Staff Software Engineer - Konnect AI Platform at Kong

Are you ready to power the World's connections? If you don’t think you meet all of the criteria below but are still interested in the job, please apply. Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others. ABOUT THE ROLE: The Senior AI/ML Engineer will be a key contributor to Kong's AI Platform team, focusing on data pipelines, knowledge systems, and vector search infrastructure. You'll build the data foundation that powers our AI capabilities, including documentation ingestion, semantic search, embedding pipelines, and knowledge base management. This role requires expertise in vector databases, embedding models, document processing, and MLOps practices. You'll ensure our AI systems have access to high-quality, privacy-compliant data that enables accurate and relevant responses. WHAT YOU'LL DO: - Design and implement documentation ingestion pipelines for AWS Bedrock Knowledge Base and other vector stores. - Build semantic chunking and document processing systems optimized for retrieval quality and context preservation. - Develop vector embedding pipelines with hybrid search strategies combining semantic and keyword search. - Create real-time telemetry collection systems for model monitoring, performance tracking, and quality assurance. - Implement data anonymization and PII detection systems to ensure privacy-compliant AI operations. - Build automated knowledge base refresh and quality monitoring systems for maintaining data freshness. - Design and implement feature stores for model inputs, prompt variables, and contextual information. - Develop training data curation pipelines for future fine-tuning and model improvement initiatives. - Optimize vector indexes and database performance for low-latency retrieval at scale. - Create metadata enrichment and entity extraction pipelines to enhance search relevance. - Build streaming data pipelines for real-time data ingestion and processing. - Collaborate with AI engineers to optimize retrieval strategies and improve RAG system performance. - Establish best practices for data quality, privacy compliance, and GDPR-compliant data handling. - Work with platform engineers to deploy and scale data infrastructure on AWS. WHAT YOU'LL BRING: - 5+ years of professional software engineering experience with 4+ years focused on ML/AI engineering and data pipelines. - Strong experience with vector databases such as Pinecone, Weaviate, Qdrant, OpenSearch, or similar technologies. - Deep knowledge of embedding models including OpenAI Ada-3, Cohere, BGE, E5, and understanding of when to use each. - Expertise in document processing, chunking strategies, and optimizing retrieval quality. - Proficiency in both Python (for ML workflows) and Go (for production systems). - Experience with MLOps tools and platforms such as Weights & Biases, MLflow, Kubeflow, or similar. - Strong knowledge of streaming data systems like Kafka, Kinesis, or similar technologies. - Understanding of GDPR, privacy regulations, and privacy-compliant data handling practices. - Experience with knowledge graphs, semantic technologies, or ontology management. - Background in information retrieval, search relevance, and ranking algorithms. - Experience with data transformation and ETL/ELT pipelines at scale. - Strong understanding of SQL and NoSQL databases for data storage and retrieval. - Knowledge of AWS data services (S3, DynamoDB, RDS, Kinesis, Glue). - Experience with experiment tracking and feature engineering for ML models. BONUS POINTS: - Experience with AWS Bedrock Knowledge Base or similar managed vector search services. - Knowledge of hybrid search algorithms combining dense and sparse retrieval. - Experience with reranking models and cross-encoders for retrieval optimization. - Familiarity with prompt compression and context window optimization techniques. - Experience with synthetic data generation for training and evaluation. - Knowledge of named entity recognition (NER) and information extraction systems. - Experience with graph databases (Neo4j, Amazon Neptune) or knowledge graph construction. - Background in natural language processing (NLP) or computational linguistics. - Experience with data versioning tools (DVC, Pachyderm) and reproducible ML pipelines. - Knowledge of model serving infrastructure (Seldon, KServe, BentoML). - Experience with distributed computing frameworks (Spark, Ray, Dask). - Understanding of semantic similarity metrics and evaluation frameworks. - Experience with A/B testing frameworks and causal inference methods. - Familiarity with data governance and lineage tracking tools. - Contributions to open-source ML/data engineering projects. - Experience with Kubernetes and containerized data pipelines. #LI-AP1 About Kong: Kong Inc., a leading developer of API and AI connectivity technologies, is building the infrastructure that powers the agentic era. trusted by the Fortune 500 and startups alike, Kong's unified API and AI platform, Kong Konnect, enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI models. For more information, visit www.konghq.com [Upgrade to PRO to see link]