AI Engineer at Distro

Role Overview We are seeking a hands-on AI Engineer with deep expertise in Large Language Model integration and production AI systems. This role will lead the design and implementation of LLM-powered capabilities within our platform, working closely with backend, mobile, and product teams. This individual will own the end-to-end AI architecture, from model selection and prompt strategy to retrieval systems, evaluation frameworks, cost optimization, and production deployment. This is not a research role. It is a systems architecture and applied AI engineering role focused on building scalable, secure, real-world AI applications. Key Responsibilities LLM Architecture & Integration Design and implement LLM-powered application workflows Architect prompt orchestration, tool calling, and multi-step reasoning pipelines Define model selection strategy (OpenAI, Anthropic, open-source models, etc.) Implement streaming responses for mobile and web clients Optimize token usage and latency for production environments Build fallback and resilience strategies across model providers RAG & Knowledge Systems Architect retrieval-augmented generation pipelines Design vector database schema and embedding workflows Implement chunking, metadata tagging, and indexing strategies Optimize semantic search relevance Integrate structured and unstructured data sources AI Infrastructure & Backend Integration Collaborate with backend architects to integrate AI services into APIs Design asynchronous processing pipelines for AI workflows Implement caching strategies for inference results Architect evaluation and monitoring frameworks for LLM output quality Build guardrails, moderation layers, and output validation Model Evaluation & Performance Define evaluation metrics for response quality Implement automated testing for LLM outputs Analyze hallucination patterns and mitigation techniques Monitor drift, cost, and performance metrics Continuously improve prompt and architecture strategies Security & Governance Implement data privacy safeguards Ensure compliance with enterprise security requirements Design safe handling of user-generated content Implement access control and audit logging Technical Leadership Guide LLM architecture decisions across the platform Mentor engineers working on AI-related components Evaluate emerging AI tools and frameworks Define long-term AI roadmap aligned with product strategy Required Qualifications 5–8+ years in software engineering with at least 2+ years focused on LLM systems Production experience integrating LLM APIs Strong experience with: Python (FastAPI preferred) Vector databases (pgvector, Pinecone, Weaviate, etc.) Embeddings and semantic search Prompt engineering and tool invocation workflows Experience building RAG systems in production Experience optimizing latency and inference costs Strong understanding of tokenization, context windows, and model limitations Experience deploying AI services in cloud environments (AWS, GCP, Azure) #Matchpoint #LI-PROMOTED #LI-Remote