Senior Data Engineer
Ami P. (L.I.O.N.)United StatesSENIORPosted 1h ago
PythonJavaAWSPySparkSQLRESTful APIMCPLLMGenerative AIAPI testingGitCI/CDAgileRAGMicroservicesApplication observability
About the Role
We are looking for a Senior Data Engineer to join our team and build scalable data pipelines, develop production-grade APIs, and integrate LLM and Generative AI capabilities into enterprise solutions.
Key Responsibilities
• Design, develop, and optimize scalable data pipelines using Python, PySpark, SQL, and AWS services.
• Build and maintain production-grade APIs, backend services, and microservices using Java and Python.
• Integrate RESTful APIs to support enterprise applications and data-driven workflows.
• Leverage AWS cloud services to deliver reliable, scalable data engineering solutions.
• Develop and integrate MCP-based solutions, LLMs, and Generative AI capabilities into enterprise workflows.
• Write reusable, maintainable code and optimize SQL queries and ETL/ELT processes.
• Collaborate with engineers, architects, data scientists, and business stakeholders to deliver technical solutions.
• Troubleshoot APIs, data pipelines, cloud integrations, and production issues.
• Participate in code reviews, automated testing, CI/CD, performance tuning, and production support.
Requirements
• Strong hands-on experience in data engineering, backend development, or software engineering.
• Expertise in AWS services such as S3, Glue, EMR, Lambda, and Redshift.
• Proven experience developing production-grade APIs and backend services using Java, Python, and RESTful API design.
• Hands-on experience with PySpark, distributed data processing, scalable data transformations, and SQL optimization.
• Strong understanding of object-oriented programming, software design principles, data structures, and maintainable architecture.
• Practical exposure to MCP, LLMs, Generative AI, or AI application integration.
• Experience with API testing, authentication, debugging, Git, CI/CD, automated testing, and Agile practices.
• Familiarity with RAG, LLM orchestration, AWS AI services, microservices, and application observability is preferred.