Jobs → Data & Analytics → Senior Data Engineer

About the Role AppOmni is seeking an experienced Senior Data Engineer to design, build, and optimize scalable data platforms supporting AI initiatives. The ideal candidate will have strong hands-on expertise in Databricks, GCP, data pipeline development, Lakehouse architecture, and AI/ML data engineering. Key Responsibilities • Design and implement scalable data architectures and production-ready pipelines • Build and optimize ETL/ELT pipelines using Databricks and GCP • Implement Lakehouse and Medallion architectures using Delta Lake • Develop batch, mini-batch, and real-time data processing solutions • Optimize pipeline performance, latency, reliability, and cloud costs • Establish data quality, governance, security, and compliance standards • Collaborate with AI/ML, MLOps, DevOps, Product, and Engineering teams • Support AI/ML workflows and evaluate AI tools to improve engineering efficiency Requirements • 12+ years of experience in Data Engineering, Data Platforms, or related fields • Hands-on experience designing and developing Databricks data pipelines on GCP • Strong expertise in Databricks Lakehouse, Delta Lake, Data Catalog, and Medallion architecture • Proficiency in Python, Pandas, and PySpark • Experience with ETL/ELT, orchestration, and production deployments • Experience processing large volumes of structured and unstructured data • Knowledge of batch, mini-batch, and real-time processing • Understanding of data modeling, platform sizing, estimation, and infrastructure design • Familiarity with Parquet, PostgreSQL, DuckDB, JSON, and PDF formats Preferred Qualifications • TensorFlow, PyTorch, Keras, LangChain, or LangSmith • Experience supporting machine learning models and workflows • Infrastructure as Code (IaC) • Background in cybersecurity, automotive, energy, or healthcare • Advanced degree in Computer Science, Engineering, AI, or Machine Learning