Jobs → Data & Analytics → Senior Data Engineer – Databricks

About the Role Lorgan Tech Solutions, LLC is seeking a Senior Data Engineer with strong Databricks, PySpark, SQL, and cloud data platform experience to design, build, and optimize enterprise data solutions. Key Responsibilities • Develop and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL. • Implement Bronze, Silver, and Gold layers using Delta Lake and medallion architecture. • Build reliable batch and streaming pipelines with incremental processing, deduplication, and recovery capabilities. • Design data models and curated datasets for reporting, analytics, and downstream applications. • Optimize Spark workloads, cluster configurations, and query performance. • Implement data quality checks, monitoring, logging, and error handling. • Manage data access, governance, and lineage using Unity Catalog. • Automate deployment using Git and CI/CD practices. • Collaborate with architects, analysts, and business stakeholders. • Mentor engineers and contribute to technical design and code reviews. Requirements • 10+ years of professional Data Engineering experience. • Strong hands-on experience with Databricks, Apache Spark, PySpark, and SQL. • Proficiency in Python and distributed data processing. • Experience with Delta Lake, medallion architecture, and data modeling. • Experience with at least one cloud platform: Azure, AWS, or GCP. • Knowledge of orchestration using Databricks Workflows, Airflow, or Azure Data Factory. • Experience implementing incremental loads, CDC, and data validation. • Familiarity with Unity Catalog, security, and data governance. • Experience with Git, CI/CD, and production troubleshooting. • Strong communication, problem-solving, and collaboration skills. Preferred Qualifications • Experience with Auto Loader, Structured Streaming, or declarative pipelines. • Familiarity with dbt, Terraform, and infrastructure automation. • Databricks certification.