Databricks Architect
Ronny D'souzaUnited StatesSENIORPosted 10d ago
DatabricksAWSSparkPySparkSQLDelta LakeMedallion ArchitectureData PartitioningCaching StrategiesZORDEROPTIMIZEAuto LoaderFile CompactionS3Glue CatalogIAMLambdaAPI GatewayKinesisCI/CDGitDevOpsTerraform
About the Role
We are actively looking for an experienced Databricks Architect to join a high-impact enterprise data transformation initiative.
Key Responsibilities
• Design and implement scalable Databricks Lakehouse solutions on AWS
• Architect high-performance data pipelines and streaming solutions
• Lead platform optimization, governance, security, and observability initiatives
• Collaborate with Data Engineering, ML, BI, and business teams
• Mentor engineers while remaining hands-on in architecture and implementation
Requirements
• 12+ years of overall IT experience
• Strong expertise in Databricks Lakehouse Architecture on AWS
• Deep experience with Spark, PySpark, SQL, Delta Lake, and Distributed Data Systems
• Hands-on with Medallion Architecture (Bronze/Silver/Gold)
• Expertise in Data Partitioning, Caching Strategies, ZORDER, OPTIMIZE, Auto Loader, and File Compaction
• Experience handling multi-terabyte structured time-series datasets
• Strong AWS knowledge including S3, Glue Catalog, IAM, Lambda, API Gateway, and Kinesis
• Experience designing low-latency data delivery and serving-layer architectures
• CI/CD, Git, DevOps, and Terraform expertise preferred
• Databricks certifications are a plus