Loading...
SeniorFull-timePythonAWSTerraformAPIData
Salary Range
$132,600 - $179,400 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are looking for a Senior Data Engineer to design, build, and evolve our modern, event-driven and near real-time Lakehouse platform on AWS + Snowflake. This is a data platform role focused on high-volume transactional data, near real-time processing, and strong architectural foundations.
Key Responsibilities
- →Design and maintain event-driven ingestion pipelines using Confluent + AWS
- →Ingest CDC streams from transactional databases using AWS DMS
- →Build and maintain custom ingestion connectors for internal and external sources
- →Ensure data consistency, ordering, idempotency, and replayability in near real-time pipelines
- →Own the Lakehouse architecture based on Apache Iceberg on S3 Tables
- →Design Iceberg tables optimized for streaming and batch workloads
- →Manage schema evolution, partitioning strategies, compaction, and time travel
- →Implement and evolve Bronze, Silver, and Gold layers aligned with event-driven ingestion patterns
- →Build near real-time and batch ELT pipelines using AWS Glue, Python, and dbt
- →Implement incremental models optimized for streaming-derived datasets
- →Implement data observability and monitoring across ingestion and transformation layers
- →Track freshness, volume, schema changes, and data quality metrics
- →Expose curated datasets through Snowflake as a governed access layer
- →Contribute to architectural decisions across ingestion, storage, and consumption
Requirements
- ✓3–6+ years of experience as a Data Engineer
- ✓Strong experience with event-driven data architectures
- ✓Hands-on experience with Kafka / Confluent in production environments
- ✓Experience ingesting data via CDC (AWS DMS or similar tools)
- ✓Solid experience designing near real-time data pipelines
- ✓Strong knowledge of Apache Iceberg
- ✓Experience with AWS (S3, Glue, Lambda, EventBridge, Firehose, etc.)
- ✓Advanced SQL and Python
- ✓Production experience using dbt
- ✓Experience using Snowflake as an analytics and governance layer
- ✓Summary Bullets
- ✓Design and build event-driven data ingestion pipelines
- ✓Manage Lakehouse architecture on AWS + Snowflake
- ✓Develop ELT pipelines using AWS Glue, Python, and dbt
- ✓Implement data observability and monitoring systems
- ✓Contribute to architectural decisions for the data platform
- ✓Ensure data quality and reliability across the pipeline
- ✓Expose curated datasets through Snowflake
- ✓Requirement Bullets
- ✓3–6+ years of Data Engineering experience
- ✓Experience with Kafka/Confluent and AWS DMS
- ✓Knowledge of Apache Iceberg and S3 Tables
- ✓Strong SQL and Python skills
- ✓Experience with dbt and Snowflake
- ✓Understanding of event-driven architectures
- ✓Experience with high-volume transactional data
- ✓Benefit Bullets
- ✓[]
