Jobs → Data & Analytics → AWS Data Engineer

About the Role The Commonwealth of Pennsylvania is seeking an AWS Data Engineer & Insights Analyst to support the Pennsylvania Longitudinal Data System (PALDS). The candidate will build and maintain data pipelines, integrate data from multiple sources, improve data quality, and develop analytics solutions that support education, workforce, and government decision-making. Key Responsibilities • Design, build, and maintain data pipelines and ETL processes. • Develop cloud-based data solutions using AWS S3, Glue, Redshift, EMR, Athena, Lambda, Kinesis, etc. • Build data workflows using Apache Airflow, AWS Step Functions, or similar tools. • Work with large and complex datasets using SQL and Big Data technologies. • Develop dashboards, reports, analytical models, and decision-support solutions. • Perform data transformation, validation, quality checks, and root-cause analysis. • Build monitoring, logging, and alerting for data pipelines. • Work with Data Scientists, Research Analysts, Policy Analysts, and business teams. • Support data integration, system modernization, and data conversion projects. • Develop solutions for education and workforce data analytics. • Ensure data governance, security, privacy, lineage, and quality standards are followed. • Work with cross-functional teams to improve data systems and analytics products. Requirements • 5+ years of AWS Data Engineering experience with S3, Glue, Redshift, EMR, Athena, Lambda, RDS, DynamoDB, etc. • 5+ years of Python, R, Java, Scala, or similar programming experience. • 5+ years of advanced SQL and relational/NoSQL database experience. • Experience with Oracle, SQL Server, PostgreSQL, and Cassandra. • Strong experience with Big Data technologies such as Spark, Hadoop, and Kafka. • Experience with Airflow or similar data pipeline/workflow tools. • Experience with Git and CI/CD pipelines. • Strong knowledge of data modeling, including Star, Snowflake, Dimensional Modeling, and Data Vault. • Experience with AWS Glue Data Catalog, metadata, data lineage, and data governance. • Experience with tools such as Informatica, MuleSoft, Talend, or TIBCO. • Knowledge of stream processing such as Kafka, Spark Streaming, or Storm. • Strong experience developing dashboards, reports, and analytical models. Important Domain Experience Candidates should have experience working with student-level higher education data, including linking education data with employment, wage, or workforce data to analyze outcomes and provide actionable insights. Experience with education analytics, workforce analytics, longitudinal data, or government data is highly preferred.