AWS Data Engineer & Insights Analyst
Karamthot Sandeep NaikUnited StatesSENIORPosted 22m ago
AWSS3EC2GlueEMRRDSRedshiftAthenaLambdaDynamoDBPythonRJavaC++ScalaSQLNoSQLOracleMS SQL ServerPostgresCassandraGitCI/CDData ModelingDimensional ModelingData VaultStar SchemaSnowflake SchemaAWS Glue Data CatalogData GovernanceData LineageData QualityREST APIHadoopSparkKafkaAzkabanLuigiAirflowInformaticaMuleSoftTalendTIBCOStormSpark-Streaming
About the Role
We are seeking an experienced AWS Data Engineer & Insights Analyst to join our team. This role involves working with cloud-based data services and big data tools to design, implement, and maintain data pipelines and analytics solutions.
Key Responsibilities
• Design and implement data solutions using AWS services such as S3, EC2, Glue, EMR, RDS, Redshift, Athena, and Lambda
• Develop and maintain data pipelines and workflows using tools like Airflow, Azkaban, and Luigi
• Work with both relational and NoSQL databases including Oracle, MS SQL Server, Postgres, and Cassandra
• Implement data modeling techniques including dimensional modeling, data vault, and star/snowflake schemas
• Utilize data cataloging and metadata management tools such as AWS Glue Data Catalog
• Ensure data governance, data lineage, and data quality standards are met
• Integrate REST APIs and work with big data tools like Hadoop, Spark, and Kafka
• Experience with data integration services from vendors such as Informatica, MuleSoft, Talend, and TIBCO
• Work with stream-processing systems like Storm and Spark-Streaming
Requirements
• Hands-on experience with cloud-based data services such as AWS (S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, DynamoDB)
• Proficiency in object-oriented/object function scripting languages: Python, R, Java, C++, Scala
• Proficiency in relational SQL and NoSQL databases, including Oracle, MS SQL Server, Postgres, Cassandra
• Experience with version control systems such as Git and CI/CD pipelines
• Knowledge of data modeling techniques including dimensional modeling, data vault, and star/snowflake schemas
• Familiarity with data cataloging and metadata management tools such as AWS Glue Data Catalog
• Understanding of data governance, data lineage, and data quality frameworks
• Knowledge of REST API design and integration patterns
• Experience with big data tools: Hadoop, Spark, Kafka
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow
• Experience with data integration services solutions from vendors such as Informatica, MuleSoft, Talend, and TIBCO
• Experience with stream-processing systems: Storm, Spark-Streaming