Junior Data Scientist
Ammar Javed KhanUnited StatesJUNIORPosted 4d ago
PySparkSpark SQLRDatabricksACID transactionsMERGE upsertsSchema enforcementTime travelOPTIMIZEZ-orderVACUUMParquetDeltaCSVADLS Gen2OPENROWSETExternal tablesViewsStar-schemaFactsDimensionsStored proceduresSCD Type 1/Type 2COPY INTOPolyBaseCETASParameterized Synapse/ADF pipelinesGitCI/CDAzure DevOpsGitHubSynapse artifactsDatabricks Asset BundlesAzure Data Engineer AssociateSQLServerPowerBIAzureDevOpsCICDDataPipelinesDataModelingETLELT
About the Role
We are looking for a Junior Data Scientist to join our team in Indianapolis, IN. This role involves working with Databricks notebooks using PySpark, Spark SQL, and R, and involves various data engineering tasks.
Key Responsibilities
• Develop Databricks notebooks using PySpark, Spark SQL, and R
• Work with ACID transactions, MERGE upserts, schema enforcement and evolution, time travel, OPTIMIZE, Z-order, and VACUUM
• Query Parquet, Delta, and CSV files in ADLS Gen2 using OPENROWSET, external tables, and views
• Build star-schema facts and dimensions, stored procedures, views, and SCD Type 1/Type 2 implementations
• Utilize COPY INTO, PolyBase, CETAS, and parameterized Synapse/ADF pipelines
• Work with Git and CI/CD using Azure DevOps or GitHub
Requirements
• Master's degree in Data Science, Analytics, Informatics, Computer Science, Mathematics, Engineering, or a related field; OR a bachelor's degree in a related field plus 2+ years of relevant experience; OR 4+ years of hands-on experience in a similar data engineering role
• Experience developing Databricks notebooks using PySpark, Spark SQL, and R
• Knowledge of ACID transactions, MERGE upserts, schema enforcement and evolution, time travel, OPTIMIZE, Z-order, and VACUUM
• Experience querying Parquet, Delta, and CSV files in ADLS Gen2
• Experience building star-schema facts and dimensions, stored procedures, views, and SCD Type 1/Type 2 implementations
• Knowledge of COPY INTO, PolyBase, CETAS, and parameterized Synapse/ADF pipelines
• Experience with Git and CI/CD using Azure DevOps or GitHub
• Databricks Certified Data Engineer Associate certification preferred
• Microsoft Certified: Azure Data Engineer Associate (DP-203) preferred