Jobs → Data & Analytics → Big Data Scientist

About the Role Norfolk Southern Corporation is seeking an experienced Big Data Scientist to design, develop, and maintain scalable data engineering, analytics, and streaming solutions. Key Responsibilities • Design, develop, and maintain scalable data ingestion, transformation, and streaming pipelines using Databricks and Apache Spark. • Build and operate production-grade ETL/ELT pipelines using PySpark, Spark SQL, Delta Lake, and Delta Live Tables. • Develop and maintain Delta Lake tables, including incremental processing, merges, upserts, schema evolution, and performance optimization. • Implement data governance and security controls using Unity Catalog and appropriate access management practices. • Integrate high-volume data streams using Apache Kafka, Confluent Cloud, or equivalent messaging platforms. • Use AWS services, including Amazon S3 and related cloud integration capabilities, to support data solutions. Requirements • Strong hands-on experience with Databricks, Delta Lake, Delta Live Tables, Unity Catalog, Apache Spark, Kafka, AWS, and SQL. • Experience with PySpark, Spark SQL, ETL/ELT processes, and cloud platforms. • Ability to collaborate with business stakeholders, data scientists, data modelers, and BI teams.