Data Engineer
Chakradhar Leela Krishna G.United StatesSENIORPosted 1h ago
PySparkETLELTSSISAzure Synapse AnalyticsAzure Data Lake StorageAzure Data FactoryDatabricksKubernetesCI/CDKafkaMQAWS SQSIAMRBACOAuthmTLS/TLSJWT
About the Role
We are looking for a Data Engineer to join the Alternative Investments team within Global Fund Services Technology. This role will focus on modernizing legacy data integrations by transforming SSIS-based processes into scalable, cloud-native PySpark ETL/ELT pipelines.
Key Responsibilities
• Design and develop PySpark-based ETL/ELT pipelines for cloud data platforms
• Lead the migration of legacy SSIS packages and on-premises workloads to Azure-based solutions
• Build robust data-processing workflows using Azure Synapse Analytics, Azure Data Lake Storage, Azure Data Factory, Databricks, and Kubernetes
• Define and execute migration strategies for legacy systems while maintaining reliability, data quality, and performance
• Apply engineering best practices for CI/CD, automated testing, observability, performance tuning, and resilient data processing
• Partner with product, security, and cloud architecture teams to deliver highly available and fault-tolerant solutions
• Embed data governance, lineage, security, and compliance throughout data engineering workflows
Requirements
• Strong hands-on experience building complex financial data-processing frameworks and workflows in large enterprise environments
• Experience with financial accounting platforms is preferred
• Solid operational and development experience with SSIS and third-party data-processing tools
• Strong knowledge of Apache Spark and PySpark for large-scale data processing
• Expertise with Microsoft Azure data technologies, including Azure Synapse Analytics, Azure Data Lake Storage, Azure Data Factory, Azure SQL Database, Azure Data Catalog and Microsoft Purview, Azure Machine Learning and Stream Analytics
• Familiarity with Databricks and Kubernetes-based data workloads
• Understanding of middleware and messaging platforms, including Kafka, MQ, and AWS SQS
• Strong knowledge of security concepts: IAM, RBAC, OAuth, mTLS/TLS, and JWT
• Experience with cloud migration, data governance, lineage, and compliance practices