Loading...
Mid LevelFull-timePythonGCPAzureDockerAgileCI/CDGitAIAPIData
Salary Range
$35,700 - $48,300 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are looking for an Azure Data Engineer to join our team at Capco Poland. This role involves designing and implementing ingestion pipelines, document processing workflows, and integrations across Azure services to enable domain agents and task agents to operate on properly prepared knowledge artifacts.
Key Responsibilities
- →Design and develop data connectors for automated document and metadata ingestion
- →Implement ingestion pipelines and processing orchestration mechanisms
- →Process documents using Azure Document Intelligence
- →Develop transformation and normalization workflows
- →Index content into Azure AI Search and Knowledge Bases
- →Prepare evaluation datasets and automate quality evaluation
- →Implement instrumentation, logging, and monitoring
- →Optimize document processing costs
- →Establish CI/CD pipelines and development standards
- →Build and maintain containerized services
- →Automate ingestion and evaluation workflows
Requirements
- ✓Strong Python skills (asyncio, FastAPI, Pydantic, multiprocessing)
- ✓Hands-on experience with Azure SDK for Python
- ✓Practical experience with Microsoft Graph API / SharePoint API
- ✓Azure AI Search: index design, indexers, skillsets, embeddings
- ✓Azure Document Intelligence (OCR, layout extraction, custom models)
- ✓Testing experience (pytest, integration tests)
- ✓GitHub Actions (build, test, scanning, artifact management)
- ✓Docker (image building, multi-stage builds, layer optimization)
- ✓Logging and monitoring (Application Insights, Log Analytics)
- ✓Experience working with large document collections
- ✓Data security best practices (PII masking, RBAC, Entra ID integration)
Free to sign up. Takes 30 seconds.
