Data Engineer at CityOfNewYork

** Please note that due to city requirements only applicants with a master’s degree and at least 3 years of related work experience will be considered. In addition, this position requires a US work authorization as no visa sponsorship is available. The position is eligible for 2 days per week remote work. About TLC: The New York City Taxi and Limousine Commission regulates for-hire transportation across NYC taxis, high volume platforms like Uber and Lyft, black cars, commuter vans, and more. We license roughly 180,000 drivers and 116,000 vehicles that conduct nearly a million trips a day. Our work on driver pay standards, accessibility, and traffic safety has become a model for regulators in cities around the world. About the Role: Our Data Analytics Unit, embedded in the Policy & Community Affairs Division, works at the intersection of data infrastructure and public policy. A lot of initial work starts with a policy question rather than a specification. The data (and analytics) engineering focus is on building and maintaining the infrastructure that makes good policy analysis possible everything from inspecting raw file submissions and initial ETL to designing pipelines and creating silver and gold tables to make analysis as streamlined as possible. Our goal is to make sure what we're producing is trustworthy: well-documented, reproducible, and reliable over time. We're a small team of analysts and engineers, so there is room for close collaboration. The data itself is rich: billions of trip records, GPS breadcrumb traces for every for-hire vehicle in the city, detailed session data across all major platforms. The infrastructure we've built Databricks, Delta Lake, Azure allows us to process this quickly and consistently at scale so we can focus on policy impact and not be bottlenecked by compute. What We're Looking For: We're looking for someone with an infrastructure focus: someone who thinks about how data gets built and maintained, not just consumed. That means caring about schemas, naming conventions, reliability, and what makes data trustworthy for the people downstream, with familiarity processing large data streams. Also important is analytical curiosity. You notice when numbers don't add up and you follow the thread. Data quality is interesting to you, and you understand what analysts and scientists are trying to do with the data you build well enough to push back or ask good questions when something doesn't make sense. You recognize that sometimes the best way to resolve data quality issues before writing a line of code may be to reach out to data providers and explain upstream issues. Beyond that: you're comfortable with ambiguity and can take a vague request and make progress without waiting for a perfect spec. Your Python and SQL are clean and readable, and version control, modular code, and documentation are habits rather than afterthoughts. You understand that LLM’s are a powerful tool but avoid copy-pasting without understanding. We're a small team that works closely together, so you'll have real ownership of your work while also being able to think out loud with others. You should also be comfortable presenting to senior staff and external stakeholders. To Apply: Please go to cityjobs.nyc.gov and search for Job ID# 776089 or click the "Apply" button below. Please include bother resume and cover letter. SUBMISSION OF A RESUME IS NOT A GUARANTEE THAT YOU WILL RECEIVE AN INTERVIEW. APPOINTMENTS ARE SUBJECT TO OVERSIGHT APPROVAL. *Proposed salary will be determined based on experience and preferred skills of the selected candidate.” Work Location: New York, NY CITY RESEARCH SCIENTIST - 21744