Senior Azure Data Engineer

DATAMAXIS • Illinois, Illinois • 3d ago

Job Summary: We are seeking a Senior Azure Data Engineer to help design, build, and operate our next-generation enterprise data platform on Microsoft Azure. You will own end-to-end delivery of data pipelines and data products that power analytics, regulatory reporting, operational dashboards, and emerging AI/ML use cases. You will partner closely with data architects, analytics engineers, data scientists, business stakeholders, and platform engineering teams to deliver reliable, performance, secure, and costefficient data solutions.

This role is ideal for an engineer with strong hands-on depth in Azure Data Factory, Azure Synapse Analytics and/or Databricks, and modern Lakehouse patterns, who is comfortable leading migration programs (e.g., Informatica-to-ADF, on-prem warehouse-to-cloud), mentoring mid-level engineers, and shaping engineering standards across the team.

Key Responsibilities:

Pipeline Design & Development

Design and build robust, reusable, parameter-driven ingestion and transformation pipeline using Azure Data Factory, Synapse Pipelines, Data Bricks and/or Microsoft Fabric Data Factory.

Implement medallion architecture (Bronze / Silver / Gold) on Azure Data Lake Storage Gen2 using Delta Lake, Parquet, and structured streaming patterns.

Build performant ELT workflows that leverage pushdown to source systems (Synapse Dedicated SQL Pool, Azure SQL, Teradata) where appropriate.

Develop and optimize PySpark notebooks and jobs on Azure Databricks or Synapse Spark.

Data Modeling & Warehousing

Design dimensional models (Kimball star/snowflake) and data vault patterns for analytics consumption.

Implement Slowly Changing Dimensions (Type 1/2/3), Change Data Capture, and late-arriving data patterns.

Tune distributed SQL workloads in Synapse Dedicated SQL Pool / Fabric Warehouse, including distribution keys, partitioning, and clustered column store indexes.

Platform Engineering & DevOps

Implement CI/CD for data pipelines using Azure DevOps (YAML pipelines, ARM/Bicep/Terraform) across Dev / SIT / UAT / Prod environments.

Instrument pipelines with robust logging, auditing, and monitoring using Azure Monitor, Log Analytics, and KQL.

Define and enforce coding standards, code review practices, branching strategies, and release management.

Migration & Modernization

Lead or contribute to legacy-to-cloud migrations - e.g., Informatica PowerCenter to Azure Data Factory, on-premises Teradata / Oracle / SQL Server to Synapse or Fabric.

Perform workload assessment, capacity planning, and cost modeling for target-state architectures.

Production incident response for critical pipelines.

Required Qualifications:

Deep hands-on expertise with Azure Data Factory: pipelines, datasets, linked services, triggers, parameterization, mapping data flows, and all three Integration Runtime @types (Azure, Self hosted, SSIS).

Strong Experience in Data Bricks and PySpark.

Production experience with one or more of: Azure Synapse Analytics (Dedicated and Serverless SQL Pools, Spark Pools) OR Azure Databricks (Delta Lake, Unity Catalog) OR Microsoft Fabric (Warehouse, Lakehouse, OneLake).

Strong working knowledge of Azure Data Lake Storage Gen2 (hierarchical namespace, RBAC + ACLs, lifecycle management, security).

Experience with Azure Key Vault, Azure AD / Entra ID (including managed identities and service principals), and private networking (VNet integration, private endpoints).

Monitoring and troubleshooting with Azure Monitor, Log Analytics, and KQL.

Advanced SQL - window functions, CTEs, query optimization, execution plan analysis, performance tuning.

Strong Python for data engineering - pandas, PySpark, REST API integration, unit testing (pytest).

Proficient in T-SQL; familiarity with Spark SQL, KQL, PowerShell, and Bash shell scripting.

Preferred Qualifications:

5+ years of data warehouse development experience.

5+ years of data modeling experience using ERWIN or similar tools.

2+ years of experience with Azure Data Factory and Snowflake.

Medicaid Domain Knowledge is a plus.

Similar Jobs

Electrical Designer

Actalent • Chicago, IL • 20min ago

IT & Engineering

An international MEP engineering firm with a growing presence in Chicago is looking to add an Electrical Designer to their team! Job Description Seeking an experienced Electrical Designer to perform full design of lighti...

20min ago

Go to Apply

Electrical Designer

Actalent • Chicago, IL • 23min ago

IT & Engineering

23min ago

Go to Apply

Electrical General Superintendent

Intrepid Solutions and Services • Lindenhurst, IL • 24min ago

IT & Engineering

Overview Shimmick is seeking a highly talented Electrical General Superintendent to be based in the greater Seattle, Washington area.This challenging position will provide an excellent growth opportunity while improving...

24min ago

Go to Apply

Electrical Project Engineer

Progress Rail Services Corp. • La Grange, IL • 25min ago

IT & Engineering

11983BR PRS Facility Location: EMD HQ LaGrange, IL - 400 Job Purpose: The Electrical Application Engineer is a highly accountable, execution-driven role responsible for the design and delivery of locomotive electrical ap...

25min ago

Go to Apply

Electrical Field Engineer Intern

Intrepid Solutions and Services • Lemont, IL • 27min ago

IT & Engineering

Overview Shimmick is actively seeking a highly motivated Field Engineer Intern for employment in our Electrical Division (Axia). Location is to be determined, but opportunities exist in Northern California, Southern Cali...

27min ago

Go to Apply

Electrical Engineer - Nationwide

Jacobs • Chicago, IL • 29min ago

IT & Engineering

Job summary: We are seeking an experienced Electrical Engineer with a strong background in performing design of large-scale electric power systems such as power generation plants, High voltage (HV)/medium voltage (MV) su...

29min ago

Go to Apply

Electrical Design Engineer

ManpowerGroup • Round Lake, IL • 30min ago

IT & Engineering

Our client, a leader in medical device innovation, is seeking a Senior Principal Electrical Engineer to join their team. As a Senior Principal Electrical Engineer, you will be part of the Engineering Department supportin...

30min ago

Go to Apply

Electrical General Superintendent

Intrepid Solutions and Services • Lemont, IL • 32min ago

IT & Engineering

Overview Shimmick is seeking a highly talented Electrical General Superintendent to be based in the greater Los Angeles, Inland Empire, or San Diego county area.This challenging position will provide an excellent growth...

32min ago

Go to Apply

Electrical Project Engineer

Intrepid Solutions and Services • Lemont, IL • 32min ago

IT & Engineering

Overview Shimmick is seeking a highly talented Electrical Project Engineer based out of Orange County, CA . This challenging position will provide an excellent growth opportunity while improving the infrastructure of the...

32min ago

Go to Apply

Agentic Software Engineer II

Deloitte • Chicago, IL • 2h ago

IT & Engineering

Our Deloitte Customer team empowers organizations to build deeper relationships with customers through innovative strategies, advanced analytics, Generative AI, transformative technologies, and creative design. We can en...

2h ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)