Data Integration Lead
Canada4w ago
Key details
- Location
- Canada
- Work style
- Onsite
- Level
- Senior
- Function
- other
- Focus area
- other
- Posted
- 4w ago
Skills
azureazure data factorydata lakesdata pipelinesdata qualitydatabricksetlkafkapysparkroot cause analysissparksql
About the role
Real-time data pipelines using Azure Databricks, PySpark, Kafka, Azure Data Factory (ADF), and Azure Data Lake Storage (ADLS), enabling reliable ingestion and transformation of 10M+ records daily. • Developed and optimized SQL and PySpark transformations within medallion-style ETL workflows, improving processing efficiency, maintainability, and data quality for downstream analytics. • Identified requirement gaps and data quality issues stemming from upstream data sources and solutioned root-cause fixes by implementing validation rules, refining transformation logic, and collaborating with stakeholders to improve downstream data reliability.
How well do you match this job?
Skills
Experience
Level
Title
Location
Salary
Freshness
Similarity
Free account needed to apply or save. It takes a minute, and we'll bring you right back to this job.