Lead Data Engineer- Databricks + Apache Pinot
Coimbatore, TN, India5mo ago
Key details
- Location
- Coimbatore, TN, India
- Work style
- Onsite
- Level
- Senior
- Job type
- Full-time
- Experience
- 10–14 yrs
- Function
- Data & ML
- Focus area
- Data engineering & Analytics
- Posted
- 5mo ago
Skills
About the role
Who are we
Fulcrum Digital is an agile and next-generation digital
accelerating company providing digital transformation and technology services
right from ideation to implementation. These services have applicability across
a variety of industries, including banking & financial services, insurance,
retail, higher education, food, healthcare, and manufacturing.
About the role
We are looking for a Data Engineering Lead with 10-14 years of experience to drive solution architecture, client engagements, pre-sales activities, and technical leadership. The ideal candidate should have deep expertise in Databricks, Apache Pinot, Spark, Kafka, Python, and SQL, along with exceptional communication and stakeholder management skills.
Key Responsibilities
- Participate in pre-sales discussions, solutioning workshops, and client stakeholder meetings.
- Lead client POCs, technical assessments, and architecture discovery sessions.
- Design and implement scalable end-to-end Data & Analytics solutions on Databricks.
- Architect and build batch and real-time data pipelines using Spark, Kafka, and Python.
- Design and implement Lakehouse architectures, Delta Lake, and Medallion frameworks.
- Drive Apache Pinot-based real-time analytics and low-latency data solutions.
- Establish data governance, security, and best practices using Unity Catalog.
- Define and implement CI/CD and DevOps best practices for data platforms.
- Perform architecture reviews, code reviews, performance tuning, and cost optimization.
- Work closely with business stakeholders, delivery teams, and client leadership.
- Mentor and guide Data Engineers and Technical Leads, fostering technical excellence and capability development across the team.
- Provide hands-on technical leadership during design, development, and production support activities.
Required Skills
- Databricks (Expert Level)
- Apache Spark / PySpark
- Python
- SQL
- Apache Kafka
- Apache Pinot
- Delta Lake & Lakehouse Architecture
- Unity Catalog
- CI/CD & DevOps
- Data Modeling & Performance Optimization
Experience
- 10-14 years of overall experience in Data Engineering.
- 5+ years of hands-on Databricks experience.
- 3+ years of participating in pre-sales calls/ architecture level discussion.
- Strong experience in client-facing roles, pre-sales, solutioning, and stakeholder management.
- Proven track record in building enterprise-scale data platforms and mentoring technical teams.
Soft Skills
- Excellent verbal and written communication skills.
- Strong presentation and stakeholder management abilities.
- Ability to engage confidently with client executives, business leaders, and technical teams.
- Proven leadership and team mentoring experience.
Requirements
Key Responsibilities
- Participate in pre-sales discussions, solutioning workshops, and client stakeholder meetings.
- Lead client POCs, technical assessments, and architecture discovery sessions.
- Design and implement scalable end-to-end Data & Analytics solutions on Databricks.
- Architect and build batch and real-time data pipelines using Spark, Kafka, and Python.
- Design and implement Lakehouse architectures, Delta Lake, and Medallion frameworks.
- Drive Apache Pinot-based real-time analytics and low-latency data solutions.
- Establish data governance, security, and best practices using Unity Catalog.
- Define and implement CI/CD and DevOps best practices for data platforms.
- Perform architecture reviews, code reviews, performance tuning, and cost optimization.
- Work closely with business stakeholders, delivery teams, and client leadership.
- Mentor and guide Data Engineers and Technical Leads, fostering technical excellence and capability development across the team.
- Provide hands-on technical leadership during design, development, and production support activities.
How well do you match this job?
Free account needed to apply or save. It takes a minute, and we'll bring you right back to this job.