Data Engineer
Oregon, United States23h ago
Key details
- Location
- Oregon, United States
- Work style
- Remote
- Level
- Mid level
- Job type
- Full-time
- Experience
- 3–6 yrs
- Function
- Data & ML
- Focus area
- Data engineering & Analytics
- Visa sponsorship
- US citizens only
- Posted
- 23h ago
Skills
About the role
Metas Solutions excels in providing strategic consulting, program management, and data-driven analytics to federal public health agencies. Our expertise enhances health systems and delivers measurable outcomes, helping agencies achieve their mission. We collaborate with federal partners to implement agile solutions for national public health initiatives, driving impactful changes through cutting-edge programs that strengthen communities and promote well-being. See www.metassolutions.com for further details about us and careers with Metas.
Job Description:
Metas Solutions has an "immediate opening" for a talented Data Engineer. In this Remote position you will develop, maintain, and support data pipelines and observability solutions that provide reliable monitoring, operational insights, and mission-critical public health analytics. The role focuses on Databricks (Delta Lake, Unity Catalog, job orchestration), Palantir Foundry (transforms, datasets, pipelines), and distributed processing with PySpark and SQL. You will build and maintain end‑to‑end pipelines within established architectural patterns, contributing to schema design and quality assurance, testing, deployment, and production support. You'll work with team to troubleshoot pipeline issues, improve transformation efficiency, partitioning strategies, and query performance.
You'll modernize existing analytical codebases (Python, SQL), contribute to shared libraries and reusable templates, and support platform onboarding/migrations including Palantir Foundry. You work will include cloud‑native orchestration using Azure Databricks, Palantir foundry, CI/CD for data services, observability and data quality, and data security & governance. Strong communication and collaboration skills are essential. You'll work with analysts, program teams, and CDC partners to clarify requirements, document solution, communicate progress and issues, and explain technical findings in clear, outcome-oriented language.
Responsibilities:
- Develop, maintain, and support scalable data pipelines and platform components using Databricks and Palantir Foundry, following established architecture and engineering standards.
- Contribute to the design, development, and continuous improvement of observability metadata models, standards, and frameworks to support monitoring, data quality, operational visibility, and platform reliability.
- Implement schemas and transformations using PySpark and SQL; troubleshoot processing issues and improve partitioning and query performance.
- Configure and maintain cloud-native orchestration workflows, contribute to CI/CD pipelines for data services, and develop automated tests and validation checks.
- Support platform onboarding and migration activities, including Palantir Foundry; contribute to reusable libraries, templates, and technical documentation.
- Implement data-quality checks, pipeline monitoring, and established security and governance controls; collaborate with stakeholders to clarify requirements and communicate progress, issues, and technical findings.
Qualifications:
- 3+ years of hands-on data engineering experience, including developing transformations with PySpark and SQL and working with distributed data processing.
- 3+ years of experience developing, maintaining, troubleshooting, and supporting production data pipelines
- 1+ year(s) of experience with Databricks, including Delta Lake, job orchestration, and familiarity with Unity Catalog
- 1+ year(s) of experience with Microsoft Azure, including services such as Azure Data Factory, Azure Functions, and Azure Storage, or equivalent services on AWS or Google Cloud Platform (GCP)
- Experience designing, developing, maintaining, and supporting observability data platforms and solutions for monitoring, troubleshooting, and performance optimization
- Experience implementing schema designs, medallion architecture patterns, partitioning strategies, and performance improvements within established data architectures
- Experience collaborating with business stakeholders to gather requirements and explain technical solutions and findings to technical and non-technical audiences
- Experience with Palantir Foundry, including datasets, transforms, pipeline development, and data modeling
- Ability to independently complete assigned engineering tasks, collaborate in cross-functional teams, and escalate technical issues appropriately
- Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements
- Bachelor's degree
- Must be a U.S. Citizen or Lawful Permanent Resident (Green Card Holder)
Additional Qualifications:
- Experience contributing to CI/CD pipelines for data engineering and application services using Azure DevOps or equivalent tooling.
- Experience implementing data-quality checks, observability, monitoring, data security, or governance controls
- Experience supporting data-platform onboarding, cloud migration, or modernization of existing Python and SQL analytics code
- Knowledge of Agile development methodologies, DevOps, and DataOps practices
- Knowledge of Generative AI concepts and tools, including retrieval-augmented generation (RAG), large language models (LLMs), and embedding models
Security Requirements:
- Must be US Citizen and with the ability to obtain a US Government security clearance (Public Trust 5) within a reasonable period
- Market competitive salary, commensurate with experience and education
- Comprehensive benefits package available, Medical, Dental, Vision and Life Insurance, Paid Time Off (PTO), 401K with company match, growth, and promotion opportunities
Notice on Candidate AI Usage
Metas Solutions is committed to maintaining a fair and competitive interview process for all candidates, based on their experience, skills, and education. To preserve the integrity and fairness of our interview process, candidates are expected to respond to all interview questions based on their own knowledge, skills, and experience. The use of artificial intelligence (AI) tools or other third-party resources to generate, assist with, or provide interview responses during phone, virtual, or in-person interviews is not permitted. Any use of AI-assisted responses during the interview process may result in disqualification from further consideration for employment.
We are an Equal Opportunity Employer/Veterans/Disabled
How well do you match this job?
Free account needed to apply or save. It takes a minute, and we'll bring you right back to this job.