Member of Technical Staff, Data Platform
New York City, NY, United States11h ago
Key details
- Salary
- $140K–$200K/yr
- Location
- New York City, NY, United States
- Work style
- Hybrid
- Level
- Senior
- Job type
- Full-time
- Function
- Data & ML
- Focus area
- Data engineering & Analytics
- Posted
- 11h ago
Skills
About the role
About Mecka AI
Mecka AI is building the data infrastructure layer for robotics and embodied AI.
We design and operate global systems for data capture, data labeling, and hardware-enabled workflows used by leading AI labs and robotics companies to train and validate humanoid and embodied AI systems.
We work closely with frontier robotics teams to bridge real-world data, simulation, learning-based systems, and deployed hardware.
The role
When a robotics team trains on Mecka’s data, they need to trust what they’re getting. You’ll define what that trust rests on: the benchmarks, audits and quality standards behind every delivery. You’ll dig into the failures that averages hide, question results that look too good and make the call on whether data is ready to ship.
You’ll work across Computer Vision, annotation, operations and delivery. We’re looking for someone with a machine learning and data science background who enjoys getting to the bottom of a difficult measurement problem, and can turn what they find into a quality bar the whole team can use.
What you’ll do
Build benchmarks we can trust. You’ll own which evaluation sets exist, how they’re sampled, which metrics apply and how versions are frozen and compared. Your scope will span 3D pose, SLAM, multimodal checks and delivered datasets.
Connect quality standards to customer experience. You’ll write customer quality specifications and audit rubrics, and keep them aligned with what customers actually reject.
Make human review count. You’ll run audits and work with annotation and operations on sampling, agreement, throughput and cost.
Automate checks without creating blind spots. You’ll build checks and graders that reduce manual review costs and are hard to game.
Own the decision to ship. You’ll track where recorded hours are lost or rejected, report quality results regularly and decide whether data meets the bar for delivery.
What you’ll bring
Experience measuring quality at scale. You’ve worked in evaluation, measurement or data quality for machine learning systems, or in data operations at scale.
A machine learning and data science background. You bring strong Python skills and practical experience applying statistics, including sampling, precision and recall, agreement and confidence intervals.
Experience building evaluations from scratch. You’ve designed evaluation protocols and ground-truth pipelines, including the decisions that make their results trustworthy.
The judgment to challenge results. You can question a research team’s numbers, investigate discrepancies and support your conclusions with enough evidence to drive a decision.
An eye for misleading metrics. You can spot how a metric could be gamed and build checks that close those gaps.
Nice to have
Experience in one or more of these areas would be valuable:
Benchmarks and metrics: 3D pose, trajectory and reconstruction metrics such as MPJPE and ATE.
Audits and data quality: Sampling design, inter-annotator agreement, tracking where captured data is lost or rejected, and managing annotators or vendors.
Automated checks and graders: Verifiers, learned scoring, active learning and human-in-the-loop evaluation.
Data curation and scoring: Quality scoring, filtering and composition of large datasets.
If you don't check every box above but believe you can do the job, we encourage you to apply — we're looking for capability and trajectory, not a perfect checklist match.
We strive to create an inclusive work environment for all; we welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, we will partner with you to meet your needs.
We may use AI-assisted tools to help evaluate applications against the requirements of this role. These tools support our recruiting team in identifying qualified candidates but do not replace human judgment or limit which applications may be reviewed or considered.
How well do you match this job?
Free account needed to apply or save. It takes a minute, and we'll bring you right back to this job.