Software Development Engineer - Agentic Data Infrastructure, Partner Data And Analytics
Chennai, TN, India3d ago
Key details
- Location
- Chennai, TN, India
- Work style
- Onsite
- Level
- Mid level
- Job type
- Full-time
- Experience
- 3–6 yrs
- Function
- Software engineering
- Focus area
- Data engineering & Analytics
- Posted
- 3d ago
Skills
agentic aialgorithmsapi designawsbig dataconcurrencydata pipelinesdata structuresdistributed systemsgenerative aihadoopjavakafkakinesiskotlinlakehousellmmodel context protocolpythonragroot cause analysisrustscalaservice-oriented architecturesparkstate managementsupply chaintrinotypescript
About the role
The Data Services team within PDNA (Partner Data And Analytics) builds and operates the Agentic Data Platform that powers Amazon's Selling Partner analytics infrastructure. We serve 900K+ Selling Partners and 30K+ Amazonians through 4,000+ standardized metrics, real-time streaming pipelines validated at Black Friday scale (142M+ events), and the Curated Data Layer (CDL) — Amazon's strategic data infrastructure for Selling Partner GenAI applications including Seller Assistant, Supply Chain Expert, MCF, and ISS. We are building the future where intelligent agents autonomously operate data pipelines, serve analytics through MCP servers, and enable self-service insights at scale.
You will build the infrastructure layer that sits between raw data and intelligent agents. Think of it as the operating system for agentic data — the MCP servers agents call, the query engines they reason through, the schema contracts they interpret, the streaming backbone they react to, and the validation services they trust. This is systems engineering applied to the agentic AI problem: you are not building agents — you are building the infrastructure that makes agents possible.
Key job responsibilities
1) Design and build MCP servers and data access APIs
2) Engineer the real-time streaming backbone using Spark Structured Streaming, Kafka/MSK, Kinesis, and Lambda that powers agentic workflows.
3) Build agent orchestration infrastructure — the tool registries, execution planners, state stores, and handoff protocols that enable specialized agents (ingestion, validation, delivery) to coordinate across the data operations lifecycle.
4) Develop trust and validation infrastructure that keeps agent-generated outputs reliable
5) Extend the lakehouse for agentic consumption — building metadata contracts, schema registries, and catalog APIs on top of our Iceberg-based infrastructure that agents can reason over autonomously.
6) Own your systems end-to-end — design, build, test, deploy, operate, and document. Your infrastructure will be observable, self-healing, and resilient. You will participate in on-call, drive root-cause analysis, and permanently resolve operational issues.
7)
- Bachelor's degree in Computer Science, Computer Engineering, or equivalent experience
- Proficiency in one or more systems languages (Java, Python, Kotlin, TypeScript, Scala, or Rust)
- Experience designing and operating production services — API design, distributed system patterns (consistency, partitioning, fault tolerance), and service-oriented architectures
- Strong fundamentals in data structures, algorithms, concurrency, and system design
- Experience with data infrastructure — lakehouse architectures (Iceberg), query engines (Athena, Trino), or metadata/catalog systems
- Familiarity with GenAI infrastructure patterns (RAG, embedding stores, evaluation frameworks)
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
null
You will build the infrastructure layer that sits between raw data and intelligent agents. Think of it as the operating system for agentic data — the MCP servers agents call, the query engines they reason through, the schema contracts they interpret, the streaming backbone they react to, and the validation services they trust. This is systems engineering applied to the agentic AI problem: you are not building agents — you are building the infrastructure that makes agents possible.
Key job responsibilities
1) Design and build MCP servers and data access APIs
2) Engineer the real-time streaming backbone using Spark Structured Streaming, Kafka/MSK, Kinesis, and Lambda that powers agentic workflows.
3) Build agent orchestration infrastructure — the tool registries, execution planners, state stores, and handoff protocols that enable specialized agents (ingestion, validation, delivery) to coordinate across the data operations lifecycle.
4) Develop trust and validation infrastructure that keeps agent-generated outputs reliable
5) Extend the lakehouse for agentic consumption — building metadata contracts, schema registries, and catalog APIs on top of our Iceberg-based infrastructure that agents can reason over autonomously.
6) Own your systems end-to-end — design, build, test, deploy, operate, and document. Your infrastructure will be observable, self-healing, and resilient. You will participate in on-call, drive root-cause analysis, and permanently resolve operational issues.
7)
Basic qualifications
- 3+ years of big data technologies such as AWS, Hadoop, Spark, Pig, Hive, Lucene/SOLR or Storm/Samza experience- Bachelor's degree in Computer Science, Computer Engineering, or equivalent experience
- Proficiency in one or more systems languages (Java, Python, Kotlin, TypeScript, Scala, or Rust)
- Experience designing and operating production services — API design, distributed system patterns (consistency, partitioning, fault tolerance), and service-oriented architectures
- Strong fundamentals in data structures, algorithms, concurrency, and system design
- Experience with data infrastructure — lakehouse architectures (Iceberg), query engines (Athena, Trino), or metadata/catalog systems
Preferred qualifications
- Experience with agentic AI infrastructure — tool registries, agent orchestration, LLM integration, or agent state management- Familiarity with GenAI infrastructure patterns (RAG, embedding stores, evaluation frameworks)
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
null
How well do you match this job?
Skills
Experience
Level
Title
Location
Salary
Freshness
Similarity
Free account needed to apply or save. It takes a minute, and we'll bring you right back to this job.