Staff Site Reliability Engineer - Dev Platform

Accountabilities

• Own the reliability, availability, and performance of the production AI Gateway, defining and maintaining service-level objectives (SLOs), error budgets, and on-call practices.

• Design and implement resilience mechanisms, including multi-provider model routing, failover, retries, rate limiting, and graceful degradation when external model providers experience disruptions.

• Set technical direction for gateway architecture, capacity planning, and scaling to accommodate increasing LLM traffic and expanding workloads.

• Develop comprehensive observability for AI traffic, including latency, errors, token consumption, and cost attribution by team and use case.

• Collaborate with Security, Finance, and AI governance stakeholders to implement access controls, usage budgets, audit logging, and appropriate data-handling policies.

• Lead incident response and conduct blameless post-incident reviews, translating findings into long-term reliability improvements.

• Automate infrastructure and operational processes through infrastructure as code, safe deployment strategies, and progressive rollouts.

• Support secure and reliable integrations with LLM providers and internal AI services.

• Mentor engineers, establish engineering best practices, and raise operational standards across the developer platform organization.

• Influence technical decisions across teams through clear design documentation, architectural guidance, and collaborative leadership.

Requirements

• Extensive experience in site reliability engineering, infrastructure engineering, or platform engineering, with a proven track record of operating high-traffic, business-critical production services.

• Deep hands-on experience with Kubernetes, a major cloud provider, and infrastructure as code tools such as Terraform.

• Strong understanding of SLOs, observability, incident management, and reliability engineering principles, with the ability to prioritize improvements based on business impact.

• Proficiency in a programming language such as Rust or Python, along with the ability to read, troubleshoot, and contribute to production codebases.

• Experience with API gateways, reverse proxies, or traffic management systems, including load balancing, rate limiting, failover, and caching.

• Practical experience with Consul and Vault for service discovery, configuration management, and secrets management.

• Knowledge of authentication and authorization mechanisms for internal platforms and APIs, including OAuth 2.0/OIDC, JWT, mutual TLS (mTLS), and API keys, particularly in multi-tenant environments.

• Experience with model evaluation systems, including building or operating evaluation pipelines to compare models, prompts, and agents and identify quality regressions before and after deployment.

• Familiarity with coding agents and agent harnesses, such as OpenHands, Claude Code, or comparable tools, including safe execution in sandboxed environments.

• Experience with agent memory systems and supporting data stores, such as vector databases, search systems, relational databases, and key-value stores, with an emphasis on reliable operations.

• Staff-level technical leadership, including the ability to establish direction across teams, write effective design documents, and build alignment around technical decisions.

• Strong judgment when balancing reliability, infrastructure costs, and developer productivity.

Nice to have:

• Experience operating LLM gateways or integrating LLM provider APIs at scale, including streaming, token-based rate limiting, model routing, and caching.

• Experience hosting and serving models using technologies such as vLLM, or operating distributed inference and training workloads with Ray, including GPU scheduling, batching, autoscaling, and cost-per-token optimization.

• Experience designing, developing, and maintaining Kubernetes controllers and operators using tools such as controller-runtime or Kubebuilder, including custom resource definitions, reconciliation loops, and safe upgrades.

• Familiarity with cost attribution, budgeting, or FinOps practices for usage-based services.

• Experience working in security-sensitive or regulated environments, such as financial services or cryptocurrency.

• Contributions to open-source projects involving gateways, proxies, or observability.

Benefits

• Remote work: Opportunity to work remotely from Brazil as part of an internationally distributed organization.

• High-impact engineering: Help build and operate critical infrastructure supporting AI-enabled software development.

• Technical ownership: Influence platform architecture, reliability strategy, and engineering standards at the staff level.

• Cross-functional collaboration: Work with engineering, security, finance, and AI governance teams on complex technical challenges.

• Professional growth: Deepen expertise in cloud-native infrastructure, LLM platforms, distributed systems, and AI operations.

• Technical leadership: Mentor engineers and help shape best practices across a growing developer platform.

• Global environment: Collaborate with colleagues and stakeholders across international teams.