At a glance
- Experience
- 4+ years experience
- Location
- India
About Coursera
- Coursera and Udemy are now one company, bringing together two mission-driven brands to create the world’s most powerful platform for turning learning into progress. Together, we help more than 300 million learners and 12,000+ enterprise customers build the skills they need for a world being reshaped by AI. Read more about the combined company by visiting our blog.
About the role
- As an AI Platform and LLMOps Engineer, you will join a fast-paced innovation team at Coursera working on AI enablement for our organization and customers.
- You will own the operational backbone that lets our agentic AI systems run reliably in production—from prompt and pipeline versioning to evaluation, monitoring, incident and cost management.
Responsibilities
- Build and maintain the operational tooling for our LLM-powered systems, including prompt/pipeline versioning, evaluation harnesses, and CI/CD workflows tailored to non-deterministic AI outputs
- Implement monitoring and observability for production LLM systems — tracking latency, token usage, cost per request, output quality, and drift over time
- Design and run automated evaluation suites to catch regressions, hallucinations, and quality degradation before they reach customers
- Manage RAG pipelines and vector store infrastructure, keeping retrieval sources fresh, accurate, and performant
- Implement safety and compliance guardrails — content filtering, PII redaction, and access controls — in line with enterprise data privacy and residency requirements
- Own cost governance for LLM usage: caching strategies, model routing, and usage reporting to keep spend predictable as adoption scales
- Collaborate closely with Product Managers and Senior Engineers to scope operational requirements and translate them into reliable systems
- Participate in sprint planning, technical grooming, and retrospective discussions
Requirements
- Bachelor’s or Master’s degree in Computer Science, Computer Engineering or a related field
- 4+ years of experience in a software engineering role, with a solid background in Backend Engineering and DevOps fundamentals
- Hands-on experience operating or supporting LLM-powered systems in production (via APIs, RAG pipelines, frontier/open weight/fine-tuned models)
- Proficiency in a backend language such as Python, Java, TypeScript or Go, and familiarity with Docker and container orchestration (Kubernetes)
- Experience implementing APIs, working with SQL and NoSQL databases, and writing automated tests
- Working knowledge of at least one cloud platform (AWS, GCP, or Azure), across both managed and self-hosted services
- Familiarity with infrastructure-as-code (e.g., Terraform) and CI/CD tooling (e.g., GitHub Actions, Jenkins)
- Comfort debugging production issues involving non-deterministic systems, with attention to detail and a bias towards reliability
- Strong belief in engineering quality and building tooling that creates leverage for others
Preferred qualifications
- Experience with LLMOps-specific tooling such as LangFuse, LangSmith, Weights & Biases-style evaluation frameworks, or vector databases like Pinecone, Weaviate, pgvector
- Working knowledge of RAG architectures, MCP implementation & governance, agent orchestration frameworks like LangGraph and LLM Gateways such as LiteLLM, Open Router & enterprise AI ecosystems such as Vertex AI, Bedrock
- Exposure to prompt management and versioning practices treated as code along with model access management, deterministic guardrails and evals
- Understanding of AI governance considerations — data privacy, residency, and compliance in enterprise AI deployments
- Ability to work in a fast-paced, ambiguous environment with a proactive, ownership-driven mindset
- Strong communication skills and comfort collaborating across engineering, product, and strategy functions
Skills
- Python
- Java
- TypeScript
- Go
- Docker
- Kubernetes
- SQL
- NoSQL
- AWS
- GCP
- Azure
- Terraform
Show all 14 skills
- GitHub Actions
- Jenkins
