Senior/Staff Software Engineer (Back-End), Inference
Remote · San Francisco · New York · Ottawa · Toronto · Montreal · Full-time · Remote · Posted Oct 15, 2026
At a glance
- Employment
- Full-time
- Workplace
- Remote
- Location
- Remote · San Francisco · New York · Ottawa · Toronto · Montreal
About the role
- This role is for a highly experienced Back-End Software Engineer who wants to work on core systems that shape how the platform (Model Vault) operates.
- You will help define the architecture of the platform where reliability, security, performance, and simplicity all matter, and where many of the problems do not have off-the-shelf solutions.
Responsibilities
- Evolve and scale the backend systems that power Model Vault, including authentication and authorization, deployment management, billing, metering, and customer lifecycle.
- Lead major architectural initiatives across identity, payments, usage systems, platform scalability, and other core product foundations.
- Build highly reliable control-plane services for provisioning, configuring, scaling, and operating model deployments across clouds and hardware platforms.
- Design APIs and distributed systems with strong guarantees around correctness, consistency, isolation, failure recovery, idempotency, and observability.
- Build security and privacy into the platform architecture, including tenant isolation, least-privilege access, auditability, secrets management, encryption, and confidential computing boundaries.
- Solve complex problems that span backend systems, cloud infrastructure, security, and AI inference, where conventional SaaS patterns are often not enough.
- Drive projects end-to-end, from architecture and threat modeling through implementation, rollout, production operations, and iteration based on real customer workloads.
Requirements
- You are a strong backend engineer with excellent fundamentals in distributed systems, databases, networking, operating systems, and security.
- You have strong programming fundamentals and experience building production systems in languages such as Go, Python, Java, C++, Rust, or similar.
- You have designed and operated production systems where correctness, availability, security, and performance mattered at significant scale.
- You can take ambiguous, high-impact technical problems, define the right architecture, and drive them through to reliable production systems.
- You have strong systems-design instincts and understand concepts such as concurrency, consistency, caching, queues, rate limiting, backpressure, failure domains, idempotency, and graceful degradation.
- You think deeply about security boundaries and understand authentication, authorization, identity, cryptography, secrets, tenant isolation, and how small implementation choices can create significant security risks.
- You are comfortable reasoning across backend services, APIs, databases, distributed systems, and the cloud infrastructure they run on.
- You are comfortable diving into unfamiliar codebases and debugging complex problems that cross service or system boundaries.
- You care about the details of production engineering, including instrumentation, capacity planning, migrations, backwards compatibility, incident response, and safe rollouts.
- You have a track record of taking technically difficult projects from an unclear starting point to systems that other engineers and customers depend on.
Benefits
- A weekly lunch stipend of $75/£75 or equivalent in your local currency for lunch.
- Full health and dental benefits, including a separate budget for mental health.
- RRSP matching, 401K, Pension Scheme.
- 100% Parental Leave top-up for up to 6 months, for either parent.
- Annual enrichment benefits including arts & culture, fitness/wellness, quality time, and a workspace improvement credit.
- Education & learning stipend for conferences, courses, and coaching.
- 6 weeks of paid vacation (30 working days)
- Budget for traveling to other offices if you are remote, plus an annual company offsite.
Workplace
- Cohere is remote-friendly, but we also have offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul with more opening soon.
- For those in the office: a daily lunch program, plenty of snacks, and regular community and social events.
- For those not near an office: a co-working benefit so you can work alongside others in your city.
- Everyone receives a $500 home office stipend to set up your workspace properly.
Skills
- Go
- Python
- Java
- C++
- Rust
