Staff Software Engineer, Applied AI

San Francisco · USD 194,000-267,000 / year · Posted Oct 15, 2026

At a glance

The Staff Software Engineer will join a team building Digital Workers, a suite of autonomous agents for identity operations and security.
This role involves working directly with Okta's CEO to develop reliable agent harnesses and evaluation systems for enterprise-scale AI applications.
The position offers the ownership and pace of a startup environment supported by the resources of a public company.

Build agent harnesses for planning, tool use, memory, context compaction, interruption, and recovery.
Integrate OpenAI and Anthropic model APIs while managing quality, latency, and cost.
Maintain harness connectivity to AWS workflow execution and approval state.
Implement application policy checks for authorization and sensitive writes.
Build evaluation datasets and replay tests for findings, access decisions, and agent onboarding.
Measure metrics including false positives, missed risks, unsafe actions, and completion rates.
Turn production failures into regression tests.
Test for prompt injection, stale evidence, conflicting instructions, and unsafe tool use.
Improve prompts, retrieval, tools, and model routing.
Evaluate fine-tuning or post-training methods when results justify them.

Strong Python engineering skills.
Experience shipping an LLM application that uses tools.
Hands-on experience building agent workflows or evaluation systems for tool-using LLM applications.
Experience integrating model APIs, structured outputs, and asynchronous services.
Depth in agent execution or evaluation systems with explainable failure analysis.
Ability to distinguish plausible model responses from supported findings or verified actions.