Senior Applied Scientist - AI Red Teaming & Model Risk

Uber • Seattle, WA • 2h ago

About the Role

As AI systems-particularly LLMs and agentic AI-become core to our products and internal platforms, understanding how these systems fail is just as important as improving their performance. We are looking for a Senior Applied Scientist to join our AI Red Teaming efforts and focus on adversarial evaluation, failure analysis, and risk discovery in AI models and AI agents.

In this role, you will systematically probe AI systems to uncover unsafe, unintended, or harmful behaviors, including prompt injection, jailbreaks, behavioral drift, tool misuse, and @context or memory poisoning. You will design experiments, build evaluation frameworks, and analyze outcomes to surface risks that traditional ML metrics do not capture.

This role is ideal for a data scientist who enjoys working at the edge of model behavior, cares deeply about safety and robustness, and wants to apply scientific rigor to securing real-world AI systems.

What the Candidate Will Need / Bonus Points

Design and execute AI red-teaming experiments against LLMs and AI agents to identify: prompt injection (direct & indirect), jailbreaking and policy bypass, model and tool poisoning, @context and memory poisoning, behavioral drift and unsafe autonomy

Develop adversarial datasets, probes, and test harnesses to systematically evaluate model and agent behavior under attack.

Define and track AI risk metrics beyond accuracy (e.g., failure rates, drift indicators, unsafe action likelihood, confidence miscalibration).

Analyze agent workflows and decision traces to understand how failures emerge across multi-step reasoning and tool use.\

Collaborate with security engineers and AI platform teams to translate findings into guardrails, mitigations, and design improvements.

Build reusable evaluation pipelines to support continuous red teaming and regression testing as models and agents evolve.

5+ years of experience as a Data Scientist, Applied Scientist, or ML Scientist.

Hands-on experience working with LLMs or generative AI systems.

Direct experience with AI red teaming, model safety, or adversarial evaluation.

Direct experience with prompt injection, jailbreaks, and LLM failure modes.

Strong background in experimental design, evaluation, and statistical analysis.

Experience analyzing complex model behavior and failure cases beyond standard metrics.

Proficiency in Python and common DS/ML tooling.

Experience evaluating agentic systems, including tool use, memory, or multi-step workflows.

Knowledge of GenAI architectures (transformers, embeddings, RAG, agent frameworks).

Experience building custom evaluation datasets or simulation environments.

Background or strong interest in security, privacy, or trust & safety.

Familiarity with AI evaluation tools (e.g., custom judges, LLM-as-judge, simulation frameworks).

For New York, NY-based roles: The base salary range for this role is USD$190,000 per year - USD$211,000 per year. For San Francisco, CA-based roles: The base salary range for this role is USD$190,000 per year - USD$211,000 per year. For Seattle, WA-based roles: The base salary range for this role is USD$190,000 per year - USD$211,000 per year. For Sunnyvale, CA-based roles: The base salary range for this role is USD$190,000 per year - USD$211,000 per year. For all US locations, you will be eligible to participate in Uber's bonus program, and may be offered an equity award & other @types of comp. All full-time employees are eligible to participate in a 401(k) plan. You will also be eligible for various benefits. More details can be found at the following link https://jobs.uber.com/en/benefits.

Uber is proud to be an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you have a disability or special need that requires accommodation, please let us know by completing this form- https://docs.google.com/forms/d/e/1FAIpQLSdb_Y9Bv8-lWDMbpidF2GKXsxzNh11wUUVS7fM1znOfEJsVeA/viewform

Uber

Go to Apply

Share this job

Similar Jobs

Flight Path Innovation Engineer

GE Aerospace • Redmond, WA • 2h ago

IT & Engineering

Job Description Summary The Flight Path Innovation Engineer will tackle the same core challenges facing both the global flight path design community and our navigation services team: delivering instrument procedures fast...

2h ago

Go to Apply

Staff Machine Learning Engineer, Platform & Production

Warner Bros. Discovery • Bellevue, WA • 2h ago

Hospital & Medical

Welcome to Warner Bros. Discovery... the stuff dreams are made of. Who We Are... When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wond...

2h ago

Go to Apply

Senior Data Scientist, Level 5

Snap Inc. • Seattle, WA • 2h ago

IT & Engineering

Snap Inc ( https://www.snap.com/en-US/ ) is a technology company. We believe the camera presents the greatest opportunity to improve the way people live and communicate. Snap contributes to human progress by empowering p...

2h ago

Go to Apply

Senior Data Scientist, Level 5

Snap Inc. • Bellevue, WA • 2h ago

IT & Engineering

2h ago

Go to Apply

Expert Data Scientist - EHS

The Boeing Company • Renton, WA • 2h ago

IT & Engineering

Job Description At Boeing, we innovate and collaborate to make the world a better place. We're committed to fostering an environment for every teammate that's welcoming, respectful and inclusive, with great opportunity f...

2h ago

Go to Apply

Finance Manager, AWS Global Sales Planning & Compensation

Amazon • Seattle, WA • 2h ago

Finance, Accounting & Banking

Description At Amazon, we're working to be the most customer-centric company on earth. To get there, we need exceptionally talented, bright, creative, and driven people. If you want to help make history and have fun, Ama...

2h ago

Go to Apply

Lead Artificial Intelligence /Machine Learning Data Scientist (Data Science)

The Boeing Company • Seattle, WA • 2h ago

IT & Engineering

2h ago

Go to Apply

Sr. Data Scientist , Companion Product & Servi (ComPAS)

Amazon • Seattle, WA • 2h ago

IT & Engineering

Description ComPAS Business Insights sits at the intersection of pricing, marketing, and consumer science for Amazon's Companion Products & Services portfolio; spanning Accessories, Pre-Owned Business (POB), and Trad...

2h ago

Go to Apply

Sr. Technical Program Manager, Shopping Core Foundations - BuyForMe

Amazon • Seattle, WA • 2h ago

Management & Administrative

Description We are building the next generation of autonomous AI agents that enable Amazon customers to seamlessly discover and purchase products across the open web. Our systems operate in highly dynamic real-world envi...

2h ago

Go to Apply

Software Development Engineer, Hiring Intelligence, Science & Experimentation (HIRISE)

Amazon • Seattle, WA • 2h ago

IT & Engineering

Description Do you want a role with deep meaning and the ability to make a major impact? As part of Intelligent Talent Acquisition (ITA), you'll have the opportunity to reinvent the hiring process and deliver unprecedent...

2h ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)