Sr Applied Scientist

Uber • Seattle, WA • 3h ago

About the Role

As AI systems-particularly LLMs and agentic AI-become core to our products and internal platforms, understanding how these systems fail is just as important as improving their performance. We are looking for a Senior Applied Scientist to join our AI Red Teaming efforts and focus on adversarial evaluation, failure analysis, and risk discovery in AI models and AI agents.

In this role, you will systematically probe AI systems to uncover unsafe, unintended, or harmful behaviors, including prompt injection, jailbreaks, behavioral drift, tool misuse, and @context or memory poisoning. You will design experiments, build evaluation frameworks, and analyze outcomes to surface risks that traditional ML metrics do not capture.

This role is ideal for a data scientist who enjoys working at the edge of model behavior, cares deeply about safety and robustness, and wants to apply scientific rigor to securing real-world AI systems.

What the Candidate Will Need / Bonus Points

Design and execute AI red-teaming experiments against LLMs and AI agents to identify: prompt injection (direct & indirect), jailbreaking and policy bypass, model and tool poisoning, @context and memory poisoning, behavioral drift and unsafe autonomy

Develop adversarial datasets, probes, and test harnesses to systematically evaluate model and agent behavior under attack.

Define and track AI risk metrics beyond accuracy (e.g., failure rates, drift indicators, unsafe action likelihood, confidence miscalibration).

Analyze agent workflows and decision traces to understand how failures emerge across multi-step reasoning and tool use.\

Collaborate with security engineers and AI platform teams to translate findings into guardrails, mitigations, and design improvements.

Build reusable evaluation pipelines to support continuous red teaming and regression testing as models and agents evolve.

5+ years of experience as a Data Scientist, Applied Scientist, or ML Scientist.

Hands-on experience working with LLMs or generative AI systems.

Direct experience with AI red teaming, model safety, or adversarial evaluation.

Direct experience with prompt injection, jailbreaks, and LLM failure modes.

Strong background in experimental design, evaluation, and statistical analysis.

Experience analyzing complex model behavior and failure cases beyond standard metrics.

Proficiency in Python and common DS/ML tooling.

Experience evaluating agentic systems, including tool use, memory, or multi-step workflows.

Knowledge of GenAI architectures (transformers, embeddings, RAG, agent frameworks).

Experience building custom evaluation datasets or simulation environments.

Background or strong interest in security, privacy, or trust & safety.

Familiarity with AI evaluation tools (e.g., custom judges, LLM-as-judge, simulation frameworks).

~~ ~~

Uber is proud to be an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you have a disability or special need that requires accommodation, please let us know by completing this form- https://docs.google.com/forms/d/e/1FAIpQLSdb_Y9Bv8-lWDMbpidF2GKXsxzNh11wUUVS7fM1znOfEJsVeA/viewform

Uber

Go to Apply

Share this job

Similar Jobs

Physical Therapist

U.S. Physical Therapy • Bellevue, WA • 1h ago

Hospital & Medical

Company Description Location: Bellevue, WA (Factoria) Salary Range : $92,000- $102,000 Schedule: Monday - Friday Bonuses : $5,000 Sign-on bonus and Production Bonuses Why Join Sports Physical Therapy? At SPT - Factoria y...

1h ago

Go to Apply

Front Office Coordinator

U.S. Physical Therapy • Kirkland, WA • 1h ago

Retail & Hospitality

Company Description Sports Physical Therapy offers performance based Physical Therapy, Athletic Training, & Personal Training. When you join our team, you will be a part of something bigger as an important contributo...

1h ago

Go to Apply

Physical Therapy Aide

ATI Physical Therapy • Seattle, WA • 1h ago

Hospital & Medical

Overview Join a collaborative team dedicated to delivering exceptional patient care as a Physical Therapy Aide , while gaining valuable healthcare skills that can launch your career. Work alongside leading experts in Phy...

1h ago

Go to Apply

Front Office Coordinator

U.S. Physical Therapy • Bellevue, WA • 1h ago

Retail & Hospitality

1h ago

Go to Apply

Senior Manager, Product Management - Ad Formats

Snap Inc. • Bellevue, WA • 3h ago

Management & Administrative

Snap Inc ( https://www.snap.com/en-US/ ) is a technology company. We believe the camera presents the greatest opportunity to improve the way people live and communicate. Snap contributes to human progress by empowering p...

3h ago

Go to Apply

Staff Data Scientist, Tech

Uber • Seattle, WA • 3h ago

IT & Engineering

About the Role In this role, you will be the primary technical lead responsible for evaluating the safety and maturity of Autonomous Vehicle (AV) partners looking to launch on the Uber platform. Your mission is to conduc...

3h ago

Go to Apply

Sr Data Scientist, Tech

Uber • Seattle, WA • 3h ago

IT & Engineering

About the Role As a Senior Systems Engineer in our Autonomous Mobility & Delivery (AM&D) group, you will play a pivotal role in shaping the future of autonomous transportation at Uber. You will be responsible for...

3h ago

Go to Apply

STINGRAI Data Engineer/ Architect (Data Scientist 2)- 29206

Huntington Ingalls Industries • Bremerton, WA • 3h ago

IT & Engineering

Requisition Number: 29206 Required Travel: 11 - 25% Employment Type: Full Time/Salaried/Exempt Anticipated Salary Range: $86,511.00 - $137,000.00 Security Clearance: Secret Level of Experience: Mid HI This opportunity re...

3h ago

Go to Apply

Sr Product Manager, Tech

Uber • Seattle, WA • 3h ago

Hospital & Medical

About the Role Are you an AI/ML enthusiast? Does working on Agentic AI excite you? Are you thrilled about building great user experiences? If yes, read on! At Uber, Generative AI is becoming a foundational technology acr...

3h ago

Go to Apply

Restaurant Server

Marriott • Seattle, WA • 3h ago

Retail & Hospitality

Additional Information Job Number 26078946 Job Category Food and Beverage & Culinary Location 2100 Alaskan Way, Seattle, Washington, United States, 98121 VIEW ON MAP ( https://www.google.com/maps?q=2100%20Alaskan%20W...

3h ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)