Applied Scientist, Artificial General Intelligence, AGI Data Services

Amazon • Bellevue, WA • 2h ago

Description

The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems.

Key job responsibilities

As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services.

A day in the life

An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.

Basic Qualifications

Master's degree in computer science, mathematics, statistics, machine learning or equivalent quantitative field

Experience programming in Java, C++, Python or related language

Experience with SQL and an RDBMS (e.g., Oracle) or Data Warehouse

Preferred Qualifications

Experience implementing algorithms using both toolkits and self-developed code

Have publications at top-tier peer-reviewed conferences or journals

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you're applying in isn't listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits .

USA, WA, BELLEVUE - 136,000.00 - 184,000.00 USD annually

Amazon

Go to Apply

Share this job

Similar Jobs

Sales Manager

Convergint Technologies • Renton, WA • 24min ago

Management & Administrative

Description Convergint is looking for a full-time, enthusiastic, results driven and forward-thinking Sales Manager to join our amazing culture. In this role, you will be responsible for sales and business development act...

24min ago

Go to Apply

Armed Security Officer

Allied Universal • Auburn, WA • 2h ago

Management & Administrative

Company Overview: Allied Universal®, North America's leading security and facility services company, offers rewarding careers that provide you a sense of purpose. While working in a dynamic, welcoming, and collaborative...

2h ago

Go to Apply

Principal Software Engineering Manager - CTJ - Top Secret

Microsoft Corporation • Redmond, WA • 5h ago

Management & Administrative

Overview Security represents the most critical priorities for our customers in a world awash in digital threats, regulatory scrutiny, and estate complexity. Microsoft Security aspires to make the world a safer place for...

5h ago

Go to Apply

NPO Product TPM, Amazon Infrastructure Service - Manufacturing Operation

Amazon • Bellevue, WA • 7h ago

Management & Administrative

Description AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers an...

7h ago

Go to Apply

Customer Service Associate - Temporary

Walgreens • Federal Way, WA • 8h ago

Management & Administrative

Job Description: Models and delivers a distinctive and delightful customer experience. Registers sales on assigned cash register, provides customers with courteous, fair, friendly, and efficient checkout service. Custome...

8h ago

Go to Apply

Customer Service - Self Storage Manager

Public Storage • Bellevue, WA • 8h ago

Management & Administrative

Company Description Public Storage is the self-storage industry leader and we are Hiring Now! Earn $22.00 Per Hour Our Benefits Total Rewards package available to our team: We work Flexible and Full-Time Schedules betwee...

8h ago

Go to Apply

Senior Supply Chain Manager, NA Sort Centers, Engineering Capacity Planning, NASC Engineering

Amazon • Bellevue, WA • 14h ago

Management & Administrative

Description Engineering Capacity Planning within the NA Sort Center (NASC) Engineering team is looking for a Sr. Supply Chain Manager with a strong delivery record and proven project management experience to own strategi...

14h ago

Go to Apply

Senior Supply Chain Manager, NA Sort Centers, Engineering Capacity Planning, NASC Engineering

Amazon • Bellevue, WA • 14h ago

Management & Administrative

14h ago

Go to Apply

Supply Chain Manager, SSD Supply Chain- design & execution

Amazon • Bellevue, WA • 14h ago

Management & Administrative

Description Join Amazon's dynamic environment as a Supply Chain Manager with a design and execution background, where you will play a crucial role in shaping and optimizing various facets of our operations, concentrating...

14h ago

Go to Apply

Part-Time Administrative Assistant

Aston Carter • Redmond, WA • 14h ago

Management & Administrative

We are seeking a dedicated Administrative Assistant who will perform a variety of complex administrative functions for two senior leaders. The role involves managing schedules and calendars, coordinating travel arrangeme...

14h ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.