Engineering Program Manager, Siri and Apple Intelligence Evaluation

Apple • Cupertino, CA • 6h ago

Role Number: 200649715-0836

Summary

Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other's ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the world, changing lives for the better. It's the diversity of our people and their thinking that inspires the innovation that runs through everything we do. When we bring everybody in, we can do the best work of our lives. Here, you'll do more than join something - you'll add something.

We're looking for a Program Manager with a strong track record of building and leading effective evaluation programs demonstrating success leading via data. As a leader in the AIML Siri and Apple Intelligence evaluation team, you will lead initiatives in evaluation for foundation models and GenAI features powering Siri and Apple Intelligence experiences that are critical to Apple's future.

Description

You will work with top tier data scientists, engineers, research teams, and product teams across Apple to help ensure we deliver high-quality, safe, and beneficial AI-powered experiences that over 1 billion customers expect and love. This role requires technical depth in evaluation methodologies combined with strong program management expertise to drive comprehensive assessment of model capabilities, safety, helpfulness, and user experience quality.

Minimum Qualifications

Bachelor's degree in Statistics, Business Intelligence, Computer Science, other Quantitative Sciences, or related field and equivalent experience

8+ years of experience in driving large scale program building machine learning powered products or analytics to support product development

5+ years of experience managing programs in AI powered product space, preferably experience in evaluation of ML/AI products

Ability to deal with ambiguities, drive disambiguation and clarities around evaluation methodologies, shepherd multiple teams to converge on rigorous measurement frameworks

Experience designing and implementing evaluation systems for machine learning models, particularly large language models or conversational AI systems

Program management skills including program structuring and managing multiple work streams interdependently across research, engineering, and product teams

Problem-solving skills with attention to details in identifying edge cases, failure modes, and capability gaps

Ability to communicate abstract ideas clearly, manage comprehensive yet succinct program status updates to all levels of audience, both verbally and in written forms

Proven adaptability and agility in making adjustments to program strategy and plan with evolving model capabilities and product decisions

Preferred Qualifications

Master's or PhD degree in Statistics, Machine Learning, Computer Science, other Quantitative Sciences, or related field and equivalent experience

Experience with statistical analysis and drawing meaningful conclusions from large-scale evaluation datasets

Deep understanding of LLM capabilities, limitations, and safety considerations

Self-sufficient in analyzing and drawing conclusions about model quality, user experience, and product opportunity from raw and refined evaluation data

Player-coach capable of personally leading large evaluation initiatives while coaching team members along the way and mentoring team members to grow evaluation expertise

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant (https://www.eeoc.gov/sites/default/files/2023-06/22-088\EEOC\KnowYourRights6.12ScreenRdr.pdf) .

Management & Administrative

Description Amazon Web Services is seeking a results-driven Account-Based Marketing (ABM) Industry Marketing Manager to design and execute high-impact ABM programs for enterprise customers for some of our largest, most c...

10h ago

Go to Apply

Account Based Marketing Manager, Strategic Accounts, AWS Industry Marketing

Amazon • Santa Clara, CA • 10h ago

Management & Administrative

10h ago

Go to Apply

Account Manager - Cybersecurity | Remote, SF Bay Area

Optiv • San Jose, CA • 10h ago

Management & Administrative

As an Account Manager, (or Client Manager (CM) as we call it) you'll be responsible for selling Optiv security services and security technology solutions to a select few strategic accounts in the San Francisco Bay Area....

10h ago

Go to Apply

Assistant Vice President, Electrical Engineer - Buildings

WSP USA • San Jose, CA • 13h ago

Management & Administrative

This Opportunity WSP is currently initiating a search for an Assistant Vice President, Electrical Engineer to join our Property & Buildings team in California . This position may be based out of any WSP office in Cal...

13h ago

Go to Apply

Administrative Assistant

Amergis • Los Gatos, CA • 22h ago

Management & Administrative

Amergis Healthcare Staffing is looking for qualified Administrative Assistants to be responsible for a variety of administrative and clerical duties to help improve efficiencies at an organization. The Administrative Ass...

22h ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.