Back to Jobs search

AI Quality Analyst-# 26-22273

US Tech Solutions • Lincolnshire, IL • 7h ago

$55-$56 per hour

Lincolnshire, IL

Contract

Duration: 5+ Months Contract

Hybrid Position

Job Description:

The AI Quality Analyst is a critical role responsible for ensuring the performance, safety, and reliability of our cutting-edge AI/ML models. You will be at the forefront of our development lifecycle, designing and executing comprehensive evaluation strategies to identify model weaknesses, potential biases, and critical edge cases. This role requires a blend of analytical rigor, technical aptitude, and a deep curiosity for how AI models behave in real-world scenarios. You will not just find bugs, but provide the actionable insights that drive model improvement and guide our research and development efforts.

Responsibilities:

Evaluation Strategy & Benchmark Development: Design, develop, and maintain a comprehensive suite of test cases and evaluation benchmarks. Proactively identify potential model failure points, including edge cases, adversarial inputs, and sources of bias.

Error Analysis & Failure Triage: Conduct systematic error analysis to categorize model failures and identify underlying patterns. Triage defects, prioritize them based on severity and impact, and work with the development team to ensure resolution.

Data Sourcing & Curation: Source, curate, and manage high-quality datasets for model evaluation and testing. This includes performing data annotation and validation to ensure the integrity of our ground-truth data.

Exploratory & Adversarial Testing (Red Teaming): Perform unscripted, exploratory testing to Client unexpected model behaviors. Participate in red teaming exercises to intentionally challenge our models and identify potential safety and security vulnerabilities.

Test Environment Management: Set up, maintain, and troubleshoot testing and demonstration environments to ensure a stable and reliable evaluation pipeline.

Reporting & Insights: Analyze and synthesize test results into clear, actionable reports for both technical and non-technical stakeholders. Translate complex findings into concrete recommendations for model improvement.

Process Improvement: Actively participate in post-hoc evaluation reviews and contribute to the continuous improvement of our testing methodologies, tools, and overall quality assurance processes.

Experience:

Proven experience in a quality assurance, testing, or data analysis role, preferably within the AI/ML domain.

A deep understanding of the machine learning lifecycle and the common failure modes of AI models.

Hands-on experience with data annotation, data validation, and managing large datasets.

Meticulous attention to detail and a methodical approach to problem-solving.

Skillsets:

Strong analytical skills with the ability to identify patterns in data and draw meaningful conclusions.

Expertise with industry-standard test automation tools and libraries (e.g., Selenium, Playwright, Cypress, REST-assured).

Experience in testing across different platforms (e.g. comprehensive testing of mobile Android/iOS and web applications)

Experience with bug tracking systems (e.g., Jira) and test case management tools.

Scripting skills (e.g., Python) for test automation and data manipulation.

(Preferred) experience in testing AI systems, including evaluating agentic responses, model performance metrics, and data integrity.

Excellent communication skills, with the ability to clearly document bugs and articulate complex technical issues.

Familiarity with computer vision or other specific AI domains relevant to our work.

Education:

Bachelors Degree

About US Tech Solutions:

US Tech Solutions is a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions. To know more about US Tech Solutions, please visit www.ustechsolutions.com .

US Tech Solutions is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, colour, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

AI Statement: By applying, you acknowledge that AI-assisted tools may be used during hiring.

LI-AS140

Similar Jobs

Inside Account Manager

O'Neal Industries, Inc. • Carol Stream, IL • 47min ago

Management & Administrative

Description Company Summary: TW Metals, LLC is a leading global distributor of Specialty Metals to companies such as Boeing, Space-X, GE, McMaster-Carr, and Eaton. TW Metals stocks and processes Tube, Pipe ( https://www....

47min ago

Go to Apply

Customer Service Associate

Walgreens • River Grove, IL • 1h ago

Management & Administrative

Models and delivers a distinctive and delightful customer experience. Registers sales on assigned cash register, provides customers with courteous, fair, friendly, and efficient checkout service. Customer Experience Enga...

1h ago

Go to Apply

Analyst Sr Supply Chain

Walgreens • Deerfield, IL • 4h ago

Manufacturing & Warehouse

Job Summary : Responsible for providing complex analysis of information, problems and/or data of a diverse scope, where analysis of data requires evaluation of identifiable factors. Works with limited supervision. Exerci...

4h ago

Go to Apply

Analyst Supply Chain

Walgreens • Deerfield, IL • 4h ago

Manufacturing & Warehouse

Job Summary: Responsible for providing moderately complex analysis of information, problems and/or data of a moderate scope, where analysis requires a review of a variety of factors. Works under general supervision and e...

4h ago

Go to Apply

Assembler

Aerotek • Buffalo Grove, IL • 4h ago

Manufacturing & Warehouse

Job Title: Production Associate Job Description We are seeking a dedicated and detail-oriented Production Associate to support assembly and production operations in a fast-paced, clean, and organized manufacturing enviro...

4h ago

Go to Apply

Assembler

Aerotek • Northbrook, IL • 4h ago

Manufacturing & Warehouse

Job Title: Mechanical Assembler - 1st Shift Job Description The Mechanical Assembler role focuses on building mechanical components, subassemblies, and finished products in accordance with engineering drawings, work inst...

4h ago

Go to Apply

Finance and Accounting Internship

Brunswick • Mettawa, IL • 4h ago

Finance, Accounting & Banking

Mettawa, IL Full time JR-051524 Are you ready for what's next? Come explore opportunities within Brunswick, a global marine leader ( https://youtube.com/watch?v=ksuQ6B2j\mA ) committed to challenging conventions and inno...

4h ago

Go to Apply

Pharmacy Operations Manager

Walgreens • Bartlett, IL • 4h ago

Hospital & Medical

Where state and federal laws/regulations allow, accountable for performing day-to-day non-clinical pharmacy operations, administrative activities; Ensures efficient pharmacy workflow and a positive patient experience. Se...

4h ago

Go to Apply

Pharmacy Operations Manager

Walgreens • Mount Prospect, IL • 4h ago

Hospital & Medical

4h ago

Go to Apply

Finance Manager, Immunology SKYRIZI Consolidation

AbbVie • Mettawa, IL • 5h ago

Finance, Accounting & Banking

Company Description About AbbVie AbbVie's mission is to discover and deliver innovative medicines and solutions that solve serious health issues today and address the medical challenges of tomorrow. We strive to have a r...

5h ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)