Details: Job Title: LLM Evaluation Expert Location: Remote Start Date: Right Away Keywords: #RemoteJobs; #LLMEvaluationExpertJobs; Compensation: This position is offering a hourly rate range from $57 - $59 an hour. A LLM Evaluation Expert Job Virtually is currently available through Belcan. In this role you will play a crucial role in assessing and improving our language models' coding capabilities. If you are interested in this role, Apply Today! Job Description: Actual Job Title: LLM Evaluation Expert - Domain: Coding The role will follow the qualifications and job duties below. Only 10 hours per week, no wiggle room on this. Company Overview: Artificial General Intelligence (AGI) Data Services is at the forefront of AI innovation, specializing in the development and refinement of large language models (LLMs). Our mission is to create AI systems that can understand and generate high-quality code, revolutionizing the software development landscape. Job Description: As an LLM Evaluation Expert specializing in Coding, you will play a crucial role in assessing and improving our language models' coding capabilities. Your expertise will be instrumental in evaluating LLM-generated code responses, making high-level judgments, and setting the standard for what constitutes excellent AI-assisted coding. Key Responsibilities:

Critically analyze and evaluate code responses generated by our LLMs across various programming languages and paradigms

Exercise expert judgment to select the most appropriate and efficient code solutions from multiple LLM-generated options

Make informed decisions on behalf of our customers, ensuring that selected code meets industry standards, best practices, and specific client needs

Develop and write coding demonstrations to illustrate "what good looks like" in AI-generated code, setting benchmarks for quality and efficiency

Provide detailed feedback and explanations for your evaluations, helping to refine and improve the LLM's understanding and output

Collaborate with the AI research team to identify areas for improvement in the LLM's coding capabilities

Stay abreast of the latest developments in software engineering, coding standards, and AI to ensure our evaluations remain cutting-edge

Required Qualifications:

Advanced degree in Computer Science, Software Engineering, or a related field

Extensive experience (5+ years) in software development across multiple programming languages and paradigms

Demonstrated ability to critically evaluate code quality, efficiency, and adherence to best practices

Strong analytical and decision-making skills, with the ability to make complex judgments under ambiguous circumstances

Excellent written and verbal communication skills, with the ability to explain technical concepts clearly

Experience in technical writing, particularly in creating coding examples or tutorials

Preferred Qualifications:

Previous experience working with or evaluating AI systems, particularly in the @context of code generation

Familiarity with a wide range of software development methodologies and architectural patterns

Understanding of machine learning concepts, particularly as they apply to natural language processing and code generation

Experience in creating or contributing to coding standards or style guides

This role requires a unique blend of technical expertise, critical thinking, and communication skills. You will be the bridge between advanced AI technology and practical, real-world coding applications. Your work will directly influence the development of next-generation AI coding assistants, shaping the future of software development. If you're passionate about code quality, have a keen eye for detail, and are excited about the potential of AI in software engineering, we encourage you to apply for this pivotal role at AGI Data Services

Belcan Corp jobs

LLM Evaluation Expert