Module Lead, Data Science (LLM-as-a-judge)
About this role
BOLD is seeking a senior professional to implement and run Testing and Quality Assurance specifically designed for AI-driven products and Data Science workflows. In this role, you will bridge the gap between traditional software testing and modern machine learning evaluation. You have a great team management and organizational skills. You’ll have a broad mastery of capabilities, techniques, technologies and tools in the Testing and QA domains. You’ll be handling QA teams of various sizes and expertise.
ABOUT THIS TEAM
The Data Science department at BOLD is responsible for discovering patterns and trends in datasets to get insights, creating predictive algorithms and data models, improving the quality of data or product offerings by utilizing machine learning techniques, distributing suggestions to other teams and top management, and using data tools such as Python and SQL.
The Data Science team actively collaborates with other vertical teams such as Engineering, Portals, BI, Product, Legal. Most of the projects are focused around problems that require a mix of natural language processing and machine learning. Some of the active projects are resume parsing, ranking, summary generation, data quality and scoring, content generation, job recommendations, and conversion analysis. Apart from the business initiatives, the team also explores state of the art methods and keeps them up to to date with technology.
WHAT YOU’LL DO
- Develop and own the QA strategy for AI-driven features, focusing on model accuracy, data drift, and performance metrics (Precision, Recall, F1-score).
- Implement state-of-the-art AI tools (e.g., GitHub Copilot, or custom LLM agents) to automate test case generation, code analysis, and synthetic data creation.
- Oversee the testing of Generative AI outputs, implementing "LLM-as-a-judge" frameworks to evaluate hallucination rates, toxicity, and prompt adherence.
- Develop and own BOLD’s quality framework from product development through to production
- Should be able to drive the initiatives (both Business and QA) in his area along with needed smooth coordination across the board to ensure a timely achievement of goal.
- Take part in Requirement Analysis, Sprint planning, Bug triage meetings.
- Liaise with product, development, and infrastructure teams to collaborate on delivery of high quality, high performance, and high availability applications
- Develop quality scripts with needed coverage to ensure defect free releases.
- Manage and mentor team members to maximize output and efficiency and promote a team culture of quality and fun
- Motivate, Encourage, Guide Teams maintaining equilibrium and satisfaction level.
- Give technical/logical solutions to problems faced by the teams and remove hurdles blocking releases.
- Evaluate and suggest new tools and technologies
WHAT YOU’LL NEED
- Solid understanding of the Machine Learning lifecycle (EDA, Training, Evaluation, Deployment).
- Experience using LLMs (GPT, Claude, Llama) and AI-coding assistants to accelerate the testing lifecycle.
- Experience in a high volume, high throughput web/ecommerce environment.
- Hands-on experience with UI/API automation (Selenium, Playwright, or Cypress) integrated with AI-driven healing capabilities.
- Experience with web service technology like XML, REST, SOAP
- Knowledge on any of the query language like SQL, JQL etc.
- Experience in testing complex web and mobile apps.
- Strong application architecture and technical knowledge
- Strong knowledge of web concepts, including web performance infrastructure.
- Able to play the role of a scrum master, with strong coordination skills
- Knowhow of basic UX standards, SEO/ SEM concepts and key tracking matrices
- Strong communication skills
EXPERIENCE-
- Module Lead, QA (AI/ML & Automation)- 5 years+
About Bold
We Transform Work Lives
As an established global organization, BOLD helps people find jobs. Our story is one of growth, success, and professional fulfillment. We create digital products that have empowered millions of people in 180 countries to build stronger resumes, cover letters, and CVs. The result of our work helps people interview confidently, finding the right job in less time. Our employees are experts, learners, contributors, and creatives.
We Celebrate And Promote Diversity And Inclusion
We value our position as an Equal Opportunity Employer. We hire based on qualifications, merit, and our business needs. We don't discriminate regarding race, color, religion, gender, pregnancy, national origin or citizenship, ancestry, age, physical or mental disability, veteran status, sexual orientation, gender identity or expression, marital status, genetic information, or any other applicable characteristic protected by law.
Under San Francisco's Fair Chance Ordinance, qualified applicants with arrest and conviction records will be considered for the position.
Frequently Asked Questions
Is the salary disclosed for the Module Lead, Data Science (LLM-as-a-judge) position at livecareer?
Where is the Module Lead, Data Science (LLM-as-a-judge) position at livecareer located?
Which team or department does the Module Lead, Data Science (LLM-as-a-judge) at livecareer belong to?
How do I apply for the Module Lead, Data Science (LLM-as-a-judge) position at livecareer?
When was the Module Lead, Data Science (LLM-as-a-judge) job at livecareer posted?
You'll be redirected to livecareer's official application page on Greenhouse.