CoreHRx Jobs

Apply in one click, save jobs and take free mock tests.

Register free Log in
Hiring? Post jobs with CoreHRx
My account
user

View & update profile

AI Evaluation Engineer - Proofline

Meraki Labs Verified employer

Full time On-site
Experience
5+ years
Location
Bangalore
Work mode
On-site
Posted 3 months ago

Free · takes a minute · your resume fills your profile

CoreHRx never asks candidates for money. If anyone does, .

Role at a glance

You will
  • Proofline is live in production with institutional pilots underway.
  • Building with AI is easy to prototype, but proving reliability in production is a major challenge.
  • In AI-native codebases, verification is the key bottleneck for scaling capabilities.
Experience
5+ years
Must-have skills
CypressPlaywrightProduction EngineeringPythonTypeScript
Work mode
On-site

How well do your skills match this job?

Sign in to see your fit: the skills, experience and location this job asks for, checked against your profile.

Job description

Location: Bangalore, India
Department: Engineering

About Us

Meraki Labs (founded by Mukesh Bansal & Peeyush Ranjan) builds and rapidly scales AI-first, "moonshot" startups. We're looking for a high-velocity, production-grade engineer to build 0-to-1 products alongside founders.

About the Role

Proofline is live in production with institutional pilots underway. Building with AI is easy to prototype, but proving reliability in production is a major challenge. In AI-native codebases, verification is the key bottleneck for scaling capabilities. As an AI Evaluation Engineer, you will take ownership of the evaluation scaffolding and quality layer, including eval harnesses for AI-facing features, test infrastructure, and release gates. You’ll play a critical role in monitoring how our product behaves across AI assistants (e.g., Claude, ChatGPT), accounting for differences by host and continual changes.


AI Evaluation Engineer — Evals & Systems Verification

Location: HSR, Bengaluru (On-site)

Experience: 5+ years

Responsibilities

  • Build and maintain evaluation harnesses for AI-facing features to measure and tune system quality (e.g., capture quality, retrieval quality, guidance quality)

  • Own end-to-end and API test infrastructure (Playwright-class), supporting a continuous, daily-release cycle

  • Design and execute host-behavior probes using scripted user sessions across diverse AI assistants, ensuring product behavior aligns with expectations

  • Gate production releases through thorough user acceptance testing (UAT), regression analysis, and quality reporting

Requirements

  • Background in SDET/QA automation or ML evaluation, with proven ownership of test or evaluation infrastructure and not just executing tests

  • Strong programming skills in Python or TypeScript, with experience in API-level testing and tools like Playwright or Cypress

  • Familiarity with LLM applications or a demonstrated interest in evaluating non-deterministic systems

  • Highly autonomous and able to define your own workflows and processes for evaluation and verification


How We Work

Small team, high trust, written decisions. Designs get adversarial review before code; PRs get automated review driven to zero open findings; features aren't done until verified on a live system. AI agents do a large share of the implementation—your leverage is judgment: framing the problem, freezing the right design, and knowing when the machine is wrong.

You Should Apply If

• You enjoy turning messy problems into concrete solutions.

• You thrive in fast-moving, low-process environments.

• You're excited about production engineering (monitoring, reliability, cost, latency).

You Should Not Apply If

• You want remote/hybrid (this is onsite Bangalore only).

• You prefer narrow tickets and minimal ambiguity.

• You prefer highly structured, slow-moving product organizations.

The Recruiter for this role is Ritesh Kalvellu. Apply to the role directly, and he will get back to you if your profile is relevant to the hiring requirements.

Key skills

Required

CypressPlaywrightProduction EngineeringPythonTypeScript

Soft skills

Ownership

Job market insights

10active Cypress jobs in Bangalore
16active Playwright jobs in Bangalore
5active Production Engineering jobs in Bangalore

Most IT Jobs posts in Bangalore ask for 3–6 years of experience (42% of 651 that say).

Hiring most for IT Jobs in Bangalore

  • Brillio114 open jobs
  • Databricks60 open jobs
  • Extreme Networks29 open jobs

Jobs asking for Cypress in Bangalore also ask for

  • CI/CD80%
  • Playwright80%
  • Python60%
  • Selenium60%
  • GitHub Actions50%

From live job posts on CoreHRx, updated 1 Nov 2026. Skill figures count posts that list their skills; salary figures only posts that show a salary.

Questions about this job

How much experience does this job need?

The post asks for 5 Yrs experience.

Where is this job based?

In Bangalore (on-site).

What kind of job is this?

It is a full time job.

This page helps Indian job seekers prepare for Python interviews for backend, data engineering and data analyst roles, whether you are a fresher facing your first technical round or an experienced developer aiming for a senior position. It has 60 questions in three levels, easy, medium and hard, each with a concise sample answer and, where it helps, runnable Python 3.12 code. Topics include data structures, OOP, generators, decorators, the GIL and concurrency, type hints, testing and performance. Answer each question aloud in your own words first, then compare with the sample answer and run the code to check your understanding.

Questions you may be asked

  1. What is the difference between mutable and immutable types in Python?
  2. What is the difference between a list and a tuple?
  3. What is the difference between is and ==?
  4. What are *args and **kwargs?

Get IT jobs in Bangalore by email

Create a free profile and we email you new IT jobs in Bangalore, at most once a week.

Similar jobs

Php Developer

Speedhire Consultancy
noida
AWSCI/CDPHP
Full time On-site 14 days ago

Free · takes a minute · your resume fills your profile