Ebin Babu Thomas

AI Engineer — agent reliability, AI control & evaluation

Ebin Babu Thomas

I build the gates, then measure what they stop.

I work on AI control and evaluation: agents whose evidence cannot be fabricated, durable human approval over tool calls, and preregistered behavioural experiments on open models. Before that, three and a half years shipping LLM, RAG and agent backends for startup clients in four countries. I publish the failures next to the passes.

Kerala, India Remote IST (UTC+5:30)

See the work Email

Measured results

434/434

agent runs recovered after being killed mid-action

IncidentGate

$0.02–$0.19

per review that takes a person 20–40 minutes

ProofPack

65.8%

of distress language trained away — the effect it reported stayed

Digital Grimace Scale

16/16

PDFs converted to editable Word, checked by re-rendering

ExactDoc

Work

Five projects, with their numbers

Each one links to its repository and, where one exists, its write-up. Limitations are on the page, not in a footnote.

AI control & agent reliability

Agents that must cite evidence and ask a person before changing anything, and harnesses that measure what the safeguards actually stop.

IncidentGate

In development

A lab measuring how policy gates, a monitor and human approval change an incident agent

434/434 kill-point cells recovered identically, 0 duplicate mutations

  • Python 3.12
  • LangGraph
  • FastMCP
  • PostgreSQL
  • OpenTelemetry
  • +3

ProofPack

Shipped

A pre-approval review agent where every “Found” has to cite a hashed, date-stamped capture

$0.02–$0.19 per review, against 20–40 minutes by hand

  • Python
  • Gemini API
  • Claude Agent SDK
  • Playwright
  • YAML checklists
  • +3

Model behaviour research

Preregistered experiments on open models, published with their failed tests.

Digital Grimace Scale

Shipped

A preregistered test of whether language models show involuntary markers of adverse treatment

65.8% of distress language trained away — the effect it reported stayed

  • Python
  • vLLM
  • Modal
  • QLoRA-DPO
  • top-20 logprob metrics
  • +1

whose-voice

Shipped

Blind attribution of the hidden principal behind a covertly poisoned training corpus

12–44% top-1 of 47 candidates, against 2.1% chance

  • Python 3.12
  • sentence-transformers
  • PyTorch
  • bootstrap and permutation statistics
  • pytest

Open source

Tools I maintain, and fixes that landed in other people’s repositories.

ExactDoc

Shipped

PDF to editable DOCX, checked by rendering the result back and diffing word positions

16/16 corpus documents converted, verified by render-back diff

  • Python
  • PDFium / pypdfium2
  • OOXML
  • LibreOffice headless
  • PyMuPDF
  • +1

Merged upstream

All work

How I work

Three rules the work follows

  1. Claims cite a run

    Every number I publish points at a committed artifact and the command that produced it, or it does not go up.

  2. Failures get published

    The preregistered test that failed sits in the repository under its own heading, next to the one that worked.

  3. Humans keep the decision

    A model can propose and a monitor can flag, but nothing mutates without a single-use token a person minted.

Now

What I’m working on this month

Updated

  • Wiring a model into IncidentGate’s decision path, so the three-condition comparison runs on model output instead of deterministic fixtures.
  • Moving IncidentGate’s three MCP servers out of process onto a real transport, then re-running the kill matrix against that.
  • Packaging ExactDoc 1.0 for PyPI and growing the expansion corpus without invalidating a published number.
  • Working through ControlArena and the AI-control literature on control evaluations and monitor games.
  • Writing up the grimace-scale channel dissociation as a standalone piece, separate from the sprint report.
Full now page

Writing

Reports and write-ups

All writing

Experience

Track record: production backends since 2022

  1. Jul 2026 – present

    Independent AI engineer & researcher

    Remote (Kerala, India)

    Building and measuring AI control systems — evidence-gated agents, durable human approval over tool calls, and preregistered behavioural experiments on open models — with every claim tied to a committed run.

    • IncidentGate: a governed incident-agent lab. 594 kill points across 27 frozen scenarios, 434/434 measured cells recovered identically, zero duplicate mutations, zero lost incidents.
    • ProofPack: an evidence-gated pre-approval reviewer at $0.02–$0.19 per review against 20–40 minutes by hand, where a fabricated citation is structurally impossible rather than discouraged.
    • Two Apart Research sprints, solo: the Digital Grimace Scale (Digital Minds, Aug 2026) and whose-voice (Secret Loyalties, Jul 2026) — both published with their failed tests intact.
    • ExactDoc 1.0: measurement-validated PDF to DOCX, 16/16 on a frozen corpus at 0.9588 live-text retention, verified by rendering each output back and diffing word positions.

  2. Jan 2022 – Jun 2025

    Zackriya Solutions Software Engineer

    Remote, India

    Shipped 0-to-1 backends and applied-AI systems end-to-end for startup clients across the US, Canada, Europe and Australia, and scoped technical requirements for incoming projects as a core developer.

    • Natural-language real-estate search: a FastAPI service turning plain-English queries into SQL over Cloud SQL via GPT-3.5, deployed on GCP Cloud Run and load-tested with Locust.
    • DocuAI semantic document search: chunked and indexed 1000+ documents in Qdrant and returned the most relevant parent documents; designed the ingestion schema and retrieval logic.
    • Speech-assessment backend for an EdTech client: scored one-minute candidate videos with Whisper ASR and the Microsoft Pronunciation API on AWS Fargate, tuned against ground truth.
    • FinBot: Mistral-7B fine-tuned with QLoRA plus an Alpaca to Bytewax to Qdrant news-retrieval pipeline, served with vLLM on a GKE L4 GPU node.

Resume (PDF) (opens in a new tab)

Contact

Get in touch

Email
GitHub
github.com/ebt55
LinkedIn
linkedin.com/in/ebinbt
Peerlist
peerlist.io/ebinbt
Kaggle
kaggle.com/ebinbt007
Hugging Face
huggingface.co/ebt005
Call
Book a call
Resume
Resume (PDF) (opens in a new tab)

I take on contract work in agent reliability, evaluation and LLM backends — remote from Kerala, India (IST), overlapping US mornings and EU afternoons. For the right team, that can become a full-time role or a research fellowship.