Research
Building the instruments
Two tracks, one habit: build the thing that measures before trusting what it measures.
Agents
Agent Infrastructure
I build reproducible agent evaluation environments and test whether benchmark scores are supported by the underlying evidence. My work includes controlled skill evaluations, runtime evidence integrity, and scientific task validation.
-
Benchmarking how well agent skills work across diverse tasks.
- Built the agent sandbox across 18 model–harness configurations, with per-trial container isolation, raw trajectories, usage accounting, and pass/fail evidence.
- Designed paired evaluations across 87 tasks with task-macro pass rates and 95% CIs; curated Skills raised pass rates from 33.9% to 50.5% (+16.6 pp).
-
Evaluating capability and safety of LLM productivity agents in simulated workspaces.
- Simulates five productivity services — Gmail, Slack, Google Calendar, Google Docs, Google Drive — as one coherent workspace.
- 44 tasks that test capability and safety together, rather than treating safety as a separate afterthought benchmark.
-
Formal model-backed instrumentation for reward integrity in LLM-agent evaluation infrastructure.
- Hardened runtime evidence checks against tampered event annotations and unsupported verdicts; added regression tests for missing evidence and action attribution.
- Built a resumable annotation runner that validates completion from generated artifacts and records token usage and cost for each model call.
-
Turning World’s Scientific Codebase into Agent Learnable Environments.
- Packaged Scirpy (single-cell immune-receptor repertoire analysis) as the bioinformatics environment in a 64-environment registry of executable scientific-code tasks, with acceptance criteria that check scientific outputs, not just successful execution.
-
The evaluation environment and runtime powering benchmark suites — SkillsBench, ClawsBench, AgentFuzzBench.
-
FDA review agent & FDABench
A full-stack review agent for FDA submissions — RAG over regulations and guidance, a multi-step review workflow with citation provenance — and FDABench, the end-to-end evaluation that scores its outputs against reference reviews.
-
Co-organizer of the first workshop on the design, evaluation, and optimization of procedural knowledge for LLM agents.
Biomedical
Biomedical AI
Single-cell and spatial assays produce far more data than anyone can interpret by hand. I build models that turn those matrices into biology: cell-type annotation, tumour microenvironment structure, and drug–disease relationships read off a knowledge graph.
-
DG-scRNA
A graph-convolutional network with cluster visualization that predicts cell types from single-cell RNA-seq — 97.2% accuracy across more than 150,000 cells. First-author manuscript.
-
Thyroid tumour microenvironment
Integrated scRNA-seq with spatial transcriptomics across 25 thyroid cancer samples; identified 50+ ligand–receptor interactions and two previously undescribed cell subtypes.
-
A drug-repositioning web server: 15+ deep learning models over a knowledge graph of 5.9M edges and 107 relationship types, with visualizations that explain why each drug is recommended rather than just ranking them.
-
CRISPR gRNA on-target prediction
Benchmarked CNN, RNN, and Transformer architectures over 50,000+ curated guide sequences to establish which inductive bias the problem actually rewards.
-
Earlier wet-lab and computational work: proteomic profiling and genome-wide ChIP-seq mapping of O-GlcNAc chromatin-associated proteins, which uncovered an O-GlcNAc-regulated genotoxic stress response.
Elsewhere