I like systems where the interesting part is below the API surface: schedulers, memory managers, serving control planes, data pipelines, and the benchmarks that make performance claims honest. My current focus is AI infrastructure for LLM inference.
Recently I built mini-vLLM, a small LLM serving engine simulator for learning and testing paged KV cache, continuous batching, preemption, and prefix caching. At ASML I designed the runtime agent infrastructure for the Tachyon SQA process. SDE intern at Amazon AI Studio (Sep 2026–present), building distributed training and inference infrastructure.
Agent runtimesDurable workflows, human-approval gates, async job execution, result reconciliation.
AI systemsReliable extraction, evaluation, schema fidelity, long-context pipelines, production constraints.
Experience
Sep 2026 - present
Software Development Engineer Intern, Amazon AI Studio
Distributed training and inference infrastructure for AI Studio.
May 2026 - Aug 2026
Software Engineering, Test Infrastructure, ASML
Designed runtime agent infrastructure for the Tachyon SQA process: daily regression tracking, a test-plan lifecycle with QA approval gates, asynchronous agent job execution with result / inspection writeback, and a human-governed self-evolution loop (session → temporary shared memory → human audit → persist into the harness). Built on Temporal-style durable workflows.
A compact implementation of the control-plane ideas behind high-throughput LLM inference. It keeps compute simulated on purpose so memory management and scheduling behavior are easy to inspect, test, and benchmark on a laptop.
CPU-only simulation benchmarks: 7× more active sequences; 1.53× sustained throughput; 54× lower p99 TTFT. Workloads & baselines.
Added deterministic workload benchmarks and unit tests for scheduler, cache, engine, and prefix behavior.
Used the project as a concrete way to study vLLM-style serving internals instead of only reading papers.
PythonPagedAttentionKV cacheSchedulerBenchmarking
Tachyon SQA runtime
private
ASML, Software Engineering in Test intern
Runtime agent infrastructure for the Tachyon SQA process. The control plane drafts a test plan, pauses for QA, runs jobs asynchronously, and writes results back — then evolves the harness itself under human audit, instead of silently accumulating prompt residue.
Technical notes for Tachyon SQA runtime
Designed the runtime agent infra for Tachyon SQA: a workspace harness plus an execution layer that owns test-plan state across draft, approval, job run, and result writeback.
Built a daily regression tracking harness that materializes review sections from regression inputs, reads QA gates from Confluence, and drives follow-up work through durable runtime state.
Encoded the test-plan lifecycle as a durable workflow: draft test plan → capture job execution details → wait for QA approval → agent executes jobs asynchronously → update job results and inspection results when each job finishes.
Built a human-governed self-evolution loop: while a human uses the harness, the agent extracts reusable memory from the session into a temporary shared-memory layer; humans periodically audit and edit that layer; after approval, the agent persists the changes into the harness internals.
Used Temporal-style durable-execution primitives: deterministic workflow orchestration, activities for side-effecting adapters, signals for QA approval, task-queue job claiming, retries/idempotent writes, event-history replay, and local sandbox recovery.
A Kubernetes-native control plane for isolated, stateful agent workspaces: provision from runtime templates, suspend and resume, then garbage-collect on TTL.
Technical notes for Ludus
Designed the sandbox control plane with CRDs and controllers, warm-pool provisioning, and lifecycle reconciliation.
Hardened untrusted-code execution with gVisor/Kata RuntimeClasses, namespace-scoped RBAC, default-deny NetworkPolicies, and resource quotas.
Standardized reproducible agent environments with immutable OCI images, short-lived secret projection, PVC-backed workspaces, and OpenTelemetry traces.
KubernetesCRDsgVisorOCIOpenTelemetry
Clinical note structure extraction
private
UCSF research, clinical NLP and LLM inference
A pipeline for turning long, unstructured clinical records into standardized structured outputs under privacy and schema constraints.
Technical notes for Clinical note structure extraction
Decomposed monolithic extraction into smaller schema-specific subtasks to reduce context waste and improve debuggability.
Built domain datasets and fine-tuning experiments with Qwen, QLoRA, self-consistency checks, and schema validation.
Optimized long-context inference with vLLM/PagedAttention and quantization to improve throughput and memory use.
A six-month path for learning LLM serving systems through vLLM: request lifecycle, PagedAttention, scheduler design, kernels, mini-vLLM, and frontier serving systems.
Technical notes for Infra Learning Path
Connects operating systems concepts to real LLM serving internals instead of teaching them as isolated theory.
Includes diagrams, source-reading anchors, self-checks, paper reading order, and a mini-vLLM project arc.
Adds a frontier serving chapter on prefill/decode disaggregation, KV transfer, prefix-aware routing, speculative decoding, and FP8/KV quantization.
A learning site that follows one cloud request down through CPU scheduling, virtual memory, I/O, virtualization, containers, networking, Kubernetes, serverless, and GPU inference.
Technical notes for Cloud Computing, from the machine up
Organizes 14 chapters around the path from a single machine to cloud-scale infrastructure.
Connects CS162-style operating systems concepts to virtual machines, containers, orchestration, and AI serving.
Published as a static learning artifact under the site’s Learning section.
A local-first illustrated textbook that walks from SQL and relational algebra down to buffer pools, B+ trees, query execution, locking, recovery, distributed commit, and modern data systems.
Technical notes for Data Systems Illustrated
Organizes 12 database systems chapters around diagrams, invariants, and I/O cost intuition.
Includes a B+ tree sandbox plus static visual chapters for sorting, joins, optimization, recovery, and distributed transactions.
Built as a personal learning artifact rather than a copy of course notes.
DatabasesCS186StorageTransactionsRecovery
Writing
Two Simple LoopsHuman-governed skill evolution and stage checkpoints for long SQA agent tasks.
Why Networks MatterGraph structure, degree distributions, and connected systems.
Swarm Coder SetupNotes on local agent workflows and coding infrastructure.
News
2026.09Wrote Two Simple Loops, on human-governed skill evolution and stage checkpoints in SQA agent runtime.
2026.09Started as an SDE intern at Amazon AI Studio, on distributed training and inference infrastructure.
2026.07Published The Shell Is a Protocol Now, on agent-native CLIs as execution and governance boundaries for enterprise workflows.
Earlier updates
2026.05Started Software Engineering, Test Infrastructure at ASML on Tachyon SQA: daily regression and async test-plan lifecycle with QA approval gates.
2026.05Published Infra Learning Path, a textbook-style AI infrastructure path anchored on vLLM and mini-vLLM.
2026.05Published Cloud Computing, from the machine up, a Chinese learning site for OS concepts, cloud architecture, Kubernetes, serverless, and GPU inference.
2026.05Published Data Systems Illustrated, a CS186-style visual companion for databases from SQL down to storage and recovery.
2026.05Released mini-vLLM: a reproducible, CPU-only simulator of modern LLM serving internals.
2026.05Updated benchmark suite for continuous batching, prefix caching, preemption, and serving latency tradeoffs.
2025.08Started UCSF research on LLM-based clinical note structure extraction.