I study computer science at Columbia and work on two sides of language models: making them cheaper to run, and figuring out what is going on inside them. When I'm not doing that I'm usually reading fiction or looking at space.

Resume GitHub LinkedIn

Open to Summer 2027 infrastructure SWE internships, May 14 to September 7

It's afternoon in New York.

Lately

Building COMET

Efficient inference for long-horizon LLM agents, with three classmates in a graduate Scaling LLM Systems seminar at Columbia.

Back from China

Spent the summer in Shanghai and Hangzhou, building messaging features at ByteDance and medical speech models at Trizen AI.

Open for Summer 2027

Looking for an infrastructure SWE internship: distributed storage and databases, backend and cloud, AI infrastructure and inference, ML systems, agents or Android. Free May 14 to September 7 and happy to relocate.

Toolbox

Tap a tool to see where I used it

Languages
AI infra and ML
Distributed and storage
Backend and data
Cloud and clusters
Agents
Android

What I work on

Pick an area

Work

Three internships, two continents. Tap a row to open it.

  • I spent the summer in Shanghai writing Kotlin in ByteDance's shared Android codebase. Both of my features were for the friend side of the product: one passed its A/B test and is now live in the app, one still in integration when I left. Both were experiment-gated, so a lot of the work was making sure every group of users saw exactly what it was supposed to.

    A messaging bar in the friend feed

    live after A/B test

    The goal was to replace the bottom action area of the friend feed's video page with a messaging bar, so you could reply, tap a suggested emoji or start a voice message without leaving the video. I owned most of the Android client.

    • Four explicit states. Server signals on each video map to one client decision: keep the default bar, show the new bar with voice, show it without voice, or leave the area empty on purpose. Separate booleans would have let "can't show messaging" quietly turn into "show nothing."
    • Module boundaries. The feed module owns the UI and lifecycle. The messaging module owns sending, quick reply, emoji suggestions and the half-screen chat. I connected them through the messaging module's exported interfaces instead of its internals, and reused the existing quick-reply flow rather than rebuilding it.
    • Checking the contract before reusing code. An older feed presenter looked reusable, but its bind method was final and cast the model to a different type, which meant a compile error or a runtime cast failure. I built on the detail-page presenter instead.
    • Lifecycle under view reuse. The page reuses a visible view before binding the next video asynchronously. I cleared the previous video's context before each bind so a stale view can't take a click, removed observers on unbind, and only logged an impression once a page was actually selected.
    • A crash in an optional cache. Emoji suggestions read a Fragment-level cache, but this page can have no Fragment. I traced the null from a Java getter through Kotlin's platform types, made the cache optional, and kept the network request going.
    • A bug only the control group could see. During rollout, control users got an empty bar on their own videos. A content rule ran before the cohort check. The fix was 36 lines: check the cohort first, plus a remote switch so the fix itself could be rolled back without a new release. Try it below.
    User group
    Video
    Server allows voice
    Client logic

      A home-screen widget install guide

      in integration

      A redesigned guide that walks people through adding a home-screen widget. One app got a new Lottie animation while the main app kept the old look, plus a new trigger when entering a chat, a fixed entry point, and frequency limits shared with the existing guide. Implemented and integrated locally on a feature branch.

      • Host isolation. The new style and experiment logic live in the app-specific module and reach shared code through a small interface. The shared module doesn't carry the animation or its dependencies, and removing the experiment later touches one place.
      • Simplifying my own design. My first version had a 220-line custom view handling loading, caching, visibility and fallbacks. I deleted it for a plain Lottie view with a local fallback image, which matched the real complexity of one half-screen animation.
      • Keeping the dependency that mattered. The new visual didn't need the old preview data, but the install tutorial after the button still needed the shared widget resource bundle, so that path stayed.
      • Isolating a silent failure. When the guide wouldn't appear even though eligibility and resources checked out, I bypassed frequency control on a test build and narrowed the block to the shared frequency-limiting chain, not my code.
      ShanghaiKotlinAndroidmodular architectureexperimentsremote rollbackLottie

    1. I worked on Trizen-ASR, a medical speech recognition model built on Alibaba's Fun-ASR that transcribes doctor-patient conversations in real time. Those transcripts become clinical advice and medical records, so a misheard drug name or dosage matters far more than a misheard filler word. That idea shaped everything I did: the data, the training and the way we scored models.

      Data

      A Python pipeline that turned 500+ hours of clinical audio into training-ready speech and text pairs with voice activity detection, forced alignment and confidence filtering. I cleaned it, mixed in noise for robustness, and kept the mix close to what the deployed model actually hears, using real collected audio for online training.

      Training

      A mix of supervised fine-tuning and continued pretraining on the encoder. On the decoder side, I put more weight on tokens that carry medical meaning but are rare in everyday speech, so the model stops rounding unusual drug and condition names toward common words.

      Evaluation

      Plain word error rate treats "uh" and "atorvastatin" the same. I designed a new benchmark around the downstream use: a weighted error rate that counts medical-term mistakes more heavily, plus medical-term recall. I also compared Fun-ASR-Nano, Paraformer and SenseVoice on accuracy and latency for real-time use.

      Which transcript is better? Depends on what you measure.

      Reference uh so the patient takes twenty milligrams of atorvastatin for hypertension

      Model A

      uh oh a patient takes twenty milligrams of atorvastatin for hypertension

      Error rate
      27.3%
      Medical recall
      4 of 4
      Model B

      uh so the patient takes twelve milligrams of rosuvastatin for hypertension

      Error rate
      18.2%
      Medical recall
      2 of 4

      • GPU tooling. Evaluation and preprocessing tooling for 800M-parameter speech models that kept domain adaptation, fine-tuning and deployment reproducible.
      HangzhouFun-ASRcontinued pretrainingSFTmedical-term weightingnoise augmentationeval designVADforced alignment

    2. Backend work on public healthcare data: taking CMS data on 15,000+ U.S. nursing homes and making it fast to search, compare and monitor.

      • APIs and pipelines. Flask REST APIs and PostgreSQL pipelines to ingest, normalize and serve the CMS data.
      • Faster search. Query tuning, indexing and pagination cut search latency by 30%.
      • Better recommendations. scikit-learn ranking models that made facility recommendations 20% more relevant.
      New YorkFlaskPostgreSQLSQL tuningscikit-learn

    Projects

    COMET

    Co-optimizing memory efficiency and test-time compute. Fall 2026, in progress.

    Agents run long, and their KV cache grows with every step. Compressing the cache frees memory and compute for more attempts at a task, but extra attempts only help if what's left in the cache is still enough to solve it.

    COMET asks how to split a fixed inference budget between keeping memory, taking more attempts, and paying to recover what was thrown away. Four of us are building it for Dr. Kaoutar El Maghraoui's graduate seminar on scaling LLM systems, and I own experiment setup and execution.

    Try evicting a sentence

    tokens kept0
    what surviveseverything
    Qwen3-8B cache at 32k tokens4.50 GiB

    StreamingLLM keeps the first few tokens as attention sinks plus the most recent ones. At 25% the sentence forgets who it was about, which is roughly the problem an agent has on step 40.

    Built with

    Language
    Python
    Serving
    SGLang (Mini-SGLang)vLLMKV-cache managementinference schedulingquantizationcheckpoint/restore
    ML
    PyTorchCUDAQwen3-8B in BF16StreamingLLM, R-KV, AgentKV
    Infra
    GCPA100 80GB GPUsSlurm
    Evaluation
    AppWorld agent benchmarklatency and throughputpeak GPU memorybootstrap confidence intervals

    The questions

    Can a recovery trigger based on things the agent can observe, its context length and tool-call count, beat never rebuilding and rebuilding on a fixed schedule? And at 80, 50 and 25% retention, how much does one rebuild per attempt buy back compared with full-cache best-of-N under the same budget?

    The setup

    Qwen3-8B in BF16 on AppWorld, whose multi-step agent tasks have outcomes that can be checked deterministically. One A100 80GB, batch size 1, sequential attempts, on an extended Mini-SGLang engine. A pilot on train and dev tasks freezes every budget, cap and seed before anything touches the test set.

    The first experiment

    Every run branches from the same saved checkpoint of the agent and its environment, taken before compression. Each branch keeps 100, 80, 50 or 25% of the cache entries full-cache inference would hold at that step, then plays out on its own, so retention is the only thing that differs.

    How a rebuild works

    A rebuild re-prefills the attempt's full raw transcript and then reapplies the eviction policy. It never undoes tool actions. Rebuilds count against the same time and VRAM budget as everything else, and a rebuild that can't fit under the memory cap isn't allowed.

    agent steps (each tool call adds tokens) KV-cache memory VRAM cap full KV out of memory: counts as a failure 50% retention rebuild: re-prefill the transcript, then evict again won't fit under the cap: not allowed
    Schematic, not measured data. Eviction slows how fast the cache grows. A rebuild briefly costs full-cache memory, so the policy only rebuilds when that spike still fits under the cap.
    Four ways to manage the cache, all running in one engine
    MethodWhat it keepsWhat it tests
    Full KVEverything, in BF16Baseline success and cost, plus best-of-N with the same budget
    StreamingLLMAttention sinks and the most recent tokensWhether recent observations are enough, and whether more attempts make up for lost history
    R-KVTokens that are important and not redundantWhether keeping useful reasoning states preserves the benefit of sampling more
    AgentKVPhase-aware, agent-specific retentionWhether early evidence survives until the later actions that need it

    What we measure

    • Oracle task success: the share of tasks solved by any of up to N attempts within budget
    • Single-attempt success and how many attempts actually ran
    • End-to-end latency, decoding throughput and estimated FLOPs
    • Peak KV-cache memory, including transient spikes during rebuilds
    • Paired comparisons across seeds, with bootstrap confidence intervals over tasks

    Schedule

    1. Oct 9Proposal submitted
    2. Oct 30Baseline experiments
    3. Nov 9Midpoint check-in with first comparisons
    4. Nov 20Recovery policy experiments
    5. Dec 4Policy tuning and final evaluation
    6. Dec 14Paper, code and a 10-minute talk

    Systems coursework

    Two courses where most of the work is making things not break.

    Distributed systems, in Go

    One key-value store, rebuilt five times. Each version survives more failure than the last, ending in the same basic architecture behind Spanner and most modern storage systems.

    Language
    Go
    Built with
    RPCgoroutinesconcurrency control
    Concepts
    Paxos consensusprimary/backup replicationsharding and shard migrationMapReducefault tolerance
    Testing
    failure injectioncrash recoverymodel checking
    1. MapReducedone

      A MapReduce library in Go. The master hands out map and reduce tasks and reassigns any whose worker dies. That's safe because workers are stateless, which is the simplest kind of fault tolerance there is.

    2. Primary/backup key-value serverdone

      The first stateful server. The primary forwards each operation to a backup before replying, and when the primary dies the backup takes over without losing acknowledged writes. Getting the handoff right is where the bugs live.

    3. Paxos key-value storedone

      Replicas agree on every operation through Paxos and apply the same log in the same order. The store stays consistent through crashes and dropped or delayed messages, and keeps serving as long as a majority is up. Retried client requests take effect once.

    4. Sharded key-value storedone

      The Paxos store split into shards across many replica groups, with a replicated configuration service that moves shards between groups while clients keep reading and writing. This is the step from fault tolerant to fault tolerant and scalable.

    5. Model checking Paxosdone

      A bare-bones model checker that explores message orderings and failures to check that Paxos never decides two different values. Testing shows the bugs you thought of; model checking finds the ones you didn't.

    Five replicas Click a server to take it down

    C++ for C Programmers (COMS W4995)

    Five labs in modern C++ built on top of systems C. Every one had to build cleanly with -Wall and run valgrind-clean: no leaks and no memory errors.

    Languages
    C++20C
    Concepts
    RAIIcopy and move semanticstemplatesC++20 conceptsexception safetyO(1) data structures
    Standard library
    containersstd::regexstd::filesystemfile streamsstd::hash
    Tooling
    g++Makevalgrind

    Parses MyMakefiles with std::regex (variables, multi-target rules, .PHONY), generates implicit compile and link rules, builds dependencies recursively with cycle detection and timestamp checks through std::filesystem, and runs commands through /bin/sh behind a custom fatal and non-fatal exception hierarchy.

    Parsed rules are cached to disk with a generic Xdb<T> serializer, and the cache invalidates itself when the MyMakefile changes or a source file appears or disappears. On a warm run the MyMakefile is never opened.

    dependency graphsincremental buildsstd::filesystemtemplatesexceptions

    An LRU cache from std::list plus unordered_map with splice, made fully generic. Then an LFU cache with one list per frequency and a tracked minimum, so every operation including eviction stays O(1), with no copies of values beyond insert and return.

    Constrained the key type with a C++20 concept to turn pages of template errors into one readable line, and specialized std::hash so a custom string class could be a key.

    O(1) designC++20 conceptsstd::hashSTL

    Wrapped a C record database in classes, then split it into reader and writer bases with multiple inheritance so read-only files work. Added an mdb-grep tool with std::regex, move constructors so one writer can fan out to several databases, and finally swapped all C stdio for C++ streams under the Rule of Zero.

    inheritancemove semanticsfstreamsRule of Zero

    Traced each constructor, copy, assignment and destructor call through a string class, with and without copy elision. Then added comparison operators, +=, and noexcept move construction and assignment, and worked out which copies became moves and why the others couldn't.

    RAIIcopy elisionnoexcept movesvalgrind

    Dijkstra's two-stack algorithm for + - * / ** with parentheses and precedence. Making ** right-associative like Python came down to one small change in when operators get popped.

    stacksparsingMakefiles

    More things I've built

    Squeeze

    2nd place

    Morgan Stanley Code to Give, March 2026

    Backend
    PythonFastAPIPostgreSQLREST APIs
    Frontend
    Next.jsTypeScript
    AI
    LangGraphAWS Bedrockstructured tool callsguardrailshuman approval

    A full-stack volunteer platform built for a nonprofit during a hackathon. I owned the FastAPI and PostgreSQL backend and the agent integration: LangGraph agents on AWS Bedrock that draft campaign content and outreach through structured tool calls, with guardrails and a human approval step before anything goes out.

    genesis

    in progress

    With a collaborator

    Focus
    agent infrastructuretool compositionverification and repair

    Infrastructure for capability improvement that compounds across organizations. Model weights stay fixed. What improves is the library of executable tools that organizations build, select, compose and repair, so experience from earlier tool-building makes later creation, verification and repair better.

    We keep three claims separate: tools pile up, tools get reused, and the tool-making process itself improves. Each needs its own evidence.

    AuscultAI

    hackathon

    Low-cost cardiac screening

    Hardware
    Raspberry PiESP32
    Signal processing
    DSPmel-spectrograms

    A Raspberry Pi and an ESP32 capture heart sounds, and a DSP pipeline turns them into mel-spectrograms for analysis. Hardware, signal processing and ML in a single build.

    UrAgent

    course project

    With teammates

    Focus
    multi-turn reinforcement learningLLM web agentspaper replication

    A replication of WebAgent-R1, which trains LLM web agents end to end with multi-turn reinforcement learning.

    More on GitHub.

    Research

    Interpretability, mostly

    • Models sometimes get drug-toxicity questions right for the wrong reasons. I wanted to know whether a model actually represents the chemistry, or is matching surface cues that happen to line up with the answer.

      What I did

      Extracted residual-stream activations on drug-toxicity benchmarks and trained linear probes on them to see how scientific concepts are laid out inside the model. Then I tried to break the probes, varying prompts and contexts and building adversarial examples meant to pull the real concept apart from the cues.

      Why it matters

      A probe with high accuracy isn't evidence of understanding if it's tracking a spurious feature. Measuring probe sensitivity and concept separability under pressure is how you tell the two apart.

      RemotePyTorchInspect AIresidual streamlinear probes

    • Does retrieval-augmented generation switch on reusable reasoning routines inside a model? I tested that with sparse autoencoder features on Gemma-3-27B at Columbia's Complex Resilient Intelligent Systems Lab.

      Setup

      Gemma-3-27B with pretrained SAEs from SAE-Lens on a Slurm GPU cluster. I built the PyTorch and Hugging Face Transformers pipelines for inference and evaluation, and used torch.profiler to keep the GPU jobs efficient.

      What happened

      One candidate feature passed ablation and differential-activation tests: removing it changed behavior, and it activated differently with retrieval than without. It then failed a pre-registered steering-specificity test. Turning it up didn't specifically produce the behavior.

      What I took from it

      Necessary isn't the same as sufficient. Following Makelov et al., a feature can matter when it's removed without being what drives the behavior on its own. That gap is exactly why the steering test was pre-registered.

      Earlier in the semester I used PCA, normalization and probing to compare how the model represents the same math problems posed in plain language and in symbols, with controlled accuracy benchmarks across both formats.

      New YorkGemma-3-27BSAE-Lenscausal interventionstorch.profilerSlurm

    Coursework

    Columbia CS, 3.96 GPA, after transferring from WashU

    Systems and infrastructure

    • Introduction to DatabasesCOMS W4111, Fall 2025
    • Fundamentals of Computer SystemsCSEE W3827, Fall 2025A+
    • Advanced ProgrammingCOMS W3157, Fall 2025
    • Data Structures and AlgorithmsWashington University in St. LouisA+
    • Computer Science TheoryCOMS W3261, Fall 2026in progress

    Machine learning and AI

    • Reinforcement LearningEECS E6892, graduate, Spring 2026A+
    • Natural Language ProcessingCOMS W4705, Spring 2026A+
    • Machine LearningCOMS W4771, Fall 2025
    • Theory of ML and Interactive Decision MakingCOMS W4995, Spring 2026
    • Introduction to AIWashington University in St. LouisA+

    Beyond CS

    • Galaxies and CosmologyASTR UN1420, Spring 2026
    • Literature Humanities I and IIHUMA CC1001–1002, 2025–26
    • Contemporary Western CivilizationCOCI CC1101–1102, 2026–27in progress
    • Music HumanitiesHUMA UN1123, Fall 2026in progress

    Off the clock

    The stuff that doesn't fit on a resume.

    Four constellations are hidden in this sky. Move your cursor around to find them.

    Pull a book off the shelf.

    Baseball

    I played as a student athlete with MLB China.

    Snooker

    My favorite game to play and to watch.

    Gym

    In the gym most weeks.

    Food

    Big foodie, and New York is a good city for it.

    Travel

    I travel whenever I get the chance.

    Space

    Subteam lead at Columbia Space Initiative on an AR project for NASA's Lunar Terrain Vehicle, one of 10 university teams invited to present at Johnson Space Center.

    Also: competitive programming to keep my algorithms sharp, and vice president of Hack WashU.

    Say hi.

    Recruiting for Summer 2027, a research idea, or a book I should read. Email is the fastest way to reach me.

    hhernye@gmail.com
    Resume (PDF) GitHub LinkedIn

    The 30-second version

    For anyone skimming between interviews.

    Who
    Columbia CS, class of 2028, 3.96 GPA
    Looking for
    A Summer 2027 infrastructure SWE internship: distributed storage and databases, backend and cloud, AI infrastructure and inference optimization, ML systems, agent systems or Android.
    Available
    May 14 to September 7, 2027. Happy to relocate.
    Experience
    ByteDance (Android messaging, Douyin Social & Messaging), Trizen AI (training a medical speech recognition model), Helios Healthcare (backend), EleutherAI SoAR fellow, Columbia CRIS Lab
    Building now
    COMET: KV-cache eviction and recovery for long-horizon agents on Mini-SGLang
    Systems
    Paxos and sharded key-value stores in Go. O(1) template caches and a make clone in C++20.
    Outside work
    Baseball (student athlete with MLB China), snooker, the gym, food and travel
    Languages
    Python, Go, C/C++, Kotlin, Java, TypeScript, SQL
    Stack
    PyTorch, CUDA, SGLang, vLLM, FastAPI, Flask, PostgreSQL, Android, GCP, AWS, Docker
    What I work onCOMETPaxosContact
    hhernye@gmail.com
    Resume (PDF)GitHubLinkedIn

    ↑↓ moveEnter goEsc close