Field notes

What the GPUs told us. Measured, then written down.

Findings from real inference runs on real hardware. Each note names the model, the GPU and the engine, shows the numbers and ends with a fix you can try.

5 min read

Your GPU says 35% busy. Busy doing what?

nvidia-smi said our H100 was 35% busy while users queued for it. Counting every kernel and memory copy found the cause, 643 MB copied off the GPU per request.

H100 and L4 · llama.cpp and vLLMRead the note →