Sailfish is a local inference stack that fine-tunes Gemma-4 on your own tool-call history, then serves it with speculative decoding for agent tasks. You point it at existing agent logs, a frontier model curates them into training data, you train on rented GPU, download the weights, and serve locally via OpenAI-compatible API. The measured claim: on an RTX 3060, this gives ~25% speedup (66 → 82 tok/s) over the base model by using a trained 156 MB draft head for speculative decoding. On datacenter hardware (A10G), 504.8 tok/s. The draft head accepts ~2× as many guesses as the stock drafter because it's trained on the machine's own sessions. Crucially, the source says this is "lossless"—every guess is verified by the base model, so answers are token-identical; the accuracy stayed at 6/6 correct across their test. The install is one-liner and auto-detects your GPU. One open question: the RTX 5090 row is an unvalidated bandwidth-scaled estimate, not measured.
reply