You describe a computation. KILN analyses it, decides how to schedule it, and emits ARM64 instructions one 32-bit word at a time — then writes them into executable memory and calls them. No LLVM. No assembler. No numpy. 3,176 lines of the Python standard library.
Every number here was produced by running the code on this machine. Nothing is copied from a previous run.
Ask numpy for (a*b + c)*a - b and it sees four separate operators.
It walks all of memory four times and builds three throwaway arrays along the way,
because a library has no way to know what you are going to ask for next.
KILN sees one expression. It compiles the whole thing into a single loop that touches each number once, keeps every intermediate value in a register, and allocates nothing at all. On 16 million numbers that is the difference between 14.0 milliseconds and 4.3.
The pattern below is the thesis: the more operators in the expression, the
bigger the gap — because that is exactly how many trips through memory the
fusion removes. a*2.5 + b has two operators and gains least. The
nine-operator expression gains most.
A speedup on its own means nothing. So before comparing against anything, KILN measures the two hard limits of the machine it is running on — using kernels it emits for the purpose. One loop of nothing but multiply-adds. One loop of nothing but reading memory. Whatever those hit is the ceiling.
The large sizes do not manage that on their own. At 2048×2048 the multiply runs at 16.2 GFLOP/s — 16% of the ceiling — because it re-reads the same slab of memory for every row of the answer. Working on cache-sized blocks instead takes it to 77.2. That 4.8× is the single largest improvement in the project, and the instructions in the inner loop are identical before and after. Only the order changed.
Getting the ceiling itself right took two attempts, and the first one was wrong in my favour. Measuring the chip's limit with only 8 independent chains of work reports 2.0 multiply-adds per cycle; with 12 it reports 3.0. Those are measurements of how long one instruction takes, not how many run at once — and the first version was also timed before the processor had raised its clock speed. It read 85.6 where the truth is 103.3. Every benchmark now warms the chip first and takes the best of five shapes.
Different operations deserve different standards. Collapsing them into one number is how real errors get hidden.
Kernels built from add, multiply, fused multiply-add, min, max and square root must match the reference on every single number, to the last bit. Seven kernel families, 77 configurations each: 0 error.
The comparison is not ordinary Python arithmetic — that rounds twice and disagrees with the hardware for reasons that have nothing to do with the compiler. KILN computes each reference value as an exact fraction and rounds it once. The first version of these tests reported an 86-in-1000 failure rate that turned out to be entirely the reference's fault.
exp, 1/x and 1/√x are
approximations by construction, so there is no correct answer to demand. The tests
report the worst error instead: exp 1, reciprocal 1, inverse-square-root 2.
tanh reaches 32 — that is the formula's own cancellation, and it is
reported rather than tuned away.
Checking instructions one at a time cannot catch a jump that lands in the wrong place. So 159 complete kernels — 16,518 instructions — get printed as assembly, handed back to Apple's compiler, and compared. Zero mismatches.
Adding up millions of numbers loses precision: once the running total grows large, the smallest digits of each new number fall off the end. There is a fix — carry the lost part in a second register and feed it back — but it costs three extra instructions per number, roughly 3× the time.
So KILN doesn't just pick one. It switches the fix on only once the drift would actually matter. Both halves of that trade are measured:
| Numbers added | Plain error | Plain | Corrected error | Corrected | numpy error | numpy |
|---|---|---|---|---|---|---|
| 65,536 | 1.7e-07 | 6.8 µs | 9.6e-08 | 13.4 µs | 7.0e-09 | 24.7 µs |
| 1,048,576 | 4.3e-06 | 120 µs | 4.4e-08 | 204 µs | 4.5e-08 | 576 µs |
| 16,777,216 | 2.8e-04 | 2,153 µs | 6.0e-08 | 3,303 µs | 6.0e-08 | 11,170 µs |
At 16 million numbers the corrected version matches numpy's accuracy to three significant figures while running 3.4× faster. Left uncorrected it would have been four thousand times less accurate — which is exactly why the switch exists.
The forward pass, the backward pass and the optimiser all run on machine code this project generated. To check it, the identical network — same starting weights, same data, same settings — is trained again in slow, high-precision numpy, and the two loss curves are compared step by step.
Mistakes in backpropagation compound. A transpose off by one row, a gradient with the wrong sign, a bias added to the wrong axis — any of those would pull the curves apart in the first few steps and never let them rejoin.
There are forty ways to arrange each loop, and the best one shifts with the size of the data and the shape of the expression. The gap between the best and worst arrangement reaches 16× on the worst kernel, so choosing badly is expensive. Timing all forty is accurate and slow.
So KILN fits a small model on measurements from this machine — instructions per number, memory traffic, register pressure, loop length — and uses it to rank all forty candidates without running any of them. Only the top five get timed.
Scored by holding out an entire kernel: the model is trained on every other kernel, then asked to schedule one it has never encountered. Training on the thing you then measure would prove only that it memorised.
Late on, one check failed that had passed a dozen times before:
tanh was off by 254 units in the last place — not a
rounding difference, a real loss of accuracy.
Two separate faults, and the second one is worse. The first: the textbook formula for tanh subtracts two nearly identical numbers when its input is near zero, and almost every meaningful digit cancels. The second: the test was seeding its random inputs from Python's built-in hash function, which is deliberately scrambled differently every time the program starts. The test data was changing from run to run, so the bug surfaced only when the dice happened to land on it.
A test whose inputs move is not a test. The seeds are now derived from a fixed checksum, and tanh was rebuilt: an exact polynomial for small inputs, the exponential formula for large ones, and a comparison that picks between them without ever branching — because four numbers travel through the processor together and can disagree about which formula they need.