From training every weight to training a handful of them. Change the numbers below and watch every method recalculate at once.
Tip: click any heading or description to edit the wording before you send this on.
TinyLoRA barely shows up on this chart. That's not a rendering bug, it really is that small.
Every weight in the matrix gets updated during training. No shortcuts, no approximations, and a memory bill to match.
Freeze the original weights. Train two small matrices, A and B, and add their product back on top.
Same shape as LoRA, but A is frozen as well. Only B ever moves, so there's even less to train.
The same trainable matrices as LoRA, except the frozen base now lives in 4-bit. Same maths, a quarter of the memory.
Skip A and B altogether. Multiply a tiny trainable vector by a fixed tensor and tie the result back across the whole matrix. N can be as small as one.