feat: Make saved state more compact on-disk (#1296)
* State load/save changes
- Only store up to `n_tokens` logits instead of full `(n_ctx, n_vocab)`
sized array.
- Difference between ~350MB and ~1500MB for example prompt with ~300
tokens (makes sense lol)
- Auto-formatting changes
* Back out formatting changes T
tc-wolf committed
4924455decd79273c8c695a8ff796306ac0df30d
Parent: 9842cbf
Committed by GitHub <noreply@github.com>
on 4/17/2024, 2:06:50 PM