SIGN IN SIGN UP

convert : refactor rope_freqs generation (#9396)

* convert : refactor rope_freqs generation

This should also fix vocab-only conversion for Phi-3.

* convert : adapt MiniCPM3 to separate rope_freqs insertion

MiniCPM3's tokenizer is treated as a SentencePiece tokenizer to avoid
having to run its custom Python code which mixes tokenization
in the same file as tool calls.

gguf-py : add long and short RoPE factors to tensor mappings

Empty, but the key names are used to populate the mappings.
C
compilade committed
1927378bcce20ba72b6c89d5977b854a4bcaeb5d
Parent: 6f1d9d7
Committed by GitHub <noreply@github.com> on 10/1/2024, 6:31:36 AM