convert : refactor rope_freqs generation (#9396)
* convert : refactor rope_freqs generation This should also fix vocab-only conversion for Phi-3. * convert : adapt MiniCPM3 to separate rope_freqs insertion MiniCPM3's tokenizer is treated as a SentencePiece tokenizer to avoid having to run its custom Python code which mixes tokenization in the same file as tool calls. gguf-py : add long and short RoPE factors to tensor mappings Empty, but the key names are used to populate the mappings.
C
compilade committed
1927378bcce20ba72b6c89d5977b854a4bcaeb5d
Parent: 6f1d9d7
Committed by GitHub <noreply@github.com>
on 10/1/2024, 6:31:36 AM