Blame: examples/hf_pull/main.py - abetlen/llama-cpp-python

Python bindings for llama.cpp

0 0 1 Python

docs(examples): Add huggingface pull example 2024-02-25 21:09:41 -05:00			`import llama_cpp`
			`import llama_cpp.llama_tokenizer`


			`llama = llama_cpp.Llama.from_pretrained(`
fix: Qwen 3.5 support (#2152) * fix: handle Qwen 3.5 hybrid prefix reuse * test: fix Qwen runtime unit mocks * test: drop Qwen runtime unit tests * docs: credit Qwen fix contributors in changelog * docs/tests: update default Qwen model to 3.5 0.8B * test: rebaseline Qwen 3.5 outputs * test: stabilize low-level Qwen sampling check * test: tighten Qwen 3.5 completion prompts 2026-03-22 22:33:31 -07:00			`repo_id="lmstudio-community/Qwen3.5-0.8B-GGUF",`
			`filename="*Q8_0.gguf",`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`tokenizer=llama_cpp.llama_tokenizer.LlamaHFTokenizer.from_pretrained(`
fix: Qwen 3.5 support (#2152) * fix: handle Qwen 3.5 hybrid prefix reuse * test: fix Qwen runtime unit mocks * test: drop Qwen runtime unit tests * docs: credit Qwen fix contributors in changelog * docs/tests: update default Qwen model to 3.5 0.8B * test: rebaseline Qwen 3.5 outputs * test: stabilize low-level Qwen sampling check * test: tighten Qwen 3.5 completion prompts 2026-03-22 22:33:31 -07:00			`"Qwen/Qwen3.5-0.8B"`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`),`
			`verbose=False,`
docs(examples): Add huggingface pull example 2024-02-25 21:09:41 -05:00			`)`

			`response = llama.create_chat_completion(`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`messages=[{"role": "user", "content": "What is the capital of France?"}],`
docs(examples): Add huggingface pull example 2024-02-25 21:09:41 -05:00			`response_format={`
			`"type": "json_object",`
			`"schema": {`
			`"type": "object",`
			`"properties": {`
			`"country": {"type": "string"},`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`"capital": {"type": "string"},`
docs(examples): Add huggingface pull example 2024-02-25 21:09:41 -05:00			`},`
			`"required": ["country", "capital"],`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`},`
docs(examples): Add huggingface pull example 2024-02-25 21:09:41 -05:00			`},`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`stream=True,`
docs(examples): Add huggingface pull example 2024-02-25 21:09:41 -05:00			`)`

			`for chunk in response:`
			`delta = chunk["choices"][0]["delta"]`
			`if "content" not in delta:`
			`continue`
			`print(delta["content"], end="", flush=True)`

fix(misc): Format 2024-07-09 12:20:17 -04:00			`print()`