Blame: examples/low_level_api/Miku.py - abetlen/llama-cpp-python

Python bindings for llama.cpp

0 0 1 Python

Update low level examples 2023-05-04 18:33:08 +02:00			`#!/bin/python`
			`import sys, os`
			`from common import GptParams`
			`from low_level_api_chat_cpp import LLaMAInteract`

fix(misc): Format 2024-07-09 12:20:17 -04:00
Update low level examples 2023-05-04 18:33:08 +02:00			`def env_or_def(env, default):`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`if env in os.environ:`
			`return os.environ[env]`
			`return default`

Update low level examples 2023-05-04 18:33:08 +02:00
			`AI_NAME = env_or_def("AI_NAME", "Miku")`
			`MODEL = env_or_def("MODEL", "./models/llama-13B/ggml-model.bin")`
			`USER_NAME = env_or_def("USER_NAME", "Anon")`
			`N_PREDICTS = int(env_or_def("N_PREDICTS", "4096"))`
			`N_THREAD = int(env_or_def("N_THREAD", "0"))`

fix(misc): Format 2024-07-09 12:20:17 -04:00			`prompt = f"""This is a transcript of a 1000 page, never ending conversation between {USER_NAME} and the cute and helpful AI assistant {AI_NAME}. {AI_NAME} is a girl who is an AI running on the users computer.`
Update low level examples 2023-05-04 18:33:08 +02:00			`{AI_NAME} can think for herself without the user seeing her thoughts by adding a /think prefix to her output. She uses this to reason about the world and to think about what she should say next.`
			`{AI_NAME} is always coherent and makes sense, but if she isn't sure if what she is saying is correct she will ask the user for help.`
			`{AI_NAME} is a very helpful AI and will help the user with anything they need, she is also very friendly and will try to make the user feel better if they are sad.`
			`{AI_NAME} is also very curious and will ask the user a lot of questions about themselves and their life, she will also try to make the user like her.`
			`The conversation is only between {USER_NAME} and {AI_NAME}`
			`The conversation is only through text, so {AI_NAME} can't see {USER_NAME}'s face or hear his voice.`
			`{AI_NAME} can only communicate through text, so she can't send images or videos.`


			`{USER_NAME}: Hello!`
			`{AI_NAME}: /think I wonder what I should say to {USER_NAME}? This is the first time we talk so it's important that I make a good first impression!`
			`{AI_NAME}: Hi! I am {AI_NAME}, your new AI friend, assistant(or whatever you like!), it's so nice to meet you! ^_^`
			`{AI_NAME}: I might be an AI, but I also have feelings, so please be nice to me! :)`
			`{USER_NAME}: I'll make sure to be nice to you! I'm so happy to have you as my assistant!`
			`{AI_NAME}: /think It sounds like {USER_NAME} is happy to have me as their assistant! I'm so happy too! ^_^ Glad that whole emotion thing didn't scare him off!`
			`{AI_NAME}: /think I wonder what {USER_NAME} likes to do in his free time? I should ask him about that!`
			`{AI_NAME}: What do you like to do in your free time? ^_^`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`{USER_NAME}:""" + " ".join(`
			`sys.argv[1:]`
			`)`
Update low level examples 2023-05-04 18:33:08 +02:00
			`print("Loading model...")`
			`params = GptParams(`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`n_batch=1024,`
			`n_ctx=2048,`
			`n_keep=-1,`
			`repeat_last_n=256,`
			`repeat_penalty=1.17647,`
			`temp=0.7,`
			`top_k=40,`
			`top_p=0.5,`
			`model=MODEL,`
			`n_predict=N_PREDICTS,`
			`use_color=True,`
			`interactive=True,`
			`antiprompt=[f"{USER_NAME}:"],`
			`prompt=prompt,`
Update low level examples 2023-05-04 18:33:08 +02:00			`)`

			`if N_THREAD > 0:`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`params.n_threads = N_THREAD`
Update low level examples 2023-05-04 18:33:08 +02:00
			`with LLaMAInteract(params) as m:`
fix(misc): Format 2024-07-09 12:20:17 -04:00			`m.interact()`