fusion_hat.llm module

Large Language Model (LLM) module

This module provides a base class for large language models (LLMs) and preset LLM classes.

Example

import a preset LLM class

>>> from fusion_hat.llm import Deepseek as LLM
>>> from fusion_hat.llm import Grok as LLM
>>> from fusion_hat.llm import Doubao as LLM
>>> from fusion_hat.llm import Qwen as LLM
>>> from fusion_hat.llm import OpenAI as LLM

initialize the LLM instance

>>> API_KEY = "your_api_key"
>>> MODEL = "your_model"
>>> llm = LLM(api_key=API_KEY, model=MODEL)

For Ollama, you don’t need api_key, but you might need to set ip.

>>> from fusion_hat.llm import Ollama as LLM
>>> llm = LLM(ip="localhost", model="deepseek-r1:1.5b")

You can also import a basic LLM class.

>>> from fusion_hat.llm import LLM as LLM

You will need to set the base url which compatible with OpenAI completion API.

>>> llm = LLM(
        base_url="https://api.deepseek.com",
        model=MODEL,
        api_key=API_KEY,
    )

Or set the whole url if it’s not ends with “/v1/chat/completions”

>>> llm = LLM(
        url="https://api.deepseek.com/v1/chat/completions",
        model=MODEL,
        api_key=API_KEY,
    )

Set instructions

>>> llm.set_instructions("You are a helpful assistant.")

Set welcome message

>>> llm.set_welcome("Hello, I am a helpful assistant. How can I help you?")

Prompt the LLM with input text

>>> input_text = "Hello"
>>> response = llm.prompt(input_text, stream=True)
>>> for next_word in response:
>>>     if next_word:
>>>         print(next_word, end="", flush=True)
>>> print("")

Prompt with image

>>> input_text = "Hello"
>>> image = "image.jpg"
>>> response = llm.prompt(input_text, image=image, stream=True)
>>> for next_word in response:
>>>     if next_word:
>>>         print(next_word, end="", flush=True)
>>> print("")

class fusion_hat.llm.LLM(api_key=None, model=None, url=None, base_url=None, max_messages=20, authorization=Authorization.BEARER, debug=False)[source]

Bases: object

LLM class

Parameters:

api_key (str, optional) – API key, default is None
model (str, optional) – Model name, default is None
url (str, optional) – URL, default is None
base_url (str, optional) – Base URL, default is None
max_messages (int, optional) – Max messages, default is DEFAULTMAX_MESSAGES
authorization (Authorization, optional) – Authorization, default is Authorization.BEARER

DEFAULTMAX_MESSAGES = 20

debug(msg, end='\n', flush=True)[source]

Debug message

Parameters:: msg (str) – Message

set_api_key(api_key)[source]

Set API key

Parameters:: api_key (str) – API key

set_base_url(base_url)[source]

Set base URL

Parameters:: base_url (str) – Base URL

set_model(model)[source]

Set model

Parameters:: model (str) – Model name

set_max_messages(max_messages)[source]

Set max messages

Parameters:: max_messages (int) – Max messages

set(name, value)[source]

Set parameter

Parameters:

name (str) – Parameter name
value (str) – Parameter value

add_message(role, content, image_path=None)[source]

Add message

Parameters:

role (str) – Role
content (str) – Content
image_path (str, optional) – Image path, default is None

get_base64_from_image(image_path)[source]

Get base64 from image

Parameters:: image_path (str) – Image path
Returns:: Base64 string
Return type:: str

get_base_64_url_from_image(image_path)[source]

Get base64 url from image

Parameters:: image_path (str) – Image path
Returns:: Base64 url
Return type:: str

set_instructions(instructions)[source]

Set instructions

Parameters:: instructions (str) – Instructions

set_welcome(welcome)[source]

Set welcome

Parameters:: welcome (str) – Welcome

chat(stream=False, **kwargs)[source]

Chat with LLM

Parameters:

stream (bool, optional) – Stream, default is False
**kwargs – Additional arguments

Returns:

Response

Return type:

requests.Response

prompt(msg, image_path=None, stream=False, **kwargs)[source]

Prompt LLM

Parameters:

msg (str or list) – Message
image_path (str, optional) – Image path, default is None
stream (bool, optional) – Stream, default is False
**kwargs – Additional arguments

Returns:

Response

Return type:

requests.Response

Raises:

ValueError – Model not set
ValueError – API key not set
ValueError – URL not set
ValueError – Prompt must be a string or a list of messages

decode_stream_response(line)[source]

Decode stream response

Parameters:: line (str) – Line
Returns:: Content
Return type:: str

_stream_response(response)[source]

Stream response

Parameters:: response (requests.Response) – Response
Yields:: str – Content

_non_stream_response(response)[source]

Non-stream response

Parameters:: response (requests.Response) – Response
Returns:: Response text
Return type:: str

print_stream(stream)[source]

Print stream

Parameters:: stream (iterable) – Stream

class fusion_hat.llm.Deepseek(*args, **kwargs)[source]

Bases: LLM

Deepseek preset LLM class

Parameters:

*args – Passed to sunfounder_voice_assistant.llm.llm.LLM
**kwargs – Passed to sunfounder_voice_assistant.llm.llm.LLM

class fusion_hat.llm.Grok(*args, **kwargs)[source]

Bases: LLM

Grok preset LLM class

Parameters:

*args – Passed to sunfounder_voice_assistant.llm.llm.LLM
**kwargs – Passed to sunfounder_voice_assistant.llm.llm.LLM

class fusion_hat.llm.Doubao(*args, **kwargs)[source]

Bases: LLM

Doubao preset LLM class

Parameters:

*args – Passed to sunfounder_voice_assistant.llm.llm.LLM
**kwargs – Passed to sunfounder_voice_assistant.llm.llm.LLM

class fusion_hat.llm.Qwen(*args, **kwargs)[source]

Bases: LLM

Qwen preset LLM class

Parameters:

*args – Passed to sunfounder_voice_assistant.llm.llm.LLM
**kwargs – Passed to sunfounder_voice_assistant.llm.llm.LLM

class fusion_hat.llm.OpenAI(*args, **kwargs)[source]

Bases: LLM

OpenAI preset LLM class

Parameters:

*args – Passed to sunfounder_voice_assistant.llm.llm.LLM
**kwargs – Passed to sunfounder_voice_assistant.llm.llm.LLM

class fusion_hat.llm.Ollama(ip: str = 'localhost', *args, api_key: str = 'ollama', **kwargs)[source]

Bases: LLM

Ollama preset LLM class

Parameters:

ip (str, optional) – IP address of Ollama server. Defaults to “localhost”.
*args – Passed to sunfounder_voice_assistant.llm.llm.LLM
api_key (str, optional) – API key of Ollama server. Defaults to “ollama”.
**kwargs – Passed to sunfounder_voice_assistant.llm.llm.LLM

add_message(role: str, content: str, image_path: str = None) → None[source]

Add message to messages list

Parameters:

role (str) – Role of message, e.g. “user”, “assistant”
content (str) – Content of message
image_path (str, optional) – Image path, default is None

Raises:

ValueError – Role must be ‘user’ or ‘assistant’

decode_stream_response(line: str) → str[source]

Decode stream response line

Parameters:: line (str) – Stream response line
Returns:: Decoded content, None if error
Return type:: str

class fusion_hat.llm.Gemini(*args, **kwargs)[source]

Bases: LLM

Gemini preset LLM class

Parameters:

*args – Passed to sunfounder_voice_assistant.llm.llm.LLM
**kwargs – Passed to sunfounder_voice_assistant.llm.llm.LLM