fusion_hat.llm モジュール

Large Language Model (LLM) module

This module provides a base class for large language models (LLMs) and preset LLM classes.

サンプル

import a preset LLM class

>>> from fusion_hat.llm import Deepseek as LLM
>>> from fusion_hat.llm import Grok as LLM
>>> from fusion_hat.llm import Doubao as LLM
>>> from fusion_hat.llm import Qwen as LLM
>>> from fusion_hat.llm import OpenAI as LLM

initialize the LLM instance

>>> API_KEY = "your_api_key"
>>> MODEL = "your_model"
>>> llm = LLM(api_key=API_KEY, model=MODEL)

For Ollama, you don't need api_key, but you might need to set ip.

>>> from fusion_hat.llm import Ollama as LLM
>>> llm = LLM(ip="localhost", model="deepseek-r1:1.5b")

You can also import a basic LLM class.

>>> from fusion_hat.llm import LLM as LLM

You will need to set the base url which compatible with OpenAI completion API.

>>> llm = LLM(
        base_url="https://api.deepseek.com",
        model=MODEL,
        api_key=API_KEY,
    )

Or set the whole url if it's not ends with "/v1/chat/completions"

>>> llm = LLM(
        url="https://api.deepseek.com/v1/chat/completions",
        model=MODEL,
        api_key=API_KEY,
    )

Set instructions

>>> llm.set_instructions("You are a helpful assistant.")

Set welcome message

>>> llm.set_welcome("Hello, I am a helpful assistant. How can I help you?")

Prompt the LLM with input text

>>> input_text = "Hello"
>>> response = llm.prompt(input_text, stream=True)
>>> for next_word in response:
>>>     if next_word:
>>>         print(next_word, end="", flush=True)
>>> print("")

Prompt with image

>>> input_text = "Hello"
>>> image = "image.jpg"
>>> response = llm.prompt(input_text, image=image, stream=True)
>>> for next_word in response:
>>>     if next_word:
>>>         print(next_word, end="", flush=True)
>>> print("")
class fusion_hat.llm.LLM(api_key=None, model=None, url=None, base_url=None, max_messages=20, authorization=Authorization.BEARER, debug=False)[ソース]

ベースクラス: object

LLM class

パラメータ:

  • api_key (str, optional) -- API key, default is None

  • model (str, optional) -- Model name, default is None

  • url (str, optional) -- URL, default is None

  • base_url (str, optional) -- Base URL, default is None

  • max_messages (int, optional) -- Max messages, default is DEFAULTMAX_MESSAGES

  • authorization (Authorization, optional) -- Authorization, default is Authorization.BEARER

DEFAULTMAX_MESSAGES = 20
debug(msg, end='\n', flush=True)[ソース]

Debug message

パラメータ:

msg (str) -- Message

set_api_key(api_key)[ソース]

Set API key

パラメータ:

api_key (str) -- API key

set_base_url(base_url)[ソース]

Set base URL

パラメータ:

base_url (str) -- Base URL

set_model(model)[ソース]

Set model

パラメータ:

model (str) -- Model name

set_max_messages(max_messages)[ソース]

Set max messages

パラメータ:

max_messages (int) -- Max messages

set(name, value)[ソース]

Set parameter

パラメータ:

  • name (str) -- Parameter name

  • value (str) -- Parameter value

add_message(role, content, image_path=None)[ソース]

Add message

パラメータ:

  • role (str) -- Role

  • content (str) -- Content

  • image_path (str, optional) -- Image path, default is None

get_base64_from_image(image_path)[ソース]

Get base64 from image

パラメータ:

image_path (str) -- Image path

戻り値:

Base64 string

戻り値の型:

str

get_base_64_url_from_image(image_path)[ソース]

Get base64 url from image

パラメータ:

image_path (str) -- Image path

戻り値:

Base64 url

戻り値の型:

str

set_instructions(instructions)[ソース]

Set instructions

パラメータ:

instructions (str) -- Instructions

set_welcome(welcome)[ソース]

Set welcome

パラメータ:

welcome (str) -- Welcome

chat(stream=False, **kwargs)[ソース]

Chat with LLM

パラメータ:

  • stream (bool, optional) -- Stream, default is False

  • **kwargs -- Additional arguments

戻り値:

Response

戻り値の型:

requests.Response

prompt(msg, image_path=None, stream=False, **kwargs)[ソース]

Prompt LLM

パラメータ:

  • msg (str or list) -- Message

  • image_path (str, optional) -- Image path, default is None

  • stream (bool, optional) -- Stream, default is False

  • **kwargs -- Additional arguments

戻り値:

Response

戻り値の型:

requests.Response

例外:

  • ValueError -- Model not set

  • ValueError -- API key not set

  • ValueError -- URL not set

  • ValueError -- Prompt must be a string or a list of messages

decode_stream_response(line)[ソース]

Decode stream response

パラメータ:

line (str) -- Line

戻り値:

Content

戻り値の型:

str

_stream_response(response)[ソース]

Stream response

パラメータ:

response (requests.Response) -- Response

列挙:

str -- Content

_non_stream_response(response)[ソース]

Non-stream response

パラメータ:

response (requests.Response) -- Response

戻り値:

Response text

戻り値の型:

str

print_stream(stream)[ソース]

Print stream

パラメータ:

stream (iterable) -- Stream

class fusion_hat.llm.Deepseek(*args, **kwargs)[ソース]

ベースクラス: LLM

Deepseek preset LLM class

パラメータ:
class fusion_hat.llm.Grok(*args, **kwargs)[ソース]

ベースクラス: LLM

Grok preset LLM class

パラメータ:
class fusion_hat.llm.Doubao(*args, **kwargs)[ソース]

ベースクラス: LLM

Doubao preset LLM class

パラメータ:
class fusion_hat.llm.Qwen(*args, **kwargs)[ソース]

ベースクラス: LLM

Qwen preset LLM class

パラメータ:
class fusion_hat.llm.OpenAI(*args, **kwargs)[ソース]

ベースクラス: LLM

OpenAI preset LLM class

パラメータ:
class fusion_hat.llm.Ollama(ip: str = 'localhost', *args, api_key: str = 'ollama', **kwargs)[ソース]

ベースクラス: LLM

Ollama preset LLM class

パラメータ:
add_message(role: str, content: str, image_path: str = None) → None[ソース]

Add message to messages list

パラメータ:

  • role (str) -- Role of message, e.g. "user", "assistant"

  • content (str) -- Content of message

  • image_path (str, optional) -- Image path, default is None

例外:

ValueError -- Role must be 'user' or 'assistant'

decode_stream_response(line: str) → str[ソース]

Decode stream response line

パラメータ:

line (str) -- Stream response line

戻り値:

Decoded content, None if error

戻り値の型:

str

class fusion_hat.llm.Gemini(*args, **kwargs)[ソース]

ベースクラス: LLM

Gemini preset LLM class

パラメータ: