Godot LLM Template 1.0 Templates 4.2 Community

Submitted by user Adriankhl; MIT; 2024-05-30

Demonstrate how to use the "Godot LLM" plugin.

# How to use
1. Get `Godot LLM` directly from the godot asset library, or download the addon from [godot-llm](https://github.com/Adriankhl/godot-llm) and place it to the `addons` folder

2. Download models
* Text generation recommendation: Meta-Llama-3-8B-Instruct-Q5_K_M.gguf from https://huggingface.co/lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF/tree/main or Phi-3-mini-4k-instruct-Q2_K.gguf from https://huggingface.co/bartowski/Phi-3-mini-4k-instruct-GGUF/tree/main for low-end devices such as Android phone
* Similarity recommendation: mxbai-embed-large-v1.Q5_K_M.gguf from https://huggingface.co/ChristianAzinn/mxbai-embed-large-v1-gguf/tree/main
* Image to text recommendation: llava-phi-3-mini-int4.gguf from https://huggingface.co/xtuner/llava-phi-3-mini-gguf/tree/main, also download the `llava-phi-3-mini-mmproj-f16.gguf` from the same page
* Vector Database setting is similar to `Similarity`

3. Open the application with Godot

You will see 4 modes: `Text Generation`, `Similarity`, `Image to Text`, and `Vector Database`

In each mode, you need to clide `Model` to point to the correct model `gguf` file. Additionally, in `Image to Text` mode, you need to click `Mmproj` to point to your `*mmproj*gguf` file.

# Text Generation

You will see there are 3 generation mode: `Simple`, `Instruct`, and `Interactive`.

## `Simple` mode
Just edit the prompt and click `Generate`. Click `Stop` to stop the generation.

You choose `None` schema or `Person` schema, if you choose `None` schema, the model will generate the information of a character with Json format.

## `Instruct` mode (currently disabled)
Click `Start` first with empty prompt, then modify the prompt and click `Continue` when it is available to talk to the AI. This is an interactive mode. Whenever you see the `Continue` button is available, either input something in the prompt and click `Continue` to send the input, or simply click `Continue` to let the AI talk itself.

## `Interactive` mode
Similar to the instruct mode, except that it relies on a good initial prompt and additoinal reverse prompt and input suffix settings to generate a smooth conversation. The preset prompt should work decently well, click the `Start` button with the preset prompt and start talkin to the AI.

# Similarity
Input prompts to compute the `Embedding`, or input two prompts to calculate their `Similarity`

# Image to Text
Select an `Image` from your device. Alternative, click `From View` to capture your game screen as the image. You can `Compute Base64` encoding for your image, or decode a base64 string back to `Compute Image`. Edit the prompt, and generate text from `Image`, `Base64`, or `View` (your game screen).

# Vector Database
Click `Metadata: id, year` to define the metadata of the tables, then click `Create tables` to create a some tables for embedding storage and retrieval. Click `Store text` to split the document into chunks and store into the database, you may also click `Split text` to see the chunks. Click `Retrieve` to retrive three most similar text chunks with the upper left prompt, which satisfy the sql-where-clause in the middle right. You may also turn on "Verbose stdout", modify the upper prompt to a valid sql statement, and click execute to run the sql statement directly, you will see the result in godot log.