How can I use a Llama 3 within Godot either via API or offline?

[Jojo-Apple]

Godot Version

Godot 4

Question

A few days ago I saw that a YouTube user named “Wici” uploaded a video titled “How to put ChatGPT into Godot Projects” and basically in the video he uses a page that is no longer the same as before called Ora where he created a custom GPT and apparently he used the API and with the 2D, background, LineEdit and Label nodes and a simple script he created an interface where the user wrote and the chat bot responded in the Label.

Also a long time ago Google and StanFord University created “SmallVille” which was basically a simulation genre game where the NPCs were a little more “alive” or “less robotic” thanks to the use of the original Chat-GPT.

Now that Meta created the Llama language models that are opensource and there is a lot of information out there… so much that even someone with little experience like me gets lost and confused… I would like to know if… there is a way to do what I saw in the Wici’s video but using the Llama 3 8B API or if it is possible to find a way for Llama 3 to communicate in GDscript via offline since I have seen that there are many ways to download it and run it locally although I feel that This option may not be the most viable (I think)…

Another YouTube video called “Endless Storytelling with AI NPCs & Simulated Realities” from the “Theoretically Media” channel, there is a comment that gave me the idea that I have in mind now that goes like this:

“There are already mods to Skyrim and Bannerlord that use the ChatGPT API in order to make the NPCs have never ending dialogue and react to the player in natural ways, that’s the future of games, we just need to base it on something open source like Llama 3 not ChatGPT”

So I am seriously interested in how to achieve this goal, I am new to the Godot community but I hope that the community will come together as a collective intelligence to better understand this topic and answer these types of questions for future developers who want to do the same.

[VBe]

Hi!
Essentially you can use a extension for GD script, for Python for example, and use it to contact the API of the language model of your choice.

From my experience getting the basics down about how to retrieve the outputs based on what you need for your game is something that will need a hands-on approach while it’s also doable.
I strongly suggest to integrate this yourself.

[Jojo-Apple]

Thank you very much for responding, so you say that what I have to do is simply use an extension for GDscript or Python and with that extension I can communicate with the Llama 3 API or any other LLM?.. By chance, can I know the name of that extension you’re talking about?
I appreciate your answer very much

[VBe]

No problem.
I intent to use embeddings in my game and can work on a basic implementation in the next few days. Will share the results here with you.

[Jojo-Apple]

Oh thank you very much! I’ll be waiting

[oijqws]

Hope godot would support llm by gdscript

[Jojo-Apple]

I found this thing last night from a user on Reddit with a link that took me to that Github repository… I’m not sure what to think, I need your wisdom

[Jojo-Apple]

good afternoon, I know this is my fifth answer and I don’t want to seem annoying so forgive me for that , but as a new member of the Godot community I just want to pass on a little news… I discovered that apparently there are 3 plugins in Godot to run Llama in it, but they seem to be very new because there are no video tutorials or more information than what they show…

personally, I just want to make a video game like the one Google made called Smallville, that is, a game where the player can talk to an NPC who can always respond more than usual
I tried to join the screenshots in paint