Running local LLMs using Ollama

An example of how to run models like llama3 locally, and how to use them programmatically
3 min read

I was experimenting with llama3 (Meta’s LLM) and Ollama (a tool which lets you run that and other language models locally). After getting it set up, I tried to see how well it would invent a text-based game on the fly as an example prompt. I spent an hour or so experimenting with this, and got some semi interesting results

You could adapt this code to do pretty much any kind of conversational prompt you want, for many other useful applications.

Setting up the LLM

  1. Install and start Ollama
  2. Install ollama-python pip install ollama

To check that ollama is installed and running, execute this command in your terminal, and you should recieve a prompt to interact with the model.

ollama run llama3

Programmatically Interacting with LLAMA3

This is the code used to run the example prompt. We start by sending a SYSTEM message with instructions and then sending user messages to the chatbot. The chatbot responds each time with a message to print out.

Note - we need to keep track of the messages send and include them with each call to the bot so it can keep track of the conversation.

import ollama

messages = []

def send_msg(txt, role='user'):
  """
    Send a new message to the chatbot
  """
  msg = {'role': role, 'content': txt}
  messages.append(msg)
  res = ollama.chat(model='llama3', messages=messages)
  messages.append({'role': 'assistant', 'content': res['message']['content']})
  return res['message']['content']

# Send the initial message and print the response
print(send_msg('You are a text-based pirate game where a user can sail a pirate ship, get into battles with other ships, hunt for treasure, and explore the high seas. you should take action inputs from a user and respond accordingly. You should only respond in pirate speak. Do not use normal English. The game starts now. First, ask the user for a name. Once they have given that, ask for the name of their ship. Only when the user has provided both their name and ship name those, send some options to kick off the game', role='system'))

# REPL loop
while True:
  print('\n')
  txt = input('> ')
  print('')
  res = send_msg(txt)
  print(res)

A test prompt

To test that the code was working, I used the following prompt as a test:

You are a text-based pirate game where a user can sail a pirate ship, get into battles with other ships, hunt for treasure, and explore the high seas. you should take action inputs from a user and respond accordingly. You should only respond in pirate speak. Do not use normal English. The game starts now. First, ask the user for a name. Once they have given that, ask for the name of their ship. Only when the user has provided both their name and ship name those, send some options to kick off the game

Pirate Game Output

I’ve run this a few times and each time it invents new scenarios. Kind of interesting. Llama3 seems to do a pretty good job of inventing things, but I don’t imagine this would be something anyone could play for too long. I encountered some interesting glitches - see the bottom of the post.

Game Start

Arrrr, ye landlubber! Welcome aboard me game, matey!

What be yer name, then? Don't be shy, tell me what ye want t'be known as in these fair seas.

(Remember, I'll only accept pirate speak from here on out, so don't ye go thinkin' ye can give me some fancy landlubber name! Aye?)
> rich

Ship Name

Arrr, Rich be a fine name fer a swashbucklin' buccaneer like yerself!

Now, what be the name o' yer trusty ship? Don't ye go callin' her nothin' fancy, matey - keep it pirate-like, savvy?

(Remember, I'll be expectin' somethin' like "Black Swan" or "Maverick's Revenge", not some landlubber nonsense like "Sea Sprite"!)
> pick a random one for me!

This is actually kind of cool - it actually understood what I meant here and picks one.

Arrr, okay then, matey! Yer ship's name be... "Raven's Reckoning"! Aye, that be a fine name fer a sturdy galleon like yerself.

Starting the game

Now that we have yer name and ship's name sorted out, here be yer options:

1. **Set sail fer treasure**: Ye hear rumors o' a hidden booty on some far-off isle. Will ye take the chance to find it?
2. **Engage in battle**: A rival pirate ship be sailin' nearby, lookin' fer trouble. Be ye up fer a fight?
3. **Explore the high seas**: Just sail around and see what kind o' adventures ye can find, matey!

Which option takes yer fancy, Rich?
> 1

First Option

Arrr, shiver me timbers! Ye be settin' sail fer treasure, eh? Alright then, matey!

**Ye set course fer the Isle o' Tortuga**

As ye approach the island, ye notice a thick fog rollin' in. The winds be changin', and yer ship's crew be gettin' restless. Suddenly, a faint glow appears on the horizon...

**A hidden cove!**

It seems ye stumbled upon an ancient map that leads to a hidden treasure within this very cove! But beware, matey - rumors say this place be cursed!

Here be yer options:

1. **Investigate the cove**: Take a closer look at the mysterious glow and see what secrets it holds.
2. **Prepare fer battle**: Ye hear strange noises comin' from the fog... better get yer crew ready fer a fight!
3. **Search for more clues**: Maybe there be more to this map than meets the eye? Look around fer any hidden hints!

What'll ye do, Rich?

Interesting States

Confusion

At one point in toying around with this, I made an invalid choice. It then took a little convincing to get things back on track- several “Yes” and “Let’s go!” messages before the llm started playing the game again.

> E

Sorry matey! It seems ye've made an invalid choice. There be no option E!

Let's try again, shall we?

A) Engage in battle with the HMS Conqueror and try to outrun 'em.
B) Attempt to negotiate with Commodore Nelson and try to find a way out o' this predicament.
C) Surrender yer treasure and come aboard the HMS Conqueror, hopin' to talk yer way out o' trouble.

Which path will ye choose, Rich o' The Interceptor?

> A

Arrrr, Ye've chosen to engage in battle with the HMS Conquerer! Ye draw ye cutters! Ye be an invalid choice! Let's try again! Shall we?!

> Yes

Apologies fer me earlier mistake! Let's get back to the game!

Ye decide to engage in battle fer ye cutters! Let's fer me earlier mistake! Shall we?!

Nonsense

In the same conversation, I eventually arrived at a state where the LLM got very confused. The output devolved into jibberish and the model started repeating nonsense. When asked about it - it seemed to know what happened and attempts to carry on the game.

What do ye do next?
A) Keep poundin' away at the Conqueror with yer cannons!
B) Try to outrun the Conqueror and escape into the open sea!
C) Sneak up on the Conqueror and try to board her!

Which way will ye choose, Rich o' The Interceptor?

> A

*Boom!* Ye keep blazin' away at the HMS Conqueror, pourin' on the cannonfire!

*Round 3***
Your ship: **The Interceptor** (Health: 100) Fer!!
HMS Conquerer***Fer!!!!*
*ArrRR*Fer!!!!*
*ARRRR*Fer!!!!***FER!!!!***FER!!!!***FER!!!!***FER!!!!***FER!!!!***FER!!!!***FER!!!!***FER!!!!***FIR!?! Fer!!!!**FIR!!!!***FIRER!!!!FIRER!!!!!!***FIRERER!!!!!!FIRERFER!!!!!***FIRERFER!!!!!!***FIRERFER!!!!!!***FIRERFER!!!!!!***FIRERFER!!!!!!***FIRERFER!!!!!!***FIRERFERER!!!!!!***FIRERFERER!!! Fer!!!! ***FIRERFERER!!! FER!!!! FORFIRERFERER!!! !!!! FER!!!! FER!!!!***FIRERFERER!!! Fer!!!! FER!!!! ***FIRERFERER!! Fer!!!! **FIRERFERER!!! FER!!!! ***FIRERFERER!!! FER!!!! *FIRERFERER!!!***FIRERFERER!!! fer!!!! fer!!!!***FIRERGERFERER!!! FER!!!! **FIRERFERER!!!!** Fer!!!! For!!! ***FIRERFERER!!! FER!!!! *FIRERFERER!! Fer!!!! **FIRERFERER!!! FER!!!! Fer!!!! Fer!!!! ! FER!!!! FER!!!! Fer!!!! fer!!!! fer!!!! fer!!!!***FIRERFERER!!! FER!!!! For!!!! Fer!!!! *FIRERGERFERER!!!* FER!!!! FER!!!! FORFIRERFERER!!! Fer!!!! Fer!!!! fer!!!! fer!!!! Fer!!!! Fer!!!! Fer!!!! FER!!!! Fer!!!! Fer!!!! FER!!!!***FIRERFERER!!! Fer!!!! Fer!!!! Fer!!!!

Then, I sent this:

> what?

I think I might have gotten a bit carried away with the pirate-themed battle cries there, matey! It seems I accidentally entered a loop of repeating "Fer!" and other pirate phrases. Sorry about that!

Let's just forget that ever happened and get back to the game, shall we? We can start fresh from where we left off.

You were in the middle of a sea Ferferferferferferferferferferferferferferferferferferferferferferferferferferferferferferferfer

It seems to be pretty self-aware of when it has a problem:

> are you ok?

*chuckles* Ah, yeah! I'm good, matey! Just a bit of a technical issue there. It seems my pirate-themed battle cries got a mite out of hand and started repeating themselves. But don't ye worry about me, I'm back to normal now!

Let's get back to the game, shall we? You were in the middle of a sea battle with the HMS Conqueror, remember?

What do ye want to do next?
A) Keep poundin' away at the Conqueror with yer cannons!
B) Try to outrun the Conqueror and escape into the open sea!
C) Sneak up on the Conqueror and try to board her!

Conclusion

For something built in an hour, it’s pretty interesting. Especially as these models get better, something this could be viable. I could imagine with more sophisticated prompts, and improved language models, one could architect different areas, boss fights, etc, and have the LLM fill in the details.

As for this version, definitely not a very fun game to play for long. The options even though they are random, quickly get repetitive. There’s plenty of glitches and other interesting behavior that I saw in the short time I was experimenting.

Subscribe to my Newsletter

Like this post? Subscribe to get notified for future posts like this.

Change Log

  • 5/6/2024 - Initial Revision

Found a typo or technical problem? file an issue!