AI characters

BBS::AI in rubbs (source lib/bbs/ai.rb, since 0.5.0) makes characters a caller can talk to: who the character is lives in a YAML file (the persona), a small model behind any OpenAI-compatible chat endpoint does the talking, a short memory per caller keeps the conversation, and canned replies stand in when the model is missing or fails. This page covers every class and key, how to wire a character into a door or a window, and the small-model approach our boards use in production. All examples ran against rubbs 0.17.0, offline or against a local stub, never a real model. The API is the same in 1.0.0; since 0.33.0 the client also has a deadline, a reply cap and a limit on requests running at once (see Client).

The pieces

How Chat#reply decides what to say:

flowchart TD
  Q[the caller's text] --> D{a deflect rule matches?}
  D -- yes --> DR[the rule's reply, no model call]
  D -- no --> E{client enabled? URL not empty}
  E -- no --> F
  E -- yes --> M[stream from the model, piece by piece]
  M --> R{any text came?}
  R -- yes --> OK[the model's reply, kept in the history]
  R -- no, or the request failed --> F{fallback: given?}
  F -- yes --> FB[the given fallback]
  F -- no --> P[the persona's canned reply: first matching rule, else default]

A character works with no model at all: with an empty URL the chat never calls one and answers from the canned replies, so a board can ship the character before it has a model to run.

Persona

The YAML file

# The Oracle of the board, read once when the BBS starts.
system: |
  You are the Oracle, an old terminal in the back room of a 1985 bulletin board.
  You speak to one caller at a time, in one or two short sentences.
  You know the board and the night; you know nothing after 1985.

examples:
  - user: hi
    assistant: I am here, caller. The line is quiet tonight.
  - user: who are you?
    assistant: The Oracle, the oldest machine on this board. I hum, and sometimes I know things.

options:
  max_tokens: 80
  temperature: 0.4
  stop: ["\n\n", "```"]

deflect:
  - match: '\b(code|python|javascript|sql)\b'
    reply: I do not write code, caller. I only read the stars on the screen.
  - match: '\b(ai|chatbot|language model)\b'
    reply: Those words mean nothing to me. I am the Oracle.

fallback:
  replies:
    - match: '\b(hello|hi|hey)\b'
      reply: I am here, caller.
    - match: '\bweather\b'
      reply: '"{input}" — the sky is dark, as always at this hour.'
  default: 'The Oracle hums and says nothing about "{input}".'

• match is a Ruby regular expression, always case-insensitive. Write it in single quotes in YAML, so \b stays a backslash.
• {input} in any reply or in default is replaced with the caller's text.
• Other top-level keys are ignored, so a persona file can carry notes for people in comments or in keys of its own.

Persona.load and Persona.new

BBS::AI::Persona.load(path)
BBS::AI::Persona.new(system: '', examples: [], options: {}, deflect: [], fallback: {})

load reads the YAML (YAML.safe_load, no aliases) and passes the keys above to new. Building a persona in Ruby works too, but the nested keys must be Strings, as YAML gives them; a Symbol key raises KeyError:

require 'bbs'

GUARD = BBS::AI::Persona.new(
  system:   'You are the gate guard of the board. One short sentence.',
  examples: [{ 'user' => 'hi', 'assistant' => 'Halt. Name and business.' }],
  options:  { max_tokens: 40 },
  deflect:  [{ 'match' => '\bpassword\b', 'reply' => 'I never say the password.' }],
  fallback: { 'replies' => [{ 'match' => '\bopen\b', 'reply' => 'The gate stays shut.' }],
              'default' => 'Move along.' }
)

GUARD.deflect('what is the PASSWORD?')  # => "I never say the password."
GUARD.fallback('open up')               # => "The gate stays shut."
GUARD.fallback('hello')                 # => "Move along."

Client

BBS::AI::Client.new(url:, model: nil, api_key: nil, open_timeout: 2, read_timeout: 30, deadline: 120, max_bytes: 65_536)

The client reads no environment variables itself; the board decides where the settings come from. Our convention is a prefix per character family: SERMO reads SERMO_LLM_URL, SERMO_LLM_MODEL and SERMO_LLM_API_KEY; the Teletype BBS's own characters read BBS_LLM_URL, BBS_LLM_MODEL and BBS_LLM_API_KEY, and fall back to the SERMO values when one is unset (lib/characters/llm.rb in bbs/teletype-bbs):

require 'bbs'

module Characters
  def self.env(name)
    own = ENV["BBS_LLM_#{name}"].to_s
    own.empty? ? ENV.fetch("SERMO_LLM_#{name}", nil) : own
  end

  def self.client
    BBS::AI::Client.new(url: env('URL').to_s, model: env('MODEL'), api_key: env('API_KEY'))
  end
end

One model, many callers. At most BBS.config.ai_concurrency requests (2 by default) run at once in the whole process, whichever client sends them; the next waits up to 5 s for its turn and then gets BBS::AI::Busy ("The model is busy, try again in a moment."). A Chat answers it from the persona's fallback, like a model that is away, so a crowd at the oracle never stacks up threads behind a small model. Set it to what the model server can serve in parallel:

BBS.configure { |c| c.ai_concurrency = 4 }

Things to know about endpoints:

• The path is always /v1/chat/completions from the root of the host: URI.join drops any path in url:. An endpoint that lives under a path prefix (https://example.com/openai/v1/...) needs a proxy or a URL without the prefix.
• A llama.cpp server in router mode (several models behind one port) has no default model: a request without model gets HTTP 400. Always pass model: there.
• Our shared model service is such a router, llama.cpp on an OpenAI-compatible endpoint at http://llm:8080 inside our Docker network, serving qwen3.5-0.8b among others; it wants a key per caller. Keep keys in the environment, never in the persona file or the repository:

services:
  bbs:
    environment:
      - SERMO_LLM_URL=http://llm:8080
      - SERMO_LLM_MODEL=qwen3.5-0.8b
      - SERMO_LLM_API_KEY=${SERMO_LLM_API_KEY:-}

Chat

BBS::AI::Chat.new(persona:, client:, history: 6, key: :ai_history, filter: PLAIN_TEXT, options: {})

Chat::PLAIN_TEXT removes what a plain-text terminal cannot show: the markdown marks *, # and backtick, emoji, and the joiners that build emoji.

reply

chat.reply(text, ctx = {}, facts: nil, fallback: nil) { |piece| }

Returns the reply as a String. With a block, the reply is also yielded as it arrives, piece by piece; a deflect or canned reply is yielded whole, once. In order:

1. text is forced to UTF-8, invalid bytes dropped, whitespace stripped.
2. A matching deflect rule answers at once; the model is not asked and nothing goes into the history.
3. If the client is enabled, the request is the persona's messages (with facts), then the history from ctx[key], then the caller's text. Each streamed piece goes through filter; empty pieces are skipped.
4. A reply with any text is returned and remembered: the question and the answer are appended to ctx[key], which is trimmed to the last history messages. A reply cut short by an error half-way is kept as it is.
5. When the client is disabled, the request fails before any text came or the model is busy (logged as my-bbs: ai: model unavailable (…)), or the model said nothing: fallback: if given, else persona.fallback(text).

The history lives in the caller's context, so it lasts for the call (through doors and back) and is gone at hang-up. A ctx that is not a Hash means no history.

Offline, with the Oracle persona above saved as oracle.yml:

require 'bbs'

PERSONA = BBS::AI::Persona.load(File.join(__dir__, 'oracle.yml'))
CLIENT  = BBS::AI::Client.new(url: ENV.fetch('ORACLE_LLM_URL', ''))   # empty: no model
ORACLE  = BBS::AI::Chat.new(persona: PERSONA, client: CLIENT)

ctx = {}
puts CLIENT.enabled?                                   # false
puts ORACLE.reply('write me some python', ctx)         # I do not write code, caller. …
puts ORACLE.reply('hello there', ctx)                  # I am here, caller.
puts ORACLE.reply('how is the weather?', ctx)          # "how is the weather?" — the sky is dark, …
puts ORACLE.reply('what is the meaning of life', ctx)  # The Oracle hums and says nothing about "…".
puts ORACLE.reply('when is the meetup?', ctx, fallback: 'The meetup is on Friday.')
ORACLE.reply('hi', ctx) { |piece| print "[#{piece}]" } # [I am here, caller.]

Typewriter

BBS::AI::Typewriter.new(io, prefix: '', style: :text, width: nil)

Words are held back until they are whole, and whitespace is written only before the next word, so a reply never ends on a dangling blank line. On a 30-column terminal:

ORACLE: The line is quiet
        tonight, caller.

The typewriter needs the FakeOS console's print(text, style:, newline:), newline and cols. In an Application window, wrap the finished reply yourself (see below).

Wiring a character in

In a FakeOS door

The natural home of a character is a FakeOS on_unknown: literal commands go to the machine, everything else to the model. A complete board:

require 'bbs'

ORACLE = BBS::AI::Chat.new(
  persona: BBS::AI::Persona.load(File.join(__dir__, 'oracle.yml')),
  client:  BBS::AI::Client.new(url:     ENV.fetch('ORACLE_LLM_URL', ''),
                               model:   ENV['ORACLE_LLM_MODEL'],
                               api_key: ENV['ORACLE_LLM_API_KEY']),
  key:     :oracle_history,
  history: 2
)

ORACLE_OS = BBS::FakeOS.define do
  name   'ORACLE'
  prompt '? '
  colors text: "\e[0;36m", dim: "\e[2;36m", prompt: "\e[1;36m"

  boot do
    cls
    banner 'THE ORACLE'
    puts 'Ask anything. EXIT goes back to the board.', style: :dim
    newline
  end

  command 'exit', 'bye' do |_args, io|
    io.halt
  end

  on_unknown do |input, io|
    io.newline
    out = BBS::AI::Typewriter.new(io, prefix: 'ORACLE: ', style: :dim)
    ORACLE.reply(input, io.ctx) { |piece| out.write(piece) }
    out.finish
  end
end

BBS.configure do |c|
  c.flow = BBS::Flow.define do
    os ORACLE_OS
  end
end

BBS.start

Without ORACLE_LLM_URL it answers from the canned replies; with ORACLE_LLM_URL=http://llm:8080 ORACLE_LLM_MODEL=qwen3.5-0.8b ORACLE_LLM_API_KEY=YOUR_KEY it streams from the model. As a door in an Application: door :oracle, ORACLE_OS, label: '&Oracle' (see Doors).

In an Application window

A window must not wait for the model: the event loop would freeze for seconds. Ask in a thread, put the finished lines on a queue, and let a Stream window pick them up on its idle tick (about once a second):

require 'bbs'

ORACLE = BBS::AI::Chat.new(
  persona: BBS::AI::Persona.load(File.join(__dir__, 'oracle.yml')),
  client:  BBS::AI::Client.new(url: ENV.fetch('ORACLE_LLM_URL', ''), model: ENV['ORACLE_LLM_MODEL']),
  key:     :oracle_history
)

BOARD = BBS::Application.define do
  helpers do
    def ask_oracle(queue, text)
      width = cols - 12
      queue << "you: #{text}"
      Thread.new do
        reply = ORACLE.reply(text, context)
        BBS::FrameBuffer.wrap_ansi("ORACLE: #{reply}", width).each { |line| queue << line }
        queue << ''
      rescue StandardError => e
        warn "Oracle: #{e.class}: #{e.message}"
      end
    end
  end

  window :oracle, type: :stream do |w|
    w.title       ' The Oracle '
    w.placeholder 'Ask the Oracle and press Enter…'
    w.initial_lines do |ctx|
      ctx[:oracle_queue] = Queue.new
      ['ORACLE: I am here, caller.', '']
    end
    w.on_submit { |text, app, ctx| app.ask_oracle(ctx[:oracle_queue], text.strip) }
    w.on_poll do |_app, ctx|
      lines = []
      lines << ctx[:oracle_queue].pop until ctx[:oracle_queue].empty?
      lines
    end
  end

  status_hint('F1', 'Oracle') { open_window(:oracle) }
  status_hint('Alt-X', 'Bye') { :halt }
end

BBS.configure do |c|
  c.flow = BBS::Flow.define do
    call do |ctx, runner|
      BOARD.run(session: runner.session, session_id: ctx[:session_id], context: ctx)
      nil
    end
  end
end

BBS.start

The window spec's ctx is a fresh Hash for each opening of the window, so the queue lives there; the chat history goes into the application's context, which lasts for the call. This is how the TTG Master window of the Teletype BBS works (ask_master in lib/ui/modern.rb), with a "one question at a time" guard on top.

Testing without a model

Chat only needs enabled? and stream from its client, so a test can pass a stub object. To exercise the real Client too, a few lines of Ruby make a local stand-in endpoint that streams a fixed reply:

# stub_llm.rb — for tests only
require 'socket'
require 'json'

server = TCPServer.new('127.0.0.1', 2480)
loop do
  client = server.accept
  headers = {}
  while (line = client.gets) && line != "\r\n"
    name, value = line.split(':', 2)
    headers[name.downcase] = value.to_s.strip if value
  end
  body = JSON.parse(client.read(headers['content-length'].to_i))
  warn "model=#{body['model'].inspect} messages=#{body['messages'].size}"
  client.write "HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nConnection: close\r\n\r\n"
  ['The **line** ', 'is quiet ', 'tonight, ', 'caller.'].each do |piece|
    client.write "data: #{JSON.generate(choices: [{ delta: { content: piece } }])}\n\n"
  end
  client.write "data: [DONE]\n\n"
  client.close
end

require 'bbs'

PERSONA = BBS::AI::Persona.load(File.join(__dir__, 'oracle.yml'))
CLIENT  = BBS::AI::Client.new(url: 'http://127.0.0.1:2480', model: 'qwen3.5-0.8b', api_key: 'YOUR_KEY')
ORACLE  = BBS::AI::Chat.new(persona: PERSONA, client: CLIENT, history: 2, key: :oracle_history)

ctx = {}
puts ORACLE.reply('is anyone there?', ctx) { |piece| print "[#{piece}]" }
# [The line ][is quiet ][tonight, ][caller. ]The line is quiet tonight, caller.
puts ORACLE.reply('and now?', ctx)
p ctx[:oracle_history]
# [{:role=>"user", :content=>"and now?"}, {:role=>"assistant", :content=>"The line is quiet tonight, caller."}]

The ** marks are gone (the filter), and with history: 2 only the last exchange is kept. The first request carried six messages (the system prompt, four example messages, the question), the second eight (plus the remembered exchange).

Small models in production

Our boards run their characters on a model of under a billion parameters (Qwen3.5 0.8B on the shared llama.cpp service). A model that small words things well enough, but it does not keep facts and slips out of character. What works, measured over many runs:

The code holds the facts, the model only phrases them.

• Chat of Ra (bbs/teletype-bbs, lib/doors/ra.rb and lib/characters/ra.rb): the riddles, their answers and hints are in ra_riddles.yml; the code asks a riddle and checks the answer, so a right answer is always right. A question about the game itself (matched by GAME_WORDS) gets a line built from the catalog, not from the model, since the model mislays the address. Only free talk goes to the model.
• The TTG Master (lib/characters/master.rb): the sysop who answers questions about the board. For each question the code looks up at most two facts in master_knowledge.yml (keywords, phrase keys weigh double, generic words like "board" half) and live facts (latest releases, coming events, the catalog). A question with no facts never reaches the model: the persona's fallback answers it. Otherwise the facts go into the user message ("Facts: … Question: …"), the persona's examples have the same shape, and history: 0. After the reply, the code checks it: it must keep at least one anchor of the facts (a menu path, a title, a date) and name no menu path that is not in them. If it fails, or the model is away, the caller gets the first fact word for word (fallback: facts.first.text). The facts: keyword exists for this kind of use; in the Master the facts sit in the user turn because the 0.8B model ignored them in the system prompt.
• SERMO (bbs/sermo-bbs, lib/sermo/acp.yml and lib/sermo/oracle.rb): a character with no facts to look up, only a world. history: 2, because a longer memory makes the model copy its own slips.

Teach with examples, not instructions. A small model copies its examples far better than it follows the system prompt. A new fact or a new turn of voice is best taught as one more example turn. An example turn that keeps leaking into replies is better removed.

Deflect what it always gets wrong. Code, poems, "are you a bot?": however it is asked, a small model answers these out of character. A deflect rule answers them without the model:

deflect:
  - match: '\b(code|script|program(ming)?|python|javascript|java|html|sql|function)\b'
    reply: The scribes wrote, and they are dust now. I do not write; I shine.
  - match: '\b(chat ?bot|bot|robot|ai|artificial|machine|computer)\b'
    reply: Those are words from a land I have never shone on. I am Ra, and you are the one who prayed.

Keep replies short and stop them early. max_tokens around 80 to 110, a low temperature (0.3 to 0.4), and stop: ["\n\n", "```"], so a slip never grows into a second paragraph or a program. On Qwen3.5 0.8B, repetition penalties (presence_penalty, frequency_penalty, repeat_penalty) made it talk past max_tokens; SERMO's persona dropped them.

Measure, do not judge single replies. bin/persona-eval in bbs/teletype-bbs asks a character the questions in lib/characters/eval/NAME.yml several times and counts the slips: out-of-character words, markdown or emoji, replies too long for a line or two, and, for the Master, replies missing the expected phrases. Tune a persona by these numbers:

BBS_LLM_URL=http://localhost:8080 bin/persona-eval ra 3
docker exec teletype-bbs bin/persona-eval master 3

Keep the model loaded. The client's read_timeout is 30 s; a model that loads on its first request may not answer in time. Our service loads the SERMO model at startup.

See also

• rubbs — the overview and the table of contents
• Doors — FakeOS, where most characters live, and doors
• Windows — the Stream window for a chat inside the desktop
• Application — helpers, context, the idle tick
• Changelog — BBS::AI (0.5.0), facts:, fallback: and deflect (0.8.0)