class BBS::AI::Client

A streaming client for an OpenAI-compatible /v1/chat/completions endpoint. At most BBS.config.ai_concurrency requests run at once in the process (see SLOTS).

Constants

DEADLINE: Seconds one request may take altogether, by default.
MAX_LINE: The most bytes of the stream kept while waiting for a line end.
MAX_REPLY: The most bytes of reply text taken from one request, by default.
QUEUE_WAIT: Seconds a request waits for its turn when the model is busy.
STREAM_LINE: One complete line at the start of the stream buffer, with its LF.

Attributes

url [R]

The base URL of the server, as a String; empty when no model is set up.

Public Class Methods

new (url:, model: nil, api_key: nil, open_timeout: 2, read_timeout: 30, deadline: DEADLINE, max_bytes: MAX_REPLY)

Source

# File lib/bbs/ai.rb, line 219
def initialize(url:, model: nil, api_key: nil, open_timeout: 2, read_timeout: 30,
               deadline: DEADLINE, max_bytes: MAX_REPLY)
  @url          = url.to_s
  @model        = model
  @api_key      = api_key
  @open_timeout = open_timeout
  @read_timeout = read_timeout
  @deadline     = deadline
  @max_bytes    = max_bytes
end

Creates a client for the server at url (its path is replaced by /v1/chat/completions).

model: sent as the request’s model field when given
api_key: sent as a Bearer token when given
open_timeout: seconds to wait for the connection
read_timeout: seconds to wait for each read of the reply
deadline: seconds the whole request may take
max_bytes: the most bytes of reply text taken

Public Instance Methods

enabled? ()

Source

# File lib/bbs/ai.rb, line 231
def enabled? = !@url.empty?

Returns whether a URL is set, so the client can be asked at all.

stream (messages, **options, &)

Source

# File lib/bbs/ai.rb, line 241
def stream(messages, **options, &)
  SLOTS.hold(limit: BBS.config.ai_concurrency, wait: QUEUE_WAIT) { request(messages, options, &) }
end

Posts a streaming chat request with messages (an Array of { role:, content: } Hashes) and options merged into the request body, and yields each piece of the reply’s text as it arrives. Raises RuntimeError ("HTTP 503" and the like) for a non-2xx answer, Busy when the model is taken (see SLOTS), Error when the request outlives its deadline, the reply passes max_bytes or a stream line MAX_LINE, and the Net::HTTP, socket, timeout and JSON errors as they come.