class BBS::Cache

A thread-safe in-memory cache of computed values, for the answers of a slow service: each value lives ttl seconds, the least recently used go first when the cache is full, and one computation runs per key at a time.

ARTICLES = BBS::Cache.new(ttl: 600, max_entries: 500, max_bytes: 8 * 1024 * 1024)
ARTICLES.fetch([lang, title]) { BBS::HTTP.get(url).body }

Callers that miss the same key at once wait for the one computation and share its value (or its error). When the computation raises and an expired value of the key is still kept, that value is served instead.

Constants

Entry: One kept value: the value, when it expires (monotonic seconds) and its size in bytes.
Flight: One running computation of a key, which the callers missing the same key wait for: its value, its error, and whether it is over.

Public Class Methods

new (ttl:, max_entries:, max_bytes: nil, size: nil)

Source

# File lib/bbs/cache.rb, line 35
def initialize(ttl:, max_entries:, max_bytes: nil, size: nil)
  @ttl         = ttl
  @max_entries = max_entries
  @max_bytes   = max_bytes
  @size        = size || ->(value) { value.respond_to?(:bytesize) ? value.bytesize : 0 }
  @entries     = {}
  @flights     = {}
  @bytes       = 0
  @mu          = Mutex.new
  @landed      = ConditionVariable.new
end

Creates an empty cache.

ttl: seconds a value stays fresh
max_entries: the most values kept
max_bytes: the most bytes kept, all values together; nil for no limit. A value bigger than this alone is not kept.
size: a callable giving the bytes of a value; by default its bytesize (0 for a value without one)

Public Instance Methods

clear ()

Source

# File lib/bbs/cache.rb, line 78
def clear
  @mu.synchronize do
    @entries.clear
    @bytes = 0
  end
  nil
end

Forgets every value. Returns nil.

delete (key)

Source

# File lib/bbs/cache.rb, line 75
def delete(key) = @mu.synchronize { drop(key) && nil }

Forgets the value of key. Returns nil.

fetch (key, &)

Source

# File lib/bbs/cache.rb, line 50
def fetch(key, &)
  flight = @mu.synchronize do
    entry = fresh(key) and return entry.value
    if (running = @flights[key])
      @landed.wait(@mu) until running.done
      return running.outcome
    end
    @flights[key] = Flight.new
  end
  fly(key, flight, &)
  flight.outcome
end

Returns the fresh value of key, or computes it with the block, keeps it and returns it. Raises what the block raises, unless an expired value of key is kept: that one is returned then.

refresh (key) { || ... }

Source

# File lib/bbs/cache.rb, line 68
def refresh(key)
  value = yield
  @mu.synchronize { store(key, value) }
  value
end

Computes the value of key anew with the block, fresh or not, keeps it and returns it. When the block raises, the value kept so far stays and the error goes on: a background refresh never empties the cache.

CATALOG.refresh(:all) { repository.load }

size ()

Source

# File lib/bbs/cache.rb, line 87
def size = @mu.synchronize { @entries.size }

Returns the number of values kept, the expired ones included.