ClusterManager
ClusterManager (from @lunibee/sharding) runs a bot as several processes. It:
- splits the shards across clusters and forks one worker process per cluster;
- hands out IDENTIFY slots for the whole deployment, so Discordâs limit holds across processes;
- routes
ShardBusmessages to the cluster that owns the target shard; - answers requests, broadcasts and health queries for the whole deployment;
- restarts crashed clusters with backoff, and can move a failed clusterâs shards to a healthy one;
- with
remote, places clusters on other machines (see ClusterHost).
ClusterManager orchestrates processes; ShardManager runs shards. The manager owns the shard-to-cluster map and keeps the workers alive; a ShardManager inside each worker opens the Gateways for that clusterâs shards. A shard is never owned by two clusters at once. The sharding guide explains how the two fit together.
Works under Bun and Node.js (child_process.fork() with IPC). The remote option needs Bun on the coordinator.
import { ClusterManager, ClusterWorker, readClusterContext } from "lunibee";// or the package on its own:import { ClusterManager, ClusterWorker, readClusterContext } from "@lunibee/sharding";import { ClusterManager, ClusterWorker, readClusterContext } from "lunibee";// or the package on its own:import { ClusterManager, ClusterWorker, readClusterContext } from "@lunibee/sharding";Quick start
Section titled âQuick startâTwo scripts: the parent, which runs the ClusterManager, and the worker, which the parent forks once per cluster.
// index.ts (parent)import { ClusterManager } from "lunibee";
const clusters = new ClusterManager({ token: process.env.DISCORD_TOKEN!, script: "./worker.ts", shardCount: "auto", shardsPerCluster: 4, waitForReady: true, startTimeout: 120_000, supervisor: { restartDelay: 1000, backoff: 2, jitter: 0.2, maxRestarts: 5 }, reassignOnGiveUp: true,});
clusters.on("clusterReady", (cluster) => console.log(`cluster ${cluster.id} ready`));clusters.on("clusterRestart", (cluster, { delay, attempt }) => console.warn(`cluster ${cluster.id} restarts in ${delay}ms (#${attempt})`),);clusters.on("error", (error) => console.error(error));
await clusters.spawn();
process.on("SIGTERM", () => void clusters.shutdownAll());// worker.ts (one process per cluster)import { ShardManager, ShardBus, GatewayIntentBits } from "lunibee";
// Inside a cluster, ShardManager reads the worker contract from the// environment: it runs only this cluster's shards, asks the parent for every// IDENTIFY, and reports ready and health to the parent.const shards = new ShardManager({ token: process.env.DISCORD_TOKEN!, intents: [GatewayIntentBits.Guilds],});
for (const id of shards.shardIds) new ShardBus(id, "my-bot").respond("guildCount", () => 42);
await shards.connect();// index.js (parent)import { ClusterManager } from "lunibee";
const clusters = new ClusterManager({ token: process.env.DISCORD_TOKEN, script: "./worker.js", shardCount: "auto", shardsPerCluster: 4, waitForReady: true, startTimeout: 120_000, supervisor: { restartDelay: 1000, backoff: 2, jitter: 0.2, maxRestarts: 5 }, reassignOnGiveUp: true,});
clusters.on("clusterReady", (cluster) => console.log(`cluster ${cluster.id} ready`));clusters.on("error", (error) => console.error(error));
await clusters.spawn();
process.on("SIGTERM", () => void clusters.shutdownAll());// worker.js (one process per cluster)import { ShardManager, ShardBus, GatewayIntentBits } from "lunibee";
const shards = new ShardManager({ token: process.env.DISCORD_TOKEN, intents: [GatewayIntentBits.Guilds],});
for (const id of shards.shardIds) new ShardBus(id, "my-bot").respond("guildCount", () => 42);
await shards.connect();The worker inherits the parentâs environment, so it reads the same DISCORD_TOKEN.
Constructor
Section titled âConstructorânew ClusterManager(options: ClusterManagerOptions)The constructor validates the options and throws; it does not fork or call Discord. spawn() does that.
Shards and clusters
Section titled âShards and clustersâ| Option | Type | Default | Description |
|---|---|---|---|
token |
string |
â | Bot token. Required; an empty one throws a TypeError. |
script |
string |
â | Worker script, forked once per cluster on this machine. Required (also with remote, where it is used only for clusters on this machine); an empty one throws a TypeError. |
shardCount |
number | "auto" |
unset | Total shards. "auto" asks /gateway/bot. When unset, spawn() also asks Discord, but autoScaleInterval only acts with an explicit "auto". |
guildsPerShard |
number |
unset | A sizing hint for "auto": at least ceil(guilds / guildsPerShard) shards, and never fewer than Discord recommends. Integer from 1 to 2500 (Discordâs limit); Discord recommends about 1000. It only picks the shard count: Discord puts each guild on shard (guild_id >> 22) % shardCount, so a shard gets about that many guilds, not exactly. Throws with a numeric shardCount. |
guildCount |
number |
asked from Discord | The guild count for guildsPerShard, instead of reading approximate_guild_count from /applications/@me. |
shardIds |
number[] |
every shard | The shards this deployment runs, out of shardCount. Must be unique integers in [0, shardCount); checked in the constructor when shardCount is a number, otherwise in spawn(). |
clusterCount |
number |
logical CPUs (os.cpus().length) |
Clusters to split the shards across, in contiguous blocks: 10 shards over 3 clusters are [0-3] [4-6] [7-9]. Positive integer. More clusters than shards gives one shard per cluster (the plan event reports the count asked for). On a one-CPU machine the default is one cluster: set clusterCount or shardsPerCluster to be sure. |
shardsPerCluster |
number |
unset | Shards per cluster; the cluster count is ceil(shards / shardsPerCluster) and the last cluster may have fewer: 10 shards at 4 per cluster are [0-3] [4-7] [8-9]. Positive integer. Setting it together with clusterCount throws a RangeError. |
IDENTIFY
Section titled âIDENTIFYâ| Option | Type | Default | Description |
|---|---|---|---|
maxConcurrency |
number |
Discordâs value when fetched, else 1 |
Discordâs max_concurrency: how many IDENTIFY rate-limit keys there are. Positive integer. When /gateway/bot was read, this can only lower Discordâs value. |
fetchLimits |
boolean |
false |
Read max_concurrency and the session-start budget from /gateway/bot even with a numeric shardCount. A failed request is reported as an error event and the start continues without them. |
startup (0.4.0) |
"safe" | "sequential" |
"safe" |
"safe": shards start as fast as the IDENTIFY rules allow, different rate-limit keys in parallel. "sequential": one shard at a time across all clusters; each holds its slot until it is READY (or identify.holdTimeout, which workers also use as their READY wait), then the next waits identify.interval. Slow, for debugging and small bots. Needs identify.coordinate. |
identify.coordinate |
boolean |
true |
Hand out IDENTIFY slots from the parent for every cluster. false lets each cluster pace its own IDENTIFYs (LUNIBEE_IDENTIFY=local). |
identify.interval |
number |
5000 |
Milliseconds between two IDENTIFYs on one key. |
identify.holdTimeout |
number |
60000 |
A granted slot that a worker never gives back is released after this many ms. |
spawnInterval |
number |
500 |
Milliseconds between forking two clusters, to spread the CPU cost of starting. IDENTIFY pacing does not depend on it. Cannot be negative. |
Readiness, requests and limits
Section titled âReadiness, requests and limitsâ| Option | Type | Default | Description |
|---|---|---|---|
waitForReady |
boolean |
false |
Make spawn() resolve only once every cluster reported ready. |
startTimeout |
number |
off | A cluster that has not reported ready this many ms after its fork is killed (SIGKILL) and restarted like a crash. Off by default, because a worker that never reports ready would be killed. Cannot be negative. Also the timeout of the take-over request in reassignment (120000 when unset). |
requestTimeout |
number |
5000 |
Default timeout of request(), broadcast() and requestCluster(), in ms. Cannot be negative. |
maxPendingRequests |
number |
10000 |
Most requests that may wait for answers at once. Positive integer. Beyond it, a new request throws. |
maxMessageBytes |
number |
8388608 (8 MiB) |
Largest payload the manager handles, in UTF-8 bytes. See Payload limits. Positive integer. |
Supervision
Section titled âSupervisionâ| Option | Type | Default | Description |
|---|---|---|---|
restartOnExit |
boolean |
true |
Re-fork a cluster that exits unexpectedly, with the same shards. false leaves it failed. |
restartDelay |
number |
5000 |
Delay before the first restart, in ms. supervisor.restartDelay wins when both are set. |
supervisor |
SupervisorOptions |
fixed 5 s delay, no limit | Backoff, jitter and a give-up limit. See Restarts. |
reassignOnGiveUp |
boolean |
false |
When a cluster gives up, move its shards to the ready cluster that owns the fewest. See Reassignment. |
shutdownTimeout |
number |
5000 |
Grace period after SIGTERM before SIGKILL, in ms, for shutdownAll() and respawn(). |
onClusterExit |
(cluster, code, signal) => void |
none | Called whenever a cluster process exits, before any restart. The clusterExit event carries the same and more. |
onGiveUp |
(cluster) => void |
none | Called when a cluster will not be restarted. Same as the clusterGiveUp event. |
A callback or listener that throws is ignored; it cannot break supervision.
Scaling, launching and hosts
Section titled âScaling, launching and hostsâ| Option | Type | Default | Description |
|---|---|---|---|
autoScaleInterval |
number |
off | With shardCount: "auto", check the recommended count every this many ms and respawn() when it changed. Integer of at least 1000. |
onAutoScaleError |
(error: Error) => void |
none | Called when a background auto-scale check fails. The next tick retries. |
fork |
(script, env) => ChildProcess |
child_process.fork(script, [], { env }) |
Starts a local worker. env is the parentâs environment plus the worker contract. Replace it for tests or a custom launcher; the returned process needs IPC (send(), message, exit). threadFork() runs clusters as worker threads. |
leader (0.3.2) |
{ lock: LeaderLock; ttl?: number } |
off | Several coordinators, one active: spawn() waits until this one holds the lock. ttl defaults to 10000 (at least 100). See Coordinator failover. |
remote |
RemoteOptions |
off | Accept ClusterHost agents on other machines and place clusters on them. |
Validation
Section titled âValidationâ| Condition | Error |
|---|---|
token or script missing or blank |
TypeError |
clusterCount and shardsPerCluster both set |
RangeError |
clusterCount, shardsPerCluster or maxConcurrency not a positive integer |
RangeError |
maxMessageBytes or maxPendingRequests not a positive integer |
RangeError |
spawnInterval, startTimeout or requestTimeout negative |
RangeError |
autoScaleInterval not an integer of at least 1000 |
RangeError |
guildsPerShard outside 1â2500, or set with a numeric shardCount |
RangeError |
shardIds duplicate or outside [0, shardCount) |
RangeError |
supervisor values out of range |
RangeError (see Restarts) |
invalid remote options |
RangeError / TypeError (see ClusterHost) |
Properties
Section titled âPropertiesâ| Property | Type | Description |
|---|---|---|
clusters |
Map<number, ClusterInfo> |
Live clusters (starting or ready) by ID. A crashed cluster leaves the map until it is restarted. |
shardCount |
number |
Total shards of the deployment. 0 before the first spawn(). |
remotePort |
number | undefined |
The port hosts connect to, once the listener is open. Useful with remote.port: 0. |
isLeader (0.3.2) |
boolean |
Whether this coordinator holds the leader lock. Always true without leader. |
Methods
Section titled âMethodsâspawn()
Section titled âspawn()âspawn(): Promise<void>Resolves the shard count, allocates shards, waits for hosts (with remote) and forks every cluster, spawnInterval apart. With waitForReady it then waits for readiness.
- Calling it again while clusters run, or while the first call is still starting, returns the same promise.
- Called during a shutdown, it waits for the shutdown and then starts.
- A failed start (Discord unreachable, hosts that did not join) rejects and can be retried.
- Inside a cluster worker it rejects: a worker must never fork the fleet again.
ready(timeoutMs?)
Section titled âready(timeoutMs?)âready(timeoutMs?: number): Promise<void>Resolves when every cluster is ready. Rejects after timeoutMs if given, or when the manager starts stopping. There is no timeout by default.
shutdownAll(timeoutMs?)
Section titled âshutdownAll(timeoutMs?)âshutdownAll(timeoutMs = shutdownTimeout): Promise<void>Graceful stop. Cancels restart timers, start timers, pending requests, IDENTIFY waits and ready() waiters; sends SIGTERM to every cluster, and SIGKILL to those still running after timeoutMs; closes the host listener. Safe to call twice or concurrently. Exits during a shutdown are never restarted. Every cluster ends stopped.
killAll()
Section titled âkillAll()âkillAll(): voidImmediate stop. Cancels the same work as shutdownAll(), calls kill() (SIGTERM) on every cluster process without waiting, closes the host listener (hosts then kill their own workers), and marks every cluster stopped. Prefer shutdownAll().
respawn(shardCount)
Section titled ârespawn(shardCount)ârespawn(newShardCount: number): Promise<void>Shuts the clusters down gracefully, forgets their crash history and spawns again with the new total. Hosts stay connected.
checkAutoScale() and fetchRecommendedShardCount()
Section titled âcheckAutoScale() and fetchRecommendedShardCount()âcheckAutoScale(): Promise<void>fetchRecommendedShardCount(): Promise<number>checkAutoScale() runs one auto-scale check: with shardCount: "auto" it compares Discordâs count (or the guildsPerShard count) with shardCount and calls respawn() when they differ. It does nothing otherwise, and passes failures to onAutoScaleError instead of throwing. fetchRecommendedShardCount() returns that count.
clusterForShard(shardId) and assignments()
Section titled âclusterForShard(shardId) and assignments()âclusterForShard(shardId: number): ClusterInfo | undefinedassignments(): ReadonlyMap<number, number>The cluster that owns a shard (whatever its state), or undefined when the shard is not part of the deployment; and a copy of the whole shard â cluster map. Ownership changes only through reassignment.
request(shardId, type, data, options?)
Section titled ârequest(shardId, type, data, options?)ârequest<R = unknown, T = unknown>(shardId: number, type: string, data: T, options?: RequestOptions): Promise<R>Sends a request to one shard, routed to the cluster that owns it, and resolves with the result of that shardâs ShardBus.respond() handler.
| Rejects with | When |
|---|---|
RangeError |
data is larger than maxMessageBytes, or the shard is not part of the deployment. |
Error("Shard 3 is down: cluster 1 is restarting.") |
The owning cluster is not running. Fails at once. |
Error("Cluster 1 exited.") |
The owning cluster exited while the request waited. |
Error(<handler message>) |
The handler threw. |
Error('Shard 3 did not reply to "stats" in time.') |
No answer within the timeout, or signal aborted. |
broadcast(type, data, options?)
Section titled âbroadcast(type, data, options?)âbroadcast<R = unknown, T = unknown>(type: string, data: T, options?: RequestOptions): Promise<BroadcastResult<R>>Asks every shard of every live cluster. It never rejects for a slow, failed or down shard (only a RangeError for oversized data, or when maxPendingRequests is reached). It settles as soon as every expected shard answered, or at the timeout.
const { replies, missing } = await clusters.broadcast<number>("guildCount", null, { timeoutMs: 5000 });const total = replies.reduce((sum, r) => sum + (r.result ?? 0), 0);requestCluster(clusterId, type, data, options?)
Section titled ârequestCluster(clusterId, type, data, options?)ârequestCluster<R = unknown, T = unknown>(clusterId: number, type: string, data: T, options?: RequestOptions): Promise<R>Sends a request to a worker process (not a shard), answered by a ClusterWorker.onRequest() handler. Rejects with a RangeError for an unknown cluster or oversized data; with Error("Cluster 1 is restarting.") when it is not running; with the handlerâs error message; or with Error('Cluster 1 did not reply to "flush" in time.').
await clusters.requestCluster(1, "flushCaches", null);health() and refreshHealth(options?)
Section titled âhealth() and refreshHealth(options?)âhealth(): ClusterManagerHealthrefreshHealth(options?: { metrics?: boolean; timeoutMs?: number }): Promise<ClusterManagerHealth>health() returns what the workers last reported; nothing is sent. refreshHealth() asks every live cluster for a fresh report first (timeout 2000 ms per cluster; process metrics only with metrics: true). A cluster that does not answer keeps its last report and is not treated as failed. See Health.
hosts()
Section titled âhosts()âhosts(): HostStatus[]The hosts of the deployment, declared or connected, with the clusters on each. Empty without remote. See ClusterHost.
on(event, listener) and off(event, listener)
Section titled âon(event, listener) and off(event, listener)âAdd or remove a listener for one of the events. Both return the manager.
RequestOptions
Section titled âRequestOptionsâ| Field | Type | Description |
|---|---|---|
timeoutMs |
number |
Milliseconds to wait. Defaults to requestTimeout (5000). |
signal |
AbortSignal |
Stops waiting. An aborted request settles like a timeout. |
BroadcastResult<R>
Section titled âBroadcastResult<R>â| Field | Type | Description |
|---|---|---|
replies |
ShardReply<R>[] |
One entry per shard that answered or failed: { shardId, result } or { shardId, error }. A shard whose cluster exited (or could not be sent to) during the broadcast is here with an error. |
missing |
number[] |
Shards that did not answer, sorted: their cluster was not running when the broadcast started, or they did not answer before the timeout or abort. |
ClusterInfo
Section titled âClusterInfoâThe object is live: the manager updates it as the cluster changes.
| Field | Type | Description |
|---|---|---|
id |
number |
Cluster ID. |
process |
ChildProcess |
The current process (the last one while a restart is pending). For a cluster on another host it is a proxy whose pid is the pid on that host. |
host |
string |
"local" or the ID of the host it runs on. |
shards |
number[] |
Shards it owns. |
state |
ClusterState |
See Cluster states. |
restarts |
number |
Restarts after a crash since spawn(). |
startedAt |
number |
When the current process was forked (epoch ms). |
readyAt |
number? |
When the current process reported ready. |
lastReadyAt |
number? |
When any process of this cluster last reported ready. |
report |
WorkerReport? |
The workerâs last health report. |
Cluster states
Section titled âCluster statesâClusterState is one of:
| State | Meaning |
|---|---|
starting |
Forked; has not reported ready. |
ready |
The worker reported its shards ready. |
restarting |
Exited unexpectedly; a restart is scheduled. |
stopping |
A deliberate shutdown is in progress. |
stopped |
Stopped on purpose (shutdownAll(), killAll(), respawn()). Final. |
failed |
Crashed too often (or restartOnExit: false) and will not be restarted. |
starting ââreadyâââś ready â â âââââ crash âââââââ´âââś restarting ââdelayâââś starting â limit reached âââś failedany live state ââ shutdownAll()/killAll() âââś stopping âââś stoppedA startTimeout that passes while starting kills the process; its exit is handled as a crash.
| Event | Arguments | When |
|---|---|---|
clusterSpawn |
cluster: ClusterInfo |
A cluster process was forked (also for each restart). |
clusterReady |
cluster: ClusterInfo |
A clusterâs worker reported ready. |
ready |
â | Every cluster is ready. Emitted again after a recovery brings them all back. |
clusterExit |
cluster, code: number | null, signal: NodeJS.Signals | null, expected: boolean |
A cluster process exited. expected is true for a deliberate shutdown. |
clusterRestart |
cluster, { delay: number; attempt: number } |
A restart was scheduled delay ms from now; attempt is the clusterâs restarts count. |
clusterUnstable |
cluster, { attempt: number; delay: number } |
Emitted with every restart once the cluster has been restarted three or more times since spawn(). |
clusterStartTimeout |
cluster: ClusterInfo |
The cluster did not report ready within startTimeout and is being killed. |
clusterGiveUp |
cluster: ClusterInfo |
The cluster exceeded supervisor.maxRestarts and will not be restarted. |
shardsReassigned |
{ shardIds: number[]; from: number; to: number } |
A failed clusterâs shards were moved to another cluster. |
hostJoin |
hostId: string |
A host authenticated (also when it reconnects). |
hostLeave |
hostId: string, reason: string |
A host disconnected or was lost. Its clusters exit and are handled as crashes. |
clusterMoved (0.3.2) |
cluster, { from: string; to: string } |
A lost hostâs cluster is restarting on another place (remote.moveClusters). |
leaderAcquired (0.3.2) |
â | This coordinator took the leader lock. |
leaderLost (0.3.2) |
reason: string |
This coordinator lost the lock, killed its clusters and is a standby again. |
plan (0.4.0) |
ClusterPlanInfo |
Each start: totalShards, clusterCount, clusterCountSource ("clusterCount", "shardsPerCluster" or "default"), requestedClusterCount, clusters ({ id, shards }), maxConcurrency, startup, identifyCoordinated. Before any process is forked. |
identify (0.4.0) |
IdentifyEvent |
The IDENTIFY queue: { type, owner, shardId, bucket, waitedMs? }, type one of "queued", "granted", "released", "expired" (a slot not given back in holdTimeout), "dropped" (its cluster exited). bucket is shardId % maxConcurrency. |
shutdown (0.4.0) |
{ clusters: number; durationMs: number } |
shutdownAll() finished: every worker exited, or was killed after the timeout. |
identifyWait |
{ shardId: number; waitMs: number } |
An IDENTIFY waits for the session-start budget to reset. Repeated waits for the same shard and second are not re-emitted. |
error |
error: Error |
Something went wrong that did not stop the deployment. See below. |
error is emitted for: a cluster whose process could not be forked (the message names the cluster and its shards; it goes through the restart policy like a crash, and the clusters after it still start); a malformed control frame from a worker; a ready report naming shards the cluster does not own; an oversized shard-bus message or health report; an asynchronous IPC send failure of a worker process; a failed fetchLimits request; a reassignment that found no target or was refused; and, with remote, a host that could not start a worker. With no error listener these are dropped.
Readiness
Section titled âReadinessâA cluster is starting until its worker sends a ready frame, which a ShardManager does once every one of its shards finished its handshake (and again after a shard dropped and recovered). Workers without a ShardManager call ClusterWorker.ready() themselves. A ready frame that names a shard the cluster does not own is rejected with an error event.
When every cluster in the deployment is ready, ready() waiters resolve and the ready event is emitted. A crash clears that state, so ready is emitted again after recovery. Readiness means every shard is served by a ready cluster: a failed cluster whose shards were reassigned no longer counts. When a cluster fails and its shards are left with no owner (no reassignOnGiveUp, restartOnExit: false, no ready cluster to take them, or the takeover failed), pending ready() calls reject with an error naming the cluster and its shards instead of waiting forever.
IDENTIFY coordination
Section titled âIDENTIFY coordinationâDiscord allows one IDENTIFY per rate-limit key (shard_id % max_concurrency) every five seconds, for the whole bot, and a daily session-start budget. With identify.coordinate on (the default), the parent holds both rules for every cluster, local or remote:
- Before a worker connects a shard, it asks the parent for a slot (
ClusterWorker.identifyGate.acquire(shardId)).ShardManagerdoes this for you, and so does aClientcreated withgateway: { shardId, shardCount }inside a worker (0.4.0); its re-IDENTIFYs after a lost session ask too. - The parent grants one slot per key at a time, at least
identify.intervalapart, in request order. - Once the IDENTIFY went out, the worker gives the slot back (
release(shardId)) and the key rests for one interval. A slot not given back withinidentify.holdTimeoutis released for the worker.
The parent denies a request for a shard the cluster does not own, and a second request for a shard that is already waiting, so a worker cannot queue more requests than it has shards. When a cluster exits, its waiting requests are dropped and the slots it held are freed.
The budget is known when /gateway/bot was read (shardCount: "auto", unset, or fetchLimits: true). Each grant spends one session; when none are left, requests wait for Discordâs reset_after (an identifyWait event says so) instead of failing, and the budget refills every 24 hours after that. Without a known budget the parent paces but never blocks.
With identify.coordinate: false, workers get LUNIBEE_IDENTIFY=local and pace themselves; a request for a slot is denied with "IDENTIFY coordination is off.". A RESUME, or a reconnect handled inside one Gateway, does not use a slot.
IdentifyCoordinator is the class the parent uses. It is exported for custom orchestrators:
import { IdentifyCoordinator, IDENTIFY_INTERVAL } from "lunibee";
const slots = new IdentifyCoordinator({ maxConcurrency: 16, interval: IDENTIFY_INTERVAL });await slots.acquire(/* owner */ 0, /* shardId */ 5);// ...send IDENTIFY...slots.release(0, 5);| Option | Default | Description |
|---|---|---|
maxConcurrency |
1 |
Number of keys. Positive integer. |
interval |
5000 |
Ms between two IDENTIFYs on one key. Cannot be negative. |
budget |
none | { total, remaining, resetAfter, period? }; period defaults to 24 h. |
holdTimeout |
60000 |
Ms after which an unreleased grant is released. Must be positive. |
onBudgetWait |
none | ({ shardId, waitMs }) => void when a request waits for the budget. |
now, schedule, cancel |
Date.now, setTimeout, clearTimeout |
Clock and timer, for tests. |
Members: acquire(owner, shardId), release(owner, shardId) (ignored unless owner holds that slot), drop(owner, reason?) (rejects the ownerâs waiters and frees its slots), close(), pump(), and the read-only maxConcurrency, remaining, waiting and holding.
Routing and payloads
Section titled âRouting and payloadsâWorkers send ShardBus messages to the parent over IPC (the worker contract sets LUNIBEE_SHARD_BUS=ipc). The parent relays only between clusters; buses in the same worker deliver to each other directly (IpcTransport delivers locally as well as sending to the parent).
| Target | What the parent does |
|---|---|
| A shard ID | Sends it only to the cluster that owns the shard. If the target is in the senderâs own cluster, or not part of the deployment, nothing is relayed. If the owner is not running, the sender gets a peer-down notice and its pending request to that shard fails at once. |
null (broadcast) |
Sends it to every other live cluster. |
the parent (-1) |
Consumed as an answer to request() or broadcast(). Answers count only from the cluster that owns the answering shard, once per shard. |
Envelope validation
Section titled âEnvelope validationâA bus frame from a worker is dropped silently unless:
typeis a string of 1 to 256 characters;idis a string of at most 256 characters;sourceis an integer;targetis an integer ornull.
Only the documented envelope is relayed (type, id, source, target, data and expectsReply when present); other fields are stripped. Anything on the IPC channel that is not a bus or cluster control frame is the botâs own process.send() traffic and is ignored. A control frame that does not parse is reported as an error event.
Payload limits
Section titled âPayload limitsâmaxMessageBytes (default 8 MiB, the same as the host link) is measured in UTF-8 bytes: a string as it is, anything else as its JSON text. A value that cannot be serialized (a cycle) counts as oversized.
| Payload | Over the limit |
|---|---|
data of a shard-bus message from a worker |
Dropped, not relayed; an error event says so. A request waiting for it times out as usual. |
A workerâs answer to requestCluster() |
The request settles at once with an error. |
| A health report from a worker | Ignored; an error event says so. The previous report is kept. |
data passed to request(), broadcast() or requestCluster() |
The call rejects with a RangeError; nothing is sent. |
RequestTracker
Section titled âRequestTrackerâThe parentâs table of pending requests, exported for custom orchestrators. Each request waits for a set of keys, each owned by one cluster, and settles when all answered, at its timeout, or when its signal aborts. An answer is accepted only from the keyâs owner and only once; late, duplicate and unknown answers are ignored. A timer per request guarantees the entry is removed.
import { RequestTracker } from "lunibee";
const tracker = new RequestTracker("parent", 10_000); // id prefix, maxPendingconst { id, done } = tracker.begin({ keys: new Map([["c:1", 1]]), timeoutMs: 5000 });tracker.answer(id, "c:1", 1, { result: "ok" }); // true when acceptedconst { replies, missing } = await done; // [{ key, result?, error? }], ["..."]failOwner(owner, error) settles every unanswered key of a cluster with error; close() settles everything with what it has; size is the number of waiting requests. begin() throws when maxPending requests are already waiting.
ClusterManagerHealth
Section titled âClusterManagerHealthâ| Field | Type | Description |
|---|---|---|
state |
"idle" | "starting" | "ready" | "degraded" | "stopping" | "stopped" |
idle before the first spawn(); stopping during a shutdown; stopped when every cluster is stopped; ready when all are ready; degraded when any is restarting or failed; otherwise starting. |
totalShards |
number |
shardCount. |
clusters |
ClusterHealth[] |
Every cluster of the deployment, sorted by ID, whatever its state. |
readyClusters |
number |
Clusters in ready. |
missingShards |
number[] |
Shards whose owning cluster is not ready, sorted. |
ClusterHealth
Section titled âClusterHealthâ| Field | Type | Description |
|---|---|---|
id |
number |
Cluster ID. |
state |
ClusterState |
Lifecycle state. |
pid |
number? |
Process ID (on its host, for a remote cluster); only while starting or ready. |
host |
string |
"local" or the host ID. |
shardIds |
number[] |
Shards it owns now. |
uptimeMs |
number |
Ms since the current process was forked; 0 when it is not running. |
restarts |
number |
Restarts after a crash. |
startedAt |
number? |
When the current process was forked. |
readyAt |
number? |
When the current process reported ready. |
lastReadyAt |
number? |
When any process of this cluster last reported ready. |
shards |
ShardHealth[] |
Per-shard health from the last report; empty until the worker reports. |
metrics |
WorkerMetrics? |
{ rssBytes, heapUsedBytes, cpuUserMicros, cpuSystemMicros }, when requested with refreshHealth({ metrics: true }) and the worker answered. |
reportedAt |
number? |
When shards and metrics were reported (epoch ms). |
ShardHealth is { id, state, ping, status, readyAt?, disconnectedAt? }, where state is the Gateway state and status the shard lifecycle (idle, connecting, ready, disconnected, unstable, destroyed). See ShardManager.
A workerâs ShardManager pushes a report each time all its shards become ready, and answers refreshHealth(). Cluster health follows the process: a Gateway problem in one shard, a reconnect storm included, never restarts a healthy cluster. Only an exited process (or a missed startTimeout) is restarted.
Recovery
Section titled âRecoveryâRestarts
Section titled âRestartsâWhen a cluster exits outside a shutdown, the parent:
- drops its IDENTIFY requests and frees its slots;
- settles its pending requests with
"Cluster <id> exited."; - calls
onClusterExitand emitsclusterExit; - tells the other clusters its shards are down, so their pending requests to those shards fail at once;
- schedules a restart with the same shards (
clusterRestart), or gives up.
The supervisor option (SupervisorOptions) decides the delay:
| Option | Default | Description |
|---|---|---|
restartDelay |
restartDelay, else 5000 |
Delay before the first restart, in ms. Cannot be negative. |
backoff |
1 |
Each further crash inside window multiplies the delay by this. At least 1; 1 is a fixed delay. |
maxDelay |
300000 |
Longest delay, in ms. Cannot be negative. |
jitter |
0 |
Random spread added to each delay, as a fraction of it (0â1). The result is still capped at maxDelay. |
maxRestarts |
unlimited | Give up after this many restarts inside window. 0 gives up on the first crash. |
window |
60000 |
The time, in ms, over which restarts are counted. A cluster that stays up longer is forgiven. Must be positive. |
The delay of the n-th restart inside the window is min(maxDelay, restartDelay Ă backoff^(nâ1)) Ă (1 + jitter Ă random), capped at maxDelay. When the limit is reached the cluster becomes failed, onGiveUp is called and clusterGiveUp is emitted. With restartOnExit: false an exit makes the cluster failed straight away, without clusterGiveUp.
A restarted process is a new one: late messages and the exit of the process it replaced are ignored.
ShardSupervisor is the class behind this, exported for custom orchestrators: new ShardSupervisor(options?, random?), next(id, now?) returns the delay in ms or null to give up, and reset(id?) forgets crashes.
Reassignment
Section titled âReassignmentâWith reassignOnGiveUp: true, a cluster that gives up hands its shards to the ready cluster that owns the fewest:
- The parent asks that clusterâs worker to start them (a built-in
requestCluster()request thatShardManageranswers withaddShards()), with a timeout ofstartTimeout, or 120000 ms. - Only after the worker confirms does ownership move:
clusterForShard()returns the new owner, the targetâsshardsgrows, the failed clusterâsshardsbecomes empty andshardsReassignedis emitted.
The failed process is already dead and ownership moves only after confirmation, so no shard ever has two owners. If no cluster is ready, or the worker refuses or times out, an error event says so and the shards stay down. Reassignment does not happen for restartOnExit: false.
Clusters as threads
Section titled âClusters as threadsâ(0.3.2) fork: threadFork(options?) runs every local cluster as a worker thread of the managerâs process instead of a forked process. The worker script does not change: ShardManager, ShardBus and ClusterWorker find the parent through the threadâs port, and IDENTIFY coordination, routing, health and restarts work as with processes.
import { ClusterManager, threadFork } from "lunibee";
const manager = new ClusterManager({ token: process.env.DISCORD_TOKEN!, script: "./worker.ts", shardsPerCluster: 1, // one shard per thread fork: threadFork({ resourceLimits: { maxOldGenerationSizeMb: 512 } }),});- Threads start faster and share one processâs memory overhead. A crash that ends the process (out of memory, a native fault) ends every cluster with it, and
ClusterInfo.process.pidis the thread ID. kill()terminates the thread: a thread cannot handle signals, soshutdownAll()ends it at once, asSIGTERMends a process without a handler. Its exit reports the signal it was asked for.envbecomes the threadâsprocess.env. Remote hosts still fork processes.threadEndpoint()returns the parent link inside such a thread (undefinedelsewhere), for code that talks to the parent itself.
Process per shard: shardsPerCluster: 1 gives every shard its own process (or thread).
Worker side
Section titled âWorker sideâEnvironment variables
Section titled âEnvironment variablesâClusterManager sets these in every worker, on top of the parentâs environment (local clusters) or the hostâs environment (remote clusters):
| Variable | Value |
|---|---|
LUNIBEE_CLUSTER_WORKER |
1. Marks a worker. A ClusterManager refuses to spawn() in such a process. |
CLUSTER_ID |
This clusterâs ID. |
CLUSTER_COUNT |
How many clusters the deployment started (clusters with shards). |
SHARD_COUNT |
Total shards of the deployment (Discordâs num_shards). |
SHARD_LIST |
Comma-separated shard IDs this cluster owns. |
LUNIBEE_SHARD_BUS |
ipc: ShardBus uses the IPC channel instead of a BroadcastChannel. |
LUNIBEE_IDENTIFY |
coordinated when the parent hands out IDENTIFY slots, otherwise local. |
LUNIBEE_STARTUP (0.4.0) |
sequential with startup: "sequential"; unset otherwise. |
LUNIBEE_READY_TIMEOUT (0.4.0) |
With startup: "sequential": identify.holdTimeout (default 60000), how long a worker waits for a shardâs READY. |
These seven are the only variables a coordinator can set in a worker on another host.
readClusterContext(env?) and isClusterWorker(env?)
Section titled âreadClusterContext(env?) and isClusterWorker(env?)âreadClusterContext(env = process.env): ClusterContext | nullisClusterWorker(env = process.env): booleanisClusterWorker() is true when LUNIBEE_CLUSTER_WORKER is 1. readClusterContext() parses the variables above into a ClusterContext, returns null outside a worker, and throws a RangeError when they are present but malformed (not non-negative integers, or shard IDs outside SHARD_COUNT or repeated).
ClusterContext field |
Type | From |
|---|---|---|
clusterId |
number |
CLUSTER_ID |
clusterCount |
number |
CLUSTER_COUNT |
shardCount |
number |
SHARD_COUNT |
shardIds |
number[] |
SHARD_LIST |
coordinatedIdentify |
boolean |
LUNIBEE_IDENTIFY === "coordinated" |
startup |
"safe" | "sequential" |
LUNIBEE_STARTUP (default "safe") |
readyTimeout |
number | undefined |
LUNIBEE_READY_TIMEOUT |
ClusterWorker
Section titled âClusterWorkerâThe worker side of the contract: it reports readiness and health to the parent, answers the parentâs requests, and asks the parent for IDENTIFY slots. A ShardManager inside a cluster uses it automatically. Use it directly when a worker runs its own Gateways, or to answer requestCluster().
import { ClusterWorker } from "lunibee";
const worker = ClusterWorker.current(); // undefined outside a clusterworker?.onRequest("flushCaches", async () => { await cache.flush(); return { ok: true };});Getting one
Section titled âGetting oneâstatic current(): ClusterWorker | undefinednew ClusterWorker(context: ClusterContext, endpoint?: WorkerEndpoint, options?: { exitOnParentLoss?: boolean; exit?: (code: number) => void })current() returns this processâs worker, creating it on first use, or undefined when the process was not forked by a ClusterManager (or has no IPC channel). It throws when the contract variables are malformed. ClusterWorker.reset() closes and forgets it (tests).
The constructor takes a context, an endpoint (process by default: anything with send, on and off) and options. It throws if the endpoint has no send().
| Option | Default | Description |
|---|---|---|
exitOnParentLoss |
true for the real process, false for an injected endpoint |
Exit with code 1 when the parentâs IPC channel disconnects. Without a parent nothing coordinates IDENTIFY or fences the process, and its Gateway sessions would keep running beside the replacement the parent starts. A closed worker never exits this way. |
exit |
process.exit |
Replaces the exit call; for tests. |
Losing the parent always rejects pending IDENTIFY requests, whatever this option says.
Members
Section titled âMembersâ| Member | Description |
|---|---|
context |
The ClusterContext it was built with. |
identifyGate |
An IdentifyGate (acquire(shardId): Promise<void>, release(shardId): void) when context.coordinatedIdentify is true, otherwise undefined. acquire() resolves when the shard may IDENTIFY and rejects when the parent denies it, the parent goes away, or the worker is closed. Call release() once the IDENTIFY was sent. |
ready(shardIds?) |
Tells the parent the cluster is ready. Defaults to all of context.shardIds. The parent marks the cluster ready on any ready frame whose shards it owns, so call it once all of them are ready. |
reportHealth(report) |
Pushes a WorkerReport ({ shards: ShardHealth[], metrics?, reportedAt }) to the parent without being asked. |
onRequest(type, handler) |
Answers requestCluster() requests of type. The handler gets data; its return value (awaited) or thrown error message goes back. Returns the worker. Throws a TypeError for a blank type or a non-function. A request with no handler is answered with No handler for "<type>". |
close() |
Stops listening, rejects pending IDENTIFY requests, and turns off exitOnParentLoss. |
Request types starting with __lunibee: are reserved: ShardManager registers the built-in health and shard-assignment handlers under them.
Own Gateways
Section titled âOwn Gatewaysâimport { Gateway, GatewayIntentBits, ClusterWorker, readClusterContext } from "lunibee";
const token = process.env.DISCORD_TOKEN!;const intents = [GatewayIntentBits.Guilds];const context = readClusterContext()!;const worker = ClusterWorker.current()!;for (const shardId of context.shardIds) { const gateway = new Gateway({ token, intents, shardId, shardCount: context.shardCount }); await worker.identifyGate?.acquire(shardId); try { const identified = new Promise<void>((resolve) => gateway.on("stateChange", (change) => { const { next } = change as { next: string }; if (next !== "CONNECT" && next !== "HELLO") resolve(); }), ); await gateway.connect(); await identified; // release only once IDENTIFY went out } finally { worker.identifyGate?.release(shardId); }}worker.ready();Gateways created outside ShardManager and without the gate are not paced by the parent.
Shard allocation helpers
Section titled âShard allocation helpersâThe pure functions the manager uses, exported for custom orchestrators and tests.
allocateShards(shardIds, clusterCount)
Section titled âallocateShards(shardIds, clusterCount)âallocateShards(shardIds: readonly number[], clusterCount: number): number[][]Splits shard IDs into contiguous blocks, in order; when they do not divide evenly the first clusters get one more. The same input always gives the same output. Lists at the end are empty when there are more clusters than shards. Throws a RangeError unless clusterCount is a positive integer. Before 0.4.0 the split was round-robin.
allocateShards([0, 1, 2, 3, 4, 5], 4); // [[0, 1], [2, 3], [4], [5]]planClusters(options) (0.4.0)
Section titled âplanClusters(options) (0.4.0)âplanClusters({ shardIds, clusterCount?, shardsPerCluster?, defaultClusterCount }): ClusterPlanWhat spawn() does with the options, without starting anything: clusterCount splits the shards with allocateShards, shardsPerCluster fills clusters of that size (the last may be partial), and neither uses defaultClusterCount. Never more clusters than shards, and no cluster is empty. Returns { clusters, source, requested }. Throws a RangeError for both options at once, a count that is not a positive integer, or no shards.
planClusters({ shardIds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], shardsPerCluster: 4, defaultClusterCount: 1 }).clusters;// [[0, 1, 2, 3], [4, 5, 6, 7], [8, 9]]assertCompleteAllocation(chunks, expected)
Section titled âassertCompleteAllocation(chunks, expected)âassertCompleteAllocation(chunks: readonly (readonly number[])[], expected: readonly number[]): voidThrows an Error naming the duplicate, missing and unexpected shard IDs unless the chunks cover expected exactly once.
validateShardIds(shardIds, total)
Section titled âvalidateShardIds(shardIds, total)âvalidateShardIds(shardIds: readonly number[], total: number): voidThrows a RangeError unless total is a positive integer and every ID is a unique integer in [0, total).
Several hosts
Section titled âSeveral hostsâremote turns the manager into a coordinator for ClusterHost agents on other machines. Clusters on a host are supervised, routed, paced and health-checked like local ones. Options, placement, authentication and fencing are on the ClusterHost page.