Skip to main content

Overview

Each live consume session holds records in the memory of the Console instance that serves it. From Console 1.49.0, you can cap how many sessions run at the same time, so a burst of consume activity, or one user opening many consume tabs, can’t take a large share of an instance’s resources. Two limits are available, and both are off (0) by default, so upgrading changes nothing: no sessions are rejected, and no background work runs until you set a limit. Each is an environment-variable change, with no new release or restart cycle needed.
  • A per-instance limit caps the live consume sessions one Console instance serves, protecting that instance’s memory under heavy use.
  • A per-user limit caps the live consume sessions one user can have open across your whole Console deployment, so one person can’t crowd out everyone else.
Limits are checked only when a session starts. A session that’s already running is never interrupted or timed out by this feature, and there’s still no limit on how long a session can stay open.

Set the limits

Set the limits in your Console configuration. Start with the two limits, and leave the advanced settings at their defaults:
Or as environment variables:
See Consume rate limit properties for every setting, including the advanced session-bookkeeping values. Use the metrics below to pick values that fit your usage before you enable them.

What a user sees when a limit is reached

When a limit is reached, the user gets a clear message:
  • If the instance is busy, Console quietly retries: another instance may have room, so the retry can land there.
  • If the user is at their own limit, Console tells them how many sessions they have open, so they can close one.

Choose limits from metrics

Console exposes these Prometheus metrics on its existing metrics endpoint. Watch them to pick sensible limits before you enable them:
  • console_consume_sessions_active — a gauge, per instance: the live consume sessions an instance is serving right now.
  • console_consume_rejected_total{scope="global"} — a counter: sessions rejected because an instance was at capacity. A sustained rate here is the signal to scale out or raise the per-instance limit.
  • console_consume_rejected_total{scope="user"} — a counter: sessions rejected because a user was at their personal limit. Spikes here usually mean a team’s workflow needs a higher per-user limit.
Console also logs one line at INFO level for each rejection, naming the user, cluster and topic, and one line when it clears sessions left behind by a crashed instance. Occasional crash-cleanup lines are normal.

Scale for consume volume

Every live consume session runs entirely on the instance that serves it, so consume capacity grows with the number of Console instances. Size the fleet for peak concurrent consume sessions and API traffic: the per-instance limit protects each node’s memory, and the automatic retry lands a rejected session on a less-loaded instance. Background work such as indexing and alerts runs on a single elected leader, so adding instances doesn’t make it faster. Give each instance enough CPU and memory to carry that load while it’s the leader.