Skip to main content
Quick navigation

Memory configuration (1.19 and later)

RUN_MODE is gone!
We now rely on container CGroups limits and use up to 80% of container memory limit for JVM max heap size. Our settings are the following

-XX:+UseContainerSupport -XX:MaxRAMPercentage=80

You now only need to care about the limits that you set to your container.

# Values.yaml
...
platform:
resources:
limits:
memory: 8Gi
...

Read this article to understand what this is about.

Memory configuration before 1.19

Conduktor Console uses the environment variable RUN_MODE to switch between different memory configuration presets.

RUN_MODETarget RAM usageConsole XmxGovernance XmxAuthenticator Xmx
nano3GB1024m256m128m
small4GB1700m512m128m
medium6GB3072m1024m256m
large8GB4096m2048m256m
customNo targetN.A.N.A.N.A.
caution

This means that Conduktor Console won't start if the container CGroup memory limit is set below the target RAM usage.

Modules memory usage

  • Console : contains administation, RBAC and kafka resources exploration. Should scale with the number of users and kafka resources.
  • Governance : contains data-masking engine, kafka indexer and kafka metrics exporter. Should scale for large kafka cluster to index.
  • Authenticator : Authentication module to interface SSO/LDAP and local users. Scale on internal user list size and number of concurrent users connected.

Override specific module memory settings

All modules memory presets can be overridden using following the environment variables :

  • CONSOLE_MEMORY_OPTS : Used by Console module
  • GOVERNANCE_MEMORY_OPTS : Used by governance module
  • AUTHENTICATOR_MEMORY_OPTS : Used by authenticator module

The other modules will still use the preset from the RUN_MODE value.

Example

RUN_MODE=small
GOVERNANCE_MEMORY_OPTS="-Xms1025m -Xmx5000m"

In this example we set around 5GB of RAM for Governance and leave other modules like Console and Authenticator with small memory presets.

caution

Be aware that in this example the target RAM usage is increased from 4GB to 8-9GB and container CGroup memory limit should be set accordingly.

Using custom RUN_MODE

If existing RUN_MODE presets are too restrictive or don't meet your needs, you can use the custom RUN_MODE and this removes the CGroup memory limit startup check and requires that you set manually the Java memory options for all internal components using the environment variables CONSOLE_MEMORY_OPTS, GOVERNANCE_MEMORY_OPTS and AUTHENTICATOR_MEMORY_OPTS.

Unlike single module override, all environment variables must be set when RUN_MODE=custom.

Example

Conduktor Console running environment variables with a target of 16GB:

RUN_MODE=custom
CONSOLE_MEMORY_OPTS="-Xms2048m -Xmx8000m"
GOVERNANCE_MEMORY_OPTS="-Xms1025m -Xmx5000m"
AUTHENTICATOR_MEMORY_OPTS="-Xms128m -Xmx512m"

In this example we set around 8GB for Console, 5GB for Governance and 512MB for Authenticator leaving a safety margin of 2.5GB for other JVM memory pools and extra internal modules.

caution

In custom mode, Conduktor will not check CGroup memory limits to prevent under provisioning that could lead to an unexpected crash.