Skip to main content
Quick navigation

Agent Setup

danger

Monitoring is changing to improve the ease of setup and usability. As a result, agent support will be removed from April 3rd, 2023. Find out more.

It should not be setup in case of a new Conduktor deployment, and will be maintained only for existing users to support their transition.

Overview

You should setup the Agent to use Monitoring at full capacity. Running Prometheus node exporter and JMX exporter will ensure you maximize the metrics you have access to within Conduktor. As there can be a wide range of unique configurations and edge cases when seting up an agent, these instructions should only be considered guidelines. Should you encounter any issues with setup, please contact us directly using the chat widget on this page.

Setup JMX Exporter

Download the agent and its configuration

Create a new directory for jmx-exporter

mkdir /opt/jmx-exporter

Download the jar into your newly generated directory:

curl https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.17.2/jmx_prometheus_javaagent-0.17.2.jar -o/opt/jmx-exporter/jmx_prometheus_javaagent-0.17.2.jar

Download the associated monitoring configuration file:

curl http://demo.conduktor.io/monitoring/kafka-broker.yml -o /opt/jmx-exporter/kafka-broker.yml

Start your Kafka service

Your Kafka server must start with the following javaagent:

-javaagent:/opt/jmx-exporter/jmx_prometheus_javaagent-0.17.2.jar=9101:/opt/jmx-exporter/kafka-broker.yml

For instance, you can set the environment variable:

KAFKA_OPTS=-javaagent:/opt/jmx-exporter/jmx_prometheus_javaagent-0.17.2.jar=9101:/opt/jmx-exporter/kafka-broker.yml

Setup Node Exporter

Install Prometheus node exporter on your server (apt based systems):

apt install prometheus-node-exporter

Note on other systems you can install it manually (docs).

Node exporter can be started with its default configuration and should listen on port 9100. However, you may want to select which filesystems are monitored. You can use the --collector.filesystem.mount-points-exclude=... option for this.