> ## Documentation Index
> Fetch the complete documentation index at: https://docs.conduktor.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Gateway metrics reference

> Prometheus metrics exposed by Conduktor Gateway: connection, latency, throughput, thread, per-user and storage metrics. Use with Grafana, Cortex, Mimir.

Gateway exposes metrics in the Prometheus format that you can scrape and send to your external log management system.

The metrics endpoint is `<gateway_host>:<gateway_port>/metrics` of your deployed Gateway instance. For example: `localhost:8888/metrics`.

<Info>
  If `GATEWAY_SECURED_METRICS` is enabled (which is the default), you'll have to use the credentials specified in `GATEWAY_ADMIN_API_USERS` to access the metrics.
</Info>

## Prometheus metric suffixes

When metrics are exported to Prometheus, suffixes are automatically appended based on the metric type:

* **Counter**: `_total` suffix (e.g., `gateway_bytes_exchanged_total`)
* **Timer (summary)**: `_seconds_count`, `_seconds_sum` and `_seconds_max` suffixes (e.g., `gateway_apiKeys_latency_request_response_seconds_count`)
* **Timer (histogram)**: same as summary, plus `_seconds_bucket` with a `le` label for histogram buckets (e.g., `gateway_latency_request_response_seconds_bucket`). Only `gateway_latency_request_response` is exported as a histogram.
* **Gauge**: no suffix (e.g., `gateway_active_connections_vcluster`)

The table below lists the Prometheus metric names as they appear when scraped. Dots (`.`) in the underlying metric name are converted to underscores (`_`).

| Prometheus metric name                                    | Metric type | Labels                                                                    | Description                                                                                                                                                     |
| --------------------------------------------------------- | ----------- | ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Upstream connection metrics**                           |             |                                                                           |                                                                                                                                                                 |
| `gateway_upstream_connection_close_rate_seconds_count`    | summary     |                                                                           | The number of connections closed                                                                                                                                |
| `gateway_upstream_connection_close_rate_seconds_sum`      | summary     |                                                                           | Total time spent closing connections                                                                                                                            |
| `gateway_upstream_connection_close_rate_seconds_max`      | gauge       |                                                                           | Maximum time spent closing a connection                                                                                                                         |
| `gateway_upstream_connection_close_total`                 | counter     |                                                                           | The total number of connections closed                                                                                                                          |
| `gateway_upstream_connection_creation_rate_seconds_count` | summary     |                                                                           | The number of new connections established                                                                                                                       |
| `gateway_upstream_connection_creation_rate_seconds_sum`   | summary     |                                                                           | Total time spent creating connections                                                                                                                           |
| `gateway_upstream_connection_creation_rate_seconds_max`   | gauge       |                                                                           | Maximum time spent creating a connection                                                                                                                        |
| `gateway_upstream_connection_creation_total`              | counter     |                                                                           | The total number of new connections established                                                                                                                 |
| `gateway_upstream_select_rate_seconds_count`              | summary     |                                                                           | The number of times the I/O layer checked for new I/O to perform                                                                                                |
| `gateway_upstream_select_rate_seconds_sum`                | summary     |                                                                           | Total time spent on I/O select operations                                                                                                                       |
| `gateway_upstream_select_rate_seconds_max`                | gauge       |                                                                           | Maximum time spent on a single I/O select operation                                                                                                             |
| `gateway_upstream_select_total`                           | counter     |                                                                           | The total number of times the I/O layer checked for new I/O to perform                                                                                          |
| `gateway_upstream_io_wait_rate_seconds_count`             | summary     |                                                                           | The number of times the I/O thread spent waiting                                                                                                                |
| `gateway_upstream_io_wait_rate_seconds_sum`               | summary     |                                                                           | Total time the I/O thread spent waiting                                                                                                                         |
| `gateway_upstream_io_wait_rate_seconds_max`               | gauge       |                                                                           | Maximum time the I/O thread spent waiting in a single wait                                                                                                      |
| `gateway_upstream_io_wait_total`                          | counter     |                                                                           | The total time the I/O thread spent waiting                                                                                                                     |
| **Connection metrics**                                    |             |                                                                           |                                                                                                                                                                 |
| `gateway_active_connections_vcluster`                     | gauge       | `vcluster`                                                                | The number of active connections per Virtual Cluster                                                                                                            |
| `gateway_upstream_connections_upstream_connected`         | gauge       | `name`, `threadId`                                                        | The number of connections from Gateway to the backing Kafka cluster                                                                                             |
| `gateway_upstream_io_nodes`                               | gauge       | `clusterId`                                                               | The number of Kafka nodes                                                                                                                                       |
| `gateway_backend_brokered_active_connections`             | gauge       |                                                                           | The number of active backend brokers on the MAIN cluster                                                                                                        |
| **Latency metrics**                                       |             |                                                                           |                                                                                                                                                                 |
| `gateway_latency_request_response_seconds_count`          | histogram   |                                                                           | The number of requests processed                                                                                                                                |
| `gateway_latency_request_response_seconds_sum`            | histogram   |                                                                           | The total round-trip latency from Gateway to Kafka and back                                                                                                     |
| `gateway_latency_request_response_seconds_max`            | gauge       |                                                                           | The maximum round-trip latency from Gateway to Kafka and back                                                                                                   |
| `gateway_latency_request_response_seconds_bucket`         | histogram   | `le`                                                                      | Histogram buckets for round-trip latency distribution                                                                                                           |
| `gateway_latency_request_response_seconds`                | gauge       | `quantile`                                                                | Round-trip latency percentiles per Gateway instance (p30, p50, p95, p99)                                                                                        |
| `gateway_apiKeys_latency_request_response_seconds_count`  | summary     | `apiKeys`                                                                 | The number of requests processed per API key                                                                                                                    |
| `gateway_apiKeys_latency_request_response_seconds_sum`    | summary     | `apiKeys`                                                                 | The total latency per API key                                                                                                                                   |
| `gateway_apiKeys_latency_request_response_seconds_max`    | gauge       | `apiKeys`                                                                 | The maximum latency per API key                                                                                                                                 |
| `gateway_apiKeys_throttle_ms_seconds_count`               | summary     | `apiKeys`                                                                 | The number of throttled responses per API key                                                                                                                   |
| `gateway_apiKeys_throttle_ms_seconds_sum`                 | summary     | `apiKeys`                                                                 | The total throttle time in Kafka broker responses per API key. Reflects throttling imposed by Kafka (for example, due to quota violations).                     |
| `gateway_apiKeys_throttle_ms_seconds_max`                 | gauge       | `apiKeys`                                                                 | The maximum throttle time in Kafka broker responses per API key                                                                                                 |
| **Throughput metrics**                                    |             |                                                                           |                                                                                                                                                                 |
| `gateway_bytes_exchanged_total`                           | counter     | `direction`                                                               | The total bytes exchanged between clients and Gateway. Tagged by direction: `upstream` (from clients toward Kafka) or `downstream` (from Kafka toward clients). |
| `gateway_bytes_exchanged_vcluster_total`                  | counter     | `vcluster`, `direction`                                                   | The total bytes exchanged between Gateway and Kafka brokers for a Virtual Cluster                                                                               |
| `gateway_bytes_exchanged_topic_total`                     | counter     | `vcluster`, `direction`, `topic`, `apiKeys`                               | The total bytes exchanged within the context of a specific topic                                                                                                |
| **Thread metrics**                                        |             |                                                                           |                                                                                                                                                                 |
| `gateway_thread_request_received_total`                   | counter     | `threadId`, `name`                                                        | Client requests received and forwarded to Kafka, per Gateway thread                                                                                             |
| `gateway_thread_request_rebuild_total`                    | counter     | `api_key`, `name`, `threadId`, `vcluster`                                 | Requests rebuilt per Gateway thread                                                                                                                             |
| `gateway_request_expired_total`                           | counter     | `thread`, `apiKey`, `vcluster`\*, `username`\*, `clusterId`\*, `nodeId`\* | The number of client requests that timed out waiting for a response from Kafka. Labels marked with \* are conditional and only present when available.          |
| `gateway_thread_tasks`                                    | gauge       | `threadId`, `name`                                                        | The number of pending tasks on the Gateway thread (where all request/response rebuilding happens)                                                               |
| **Per-user metrics**                                      |             |                                                                           |                                                                                                                                                                 |
| `gateway_error_per_apiKeys_total`                         | counter     | `vcluster`, `username`, `apiKeys`                                         | The total Gateway processing exceptions per API key for the specified Virtual Cluster and username                                                              |
| `gateway_current_inflight_apiKeys_total`                  | counter     | `vcluster`, `username`, `apiKeys`                                         | The current inflight API keys of the specified Virtual Cluster and username                                                                                     |
| `gateway_failed_authentications_total`                    | counter     | `vcluster`, `user`, `apiKeys`                                             | The number of authentication attempts that failed for each user                                                                                                 |
| **Storage metrics**                                       |             |                                                                           |                                                                                                                                                                 |
| `gateway_kcache_size`                                     | gauge       | `type`                                                                    | The size of the kcache (equal to the number of key-value pairs in the cache)                                                                                    |
| `gateway_topic_log_end_offset`                            | gauge       | `vcluster`, `topic`, `partition`                                          | The log end offset of logical topics                                                                                                                            |
| `gateway_topic_current_offset`                            | gauge       | `vcluster`, `group`, `topic`, `partition`                                 | The current offset of a consumer group on a logical topic                                                                                                       |
| **License metrics**                                       |             |                                                                           |                                                                                                                                                                 |
| `gateway_license_remaining_days`                          | gauge       |                                                                           | The number of days remaining on your Gateway license                                                                                                            |

<Info>
  The `gateway_topic_log_end_offset` and `gateway_topic_current_offset` metrics are currently only populated for topics using [topic concentration](/guide/conduktor-concepts/logical-topics). For non-concentrated topics, use standard Kafka consumer lag monitoring.
</Info>

## Related resources

* [Set up alerts and notifications](/guide/monitor-brokers-apps/alerts)
* [Deploy Cortex](/guide/conduktor-in-production/deploy-artifacts/deploy-cortex)
* [Give us feedback/request a feature](https://conduktor.io/roadmap) <Icon icon="up-right-from-square" />
