To enable monitoring via Console, you have to deploy and configure the console-cortex artifact.

​
Deploy and configure Cortex

The only required property is CDK_CONSOLE-URL, everything else is related to storage for the metrics. By default, data will be stored in /var/conduktor/monitoring inside the running image. You can mount a volume on this folder to keep metrics data between updates. Alternatively, you can use the storage parameters described below to store the data using either s3, gcs, azure or swift.
Environment variableDescriptionMandatoryTypeDefaultSince
CDK_CONSOLEURLConsole URL and port (example: "http://conduktor-console:8080").truestring∅1.18.0
CDK_SCRAPER_SKIPSSLCHECKDisable TLS check when scraping metrics from Console.falseboolfalse1.18.2
CDK_SCRAPER_CAFILEPath to CA certificate file inside the container to perform TLS check when scraping metrics from Console.falsestring∅1.18.2
S3
CDK_MONITORING_STORAGE_S3_ENDPOINTS3 storage endpoint.falsestring∅1.18.0
CDK_MONITORING_STORAGE_S3_REGIONS3 storage region.falsestring∅1.18.0
CDK_MONITORING_STORAGE_S3_BUCKETS3 storage bucket name.truestring∅1.18.0
CDK_MONITORING_STORAGE_S3_INSECURES3 storage SSL/TLS check flag.falseboolfalse1.18.0
CDK_MONITORING_STORAGE_S3_ACCESSKEYIDAccess key ID of an AWS IAM identity for monitoring to upload logs to S3. If set to false, and if you are running on a Kubernetes deployment, Monitoring can attempt to leverage AWS IRSA for the Pod’s service account when connecting to S3.falsestring∅1.18.0
CDK_MONITORING_STORAGE_S3_SECRETACCESSKEYSecret access key of an AWS IAM identity for monitoring to upload logs to S3. If set to false, and if you are running on a Kubernetes deployment, Monitoring can attempt to leverage AWS IRSA for the Pod’s service account when connecting to S3.falsestring∅1.18.0
GCS
CDK_MONITORING_STORAGE_GCS_BUCKETNAMEGCS storage bucket name.truestring∅1.18.0
CDK_MONITORING_STORAGE_GCS_SERVICEACCOUNTGCS storage service account JSON content.truestring∅1.18.0
Azure
CDK_MONITORING_STORAGE_AZURE_ACCOUNTNAMEAzure storage account name.truestring∅1.18.0
CDK_MONITORING_STORAGE_AZURE_ACCOUNTKEYAzure storage account key.truestring∅1.18.0
CDK_MONITORING_STORAGE_AZURE_CONTAINERNAMEAzure storage container name.truestring∅1.18.0
CDK_MONITORING_STORAGE_AZURE_ENDPOINTSUFFIXAzure storage endpoint suffix.falsestringblob.core.windows.net1.18.0
Swift
CDK_MONITORING_STORAGE_SWIFT_AUTHURLSwift storage authentication URL.truestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_PASSWORDSwift storage user password.truestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_CONTAINERNAMESwift storage container name.truestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_USERIDSwift storage user ID.falsestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_USERNAMESwift storage user name.falsestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_USERDOMAINNAMESwift storage user domain name.falsestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_USERDOMAINIDSwift storage user domain ID.falsestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_DOMAINIDSwift storage user domain ID.falsestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_DOMAINNAMESwift storage user domain name.falsestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_PROJECTIDSwift storage project ID.falsestring∅1.18.0
CDK_MONITORING_STORAGE_SWIFT_REGIONNAMESwift storage region name.falsestring∅1.18.0
Logs
CORTEX_ROOT_LOG_LEVELCortex log level.falsestringinfo1.18.0
CORTEX_ALERT_ROOT_LOG_LEVELAlert manager log level.falsestringinfo1.18.0
PROMETHEUS_ROOT_LOG_LEVELPrometheus log level.falsestringinfo1.18.0

​
Sample configuration

In a docker compose it may look like the following: 
version: '3.8'
services:
  conduktor-console:
    image: conduktor/conduktor-console
    ports:
      - "8080:8080"
    environment:
      CDK_MONITORING_CORTEX-URL: http://conduktor-monitoring:9009/
      CDK_MONITORING_ALERT-MANAGER-URL: http://conduktor-monitoring:9010/
      CDK_MONITORING_CALLBACK-URL: http://conduktor-console:8080/monitoring/api/
      CDK_MONITORING_NOTIFICATIONS-CALLBACK-URL: http://localhost:8080
  conduktor-monitoring:
    image: conduktor/conduktor-console-cortex
    ports:
      - "9009:9009" # cortex api
      - "9010:9010" # alertmanager api
      - "9090:9090" # prometheus api
    environment:
      CDK_CONSOLE-URL: "http://conduktor-console:8080"

​
Overriding configuration

​
Overriding with YAML

Cortex
Cortex configuration can be patched by mounting a YAML file into path /opt/override-configs/cortex.yaml. For an alternative path set the location using the environment variable CORTEX_OVERRIDE_CONFIG_FILE. This is not currently available for Alert Manager. For example, create a file cortex.yaml add in only your overrides: 
limits:
  ingestion_rate: 50000
  max_series_per_metric: 100000
Mount to /opt/override-configs/cortex.yaml. Spin up the container. Exec into the container and confirm the contents, replace 2 with the number of lines of override you wish to see, or remove grep to get the whole file: cat /var/conduktor/configs/monitoring-cortex.yaml | grep limits -A2. You should see a similar entry to the below in the opening logs: 
INFO monitoring_entrypoint - Patch "/var/conduktor/configs/monitoring-cortex.yaml" configuration with "/opt/override-configs/cortex.yaml" fragment
Prometheus Prometheus configuration can be overridden with replace, by mounting a YAML file into path /opt/override-configs/prometheus.yaml. For an alternative path set the location using an environment variable PROMETHEUS_OVERRIDE_CONFIG_FILE. You should see a similar entry to the below in the opening logs: 
INFO monitoring_entrypoint - Replace "/var/conduktor/configs/monitoring-prometheus.yaml" configuration with "/opt/override-configs/prometheus.yaml" content

​
Overriding with ConfigMap

If you are deploying Cortex using our Helm charts you may expand the input with a custom ConfigMap for patching configuration such as retention time within Cortex. You can also replace the Prometheus config as described below. 
apiVersion: v1
kind: ConfigMap
metadata:
  name: conduktor-console-cortex-config
  labels:
    app.kubernetes.io/name: console
    app.kubernetes.io/instance: conduktor
    app.kubernetes.io/component: conduktor-platform-cortex
data:
  cortex.yaml: |
    blocks_storage:
      tsdb:
        retention_period: 24h
  prometheus.yaml: |
    global:
      scrape_interval: 15s
      evaluation_interval: 15s
On chart values.yaml : 
platformCortex:
  extraVolumes:
    - name: cortex-config-override
      configMap:
        name: conduktor-console-cortex-config
  extraVolumeMounts:
        - name: cortex-config-override
          mountPath: /opt/override-configs/cortex.yaml
          subPath: cortex.yaml
        - name: cortex-config-override
          mountPath: /opt/override-configs/prometheus.yaml
          subPath: prometheus.yaml

​
Monitoring endpoints

Monitoring is designed to be interacted with through the API endpoints by Console (not users). Therefore, ingress isn’t available externally and you shouldn’t set it up because an authentication mechanism doesn’t exist.

​
Cortex endpoint

/ready on port 9009 If Cortex is running, returns the 200 status with ready.
cURL example
curl -s "http://localhost:9009/ready"

​
Alertmanager endpoint

/ready on port 9010 If Alertmanager is running, returns the 200 status with ready.
cURL example
curl -s "http://localhost:9010/ready"

​
Prometheus endpoint

/-/healthy on port 9090 If Prometheus is running, returns the 200 status with Prometheus Server is Healthy..
cURL example
curl -s "http://localhost:9090/-/healthy"

​
Troubleshoot

​
No metrics in the monitoring page

Go to http://localhost:9090/targets to see Prometheus scraping target status. If it fails, check that you can query metrics endpoint from conduktor-console-cortex container. You might also have to configure CDK_SCRAPER_SKIPSSLCHECK or CDK_SCRAPER_CAFILE if conduktor-console is configured with TLS termination.