> ## Documentation Index
> Fetch the complete documentation index at: https://docs.conduktor.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Gateway service accounts and ACLs

> Manage Gateway service accounts and Kafka ACLs using the Conduktor CLI. Create local service accounts, assign permissions.

## Manage a local service account

In this tutorial, you'll learn how to manage service accounts on Gateway. We'll cover the **how to create both local and external service accounts** and how to **assign ACLs to them**. In our scenario:

* The SASL\_PLAINTEXT security protocol is used for communication between Clients > Gateway > Kafka
* The ACLs are enabled on the *passthrough* Virtual Cluster (`GATEWAY_ACL_ENABLED: true`)
* The ACLs super-user is called `GATEWAY_SUPER_USERS: local-acl-admin`
* The Gateway API admin credentials are the defaults

In this guide we'll use the [Gateway API](https://developers.conduktor.io/?product=gateway) <Icon icon="up-right-from-square" /> to create and manage service accounts but these instructions will also work with [Conduktor CLI](/guide/conduktor-in-production/automate/cli-automation).

<Info>
  For local deployments, the Gateway API documentation is available at [`http://localhost:8888`](http://localhost:8888). In this guide, we'll use the `service-account` and the `token` endpoints.
</Info>

In the `service-account` section of the Gateway API documentation, you'll notice that to create a service account on Gateway, you have to chose between a `local` or `external` service account.

A `local` service account is managed by Gateway itself, while an `external` service account is managed by an external OIDC identity provider.

[Find out more about Gateway authentication and authorization](/guide/conduktor-concepts/gateway-authentication-authorization).

### Prerequisites

* Docker and Docker Compose installed
* A valid Conduktor Gateway license key — set it as an environment variable before starting:

```bash theme={null}
export GATEWAY_LICENSE_KEY="your-license-key"
```

### 1. Start the local deployment

Here's the `docker-compose` file to start a local Gateway with the above configuration.

<Tabs>
  <Tab title="Start the environment">
    ```bash theme={null}
    docker compose -f docker-compose.yaml up -d
    ```
  </Tab>

  <Tab title="docker-compose.yaml">
    ```yaml title="docker-compose.yml" theme={null}
    services:
      conduktor-gateway:
        image: conduktor/conduktor-gateway:latest
        hostname: conduktor-gateway
        container_name: conduktor-gateway
        ports:
          - 8888:8888
          - 6969-6971:6969-6971
        environment:
          # Gateway > Kafka connection
          KAFKA_BOOTSTRAP_SERVERS: kafka:9092
          KAFKA_SASL_MECHANISM: PLAIN
          KAFKA_SECURITY_PROTOCOL: SASL_PLAINTEXT
          KAFKA_SASL_JAAS_CONFIG: org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";
          # Clients > Gateway connection
          GATEWAY_SECURITY_MODE: GATEWAY_MANAGED
          GATEWAY_LISTENER_DEFAULT_SECURITY_PROTOCOL: SASL_PLAINTEXT
          GATEWAY_LISTENER_DEFAULT_ROUTING: port
          GATEWAY_LISTENER_DEFAULT_PORTS: 6969-6971
          GATEWAY_LISTENER_DEFAULT_ADVERTISED_HOST: localhost # Considering your clients are running on your machine, outside of the Docker network
          # GATEWAY_OAUTH_JWKS_URL: "TO_FILL"
          # GATEWAY_OAUTH_EXPECTED_ISSUER: "TO_FILL"
          # GATEWAY_OAUTH_EXPECTED_AUDIENCES: "TO_FILL"
          # Gateway configuration
          GATEWAY_MIN_BROKERID: 1
          # Enable ACLs on the passthrough virtual cluster, with the super user
          GATEWAY_ACL_ENABLED: true
          GATEWAY_SUPER_USERS: local-acl-admin
          GATEWAY_USER_POOL_SECRET_KEY: "Y2hhbmdlbWUtc2VjcmV0LWtleS1mb3ItbG9jYWwtZGV2" # base64-encoded secret for local dev only
          GATEWAY_LICENSE_KEY: ${GATEWAY_LICENSE_KEY}
        healthcheck:
          test: curl -s http://localhost:8888/health | grep -q UP || exit 1
          start_period: 10s
          interval: 5s
          retries: 25
        depends_on:
          kafka: { condition: service_healthy }

      kafka:
        image: confluentinc/cp-kafka:7.7.0
        container_name: kafka
        hostname: kafka
        ports:
          - 19092:19092
        environment:
          CLUSTER_ID: MkU3OEVBNTcwNTJENDM2Qk
          KAFKA_NODE_ID: 1
          KAFKA_PROCESS_ROLES: broker,controller
          KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka:9093"
          KAFKA_LISTENERS: "INTERNAL://kafka:9092,EXTERNAL://:19092,CONTROLLER://kafka:9093"
          KAFKA_ADVERTISED_LISTENERS: "INTERNAL://kafka:9092,EXTERNAL://localhost:19092"
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:SASL_PLAINTEXT,EXTERNAL:PLAINTEXT,CONTROLLER:SASL_PLAINTEXT"
          KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
          KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
          KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
          KAFKA_SASL_MECHANISM_CONTROLLER_PROTOCOL: PLAIN
          KAFKA_LISTENER_NAME_INTERNAL_SASL_ENABLED_MECHANISMS: PLAIN
          KAFKA_LISTENER_NAME_CONTROLLER_SASL_ENABLED_MECHANISMS: PLAIN
          KAFKA_LISTENER_NAME_INTERNAL_PLAIN_SASL_JAAS_CONFIG: "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-secret\" user_admin=\"admin-secret\";"
          KAFKA_LISTENER_NAME_CONTROLLER_PLAIN_SASL_JAAS_CONFIG: "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-secret\" user_admin=\"admin-secret\";"
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
          KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
          KAFKA_AUTHORIZER_CLASS_NAME: org.apache.kafka.metadata.authorizer.StandardAuthorizer
          KAFKA_SUPER_USERS: "User:admin;User:ANONYMOUS"
          KAFKA_HEAP_OPTS: "-Xmx512M -Xms512M"
        healthcheck:
          test: nc -zv kafka 9092 || exit 1
          interval: 5s
          retries: 25
          start_period: 20s
    ```
  </Tab>
</Tabs>

### 2. Export Java security manager config (optional)

Depending on your version of Java, you may have to run the below command in your shell session.

Newer versions of Java don't support security manager and current versions of Kafka CLI commands will fail without this being set.

If you get errors when running the later commands with authentication, run this command:

```bash theme={null}
export KAFKA_OPTS="-Djava.security.manager=allow"  
```

### 3. Create a few topics on Kafka

Let's create a few topics in Kafka by running the following command:

```bash theme={null}
kafka-topics --create --bootstrap-server localhost:19092 --topic finance-data
kafka-topics --create --bootstrap-server localhost:19092 --topic finance-report
```

### 4. Create a local service account

A local service account is managed by Gateway itself. This means that we have to ask Gateway to create it for us by giving it a name.

The first step is to **create a reference to this new service account**. In our case, we'll call this service account `local-app-finance-dev` and we want it to exist in the *passthrough* Virtual Cluster:

```bash title="Create the service account" theme={null}
curl \
  --request PUT \
  --url 'http://localhost:8888/gateway/v2/service-account' \
  --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "kind" : "GatewayServiceAccount",
    "apiVersion" : "gateway/v2",
    "metadata" : {
      "name" : "local-app-finance-dev",
      "vCluster" : "passthrough"
    },
    "spec" : { "type" : "LOCAL" }
  }'
```

Then, we need to **get the secret key of this service account**, that has a limited lifetime:

```bash title="Get the secret key" theme={null}
curl \
  --request POST \
  --url 'http://localhost:8888/gateway/v2/token' \
  --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "vCluster": "passthrough",
    "username": "local-app-finance-dev",
    "lifeTimeSeconds": 3600000
  }'
```

This will return a JSON object with the `token` field containing the secret key.

```json title="Response" theme={null}
{
  "token": "eyJhbGciOiJIUzI1NiJ9.eyJ1c2VybmFtZSI6ImxvY2FsLWFwcC1maW5hbmNlLWRldiIsInZjbHVzdGVyIjoicGFzc3Rocm91Z2giLCJleHAiOjE3MzIwOTUzNjN9.-rivmwcI-zvTTqLPeO_0l3xUALz5mKtopp1YMaTswFk"
}
```

This means that, we can now connect to Gateway *passthrough* Virtual Cluster using the `local-app-finance-dev` service account and its secret key.

### 5. Connect to Gateway

Create a properties file `local-client.properties` with the credentials we've just generated to connect to Gateway:

```properties title="local-client.properties" theme={null}
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="local-app-finance-dev" password="eyJhbGciOiJIUzI1NiJ9.eyJ1c2VybmFtZSI6ImxvY2FsLWFwcC1maW5hbmNlLWRldiIsInZjbHVzdGVyIjoicGFzc3Rocm91Z2giLCJleHAiOjE3MzIwOTUzNjN9.-rivmwcI-zvTTqLPeO_0l3xUALz5mKtopp1YMaTswFk";
```

List topics using the Kafka CLI, authenticating using our service account:

```bash title="List topics" theme={null}
kafka-topics --list --bootstrap-server localhost:6969 --command-config local-client.properties
```

In this case, the command doesn't return anything because we have enabled ACLs on this *passthrough* Virtual Cluster (`GATEWAY_ACL_ENABLED: true`). It means that my local service account doesn't have the right permissions to see any resources, it's not authorized.

Let's modify the ACLs, so that this service account can list topics.

### 6. Create an ACL admin

To modify the ACLs, we recommend that you define a dedicated **ACL admin service account**.

This is a privileged service account and has to be defined in the Gateway configuration using the environment variable **`GATEWAY_SUPER_USERS`** in the case of the *passthrough* Virtual Cluster.

In our example, we've called it `local-acl-admin`.

Repeat the steps as before, using the name `local-acl-admin`. Create the service account, get its credentials and save them to file.

<Tabs>
  <Tab title="Create the service account">
    ```bash theme={null}
    curl \
      --request PUT \
      --url 'http://localhost:8888/gateway/v2/service-account' \
      --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
      --header 'Content-Type: application/json' \
      --data-raw '{
        "kind" : "GatewayServiceAccount",
        "apiVersion" : "gateway/v2",
        "metadata" : {
          "name" : "local-acl-admin",
          "vCluster" : "passthrough"
        },
        "spec" : { "type" : "LOCAL" }
      }'
    ```
  </Tab>

  <Tab title="Get its credentials">
    ```bash theme={null}
    curl \
      --request POST \
      --url 'http://localhost:8888/gateway/v2/token' \
      --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
      --header 'Content-Type: application/json' \
      --data-raw '{
        "vCluster": "passthrough",
        "username": "local-acl-admin",
        "lifeTimeSeconds": 3600000
      }'
    ```
  </Tab>
</Tabs>

Store the generated credentials in a new file, `local-acl-admin.properties`.

```properties title="local-acl-admin.properties" theme={null}
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="local-acl-admin" password="eyJhbGciOiJIUzI1NiJ9.eyJ1c2VybmFtZSI6ImxvY2FsLWFjbC1hZG1pbiIsInZjbHVzdGVyIjoicGFzc3Rocm91Z2giLCJleHAiOjE3MzIxNjEwOTB9.m8U_DVv4MTOY9mKiKY2tHeUGjxsUvhC9ssE6iAI3eJc";
```

As this user is an ACLs admin, they have access to all the Gateway topics and can create and modify ACLs for the other service accounts.

### 7. Create ACLs for other local service accounts

Use the **ACL admin service account** to create ACLs for other local service accounts.

In order for the `local-app-finance-dev` service account to be able to interact with its topics, we have to give it the `WRITE` permission on its prefix. Run the following command to do so:

```bash title="Give ACLs to local-app-finance-dev" theme={null}
kafka-acls --bootstrap-server localhost:6969 \
  --command-config local-acl-admin.properties \
  --add \
  --allow-principal User:local-app-finance-dev \
  --operation write \
  --topic finance- \
  --resource-pattern-type prefixed
```

```bash title="Response" theme={null}
Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=finance-, patternType=PREFIXED)`:
    (principal=User:local-app-finance-dev, host=*, operation=WRITE, permissionType=ALLOW)

Current ACLs for resource `ResourcePattern(resourceType=TOPIC, name=finance-, patternType=PREFIXED)`:
    (principal=User:local-app-finance-dev, host=*, operation=WRITE, permissionType=ALLOW)
```

Finally, let's list the topics using the `local-app-finance-dev` service account:

```bash title="List topics as local-app-finance-dev" theme={null}
kafka-topics --list --bootstrap-server localhost:6969 --command-config local-client.properties
```

```bash title="Response" theme={null}
finance-data
finance-report
```

## Manage an external service account

An external service account is managed by an external OIDC identity provider. This means we only have to make Gateway aware of this external service account by giving it its OIDC principal (this is the `externalNames`). The credentials that will be used by this application are already defined in the OIDC identity provider.

To follow these steps on your machine, you'll need to have an OAUTHBEARER provider connected in the config of the Docker Compose you're using, otherwise use it as a reference.

To create this external service account reference on Gateway, you can run the following command to create a Gateway service account called `azure-app-billing-dev` which is recognized by it's OIDC principal (`"externalNames" : [ "TO_FILL" ]`):

```bash title="Create the service account" theme={null}
curl \
  --request PUT \
  --url 'http://localhost:8888/gateway/v2/service-account?dryMode=false' \
  --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "kind" : "GatewayServiceAccount",
    "apiVersion" : "gateway/v2",
    "metadata" : {
      "name" : "azure-app-billing-dev",
      "vCluster" : "passthrough"
    },
    "spec" : {
      "type" : "EXTERNAL",
      "externalNames" : [ "TO_FILL" ]
    }
  }'
```

Now you can apply some interceptors to this service account, by referring to the service account name, `azure-app-billing-dev`.

<Info>
  When configuring Interceptor scopes for external service accounts, always use the internal name (`azure-app-billing-dev` in this case), **not** the external name. The external name is only used for mapping external identities to Gateway service accounts.
</Info>

### 1. Connect to Gateway

You can now connect to Gateway using the `azure-app-billing-dev` service account.

Here's the type of properties file you may use to connect to Gateway using OAUTHBEARER:

```properties title="external-client.properties" theme={null}
security.protocol=SASL_PLAINTEXT
sasl.mechanism=OAUTHBEARER
sasl.login.callback.handler.class=org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerLoginCallbackHandler
sasl.oauthbearer.token.endpoint.url="TO_FILL"
sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required clientId="TO_FILL" clientSecret="TO_FILL" scope=".default";
```

And here's an example of using the Kafka CLI to list the topics, using this service account:

```bash title="List topics" theme={null}
kafka-topics --list --bootstrap-server localhost:6969 --command-config external-client.properties
```

In this case, the command wouldn't return anything because we have enabled the ACLs on this passthrough virtual cluster (`GATEWAY_ACL_ENABLED: true`). It means that my local service account doesn't have the right permissions to see any resources, it's not authorized. The next step is then to give it some ACLs so it can list topics.

### 2. Create ACLs for an external service account

The steps here are exactly the same as the ones for the [local service account](#6-create-acls-for-other-local-service-accounts).

Follow those instructions but use the `azure-app-billing-dev` service account (instead of *local-app-finance-dev*).

```bash title="Give ACLs to azure-app-billing-dev" theme={null}
kafka-acls --bootstrap-server localhost:6969 \
  --command-config local-acl-admin.properties \
  --add \
  --allow-principal User:azure-app-billing-dev \
  --operation write \
  --topic finance- \
  --resource-pattern-type prefixed
```

#### Differences if using Virtual Clusters

The example above is using a default `passthrough` Virtual Cluster. If you are using your own Virtual Clusters, you need to make a few changes.

First, let's see how to create a Virtual Cluster with the ACLs enabled, and a super user declared. Then, we'll see how to create the super user credentials, in order to give permissions to the applications service account.

### 3. Create the Virtual Cluster with an ACL admin

The below creates a Virtual Cluster called `my-vcluster` that will have ACLs enabled and a super user named `local-acl-admin`.

```bash title="Create a Virtual Cluster with ACLs Enabled" theme={null}
curl \
  --request PUT \
  --url 'http://localhost:8888/gateway/v2/virtual-cluster?dryMode=false' \
  --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "kind" : "VirtualCluster",
  "apiVersion" : "gateway/v2",
  "metadata" : { "name" : "my-vcluster" },
  "spec" : {
    "aclEnabled" : true,
    "superUsers" : [ "local-acl-admin" ]
  }
}'
```

### 4. Creating a service account in a Virtual Cluster

Now that the Virtual Cluster `my-vcluster` exists, create the local service account for the super user:

```bash title="Create the Service Account" theme={null}
curl \
  --request PUT \
  --url 'http://localhost:8888/gateway/v2/service-account' \
  --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "kind" : "GatewayServiceAccount",
    "apiVersion" : "gateway/v2",
    "metadata" : {
      "name" : "local-acl-admin",
      "vCluster" : "my-vcluster"
    },
    "spec" : { "type" : "LOCAL" }
  }'
```

Finally, get its secret key:

```bash title="Get the secret key" theme={null}
curl \
  --request POST \
  --url 'http://localhost:8888/gateway/v2/token' \
  --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "vCluster": "my-vcluster",
    "username": "local-acl-admin",
    "lifeTimeSeconds": 3600000
  }'
```

The same modification applies for external service accounts.

You can now create a properties file `local-acl-admin.properties` using the credentials you've just generated.

## Auto-create topic authorization

When `GATEWAY_AUTO_CREATE_TOPICS_ENABLED` is set to `true`, users require specific ACL permissions to automatically create topics when producing or consuming through Gateway.

### Required permissions

You have to have one of the following ACL permissions to create topics automatically:

1. **CLUSTER resource with CREATE operation** - allows creating any topic
2. **TOPIC resource with CREATE operation** - allows creating specific topics

### Example ACLs for auto-create topics

#### Allow creating any topic (CLUSTER level)

```bash title="Allow creating any topic" theme={null}
kafka-acls --bootstrap-server localhost:6969 \
  --command-config local-acl-admin.properties \
  --add \
  --allow-principal User:local-app-finance-dev \
  --operation create \
  --cluster
```

#### Allow creating specific topics (TOPIC level)

```bash title="Allow creating specific topics" theme={null}
kafka-acls --bootstrap-server localhost:6969 \
  --command-config local-acl-admin.properties \
  --add \
  --allow-principal User:local-app-finance-dev \
  --operation create \
  --topic finance- \
  --resource-pattern-type prefixed
```

<Warning>
  When auto-create topics is enabled, concentrated topics are not supported. Topics that would normally be concentrated will be created as physical topics instead.

  Auto-create topics is disabled by default (`GATEWAY_AUTO_CREATE_TOPICS_ENABLED=false`).
</Warning>

## Related resources

* [Find out more about Gateway service accounts](/guide/conduktor-concepts/gateway-service-accounts)
* [Find out more about Gateway authentication](/guide/conduktor-concepts/gateway-authentication-authorization)
* [Give us feedback/request a feature](https://conduktor.io/roadmap) <Icon icon="up-right-from-square" />