Release cadence
Conduktor provides a new release roughly every month. We suggest that you upgrade no more than two versions apart at a time. To get notified when a new release comes out, subscribe to the release notes RSS feed.Overview
Each Conduktor component follows an independent release cycle, so versioning is separate for Helm charts, Console/Gateway (the Docker image which includes the related API) and the Terraform provider. All the components follow the semantic versioning of:major.minor.patch.
Before any upgrade we strongly recommend reviewing Conduktor release notes to ensure that there are no breaking changes that might impact your deployment.
Upgrade Console
- Docker
- Kubernetes
For Docker deployments, we recommend to:
- Back up your database before proceeding.
- Upgrade the Docker image version to the most recent one.
- Re-deploy Console with the updated image version.
Upgrade Gateway
- Docker
- Kubernetes
For Docker deployments, we recommend to:
- Upgrade the Docker image version to the most recent one.
- Re-deploy Gateway with the updated image version.
Terraform compatibility
The Terraform provider compatibility is managed at the resource level, not the overall provider or Console version. New resources may require newer Console API versions. Compatibility details are:- in each Terraform resource’s documentation
- as inline warnings/errors when running
terraform apply.
Migrate Gateway API from v1 to v2
Gateway v3.3.0 introduces API v2 to enhance user experience when interacting with Gateway resources. When migrating from API v1 to v2, your Gateway resources will be preserved (you can interact with old resources via the v2 API) and only minimal attention is required for two resources: Virtual Clusters and Gateway service account.Virtual Clusters
When migrating to v2:- the boolean value for
aclEnabledwill be automatically derived from the previously usedGATEWAY_ACL_STORE_ENABLEDvariable but - you have to populate the
superUserslist, as it won’t be done automatically when migrating.
Gateway service account
When migrating to v2, user mappings become GatewayServiceAccount. Every existing user mapping is migrated to v2 asEXTERNAL GatewayServiceAccount.
If you were using LOCAL users (and generating tokens) with the v1 API, you have to:
- Declare LOCAL users in the v2 API.
- Generate tokens.
Migrate Gateway service accounts storage
From Gateway v3.23.0, service accounts are stored in the_conduktor_${GATEWAY_CLUSTER_ID}_serviceaccounts topic instead of _conduktor_${GATEWAY_CLUSTER_ID}_usermappings. The migration is automatic.
What happens on upgrade
- At startup, before it serves traffic, Gateway copies every service account from
_usermappingsto_serviceaccounts. It never overwrites a record that already exists. - Gateway then writes a completion marker in
_serviceaccounts. Later startups see the marker and skip the migration. - Several Gateway nodes can run the migration at the same time, and a migration interrupted part-way through is safe. The result is the same.
- If the migration fails, Gateway stops instead of serving traffic. Fix the cause and restart Gateway: the migration runs again from the start.
- Gateway never writes to or deletes
_usermappings. It’s kept so that you can roll back. At each startup, Gateway logs that a non-empty_usermappingstopic can be deleted: delete it only once you no longer need to roll back. - SCRAM credentials are always available. Gateway ignores the
GATEWAY_FEATURE_FLAGS_SERVICE_ACCOUNTSvariable, but keep it set if you use SCRAM and might roll back.
During a rolling upgrade
Don’t create, update or delete service accounts or group memberships until every Gateway node runs v3.23.0. A change made through an earlier node isn’t migrated, and the upgraded nodes don’t see it. A group membership removed through an upgraded node still applies on the earlier nodes.Roll back
You can roll back to an earlier Gateway version. Client traffic keeps working if:- the
_usermappingstopic still exists. Without it, the earlier version has no service accounts. - the earlier version runs with
GATEWAY_FEATURE_FLAGS_SERVICE_ACCOUNTS=true, if you use SCRAM. Without it, the earlier version doesn’t read_serviceaccountsand every SCRAM login fails.
_usermappings, which the upgrade didn’t change, so service accounts return to their state at upgrade time:
- Service accounts created after the upgrade disappear. Their SCRAM credentials remain, but their SCRAM logins fail until you re-create the accounts.
- Service accounts deleted after the upgrade come back and can authenticate again. Their SCRAM credentials stay deleted.
- Group memberships added after the upgrade are kept. Memberships removed after the upgrade can apply again.
_serviceaccounts record it can’t read. These errors are expected and harmless.
Upgrade again after a rollback
The migration doesn’t run again, because the completion marker is already in_serviceaccounts. Changes made while rolled back aren’t migrated. You can either redo these changes, or run the migration again.
Redo the changes made while rolled back
After the upgrade, redo the changes through the API. If you manage service accounts with the Conduktor CLI or Terraform, apply your configuration again.- Re-create the service accounts created while rolled back.
- Delete again the service accounts deleted while rolled back. Until you do, they can authenticate.
- Apply again the changes made to service accounts while rolled back.
Run the migration again
To copy_usermappings again as it is now, delete the _serviceaccounts topic before the upgrade:
- Stop every Gateway node.
- Delete the
_conduktor_${GATEWAY_CLUSTER_ID}_serviceaccountstopic. - Wait until the topic no longer appears in the topic list. Kafka deletes topics asynchronously.
- Start the Gateway nodes on v3.23.0. Gateway creates the topic again and runs the migration.