Skip to main content
Quick navigation

Data masking

Overview

Field level data masking Interceptor masks sensitive fields within messages as they are consumed.

Configuration

The policies will be applied when consuming messages.

KeyTypeDefaultDescription
topicString.*Topics that match this regex will have the Interceptor applied
policiesPolicy listList of your masking policies
errorPolicyStringfail_fetchDetermines the plugin behavior when it can't parse a fetched message: fail_fetch or skip_masking

Policy

KeyTypeDescription
nameStringUnique name to identify your policy
fieldslistList of fields that should be obfuscated with the masking rule. Fields can be in a nested structure with dot .. For example: education.account.username, banks[0].accountNo or banks[*].accountNo
ruleRuleRule
schemaRegistryConfigSchema registryThe schema registry in use.

Rule

KeyTypeDefaultDescription
typeMasking typeMASK_ALLThe type of masking (see below).
maskingCharchar*The character used for masking data.
numberOfCharsnumberNumber of masked characters, required if type != MASK_ALL

Masking type

  • MASK_ALL: all data will be masked
  • MASK_FIRST_N: the first n characters will be masked
  • MASK_LAST_N: the last n characters will be masked

Error policy

You can control the plugin behavior when it can't parse a fetched message through its errorPolicy which can be set to fail_fetch or skip_masking.

The default is fail_fetch. In this mode, the plugin will return a failure to read the batch which the fetch record is part of, effectively blocking any consumer.

In skip_masking mode, if there's a failure to parse a message being fetched (e.g. an encrypted record is read in), then that record is skipped and returned un-masked.

Schema registry

KeyTypeDefaultDescription
typestringCONFLUENTThe type of schema registry to use. Choose CONFLUENT (for Confluent-like schema registries including OSS Kafka) or AWS for AWS Glue schema registries.
additionalConfigsmapAdditional properties that map to specific security-related parameters. For enhanced security, you can hide the sensitive values using environment variables as secrets.​
Confluent LikeConfiguration for Confluent-like schema registries
hoststringURL of your schema registry.
cacheSizestring50Number of schemas that can be cached locally by this Interceptor so that it doesn't have to query the schema registry every time.
AWS GlueConfiguration for AWS Glue schema registries
regionstringThe AWS region for the schema registry, e.g. us-east-1.
registryNamestringThe name of the schema registry in AWS. Leave blank for the AWS default of default-registry.
basicCredentialsstringAccess credentials for AWS.
AWS credentialsAWS credential configuration
accessKeystringThe access key for the connection to the schema registry.
secretKeystringThe secret key for the connection to the schema registry.
validateCredentialsbooltrueThe true or false flag determines whether the credentials provided should be validated when set.
accountIdstringThe Id for the AWS account to use.

If you don't provide a basicCredentials section for the AWS Glue schema registry, the client we use to connect will instead attempt to find the connection information is needs from the environment and the credentials required can be passed this way to Gateway as part of its core configuration. Find out more about setting up AWS.

Read our blog about schema registries.

Limitation for non-structured keys/values

When the message key or value can't be transformed into a JSON-like structure, the whole message won't be displayed.

Example

{
"name": "myFieldLevelDataMaskingPlugin",
"pluginClass": "io.conduktor.gateway.interceptor.FieldLevelDataMaskingPlugin",
"priority": 100,
"config": {
"schemaRegistryConfig": {
"host": "http://schema-registry:8081"
},
"policies": [
{
"name": "Mask password",
"rule": {
"type": "MASK_ALL"
},
"fields": [
"password"
]
},
{
"name": "Mask visa",
"rule": {
"type": "MASK_LAST_N",
"maskingChar": "X",
"numberOfChars": 4
},
"fields": [
"visa"
]
}
]
}
}

Secured schema registry

{
"name": "myFieldLevelDataMaskingPlugin",
"pluginClass": "io.conduktor.gateway.interceptor.FieldLevelDataMaskingPlugin",
"priority": 100,
"config": {
"schemaRegistryConfig": {
"host": "http://schema-registry:8081",
"additionalConfigs": {
"schema.registry.url": "${SR_URL}",
"basic.auth.credentials.source": "${SR_BASIC_AUTH_CRED_SRC}",
"basic.auth.user.info": "${SR_BASIC_AUTH_USER_INFO}"
}
},
"policies": [
{
"name": "Mask password",
"rule": {
"type": "MASK_ALL"
},
"fields": [
"password"
]
},
{
"name": "Mask visa",
"rule": {
"type": "MASK_LAST_N",
"maskingChar": "X",
"numberOfChars": 4
},
"fields": [
"visa"
]
}
]
}
}