Amazon MSK is a self-managed service that makes it easy to build and run applications that use Apache Kafka to process streaming data.
It lacks several important Apache Kafka features like Kafka Connect, Kafka Streams, ksqlDB, and is not cloud-native (serverless, like S3 or Kinesis) but is just a provisioned infrastructure
Conduktor, which is running on your computer, has no access by default to MSK. Still, it's possible to connect it to the cluster by using a specialized kafka proxy in-between.
To make it work:
Start a proxy https://github.com/dajudge/kafkaproxy/ on a EC2 instance that has access to the cluster. For instance, using Docker:
$ sudo docker run --net host \-e KAFKAPROXY_HOSTNAME=localhost\-e KAFKAPROXY_BASE_PORT=4000 \-e KAFKAPROXY_BOOTSTRAP_SERVERS=MYBROKER1:9092,broker2:9092,broker3:9092 \dajudge/kafkaproxy:0.0.13
On your local machine, do a ssh-tunnel to this EC2 instance:
$ ssh -i ~/.ssh/ec2-key.pem -N \-L 4000:localhost:4000 \-L 4001:localhost:4001 \-L 4002:localhost:4002 \<ec2instance>
Connect Conduktor using localhost:4000