Kafka Connect CLI tricks

I do lots of work with Kafka Connect, almost entirely in Distributed mode—even just with 1 node -> makes scaling out much easier when/if needed. Because I’m using Distributed mode, I use the Kafka Connect REST API to configure and manage it. Whilst others might use GUI REST tools like Postman etc, I tend to just use the commandline. Here are some useful snippets that I use all the time.

I’m showing the commands split with a line continuation character (\) but you can of course run them on a single line. You might also choose to get fancy and set the Connect host and port as environment variables etc, but I leave that as an exercise for the reader :)

List all running connectors: 🔗

curl -s "http://localhost:8083/connectors"| \
jq '.[]'| \
xargs -I{connector_name} curl -s "http://localhost:8083/connectors/"{connector_name}"/status"| jq -c -M '[.name,.connector.state,.tasks[].state]|join(":|:")'| \
column -s : -t| \
sed 's/\"//g'| \
sort

Selectively delete a connector 🔗

(h/t to @madewithtea for the idea of using peco)

curl -s "http://localhost:8083/connectors"| \
jq '.[]'| \
peco | \
xargs -I{connector_name} curl -s -XDELETE "http://loc
alhost:8083/connectors/"{connector_name}

Delete all connectors 🔗

CAUTION with this one, as it will delete all your connectors!

curl -s "http://localhost:8083/connectors"| \
jq '.[]'| \
xargs -I{connector_name} curl -s -XDELETE "http://localhost:8083/connectors/"{connector_name}