I started hitting problems when trying Debezium against MySQL v8. When creating the connector:
curl -i -X PUT -H "Accept:application/json" \
-H "Content-Type:application/json" http://localhost:8083/connectors/source-debezium-mysql-00/config \
-d '{
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"database.hostname": "mysql",
"database.port": "3306",
"database.user": "debezium",
"database.password": "dbz",
"database.server.id": "42",
"database.server.name": "asgard",
"table.whitelist": "demo.customers",
"database.history.kafka.bootstrap.servers": "kafka:29092",
"database.history.kafka.topic": "asgard.dbhistory.demo" ,
"include.schema.changes": "true"
}'
I’d get the error
{
"error_code": 400,
"message": "Connector configuration is invalid and contains the following 1 error(s):\nUnable to connect: Public Key Retrieval is not allowed\nYou can also find the above list of errors at the endpoint `/{connectorType}/config/validate`"
}
The fix for this, courtesy of Jiri on the Debezium gitter.im chat room, is to add to the connector configuraton database.allowPublicKeyRetrieval=true
:
curl -i -X PUT -H "Accept:application/json" \
-H "Content-Type:application/json" http://localhost:8083/connectors/source-debezium-mysql-00/config \
-d '{
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"database.hostname": "mysql",
"database.port": "3306",
"database.user": "debezium",
"database.password": "dbz",
"database.server.id": "42",
"database.allowPublicKeyRetrieval":"true",
"database.server.name": "asgard",
"table.whitelist": "demo.customers",
"database.history.kafka.bootstrap.servers": "kafka:29092",
"database.history.kafka.topic": "asgard.dbhistory.demo" ,
"include.schema.changes": "true"
}'
After this the connector was created successfully, but immediately FAILED
with the following error in the Kafka Connect worker log:
org.apache.kafka.connect.errors.ConnectException: Failed to authenticate to the MySQL database at mysql:3306 with user 'debezium'
[…]
com.github.shyiko.mysql.binlog.network.AuthenticationException: Client does not support authentication protocol requested by server; consider upgrading MySQL client
A bit of Googling threw up this issue on GitHub with a solution that worked—add WITH mysql_native_password
to the authentication settings for the Debezium user:
ALTER USER 'debezium'@'%' IDENTIFIED WITH mysql_native_password BY 'dbz';
After that, the connector ran successfully 👍