The Replicat was kapput: GGSCI (localhost.localdomain) 3> info rkconnoe REPLICAT RKCONNOE Last Started 2017-09-12 17:06 Status ABENDED Checkpoint Lag 00:00:00 (updated 00:46:34 ago) Log Read Checkpoint File /u01/app/ogg/dirdat/oe000000 First Record RBA 0 So checking the OGG error log ggserr.log showed 2017-09-12T17:06:17.572-0400 ERROR OGG-15051 Oracle GoldenGate Delivery, rkconnoe.prm: Java or JNI exception: oracle.goldengate.util.GGException: Error detected handling operation added event. 2017-09-12T17:06:17.572-0400 ERROR OGG-01668 Oracle GoldenGate Delivery, rkconnoe.prm: PROCESS ABENDING. So checking the replicat log dirrpt/RKCONNOE_info_log4j.

tl;dr Make sure that key.converter.schema.registry.url and value.converter.schema.registry.url are specified, and that there are no trailing whitespaces. I’ve been building on previous work I’ve done with Oracle GoldenGate and Kafka Connect, looking at how to have the change records from the Oracle database come through to Kafka in Avro format rather than the default JSON that the sample configuration gives. Simply changing the Kafka Connect OGG configuration file (confluent.properties) from value.converter=org.apache.kafka.connect.json.JsonConverter key.

I’ve been doing some noodling around with Confluent’s Kafka Connect recently, as part of gaining a wider understanding into Kafka. If you’re not familiar with Kafka Connect this page gives a good idea of the thinking behind it. One issue that I hit defeated my Google-fu so I’m recording it here to hopefully help out fellow n00bs. The pipeline that I’d set up looked like this: Eneco’s Twitter Source streaming tweets to a Kafka topic Confluent’s HDFS Sink to stream tweets to HDFS and define Hive table automagically over them It worked great, but only if I didn’t enable the Hive integration part.

The Oracle by Example (ObE) here demonstrating how to use Goldengate to replicate transactions big data targets such as HDFS is written for the BigDataLite 4.2.1, and for me didn’t work on the current latest version, 4.4.0. The OBE (and similar Hands On Lab PDF) assume the presence of pmov.prm and pmov.properties in /u01/ogg/dirprm/. On BDL 4.4 there’s only the extract to from Oracle configuration, emov. Fortunately it’s still possible to run this setup out of the box in BDL 4.