Jul 17, 2020

Learning Golang (some rough notes) - S02E08 - Checking Kafka advertised.listeners with Go

At the beginning of all this my aim was to learn something new (Go), and use it to write a version of a utility that I’d previously hacked together in Python that checks your Apache Kafka broker configuration for possible problems with the infamous advertised.listeners setting. Check out a blog that I wrote which explains all about Apache Kafka and listener configuration. You can find the code at https://github.com/rmoff/kafka-listeners It’s been a fun journey, and now I am pleased to be able to show the results of it.

Jul 16, 2020

Learning Golang (some rough notes) - S02E07 - Splitting Go code into separate source files and building a binary executable

So far I’ve been running all my code either in the Go Tour sandbox, using Go Playground, or from a single file in VS Code. My explorations in the previous article ended up with a a source file that was starting to get a little bit unwieldily, so let’s take a look at how that can be improved. Within my most recent code, I have the main function and the doProduce function, which is fine when collapsed down:

Jul 15, 2020

Learning Golang (some rough notes) - S02E06 - Putting the Producer in a function and handling errors in a Go routine

When I set out to learn Go one of the aims I had in mind was to write a version of this little Python utility which accompanies a blog I wrote recently about understanding and diagnosing problems with Kafka advertised listeners. Having successfully got Producer, Consumer, and AdminClient API examples working, it is now time to turn to that task.

Jul 15, 2020

Learning Golang (some rough notes) - S02E05 - Kafka Go AdminClient

Having ticked off the basics with an Apache Kafka producer and consumer in Go, let’s now check out the AdminClient. This is useful for checking out metadata about the cluster, creating topics, and stuff like that.

Jul 14, 2020

Learning Golang (some rough notes) - S02E04 - Kafka Go Consumer (Function-based)

Last time I looked at creating my first Apache Kafka consumer in Go, which used the now-deprecated channel-based consumer. Whilst idiomatic for Go, it has some issues which mean that the function-based consumer is recommended for use instead. So let’s go and use it! Instead of reading from the Events() channel of the consumer, we read events using the Poll() function with a timeout. The way we handle events (a switch based on their type) is the same:

Jul 14, 2020

Learning Golang (some rough notes) - S02E03 - Kafka Go Consumer (Channel-based)

Having written my first Kafka producer in Go, and even added error handling to it, the next step was to write a consumer. It follows closely the pattern of Producer code I finished up with previously, using the channel-based approach for the Consumer:

Jul 10, 2020

Learning Golang (some rough notes) - S02E02 - Adding error handling to the Producer

I looked last time at the very bare basics of writing a Kafka producer using Go. It worked, but only with everything lined up and pointing the right way. There was no error handling of any sorts. Let’s see about fixing this now.

Jul 8, 2020

Learning Golang (some rough notes) - S02E01 - My First Kafka Go Producer

Jul 8, 2020

Learning Golang (some rough notes) - S02E00 - Kafka and Go

With the first leg of my journey with Go done (starting from a very rudimentary base), the next step for me was to bring it into my current area of interest and work - Apache Kafka.

Jul 3, 2020

Learning Golang (some rough notes) - S01E10 - Concurrency (Web Crawler)

👉 A Tour of Go : sync.Mutex

In the previous exercise I felt my absence of a formal CompSci background with the introduction of Binary Sorted Trees, and now I am concious of it again with learning about mutex. I’d heard of them before, mostly when Oracle performance folk were talking about wait types - TIL it stands for mutual exclusion!

rmoff’s random ramblings

✨ Data Engineering, Kafka, and other random geekery 🤓