SQL. Three simple letters.
Ess Queue Ell.
/ˌɛs kjuː ˈɛl/.
In the data world they bind us together, yet separate us.
As the saying goes, England and America are two countries divided by the same language, and the same goes for the batch and streaming world and some elements of SQL.
In my last post I wrote all about watermarks in Apache Flink. As you’ll have realised—or been aware of already—it can get fairly hairy. That’s why I’m keen to see how Confluent Cloud for Apache Flink deals with watermarks, to see if it makes life any easier for the developer.
Whether you’re processing data in batch or as a stream, the concept of time is an important part of accurate processing logic.
Because we process data after it happens, there are a minimum of two different types of time to consider:
When it happened, known as Event Time
When we process it, known as Processing Time (or system time or wall clock time)
