Architecturally, OBIEE 12c is - on the surface - pretty similar to OBIEE 11g. Sure, we’ve lost OPMN in favour of Node Manager, but all the old favourites are there - WebLogic Servers, BI Server (nqsserver / OBIS), Presentation Services (sawserver / OBIPS), and so on.

But, scratch beneath the surface, or have a gander at slide decks such as this one from BIWA this year, and you realise that change is afoot. Whilst the OBIEE core is still built around proprietary ‘black box’ protocols (SAW from analytics to sawserver on port 9710, NQS ODBC from sawserver to nqsserver, cluster management on 9706 to nqsclustercontroller), there are now REST-based web services springing up (in addition to the existing SOAP services that have been there since at least 10g). Whilst the REST services are there under the covers, they are not documented nor user-servicable, but they are there. But let me re-iterate:

I wrote recently about Kibana’s excellent Timelion feature, which brings time-series visualisations to Kibana. In the comments Ben Huang asked:

do you know how to show whats the difference between this Friday and last Friday by Timelion?

So I thought I’d answer properly here.

Timelion includes mathematical functions including add and subtract, as well as the ability to show data offset by an amount of time. So to answer Ben’s query, we combine the two.

Running the OBIEE 12c startup on Windows:

C:\app\oracle\fmw\user_projects\domains\bi\bitools\bin\start.cmd

Just hangs at:

Starting AdminServer ...

No CPU being consumed, very odd. But then … looking at DOMAIN_HOME\servers\AdminServer\logs\AdminServer.out shows the last log entry was:

Enter username to boot WebLogic server:

And that’s bad news, cos that’s an interactive prompt, but not echo’d to the console output of the startup command, and there’s no way to interact with it.

The start.cmd was being called by adding it to the Startup folder (C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp), and I guess it was something about this that stopped the prompt coming back to the console, because when I ran it manually from the command prompt, I got this:

Another quick note on OBIEE 12c, this time on the importServiceInstance command. If you run it with a BAR file that doesn’t exist, it’ll fail (obviously), but the error at the end of the stack trace is slightly confusing:

oracle.bi.bar.exceptions.UnSupportedBarException: 
The Bar file provided as input is not supported in this BI Platfrom release.

Scrolling back up the stack trace does show the error message:

SEVERE: Failed in reading bar file. [...]
java.io.FileNotFoundException: [...] 
(The system cannot find the file specified)

So … RTEM (Read the Fantastic Error Message) in full, don’t just skim to the end…

Interesting quirk in running Baseline Validation Tool for OBIEE here. If you invoke obibvt from the bin folder, it errors with Parameter ‘directory’ is not a directory

PS C:\OracleBI-BVT> cd bin
PS C:\OracleBI-BVT\bin> .\obibvt -config C:\OracleBI-BVT\bin\bvt-config.xml -deployment current
 INFO: Result folder: Results\current
Throwable: Parameter 'directory' is not a directory
Thread[main,5,main]
SEVERE: Unhandled Exception
SEVERE: java.lang.IllegalArgumentException: Parameter 'directory' is not a directory
       at org.apache.commons.io.FileUtils.validateListFilesParameters(FileUtils.java:545)
       at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:521)
       at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:691)
       at com.oracle.biee.bvt.UpgradeTool.loadPlugins(UpgradeTool.java:537)
       at com.oracle.biee.bvt.UpgradeTool.runPluginTests(UpgradeTool.java:644)
       at com.oracle.biee.bvt.UpgradeTool.run(UpgradeTool.java:812)
       at com.oracle.biee.bvt.UpgradeTool.main(UpgradeTool.java:999)

PS C:\OracleBI-BVT\bin>

Solution? Run the exact same command, but from the folder above:

Yesterday I wrote about Monitoring Logstash Ingest Rates with InfluxDB and Grafana, in which InfluxDB provided the data store for the ingest rate data, and Grafana the frontend.

Mark Walkom reminded me on twitter that the next release of Logstash will add more functionality in this area - and that it’ll integrate back into the Elastic stack:

@rmoff nice, LS 5.0 will have APIs exposing metrics too. they’ll be integrated back into Marvel/Monitoring! :)

In this article I’m going to show you how to easily monitor the rate at which Logstash is ingesting data, as well as in future articles the rate at which Elasticsearch is indexing it. It’s a nice little touch to add to any project involving Logstash, and it’s easy to do.

Logstash is powerful tool for data ingest, processing, and distribution. It originated as simply the pipe to slurp at log files and put them into Elasticsearch, but has evolved into a whole bunch more. With connectors to JDBC and Kafka, as well as many other input and output options (not to mention the filtering possibilities), it really is a great bit of software to use. I’ve used it over the years with OBIEE, as well as more recently to pull data from Oracle, and even IRC. Another great set of tools is InfluxDB and Grafana, which for me really round off the standalone Elastic platform (previously known as ELK - Elasticsearch, Logstash, and Kibana). What InfluxDB and Grafana give is a powerful dedicated time series database and flexible time series-based dashboarding tool respectively. A topic for another day is the Elasticsearch vs InfluxDB overlap, and Kibana vs Grafana - but for now, just take it as read that it’s horses for course, right tool for the right job, etc.

Here’s a collection of useful articles that I’ve found over the years that give good advice on writing a good abstract, mistakes to avoid, etc:

• Adam Machanic - Capturing Attention: Writing Great Session Descriptions
• Gwen Shapira - Concrete Advice for Abstract Writers
• Kellyn Pot’Vin-Gorman - Abstracts, Reviews and Conferences, Oh My!
• Russ Unger - Conference Proposals that Don’t Suck
• Martin Widlake - Tips on Submitting an Abstract to Conference
• Bridget Kromhout - give actionable takeaways

(post photo courtesy of Calum MacAulay on https://unsplash.com)

Kibana is a tool from Elastic that makes analysis of data held in Elasticsearch really easy and very powerful. Because Elasticsearch has very loose schema that can evolve on demand it makes it very quick to get up and running with some cool visualisations and analysis on any set of data. I demonstrated this in a blog post last year, taking a CSV file and loading it into Elasticsearch via Logstash.

This is all great, but the one real sticking point with analytics in Elasticsearch/Kibana is that it needs the data to be denormalised. That is, you can’t give it a bunch of sources of data and it perform the joins for you in Kibana - it just doesn’t work like that. If you’re using Elasticsearch alone for analytics, maybe with a bespoke application, there are ways of approaching it, but not through Kibana. Now, depending on where the data is coming from, this may not be a problem. For example, if you use the JDBC Logstash input to pull from an RDBMS source you can specify a complex SQL query going across multiple tables, so that the data when it hits Elasticsearch is nice and denormalised and ready for fun in Kibana. But, source data doesn’t always come this way, and it’s useful to have a way to work with the data still when it is like this.

Two vulns for OBIEE in the latest critical patch update (CPU): http://www.oracle.com/technetwork/security-advisory/cpuapr2016v3-2985753.html?elq_mid=45463&sh=91225181314122121267715271910&cmid=WWMK10067711MPP001C140

Patches is bundle patch .160419:

• 12.2.1: https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?parent=DOCUMENT&sourceId=2102148.1&patchId=22734181
• 11.1.1.9: https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?parent=DOCUMENT&sourceId=2102148.1&patchId=22393988
• 11.1.1.7: https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?parent=DOCUMENT&sourceId=2102148.1&patchId=22225110

Note that April 2016 is the last regular patchset for 11.1.1.7, ref: https://support.oracle.com/epmos/faces/DocumentDisplay?id=2102148.1#mozTocId410847. If you’re still on it, or earlier, time to upgrade!

(Photo credit: https://unsplash.com/@jenlittlebirdie)

Recently added to the oracledi project over at java.net is an adaptor enabling Oracle GoldenGate (OGG) to send data to Elasticsearch. This adds a powerful alternative to [micro-]batch extract via JDBC from Oracle to Elasticsearch, which I wrote about recently over at the Elastic blog.

Elasticsearch is a ‘document store’ widely used for both search and analytics. It’s something I’ve written a lot about (here and here for archives), as well as spoken about - preaching the good word, as it were, since the Elastic stack as a whole is very very good at what it does and a pleasure to work with. So, being able to combine that with my “day job” focus of Oracle is fun. Let’s get started!

I’ve recently been playing around with the ELK stack (now officially known as the Elastic stack) collecting data from an IRC channel with Elastic’s Logstash, storing it in Elasticsearch and analysing it with Kibana. But, this isn’t an “ELK” post - this is a Kafka post! ELK is just some example data manipulation tooling that helps demonstrate the principles.

As I wrote about last year, Apache Kafka provides a handy way to build flexible “pipelines”. Today I’m writing up a short real-world example of this in practice. There are three elements to the flexibility that I really want to highlight:

On a family holiday in South Devon last week I had some good food experiences. Top of the pile was a Lardy Cake, a delicacy new to me but which I’ll be sure to be searching out again. It reminded me of an Eccles cake, but bigger and lardier!

I got mine from Vinnicombes on the High Street in Sidmouth.

Just down the coast from Sidmouth is a village called Beer. I didn’t have any food of note there, but it’s called Beer which is certainly worth recording :)

Timelion was released in November 2015 and with the 4.4.2 release of Kibana is available as a native visualisation once installed. It adds some powerful capabilities to Kibana as an timeseries analysis tool, using its own data manipulation language.

Installing Timelion is a piece of cake:

./bin/kibana plugin -i kibana/timelion

After restarting Kibana, you’ll see it as an option from the application picker

There’s a bit of a learning curve with Timelion, but it’s worth it. The blog gives some basics, and the built-in help is really good too:

OBIEE supports JDBC as a connection protocol, using the driver available on all installations of OBIEE, bijdbc.jar. This makes connecting to OBIEE from custom or third-party applications very easy. Once connected, you issue “Logical SQL” against the “tables” of the Presentation Layer. An example of logical SQL is:

SELECT "Time"."T05 Per Name Year" saw_0 FROM "A - Sample Sales"

To find more Logical SQL simply inspect your nqquery.log (obis-query.log in 12c), or Usage Tracking.

OK, maybe that’s not entirely true. But my read-only client, certainly.

I was perusing the Logstash input plugins recently when I noticed that there was one for IRC. Being a fan of IRC and a regular on the #obihackers channel, I thought this could be fun and yet another great example of how easy the Elastic stack is to work with.

Installation is a piece of cake:

wget https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/zip/elasticsearch/2.2.1/elasticsearch-2.2.1.zip
wget https://download.elastic.co/logstash/logstash/logstash-2.2.2.zip
wget https://download.elastic.co/kibana/kibana/kibana-4.4.2-linux-x64.tar.gz
unzip \*.zip
tar -xf kibana-4.4.2-linux-x64.tar.gz
sudo mv elasticsearch-2.2.1 logstash-2.2.2 kibana-4.4.2-linux-x64 /opt

(you’ll also need Oracle JDK installed if not already, here’s a handy way to get it from the CLI).

One of the perks of my job is that I get to travel to some pretty nice places (hi, San Francisco, Bergen, Åland Islands), and get to eat some pretty good food too. If you’re looking for some techie content, then move along and go and read about Kafka, but if you enjoy food pr0n then stay put.

I was working for a client in the centre of Manchester this week, staying as usual at a Premier Inn. I’m a big fan of my fried breakfasts, and whilst never anything to write home about (or indeed write a blog about) Premier Inn breakfasts are generally fine for what they are – but this one was a real disappointment. Even with the excitement of seeing bubble and squeak, it was a real let down; luke warm, and the sausages in particular tasted really artificially flavoured.

I got this lovely failure during a fresh install of OBIEE 11.1.1.9. I emphasise that it was during the install because there’s other causes for this error on an existing system to do with corrupted credential stores etc – not the case here.

The install had copied in the binaries and was in the process of building the domain. During the early stages of this where it starts configuring and restarting the AdminServer it failed, with the AdminServer.log showing the following: (I’ve extracted the salient errors from the log)

The Oracle by Example (ObE) here demonstrating how to use Goldengate to replicate transactions big data targets such as HDFS is written for the BigDataLite 4.2.1, and for me didn’t work on the current latest version, 4.4.0.

The OBE (and similar Hands On Lab PDF) assume the presence of pmov.prm and pmov.properties in /u01/ogg/dirprm/. On BDL 4.4 there’s only the extract to from Oracle configuration, emov.

Fortunately it’s still possible to run this setup out of the box in BDL 4.4, with bells on because it includes Kafka too. And, who doesn’t like a bit of Kafka nowadays?

I’ve always defaulted to Slideshare for hosting slides from presentations that I’ve given, but it’s become more and more crap-infested. The UI is messy, and the UX sucks - for example, I want to download a slide deck, I most definitely 100% am not interested in “clipping” it…even if you ask me every. damn. time:

Looking around it seems the other popular option is Speakerdeck. The UI is clean and simple, and I as both a user and uploader I feel like I’m there to read and share slides rather than be monetised as an eyeball on the site.