Installing CDH 5.7 on Linux Containers (LXC) hosted on Proxmox 4. Everything was going well until Cluster Setup, and which point it failed on Start YARN (MR2 included) Completed only 0/1 steps. First failure: Failed to execute command Start on service YARN (MR2 Included) Log /var/log/hadoop-yarn/hadoop-cmf-yarn-NODEMANAGER-cdh57-01-node-02.moffatt.me.log.out showed: org.apache.hadoop.service.AbstractService: Service containers-monitor failed in state INITED; cause: java.lang.NumberFormatException: For input string: "18446744073709550364" java.lang.NumberFormatException: For input string: "18446744073709550364" Looking down the stack trace, this came from org.

Continue Reading

I’ve got an EdgeRouter LITE (ERL) which I used as my home router until a powercut fried it a while ago (looks like I’m not the only one to have this issue). The symptoms were it powering on but not giving any DHCP addresses, or after a factory reset responding on the default IP of 192.168.1.1. It was a real shame, because it had been a great bit of kit up until then.

Continue Reading

(Previously, previously, previously) Since Proxmox 4 has a recent Linux kernel and mainline one at that, it means that Docker can be run on it. I’ve yet to really dig into Docker and work out when it makes sense in place of Linux Containers (LXC), so this is going to be a learning experience for me. To install Docker, add Backports repo to apt: root@proxmox01:~# cat /etc/apt/sources.list.d/backports.list deb http://ftp.debian.org/debian jessie-backports main And then install:

Continue Reading

(Previously, previously) I’ve got a bunch of existing VirtualBox and VMWare VMs that I want to run on Proxmox. Eventually I’ll migrate them to containers, but for the time being run them as “fat” VMs using Proxmox’s KVM virtualisation. After copying the OVA files that I had to the server, I uncompressed them: root@proxmox01:/data04/vms/bdl44-biwa# cd ../bdl44 root@proxmox01:/data04/vms/bdl44# ll total 27249328 -rw------- 1 root root 27903306752 Jun 1 10:14 BigDataLite440.ova root@proxmox01:/data04/vms/bdl44# tar -xf BigDataLite440.

Continue Reading

(Previously) With my server in place, I ran a memtest on it … which with 128G took a while ;) And then installed Proxmox 4, using a bootable USB that I’d created on my Mac from the ISO downloaded from Proxmox’s website. To create the bootable USB, create the img file: hdiutil convert -format UDRW -o target.img source.iso and then burn it to USB: sudo dd if=target.img of=/dev/rdiskN bs=1m Replace N with the correct device based on diskutil list output.

Continue Reading

After a long and painful delivery, I’m delighted to announce the arrival of a new addition to my household … : This custom-build from Scan 3XS is sat in my study quietly humming away. I’m going to use it for hosting VMs for R&D on OBIEE, Big Data Lite, Elastic, InfluxDB, Kafka, etc. I’ll blog various installations that I’ve done on it as a reference for myself, and anyone else interested.

Continue Reading

Oracle’s excellent Big Data Lite VM has been updated, to version 4.5. Download it here Changes: CDH 5.5 -> 5.7 Big Data Spatial and Graph 1.1 -> 1.2 Big Data Discovery 1.1 -> 1.2 Oracle Big Data Connectors 4.4 -> 4.5 Oracle NoSQL 3.5 -> 4.0 GoldenGate 12.2.0.1 -> 12.2.0.1.1

Continue Reading

I’ve been spending some interesting hours digging into OBIEE 12c recently, with some interesting blog posts to show for it. Some of it is just curiosities discovered along the way, but the real meaty stuff is the in the RESTful APIs - lots of potential here for cool integrations I think… Lifting the Lid on OBIEE 12c Web Services - Part 1 Lifting the Lid on OBIEE 12c Web Services - Part 2 Extended Subject Areas (XSA) and the Data Set Service Changes in BI Server Cache Behaviour in OBIEE 12c : OBIS_REFRESH_CACHE Dynamic Naming of OBIEE 12c Service Instance Exports OBIEE 12c - “Add Data Source” in Answers (Photo credit: https://unsplash.

Continue Reading

Presentation Services can provide some very detailed logs, useful for troubleshooting, performance tracing, and general poking around. See here for details. There’s no bi-init.sh in 12c, so need to set up the LD_LIBRARY_PATH ourselves: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/app/oracle/biee/bi/bifoundation/web/bin/:/app/oracle/biee/bi/lib/:/app/oracle/biee/lib/:/app/oracle/biee/bi/bifoundation/odbc/lib/ Run sawserver with flag to list all log sources /app/oracle/biee/bi/bifoundation/web/bin/sawserver -logsources > saw_logsources_12.2.1.txt Full list: https://gist.github.com/rmoff/e3be9009da6130839c71181cb58509a0

Continue Reading

In OBIEE 12c data-model-cmd is a wrapper for some java code which ultimately calls an internal RESTful web service in OBIEE 12c, bi-lcm. We saw in the previous post how these internal web services can be opened up slightly, and we’re going to do the same again here. Which means, time for the same caveat: None of these Web Services are documented, and they should therefore be assumed to be completely unsupported by Oracle.

Continue Reading

exportServiceInstance will export the RPD, Presentation Catalog, and Security model (application roles & policies etc – but not WLS LDAP) into a single .bar file, from which they can be imported to another environment, or restored to the same one at a later date (e.g. for backup/restore). To run exportServiceInstance you need to launch WLST first. The following demonstrates how to call it, and embeds the current timestamp & machine details in the backup (useful info, and also makes the backup name unique each time).

Continue Reading

So this had me scratching my head for a good hour today. Comparing SampleApp v511 against a vanilla OBIEE 12c install I’d done, one had “Add Data Source” as an option in Answers, the other didn’t. The strange thing was that the option wasn’t there in SampleApp – and usually that has all the bells and whistles enabled. After checking and re-checking the Manage Privileges option, and even the Application Policy grants, and the manual, I hit MoS - and turned up Doc ID 2093886.

Continue Reading

I had the pleasure of not one but two fry-ups in York, UK last weekend. The first was courtesy of Bill’s Restaurant Overall, pretty good, and I’ve had much worse. All the ingredients seemed decent. The black pudding was overcooked and almost biscuit-like, but that’s my only serious grumble. The bacon was cooked well. That black pudding, beans and the mashed/friend potato thing were each extra charges annoyed me. Particularly with a hangover, I just want to be able to order a full english, without playing Mastermind to work out what’s in or not.

Continue Reading

Architecturally, OBIEE 12c is - on the surface - pretty similar to OBIEE 11g. Sure, we’ve lost OPMN in favour of Node Manager, but all the old favourites are there - WebLogic Servers, BI Server (nqsserver / OBIS), Presentation Services (sawserver / OBIPS), and so on. But, scratch beneath the surface, or have a gander at slide decks such as this one from BIWA this year, and you realise that change is afoot.

Continue Reading

I wrote recently about Kibana’s excellent Timelion feature, which brings time-series visualisations to Kibana. In the comments Ben Huang asked: do you know how to show whats the difference between this Friday and last Friday by Timelion? So I thought I’d answer properly here. Timelion includes mathematical functions including add and subtract, as well as the ability to show data offset by an amount of time. So to answer Ben’s query, we combine the two.

Continue Reading

Running the OBIEE 12c startup on Windows: C:\app\oracle\fmw\user_projects\domains\bi\bitools\bin\start.cmd Just hangs at: Starting AdminServer ... No CPU being consumed, very odd. But then … looking at DOMAIN_HOME\servers\AdminServer\logs\AdminServer.out shows the last log entry was: Enter username to boot WebLogic server: And that’s bad news, cos that’s an interactive prompt, but not echo’d to the console output of the startup command, and there’s no way to interact with it. The start.cmd was being called by adding it to the Startup folder (C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp), and I guess it was something about this that stopped the prompt coming back to the console, because when I ran it manually from the command prompt, I got this:

Continue Reading

Another quick note on OBIEE 12c, this time on the importServiceInstance command. If you run it with a BAR file that doesn’t exist, it’ll fail (obviously), but the error at the end of the stack trace is slightly confusing: oracle.bi.bar.exceptions.UnSupportedBarException: The Bar file provided as input is not supported in this BI Platfrom release. Scrolling back up the stack trace does show the error message: SEVERE: Failed in reading bar file.

Continue Reading

Interesting quirk in running Baseline Validation Tool for OBIEE here. If you invoke obibvt from the bin folder, it errors with Parameter ‘directory’ is not a directory PS C:\OracleBI-BVT> cd bin PS C:\OracleBI-BVT\bin> .\obibvt -config C:\OracleBI-BVT\bin\bvt-config.xml -deployment current INFO: Result folder: Results\current Throwable: Parameter 'directory' is not a directory Thread[main,5,main] SEVERE: Unhandled Exception SEVERE: java.lang.IllegalArgumentException: Parameter 'directory' is not a directory at org.apache.commons.io.FileUtils.validateListFilesParameters(FileUtils.java:545) at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:521) at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:691) at com.oracle.biee.bvt.UpgradeTool.loadPlugins(UpgradeTool.java:537) at com.

Continue Reading

Yesterday I wrote about Monitoring Logstash Ingest Rates with InfluxDB and Grafana, in which InfluxDB provided the data store for the ingest rate data, and Grafana the frontend. Mark Walkom reminded me on twitter that the next release of Logstash will add more functionality in this area - and that it’ll integrate back into the Elastic stack: @rmoff nice, LS 5.0 will have APIs exposing metrics too. they’ll be integrated back into Marvel/Monitoring!

Continue Reading

In this article I’m going to show you how to easily monitor the rate at which Logstash is ingesting data, as well as in future articles the rate at which Elasticsearch is indexing it. It’s a nice little touch to add to any project involving Logstash, and it’s easy to do. Logstash is powerful tool for data ingest, processing, and distribution. It originated as simply the pipe to slurp at log files and put them into Elasticsearch, but has evolved into a whole bunch more.

Continue Reading