A few weeks back I wrote a post in which I visualized Dublin (CA) 2014 weather in Edward Tufte's style of New York 2003 weather visualization. That post used a dataset containing 15 years of daily weather data from 2000 to 2014 and showed daily temperature highs and lows against a background of daily record highs and lows and average highs and lows of years before 2014. I wanted to write an interactive application using Shiny, RStudio's web application framework for R, and the Dublin weather visualization seemed like a great candidate. I wrote the Shiny app and in the chart below you see the app in action -- an interactive version of that visualization. The user can select the range of years of temperature data to form the background, anywhere between years 2000 and 2014 inclusive.

I had been thinking of spiffying up my plain-looking WordPress-hosted blog for some time now and a few days back I got to do just that. Actually I did more than that: I ended up building a brand new website! While searching online for the how-tos I discovered GitHub Pages and Jekyll. GitHub Pages allows you to host your website for free and Jekyll lets you build static websites. It costs me ~$50 per year to host my blog on Wordpress so the free hosting on GitHub sounded like a great deal. On top of it the Jekyll website generator would allow the look and feel of the blog — the whole design — to be completely under my control. And so I took on the project. The project required git which was already installed on my desktop PC.

In the data visualization community Edward Tufte's chart of New York City 2003 weather is well-known. A few weeks ago Brad Boehmke published a blogpost with a similar chart for his city, Dayton, titled Dayton's weather in 2014 which inspired me to do a similar visualization for my city, Dublin, California. The result is the chart below. The R code to build the chart is located here and it draws from Boehmke's post but also includes original code. Further below I describe the steps used to arrive at the chart.

I needed to run a 64-bit Virtual Machine on my laptop when I found out that it did not support virtualization even though it had a 64-bit CPU. That got me started on acquiring a new PC. I decided that I wanted a mid-to-high-end PC -- not quite a high-end gaming PC -- but one with sufficient horsepower to be able to run multi-threaded applications with ease. I settled on the following minimum configuration: a 64-bit CPU with hardware support for virtualization, 8GB RAM, and 256GB hard drive. Superior graphics performance was not on my list of must-haves.

Last week (Nov 11-14) I attended the Cloudera Developer Training for Apache Hadoop course and in this post I share my experience and takeaways from that training. But first, a brief bit about me to lay out the context for my experience with this training. I have worked in the Telecommunications industry and have several years experience with Embedded Systems software design and development and about 2 months ago I decided to work in the Big Data space, which led me to this training course.