Back in June 2014, at the annual Google IO in San Francisco, Google unveiled their newest, and much hyped cloud product, Cloud Dataflow. The demo they did that day, using a live twitter feed to analyze supporter sentiment during the 2014 world cup, got my mouth watering at the prospect of working with it. It looked downright freaking awesome, and I just couldn’t wait to get my hands on it to take it for a spin.
Posted in Cloud, Java, Testing
Tagged Architecture, Big Data, BigQuery, BQ, Cloud, Cloud Dataflow, ETL, Google, Google Cloud, Google Cloud Platform, Google Cloud Storage, Java, SDK
WHAT A WEEKEND! I’m still recovering from the 24 hours of coding and the unhealthy amount of coffee and snacks I’ve ingested during this hackathon.
Two weeks ago, Melbourne was host to the first Battlehack of 2015. “What’s a Battlehack?” I hear you ask. Well Battlehacks are competitions sponsored by Braintree (a Paypal company) where teams of developers face off over 24 hours to create the most creative hack that benefits the local or global community. The venue, which was the Plaza Ballroom on Collins St was well worth a mention. The vast ballroom was decked out with food stations, sleep and massage stations, and most importantly, our hacking stations, or tables of 4 for the long night ahead. Continue reading
A clash of the old-school and new-school
Over the last couple of years at Shine we’ve built a number of Single Page Apps (SPAs) for large businesses. An app we recently built had to integrate with Oracle Access Manager (OAM), an Identity Management System (IDM) from everybody’s favourite enterprise software company. In this post I’m going to talk about a strategy that we used to successfully work around mismatches between the way that SPAs and OAM do things.
Shine’s Graham Polley has been featured on the Google Cloud Platform Blog talking about his experiences with Google BigQuery.
In the post Graham discusses how he was able to stream massive amounts of data into BigQuery for a customer, and then query it with a SQL-like interface.
Check it out if you want to learn more!
Posted in News
Rewire your brain. Do it now. That’s what Oracle wants you to do with the introduction of the Java 8 SE. Unnecessarily dramatic statements aside, the “JCP” have approved a number of new language features that bring functional programming to Java – and which Oracle are promoting so heavily, they may as well be walking around with “declarative programming is great” inked on their collective foreheads.
We love Git Flow. It’s awesome to have a standard release process where everybody is using the same terminology and tooling. It’s also great to have out-of-the-box answers to the same questions that get asked at the start of every project. For example:
- “How are we going to develop features in isolation?”
- “How are we going to separate release candidates from ongoing development?”
- “How are we going to deal with hotfixes?”
Now it’s enough to just say ‘We use Git Flow’, and everybody’s on the same page.
Well, mostly. Whilst Git Flow is terrific for managing features and separating release candidates from ongoing development, things get a little hazier when it comes time to actually release to production. This is because of a mismatch between the way that Git Flow works and another practice that is common in large development projects: immutable build artifacts.
A few months ago I joined an exciting project with fellow Shiner Graham Polley, who you might know from such hits as Put on your streaming shoes. This is a follow-up article, discussing the elegant way in which we solved a hideous asynchronous limitation in PHP.
My role on the project was DevOps-based, and I was there to build some infrastructure using Amazon Web Services. As the cool kids would put it, I was there to put our client’s enterprise application INTO the cloud, or, more succinctly, to build a solution coupling services from two rival cloud service providers and provide a new league of scalability and flexibility.
The solution was pretty simple, but, like any simple solution, the little complexities come out along the way, and when you’re least expecting them.