Shine wins Computerworld Data+ Award!

cw_2014_cwd02_data_shine_technologies-100410219-large.idge

Shine is proud to have been awarded the Computerworld Data+ award for our work with Google BigQuery.  The innovative work is a great example of using great technology to deliver business benefit.  You can see the write up of the award here:

http://www.computerworld.com/article/2598539/shine-technologies.html

Congratulations to the Shiners on the team: Luke, Pablo, Graham and Kon!

Posted in News | Tagged , , , | Leave a comment

Shiner Ben Teese to speak at YOW! Connected on Mobile Web Development

yow-connected

Shine Senior Developer Ben Teese will be speaking at YOW! Connected next week.

Held in Melbourne, Australia on September 8 and 9, YOW! Connected covers both Mobile Development and The Internet of Things.

Ben’s talk topic is ‘The State of the Mobile Web’ and is scheduled for 3:30pm on Tuesday. It’ll include advice about when to go web and when to go native, best practices for mobile web development, and a discussion of hybrid mobile apps.

If you’re attending the conference and run into Ben, be sure to say Hi!

 

Posted in Mobile, News, Web | Leave a comment

Using ‘HugeCollections’ To Manage Big Data

OLYMPUS DIGITAL CAMERA

Introduction

When writing complex software things don’t always go as planned. You implement a new feature that works perfectly well locally and in a test environment, but when your code hits the real world everything falls apart. Sadly, that’s one of the things we all have to deal with as software developers. On a recent project for a major telecommunications client we needed to be able to process more than 20 million records every night. That equated to 5GB of storage and unfortunately the environment where our process was running had up to 4GB of memory.

Processing such a vast amount of data brings a lot of challenges along with it, especially when you also need to combine it with a few more million records that are located in a database. Adding code to retrieve associated information and transform raw data might take an extra few milliseconds per each record. However, when you repeat that operation 20 million times those few milliseconds can easily turn into hours.

So there we were asking ourselves why it was taking so long. Is it an index we forgot to add to the database? A network latency problem? These things can be very hard to pin down.

We needed to think outside the box to get around this one.

Continue reading

Posted in Java

Put On Your Streaming Shoes

Saturday-Night-Fever-2

The Kick-Off Meeting

It went something along the lines of:

  • Client: “We have a new requirement for you..”
  • Shiners: “Shoot..”
  • Client: “We’d like you to come up a solution that can insert 2 million rows per hour into a database and be able to deliver real-time analytics and some have animated charts visualising it. And, it should go without saying, that it needs to be scalable so we can ramp up to 100 million per hour.”
  • Shiners: [inaudible]
  • Client: “Sorry, what was that?”
  • Shiners: [inaudible]
  • Client: “You’ll have to speak up guys..”
  • Shiners: “Give us 6 weeks”

We delivered it less than 4.

Continue reading

Posted in Cloud, Java, Javascript, Opinion | Tagged , , , , , , , , , , | Leave a comment

SSH through a Raspberry Pi Railway Signal

lightipadWe’ve all been there. You are in the supermarket with two bottles of diet cola in one arm and a packet of brown rice with quinoa in the other. Your site lead calls you with a request from a client who has locked themselves out of their account. Normally you would direct them to the administration interface but, because of the paradox inducing way in which they have bent the fabric of space and time, this will require some manual intervention.  You need to apply some subtle but distinct database changes. Simply delete a row or two from the QUANTUM_PARADOX table. Well, it’s actually a view… but that’s not important right now.

Continue reading

Posted in Linux | 1 Comment

Swift from an Objective-C developer’s perspective

After the most recent WWDC, most iOS developers aren’t talking about cool new iOS 8 features or APIs. Instead,  they’re talking about a whole new language: Swift. Yes, you heard right – a whole new programming language. How exciting it is!

Apple has been working on Swift secretly for a few years. It is a modern programming language that takes the strengths of other popular languages (for example, Python) and avoids the bad things about Objective-C (for example, poor manual memory-management and awful block syntax). As Apple declared, it’s Objective-C without the C. Not only this, Swift is also able to access the Apple Cocoa Touch framework and share the same LLVM compiler as Objective-C, so they can be seamlessly mixed in a same project.

Now you might be just like me, wondering why Apple would introduce a new language for iOS and Mac OS programming. Let’s walk through some most exciting parts of Swift to see if we can find a reason.

Continue reading

Posted in iPhone, Mobile | Tagged , | Leave a comment

Geekapalooza: My recent visit to Google I/O

https://plus.google.com/u/0/communities/105113874753230740413
src: https://plus.google.com/u/0/communities/105113874753230740413

Last week I had to pry myself away from my day-to-day life as a software developer and fly over the Pacific to attend Google I/O. Okay maybe I didn’t have to pry myself away – rather, it was more like me excitedly not sleeping for 2 days, giddy as a school kid, prior to leaving. I was lucky enough to be personally invited this year to both the conference and to other GDG organiser events to meet other community leaders across the globe. It’s safe to say I had high expectations.

I was expecting it to be big, but what I found was a mecca for geeks. A place where being a nerd was cool, wearing glass and wearables was not only normal but encouraged, and a place where everyone wanted to talk to you about the products they have been building and what tech stack and services made it possible. To say that I was blown away is an understatement — and this is just the people and culture surrounding the event. Let’s not forget the main reason I was there: I/O is the main stage for Google to announce their new product pipeline for the year. In this blog entry I’ll highlight the big announcements, albeit at a very high level.

Continue reading

Posted in Android, Cloud, Mobile, Opinion, UI | Leave a comment