Shiner Ben Teese to speak at YOW! Connected on Mobile Web Development


Shine Senior Developer Ben Teese will be speaking at YOW! Connected next week.

Held in Melbourne, Australia on September 8 and 9, YOW! Connected covers both Mobile Development and The Internet of Things.

Ben’s talk topic is ‘The State of the Mobile Web’ and is scheduled for 3:30pm on Tuesday. It’ll include advice about when to go web and when to go native, best practices for mobile web development, and a discussion of hybrid mobile apps.

If you’re attending the conference and run into Ben, be sure to say Hi!


Posted in Mobile, News, Web | Leave a comment

Using ‘HugeCollections’ To Manage Big Data



When writing complex software things don’t always go as planned. You implement a new feature that works perfectly well locally and in a test environment, but when your code hits the real world everything falls apart. Sadly, that’s one of the things we all have to deal with as software developers. On a recent project for a major telecommunications client we needed to be able to process more than 20 million records every night. That equated to 5GB of storage and unfortunately the environment where our process was running had up to 4GB of memory.

Processing such a vast amount of data brings a lot of challenges along with it, especially when you also need to combine it with a few more million records that are located in a database. Adding code to retrieve associated information and transform raw data might take an extra few milliseconds per each record. However, when you repeat that operation 20 million times those few milliseconds can easily turn into hours.

So there we were asking ourselves why it was taking so long. Is it an index we forgot to add to the database? A network latency problem? These things can be very hard to pin down.

We needed to think outside the box to get around this one.

Continue reading

Posted in Java

Put On Your Streaming Shoes


The Kick-Off Meeting

It went something along the lines of:

  • Client: “We have a new requirement for you..”
  • Shiners: “Shoot..”
  • Client: “We’d like you to come up a solution that can insert 2 million rows per hour into a database and be able to deliver real-time analytics and some have animated charts visualising it. And, it should go without saying, that it needs to be scalable so we can ramp up to 100 million per hour.”
  • Shiners: [inaudible]
  • Client: “Sorry, what was that?”
  • Shiners: [inaudible]
  • Client: “You’ll have to speak up guys..”
  • Shiners: “Give us 6 weeks”

We delivered it less than 4.

Continue reading

Posted in Cloud, Java, Javascript, Opinion | Tagged , , , , , , , , , , | Leave a comment

SSH through a Raspberry Pi Railway Signal

lightipadWe’ve all been there. You are in the supermarket with two bottles of diet cola in one arm and a packet of brown rice with quinoa in the other. Your site lead calls you with a request from a client who has locked themselves out of their account. Normally you would direct them to the administration interface but, because of the paradox inducing way in which they have bent the fabric of space and time, this will require some manual intervention.  You need to apply some subtle but distinct database changes. Simply delete a row or two from the QUANTUM_PARADOX table. Well, it’s actually a view… but that’s not important right now.

Continue reading

Posted in Linux | 1 Comment

Swift from an Objective-C developer’s perspective

After the most recent WWDC, most iOS developers aren’t talking about cool new iOS 8 features or APIs. Instead,  they’re talking about a whole new language: Swift. Yes, you heard right – a whole new programming language. How exciting it is!

Apple has been working on Swift secretly for a few years. It is a modern programming language that takes the strengths of other popular languages (for example, Python) and avoids the bad things about Objective-C (for example, poor manual memory-management and awful block syntax). As Apple declared, it’s Objective-C without the C. Not only this, Swift is also able to access the Apple Cocoa Touch framework and share the same LLVM compiler as Objective-C, so they can be seamlessly mixed in a same project.

Now you might be just like me, wondering why Apple would introduce a new language for iOS and Mac OS programming. Let’s walk through some most exciting parts of Swift to see if we can find a reason.

Continue reading

Posted in iPhone, Mobile | Tagged , | Leave a comment

Geekapalooza: My recent visit to Google I/O

Last week I had to pry myself away from my day-to-day life as a software developer and fly over the Pacific to attend Google I/O. Okay maybe I didn’t have to pry myself away – rather, it was more like me excitedly not sleeping for 2 days, giddy as a school kid, prior to leaving. I was lucky enough to be personally invited this year to both the conference and to other GDG organiser events to meet other community leaders across the globe. It’s safe to say I had high expectations.

I was expecting it to be big, but what I found was a mecca for geeks. A place where being a nerd was cool, wearing glass and wearables was not only normal but encouraged, and a place where everyone wanted to talk to you about the products they have been building and what tech stack and services made it possible. To say that I was blown away is an understatement — and this is just the people and culture surrounding the event. Let’s not forget the main reason I was there: I/O is the main stage for Google to announce their new product pipeline for the year. In this blog entry I’ll highlight the big announcements, albeit at a very high level.

Continue reading

Posted in Android, Cloud, Mobile, Opinion, UI | Leave a comment

Think a Bit, Code a Bit, Test a Bit


The best piece of advice I ever got regarding a personal software development process was from a grizzled old Unix developer with a neckbeard. OK, that’s not true – it was actually from a clean-shaven principal consultant and architect at a company I used to work at, but that doesn’t sound nearly as impressive.

Nevertheless, the process went something like this:

  1. Think a bit
  2. Code a bit
  3. Test a bit
  4. Go to Step 1

Having tried all manner of processes over the years, this is the one that has served me best. Let me break it down for you if you’re having trouble understanding it.

Step 1: Think a bit

Do you equate productivity with the sound of keys clicking on a keyboard? If so, my advice is to slowly step away from the computer and actually have a think about the problem you’re trying to solve before leaping into the fray.

When it comes to thinking, some people have trouble knowing where to start. If that’s you, here’s a suggested starting point: what are you actually supposed to be doing? Sometimes a few moments of quiet reflection will reveal that what you were thinking of typing doesn’t really relate to what is actually needed, or that you need more information before you can continue.

The next thing I recommend thinking about is: at this point in time, what’s The Simplest Thing That Will Possibly Work? I have never regretted asking myself this question; I regularly regret not asking it before I started typing. Also known as Keep It Simple, Stupid (KISS) or You Ain’t Going to Need It (YAGNI), the quest for The Simplest Thing That Will Possibly Work is a noble and worthy goal for a software developer. 

Feel that The Simplest Thing That Will Possibly Work is insufficiently challenging for your superior skills? Maybe, maybe not. Sometimes, when you actually force yourself to think about it, finding and executing The Simplest Thing That Can Possibly Work can actually be quite a challenge in itself.

Complexity will come and find you eventually; there’s no need to seek it out prematurely. Besides, what’s the worst that can happen when you do The Simplest Thing That Will Possibly Work? You can always just go to the pub if you finish early (although funnily enough, this has never happened to me, ever).

That said, you can only do so much thinking about the problem you’re trying to solve and how you’re going to solve it. It’s important not to go overboard and enter a state of analysis paralysis – a debilitating condition I am personally prone to. My strategy: as soon as I feel the symptoms of paralysis setting in, I know that it’s time to move on to the next step. Which brings us to…

Step 2: Code a bit

Now for the fun part. I’m not even really going to go into this one too much – it’s why we’re all in the computer programming game, right?

But of course, one can have too much of a good thing and coding is no exception. Soon unwanted thoughts and questions will begin to enter your head – for example: how do you know if your program is going to…you know…DO WHAT IT’S SUPPOSED TO DO?

So let’s now artfully jump-cut past your ecstatic communion with the keyboard and go straight to the next step, when it’s time to…

Step 3: Test a bit

I’m not going to lie to you and pretend that this is as much fun as coding. That said, this step isn’t really negotiable.

“But Ben”, I hear you say, “I don’t want to do stuff that’s boring”. To you, I say: too bad, you’re not being paid to produce code that doesn’t work. Never in the history of computing has it been easier to write automated tests. So stop procrastinating and write some tests already. If you really want to write code without tests, do it in your own time, on your own open source project (which no-one will use, by the way, because it has no tests).

And perhaps, to be honest, our aversion to testing stems not so much from the fact that it’s boring, but from the fact that’s it can actually be quite hard. In much of the work I do these days, testing is probably the most challenging part because, unlike the coding phase, the optimal solution is less obvious. 

Writing tests that are both meaningful (i.e., not just mocking the heck out of everything) yet practical (i.e., not excessively fragile or long-running) is hard. I’d call it more art than science, and the ideal solution differs from project to project. Unlike the noble quest for the Simplest Thing That’s Going to Work when coding, testing can be a messy, uncertain affair.

Some people like to do this step before they code. I say: good for you; it’s nice if you can do it, but don’t get hung up on it. If what I’m doing is highly algorithmic, test-first can be useful. But that doesn’t happen to me often – most of the work that I do is essentially integrating existing pieces. And sometimes I don’t know exactly how to do that until it’s been done, which kind of makes TDD hard. 

That said, whether you test-first or test-after, you should always test.

Step 4: Rinse and Repeat

We can’t expect to solve all of a problem in one hit; iteration and feedback are critical. Developing a piece of software is a process of continuous learning – about both the technology and the problem domain. A tight cycle of planning, execution and reflection is the optimal strategy.

As we progress, this cycle may accelerate, sometimes to the point where the steps will appear to blur into one another. But if we hit a roadblock and progress falters, this does not mean that the steps should be skipped – we should just slow things down and continue our cycle of Think A Bit, Code A Bit, Test A Bit.


So there you have it, a simple 4-step personal software development process. It’s served me well for most of my career, and the good news is that modern software development tools are making it easier and easier to work this way. Furthermore, whilst this approach predates the agile movement, it obviously ties in with the agile philosophy.

So try thinking a bit, coding a bit, and testing a bit – I promise that you won’t be disappointed.

Posted in Opinion | 1 Comment