Shine Technologies Blog

My name is Mark. And I like rich clients.

There, I said it. I don’t like web applications. I never got Gmail as an interface. It never quite worked the way I wanted it to. I like eBay as a service, but not as an experience.

Over the last year I have found myself using NetNewsWire, not Google Reader.  I use OmniFocus, not RememberTheMilk.  I like having my data available everywhere, I just don’t want to use a browser to see or manipulate it.

I get that the interweb has allowed us to explore, connect and transact. Awesome. But now that we have the data and services the focus must shift to how we use it. The usability of most web applications is still well behind that of rich clients.  We need to get better.

So, we are faced with two alternatives. Try to make web applications better, or move to web enabled rich applications.

I favour the latter. Here’s why:

Rich clients work faster. Rich clients have tighter integration with the keyboard. You don’t need to be connected to the Internet all the time. Enough?

No! They also have tighter integration with the device they are running on. An iPhone application knows it has GPS and an accelerometer. It can do things you can’t do through a browser.

So let’s use the device well. Let’s optimise the experience for the device we are on. If we are on a mac I want Growl support. If we are on Windows then save data frequently.

But that is expensive to develop I hear you say. Yes. I didn’t say this would be cheap. However, in recent years we have gotten much better at building services faster (Spring: thank you.  Rails: thank you. SOAP: are you still here?).  So now that we have saved all that time on the back end, let’s spend a little more time on the front end.  

Please?!?

The following is a write-up of the highlights during the Sun Developer Day which I just attended. The early day kicked off with the usual registration and light refreshments before moving on to the ballroom for the opening keynote by Sun’s Director of Technology Outreach, Reginald Hutcherson.

The keynote addressed the possibilities of JavaFX in the non-PC arena such as televisions and mobile phones or according to Reginald, “any screen you will ever come across”. While it is a bold claim for a new technology in a world with existing competitors such as Flex, Silverlight, GWT …etc, it would be interesting to see how JavaFX performs in 2009. The keynote wrapped up with the advocacy of using open-source technology which was not surprising.

The demo shootout showcased the more interesting bits of the event. The first demo demonstrated the capabilities of JavaFX such as widget animation, a Flickr demo where click-able images are downloaded on the fly while floating across a canvas and finally, a video puzzle game with the video playing in the background. Performance was great and seamless considering the fact that they were all done on MacBook Pros. The next demo, showcased a compiz/beryl-like desktop for open Solaris and virtualization using VirtualBox which is now owned by Sun. Next up, is a quick 5 minute walkthrough of using NetBeans to produce a JavaFX app. This walkthrough shows off the tools within Netbeans such as drag and drop code generation and on the fly coding and previewing which is really impressive from a usability point of view. A feature which allows importing of Illustrator-created graphics was also mentioned, although it was never demonstrated.

The next bit discussed on the direction Java SE is heading and covered topics such as closure, Java Modules (JAM - JSR 277), and the usage of annotations in Swing, such as event handling…etc. While the morning covered the main highlights, the rest of the day was spent on code demonstrations on various topics such as REST, more JavaFX, SOA, MySQL, Dtrace and xVM (Virtual Box). Also, we got to see many of the features within Netbeans which were used during the demos. Sun is trying to market Netbeans to a wider range of developers (PHP, Ruby…etc.), which may be a challenge where there are many who have already got used to the Eclipse environment.

In general, the demos were well conducted and interesting. However, there were a few that were done in a hurry or needed more time for explanation as they were heavy topics (eg. SOA with OpenESB and Java CAPS). Unfortunately, there was not enough time for a Q&A session. Its understandable Q&As can potentially take up more time, but I’m sure there is always time for 1 or 2.

Personally, my main aim for attending the event was to learn more about JavaFX. I was impressed with what I saw and I appreciate the fact that Sun is stepping up their game in the IDE market. This is important, if they are to compete with the likes of Microsoft’s Visual Studio. However, one itching question remains. Is JavaFX able to compete with other web technologies in terms of performance on older hardware or will it be another slow Swing app?

One of my pet hates in software development is repetitive tasks: a complicated deployment process, tricky configuration of an application, repetitive editing motions that are just a little too messy for a find/replace. All of the above have, in the past, killed my concentration, dulled my senses and otherwise numbed my brain. I quickly realized that learning the ins and outs of a few powerful tools can take away a lot of the drudgery that sometimes crops up in my day-to-day work. Further, combining these tools makes it possible to automate just about everything that I need to do on a regular (or even not-so-regular) basis.

What’s important about everything that follows isn’t necessarily the process/tools itself, but what you get out of learning them. This is just what works for me, what has accumulated over a few years.

0. Pick a Good Editor (or IDE)

This first point is a half-hearted gesture because I don’t actually expect anyone to listen to me.

Use vim.

In all seriousness, if your editor doesn’t support basic macros and/or keyboard recording you’re going to encounter situations where you get slowed down by dull, repetitive editing. Even in an IDE that supports refactoring, macros will save you time and brain-dead copy/pastes. Automating repetitive tasks in editing source code is one of the simplest ways you can help yourself.

1. Learn Your Shell (or Learn Your IDE)

I think the Pragmatic Programmer said it well before I even knew what a “shell” was: the command-line is powerful.

Becoming very familiar with a powerful shell is one of the best things you can do for yourself as a developer. Yes, the syntax can be funky and the semantics confusing but over time you come to love it. For me, it’s the Unix shell: in *nix, the shell becomes the glue between all the other tools you use to get work done — including custom scripts and programs that take care of the heavy lifting for you. Windows users might be interested in PowerShell, which is a similar offering from Microsoft — you have the power of the .NET framework at your fingertips.

I understand, too, that folks accustomed to IDEs tend to get a little irked by shell programming. I still strongly recommend giving it a go, but scripting or extending your IDE to automate certain tasks you perform on a regular basis is a great start. If you find you lose a lot of time uploading and modifying configuration files to various servers, you may want to write an Eclipse plugin to make that process as simple as the click of a button.

Whatever path you choose — learning the more exotic aspects of a shell or the plugin API of your favourite IDE — learn it well. The only way to learn it well is to use it all the time.

2. Learn a Dynamically Typed Programming Language

If your shell scripts start to look like a train wreck when you need to perform complex operations, you may want to consider falling back to a scripting language: Perl, Python and/or Ruby are all great choices — bonus points if you mix the three to meet your needs. Languages like these will give you a more structured environment in which to express what you’re trying to achieve (at the cost of it being slightly more difficult to invoke external processes than, say, Bash — a standard Unix shell).

I think it’s important to note that I mention “dynamically typed” languages here because they generally don’t have a separate compilation step: bytecode compilation occurs as part of the execution of a Perl/Python/Ruby program. Thus, a small change doesn’t require an ant/make call to pull your code together again. Remember: you’re trying to automate a process, not create more process. Build management is overhead.

By the same token, if your program needs low-level access to hardware or raw speed for processing large quantities of data — don’t be afraid to fall back to a language like C or Java if you’re sure of your reasons for doing so. Needless to say, situations such as this are rare but not impossible.

3. Appreciation of the Supporting Cast

ssh, scp, rsync, svn/cvs/git, diff, patch, wget, sed, grep, find … the list of handy *nix command-line tools goes on and on. These are smaller tools that — with a good shell — can easily be combined to automate more complicated process. The output of these programs can also be piped into a script of your own doing to perform more complicated operations; then, the output of that program can be piped into another and so on.

This is the big thing I miss when working in Windows: even with PowerShell in play to alleviate the pain that is DOS, it has terrible tool support out of the box. Having the .NET framework on hand is great, but even that’s no substitute for the *nix command-line tools mentioned above. Rant aside, there are ways around most of the shortcomings in Windows and it’s possible to emulate some of the more useful command-line *nix tools. It just means you may need to put in more effort than a *nix counterpart at times.

For folks interested in automating with their IDE, your “supporting cast” might mean third party Java/.NET libraries which your automation plugin(s)/scripts can use to perform more complicated operations.

4. Invest Time in Learning Your Tools

Really, it all comes down to learning the tools from steps 1-3 backwards. Windows, Linux or OSX — you should be an expert in how your tools work and ideally use them on a daily basis. Really getting to know your tools might mean stepping out of your comfort zone a little to learn the nuts and bolts of how your shell ticks, or taking some time out of your day to learn certain esoteric API calls, or the semantics of some new construct in your scripting language of choice.

The goal of such practice is to surround yourself with an easily extensible suite of tools where every action feels like it’s an obvious choice. These tools will help to take away some of the unavoidable “noise” on the fringes of any software development project, and let you get on with getting your job done.

5. Write Lots of Scripts to Automate Common Tasks

This is at the core of automation: combine the above tools to get things done. Copy files to a server using scp, have your script extract and install it via ssh, manipulate configuration files using sed, grep and perl before finally starting the application and storing the process ID in a file with a quick shell command. All without you typing much more than “deploy”.

Automated tasks should be easy to kick off, require little-to-no input from you and — ideally — be fast, so you don’t have to wait long to determine if anything went haywire.

Automate at the First Sign of Resistance

Once you become accustomed to your tools — and again, by no means am I implying that those described here are the best or the only — you’ll step back when you start feel some resistance performing a task throughout your day:

Hmm. I don’t want to have to go through that 12-step deployment process all over again.

It becomes second nature to whip up a script that not only does the complicated deployment but verifies that the application is up and running as expected, checking a process exists and scanning the logs for hints indicating that everything’s fine. Over time you should find your automated tasks are easier to write and “feel” much more reliable than any manual process.

And, hopefully, you’ll be happier for it.

Having accomplished one of my main short term goals in life, I thought I would reflect on the recent events that earned me my first certification outside of Uni.

You might be thinking, whats the big deal about answering a couple of Java questions, nabbing the cert and moving on with life. Besides, we all sat for the tricky test tailored by Shine that got us this job. The only way to answer your question is to try the questions in the diagnostic exam by Whizlabs or head over to the Java Ranch website and try their mock exams.

A couple of us sitting for the exam tried out the questions on Whizlabs and had our butt kicked. Things we thought we knew about Java. The hardest part about achieving our goal was the preparation. Our initial plan was to pass the exam by December last year, however, other commitments at the client site threw our schedule off course. By the time we got back to studying which was around February, our knowledge of the first few chapters were stale. To make it worse, I had to regain my motivation to put in additional hours into the preparation which is seperate from Shine’s training time.

February was rather short and as March approached I was entering panic mode. By this time, I had covered about five chapters which left me with another five more to go. The next few weeks were spent flying through the remaining chapters and trying out the self-tests that came with at the end of each chapter. I tried out the Whizlabs exam to get an estimate of where I am at and I was rather disappointed with my score of 30%. The good thing is, I get to study the questions I got wrong and work on my weak points. I tried the questions on the CD that came attached with the SCJP book I was studying and I found the questions much tougher and trickier. The whole objective is to thoroughly test your knowledge.

I decided to stop doing the questions on the CD as the results I was getting were rather demoralizing. I spent Easter finishing up the last chapter and doing some practice exams with Whizlabs. One of the things that I look out for while doing an exam, is the clock. For example, if you had 72 questions to answer within 3 hours, you would have about 2.5 minutes to answer every one of them. I adopted an approach of answering the “easy” questions that I am confident with first and “Marking” those that I could not answer or I was unsure off. If you still have about an hour left after your first run, you can relax a little and review those questions that you have not answered. Try your best to give an answer and since its multiple-choice, analyze each answer and deduce which one feels right. You should still have abit of time left after answering your questions, so go back to the ones that you have marked. From personal experience, I found that I was able to tackle the questions that I was unsure about after I had answered the rest of them. Some tough questions really do not mix well with stress and you will be amazed at your ability to tackle them after the initial hurdle.

I was really nervous at the actual exam but the valuable experience gained with Whizlabs gave me that confidence to see it through. The questions are actually easier than the Whizlab’s ones but they are designed to keep you engrossed! So whatever you do, watch the clock, mark tough questions and review!

P.S. I have never passed a practice exam on Whizlabs before. Nearly did, but not counted as a pass. I scored 70% on the actual exam and its the first exam I ever passed. If you are coping well with the Whizlabs exams, you know you are doing well.

Some people love it, others hate it - many people just don’t get it. So what is Test Driven Development ?

Wikipedia has a page on it, there are heaps of blogs talking about it and our own Ben Teese has put together a great presentation on how to get started.

Jason Gorman has got a great analogy that helps to make it a bit more real - Test Driven Kitchen Design.

He didn’t invent the phrase “TDD is specification by example”, but it sums things up well.

So what’s in a name … ? Does it matter what you call your chunks of work ?

In many ways it doesn’t - different people use the terms inter-changeably. Sprint is specifically a Scrum term, XP tends to use Iteration, each different Agile approach has different ways of defining and using the terms. However the common ground is that they are regular, repeated timeboxes used to manage the work to be done so that there are regular checkpoints and feedback as part of the project process.

However there are some important differences between the processes of iterating versus incrementing.

Jeff Paton has posted a nice summary of the difference, along with some pictures that help to make it clear:

Iterating: Starting with an idea of what is wanted and refining that idea to get the desired result

Incrementing: Starting with a much more definite understanding and building piece by piece

Both still have the benefit of regular feedback and both can still deliver usable software at the end of the timebox. Incrementing is probably more useful when requirements are more stable or better understood, whilst iterating allows a more agile response to changing or unclear requirements. There’s no reason why a project can’t use both, at different times and for different features - in fact most projects probably should.

So - whatever you call your timebox periods, make sure you understand whether you want to iterate or increment.

If you think you are doing Scrum, how do you know ?

There are lots of good books around, but - whilst Scrum is deliberately quite simple - it can sometimes be easy to lose sight of the principles. Over on the Agile and Business blog Joe Little has posted the test used by Nokia to determine whether a team is using Scrum or not.

I’m happy to say that some of the recent work I have been involved in does quite well on the test, other work not so much …

First came light-weight containers such as Spring which provide some of the useful features of application servers, but with less constraints over both design and deployment. Now we have Web Beans:

1. A Web Bean does not have to be an EJB
2. Web Beans should be executable outside the EE environment

This is further support for the move away from the over-engineered and invasive approach in the original J2EE specification to a simpler POJO based model. However the Web Beans proposal attempts to stay within the formal J2EE fold by choosing to delegate some services to the application server:

” … not every component needs the services that EJB provides (transaction demarcation, authorization, etc). However, Web Beans will not duplicate this functionality, so when these services are needed, the Web Bean should be written as a session bean.”

In contrast, Spring implements some services (such as transaction management) and supports integration with other components to support others - for example Hibernate for persistence, Terracota for clustering - and does not rely on an application server (although it can be deployed within one).

It will be interesting to see where this ends up, and whether Web Beans ends up moving closer to or further away from the model adopted by Spring.

But what about the application server ? Certainly not dead, but perhaps leaner.

Hopefully these POJO based approaches will make design decisions less about the deployment architecture and specification constraints and more about implementing the business logic.

Technology is always making our lives easier. In fact a lot of the time we completely forget what it was that technology is doing for us. I believe that one of these areas is software quality assurance (QA). Tools such as code formatters, Checkstyle, EMMA, jUnit, Jester, Cruise Control, all assist in the quality assurance process; but are these tools (plus others not listed) good enough to no longer warrant manual QA?

There are many ways to skin a cat, and the same principle applies to building software applications. It’s just lucky we have an automatic tool for always identifying the best design/solution? Wait a minute, no we don’t. But surely design doesn’t matter, as long as the software works right? Wrong, design is a fundamental step in the software life cycle and if done poorly/inconsistently can lead longer development times and maintenance nightmares (and a slew of other problems).

A valuable technique for solving this problem and many others are good old manual (gasp) code reviews. Code reviews can be as quick as five minutes or as long as an hour (depends on the code). The advantages of code reviews are that they help to identify:

• Deviations from the design/architecture
• Software bugs
• Quality of test cases
• Overly complex code
• Adequate code commenting
• Deviations from the coding standards that are not automatically detectable
• Requirement tracing

Above all else, code reviews provide education and promote communication within the team. This ensures that all members are on the same page. Code review checklists are not hard to make and there are many templates available on the net. I find the best checklists are the short ones made specifically for the project. The downside to performing code reviews is obviously the ‘fourth dimension’ - time. But do the benefits out weigh the cost?

I think so.

There is a great article on the importance of code reviews here.

One of the common criticisms levelled at Agile development - particularly by those who haven’t experience it for themselves - is that it is little better than hacking. In particular, that the reduced focus on documentation is evidence that the processes aren’t managed or controlled.

So, for a corporation that has invested significant time and money in process improvement and has reached CMMI level 3 or above surely Agile development is not an option ?

Not according to Mark Paulk who looked at Extreme Programming in the context of CMM(I):

“XP has disciplined processes, and the XP process itself is clearly well defined. We can thus consider CMM and XP complementary.”

So who is Mark Paulk ? He worked at the Software Engineering Institute as leader of development of the CMM for software.