Shine Technologies Blog

As a relative newby to the world of Cocoa programming (on the iPhone in particular), I have spent some time trying to understand if and when you’d use a run-loop instead of launching a separate thread. I was unable to find any definitive answer on the web, so ended up joining the dots myself. What follows is my understanding of when you’d want to use one or the other. Cocoa experts are welcome to comment if I’ve got it wrong.

The Problem

Touches aren’t the only source of input to an iPhone application. For example, another source can be a socket - sometime you want to listen to a socket for data. But you don’t want the UI to lock up whilst it’s listening - you still want input from the user to be dealt with promptly. Similarly, you might want events to be triggered automatically at certain time intervals, but without locking up the application in the interim.

Coming from other UI frameworks, you might think that the way to deal with this is to to use a separate thread. That way, the thread can block on the socket or sleep for a particular time interval. However, as we all know, the introduction of multiple threads immediately introduces a bunch of potential defects that are difficult to reproduce and fix.

The Solution

Enter run loops. Or more specifically, the run loop - each iPhone application has one by default and for our purposes, this is all we need.

So what exactly is a run loop?

Well, first consider this assertion:the vast majority of the time that your Cocoa application is running, it’s doing nothing. More specifically, it’s waiting for input. However, as soon as you touch the screen, an event gets triggered, which may in turn result in some of your code being executed. If some data comes into a socket, or a timer fires, the same applies.

The key things is that once this code has been executed, the application goes back to waiting for input. Furthermore, in many cases the execution time of your code will be very small relative to the time the application spends waiting for input.

I think of run loops as a mechanism that exploits this fact.

A run loop is essentially an event-processing loop running on a single thread. You register potential input sources on it, pointing it to the code that it should execute whenever input is available on those sources.

Then when input comes into a particular source, the run loop will execute the appropriate code, then go back to waiting for input to come in again to any of it’s registered sources. If input comes into a registered source whilst the run-loop is executing another piece of code, it’ll finish executing the code before it handles the new input.

The upside of this is that whilst you mightn’t know exactly what order things are going to come in, at least you know that they’ll be processed one after the other instead of in parallel. This means that you avoid all of those nasty multi-threading issues that were described earlier. And that’s why run loops are useful.

Run loop scheduling in action

By default, all touch events received by an iPhone application are queued for processing by the application’s main run loop, so there’s nothing special you need to do for UI components. However, other sources of input require additional coding.

To schedule an NSInputStream on a run loop, you’d do something like this:

This code sets it up so that whenever input is available on ‘iStream’, a ’stream:handleEvent’ message will be sent to ’self’. Note that the stream could be from any sort of source, including a socket.

Another object that can be scheduled on a run loop is a timer. For example:

will schedule a timer on the current run loop to send a ‘doStuff’ message to ’self’ every two seconds.

When not to use a run loop

So when wouldn’t you use a run loop? Well, if you had some event-handling code that was going to take a long time to execute (for example, performing some CPU-intensive calculation), then everything else in the event-handling queue won’t get handled until it’s finished. This would cause your application to become unresponsive until the processing has finished. In that sort of scenario, you might want to consider using a separate thread to do the processing.

However, for the vast majority of cases, our code for handling events - be they from the screen, sockets or timers - takes a very short time to execute. And that’s why it’s easier (and safer) to just use the main run loop to handle those events.

The trade-offs

The only downside to using a run loop instead of a thread is that instead of just whacking a thread around a whole section of code that you know will block in one or more places, you have to go to each potential blocking point, register the source on the run loop, and implement a callback to process events that are generated from that source.

Whilst this may seem like some effort, it pales in comparison to the pain that can result from poorly-considered threading. So next time you’re tempted to use a thread to read from a blocking input source, consider taking the time to use a run loop. It could well save you a lot of time in the long run.

A while back Mark expressed interest to me in using git and git-svn for version control on his own machine, against a remote Subversion repository. However, when I followed up with him recently, he admitted that in the small amount of time he’d spent looking at it, he hadn’t really got his head around how it was all going to hang together.

I can relate. I found Git to have a steep learning curve. It took me a while - and some assistance (thanks Tom) - to figure out the magical incantations to accomplish what I wanted. But now that I know them, I’ve found Git very useful for local version control.

I was going to walk Mark through my rough git-svn workflow, figuring it’d either get him started or scare him off for good Then I decided I might as well share it with the world.

I’ve covered my motivations for using git for local version control in a previous post, so I won’t repeat them here. Nor am I going to provide a tutorial on git and git-svn - there’s plenty of those out there already. Instead, I’m going to run through an example taken from my day-to-day work to try and show how I use git and git-svn in real-life.

The Basics

1. The first step is something I normally only do very occasionally: create a local Git clone of my remote Subversion repository. So say that I want to create a git clone of the ‘remote_maintenance’ project on the Shine Subversion repository:
   git svn clone svn+ssh://subversion.shinetech.com/home/svn/remote_maintenance/trunk remote_maintenance

   This checks out the contents of the ‘trunk’ branch into a local directory called ‘remote_maintenance’. This local checkout is known as a ‘working tree’.

   The most significant thing I can say about ‘git svn clone’ is that it will take a while if your SVN repository has a big history, as git will create an entire replica of this history. This might sound like a drag but can be very useful later if you’re working offline.

2. Having cloned the repository, I merrily jump into the code in the working tree and start modifying, adding, deleting and moving files. Note that if I want to move a file and have git track a move, I use ‘git mv’. For example, to rename the ‘README’ file to ‘README.txt’, I’d do:
   bent:remote_maintenance bent$ git mv README README.txt

   I could just move it using ‘mv’, but git wouldn’t be able to track the change.

3. When I’m curious to know what my changes to the working tree have been since my last commit, I use ‘git status’. For example, having renamed a file and added a new file, I’d get the following output:
   bent:remote_maintenance bent$ git status
   # On branch master
   # Changes to be committed:
   # (use “git reset HEAD …” to unstage)
   #
   # renamed: README -> README.txt
   #
   # Untracked files:
   # (use “git add …” to include in what will be committed)
   #
   # LICENCE.txt

   If I want to look at my changes in more detail, I run:

   git diff

   Which gives me a line-by-line diff.

4. When I’m ready to commit my changes to git, I first run:
   git add .

   This will add any new files to the local index. I’ll cover what this means in more detail in the next step. After I’ve done run this command, running ‘git status’ again will yield the following:

   bent:remote_maintenance bent$ git status
   # On branch master
   # Changes to be committed:
   # (use “git reset HEAD …” to unstage)
   #
   # new file: LICENCE.txt
   # renamed: README -> README.txt
   #
5. Next, I commit all my changes by running:
   git commit -a -m ‘Miscellaneous changes’

   The ‘-a’ means that git will automatically add any changed files to the local index before doing the commit. For git newbies, git differs from Subversion in that shepherding changes into a git repository is a two-stage process - first you add a changed file to a local index, then you commit the changes to the repository itself. ‘git commit -a’ is a convenient way of combining these two steps into one.

   Unfortunately, if I have added brand new files to my working tree, ‘git commit -a’ won’t pick them up. Thus the need to explicitly add them to the repository using ‘git add’ in the previous step.

6. Having done my commit, I can return to step 2 and repeat as often as I want: make changes to my working tree, add them to the local index, and commit them.

Branching

1. So what if I want to branch? Well firstly, let’s examine the history of my commits by using ‘git log’:
   bent:remote_maintenance bent$ git log
   commit 76b1ebf06586843e990a29423df39f2def2492b5
   Author: Ben Teese
   Date: Tue Jan 6 12:04:49 2009 +1100

   Miscellaneous changes

   commit 21873ad305a06b97504c0c2270549a9a20238596
   Author: Ben Teese
   Date: Wed Jan 7 16:09:34 2009 +1100

   Made it that ground server binds TCP server to provided IP address only, not all network interface

   commit 69b5b7421bc989538689711198510702f129d8f6
   Author: Ben Teese
   Date: Wed Jan 7 14:06:05 2009 +1100

   Made it that transfer service TCP server only binds to IP address provided. Added logging.
   …

2. To see what branches we’ve got at the moment, we use ‘git branch’:
   bent:remote_maintenance bent$ git branch
   * master

   We see that we currently only have one branch called ‘master’ - the default one that you get when you create a new repository.

3. Say that I want to try an alternate approach to the change I just committed. To do that, I’d create a create a local branch from the previous commit and check it out. I can do this in one step using:
   git checkout -b new_branch 21873ad305a06b97504c0c2270549a9a20238596

   The long string is a SHA that represents the commit we want to branch from. There are other, shorter, ways to refer to commits, but I find it just as easy to copy the SHA from the output of ‘git log’ directly into the ‘git checkout’ command.

4. Now let’s use ‘git branch’ to see what branches we’ve now got:
   bent:remote_maintenance bent$ git branch
   master
   * new_branch

   We see that out new branch appears and that it has an asterisk next to it. This means it is the currently checked-out branch.

5. Now, if I do a ‘git log’:
   commit 21873ad305a06b97504c0c2270549a9a20238596
   Author: Ben Teese
   Date: Wed Jan 7 16:09:34 2009 +1100

   Made it that ground server binds TCP server to provided IP address only, not all network interface

   commit 69b5b7421bc989538689711198510702f129d8f6
   Author: Ben Teese
   Date: Wed Jan 7 14:06:05 2009 +1100

   Made it that transfer service TCP server only binds to IP address provided. Added logging.
   …

   I see that my new branch only goes as far as the previous commit. You could now commit changes to this branch, and they wouldn’t appear in the ‘master’ branch.

6. We’ll switch back to the ‘master’ branch to continue with this demo:
   bent:remote_maintenance bent$ git checkout master
   Switched to branch “master”

Synchronizing with Subversion

1. When I want to synchronize my local git branch with the remote Subversion repository, I run:
   git svn rebase

   It’ll spit out something like this:

   M test/unit/csdb_inventory_content_test.rb
   M test/unit/resource_entity_test.rb
   M test/unit/ground_server_test.rb
   r3743 = 64471395cf084217f7eab91e07abb03297492c83 (git-svn)
   M test/unit/csdb_inventory_content_test.rb
   M test/unit/resource_entity_test.rb
   M test/unit/ground_server_test.rb
   r3744 = 1828baa8e7b91cd8861dfaaecc26eb66b0f9264e (git-svn)
   First, rewinding head to replay your work on top of it…
   Applying: Made it that transfer service TCP server only binds to IP address provided. Added logging.
   Applying: Miscellaneous changes
   /Users/bent/NetBeansProjects/remote_maintenance/.git/rebase-apply/patch:31: trailing whitespace.

   A rebase temporarily winds back the commits that you’ve made on the branch since the last time you rebased, applies the commits from Subversion to the branch, then reapplies your commits to the branch. The great thing about this is that git-svn keeps track of the last rebase you did, so you never have to tell it that you only want to rebase from a particular point. This avoids some of the problems that Subversion has when you repeatedly merge from one branch to another.

2. But what if somebody has changed a file that I’ve changed and we have a merge conflict? Git will report the problem as it is trying to reapply the local commits:
   Auto-merged test/unit/command/reader_test.rb
   CONFLICT (content): Merge conflict in test/unit/command/reader_test.rb
   Failed to merge in the changes.
   Patch failed at 0002.

   When you have resolved this problem run “git rebase –continue”.
   If you would prefer to skip this patch, instead run “git rebase –skip”.
   To restore the original branch and stop rebasing run “git rebase –abort”.

3. To resolve the merge conflict, I open up test/unit/command/reader_test.rb, resolve the merge conflicts, then run:
   git add test/unit/command/reader_test.rb
4. I repeat step the previous step for any other file that reported a conflict during the re-application of that particular commit. When I’m done, I run:
   git rebase –continue
5. Additional merge conflicts may occur when git reapplies later commits, in which case I repeat steps 3 and 4.
6. Now I have a look at my log on the current branch:
   bent:remote_maintenance bent$ git log
   commit 21873ad305a06b97504c0c2270549a9a20238596
   Author: Ben Teese
   Date: Wed Jan 7 16:09:34 2009 +1100

   Made it that ground server binds TCP server to provided IP address only, not all network interface

   commit 69b5b7421bc989538689711198510702f129d8f6
   Author: Ben Teese
   Date: Wed Jan 7 14:06:05 2009 +1100

   Made it that transfer service TCP server only binds to IP address provided. Added logging.

   commit 1828baa8e7b91cd8861dfaaecc26eb66b0f9264e
   Author: danielw
   Date: Tue Jan 6 22:13:34 2009 +0000

   converted the first letter of the test names to be lower case

   git-svn-id: svn+ssh://svn.shinetech.com/home/svn/remote_maintenance/trunk@3744 25b05753-7f2e-0410-

   We can see that the changes in my local git repository have been applied on top of the latest change from Subversion.

Squashing Changes Together

1. Now I want to squash together the first 2 of the changes I’ve made to my local git respository into a single commit. To do this, I run:
   bent:remote_maintenance bent$ git rebase -i git-svn

   My local terminal editor will appear with the following:

   pick 69b5b74 Made it that transfer service TCP server only binds to IP address provided. Added logging.
   pick 21873ad Made it that ground server binds TCP server to provided IP address only, not all network interfaces.
   pick 76b1ebf Added dvd stub that responds correctly to ProtocolRequest from iPhone.

   # Rebase 1828baa..76b1ebf onto 1828baa
   #
   # Commands:
   # p, pick = use commit
   # e, edit = use commit, but stop for amending
   # s, squash = use commit, but meld into previous commit
   #
   # If you remove a line here THAT COMMIT WILL BE LOST.
   # However, if you remove everything, the rebase will be aborted.
   #

2. To squash the second commit into the first, I edit the text as follows:
   pick 69b5b74 Made it that transfer service TCP server only binds to IP address provided. Added logging.
   squash 21873ad Made it that ground server binds TCP server to provided IP address only, not all network interfaces.
   pick 76b1ebf Miscellaneous changes

   # Rebase 1828baa..76b1ebf onto 1828baa
   #
   # Commands:
   # p, pick = use commit
   # e, edit = use commit, but stop for amending
   # s, squash = use commit, but meld into previous commit
   #
   # If you remove a line here THAT COMMIT WILL BE LOST.
   # However, if you remove everything, the rebase will be aborted.
   #

   and save and exit the editor.

3. git immediately pops up another editor that allows me to merge the comments for these two commits that I have said I want to squash together:
   # This is a combination of two commits.
   # The first commit’s message is:

   Made it that transfer service TCP server only binds to IP address provided. Added logging.

   # This is the 2nd commit message:

   Made it that ground server binds TCP server to provided IP address only, not all network interfaces.

   # Please enter the commit message for your changes. Lines starting
   # with ‘#’ will be ignored, and an empty message aborts the commit.
   #
   # Committer: Ben Teese
   #
   # Not currently on any branch.
   # Changes to be committed:
   # (use “git reset HEAD …” to unstage)
   #
   # new file: lib/dvd_stub.rb
   # modified: lib/ground_server.rb
   # modified: lib/rmu_stub.rb
   #

4. Which I do accordingly:
   Made it that TCP servers only uses IP address provided. This ensures it only binds to the network inte
   rface of that IP address, not all network interfaces. Added logging.
   # Please enter the commit message for your changes. Lines starting
   # with ‘#’ will be ignored, and an empty message aborts the commit.
   #
   # Committer: Ben Teese
   #
   # Not currently on any branch.
   # Changes to be committed:
   # (use “git reset HEAD …” to unstage)
   #
   # new file: lib/dvd_stub.rb
   # modified: lib/ground_server.rb
   # modified: lib/rmu_stub.rb
   #

   and save and exit. Git will then squash the two commits together.

5. If we do a ‘git log’, we see that git has merged the two commits into one, complete with a new message:
   bent:remote_maintenance bent$ git log
   commit 6a29acd801f280360699ac5278fcb63bb9c8744a
   Author: Ben Teese
   Date: Tue Jan 6 12:04:49 2009 +1100

   Miscellaneous changes

   commit 1fb107af4c765d2220d6674d61edca3fb13f5dc2
   Author: Ben Teese
   Date: Wed Jan 7 14:06:05 2009 +1100

   Made it that TCP servers only uses IP address provided. This ensures it only binds to the network

   commit 1828baa8e7b91cd8861dfaaecc26eb66b0f9264e
   Author: danielw
   Date: Tue Jan 6 22:13:34 2009 +0000

   converted the first letter of the test names to be lower case

   git-svn-id: svn+ssh://svn.shinetech.com/home/svn/remote_maintenance/trunk@3744 25b05753-7f2e-0410-
   …

Commiting to Subversion

Having squashed some of our commits together, commiting to Subversion is relatively straightforward:

This will create one commit in Subversion for each git commit.

Keeping Branches Synchronized

So what if you’ve made changes in one branch and you want to get them into another branch? The key piece of advice I can give is to use the Subversion repository as the transport mechanism. Once you’ve finished making changes in one branch, check them into the Subversion repository. Then switch to the other branch and rebase from the Subversion repository.

But what if you want to move code from one branch to another, without commiting to Subversion? Well, I have never done a direct git merge from one branch to another, and the ‘Caveats’ section of the git-svn man page recommends against it if you’re using git-svn. To be perfectly honest, it’s never been an issue for me. I tend to work on distinct features in separate branches so there is little code overlap. If there is some code overlap, I just have to fix the merge conflicts when I rebase from Subversion. However, given that I made both sets of changes, this usually isn’t too difficult.

Conclusion

From this demo you’ll see that my day-to-day usage of git and git-svn essentially boils down to 10 commands. I’ve listed them below, along with a quick summary of what they do:

• git status :See an overview of the files I’ve changed
• git diff: See a diff of the changes I’ve made
• git add: Add new or merged files to the index.
• git commit -a: Add changed files to the index and commit
• git log: See the history of commits I’ve made.
• git branch: See what branch is currently checked out.
• git checkout -b: Checkout a new branch from a particular point.
• git rebase -i: Squash together a bunch of git commits.
• git svn rebase: Get the changes in the Subversion repository.
• git svn dcommit: Send git commits to the Subversion repository.

Once I figured these out, I never looked back. I hope that you find them as useful as I did when using git for local version control with a remote Subversion repository.

The Problem

A while back we had a Flex client that needed to be able to display search results received from a server. The server was designed RESTfully, returning XML results to the client. The Flex client would display these results nicely to the user, and when the user clicked on a result, their browser would be directed to a HTML page for editing the actual item.

The problem was that, when a client clicked on a result, we were manually constructing the resource URLs in the Flex client from the IDs buried in the search results. The Flex code looked a bit like this:

Furthermore, we had an impending need to support lots of different types of resources, so our logic for assembling URLs was about to become rather convoluted.

A Solution

Put the URLs into the search results. It’s blindingly simple (when you think about it, we do it all the time with pure HTML sites), but requires a subtle shift in mindset when you’re writing a fat client.

The solution was prompted by an article on REST Anti-Patterns I had read on InfoQ. - in particular, an anti-pattern called ‘Forgetting hypermedia’:

The first indicator of the “Forgetting hypermedia” anti-pattern is the absence of links in representations. There is often a recipe for constructing URIs on the client side, but the client never follows links because the server simply doesn’t send any.

It goes on to suggest that:

a client should have to know a single URI only; everything else…should be communicated via hypermedia, as links within resource representations.

You wouldn’t want to take this to an extreme, but it’s worth aiming for.

This approach helped simplify our code. We tweaked our server to write the resource URL into our XML, then had the Flex client simply navigate to the URL.

Furthermore, this solution was somewhat polymorphic - a link to any sort of resource could be passed back and the client would always be able to navigate to it.

Caveats

The only catch was that we had to tweak the server to manually write the paths into the XML. As our server was a Rails app, we found it easiest to just add a ‘.xml.erb’ file to our views directory:

xml.results do
  @results.each do |result|
    xml.result do
      xml.tag! :name, result.name
      xml.tag! :description, result.description
      xml.tag! :edit_path, edit_polymorphic_path(result)
    end
  end
end

Note the use of edit_polymorphic_path to ensure that the path to any sort of resource can be written out.

My name is Mark. And I like rich clients.

There, I said it. I don’t like web applications. I never got Gmail as an interface. It never quite worked the way I wanted it to. I like eBay as a service, but not as an experience.

Over the last year I have found myself using NetNewsWire, not Google Reader.  I use OmniFocus, not RememberTheMilk.  I like having my data available everywhere, I just don’t want to use a browser to see or manipulate it.

I get that the interweb has allowed us to explore, connect and transact. Awesome. But now that we have the data and services the focus must shift to how we use it. The usability of most web applications is still well behind that of rich clients.  We need to get better.

So, we are faced with two alternatives. Try to make web applications better, or move to web enabled rich applications.

I favour the latter. Here’s why:

Rich clients work faster. Rich clients have tighter integration with the keyboard. You don’t need to be connected to the Internet all the time. Enough?

No! They also have tighter integration with the device they are running on. An iPhone application knows it has GPS and an accelerometer. It can do things you can’t do through a browser.

So let’s use the device well. Let’s optimise the experience for the device we are on. If we are on a mac I want Growl support. If we are on Windows then save data frequently.

But that is expensive to develop I hear you say. Yes. I didn’t say this would be cheap. However, in recent years we have gotten much better at building services faster (Spring: thank you.  Rails: thank you. SOAP: are you still here?).  So now that we have saved all that time on the back end, let’s spend a little more time on the front end.  

Please?!?

The following is a write-up of the highlights during the Sun Developer Day which I just attended. The early day kicked off with the usual registration and light refreshments before moving on to the ballroom for the opening keynote by Sun’s Director of Technology Outreach, Reginald Hutcherson.

The keynote addressed the possibilities of JavaFX in the non-PC arena such as televisions and mobile phones or according to Reginald, “any screen you will ever come across”. While it is a bold claim for a new technology in a world with existing competitors such as Flex, Silverlight, GWT …etc, it would be interesting to see how JavaFX performs in 2009. The keynote wrapped up with the advocacy of using open-source technology which was not surprising.

The demo shootout showcased the more interesting bits of the event. The first demo demonstrated the capabilities of JavaFX such as widget animation, a Flickr demo where click-able images are downloaded on the fly while floating across a canvas and finally, a video puzzle game with the video playing in the background. Performance was great and seamless considering the fact that they were all done on MacBook Pros. The next demo, showcased a compiz/beryl-like desktop for open Solaris and virtualization using VirtualBox which is now owned by Sun. Next up, is a quick 5 minute walkthrough of using NetBeans to produce a JavaFX app. This walkthrough shows off the tools within Netbeans such as drag and drop code generation and on the fly coding and previewing which is really impressive from a usability point of view. A feature which allows importing of Illustrator-created graphics was also mentioned, although it was never demonstrated.

The next bit discussed on the direction Java SE is heading and covered topics such as closure, Java Modules (JAM - JSR 277), and the usage of annotations in Swing, such as event handling…etc. While the morning covered the main highlights, the rest of the day was spent on code demonstrations on various topics such as REST, more JavaFX, SOA, MySQL, Dtrace and xVM (Virtual Box). Also, we got to see many of the features within Netbeans which were used during the demos. Sun is trying to market Netbeans to a wider range of developers (PHP, Ruby…etc.), which may be a challenge where there are many who have already got used to the Eclipse environment.

In general, the demos were well conducted and interesting. However, there were a few that were done in a hurry or needed more time for explanation as they were heavy topics (eg. SOA with OpenESB and Java CAPS). Unfortunately, there was not enough time for a Q&A session. Its understandable Q&As can potentially take up more time, but I’m sure there is always time for 1 or 2.

Personally, my main aim for attending the event was to learn more about JavaFX. I was impressed with what I saw and I appreciate the fact that Sun is stepping up their game in the IDE market. This is important, if they are to compete with the likes of Microsoft’s Visual Studio. However, one itching question remains. Is JavaFX able to compete with other web technologies in terms of performance on older hardware or will it be another slow Swing app?

Time to try Ganymede install again as the current version has got itself very confused! Install from scratch should take no more than 10-15mins (excluding any download times). This should give a good eclipse setup to do GWT and Maven work.

• Ganymede - Latest Enterprise install eclipse-jee-ganymede-macosx-carbon.tar.gz
• Sysdeo - I still use the old sysdeo plugin for basic tomcat running, used it for many years and its just easy. Installing 3.2.1 from sysdeo
• Cypal - For GWT work, get the latest cypal plugin
• Maven 2 - Update Manager new site, http://m2eclipse.sonatype.org/update/
• Subclipse - Update Manager new site, http://subclipse.tigris.org/update_1.4.x

One of my pet hates in software development is repetitive tasks: a complicated deployment process, tricky configuration of an application, repetitive editing motions that are just a little too messy for a find/replace. All of the above have, in the past, killed my concentration, dulled my senses and otherwise numbed my brain. I quickly realized that learning the ins and outs of a few powerful tools can take away a lot of the drudgery that sometimes crops up in my day-to-day work. Further, combining these tools makes it possible to automate just about everything that I need to do on a regular (or even not-so-regular) basis.

What’s important about everything that follows isn’t necessarily the process/tools itself, but what you get out of learning them. This is just what works for me, what has accumulated over a few years.

0. Pick a Good Editor (or IDE)

This first point is a half-hearted gesture because I don’t actually expect anyone to listen to me.

Use vim.

In all seriousness, if your editor doesn’t support basic macros and/or keyboard recording you’re going to encounter situations where you get slowed down by dull, repetitive editing. Even in an IDE that supports refactoring, macros will save you time and brain-dead copy/pastes. Automating repetitive tasks in editing source code is one of the simplest ways you can help yourself.

1. Learn Your Shell (or Learn Your IDE)

I think the Pragmatic Programmer said it well before I even knew what a “shell” was: the command-line is powerful.

Becoming very familiar with a powerful shell is one of the best things you can do for yourself as a developer. Yes, the syntax can be funky and the semantics confusing but over time you come to love it. For me, it’s the Unix shell: in *nix, the shell becomes the glue between all the other tools you use to get work done — including custom scripts and programs that take care of the heavy lifting for you. Windows users might be interested in PowerShell, which is a similar offering from Microsoft — you have the power of the .NET framework at your fingertips.

I understand, too, that folks accustomed to IDEs tend to get a little irked by shell programming. I still strongly recommend giving it a go, but scripting or extending your IDE to automate certain tasks you perform on a regular basis is a great start. If you find you lose a lot of time uploading and modifying configuration files to various servers, you may want to write an Eclipse plugin to make that process as simple as the click of a button.

Whatever path you choose — learning the more exotic aspects of a shell or the plugin API of your favourite IDE — learn it well. The only way to learn it well is to use it all the time.

2. Learn a Dynamically Typed Programming Language

If your shell scripts start to look like a train wreck when you need to perform complex operations, you may want to consider falling back to a scripting language: Perl, Python and/or Ruby are all great choices — bonus points if you mix the three to meet your needs. Languages like these will give you a more structured environment in which to express what you’re trying to achieve (at the cost of it being slightly more difficult to invoke external processes than, say, Bash — a standard Unix shell).

I think it’s important to note that I mention “dynamically typed” languages here because they generally don’t have a separate compilation step: bytecode compilation occurs as part of the execution of a Perl/Python/Ruby program. Thus, a small change doesn’t require an ant/make call to pull your code together again. Remember: you’re trying to automate a process, not create more process. Build management is overhead.

By the same token, if your program needs low-level access to hardware or raw speed for processing large quantities of data — don’t be afraid to fall back to a language like C or Java if you’re sure of your reasons for doing so. Needless to say, situations such as this are rare but not impossible.

3. Appreciation of the Supporting Cast

ssh, scp, rsync, svn/cvs/git, diff, patch, wget, sed, grep, find … the list of handy *nix command-line tools goes on and on. These are smaller tools that — with a good shell — can easily be combined to automate more complicated process. The output of these programs can also be piped into a script of your own doing to perform more complicated operations; then, the output of that program can be piped into another and so on.

This is the big thing I miss when working in Windows: even with PowerShell in play to alleviate the pain that is DOS, it has terrible tool support out of the box. Having the .NET framework on hand is great, but even that’s no substitute for the *nix command-line tools mentioned above. Rant aside, there are ways around most of the shortcomings in Windows and it’s possible to emulate some of the more useful command-line *nix tools. It just means you may need to put in more effort than a *nix counterpart at times.

For folks interested in automating with their IDE, your “supporting cast” might mean third party Java/.NET libraries which your automation plugin(s)/scripts can use to perform more complicated operations.

4. Invest Time in Learning Your Tools

Really, it all comes down to learning the tools from steps 1-3 backwards. Windows, Linux or OSX — you should be an expert in how your tools work and ideally use them on a daily basis. Really getting to know your tools might mean stepping out of your comfort zone a little to learn the nuts and bolts of how your shell ticks, or taking some time out of your day to learn certain esoteric API calls, or the semantics of some new construct in your scripting language of choice.

The goal of such practice is to surround yourself with an easily extensible suite of tools where every action feels like it’s an obvious choice. These tools will help to take away some of the unavoidable “noise” on the fringes of any software development project, and let you get on with getting your job done.

5. Write Lots of Scripts to Automate Common Tasks

This is at the core of automation: combine the above tools to get things done. Copy files to a server using scp, have your script extract and install it via ssh, manipulate configuration files using sed, grep and perl before finally starting the application and storing the process ID in a file with a quick shell command. All without you typing much more than “deploy”.

Automated tasks should be easy to kick off, require little-to-no input from you and — ideally — be fast, so you don’t have to wait long to determine if anything went haywire.

Automate at the First Sign of Resistance

Once you become accustomed to your tools — and again, by no means am I implying that those described here are the best or the only — you’ll step back when you start feel some resistance performing a task throughout your day:

Hmm. I don’t want to have to go through that 12-step deployment process all over again.

It becomes second nature to whip up a script that not only does the complicated deployment but verifies that the application is up and running as expected, checking a process exists and scanning the logs for hints indicating that everything’s fine. Over time you should find your automated tasks are easier to write and “feel” much more reliable than any manual process.

And, hopefully, you’ll be happier for it.

We have a new release of the GWT/Spring Shine Reference project out here (ver 0.3), which is an important improvement on the previous releases.

• Now includes proper Data Layer : using Spring 2.5 annotations, Hibernate 3.2 and integrated HSQLDB runtime memory database.
• Upgraded GWT gui to include save/delete/select functionality linking into the exposed Spring services
• Maven now builds deployable war files for both the presentation (gwt) and service (spring) layers that can be dropped into a standard Tomcat install.

All of this should “just work” straight out of the box - tested on linux (fedora), mac and windows systems. Follow the “Getting Started” instructions on the project page.

This was just too painful! Unfortunately I would have to recommend to people to stay away from Ganymede if you use Subversion for version control until they sort out connectors, update sites etc. After installing on 4 machines (3 macs, 1 linux) i’ve finally got 3 out of 4 working - but one of them (one of the identical macs!) just wont fix and the others were such hit and miss affairs. In theory, all you need to do is add in the update site as detailed on the Polarian site

Update Sites - Direct your Eclipse update manager to both the following update sites…
* Subversive plug-in update site :
http://download.eclipse.org/technology/subversive/0.7/update-site/

* Subversive SVN Connectors update site
http://www.polarion.org/projects/subversive/download/eclipse/2.0/update-site/

I realise that its to do with license issues etc, but Eclipse really has to “just work” out of the box with Subversion soon. The update manager really needs work too. Whoever put the “Close” button where they did while hiding off the “Install” button needs to go on a design course - so fed up clicking “close” just when i’ve chosen “install”….!

The Problem

As I work, I like to be able to do very regular checkins to our Subversion server (ensuring the tests pass beforehand, of course). This gives me an easy fallback position if I make some change that causes test breakages. Sometimes it’s just easier to rollback and start again than to try and figure out how I broke it. This is especially the case when refactoring.

We use Crucible for our code reviews, connected to our Subversion server. The problem is that when I want somebody to review a new feature or bug fix, the reviews might need to span a bunch of these fine-grained changesets. Crucible allows you to have multiple changesets in a review, but I don’t want people to have to review every little changeset - I want them to be able to just look at the difference between the start point and the end point.

A Solution

To get around this, I’ve been using Git to track my local changes, a custom script to bundle my changes into a single changeset, then git-svn to commit them to my Subversion server when I am ready.

I’m not going to give you a tutorial on Git here, but here are the basic steps involved:

1. Get Git and git-svn.
2. Clone your remote SVN repository into a local Git repository. If your SVN repository has a lot of history, this may take a while - but this is a one-off event. Go get a coffee whilst you wait.
3. Once it’s done, start coding. At this point it might be worth reading some of the introductory Git documentation to get an understanding of how it works. Commit your changes to your local Git repository as per normal Git usage.
4. Periodically rebase to the remote SVN repository using
   git svn rebase

   . This will ensure you’ve got the latest changes in the SVN repository.

5. Repeat steps 3 and 4 until you’re happy with your changes. Create branches if you want, but don’t merge between them (we’ll discuss why later).
6. When you’re ready to check-in to SVN, first use a custom bash script that was written by my colleague Tom Lee. Why do this? Well, to commit from Git to Subversion, you’d normally just use:
   git svn dcommit

   However, the problem with this is that’ll it commit a new changeset to SVN for each Git commit you’ve done. We don’t want this. Instead, we want to bundle everything up into a single commit first.

   The way to do is to create a temporary Git branch, and merge all of your changes into it. This will squash them into a single changeset, which you can then commit to SVN in one hit. Whilst you can do this manually if you want, Tom’s script does this for you automatically.

   Put the script in the same directory as your other Git executables and make it executable too. You can then run it as follows:

   git-prepare-svn-commit -m ‘Added some new feature’

   It’ll bundle everything up into a single changeset with the provided commit message. Note that this script has only been tested with Ubuntu - your mileage may vary.

7. Now you can check it in:
   git svn dcommit

   and it’ll just commit a single bundled changeset to your Subversion repository.

Additional Benefits of using Git

Git has a number of features that are very appealing. One that stood out for me is the ability to very quickly and easily create branches locally and then instantly switch between them. Often I would have a number of branches going at the same time, each one for different code enhancements or defect fixes.

In theory I could probably do all of this using Subversion branches, but Git offered one additional advantage - I don’t require a network connection to use it. I regularly work with it offline, only having to find a network connection when I’m ready to upload my changes to Subversion. I also have the added bonus that I don’t end up with a bazillion branches hanging around on the Subversion server.

It’s worth noting that I’ve never merged between local Git branches that originated from the Subversion server - indeed, the git-svn documentation recommends against it (see the ‘Caveats’ section of the git-svn Manual Page). However, this wasn’t ever a problem for me; by committing finished work to Subversion, switching to a Git branch containing unfinished work and then rebasing against the Subversion repository, code could easily be transmitted between branches.

Drawbacks of using Git
I found Git to have a steep learning curve. In particular, it’s two-stage approach to commits took a little getting used to. This problem was exacerbated by the fact that I was using git-svn as well. The best example of this was the extreme bewilderment I experienced when I first encountered merge conflicts during a rebase against the Subversion repository.

The best piece of advice I can give you if you encounter such a conflict during a rebase is to do what Git tells you. Git will provide a number of ‘what you can do next’ instructions when it encounters a conflict that it needs you to resolve. Read these instructions carefully and follow them. I tried to short-cut them and ended up hopelessly tangled up.

Finally, there’s not much Git tool support out there at the moment, so you’re pretty much gonna be doing it all from the command-line.

Credits
Thanks to Tom Lee for breaking a trail to this solution - I knew what I wanted and thought Git might provide a solution, but Tom was the one who actually figured it out and put it all together. He also pulled me out of a few holes that I dug myself into during the learning process.