Archive for the Rails Category

On an early Rails project I elected to use instantiated fixtures. My rationale was that it would make my test code cleaner and easier to understand. Sure, test_helper.rb warned me that they would be slow, but how bad could it be?

Well, pretty bad as it turns out. The average execution time of my test suite (with 1052 tests) using instantiated fixtures was around 590 seconds. That’s almost 10 minutes.

Suspecting that it might be the instantiated fixtures that were slowing things down, I modified a couple of tests to see if they could be sped up by switching to standard fixtures. The results were promising, so I switched everything.

The average execution time fell to 123 seconds. That’s a factor of 4.77 difference. In other words, my test time dropped from almost 10 minutes to just over two minutes.

The funny part is that instantiated fixtures didn’t make the code that much cleaner anyway, the reason being that model names would often end up embedded in fixture names. For example, if I had a User model, I might end up referencing a fixture as follows:

@user_in_more_than_one_group

The fixture was named this way because it wouldn’t make much sense if the name wasn’t prefixed with ‘user’. But when you think about it, there’s not much difference between this and the following:

users(:in_more_than_one_group)

So when I shifted away from instantiated fixtures, my code got only slightly more verbose, but the test execution time dropped by almost 80%.

Now it could be argued that these aren’t particular good fixture names, and that if they are for testing specific scenarios, I’d be better off using something like Factory Girl or Machinist. That’s a good point, but unfortunately I’m not in a position to make that change at the moment.

The bottom line is that I got caught out by just how slow instantiated fixtures are. Your results may differ, but it’s definitely worth investigating if your tests are slow and you’re using instantiated fixtures.

This is a short one, but we’ve had to do it a couple of times so I thought I’d put it up.

As you may be aware, Rails 2.2 introduced a new format for test names. Where you once might have had:

def test_should_do_stuff
  ...
end

You can now have:

test 'should do stuff' do
  ...
end

We’ve found this much easier to read and type - especially when your test names start to get big, or would be more readable if they contained characters that aren’t valid in Ruby method names.

However, if you’ve got lots of existing tests and want to shift them across to this format, it can be a pain to do it manually. Unless you’ve got a Ruby script, of course:

directory = 'unit'
Dir.mkdir("#{directory}.new")
Dir.new(directory).entries.each do |entry|
  original_filename = "#{directory}/#{entry}"
  File.open("#{directory}.new/#{entry}", 'w+') do |new_file|
    File.open(original_filename).readlines.each do |line|
      new_file.puts(line.gsub(/def test_*(.*)/) do |match|
          "test '#{$1.gsub('_', ' ')}' do"
        end)
    end
  end unless File.directory?(original_filename)
end

..which you’ll also find in this gist.

If you go into your ‘test’ folder and run this script, it’ll create a new directory called ‘unit.new’, that contains copies of all of your original unit tests, converted to the new format. Change ‘unit’ to ‘functional’, and it’ll do the same for your functional tests. Note that this will not do tests in subdirectories. YMMV. Feel free to steal and modify as you see fit.

The Problem

A while back we had a Flex client that needed to be able to display search results received from a server. The server was designed RESTfully, returning XML results to the client. The Flex client would display these results nicely to the user, and when the user clicked on a result, their browser would be directed to a HTML page for editing the actual item.

The problem was that, when a client clicked on a result, we were manually constructing the resource URLs in the Flex client from the IDs buried in the search results. The Flex code looked a bit like this:

navigateToURL(new URLRequest(’/item/’ + result.id +”/edit”),”_self”)

Furthermore, we had an impending need to support lots of different types of resources, so our logic for assembling URLs was about to become rather convoluted.

A Solution

Put the URLs into the search results. It’s blindingly simple (when you think about it, we do it all the time with pure HTML sites), but requires a subtle shift in mindset when you’re writing a fat client.

The solution was prompted by an article on REST Anti-Patterns I had read on InfoQ. - in particular, an anti-pattern called ‘Forgetting hypermedia’:

The first indicator of the “Forgetting hypermedia” anti-pattern is the absence of links in representations. There is often a recipe for constructing URIs on the client side, but the client never follows links because the server simply doesn’t send any.

It goes on to suggest that:

a client should have to know a single URI only; everything else…should be communicated via hypermedia, as links within resource representations.

You wouldn’t want to take this to an extreme, but it’s worth aiming for.

This approach helped simplify our code. We tweaked our server to write the resource URL into our XML, then had the Flex client simply navigate to the URL.

navigateToURL(new URLRequest(result.edit_path),”_self”)

Furthermore, this solution was somewhat polymorphic - a link to any sort of resource could be passed back and the client would always be able to navigate to it.

Caveats

The only catch was that we had to tweak the server to manually write the paths into the XML. As our server was a Rails app, we found it easiest to just add a ‘.xml.erb’ file to our views directory:

xml.results do
  @results.each do |result|
    xml.result do
      xml.tag! :name, result.name
      xml.tag! :description, result.description
      xml.tag! :edit_path, edit_polymorphic_path(result)
    end
  end
end

Note the use of edit_polymorphic_path to ensure that the path to any sort of resource can be written out.

Ruby on Rails comes with a lot of nice helper methods for generating the JavaScript driving the AJAX calls to your controllers. Handling the responses from the HTTP server becomes a snap too, with Rails providing a few simple callbacks to handle the response from the server:

But what if the server doesn’t get a chance to respond at all? What if the user’s browser has been unexpectedly disconnected from the Internet? What if your server - god forbid - has crashed?

Most AJAX applications in the wild will simply sit there forever waiting for a response. Certainly, that’s what happened to us when we first tested such a scenario. This is unacceptable: if the browser is unable to communicate with the server, we need to let the user know somehow.

Internally, Rails uses a cross-platform Javascript library called Prototype to do the heavy lifting when making AJAX calls. When you call a helper method like link_to_remote, Rails generates code to instantiate a Prototype Ajax.Request object (or a Ajax.Updater object in some cases) to make the remote call when a certain event occurs. In the case of link_to_remote, this will be when a user clicks on the generated link.

Many of the helper methods accept a common set of options which provide callbacks for handling success or failure using the following callbacks:

  • :success will be executed if the server responds to the AJAX request with any sort of 200 HTTP OK code.
  • :failure will be executed if the server responds to the AJAX request with anything *but* a 200 code.
  • :complete will be executed after the AJAX request has finished, irrespective of the result.

These callbacks can be mapped to corresponding callbacks in Prototype on a one-to-one basis. For the above callbacks, the corresponding Prototype callbacks are called onSuccess, onFailure and onComplete, respectively. Unfortunately, only the “complete” callback will be triggered if some fatal error occurs when communicating the server - “failure” will only be triggered if we get a response back from the remote host indicating a server-side. We have no way of determining what will happen if the browser is unable to communicate with the server at all.

So how do we detect server outage? Well it just turns out that if your browser is unable to communicate with the server, an exception will be thrown. We can actually use this as a basis for detecting server outages and other classes of errors that might occur browser-side. For this purpose, Prototype provides onException which is triggered by your browser if any sort of exception is thrown while the AJAX call is in progress. If we can tap into this callback, we can ensure our AJAX methods will dutifully report any problems that might occur.

Unfortunately Rails doesn’t yet provide access to this callback out-of-the-box, so there is no “:exception” callback available to the Rails helpers. We don’t want to clutter our templates with ugly JavaScript: we’d prefer to keep with the existing Rails conventions if we can. So what can we do?

PrototypeHelper#build_callbacks is used by remote_function to generate the Javascript responsible for making an AJAX call in helper methods like link_to_remote, form_remote_tag and others. It turns out that this little method exposes virtually all the callbacks available to Prototype *but* onException.

This is the hook we’re looking for: here we override and adapt the original code in PrototypeHelper#build_callbacks to add the following method to our ApplicationHelper:

include ActionView::Helpers::PrototypeHelper # ... def build_callbacks(options) callbacks = {} options.each do |callback, code| name = 'on' + callback.to_s.capitalize if CALLBACKS.include?(callback) callbacks[name] = "function(request){#{code}}" elsif callback == :exception callbacks[name] = "function(request,exception){#{code}}" end end callbacks end

This makes the :exception callback available to Rails using the familiar notation seen for the other types of callback. :exception is somewhat different in that it provides the exception object thrown internally available to your callbacks as the “exception” variable. Outside of that, it’s just like any other callback.

With this new callback available to us, we can then override the AJAX helper methods to include our error handling code in ApplicationHelper. As an example, here’s how we’d do it for form_remote_tag:

alias old_form_remote_tag form_remote_tag def form_remote_tag(options={}, &block) # In the event of a network failure (or some other error), we want to display an error message options[:exception] = (options[:exception] || '') + 'alert("Anerror occured while communicating with the server. Please try again later.");' + # Bubble the javascript exception up to prevent the caller from continuing execution (failure # to do this may result in multiple error messages being displayed!). 'throw exception;' old_form_remote_tag(options, &block) end end

With this code in place, any attempts to submit a form via AJAX will include our special error detection code, so we can rest at ease knowing our users will be notified if something goes horribly wrong at the network or language leve.

When using this technique to check, you have to be quite careful in your JavaScript code. Where any language- or browser-level exception that was thrown during the AJAX request was once ignored, it will be caught by the :exception callback. This means syntax errors in the response Javascript, bad content types (e.g. a content type of “text/javascript” with a body of HTML) or other errors will trigger the :exception callback. Unfortunately there doesn’t seem to be an obvious way to get more information about the nature of the error. For our purposes, however, this seemed to be enough if we were careful with the AJAX responses.

A further improvement to this approach might use a timeout mechanism to provide a hard limit for AJAX requests: if the server doesn’t respond with data within X seconds, abort the request and display an error. This would remove the reliance on the browser reporting errors via an exception, resulting in somewhat more robust code - the feedback may not be as immediate as if one were using exceptions, however.

Don’t leave your users out in the dark: keep them informed if things go wrong with an AJAX request.

Leopard and Rails


After having gone through the pain of installing Ruby on Rails on my MacBook Pro running Tiger (OS X 10.4) I was pleased to see that Leopard (OS X 10.5) ships with Rails ‘pre-installed’ and ready to go.

So, after I upgraded my installation on Friday night I was perplexed when trying to run my Rails apps. Why was it broken?

In the end, it turns out there were several problems.
(more…)

It takes almost ten seconds to start a Rails test on my Windows XP box. The test may report that it took it a fraction of a second to run, but it actually takes about ten seconds for it to start. The Rails Console and Mongrel have the same problem, but I can deal with that. However, with unit tests, it’s not acceptable because rapid feedback from tests is the key to rapid development.

I’ve had JUnit tests using Spring and Hibernate that start faster than this. And in Java, incremental compilers (for example, the one that comes with Eclipse) tell me immediately when I’ve stuffed up syntax. With interpreted Ruby on Rails, I have to wait ten seconds just to be told that I’ve made a typo.

To experience the problem, all I have to do is create a new Rails project, create a single model, and then run the test for that model. It takes ten seconds to start the test, give or take a second or so. Trying JRuby is not an option at this stage because at the end of the day I need to use ferret on my project, which requires native C extensions.
(more…)

Last month I attended a seminar about Virtual Worlds and in particular Second Life, during which it was suggested that Second Life had been through the hype stage, passed over the bell curve of growth and the “cool” factor, and was now on the other side of the curve with as many articles in the press concerning problems and issues than there were previously praising it.

Is the same happening with Rails?

For the last couple of years there has been a great deal written and tried with Rails which has been overwhelmingly positive (and in the main rightly so). However, I am detecting a swing in the coverage, and in particular the recent blog from cdbaby.com describing why a Rails re-write of a PHP system was completely abandoned, details here.

There have been a number of successes at Shine using Rails, and it is a great technology, but is it really that good a fit in the enterprise beyond new greenfield systems? Is there compelling reasons to replace the (in the main) Java systems we have built and maintained over the last 10 years? Is GWT a better solution for existing corporate systems - even just from the point of view that the retraining involved in GWT is trivial for a Java developer as GWT development is all written in Java? Or will improvements in Rails, for instance JRuby, allow greater integration into existing systems and hence speed its adoption?

So back to the original point, where on the bell curve is Rails….

For the first time in many years I am learning a new programming language - Ruby. I started about 6 months ago and it occurred to me the other day that they way I go about acquiring knowledge now is so much different from 1999 when I was getting into Java.

For me the standard process was:

  1. Buy Idiot’s Guide or equivalent
  2. Read book front to back, stopping very occasionally to try out some tutorials
  3. Implement a pet project in my spare time, and start buying good reference books
  4. Start to specialise and buy books about particular parts of the language I needed at that point in time.

    The Buggles - Video Killed the Radio Star

This practice worked well for me a number of times (Fortran, Pascal, C, COBOL, PowerBuilder, Delphi, VB and Java). The only problem was, it was about step 4 where I actually started to get productive. Still, if it ain’t broke, why fix it?

Well, for a start there was a limited selection of good Ruby and Ruby on Rails books. I used the excellent Agile Development With Ruby on Rails to get started, but before I had even gotten too far in I was already branching out to write some code and really never got back to it. Why would I, when I writing code was so much fun?

Of course, I kept coming up against blind spots in my knowledge. So I went to the web and started searching. Inevitably I found a blog item somewhere with my exact problem and at least one potential solution. Good, but still not that satisfying.

Then I found Railscasts, an episodic vodcast to help you learn Ruby on Rails. Forget that it is about Rails, the concept just works. Nothing beats watching someone code and having them talk you through the process. This is the exact format of the episodes, and thanks to some nice technology (you get to see the ‘hidden’ keystrokes in a Growl pane throughout), Ryan’s great technical knowledge and the tasty topics covered it is fair to say I am hooked.

Imagine having a guru in your company talking through best practice techniques weekly. Awesome. In addition, even just watching how Ryan Bates works is sometimes interesting.

So, my new learning process is:

  1. Watch video
  2. Play with code
  3. Go to step 1

For me, this has resulted in me not only being more motivated (”I’m writing code already!”) but my retention of the information is much better since I am implementing as I learn.This isn’t an isolated experience for me. In addition, Luke from Shine put me on to a YouTube video on how to use Quicksilver (a must have for Mac users). In a short period of time I learned a couple of really interesting tips that wouldn’t have seemed so useful written down in a book.

Another interesting example was when I was searching for more information on a Rails plugin called Chronic. A great package to parse language into dates (”Yesterday”, or even “last monday at 2am”), but it doesn’t have a lot of support out there in the blogs yet. I did, however, come across an entire video presentation that Tom did to a Rails User Group. Through that presentation I got an overview of how the entire plugin was built in a much shorter time frame than having to read all the code ‘cold’.

So I guess the question is, will we even need the Idiot’s Guide to X in the future? Maybe all we need is a basic introduction, get playing and go back and watch the videos as you need them. This seamless blend of learning and experimentation seems to me to closely match the learning process rather than the sterile chapter by chapter rote learning approach.

So what do you think? Are you using video to learn instead of books? Has video killed the Idiot’s Guide?

Gaining entry to a sessionBen and I attended a session on the state of Ruby Tooling. Again, since this was a presentation from Sun employees, they admitted up front that it was really a NetBeans presentation. Still, they did a good job of identifying other alternatives so I won’t give them too hard a time.

Overall, I am probably lucky to be still conscious. I tried really hard to follow the details of what they are doing, but these guys were the sort of people that rockets scientists say ‘wow, those guys are geeks’. So, let me try to explain why.

The Code Completion Issue
Ruby is a dynamic language, so compared to static languages like Java it is pretty hard to work out what methods are available since you can’t use the static typing. So, the easy answer is that you start with just plain string matching. If you type ‘file.o’ and try to code complete the IDE will find any method that starts with ‘o’ - everywhere (Ruby, gems, your code). OK, this falls down pretty quickly since the results will overwhelm you. So, these guys then started to add more intelligence to the searching to narrow down appropriate results. Interestingly, they used Lucene to do this.

The IDE uses a number of approaches to apply weighting to results so that the code completion is returning fuzzy results in the best order it can determine. In some cases there are ‘exact’ matches as well as ‘fuzzy’ matches. The methods they use to weight are quite frankly berserk. For instance, to help work out the type of an argument (they are all of type Object until execution) they look for instances where the method is called from and inspect the types of the objects that are passed in. They can then determine the common methods on the different object types to try to hint the most likely object types. Riiiiight. That was just one of half a dozen techniques they are using, so safe to say that the fuzzy matching is quite good and will likely get better over time.

It is probably worth noting a couple of special cases to the above. For common Rails librarys like ActionController (but they didn’t mention ActievRecord!), they are documenting methods and return types and if they determine that you are using one of these types they will provide exact matches. In addition, they are supporting an annotation standard where if you use the right tags (similar to JavaDoc) in your method headers then the IDE can use this to display the valid methods. This is a little tricky since the implementation is going to win over the documentation every time so if the documentation is wrong it could lead you astray.

Debugging
NetBeans and most of the alternatives (command line, RadRails/RDT, even TextMate) have debugging. Apparently there is a ’slow’ and a ‘fast’ debugging library and depending on the IDE it might have used one or both. There is a lot of work going into the debug-commons project trying to create a single, fast debug library that everyone can use so over time all IDEs should have fast debugging. For now, I don’t think it is a big issue so we can pretty much mark off debugging as being available.

Refactoring
This is something that all the vendors/projects are scrambling to get to. Now that debugging and code completion are mostly done, they are moving on to refactoring. As per the above discussion, refactoring also poses some pretty interesting difficulties. Despite this, the NetBeans guys think they have a way to make method renaming possible (they already have the trivial case of local variable renaming). We also spoke to an Aptana guy in the JavaOne Pavilion who reassured us that the project was still live and that they are hoping to soon release refactoring support. I don’t have any details on what that might be, but it is reassuring to see that the project is still alive for all those Eclipse fans out there. Interestingly I asked him about code completion for ActiveRecord and he suggested that they would be using information in Migrations to determine the attributes rather than going to the database directly.

Summary

In summary, NetBeans has debugging, decent code completion and very little refactoring. I would expect to see better refactoring in the near future, probably by the end of the year. Even still, these functions in Ruby are always going to be ‘fuzzy’ compared to similar functionality in Java so there is going to be some adjustment by developers to get used to the new approach.

A day ago I attended a presentation run by a couple of young Sun engineers entitled ‘Java Platform Web Applications Vs Ruby on Rails: This Time With Tools’. They were big fans of Rails, and set about examining two important questions:

  • Can you develop a JSF webapp as rapidly as a Rails one if you have the right tools?
  • Is the Ruby and Rails tooling sufficient?

By ‘tools’ they really meant NetBeans, but this was understandable given that they are Sun employees.

They gave a little demo of NetBeans generating the skeleton of a JSF web application, in a similar manner to how Rails can generate an application skeleton. And superficially speaking, the resultant directory tree certainly looked no worse than that generated by Rails.

The chief concession they made was that with the Java solution you need several frameworks, whereas with Rails you get a full stack. What they didn’t mention was that with Java, support for important things like migrations, fixtures and integration testing is missing. This is where I think that Rails still has an edge.

They ran the server from NetBeans, demonstrating that compilation and deployment of JSF apps is now down to around the 10 second mark, making development speed on par with Rails a possibility. They did admit, however, that memory problems and server restarts are still an issue.

With regards to the Ruby on Rails IDE support, I think that NetBeans has definitely advanced the state of things. I’ve already covered the NetBeans Ruby support, but the Rails support is also pretty cool, although it’s really just a wrapper for the Rails infrastucture.

We already know that Rails offers a great environment for rapidly getting a web application up and running. Whilst we acknowledge that Rails may well be subject to the 80/20 rule, frankly I wasn’t convinced that the NetBeans JSF support was helping get a new JSF project very far at all.

As a framework Rails still seems more potentially productive. Consequently the real benefit of NetBeans that I see is not that it can build a great JSF app for you, but that it provides great support for Ruby and Rails - which is already a great framework.