Shine Technologies Blog

Session 3: Build In Enterprise, Ant, Maven, Dependency Management

• Better builds with Maven has misled a lot of developers, it encouraged bad practice of using child modules. It was suggested that the book should be renamed to ‘Broken builds with Maven.’ There’s a new Maven book being written about using Maven the right way.
• Good thing about Maven is that it encapsulates procedural steps in a declarative manner.
• Teams that use Ant successfully tend to employ a certain level of standard and convention.
• Easy Ant - Ant with standards.
• If Ant works for you, there is no reason for adopting Maven.
• It’s always easier to figure out a Maven POM rather than trying to understand an Ant build script.
• Build is never easy, it looks easy, but it’s not.
• You want everyone in your team to be able to modify code, but you don’t want everyone to modify build.
• You shouldn’t avoid maintaining build script, the technical debt will be too much at the end.
• The quality of artifacts available from Maven central repository can be improved by enforcing tests to be part of the development culture, a la Perl testing and CPAN.
• Maven central repository needs to improve metadata quality, lots of broken POM.
• There’s an opportunity for an open quality dashboard system in continuous integration marketplace.
• How you build your product is often part of the audit process, specially when another company wants to acquire your product.
• Sonar is handy to calculate your technical debt.
• Crap4j measures code quality using an algorithm.
• Avoid setting latest/SNAPSHOT as a dependency to avoid crap release creeping in to your build, use version or version range.

Session 4: Continuous Integration Tools

• The pricing of CI tools can be roughly grouped into developer budget (i.e. free ones like Hudson, CruiseControl, Luntbuild, Buildbot), manager budget (i.e. not free but not expensive like Atlassian Bamboo), and director++ budget (e.g. Anthill Pro, IBM Rational BuildForge).
• The price gap between manager budget and director budget is pretty wide, this is because you can’t sell moderately expensive software.
• Interestingly, AnthillPro finds projects which already use Hudson to be a good starting point to migrate from.
• Deployment should be scripted, use the same script for all environments, use the script as early as possible (in dev environment).
• When you have any doubt about Hudson scalability, add more slaves.
• You’ll be surprised at how there are still projects out there without any proper build script.
• Git bisection is handy to identify point of failure and rollback.

Session 5: Continuous Deployment

• IMVU deploys to production fifty times a day.
• Flickr deploys 10+ times per day.
• When you deploy to production on each commit, you care less about zero defect, but you care more about having really good tests. And if there’s a defect, you handle each defect exactly once by adding more tests around the defect.
• Deploy on commit allows developer to concentrate on changes in small, more manageable, chunks.
• Feature flags give you the opportunity to push bug fixes to production with half-baked new features disabled. You can then enable those features in the future when they’re ready.
• Feature flag is also handy for doing A/B testing
• Deploy straight to production requires more care when there are database changes involved, specially on backward patch.
• Continuous deployment is not to be confused with cowboy development. Cowboy development is not about the frequency, but more on the lack of quality. Continuous deployment care about having good tests.
• Large corporate culture of having management signing off every little change can be a blocker to employing continuous deployment. One solution to this is to have the management sign off the process, instead of the change itself.
• Continuous deployment is not for all projects. It won’t work for legacy system lacking good tests. It won’t work for projects with large data that requires time-consuming post-deployment processing.
• It’s questionable how well the concept of continuous deployment is going to work in finance industry where a single error can have a legal implication.

Day 2 ended with a retrospective session where many of us highlighted how well the open space format worked, and how good the quality of the discussions is. My favourite sessions were the ‘Evolution of Agile and Continuous Integration’ talk facilitated by Jeff, and ‘Continuous Deployment’ talk facilitated by Nigel McNie. I learned a great deal from those discussions, I had my a-ha moments, and there were a number of ideas I could pass on to others in my organisation/team.

I believe that the main reason why open space worked really well was the participants, everyone who was there, they were there because they cared, and somehow the level of passion was higher compared to other events I attended in the past. CITCON rocked!

Day 2 started at 9am with the early attendees ‘refactoring’ the talks to fit into the schedule, this involved spreading the most voted talks into larger rooms, combining talks around similar topics, and at the same time making sure that no facilitator ended up with multiple sessions running on the same time period. There were about 30 proposed talks with 20 time slots, 4 sessions running in parallel per time period.

Refactoring session.

It was raining all day, a typical Wellington weather I was told.

And here are some short notes from the sessions I attended:

Session 1: ATDD and BDD

• A good way to make developers realise the value of BDD is by involving them in a project they’re not familiar with, that already have well defined tests/behaviours. Some open source projects can be a good example.
• BDD can be seen as a subset of ATDD, ATDD involves implementing requirements into executables.
• FitNesse works really well for some teams, others totally dislike it.
• 100% control of large test data (e.g. in finance industry) is often not possible, specially when the data can’t be easily copied (e.g. due to legacy system restriction), or when there are multiple teams utilising the same set of data.
• Some people strongly believe that web testing should be about testing the presentation layer only and hence can be made minimal, time is better spent on heavily testing the data/model. Others disagree and believe that a complete end to end web testing is always needed, nothing can replace humans.

Session 2: Evolution of Agile and Continuous Integration

• Check out Rapid Development by Steve McConnell, published in 1996, you’ll soon realise that there are still many companies which development practices are only 15 years behind.
• Pre & post chasm diagram, having started employing CI in 2002/3, did that make me an early adopter?
• At the beginning, most people identified agile as XP “Yea, we do XP!” Now, most people are able to differentiate between Scrum and XP.
• Use CI Maturity Model to identify where your organisation is at in terms of CI practices.
• That maturity model is also an excellent resource to convince others in your organisation that some existing policies need to be updated to allow transition to the next level.
• Back then, only developers adopted agile practices. By now, QA is already involved. Next up is operations and infrastructures. My personal take is that it will be 3-5 years from now.
• Nowadays agile is _understood_ by development team and QA, the problem is with governance.
• Jeff emphasized that the future will be in lean management. The key to lean is to reduce cycle time.
• Sometimes a rubber chicken is all that’s needed to do continuous integration.
• Would be nice to have a push-button deployment to production.
• Someone from operations group must be brought into the team, be involved in the iterations/cycles/sprints, just like QA/testing is now.
• Broken builds often get ignored, this is where team must figure out what works for them, whether it be lava lamps, flashing light, IRC notification, or other feedback mechanisms.
• The main problem with CI in enterprise is when it gets centralized and development teams suddenly lose visibility. Instead of teams running their own CI tools (e.g. a build server under someone’s desk), it’s now the build engineers who control those tools and often restricting access.
• When operations team is against the idea of transitioning CI to the next step, start with empathy and try to look at the situation from their point of view. Try showing them the business value behind the transition, involve someone from higher up in management.
• The culture of an organization is often a blocker to further transition in the maturity model.
• Know when to stop. When the culture resists further transition and you can’t change the culture, stop, you’re fighting a losing battle.

Continue reading CITCON Australia/New Zealand 2010 - Day 2 (Part 2).

I attended CITCON A/NZ 2010 in Wellington, New Zealand. CITCON is an unconference on Continuous Integration and Testing, it uses an open space format (which worked really really well), and CITCON rocked! (more about this later)

CITCON breaks Wellington

My flight arrived at 3.30pm, and day 1 activities started at 6.30pm. Paul Julius and Jeffrey Fredrick kicked-off the show by introducing what CITCON was about and how it was going to work, the rest was then up to us, the attendees.

PJ and Jeff.

The attendees then suggested the topics they would like to facilitate and voted on the ones they would like to attend. Given that it was my first open space conference, I wasn’t sure how open space could possibly work out. I read about how it’s _supposed_ to work, but I still wondered “really?”

I was introduced to the awesomeness of Sharpie.

The voting then continued during social hour at Foxglove. I had a chance to talk to Bruce Chapman who suggested an informal Hudson BOF session on the topic of the pain and pleasure of using Hudson plugins. In summary we agreed that Hudson core is solid, but there’s a need for better orchestration on those myriad of plugins, the question is who’s going to do it because clearly leaving it to the plugin maintainers haven’t really worked out (myself included, guilty).

Does a plugin need to have five features when most people only need one? Should a plugin do one thing only and leave it to the users to combine the plugins? There’s also clearly a need for better plugin information/metadata, it’s currently hard to find out which plugins to use if you need to do certain things. Perhaps a set of Hudson recipes need to be documented and shared on the wiki. Those were the questions and ideas that I would like to raise with others in Hudson community.

That’s it for day 1, we basically prepared the topics followed by some informal discussions over drinks.
Continue reading CITCON Australia/New Zealand 2010 - Day 2 (Part 1)