I’d like to take a moment to publicly discuss my thoughts on Software Craftsmanship, a topic that has seen a bit of back and forth in the blog world after Dan North unleashed Programming is Not a Craft. You can find some of the responses at the bottom of this post by Martin Fowler.

The two main criticisms of the Software Craftsmanship Manifesto are that it encourages programmers to focus only on the code side, and the low barrier to entry to calling yourself a “Software Craftsman.” Allow me to address these points of criticism.

Criticism #1: Programmers will fall in love with the software at a technical level, at the expense of building software that makes users go WOW

Martin Fowler may be right that “those people who primarily self-identify as programmers feel a large part of their life is no longer important in the agile world.” They are important, obviously, but if they don’t feel important then that’s something that needs to be addressed. Part of me thinks that SC is a reaction to broken software organizations that discourage interaction between programmers and users, and so the programmers retreat into their coding cubbies and focus on the only thing they have control over - their technical practices.

Back when Alan Cooper was stuck in an MS-dominated paradigm, he wrote, “Programmers aren’t evil. They work hard to make their software easy to use. Unfortunately, their frame of reference is themselves, so they only make it easy to use for other software engineers, not for normal human beings.” Some people seem to fear that a focus on craftsmanship will lead to a world of well-tested, well-designed code that powers ugly software that is painful to use. I don’t think that’s the case. The common theme throughout Agile Roots 2010 was user experience, and how to get UX people working closely with programmers. The Agile testing crowd is trying to get the message out that testing is about understanding user expectations. When you get programmers, UXers, and testers collaborating, you have the holy trinity of software development - quality software that solves user’s problems, with no hidden surprises.

Criticism #2: Low barrier to entry to calling yourself a Software Craftsman

Anyone can sign his name to the manifesto and call himself a software craftsman. I can add my dog as a signatory if I want to. Apparently I’m supposed to believe that this cheapens the values expressed in the manifesto. The reality is that anyone can call himself anything he wants! I’m the President of the United States. I’m dating Angelina Jolie. I’m a member of the World Cup-winning US soccer team. You get the idea… Simply calling yourself something doesn’t make it so. Who decides what is a “true” software craftsman anyway? It’s up to you to decide whether people you interact with practice what they preach. The manifesto doesn’t say, “we are craftsmen” - it says “we aspire to be craftsmen.”

I can’t accurately assess someone’s technical abilities or theory of software development based on the presence of his or her name on the SC Manifesto. I’d be an idiot for trying. But I bet that I could share a beer with anyone on that list and have an interesting discussion about software. Even if they don’t test like I do. Especially if they don’t test like I do. It’s perfectly valid for people to hold different theories on how to build software. I think we’re all better off if we let those theories mingle with each other, evolve, and even blow up in our faces on occasion. At the very least, Software Craftsmanship provides one means of recognizing people who give a shit and are worth talking to.

What’s missing from this discussion?

SC discussions have focused primarily on the programming side, but it’s called the Software Craftsmanship Manifesto, not the Typing-code-into-my-One-True-Editor Manifesto. Programming is only one role in the software creation process. Wouldn’t it be great if testers and UXers could organize themselves into groups with hokey names, just like the Software Craftsmen? Then we could throw one big mixer and invent 99% of the world’s useful software. Perhaps they can rally under the banner of Software Craftsmanship. They should rally under the banner of Software Craftsmanship.

What are we really talking about?

I see the Software Craftsmanship Manifesto as an evolution of the Agile Manifesto. The key phrasing is not the “Not only…but” clauses, it’s “raising the bar of professional software development by practicing it and helping others learn the craft.” Let’s examine it in detail:

• raising the bar - Nobody has a clue how to objectively assess the best software people. That’s why licensing boards are so potentially dangerous to the industry, and why certifications are so useless. If we want to achieve progress, it is the practitioners that must take responsibility for elevating the industry.

• practicing software development - Most professions devote as much time to practicing as they do to performing. Yet for some reason we approach every day like we’re completely prepared for what’s to come, despite working on problems riddled with uncertainty. SC says that we must acknowledge, respect, and prepare for the difficulty inherent in our work. SC says that the state of the art changes so rapidly that you must continually hone your skills if you are to build useful software over a span of years or decades. SC says that we must not let ourselves become complacent in our abilities, or make excuses for cutting corners. If your programming environment doesn’t provide automated refactorings, then you had better practice common reactorings so that they become second nature to you, and so that you can execute them swiftly and safely on production code. Above all, SC says never to let “too much work” get in the way of “a job well done.”

• helping others learn - this is the crux of the manifesto, because software development is fundamentally an iterative learning process. When we share our experience and knowledge with others, we allow them to benefit from our past successes and failures. When we teach, we also learn, and the mixing of ideas allows us to expand the adjacent possible.

Story time

I was in Chicago for Agile 2009, the huuuge Agile conference filled with way more scrummasters than programmers. It’s where I met Corey Haines, or more specifically I was at David Chelimsky’s house pairing on RSpec when Corey Haines walked through the door and greeted me with a big hug. If you know Corey, you know that’s not a strange way for him to greet a stranger. Corey told me about the Software Craftsmanship conference taking place across the street from AgileConf and said that I should come. Some folks whose work I admired would be speaking, like Uncle Bob and Michael Feathers, and I would be exposed to other smart people in the software world who I hadn’t heard of before. I almost didn’t go, because I didn’t have enough money in my checking account to both pay for the conference and get my bills paid that month, but after a few hours of deliberation I held my breath and fired off an email to Corey asking if he could spot me the registration fee. He was in the middle of his pair programming tour where he basically worked for peanuts to put gas in his car to get from place to place, so he wasn’t exactly in the best position to lend me cash. I’m incredibly grateful that he did, because it was one of the best conferences I’ve been to in my life. There were talks about design, refactoring, compilers, required reading for programmers…Ken Auer even spoke about achieving a balanced life by picking four key areas of focus. Programmers…talking about life balance??? I thought we’re supposed to be locked away in our code caves for 72 hours straight surviving on cold pizza and jolt cola! Clearly this whole software craftsmanship thing is a different way of looking at software development. But how exactly?

The Software Craftsmanship Manifesto is valuable because it provides a goal - raising the bar - and a blueprint for getting there - practice, practice, practice, and help others learn. SC may run the risk of creating an echo chamber and excluding people who are capable of contributing valuable insight to the conversation, but I’m not afraid of that happening because at its core, SC is about sharing and being open to ideas for how to build software. Without SC and similar movements, we run a far greater risk of believing that we’re alone in the battle against crappy software, instead of recognizing that we’re part of a vibrant community of thinkers and practitioners who want to change the world.

I have always been a big reader. When I was a kid I used to stay up until 2am reading, and I always had a pile of books at the foot of my bed. When the Kindle came out a few years ago, I had to have one. Carrying around thousands of books at once? Yes please. Downloading new books and periodicals at the click of a button? Don’t mind if I do. Some people will argue that you don’t need to carry thousands of books at a time, and they’re probably right. My kindle has about 75 items on it. I generally read between 3 and 5 books at a time, because I have ADD and feel like reading something different every day, or at different times in the same day. Being able to carry those around with me in one device is a big win.

For the most part, I stick to non-fiction. My goal in reading any book is to acquire new information, or ideally to elevate my understanding. Those are different goals, and a lot of books that satisfy the first don’t satisfy the second. Either way, in order to get the most out of a book, I follow a basic process:

1. First, I do a quick skim of the book, reading the table of contents, the index, intros and conclusions of chapters, the final few pages of the book, and just generally flipping through to find interesting sections or visuals.
2. Next I read through the book as quickly as I can, bookmarking pages that seem important or are difficult to understand. I may highlight short passages for later reference, but I don’t really take notes at this point.
3. Once I’ve finished the book, I go back and look over the sections that gave me difficulty. This is where I begin to go into more depth, taking notes, writing down my questions and trying to figure out the answers to them.

This process comes pretty much straight from How to Read a Book by Mortimer Adler. I tell you what, friends give you funny looks when they see that book on your desk. I don’t mind though, because since I started reading this way, I am able to go through books faster than I was able to before, and I get a lot more out of them. Presumably this is the sort of skill I would have learned in college, but I didn’t attend for long enough.

Digital formats change the way I read

I’ve been using this process on paper books for a long time. It’s taken me a while to completely adapt it to digital books. At first I tried to pick a single platform to handle all of my reading. The biggest problem is that every digital reader has some strengths and weaknesses. The kindle provides the best reading experience for me. I think the e-ink is gorgeous, and over all it is the closest I can get to a real book. It’s better than a real book in a lot of ways, because the computer books I like to read are hefty and awkward to hold. The kindle has one glaring weakness for me: navigation. I can set bookmarks and highlight passages, and fairly easily jump between them, but it isn’t as natural or efficient as dog-earing pages or sticking my fingers in between the pages to hold a spot. The kindle provides an excellent linear reading experience, but I find it cumbersome to jump back and forth between different parts of the book.

The other major downside to kindle is the DRM-protected books you buy from amazon.com. I understand why companies do DRM, especially for a technology as new as e-readers, where nobody has a secure hold on the market yet. But the promise of digital books to me is that I can read content on a number of devices, each with different capabilities. Amazon has a kindle reader for basically every platform, but each one is still locked down with DRM, and most importantly I am limited by amazon’s ability and willingness to provide features.

I’ve started to buy more ebooks directly from publishers rather than amazon, because publishers often let you download the book in more than one format. Pragmatic Press comes to my mind first, but there are others such as Smashwords who have recently made all of Gerald Weinberg’s excellent books available in PDF and Kindle format for only 10 bucks. This is important to me because my process for reading digital books, after a good bit of trial and error, looks like this:

1. Load up the PDF on my computer and do my pre-reading, studying the table of contents and index, and skimming the book a little
2. Read through the book on my kindle as quickly as possible, bookmarking difficult pages or highlighting interesting passages
3. Return to my computer to go into the book in more detail, take notes, and all that

Steps 1 and 2 involve randomly accessing the book’s contents. This is not the kindle’s strong point, in my experience. On the other hand, kindle kicks ass when reading linearly, which is what step 2 is all about.

In addition to random access, PDF’s strong point is that it’s a standard format with excellent tool support. I have a simple use for it, creating rich text notes and copying the links to particular pages of a PDF to embed in the note, or just copying and pasting text from the PDF into the note. There are more advanced annotation tools that let you add annotations directly to the PDF, if that interests you.

When I can download a single book in multiple formats, I have the best of all worlds. Kindle lets me burn through books quickly and comfortably, with an easy mechanism for bookmarks and highlights to refer to later. PDF lets me pull it up on my computer and jump around, or copy and paste text and images, or links to specific pages. The multiple formats allow me to use the best tool for the job, so to speak. Because of this, I have tended to purchase non-fiction books directly from publishers that let me download a PDF version in addition to a kindle version. I find that I simply get more value from my reading when I can read books in this way.

What does this mean for readers?

I am interested to see how the digital book revolution plays out. I suspect that more authors will move toward self-publishing, for a variety of reasons. One is that they have more control over how their content is distributed. I think creative folks are coming around to the idea that it’s more important that their work gets exposure to a large number of people than it is to be fairly compensated by every single person who consumes that work. Put another way, most authors would rather sell 100k copies of a digital book and have 300k stolen copies out there, than sell 50k copies and have zero piracy. I have no clue if that scenario is realistic, but the point is that if you can earn double the revenue and have 8x the readership, then you’d be silly to focus your efforts on combating piracy.

The other reason is that I’m pretty sure traditional publishing is dead unless they embrace technology much more than they have so far. As Jeremy McAnnally points out in his eBook Authoring eBooks, he was able to publish his Rails 3 upgrade guide right as Rails was released, but the first dead-tree books didn’t hit stores until a year later. Many books, especially technical books, are available as “rough cuts” or something similar, but there’s no denying that the publishing industry in general moves at a snail’s pace when compared with digital publishing.

What about you?

I am curious to know what other people think. How are you reading books these days? Are you addicted to digital readers or do you still prefer paper books? If you do read digital books, what influences your purchasing decisions? Is it convenience, price, availability of formats, or something else? For authors or aspiring authors, how do you plan to publish your content in this day and age?

What is it with all of the presentations and blog posts titled “X sucks”? Don’t pretend that you created a content-rich presentation but slapped on a provocative title just for effect. If your goal is to immediately bias the listeners toward your position, job well done. But I guarantee you that it does nothing to add to intelligent discussion in the software development community.

My biggest problem with “X sucks” talks is that they are so fucking insulting to my intelligence. News flash: any tool or technique sucks when used incorrectly or in the wrong context. And presenting only one side of an argument is the surest sign that you either don’t know what you’re talking about, or that you’re a self-indulgent asshole. You know who else only presents one side of an argument, and discredits other opinions through hand-waving and strawmen? Politicians and lawyers. Congrats on joining such esteemed company!

When you stand up to present to your local user’s group, you have a responsibility to give them the highest quality information you can. This can be in the form of hard data, or experience, or references to other people’s work, and the ideal presentation will use a combination of these. Do your best to explain and argue your position, but make sure that you make people aware that there is another side to the debate, one that you aren’t presenting, that they should go search for themselves if they’re interested. I have yet to come across a software-related topic that can be fully explored in a one-hour presentation. Don’t insult your audience’s intelligence by presenting an extremist stance. At best they’ll think you’re an idiot. At worst you’ll fuel the FUD machine that has been plowing through the software industry all these years.

An interesting point about “X sucks” is that 6 months later, you will probably find out that you’re wrong! I find that desirable, as it demonstrates maturation as a developer and as a person. In that sense, “X sucks” might not be such a bad thing as it represents one step on your path to enlightenment. Just be sure to heavily qualify your talk as presenting your personal experience at this moment in time, and that there are plenty of people who disagree with you, and that a few months from now you may completely reverse your position. Realistically, you will probably bounce from one extreme to another, with the extremes gradually moving towards each other like the peaks of a fading sine wave, until you reach an equilibrium point. This equilibrium point is that fine line between “X sucks” and “X rocks,” where at any given moment you can only know which side you’re on based on tons of experience.

The next time you sit down to write an “X sucks” blog post or talk, please reconsider. Honor your abilities as a thinking developer by taking the time to advance our industry’s dialogue rather than making yourself look clever by trashing other people or techniques. We’ll all be better for it.

This morning started off not so hot when I discovered that iTunes had randomly deleted a few of my favorite songs. I went looking through Time Machine for them, and they were nowhere to be found. I managed to recover all but one of them using Senuti to pull the missing files from my iPhone and place them in my iTunes library.

After I did that, I was curious to know if I had any other broken files in my library. iTunes identifies broken files with an exclamation point – but only after you try to play the file! I spent about an hour googling trying to find a quick way to show me all the broken files. All the tips I found were from 2006 and earlier, and none worked on iTunes 10 on my computer. That’s when aniero suggested I write a script to parse my iTunes library XML file and check it against the file system.

Here’s the script I came up with. No guarantees that it’s perfect, but it worked for me:

If you look at any code I’ve written that takes advantage of open classes, you’ll see the following pattern:

Some::Class::Name.class_eval do
  def my_added_method
    # something here
  end

  def existing_method_with_my_extension
    # something here
    existing_method_without_my_extension
  end
  alias_method_chain :existing_method, :my_extension
end

I use class_eval because doing so ensures that my code actually does what I think it does. For this code to work, Some::Class::Name must already be defined, otherwise I’ll get an unknown constant error. That helps me avoid two time-wasting pitfalls: a bad load path, and typos.

Get your path right

When you extend a gem, you need to have the gem installed in the system and in your load path, and then require it. If you’re using Rails, it will automatically require it for you when you reference the constant name. By using class_eval, I know immediately when my environment isn’t set up properly, because I get an error.

Busting ghost classes

Another problem I run into is stupid typos when writing out the class definition, or getting the module heirarchy wrong. Something like this:

module Sum
  module Klass
    class Naem
      def my_added_method
        # something here
      end
    end
  end
end

In this case, Ruby is all too happy to create a second class for me called Sum::Klass::Naem even though I’m trying to extend Some::Class::Name. I call these “ghost classes.” I can’t figure them out just by looking at them, and they cause lots of mischief.

Order of extension

As a final note, when you extend a class, order often matters. It’s possible to load your “extension” before the actual implementation and have it work (assuming you fully-qualified the class name properly, including its superclass, otherwise you get that funky superclass type mismatch error). Order matters when you want to redefine a method, because Ruby will use the last method definition it comes across. That means if you and the gem both define #foo, and you load your extension before loading the gem, Ruby will use the gem’s version. In order to get the behavior you want, you need to load the gem first. This also applies to alias_method_chain, which would blow up on you because the method you want to alias doesn’t exist yet.

This is just one way to reopen classes in Ruby, it’s not “the” way. I’ve found it to be helpful in avoiding stupid mistakes, or catching environment issues, so I use it. It has the added benefit of being more explicit, I think, about the fact that I’m opening an existing class. I hope you find it useful, and if you’ve got other tricks for opening classes, please share in the comments!

Here are some quick thoughts after participating in this year’s Rails Rumble:

Avoid technical risk late in the game

You don’t want to bork your application minutes before the deadline. The first time I participated in the Rails Rumble, I broke the signup functionality at the last minute. Nobody was able to test out our app. That was a bummer.

That doesn’t mean you shouldn’t use unfamiliar technologies. By all means, take advantage of the weekend to learn something new. Just do your experimentation early on. I played with Neo4j’s Ruby integration for a couple hours leading up to the rumble, and then tried to make it work for a few more hours after starting. I wish it had gone smoothly, but it didn’t, so I scrapped it. Your technology stack should be settled by the end of day 1.

Create a stupid simple deploy setup

You shouldn’t be wasting time on deployment. Your deployment process should be one step, start to finish. We set up continuous deployment and it worked great. There was no need for anything heavy like capistrano to do the deploy. I just set up CI Joe and a github post-receive hook, so whenever we pushed to github CI Joe would run the tests, then cd into the production directory, pull the latest code, install gems, migrate and restart nginx. None of the other guys even knew what the deploy setup was - they just pushed their code, and a minute later it was live.

Keep your code clean

This is an interesting one because it sort of flies in the face of some conventional programmer wisdom, namely that code cleanliness doesn’t matter on short-term projects. The argument (I think) is that the code base is small and easily understood, there won’t be extended maintenance, and quite frankly there just isn’t enough time to refactor and keep the code clean.

That argument is, in my opinion, bullshit.

We did a good job of keeping the code clean, and paid special attention to the more complex areas of our code. We broke stuff up into extremely small objects (four of the core classes in the system were plain Ruby objects with less than 25 lines of code each), that at first I suspected might be decoupling overkill. Then we made three or four revisions to the core behavior and structure of our app, and I was sold. We were able to quickly make important changes that we probably wouldn’t have otherwise. This also plays into the point on avoiding technical risk.

Oh, write automated tests, duh.

Sleep

There’s definitely some machismo surrounding the Rails Rumble. People stock up on red bull and stay up all night in order to maximize their time spent coding. That’s cool if you want to, but don’t feel like you have to. I admit that the first night I was up until 7am hacking. I think I woke up the next day at around 11:30am. The fact is, if you’re really excited to work on something, your brain will force you to work on it and won’t let your body sleep. But at a certain point you’ll hit a wall and there’s really no sense in pushing past it. For creativity-intensive tasks like the Rails rumble, using caffeine to keep you up for an extra hour or two isn’t going to be very effective.

If you’ve followed my blog or seen me speak, you may already know that I’m not the biggest fan of ActiveRecord callbacks. If you didn’t know that, well, I’m not the biggest fan of ActiveRecord callbacks :)

As much as I love Rails, and in particular convention over configuration, it bums me out when people sacrifice basic OOP design in the name of convention over configuration [1]. Let me share a brief example.

Let’s say we’re building an app that allows people to subscribe to magazines. The three models we’ll use are Account, Magazine, and Subscription. They’re related as follows:

class Account < ActiveRecord::Base
  has_many :subscriptions
  has_many :magazines, :through => :subscriptions
end

class Subscription < ActiveRecord::Base
  belongs_to :account
  belongs_to :magazine
end

class Magazine
  has_many :subscriptions
  has_many :accounts, :through => :subscriptions
end

The Rails convention for creating a subscription would be

Subscription.create!(:account => some_account_object, :magazine => some_magazine_object).

So now that we’ve got the model set up, how do we send a confirmation email to the user when they subscribe to a magazine? Conventional Rails code would be something like this:

class Subscription < ActiveRecord::Base
  belongs_to :account
  belongs_to :magazine

  after_create :deliver_confirmation_email

  def deliver_confirmation_email
    SubscriptionMailer.deliver_confirmation_email self
  end
end

Nothing surprising there.

In the interest of keeping this short, I’m just going to show an alternative approach to solving this problem, and keep the commentary to a minumum. It’s pretty simple: just write a behavioral method.

class Magazine
  has_many :subscriptions
  has_many :accounts, :through => :subscriptions

  def subscribe(account)
    subscription = Subscription.create! :magazine => self, :account => account
    SubscriptionMailer.deliver_confirmation_email subscription
  end
end

This approach neatly solves the callback problem, and the resulting code stays true to the domain language. People don’t create a subscription to a newspaper, they subscribe to a newspaper. If there’s a domain-specific action called subscribe, shouldn’t there be a corresponding method somewhere in the code base? I think so.

There are, of course tradeoffs. We’re sacrificing the automatic transaction semantics that callbacks give us. It doesn’t work with resource_controller, or other frameworks that rely on conventions. Rails programmers new to your team might be confused by the use of extra methods instead of just going through the ActiveRecord API. The first two I think can be solved with a little bit of code. If you’re doing something like this a lot, you’ll figure out a way to make it nice and clean. The third one is easily solved by pair programming (as is basically every problem of knowledge transfer across a team).

One question you’ll encounter with this approach is where to place the methods. After all, we could have just as easily chosen account.subscribe(magazine), and indeed reading that aloud is perhaps more natural than magazine.subscribe(account). I’ve found the book Streamlined Object Modeling to be an excellent resource for thinking through this kind of problem. It goes into great detail on how and why to structure different kinds of object collaborations.

[1] Any suggestions for how to shorten “convention over configuration?” CoC has an unfortunate pronunciation… conv / conf maybe? We need a convention for this.

A while ago I read some Cucumber examples that Elisabeth Hendrickson wrote. I noticed something about her style that was a bit different from how I had been writing steps up to that point. Here’s a quick example:

Scenario: Successful registration sends email
    Given I am at the home page
    When I sign up with the email address "pat@example.com"
    Then "pat@example.com" receives an email

The difference is in the Then step, which I would have always written as

    Then "pat@example.com" should receive an email

She says “receives” while I say “should receive.” A holdover from all the RSpec examples I’ve written over the years, I guess.

I like Elisabeth’s style because it is more declarative and straight-forward. I did have one hesitation though, and it quickly surfaced in practice as I began to adopt her style. Some of my Given steps (meant to set up initial state) used the same language as my Then steps (meant to verify outcomes). What do you do when the steps are ambiguous?

Let me show you an example of what I mean. Here’s a version that uses the “should” language:

Scenario: Successful registration adds user to the system
    Given there are 0 users
    When I sign up with the email address "pat@example.com"
    Then there should be 1 user

If I drop the should, I’ll get something like the following:

Scenario: Successful registration adds user to the system
    Given there are 0 users
    When I sign up with the email address "pat@example.com"
    Then there is 1 user

Now I’ve got a problem. Somewhere I was using the following step matcher:

Given /^ there are 0 users$/ do
  User.destroy_all
end

but my Then matcher will also match that line:

Then /^ there (?:is|are) (\d+) users?$/ do |num_users|
  User.count.should == num_users.to_i
end

In cases like this, I’ve begun a convention of renaming my Given steps to “Given that.” Rewriting the example, it would be:

    Given that there are 0 users
    When I sign up with the email address "pat@example.com"
    Then there is 1 user

Now I’m able to write my Then step without “should,” and Cucumber doesn’t complain about ambiguous step definitions.

Occasionally I will change the language of the Given step definition, if I want to handle one or two special cases. For example, instead of writing “Given that there are 0 users” I might write “Given there are no users” and have that step definition destroy any users in the system.

At RailsConf 2009, BJ Clark and I gave a talk about working with legacy Rails apps. In that talk, we spent some time talking about technical debt. Ward Cunningham originally coined the term 18 years ago, and it has enjoyed a resurgence in blog posts and conference talks throughout the past year.

I’ve noticed that most discussions about technical debt, including BJ’s and mine, miss the mark when it comes Ward’s original point.

Ward invented the Debt Metaphor to explain how a software project benefits from delivering software early on in the project’s lifecycle. He was working on WyCash, an investment portfolio system that modeled a complex set of financial instruments. Because Ward and the rest of the programming team were programmers and not financial experts, a good deal of their effort went into learning the complexities of the domain they were working in. This sort of learning is best done iteratively, by creating software and observing how it works, and then taking what you learn and investing it back into the software. Steve Freeman, listening to Ward speak about technical debt, captured this process:

• Use what you know
• Feel it work
• Share the experience
• Wait for insight
• Refactor to include it

The most interesting bit to me is “refactor to include insight.” We tend to interpret refactoring as improving the design of existing code, but if you listen to Ward talk it’s clear that he literally means changing the factorization of the code over time. As his team learned about the problem, they modified the program “to look as if we had known what we were doing all along, and to look as if it had been easy to do.” It was crucial to him that the software reflect the team’s current understanding of the problem, and that it be continually updated to reflect any new insights they learned.

So where does the debt metaphor come into play? At the begining of the project, the programming team knows very little about the domain problem. Any code they write will look very different just a few iterations later. Writing code before you have a complete understanding of the problem allows you to jumpstart a project. It’s like taking out a loan, which allows you to get moving sooner than if you had to have all of the necessary capital upfront.

Technical Debt is not a license to write bad code with the intention of cleaning it up later. In fact, successfully using the Technical Debt metaphor hinges upon a team’s ability to write clean code. The code needs to clearly and concisely reflect the team’s current understanding of the problem, so that as they learn more they can easily modify the program to reflect their new understanding. It’s important to make sure the code doesn’t drift from the team’s understanding of the problem. Failure to do so, as Ward puts it, results in “continually stumbling over the disagreement [between code and understanding], which slows us down and is like paying interest.”

Other interpretations of technical debt suggest that interest payments come from the time taken to understand ugly code. That wasted time, and the bugs that typically come out of ugly code, are more like penalties on the debt. Whenever a team has to deal with ugly, buggy code, they are penalized for not writing the cleanest code they possibly can.

Michael Feathers gets it right when he says, “we need to see clean flexible code as an asset and count it as an asset.” Ward believes this is the practical approach, but as a lot of folks know it often turns out to be much harder in practice. Whether it’s deadlines, dispersed teams, or unfamiliarity with a new set of libraries or technologies, it’s easy to let code rot.

Keep this in mind the next time you hear someone talk about technical debt. It sounds great in theory, and if you listen to Ward’s words it makes a lot of sense. Let me caution you, though, to beware of any suggestion that it’s okay to commit unclean code, even in the name of shipping sooner. Software development is difficult enough as is, and we already have a term for giving undue weight to our present situation versus the future: being irresponsible.

Also, be sure to check out Ward’s video explanation of the Debt Metaphor

Corey Haines is a good friend of mine, and as some of you know last year he went on his Pair Programming Journeyman’s Tour. I’ve said to myself numerous times, “one of these days I’m going to sell up all of my stuff, pack my dogs up in an RV and travel around the country Corey Haines style.” I keep waiting for the right time, but my dogs would be hell to deal with, as would my girlfriend who is not fond of me skipping town for 6-8 weeks, even if it is to hang out with a bunch of computer nerds. So the other day I thought what if I didn’t make such a radical lifestyle change, and instead just made week-long visits to various programming shops or folks that I’m interested in working with. I brought that up with Nicole before going to bed and she liked that idea much more than my RV adventure, and that was that.

Well today I’m happy to say that it’s about to begin, or maybe is already underway. The amazing Elisabeth Hendrickson is hosting me for a week at her Agilistry studio to continue work on a demo project she made for GoGaRuCo.

Maybe I can visit you?

Agilistry is my first stop on this new programming tour. Next up is…well, I haven’t looked that far ahead yet :) I have a couple places in mind that I’ll get in contact with. In the mean time, if you know of any cool places that I could work at for a short time and learn some new stuff, shoot me an email at my twitter handle@me.com

Just so you know what I’ve got in mind, my plan would be to work alongside you and/or your team for a week. I’m trying to get exposure to new environments, be around smart people, etc. I won’t doing this for pay. That said, I can’t afford to fly all over the country, so I’d give strong preference to anybody wanting to cover my expenses. So if you’re

1. Really cool
2. Interested in working with me for a week
3. Can cover my flight / accommodations / food

then let’s get in touch!

I spent a year working on a team that was distributed across four time zones. During the first six months I worked from home, and then I moved up to where the company’s headquarters are located for the final six months. I’m not going to rehash arguments about distributed teams here, I just want to share some of my personal feelings as a result of my experience.

1. Remote pair programming sucks

The technology just isn’t there yet to make remote pair programming consistently enjoyable for me. I feel like I operated at about 70% the effectiveness when pairing remotely as I did in-person. That’s probably being generous. Remote pairing is extremely frustrating compared to in-person (and that’s just the technology - it doesn’t take into account your pair’s mood that day!) Moreover, we had tough times synching up to pair. Because of our meeting schedule, we’d typically have one hour to pair in the morning. Then the east coast guy would go to lunch, and when he got back the west coast guy was taking off soon. Maybe another hour or two of pairing in the afternoon before east coast guy logs off for the day. Result: not very much pairing.

2. Culture is important

One of the most profound experiences I’ve had was working on a for-fun weeklong agile project when Dale Emery rang a little bell to announce that he had pushed a new feature to production. He explained briefly what it was, everyone clapped, and went back to work. After that, people started ringing the bell to announce new deploys. It was fun to have a moment to take credit for your work and receive the warm fuzzies from your team’s applause. If warm fuzzies don’t interest you, you might appreciate the rapid knowledge transfer that occurred as a result. In 10 seconds, the entire team got an update on the current state of the product. If the announcement sounded like it could affect a pair’s work-in-progress, they could flag down the announcer and have a chat to sort things out.

The key takeaway for me was that the Agilistry deploy bell developed organically. There was no proclamation that “though shalt ring the deploy bell upon deployment.” Even after Dale did it, nobody said, “hey we should all do that from now on.” Well maybe someone did, I’m not sure, but it certainly didn’t get codified up in any guidelines or something like that. Dale rang the bell and talked briefly. Then later someone else did it, and it snowballed from there.

One of my favorite things about working in teams is growing as a group. I am of course interested in developing my personal skills, but I think it’s more interesting to be a part of a team that gels and evolves from day to day. I think that kind of gelling is much tougher when the team is remote, and the evolution a lot slower.

3. Partially distributed teams need even more communication

At RubyWebConf a few weeks ago, someone made the comment that a partially distributed team has tougher communications challenges than a fully distributed team does. That seems to fit with my experience. When one or two people on the team are in the office and talking with the business people all day long, they get a lot more information than the rest of the team does. This can cause problems when a few of the team members are in sync and the rest are out of sync, or sometimes the remote guys get anxious because they know that there are conversations going on without them. There was more than one occasion where someone’s whole workday to that point was wasted effort because the PM decided to change the requirements, but nobody remembered to tell the guy working on it. Hey, they’re easy to forget sometimes, with nothing but a line item in Campfire to represent them.

A lot of teams are experimenting with going partially or entirely remote and are having varying degrees of success. I want to pair program all the time, and I like working in a room and talking face to face with my team, and sketching ideas out on paper or a whiteboard. So for the most part, I’m not interested in working in a long-term distributed team. Different people are going to make different tradeoffs than I am. And I know there’s a lot of work going into solving some of the challenges that I touched on. Avdi Grimm runs his Wide Teams blog with lots of articles and interviews about managing distributed teams (including an article on building an intentional culture). Naresh Jain talks about distributed Agile a lot - here’s a good slideshow from him on the subject. If you’re interested in making your distributed teams more effective, I suggest starting at those web pages.

If your application uses a web-based service, you probably have to pass the service an identifier so that you can correlate the service’s data with your app’s local data. When using ActiveRecord within your Rails app, the obvious identifier to use is the id primary key. There are a few reasons you may not want to do that:

• passing the id requires that the object actually be saved (which could mean manual cleanup if saving to the service doesn’t work)
• inconsistent identifiers if you move from a SQL database to a NoSQL store
• automated tests can conflict when run from multiple machines

There are ways to get around all of these issues but in my experience it’s simpler to generate a GUID to use as the service identifier. You can do it using the uuidtools gem.

First you need to add a new string field with a limit of 32 characters to your model. Then automatically populate it with a before_create:

Then you just use it when creating your record. Here’s a snippet from a controller in which I create a local ActiveRecord object, as well as an ActiveResource object that uses the GUID identifier:

I just got back from the RubyWeb conference in Snowbird, Utah, the birthplace of Agile - where fourteen software nerds got together and drew up the Agile Manifesto. Alistair Cockburn gave a thought-provoking talk in which he said that to him the Agile Manifesto marked the end of something rather than a beginning. I’m still not entirely sure what that means. According to Alistair, the group wordsmithed it until every single member agreed on every single word, and it went from inception to delivery in something like six hours. That’s getting your dinner, as Alistair put it.

RubyWebConf is one more notch in my belt of regional conferences, and one I’m definitely looking forward to for next year. There were about 60 folks in the conference room which meant a venue intimate enough to enable heckling but not so large I felt stressed out. At events like RailsConf I spend most of my time tracking down folks I know because (a) I like to hang out with my geek friends that I rarely get to see and (b) the sea of people makes me feel awkward initiating conversations with new people. That’s why speaking at those conferences is so good - if you speak in front of a group of people and do a decent job someone is bound to approach you afterward. Solved! The small venue at RubyConf meant I was more comfortable chatting with strangers (we’re just geeks hanging around after all), and I kept running into the folks I talked to, so I could continue a conversation thread and be introduced to still other people. Long story short, smaller conferences are a lot of fun for me.

The following are a few thoughts I had after participating in the RubyWeb conference.

Novice presenters are good for the conference

I was impressed with the quality of presentations at RubyWebConf (a result of the single-track format imo). The content was great but the speakers themselves were excellent as well. I can definitely pick out the novices from the more experienced presenters, but I think the novices did a great job over all. Teaching is one of the best ways to learn, and even when I already knew the content of a talk I appreciated the effort people went through to create and deliver useful talks. Plus, as someone who’s given a few talks myself, I know how nerve-wracking it can be to speak to a crowd of intelligent, critical peers! By the way for anyone speaking at conferences, or anyone who wants to do more than sit in front of emacs/vi/whatever all day, checking out your local Toastmasters clubs is a good idea. It costs like $30 for a year and you’ll improve your spoken communication abilities like whoah.

It’s okay to disengage a bit during talks

Some talks I’m more interested in than others. Some may be interesting for me but too much to process at that point, or something I want to think about and explore at another time. I’m so stoked that Coby at Confreaks has been recording so many talks and putting them online. It’s tough to know which ones to watch, but I think he’ll be adding some features to help with that like showing # of views or integration with Speakerrate or something like that. One of the reasons I go to conferences is to get an overview of what’s going on, with minimal noise. Then afterward I can look through the slides from the talk, or rewatch a video, and dig in to those topics more deeply. I’ve noticed that a lot of other people focus on the talks more and take notes in a notebook or on computer though, so to each his own. Most people do this for the first day or half day though, it’s difficult to sustain that concentration over several days. Plus with the conferences people during the workweek, we all get emails or phone calls from people requesting our attention (I support weekday conferences though - I treat conferences as an important part of my career, and I enjoy spending my weekends at home.

Soft skills matter

Joe O’Brien gave a great talk on communicating with customers and teammates. One of my favorite parts was how minicultures like the programming community have erected a set of beliefs and customs that bond us but that can limit the frequency and effectiveness of interactions with people outside of our culture. We should embrace our culture, but be aware of our idiosyncrasies and how they can affect our perception of other people, or their perceptions of us. I’m a bit of a communications geek you could say, but I think it was an important talk for everyone, and Joe delivers his message with a great deal of passion and insight.

I go to technical conferences and I want technical material. There are other aspects of our world though that we can benefit from learning, in particular for people who want to build businesses. Frankly I think that I and a lot of the people I know are too smart to not only make money while we’re working, and technical chops alone won’t earn us money while we sleep. One soft skill per day, delivered by an advanced speaker, is the sweet spot as far as I’m concerned.

That’s all I’ve got for now. Big ups to Mike Moore and the rest of the RubyWeb conference crew (whoever they are - it couldn’t have been Mike alone, could it?!). They put on a killer conference. Connectivity was great too, a conference first for me!

In the first edition of testing tips we take a peek at how to test ActiveRecord finders

Last week I had a fascinating conversation with Jeff Patton. It began with me telling Jeff that I think there’s a rift between the people who know Agile (the folks who have been doing it for the last 10 years), and the people who can actually be agile (Rubyists, for example). Agile consultants spend their time trying to get shops to do TDD & iterative development, which are taken for granted at a lot of Ruby shops (though the quality and extent to which this is done may be up for debate). I think it would be interesting, to an Agile expert like Jeff, to work with a team that doesn’t have the same sort of organizational friction to adopting Agile practices that he may be used to.

Jeff’s reply shocked me:

“The Ruby community cares about building high-quality apps, but doesn’t necessarily care about shipping high-value apps.”

Jeff went on to say that the Ruby community is obsessive about craftsmanship. This is a good thing, of course. We test. We write clean code. We take the time and care to build applications that are beautiful and do what our customers ask for.

Therein lies the rub: what customers ask for is rarely what they want, and almost never what they need. As Henry Ford put it, “If I had asked what people wanted, they would have said faster horses.” Or as I put it, your customer may pay you $1000 to deliver him a knuckle sandwich, but no amount of precision or strength training is going to leave you with a happy customer.

It turns out that constructing a high-quality application is not enough - you have to conceptualize and design an application that users will actually find useful. Doing this is every bit as difficult as constructing the software, if not harder. It requires a combination of research - generating new ideas from asking questions & identifying problems - and feedback - testing out ideas you’ve created. The Ruby & Agile worlds have been primarily focused on getting user feedback, without doing the all-important research.

Perhaps the most interesting thing he said was that there’s an important shift in responsibility that needs to take place. The people who know software need to take more responsibility for the success of software projects, and that means being more active in the research, conceptualization, and design of products. He said that there are a number of well-known “elite” development firms out there that will gladly cash your company’s $250k check and build an app that they know is not likely to make any money. It’s easy to point to the numbers in order to shirk responsibility - 90% of businesses fail within a year of opening, and 90% of the ones that do make it past the first year fail within the next five. The numbers are pretty consistent with Jeff’s experiences. He claims that fewer than 10% of products built by consultant shops reach a point where they actually make money from the product.

He told me a story of a project he worked on a while back with some really smart people, and the project was regarded, by the programmers at least, as an “Agile success story.” The team did all the practices, delivered software regularly, and had low defect rates. The only problem? Customers hated the software, and the business went extinct.

I’m not 100% sure what to make of his comments yet. I’m proud to be a part of a community that places such high importance on craftsmanship. But it seems clear to me that there’s a piece missing from our discussions. Maybe we’re doing the software equivalent of bicep curls, and effectively punching our end users in the face. Just something to think about.

Join the discussion on Google+