ReinH Blog

Incremental Stories and Micro‑Releases

2008-08-29T17:14:23-04:00

Agile estimating and planning is difficult. One of the practices that we are learning and adopting at Hashrocket that has truly improved our ability to ship is micro-releases. The idea, championed by author Joshua Kerievsky, is that constantly planning and shipping small, feature based incremental releases within the iterative cycle drives the delivery of tangible business value. In other words, the most important stuff gets shipped sooner.

Committing to micro-releases ensures that we are constantly working on delivering the highest priority features that represent real business value in an agile, incremental way. Complex stories that won’t fit into a micro-release cycle are broken down into stories that deliver core value and supporting stories that can be deferred to a later iteration.

A Complex Story

As A User I Want To View A List Of Projects
So that I can find a project that interests me

#1

Acceptance:

All projects shown in list
List is paginated
List sorts by age by default
List is sortable by the following criteria
- Age
- Owner
List is filterable by the following criteria
- Status
- Category

Cost: 4 Points

This is a reasonably good story. It is clear, concise and deliverable. Its acceptance criteria are testable. It adds real business value. A short time ago ago I would have been quite happy to write stories just like this. That’s not the case today.

Now that our development process is becoming more geared towards focused micro-releases, the value of incremental stories is becoming exceedingly clear. Writing incremental stories allows us to break complex stories apart into a first increment focused on delivering core value and a set of secondary stories that can be added as later increments to deliver supplementary value.

A Simple Primary Story

As A User I Want To View A List Of Projects
So that I can find a project that interests me

#1

Acceptance:

All projects shown in list
List is paginated
List is sorted by age

Cost: 1 Point

This story is smaller and simpler but it delivers most of the value of the original complex story. Users can browse a list and find projects. Additional value is certainly provided by selecting sort and filtering criteria. But that’s the point: these provide additional value. The simple primary story is an increment that delivers on the core value behind this feature: users want to discover projects. It’s the minimum functionality needed to start using the feature.

Some Simple Supplementary Stories

As A User I Want To Sort The Project List
So that I can better discover projects based on certain criteria

#1.1

Requires:

A list of projects

Acceptance:

List is sortable by the following criteria
- Age
- Owner

Cost: 1 Point

As A User I Want To Filter The Project List
So that I can browse only projects that match a given criterion

#1.2

Requires:

A list of projects

Acceptance:

List is filterable by the following criteria
- Status
- Category

Cost: 2 Points

One interesting and important observation is that the primary story is a one point story while the secondary stories total to three points. This means that these secondary stories represent fully three fourths of the original value of the complex story. In other words, shipping on the core value of the feature becomes much, much easier once it’s broken up into increments.

Ship Leaner Features, Sooner

Breaking complex stories into simpler incremental stories becomes especially important in situations where your iterations are extremely time-sensitive. This is precisely the case in our 3-2-1 Launch projects. Breaking stories into increments allows us to deliver more of the application’s core value sooner. It also helps clients make real-time decisions about the importance of the primary and secondary value stories separately. Once they can get their hands on the core functionality they often re-prioritize the secondary stories more effectively.

In the case of this particular story, once the basic list and pagination were out there – and once project search (another core value feature) was implemented – it became clear that the secondary project listing stories only duplicated functionality. These secondary stories are cooling off in the icebox as we speak. Meanwhile, we shipped a feature four times leaner and four times sooner.

Hack && Ship

2008-08-27T21:31:09-04:00

When the OG Consulting guys were down at Hashrocket working on our latest 3-2-1, they introduced us to a pair of bash scripts called hack and ship that they use to streamline their everyday git workflow. They’re so useful that we adopted them immediately and we’ve been using them religiously ever since. I estimate that these little scripts save me about an hour a day and, what’s more, they make it easy to follow the commit early, commit often mindset that’s so useful to the agile process.

Simple Software Process

Here’s our typical workflow (before hack and ship):

Start a feature in Pivotal Tracker
Checkout a working branch for this feature
Write a test
Write an implementation to make the test pass
Repeat 3 and 4 until the feature is complete
Commit changes to git with a useful commit message
Checkout the master branch
Update the master branch
Checkout the working branch
Rebase the master branch into the working branch
Checkout the master branch
Merge the working branch into the master branch
Push the changes to the origin repository (usually on github)
Repeat.

This is pretty simple (if a bit longwinded). We each probably do this dozens of times a day. Could it be simpler? You bet!

Simplified Software Process

I’d like to introduce you to the Simplified Software Process:

While it may not be a very good process, it does have one thing going for it: it’s simple. We like simple. So when we saw Rick Bradley’s super simple bash scripts for automating these common git tasks, we jumped on them. These scripts are designed to work with the basic git workflow we outlined above. And they have awesome names.

Here’s hack.

And ship.

Now our process looks like this:

Start a feature in Pivotal Tracker
Checkout a working branch for this feature
Write a test
Write an implementation to make the test pass
Repeat 3 and 4 until the feature is complete
Commit changes to git with a useful commit message
hack
ship
Repeat.

We even wrote ourselves a little alias to make this even easier:

Which brings the workflow down to three easy steps:

Test
Implement
ssp

Combine this with a Continuous Integration build like cruisecontrol.rb and a deployment to a staging server that takes less than a minute – we love EngineYard – and you have the perfect recipe for agile, iterative, test-driven, micro-release oriented development. I’ve never been happier with my development process.

Hack && Test && Ship

2008-08-28T12:30:01-04:00

Devoted reader Kevin Barnes of OGC pointed out in a comment to my last post that people make mistakes. This is news to me. In consideration of this, we have decided that it is important to test between hacking and shipping to ensure that any merging that goes on doesn’t result in a broken build due to other people’s mistakes.

When multiple teams are working on a project, two teams can commit mutually incompatible changes. These changes may be in different areas and may not cause actual merge conflicts but may still break the build. Git is dumb. It won’t fix these for you. This is why it’s important to run your tests after you merge.

I have thus updated the hack and ship protocol to include this important new step.

Running one last test before you merge your changes into master will also be a final catch-all and allow you to fix any problems before merging back to master. You can even git commit --amend them to prevent anyone else from seeing your own mistakes.

NB: hack && ship won’t ship if there are merge conflicts in the hack step. hack returns a non-zero exit code and the && prevents ship from running. Likewise, hack && test && ship won’t ship unless hack and test are both successful.

Thanks, Kevin.

Rocking the Hoedown

2008-08-10T14:02:38-04:00

We had a great time at the Ruby Hoedown this weekend. The Hashrocket crew was out in force (and there was much debauchery in the Hashrocket RV). We got to meet or get better acquainted with a lot of great rubyists and hear a lot of entertaining, informative talks.

Jim and Joe’s Mock Dialog

Jim Weirich and Joe O’Brien presented their talk about mocks and stubs in the form of a short play in three acts. This was very timely for me since mocking properly in tests is area we have been discussing quite a bit at Hashrocket.

One thing they said that struck a chord was that complex mocks are a code smell that can clue you in to problems or hot spots for refactoring in the code under test.

The presentation style was fresh and the tips they provided should be very valuable to anyone using mocks in testing (which should be everyone).

Ruby Best Practice Patterns

I’m afraid I played a little trick on poor Jeremy and my dear audience. Instead of spending an hour giving examples of patterns and practices, I thought it might be fun to take a more satirical slant on the material. The resulting talk, Unfactoring From Patterns: Job Security Through Code Obscurity, turned out to be a modest success (if I do say so myself) and I think rather more entertaining to boot.

I did have time to show a few actual patterns but the dialog during the question and answer period was especially valuable. David Black, Yehuda Katz and others made some great points and I really enjoyed getting a chance to discuss one of my favorite topics with such a receptive and informed audience.

We Ain’t Got No Keynote

Chris decided to forgo the slides to tell us a more personal story about his growth from a lowly PHP hacker to the successful rubyist, entreprenuer and open-source champion that he is today. Chris is an exemplar for me of the right way to go about creating a personal and company brand.

Chris’s years of passion for tinkering, experimenting and exploring really have payed off in a big way. I think his story could rightly be considered inspirational. I’m sure a lot of people left his talk and started side projects this very weekend. I really enjoyed his talk (even if he did tell people not to buy my book).

Flog Your Tests, Test Your Flog

Testing is such a crucial part of my process that I was very excited to hear Rick Bradley’s account of his rather epic battle to add tests to flog. The irony here, of course, is that flog itself is a testing tool.

This talk combined two of my favorite topics: testing and refactoring. Rick’s war stories had some great takeaways: the importance of integration tests to characterize the behavior of the existing system, the ways in which code that is written test first differs from code that is written without tests, the utter importance of testing all the f—king time. Bryal Liles will have more to say on that last topic in a bit.

I really can’t say enough about Rick’s talk. Hashrocket often takes on applications with little or no testing for our Rescue Missions and Rick’s deliberate, careful, comprehensive methodology is exactly the process we try to follow when we refactor and resuscitate our clients’ code back to health. Rick could very well write the Hashrocket Rescue Mission manual.

MIDI Machinations and Hungry Hungry Hippos

Giles Bowkett is a mad genius. Perhaps more importantly, he’s an artist who can ship. He lured us all to a talk about archaeopteryx, his amazing MIDI generator, and then proceeded to smack us all with the enlightenment stick.

I won’t spoil his talk for people who haven’t seen it. Suffice it to say that it was far more than I expected. Giles’s opinions on software development are unique and somewhat iconoclastic. Be careful, the code he writes might blow some people’s minds.

Giles’s presentation style is engaging and he can be down right hilarious at times. Also, hippos are scary!

Wrap It Up!

Taking a road trip down to the Hoedown with the Hashrocket crew was one of the best conference experiences I’ve had so far. The talks were great, but I think the after-conference festivities were the real hilight. Watching Obie and Jason play beer pong on a table constructed of pool noodles (watch the video), playing werewolf with a crew of drunken rubyists and spending time with some of my favorite conference friends. Who could ask for more? Also, Jeremy has big plans for next year’s Hoedown. It involves words like “free” and “Nashville” – I would be inclined to add “awesome” as well.

REST & Rails

2007-11-13T12:29:16-05:00

I am giving a presentation on Rails, REST, and the Resource Oriented Architecture today entitled “REST & Rails: Web Services for the Rails World”. Check out the slides.

To Design A Blog (act 1)

2008-02-25T15:22:24-05:00

Your blog’s role as personal soapbox is obvious; its role in establishing identity is often overlooked. As a designer, your blog speaks volumes not only by what it says, but also by how it says it. If you can’t design your own blog – they will say – how can you design anything else? It was with this in mind that I set out to redesign my own blog.

As an entry point for colleagues and potential clients, your blog’s first impression is crucial. Snap judgements will be made that you may never be able to escape. Your design must inform your readers as to your abilities with grace and flair. It must establish your visual style distinctly and succinctly. It must be both well orchestrated and well rehearsed. It must, in short, be great.

The Design Process

Before setting pen to paper, hand to mouse or fingers to keyboard, I began by thinking about what kind of response I wanted to elicit from my readers. I decided that want it to be warm and inviting, rich in information, and clean and elegant; it should be visually consistent, easy to read, and easy to navigate; most importantly, it should look like me.

This lead to a few clear design goals:

Strong, consistent typography
Simple, uncomplicated layout
Minimal but effective use of color

The design itself went through a number of iterations, beginning with simple pen-and-paper sketches. At the beginning, I imposed a few constraints to help focus my efforts: Single column, 800px width, gridded layout, three fonts. The content would reign supreme. Consistent typography would establish a visual hierarchy.

In The Beginning Was The Typography

(Figure 1.1)

Sorting out the typography goes a long way towards giving your design a strong, cohesive style. It is important to chose two or three fonts (and no more) and to use them consistently. It is also important to use consistent font sizing to establish a visual hierarchy.

I decided on Helvetica (specifically Helvetica Neue when available) for headers and Verdana for body text. Helvetica was chosen for its readability at medium to large point sizes and flat out gorgeousness. Verdana was chosen for its ubiquity and readability on screen at small to medium point sizes.

Rock Out With Your Layout

Most blogs use a two-column layout with a sidebar on the right to display subordinate information. Here, I choose a more minimal approach: a single column with information presented serially according to importance. The home page is streamlined to encourage the reader to discover subordinate content, as will navigation links at the top.

Also, the typography was further refined. A serif font is added for subtitles and meta information (like post date). Finally. the main heading font is pumped a bit to help the major sections pop.

(Figure 1.2)

A quick look at Figure 1.2 will reveal that the initial read is much improved. The main content sections (the latest post and the list of recent posts) stand out as clearly differentiated. There remain, however, some issues with the font sizing and the spacing of elements. Also, the footer’s columns appear rather haphazardly placed.

Coming up in act two, the layout and typography get cleaned up and we add a splash of color. Don’t miss it!

Custom Textile Tags For Great Justice

2008-04-09T20:56:01-04:00

fig. 1.1 | elem-typo-style.png

Taking some inspiration (and code) from the quirky but lovable _why’s original post on adding yer custom blocks to RedCloth and the inimitable Geoff Grosenbach’s foray into the world of custom textile figure tags, we’ve build a custom tag (or prefix) for this blog’s own figures, like figure-1-1) there on the right.

RedCloth’s textile implementation uses some simple metaprogramming to create its own tags, which allows you to (more or less) easily create new tags to suit your own needs. The basic formula is: write a textile_#{ tag } method, where tag is the name of the tag you want to create. This method takes four arguments, tag, atts, cite and content, which are parsed from the textile by the RedCloth engine.

For our purposes, we’re only concerned with content and atts, so I’ll leave the rest as an exercise for the gentle reader. The html we’re trying to create looks like this:

<div class="img" id="figure-1-1">
  <a class="fig" href="/images/image.jpb">
    <img src="/images/thumbs/image.jpg" alt="Figure 1.1" />
  </a>
  <p>Figure 1.1</p>
</div>

The tag we want to use to create it looks like this:

fig. 1.1 | image.jpg

That’s quite a bit shorter and more elegant. Jumping right in to the good stuff, the method definition for our new tag looks like this:

def textile_fig(tag, atts, cite, content)
  span_class = "img "
  if atts =~ /class="([^\"]+)"/
    span_class += $1
  end
  (figure_number, img_url) = content.split("|").map { |w| w.strip }
  figure_name = "Figure #{figure_number}"
  figure_id = "figure-#{figure_number}".tr(".", "-")

  %{
  <div class="#{span_class}" id="#{figure_id}">
    <a class="fig" href="/images/#{img_url}">
      <img src="/images/thumbs/#{img_url}" alt="#{figure_name}" />
    </a>
    <p>#{figure_name}</p>
  </div>
  }
end

Let’s break that down.

span_class = "img "
if atts =~ /class="([^\"]+)"/
  span_class += $1
end

This adds any classes in atts (in the form of class="foo") to the base class, img.

(figure_number, img_url) = content.split("|").map { |w| w.strip }
figure_name = "Figure #{figure_number}"
figure_id = "figure-#{figure_number}".tr(".", "-")

This breaks “fig 1.1 | image.jpg” down into two parts by splitting on the “|” and then normalizes them a bit to be used later.

Finally, the relevant parts are jammed into the html prototype and spit back as the method’s return value for insertion into your textile document (and for great justice, of course).

fig. 1.2 | ReinH-07.png

A later post might show the way __END__, DATA and if __FILE__ == $PROGRAM_NAME were used with the TextMate Ruby Bundle’s ⌘R command to preview the output of the textile tag as it was being written for a faster workflow or the jQuery used to activate the thumbnail’s hover effect while an inline link like figure-1-2) is hovered. But I wouldn’t hold your breath.

To Design A Blog (act 2)

2008-02-26T21:28:16-05:00

fig. 1.1 | ReinH-02.png

In our first installment, the typography and layout of the blog were mostly decided. With this ground work in place, it’s time to start focusing on the details. But first, it’s time to give our eyes and brain a chance to relax and reset. Staring at the same design for hours tends to give you tunnel-vision and we want to be able to look at the site afresh.

Now that we’re ready to continue, the most obvious issue is the arrangement of the sections in the footer. The rest of the design follows a strict grid system but the haphazard placement of the bottom columns is jarring to the eye and the whitespace between them is also problematic. figure-1-1).

fig. 1.2 | ReinH-03.png

A third column is added and the columns are properly aligned with the grid layout once again figure-1-2). Great success! The remaining changes are all relatively minor and mainly involve tweaks to font size, color, and spacing. Bonus points if you can spot them.

Now then: Welcome to the new blog! Enjoy your stay.

Adding Epicycles: Copernicus on Software Development

2008-07-28T19:48:15-04:00

The geocentric model of the solar system is a very old one. First postulated by Plato and Aristotle in the 6th century BC, the idea that the Earth is at the center of the universe was later refined and standardized in Claudius Ptolemy’s main astronomical book, Almagest. It is also an excellent example of a very common design failure in software development.

NB: The following contains a completely gratuitous lesson on the history of sclence. If you don’t like learning things, I guess you can skip a bit.

Ptolemy’s Faulty Assumption

It seems like a pretty obvious idea: sun, stars, planets… everything looks as if it goes around the Earth. Indeed, it was almost two thousand years before Copernicus was able to provide a convincing alternative.

Ptolemy’s Great Design Failure

As people began to make better and better observations of the motions of the various heavenly bodies, they began to notice that their movements were often far more complex than the Ptolemaic system could allow. The planets were particularly troublesome in this regard.

The logical step when a hypothesis no longer fits with observation is to reconsider that hypothesis and possibly discard it in place of a more fitting candidate. This is precisely what did not happen when Ptolemy set out to catalog the arrangements and movements of the celestial spheres.

What he did instead was add smaller circles that the planets move in, and then have these circles move on top of their orbital circles. These circles on top of circles – or epicycles – form the basis of Ptolemy’s new and “improved” geocentric model.

Ptolemy’s greatest mistake was not made in adopting the geocentric model in the first place. It was made in not abandoning it when it no longer fit observed fact. Rather than throw out his bad design, he added more and more layers of bad design to try to fit the new observations.

Copernicus’s Great Refactor

This process of adding more and more complexity onto complexity continued through iteration after iteration for almost two thousand years until Copernicus stepped in. He fixed the bad design at the root of the ridiculously complex and unwieldy Ptolemaic system by putting the Sun at the center. This simple change had a profoundly simplifying effect on our understanding of the motion of the planets and other heavenly bodies.

Copernicus’s heliocentric model is in retrospect a rather simple and obvious paradigm shift. Nevertheless, it marked the start of the Scientific Revolution and is often regarded as the starting point of modern astronomy.

Copernicus on Software Development

What does all of this mean for software developers? It means that if your system is built on a bad design, trying to accomodate that problem will only lead you to add more and more levels of bad design. Eventually the whole system will come crashing down under the weight of this accumulated technical debt.

This process of iteratively adding layers of complexity and bad design to attempt to shore up a faulty assumption or hopelessly flawed abstraction is called Adding Epicycles in honor of Ptolemy and his great design failure. The right thing to do is to follow Copernicus’s lead and correct the faulty assumption or poor abstraction that is at the heart of your design failure. The sooner you do this, the sooner you can start simplifying your code.

Correcting a critical design flaw will often cause these layers of circle upon circle upon preposterous circle of complexty to vanish, leaving your code clean and clear and simple. Just try not to anger the Pope.

Git It, Got It? Good!

2008-02-19T17:07:08-05:00

Last week, I gave a presentation on the git version control system. Apparently it went pretty well. The slides are available for those who are interested.

ReinH Blog

Incremental Stories and Micro‑Releases

A Complex Story

As A User I Want To View A List Of Projects So that I can find a project that interests me

#1

Acceptance:

Cost: 4 Points

A Simple Primary Story

As A User I Want To View A List Of Projects So that I can find a project that interests me

#1

Acceptance:

Cost: 1 Point

Some Simple Supplementary Stories

As A User I Want To Sort The Project List So that I can better discover projects based on certain criteria

#1.1

Requires:

Acceptance:

Cost: 1 Point

As A User I Want To Filter The Project List So that I can browse only projects that match a given criterion

#1.2

Requires:

Acceptance:

Cost: 2 Points

Ship Leaner Features, Sooner

Hack && Ship

Simple Software Process

Simplified Software Process

Hack && Test && Ship

Rocking the Hoedown

Jim and Joe’s Mock Dialog

Ruby Best Practice Patterns

We Ain’t Got No Keynote

Flog Your Tests, Test Your Flog

MIDI Machinations and Hungry Hungry Hippos

Wrap It Up!

REST & Rails

To Design A Blog (act 1)

The Design Process

In The Beginning Was The Typography

Rock Out With Your Layout

Custom Textile Tags For Great Justice

To Design A Blog (act 2)

Adding Epicycles: Copernicus on Software Development

Ptolemy’s Faulty Assumption

Ptolemy’s Great Design Failure

Copernicus’s Great Refactor

Copernicus on Software Development

Git It, Got It? Good!

As A User I Want To View A List Of Projects
So that I can find a project that interests me

As A User I Want To View A List Of Projects
So that I can find a project that interests me

As A User I Want To Sort The Project List
So that I can better discover projects based on certain criteria

As A User I Want To Filter The Project List
So that I can browse only projects that match a given criterion