An Agile Functional Specification

In my previous post I described my experience as a business analyst on an agile project.

One of the key artifacts I produced on the project was the functional specification (FS). In this post I’m going to get right under the covers of the FS and explain exactly what it was and how it worked. My intention is to provide enough detail so that others could actually try it out on their own projects. So you’ll have to excuse me if I go on a bit – I’m sharing some love here. :) I’m not saying it’s perfect, but it worked for me!

To help this article make more sense, you might want to download a copy of the Functional Specification.

Can a Functional Specification really be Agile?

Is there any such thing as an agile functional specification?

Given that the agile movement values working software over comprehensive documentation, you might well ask whether there is any place for a functional specification on an agile project. In Extreme Programming, the requirements are conveyed verbally, directly to the developer, with just a few notes scribbled on an index card as an aide memoire. The only documentation is the code itself plus the accompanying suite of automated acceptance tests.

The debate as to how well this approach works in practice is ongoing. Do good developers make good analysts? Is no documentation enough? Does the analyst role help or hinder the overall process? I’m not even sure which side of the fence I sit on myself yet!

Either way, a “no analyst, no documentation” approach wasn’t appropriate for this particular project – primarily for geographical reasons. The business team was located in Leeds (UK) and the development team was 200 miles away in Exeter, with no real option to re-locate. Maintaining a rich verbal dialogue directly between the two groups was always going to be tricky, and it made more sense to have a dedicated business analyst (i.e. me) bridging the gap.

And besides, I would have written myself out of a job!

However, this was not like any functional specification I had ever written before. I was keen to avoid all the excess baggage that normally comes with an FS and try to keep things as pared down as possible. Let’s have a look at some of its key features, and hopefully you will agree that it is possible to have an agile functional specification.

Purpose and Objectives

It’s worth stating this explicitly, because different people seem to have different views on this…

The purpose of my functional specification was to specify the externally-visible behaviour of the system, in enough detail to allow the development team to build the system.

It existed alongside an (HTML) prototype which provided guidance on the user interface for the system.

It was in my view a design document (a functional design, not a technical design). Some people would call it a requirements document, but to me that’s just down to terminology, and perhaps mindset. But that’s another article.

Format and Authoring Tool

The first and most notable feature of my functional specification is that it was a spreadsheet. In the past I have always used a word processor for capturing requirements, so this was a bit of a change for me.

Why did I decide to do it this way?

Well, I knew I wanted to capture the system requirements as user stories, complete with acceptance criteria. And I also knew I wanted to create a living document that could easily be updated throughout the project – in particular the Product Backlog (the master list of user stories). I was initially considering using an agile CASE tool such as Mingle, but in the end I decided to go for something more low tech and see how it went. So I chose MS Excel. You don’t get much more lo-fi than that, although I suppose if you were a masochist you could always do the whole thing in Notepad! :)

Some of the benefits of using a spreadsheet will become apparent as I explain the other features of the FS – the ability to add columns for various purposes being particularly useful. But one of the key benefits came as quite a surprise, and deserves a special mention.

I’ve always found then when using a word processor to author a document (like now, for example), I spend a lot of time on the format, structure and grammar. I think that’s because, in my mind, poor presentation reflects poor content. What happens then is the document becomes an end in itself, rather than a means to the real end (working software).

But with a spreadsheet it was different. I felt no compunction to perfect the structure and grammar of each sentence, nor was there any real need to format the document very much. I was free to bang out the content and let the ideas flow. It was extremely liberating! And it saved a whole load of time too.

In summary, the spreadsheet allowed me to focus on content over form.

Story Index

The ‘Summary’ tab within the FS contains a list of all the user stories (in a Scrum project this would be called the Product Backlog). This is where new stories got added as I worked through the project’s Feature List (which was really a kind of wish list) and elaborated the features into something more concrete (more on the process later).

There are quite a few columns on the Summary tab. Most of them are self-explanatory, and some of them also have comments against the header to help out. Here are a few useful notes:

  • Story ID – I gave each story a unique ID, using the format ‘Snnn’ e.g. S001, S002 etc. The ‘S’ prefix turned out to be very handy because stories could be referred to by their IDs alone without having to bother with qualifiers like ‘story number’, as in, “Hey Joe, I’ve just been looking at S078 and I have a question…”
  • Story Name – A simple name for the story, rather like a Use Case name – which was quicker and easier to digest than the “As A…I Can…” summary.
  • Status – indicated where the story was in the delivery lifecycle – with colour coding – so we could all easily see which stories were ready for development, test, fixing etc. and which were withdrawn.
  • Priority – I gave each story a MoSCoW priority, which sort of helped to work out what order to deliver the stories in, but in reality deciding the order was much more complicated (and subtle than that) – we needed to take into account inter-dependencies and also which stories we most wanted to show to the users sooner.
  • Dev Increment – which increment the story had been allocated to. Of course, stories were only allocated to increments at the start of each increment once we had done the estimates and worked out what would fit.

Story Details

The ‘Details’ tab contains the meat of the specification. Each story from the ‘Summary’ tab is expanded out into its full form.

I spent quite some time trying to find this full form. User stories are typically defined in terms of acceptance criteria – which state what it means for the story to be “done”. But badly-written acceptance criteria are as bad as badly-written requirements, and there are two pitfalls that I have found:

  1. Ambiguity – not giving enough detail
  2. Lack of context – not saying when or how the acceptance criterion applies

So rather than having an unstructured list of acceptance criteria, I decided to work them into a tried-and-tested format: use cases. And in particular, Alistair Cockburn-style use cases. I used a two column format – the “When…” column states the actor’s (user’s) actions and the “Then…” column states the system’s responses (which also double up as the acceptance criteria). If I had multiple acceptance criteria against a single action, I put them all on separate rows, so that I could track them individually.

Here’s an example of a “Log In” story:

Story ID As a… I can… So that…
S001 User log in to the system I can gain access to restricted areas of the site
Flow Step When… Then…
Main 1 the user is not logged in
2 the user requests to view any “restricted” page
3 the system displays the log in page
4 the log in page includes data entry fields for user ID and password
5 the maximum length for user ID is 16 characters
6 The password field is a “masked” data entry field (it does not display the characters the user types)
7 the user enters a valid user ID and password, and requests to log in
8 the system changes user’s status to “logged in”
9 the system captures an audit trail of the date and time the user logged in
10 the system takes the use the the page they initially requested
7a 1 the user enters a valid user ID but an invalid password
2 the system re-displays the log in page
3 the system displays an error message: “Incorrect user ID or password”
7b 1 the user enters an invalid user ID
2 the system re-displays the log in page
3 the system displays an error message: “Incorrect user ID or password”

Notes:

  1. The “When…Then…” terminology is borrowed from Behaviour Driven Development, but really, what you are looking at here are use cases.
  2. The “Flow” column identifies the main (“happy day”) flow and the alternative flows of the use case. For example, flow 7a is the first alternative flow to branch off main flow step 7, and flow 7b is the second to branch off the same step.

I used to spend a lot of time deliberating over the exact language and level of detail in my use cases – making sure I used consistent terminology for all my actors and entities; trying not to get too drawn in to UI design. This time I threw caution to the wind and just wrote down, as quickly and as clearly as possible, what I wanted the system to do. Some acceptance criteria might be quite high level (e.g. “the system displays the log in page”) whereas other might be very specific (e.g. “the maximum length for user ID is 16 characters”). If there was some detail that I cared about, it went in. If I didn’t care too much, or if I was confident the developers would get it right without too much guidance, I left it out.

I also used to spend valuable time making sure my use cases were cumulative – so if I needed to modify or extend an existing system function, I would extend or modify the existing use case (being careful to mark up the changes to show what is new) – thus providing a useful document to refer back to in future. Whether or not this is worth the effort is a hotly debated topic. At the moment I fall on the “it’s not worth the effort” side of the debate, and so if I want to modify or extend an existing function, I just write a new story to describe the changes. Easy!

A Day In The Life

A large part of what made the FS agile is more to do with how (and when) I created it.

One key point is that I wrote the FS very much “just in time”. We were operating an incremental development process – each increment was 3 weeks long, so in any given 3 week period I was working on elaborating the stories that would go in the next increment. More precisely, I elaborated enough stories to be sure of having enough scope for the next increment – because the exact scope wasn’t known until we did the increment planning session at the start of the increment.

Stories were elaborated in priority order, and as we worked our way through the increments, the stories were continuously being re-prioritised, based on our unfolding understanding of the overall system. After some of the increments we made the system available to users (a sort of soft UAT) to get their feedback. This led to a whole raft of proposed tweaks, which were added into later increments as new stories – duly prioritised alongside all the other stories.

As mentioned earlier, I also developed an HTML prototype in parallel with the functional specification, which mainly served to illustrate the concepts in the functional specification but also provided look & feel guidance for the developers.

On this particular project, I was lucky enough to have ready access to my business stakeholders. So I ran twice-weekly workshops in which we went through the latest stories. Using my laptop and a projector, I had the HTML prototype up on the big screen for all to see and the FS on my laptop screen, so the stakeholders had something visual to look at but I had the details in the FS to refer back to. I would read out the acceptance criteria, get agreement around the room (or amend as appropriate) and then insert today’s date in the “business agreed” column in the FS. The design was signed off there and then, and the users never even had to read the spec. To my mind that was a big advantage of this process.

A Living Document

The FS was, of course, a living document – whilst the developers were using it to work on increment N, I was adding stories for increment N+1, and also amending the stories in increment N when the dev team spotted holes in the plot.

It lived on a shared network drive so that everybody could easily access it. To keep things simple, we didn’t use the “sharing” feature of MS Excel, so that meant only one person could edit it at a time (MS Excel handles the locking and notification for you – pretty nifty). Mostly this wasn’t a problem, but occasionally we had a bit of contention. That said, I don’t think this approach would scale beyond a team of around 6 people.

Testing

Not content with using a spreadsheet as a CASE tool, we also doubled it up to use it as a test tool too! The testers (myself and one other) used the use cases directly as test scripts, and if you look at columns N through R on the ‘Details’ tab, you will see that the spreadsheet also captured the results of testing, as well as any defects found. Again, this was a pretty lo-fi solution which wouldn’t scale that well, but it did have the big advantage that the developers were able to see exactly what has failed testing and in what context, without having to cross-refer to multiple documents.

Tracking and Progress Monitoring

If you know your agile, you will know that burn-down charts are all the rage for tracking progress. In my FS there was a separate tab showing a burn-down chart for each iteration. Actually, my charts were burn-up charts because they tracked work done, rather than work remaining, but the principle was the same. The burn-up charts were calculated automatically based on the date entered in column K of the ‘Summary’ sheet. Note that they only showed development complete, not test complete.

Retrospective

As I mentioned at the start of this article – I don’t claim that my FS is an agile analyst’s silver bullet. Here are some thoughts on its overall success on my particular project.

What went well:

  • Everyone liked the low-tech approach and lack of enslavement to tools
  • Business users enjoyed not having to read and sign off documentation
  • Business analyst enjoyed the freedom to be slap-dash with language and terminology!
  • No complaints over lack of clarity in requirements

Could do better:

  • Solution won’t scale well (beyond a team of around 6)
  • As the document grew it became a bit tricky to track through it looking for outstanding defects to fix/test.

The Future

There is a lot of talk these days about the system being documented as a series of executable acceptance tests. It seems to me that the next natural step would be to move towards the BA and/or QA writing these tests themselves (currently it is seen as a developer task). Behaviour Driven Development seems to be heading in this direction, but can we get to a point where executable tests can be written by a non-programmer? Or at least co-written? I have tried writing executable acceptance tests directly using Selenium, but I find that I get too bogged down in the technical aspects of getting the tests to work, and get drawn away from the analysis task. So we’re not quite there yet.

Downloads

You can download the example documents described in this article on the downloads page. The examples are real documents from a real project (but with the details obfuscated for obvious reasons).

Further Reading

There’s more to business analysis than just writing Functional Specifications. I’ve tried to capture the end-to-end process I follow in the article Business Analyst Designer Method.

I’ve also written an article describing an alternative approach using a Graphical Functional Specification.

And if you want to learn how to put what you’ve just read into practice, you might want to look at my Distance Learning Course.

69 thoughts on “An Agile Functional Specification

  1. Glen B Alleman

    An interesting issue. One question would be, what type of system development can support the direct definition of features in the absence of an overall vision of “done,” an architecture where these features live, and the discovery of the interdependence of these features?

    You mentioned limits in the last paragraphs, with 6 or so developers, would this team be working on a standalone development, or could they be embedded in a larger project.

  2. Tony Heap Post author

    @ Sudhi – you can download an example Functional Specification via the link near the top of the article – is that what you mean?

    @ Glen – if you have a look at my previous article, you will see the other artifacts I produced on this proved – a Vision and a Feature List, both of which provided context for the Functional Specification. Because this project was an enhancement to an existing system, the architecture was already defined.

    Feature interdependence is an interesting one – there were some features which played off one another, and we did have to go back and re-work a couple of earlier features to take account of discoveries we made in later increments. There was one feature in particular where I had the developer pretty much completely re-build it because I just hadn’t got it right first time round – which did feel like a bit of a waste of time. You would remove some of that re-work by doing all your analysis up-front, but of course then you would lose all the other benefits of incremental development.

    The limits I mentioned were purely down to the practicalities of using an Excel spreadsheet on a shared network drive. If you wanted to use this approach on a larger project, you would need to find a way of splitting the project down into small parts so that there were 6 or fewer people working on each part. On my current project (a large and complex eCommerce system), we are producing one Functional Specification per “feature” – where a feature is a coherent set of stories that together deliver some useful business benefit (some people would call this an “epic”) – containing somewhere between 1 and 20 stories. We aren’t using the Excel FS format on this project, but I could see it working.

  3. Nick Panagopoulos

    Nice article, Tony. These trade secrets takes years to perfect…I can’t believe you’re giving these secrets away!!!
    You’re approach seems to be well balanced. The amount of detail is always a challenge, and I think the level of detail you provide gives the user the information they need to know what they’re getting and the devs enough detail to do their work. Users love not having to read any extra documents; meetings work best with them. I like your review process where sign off occurs immediately, but I usually get users asking to see the end document, even if I’m reviewing with them live on the call – perhaps only because they are half paying attention.

    I recall in your earlier posts, you were using dreamweaver for your html prototypes. Why HTML prototypes? Only because this was a web app? Did devs use your prototypes as final code sometimes? Many BAs I know won’t bother trying to create HTML mockups.

    A question to you and your team for that project: if they were to work with the same team, would they use the same method again?

    Finally, I think that the maintenance of such a specification over the lifecycle of a product is difficult. Describing a new story instead of changing an existing story, as you suggested, may leave some steps out, which can be important to the business or the product…Is it up to the BA to decide what is important or not?

  4. Tony Heap Post author

    @ Nick – glad you enjoyed the article. I’m more than happy to share my “trade secrets” – where would we be as a race if nobody ever helped anybody else out? :)

    This was indeed a web app – HTML prototypes wouldn’t have been appropriate otherwise. I’ve discovered it’s by far the best way to convey the look & feel of a web to both the users and the dev team. The users are so much more engaged in the workshop than they would be with a wireframe or drawing because feels like they are looking at the actual finished product. And the devs love it because the look & feel piece is already done for them. I made sure I was using the same HTML conventions and CSS styles as the live application so the devs could use the prototype as a starting point for coding.

    It does require you to have HTML skills as a BA though, which I know not everyone does have. IMHO it’s worth learning. And it does take up more time than just doing a wireframe or sketch, so it depends on your available bandwidth. On my current project we have a dedicated UX person who produces the HTML mockups on behalf of the BA team, which is working well.

    Would we use the same method again? Well we went through 7 increments and after each one we had a retrospective to check & monitor the process, and every time we agreed that it was working fine. So I guess the answer would be “yes”. Mind you, on my current project we are doing things very differently and using Visio to produce some very visual Functional Specs, and I still haven’t decided which method I prefer!

    Re. maintenance – as I’ve mentioned before, these days I’m not convinced that maintaining comprehensive, detailed product documentation is a valuable exercise in the long term – and this was definitely a “throw away” spec.

  5. Michelle Swoboda

    Thank you Tony for a well written and informative article. As a BA I have learned from you – thank you!! I love to learn, to see what templates people and using. I share and I am happy that you share too.
    I am looking forward to an article on the use of Visio for this application. Currently I am working with requirements to provide a visual to them with Visio – it is coming together but it takes a while to think it through.
    Do you see this as replacing the final design document and sign off? I know we all like the diagrams in that document :-)

  6. Mike Hibbard

    Excellent article Tony.

    Having worked as a BA in the City of London these past four years, I can tell you that the financial technology space is crying out for this sort of marriage of a case methodology with agile development.

    Agile development is the method du jour for banking tech teams, but culturally the end users and – crucially – the business stakeholders still expect the usual lengthy document. It was thus in my last role I found I was communicating verbally to the developers what they needed to build, and then retro-fitting my Use Cases in the FS to describe what had already been built to keep the stakeholders happy!

    What I really like about your FS is that it fundamentally is a set of User Stories – so the developers are happy – but underpinned by quasi-Use Case descriptions – which still facilitate effective user testing. Crucially, this was light-weight enough for you to keep up to date as changes were discovered.

    – The acknowledgement in Agile that it’s okay to ‘discover’ requirements as you go along is for me, a revalation for business analysts.
    – What’s lacking – in my humble opinion – is the corresponding light-weight method for documenting requirements that allows for this. I think you may have just found this…

    As a final comment, I’d say that choosing excel because it was lo-fi was probably a very smart move. Had you chosen a CASE tool – even one specifically for Agile – I suspect you’d have reached a point where the overhead of keeping things up-to-date outweighed the benefits of doing so.

    cheers, Mike

  7. Yvonne

    Hi Tony,
    Thank you for your timely article. It is the answer that I was seeking in how to track the development and user stories together. It is in my mind “Traceability” of the requirements.

    There is little out there on the BA aspect of Agile and more on the Project Management and Development of an agile project. The aspect of tracking and managing requirements is always either glossed over or some hand-waving by many experts and authors.

    Please keep me in the loop about what else was done on your Agile project and the project turned out.

    With much appreciation,
    Yvonne

  8. Tony Heap Post author

    @ Michelle – yes, we are signing off Visio docs directly, and the doc *is* the specification – it contains a front page with version control and all that usual guff. I suppose it’s a bit like a (building) architect’s drawings – everything you need to know is in the drawings, or in the annotations on the drawings.

    @ Mike & Yvonne – glad you found the article useful, thanks for your comments. It does sound like there are people out there trying to find the “middle ground” between formal old school rigid methods and agile.

  9. Shireen

    Hi Tony,
    What an interesting read! As a Business Analyst, I believe in fit-for-purpose. Excel and Visio have always been my weapons of choice but I’ve always found them lacking/missing something. Your article has bridged some of those gaps for me. And for the stakeholders who still expect a lengthy FS document, your template allows me to quickly summarise into a Word doc. Thanks!

  10. Michelle Swoboda

    Tony, the signing off of the Visio is quite intriguing. Good for you for finding new and innovative ways to document for the project.

  11. Anh Doan

    Your articles are really a big guidance to me. As I just an Intern BA for Agile and I’m the only BA in my company. However, your articles seem know all problems I facing and help me solve each of them. I’m now just work for few months but become the product owner for a biggest project in the company and customer and my boss is please of my work.
    I figure out that agile developers prefer to work on prototype more than reading text, so sometimes I just create wire-frame and put descriptions on it.
    As an inexperience BA like me, there is still some problems I face, especially at the start of project:
    -One developer want me to do deeper analysis, he want me to analyse and do more UML, architecture analysis…He feel not comfortable with the ambiguity of the system architect (I think because he may too familiar with traditional style that provide big document and design upfront). When others care more about the prototype when they developing.
    – The growth of requirement and sometimes it tricky for me to fulfill the request from client, when the client pushing the time.
    – Underestimate the task in planning meeting because sometimes I did not have deep analysis on the function and may get it wrong.

    These are some problems prevent me most at the time, I think I will have some suggestions from you.

    Many thanks

    1. Yvonne

      Anh,
      I can completely empathized with you. My organization is going thorough the same “growing”/transition pains. The flip-flopping of between Traditional Waterfalll and Agile methodology is tough.

      However, your last point in your comment regarding the “- Underestimate the task in planning meeting because sometimes I did not have deep analysis on the function and may get it wrong.” has tweaked my interest.

      Have you or anyone else out there know how your can ensure that you captured enough information for the task to ensure proper estimation?

      I have asked my developers and often times they will “just enough” but how does one know when they have “just enough”.

      Thanks for your help in this matter in advance.
      Yvonne

  12. Tony Heap Post author

    @ Anh & Yvonne,

    In my experience, by far the most common reason for under-estimated stories/functions is a failure to properly analyse the alternative flows.

    The alternative flows are all the “failure” scenarios for a given function, and if you forget to capture them up-front, you tend to have a lot of conversations like this:

    BA: OK, so when they user enters their user name and password, the system logs them in.
    Dev: OK, cool, that will be 2 days’ work.

    Time passes…

    Dev: I’ve started coding this “log in” story and I was wondering, what happnes if they enter the wrong username and password?
    BA: Good point. Well “obviously”, the system doesn’t let them in.
    Dev: What do you mean?
    BA: Well, erm, re-display the page and display an error message.
    Dev: OK I didn’t account for that. That’s an extra day’s work. Now what about if they keep trying to log in with the wrong username and password?
    BA: Erm…lock them out after 3 tries.
    Dev: OK will do. That’s an extra day. Oh, and how will they get their account unlocked?
    BA: Ah, yes, we need an admin function to do that don’t we. Will that take long?
    Dev: (sigh).

    Typically, the “happy day” scenarios account for around 20% of your total build effort (hence the 80-20 rule – that it takes 80% of the time to do the last 20% of the work).

    As I have mentioned previously, I try to keep one increment ahead of the dev team – so I go into the increment planning session with all the stories (including the alternative flows) already worked out. That tends to result in better estimates.

    One of the problems I have experienced with capturing a story’s details as just a set of acceptance criteria is that it doesn’t help you to spot the alternative flows. Whereas capturing it as a use case (a step-by-step sequence of events) allows you to consider “what could go wrong” at each step in the sequence.

    Assuming this is indeed your problem, the most valuable thing you could do to improve your skills in this area is to read “Writing Effective Use Cases” by Alistair Cockburn, which has a whole load of good advice on how to identify alternative flows.

    The alternative approach (which I first heard suggested by Kent Beck, one of the fathers of agile and XP), is to simply double all the estimates and then “add a bit more for good luck”. I have tried this in the past and is surprisingly accurate. :)

    1. croga

      [quote]Typically, the “happy day” scenarios account for around 20% of your total build effort (hence the 80-20 rule – that it takes 80% of the time to do the last 20% of the work).[/quote]

      One possible solution is to move away from time-based estimations. A widely used concept in Agile is “planning poker” and estimating in story-weight (sometimes called Gummybears but use whatever quantity you like as long as it’s not time-based). That way you will never run into the old 80/20 rule as it’ll be automatically in place for all the work you do.

      The burn chart will show you how many “points” can be completed per itteration and this number (“velocity”) can then be used to estimated the work that can be done in the next itteration.

  13. Nick Panagopoulos

    Another secret revealed! Estimates times two is safe, but for those projects with many unknowns, multiply by pi.
    I agree with you, Tony. Use cases are still important, even in agile methodologies. Understanding the user flow sunny day, and all the alternate flows, is an art, which requires focus, iterations, and room for mistakes or misses.
    Merge this with a good understanding of how the GUI transitions from one step in the use case to another, and you have a good spec.
    My rule of thumb about use case detail (system level) is, if the user can see it, it should be in the use case.
    Tony , what do you think about that? Also, how did you deal with non-functional reqs?

  14. Tony Heap Post author

    @ Nick:

    Re. level of detail – I’m firmly of the view that the BA role is a design role, not a “requirements capture” role (in fact, that’s the subject of my next article!), and I agree with you that anything the user can see is fair game for a use case.

    I used to write specs that covered *everything* the user could see, with no ambiguity. I’ve loosened up a bit recently (got a bit more “agile” I suppose), and I tend to only spec the details that I think really matter or that the dev team or QA team might otherwise get wrong. That’s all a matter of judgment of course, and you can never over-estimate the dev team’s ability to make the wrong assumption :).

    The other problem I’ve had in the past with over-detailed use cases is that they are very hard to read with all that detail in the way of the main storyline. The current format I’m using is pretty cool – it’s graphical, with the main use case flow shown as a process flow down the middle of the page, and with the details as annotations around the edges.

    Re. NFRs – pretty easy really, I just write them in use case format just like any other story, so that they read pretty much like a non-functional test e.g.

    1) 10,000 users log on simultaneously and perform various transactions
    2) The system responds to transactions with an average response time of under 2 seconds (or perhaps refer to a table of response SLAs somewhere).

    There’s a good article I read on the web somewhere about how to write NFRs as “scenarios” – with a key point being to define acceptance criteria. That’s pretty much what I do.

  15. Vin D'Amico

    I like the way you used both stories and use cases. The two really are different. They offer alternative views of the same problem. Some people try to equate the two or debate which is better. They are just different. You employed them both to great advantage.

    Well done and thanks for sharing!

  16. Richard Sheldon

    Fascinating article – thank you very much. One (slightly tangential) question arising from this (and Agile in general) – how do you predict costs for this sort of development? If, as part of the approach, you are defining / refining reqs as you go along, how do you get a reasonable estimate of size / shape for senior stakeholders to sign off?

    Thanks again!

    1. Tony Heap Post author

      Hi Richard,

      If you have a look at my previous article (link in first paragraph) you will find details of how we did the estimating for the project.

      The main feature of the estimating was that the project was fixed in terms of duration and cost, not in terms of scope.

      I made sure I spent a lot of time getting the stakeholders to buy into this approach – the idea that they would *not* get everything on their original scope list, nor would they eventually want all of those things – because things change during the course of the project.

      I was lucky that my stakeholders understood this approach. And the burn-up charts were very useful too because the way I did them meant that the stakeholders could see how much the overall scope had grown during the project and also how we had delivered the same *quantity* of functionality that they had originally asked for.

  17. Yvonne

    Hi Tony,
    I am looking to adapt your spreadsheet into my current Agile project. I was reviewing the formulas and numbers that you had entered in the spreadsheet. I am curious about the tab “Stats (i8)” in columns P-T. It has name int the column headers which I assume are the developers but I am not understanding the numbers under each developers. For each week there is a 1 or a 0.
    What does the 1 signify? The 0(zero)?
    How do you determine whether it is a 1 or a 0 for the week?

    Thank you for your direction and help in this matter in advance.
    Yvonne

    1. Tony Heap Post author

      Hi Yvonne,

      I’m glad you are finding the spreadsheet useful. The values in columns P through T capture whether each developer is working a full day on the given day (each row in the sheet is a single day, not a whole week). A “1” means they are working a full day and a “0” means they are on holiday. If you look at column J you will see that it is the total of the developers’ individual days (i.e. the total capacity for each day), and column K is the cumulative capacity over the course of the increment. These then feed into the graph (via column J) to show the expected progress day-by-day (the lilac line on the graph).

  18. kai

    Thanks for the article couple of questions. Did you give this spreadsheet to the developer assigned to lets say Story S13?
    1. where did you store ur diagrams like the Prototype of the Web screen?
    2. how do you write user stories for something like “remove the name field and name field required message box.” I have alot of edit, maintenance, backend processes..not really strainforward new software development.

    3. As a BA did you test the story prior to it going to QA? If so is that the result u stored in the “test result”. Was the acceptance criteria sufficeint or did they also expect a test case of “How you tested, what data u passed” etc.

    4. I am trying to undertand how we go from one interation to the next. Once development starts for iteration 1, then the ba is doing what? When the iteration ends the ba tests the stories in interation 1 what are the devlopment team doing. What is discussed in the iteraion meeting. Does QA get the release once all stories have been integrated into the release?

    5. I really like ur layout we are not an agile shop but would love to practice user stories. I did a spreadsheet in sharepoint but not sure how friendly sharepoint is with printing it out. Your tabs are associated to sharepoints Lists. So I have a list called “Features: maynot be the right term but my columns were Type=Security,contrain,UI,function, Business rule
    Feature-ties to the SRS
    Requirement- ties to the SRS
    acceptance criteria but i did not place on a separate line like yours which i will change
    Diagram-not sure if that is the right title
    Test status
    notes
    priority
    status=anaysis, development,ba testing, qa testing,done

    I had another list called “defects” but I may just add it all to the other tab instead of spreading into two tabs.

    I have another one called ‘TestCases”. Personally i dont want to do any test cases…not sure if its really needed or what i should put here. I think they want to see how i tested..not sure..maybe i only need testcases for things thatr warrant aot of algorithms. QA will also be testing the release.
    Thanks

    1. Tony Heap Post author

      Hi Kai,

      You might find a few answers to your questions in my previous article http://www.its-all-design.com/being-an-agile-business-analyst/
      Specific answers:
      0) The spreadsheet was kept on a shared drive so all developers had access to it
      1) The prototype went on the shared drive too
      2) “Remove” stories are just a simple as “add” stories, you just use the word “not”. So, for example, to remove the postcode from the address page:
      3) I was the QA. There were no separate test cases – the stories were the test cases. Neat, eh?
      4) When the dev team is working on increment one, the BA is writing the stories for increment 2. When the dev team moves on to increment 2, the BA works on the stories for increment 3 and also the QA for increment 1. Note that this is different from standard Scrum or XP – in Scrum, you are supposed to write the stories, code and test all within a single increment. I don’t think that works too well.
      5) I’ve used Sharepoint a bit and I imagine it would lower your productivity because it’s not as dynamic to use as Excel. It would probably be a bit more scaleable though. My advice is use the simplest tool you can get away with.

      When I request to view my address
      Then the system displays the address page
      And the address page does *not* include my postcode

      1. kai

        Currently I did a spreadsheet like yours but the business is used to a word requirement with screen shots and broken down by feature etc. It makes it easier for me to make notes, revise and not worry about it being “pretty”. However that is how they are used to submitting it. Maybe I can find a way that I can print it in a user friendly format that they may be perceptive to reading it instead of a word doc. do you keep up with the excel and the word document? Or for my sanity I should probally pick one or the other.

        FYI-I just saw some testcases one of the ba’s on my team did I almost suffered a massive headache. She really should not have to do all those test cases if there is a collobration with QA. I did mine like yours lets see how its perceived by management.

  19. kai

    Also I am noticing spreadsheets hard to place screen shots of screen edits.
    Or
    for a requirment like “When saving new user file the new customer email should not be sent and the new user messgebox removed”…dont think i can write this as a story or feature?

    1. Tony Heap Post author

      I didn’t put screenshots in the spreadsheet – I had a separate HTML prototype to show the changes

      Here’s a story for your requirement:

      When I save a new user file
      Then the system does *not* send a new customer email
      And the system does *not* display the new user message box

      Anything goes with stories, so long as it makes sense to the developer and the QA. Just imagine you are writing a test script.

  20. Amanda Abelove

    Hi – I just did a spec for a site redesign in a way similar to how you did, except I used MS word because I wanted the cross-reference and bookmark features (that way when a story, case, extension was specified you can just ctrl+click on it to go there). It came out to about 500 pages. When it went for review, I was told it was going to be too hard for other people to edit. My response was that it saved manually changing numbers and titles by hand and that people should learn how to use MS word, which was unpopular. Would you push to make the other product people learn how to use bookmarks in MS Word (it’s not THAT fancy) or would you just go back and replace every reference in your entire spec with static text?

    The goal as always being working software…

    1. Tony Heap Post author

      First of all I would be worried about having a 500 page document going out for review. My XL spreadsheet is thousands of lines long, but it never got reviewed all in one go. In fact it never got reviewed at all. Have you considered a more incremental approach, breaking it down into smaller pieces?

      To your original question, I do tend to go with what people are comfortable with and keep things as simple as possible. Generally, people will naturally adopt a tool or technique if its benefits clearly outweigh the cost/effort of doing it. So if you think you have a strong case for your technique then persevere with it and soon enough you will probably find some supporters within your organisation. Back in the say when I still wrote Functional Specifications in Word, I used to use cross-references, but all my use cases had section numbers so I used to reference those, rather than create bookmarks. That was easy enough for people to follow.

  21. Yvonne

    Hi Tony,

    Did you have a user story which the users asked for and agreed upon. Then when you presented the story to the developers they all agreed that it was technically not feasible. How did you indicate this information in your spreadsheet?

    1. Tony Heap Post author

      Not really. I collaborated with the lead developer right from the start of the process – so we already knew about any technical issues long before I had written the user story. Plus I am from a technical background myself.

      We did have a few stories that got withdrawn – which I indicated using the “Status” column on the Summary sheet.

  22. kai

    1. I am wrting out my acceptance criteria in the (When I, then) format. I am noticing that I am having to combine 1-2 stories in order to cpmplete the whole action. Is it possible I made my stories too small?

    2. Do you write all your requirments in the form of a story or only those that relate to the user interaction with the system? for example you would not write a story that deals with screen edits, renaming of GUI objects(like buttons, reports,labels) processes that are to be kicked off, updated values in a table would you?

    1. Tony Heap Post author

      1. This is a whole topic in itself. A good rule of thumb is that a (single) story should have some useful benefit to the end user. So if you need multiple stories to achieve that, it’s possible that your stories are too granular. On the other hand, a story is supposed to be small enough to implement within a single dev increment, with an idea code & unit test duration of up to 5 man days. Stories that are “too big” are called “epics” and are generally split down into bite-sized chunks. Google “agile epic” for more details.

      2. I wrote all my requirements as stories on this project. For UI changes, the acceptance criteria might be something like “the size & position of widget X is as per screen Y in the HTML prototype” – and then make sure the prototype is correct, of course.

  23. Yvonne

    How did you go about documenting the Non-Functional requirements? Was it also recorded within the spreadsheet or in a separate document?

    1. Tony Heap Post author

      I didn’t have many because the system was already built (these were enhancements) and most of the NFRs were already covered. When I had any I wrote them as user stories too e.g.

      When I click on the “Submit” button
      Then the system responds within 2 seconds 95% of the time

      Probably better would be to write a user story as a “hook” into an NFRs doc e.g.

      When I click on the “Submit” button
      Then the system responds within the SLAs set out in Document X

      There’s a lot of good stuff out there on writing NFRs as testable scenarios – which is exactly what user stories are – so I do think they are an appropriate format.

  24. kai

    1. What if you wanted to test a story 3 different ways like with 3 different users permissions. Would this be 3 different stories?
    2. Do you make notes of the data you used for your tests?

    1. Tony Heap Post author

      1. It depends. Usually I would do this via different acceptance criteria within a single story, but sometimes I would do differnt stories. SOmetimes with permissions it’s better to have a master premissions doc (showing the functions that each role has access to) and refer to that from the story.

      2. No

  25. vadaliprasad

    Thanks a lot for this wonderful article. I have been working as a Quality analyst and aspiring to become Business Analyst. Please suggest me more references to build up my career as a BA.

  26. kai

    Help meeting in the morning! Can I use this spreadsheet in a functionality walk-through meeting even if I don’t have a prototype? I have a bunch of screenshots that I have in the original requirement I wrote. How could I facilitate this? Print off the screenshots as a handout and then show the spreadsheet on the overhead?

  27. BlueTone

    Hi Tony,
    Love the article. My development background is almost identical to the one you describe: development, RUP, UML, agile etc. Spookily I even seem to have ended up with the same hybrid analysis process that you describe. (Right down to the use of burn-UP charts, doing just enough up front analysis before the sprint starts, and most importantly learning how to couch my process steps in the latest Scrum terminology :-)).

    One slight tooling difference – I now make extensive use of Jira & Confluence, which between them do the job of your network spreadsheet. (I actually also have the spreadsheet, but the feature list it contains is extracted automatically from Jira to avoid duplication.)

    The reason I mention these tools is that the latter helped me find a useful middle ground in your quest for the cumulative ‘font of all knowledge’. I also found that the completist approach was time-intensive and yielded a great document that nobody read. However, where we ended up was to produce two levels of documentation. The first was cumulative, at the level of a user guide for the software-we-are-planning-to-build and was full of screen mock-ups and diagrams. It was about 1/3 of the size of the original cumulative document and hence did get read. The remaining 2/3 was written as Use Cases, just in time, and were effectively used as work instructions for the developers. They were archived, but no attempt was made to keep them cumulative or up to date – they just described a series of delta changes. This is where the tooling came in handy – by using a wiki, it was very natural for these two documentation levels to co-exist simply by judicious hyperlinking.

    So, for example, each use case would have no introductory text – it would instead give one or more links to the relevant pages in the cumulative document. Conversely, the cumulative doc would have lots of links ‘for more detail click here’ which took the reader to the use case.

    The task management software completed the picture by allowing us to be precise about the degree of richness that we had prioritised for a given iteration. The cumulative doc would always describe where we are (realistically) trying to get to, but individual Use Cases / and their counterpart task in the database my explicitly exclude some aspect of functionality, which will be provided at a later date. (This is where I decided Use Case ‘extends’ fit in :-))

    Anyway, I was wondering if you had tried something similar and, if so, what your experiences were?

    Keep up the good work!

    1. Tony Heap Post author

      Hi BlueTone,

      I like the sound of your approach, it sounds like a good balance in terms of keeping the documentation alive whilst not spending too much time on it.

      Funny you should mention using a Wiki – it is something we are experimenting with at my current client, and something that has been written about previously on this site by Adriana Beal (http://www.bridging-the-gap.com/how-to-add-value-to-a-project-using-smart-knowledge-sharing-strategies-part-i/).

      We’re using the (SharePoint) Wiki for higher level info on the systems, but what we need to start doing is linking it to our various (delta) Functional Design documents, so the reader is able to refer to the detail more easily. The trouble is there is no impetus to do it because it’s one of those “low priority” tasks – we get by with the knowledge in our heads and in each others’ heads – and we’re always too busy writing the FDDs!

      We’re also pretty poor at producing User Guides – but I also think this would be a great way to drive system development. In fact I even have a catchy name for it – User Guide Driver Development (after Test Driven Development, you see!)

  28. Todd Tullis

    Hi Tony – thanks this is a great concept as I too tend to over-format documents and am trying to stay as agile within a regulated software development environment.

    I’ve adopted your concept to create functional specifications for a web-based app but have run into a couple issues, hoping you can provide some experience-based advice:

    1) How did you handle cases where your spec required a branch within a branch? This seems to throw off the numbering scheme any way I cut it.

    2) How did you handle validation specs for multiple data fields on a form…thus far in my experience I end up trying to come up with all possible combinations of missing or invalid data, which quickly escalates to a large number of “When I…” steps

    Any advice appreciated – thank you!

    BTW – my approach this far is to ‘sketch’ out the flow diagrams and then fill in the spreadsheet – the visual look just makes it much easier to understand the branches & actions – but unfortunately does not provide ready-made test scripts like the spreadsheet does (unless others have found a good tool for this).

  29. Tony Heap Post author

    Hi Todd,

    Glad you’ve found the approach useful. Here’s some answers:

    1) To branch off a branch I use a “dot”. For example, to branch off flow 2b at step 3, I call the new flow “2b.3a”. If I need to branch off the same step again, it’s “2b.3b” and so on. There are a few examples of this in the example FS if you search (e.g. row 334).

    3) For validation of data fields, I tend to have a single line in the main flow for each validation and a single alternative flow for each failure. I don’t try to create a separate flow for each failure permutation. See row 228 for an example (although note that the alternative flows seem to have the wrong IDs on this one!). For an alternative approach see row 581.

    Re. the flow diagram “sketches” – I agree this is an excellent idea and it’s an approach I have been using recently myself. But instead of creating the spreadsheet, I’ve been tagging the acceptance criteria directly to the flow diagram itself (in Visio). There are pros and cons of this approach and I hope to write an article on it soon!

  30. Pingback: To spec or not to spec, that is the question… » pseudoplace.com

  31. Kamal

    What would you suggest if someone has to develop an application like MS Word or Visio where there is complex UI involved but the essential goal is a document. What could be the possible use cases for such a scenario?

    1. Tony Heap

      Hi Kamal,

      That’s really not too different from a “normal” application – you still need to specify what the user does and how the system responds to that – but I guess you would have a greater focus on having a working prototype because the usability isso important.

      1. Kamal

        Well the issue is the that as per traditional definition for a use case, one should not put UI details in a use case.
        Now for example I have to provide a developer with functional details for a designer UI which has got a toolbar using which one could drop different kind of objects to a layout area and then user could have scenarios like rotating, re-sizing, moving an object.
        Now in such a case the developer should get the UI level behavior details, which are by definition not included in a use case, more over there would be many use cases that are only related to UI behavior and can not be foreseen before prototyping.

        My question is what is the best approach to document such details for a developer?

  32. Shantul Sharma

    Hi Tony,

    I am in the process of streamlining the documents used on our projects and this article, especially the spreadsheet, is a godsend.

    I have just one question though and pardon me if I have missed it in your spreadsheet / article. We are required to document the validations on each field on the screens. For example, we usually capture what is the length of the field, what characters it accepts, whether it is mandatory or not, and whether it is read-only or editable for the user in question. Did you capture this information anywhere and if not, where would you suggest that it should be captured?

    1. Tony Heap Post author

      Hi Shantul,

      The easiest way to do this is to include the field rules within the flow of the story itself, for example:

      When…
      I navigate to the Enter Address page
      Then…
      There is a field “Street Name”
      The Street Name field accepts max 40 characters
      The Street name field accepts only alphanumeric characters, spaces and commas (if you are restricting the characters the user can actually type)
      When…
      I enter my address
      I Click on Save
      Then…
      The system validates that I have entered a Street Name (i.e. Street name is mandatory)
      The system validates that Street Name contains only alphanumeric characters, spaces and commas (if you are going to allow the user to type anything up front and validate it only after they click “Save”)

      However, this gets a bit messy if there are lots of fields. So they you could consider including the field validation rules in the accompanying prototype instead. Max length is easily specified in HTML. Mandatory fields can be indicated visually with an asterisk or similar. Restricted characters are more difficult – you could put the rules in a tooltip maybe.

      If you take this approach, your story would look something like this:

      When…
      I navigate to the Enter Address page
      Then…
      There are a number of input fields as per the prototype
      The field max lengths are as per the prototype
      The fields accept only the characters specified in the prototype tooltips
      When…
      I enter my address
      I Click on Save
      Then…
      The system validates that I have entered all mandatory fields as per the prototype

      The main rule is that there are no rules – just do whatever works best for you.

  33. Doug

    Hi Tony
    This was a great article. I previously used Excel as a Spec Template as well, but really enjoyed / learned from some of the items you listed. I’ve since moved on to Google Spreadsheets – which makes sharing and collaborating of a live document much easier. It may address your sharing / collaboration concerns above.
    Cheers,
    Doug

  34. Venkat Ram

    Excellent work Tony !
    You just uncomplicated the way we gather requirements, and made it so easy for all the stake holders of the project to understand them.
    Keep Going !!

  35. Mohammad Khan

    Hi Tony,
    Just a real brilliant FSD. I’m actually looking to use it for my next deliverable on a project. Would you mind if I used it?

  36. Sandy

    Tony,

    That was a brilliant job, That makes me feel really lite. Am trying adopt it would you mind?

    I have a questions

    1. If i wish to specify the controls, say for an eg.. data entry fields in the form like txt box, list box, button..
    I get a feel that could help the developer in a better way, what is your suggestion on this?? How do we include this if we could do that.

  37. Tony Heap

    Hi Sandy,

    My rule is that you are fine so long as you stick to specifying things that are user-visible – so text boxes & radio buttons are OK. Just include them in the screen mock-up, and if you want to be specificy you can add acceptance criteria like “There is a text box labelled ‘Name’ with a max field size of 30 characters. It does not accept the following special characters: #@£%”

  38. Reno

    I was on trial of an online kanban tool (using it as PM/RM tool). Desperately looking for a template to wrap up all project doc.
    Thank you for saving me the trouble re-inventing the wheel. :)

  39. Brian

    Do you have experience on making these specs for something that’s not necessarily software but hardware? How would these elements change?

    Curious about how to implement this for various products outside of the software/agile world.

    Any help/links would be greatly appreciated!

    Thanks!
    B

    1. Tony Heap Post author

      Hi Brian,

      My experience is only with designing software and business processes. In theory, the notation used ought to be appropriate for describing anything that has *behaviour* i.e. it does stuff over time, and in particular if its behaviour depends on how its environment (users or other entities) interact with it.

      But bear in mind that agile delivery is not appropriate for building in all media. Software and business processes are particular in that they are malleable – it’s relatively easy to develop/change them incrementally and experimentally, whereas other media (buildings for example and possibly hardware) might be better being designed up-front because of the higher cost of re-work.

  40. Lino

    Hi Tony,
    this article is a very good example showing that also agile process needs analysis and design.
    I would like to understand , apart from functoniality, how other information are documented expecially entity attributes and components distribution.

    Thanks

    1. Tony Heap Post author

      Hi Lino,

      There are some ideas on how to document data entities and attributes in this article.

      For component distribution – it depends what you mean exactly. If you want to specify how multiple components within a system interact with one another, then the use case/scenario format is still appropriate – you just need to describe which components speak to which and in what order. For example:
      1) User enters a postcode and requests an address search from system A
      2) System A passes postcode to system B and requests an address search
      3) System B returns a list of matching addresses
      4) System A displays the list of matching addresses to the user
      5) User selects an address from the list…and so on

  41. Tim

    In your comment about extreme programming, you described their philosophy this way:

    “In Extreme Programming, the requirements are conveyed verbally, directly to the developer, with just a few notes scribbled on an index card as an aide memoire.”

    The company I currently work for has adapted a version of agile methodology for our software development. On the project to which I am currently assigned, we are developing a new feature for the product. The business team and the development team met, discussed the requirements for the feature. The development team took mental notes and made a few paper notes to themselves and then went on to develop the feature. Then the development team, the business team, and the QA team met so QA could ask more questions about the feature. In this discussion it turned out that the development team development team had not implemented the feature the way the business team thought they had explained it to work. So development and business teams had further discussions and the development team reworked the feature. Another meeting with QA, development, and business teams occurred and, once again, the development team had incorrectly (from the point-of-view of the business team) critical aspects of the feature. This cycle was repeated several more times before business was satisfied.

    If the development team and written a spec and reviewed it with the business team and the QA team before implementing the feature, it is likely that the errors would have been caught at that stage and no redevelopment would have happened.

    I don’t understand how extreme programming addresses this problem. It seems like extreme programming would exacerbate this tendency if there are no written spec of any sort. My experience over the years has taught me that programmers do not like writing documentation. Is extreme programming just a thinly-veiled excuse for developers to not write documentation?

    1. Tony Heap Post author

      Hi Tim,

      Sorry for the delay in replying. Your example is an interesting one. Here are my thoughts.

      Firstly, in my experience, having a specification doesn’t necessarily mean you’ll get it right first time. Many times I have produced a spec, had it signed off, had it built, then shown it to the business, only for them to tell us it’s “not right”. Sometimes it’s only when the business gets to touch and feel the built product that they realise what they really want (or don’t want).

      XP and other agile methods attempt to cater for the fact that you won’t necessarily get it right first time. Specifically, phased delivery allows teams to (a) deliver high value functionality early and (b) receive business feedback on that built functionality and rework it until it’s fit for purpose.

      Second, specifications have pros and cons. On the good side, they force the team to think about what they are building before they build it, and as you say they give the *possibility* of spotting that the design is wrong before building it, which might be cheaper than spotting it’s wrong after it’s built. But as mentioned above, that’s only a possibility – it’s still possible to get it wrong even with a spec.

      On the down side, specs take time and effort to produce, especially when you get into lengthy review and approval cycles. So there’s a risk that you actually spend *more* time and money specifying and building a system before you find out it’s wrong. The next problem is that specs can actually become *barriers* to communication. Instead of talking, IT and business communicate only via the spec (reviews, comments, approvals). Collaboration decreases and the chance of getting the right system built also decreases. One of the objectives of XP was to increase collaboration, and one of the ways of doing it was to decrease documentation and thus force verbal communication.

      In summary, it’s not clear cut that a specification will always help. That said, my own experience is that a lightweight specification of some sort, coupled with lots of face-to-face collaboration, usually gives a net benefit. For every new task I try to evaluate whether it’s worth producing a spec. If the task at hand is well-understood and unlikely to be misinterpreted, then maybe not.

      But I also agree with you that some (not all) developers, would rather just wade in and code, not write (it’s more fun)!

  42. George

    Tony excellent info!! Am I missing this? In the spreadsheet I see ‘As a…’ and ‘I can…’ but I am not seeing a column for ‘So That…’ Where is this info captured? I have clients that want to see all 3 columns for each User Story. Thank you!!

    1. Tony Heap Post author

      Ah, you’re right, the example spreadsheet doesn’t actually have the “so that” column, does it? I’d suggest the following two-step process:
      1) Insert a column to the right of the “I can” column (column D) and put it there.
      2) Head over to the Downloads page and show your immense gratitude with a donation :)

  43. Richard Haynes

    Hi Tony, agree that having the BA writing the tests is the way of the future. We practice this at our business (software company with one core product). We use Selenium IDE to get the tests to a point where a developer can add them to the continuous integration process. There is of course some massaging and a learning curve for BAs but it’s definitely the way to go. I wish I had a dollar for every time a unit test passed but the UI was displaying the wrong info! I am a big fan of front end automated testing. Nice article. Rich

Comments are closed.