I got an email at the end of last week from a developer asking about Agile development. It highlights a few problems with development in general, and with Agile as a "badge of honour" that are worth exploring. It deserved a fairly detailed reply, so excerts of the email follow:
I just came into an Agile project with some difficulties, they have stopped doing Agile development and gone back to a Waterfall approach. Coming into the project late, I saw numerous problems with the approach that are not helping.
It strikes me that developers rarely come onto a project and think how wonderful it is and how well it is running. This could be because they just have different ideas to everyone else, we all know developers are pretty opinionated, and believe they have the "right" answer to all the questions.
There is however a more likely reason, when you are close to a project, it is sometimes hard to see what is glaringly obvious to anyone with fresh eyes.
Agile can be daunting, you are fully exposed, you need to have Courage, you need to resist the urge to falling back to lines like "well we don't have written requirements", "it will be fixed in testing", "it isn't my code", and all the other Waterfall phrases.
Developers would be working on stories but things were not available to them as they had not been created yet - those items were in other stories
This sounds like a bad planning excercise. If you are using user stories, then it is the job of the developers to be involved in the planning of these, and to be able to assess up front what dependencies these stories have upon each other, and to properly break these down to more manageable, and more appropriate, stories.
I have seen similar problems, and it was generally caused by project managers deciding upon the priorities for things, and not listening to developer feedback (or not even including them). A good project manager is vital to a project to act as an "enabler", someone who can make things happen, can remove stumbling blocks, and can generally "grease the wheels". A bad project manager is the worst thing that can happen to a project.
Requirements changes by the business could cause certain functionality to have to be reworked significantly, causing a lot of problems, if the stakeholders changed their mind every time they saw something, more changes would be introduced. The more they saw, the more they wanted to change.
Well, Agile dictates (in so far as Agile dictates anything) that the only certainty in a project is that the requirements will change. We embrace change. Agile methodologies are all designed to allow change to happen as easily as possible, and to encourage change to be a core part of the driving process.
There is a point at which this can easily go wrong though, and again it comes down to either the wrong people in the planning excercise, or someone not listening.
Understanding change happens is only the first step.
You have to also explain to the business that change is inevitable, but it is certainly not free. The decision rests with them, but it is the job of the development team to properly evaluate requests and user stories to see what impact they will have on the project now, and into the future. Business users presented with an option to change, but no consequences of doing so, will of course choose change. When asked whether they want to make the change they think is needed, or do the other three user stories that were scheduled for this week instead, the choice becomes rather different. Agile encourages everyone in the process to become a stakeholder, but everyone also has to bear the cost of change.
Then you have to be able to think ahead and to design your system in such a way as to allow for change. This is a technical challenge, largely only obtained via experience, both of the development language and tools you are using, but also of the business team you are dealing with - how often do they change their minds, how often do they express things badly, how often do they get the wrong person to make the decision. In the sales game, you always want to be talking to the decision maker, becasue only they can give you the answer you need - make sure as a development team you are talking to the decision maker too.
A well designed architecture is fundamental to a successful project, and a key quality of a well designed architecture is the ability to make changes with relatively little impact upon other system components. Adding or modifying behaviour should not be a traumatic process - it may not be trivial either, but it should never be traumatic. I covered some of the aspects of this previously, and although that post was geared towards why Inversion of Control can benefit your code, it also deals with Robert C Martin's observations on a Rotting Design. Make no mistake, it is perfectly possible to start a greenfield project and have a rotting design from day one. It is also a large risk to any project that the once elegant code base rapidly slips into being a rotting design when time pressures are on, and people start taking shortcuts.
It is hard to know when it will be finished when things are always open to change
Agile methodologies do not say we have no delivery date, in fact almost to the contrary. Generally under an Agile project you should be able to deliver a working version of the software at any point in the development.
Admittedly the functionality present within that software will be largely restricted the earlier on in the process you decide to deliver, but 75% of functionality with a high quality code base is a lot better than 100% functionality which it is heavily bugged and fragile.
I often draw a triangle for clients, with Time, Quality, Functionality as the three points (Resources is often included here, but I ignore it for this purpose, Brooks Law usually applies by the point this matters). I tell them they can pick any two of those aspects and fix them, but that means the last one will have to vary. They cannot fix all three.
If Time is a critical deliverable, then you can choose to sacrifice Quality or Functionality (or both). The perfectionist in me says sacrifice Functionality, Quality is too precious to be the one to slip.
If Quality is a critical deliverable, then choose to slip Time or Functionality. The choice here only comes down to whether you want it fast or complete, or what combination of the two you want.
If Functionality is the key deliverable, then you can slip Time or Quality, and again I would say that Time has to slip, as Quality is too precious.
My personal opinion is therefore that Quality comes first, Functionality and Time are the things you can alter - but not only does this vary by project, but also by business circumstances. Sometimes it really is better to get to market with a low quality product, in the lowest possible time, with limited functionality (Twitter immediately comes to mind), just to steal a march on the opposition.
But what is really important, is that you express these aspects of development explicitly. If you are sacrificing quality because the business has asked for something unachievable, then let them know you are doing this, and make it an express design decision. If later on the business tells you the software is bugged to hell, you can point out when they took that decision to satisfy another need. If you need to take two more months because the business just changed the business model, then fine, but again make it an expressed and documented design decision.
Too many of these things get left as "assumptions" ... if you don't say otherwise, the business will assume a really high quality product, with massive functionality, in a very short timescale ... it isn't possible, so let them make the choice.