Somewhere, over the rainbow, there is a magical universe where projects always deliver on time, on budget, and to specification.

Unfortunately, not many of us visit that world too often. We spend most of our time in a place where things go wrong. Tasks take longer than they were supposed to. You discover requirements you had not anticipated. The hardware doesn't show up. The designer falls off her snowboard and breaks both arms. Enter our familiar friend, the Iron Triangle:

Schedule, Cost and Scope. You can get up to two of these back on track, but not all three. You can still get everything you wanted when you wanted it, but only if you hire more effort. Or you can keep to the same spend, at the expense of fewer features. Usually there will be some compromise adjustment of all three. We've all been there.
In a waterfall development the procedure will be to have a replanning meeting or initate a 'change request' process. You adjust the requirements up here, then adjust the development estimate down there, which then alters the testing schedule.. a series of interlinked changes ripple up-and-down a Gantt chart covering a period of many months. And then the following week something else happens and you go through the whole exercise
again..

This process can become pretty painful. I have seen projects whose entire energy winds up being absorbed into this planning spiral. I have seen projects run for an entire year, and get as little as two weeks' actual development done. I've even seen projects abandoned completely because they were no longer worth the candle.
Agile Development fixes all of this though, right? By planning a little and often and being flexible you take the heat right out of these decisions, projects proceed without pain and there are no nasty surprises.
Yeah, right.
When Agile Projects Go Bad
Developing using an agile methodology does not change the fundamental objective - you are still attempting to develop a product with certain characteristics, on a particular timescale, using a given budget. And while you may avoid months of 'analysis-paralysis', or dozens of tedious and demoralising change-request meetings, ultimately the same problem often arises - it becomes clear you aren't going to be able to release what you wanted on time and budget. Why does this happen, and what do you do about it?
The first part is easy, it happens because we are human. Nobody can look at a project of any complexity, anticipate all problems involved, and accurately estimate what it will take to overcome them. By rejecting upfront specifications and planning agile explicitly acknowledges this. However agile skeptics would argue that this simply creates a vacuum, allowing for open-ended projects which drift along without proper control -
and they are absolutely right.

One of the most common problems with agile projects is that the team keep discovering interesting new features to build, or elegant code refactorings to undertake, or simply keep changing things because they can... and x months later find they have spent the budget without producing a market-ready product. That doesn't have to happen too many times before companies decide that agile ain't working, and return to waterfall development, or try fixed-bid contracts, or whatever.
Keeping on Track
So how do you avoid this? Well, look again at the Iron Triangle. It may be a triangle, but it isn't a very equilateral one. In many situations 'time' and 'cost' are quite close to being the same thing - the spend is a function of how many man-hours are put into the project. Now if there is one thing companies hate it is losing control of costs, which implies that you should try to keep to schedule. In which case the thing which must give is scope - the angle which agile projects often seem to be worst at governing.
However that need not be the case. The thing to do is start with a very clear vision of how the overall product should look, what the absolute must-have features are, and make sure that you build them first. Suppose you are building a photo-sharing website like Flickr. You need a site where people can create an account, upload photos, view them, tag them with search terms, and search. That's a minimum, without which there is no product. Now what may happen is that you put some really cool, 'web 2.0' photo-networking feature in your prototype, your test users love it, the team throw themselves into building it - and totally neglect some boring fundamentals like user registration. Come the expected end-date of the project, you have half a photosharing site with one really exciting feature, but you can't launch or do anything with it. You have no choice but to either spend more money or give up, and your bosses aren't happy.

In contrast, if you keep your eyes on the prize and try to ensure that you have a shippable, if minimal product by the desired date, then the engineering team have done their job. Sure, it may be the case that without the cool extra feature you don't feel able to take the product to market - but it's a
business-based decision. Perhaps you will do a soft-launch, or a closed beta. Perhaps you get a little more funding to add that extra killer feature on top. You're in control of events, and in a strong position to choose.
This then, is the way I believe you can deal with the tyranny of the iron triangle:
- Start with a clear product vision
- Develop iteratively, most important things first
- Try to hit the end date, and thus control costs
- Adjust scope to make this possible
Being able to deal with the day-to-day detail of changing stories without losing sight on the end vision is one of the most important parts of the Product Owner's job, requiring a strong sense of prorities. At the same time the engineering team need to keep the business objective in mind when deciding how much refactoring they can afford to do. On reflection a lot of this is about remembering the macroscopic project goal and not getting lost in the microscopic mechanics. Ye cannae change the laws of physics. But if you look up often enough you're a lot less likely to get smacked on the head by an apple.