Joel on Estimation

Note: This is an interview I conducted with Joel. He was so kind to provide long and clear explanations that I decided to leave them alone rather than make it into an article. The beginning and end are my comments and everything in between his Joel. Thanks, Joel.
Estimation is daunting in Web design with its many variables and the differing for each project. If I could teach everyone one thing about estimation, it’s this: the shorter the tasks, the better the estimate. I tell people to break every schedule down to tasks that are between 4 and 16 hours. If you have a task on your schedule that says something will take a week, it probably means you haven’t thought enough about that steps are involved and you’re pulling the estimate out of thin air. If the requirements indicate, “We need a logon module,” then how long will that take? Aw geez, maybe a week, you guesstimate. Thin air. Back to reality, think about what’s involved in the logon module and break it into manageable tasks.
* Create user table in the database (4 hours)
* Develop a way to enter new users (2 days)
* Ability for users to get password emailed to them if they forget it (1 day)
* Create logon page (4 hours)
* Add password checking capability (1 day)
* Add back-end security (2 days)
* Add HTTPS server (1 day)
* Build code in every page to check that the user has logged on (2 days)
That adds up to TWO weeks. The guesstimate means trouble just for this one requirement. Imagine under-guesstimating for more and finding the project way behind schedule.
By thinking about small tasks you get an estimate, which is not only more precise, but also vastly more accurate. Of course, the client doesn’t know everything that’s needed while working on the requirements.
For the first 11 years of my career, I went around assuming that clients somehow know what they want and our job was to pin them down, maybe by writing a super-detailed spec and getting them to sign every page of it (as some have suggested).
You can try that; it won’t work. It’s too hard to envision the whole system working together until it works together. You’ll start to notice all kinds of things that you thought were requirements, which you never use, and all kinds of things that never occurred to anyone, least of all the client, which turn out to be ultra-crucial.
Manage your client relationship in a way to give you room to frequently adjust even in the middle of the project. I approach handling requirements by suggesting to the client that I’ll start building the minimal system that implements some of the requirements and show the client how it works. Afterwards, decide on which requirements to add next even if it’s not on the original list of requirements. Deliver the next product with the added requirements and continue the process until the system works as expected.
This approach allows repeatedly changing direction while the client learns about the system and how it serves his needs. Requirements that seem important may turn out not to be needed, thus saving money. Regularly delivering functionality builds the client’s confidence in the developer’s ability to stay on track.
Unfortunately, this process doesn’t help with estimating the cost of the whole project before starting. Work through this by explaining that you’ll keep adding and delivering features that have the biggest ROI until we’ve run out of features that are cost effective.
Expect the client to keep requesting more features beyond the budget. When it happens, demonstrate the cost is higher than the budget and it’s time to prioritize. Be ready for politics. The person signing the check
is usually not involved in the project activities.
Here is where the incremental system works since you and the client
address the higher priority items first and keep adding to it until you
run out of time and money. By then, you’ve proven yourself by regularly
delivering functionality.
What do you think? How do you estimate? How accurate is it? I’ve been reading about Galorath’s SEER-SEM estimation tool and hear great things about it. Has anyone used it?

Bio:
Joel Spolsky is a software developer in New York City who has worked at Microsoft, Viacom, and Juno Online Services. Currently, he runs his own company, Fog Creek Software, which makes CityDesk content management software. He is the author of Joel on Software: (long subtitle follows and I am avoiding carpal tunnel – grin). Apress, August 2004.