At DeveloperTown we’ve built a fairly comprehensive design process that we use to kick off most of our client engagements. When I look back at our kickoff design process when we launched in 2010 to where we’ve matured today, I’m taken aback by how good it is. I’m simultaneously amazed by the work product we can produce in a mere four weeks, and eager to see how much better it gets over the next four years as others add their mark to the process and we continue to refine it.
It’s this process (and a recent argument with my partners) that has convinced me that most Requests for Proposals (RFPs) are fundamentally broken. They ignore (or grossly gloss over) the unknowns of a project, remove the ability to discuss the real objectives and the tradeoffs associated with technical decisions, and increase cost and risk for the people who submit the request. As I look at the types of outcomes we can create for clients now with our process, I see nothing but waste in most RFPs that I see.
In our process – or any good process of product design and technical discovery - it’s not uncommon for us to:
- Fundamentally influence the business model of the product, moving the solution in a slightly different direction than what was originally requested;
- Come up with a breakthrough design that dramatically simplifies the product, requiring us to build less than was originally conceived – or build it in a different way;
- Discover that some piece of technical wizardry isn’t going to be as easy as originally assumed at the outset, requiring us to fundamentally rethink the solution;
- Create product roadmaps that release features into the market in smaller increments allowing for feedback and tuning;
- Take on short term technical debt or leverage a third party solution until a feature is proven in the marketplace before spending more to build it out completely;
- Look at the long term (12 to 24 month) finances of the project - not just the next three to six months, where most RFPs seem to focus;
- Or other tradeoffs that emerge based on debate and discussion or feedback from early validation of ideas with potential customers and users.
Our process – while unique – isn’t the only way to make these tradeoffs. But I’ve yet to see the RFP that says “We think we want X, but we’re open to the idea that we may really want Y. Help us figure that out.” They always seem to assume the end solution proposed is the right one. There’s never any discovery, learning, or long-term planning baked in. Or if there is, I haven’t seen it. I would openly applaud the company who submitted an RFP that did contain those parameters.
Why does an RFP process lead to increased cost? Without discussion of the many tradeoffs noted above, there’s no room for the technology team to recommend alternative methods to accomplish the end goals. Ideally, you’d like to find a team to help you solve a problem. Instead, most RFPs tend to lead to teams implementing a given narrow solution. Which assumes the problem is fully understood and adequately solved by the submitted proposal. It costs more to be wrong and have to go back to first principles than it does to move in a more exploratory way at the onset where you continuously question core assumptions and validate those assumptions over time as the project unfolds.
More importantly, consulting companies will likely charge more for an RFP than for a product that evolves over time. If I attempt to do pricing for an RFP, here is my process:
- Do some top-down estimation in my head based on my experience in software development and past projects that I’ve seen. Is this product $300k to $500k? Or is it $1.5M to $2M? Figure out what the primary drivers of cost are likely to be.
- Assemble a very small team of designers, developers, and testers to have them help me quickly estimate the work from the bottom-up. Figure out the high-level tasks for the project, and have the team put hourly estimates on those tasks.
- Iteratively work with the team to get my bottom-up and top-down estimates to mesh. It’s likely somewhere in between the two. For this illustration, let’s assume that what the team and I believe is a “likely” cost to build a product is $100k.
- Once we have that “likely” cost, add a contingency. Because who knows, we could be wrong. (And we likely are.) For argument, say that contingency is 20%. For most purposes, this adjusted cost estimate - $120k in this case – is the good faith estimate I’d want to provide a client.
- But I’m not providing a good faith estimate in most RFPs. I’m either fixed-bidding the work, or doing smaller fixed bids in phases, or doing some sort of not-to-exceed, or setting myself up for a “Well we told you everything you needed to know upfront, so if it costs more than you estimated then that’s your fault.” So I now need to price in the risk of missed client expectations, re-work, and evolving scope. Evolving scope includes all the things the client wants that they didn’t include in the RFP... And how could they know what they really wanted without seeing designs or a working product yet? Let us assume I price the “expectation gap” risk at an additional 50% on top of our $120k. So I’ll bid $180k.
- Now I need to list out all of the legitimate assumptions for my estimates (we think were only building this for iOS, not Android, we only plan to support IE 9 and above, etc) as well as ridiculous assumptions that should immediately cause you to fire someone on the spot if they include them in a document they deliver to you. Actual examples from past proposals I’ve seen: “… the client responds in a timely fashion … the project team will have continuous access to development and test services/environments … key business resources are available throughout the engagement.” Seriously? If you include crap like this, you don’t know what you’re doing. It’s like saying, “If the sun doesn’t come up tomorrow, we may fall behind schedule.” Really? Shocking insight! However, if I don’t include it, I can’t “point to it” later when you come back and yell at me about increased costs. So in this crazy RFP world, I have a perverse incentive to include more and more of this crap, and the more outlandish and vague it is, the better for me.
- Finally, while I have a price and all my assumptions to guard me, I still have one more ace in the hole. The ever-present “change request” process. I know you don’t know what you really want yet. So I know the requirements will change. They almost have to or you’re likely guaranteed failure upon launch because you can’t possibly know everything months in advance. And I know when you change requirements – then I get to submit a change request. And with each change request, I get to increase the cost.
Final result? A bloated quote for a solution I know you won’t really want, with a process and incentives setup to ensure that we negotiate more than we collaborate. It’s an awful awful process. All the incentives are wrong. An RFP buys the false illusion of certainty, and does so expensively.
A more “sane” consulting company (and there are many of them – not just us) will ask you to engage in an ongoing team-based model. Form a team, have that team deeply learn the problem you’re trying to solve and the business constraints on the project today, and then that team works with you to iteratively solve the problem as quickly and safely as possible. You lose the false appearance of “certainty” that the RFP process provides – and that’s difficult to do – but you gain better risk control, which in the long run will yield lower costs and a better product. A better product also leads to increased revenues, increased user adoption, or whatever your business objectives are.
In addition to the up-front pricing problem, there’s also no opportunity for deep discussions around tradeoffs. There might be three, five, or twenty ways to accomplish some feature, task, or product goal. All of them have different short term and long term costs, different short term and long term implications on user experience, different implications to technical debt and product quality, etc. Those decisions should be ongoing and ever-evolving discussions – not a list of assumptions made by someone who doesn’t know your users, long-term business goals, competitors, or technical competency.
All of this leads to increased risk. RFPs codify early assumptions into contracts. They remove incentive to learn and adapt throughout the project. They remove the ability (through change requests and CYA activities) for people to change their mind based on new information. They encourage developers to take shortcuts since pricing is fixed (or feels fixed) and the goals are short-term goals. You want your developers to think long-term, but then simultaneously act short-term with that long-term goal in mind. You want them to behave as if they were spending their own money. You should want them to argue with you, debate tradeoffs, and care about outcomes.
Dave Christiansen, a former “Townie” and close friend, likes to say, “Software development is hard.” He says it out loud and he says it often. He says it because it’s easy for us to forget. It’s easy to think we can know what we want, know the best way to do it, know the unintended consequences of our decisions, know how long it will take to do something that’s never been done before. He’s right. It is hard. Most RFPs seem to assume it’s easy. Building and launching a new product is one of the most difficult tasks you can undertake in business. Building and launching a new software product is even harder.
So I bid farewell to the RFP. I suspect we will still get them, and that I’ll have to come up with a well crafted response that says, “Thank you for the interest in working with us. We’d love to help. Here’s how we can best do that…” While this likely closes some doors, hopefully it opens others.
First published on the DeveloperTown blog.