Software Consultants
I worked over a decade as a consultant, and used and managed them for a couple decades more. As a consultant I've worked for organizations (agencies) and as an independent. I have nothing against consultants or consulting (they're a very valuable resource), but there is an art to using consultants or consulting organizations wisely, and most companies don't quite have the artistry required. This article will try to explain some of the pitfalls and ways to use consultants better.
Contents
Basics
For the most part, you can think of independent consultants as just "little" agencies; with higher risk, but lower overhead, usually lower cost and your contract is more important to them. Rather than outsourcing whole solutions to independents, most act as "guns for hire" and do a lot more of being hired in "late" on struggling projects, to help bail water or stomp fires. There are too few used to help prevent problems or sanity check directions before it is too late. But the point is that there are different specialties in independent consulting. Assuming that your consultant is acting as their own agency, then you can treat them accordingly. But if you don't need a full time employee or can't find them, then sometimes the "gun-for-hire" type can be used more as a pseudo-employee, especially if you put them on retainer or other long-term contracts (so their interests more match yours).
The first thing to remember when working with consulting agencies is "their interests are not your interests".
The agencies interests are to get your business (sell you their services) for as long as possible and get as much money from you as is possible. You want to get a solution as quickly and cheaply (as little money) as is possible. Your interests are in direct conflict - you are not just allies but also competitors. Now many partnerships have an element of competition, and it doesn't mean that they are "out to get you" -- in fact, most want to do a satisfactory job, so they can get your future business. But many forget this aspect of the relationship and what it means when it comes to software contracts.
Time is money; for them
Software people already have problems with "underestimating" how much time project take - but consultants are usually worse. Consultants have a vested interest in "low-balling" how long it will take to get a job done; the lowest bid usually gets the contract, and they all know this. So they don't lie, per say, they're just very optimistic. The more optimistic they are, the more contracts they get.
In most cases, consultants or agencies are paid by time. The longer the task takes, the more money they make. If they miss-bid, then it is your loss because you'll keep paying to get the job completed. Or if you don't, you throw away a lot of work/money - so most people will keep throwing money at the problem. Even if you won't keep paying and you decide to cut your losses and walk-away after a few months or a year or whatever, they know that they still made more money on that partially completed contract than they would have made if you had never picked them in the first place. So if agencies want to win contracts, they must be very aggressive with schedules; like politics and law, the biggest liars win, and put pressure on all the rest.
Other pressures effect them as well. If the market is down, they have more pressures to get the business, even if that means more "creative interpretation" of the issues (or time) in order to get the business. If business is up, then the higher a premium you're going to have to pay to get their services.
Fixed Price Contracts
Now some people think that fixed price contracts will help eliminate project / time slipping problems, because consultants aren't paid by time but on delivery (milestones). This can actually cost you more in both time and money.
In these cases, the agencies have a vested interest in delivering the minimum required to meet the contract, in the least amount of time. But if you think that means you hit your goals sooner, you'd probably be wrong.
Fixed price contracts are all based on the requirements - but requirements can be interpreted many ways, and odds are the agency will interpret things in different ways than your companies goals. In these cases, the agencies want to interpret the requirements in ways that take the least amount of time to hit. But your objectives (and what you meant by the requirements) are probably not the easiest way to hit the requirements, and are often the hardest way to hit the requirement. So once again your goals and the agencies goals are in direct conflict.
If your requirements would hurt the consulting agency, then they can't afford to give you what you really want, so they choose to give you what you asked for (in ways that cost them the least energy). By doing what you said, the agencies save time (and money) - but they have a bigger win than that. By meeting the requirements to the "letter" of the law, but not the spirit, then they have a double win, because now they've not only saved money up front (and met the obligations in the cheapest way), but you didn't get quite what you wanted. That might not sound like a win, but in order for you to get what you want, then they get to renegotiate for the next set of goals, and you'll pay them to try to hit those 'revised' goals; which means more money and another round of consulting for them. This becomes a game of ever circling your goals while never quite hitting them. As long as they don't quite hit your goals, then you'll keep paying them to get closer while probably adding new goals to each version as well. So they want to get close enough to keep you happy, but never quite get there, so you'll keep on going for the next round.
This whole problem can become much worse when you're having the agency itself generate the requirements that they have to meet. They have lots of experience in creating requirements in ways that they can interpret to save themselves the most money (and cost you the most later on); imagine having your competitor represent your interest in contract negotiations and you're getting the idea.
Right solution, or only for them
Consulting agencies often know one tool or way to solve a problem; they are most often specialists. There is a saying, "if your only tool is a hammer, then every problem will look like a nail".
If you ask a consultant or agency that is known for one thing, what tool you should use; then you can guess what tool they will suggest. I don't think they'll lie; but they're good at telling you what you want to hear, or in defending the tool they are using, and believing it - whether it is the best choice for your goals or not. Again, their interests are not yours. They usually know one tool and so see solutions in terms of that tool.
Even if they have multiple specialties (in-house), they still have pressures to interpret the "best tool" for you, based on what resources they have free. Or the other side is that they might recommend the most expensive solution they have; like the results of asking a car salesman which should you buy the deluxe model or economy car. It's only money.
Easy way or the hard way
Because it is expensive to train all your own people on complex tools or hire people with expertise (of complex tools), consultants have a vested interest in complexity (not simplicity). If they choose a large and complex tool, and add tons of their own complexity to it (their own libraries and code base), then they provide the most value (in contrast to what it would cost you to buy that expertise). And so consultants tend to gravitate towards the most complex tools and the most complex solutions to your problems. But that expertise is illusionary, since you might not have needed that tool or complexity in the first place, and do you think they're going to correct you on that?
Consultants often push for very complex things in the solutions as well (in the bidding part), the more they can get into the contract, then the more money they can make. The more they can "interpret" out of the contract that they already have, then the more they make.
So consulting agencies inflate the expertise of the solution required (by the tools they choose), and then that increases the cost and the need for more consultants. And early they try to convince you that you need more than you need; features and complexity, just like other salespeople. And they know that if they can make it so complex that no one can figure it out, then you'll have to come back to them for fixes and additions later on. Again, this is not nefarious and is a tad more subtle than it sounds. But there's a pressure for them to use complex tools, and create complex solution that not anyone can fix, so that you will need them. And they want to add every possible option, and charge you for it.
Long term or short term
Consultants or agencies interests are shorter term; usually for the duration of the contract. Usually they aren't going to get the maintenance contracts, and if they do, then you'll pay. So they have a vested interest in delivering the best solution to get them paid; but if it costs you more in maintenance, then that is your problem.
Most agencies and consultant don't want to do "the wrong thing". But there is an economic incentive to do it "quick" and increase their short term savings over your long term maintainability or long term goals.
While developing the solution, if you want to do something "the wrong way", even if it is a mistake that you'll have to fix later, or that will cost you more time later, then consultant know to shut up. This is not only safer politically (not making enemies), but later you'll have to pay them to fix it. That doesn't mean they'll lie to you to fake you out - but their incentive to fight you to do "the right thing" is a negative incentive. So you are more likely to get the one, "I don't think that is the best way to do that", and that minimal effort covers their butt for later - after that, the problem is yours.
Employees have plenty of problems as well. There is the politics and other things going on. But ultimately, most employees have a larger vested interest in delivering what companies want, with the best long term tradeoffs, in the least amount of time. They know that if they do something wrong, then they have to fix it, maintain it, support it and will get the blame, all under crunched deadlines with little or no overtime pay and so on. So your employees are generally longer term thinkers than consultants.
Not black and white
Now there are different degrees of integrity between organizations and people. And all of them have to get close to your goals, or to convince you that the reasons for the delays in time is because of your interpretations of the requirements (or other things that are your organizations fault). And in most cases they are your problem; caveat emptor (let the buyer beware), and you need to know the rules before you play the game.
Consultants don't want to hurt their reputation and they want you to keep using them. They do this by delivering as much as they can, and as close to your goals as they can; so it is not like they are out to screw you, or they are all rat-bastards. Quite the opposite, they are trying to deliver something to you - but the realities of the market put pressures on them that will get passed on to you. And their interests are in conflict with your interests, and when one has to give, odds are it will be yours.
Employees come with their own set of issues as well. So the point is not to avoid consultants; just know how their interests differ from yours, and how to use them wisely.
Solutions
Now there are ways to help solve all of these problems.
Generally fixed price contracts result in low quality or missing expectations, and rate/time contracts usually mean missing on deadlines. So balance the two. Try a little of both, paying a rate, and paying incentive for on-time delivery and on-expectation delivery (the latter being harder to quantify, but if you keep control it can still be a motivator). If done well, then you can balance the extreme pressure and get better overall results.
In order to use consultants effectively, you need the expertise in-house to know exactly what you want, and how to ask for it. If you are really good at creating the requirements, then you minimize the problems with "misinterpreted" requirements. So the better your communications, the less room for communication errors.
You also need to pick the tools you want to use, and have justifications as to why, before you choose a consultant. Or choose one agency to do the business analysis and "tool choice" with them knowing your business needs and full well that there is no chance for them to get the follow-on contract or recommend to you who will; so that reduces some of the bias.
You need to take the time, energy and resources to monitor every step of the agencies progress and keep them on track. The sooner you catch things then the less they cost to correct, the more you know what you are getting, and the more you can refine or revise before the pressures push you to just "ship it".
You need enough on-staff experience that they can be working with the agencies and looking out for your technical interests. If you don't have it, then you can consult out that service or maintenance contract to someone who has no ties with the other agency, but it is safer to have that knowledge in-house. Since your staff will have to maintain things, they have interests more in line with your organization. This includes fighting for less unnecessary complexity and simpler solutions, trying to make sure that designs have long term maintainability, often fighting for more documentation and built in test/support code, and so on.
Remember, consultants are not better or worse than employees for organizations; there are good and bad variants of both. And both can be thought of as tools or resources to your organization (while still remembering that they are people). Your success or failure with either will be based on how good you are at utilizing both of their unique strengths and weaknesses for the job you're trying to accomplish.
Written 2002.03.12