Vishal BhatiaPART 1
There are several different ways you can measure your outsourcing and the one you choose will mainly depend on your personal preference.
If you have your own experience with software development then you could measure your outsourcing against your own expertise. You can judge the quality of programming yourself by how well it works, how many bugs there are, and how fast it’s being delivered. You’ll probably always have a gut feeling, and if you’re a software engineer yourself, you’d be wise to trust it.
Of course, you may be the sort of person who prefers to measure things with quantifiable data. If that’s the case then you need to decide on some metrics to measure. These will allow you to objectively measure how well your outsourcers are performing. What’s more, you can create improvement targets that can be easily tracked.
If you’re running a small organization or project, however, measuring by metrics may well be overkill. With less complex projects, you can generally be a little more informal because it’s much easier to tell if things are on track. As long as you’re getting working code on time and on budget, it’s probably not worth worrying about measuring anything else.
Can and should you use metrics to measure your outsourcing?
These days you can outsource many different business processes from accounting, to sales, to customer support. Most of these are easy to track and quantify so the most successful outsourcing vendors have very value-driven propositions. In other words, they win business by showing you EXACTLY how much money they can save you and then proving it by tracking their progress. As time goes on, this approach is becoming more popular in software development as well.
There is increasingly more pressure to make software development and its cost, in particular, more predictable and reliable.
If your project is more than a simple, short, one-off affair, then you should seriously consider taking advantage of this trend and using proper metrics to track its success.
Of course, if that sounds like you then the question becomes: what metrics should you track and what action do you take if you’re not happy with what you find?
Only measure what matters to you
Before you start measuring anything, you really need to decide what’s important to you; what metrics do you care about?
For example, you can choose to measure it in its entirety down to the minutest details so you can make everything as efficient as is humanly possible. Remember, though, there are costs associated with this in and of itself. Or, you can decide that as long as everything is delivered on time and to a reasonable standard, you’ll be happy.
Ideally, you need to decide in advance what you want to track so you can have your requirements included in the written Service Level Agreement (SLA) you have with your vendor.
Your SLA should include your expectations of performance from your vendor. However, it’s only going to be useful if it’s based on objective and measurable metrics you can track. It can be a lot of work to get one properly made up and you may be tempted not to bother, but that would be a huge mistake. If you don’t track progress, you’re asking for trouble; and you may not notice any problems before it’s too late.
The details of your project will determine what you need to track. If you have a one-off project with a fixed set of features, then your main concern is going to be getting everything done on time, on budget and on spec.
On the other hand, if you’re creating, for example, an online consumer product that is going to need constant updating with bug fixes and new features, your metrics will need to be significantly different. You’ll need to track things like response times, speed of implementation – that kind of thing.
You see, there’s no one-size-fits-all metric for measuring software development; it just doesn’t exist. Nor should it; you need to track what matters to you.
How to measure new software development
If you’re developing new software, rather than adding to something that already exists, then you need to track some very particular metrics.
Primarily, you need to track how quickly new features are added and how well they work. In order to do this successfully, you’ll need to split the programming required into trackable chunks, sometimes called ‘work units.’
The more closely you measure these particular metrics, the better; and daily reports are best.
Next, you’ll want to measure how effective your engineers are at predicting their speed of output. At first, they probably won’t be any good: engineers are well known for overestimating their output, but if you track it and give them a metric to measure their output against, they’ll improve quickly.
This will make it much easier to predict the progress of your project and thus improve the quality of output.
How to measure software maintenance
If your project is focused on maintaining or improving a piece of existing software, you’ll also need to focus on a particular set of metrics.
The two main ones are: how quickly work units are completed, and how quickly bugs are fixed. You’ll also need to track the success rate of bug fixes. In other words, how often they pass QA tests.
Getting your SLA right
The quality of your SLA will, to a large extent, determine the success of your project. Its purpose is to standardize the process of developing your software so you and your vendor are always on the same page and always improving.
It’s NOT about allowing you to micromanage, point fingers, or generally try to catch your team in any way. That kind of negative relationship will just make things a lot harder for both of you.
You’re probably wondering how to avoid that kind of negativity, and the best advice we can offer is quite simple…
You both need to agree to the details in advance. Have your vendor commit to a regular schedule for delivering work such as, ‘X work units per day’ or ‘Y features per week.’
Then have them agree to the specifics of how you will define work units, and find a productivity schedule you can both agree to.
What this gives both of you, more than anything, is freedom, because it defines success in terms of the things that matter to you ― like how quickly work is completed ― and allows the vendor to manage the project in the way that works best for them.
For instance, the last thing you want to do is manage the output of individual programmers; and the last thing your vendor wants is you telling them how to manage their people. [A good SLA with set productivity targets for the vendor’s team as a whole so you don’t have to concern yourself with the minute details.]
There is a whole raft of tools for tracking metrics, and your vendor will be able to suggest the most suitable options from which to choose.
In Part 2 we’ll show you the things to measure, and how to keep it balanced, to ensure overall success for your business.
