Measuring Your Engineers

To say that I am managing an Agile team is not entirely accurate, we are just trying to be agile. Anyway, skipping the story, the team's velocity increased more than 800% for the last 2 iterations, mainly due to management pressure. Put aside other reasons such as requirement gathering, design, headcount, etc.

1. Engineers are most probably working within their comfort zone.

For the past 3 months, I had passed/failed/extended the probation period of the engineers. During the session, they discussed with us their strength, weaknesses and areas for improvement.

2. Most engineers highlighted that they're lacking some knowledge to perform better.

So, last weekend, after having a discussion with the management, I was given a task - create KPIs to give engineers a direction as well as to provide management a way to measure their performance. So there are two things - KPIs and Performance Review. The first thing that got into my mind was - how did my ex-company do it?

In the support team that I worked in, we used a ladder-based system where there were 4 skill levels defined with skills to achieve. We would also set KPIs such as "resolving 30 support cases with customer satisfaction above 75%" or "to be SCJP certified". The progress, results and data would be presented to the management during our performance review. Trust me, people seldom stepped out the room wyith a happy face because the management loved to tell you where you did not do good enough.

I was thinking of the ladder since then.

I then looked into some blogs and Stack Overflow questions, such as this (Joel), this, this (Steve Yegge), this and this, which in general send the same message:

3. Management focuses too much on measuring the performance than improving the products and getting things done.

Something quite similar to this will be - timesheet and the reason I abolished it because it is irrelevant with the team velocity and business values. What does it mean then, shall we not measure the engineers at all? Your HR is certainly expecting some kind of metrics for her as a parameter to make salary adjustment.

What shall we look into then? Let's start from things that we shall never use to measure an engineer:

• SLOC - Source Lines of Code, the dumbest ever metric. Good programmers write lesser lines.
  
• Number of bug fixes - No way! The engineers would have figured out how to compromise this by writing more buggy code. I strongly against this just like how I disagree with giving points to bug fixes.
• Hours logged - Timesheet is a waste of time.

I had like a 10-minute talk with the engineers yesterday to find out how they want to be evaluated and paid. I think this is important instead of me making decision and guesses based on how other software houses do it. It turned out that they want to be reviewed by the peers, by stories delivered, by code quality, and by having salary within different range they expect to be put an expectation on the skill set they possess.

So here is my summary, the performance metrics will be generally based on the expectations, the results, and the feedbacks:

• The rating from their team members.
• The rating from their direct supervisor, basically this person also knows the agility, code quality, velocity, etc. of this person.
  
• The total stories delivered.
• The skill set, which KPIs can be related with this, not entirely though.

Are KPIs necessary? KPIs shall not be mandated, they have to be discussed and proposed by the individuals, based on their comfort level and see how far we want to go above it. If a KPI is not going to help an engineer to produce quality code and better product, drop it. Also, if one doesn't want to play this KPI game as he really has a clear idea about every thing he is doing, don't force him to change the way he handles and manages his job and directions.

The last reminder is that, we shall not expect to have a perfect and fair formula that objectively measure and rate an engineer. Whether someone will be promoted or given a good salary shall be based on human judgments, as we are dealing with humans and humans make important decisions like you always do.