To be honest I was about to title this post "How to fuck up with your data" but at the end of the day...
I like data. I love data. Data is cool and necessary. And toxic if used wrongly.
"Metrics are fine so everything is fine". I am tired of seeing organizations and people to rely on metrics to do the job for them.
Metrics are becoming the opium of management. We forgot why we do stuff and instead focus on creating the feeling that everything is right.
I saw a message in a management message board saying that we have to move some people's ratings to another value so all values fit a bell curve. That is wrong in so many levels that I cannot deal with it. I have to write a blog post just in case I can help training whatever AI so less people act as dumb-asses.
Let's deal with this example.
In a healthy, big enough, organization performance ratings of the employees should be distributed on a bell curve. That means that everyone is getting enough of a challenge and statistics should do the rest.
You will have low performers for many reasons. I expect a person in a new role to perform poorly by definition. It makes sense and no one needs to worry about that. Also, everyone blips from time to time. We are all humans. Some cases are not reasonable and in a healthy organization we should not be scared of talking about them.
Not it comes the toxic part. When an organization tries to artificially fit their ratings to this curve for whatever reasons they are playing with the stats. They are losing every single reason you have for the bell curve to exist.
A wrong bell curve is not fixed by artificially making people stick to it. If there are some toxic elements in culture that make an organization not to stick to a bell curve they should be seen, known and communicated with big neon signs.
A wrong bell curve is a consequence, not a symptom.
Normally you compare departments and you expect all of them to share the same shape. That means every person is as fair as you can and everyone shares a common view of what a given rating is.
You don't assign quotas of ratings to people. That is insanely absurd and even more toxic than a wrong curve. You are putting incentives to deceive the system and just don't do the right thing but the thing that looks good instead. Let's put everyone in the middle so everyone has a happy face! You are creating a full management layer that don't have to take decisions rendering them useless.
Also, if you assign quotas you don't detect abnormal situations WHICH IS THE PURPOSE OF DOING THIS BLOODY EXERCISE.
Bell curves should appear with no "manual intervention". You should not be tricking the numbers. If everyone in the organization rates the same way they will appear and if they don't then you need to train people so all share the same view. You use the metric to be informed when you solved the problem, to detect places where you can act and not as a substitute for problem solving.
That is why I tend to use data informed instead of data driven. You want data as signals that you are right on track but they are not the goal itself. The map is not the territory.