Indeed. Love it!

As usual:

Step 1: metric is used to indicate performance

Step 2: developer finds a way to hack the metric

Step 3: developer profit

Step 4: metric now considered unusable

LOCs, #of issues fixed, ... all in the same bag.

Tell that to people doing machine learning!

Isn't that kind of why they use a training set and a separate evaluation set?