People & HR

Did the same employees score higher after training than before?

Whether training moved scores is a common people-analytics question. This runs on the library's sample before-and-after scores for employees.

The data. standard_paired_comparison_v2.csv, the published sample file for Paired comparison: 120 rows; columns Employee ID, Team, Score Before, Score After, Passed Before, Passed After. Sample data, not a customer's.

The calculation. Python, standard library only. This exact code ran against the file when this page was built, and printed the output below it.

import csv, statistics as st
rows = list(csv.DictReader(open("standard_paired_comparison_v2.csv")))
diffs = [float(r["Score After"]) - float(r["Score Before"])
         for r in rows if r["Score Before"] and r["Score After"]]
k = len(diffs)
m, se = st.mean(diffs), st.stdev(diffs) / k ** 0.5
t = 1.98  # t critical value, 95%, about 117 degrees of freedom
print(k, "complete pairs of", len(rows), "rows")
print("mean change", round(m, 2), "points, 95% interval",
      round(m - t * se, 2), "to", round(m + t * se, 2))
print(sum(x > 0 for x in diffs), "improved,",
      sum(x == 0 for x in diffs), "unchanged,",
      sum(x < 0 for x in diffs), "declined")

118 complete pairs of 120 rows
mean change 3.29 points, 95% interval 1.93 to 4.66
73 improved, 1 unchanged, 44 declined

What it says. Across 118 employees with both scores, the average change is +3.29 points, with a 95% interval of 1.93 to 4.66. Because each person is compared with themselves, differences between people drop out, which is why a paired design usually needs fewer people than comparing two separate groups.

What it does not say. 44 of 118 people still scored lower afterwards: an average gain is not a gain for everyone. Without a group that skipped the training, this shows change, not what the training caused. 2 rows missing a score were dropped.

Run Paired comparison on this file or your own, or open its stored sample report.