Looks like it was a bug.
In today's release notes of the CS:S beta:
Players are no longer kicked for team killing during mp_spawnprotectiontime if mp_autokick is not enabled.
We do have mp_autokick off which is described as
Kick idle/team-killing players
if it is on, so we've definitely been suffering from this bug.
I've turned mp_spawnprotectiontime (which I didn't even know existed), to counter the bug until the update is pushed to the non-beta version.