Team Glory decrease

Has anyone noticed the team glory total go down after attacking Castle guards or when players are out attacking each other in NML. We noticed this the other day and I cant find any reason why it should go down. and no we did not hire any castle guards. Did i miss a update or maybe never noticed until now?