You, as a manager, are judged on results and not on the work you do and the performance of the team. Imagine paying a guy a huge amount of money and then judging him not on the things he can control, but on those he can't.