Because the principal and agent aren't the same thing. If an agent commits a crime without the knowledge of the principal, then only the agent should have criminal liability.
You forgot about the oversight responsibilities, though. Just because you delegate work to an agent doesn't automatically mean that you are not responsible for the consequences of any action of the agent. If what you say was true not a single company ever would be successfully prosecuted because companies don't do anything on their own, all their actions are through their agents.
IANAL also; but I don't believe English law works like that. The actions your agent performs while acting under your instructions are, on the face of it, your actions. That doesn't protect your agent from liability, but you may have to defend allegations that you were e.g. recklessly negligent.
Because the principal and agent aren't the same thing. If an agent commits a crime without the knowledge of the principal, then only the agent should have criminal liability.