How Many Agents Does it Take to Beat PyTorch? (surprisingly not that much)