Doesn't the Horde start working against him as well. Just because he is our warchief doesn't mean we like him. I mean I didn't vote for him. Thrall is why I joined the horde on day one as a shaman.
Don't worry, all will be right with the world soon :) For the Horde!