To be specific, this is not an issue with lag. Each time it happens, I can see monsters moving around on my screen, and when I am rubber-banded back to some location, I don't see any change in the normal behavior and movement of monsters, they seem to be unaffected. I've also had this occur in both multiplayer and single player sessions.
I have a suspicion that this bug is tied directly to desecration. Elites with that affix cast desecrate on me, and then when I try to Tempest rush away, I get rubber-banded back on to that exact spot. It's very frustrating! This bug occurs against elites that do not have the vortex affix, so I'm certain that I am not being pulled back by that ability.