Some of the oddities are:
-The land it sits on is similar in shape to a Human skull.
-There was an explosion in the area at an unknown time that "weakened reality" in the area.
-It is unknown who built the tower.
-People who visit see visions of the past and future.
-It is believed that all ley line in Azeroth meet at some point underneath the tower.
-The top of Karazhan is a portal to the twisting nether.
-The tower has an upside down mirror version of it underneath normal Karazhan.
-The tower appears to be at least partially sentient in that it sealed off reverse Karazhan on its own.
-Yogg-Saron possibly refers to it in the quote "The giant rook watches from the dead trees. Nothing breathes beneath his shadow".
-The Maiden of Virtue, a titanic watcher, what is she doing in Karazhan?
-The Dark Riders appear to come from Karazhan.
-There is a room full of corpses suspended upside down on hooks underwater called "the upside down sinners".
What do you think this all adds up to?