But that's assuming you need a gun to penetrate armor, but you don't. You can penetrate armor with HE or HEAT warheads, and since they don't require high impact velocities to be effective, you can launch them with methods other than cannons. For example: Rockets/missiles (ballistic or cruise), mortars (I know, very close to a gun), gravity bombs, glider bombs, etc. Or you can even not launch them at all (landmines).
With ^, you can penetrate extremely thick armor while carrying much less than an entire cannon. There's only two downsides to these types of weapons:
1) The ammunition (including propellant) is quite large, and thus it is difficult to carry much of it.
2) These weapons are more vulnerable to spaced armor and active defense systems than kinetic perpetrators.
With active defense technology improving, this looks bad for rockets and good for cannons, but we could see great improvements in decoys, jammers, etc. to combat these systems
But, what I would like to mainly discuss on this thread is how much of an impact nano-themite will have on this balance, because I've heard it'll be a large improvement to today's explosives, and the technology would likely affect not only the size of the HE/HEAT warhead, but also the rocket propellants as well (and I'm guessing it'll also affect gunpowder too).
Of course, rail guns could also have an effect.
In general, if smaller weapons become more effective at taking out larger, tougher targets, then the concept of the tank may disappear, leaving vehicles which are more like modern destroyers, in that it's not what armor they have, but what detection, defense, and attack systems they hold (also stealth, speed, and affordability are desirable).
Note: Large naval vessels have evolved in such a way because the balance between rocket and cannon fell strongly into the rocket's favor, but the cannon's downfall wasn't only due to size and weight, but mostly due to range restrictions.