I do not believe dex has any say at all.

The Ac bonus is the armor you are wearing, think about it that way to make the best use of it.
Giants likely do well because of the heavier armor you can wear, the more choices you can justify; Dwarf/Duergar, yes and yes, I seen a svirf rock it; anyone who knows gear can use it well......Its a legacy that is useless until you can regear to a decent amount though which can be frustrating, because you spend so much time getting the ac to a good level and start over when you lose it