I think it matters with overhead, but axes are heavy weapons and something like a 'pincer' from a giant (whose arm is bigger than a svirf's entire body) seems like it would hurt a lot more than from a shorter person. Add in the effect of leverage (no, not the staff spec skill) and it would make giants the hands down winners.
I don't like seeing giant warriors being the less viable option for strength-based weapon specs. I don't think they should be as good with swords, but I think axes and polearms should be better with them. Possibly maces.
Anyway I'm rambling.