Is there a formula that determines how much experience you get from a mob?

*Of course there's a formula, doh! Dunno if someone's has bothered to estimate one, because we don't really need to know it. The higher the level of the mob compared to your character's level, the better the exp. Opposite alignment mob yields more exp. If mob has special abilities, i.e. is somehow tough(er), it generally gives more exp.

What’s the multiplier for two in a group and three in a group?

* It's significant, i.e. noticeable for two and better (best) for three. Again, the exact multiplier does not really matter.

I’ve heard there’s a way to use mercs to level now. How does that work?

* Haven't tried them myself, but I understand it's the same as if you were with another pc for those mercs that allow it.

What if I’m in a group where I’m level 30 and one is level 35 and the other is 40 compared to all of us being 30?

* In such groups mob level tends to be the most important factor, i.e. gains would be sucky for the L40 guy if you were fighting mobs say L40 or just below, but nice gains for you at L30. Anyway, I suggest just playing the game, it shouldn't be too hard to find a suitable place to rank for your group!