I have had a few cavern-dwellers. Two Drow, two Svirf. They are supreme in their home terrains as you said, but only average if not lackluster everywhere else.
Unless you are ranking in a place where you can take advantage of it, you aren't going to have a lot to do. Trying to get people to fight you in caves is going to be difficult, if not impossible. But if you get somebody stupid or unaware, you can pretty much guarantee they die. Meanwhile, every time you step out of the wilderness, you are just easy prey.
I like the rangers, they can pull off a lot of impressive moves that would kill anybody else. It just gets boring after awhile being so specialized and tied to a few specific areas.