I think its entirely possible that if you're experienced, you can beat even well-prepped invokers 4/5 times with a villager. But killing them is another matter altogether, and is probably beyond most solo ragers.

Meanwhile from the invoker killing the rager perspective, unless you're popping gossamer wings or willing to eat vuln_lightning, you're going to die to quicksand/dig/vortex/web/etc. A lot. Invoker + one gang buddy = GG most ragers.