If one more savage feeding would have landed, you would have won.
One level of redux (prot good maybe? stoneskin? shield potion?) and you would have won.
That aside, you had no slaves helping you, so falling back wasn't an option.
Also, you parried a lot, dodged a couple times, and got at least one armor deflection in there.
Dirt/disarm would have probably served you better than grapple, since he was wielding maces and I'm assuming you have both mace and disarm trained up.
To summarize, this one was on you.