Number of people on CF should not impact you. Also, it's not like you're streaming some bandwidth-heavy resources, like online video. Text based gaming is pretty much the lightest impact you can get.

What might be happening is that your ISP might start dropping packets. Here's a simple test: when you start mudding, open up a terminal window and ping carrionfields.net (or google.com) indefinitely. When you start noticing lag in the mud, check your ping replies. If they start timing out, it's not CF, it's your ISP.

In windows, run hit "windows" key and type "cmd'. This will open up a terminal. Type "ping -t carrionfields.net" or "ping -t google.com" and this will start an infinite ping loop to your host. Hit "ctrl+c" when you want to break it.