While you aren't directly taking something(removing a posession) from someone's person when you download it - you are removing their ownership of the item, and their abiltiy to control how that item is exposed.
You're not home. I go into your house and live there for a while. I don't tell you. You come back and everything still looks fine - just like new. While I didn't steal anything, I've still taken away your ownership of the house, I've removed the ability for you to say when it is used and how - which you should have, being as it is your house, right?
Maybe that's a bad example....Bleh.
It's not that I like copyright stuff, or think the law is awesome - but I do think it's there for a reason and is useful.