I don't really collect card backs, but I have a few from various deals and purchases.
I think like Darth Father has said, a lot of it has to do with the nostalgic feeling of seeing those cards, and remembering what it felt like to tear a figure out. Some people actually took time and care, and kept their card backs in nice condition. It can be done with a little patience. Other people may just want to try and have the figure as close to 100% as they can. Focus collectors may want to have every version of a card their focus figure was shipped on.
There is a local comic shop near me that has a few card backs for sale. I picked up a Luke Stormtrooper, just because I have a special tie to that figure. Not a good tie, but a special one.
When I was a kid, my mother and I were in a Zayer store. I saw a Luke Stormtrooper figure, and when I ran over to my mom to ask if I could get it, she said I could. Problem was, I didn't take the figure WITH me. I left it on the peg until I asked her. In that two minutes, someone came along and bought it right out from under me! I went back, searched EVERY figure n EVERY peg. GONE!!
I like to think that whoever got it really didn't know they took it from a kid, but if it's anything like snipers out there today, they don't care.