Oh I'm sure it does affect the paint, as does being out in the environment generally, but just not as much as people make out as long you follow some basic common sense practices. I can honestly say that after keeping the Z under a cover for over four years it still looked immaculate on the day I sold it. Did it have swirls if you looked close enough in the right light? Very probably, but so do cars that aren't covered.
Using a cover isn't necessarily about thinking the car will dissolve if it's not covered; for me it was about being able to wash it on a nice day after a drive, leave it in the street for a week and go back to it still looking pretty good. It reduced the amount of time I had to spend cleaning the car (because I liked it to look good when I drove it) and also made what was a really quite conspicuous car less noticeable in the street (trust me - the car got a lot of attention when it was uncovered and none when it was covered - luckily it was always positive attention, but there always that risk with it being parked on the road).
We've moved to a place with a garage now but I'd defo use a cover again if I needed to.
I do agree that if your car is a daily driver a cover is probably more trouble than it's worth but if like me you got to drive only at the weekends and had to park in the street, it was a worthwhile investment. Horses for courses and all that.