I've only just installed my spacers so can't comment on the difference of handling yet, but I chose to go for less extreme widths (15mm front, 20mm rears) as I was also worried about the negative effects of handling.
At these widths the wheels now fill most of the arches and don't look as obviously "retracted" as OEM. However on reading about the negative effects on handling it doesn't appear that many users have had reliability issues. I think installing lowering springs causes significantly more wear on the suspension in contrast to the installation of spacers but I stand to be corrected.
My personal opinion reflects those of Jetpilot and Sargara: the benefit of having wheels that look like they fit is experienced 100% of the time I drive the car, versus perhaps the 10% of compromised handling when the car is driven enthusiastically.