You're adding weight to the worst possible place, so there will be *some* negativity. There will be resistance to turn, slower acceleration and longer braking distances.
That said, the differences are negligible to most. Some will notice it and hate it, some accept it because the car looks better, most don't notice that much if at all. I run spacers on my car, however when I changed wheels I made sure they were lighter so that even with the spacers I've lost weight on each corner. If you're not changing wheels then expect the car to feel a bit different, but tbh a sharper geo can sort that out.