Without a doubt he's hurting inside far more than you are, otherwise why make a derogatory comment at all?
Little people like him always try to spoil the pleasure of others in order to try and bring them down to their own miserable level.
If I were you I'd go into subtle mode, just give him a beep on the horn, a little wave, a and a blip on the accelerator each time you see him getting into his crapy little car or walking for the bus and you’ll have this prat seething even more than he is now.