As above.
I'm relatively new to quality car audio, but I know a bit about quality home audio and the amp makes a huge difference to your overall sound. The speakers are simply the last line in the production of sound. It starts with the media (e.g. cd or ipod), then source (the head unit/cd player), then the cables to the amp, then the amp, then the cables from the amp to the speakers, than lastly the speakers themselves. This is like a big river, flowing from source (head unit) to mouth (speaker). To get the best from the speakers everything "upstream" must be as good or better than the speakers themselves. If the head unit is crap, million pound speakers will still sound poor, likewise for the amp. Imagine if you play a rubbish tape on a great hi-fi, it can't make the tape sound great.
If the "flow" from the source is restricted along the route, you can't ever get that back downstream, it is held back by the upstream components before it ever reaches the speakers. The idea of any system is to get as true a representation from the source out of the speakers as possible, quality can only ever be lost along the way, it can't be added in.
Bit of a clumsy analogy but I think it just about makes sense!