As the others have said, its probably a bit of a few things that compound into making it hard to buff off. Collinite is notourously hard wax to apply but does last long so has its rewards. Next time, make sure you do nice thin layers, should hardly see it on the car, just a blurry look to it, doing a panel at a time, and it should be easier to shift. The cold wont have helped either as it would be hard to get it to spread evenly and thinly if it was cold.