Can't hurt, but removing weight from over the driven wheels may mean you lose a bit of traction? You might notice in the wet when turning sharp corners, but sounds like you are putting weight back in to replace it.
I'm sure there are people who have removed it and haven't instantly come off the road! Just get some spray repair if you get a puncture.