I've not had any experience myself, but know a few people who use them.
I don't think you have anything to be nervous about, as CSC system cameras are a growing trend, particularly amongst DSLR users who are fed up lugging heavy gear around all the time.
I've only heard good reports up to now.
I know one chap who owns a Canon 5D Mk3 with a good selection of FX lenses and finds himself using the Olympus 90% of the time as the Canon gets left at home more and more frequently.
My daughter uses a Sony RX-10 with a smaller 1 inch sensor than micro 4/3 and the pic quality is excellent, so you shouldn't have a problem in that respect.
The lenses are fast and very reasonably priced....whats not to like!!