Potentially there’s a seal gone somewhere, in the calliper itself. You won’t notice fluid loss, but the fluid passes the seal upon application so you get an initial soft pedal until you get the compression. I’d suggest redoing the calliper piston seals as it’s cheap, along with the obvious fluid change and bleed procedure, see how you get on.
That being said, if it only does it once a year then maybe just bleed it to fix it and consider it an annual maintenance job.