I feel with you! I wish I could give you some good advice but I have non. I often go to the bathroom I think people would expect me because of safety reasons, but then I'm always so damn uncomfortable.
Depending on the building I go to the most remote bathroom existing and decide there if I have the courage or not.
Is there maybe a room besides the bathroom to go to for taking off your binder?
What I can tell you: You are not alone. And I think you're doing great in taking care of you and giving yourself a break from the binder despite your situation.