A few points I agree on: the difficulty, the reaction. the failure to accommodate.
I have my own trouble getting to my doctor, pdoc, and T appointments. Part of this is my fear to parallel park. I am terrified that the street the offices fall on will be packed enough forcing me to do so. I have spent a great deal of money on cab rides. I also am fearful of requesting time off work (I don't wish to trouble them or be a pain if that makes sense) which makes scheduling appointments all that much more difficult. When the day of my appoints arise I panic to the point I throw up. I have simply called in at times with a lame excuse.
I have felt my healthcare team fails to understand these triggers and why it is so difficult for me. Certainly the office workers are part of the problem - admonishing me for not being able to showl. Another dread about appointments.
Sitting in a waiting room is difficult. I worry about what others may be thinking about me.
Whereas the doctors get angry and harp on my failures the T is at least more helpful. When I do miss an appointment she will call me and conduct my session by phone.
I see no reason you own doctor can't accommodate you by doing the same.
Also, is it possible to get an appointment during a less busy time - like first or last thing in the day?