Yer, you're totally right, it's not the most damaging thing really. But I know what you mean about triggers. To be honest I simply find the word 'shrinking' jarring and don't use it in conversation, that's about it. Fortunately.
Some women have mentioned it in conversation, not knowing about my fetish before, but I've not been too uncomfortable with that.