After some time, really thinking about it, I do think it's a mixture of not knowing what to say or do, and anxiety. I just realized I have felt anxiety of saying certain things to certain people.

I think my problem is becoming a little bit clearer with that. I couldn't really see myself stopping myself out of anxiety until now, as I thought about it in hindsight. So maybe something conditioned me to have that anxiety.

But I still don't know how to work on it. But it sort of helps knowing that anxiety, confidence issues, really is a part of the problem. It means that the next time I feel that anxiety again, it's because I had a thought that I rejected using, and I can try to make the choice to use it.