I have stayed in a womens shelter and gone through years of counseling for domestic violence.
We always class abuse as somebody with the intent to control by using fear.
Do you ever want to make your husband fearful?? Probably not.
Does your husband support you at all in your anxiety troubles? It sounds like he may add to them.
I would stand up for yourself next time and assert that you are in no way abusive. Hope you feel better