He doesn't respect you. If this continues, you should leave him. Self-care does not equal selfishness.