Yikes, it sounds like there are a lot of issues going on. I think since you do live together and the kids are caught in the middle you do need to get couples counseling ASAP. It does sound like your BF needs to set better boundaries, rules and expectations and you probably need to step back a bit and try to be more nuturing. All the kids are still so young. If you are going to continue to live together please do not leave when his daughter comes to stay with you. That is sending a terrible message to her and will affect her for the rest of her life. It sounds like both of his kids have been abandoned by their mothers and they bounce around between the grandmother's houses and your house. That's going to affect them and they need a good balance between disciple and love and support.

Good luck!