The warning bells and spidey sense is going off.
Mother's tend to think quite highly and defensive of their sons. Thus, for your boyfriend's mom to speak in less than glowing terms - indeed to warn you - about him is truly something to take heed of. Now, there is a chance she is using such a tactic to split the two of you up because she isn't fond of you but many more ways to drive a wedge between the two of you come to mind. If this was the case there is no way your boyfriend would leave she and you alone together. No, I would then take her warnings seriously.