I think an important thing here is, you don't like it when he rubs your leg, and you do not have to put up with it. It is absolutely your right to decide how you want to be touched and by who. Regardless of what his reasons are, you can say, "Hey Dad, I don't like that, please stop it."