Dogs don't like being stared at, especially by people they don't like.
And maybe he suffers from little dog syndrome, seems to me many of them do.