Like I said, his mother was human. He might hate his human side but he seems rather indifferent towards humans in general, if one gets in his way he'll undo them but otherwise, they're there, who cares. He also doesn't really have a close relationship with his other side of the family since he kills demons left and right, too.
Anyway, it's not impossible for Vergil to become enamored with a woman if that woman shares any traits with his mother, whether it be a physical trait or other miens. The point is it's not impossible for him to have affections towards a human woman if the conditions are right. The same could be argued of a demon woman.
PS- I think this might give the impression that I think that it is infact true that Nero is Vergil's kid. Actually I don't think it's true, I'm simply arguing that Vergil loves his human mother and to love another human is not that much of a stretch.