I think that he's guilty for not being able to protect his mother. In the manga that was never finished it was made clear that he saw the demons killing her mother and Dante was hidden in a closet, Dante shares the same feeling but the trauma not made him go so far to compensate that tragedy.
I think the game itself throws tips that it's guilty, notably because Vergil tries very hard to not show emotions and be more like a demon, and they never talk about their mother too, it's kinda weird but i think they do not talk about this directly because it's really painful for both. But in the end of the game he gave up to act like this (but in a classy and very "Vergile" way), look again at the scene where he falls in that waterfall in the demon world, he leaves other mark in Dante's gloves (and show Dante keeping this glove in his shop) to leave a keepsake of him with his brother. He cares for the peoples he loves, but his trauma made him go after power at all costs. And one point that is recurrent on nearly every DMC game is the "human factor", the human abilities to love and care for each other, and he barely have a conversation with Dante that goes this way when he say to Vergil that in their veins flow the blood of Sparda, but "more importantly, his soul", i read this like his ability to fight for the humans and care for them that only Sparda had within all the other demons. And in the end, sounds like Vergil understanded this message when he tries and fight Mundus in the demon world, he says: "If my father did it, i should be able to do it too!", when in the whole game he shows resistance in calling Sparda "my father" (and it feels like a emotion of "betrayal", by his father disappearing and leting his wife and childs without protection, Dante look to see it in the same way at the beginning of DMC3). In the end he says proud that he is a son of Sparda when decides to keep his amulet, and even decide that he will stay in the demon world because "it was our father's home"
Obviously, i'm mixing my own speculations with hints given in the game, but is as this that i see it. I think he let his feeling of impotence while his mother was getting killed spread on all his life and dictates for what he would live for. He thinks he needs power to protect whatever he comes to love, or besides he wanted that power to kill once and for all his humanity and never feel "sad" anymore.
Check this cinematic and the other before this, try to see the way i put it here and something new might come to your mind.