Couple of things.
For one, being away from the Hellgate does not cut off Mundus from his immortality and power, so he has nothing to fear from being out of his office. We've seen photos of him totally not in his office. Y'know, the ones bouncing around on the walls when he and Lilith are bumping uglies.
Secondly, Mundus knew of Vergil, but not that he was anything more than a dude privy to the demon's plan and a thorn in Mundus' side. Mundus also was not able to sense Dante, he sensed the death of his subjects, but could not sense Dante in any way, shape, or form. That's why the Hunter demons exist in the first place, and if he could have sensed Dante, there's no reason why he couldn't have killed him when he was a child.
Also - the question of "they knew where Dante was when he was younger, so why did they let him go?" is silly, because it ignores the fact that they had only a first name to go off of. I'll tell you to go find a kid named Jimmy in an orphanage and we'll see how well you pinpoint the exact one I was looking for. Then of course, as soon as Dante realizes he's in the presence of demons, he goes batsh!t and cuts his way out of the establishment. He never stays long enough to get caught. Hell, I'm still skeptical if they had his name when he was a kid, even, and not something else they picked up while searching for "the nephilim".
Mundus only knew of Dante because he saw the kid while he was busying killing Eva. Vergil was nowhere in sight, and neither Sparda nor Eva would reveal the existence of their children. Dante just happened to be there, so they knew. They didn't even have Dante's "scent" up until recently, only having a name to go off of for the longest time. Vergil got away cleanly, they had no idea he even existed, and Vergil became hella good at covering his tracks, given that he could operate a global network of anti-demon conspirators without anyone finding out anything about them that the Order didn't want them to know.
How did they know it would work? Pretty much because the legend said they had the power to. Take that for what you will, but when a Nephilim is destined to be the only thing that can kill a Demon King by shutting the Hellgate and all that jazz, you take what you can get, and that was a slam dunk.
So let's recap - the only way Mundus could have lost his power and immortality is if a Nephilim got close to the Hellgate. As he knew of Dante as the only nephilim at the time, he figured he'd be totally fine tackling Dante through a few buildings in a fit of rage. He had no reason to assume another Nephilim existed that could have been there.
