There's too much risk in him being here for it to be practical. If he were to slip for even a second, and let one of us get to him? I'd like to believe he's smart enough to realize that would be bad for him.

He can create the same effect from speaking outside the building, or at the very least, somewhere absolutely impervious and undetectable within this building. If he's in the lower floors, then he's leading us to him, and I don't imagine that bodes well for us.