"Oh, yes. Of course. While I am being unnecessarily paranoid, should somehow anyone outside this building ask, my brother's visit is not to be discussed." As extremely doubtful as it is that this needed saying, better safe than sorry.