This indicates that the man was in service in the same part of the imperial palace as Izumi Shikibu, and so she could hardly help but come across him in the normal performance of her duties.