This is explained in "The Time of the Doctor". The Time Lords are locked in a pocket universe (shown in "The Day of the Doctor"). They are able to communicate through the Cracks in Time. If the Doctor answers the question correctly, then the Time Lords would know they'd found their home universe & could return to it. The Silence fear that the return of the Time Lords would restart the Last Great Time War, with all its attendant horrors.