We know the Silence were able to blow it up. We have yet to learn how and why they blew it up.
We don't know the Silence blew it up. We only know that a voice (not yet identified) said, "Silence will fall." Moffat is famous (?notorious?) for using red herrings to mislead the audience & that could be one of them. It's not safe to assume it means what it seems to mean.