The Silence, really, is The Church. And all The Church wants to do is avoid the possible continuation of the Time War.
However, their methods of doing so have been rather monstrous. They would have destroyed the universe in "The Pandorica Opens"/"The Big Bang" had the Doctor not rebooted it, they treated a living, conscious child (Melody Pond) like an object, and manipulated her into becoming a war machine - not to mention taking her from her parents.
And, they have, of course, attempted to kill the Doctor on many occasions.
While they believe they are doing all this for the greater good, The Silence are, in most people's books, monsters.
Note: 'The Silence' refers to the religious order constructed to kill the Doctor and stop the Time War. 'Silents' are a species that work for The Silence.