As Amy said in "Asylum of the Daleks", whatever Kovarian & the Silence did to her while she was pregnant with Melody left her unable to have any more children. We don't have details of just what was done to her or why it had this effect. Her inability to have more children may not have been the purpose of what was done, just a side-effect of whatever they were doing to turn baby Melody (River Song) into a weapon against the Doctor (see "A Good Man Goes to War"). The Silence would be unlikely to care what damage they did or didn't do to Amy, as long as they got the baby they wanted. (Madame Kovarian, though, showed that she was the type who'd positively enjoy doing damage.) It also could simply have been the trauma of those events.