From what we've seen of the process, it's the time period. It's not truly forgotten, however. It remains in the subconscious but isn't consciously remembered. That makes it ideal for implanting instructions that function in the same way as post-hypnotic suggestions: The victim is unaware of being given the instruction but does obey it (where possible).
When the Doctor showed an image of a Silent to Canton & accompanied that image with an instruction to straighten the Doctor's bow tie, Canton couldn't remember either the image or the Doctor's instruction. It wasn't only the sight of the Silent that he'd forgotten but everything that happened while he was looking at it. He still did straighten the Doctor's bow tie, as he'd been told to do, even though he'd no idea why he was doing it.