The composers/arrangers do it because they're told to. If someone hires you to provide a new version of the theme, and you give them back the exact same version as the original, they're not going to be happy. The only exception to this is Murray Gold's 2008 rearrangement of the opening - he was originally asked only to change the closing theme (because the BBC's new credits policy required the closing credits to be redone), but decided to revamp the opening theme as well.
The producers originally conceived of the idea of rearranging the theme for the beginning of Patrick Troughton's run, as a way of indicating that the show had changed. (In fact, even before that change, the producers had asked for a change between the pilot and the original story.) Later producers have just been following the lead of the originals, using a new theme to indicate major changes to the show, usually but not always a new Doctor.
Major changes were made with:
- The start of the 2nd Doctor (although accidentally not used until season 4).
- The middle of the 4th Doctor, starting with "The Leisure Hive", heralding the change in style brought by new producer JNT. (This is the one Peter Howell did.)
- The middle of the 6th Doctor, starting with Trial of a Time Lord, heralding the change in tone ordered by the BBC on revival of the show from hiatus and near-cancellation.
- The start of the 7th Doctor.
- The start of the 8th Doctor in the TV movie.
- The start of the 9th Doctor in the revived series.
- The middle of the 10th Doctor, starting with "Voyage of the Damned", originally intended to be only a major change to the closing credits (because of the new BBC credits standards), but the opening ended up dramatically changed as well.
- The start of the 11th Doctor, along with the change to HD and a new production team.
- The start of the 12th Doctor.
Minor changes were made with:
- The second episode.
- The start of the 3rd Doctor.
- The middle of the 3rd Doctor, multiple times, as they re-edited the new arrangement to different lengths and played with adding and removing a stutter effect and a few other small changes, and even went back to the 2nd Doctor version sometimes.
- The start of the 10th Doctor (big changes to the closing theme, but not to the opening).
- The middle of the 11th Doctor (the lightning sound effects have been removed and restored).