AFAIK, although Windows will have appropriate drivers at the ready for external keyboards and mice, if they are not in use, the drivers won't be loaded.
In the greater scheme of things, the drivers required to operate these devices will be trivial both in terms of size and complexity, when compared with the 101 other drivers Windows 7 needs for all the other components. It's simply not an issue.
If you are noticing any difference when using external peripherals, you may have a separate problem, but that problem should be tackled explicitly.
Do you regard this as some sort of optimisation, or are you experiencing some sort of problem?