There is some interesting answers on the msdn blog about this. As to why it is hard:
Estimating the time remaining to complete a copy is nearly impossible
to do with any precision because there are many unpredictable and
uncontrollable variables involved – for instance, how much network
bandwidth will be available for the length of the copy job? Will your
anti-virus software spin up and start scanning files? Will another
application need to access the hard drive? Will the user start another
And how they are improving,
Rather than invest a lot of time coming up with a low confidence
estimate that would be only slightly improved over the current one, we
focused on presenting the information we were confident about in a
useful and compelling way. This makes the most reliable information we
have available to you so you can make more informed decisions.
That said, if you really want to improve just the given estimate and keep the progress bar as it is, you could do something suggested on the slashdot comment:
Maintain a table of expected speeds for each storage device on the
filesystem. Record how long it takes to read the filesystem
information. When a device is mounted, if it's reasonable for the
device type, seek to the middle and end, measuring speeds there, too.
Get approximate curves for the read and write speeds across locations,
and use those for future estimates. For future read and write
operations, take note of where they are and how fast they go, and
adjust the curves accordingly.
When an operation starts, look at the curves for input and output for
the respective devices. Find the expected speed for the target
location. Whichever speed is lower should be used for the estimate.