The method for calculating download speed varies from software to software. The first thing to keep in mind is that speed is always calculated in the context of a certain time duration. Some software may display download speed for the entire time you've been downloading (could be hours), while other software may display download speed for only the most recent x number of seconds. The shorter the time duration for the speed calculation, the more real-time the value is.
Further, when a download first begins, there is a choice to be made whether the software assumes an initial speed of 0, or whether the software assumes (and displays) no initial speed whatsoever until it has more data to make an estimate with.
In the situation you've described, it is possible that the software is such as the former above, and so it takes a few moments for the calculated value to average up (from 0) to a more accurate value.