You have 3MB free, not 85MB. Since you have 255MB of swap used and only 88MB of memory that could potentially be used to hold data that had been swapped, the kernel has no choice but to swap at least 167MB. The reason it has swapped more than that is that the kernel judged that performance would be better if it used that memory for a disk cache (technically, for page caching) rather than using it to hold application data that hadn't been recently accessed.
The reason 3MB are kept free is to handle bursts of load in contexts where memory can't easily be made free or re-purposed. For example, if there's a burst of network traffic or when disk I/O completes.