In according to my tests on Debian Wheezy, all 3 scenarios may lead to both results (file1 gets sorted and written back to itself OR nothing gets sorted and nothing gets written to file1.
I believe that this is normal behavior and comes from the way Linux works with files. Think about the command - the sort command starts to read file1 and immediately sends it's output to tee. Tee reads the output, writes it back to file1 and prints it to /dev/null. In case sort is quick enough to read the whole file1, tee gets sorted output. But in case tee gets it's lock on the file, it erases it (tee always erases the output file, except when append option is used) and that's pretty much what's happening in all your 3 scenarios.
To make it shorter, lets say, that sometimes sort is not quick enough to read file1. In such case tee erases the file BEFORE sort can read it.
I would recommend the following procedure:
cat file1 | sort > /tmp/sorting.tmp; mv /tmp/sorting.tmp file1
In case you want to see the sorted output on stdout, do it like this:
cat file1 | sort | tee /tmp/sorting.tmp; mv /tmp/sorting.tmp file1
It's not a good idea to let 2 different commands working with 1 file on multiprocessor systems - you can never be sure which one gets executed first. On a single threaded system, the behavior would be different - sequential.
GNU sort
you can use-o
option:sort file1 -o file1
.