Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a C++ application test that creates 10,000 files in an NFS mounted directory, but my test recently failed once due to one file appearing twice with the same name in that directory with all the other 10,000 files. This can be seen on either Linux Centos v4 or v5 where the directory is NFS mounted, but not on the host machine where the disk resides.

How is it even possible to have two files with the same name in the same directory?

[centos4x32 destination] ls -al ./testfile03373
-rwx------  1 user root 3373 Sep  3 03:23 ./testfile03373*
[centos4x32 destination] ls -al ./testfile03373*
-rwx------  1 user root 3373 Sep  3 03:23 ./testfile03373*
-rwx------  1 user root 3373 Sep  3 03:23 ./testfile03373*
[centos4x32 destination] ls -al *testfile03373
-rwx------  1 user root 3373 Sep  3 03:23 testfile03373*
-rwx------  1 user root 3373 Sep  3 03:23 testfile03373*
[centos4x32 destination] ls -alb test*file03373
-rwx------  1 user root 3373 Sep  3 03:23 testfile03373*
-rwx------  1 user root 3373 Sep  3 03:23 testfile03373*

Running the Perl script suggested in one of the answers below:

ls -la *03373* | perl -e 'while(<>){chomp();while(/(.)/g){$c=$1;if($c=~/[!-~]/){print("$c");}else{printf("\\x%.2x",ord($c));}}print("\n");}'

gives:

-rwx------\x20\x201\x20user\x20root\x203373\x20Sep\x20\x203\x2003:23\x20testfile03373*
-rwx------\x20\x201\x20user\x20root\x203373\x20Sep\x20\x203\x2003:23\x20testfile03373*

Printing with the inode (-i) values shows the two copies have the same inode entry (36733444):

[h3-centos4x32 destination] ls -alib te*stfile03373
36733444 -rwx------  1 user root 3373 Sep  3 03:23 testfile03373*
36733444 -rwx------  1 user root 3373 Sep  3 03:23 testfile03373*

It would seem the directory entry is corrupted somehow.

Could my application have legitimately created this situation or is this a bug in the operating system? Is there anything I can do to protect against this in my program that creates the files?

I'm thinking there is some kind of bug in the NFS mounting software. Also 'umount' and then 'mount' of the NFS drive that has the issue does not resolve it, the repeated entry remains after remount.


Update 1: I've now hit this issue a second time, a few hours later, and the really strange thing is it happened on the exact same file, testfile03373, although it got a different inode this time, 213352984, for the doubled files. I'll also add that the file is being created on the Centos 5 machine where disk is being hosted, so it is being created locally, and showing correct locally, but all the other machines that NFS mount it are seeing the doubled entry.


Update 2: I mounted the drive on a Centos v6 machine and found the following in /var/log/messages after listing and seeing the double entry there:

[root@c6x64 double3373file]# ls -laiB testfile03373* ; tail -3 /var/log/messages
36733444 -rwx------. 1 user root 3373 Sep  3 03:23 testfile03373
36733444 -rwx------. 1 user root 3373 Sep  3 03:23 testfile03373
...
Sep  4 14:59:46 c6x64 kernel: NFS: directory user/double3373file contains a readdir loop.Please contact your server vendor.  The file: testfile03373 has duplicate cookie 7675190874049154909
Sep  4 14:59:46 c6x64 kernel: NFS: directory user/double3373file contains a readdir loop.Please contact your server vendor.  The file: testfile03373 has duplicate cookie 7675190874049154909

Additionally, I found that renaming the file causes the double entry to disappear, but renaming it back causes it to reappear doubled, or alternatively, just touching a new file with the name testfile03373, causes a double entry to appear, but this only happens in the two directories where this double entry has been seen.

share|improve this question
    
AFAIK, it is impossible for two files with the same name and extension coexisitng in the same directory in any filesystem. You could use some exception mechanism in your program to prevent failure, other that this... –  Doktoro Reichard Sep 3 '13 at 14:16
    
What filesystem are you using? –  Doktoro Reichard Sep 3 '13 at 14:25
    
Are they precisely the same? E.g. no leading or trailing whitespace? no UTF-16 chars, ... –  Hennes Sep 3 '13 at 14:25
    
What other tests can I perform to confirm they are exactly the same? –  WilliamKF Sep 3 '13 at 14:41
    
Sounds like you learned how to do an end run around a vital OS sanity check. –  Fiasco Labs Sep 3 '13 at 15:03

4 Answers 4

The disk is an NFS mounted disk. When I go to the host computer that publishes the drive, the file is only listed once.

Probably a bug, issue, or race condition with NFS.

It's possible to have two files of the same name if you directly edit the filesystem structures using a hex editor. However I'm not sure what would happen if you try to delete or open the files. I'm unsure of what tools exist on Linux to access a file by inode number (which can't be duplicated) but that may work.

Duplicate file names are something fsck would likely catch and try to fix.

Make sure none of the files have differing trailing spaces though.

share|improve this answer
    
I was going to suggest that the amount of writing on the filesystem ultimately broke something and allowed the existence of two identical files. –  Doktoro Reichard Sep 3 '13 at 17:55
    
Running fsck found no issues. Rebooted both host and client machines, issue still shows. –  WilliamKF Sep 3 '13 at 18:43
    
I should have been more clear - fsck is probably only going to work on the local file system, not an NFS mounted one. You probably need to upgrade/patch your nfs packages and possibly your kernel. As @somequixotic mentions, your CentOS is old and the issues you are having may have been resolved in future updates. –  ultrasawblade Sep 3 '13 at 18:46
up vote 5 down vote accepted

A friend helped me track this down and found this is a bug as recorded in Bugzilla 38572 for the Linux kernel here. The bug is supposedly fixed in version 3.0.0 of the kernel, but present at least in version 2.6.38.

The issue is that the server's ReadDIR() RPC call returns incorrect results. This occurs because of the following:

When the client reads a directory, it specifies a maximum buffer size and zeroes a cookie. If the directory is too large, the reply indicates that the reply is only partial and updates the cookie. Then the client can re-execute the RPC with the updated cookie to get the next chunk of data. (The data is sets of file handles and names. In the case of ReadDirPlus(), there is also stat/inode/vnode data.) The documentation does not indicate that this is a bug with ReadDirPlus(), but it probably is there as well.

The actual problem is that the last file in each chunk (name, handle tuple) is sometimes returned as the first file in the next chunk.

There is an bad interaction with the underlying filesystems. Ext4 exhibits this, XFS does not.

This is why the problem appears in some situations but not in others and rarely occurs on small directories. As seen in the question description, the files show the same inode number and the names are identical (not corrupted). Since the Linux kernel calls the vnode operations for underlying operations such as open(), etc., the file system's underlying routines decide what happens. In this case, the NFS3 client just translates the vnode operation into an RPC if the required information isn't in its attribute cache. This leads to confusion since the client believes the server can't do this.

share|improve this answer

There is a chance that you have a hidden non-printable character or whitespace in one of the filenames. You can check with by providing the -b option to ls, e.g.:

user@server:~/test$ ls -lab
total 8
drwxr-xr-x 2 user user 4096 Sep  3 12:20 .
drwx------ 8 user user 4096 Sep  3 12:20 ..
-rw-r--r-- 1 user user    0 Sep  3 12:19 hello
-rw-r--r-- 1 user user    0 Sep  3 12:19 hello\

Note the \ signifying the space at the end of that filename.

   -b, --escape
          print C-style escapes for nongraphic characters

As an alternative (though the above should work), you can pipe the output through this perl script to replace anything that isn't a printable ASCII character with its hex code. For example, a space becomes \x20.

while (<>) {
    chomp();
    while (/(.)/g) {
        $c = $1;
        if ($c=~/[!-~]/) {
            print("$c");
        } else {
            printf("\\x%.2x", ord($c));
        }
    }
    print("\n");
}

Usage:

ls -la | perl -e 'while(<>){chomp();while(/(.)/g){$c=$1;if($c=~/[!-~]/){print("$c");}else{printf("\\x%.2x",ord($c));}}print("\n");}'
share|improve this answer

No filesystem supports files with the same name in a directory. Linux are case sensitive so they must at least differ in case

share|improve this answer
    
The files have the same exact names, including case. –  WilliamKF Sep 3 '13 at 14:23
2  
Maybe there's some hidden character that you don't see –  Lưu Vĩnh Phúc Sep 3 '13 at 14:29
1  
How can I get the file system to show that if it were the case? –  WilliamKF Sep 3 '13 at 14:41
4  
Be careful with claims like "No filesystem supports files with the same name in a directory". While I know of no specific file system that allows it, the virtual file system of Plan 9 (and Inferno) does allow a directory to have multiple files each with the same name and different content. The easiest way to accomplish this is by union binding one directory over another. Access by name always accesses the first file, but directory enumeration (like done internally by ls) can allow access to one of the others with the duplucate name. –  Kevin Cathcart Sep 3 '13 at 19:32
    
@KevinCathcart If you're combining multiple directories then windows libraries allow multiple files with the same name in the same "folder" too –  Lưu Vĩnh Phúc Sep 3 '13 at 23:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.