9

Around the web many people say you can truncate a file using > filename or truncate -s0 filename while file begin used

I know everytime a process write in a file, the process uses a offset for write in a file, doing a test with an script like this.

#!/usr/bin/env python

import os, time

with open("passwd","w") as f: #copy of passwd file in my current directory
        f.seek(0)
        for x in xrange(1,1000):
                f.write("hello world \n" + time.ctime() + "\n")
                f.flush()
                time.sleep(2)

every time my script make a write syscall the offset in /proc/pid_number/fdinfo/3 pos field is changed, but when i try to truncate the the file using the method listed above, in my file i see many characters like this ^@ when i open the file using vim or less -u and the file type is change from ASCII text to data and when i use ls -l filename the size isn't changed

So, when truncate the file the offset of the file are not report back, i'm testing this in Centos 7 and in Redhat 5, so i can tell changed the file size while the file is in use by processes doesn't free space and make dirty my file.

So my questions is, if my process has an opened file in pos 1000 and i did truncate -s0 filename, if the truncate works, what happening in the next process write?

strace truncate -s0 passwd
open("passwd", O_WRONLY|O_CREAT|O_NONBLOCK, 0666) = 3
ftruncate(3, 0)                         = 0
close(3)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?

 ls -l passwd 
 -rw-rw-r--. 1 user91 users 13832 Feb 23 17:16 passwd

As you can see my file wasn't truncated

This problem doesn't happen if i open the in append mode, for example with this code.

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/stat.h>


int main(){
        int range = 1000;
        int x; x = open("passwd", O_WRONLY|O_CREAT|O_APPEND);
        int i = 0;
        for( i = 0; i <= range; range++)
                write(x,"hello world\n",12);
                sleep(2);
}
2
  • 1
    This seems more like a general computing question rather then anything related to system administration.
    – Zoredache
    Feb 23, 2015 at 18:06
  • 3
    @Zoredache I disagree. I feel this is a question of file semantics in posix systems and is a platform question. I am answering. Feb 23, 2015 at 18:09

1 Answer 1

15

Note although the system call is called truncate, it actually is better interpreted as saying "Make my file report this many bytes in size". As per the system call manpage:

The truncate() and ftruncate() functions cause the regular file named by path or referenced by fd to be truncated to a size of precisely length bytes.

If the file previously was larger than this size, the extra data is lost. If the file previously was shorter, it is extended, and the extended part reads as null bytes ('\0').

So, one can truncate a file and have it be larger, rather than smaller.

So my questions is, if my process has an opened file in pos 1000 and i did truncate -s0 filename, if the truncate works, what happening in the next process write?

  • You have truncated. The file size at this stage is 0 bytes. The offset is 1000.
  • The write at position 1001 occurs.
  • The file size is 1002 bytes. Bytes 0-1000 contain '\0' (null). Bytes 1001+ contain data written.

When you write to a file from a position larger than the file itself, the data between the end of the file and the new write becomes null bytes and the file data between those two points is referred as being sparse.

Indeed, you can do the following and produce the same affect.

import os, sys

f = open('data.txt','w')
f.seek(1048576)
f.write('a')
f.flush()
f.close()

You also mentioned that opening in append mode avoids this behaviour. This is true because you are instructing the kernel in that case to "write to the actual end of the file every time". If you truncate then the end of the file does change. In append you cannot reposition your file pointer.

Here is a sample program which demonstrates what happens to the file, the offsets and the data in a file that has been truncated.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <err.h>

#define FPATH "/tmp/data.txt"
#define FSIZE 65536

int main() {
  int a,b;
  char buf[FSIZE];
  char byte;
  struct stat st;
  memset(buf, 'A', FSIZE);

  a = open(FPATH, O_WRONLY|O_CREAT);
  b = open(FPATH, O_RDONLY);

  if (a < 0 || b < 0)
    errx(EXIT_FAILURE, "Could not open file");

  printf("Writing %d * 'A' into file\n", FSIZE);
  /* Write some bytes */
  if(write(a, buf, FSIZE) != FSIZE)
    errx(EXIT_FAILURE, "Couldn't write complete size out");

  /* Seek to a  new position in the file */
  lseek(b, FSIZE/2, SEEK_SET);

  printf("Current position of handle 'a': %d\n", lseek(a, 0, SEEK_CUR));
  printf("Current position of handle 'b': %d\n", lseek(b, 0, SEEK_CUR));
  stat(FPATH, &st);
  printf("Reported size on filesystem of %s: %d\n", FPATH, st.st_size);

  /* OK -- now, read the byte at the position */
  if (read(b, &byte, 1) < 0)
    err(EXIT_FAILURE, "Could not read file");
  printf("Character at current position of handle 'b': '%c'\n", byte);

  /* Truncate the file in the 'a' handle */
  printf("Performing truncate...\n");
  if (ftruncate(a, 0) < 0)
    err(EXIT_FAILURE, "Cannot truncate file");

  printf("Current position of handle 'a': %d\n", lseek(a, 0, SEEK_CUR));
  printf("Current position of handle 'b': %d\n", lseek(b, 0, SEEK_CUR));
  stat(FPATH, &st);
  printf("Reported size on filesystem of %s: %d\n", FPATH, st.st_size);

  printf("Writing one byte via handle 'a'\n");
  if (write(a, buf, 1) < 0)
    err(EXIT_FAILURE, "Cannot perform second write");

  printf("Current position of handle 'a': %d\n", lseek(a, 0, SEEK_CUR));
  printf("Current position of handle 'b': %d\n", lseek(b, 0, SEEK_CUR));
  stat(FPATH, &st);
  printf("Reported size on filesystem of %s: %d\n", FPATH, st.st_size);

  if (read(b, &byte, 1) < 0)
    err(EXIT_FAILURE, "Could not read file");
  printf("Character at current position of handle 'b': '%c'\n", byte);


  close(a);
  close(b);
  exit(0);
}

This results in the following output;

Writing 65536 * 'A' into file
Current position of handle 'a': 65536
Current position of handle 'b': 32768
Reported size on filesystem of /tmp/data.txt: 65536
Character at current position of handle 'b': 'A'
Performing truncate...
Current position of handle 'a': 65536
Current position of handle 'b': 32769
Reported size on filesystem of /tmp/data.txt: 0
Writing one byte via handle 'a'
Current position of handle 'a': 65537
Current position of handle 'b': 32769
Reported size on filesystem of /tmp/data.txt: 65537
Character at current position of handle 'b': ''
1
  • yes, looking in the source kernel code i saw when the file is opened with O_APPEND, the pos = i_read_size(inode), anyway your answer solved my problem.
    – c4f4t0r
    Feb 23, 2015 at 21:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .