I want to print the first 1000 characters in an UTF-8 encoded file. I know that the head tool can print the first n bytes of a file, but it may cut a character in the middle so that I get garbled output at the end.

I can write a awk program to do this, but may I know if there is any simpler way?

PS. I found it unreasonable that head and tail do not support character encoding (the LANG environment variable), while other tools such as cut, wc, sed and awk all support character encoding.

  • One other thing to think about - even if you output whole code points, you may still end up separating base characters from their subsequent combining characters. If that’s a problem for your application you’ll need to do something a bit more sophisticated than the answer so far. – Richard Kettlewell Jul 19 '12 at 7:55

Not sure it is simpler, but this my way:

cat file | iconv -t UTF-32 | head -c $[1000 *4+4] | iconv -f UTF-32

This converts to a fixed-width form of Unicode so that the 1000 will always represent whole characters.

  • iconv (the one included in glibc) has a bug of buffering the whole input before making any output. This has a huge performance impact. Nevertheless, your approach works. – Wu Yongzheng Jan 30 '13 at 4:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.