Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want to print the first 1000 characters in an UTF-8 encoded file. I know that the head tool can print the first n bytes of a file, but it may cut a character in the middle so that I get garbled output at the end.

I can write a awk program to do this, but may I know if there is any simpler way?

PS. I found it unreasonable that head and tail do not support character encoding (the LANG environment variable), while other tools such as cut, wc, sed and awk all support character encoding.

share|improve this question
    
One other thing to think about - even if you output whole code points, you may still end up separating base characters from their subsequent combining characters. If that’s a problem for your application you’ll need to do something a bit more sophisticated than the answer so far. –  Richard Kettlewell Jul 19 '12 at 7:55

1 Answer 1

up vote 0 down vote accepted

Not sure it is simpler, but this my way:

cat file | iconv -t UTF-32 | head -c $[1000 *4+4] | iconv -f UTF-32

This converts to a fixed-width form of Unicode so that the 1000 will always represent whole characters.

share|improve this answer
    
iconv (the one included in glibc) has a bug of buffering the whole input before making any output. This has a huge performance impact. Nevertheless, your approach works. –  Wu Yongzheng Jan 30 '13 at 4:44

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.