5

I am experimenting a strange behaviour which I don't know how to solve. I will explain the scenario:

  • From a Python script I'm getting a json from a simple application hosted on parse.
  • Once I get the text, I get a sentence from it and save it to a local "txt" file saving it as iso-8859-15.
  • Finally I send it to a text to speech processor, which expects receiving it on ISO-8859-15

The weird thing is that once the python script runs, if I run

file my_file.txt

The output is:

my_file.txt: ASCII text, with no line terminators

But if I open my_file.txt with vim, then remove the last "dot" of the sentence, write it again, and save the file: if I do again:

file my_file.txt

now the output is:

my_file.txt: ASCII text

Which solves some problems when processing the voice synthesizer. So, how can I force this behaviour automatically without doing the vim stuff? I have also done many tries with iconv with no success.

Any help would be much appreciated

Edit:

i@raspberrypi ~/main $ hexdump -C my_file.txt

00000000  73 61 6d 70 6c 65 20 61  6e 73 77 65 72 2e 2e     |sample answer..|
0000000f

pi@raspberrypi ~/main $ file my_file.txt
my_file.txt: ASCII text, with no line terminators
pi@raspberrypi ~/main $ vim my_file.txt
pi@raspberrypi ~/main $ file my_file.txt
my_file.txt: ASCII text
pi@raspberrypi ~/main $ hexdump -C my_file.txt

00000000  73 61 6d 70 6c 65 20 61  6e 73 77 65 72 2e 2e 0a  |sample answer...|
00000010

Sample file

Python code:

import json,httplib
from random import randint
import codecs

connection = httplib.HTTPSConnection('api.parse.com', 443)
connection.connect()
connection.request('GET', '/1/classes/XXXX', '', {
       "X-Parse-Application-Id": "xxxx",
       "X-Parse-REST-API-Key": "xxxx"
     })
result = json.loads(connection.getresponse().read())

pos = randint(0,len(result['results'])-1)
sentence = result['results'][pos]['sentence'].encode('iso-8859-15')
response = result['results'][pos]['response'].encode('iso-8859-15')

text_file = codecs.open("sentence.txt", "w","ISO-8859-15")
text_file.write("%s" % sentence)
text_file.close()

text_file = open("response.txt","w")
text_file.write("%s" % response)
text_file.close()
  • Can you upload the file with no line terminators? I would like to have a look at it. – Nidhoegger Oct 17 '15 at 9:23
  • 1
    Is it removing the 'dot', or does any edit fix it? It might be that editing the file adds the end of line marker, rather than the dot causing the problem. – Paul Oct 17 '15 at 9:24
  • So it's a single line in that text file? And does it have a line terminator? And are you sure you're only removing the dot? You can validate using hexdump -C. When typing in vim, lines always seem to end with 0x0a, even though you cannot move the cursor to the next empty line. So I guess vim is indeed adding it when you remove the dot, or make any edit. – Arjan Oct 17 '15 at 9:24
  • many thanks! yes, you are all right, just opening and saving the file with vim is enough – cor Oct 17 '15 at 9:27
  • thank you @Arjan I edited the post with the command results – cor Oct 17 '15 at 9:40
6

The standard /bin/echo can be used to add that newline to the end of the file for you:

$ echo -n 'ssss'>test
$ file test
test: ASCII text, with no line terminators
$ hexdump -C test 
00000000  73 73 73 73                                       |ssss|
00000004
$ echo >> test
$ file test
test: ASCII text
$ hexdump -C test 
00000000  73 73 73 73 0a                                    |ssss.|
00000005
$ 

Another option would be to add it in your Python code:

text_file = open("response.txt","w")
text_file.write("%s" % response)
text_file.write("\n")  # <-- newline added here
text_file.close()
  • Or: text_file.write("%s\n" % response) ;-) – Arjan Oct 17 '15 at 11:01
  • 1
    @Arjan that's probably how I would do it because I like things to be ultra-concise, but I wanted the extra verbosity for illustrative purposes here. :) – Scott Johnson Oct 17 '15 at 11:03
3

The simplest solution is to append the newline in the write command:

text_file.write("%s\n" % sentence)

My sample program to demonstrate

import codecs
sentence = 'something'
text_file = codecs.open("sentence.txt", "w","ISO-8859-15")
text_file.write("%s" % sentence)
text_file.close()
text_file = codecs.open("sentence2.txt", "w","ISO-8859-15")
text_file.write("%s\n" % sentence)
text_file.close()

And the result:

$ file sentence.txt 
sentence.txt: ASCII text, with no line terminators
$ file sentence2.txt 
sentence2.txt: ASCII text

The explanation is that the variable you are writing does not contain the newline and write() writes exactly hat you give it.

  • Thank you, it works! your answer could be the acepted one perfectly, but Scott was quicker. – cor Oct 17 '15 at 11:13

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.