0

I have created a file with a list of approximately 8 million import statements to be imported into an SQLite database. I have been redirecting errors to a file. Every few thousand lines there is an error, which let me know the progress of the import. As of yesterday the import was at line 5 million, but as far as I can tell nothing new has happened. [The import process - cat listofsqlstatements.sql| sqlite3 mydb.db 2>errors.txt - is still running.] The error log has not changed; the file size of the db seems unchanged (using du database.db); and an - sqlite3 mydb.db "select distinct * from sometable"|wc -l does not show any change. However, looking at htop I see the import process has high CPU use. What might be going on? How might I determine this? Can I check to see what the last data imported to the DB was?

1 Answer 1

2

Use strace -p to attach to the cat process. See if it's still reading from the file.

By the way, if you're not doing your import inside a transaction, it will run several hundred times slower than it should.

1
  • So true. I've revised the code that generates the import statements. Now, I'll just use the sqlite3 .import with a csv file delimited by |.
    – d-cubed
    Feb 3, 2013 at 23:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .