1

I have shell file like this:

grep 'INSERT INTO container' $file | mysql database -f

Problem I have is MySQL gets too many inserts and it slow down a lot (database is mostly read-only and this is only insert).

I want to have something that can slow down the speed / control pipe "bandwidth", similar to this one:

grep 'INSERT INTO container' $file | gzip -9 | gunzip | mysql database -f

Here gzip -9 | gunzip will slow down the process and pressure to MySQL will be low.

pv command happen to have abilities like this, but it is interactive command. I am looking for something non-interactive.

3

2 Answers 2

3

You should use pv with the option -q which from the manual :

-q, --quiet
    No output.  Useful if the -L option is being used on its own to just limit the transfer rate of a pipe.

So using the command grep 'INSERT INTO container' $file | pv -L 100 -q | mysql database -f should do what you want. Maybe 100 is too slow, but you can put like 5k or more if you want.

1
1

You can use insert-delayed (deprecated in latest version) http://dev.mysql.com/doc/refman/5.7/en/insert-delayed.html

1
  • this is the other thing we are checking in parallel
    – Nick
    Oct 29, 2015 at 15:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .