Running script with Cygwin on Windows XP with Dual Core and 4GB Ram
cat url_list.txt | xargs -P50 wget -i
I am trying to trawl through 4GB of URL to download (approx 43 Million)
Works okay for about the first hour, then the Bash shell and downloads stop even though its only 2% through the URL list.
Any ideas at what could be wrong?
What is the best way to debug why this is stoping after an hour?