2
votes
1answer
40 views

How can I get a list of all messages in my gmail account in linux command line?

How can I get a list of all messages in my gmail account in linux command line? Ideally I would want to see something similar to: from:[email protected] to:[email protected] Date:11/22/33 subj:Foo1 ...
0
votes
3answers
51 views

Mirroring a wordpress website with wget

I'm trying to download a wordpress web-site, my blog actually, and to get the php files as well. So far I've tried - wget -rkp -l3 -np -nH --cut-dirs=1 http://www.sharons.org.uk/ wget -r ...
0
votes
2answers
37 views

wget error 400 CentOS

I have problem with calling wget command in CentOS 6.5. Im using following command: (I spaced the https, because of my user level on thi forum) wget --certificate=/root/o2key.pem ...
0
votes
0answers
48 views

Pinging local machine with external IP

On a Gentoo server, I am trying to wget or curl a page from its url: wget http://example.com/test.html (I can get result for this url from my browser) All I receive is Timeout. With cUrl I get ...
0
votes
0answers
57 views

wget input/output error (on linux CentOS / VMware)

I have booted a Linux OS (CentOS-6.5-x86_64-LiveCD.ISO), in VMWare Workstation (host windows Home). When i type this command i get the error as follows: $ wget input/output error I wonder if it is ...
0
votes
1answer
96 views

How to mirror ENTIRELY a website with wget [duplicate]

I really need the command to download everything from a website. It is arnos.gr and has a lot of content on it which I want to browse locally without Internet I have an account for login
0
votes
1answer
26 views

Use tmux for managing multiple downloaders as a supervisorctl service?

I have many data-servers I need to download data from via http as soon as it is available. For each server I start a bash "while true"-loop and within that a wget to poll the server for new data. To ...
0
votes
1answer
34 views

How can I manage a tmux session using supervisord?

I want do mirror the data of some 15 different servers that aperiodically but continuously provide files that I need. For this, I created a tmux conf that opens 15 tmux-windows with shells that run ...
2
votes
1answer
53 views

wget download from mirror download links

I'm trying to download using wget a tar.gz. The file is redirected based on the url parameters, so all downloads are redirected automatically. for example: wget ...
-1
votes
1answer
133 views

Iptables: allow wget

for some reason the following rules block wget. iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT iptables -A INPUT -p tcp --dport 80 -j ACCEPT iptables -A INPUT -p tcp ...
0
votes
1answer
89 views

Does wget have a download history?

I have used wget to download several compressed packages(in a failed attempt to manually emulate apt-get, if that is of any significance), and I don't remember the packages' locations. Is it possible ...
0
votes
0answers
65 views

Installing a .cer in CentOS

I have a .cer file that I have to install on a CentOS machine. How can I do that? What I am trying to do is make a request to https://myaddress.com but doing it through wget https://myaddress.com ...
0
votes
3answers
40 views

Linux debian command

i tried to install isp cp omega on my debian dedicated server os, i created a new dir with name /panel/, and i wrote a command cd /panel, then I run the code wget ...
0
votes
1answer
73 views

How to get Wget to retrieve entire url, including search terms

I'm trying to use a wget to retrieve a web page that that displays the results for a searched string. The page retrieved however, displays only the results of an empty string. From the looks of it, ...
1
vote
1answer
456 views

How would I `wget` files and then save them by date downloaded rather than filename?

My goal: To download 131 JPEGs and save them in a file name format that is relative to the date/time format rather than their file name. I have already tried things that involve changing the files' ...
1
vote
0answers
54 views

Ncat throws errors while being run from crontab

I have a simple script set up on my OpenWrt box to check if one of the computers in a local network is streaming, and if it does, the script starts wget | madplay to play the stream. Script works ...
0
votes
1answer
46 views

Multiple versions of wget on the server

I would like to have multiple versions of wget installed on my server. I am currently on Debian 6.0 and have wget 1.12. For testing purposes I need also wget 1.14 installed parallel to 1.12. This ...
1
vote
1answer
406 views

wget execute script after download

I need every time you complete a transfer program wget this run a script. this is my full command line: wget -i urls-to-downloads.txt My problem is that I want to go subtracting it downloads the ...
0
votes
1answer
342 views

How do I use wget to download a list of files from different URLs on the same site and store the files in the same folder structure as the URL?

I have a list of files I need to download at different URLs on the same site. Using the command: wget --http-user:[email protected] --http-password:password -i download.txt The files download to ...
3
votes
2answers
174 views

Dowloading all Urls accessible under a given domain with wget without saving the actual pages?

Hi trying to determine all the valid urls under a given domain without having the mirror the site locally. People generally want to download all the pages but I just want to get a list of the direct ...
1
vote
1answer
59 views

How to resume wget mirroring from a different location?

I am out of space in my home folder , where iam currently mirroring this site for the pupose of a cygwin installation in my windows machine using wget from my linux machine. i am aware that i can ...
0
votes
2answers
181 views

Script for CRON (FTP get files EVERY MINUTE)

I want to create a script that must run with a CRON every minute (Q: Is it wise to run CRON FTP/wget every minute?) compares local directory with remote directory (Q: do I use FTP or wget and how ...
0
votes
2answers
468 views

Diagnose internet connection issues using command line

I'm having multiple issues trying to get CentOS to download anything. I can't get yum to work because there is a time out when trying to access a file. Anyways, is there a way to trace why a command ...
0
votes
2answers
62 views

Automating OpenOffice Installation

I am attempting to write a shell script which will download the latest version of OpenOffice using wget, and then use the command-line to install the application. I have seen that installing the ...
3
votes
1answer
277 views

How can I use a command line tool like wget to log into an OpenID site?

Specifically, I would like to be able to download specific pages from my user profile on the various SE sites. I would, however, like to do this automatically (using a cron job), from the command line ...
0
votes
0answers
125 views

Galera install failure on Fedora 18

I've been trying to reinstall MariaDB and have been encountering multiple issues, $ yum install Mariadb-Galera-server Error: Package: MariaDB-Galera-server-5.5.29-1.i386 (mariadb) ...
7
votes
2answers
1k views

how can I use wget to download large files?

I'm using Linux Mint 15 Cinnamon running from an 8GB pendrive. I want to get the ISO for 'Linux Mint 14 “Nadia” KDE '. I tried using wget in the terminal. Here is exactly what I typed: wget ...
2
votes
1answer
272 views

What is more transparent, curl or wget in this web crawler project?

I have files on a Windows 7, xampp server (using altered port, 181). I am triggering the script from another computer (linux) using curl http://xxx.xxx.xxx:181/phpproject.php This PHP project is a ...
0
votes
2answers
403 views

Install yum in CentOS without make

I'm working with GoDaddy via SSH, and I noticed that the OS (CentOS 5.8) doesn't have yum. It does seem to have wget and rpm, but it doesn't have make, so I ran into a wall when trying to install yum ...
4
votes
3answers
230 views

Spawning multiple parallel wgets and storing results in a bash array to be pretty printed when all wgets are done

I have a long list of urls on my own website listed in a carriage return seperated text file. So for instance: http:/www.mysite.com/url1.html http:/www.mysite.com/url2.html ...
0
votes
1answer
347 views

Mirror website with wget but only matching url pattern

I want to mirror a website. It's built-on like this. abc.com/A/B/1/... abc.com/A/D/2/... abc.com/A/R/3/... abc.com/A/G/4/... abc.com/A/F/5/... abc.com/A/B/6/... And I only want to get all links ...
1
vote
2answers
87 views

Is it possible to download only a certain percentage of a file wit Kget (or wget)?

I need to download some large files (ISOs of some Linux distros) and I'm connected to the internet with a mobile broadband connection, which has a limited amount of daily total data transfer size. ...
2
votes
1answer
157 views

How do I disable my internet using curl or wget to hit my router in a script?

I want to be able to run some kind of shell script/curl commands to toggle my TV services by accessing a form on my router page using curl or wget. I have been able to get past authentication using ...
1
vote
1answer
523 views

wget download pdf files fail

I am using: wget -r -A pdf https://labraj.uni-mb.si but instead of pdf's I get this output in console: --2013-03-23 15:11:03-- https://labraj.uni-mb.si/ Resolving labraj.uni-mb.si ...
2
votes
1answer
139 views

wget downloads empty files even with proper referer and cookies

I'm trying to scrape Brenda Enzymes for a bunch of molecule files with a script. If I just download by file URL (it's a PHP redirect), I get a 0Kb file, so I think they're checking for cookies and/or ...
0
votes
0answers
103 views

wget truncated my files

wget -c -b --content-disposition 'http://example.com/somepath.mp4' I had some connectivity problems so I had to rerun this command for a few remote files. According to wget's man page, it will ...
0
votes
1answer
214 views

How to check for a value after curl or wget?

I need to curl/wget a page and make sure it returned a particular value (else return 1), something like this: curl http://example.com/something/run | grep -e '^success!$' || return 1 Which flags ...
5
votes
4answers
446 views

Trouble using wget or httrack to mirror archived website

I am trying to use wget to create a local mirror of a website. But I am finding that I am not getting all the linking pages. Here is the website ...
2
votes
3answers
55 views

mirror full website1 and all links to specific website2

I want to mirror a website that will be taken down soon, the problem im facing at the moment is simple -> mirror whole website1 and all links (will be files/images/similiar) to website2 too, so i got ...
0
votes
1answer
199 views

wget timestamp alternatives

I have a script that works in a loop and checks for mysqlbackup copies on another server. couple of times a day wget downloads the file even though no change has been made. what I've tried is using ...
3
votes
3answers
3k views

force wget timeout

How can i force wget to stop after X seconds? I have a script that downloads images, from time to time it gets stuck and refuses to "timeout" what i've tried: --tries=3 --connect-timeout=30 from ...
1
vote
2answers
48 views

wget : how to get a directory to a specify directory

The command I am using is: wget -r -nH ftp://some.domain.com/home/work/test/* -P ./my The output directory becomes: ./my/home/work/test/* How can I make it to just ./my/*?
2
votes
1answer
216 views

Download an entire sites jpegs with wget instead of just thumbnails

When I try to pull all the jpegs off of this website (http://www.zodiackillerfacts.com/gallery/) using wget: wget -A jpg,jpeg -r http://www.zodiackillerfacts.com/gallery/ it only downloads the ...
1
vote
3answers
498 views

Linux wget, how to display progress percentage after session has reload?

I am running debian Squeeze in console mode on a plug computer. I control it opening an SSH session from a Windows machine, on the same local network. I started downloading a large file using wget. ...
0
votes
1answer
563 views

How to download all file content from a folder using wget and http

I am trying to use wget and http to download all contents from folderAA below to directory /root/sstest wget -r --directory-prefix="/root/sstest" -o /root/sstest2.log ...
3
votes
2answers
268 views

What is the default user agent when using 'wget' on Linux?

What is the default user agent when using the 'wget' command on Linux. I'm not trying to fake out a website when it's being used, I simply want to know what appears in the Apache logs. Thanks!
2
votes
3answers
2k views

How to retry connections with wget?

I have a very unstable internet connection, and sometimes have to download files as large as 200 MB. The problem is that the speed frequently drops and sits at --, -K/s and the process remains ...
2
votes
4answers
1k views

How to know if file is already finished downloading in linux command line?

If you're downloading a large file in linux via command line, is there a command to know if it's already completed? Currently I am only checking if the file size is the same as the expected file size ...
0
votes
1answer
60 views

Transfer a file from Server A to B, using your PC as terminal then turning off your PC?

I need to copy server-a.com/file.zip to server-b.com. Let's assume the file is really big. Using my home PC, I SSH to server-b.com and type wget server-a.com/file.zip How do you do it so that the ...
4
votes
2answers
376 views

wget recursive limited within subdomain

I want to download the following subdomain with the recursive option using wget: www.example.com/A/B So if that URL has links to www.example.com/A/B/C and www.example.com/A/B/D, these two should ...