Questions tagged [wget]

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, Cron jobs, terminals without X Window System (X11) support, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
71 views

Inconsistent SSL connection error when using wget

I'm trying to download some .nc files from a secure online database. It seems like for roughly every other attempted file download, I get an 'Unable to establish SSL connection' message, and then wget ...
user avatar
0 votes
0 answers
30 views

YARN resource manager "HTTP request sent, awaiting response..."

we have Hadoop cluster with active/stand by resource manager services the active resource manager is on master1 machine and the stand by resource manager is on master2 machine in our cluster YARN ...
King David's user avatar
-1 votes
1 answer
42 views

how to get back the file that has been downloaded using Invoke-Webrequest without providing the output switch?

I downloaded a large file, using wget which is just alias for Invoke-WebRequest commandlet in powershell, and it asked me url i provided, it completed, and gave me the Headers of HTTP request instead ...
Yanjan. Kaf.'s user avatar
2 votes
1 answer
279 views

Stop wget from making index.html for Options +Indexes directories

I am using this command to mirror a directory tree on my server: wget -m -e robots=off --no-parent --reject-regex index.html --reject-regex aaa --no-verbose --timestamp https://example.com/folders/ &...
KevinHJ's user avatar
  • 187
1 vote
1 answer
69 views

Getting "Cannot write to" message after successful mirroring of remote directory with wget

I have been trying to mirror a directory on my server using wget, then afterward update the downloaded tree with only newer files. The local machine is running MacOS 12.6.3; the server is CloudLinux, ...
KevinHJ's user avatar
  • 187
0 votes
0 answers
55 views

WSL wget command returns HTML rather than downloadable csv file

I am running WSL (Windows Subsystem for Linux) and using Ubuntu 22.04 to test my homegrown bash scripts as a part of my CS course. The task is to download a CSV (comma separated values) file from the ...
Robert Thornton's user avatar
0 votes
0 answers
82 views

Why won't wget mirror the .zip files from this website?

This website has links to a bunch of sub-pages like /zx-origins/sub-page, and those pages in turn have a link to a .zip file on a sub-domain. My goal is to get those .zip files in one nice command, I ...
Stefan Midjich's user avatar
1 vote
0 answers
225 views

Can't get past the login page with wget

I know that there are many similar questions, but they are very dated, and I have already tried pretty much everything and it still didn't work. I'm currently trying to download a whole website using ...
kurapica's user avatar
0 votes
0 answers
375 views

curl not working on an RHEL8 Server

I have 3 running RHEL 8 VM's as Servers connected to my Company's network and for Setup purposes I need to download a script using curl. The VM's are properly connected to the internet for example I ...
Oberwalder Sven's user avatar
0 votes
0 answers
35 views

Wget starts fine but gets locked out on chromebook. Why?

Did I try to connect to something I shouldn't have, locking wget out until reboot? On a chromebook, enable Debian, and use apt to install wget. Then if we do $ wget http://www.example.com/ -O /tmp/x --...
Dan Jacobson's user avatar
2 votes
1 answer
10k views

curl: (35) schannel: next InitializeSecurityContext failed - The revocation function was unable to check revocation for the certificate

tl;dr How can I fix "The revocation function was unable to check revocation for the certificate" when using a custom proxy+cert combination? Can I disable this check somehow for the whole ...
ctote's user avatar
  • 379
0 votes
0 answers
294 views

wget recursive folder/file download only returns multiple index.html

wget --no-parent --recursive --level=4 https://www.example.com/jsons/ I'm using the above line to try & download all files in subfolders of that URL. The folder structure is as: https://www....
DaveF's user avatar
  • 37
1 vote
0 answers
155 views

Wget failing to detect SVG files when downloading a website

I used this command to recursively download the website, but the linked SVG files hosted on the asset's subdomain are not downloaded. wget --load-cookies cookies.txt --mirror --recursive --level=5 --...
ChickenHead's user avatar
0 votes
1 answer
605 views

Where do files downloaded using wget usually go to on Pop_OS?

I recently ran a download speedtest using: wget --output-document=/dev/null http://speedtest.wdc01.softlayer.com/downloads/test500.zip I got the output: ... HTTP request sent, awaiting response... ...
VBtheHun's user avatar
2 votes
0 answers
651 views

What is the difference between wget --mirror and --recursive?

My question pertains to wget, a command line tool used for downloading entire websites off the internet. I am having a confusion between the two options --mirror and --recursive. Are these two options ...
Galaxy's user avatar
  • 223
1 vote
0 answers
83 views

Automatic connect and disconnect to an audio stream (wget) (Ubuntu)

it's about an audio stream. I already have a code. After that comes my question. :) This is about a connect on an audio stream. I start the whole thing via ./.sh #!/bin/bash # Basic while loop pkill ...
Steven Burghardt's user avatar
2 votes
0 answers
499 views

wget and curl switch default to background task

When I use wget and curl from bash, it automatically switches to a background task by default, without me having provided the required arguments. This is making it hard to use these commands in ...
Nick Durcholz's user avatar
2 votes
1 answer
285 views

Is there a way to Grep multiple HTML pages on a domain without downloading the files?

I essentially want to Ctrl+F multiple sites in a browser. I have a range of URLs on a domain https://exampleblog.com/posts/{1-50} I want to grep all of the pages under /posts/ at once. They are each ...
dcannistraro's user avatar
0 votes
0 answers
144 views

wget -p does not download referenced files from external systems

Using Windows I would like to download an entire website using wget -p -e robots=off. The site references Google fonts from the Google server, so it includes a line <link rel='stylesheet' id='...
Thomas's user avatar
  • 281
1 vote
0 answers
80 views

RestAPI Query Value With Wget

I'm learning about the REST API fundamentals and randomly landed on webpage used for their API. https://isc.sans.edu/api/threatcategory/research/ I wanted to see if I could query individual items, ...
savdanorum's user avatar
0 votes
0 answers
550 views

Error when downloading an account restricted file from Google drive with wget. I get the error of "Result too large"

How do you download a file from Google drive with wget? The file I want to download is account restricted meaning only a person with a particular Google account can access the file. (i.e. the file has ...
barlop's user avatar
  • 23.8k
0 votes
2 answers
512 views

windows command line utility to download from both http:// and file:// with progress

is there a windows command line utility that supports downloading from both http:// and file:// with progress? already tried: powershell Invoke-WebRequest : not working on windows 7 powershell (New-...
geek175's user avatar
  • 131
0 votes
2 answers
1k views

wget how to get OpenSSL version?

How do i get which version of OpenSSL a wget was compiled with? when i do wget --version i just get $ wget --version GNU Wget 1.21.2 built on linux-gnu. -cares +digest -gpgme +https +ipv6 +iri +...
hanshenrik's user avatar
  • 1,665
3 votes
2 answers
12k views

How to install mysql 5.7 on ubuntu 22.04?

I have upgraded server from centos to ubuntu 22.04, but I want to keep mysql 5.7. I use ansible for provisioning By default ubuntu 22.04 apt gives mysql 8, so I think I need to add repo, gpg keyring ...
Lizard Derad's user avatar
0 votes
0 answers
458 views

How do you download something using wget and the download URL when the web page requires a checkbox ticking?

I want to download some data. On the web page where the download is, there is a box that needs checking to confirm that the terms have been agreed before allowing this download. When I try to download ...
MelParker101's user avatar
4 votes
1 answer
4k views

How to dowload files with original timestamp?

Many web sites ask us to login before downloading real contents files, and restrict us to download files from it with a cli program such as wget or curl. So we must download files with a browser, ...
Leon's user avatar
  • 239
0 votes
1 answer
2k views

-P option on wget does not work on windows 11

If I run: !wget -P ./arcface_model https://github.com/neuralchen/SimSwap/releases/download/1.0/arcface_checkpoint.tar from my jupyter notebook on windows11 It does not recognize !wget, even though ...
kevbuntu's user avatar
  • 119
1 vote
1 answer
631 views

Download from a redirect using wget or curl

I want to download something from https://www.articy.com/en/downloads/server/. The link I have to click to download the file is https://www.articy.com/download/articyserver3. I tried using curl -OJL ...
Deses's user avatar
  • 250
1 vote
0 answers
217 views

wget downloads different file than browser from same URL

When downloading the Hard Disk utility tool off Seagate's website I noticed something truly strange. The Seatools (Linux) link on that site links directly to a .run file which, on my system, firefox ...
JMC's user avatar
  • 158
0 votes
1 answer
33 views

Download the Tour of Heroes app and tutorial webpages having robots.txt

I like to download the Tour of Heroes app and tutorial webpages (https://v13.angular.io/tutorial) for offline viewing. I tried wget on Linux Mint 20.3 Cinnamon. I suspect that the robots.txt file ...
albertkao9's user avatar
0 votes
1 answer
810 views

How to download a link that contains UTF-8 coding with WGET

I use wget (Windows) to download webpages. On a site all works well but some webpages return a zero file. Here are commands i execute The first loads well with WGET, the second returns an empty file ...
clement9700's user avatar
1 vote
1 answer
688 views

ACL wget permission denied

ls -la: total 0 drwxrwxr-x+ 2 root root 4096 Jul 11 21:40 ./ drwxrwxr-x 2 root root 4096 Jul 11 21:40 ../ the permissions on . have a plus so ACL is active... getfacl .: # file: . # owner: root # ...
Simon's user avatar
  • 23
2 votes
1 answer
876 views

curl over HTTPS

I've just setup a website/webserver, and just got it verified by ZeroSSL, but when i try and run curl or wget over it, it errors. I could bypass this by running wget --no-check-certificate or curl -k ...
Jack Wu's user avatar
  • 23
-1 votes
1 answer
40 views

How to download part of JavaScript code

I am looking at this website with a public table with some information I am interested in. I can download the info saving the page through Firefox, but it will take too much time to save all tables (I ...
Emerson Soares's user avatar
3 votes
0 answers
109 views

Is there a way to make wget handle redirects more intelligently, for example don't follow cross-domain redirects if host spanning is off?

The way that wget handles redirects seems to be pretty bad I'm aware that following redirect can be disabled with --max-redirect=0 but that's not what I want I've identified a couple scenarios where ...
Displayname71's user avatar
0 votes
0 answers
184 views

wget does not download all images

Every day I have to save images from a web page which is updated daily. To do this I use wget command, but unfortunately on about 2000 images I download about 1000 and then it stops. The command I use ...
ebr15's user avatar
  • 1
1 vote
1 answer
982 views

I want to download a 1.2 GB file, but the transfer always fails at 1.08 GB. Is it possible to download the file in two pieces?

I can download 1.08 GB of the file without any difficulty, but the download invariably fails at that point. I am fairly sure this is because nginx writes temp files to disk before sending them to the ...
Custom Soundtrack's user avatar
-6 votes
1 answer
197 views

How to: Download Driver

I would like to download this package 3.16 - 5.14 drivers/staging/rtl8188eu/os_dep/usb_intf.c CONFIG_R8188EU CONFIG_STAGING 2357:010c * From here: https://linux-hardware.org/?id=usb:2357-...
mister mcdoogle's user avatar
1 vote
0 answers
267 views

Download "10yard.zip" with wget, not "10yard.zip/"

I'm trying to download all the zip file from a page in archive.org using wget: wget -r -np -nc -l 1 -A zip https://archive.org/download/mame-merged/mame-merged/ I have found the previous command in ...
VansFannel's user avatar
1 vote
1 answer
2k views

How to download all files in a folder with wget

I need to download all files in a domain folder says https://example.com/folder/subfolder. The subfolder files dont have unique increment, means the file names are random string. I want to download ...
J C's user avatar
  • 111
1 vote
0 answers
772 views

wget download only files from a specific folder

wget - I'm looking for a command to transfer files from http://1.2.3.4:1234/ - only files and not subfolders or their contents download... URL/image3.jpg URL/prog.exe URL/doc.txt URL/. no download... ...
celjan48's user avatar
0 votes
1 answer
42 views

Wget over https segfaults after system recovery

Running Raspbian, I had a microSD go bad. I used ddrescue to image it and put the image on a good microSD. The system boots, I can ssh in, plex seems to work, samba seems to work, most things seem to ...
Felix's user avatar
  • 613
0 votes
1 answer
3k views

How can I only show the request header/response headers and output to /dev/null when using wget?

How can I only show the request header/response headers and output to /dev/null when using wget? I use --debug it shows the request headers and response headers: wget --debug https://sb....
qg_java_17137's user avatar
1 vote
0 answers
74 views

use bash script to remove tracking from google scholar links into non tracking links for wget

I Get regular google scholar emails. I am trying to make a simple script that lets me download the pdfs / articles, without google tracking me via their urls. Up till this point I have been copy ...
crowdfundinxmr's user avatar
2 votes
1 answer
531 views

how to append output documents fetched (-O) by wget under windows

I am running wget (under windows) in a loop, every 10 seconds, to test the communication to my arduino server. Currently, the server response goes to a file which I rename per the current time hence, ...
g2c's user avatar
  • 21
1 vote
0 answers
556 views

Is it possible to download an MKV file starting from half-way over HTTP?

I have an MKV video file that I need to download starting form half-way through. The video encoded as h264, the audio is encoded as AAC. Normally HTTP downloads sequentially so if your download speed ...
Baa's user avatar
  • 411
0 votes
1 answer
384 views

Unable to wget/curl the zipped file on remote server

I've been trying solution similar to my case but none solved my problem. I have zip file on remote server, but when I download it using wget or curl, it keeps returning 404. To make sure that I have ...
afikri's user avatar
  • 113
3 votes
0 answers
474 views

Why is Wget option --reject not working with redirections?

I am trying to download recursively a website by filtering out some zip files with this (simplified) command : wget <other_args...> --reject 'zip' https://www.domain.tld This is not working! ...
fft's user avatar
  • 31
0 votes
0 answers
399 views

wget timeout on Let'Encrypt certificate server

I'am on a CentOS server trying to understand why I cannot connect to another website that is exposing a Let'sEncrypt R3 issued certificate. # cat /etc/centos-release CentOS Linux release 7.9.2009 (...
1Z10's user avatar
  • 101
0 votes
2 answers
10k views

How do I add a self signed root CA to Ubuntu 21.04

I have an EC2 instance that I am trying to get a local wget to work. I have my own CA set of keys,crt,pem I have been using it to sign my company's certificate cert requests, and has been working ...
mmaceachran's user avatar

1
2 3 4 5
19