Questions tagged [wget]

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, Cron jobs, terminals without X Window System (X11) support, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
22 views

I want to download a 1.2 GB file, but the transfer always fails at 1.08 GB. Is it possible to download the file in two pieces?

I can download 1.08 GB of the file without any difficulty, but the download invariably fails at that point. I am fairly sure this is because nginx writes temp files to disk before sending them to the ...
user avatar
0 votes
0 answers
14 views

Gunzip downloading files, piping

I want to unzip a series of files iteratively. That is, upon finishing the download process of a file with wget, I want it to, then, get unarchived with gunzip That is what I've already tried which ...
user avatar
0 votes
0 answers
33 views

Force browser to use IP address

Internet Archive is probably banned in my country. I don't know how they implemented the ban but I cannot access it mostly because my browsers are being unhelpful. Here is some transcript I made with ...
user avatar
-6 votes
1 answer
46 views

How to: Download Driver

I would like to download this package 3.16 - 5.14 drivers/staging/rtl8188eu/os_dep/usb_intf.c CONFIG_R8188EU CONFIG_STAGING 2357:010c * From here: https://linux-hardware.org/?id=usb:2357-...
user avatar
0 votes
0 answers
68 views

wget big data file (15+ GB) pipe to s3

I'm using wget to pipe straight from a www.backblaze.com url to S3 with the following command: > wget -qO- '<url>' | aws s3 cp - s3://<bucket>/{data_file} The is to a file, which can ...
user avatar
  • 1
1 vote
0 answers
78 views

Download "10yard.zip" with wget, not "10yard.zip/"

I'm trying to download all the zip file from a page in archive.org using wget: wget -r -np -nc -l 1 -A zip https://archive.org/download/mame-merged/mame-merged/ I have found the previous command in ...
user avatar
1 vote
1 answer
158 views

How to download all files in a folder with wget

I need to download all files in a domain folder says https://example.com/folder/subfolder. The subfolder files dont have unique increment, means the file names are random string. I want to download ...
user avatar
  • 11
1 vote
0 answers
46 views

wget download only files from a specific folder

wget - I'm looking for a command to transfer files from http://1.2.3.4:1234/ - only files and not subfolders or their contents download... URL/image3.jpg URL/prog.exe URL/doc.txt URL/. no download... ...
user avatar
0 votes
1 answer
28 views

Wget over https segfaults after system recovery

Running Raspbian, I had a microSD go bad. I used ddrescue to image it and put the image on a good microSD. The system boots, I can ssh in, plex seems to work, samba seems to work, most things seem to ...
user avatar
  • 613
0 votes
1 answer
148 views

How can I only show the request header/response headers and output to /dev/null when using wget?

How can I only show the request header/response headers and output to /dev/null when using wget? I use --debug it shows the request headers and response headers: wget --debug https://sb....
user avatar
0 votes
0 answers
26 views

Issue installing R-3.4.2 with Docker file

I am using the following Docker file: FROM python:3.7 RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7638D0442B90D010 CBF8D6FD518E17E1 RUN apt-get update && apt-get ...
user avatar
  • 101
1 vote
0 answers
22 views

use bash script to remove tracking from google scholar links into non tracking links for wget

I Get regular google scholar emails. I am trying to make a simple script that lets me download the pdfs / articles, without google tracking me via their urls. Up till this point I have been copy ...
user avatar
0 votes
0 answers
55 views

wget2 mirroring site "filename too long"

Have a site I try to mirror with wget2 (works on wget). wget2 --recursive --no-parent --convert-links --mirror --page-requisites --max-threads=50 https://example.com I get a filename too long-error: ...
user avatar
  • 101
0 votes
0 answers
73 views

Mirroring site with wget2 fails to convert links - wget works

I just test on a random small site I want to mirror. It's all ok if I do it with wget: wget --no-parent --convert-links --mirror --adjust-extension --page-requisites https://semesterbostad.nu If I try ...
user avatar
  • 101
1 vote
1 answer
21 views

how to append output documents fetched (-O) by wget under windows

I am running wget (under windows) in a loop, every 10 seconds, to test the communication to my arduino server. Currently, the server response goes to a file which I rename per the current time hence, ...
user avatar
  • 11
1 vote
0 answers
103 views

Is it possible to download an MKV file starting from half-way over HTTP?

I have an MKV video file that I need to download starting form half-way through. The video encoded as h264, the audio is encoded as AAC. Normally HTTP downloads sequentially so if your download speed ...
user avatar
  • 171
0 votes
1 answer
93 views

Unable to wget/curl the zipped file on remote server

I've been trying solution similar to my case but none solved my problem. I have zip file on remote server, but when I download it using wget or curl, it keeps returning 404. To make sure that I have ...
user avatar
  • 113
0 votes
0 answers
11 views

wget bind-address modem and VPN act different

I use Ubuntu 20.04 Desktop and trying to check if wget --bind-address works. This is ip a output: 3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen ...
user avatar
  • 189
2 votes
0 answers
64 views

Why is Wget option --reject not working with redirections?

I am trying to download recursively a website by filtering out some zip files with this (simplified) command : wget <other_args...> --reject 'zip' https://www.domain.tld This is not working! ...
user avatar
  • 21
0 votes
0 answers
86 views

wget timeout on Let'Encrypt certificate server

I'am on a CentOS server trying to understand why I cannot connect to another website that is exposing a Let'sEncrypt R3 issued certificate. # cat /etc/centos-release CentOS Linux release 7.9.2009 (...
user avatar
  • 101
0 votes
0 answers
408 views

How do I add a self signed root CA to Ubuntu 21.04

I have an EC2 instance that I am trying to get a local wget to work. I have my own CA set of keys,crt,pem I have been using it to sign my company's certificate cert requests, and has been working ...
user avatar
0 votes
1 answer
377 views

How do I install (anything) on debian: 'apt is unknown instruction' in Dockerfile

How do I install (anything) on debian: 'apt is unknown instruction' in Dockerfile If I search: [how to install wget on debian] I set articles that say: $sudo apt install wget So I try to do that in ...
user avatar
0 votes
1 answer
35 views

My windows 7 downloads files with appended lines

I have a windows 7 on my old laptop which don't support newer OS so i keep that as a test machine for various codes. Most of the times I use wget for transferring files from my new laptop to the old ...
user avatar
1 vote
1 answer
349 views

How do I stop a currently running cron job executing wget?

I added a cron job to prime the cache of my website. The command is: cd crawler && wget --recursive --no-parent --domains=example.com --level=inf --reject css,js,png,jpg,mp4,psd,pdf --delete-...
user avatar
0 votes
0 answers
70 views

Which directory to install Wget

I have installed Wget. But I'm just not sure where I need to put the wget.exe file. Because I already have searched where to put wget.exe. But didn't find where the best place is to put it. Do I need ...
user avatar
0 votes
0 answers
66 views

Download webpage and files in it and follow links to download files with in the links

I want to download a webpage and files in it and also follow the links in it and download the files in them, more specifically images and pdfs if possible. I think wget is the way to go but all the ...
user avatar
2 votes
1 answer
887 views

Download whole website wget

I'm trying to download the entire contents of the site http://julesverne.ca/ using wget. I'm running the following command: wget --wait 20 --limit-rate=20K --recursive --no-clobber --page-requisites --...
user avatar
0 votes
0 answers
17 views

Wget, curl and Apache Benchmark not working for one of my machines to connect internal services

This one is better showing the results straight away. Good machine: ping google.com ...
user avatar
0 votes
0 answers
79 views

Wget only downloads files from first directory and from all other susequent directories, it just downloads index.html files

I am trying to download whole website including everything, pictures, zips, pdfs, everything. Wget works fine for downloading files in immediate folder but for all sub-folders, it just downloads a ...
user avatar
  • 1
0 votes
0 answers
34 views

HTTP requests from Linux server CLI to a website with dynamic content that requires Webdriver (JS emulation)?

Are there any simple solutions to making GET or POST requests from a CLI on a linux server to dynamically generated websites that require JavaScript to run in order to return HTML? I know that ...
user avatar
  • 183
0 votes
1 answer
495 views

wget failed: Connection timed out

I have the following command to copy the website, as it tried to hit sun.com it got connection timed out. I would like the wget to exclude the sun.com so that wget would proceed to the next thing. ...
user avatar
0 votes
1 answer
918 views

How to download pdf only viewable in pdf.js?

I use Firefox Version 90 on macOS 11.5 and would like to download a pdf I can view via pdf.js. The link is https://www.laprensa.com.ni/wp-content/plugins/mix-eviewer/pdfjs/web/viewer.php?file=https://...
user avatar
  • 103
0 votes
0 answers
47 views

How to get the average download speed of a file in Windows 7?

I want to get the download speed of a online file. In Linux, I can use the following command: wget https://www.sample.com/myfile.exe which will report the average download speed after the download is ...
user avatar
  • 113
0 votes
0 answers
51 views

Replay an http request stored in file

When trying to work out an http request, I often end up capturing the request content, e.g. like this: POST /some/endpoint?options=1 HTTP/1.1 Host: www.example.com Connection: keep-alive accept: */* .....
user avatar
  • 502
-1 votes
1 answer
251 views

Multiple domains into wget --accept-regex?

I want to download, recursively, the page https://en.wikipedia.org/wiki/Main_Page, but only the files from upload.wikimedia.org and de.wikipedia.org It works by using these commands: wget -H -r -l1 --...
user avatar
  • 589
0 votes
0 answers
2k views

wget stuck on "HTTP request sent, awaiting response..." not timing out

I have wget set up to download pages from a site with the following parameters: wget -r --level=400 --retry-connrefused --waitretry=2 --read-timeout=15 --timeout=15 -t 0 [ site ] The timeout feature ...
user avatar
1 vote
1 answer
425 views

How to download a file using wget when url has parameters?

The URL is this: https://software-download.microsoft.com/db/Win10_21H1_English_x64.iso?t=52dbec9b-63f5-47e9-b4b2-ccbcbb6beb7a&e=1627124449&h=42e9a2a8e1f843e5a4573960dc07feaa The error is: wget ...
user avatar
  • 13
2 votes
1 answer
201 views

wget gets stuck on logging in as anonymus from ftp server

I´m trying to download a file from ftp://ftp.perforce.com with wget. I´m using the following command: wget -t 3 --timeout=10 ftp://ftp.perforce//path/to/file I get the following output: [xxxx@xxxxxx~]$...
user avatar
  • 21
1 vote
1 answer
375 views

wget fails to download some images in a webpage

So when I tried to download this webpage with wget, the text and styling works well, but some images are missing. Upon further research, the files fail to download because the url wget tries to ...
user avatar
0 votes
0 answers
26 views

wget reconnects every 2:30 exactly

I am trying to wget a 42Gb file. Every 2:30, pretty much to the second, the connection closes and wget then reconnects. Fortunately I was able to use -c to continue the transfer and --tries=100 to ...
user avatar
  • 111
0 votes
1 answer
1k views

Unable to establish SSL connection (Windows)

I am trying to run the following command in Command Prompt on Windows, using GNU Wget 1.11.4: wget -A pdf -m -p -E -k -K -nd https://www.ceratizit.com/ but I keep getting the following error: ...
user avatar
  • 117
0 votes
1 answer
79 views

Batch download webpages as displayed by browser

Wget doesn't work because the page is dynamic, and no matter what options I put it won't download some of the text shown in Firefox. I Googled quite a bit, but all the solutions I find are quite ...
user avatar
  • 174
1 vote
1 answer
1k views

How to download a single file from local BitBucket server using cURL or wget, not cloning the repo

I am trying to download a single file from BitBucket on the linux server. I do have access to the repositories when I try to access it from the web, so it is not permissions issue I used to be able ...
user avatar
  • 131
1 vote
0 answers
21 views

Incremental renaming files in wget2

How I can rename files in incremental img001.jpg, img002.jpg,img003.jpg in below. i tried -o, as i am downloading multiple files from text files, the files gets truncated. -o is building a one large ...
user avatar
  • 340
0 votes
1 answer
16 views

wget unable to download large directories completely

wget works fine for downloading smaller directories. But it is unable to download directories that have many sub-folders and files in them. It doesn't throw any errors. But I can see that it has ...
user avatar
0 votes
0 answers
21 views

wget not converting links that start with //

If I have a <script> tag in an html file that is like so: <script src="//cdn.website.net/js/file.min.js"></script> Convert links will change it to: <script src="...
user avatar
  • 111
0 votes
0 answers
362 views

wget ERROR: cannot verify localhost's certificate, ..Self-signed certificate encountered

How Can I implement a secure https connection on ngnix I want to implement https on my localhost.I am running http server nginx on ubuntu 20.04 What I did was i issued the command sudo openssl req -...
user avatar
0 votes
0 answers
57 views

ZScalar 'proceed' with download with wget

I am using PowerShell Invoke-WebRequest (alias wget) to download miniconda. When using the Windows 10 GUI, Zscaler will warn me, but I can still click on the "Continue" button to continue ...
user avatar
  • 451
0 votes
1 answer
38 views

How do I mirror a private google site?

We have an internal wiki hosted on google sites. It's only accessible to persons in our organization (we have a G Suite Domain). I need to make a mirror of it (because we're moving some things to a ...
user avatar
0 votes
0 answers
50 views

Why does streaming an MP4 video file from a WordPress site work fine in the browser yet fail when I'm trying to save it to a file?

I am trying to learn more about streaming and why a video that plays fine in the browser is so tricky to download to a local hard drive for offline playing. It's hosted on WordPress and the full URL ...
user avatar
  • 161

1
2 3 4 5
18