Questions tagged [wget]

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, Cron jobs, terminals without X Window System (X11) support, etc.

Filter by
Sorted by
Tagged with
0
votes
1answer
11 views

How do I mirror a private google site?

We have an internal wiki hosted on google sites. It's only accessible to persons in our organization (we have a G Suite Domain). I need to make a mirror of it (because we're moving some things to a ...
0
votes
0answers
32 views

Why does streaming an MP4 video file from a WordPress site work fine in the browser yet fail when I'm trying to save it to a file?

I am trying to learn more about streaming and why a video that plays fine in the browser is so tricky to download to a local hard drive for offline playing. It's hosted on WordPress and the full URL ...
0
votes
1answer
18 views

How can i download folder from google drive in linux

I want to download a directory that i puted in my drive to my ubuntu from terminal !! How can i download a directory from google drive to my ubuntu 20.04 with wget ?? or even another one! i searched ...
0
votes
2answers
30 views

wget cannot verify GitHub.com's certificate

I am using macOS 10.15.7. I have installed wget via homebrew. My goal is to install orpie via opam. opam uses wget. My installation of wget cannot find the certificates it needs. How can I resolve ...
0
votes
3answers
1k views

How to download/ save my own website FOR A NOOB

Basically I can no longer afford my site. I want to save it dearly but I cannot pay. I am a noob at this stuff but I want to download/ save the site so that when I can host I can just re-upload its ...
0
votes
0answers
3 views

How to print all .apk urls from open directory to a text file?

I have this open directory http://misfitmods. com/apks2/ How I can print/save all the files urls from above directory to a .txt file using Linux Terminal. I tried wget,but it only downloads a .html ...
1
vote
0answers
48 views

wget ignores files despite *? wildcard search

i run the following command: wget -np -c -r -v -l1 -nd -A "*,*multistream-index*,*pages-articles.xml.bz2,*pages-meta-current.xml.bz2,*abstract.xml.gz,*pages-logging.xml.gz,*stub-articles.xml.gz,*...
0
votes
0answers
36 views

How to download a website using WGET?

I'm not sure if anyone on this forum can help me. Is it a ridiculous idea for you all to show me the steps to download a website that requires a username and password, I currently have? https://app....
0
votes
0answers
10 views

Renaming files in wget2

I am batch downloading images from links saved in text files. How I can rename files(adding prefix or completely new name) on the go as file gets downloaded to maintain order as images don't have ...
1
vote
0answers
19 views

How can I batch-download expiring links?

I have a few hundred files to download. The URLs are 'expiring' and are of the format: https://website.com/directory/file1.iso?expires=xxxxxxx&token=yyyyyyy xxxxxxx is the Unix epoch time. Each ...
1
vote
2answers
50 views

for loop wget directory creation error

As a Ubuntu beginner. I am trying to download links from multiple text files with wget2 in batch. The -P "$f" creates directory name with .txt suffix. Is it possible to ignore .txt for -P &...
1
vote
0answers
53 views

WSL2 trying to launch VSCode with code . results in error “Please install missing certificates.”

I'm trying to launch VS Code from WSL2 using the command code ., the command spots that there is an update available for VS Code server and tries downloading it, however the download fails with a ...
0
votes
0answers
24 views

Batch Downloading files from urls saved in multiple text files

I have multiple text files with urls saved in it. t001.txt,ty002.txt,a003.txt....more that 100. How I can batch download files from these multiple text files. I asked this question earlier here too. ...
0
votes
1answer
27 views

How to go around the “failed: Connection refused” error?

I need to download all the files in some URL which points to a folder that has a couple thousand files of approximately 10kB each. I've tried using -w 1 to avoid overwhelming the server, but it still ...
2
votes
1answer
55 views

I have downloaded large gzipped xml files from Pubchem but my computer says they are corrupted

I have tried twice downloading the large gzipped xml files from PubChem FTP server, once through wget and one through chrome manually, but both times result in a corrupt file. For example, the first ...
1
vote
0answers
31 views

Browse https://dl.google.com using wget

Is there any way to browse the https://dl.google.com using wget? Ex - I have this link https://dl.google.com/android/repository/platform-tools_r30.0.5-linux.zip and I want to list all the contents in ...
0
votes
0answers
34 views

Linux: can't resolve any hostname

I am having a Debian server and it can't resolve any hostname, like with wget. Unfortunately I can't even install any packages, so I can't install the host command or similar. I am not an expert for ...
0
votes
0answers
25 views

Many bash and curl processes being run by www-data [duplicate]

I recently restarted my VPS on the Digital Ocean (DO) and today i was not able to get the data from my API, on DO it was showing very high usage on the graphs so i logged in via ssh, which was very ...
0
votes
1answer
204 views

download files with powershell, powershell version of wget

I am one of the few that decided to learn powershell instead of bash. I'd like to learn how to do the same thing as wget, for powershell. How to I do the following commands in powershell. wget -r --no-...
0
votes
1answer
32 views

How to archive URLs with special chars with wget on Windows?

I'm trying to archive a website but wget isn't linking some URLs to local files, the URLs that aren't linked to local files contain "´" and "²" to give some examples. The URL of ...
1
vote
1answer
60 views

Wget a script and run it with parameters

My question is similar to Wget a script and run it but i want to additionally pass some parameters to the script. Is it possible to do?
0
votes
0answers
30 views

Is there cli tool support multithreaded recursive download from http or ftp and resume the downloading?

I need to be able to download an entire directory and pause/resume the download. so far wget can be used to download http/ftp directory recursively, however it does not support mutlithreading/multi-...
0
votes
0answers
32 views

Wget or similar for WinCE 6.0 ARM

We have a bunch of handheld computers running WinCE 6.0 on ARM CPU's. Device = Datalogic Skorpio Input = Stylus + On-Screen keyboard + Numeric pad They need to be reset from time to time due to ...
0
votes
0answers
10 views

Web scraping with cookies

There is a website where I want to download all its articles. The site not necessary access to the URL but if I wanna show full articles, need credentials. So firstly, exported the site cookies to a ...
1
vote
1answer
781 views

wget crashes on MacOS Big Sur 11.1 (M1 Apple Silicon)

I get the same result from a homebrew installed version of wget and also a locally compiled one. when running wget it exits immediately with exit code 137 bash$ wget --help Killed: 9 zsh$ wget --help ...
0
votes
0answers
56 views

How to use wget to download Twitter archive

I already generated the Twitter archive download link on Twitter. But I could not download it via wget. It returns the 401 not authentication error. I have tried to add the username and the password ...
2
votes
1answer
53 views

Unzip and 7z with password, without temporary files

Every now and then, I receive a download link for a 6 GB .zip data archive that I'd like to: download unzip recompress with 7z, with a password I'd like to avoid using temporary files for each ...
0
votes
1answer
59 views

Using wget to download from site with DDos protection

When trying to load pages with wget I have the following error: $ wget --user-agent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" "https://www.cruisersforum.com/forums/&...
0
votes
0answers
144 views

Wget a file from 1fichier

I'm trying to download a file on 1fichier using wget on a Raspberry Pi 4 but, when I do wget <my url> it does not download the file, but a file without extension. I've tried using --referer and -...
1
vote
2answers
98 views

How to download a zip file using the command line

I would like to download a heavy compressed zip file from either dropbox or the UCF-Crime video dataset. The dataset's website points out these two links to get the dataset from : https://www.dropbox....
0
votes
1answer
53 views

wget post file. data body is undefined

I don't have curl installed and don't want it. The receiving servers data body is undefined when I try to post my file contents (my file is not blank, it has a json object in it) (Google is not ...
-1
votes
1answer
88 views

wget and ftp passive mode

When I do wget ftp://x.x.x.x:myport/a.jpg --date time-- ftp://x.x.x.x:myport/a.jpg => ‘a.jpg’ Connecting to x.x.x.x:myport... connected. Logging in as anonymous ... Logged in! ==> ...
0
votes
0answers
906 views

Unable to establish SSL connection

When I try to download the package from this link I get a error; In curl; curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number In wget; --2020-10-27 19:17:40-- https://www....
0
votes
0answers
106 views

wget - CentOS 8 Repo mirror

I am setting up an Ubuntu server to work as a repository for both apt and yum, and I'm running into some problems getting the CentOS 8 files to download for some reason.. I have a crontab with the ...
0
votes
1answer
301 views

How do you download an entire website for offline viewing with Wget?

I would like to download the entirety of a relatively small website (~50 pages) for offline viewing. I could manually open every page in a browser and download them via ctrl + s, and this would create ...
0
votes
1answer
429 views

How do I download only the file names of webpages using wget on Windows?

I am currently using this command: wget -m https://example.org --limit-rate=25m --convert-links --timestamps --wait=0.5 --execute="robots = off" And it works wonderfully. However, I would ...
2
votes
1answer
28 views

wget download only url with specific words in it

how to make wget download ONLY an url that contains the exact words? example https://ragnarok-europe.net/?module=item&action=view&id=500 I want wget to only download html pages like this. ...
-1
votes
2answers
119 views

How can I download a PDF file with WGET authentication and set the PDFextension on the dowloaded file?

I want to download a pdf file with wget The URL is like this: example.com/pdfdl/?pdfnumberid=245 This URL doesn’t have .pdf extension at the end of the parameter and for downloading it need just ...
0
votes
0answers
45 views

Replace server address by IP in wget

EDIT: It was suggested this is a duplicate of another question. That is not clear at all for me. First, the accepted answer suggests 3 possible reasons (which may actually be 2), and one of them was ...
0
votes
1answer
26 views

“wget” failing to retrieve input file because of name on my computer

I've been doing my very best to try a solution that works. I hope I can get my answer here. I'm trying to download a lot of images from my bookmarks folder, and I'm trying to extract them separately. ...
0
votes
1answer
42 views

Add domain url to wget result page

i'm trying to make a list of websites into static websites, and i'm using this wget command wget --quiet -E -H -k -nd -K -p -e robots=off #{url} the result is mostly what i want, but all the urls on ...
0
votes
1answer
641 views

TLS options for curl/wget

Is it possible to configure curl and/or wget to reject a DH key-exchange of less than or equal to 1024 bits. As a functional test, using curl/wget on https://dh1024.badssl.com/ should fail. I was ...
0
votes
0answers
234 views

503 Service Unavailable only when a particular URL is accessed from the terminal command line tools like cUrl or wget

I am behind a proxy in a corporate environment: $ printenv | grep proxy http_proxy=http://aproxy.corproot.net:8080/ https_proxy=http://aproxy.corproot.net:8080/ and trying to access the following 2 ...
0
votes
1answer
25 views

linux - How to download a journal pdf from terminal?

I want to download a pdf file which is hidden inside of website. In my home, I can not access the pdf file but if I connect to school network, I can access the the document. The thing I want to do is ...
0
votes
1answer
31 views

wget downloads malformed directories when using -r flag

I am trying to download a directory from github along with all the subdirectories and files. I don't want the whole repository. I use this command wget -np -nH --cut-dirs 4 -r -e robots=off https://...
0
votes
0answers
181 views

Downloading files through wget getting 404 error

I am need to copy entire file from S3 to HDFS. Sample file location: https://example.com/disabilities/state/city/MMYY/file.csv state and city having multiple values, I need to download all csv files ...
0
votes
1answer
23 views

How do I download and extract a tar in one command, while extracting only a single file to a specific location?

I'm trying to accomplish what it says in the title ... but for the life of me I can't figure out the magic invocation. The LLVM archive I am downloading contains the path clang+llvm-10.0.0-x86_64-...
0
votes
0answers
93 views

How to retry wget if specific error is found?

I have been trying to download a very large folder (>1Tib) with multiple files with wget, but it keeps giving me "segmentation fault" error. usr/local/bin/wget -r -c -N -l inf –no-iri --...
2
votes
1answer
31 views

Getting the same filename when renaming with wget

Using Windows 10 and wget to download files from a specific location, installed and configured WSL2. Once the setup was done, opened and ran the following command sudo apt-get install wget Then, ran ...
0
votes
0answers
88 views

wget: preserve filename with S3 pre-signed URL

How can I download a pre-signed S3 url using wget and preserve the filename? I tried --content-disposition but the filename still includes the query string. I think the server does not include that ...

1
2 3 4 5
17