If I have a .txt file containing a list of URLs, how can I get Chrome or Firefox to open them, one URL per tab and then save the page in each tab separately?
Any platform will do: Windows, Mac, Linux.
If I have a .txt file containing a list of URLs, how can I get Chrome or Firefox to open them, one URL per tab and then save the page in each tab separately?
Any platform will do: Windows, Mac, Linux.
Opening the URLs in tabs can be done just pasting in a console prompt cmd
your URLs like this.
start chrome http://yoururls/foo.html &
start chrome http://yoururls/bar.html &
.
.
.
or
start firefox http://yoururls/foo.html &
start firefox http://yoururls/bar.html &
.
.
.
timeout 1 &
between each line. Not sure if there is a better way to do it.
May 6, 2023 at 7:57
Although the question specifically mentions Chrome and Firefox, if what you are trying to do is to save the content of each page, wget is made for this task. E.g.:
wget --input-file=list_of_urls.txt
You can even specify a specific user-agent in case you want the server to see the request as a certain browser:
wget --user-agent=some_specific_user_agent_string --input-file=list_of_urls.txt
Answering my own question, I was unsure of some plug-ins I had already seen but a closer look confirmed that they should do the job.
Two stages:
Run Firefox or Chrome from the command line, passing in the list of sites as parameter or a file containing a list, using one of the following methods:
Then use one of the following plug-in as appropriate to save all the contents of each tab
Firefox:
Quoted from the plug-in page (particular features of interest for this question):
About this Add-on: UnMHT provides following features:
- Save webpage as MHT file.
- Insert URL of the webpage and date you saved into saved MHT file.
- Save multiple tabs as MHT files at once.
- Save multiple tabs into a single MHT file.
- Save webpage by single click into prespecified directory with Quick Save feature.
- Convert HTML files and directory which contains files used by the HTML into MHT file.
- View the MHT file saved by UnMHT, IE, PowerPoint, etc.
Chrome:
From the extension page:
- save multiple opened pages into a zip file containing single HTML files with all resources included (images, stylesheets, frames...)
- open a zip file containing archives into tabs Notes:
- this extension needs "SingleFile Core" to be already installed (follow install instructions)
For macOS you can just open terminal and use open <URL>
to open all URLs separated by &
like:
open https://www.google.com &
open https://www.youtube.com &
open https://www.facebook.com &
open https://www.stackoverflow.com
But keep in mind that tabs order won't be according to the command flow (i.e it won't open google first and StackOverflow last it will open in different orders).
If you have a .txt file you can use this command to open all URLs using terminal
cd <filePath> && while read -r url; do open "$url"; done < <filename>.txt
Note: This command will use your default browser.
Open all at once one on each tab from the command line (CLI)
Chromium:
xargs chromium-browser --new-tab < urls.txt
Firefox:
xargs -L1 firefox -new-tab < urls.txt
or slightly more efficiently as per: https://unix.stackexchange.com/questions/365399/xargs-append-each-argument-with-a-parameter
xargs printf -- '-new-tab %s\n' < urls.txt | xargs firefox
Test file:
urls.txt
https://example.com/1
https://example.com/2
https://example.com/3
Firefox needs a bit more work because it requires one -new-tab
for every URL:
firefox -new-tab https://example.com/1 \
-new-tab https://example.com/2 \
-new-tab https://example.com/3
For the saving, you should just use wget
instead of a browser as mentioned at: https://superuser.com/a/965286/128124
Tested on Ubuntu 20.10, Firefox 84, Chromium 85.
Web-based URL openers
There are several JavaScript based services out there, e.g. https://url-opener.com/ which are a good newb-proof solution that don't require installing anything.
You just paste the list of URLs into them, click "Open All", and they open one URL per tab.
You have to enable popups/multi-tab opening for the domain however, as those are blocked by default on most browsers to prevent ad spam. Browsers usually show a "Popup blocked" warning, which you can then click to disable.
TODO can't find an open source one on GitHub pages after searching for five minutes...
Getting all of the tabs to open separate through batch isn't too hard.
Code:
@echo off
open chrome [enter URL 1]
open chrome [enter URL 2]
...
exit
As for the saving, not sure if it's possible through batch. I could be wrong though.
The easiest way that I found in Firefox, is to install and use the SingleFile Extension, which has the "Batch save URLS" option ( and even the "Save selected links", "Save all tabs" etc. options, if you need it )
Here is a screenshot of its "Batch save URLS" option :