21

wget --load-cookies will load cookies as a "textual file in the format originally used by Netscape's cookies.txt file". However, Firefox keeps its cookies in an SQLite database.

Is there a way to extract the "Netscape's cookies.txt file" from the Firefox cookies.sqlite file?

15

There are cookie exporter extensions that you can use to export a cookie.txt format file that can be used with wget.

Alternatively, you can create your own. Cookies are viewable in Options / Privacy / remove individual cookies. You can find the cookie you are after and create a .txt file containing the information:

domain - The domain that created AND that can read the variable. 
flag - A TRUE/FALSE value indicating if all machines within a given domain can access the variable.  Say "true" 
path - The path within the domain that the variable is valid for.  Use / for any url
secure - A TRUE/FALSE value indicating if a secure connection with the domain is needed to access the variable. Use false to allow http://
expiration - The UNIX time that the variable will expire on.  Set something far in the future
name - The name of the variable. 
value - The value of the variable.

So one might look like this for example:

.domain.com TRUE  / FALSE 4102358400 SESSIONID dfjdfkjsjwere090fusfdkljf
3
12

If you're using wget, you are probably comfortable from the command line. In that case, instead of a Firefox extension, you can use a simple shell script:

extract_cookies.sh > mycookies.txt
wget --load-cookies mycookies.txt examplehost.com

You can download the extract_cookies.sh script from https://gist.github.com/hackerb9/d382e09683a52dcac492ebcdaf1b79af or cut and paste the following:

#!/bin/sh -e
# extract_cookies.sh:
#
# Convert from Firefox's cookies.sqlite format to Netscape cookies,
# which can then be used by wget and curl. (Why don't wget and curl
# just use libsqlite if it's installed? Mysteries abound.)

# USAGE:
#
# $ extract_cookies.sh > /tmp/cookies.txt
# or
# $ extract_cookies.sh ~/.mozilla/firefox/*default*/cookies.sqlite > /tmp/cookies.txt

# USING WITH WGET:
# $ wget --load-cookies=/tmp/cookies.txt http://example.com

# USING WITH CURL:
# $ curl --cookie /tmp/cookies.txt http://example.com

# Note: If you do not specify an SQLite filename, this script will
# intelligently find it for you.
#
# A) Usually it will check all profiles under ~/.mozilla/firefox/ and
# use the cookies.sqlite that was updated most recently.
#
# B) If you've redirected stdin (with < or |) , then that will be used.


# HISTORY: I believe this is circa 2010 from:
# http://slacy.com/blog/2010/02/using-cookies-sqlite-in-wget-or-curl/
# However, that site is down now.

# Cleaned up by Hackerb9 (2017) to be more robust and require less typing.


cleanup() {
    rm -f $TMPFILE
    exit 0
}
trap cleanup  EXIT INT QUIT TERM


if [ "$#" -ge 1 ]; then
    SQLFILE="$1"
else
    if tty -s; then
    SQLFILE=$(ls -t ~/.mozilla/firefox/*/cookies.sqlite | head -1)
    else
    SQLFILE="-"     # Will use 'cat' below to read stdin
    fi
fi

if [ "$SQLFILE" != "-" -a ! -r "$SQLFILE" ]; then
    echo "Error. File $SQLFILE is not readable." >&2
    exit 1
fi

# We have to copy cookies.sqlite, because FireFox has a lock on it
TMPFILE=`mktemp /tmp/cookies.sqlite.XXXXXXXXXX`
cat "$SQLFILE" >> $TMPFILE


# This is the format of the sqlite database:
# CREATE TABLE moz_cookies (id INTEGER PRIMARY KEY, name TEXT, value TEXT, host TEXT, path TEXT,expiry INTEGER, lastAccessed INTEGER, isSecure INTEGER, isHttpOnly INTEGER);

echo "# Netscape HTTP Cookie File"
sqlite3 -separator $'\t' $TMPFILE <<- EOF
    .mode tabs
    .header off
    select host,
    case substr(host,1,1)='.' when 0 then 'FALSE' else 'TRUE' end,
    path,
    case isSecure when 0 then 'FALSE' else 'TRUE' end,
    expiry,
    name,
    value
    from moz_cookies;
EOF

cleanup
5
  • 1
    This doesn't work for cookies that are kept only during given browser session. (so probalby all session cookies) Sep 4 '17 at 13:14
  • I've wrapped this up in a command called curlfire. curlfire http://www.example.com/ and culfire -P newprofile http://www.example.com
    – Att Righ
    Nov 7 '17 at 22:13
  • 1
    This is great. Doesn't interfere with multiprocess or newer versions of FF, and can be scripted. Nov 8 '17 at 7:53
  • For attribution, that script comes from kb.mozillazine.org/Cookies.sqlite
    – Cliff
    Nov 25 at 22:43
  • @Cliff Thanks, I do like to give credit where possible. Unfortunately, that is not the source. That's a copy that I or someone else posted to MozillaZine in 2017. (Note that it has my name embedded in it).
    – hackerb9
    Nov 27 at 5:06
1

The way you find the sqlite file doesn't work on most systems.

Also what if you have multiple sqlite files because you have multiple Firefox profiles.

So here's how I do it:

Get all the cookies.sqlite files, sort them by line number and assume the one with the most lines is the one you are actually using the most. Then return the path for that file.

So I changed your line to this:

SQLFILE=$(find ~ -type f -name cookies.sqlite -exec wc -l {} \+ | sort -rn |grep -v total| head -1 |egrep -o "/.*")
2
  • Interesting. So what version of Firefox are you using that my script doesn't find all the profiles by default? Where are the cookies being stored? Surely, you don't have to search a user's entire home directory to find them.
    – hackerb9
    Sep 11 '17 at 9:07
  • I think it is a mistake to default to using the SQLite file which has the most newlines rather than the most recently used. I will often create throwaway Firefox profiles just to get some cookies from a site that is giving wget grief, so the relevant cookie jar will be tiny, but the most recently updated. By the way, why count the number of newlines in the database, which is binary, instead of using the filesize? You do not need to change my script much to do so; just swap ls -t with ls -S. (Or you can use my script as a filter by piping into it if you prefer find).
    – hackerb9
    Sep 11 '17 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.