I am trying to crawl the internet starting at a certain website and get a list of URLs that were visited. This is what I have so far:
wget --spider -r --no-verbose -o out.txt <url>
The problem with this is that it creates a folder structure on the disk of the websites it visited. I don't want to download any files or create folders on disk. How can I achieve this?