-1

Below is what I have so far, which opens chrome, goes to the URL, then opens a new .txt file. I need to select all the text on the page, copy it, paste it into the BigData.txt file, then save the file to the desktop. Is that possible?

CD C:\Program Files (x86)\Google\Chrome\Application\   
start chrome.exe http://en.wikipedia.org/wiki/Big_data
CD C:\Users\TomD\Desktop
echo >BigData.txt
end;

Thanks everyone

4
  • 2
    This looks dangerously like an XY problem. What are you really trying to do? As in, what do you expect to see in BigData.txt in this example? You want the page as it shows in the browser? The HTML? All the text (including the menus, etc) with no formatting? Just the body text with no formatting? Something else? More importantly - what's the end goal? You're hoping to scraping Wikipedia to do ____ ?
    – ernie
    Mar 19, 2013 at 20:31
  • Just copy the text in the easiest way possible - it doesn't have to be formatted. So I guess just the body text with no formatting. I just used that wikipedia page as an example. This project isn't as much about the end result as it is the process of getting there. Thanks for your reply! Mar 19, 2013 at 20:36
  • Is this a homework question? If the end result isn't important, this violates our FAQ, as all questions on SuperUser need to be about an actual problem you are having, not just something for casual interest. superuser.com/faq Mar 19, 2013 at 20:46
  • No, you cannot 'read' text out of IE from the command window. There are 3rd party utilities that can return the raw markup of the webpage, but it would still be up to you to sort through it. There aren't any built in functions in Windows to do what your wanting to do. Mar 19, 2013 at 22:13

2 Answers 2

4

You're really talking about web scraping, which will be difficult with batch files alone, especially if you're looking for just a specific part of a page.

If you wanted to get the whole web page, I'd suggest you get a windows version of wget or curl. Both of these could easily be integrated into a batch script and would download the HTML for a webpage. Note the real reason to use these instead of Chrome is that they are command line tools, as opposed to a GUI based browser like Chrome.

Once you have the HTML, you'll probably need to use a real scripting language to parse the HTML file, and extract just the main article text.

2

First, download "wget.exe" and place it on the same folder of your .bat (or in any folder in your path).

Then use:

set urlPath=www.wikipedia.com
set urlFile=index.html
set urlPathOuput="C:\users\%username%\Desktop"
set urlFileOutput=WiKiPeDiA.txt
wget %urlPath%/%urlFile%
type %urlFile% > %urlPathOuput%\%urlFileOutput%
del %urlFile%

Change:

www.wikipedia.com to the web page folder

index.html to the web page file

C:\users\%username%\Desktop to a folder on your pc

WiKiPeDiA.txt to the file to be generated in the folder previously defined

STEP BY STEP

First, the code saves some variables. Then, wget downloads the file of the web page to the %cd% folder(here, it would download "www.wikipedia.com/index.html").

Then, the contents of the .html downloaded will be saved in a plain text file(here, a new file will be created in "C:\users\%username%\Desktop" (the desktop for the current user) called "WiKiPeDiA.txt").

Finally, the web page file is deleted.

Note:

If you want to save the contents of the file with the tags, this would be the code.

If what you want is to save the text displayed by the browser, you woul have to analise and filter it before saving it to the file.

I have never done that, but this is what I would do:

1. set linenumber=0
2. set characternumber=0
3. find the first "<", after linenumber&characternumber and save it in linenumber2 and the characternumber2.
4. echo everything from linenumber&characternumber to linenumber2&characternumber2.
5. find the first ">" after the previous "<" which was in linenumber2, characternumber2. Save it in linenumber and characternumber.
6. goto step 3

Hope it helped!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .