How to Extract Web Data for Shopify Bulk Import [closed]

Question

I want to import a huge database of products (5000+) to Shopify. My supplier only gave me the website, he refused to give me the database file.

How do I extract the web data to a CSV format so that I can do a bulk import for Shopify?

The CSV format of Shopify: http://www.okah.com/shopify-products.csv

The sample web data that I would like to extract to a CSV: http://www.okah.com

I can do a simple data import on Excel, but it still lacks of a few things:

1. the detailed description of each product (which is linked from the main table)

2. the image URL on the last column (excel will only capture the display name of the image link: Pic1, Pic 2, Pic 3 - instead of the image URL)

Can anyone please help?

Thanks.

Answer 1

This is not an answer but is too big for a comment.

Introduction to the requirement

In my experience most tools that import HTML or XML as data ignore the attributes but some of the data you want is held as attributes. For example:

  <a href="xrgEAsgZ8N.jpg"
     title="Orange Cotton&Lace Feathers Printed Fashion T-Shirt"
     rel="lightbox[gal12615]">
     Pic&nbsp;1
  </a>

I have added newlines to the above for easier reading but they are of no significance within HTML. I will assume your knowledge of HTML is nil and explain this.

<a xxxx>yyyy</a> defines an Anchor or hyperlink. xxxx are attributes and yyyy is the value displayed to the user.

In this example yyyy is Pic 1.   represents non-break-space which means Pic 1 must appear together on one line.

Each attribute is of the form vvv="www".

href="xrgEAsgZ8N.jpg" defines the destination URL. This URL is relative to the web page so the full URL is http://www.okah.com/xrgEAsgZ8N.jpg.

title="Orange Cotton&Lace Feathers Printed Fashion T-Shirt" is for browsers that cater for the visually impaired or cater for devices that cannot display pictures.

rel="lightbox[gal12615]" is a forward link. The definition from the HTML specification is: This attribute describes the relationship from the current document to the anchor specified by the href attribute. The value of this attribute is a space-separated list of link types. lightbox[gal12615] is not a standard link type nor can I discover a definition.

There are other attributes you might want includes the hyperlink from the item code and picture (for example, /12616.html) to a page that contains information like:

Product specification 
Item code     gA4-3841ge 
Gross weight  330 Gram 
Fabric        Cotton&Polyester 
Size          Free  

Obtaining the attribute values

With a fuller understanding the data you need from the HTML page, you may be able to Google for a tool that extracts attribute values.

The example HTML you provide is very simple. A bespoke program in VBA or other language to extract the data you seek would not be difficult to write. To be honest, I think this is the only option; I cannot imagine a general tool that could extract the example CSV file below from the example HTML.

What would you do with this information?

Suppose you could get something like the following:

Item code href,Item code value,Product name
/12616.html,gA4-3841ge,Green Cotton&Lace Feathers ... 
12615.html,gA4-3841or,Orange Cotton&Lace Feathers ...

How are you going to use this? There is little point in getting this data as a CSV file unless you have a plan for using it.