1

I hit a behavior of wget from FTP server (URL: ftp://foo.com/bar/...) that was unfamiliar to me. Can you direct me to some documentation or explain how is the following possible?

If I wget the folder's URL I get an index.html (shown at the bottom of the post) with the following listing which shows that only the zipped file is available: os_image_123_0-incr-repo.iso.gz

I can wget this file's URL and I will get a gzip file with specific MD5 checksum.

c4fd702b3ad296561b2ec7e4b888e624  os_image_123_0-incr-repo.iso.gz
os_image_123_0-incr-repo.iso.gz:        gzip compressed data - deflate method , original file name

However I can also wget the same file URL omitting the .gz extension, leaving only the filename os_image_123_0-incr-repo.iso and I will now get an iso file, which is not listed in the index.html.

08daf88aca95325c3aa9ae6406e02198  os_image_123_0-incr-repo.iso
os_image_123_0-incr-repo.iso:   ISO 9660 filesystem image

It's as if wget "knows" to extract the iso from the iso.gz file.

Is this what happens? How does it work?

This is the content listed in the index.html for the folder:

2016 Apr 13 14:45  Directory   <a href="ftp://foo.com/bar/05/.RE/">.RE/</a> 
2016 Apr 13 14:36  File        <a href="ftp://foo.com/bar/05/README.11.3.7">README.11.3.7</a>  (3097 bytes)
2016 Apr 13 14:36  File        <a href="ftp://foo.com/bar/05/md5sum-repo-iso-123_0.txt">md5sum-repo-iso-123_0.txt</a>  (63 bytes)
2016 Apr 13 13:02  File        <a href="ftp://foo.com/bar/05/md5sums.txt">md5sums.txt</a>  (321 bytes)
2016 Apr 13 09:52  File        <a href="ftp://foo.com/bar/05/os_image_123_0-ai-sparc.iso">os_image_123_0-ai-sparc.iso</a>  (570142720 bytes)
2016 Apr 13 09:53  File        <a href="ftp://foo.com/bar/05/os_image_123_0-ai-x86.iso">os_image_123_0-ai-x86.iso</a>  (496521216 bytes)
2016 Apr 13 13:02  File        <a href="ftp://foo.com/bar/05/os_image_123_0-fallback_boot-sparc.pkg">os_image_123_0-fallback_boot-sparc.pkg</a>  (103859114 bytes)
2016 Apr 13 14:37  File        <a href="ftp://foo.com/bar/05/os_image_123_0-incr-repo.iso.gz">os_image_123_0-incr-repo.iso.gz</a>  (3613854717 bytes)
2016 Apr 13 09:52  File        <a href="ftp://foo.com/bar/05/os_image_123_0-text-sparc.iso">os_image_123_0-text-sparc.iso</a>  (885370880 bytes)
2016 Apr 13 09:53  File        <a href="ftp://foo.com/bar/05/os_image_123_0-text-x86.iso">os_image_123_0-text-x86.iso</a>  (795648000 bytes)
2016 Apr 13 14:38  Directory   <a href="ftp://foo.com/bar/05/zipped-repo/">zipped-repo/</a> 

Adding wget output.

[my_user 503 0 /tmp]$ wget ftp://foo.bar.com/products/Solaris_11/s11.3/support/sru07/05/sol-11_3_7_5_0-incr-repo.iso.gz
--2016-05-10 06:39:19--  ftp://foo.bar.com/products/Solaris_11/s11.3/support/sru07/05/sol-11_3_7_5_0-incr-repo.iso.gz
           => ‘sol-11_3_7_5_0-incr-repo.iso.gz’
           Resolving foo.bar.com (foo.bar.com)... 10.10.11.11
           Connecting to foo.bar.com (foo.bar.com)|10.10.11.11|:21... connected.
           Logging in as anonymous ... Logged in!
           ==> SYST ... done.    ==> PWD ... done.
           ==> TYPE I ... done.  ==> CWD (1) /products/Solaris_11/s11.3/support/sru07/05 ... done.
           ==> SIZE sol-11_3_7_5_0-incr-repo.iso.gz ... 3613854717
           ==> PASV ... done.    ==> RETR sol-11_3_7_5_0-incr-repo.iso.gz ... done.
           Length: 3613854717 (3.4G) (unauthoritative)

           sol-11_3_7_5_0-incr-repo.iso.gz        100%[===========================================================================>]   3.37G  99.5MB/

           2016-05-10 06:39:56 (94.4 MB/s) - ‘sol-11_3_7_5_0-incr-repo.iso.gz’ saved [3613854717]

[my_user 505 0 /tmp]$ wget ftp://foo.bar.com/products/Solaris_11/s11.3/support/sru07/05/sol-11_3_7_5_0-incr-repo.iso
--2016-05-10 06:43:25--  ftp://foo.bar.com/products/Solaris_11/s11.3/support/sru07/05/sol-11_3_7_5_0-incr-repo.iso
           => ‘sol-11_3_7_5_0-incr-repo.iso’
           Resolving foo.bar.com (foo.bar.com)... 10.10.11.11
           Connecting to foo.bar.com (foo.bar.com)|10.10.11.11|:21... connected.
           Logging in as anonymous ... Logged in!
           ==> SYST ... done.    ==> PWD ... done.
           ==> TYPE I ... done.  ==> CWD (1) /products/Solaris_11/s11.3/support/sru07/05 ... done.
           ==> SIZE sol-11_3_7_5_0-incr-repo.iso ... done.

           ==> PASV ... done.    ==> RETR sol-11_3_7_5_0-incr-repo.iso ... done.

           sol-11_3_7_5_0-incr-repo.iso               [                                                            <=>             ]   4.04G  32.3MB/

           2016-05-10 06:45:39 (30.9 MB/s) - ‘sol-11_3_7_5_0-incr-repo.iso’ saved [4337887232]
8
  • You wrote "shows that only the zipped file is available", yes the listing just below shows 08daf88aca95325c3aa9ae6406e02198 os_image_123_0-incr-repo.iso. Also I do not understand your two separate listings, both labeled index.html May 9, 2016 at 9:10
  • Also what's the URL? Is it ftp:// or http://? What the server is? May 9, 2016 at 9:15
  • Thanks @MartinPrikryl - I edited the post to make it more readable. It is a FTP:// server. It's an internal server at my workplace, so can't share it for anyone to try it themselves.
    – RaamEE
    May 10, 2016 at 5:47
  • But what FTP server software that is? May 10, 2016 at 5:50
  • I dont have this information.
    – RaamEE
    May 10, 2016 at 6:27

1 Answer 1

2

It's likely the server is configured to look for .gz file, when the requested file is not found and return an extracted contents.

With HTTP, it's probably easy to set this up with Apache mod_rewrite and mod_deflate modules. Not sure, if some FTP server supports this.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .