2

So I have a LAMP webserver hosted on a Raspberry Pi 3. The Raspberry Pi has a RAID 10 connected to it through a hardware RAID controller. My goal is to be able to upload large files through my webpage using PHP via a LAMP stack setup on the Raspberry Pi.

I am able to successfully upload a file that is 0.97G but not one that is 1.32G so something must be limiting my upload file size to approximately 1G is my guess.

In php.ini I have the following set right now:

  • memory_limit: 1000M
  • post_max_size: 4G
  • upload_max_filesize: 4G
  • upload_tmp_dir: /mnt/raid10/Files/Temp (This is on the connected RAID so it isn't just dump the file to the SD card that the Raspberry Pi uses. Note that adding this did help up the max size from the before lower on.)

I can't figure out which other settings I should try adjusting and am open to any suggestions.

  • Which webserver are you using to host the site? That may also have a maximum file size limit. – Worthwelle Aug 23 '18 at 20:47
  • The Pi3 has 1GB of physical RAM onboard. My guess is that that is a limiting factor. – davidgo Aug 23 '18 at 20:55
  • Try restarting Apache with sudo service apache2 restart. It may not have picked up the changes from php.ini yet. This is the only reference to an Apache specific configuration I see, but is pretty outdated. – Worthwelle Aug 23 '18 at 20:55
  • 1
    Are you trying to upload locally, and how long is this taking? Another possibiity is you are running foul of the max_execution time. – davidgo Aug 23 '18 at 20:57
  • @davidgo superuser.com/questions/1351721/… Per the answer there, it seems that the memory limit should not affect the upload size as it is a direct stream. I thought the same thing which is why I posted this originally. Let me try increasing the max_execution time though – Eric F Aug 23 '18 at 20:58
2

Removing upload file size limitations

In order to configure a LAMP (Linux, Apache, MySQL, PHP) setup that allows a user to upload large files to their server the following has to be done:

Change the following variables in php.ini

  1. memory_limit = 1000M - Set memory limit to what is available on the system
  2. post_max_size = 4G - Ensure that this value is greater or equal to the upload_max_filesize value
  3. upload_max_filesize = 4G - Set this to whatever you want the maximum to be
  4. upload_tmp_dir: /mnt/raid10/Files/Temp - By default this is disabled which makes the temporary location be on the same disk as the OS is installed. In my case of running everything off of a Raspberry Pi, that makes it the SD card. To have less restriction, I set this to my network drive location instead.

After doing the above, the limitation in file upload size is the physical memory available to the device, whatever is set for memory_limit, and the file system limitations (4GB for a FAT file structure). In the case of my Raspberry Pi 3, that was 1GB.

Bypassing the memory limit to upload a file bigger than the amount of memory available

The only way to go past the memory limit is to chunk the upload. Once chunking is enabled, what happens during an upload is the following:

  1. Upload initiated on webpage.
  2. Data from source file is streamed to the memory of webserver.
  3. Data is dumped from memory of webserver to temporary file location.
  4. Once the temporary file reaches the chunk size that was set, then that data is dumped into the final destination location file and the temp file is deleted.
  5. Repeat step 2 until file is completed.

How to use chunking

If you are using a download handler similar to this one (https://github.com/blueimp/jQuery-File-Upload/blob/master/server/php/UploadHandler.php) then there are many built in options available.

In my case, where the #fileupload is declared, I added a line for maxChunkSize:

$('#fileupload').fileupload({
    maxChunkSize: 10 * 1024 * 1024, // 10 MB <-----This is the line I added
    add: function (e, data) {
       blah blah blah
    }
    });
}
});

Whatever value is placed here will be the chunk size used. The size that you chose takes a little experimentation as one that is tiny will slow down the upload speed quite a bit, and one too large will result in the user of the site seeing no progress on their progress bar for quite some time. For me 10MB was a happy medium.

*Do note that the progress bar for the upload will move only when each chunk starts, so it becomes more choppy than a nice continuous progress bar but I am actually ok with this.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.