0

I found today I can download multiple files in a range using curl and this is great.

$ curl -O "http://example.com/[001-100].png" (https://everything.curl.dev/cmdline/globbing)

Also I found I can use -f flag to prevent saving empty files in case of missing (let's say we check all the files in range 001-100, but there are only files from 001 to 050).

The problem I have is my range is really large. I want to save some videos from an online streaming (and they are splits by ~5 seconds, to be merged in a single video later by my script). The range is kind of 00001-10000, but some videos are really short (00001-00100).

I need a way to stop the try-to-download video files, when the returned file is empty instead of ignoring this case. Kind of loop/break.

Is it possible with curl? Do you have any suggestion? (I just want to save all the video fragments that exists in that range 00001-10000, but stop the try-to-download when nothing more is found).

Update:

The context: a long video is split in small .ts and its fragments named with consecutive integers. If 070 is a valid file and 071 is an empty one, then it means the video has only 70 parts I will merge then. These are my final steps to get the video as a single file after downloading the parts.

$ printf "file '%s'\n" ./*.ts > mylist.txt

$ ffmpeg -f concat -safe 0 -i mylist.txt -c copy final-video.ts

3
  • How do you intend to distinguish between multiple sequential "missing" files and no more files?. I don't think that is possible.
    – DavidPostill
    Oct 27, 2021 at 10:05
  • @DavidPostill In my case I just want to break when the first empty file appears. If there in not an ...00050 file, I am sure there will not be any ...00051 or 52, 53. So, I want to make sure the script stops with the first fail and doesn't consume unnecessary time searching for nothing.
    – GhitaB
    Oct 27, 2021 at 10:19
  • everything.curl.dev/usingcurl/returns
    – DavidPostill
    Oct 27, 2021 at 10:56

1 Answer 1

0

Solved using for/if/break.

#!/bin/bash
URL_prefix='http://example.com/filename_'
URL_var_range_start=1
URL_var_range_stop=10000
URL_var_range_zero_padding=5
URL_sufix='.ts'

echo "Creating videos folder..."
mkdir videos && cd "$_"

echo "Creating video folder..."
mkdir video && cd "$_"

echo "Downloading ts files..."
for i in $(seq -f "%0"$URL_var_range_zero_padding"g" $URL_var_range_start $URL_var_range_stop)
do
  DOWNLOAD_URL=$URL_prefix$i$URL_sufix
  SAVED_FILE="video_"$i".ts"
  CURL_COMMAND="curl -f -s "$DOWNLOAD_URL" -o "$SAVED_FILE
  if $CURL_COMMAND; then
    echo "Downloading "$SAVED_FILE"..."
  else
    echo "No more files to download."
    break
  fi;
done


echo "Creating playlist mylist.txt..."
printf "file '%s'\n" ./*.ts > mylist.txt

echo "Saving the video files into a single final-video.ts file..."
ffmpeg -f concat -safe 0 -i mylist.txt -c copy final-video.ts

echo "Cleaning unused files..."
rm video*.* mylist.txt

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .