I'm trying to parse the nickjr.com site, specifically "http://www.nickjr.com/paw-patrol/videos/" URL for full episodes only.
nickjr.com has their site laid out by show title, then by games or videos, at least with paw patrol, so the URL will remain the same until I actually click to play a video. There are multiple video's on the site, but the full episodes have the word "full-episode" in the URL.
Example that's not a full episode:
Example that's a full episode:
My idea is to use youtube-dl, which does work on their site (tested), to download episodes, but I don't want to download all their episodes because some are not full episodes. So what I'm doing now is going to each video and having to copy the URL, then copy that into youtube-dl. It works, but nickjrs site sucks trying to navigate it.
I know web spiders exist, but I wasn't really able to find exactly what I'm looking for and it didn't really make a whole lot of sense. But my plan would be to search the site, and use regex on the URL but only to write the URL's that match the full-episode string to a text file which I could then import into youtube-dl.
So, I really need help parsing the site for full episodes only, that can be saved into a text file.
Also, I would prefer the program to be compatible with Ubuntu 16.04, but I can also use Windows 10 if need be.