yt-dlp scripts

I use yt-dlp in order to be able to view videos from channels I follow offline, and wrote this set of scripts in order to help manage it.

Put a list of things you want to download in channels.txt and run this on a regular basis. Already downloaded videos are stored in archive.txt. You might want to change the sleep settings, the settings of 5 is reasonably conservative and I’ve not ran into any issues with no sleeping at all in the past. But if you run this daily or more often with a big channel list, you might get your IP flagged.

#!/usr/bin/env bash

export LC_ALL=en_GB.utf8

shuf channels.txt > shuffled-channels.txt

timeout --kill-after=60s --foreground 8h yt-dlp \
    --download-archive archive.txt \
    --force-download-archive \
    --batch-file shuffled-channels.txt \
    --embed-subs \
    --sub-langs all,-live_chat \
    --no-playlist \
    --sleep-requests 5 \
    --sleep-interval 5 \
    --max-sleep-interval 10 \
    --fragment-retries 30 \
    --sponsorblock-mark "all" \
    --sponsorblock-remove "sponsor" \
    --format "bestvideo[height<=1080]+bestaudio/best[height<=1080]/best" \
    --output 'output/%(uploader)s/%(title)s.%(ext)s' 2>&1 | tee --append log.txt 

rmdir --ignore-fail-on-non-empty output/*

Also, you might run into issues like

# yt-dlp 'https://www.youtube.com/watch?v=xpDx0vAsl_k'
[youtube] Extracting URL: https://www.youtube.com/watch?v=xpDx0vAsl_k
[youtube] xpDx0vAsl_k: Downloading webpage
[youtube] xpDx0vAsl_k: Downloading tv client config
[youtube] xpDx0vAsl_k: Downloading player 73381ccc-main
[youtube] xpDx0vAsl_k: Downloading tv player API JSON
[youtube] xpDx0vAsl_k: Downloading ios player API JSON
ERROR: [youtube] xpDx0vAsl_k: Join this channel to get access to members-only content like this video, and other exclusive perks.

In which case, you can use cat log.txt | rg "ERROR: \[youtube\] (.*):" -or 'youtube $1' to get a list of lines to append to archive.txt so you don’t try and redownload those videos every time.

This ignore.sh is also very handy for adding new channels to the download list without needing to download every one of their files.

#!/usr/bin/env bash

export LC_ALL=en_GB.utf8

yt-dlp \
    --flat-playlist \
    --dump-json "$@" \
    | jq -r '"\(.extractor) \(.id)"' >> archive.txt