Add configurable YouTube request throttling and rate-limit backoff

[samcro1967]

Select Themerr Product

Themerr-plex

Select Topic Area

Feature Request

Body

Would it be possible to consider adding some optional throttling/backoff controls for YouTube theme downloads?

As has been discussed previously, YouTube's anti-bot measures can sometimes result in yt-dlp receiving errors such as:

Sign in to confirm you're not a bot.

This seems particularly likely when processing a number of themes in succession. I understand that this is ultimately a YouTube/yt-dlp issue and not something Themerr can directly fix.

For users who prefer not to provide authenticated YouTube cookies, though, it might be helpful if Themerr had an option to process YouTube requests more conservatively.

With an earlier version of Themerr, I experimented with a simple external retry script that added delays between attempts. I used:

ITEM_DELAY = 5
BATCH_SIZE = 2
BATCH_DELAY = 30

So it would process two items, with a five-second delay between them, and then wait 30 seconds before continuing. This seemed to work well for me and allowed previously failed themes to download successfully.

That script depended on internals from an older version of Themerr, so I'm not suggesting it as a current solution or implementation. I'm happy to share it if it would be useful.

Would it make sense to add optional native controls for something along these lines?

• A configurable delay between YouTube requests
• An optional batch size and delay between batches
• A way to limit concurrent YouTube downloads
• Automatic cooldown/backoff when yt-dlp detects the YouTube bot/rate-limit response

The exact implementation or defaults aren't important to me. The main idea would be to give users the option to trade download speed for a more conservative request rate.

An automatic cooldown might also prevent continuing through the queue once YouTube has started rejecting requests, allowing processing to resume later instead.

This could provide another option for users who encounter YouTube's anti-bot measures but would prefer not to authenticate to YouTube with cookies.

One other related enhancement that might be useful would be an API endpoint to retry processing a single item (or selected failed items). That could allow individual failures to be retried without triggering another full library scan, and would also make it possible for external tools to implement their own conservative retry schedules if desired.

Thanks for considering it.

[ReenigneArcher]

For users who prefer not to provide authenticated YouTube cookies

You don't actually have to sign in. You can just do it via a private browser. https://docs.lizardbyte.dev/projects/themerr/latest/about/usage.html#youtube-cookies

I was able to process almost 1k items like this in a row, not signed in.

A way to limit concurrent YouTube downloads

You already can.