feat: yt-dlp cookie support + surface real failure reason; default workers 4
Bulk --repair on unauthenticated YouTube trips the bot-check (HTTP 429 "Sign in to confirm you're not a bot"), after which every call fails until the IP flag clears. Add cookie support so authenticated requests bypass it: - --cookies FILE / --cookies-from-browser BROWSER (and $YTDLP_COOKIES / $YTDLP_COOKIES_FROM_BROWSER for the API container), threaded into every yt-dlp invocation (search, probe, download, repair metadata fetch). - run_yt_dlp_get_metadata now logs yt-dlp's last stderr line (the actual 429 / bot-check / network reason) instead of a bare exit code. - Default --repair workers lowered 8 -> 4 (safe without cookies; raise with). - compose: optional YTDLP_COOKIES env + commented cookies mount. - README: how to obtain cookies (Chrome/Firefox, browser-read vs cookies.txt export); gitignore cookies.txt. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
73
README.md
73
README.md
@@ -95,7 +95,9 @@ export LIDARR_API_KEY="your-lidarr-api-key"
|
||||
| `-o`, `--root PATH` | Output root folder (default `/media/music`). |
|
||||
| `--search-all` | Search all albums when adding an artist to Lidarr. |
|
||||
| `--repair` | Re-tag existing downloads under `--root` from source metadata (see below). |
|
||||
| `--workers N` | Parallel metadata fetches during `--repair` (default 8). |
|
||||
| `--workers N` | Parallel metadata fetches during `--repair` (default 4). |
|
||||
| `--cookies FILE` | yt-dlp `cookies.txt` for authenticated YouTube (avoids bot-check / rate limits). |
|
||||
| `--cookies-from-browser BROWSER` | Load YouTube cookies from a local browser (e.g. `firefox`). |
|
||||
| `--retag-from-path` | Offline: re-tag artist/title from folder + filename (see below). |
|
||||
| `-x`, `--exclude NAME` | Folder under `--root` to skip during `--repair`/`--retag-from-path` (repeatable). |
|
||||
| `--debug` | Verbose output. |
|
||||
@@ -139,10 +141,71 @@ title. A bogus `NA [<id>].<ext>` filename is renamed to the recovered title, and
|
||||
`NA` album with no source album is normalised to `Unknown Album`.
|
||||
|
||||
Each file is its own yt-dlp network round-trip, so repair runs them in a thread pool;
|
||||
`--workers N` (default 8) caps concurrency — lower it if YouTube starts rate-limiting
|
||||
(HTTP 429/403), raise it to go faster on a large library. Progress prints every 100 files.
|
||||
Requires `mutagen` (a yt-dlp dependency, usually already present). CLI-only — not exposed via
|
||||
the REST API.
|
||||
`--workers N` (default 4) caps concurrency. Progress prints every 100 files. Requires
|
||||
`mutagen` (a yt-dlp dependency, usually already present). CLI-only — not exposed via the REST API.
|
||||
|
||||
**Cookies (important for bulk repair).** Unauthenticated YouTube requests get throttled fast —
|
||||
a large `--repair` (or even a `--dry-run`, which still fetches) will trip *"Sign in to confirm
|
||||
you're not a bot"* (HTTP 429) and every subsequent call fails until the IP-level flag clears.
|
||||
Pass authenticated cookies to avoid it:
|
||||
|
||||
```bash
|
||||
./musicfetch --repair --cookies /path/cookies.txt -o /media/music # exported cookies.txt
|
||||
./musicfetch --repair --cookies-from-browser firefox -o /media/music # or read from a browser
|
||||
```
|
||||
|
||||
With cookies you can raise `--workers`; without them keep it low (≤4) and expect occasional
|
||||
throttling. Cookies also apply to normal fetches/downloads. The same can be set for the API
|
||||
container via `$YTDLP_COOKIES` / `$YTDLP_COOKIES_FROM_BROWSER`. If you do get flagged, **stop** —
|
||||
retrying extends it; wait ~30-60 min (429) or longer for a bot-check.
|
||||
|
||||
#### Getting YouTube cookies
|
||||
|
||||
> ⚠️ Use a **throwaway / secondary Google account**, not your main one — bulk automated
|
||||
> requests can get the account flagged. You must be **logged in to YouTube** in the browser
|
||||
> first.
|
||||
|
||||
**Option A — read straight from the browser (simplest, host CLI only).**
|
||||
`--cookies-from-browser` reads the browser's own cookie store, so there's nothing to export:
|
||||
|
||||
```bash
|
||||
./musicfetch --repair --cookies-from-browser firefox -o /media/music
|
||||
./musicfetch --repair --cookies-from-browser chrome -o /media/music
|
||||
```
|
||||
|
||||
- **Firefox:** works while open; just be logged in to YouTube.
|
||||
- **Chrome / Chromium / Brave / Edge:** must be **fully quit** when you run this (Chrome locks
|
||||
its cookie DB, and newer versions encrypt it — close the browser entirely first). On Linux a
|
||||
running Chrome will usually fail with a "could not copy cookie database / locked" error.
|
||||
- Specify a profile if not the default, e.g. `--cookies-from-browser "chrome:Profile 1"`.
|
||||
|
||||
This only works where the browser lives (your host), **not** inside the Docker container.
|
||||
|
||||
**Option B — export a `cookies.txt` (works anywhere, incl. the container/server).**
|
||||
Use a Netscape-format cookie exporter, then point `--cookies` / `$YTDLP_COOKIES` at the file:
|
||||
|
||||
1. Install a cookies exporter extension:
|
||||
- Firefox: *"cookies.txt"* (a.k.a. *Export Cookies*).
|
||||
- Chrome: *"Get cookies.txt LOCALLY"* (pick a **LOCALLY**-running one — avoid extensions that
|
||||
upload your cookies anywhere).
|
||||
2. Log in to <https://www.youtube.com>, click the extension, **Export** → save `cookies.txt`.
|
||||
3. Use it:
|
||||
|
||||
```bash
|
||||
./musicfetch --repair --cookies ~/cookies.txt -o /media/music
|
||||
```
|
||||
|
||||
For the API container, mount it and set the env var (see `server/docker-compose.yml`):
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
YTDLP_COOKIES: "/cookies.txt"
|
||||
volumes:
|
||||
- /host/path/cookies.txt:/cookies.txt:ro
|
||||
```
|
||||
|
||||
Cookies expire — if YouTube starts rejecting them, re-export. Treat `cookies.txt` like a
|
||||
password (it *is* your logged-in session); keep it out of git (`.gitignore` it).
|
||||
|
||||
```bash
|
||||
# Preview what would change (writes nothing)
|
||||
|
||||
Reference in New Issue
Block a user