add hotbunny provider + fix domain cookie scoping in requester

New channel: hotbunny (AI-generated hentai, hotbunny.ai JSON API).
Includes thumbnail proxy (/proxy/hotbunny-thumb/) since assets.hotbunny.ai
is CF bot-managed. check.py updated to treat CF-protected format URLs as
warnings rather than errors.

requester.rs: store_response_cookies now honours the Domain attribute in
Set-Cookie headers — cookies scoped to .domain.com are registered against
the parent domain so the wreq Jar returns them for sub.domain.com requests
automatically, without per-provider workarounds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Simon
2026-06-24 18:40:57 +00:00
parent 8373d49593
commit 6d5771a038
9 changed files with 464 additions and 10 deletions

View File

@@ -24,6 +24,7 @@ This is the current implementation inventory as of this snapshot of the repo. Us
| `hentaihaven` | `hentai-animation` | no | no | HTML scraper for hentaihaven.xxx (WordPress/Madara theme), Cloudflare-protected so the provider is gated behind `FLARE_URL` in `skip_reason_for_provider` (mod.rs); the shared requester clears CF directly (wreq Firefox136 emulation currently passes for the listing/search/watch/episode/`player.php` GETs) and falls back to Jina/FlareSolverr. Latest feed `/hentai/page/{N}/`, search `/?s={query}` (search is single-page — page>1 returns empty); listing/search cards link to series watch pages `https://hentaihaven.xxx/watch/{slug}/`. Per-series media resolution (the UUID exists nowhere in page HTML, so enrichment is unavoidable): watch page → episode links `…/watch/{slug}/episode-K` (in `manga-chapters-holder`) → episode page → `<iframe src="…/wp-content/plugins/player-logic/player.php?data=…">``player.php``<meta name="x-secure-token" content="sha512-…">` → decode token (strip `sha512-`, then 3× of rot13→base64-decode, then `JSON.parse`) → `{en, iv, uri, hot_domains, …}` → POST `…/wp-content/plugins/player-logic/api.php` with `action=zarat_get_data_player_ajax&a={en}&b={iv}` (urlencoded; this one POST uses a dedicated `wreq` Chrome137 client, not the shared requester) → `{"status":true,"data":{"sources":[{"src":"…m3u8"}],"isOctopus":bool}}`. A multi-episode series collapses into one `VideoItem` titled `"… (N Episodes)"` with one `m3u8` `VideoFormat` per episode (`format_note`/`format_id` = "Episode K"); each format carries `Referer`/`Origin: https://hentaihaven.xxx` + a Firefox `User-Agent`. `video.url` is the `watch/{slug}/` page (no yt-dlp extractor exists for the site, so `formats` are populated rather than relying on `video.url`). Two CDN shapes are returned: newer content-addressed `octopusmanifest.org/{uuid}/playlist.m3u8` (no token, portable across IPs) and older signed `master-lengs.org/api/v3/hh/{slug}/master.m3u8?hash=…` (~2.5h). **Gotcha:** both CDNs (same IP) aggressively per-IP rate-limit/ban with a TCP RST on 80/443 once tripped — looks like "host down" but is an IP ban; browsers play fine over HTTP/3 (QUIC) while TCP clients (curl/yt-dlp/wreq) get refused, so segment fetches can fail from a tripped IP even though the manifest URL is valid. Tags from the series "Genre(s)" block; `views` from the "Viewed … Total" counter; thumbnails (`img.hentaihaven.xxx`) load directly (no proxy/referer). Resolution is slow (each listing page = ~25 series × multi-episode player-API calls), so the provider is DB-first: it fetches the listing once for the ordered watch URLs, serves already-resolved `VideoItem`s from the `videos` SQLite table instantly (`db::upsert_video` to avoid duplicate-row staleness), and `spawn_refresh`es the whole page in the background (in-memory `VideoCache` soft-TTL 1h / hard-TTL 24h, per-listing in-flight guard). No `/api/uploaders` (no uploader identity), no proxy. |
| `hentaitv` | `hentai-animation` | no | yes | Next.js hentai site (hentai.tv) backed by a clean JSON API: `GET /api/browse?page=N&sort=<Label>&genres=<ExactName>` (`{videos:[28],total,pages}`, real pagination) and `GET /api/search?q=Q` (`{videos:[...]}`, single-page — `page` is ignored, so page>1 returns empty). Unlike `animeidhentai`, browse honors both `sort` (labels `Most Recent`/`Most Viewed`/`Trending`, mapped from option ids `new`/`views`/`trending`) and `genres` (the **exact case-sensitive** stored genre name, e.g. `Big Boobs`, `incest`), so genre archives go through `/api/browse?genres=` and paginate. The 68-genre catalogue (exact names) is background-loaded from the `/browse` page HTML (`"genres":[{"name","count"}]`, not exposed by the JSON API) and powers the `categories` filter plus keyword→genre routing. Each episode JSON has `slug`, `title`/`ep`, `tags[]`, `views`, `rating` (0-10 → ×10), `duration` ("MM:SS"), `brand` (studio → `uploader`), `thumb`/`backdrop`/`cover` (relative, served from `hentai.tv/uploads/...`, no referer), and `embedUrl=https://nhplayer.com/v/{embedId}/`. `video.url` is the reachable watch page `https://hentai.tv/hentai/{slug}`; `genre:`/`cat:`/`category:` prefixes and bare keywords that exactly match a genre route to the genre archive, everything else to search. Playback shares the **same nhplayer→`r2.1hanime.com` signed-CDN backend as `animeidhentai`**: `/proxy/hentaitv/{embedId}.mp4` is a redirect proxy that replicates nhplayer's PoW+DOM challenge (`player.php``player-core-v2.php``get-video-url-v2.php`, SHA-256-first-byte-zero PoW, ≥700ms dwell, fixed fingerprint) to mint a signed `?verify=<ts>-<sig>` URL — HEAD→200, GET→302 to the CDN URL (cached 150s). The CF wall is JA3-based not IP-based, so the signed URL is verifiable from anywhere with `yt-dlp --impersonate chrome` even though plain `curl`/`wreq` get 403. `src/proxies/hentaitv.rs` is a near-copy of `src/proxies/animeidhentai.rs` (only `SITE_REFERER` differs). No `/api/uploaders` (brand is studio-only). |
| `homoxxx` | `gay-male` | no | no | Gay category grouping example. |
| `hotbunny` | `ai` | no | yes (thumbs only) | JSON-API provider for hotbunny.ai (AI-generated hentai community). Feeds: `GET /api/post/feed?limit=N` (hot), `/api/post/feed/popular`, `/api/post/feed/recents`, `/api/post/category/{cuid}?limit=N`, `/api/post/search?query=Q&limit=N`, `/api/post/author/{name}/all?limit=N`. Pagination: fetch `page × per_page` items in one call and slice (cursor-based API, but limit works up to 200 total). Key fields: `id`, `imageUrl` (relative thumbnail path on CDN), `authorName`, `likes` (→ `views`), `sharedVideos[].videoUrl` (first video path, relative to CDN), `width`/`height`, `galleries[].label` (used as title prefix). 15 hardcoded categories from `/api/categories` (stable CUIDs). `video.url` is the post page `https://hotbunny.ai/post/{id}` (React SPA, not yt-dlp-resolvable); `formats[0].url` = `https://assets.hotbunny.ai/{sharedVideos[0].videoUrl}` with `Referer: https://hotbunny.ai/` header. `assets.hotbunny.ai` is Cloudflare bot-management protected (returns 404 to plain curl/yt-dlp; iOS AVFoundation passes with real Apple TLS — no proxy wrapping needed for video). Thumbnails proxied via `/proxy/hotbunny-thumb/...` to inject the required `Referer`. Supports `uploader:`/`author:`, `cat:`/`category:`/`#tag` query shortcuts, `categories` filter option, and `sort` option (feed/popular/recent). |
| `hqporner` | `studio-network` | no | yes | Uses thumb and redirect proxy helpers. |
| `hsex` | `chinese` | yes | no | Strong template for tags, uploaders, and direct HLS formats. |
| `hypnotube` | `fetish-kink` | no | no | Fetish/tube hybrid. |
@@ -117,6 +118,7 @@ These return binary media or images, sometimes rewriting manifests or forwarding
- `/proxy/fikfap-thumb/{endpoint}*`
- `/proxy/hqporner-thumb/{endpoint}*`
- `/proxy/porndish-thumb/{endpoint}*`
- `/proxy/hotbunny-thumb/{endpoint}*`
- `/proxy/pornhub-thumb/{endpoint}*`
## Best Copy Sources By Problem