melonstube fix

docs/provider-catalog.md

@@ -29,7 +29,7 @@ This is the current implementation inventory as of this snapshot of the repo. Us
 | `hsex` | `chinese` | yes | no | Strong template for tags, uploaders, and direct HLS formats. |
 | `hypnotube` | `fetish-kink` | no | no | Fetish/tube hybrid. |
 | `javtiful` | `jav` | no | no | JAV channel family. |
-| `melonstube` | `mainstream-tube` | no | yes | Meta-search aggregator for melonstube.com — every card is an `/out/?l=<base64>&c=<hash>&v=3` redirect link whose destination (a third-party host) is embedded, zero-network, in a MessagePack blob under the `l` param (mixed binary/string fields: view/click counts, a JSON date-range blob, an array of related-video ids, and the destination URL itself as one msgpack string). Decoding is local: base64-decode `l`, then regex-extract the `https?://...` destination directly out of the raw (lossy-UTF8-converted) msgpack bytes — **the regex must be a positive allowlist of legal URI characters** (`[A-Za-z0-9\-._~:/?#\[\]@!$&'()*+,;=%]+`), not a denylist of a few excluded characters: the byte immediately following the msgpack string (a length-prefix marker for the next field, e.g. `\xcd`) lossy-converts to U+FFFD (code point 0xFFFD), which a denylist limited to ASCII control chars (`[^\x00-\x1f\\"']`) fails to exclude, silently appending a corrupt trailing character to the URL. Feeds: `/new` (latest), `/popular`, `/rating` (all three are static curated lists, ~120 items, true pagination via `?page=N`); search via `/search?q=`. Destination hosts fall into three buckets: (1) the common case — yt-dlp (`--impersonate chrome-120`) resolves the destination directly, so `video.url` is just the decoded destination URL, no proxy; (2) `JUNK_HOSTS` (currently `fhgte.com`) — observed dead-end paywall/signup funnels with no free playable video, so cards pointing there are dropped entirely rather than surfaced as false-positive results; (3) `HARD_HOSTS` (`manysex.com`, `videomanysex.com`) — yt-dlp cannot resolve these, so `video.url` is routed through `/proxy/melonstube/{host}/{path}`. The proxy (`src/proxies/melonstube.rs`) ports vjav.rs's Cyrillic-homoglyph-obfuscated base64 decode chain to pull the real `get_file` path/query out of `videofile.php`'s `video_url` field, then makes two manual (non-auto-redirect) hops to the final CDN URL: hop 1 needs the manysex.com/videomanysex.com page as `Referer`; hop 2 (to the signed `ahcdn.com` URL) must be sent with **no** Referer at all, because the CDN's signed URL embeds a literal `referer=none,.manysex.com,.gstatic.com` allow-list that rejects the videomanysex.com Referer used on hop 1 — auto-redirect clients that forward the same Referer to every hop get a 403 at hop 2. Before returning the 302, an anti-false-positive check confirms the resolved URL string contains the requested numeric video id and that a ranged GET (`Range: bytes=0-65535`) returns 200/206 with a `video/*`/`octet-stream` content-type, so a paywall/ad/error page can't masquerade as the real stream. Thumbnails (`ttcache.com`) load directly, no proxy. No `/api/uploaders` (aggregator has no stable uploader identity). |
+| `melonstube` | `mainstream-tube` | no | yes | Meta-search aggregator for melonstube.com — every card is an `/out/?l=<base64>&c=<hash>&v=3` redirect link whose destination (a third-party host) is embedded, zero-network, in a MessagePack blob under the `l` param (mixed binary/string fields: view/click counts, a JSON date-range blob, an array of related-video ids, and the destination URL itself as one msgpack string). Decoding is local: base64-decode `l`, then regex-extract the `https?://...` destination directly out of the raw (lossy-UTF8-converted) msgpack bytes — **the regex must be a positive allowlist of legal URI characters** (`[A-Za-z0-9\-._~:/?#\[\]@!$&'()*+,;=%]+`), not a denylist of a few excluded characters: the byte immediately following the msgpack string (a length-prefix marker for the next field, e.g. `\xcd`) lossy-converts to U+FFFD (code point 0xFFFD), which a denylist limited to ASCII control chars (`[^\x00-\x1f\\"']`) fails to exclude, silently appending a corrupt trailing character to the URL. Feeds: `/new` (latest), `/popular`, `/rating` (all three are static curated lists, ~120 items, true pagination via `?page=N`); search is NOT the vestigial `/search?q=` GET endpoint (returns generic filler content behind a fake static result counter regardless of query) — the real mechanism mirrors the site's own search form: POST `search_query[query]=<term>` to `/searching/by-form`, which 303-redirects (cookies + redirect auto-followed by the shared `Requester`) to a resolved, genuinely query-relevant destination depending on entity match — `/pornstar/<slug>`, `/category/<slug>`, `/search/a/<term>` (studio), or `/search/<term>` (generic free text) — all of which support `?page=N` pagination; a non-matching query resolves to a page with zero result cards, which `parse_listing` naturally returns as empty. Destination hosts fall into three buckets: (1) the common case — yt-dlp (`--impersonate chrome-120`) resolves the destination directly, so `video.url` is just the decoded destination URL, no proxy; (2) `JUNK_HOSTS` (currently `fhgte.com`) — observed dead-end paywall/signup funnels with no free playable video, so cards pointing there are dropped entirely rather than surfaced as false-positive results; (3) `HARD_HOSTS` (`manysex.com`, `videomanysex.com`) — yt-dlp cannot resolve these, so `video.url` is routed through `/proxy/melonstube/{host}/{path}`. The proxy (`src/proxies/melonstube.rs`) ports vjav.rs's Cyrillic-homoglyph-obfuscated base64 decode chain to pull the real `get_file` path/query out of `videofile.php`'s `video_url` field, then makes two manual (non-auto-redirect) hops to the final CDN URL: hop 1 needs the manysex.com/videomanysex.com page as `Referer`; hop 2 (to the signed `ahcdn.com` URL) must be sent with **no** Referer at all, because the CDN's signed URL embeds a literal `referer=none,.manysex.com,.gstatic.com` allow-list that rejects the videomanysex.com Referer used on hop 1 — auto-redirect clients that forward the same Referer to every hop get a 403 at hop 2. Before returning the 302, an anti-false-positive check confirms the resolved URL string contains the requested numeric video id and that a ranged GET (`Range: bytes=0-65535`) returns 200/206 with a `video/*`/`octet-stream` content-type, so a paywall/ad/error page can't masquerade as the real stream. Thumbnails (`ttcache.com`) load directly, no proxy. No `/api/uploaders` (aggregator has no stable uploader identity). |
 | `missav` | `jav` | no | no | HLS format pattern. |
 | `noodlemagazine` | `mainstream-tube` | no | yes | Best template for media and thumbnail proxying. |
 | `okporn` | `mainstream-tube` | no | no | Simple mainstream archive. |