simon/jacuzzi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Race the CDN and the proxy, for thumbnails and for playback

Browse Source 
A thumbnail used to try the provider and only ask /api/image once that
had failed, so every hotlink-blocked host cost a wasted request per card
before anything appeared. Both routes now go out together for the first
thumbnail of a host, and the rest of the batch waits on that one answer
rather than each rediscovering it. Speed decides which image is shown;
capability decides what the host is remembered as, since the proxy tends
to win first contact merely for being same-origin -- pinning a host to it
over that would push a whole page of thumbnails through our own server.

Playback asks the same question, but per video and at play time: one
provider can spread its media over several CDNs, so there is nothing
useful to pre-compute, and the old per-card probe answered for whichever
card happened to scroll past. The direct route is now tested alongside
the proxied playback and takes over if it answers before a frame is
decoded. Whatever loses is cancelled -- the token guards stopped stale
callbacks but left their requests running, so the losing route kept
pulling bytes and the server kept an upstream connection open for them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QBDkEXP4htyXTCZUwMLphd
This commit is contained in:
Simon
2026-09-08 12:48:18 +00:00
parent e2632c962d
commit 74b719b2ea
5 changed files with 444 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files

415
frontend/js/videos.js
View File

@@ -123,27 +123,290 @@ App.videos = App.videos || {};
}
};
App.videos.attachNoReferrerRetry = function(img) {
if (!img) return;
if (!img.dataset.originalSrc) {
img.dataset.originalSrc = img.currentSrc || img.src || '';
// ---------------------------------------------------------------------
// Thumbnails
//
// A thumbnail can come from two places: the provider's own CDN, or our
// /api/image proxy. Neither is reliably the faster one -- the CDN is a hop
// closer, but plenty of them hotlink-block or rate-limit, and learning that
// used to cost a whole failed request before the proxy was even asked. That
// serial retry is the wait worth removing.
//
// So the first thumbnail from a host is raced: both requests go out at once
// and whichever answers first is the one displayed. The winner is then
// remembered per host -- hotlink and CORS policy are origin-level, the same
// assumption the direct-playability probe below makes -- so every later
// thumbnail from that host goes straight down the route that already
// worked. One race per host, not one per image: racing every card would
// double the image traffic of a whole grid to learn something we already
// know by the second card.
// ---------------------------------------------------------------------
const IMAGE_DIRECT = 'direct';
const IMAGE_PROXY = 'proxy';
const imageRoutes = new Map(); // host -> winning route; absent = unknown
const imageRacing = new Set(); // hosts with a race already deciding
const imageWaiting = new Map(); // host -> images held until it decides
// A page of cards is built in one go, so every thumbnail from a host is
// attached before the first one has come back. Sending them all down the
// optimistic route is how the old serial retry hurt: on a host that blocks
// us, each card paid its own failed request before asking the proxy. So
// while a host is being decided the rest of its images wait for the answer
// -- at most as long as the fastest route takes -- and then load once, the
// right way round. If the race somehow stalls they go anyway.
const RACE_PATIENCE_MS = 2500;
// Direct is the route we'd rather settle on, so when the proxy comes home
// first the held images give direct this much longer to answer before
// committing to the proxy. Long enough that a provider merely a little
// slower than same-origin still wins its hosts; short enough that one which
// hangs -- the case this whole thing exists for -- doesn't hold up a grid.
const DIRECT_GRACE_MS = 200;
const imageHostOf = function(url) {
try {
return new URL(url, window.location.href).host;
} catch (err) {
return '';
}
img.dataset.noReferrerRetry = '0';
img.addEventListener('error', () => {
if (img.dataset.noReferrerRetry === '1') return;
img.dataset.noReferrerRetry = '1';
img.referrerPolicy = 'no-referrer';
img.removeAttribute('crossorigin');
const original = img.dataset.originalSrc || img.currentSrc || img.src || '';
const proxyUrl = App.videos.buildImageProxyUrl(original);
if (proxyUrl) {
img.src = proxyUrl;
} else if (original) {
img.src = original;
};
// Where a thumbnail from this host should be fetched from. Answers with the
// provider while the host is still unknown -- the optimistic route, and the
// one a race starts on anyway.
App.videos.thumbnailUrl = function(url) {
if (!url) return '';
return imageRoutes.get(imageHostOf(url)) === IMAGE_PROXY
? (App.videos.buildImageProxyUrl(url) || url)
: url;
};
// A src-less <img> counts as "unavailable", and the browser paints its alt
// text across the thumbnail box. Since a thumbnail now waits for its host's
// route before it gets a src, the caption is held back in a data attribute
// and put on only once there is an image to caption -- otherwise every card
// spells out its own title over the placeholder while the host is being
// decided, and permanently for an item that has no thumbnail at all.
const showThumbnail = function(img, url) {
if (img.dataset.alt !== undefined) {
img.alt = img.dataset.alt;
delete img.dataset.alt;
}
img.src = url;
};
// Last resort on a route that normally works: one expired or missing image
// shouldn't be left broken just because its host is fine in general.
const attachProxyFallback = function(img, proxyUrl) {
if (!proxyUrl) return;
img.addEventListener('error', () => { showThumbnail(img, proxyUrl); }, { once: true });
};
// Releases the images held for `host`. `route` is the winner, or null when
// the race told us nothing (both routes failed, or it stalled) -- in which
// case they take the optimistic route with the proxy behind it, exactly as
// an undecided host used to.
const releaseWaiting = function(host, route) {
const waiting = imageWaiting.get(host);
if (!waiting) return;
imageWaiting.delete(host);
waiting.forEach((entry) => {
if (route === IMAGE_PROXY) {
showThumbnail(entry.img, entry.proxyUrl);
return;
}
if (route !== IMAGE_DIRECT) attachProxyFallback(entry.img, entry.proxyUrl);
showThumbnail(entry.img, entry.directUrl);
});
};
// Two questions, and they don't have the same answer:
//
// which is quicker *now* -> what this image should display
// does direct work at all -> what the host is remembered as
//
// The proxy often wins the first question purely because it's same-origin:
// the browser already holds that connection, while the provider costs a
// fresh DNS lookup and TLS handshake. That says nothing about the provider,
// and pinning a host to the proxy over it would push every thumbnail on the
// page through our own server for no reason. So speed decides the pixels,
// and capability decides the memory: direct is remembered whenever it works
// at all, because it costs no server hop.
//
// Both routes are therefore fetched off-screen, and the visible image is
// pointed at the first one home. Racing on the visible element instead
// would abort the loser -- and the loser is the request that answers the
// second question.
const raceThumbnail = function(img, directUrl, proxyUrl, host) {
let shown = false;
let outstanding = 2;
let routeFinal = false; // the direct verdict is in; nothing can revise it
let abandoned = false; // took too long; a later race owns the host now
let directSettled = false;
let patience = null;
let grace = null;
const show = function(url) {
if (shown) return;
shown = true;
showThumbnail(img, url); // a cache hit; the probe has the bytes
};
// Records the host's route and lets go of everything held for it. The
// direct verdict is final; a route taken because direct was too slow to
// wait for is not, so a direct probe that comes home late still upgrades
// the host rather than leaving it on the proxy for the whole session.
//
// Crucially this happens the moment direct answers, not when both probes
// have finished: every thumbnail attached in the meantime is queued on
// exactly this answer, and making them wait on the *other* probe too
// leaves them blank for no reason.
const settleRoute = function(route, final) {
if (abandoned || routeFinal) return;
routeFinal = !!final;
if (final && patience) { clearTimeout(patience); patience = null; }
imageRoutes.set(host, route);
imageRacing.delete(host);
releaseWaiting(host, route);
};
// Nothing usable came home in time. Show whatever the host settled on --
// broken rather than blank, as it would have been without any of this --
// and keep the proxy behind an untested direct.
const giveUp = function() {
if (shown) return;
shown = true;
if (imageRoutes.get(host) === IMAGE_PROXY) {
showThumbnail(img, proxyUrl);
return;
}
attachProxyFallback(img, proxyUrl);
showThumbnail(img, directUrl);
};
const decide = function() {
if (--outstanding > 0) return;
if (patience) { clearTimeout(patience); patience = null; }
if (grace) { clearTimeout(grace); grace = null; }
giveUp(); // no-op if either route came home
};
// Off-screen, and low priority: neither may take bandwidth from anything
// the reader is already looking at.
const newProbe = function() {
const image = new Image();
image.decoding = 'async';
image.fetchPriority = 'low';
return image;
};
const directProbe = newProbe();
directProbe.referrerPolicy = 'no-referrer';
// The direct verdict alone decides the route -- either way round. One
// probe discovers the host's answer and every held image acts on it,
// instead of each rediscovering it at the cost of its own request.
directProbe.onload = function() {
directSettled = true;
if (grace) { clearTimeout(grace); grace = null; }
show(directUrl);
settleRoute(IMAGE_DIRECT, true);
decide();
};
directProbe.onerror = function() {
directSettled = true;
if (grace) { clearTimeout(grace); grace = null; }
// Direct is out for this host, so the proxy is the answer even if it
// hasn't reported yet -- there is nothing else left to be.
settleRoute(IMAGE_PROXY, true);
decide();
};
const proxyProbe = newProbe();
proxyProbe.onload = function() {
show(proxyUrl); // first one home gets the pixels on screen
// Don't strand anything behind a provider that may never answer:
// give it the grace window, then take the route that works. Marked
// provisional, so a slow-but-working provider still wins its host
// when it finally reports.
if (!directSettled && !grace) {
grace = setTimeout(function() {
grace = null;
settleRoute(IMAGE_PROXY, false);
}, DIRECT_GRACE_MS);
}
decide();
};
proxyProbe.onerror = function() { decide(); };
imageRacing.add(host);
// A probe that never answers -- a hung connection rather than a refused
// one -- must not leave the host mid-race forever, with every later
// thumbnail queueing behind a decision that will never come. Once that
// happens this race stops touching the shared maps entirely: the next
// thumbnail starts a fresh one, and a late answer here must not reach in
// and overwrite what *that* race decides.
patience = setTimeout(function() {
patience = null;
if (grace) { clearTimeout(grace); grace = null; }
if (!routeFinal) {
abandoned = true;
imageRacing.delete(host);
releaseWaiting(host, imageRoutes.get(host) || null);
}
giveUp();
}, RACE_PATIENCE_MS);
directProbe.src = directUrl;
proxyProbe.src = proxyUrl;
};
// Points `img` at `url` by whichever route is known to work for its host,
// racing the two the first time that host is seen.
App.videos.attachThumbnail = function(img, url) {
const directUrl = url || (img && img.dataset.thumb) || '';
if (!img) return;
// Held back until there is an image to caption -- see showThumbnail. An
// item with no thumbnail keeps an empty alt: the card's own title sits
// directly beneath the box, so there is nothing for it to add.
if (img.alt && img.dataset.alt === undefined) {
img.dataset.alt = img.alt;
img.alt = '';
}
if (!directUrl) return;
// A cross-origin Referer is what most hotlink protection keys on, and an
// image needs none. Sending none is what lets the direct route work at
// all on a fair number of providers -- and the direct route is the one
// that costs us no server hop.
img.referrerPolicy = 'no-referrer';
const proxyUrl = App.videos.buildImageProxyUrl(directUrl);
const host = imageHostOf(directUrl);
const route = imageRoutes.get(host);
if (route === IMAGE_PROXY) {
showThumbnail(img, proxyUrl || directUrl);
return;
}
if (imageRacing.has(host)) {
// A race is already deciding for this host. Wait for it rather than
// guessing: guessing wrong costs this image a whole failed request
// before it even asks the route that was about to be proven.
const waiting = imageWaiting.get(host) || [];
waiting.push({ img: img, directUrl: directUrl, proxyUrl: proxyUrl });
imageWaiting.set(host, waiting);
return;
}
if (route === IMAGE_DIRECT || !host || !proxyUrl) {
// Known good, or nothing to race against: take the provider and keep
// the proxy as this image's own fallback.
attachProxyFallback(img, proxyUrl);
showThumbnail(img, directUrl);
return;
}
raceThumbnail(img, directUrl, proxyUrl, host);
};
// Each channel in a group sends back a different number of videos per
// page, so a small per-channel count keeps any one channel from
// dominating a single interleaved batch.
@@ -279,7 +542,11 @@ App.videos = App.videos || {};
};
const warmThumbnails = function(items) {
const urls = (items || []).map((v) => v && v.thumb).filter(Boolean);
// Warm down the route the host has already settled on -- warming a URL
// the cards won't ask for would leave them waiting anyway.
const urls = (items || [])
.map((v) => v && App.videos.thumbnailUrl(v.thumb))
.filter(Boolean);
if (!urls.length) return Promise.resolve();
const loads = urls.map((url) => new Promise((resolve) => {
// Off-screen and not needed yet: low priority, started when the
@@ -289,6 +556,12 @@ App.videos = App.videos || {};
const img = new Image();
img.decoding = 'async';
img.fetchPriority = 'low';
// Same terms the card and the probe fetch on. Warming with a
// Referer the real request won't send would warm the wrong
// thing: on a hotlink-protecting host it earns a 403, which is
// both a wasted warm and a cached refusal the probe may then be
// handed -- pinning a host to the proxy that works direct.
img.referrerPolicy = 'no-referrer';
img.onload = resolve;
img.onerror = resolve; // the card's own retry handles failures
img.src = url;
@@ -440,7 +713,7 @@ App.videos = App.videos || {};
// Builds a fully-wired video card element for `v`. Kept separate from
// mounting so the virtualizer can create a card the moment it needs to be
// on screen and throw it away once it scrolls out of the window.
App.videos.buildCard = function(v) {
App.videos.buildCard = function(v, options) {
const favoritesSet = App.favorites.getSet();
const card = document.createElement('div');
card.className = 'video-card';
@@ -462,7 +735,7 @@ App.videos = App.videos || {};
<button class="video-menu-item" type="button" data-action="download" role="menuitem">Download</button>
</div>
<div class="video-thumb">
<img src="${v.thumb}" alt="${v.title}" loading="lazy" decoding="async">
<img alt="${v.title}" loading="lazy" decoding="async">
<div class="video-loading" aria-hidden="true">
<div class="video-loading-spinner"></div>
</div>
@@ -473,7 +746,13 @@ App.videos = App.videos || {};
${tagsMarkup}
`;
const thumb = card.querySelector('img');
App.videos.attachNoReferrerRetry(thumb);
// The layout probe (see shapeHeight) needs the card's shape, never its
// pixels: it measures against the CSS 16:9 placeholder and is removed in
// the same frame, so loading a thumbnail for it -- let alone racing one
// -- would be pure waste.
if (!(options && options.skipThumbnail)) {
App.videos.attachThumbnail(thumb, v.thumb);
}
const favoriteBtn = card.querySelector('.favorite-btn');
if (favoriteBtn && favoriteKey) {
App.favorites.setButtonState(favoriteBtn, favoritesSet.has(favoriteKey));
@@ -554,7 +833,7 @@ App.videos = App.videos || {};
App.player.open(v, { originEl: card });
};
cardVideo.set(card, v);
card.addEventListener('pointerenter', () => App.videos.resolveAndProbe(v), { once: true });
card.addEventListener('pointerenter', () => App.videos.ensureFormats(v), { once: true });
return card;
};
@@ -755,7 +1034,7 @@ App.videos = App.videos || {};
if (cached != null) return cached;
const el = grid();
if (!el) return 240;
const probe = App.videos.buildCard(v);
const probe = App.videos.buildCard(v, { skipThumbnail: true });
probe.style.position = 'absolute';
probe.style.visibility = 'hidden';
probe.style.left = '-99999px';
@@ -886,13 +1165,13 @@ App.videos = App.videos || {};
}
// Marquee + direct-playability probe only matter for on-screen cards.
requestAnimationFrame(() => { if (mounted.get(i) === card) measureTitle(card); });
probeObserver.observe(card);
resolveObserver.observe(card);
};
const unmount = function(i) {
const card = mounted.get(i);
if (!card) return;
probeObserver.unobserve(card);
resolveObserver.unobserve(card);
if (titleObserver) {
titleObserver.unobserve(card);
titleVisibility.delete(card);
@@ -1390,22 +1669,42 @@ App.videos = App.videos || {};
let ok = false;
let detail = '';
try {
// A simple GET (no custom headers) avoids a CORS preflight. If
// the response is readable and successful, CORS + reachability
// are both proven; we abort immediately so the body isn't
// downloaded (it can be a whole video file).
// A simple GET (no custom headers) avoids a CORS preflight.
const res = await fetch(url, {
method: 'GET',
mode: 'cors',
credentials: 'omit',
signal: controller.signal
});
ok = res.ok || res.status === 206;
detail = `HTTP ${res.status}`;
if (!(res.ok || res.status === 206)) {
ok = false;
detail = `HTTP ${res.status}`;
} else if (res.body && typeof res.body.getReader === 'function') {
// Run it until actual media bytes arrive. A readable status
// line is weaker evidence than it looks: the question is
// whether this origin will hand *the player* video data
// cross-origin, and that isn't settled until some has
// arrived. Then stop -- the rest of the file is not our
// business, and it can be a whole film.
const reader = res.body.getReader();
const chunk = await reader.read();
const bytes = (!chunk.done && chunk.value && chunk.value.length) || 0;
ok = bytes > 0;
detail = `HTTP ${res.status}, ${bytes} bytes`;
reader.cancel().catch(() => {});
} else {
// No readable stream to sample (an old browser): the status
// line is all the evidence on offer.
ok = true;
detail = `HTTP ${res.status}, headers only`;
}
controller.abort();
} catch (err) {
// A CORS refusal lands here as a TypeError with no status --
// the browser won't say more than "failed" about a response it
// wouldn't let us read.
ok = false;
detail = (err && err.name === 'AbortError') ? 'timeout' : (err && err.message) || 'fetch failed';
detail = (err && err.name === 'AbortError') ? 'timeout' : (err && err.message) || 'blocked (CORS)';
} finally {
clearTimeout(timer);
}
@@ -1418,37 +1717,18 @@ App.videos = App.videos || {};
return promise;
};
// Kicks off a background probe of a video's best (first-played) source so a
// later playback can skip the proxy if its host is proven reachable. Only
// runs once the video has resolved formats (see resolveAndProbe): those are
// real media URLs (or redirects to them), whereas a bare listing item only
// carries a page URL that the player can't use directly.
App.videos.probeVideoSources = function(video) {
if (!video || typeof video !== 'object') return;
const meta = video.meta || video;
if (!meta || !Array.isArray(meta.formats) || !meta.formats.length) return;
let sources;
try {
sources = App.videos.resolveStreamSources(video);
} catch (err) {
return;
}
const best = sources && sources[0];
if (!best || !best.url || best.isLive) return;
// Sources that require a specific upstream Referer can't be fetched
// directly by the browser (it can't forge a cross-origin Referer), so a
// probe would always fail -- leave them to the proxy.
if (best.refererRequired) return;
App.videos.probeDirect(best.url);
};
// Listing items arrive without formats (meta is null) -- only a page URL --
// so there's nothing direct-playable to probe up front. This resolves a
// video's real media formats via the backend (yt-dlp), attaches them as
// `video.meta` so the player and probe can use them, then probes the best
// source. Resolution is per-video and deduped: it runs at most once per
// video, triggered lazily by hover/scroll so we don't resolve cards the
// user never looks at.
// so this resolves a video's real media formats via the backend (yt-dlp) and
// attaches them as `video.meta`, which is what playback, the quality menu
// and the hover preview all need. Resolution is per-video and deduped: it
// runs at most once per video, triggered lazily by hover/scroll so we don't
// resolve cards the user never looks at.
//
// It deliberately does *not* test direct playability. That question belongs
// to the video actually being played (see raceDirect in player.js): a
// provider can spread its media over several CDNs, so an answer taken from
// whichever card happened to scroll past need not hold for the one the
// reader picks.
const cardVideo = new WeakMap();
// Session cache of `/api/resolve` results, keyed by video id (falling back
@@ -1559,21 +1839,16 @@ App.videos = App.videos || {};
return promise;
};
const probeObserver = new IntersectionObserver((entries) => {
const resolveObserver = new IntersectionObserver((entries) => {
entries.forEach((entry) => {
if (!entry.isIntersecting) return;
probeObserver.unobserve(entry.target);
resolveObserver.unobserve(entry.target);
const video = cardVideo.get(entry.target);
if (video) App.videos.resolveAndProbe(video);
if (video) App.videos.ensureFormats(video);
});
}, { rootMargin: '200px' });
App.videos.resolveAndProbe = function(video) {
if (!video || typeof video !== 'object') return Promise.resolve();
return App.videos.ensureFormats(video).then((meta) => {
if (meta) App.videos.probeVideoSources(video);
});
};
// Builds a proxied stream URL. Extra params other than `url` are forwarded
// by the backend as request headers, so use real header names here.