Compare commits

11 Commits

Author SHA1 Message Date
Simon
ba6045a2b1 Keep a favorite's formats when saving it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-11-02 22:50:40 +00:00
Simon
93463bce3a Ask the provider again for a favorite yt-dlp can't resolve
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-11-02 22:29:00 +00:00
Simon
2fcc17877d Let saved searches from different sources be read as one
A saved search is one source, one channel on it, that channel's filters
and the words. Two of them on different sources had no way to be read
together, and a channel group is not it: every channel in a group belongs
to the same server and shares one query, where the interesting pair is
"uncensored JAV over here" and "amateur 1080p over there".

So a saved search can now hold several of those instead of one. Opening it
feeds a single grid from all of them, taken alternately -- one from each in
turn, the way a group zips its channels -- rather than one source's page
and then the next. Each member keeps its own source, channel, filters and
words, because having its own words is most of why it was worth saving
separately.

Every entry is read as a list of members, a plain search being a merge of
one. That is what lets the daily check, mark-all-read and the bookmark go
on working without learning what a merge is: they ask each member and zip
the answers, so the badge counts what the grid would have shown.

The search box is turned off while a merge is open and says why -- there
is nothing for one box to say for several members. A tag or an uploader
tapped on a card steps out of the merge onto one real channel instead,
since otherwise it would reload the same videos and look broken.

A merge answers to no single server, so it is not stored as a channel
preference and is not looked up from a server's status: its members ride
on the session, and a reload rebuilds it from the saved search so a rename
is picked up rather than frozen in. One source failing drops that member
and leaves the rest readable. Merging a merge flattens it, so "the
searches in it" means the same however it was assembled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-11-01 07:59:12 +00:00
Simon
c60f5e01dc Re-anchor a moving reel on where it is, not what it was playing
A viewport change re-anchors the reel on the video that is playing,
resolved by id rather than by a scroll position the change has already
invalidated. That is what keeps a rotation on the same video.

Mid-swipe it names the wrong one. setActive runs from onScroll's next
frame, so a resize landing before that frame still reads the video being
swiped *away from* -- and realignToActive writes its position straight to
scrollTop, hauling the reel back to it. A phone slides its chrome on every
scroll, so the resize lands mid-swipe on nearly every swipe: the reel went
where the thumb put it, far enough to show the top of the next video at
the bottom of the screen, and was then written back a beat later.

The input guard in onScroll could not catch this. There is no scroll event
to guard -- the write happens inside the resize.

So while the reader is demonstrably driving, take the reel's own position
as the authority and re-anchor on the step it is nearest at the new
height. It still lands on a snap point, so nothing is left straddling two
screens, but the swipe stands. Only a step's worth of disagreement counts
as theirs: a scroller just re-parented into or out of the
picture-in-picture window reads 0, and following that would land them on
the first video instead of their own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-10-27 22:26:39 +00:00
Simon
7033288bc4 Hold the slides for the whole turn, not until it first pauses
The slides are held at the size a turn is heading for so the picture steps
once, straight into the shape it keeps, instead of being re-letterboxed
through every shape the viewport passes on its way round.

The hold came off once the height had gone six frames unchanged, which
stood in for "the viewport has stopped moving". It does not: a turn
arrives in steps, and the gap between two of them is routinely longer
than six frames -- on a phone and in a loaded tab alike. So the pin came
off mid-turn, the slides took an intermediate shape, and the picture
jumped twice on the way round. That is the stutter the pin exists to
remove, and TURN_HOLD_FRAMES was meant to be "long enough that a viewport
animating round in steps is not mistaken for one that has arrived" -- six
frames is not.

The device does say when a turn is underway, so believe it: while the
announcement stands the pin holds, and the frame count is left to end the
pins nothing announced, where there is no signal to go on and no turn to
ruin. The realign window has to stop concluding mid-turn too, since
closing it releases the pin by the other door; SETTLE_MAX_MS outlasts the
announcement and still guarantees both of them end.

The cost is that an announced turn keeps the old size until the
announcement lapses rather than letting go the moment the viewport looks
still. That buys the one step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-10-27 22:26:20 +00:00
Simon
872786a657 Let a swipe count while the viewport is still moving
A viewport change opens a window in which the feed holds the reader's
video in place, because the scroll-snap container re-anchors on positions
measured against the height from before it and must not be allowed to
flip the video. The window answered every scroll it saw with a realign.

On a phone that is the worst moment to do it: the chrome sliding out of
the way is caused by the swipe, so the window stood open at exactly the
moment the reader was swiping. The reel moved under the finger, far
enough to show the top of the next video at the bottom of the screen, and
was then pulled back to where it started -- a second later, once the
settle window ran out.

A re-anchor has nothing behind it and a swipe has a finger or a wheel, so
record the input and let a scroll that follows one close the window
instead of being reverted into it. Closing it that way skips the realign,
or the close would undo the swipe that caused it. Only input that means
to scroll counts -- a finger that has moved rather than one that has
merely landed, since a tap must not sign for the re-anchor after it.

Telling a turn apart from the chrome moving was reading the orientation
flip alone, which a few pixels of height cross whenever the viewport is
near enough to square. That pinned the slides and turned snapping off for
a second over a height change, which is the one thing it is there to tell
a turn apart from; a turn swaps the long side, and swapping it moves the
width too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-10-27 21:39:18 +00:00
Simon
7fc718158a Check a direct route hands back the video it was asked for
The proxy exists to get around CORS, hotlink protection and TLS
fingerprinting, so the player probes whether a CDN will serve the browser
itself and skips the hop when it will. The probe took a readable status
line and some bytes as proof.

Not every hotlink guard refuses a foreign Referer with a 403. eporner's
CDN answers one with a 302 to a one-minute "watch this at our site
instead" clip, served 206 with Access-Control-Allow-Origin: *. fetch()
follows that redirect without comment, so the probe read the stand-in as
a win, cached the host as direct, and the player restarted playback on
the raw URL -- where the browser sent the page's own Referer again and
got the stand-in a second time. An eporner video played as 61 seconds of
advertisement instead of itself.

Only the resolved path was affected: yt-dlp's eporner formats carry no
Referer, so refererRequired was false and the race ran. The listing's own
formats name one and were always proxied.

So ask the proxy, which carries the Referer the origin wants, how long
the media is, and refuse a direct route that reports another length. The
question goes out alongside the direct fetch rather than after it, and an
unknown length on either side leaves the old bytes test standing, so
chunked responses and playlists decide as before. No Range header: that
would cost a CORS preflight.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-10-22 20:58:45 +00:00
Simon
4f772773e1 Let the reader have the captured pictures taken again
A frame taken out of a video is kept for good: that is what makes a dead
favorite cheap to scroll past, and it is also the one thing that can go wrong
with no way out. A picture from the wrong moment, or one kept by a version of
this that would keep a frame with nothing in it, stays on the card for the life
of the item.

So Settings grows a Captured Thumbnails section: what is kept and how much room
it takes, and a button that throws the lot away. The cards on screen go back to
the start of the thumbnail ladder rather than waiting to be scrolled out of the
window and back, which is where a fresh capture comes from, and the session's
memory of which items there was no point asking about goes with the pictures --
otherwise "take them again" would quietly skip every item that failed once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-10-21 21:08:06 +00:00
Simon
be058dba37 Keep a captured thumbnail only if there is a picture in it
A favorite whose thumbnail is gone gets a frame out of the video instead. The
one this came from got a black box: 1.6 KB of pure black, kept for good, on an
item whose video is black nowhere -- ffmpeg puts its opening frame at YAVG 185
and its quarter mark at 130.

So that was not a frame of the video at all. Drawing a video element that has
not presented a frame gives back the canvas untouched, and transparent black
JPEGs as pure black at about 1.6 KB -- comfortably past the only test a capture
had to pass, which was being over 512 bytes long. Nothing then asked whether
the picture had anything in it, and the cache exists precisely so that nothing
asks again.

The frame now has to be a picture of something, and both ways it can fail to be
one have the same answer: take it somewhere else. The brightest sample in the
drawn frame decides -- an untouched canvas has no painted pixels at all, and a
frame that genuinely is black (a lead-in, a fade, a cut) is no better a
thumbnail than one of nothing. A blank frame moves on: the quarter mark again
with the element playing, since an element that is playing has to present
frames while a seeked one does not, and then the half mark. The frame is taken
on `requestVideoFrameCallback` where there is one -- the browser saying a frame
is on screen now -- rather than on `seeked` alone, which Chromium backs with a
decoded frame but Safari does not for an element that is off-screen and paused.

Nobody will say how long this item is (the extractor reports no duration and
the favorite kept a zero), and a clip with no length had no quarter mark to
seek to, so the capture settled for the first frame the element decoded -- the
opening one, which is the frame most likely to be black. It now seeks blind to
a fixed mark, clamped back into the file if that is past the end.

And the black pictures already kept heal: an index entry carries a marker
saying its picture was checked, every capture writes one, and an entry without
one is measured once -- behind the picture the card has already painted -- then
dropped and taken again if it turns out to be a frame of nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-10-21 20:20:05 +00:00
Simon
6dac1028a3 Take a thumbnail out of the video when the listed one is gone
A favorite outlives its thumbnail. Sources sign their CDN paths and the
signature expires, so a video kept months ago has a `thumb` that 404s and
a card that is a grey box with a title under it. Asking the source page
for a fresh picture covers the case where the page still exists; nothing
covered the case where it doesn't.

The video does, because playback re-resolves from the page URL every time.
So once every route to the listed picture is exhausted, a frame a quarter
of the way in becomes the card's picture -- past the black lead-in, and a
fair guess at what the video looks like. It goes through our own proxy, so
the canvas can be read back, is scaled to card width and JPEG'd to a
handful of KB, and is kept: later mounts and later sessions paint it
without a request, instead of asking again for a picture that is known to
be gone.

Only for favorites, once per item, one at a time, and never for a live
stream or a manifest. Two things worth knowing, both found with the
browser rather than reasoned about: a browser fires `seeked` for its own
initial seek to zero, so taking the frame on the first one captures the
opening of every video; and the same item is on screen in more than one
place, so the captures are deduped by a promise rather than the first
asker getting the picture and the rest being told there isn't one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-10-20 07:33:38 +00:00
Simon
e8eac70f4d Write a check into the list instead of rebuilding it
A check asks each saved search in turn and writes back what it found, and
the list was built again from scratch on every one of those writes. So a
background check -- which has no business being visible at all beyond a
new count and a new time -- replayed the whole list's entry animation,
threw away the reader's scroll position, and took the pointer out of
whatever row it was over, several times over.

Rows are now matched by id and told what changed: the count, the name, the
line underneath, nothing else. A row is only built when it is new, gone, or
changes shape, and the entry animation belongs to a row that has arrived
rather than to every row every time. One mid-rename or mid-confirm is left
alone entirely, so a check can no longer type over the reader.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
2026-10-20 07:33:27 +00:00
20 changed files with 3018 additions and 89 deletions

View File

@@ -160,6 +160,13 @@ body.theme-light .top-bar {
color: var(--text-secondary);
}
/* A merge reads its members' own words, so the box has nothing to say for it.
Dimmed rather than hidden: the placeholder is where it says why. */
.search-container input.is-inert {
opacity: 0.6;
cursor: default;
}
.search-clear-btn {
position: absolute;
top: 50%;
@@ -3279,6 +3286,11 @@ body.theme-light .saved-badge { box-shadow: 0 0 0 2px rgba(246, 241, 231, 0.95);
align-items: center;
gap: 4px;
border-bottom: 1px solid var(--border);
}
/* Rows animate in when they arrive, and only then: a row already on screen
that a background check wrote a new count into has not arrived anywhere. */
.saved-row.is-new {
animation: saved-row-in 0.3s cubic-bezier(0.16, 1, 0.3, 1) both;
animation-delay: calc(var(--i, 0) * 40ms);
}
@@ -3332,6 +3344,22 @@ body.theme-light .saved-badge { box-shadow: 0 0 0 2px rgba(246, 241, 231, 0.95);
.saved-row.is-unreachable .saved-count { border-style: dashed; }
/* Picking rows to merge. The count becomes a tick box: empty until the row is
chosen, so a glance down the list says which ones are going in. */
.saved-count.saved-pick {
border-color: var(--border-hover);
background: transparent;
color: var(--accent);
font-size: 15px;
}
.saved-row.is-picked .saved-count.saved-pick {
background: rgba(201, 165, 103, 0.16);
border-color: var(--accent);
}
.saved-row.is-picked { background: var(--bg-tertiary); }
.saved-row-text {
min-width: 0;
display: flex;

View File

@@ -163,6 +163,14 @@
</div>
<div id="sources-list" class="sources-list"></div>
</div>
<div class="sidebar-section">
<h4 class="sidebar-subtitle">Captured Thumbnails</h4>
<div class="setting-item">
<label for="clear-posters-btn">Kept Pictures</label>
<button id="clear-posters-btn" class="btn-secondary" type="button">Take them again</button>
<p id="clear-posters-status" class="setting-note" role="status" aria-live="polite"></p>
</div>
</div>
<div class="sidebar-section">
<h4 class="sidebar-subtitle">Hot Tub Backup</h4>
<div class="setting-item">
@@ -251,6 +259,7 @@
<p class="saved-head-note" id="saved-head-note"></p>
</div>
<div class="saved-head-actions">
<button id="saved-select-btn" class="btn-link" type="button">Merge…</button>
<button id="saved-read-btn" class="btn-link" type="button">Mark all read</button>
<button id="saved-check-btn" class="btn-link" type="button">Check now</button>
<button id="saved-close-btn" class="close-btn" type="button" aria-label="Close">✕</button>
@@ -262,6 +271,7 @@
then press the bookmark in the search bar to keep the whole thing.
</div>
<div class="saved-foot">
<button id="saved-merge-btn" class="btn-secondary saved-add" type="button" hidden>Merge selected</button>
<button id="saved-add-btn" class="btn-secondary saved-add" type="button">Save the current search</button>
</div>
</div>
@@ -276,6 +286,7 @@
<script src="static/js/player.js"></script>
<script src="static/js/favorites.js"></script>
<script src="static/js/favoritesView.js"></script>
<script src="static/js/posters.js"></script>
<script src="static/js/sqlite.js"></script>
<script src="static/js/hottubBackup.js"></script>
<script src="static/js/videos.js"></script>

View File

@@ -16,13 +16,9 @@ App.favorites = App.favorites || {};
const raw = localStorage.getItem(FAVORITES_KEY);
const parsed = raw ? JSON.parse(raw) : [];
if (!Array.isArray(parsed)) return [];
// Two things are repaired on the way in, and written back once if
// One thing is repaired on the way in, and written back once if
// anything changed, so the fix happens exactly one time:
//
// `meta`, from older versions, is a blob of resolved formats whose
// URLs are signed and long expired -- dropped so no code path can
// reach for one; everything re-resolves from `url` at play time.
//
// `favoriteDate` didn't exist before sorting needed it. There's no
// way to recover when an old favorite was actually saved, so it
// gets now: they sort together, as one batch, at the point the
@@ -31,10 +27,9 @@ App.favorites = App.favorites || {};
const now = new Date().toISOString();
const items = parsed.map((item) => {
if (!item || typeof item !== 'object') return item;
if (!item.meta && item.favoriteDate) return item;
if (item.favoriteDate) return item;
const clean = Object.assign({}, item);
delete clean.meta;
if (!clean.favoriteDate) clean.favoriteDate = now;
clean.favoriteDate = now;
repaired = true;
return clean;
});
@@ -103,6 +98,18 @@ App.favorites = App.favorites || {};
return video.key || meta.key || video.id || meta.id || video.url || meta.url || null;
};
const formatsOf = function(video, meta) {
const src = Array.isArray(video.formats) && video.formats.length ? video
: (meta && Array.isArray(meta.formats) && meta.formats.length ? meta : null);
if (!src) return undefined;
return {
url: src.url || video.url || '',
formats: src.formats,
http_headers: src.http_headers || {},
isLive: !!(src.isLive || video.isLive)
};
};
App.favorites.normalize = function(video) {
const key = App.favorites.getKey(video);
if (!key) return null;
@@ -110,9 +117,7 @@ App.favorites = App.favorites || {};
return {
key,
id: video.id || null,
// The page/source URL (e.g. the YouTube watch URL), not a resolved
// CDN media URL -- those expire, so favorites must always re-resolve
// via the server at play time instead of caching a stream link.
// The page/source URL (e.g. the YouTube watch URL).
url: video.url || (meta && meta.url) || '',
title: video.title || '',
thumb: video.thumb || '',
@@ -122,13 +127,10 @@ App.favorites = App.favorites || {};
isLive: !!(video.isLive || (meta && meta.isLive)),
// When it was saved. An import carries the date the other client
// recorded; anything saved here is saved now.
favoriteDate: video.favoriteDate || new Date().toISOString()
// No `meta` field: persisting resolved formats would freeze their
// (expiring) CDN URLs into localStorage. Leaving it unset makes a
// favorite look like a fresh, unresolved listing item again, so
// playback/download/info all re-resolve through the backend from
// `url` -- see resolveStreamSources' no-formats fallback, which the
// backend resolves live via yt-dlp (main.py stream_video).
favoriteDate: video.favoriteDate || new Date().toISOString(),
// The formats the video was saved with, so it plays without being
// resolved again. Only when there are some.
meta: formatsOf(video, meta)
};
};
@@ -251,7 +253,7 @@ App.favorites = App.favorites || {};
'uploader', 'duration', 'isLive', 'favoriteDate'];
const sameEntry = function(a, b) {
return ENTRY_FIELDS.every((field) =>
return JSON.stringify(a.meta || null) === JSON.stringify(b.meta || null) && ENTRY_FIELDS.every((field) =>
String(a[field] === undefined || a[field] === null ? '' : a[field]) ===
String(b[field] === undefined || b[field] === null ? '' : b[field]));
};
@@ -494,14 +496,11 @@ App.favorites = App.favorites || {};
`;
const thumb = card.querySelector('img');
if (App.videos && typeof App.videos.attachThumbnail === 'function') {
App.videos.attachThumbnail(thumb, item.thumb, App.videos.sourcePageUrl(item));
App.videos.attachThumbnail(thumb, item.thumb, App.videos.sourcePageUrl(item), item);
}
card.onclick = () => {
if (card.classList.contains('is-loading')) return;
card.classList.add('is-loading');
// Ignore any stale `meta` a favorite saved before this fix may
// still carry in localStorage -- always re-resolve from `item`
// (id/url) so playback never reuses an expired stream URL.
App.player.open(item, { originEl: card });
};
const favoriteBtn = card.querySelector('.favorite-btn');
@@ -525,9 +524,6 @@ App.favorites = App.favorites || {};
showInfoBtn.onclick = (event) => {
event.stopPropagation();
App.videos.closeAllMenus();
// Favorites deliberately store no resolved metadata; the
// panel opens on what the entry holds and resolves the rest
// itself.
App.ui.openInfo(item);
};
}

View File

@@ -44,10 +44,21 @@ App.feed = App.feed || {};
// scroll events with positions that no longer map to the active slide;
// onResize sets this for the brief realign window so those events don't
// flip the active video -- rotating the device must never change which
// slide is playing. Normal swipes (no resize in flight) are unaffected.
// slide is playing.
//
// What it must not ignore is the reader. A re-snap has nothing behind it;
// a swipe has a finger or a wheel. From the scroll event alone the two are
// identical, so the input is recorded and a scroll that follows one closes
// the window instead of being reverted into it. Without that, a swipe on a
// phone was routinely thrown away: the chrome sliding out of the way *is*
// the scroll, so the window was open at exactly the moment it did most
// harm -- the reel moved under the finger, showing the next video coming
// in, and was then pulled back to where it started.
let suppressScroll = false;
let resizeSettleRaf = null;
let resizeSettleTimer = null;
const INPUT_GRACE_MS = 700;
let lastInputAt = 0;
// A rotation is not one event. The viewport, the browser's own chrome and
// the scroll-snap container settle over many frames and often several
// `resize` events, so the realign window is held open until the slide
@@ -807,7 +818,7 @@ App.feed = App.feed || {};
</div>
`;
const poster = pane.querySelector('.feed-poster');
App.videos.attachThumbnail(poster, v.thumb, App.videos.sourcePageUrl(v));
App.videos.attachThumbnail(poster, v.thumb, App.videos.sourcePageUrl(v), v);
const slideVideo = pane.querySelector('.feed-video');
bindTimeline(pane, slideVideo);
bindSharedControls(pane, slideVideo, v);
@@ -1319,8 +1330,14 @@ App.feed = App.feed || {};
// in the event, is what keeps that position from being painted --
// waiting for the next frame means showing it first.
if (suppressScroll) {
realignToActive();
return;
// Unless it is the reader scrolling. A swipe closes the window
// instead of being reverted into it -- and closes it without a
// realign, or the close would undo the swipe that caused it.
if (Date.now() - lastInputAt >= INPUT_GRACE_MS) {
realignToActive();
return;
}
closeSettle(false);
}
if (scrollRaf) return;
scrollRaf = requestAnimationFrame(() => {
@@ -1358,6 +1375,12 @@ App.feed = App.feed || {};
const isTurn = function(before, now) {
if (Date.now() - turnSignalAt < TURN_SIGNAL_MS) return true;
if (!before) return false;
// Swapping the long side moves both of them. Reading the flip on its
// own calls a few pixels of height a rotation whenever the viewport is
// near enough to square for them to cross it -- and then pins the
// slides and turns snapping off for a second over the chrome sliding
// away, which is the one thing this is here to tell a turn apart from.
if (before.w === now.w) return false;
return (before.w > before.h) !== (now.w > now.h);
};
@@ -1390,16 +1413,40 @@ App.feed = App.feed || {};
// video playing/focused rather than snapping to a neighbour.
const realignToActive = function() {
if (stepCount() === 0) return;
const h = slideHeight();
const scroller = getScroller();
// Anchoring by id is what keeps a rotation on the same video, because
// the scroll position it would otherwise read has been invalidated by
// the very change being answered. But mid-swipe that id names the video
// being *left*: setActive runs from onScroll's next frame, so a resize
// landing before it still reads the old one -- and writing its position
// hauls the reel back to it. A phone slides its chrome on every scroll,
// so that happened on nearly every swipe.
//
// While the reader is demonstrably driving, the reel's own position is
// the better authority: re-anchor on the step it is nearest at the new
// height. That still puts it on a snap point -- no slide left straddling
// two screens -- without undoing the swipe.
let index = state.feedActiveIndex;
if (state.feedActiveVideoId != null) {
const found = (state.loadedVideos || [])
.findIndex((v) => String(v.id) === String(state.feedActiveVideoId));
if (found >= 0) index = Math.floor(found / paneCount());
}
const driving = Date.now() - lastInputAt < INPUT_GRACE_MS;
if (driving && scroller && h) {
// A swipe moves by a step, so only a step's worth of disagreement
// is the reader's doing. Anything further apart is the position
// being meaningless rather than ahead -- a scroller that has just
// been re-parented into (or out of) the picture-in-picture window
// reads 0, and following that would land the reel on the first
// video instead of theirs.
const atScroll = clampIndex(Math.round(scroller.scrollTop / h));
if (atScroll >= 0 && Math.abs(atScroll - index) <= 1) index = atScroll;
}
index = clampIndex(index);
if (index < 0) return;
state.feedActiveIndex = index;
const h = slideHeight();
const start = windowBounds(index).start;
// Only write what actually differs. This runs on every frame of a
// turn, and a spacer height or a scrollTop written back unchanged
@@ -1408,7 +1455,6 @@ App.feed = App.feed || {};
const spacer = getTopSpacer();
const spacerHeight = `${start * h}px`;
if (spacer && spacer.style.height !== spacerHeight) spacer.style.height = spacerHeight;
const scroller = getScroller();
const top = index * h;
if (scroller && Math.abs(scroller.scrollTop - top) > 0.5) scroller.scrollTop = top;
const activeSlide = slidesByIndex.get(index);
@@ -1420,8 +1466,11 @@ App.feed = App.feed || {};
}
};
// Ends the realign window: one last realign, then real swipes count again.
const finishSettle = function() {
// Ends the realign window, and `realign` says whether to re-anchor on the
// way out. A window that closes on its own does -- holding the reader's
// video in place is what it was opened for. One cut short by a swipe must
// not, or closing it would undo the very swipe that closed it.
const closeSettle = function(realign) {
if (resizeSettleRaf) {
cancelAnimationFrame(resizeSettleRaf);
resizeSettleRaf = null;
@@ -1431,7 +1480,7 @@ App.feed = App.feed || {};
resizeSettleTimer = null;
}
releaseTurn();
realignToActive();
if (realign) realignToActive();
suppressScroll = false;
lastShape = viewportShape();
const after = settleDone;
@@ -1439,6 +1488,11 @@ App.feed = App.feed || {};
if (after) after();
};
// Ends the realign window: one last realign, then real swipes count again.
const finishSettle = function() {
closeSettle(true);
};
// Watches the viewport settle, frame by frame, holding the active video in
// place the whole way.
const settleStep = function() {
@@ -1461,15 +1515,29 @@ App.feed = App.feed || {};
if (!turnPin && !turnResolved && isTurn(settleFromShape, shape)) {
pinTurn(settleFromShape);
}
// A turn the device announced is not over because the viewport paused.
// It arrives in steps, and the gap between two of them is routinely
// longer than the handful of frames that stands in for "stopped" --
// which is how the pin came off mid-turn and let the picture take an
// intermediate shape, the one thing it is here to prevent. So while
// the announcement stands the pin holds, and the frame count is left
// to end the pins nothing announced: a window dragged across the
// square, where there is no signal to go on and no turn to ruin.
const announced = Date.now() - turnSignalAt < TURN_SIGNAL_MS;
// The viewport has stopped moving, so what it is now is better than
// what the pin guessed it would be. Letting go here, with the realign
// below in the same frame, is the one step the picture takes.
if (turnPin && settleStableFrames >= TURN_HOLD_FRAMES) {
if (turnPin && !announced && settleStableFrames >= TURN_HOLD_FRAMES) {
releaseTurn();
turnResolved = true;
}
realignToActive();
if (settleStableFrames < SETTLE_STABLE_FRAMES &&
// The window may not conclude mid-turn either: closing it releases the
// pin on the way out, which would put the picture through the same
// intermediate shape by the other door. An announced turn therefore
// holds the window open too, and SETTLE_MAX_MS -- longer than the
// announcement -- is what still guarantees it ends.
if ((announced || settleStableFrames < SETTLE_STABLE_FRAMES) &&
(Date.now() - settleStartedAt) < SETTLE_MAX_MS) {
resizeSettleRaf = requestAnimationFrame(settleStep);
return;
@@ -1584,7 +1652,7 @@ App.feed = App.feed || {};
const titleText = pane.querySelector('.feed-title-text');
if (titleText) titleText.textContent = v.title || '';
const poster = pane.querySelector('.feed-poster');
if (poster) App.videos.attachThumbnail(poster, v.thumb, App.videos.sourcePageUrl(v));
if (poster) App.videos.attachThumbnail(poster, v.thumb, App.videos.sourcePageUrl(v), v);
loadSlideSource(pane, v, true);
setPipMetadata(v);
};
@@ -1940,6 +2008,19 @@ App.feed = App.feed || {};
if (!scrollBound) {
scroller.addEventListener('scroll', onScroll, { passive: true });
// How the realign window tells a swipe from the container
// re-anchoring itself (see suppressScroll): a scroll with one of
// these behind it is the reader's, and outranks the window.
// Only input that means to scroll counts -- a finger that has
// moved, not one that has merely landed, since a tap must not
// sign for the re-anchor that follows it.
const noteInput = () => { lastInputAt = Date.now(); };
['wheel', 'touchmove'].forEach((type) => {
scroller.addEventListener(type, noteInput, { passive: true });
});
// Arrow keys and Page Up/Down scroll the container too, and the
// keystroke lands on whatever has focus rather than the scroller.
window.addEventListener('keydown', noteInput, { passive: true });
window.addEventListener('resize', onResize);
// `orientationchange` fires before the new metrics are readable and
// the visual viewport moves on its own as the browser's chrome

View File

@@ -830,7 +830,7 @@ App.player = App.player || {};
// a browser, so there is nothing to find out.
if (resolved.refererRequired) return;
if (directProven(resolved.url)) return;
App.videos.probeDirect(resolved.url).then((ok) => {
App.videos.probeDirect(resolved).then((ok) => {
if (!ok || token !== cp.attemptToken) return;
// readyState >= HAVE_CURRENT_DATA means a frame is up; leave a
// playing video alone rather than trading a visible stall for a

583
frontend/js/posters.js Normal file
View File

@@ -0,0 +1,583 @@
window.App = window.App || {};
App.posters = App.posters || {};
// ---------------------------------------------------------------------------
// Captured posters
//
// A favorite outlives its thumbnail. The picture a listing hands out is a
// signed CDN path on most sources, and the signature expires -- so a video kept
// months ago has a `thumb` that 404s, and the card that stood for it is a grey
// box with a title under it. Asking the source page for a fresh one (the
// /api/poster repair in videos.js) covers the case where the page still exists
// and still shows a picture; nothing covers the case where it doesn't.
//
// The video itself does, though: it is still playable, because playback
// re-resolves from the page URL every time. So once every route to the listed
// picture is exhausted, a frame a quarter of the way into the clip becomes the
// card's picture -- far enough in to be past a title card or a black lead-in,
// and a fair guess at what the video looks like.
//
// A frame costs a resolve and the stretch of video around that point, which is
// why this happens once per item, for favorites only, and never again: the
// frame is scaled down, JPEG'd, and kept in localStorage, so the next mount and
// every later session paint it straight away without touching the network.
//
// Which is also why a capture has to be looked at before it is kept. A frame
// with nothing in it -- an element that decoded nothing draws as nothing, and a
// lead-in or a fade is black in its own right -- is a black box on the card for
// the life of the item, and asking again is exactly what the cache is there to
// avoid. So a blank frame is taken somewhere else in the clip instead (see
// planFor), and one kept by a version of this that did not check gets measured
// once and replaced (see recheck).
// ---------------------------------------------------------------------------
(function() {
const INDEX_KEY = App.constants.POSTER_CACHE_KEY;
const ENTRY_PREFIX = `${INDEX_KEY}:`;
// Marks an entry whose picture was checked for having a picture in it,
// which every capture now is and none of the older ones were.
const CHECKED = 1;
// A thumbnail box is around 300 CSS px wide in the grid and half a phone
// screen in the favorites bar, so this is a picture at roughly card size
// and not a frame of video: the point of the cache is to be small.
const WIDTH = 360;
const QUALITY = 0.72;
// How far into the clip the frame comes from.
const SEEK_FRACTION = 0.25;
// What the whole cache may take. Stored base64, so the string is a third
// larger than the JPEG behind it -- around 20 KB an entry, which leaves
// room for a hundred or so of them inside a localStorage budget that is
// shared with the favorites themselves.
const BUDGET_BYTES = 3 * 1024 * 1024;
// How long a seek gets before the frame it was decoding anyway will do.
const SEEK_PATIENCE_MS = 6000;
// How long a frame gets to be presented once the data for it has arrived.
const PRESENT_PATIENCE_MS = 500;
// A capture is a handful of seeks into one file. If none of them has
// produced a picture by now, the source is not going to give us one.
const FRAME_TIMEOUT_MS = 30000;
// How bright the brightest sample in a frame has to be for the frame to be
// a picture of anything at all. A floor, not a threshold: this is the
// difference between a frame and no frame, not between a dark scene and a
// bright one.
const BLACK_LEVEL = 10;
// Every fourth pixel answers that question, and keeps the check off the
// cost of a capture.
const SAMPLE_STRIDE = 4;
// Where to look in a clip whose length nobody will say.
const BLIND_MARK_SECONDS = 10;
// How many of the ranked sources are worth trying before giving up on an
// item for this session.
const SOURCE_TRIES = 2;
// Both halves of a 64-bit-ish hash of the item's identity: a cache key has
// to be short enough to be a localStorage key and distinct enough that two
// favorites never share a picture.
const hash = function(text) {
let fnv = 0x811c9dc5;
let djb = 5381;
for (let i = 0; i < text.length; i++) {
const code = text.charCodeAt(i);
fnv = (fnv ^ code) >>> 0;
fnv = (fnv + ((fnv << 1) + (fnv << 4) + (fnv << 7) + (fnv << 8) + (fnv << 24))) >>> 0;
djb = (((djb << 5) + djb) + code) >>> 0;
}
return fnv.toString(16).padStart(8, '0') + djb.toString(16).padStart(8, '0');
};
// The identity to cache under: the item's own URL, normalized the way
// favorites normalize one, so the same video saved from a listing and
// imported from a backup lands on the same picture.
const keyOf = function(video) {
if (!video || typeof video !== 'object') return '';
const url = video.url || (video.meta && video.meta.url) || '';
const basis = (url && App.favorites && App.favorites.urlKey(url)) || url ||
(App.favorites && App.favorites.getKey(video)) || video.id || '';
return basis ? hash(String(basis)) : '';
};
// ---- the cache -------------------------------------------------------
// [{ k, n, at }], oldest capture first. Read once and kept: a card asks
// whether it has a picture as it mounts, and parsing the index again for
// every card of a scrolling grid is the one thing this must not cost.
let index = null;
const readIndex = function() {
if (index) return index;
try {
const parsed = JSON.parse(localStorage.getItem(INDEX_KEY));
index = Array.isArray(parsed) ?
parsed.filter((entry) => entry && entry.k).map((entry) => ({
k: String(entry.k),
n: Number(entry.n) || 0,
at: Number(entry.at) || 0,
// Whether anything ever looked at what this entry holds.
// Absent on everything written before that was checked --
// see recheck.
v: Number(entry.v) || 0
})) : [];
} catch (err) {
index = [];
}
return index;
};
const writeIndex = function() {
try {
localStorage.setItem(INDEX_KEY, JSON.stringify(index));
} catch (err) { /* nothing here is worth failing a card over */ }
};
const drop = function(entry) {
try {
localStorage.removeItem(ENTRY_PREFIX + entry.k);
} catch (err) { /* already gone */ }
};
const store = function(key, src) {
const list = readIndex().filter((entry) => entry.k !== key);
list.push({ k: key, n: src.length, at: Date.now(), v: CHECKED });
index = list;
// Oldest captures go first. Not least-recently-seen: tracking that
// would mean writing to storage every time a card mounts, which is a
// worse trade than occasionally re-capturing a picture.
let total = list.reduce((sum, entry) => sum + entry.n, 0);
while (total > BUDGET_BYTES && index.length > 1) {
const oldest = index.shift();
total -= oldest.n;
drop(oldest);
}
try {
localStorage.setItem(ENTRY_PREFIX + key, src);
} catch (err) {
// Out of room, and not necessarily because of us -- the store is
// shared with the favorites and everything else. Make room by
// giving up the oldest third of the cache and try once more.
const giveUp = Math.max(1, Math.floor(index.length / 3));
index.splice(0, giveUp).forEach(drop);
try {
localStorage.setItem(ENTRY_PREFIX + key, src);
} catch (again) {
index = index.filter((entry) => entry.k !== key);
writeIndex();
return '';
}
}
writeIndex();
return src;
};
const read = function(key) {
try {
return localStorage.getItem(ENTRY_PREFIX + key) || '';
} catch (err) {
return '';
}
};
// The picture already captured for this item, if there is one.
App.posters.cached = function(video) {
const key = keyOf(video);
if (!key) return '';
// The index says what we have without touching the entries, so an item
// with no captured picture -- almost all of them -- costs a set lookup
// rather than a read of a data URL out of storage.
if (!readIndex().some((entry) => entry.k === key)) return '';
const src = read(key);
if (!src) {
index = readIndex().filter((entry) => entry.k !== key);
writeIndex();
}
return src;
};
App.posters.forget = function(video) {
const key = keyOf(video);
if (!key) return;
readIndex().filter((entry) => entry.k === key).forEach(drop);
index = readIndex().filter((entry) => entry.k !== key);
writeIndex();
};
// Throws every captured picture away, and with it this session's memory of
// which items there was no point asking about. That memory is the whole
// reason a dead favorite is cheap to scroll past, so it only goes when the
// reader asks for the pictures to be taken again (Settings -> Captured
// Thumbnails) -- otherwise an item that failed once would be retried on
// every mount for the rest of the session.
App.posters.clear = function() {
readIndex().forEach(drop);
index = [];
writeIndex();
failed.clear();
rechecks.clear();
};
App.posters.stats = function() {
const list = readIndex();
return { count: list.length, bytes: list.reduce((sum, entry) => sum + entry.n, 0) };
};
// ---- taking the frame ------------------------------------------------
// Is there a picture in what was just drawn?
//
// Two things come out of a capture looking identical, and both are useless.
// A video that has decoded nothing draws as nothing: the canvas keeps its
// transparent black, which becomes a pure black JPEG about 1.6 KB long --
// large enough to pass for a picture, and kept for the life of the item.
// And a frame that genuinely is black (a lead-in, a fade, a cut between
// scenes) is no better a thumbnail than that one. The answer to both is to
// take the frame somewhere else, so they are the same question.
const isBlank = function(ctx, width, height) {
// Tainted canvas throws here, which is why the frame is taken from our
// own proxy rather than from the CDN: /api/stream is same-origin, so
// the pixels can be read back.
const px = ctx.getImageData(0, 0, width, height).data;
let painted = false;
let brightest = 0;
for (let i = 0; i < px.length; i += 4 * SAMPLE_STRIDE) {
if (px[i + 3] > 0) painted = true;
const value = Math.max(px[i], px[i + 1], px[i + 2]);
if (value > brightest) brightest = value;
}
return !painted || brightest <= BLACK_LEVEL;
};
// Draws what the element is showing, at card size rather than video size.
// Empty when there turns out to be nothing in it, which is the caller's cue
// to look elsewhere in the clip rather than keep it.
const drawFrame = function(el) {
const width = el.videoWidth;
const height = el.videoHeight;
if (!width || !height) throw new Error('nothing was decoded');
const scale = Math.min(1, WIDTH / width);
const canvas = document.createElement('canvas');
canvas.width = Math.max(1, Math.round(width * scale));
canvas.height = Math.max(1, Math.round(height * scale));
const ctx = canvas.getContext('2d');
ctx.drawImage(el, 0, 0, canvas.width, canvas.height);
if (isBlank(ctx, canvas.width, canvas.height)) return '';
const src = canvas.toDataURL('image/jpeg', QUALITY);
if (!src || src.length < 512) throw new Error('the frame came out empty');
return src;
};
// Where a frame comes from, and what to do when it comes back blank. A
// blank frame means one of two things and each has its own answer: the
// element decoded nothing (so play it for a moment -- an element that is
// playing has to present frames, which a seeked one is under no obligation
// to do), or the frame at that mark really is black (so take it somewhere
// else).
const planFor = function(length) {
if (length > 0 && isFinite(length)) {
return [
{ at: length * SEEK_FRACTION, play: false },
{ at: length * SEEK_FRACTION, play: true },
{ at: length * 0.5, play: true }
];
}
// Nobody will say how long this is: an extractor that reports no
// duration, a file the browser won't commit to a length for. A blind
// seek to a fixed mark still beats the opening frame, which is a black
// one often enough to be half the reason this check exists -- and a
// mark past the end comes back clamped into the file, so the guess
// costs nothing.
return [
{ at: BLIND_MARK_SECONDS, play: false },
{ at: 0, play: true }
];
};
const grabFrame = function(url, duration) {
return new Promise((resolve, reject) => {
const el = document.createElement('video');
el.muted = true;
el.defaultMuted = true;
el.playsInline = true;
el.preload = 'auto';
el.crossOrigin = 'anonymous';
el.setAttribute('aria-hidden', 'true');
// Off-screen, but in the document: an element the browser does not
// consider part of a page is under no obligation to decode
// anything.
el.style.cssText = 'position:fixed;left:-9999px;top:0;width:2px;height:2px;' +
'opacity:0;pointer-events:none';
let settled = false;
let budget = null;
let patience = null;
// The marks to try, and which of them we are on.
let plan = [];
let step = -1;
// Where we asked to be, and whether we asked at all. A browser
// fires `seeked` for its own initial seek to zero as the file
// opens, and taking the frame on that one would mean every
// captured picture is the first frame of the video -- usually a
// black one -- however far in we meant to go.
let target = 0;
let asked = false;
// A frame is already being waited for. A completed seek and the
// patience timer can both decide it is time, and the second one to
// decide must not start a second attempt on the same frame.
let pending = false;
const done = function(err, src) {
if (settled) return;
settled = true;
if (budget) clearTimeout(budget);
if (patience) clearTimeout(patience);
try { el.pause(); } catch (stop) { /* never started */ }
// Told to stop downloading before it leaves, or the request
// outlives the element that made it.
el.removeAttribute('src');
try { el.load(); } catch (stop) { /* nothing to stop */ }
el.remove();
if (err) reject(err);
else resolve(src);
};
const take = function() {
pending = false;
let src = '';
try {
src = drawFrame(el);
} catch (err) {
done(err);
return;
}
if (src) {
done(null, src);
return;
}
// Nothing in it, so this mark is spent.
attempt();
};
// Drawing an element that has not presented a frame gives an empty
// canvas, and the engines disagree about when that is: Chromium has
// a decoded frame by the time it says a seek is done, while Safari
// will not present one for an off-screen paused element until it
// has to. `requestVideoFrameCallback` is the browser saying a frame
// is on screen now, which is the only direct answer there is; where
// there is none, a decoded readyState after a paint is the nearest
// thing to one.
const whenPresented = function() {
if (settled || pending) return;
pending = true;
let taken = false;
const once = function() {
if (taken || settled) return;
taken = true;
take();
};
if (typeof el.requestVideoFrameCallback === 'function') {
el.requestVideoFrameCallback(() => once());
// An element that is paused on the frame it already
// presented may never present another, so the callback on
// its own would wait for good.
setTimeout(() => {
if (el.readyState >= 2) once();
else pending = false;
}, PRESENT_PATIENCE_MS);
return;
}
requestAnimationFrame(() => {
if (el.readyState >= 2) once();
else setTimeout(once, PRESENT_PATIENCE_MS);
});
};
const attempt = function() {
step += 1;
if (step >= plan.length) {
done(new Error('every frame came out blank'));
return;
}
if (patience) clearTimeout(patience);
const mark = plan[step];
target = mark.at;
if (mark.play) {
const playing = el.play();
if (playing && typeof playing.catch === 'function') playing.catch(() => {});
}
if (target > 0 && Math.abs(el.currentTime - target) > 1) {
asked = true;
try { el.currentTime = target; } catch (err) { done(err); return; }
} else {
// Already where this attempt wanted to be: all that has
// changed is that the element is now playing.
whenPresented();
}
// Some sources cannot be seeked in at all: no byte ranges, or
// a length the browser does not trust. They still have an
// opening frame, and a picture of the opening beats a grey box
// -- but only once the seek has had its chance.
patience = setTimeout(() => {
if (el.readyState >= 2) whenPresented();
}, SEEK_PATIENCE_MS);
};
budget = setTimeout(() => done(new Error('no frame in time')), FRAME_TIMEOUT_MS);
el.addEventListener('error', () => done(new Error('the source would not play')));
el.addEventListener('loadedmetadata', () => {
plan = planFor(Number(el.duration) || Number(duration) || 0);
step = -1;
attempt();
});
el.addEventListener('seeked', () => {
// Not ours: the browser's own seek to zero as the file opens.
if (!asked || (target > 0 && el.currentTime <= 0)) return;
// Near enough: a seek lands on the nearest frame it can decode,
// which is the keyframe before the mark rather than the mark
// itself, and a mark past the end comes back clamped to as
// close as that file gets.
whenPresented();
});
document.body.appendChild(el);
el.src = url;
});
};
const captureOne = async function(video) {
// Favorites keep no resolved formats on purpose (their URLs expire), so
// this is where the item becomes playable again.
try {
await App.videos.ensureFormats(video);
} catch (err) { /* the page-URL fallback below may still play */ }
const sources = App.videos.resolveStreamSources(video, {
applyPreferredQuality: false,
// The smallest format there is: this is going to end up 360 px
// wide, so a 1080p stream would be bytes spent on pixels that get
// thrown away.
cheapest: true
}).filter((source) => {
// A manifest is a playlist of segments, not something a bare
// <video> can open outside Safari, and the player's hls.js is a
// heavier thing than a thumbnail is worth.
const kind = App.videos.classifySource(source);
return !kind.isHls && !source.isLive;
}).slice(0, SOURCE_TRIES);
for (let i = 0; i < sources.length; i++) {
const url = App.videos.buildStreamUrlFromSource(sources[i]);
if (!url) continue;
try {
return await grabFrame(url, Number(video.duration) || 0);
} catch (err) {
// Try the next format; if that was the last one, the item has
// no picture and no amount of asking changes it.
}
}
return '';
};
// The same item is on screen in more than one place -- the favorites bar
// and the grid behind it, a card and the reel's own poster -- and all of
// them ask. They wait on the one capture and all get the picture; the
// alternative is the first one to ask getting a picture and the rest being
// told there isn't one.
const inFlight = new Map();
// And one try per item per session once it has come back empty: a failed
// capture is a resolve plus a download, and a grid that scrolls past the
// same dead favorite ten times must not pay for it ten times.
const failed = new Set();
// One capture at a time. The reader is watching something; a thumbnail has
// no business competing with it for bandwidth.
let queue = Promise.resolve();
App.posters.capture = function(video) {
const key = keyOf(video);
if (!key) return Promise.resolve('');
const have = App.posters.cached(video);
if (have) return Promise.resolve(have);
const running = inFlight.get(key);
if (running) return running;
if (failed.has(key)) return Promise.resolve('');
// A live stream has no quarter mark, and whatever it is showing now is
// not what it will be showing when the card is next looked at.
if (video.isLive || (video.meta && video.meta.isLive)) return Promise.resolve('');
const run = queue.then(() => captureOne(video)).then((src) => {
inFlight.delete(key);
if (!src) {
failed.add(key);
return '';
}
return store(key, src);
}, () => {
inFlight.delete(key);
failed.add(key);
return '';
});
inFlight.set(key, run);
queue = run.catch(() => '');
return run;
};
// ---- pictures kept before captures were checked ----------------------
// Decodes a kept picture and says whether there is one in it -- the same
// test a fresh capture passes, run on the stored JPEG rather than on a
// frame of video.
const hasPicture = function(src) {
return new Promise((resolve) => {
const probe = new Image();
probe.onload = function() {
const canvas = document.createElement('canvas');
canvas.width = probe.naturalWidth || 1;
canvas.height = probe.naturalHeight || 1;
const ctx = canvas.getContext('2d');
ctx.drawImage(probe, 0, 0);
try {
resolve(!isBlank(ctx, canvas.width, canvas.height));
} catch (err) {
// No way to tell, so leave it where it is.
resolve(true);
}
};
probe.onerror = () => resolve(false);
probe.src = src;
});
};
// One measurement per item per session, however many cards show it -- and
// all of them wait on that one and get the replacement. The same item is on
// screen in more than one place (the favorites bar and the grid behind it),
// so the first asker getting the picture and the rest being told the kept
// one stands would leave the black box up on every card but one.
const rechecks = new Map();
// A picture kept by a version of this that did not check its captures can
// be a frame of nothing -- pure black, and kept for the life of the item,
// since the only test it had to pass was being a few hundred bytes long.
// Each unchecked entry is measured once, after the card has already painted
// what there is, and one with nothing in it is dropped and taken again.
// Resolves with a replacement picture, or '' when the kept one stands.
App.posters.recheck = function(video) {
const key = keyOf(video);
if (!key) return Promise.resolve('');
const running = rechecks.get(key);
if (running) return running;
const entry = readIndex().find((item) => item.k === key);
if (!entry || entry.v) return Promise.resolve('');
const src = read(key);
if (!src) return Promise.resolve('');
const run = hasPicture(src).then((keep) => {
if (keep) {
// Nothing wrong with it. Said so, so no later session measures
// this one again.
entry.v = CHECKED;
writeIndex();
return '';
}
App.posters.forget(video);
return App.posters.capture(video);
}, () => '');
rechecks.set(key, run);
return run;
};
})();

View File

@@ -31,6 +31,8 @@ App.savedSearches = App.savedSearches || {};
const PROBE_PER_PAGE = 12;
// A group asks each of its channels, so each one asks for less.
const GROUP_PROBE_PER_PAGE = 4;
// So does a merge, of each of its searches.
const MERGED_PROBE_PER_PAGE = 6;
// Above this the badge says "9+" rather than a number.
const NEW_CAP = 9;
const DAY_MS = 24 * 60 * 60 * 1000;
@@ -48,9 +50,44 @@ App.savedSearches = App.savedSearches || {};
// ---- store ----------------------------------------------------------
// One search's worth of "where to look": a source, a channel on it, that
// channel's filters and the words. A plain saved search is one of these
// held flat on the entry; a merged one is a list of them.
const cleanMember = function(member, i) {
if (!member || typeof member !== 'object') return null;
if (!member.server || !member.channelId) return null;
return {
memberId: String(member.memberId || `m${i}`),
server: String(member.server),
channelId: String(member.channelId),
channelName: typeof member.channelName === 'string' ? member.channelName : '',
query: typeof member.query === 'string' ? member.query : '',
options: member.options && typeof member.options === 'object' ? member.options : {},
optionLabels: Array.isArray(member.optionLabels) ? member.optionLabels.map(String) : []
};
};
const clean = function(entry) {
if (!entry || typeof entry !== 'object') return null;
if (!entry.id || !entry.server || !entry.channelId) return null;
if (!entry.id) return null;
// A merge keeps its members and no flat source of its own: there is no
// single server or channel that would be the truth for it.
if (Array.isArray(entry.members) && entry.members.length) {
const members = entry.members.map(cleanMember).filter(Boolean);
if (!members.length) return null;
return {
id: String(entry.id),
name: typeof entry.name === 'string' && entry.name.trim() ? entry.name : 'Merged search',
members: members,
createdAt: Number(entry.createdAt) || 0,
seenIds: Array.isArray(entry.seenIds) ? entry.seenIds.map(String).slice(0, SEEN_KEEP) : [],
lastViewedAt: Number(entry.lastViewedAt) || 0,
lastCheckedAt: Number(entry.lastCheckedAt) || 0,
newCount: Math.max(0, Number(entry.newCount) || 0),
unreachable: !!entry.unreachable
};
}
if (!entry.server || !entry.channelId) return null;
return {
id: String(entry.id),
name: typeof entry.name === 'string' && entry.name.trim() ? entry.name : 'Saved search',
@@ -90,6 +127,27 @@ App.savedSearches = App.savedSearches || {};
return App.savedSearches.getAll().find((entry) => entry.id === id) || null;
};
App.savedSearches.isMerged = function(entry) {
return !!(entry && Array.isArray(entry.members) && entry.members.length);
};
// Every entry read as a list of places to look, so the code that asks a
// source something does not have to care which kind it was handed. A plain
// search is a merge of one.
App.savedSearches.membersOf = function(entry) {
if (!entry) return [];
if (App.savedSearches.isMerged(entry)) return entry.members;
return [{
memberId: 'm0',
server: entry.server,
channelId: entry.channelId,
channelName: entry.channelName,
query: entry.query || '',
options: entry.options || {},
optionLabels: entry.optionLabels || []
}];
};
// Every write goes through here: the list is read again first, so a check
// that lands while the reader is renaming or deleting can't write back a
// copy of the list from before they did it.
@@ -126,6 +184,25 @@ App.savedSearches = App.savedSearches || {};
App.savedSearches.current = function() {
const session = App.storage.getSession();
if (!session || !session.channel) return null;
// A merge is described by the searches it is made of, not by the
// session's server and channel -- there is no one of either. Its
// members are what the bookmark has to recognise it by, whichever of
// them happen to be toggled on just now.
if (session.channel.isMerged) {
return {
name: session.channel.name,
savedSearchId: session.channel.savedSearchId,
members: (session.channel.members || []).map((member) => ({
memberId: member.memberId,
server: member.server,
channelId: member.channelId,
channelName: member.channelName,
query: member.query || '',
options: member.options || {},
optionLabels: member.optionLabels || []
}))
};
}
const box = document.getElementById('search-input');
return {
server: session.server,
@@ -137,19 +214,29 @@ App.savedSearches = App.savedSearches || {};
};
};
// Two searches are the same search when they would send the same request.
App.savedSearches.signature = function(desc) {
if (!desc) return '';
const options = desc.options || {};
// One member's worth of identity: the request it would send.
const memberSignature = function(member) {
const options = member.options || {};
const parts = Object.keys(options).sort().map((key) => {
const value = options[key];
const flat = Array.isArray(value) ? value.slice().sort().join(',') : String(value);
return `${key}=${flat}`;
});
return [desc.server, desc.channelId, (desc.query || '').toLowerCase().trim()]
return [member.server, member.channelId, (member.query || '').toLowerCase().trim()]
.concat(parts).join('||');
};
// Two searches are the same search when they would send the same request.
// A merge is the same merge when it is made of the same searches -- in any
// order, since which one was ticked first is not part of what it is.
App.savedSearches.signature = function(desc) {
if (!desc) return '';
if (Array.isArray(desc.members) && desc.members.length) {
return `merged::${desc.members.map(memberSignature).sort().join('&&')}`;
}
return memberSignature(desc);
};
// The saved search the app is currently showing, if any -- what makes the
// bookmark in the search bar light up.
App.savedSearches.findMatching = function() {
@@ -162,10 +249,19 @@ App.savedSearches = App.savedSearches || {};
// What to call a search nobody has named: the words searched for, or the
// channel when there were none.
// What to call a merge nobody has named: the searches in it, which is the
// only thing it is. Two names fit on a row; past that it counts.
const mergedAutoName = function(members) {
const names = members.map((member) => member.channelName || member.channelId);
if (names.length <= 2) return names.join(' + ');
return `${names[0]} + ${names.length - 1} more`;
};
const autoName = function(desc) {
if (Array.isArray(desc.members) && desc.members.length) return mergedAutoName(desc.members);
if (desc.query) return desc.query;
const channel = desc.channelName || desc.channelId;
const narrowed = desc.optionLabels.slice(0, 2).join(' · ');
const narrowed = (desc.optionLabels || []).slice(0, 2).join(' · ');
return narrowed ? `${channel} · ${narrowed}` : channel;
};
@@ -213,6 +309,61 @@ App.savedSearches = App.savedSearches || {};
return entry;
};
// Merges several saved searches into one listing whose results come from
// all of them at once. The sources need nothing in common: that is the
// point, and it is why this cannot be a channel group (a group lives on
// one server and shares one query).
//
// The searches that went in are replaced by the merge. Merging a merge
// flattens it rather than nesting, so a merge is always one level deep and
// "the searches in it" means the same thing however it was assembled.
App.savedSearches.merge = function(ids, name) {
const wanted = Array.isArray(ids) ? ids : [];
const entries = App.savedSearches.getAll();
const chosen = wanted
.map((id) => entries.find((entry) => entry.id === id))
.filter(Boolean);
if (chosen.length < 2) return null;
// Deduped by what each member would ask for, so merging two searches
// that happen to be the same search does not ask twice.
const members = [];
const seenSigs = new Set();
chosen.forEach((entry) => {
App.savedSearches.membersOf(entry).forEach((member) => {
const sig = memberSignature(member);
if (seenSigs.has(sig)) return;
seenSigs.add(sig);
members.push(Object.assign({}, member, { memberId: `m${members.length}` }));
});
});
if (members.length < 2) return null;
const entry = clean({
id: `ss_${now().toString(36)}_${Math.random().toString(36).slice(2, 7)}`,
name: (name || '').trim() || mergedAutoName(members),
members: members,
createdAt: now(),
// No mark yet, so the first check sets one instead of counting a
// whole first page as news (see checkOne).
seenIds: [],
lastViewedAt: 0,
lastCheckedAt: 0,
newCount: 0
});
if (!entry) return null;
const chosenIds = new Set(chosen.map((item) => item.id));
const kept = entries.filter((item) => !chosenIds.has(item.id));
// Where the first of them was, rather than at the top: the reader put
// the list in the order they wanted.
const at = entries.findIndex((item) => chosenIds.has(item.id));
kept.splice(at < 0 ? 0 : at, 0, entry);
setAll(kept);
changed();
return entry;
};
// Opened, so whatever it shows now counts as seen and the badge goes out.
App.savedSearches.markViewed = function(id) {
const seen = visibleIds();
@@ -256,20 +407,34 @@ App.savedSearches = App.savedSearches || {};
// The ids on the first page of a saved search, as the server orders them.
// Not the grid's request: the grid's paging state belongs to what the
// reader is looking at, and a check must not disturb it.
const probeIds = async function(entry, signal) {
// One from each list in turn, so "the newest" means here what it means in
// the grid: the sources taken alternately rather than one exhausted first.
const interleave = function(lists) {
const out = [];
const longest = lists.reduce((max, ids) => Math.max(max, ids.length), 0);
for (let i = 0; i < longest; i++) {
lists.forEach((ids) => { if (ids[i]) out.push(ids[i]); });
}
return out;
};
// The first page of one member -- one source, one channel on it, its own
// filters and its own words. A member may itself name a channel group, in
// which case its own channels are interleaved first.
const probeMember = async function(member, perPage, signal) {
const serverEntry = App.storage.getServerEntries()
.find((candidate) => candidate.url === entry.server);
.find((candidate) => candidate.url === member.server);
const serverData = serverEntry ? serverEntry.data : null;
const channel = serverData ? App.session.resolveChannelById(serverData, entry.channelId) : null;
const channel = serverData ? App.session.resolveChannelById(serverData, member.channelId) : null;
if (!channel) throw new Error('channel is no longer listed');
const ask = async function(channelId, perPage, extra) {
const ask = async function(channelId, size, extra) {
const body = Object.assign({
channel: channelId,
query: entry.query || '',
query: member.query || '',
page: 1,
perPage: perPage,
server: entry.server
perPage: size,
server: member.server
}, extra || {});
const response = await fetch('/api/videos', {
method: 'POST',
@@ -284,26 +449,36 @@ App.savedSearches = App.savedSearches || {};
};
if (channel.isGroup) {
const selected = entry.options ? entry.options[App.session.GROUP_CHANNELS_OPTION_ID] : null;
const selected = member.options ? member.options[App.session.GROUP_CHANNELS_OPTION_ID] : null;
const enabled = (Array.isArray(selected) && selected.length ? selected : channel.channelIds)
.filter((id) => channel.channelIds.includes(id));
if (!enabled.length) return [];
const perChannel = await Promise.all(enabled.map((id) => ask(id, GROUP_PROBE_PER_PAGE)));
// Interleaved, so "the newest" means the same thing it does in the
// grid: one from each channel in turn.
const merged = [];
const longest = perChannel.reduce((max, ids) => Math.max(max, ids.length), 0);
for (let i = 0; i < longest; i++) {
perChannel.forEach((ids) => { if (ids[i]) merged.push(ids[i]); });
}
return merged;
return interleave(await Promise.all(
enabled.map((id) => ask(id, GROUP_PROBE_PER_PAGE))));
}
const extra = {};
Object.entries(entry.options || {}).forEach(([key, value]) => {
Object.entries(member.options || {}).forEach(([key, value]) => {
extra[key] = Array.isArray(value) ? value.join(', ') : value;
});
return ask(entry.channelId, PROBE_PER_PAGE, extra);
return ask(member.channelId, perPage, extra);
};
const probeIds = async function(entry, signal) {
const members = App.savedSearches.membersOf(entry);
if (members.length === 1) return probeMember(members[0], PROBE_PER_PAGE, signal);
// A merge asks every member and interleaves them, so the badge counts
// what the grid would actually have put in front of the reader. One
// member whose source has gone is not the end of the merge: the rest
// still answer, and only a merge where nothing answers is a failure.
const settled = await Promise.allSettled(
members.map((member) => probeMember(member, MERGED_PROBE_PER_PAGE, signal)));
const ok = settled.filter((r) => r.status === 'fulfilled').map((r) => r.value);
if (!ok.length) {
const first = settled.find((r) => r.status === 'rejected');
throw (first && first.reason) || new Error('no member could be reached');
}
return interleave(ok);
};
// How many of the videos the source lists now are ones the reader has not
@@ -441,6 +616,53 @@ App.savedSearches = App.savedSearches || {};
// ---- opening one -----------------------------------------------------
// Opens a merge: one listing fed by all of its searches at once.
//
// The search box is left empty and stays out of it. Every member carries
// its own words -- that is what makes merging two different searches worth
// doing -- so there is nothing for one box to say for all of them.
//
// Not saved as a channel preference either: a preference is stored per
// server against a channel that server lists, and a merge is neither.
// Reload restores it from the session instead (see ensureSessionFromCache).
const applyMerged = async function(entry) {
const channel = App.session.buildMergedChannel(entry);
if (!channel) return false;
const configured = new Set(App.storage.getServerEntries()
.filter((candidate) => candidate.data)
.map((candidate) => candidate.url));
const reachable = channel.members.filter((member) => configured.has(member.server));
if (!reachable.length) {
App.ui.showError(`${entry.name}: none of its sources are set up on this device any more.`);
patch(entry.id, { unreachable: true });
changed();
return false;
}
if (reachable.length < channel.members.length) {
const missing = channel.members
.filter((member) => !configured.has(member.server))
.map((member) => member.server);
App.ui.showError(`${entry.name}: showing it without ${missing.join(', ')} — not set up on this device.`);
}
App.storage.setSession({
server: reachable[0].server,
channel: channel,
options: App.session.buildDefaultOptions(channel)
});
App.videos.setSearchBox('');
if (App.favoritesView && App.favoritesView.isActive()) {
App.favoritesView.close({ silent: true });
}
App.ui.renderMenu();
App.videos.resetGrid();
await App.videos.loadVideos();
App.savedSearches.markViewed(entry.id);
return true;
};
// Puts the whole saved search back: source, channel, filters, query. The
// grid is reloaded here rather than left to the caller, because "applied"
// means the reader is looking at it -- which is also when it stops being
@@ -449,6 +671,8 @@ App.savedSearches = App.savedSearches || {};
const entry = App.savedSearches.get(id);
if (!entry) return false;
if (App.savedSearches.isMerged(entry)) return applyMerged(entry);
const serverEntry = App.storage.getServerEntries()
.find((candidate) => candidate.url === entry.server);
if (!serverEntry || !serverEntry.data) {

View File

@@ -15,7 +15,8 @@ App.state = {
feedMuted: true,
feedActiveIndex: -1,
feedActiveVideoId: null,
groupCursors: null
groupCursors: null,
mergedCursors: null
};
// Local storage keys used across modules.
@@ -26,7 +27,10 @@ App.constants = {
PREFERRED_QUALITY_KEY: 'preferredQuality',
FEED_END_BEHAVIOR_KEY: 'feedEndBehavior',
SEARCH_HISTORY_KEY: 'searchHistory',
SAVED_SEARCHES_KEY: 'savedSearches'
SAVED_SEARCHES_KEY: 'savedSearches',
// Thumbnails captured out of a video whose listed picture is gone:
// the index, plus one entry per picture under `<key>:<hash>`.
POSTER_CACHE_KEY: 'posterCache'
};
// Lazily injects hls.js the first time a stream actually needs it. Sessions

View File

@@ -164,6 +164,7 @@ App.session = App.session || {};
// channels as a toggleable multi-select, so the user can browse "All <group>"
// while disabling individual channels.
App.session.GROUP_CHANNELS_OPTION_ID = '__groupChannels';
App.session.MERGED_MEMBERS_OPTION_ID = '__mergedMembers';
// Options/session helpers that power channel selection and filters.
App.session.serializeOptions = function(options) {
@@ -233,6 +234,45 @@ App.session = App.session || {};
};
};
// Builds a pseudo-channel representing a merged saved search: several
// saved searches, each with its own source, channel, filters and words,
// browsed as one listing. A channel group does something similar but
// cannot express this -- every channel in a group belongs to the same
// server and shares one query, where a merge is held together by nothing
// except the reader having put it together (see fetchMergedBatch).
//
// The members are carried on the channel itself rather than looked up from
// a server's status, because no single server knows about them. That is
// also what lets a merged session survive a reload.
App.session.buildMergedChannel = function(entry) {
if (!entry || !Array.isArray(entry.members) || entry.members.length === 0) return null;
const members = entry.members.map((member, i) => Object.assign({}, member, {
// Stable within the merge, so a member toggle keeps meaning the
// same member even if two of them name the same channel.
memberId: member.memberId || `m${i}`
}));
const memberOptions = members.map((member) => ({
id: member.memberId,
title: member.channelName || member.channelId
}));
return {
id: `merged:${entry.id}`,
name: entry.name || 'Merged search',
isMerged: true,
savedSearchId: entry.id,
members: members,
// Same shape as a group's "Channels" filter, so the existing
// filter UI shows it without knowing what a merge is.
options: memberOptions.length > 0 ? [{
id: App.session.MERGED_MEMBERS_OPTION_ID,
title: 'Searches',
multiSelect: true,
selectAllDefault: true,
options: memberOptions
}] : []
};
};
// Resolves a stored channel id (which may reference a single channel or a
// "group:<id>" pseudo-channel) against a server's status payload.
App.session.resolveChannelById = function(serverData, channelId) {
@@ -350,6 +390,28 @@ App.session = App.session || {};
const serverKeys = (config.servers || []).map((serverObj) => Object.keys(serverObj)[0]);
const existingSession = App.storage.getSession();
// A merged search answers to no single server, so the check below --
// "does this server still list this channel" -- cannot speak for it.
// It is valid while the saved search it came from is still there and
// at least one member's source is still set up. Rebuilt from that
// saved search rather than trusted as stored, so a rename or an edited
// member is picked up instead of being frozen into the session.
if (existingSession && existingSession.channel && existingSession.channel.isMerged) {
const stored = App.savedSearches && typeof App.savedSearches.get === 'function'
? App.savedSearches.get(existingSession.channel.savedSearchId)
: null;
const channel = stored ? App.session.buildMergedChannel(stored) : existingSession.channel;
const members = (channel && channel.members) || [];
if (members.some((member) => serverKeys.includes(member.server))) {
App.storage.setSession({
server: members[0].server,
channel: channel,
options: App.session.hydrateOptions(channel, App.session.serializeOptions(existingSession.options))
});
return true;
}
}
// Leave a still-valid session untouched so we don't disturb the user's
// current server/channel selection on refresh.
if (existingSession && existingSession.channel && serverKeys.includes(existingSession.server)) {

View File

@@ -288,6 +288,11 @@ App.ui = App.ui || {};
if (otherBtn) otherBtn.classList.remove('active');
if (willOpen) {
// What the captured-thumbnail note says is a count, and captures
// happen while the drawer is shut, so it is read as it opens.
if (!isMenu && typeof App.ui.syncPosterCache === 'function') {
App.ui.syncPosterCache();
}
targetDrawer.classList.add('open');
if (targetBtn) targetBtn.classList.add('active');
overlay.classList.add('open');
@@ -506,7 +511,25 @@ App.ui = App.ui || {};
// Where the search points, in one line: the channel, what the filters
// narrow it to, and the words themselves.
const hostOf = function(url) {
try { return new URL(url).host; } catch (err) { return String(url || ''); }
};
const metaLine = function(entry) {
// A merge has no channel or words of its own -- what it is, is the
// searches in it, so those are what the row says. Each one names
// where it looks and which source it looks on, because two members
// differing only by source is the whole reason to merge.
if (App.savedSearches.isMerged(entry)) {
const shown = entry.members.slice(0, 3).map((member) => {
const where = member.channelName || member.channelId;
const words = member.query ? ` “${member.query}”` : '';
return `${where}${words} · ${hostOf(member.server)}`;
});
const rest = entry.members.length - shown.length;
const tail = rest > 0 ? ` + ${rest} more` : '';
return `${entry.members.length} searches · ${shown.join(' + ')}${tail}`;
}
const parts = [entry.channelName || entry.channelId];
if (entry.optionLabels && entry.optionLabels.length) {
parts.push(entry.optionLabels.slice(0, 3).join(' · '));
@@ -609,10 +632,27 @@ App.ui = App.ui || {};
return strip;
};
// Picking rows to merge. A merge is made here rather than from a row's
// own menu because it is about several rows at once -- and because
// "which ones" is the whole question.
let selecting = false;
let picked = new Set();
// Which shape a row is in. A row only has to be built again when it
// changes shape; otherwise the one on screen is updated where it
// stands (see render). Being picked is a shape too: the row swaps its
// count for a tick and its tools for nothing.
const modeOf = function(entry) {
if (selecting) return picked.has(entry.id) ? 'pick-on' : 'pick-off';
return (pending && pending.id === entry.id) ? pending.mode : 'plain';
};
const buildRow = function(entry, index) {
const row = document.createElement('div');
row.className = 'saved-row';
// Only a row that was not on screen a moment ago animates in.
row.className = 'saved-row is-new';
row.dataset.id = entry.id;
row.dataset.mode = modeOf(entry);
row.style.setProperty('--i', String(index));
if (entry.unreachable) row.classList.add('is-unreachable');
@@ -648,9 +688,30 @@ App.ui = App.ui || {};
text.appendChild(stamp);
main.appendChild(count);
main.appendChild(text);
main.addEventListener('click', () => choose(entry.id));
main.addEventListener('click', () => {
if (selecting) {
if (picked.has(entry.id)) picked.delete(entry.id);
else picked.add(entry.id);
render();
return;
}
choose(entry.id);
});
row.appendChild(main);
// While picking, a tap means "this one too" rather than "open
// this", so the row says so: a tick where the count was, and no
// rename or remove to catch the thumb on the way past.
if (selecting) {
row.classList.add('is-selecting');
row.classList.toggle('is-picked', picked.has(entry.id));
count.classList.remove('is-quiet');
count.classList.add('saved-pick');
count.textContent = picked.has(entry.id) ? '✓' : '';
count.removeAttribute('title');
return row;
}
if (pending && pending.id === entry.id && pending.mode === 'rename') {
const field = renameField(entry, row);
name.replaceWith(field);
@@ -674,6 +735,45 @@ App.ui = App.ui || {};
tools.appendChild(rename);
tools.appendChild(remove);
row.appendChild(tools);
// What updateRow writes to, so it doesn't have to search the row
// again on every check.
row._parts = { count: count, name: name, meta: meta, stamp: stamp,
rename: rename, remove: remove };
return row;
};
// The same row, told what changed. A check writes a new count and a new
// "checked" time on every saved search it asks about; rebuilding the
// list for that would play the whole list's entry animation again,
// drop the reader's scroll position and take the pointer out of
// whatever it was over. A background check has no business doing any
// of that, so nothing is replaced -- the few strings that differ are
// written in place.
const updateRow = function(row, entry, index) {
const parts = row._parts;
if (!parts) return row;
row.classList.remove('is-new');
row.style.setProperty('--i', String(index));
row.classList.toggle('is-unreachable', !!entry.unreachable);
const label = App.savedSearches.badgeLabel(entry.newCount);
const countText = label || '·';
if (parts.count.textContent !== countText) parts.count.textContent = countText;
parts.count.classList.toggle('is-quiet', !label);
if (label) parts.count.title = `${entry.newCount} new since you last opened this`;
else parts.count.removeAttribute('title');
if (parts.name.textContent !== entry.name) {
parts.name.textContent = entry.name;
parts.rename.title = `Rename ${entry.name}`;
parts.rename.setAttribute('aria-label', `Rename ${entry.name}`);
parts.remove.title = `Remove ${entry.name}`;
parts.remove.setAttribute('aria-label', `Remove ${entry.name}`);
}
const meta = metaLine(entry);
if (parts.meta.textContent !== meta) parts.meta.textContent = meta;
const stamp = stampLine(entry);
if (parts.stamp.textContent !== stamp) parts.stamp.textContent = stamp;
return row;
};
@@ -684,11 +784,32 @@ App.ui = App.ui || {};
const add = el('saved-add-btn');
const check = el('saved-check-btn');
const read = el('saved-read-btn');
const select = el('saved-select-btn');
const mergeBtn = el('saved-merge-btn');
if (!list) return;
const entries = App.savedSearches.getAll();
list.textContent = '';
entries.forEach((entry, index) => list.appendChild(buildRow(entry, index)));
const onScreen = new Map();
Array.from(list.children).forEach((row) => onScreen.set(row.dataset.id, row));
entries.forEach((entry, index) => {
const had = onScreen.get(entry.id);
onScreen.delete(entry.id);
// A row mid-rename or mid-confirm is a different row: it has
// its own field and buttons, so that one is built.
let row;
if (had && had.dataset.mode === modeOf(entry)) {
row = updateRow(had, entry, index);
} else {
row = buildRow(entry, index);
// A row that changed shape where it stood has not arrived
// either -- it is the same row, asking something.
if (had) row.classList.remove('is-new');
}
if (list.children[index] !== row) {
list.insertBefore(row, list.children[index] || null);
}
});
onScreen.forEach((row) => row.remove());
if (empty) empty.hidden = entries.length > 0;
if (note) {
@@ -709,13 +830,33 @@ App.ui = App.ui || {};
read.disabled = marking || !App.savedSearches.totalNew();
read.textContent = marking ? 'Marking…' : 'Mark all read';
}
if (select) {
// There is nothing to merge with fewer than two of them.
select.hidden = entries.length < 2;
select.textContent = selecting ? 'Cancel' : 'Merge…';
}
if (mergeBtn) {
mergeBtn.hidden = !selecting;
mergeBtn.disabled = picked.size < 2;
mergeBtn.textContent = picked.size < 2
? 'Pick two or more to merge'
: `Merge ${picked.size} selected`;
}
if (add) {
// While picking, the merge is the only action that means
// anything down here.
add.hidden = selecting;
const match = App.savedSearches.findMatching();
add.disabled = !!match || !App.savedSearches.current();
add.textContent = match ? `Already saved as “${match.name}”` : 'Save the current search';
}
};
const stopSelecting = function() {
selecting = false;
picked = new Set();
};
const choose = function(id) {
close();
App.savedSearches.apply(id);
@@ -725,6 +866,7 @@ App.ui = App.ui || {};
const box = panel();
if (!box) return;
pending = null;
stopSelecting();
render();
box.classList.add('open');
box.setAttribute('aria-hidden', 'false');
@@ -748,6 +890,7 @@ App.ui = App.ui || {};
box.setAttribute('aria-hidden', 'true');
}
pending = null;
stopSelecting();
const trigger = el('saved-searches-btn');
if (trigger) {
trigger.setAttribute('aria-expanded', 'false');
@@ -833,6 +976,32 @@ App.ui = App.ui || {};
});
}
const select = el('saved-select-btn');
if (select) {
select.addEventListener('click', () => {
if (selecting) stopSelecting();
else { selecting = true; picked = new Set(); pending = null; }
render();
});
}
const mergeBtn = el('saved-merge-btn');
if (mergeBtn) {
mergeBtn.addEventListener('click', () => {
// In the order they are listed, not the order they were
// ticked: the list is the reader's own arrangement.
const order = App.savedSearches.getAll()
.map((entry) => entry.id)
.filter((id) => picked.has(id));
const made = App.savedSearches.merge(order);
stopSelecting();
// Straight into renaming it: a merge's own name is the
// only thing about it nobody has said yet.
if (made) pending = { id: made.id, mode: 'rename' };
render();
});
}
const check = el('saved-check-btn');
if (check) {
check.addEventListener('click', () => {
@@ -1213,8 +1382,36 @@ App.ui = App.ui || {};
})();
// Settings + menu rendering.
// A merge carries its members' own words, and one box cannot stand for
// several (see fetchMergedBatch, which does not read it). So rather than
// leave a live-looking box that changes nothing, it is turned off and says
// why. Switching to any single channel turns it back on.
let searchPlaceholder = null;
App.ui.syncSearchForSession = function(session) {
const searchInput = document.getElementById('search-input');
if (!searchInput) return;
// Kept from the markup the first time through, so putting it back does
// not depend on remembering what it said.
if (searchPlaceholder === null) searchPlaceholder = searchInput.placeholder;
const merged = !!(session && session.channel && session.channel.isMerged);
searchInput.disabled = merged;
searchInput.classList.toggle('is-inert', merged);
if (merged) {
searchInput.value = '';
searchInput.placeholder = 'Each search in this merge keeps its own words';
} else {
searchInput.placeholder = searchPlaceholder;
}
const clearBtn = document.getElementById('search-clear-btn');
if (clearBtn && merged) {
clearBtn.classList.remove('is-visible');
clearBtn.disabled = true;
}
};
App.ui.renderMenu = function() {
const session = App.storage.getSession();
App.ui.syncSearchForSession(session);
const serverEntries = App.storage.getServerEntries();
const sourceSelect = document.getElementById('source-select');
const filtersContainer = document.getElementById('filters-container');
@@ -1629,6 +1826,56 @@ App.ui = App.ui || {};
});
};
// Settings -> Captured Thumbnails. A favorite whose listed picture is gone
// gets a frame out of the video instead, kept for good so no later session
// pays for it again (see App.posters). Kept for good is also the problem:
// a picture taken from the wrong moment, or from a version of this that
// kept frames with nothing in them, has no other way out. So the reader
// can throw the lot away and have them taken again.
const posterCacheNote = function() {
if (!App.posters || typeof App.posters.stats !== 'function') return '';
const stats = App.posters.stats();
if (!stats.count) {
return 'No pictures taken from videos yet. A favorite gets one when ' +
'its own thumbnail is gone for good.';
}
const plural = stats.count === 1 ? 'picture' : 'pictures';
const kb = Math.max(1, Math.round(stats.bytes / 1024));
return `${stats.count} ${plural} kept, ${kb} KB. Throwing them away has ` +
'each one taken again from the video the next time its card is shown.';
};
App.ui.syncPosterCache = function() {
const status = document.getElementById('clear-posters-status');
if (status) status.textContent = posterCacheNote();
};
App.ui.bindPosterCacheControls = function() {
const button = document.getElementById('clear-posters-btn');
if (!button) return;
App.ui.syncPosterCache();
button.addEventListener('click', () => {
const before = App.posters.stats().count;
App.posters.clear();
// The cards on screen are still showing what was just thrown away,
// so they go back to the start of the thumbnail ladder -- which is
// where a fresh capture comes from.
if (typeof App.videos.refreshThumbnails === 'function') {
App.videos.refreshThumbnails();
}
if (App.favorites && typeof App.favorites.renderBar === 'function') {
App.favorites.renderBar();
}
const status = document.getElementById('clear-posters-status');
if (status) {
status.textContent = before
? `Threw away ${before} ${before === 1 ? 'picture' : 'pictures'}. ` +
'They are being taken again.'
: posterCacheNote();
}
});
};
// Favorites bar header: the sort order (which applies to the bar and the
// favorites grid alike) and the toggle into that grid.
App.ui.bindFavoritesControls = function() {
@@ -1654,6 +1901,7 @@ App.ui = App.ui || {};
App.ui.bindGlobalHandlers = function() {
App.ui.bindBackupImport();
App.ui.bindPosterCacheControls();
App.ui.bindFavoritesControls();
window.toggleDrawer = App.ui.toggleDrawer;

View File

@@ -270,9 +270,24 @@ App.videos = App.videos || {};
// being asked.
const posterAsked = new Map(); // page URL -> promise of a replacement, or ''
// Last of all, once the listed picture is gone and the page has no
// better one to offer: a frame out of the video itself. Only for something
// the reader kept -- a capture costs a resolve and a stretch of video,
// which is not worth spending on a card that scrolls past once -- and the
// picture is cached, so an item pays for it once and no later session pays
// at all. See App.posters.
const capturePoster = function(img, token) {
const video = img._thumbVideo;
if (!video || !App.posters || !App.favorites || !App.favorites.has(video)) return;
App.posters.capture(video).then((src) => {
if (!src || img.dataset.thumbToken !== token) return;
showThumbnail(img, src, token);
});
};
const repairThumbnail = function(img, token) {
const page = img.dataset.thumbPage || '';
if (!page) return;
if (!page) { capturePoster(img, token); return; }
let pending = posterAsked.get(page);
if (!pending) {
pending = fetch(`/api/poster?url=${encodeURIComponent(page)}`)
@@ -282,7 +297,9 @@ App.videos = App.videos || {};
posterAsked.set(page, pending);
}
pending.then((replacement) => {
if (!replacement || img.dataset.thumbToken !== token) return;
if (img.dataset.thumbToken !== token) return;
// The page had nothing either, so there is one thing left to try.
if (!replacement) { capturePoster(img, token); return; }
// No second repair for this mount: the ladder below must not come
// back here and ask the same page for the same picture forever.
delete img.dataset.thumbPage;
@@ -525,15 +542,32 @@ App.videos = App.videos || {};
}
};
// Puts every thumbnail on the page back to the start of the ladder. For the
// settings action that throws the captured pictures away (see
// App.posters.clear): a card that is already mounted would otherwise keep
// showing the picture that was just discarded until something scrolled it
// out of the window and back.
App.videos.refreshThumbnails = function() {
document.querySelectorAll('img[data-thumb-token]').forEach((img) => {
const video = img._thumbVideo;
if (!video) return;
App.videos.attachThumbnail(img, video.thumb || '',
img.dataset.thumbPage || App.videos.sourcePageUrl(video), video);
});
};
// Points `img` at `url` by whichever route is known to work for its host,
// racing the two the first time that host is seen. `page` is optional: the
// item's own page, asked for a replacement if the picture turns out to be
// gone.
App.videos.attachThumbnail = function(img, url, page) {
App.videos.attachThumbnail = function(img, url, page, video) {
// An address that isn't one is the same thing as no thumbnail: the card
// keeps its placeholder rather than chasing it. See usableThumbUrl.
const directUrl = usableThumbUrl(url || (img && img.dataset.thumb) || '');
if (!img) return;
// The item this picture is of, kept for the capture at the end of the
// ladder (see capturePoster).
img._thumbVideo = video || null;
// Held back until there is an image to caption -- see showThumbnail. An
// item with no thumbnail keeps an empty alt: the card's own title sits
// directly beneath the box, so there is nothing for it to add.
@@ -541,7 +575,39 @@ App.videos = App.videos || {};
img.dataset.alt = img.alt;
img.alt = '';
}
if (!directUrl) return;
// Every attach is a new generation, so work started for a previous one
// stops being able to touch this element.
const token = String(++thumbSeq);
img.dataset.thumbToken = token;
if (page) img.dataset.thumbPage = page;
else delete img.dataset.thumbPage;
// A picture captured for this item earlier wins outright. The cache
// only ever holds one because every route to the listed picture had
// already failed, so going back to the wire for it would be a request
// that is known to fail, on every mount, for the rest of the item's
// life.
const captured = App.posters ? App.posters.cached(video) : '';
if (captured) {
showThumbnail(img, captured, token);
// A picture kept before captures were checked for having anything
// in them can be a frame of nothing -- a black box where the grey
// one used to be. Measured once, behind the picture that is already
// up, and replaced if that is what it turns out to be.
if (typeof App.posters.recheck === 'function') {
App.posters.recheck(video).then((fresh) => {
if (fresh && img.dataset.thumbToken === token) showThumbnail(img, fresh, token);
});
}
return;
}
if (!directUrl) {
// Nothing listed to fail, so there is nothing to walk a ladder
// down: a frame out of the video is the only picture this item is
// ever going to have.
capturePoster(img, token);
return;
}
// A cross-origin Referer is what most hotlink protection keys on, and an
// image needs none. Sending none is what lets the direct route work at
// all on a fair number of providers -- and the direct route is the one
@@ -551,12 +617,6 @@ App.videos = App.videos || {};
const proxyUrl = App.videos.buildImageProxyUrl(directUrl);
const host = imageHostOf(directUrl);
const route = imageRoutes.get(host);
// Every attach is a new generation, so work started for a previous one
// stops being able to touch this element.
const token = String(++thumbSeq);
img.dataset.thumbToken = token;
if (page) img.dataset.thumbPage = page;
else delete img.dataset.thumbPage;
if (route === IMAGE_PROXY) {
attachRetry(img, retryPlan(directUrl, proxyUrl, IMAGE_PROXY), token);
@@ -590,6 +650,7 @@ App.videos = App.videos || {};
img.dataset.thumbToken = String(++thumbSeq);
delete img.dataset.alt;
delete img.dataset.thumbPage;
img._thumbVideo = null;
detachRetry(img);
};
@@ -694,6 +755,94 @@ App.videos = App.videos || {};
};
};
// Fetches one page from every search in a merge and zips them together,
// the way a group does its channels -- except that a merge's members share
// nothing. Each one carries its own server, channel, filters and words, so
// each gets its own request built from itself rather than from the session.
//
// The search box is deliberately not consulted: a merge is several
// searches, each with its own words, and one box cannot speak for all of
// them (see applyMerged).
const fetchMergedBatch = async function(session, signal) {
const merged = session.channel;
const all = Array.isArray(merged.members) ? merged.members : [];
// The per-merge "Searches" multi-select, so a member can be switched
// off while browsing. Falls back to all of them when nothing is
// selected.
const selected = session.options ? session.options[App.session.MERGED_MEMBERS_OPTION_ID] : null;
const enabledIds = Array.isArray(selected) && selected.length > 0
? selected.map((opt) => opt.id)
: all.map((member) => member.memberId);
const members = all.filter((member) => enabledIds.includes(member.memberId));
const signature = members.map((member) => member.memberId).join(',');
if (!state.mergedCursors || state.mergedCursors.mergeId !== merged.id ||
state.mergedCursors.signature !== signature) {
state.mergedCursors = {
mergeId: merged.id,
signature: signature,
members: members.map((member) => ({ member: member, page: 1, hasNextPage: true }))
};
}
const active = state.mergedCursors.members.filter((cursor) => cursor.hasNextPage);
if (active.length === 0) {
return { items: [], hasNextPage: false };
}
const results = await Promise.all(active.map(async (cursor) => {
const member = cursor.member;
const page = claimPage(cursor);
const body = {
channel: member.channelId,
query: member.query || '',
page: page,
perPage: GROUP_CHANNEL_PAGE_SIZE,
server: member.server
};
Object.entries(member.options || {}).forEach(([key, value]) => {
body[key] = Array.isArray(value) ? value.join(', ') : value;
});
try {
const response = await fetch('/api/videos', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
signal: signal
});
const data = await response.json();
const items = data && Array.isArray(data.items) ? data.items : [];
cursor.hasNextPage = items.length > 0 &&
(data && data.pageInfo ? data.pageInfo.hasNextPage !== false : true);
return items;
} catch (err) {
returnPage(cursor, page);
// One source failing is not the merge failing: the others
// still have pages, and a merge that drops to one member is
// still worth reading. An aborted load is the exception --
// that is the grid being reset under us, and it has to
// propagate or the reset races the batch it cancelled.
if (err && err.name === 'AbortError') throw err;
cursor.hasNextPage = false;
return [];
}
}));
const interleaved = [];
const maxLen = results.reduce((max, items) => Math.max(max, items.length), 0);
for (let i = 0; i < maxLen; i++) {
results.forEach((items) => {
if (items[i]) interleaved.push(items[i]);
});
}
return {
items: interleaved,
hasNextPage: state.mergedCursors.members.some((cursor) => cursor.hasNextPage)
};
};
const fetchChannelBatch = async function(session, signal) {
const searchInput = document.getElementById('search-input');
const query = searchInput ? searchInput.value : "";
@@ -735,6 +884,7 @@ App.videos = App.videos || {};
};
const fetchNextBatch = function(session, signal) {
if (session.channel.isMerged) return fetchMergedBatch(session, signal);
return session.channel.isGroup
? fetchGroupBatch(session, signal)
: fetchChannelBatch(session, signal);
@@ -1126,7 +1276,7 @@ App.videos = App.videos || {};
// the same frame, so loading a thumbnail for it -- let alone racing one
// -- would be pure waste.
if (!(options && options.skipThumbnail)) {
App.videos.attachThumbnail(cardRefs(card).img, v.thumb, App.videos.sourcePageUrl(v));
App.videos.attachThumbnail(cardRefs(card).img, v.thumb, App.videos.sourcePageUrl(v), v);
}
return card;
};
@@ -1337,7 +1487,43 @@ App.videos = App.videos || {};
}
};
// Steps out of a merge onto one real channel: its first member whose
// source is still set up. Returns false when none of them is, in which
// case the merge is left alone -- there is nothing better to show.
App.videos.leaveMerge = function() {
const session = App.storage.getSession();
if (!session || !session.channel || !session.channel.isMerged) return false;
const entries = App.storage.getServerEntries();
for (const member of (session.channel.members || [])) {
const serverEntry = entries.find((candidate) => candidate.url === member.server);
const data = serverEntry ? serverEntry.data : null;
const channel = data ? App.session.resolveChannelById(data, member.channelId) : null;
if (!channel) continue;
const next = {
server: member.server,
channel: channel,
options: App.session.hydrateOptions(channel, member.options || {})
};
App.storage.setSession(next);
App.session.savePreference(next);
if (App.ui && typeof App.ui.renderMenu === 'function') App.ui.renderMenu();
return true;
}
return false;
};
App.videos.handleSearch = function(value) {
// Searching while a merge is open leaves it. A merge is held together
// by its members' own words (see fetchMergedBatch, which does not read
// the box), so there is nowhere in it to put new ones -- and a tag or
// an uploader tapped from a card would otherwise reload the same
// videos and look like nothing happened. The source and channel it
// lands on are the merge's first member, which is also the one the
// session was already pointed at.
const open = App.storage.getSession();
if (open && open.channel && open.channel.isMerged && typeof value === 'string' && value.trim()) {
App.videos.leaveMerge();
}
if (typeof value === 'string') {
App.videos.setSearchBox(value);
}
@@ -1367,6 +1553,7 @@ App.videos = App.videos || {};
state.renderedVideoIds.clear();
state.loadedVideos = [];
state.groupCursors = null;
state.mergedCursors = null;
App.virtualGrid.reset();
if (App.feed && typeof App.feed.reset === 'function') {
App.feed.reset();
@@ -1824,7 +2011,7 @@ App.videos = App.videos || {};
if (!card || !v) return;
unfilled.delete(i);
const img = cardRefs(card).img;
App.videos.attachThumbnail(img, v.thumb, App.videos.sourcePageUrl(v));
App.videos.attachThumbnail(img, v.thumb, App.videos.sourcePageUrl(v), v);
// Once the thumbnail loads, drop the 16:9 placeholder so it shows at
// its true aspect ratio, clear the shimmer, then correct the height.
if (img) {
@@ -2394,12 +2581,48 @@ App.videos = App.videos || {};
// same for every media URL served by a given host: we probe (and cache)
// once per host and let the player skip the proxy for any URL on a host
// that's been proven.
//
// Readable bytes alone don't settle it. Not every hotlink guard answers a
// foreign Referer with a 403: eporner's CDN answers one with a 302 to a
// one-minute "watch this at our site instead" clip, served 206 with
// `Access-Control-Allow-Origin: *`. fetch() follows that redirect without
// comment, so it reads as a win and plays as a different video. The probe
// therefore also checks it was handed *this* resource, by asking the proxy
// -- which carries the Referer the origin wants -- how long the media is.
// A direct route reporting some other length is a substitution, not a
// shortcut, and belongs on the proxy.
const DIRECT_PROBE_TIMEOUT_MS = 8000;
const directHostOf = (url) => {
try { return new URL(url).host; } catch (err) { return ''; }
};
// The total size of the media a response describes, or 0 when it won't say.
// An un-ranged GET states it in Content-Length, which CORS exposes to
// script without the origin having to opt in; a ranged one puts it after
// the slash in Content-Range.
const totalLengthOf = function(res) {
const match = /\/\s*(\d+)\s*$/.exec(res.headers.get('Content-Range') || '');
if (match) return parseInt(match[1], 10) || 0;
return parseInt(res.headers.get('Content-Length') || '0', 10) || 0;
};
// How long the proxied route says this media is, or 0 if it won't say. The
// proxy answers a HEAD with the whole resource's length (see stream_video's
// head_probe) and fetches with the headers the extractor attached, so this
// is the length of the media the viewer is actually meant to get.
const proxiedLength = async function(resolved, signal) {
try {
const streamUrl = App.videos.buildStreamUrlFromSource(resolved);
if (!streamUrl) return 0;
const res = await fetch(streamUrl, { method: 'HEAD', signal: signal });
if (!(res.ok || res.status === 206)) return 0;
return totalLengthOf(res);
} catch (err) {
return 0;
}
};
// host -> true (proven directly playable) | false (proven not). Absent
// means unknown/unprobed, in which case the proxy is used.
App.videos._directStatus = new Map();
@@ -2409,7 +2632,12 @@ App.videos = App.videos || {};
return App.videos._directStatus.get(directHostOf(url)) === true;
};
App.videos.probeDirect = function(url) {
// Takes a resolved source (as resolveStreamSources builds them) so the
// proxied cross-check can be addressed with the same headers playback uses.
// A bare URL string still works; it just can't be cross-checked as closely.
App.videos.probeDirect = function(source) {
const resolved = (typeof source === 'string') ? { url: source } : (source || {});
const url = resolved.url;
if (!url) return Promise.resolve(false);
const host = directHostOf(url);
if (!host) return Promise.resolve(false);
@@ -2421,7 +2649,17 @@ App.videos = App.videos || {};
}
const promise = (async () => {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), DIRECT_PROBE_TIMEOUT_MS);
// The cross-check gets its own controller: the direct fetch aborts
// as soon as it has its sample, and that must not cancel a HEAD the
// comparison still needs. One timer bounds them both.
const headController = new AbortController();
const timer = setTimeout(() => {
controller.abort();
headController.abort();
}, DIRECT_PROBE_TIMEOUT_MS);
// Asked alongside the direct fetch rather than after it: the probe
// runs beside live playback and shouldn't cost two serial trips.
const expectedLength = proxiedLength(resolved, headController.signal);
let ok = false;
let detail = '';
try {
@@ -2454,6 +2692,18 @@ App.videos = App.videos || {};
ok = true;
detail = `HTTP ${res.status}, headers only`;
}
if (ok) {
// The same media, or a stand-in? Only decided when both
// sides state a length: a chunked response or a playlist
// states none, and an unknown length is no evidence of
// substitution.
const direct = totalLengthOf(res);
const expected = await expectedLength;
if (direct && expected && direct !== expected) {
ok = false;
detail += `, ${direct} bytes but proxied is ${expected} -- substituted`;
}
}
controller.abort();
} catch (err) {
// A CORS refusal lands here as a TypeError with no status --
@@ -2463,6 +2713,10 @@ App.videos = App.videos || {};
detail = (err && err.name === 'AbortError') ? 'timeout' : (err && err.message) || 'blocked (CORS)';
} finally {
clearTimeout(timer);
// A refused direct fetch never waits on the comparison, so drop
// the HEAD rather than leaving it to run for an answer nobody
// will read. Harmless once it has already been awaited.
headController.abort();
}
App.videos._directStatus.set(host, ok);
directPending.delete(host);
@@ -2513,6 +2767,39 @@ App.videos = App.videos || {};
}
};
// Some providers (rule34video among them) hand the listing their media
// formats themselves, and yt-dlp can't extract their pages. A favorite keeps
// no formats, so for those /api/resolve comes back empty -- ask the
// providers again instead, the way the listing did: search the channel for
// the title and take the item that is this video.
const formatsFromProvider = async function(video) {
const idHead = String(video.id || '').split(':')[0];
const channelId = (idHead && idHead !== String(video.id) ? idHead : String(video.channel || '')).toLowerCase();
if (!channelId) return null;
const servers = (App.storage.getConfig().servers || []).map((entry) => Object.keys(entry)[0]);
const wanted = App.favorites && App.favorites.urlKey ? App.favorites.urlKey(video.url) : video.url;
const same = (item) => item && (item.id === video.id ||
(item.url && (App.favorites && App.favorites.urlKey ? App.favorites.urlKey(item.url) : item.url) === wanted));
for (const server of servers) {
try {
const response = await fetch('/api/videos', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ channel: channelId, query: video.title || '', page: 1, perPage: 24, server })
});
if (!response.ok) continue;
const data = await response.json();
const found = (data && Array.isArray(data.items) ? data.items : []).find(same);
if (found && hasFormats(found)) {
return { url: found.url, formats: found.formats, http_headers: found.http_headers || {}, isLive: !!found.isLive };
}
} catch (err) {
// Next server.
}
}
return null;
};
App.videos.ensureFormats = function(video) {
if (!video || typeof video !== 'object') return Promise.resolve(null);
if (hasFormats(video.meta)) return Promise.resolve(video.meta);
@@ -2539,13 +2826,14 @@ App.videos = App.videos || {};
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url: video.url })
});
if (!response.ok) return null;
const data = await response.json();
return hasFormats(data) ? data : null;
if (response.ok) {
const data = await response.json();
if (hasFormats(data)) return data;
}
} catch (err) {
// Best-effort: playback still works through the proxy.
return null;
}
return await formatsFromProvider(video);
})();
metaCache.set(cacheKey, promise);
}

BIN
tests/fixtures/dark.mp4 vendored Normal file

Binary file not shown.

BIN
tests/fixtures/wide.mp4 vendored Normal file

Binary file not shown.

View File

@@ -0,0 +1,77 @@
#!/usr/bin/env python3
"""A favorite whose site yt-dlp can't extract still gets its formats.
Providers like rule34video hand the listing their formats; a favorite keeps
none, and /api/resolve comes back empty for them. ensureFormats must then ask
the providers again and take the item that is this video.
backend/main.py &
.venv/bin/python tests/smoke_favorite_formats.py
"""
import json
import sys
from playwright.sync_api import sync_playwright
BASE = "http://127.0.0.1:5000/"
SERVER = "https://a.fixture.test"
FAV = {"key": "rule34video:4511669", "id": "rule34video:4511669",
"url": "https://rule34video.com/video/4511669/sindel-cassie-sonya/",
"title": "Sindel, Cassie, Sonya", "thumb": "", "channel": "Rule34video",
"uploader": "Nyl", "duration": 510, "isLive": False,
"favoriteDate": "2026-11-02T22:00:05.356Z"}
STATUS = {"id": "a", "name": "a", "online": True, "channels": [
{"id": "rule34video", "name": "Rule34video", "description": "x", "options": []}]}
FMT = [{"url": "https://rule34video.com/get_file/1/x/4511669_720p.mp4/", "height": 720, "ext": "mp4"}]
asked = []
def videos(route):
body = json.loads(route.request.post_data or "{}")
asked.append(body)
items = [{"id": "rule34video:1", "url": "https://rule34video.com/video/1/other/", "formats": []},
{"id": FAV["id"], "url": FAV["url"], "formats": FMT}]
route.fulfill(status=200, content_type="application/json", body=json.dumps({"items": items}))
def main():
failed = 0
with sync_playwright() as p:
b = p.chromium.launch(args=["--no-sandbox", "--disable-gpu"])
page = b.new_page()
page.route(f"{SERVER}/api/status", lambda r: r.fulfill(
status=200, content_type="application/json", body=json.dumps(STATUS)))
page.route("**/api/resolve", lambda r: r.fulfill(
status=200, content_type="application/json",
body='{"formats":[],"http_headers":{},"isLive":false,"url":null}'))
page.route("**/api/videos", videos)
page.goto(BASE, wait_until="domcontentloaded")
page.evaluate("""([s, f]) => {
localStorage.setItem('config', JSON.stringify({servers: [{[s]: {}}]}));
localStorage.setItem('favorites', JSON.stringify([f]));
}""", [SERVER, FAV])
page.goto(BASE, wait_until="load")
meta = page.evaluate("""async (f) => {
const v = Object.assign({}, f);
const m = await App.videos.ensureFormats(v);
return { m, attached: !!v.meta };
}""", FAV)
ok = bool(meta["m"]) and meta["m"]["formats"] == FMT and meta["attached"]
print(f" [{'PASS' if ok else 'FAIL'}] favorite gets the provider's formats {meta}")
failed += 0 if ok else 1
ok = bool(asked) and asked[0]["channel"] == "rule34video" and asked[0]["server"] == SERVER
print(f" [{'PASS' if ok else 'FAIL'}] asked the right channel and server {asked[:1]}")
failed += 0 if ok else 1
saved = page.evaluate("""(item) => {
localStorage.setItem('favorites', '[]');
App.favorites.toggle(Object.assign({}, item, { formats: [{ url: 'https://x/y.mp4', height: 720 }] }));
return App.favorites.getAll()[0];
}""", FAV)
ok = bool(saved) and bool(saved.get("meta")) and saved["meta"]["formats"][0]["url"] == "https://x/y.mp4"
print(f" [{'PASS' if ok else 'FAIL'}] a saved favorite keeps its formats")
failed += 0 if ok else 1
b.close()
return 1 if failed else 0
if __name__ == "__main__":
sys.exit(main())

248
tests/smoke_feed_swipe.py Normal file
View File

@@ -0,0 +1,248 @@
#!/usr/bin/env python3
"""A swipe counts even when the viewport just moved.
Run against a locally running backend:
backend/main.py &
.venv/bin/python tests/smoke_feed_swipe.py
Both the status and the listing are fixtures, so the reel is long enough to
swipe through and the test never waits on a real source.
A viewport change opens a realign window in which the feed holds the reader's
video in place, because the scroll-snap container re-anchors on positions
measured against the old height and must not be allowed to flip the video (see
smoke_rotate). The window used to revert *every* scroll it saw, and on a phone
that is the worst possible moment to do it: the chrome sliding out of the way
is caused by the swipe, so the window was open exactly when the reader was
swiping. The reel moved under the finger -- far enough to show the next video
coming in at the bottom -- and was then pulled back to where it started.
So the two halves have to be told apart, and the test asks for both: a scroll
with real input behind it is the reader's and is kept, a bare re-anchor is not
and is still undone.
"""
import json
import os
import re
import sys
from playwright.sync_api import sync_playwright
CLIP = os.path.join(os.path.dirname(__file__), "fixtures", "clip.mp4")
BASE = "http://127.0.0.1:5000/"
SERVER = "https://fixture.test"
CHANNEL = "clips"
PORTRAIT = {"width": 430, "height": 930}
# Chrome sliding away: height only, so this is not a turn -- it is the resize
# that used to eat the swipe that caused it.
SHORTER = {"width": 430, "height": 860}
# Near enough to square that a little height crosses it. Reading the `w > h`
# flip on its own called that a rotation, pinned the slides and turned
# snapping off for about a second.
SQUARE = {"width": 800, "height": 810}
SQUARE_SHORT = {"width": 800, "height": 790}
STATUS = {
"online": True,
"channels": [{
"id": CHANNEL,
"name": "Fixture Clips",
"description": "A channel that only exists for this test.",
"options": [],
}],
}
SEED = """([server, channel]) => {
localStorage.setItem('config', JSON.stringify({ servers: [{ [server]: {} }] }));
localStorage.setItem('preferences', JSON.stringify({ [server]: { channelId: channel } }));
localStorage.removeItem('session');
}"""
WHERE = """() => {
const scroller = document.getElementById('feed-scroll');
const root = document.getElementById('feed-view');
const active = document.querySelector('.feed-slide.is-active');
const h = scroller.clientHeight;
return {
index: App.state.feedActiveIndex,
videoId: App.state.feedActiveVideoId,
scrollTop: Math.round(scroller.scrollTop),
expected: App.state.feedActiveIndex * h,
height: h,
activeTop: active ? Math.round(active.getBoundingClientRect().top) : None_,
turning: root.classList.contains('is-turning'),
snap: getComputedStyle(scroller).scrollSnapType,
offHeight: Array.from(document.querySelectorAll('.feed-slide'))
.map((s) => s.getBoundingClientRect().height)
.filter((sh) => Math.abs(sh - h) > 1),
};
}""".replace("None_", "null")
GO = """(step) => {
const scroller = document.getElementById('feed-scroll');
scroller.scrollTop = step * scroller.clientHeight;
return App.state.feedActiveIndex;
}"""
# The container's own re-anchor: a scroll position and nothing behind it.
RESNAP = """(top) => {
document.getElementById('feed-scroll').scrollTop = top;
}"""
# A swipe, and the chrome reporting itself before the feed has caught up.
# setActive runs from onScroll's next frame, so doing both in one block --
# no frame boundary in between -- leaves feedActiveVideoId still naming the
# video being left, which is the state a phone resize routinely lands in.
# onResize then re-anchors synchronously, with no scroll event for the
# input guard in onScroll to see.
SWIPE_THEN_RESIZE = """() => {
const s = document.getElementById('feed-scroll');
const before = App.state.feedActiveIndex;
s.dispatchEvent(new WheelEvent('wheel', { deltaY: 120, bubbles: true }));
s.scrollTop = s.scrollTop + s.clientHeight;
const moved = s.scrollTop;
window.dispatchEvent(new Event('resize'));
return { before: before, moved: moved, after: s.scrollTop, h: s.clientHeight };
}"""
class Checks:
def __init__(self):
self.failed = 0
def ok(self, label, condition, detail=""):
mark = "PASS" if condition else "FAIL"
if not condition:
self.failed += 1
print(f" [{mark}] {label}" + (f" -- {detail}" if detail and not condition else ""))
def items():
return [{"id": f"vid-{i}", "title": f"clip {i}",
"url": f"https://fixture.test/v{i}.mp4", "thumb": "", "channel": CHANNEL,
"duration": 30, "isLive": False, "tags": []} for i in range(36)]
def swipe_to(page, step):
for n in range(1, step + 1):
page.evaluate(GO, n)
page.wait_for_timeout(250)
page.wait_for_timeout(400)
def main():
c = Checks()
with sync_playwright() as p:
browser = p.chromium.launch(args=[
"--no-sandbox", "--disable-dev-shm-usage", "--disable-gpu",
"--autoplay-policy=no-user-gesture-required",
"--renderer-process-limit=1", "--js-flags=--max-old-space-size=512",
])
page = browser.new_page(viewport=PORTRAIT)
crashes = []
page.on("pageerror", lambda e: crashes.append(str(e)))
page.route("**/api/status", lambda route: route.fulfill(
status=200, content_type="application/json", body=json.dumps(STATUS)))
page.route("**/api/videos", lambda route: route.fulfill(
status=200, content_type="application/json",
body=json.dumps({"items": items(), "pageInfo": {"hasNextPage": False}})))
page.route(re.compile(r"\.mp4"), lambda route: route.fulfill(
status=200, content_type="video/mp4", body=open(CLIP, "rb").read()))
page.goto(BASE, wait_until="domcontentloaded")
page.evaluate(SEED, [SERVER, CHANNEL])
page.goto(BASE, wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
page.wait_for_timeout(500)
page.evaluate("() => App.feed.open()")
page.wait_for_selector(".feed-slide", timeout=20000)
page.wait_for_timeout(800)
print("\nseven videos into the reel")
swipe_to(page, 7)
start = page.evaluate(WHERE)
c.ok("that is where the reel is", start["index"] == 7, str(start))
print("\na swipe while the chrome is still sliding away")
page.mouse.move(215, 500)
page.set_viewport_size(SHORTER)
page.wait_for_timeout(50) # well inside the realign window
page.mouse.wheel(0, SHORTER["height"])
page.wait_for_timeout(2200)
swiped = page.evaluate(WHERE)
c.ok("the swipe moved the reel on",
swiped["index"] == start["index"] + 1,
f"step {start['index']} -> {swiped['index']}")
c.ok("onto the next video",
swiped["videoId"] != start["videoId"],
f"{start['videoId']} -> {swiped['videoId']}")
c.ok("and it is sitting on that step, not between two",
abs(swiped["scrollTop"] - swiped["expected"]) <= 1, str(swiped))
c.ok("so no second video shows at the bottom of the screen",
swiped["activeTop"] == 0, str(swiped["activeTop"]))
print("\na bare re-anchor in the same window is still undone")
page.wait_for_timeout(900) # let the input grace lapse
held = page.evaluate(WHERE)
page.set_viewport_size(PORTRAIT)
page.wait_for_timeout(50)
# A position measured against the height from before the resize, which
# is what the container itself hands back mid-rotation.
page.evaluate(RESNAP, held["index"] * held["height"])
page.wait_for_timeout(1800)
resnapped = page.evaluate(WHERE)
c.ok("the same video is still playing",
resnapped["videoId"] == held["videoId"],
f"{held['videoId']} -> {resnapped['videoId']}")
c.ok("on the same step",
resnapped["index"] == held["index"],
f"{held['index']} -> {resnapped['index']}")
c.ok("re-anchored onto its own snap point",
abs(resnapped["scrollTop"] - resnapped["expected"]) <= 1, str(resnapped))
print("\nthe chrome reporting itself before the feed has caught up")
page.wait_for_timeout(900)
pre = page.evaluate(WHERE)
got = page.evaluate(SWIPE_THEN_RESIZE)
page.wait_for_timeout(2200)
kept = page.evaluate(WHERE)
print(f" (swiped to {got['moved']}px; the resize left it at {got['after']}px)")
c.ok("the resize did not haul the reel back to the video being left",
got["after"] >= got["moved"] - 1,
f"{got['moved']} -> {got['after']}")
c.ok("the swipe still landed on the next step",
kept["index"] == pre["index"] + 1,
f"step {pre['index']} -> {kept['index']}")
c.ok("and on that step's snap point",
abs(kept["scrollTop"] - kept["expected"]) <= 1, str(kept))
print("\na little height on a near-square viewport is not a rotation")
page.set_viewport_size(SQUARE)
page.wait_for_timeout(1800)
page.set_viewport_size(SQUARE_SHORT)
page.wait_for_timeout(120) # mid-window, when the pin would be on
pinned = page.evaluate(WHERE)
c.ok("the slides are not pinned for it", not pinned["turning"], str(pinned["turning"]))
c.ok("so snapping stays on", pinned["snap"] != "none", str(pinned["snap"]))
page.wait_for_timeout(1800)
settled = page.evaluate(WHERE)
c.ok("every slide is exactly a screenful",
settled["offHeight"] == [], str(settled["offHeight"]))
c.ok("and the reel rests on its snap point",
abs(settled["scrollTop"] - settled["expected"]) <= 1, str(settled))
c.ok("still on the same video",
settled["videoId"] == kept["videoId"],
f"{kept['videoId']} -> {settled['videoId']}")
c.ok("nothing threw along the way", not crashes, str(crashes[:2]))
browser.close()
print(f"\n{'FAILED' if c.failed else 'OK'}: {c.failed} check(s) failed")
return 1 if c.failed else 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,263 @@
#!/usr/bin/env python3
"""Merging saved searches from different providers into one listing.
Run against a locally running backend:
backend/main.py &
.venv/bin/python tests/smoke_merge_searches.py
Two fixture providers, one saved search on each, merged into one. What the
merge has to do that a channel group cannot: a group's channels all live on
one server and share one query, where these two share nothing at all. So the
load-bearing check is that the grid comes back holding videos from *both*
providers, taken alternately rather than one provider's page and then the
other's.
Both providers are fixtures, so "interleaved" is something the test can read
off the ids rather than infer: provider A answers a-0, a-1 ... and B answers
b-0, b-1 ...
"""
import json
import sys
from playwright.sync_api import sync_playwright
BASE = "http://127.0.0.1:5000/"
SERVER_A = "https://a.fixture.test"
SERVER_B = "https://b.fixture.test"
CHANNEL_A = "alpha"
CHANNEL_B = "beta"
# fetchMergedBatch asks each member for this many (GROUP_CHANNEL_PAGE_SIZE).
PER_MEMBER = 4
def status(name, channel_id, channel_name):
return {
"id": name,
"name": name,
"online": True,
"channels": [{
"id": channel_id,
"name": channel_name,
"description": "A channel that only exists for this test.",
"options": [],
}],
}
STATUS_A = status("alpha-co", CHANNEL_A, "Alpha Tube")
STATUS_B = status("beta-co", CHANNEL_B, "Beta Tube")
# Two saved searches, one per provider, seeded straight into the store -- the
# saving of a search is smoke_saved_searches' business, not this test's.
SEED = """([a, b, ca, cb]) => {
localStorage.setItem('config', JSON.stringify({
servers: [{ [a]: {} }, { [b]: {} }]
}));
localStorage.removeItem('session');
localStorage.setItem('favorites', JSON.stringify([]));
localStorage.setItem('savedSearches', JSON.stringify([
{ id: 'ss_a', name: 'Alpha pick', server: a, channelId: ca,
channelName: 'Alpha Tube', query: 'reef', options: {}, optionLabels: [],
createdAt: 1, seenIds: [], lastViewedAt: 1, lastCheckedAt: 1, newCount: 0 },
{ id: 'ss_b', name: 'Beta pick', server: b, channelId: cb,
channelName: 'Beta Tube', query: 'ridge', options: {}, optionLabels: [],
createdAt: 2, seenIds: [], lastViewedAt: 2, lastCheckedAt: 2, newCount: 0 }
]));
}"""
GRID_IDS = """() => (App.state.loadedVideos || []).map(v => v.id)"""
SESSION = """() => {
const s = App.storage.getSession();
return {
merged: !!(s && s.channel && s.channel.isMerged),
channelId: s && s.channel ? s.channel.id : null,
members: s && s.channel && s.channel.members
? s.channel.members.map(m => m.server + '|' + m.channelId + '|' + m.query)
: null,
name: s && s.channel ? s.channel.name : null
};
}"""
STORE = """() => App.savedSearches.getAll().map(e => ({
id: e.id, name: e.name,
members: e.members ? e.members.map(m => m.server) : null
}))"""
class Checks:
def __init__(self):
self.failed = 0
def ok(self, label, condition, detail=""):
mark = "PASS" if condition else "FAIL"
if not condition:
self.failed += 1
print(f" [{mark}] {label}" + (f" -- {detail}" if detail and not condition else ""))
def install_routes(page, seen):
page.route(f"{SERVER_A}/api/status", lambda route: route.fulfill(
status=200, content_type="application/json", body=json.dumps(STATUS_A)))
page.route(f"{SERVER_B}/api/status", lambda route: route.fulfill(
status=200, content_type="application/json", body=json.dumps(STATUS_B)))
def videos(route):
try:
body = json.loads(route.request.post_data or "{}")
except ValueError:
body = {}
server = body.get("server") or ""
page_no = int(body.get("page") or 1)
per = int(body.get("perPage") or 12)
tag = "a" if server == SERVER_A else ("b" if server == SERVER_B else "?")
seen.append({"server": server, "channel": body.get("channel"),
"query": body.get("query"), "page": page_no, "perPage": per})
start = (page_no - 1) * per
items = [{
"id": f"{tag}-{start + i}",
"title": f"{tag} clip {start + i}",
"url": f"{server}/v{start + i}",
"thumb": "", "channel": body.get("channel"),
"duration": 30, "isLive": False, "tags": [],
} for i in range(per)]
route.fulfill(status=200, content_type="application/json",
body=json.dumps({"items": items, "pageInfo": {"hasNextPage": True}}))
page.route("**/api/videos", videos)
def main():
c = Checks()
with sync_playwright() as p:
browser = p.chromium.launch(args=[
"--no-sandbox", "--disable-dev-shm-usage", "--disable-gpu",
"--renderer-process-limit=1", "--js-flags=--max-old-space-size=512",
])
page = browser.new_page(viewport={"width": 1280, "height": 900})
crashes = []
page.on("pageerror", lambda e: crashes.append(str(e)))
seen = []
install_routes(page, seen)
page.goto(BASE, wait_until="domcontentloaded")
page.evaluate(SEED, [SERVER_A, SERVER_B, CHANNEL_A, CHANNEL_B])
page.goto(BASE, wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
page.wait_for_timeout(800)
print("\nmerging the two")
page.click("#saved-searches-btn")
page.wait_for_selector("#saved-panel.open", timeout=10000)
rows = page.eval_on_selector_all(".saved-row", "els => els.length")
c.ok("both saved searches are listed", rows == 2, str(rows))
page.click("#saved-select-btn")
page.wait_for_timeout(200)
page.click('.saved-row[data-id="ss_a"] .saved-row-main')
page.click('.saved-row[data-id="ss_b"] .saved-row-main')
page.wait_for_timeout(200)
label = page.eval_on_selector("#saved-merge-btn", "el => el.textContent")
c.ok("the merge action counts what is picked",
"2" in label, label)
page.click("#saved-merge-btn")
page.wait_for_timeout(400)
# The new merge opens straight into its rename field; Enter keeps the
# name it was given.
page.keyboard.press("Enter")
page.wait_for_timeout(300)
store = page.evaluate(STORE)
c.ok("the two originals were replaced by one merge", len(store) == 1, str(store))
c.ok("and it holds a member from each provider",
len(store) == 1 and sorted(store[0]["members"] or []) == sorted([SERVER_A, SERVER_B]),
str(store))
print("\nopening it")
seen.clear()
page.click(f'.saved-row[data-id="{store[0]["id"]}"] .saved-row-main')
page.wait_for_timeout(2500)
session = page.evaluate(SESSION)
c.ok("the session is a merge", session["merged"], str(session))
c.ok("carrying both members, each with its own words",
sorted(session["members"] or []) == sorted([
f"{SERVER_A}|{CHANNEL_A}|reef", f"{SERVER_B}|{CHANNEL_B}|ridge"]),
str(session["members"]))
asked = {(r["server"], r["query"]) for r in seen}
c.ok("both providers were asked, each for its own search",
asked == {(SERVER_A, "reef"), (SERVER_B, "ridge")}, str(asked))
ids = page.evaluate(GRID_IDS)
kinds = [i.split("-")[0] for i in ids]
c.ok("the grid holds videos from both providers",
"a" in kinds and "b" in kinds, str(ids[:10]))
# The point of the exercise: alternated, not one provider then the other.
head = kinds[:2 * PER_MEMBER]
c.ok("taken alternately rather than one provider at a time",
head == ["a", "b"] * PER_MEMBER, str(head))
print("\nthe search box stays out of it")
box = page.evaluate("""() => {
const el = document.getElementById('search-input');
return { disabled: el.disabled, value: el.value, placeholder: el.placeholder };
}""")
c.ok("the box is off while a merge is open", box["disabled"], str(box))
c.ok("and says why", "own words" in box["placeholder"], str(box["placeholder"]))
print("\nit survives a reload")
page.reload(wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
page.wait_for_timeout(2000)
again = page.evaluate(SESSION)
c.ok("still a merge after reloading", again["merged"], str(again))
c.ok("with the same members",
sorted(again["members"] or []) == sorted(session["members"] or []),
str(again["members"]))
reloaded = [i.split("-")[0] for i in page.evaluate(GRID_IDS)]
c.ok("and still fed by both providers",
"a" in reloaded and "b" in reloaded, str(reloaded[:10]))
print("\nturning one member off")
page.evaluate("""(host) => {
const s = App.storage.getSession();
const keep = s.channel.members.filter(m => m.server.indexOf(host) !== -1);
s.options = { __mergedMembers: keep.map(m => ({ id: m.memberId, title: m.channelName })) };
App.storage.setSession(s);
}""", "a.fixture")
page.evaluate("() => { App.videos.resetGrid(); return App.videos.loadVideos(); }")
page.wait_for_timeout(2000)
only = [i.split("-")[0] for i in page.evaluate(GRID_IDS)]
c.ok("only the member left on is asked",
only and set(only) == {"a"}, str(only[:10]))
print("\nsearching from inside a merge steps out of it")
# A tag or an uploader tapped on a card comes through handleSearch.
# There is nowhere in a merge to put new words, so rather than reload
# the same videos and look broken, it lands on one real channel.
page.evaluate("() => App.videos.handleSearch('anemone')")
page.wait_for_timeout(2000)
after = page.evaluate(SESSION)
c.ok("the session is no longer a merge", not after["merged"], str(after))
c.ok("it landed on a member's own channel",
after["channelId"] in (CHANNEL_A, CHANNEL_B), str(after["channelId"]))
box2 = page.evaluate("""() => {
const el = document.getElementById('search-input');
return { disabled: el.disabled, value: el.value };
}""")
c.ok("and the box is usable again", not box2["disabled"], str(box2))
c.ok("holding what was searched for", box2["value"] == "anemone", str(box2))
searched = [r for r in seen if r["query"] == "anemone"]
c.ok("which is what the provider was asked for", bool(searched), str(seen[-3:]))
c.ok("nothing threw along the way", not crashes, str(crashes[:2]))
browser.close()
print(f"\n{'FAILED' if c.failed else 'OK'}: {c.failed} check(s) failed")
return 1 if c.failed else 0
if __name__ == "__main__":
sys.exit(main())

292
tests/smoke_poster_black.py Normal file
View File

@@ -0,0 +1,292 @@
#!/usr/bin/env python3
"""A captured thumbnail has to be a picture of something.
Run against a locally running backend:
backend/main.py &
.venv/bin/python tests/smoke_poster_black.py
Two ways a card ends up a black box rather than a picture, both of which the
capture used to keep for the life of the item -- the only test a frame had to
pass was being a few hundred bytes long:
* the frame at the mark is black. A video that opens on a fade, or whose
length nobody will say (so there is no quarter mark to seek to and the
opening frame is all there is), hands over exactly that.
* the element decoded nothing. Drawing one that has not presented a frame
gives an untouched canvas, which is a pure black JPEG about 1.6 KB long.
The fixture is black for its first three and a half seconds -- past the quarter
mark a capture aims for -- and flat orange after, so the picture on the card
says whether anything looked at what it was keeping.
"""
import json
import os
import re
import sys
from playwright.sync_api import sync_playwright
CLIP = os.path.join(os.path.dirname(__file__), "fixtures", "dark.mp4")
BASE = "http://127.0.0.1:5000/"
SERVER = "https://fixture.test"
CHANNEL = "clips"
CACHE_KEY = "posterCache"
STATUS = {
"online": True,
"channels": [{
"id": CHANNEL,
"name": "Fixture Clips",
"description": "A channel that only exists for this test.",
"options": [],
}],
}
# Saved with the thumbnail it had at the time, and with nothing known about how
# long it is -- which is the state the item this test comes from is in.
SEED = """([server, channel]) => {
localStorage.setItem('config', JSON.stringify({ servers: [{ [server]: {} }] }));
localStorage.setItem('preferences', JSON.stringify({ [server]: { channelId: channel } }));
localStorage.removeItem('session');
localStorage.removeItem('posterCache');
Object.keys(localStorage).forEach((k) => {
if (k.indexOf('posterCache:') === 0) localStorage.removeItem(k);
});
localStorage.setItem('favorites', JSON.stringify([{
key: 'vid-0',
id: 'vid-0',
url: 'https://fixture.test/v0.mp4',
title: 'kept clip 0',
thumb: 'https://pictures.fixture.test/gone/0.jpg',
channel: channel,
uploader: '',
duration: 0,
isLive: false,
favoriteDate: new Date().toISOString()
}]));
}"""
CAPTURED = """(id) => {
const card = document.querySelector(`.video-card[data-video-id="${id}"]`);
if (!card) return false;
const img = card.querySelector('.video-thumb img');
return (img.getAttribute('src') || '').indexOf('data:image/jpeg') === 0;
}"""
# What the card is showing, measured: the brightest sample in it and the
# average, which is all it takes to tell a picture from a black box.
LIGHT = """(id) => new Promise((resolve) => {
const card = document.querySelector(`.video-card[data-video-id="${id}"]`);
const img = card ? card.querySelector('.video-thumb img') : null;
const src = img ? (img.getAttribute('src') || '') : '';
if (!src) { resolve(null); return; }
const probe = new Image();
probe.onload = () => {
const canvas = document.createElement('canvas');
canvas.width = probe.naturalWidth;
canvas.height = probe.naturalHeight;
const ctx = canvas.getContext('2d');
ctx.drawImage(probe, 0, 0);
const px = ctx.getImageData(0, 0, canvas.width, canvas.height).data;
let sum = 0, brightest = 0;
for (let i = 0; i < px.length; i += 4) {
const value = Math.max(px[i], px[i + 1], px[i + 2]);
sum += value;
if (value > brightest) brightest = value;
}
resolve({ brightest, mean: Math.round(sum / (px.length / 4)),
bytes: src.length, w: canvas.width, h: canvas.height });
};
probe.onerror = () => resolve(null);
probe.src = src;
})"""
CACHE = """() => JSON.parse(localStorage.getItem('posterCache') || '[]')"""
FORGET = """() => {
localStorage.removeItem('posterCache');
Object.keys(localStorage).forEach((k) => {
if (k.indexOf('posterCache:') === 0) localStorage.removeItem(k);
});
}"""
# The item this test comes from has no duration: the extractor reports none and
# the favorite kept a zero. Where the browser can't supply one either, there is
# no quarter mark to seek to -- and the frame the element decodes on its own is
# the opening one, which is the frame this clip is black for.
NO_DURATION = """
Object.defineProperty(HTMLVideoElement.prototype, 'duration', {
configurable: true,
get: function() { return NaN; }
});
"""
# What the version before this kept: an untouched canvas, JPEG'd. Written over
# the picture in the cache, with the "this was looked at" marker taken off the
# index entry, which is exactly the state those sessions left behind.
POISON = """() => {
const canvas = document.createElement('canvas');
canvas.width = 360;
canvas.height = 203;
const black = canvas.toDataURL('image/jpeg', 0.72);
const index = JSON.parse(localStorage.getItem('posterCache') || '[]');
index.forEach((entry) => {
delete entry.v;
entry.n = black.length;
localStorage.setItem(`posterCache:${entry.k}`, black);
});
localStorage.setItem('posterCache', JSON.stringify(index));
return { black: black, entries: index.length };
}"""
class Checks:
def __init__(self):
self.failed = 0
def ok(self, label, condition, detail=""):
mark = "PASS" if condition else "FAIL"
if not condition:
self.failed += 1
print(f" [{mark}] {label}" + (f" -- {detail}" if detail and not condition else ""))
def items():
return [{"id": "vid-0", "title": "clip 0",
"url": "https://fixture.test/v0.mp4",
"thumb": "https://pictures.fixture.test/gone/0.jpg",
"channel": CHANNEL, "duration": 0, "isLive": False, "tags": []}]
def main():
c = Checks()
streamed = []
with sync_playwright() as p:
browser = p.chromium.launch(args=[
"--no-sandbox", "--disable-dev-shm-usage", "--disable-gpu",
"--autoplay-policy=no-user-gesture-required",
"--renderer-process-limit=1", "--js-flags=--max-old-space-size=512",
])
page = browser.new_page(viewport={"width": 1180, "height": 920})
crashes = []
page.on("pageerror", lambda e: crashes.append(str(e)))
page.route("**/api/status", lambda route: route.fulfill(
status=200, content_type="application/json", body=json.dumps(STATUS)))
page.route("**/api/videos", lambda route: route.fulfill(
status=200, content_type="application/json",
body=json.dumps({"items": items(), "pageInfo": {"hasNextPage": False}})))
page.route(re.compile(r"pictures\.fixture\.test"),
lambda route: route.fulfill(status=404, body=""))
page.route(re.compile(r"/api/image"),
lambda route: route.fulfill(status=404, body=""))
page.route(re.compile(r"/api/poster"), lambda route: route.fulfill(
status=200, content_type="application/json", body=json.dumps({"thumb": ""})))
# Byte ranges and all: a response a browser cannot seek in is one it
# will not take a frame out of the middle of.
def stream(route):
streamed.append(route.request.url)
body = open(CLIP, "rb").read()
span = re.match(r"bytes=(\d+)-(\d*)", route.request.headers.get("range") or "")
if span:
start = int(span.group(1))
end = int(span.group(2)) if span.group(2) else len(body) - 1
chunk = body[start:end + 1]
route.fulfill(status=206, body=chunk, headers={
"Content-Type": "video/mp4",
"Accept-Ranges": "bytes",
"Content-Range": f"bytes {start}-{end}/{len(body)}",
"Content-Length": str(len(chunk)),
})
return
route.fulfill(status=200, body=body, headers={
"Content-Type": "video/mp4",
"Accept-Ranges": "bytes",
"Content-Length": str(len(body)),
})
page.route(re.compile(r"/api/stream"), stream)
page.goto(BASE, wait_until="domcontentloaded")
page.evaluate(SEED, [SERVER, CHANNEL])
page.goto(BASE, wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
print("\nthe quarter mark of this clip is black")
page.wait_for_function(CAPTURED, arg="vid-0", timeout=60000)
shot = page.evaluate(LIGHT, "vid-0")
c.ok("the card ended up with a captured picture", shot is not None, str(shot))
c.ok("and it is not a black box",
shot and shot["brightest"] > 40, str(shot))
c.ok("it is the orange the clip shows past its lead-in",
shot and shot["mean"] > 60, str(shot))
c.ok("at card width, so it is a thumbnail and not a frame of video",
shot and shot["w"] == 360, str(shot))
print("\nand the picture it kept says it was looked at")
cached = page.evaluate(CACHE)
c.ok("one picture is in the cache", len(cached) == 1, str(cached))
c.ok("marked as checked", cached and cached[0].get("v") == 1, str(cached))
print("\na black picture kept by an earlier version is replaced")
poisoned = page.evaluate(POISON)
c.ok("the cache now holds what that version would have kept",
poisoned["entries"] == 1, str(poisoned["entries"]))
streamed.clear()
page.goto(BASE, wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
# A flat frame weighs about what a blank one does -- both are a single
# colour -- so what says the card moved on is the picture itself.
page.wait_for_function(
"""([id, black]) => {
const card = document.querySelector(`.video-card[data-video-id="${id}"]`);
const img = card && card.querySelector('.video-thumb img');
const src = img ? (img.getAttribute('src') || '') : '';
// The listed picture is on the card while the ladder walks
// down it, so what counts is a captured picture that is not
// the one that was there.
return src.indexOf('data:image/jpeg') === 0 && src !== black;
}""", arg=["vid-0", poisoned["black"]], timeout=60000)
healed = page.evaluate(LIGHT, "vid-0")
c.ok("the card shows a picture again", healed is not None, str(healed))
c.ok("not the black one it was given", healed and healed["brightest"] > 40, str(healed))
c.ok("taken from the video, which was fetched for it",
any("/api/stream" in url for url in streamed), str(streamed[:2]))
kept = page.evaluate(CACHE)
c.ok("and the replacement is what is kept now",
len(kept) == 1 and kept[0].get("v") == 1, str(kept))
print("\nand it is not measured a second time")
streamed.clear()
page.goto(BASE, wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
page.wait_for_timeout(2000)
again = page.evaluate(LIGHT, "vid-0")
c.ok("the kept picture is on the card straight away",
again and again["brightest"] > 40, str(again))
c.ok("without going back to the video", streamed == [], str(streamed))
print("\nand a clip nobody will say the length of is not left on frame one")
page.evaluate(FORGET)
page.add_init_script(NO_DURATION)
streamed.clear()
page.goto(BASE, wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
c.ok("the browser gives no length to go by",
page.evaluate("() => String(document.createElement('video').duration)") == 'NaN')
page.wait_for_function(CAPTURED, arg="vid-0", timeout=60000)
blind = page.evaluate(LIGHT, "vid-0")
c.ok("a frame came out of the video anyway", blind is not None, str(blind))
c.ok("and it is not the black one it opens on",
blind and blind["brightest"] > 40, str(blind))
c.ok("nothing threw along the way", not crashes, str(crashes[:2]))
browser.close()
print(f"\n{'OK' if not c.failed else 'FAILED'}: {c.failed} check(s) failed")
return 1 if c.failed else 0
sys.exit(main())

View File

@@ -0,0 +1,308 @@
#!/usr/bin/env python3
"""A favorite whose thumbnail has expired gets a frame out of the video instead.
Run against a locally running backend:
backend/main.py &
.venv/bin/python tests/smoke_poster_capture.py
Everything the client would reach for is a fixture, and every one of them is
dead on purpose: the listed picture 404s on both routes, the image proxy 404s,
and the source page has no replacement to offer. That is the state a favorite
saved months ago is actually in -- the signature on its CDN thumbnail expired --
and what the test asks is that the card still ends up with a picture of the
video, scaled down, kept, and not fetched again.
"""
import json
import os
import re
import sys
from playwright.sync_api import sync_playwright
# 1280x720, eight seconds. Wide on purpose: the capture has to scale down to
# card size, and a fixture no wider than the cache's own width would never show
# whether it does.
CLIP = os.path.join(os.path.dirname(__file__), "fixtures", "wide.mp4")
BASE = "http://127.0.0.1:5000/"
SERVER = "https://fixture.test"
CHANNEL = "clips"
# What App.posters scales a frame down to, and what it keeps the cache under.
POSTER_WIDTH = 360
CACHE_KEY = "posterCache"
STATUS = {
"online": True,
"channels": [{
"id": CHANNEL,
"name": "Fixture Clips",
"description": "A channel that only exists for this test.",
"options": [],
}],
}
# The favorite is the first item of the listing, saved with the thumbnail it
# had at the time -- the one that no longer loads.
SEED = """([server, channel]) => {
localStorage.setItem('config', JSON.stringify({ servers: [{ [server]: {} }] }));
localStorage.setItem('preferences', JSON.stringify({ [server]: { channelId: channel } }));
localStorage.removeItem('session');
localStorage.removeItem('posterCache');
Object.keys(localStorage).forEach((k) => {
if (k.indexOf('posterCache:') === 0) localStorage.removeItem(k);
});
localStorage.setItem('favorites', JSON.stringify([{
key: 'vid-0',
id: 'vid-0',
url: 'https://fixture.test/v0.mp4',
title: 'kept clip 0',
thumb: 'https://pictures.fixture.test/gone/0.jpg',
channel: channel,
uploader: '',
duration: 8,
isLive: false,
favoriteDate: new Date().toISOString()
}]));
}"""
# The picture on a card, and whether it is one: a broken <img> has no
# intrinsic size.
SHOT = """(id) => {
const card = document.querySelector(`.video-card[data-video-id="${id}"]`);
if (!card) return null;
const img = card.querySelector('.video-thumb img');
return {
src: (img.getAttribute('src') || '').slice(0, 24),
length: (img.getAttribute('src') || '').length,
width: img.naturalWidth,
height: img.naturalHeight
};
}"""
CACHE = """() => {
const index = JSON.parse(localStorage.getItem('posterCache') || '[]');
return index.map((entry) => ({
bytes: entry.n,
stored: (localStorage.getItem(`posterCache:${entry.k}`) || '').slice(0, 24),
length: (localStorage.getItem(`posterCache:${entry.k}`) || '').length
}));
}"""
PIXEL = """([id, x, y]) => new Promise((resolve) => {
const card = document.querySelector(`.video-card[data-video-id="${id}"]`);
const img = card ? card.querySelector('.video-thumb img') : null;
if (!img) { resolve(null); return; }
const probe = new Image();
probe.onload = () => {
const canvas = document.createElement('canvas');
canvas.width = probe.width;
canvas.height = probe.height;
const ctx = canvas.getContext('2d');
ctx.drawImage(probe, 0, 0);
const px = ctx.getImageData(x, y, 1, 1).data;
resolve({ r: px[0], g: px[1], b: px[2] });
};
probe.onerror = () => resolve(null);
probe.src = img.getAttribute('src');
})"""
CAPTURED = """(id) => {
const card = document.querySelector(`.video-card[data-video-id="${id}"]`);
if (!card) return false;
const img = card.querySelector('.video-thumb img');
return (img.getAttribute('src') || '').indexOf('data:image/jpeg') === 0;
}"""
class Checks:
def __init__(self):
self.failed = 0
def ok(self, label, condition, detail=""):
mark = "PASS" if condition else "FAIL"
if not condition:
self.failed += 1
print(f" [{mark}] {label}" + (f" -- {detail}" if detail and not condition else ""))
def items():
# The Referer is what gives an item a source page, which is the repair that
# comes before a capture. A favorite has none -- normalize() keeps no
# headers, since theirs expire -- so a saved item goes straight to the
# frame, while the same item in the listing walks the whole ladder.
return [{"id": f"vid-{i}", "title": f"clip {i}",
"url": f"https://fixture.test/v{i}.mp4",
"thumb": f"https://pictures.fixture.test/gone/{i}.jpg",
"http_headers": {"Referer": "https://page.fixture.test/"},
"channel": CHANNEL, "duration": 8, "isLive": False, "tags": []}
for i in range(3)]
def main():
c = Checks()
streamed = []
posters = []
with sync_playwright() as p:
browser = p.chromium.launch(args=[
"--no-sandbox", "--disable-dev-shm-usage", "--disable-gpu",
"--autoplay-policy=no-user-gesture-required",
"--renderer-process-limit=1", "--js-flags=--max-old-space-size=512",
])
page = browser.new_page(viewport={"width": 1180, "height": 920})
crashes = []
page.on("pageerror", lambda e: crashes.append(str(e)))
page.route("**/api/status", lambda route: route.fulfill(
status=200, content_type="application/json", body=json.dumps(STATUS)))
page.route("**/api/videos", lambda route: route.fulfill(
status=200, content_type="application/json",
body=json.dumps({"items": items(), "pageInfo": {"hasNextPage": False}})))
# Every way to the listed picture is gone: the provider, and our own
# image proxy standing in front of it.
page.route(re.compile(r"pictures\.fixture\.test"),
lambda route: route.fulfill(status=404, body=""))
page.route(re.compile(r"/api/image"),
lambda route: route.fulfill(status=404, body=""))
# And the source page has no fresher one to read off.
def poster(route):
posters.append(route.request.url)
route.fulfill(status=200, content_type="application/json",
body=json.dumps({"thumb": ""}))
page.route(re.compile(r"/api/poster"), poster)
# The video itself, which is the one thing that still works. Served
# the way the real proxy serves it -- byte ranges and all -- because a
# response a browser cannot seek in is one it will not capture a frame
# a quarter of the way into.
def stream(route):
streamed.append(route.request.url)
body = open(CLIP, "rb").read()
asked = route.request.headers.get("range") or ""
span = re.match(r"bytes=(\d+)-(\d*)", asked)
if span:
start = int(span.group(1))
end = int(span.group(2)) if span.group(2) else len(body) - 1
chunk = body[start:end + 1]
route.fulfill(status=206, body=chunk, headers={
"Content-Type": "video/mp4",
"Accept-Ranges": "bytes",
"Content-Range": f"bytes {start}-{end}/{len(body)}",
"Content-Length": str(len(chunk)),
})
return
route.fulfill(status=200, body=body, headers={
"Content-Type": "video/mp4",
"Accept-Ranges": "bytes",
"Content-Length": str(len(body)),
})
page.route(re.compile(r"/api/stream"), stream)
page.goto(BASE, wait_until="domcontentloaded")
page.evaluate(SEED, [SERVER, CHANNEL])
page.goto(BASE, wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
print("\nthe listed picture is gone, on every route there is")
c.ok("the module is there", page.evaluate("() => typeof App.posters.capture") == 'function')
c.ok("nothing is cached to begin with", page.evaluate(CACHE) == [], str(page.evaluate(CACHE)))
print("\nso the card takes a frame out of the video")
page.wait_for_function(CAPTURED, arg="vid-0", timeout=40000)
shot = page.evaluate(SHOT, "vid-0")
c.ok("the favorite's thumbnail is a captured frame",
shot and shot["src"].startswith("data:image/jpeg"), str(shot))
c.ok("and it decoded, so it is a picture and not an empty box",
shot["height"] > 0, str(shot))
c.ok("scaled down to card width, not kept at video size",
shot["width"] == POSTER_WIDTH, str(shot["width"]))
c.ok("the aspect ratio came through",
abs(shot["height"] - round(POSTER_WIDTH * 720 / 1280)) <= 1, str(shot))
c.ok("it was asked for at the quarter mark, from the video itself",
any("/api/stream" in url for url in streamed), str(streamed[:2]))
c.ok("and the source page was asked for a fresh picture before that",
any("/api/poster" in url for url in posters), str(posters[:2]))
print("\nthe frame is the one a quarter of the way in")
# The fixture is three bands of flat colour, so the frame says when it
# was taken: red for the first second and a half, orange from there to
# two and a half -- which is where a quarter of eight seconds falls --
# and navy for the rest. A capture that drew the first frame it decoded
# instead of seeking would come out red, and one that waited for the
# middle, navy.
frame = page.evaluate(PIXEL, ["vid-0", 180, 100])
c.ok("the colour is the one the clip shows two seconds in",
frame is not None and frame["r"] > 190 and 100 < frame["g"] < 180 and frame["b"] < 110,
str(frame))
c.ok("so it is neither the opening frame nor the middle",
frame is not None and frame["g"] > 80 and not (frame["b"] > frame["r"]), str(frame))
print("\nand keeps it")
cached = page.evaluate(CACHE)
c.ok("one picture is in the cache", len(cached) == 1, str(cached))
c.ok("stored as a JPEG data URL",
cached[0]["stored"].startswith("data:image/jpeg"), str(cached[0]["stored"]))
c.ok("small enough to keep a hundred of them",
0 < cached[0]["length"] < 60000, str(cached[0]["length"]))
c.ok("and the index says how big it is",
cached[0]["bytes"] == cached[0]["length"], str(cached[0]))
print("\nthe videos nobody kept are left alone")
others = [page.evaluate(SHOT, "vid-1"), page.evaluate(SHOT, "vid-2")]
c.ok("a card that is not a favorite captures nothing",
all(not (shot and shot["src"].startswith("data:")) for shot in others), str(others))
c.ok("so exactly one video was fetched for a picture",
len(streamed) == 1, str(streamed))
print("\nand the next session does not fetch it again")
streamed.clear()
posters.clear()
page.goto(BASE, wait_until="load")
page.wait_for_selector(".video-card", timeout=30000)
page.wait_for_timeout(1500)
shot = page.evaluate(SHOT, "vid-0")
c.ok("the kept picture is on the card straight away",
shot and shot["src"].startswith("data:image/jpeg"), str(shot))
c.ok("without going back to the video", streamed == [], str(streamed))
c.ok("or to the source page", posters == [], str(posters))
c.ok("and it is still the same one picture in the cache",
len(page.evaluate(CACHE)) == 1, str(page.evaluate(CACHE)))
print("\nthe favorites bar shows it too")
bar = page.evaluate("""() => {
const img = document.querySelector('.favorite-card img');
return img ? (img.getAttribute('src') || '').slice(0, 24) : null;
}""")
c.ok("the bar card is the captured frame as well",
bar is not None and bar.startswith("data:image/jpeg"), str(bar))
print("\nand the reader can have them taken again")
streamed.clear()
page.click(".settings-toggle")
c.ok("the setting says what is being kept",
"1 picture kept" in page.inner_text("#clear-posters-status"),
page.inner_text("#clear-posters-status"))
page.click("#clear-posters-btn")
c.ok("nothing is kept any more", page.evaluate(CACHE) == [], str(page.evaluate(CACHE)))
c.ok("and it says what it threw away",
"Threw away 1 picture" in page.inner_text("#clear-posters-status"),
page.inner_text("#clear-posters-status"))
page.wait_for_function(CAPTURED, arg="vid-0", timeout=40000)
shot = page.evaluate(SHOT, "vid-0")
c.ok("the card has a frame out of the video again",
shot and shot["src"].startswith("data:image/jpeg"), str(shot))
c.ok("which was fetched for it, this being the point of the setting",
any("/api/stream" in url for url in streamed), str(streamed[:2]))
c.ok("and kept again, as one picture",
len(page.evaluate(CACHE)) == 1, str(page.evaluate(CACHE)))
c.ok("nothing threw along the way", not crashes, str(crashes[:2]))
browser.close()
print(f"\n{'OK' if not c.failed else 'FAILED'}: {c.failed} check(s) failed")
return 1 if c.failed else 0
sys.exit(main())

View File

@@ -58,6 +58,20 @@ ROWS = """() => Array.from(document.querySelectorAll('#saved-panel .saved-row'))
stamp: row.querySelector('.saved-row-stamp').textContent
}))"""
MARK_ROWS = """() => {
// A mark on the elements themselves: a rebuilt row cannot carry it.
const rows = Array.from(document.querySelectorAll('#saved-panel .saved-row'));
rows.forEach((row, i) => { row.dataset.probe = `p${i}`; });
return rows.length;
}"""
PROBES = """() => Array.from(document.querySelectorAll('#saved-panel .saved-row')).map((row) => ({
probe: row.dataset.probe || '',
count: row.querySelector('.saved-count').textContent,
fresh: row.classList.contains('is-new'),
stamp: row.querySelector('.saved-row-stamp').textContent
}))"""
READ_BTN = """() => {
const button = document.getElementById('saved-read-btn');
return { disabled: button.disabled, text: button.textContent };
@@ -378,6 +392,20 @@ def main():
c.ok("and the button turns itself off with nothing left to clear",
page.evaluate(READ_BTN)["disabled"] is True, str(page.evaluate(READ_BTN)))
print("\na check writes into the list without rebuilding it")
c.ok("the list is open with a row in it", page.evaluate(MARK_ROWS) == 1)
offset["at"] = offset["at"] + 2
page.evaluate("() => App.savedSearches.check({ force: true })")
page.wait_for_timeout(1000)
rows = page.evaluate(PROBES)
c.ok("the row is the same element it was before the check",
len(rows) == 1 and rows[0]["probe"] == "p0", str(rows))
c.ok("with the count written into it where it stood",
rows[0]["count"] == "2", str(rows[0]["count"]))
c.ok("and the time of the check with it",
"checked" in rows[0]["stamp"].lower(), str(rows[0]["stamp"]))
c.ok("nothing animated itself back in", rows[0]["fresh"] is False, str(rows[0]))
c.ok("nothing threw along the way", not crashes, str(crashes[:2]))
browser.close()

188
tests/unit_direct_probe.js Normal file
View File

@@ -0,0 +1,188 @@
#!/usr/bin/env node
/* The direct-playability probe, tested without a browser.
*
* node tests/unit_direct_probe.js
*
* The probe decides whether the player may skip the proxy and let the browser
* fetch a CDN URL itself. Getting that wrong is not a slow video but a wrong
* one: eporner's CDN answers a browser-origin Referer with a 302 to a
* one-minute "watch this at our site instead" clip, served 206 with
* `Access-Control-Allow-Origin: *`, so "bytes arrived" was satisfied by a
* stand-in and the player switched to it mid-video. What settles it is whether
* the direct route hands back the same media the proxied one does, which is
* two stubbed fetches to check -- no server, no Chromium, no network.
*/
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const noop = () => {};
const element = () => ({
style: { setProperty: noop, removeProperty: noop },
classList: { add: noop, remove: noop, toggle: noop, contains: () => false },
dataset: {},
querySelector: () => null,
querySelectorAll: () => [],
addEventListener: noop,
removeEventListener: noop,
appendChild: noop,
removeChild: noop,
remove: noop,
getBoundingClientRect: () => ({ width: 0, height: 0, top: 0, bottom: 0, left: 0, right: 0 }),
setAttribute: noop,
removeAttribute: noop,
getAttribute: () => null,
cloneNode: element,
content: { firstElementChild: { cloneNode: element } },
children: [],
childElementCount: 0,
});
const sandbox = {
console,
setTimeout,
clearTimeout,
URL,
Response,
AbortController,
Image: function () { return element(); },
IntersectionObserver: function () {
return { observe: noop, unobserve: noop, disconnect: noop };
},
requestAnimationFrame: noop,
requestIdleCallback: noop,
performance: { now: () => 0 },
localStorage: { getItem: () => null, setItem: noop, removeItem: noop },
};
sandbox.addEventListener = noop;
sandbox.removeEventListener = noop;
sandbox.window = sandbox;
sandbox.self = sandbox;
sandbox.globalThis = sandbox;
sandbox.document = {
getElementById: () => null,
createElement: element,
querySelectorAll: () => [],
addEventListener: noop,
documentElement: element(),
body: element(),
head: element(),
};
sandbox.window.matchMedia = () => ({ matches: false, addEventListener: noop });
sandbox.window.location = { href: 'http://localhost/' };
const context = vm.createContext(sandbox);
sandbox.App = { state: {}, constants: {}, favorites: { getKey: () => null, has: () => false,
setButtonState: noop }, storage: { getPreferredQuality: () => 'auto' } };
vm.runInContext(
fs.readFileSync(path.join(__dirname, '..', 'frontend', 'js', 'videos.js'), 'utf8'),
context, 'videos.js');
const { probeDirect } = sandbox.App.videos;
let failed = 0;
const ok = (label, cond, detail) => {
if (!cond) failed++;
console.log(` [${cond ? 'PASS' : 'FAIL'}] ${label}` + (!cond && detail ? ` -- ${detail}` : ''));
};
// Stands in for both routes. `direct` is what the CDN hands the browser,
// `proxied` what /api/stream reports -- either given as a byte count, or null
// for a response that states no length at all. Every call is recorded so the
// test can check what was asked of whom.
let calls = [];
const stubFetch = function(direct, proxied) {
calls = [];
sandbox.fetch = async (url, opts) => {
calls.push({ url: String(url), method: (opts && opts.method) || 'GET' });
if (String(url).indexOf('/api/stream') === 0) {
const headers = proxied === null ? {} : { 'Content-Length': String(proxied) };
return new Response('', { status: 200, headers: headers });
}
const headers = direct === null ? {} : { 'Content-Length': String(direct) };
return new Response(new Uint8Array(16), { status: 206, headers: headers });
};
};
// A fresh host each time: the probe caches its answer per host for the session,
// which is the point of it, so reusing one would answer from the cache.
let hostSeq = 0;
const sourceOn = (host) => ({
url: `https://${host}/v4/token/10299874-1080p.mp4`,
referer: 'https://www.eporner.com/',
headers: { Referer: 'https://www.eporner.com/' },
});
const freshSource = () => sourceOn(`cdn${++hostSeq}.example.com`);
const run = async () => {
console.log('\nwhat the two routes say about the same URL');
stubFetch(486933813, 486933813);
ok('the same length both ways is a shortcut', await probeDirect(freshSource()) === true);
// The bug: 642497 is static.eporner.com/na.mp4, the stand-in clip.
stubFetch(642497, 486933813);
ok('a shorter file than the proxy reports is a substitution',
await probeDirect(freshSource()) === false);
// Not every mismatch is a decoy -- a longer one is just as wrong.
stubFetch(486933813, 642497);
ok('any disagreement about length is refused',
await probeDirect(freshSource()) === false);
console.log('\nwhen a length cannot be had');
stubFetch(642497, null);
ok('a proxy that states no length leaves the bytes test standing',
await probeDirect(freshSource()) === true);
stubFetch(null, 486933813);
ok('a direct route that states no length is not accused',
await probeDirect(freshSource()) === true);
stubFetch(null, null);
ok('neither stating one still plays direct',
await probeDirect(freshSource()) === true);
console.log('\nhow the cross-check is addressed');
stubFetch(486933813, 486933813);
const src = freshSource();
await probeDirect(src);
const head = calls.find((c) => c.url.indexOf('/api/stream') === 0);
ok('the proxy is asked, by HEAD', !!head && head.method === 'HEAD',
head ? head.method : 'never asked');
ok('with the Referer the origin wants, not the page\'s',
!!head && head.url.indexOf(`referer=${encodeURIComponent('https://www.eporner.com/')}`) !== -1,
head ? head.url : '');
ok('about the very URL being probed',
!!head && head.url.indexOf(encodeURIComponent(src.url)) !== -1);
console.log('\nthe answer is kept per host');
stubFetch(486933813, 486933813);
const repeat = sourceOn('cached.example.com');
await probeDirect(repeat);
const afterFirst = calls.length;
await probeDirect(repeat);
ok('a second probe of a known host asks nobody', calls.length === afterFirst,
`${calls.length - afterFirst} extra call(s)`);
console.log('\na bare URL still works');
stubFetch(642497, 486933813);
ok('a string is probed, and still cross-checked',
await probeDirect('https://bare.example.com/v.mp4') === false);
stubFetch(486933813, 486933813);
ok('nothing to probe is not playable', await probeDirect('') === false);
ok('neither is a source without a URL', await probeDirect({ url: '' }) === false);
};
run().then(() => {
console.log(`\n${failed ? 'FAILED' : 'OK'}: ${failed} check(s) failed`);
process.exit(failed ? 1 : 0);
}).catch((err) => {
console.error(err);
process.exit(1);
});