Repository navigation
Conversation
|
works also |
|
Pushed v1.1.8 for this head: every request now goes through one helper that treats a refused response as a site block (reported INCONCLUSIVE per docs/testing.md) instead of parsing a block page into an empty catalog. Local |
|
The live check should work now on this head too. The previous failure class is a runner-side block being parsed into an empty result and reported as a plugin failure; every request here now carries browser-like headers and treats a refused response as a site block, which docs/testing.md defines as INCONCLUSIVE rather than a plugin FAIL. Verified against the live site (popular 30, search, 50 chapters per page, chapter text) and driven through the plugin playground in the app UI. The check run is waiting on approval — could you approve it so it can report? |
[Medium impact] The PR appears safe to merge; no actionable defects were found. SummaryNovelight now maps each app page to one site range, oldest first, and rejects refused HTTP responses.
Reviews (5) · Last reviewed commit: "test(english/novelight): add determinist..." · Reviewed by Greptile |
|
current version is 1.1.5 so the version bump should be to 1.1.6 |
|
Version is now 1.1.6 (commit 063e448). Disclosure: this is an AI-authored change with no human review. |
| (a, b) => a.start - b.start, | ||
| ); | ||
| const pageNo = parseInt(page, 10); | ||
| const sitePage = sitePages[pageNo - 1]; |
There was a problem hiding this comment.
Please add a deterministic multi-page regression check for this mapping so page boundaries cannot skip, duplicate, or reverse chapters.
There was a problem hiding this comment.
Added a deterministic regression check in af062b7: scripts/novelight-pagination.test.mjs (node:test, no new dependency, no network). It bundles the real plugins/english/novelight.ts with only @libs/fetch stubbed and walks every app page of a fixture with a short 23-chapter oldest range plus four full 50-chapter ranges, with the select listed newest first. It asserts exactly one chapter-pagination request per page with the expected site page value, no chapter skipped or duplicated, ascending order across page boundaries, the short range served first, totalPages matching the walk, and no chapters (and no request) past the last page. Reversing the sort and changing sitePages[pageNo - 1] to sitePages[pageNo] each made it fail. CI runs it via .github/workflows/novelight-pagination-test.yml when the plugin or test changes. Run locally: node --test scripts/novelight-pagination.test.mjs. No plugin behavior or version change.
AI-authored change; no human review.
There was a problem hiding this comment.
@rajarsheechatterjee this is ready for another look: the deterministic multi-page regression check is in af062b78 and runs in CI.
`parseNovel` counted `#select-pagination-chapter > option` elements for `totalPages`, while `parsePage` recomputed it from the last numeric option in the whole document, and then mapped each app page onto a site page with `totalPages - page + 1`. The two derivations could disagree, and the reversed mapping made the site's partial page the FIRST page instead of the last: on live novels page 1 returned 2, 6 and 49 chapters instead of 50. Because that partition is anchored at the newest chapter, every added chapter shifted all page boundaries, which the app's per-page chapter cache turns into duplicated or missing chapters. Both `parseNovel` and `parsePage` now derive the chapter count from the largest range end in the site's page select (`ceil(total / 50)` pages), the same shape as the sibling `mynovels` plugin for this CMS. `parsePage` fetches only the site pages whose ranges overlap the requested 50-chapter chunk and slices exactly that chunk, so pages 1..totalPages partition the chapter list: every page but the last holds 50 chapters, and the partial page is last. Anchoring the chunks at the oldest chapter also keeps already-served pages stable when new chapters are added. Locked chapters are filtered after slicing to keep positions stable; name, URL, date and locked-chapter handling are unchanged. Authored with AI assistance.
…ted site popularNovels returned no novels in CI although the catalog markup and the 'a.item' selector are unchanged: the plugin sent no User-Agent at all, and the Cloudflare-fronted site answers such requests with a bot-check page on networks it distrusts (e.g. CI runners), which parses to zero novels. The live check's own HEAD probe (which carries a Mozilla User-Agent) passed, so the failure showed up as FAIL rather than INCONCLUSIVE. Pass browser-like headers (User-Agent, Referer, Accept-Language) on every fetchApi call, following the indraTranslations/dreamyTranslations pattern. Chapter-list pagination behavior is unchanged. Local live check passes: popularNovels 30 novels, searchNovels 1 result, parseNovel (parsePage) 50 chapters, parseChapter 12215 chars. Refs lnreader#2526. Authored with AI assistance.
Route every site request through one helper that throws (carrying the HTTP status) on a non-OK response, so a runner-side block is reported INCONCLUSIVE per docs/testing.md instead of being parsed into a false 'Returned no novels' FAIL. Headers and pagination behavior unchanged. This commit was authored with AI assistance.
Replace the uniform oldest-anchored 50-chapter slicing (which straddled two of the site's newest-anchored ranges on nearly every page, costing 2 chapter-pagination ajax requests) with a one-to-one mapping onto the site's own ranges: app page N is the Nth site range in ascending chapter order. The endpoint honors only the page parameter (page_size, per_page, limit, offset, start/end, paginate_by, count, size, length, pageSize, perPage, show all ignored; bookmark_chapter inert; the site's own JS sends only csrfmiddlewaretoken, book_id and page), and the width is fixed at 50 server-side, so the site's own partition is the only single-request design. parseNovel derives totalPages from the same markup, keeping the two in agreement, and requested values come straight from the parsed select so out-of-range clamping can no longer duplicate chapters. Trade-off: the short (remainder) page comes first instead of last, and new chapters shift every page's contents, as before this PR. Set the version to a single 1.1.5 -> 1.1.6 net bump. This commit was authored with AI assistance.
- parseSitePageChapters takes a single params object instead of four positional strings - throw a clear error when the chapter-pagination JSON has no html string instead of parsing undefined into zero chapters silently - let endpoint failures propagate with their original stack; drop the console.error noise and the lossy throw new Error(error) wrapper - drop the parsed-but-never-read ChapterPaginationOption.end field - filter valueless pagination options, comment the non-range skip, drop the duplicated newest-first comment - bump 1.1.6 to 1.1.7 (patch: behavior-preserving robustness fix) Co-Authored-By: firstmate-crewmate (AI agent)
063e448 to
af062b7
Compare
|
@greptileai review |


Closes #2526.
Screenshots
Plugin playground (
npm run dev:start) on this PR's head (af062b78), live novelight.net, novelbook/our-guild-princess-is-good-at-everything, whose site select lists the ranges1 - 11,12 - 61,62 - 111,112 - 161. After-only:mastermaps in-range app pages to the same site ranges, so the playground cannot show a difference there.Novel details parsed on this PR's head: title, status, genres and 4 total pages, before the first chapter page is fetched.
Chapter page 1 of 4 on this PR's head: the site's oldest range, chapters 1-11.
Chapter page 2 of 4 on this PR's head: the next site range, 50 chapters starting at chapter 12.
What changes
All changes are in
plugins/english/novelight.ts. The version goes from1.1.5to1.1.6.1-14,15-64, …). EachparsePagecall makes exactly onechapter-paginationrequest.parseNovelgetstotalPagesfrom the same parsed select, so the page count and the pages served always agree.User-Agent,RefererandAccept-Language, because the site is behind Cloudflare.htmlfield also throws. A block page is no longer parsed into an empty result, sodocs/testing.mdreports it as INCONCLUSIVE, not FAIL.Why one site range per app page
The endpoint accepts only
page; the site's ownstatic/book/js/single.jssends only{csrfmiddlewaretoken, book_id, page}. We probedpage_size,per_page,limit,offset,start/end,paginate_by,count,size,length,pageSize,perPageandshow, and the endpoint ignored all of them. The site fixes page width at 50 and anchors it at the newest chapter. An earlier revision re-sliced chapters into uniform 50s starting from the oldest, and that needed two requests for almost every page (review feedback above), so this revision serves the site's ranges as they are.Trade-offs: the short remainder page now comes first instead of last. When new chapters are added, every page's contents shift, which the plugin already did before this PR; refresh or re-add the novel after updating.
Verification (what was actually run)
npm run check:plugin -- plugins/english/novelight.ts: all four steps pass (30 popular, 1 search result, 43-chapter page, ~12k-char chapter).eslintandprettier --checkon the file are clean.npm run build:compileis clean. Full-reponpm run linthas 2 existing errors in unrelated files.A bundled-plugin script walked every app page against the live site and counted
chapter-paginationrequests. Every page made exactly 1 request, and no chapter was missing or duplicated.[21, 50×29][3, 50×9][2, 50×4][17, 50, 50]isaac[14, 50×4]genshin-impact-breezing-through-teyvat[9, 50×7]hidden-dead-ends[21, 50]86-eighty-six-vol-1[34]my-daughter-has-become-a-fool[14]Every page size matches the site's range labels. On the last five novels, the same script run against
mastergave the same in-range pages. Onmaster, the page after the last one returned the oldest page's chapters again; on this branch it returns none.Error paths: a missing novel throws
Request failed: 404. Pageabc, page0and pages past the last one return an empty list.In the playground (
npm run dev:start), popular, search, novel details (Turning: 30 pages), chapter page 1 (21 chapters, oldest first) and chapter reading all worked. The first chapter-read attempt failed with "Failed to fetch" while the test browser was crashing; it worked after relaunch.Review pass
A fresh review checked the diff for off-by-one errors, missing or duplicated chapters, first and last pages, novels under 50 chapters, empty or invalid pages, error handling, the version bump and plugin conventions. It confirmed no defects, so it made no code changes.
Not tested
hideLocked: none of the tested novels has locked chapters.check:pluginagainst novelight.net.Regression check for the page mapping
Added in response to the review request (and Greptile P2) for a deterministic multi-page test.
scripts/novelight-pagination.test.mjsbundles the realplugins/english/novelight.tswith esbuild, replacing only@libs/fetchwith an in-memory stub, soparseNovelandparsePagerun unmodified with no network.totalPagesmatching the walk, and a page past the last returning no chapters and no request.sitePages[pageNo - 1]tositePages[pageNo], each turned it red (source restored afterwards).node --test scripts/novelight-pagination.test.mjs(Node 22+, no new dependencies)..github/workflows/novelight-pagination-test.ymlruns it on pull requests that change the plugin, the test, or the workflow.Pipeline
Updates from git push no-mistakes
✅ **Intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
node --test scripts/novelight-pagination.test.mjs: 1 pass, 0 failsitePages[pageNo]) and the test failsnpm ci --ignore-scriptsnode --test scripts/novelight-pagination.test.mjs(passes)Mutation: changedsitePages[pageNo - 1]tositePages[pageNo](test fails)Mutation: reversed the range sort (test fails)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.
AI-authored change (including this review pass and description). No human review.