Repository navigation
fix(ar/galaxynovels): read the full chapter index and the real chapter body - #2584
hamedhani1998 wants to merge 3 commits into
Conversation
…r body Three defects, all on long-running novels. 1. Only 30 of 3155 chapters were listed. The series page ships a static chapter index for its own reader to hydrate from: a small manifest naming a pack, and the pack holds every chapter. The page itself only renders the first page of it, which is all the plugin was reading. It now follows `data-manifest-url` to the pack and lists the lot. 2. Chapter links 404'd. The plugin synthesised `<novel>/chapter-<id>/`, but the site requires the slug form `/chapter-<id>/<percent-encoded-title>/`. The index already carries the correct URL, so it is now used verbatim. 3. `parseChapter` matched nothing. All three of its original selectors are absent from the site; the prose lives in `.wor-reader-text-surface`, which is now first in the list. Version 1.1.0 -> 1.2.0. Tested on https://galaxynovels.com/novel/i-can-copy-talents/ (3155 chapters): chapters 1, 31, 1501 and 3155 all return 200 with 13k-15k characters, and the list is ordered 1 -> 3155. `npm run check:plugin -- plugins/arabic/galaxynovels.ts` reports popularNovels 20 novels, searchNovels 1 result, parseChapter 16993 chars. Co-Authored-By: Claude <noreply@anthropic.com>
|
| const path = ch.url?.startsWith('http') | ||
| ? new URL(ch.url).pathname | ||
| : `${novelPath}chapter-${ch.id}/`; |
| const paragraphs = content.find('p').toArray(); | ||
| if (paragraphs.length) { | ||
| const body = paragraphs | ||
| .map(el => $.html(el) || '') | ||
| .join('') | ||
| .trim(); | ||
| if (body) { | ||
| return body; |
There was a problem hiding this comment.
| const manifest = await this.fetchJson<ChaptersManifest>(manifestUrl); | ||
| const pack = await this.fetchJson<ChaptersIndex>(manifest.pack_url); |
There was a problem hiding this comment.
Relative manifest URLs truncate chapters If either
data-manifest-url or pack_url is relative, it is passed to fetchJson without being resolved against the site. The fetch fails, and the caught error leaves the plugin parsing only the first page of chapters rendered in the novel page. The existing index branch already resolves relative URLs.
The reader cache urls reach the plugin from two places — a data-* attribute on the novel page and a field in the index JSON — and the site has published them both absolute and relative. A relative one was handed to fetchApi unresolved, so the fetch failed, the catch swallowed it, and the plugin fell back to parsing the novel page, which only renders the first 30 chapters of 3155. Resolving against the site keeps that silent truncation from coming back. Chapter paths now go through the same resolver, and the comment on the id fallback no longer claims the bare /chapter-<n>/ form works: every such form 404s, including the internal chapter id. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Checked all three against the live site before changing anything. One held Fixed — Relative manifest URLs truncate chapters ( The reader-cache URLs reach the plugin from two places, a Not actioned — Relative chapter links lose their slugs The current code does use So building a path from the id cannot work at all, and the id-based line is Not actioned — Non-paragraph chapter content disappears The Re-verified after the change: 20 novels, 1 search result, 651 chapters, 🤖 Generated with Claude Code |
…p duplicated url handling Review flagged two things in parseChapter and the chapter-index load. The paragraph filter looked like it dropped content, so record what the non-paragraph markup actually is: on a live chapter the surface holds 129 child elements, 97 of them <p>, and the other 32 are <small>/<i>/<span>/<b> carrying hidden, aria-hidden, role="presentation" and data-wor-decoy="1" — SEO bait the site injects into the text surface. Returning the whole block would splice that into every chapter, so the filter is what keeps it out. The comment now says so, since the code reads as lossy without it. The index branch rebuilt the absolute/relative check by hand where resolveUrl already does it, and resolveUrl is what the manifest and pack branches use. Route it through the helper so all three resolve the same way. Verified: 20 novels, search, 651 chapters, 16993 chars in parseChapter. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Checked all three findings against the live site. Two were already handled; the third turned out to be the opposite of what the review describes, and I have added the measurement so the next reader does not have to re-derive it. 1. Relative chapter links lose their slug — already fixed in this branch
const path = ch.url
? new URL(this.resolveUrl(ch.url), this.site).pathname
: `${novelPath}chapter-${ch.id}/`;The slug is preserved. The bare 2. Relative manifest URLs truncate chapters — already fixed, now consistent Both the manifest and the pack go through 3. Non-paragraph chapter content disappears — the paragraphs are the content This one is worth being precise about, because the numbers do not support it. On a live chapter (651 chapters in that novel's index, so there is no thin-content edge case):
Every one of the 32 carries <small hidden="" aria-hidden="true" role="presentation" data-nosnippet="" data-wor-decoy="1" ...>That is SEO bait the site injects inside the prose surface, not content the parser is dropping. There are no images, headings or lists in the block at all — the counts for those are 0. Returning the whole block would splice that hidden markup into every chapter, so the paragraph filter is the thing keeping it out. I have recorded this in a comment at the filter, because read cold the line does look lossy and that is a fair thing to flag. Live check on this branch after the change:
🤖 Generated with Claude Code |
|
Closing this as superseded by the broader GalaxyNovels fixes in #2529. |
Checklist
Problems
Reported against a 3155-chapter novel
(https://galaxynovels.com/novel/i-can-copy-talents/), which showed only 30
chapters and no readable text.
1. Only the first 30 chapters were listed. The series page ships a static
chapter index for its own reader to hydrate from: a small manifest naming a
pack, and the pack holds every chapter. The page renders only the first page of
it, and that was all the plugin read. It now follows
data-manifest-urlto thepack and lists every chapter.
2. Chapter links 404'd. The plugin synthesised
<novel>/chapter-<id>/, butthe site requires the slug form
/chapter-<id>/<percent-encoded-title>/. Theindex already carries the correct URL, so it is now used verbatim. (Verified:
bare path 404s, index URL returns 200.)
3.
parseChaptermatched nothing. All three of its original selectors areabsent from the site. The prose lives in
.wor-reader-text-surface, which isnow first in the list; the remaining three are kept as fallbacks.
Testing
On the 3155-chapter novel: chapters 1, 31, 1501 and 3155 all return HTTP 200
with 13k-15k characters, and the list is ordered 1 -> 3155.
popularNovelssearchNovelsparseNovelparseChapter🤖 Generated with Claude Code