Skip to content

fix(arabic): fix empty chapters and missing chapter bodies in Arabic sources - #2529

Open
RibatTRW wants to merge 12 commits into
lnreader:masterfrom
RibatTRW:fm/lnreader-arabic-empty-chapters
Open

RibatTRW wants to merge 12 commits into
lnreader:masterfrom
RibatTRW:fm/lnreader-arabic-empty-chapters

Conversation

@RibatTRW

@RibatTRW RibatTRW commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Closes #2522, #2568, #2486, #2483, #2505, #2465, #2501, #2370, #1566, #2442, #2576, #2578, #2667, #2490

Opening a chapter in these Arabic sources showed "no content", even though the cover, description and chapter list still loaded. Each source broke for a different reason. Four are fixed. The fifth, Markazriwayat, could not be checked at all, so it is listed as not verified instead of fixed.

Source Cause Status
Galaxy Novels chapter links were built from a WordPress post id (…/chapter-<post-id>/, which 404s), and the JSON endpoint the plugin preferred answers Cloudflare 403 to third-party clients fixed
Riwyat $('.text-left') || … never reached its later selectors (a cheerio selection is always truthy), and the theme's invisible decoy blocks were returned as text fixed
Kol Novel the body regex required a bottomnav terminator that the theme dropped, and hard-coded a <div> wrapper where the theme serves <article> fixed; verified through a proxy only (2026-09-28). The site is down as of 2026-10-11
Free Kol Novel the same regex break, plus chapter links point at kolnovel.com, so the plugin requested free.kolnovel.com/https://kolnovel.com/… fixed; verified through a proxy only (2026-09-28). The site is down as of 2026-10-11
Markazriwayat not reproduced: the site serves a "Coming Soon" page and 404s every other path not verified; no code changed

What master fixed separately. The Kol Novel family and TC & Sega "empty chapter" bodies (bottomnav terminator, <article class="epcontent">) are fixed on master by #2593, #2611 and #2632, and this PR no longer carries that selector change. This PR still changes plugins/multisrc/lightnovelwp/template.ts for two things master does not have: it keeps only the path when a chapter link is an absolute URL on another host (Free Kol Novel links kolnovel.com, and parseChapter re-prefixes the site, giving free.kolnovel.com/https://kolnovel.com/…), and parseChapter throws Chapter text not found on the page, try to open in webview. instead of returning an empty string. The lightnovelwp template version goes to 1.2.12 and the madara one to 2.5.

This PR also fixes Riwyat's novel metadata on the site's new theme, where name, cover, summary, author, genres, status and rating all came back empty. The fix adds fallbacks that run only after the existing selectors find nothing, so sources on the classic theme keep their current behaviour. Issues #2568, #2576 and #2578 report empty Kol Novel-family chapters; they come from the same template defect and this fix covers them.

Strengthening pass

A second pass reviewed the changes to the two shared templates (madara and lightnovelwp), because dozens of other sources use them. It made three small changes. Each one prevents a specific failure, shown before and after in the evidence below:

  1. madara: image-only chapters render again. The new selector walk kept the first candidate that contains text. An illustration or comic chapter with only <img> inside .text-left has no text, so this PR's first version returned an empty body where master showed the images. A candidate that contains images now counts as content too.
  2. madara and lightnovelwp: missing bodies raise a clear error. When no chapter body is found (a login wall, a locked chapter, or a future markup change), parseChapter used to return "". It now throws Chapter text not found on the page, try to open in webview. The error only replaces cases that returned an empty string, so any chapter that returned text before returns the same text.
  3. Galaxy Novels: no fake chapter text. Any 200 page without a chapter body, such as the novel page, returned <p>Content not available.</p> as though it were the chapter, and a download would have saved that text. The plugin now throws instead.

This pass made no other changes. Kol Novel and Free Kol Novel still cannot be reached from the machine used for testing (TCP timeout), so they remain verified through a proxy only.

Issue #2522

#2522 reports "no content" when opening a chapter on Kol Novel, Free Kol Novel and Riwyat (its labels name Galaxy Novels and Markazriwayat, but the report text names the Kol Novel family and Riwyat). Against today's live sites this PR covers it as follows:

  • Riwyat: fixed and verified. Novel metadata and chapter bodies return on novels that have chapters (see the 2026-09-28 re-verification below: 120 chapters with full metadata and a 9,011-char body; 70 chapters with a 634-char body).
  • Kol Novel and Free Kol Novel: fixed, but both sites are down right now (2026-10-11), so the fix cannot be tested end-to-end until they return. On 2026-09-28 both hosts still resolved but time out over HTTPS from the test machine, so no end-to-end check:plugin run can reach them (fetch failed → INCONCLUSIVE). The fix was re-verified on live chapter HTML fetched through a proxy: the old regex returns 0 chars and the new selector returns the full body (29,415 chars / 254 paragraphs on the sampled chapter). Free Kol Novel's cross-host chapter links were also re-confirmed.
  • Markazriwayat: still not verified. The site answers direct requests with a Cloudflare challenge and shows a "Coming Soon" page through a reader proxy. No code in this PR touches it.

Currency (2026-09-28)

master has 16 newer commits since this branch's base. They touch the shared madara template (a Translator-author fallback and a Summary fallback) and madara/sources.json (add Tangerine Archive, drop two dead entries), plus unrelated plugins. None of them invalidate this change, the branch is still mergeable, and the head is unchanged (184b85a), so history was not rewritten. The stacked fork branch for the Galaxy full chapter index sits on an earlier point of this branch and is untouched.

Galaxy Novels full chapter list, review fixes and screenshots (2026-10-11)

Full chapter list (#2667, #2490). The novel page only renders the latest 30 chapters, and its data-index-url is empty. parseNovel now reads the chapter container's data-manifest-url. The manifest's pack_url holds every chapter, and its live_tail holds the newest ones. These are merged by chapter id the way the site's own reader merges them, then sorted oldest first. When the manifest is missing or malformed, the plugin falls back to the 30 chapters on the page. Galaxy Novels goes from 1.1.1 to 1.1.2. This is the change from RibatTRW#13, cherry-picked unchanged.

Checked against the live site, comparing with each novel's manifest total:

Novel Manifest total Before (1.1.1) After (1.1.2) Chapter 1 body
i-can-copy-talents 3155 30 (3164 → 3135, newest first) 3155, numbered 1..3155 8,414 chars
after-billions-of-deaths-i-copy-talents-and-slaughter-all-races 156 30 (156 → 127) 156, numbered 1..156 7,199 chars
the-auction-returns-ten-thousand-times-and-an-auction-house-opens 1797 30 1797, numbered 1..1797 9,382 chars

Before this change chapter 1 was unreachable on every novel with more than 30 chapters. chapterNumber is the manifest position, so numbering is contiguous. The chapter name keeps the site's own label, which can differ where the site skips numbers (position 3155 is "الفصل 3164"). The fallback was exercised by making the manifest return 404 and by serving a malformed pack. Both returned the 30 page chapters, which is the behaviour these novels showed before this change.

Review fixes. A code review of the whole diff found two defects, both fixed:

  1. Riwyat status. The NHV-theme status fallback mapped every non-ongoing novel to Completed. The site marks stopped and on-hold novels "متوقفة", so cont/surviving/ showed Completed. That status now maps to On Hiatus. Completed ("مكتملة") and Ongoing ("مستمرة") are unchanged.
  2. Galaxy fallback order. The page-derived fallback list kept the page's newest-first order. It is now reversed, so the oldest chapter comes first, like the manifest path.

Merge with master. The branch had fallen into conflict with master. A normal merge commit brings it up to date, with no history rewrite or force push. The madara cover conflict is resolved as extractCover on .summary_image first, then the NHV cover, then defaultCover. master's own lightnovelwp .epcontent fixes (#2593, #2632) already cover what this PR's selector change did, so the merged template keeps one copy. This PR's remaining lightnovelwp changes are the cross-host chapter-link fix and the "chapter text not found" errors.

Checks on head be436fd:

  • npm run build:compile exits 0.
  • npm run check:plugin:
    • Galaxy Novels: popular 20, search 1, parseNovel 1797 chapters, parseChapter 31,056 chars.
    • Riwyat: popular 12, search 12, parseNovel 9 chapters, parseChapter 35,653 chars.
    • ArNovel (madara): 12 / 1 / 279 chapters / 6,562 chars.
    • KnoxT (lightnovelwp): 20 / 0 / 5 chapters / 9,729 chars.
    • BoxNovel: INCONCLUSIVE (HTTP 403 Cloudflare).
  • The TC&Sega check could not run: the checker cannot load a plugin file whose name contains &.
  • The Riwyat cover still resolves after the merge on both a classic and an NHV page.

Screenshots. These come from this repo's plugin playground, run in headless Chromium, not from the LNReader app.

Galaxy Novels, before: 30 chapters, newest first.

Galaxy Novels before: 30 chapters, newest first

Galaxy Novels, after: all 156 chapters, from chapter 1 to chapter 156.

Galaxy Novels after: 156 chapters, chapter 1 first
Galaxy Novels after: list ends at chapter 156

Galaxy Novels, after: 3155 chapters on i-can-copy-talents, first and last.

Galaxy Novels after: i-can-copy-talents lists 3155 chapters
Galaxy Novels after: list ends at chapter 3164 (position 3155)

Galaxy Novels, after: chapter 1 opens with its text (19,107 chars).

Galaxy Novels after: chapter 1 opens with its text

Riwyat, before and after the status fix: an on-hold novel showed Completed and now shows On Hiatus.

Riwyat before: on-hold novel shown as Completed
Riwyat after: on-hold novel shown as On Hiatus

Riwyat, after: novel metadata on the NHV theme (cover, status, author, rating, genres, summary, 220 chapters) and a chapter body (13,426 chars).

Riwyat after: novel metadata and chapter list
Riwyat after: chapter body

Kol Novel and Free Kol Novel could not be screenshotted because both sites are down as of 2026-10-11. Both hostnames resolve to the same server (148.251.110.232). It does not answer on ports 80 or 443 from the test machine (TCP timeout), and a second, independent network path gets "connection refused" on 443. A regular browser or the app cannot reach them right now either, so the Kol Novel fixes stay verified only on the HTML captured through a proxy on 2026-09-28.

Known, not changed here. Riwyat chapter release dates show "Invalid Date". That comes from the shared madara date parsing, which this PR does not touch, and it happens on master too.

Not closed

Verification evidence

First pass

  • Galaxy Novels: npm run check:plugin shows popular 20 ✅, search 1 ✅, parseNovel 30 chapters ✅, parseChapter 13258 chars ✅ (was a 404).
  • Riwyat: popular 12 ✅, search 1 ✅, parseNovel 15 chapters ✅, parseChapter 16910 chars ✅ (was 0 chars).
  • The shared lightnovelwp fix was reproduced independently on a reachable source, TC & Sega: parseChapter FAIL (0 chars) → PASS 5905 chars.
  • Regression sweeps: all 36 lightnovelwp sources before and after showed exactly one status change (TC & Sega, FAIL → PASS). 20 madara sources were identical, and 12 madara sources swept for the metadata fallbacks showed no status or body-length change.
  • kolnovel.com and free.kolnovel.com time out over TCP from the test machine. They were verified by running the plugin's own extraction on live HTML fetched through a text proxy, which gave 14002 chars from the same chapter on both hosts. On a 12-paragraph sample, the old regex returned 0 chars and the new selector returned 1,006. At that time the sites themselves were up and only that network route timed out. As of 2026-10-11 both sites are down (see the Galaxy Novels section below).
  • Checked against this PR's base (8522c771) at dc58567e: Galaxy Novels parseChapter went from FAIL Could not reach site (404) to PASS with 49,502 chars, and Riwyat parseNovel went from FAIL Missing novel name to PASS with full metadata, 100 chapters and a 9,002-char chapter body.
  • A spot sweep of sources this PR does not name found KnoxT, ArNovel, LightNovelHeaven and Ninja identical on head and base. UniversalNovel and MostNovel fail the same way on both, so those failures existed before this PR.

Strengthening pass (head 184b85a vs dc58567)

Fixture before/after, calling the bundled plugins' parseChapter:

Input dc58567 184b85a
madara, .text-left holding only <img> (master returned the images) returns "" returns the <img> paragraphs
madara, page with no chapter body returns "" throws Chapter text not found on the page, try to open in webview.
lightnovelwp, page without .epcontent returns "" throws the same error
lightnovelwp, normal <article class="epcontent"> chapter two paragraphs the same two paragraphs
Galaxy Novels, live novel page served as a chapter (HTTP 200) returns <p>Content not available.</p> throws Chapter text not found on the page, open it in WebView.

Full regression sweep over all 78 madara and 36 lightnovelwp sources plus Galaxy Novels. Every HTTP response was recorded once and replayed to both heads, so both saw identical pages. For each source the sweep ran popular → first novel → first chapter and compared status, name, author, genres, status, rating, summary length, cover, chapter count, body length and a hash of the body. 115 of 115 were identical. Reachability from the test machine:

Template Chapter body OK No chapters No popular list Blocked (403/404/410/503/captcha) Unreachable
madara (78) 26 2 11 19 20
lightnovelwp (36) 8 2 4 12 10

On every reachable lightnovelwp chapter, the text extracted by the old regex and by this PR's selector was identical. Only the HTML serialization differs. On every reachable classic madara chapter, .text-left holds the text, so the selector walk still stops at .text-left, as it did before.

npm run check:plugin on 184b85a:

  • Galaxy Novels: popular 20 ✅, search 1 ✅, parseNovel 30 chapters ✅, parseChapter 49502 chars ✅
  • Riwyat: popular 12 ✅, search 3 ✅, parseNovel 110 chapters ✅, parseChapter 9002 chars ✅
  • Kol Novel, Free Kol Novel: ⚠️ INCONCLUSIVE (fetch failed, TCP timeout on the test machine's route), still verified through a proxy only

npm run build:compile exits 0. eslint and prettier --check pass on all changed files.

Re-verified 2026-09-28 (head 184b85a, no code changes)

Direct probes: galaxynovels.com HTTP 200, cenele.com HTTP 200, kolnovel.com and free.kolnovel.com TCP timeout on 443 (fetch failed), markazriwayat.com HTTP 403 (Cloudflare challenge; reader proxy shows "Coming Soon").

  • Galaxy Novels (npm run check:plugin -- plugins/arabic/galaxynovels.ts): popular 20 ✅, search 1 ✅, parseNovel 30 chapters ✅, parseChapter 49502 chars ✅.
  • Riwyat (npm run check:plugin -- "plugins/arabic/Riwyat[madara].ts"): popular 12 ✅, search 1 ✅, parseNovel ❌ FAIL "No chapters returned" — the harness checks the first popular novel (cont/pain-novel/), whose own ajax/chapters/ endpoint currently answers "Manga has no chapter yet", so the site itself lists zero chapters there. Targeted runs on novels that have chapters still pass: cont/i-can-see-through-e/ → 120 chapters, full metadata (author, genres, Ongoing status, rating 8.4, cover, 905-char summary), first-chapter body 9,011 chars; cont/starting-over/ → 70 chapters, first-chapter body 634 chars.
  • Kol Novel / Free Kol Novel (npm run check:plugin on both generated plugins): ⚠️ INCONCLUSIVE at siteReachability (fetch failed) — the direct route still times out, so end-to-end verification remains impossible from here. Proxy re-verification on live HTML: the sampled Kol chapter (Chrysalis 162) serves <div class="epcontent entry-content"> with no bottomnav; the old regex returns 0 chars and this PR's $('.epcontent').first().find('p') selector returns 29,415 chars (254 paragraphs). The Free novel page still lists chapter links as absolute kolnovel.com URLs, and the path normalization was unit-verified (old: free.kolnovel.com/https://kolnovel.com/…, new: /…); the same Free chapter's content was confirmed present through a reader proxy.
  • Markazriwayat (npm run check:plugin -- plugins/arabic/Markazriwayat.ts): ⚠️ INCONCLUSIVE (HTTP 403 Cloudflare). Still not verified; no code changed.
  • npm run build:compile exits 0; eslint and prettier --check pass on all changed files.

This PR was written by AI coding agents working for the PR author (the strengthening pass by Claude Opus 5.5, the earlier description refresh by Muse Spark, and the Galaxy full chapter list fold, review fixes, master merge and this update by Claude Opus 5.5). No human has reviewed the code, so please review it with that in mind.

Status mapping and master update (2026-11-04)

Addresses the review comment on the NHV status mapping and the merge conflicts with master.

  • Status mapping. In plugins/multisrc/madara/template.ts the NHV fallback now maps only recognized labels: مستمرة to Ongoing, متوقفة to On Hiatus and مكتملة to Completed. Any other label leaves status unset instead of Completed. (Correction, 2026-11-19: مكتملة does appear on live pages, see the maintainer-review section below.)
  • .manga-status fallback. It ran whenever status was unset, and empty text counts as Completed, so an unknown NHV label would still have ended up Completed. It is now skipped when an .nhv-novel-status label is present. Pages with no NHV label behave as before, and the older non-NHV status branches are unchanged.
  • Master update. The branch is rebased onto current master, which removes the conflicts. The conflict in galaxynovels.ts was resolved keeping both master's changes and this PR's fixes.
  • Checks after the update. npm run build:compile passes, and prettier passes. eslint reports only the existing no-explicit-any warning in template.ts. check:plugin: Galaxy Novels passes (popular 20, search 1, parseNovel 168 chapters, parseChapter 26,798 chars). Riwyat's default novel has no chapters on the site ("There is no chapters"), so its parseNovel check fails on that data; on another Riwyat novel it passes (302 chapters, 10,758-char body).
  • Restored after the rebase. The first rebase dropped this PR's lightnovelwp/template.ts changes. They are restored in c87fddbe: the cross-host chapter path fix, the two missing-body errors, and the version bumps (lightnovelwp 1.2.12, madara 2.5). The epcontent selection is not restored because master already has it (fix(lightnovelwp): parse chapter text when bottomnav is absent #2593, fix(lightnovelwp): support article.epcontent chapters and fix search URL (#2483) #2632). Nothing else was dropped: the Galaxy Novels, Madara template and Madara sources changes are all still in the diff. check:plugin on KnoxT passes; Kol Novel and Free Kol Novel are still unreachable from the test machine (inconclusive).
  • No new screenshots: the only visible change is that unrecognized NHV labels no longer show Completed.

This section was written by an AI coding agent (Claude Sonnet 5.5). No human has reviewed the code.

Maintainer review follow-up (2026-11-19)

Addresses the three review comments of 2026-11-18. Commit 5e19ac27.

  1. Galaxy Novels hidden text. parseChapter removes [data-wor-decoy], [hidden] again before it picks the chapter body, so a candidate holding only hidden text no longer counts as the chapter. On 9 live chapters of 4 novels the returned HTML held 26 to 63 hidden elements with text before and 0 after, and the visible text was identical. Playground, same chapter (30,821 characters before, 17,307 after):

    Before After
    Galaxy Novels chapter before Galaxy Novels chapter after
  2. Madara promo-only candidate. parseChapter now runs the source's customJs cleanup on a candidate and takes the first one that still has text or images afterwards. A rejected candidate is cleaned and the next one is read from the pristine page HTML, because some cleanups (Ranovel's) are not idempotent. Sources without customJs follow exactly the old path. A promo-only .text-left before .reading-content returned an empty string before and returns the chapter now. If every candidate is empty after cleanup it now throws the existing "Chapter text not found" error instead of returning an empty body.

  3. Madara status labels. The label is read from the value element only (.nhv-novel-status also holds the "الحالة" caption), normalized (NFKC, tatweel and diacritics removed, whitespace collapsed) and matched exactly. Live cenele.com, 261 novel pages: مستمرة 203, مكتملة 54, متوقفة 4, nothing else, so those three map as before. غير مكتملة does not occur on the live site; it now stays unset (before: Completed). I left it unset rather than guessing Ongoing or On Hiatus, since the site never uses it.

How it was checked (machine with partial network reach; AI-written, no human review):

  • Fixtures reproducing each case, run against the previous head and this one, with full bodies compared. The promo-only cases fail before and pass after; every no-customJs case (.text-right, .text-content, .text-chapter-content, novel-chapter, .entry-content, .c-blog-post) is identical.
  • Madara regression sample: 76 sources, 22 reachable from here (the rest refused, reset or did not resolve). On 3 chapters each (first, middle, last), previous and new template returned byte-identical bodies on all 58 readable chapters of 20 sources. Selectors seen live: .text-left (54), .reading-content (3), .entry-content (1). The other selectors were checked only on fixtures. Morenovel and Ranovel (6 chapters) gave the same error before and after.
  • check:plugin passes for Galaxy Novels, Riwyat and ArNovel; lint, prettier and build:compile pass.

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 6 of 6 scenarios driven live against the product
Scenario Result Live Evidence
galaxynovels parseChapter removes 28 data-wor-decoy and 28 hidden elements (mocked fetch, synthetic page) ✅ pass live galaxy-results.txt
galaxynovels skips a candidate holding only hidden text and uses the next candidate (mocked fetch) ✅ pass live galaxy-results.txt
galaxynovels throws an error instead of placeholder text when only hidden text exists (mocked fetch) ✅ pass live galaxy-results.txt
madara promo-only .text-left falls through to .reading-content (riwyat customJs, mocked fetch) ✅ pass live madara-results.txt
madara normal .text-left still wins and its promo is stripped; image-only chapters still return; promo-only pages error (mocked fetch) ✅ pass live madara-results.txt
madara status mapping: غير مكتملة stays unset; مكتملة, مستمرة, متوقفة and a diacritic variant map to the right status; an unknown label stays unset (mocked fetch) ✅ pass live madara-results.txt
  • node h/gal.mjs: galaxynovels parseChapter against synthetic pages
  • node h/run.mjs: madara riwyat parseChapter and parseNovel against synthetic pages
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@RibatTRW

Copy link
Copy Markdown
Contributor Author

bro the markaz riwayat website is not even up... i need help with kol novel also, i am able to get the words and characters but the website itself is not up for me...

@RibatTRW RibatTRW changed the title fix(ar): empty chapter bodies in Galaxy Novels, Riwyat, Kol Novel and Free Kol Novel fix(ar): empty chapter bodies and Riwyat novel metadata (Galaxy Novels, Riwyat, Kol Novel, Free Kol Novel) Sep 24, 2026
@RibatTRW RibatTRW changed the title fix(ar): empty chapter bodies and Riwyat novel metadata (Galaxy Novels, Riwyat, Kol Novel, Free Kol Novel) fix(arabic): list all Galaxy Novels chapters and fix empty chapter bodies Oct 11, 2026
@RibatTRW

Copy link
Copy Markdown
Contributor Author

bump

Copy link
Copy Markdown
Member

Could we map only recognized status labels and leave unknown values unset? Any nonempty label outside these two Arabic strings is currently marked Completed.

@RibatTRW
RibatTRW force-pushed the fm/lnreader-arabic-empty-chapters branch from be436fd to 13ba9a5 Compare November 3, 2026 22:49
@RibatTRW RibatTRW changed the title fix(arabic): list all Galaxy Novels chapters and fix empty chapter bodies fix(arabic): fix empty chapters in Galaxy Novels and Madara Arabic source Nov 3, 2026
@RibatTRW

RibatTRW commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Done in 63cf1d1, now on the PR head c87fddb after rebasing onto current master.

  • The NHV status block in plugins/multisrc/madara/template.ts now maps only مستمرة (Ongoing), متوقفة (On Hiatus) and مكتملة (Completed). Any other label leaves status unset. I did not find مكتملة on a live novel page (all 23 sampled cenele.com novels were ongoing); it comes from the site's novel-wiki page and the theme's is-completed CSS class.
  • The .manga-status fallback below it would have turned an unknown NHV label back into Completed, so it is now skipped when an .nhv-novel-status label is present. The older non-NHV branches are unchanged.
  • The first rebase had dropped this PR's lightnovelwp template changes (cross-host chapter path for Free Kol Novel, errors on a missing chapter body); they are restored in c87fddb with version bumps. The epcontent selection is not restored because master already has it (fix(lightnovelwp): parse chapter text when bottomnav is absent #2593, fix(lightnovelwp): support article.epcontent chapters and fix search URL (#2483) #2632).
  • The conflicts with master are gone (the PR reports mergeable and clean).

Re-checked after the update: build:compile and prettier pass; check:plugin passes for Galaxy Novels (168 chapters, chapter body returned) and for Riwyat on a novel that has chapters (302). Riwyat's default check novel has no chapters on the site, so that check fails on that data.

This change was made by an AI coding agent; no human has reviewed it.

@RibatTRW RibatTRW changed the title fix(arabic): fix empty chapters in Galaxy Novels and Madara Arabic source fix(arabic): resolve empty and missing chapters in Galaxy Novels, Madara and LightNovelWP sources Nov 3, 2026
]) {
const candidate = $(selector);
if (candidate.length && candidate.text().trim()) {
return candidate.html() || '';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we restore the [data-wor-decoy] and [hidden] cleanup before extracting the chapter? Two live chapter bodies contained 28 and 25 hidden elements with text.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 5e19ac2: parseChapter removes [data-wor-decoy], [hidden] right after loading the page, before a candidate is chosen, so a candidate holding only hidden text no longer counts as the chapter.

Checked live on 9 chapters of 4 novels (first, middle and last chapter of each): the returned HTML had 26 to 63 hidden elements with text before the change and 0 after, with identical visible text. In the plugin playground the first chapter of "Sign in to become a saint" went from 30,821 to 17,307 characters (before/after screenshots are in the PR description). check:plugin passes.

This reply and the change were written by an AI coding agent; no human has reviewed them.

Comment thread plugins/multisrc/madara/template.ts Outdated
'.c-blog-post > div > div:nth-child(2)',
]) {
const candidate = loadedCheerio(selector);
if (candidate.text().trim() || candidate.find('img').length) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we check for text or images after cleanup? A promo-only .text-left currently wins, then cleanup leaves it empty even when .reading-content contains the chapter.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 5e19ac2: the template now runs the source's customJs cleanup on a candidate and takes the first one that still has text or images afterwards. A rejected candidate is cleaned and the next one is read from the pristine page HTML, because some cleanups (Ranovel's) are not idempotent. Sources without customJs follow the same path as before. If every candidate is empty after cleanup it now throws the existing "Chapter text not found" error instead of returning an empty body.

Checked with fixtures: a promo-only .text-left before .reading-content (and before .entry-content) returned an empty string before and returns the chapter now. Every other selector case without customJs returns an identical body. Because the template is shared, I also compared the previous and new template on live chapters (first, middle, last) of the 22 Madara sources reachable from this machine (of 76): all 58 readable chapters of 20 sources were byte-identical. Live selectors seen were .text-left, .reading-content and .entry-content; the others were only covered by fixtures.

This reply and the change were written by an AI coding agent; no human has reviewed them.

Comment thread plugins/multisrc/madara/template.ts Outdated
if (nhvStatus.includes('مستمرة')) novel.status = NovelStatus.Ongoing;
else if (nhvStatus.includes('متوقفة'))
novel.status = NovelStatus.OnHiatus;
else if (nhvStatus.includes('مكتملة'))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we match normalized status labels exactly? includes('مكتملة') classifies غير مكتملة (incomplete) as Completed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 5e19ac2: the label is read from the value element only (.nhv-novel-status also contains the "الحالة" caption), normalized (NFKC, tatweel and diacritics removed, whitespace collapsed) and matched exactly against مستمرة, متوقفة and مكتملة. Anything else stays unset.

Live labels on cenele.com (261 novel pages): مستمرة 203, مكتملة 54, متوقفة 4, nothing else, so those map as before. غير مكتملة is not used on the site; with the change it stays unset instead of Completed (before: Completed). I left it unset rather than guess Ongoing or On Hiatus, since the site never produces it. Fixtures confirm غير مكتملة, with or without extra spaces, is unset, while مُكتملة and مكـتملة map to Completed.

This reply and the change were written by an AI coding agent; no human has reviewed them.

RibatTRW and others added 8 commits November 19, 2026 06:49
parseNovel preferred the data-chapter-id attribute (a WordPress post id)
over the chapter link and built `…/chapter-<post-id>/` paths, which the
site answers with 404 — only `…/chapter-<number>/<slug>/` resolves, so
every chapter list it produced was unusable.

parseChapter then asked `wp-json/wor-reader-app/v1/chapters/<id>` first;
that route exists but Cloudflare answers 403 to every non-app client
(reproduced from a real browser with a same-origin fetch), and the HTML
fallback looked for selectors the theme no longer ships. Read the body
from the article marked up as https://schema.org/Chapter instead, taking
the first candidate that actually contains text.

Fixes the chapter body for Galaxy Novels (issue lnreader#2522).
`$('.text-left') || $('.text-right') || …` never reaches the second
selector: a cheerio selection object is always truthy, so a page without
`.text-left` returned an empty chapter instead of trying the rest. Walk
the selectors and keep the first one that actually contains text, and add
the anchors the Madara-derived themes read from (`.text-content`,
`.text-chapter-content`, `novel-chapter`, `.reading-content`).

The riwyat source (cenele.com) also hides scraped-text decoys in
`<section inert data-nosnippet>` blocks that its inline CSS makes
invisible and wraps the body in a per-request random class, so its
customJs now drops those sections along with in-body style/script tags
and the app-promo blocks.

Fixes the chapter body for Riwyat (issue lnreader#2522).
Riwyat (cenele.com) migrated to a custom NHV theme whose novel pages no
longer render the classic Madara detail blocks, so parseNovel returned an
empty name, cover, summary, author and genres. Try each existing selector
first and only fall through to the theme-specific markup (h1.nhv-novel-title,
.nhv-novel-cover img, .nhv-novel-synopsis p, .nhv-novel-meta author links,
.nhv-novel-genres a, .nhv-novel-status, .nhv-simple-rating__avg) when it
yields nothing, leaving classic-theme sources unchanged.

Pairs with the parseChapter selector-chain fix for the same source; this
change stays out of parseChapter and sources.json.

AI-authored change, no human review claimed.
…odies

- madara: a chapter candidate that holds only images (illustration or comic
  chapters) counts as content again; the text-only test skipped it and
  returned an empty body where master rendered the images.
- madara, lightnovelwp: throw a clear error when no chapter body is found
  instead of returning an empty string.
- galaxynovels: throw instead of returning "Content not available." as if it
  were the chapter text.
The novel page only renders the latest 30 chapters and data-index-url
is empty, so parseNovel now reads data-manifest-url from the chapters
container, merges the pack file with live_tail the way the site reader
does, and falls back to the page-derived list when the manifest is
absent or malformed.

This change was authored by an AI coding agent.

Co-Authored-By: firstmate-crewmate <crewmate@firstmate.local>
…ers oldest first

The NHV-theme status fallback mapped every non-ongoing novel to Completed,
so novels the site marks متوقفة (stopped/on hold) showed as Completed; they
now map to On Hiatus. Galaxy Novels' page-derived fallback list kept the
page's newest-first order; it is now reversed so the oldest chapter comes
first, like the manifest path.

This change was authored by an AI coding agent.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… and status labels

- galaxynovels: drop [data-wor-decoy] and [hidden] elements before choosing the chapter body.
- madara: choose the first chapter candidate that still has text or images after the source's customJs cleanup; rejected candidates are cleaned on a throwaway copy of the page.
- madara: match NHV status labels exactly after Unicode/diacritic/whitespace normalization, reading only the value element so "غير مكتملة" is no longer Completed.
@RibatTRW
RibatTRW force-pushed the fm/lnreader-arabic-empty-chapters branch from c87fddb to 5e19ac2 Compare November 18, 2026 22:51
@RibatTRW RibatTRW changed the title fix(arabic): resolve empty and missing chapters in Galaxy Novels, Madara and LightNovelWP sources fix(arabic): fix empty chapters and missing chapter bodies in Arabic sources Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment