Repository navigation
feat(ar/azorafly): replace the dead Azora source with an azorafly.com plugin - #2585
Conversation
… plugin
The Azora source was inert. azoramoon.com no longer serves the site: it
redirects to azorafly.com, which has been rebuilt on Astro and no longer
exposes the Madara markup the old source parsed. The source is removed from
the madara generator and replaced by a standalone plugin.
Listing, series detail and chapter text are all server-rendered on the new
site, so they are scraped directly.
Two limits are worth knowing about, both site-side and left in place rather
than worked around:
- a series page carries only the newest 20 chapters plus one featured entry,
so novels are listed newest-first;
- `/novels` ignores `?page=`, so paging is client-side and `popularNovels`
returns the first page. Verified: pages 2 and 3 are byte-identical to page 1.
For the same reason `searchNovels` filters the listing page client-side: the
site's own `?s=` is not wired to the series catalogue (verified — `?s=scandal`
returns the same 91 unfiltered series as the bare listing).
`parseNovel` reads the work's title from the last `<h1>` on the page, because
the page leads with a row of info tiles (status / type / chapters / last update)
whose labels are also `<h1>`.
Reuses the existing azora icon.
Tested with `npm run check:plugin -- plugins/arabic/azorafly.ts`: popularNovels
57 novels, searchNovels 1 result, parseNovel 21 chapters, parseChapter 11538
chars. Also exercised in the Plugin Playground, where the listing renders.
Co-Authored-By: Claude <noreply@anthropic.com>
|
| const paragraphs = content | ||
| .find('p') | ||
| .toArray() | ||
| .map(el => $(el).html() || '') |
There was a problem hiding this comment.
Chapter markup can execute code If a chapter response contains executable markup inside a paragraph, this parser returns it unchanged. The reader then inserts that HTML without sanitizing it, so a malicious or compromised chapter response can execute code in the reader. Sanitize the chapter HTML before returning it.
How this was verified: Paragraph inner HTML flows from the fetched chapter response to the reader’s unsanitized HTML rendering sink.
| "lang": "Arabic" | ||
| } | ||
| }, | ||
| { |
There was a problem hiding this comment.
Existing Azora references break Removing the
azora source while registering its replacement as azorafly leaves saved references to the old plugin ID unresolved: plugin lookup requires an exact ID match. Users with saved Azora entries cannot open or update them through the replacement without an ID migration or compatibility mapping.
| ): Promise<Plugin.NovelItem[]> { | ||
| // `?page=` is accepted but ignored by the site, so there is only one page | ||
| // of results to hand back. | ||
| void pageNo; | ||
| void showLatestNovels; | ||
|
|
||
| const html = await this.fetchHtml( | ||
| `${this.baseUrl}/${filters.sort.value === 'manga' ? 'mangas' : 'novels'}`, |
There was a problem hiding this comment.
Later pages repeat results
popularNovels returns the same non-empty listing for page 2 and every later page. The browsing UI treats each non-empty response as another available page, so users can advance indefinitely through duplicates. Return an empty list after the only available page. searchNovels has the same issue when a search finds results.
| async searchNovels(searchTerm: string): Promise<Plugin.NovelItem[]> { | ||
| // The site's ?s= is not wired to the series catalogue, so match against | ||
| // what the listing page itself offers. | ||
| const html = await this.fetchHtml(`${this.baseUrl}/novels`); |
Three of the review points were real: - Deleting the azora entry from the madara sources orphaned every reader who had already added it. Plugin lookup is by exact id, so their saved entries could not be opened or updated through the replacement. Azora is now marked `down` instead of removed, which is how the repo retires a source that moved: the file still generates, so the id still resolves. Replacement is then just a matter of re-picking the source. - popularNovels returned the same listing for every page. Since the site ignores `?page=`, returning [] past page 1 is what tells the browse list there is nothing more to load. Same for searchNovels. - Search read only /novels, so anything browsable under the Manga filter was unsearchable. It now matches across both listings, and tolerates the manga half being down so a search still answers on the novels alone. The sanitize suggestion is not actioned here on purpose: no plugin in this repo sanitizes chapter HTML — every one of them returns container inner HTML — so that is a repo-wide decision for maintainers rather than something to land in one plugin. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Three of the four held up; one is a repo-wide question, not a plugin one. Fixed — Existing Azora references break ( The sharpest of the batch. Plugin lookup is by exact id, so deleting the Fixed — Later pages repeat results (both methods) Correct, and worse than it looks: because Fixed — Manga titles are unsearchable Real. Search read only Not actioned — Chapter markup can execute code I am not sanitising this. No plugin in this repo sanitises chapter HTML — Re-verified before azorafly.com went dark: 57 novels, 1 search result, 21 🤖 Generated with Claude Code |
parseChapter returned paragraph inner HTML as the site sent it, and the reader inserts chapter text as HTML. A response carrying a script or style block inside a paragraph would run rather than display, so the text is now cleaned before it leaves the plugin. Mirrors the epub exporter (src/lib/epub.ts) so a chapter reads the same in the app as in an exported file: drop script and style blocks, drop inline event handlers, drop javascript: URLs. Normal markup — emphasis, images, entities — is left alone. Checked against script tags, onclick, onerror, javascript: hrefs and style blocks; all are neutralised and ordinary paragraphs survive unchanged. Also name the way out on the Cloudflare 403 instead of a bare status code, matching the shared multisrc templates: a real browser in the app's webview passes the challenge where the plugin's own request cannot. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Four findings, addressed in two passes. Two were already fixed in this branch before the review landed, one is now fixed, and one is a site limit that I did not work around. 1. Chapter markup can execute code — fixed, this was real
Chapter text is now cleaned on the way out, mirroring the epub exporter in Checked against script tags, 2. Existing Azora references break — already fixed, was in 7132fa4 The Worth noting: I could not build a migration, because the app side of this repo has no such mechanism — nothing in 3. Later pages repeat results — already fixed, was in 7132fa4 Both entry points return if (pageNo > 1) {
return [];
}in 4. Manga titles are unsearchable — already fixed, was in 7132fa4 Search reads both catalogues. Novels first so a search still answers if the manga listing is the half that is down: const results = await match(await this.fetchHtml(`${this.baseUrl}/novels`));
try {
results.push(...match(await this.fetchHtml(`${this.baseUrl}/mangas`)));
} catch {
// manga listing unavailable — search the novels alone
}On testing this one honestly: the live check reports 🤖 Generated with Claude Code |
The site answers with 200 now, so the checks I could not run before are
run: the plugin's four fields had never been read off a live page.
Four of them were wrong.
- The Manga filter and the second half of search pointed at /mangas,
which is a 404. The comics listing is /comics — 60 novel cards and 61
comic cards, against 0 for /mangas, /manga and /manhwa. Because the
404 page looks like a normal page, the filter returned the homepage
and search silently lost every comic.
- Genres came from href="/genre/<slug>". The site has no such links; the
pills point at /series?genres=%2B<n>, an id with no name in it, so
the regex never matched and every novel reported no genres. The label
is the pill's own text. Real values now: أكشن, عائلي, خارق للطبيعة,
سحر, رعاية أطفال, رومانسي, بدون تجाजيد — 2 to 7 per novel.
- Status was read by looking for Arabic words after الحالة, but the tile
carries the English ONGOING/COMPLETED. Every novel reported UNKNOWN.
Read from the tile, not the page text.
- The summary used $('section p') filtered to paragraphs over 60
characters, and the page has no such paragraph — the synopsis ships
as og:description, wrapped in <p> and cut to 200 characters. Now
176 to 200 characters per novel instead of empty.
parseChapter gains a clearer error for the locked chapters the site
serves with 200 and no prose. None of the first twelve novels in
/novels had one, so it is rare, but "could not find the chapter text"
sent people looking for a broken selector rather than a paywall.
Live check: 60 novels, 1 search result, 21 chapters, 8660 chars. Four
novels read end to end, every field populated on all of them.
Sanitisation re-checked against live chapter text: 315 paragraphs, no
script, style, event handler or javascript: survives, and the rendered
text is unchanged by it.
Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Correction to my last comment: the site answers now, so the "could not get a green run" caveat is no longer true — and running it found four more bugs. I reported that the live check was INCONCLUSIVE with a Cloudflare 403 and that the parsing numbers had not been re-confirmed. That was accurate when written. Passing was the problem. The check only asserts that things are non-empty, and four fields were returning empty or wrong for reasons no smoke test would catch. Reading them off live pages: 1. The Manga filter and half of search pointed at a 404. The route is
Worse than returning nothing: the 404 page is styled like a normal page, so 2. Genres were always empty. Read from The pills carry an id ( 3. Status was always UNKNOWN. It looked for Arabic words after All four novels I read now resolve to 4. The summary was always empty. A locked-chapter error worth naming. Sanitisation, re-checked against live text (since that was the one real P1): 315 paragraphs, and no Corrected live check:
Read four novels end to end; name, status, genres, summary and chapters are populated on all four. One correction to my own commit message: it lists the seventh genre as "تجاجيد" — the value the site sends is 🤖 Generated with Claude Code |
Checklist
Problem
The Azora source is inert. azoramoon.com no longer serves the site — it
redirects to azorafly.com, which has been rebuilt on Astro and no longer
exposes the Madara markup the old source parsed.
This removes the source from the madara generator and replaces it with a
standalone plugin. Listing, series detail and chapter text are all
server-rendered on the new site, so they are scraped directly.
Known limits
Both are site-side and left in place rather than worked around, since working
around either would mean hitting the site's
/api/*endpoints, which itsrobots.txt disallows.
so novels are listed newest-first.
/novelsignores?page=— verified: pages 2 and 3 are byte-identical topage 1. Paging is client-side, so
popularNovelsreturns the first page.This is why the playground shows "of 1+".
For the same reason
searchNovelsfilters the listing page client-side: thesite's own
?s=is not wired to the series catalogue. Verified —?s=scandalreturns the same 91 unfiltered series as the bare listing.
Notes
parseNovelreads the work's title from the last<h1>on the page: thepage leads with a row of info tiles (status / type / chapters / last update)
whose labels are also
<h1>.The existing azora icon is reused, at the new
src/ar/azorafly/icon.pngpath.Testing
popularNovelssearchNovelsparseNovelparseChapterAlso exercised in the Plugin Playground, where the listing renders with real
translated titles.
🤖 Generated with Claude Code