Filter, clean, and sample URLs for web crawling, with language, spam, and tracker filters, link extraction, and a URL store for crawl scheduling. Python and CLI
-
Updated
Nov 16, 2026 - Python
Filter, clean, and sample URLs for web crawling, with language, spam, and tracker filters, link extraction, and a URL store for crawl scheduling. Python and CLI
Keyword Search & Information Gathering Tool
🔨 extract all links from website
Crawl-Res是一款基于 Django + Scrapy + Celery 开发的全流程自动化资源检索系统,采用“配置即爬虫”的插件化设计,支持多站点异步并行抓取、智能链接提取与邮件自动推送,无需编写代码即可快速适配全网资源站。
WebScraper-Plus is a powerful and flexible Python library for extracting text, links, documents, and images from websites with OCR support, customizable output, and robust CLI/API options.
Extract a website's social media profile links (X/Twitter, LinkedIn, Instagram, Facebook, YouTube, TikTok, GitHub, Threads, Bluesky), skipping share buttons.
Multi-purpose web scraper that crawls websites and extracts links with configurable depth control
A modular FastAPI backend for email threat analysis. Supports rule‑based detection, link extraction, risk scoring, and sender reputation tracking. Includes a structured SQLAlchemy schema, a full rule engine, and utility scripts for seeding rules and importing example emails.
WSGI app that provides bounce URLs to HTML-driven webrings
To associate your repository with the link-extraction topic, visit your repo's landing page and select "manage topics."