Scrapy project (original) (raw)
An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.
Pinned Loading
- Scrapy, a fast high-level web crawling & scraping framework for Python.
Python 55.2k 10.8k - The scrapy.org website
HTML 64 142
Repositories
Type
Select type
All Public Sources Forks Archived Mirrors Templates
Language
Select language
All C++ DIGITAL Command Language HTML Python Shell
Sort
Select order
Last updated Name Stars
Showing 10 of 27 repositories
- scrapy/scrapyd-client’s past year of commit activity
Python
771
BSD-3-Clause
146 5 0
Updated May 15, 2025 - scrapy Public
Scrapy, a fast high-level web crawling & scraping framework for Python.
scrapy/scrapy’s past year of commit activity - parsel Public
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors
scrapy/parsel’s past year of commit activity - scrapy/scrapy.org’s past year of commit activity
HTML
64 142 1 1
Updated May 8, 2025 - scrapy/cssselect’s past year of commit activity
- w3lib Public
Python library of web-related functions
scrapy/w3lib’s past year of commit activity - scrapyd Public
A service daemon to run Scrapy spiders
scrapy/scrapyd’s past year of commit activity
Python
3,028
BSD-3-Clause
571 9 0
Updated Apr 12, 2025 - queuelib Public
Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python
scrapy/queuelib’s past year of commit activity
Python
277
BSD-3-Clause
55 3 2
Updated Mar 31, 2025 - protego Public
A pure-Python robots.txt parser with support for modern conventions.
scrapy/protego’s past year of commit activity - itemloaders Public
Library to populate items using XPath and CSS with a convenient API
scrapy/itemloaders’s past year of commit activity
Python
48
BSD-3-Clause
15 17 5
Updated Mar 24, 2025