Selecting dynamically-loaded content Finding the data source Inspecting the source code of a webpage Reproducing requests Handling different response formats Parsing JavaScript...

Selectors Using selectors Constructing selectors Using selectors Extensions to CSS Selectors Nesting selectors Selecting element attributes Using selectors with regular expres...

Requests and Responses Request objects Passing additional data to callback functions Using errbacks to catch exceptions in request processing Accessing additional data in errback...

Requests and Responses Request objects Passing additional data to callback functions Using errbacks to catch exceptions in request processing Accessing additional data in errback...

Requests and Responses Request objects Other functions related to requests Passing additional data to callback functions Using errbacks to catch exceptions in request processing ...

Spiders Contracts 自定义Contracts Spiders Contracts 0.15 新版功能. 注解 这是一个新引入(Scrapy 0.15)的特性，在后续的功能/API更新中可能有所改变，查看release notes 来了解更新。 测试spider是一件挺烦人的事情，尤其是只能编写单元测试(unit test)没...

核心API Crawler API 设置(Settings) API SpiderLoader API 信号(Signals) API 状态收集器(Stats Collector) API 核心API 0.15 新版功能. 该节文档讲述Scrapy核心API，目标用户是开发Scrapy扩展(extensions)和中间件(middlewa...

Signals Deferred signal handlers Built-in signals reference Engine signals engine_started engine_stopped Item signals item_scraped item_dropped item_error Spider signals sp...

Item Loaders Using Item Loaders to populate items Input and Output processors Declaring Item Loaders Declaring Input and Output Processors Item Loader Context ItemLoader objec...

Link Extractors 内置Link Extractor 参考 LxmlLinkExtractor Link Extractors Link Extractors 是那些目的仅仅是从网页(scrapy.http.Response 对象)中抽取最终将会被follow链接的对象｡ Scrapy提供了 scrapy.linkextrac...