Scrapy at a glance Walk-through of an example spider What just happened? What else? What’s next? Scrapy at a glance Scrapy is an application framework for crawling web site...

Scrapy shell Configuring the shell Launch the shell Using the shell Available Shortcuts Available Scrapy objects Example of shell session Invoking the shell from spiders to i...

Scrapy shell Configuring the shell Launch the shell Using the shell Available Shortcuts Available Scrapy objects Example of shell session Invoking the shell from spiders to i...

Scrapy shell Configuring the shell Launch the shell Using the shell Available Shortcuts Available Scrapy objects Example of shell session Invoking the shell from spiders to i...

数据收集(Stats Collection) 常见数据收集器使用方法 可用的数据收集器 MemoryStatsCollector DummyStatsCollector 数据收集(Stats Collection) Scrapy提供了方便的收集数据的机制。数据以key/value方式存储，值大多是计数值。该机制叫做数据收集器(Stats Co...

Scrapy at a glance Walk-through of an example spider What just happened? What else? What’s next? Scrapy at a glance Scrapy is an application framework for crawling web site...

Spider组件 Spider组件

Logging Log levels How to log messages Logging from Spiders Logging configuration Logging settings Command-line options Custom Log Formats Advanced customization scrapy.uti...

Benchmarking Benchmarking 0.17 新版功能. Scrapy提供了一个简单的性能测试工具。其创建了一个本地HTTP服务器，并以最大可能的速度进行爬取。该测试性能工具目的是测试Scrapy在您的硬件上的效率，来获得一个基本的底线用于对比。其使用了一个简单的spider，仅跟进链接，不做任何处理。 运行: scrapy ...

Release notes 1.0.5 (2016-02-04) 1.0.4 (2015-12-30) 1.0.3 (2015-08-11) 1.0.2 (2015-08-06) 1.0.1 (2015-07-01) 1.0.0 (2015-06-19) Support for returning dictionaries in spiders ...