asyncio
New in version 2.0.
Scrapy has partial support asyncio
. After you install the asyncio reactor, you may use asyncio
and asyncio
-powered libraries in any coroutine.
Warning
asyncio
support in Scrapy is experimental. Future Scrapy versions may introduce related changes without a deprecation period or warning.
Installing the asyncio reactor
To enable asyncio
support, set the TWISTED_REACTOR
setting to 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'
.
If you are using CrawlerRunner
, you also need to install the AsyncioSelectorReactor
reactor manually. You can do that using install_reactor()
:
install_reactor('twisted.internet.asyncioreactor.AsyncioSelectorReactor')
Using custom asyncio loops
You can also use custom asyncio event loops with the asyncio reactor. Set the ASYNCIO_EVENT_LOOP
setting to the import path of the desired event loop class to use it instead of the default asyncio event loop.