关于
This skill provides a comprehensive framework for building and maintaining robust PySpider crawlers. It offers domain-specific guidance on handling complex scraping scenarios such as JavaScript rendering, proxy rotation, authentication, and multi-page pagination. By enforcing a structured workflow and highlighting critical anti-patterns—such as avoiding blocking operations and correctly utilizing the on_message method—it ensures developers create reliable, scalable data collection pipelines that adhere to enterprise standards.
