Top 20 NuGet crawler Packages

dcsoup is a .NET library for working with real-world HTML. It provides a very convenient API for extracting and manipulating data, using the best of DOM, CSS, and jquery-like methods. This library is basically a port of jsoup, a Java HTML parser library. see also: http://jsoup.org/ API reference is ...

Web Exploration Model: crawlers, crawler reports, web analytic console, web loader, web crawler experiment setup...

HtmlMonkey is a lightweight HTML/XML parser written in C#. It allows you to parse an HTML or XML string into a hierarchy of node objects, which can then be traversed or queried using jQuery-like selectors. The library also supports creating node objects from code and producing HTML or XML from those...

Sop.Spider a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

The Sqreen in app agent for .NET. Defense in depth for OWASP Top-10 attacks that’s easy to install, manage and scale.

A library for reading/writing WARC files and scraping websites.

crawler framework , distributed crawler extractor. try ruiji scraper --- chrome web crawler https://chrome.google.com/webstore/detail/ruiji-scraper/klhahkhllngppofpkjdlbmnglnmnbbol?hl=zh-CN&authuser=0

Web Crawling and Scraping Framework

ExcavatorSharp is a multi-threaded server for scraping web data. It converts HTML code into a structured array of data. The library allows data scraping from multiple sites in parallel mode, within a single running application. Create scraping tasks and perform data extraction on a schedule. The l...

The Sqreen in app agent for .NET. Defense in depth for OWASP Top-10 attacks that’s easy to install, manage and scale.