Top 20 NuGet packages for crawler

HtmlAgilityPack icon

HtmlAgilityPack

This is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT (you actually don't HAVE to understand XPATH nor XSLT to use it, don't worry...). It is a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world" m...

vote Vote for this package

Score: 7.8 | votes (0) | comments (0) | 7/6/2017 | v 1.5.1


AbotX icon

AbotX Web Crawler

A powerful C# web crawler that makes advanced crawling features easy to use. AbotX builds upon the open source Abot C# Web Crawler by providing a powerful set of wrappers and extensions.

vote Vote for this package

Score: 6 | votes (0) | comments (0) | 7/16/2017 | v 1.2.130


Abot icon

Abot Web Crawler

Abot is an open source C# web crawler built for speed and flexibility. It takes care of the low level plumbing (multithreading, http requests, scheduling, link parsing, etc..). You just register for events to process the page data. You can also plugin your own implementations of core interfaces to t...

vote Vote for this package

Score: 5.7 | votes (1) | comments (0) | 3/31/2017 | v 1.5.1.69


TeamFortressOutpostApi icon

TeamFortressOutpostApi

An API wrapper for the TF2 Outpost platform. A platform to find great deals for your Team Fortress 2, Counter-Strike: Global Offensive and Dota 2 items with zero hassle.

vote Vote for this package

Score: 5.3 | votes (0) | comments (0) | 4/26/2017 | v 0.1.1


BackpackLogin icon

BackpackLogin

A .net core library to handle the login to Backpack.tf. Backpack.tf is a trading site for Team Fortress 2, Counter-Strike: Global Offensive, and Dota 2. Community item pricing, item trading and stats, and much more.

vote Vote for this package

Score: 5.3 | votes (0) | comments (0) | 4/26/2017 | v 0.1.0


dcsoup icon

dcsoup HTML Parser

dcsoup is a .NET library for working with real-world HTML. It provides a very convenient API for extracting and manipulating data, using the best of DOM, CSS, and jquery-like methods. This library is basically a port of jsoup, a Java HTML parser library. see also: http://jsoup.org/ API reference is ...

vote Vote for this package

Score: 5.1 | votes (0) | comments (0) | 6/21/2017 | v 1.0.0


Shaman.Scraping icon

Shaman.Scraping

A library for reading/writing WARC files and scraping websites.

vote Vote for this package

Score: 5 | votes (0) | comments (0) | 3/30/2017 | v 1.0.1.4


DotnetSpider2.Core icon

DotnetSpider2.Core

Offical package. DotnetSpider is a high performance, light weight cralwer developed by C#.

vote Vote for this package

Score: 5 | votes (0) | comments (0) | 3/6/2017 | v 1.0.13


DotnetSpider2.Extension icon

DotnetSpider2.Extension

Offical package. DotnetSpider is a high performance, light weight cralwer developed by C#.

vote Vote for this package

Score: 5 | votes (0) | comments (0) | 4/18/2017 | v 1.0.10


HtmlAgilityPack.NetCore icon

HtmlAgilityPack for .NET Core

This is a port of HtmlAgilityPack library created by Simon Mourrier and Jeff Klawiter for .NET Core platform. This NuGet package supports can be used with Universal Windows Platform, ASP.NET 5 (using .NET Core) and full .NET Framework 4.6. Original description: This is an agile HTML parser that bu...

vote Vote for this package

Score: 4.3 | votes (0) | comments (0) | 7/22/2016 | v 1.5.0.1


SkyScraper icon

SkyScraper

Web scraper / crawler / spider. Supports robots protocol and user agent.

vote Vote for this package

Score: 4.1 | votes (0) | comments (0) | 8/26/2016 | v 1.0.46


NScrape icon

NScrape Web Scraping Framework

A web scraping framework for .Net

vote Vote for this package

Score: 3.5 | votes (0) | comments (0) | 9/21/2016 | v 0.3.0.0


HtmlDexterityPack icon

HtmlDexterityPack

This is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT (you actually don't HAVE to understand XPATH nor XSLT to use it, don't worry...). It is a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world" m...

vote Vote for this package

Score: 3.5 | votes (0) | comments (0) | 4/22/2017 | v 1.5.1


CrawlerLib.Engine.TestHelper icon

Crawler-Lib Engine Test Helper

The Crawler-Lib Engine Test Helper simplifies the test of tasks. It can be used to develop unit tests and integration tests for tasks.

vote Vote for this package

Score: 3 | votes (0) | comments (0) | 5/1/2015 | v 2.3.5544.21265


NCrawler icon

NCrawler

Crawler and scrapping framework which is written in C#

vote Vote for this package

Score: 3 | votes (0) | comments (0) | 8/11/2015 | v 3.0.1


NCrawler.EntityFramework icon

NCrawler.EntityFramework

Provides storing crawler data using EF for NCrawler

vote Vote for this package

Score: 3 | votes (0) | comments (0) | 4/23/2015 | v 3.0.2


HtmlAgilityPack.NetCoreCodePages icon

HtmlAgilityPack for .NET Core

This is a port of HtmlAgilityPack library created by Simon Mourrier and Jeff Klawiter for .NET Core platform. This NuGet package supports can be used with Universal Windows Platform, ASP.NET 5 (using .NET Core) and full .NET Framework 4.6. Original description: This is an agile HTML parser that bu...

vote Vote for this package

Score: 2.8 | votes (0) | comments (0) | 1/23/2017 | v 1.5.0.2


NCrawler.EsentServices icon

NCrawler.EsentServices

This is ESENT storage providers for the NCrawler

vote Vote for this package

Score: 2.5 | votes (0) | comments (0) | 4/23/2015 | v 3.0.0


NCrawler.FileStorageServices icon

NCrawler.FileStorageServices

Storage in file system for the NCrawler

vote Vote for this package

Score: 2.5 | votes (0) | comments (0) | 4/23/2015 | v 3.0.0


NCrawler.HtmlProcessor icon

NCrawler.HtmlProcessor

HTML processing of crawled data for NCrawler

vote Vote for this package

Score: 2.5 | votes (0) | comments (0) | 8/11/2015 | v 3.0.0