Info
| Version: | 0.1.13-beta |
| Author(s): | AbotCore |
| Last Update: | Monday, November 21, 2016 |
| .NET Fiddle: | Create the first Fiddle |
| Project Url: | https://github.com/sjdirect/abot-dotnet-core |
| NuGet Url: | https://www.nuget.org/packages/AbotCore |
Install
Install-Package AbotCore
dotnet add package AbotCore
paket add AbotCore
AbotCore Download (Unzip the "nupkg" after downloading)
Dependencies
.NETStandard 1.6
- System.Net.Http(>= 4.1.0)
- AngleSharp(>= 0.9.9)
- HtmlAgilityPack.NetCore(>= 1.5.0.1)
- Microsoft.Extensions.Logging(>= 1.0.0)
- NETStandard.Library(>= 1.6.0)
- System.Collections.Specialized(>= 4.0.1)
- NRobotsCore(>= 1.1.2)
- AutoMapper(>= 5.1.1)
- System.Reflection.TypeExtensions(>= 4.1.0)
- System.Threading.ThreadPool(>= 4.0.10)
- System.Threading.Thread(>= 4.0.0)
- Microsoft.Extensions.Configuration.Abstractions(>= 1.0.0)
- Microsoft.Extensions.Configuration(>= 1.0.0)
- Microsoft.Extensions.Configuration.Json(>= 1.0.0)
- Microsoft.Net.Http(>= 2.2.22)
- AngleSharp(>= 0.9.9)
- HtmlAgilityPack.NetCore(>= 1.5.0.1)
- Microsoft.Extensions.Logging(>= 1.0.0)
- NETStandard.Library(>= 1.6.0)
- System.Collections.Specialized(>= 4.0.1)
- NRobotsCore(>= 1.1.2)
- AutoMapper(>= 5.1.1)
- System.Reflection.TypeExtensions(>= 4.1.0)
- System.Threading.ThreadPool(>= 4.0.10)
- System.Threading.Thread(>= 4.0.0)
- Microsoft.Extensions.Configuration.Abstractions(>= 1.0.0)
- Microsoft.Extensions.Configuration(>= 1.0.0)
- Microsoft.Extensions.Configuration.Json(>= 1.0.0)
Tags
Abot is an open source C# web crawler built for speed and flexibility. It takes care of the low level plumbing (multithreading, http requests, scheduling, link parsing, etc..).
You just register for events to process the page data. You can also plugin your own implementations of core interfaces to take complete control over the crawl process.