Top 20 NuGet robots Packages

SimpleSitemap is a lite library that helps you create web sitemaps for collections or lists of items. These sitemaps follow the Sitemap Protocol. Both sitemapindex and urlset links are generated, based upon the data collection size and 'page size'. Examples of this could be a list of your users, ...
SEO plugin for ASP.NET Core applications.
robots.txt middleware for .NET core
Sitemap and robots dynamic generator core classes
Sitemap and robots dynamic generator with gadget and admin plugins for EPiserver
Allows to serve a robots instruction file.
A simple middleware built on reflection designed to support your Search Engine Optimized (SEO) app by dynamically creating a Sitemap.xml and Robots.txt. This package is designed to be very simple for simple use cases but allows for custom disallowed routes, user agents, and custom node endpoints by ...
A simple middleware built on reflection designed to support your Search Engine Optimized (SEO) app by dynamically creating a Sitemap.xml and Robots.txt. This package is designed to be very simple for simple use cases but allows for custom disallowed routes, user agents, and custom node endpoints by ...
Configurable robots.txt handler supporting web.config based configuration. Icon supplied by Freepik (http://www.flaticon.com/authors/freepik)
Package Description
Robots.txt parsing and querying utility
The Robots Handler package provides editors with the ability to dynamically change the contents of a site's robots file. Instead of storing the contents of a robots file on the file system, an editor can specify its contents in an Umbraco content page. The property content is then served via a Http ...
A friendly tool for creating dynamic robots.txt files in Umbraco 8
Automatic generation of sitemap.xml and robots.txt for ASP.NET Core application. Uses routing tables to generate a standards compliant sitemap file. Includes HTTPs override, and controller or action level exclusions.
IDeliverable.Seo is module for the Orchard CMS that allows site owners robust and granular control over SEO aspects. The module lets site owners control page titles, meta keywords and descriptions, robots.txt and sitemap.xml. Additionally, it allows 301/302 redirects to be configured for changed URL...
Fornece todos os recursos necessários para o spidering de aplicações web.
AngleSharp IHtmlParser Extension for Mechanize.NET
Stateful programmatic web browsing, based on Python-Mechanize, which is based on Andy Lester’s Perl module WWW::Mechanize.
Sitecore Multisite Http module, allows mulitsite handling of 404, & 500 errors, along with custom robots.txt generation
robots.text implementation for EPiServer