User Agent

Select which bot this rule applies to.

Default Access

Allow All

Disallow All

Optimization Delay (Crawl-delay)

Sitemap URL

Restricted Directories

User-agent: *
Disallow: /cgi-bin/