User-Agent
TopCrawler identifies itself with this user-agent:
TopCrawler/0.1 (+https://toplist.fun/crawler-info.php)
Crawler transparency
TopCrawler
TopCrawler is the crawler used by TopList Search to discover public web pages, links, and images for the TopList.fun search index.
What it collects
TopCrawler indexes public page titles, descriptions, visible text, outbound links, image URLs, alt text, and basic source metadata.
Robots.txt
TopCrawler is designed to respect robots.txt rules and standard crawler blocking.
Blocking TopCrawler
To block TopCrawler, add this to your robots.txt file:
User-agent: TopCrawler Disallow: /