Google has completely reorganized its for-seo-training-course">crawlers and user-triggered fetchers documentation. It used to be all on one page and now it is in several pages. Most of the changes were just moving content around but Google did add sections for what product each crawler affects, and added a robots.txt snippet for each crawler to demonstrate how to use the user agent tokens.
Nigeria's #1 Blog SEO Resource: Get Ranked & Seen: Unlock the power of SEO for your Nigerian blog. Learn proven strategies to dominate search engines, attract more readers, and grow your online presence. [invalid URL removed].ng – Your one-stop shop for Nigerian blogger SEO success.