Say hi to bots and crawlers with Kriti.blog.

All you have to do is … nothing. Unless you want customization.

Suggesting crawlers on how to index a page

The default meta tag for crawlers is <meta name="robots" content="index, dofollow">. You don’t have to add it every crawler will adhere to this specification. But what if noindex, nofollow is required for a particular page? Add following to your frontmatter

params:
    crawlers:
        robots: noindex, nofollow

Great! what about googlebot?

params:
    crawlers:
        googlebot: notranslate, ...

Same goes for any other bot, add its name under params.crawlers and corresponding directive.

You don’t have to do it for every page

It can be tiresome to add it to every page, so the alternative is to add it site wide from dashboard.

  1. Go to project dashboard
  2. Navigate to Publish section
  3. Expand the Meta Data sub section
  4. Add your required data