Add support for :api_reference_noindex config

[robertoaloi]

Adds a new :api_reference_noindex config option (default: false) that adds <meta name="robots" content="noindex"> to the generated api-reference.html, asking search engines not to index it.

Motivation

Since the individual module and function pages are already indexed, the aggregate page is redundant for search. This option lets you keep the page available for navigation while excluding it from indexing.

Usage

defp docs do
  [
    [...]
    api_reference_noindex: true
  ]
end

The default is false, so behavior is unchanged for existing projects — including those that use the API Reference page as their indexed landing page.

Notes

• Applies to the HTML formatter.
• Relies on crawlers honoring the noindex robots meta tag.

[josevalim]

💚 💙 💜 💛 ❤️