Faxobot

Faxobot is the name of Faxo's indexing agent, also known as a spider. Faxobot is like a thousand internet users busily surfing the web. But it moves from page to page indexing the content it finds.

Faxobot obeys the robots.txt standard. robots.txt is a file that you place in your web server's root html directory that tells search engines what pages you do not want indexed.

Faxobot supports the Crawl-delay directive.

Faxobot also obeys the following nofollow, noarchive and noindex meta tags:

• <meta name="robots" content="noindex"> // Faxobot will not index a page that has this tag
• <meta name="robots" content="noarchive"> // Faxobot will not cache a page that has this tag
• <meta name="robots" content="nofollow"> // Faxobot will not follow links on a page that has this tag