Search Engine Optimization
25 views
Robots.txt
Quick Definition
Text file instructing crawlers which pages to crawl
Full Definition
A text file instructing web crawlers which pages or sections of a site to crawl or skip.
Examples
crawl budget management, blocking pages, search engine directives
Related Terms
technical-seo
xml-sitemap
crawlability
More Search Engine Optimization Terms
NAP Consistency
Ensuring consistent business info across all listings
Title Tag
HTML element specifying page title for search and browsers
On-Page SEO
Optimizing individual web pages for search rankings
Page Speed
How fast a web page loads and becomes interactive
Anchor Text
Clickable text in a hyperlink providing context
Keyword Research
Discovering and analyzing search terms for content strategy