Noindex & Nofollow: Control Search Engine Indexing
9 min read
Noindex & Nofollow: Control Search Engine Indexing
Reading time: 9 minutes
Quick Definition:
- Noindex tells search engines “Don’t show this page in search results”
- Nofollow tells search engines “Don’t pass link equity through links on this page”
Unlike robots.txt (which blocks crawling), these tags allow crawling but control what happens after.
TLDR
Noindex prevents pages from appearing in search results. use it for thank-you pages, search results, and duplicate content. Nofollow prevents passing link equity. use it for paid links, user-generated content, and untrusted links. Critical mistake: combining noindex with robots.txt blocks creates conflicts where Google can’t see the noindex tag. Most common combination is noindex with follow, which blocks the page but follows its links. Verify changes with Google Search Console and allow 1-4 weeks for processing.
Noindex: Remove Pages from Search Results
What It Does
Noindex prevents a page from appearing in Google search results, even if other sites link to it.
Use cases:
- Thank you pages after form submissions
- Internal search result pages (
/search?q=shoes) - Admin/login pages (combine with password protection)
- Duplicate content you can’t delete
- Thin/low-value pages (tag archives, author pages)
- Private content that’s not truly secret
Don’t use for:
- Pages you want to rank (obviously)
- Pages already blocked in robots.txt (Google can’t see the noindex tag if it can’t crawl)
- Truly sensitive content (use password protection instead)
How to Implement Noindex
HTML meta tag (in <head>):
<meta name="robots" content="noindex">Target specific search engines:
<!-- Only block Google -->
<meta name="googlebot" content="noindex">
<!-- Only block Bing -->
<meta name="bingbot" content="noindex">
<!-- Block all bots -->
<meta name="robots" content="noindex">HTTP header (for PDFs, images, non-HTML files):
X-Robots-Tag: noindexWordPress Implementation
Yoast SEO:
- Edit page/post
- Yoast SEO section → Advanced
- Set “Allow search engines to show this page in search results?” to No
Rank Math:
- Edit page/post
- Rank Math section → Advanced
- Toggle “Robots Meta” → Select noindex
Manual (in theme):
<?php
// In header.php or functions.php
if (is_search() || is_404()) {
echo '<meta name="robots" content="noindex, follow">';
}
?>Checking If Noindex Works
Method 1: View page source
- Visit the page
- Right-click → “View Page Source”
- Search (Ctrl+F) for “noindex”
Method 2: Google Search Console
- URL Inspection tool
- Enter the URL
- Look for “Indexing allowed? No: ‘noindex’ detected in ‘robots’ meta tag”
Method 3: Browser extension
- META SEO inspector (Chrome)
- Shows meta robots tags in toolbar
Nofollow: Control Link Equity
What It Does
Nofollow tells search engines “Don’t pass PageRank/link equity through the links on this page.”
Two implementations:
- Page-level nofollow (entire page)
- Link-level nofollow (specific links)
Page-Level Nofollow
Syntax:
<meta name="robots" content="nofollow">What happens:
- Google crawls the page
- Google indexes the page
- Google ignores ALL links on the page for ranking purposes
Use cases:
- User-generated content pages (comments, forums)
- Pages with many external links you don’t vouch for
- Affiliate disclosure pages
Rarely needed because link-level nofollow (below) is more precise.
Link-Level Nofollow
Syntax:
<a href="https://example.com" rel="nofollow">Link text</a>Use cases:
- Paid links (Google requires this)
- Affiliate links
- Untrusted user-generated content
- Login/signup links (don’t waste crawl budget)
- Links to your own unimportant pages (cart, account, etc.)
Example: Affiliate link
<a href="https://affiliate.example.com/product?ref=123" rel="nofollow">
Buy Now
</a>Modern Link Attributes (2019+)
Google introduced more specific link attributes:
| Attribute | Purpose | When to Use |
|---|---|---|
rel="sponsored" | Paid/affiliate links | Ads, sponsored content, affiliate links |
rel="ugc" | User-generated content | Comments, forum posts, user bios |
rel="nofollow" | Don’t pass link equity | General untrusted links |
Multiple attributes:
<a href="https://example.com" rel="nofollow sponsored">Sponsored Link</a>Which to use?
- Affiliate links:
rel="sponsored"(ornofollow sponsored) - Comment links:
rel="ugc"(ornofollow ugc) - General untrusted:
rel="nofollow"
Combining Noindex and Nofollow
Common Combinations
Noindex, follow (most common):
<meta name="robots" content="noindex, follow">- Don’t index this page
- DO follow links on this page
Use for: Thank you pages, search results, tag archives
Noindex, nofollow (strict):
<meta name="robots" content="noindex, nofollow">- Don’t index this page
- Don’t follow links on this page
Use for: Private pages, low-quality scraped content, spam
Index, nofollow (rare):
<meta name="robots" content="index, nofollow">- Index this page
- Don’t follow links on this page
Use for: Disclaimer pages, pages with many affiliate links
Index, follow (default):
<meta name="robots" content="index, follow">
<!-- Or just omit the tag entirely -->- Index this page (it can rank)
- Follow links on this page
Default behavior if no robots meta tag is present.
Common Mistakes
Noindexing Important Pages
Scenario: You accidentally add noindex to your homepage or key product pages.
How it happens:
- Yoast SEO checkbox accidentally ticked
- Staging site setting copied to production
- WordPress “Discourage search engines” setting enabled
Result: Pages disappear from Google within weeks.
Fix:
- Remove the noindex tag
- Request re-indexing in Google Search Console
- Check other important pages with site search:
If key pages are missing, check for noindex.site:example.com
Blocking Crawling AND Using Noindex
Wrong:
# In robots.txt:
Disallow: /thank-you/
# On thank-you page:
<meta name="robots" content="noindex">Problem: Google can’t crawl the page, so it never sees the noindex tag.
Result: Page may still appear in search results (with “A description for this result is not available” text).
Fix: Remove the robots.txt block. Let Google crawl it so it can see the noindex tag.
Right approach:
- Allow crawling in robots.txt
- Add noindex meta tag to the page
- Google crawls → sees noindex → removes from results
Nofollowing Internal Links
Bad practice:
<a href="/about" rel="nofollow">About Us</a>
<a href="/contact" rel="nofollow">Contact</a>Why it’s bad:
- Wastes your own PageRank (doesn’t pass to your own pages)
- Confuses Google about your site structure
- Slows down discovery of new pages
When it’s okay:
- Login/signup links (low SEO value)
- Cart/checkout pages (transactional, not SEO targets)
Using Nofollow for Duplicate Content
Wrong assumption: “I’ll nofollow the duplicate page to avoid a penalty.”
Reality: Nofollow doesn’t fix duplicate content. Use:
- Canonical tags to point to the original
- 301 redirects to merge duplicates
- Noindex if you can’t consolidate
Robots Meta Tag vs X-Robots-Tag
Meta Tag (HTML pages)
<head>
<meta name="robots" content="noindex, nofollow">
</head>Pros:
- Easy to add
- Works in all CMS platforms
Cons:
- Only works for HTML pages
- Can’t apply to PDFs, images, etc.
X-Robots-Tag (HTTP header)
Server-level implementation:
# Apache .htaccess
<FilesMatch "\.pdf$">
Header set X-Robots-Tag "noindex, nofollow"
</FilesMatch># Nginx config
location ~* \.pdf$ {
add_header X-Robots-Tag "noindex, nofollow";
}Pros:
- Works on ANY file type (PDFs, images, videos)
- Can apply to entire directories
- Harder for users to accidentally change
Cons:
- Requires server access
- More technical to implement
Testing Your Implementation
View Page Source
- Visit the page
- Right-click → View Page Source
- Search for
<meta name="robots"
Google Search Console
- URL Inspection tool
- Enter your URL
- Check “Coverage” section for indexing status
Site Search
site:example.com "exact page title"If the page appears in results:
- Noindex is not working (or not processed yet)
If it doesn’t appear:
- ✓ Noindex is working
Note: Changes can take 1-4 weeks to fully process.
Browser Extensions
Quick Decision Guide
Should I use noindex?
| Scenario | Use Noindex? |
|---|---|
| Thank you page after form submit | Yes |
Search results page (/search?q=) | Yes |
| Paginated pages (page 2, 3, 4…) | Maybe (use rel=“prev/next” instead) |
| Category/tag pages with thin content | Yes |
| Duplicate product pages | Yes (or use canonical tag) |
| Admin/login pages | Yes (+ password protect) |
| Pages you want to rank | No |
| Your homepage | No (unless staging site) |
Should I use nofollow?
| Scenario | Use Nofollow? |
|---|---|
| Affiliate link | Yes (rel="sponsored") |
| Paid advertisement | Yes (rel="sponsored") |
| Comment link | Yes (rel="ugc") |
| Link to your own pages | No |
| Editorial link to trusted source | No |
| Login/cart links | Maybe (low priority) |
What Surmado Checks
Surmado Scan looks for:
- Important pages accidentally noindexed (homepage, key products)
- Noindex + robots.txt block conflict
- High-value pages with nofollow on internal links
- Inconsistent indexing directives
→ Related: Robots.txt Essentials | Canonical URLs | Crawl Budget
Was this helpful?
Thanks for your feedback!
Have suggestions for improvement?
Tell us moreHelp Us Improve This Article
Know a better way to explain this? Have a real-world example or tip to share?
Contribute and earn credits:
- Submit: Get $25 credit (Signal, Scan, or Solutions)
- If accepted: Get an additional $25 credit ($50 total)
- Plus: Byline credit on this article