On large sites, Googlebot has a finite amount of attention. Every crawl spent on a duplicate filter combination, a soft-404, or an orphaned parameter URL is a crawl not spent on the pages you actually want indexed.
Where the waste hides
- Faceted navigation generating thousands of near-duplicate URL combinations.
- Soft 404s that return a 200 status but show empty or "not found" content.
- Legacy redirect chains stacking three or four hops deep.
The fix
Audit your log files to see what Googlebot is actually requesting, block low-value parameter combinations in robots.txt, consolidate duplicate content with canonical tags, and flatten redirect chains to a single hop.