Using Screaming Frog SEO Spider to Crawl and Detect Broken Links

To use Screaming Frog SEO Spider to crawl and detect broken links, follow these key steps:

1. Start a Crawl: Enter your website URL in Screaming Frog and click the "Start" button to begin crawling your site. The tool will scan all accessible pages and links.

2. Locate Broken Links: Once the crawl completes (indicated by 100% at the bottom right), go to the "Response Codes" tab. Set the filter to "Client Errors (4XX)" to isolate broken links, typically 404 errors.

3. Analyze Sources of Broken Links: Select a broken link URL to view its "Inlinks" tab, which shows the pages linking to the broken URL, anchor text, and link attributes (follow/nofollow). This helps identify where the broken links originate.

4. Export Results: Export the list of broken links and their source URLs via Bulk Export > Response Code > Client Errors (4XX) for further analysis or to share with developers for fixing.

Additional tips:

• If you want to check a specific list of URLs rather than crawling a whole site, use List Mode (Mode > List), where you can upload or paste URLs to check for broken links without crawling the entire site.

• To detect broken jump links (anchors/bookmarks within pages), enable "Crawl Fragment Identifiers" in Config > Spider > Advanced and filter for broken bookmarks after crawling.

• The free version of Screaming Frog allows crawling up to 500 URLs, suitable for small sites. Larger sites require a paid license.

This method efficiently identifies broken internal and external links causing 404 errors, helping improve user experience and SEO by fixing or removing these links.