Screaming Frog’s SEO Spider is a desktop tool used for auditing sites by letting you crawl an entire sites images, links, CSS, scripts etc on demand from an SEO perspective. But that’s not all it can be used for! Below is a list of some other ways Screaming Frog can be used for SEO.
1. Creating a Sitemap
Screaming Frog provides a really easy way to create an XML sitemap, but it’s important that you configure the crawl settings properly before you begin so that you don’t end up with URL’s that shouldn’t be in there. This is especially the case with WordPress sites which tend to create pages for example when you upload image attachments, or someone comments on your post. These types of pages you will not want indexed nor will you want them in your sitemap. All you have to do is go to the configuration settings and use the exclude tool. Here is a video explaining the Screaming Frog SEO Spider Exclude Function.
2. Auditing Backlinks
For those of you who like to keep tabs on your backlinks, or during certain processes of outreach where you might have a big list of websites you are expecting backlinks from, this can be a tiresome process. However Screaming Frogs list mode can help you speed up the process by checking a list of URL’s very quickly. For a more detailed explanation of how you can use Screaming Frog to audit your backlinks you can view their blog article here.
3. Link Building
Often when doing outreach it’s easier to make that first point of contact through platforms like Twitter to get yourself on the target’s radar. A nice innocent way of letting them know you read their blog and to draw attention to yourself is to crawl their site using Screaming Frog and let them know of any blog posts which return a 404 error. This way, when you actual email them for outreach, it isn’t completely out of the blue and they may recognise your name or face, giving you a better chance of reply.
4. Verifying Crawl Errors on GWT
Although Google Webmaster Tools is a great tool for auditing sites and has a number of useful features, it doesn’t always update the crawl errors frequently enough. This means that sometimes 404 errors can be flagged up on GWT which have actually already been fixed. Screaming Frog is a tool which can be used to solve this problem, and below are the steps you can take to achieve this:
- Crawl the site using Screaming Frog and put any 404 errors into a spreadsheet
- Download any 404 errors from GWT and add them to the spreadsheet
- Remove any duplicates
- Copy, paste and save all of the 404 errors into a text document
- Upload the 404.txt document and re-crawl using the Screaming Frog list mode
- From this you can then export any remaining 404 errors and fix them