Which?pages your site promotes as the most important is critical for search engine optimization. Remembering the details of past internal linking structures is impossible, unless you use the Wayback Machine for SEO time travel.
For example, team members at iHerb probably remember very clearly that they launched a dramatic redesign in the spring of 2015. They might remember that the tabbed ?Best Selling? feature was a new addition on the home page, and that the header navigation changed, as shown below.
iHerb.com’s home page in February 2015 did not include the “Best Selling” tab in the middle of the page, among other differences from the current site.
—
iHerb.com’s February 2016 home page has a “Best Selling” section in the middle, as well as category names in the green bar that are different from February 2015.
But does iHerb’s staff remember that they actually had two redesigns in April 2015? And do they remember exactly which URLs were added to and removed from the header navigation in each case?
The Wayback Machine remembers.
The design of the cached version of the home page on Archive.org?s Wayback Machine changed between April 15 and April 17, 2015, and again on April 30, 2015.
Before and after the two iHerb.com redesigns in April 2015.
The home page is an important page, without question. The pages linked from the home page have a higher amount of internal link authority passed to them than pages that aren?t linked to from the home page. For more on link authority and how it flows through a site, see my article ?SEO: How Google PageRank Works on an Ecommerce Site.?
More important than just looking at the home page, the Wayback Machine can show you snapshots of other pages on the site, the source code of each page, the links that were and were not available on each page, and what can be crawled like a regular web site. Anything you can do on a regular web page you can do on the Wayback Machine, provided it has cached the page you?re looking for in the timeframe you want.
While the home page is very powerful for SEO, the header and footer navigation on every page of the site are even more powerful. Just as the home page passes some of its large amount of link authority to the pages it links to, each link in the header and footer passes some of its link authority to every page linked to in the header and footer.
Every page in your header and footer are receiving the most internal link authority possible. Consequently, along with the home page of the site, the pages in the header and footer have the best chance of ranking and driving organic search traffic and sales.
Tracking which pages are added to and removed from the header and footer navigation is critically important to understanding your organic search performance. And when you need to remember which pages were or were not linked to from the header and footer, the Wayback Machine remembers.
Based on analysis of the source code for the header navigation on April 15, 17, and 30, we can see that the pages linked to were inconsistent. This is very common in a redesign, and something to watch for carefully because it affects the flow of link authority to the pages removed and therefore their ability to rank.
In the table below you can see that the number of links in the header navigation grew between April 15 and 17 by about 10 percent. However, the pages for ?Attitude? and ?Blood Pressure? were removed from the April 17 navigation. ?Blood Pressure? was added back into the April 30 navigation, but the link to the brand ?Ark Naturals? was removed.
The number of links in the header navigation changed three times for iHerb.com in April 2015 ? from 354 on April 15, to 391 on April 17, to 393 on April 30.
This kind of adding and removing of links in the navigation is disruptive to the ability of those pages to drive organic search traffic and sales. It?s akin to telling someone that she’s your best friend, then changing your mind the next day and saying that you don?t really care about her. Pages whose links have been added and removed will be diminished in their ability to perform because they won?t continue to have the higher amount of internal link authority that they previously had.
The same thing happens when you change URLs in the navigation ? when the page name stays the same but the URL for the page changes. That new URL is an entirely new page to the search engines, even if you see the same content when the page loads. You?ve declared a new best friend in your navigation, and diminished the old page?s ability to perform.
If you 301 redirected the old URL to the new, then you?ve done what you can to pass the link authority to the new page. The performance should be similar to the old page. But if you changed the URL, linked to the new URL, and left the old URL live as an orphaned page, performance will suffer.
The individual pages and URLs linked to in the navigation are nearly impossible to remember from month to month, and they may change without you knowing it based on some other update in your development team?s release cycle or decision your merchandisers made. You may see symptoms of the problem in your analytics and have no idea why certain previously performing pages are not performing well now.
The Wayback Machine gives you the ability to discover and analyze the changes in your site?s design and navigation in an effort to identify why your site?s performance has changed.
Archive.org has limited resources and can?t crawl everywhere at once. Sites with fewer links to them from other sites will have fewer ? or no ? snapshots available on archive.org. If your site hasn?t ever been crawled, you can request a one-time crawl using the ?Save this URL? link shown below on the error page.
The link to save a page on the Wayback Machine?s error page.
Visit the Wayback Machine regularly to learn more about what has changed on your site that you might not have known about, and to indicate your interest in what it captures. The Wayback Machine doesn?t guarantee inclusion, but it?s something you can do to try to boost the number of your site?s snapshots for future use.