Webmastersite.net

Search    Register    Log In   

Details

OpenCage and Wayback Links
By
Nov 24, 2018 at 7:19 AM

Turns out Geocodio only supports USA and Canada, and for some reason they don't bother to mention that anywhere prominent. So I had to quickly do more research and found that OpenCage appears the best worldwide geocoding option, with a permissive TOS. OpenCage's data is "interpolated" rather than "rooftop" level accuracy, meaning the pin should be close enough to see your destination but not necessarily exactly precise, so Geocodio is better for USA/Canada addresses. I've set up WSN so that (if all the API keys have been entered) it uses Geocodio on addresses that contain "United States" or "Canada" but uses OpenCage for all other countries. If you still want to use the google maps geocoder even though it breaks the TOS, WSN will fall back to that if you have no other API keys entered.

Today's project is for WSN Links. As you've no doubt noticed, link rot is a big problem these days -- websites are coming and going all the time, and reorganizing their URLs without leaving redirects too. WSN's dead link checker and content checker help you catch these, but just having to delete stuff all the time isn't ideal. Thus, I'm adding an option to convert the listings found dead into archive.org wayback machine links. The wayback machine, as you probably know, keeps an incomplete but extensive archive of historical webpages.

Now, there's a couple ways to do that. One way is to just request the latest archived version of the URL -- simple. But what if the website has changed or might eventually change into a spammy page full of viagra links? Archive.org would happily archive the spammy page and serve it up. So I think it's better to instead request a version of the page from a specific time in the past when we know the site was good. The date on which the link was submitted to your site makes the most sense to me. So, when you tell WSN to convert a link to the wayback machine version, it will automatically request the version from the date of the link submission. Archive.org will serve up the soonest it has after that date.

If the dead link checker finds that an archive.org link is serving up a 404 error (which the wayback machine will do when it doesn't have a copy of the page), it'll then automatically suspend and hide the link when you try to archive it again.




Views 32 views. Averaging 2 views per day.
Similar Listings
Microdata and Password Reset Links

Sorry, you don't have permission to post. Log in, or register if you haven't yet.

Please login or register.