Link rot

"Broken Link" redirects here. For theStar Trek: Deep Space Nineepisode, seeBroken Link (Star Trek: Deep Space Nine).

For link rot on Wikipedia, seeWikipedia:Link rot.

A rotten link usually leads to an error message

Link rot(also calledlink death,link breaking,orreference rot) is the phenomenon ofhyperlinkstending over time to cease to point to their originally targetedfile,web page,orserverdue to that resource being relocated to a new address or becoming permanently unavailable. A link that no longer points to its target, often called abroken,dead,ororphanedlink, is a specific form ofdangling pointer.

The rate of link rot is a subject of study and research due to its significance to theinternet's ability to preserve information. Estimates of that rate vary dramatically between studies. Information professionals have warned that link rot could make important archival data disappear, potentially impacting the legal system and scholarship.

Commonly, broken website links may immediately redirect the user to the home page of the website, confusing users even more and resulting in it being difficult to obtain the URL of the broken link.

Prevalence

A number of studies have examined the prevalence of link rot within theWorld Wide Web,in academic literature that usesURLsto cite web content, and withindigital libraries.

A 2002 study suggested that link rot within digital libraries is considerably slower than on the web, finding that about 3% of the objects were no longer accessible after one year(equating to ahalf-lifeof nearly 23 years).

A 2003 study found that on the Web, about one link out of every 200 broke each week,suggesting ahalf-lifeof 138 weeks. This rate was largely confirmed by a 2016–2017 study of links inYahoo! Directory(which had stopped updating in 2014 after 21 years of development) that found the half-life of the directory's links to be two years.

A 2004 study showed that subsets of Web links (such as those targeting specific file types or those hosted by academic institutions) could have dramatically different half-lives.The URLs selected for publication appear to have greater longevity than the average URL. A 2015 study by Weblock analyzed more than 180,000 links from references in the full-text corpora of three major open access publishers and found a half-life of about 14 years,generally confirming a 2005 study that found that half of theURLscited inD-Lib Magazinearticles were active 10 years after publication.Other studies have found higher rates of link rot in academic literature but typically suggest a half-life of four years or greater.A 2013 study inBMC Bioinformaticsanalyzed nearly 15,000 links in abstracts from Thomson Reuters'sWeb of Sciencecitation index and found that the median lifespan of web pages was 9.3 years, and just 62% were archived.A 2021 study of external links inNew York Timesarticles published between 1996 and 2019 found a half-life of about 15 years (with significant variance among content topics) but noted that 13% of functional links no longer lead to the original content—a phenomenon calledcontent drift.

A 2013 study found that 49% of links in U.S. Supreme court opinions are dead.

A 2023 study looking at United StatesCOVID-19dashboards found that 23% of the state dashboards available in February 2021 were no longer available at the previous URLs in April 2023.

Pew Researchfound that, in 2023, 38% of pages from 2013 went missing. Also in 2023, 54% ofEnglish Wikipediaarticles had a dead link in the 'references' section and 23% ofnews articleslinked to a dead URL.

Causes

Link rot can result from several occurrences. A target web page may be removed. The server that hosts the target page could fail, be removed from service, or relocate to a newdomain name.As far back as 1999, it was noted that with the amount of material that can be stored on a hard drive, "a single disk failure could be like the burning of the library at Alexandria."A domain name's registration may lapse or be transferred to another party. Some causes will result in the link failing to find any target and returning an error such asHTTP 404.Other causes will cause a link to target content other than what was intended by the link's author.

Other reasons for broken links include:

• the restructuring of websites that causes changes in URLs (e.g.domain.net/pine_treemight be moved todomain.net/tree/pine)
• relocation of formerly free content to behind apaywall
• a change in server architecture that results in code such asPHPfunctioning differently
• dynamic page content such as search results that changes by design
• deletion of the target page and/or its content
• the presence of user-specific information (such as a login name) within the link
• deliberate blocking bycontent filtersorfirewalls
• the expiration of adomain name registration

Prevention and detection

Strategies for preventing link rot can focus on placing content where its likelihood of persisting is higher, authoring links that are less likely to be broken, taking steps to preserve existing links, or repairing links whose targets have been relocated or removed.

The creation of URLs that will not change with time is the fundamental method of preventing link rot. Preventive planning has been championed byTim Berners-Leeand other web pioneers.

Strategies pertaining to the authorship of links include:

• linking to primary rather than secondary sources and prioritizing stable sites
• avoiding links that point to resources on researchers' personal pages
• usingclean URLsor otherwise employingURL normalizationorURL canonicalization
• usingpermalinksandpersistent identifierssuch as ARKs,DOIs,Handle System references,PURLs,orcontent addressing
• avoiding linking to documents other than web pages
• avoidingdeep linking
• linking toweb archivessuch as theInternet Archive,WebCite,archive.today,Perma.cc,Amber,or Arweave

Strategies pertaining to the protection of existing links include:

• usingredirectionmechanisms such asHTTP 301to automatically refer browsers and crawlers to relocated content.
• usingcontent management systemswhich can automatically update links when content within the same site is relocated or automatically replace links with canonical URLs
• integrating search resources intoHTTP 404pages

The detection of broken links may be done manually or automatically. Automated methods includeplug-insforcontent management systemsas well as standalone broken-link checkers such as likeXenu's Link Sleuth.Automatic checking may not detect links that return asoft 404or links that return a200 OKresponse but point to content that has changed.

See also

• Archive Team,web archiving team
• Dead Internet theory
• Deletionism and inclusionism in Wikipedia
• Digital preservation
• Infodemic
• Software rot

Further reading

• Markwell, John; Brooks, David W. (2002). "Broken Links: The Ephemeral Nature of Educational WWW Hyperlinks".Journal of Science Education and Technology.11(2): 105–108.doi:10.1023/A:1014627511641.S2CID60802264.
• Gomes, Daniel; Silva, Mário J. (2006)."Modelling Information Persistence on the Web"(PDF).Proceedings of the 6th International Conference on Web Engineering.ICWE'06. Archived fromthe original(PDF)on 2011-07-16.Retrieved14 September2010.
• Dellavalle, Robert P.; Hester, Eric J.; Heilig, Lauren F.; Drake, Amanda L.; Kuntzman, Jeff W.; Graber, Marla; Schilling, Lisa M. (2003)."Going, Going, Gone: Lost Internet References".Science.302(5646): 787–788.doi:10.1126/science.1088234.PMID14593153.S2CID154604929.
• Koehler, Wallace (1999). "An Analysis of Web Page and Web Site Constancy and Permanence".Journal of the American Society for Information Science.50(2): 162–180.doi:10.1002/(SICI)1097-4571(1999)50:2<162::AID-ASI7>3.0.CO;2-B.
• Sellitto, Carmine (2005)."The impact of impermanent Web-located citations: A study of 123 scholarly conference publications"(PDF).Journal of the American Society for Information Science and Technology.56(7): 695–703.CiteSeerX10.1.1.473.2732.doi:10.1002/asi.20159.

External links

• Future-Proofing Your URIs
• Nielsen, Jakob(14 June 1998)."Fighting Linkrot".Archived fromthe originalon 23 December 2012.