Module:scripts

The followingdocumentationis located atModule:scripts/documentation.^[edit]

Useful links:subpage list•links•transclusions•testcases•sandbox

This module is used to retrieve and manage Wiktionary's various writing systems and the information associated with them. SeeWiktionary:Scriptsfor more information.

The information itself is stored inModule:scripts/data.The data module shouldnotbe used directly by any other module, the data should only be accessed through the functions provided by Module:scripts.

For functions that allow templates to use this module, seeModule:scripts/templates.

Finding and retrieving scripts

The module exports a number of functions that are used to find scripts.

export.makeObject

functionexport.makeObject(code,data,useRequire)

export.getByCode

functionexport.getByCode(code,paramForError,disallowNil,useRequire)

Finds the script whose code matches the one provided. If it exists, it returns aScriptobject representing the script. Otherwise, it returnsnil,unlessparamForErroris given, in which case an error is generated. IfparamForErroristrue,a generic error message mentioning the bad code is generated; otherwiseparamForErrorshould be a string or number specifying the parameter that the code came from, and this parameter will be mentioned in the error message along with the bad code.

export.getByCanonicalName

functionexport.getByCanonicalName(name,useRequire)

export.charToScript

functionexport.charToScript(char)

Takes a codepoint or a character and finds the script code (if any) that is appropriate for it based on the codepoint, using the data moduleModule:scripts/recognition data.The data module was generated from the patterns inModule:scripts/datausingModule:User:Erutuon/script recognition.

Converts the character to a codepoint. Returns a script code if the codepoint is in the list of individual characters, or if it is in one of the defined ranges in the 4096-character block that it belongs to, else returns "None".

export.findBestScriptWithoutLang

functionexport.findBestScriptWithoutLang(text,none_is_last_resort_only)

Returns the code for the script that has the greatest number of characters intext.Useful for script tagging text that is unspecified for language. UsesModule:scripts/recognition datato determine a script code for a character language-agnostically. Specifically, it works as follows: Convert each character to a codepoint. Iterate the counter for the script code if the codepoint is in the list of individual characters, or if it is in one of the defined ranges in the 4096-character block that it belongs to. Each script has a two-part counter, for primary and secondary matches. Primary matches are when the script is the first one listed; otherwise, it's a secondary match. When comparing scripts, first the total of both are compared (i.e. the overall number of matches). If these are the same, the number of primary and then secondary matches are used as tiebreakers. For example, this is used to ensure thatGrektakes priority overPolytif no characters which exclusively matchPolytare found, asGrekis a subset ofPolyt.Ifnone_is_last_resort_onlyis specified, this will never return"None"if any characters intextbelong to a script. Otherwise, it will return"None"if there are more characters that don't belong to a script than belong to any individual script. (FIXME: This behavior is probably wrong, andnone_is_last_resort_onlyshould probably become the default.)

Script objects

AScriptobject is returned from one of the functions above. It is a Lua representation of a script and the data associated with it. It has a number of methods that can be called on it, using the:syntax. For example:

localm_scripts=require("Module:scripts")
localsc=m_scripts.getByCode("Latn")
localname=sc:getCanonicalName()
-- "name" will now be "Latin"

Script:getCode

functionScript:getCode()

Returns the script code of the script. Example:"Cyrl"for Cyrillic.

Script:getCanonicalName

functionScript:getCanonicalName()

Returns the canonical name of the script. This is the name used to represent that script on Wiktionary. Example:"Cyrillic"for Cyrillic.

Script:getDisplayForm

functionScript:getDisplayForm()

Returns the display form of the script. For scripts, this is the same as the value returned by:getCategoryName( "nocap" ),i.e. it reads "NAME script" (e.g."Arabic script"). The displayed text used in:makeCategoryLinkis always the same as the display form.

Script:getOtherNames

functionScript:getOtherNames(onlyOtherNames)

Script:getAliases

functionScript:getAliases()

Script:getVarieties

functionScript:getVarieties(flatten)

Script:getIETFSubtag

functionScript:getIETFSubtag()

Returns theIETF subtagused for the script, which should always be a validISO 15924script code. This is used when constructing HTMLlang=tags. Theietf_subtagvalue from the script's data file is used, if present; otherwise, the script code is used. For script codes which contain a hyphen, only the part after the hyphen is used (e.g."fa-Arab"becomes"Arab").

Script:getParent

functionScript:getParent()

Returns the parent of the script. Example:"Arab"for"fa-Arab".It returns"top"for scripts without a parent, like"Latn","Grek",etc.

Script:getSystemCodes

functionScript:getSystemCodes()

Script:getSystems

functionScript:getSystems()

Script:isSystem

functionScript:isSystem(...)

Check whether the script is of typesystem,which can be a writing system code or object. If multiple systems are passed, return true if the script is any of the specified systems.

Script:hasType

functionScript:hasType(...)

Given a list of types as strings, returns true if the script has all of them.

Currently the only possible type isscript;usehasType("script")to determine if an object that may be a language, family or script is a script.

Script:getCategoryName

functionScript:getCategoryName(nocap)

Returns the name of the main category of that script. Example:"Cyrillic script"for Cyrillic, whose category is atCategory:Cyrillic script.Unless optional argumentnocapis given, the script name at the beginning of the returned value will be capitalized. This capitalization is correct for category names, but not if the script name is lowercase and the returned value of this function is used in the middle of a sentence. (For example, the script with the codeSemaphas the name"flag semaphore",which should remain lowercase when used as part of the category nameCategory:Translingual letters in flag semaphorebut should be capitalized inCategory:Flag semaphore templates.) If you are considering usinggetCategoryName( "nocap" ),usegetDisplayForm()instead.

Script:makeCategoryLink

functionScript:makeCategoryLink()

Script:getWikipediaArticle

functionScript:getWikipediaArticle()

Returns thewikipedia_articleitem in the script's data file, or else callsScript:getCategoryName().

Script:getCharacters

functionScript:getCharacters()

Returns the charset defining the script's characters from the script's data file. This can be used to search for words consisting only of this script, but see the warning above.

Script:countCharacters

functionScript:countCharacters(text)

Returns the number of characters in the text that are part of this script.Note:You should never assume that text consists entirely of the same script. Strings may contain spaces, punctuation and even wiki markup or HTML tags. HTML tags will skew the counts, as they contain Latin-script characters. So it's best to avoid them.

Script:hasCapitalization

functionScript:hasCapitalization()

Script:hasSpaces

functionScript:hasSpaces()

Script:isTransliterated

functionScript:isTransliterated()

Script:sortByScraping

functionScript:sortByScraping()

Returns true if the script is (sometimes) sorted by scraping page content, meaning that it is sensitive to changes in capitalization during sorting.

Script:getDirection

functionScript:getDirection()

Returns the text direction. Horizontal scripts return"ltr"(left-to-right) or"rtl"(right-to-left), while vertical scripts return"vertical-ltr"(vertical left-to-right) or"vertical-rtl"(vertical right-to-left).

Script:getRawData

functionScript:getRawData()

Script:hasNormalizationFixes

functionScript:hasNormalizationFixes()

Returnstrueif the script contains characters that require fixes to Unicode normalization under certain circumstances,falseif it doesn't.

Script:fixDiscouragedSequences

functionScript:fixDiscouragedSequences(text)

Corrects discouraged sequences of Unicode characters to the encouraged equivalents.

Script:toFixedNFC

functionScript:toFixedNFC(text)

Script:toFixedNFD

functionScript:toFixedNFD(text)

Script:toFixedNFKC

functionScript:toFixedNFKC(text)

Script:toFixedNFKD

functionScript:toFixedNFKD(text)

Script:toJSON

functionScript:toJSON()

Subpages

scripts/by name scripts/by name/documentation scripts/chain scripts/chain/documentation scripts/charToScript scripts/charToScript/documentation scripts/code to canonical name scripts/code to canonical name/documentation scripts/codes/documentation scripts/data scripts/data/documentation scripts/documentation scripts/print scripts/print/documentation scripts/recognition data scripts/recognition data/documentation scripts/style.css scripts/templates scripts/templates/documentation scripts/testcases scripts/testcases/documentation

See also

• Module:script utilities

---

localm_str_utils=require("Module:string utilities")

localconcat=table.concat
localexplode=m_str_utils.explode_utf8
localgsplit=m_str_utils.gsplit
localmatch=string.match
localselect=select
localsplit=m_str_utils.split
localtoNFC=mw.ustring.toNFC
localtoNFD=mw.ustring.toNFD
localtoNFKC=mw.ustring.toNFKC
localtoNFKD=mw.ustring.toNFKD
localtype=type
localugsub=m_str_utils.gsub
localumatch=m_str_utils.match

localexport={}

localScript={}

--[==[Returns the script code of the script. Example: {{lua| "Cyrl" }} for Cyrillic.]==]
functionScript:getCode()
returnself._code
end

--[==[Returns the canonical name of the script. This is the name used to represent that script on Wiktionary. Example: {{lua| "Cyrillic" }} for Cyrillic.]==]
functionScript:getCanonicalName()
returnself._rawData[1]orself._rawData.canonicalName
end

--[==[Returns the display form of the script. For scripts, this is the same as the value returned by <code>:getCategoryName( "nocap" )</code>, i.e. it reads "NAME script" (e.g. {{lua| "Arabic script" }}). The displayed text used in <code>:makeCategoryLink</code> is always the same as the display form.]==]
functionScript:getDisplayForm()
returnself:getCategoryName("nocap")
end

functionScript:getOtherNames(onlyOtherNames)
returnrequire("Module:language-like").getOtherNames(self,onlyOtherNames)
end

functionScript:getAliases()
returnself._rawData.aliasesor{}
end

functionScript:getVarieties(flatten)
returnrequire("Module:language-like").getVarieties(self,flatten)
end

--[==[Returns the {{w|IETF language tag#Syntax of language tags|IETF subtag}} used for the script, which should always be a valid {{w|ISO 15924}} script code. This is used when constructing HTML {{code|html|lang{{=}}}} tags. The {{lua|ietf_subtag}} value from the script's data file is used, if present; otherwise, the script code is used. For script codes which contain a hyphen, only the part after the hyphen is used (e.g. {{lua| "fa-Arab" }} becomes {{lua| "Arab" }}).]==]
functionScript:getIETFSubtag()
localcode=self._ietf_subtag
ifcode==nilthen
code=self._rawData.ietf_subtagormatch(self._code,"[^%-]+$")
self._ietf_subtag=code
end
returncode
end

--[==[Returns the parent of the script. Example: {{lua| "Arab" }} for {{lua| "fa-Arab" }}. It returns {{lua| "top" }} for scripts without a parent, like {{lua| "Latn" }}, {{lua| "Grek" }}, etc.]==]
functionScript:getParent()
returnself._rawData.parent
end

functionScript:getSystemCodes()
ifnotself._systemCodesthen
localsystem_codes=self._rawData[2]
iftype(system_codes)=="table"then
self._systemCodes=system_codes
elseiftype(system_codes)=="string"then
self._systemCodes=split(system_codes,"%s*,%s*",true)
else
self._systemCodes={}
end
end
returnself._systemCodes
end

functionScript:getSystems()
ifnotself._systemObjectsthen
localm_systems=require("Module:writing systems")
self._systemObjects={}

for_,wsinipairs(self:getSystemCodes())do
table.insert(self._systemObjects,m_systems.getByCode(ws))
end
end

returnself._systemObjects
end

--[==[Check whether the script is of type `system`, which can be a writing system code or object. If multiple systems are passed, return true if the script is any of the specified systems.]==]
functionScript:isSystem(...)
for_,systeminipairs{...}do
iftype(system)=="table"then
system=system:getCode()
end
for_,sinipairs(self:getSystemCodes())do
ifsystem==sthen
returntrue
end
end
end
returnfalse
end

--function Script:getAllNames()
-- return self._rawData.names
--end

--[==[Given a list of types as strings, returns true if the script has all of them.

Currently the only possible type is {script}; use {{lua|hasType( "script" )}} to determine if an object that
may be a language, family or script is a script.
]==]
functionScript:hasType(...)
localtypes=self._types
iftypes==nilthen
types={script=true}
localrawtypes=self._rawData.type
ifrawtypesthen
forrawtypeingsplit(rawtypes,"%s*,%s*",true)do
types[rawtype]=true
end
end
self._types=types
end
fori=1,arg.ndo
ifnottypes[arg[i]]then
returnfalse
end
end
returntrue
end

--[==[Returns the name of the main category of that script. Example: {{lua| "Cyrillic script" }} for Cyrillic, whose category is at [[:Category:Cyrillic script]].
Unless optional argument <code>nocap</code> is given, the script name at the beginning of the returned value will be capitalized. This capitalization is correct for category names, but not if the script name is lowercase and the returned value of this function is used in the middle of a sentence. (For example, the script with the code <code>Semap</code> has the name <code> "flag semaphore" </code>, which should remain lowercase when used as part of the category name [[:Category:Translingual letters in flag semaphore]] but should be capitalized in [[:Category:Flag semaphore templates]].) If you are considering using <code>getCategoryName( "nocap" )</code>, use <code>getDisplayForm()</code> instead.]==]
functionScript:getCategoryName(nocap)
localname=self:getCanonicalName()

-- If the name already has "script", "code" or "semaphore" at the end, don't add it.
ifnot(
name:find("[ %-][Ss]cript$")or
name:find("[ %-][Cc]ode$")or
name:find("[ %-][Ss]emaphore$")
)then
name=name.."script"
end
ifnotnocapthen
name=mw.getContentLanguage():ucfirst(name)
end
returnname
end

functionScript:makeCategoryLink()
return"[[:Category:"..self:getCategoryName().."|"..self:getDisplayForm().."]]"
end

--[==[Returns the {{lua|wikipedia_article}} item in the script's data file, or else calls {{lua|Script:getCategoryName()}}.]==]
functionScript:getWikipediaArticle()
returnself._rawData.wikipedia_articleorself:getCategoryName()
end

--[==[Returns the charset defining the script's characters from the script's data file.
This can be used to search for words consisting only of this script, but see the warning above.]==]
functionScript:getCharacters()
returnself.charactersornil
end

--[==[Returns the number of characters in the text that are part of this script.
'''Note:''' You should never assume that text consists entirely of the same script. Strings may contain spaces, punctuation and even wiki markup or HTML tags. HTML tags will skew the counts, as they contain Latin-script characters. So it's best to avoid them.]==]
functionScript:countCharacters(text)
localcharset=self._rawData.characters
ifcharset==nilthen
return0
end
returnselect(2,ugsub(text,"["..charset.."]",""))
end

functionScript:hasCapitalization()
returnnotnotself._rawData.capitalized
end

functionScript:hasSpaces()
returnself._rawData.spaces~=false
end

functionScript:isTransliterated()
returnself._rawData.translit~=false
end

--[==[Returns true if the script is (sometimes) sorted by scraping page content, meaning that it is sensitive to changes in capitalization during sorting.]==]
functionScript:sortByScraping()
returnnotnotself._rawData.sort_by_scraping
end

--[==[Returns the text direction. Horizontal scripts return {{lua| "ltr" }} (left-to-right) or {{lua| "rtl" }} (right-to-left), while vertical scripts return {{lua| "vertical-ltr" }} (vertical left-to-right) or {{lua| "vertical-rtl" }} (vertical right-to-left).]==]
functionScript:getDirection()
returnself._rawData.directionor"ltr"
end

functionScript:getRawData()
returnself._rawData
end

--[==[Returns {{lua|true}} if the script contains characters that require fixes to Unicode normalization under certain circumstances, {{lua|false}} if it doesn't.]==]
functionScript:hasNormalizationFixes()
returnnotnotself._rawData.normalizationFixes
end

--[==[Corrects discouraged sequences of Unicode characters to the encouraged equivalents.]==]
functionScript:fixDiscouragedSequences(text)
ifself:hasNormalizationFixes()then
localnorm_fixes=self._rawData.normalizationFixes
localto=norm_fixes.to
iftothen
fori,vinipairs(norm_fixes.from)do
text=ugsub(text,v,to[i]or"")
end
end
end
returntext
end

do
localcombiningClasses

-- Implements a modified form of Unicode normalization for instances where there are identified deficiencies in the default Unicode combining classes.
localfunctionfixNormalization(text,self)
ifnotself:hasNormalizationFixes()then
returntext
end
localnorm_fixes=self._rawData.normalizationFixes
localnew_classes=norm_fixes.combiningClasses
ifnot(new_classesandumatch(text,"["..norm_fixes.combiningClassCharacters.."]"))then
returntext
end
-- Obtain the list of default combining classes.
combiningClasses=combiningClassesormw.loadData("Module:Unicode data/combining classes")
text=explode(text)
-- Manual sort based on new combining classes.
-- We can't use table.sort, as it compares the first/last values in an array as a shortcut, which messes things up.
fori=2,#textdo
localchar=text[i]
localclass=new_classes[char]orcombiningClasses[char]
ifclassthen
repeat
i=i-1
localprev=text[i]
if(new_classes[prev]orcombiningClasses[prev]or0)<classthen
break
end
text[i],text[i+1]=char,prev
untili==1
end
end
returnconcat(text)
end

functionScript:toFixedNFC(text)
returnfixNormalization(toNFC(text),self)
end

functionScript:toFixedNFD(text)
returnfixNormalization(toNFD(text),self)
end

functionScript:toFixedNFKC(text)
returnfixNormalization(toNFKC(text),self)
end

functionScript:toFixedNFKD(text)
returnfixNormalization(toNFKD(text),self)
end
end

functionScript:toJSON()
ifnotself._typesthen
self:hasType()
end
localtypes={}
fortypeinpairs(self._types)do
table.insert(types,type)
end

localret={
canonicalName=self:getCanonicalName(),
categoryName=self:getCategoryName("nocap"),
code=self._code,
otherNames=self:getOtherNames(true),
aliases=self:getAliases(),
varieties=self:getVarieties(),
type=types,
direction=self:getDirection(),
characters=self:getCharacters(),
parent=self:getParent(),
systems=self:getSystemCodes(),
wikipediaArticle=self._rawData.wikipedia_article,
}

returnrequire("Module:JSON").toJSON(ret)
end

Script.__index=Script

functionexport.makeObject(code,data,useRequire)
returndataandsetmetatable({
_rawData=data,
_code=code,
characters=data.characters
},Script)ornil
end

--[==[Finds the script whose code matches the one provided. If it exists, it returns a {{lua|Script}} object representing the script. Otherwise, it returns {{lua|nil}}, unless <span class= "n" >paramForError</span> is given, in which case an error is generated. If <code class= "n" >paramForError</code> is {{lua|true}}, a generic error message mentioning the bad code is generated; otherwise <code class= "n" >paramForError</code> should be a string or number specifying the parameter that the code came from, and this parameter will be mentioned in the error message along with the bad code.]==]
functionexport.getByCode(code,paramForError,disallowNil,useRequire)
-- Track uses of paramForError, ultimately so it can be removed, as error-handling should be done by [[Module:parameters]], not here.
ifparamForError~=nilthen
require("Module:debug/track")("scripts/paramForError")
end

ifcode==nilandnotdisallowNilthen
returnnil
end

localdata
ifuseRequirethen
data=require("Module:scripts/data")[code]
else
data=mw.loadData("Module:scripts/data")[code]
end

localretval=export.makeObject(code,data,useRequire)

ifnotretvalandparamForErrorthen
require("Module:languages/error")(code,paramForError,"script code",nil,"not real lang")
end

returnretval
end

functionexport.getByCanonicalName(name,useRequire)
localcode
ifuseRequirethen
code=require("Module:scripts/by name")[name]
else
code=mw.loadData("Module:scripts/by name")[name]
end

returnexport.getByCode(code,nil,nil,useRequire)
end

--[==[
Takes a codepoint or a character and finds the script code (if any) that is
appropriate for it based on the codepoint, using the data module
[[Module:scripts/recognition data]]. The data module was generated from the
patterns in [[Module:scripts/data]] using [[Module:User:Erutuon/script recognition]].

Converts the character to a codepoint. Returns a script code if the codepoint
is in the list of individual characters, or if it is in one of the defined
ranges in the 4096-character block that it belongs to, else returns "None".
]==]
functionexport.charToScript(char)
returnrequire("Module:scripts/charToScript").charToScript(char)
end

--[==[
Returns the code for the script that has the greatest number of characters in `text`. Useful for script tagging text
that is unspecified for language. Uses [[Module:scripts/recognition data]] to determine a script code for a character
language-agnostically. Specifically, it works as follows:

Convert each character to a codepoint. Iterate the counter for the script code if the codepoint is in the list
of individual characters, or if it is in one of the defined ranges in the 4096-character block that it belongs to.

Each script has a two-part counter, for primary and secondary matches. Primary matches are when the script is the
first one listed; otherwise, it's a secondary match. When comparing scripts, first the total of both are compared
(i.e. the overall number of matches). If these are the same, the number of primary and then secondary matches are
used as tiebreakers. For example, this is used to ensure that `Grek` takes priority over `Polyt` if no characters
which exclusively match `Polyt` are found, as `Grek` is a subset of `Polyt`.

If `none_is_last_resort_only` is specified, this will never return { "None" } if any characters in `text` belong to a
script. Otherwise, it will return { "None" } if there are more characters that don't belong to a script than belong to
any individual script. (FIXME: This behavior is probably wrong, and `none_is_last_resort_only` should probably
become the default.)
]==]
functionexport.findBestScriptWithoutLang(text,none_is_last_resort_only)
returnrequire("Module:scripts/charToScript").findBestScriptWithoutLang(text,none_is_last_resort_only)
end

returnexport