% language=us runpath=texruns:manuals/fonts \startcomponent fonts-features \environment fonts-environment % windows: seguiemj.ttf (windows 10) % public : emojionecolor-svginot.ttf (https://github.com/eosrei/emojione-color-font) \startMPextensions vardef MyRectangle(expr n, w, h, x, y, c) = image ( fill unitsquare xyscaled (w,h) shifted (x,y) withcolor c ; draw textext("\tttf " & decimal n) xsized (1/2) shifted (w/2,h/2) shifted (x,y) withcolor white ; ) enddef ; vardef MyDot(expr x, y) = image ( draw (x,y) withpen pencircle scaled (2/3) withcolor white ; draw (x,y) withpen pencircle scaled (1/2) withcolor black ; ) enddef ; \stopMPextensions \startchapter[title=Features][color=darkmagenta] \startsection[title=Introduction] If you look into fonts, it is hard not to bump into kerns (spacing between characters) and ligatures (combining multiple shapes into one) and apart from monospaced fonts most \TYPEONE\ fonts have them. In the \OPENTYPE\ universe we call these properties features and in such a font there can be many such features. For those who grew up with \TEX\ or still remember the times of eight bit fonts, it is no secret that \TEX\ macro packages did some magic to get most out of a font: replacing missing glyphs, fixing metrics, using commands to access shapes that had a weird code point, to mention a few. As there is absolutely no guarantee that an \OPENTYPE\ font does better, there is a good reason to continue messing around with fonts. After all, it's what \TEX\ users seem to like: control. So, when we started writing support for \OPENTYPE\ quite soon a mechanism has been created that permits adding our own features to the repertoire that comes with a font. Because \OPENTYPE\ features demand a configuration and control mechanism, it made sense to generalize that and provide a single interface. This means that when we talk about font features, we don't limit ourselves to those provided by the font, but also those provided by \CONTEXT. As with font features, they are enabled per font. Some of the extra features are sort of generic, others are very font specific and their properties are somewhat bound to a font. Such features are defined in a font goodie files. Consider these goodies a font extension mechanism. Some features need information that only the engine can provide. This is why we have analyzers. Some are generic, others are bound to scripts. They come in action before features are applied. Rather special is applying features in combination with paragraph building. This is something very specific to \CONTEXT\ but it depends on properties of the font. It falls into the category \quote {optimizing}. It is clear that when we talk of features many aspects of a font play a role. In this chapter we will discuss all the mentioned aspects. There is quite a bit of \LUA\ code shown in this chapter, but don't worry, users will seldom need to tweak fonts this way. On the other hand it's good to see what is possible. \stopsection \startsection[title=Regulars] \startsubsection[title=Introduction] The \OPENTYPE\ specification, which can be found on the \MICROSOFT\ website is no easy reading. Some of the concepts are easy to understand, like relative positioning (that we call kerning in \TEX) or ligature substitution (as we have ligatures in \TEX\ too). It makes no sense to discuss the bitwise composition of an \OPENTYPE\ or \TRUETYPE\ file here. First of all, all we get to see is a \LUA\ table, and in \CONTEXT\ even that one gets sanitized and optimized into a more useable table. However, as the data that comes with a font is a good indication of what a font is capable of, we will discuss some of it in an appendix. In this section we will discuss the basic principles and categories of features. \stopsubsection \startsubsection[title=Feature sets] Because in the next examples we will demonstrate features, we need to know how we can tell \CONTEXT\ what features to use. Although you can add explicit feature definitions to a font specification, I strongly advice you not to do this but use the more abstract mechanism of feature sets. These are defined as follows: \starttyping \definefontfeature [MyFeatureSet] [alpha=yes, beta=no, gamma=123] \stoptyping Such a set is bound to a font with the \type {*} specifier, as in: \starttyping \definefont [MyFontInstance] [MyNiceFont*MyFeatureSet at 12pt] \stoptyping In most cases the already defined \type {default} feature set will suffice. It often makes sense to use that one as base for new definitions: \starttyping \definefontfeature [MyFeatureSet] [default] [alpha=yes, beta=no, gamma=123] \stoptyping The second argument can be a list, as in: \starttyping \definefontfeature [MyFeatureSet] [MyFirstSet,MySecondSet] [alpha=yes, beta=no, gamma=123] \stoptyping Of course you need to know what features a font support, and one way to find out is: \starttyping mtxrun --script font --list --info --pattern=pagella \stoptyping Don't be too surprised if different fonts show different features and even similar features can be implemented differently. Sometimes you really need to know the font, but fortunately many fonts come with examples. There are many features and there values are kept with the font when it gets defined. This means that when you change a featureset, it will not affect already defined fonts. Because fonts are often defined on demand, you need to be aware of the fact that a redefinition of a featureset can have consequences for already defined fonts. For instance, a bodyfont switch only sets up the fonts and delays defining them. Although features are a sort of abstractions it can be interesting to see what features and values are actually used: \starttyping \usemodule[fonts-features] \showusedfeatures \stoptyping You will notice that we have more features than \OPENTYPE\ fonts can offer. This is because in \CONTEXT\ features is a more general concept. \showusedfeatures \stopsubsection \startsubsection[title=Main categories] There are two (but potentially more) main groups of features: those that deal with substitution, and those that lead to positioning. It is not really needed to know the gory details, but it helps to know at least a bit about them as it can help to track down issues with fonts. There are several substitutions possible: \startitemize[packed] \startitem a single substitution replaces one glyph by another \stopitem \startitem a multiple substitution replaces one glyph by one or more \stopitem \startitem a ligature substitution replaces multiple glyphs by one glyph \stopitem \startitem an alternate substitution replaces one glyph by one out of a set \stopitem \stopitemize Like it or not, but these categories are not always used as intended: they just are a way of replacing one or more glyphs by one or more other glyphs. This means that when for instance \type {ij} gets replaced by one glyph (given that the font supports it) a ligature substitution is used, even when in fact we have to do with a diftong that can be represented by one character. No matter what features you will use, keep in mind that they are nothing more than a combination of substitutions and positioning directives. So, the de facto standard ligature building feature \type {liga} indeed uses a ligature substitution, but other features with names that resemble no ligatures might use this substitution as well. An example of a single substitution is an oldstyle (\type {onum}) although it can as well be implemented as a choice out of alternate glyphs. Another example is smallcaps (\type {smcp}). Nowdays these are more or less standard features for a grown up font, while in the past they came as separate fonts. So, instead of loading an extra font, one sticks to one and selects the feature that does the substitution. A second category concerns relative positioning. Again we have several variants: \startitemize[packed] \startitem a single positioning moves a glyph over one of two axis and can change the width and|/|or height \stopitem \startitem a pair positioning also moved glyphs but concerns two adjacent glyphs \stopitem \startitem a cursive positioning operates on a range of glyphs and is used to visually connect them \stopitem \stopitemize In addition there are three ways to anchor marks onto glyphs: \startitemize[packed] \startitem a mark can be anchored on a base glyph \stopitem \startitem a mark can be anchored on a specific (visual) component of a ligature \stopitem \startitem a mark can be anchored on another mark \stopitem \stopitemize In base mode the single, alternate and ligature substitutions can rather easily be mapped onto the traditional \TEX\ font handling mechanism and this is what happens in base mode. A single substitution is just another instance of a glyph so there we just replace the original index into the glyph table by another one. In the case of an alternate we change the default index into one of several possible replacements in the alternate set. Ligatures can be mapped onto \TEX s ligature mechanism. The single positioning maps nicely on \TEX s kerning mechanism and pairwise positioning is not applicable in base mode. In node mode we don't do any remapping at loading time but delegate that to \LUA\ when processing the node lists. Marks are special in the sense that they normally only occur in scripts that also use substitution and positioning which in turn means that some more housekeeping is involved. After all, we need to keep track to what a mark applies. Of course a font can provide regular latin accents as marks but that is ill practice because cut and paste might not work out as expected. A proper font will support composed characters and provide glyphs that have the accents built in. Marks are not dealt with in base mode. Talking of complex scripts, the above set of operations is far from enough. Take for instance Arabic, where a sequence of 5~characters with 3~marks can easily become two glyphs glued together with two marks only. In the process we can have single substitutions, ligatures being built, marks being anchored and glyphs being cursively positioned. But, in order to do this well, some contextual analysis has to be done as well. Again we have several variants of this: \startitemize[packed] \startitem with contextual substitution a replacement takes place depending on a matching sequence of glyphs, optionally preceded or followed by matches \stopitem \startitem with contextual positioning shifting and anchoring happens based on a matching sequence of glyphs, optionally preceded or followed by matches \stopitem \startitem multiple contextual substitutions or positionings can be chained together \stopitem \startitem this can also happen in the reverse order (for right|-|to|-|left scripts) \stopitem \stopitemize In practice there is no fundamental difference between these and we can collapse them all in a sequence of lookups resulting in a sequence of whatever other manipulation is wanted. Given this, what is a feature? It's mostly a sequence of actions expressed in the above. And although there is a whole repertoire of semi|-|standardized features like \type {liga} and \type {onum}, there is no real hard coded support for them in \CONTEXT. Instead we have a generic feature processor that deals with all of them. A feature, say \type {abcd}, has a definition that boils down to a sequence of lookups. A lookup is just a name that is associated to one of the mentioned actions. So, \type {abcd} can do a decomposition (multiple substitution), then a replacement (single substitution) based on neighbouring glyphs, then do some ligature building (ligature substitution) and finally position the resulting glyphs relative to each other (like cursive positioning and anchoring marks). Imagine that we start out with 5 characters in the input. Instead of real glyphs we represent them by rectangles. The third one is a mark. \startlinecorrection \startMPcode draw MyRectangle(1,2,6, 0,0,.5red ) ; draw MyRectangle(2,2,4, 3,0,.5green ) ; draw MyDot(4,4.25) ; draw MyRectangle(3,2,1, 6,5,.5blue ) ; draw MyDot(7,4.75) ; draw MyRectangle(4,2,5, 9,0,.5yellow ) ; draw MyRectangle(5,2,5,12,0,.5magenta) ; currentpicture := currentpicture ysized(4cm) ; \stopMPcode \stoplinecorrection In the next variant we see that four and five have been replaced by number six. This is a ligature replacement. \startlinecorrection \startMPcode draw MyRectangle(1,2,6,0,0,.5red ) ; draw MyRectangle(2,2,4,3,0,.5green) ; draw MyDot(4,4.25) ; draw MyRectangle(3,2,1,6,5,.5blue ) ; draw MyDot(7,4.75) ; draw MyRectangle(6,3,5,9,0,.5cyan ) ; currentpicture := currentpicture ysized(4cm) ; \stopMPcode \stoplinecorrection The mark is an independent entity. Sometimes it has a width, sometimes it hasn't. In both cases we can position it. Here we move the shape left and down. There are two ways to do this: simple pairwise kerning but better is to use anchors. Here we have one anchor per shape but there can be many. \startlinecorrection \startMPcode draw MyRectangle(1,2,6,0,0 ,.5red ) ; draw MyRectangle(2,2,4,3,0 ,.5green) ; draw MyRectangle(3,2,1,3,4.5,.5blue ) ; draw MyDot(4,4.25) ; draw MyRectangle(6,3,5,6,0 ,.5cyan ) ; currentpicture := currentpicture ysized(4cm) ; \stopMPcode \stoplinecorrection Next we apply some kerning. Of course the anchored marks need to move as well. \startlinecorrection \startMPcode draw MyRectangle(1,2,6,0, 0 ,.5red ) ; draw MyRectangle(2,2,4,2.5,0 ,.5green) ; draw MyRectangle(3,2,1,2.5,4.5,.5blue ) ; draw MyDot(3.5,4.25) ; draw MyRectangle(6,3,5,5, 0 ,.5cyan ) ; currentpicture := currentpicture ysized(4cm) ; \stopMPcode \stoplinecorrection Alternatively we can connect the shapes in a cursive way. The name cursive is somewhat misleading as it just boils down to shifting. The cursive indicates that the shifts accumulate within a word. \startlinecorrection \startMPcode draw MyRectangle(1,2,6,0,0 ,.5red ) ; draw MyRectangle(2,2,4,2,0.5,.5green) ; draw MyRectangle(3,2,1,2,5 ,.5blue ) ; draw MyDot(3,4.75) ; draw MyRectangle(6,3,5,4,1 ,.5cyan ) ; currentpicture := currentpicture ysized(4cm) ; \stopMPcode \stoplinecorrection \stopsubsection \startsubsection[title={Single substitution}] Single substitutions are probably the most used ones. For instance, when you ask for small caps, a lot of glyphs get replaced. When using oldstyle numerals only digits get replaced but even then each glyph has to be checked. This can be demonstrated with the Latin Modern fonts. \startlinecorrection \scale [height=1cm] {\strut {\definedfont[lmroman10-bold*default]\$123.45}% \quad {\definedfont[lmroman10-bold*oldstyle]\$123.45}} \stoplinecorrection As you can see here, Latin Modern has an oldstyle dollar sign. If you don't like that one, you're in troubles as it comes with the rest of the oldstyles. The only way out is to apply the oldstyle numerals to digits only which involves more tagging than you might be willing to add. So, whenever you choose a substitution, be aware that you have not that much control over what gets substituted: it's the font that drives it. Here are some examples: \starttyping \definefontfeature[capsandold][smallcaps,oldstyle] \showotfcomposition{dejavu-serif*capsandold at 24pt}{}{It's 2013!} \showotfcomposition{cambria*capsandold at 24pt}{}{It's 2013!} \showotfcomposition{lmroman10regular*capsandold at 24pt}{}{It's 2013!} \showotfcomposition{texgyrepagellaregular*capsandold at 24pt}{}{It's 2013!} \stoptyping \definefontfeature[capsandold][smallcaps,oldstyle] \blank \showotfcomposition{dejavu-serif*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank \blank \showotfcomposition{cambria*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank \blank \showotfcomposition{lmroman10regular*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank \blank \showotfcomposition{texgyrepagellaregular*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank \stopsubsection \startsubsection[title={Multiple substitution}] In a multiple substitution a sequence of characters (glyphs) gets replaced by another sequence. In fact, you might wonder why one||to||one, multiple||to||one and multiple||to||multiple are not all generalized into this variant. Efficiency is probably the main reason. \footnote {Isn't it strange that complex mechanisms are designed to save a few bytes while at the same time we produce ridiculous large pictures with cameras.} For instance the many||to||one is often used for ligatures (\type {liga}) and as a consequence \type {liga} is often misused also for non||ligatures. One usage of multiple replacements is to avoid and or undo other replacements. In the next example we see a language dependent \type {fi} ligature. Take the dutch \type {ij} and \type {ie} diftongs. Here we need to prevent the \type {i} becoming combined with the \type {f} as it would look weird. Among the solutions for this are: context dependent ligatures (which involves a lot of rules), or multiple to multiple replacements (looking at the \type {fij} sequence). \startbuffer[definitions] \definefontfeature[default-fijn-en][default][language=eng,script=latn] \definefontfeature[default-fijn-nl][default][language=nld,script=latn] \stopbuffer \getbuffer[definitions] \typebuffer[definitions] \starttyping \definedfont[lmroman10-regular*default-fijn-en]\en effe fijn fietsen \definedfont[lmroman10-regular*default-fijn-nl]\nl effe fijn fietsen \stoptyping This gives: \startlinecorrection[blank] \scale [width=\textwidth] \bgroup \framed [offset=overlay,frame=off,foregroundcolor=maincolor,align=normal,strut=no] \bgroup \definedfont[lmroman10-regular*default-fijn-en]\en effe fijn fietsen\vskip-1ex \definedfont[lmroman10-regular*default-fijn-nl]\nl effe fijn fietsen\par \egroup \egroup \stoplinecorrection Of course from this result one cannot see what (combination of) substitution(s) was used, but it's a nice exercise to work out a solution. Multiple substitutions are mostly used for scripts more complex than latin or special fonts like Zapfino where advanced contextual analysis happens. \stopsubsection \startsubsection[title={Alternate substitution}] Alternates are simple one||to||one substitutions. Popular examples are small capitials and oldstyle numerals. A nice application of alternates is the punk font. This font is a Knuth original. As part of experimenting with the \METAPOST\ library in the early days of \LUATEX\ and \MKIV, runtime randomization was implemented. However, that variant used virtual fonts and was somewhat resource hungry. So, in a later stage Khaled Hosny made an \OPENTYPE\ version using \METAPOST\ output. Randomization is implemented through the \type {rand} feature. In \MKIV\ the \type {rand} feature is not really special and behaves just like any other (stylistic) alternate. The only difference is that for this feature a value of \type {yes} equals \type {random}. This also means that any feature that uses alternates use them randomly. \startbuffer \definefontfeature[punknova-first] [mode=node,kern=yes,rand=first] \definefontfeature[punknova-2] [mode=node,kern=yes,rand=2] \definefontfeature[punknova-yes] [mode=node,kern=yes,rand=yes] \definefontfeature[punknova-random][mode=node,kern=yes,rand=random] \stopbuffer \typebuffer \getbuffer We use this is: \startbuffer[sample] The original punk font is designed by Don Knuth: xxxxxxxxxxxx \stopbuffer \startbuffer \definedfont[punknova-regular at 15pt] \getbuffer[sample] \definedfont[punknova-regular*punknova-first at 15pt] \getbuffer[sample] \definedfont[punknova-regular*punknova-2 at 15pt] \getbuffer[sample] \definedfont[punknova-regular*punknova-yes at 15pt] \getbuffer[sample] \definedfont[punknova-regular*punknova-random at 15pt] \getbuffer[sample] \stopbuffer \typebuffer[sample] \typebuffer In order to illustrate the variants we show a sequence of \type {x}'s. There are upto ten different variants per characters. \startlines[color=maincolor] \getbuffer \stoplines There is one pitfall with random alternates: if each run leads to a different outcome, we can end up in oscillation: different shapes give different paragraphs and we can get more pages or cross references etc.\ that can end up differently so this is why \CONTEXT\ always uses the same random seed (which gets reset when you purge the auxiliary files. \stopsubsection \startsubsection[title={Ligature substitution}] A ligature is traditionally a combination of several characters into one. Popular ligatures are \quote {fi}, \quote {fl}, \quote {ffi} and , \quote {ffl}. Occasionally we see \quote {\ae}, \quote {\oe} and some more. Often ligatures are language dependant. For instance in languages like Dutch and German there can be compound words where one part ends with an \type {f} and the next part starts with an \type {f} and that looks bad or at least not intuitive. To some extent one can wonder if this tradition of ligatures is a good one. It definitely made sense ages ago, but I wouldn't be surprised if they are often added to fonts because the encoding vectors have them. After all, nothing prevents to go ahead and come up with way more ligatures. There can be many ligature features in a font. Although we support arbitrary features, that is: those not registered as being official one way or the other, the following are known by description: \startluacode context.starttabulate { "|lTCT{maincolor}|l|" } for k, v in table.sortedhash(fonts.handlers.otf.tables.features) do if string.find(v,"ligature") then context.NC() context(k) context.NC() context(v) context.NR() end end context.stoptabulate() \stopluacode The \type {default} feature set has type {liga} as wel as the \TEX\ specific \type {tlig} that replaces successive hyphen signs into en- and emdashes. The \type {arabic} feature set also has \type {rlig} enabled. Now, there is one thing you should realize when we discuss specific features and the underlaying mechanisms: there is no real relationship between the features's name and the mechanisms used: any feature can use any underlying mechanism or combination. This is why deep down we see that what is internally called ligature gets used for any purpose where multiple||to||one replacements happen, and why the \type {liga} feature can use single substitutions or alternates to swap in another rendering so that the dot of the \type {i} stays free of the preceding \type {f}. And for some fonts relative positioning can be used to achieve a ligature effect. The next examples demonstrate how the \type {liga} feature deals with \type {ffi}. Possible solutions are: replace all three at once, replace the first two first and in a next step, combine a ligature and following character, replace one or more components by variants that have no interference with the dot of the~\quote{i}. \starttyping \showotfcomposition{dejavu-serif*default at 48pt}{}{ffi} \showotfcomposition{cambria*default at 48pt}{}{ffi} \showotfcomposition{lmroman10regular*default at 48pt}{}{ffi} \showotfcomposition{texgyrepagellaregular*default at 48pt}{}{ffi} \stoptyping \blank \showotfcomposition{dejavu-serif*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank \blank \showotfcomposition{cambria*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank \blank \showotfcomposition{lmroman10regular*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank \blank \showotfcomposition{texgyrepagellaregular*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank \stopsubsection \startsubsection[title={Single positioning}] Single positioning is also known as kerning, moving characters closer together so that we get a more uniformly spaced sequence of glyphs. It is a mistake to think that kerning is always needed! There are fonts that have hardly any kerns or no kerns at all and still look good. \start \showfontkerns \blank \definedfont[dejavu-serif*default at 8pt]Dejavu Serif: \input tufte (E.R. Tufte)\blank \definedfont[cambria*default at 9pt]Cambria: \input tufte (E.R. Tufte)\blank \definedfont[lmroman10regular*default at 10pt]Latin Roman Regular: \input tufte (E.R. Tufte)\blank \definedfont[lucidabrightot*default at 8pt]Lucida Bright: \input tufte (E.R. Tufte)\blank \definedfont[texgyrepagellaregular*default at 9pt]Pagella Regular: \input tufte (E.R. Tufte)\blank \stop The next couple of examples show the action for a few words: \blank \showotfcomposition{dejavu-serif*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank \blank \showotfcomposition{cambria*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank \blank \showotfcomposition{lmroman10regular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank \blank \showotfcomposition{lucidabrightot*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank \blank \showotfcomposition{texgyrepagellaregular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank \stopsubsection \startsubsection[title={Pairwise positioning}] This variant of positioning involved the first, second or both glyphs of a glyph pair. The specification can influence the horizontal and vertical positions we well as the widths of the positioned glyphs. \startnotabene We need an example here. \stopnotabene \stopsubsection \startsubsection[title={Mark positioning}] Marks are (often) small symbols that represent accents (in latin) or vowels (in arabic) that get attached to base glyphs. In the input stream they come after the character that they apply to. Many fonts come with precomposed latin characters which means that an \type {à} in the input is mapped directly onto its corresponding shape. When the input contains an \type {a} followed by a \type{̀ } input normalization will normally turn this into an \type {à}. But, when this doesn't happen, the font machinery has to make sure that the mark gets positioned right onto the base character. In traditional \TYPEONE\ fonts that more or less happened automatically by overlaying the shapes. In \OPENTYPE\ (single) positioning is used to place the mark right. \startnarrowtyping \showotfcomposition{dejavu-serif*default at 24pt}{}{à a\utfchar{"0300} à} \showotfcomposition{cambria*default at 24pt}{}{à a\utfchar{"0300} à} \showotfcomposition{lmroman10regular*default at 24pt}{}{à a\utfchar{"0300} à} \showotfcomposition{lucidabrightot*default at 24pt}{}{à a\utfchar{"0300} à} \showotfcomposition{texgyrepagellaregular*default at 24pt}{}{à a\utfchar{"0300} à} \stopnarrowtyping Of course a font can contain logic that replaces a sequence of base and mark into precomposed characters with the right \UNICODE\ entry. \blank \showotfcomposition{dejavu-serif*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank \blank \showotfcomposition{cambria*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank \blank \showotfcomposition{lmroman10regular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank \blank \showotfcomposition{lucidabrightot*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank \blank \showotfcomposition{texgyrepagellaregular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank You can imagine that when marks are bound to characters that have become ligatures the anchoring is more complex as the font machinery has to keep track of onto which component the mark goes. For this purpose marks as well as base characters and base ligatures have anchors and feature lookups can explicitly refer to them. \stopsubsection \startsubsection[title={Contextual analysis}] What actually happens when turning a list of characters into a list of glyphs can range from real simple to pretty complex. For instance the \type {smcp} feature only has to run over the list and relate characters to a smallcaps shape. A slightly more complex feature might also demand some positioning. One step further is the use of contextual analysis, i.e. looking at previous, current and following characters (or glyphs). Because features can range from simple to complex the actual processing is not per feature! A font comes with a sequence of so called lookups that relate to a feature, script and language. Also, each feature can use one||to||one, multiple||to||one and many||to||many replacements as well as relative positioning. So, what actually happens is not that a feature is processed, but that all features are dealt with at the same time, in the order that the font prescribes. Enabling a specific feature means that a step is executed, while a disabled feature skips all steps that are tagged as belonging to that feature. And, as each feature can use contextual analysis, you can imagine that the effective sequence of actions can be a complex mix. A nice example of a contextual substitution is the centered period character in catalan in \type {ebgaramond}: \startbuffer \definefontfeature [example] [default] [locl=yes,script=latn,language=cat] \definedfont[file:ebgaramond12-regular.otf*default at 40pt]l·l\quad \definedfont[file:ebgaramond12-regular.otf*example at 40pt]l·l \stopbuffer \typebuffer We show the boundingbox of the glyphs. The centered period between two l's is is replaced by a raised variant with no width. \blank \start \showglyphs \maincolor \midaligned{\getbuffer} \stop \blank It will be clear that in order to use such features you need to know what the font provides. For some fonts you need to explicitly enable the latin script (while others use default). Such a feature can be part of localized support but that is no rule. In that respect \OPENTYPE\ features are a rather unpredictable mess. For instance, nothing prevents such a feature to be a ligature, and in case you find that strange, especially ligature features are often abused for any purpose. \stopsubsection \startsubsection[reference=ligatures:hyphenation,title=Ligatures and hyphenation] In this section we will say a few words on how hyphenation interferes with (especially) ligature building. For this you need to know that: \starttyping effe \stoptyping But when hyphenation is permitted between the two \type{s}'s we actually have internally: \starttyping ef{-}{}{}fe \stoptyping The first snippet comes at the end of a line, the second at the beginning of a the next line and the last snippet is used when no hyphenation is needed. Such triplets need to be taken into account when we do replacements and positioning and also when we do contextual lookups. An \OPENTYPE\ font is just a container that collects the following: \startitemize[packed] \startitem graphic representations of characters and symbols \stopitem \startitem information about what characters the shapes represent \stopitem \startitem rules about converting (sequences of) characters into one or more representations \stopitem \startitem rules about positioning representations relative to each other \stopitem \stopitemize Although the way this information is stored is standardized, the rules are not. You can imagine that there would be some standard way to turn an \type {f} and \type {i} into an \quote{fi} but we already saw that this is not the case. Here are some possibilities: \startitemize[packed] \startitem The two characters get their own standard glyph, maybe with some kerning. \stopitem \startitem The two characters are combined into one shape. \stopitem \startitem The \type{f} gets a narrow representation and is kept close to the standard \type{i}. \stopitem \startitem A standard \type {f} is kerned with a dotless \type{i} (not to be confused with the \UNICODE\ character). \stopitem \startitem A special \type {f} is combined with a special \type {i}. \stopitem \stopitemize % maybe mark lig components when separate chars so that we can do spacing If the two characters are represented by their own shape, some contextual analysis takes place. Again there are several approaches to this: \startitemize[packed] \startitem When an \type{f} is seen in the input, the next character is checked and one or both gets replaced. \stopitem \startitem When an \type{i} is seen in the input, the previous character is checked and the \type {i} gets replaced. \stopitem \startitem When an \type{f} several following characters are checked, for instance to see if we need to take \type {ij} into account. \stopitem \stopitemize Traditionally the \type {f} followed by an \type{f}, \type{l} and \type{i} get a treatment, but some fonts also combine the \type {f} with \type {k}, \type {j}, \type {b}, \type {t} and more. The \MKIV\ font handler is rather generic in the sense that it support what the font requires. However, a complication is that the scripts (languages) that use these diverse methods also expect hyphenation within such a ligature. Script like Arabic that are more demanding don't hyphenate so there interference with hyphenation is not a problem. Some ligatures are sensitive for languages. In languages that have compound words it might be undesirable to have a ligature at a word boundary, or in the Dutch word \type {fijn} we like to have a nice glyph (or combinations) for \type {ij} but no \type {fi} ligature. In a similar way hyphenation patterns can have rules and it will be no surprise that the hyphenation mechanism can compete with the ligature building for the best solution. This gets complicated by the fact that there is no real way to recognize in the font handler if we really are dealing with ligature building. Not only is the \type {liga} feature (and deep down the ligature gsub handling) not bound to ligatures (but simply a many|-|to|-|one mapper), some of the mentioned pseudo ligature builders use simple substitution and kerning and there is no way to recognize that as a ligature. Although it is possible to come up with a solution that is acceptable for many cases, there is no way to predict what kind of tricks font designers will use. A hyphenation point can be seen as follows: \starttabulate[||||] \NC \type{effe} \NC \type{ef-fe} \NC \type{e{f-}{f}{ff}e} \NC \NR \NC \type{efficient} \NC \type{ef-fi-cient} \NC \type{e{ffi-}{}{ffi}cient} \NC \NR \stoptabulate In the second case the larger ligatures has replaced the previous one. We could have kept the first one because there are ways to manage two|-|step bounding ligatures but it's not worth the trouble (read: way more complex code and increased runtime for the whole mechanism). Here the \type {{ff}} and \type {{ffi}} can be individual shapes or just one shape. The three components of a hyphenation point: the pre, post and replacement text need to be looked at independently so that we get the proper kerning with the preceding and following characters. Also, in more complex (chained) lookups we need to compare each element with its surrounding. A fully expanded solution tree is too time consuming so we take some shortcuts and limits the checks to the level that it has no big impact on performance. The occasionally needed backtracking and inspection of components is currently quite reasonable. We need to trade quality with convenience: the result should look okay but processing speed should also be as high as possible. There is no need to let other scripts or regular fonts suffer too much from excessive script demands of fonts that could have be done better. The complication is that we not only need to check and replace but also need to check the kerning with preceding and following characters. We also need to take the hyphen into account (here one, but there can also be one after the break. It is for this reason that in \MKIV\ we have a (we think) acceptable mix of heuristics around hyphenation points that deal with single and multiple substitution as well as kerning. It will never be 100\% pertect but we consider it better to drop an occasional hyphenation in favor of proper font handling. In practice \TEX\ is clever enough to break a paragraph in lines within these restrictions. In \CONTEXT\ we have the traditional \TEX\ hyphenator but also provide an extensible \LUA\ reimplementation. That one might become the default in future versions. In traditional \TEX\ there are several low level hyphenation representations: simple hyphen only points, injected by the hyphenator, explicitly injected by the user or originating from a hyphen character. Then there is the generic (pre, post, replace) discretionary that can be explicitly injected by the user (or a macro). In \MKIV\ all hyphenation points get normalized to this generic discretionary. There is no need for old|-|time optimizations and a consistent (expanded) representation is easier to deal with in other extensions. However, because the font handler is supposed to also work outside \CONTEXT\ we need to deal with traditional cases too. But \unknown\ the results might differ a bit. \stopsubsection \startsubsection[title=Color] % TODO: use emojionecolor-svginot-archived.ttf A recent new (and evolving) addition to \OPENTYPE\ is colored glyphs. One variant (by \MICROSOFT) uses overlays and this method is quite efficient. \startbuffer \definefontfeature[colored][colr=yes] \definefontsynonym[Emoji][file:seguiemj.ttf*default,colored] \definesymbol[bug][\getglyphdirect{Emoji}{\char"1F41B}] \definesymbol[ant][\getglyphdirect{Emoji}{\char"1F41C}] \definesymbol[bee][\getglyphdirect{Emoji}{\char"1F41D}] \stopbuffer \typebuffer \getbuffer Here we see a \symbol[bug], \symbol[ant] and \symbol[bee], and they come in color! Once \UNICODE\ started adding such symbols (and more get added) the distinction between characters and symbols get even fuzzier. Of course one can argue that we communicate in pictograms but even then, given that mankind lasts a while, the \UNICODE\ repertoire will explode. \startplacefigure[title={A few emojis from \type {seguiemj.ttf}}] \startcombination [3*1] {\scale[width=.3\textwidth]{\symbol[bug]}} {\type{U+1F41B}: bug} {\scale[width=.3\textwidth]{\symbol[ant]}} {\type{U+1F41C}: ant} {\scale[width=.3\textwidth]{\symbol[bee]}} {\type{U+1F41D}: bee} \stopcombination \stopplacefigure Here we use \type {seguiemj.ttf}, a font that comes with \MSWINDOWS. Colors are achieved by combining glyphs rendered in different colors. A variant that uses \SVG\ instead of overlays is \type {emojionecolor-svginot.ttf}: \startbuffer \definefontfeature[svg][svg=yes] \definefontsynonym[Emoji][file:emojionecolor-svginot.ttf*default,svg] \stopbuffer \typebuffer \getbuffer This time we get \symbol[bug], \symbol[ant] and \symbol[bee] and they look quite different. Both fonts also have ligatures and you can wonder what sense that makes. It makes it impossible to swap fonts and as there is no standard one never knows what to expect. \startplacefigure[title={A few emojis from \type {emojionecolor-svginot.ttf}}] \startcombination [3*1] {\scale[width=.3\textwidth]{\symbol[bug]}} {\type{U+1F41B}: bug} {\scale[width=.3\textwidth]{\symbol[ant]}} {\type{U+1F41C}: ant} {\scale[width=.3\textwidth]{\symbol[bee]}} {\type{U+1F41D}: bee} \stopcombination \stopplacefigure \definefont[Emoji][file:emojionecolor-svginot.ttf*default,svg] \def\FourFaces{\char128104\zwj\char128105\zwj\char128102\zwj\char128102\relax} \def\Man {\char"1F468\relax} \def\Woman{\char"1F469\relax} \def\Boy {\char"1F466\relax} \def\Girl {\char"1F467\relax} How do we know what faces add up to the ligature {\Emoji\Man \zwj \Woman \zwj \Girl \zwj \Boy} and how are we supposed to know that there should {\darkgray \type {zwj}} in between? When we input four faces separated by zero width joiners, we get a four face symbol instead. The reason for having the joiners in between is probably to avoid unexpected ligatures. The sequence \type {man}, \type {woman}, \type {boy}, \type {boy} gives \type {family}: % {\Emoji\Man} + {\darkgray \type {zwj}} {\Emoji\Woman} + {\darkgray \type {zwj}} {\Emoji\Boy} + {\darkgray \type {zwj}} {\Emoji\Boy} = {\Emoji\Man \zwj \Woman \zwj \Boy \zwj \Boy}, % but two girls also work: % {\Emoji\Man} + {\darkgray \type {zwj}} {\Emoji\Woman} + {\darkgray \type {zwj}} {\Emoji\Girl} + {\darkgray \type {zwj}} {\Emoji\Girl} = {\Emoji\Man \zwj \Woman \zwj \Girl \zwj \Girl}, % so does a mixture of kids: % {\Emoji\Man} + {\darkgray \type {zwj}} {\Emoji\Woman} + {\darkgray \type {zwj}} {\Emoji\Girl} + {\darkgray \type {zwj}} {\Emoji\Boy} = {\Emoji\Man \zwj \Woman \zwj \Girl \zwj \Boy}, % although (at least currently): % {\Emoji\Man} + {\darkgray \type {zwj}} {\Emoji\Woman} + {\darkgray \type {zwj}} {\Emoji\Boy} + {\darkgray \type {zwj}} {\Emoji\Girl} = {\Emoji\Man \zwj \Woman \zwj \Boy \zwj \Girl}, % gives twin boys. Of course the real family emoj is {\Emoji\char"1F46A}. In our times for sure many combinations are possible, so: % {\Emoji\Man} + {\darkgray \type {zwj}} {\Emoji\Man} + {\darkgray \type {zwj}} {\Emoji\Girl} + {\darkgray \type {zwj}} {\Emoji\Girl} = {\Emoji\Man \zwj \Man \zwj \Girl \zwj \Girl}, % indeed gives a family, but I wonder at what point cultural bias will creep into font design. One can even wonder how clothing and haircut will demand frequent font updates: {\Emoji\char"1F46B}, {\Emoji\char"1F46C}, {\Emoji\char"1F46D}. In the math alphabets we have a couple of annoying holes because some characters were already present in \UNICODE. The bad thing here is that we now always have to deal with these exceptions. But not so with emojis because here eventually all variants will show up. Where a character \type {A} in red or blue uses the same code point, a white telephone {\Emoji\char"1F57E} and black telephone {\Emoji\char"1F57F} have their own. And because obsolete scripts are already supported in \UNICODE\ and more get added, we can expect old artifacts also showing up at some time. Soon the joystick {\Emoji\char"1F579} will be an unknown item to most of us, while the \MICROSOFT\ hololens migth get its slot. \startplacefigure[title={Will all animals come in stages of development?}] \startcombination [3*1] {\scale[width=.3\textwidth]{\Emoji\char"1F423}} {\type{U+1F423}: hatching chick} {\scale[width=.3\textwidth]{\Emoji\char"1F424}} {\type{U+1F424}: baby chick} {\scale[width=.3\textwidth]{\Emoji\char"1F425}} {\type{U+1F425}: front-facing baby chick} \stopcombination \stopplacefigure For sure these mechanisms will evolve and to what extent we support them depends on what users want. At least we have the basics implemented. \stopsubsection \stopsection \startsection[title=Extras] \startnotabene Todo. \stopnotabene \stopsection \startsection[reference=goodies,title=Goodies] Goodies range from simple to complex. They share that they are defined in files and loaded at runtime. There is a good change that when you read this, that there are already more goodies than mentioned here. Here we will just mention a couple of goodies. More details can be found in the files that ship with \CONTEXT\ and have suffix \type {lfg}. A goodie file is a regular \LUA\ file and is supposed to return a table. This table collects data that is used for implementing the goodie or relates to a regular feature. It can also provide information that is used for patching a font. An example of a simple goodie file is the ones that accompanies the first release of the \OPENTYPE\ Lucida fonts. \starttyping return { name = "lucida-opentype-math", version = "1.00", comment = "Goodies that complement lucida opentype.", author = "Hans Hagen", copyright = "ConTeXt development team", mathematics = { alternates = { italic = { feature = 'ss01', value = 1, comment = "Mathematical Alternative Italic" }, } } } \stoptyping This goodie file is only providing information about the meaning of a stylistic alternate. These have abstract tags like \type {ss01} and in this case this category collects alternative italic (calligraphic) shapes. Because math does not follow the same rules as text, this feature is enabled explicitly. In the goodie file of Xits math the alternates table has more entries: \startnarrowtyping alternates = { cal = { ... comment = "Mathematical Calligraphic Alphabet" }, greekssup = { ... comment = "Mathematical Greek Sans Serif Alphabet" }, greekssit = { ... comment = "Mathematical Italic Sans Serif Digits" }, monobfnum = { ... comment = "Mathematical Bold Monospace Digits" }, mathbbbf = { ... comment = "Mathematical Bold Double-Struck Alphabet" }, mathbbit = { ... comment = "Mathematical Italic Double-Struck Alphabet" }, mathbbbi = { ... comment = "Mathematical Bold Italic Double-Struck Alphabet" }, upint = { ... comment = "Upright Integrals" }, vertnot = { ... comment = "Negated Symbols With Vertical Stroke" }, } \stopnarrowtyping An alternate is triggered at the \TEX\ end with: \starttyping $ABC$ $\cal ABC$ $\mathalternate{cal}\cal ABC$ \stoptyping This is an example of a dynamic feature that gets applied when enabled at a specific location in the input. The \type {cal} is only recognized when it is defined in a goodies file, where the value is defined (in all of the above cases the value is~\type {1}). The Xits math fonts has a goodie files that starts with: \starttyping return { name = "xits-math", version = "1.00", comment = "Goodies that complement xits (by Khaled Hosny).", author = "Hans Hagen", copyright = "ConTeXt development team", mathematics = { italics = { ["xits-math"] = italics, }, alternates = { \stoptyping Here the \type {italics} variable is a table defined before the \type {return} that looks as follows: \starttyping local italics = { defaultfactor = 0.025, disableengine = true, corrections = { -- [0x1D44E] = 0.99, -- a (fraction of quad) -- [0x1D44F] = 100, -- b (font points) [0x1D453] = -0.0375, -- f } } \stoptyping This rather specific table tells \CONTEXT\ that (when enabled) it has to apply italic correction. It disables support built into the \TEX\ engine (which in the case of \LUATEX\ is close to absent anyway). It will apply a default italic correction of \type {0.025} but for some shapes a different value is used. Again we have some commands at the \TEX\ end: \starttyping \setupmathematics[italics=1] % apply italics with we have an italic font \setupmathematics[italics=2] % apply italics anyway \setupmathematics[italics=3] % apply italics only when italic or bolditalic shapes \setupmathematics[italics=4] % combination of 1 and 3 \stoptyping An alternative is this: \starttyping \definefontfeature[mathextra][mathextra][collapseitalics=yes] \stoptyping This extends the \type {mathextra} feature to move the italic correction into the character's width. Often this works out ok. Because (definitely at the start of the \LUATEX\ project) we had no proper \OPENTYPE\ math fonts, but at the same time wanted to move on to \OPENTYPE\ and \UNICODE\ math and no longer struggle with all those math families and definitions. The way out of this problem is to define a virtual math font. The code for doing this is built into the \MKIV\ core but is controlled by a goodie definition. Take for instance Antykwa Math: \startnarrowtyping return { name = "antykwa-math", version = "1.00", comment = "Goodies that complement antykwa math.", author = "Hans, Mojca, Aditya", copyright = "ConTeXt development team", mathematics = { mapfiles = { "antt-rm.map", "antt-mi.map", "antt-sy.map", "antt-ex.map", "mkiv-base.map", }, virtuals = { ["antykwa-math"] = { { name = "file:AntykwaTorunska-Regular", features = "virtualmath", main = true }, { name = "mi-anttri.tfm", vector = "tex-mi", skewchar=0x7F }, { name = "mi-anttri.tfm", vector = "tex-it", skewchar=0x7F }, { name = "sy-anttrz.tfm", vector = "tex-sy", skewchar=0x30, parameters = true } , { name = "ex-anttr.tfm", vector = "tex-ex", extension = true } , { name = "msam10.tfm", vector = "tex-ma" }, { name = "msbm10.tfm", vector = "tex-mb" }, }, \stopnarrowtyping Normally users will not define such tables but the keys give an indication of what is involved. The same is true for the previously shown tables: they are just provided in the \CONTEXT\ distribution. Text fonts also can have goodies. We start with a rather dumb one and there will be not that many of those. This one is needed to turn a \TYPEONE\ font with a rather special encoding into a \UNICODE\ font. The next mapping is possible because the dingbats are part of \UNICODE. \starttyping return { name = "dingbats", version = "1.00", comment = "Goodies that complement dingbats (funny names).", author = "Hans Hagen", copyright = "ConTeXt development team", remapping = { tounicode = true, unicodes = { a1 = 0x2701, a10 = 0x2721, a100 = 0x275E, a101 = 0x2761, a102 = 0x2762, \stoptyping Applying this encoding happens in two steps. Because goodies like this are just features, we need to define a proper font feature set: \starttyping \definefontfeature [dingbats] [mode=base, goodies=dingbats, unicoding=yes] \stoptyping We have a base mode font, so no special processing takes place. The \type {goodies} key is used to communicate the goodies file. The \type {unicoding} key is used to apply the encoding. Of course this only works because the remapper code is present in the core and is hooked in to the font initialization code. The \type {dingbats} feature set is predefined, just as the font definition: \starttyping \definefontsynonym [ZapfDingbats] [file:uzdr] [features=dingbats] \stoptyping Here is a goodie file that I made a while ago: \starttyping return { name = "oxoniensis", version = "1.00", comment = "Oxoniensis test file for Thomas Schmitz.", author = "Hans Hagen", copyright = "ConTeXt development team", features = { lunatesigma = { type = "substitution", data = { sigma = 0x03F2, sigma1 = 0x03F2, Sigma = 0x03F9, phi = phi1, }, } }, } \stoptyping There is not that much to say about this, apart from that it's a sort of fake feature that gets enabled as regular one: \starttyping \definefontfeature[test] [mode=node, kern=yes, lunatesigma=yes, goodies=oxoniensis] \definefont[somefont][file:oxoniensis*test] \stoptyping A complete different kind of goodie is the following. At one of the \CONTEXT\ meetings Mojca Miklavec discussed the possibility to have an additional mechanism for defining combinations of fonts. Often fonts come in a set of four (regular, italic, bold and bold italic). In \MKII\ the complexity of typescripts depends on the amount of encodings that need to be supported but in \MKIV\ things are easier. For a set of four fonts a typescript looks as follows: \starttyping \starttypescript [sans] [somesansfont] [name] \setups[font:fallback:sans] \definefontsynonym [Sans] [file:somesans] [features=default] \definefontsynonym [SansBold] [file:somesansb] [features=default] \definefontsynonym [SansItalic] [file:somesansi] [features=default] \definefontsynonym [SansBoldItalic] [file:somesansz] [features=default] \stoptypescript \stoptyping We still have the abstract notion of a \type {Sans} font so that we can refer to the regular shape without knowing the real name but the number of lines needed is small. Such a definition can then be referred to using: \starttyping \starttypescript[somefontset] \definetypeface [somefontset] [rm] [serif] [someserif] [default] \definetypeface [somefontset] [ss] [sans] [somesans] [default] \definetypeface [somefontset] [tt] [mono] [somemono] [default] \definetypeface [somefontset] [mm] [math] [somemath] [default] \stoptypescript \stoptyping So far things look simple. Given that many fonts follow a similar naming scheme Wolfgang made a module that avoids such definitions altogether. However, being involved in the development of the Antykwa fonts, Mojca ran into the situation that not just four fonts were part of the set but many more. There are several weight (think of light and heavy variants) as well as condensed variants and of course the whole set is not per se a multiple of four. In the meantime, in addition to the \type {file:} and \type {name:} accessors, \CONTEXT\ had an additional one tagged \type {spec:} where a string made out of weight, style, width etc.\ is turned into a (best guessed) font name. Therefore the most natural way to deal with the many|-|fonts|-|in|-|a|-|set dilemma was to provide an additional interface between this specification and the font set and the most robust method was to define all in a goodie file. In this case the goodies are loaded independent of the font, that is: not as a feature. For instance: \starttyping \loadfontgoodies[antykwapoltawskiego] \stoptyping This file maps obscure fontnames onto the \type {spec:} interface so that we can access them in a robust way. \starttyping \definefont [MyFontA] [file:Iwona-Regular*smallcaps] \definefont [MyFontB] [file:AntykwaTorunska-Regular*smallcaps] \definefont [MyFontC] [file:antpoltltcond-regular*smallcaps] \definefont [MyFontD] [spec:antykwapoltawskiego-bold-italic-condensed-normal*smallcaps] \definefont [MyFontE] [spec:antykwapoltawskiego-bold-italic-normal] \stoptyping The goodies file looks as follows: \starttyping return { name = "antykwa-poltawskiego", version = "1.00", comment = "Goodies that complement Antykwa Poltawskiego", author = "Hans & Mojca", copyright = "ConTeXt development team", files = { name = "antykwapoltawskiego", -- shared list = { ["AntPoltLtCond-Regular.otf"] = { weight = "light", style = "regular", width = "condensed", }, ... ["AntPoltExpd-BoldItalic.otf"] = { weight = "bold", style = "italic", width = "expanded", }, }, }, typefaces = { ["antykwapoltawskiego-light"] = { shortcut = "rm", shape = "serif", fontname = "antykwapoltawskiego", normalweight = "light", boldweight = "medium", width = "normal", size = "default", features = "default", }, ... }, } \stoptyping In addition to the files|-|to|-|specification mapping, there is also a typeface specification table. This permits the definition of a typeface in the following way: \starttyping \definetypeface [name=mojcasfavourite, preset=antykwapoltawskiego, normalweight=light, boldweight=bold, width=expanded] \setupbodyfont [mojcasfavourite] \stoptyping When resolving the definition, the best possible match will be taken from the typeface table in the goodie file. Of course this is not something that we expect the average user to deliver and deal with. As the Antykwa font is somewhat atypical and not used in everyday typesetting, you might wonder if all this overhead makes sense. However, there are type foundries that do ship their fonts in many weights and for those using a \LUA\ goodie file instead of many typescripts in \TEX\ coding makes sense. Take for instance TheMix: \starttyping \loadfontgoodies [themix] \definetypeface [name=themix, preset=themix-light] \definetypeface [name=themix, preset=themixmono-light] \setupbodyfont [themix] \stoptyping In this case the goodie file can serve as a template for more such fonts. In order to be efficient this goodie file uses a couple of local tables (we could have used metatables instead). \starttyping local themix = { name = "themix", shortcut = "ss", shape = "sans", fontname = "themix", width = "normal", size = "default", features = "default", } local themixmono = { name = "themixmono", shortcut = "tt", shape = "mono", fontname = "themixmono", width = "normal", size = "default", features = "default", } \stoptyping The main goodie table defines a lot of weights: \startnarrowtyping return { name = "themix", version = "1.00", comment = "Goodies that complement TheMix (by and for sale at www.lucasfonts.com).", author = "Hans Hagen", copyright = "ConTeXt development team", files = { list = { ["TheMixOsF-ExtraLight"] = { name = "themix", weight = "extralight", style = "regular", width = "normal" }, ["TheMixOsF-ExtraLightItalic"] = { ... }, ... ["TheMixOsF-Black"] = { ... }, ["TheMixOsF-BlackItalic"] = { ... }, ... -- ["TheMixMono-W2ExtraLight"] = { name = "themixmono", weight = "extralight", style = "regular", width = "normal" }, ... ["TheMixMono-W9BlackItalic"] = { ... }, }, }, typefaces = { ["themix-extralight"] = table.merged(themix, { normalweight = "extralight", boldweight = "semilight" }), ["themix-light"] = table.merged(themix, { normalweight = "light", boldweight = "normal" }), ... ["themixmono-bold"] = table.merged(themixmono, { normalweight = "bold", boldweight = "black" }), }, } \stopnarrowtyping It's now time for some generic goodies. In the \CONTEXT\ distribution there is a goodie file that (at the time of this writing) looks as follows: \starttyping local default = { analyze = "yes", mode = "node", language = "dflt", script = "dflt", } local smallcaps = { smcp = "yes", } local function statistics(tfmdata) commands.showfontparameters(tfmdata) end local function squeeze(tfmdata) for k, v in next, tfmdata.characters do v.height = 0.75 * (v.height or 0) v.depth = 0.75 * (v.depth or 0) end end return { name = "demo", version = "1.01", comment = "An example of goodies.", author = "Hans Hagen", featuresets = { default = { default, }, smallcaps = { default, smallcaps, }, }, colorschemes = { default = { [1] = { "one", "three", "five", "seven", "nine", }, [2] = { "two", "four", "six", "eight", "ten", }, }, all = { [1] = { "*", }, }, some = { [1] = { "0x0030:0x0035", }, }, }, postprocessors = { statistics = statistics, squeeze = squeeze, }, } \stoptyping This demo file implements several goodies: featuresets, colors and postprocessors. Keep in mind that a goodie file can provide whatever information it wants but of course only known subtables will be dealt with. The coloring of glyphs can happen by name, which assumes that glyph names are used, or by number. Here we use generic glyph names, but for a specific font one might need to provide a special goodie file. For instance, the color section of the goodie file for the husayni font has entries like: \startnarrowtyping [3] = { "Ttaa.waqf", "SsLY.waqf", "QLY.waqf", "Miim.waqf", "LA.waqf", "Jiim.waqf", "Threedotsabove.waqf", "Siin.waqf", "Ssaad.waqf", "Qaaf.waqf", "SsL.waqf", "QF.waqf", "SKTH.waqf", "WQFH.waqf", "Kaaf.waqf", "Ayn.ruku", "Miim.nuun_high", "Siin.Ssaad", "Nuunsmall", "emptydot_low", "emptydot_high", "Sifr.fill", "Miim.nuun_low", "Nuun.tanwiin", }, \stopnarrowtyping Of course such a definition can only be made when the internals of the font are known and in this case it concerns a pretty complex font. \startbuffer \definefontfeature [demo-colored] [goodies=demo, colorscheme=default, featureset=default] \definefontfeature [demo-colored-all] [goodies=demo, colorscheme=all, featureset=default] \definefontfeature [demo-colored-some] [goodies=demo, colorscheme=some, featureset=default] \definefont[DemoFontA][MonoBold*demo-colored at 10pt] \definefont[DemoFontB][MonoBold*demo-colored-all at 10pt] \definefont[DemoFontC][MonoBold*demo-colored-some at 10pt] \stopbuffer \typebuffer \getbuffer % \definecolor[colorscheme:1:1][s=.75] % \definecolor[colorscheme:1:2][r=.75] % \definecolor[colorscheme:1:3][g=.75] % \definecolor[colorscheme:1:4][b=.75] % \definecolor[colorscheme:1:5][c=.75] % \definecolor[colorscheme:1:6][m=.75] % \definecolor[colorscheme:1:7][y=.75] % \definecolor[colorscheme:2:7][s=.75] % \definecolor[colorscheme:2:6][r=.75] % \definecolor[colorscheme:2:5][g=.75] % \definecolor[colorscheme:2:4][b=.75] % \definecolor[colorscheme:2:3][c=.75] % \definecolor[colorscheme:2:2][m=.75] % \definecolor[colorscheme:2:1][y=.75] \startbuffer \starttabulate[||||] \NC \DemoFontA \resetfontcolorscheme test 1234567890 \NC \DemoFontA \setfontcolorscheme [1]test 1234567890 \NC \DemoFontA \setfontcolorscheme [2]test 1234567890 \NC \NR \NC \DemoFontB \resetfontcolorscheme test 1234567890 \NC \DemoFontB \setfontcolorscheme [1]test 1234567890 \NC \DemoFontB \setfontcolorscheme [2]test 1234567890 \NC \NR \NC \DemoFontC \resetfontcolorscheme test 1234567890 \NC \DemoFontC \setfontcolorscheme [1]test 1234567890 \NC \DemoFontC \setfontcolorscheme [2]test 1234567890 \NC \NR \stoptabulate \stopbuffer \typebuffer \getbuffer Here is an example that I made at the TUG 2013 conference in Japan, after a presentation by Chof. The font (adapted by by Dohyun Kim) can be downloaded from: \hyphenatedurl {http://ftp.ktug.org/KTUG/hcr-lvt/1.910_nomac/}. \startbuffer[korean-demo] \definefontfeature [korean-composed] [goodies=hanbatanglvt, colorscheme=default, mode=node, ljmo=yes, tjmo=yes, vjmo=yes, script=hang, language=kor] \definefont [KoreanJMO] [hanbatanglvt*korean-composed] \definecolor[colorscheme:100:1][r=.75] \definecolor[colorscheme:100:2][g=.75] \definecolor[colorscheme:100:3][b=.75] \definecolor[colorscheme:101:1][g=.75,b=.75] \definecolor[colorscheme:101:2][r=.75,b=.75] \definecolor[colorscheme:101:3][r=.75,g=.75] \stopbuffer \typebuffer[korean-demo] \getbuffer[korean-demo] \startbuffer % Hunminjeongeum: http://en.wikipedia.org/wiki/Hunminjeongeum 나랏말ᄊᆞ미中듕國귁에달아문ᄍᆞᆼ와로서르ᄉᆞᄆᆞᆺ디아니ᄒᆞᆯᄊᆡ% 사ᄅᆞᆷ마다ᄒᆡᅇᅧ수ᄫᅵ니겨나...% ᆯ로ᄡᅮ메便뼌安ᅙᅡᆫ킈ᄒᆞ고져ᄒᆞᇙᄯᆞᄅᆞ미니라 \stopbuffer \startlinecorrection \startcombination[1*3] {\framed{\startscript[hangul]\KoreanJMO \getbuffer\stopscript}} {no colorscheme} {\framed{\startscript[hangul]\KoreanJMO\setfontcolorscheme[100]\getbuffer\stopscript}} {colorscheme 100} {\framed{\startscript[hangul]\KoreanJMO\setfontcolorscheme[101]\getbuffer\stopscript}} {colorscheme 101} \stopcombination \stoplinecorrection The goodie definition looks as follows (watch how we use ranges): \starttyping return { name = "hanbatanglvt", version = "1.00", comment = "Goodies that complement the hanbatanglvt fonts.", author = "Hans Hagen", colorschemes = { default = { { "0x01100:0x0115F" }, -- jamo_initial (r/c) { "0x01160:0x011A7" }, -- jamo_medial (g/m) { "0x011A8:0x011FF" }, -- jamo_final (b/y) } } } \stoptyping This is much shorter (and efficent) that defining a whole vector, as in: \starttyping local f_uni_base = string.formatters["uni%04X"] local f_uni_plus = string.formatters["uni%04X.y%s"] local function range(first,last) local t = { } for i=first,last do t[#t+1] = f_uni_base(i) for j=0,19 do t[#t+1] = f_uni_plus(i,j) end end return t end return { name = "hanbatanglvt", version = "1.00", comment = "Goodies that complement the hanbatanglvt fonts.", author = "Hans Hagen", colorschemes = { default = { range(0x01100,0x0115F), -- jamo_initial (r/c) range(0x01160,0x011A7), -- jamo_medial (g/m) range(0x011A8,0x011FF), -- jamo_final (b/y) } } } \stoptyping By using names we don't depend on \UNICODE\ which sometimes is needed when glyphs have ended up in the private space. However, by default, after glyphs have been mapped to colors, an extra pass will make sure that characters pushed into private space will get the same mapping as their regular \UNICODE\ has gotten (given that the number is known). Of course explicitly assigned colors will be preserved. Another generic demo feature is postprocessing. In principle one can add additional postprocessors but for that the source code needs to be consulted which in turn assumes some knowledge of fonts and \CONTEXT\ internals. \startbuffer \definefontfeature [justademoa] [default] [goodies=demo, postprocessor=squeeze] \definefontfeature [justademob] [default] [goodies=demo, postprocessor=statistics] \definefontfeature [justademoc] [default] [goodies=demo, postprocessor={statistics,squeeze}] \stopbuffer \typebuffer \getbuffer The statistics just print some font parameters to the log so that one is not showing up here. The squeeze looks as follows: \startbuffer \definefont[DemoFontD][Serif*default at 30pt] \definefont[DemoFontE][Serif*justademoa at 30pt] \stopbuffer \typebuffer \getbuffer \startlinecorrection \hbox\bgroup \ruledhbox{\color[maincolor]{DemoFontD height \& depth}}\quad \ruledhbox{\color[maincolor]{DemoFontE height \& depth}} \egroup \stoplinecorrection The squeezer just makes the height and depth of glyphs a bit smaller and it is not that hard to imagine other manipulations. The demo goodie file is good place to start playing with such things. Because there is less standardization with respect to features than one might suspect, goodie files provide a mean to define featuresets. We can use such a set in another definition: \starttyping \definefontfeature [demo-smallcaps] [goodies=demo, featureset=smallcaps] \stoptyping Of course this only makes sense for more complex combinations. The already mentioned husayni font comes with many features and most of these work together. The basic goodie table looks as follows: \startnarrowtyping return { name = "husayni", version = "1.00", comment = "Goodies that complement the Husayni font by Idris Samawi Hamid.", author = "Idris Samawi Hamid and Hans Hagen", featuresets = { }, solutions = { }, stylistics = { }, colorschemes = { }, } \stopnarrowtyping We already saw the color schemes and now we will fill in the other tables. First we define a couple of sets: \startnarrowtyping local basics = { analyze = "yes", mode = "node", language = "dflt", script = "arab", } local analysis = { ccmp = "yes", init = "yes", medi = "yes", fina = "yes", } local regular = { rlig = "yes", calt = "yes", salt = "yes", anum = "yes", ss01 = "yes", ss03 = "yes", ss07 = "yes", ss10 = "yes", ss12 = "yes", ss15 = "yes", ss16 = "yes", ss19 = "yes", ss24 = "yes", ss25 = "yes", ss26 = "yes", ss27 = "yes", ss31 = "yes", ss34 = "yes", ss35 = "yes", ss36 = "yes", ss37 = "yes", ss38 = "yes", ss41 = "yes", ss42 = "yes", ss43 = "yes", js16 = "yes", } local positioning = { kern = "yes", curs = "yes", mark = "yes", mkmk = "yes", } local minimal_stretching = { js11 = "yes", js03 = "yes", } local medium_stretching = { js12="yes", js05="yes", } local maximal_stretching= { js13 = "yes", js05 = "yes", js09 = "yes", } local wide_all = { js11 = "yes", js12 = "yes", js13 = "yes", js05 = "yes", js09 = "yes", } local shrink = { flts = "yes", js17 = "yes", ss05 = "yes", ss11 = "yes", ss06 = "yes", ss09 = "yes", } local default = { basics, analysis, regular, positioning, -- xxxx = "yes", yyyy = 2, } \stopnarrowtyping Next we define some featuresets and we use the default as starting point: \startnarrowtyping featuresets = { default = { default, }, minimal_stretching = { default, js11 = "yes", js03 = "yes", }, medium_stretching = { default, js12="yes", js05="yes", }, maximal_stretching= { default, js13 = "yes", js05 = "yes", js09 = "yes", }, wide_all = { default, js11 = "yes", js12 = "yes", js13 = "yes", js05 = "yes", js09 = "yes", }, shrink = { default, flts = "yes", js17 = "yes", ss05 = "yes", ss11 = "yes", ss06 = "yes", ss09 = "yes", }, } \stopnarrowtyping When defining the font at the \TEX\ end we can now refer to for instance \type {wide_all} which saves us some typing. However, it does not stop here. In a later paragraph we will see how fonts can work in tandem with the parbuilder. For that purpose the goodie table has a \type {solutions} subtable: \startnarrowtyping solutions = { experimental = { less = { "shrink" }, more = { "minimal_stretching", "medium_stretching", "maximal_stretching", "wide_all" }, }, } \stopnarrowtyping Here we define an experimental solution for optimizing the lines in a paragraph: we can narrow words or we can widen them according to a specific featureset. In order to reach the optimal solution the text will be retypeset under a different feature regime. {\em TODO: show how to apply.} %D \starttyping %D \setupfontsolutions[method={random,preroll},criterium=1,randomseed=101] %D %D \definefontsolution % actually only the last run needs to be done this way %D [FancyHusayni] %D [goodies=husayni, %D solution=experimental] %D %D \definedfont[husayni*husayni-default at 24pt] %D \setupinterlinespace[line=36pt] %D \righttoleft %D \enabletrackers[parbuilders.solutions.splitters.colors] %D \setfontsolution[FancyHusayni] %D alb alb alb \par %D \resetfontsolution %D \disabletrackers[parbuilders.solutions.splitters.colors] %D \stoptyping Because there are a some 55 stylistic and 21 justification variants the goodie file also provides a \type {stylistics} table and for tracing purposes the {colorschemes} table is populated. Yet another demonstration of manipulation is the following. Not all fonts come with all combined glyphs. Although we have an auto|-|compose feature in \CONTEXT\ it sometimes helps to be specific with respect to some combinations. This is where the \type {compositions} goodie kicks in: \starttyping local compose = { [0x1E02] = { anchored = "top", }, [0x1E04] = { anchored = "bottom", }, [0x0042] = { -- B anchors = { top = { x = 300, y = 700, }, bottom = { x = 300, y = -30, }, }, }, [0x0307] = { anchors = { top = { x = -250, y = 550, }, }, }, [0x0323] = { anchors = { bottom = { x = -250, y = -80, }, }, }, } return { name = "lm-compose-test", version = "1.00", comment = "Goodies that demonstrate composition.", author = "Hans and Mojca", copyright = "ConTeXt development team", compositions = { ["lmroman12-regular"] = compose, } } \stoptyping Of course this assumes some knowledge of the font metrics (in base points) and \UNICODE\ slots, but it might be worth the trouble. After all, one only needs to figure it out once. But keep in mind that it will always be a kludge. A slightly different way to define such compositions is the following: \starttyping local defaultunits = 193 - 30 local compose = { DY = defaultunits, -- [0x010C] = { DY = defaultunits }, -- Ccaron -- [0x02C7] = { DY = defaultunits }, -- textcaron } -- fractions relative to delta(X_height - x_height) local defaultfraction = 0.85 local compose = { DY = defaultfraction, -- uppercase compensation } return { name = "lucida-one", version = "1.00", comment = "Goodies that complement lucida.", author = "Hans and Mojca", copyright = "ConTeXt development team", compositions = { ["lbr"] = compose, ["lbi"] = compose, ["lbd"] = compose, ["lbdi"] = compose, } } \stoptyping Of course no one really needs this because \OPENTYPE\ Lucida fonts have replaced the \TYPEONE\ versions. The next goodie table is dedicated to the de facto standard \TEX\ font Latin Modern. There is a bit of history behind this file. When we started writing \CONTEXT\ there were not that many fonts available and so we ended up with a font system that was rather well suited for the predecessor of Latin Modern, called Computer Modern. And because these fonts came in design sizes the font system was made such that it could cope efficiently with many files in a font set. Although there is no additional overhead compared to small font sets, apart from more files, there is some burden in defining them. And, as they are the default fonts, these definitions slow down the initialization of \CONTEXT\ (which is due to the fact that the large typescript definitions were loaded and parsed). So, at some point the decision was made to kick out these definitions and move the burden of figuring out the right size to \LUA. When Latin Modern is chosen as font the effect is the same when design sizes are enabled. But, instead of many definitions (one for each combination of size and style) we now have an option. A non|-|designsize typeface is defined as follows: \startnarrowtyping \starttypescript [modern,modern-base] \definetypeface [\typescriptone] [rm] [serif] [modern] [default] \definetypeface [\typescriptone] [ss] [sans] [modern] [default] \definetypeface [\typescriptone] [tt] [mono] [modern] [default] \definetypeface [\typescriptone] [mm] [math] [modern] [default] \quittypescriptscanning \stoptypescript \stopnarrowtyping The designsize variant looks like this: \startnarrowtyping \starttypescript [modern-designsize] \definetypeface [\typescriptone] [rm] [serif] [latin-modern-designsize] [default] [designsize=auto] \definetypeface [\typescriptone] [ss] [sans] [latin-modern-designsize] [default] [designsize=auto] \definetypeface [\typescriptone] [tt] [mono] [latin-modern-designsize] [default] [designsize=auto] \definetypeface [\typescriptone] [mm] [math] [latin-modern-designsize] [default] [designsize=auto] \quittypescriptscanning \stoptypescript \stopnarrowtyping Of course there are accompanying typescripts that map the sans, serif, mono and math styles onto files. The \type {designsize} magic uses the following table. We show only part of the file, as it is in the \CONTEXT\ distribution. \starttyping return { name = "latin modern", version = "1.00", comment = "Goodies that complement latin modern.", author = "Hans Hagen", copyright = "ConTeXt development team", mathematics = { tweaks = { aftercopying = { mathematics.tweaks.fixbadprime, -- prime is too low }, }, }, designsizes = { ["LMMathRoman-Regular"] = { ["4pt"] = "LMMath5-Regular@lmroman5-math", ... ["12pt"] = "LMMath12-Regular@lmroman12-math", default = "LMMath10-Regular@lmroman10-math" }, ["LMMathRoman-Bold"] = { -- not yet ready ... }, ["LMRoman-Regular"] = { ["4pt"] = "file:lmroman5-regular", ... ["12pt"] = "file:lmroman12-regular", default = "file:lmroman10-regular", }, ["LMRoman-Bold"] = { ... }, ["LMRoman-Demi"] = { default = "file:lmromandemi10-regular", }, ["LMRoman-Italic"] = { ... }, ... ["LMRoman-Unslanted"] = { default = "file:lmromanunsl10-regular", }, ["LMSans-Regular"] = { ... }, ["LMTypewriter-Regular"] = { ... }, ... ["LMTypewriterVarWd-DarkOblique"] = { default = "file:lmmonoproplt10-boldoblique", }, ... ["LMTypewriter-CapsOblique"] = { default = "file:lmmonocaps10-oblique", }, } } \stoptyping The \type {auto} option will choose a best fit compatible to the \MKII\ implementation. When \type {default} is used instead, the default filename will be taken. Of course one might wonder if there will ever be similar goodie files because design sizes are not that popular nowadays. Not all fonts are perfect and of course the \LUATEX\ engine can have flaws as well. For this reason we can implement patches. Here is another example of a goodie file that has some more code than just a table: \starttyping local patches = fonts.handlers.otf.enhancers.patches local function patch(data,filename,threshold) local m = data.metadata.math if m then local d = m.DisplayOperatorMinHeight or 0 if d < threshold then patches.report("DisplayOperatorMinHeight(%s -> %s)",d,threshold) m.DisplayOperatorMinHeight = threshold end end end patches.register("after","analyze math","asana", function(data,filename) patch(data,filename,1350) end) local function less(value,target,original) return 0.25 * value end local function more(value,target,original) local o = original.mathparameters.DisplayOperatorMinHeight if o < 2800 then return 2800 * target.parameters.factor else return value -- already scaled end end return { name = "asana-math", version = "1.00", comment = "Goodies that complement asana.", author = "Hans Hagen", copyright = "ConTeXt development team", mathematics = { parameters = { DisplayOperatorMinHeight = more, StackBottomDisplayStyleShiftDown = less, StackBottomShiftDown = less, StackDisplayStyleGapMin = less, StackGapMin = less, StackTopDisplayStyleShiftUp = less, StackTopShiftUp = less, StretchStackBottomShiftDown = less, StretchStackGapAboveMin = less, StretchStackGapBelowMin = less, StretchStackTopShiftUp = less, } } } \stoptyping In fact, in addition to already mentioned \type {mapfiles} and \type {virtuals} subtables, we can pass variables and overload parameters. \starttyping return { name = "lm-math", ... mathematics = { mapfiles = { ... }, virtuals = { ... variables = { joinrelfactor = 3, -- default anyway }, parameters = { -- test values -- FactorA = 123.456, -- FactorB = false, -- FactorC = function(value,target,original) -- return 7.89 * target.factor -- end, -- FactorD = "Hi There!", }, } } \stoptyping This kind of goodie functionality is typical for the development of \LUATEX\ and experimental math fonts and no user should ever be bothered with it. However, it demonstrates that we're not stuck with only features built in the fonts. % mathdimensions It can be that a user is not satisfied by some aspects of a math font design. There is not much that we can do about the shapes, but we can manipulate for instance dimensions. For this there are two mechanism available: automatically applied dimensional fixes and a \type {mathdimensions} feature. Both work with the same goody specification. \starttyping mathematics = { ... dimensions = { }, ... } \stoptyping The entries in a dimensions table are tables themselves. There can be many of them so one can organize dimensional tweaks in groups. The \type {default} group is always applied, while others are applied on demand. Say that want to tweak all \type {±} and \type {∓}. \footnote {In fact, this mechanism is a a response to a mail on the \CONTEXT\ mailing list.} \starttyping mathematics = { dimensions = { default = { [0x00B1] = { -- ± height = 500, depth = 0, }, [0x2213] = { -- ∓ height = 500, depth = 0, }, }, }, } \stoptyping This will give these two characters a different height and depth. However, this will not have much effect in rendering (much larger dimensions might have). \starttyping mathematics = { dimensions = { default = { [0x00B1] = { -- ± yoffset = 100, }, [0x2213] = { -- ∓ yoffset = -100, }, }, }, } \stoptyping This will raise and lower the glyphs in their bounding boxes and give them an appearance more close to their ancestors. But defined this way, they are always applied and that might not be what we want. So, we can do this: \starttyping mathematics = { dimensions = { signs = { [0x00B1] = { -- ± yoffset = 100, }, [0x2213] = { -- ∓ yoffset = -100, }, }, }, } \stoptyping This time the application is feature driven. As with all features, setting them up has to happen {\em before} fonts are loaded. This will do the trick: \starttyping \definefontfeature [lm-math] [mathdimensions=signs] \stoptyping The \type {lm-math} feature is not defined by default but can be used for such purposes. It {\em is} defined with the fontname: \starttyping \definefontsynonym [LMMathRoman-Regular] [file:latinmodern-math-regular.otf] [features={math\mathsizesuffix,lm-math}, goodies=lm] \stoptyping A rather specialized goodie is the one that is used to specify math cut|-|ins. A good quality math font has these kerns already defined but even then you might want to add or replace some by your own. Here is an example of such a patch. Normally there are multiple goodies defined in one file but we only show kerns here: \starttyping local kern_200 = { bottomright = { { kern = -200 } } } local kern_100 = { bottomright = { { kern = -100 } } } return { name = "pagella-math", version = "1.00", comment = "Goodies that complement pagella.", author = "Hans Hagen", copyright = "ConTeXt development team", mathematics = { kerns = { [0x1D449] = kern_200, -- math italic V [0x1D44A] = kern_100, -- math italic W }, }, } \stoptyping As with other goodies the file is loaded with: \starttyping \definefontsynonym [MathRoman] % names used in definitions [file:texgyrepagella-math.otf] % the file to be loaded [features=math\mathsizesuffix, % size dependent features goodies=pagella-math] % the goodie file to be applied \stoptyping This is typically a goodie that is always applied and not driven by a feature. After all, the values given are passed to the engine (after being scaled). Most goodies are bound to fonts of collections of fonts. This is different for treatments. These ship with the distribution but you can also provide your own. As this is still somewhat experimental we just mention a few aspects. The entries are filenames that point to tables. \starttyping return { name = "treatments", version = "1.00", comment = "Goodies that deals with some general issues.", author = "Hans Hagen", copyright = "ConTeXt development team", treatments = { ["adobeheitistd-regular.otf"] = { embedded = false, -- not yet used comment = "this font is part of acrobat", }, ["crap.ttf"] = { ignored = true, comment = "a text file with suffix ttf", }, ["latinmodern-math.otf"] = { comment = "experimental", }, ["rubish-regular.ttf"] = { comment = "check output for missing à and á", } }, } \stoptyping The comment entry in such a table becomes part of the message at the end of a run: \startnarrowtyping mkiv lua stats > loaded fonts: 2 files: latinmodern-math.otf (experimental), lmroman12-regular.otf \stopnarrowtyping The ignored flag signals the font name database builder to ignore the file. This means that the font can still be known as file, but that its (name based) properties are not collected. As you asked explicitly for a file, the file can still be loaded. You can use this trick to avoid issues with the database builder in case of a problematic file, but a real run will still try to load the file. After all, you get what you ask for. If loading and usage is successful you get at least the message reported at the end of the run. \stopsection \startsection[title=Analyzers] An \OPENTYPE\ font is kind of special in the sense that it provides some information on how to turn sequences of characters into sequences of glyphs. In fact, if all fonts had a reasonable repertoire of glyphs most of the information that concerns combining, remapping and shuffling the input and|/|or mapping onto glyphs could as well happen in the renderer. This means that fonts have many of their internal features tables in common, or more precisely could share many gsub related issues, if only there had been some predefined sets of substitutional features. So, for most of the time, a feature processor just does what the font demands and the font provides the information. There are however a few cases where font only provide part of the logic. Take for instance the \type {init}, \type {medi}, \type {fina} and \type {isol} features that relate to positions in the word: the start, the end, in the middle or isolated. For these features to work the engine has to provide information about the state of a character (glyph) and this is where analysis kicks in. Just watch this: \startbuffer \definefontfeature [default-with-analyze] [default] [script=latn,mode=node, init=yes,medi=yes,fina=yes,isol=yes] \showotfcomposition {dejavu-serif*default-with-analyze at 24pt} {} {I don't wanna know tha\utfchar{"300}t!} \stopbuffer \typebuffer In the tracer the different categories are colored. This kind of information is especially important for typesetting Arabic. Normally \CONTEXT\ can figure out itself when this is needed so you don't have to worry too much about this kind of additional actions. \blank \getbuffer \blank \stopsection \startsection[title=Processors] \startnotabene Todo. \stopnotabene \stopsection \startsection[title=Optimizing] \startnotabene Todo. \stopnotabene \stopsection \startsection[title=Tracing] There are a lot of tracing options in \MKIV, but most will never be seen by users. Most are enabled using the tracker mechanism. Some have a bit more visibility and have a dedicated command to trigger them. When something is going terribly wrong, you will always get a message but sometimes even an end|-|user has to request for more information. An example are missing characters. There are several ways to get them reported: \starttyping \enabletrackers[fonts.missing=replace] \enabletrackers[fonts.missing=remove] \enabletrackers[fonts.missing] \stoptyping For historic reasons we also have: \starttyping \checkcharactersinfont \removemissingcharacters \replacemissingcharacters \stoptyping which happens automatically when you enable the tracker. There is some extra overhead involved so you might want to turn on this feature on only if you really expect characters not to be present. Say that we use Latin Modern fonts and ask for some of the rare fractions: \startbuffer \definedfont[lmroman10-regular*default-with-missing at 10pt] a b c ½ ⅓ ¼ ⅕ ⅙ ⅛ Ɣ ɣ ʤ ʭ ʮ α β γ \stopbuffer \typebuffer \enabletrackers[fonts.missing=replace] We get this: \start \getbuffer \stop \removeunwantedspaces . \space In the log file you will find something like this: \par \disabletrackers[fonts.missing] \starttyping fonts > characters > start missing characters: lmroman10-regular.otf missing > U+00194 Ɣ LATIN CAPITAL LETTER GAMMA missing > U+00263 ɣ LATIN SMALL LETTER GAMMA missing > U+002A4 ʤ LATIN SMALL LETTER DEZH DIGRAPH missing > U+002AD ʭ LATIN LETTER BIDENTAL PERCUSSIVE missing > U+002AE ʮ LATIN SMALL LETTER TURNED H WITH FISHHOOK missing > U+003B1 α GREEK SMALL LETTER ALPHA missing > U+003B2 β GREEK SMALL LETTER BETA missing > U+003B3 γ GREEK SMALL LETTER GAMMA missing > U+02153 ⅓ VULGAR FRACTION ONE THIRD missing > U+02155 ⅕ VULGAR FRACTION ONE FIFTH missing > U+02159 ⅙ VULGAR FRACTION ONE SIXTH missing > U+0215B ⅛ VULGAR FRACTION ONE EIGHTH fonts > characters > stop missing characters \stoptyping If you're lucky your editor will use a font that shows the missing characters (dejavu monospace is a good candidate). The replacement characters can help you to locate the spots where something is missing so that an alternative can be considered. The replacements resemble the category of the missing character. \showmissingcharacterslegend You can call up this legend after loading an extra module: \starttyping \usemodule[s][fonts-missing] \showmissingcharacterslegend \showmissingcharacters \stoptyping The last command shows a detailed list of missing characters \showmissingcharacters Here the characters are shown, because we use a monospaced font that happens to have them. Of course this example uses characters that are rarely used and are unlikely to show up in future versions of the Latin Modern fonts. \startnotabene Here a few more relevant trackers will be mentioned. \stopnotabene \stopsection % \startsection[title=Discretionaries] % % \startbuffer % \definedfont[cambria*default] % 12\discretionary % {3} {4} {5}% % 67\par % 12{\oldstyle\discretionary % {3} {4} {5}}% % 67\par % 12\discretionary % {3{\oldstyle3}} {{\oldstyle4}4} {5{\oldstyle5}5}% % 67\par % \stopbuffer % % The font handler has to do some magick to get features working with and across % discretionaries. To some extend you can use font switches inside discretionaries % but for sure border cases are not dealt with. This works: % % \startlinecorrection[blank] % \startcombination[nx=4,ny=1,location=top] % {\framed[align=normal]{\enabledirectives [otf.alwaysdisc]\setupwhitespace[line]\getbuffer}} {1} % {\framed[align=normal]{\enabledirectives [otf.alwaysdisc]\hsize1mm\getbuffer}} {2} % {\framed[align=normal]{\disabledirectives[otf.alwaysdisc]\setupwhitespace[line]\getbuffer}} {3} % {\framed[align=normal]{\disabledirectives[otf.alwaysdisc]\hsize1mm\getbuffer}} {4} % \stopcombination % \stoplinecorrection % % The first two examples have \type {otf.alwaysdisk} enabled, the last two have it % disabled. % % \typebuffer % % \stopsection \startsection[title=Some remarks] If you talk about features and fonts it is not difficult to end up speaking \OPENTYPE . However, in \CONTEXT\ we use the term in a more general way, if only because we provide more features. In traditional \TEX\ we have a few features: ligatures, smallcaps and kerns, and to some extent we can see oldstyle numerals also as feature. It is however important to notice that in \OPENTYPE\ ligatures are just a synonym for combining multiple characters into on. From the user interface point of view these operations are grouped into \type {liga}, \type {dlig}, \type {clig} and \type {rlig} and for \TEX ies we have \type {tlig}. The distinction is not as clear as one might think: any feature can use the ligature builder. And as a consequence we see that happen too, for instance some fonts use \type {ccmp} for constructing mandatory ligatures. Some of these interpretations (or maybe even tricks) are side effects of for instance user interfaces. If one can for instance not turn on or off the \type {ccmp} feature, but can do that for \type {liga}, then one way to keep some ligatures in for instance letter spaced text, is to put them into \type {ccmp}, assuming that this one will always be enabled. Eventually that then becomes a sort of standard. Personally I don't like such pseudo standards but we have to live with them. Another example of such a standard is the used of non breakable spaces to influence treatment of some Devanagari characters. Where \UNICODE\ has special characters to influence mechanisms that combine and replace characters, the lack of some triggers others to be used and eventually that becomes a standard. Similar ambiguities arise with math: we have no way to indicate math (while we do have ways to indicate a change in writing order). Talking of math, take \OPENTYPE\ math: at some point there is a draft, that then gets implemented in one word processor using one font, but omissions or imperfections that surface (maybe because more fonts and engines are developed) stay around because the initial implementation is published and frozen, simply because there are many users that stick to expectations. Where \TEX ies accept a few years of development, this is not true for commercial applications. \footnote {Of course \HTML\ is the biggest example of this: we're stuck forever with open tags without close tags, mixed uppercase and lowercase tags, attributes without value or values without quotes.} So, although there is without doubt progress, some annoyances stay. The \TEX\ community has always been able to adapt, and this is one reason why a \LUA\ implementation is nice: it gives room for experiments, extensions, variants, etc. Of course it also makes a bit more independent, although one may wonder if that matters any longer in a rapidly changing world. The original idea behind \TEX, that it should be useable for ages, will survive, but users might see more changes in a lifetime than foreseen when \TEX\ showed up. \stopsection \startsection[title=Different spaces] The width of the space and its stretch and shrink are taken from the font. The so called emspace is the reference for much spacing related parameters. It is the width of character \type {0x2014}. The regular space width is taken from \type {0x0020}, the space character. When there is no space character, in the case of a monospaced font we take the emwidth, otherwise we take half the emwidth. As a last resort we can take the average width of characters. And of even that fails we take half of the font units. When there is no emwidth that one is set to the font units. In the \CONTEXT\ font loader we use a stretch that is 1/2 of the width of a space and the shrink is 1/3 the width of a space, so we use values that are quite similar to what \TEX\ always used. You can overload these values when a font is loaded and the overload is implemented as a feature. The next example demonstrates how this is done: \startbuffer \definefontfeature[morespace][spacing=0.50 plus 0.50 minus 0.250] \definefontfeature[lessspace][spacing=0.25 plus 0.25 minus 0.125] \definedfont[Serif*default] \samplefile{klein}\blank \definedfont[Serif*default,morespace]\samplefile{klein}\blank \definedfont[Serif*default,lessspace]\samplefile{klein}\blank \definedfont[Serif*default] \samplefile{klein}\blank \stopbuffer \typebuffer \blank \getbuffer \blank \stopsection \startsection[title=Dynamic features] We can enable and disable features any time in the input by using the \type {\feature} command. he following example was posted on the list: \startbuffer \definefont [WeirdShapes] [file:libertiner*default] \definefontfeature [hist] [hlig=yes] \definefontfeature [rare] [dlig=yes] \setupquotation [style={\feature[+][hist,rare]}] \startlines \WeirdShapes strict {\feature[+][hist]strict} wurtzite {\feature[+][rare]wurtzite} \quotation{strict wurtzite} \stoplines \stopbuffer \typebuffer Or typeset: \getbuffer The \type {\feature} command takes as first argument a directive of what do do: \starttabulate[|T||] \NC + more \NC add set to previous set and combine with font set \NC \NR \NC - less \NC subtract set to previous set and combine with font set \NC \NR \NC = new \NC replace font set \NC \NR \NC ! < reset \NC forget sets and revert to font set \NC \NR \NC > old default \NC make sure the current set is used on top of the font set \NC \NR \stoptabulate \stopsection \startsection[title=Spacekerns] Some fonts kern glyphs with spaces. Although \TEX\ doesn't really have spaces we do support this. However, it's implemented as part of kerning so when you define such kerns you need to hook it into for instance the \type {kern} feature: \starttyping \startluacode local kern = -50 local pair = { [32] = kern } fonts.handlers.otf.addfeature { name = "kern", -- spacekerns assume kern type = "kern", data = { A = pair, V = pair, W = pair, [32] = { A = kern, V = kern, W = kern, }, } } \stopluacode \stoptyping Of course this depends on font properties so one can wonder how useful this is. \stopsection \stopchapter \stopcomponent