fonts-features.tex /size: 95 Kb    last modification: 2021-10-28 13:50
1% language=us runpath=texruns:manuals/fonts
2
3\startcomponent fonts-features
4
5\environment fonts-environment
6
7% windows: seguiemj.ttf              (windows 10)
8% public : emojionecolor-svginot.ttf (https://github.com/eosrei/emojione-color-font)
9
10\startMPextensions
11    vardef MyRectangle(expr n, w, h, x, y, c) =
12        image (
13            fill unitsquare xyscaled (w,h) shifted (x,y) withcolor c ;
14            draw textext("\tttf " & decimal n) xsized (1/2) shifted (w/2,h/2) shifted (x,y) withcolor white ;
15        )
16    enddef ;
17    vardef MyDot(expr x, y) =
18        image (
19            draw (x,y) withpen pencircle scaled (2/3) withcolor white ;
20            draw (x,y) withpen pencircle scaled (1/2) withcolor black ;
21        )
22    enddef ;
23\stopMPextensions
24
25\startchapter[title=Features][color=darkmagenta]
26
27\startsection[title=Introduction]
28
29If you look into fonts, it is hard not to bump into kerns (spacing between
30characters) and ligatures (combining multiple shapes into one) and apart from
31monospaced fonts most \TYPEONE\ fonts have them. In the \OPENTYPE\ universe we
32call these properties features and in such a font there can be many such
33features.
34
35For those who grew up with \TEX\ or still remember the times of eight bit fonts,
36it is no secret that \TEX\ macro packages did some magic to get most out of a
37font: replacing missing glyphs, fixing metrics, using commands to access shapes
38that had a weird code point, to mention a few. As there is absolutely no
39guarantee that an \OPENTYPE\ font does better, there is a good reason to continue
40messing around with fonts. After all, it's what \TEX\ users seem to like:
41control.
42
43So, when we started writing support for \OPENTYPE\ quite soon a mechanism has
44been created that permits adding our own features to the repertoire that comes
45with a font. Because \OPENTYPE\ features demand a configuration and control
46mechanism, it made sense to generalize that and provide a single interface.
47
48This means that when we talk about font features, we don't limit ourselves to
49those provided by the font, but also those provided by \CONTEXT. As with font
50features, they are enabled per font.
51
52Some of the extra features are sort of generic, others are very font specific and
53their properties are somewhat bound to a font. Such features are defined in a
54font goodie files. Consider these goodies a font extension mechanism.
55
56Some features need information that only the engine can provide. This is why we
57have analyzers. Some are generic, others are bound to scripts. They come in
58action before features are applied. Rather special is applying features in
59combination with paragraph building. This is something very specific to \CONTEXT\
60but it depends on properties of the font. It falls into the category \quote
61{optimizing}.
62
63It is clear that when we talk of features many aspects of a font play a role. In
64this chapter we will discuss all the mentioned aspects. There is quite a bit of
65\LUA\ code shown in this chapter, but don't worry, users will seldom need to
66tweak fonts this way. On the other hand it's good to see what is possible.
67
68\stopsection
69
70\startsection[title=Regulars]
71
72\startsubsection[title=Introduction]
73
74The \OPENTYPE\ specification, which can be found on the \MICROSOFT\ website
75is no easy reading. Some of the concepts are easy to understand, like relative
76positioning (that we call kerning in \TEX) or ligature substitution (as we
77have ligatures in \TEX\ too). It makes no sense to discuss the bitwise composition
78of an \OPENTYPE\ or \TRUETYPE\ file here. First of all, all we get to see is
79a \LUA\ table, and in \CONTEXT\ even that one gets sanitized and optimized
80into a more useable table. However, as the data that comes with a font is
81a good indication of what a font is capable of, we will discuss some of it in
82an appendix. In this section we will discuss the basic principles and categories
83of features.
84
85\stopsubsection
86
87\startsubsection[title=Feature sets]
88
89Because in the next examples we will demonstrate features, we need to know how
90we can tell \CONTEXT\ what features to use. Although you can add explicit
91feature definitions to a font specification, I strongly advice you not to do this
92but use the more abstract mechanism of feature sets. These are defined as follows:
93
94\starttyping
95\definefontfeature
96  [MyFeatureSet]
97  [alpha=yes,
98   beta=no,
99   gamma=123]
100\stoptyping
101
102Such a set is bound to a font with the \type {*} specifier, as in:
103
104\starttyping
105\definefont
106  [MyFontInstance]
107  [MyNiceFont*MyFeatureSet at 12pt]
108\stoptyping
109
110In most cases the already defined \type {default} feature set will suffice. It often
111makes sense to use that one as base for new definitions:
112
113\starttyping
114\definefontfeature
115  [MyFeatureSet]
116  [default]
117  [alpha=yes,
118   beta=no,
119   gamma=123]
120\stoptyping
121
122The second argument can be a list, as in:
123
124\starttyping
125\definefontfeature
126  [MyFeatureSet]
127  [MyFirstSet,MySecondSet]
128  [alpha=yes,
129   beta=no,
130   gamma=123]
131\stoptyping
132
133Of course you need to know what features a font support, and one way to find
134out is:
135
136\starttyping
137mtxrun --script font --list --info --pattern=pagella
138\stoptyping
139
140Don't be too surprised if different fonts show different features and even similar
141features can be implemented differently. Sometimes you really need to know the font,
142but fortunately many fonts come with examples.
143
144There are many features and there values are kept with the font when it gets
145defined. This means that when you change a featureset, it will not affect already
146defined fonts. Because fonts are often defined on demand, you need to be aware of
147the fact that a redefinition of a featureset can have consequences for already
148defined fonts. For instance, a bodyfont switch only sets up the fonts and delays
149defining them.
150
151Although features are a sort of abstractions it can be interesting to see what features
152and values are actually used:
153
154\starttyping
155\usemodule[fonts-features] \showusedfeatures
156\stoptyping
157
158You will notice that we have more features than \OPENTYPE\ fonts can offer. This
159is because in \CONTEXT\ features is a more general concept.
160
161\showusedfeatures
162
163\stopsubsection
164
165\startsubsection[title=Main categories]
166
167There are two (but potentially more) main groups of features: those that deal
168with substitution, and those that lead to positioning. It is not really needed
169to know the gory details, but it helps to know at least a bit about them as
170it can help to track down issues with fonts.
171
172There are several substitutions possible:
173
174\startitemize[packed]
175\startitem a single substitution replaces one glyph by another \stopitem
176\startitem a multiple substitution replaces one glyph by one or more \stopitem
177\startitem a ligature substitution replaces multiple glyphs by one glyph \stopitem
178\startitem an alternate substitution replaces one glyph by one out of a set \stopitem
179\stopitemize
180
181Like it or not, but these categories are not always used as intended: they just
182are a way of replacing one or more glyphs by one or more other glyphs. This means
183that when for instance \type {ij} gets replaced by one glyph (given that the font
184supports it) a ligature substitution is used, even when in fact we have to do
185with a diftong that can be represented by one character.
186
187No matter what features you will use, keep in mind that they are nothing more
188than a combination of substitutions and positioning directives. So, the de facto
189standard ligature building feature \type {liga} indeed uses a ligature
190substitution, but other features with names that resemble no ligatures might use
191this substitution as well.
192
193An example of a single substitution is an oldstyle (\type {onum}) although it can
194as well be implemented as a choice out of alternate glyphs. Another example is
195smallcaps (\type {smcp}). Nowdays these are more or less standard features for a
196grown up font, while in the past they came as separate fonts. So, instead of loading
197an extra font, one sticks to one and selects the feature that does the
198substitution.
199
200A second category concerns relative positioning. Again we have several variants:
201
202\startitemize[packed]
203\startitem a single positioning moves a glyph over one of two axis and can change the width and|/|or height \stopitem
204\startitem a pair positioning also moved glyphs but concerns two adjacent glyphs \stopitem
205\startitem a cursive positioning operates on a range of glyphs and is used to visually connect them \stopitem
206\stopitemize
207
208In addition there are three ways to anchor marks onto glyphs:
209
210\startitemize[packed]
211\startitem a mark can be anchored on a base glyph \stopitem
212\startitem a mark can be anchored on a specific (visual) component of a ligature \stopitem
213\startitem a mark can be anchored on another mark \stopitem
214\stopitemize
215
216In base mode the single, alternate and ligature substitutions can rather easily
217be mapped onto the traditional \TEX\ font handling mechanism and this is what
218happens in base mode. A single substitution is just another instance of a glyph
219so there we just replace the original index into the glyph table by another one.
220In the case of an alternate we change the default index into one of several
221possible replacements in the alternate set. Ligatures can be mapped onto \TEX s
222ligature mechanism. The single positioning maps nicely on \TEX s kerning
223mechanism and pairwise positioning is not applicable in base mode. In node mode
224we don't do any remapping at loading time but delegate that to \LUA\ when
225processing the node lists.
226
227Marks are special in the sense that they normally only occur in scripts that also
228use substitution and positioning which in turn means that some more housekeeping
229is involved. After all, we need to keep track to what a mark applies. Of course a
230font can provide regular latin accents as marks but that is ill practice because
231cut and paste might not work out as expected. A proper font will support composed
232characters and provide glyphs that have the accents built in. Marks are not dealt
233with in base mode.
234
235Talking of complex scripts, the above set of operations is far from enough. Take
236for instance Arabic, where a sequence of 5~characters with 3~marks can easily
237become two glyphs glued together with two marks only. In the process we can have
238single substitutions, ligatures being built, marks being anchored and glyphs
239being cursively positioned. But, in order to do this well, some contextual
240analysis has to be done as well. Again we have several variants of this:
241
242\startitemize[packed]
243\startitem with contextual substitution a replacement takes place depending on a matching sequence of glyphs,
244optionally preceded or followed by matches \stopitem
245\startitem with contextual positioning shifting and anchoring happens based on a matching sequence of glyphs,
246optionally preceded or followed by matches \stopitem
247\startitem multiple contextual substitutions or positionings can be chained together \stopitem
248\startitem this can also happen in the reverse order (for right|-|to|-|left scripts) \stopitem
249\stopitemize
250
251In practice there is no fundamental difference between these and we can collapse
252them all in a sequence of lookups resulting in a sequence of whatever other
253manipulation is wanted.
254
255Given this, what is a feature? It's mostly a sequence of actions expressed in the
256above. And although there is a whole repertoire of semi|-|standardized features
257like \type {liga} and \type {onum}, there is no real hard coded support for them
258in \CONTEXT. Instead we have a generic feature processor that deals with all of
259them. A feature, say \type {abcd}, has a definition that boils down to a sequence
260of lookups. A lookup is just a name that is associated to one of the mentioned
261actions. So, \type {abcd} can do a decomposition (multiple substitution), then a
262replacement (single substitution) based on neighbouring glyphs, then do some
263ligature building (ligature substitution) and finally position the resulting
264glyphs relative to each other (like cursive positioning and anchoring marks).
265
266Imagine that we start out with 5 characters in the input. Instead of real glyphs
267we represent them by rectangles. The third one is a mark.
268
269\startlinecorrection
270\startMPcode
271  draw MyRectangle(1,2,6, 0,0,.5red    ) ;
272  draw MyRectangle(2,2,4, 3,0,.5green  ) ; draw MyDot(4,4.25) ;
273  draw MyRectangle(3,2,1, 6,5,.5blue   ) ; draw MyDot(7,4.75) ;
274  draw MyRectangle(4,2,5, 9,0,.5yellow ) ;
275  draw MyRectangle(5,2,5,12,0,.5magenta) ;
276  currentpicture := currentpicture ysized(4cm) ;
277\stopMPcode
278\stoplinecorrection
279
280In the next variant we see that four and five have been replaced by
281number six. This is a ligature replacement.
282
283\startlinecorrection
284\startMPcode
285  draw MyRectangle(1,2,6,0,0,.5red  ) ;
286  draw MyRectangle(2,2,4,3,0,.5green) ; draw MyDot(4,4.25) ;
287  draw MyRectangle(3,2,1,6,5,.5blue ) ; draw MyDot(7,4.75) ;
288  draw MyRectangle(6,3,5,9,0,.5cyan ) ;
289  currentpicture := currentpicture ysized(4cm) ;
290\stopMPcode
291\stoplinecorrection
292
293The mark is an independent entity. Sometimes it has a width, sometimes it hasn't.
294In both cases we can position it. Here we move the shape left and down. There are
295two ways to do this: simple pairwise kerning but better is to use anchors. Here
296we have one anchor per shape but there can be many.
297
298\startlinecorrection
299\startMPcode
300  draw MyRectangle(1,2,6,0,0  ,.5red  ) ;
301  draw MyRectangle(2,2,4,3,0  ,.5green) ;
302  draw MyRectangle(3,2,1,3,4.5,.5blue ) ; draw MyDot(4,4.25) ;
303  draw MyRectangle(6,3,5,6,0  ,.5cyan ) ;
304  currentpicture := currentpicture ysized(4cm) ;
305\stopMPcode
306\stoplinecorrection
307
308Next we apply some kerning. Of course the anchored marks need to move as well.
309
310\startlinecorrection
311\startMPcode
312  draw MyRectangle(1,2,6,0,  0  ,.5red  ) ;
313  draw MyRectangle(2,2,4,2.5,0  ,.5green) ;
314  draw MyRectangle(3,2,1,2.5,4.5,.5blue ) ; draw MyDot(3.5,4.25) ;
315  draw MyRectangle(6,3,5,5,  0  ,.5cyan ) ;
316  currentpicture := currentpicture ysized(4cm) ;
317\stopMPcode
318\stoplinecorrection
319
320Alternatively we can connect the shapes in a cursive way. The name cursive is
321somewhat misleading as it just boils down to shifting. The cursive indicates that
322the shifts accumulate within a word.
323
324\startlinecorrection
325\startMPcode
326  draw MyRectangle(1,2,6,0,0  ,.5red  ) ;
327  draw MyRectangle(2,2,4,2,0.5,.5green) ;
328  draw MyRectangle(3,2,1,2,5  ,.5blue ) ; draw MyDot(3,4.75) ;
329  draw MyRectangle(6,3,5,4,1  ,.5cyan ) ;
330  currentpicture := currentpicture ysized(4cm) ;
331\stopMPcode
332\stoplinecorrection
333
334\stopsubsection
335
336\startsubsection[title={Single substitution}]
337
338Single substitutions are probably the most used ones. For instance, when you
339ask for small caps, a lot of glyphs get replaced. When using oldstyle numerals
340only digits get replaced but even then each glyph has to be checked. This can be
341demonstrated with the Latin Modern fonts.
342
343\startlinecorrection
344\scale
345  [height=1cm]
346  {\strut
347   {\definedfont[lmroman10-bold*default]\$123.45}%
348   \quad
349   {\definedfont[lmroman10-bold*oldstyle]\$123.45}}
350\stoplinecorrection
351
352As you can see here, Latin Modern has an oldstyle dollar sign. If you don't like
353that one, you're in troubles as it comes with the rest of the oldstyles. The only
354way out is to apply the oldstyle numerals to digits only which involves more tagging
355than you might be willing to add. So, whenever you choose a substitution, be aware
356that you have not that much control over what gets substituted: it's the font that
357drives it. Here are some examples:
358
359\starttyping
360\definefontfeature[capsandold][smallcaps,oldstyle]
361
362\showotfcomposition{dejavu-serif*capsandold          at 24pt}{}{It's 2013!}
363\showotfcomposition{cambria*capsandold               at 24pt}{}{It's 2013!}
364\showotfcomposition{lmroman10regular*capsandold      at 24pt}{}{It's 2013!}
365\showotfcomposition{texgyrepagellaregular*capsandold at 24pt}{}{It's 2013!}
366\stoptyping
367
368\definefontfeature[capsandold][smallcaps,oldstyle]
369
370\blank \showotfcomposition{dejavu-serif*capsandold          at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank
371\blank \showotfcomposition{cambria*capsandold               at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank
372\blank \showotfcomposition{lmroman10regular*capsandold      at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank
373\blank \showotfcomposition{texgyrepagellaregular*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank
374
375\stopsubsection
376
377\startsubsection[title={Multiple substitution}]
378
379In a multiple substitution a sequence of characters (glyphs) gets replaced by
380another sequence. In fact, you might wonder why one||to||one, multiple||to||one
381and multiple||to||multiple are not all generalized into this variant. Efficiency
382is probably the main reason. \footnote {Isn't it strange that complex mechanisms
383are designed to save a few bytes while at the same time we produce ridiculous
384large pictures with cameras.} For instance the many||to||one is often used for
385ligatures (\type {liga}) and as a consequence \type {liga} is often misused also
386for non||ligatures.
387
388One usage of multiple replacements is to avoid and or undo other replacements. In
389the next example we see a language dependent \type {fi} ligature. Take the dutch
390\type {ij} and \type {ie} diftongs. Here we need to prevent the \type {i}
391becoming combined with the \type {f} as it would look weird. Among the solutions
392for this are: context dependent ligatures (which involves a lot of rules), or
393multiple to multiple replacements (looking at the \type {fij} sequence).
394
395\startbuffer[definitions]
396\definefontfeature[default-fijn-en][default][language=eng,script=latn]
397\definefontfeature[default-fijn-nl][default][language=nld,script=latn]
398\stopbuffer
399
400\getbuffer[definitions] \typebuffer[definitions]
401
402\starttyping
403\definedfont[lmroman10-regular*default-fijn-en]\en effe fijn fietsen
404\definedfont[lmroman10-regular*default-fijn-nl]\nl effe fijn fietsen
405\stoptyping
406
407This gives:
408
409\startlinecorrection[blank]
410\scale [width=\textwidth] \bgroup
411    \framed [offset=overlay,frame=off,foregroundcolor=maincolor,align=normal,strut=no] \bgroup
412        \definedfont[lmroman10-regular*default-fijn-en]\en effe fijn fietsen\vskip-1ex
413        \definedfont[lmroman10-regular*default-fijn-nl]\nl effe fijn fietsen\par
414    \egroup
415\egroup
416\stoplinecorrection
417
418Of course from this result one cannot see what (combination of) substitution(s)
419was used, but it's a nice exercise to work out a solution.
420
421Multiple substitutions are mostly used for scripts more complex than latin or
422special fonts like Zapfino where advanced contextual analysis happens.
423
424\stopsubsection
425
426\startsubsection[title={Alternate substitution}]
427
428Alternates are simple one||to||one substitutions. Popular examples are small
429capitials and oldstyle numerals.
430
431A nice application of alternates is the punk font. This font is a Knuth original.
432As part of experimenting with the \METAPOST\ library in the early days of
433\LUATEX\ and \MKIV, runtime randomization was implemented. However, that variant
434used virtual fonts and was somewhat resource hungry. So, in a later stage Khaled
435Hosny made an \OPENTYPE\ version using \METAPOST\ output. Randomization is
436implemented through the \type {rand} feature.
437
438In \MKIV\ the \type {rand} feature is not really special and behaves just like
439any other (stylistic) alternate. The only difference is that for this feature a
440value of \type {yes} equals \type {random}. This also means that any feature that
441uses alternates use them randomly.
442
443\startbuffer
444\definefontfeature[punknova-first] [mode=node,kern=yes,rand=first]
445\definefontfeature[punknova-2]     [mode=node,kern=yes,rand=2]
446\definefontfeature[punknova-yes]   [mode=node,kern=yes,rand=yes]
447\definefontfeature[punknova-random][mode=node,kern=yes,rand=random]
448\stopbuffer
449
450\typebuffer \getbuffer
451
452We use this is:
453
454\startbuffer[sample]
455The original punk font is designed by Don Knuth: xxxxxxxxxxxx
456\stopbuffer
457
458\startbuffer
459\definedfont[punknova-regular                 at 15pt] \getbuffer[sample]
460\definedfont[punknova-regular*punknova-first  at 15pt] \getbuffer[sample]
461\definedfont[punknova-regular*punknova-2      at 15pt] \getbuffer[sample]
462\definedfont[punknova-regular*punknova-yes    at 15pt] \getbuffer[sample]
463\definedfont[punknova-regular*punknova-random at 15pt] \getbuffer[sample]
464\stopbuffer
465
466\typebuffer[sample]
467
468\typebuffer
469
470In order to illustrate the variants we show a sequence of \type {x}'s. There are
471upto ten different variants per characters.
472
473\startlines[color=maincolor] \getbuffer \stoplines
474
475There is one pitfall with random alternates: if each run leads to a different
476outcome, we can end up in oscillation: different shapes give different paragraphs
477and we can get more pages or cross references etc.\ that can end up differently
478so this is why \CONTEXT\ always uses the same random seed (which gets reset when
479you purge the auxiliary files.
480
481\stopsubsection
482
483\startsubsection[title={Ligature substitution}]
484
485A ligature is traditionally a combination of several characters into one. Popular
486ligatures are \quote {fi}, \quote {fl}, \quote {ffi} and , \quote {ffl}.
487Occasionally we see \quote {\ae}, \quote {\oe} and some more. Often ligatures are
488language dependant. For instance in languages like Dutch and German there can be
489compound words where one part ends with an \type {f} and the next part starts with
490an \type {f} and that looks bad or at least not intuitive. To some extent one
491can wonder if this tradition of ligatures is a good one. It definitely made
492sense ages ago, but I wouldn't be surprised if they are often added to fonts
493because the encoding vectors have them. After all, nothing prevents to go ahead
494and come up with way more ligatures.
495
496There can be many ligature features in a font. Although we support arbitrary
497features, that is: those not registered as being official one way or the other,
498the following are known by description:
499
500\startluacode
501context.starttabulate { "|lTCT{maincolor}|l|" }
502for k, v in table.sortedhash(fonts.handlers.otf.tables.features) do
503    if string.find(v,"ligature") then
504        context.NC()
505        context(k)
506        context.NC()
507        context(v)
508        context.NR()
509    end
510end
511context.stoptabulate()
512\stopluacode
513
514The \type {default} feature set has type {liga} as wel as the \TEX\ specific \type {tlig}
515that replaces successive hyphen signs into en- and emdashes. The \type {arabic} feature
516set also has \type {rlig} enabled.
517
518Now, there is one thing you should realize when we discuss specific features and
519the underlaying mechanisms: there is no real relationship between the features's
520name and the mechanisms used: any feature can use any underlying mechanism or
521combination. This is why deep down we see that what is internally called ligature
522gets used for any purpose where multiple||to||one replacements happen, and why the
523\type {liga} feature can use single substitutions or alternates to swap in
524another rendering so that the dot of the \type {i} stays free of the preceding
525\type {f}. And for some fonts relative positioning can be used to achieve a
526ligature effect.
527
528The next examples demonstrate how the \type {liga} feature deals with \type {ffi}.
529Possible solutions are: replace all three at once, replace the first two first and
530in a next step, combine a ligature and following character, replace one or more
531components by variants that have no interference with the dot of the~\quote{i}.
532
533\starttyping
534\showotfcomposition{dejavu-serif*default          at 48pt}{}{ffi}
535\showotfcomposition{cambria*default               at 48pt}{}{ffi}
536\showotfcomposition{lmroman10regular*default      at 48pt}{}{ffi}
537\showotfcomposition{texgyrepagellaregular*default at 48pt}{}{ffi}
538\stoptyping
539
540\blank \showotfcomposition{dejavu-serif*default          at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank
541\blank \showotfcomposition{cambria*default               at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank
542\blank \showotfcomposition{lmroman10regular*default      at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank
543\blank \showotfcomposition{texgyrepagellaregular*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank
544
545\stopsubsection
546
547\startsubsection[title={Single positioning}]
548
549Single positioning is also known as kerning, moving characters closer together so
550that we get a more uniformly spaced sequence of glyphs. It is a mistake to think
551that kerning is always needed! There are fonts that have hardly any kerns or no
552kerns at all and still look good.
553
554\start
555    \showfontkerns                                                                       \blank
556    \definedfont[dejavu-serif*default          at  8pt]Dejavu Serif:        \input tufte (E.R. Tufte)\blank
557    \definedfont[cambria*default               at  9pt]Cambria:             \input tufte (E.R. Tufte)\blank
558    \definedfont[lmroman10regular*default      at 10pt]Latin Roman Regular: \input tufte (E.R. Tufte)\blank
559    \definedfont[lucidabrightot*default        at  8pt]Lucida Bright:       \input tufte (E.R. Tufte)\blank
560    \definedfont[texgyrepagellaregular*default at  9pt]Pagella Regular:     \input tufte (E.R. Tufte)\blank
561\stop
562
563The next couple of examples show the action for a few words:
564
565\blank \showotfcomposition{dejavu-serif*default          at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank
566\blank \showotfcomposition{cambria*default               at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank
567\blank \showotfcomposition{lmroman10regular*default      at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank
568\blank \showotfcomposition{lucidabrightot*default        at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank
569\blank \showotfcomposition{texgyrepagellaregular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank
570
571\stopsubsection
572
573\startsubsection[title={Pairwise positioning}]
574
575This variant of positioning involved the first, second or both glyphs of a glyph
576pair. The specification can influence the horizontal and vertical positions we
577well as the widths of the positioned glyphs.
578
579\startnotabene
580    We need an example here.
581\stopnotabene
582
583\stopsubsection
584
585\startsubsection[title={Mark positioning}]
586
587Marks are (often) small symbols that represent accents (in latin) or vowels (in
588arabic) that get attached to base glyphs. In the input stream they come after the
589character that they apply to. Many fonts come with precomposed latin characters
590which means that an \type {à} in the input is mapped directly onto its
591corresponding shape. When the input contains an \type {a} followed by a \type{̀ }
592input normalization will normally turn this into an \type {à}. But, when this
593doesn't happen, the font machinery has to make sure that the mark gets positioned
594right onto the base character. In traditional \TYPEONE\ fonts that more or less
595happened automatically by overlaying the shapes. In \OPENTYPE\ (single)
596positioning is used to place the mark right.
597
598\startnarrowtyping
599\showotfcomposition{dejavu-serif*default          at 24pt}{}{ a\utfchar{"0300} à}
600\showotfcomposition{cambria*default               at 24pt}{}{ a\utfchar{"0300} à}
601\showotfcomposition{lmroman10regular*default      at 24pt}{}{ a\utfchar{"0300} à}
602\showotfcomposition{lucidabrightot*default        at 24pt}{}{ a\utfchar{"0300} à}
603\showotfcomposition{texgyrepagellaregular*default at 24pt}{}{ a\utfchar{"0300} à}
604\stopnarrowtyping
605
606Of course a font can contain logic that replaces a sequence of base and mark into
607precomposed characters with the right \UNICODE\ entry.
608
609\blank \showotfcomposition{dejavu-serif*default          at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ a\utfchar{"0300} à}} \blank
610\blank \showotfcomposition{cambria*default               at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ a\utfchar{"0300} à}} \blank
611\blank \showotfcomposition{lmroman10regular*default      at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ a\utfchar{"0300} à}} \blank
612\blank \showotfcomposition{lucidabrightot*default        at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ a\utfchar{"0300} à}} \blank
613\blank \showotfcomposition{texgyrepagellaregular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ a\utfchar{"0300} à}} \blank
614
615You can imagine that when marks are bound to characters that have become
616ligatures the anchoring is more complex as the font machinery has to keep track
617of onto which component the mark goes. For this purpose marks as well as base
618characters and base ligatures have anchors and feature lookups can explicitly
619refer to them.
620
621\stopsubsection
622
623\startsubsection[title={Contextual analysis}]
624
625What actually happens when turning a list of characters into a list of glyphs can
626range from real simple to pretty complex. For instance the \type {smcp} feature
627only has to run over the list and relate characters to a smallcaps shape. A
628slightly more complex feature might also demand some positioning. One step further
629is the use of contextual analysis, i.e. looking at previous, current and following
630characters (or glyphs). Because features can range from simple to complex the actual
631processing is not per feature! A font comes with a sequence of so called lookups that
632relate to a feature, script and language. Also, each feature can use one||to||one,
633multiple||to||one and many||to||many replacements as well as relative positioning.
634
635So, what actually happens is not that a feature is processed, but that all features
636are dealt with at the same time, in the order that the font prescribes. Enabling a
637specific feature means that a step is executed, while a disabled feature skips all
638steps that are tagged as belonging to that feature. And, as each feature can use
639contextual analysis, you can imagine that the effective sequence of actions can be
640a complex mix.
641
642A nice example of a contextual substitution is the centered period character in
643catalan in \type {ebgaramond}:
644
645\startbuffer
646\definefontfeature
647  [example]
648  [default]
649  [locl=yes,script=latn,language=cat]
650
651\definedfont[file:ebgaramond12-regular.otf*default at 40pt]l·l\quad
652\definedfont[file:ebgaramond12-regular.otf*example at 40pt]l·l
653\stopbuffer
654
655\typebuffer
656
657We show the boundingbox of the glyphs. The centered period between two l's is
658is replaced by a raised variant with no width.
659
660\blank \start \showglyphs \maincolor \midaligned{\getbuffer} \stop \blank
661
662It will be clear that in order to use such features you need to know what the font
663provides. For some fonts you need to explicitly enable the latin script (while others
664use default). Such a feature can be part of localized support but that is no rule.
665In that respect \OPENTYPE\ features are a rather unpredictable mess. For instance,
666nothing prevents such a feature to be a ligature, and in case you find that strange,
667especially ligature features are often abused for any purpose.
668
669\stopsubsection
670
671\startsubsection[reference=ligatures:hyphenation,title=Ligatures and hyphenation]
672
673In this section we will say a few words on how hyphenation interferes with
674(especially) ligature building. For this you need to know that:
675
676\starttyping
677effe
678\stoptyping
679
680But when hyphenation is permitted between the two \type{s}'s we actually have
681internally:
682
683\starttyping
684ef{-}{}{}fe
685\stoptyping
686
687The first snippet comes at the end of a line, the second at the beginning of a
688the next line and the last snippet is used when no hyphenation is needed. Such
689triplets need to be taken into account when we do replacements and positioning
690and also when we do contextual lookups.
691
692An \OPENTYPE\ font is just a container that collects the following:
693
694\startitemize[packed]
695\startitem
696    graphic representations of characters and symbols
697\stopitem
698\startitem
699    information about what characters the shapes represent
700\stopitem
701\startitem
702    rules about converting (sequences of) characters into one or more
703    representations
704\stopitem
705\startitem
706    rules about positioning representations relative to each other
707\stopitem
708\stopitemize
709
710Although the way this information is stored is standardized, the rules are not.
711You can imagine that there would be some standard way to turn an \type {f} and
712\type {i} into an \quote{fi} but we already saw that this is not the case. Here
713are some possibilities:
714
715\startitemize[packed]
716\startitem
717    The two characters get their own standard glyph, maybe with some kerning.
718\stopitem
719\startitem
720    The two characters are combined into one shape.
721\stopitem
722\startitem
723    The \type{f} gets a narrow representation and is kept close to the standard
724    \type{i}.
725\stopitem
726\startitem
727    A standard \type {f} is kerned with a dotless \type{i} (not to be confused
728    with the \UNICODE\ character).
729\stopitem
730\startitem
731    A special \type {f} is combined with a special \type {i}.
732\stopitem
733\stopitemize
734
735% maybe mark lig components when separate chars so that we can do spacing
736
737If the two characters are represented by their own shape, some contextual
738analysis takes place. Again there are several approaches to this:
739
740\startitemize[packed]
741\startitem
742    When an \type{f} is seen in the input, the next character is checked and one
743    or both gets replaced.
744\stopitem
745\startitem
746    When an \type{i} is seen in the input, the previous character is checked and
747    the \type {i} gets replaced.
748\stopitem
749\startitem
750    When an \type{f} several following characters are checked, for instance to
751    see if we need to take \type {ij} into account.
752\stopitem
753\stopitemize
754
755Traditionally the \type {f} followed by an \type{f}, \type{l} and \type{i} get a
756treatment, but some fonts also combine the \type {f} with \type {k}, \type {j},
757\type {b}, \type {t} and more.
758
759The \MKIV\ font handler is rather generic in the sense that it support what the
760font requires. However, a complication is that the scripts (languages) that use
761these diverse methods also expect hyphenation within such a ligature. Script like
762Arabic that are more demanding don't hyphenate so there interference with
763hyphenation is not a problem.
764
765Some ligatures are sensitive for languages. In languages that have compound words
766it might be undesirable to have a ligature at a word boundary, or in the Dutch
767word \type {fijn} we like to have a nice glyph (or combinations) for \type {ij}
768but no \type {fi} ligature. In a similar way hyphenation patterns can have
769rules and it will be no surprise that the hyphenation mechanism can compete with
770the ligature building for the best solution. This gets complicated by the fact
771that there is no real way to recognize in the font handler if we really are
772dealing with ligature building. Not only is the \type {liga} feature (and deep
773down the ligature gsub handling) not bound to ligatures (but simply a
774many|-|to|-|one mapper), some of the mentioned pseudo ligature builders use simple
775substitution and kerning and there is no way to recognize that as a ligature.
776
777Although it is possible to come up with a solution that is acceptable for many cases,
778there is no way to predict what kind of tricks font designers will use. A hyphenation
779point can be seen as follows:
780
781\starttabulate[||||]
782\NC \type{effe}      \NC \type{ef-fe}       \NC \type{e{f-}{f}{ff}e}       \NC \NR
783\NC \type{efficient} \NC \type{ef-fi-cient} \NC \type{e{ffi-}{}{ffi}cient} \NC \NR
784\stoptabulate
785
786In the second case the larger ligatures has replaced the previous one. We could
787have kept the first one because there are ways to manage two|-|step bounding
788ligatures but it's not worth the trouble (read: way more complex code and
789increased runtime for the whole mechanism). Here the \type {{ff}} and \type
790{{ffi}} can be individual shapes or just one shape.
791
792The three components of a hyphenation point: the pre, post and replacement text
793need to be looked at independently so that we get the proper kerning with the
794preceding and following characters. Also, in more complex (chained) lookups we
795need to compare each element with its surrounding. A fully expanded solution tree
796is too time consuming so we take some shortcuts and limits the checks to the
797level that it has no big impact on performance. The occasionally needed
798backtracking and inspection of components is currently quite reasonable. We need
799to trade quality with convenience: the result should look okay but processing
800speed should also be as high as possible. There is no need to let other scripts
801or regular fonts suffer too much from excessive script demands of fonts that
802could have be done better.
803
804The complication is that we not only need to check and replace but also need to
805check the kerning with preceding and following characters. We also need to take
806the hyphen into account (here one, but there can also be one after the break.
807
808It is for this reason that in \MKIV\ we have a (we think) acceptable mix of
809heuristics around hyphenation points that deal with single and multiple
810substitution as well as kerning. It will never be 100\% pertect but we consider
811it better to drop an occasional hyphenation in favor of proper font handling. In
812practice \TEX\ is clever enough to break a paragraph in lines within these
813restrictions.
814
815In \CONTEXT\ we have the traditional \TEX\ hyphenator but also provide an
816extensible \LUA\ reimplementation. That one might become the default in future
817versions. In traditional \TEX\ there are several low level hyphenation
818representations: simple hyphen only points, injected by the hyphenator,
819explicitly injected by the user or originating from a hyphen character. Then
820there is the generic (pre, post, replace) discretionary that can be explicitly
821injected by the user (or a macro). In \MKIV\ all hyphenation points get
822normalized to this generic discretionary. There is no need for old|-|time
823optimizations and a consistent (expanded) representation is easier to deal with
824in other extensions. However, because the font handler is supposed to also work
825outside \CONTEXT\ we need to deal with traditional cases too. But \unknown\ the
826results might differ a bit.
827
828\stopsubsection
829
830    \startsubsection[title=Color]
831
832    % TODO: use emojionecolor-svginot-archived.ttf
833
834    A recent new (and evolving) addition to \OPENTYPE\ is colored glyphs. One variant
835    (by \MICROSOFT) uses overlays and this method is quite efficient.
836
837    \startbuffer
838    \definefontfeature[colored][colr=yes]
839    \definefontsynonym[Emoji][file:seguiemj.ttf*default,colored]
840
841    \definesymbol[bug][\getglyphdirect{Emoji}{\char"1F41B}]
842    \definesymbol[ant][\getglyphdirect{Emoji}{\char"1F41C}]
843    \definesymbol[bee][\getglyphdirect{Emoji}{\char"1F41D}]
844    \stopbuffer
845
846    \typebuffer \getbuffer
847
848    Here we see a \symbol[bug], \symbol[ant] and \symbol[bee], and they come in
849    color! Once \UNICODE\ started adding such symbols (and more get added) the
850    distinction between characters and symbols get even fuzzier. Of course one
851    can argue that we communicate in pictograms but even then, given that
852    mankind lasts a while, the \UNICODE\ repertoire will explode.
853
854    \startplacefigure[title={A few emojis from \type {seguiemj.ttf}}]
855        \startcombination [3*1]
856            {\scale[width=.3\textwidth]{\symbol[bug]}} {\type{U+1F41B}: bug}
857            {\scale[width=.3\textwidth]{\symbol[ant]}} {\type{U+1F41C}: ant}
858            {\scale[width=.3\textwidth]{\symbol[bee]}} {\type{U+1F41D}: bee}
859        \stopcombination
860    \stopplacefigure
861
862    Here we use \type {seguiemj.ttf}, a font that comes with \MSWINDOWS. Colors are
863    achieved by combining glyphs rendered in different colors. A variant that uses
864    \SVG\ instead of overlays is \type {emojionecolor-svginot.ttf}:
865
866    \startbuffer
867    \definefontfeature[svg][svg=yes]
868    \definefontsynonym[Emoji][file:emojionecolor-svginot.ttf*default,svg]
869    \stopbuffer
870
871    \typebuffer \getbuffer
872
873    This time we get \symbol[bug], \symbol[ant] and \symbol[bee] and they look
874    quite different. Both fonts also have ligatures and you can wonder what sense
875    that makes. It makes it impossible to swap fonts and as there is no standard
876    one never knows what to expect.
877
878    \startplacefigure[title={A few emojis from \type {emojionecolor-svginot.ttf}}]
879        \startcombination [3*1]
880            {\scale[width=.3\textwidth]{\symbol[bug]}} {\type{U+1F41B}: bug}
881            {\scale[width=.3\textwidth]{\symbol[ant]}} {\type{U+1F41C}: ant}
882            {\scale[width=.3\textwidth]{\symbol[bee]}} {\type{U+1F41D}: bee}
883        \stopcombination
884    \stopplacefigure
885
886    \definefont[Emoji][file:emojionecolor-svginot.ttf*default,svg]
887
888    \def\FourFaces{\char128104\zwj\char128105\zwj\char128102\zwj\char128102\relax}
889
890    \def\Man  {\char"1F468\relax}
891    \def\Woman{\char"1F469\relax}
892    \def\Boy  {\char"1F466\relax}
893    \def\Girl {\char"1F467\relax}
894
895    How do we know what faces add up to the ligature {\Emoji\Man \zwj \Woman \zwj
896    \Girl \zwj \Boy} and how are we supposed to know that there should {\darkgray
897    \type {zwj}} in between? When we input four faces separated by zero width
898    joiners, we get a four face symbol instead. The reason for having the joiners in
899    between is probably to avoid unexpected ligatures. The sequence \type {man},
900    \type {woman}, \type {boy}, \type {boy} gives \type {family}:
901    %
902    {\Emoji\Man}   + {\darkgray \type {zwj}}
903    {\Emoji\Woman} + {\darkgray \type {zwj}}
904    {\Emoji\Boy}   + {\darkgray \type {zwj}}
905    {\Emoji\Boy}   = {\Emoji\Man \zwj \Woman \zwj \Boy \zwj \Boy},
906    %
907    but two girls also work:
908    %
909    {\Emoji\Man}   + {\darkgray \type {zwj}}
910    {\Emoji\Woman} + {\darkgray \type {zwj}}
911    {\Emoji\Girl}  + {\darkgray \type {zwj}}
912    {\Emoji\Girl}  = {\Emoji\Man \zwj \Woman \zwj \Girl \zwj \Girl},
913    %
914    so does a mixture of kids:
915    %
916    {\Emoji\Man}   + {\darkgray \type {zwj}}
917    {\Emoji\Woman} + {\darkgray \type {zwj}}
918    {\Emoji\Girl}  + {\darkgray \type {zwj}}
919    {\Emoji\Boy}   = {\Emoji\Man \zwj \Woman \zwj \Girl \zwj \Boy},
920    %
921    although (at least currently):
922    %
923    {\Emoji\Man}   + {\darkgray \type {zwj}}
924    {\Emoji\Woman} + {\darkgray \type {zwj}}
925    {\Emoji\Boy}   + {\darkgray \type {zwj}}
926    {\Emoji\Girl}  = {\Emoji\Man \zwj \Woman \zwj \Boy \zwj \Girl},
927    %
928    gives twin boys. Of course the real family emoj is {\Emoji\char"1F46A}.
929
930    In our times for sure many combinations are possible, so:
931    %
932    {\Emoji\Man}  + {\darkgray \type {zwj}}
933    {\Emoji\Man}  + {\darkgray \type {zwj}}
934    {\Emoji\Girl} + {\darkgray \type {zwj}}
935    {\Emoji\Girl} = {\Emoji\Man \zwj \Man \zwj \Girl \zwj \Girl},
936    %
937    indeed gives a family, but I wonder at what point cultural bias will creep into
938    font design. One can even wonder how clothing and haircut will demand frequent
939    font updates: {\Emoji\char"1F46B}, {\Emoji\char"1F46C}, {\Emoji\char"1F46D}.
940
941    In the math alphabets we have a couple of annoying holes because some characters
942    were already present in \UNICODE. The bad thing here is that we now always have
943    to deal with these exceptions. But not so with emojis because here eventually all
944    variants will show up. Where a character \type {A} in red or blue uses the same
945    code point, a white telephone {\Emoji\char"1F57E} and black telephone
946    {\Emoji\char"1F57F} have their own. And because obsolete scripts are already
947    supported in \UNICODE\ and more get added, we can expect old artifacts also
948    showing up at some time. Soon the joystick {\Emoji\char"1F579} will be an unknown
949    item to most of us, while the \MICROSOFT\ hololens migth get its slot.
950
951    \startplacefigure[title={Will all animals come in stages of development?}]
952        \startcombination [3*1]
953            {\scale[width=.3\textwidth]{\Emoji\char"1F423}} {\type{U+1F423}: hatching chick}
954            {\scale[width=.3\textwidth]{\Emoji\char"1F424}} {\type{U+1F424}: baby chick}
955            {\scale[width=.3\textwidth]{\Emoji\char"1F425}} {\type{U+1F425}: front-facing baby chick}
956        \stopcombination
957    \stopplacefigure
958
959    For sure these mechanisms will evolve and to what extent we support them depends
960    on what users want. At least we have the basics implemented.
961
962    \stopsubsection
963
964    \stopsection
965
966\startsection[title=Extras]
967
968\startnotabene
969    Todo.
970\stopnotabene
971
972\stopsection
973
974\startsection[reference=goodies,title=Goodies]
975
976Goodies range from simple to complex. They share that they are defined in files
977and loaded at runtime. There is a good change that when you read this, that there
978are already more goodies than mentioned here. Here we will just mention a couple
979of goodies. More details can be found in the files that ship with \CONTEXT\ and
980have suffix \type {lfg}.
981
982A goodie file is a regular \LUA\ file and is supposed to return a table. This
983table collects data that is used for implementing the goodie or relates to a
984regular feature. It can also provide information that is used for patching a
985font. An example of a simple goodie file is the ones that accompanies the first
986release of the \OPENTYPE\ Lucida fonts.
987
988\starttyping
989return {
990  name = "lucida-opentype-math",
991  version = "1.00",
992  comment = "Goodies that complement lucida opentype.",
993  author = "Hans Hagen",
994  copyright = "ConTeXt development team",
995  mathematics = {
996    alternates = {
997      italic = {
998        feature = 'ss01',
999        value = 1,
1000        comment = "Mathematical Alternative Italic"
1001      },
1002    }
1003  }
1004}
1005\stoptyping
1006
1007This goodie file is only providing information about the meaning of a stylistic
1008alternate. These have abstract tags like \type {ss01} and in this case this
1009category collects alternative italic (calligraphic) shapes. Because math does
1010not follow the same rules as text, this feature is enabled explicitly.
1011
1012In the goodie file of Xits math the alternates table has more entries:
1013
1014\startnarrowtyping
1015alternates = {
1016  cal       = { ... comment = "Mathematical Calligraphic Alphabet" },
1017  greekssup = { ... comment = "Mathematical Greek Sans Serif Alphabet" },
1018  greekssit = { ... comment = "Mathematical Italic Sans Serif Digits" },
1019  monobfnum = { ... comment = "Mathematical Bold Monospace Digits" },
1020  mathbbbf  = { ... comment = "Mathematical Bold Double-Struck Alphabet" },
1021  mathbbit  = { ... comment = "Mathematical Italic Double-Struck Alphabet" },
1022  mathbbbi  = { ... comment = "Mathematical Bold Italic Double-Struck Alphabet" },
1023  upint     = { ... comment = "Upright Integrals" },
1024  vertnot   = { ... comment = "Negated Symbols With Vertical Stroke" },
1025}
1026\stopnarrowtyping
1027
1028An alternate is triggered at the \TEX\ end with:
1029
1030\starttyping
1031$ABC$ $\cal ABC$ $\mathalternate{cal}\cal ABC$
1032\stoptyping
1033
1034This is an example of a dynamic feature that gets applied when enabled at a
1035specific location in the input. The \type {cal} is only recognized when it
1036is defined in a goodies file, where the value is defined (in all of the above cases
1037the value is~\type {1}).
1038
1039The Xits math fonts has a goodie files that starts with:
1040
1041\starttyping
1042return {
1043  name = "xits-math",
1044  version = "1.00",
1045  comment = "Goodies that complement xits (by Khaled Hosny).",
1046  author = "Hans Hagen",
1047  copyright = "ConTeXt development team",
1048  mathematics = {
1049    italics = {
1050      ["xits-math"] = italics,
1051    },
1052    alternates = {
1053\stoptyping
1054
1055Here the \type {italics} variable is a table defined before the \type {return}
1056that looks as follows:
1057
1058\starttyping
1059local italics = {
1060  defaultfactor = 0.025,
1061  disableengine = true,
1062  corrections   = {
1063   -- [0x1D44E] = 0.99,    -- a (fraction of quad)
1064   -- [0x1D44F] = 100,     -- b (font points)
1065      [0x1D453] = -0.0375, -- f
1066  }
1067}
1068\stoptyping
1069
1070This rather specific table tells \CONTEXT\ that (when enabled) it has to apply
1071italic correction. It disables support built into the \TEX\ engine (which in the
1072case of \LUATEX\ is close to absent anyway). It will apply a default italic
1073correction of \type {0.025} but for some shapes a different value is used. Again
1074we have some commands at the \TEX\ end:
1075
1076\starttyping
1077\setupmathematics[italics=1] % apply italics with we have an italic font
1078\setupmathematics[italics=2] % apply italics anyway
1079\setupmathematics[italics=3] % apply italics only when italic or bolditalic shapes
1080\setupmathematics[italics=4] % combination of 1 and 3
1081\stoptyping
1082
1083An alternative is this:
1084
1085\starttyping
1086\definefontfeature[mathextra][mathextra][collapseitalics=yes]
1087\stoptyping
1088
1089This extends the \type {mathextra} feature to move the italic correction into
1090the character's width. Often this works out ok.
1091
1092Because (definitely at the start of the \LUATEX\ project) we had no
1093proper \OPENTYPE\ math fonts, but at the same time wanted to move on
1094to \OPENTYPE\ and \UNICODE\ math and no longer struggle with all
1095those math families and definitions. The way out of this problem
1096is to define a virtual math font. The code for doing this is built
1097into the \MKIV\ core but is controlled by a goodie definition. Take
1098for instance Antykwa Math:
1099
1100\startnarrowtyping
1101return {
1102  name = "antykwa-math",
1103  version = "1.00",
1104  comment = "Goodies that complement antykwa math.",
1105  author = "Hans, Mojca, Aditya",
1106  copyright = "ConTeXt development team",
1107  mathematics = {
1108    mapfiles = {
1109      "antt-rm.map",
1110      "antt-mi.map",
1111      "antt-sy.map",
1112      "antt-ex.map",
1113      "mkiv-base.map",
1114    },
1115    virtuals = {
1116      ["antykwa-math"] = {
1117        { name = "file:AntykwaTorunska-Regular", features = "virtualmath", main = true },
1118        { name = "mi-anttri.tfm", vector = "tex-mi", skewchar=0x7F },
1119        { name = "mi-anttri.tfm", vector = "tex-it", skewchar=0x7F },
1120        { name = "sy-anttrz.tfm", vector = "tex-sy", skewchar=0x30, parameters = true } ,
1121        { name = "ex-anttr.tfm", vector = "tex-ex", extension = true } ,
1122        { name = "msam10.tfm", vector = "tex-ma" },
1123        { name = "msbm10.tfm", vector = "tex-mb" },
1124      },
1125\stopnarrowtyping
1126
1127Normally users will not define such tables but the keys give an indication of
1128what is involved. The same is true for the previously shown tables: they are just
1129provided in the \CONTEXT\ distribution.
1130
1131Text fonts also can have goodies. We start with a rather dumb one and there
1132will be not that many of those. This one is needed to turn a \TYPEONE\ font
1133with a rather special encoding into a \UNICODE\ font. The next mapping is
1134possible because the dingbats are part of \UNICODE.
1135
1136\starttyping
1137return {
1138  name = "dingbats",
1139  version = "1.00",
1140  comment = "Goodies that complement dingbats (funny names).",
1141  author = "Hans Hagen",
1142  copyright = "ConTeXt development team",
1143  remapping = {
1144    tounicode = true,
1145    unicodes = {
1146      a1   = 0x2701,
1147      a10  = 0x2721,
1148      a100 = 0x275E,
1149      a101 = 0x2761,
1150      a102 = 0x2762,
1151\stoptyping
1152
1153Applying this encoding happens in two steps. Because goodies like this are just
1154features, we need to define a proper font feature set:
1155
1156\starttyping
1157\definefontfeature
1158  [dingbats]
1159  [mode=base,
1160   goodies=dingbats,
1161   unicoding=yes]
1162\stoptyping
1163
1164We have a base mode font, so no special processing takes place. The \type {goodies}
1165key is used to communicate the goodies file. The \type {unicoding} key is used
1166to apply the encoding. Of course this only works because the remapper code is present
1167in the core and is hooked in to the font initialization code. The \type {dingbats}
1168feature set is predefined, just as the font definition:
1169
1170\starttyping
1171\definefontsynonym [ZapfDingbats] [file:uzdr] [features=dingbats]
1172\stoptyping
1173
1174Here is a goodie file that I made a while ago:
1175
1176\starttyping
1177return {
1178  name = "oxoniensis",
1179  version = "1.00",
1180  comment = "Oxoniensis test file for Thomas Schmitz.",
1181  author = "Hans Hagen",
1182  copyright = "ConTeXt development team",
1183  features = {
1184    lunatesigma = {
1185      type = "substitution",
1186      data = {
1187        sigma  = 0x03F2,
1188        sigma1 = 0x03F2,
1189        Sigma  = 0x03F9,
1190        phi    = phi1,
1191      },
1192    }
1193  },
1194}
1195\stoptyping
1196
1197There is not that much to say about this, apart from that it's a sort of fake
1198feature that gets enabled as regular one:
1199
1200\starttyping
1201\definefontfeature[test]
1202  [mode=node,
1203   kern=yes,
1204   lunatesigma=yes,
1205   goodies=oxoniensis]
1206
1207\definefont[somefont][file:oxoniensis*test]
1208\stoptyping
1209
1210A complete different kind of goodie is the following. At one of the \CONTEXT\ meetings
1211Mojca Miklavec discussed the possibility to have an additional mechanism for
1212defining combinations of fonts. Often fonts come in a set of four (regular, italic,
1213bold and bold italic). In \MKII\ the complexity of typescripts depends on the amount of
1214encodings that need to be supported but in \MKIV\ things are easier. For a set of four fonts
1215a typescript looks as follows:
1216
1217\starttyping
1218\starttypescript [sans] [somesansfont] [name]
1219  \setups[font:fallback:sans]
1220  \definefontsynonym [Sans]           [file:somesans]  [features=default]
1221  \definefontsynonym [SansBold]       [file:somesansb] [features=default]
1222  \definefontsynonym [SansItalic]     [file:somesansi] [features=default]
1223  \definefontsynonym [SansBoldItalic] [file:somesansz] [features=default]
1224\stoptypescript
1225\stoptyping
1226
1227We still have the abstract notion of a \type {Sans} font so that we can refer to
1228the regular shape without knowing the real name but the number of lines needed
1229is small. Such a definition can then be referred to using:
1230
1231\starttyping
1232\starttypescript[somefontset]
1233  \definetypeface [somefontset] [rm] [serif] [someserif] [default]
1234  \definetypeface [somefontset] [ss] [sans]  [somesans]  [default]
1235  \definetypeface [somefontset] [tt] [mono]  [somemono]  [default]
1236  \definetypeface [somefontset] [mm] [math]  [somemath]  [default]
1237\stoptypescript
1238\stoptyping
1239
1240So far things look simple. Given that many fonts follow a similar naming scheme
1241Wolfgang made a module that avoids such definitions altogether. However, being
1242involved in the development of the Antykwa fonts, Mojca ran into the situation
1243that not just four fonts were part of the set but many more. There are several
1244weight (think of light and heavy variants) as well as condensed variants and of
1245course the whole set is not per se a multiple of four.
1246
1247In the meantime, in addition to the \type {file:} and \type {name:} accessors,
1248\CONTEXT\ had an additional one tagged \type {spec:} where a string made out of
1249weight, style, width etc.\ is turned into a (best guessed) font name. Therefore
1250the most natural way to deal with the many|-|fonts|-|in|-|a|-|set dilemma was to
1251provide an additional interface between this specification and the font set and
1252the most robust method was to define all in a goodie file.
1253
1254In this case the goodies are loaded independent of the font, that is: not
1255as a feature. For instance:
1256
1257\starttyping
1258\loadfontgoodies[antykwapoltawskiego]
1259\stoptyping
1260
1261This file maps obscure fontnames onto the \type {spec:} interface so that
1262we can access them in a robust way.
1263
1264\starttyping
1265\definefont
1266  [MyFontA]
1267  [file:Iwona-Regular*smallcaps]
1268\definefont
1269  [MyFontB]
1270  [file:AntykwaTorunska-Regular*smallcaps]
1271\definefont
1272  [MyFontC]
1273  [file:antpoltltcond-regular*smallcaps]
1274\definefont
1275  [MyFontD]
1276  [spec:antykwapoltawskiego-bold-italic-condensed-normal*smallcaps]
1277\definefont
1278  [MyFontE]
1279  [spec:antykwapoltawskiego-bold-italic-normal]
1280\stoptyping
1281
1282The goodies file looks as follows:
1283
1284\starttyping
1285return {
1286  name = "antykwa-poltawskiego",
1287  version = "1.00",
1288  comment = "Goodies that complement Antykwa Poltawskiego",
1289  author = "Hans & Mojca",
1290  copyright = "ConTeXt development team",
1291  files = {
1292    name = "antykwapoltawskiego", -- shared
1293    list = {
1294      ["AntPoltLtCond-Regular.otf"] = {
1295        weight = "light",
1296        style  = "regular",
1297        width  = "condensed",
1298      },
1299      ...
1300      ["AntPoltExpd-BoldItalic.otf"] = {
1301        weight = "bold",
1302        style  = "italic",
1303        width  = "expanded",
1304      },
1305    },
1306  },
1307  typefaces = {
1308    ["antykwapoltawskiego-light"] = {
1309      shortcut     = "rm",
1310      shape        = "serif",
1311      fontname     = "antykwapoltawskiego",
1312      normalweight = "light",
1313      boldweight   = "medium",
1314      width        = "normal",
1315      size         = "default",
1316      features     = "default",
1317    },
1318    ...
1319  },
1320}
1321\stoptyping
1322
1323In addition to the files|-|to|-|specification mapping, there is
1324also a typeface specification table. This permits the definition
1325of a typeface in the following way:
1326
1327\starttyping
1328\definetypeface
1329  [name=mojcasfavourite,
1330   preset=antykwapoltawskiego,
1331   normalweight=light,
1332   boldweight=bold,
1333   width=expanded]
1334
1335\setupbodyfont
1336  [mojcasfavourite]
1337\stoptyping
1338
1339When resolving the definition, the best possible match will be taken from the
1340typeface table in the goodie file. Of course this is not something that we expect
1341the average user to deliver and deal with.
1342
1343As the Antykwa font is somewhat atypical and not used in everyday typesetting,
1344you might wonder if all this overhead makes sense. However, there are type
1345foundries that do ship their fonts in many weights and for those using a \LUA\
1346goodie file instead of many typescripts in \TEX\ coding makes sense. Take for
1347instance TheMix:
1348
1349\starttyping
1350\loadfontgoodies
1351  [themix]
1352
1353\definetypeface
1354  [name=themix,
1355   preset=themix-light]
1356
1357\definetypeface
1358  [name=themix,
1359   preset=themixmono-light]
1360
1361\setupbodyfont
1362  [themix]
1363\stoptyping
1364
1365In this case the goodie file can serve as a template for more such fonts.
1366In order to be efficient this goodie file uses a couple of local
1367tables (we could have used metatables instead).
1368
1369\starttyping
1370local themix = {
1371  name     = "themix",
1372  shortcut = "ss",
1373  shape    = "sans",
1374  fontname = "themix",
1375  width    = "normal",
1376  size     = "default",
1377  features = "default",
1378}
1379
1380local themixmono = {
1381  name     = "themixmono",
1382  shortcut = "tt",
1383  shape    = "mono",
1384  fontname = "themixmono",
1385  width    = "normal",
1386  size     = "default",
1387  features = "default",
1388}
1389\stoptyping
1390
1391The main goodie table defines a lot of weights:
1392
1393\startnarrowtyping
1394return {
1395  name = "themix",
1396  version = "1.00",
1397  comment = "Goodies that complement TheMix (by and for sale at www.lucasfonts.com).",
1398  author = "Hans Hagen",
1399  copyright = "ConTeXt development team",
1400  files = {
1401    list = {
1402      ["TheMixOsF-ExtraLight"] = {
1403        name   = "themix",
1404        weight = "extralight",
1405        style  = "regular",
1406        width  = "normal"
1407      },
1408      ["TheMixOsF-ExtraLightItalic"] = {
1409        ...
1410      },
1411      ...
1412      ["TheMixOsF-Black"] = {
1413        ...
1414      },
1415      ["TheMixOsF-BlackItalic"] = {
1416        ...
1417      },
1418      ...
1419      --
1420      ["TheMixMono-W2ExtraLight"] = {
1421        name   = "themixmono",
1422        weight = "extralight",
1423        style  = "regular",
1424        width  = "normal"
1425      },
1426      ...
1427      ["TheMixMono-W9BlackItalic"] = {
1428        ...
1429      },
1430    },
1431  },
1432  typefaces = {
1433    ["themix-extralight"] = table.merged(themix, {
1434      normalweight = "extralight",
1435      boldweight   = "semilight"
1436    }),
1437    ["themix-light"] = table.merged(themix, {
1438      normalweight = "light",
1439      boldweight   = "normal"
1440    }),
1441    ...
1442    ["themixmono-bold"] = table.merged(themixmono, {
1443      normalweight = "bold",
1444      boldweight   = "black"
1445    }),
1446  },
1447}
1448\stopnarrowtyping
1449
1450It's now time for some generic goodies. In the \CONTEXT\ distribution there
1451is a goodie file that (at the time of this writing) looks as follows:
1452
1453\starttyping
1454local default = {
1455  analyze  = "yes",
1456  mode     = "node",
1457  language = "dflt",
1458  script   = "dflt",
1459}
1460
1461local smallcaps = {
1462  smcp = "yes",
1463}
1464
1465local function statistics(tfmdata)
1466  commands.showfontparameters(tfmdata)
1467end
1468
1469local function squeeze(tfmdata)
1470  for k, v in next, tfmdata.characters do
1471    v.height = 0.75 * (v.height or 0)
1472    v.depth  = 0.75 * (v.depth  or 0)
1473  end
1474end
1475
1476return {
1477  name = "demo",
1478  version = "1.01",
1479  comment = "An example of goodies.",
1480  author = "Hans Hagen",
1481  featuresets = {
1482    default = {
1483      default,
1484    },
1485    smallcaps = {
1486      default, smallcaps,
1487    },
1488  },
1489  colorschemes = {
1490    default = {
1491      [1] = {
1492        "one", "three", "five", "seven", "nine",
1493      },
1494      [2] = {
1495        "two", "four", "six", "eight", "ten",
1496      },
1497    },
1498    all = {
1499      [1] = {
1500        "*",
1501      },
1502    },
1503    some = {
1504      [1] = {
1505        "0x0030:0x0035",
1506      },
1507    },
1508  },
1509  postprocessors = {
1510    statistics = statistics,
1511    squeeze    = squeeze,
1512  },
1513}
1514\stoptyping
1515
1516This demo file implements several goodies: featuresets, colors and
1517postprocessors. Keep in mind that a goodie file can provide whatever information
1518it wants but of course only known subtables will be dealt with.
1519
1520The coloring of glyphs can happen by name, which assumes that glyph names are
1521used, or by number. Here we use generic glyph names, but for a specific font one
1522might need to provide a special goodie file. For instance, the color section of
1523the goodie file for the husayni font has entries like:
1524
1525\startnarrowtyping
1526[3] = {
1527  "Ttaa.waqf", "SsLY.waqf", "QLY.waqf", "Miim.waqf", "LA.waqf", "Jiim.waqf",
1528  "Threedotsabove.waqf", "Siin.waqf", "Ssaad.waqf", "Qaaf.waqf", "SsL.waqf",
1529  "QF.waqf", "SKTH.waqf", "WQFH.waqf", "Kaaf.waqf", "Ayn.ruku", "Miim.nuun_high",
1530  "Siin.Ssaad", "Nuunsmall", "emptydot_low", "emptydot_high", "Sifr.fill",
1531  "Miim.nuun_low", "Nuun.tanwiin",
1532},
1533\stopnarrowtyping
1534
1535Of course such a definition can only be made when the internals of the font are
1536known and in this case it concerns a pretty complex font.
1537
1538\startbuffer
1539\definefontfeature
1540  [demo-colored]
1541  [goodies=demo,
1542   colorscheme=default,
1543   featureset=default]
1544
1545\definefontfeature
1546  [demo-colored-all]
1547  [goodies=demo,
1548   colorscheme=all,
1549   featureset=default]
1550
1551\definefontfeature
1552  [demo-colored-some]
1553  [goodies=demo,
1554   colorscheme=some,
1555   featureset=default]
1556
1557\definefont[DemoFontA][MonoBold*demo-colored      at 10pt]
1558\definefont[DemoFontB][MonoBold*demo-colored-all  at 10pt]
1559\definefont[DemoFontC][MonoBold*demo-colored-some at 10pt]
1560\stopbuffer
1561
1562\typebuffer \getbuffer
1563
1564% \definecolor[colorscheme:1:1][s=.75]
1565% \definecolor[colorscheme:1:2][r=.75]
1566% \definecolor[colorscheme:1:3][g=.75]
1567% \definecolor[colorscheme:1:4][b=.75]
1568% \definecolor[colorscheme:1:5][c=.75]
1569% \definecolor[colorscheme:1:6][m=.75]
1570% \definecolor[colorscheme:1:7][y=.75]
1571
1572% \definecolor[colorscheme:2:7][s=.75]
1573% \definecolor[colorscheme:2:6][r=.75]
1574% \definecolor[colorscheme:2:5][g=.75]
1575% \definecolor[colorscheme:2:4][b=.75]
1576% \definecolor[colorscheme:2:3][c=.75]
1577% \definecolor[colorscheme:2:2][m=.75]
1578% \definecolor[colorscheme:2:1][y=.75]
1579
1580\startbuffer
1581\starttabulate[||||]
1582\NC
1583    \DemoFontA \resetfontcolorscheme   test 1234567890 \NC
1584    \DemoFontA \setfontcolorscheme  [1]test 1234567890 \NC
1585    \DemoFontA \setfontcolorscheme  [2]test 1234567890 \NC
1586\NR
1587\NC
1588    \DemoFontB \resetfontcolorscheme   test 1234567890 \NC
1589    \DemoFontB \setfontcolorscheme  [1]test 1234567890 \NC
1590    \DemoFontB \setfontcolorscheme  [2]test 1234567890 \NC
1591\NR
1592\NC
1593    \DemoFontC \resetfontcolorscheme   test 1234567890 \NC
1594    \DemoFontC \setfontcolorscheme  [1]test 1234567890 \NC
1595    \DemoFontC \setfontcolorscheme  [2]test 1234567890 \NC
1596\NR
1597\stoptabulate
1598\stopbuffer
1599
1600\typebuffer \getbuffer
1601
1602Here is an example that I made at the TUG 2013 conference in Japan,
1603after a presentation by Chof. The font (adapted by by Dohyun Kim) can
1604be downloaded from: \hyphenatedurl {http://ftp.ktug.org/KTUG/hcr-lvt/1.910_nomac/}.
1605
1606\startbuffer[korean-demo]
1607\definefontfeature
1608  [korean-composed]
1609  [goodies=hanbatanglvt,
1610   colorscheme=default,
1611   mode=node,
1612   ljmo=yes,
1613   tjmo=yes,
1614   vjmo=yes,
1615   script=hang,
1616   language=kor]
1617
1618\definefont
1619  [KoreanJMO]
1620  [hanbatanglvt*korean-composed]
1621
1622\definecolor[colorscheme:100:1][r=.75]
1623\definecolor[colorscheme:100:2][g=.75]
1624\definecolor[colorscheme:100:3][b=.75]
1625
1626\definecolor[colorscheme:101:1][g=.75,b=.75]
1627\definecolor[colorscheme:101:2][r=.75,b=.75]
1628\definecolor[colorscheme:101:3][r=.75,g=.75]
1629\stopbuffer
1630
1631\typebuffer[korean-demo] \getbuffer[korean-demo]
1632
1633\startbuffer
1634    % Hunminjeongeum: http://en.wikipedia.org/wiki/Hunminjeongeum
1635    나랏말ᄊᆞ미中듕國귁에달아문ᄍᆞᆼ와로서르ᄉᆞᄆᆞᆺ디아니ᄒᆞᆯᄊᆡ%
1636    사ᄅᆞᆷ마다ᄒᆡᅇᅧ수ᄫᅵ니겨나...% ᆯ로ᄡᅮ메便뼌安ᅙᅡᆫ킈ᄒᆞ고져ᄒᆞᇙᄯᆞᄅᆞ미니라
1637\stopbuffer
1638
1639\startlinecorrection
1640\startcombination[1*3]
1641   {\framed{\startscript[hangul]\KoreanJMO                        \getbuffer\stopscript}} {no colorscheme}
1642   {\framed{\startscript[hangul]\KoreanJMO\setfontcolorscheme[100]\getbuffer\stopscript}} {colorscheme 100}
1643   {\framed{\startscript[hangul]\KoreanJMO\setfontcolorscheme[101]\getbuffer\stopscript}} {colorscheme 101}
1644\stopcombination
1645\stoplinecorrection
1646
1647The goodie definition looks as follows (watch how we use ranges):
1648
1649\starttyping
1650return {
1651    name = "hanbatanglvt",
1652    version = "1.00",
1653    comment = "Goodies that complement the hanbatanglvt fonts.",
1654    author = "Hans Hagen",
1655    colorschemes = {
1656        default = {
1657            { "0x01100:0x0115F" }, -- jamo_initial (r/c)
1658            { "0x01160:0x011A7" }, -- jamo_medial  (g/m)
1659            { "0x011A8:0x011FF" }, -- jamo_final   (b/y)
1660        }
1661    }
1662}
1663\stoptyping
1664
1665This is much shorter (and efficent) that defining a whole vector, as in:
1666
1667\starttyping
1668local f_uni_base = string.formatters["uni%04X"]
1669local f_uni_plus = string.formatters["uni%04X.y%s"]
1670
1671local function range(first,last)
1672    local t = { }
1673    for i=first,last do
1674        t[#t+1] = f_uni_base(i)
1675        for j=0,19 do
1676            t[#t+1] = f_uni_plus(i,j)
1677        end
1678    end
1679    return t
1680end
1681
1682return {
1683    name = "hanbatanglvt",
1684    version = "1.00",
1685    comment = "Goodies that complement the hanbatanglvt fonts.",
1686    author = "Hans Hagen",
1687    colorschemes = {
1688        default = {
1689            range(0x01100,0x0115F), -- jamo_initial (r/c)
1690            range(0x01160,0x011A7), -- jamo_medial  (g/m)
1691            range(0x011A8,0x011FF), -- jamo_final   (b/y)
1692        }
1693    }
1694}
1695\stoptyping
1696
1697By using names we don't depend on \UNICODE\ which sometimes is needed when glyphs
1698have ended up in the private space. However, by default, after glyphs have been
1699mapped to colors, an extra pass will make sure that characters pushed into
1700private space will get the same mapping as their regular \UNICODE\ has gotten
1701(given that the number is known). Of course explicitly assigned colors will be
1702preserved.
1703
1704Another generic demo feature is postprocessing. In principle one can
1705add additional postprocessors but for that the source code needs to
1706be consulted which in turn assumes some knowledge of fonts and \CONTEXT\
1707internals.
1708
1709\startbuffer
1710\definefontfeature
1711  [justademoa]
1712  [default]
1713  [goodies=demo,
1714   postprocessor=squeeze]
1715
1716\definefontfeature
1717  [justademob]
1718  [default]
1719  [goodies=demo,
1720   postprocessor=statistics]
1721
1722\definefontfeature
1723  [justademoc]
1724  [default]
1725  [goodies=demo,
1726   postprocessor={statistics,squeeze}]
1727\stopbuffer
1728
1729\typebuffer \getbuffer
1730
1731The statistics just print some font parameters to the log so that one
1732is not showing up here. The squeeze looks as follows:
1733
1734\startbuffer
1735\definefont[DemoFontD][Serif*default    at 30pt]
1736\definefont[DemoFontE][Serif*justademoa at 30pt]
1737\stopbuffer
1738
1739\typebuffer \getbuffer
1740
1741\startlinecorrection
1742\hbox\bgroup
1743    \ruledhbox{\color[maincolor]{DemoFontD height \& depth}}\quad
1744    \ruledhbox{\color[maincolor]{DemoFontE height \& depth}}
1745\egroup
1746\stoplinecorrection
1747
1748The squeezer just makes the height and depth of glyphs a  bit smaller and it is
1749not that hard to imagine other manipulations. The demo goodie file is good
1750place to start playing with such things.
1751
1752Because there is less standardization with respect to features than one might
1753suspect, goodie files provide a mean to define featuresets. We can use such a set
1754in another definition:
1755
1756\starttyping
1757\definefontfeature
1758  [demo-smallcaps]
1759  [goodies=demo,
1760   featureset=smallcaps]
1761\stoptyping
1762
1763Of course this only makes sense for more complex combinations. The already mentioned
1764husayni font comes with many features and most of these work together.
1765
1766The basic goodie table looks as follows:
1767
1768\startnarrowtyping
1769return {
1770  name         = "husayni",
1771  version      = "1.00",
1772  comment      = "Goodies that complement the Husayni font by Idris Samawi Hamid.",
1773  author       = "Idris Samawi Hamid and Hans Hagen",
1774  featuresets  = { },
1775  solutions    = { },
1776  stylistics   = { },
1777  colorschemes = { },
1778}
1779\stopnarrowtyping
1780
1781We already saw the color schemes and now we will fill in the other tables. First
1782we define a couple of sets:
1783
1784\startnarrowtyping
1785local basics = {
1786  analyze  = "yes",
1787  mode     = "node",
1788  language = "dflt",
1789  script   = "arab",
1790}
1791
1792local analysis = {
1793  ccmp = "yes",
1794  init = "yes", medi = "yes", fina = "yes",
1795}
1796
1797local regular = {
1798  rlig = "yes", calt = "yes", salt = "yes", anum = "yes",
1799  ss01 = "yes", ss03 = "yes", ss07 = "yes", ss10 = "yes", ss12 = "yes",
1800  ss15 = "yes", ss16 = "yes", ss19 = "yes", ss24 = "yes", ss25 = "yes",
1801  ss26 = "yes", ss27 = "yes", ss31 = "yes", ss34 = "yes", ss35 = "yes",
1802  ss36 = "yes", ss37 = "yes", ss38 = "yes", ss41 = "yes", ss42 = "yes",
1803  ss43 = "yes", js16 = "yes",
1804}
1805
1806local positioning = {
1807  kern = "yes", curs = "yes", mark = "yes", mkmk = "yes",
1808}
1809
1810local minimal_stretching = {
1811  js11 = "yes", js03 = "yes",
1812}
1813
1814local medium_stretching = {
1815  js12="yes", js05="yes",
1816}
1817local maximal_stretching= {
1818  js13 = "yes", js05 = "yes", js09 = "yes",
1819}
1820
1821local wide_all = {
1822  js11 = "yes", js12 = "yes", js13 = "yes", js05 = "yes", js09 = "yes",
1823}
1824
1825local shrink = {
1826  flts = "yes", js17 = "yes", ss05 = "yes", ss11 = "yes", ss06 = "yes",
1827  ss09 = "yes",
1828}
1829
1830local default = {
1831  basics, analysis, regular, positioning, -- xxxx = "yes", yyyy = 2,
1832}
1833\stopnarrowtyping
1834
1835Next we define some featuresets and we use the default as starting point:
1836
1837\startnarrowtyping
1838  featuresets = {
1839    default = {
1840      default,
1841    },
1842    minimal_stretching = {
1843      default, js11 = "yes", js03 = "yes",
1844    },
1845    medium_stretching = {
1846      default, js12="yes", js05="yes",
1847    },
1848    maximal_stretching= {
1849      default, js13 = "yes", js05 = "yes", js09 = "yes",
1850    },
1851    wide_all = {
1852      default, js11 = "yes", js12 = "yes", js13 = "yes", js05 = "yes",
1853      js09 = "yes",
1854    },
1855    shrink = {
1856      default, flts = "yes", js17 = "yes", ss05 = "yes", ss11 = "yes",
1857      ss06 = "yes", ss09 = "yes",
1858    },
1859  }
1860\stopnarrowtyping
1861
1862When defining the font at the \TEX\ end we can now refer to for instance \type
1863{wide_all} which saves us some typing. However, it does not stop here. In a later
1864paragraph we will see how fonts can work in tandem with the parbuilder. For that
1865purpose the goodie table has a \type {solutions} subtable:
1866
1867\startnarrowtyping
1868solutions = {
1869  experimental = {
1870    less = {
1871      "shrink"
1872    },
1873    more = {
1874      "minimal_stretching", "medium_stretching", "maximal_stretching", "wide_all"
1875    },
1876  },
1877}
1878\stopnarrowtyping
1879
1880Here we define an experimental solution for optimizing the lines in a paragraph:
1881we can narrow words or we can widen them according to a specific featureset. In
1882order to reach the optimal solution the text will be retypeset under a different
1883feature regime.
1884
1885{\em TODO: show how to apply.}
1886
1887%D \starttyping
1888%D \setupfontsolutions[method={random,preroll},criterium=1,randomseed=101]
1889%D
1890%D \definefontsolution % actually only the last run needs to be done this way
1891%D   [FancyHusayni]
1892%D   [goodies=husayni,
1893%D    solution=experimental]
1894%D
1895%D \definedfont[husayni*husayni-default at 24pt]
1896%D \setupinterlinespace[line=36pt]
1897%D \righttoleft
1898%D \enabletrackers[parbuilders.solutions.splitters.colors]
1899%D \setfontsolution[FancyHusayni]
1900%D alb alb alb \par
1901%D \resetfontsolution
1902%D \disabletrackers[parbuilders.solutions.splitters.colors]
1903%D \stoptyping
1904
1905Because there are a some 55 stylistic and 21 justification variants the
1906goodie file also provides a \type {stylistics} table and for tracing purposes
1907the {colorschemes} table is populated.
1908
1909Yet another demonstration of manipulation is the following. Not all fonts come
1910with all combined glyphs. Although we have an auto|-|compose feature in \CONTEXT\
1911it sometimes helps to be specific with respect to some combinations. This is
1912where the \type {compositions} goodie kicks in:
1913
1914\starttyping
1915local compose = {
1916  [0x1E02] = {
1917    anchored = "top",
1918  },
1919  [0x1E04] = {
1920    anchored = "bottom",
1921  },
1922  [0x0042] = { -- B
1923    anchors = {
1924      top = {
1925        x = 300,
1926        y = 700,
1927      },
1928      bottom = {
1929        x = 300,
1930        y = -30,
1931      },
1932    },
1933  },
1934  [0x0307] = {
1935    anchors = {
1936      top = {
1937        x = -250,
1938        y = 550,
1939      },
1940    },
1941  },
1942  [0x0323] = {
1943    anchors = {
1944      bottom = {
1945        x = -250,
1946        y = -80,
1947      },
1948    },
1949  },
1950}
1951
1952return {
1953  name = "lm-compose-test",
1954  version = "1.00",
1955  comment = "Goodies that demonstrate composition.",
1956  author = "Hans and Mojca",
1957  copyright = "ConTeXt development team",
1958  compositions = {
1959    ["lmroman12-regular"] = compose,
1960  }
1961}
1962\stoptyping
1963
1964Of course this assumes some knowledge of the font metrics (in base points) and
1965\UNICODE\ slots, but it might be worth the trouble. After all, one only needs to
1966figure it out once. But keep in mind that it will always be a kludge.
1967
1968A slightly different way to define such compositions is the following:
1969
1970\starttyping
1971local defaultunits = 193 - 30
1972
1973local compose = {
1974                 DY = defaultunits,
1975 -- [0x010C] = { DY = defaultunits }, -- Ccaron
1976 -- [0x02C7] = { DY = defaultunits }, -- textcaron
1977}
1978
1979-- fractions relative to delta(X_height - x_height)
1980
1981local defaultfraction = 0.85
1982
1983local compose = {
1984  DY = defaultfraction, -- uppercase compensation
1985}
1986
1987return {
1988  name = "lucida-one",
1989  version = "1.00",
1990  comment = "Goodies that complement lucida.",
1991  author = "Hans and Mojca",
1992  copyright = "ConTeXt development team",
1993  compositions = {
1994    ["lbr"]  = compose,
1995    ["lbi"]  = compose,
1996    ["lbd"]  = compose,
1997    ["lbdi"] = compose,
1998  }
1999}
2000\stoptyping
2001
2002Of course no one really needs this because \OPENTYPE\ Lucida fonts
2003have replaced the \TYPEONE\ versions.
2004
2005The next goodie table is dedicated to the de facto standard \TEX\ font Latin
2006Modern. There is a bit of history behind this file. When we started writing
2007\CONTEXT\ there were not that many fonts available and so we ended up with a font
2008system that was rather well suited for the predecessor of Latin Modern, called
2009Computer Modern. And because these fonts came in design sizes the font system
2010was made such that it could cope efficiently with many files in a font set. Although
2011there is no additional overhead compared to small font sets, apart from more files,
2012there is some burden in defining them. And, as they are the default fonts, these
2013definitions slow down the initialization of \CONTEXT\ (which is due to the fact that
2014the large typescript definitions were loaded and parsed). So, at some point
2015the decision was made to kick out these definitions and move the burden of figuring
2016out the right size to \LUA. When Latin Modern is chosen as font the effect is the
2017same when design sizes are enabled. But, instead of many definitions (one for each
2018combination of size and style) we now have an option. A non|-|designsize typeface
2019is defined as follows:
2020
2021\startnarrowtyping
2022\starttypescript [modern,modern-base]
2023  \definetypeface [\typescriptone] [rm] [serif] [modern] [default]
2024  \definetypeface [\typescriptone] [ss] [sans]  [modern] [default]
2025  \definetypeface [\typescriptone] [tt] [mono]  [modern] [default]
2026  \definetypeface [\typescriptone] [mm] [math]  [modern] [default]
2027  \quittypescriptscanning
2028\stoptypescript
2029\stopnarrowtyping
2030
2031The designsize variant looks like this:
2032
2033\startnarrowtyping
2034\starttypescript [modern-designsize]
2035  \definetypeface [\typescriptone]
2036    [rm] [serif] [latin-modern-designsize] [default] [designsize=auto]
2037  \definetypeface [\typescriptone]
2038    [ss] [sans]  [latin-modern-designsize] [default] [designsize=auto]
2039  \definetypeface [\typescriptone]
2040    [tt] [mono]  [latin-modern-designsize] [default] [designsize=auto]
2041  \definetypeface [\typescriptone]
2042    [mm] [math]  [latin-modern-designsize] [default] [designsize=auto]
2043  \quittypescriptscanning
2044\stoptypescript
2045\stopnarrowtyping
2046
2047Of course there are accompanying typescripts that map the sans, serif, mono
2048and math styles onto files. The \type {designsize} magic uses the following
2049table. We show only part of the file, as it is in the \CONTEXT\ distribution.
2050
2051\starttyping
2052return {
2053  name = "latin modern",
2054  version = "1.00",
2055  comment = "Goodies that complement latin modern.",
2056  author = "Hans Hagen",
2057  copyright = "ConTeXt development team",
2058  mathematics = {
2059    tweaks = {
2060      aftercopying = {
2061        mathematics.tweaks.fixbadprime, -- prime is too low
2062      },
2063    },
2064  },
2065  designsizes = {
2066    ["LMMathRoman-Regular"] = {
2067      ["4pt"]  = "LMMath5-Regular@lmroman5-math",
2068      ...
2069      ["12pt"] = "LMMath12-Regular@lmroman12-math",
2070      default  = "LMMath10-Regular@lmroman10-math"
2071    },
2072    ["LMMathRoman-Bold"] = { -- not yet ready
2073      ...
2074    },
2075    ["LMRoman-Regular"] = {
2076      ["4pt"]  = "file:lmroman5-regular",
2077      ...
2078      ["12pt"] = "file:lmroman12-regular",
2079      default  = "file:lmroman10-regular",
2080    },
2081    ["LMRoman-Bold"] = {
2082      ...
2083    },
2084    ["LMRoman-Demi"] = {
2085      default  = "file:lmromandemi10-regular",
2086    },
2087    ["LMRoman-Italic"] = {
2088      ...
2089    },
2090    ...
2091    ["LMRoman-Unslanted"] = {
2092        default  = "file:lmromanunsl10-regular",
2093    },
2094    ["LMSans-Regular"] = {
2095        ...
2096    },
2097    ["LMTypewriter-Regular"] = {
2098        ...
2099    },
2100    ...
2101    ["LMTypewriterVarWd-DarkOblique"] = {
2102      default  = "file:lmmonoproplt10-boldoblique",
2103    },
2104    ...
2105    ["LMTypewriter-CapsOblique"] = {
2106      default  = "file:lmmonocaps10-oblique",
2107    },
2108  }
2109}
2110\stoptyping
2111
2112The \type {auto} option will choose a best fit compatible to the
2113\MKII\ implementation. When \type {default} is used instead, the
2114default filename will be taken. Of course one might wonder if
2115there will ever be similar goodie files because design sizes
2116are not that popular nowadays.
2117
2118Not all fonts are perfect and of course the \LUATEX\ engine can have flaws as
2119well. For this reason we can implement patches. Here is another example of a
2120goodie file that has some more code than just a table:
2121
2122\starttyping
2123local patches = fonts.handlers.otf.enhancers.patches
2124
2125local function patch(data,filename,threshold)
2126  local m = data.metadata.math
2127  if m then
2128    local d = m.DisplayOperatorMinHeight or 0
2129    if d < threshold then
2130      patches.report("DisplayOperatorMinHeight(%s -> %s)",d,threshold)
2131      m.DisplayOperatorMinHeight = threshold
2132    end
2133  end
2134end
2135
2136patches.register("after","analyze math","asana",
2137    function(data,filename) patch(data,filename,1350) end)
2138
2139local function less(value,target,original)
2140  return 0.25 * value
2141end
2142
2143local function more(value,target,original)
2144  local o = original.mathparameters.DisplayOperatorMinHeight
2145  if o < 2800 then
2146      return 2800 * target.parameters.factor
2147  else
2148      return value -- already scaled
2149  end
2150end
2151
2152return {
2153  name = "asana-math",
2154  version = "1.00",
2155  comment = "Goodies that complement asana.",
2156  author = "Hans Hagen",
2157  copyright = "ConTeXt development team",
2158  mathematics = {
2159    parameters = {
2160      DisplayOperatorMinHeight         = more,
2161      StackBottomDisplayStyleShiftDown = less,
2162      StackBottomShiftDown             = less,
2163      StackDisplayStyleGapMin          = less,
2164      StackGapMin                      = less,
2165      StackTopDisplayStyleShiftUp      = less,
2166      StackTopShiftUp                  = less,
2167      StretchStackBottomShiftDown      = less,
2168      StretchStackGapAboveMin          = less,
2169      StretchStackGapBelowMin          = less,
2170      StretchStackTopShiftUp           = less,
2171    }
2172  }
2173}
2174\stoptyping
2175
2176In fact, in addition to already mentioned \type {mapfiles} and
2177\type {virtuals} subtables, we can pass variables and
2178overload parameters.
2179
2180\starttyping
2181return {
2182  name = "lm-math",
2183  ...
2184  mathematics = {
2185    mapfiles = {
2186      ...
2187    },
2188    virtuals = {
2189      ...
2190    variables = {
2191      joinrelfactor = 3, -- default anyway
2192    },
2193    parameters = { -- test values
2194     -- FactorA = 123.456,
2195     -- FactorB = false,
2196     -- FactorC = function(value,target,original)
2197     --   return 7.89 * target.factor
2198     -- end,
2199     -- FactorD = "Hi There!",
2200    },
2201  }
2202}
2203\stoptyping
2204
2205This kind of goodie functionality is typical for the development of \LUATEX\ and
2206experimental math fonts and no user should ever be bothered with it. However, it
2207demonstrates that we're not stuck with only features built in the fonts.
2208
2209% mathdimensions
2210
2211It can be that a user is not satisfied by some aspects of a math font design.
2212There is not much that we can do about the shapes, but we can manipulate for
2213instance dimensions.
2214
2215For this there are two mechanism available: automatically applied dimensional
2216fixes and a \type {mathdimensions} feature. Both work with the same goody
2217specification.
2218
2219\starttyping
2220mathematics = {
2221  ...
2222  dimensions = {
2223  },
2224  ...
2225}
2226\stoptyping
2227
2228The entries in a dimensions table are tables themselves. There can be many
2229of them so one can organize dimensional tweaks in groups. The \type {default}
2230group is always applied, while others are applied on demand. Say that want
2231to tweak all \type {±} and \type {}. \footnote {In fact, this mechanism is a
2232a response to a mail on the \CONTEXT\ mailing list.}
2233
2234\starttyping
2235mathematics = {
2236  dimensions = {
2237    default = {
2238      [0x00B1] = { -- ±
2239        height = 500,
2240        depth  = 0,
2241      },
2242      [0x2213] = { -- 
2243        height = 500,
2244        depth  = 0,
2245      },
2246    },
2247  },
2248}
2249\stoptyping
2250
2251This will give these two characters a different height and depth. However, this
2252will not have much effect in rendering (much larger dimensions might have).
2253
2254\starttyping
2255mathematics = {
2256  dimensions = {
2257    default = {
2258      [0x00B1] = { -- ±
2259        yoffset =  100,
2260      },
2261      [0x2213] = { -- 
2262        yoffset = -100,
2263      },
2264    },
2265  },
2266}
2267\stoptyping
2268
2269This will raise and lower the glyphs in their bounding boxes and give them
2270an appearance more close to their ancestors. But defined this way, they are
2271always applied and that might not be what we  want. So, we can do this:
2272
2273\starttyping
2274mathematics = {
2275  dimensions = {
2276    signs = {
2277      [0x00B1] = { -- ±
2278        yoffset =  100,
2279      },
2280      [0x2213] = { -- 
2281        yoffset = -100,
2282      },
2283    },
2284  },
2285}
2286\stoptyping
2287
2288This time the application is feature driven. As with all features, setting them
2289up has to happen {\em before} fonts are loaded. This will do the trick:
2290
2291\starttyping
2292\definefontfeature [lm-math] [mathdimensions=signs]
2293\stoptyping
2294
2295The \type {lm-math} feature is not defined by default but can be used for such
2296purposes. It {\em is} defined with the fontname:
2297
2298\starttyping
2299\definefontsynonym
2300  [LMMathRoman-Regular]
2301  [file:latinmodern-math-regular.otf]
2302  [features={math\mathsizesuffix,lm-math},
2303   goodies=lm]
2304\stoptyping
2305
2306A rather specialized goodie is the one that is used to specify math cut|-|ins. A
2307good quality math font has these kerns already defined but even then you might
2308want to add or replace some by your own. Here is an example of such a patch.
2309Normally there are multiple goodies defined in one file but we only show kerns
2310here:
2311
2312\starttyping
2313local kern_200 = { bottomright = { { kern = -200 } } }
2314local kern_100 = { bottomright = { { kern = -100 } } }
2315
2316return {
2317    name = "pagella-math",
2318    version = "1.00",
2319    comment = "Goodies that complement pagella.",
2320    author = "Hans Hagen",
2321    copyright = "ConTeXt development team",
2322    mathematics = {
2323        kerns = {
2324            [0x1D449] = kern_200, -- math italic V
2325            [0x1D44A] = kern_100, -- math italic W
2326        },
2327    },
2328}
2329\stoptyping
2330
2331As with other goodies the file is loaded with:
2332
2333\starttyping
2334\definefontsynonym
2335  [MathRoman]                     % names used in definitions
2336  [file:texgyrepagella-math.otf]  % the file to be loaded
2337  [features=math\mathsizesuffix,  % size dependent features
2338   goodies=pagella-math]          % the goodie file to be applied
2339\stoptyping
2340
2341This is typically a goodie that is always applied and not driven by a feature.
2342After all, the values given are passed to the engine (after being scaled).
2343
2344Most goodies are bound to fonts of collections of fonts. This is different for
2345treatments. These ship with the distribution but you can also provide your own.
2346As this is still somewhat experimental we just mention a few aspects. The entries
2347are filenames that point to tables.
2348
2349\starttyping
2350return {
2351  name = "treatments",
2352  version = "1.00",
2353  comment = "Goodies that deals with some general issues.",
2354  author = "Hans Hagen",
2355  copyright = "ConTeXt development team",
2356  treatments = {
2357    ["adobeheitistd-regular.otf"] = {
2358      embedded = false, -- not yet used
2359      comment  = "this font is part of acrobat",
2360    },
2361    ["crap.ttf"] = {
2362      ignored = true,
2363      comment = "a text file with suffix ttf",
2364    },
2365    ["latinmodern-math.otf"] = {
2366      comment = "experimental",
2367    },
2368    ["rubish-regular.ttf"] = {
2369      comment = "check output for missing à and á",
2370    }
2371  },
2372}
2373\stoptyping
2374
2375The comment entry in such a table becomes part of the message at the end
2376of a run:
2377
2378\startnarrowtyping
2379mkiv lua stats  > loaded fonts: 2 files: latinmodern-math.otf (experimental), lmroman12-regular.otf
2380\stopnarrowtyping
2381
2382The ignored flag signals the font name database builder to ignore the file. This
2383means that the font can still be known as file, but that its (name based)
2384properties are not collected. As you asked explicitly for a file, the file can
2385still be loaded. You can use this trick to avoid issues with the database builder
2386in case of a problematic file, but a real run will still try to load the file. After
2387all, you get what you ask for. If loading and usage is successful you get at least
2388the message reported at the end of the run.
2389
2390\stopsection
2391
2392\startsection[title=Analyzers]
2393
2394An \OPENTYPE\ font is kind of special in the sense that it provides some
2395information on how to turn sequences of characters into sequences of glyphs. In
2396fact, if all fonts had a reasonable repertoire of glyphs most of the information
2397that concerns combining, remapping and shuffling the input and|/|or mapping onto
2398glyphs could as well happen in the renderer. This means that fonts have many of
2399their internal features tables in common, or more precisely could share many gsub
2400related issues, if only there had been some predefined sets of substitutional
2401features.
2402
2403So, for most of the time, a feature processor just does what the font demands and
2404the font provides the information. There are however a few cases where font only
2405provide part of the logic. Take for instance the \type {init}, \type {medi},
2406\type {fina} and \type {isol} features that relate to positions in the word: the
2407start, the end, in the middle or isolated. For these features to work the engine
2408has to provide information about the state of a character (glyph) and this is where
2409analysis kicks in. Just watch this:
2410
2411\startbuffer
2412\definefontfeature
2413  [default-with-analyze]
2414  [default]
2415  [script=latn,mode=node,
2416   init=yes,medi=yes,fina=yes,isol=yes]
2417
2418\showotfcomposition
2419  {dejavu-serif*default-with-analyze at 24pt}
2420  {}
2421  {I don't wanna know tha\utfchar{"300}t!}
2422\stopbuffer
2423
2424\typebuffer
2425
2426In the tracer the different categories are colored. This kind of information is
2427especially important for typesetting Arabic. Normally \CONTEXT\ can figure out
2428itself when this is needed so you don't have to worry too much about this kind of
2429additional actions.
2430
2431\blank \getbuffer \blank
2432
2433\stopsection
2434
2435\startsection[title=Processors]
2436
2437    \startnotabene
2438        Todo.
2439    \stopnotabene
2440
2441\stopsection
2442
2443\startsection[title=Optimizing]
2444
2445    \startnotabene
2446        Todo.
2447    \stopnotabene
2448
2449\stopsection
2450
2451\startsection[title=Tracing]
2452
2453There are a lot of tracing options in \MKIV, but most will never be seen by users. Most
2454are enabled using the tracker mechanism. Some have a bit more visibility and have a dedicated
2455command to trigger them.
2456
2457When something is going terribly wrong, you will always get a message but sometimes even an
2458end|-|user has to request for more information. An example are missing characters. There are
2459several ways to get them reported:
2460
2461\starttyping
2462\enabletrackers[fonts.missing=replace]
2463\enabletrackers[fonts.missing=remove]
2464\enabletrackers[fonts.missing]
2465\stoptyping
2466
2467For historic reasons we also have:
2468
2469\starttyping
2470\checkcharactersinfont
2471\removemissingcharacters
2472\replacemissingcharacters
2473\stoptyping
2474
2475which happens automatically when you enable the tracker. There is some extra
2476overhead involved so you might want to turn on this feature on only if you really
2477expect characters not to be present.
2478
2479Say that we use Latin Modern fonts and ask for some of the rare fractions:
2480
2481\startbuffer
2482\definedfont[lmroman10-regular*default-with-missing at 10pt]
2483a b c ½  ¼    Ɣ ɣ ʤ ʭ ʮ α β γ
2484\stopbuffer
2485
2486\typebuffer
2487
2488\enabletrackers[fonts.missing=replace]
2489We get this: \start \getbuffer \stop
2490\removeunwantedspaces . \space
2491In the log file you will find something like this:
2492\par \disabletrackers[fonts.missing]
2493
2494\starttyping
2495fonts   > characters > start missing characters: lmroman10-regular.otf
2496
2497missing > U+00194  Ɣ  LATIN CAPITAL LETTER GAMMA
2498missing > U+00263  ɣ  LATIN SMALL LETTER GAMMA
2499missing > U+002A4  ʤ  LATIN SMALL LETTER DEZH DIGRAPH
2500missing > U+002AD  ʭ  LATIN LETTER BIDENTAL PERCUSSIVE
2501missing > U+002AE  ʮ  LATIN SMALL LETTER TURNED H WITH FISHHOOK
2502missing > U+003B1  α  GREEK SMALL LETTER ALPHA
2503missing > U+003B2  β  GREEK SMALL LETTER BETA
2504missing > U+003B3  γ  GREEK SMALL LETTER GAMMA
2505missing > U+02153    VULGAR FRACTION ONE THIRD
2506missing > U+02155    VULGAR FRACTION ONE FIFTH
2507missing > U+02159    VULGAR FRACTION ONE SIXTH
2508missing > U+0215B    VULGAR FRACTION ONE EIGHTH
2509
2510fonts   > characters > stop missing characters
2511\stoptyping
2512
2513If you're lucky your editor will use a font that shows the missing characters (dejavu
2514monospace is a good candidate).
2515
2516The replacement characters can help you to locate the spots where something is missing
2517so that an alternative can be considered. The replacements resemble the category
2518of the missing character.
2519
2520\showmissingcharacterslegend
2521
2522You can call up this legend after loading an extra module:
2523
2524\starttyping
2525\usemodule[s][fonts-missing]
2526
2527\showmissingcharacterslegend
2528
2529\showmissingcharacters
2530\stoptyping
2531
2532The last command shows a detailed list of missing characters
2533
2534\showmissingcharacters
2535
2536Here the characters are shown, because we use a monospaced font that happens to
2537have them. Of course this example uses characters that are rarely used and are
2538unlikely to show up in future versions of the Latin Modern fonts.
2539
2540\startnotabene
2541    Here a few more relevant trackers will be mentioned.
2542\stopnotabene
2543
2544\stopsection
2545
2546% \startsection[title=Discretionaries]
2547%
2548% \startbuffer
2549% \definedfont[cambria*default]
2550% 12\discretionary
2551%     {3} {4} {5}%
2552% 67\par
2553% 12{\oldstyle\discretionary
2554%     {3} {4} {5}}%
2555% 67\par
2556% 12\discretionary
2557%     {3{\oldstyle3}} {{\oldstyle4}4} {5{\oldstyle5}5}%
2558% 67\par
2559% \stopbuffer
2560%
2561% The font handler has to do some magick to get features working with and across
2562% discretionaries. To some extend you can use font switches inside discretionaries
2563% but for sure border cases are not dealt with. This works:
2564%
2565% \startlinecorrection[blank]
2566% \startcombination[nx=4,ny=1,location=top]
2567%     {\framed[align=normal]{\enabledirectives [otf.alwaysdisc]\setupwhitespace[line]\getbuffer}} {1}
2568%     {\framed[align=normal]{\enabledirectives [otf.alwaysdisc]\hsize1mm\getbuffer}} {2}
2569%     {\framed[align=normal]{\disabledirectives[otf.alwaysdisc]\setupwhitespace[line]\getbuffer}} {3}
2570%     {\framed[align=normal]{\disabledirectives[otf.alwaysdisc]\hsize1mm\getbuffer}} {4}
2571% \stopcombination
2572% \stoplinecorrection
2573%
2574% The first two examples have \type {otf.alwaysdisk} enabled, the last two have it
2575% disabled.
2576%
2577% \typebuffer
2578%
2579% \stopsection
2580
2581\startsection[title=Some remarks]
2582
2583If you talk about features and fonts it is not difficult to end up speaking
2584\OPENTYPE . However, in \CONTEXT\ we use the term in a more general way, if only
2585because we provide more features. In traditional \TEX\ we have a few features:
2586ligatures, smallcaps and kerns, and to some extent we can see oldstyle numerals
2587also as feature. It is however important to notice that in \OPENTYPE\ ligatures
2588are just a synonym for combining multiple characters into on. From the user
2589interface point of view these operations are grouped into \type {liga}, \type
2590{dlig}, \type {clig} and \type {rlig} and for \TEX ies we have \type {tlig}. The
2591distinction is not as clear as one might think: any feature can use the ligature
2592builder. And as a consequence we see that happen too, for instance some fonts use
2593\type {ccmp} for constructing mandatory ligatures.
2594
2595Some of these interpretations (or maybe even tricks) are side effects of for
2596instance user interfaces. If one can for instance not turn on or off the \type
2597{ccmp} feature, but can do that for \type {liga}, then one way to keep some
2598ligatures in for instance letter spaced text, is to put them into \type {ccmp},
2599assuming that this one will always be enabled. Eventually that then becomes a
2600sort of standard. Personally I don't like such pseudo standards but we have to
2601live with them.
2602
2603Another example of such a standard is the used of non breakable spaces to
2604influence treatment of some Devanagari characters. Where \UNICODE\ has special
2605characters to influence mechanisms that combine and replace characters, the lack
2606of some triggers others to be used and eventually that becomes a standard.
2607Similar ambiguities arise with math: we have no way to indicate math (while we do
2608have ways to indicate a change in writing order).
2609
2610Talking of math, take \OPENTYPE\ math: at some point there is a draft, that then
2611gets implemented in one word processor using one font, but omissions or
2612imperfections that surface (maybe because more fonts and engines are developed)
2613stay around because the initial implementation is published and frozen, simply
2614because there are many users that stick to expectations. Where \TEX ies accept a
2615few years of development, this is not true for commercial applications. \footnote
2616{Of course \HTML\ is the biggest example of this: we're stuck forever with open
2617tags without close tags, mixed uppercase and lowercase tags, attributes without
2618value or values without quotes.}
2619
2620So, although there is without doubt progress, some annoyances stay. The \TEX\
2621community has always been able to adapt, and this is one reason why a \LUA\
2622implementation is nice: it gives room for experiments, extensions, variants, etc.
2623Of course it also makes a bit more independent, although one may wonder if that
2624matters any longer in a rapidly changing world. The original idea behind \TEX,
2625that it should be useable for ages, will survive, but users might see more
2626changes in a lifetime than foreseen when \TEX\ showed up.
2627
2628\stopsection
2629
2630\startsection[title=Different spaces]
2631
2632The width of the space and its stretch and shrink are taken from the font. The so
2633called emspace is the reference for much spacing related parameters. It is the
2634width of character \type {0x2014}. The regular space width is taken from \type
2635{0x0020}, the space character. When there is no space character, in the case of a
2636monospaced font we take the emwidth, otherwise we take half the emwidth. As a
2637last resort we can take the average width of characters. And of even that fails
2638we take half of the font units. When there is no emwidth that one is set to the
2639font units.
2640
2641In the \CONTEXT\ font loader we use a stretch that is 1/2 of the width of a space
2642and the shrink is 1/3 the width of a space, so we use values that are quite
2643similar to what \TEX\ always used.
2644
2645You can overload these values when a font is loaded and the overload is
2646implemented as a feature. The next example demonstrates how this is done:
2647
2648\startbuffer
2649\definefontfeature[morespace][spacing=0.50 plus 0.50 minus 0.250]
2650\definefontfeature[lessspace][spacing=0.25 plus 0.25 minus 0.125]
2651
2652\definedfont[Serif*default]          \samplefile{klein}\blank
2653\definedfont[Serif*default,morespace]\samplefile{klein}\blank
2654\definedfont[Serif*default,lessspace]\samplefile{klein}\blank
2655\definedfont[Serif*default]          \samplefile{klein}\blank
2656\stopbuffer
2657
2658\typebuffer \blank \getbuffer \blank
2659
2660\stopsection
2661
2662\startsection[title=Dynamic features]
2663
2664We can enable and disable features any time in the input by using the
2665\type {\feature} command. he following example was posted on the list:
2666
2667\startbuffer
2668\definefont
2669  [WeirdShapes]
2670  [file:libertiner*default]
2671
2672\definefontfeature
2673  [hist]
2674  [hlig=yes]
2675
2676\definefontfeature
2677  [rare]
2678  [dlig=yes]
2679
2680\setupquotation
2681  [style={\feature[+][hist,rare]}]
2682
2683\startlines
2684\WeirdShapes
2685strict {\feature[+][hist]strict}
2686wurtzite {\feature[+][rare]wurtzite}
2687\quotation{strict wurtzite}
2688\stoplines
2689\stopbuffer
2690
2691\typebuffer
2692
2693Or typeset:
2694
2695\getbuffer
2696
2697The \type {\feature} command takes as first argument a directive of what
2698do do:
2699
2700\starttabulate[|T||]
2701\NC + more        \NC add set to previous set and combine with font set \NC \NR
2702\NC - less        \NC subtract set to previous set and combine with font set \NC \NR
2703\NC = new         \NC replace font set \NC \NR
2704\NC ! < reset     \NC forget sets and revert to font set \NC \NR
2705\NC > old default \NC make sure the current set is used on top of the font set \NC \NR
2706\stoptabulate
2707
2708\stopsection
2709
2710\startsection[title=Spacekerns]
2711
2712Some fonts kern glyphs with spaces. Although \TEX\ doesn't really have spaces we do
2713support this. However, it's implemented as part of kerning so when you define such
2714kerns you need to hook it into for instance the \type {kern} feature:
2715
2716\starttyping
2717\startluacode
2718    local kern = -50
2719    local pair = { [32] = kern }
2720
2721    fonts.handlers.otf.addfeature {
2722        name    = "kern", -- spacekerns assume kern
2723        type    = "kern",
2724        data    = {
2725            A = pair, V = pair, W = pair,
2726            [32] = {
2727                A = kern,
2728                V = kern,
2729                W = kern,
2730            },
2731        }
2732    }
2733\stopluacode
2734\stoptyping
2735
2736Of course this depends on font properties so one can wonder how useful this is.
2737
2738\stopsection
2739
2740\stopchapter
2741
2742\stopcomponent
2743