fonts-extensions.tex /size: 80 Kb    last modification: 2021-10-28 13:50
1% language=us runpath=texruns:manuals/fonts
2
3\startcomponent fonts-extensions
4
5\environment fonts-environment
6
7\startchapter[title=Extensions][color=darkorange]
8
9\startsection[title=Introduction]
10
11One of the benefits of using \TEX\ is that you can add your own features and try
12to optimize the look and feel. Of course this can also go wrong and output can
13look pretty awful when you don't know what you're doing, but on the average it
14works out well. In many aspects the move to an \UNICODE\ data path and \OPENTYPE\
15fonts is a good one and solves a lot of problems with traditional \TEX\ engines
16and helps us to avoid complex and ugly hacks. But, if you look into the source
17code of \CONTEXT\ you will notice that there's still quite some complex coding
18needed. This is because we want to control mechanisms, even if it's only for
19dealing with some border cases. It's also the reason why \LUATEX\ is what it is:
20an extensible engine, building on tradition.
21
22As always with \TEX, fonts are an area where many tuning happens and this is also
23true in \CONTEXT. In this chapter some of the extensions will be discussed. Some
24extensions run on top of the (rather generic) feature mechanism and some are
25using dedicated code.
26
27\stopsection
28
29\startsection[title=Italics]
30
31Although \OPENTYPE\ fonts are more rich in features than traditional \TEX\ and
32\TYPEONE\ fonts, one important feature is missing: italic correction. This might
33sound strange but you need to keep in mind that in practice it's a feature that
34needs to be applied manually.
35
36\starttyping
37test {\it test\/} test
38\stoptyping
39
40It is possible to automate this mechanism and this is what the \type {\em} command
41does in \MKII:
42
43\starttyping
44test {\em test} test
45\stoptyping
46
47This command knows that it switches to italic (or slanted) and when used nested it
48knows to switch back. It also knows if a bold italic or slanted font is used. Therefore
49it can add italic correction between an italic and upright shape.
50
51An italic correction is bound to a glyph and bound to a font. In \in {figure}
52[latinmodern-italic] we see how an italic shape extends out of the bounding box.
53This is not the case in Dejavu: watch \in {figure} [dejavu-italic].
54
55\startplacefigure[reference=latinmodern-italic,title={Italic overshoot in Latin Modern.}]
56    \startcombination
57        \startcontent
58            \backgroundline[gray]{\color[maincolor]{\definedfont[lmroman10-regular*default sa 8]test}}
59        \stopcontent
60        \startcaption
61            Latin Modern Roman Regular
62        \stopcaption
63        \startcontent
64            \backgroundline[gray]{\color[maincolor]{\definedfont[lmroman10-italic*default sa 8]test}}
65        \stopcontent
66        \startcaption
67            Latin Modern Roman Italic
68        \stopcaption
69    \stopcombination
70\stopplacefigure
71
72\startplacefigure[reference=dejavu-italic,title={Italic overshoot in Dejavu Serif.}]
73    \startcombination
74        \startcontent
75            \backgroundline[gray]{\color[maincolor]{\definedfont[dejavuserif*default sa 8]test}}
76        \stopcontent
77        \startcaption
78            Dejavu Regular
79        \stopcaption
80        \startcontent
81            \backgroundline[gray]{\color[maincolor]{\definedfont[dejavuserifitalic*default sa 8]test}}
82        \stopcontent
83        \startcaption
84            Dejavu Italic
85        \stopcaption
86    \stopcombination
87\stopplacefigure
88
89This means that the application of italic correction should never been applied without
90knowing the font. In  \in {figure} [italic-upright] we see an upright word following
91an italic. The space is determined by the upright one.
92
93\startplacefigure[reference=italic-upright,title={Italic followed by upright.}]
94    \startcombination
95        \startcontent
96            \backgroundline
97                [gray]
98                {\color[maincolor]{\definedfont[lmroman10-italic*default  sa 4]test}
99                 \color[maincolor]{\definedfont[lmroman10-regular*default sa 4]\space test}}
100        \stopcontent
101        \startcaption
102            Latin Modern
103        \stopcaption
104        \startcontent
105            \backgroundline
106                [gray]
107                {\color[maincolor]{\definedfont[dejavuserifitalic*default sa 4]test}%
108                 \color[maincolor]{\definedfont[dejavuserif*default       sa 4]\space test}}
109        \stopcontent
110        \startcaption
111            Dejavu
112        \stopcaption
113    \stopcombination
114\stopplacefigure
115
116Because it is to be used with care you need to enable this feature per font, You
117also need to explicitly enable the application of this correction. in \in {figure}
118[italic-one] we see italic correction in action.
119
120\startbuffer
121\definefontfeature
122  [italic]
123  [default]
124  [itlc=yes]
125\stopbuffer
126
127\typebuffer
128
129\getbuffer
130
131\startplacefigure[reference=italic-one,title={Italic correction.}]
132    \startcombination
133        \startcontent
134            \backgroundline
135                [maincolor]
136                {\color[white]{\definedfont[lmroman10-italic*default  sa 4]test}
137                 \color[white]{\definedfont[lmroman10-regular*default sa 4]\space test}}
138        \stopcontent
139        \startcaption
140            \backgroundline
141                [maincolor]
142                {\setupitaliccorrection[text]%
143                 \color[white]{\definedfont[lmroman10-italic*italic   sa 4]test}
144                 \color[white]{\definedfont[lmroman10-regular*default sa 4]\space test}}
145        \stopcaption
146        \startcontent
147            \backgroundline
148                [maincolor]
149                {\color[white]{\definedfont[dejavuserifitalic*default sa 4]test}
150                 \color[white]{\definedfont[dejavuserif*default       sa 4]\space test}}
151        \stopcontent
152        \startcaption
153            \backgroundline
154                [maincolor]
155                {\setupitaliccorrection[text]%
156                 \color[white]{\definedfont[dejavuserifitalic*italic sa 4]test}
157                 \color[white]{\definedfont[dejavuserif*default      sa 4]\space test}}
158        \stopcaption
159    \stopcombination
160\stopplacefigure
161
162This only signals the font constructor that additional italic information has
163to be added to the font metrics. As we already mentioned, the application of
164correction is driven by the \type {\/} primitive and that one consults the
165font metrics. Because the correction is not part of the original font
166metrics it is calculated automatically by adding a small value to the
167width. This value is calculated as follows:
168
169\starttyping
170factor * (parameters.uwidth or 40) / 2
171\stoptyping
172
173The \type {uwidth} parameter is sometimes part of the specification but if not, we
174take a reasonable default.  The factor is under user control:
175
176\startbuffer
177\definefontfeature
178  [moreitalic]
179  [default]
180  [itlc=5]
181\stopbuffer
182
183\typebuffer
184
185\getbuffer
186
187This is demonstrated in \in {figure} [italic-two]. You will notice that for Latin
188Modern (any) correction makes sense, but for Dejavu it probably makes things look
189worse. This is why italic correction is disabled by default. When enabled there
190are several variants:
191
192\starttabulate[|Bl|l|]
193\NC global \NC always apply correction \NC \NR
194\NC text   \NC only apply correction to text \NC \NR
195\NC always \NC apply correction between text and boxes \NC \NR
196\NC none   \NC forget about correction \NC \NR
197\stoptabulate
198
199We keep track of the state using attributes but that comes at a (small) price in terms
200of extra memory and runtime. The \type {global} option simply assumes that we always
201need to check for correction (of course only for fonts that have this feature enables).
202In the given example we used:
203
204\starttyping
205\setupitaliccorrection
206  [text]
207\stoptyping
208
209You can combine keys:
210
211\starttyping
212\setupitaliccorrection
213  [global,always]
214\stoptyping
215
216\startplacefigure[reference=italic-two,title={Italic correction (factor 5).}]
217    \startcombination
218        \startcontent
219            \backgroundline
220                [maincolor]
221                {\color[white]{\definedfont[lmroman10-italic*default  sa 4]test}
222                 \color[white]{\definedfont[lmroman10-regular*default sa 4]\space test}}
223        \stopcontent
224        \startcaption
225            \backgroundline
226                [maincolor]
227                {\setupitaliccorrection[text]%
228                 \color[white]{\definedfont[lmroman10-italic*italic   sa 4]test}
229                 \color[white]{\definedfont[lmroman10-regular*default sa 4]\space test}}
230        \stopcaption
231        \startcontent
232            \backgroundline
233                [maincolor]
234                {\color[white]{\definedfont[dejavuserifitalic*default sa 4]test}
235                 \color[white]{\definedfont[dejavuserif*default       sa 4]\space test}}
236        \stopcontent
237        \startcaption
238            \backgroundline
239                [maincolor]
240                {\setupitaliccorrection[text]%
241                 \color[white]{\definedfont[dejavuserifitalic*italic  sa 4]test}
242                 \color[white]{\definedfont[dejavuserif*default       sa 4]\space test}}
243        \stopcaption
244    \stopcombination
245\stopplacefigure
246
247The \type {itlc} feature controls if a font gets italic correction applied. In
248principle this is all that the user needs to do, given that the mechanism is
249enabled. These is an extra feature that controls the implementation:
250
251\starttabulate[|T|T|p|]
252\NC itlc        \NC no    \NC don't apply italic correction (default) \NC \NR
253\NC             \NC yes   \NC apply italic correction \NC \NR
254\NC textitalics \NC no    \NC precalculate italic corrections (permit engine usage) \NC \NR
255\NC             \NC yes   \NC precalculate italic corrections (inhibit engine) \NC \NR
256\NC             \NC delay \NC delay calculation of corrections \NC \NR
257\stoptabulate
258
259When \type {textitalics} is set to \type {yes} or \type {delay} the mechanism
260built into the engine is completely disabled. When set to \type {no} the engine
261can kick in but normally the alternative method takes precedence so that the
262engine sees no reason for further action. You can trace italic corrections with:
263
264\starttyping
265\enabletrackers[typesetters.italics]
266\stoptyping
267
268\stopsection
269
270\startsection[title=Bounding boxes]
271
272\startbuffer
273\definefontfeature
274  [withbbox]
275  [boundingbox=yes]
276
277\definefont
278  [FontWithBB]
279  [Normal*withbbox]
280\stopbuffer
281
282\start \getbuffer \FontWithBB
283
284There are some features that are rather useless and only make sense when figuring out
285issues. An example of such a feature is the following:
286
287\typebuffer
288
289This feature adds a background to each character in a font. In some fonts a glyph
290has a tight bounding box, while on other fonts some extra space is put on the left
291and right. Keep in mind that this feature blocks colored text.
292
293\par \stop
294
295\stopsection
296
297\startsection[title=Math italics]
298
299In the traditional \TEX\ fonts the width of a glyph was not the real width because
300one had to add the italic correction to it. The engine then juggles a bit with
301these properties. If you run into fonts that are designed this way, you can do this:
302
303\starttyping
304\definefontfeature[mathextra][italicwidths=yes] % fix latin modern
305\stoptyping
306
307This might make \type {$\left|V\right| = \left|W\right|$} look better for such
308fonts. Of course there can be side effects because these fonts assume a
309traditional engine.
310
311\stopsection
312
313\startsection[title=Slanting]
314
315This features (as well as the one described in the next section) are seldom used
316but provided because they were introduced in \PDFTEX.
317
318\startbuffer[define]
319\definefontfeature
320  [abitslanted]
321  [default]
322  [slant=.1]
323
324\definefontfeature
325  [abitmoreslanted]
326  [default]
327  [slant=.2]
328\stopbuffer
329
330\startbuffer[sample]
331\definedfont[Normal*abitslanted]This is a bit slanted.
332\definedfont[Normal*abitmoreslanted]And this is a bit more slanted.
333\stopbuffer
334
335\typebuffer[define,sample]
336
337The result is:
338
339\getbuffer[define]
340
341\startlines
342\getbuffer[sample]
343\stoplines
344
345\stopsection
346
347\startsection[title=Extending]
348
349The second manipulation is extending the shapes horizontally:
350
351\startbuffer[define]
352\definefontfeature
353  [abitbolder]
354  [default]
355  [extend=1.3]
356
357\definefontfeature
358  [abitnarrower]
359  [default]
360  [extend=0.7]
361\stopbuffer
362
363\startbuffer[sample]
364\definedfont[Normal*abitbolder]This looks a bit bolder.
365\definedfont[Normal*abitnarrower]And this is a bit narrower.
366\stopbuffer
367
368\typebuffer[define,sample]
369
370The result is:
371
372\getbuffer[define]
373
374\startlines
375\getbuffer[sample]
376\stoplines
377
378We can also combine slanting and extending:
379
380\startbuffer[define]
381\definefontfeature
382  [abitofboth]
383  [default]
384  [extend=1.3,
385   slant=.1]
386\stopbuffer
387
388\startbuffer[sample]
389\definedfont[Normal*abitofboth]This is a bit bolder but also slanted.
390\stopbuffer
391
392\typebuffer[define,sample]
393
394If you remember those first needle matrix printers you might recognize the
395next rendering:
396
397\getbuffer[define]
398
399\startlines
400\getbuffer[sample]
401\stoplines
402
403\stopsection
404
405\startsection[title=Fixing] % dimensions
406
407This is a rather special one. First we show a couple of definitions:
408
409\startbuffer
410\definefontfeature
411  [dimensions-a]
412  [default]
413  [dimensions={1,1,1}]
414
415\definefontfeature
416  [dimensions-b]
417  [default]
418  [dimensions={1,2,3}]
419
420\definefontfeature
421  [dimensions-c]
422  [default]
423  [dimensions={1,3,2}]
424
425\definefontfeature
426  [dimensions-d]
427  [default]
428  [dimensions={3,3,3}]
429\stopbuffer
430
431\typebuffer \getbuffer
432
433When you don't want a dimension to change you leave an entry empty, so
434valid entries are for instance: \type {,3,} and \type {1,,}.
435
436As usual you apply such a feature as follows:
437
438\starttyping
439\definefont[MyFont][Serif*dimensions-a sa 2]
440\stoptyping
441
442Alternatively you can use such a feature on its own:
443
444\starttyping
445\definefontfeature
446  [dimensions-333]
447  [dimensions={3,3,3}]
448\definefont[MyFont][Serif*default,dimensions-333 sa 2]
449\stoptyping
450
451In \in {figure} [dimensions-side-by-side] you see these four definitions in
452action. The leftmost rendering is the default rendering. The three numbers in the
453definitions represent the width (in em), height and depth (in ex).
454
455\startplacefigure[reference={dimensions-side-by-side},title={Freezing dimensions of glyphs.}]
456    \startcombination[5*1]
457        \startcontent \hbox to 7em {\hss\ruledhbox{\definedfont[Serif*default      sa 2]g}\hss}\stopcontent \startcaption default     \stopcaption
458        \startcontent \hbox to 7em {\hss\ruledhbox{\definedfont[Serif*dimensions-a sa 2]g}\hss}\stopcontent \startcaption \hbox{1em 1ex 1ex} \stopcaption
459        \startcontent \hbox to 7em {\hss\ruledhbox{\definedfont[Serif*dimensions-b sa 2]g}\hss}\stopcontent \startcaption \hbox{1em 2ex 3ex} \stopcaption
460        \startcontent \hbox to 7em {\hss\ruledhbox{\definedfont[Serif*dimensions-c sa 2]g}\hss}\stopcontent \startcaption \hbox{1em 3ex 2ex} \stopcaption
461        \startcontent \hbox to 7em {\hss\ruledhbox{\definedfont[Serif*dimensions-d sa 2]g}\hss}\stopcontent \startcaption \hbox{3em 3ex 3ex} \stopcaption
462    \stopcombination
463\stopplacefigure
464
465This feature only makes sense for fonts that need a fixed width, like the
466\CJK\ fonts that are used for asian scripts. Normally those fonts already
467have fixed dimensions, but this feature can be used to fix problematic
468fonts or add some more space. However, for such large fonts this also brings a
469larger memory footprint.
470
471A special case is the following:
472
473\startbuffer
474\definefontfeature
475  [dimensions-e]
476  [dimensions=strut]
477\stopbuffer
478
479\typebuffer \getbuffer
480
481This will make the height and depth the same as the {\em current} strut height
482and depth:
483
484\startbuffer
485\ruledhbox{\definedfont[Serif*default,dimensions-e at  8pt]clipped}
486\ruledhbox{\definedfont[Serif*default,dimensions-e at 12pt]clipped}
487\ruledhbox{\definedfont[Serif*default,dimensions-e at 24pt]clipped}
488\stopbuffer
489
490\typebuffer
491
492The dimensions are (in this case) limited:
493
494\startlinecorrection[blank] \dontleavehmode \hpack{\maincolor\inlinebuffer} \stoplinecorrection
495
496Another special case is \type {dimensions=mono} which will make an characters the
497fonts em|-|width. This is handy when you define font fallbacks where glyphs come
498from a non monospaced font.
499
500\stopsection
501
502\startsection[title=Unicoding]
503
504Nowadays we will mostly use fonts that ship with a \UNICODE\ aware encoding. And
505in \CONTEXT, even if we use a \TYPEONE\ font, it gets mapped onto \UNICODE.
506However, there are some exceptions, for instance the Zapf Dingbats in \TYPEONE\
507format. These have a rather obscure private encoding and the glyph names run from
508\type {a1} upto \type {a206} and have no relation to what the glyph represents.
509
510In the case of Dingbats we're somewhat lucky that they ended up in \UNICODE, so
511we can relocate the glyphs to match their rightful place. This is done by means
512of a goodies file. We already discussed this in \in {section} [goodies] so we
513only repeat the usage.
514
515\startbuffer
516\definefontfeature
517  [dingbats]
518  [mode=base,
519   goodies=dingbats,
520   unicoding=yes]
521
522\definefontsynonym
523  [ZapfDingbats]
524  [file:uzdr.afm]
525  [features=dingbats]
526\stopbuffer
527
528\typebuffer \getbuffer
529
530I tend to qualify the Dingbat font in \TEX\ distributions as rather unstable
531because of name changes and them either or not being included. Therefore it's best to
532use the hard coded name because that triggers the most visible error message when
533the font is not found.
534
535A font like this can for instance be used with the glyph placement macros as is
536demonstrated below. In the last line we see that a direct \UTF\ input also works
537out well.
538
539\starttabulate[|||T|]
540\HL
541\NC \type{\getglyphdirect     {ZapfDingbats*dingbats}{\number"2701}} \NC \getglyphdirect     {ZapfDingbats*dingbats}{\number"2701} \NC \NC \NR
542\NC \type{\getglyphdirect     {ZapfDingbats*dingbats}{\char"2701}}   \NC \getglyphdirect     {ZapfDingbats*dingbats}{\char"2701}   \NC \NC \NR
543\NC \type{\getnamedglyphdirect{ZapfDingbats*dingbats}{a1}}           \NC \getnamedglyphdirect{ZapfDingbats*dingbats}{a1}           \NC \NC \NR
544\NC \type{\getnamedglyphdirect{ZapfDingbats*dingbats}{a11}}          \NC \getnamedglyphdirect{ZapfDingbats*dingbats}{a11}          \NC \NC \NR
545\HL
546\NC \type{\getglyphdirect     {ZapfDingbats}{\number"2701}}          \NC \getglyphdirect     {ZapfDingbats}{\number"2701}          \NC unknown \NC \NR
547\NC \type{\getglyphdirect     {ZapfDingbats}{\char"2701}}            \NC \getglyphdirect     {ZapfDingbats}{\char"2701}            \NC unknown \NC \NR
548\NC \type{\getnamedglyphdirect{ZapfDingbats}{a1}}                    \NC \getnamedglyphdirect{ZapfDingbats}{a1}                    \NC \NC \NR
549\NC \type{\getnamedglyphdirect{ZapfDingbats}{a11}}                   \NC \getnamedglyphdirect{ZapfDingbats}{a11}                   \NC \NC \NR
550\HL
551\NC \type{\definedfont[ZapfDingbats*dingbats]}                      \NC \definedfont[ZapfDingbats*dingbats]                      \NC \NC \NR
552\HL
553\stoptabulate
554
555Keep in mind that fonts like Dejavu (that we use here as document font) already
556has these characters which is why it shows up in the verbose part of the table.
557
558\stopsection
559
560\startsection[title=Protrusion]
561
562Protrusion is a feature that \LUATEX\ inherits from \PDFTEX. It is sometimes
563referred to as hanging punctuation but in our case any character qualifies. Also,
564hanging is not frozen but can be tuned in detail. Currently the engine defines
565protrusion in terms of the emwidth which is unfortunate and likely to change.
566\footnote {In general the low level implementation can be optimized as there are
567better mechanisms in \LUATEX.}
568
569It is sometimes believed that protrusion improves for instance narrower columns,
570but I'm pretty sure that this is not the case. It is true that it is taken into
571account when breaking a paragraph into lines, and that we then have a little bit
572more width available, but at the same time it is an extra constraint: if we
573protrude we have to do it for each line (and the whole main body of text) so it's
574just a different solution space. The main reason for applying this feature is
575{\em not} that the lines look better or that we get better looking narrow lines
576but that the right and left margins look nicer. Personally I don't like half
577protrusion of punctuation and hyphens. Best is to have small values for regular
578characters to improve the visual appearance and use full protrusion for hyphens
579(and maybe punctuation).
580
581\startsubsubject[title=protrusion classes]
582
583In \CONTEXT\ we've always defined protrusion as a percentage of the width of a
584glyph. From \MKII\ we inherit the level of control as well as the ability to
585define vectors. The shared properties are collected in so called classes and the
586character specific properties in vectors. The following classes are predefined:
587
588\showprotrusionclass
589
590The names are used in the definitions:
591
592\starttyping
593\definefontfeature[default][protrusion=quality]
594\stoptyping
595
596Currently adding a class only has a \LUA\ interface.
597
598\startbuffer
599\startluacode
600fonts.protrusions.classes.myown = {
601    vector = 'myown',
602    factor = 1,
603}
604\stopluacode
605\stopbuffer
606
607\typebuffer \getbuffer
608
609\stopsubsubject
610
611\startsubsubject[title=protrusion vectors]
612
613Vectors are larger but not as large as you might expect. Only a subset of
614characters needs to be defined. This is because in practice only latin scripts
615are candidates and these scripts have glyphs that look a lot like each other. As
616we only operate on the horizontal direction characters like \quote
617{aàáâãäå} look the same from the left and right so we only have to define
618the protrusion for \quote {a}.
619
620As with classes, you can define your own vectors:
621
622\startbuffer
623\startluacode
624fonts.protrusions.vectors.myown = table.merged (
625    fonts.protrusions.vectors.quality,
626    {
627        [0x002C] = { 0, 2 }, -- comma
628    }
629)
630\stopluacode
631\stopbuffer
632
633\typebuffer \getbuffer
634
635\stopsubsubject
636
637\startsubsubject[title=protrusion vector pure]
638    \showprotrusionvector[name=pure]
639\stopsubsubject
640
641\startsubsubject[title=protrusion vector punctuation]
642    \showprotrusionvector[name=punctuation]
643\stopsubsubject
644
645\startsubsubject[title=protrusion vector alpha]
646    \showprotrusionvector[name=alpha]
647\stopsubsubject
648
649\startsubsubject[title=protrusion vector quality]
650    \showprotrusionvector[name=quality]
651\stopsubsubject
652
653\startsubsubject[title=examples of protrusion]
654
655Next we show the quality protrusion. For this we use \type {tufte.tex} as
656this one for sure will result in punctuation and other candidates for
657protrusion.
658
659\startbuffer[define]
660\definefontfeature
661  [whatever]
662  [default]
663  [protrusion=quality]
664
665\definefont[MyTestA][Serif*default  at 10pt]
666\definefont[MyTestB][Serif*whatever at 10pt]
667\stopbuffer
668
669\startbuffer[example]
670\startoverlay
671    {\ruledvbox \bgroup
672        \hsize\textwidth
673        \MyTestA
674        \setupalign[normal]
675        \input{tufte}
676     \egroup}
677    {\ruledvbox \bgroup
678        \hsize\textwidth
679        \MyTestB
680        \setupalign[hanging,normal]
681        \maincolor
682        \input{tufte}
683     \egroup}
684\stopoverlay
685\stopbuffer
686
687\typebuffer[define]
688\getbuffer [define]
689
690We use the following example. The results are shown in \in {figure}
691[protrusion:quality]. The colored text is the protruding one.
692
693\typebuffer[example]
694
695\startplacefigure[reference=protrusion:quality,title=The difference between no protrusion and quality protrusion.]
696    \getbuffer [example]
697\stopplacefigure
698
699The previously defined own class and vector is somewhat more extreme:
700
701\startbuffer[define]
702\definefontfeature
703  [whatever]
704  [default]
705  [protrusion=myown]
706
707\definefont[MyTestA][Serif*default  at 10pt]
708\definefont[MyTestB][Serif*whatever at 10pt]
709\stopbuffer
710
711\typebuffer[define]
712\getbuffer [define]
713
714In \in {figure} [protrusion:myown] we see that the somewhat extreem definition of
715the comma also pulls the preceding character into the margin.
716
717\startplacefigure[reference=protrusion:myown,title=The influence of extreme protrusion on preceding characters.]
718    \getbuffer [example]
719\stopplacefigure
720
721\stopsubsubject
722
723\stopsection
724
725\startsection[title=Expansion]
726
727Expansion is also an inheritance of \PDFTEX. \footnote {As with protrusion the
728implementation in the engine is somewhat suboptimal and inefficient and will be
729upgraded to a more \LUATEX-ish way.} This mechanism selectively expands
730characters, normally upto 5\%. One reason for applying it is that we have less
731visually incompatible spacing, especially when we have underfull or cramped
732lines. For each (broken) line the badness is reconsidered with either shrink or
733stretch applied to all characters in that line. So, in the worst case a shrunken
734line is followed by a stretched one and that can be visible when the scaling
735factors are chosen wrong.
736
737As with protrusion, the solution space is larger but so are the constraints. But
738contrary to protrusion here the look and feel of the whole line can be made
739better but at the cost of much more runtime and larger (\PDF) files.
740
741\startsubsubject[title=protrusion classes]
742
743The amount of expansion depends in the shape of the character. Vertical strokes
744are more sensitive for expansion then horizontal ones. So an \quote {o} can
745get a different scaling than an \quote {m}. As with protrusion we have collected
746the properties in classes:
747
748\showexpansionclass
749
750The smaller the step, the more instances of a font we get, the better it
751looks, and the larger the files become. it's best not to use too many stretch
752and shrink steps. A stretch of 2 and shrink of 2 and step of .25 results in
753upto 8~instances plus the regular sized one.
754
755\stopsubsubject
756
757\startsubsubject[title=expansion vectors]
758
759We only have one vector: \type {quality}:
760
761\showexpansionvector[name=quality]
762
763\stopsubsubject
764
765\startsubsubject[title=an example of expansion]
766
767We use \type {zapf.tex} as example text, if only because Hermann Zapf introduced
768this optimization. Keep in mind that you can combine expansion and protrusion.
769
770\startbuffer[define]
771\definefontfeature
772  [whatever]
773  [default]
774  [expansion=quality]
775
776\definefont[MyTestA][Serif*default  at 10pt]
777\definefont[MyTestB][Serif*whatever at 10pt]
778\stopbuffer
779
780\startbuffer[example]
781\startoverlay
782    {\ruledvbox \bgroup
783        \hsize\textwidth
784        \MyTestA
785        \setupalign[normal]
786        \input{tufte}
787     \egroup}
788    {\ruledvbox \bgroup
789        \hsize\textwidth
790        \MyTestB
791        \setupalign[hz,normal]
792        \maincolor
793        \input{tufte}
794     \egroup}
795\stopoverlay
796\stopbuffer
797
798\typebuffer[define]
799\getbuffer [define]
800
801We use the following example. The results are shown in \in {figure}
802[expansion:quality]. The colored text is the protruding one.
803
804\typebuffer[example]
805
806\startplacefigure[reference=expansion:quality,title=The difference between no expansion and quality expansion.]
807    \getbuffer [example]
808\stopplacefigure
809
810You can see what happens in \in {figure} [expansion:visualized].
811
812\startbuffer[example]
813    \setupalign[hz]
814    \enabletrackers[*expansion*]
815    \definefontfeature[boundingbox][boundingbox={frame,empty}]
816    \definedfont[Serif*default,quality,boundingbox @ 12.1pt]
817    \samplefile{sapolsky}\par
818    \disabletrackers[*expansion*]
819\stopbuffer
820
821\typebuffer[example]
822
823\startplacefigure[reference=expansion:visualized,title=The injected expansion kerns.]
824    \getbuffer [example]
825\stopplacefigure
826
827\stopsubsubject
828
829\startsubsubject[title=Expansion and kerning]
830
831When we expand glyphs we also need to look at the font kerns between them. In the
832original implementation taken from \PDFTEX\ expansion was implemented using pseudo
833fonts (with expanded glyph widths) and expansion of inter|-|character kerns was
834based on font information. In \LUATEX\ we have expansion factors in glyph nodes
835instead which is more efficient and gives a cleaner separation between front- and
836backend as the backend has no need to consult the font.
837
838For the font kerns we set the kern compensation directly and for that we use the
839average expansion factors of the neighbouring fonts so technically we support
840kerns between different fonts). This also has the advantage that kerns injected
841in node mode are treated well, given that they are tagged as font kern.
842
843So what is the effect (and need) of scaling font kerns? Let's look at an example.
844Kerns can be positive but also negative:
845
846\startlinecorrection
847\startcombination
848    {\vbox {
849        \forgetall
850        \hpack to 3cm{\hss\ruledhbox{\maincolor V\kern-1ptA}\hss}
851        \hpack to 3cm{\hss\ruledhbox{\maincolor V\kern 0ptA}\hss}
852    }} {negative}
853    {\vbox {
854        \forgetall
855        \hpack to 3cm{\hss\ruledhbox{\maincolor I\kern.25ptI}\hss}
856        \hpack to 3cm{\hss\ruledhbox{\maincolor I\kern  0ptI}\hss}
857    }} {positive}
858\stopcombination
859\stoplinecorrection
860
861If we use a rediculous amount of stretch we get the following. In the top line we
862scale the kern, in the bottom line we don't.
863
864\startlinecorrection
865\startcombination
866    {\vbox {
867        \definedfont[file:texgyrepagella-regular.otf at 12pt]%
868        \forgetall
869        \hpack to 3cm{\maincolor \hss\strut \scale[xscale=5000]{V}\kern-5pt\scale[xscale=5000]{A}\hss}
870        \hpack to 3cm{\maincolor \hss\strut \scale[xscale=5000]{V}\kern-1pt\scale[xscale=5000]{A}\hss}
871    }} {negative}
872    {\vbox {
873        \definedfont[file:texgyrepagella-regular.otf at 12pt]%
874        \forgetall
875        \hpack to 3cm{\maincolor \hss\strut \scale[xscale=5000]{I}\kern1.25pt\scale[xscale=5000]{I}\hss}
876        \hpack to 3cm{\maincolor \hss\strut \scale[xscale=5000]{I}\kern0.25pt\scale[xscale=5000]{I}\hss}
877    }} {positive}
878\stopcombination
879\stoplinecorrection
880
881The reason that we mention this is that when we apply \OPENTYPE\ features,
882positioning not necessarily result in font kerns. For instance ligatures can be
883the result of careful applied kerns and in some scripts kerns are used to connect
884glyphs. This means that we best cannot expand kerns by default. How bad is
885that? By looking at the examples above one would say \quotation {real bad}.
886
887But say that we have about 1pt of font kerns, then a 5\% expansion (which is
888already a lot) amounts to 0.05pt so to \blackrule [width=1pt, height=max,
889depth=max] we add \blackrule [width=.05pt, height=max, depth=max] which is so
890little that it probably goes unnoticed. Even if we use extreme kerns, as between
891VA, in practice the small amount of stretch or shrink added to a font kern goes
892unnoticed.
893
894In \in {figure} [hz:natural] we have overlayed the different strategies. The
895sample and width is chosen such that we see something. On a display you can
896scale up these examples and inspect if there is really something to see,
897but on paper zooming in helps, as in \in {figure} [hz:zoomed]. Even then the
898effect of expanded kerns is invisible. The used definitions are:
899
900\definecolor[hz:test:tr][r=1,a=1,t=.5]
901\definecolor[hz:test:tg][g=1,a=1,t=.5]
902\definecolor[hz:test:tb][b=1,a=1,t=.5]
903
904\startbuffer
905\setupfontexpansion
906    [extremehz]
907    [stretch=5,shrink=5,step=.5,vector=default,factor=1]
908\setupfontexpansion
909    [regularhz]
910    [stretch=2,shrink=2,step=.5,vector=default,factor=1]
911\setupfontexpansion
912    [minimalhz]
913    [stretch=2,shrink=2,step=.5,vector=default,factor=.5]
914
915\definefontfeature
916    [extremehz] [default]
917    [mode=node,expansion=extremehz]
918\definefontfeature
919    [regularhz] [default]
920    [mode=node,expansion=regularhz]
921\definefontfeature [minimalhz] [default]
922    [mode=node,expansion=minimalhz]
923
924\definefont
925    [ExtremeHzFont]
926    [file:texgyrepagella-regular.otf*extremehz at 10pt]
927\definefont
928    [RegularHzFont]
929    [file:texgyrepagella-regular.otf*regularhz at 10pt]
930\definefont
931    [MinimalHzFont]
932    [file:texgyrepagella-regular.otf*minimalhz at 10pt]
933\stopbuffer
934
935\typebuffer \getbuffer
936
937\edef\HzSampleText{\cldloadfile{ward}}
938
939\def\NoHzSample  {\vbox{\hsize 10cm \color[hz:test:tr]{\setupalign  [nohz]\HzSampleText\par}}}
940\def\HzSample    {\vbox{\hsize 10cm \color[hz:test:tg]{\setupalign    [hz]\HzSampleText\par}}}
941\def\FullHzSample{\vbox{\hsize 10cm \color[hz:test:tb]{\setupalign[fullhz]\HzSampleText\par}}}
942
943\startplacefigure[reference=hz:natural,title={The two expansion methods compared.}]
944    \showfontkerns
945    \dontcomplain
946    \startcombination[1*3]
947        {\ExtremeHzFont\ruledhpack{\startoverlay {\NoHzSample} {\HzSample    } \stopoverlay}} {no hz \& hz}
948        {\ExtremeHzFont\ruledhpack{\startoverlay {\NoHzSample} {\FullHzSample} \stopoverlay}} {no hz \& full hz}
949        {\ExtremeHzFont\ruledhpack{\startoverlay {\HzSample  } {\FullHzSample} \stopoverlay}} {hz \& full hz}
950    \stopcombination
951\stopplacefigure
952
953\startplacefigure[reference=hz:zoomed,title={The two expansion methods compared (zoomed in).}]
954    \showfontkerns
955    \dontcomplain
956    \startcombination[3*3]
957
958        {\ExtremeHzFont
959         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\NoHzSample} {\HzSample    } \stopoverlay}} {extreme: no hz \& hz}
960        {\ExtremeHzFont
961         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\NoHzSample} {\FullHzSample} \stopoverlay}} {extreme: no hz \& full hz}
962        {\ExtremeHzFont
963         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\HzSample  } {\FullHzSample} \stopoverlay}} {extreme: hz \& full hz}
964
965        {\RegularHzFont
966         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\NoHzSample} {\HzSample    } \stopoverlay}} {regular: no hz \& hz}
967        {\RegularHzFont
968         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\NoHzSample} {\FullHzSample} \stopoverlay}} {regular: no hz \& full hz}
969        {\RegularHzFont
970         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\HzSample  } {\FullHzSample} \stopoverlay}} {regular: hz \& full hz}
971
972        {\MinimalHzFont
973         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\NoHzSample} {\HzSample    } \stopoverlay}} {minimal: no hz \& hz}
974        {\MinimalHzFont
975         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\NoHzSample} {\FullHzSample} \stopoverlay}} {minimal: no hz \& full hz}
976        {\MinimalHzFont
977         \clip[nx=6,ny=5,x=2,y=2,sx=2]{\startoverlay {\HzSample  } {\FullHzSample} \stopoverlay}} {minimal: hz \& full hz}
978
979    \stopcombination
980\stopplacefigure
981
982In \CONTEXT\ the \type {hz} alignment option only enables expansion of glyphs,
983while \type {fullhz} also applies it to kerns. It will be clear that you can just
984stick to using the \type {hz} directive (if you want expansion at all) because
985this directive is normally disabled and because most fonts are processed in node
986mode.
987
988\stopsubsubject
989
990\stopsection
991
992\startsection[title=Composing]
993
994This feature is seldom needed but can come in handy for old fonts or when
995some special language is to be supported. When writing this section I tested
996this feature with Dejavu and only two additional characters were added:
997
998\definefontfeature
999  [default-plus-compose]
1000  [compose=yes]
1001
1002\definefont
1003  [MyComposedSerif]
1004  [Serif*default-plus-compose]
1005
1006% we need to cheat a bit as we don't have the main character in mono
1007
1008\startlines \MyComposedSerif
1009\type{fonts > combining > }\hbox to .5em{\hss Ѷ\hss}\type{ (U+00476) = }\hbox to .5em{\hss Ѵ\hss}\type{ (U+00474) + ̏ (U+0030F)}
1010\type{fonts > combining > }\hbox to .5em{\hss ѷ\hss}\type{ (U+00477) = }\hbox to .5em{\hss ѵ\hss}\type{ (U+00475) + ̏ (U+0030F)}
1011\stoplines
1012
1013This trace showed up after giving:
1014
1015\starttyping
1016\enabletrackers
1017  [fonts.composing.define]
1018
1019\definefontfeature
1020  [default-plus-compose]
1021  [compose=yes]
1022
1023\definefont
1024  [MyFont]
1025  [Serif*default-plus-compose]
1026\stoptyping
1027
1028Fonts like Latin Modern have lots of glyphs but still lack some. Although the
1029composer can add some of the missing, some of those new virtual glyphs probably
1030will never look real good. For instance, putting additional accents on top of
1031already accented uppercase characters will fail when that character has a rather
1032tight (or even clipped) boundingbox in order not to spoil the lineheight. You can
1033get some more insight in the process by turning on tracing:
1034
1035\starttyping
1036\enabletrackers[fonts.composing.visualize]
1037\stoptyping
1038
1039One reason why composing can be suboptimal is that it uses the boundingbox of the
1040characters that are combined. If you really depend on a specific font and need
1041some of the missing characters it makes sense to spend some time on optimizing
1042the rendering. This can be done via the goodies mechanism. As an example we've
1043added \type {lm-compose-test.lfg} to the distribution. First we show how it
1044looks at the \TEX\ end:
1045
1046\startbuffer
1047\enabletrackers[fonts.composing.visualize]
1048
1049\definefontfeature
1050  [default-plus-compose]
1051  [compose=yes]
1052
1053\loadfontgoodies
1054  [lm-compose-test] % playground
1055
1056\definefont
1057  [MyComposedSerif]
1058  [file:lmroman10regular*default-plus-compose at 48pt]
1059\stopbuffer
1060
1061\typebuffer \getbuffer
1062
1063\blank
1064\backgroundline
1065  [halfcolor]
1066  {\MyComposedSerif B\quad\char"1E02\quad\char"1E04}
1067\blank
1068
1069The positions of the dot accents on top and below the capital B is defined
1070in a goodie file:
1071
1072\starttyping
1073return {
1074  name = "lm-compose-test",
1075  version = "1.00",
1076  comment = "Goodies that demonstrate composition.",
1077  author = "Hans and Mojca",
1078  copyright = "ConTeXt development team",
1079  compositions = {
1080    ["lmroman12-regular"] = compose,
1081  }
1082}
1083\stoptyping
1084
1085As this is an experimental feature there are several ways to deal with
1086this. For instance:
1087
1088\starttyping
1089local defaultfraction = 10.0
1090
1091local compose = {
1092  dy       = defaultfraction,
1093  [0x1E02] = { -- B dot above
1094      dy = 150
1095  },
1096  [0x1E04] = { -- B dot below
1097      dy = 150
1098  },
1099}
1100\stoptyping
1101
1102Here the fraction is relative to the difference between the height of the
1103accentee and the accent. A better solution is the following:
1104
1105\starttyping
1106local compose = {
1107  [0x1E02] = { -- B dot above
1108    anchored = "top",
1109  },
1110  [0x1E04] = { -- B dot below
1111    anchored = "bottom",
1112  },
1113  [0x0042] = { -- B
1114    anchors = {
1115      top = {
1116        x = 300, y = 700,
1117      },
1118      bottom = {
1119        x = 300, y = -30,
1120      },
1121    },
1122  },
1123  [0x0307] = {
1124    anchors = {
1125      top = {
1126        x = -250, y =  550,
1127      },
1128    },
1129  },
1130  [0x0323] = {
1131    anchors = {
1132      bottom = {
1133        x = -250, y =  -80,
1134      },
1135    },
1136  },
1137}
1138\stoptyping
1139
1140This approach is more or less the same as \OPENTYPE\ anchoring. It takes a bit
1141more effort to define these tables but the result is better.
1142
1143\stopsection
1144
1145\startsection[title=Kerning]
1146
1147Inter|-|character kerning is not supported at the font level and with good
1148reason. The fact that something is conceptually possible doesn't mean that we
1149should use or support it. Normally proper kerning (or the lack of it) is part
1150of a font design and for some scripts different kerning is not even an option.
1151
1152On the average \TEX\ does a proper job on justification but not all programs
1153are that capable. As a consequence designers (at least we ran into it) tend to
1154stick to flush left rendering because they don't trust their system to do a
1155proper job otherwise. On the other hand they seem to have no problem with
1156messing up the inter|-|character spacing and even combine that with excessive
1157inter|-|word spacing {\em if} they want to achieve justification (without
1158hyphenation). And it can become even worse when extreme glyph expansion (like
1159hz) is applied.
1160
1161Anyhow, it will be clear that consider messing with properties like kerning that
1162are part of the font design is to be done careful.
1163
1164\definecharacterkerning [extremekerning] [factor=.125]
1165
1166\start \setcharacterkerning[extremekerning]
1167
1168For running text additional kerning makes no sense. It not only looks
1169bad, it also spoils the grayness of a text. When it is applied we need
1170to deal with special cases. For instance ligatures make no sense so they
1171should be disabled. Additional kerning should relate to already present
1172kerning and interword spacing should be adapted accordingly. Embedded
1173non|-|characters also need to be treated well.
1174
1175\par \stop
1176
1177This paragraph was typeset as follows:
1178
1179\starttyping
1180\definecharacterkerning [extremekerning] [factor=.125]
1181
1182\setcharacterkerning[extremekerning] ... text ...
1183\stoptyping
1184
1185Where additional kerning can make sense, is in titles. The previous
1186command can do that job. In addition we have a mechanism that
1187fills a given space. This mechanism uses the following definition:
1188
1189\starttyping
1190\setupcharacterkerning
1191  [stretched]
1192  [factor=max,
1193   width=\availablehsize]
1194\stoptyping
1195
1196\startbuffer
1197\stretched{\bfd to the limit}
1198\stopbuffer
1199
1200\typebuffer
1201
1202\blank \start \color[maincolor]{\getbuffer} \stop \blank
1203
1204The following does not work:
1205
1206\startbuffer
1207\ruledhbox to 5cm{\stretched{\bfd to the limit}}
1208\stopbuffer
1209
1210\typebuffer
1211
1212\blank \start \color[maincolor]{\getbuffer} \stop \blank
1213
1214But this works ok:
1215
1216\startbuffer
1217\setupcharacterkerning
1218  [stretched]
1219  [width=]
1220
1221\stretched{\bfd to the limit}
1222\stopbuffer
1223
1224\typebuffer
1225
1226\blank \start \color[maincolor]{\getbuffer} \stop \blank
1227
1228You can also say this:
1229
1230\startbuffer
1231\stretched[width=]{\bfd to the limit}
1232\stopbuffer
1233
1234\typebuffer
1235
1236\blank \start \color[maincolor]{\getbuffer} \stop \blank
1237
1238or:
1239
1240\startbuffer
1241\ruledhbox{\stretched[width=10cm]{\bfd to the limit}}
1242\stopbuffer
1243
1244\typebuffer
1245
1246\blank \start \color[maincolor]{\getbuffer} \stop \blank
1247
1248You can get some insight in what kerning does to your font by the following
1249command:
1250
1251\startbuffer
1252\usemodule[typesetting-kerning]
1253
1254\starttext
1255  \showcharacterkerningsteps
1256    [style=Bold,
1257     sample=how to violate a proper font design,
1258     text=rubish,
1259     first=0,
1260     last=45,
1261     step=5]
1262\stoptext
1263\stopbuffer
1264
1265\typebuffer
1266
1267\blank \getbuffer \blank
1268
1269\stopsection
1270
1271\startsection[title=Extra font kerns]
1272
1273Fonts are processed independent of each other. Sometimes that is unfortunate for
1274kerning, although in practice it won't happen that often. We can enable an
1275additional kerning mechanism to deal with these cases. The \type
1276{\setextrafontkerns} command takes one argument between square brackets. The
1277effect can be seen below:
1278
1279\startbuffer
1280    VA {\smallcaps va} V{\smallcaps a}
1281    VA {\bf VA} V{\bf A} {\bf V}A
1282    V{\it A}
1283\stopbuffer
1284
1285\starttabulate[|Tl|l|p|]
1286\HL
1287\BC key \BC result \BC logic \NC \NR
1288\HL
1289\NC no kerns \NC \showfontkerns\setextrafontkerns[reset]\subff{f:kern}\inlinebuffer \NC no kerns at all \NC \NR
1290\NC kerns    \NC \showfontkerns\setextrafontkerns[reset]\inlinebuffer \NC kerns within a font (feature) run \NC \NR
1291\HL
1292\NC none     \NC \showfontkerns\setextrafontkerns [none]\inlinebuffer \NC only extra kerns within fonts \NC \NR
1293\NC min      \NC \showfontkerns\setextrafontkerns  [min]\inlinebuffer \NC minimal kerns within and across fonts \NC \NR
1294\NC max      \NC \showfontkerns\setextrafontkerns  [max]\inlinebuffer \NC maximum kerns within and across fonts \NC \NR
1295\NC mixed    \NC \showfontkerns\setextrafontkerns[mixed]\inlinebuffer \NC averaged kerns within and across fonts \NC \NR
1296\HL
1297\stoptabulate
1298
1299The content is defined as:
1300
1301\typebuffer
1302
1303This mechanism obeys grouping so you have complete control over where and when
1304it gets applied. The \type {\showfontkerns} command can be used to trace the
1305injection of (font) kerns.
1306
1307\stopsection
1308
1309\startsection[title=Ligatures]
1310
1311For some Latin fonts ligature building is quite advanced, take Unifraktur. I have no
1312problem admitting that I find fraktur hard to read, but this one actually is sort of
1313an exception. It's also a good candidate for a screen presentation where you mainly
1314made notes for yourself: no one has to read it, but it looks great, especially if
1315you consider it to be drawn by a pen.
1316
1317Anyway, we will use the following code as example (based on some remarks on the
1318fonts website).
1319
1320\startbuffer[sample]
1321sitzen / ſitzen / effe fietsen / ch ck ſt tz ſi fi
1322\stopbuffer
1323
1324\typebuffer[sample]
1325
1326Some ligatures are implemented in the usual way, using the \type {liga} and \type {dlig}
1327features, others kick in thanks to \type {ccmp}. This fact alone is an illustration that
1328the low level \OPENTYPE\ ligature feature is not related to ligatures at all but a more
1329generic mechanism: you can basically combine multiple shapes into one in all features
1330exposed to the user.
1331
1332We define a bunch of specific feature sets:
1333
1334\startbuffer
1335\definefontfeature
1336  [unifraktur-a]
1337  [default]
1338\definefontfeature
1339  [unifraktur-b]
1340  [default]
1341  [goodies=unifraktur,keepligatures=yes]
1342\definefontfeature
1343  [unifraktur-c]
1344  [default]
1345  [ccmp=yes]
1346\definefontfeature
1347  [unifraktur-d]
1348  [default]
1349  [ccmp=yes,goodies=unifraktur,keepligatures=yes]
1350\definefontfeature
1351  [unifraktur-e]
1352  [default]
1353  [liga=no,rlig=no,clig=no,dlig=no,ccmp=yes,keepligatures=auto]
1354\stopbuffer
1355
1356\getbuffer \typebuffer
1357
1358and also some fonts:
1359
1360\startbuffer
1361\definefont[TestA][UnifrakturCook*unifraktur-a sa 0.9]
1362\definefont[TestB][UnifrakturCook*unifraktur-b sa 0.9]
1363\definefont[TestC][UnifrakturCook*unifraktur-c sa 0.9]
1364\definefont[TestD][UnifrakturCook*unifraktur-d sa 0.9]
1365\definefont[TestE][UnifrakturCook*unifraktur-e sa 0.9]
1366\stopbuffer
1367
1368\getbuffer \typebuffer
1369
1370We show these five alternatives here:
1371
1372\starttabulate[|T||]
1373\NC liga                        \NC \TestA\getbuffer[sample] \NC \NR
1374\NC liga + keepligatures        \NC \TestB\getbuffer[sample] \NC \NR
1375\NC liga + ccmp                 \NC \TestC\getbuffer[sample] \NC \NR
1376\NC liga + ccmp + keepligatures \NC \TestD\getbuffer[sample] \NC \NR
1377\NC ccmp + keepligatures        \NC \TestE\getbuffer[sample] \NC \NR
1378\stoptabulate
1379
1380The real fun starts when we want to add extra spacing between characters. Some
1381ligatures need to get broken and some kept.
1382
1383\startbuffer
1384\setupcharacterkerning[kerncharacters][factor=0.5]
1385\setupcharacterkerning[letterspacing] [factor=0.5]
1386\stopbuffer
1387
1388\getbuffer \typebuffer
1389
1390\enabletrackers[typesetters.kerns.ligatures]
1391
1392Next we will see how ligatures behave depending on how the mechanisms are set
1393up. The colors indicate what trickery is used:
1394
1395\starttabulate[|T||]
1396\NC \color[darkred]  {red}   \NC kept by dynamic feature \NC \NR
1397\NC \color[darkgreen]{green} \NC kept by static feature  \NC \NR
1398\NC \color[darkblue] {blue}  \NC keep by goodie          \NC \NR
1399\stoptabulate
1400
1401First we use \type {\kerncharacters}:
1402
1403\starttabulate[|T||]
1404\NC liga                        \NC \kerncharacters {\TestA\getbuffer[sample]} \NC \NR
1405\NC liga + keepligatures        \NC \kerncharacters {\TestB\getbuffer[sample]} \NC \NR
1406\NC liga + ccmp                 \NC \kerncharacters {\TestC\getbuffer[sample]} \NC \NR
1407\NC liga + ccmp + keepligatures \NC \kerncharacters {\TestD\getbuffer[sample]} \NC \NR
1408\NC ccmp + keepligatures        \NC \kerncharacters {\TestE\getbuffer[sample]} \NC \NR
1409\stoptabulate
1410
1411In the next example we use \type {\letterspacing}:
1412
1413\starttabulate[|T||]
1414\NC liga                        \NC \letterspacing {\TestA\getbuffer[sample]} \NC \NR
1415\NC liga + keepligatures        \NC \letterspacing {\TestB\getbuffer[sample]} \NC \NR
1416\NC liga + ccmp                 \NC \letterspacing {\TestC\getbuffer[sample]} \NC \NR
1417\NC liga + ccmp + keepligatures \NC \letterspacing {\TestD\getbuffer[sample]} \NC \NR
1418\NC ccmp + keepligatures        \NC \letterspacing {\TestE\getbuffer[sample]} \NC \NR
1419\stoptabulate
1420
1421\disabletrackers[typesetters.kerns.ligatures]
1422
1423The difference is that the letterspacing variant dynamically adds the predefined
1424featureset \type {letterspacing} which is defined in a similar way as \type
1425{unifraktur-e}. In the case of this font, this variant is the better one to use.
1426In fact, this variant probably works okay with most fonts. However, by not hard
1427coding this behaviour we keep control, as one never knows what the demands are.
1428When no features are used, information from the (given) goodie file \type
1429{unifraktur.lfg} is consulted:
1430
1431\starttyping
1432letterspacing = {
1433  -- watch it: zwnj's are used (in the tounicodes too)
1434  keptligatures = {
1435    ["c_afii301_k.ccmp"] = true, -- ck
1436    ["c_afii301_h.ccmp"] = true, -- ch
1437    ["t_afii301_z.ccmp"] = true, -- tz
1438    ["uniFB05"]          = true, -- ſt
1439  },
1440}
1441\stoptyping
1442
1443These kick in when we don't disable ligatures by setting features (case~e).
1444
1445There are two pseudo features that can help us out when a font doesn't provide
1446the wanted ligatures but has the right glyphs for building them. The \UNICODE\
1447database has some information about how characters can be (de)composed and we can
1448use that information to create virtual glyphs:
1449
1450\starttyping
1451\definefontfeature
1452  [default] [default]
1453  [char-ligatures=yes,mode=node]
1454\stoptyping
1455
1456and:
1457
1458\starttyping
1459\definefontfeature
1460  [default] [default]
1461  [compat-ligatures=yes,mode=node]
1462\stoptyping
1463
1464This feature was added after some discussion on the \CONTEXT\ mailing list about
1465the following use case.
1466
1467\startbuffer
1468\definefontfeature
1469  [default-l] [default]
1470  [char-ligatures=yes,
1471   compat-ligatures=yes,
1472   mode=node]
1473
1474\definefont[LigCd][cambria*default]
1475\definefont[LigPd][texgyrepagellaregular*default]
1476\definefont[LigCl][cambria*default-l]
1477\definefont[LigPl][texgyrepagellaregular*default-l]
1478\stopbuffer
1479
1480\typebuffer \getbuffer
1481
1482These definitions result in:
1483
1484\starttabulate[|l|l|l|l|l|]
1485\NC                    \NC \type {\LigCd}     \NC \type {\LigPd}     \NC \type {\LigCl}     \NC \type {\LigPl}     \NC \NR
1486\NC \type{PEL·LÍCULES} \NC \LigCd PEL·LÍCULES \NC \LigPd PEL·LÍCULES \NC \LigCl PEL·LÍCULES \NC \LigPl PEL·LÍCULES \NC \NR
1487\NC \type{pel·lícules} \NC \LigCd pel·lícules \NC \LigPd pel·lícules \NC \LigCl pel·lícules \NC \LigPl pel·lícules \NC \NR
1488\NC \type{PEĿLÍCULES}  \NC \LigCd PEĿLÍCULES  \NC \LigPd PEĿLÍCULES  \NC \LigCl PEĿLÍCULES  \NC \LigPl PEĿLÍCULES  \NC \NR
1489\NC \type{peŀlícules}  \NC \LigCd peŀlícules  \NC \LigPd peŀlícules  \NC \LigCl peŀlícules  \NC \LigPl peŀlícules  \NC \NR
1490\stoptabulate
1491
1492Of course one can wonder is this is the right approach and if it's not better to
1493use a font that provides the needed characters in the first place.
1494
1495\stopsection
1496
1497\startsection[title=New features]
1498
1499\startsubsection[title=Substitution]
1500
1501It is possible to add new features via \LUA. Here is an example of a single
1502substitution:
1503
1504\startbuffer
1505\startluacode
1506    fonts.handlers.otf.addfeature {
1507        name = "stest",
1508        type = "substitution",
1509        data = {
1510            a = "X",
1511            b = "P",
1512        }
1513    }
1514\stopluacode
1515\stopbuffer
1516
1517\typebuffer \getbuffer
1518
1519We show an overview at the end of this section, but here is a simple example
1520already. You need to define the feature before defining a font because otherwise
1521the font will not know about it.
1522
1523\startbuffer
1524\definefontfeature[stest][stest=yes]
1525\definedfont[file:dejavu-serifbold.ttf*default]
1526abracadabra: \addff{stest}abracadabra
1527\stopbuffer
1528
1529\typebuffer \start \blank \maincolor \getbuffer \blank \stop
1530
1531Instead of (more readable) glyph names you can also give \UNICODE\ numbers:
1532
1533\starttyping
1534\startluacode
1535    fonts.handlers.otf.addfeature {
1536        name = "stest",
1537        type = "substitution",
1538        data = {
1539            [0x61] = 0x58
1540            [0x62] = 0x50
1541        }
1542    }
1543\stopluacode
1544\stoptyping
1545
1546The definition is quite simple: we just map glyph names (or unicodes) onto
1547other ones. An alternate is also possible:
1548
1549\startbuffer
1550\startluacode
1551    fonts.handlers.otf.addfeature {
1552        name = "atest",
1553        type = "alternate",
1554        data = {
1555            a = { "X", "Y" },
1556            b = { "P", "Q" },
1557        }
1558    }
1559\stopluacode
1560\stopbuffer
1561
1562\typebuffer \getbuffer
1563
1564Less useful is a multiple substitution. Normally this one is part of a chain of
1565replacements.
1566
1567\startbuffer
1568\startluacode
1569    fonts.handlers.otf.addfeature {
1570        name = "mtest",
1571        type = "multiple",
1572        data = {
1573            a = { "X", "Y" },
1574            b = { "P", "Q" },
1575        }
1576    }
1577\stopluacode
1578\stopbuffer
1579
1580\typebuffer \getbuffer
1581
1582A ligature (or multiple to one) is also possible but normally only makes sense when
1583there is indeed a ligature. We use a similar definition for mapping the \TEX\ input
1584sequence \type {---} onto an \emdash.
1585
1586\startbuffer
1587\startluacode
1588    fonts.handlers.otf.addfeature {
1589        name = "ltest",
1590        type = "ligature",
1591        data = {
1592            ['1'] = { "a", "b" },
1593            ['2'] = { "d", "a" },
1594        }
1595    }
1596\stopluacode
1597\stopbuffer
1598
1599\typebuffer \getbuffer
1600
1601\stopsubsection
1602
1603\startsubsection[title=Positioning]
1604
1605You can define a kern feature too but when doing so you need to use measures in
1606font units.
1607
1608\startbuffer
1609\startluacode
1610    fonts.handlers.otf.addfeature {
1611        name = "ktest",
1612        type = "kern",
1613        data = {
1614            a = { b = -500 },
1615        }
1616    }
1617\stopluacode
1618\stopbuffer
1619
1620\typebuffer \getbuffer
1621
1622Pairwise positioning is more complex and involves two (optional) arrays
1623that specify \type {{dx dy wd ht}} for each of the two glyphs. In the next
1624example we only displace the second glyph.
1625
1626\startbuffer
1627\startluacode
1628    fonts.handlers.otf.addfeature {
1629        name = "ptest",
1630        type = "pair",
1631        data = {
1632            ["a"] = { ["b"] = { false,  { -1000, 1200, 0, 0 } } },
1633        }
1634    }
1635\stopluacode
1636\stopbuffer
1637
1638\typebuffer \getbuffer
1639
1640Of course you need to know a bit about the metrics of the glyphs involved so in
1641practice this boils down to trial and error.
1642
1643A single character (glyph) can also be tweaked, although normally this is done
1644better in a manipulator when loading the font. Anyway:
1645
1646\startbuffer
1647\startluacode
1648    fonts.handlers.otf.addfeature {
1649        name = "stest",
1650        type = "single",
1651        data = {
1652            a = { -30, 0, -50, 0 },
1653        }
1654    }
1655\stopluacode
1656\stopbuffer
1657
1658\typebuffer \getbuffer
1659
1660This will reduce the left and right edges and make the glyph a pretty tight one. The
1661values are for Latin Modern.
1662
1663\stopsubsection
1664
1665\startsubsection[title=Examples]
1666
1667We didn't show usage yet. This is because we need to define a feature before we
1668define a font. New features will be added to a font when it gets defined.
1669
1670\startbuffer
1671\definefontfeature[stest][stest=yes]
1672\definefontfeature[atest][atest=2]
1673\definefontfeature[mtest][mtest=yes]
1674\definefontfeature[ltest][ltest=yes]
1675\definefontfeature[ktest][ktest=yes]
1676\definefontfeature[ptest][ptest=yes]
1677\definefontfeature[ctest][ctest=yes]
1678
1679\definedfont[file:dejavu-serif.ttf*default]
1680
1681\starttabulate[|l|l|l|]
1682\NC operation    \NC feature       \NC              abracadabra \NC \NR
1683\HL
1684\NC substitution \NC \type {stest} \NC \addff{stest}abracadabra \NC \NR
1685\NC alternate    \NC \type {atest} \NC \addff{atest}abracadabra \NC \NR
1686\NC multiple     \NC \type {mtest} \NC \addff{mtest}abracadabra \NC \NR
1687\NC ligature     \NC \type {ltest} \NC \addff{ltest}abracadabra \NC \NR
1688\NC kern         \NC \type {ktest} \NC \addff{ktest}abracadabra \NC \NR
1689\NC pair         \NC \type {ptest} \NC \addff{ptest}abracadabra \NC \NR
1690\NC chain sub    \NC \type {ctest} \NC \addff{ctest}abracadabra \NC \NR
1691\stoptabulate
1692\stopbuffer
1693
1694\typebuffer \getbuffer
1695
1696\stopsubsection
1697
1698\startsubsection[title=Contexts]
1699
1700A more complex substitution is the following:
1701
1702\startbuffer
1703\startluacode
1704    fonts.handlers.otf.addfeature {
1705        name    = "ytest",
1706        type    = "chainsubstitution",
1707        lookups = {
1708            {
1709                type = "substitution",
1710                data = {
1711                    ["b"] = "B",
1712                    ["c"] = "C",
1713                },
1714            },
1715        },
1716        data = {
1717            rules = {
1718                {
1719                    before  = { { "a" } },
1720                    current = { { "b", "c" } },
1721                    lookups = { 1 },
1722                },
1723            },
1724        },
1725    }
1726\stopluacode
1727\stopbuffer
1728
1729\typebuffer \getbuffer
1730
1731Here the dataset is a sequence of rules. There can be a \type {before}, \type
1732{current} and \type {after} match. The replacements are specified with the \type
1733{lookups} entry and the numbers are indices in the provided \type {lookups}
1734table.
1735
1736Here is another example. This one demonstrates that one can check against spaces
1737(some fonts kerns against them) and against boundaries as well. The later is
1738something \CONTEXT\ specific. First we define a feature that create ligatures but
1739only when we touch a space:
1740
1741\startbuffer
1742\startluacode
1743    fonts.handlers.otf.addfeature {
1744        name    = "test-a",
1745        type    = "chainsubstitution",
1746        lookups = {
1747            {
1748                type = "ligature",
1749                data = {
1750                    ['1'] = { "a", "b" },
1751                    ['2'] = { "c", "d" },
1752                },
1753            },
1754        },
1755        data = {
1756            rules = {
1757                {
1758                    before  = { { " " } },
1759                    current = { { "a" }, { "b" } },
1760                    lookups = { 1 },
1761                },
1762                {
1763                    current = { { "c" }, { "d" } },
1764                    after   = { { " " } },
1765                    lookups = { 1 },
1766                },
1767            },
1768        },
1769    }
1770\stopluacode
1771\stopbuffer
1772
1773\typebuffer \getbuffer
1774
1775The next example also checks against whatever boundary we have.
1776
1777\startbuffer
1778\startluacode
1779    fonts.handlers.otf.addfeature {
1780        name    = "test-b",
1781        type    = "chainsubstitution",
1782        lookups = {
1783            {
1784                type = "ligature",
1785                data = {
1786                    ['1'] = { "a", "b" },
1787                    ['2'] = { "c", "d" },
1788                },
1789            },
1790        },
1791        data = {
1792            rules = {
1793                {
1794                    before  = { { " ", 0xFFFC } },
1795                    current = { { "a" }, { "b" } },
1796                    lookups = { 1 },
1797                },
1798                {
1799                    current = { { "c" }, { "d" } },
1800                    after   = { { 0xFFFC, " " } },
1801                    lookups = { 1 },
1802                },
1803            },
1804        },
1805    }
1806\stopluacode
1807\stopbuffer
1808
1809\typebuffer \getbuffer
1810
1811We can actually simplify this one to:
1812
1813\startbuffer
1814\startluacode
1815    fonts.handlers.otf.addfeature {
1816        name    = "test-c",
1817        type    = "chainsubstitution",
1818        lookups = {
1819            {
1820                type = "ligature",
1821                data = {
1822                    ['1'] = { "a", "b" },
1823                    ['2'] = { "c", "d" },
1824                },
1825            },
1826        },
1827        data = {
1828            rules = {
1829                {
1830                    before  = { { 0xFFFC } },
1831                    current = { { "a" }, { "b" } },
1832                    lookups = { 1 },
1833                },
1834                {
1835                    current = { { "c" }, { "d" } },
1836                    after   = { { 0xFFFC } },
1837                    lookups = { 1 },
1838                },
1839            },
1840        },
1841    }
1842\stopluacode
1843\stopbuffer
1844
1845\typebuffer \getbuffer
1846
1847As a bonus we show how to do more complex things:
1848
1849\startbuffer
1850\startluacode
1851    fonts.handlers.otf.addfeature {
1852        name    = "test-d",
1853        type    = "chainsubstitution",
1854        lookups = {
1855            {
1856                type = "substitution",
1857                data = {
1858                    ["a"] = "A",
1859                    ["b"] = "B",
1860                    ["c"] = "C",
1861                    ["d"] = "D",
1862                },
1863            },
1864            {
1865                type = "ligature",
1866                data = {
1867                    ['1'] = { "a", "b" },
1868                    ['2'] = { "c", "d" },
1869                },
1870            },
1871        },
1872        data = {
1873            rules = {
1874                {
1875                    before  = { { 0xFFFC } },
1876                    current = { { "a" }, { "b" } },
1877                    lookups = { 2 },
1878                },
1879                {
1880                    current = { { "c" }, { "d" } },
1881                    after   = { { 0xFFFC } },
1882                    lookups = { 2 },
1883                },
1884                {
1885                    current = { { "a" } },
1886                    after   = { { "b" } },
1887                    lookups = { 1 },
1888                },
1889                {
1890                    current = { { "c" } },
1891                    after   = { { "d" } },
1892                    lookups = { 1 },
1893                },
1894            },
1895        },
1896    }
1897\stopluacode
1898\stopbuffer
1899
1900\typebuffer \getbuffer
1901
1902\definefontfeature[test-a][test-a=yes]
1903\definefontfeature[test-b][test-b=yes]
1904\definefontfeature[test-c][test-c=yes]
1905\definefontfeature[test-d][test-d=yes]
1906
1907\startbuffer
1908abababcdcd abababcdcd abababcdcd
1909\stopbuffer
1910
1911With the test text:
1912
1913\typebuffer
1914
1915These four result in:
1916
1917\blank \start
1918
1919    \definedfont[file:dejavu-serif.ttf*default]
1920
1921    \start \addff{test-a} \getbuffer \stop\par
1922    \start \addff{test-b} \getbuffer \stop\par
1923    \start \addff{test-c} \getbuffer \stop\par
1924    \start \addff{test-d} \getbuffer \stop\par
1925
1926\stop \blank
1927
1928\stopsubsection
1929
1930\startsubsection[title={Language dependencies}]
1931
1932When \OPENTYPE\ was not around we only had to deal with ligatures, smallcaps and
1933oldstyle and of course kerns. Their number was so small that the term \quote
1934{features} was not even used. In practice one just loaded a font that had
1935oldstyle or smallcaps or none of that and was done. There were different fonts and
1936sold separately.
1937
1938In \OPENTYPE\ we have more variation and although these fonts can be much more
1939advanced the lack of standardization (for instance what gets initialized, or what
1940shapes are in the default slots) can lead to messy setups. Some fonts bind
1941features to scripts, some don't, which means that:
1942
1943\starttyping
1944\definefontfeature[smallcaps][smcp=yes,script=dflt]
1945\definefontfeature[smallcaps][smcp=yes,script=latn]
1946\definefontfeature[smallcaps][smcp=yes,script=cyrl]
1947\stoptyping
1948
1949are in fact different and you don't know in advance if you need to specify \type
1950{dflt} or \type {latn}. In practice for a feature like smallcaps there is no
1951difference between languages, but for ligatures there can be.
1952
1953When we extend an existing feature we can think of:
1954
1955\starttyping
1956\definefontfeature[smallcaps][default][smcp=yes,script=auto]
1957\definefontfeature[smallcaps][default][smcp=yes,script=*]
1958\stoptyping
1959
1960but that can have side effects too (for instance disabling language specific
1961features). The easiest way to explore this language dependency is to make
1962a feature of our own.
1963
1964\startbuffer
1965\startluacode
1966fonts.handlers.otf.addfeature {
1967    name     = "simplify",
1968    type     = "multiple",
1969    prepend  = true,
1970    features = {
1971        ["*"] = {
1972            ["deu"] = true
1973        }
1974    },
1975    data     = {
1976        [utf.byte("ä")] = { "a", "e" },
1977        [utf.byte("Ä")] = { "A", "E" },
1978        [utf.byte("ü")] = { "u", "e" },
1979        [utf.byte("Ü")] = { "U", "E" },
1980        [utf.byte("ö")] = { "o", "e" },
1981        [utf.byte("Ö")] = { "O", "E" },
1982        [utf.byte("ß")] = { "s", "z" },
1983        [utf.byte("")] = { "S", "Z" },
1984    },
1985}
1986\stopluacode
1987\stopbuffer
1988
1989\typebuffer \getbuffer
1990
1991Here we implement a language specific feature that we use at the \TEX\ end:
1992
1993\startbuffer
1994\definefontfeature
1995  [simplify-de]
1996  [simplify=yes,
1997   language=deu]
1998\stopbuffer
1999
2000\typebuffer \getbuffer
2001
2002that we can use as:
2003
2004\startbuffer
2005\definedfont[Serif*default,simplify-de]%
2006äüöß
2007{\de äüöß}
2008{\nl äüöß}
2009\stopbuffer
2010
2011\typebuffer
2012
2013and get: \start \maincolor \inlinebuffer \stop, but as you see, both German and
2014Dutch get the same treatment, which might not be what you want, because in Dutch
2015the diearesis has a different meaning.
2016
2017\startbuffer
2018\definedfont[Serif*default]%
2019                       äüöß
2020{\de\addff{simplify-de}äüöß}
2021{\nl                   äüöß}
2022\stopbuffer
2023
2024\typebuffer
2025
2026The above is restricts the usage so now we get: \start \maincolor \inlinebuffer
2027\stop, which is more language bound. You don't need much imagination for
2028extending this:
2029
2030\startbuffer
2031\definefontfeature
2032  [simplify]
2033  [simplify=yes,
2034   language=deu]
2035\stopbuffer
2036
2037\typebuffer \getbuffer
2038
2039\startbuffer
2040\definedfont[Serif*default]%
2041                    äüöß
2042{\de\addff{simplify}äüöß}
2043{\nl\addff{simplify}äüöß}
2044\stopbuffer
2045
2046So what do we expect with the next?
2047
2048\typebuffer
2049
2050We get: \start \maincolor \inlinebuffer \stop, and we see that the language
2051setting is not taken into account! This is because the font already has been set
2052up with a script and language combination. The solution is to temporary set the
2053font related language explicitly:
2054
2055\definefontfeature
2056  [simplify]
2057  [simplify=yes]
2058
2059\startbuffer
2060\definedfont[Serif*default]%
2061                                  äüöß
2062{\de\addfflanguage\addff{simplify}äüöß}
2063{\nl\addfflanguage\addff{simplify}äüöß}
2064\stopbuffer
2065
2066\typebuffer
2067
2068So we can automatically switch to language specific features if we want to:
2069\start \maincolor \inlinebuffer \stop.
2070
2071Let's now move to another level of complexity: support for more than one language
2072as in fact this example was made for Dutch in the first place, but the German
2073outcome is a bit more visible.
2074
2075\startbuffer
2076\startluacode
2077fonts.handlers.otf.addfeature {
2078    name     = "simplify",
2079    type     = "multiple",
2080    prepend  = true,
2081 -- prepend  = "smcp",
2082    dataset  =
2083    {
2084        {
2085            features = {
2086                ["*"] = {
2087                    ["nld"] = true
2088                }
2089            },
2090            data     = {
2091             -- [utf.byte("ä")] = { "a" },
2092             -- [utf.byte("Ä")] = { "A" },
2093             -- [utf.byte("ü")] = { "u" },
2094             -- [utf.byte("Ü")] = { "U" },
2095             -- [utf.byte("ö")] = { "o" },
2096             -- [utf.byte("Ö")] = { "O" },
2097                [utf.byte("ij")] = { "i", "j" },
2098                [utf.byte("IJ")] = { "I", "J" },
2099                [utf.byte("æ")] = { "a", "e" },
2100                [utf.byte("Æ")] = { "A", "E" },
2101            },
2102        },
2103        {
2104         -- type     = "multiple", -- local values possible
2105            features = {
2106                ["*"] = {
2107                    ["deu"] = true
2108                }
2109            },
2110            data     = {
2111                [utf.byte("ä")] = { "a", "e" },
2112                [utf.byte("Ä")] = { "A", "E" },
2113                [utf.byte("ü")] = { "u", "e" },
2114                [utf.byte("Ü")] = { "U", "E" },
2115                [utf.byte("ö")] = { "o", "e" },
2116                [utf.byte("Ö")] = { "O", "E" },
2117                [utf.byte("ß")] = { "s", "z" },
2118                [utf.byte("")] = { "S", "Z" },
2119            },
2120        }
2121    }
2122}
2123\stopluacode
2124\stopbuffer
2125
2126\typebuffer \getbuffer
2127
2128For this we use the following example:
2129
2130\startbuffer
2131\definedfont[Serif*default,simplify]%
2132                   äüöß ijæ
2133{\de\addfflanguage äüöß ijæ}
2134{\nl\addfflanguage äüöß ijæ}
2135\stopbuffer
2136
2137\typebuffer
2138
2139Because the Dutch is hard to check we use an \type {æ} replacement too and
2140commented the similarities with German: \start \maincolor \inlinebuffer \stop.
2141But still we're not done, say that we want smallcaps too:
2142
2143\startbuffer
2144\definefontfeature[alwayssmcp][smcp=always]%
2145\definedfont[Serif*default,simplify,alwayssmcp]%
2146                   äüöß ijæ
2147{\de\addfflanguage äüöß ijæ}
2148{\nl\addfflanguage äüöß ijæ}
2149\stopbuffer
2150
2151\typebuffer
2152
2153This comes out as: \start \maincolor \inlinebuffer \stop.
2154
2155The reason for specifying \type{smcp} as \type {always} is that otherwise we
2156get language specific smallcaps while often they are not bound to a language
2157but to the defaults. The good news is that we can do this automatically:
2158
2159\startbuffer
2160\setupfonts[language=auto]%
2161\definefontfeature[alwayssmcp][smcp=always]%
2162\definedfont[Serif*default,simplify,alwayssmcp]%
2163     äüöß ijæ
2164{\de äüöß ijæ}
2165{\nl äüöß ijæ}
2166\stopbuffer
2167
2168\typebuffer
2169
2170But be aware that this applies to all situations. Here we get: \start \maincolor
2171\inlinebuffer \stop.
2172
2173\stopsubsection
2174
2175\startsubsection[title=Syntax summary]
2176
2177In the examples we have seen several ways to define features. One of the
2178differences is that you either set a \type {data} field directly, or that you
2179specify a dataset. The fields in a dataset entry overload the ones given at the
2180top level or when not set the top level value will be taken. There is a bit
2181of (downward compatibility) tolerance built in, but best not depend on that.
2182
2183\starttyping
2184fonts.handlers.otf.addfeature {
2185    name     = "demo",
2186    features = {
2187        [<script>] = {
2188            [<language>] = true
2189        }
2190    },
2191    prepend  = true | featurename | position,
2192    dataset  = {
2193        {
2194            type = "substitution",
2195            data = {
2196                [<char|code>] = <char|code>,
2197            }
2198        },
2199        {
2200            type = "alternate",
2201            data = {
2202                [<char|code>] = { <char|code>, <char|code>, ... },
2203            }
2204        },
2205        {
2206            type = "multiple",
2207            data = {
2208                [<char|code>] = { <char|code>, <char|code>, ... },
2209            }
2210        },
2211        {
2212            type = "ligature",
2213            data = {
2214                [<char|code>] = { <char|code>, <char|code>, ... },
2215            }
2216        },
2217        {
2218            type = "kern",
2219            data = {
2220                [<char|code>] = { [<char|code>] = <value> },
2221            }
2222        },
2223        {
2224            type = "pair",
2225            data = {
2226                [<char|code>] = {
2227                    [<char|code>] = {
2228                        false | { <value>, <value>, <value>, <value> },
2229                        false | { <value>, <value>, <value>, <value> }
2230                    }
2231                }
2232            }
2233        },
2234        {
2235            type    = "chainsubstitution",
2236            lookups = {
2237                {
2238                    type = <typename>,
2239                    data = <mapping>,
2240                },
2241            },
2242            data = {
2243                rules = {
2244                    {
2245                        before  = { { [<char|code>], ... } },
2246                        current = { { [<char|code>], ... } },
2247                        after   = { { [<char|code>], ... } },
2248                        lookups = { <index>, ... },
2249                    },
2250                },
2251            },
2252        },
2253    },
2254}
2255\stoptyping
2256
2257\stopsubsection
2258
2259\startsubsection[title=Extra characters]
2260
2261\startbuffer[hyphenchars]
2262\startluacode
2263
2264    local privateslots = fonts.constructors.privateslots
2265
2266    local function addspecialhyphen(tfmdata)
2267
2268        local exheight = tfmdata.parameters.xheight
2269        local emwidth  = tfmdata.parameters.quad
2270        local width    = emwidth  /  4
2271        local height   = exheight / 10
2272        local depth    = exheight /  2
2273        local offset   = emwidth  /  6
2274
2275        tfmdata.characters[privateslots.righthyphenchar] = {
2276            -- no dimensions
2277            commands = {
2278
2279                { "right", offset },
2280
2281                { "push" },
2282                { "right", -width },
2283                { "down", depth },
2284                { "rule", height, width },
2285                { "pop" },
2286
2287                { "right", -width/5 },
2288                { "down", depth + height },
2289                { "rule", 3*height, width/5 },
2290
2291            }
2292        }
2293
2294        tfmdata.characters[privateslots.lefthyphenchar] = {
2295            -- no dimensions
2296            commands = {
2297
2298                { "right", -offset },
2299
2300                { "push" },
2301                { "down", depth + height },
2302                { "rule", 3*height, width/5 },
2303                { "pop" },
2304
2305                { "down", depth },
2306                { "rule", height, width },
2307
2308            }
2309        }
2310
2311    end
2312
2313    fonts.constructors.features.otf.register {
2314        name        = "specialhyphen",
2315        description = "special hyphen",
2316        manipulators = {
2317            base = addspecialhyphen,
2318            node = addspecialhyphen,
2319        }
2320    }
2321
2322\stopluacode
2323\stopbuffer
2324
2325You can add virtual characters to fonts. Here we give an example that is derived
2326from an example posted on the mailing list. By default, when we hyphenated a word,
2327we get this:
2328
2329\definefont[DemoFont] [Serif*default]
2330
2331\blank \start \DemoFont \maincolor \hsize 1mm averylongword \par \stop \blank
2332
2333The default character that is appended at the end and beginning of a line
2334can be specified as follows:
2335
2336\startbuffer
2337\setuplanguage
2338  [en]
2339  [righthyphenchar=45,
2340   lefthyphenchar=45]
2341\stopbuffer
2342
2343\typebuffer
2344
2345So now we get:
2346
2347\blank \start \getbuffer \DemoFont \maincolor \hsize 1mm averylongword \par \stop \blank
2348
2349Say that we want a different signal, for instance some rule. Here is how that can
2350be done:
2351
2352\typebuffer[hyphenchars]
2353
2354\getbuffer[hyphenchars]
2355
2356Watch the way we use private slots. You can best use a unique glyph name as these
2357numbers are shared between fonts. With:
2358
2359\startbuffer
2360\definefontfeature
2361  [default]
2362  [default]
2363  [specialhyphen=yes]
2364\definefont
2365  [DemoFont]
2366  [Serif*default at 24pt]
2367\setuplanguage
2368  [en]
2369  [righthyphenchar=\getprivateglyphslot{righthyphenchar},
2370   lefthyphenchar=\getprivateglyphslot{lefthyphenchar}]
2371\stopbuffer
2372
2373\typebuffer
2374
2375We get:
2376
2377\startlinecorrection[blank]
2378\getbuffer
2379\framed
2380  [foregroundstyle=\DemoFont \setupinterlinespace,
2381   offset=none,
2382   frame=no,
2383   width=1mm,
2384   align={flushleft}]
2385  {\hsize 1mm \maincolor averylongword\par}
2386\stoplinecorrection
2387
2388You need to keep in mind that some of these settings are global but in practice that is
2389not a real problem. Here is how you reset:
2390
2391\startbuffer
2392\definefontfeature
2393  [default]
2394  [default]
2395  [specialhyphen=no]
2396\setuplanguage
2397  [en]
2398  [righthyphenchar=45,
2399   lefthyphenchar=0]
2400\stopbuffer
2401
2402\typebuffer \getbuffer
2403
2404\stopsubsection
2405
2406\startsubsection[title=Goodies]
2407
2408The examples above extend a font in the \TEX\ document (normally a style) but you
2409can use a goodies file too, for instance \type {cambria.lfg}.
2410
2411\starttyping
2412return {
2413    name = "cambria",
2414    version = "1.00",
2415    comment = "Goodies that complement cambria.",
2416    author = "Hans Hagen",
2417    copyright = "ConTeXt development team",
2418    extensions = {
2419        {
2420            name = "kern", -- adds to kerns
2421            type = "pair",
2422            data = {
2423                [0x0153] = { -- combining acute
2424                    [0x0301] = { -- aeligature
2425                        false,
2426                        { -500, 0, 0, 0 }
2427                    }
2428                },
2429            }
2430        }
2431    }
2432}
2433\stoptyping
2434
2435Here we use the feature name \type {kern} and therefore we don't have to define a
2436specific (new) feature for it. Such a goodie is then used as follows:
2437
2438\starttyping
2439\definefontsynonym
2440  [Serif]
2441  [cambria]
2442  [features=default,
2443   goodies=cambria]
2444\stoptyping
2445
2446You can find such definitions in the \type {type-imp-*.mkiv} files.
2447
2448\stopsubsection
2449
2450\stopsection
2451
2452\startsection[title=Spacing]
2453
2454% By default the font loader deduces the spacing from the space character or
2455% other font properties. You can influence this by the \type {space} feature.
2456%
2457% \starttyping
2458% \definefontfeature
2459%   [korean]
2460%   [default]
2461%   [script=hang,
2462%    language=kor,
2463%    space=local] % or locl
2464% \stoptyping
2465%
2466% Instead of the usual \type {yes} (which means: use character 32), \type {local}
2467% or \type {locl} (which means: use a replacement provided by the \type{locl}
2468% feature), you can also pass a character, so
2469%
2470% \starttyping
2471% \definefontfeature
2472%   [spacy]
2473%   [default]
2474%   [space=A]
2475% \stoptyping
2476%
2477% is valid.
2478
2479As you probably know, \TEX\ has no space character. When the input is read,
2480characters tagged as space are intercepted and become glue. Compare this:
2481
2482\startlinecorrection[blank]
2483    \startcombination
2484        {\framed
2485            [width=3cm,height=15mm,align={middle,lohi},foregroundcolor=maincolor]
2486            {\dorecurse{5}{test }}}
2487        {\type{text test...}}
2488        {\framed
2489            [width=3cm,height=15mm,align={middle,lohi},foregroundcolor=maincolor]
2490            {\dorecurse{5}{test\char32\relax}}}
2491        {\type{text\char32test...}}
2492    \stopcombination
2493\stoplinecorrection
2494
2495Most fonts have a space character and you can actually use it and indeed a space
2496character will be injected but as it is not glue, the line break algorithm will
2497not see it as space.
2498
2499Al the magic done with space characters other than the native space character
2500(decimal 32) are at some point translated into glue.
2501
2502\starttabulate[||T|p|]
2503\NC \bf command \NC \UNICODE \NC width \NC \NR
2504
2505\NC \type{\nobreakspace}
2506    \type{\nbsp}                     \NC U+00A0 \NC space           \NC \NR
2507\NC \type{\ideographicspace}         \NC U+2000 \NC quad/2          \NC \NR
2508\NC \type{\ideographichalffillspace} \NC U+2001 \NC quad            \NC \NR
2509\NC \type{\twoperemspace}
2510    \type{\enspace}                  \NC U+2002 \NC quad/2          \NC \NR
2511\NC \type{\emspace}
2512    \type{\quad}                     \NC U+2003 \NC quad            \NC \NR
2513\NC \type{\threeperemspace}          \NC U+2004 \NC quad/3          \NC \NR
2514\NC \type{\fourperemspace}           \NC U+2005 \NC quad/4          \NC \NR
2515\NC \type{\fiveperemspace}           \NC        \NC quad/5          \NC \NR
2516\NC \type{\sixperemspace}            \NC U+2006 \NC quad/6          \NC \NR
2517\NC \type{\figurespace}              \NC U+2007 \NC width of zero   \NC \NR
2518\NC \type{\punctuationspace}         \NC U+2008 \NC width of period \NC \NR
2519\NC \type{\breakablethinspace}       \NC U+2009 \NC quad/8          \NC \NR
2520\NC \type{\hairspace}                \NC U+200A \NC quad/8          \NC \NR
2521\NC \type{\zerowidthspace}           \NC U+200B \NC 0               \NC \NR
2522\NC \type{\zerowidthnonjoiner}
2523    \type{\zwnj}                     \NC U+200C \NC 0               \NC \NR
2524\NC \type{\zerowidthjoiner}
2525    \type{\zwj}                      \NC U+200D \NC 0               \NC \NR
2526\NC \type{\narrownobreakspace}       \NC U+202F \NC quad/8          \NC \NR
2527\NC \type{\zerowidthnobreakspace}    \NC U+FEFF \NC                 \NC \NR
2528\NC \type{\optionalspace}            \NC        \NC space when not followed by punctuation \NC \NR
2529\stoptabulate
2530
2531% "205F  % space/8 (math)
2532
2533The last one is not un \UNICODE\ and the fifths of an emspace is not in \UNICODE\
2534either. This emspace (or quad in \TEX\ speak) is a font property. The width of
2535the space used by \CONTEXT\ is dreived form this value. In case of a monospace
2536fonts, the following logic is applied:
2537
2538\startitemize
2539    \startitem
2540        When there is a space character, the width of that character is used.
2541    \stopitem
2542    \startitem
2543        Otherwise, when there is an emdash present, the width if that character
2544        is used.
2545    \stopitem
2546    \startitem
2547        Otherwise, when there is an \type {charwidth} property available (the
2548        average width), that valua is used.
2549    \stopitem
2550\stopitemize
2551
2552When a proportional font is used, we do as follows:
2553
2554\startitemize
2555    \startitem
2556        When there is a space character, the width of that character is used.
2557    \stopitem
2558    \startitem
2559        Otherwise, when there is an emdash present, the width of that character
2560        divided by two is used.
2561    \stopitem
2562    \startitem
2563        Otherwise, when there is an \type {charwidth} property available (the
2564        average width), that value is used.
2565    \stopitem
2566\stopitemize
2567
2568In both cases, when no value is set we use the units of the font (often 1000 or
25692048). In \TEX\ a space glue also has stretch and shrink. Here we follow the
2570traditional \TEX\ logic:
2571
2572\startitemize
2573    \startitem
2574        The stretch is set to half the width of a space but to zero with a mono
2575        spaced font.
2576    \stopitem
2577    \startitem
2578        The shrink is set to one third of the width of a space but to zero with a
2579        mono spaced font.
2580    \stopitem
2581\stopitemize
2582
2583The xheight is set to the values specified by the font and when this is unset the
2584height of the character \type {x} will be used but when this character is not in
2585the font, we use two fifths of the font's units (normally the same as the
2586emwidth). The italic angle is also taken from the font (and is of course zero for
2587a not italic font). Most fonts have these properties set so we seldom have to
2588fall back to a guess.
2589
2590\stopsection
2591
2592\startsection[title=Ligatures]
2593
2594Not all fonts provide ligature control (normally related to languages), so here is a
2595trick.
2596
2597\starttyping
2598\blockligatures[fi,ff]
2599\blockligatures[fl]
2600
2601\definefontfeature
2602  [default]
2603  [default]
2604  [blockligatures=yes]
2605
2606\setupbodyfont[pagella]
2607
2608...
2609\stoptyping
2610
2611This way it works globally. Of course you can also bind it to a font instance:
2612
2613\startbuffer
2614\blockligatures[fi,fl]
2615
2616\definefontfeature
2617  [default:blockligs]
2618  [default]
2619  [blockligatures=yes]
2620
2621\definefont[DemoBlockY][Serif*default:blockligs at 20pt]
2622\definefont[DemoBlockN][Serif*default           at 20pt]
2623
2624Here we have no ligatures: {\DemoBlockY fi ff fl}, while here we get
2625them: {\DemoBlockN fi ff fl}. Of course it also depends on the font.
2626\stopbuffer
2627
2628\typebuffer \start \showfontkerns \getbuffer \par \stop
2629
2630There is one limitation: you need to specify the blocked ligatures before a font
2631gets defined and because we share resources it even has to happen before the
2632first font gets loaded. So, the \type {\blockligatures} commands go before
2633setting up the body font. This is no real problem because it's a hack anyway.
2634
2635The next example combines several tricks:
2636
2637\startbuffer[definitions]
2638\startluacode
2639    fonts.handlers.otf.addfeature {
2640        name = "kernligatures",
2641        type = "kern",
2642        data = {
2643            f = { i = 50, l = 50 },
2644        }
2645    }
2646\stopluacode
2647
2648\blockligatures[u:fl:a]
2649
2650\definefontfeature[default:b][default][blockligatures=yes]
2651\definefontfeature[default:k][default][blockligatures=yes,kernligatures=yes]
2652
2653\showfontkerns
2654\stopbuffer
2655
2656\startbuffer[demo]
2657{\definedfont[Brill*default   @ 11pt]auflage}\par
2658{\definedfont[Brill*default:b @ 11pt]auflage}\par
2659{\definedfont[Brill*default:k @ 11pt]auflage}\par
2660\stopbuffer
2661
2662\typebuffer[definitions,demo] \getbuffer[definitions]
2663
2664\startlinecorrection
2665    \externalfigure[demo.buffer][width=4cm]
2666\stoplinecorrection
2667
2668Processing fonts is complicated by the fact that a text can be hyphenated. This
2669complicates for instance ligature building which can cross the pre, post and|/|or
2670replace bounds. The current implementation does a decent job although there will
2671always be border cases. And, figuring out what goes wrong is a pain. There are
2672several ways to trace what happens and here's one. As mentioned, blocking only
2673works when we haven't not yet defined a font instance, so we use a funny size
2674here.
2675
2676\startbuffer
2677\blockligatures[u:fl:a]
2678
2679\definefontfeature
2680  [blockligatures]
2681  [default]
2682  [blockligatures=yes]
2683
2684\startotfcompositionlist{texgyrepagella-regular*blockligatures @ 14.5pt}{l2r}
2685    \HL
2686    \showotfcompositionsample{auflage}
2687    \showotfcompositionsample{a\discretionary{-}{}{}uflage}
2688    \showotfcompositionsample{au\discretionary{-}{}{}flage}
2689    \showotfcompositionsample{auf\discretionary{-}{}{}lage}
2690    \showotfcompositionsample{aufl\discretionary{-}{}{}age}
2691    \showotfcompositionsample{aufla\discretionary{-}{}{}ge}
2692    \showotfcompositionsample{auflag\discretionary{-}{}{}e}
2693    \HL
2694    \showotfcompositionsample{auflegt}
2695    \showotfcompositionsample{a\discretionary{-}{}{}uflegt}
2696    \showotfcompositionsample{au\discretionary{-}{}{}flegt}
2697    \showotfcompositionsample{auf\discretionary{-}{}{}legt}
2698    \showotfcompositionsample{aufl\discretionary{-}{}{}egt}
2699    \showotfcompositionsample{aufle\discretionary{-}{}{}gt}
2700    \showotfcompositionsample{aufleg\discretionary{-}{}{}t}
2701    \HL
2702\stopotfcompositionlist
2703\stopbuffer
2704
2705\typebuffer \getbuffer
2706
2707Here is another example. This one demonstrates that ligatures can force
2708collapsing of discretionaries.
2709
2710\startbuffer
2711\startotfcompositionlist{Serif*default @ 11pt}{l2r}
2712    \HL
2713    \showotfcompositionsample{effe}
2714    \showotfcompositionsample{efficient}
2715    \HL
2716\stopotfcompositionlist
2717\stopbuffer
2718
2719\typebuffer \getbuffer
2720
2721\stopsection
2722
2723\startsection[title=Collections]
2724
2725    {\em Todo.}
2726
2727\stopsection
2728
2729\stopchapter
2730