luametatex-fonts.tex /size: 51 Kb    last modification: 2025-02-21 11:03
1% language=us runpath=texruns:manuals/luametatex
2
3\environment luametatex-style
4
5\startdocument[title=Fonts]
6
7\startsection[title={Introduction}]
8
9The \LUATEX\ engine changed the approach to loading fonts and processing kerns
10and ligatures by introducing a \LUA\ loader and callbacks for processing
11replacement and positioning features. In \LUAMETATEX\ we go a step further and no
12longer load fonts otherwise than with \LUA. In the end, all that \TEX\ needs are
13a few dimensions and optionally ligature and kerning tables. Of course for math a
14bit more is needed but even there we can safely delegate all loading to \LUA. In
15\LUAMETATEX\ we still have the traditional kerning and ligature built in because
16after all that method is the reference for traditional fonts and the amount of
17code needed is relatively small.
18
19The backend is gone, so here the final font inclusion is also done by \LUA. This
20means that in the engine the amount of code involved in that is zero. In the
21engine we have glyphs and glyphs traditionally carry a font identifier (an
22number) and a glyph reference (also a number). Both are used to fetch the width,
23height, depth, italic correction ans some more from the fonts registered in the
24engine. For \TEX\ a font is more of an abstraction that from \LUA, where we can
25manipulate details and deal with the real shapes.
26
27In \LUAMETATEX\ the situation is simplified on the one hand, read: no font
28loader, but complicated on the other, for instance because we have dynamic
29scaling. In this chapter we discuss what data is stored in glyphs, what
30primitives are involved, and how loading takes place. Because a lot can be done
31in \LUA\ and because there are no standards involved, we don't need to discuss
32how a macro package is supposed to deal with all this; one can consider \CONTEXT\
33as a reference implementation if needed.
34
35Removing the font loader and backend had relatively little impact on \CONTEXT\
36because we already did most in \LUA, but as we developed \LUAMETATEX\ both
37subsystems evolved further. Especially moving more backend processing to \LUA\
38had some impact on performance but in the end the engine is much faster so we
39gained that back. Additions to the font system, like dynamic scaling of course
40have impact too but we could also limit the amount of fonts that get loaded which
41compensates for any loss in performance. The most complicated and demanding part
42of the backend code is that what deals with fonts: sharing, subsetting,
43devirtualizing, scaling, effects like weight, slanting, expansion, accuracy,
44accessibility, \unknown, all of that has to be dealt with.
45
46In this chapter we discuss a few aspects like primitives, defining fonts, \LUA\
47helpers, and virtual fonts, but for a more complete picture one really has to
48read the documents that describe how all evolved, how fonts are used in \CONTEXT\
49as well as look at how we apply all this. There is no reason to repeat everything
50here, especially because for most users this is not something they need to know.
51There are dedicated manuals and articles that cover different aspects.
52
53\stopsection
54
55\startsection[title={Primitives}]
56
57\startsubsection[title=Basic properties]
58
59Although primitives are discusses in their own chapter we repeat some here
60because it impacts following sections. Let's start with the commands that change
61the look and feel of a font:
62
63\startbuffer
64\begingroup                   glyphs represent characters \endgroup
65\begingroup \glyphscale  1200 glyphs represent characters \endgroup
66\begingroup \glyphxscale 1200 glyphs represent characters \endgroup
67\begingroup \glyphyscale  800 glyphs represent characters \endgroup
68\begingroup \glyphslant   200 glyphs represent characters \endgroup
69\begingroup \glyphweight  200 glyphs represent characters \endgroup
70\stopbuffer
71
72\typebuffer[option=TEX]
73
74This results in:
75
76\startlines \getbuffer \stoplines
77
78These parameters are applied to glyphs that get added to the current list of
79nodes. Whenever the engine (or the \LUA\ end) needs a dimension, two scales have
80to be applied, depending on the dimension being horizontal or vertical. Sometimes
81the slant and weight also have to be taken into account. Later we will see that
82we have additional math scaling so you can imagine that applying a handful of
83scales has a bit of impact on the code and also performance. However, the later
84will not be noticed because computers are fast enough.
85
86Here is how we can apply the scaling factors to dimensions:
87
88\startbuffer
89{\glyphxscale 1500                  \the\glyphxscaled 100pt} and
90{\glyphyscale  750                  \the\glyphyscaled 100pt} and
91{\glyphscale  1500 \glyphxscale 500 \the\glyphxscaled 100pt}
92\stopbuffer
93
94\typebuffer[option=TEX]
95
96We get: \inlinebuffer. In scenarios like these you need to keep in mind that the
97currently set scales also apply. The main reason why we use these 1000 based
98factor is that it is the way \TEX\ does things. We could have used posits instead
99but those were added later so for now it's factors that dominate.
100
101\stopsubsection
102
103\startsubsection[title=Specifications]
104
105A font is loaded at a specific size, so these properties start from that: the
106design size and the requested size which results in a scaling factor. Every font
107has a number so here we have:
108
109\startbuffer
110\tf \the \fontid \font \hskip1cm
111\bf \the \fontid \font \hskip1cm
112\sl \the \fontid \font
113\stopbuffer
114
115\typebuffer[option=TEX]
116
117\start \getbuffer \stop
118
119A set of settings can be combined in specification, here \type {\font} is the
120current font, from which the specification takes the identifier.
121
122\startbuffer
123\fontspecdef \MyFontA \font xscale 2000 yscale 800 weight 200 slant 200 \relax
124\fontspecdef \MyFontB \font all 1000 1500 800 250 150 \relax
125
126\begingroup \MyFontA Is this neat or not? \endgroup
127\begingroup \MyFontB Is this neat or not? \endgroup
128\stopbuffer
129
130\typebuffer[option=TEX]
131
132\startlines \getbuffer \stoplines
133
134Instead of an id an already defined specification can be given in which case we
135start from a copy:
136
137\starttyping[option=TEX]
138\fontspecdef \MyFontA 2 all 1000
139\fontspecdef \MyFontB \MyFontA xscale 1200
140\stoptyping
141
142Say that we have:
143
144\startbuffer
145\fontspecdef\MyFoo\font xscale 1200 \relax
146\stopbuffer
147
148\typebuffer[option=TEX] \getbuffer
149
150The four properties of such a specification can then be queried as follows:
151
152\startbuffer
153[\the\fontspecid       \MyFoo]
154[\the\fontspecscale    \MyFoo]
155[\the\fontspecxscale   \MyFoo]
156[\the\fontspecyscale   \MyFoo]
157[\the\fontspecifiedsize\MyFoo]
158[\fontspecifiedname    \MyFoo]
159\stopbuffer
160
161\typebuffer[option=TEX] \getbuffer
162
163A font specification obeys grouping but is not a register. Like \type
164{\integerdef} and \type {\dimendef} it is just a control sequence with a special
165meaning.
166
167If you read about compact font mode in \CONTEXT, this is what we're using there.
168It started out by more aggressive sharing and scaling but eventually all five
169properties were integrated in a fast font switch. However, setting these five
170properties, even with one command has some overhead because they are saved on the
171save stack. Okay, that was a bit if a lie: no one will notice that overhead:
172
173\startbuffer
174\fontspecdef \MyFontA \font
175    scale 1100 xscale 2000 yscale 800 weight 200 slant 200
176\relax
177\fontspecdef \MyFontB \font
178    scale 1200 xscale 1000 yscale 200 weight 100 slant 100
179\relax
180\stopbuffer
181
182\typebuffer [option=TEX]
183
184A 100.000 times \type {{\MyFontA \MyFontB}} grouped expansion takes 0.02 seconds
185runtime on my 2018 laptop, which is just noise once we start processing text:
186100.000 times \type {{\MyFontA efficient \MyFontB efficient}} takes 1.4 seconds
187and 100.000 times \type {{\MyFontA test \MyFontB test}} takes 0.4 seconds. Guess
188why.
189
190\stopsubsection
191
192\startsubsection[title=Offsets]
193
194These two parameters control the horizontal and vertical shift of glyphs with,
195when applied to a stretch of them, the horizontal offset probably being the least
196useful. The values default to the currently set values. Here is a \CONTEXT\ example:
197
198\startbuffer
199\ruledhbox \bgroup
200    \ruledhbox {\glyph yoffset 1ex xoffset -.5em 123}
201    \ruledhbox {\glyph yoffset 1ex               125}
202    \ruledhbox \bgroup
203        baseline
204        \glyphyoffset 1ex \glyphxscale 800 \glyphyscale \glyphxscale
205        raised%
206    \egroup
207\egroup
208\stopbuffer
209
210\typebuffer
211
212Visualized:
213
214\startlinecorrection[blank]
215\getbuffer
216\stoplinecorrection
217
218\stopsubsection
219
220\startsubsection[title=Math scales and identifiers]
221
222More details about fonts in math mode can be found in the chapters about math and
223primitives so here we just mention a few of these primitives. The internal \type
224{\glyphtextscale}, \type {\glyphscriptscale} and \type {\glyphscriptscriptscale}
225registers can be set to enforce additional scaling of math, like this:
226
227\startbuffer
228$                            a = b^2 = c^{d^2}$
229$\glyphtextscale         800 a = b^2 = c^{d^2}$
230$\glyphscriptscale       800 a = b^2 = c^{d^2}$
231$\glyphscriptscriptscale 800 a = b^2 = c^{d^2}$
232\stopbuffer
233
234\typebuffer[option=TEX]
235
236You can of course set them all in any mix as long as the value is larger than
237zero and doesn't exceed 1000. In \CONTEXT\ we use this for special purposes so
238don't mess with it there. as there can be unexpected (but otherwise valid) side
239effects.
240
241\startlines
242\getbuffer
243\stoplines
244
245The next few reported values depend on the font setup. A math font can be loaded
246at a certain scale and further scaled on the fly. An open type math font comes
247with recommended script and script script scales and gets passed to the engine
248scaled. The values reported by \type {\mathscale} are {\em additional} scales.
249
250\startbuffer
251$\the\mathscale\textfont        \zerocount$
252$\the\mathscale\scriptfont      \zerocount$
253$\the\mathscale\scriptscriptfont\zerocount$
254\stopbuffer
255
256\typebuffer[option=TEX]
257
258gives: \inlinebuffer
259
260In math mode the font id depends on the style because there we have a family of
261three related fonts or the same font with different scales. In this document we
262get the following identifiers:
263
264\startbuffer
265$\the\mathstylefontid\scriptscriptstyle \fam$
266$\the\mathstylefontid\scriptstyle       \fam$
267$\the\mathstylefontid\textstyle         \fam$
268\stopbuffer
269
270\typebuffer[option=TEX]
271
272Gives: \inlinebuffer, which is no surprise because we use the same font for all
273sizes combined with the \type {smaller} field options discusses later. In \CONTEXT\
274math uses compact font mode with in-place scaling by default.
275
276\stopsubsection
277
278\startsubsection[title={Scaled fontdimensions}]
279
280When you use \type {\glyphscale}, \type {\glyphxscale} and|/|or \type
281{\glyphyscale} the font dimensions also scale. The values that are currently used
282can be queried:
283
284\starttabulate[|l|c|c|c|]
285\FL
286\BC dimension                       \BC scale \BC xscale \BC yscale \NC\NR
287\TL
288\NC \type {\scaledemwidth}          \NC \star \NC \star  \NC        \NC\NR
289\NC \type {\scaledexheight}         \NC \star \NC        \NC \star  \NC\NR
290\NC \type {\scaledextraspace}       \NC \star \NC \star  \NC        \NC\NR
291\NC \type {\scaledinterwordshrink}  \NC \star \NC \star  \NC        \NC\NR
292\NC \type {\scaledinterwordspace}   \NC \star \NC \star  \NC        \NC\NR
293\NC \type {\scaledinterwordstretch} \NC \star \NC \star  \NC        \NC\NR
294\NC \type {\scaledslantperpoint}    \NC \star \NC \star  \NC        \NC\NR
295\LL
296\stoptabulate
297
298The next table shows the effective sized when we scale by 2000. The last two
299columns scale twice: the shared scale and the $x$ or $y$ scale.
300
301\def\ShowThem#1%
302  {\normalexpanded{
303   \BC \small \prm {\csstring\parametermark1}
304   \NC {\localcontrolled{\glyphscale2000 \glyphxscale1000 \glyphyscale 1000} \withoutpt\parametermark1}
305   \NC {\localcontrolled{\glyphscale1000 \glyphxscale2000 \glyphyscale 1000} \withoutpt\parametermark1}
306   \NC {\localcontrolled{\glyphscale1000 \glyphxscale1000 \glyphyscale 2000} \withoutpt\parametermark1}
307   \NC {\localcontrolled{\glyphscale2000 \glyphxscale2000 \glyphyscale 1000} \withoutpt\parametermark1}
308   \NC {\localcontrolled{\glyphscale2000 \glyphxscale1000 \glyphyscale 2000} \withoutpt\parametermark1}
309   \NC \NR}}
310
311\starttabulate[|l|c{.}|c{.}|c{.}|c{.}|c{.}|]
312    \ShowThem\scaledemwidth
313    \ShowThem\scaledexheight
314    \ShowThem\scaledextraspace
315    \ShowThem\scaledinterwordshrink
316    \ShowThem\scaledinterwordspace
317    \ShowThem\scaledinterwordstretch
318    \ShowThem\scaledslantperpoint
319\stoptabulate
320
321\stopsubsection
322
323\startsubsection[title=Character properties]
324
325The \type {\fontcharwd}, \type {\fontcharht}, \type {\fontchardp} and \type
326{\fontcharic} give access to character properties. To this repertoire
327\LUAMETATEX\ adds the top and bottom accent accessors \type {\fontcharta} and
328\type {\fontcharba} that came in handy for tracing. You pass a font reference and
329character code. Normally only \OPENTYPE\ math fonts have this property.
330
331\stopsubsection
332
333\startsubsection[title=Glyph options]
334
335In \LUATEX\ the \type {\noligs} and \type {\nokerns} primitives suppress these
336features but in \LUAMETATEX\ these primitives are gone. They are replace by a
337more generic control primitive \type {\glyphoptions}. This numerical parameter is
338a bitset with the following fields:
339
340\starttworows
341\getbuffer[engine:syntax:glyphoptions]
342\stoptworows
343
344The effects speak for themselves. They provide detailed control over individual
345glyph, this because the current value of this option is stored with glyphs. In
346\CONTEXT\ we have commands that set flags like that and also make sure that there
347is no interference in setting them. it's good to know that some of these options
348are there so that we can properly demonstrate, discuss and document \LUAMETATEX\
349behavior. The current value of this parameter is {\tt 0x\tohexadecimal
350\glyphoptions} but that can of course change because we experiment with options
351and bit positions might change over time, which is why we can query the engine.
352
353\stopsubsection
354
355\stopsection
356
357\startsection[title={Nodes}]
358
359This chapter is not about nodes so we keep this section short. A glyph node is an
360important one and a page easily has a few thousand of them. When a list that has
361glyphs nodes is processed, depending on the font quite some passes over that list
362are made in order to sort out substitutions, alternatives and ligatures as well
363as font kerning and anchoring. When the paragraph is constructed these glyphs are
364consulted and dimensions and expansion properties are accessed and scaling can
365happen. These glyph nodes are among the largest and have many fields. To what
366extend you can use these fields depends on the macro package and the reason is
367that some of these fields also affect the backend and the backend is provided by
368the macro package. When the script|/|language combination that you use supports
369hyphenation, there can be discretionary nodes that have a pre, post and|/|or
370replace component set that are node lists that can contain glyph nodes and
371whenever we mess around with glyphs we also need to check these.
372
373The most important fields are \type {font} and \type {character}, as these
374uniquely point to what shape is used. That also means that at the \LUA\ end we
375can have more information than \TEX\ needs and can do things that \TEX\ in its
376role as constructor is unaware of. The par builder doesn't really care what it
377deals with, it only needs dimensions and maybe some properties.
378
379The \type {data}, \type {state}, \type {script} and \type {protected} fields are
380used for instance by \CONTEXT\ and in particular the font handler. There are
381primitives that can query and set these fields, like \typ {\glyphdatafield}, \typ
382{\glyphscriptfield} and \typ {\glyphstatefield}.
383
384These primitives can be used to set an additional glyph properties. Of course
385it's very macro package dependent what is done with that. It started with just
386the first one as experiment, simply because we had some room left in the glyph
387data structure. It's basically an single attribute. Then, when we got rid of the
388ligature pointer we could either drop it or use that extra field for some more,
389and because \CONTEXT\ already used the data field, that is what happened. The
390script and state fields are shorts, that is, they run from zero to \type {0xFFFF}
391where we assume that zero means \quote {unset}. Although they can be used for
392whatever purpose their use in \CONTEXT\ is fixed. So far for a historical note.
393
394The \typ {language} field is used by the hyphenator but can also be used by the
395macro package. The \type {lhmin} and \type {rhmin} are only useful for the
396hyphenator and these values are set by the language mechanisms and primitives.
397The \type {discpart} bitset registers what the engine did which can be handy for
398tracing.
399
400We already mentioned scales, slant and weight and these go to fields \type
401{scale}, \type {xscale}, \type {yscale}, \type {slant} and \type {weight}. The
402\type {expansion}, \type {raise}, \type {left}, \type {right}, \type {xoffset}
403and \type {yoffset} can be set by \TEX\ but also by the font handler. Messing
404with any of these fields at the \TEX\ end is easy but one really should take into
405account what the macro packages needs them for and does with them at the \LUA\
406end and in the backend. In that respect \LUAMETATEX\ lets the user free but it
407also means that you cannot expect macro packages (assuming that \CONTEXT\ is not
408the only user) to behave the same.
409
410The various math subsystems use \type {properties}, \type {group} and \type
411{index} and again this also macro package specific. The \type {options} bitset
412controls all kind of processes in the engine when it comes to using glyphs (user
413level \type {\glyphoptions}) as do \type {control} and \type {hyphenate}.
414
415It would take many pages to explain all this so again we just refer to how
416\CONTEXT\ uses these fields, the way they can be set from \TEX\ and accessed in
417\LUA. In the end, all the users see of this is shapes anyway, while macro
418packages integrate and present these as features.
419
420\stopsection
421
422\startsection[title={Loading}]
423
424A font is normally defined by \type {\font} which in \LUAMETATEX\ is just a
425trigger for a callback. You can even do without that primitive because you can
426load a font and then use \type {\setfontid} or the previously mentioned
427specification to switch to a font. The callback, discussed in the callbacks
428chapter, gets a name and size, and is supposed to return a font identifier. You
429can use the name to locate and load a font, register the font using the following
430function, which gives you an identifier that satisfies the callback.
431
432\starttyping [option=LUA]
433function font.define ( <t:table> font, <t:integer> id )
434    return <t:integer> id
435end
436\stoptyping
437
438with respect to \type {\font} it's good to know that the engine accept a braced
439argument as a font name:
440
441\starttyping[option=TEX]
442\font\myfont = {My Fancy Font}
443\stoptyping
444
445This allows for embedded spaces, without the need for double quotes. Macro
446expansion takes place inside the argument. Although in \CONTEXT\ \LMTX\ we don't
447use the \type {\font} for defining fonts, it still can be uses.
448
449The font table is mandate but the identifier is optional. The table has the
450following fields, most of which concern math. The \type {name} field is mandate
451because it is needed in various feedback scenarios.
452
453% \showenginekeylist{font.getfontfields()}
454
455\starttabulate[|l|l|pl|]
456\FL
457\BC key                        \BC type    \BC description \NC \NR
458\ML
459\NC \type {name}               \NC string  \NC metric (file) name \NC \NR
460\NC \type {original}           \NC string  \NC the name used in logging and feedback \NC \NR
461\NC \type {designsize}         \NC number  \NC expected size (default: 655360 == 10pt) \NC \NR
462\NC \type {size}               \NC number  \NC the required scaling (by default the same as \type {designsize}) \NC \NR
463\HL
464\NC \type {compactmath}        \NC boolean \NC use the \type {smaller} fields in lookups        \NC \NR
465\NC \type {mathcontrol}        \NC bitset  \NC this controls various options in the math engine \NC \NR
466\NC \type {textcontrol}        \NC bitset  \NC this controls various options in the text engine \NC \NR
467\NC \type {nomath}             \NC boolean \NC don't check for math parameters and properties   \NC \NR
468\HL
469\NC \type {characters}         \NC table   \NC the defined glyphs of this font \NC \NR
470\NC \type {fonts}              \NC table   \NC locally used fonts              \NC \NR
471\NC \type {parameters}         \NC table   \NC parameters by index and/or key  \NC \NR
472\NC \type {MathConstants}      \NC table   \NC \OPENTYPE\ math parameter       \NC \NR
473\HL
474\NC \type {hyphenchar}         \NC number  \NC default: \TEX's \type {\hyphenchar} \NC \NR
475\NC \type {skewchar}           \NC number  \NC default: \TEX's \type {\skewchar}   \NC \NR
476\HL
477\NC \type {textscale}          \NC number  \NC scale applied to math text          \NC \NR
478\NC \type {scriptscale}        \NC number  \NC scale applied to math script        \NC \NR
479\NC \type {scriptscriptscale}  \NC number  \NC scale applied to math script script \NC \NR
480\HL
481\NC \type {textxscale}         \NC number  \NC horizontal scale applied to math text          \NC \NR
482\NC \type {scriptxscale}       \NC number  \NC horizontal scale applied to math script        \NC \NR
483\NC \type {scriptxscriptscale} \NC number  \NC horizontal scale applied to math script script \NC \NR
484\HL
485\NC \type {textyscale}         \NC number  \NC vertical scale applied to math text          \NC \NR
486\NC \type {scriptyscale}       \NC number  \NC vertical scale applied to math script        \NC \NR
487\NC \type {scriptyscriptscale} \NC number  \NC vertical scale applied to math script script \NC \NR
488\HL
489\NC \type {textweight}         \NC number  \NC weight  applied to math text          \NC \NR
490\NC \type {scriptweight}       \NC number  \NC weight  applied to math script        \NC \NR
491\NC \type {scriptscriptweight} \NC number  \NC weight  applied to math script script \NC \NR
492\LL
493\stoptabulate
494
495% % The \type {mathcontrol} bitset is mostly there for experimental purposes. Because
496% % there is inconsistency in the \OPENTYPE\ math fonts with respect to for instance
497% % glyph dimensions, it is possible to force the traditional code path. We just mention
498% % the possible flags:
499% %
500% % \startluacode
501% %     context.starttabulate { "|||" }
502% %     context.DB() context("value") context.BC() context("effect") context.NC() context.NR()
503% %     context.TB()
504% %     for k, v in table.sortedhash(tex.getmathcontrolvalues()) do
505% %         context.NC() context("0x%04X",k) context.NC() context(v) context.NC() context.NR()
506% %     end
507% %     context.LL()
508% %     context.stoptabulate()
509% % \stopluacode
510%
511%
512% % The \type {textcontrol} field is used to control some aspects of text processing.
513% % More options might be added in the future.
514%
515% % \startluacode
516% %     context.starttabulate { "|||" }
517% %     context.DB() context("value") context.BC() context("effect") context.NC() context.NR()
518% %     context.TB()
519% %     for k, v in table.sortedhash(tex.gettextcontrolvalues()) do
520% %         context.NC() context("0x%04X",k) context.NC() context(v) context.NC() context.NR()
521% %     end
522% %     context.LL()
523% %     context.stoptabulate()
524% % \stopluacode
525% %
526% % In \CONTEXT\ these are interfaced via pseudo features. The math control flags of
527% % a font can be overloaded by \prm {mathcontrolmode} on the spot and the set
528% % controls of a font can be queried by \prm {fontmathcontrol}. The text control
529% % flags in a font always win over the ones set by other parameters, like \prm
530% % {hyphenationmode}. They can be queried with \prm {fonttextcontrol}.
531
532There are three tables that need their own explanation. The \type {parameters}
533table is a hash with mixed key types. There are seven possible string keys, as
534well as a number of integer indices. The seven strings are actually used instead
535of the bottom seven indices, because that gives a nicer user interface. There are
536additional indexed entries possible for math fonts but nowadays one will use
537\OPENTYPE\ math fonts so these no longer make sense.
538
539% \showenginekeylist{font.getparameterfields()}
540
541\starttabulate[|l|c|]
542\FL
543\BC name                 \BC index \NC \NR
544\ML
545\NC \type {slant}        \NC 1 \NC \NR
546\NC \type {space}        \NC 2 \NC \NR
547\NC \type {spacestretch} \NC 3 \NC \NR
548\NC \type {spaceshrink}  \NC 4 \NC \NR
549\NC \type {xheight}      \NC 5 \NC \NR
550\NC \type {quad}         \NC 6 \NC \NR
551\NC \type {extraspace}   \NC 7 \NC \NR
552\LL
553\stoptabulate
554
555The \type {characters} table can be pretty large when we have \OPENTYPE\ fonts.
556In \CONTEXT\ we use \UNICODE\ as encoding which means that glyphs are organized
557as such. This also means that we have a hash and not an indexed array due to
558gaps. There can be more data in the glyph sub tables than the engine needs
559because the engine only picks up those that it needs. You can also later decide
560to pass additional properties and even glyphs to the engine, but changes can of
561course have consequences because at some point the backend will pick up data and
562use that. Additions are fine but changes have to be consistent. Of course it all
563depends on how you implement a backend.
564
565When a character in the input is turned into a glyph node, it gets a character
566code that normally refers to an entry in that table. For proper paragraph
567building and math rendering the fields in the tables below can best be present in
568an entry in the \type {characters} table. As said, you can of course add all kind
569of extra fields. The engine only uses those that it needs for typesetting a
570paragraph or formula. The sub tables that define ligatures and kerns are also
571hashes with integer keys, and these indices should point to entries in the main
572characters table. The fields common to text and math chartacters are:
573\showenginekeylist {font.gettextcharacterfields()}.
574
575Providing ligatures and kerns via this table permits \TEX\ to construct ligatures
576and add inter|-|character kerning. However, normally you will use an \OPENTYPE\
577font in combination with \LUA\ code that does this. In \CONTEXT\ we have base
578mode that uses the engine, and node mode that uses \LUA. A mono spaced font
579normally has no ligatures and inter character kerns and is normally not processed
580at all.
581
582We can group the parameters. All characters have the following base set. It must
583be noted here that \OPENTYPE\ doesn't have a italic property and that the height
584and depth are also not part of the design: one can choose to derive them from the
585bounding box.
586
587\starttabulate[|l|l|pl|]
588\FL
589\BC key            \BC type   \BC description                         \NC \NR
590\ML
591\NC \type {width}  \NC number \NC width in sp (default 0)             \NC \NR
592\NC \type {height} \NC number \NC height in sp (default 0)            \NC \NR
593\NC \type {depth}  \NC number \NC depth in sp (default 0)             \NC \NR
594\NC \type {italic} \NC number \NC italic correction in sp (default 0) \NC \NR
595\LL
596\stoptabulate
597
598There are four parameters that are more optional and relate to advanced optical
599paragraph optimization:
600
601\starttabulate[|l|l|pl|]
602\FL
603\BC key                     \BC type   \BC description                               \NC \NR
604\ML
605\NC \type {leftprotruding}  \NC number \NC left protruding factor  (\type {\lpcode}) \NC \NR
606\NC \type {rightprotruding} \NC number \NC right protruding factor (\type {\rpcode}) \NC \NR
607\NC \type {expansion}       \NC number \NC expansion factor        (\type {\efcode}) \NC \NR
608\NC \type {compression}     \NC number \NC compression factor      (\type {\cfcode}) \NC \NR
609\LL
610\stoptabulate
611
612The left and right protrusion factors as well as the expansion factor are
613comparable to the ones introduced by \PDFTEX, but compression is new and
614complements expansion. In \LUAMETATEX\ the expansion mechanism is also available
615in math. You might have noticed that we don't have expansion related parameters
616in the main font table. This is because we have a more dynamic model. These
617values are anyway only used when \type {\protrudechars} and|/|or \type
618{\adjustspacing} are set. The later can also be controlled by so called par
619passes and thereby applied more selectively. Because setting these fields using
620specific glyph properties can take time, it is also possible to delay these
621settings till a dedicated callback is triggered when they are needed.
622
623From \TEX\ we inherit the following tables. Ligatures are only used in so call
624base mode, when the engine does the font magic. Kerns are used in base mode text
625and optionally in math.
626
627\starttabulate[|l|l|pl|]
628\FL
629\BC key               \BC type  \BC description            \NC \NR
630\ML
631\NC \type {ligatures} \NC table \NC ligaturing information \NC \NR
632\NC \type {kerns}     \NC table \NC kerning information    \NC \NR
633\LL
634\stoptabulate
635
636The next fields control the engine and are a variant on \TEX's \TFM\ tag
637property. In a future we might provide a bit more (local) control although
638currently we see no need. Originally the \type {tag} and \type {next} field were
639combined into a packed integer but in current \LUAMETATEX\ we have a 32 bit tag
640and the next field moved to the math blob as it only is used as variant selector.
641
642\starttabulate[|l|l|pl|]
643\FL
644\BC key              \BC type    \BC description                            \NC \NR
645\ML
646\NC \type {tag}      \NC number  \NC a bitset, currently not really exposed \NC\NR
647%NC \type {reserved} \NC number  \NC note for myself \NC \NR
648\stoptabulate
649
650In a math font characters have many more fields: \showenginekeylist
651{font.getmathcharacterfields()}.
652
653\starttabulate[|l|l|pl|]
654\FL
655\BC key                      \BC type    \BC description \NC \NR
656\ML
657\NC \type {smaller}          \NC number  \NC the next smaller math size character               \NC \NR
658\NC \type {mirror}           \NC number  \NC a right to left alternative                        \NC \NR
659\NC \type {flataccent}       \NC number  \NC an accent alternative with less height (\OPENTYPE) \NC \NR
660\NC \type {next}             \NC number  \NC \quote {next larger} character index               \NC \NR
661\HL
662\NC \type {topleft}          \NC number  \NC alternative script kern \NC \NR
663\NC \type {topright}         \NC number  \NC alternative script kern \NC \NR
664\NC \type {bottomleft}       \NC number  \NC alternative script kern \NC \NR
665\NC \type {bottomright}      \NC number  \NC alternative script kern \NC \NR
666\HL
667\NC \type {topmargin}        \NC number  \NC alternative accent calculation margin \NC \NR
668\NC \type {bottomargin}      \NC number  \NC alternative accent calculation margin \NC \NR
669\NC \type {leftmargin}       \NC number  \NC alternative accent calculation margin \NC \NR
670\NC \type {rightmargin}      \NC number  \NC alternative accent calculation margin \NC \NR
671\HL
672\NC \type {topovershoot}     \NC number  \NC accent width tolerance \NC \NR
673\NC \type {bottomovershoot}  \NC number  \NC accent width tolerance \NC \NR
674\HL
675\NC \type {topanchor}        \NC number  \NC horizontal top accent alignment position    \NC \NR
676\NC \type {bottomanchor}     \NC number  \NC horizontal bottom accent alignment position \NC \NR
677\HL
678\NC \type {innerlocation}    \NC string  \NC \type {left} or \type {right}      \NC \NR
679\NC \type {innerxoffset}     \NC number  \NC radical degree horizontal position \NC \NR
680\NC \type {inneryoffset}     \NC number  \NC radical degree vertical position   \NC \NR
681\HL
682\NC \type {parts}            \NC table   \NC constituent parts of an extensible                \NC \NR
683\NC \type {partsitalic}      \NC number  \NC the italic correction applied with the extensible \NC \NR
684\NC \type {partsorientation} \NC number  \NC \type {horizontal} or \type {vertical}            \NC \NR
685\HL
686\NC \type {mathkerns}        \NC table   \NC math cut-in specifications \NC \NR
687\HL
688\NC \type {extensible}       \NC table   \NC stretch a fixed width accent to fit \NC \NR
689\LL
690\stoptabulate
691
692In \LUAMETATEX\ combined with \CONTEXT\ \MKXL\ we go beyond \OPENTYPE\ math and
693have more fields here than in \LUATEX. In \CONTEXT\ those values are set with so
694called tweaks and defined in so called font goody files. This relates to the
695extended math rendering engine in \LUAMETATEX.
696
697Bidirectional math is also supported and driven by (in \CONTEXT\ speak) tweaks
698which means that it has to be set up explicitly as it uses a combination of
699fonts. The \type {mirror} field points to an alternative glyph. The \type
700{smaller} field points to a script glyph alternative and that glyph can then
701point to a script script one (in \OPENTYPE\ speak \type {ssty} alternates
702respectively one 1 and 2). In \CONTEXT\ is also uses specific features of the
703font subsystems that hook into the backend where we have a more advanced virtual
704font subsystem than in \LUATEX. Because this is macro package dependent it will
705not be discussed here.
706
707Here is the character \quote {f} (decimal 102) in the font \type {cmr10 at 10pt}.
708The numbers that represent dimensions are in scaled points. Of course you will
709use Latin Modern \OPENTYPE\ instead but the principles are the same:
710
711\starttyping[option=LUA]
712[102] = {
713    ["width"]  = 200250,
714    ["height"] = 455111,
715    ["depth"]  = 0,
716    ["italic"] = 50973,
717    ["kerns"]  = {
718        [63] = 50973,
719        [93] = 50973,
720        [39] = 50973,
721        [33] = 50973,
722        [41] = 50973
723    },
724    ["ligatures"] = {
725        [102] = { ["char"] = 11, ["type"] = 0 },
726        [108] = { ["char"] = 13, ["type"] = 0 },
727        [105] = { ["char"] = 12, ["type"] = 0 }
728    }
729}
730\stoptyping
731
732In \CONTEXT, when they are really needed, we normally turn these traditional
733eight bit fonts into emulated \OPENTYPE\ (\UNICODE) fonts so there you will only
734encounter tables like this when we process a font in base mode.
735
736Two very special string indexes can be used also: \type {leftboundary} is a
737virtual character whose ligatures and kerns are used to handle word boundary
738processing. \type {rightboundary} is similar but not actually used for anything
739(yet).
740
741The values of \type {topanchor}, \type {bottomanchor} and \type {mathkern} are
742used only for math accent and superscript placement, see \at {page} [math] in
743this manual for details. The italic corrections are a story in themselves and
744discussed in detail in other manuals. The additional parameters that deal with
745kerns, margins, overshoots, inner anchoring, etc. are engine specific and not
746part of \OPENTYPE. More information can be found in the \CONTEXT\ distribution;
747they relate the upgraded math engine project by Mikael and Hans.
748
749A math character can have a \type {next} field that points to a next larger
750shape. However, the presence of \type {extensible} will overrule \type {next}, if
751that is also present. The \type {extensible} field in turn can be overruled by
752\type {parts}, the \OPENTYPE\ version. The \type {extensible} table is very
753simple:
754
755\starttabulate[|l|l|p|]
756\FL
757\BC key        \BC type   \BC description                \NC \NR
758\ML
759\NC \type{top} \NC number \NC top character index        \NC \NR
760\NC \type{mid} \NC number \NC middle character index     \NC \NR
761\NC \type{bot} \NC number \NC bottom character index     \NC \NR
762\NC \type{rep} \NC number \NC repeatable character index \NC \NR
763\LL
764\stoptabulate
765
766The \type {parts} entry is an arrays of components. Each of those components is
767itself a hash of up to five keys:
768
769\starttabulate[|l|l|p|]
770\FL
771\BC key             \BC type   \BC description                                          \NC \NR
772\ML
773\NC \type{glyph}    \NC number \NC character index                                      \NC \NR
774\NC \type{extender} \NC number \NC (1) if this part is repeatable, (0) otherwise        \NC \NR
775\NC \type{start}    \NC number \NC maximum overlap at the starting side (scaled points) \NC \NR
776\NC \type{end}      \NC number \NC maximum overlap at the ending side (scaled points)   \NC \NR
777\NC \type{advance}  \NC number \NC advance width of this item (width is default)        \NC \NR
778\LL
779\stoptabulate
780
781The traditional (text and math) \type {kerns} table is a hash indexed by
782character index (and \quote {character index} is defined as either a
783non|-|negative integer or the string value \type {rightboundary}), with the
784values of the kerning to be applied, in scaled points.
785
786The traditional (text) \type {ligatures} table is a hash indexed by character
787index (and \quote {character index} is defined as either a non|-|negative integer
788or the string value \type {rightboundary}), with the values being yet another
789small hash, with two fields:
790
791\starttabulate[|l|l|p|]
792\FL
793\BC key         \BC type   \BC description                                   \NC \NR
794\ML
795\NC \type{type} \NC number \NC the type of this ligature command (default 0) \NC \NR
796\NC \type{char} \NC number \NC the character index of the resultant ligature \NC \NR
797\LL
798\stoptabulate
799
800The \type {char} field in a ligature is required. The \type {type} field inside a
801ligature is the numerical or string value of one of the eight possible ligature
802types supported by \TEX. When \TEX\ inserts a new ligature, it puts the new glyph
803in the middle of the left and right glyphs. The original left and right glyphs
804can optionally be retained, and when at least one of them is kept, it is also
805possible to move the new \quote {insertion point} forward one or two places. The
806glyph that ends up to the right of the insertion point will become the next
807\quote {left}.
808
809\starttabulate[|l|c|l|l|]
810\FL
811\BC textual (Knuth)       \BC number \BC string        \BC result      \NC \NR
812\ML
813\NC \type{l + r =: n}     \NC 0      \NC \type{=:}     \NC \type{|n}   \NC \NR
814\NC \type{l + r =:| n}    \NC 1      \NC \type{=:|}    \NC \type{|nr}  \NC \NR
815\NC \type{l + r |=: n}    \NC 2      \NC \type{|=:}    \NC \type{|ln}  \NC \NR
816\NC \type{l + r |=:| n}   \NC 3      \NC \type{|=:|}   \NC \type{|lnr} \NC \NR
817\NC \type{l + r  =:|> n}  \NC 5      \NC \type{=:|>}   \NC \type{n|r}  \NC \NR
818\NC \type{l + r |=:> n}   \NC 6      \NC \type{|=:>}   \NC \type{l|n}  \NC \NR
819\NC \type{l + r |=:|> n}  \NC 7      \NC \type{|=:|>}  \NC \type{l|nr} \NC \NR
820\NC \type{l + r |=:|>> n} \NC 11     \NC \type{|=:|>>} \NC \type{ln|r} \NC \NR
821\LL
822\stoptabulate
823
824The default value is~0, and can be left out. That signifies a \quote {normal}
825ligature where the ligature replaces both original glyphs. In this table the~\type {|}
826indicates the final insertion point.
827
828The third table has the \type {MathConstants} as the camel case name suggests.
829These are not discussed here. The \type {fonts} table relates to virtual fonts
830that are discussed later.
831
832\stopsection
833
834\startsection[title=Helpers]
835
836Without argument this function returns the current font identifier and when
837an identifier is passed that one is made current.
838
839\starttyping [option=LUA]
840function font.current ( <t:nil> | <t:integer> )
841    -- no return value
842end
843\stoptyping
844
845This returns the maximum font identifier in use:
846
847\starttyping [option=LUA]
848function font.max ( )
849    return <t:integer> -- identifier
850end
851\stoptyping
852
853This one defines a font but needs an identifier, for instance reserved by \type
854{font.nextid}. The table is the same as with \type {font.define}.
855
856\starttyping [option=LUA]
857function font.setfont ( <t:integer> identifier, <t:table> data )
858    -- no return value
859end
860\stoptyping
861
862The next function can be used to add characters to a font. The table is the same
863as the table used when defining the characters in a font. The identifier must be
864known.
865
866\starttyping [option=LUA]
867function font.addcharacters ( <t:integer> identifier, <t:table> characters )
868    -- no return value
869end
870\stoptyping
871
872When protrusion or expansion data is needed for a character in a font and the
873relevant values are not yet known, a callback can be triggered and the next function
874can then be used to assign these.
875
876\starttyping [option=LUA]
877function font.addquality (
878    <t:integer> identifier,
879    <t:table>   characters
880)
881    -- no return value
882end
883\stoptyping
884
885The table looks like this:
886
887\starttyping [option=LUA]
888{
889    [index] = {
890        leftprotrusion  = <t:integer>,
891        rightprotrusion = <t:integer>,
892        expansion       = <t:integer>,
893        compression     = <t:integer>,
894    },
895    ...
896}
897\stoptyping
898
899Sometimes it can be handy to check what the next identifier will be. The optional
900boolean, when true, makes that the font is allocated.
901
902\starttyping [option=LUA]
903function font.nextid ( <t:nil> | <t:boolean> )
904    return <t:integer> -- identifier
905end
906\stoptyping
907
908This function does a lookup by name and returns the font identifier when it's
909known:
910
911\starttyping [option=LUA]
912function font.id ( <t:string> name )
913    return <t:integer> -- identifier
914end
915\stoptyping
916
917The value that gets returned or is assigned is always an integer because that is
918what these parameters are: scaled dimensions, percentages, factors.
919
920\starttyping [option=LUA]
921function font.getfontdimen (
922    <t:integer> identifier,
923    <t:integer> parameter
924)
925    return <t:integer> -- value
926end
927\stoptyping
928
929\starttyping [option=LUA]
930function font.setfontdimen (
931    <t:integer> identifier,
932    <t:integer> parameter,
933    <t:integer> value
934)
935    -- no return value
936end
937\stoptyping
938
939This one returns the properties that relate to a \type {\fontspecdef}:
940
941\starttyping [option=LUA]
942function font.getfontspec ( <t:string> name )
943    return
944        <t:integer>, -- identifier
945        <t:integer>, -- scale
946        <t:integer>, -- xscale
947        <t:integer>, -- yscale
948        <t:integer>, -- slant
949        <t:integer>  -- weight
950end
951\stoptyping
952
953Math characters are not really defined along with a font but their family can
954bind them to one. However, in \CONTEXT\ we have them decoupled and families are
955assigned fonts when the need is there.
956
957\starttyping [option=LUA]
958function font.getmathspec ( )
959    return
960        <t:integer>, -- class
961        <t:integer>, -- family
962        <t:integer>  -- character
963
964end
965\stoptyping
966
967Internally a math font parameter has a number. This function returns that number
968plus a boolean indicating if we have an variable that is not officially in
969\OPENTYPE\ math but an addition to the \LUAMETATEX\ engine.
970
971\starttyping [option=LUA]
972function font.getmathindex ( <t:string> | <t:number> )
973    return
974        <t:number>  -- index
975        <t:boolean> -- engine
976end
977\stoptyping
978
979These two don't operate on a font but multiply the given value by the \type
980{\glyphscale} and \type {\glyphxscale} respectively
981\type {\glyphyscale}.
982
983\starttyping [option=LUA]
984function font.xscaled ( <t:number> value)
985    return <t:number> -- scaled value
986end
987\stoptyping
988
989\starttyping [option=LUA]
990function font.yscaled ( <t:number> value)
991    return <t:number> -- scaled value
992end
993\stoptyping
994
995Like in other places the engine can report what fields we have, which is handy
996when we want to check manuals like this one.
997
998\starttyping [option=LUA]
999function font.getparameterfields     ( ) return <t:table> end
1000function font.getfontfields          ( ) return <t:table> end
1001function font.gettextcharacterfields ( ) return <t:table> end
1002function font.getmathcharacterfields ( ) return <t:table> end
1003\stoptyping
1004
1005\stopsection
1006
1007\startsection[title={Virtual fonts}]
1008
1009Virtual fonts have been introduced in \TEX\ because they permit combining fonts
1010and constructing for instance accented characters from several glyphs and they
1011are what one nowadays tags as a \quote {cool} feature, especially because in
1012\LUATEX\ we can use this mechanism runtime. The nice thing is that because all
1013that \TEX\ needs is dimensions, the hard work is delegated to the backend which
1014means that the front end can be agnostic when it comes to virtual fonts.
1015
1016So, in the beginning they were mostly used for providing a direct mapping from
1017for instance accented characters onto a glyph btu we use it for a lot of other
1018situations, like math. But keep in mind that because we basically define the
1019backend ourselves and because we also control everything fonts, we can go way
1020further in \CONTEXT\ than in other engines and macro packages.
1021
1022A character is virtual when it has a \type {commands} array as part of the data.
1023A virtual character can itself point to virtual characters but be careful with
1024nesting as you can create loops and overflow the stack (which often indicates an
1025error anyway).
1026
1027At the font level there can be a an (indexed) \type {fonts} table. The values are
1028one- or two|-|key hashes themselves, each entry indicating one of the base fonts
1029in a virtual font. In case your font is referring to itself in for instance a
1030virtual font, you can use the \type {slot} command with a zero font reference,
1031which indicates that the font itself is used. So, a table looks like this:
1032
1033\starttyping
1034fonts = {
1035  { name = "ptmr8a", size = 655360 }, -- referenced as font 1
1036  { name = "psyr", size = 600000 },   -- referenced as font 2
1037  { id = 38 }                         -- referenced as font 3
1038}
1039\stoptyping
1040
1041The first referenced font (at index~1) in this virtual font is \type {ptrmr8a}
1042loaded at 10pt, and the second is \type {psyr} loaded at a little over 9pt. The
1043third one is a previously defined font that is known to \LUAMETATEX\ as font
1044id~38. The array index numbers are used by the character command definitions that
1045are part of each character.
1046
1047However, the only place in \CONTEXT\ where we really need this \type {fonts}
1048table is in some math fonts where we, also as illustration and as recognition of
1049past work, assemble a \UNICODE\ math font from sort of obsolete \TYPEONE\ fonts.
1050In most cases the virtual glyphs use glyphs that are also in the font. In that
1051case we can use \type {id} zero which is resolved to the font identifiers of the
1052font itself.
1053
1054The \type {commands} array is a hash where each item is another small array, with
1055the first entry representing a command and the extra items being the parameters
1056to that command. The frontend is only interested in the dimensions, ligatures and
1057kerns of a font, which is the reason why the \TEX\ engine didn't have to be
1058extended when virtual fonts showed up: dealing with it is up to the driver that
1059comes after the backend. The first block in the next table is what the standard
1060mentions. These two engines also support the \type {special} and \LUATEX\ brings
1061the \type {pdf} and \type {pdfmode} commands but in \LUAMETATEX\ we dropped all
1062three and also \LUATEX's \type {image}.
1063
1064But \unknown\ in \LUAMETATEX\ there is no backend built in but we might assume
1065that the one provided deals with the standard entries. However, a provided
1066backend can provide more and that is indeed what happens in \CONTEXT. Because we
1067no longer have compacting (of passed tables) and unpacking (when embedding) of
1068these tables going on we stay in the \LUA\ domain. None of the virtual
1069specification is ever seen in the engine.
1070
1071\starttabulate[|l|l|l|p|]
1072\FL
1073\BC command          \BC arguments \BC type      \BC description \NC \NR
1074\ML
1075\NC \type{font}      \NC 1         \NC number    \NC select a new font from the local \type {fonts} table \NC \NR
1076\NC \type{char}      \NC 1         \NC number    \NC typeset this character number from the current font,
1077                                                     and move right by the character's width \NC \NR
1078\NC \type{slot}      \NC 2         \NC 2 numbers \NC a shortcut for the combination of a font and char command\NC \NR
1079\NC \type{push}      \NC 0         \NC           \NC save current position\NC \NR
1080\NC \type{pop}       \NC 0         \NC           \NC pop position \NC \NR
1081\NC \type{rule}      \NC 2         \NC 2 numbers \NC output a rule $ht*wd$, and move right. \NC \NR
1082\NC \type{down}      \NC 1         \NC number    \NC move down on the page \NC \NR
1083\NC \type{right}     \NC 1         \NC number    \NC move right on the page \NC \NR
1084\HL
1085\NC \type{nop}       \NC 0         \NC           \NC do nothing \NC \NR
1086\NC \type{node}      \NC 1         \NC node      \NC output this node (list), and move right
1087                                                     by the width of this list\NC \NR
1088\NC \type{lua}       \NC 1         \NC string,
1089                                       function  \NC execute a \LUA\ script when the glyph is embedded; in case of a
1090                                                     function it gets the font id and character code passed \NC \NR
1091\NC \type{comment}   \NC any       \NC any       \NC the arguments of this command are ignored \NC \NR
1092\LL
1093\stoptabulate
1094
1095The default value for \type {font} is always~1 at the start of the \type
1096{commands} array. Therefore, if the virtual font is essentially only a
1097re|-|encoding, then you do usually not have created an explicit \quote {font}
1098command in the array. Rules inside of \type {commands} arrays are built up using
1099only two dimensions: they do not have depth. For correct vertical placement, an
1100extra \type {down} command may be needed. Regardless of the amount of movement
1101you create within the \type {commands}, the output pointer will always move by
1102exactly the width that was given in the \type {width} key of the character hash.
1103Any movements that take place inside the \type {commands} array are ignored on
1104the upper level.
1105
1106In addition to the above in \CONTEXT\ we have \type {use}, \type {left}, \type
1107{up}, \type {offset}, \type {stay}, \type {compose}, \type {frame}, \type {line},
1108\type {inspect}, \type {trace} and a plugin feature so that we can add more
1109commands (which we do). These not only provide more advanced trickery but also
1110make for smaller command tables. For some features we don't even need virtual
1111magic but have additional parameters in the glyph tables. But all that is not
1112part of the engine and its specification so it will be discussed elsewhere.
1113
1114\stopsection
1115
1116\startsection[title={Callbacks}]
1117
1118The traditional \TEX\ ligature and kerning routines are build into the engine but
1119anything more (like \OPENTYPE\ rendering) has to be implemented in \LUA. The same
1120is true for math: the engine has some expectations, for instance with respect to
1121script and script script sizes, larger sizes and extensibles and needs to know at
1122least dimensions and slots in fonts in order to assemble the math. Actually there
1123are additional scaling factors in play here because math has its own scaling
1124demands.
1125
1126\stopsection
1127
1128\startsection[title=Protrusion]
1129
1130This is more an implementation note. Compared to \PDFTEX\ and \LUATEX\ the
1131protrusion detection mechanism is enhanced a bit to enable a bit more complex
1132situations. When protrusion characters are identified some nodes are skipped:
1133
1134\startitemize[packed,columns,two]
1135\startitem zero glue \stopitem
1136\startitem penalties \stopitem
1137\startitem empty discretionaries \stopitem
1138\startitem normal zero kerns \stopitem
1139\startitem rules with zero dimensions \stopitem
1140\startitem math nodes with a surround of zero \stopitem
1141\startitem dir nodes \stopitem
1142\startitem empty horizontal lists \stopitem
1143\startitem local par nodes \stopitem
1144\startitem inserts, marks and adjusts \stopitem
1145\startitem boundaries \stopitem
1146\startitem whatsits \stopitem
1147\stopitemize
1148
1149Because this can not be enough, you can also use a protrusion boundary node to
1150make the next node being ignored. When the value is~1 or~3, the next node will be
1151ignored in the test when locating a left boundary condition. When the value is~2
1152or~3, the previous node will be ignored when locating a right boundary condition
1153(the search goes from right to left). This permits protrusion combined with for
1154instance content moved into the margin:
1155
1156\starttyping
1157\protrusionboundary1\llap{!\quad}«Who needs protrusion?»
1158\stoptyping
1159
1160\stopsection
1161
1162\startsection[title=Spaces]
1163
1164There are officially no spaces in \TEX, there is only glue. This is not problem,
1165on the contrary, it is what makes the rendering so good. In \CONTEXT\ the backend
1166can convert glue to spaces in a font but that's not an engine feature.
1167
1168The \type {\nospaces} primitive can be used to overrule the usual \type
1169{\spaceskip} related heuristics when a space character is seen in a text flow.
1170The value~\type{1} triggers no injection while \type{2} results in injection of a
1171zero skip. In \in {figure} [fig:nospaces] we see the results for four characters
1172separated by a space.
1173
1174\startplacefigure[reference=fig:nospaces,title={The \type {nospaces} options.}]
1175\startcombination[3*2]
1176    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 10mm}}
1177    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 10mm}}
1178    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 10mm}}
1179    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 1mm}}
1180    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 1mm}}
1181    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 1mm}}
1182\stopcombination
1183\stopplacefigure
1184
1185\start \showmakeup[space]
1186You can, in \CONTEXT\ see where spaces are added by enabling a visualizer: \type
1187{\showmakeup [space]} does the trick, as in this paragraph. We see
1188regular spaces as well as spaces that have a space factor applied (after
1189punctuation).
1190\stop
1191
1192\stopsection
1193
1194\stopdocument
1195