fonts-hooks.tex /size: 33 Kb    last modification: 2021-10-28 13:50
1% language=us runpath=texruns:manuals/fonts
2
3\startcomponent fonts-hooks
4
5\environment fonts-environment
6
7\startchapter[title=Hooks][color=darkcyan]
8
9\startsection[title=Introduction]
10
11One of the virtues of \TEX\ is its flexibility. Because we cannot predict what
12users want to mess around with, much of the underlying code has hooks. And because
13it's not too hard to add functionality that will break things we will not advocate
14all of it. Of course you can study the code and figure out what can be done and
15there is no problem with that. It's just that you shouldn't expect much support.
16
17In this chapter we collect some of these hooks. If you run into interesting ones
18that are worth mentioning, you can always ask us to add description here.
19
20\stopsection
21
22\startsection[title=Safe hooks]
23
24\startsubsection[title=Trimming fonts]
25
26Because we store font related information in \LUA\ tables there can be situations
27where the resources used outgrow memory. An example of such a font is \type
28{lastresort} that basically defined the whole \UNICODE\ range. The font is
29actually not that large as it uses similar placeholders for glyphs in a range,
30but it has rather verbose (redundant) names. As we normally don't need these, you
31can decide to strip them away.
32
33\starttyping
34\startluacode
35    fonts.handlers.otf.readers.registerextender {
36        name   = "remove names from lastresort",
37        action = function(fontdata)
38            if fontdata.metadata.fullname == "LastResort" then
39                for k, v in next, fontdata.descriptions do
40                    v.name = nil
41                end
42            end
43        end
44    }
45\stopluacode
46
47\definedfont[LastResort][lastresort*default sa 1]
48\stoptyping
49
50This will result in a much smaller font, one that has less change to crash the
51engine due to lack of memory. Extenders like this are applied once the font has
52been loaded but before it gets saved.
53
54\stopsubsection
55
56\stopsection
57
58\startsection[title=Loading]
59
60\startsubsection[title=Introduction]
61
62We basically have to deal with three font formats that can easily be recognized
63by the suffix of the files involved: \type {tfm} and \type {vf} files that
64describe 8 bit fonts, traditionally bitmap fonts, but as they carry only metric
65information, any 8 bit font can be described. Then there are \type {afm} files
66that contain metrics related to \TYPEONE\ fonts (stored in \type {pfb} files).
67Although such fonts could contain more than 256 shapes, the implementation was
68limited to 8 bits too. By converting \type {afm} files to \type {tfm} files,
69traditional \TEX\ can deal with \TYPEONE\ given that the backend can include them
70in the final result.
71
72In this section we will discuss some aspects of the \OPENTYPE\ font reader. As
73\TEX\ only deals with metrics (in the frontend) we need to parse them, filter
74information from it and pass the metrics to \TEX. In addition, we can use all
75kind of extra information to manipulate the so called node list but in the end
76\TEX\ is only interested in font id's (that point to a font resource) and glyph
77indexes.
78
79To overcome the 256 limitation of \TYPEONE\ fonts, in \CONTEXT\ we moved away
80from \type {tfm} files (we can of course still deal with them) and turn \type
81{afm} files into so called wide fonts. Basically we turn them in a more rich
82format that looks similar to the internal \OPENTYPE\ format we use. We will not
83go into much detail about that because \TYPEONE\ is kind of obsolete and being
84replaced by \OPENTYPE, but we will of course support the old formats simply
85because we have all these fonts around.
86
87Already early in the development of \LUATEX\ a font loader library was created
88that can turn an \OPENTYPE\ (but also a \TYPEONE) font into a \LUA\ table. This
89library is derived from \FONTFORGE\ which makes it possible to look into a font
90using that editor and at the same time get a similar view on the font in \LUA,
91which is quite handy. However, at some point in \CONTEXT\ we wanted to play with
92outlines in \METAPOST\ and for that purpose an \OPENTYPE\ reader was written in
93\LUA\ that could extract the data. Because \TYPEONE\ fonts already were done in
94\LUA\ it was a logical step to also do \OPENTYPE\ in \LUA\ so now we use an
95alternative loader that doesn't depend in the \FONTFORGE\ library. This not only
96gives more flexibility but also makes it possible to avoid some conversions
97needed to provide the \CONTEXT\ font handler with the needed information in an
98efficient way.
99
100\stopsubsection
101
102\startsubsection[title=Loading \OPENTYPE\ fonts]
103
104As with most binary media formats today an \OPENTYPE\ font file is a linked list
105of records. The top level structure is called table. There are two flavours of
106\OPENTYPE\ where the main difference is in the way the shapes are defined: they
107can be \TRUETYPE\ outlines using quadratric bezier curves or cff files using
108cubic bezier curves. The last variant is the same as \POSTSCRIPT\ \TYPEONE\
109fonts. Simplified, a quadratic curve defines the shape in points with a control
110point in between, while a quadratic one also has points but each with two control
111points (as in \METAPOST).
112
113An \OPENTYPE\ font can be large: there can be upto 65536 glyphs and lots of extra
114properties and features. In order to save space the data is rather packed using
115different numeric data types. Of course one can wonder if size really matters now
116that most bandwidth is taken by audio, video and pictures but we have to live
117with it.
118
119The definition of \OPENTYPE\ can be found on the \MICROSOFT\ website:
120\hyphenatedurl {https://www.microsoft.com/typography/otspec}. Most tables then
121could make sense for us are mentioned in the following list:
122
123\starttabulate[|Bl|l|l|]
124\NC required    \NC cmap \NC character to glyph mapping \NC \NR
125\NC             \NC head \NC font header \NC \NR
126\NC             \NC hhea \NC horizontal header \NC \NR
127\NC             \NC hmtx \NC horizontal metrics \NC \NR
128\NC             \NC maxp \NC maximum profile \NC \NR
129\NC             \NC name \NC naming table \NC \NR
130\NC             \NC os/2 \NC os/2 and windows specific metrics \NC \NR
131\NC             \NC post \NC postScript information \NC \NR
132\NC truetype    \NC glyf \NC glyph data \NC \NR
133\NC             \NC loca \NC index to location \NC \NR
134\NC postscript  \NC cff  \NC compact font format \NC \NR
135\NC             \NC vorg \NC vertical origin \NC \NR
136\NC typographic \NC base \NC baseline data \NC \NR
137\NC             \NC gdef \NC glyph definition data \NC \NR
138\NC             \NC gpos \NC glyph positioning data \NC \NR
139\NC             \NC gsub \NC glyph substitution data \NC \NR
140\NC             \NC jstf \NC justification data \NC \NR
141\NC             \NC math \NC math layout data \NC \NR
142\NC extras      \NC kern \NC kerning \NC \NR
143\NC             \NC ltsh \NC linear threshold data \NC \NR
144\NC             \NC vhea \NC vertical metrics header \NC \NR
145\NC             \NC vmtx \NC vertical metrics \NC \NR
146\NC             \NC colr \NC color table \NC \NR
147\NC             \NC cpal \NC color palette table \NC \NR
148\stoptabulate
149
150When we read these tables it depends on what we want to do with the result how
151much we will really read. For instance when we only want to identify a font and
152get some basic information we don't need to read all tables and certainly don't
153need to read them completely. If we want to have the outlines we need to read the
154\type {glyf} or \type {cff} table. If we also want to boundingbox of \POSTSCRIPT\
155shapes we even need to process the shapes so that we know the dimensions of the
156result. There is no need to summarize the format here in detail because you can
157find it on the \MICROSOFT\ site. Here I only cover some aspects that influence
158the way \TEX\ can use the fonts.
159
160One of the main differences between the readers is that the \FONTFORGE\ reader
161has a lot of (recovery) heuristics for bad fonts. Nowadays most fonts are quite
162okay, and in \CONTEXT\ we prefer to just reject bad ones. In the process of
163loading the built|-|in loader gives each glyph a name (it makes them up for
164variants needed for features). It also tries to figure out some font properties,
165like the weight. If does a pretty good job on that but it is also hard to repair
166at the \LUA\ end when it makes a bad guess. The \LUA\ variants stays closer to
167the specification, but delegates more to the final user, which is good because we
168need and want that level of control as controls is what \TEX\ is about. It also
169made it possible to support for instance colored fonts without too much effort.
170
171So what data needs to be collected? If we look at what we get eventually the list
172of glyphs is the bulk. For each glyph we collect some metric information. For
173instance we fetch the (advance) width of the glyph but also the boundingbox,
174which gives us the the height and depth.
175
176In the font file the list of glyphs starts at zero and runs up tot the total
177number of glyphs. The index in this table is used in for instance the tables that
178define the font features, for instance kerning between glyphs, or multiple glyphs
179that are turned into ligatures. Each glyph gets a name. That can be a meaningful
180one but also a rather dumb one, for instance the index number.
181
182Eventually (at least in \CONTEXT) we don't order by glyph index but by \UNICODE.
183The font file contains information about the mapping from index to \UNICODE. In
184principle other encodings are possible but we stick to \UNICODE. But, because
185many glyphs can refer to one \UNICODE\ slot, for instance a regular shape as well
186as a smallcaps or oldstyle variant. These extra glyphs we let end up in the
187private \UNICODE\ areas. This also means that with each glyph in the final table
188there is also a field that has the \UNICODE. Because we order by \UNICODE\ we
189also need to store the index. An example from a Latin Modern font is:
190
191\starttyping
192[97] = {
193    boundingbox = { 34, -10, 474, 446 },
194    index       = 28,
195    name        = "a",
196    unicode     = 97,
197    width       = 490,
198}
199\stoptyping
200
201Another example is the following. Here we end up in private space:
202
203\starttyping
204[983059] = {
205    boundingbox = { 30, -10, 734, 446 },
206    index       = 19,
207    name        = "oe.dup",
208    unicode     = 339,
209    width       = 762,
210}
211\stoptyping
212
213Yet another entry is:
214
215\starttyping
216[306] = {
217   boundingbox = { 28, -22, 790, 683 },
218   index       = 357,
219   name        = "I_J",
220   unicode     = { 73, 74 },
221   width       = 839,
222  },
223\stoptyping
224
225Here you see two \UNICODE\ numbers. That kind of information is deduced from the
226name of the glyph, using knowledge on how such names are supposed to be
227constructed, or, when that is not possible, from ligature information in the
228fonts.
229
230It makes no sense to discuss the whole font table in detail, if only because most users
231will never (need to) see it. But if your curious you can have a look at the fonts
232in the cache tree, in the \CONTEXT\ distribution from the \CONTEXT\ garden this is
233
234\starttyping
235.../tex/texmf-cache/luatex-cache/context/<somehash>/fonts/otl
236\stoptyping
237
238There can be three kind of files there, with suffixes \type {tma}, \type {tmc}
239and \type {tmb}. The first one is the table as converted from the binary font
240file. The second and third variants are just bytecode compilations of this file
241(for \LUATEX\ and|/|or \LUAJITTEX). The bytecode variants are smaller but more
242important, they load a bit faster. On my disk the largest \type{tma} file is just
243below 10 MByte (an extensive \CJK\ font) but normally they are in the few hundred
244KByte range (some are real small), with the bytecode files of course being
245relatively small to their original.
246
247However, there is a bit of cheating here. If we run the command:
248
249\starttyping
250mtxrun --script font --convert lmroman10-regular.otf
251\stoptyping
252
253A \LUA\ file is generated: \type {lmroman10-regular.lua}. This file is much larger
254than the \type {tma} file in the cache:
255
256\starttabulate[|T|T|]
257\NC 643.924 \NC lmroman10-regular.lua \NC 0.029 \NR
258\NC 209.950 \NC lmroman10-regular.tma \NC 0.010 \NR
259\NC 121.541 \NC lmroman10-regular.tmb \NC \NR
260\NC 134.564 \NC lmroman10-regular.tmc \NC 0.003 \NR
261\stoptabulate
262
263The reason for this is the following. Most information is stored in tables.
264Especially tables that describe font features can be the same all over the place.
265This is why we pack the table in a more compact format before saving it in the
266cache, and unpack it after loading. The effects on loading are neglectable but
267and it has the benefit that it saves a lot of memory. By looking at such numbers
268one should be careful with conclusions, but (assuming proper garbage collection)
269we see a memory footprint of the \type {lua} file of 2836 Kbyte, while the
270unpacked variant takes 704 Kbyte. You can imagine what happens with large \CJK\
271fonts. Loading the (larger unpacked) \type {lua} file currently costs me 0.029
272seconds, while loading and unpacking the \type {tma} file takes 0.010 seconds and
273the bytecode variant \type {tmc} 0.003 seconds.
274
275\stopsubsection
276
277\startsubsection[title=Loading \TYPEONE\ fonts]
278
279When we started with \CONTEXT\ \MKIV\ (which is shortly after we started with
280\LUATEX) the only \TFM\ files that were loaded, were those to make virtual
281\UNICODE\ math fonts, awaiting real \OPENTYPE\ math fonts. Math fonts are kind
282of special with respect to metrics and such.
283
284For \TYPEONE\ text fonts we didn't use the \TFM\ files but went for parsing \AFM\
285files. That way we could use all the glyphs provided by fonts and not be limited
286to 256 slots. So, effectively we made them \UNICODE\ and similar to \OPENTYPE. Of
287course the only features were ligatures, kerns and some special ones like \TEX\
288ligatures and replacements. With the old loader code, we always made them base
289mode fonts, which means that processing was delegated to \TEX. In the new loader
290we implement ligatures and kerns as node mode features, which means that we can
291use those fonts in base mode as well as node mode. The last options therefore
292permits to add or adapt features to \TYPEONE\ fonts as well.
293
294In the next sections we will focus on \OPENTYPE\ but as the \TYPEONE\ fonts are
295organized in a similar way, some of it also applies to this older type. The most
296important to keep in mind is that we only have \type {liga}, \type {kern} and a
297few \CONTEXT\ specific features.
298
299\stopsubsection
300
301\stopsection
302
303\startsection[title=The tables]
304
305\startsubsection[title=Structure]
306
307Getting a font read for \TEX\ happens in stages. The original \OPENTYPE\ file is
308read only once. At that moment the shapes are described in the \type
309{descriptions} subtable while by the time that we pass the information to \TEX\
310they are in \type {characters}. The reason is that we go from dimensions in font
311units to dimensions in scaled points. We start with the following table:
312
313\ctxlua{context.tocontext(fonts.tables.data.original,"original_table")}
314
315The table passed \TEX\ is constructed from this one and looks like:
316
317\ctxlua{context.tocontext(fonts.tables.data.scaled,"scaled_table")}
318
319There might be a few more (often obscure) fields for special purposes. The
320characters subtable conforms to what \TEX\ expects, while the descriptions stays
321closer to \OPENTYPE. The \type {kerns} and \type {ligatures} subtables are there
322for base mode and are not present in \type {node} mode. The \type {commands} and
323\type {fonts} subtables relate to virtual fonts.
324
325\startitemize[packed]
326\startitem
327    Start with the (already) loaded \OPENTYPE\ table.
328\stopitem
329\startitem
330    Copy relevant information from \type {descriptions} to \type {characters} etc.
331\stopitem
332\startitem
333    Construct \type {properties} and \type {parameters} tables.
334\stopitem
335\startitem
336    Apply additional manipulators, for instance extend the \type {characters}
337    table, with expansion and protrusion.
338\stopitem
339\startitem
340    Scale the \type {characters}, \type {properties} and \type {parameters}.
341\stopitem
342\startitem
343    Apply additional manipulators.
344\stopitem
345\startitem
346    Pass the table to \TEX, but keep it around for later access.
347\stopitem
348\stopitemize
349
350One of the things you need to be aware of is that all references to glyphs are
351\UNICODE\ slots, either natural ones (representing a character) or a private one
352(representing an alternative representation). In \OPENTYPE\ features are defined
353in terms of glyph indices but we prefer \UNICODE\ because that is easier to deal
354with when we run over the node list. Before font processing the character field
355in a glyph node is a \UNICODE\ slot and afterwards it's still a \UNICODE\ but
356when it's a private one it can always be resolved to a non private slot of
357sequence of slots. Of course that could also be done with indices but it's just
358more natural this way.
359
360Another thing to note is that in the descriptions we're still working with font
361units ranging from $-1000$ to $+1000$, $-2048$ to $+2048$ or similar ranges. At
362the \TEX\ end we need scaled points which are much larger numbers.
363
364The question is: how often do users need to access the raw data in a font? After
365a decade of \MKIV\ and \LUATEX\ hardly any user has requested such access,
366probably because when needed easier interfaces were provided. Also, in the
367\CONTEXT\ distrubution there are some examples of manipulations that can be
368copied and adapted to personal use. There's also a danger is messing with the
369fonts (similar messing with the node lists): you never know how it interferes
370with other (maybe future) features.
371
372If you still want to do it, best is probably to start with saving the
373to|-|be|-|passed|-|to|-|\TEX\ table in a file and have a look at it. The most
374prominent subtable is the \type {characters} table and messing a bit with
375dimensions is rather harmless. You could add characters, for instance virtual
376ones, which again is harmless unless you use invalid commands. You probably want
377to stay away from the resources subtable, if only because some of its subtables
378are shared and therefore adapting them can have side effects. The top level \type
379{shared} and \type {unscaled} subtable are off limits as is the \type
380{specification}.
381
382You can save a font by consulting one of the hashes but for a specific font
383you need to know its id. You can do this by using low level accessors but better
384is to use the helpers made for this, because they prevent saving redundant
385data.
386
387% \starttyping
388% \startluacode
389% local nullfont    = fonts.hashes.identifiers[false]
390% local currentfont = fonts.hashes.identifiers[true]
391%
392% local id, tfmdata = fonts.definers.define {
393%     name = "dejavusansmono*default",
394%     size = tex.sp("6pt")
395% }
396%
397% table.save("temp-nullfont.lua",   nullfont)
398% table.save("temp-currentfont.lua",currentfont)
399% table.save("temp-definedfont.lua",tfmdata)
400% table.save("temp-definedfont.lua",fonts.hashes.identifiers[id])
401% \stopluacode
402% \stoptyping
403
404\starttyping
405\startluacode
406fonts.tables.save  {
407    filename = "temp-font-scaled.lua",
408    fontname = "dejavusansmono*default",
409    method   = "original",
410}
411\stopluacode
412\stoptyping
413
414At the \TEX\ end you can use:
415
416\starttyping
417\savefont
418  [name=dejavusansmono*default,
419   file=temp-o.lua,
420   method=original]
421\savefont
422  [name=dejavusansmono*default,
423   file=temp-s.lua,
424   method=scaled]
425\stoptyping
426
427When no \type {name} is given, the current font is used and when no \type {file}
428is given a filename is made up. The default \type {method} is \type {scaled}. The
429saved name is reported.
430
431\stopsubsection
432
433\startsubsection[title=Plug-ins]
434
435There are several places where you can hook in code: before scaling
436(initalizers), after scaling (manipulators) and while processing (processors).
437Only the first two are meant for tweaks.
438
439\starttyping
440local do_something = {
441    name        = "something",
442    description = "doing something",
443    initializers = {
444     -- position = 1,
445        base     = function(tfmdata,value,features) ... end,
446        node     = function(tfmdata,value,features) ... end,
447    },
448    manipulators = {
449     -- position = 1,
450        base     = function(tfmdata,feature,value) ... end,
451        node     = function(tfmdata,feature,value) ... end,
452    },
453    processors = {
454     -- position = 1,
455        base     = function(tfmdata,font,attr) ... end,
456        node     = function(tfmdata,font,attr) ... end,
457    }
458}
459
460fonts.constructors.features.register.otf(so_something)
461fonts.constructors.features.register.afm(so_something)
462\stoptyping
463
464A \type {initializer} is applied just before the font gets scaled. This means
465that the characterm properties and parameters are unscaled! Initializers can for
466instance be used to add extra features to fonts. You can provide an \type
467{position} key with a number to force a place in the list of initializers but of
468course you can never be sure of interference.
469
470A \type {manipulator} is applied when the font is scaled but before it gets
471passed to \TEX. It's a good place to tweak dimensions. Here you can also probide
472a \type {position}.
473
474The processors are applied when the node list gets processed, hence the \type
475{font} and optional \type {attr} arguments. The action is only applied to the
476specified font (id) and when an attribute gets passed, this is tested for a
477value. When an attribute is used, an unset attribute on the node will skip the
478action.
479
480If adapting characters and their properties is your main objetive, then there is a
481better plugin mechanism using sequencers. We illustrate this with a fake example:
482
483\starttyping
484\startluacode
485
486function document.b_copying(tfmdata)
487    logs.report("fonts","before copying: %s",tfmdata.properties.filename)
488end
489function document.a_copying(tfmdata)
490    logs.report("fonts","after copying: %s",tfmdata.properties.filename)
491end
492
493function document.b_math(tfmdata)
494    logs.report("fonts","before math: %s",tfmdata.properties.filename)
495end
496function document.a_math(tfmdata)
497    logs.report("fonts","after math: %s",tfmdata.properties.filename)
498end
499
500utilities.sequencers.appendaction(
501    "beforecopyingcharacters",
502    "before",
503    "document.a_copying"
504)
505
506utilities.sequencers.appendaction(
507    "aftercopyingcharacters",
508    "after",
509    "document.b_copying"
510)
511
512utilities.sequencers.appendaction(
513    "mathparameters",
514    "before",
515    "document.b_math"
516)
517
518utilities.sequencers.appendaction(
519    "mathparameters",
520    "after",
521    "document.a_math"
522)
523\stopluacode
524\stoptyping
525
526When we call the next command:
527
528\starttyping
529\definedfont[MathRoman at 3pt]
530\stoptyping
531
532we get this reported:
533
534\starttyping
535fonts > before math: ...../public/dejavu/texgyredejavu-math.otf
536fonts > after math: ...../public/dejavu/texgyredejavu-math.otf
537fonts > after copying: ...../public/dejavu/texgyredejavu-math.otf
538fonts > before copying: ...../public/dejavu/texgyredejavu-math.otf
539\stoptyping
540
541In between \type {before} and \type {after} we have \type {system} which is
542reserved for \CONTEXT\ actions. These actions are executed in the scaler
543function. The function get two tables passed: the original data as well as the
544target. If you ever need these hooks, you can probably best run an \type
545{inspect} on these arguments to see what you're dealing with.
546
547Fonts get reused when possible and for that a hash is calculated depending on the
548enabled features and size. If for some reason you want to adapt that hash you can
549use postprocessors. When the \type {tfmdata} table has a subtable \type
550{postprocessors}, then the actions in that subtable will be applied. When an
551action returns a string, the string will be combined with the hash. You can set
552(o rextend) the postprocessors table using the previopusly mentioned commands.
553However, in \CONTEXT\ you can best stay away from this as it might interfere. This
554mechanism is mostly provided for generic use.
555
556\stopsubsection
557
558\stopsection
559
560\startsection[title=Goodies]
561
562The font goodies are already discussed as an official mechanism to extend or enhance
563fonts with additional features. There are quite some goodies defined and for sure more will
564show up. Here is the full repertoire:
565
566\ctxlua{context.tocontext(fonts.tables.data.goodies,"goodie_table")}
567
568Of course you will never use all the options at the same time. The best place to
569look for examples are the \type {lfg} files in the \CONTEXT\ distribution.
570\footnote {At some point we might decide to also support goodies in the generic
571version.}
572
573\stopsection
574
575\startsection[title=Extra characters]
576
577When \TEX\ loads a font it gets stored in an internal format. Although \LUATEX\
578can still load \TFM\ files, normally we will load font data using \LUA\ and then
579pass it to \TEX. When that is done the font is basically frozen. After all, once
580you entered text and that text has been processed to a sequence of glyphs, the
581backend has to be sure that it can include the right data in the result. What
582matters there is:
583
584\startitemize[packed]
585\startitem the width of a glyph \stopitem
586\startitem the index of a glyph in the font \stopitem
587\startitem properties of a font, like its name \stopitem
588\startitem all kind manipulations don't with a virtual glyph\stopitem
589\stopitemize
590
591So, changing an already used glyph is not done. But, adding a new one should not
592be a big deal. So, starting with \LUATEX\ 1.0.5 we can add characters (that
593become glyphs) to a font after it got passed to \TEX.
594
595Of course there are some limitations to this. For instance, when \OPENTYPE\
596features are needed, you also need to extend that bit and it's not that trivial.
597But adding independent special characters is no problem. Also, you can decide to
598replace an already processed glyph by another one newly created with the same
599dimensions but a different rendering.
600
601Here I'll give a few (simple) examples. First we define a helper that creates a
602rule. After that we use all kind of \CONTEXT\ data structures and helpers but the
603general setup is not that complicated.
604
605\startbuffer
606\startluacode
607    function document.MakeGlyph(w)
608        local v = 655360
609        local w = w*v
610        local h = v
611        local d = v/3
612        return {
613            width  = w,
614            height = h,
615            depth  = d,
616            commands = {
617                { "down", d },
618                { "rule", h + d, w }
619            },
620        }
621    end
622\stopluacode
623\stopbuffer
624
625\typebuffer \getbuffer
626
627Of course, when one defines a font to be passed to \TEX\ it needs to conform to
628the standard. The \LUATEX\ manual describes what parameters and other properties
629have to be set. We cheat and copy the so called null font. We also create a fonts
630sub table. In such a table, a reference to id zero will be resolved to the id of
631the font.
632
633After defining the font table, we pass it to \TEX\ with \type {font,define} watch
634how we also pass an already reserved id. Then we add some characters and we also
635replace character 122 which is no problem because, after all, we haven't used it
636yet. We just use numbers so don't think we're doing \UNICODE\ here.
637
638\startbuffer
639\startluacode
640    local fontdata = fonts.hashes.identifiers
641
642    local id = font.nextid(true) -- true reserves the id
643    local nf = table.copy(fontdata[0])
644
645    local make = document.MakeGlyph
646    local add  = font.addcharacters
647
648    nf.name       = "foo"
649    nf.type       = "virtual"
650    nf.fonts      = { { id = 0 } }
651    nf.characters = {
652        [122] = make(1),
653        [123] = make(2),
654    }
655
656    font.define(id,nf)
657
658    fontdata[id] = nf
659
660    local t = make(3)
661    nf.characters[124] = t
662    add(id, {
663        fonts      = nf.fonts,
664        characters = { [124] = t }
665    })
666
667    local t = make(4)
668    nf.characters[125] = t
669    add(id, {
670        fonts      = nf.fonts,
671        characters = { [125] = t }
672    })
673
674    local t = make(8)
675    nf.characters[122] = t
676    add(id, {
677        fonts      = nf.fonts,
678        characters = { [122] = t }
679    })
680
681    interfaces.setmacro("MyDemoFontId",id)
682\stopluacode
683\stopbuffer
684
685\typebuffer \getbuffer
686
687\startbuffer
688\def\MyDemoFont{\setfontid\MyDemoFontId}
689\stopbuffer
690
691We also define a command to access this font:
692
693\typebuffer \getbuffer
694
695\startbuffer
696{\MyDemoFont \type{[122=}\char122\type{]}}
697{\MyDemoFont \type{[123=}\char123\type{]}}
698{\MyDemoFont \type{[124=}\char124\type{]}}
699{\MyDemoFont \type{[125=}\char125\type{]}}
700\stopbuffer
701
702and we test this font as follows:
703
704\typebuffer
705
706This gives:
707
708\startlines \getbuffer \stoplines
709
710Next we extend an existing font and demonstrate several methods for extending a
711font. First we define a font that we will patch.
712
713\startbuffer
714\definefontfeature[myextension-one][default][myextension=1]
715
716\definefont[MyDemoOne][Serif*myextension-one]
717\stopbuffer
718
719\typebuffer \getbuffer
720
721\startbuffer
722\startluacode
723    local currentfont   = font.current()
724    local fontdata      = fonts.hashes.identifiers[currentfont]
725    local characters    = fontdata.characters
726    local cfonts        = fontdata.fonts
727    local addcharacters = font.addcharacters
728
729    local make = document.MakeGlyph
730
731    local function startextension()
732        statistics.starttiming()
733    end
734
735    local function stopextension(n)
736        context.NC() context.formatted.type("\\char%s",n)
737        context.NC() context.char(n)
738        context.NC() context("%0.3f",statistics.stoptiming())
739        context.NC() context.NR()
740    end
741
742    context.starttabulate { "||||" }
743
744    startextension()
745        for i=1000,1999 do
746            local t = make(3)
747            characters[i] = t
748            addcharacters(currentfont, {
749                fonts      = cfonts,
750                characters = { [i] = t }
751            })
752        end
753    stopextension(1500)
754
755    startextension()
756        local t = {
757            fonts      = cfonts,
758            characters = characters
759        }
760        for i=2000,2999 do
761            characters[i] = make(5)
762            addcharacters(currentfont, t)
763        end
764    stopextension(2500)
765
766    startextension()
767        for i=3000,3999 do
768            characters[i] = make(7)
769        end
770        addcharacters(currentfont, {
771            fonts      = cfonts,
772            characters = characters
773        })
774    stopextension(3500)
775
776    startextension()
777        local t = { }
778        for i=4000,4999 do
779            characters[i] = make(9)
780            t[i] = characters[i]
781        end
782        addcharacters(currentfont, {
783            fonts      = cfonts,
784            characters = t
785        })
786    stopextension(4500)
787
788    local addcharacters = fonts.constructors.addcharacters
789
790    startextension()
791        local t = { }
792        for i=5000,5999 do
793            t[i] = make(11)
794        end
795        addcharacters(currentfont, {
796            fonts      = cfonts,
797            characters = t
798        })
799    stopextension(5500)
800
801    context.stoptabulate()
802\stopluacode
803\stopbuffer
804
805Watch how we only pass the new characters. We also need to make sure that the
806table at the \LUA\ end gets updated, because we might need the data later. You
807can see that not all methods are equally efficient. The last extension uses a
808helper that also makes sure that the main character table at the \LUA\ end gets
809updated.
810
811\typebuffer \start \MyDemoOne \getbuffer \stop
812
813\startbuffer
814\startluacode
815    local addcharacters = fonts.constructors.addcharacters
816    local currentfont   = font.current()
817    local parameters    = fonts.hashes.parameters[currentfont]
818
819    local m = metapost.simple
820    local f = string.formatters
821        ["draw fullsquare rotated %i scaled %b randomized 2bp withcolor %s"]
822
823    local push = { "push" }
824    local pop  = { "pop" }
825
826    function make()
827        local r = parameters.size
828        local o = r/2
829        local p1 = m("metafun",f( 0, r, "red"))
830        local p2 = m("metafun",f(30, r, "green"))
831        local p3 = m("metafun",f(60, r, "blue"))
832        return {
833            width  = o + r,
834            height = 2*o,
835            depth  = o,
836            commands = {
837                { "down", -o/2 }, { "right", o/2 + o },
838                push, { "pdf", "origin", p1 }, pop,
839                push, { "pdf", "origin", p2 }, pop,
840                push, { "pdf", "origin", p3 }, pop,
841            },
842        }
843    end
844
845    local t = { }
846    for i=6000,6010 do
847        t[i] = make()
848    end
849    addcharacters(currentfont, {
850        fonts      = cfonts,
851        characters = t
852    })
853\stopluacode
854\stopbuffer
855
856In this example we use \METAPOST\ to generate a shape. There is some juggling
857with dimensions and we need to shift the image in order to get a proper baseline.
858
859\typebuffer \start \MyDemoOne \showglyphs \getbuffer \stop
860
861These shapes show up as follows. We show the bounding box too:
862
863\startbuffer
864\scale [width=\textwidth] {%
865    \char6000 \space
866    \char6001 \space
867    \char6002 \space
868    \char6003
869}
870\stopbuffer
871
872\typebuffer \startlinecorrection \MyDemoOne \showglyphs \getbuffer \stoplinecorrection
873
874When defining virtual characters you need to keep in mind that there are limits to
875how large a character can be. If you go too far \LUATEX\ will quit with a scale
876related message.
877
878In \CONTEXT\ there are a couple of mechanism that were implemented when \LUATEX\
879came around that can be updated to use the tricks described here. I'm not sure if
880I'll do that. After all, it works now too. The main benefit of the fact that we
881can extend a font within reasonable bounds is that future mechanism can benefit
882from it.
883
884There is one thing to keep in mind. Say that we define a font like this:
885
886\starttyping
887\definefont[MyDemoOneX][Serif*myextension-one]
888\stoptyping
889
890Because we already defined a font with the same size, we automatically get the characters
891\type {6000} upto \type {6003}. If we don't want this and want a fresh instance, we can do:
892
893\starttyping
894\definefontfeature[myextension-two][default][myextension=2]
895\definefont[MyDemoOneX][Serif*myextension-two]
896\stoptyping
897
898or just:
899
900\starttyping
901\definefont[MyDemoOneX][Serif*default]
902\stoptyping
903
904Normally this kind of hackery is not arbitrary and part of a well designed set up
905so one knows what one gets.
906
907\stopsection
908
909% - features
910% - subfonts
911% - outlines
912% - math
913% - hashes
914
915\stopchapter
916
917\stopcomponent
918