1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324\environment luametatexstyle
325
326\startdocument[title=Nodes]
327
328\startsection[title={Introduction}]
329
330The (to be) typeset content is collected in a double linked list of so called
331nodes. A node is an array of values. When looked at from the \LUA\ end you can
332either seen them as \type [option=LUA] {<t:userdata>} or as \type [option=LUA]
333{<t:integer>}. In the case of userdata you access fields like this:
334
335\starttyping [option=LUA]
336local width = foo.width foo is userdata
337\stoptyping
338
339while the indexed variant uses:
340
341\starttyping [option=LUA]
342local width = nodes.direct.getwidth(foo) foo is an integer
343\stoptyping
344
345In \CONTEXT\ we mostly use the second variant but its a matter of taste so users
346can you whatever they like most. When you print a userdata node you see something
347like this:
348
349\startlines
350\startluacode
351context.type(tostring(node.new("glyph"))) context.par()
352context.type(tostring(node.new("hlist"))) context.par()
353context.type(tostring(node.new("glue" ))) context.par()
354\stopluacode
355\stoplines
356
357The number in the middle is the one you would also see if you use the indexed
358approach and often these numbers are kind of large. A number \type {13295}
359doesnt mean that we have that many nodes. The engine has a large array of memory
360words (pairs of 32 bit integers) and a node is a slice of then with the index
361pointing to where we start. So, if we have a node that has 5 value pairs, the
362slice runs from \type {13295} upto \type {13299} that consume 40 bytes.
363
364In this chapter we introduce the nodes that are exposed to the user. We will
365discuss the relevant fields as well as ways to access them. Because there are
366similar fields in different nodes, we can share accessors.
367
368It is important to notice that not all fields that can be accessed (set and get)
369are under full user control. For instance, in math we have a \type {noad} type
370that is actually shared between several construct (like atoms, accents and
371fences) and not all parameters make sense for each of them. Some properties are
372set while the formula is assembled. It fits in the \LUAMETATEX\ concept to open
373up everything but abusing this can lead to side effects. It makes no sense to add
374all kind of safeguards against wrong or unintended usage because in the end only
375a few users will go that low level anyway.
376
377Not all fields mentioned are accessible in the userdata variant. It is also good
378to notice that some fields are fabricated, for instance \type {total} is the sum
379of \type {height} and \type {depth}.
380
381\stopsection
382
383\startsection[title={\LUA\ node representation}]
384
385As mentioned, nodes are represented in \LUA\ as user data objects with a variable
386set of fields or by a numeric identifier when requested and we showed that when
387you print a node user data object you will see these numbers.
388
389\startfourrows
390\ctxlua{moduledata.node.codes("types")}
391\stopfourrows
392
393You can ask for a list of fields with \type {node.fields} and for valid subtypes
394with \type {node.subtypes}. There are plenty specific field values and you can
395some idea about them by calling \type {tex.get*values()} which returns a table if
396numbers (exclusive numbers or bits). We use these to get the tables that are
397shown with each node type.
398
399There are a lot of helpers and below we show them per node type. In later
400sections some will come back organized by type of usage. Trivial getters and
401setters will not be discussed. Its good to know that some getters take more
402arguments where the second one can for instance trigger more return values. The
403number of arguments to a setter can also be more than a few. As with everything
404\LUAMETATEX\ the \CONTEXT\ sources can also be seen as a reference.
405
406\stopsection
407
408\startsection[title={Main text nodes}]
409
410These are the nodes that comprise actual typesetting commands. A few fields are
411present in all nodes regardless of their type, these are: \type {next}, \type
412{id} and \type {subtype}. The \type {subtype} is sometimes just a dummy entry
413because not all nodes actually use the \type {subtype}, but this way you can be
414sure that all nodes accept it as a valid field name, and that is often handy in
415node list traversal. In the following tables \type {next} and \type {id} are not
416explicitly mentioned. Besides these three fields, almost all nodes also have an
417\type {attr} field, and there is a also a field called \type {prev}.
418
419\startsubsection[title={hlist and vlist, aka boxes}]
420
421These lists share fields and subtypes although some subtypes can only occur in
422horizontal lists while others are unique for vertical lists.
423
424\showenginefields {hlist}
425\showenginesubtypes {hlist}
426\showenginevalues {directionvalues}
427\showenginevalues {listgeometryvalues}
428\showenginevalues {listanchorvalues}
429\showenginevalues {listsignvalues}
430
431The \type {shift} is a displacement perpendicular to the character (horizontal) or
432line (vertical) progression direction.
433
434The \type {orientation}, \type {woffset}, \type {hoffset}, \type {doffset}, \type
435{xoffset} and \type {yoffset} fields are special. They can be used to make the
436backend rotate and shift boxes which can be handy in for instance vertical
437typesetting. Because they relate to (and depend on the) the backend they are not
438discussed here (yet). The \type {pre} and \type {post} fields refer to migrated
439material in both list types, while the adjusted variants only make sense in
440horizontal lists.
441
442\showengineusagepernode{hlist}
443
444\stopsubsection
445
446\startsubsection[title={rule}]
447
448Contrary to traditional \TEX, \LUATEX\ has more subtypes subtypes because we also
449use rules to store reuseable objects and images. However, in \LUAMETATEX\ these
450are gone but we reserve these subtypes. Apart form the basic rules a lot is up to
451the backend.
452
453\showenginefields {rule}
454\showenginesubtypes {rule}
455\showenginevalues {ruleoptionvalues}
456
457The width, height and depth of regular rules defaults to the special value of
458$\cldcontext {tex . magicconstants . runningrule}$ which indicates a running rule
459that adapts its dimensions to the box that it sits in.
460
461The \type {left} and type {right} keys are somewhat special (and experimental).
462When rules are auto adapting to the surrounding box width you can enforce a shift
463to the right by setting \type {left}. The value is also subtracted from the width
464which can be a value set by the engine itself and is not entirely under user
465control. The \type {right} is also subtracted from the width. It all happens in
466the backend so these are not affecting the calculations in the frontend (actually
467the auto settings also happen in the backend). For a vertical rule \type {left}
468affects the height and \type {right} affects the depth. There is no matching
469interface at the \TEX\ end (although we can have more keywords for rules it would
470complicate matters and introduce a speed penalty.) However, you can just
471construct a rule node with \LUA\ and write it to the \TEX\ input. The \type
472{outline} subtype is just a convenient variant and the \type {transform} field
473specifies the width of the outline. The \type {xoffset} and \type {yoffset}
474fields can be used to shift rules. Because they relate to (and depend on the) the
475backend they are not discussed here (yet). Of course all this assumes that the
476backend deals with it. Internally fields with different names can use the same
477variable, depending on the subtype; dedicated names just make more sense.
478
479\showengineusagepernode{rule}
480
481\stopsubsection
482
483\startsubsection[title={insert}]
484
485This node relates to the \type {\insert} primitive and support the fields:
486
487\showenginefields {insert}
488
489Here the subtype indicates the class of the insert and that number is also used
490to access the box, dimen and skip registers that relate to the insert, if we
491use inserts in the traditional way.
492
493\showengineusagepernode{insert}
494
495\stopsubsection
496
497\startsubsection[title={mark}]
498
499This one relates to the \type {\marks} primitive and only has a few fields, one
500being a token list as field which is kind of rare.
501
502\showenginefields {mark}
503\showenginesubtypes {mark}
504
505\showengineusagepernode{mark}
506
507\stopsubsection
508
509\startsubsection[title={adjust}]
510
511This node results from \type {\vadjust} usage:
512
513\showenginefields {adjust}
514\showenginesubtypes {adjust}
515\showenginevalues {adjustoptionvalues}
516
517\showengineusagepernode{vadjust}
518
519\stopsubsection
520
521\startsubsection[title={disc (discretionary)}]
522
523The \typ {\discretionary}, \typ {\explicitdiscretionary} and \typ
524{\automaticdiscretionary} primitives as well as the discretionary that comes from
525hyphenation all have the pre, post and replace lists. Because these lists have
526head and tail pointers the getters and setters handle this for you.
527
528\showenginefields {disc}
529\showenginesubtypes {disc}
530\showenginevalues {discoptionvalues}
531
532\showengineusagepernode{disc}
533
534\stopsubsection
535
536\startsubsection[title={math}]
537
538Math nodes represent the boundaries of a math formula, normally wrapped between
539\type {$} and \type {$}. The glue fields are only used when the \type {surround}
540field is zero.
541
542\showenginefields {math}
543\showenginesubtypes {math}
544
545\showengineusagepernode{math}
546
547\stopsubsection
548
549\startsubsection[title={glue}]
550
551Skips are about the only type of data objects in traditional \TEX\ that are not a
552simple value. They are inserted when \TEX\ sees a space in the text flow but also
553by \type {\hskip} and \type {skip}. The structure that represents the glue
554components of a skip internally is called a \type {gluespec}. In \LUAMETATEX\ we
555dont use the spec itself but just its values.
556
557\showenginefields {glue}
558\showenginesubtypes {glue}
559\showenginevalues {glueoptionvalues}
560
561Note that we use the key \type {width} in both horizontal and vertical glue. This
562suited the \TEX\ internals well so we decided to stick to that naming.
563
564The effective width of some glue subtypes depends on the stretch or shrink needed
565to make the encapsulating box fit its dimensions. For instance, in a paragraph
566lines normally have glue representing spaces and these stretch or shrink to make
567the content fit in the available space. The \type {effectiveglue} function that
568takes a glue node and a parent (hlist or vlist) returns the effective width of
569that glue item. When you pass \type {true} as third argument the value will be
570rounded.
571
572\showengineusagepernode{glue}
573
574\stopsubsection
575
576\startsubsection[title={gluespec}]
577
578Internally \LUAMETATEX\ (like its ancestors) also uses nodes to store data that
579is not seen in node lists. For instance the state of expression scanning (\type
580{\dimexpr} etc.) and conditionals (\type {\ifcase} etc.) is also kept in lists of
581nodes. A glue, which has five components, is stored in a node as well, so, where
582most registers store just a number, a skip register (of internal quantity) uses a
583pointer to a glue spec node. It has similar fields as glue nodes, which is not
584surprising because in the past (and other engines than \LUATEX) a glue node also
585has its values stored in a glue spec. This has some advantages because often the
586values are the same, so for instance spacing related skips were not resolved
587immediately but pointed to the current value of a space related internal register
588(like \type {\spaceskip}). But, in \LUATEX\ and therefore \LUAMETATEX\ we do
589resolve these quantities immediately and we put the current values in the glue
590nodes.
591
592\showenginefields {gluespec}
593
594You will only find these nodes in a few places, for instance when you query an
595internal quantity. In principle we could do without them as we have interfaces
596that use the five numbers instead. For compatibility reasons we keep glue spec
597nodes exposed but this might change in the future. Of course there are no
598subtypes here because its just a data store.
599
600\showengineusagepernode{gluespec}
601
602\stopsubsection
603
604\startsubsection[title={kern}]
605
606The \type {\kern} command creates such nodes but for instance the font and math
607machinery can also add them.
608
609\showenginefields {kern}
610\showenginesubtypes {kern}
611
612\showengineusagepernode{kern}
613
614\stopsubsection
615
616\startsubsection[title={penalty}]
617
618The \type {\penalty} command is one that generates these nodes. There is not much
619to tell about them, apart from that in \LUAMETATEX\ they have options and a
620possible spread related \type {nepalty} field that is used internally.
621
622\showenginefields {penalty}
623\showenginesubtypes {penalty}
624\showenginevalues {penaltyoptionvalues}
625
626\showengineusagepernode{penalty}
627
628\stopsubsection
629
630\startsubsection[title={glyph}]
631
632These are probably the mostly used nodes and although you can push them in the
633current list with for instance \prm {char} \TEX\ will normally do it for you when
634it considers some input to be text. Glyph nodes are relatively large and have many
635fields.
636
637\showenginefields {glyph}
638\showenginesubtypes {glyph}
639\showenginevalues {glyphoptionvalues}
640\showenginevalues {glyphdiscvalues}
641\showenginevalues {discpartvalues}
642\showenginevalues {glyphprotectionvalues}
643
644The \type {width}, \type {height} and \type {depth} values are readonly.
645In \LUATEX\ \type {expansion} has been introduced as part of the separation between
646front and backend. It is the result of extensive experiments with a more
647efficient implementation of expansion. Early versions of \LUATEX\ already
648replaced multiple instances of fonts in the backend by scaling but contrary to
649\PDFTEX\ in \LUATEX\ we now also got rid of font copies in the frontend and
650replaced them by expansion factors that travel with glyph nodes. Apart from a
651cleaner approach this is also a step towards a better separation between front
652and backend.
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670\showengineusagepernode{glyph}
671
672\stopsubsection
673
674\startsubsection[title={boundary}]
675
676This node relates to the \type {\noboundary}, \type {\boundary}, \type
677{\protrusionboundary}, \type {\wordboundary} etc. These are relative small nodes
678that determine what happens before and after them.
679
680\showenginefields {boundary}
681\showenginesubtypes {boundary}
682\showenginevalues {protrusionboundaryvalues}
683
684\showengineusagepernode{boundary}
685
686\stopsubsection
687
688\startsubsection[title={par}]
689
690This node is inserted at the start of a paragraph. You should not mess too much
691with this one. They are also inserted when \type {\local...} primitives are used
692that relate boxes to positions in the line and overload certain parameters that
693play a role in the line break routine. There are many fields!
694
695\showenginefields[2]{par}
696\showenginesubtypes {par}
697
698\showengineusagepernode{par}
699
700\stopsubsection
701
702\startsubsection[title={dir}]
703
704Direction nodes mark parts of the running text that need a change of direction
705and the \type {\textdirection} command generates them. Contrary to \LUATEX\
706we only have two directions.
707
708\showenginefields {dir}
709\showenginesubtypes {dir}
710
711\showengineusagepernode{dir}
712
713\stopsubsection
714
715\startsubsection[title={whatsit}]
716
717A whatsit node is a real simple one and it only has a subtype. It is even less
718than a user node (which it actually could be) and uses hardly any memory. What
719you do with it it entirely up to you: its is real minimalistic. You can assign a
720subtype and it has attributes. It is all up to the user (and the backend) how
721they are handled.
722
723\showenginefields {whatsit}
724
725\showengineusagepernode{whatsit}
726
727\stopsubsection
728
729\startsubsection[title={attribute}]
730
731This is a small node but used a lot. When an attribute is set and travels with a
732node, we actually have a forward (only) linked list with a head node that keeps a
733reference count. These lists are (to be) sorted by attribute index. Normally you
734will {\em not} mess directly with these list because you can get unwanted side
735effects.
736
737\showenginefields {attribute}
738\showenginesubtypes {attribute}
739
740\showengineusagepernode{attribute}
741
742\stopsubsection
743
744\startsubsection[title={alignrecord}]
745
746This node can be encountered in alignments and will eventually become a \type
747{hlist} or \type {vlist} node. It therefore has the same size and fields as those
748nodes. However, the following fields are overloaded by other parameters: \type
749{woffset}, \type {hoffset}, \type {doffset}, \type {xoffset}, \type {yoffset},
750\type {orientation}, \type {pre} and \type {post}. Be careful!
751
752\showenginefields {alignrecord}
753
754\showengineusagepernode{alignrecord}
755
756\stopsubsection
757
758\startsubsection[title={unset}]
759
760This node can be encountered in alignments and will eventually become a \type
761{hlist} or \type {vlist} node. It therefore has the same size and fields as those
762nodes. However, the following fields are (at least temporarily) there and they
763use the slots of \type {woffset}, \type {hoffset}, \type {doffset} and \type
764{orientation}. Be careful!
765
766\showenginefields {unset}
767
768\showengineusagepernode{unset}
769
770\stopsubsection
771
772\stopsection
773
774\startsection[title={Math nodes}]
775
776\startsubsection[title=The concept]
777
778Many object fields in math mode are either simple characters in a specific family
779or math lists or node lists: \type {mathchar}, \type {mathtextchar}, {subbox}
780and \type {submlist} and \type {delimiter}. These are endpoints and therefore the
781\type {next} and \type {prev} fields of these these subnodes are unused.
782
783There is a subset of nodes dedicated to math called noads. These are used for
784simple atoms, fractions, fences, accents and radicals. When you enter a formula,
785\TEX\ creates a node list with regular (math) nodes and noads. Then it hands over
786the list the math processing engine. The result of that is a nodelist without
787noads. Most of the noads contain subnodes so that the list of possible fields is
788actually quite small. Math formulas are both a linked list and a tree. For
789instance in $e = mc2$ there is a linked list \type {e = m c} but the \type {c}
790has a superscript branch that itself can be a list with branches.
791
792Eventually I might give a more detailed description of the differences between
793the five noad variants but for now the following has to do. One will quite likely
794not set that many fields at the \LUA\ end but running over the many sub lists can
795make sense. One has to know what the engine is doing anyway.
796
797\stopsubsection
798
799\startsubsection[title=noad]
800
801First, there are the objects (the \TEX book calls them \quote {atoms}) that are
802associated with the simple math objects: ord, op, bin, rel, open, close, punct,
803inner, over, under, vcenter. These all have the same fields, and they are
804combined into a single node type with separate subtypes for differentiation.
805However, before reading on you should realize that \LUAMETATEX\ has an extended
806math engine. We hale not only more classes, we also have many more keys in the
807nodes. We wont cover these details here.
808
809\showenginefields {noad}
810\showenginesubtypes {noad}
811\showenginevalues {noadoptionvalues}
812
813In addition to the subtypes (related to classes) that the engines knows of, there
814can be user defined subtypes. Not all fields make sense for every derives noad:
815\type {accent}, \type {fence}, \type {fraction} or \type {radical} but there we
816(currently) only mention the additional ones. These additional fields are taken
817from a pool of extra fields. Not all fields are always accessible for these nodes.
818
819\showengineusagepernode{noad}
820
821\stopsubsection
822
823\startsubsection[title=mathchar]
824
825The \type {mathchar} is the simplest subnode field, it contains the character and
826family for a single glyph object. The family eventually resolves on a reference
827to a font. Internally this nodes is one of the math kernel nodes.
828
829\showenginefields {mathchar}
830
831\showenginevalues {kerneloptionvalues}
832
833\showengineusagepernode{mathchar}
834
835\stopsubsection
836
837\startsubsection[title=mathtextchar]
838
839The \type {mathtextchar} is a special case that you will not normally encounter,
840it arises temporarily during math list conversion (its sole function is to
841suppress a following italic correction). Internally this nodes is one of the math
842kernel nodes.
843
844\showenginefields {mathtextchar}
845
846\showenginevalues {kerneloptionvalues}
847
848\showengineusagepernode{mathtextchar}
849
850\stopsubsection
851
852\startsubsection[title=subbox]
853
854These \type {subbox} subnode is used for subsidiary list items where
855the \type {list} points to a \quote {normal} vbox or hbox.
856
857\showenginefields {subbox}
858
859
860\showengineusagepernode{subbox}
861
862\stopsubsection
863
864\startsubsection[title=submlist]
865
866In \type {submlist} subnode the \type {list} points to a math list that is yet to
867be converted. Their fields
868
869\showenginefields {submlist}
870
871
872\showengineusagepernode{submlist}
873
874\stopsubsection
875
876\startsubsection[title={delimiter}]
877
878There is a fifth subnode type that is used exclusively for delimiter fields. As
879before, the \type {next} and \type {prev} fields are unused, but we do have:
880
881\showenginefields {delimiter}
882
883
884The fields \type {largechar} and \type {largefamily} can be zero, in that case
885the font that is set for the \type {smallfamily} is expected to provide the large
886version as an extension to the \type {smallchar}.
887
888\showengineusagepernode{delimiter}
889
890\stopsubsection
891
892\startsubsection[title={accent}]
893
894Accent nodes deal with stuff on top or below a math constructs.
895
896\showenginefields {accent}
897\showenginesubtypes {accent}
898
899{\em For more fields see \type {noad}. At some point we might move fields from
900that list to here but only when the engine also gets that split.}
901
902\showengineusagepernode{accent}
903
904\stopsubsection
905
906\startsubsection[title={style}]
907
908These nodes are signals to switch to another math style. Currently the subtype is
909actually used to store the style but dont rely on that for the future.
910
911\showenginefields {style}
912
913\showenginevalues {mathstylenamevalues}
914\showenginevalues {mathstylevalues}
915
916\showengineusagepernode{style}
917
918\stopsubsection
919
920\startsubsection[title={parameter}]
921
922These nodes are used to (locally) set math parameters. The subtype reflects a math style.
923
924\showenginefields {parameter}
925
926\showengineusagepernode{parameter}
927
928\stopsubsection
929
930\startsubsection[title={choice}]
931
932Most of the fields of this node are lists. Depending on the subtype different
933field names are used.
934
935\showenginefields {choice}
936\showenginesubtypes {choice}
937
938\showengineusagepernode{choice}
939
940\stopsubsection
941
942\startsubsection[title={radical}]
943
944Radical nodes are the most complex as they deal with scripts as well as
945constructed large symbols. Warning: never assign a node list to the \type
946{nucleus}, \type {sub}, \type {sup}, \type {left}, or \type {degree} field unless
947you are sure its internal link structure is correct, otherwise an error can be
948triggered.
949
950\showenginefields {radical}
951\showenginesubtypes {radical}
952
953{\em For more fields see \type {noad}. At some point we might move fields from
954that list to here but only when the engine also gets that split.}
955
956\showengineusagepernode{radical}
957
958\stopsubsection
959
960\startsubsection[title={fraction}]
961
962Fraction nodes are also used for delimited cases, hence the \type {left} and
963\type {right} fields among.
964
965\showenginefields {fraction}
966\showenginesubtypes {fraction}
967
968{\em For more fields see \type {noad}. At some point we might move fields from
969that list to here but only when the engine also gets that split.}
970
971\showengineusagepernode{fraction}
972
973\stopsubsection
974
975\startsubsection[title={fence}]
976
977Fence nodes come in pairs but either one can be a dummy (this period driven empty
978fence). Some of these fields are used by the renderer and might get adapted in
979the process.
980
981\showenginefields {fence}
982\showenginesubtypes {fence}
983
984{\em For more fields see \type {noad}. At some point we might move fields from
985that list to here but only when the engine also gets that split.}
986
987\showengineusagepernode{fence}
988
989\stopsubsection
990
991\stopsection
992
993\startsection[title=Helpers]
994
995\startsubsection[title={Introduction}]
996
997The userdata node variant has accessors on that object but when we use the
998indexed variant we use functions. As a consequence there are more helpers for
999direct nodes that for userdata nodes and many of them accept more arguments or
1000have multiple return values. When you use \CONTEXT\ you will notice that instead
1001of the \typ {node.direct} namespace we use \type {nuts}. Among the reasons is
1002that we had an intermediate variant in \CONTEXT\ \MKIV\ before we had these
1003direct nodes. That variant was more efficient than the userdata accessors and
1004triggered the introduction of direct nodes after which we dropped the
1005intermediate variant. So, for \CONTEXT\ users direct nodes are nuts.
1006
1007\stopsubsection
1008
1009\startsubsection[title={Housekeeping}]
1010
1011This function returns an array that maps node id numbers to node type strings,
1012providing an overview of the possible toplevel \type {id} types.
1013
1014\starttyping[option=LUA]
1015function node.types ( )
1016 return <t:table> identifiers
1017end
1018\stoptyping
1019
1020This shows the names of the nodes and their internal numbers. Not all nodes are
1021visible unless one goes really deep down into lists. The next two convert a name
1022to its internal numeric representation and vise versa. The numbers dont relate
1023to importance or some ordering; they just appear in the order that is handy for
1024the engine. Commands like this are rather optimized so performance should be ok
1025but you can of course always store the id in a \LUA\ number.
1026
1027\starttyping[option=LUA]
1028function node.id ( <t:string> name )
1029 return <t:integer> identifier
1030end
1031
1032function node.type ( <t:integer> identifier )
1033 return <t:string> name
1034end
1035\stoptyping
1036
1037This function returns an indexed table with valid field names for a particular
1038type of node. Some fields (like \type {total}) can be constructed from other
1039fields.
1040
1041\starttyping[option=LUA]
1042function node.fields ( <t:integer> identifier <t:string> name )
1043 return <t:table> fields
1044end
1045\stoptyping
1046
1047The \type {hasfield} function returns a boolean that is only true if \type {n} is
1048actually a node, and it has the field.This function probably is not that useful
1049but some nodes dont have a \type {subtype}, \type {attr} or \type {prev} field
1050and this is a way to test for that.
1051
1052\starttyping[option=LUA]
1053function node.direct.hasfield ( <t:direct> n <t:string> name )
1054 return <t:boolean> okay
1055end
1056\stoptyping
1057
1058The \type {new} function creates a new node. All its fields are initialized to
1059either zero or \type {nil} except for \type {id} and \type {subtype}. Instead of
1060numbers you can also use strings (names). If you pass a second argument the
1061subtype will be set too.
1062
1063\starttyping[option=LUA]
1064function node.direct.new (
1065 <t:number> id <t:string> name
1066)
1067 return <t:direct.> node
1068end
1069
1070function node.direct.new (
1071 <t:number> id <t:string> name,
1072 <t:number> <t:string> subtype
1073)
1074 return <t:direct.> node
1075end
1076\stoptyping
1077
1078As already has been mentioned, you are responsible for making sure that nodes
1079created this way are used only once, and are freed when you dont pass them
1080back somehow.
1081
1082The next one frees node \type {n} from \TEXs memory. Be careful: no checks are
1083done on whether this node is still pointed to from a register or some \type
1084{next} field: it is up to you to make sure that the internal data structures
1085remain correct. Fields that point to nodes or lists are flushed too. So, when
1086you used their content for something else you need to set them to nil first.
1087
1088\starttyping[option=LUA]
1089function node.direct.free ( <t:direct> n )
1090 return <t:direct> next
1091end
1092\stoptyping
1093
1094The \type {free} function returns the next field of the freed node, while the
1095\type {flushnode} alternative returns nothing.
1096
1097\starttyping[option=LUA]
1098function node.direct.flush ( <t:direct> n )
1099 no return values
1100end
1101\stoptyping
1102
1103A list starting with node \type {n} can be flushed from \TEXs memory too. Be
1104careful: no checks are done on whether any of these nodes is still pointed to
1105from a register or some \type {next} field: it is up to you to make sure that the
1106internal data structures remain correct.
1107
1108\starttyping[option=LUA]
1109function node.direct.flushlist ( <t:direct> n )
1110 no return values
1111end
1112\stoptyping
1113
1114When you free for instance a discretionary node, \type {flushlist} is applied to
1115the \type {pre}, \type {post}, \type {replace} so you dont need to do that
1116yourself. Assigning them \type {nil} wont free those lists!
1117
1118This creates a deep copy of node \type {n}, including all nested lists as in the case
1119of a hlist or vlist node. Only the \type {next} field is not copied.
1120
1121\starttyping[option=LUA]
1122function node.direct.copy ( <t:direct> n )
1123 return <t:direct> copy
1124end
1125\stoptyping
1126
1127A deep copy of the node list that starts at \type {n} can be created too. If
1128\type {m} is also given, the copy stops just before node \type {m}.
1129
1130\starttyping[option=LUA]
1131function node.direct.copylist ( <t:direct> n )
1132 return <t:direct> copy
1133end
1134
1135function node.direct.copylist ( <t:direct> n, <t:direct> m )
1136 return <t:direct> copy
1137end
1138\stoptyping
1139
1140Note that you cannot copy attribute lists this way. However, there is normally no
1141need to copy attribute lists because when you do assignments to the \type {attr}
1142field or make changes to specific attributes, the needed copying and freeing
1143takes place automatically. When you change a value of an attribute {\em in} a
1144list, it will affect all the nodes that share that list.
1145
1146\starttyping[option=LUA]
1147function node.direct.write ( <t:direct> n )
1148 no return values
1149end
1150\stoptyping
1151
1152This function will append a node list to \TEXs \quote {current list}. The node
1153list is not deepcopied! There is no error checking either! You might need to
1154enforce horizontal mode in order for this to work as expected.
1155
1156\stopsubsection
1157
1158\startsubsection[title={Manipulating lists}]
1159
1160Unless there is a bug or a callback messes up a node list is dual linked. In original
1161\TEX\ nodes had to be small so nodes only had a next pointer. If you run into an issue
1162you can use the next helper to sure that the node list is double linked.
1163
1164\starttyping[option=LUA]
1165function node.direct.slide ( <t:direct> n)
1166 return <t:direct> tail
1167end
1168\stoptyping
1169
1170In most cases \TEX\ itself only uses \type {next} pointers but your other
1171callbacks might expect proper \type {prev} pointers too. So, when you run into
1172issues or are in doubt, apply the slide function before you return the list. You
1173can also get the tail without sliding:
1174
1175\starttyping[option=LUA]
1176function node.direct.tail ( <t:direct> n )
1177 return <t:direct> tail
1178end
1179\stoptyping
1180
1181For tracing purposes we have a few counters. The first one returns the number of
1182nodes contained in the node list that starts at \type {n}. If \type {m} is also
1183supplied it stops at \type {m} instead of at the end of the list. The node \type
1184{m} is not counted.
1185
1186\starttyping[option=LUA]
1187function node.direct.length (
1188 <t:direct> n
1189)
1190 return <t:integer>
1191end
1192
1193function node.direct.length (
1194 <t:direct> n,
1195 <t:direct> m
1196)
1197 return <t:integer>
1198end
1199\stoptyping
1200
1201The second one the number of nodes contained in the node list that starts at
1202\type {n} that have a matching \type {id} field. If \type {m} is also supplied,
1203counting stops at \type {m} instead of at the end of the list. The node \type {m}
1204is not counted. This function also accept string \type {id}s.
1205
1206\starttyping[option=LUA]
1207function node.direct.count (
1208 <t:integer> id,
1209 <t:direct> n
1210)
1211 return <t:integer>
1212end
1213
1214function node.direct.count (
1215 <t:integer> id,
1216 <t:direct> n,
1217 <t:direct> m
1218)
1219 return <t:integer>
1220end
1221\stoptyping
1222
1223This function removes the node \type {current} from the list following \type
1224{head}. It is your responsibility to make sure it is really part of that list.
1225The return values are the new \type {head} and \type {current} nodes. The
1226returned \type {current} is the node following the \type {current} in the calling
1227argument, and is only passed back as a convenience (or \type {nil}, if there is
1228no such node). The returned \type {head} is more important, because if the
1229function is called with \type {current} equal to \type {head}, it will be
1230changed. When the third argument is passed, the node is freed.
1231
1232\starttyping[option=LUA]
1233function node.direct.remove ( <t:direct> head, <t:direct> current )
1234 return
1235 <t:direct> head,
1236 <t:direct> current,
1237 <t:direct> removed
1238end
1239
1240function node.direct.remove ( <t:direct> head, <t:direct> current, <t:boolean> free)
1241 return
1242 <t:direct> head,
1243 <t:direct> current
1244end
1245\stoptyping
1246
1247This function inserts the node \type {new} before \type {current} into the list
1248following \type {head}. It is your responsibility to make sure that \type
1249{current} is really part of that list. The return values are the (potentially
1250mutated) \type {head} and the node \type {new}, set up to be part of the list
1251(with correct \type {next} field). If \type {head} is initially \type {nil}, it
1252will become \type {new}.
1253
1254\starttyping[option=LUA]
1255function node.direct.insertbefore (
1256 <t:direct> head,
1257 <t:direct> current,
1258 <t:direct> new
1259)
1260 return
1261 <t:direct>, head
1262 <t:direct> new
1263end
1264\stoptyping
1265
1266This function inserts the node \type {new} after \type {current} into the list
1267following \type {head}. It is your responsibility to make sure that \type
1268{current} is really part of that list. The return values are the \type {head} and
1269the node \type {new}, set up to be part of the list (with correct \type {next}
1270field). If \type {head} is initially \type {nil}, it will become \type {new}.
1271
1272\starttyping[option=LUA]
1273function node.direct.insertafter (
1274 <t:direct> head,
1275 <t:direct> current,
1276 <t:direct> new
1277)
1278 return
1279 <t:direct>, head
1280 <t:direct> new
1281end
1282\stoptyping
1283
1284You can also mess with the list by changing the \type {next} or \type {prev} fields, using:
1285
1286\starttyping[option=LUA]
1287function node.direct.setprev ( <t:direct> n, <t:direct> prv ) end
1288function node.direct.setnext ( <t:direct> n, <t:direct> nxt ) end
1289function node.direct.setboth ( <t:direct> n, <t:direct> prv, <t:direct> nxt ) end
1290\stoptyping
1291
1292The next function pops the last node from \TEXs \quote{current list}. It returns
1293that node, or \type {nil} if the current list is empty.
1294
1295\starttyping[option=LUA]
1296function node.direct.lastnode ( )
1297 return <t:direct> n
1298end
1299\stoptyping
1300
1301This helper returns the location of the first match at or after node \type {n}:
1302
1303\starttyping[option=LUA]
1304function node.direct.findnode ( <t:direct> n, <t:integer> subtype )
1305 return <t:direct> n
1306end
1307
1308function node.direct.findnode ( <t:direct> n )
1309 return
1310 <t:direct>, n
1311 <t:integer> subtype
1312end
1313\stoptyping
1314
1315\stopsubsection
1316
1317\startsubsection[title={Traversing}]
1318
1319The easiest doityourself approach to run over a list of nodes is to use one of
1320the following functions:
1321
1322\starttyping[option=LUA]
1323function node.direct.getnext ( <t:direct> n )
1324 return <t:direct> <t:nil>
1325end
1326
1327function node.direct.getprev ( <t:direct> n )
1328 return <t:direct> <t:nil>
1329end
1330
1331function node.direct.getboth ( <t:direct> n )
1332 return
1333 <t:direct> <t:nil>, prev
1334 <t:direct> <t:nil> next
1335end
1336\stoptyping
1337
1338Instead of using these you can use one of the iterators that loops over the node
1339list that starts at \type {n}.
1340
1341\starttyping[option=LUA]
1342function node.direct.traverse ( <t:direct> n )
1343 return
1344 <t:direct> t,
1345 <t:integer> id,
1346 <t:integer> subtype
1347end
1348\stoptyping
1349
1350Typically code looks like this:
1351
1352\starttyping[option=LUA]
1353for n in node.traverse(head) do
1354 whatever
1355end
1356\stoptyping
1357
1358which is functionally equivalent to:
1359
1360\starttyping[option=LUA]
1361do
1362 local n
1363 local function f (head,var)
1364 local t
1365 if var == nil then
1366 t = head
1367 else
1368 t = var.next
1369 end
1370 return t
1371 end
1372 while true do
1373 n = f (head, n)
1374 if n == nil then
1375 break
1376 end
1377 whatever
1378 end
1379end
1380\stoptyping
1381
1382It should be clear from the definition of the function \type {f} that even though
1383it is possible to add or remove nodes from the node list while traversing, you
1384have to take great care to make sure all the \type {next} (and \type {prev})
1385pointers remain valid.
1386
1387If the above is unclear to you, see the section \quote {For Statement} in the
1388\LUA\ Reference Manual.
1389
1390This is an iterator that loops over all the nodes in the list that starts at
1391\type {n} that have a matching \type {id} field. See the previous section for
1392details. The change is in the local function \type {f}, which now does an extra
1393while loop checking against the upvalue \type {id}, kind of like:
1394
1395\starttyping
1396local function f(head,var)
1397 local t
1398 if var == nil then
1399 t = head
1400 else
1401 t = var.next
1402 end
1403 while not t.id == id do
1404 t = t.next
1405 end
1406 return t
1407end
1408\stoptyping
1409
1410This and the previously discussed \type {traverse} are the only traverses
1411provided for userdata nodes.
1412
1413\starttyping[option=LUA]
1414function node.direct.traverseid ( <t:integer> id, <t:direct> n )
1415 return
1416 <t:direct> t,
1417 <t:integer> subtype
1418end
1419\stoptyping
1420
1421The \type {traversechar} iterator loops over the \type {glyph} nodes in a list.
1422Only nodes with a subtype less than 256 are seen.
1423
1424{\em NEEDS CHECKING: protected check}
1425
1426\starttyping[option=LUA]
1427function node.direct.traversechar ( <t:direct> n )
1428 return
1429 <t:direct>, n
1430 <t:integer>, char
1431 <t:integer> font
1432end
1433\stoptyping
1434
1435The \type {traverseglyph} iterator loops over a list and returns the list and
1436filters all glyphs:
1437
1438\starttyping[option=LUA]
1439function node.direct.traverseglyph ( <t:direct> n )
1440 return
1441 <t:direct>, n
1442 <t:integer>, char
1443 <t:integer> font
1444end
1445\stoptyping
1446
1447This iterator loops over the \type {hlist} and \type {vlist} nodes in a list. The
1448four return values can save some time compared to fetching these fields but in
1449practice you seldom need them all.
1450
1451\starttyping[option=LUA]
1452function node.direct.traverselist ( <t:direct> n )
1453 return
1454 <t:direct>, n
1455 <t:integer>, identifier
1456 <t:integer>, subtype
1457 <t:direct> list
1458end
1459\stoptyping
1460
1461This iterator loops over nodes that have content: \type {hlist}, \type {vlist},
1462\type {glue} with leaders, \type {glyph}, \type {disc} and \type {rule} nodes.
1463
1464\starttyping[option=LUA]
1465function node.direct.traversecontent ( <t:direct> n )
1466 return
1467 <t:direct>, n
1468 <t:integer>, identifier
1469 <t:integer>, subtype
1470 <t:direct> listorleader
1471end
1472\stoptyping
1473
1474The traversers also support backward traversal. An optional extra boolean triggers
1475this. Yet another optional boolean will automatically start at the end of the
1476given list. So, if we want both we use:
1477
1478\starttyping[option=LUA]
1479function node.direct.traverse (
1480 <t:direct> n,
1481 <t:boolean> reverse,
1482 <t:boolean> startatend
1483)
1484 return
1485 <t:direct> t,
1486 <t:integer> id,
1487 <t:integer> subtype
1488end
1489\stoptyping
1490
1491\stopsubsection
1492
1493\startsubsection[title={Glyphs}]
1494
1495Glyphs have a lot of parameters and there are many setters and getters that
1496can access them. Some generic ones, like \type {getwidth} are discussed
1497in other subsections, some are more specific to glyphs:
1498
1499\starttyping[option=LUA]
1500function node.direct.getslant ( <t:direct> g ) return <t:integer> end
1501function node.direct.getweight ( <t:direct> g ) return <t:integer> end
1502\stoptyping
1503
1504and
1505
1506\starttyping[option=LUA]
1507function node.direct.setslant ( <t:direct> g, <t:integer> slant ) end
1508function node.direct.setweight ( <t:direct> g, <t:integer> weight ) end
1509\stoptyping
1510
1511\stopsubsection
1512
1513\startsubsection[title={Glue}]
1514
1515You can set the five properties of a glue in one go. If a nonnumeric value is
1516passed the property becomes zero.
1517
1518\starttyping[option=LUA]
1519function node.direct.setglue ( <t:direct> n )
1520 no return values
1521end
1522
1523function node.direct.setglue (
1524 <t:direct> n,
1525 <t:integer> width,
1526 <t:integer> stretch,
1527 <t:integer> shrink,
1528 <t:integer> stretchorder,
1529 <t:integer> shrinkorder
1530)
1531 no return values
1532end
1533\stoptyping
1534
1535When you pass values, only arguments that are numbers are assigned so the next
1536call will only adapt the width and shrink.
1537
1538\starttyping[option=LUA]
1539node.direct.setglue(n,655360,false,65536)
1540\stoptyping
1541
1542
1543When a list node is passed, you set the glue, order and sign instead. The next
1544call will return five values or nothing when no glue is passed.
1545
1546\starttyping[option=LUA]
1547function node.direct.getglue ( <t:direct> n )
1548 return
1549 <t:integer>, width
1550 <t:integer>, stretch
1551 <t:integer>, shrink
1552 <t:integer>, stretchorder
1553 <t:integer> shrinkorder
1554\stoptyping
1555
1556When the second argument is false, only the width is returned (this is consistent
1557with \type {tex.get}). When a list node is passed, you get back the glue that is
1558set, the order of that glue and the sign.
1559
1560This function returns \type {true} when the width, stretch and shrink properties
1561are all zero.
1562
1563\starttyping[option=LUA]
1564function node.direct.iszeroglue ( <t:direct> n )
1565 return <t:boolean> allzero
1566end
1567\stoptyping
1568
1569Glue is not only, well, glue. The to be filled space can also be occupied by a rule,
1570boxes, glyphs and what more. You can get the list that makes this with:
1571
1572\starttyping[option=LUA]
1573function node.direct.getleader ( <t:direct> n )
1574 return <t:direct> list
1575end
1576\stoptyping
1577
1578and set the list with
1579
1580\starttyping[option=LUA]
1581function node.direct.setleader ( <t:direct> n, <t:direct> l <t:nil> )
1582 no return values
1583end
1584\stoptyping
1585
1586\stopsubsection
1587
1588\startsubsection[title={Attributes}]
1589
1590Assignments to attributes registers result in assigning lists with set attributes
1591to nodes and the implementation is nontrivial because the value that is
1592attached to a node is essentially a (sorted) sparse array of keyvalue pairs.
1593It is generally easiest to deal with attribute lists and attributes by using the
1594dedicated functions in the \type {node} library.
1595
1596An \type {attribute} comes in two variants, indicated by subtype. Because
1597attributes are stored in a sorted linked list, and because they are shared, the
1598first node is a list reference node and the following ones are value nodes. So,
1599most attribute nodes are value nodes. These are forward linked lists. Because
1600there are assumptions to how these list are build you should rely on the helpers,
1601also because details might change.
1602
1603This returns the currently active list of attributes, if there is one.
1604
1605\starttyping[option=LUA]
1606function node.direct.currentattr()
1607 return <t:direct> list
1608end
1609\stoptyping
1610
1611The intended usage of \type {currentattr} is as follows (we use the userdata
1612interface here):
1613
1614\starttyping[option=LUA]
1615local x1 = node.new("glyph")
1616x1.attr = node.currentattr()
1617local x2 = node.new("glyph")
1618x2.attr = node.currentattr()
1619\stoptyping
1620
1621or:
1622
1623\starttyping[option=LUA]
1624local x1 = node.new("glyph")
1625local x2 = node.new("glyph")
1626local ca = node.currentattr()
1627x1.attr = ca
1628x2.attr = ca
1629\stoptyping
1630
1631The attribute lists are reference counted and the assignment takes care of
1632incrementing the count. You cannot expect the value \type {ca} to be valid any
1633more when you assign attributes (using \type {tex.setattribute}) or when control
1634has been passed back to \TEX.
1635
1636\starttyping[option=LUA]
1637<number> v = node.hasattribute ( <node> n, <number> id )
1638<number> v = node.hasattribute ( <node> n, <number> id, <number> val )
1639\stoptyping
1640
1641Tests if a node has the attribute with number \type {id} set. If \type {val} is
1642also supplied, also tests if the value matches \type {val}. It returns the value,
1643or, if no match is found, \type {nil}.
1644
1645\starttyping[option=LUA]
1646function node.direct.getattribute ( <t:direct> n, <t:integer> id )
1647 return <t:integer> value
1648end
1649\stoptyping
1650
1651The previous function tests if a node has an attribute with number \type {id}
1652set. It returns the value, or, if no match is found, \type {nil}. If no \type
1653{id} is given then the zero attributes is assumed.
1654
1655
1656\starttyping[option=LUA]
1657function node.direct.findattribute ( <t:direct> n, <t:integer> id )
1658 return
1659 <t:integer>, value
1660 <t:direct> node
1661end
1662\stoptyping
1663
1664Finds the first node that has attribute with number \type {id} set. It returns
1665the value and the node if there is a match and otherwise nothing.
1666
1667\starttyping[option=LUA]
1668function node.direct.setattribute ( <t:direct> n, <t:integer> id, <t:integer> value )
1669 no return values
1670end
1671\stoptyping
1672
1673Sets the attribute with number \type {id} to the value \type {value}. Duplicate
1674assignments are ignored.
1675
1676\starttyping[option=LUA]
1677function node.direct.unsetattribute ( <t:direct> n, <t:integer> id )
1678 return <t:integer> value
1679end
1680
1681function node.direct.unsetattribute ( <t:direct> n, <t:integer> id, <t:integer> value )
1682 return <t:integer> value
1683end
1684\stoptyping
1685
1686Unsets the attribute with number \type {id}. If \type {value} is also supplied,
1687it will only perform this operation if the value matches \type {value}. Missing
1688attributes or attributevalue pairs are ignored. If the attribute was actually
1689deleted, the function returns its old value, otherwise it returns \type {nil}.
1690
1691\stopsubsection
1692
1693\startsubsection[title={Glyph handling}]
1694
1695Processing a character stream into a visual representation using glyphs is one of
1696the important processes in the engine. In \TEX82 this happens in two places. When
1697the text is read ligaturing and kerning takes place and the list can, if needed,
1698be packed into a box because the dimensions are now known. When that list is to
1699become a paragraph it might be that lines get split and when a word can be
1700hyphenated the ligaturing and kerning is reverted, the word gets hyphenated,
1701ligatures and kerns get reapplied and the process goes on.
1702
1703In \OPENTYPE\ processing characters is way more complex. Even if we delegate this
1704to a library, the fact that we have a mix of text and whatever, potential
1705hyphenation as well as spaces turned glue, means that we need to do some juggling
1706with nodes. For that reason hyphenation (of the whole list), ligaturing and
1707kerning has been split into clearly separates stages. One can still apply the
1708original \TEX\ variants but in practice it is \LUA\ that does the juggling of
1709nodes in more complex situations. And were not only talking of font processing.
1710For instance, additional intercharacter kerning can be done in \LUA\ too.
1711
1712This all means that we have quite a repertoire of helpers that deal with glyph
1713processing efficiently.
1714
1715We can locate the first node in the list starting at \type {n} that is a glyph node
1716with a subtype indicating it is a glyph, or \type {nil}. If \type {m} is given,
1717processing stops at (but including) that node, otherwise processing stops at the
1718end of the list. The \type {char} and \type {glyph} variants check for the
1719protected field being (yet) unset or (already) set.
1720
1721\starttyping[option=LUA]
1722function node.direct.firstglyphnode ( <t:direct> n )
1723 return <t:direct> n
1724end
1725
1726function node.direct.firstglyphnode ( <t:direct> n, <t:direct> m )
1727 return <t:direct> n
1728end
1729\stoptyping
1730
1731The next functions can be used to determine if processing is needed. We distinguish between
1732a character (unprocessed) and a glyph (processed or unprocessed). When we check for a glyph
1733there are three possible outcomes:
1734
1735\starttyping[option=LUA]
1736function node.direct.isglyph ( <t:direct> n )
1737 return
1738 <t:nil>,
1739 <t:nil>
1740end
1741
1742function node.direct.isglyph ( <t:direct> n )
1743 return
1744 <t:false>,
1745 <t:integer> identifier
1746end
1747
1748function node.direct.isglyph ( <t:direct> n )
1749 return
1750 <t:integer>, character
1751 <t:integer> font
1752end
1753\stoptyping
1754
1755Checking for a processed character is more complicated. If the glyph has been processed and
1756the protected property has been set, we get this:
1757
1758\starttyping[option=LUA]
1759function node.direct.ischar ( <t:direct> n )
1760 return <t:false>
1761end
1762\stoptyping
1763
1764If thats not the case additional arguments are checked. If we dont pass a valid
1765integer, the character value is returned:
1766
1767\starttyping[option=LUA]
1768function node.direct.ischar ( <t:direct> n, <t:integer> font )
1769 return <t:integer> character
1770end
1771\stoptyping
1772
1773btu when we passed a font identifier indeed we check if that one matches the one in the
1774glyph and if not again we get:
1775
1776\starttyping[option=LUA]
1777function node.direct.ischar ( <t:direct> n, <t:integer> font )
1778 return <t:false>
1779end
1780\stoptyping
1781
1782From there on we check for more arguments to match the glyph fields:
1783
1784\starttyping[option=LUA]
1785function node.direct.ischar (
1786 <t:direct> n,
1787 <t:integer> font,
1788 <t:integer> data
1789)
1790 return <t:false> <t:integer> character
1791
1792end
1793
1794function node.direct.ischar (
1795 <t:direct> n,
1796 <t:integer> font,
1797 <t:integer> data,
1798 <t:integer> state
1799)
1800 return <t:false> <t:integer> character
1801
1802end
1803
1804function node.direct.ischar (
1805 <t:direct> n,
1806 <t:integer> font,
1807 <t:integer> scale,
1808 <t:integer> xscale,
1809 <t:integer> yscale,
1810)
1811 return <t:false> <t:integer> character
1812
1813end
1814
1815function node.direct.ischar (
1816 <t:direct> n,
1817 <t:integer> font,
1818 <t:integer> data,
1819 <t:integer> scale,
1820 <t:integer> xscale,
1821 <t:integer> yscale,
1822)
1823 return <t:false> <t:integer> character
1824
1825end
1826\stoptyping
1827
1828There are reasons for these combined tests and they can be found in the \CONTEXT\
1829font handler. A related helper is one that compares the \type {font}, \type
1830{data}, \type {scale}, \type {xscale}, \type {yscale}, \type {slant} and \type
1831{weight}.
1832
1833\starttyping[option=LUA]
1834function node.direct.issimilarglyph ( <t:direct> one, <t:direct> two )
1835 return <t:boolean> similar
1836end
1837\stoptyping
1838
1839This function returns the first glyph or disc node in the given list:
1840
1841\starttyping[option=LUA]
1842function node.direct.hasglyph ( <t:direct> n )
1843 return <t:direct> n
1844end
1845\stoptyping
1846
1847Traditional \TEX\ ligature processing can be achieved with the next helper. This
1848assumes that the ligature information is present in the font. In \CONTEXT\ we
1849call this base mode processing.
1850
1851\starttyping[option=LUA]
1852function node.direct.ligaturing ( <t:direct> first )
1853 return
1854 <t:direct>, head
1855 <t:direct>, tail
1856 <t:boolean> success
1857end
1858
1859function node.direct.ligaturing ( <t:direct> first, <t:direct> last )
1860 return
1861 <t:direct>, head
1862 <t:direct>, tail
1863 <t:boolean> success
1864end
1865\stoptyping
1866
1867Traditional \TEX\ font kern processing can be achieved with the next helper. This
1868assumes that the kern information is present in the font. In \CONTEXT\ we
1869call this base mode processing.
1870
1871\starttyping[option=LUA]
1872function node.direct.kerning ( <t:direct> first )
1873 return
1874 <t:direct>, head
1875 <t:direct>, tail
1876 <t:boolean> success
1877end
1878
1879function node.direct.kerning ( <t:direct> first, <t:direct> last )
1880 return
1881 <t:direct>, head
1882 <t:direct>, tail
1883 <t:boolean> success
1884end
1885\stoptyping
1886
1887When processing is done, you can mark the glyph nodes as protected in order to
1888prevent redundant processing, for instance because boxed material gets unboxed.
1889Where in \LUATEX\ the subtype gets changed by adding or subtracting 256, in
1890\LUAMETATEX\ we have a dedicated (small) protection field.
1891
1892\starttyping[option=LUA]
1893function node.direct.protectglyph ( <t:direct> n )
1894 no return values
1895end
1896
1897function node.direct.protectglyphs ( <t:direct> first, <t:direct> last )
1898 no return values
1899end
1900\stoptyping
1901
1902The opposite action can also be done.
1903
1904\starttyping[option=LUA]
1905function node.direct.unprotectglyph ( <t:direct> n )
1906 no return values
1907end
1908
1909function node.direct.unprotectglyphs ( <t:direct> first, <t:direct> last )
1910 no return values
1911end
1912\stoptyping
1913
1914The next function checks if protrusion is active at a line boundary, in which case the glyph
1915node can be skipped. Its not that useful in the end.
1916
1917\starttyping[option=LUA]
1918function node.direct.protrusionskipable ( <t:direct> n )
1919 return <t:boolean> skippable
1920end
1921\stoptyping
1922
1923Once were done we can freeze leaders: apply the glue to the leader and freeze
1924the boxes or whatever is at hand.
1925
1926\starttyping[option=LUA]
1927function node.direct.flattenleaders ( <t:direct> n )
1928 return
1929 <t:direct>, head
1930 <t:integer> count
1931end
1932\stoptyping
1933
1934\stopsubsection
1935
1936\startsubsection[title=Discretionaries]
1937
1938Discretionaries and glyphs are the carriers of text. Where the core of \type
1939{glyph} nodes are the \type {font} and \type {char} fields, in \type {disc} nodes
1940we have to focus on the \type {pre}, \type {post} and \type{replace} fields.
1941These point to linked lists that are a mix of glyph, kerns and (in \LUAMETATEX\
1942fixed width) glue. here are the accessors: \footnote {These are a bit more
1943generic because they also return fields from \type {choice} nodes and possibly
1944\type {hlist} and \type {vlist} nodes.}
1945
1946\starttyping[option=LUA]
1947function node.direct.getpost ( <t:direct> d, <t:boolean> tailtoo )
1948 return
1949 <t:direct>, head
1950 <t:direct> tail
1951end
1952
1953function node.direct.getpre ( <t:direct> d, <t:boolean> tailtoo )
1954 return
1955 <t:direct>, head
1956 <t:direct> tail
1957end
1958
1959function node.direct.getreplace ( <t:direct> d, <t:boolean> tailtoo )
1960 return
1961 <t:direct>, head
1962 <t:direct> tail
1963end
1964\stoptyping
1965
1966\starttyping[option=LUA]
1967function node.direct.getdisc ( <t:direct> d, <t:boolean> tailtoo )
1968 return
1969 <t:direct>, prehead
1970 <t:direct>, posthead
1971 <t:direct>, replacehead
1972 <t:direct>, pretail
1973 <t:direct>, posttail
1974 <t:direct> replacetail
1975end
1976\stoptyping
1977
1978We also have setters:
1979
1980\starttyping[option=LUA]
1981function node.direct.setpost ( <t:direct> d, <t:direct> <t:nil> ) end
1982function node.direct.setpre ( <t:direct> d, <t:direct> <t:nil> ) end
1983function node.direct.setreplace ( <t:direct> d, <t:direct> <t:nil> ) end
1984\stoptyping
1985
1986A major update can be done with this one:
1987
1988\starttyping[option=LUA]
1989function node.direct.setdisc (
1990 <t:direct>, discretionary
1991 <t:direct> <t:nil>, pre
1992 <t:direct> <t:nil>, post
1993 <t:direct> <t:nil>, replace
1994 <t:subtype> <t:nil>, subtype
1995 <t:subtype> <t:nil> penalty
1996)
1997 no return values
1998end
1999\stoptyping
2000
2001From this you can deduce that we can also say:
2002
2003\starttyping[option=LUA]
2004function node.direct.getpenalty ( <t:direct> d )
2005 return <t:integer> penalty
2006end
2007
2008function node.direct.setpenalty ( <t:direct> d, <t:integer> penalty )
2009 no return value
2010end
2011\stoptyping
2012
2013The next pair targets glyphs and normally you will not use the setter, because
2014the engine takes care of setting that state.
2015
2016\starttyping[option=LUA]
2017function node.direct.getdiscpart ( <t:direct> g )
2018 return
2019 <t:integer>, part
2020 <t:integer>, after
2021 <t:integer> code
2022
2023end
2024
2025function node.direct.setdiscpart (
2026 <t:direct> g,
2027 <t:integer> part
2028 <t:integer> after
2029 <t:integer> code
2030)
2031 no return value
2032end
2033\stoptyping
2034
2035The part and after properties relate to discretionary nodes that might have been
2036flattened. The complication in (tracing) here is that information is lost so we
2037store the states in the glyph node.
2038
2039\showenginevalues{discpartvalues}
2040
2041The code properties relate to where the (usually hyphen) character comes from:
2042
2043\showenginevalues{glyphdiscvalues}
2044
2045When you fool around with disc nodes you need to be aware of the fact that they
2046have a special internal data structure. As long as you reassign the fields when
2047you have extended the lists its ok because then the tail pointers get updated,
2048but when you add to list without reassigning you might end up in trouble when
2049the linebreak routine kicks in. You can call this function to check the list for
2050issues with disc nodes.
2051
2052\starttyping[option=LUA]
2053function node.direct.checkdiscretionary ( <t:direct> n )
2054 no return values
2055end
2056\stoptyping
2057
2058The plural variant runs over all disc nodes in a list, the singular variant
2059checks one node only (it also checks if the node is a disc node).
2060
2061\starttyping[option=LUA]
2062function node.direct.checkdiscretionaries ( <t:direct> head )
2063 no return values
2064end
2065\stoptyping
2066
2067This function will remove the discretionaries in the list and inject the replace
2068field when set.
2069
2070\starttyping[option=LUA]
2071function node.direct.flattendiscretionaries ( <t:direct> n )
2072 return
2073 <t:direct>, head
2074 <t:integer> count
2075end
2076\stoptyping
2077
2078\stopsubsection
2079
2080\startsubsection[title=Packaging and dimensions]
2081
2082At some point a node list has to be packed in either a horizontal or vertical
2083box. There are restrictions to what can get packed, for instance you cannot have
2084glyphs in a vertical list.
2085
2086
2087The \type {hpack} function creates a new hlist by packaging the list that begins
2088at node \type {n} into a horizontal box. With only a single argument, this box is
2089created using the natural width of its components. In the three argument form,
2090\type {info} must be either \type {additional} or \type {exactly}, and \type {w}
2091is the additional (\type {\hbox spread}) or exact (\type {\hbox to}) width to be
2092used. The second return value is the badness of the generated box.
2093
2094\starttyping[option=LUA]
2095function node.direct.hpack (
2096 <t:direct> list
2097)
2098 return
2099 <t:direct>, box
2100 <t:integer> badness
2101end
2102
2103function node.direct.hpack (
2104 <t:direct> list,
2105 <t:integer> width,
2106 <t:string> info "additional" "exactly"
2107)
2108 return
2109 <t:direct>, box
2110 <t:integer> badness
2111end
2112
2113function node.direct.hpack (
2114 <t:direct> list,
2115 <t:integer> width,
2116 <t:string> info, "additional" "exactly"
2117 <t:integer> direction
2118)
2119 return
2120 <t:direct>, box
2121 <t:integer> badness
2122end
2123\stoptyping
2124
2125
2126
2127
2128
2129
2130
2131
2132The \type {vpack} function creates a new vlist by packaging the list that begins
2133at node \type {n} into a vertical box. With only a single argument, this box is
2134created using the natural height of its components. In the three argument form,
2135\type {info} must be either \type {additional} or \type {exactly}, and \type {w}
2136is the additional (\type {\vbox spread}) or exact (\type {\vbox to}) height to be
2137used.
2138
2139\starttyping[option=LUA]
2140function node.direct.vpack (
2141 <t:direct> list
2142)
2143 return
2144 <t:direct>, box
2145 <t:integer> badness
2146end
2147
2148function node.direct.vpack (
2149 <t:direct> list,
2150 <t:integer> height,
2151 <t:string> info "additional" "exactly"
2152)
2153 return
2154 <t:direct>, box
2155 <t:integer> badness
2156end
2157
2158function node.direct.vpack (
2159 <t:direct> list,
2160 <t:integer> height,
2161 <t:string> info, "additional" "exactly"
2162 <t:integer> direction
2163)
2164 return
2165 <t:direct>, box
2166 <t:integer> badness
2167end
2168\stoptyping
2169
2170This function calculates the natural inline dimensions of the node list starting
2171at node \type {first} and terminating just before node \type {last} (or the end of the
2172list, if there is no second argument). The return values are scaled points.
2173
2174\starttyping[option=LUA]
2175function node.direct.dimensions (
2176 <t:direct> first,
2177 <t:direct> last
2178)
2179 return
2180 <t:integer>, width
2181 <t:integer>, height
2182 <t:integer> depth
2183end
2184\stoptyping
2185
2186This alternative calling method takes glue settings into account and is especially useful for
2187finding the actual width of a sublist of nodes that are already boxed, for
2188example in code like this, which prints the width of the space in between the
2189\type {a} and \type {b} as it would be if \type {\box0} was used asis:
2190
2191\starttyping
2192\setbox0 = \hbox to 20pt {a b}
2193
2194\directlua{print (node.dimensions(
2195 tex.box[0].glueset,
2196 tex.box[0].gluesign,
2197 tex.box[0].glueorder,
2198 tex.box[0].head.next,
2199 node.tail(tex.box[0].head)
2200)) }
2201\stoptyping
2202
2203You need to keep in mind that this is one of the few places in \TEX\ where floats
2204are used, which means that you can get small differences in rounding when you
2205compare the width reported by \type {hpack} with \type {dimensions}.
2206
2207\starttyping[option=LUA]
2208function node.direct.dimensions (
2209 <t:number> glueset,
2210 <t:integer> gluesign
2211 <t:integer> glueorder,
2212 <t:direct> first,
2213 <t:direct> last
2214)
2215 return
2216 <t:integer>, width
2217 <t:integer>, height
2218 <t:integer> depth
2219end
2220\stoptyping
2221
2222This alternative saves a few lookups and can be more convenient in some
2223cases:
2224
2225\starttyping[option=LUA]
2226function node.direct.rangedimensions (
2227 <t:direct> parent,
2228 <t:direct> first,
2229 <t:direct> last
2230)
2231 return
2232 <t:integer>, width
2233 <t:integer>, height
2234 <t:integer> depth
2235end
2236\stoptyping
2237
2238If you only need the width, a simple and somewhat more efficient variant is this,
2239where again \type {last} is optional:
2240
2241\starttyping[option=LUA]
2242function node.direct.naturalwidth (
2243 <t:direct> first,
2244 <t:direct> last
2245)
2246 return <t:integer> width
2247end
2248\stoptyping
2249
2250More low level are the following helpers. They accept various kind of nodes
2251\type {hlist}, \type {vlist}, \type {unset}, \type {rule}, \type {glyph} or \type {glue}
2252(because these can have a leader).
2253
2254\starttyping[option=LUA]
2255function node.direct.getwhd ( <t:direct> n )
2256 return
2257 <t:dimension>, width
2258 <t:dimension>, height
2259 <t:dimension> depth
2260end
2261\stoptyping
2262
2263In case of as glyph you can also get the expansion:
2264
2265\starttyping[option=LUA]
2266function node.direct.getwhd ( <t:direct> n, <t:true> expansion )
2267 return
2268 <t:dimension>, width
2269 <t:dimension>, height
2270 <t:dimension>, depth
2271 <t:integer> expansion
2272end
2273\stoptyping
2274
2275The \type {getwidth} accepts even more node types: \type {hlist}, \type {vlist},
2276\type {unset}, \type {align}, \type {rule}, \type {glue}, \type {gluespec}, \type
2277{glyph}, \type {kern} and \type {math} (surround).
2278
2279\starttyping[option=LUA]
2280function node.direct.getwidth ( <t:direct> n )
2281 return <t:dimension> width
2282end
2283\stoptyping
2284
2285And for glyphs:
2286
2287\starttyping[option=LUA]
2288function node.direct.getwidth ( <t:direct> n, <t:true> expansion )
2289 return
2290 <t:dimension>, width
2291 <t:dimension> expansion
2292end
2293\stoptyping
2294
2295The getter for height operates on \type {hlist}, \type {vlist}, \type {unset},
2296\type {rule}, \type {insert} and \type {fence}.
2297
2298\starttyping[option=LUA]
2299function node.direct.getheight ( <t:direct> n )
2300 return <t:dimension> height
2301end
2302\stoptyping
2303
2304For the depth we have a different repertoire: \type {hlist}, \type {vlist}, \type
2305{unset}, \type {rule}, \type {insert}, \type {glyph} and \type {fence}.
2306
2307\starttyping[option=LUA]
2308function node.direct.getdepth ( <t:direct> n )
2309 return <t:dimension> depth
2310end
2311\stoptyping
2312
2313For \type {hlist}, \type {vlist}, \type {unset}, \type {rule}, \type
2314{insertnode:}, \type {glyph} and \type {fence} we can get the total of height
2315and depth:
2316
2317\starttyping[option=LUA]
2318function node.direct.gettotal ( <t:direct> n )
2319 return <t:dimension> height depth
2320end
2321\stoptyping
2322
2323Only \type {hlist} and \type {vlist} have a (vertical or horizontal) shift:
2324
2325\starttyping[option=LUA]
2326function node.direct.getshift ( <t:direct> n )
2327 return <t:dimension> shift
2328end
2329\stoptyping
2330
2331This one is only valid for \type {glyph} and \type {kern} nodes:
2332
2333\starttyping[option=LUA]
2334function node.direct.getexpansion ( <t:direct> n )
2335 return <t:dimension> expansion
2336end
2337\stoptyping
2338
2339Before we move on we mention the setters:
2340
2341\starttyping[option=LUA]
2342function node.direct.setwidth ( <t:direct> n, <t:dimension> width ) end
2343function node.direct.setheight ( <t:direct> n, <t:dimension> height ) end
2344function node.direct.setdepth ( <t:direct> n, <t:dimension> depth ) end
2345function node.direct.setshift ( <t:direct> n, <t:dimension> shift ) end
2346function node.direct.setexpansion ( <t:direct> n, <t:integer> expansion ) end
2347\stoptyping
2348
2349The combined one ignores values that are no number, so passing (e.g.) \type {nil}
2350or (nicer) \type {false} will retain the value.
2351
2352\starttyping[option=LUA]
2353function nodedirect.setwhd (
2354 <t:direct> node,
2355 <t:dimension> width,
2356 <t:dimension> height,
2357 <t:dimension> depth,
2358 no return values
2359end
2360\stoptyping
2361
2362These \type {hlist} and \type {vlist} nodes (but others as well have) a field
2363called \type {list}:
2364
2365\starttyping[option=LUA]
2366function node.direct.getlist ( <t:direct> b )
2367 return <t:direct> list
2368end
2369
2370function node.direct.setlist ( <t:direct> b, <t:direct> list )
2371 nothing to return
2372end
2373\stoptyping
2374
2375When a list is packages, glue is resolved and the list node gets its glue properties
2376set so that the backend can apply the stretch and shrink to the glue amount. There might
2377be situations where you want to do this explicitly, which is why we provide:
2378
2379\starttyping[option=LUA]
2380function node.direct.freeze ( <t:direct> b )
2381 nothing to return
2382end
2383\stoptyping
2384
2385In \LUAMETATEX\ we can handle nested marks, inserts and adjusts, and
2386pre and post material can get bound to a box. We can use these to access them:
2387
2388\starttyping[option=LUA]
2389function node.direct.getpost ( <t:direct> b, <t:boolean> tailtoo )
2390 return
2391 <t:direct>, head
2392 <t:direct> tail
2393end
2394
2395function node.direct.getpre ( <t:direct> b, <t:boolean> tailtoo )
2396 return
2397 <t:direct>, head
2398 <t:direct> tail
2399end
2400\stoptyping
2401
2402and these to set them, although they are unlikely candidates for that.
2403
2404\starttyping[option=LUA]
2405function node.direct.setpost ( <t:direct> b, <t:direct> <t:nil> ) end
2406function node.direct.setpre ( <t:direct> b, <t:direct> <t:nil> ) end
2407\stoptyping
2408
2409\stopsubsection
2410
2411\startsubsection[title={Math}]
2412
2413We start with the function that runs the internal \quote {mlist to hlist}
2414conversion that turns a the yet unprocessed math list into a horizontal list. The
2415interface is the same as for the callback callback {mlisttohlist}.
2416
2417\starttyping[option=LUA]
2418function node.direct.mlisttohlist (
2419 <t:direct> list,
2420 <t:string> displaytype,
2421 <t:boolean> penalties
2422)
2423 <t:direct> result
2424end
2425\stoptyping
2426
2427When you have a horizontal list with math you can locate the relevant portion
2428with:
2429
2430\starttyping[option=LUA]
2431function node.direct.beginofmath ( <t:direct> n ) return <t:direct> end
2432function node.direct.endofmath ( <t:direct> n ) return <t:direct> end
2433\stoptyping
2434
2435You can for instance use these helpers to skip over math in case youre
2436processing text.
2437
2438The math noads have a nucleus and scripts. In \LUAMETATEX\ we have the usual
2439super and subscript but also prescripts and a primescript, so five scripts in
2440total so naturally we have getters for these:
2441
2442\starttyping[option=LUA]
2443function node.direct.getnucleus ( <t:direct> n ) return <t:direct> <t:nil> end
2444function node.direct.getprime ( <t:direct> n ) return <t:direct> <t:nil> end
2445function node.direct.getsup ( <t:direct> n ) return <t:direct> <t:nil> end
2446function node.direct.getsub ( <t:direct> n ) return <t:direct> <t:nil> end
2447function node.direct.getsuppre ( <t:direct> n ) return <t:direct> <t:nil> end
2448function node.direct.getsubpre ( <t:direct> n ) return <t:direct> <t:nil> end
2449\stoptyping
2450
2451plus:
2452
2453\starttyping[option=LUA]
2454function node.direct.getscripts ( <t:direct> n )
2455 return
2456 <t:direct>, primescript
2457 <t:direct>, superscript
2458 <t:direct>, subscript
2459 <t:direct>, superprescript
2460 <t:direct> subprescript
2461end
2462\stoptyping
2463
2464These are complemented by setters. When the second argument is not passes (or nil)
2465the field is reset.
2466
2467\starttyping[option=LUA]
2468function node.direct.setnucleus ( <t:direct> n, <t:direct> nucleus ) end
2469function node.direct.setprime ( <t:direct> n, <t:direct> primescript ) end
2470function node.direct.setsup ( <t:direct> n, <t:direct> superscript ) end
2471function node.direct.setsub ( <t:direct> n, <t:direct> subscript ) end
2472function node.direct.setsuppre ( <t:direct> n, <t:direct> superprescript ) end
2473function node.direct.setsubpre ( <t:direct> n, <t:direct> subprescript ) end
2474\stoptyping
2475
2476And of course:
2477
2478\starttyping[option=LUA]
2479function node.direct.getscripts (
2480 <t:direct> primescript,
2481 <t:direct> superscript,
2482 <t:direct> subscript,
2483 <t:direct> superprescript,
2484 <t:direct> subprescript
2485)
2486 no return values
2487end
2488\stoptyping
2489
2490In the discretionaries subsection we mention accessing pre, post and replace
2491fields. These functions can also be used for \type {choice} nodes. Discussing
2492this is currently beyond this manual.
2493
2494\stopsubsection
2495
2496\startsubsection[title={MVL}]
2497
2498Some properties of the currently used main vertical list can be fetched with:
2499
2500\starttyping[option=LUA]
2501function node.direct.getmvllist (
2502 currently no parameters
2503)
2504 return
2505 <t:direct>, head
2506 <t:direct>, tail
2507 <t:integer> mvl
2508end
2509\stoptyping
2510
2511\stopsubsection
2512
2513\startsubsection[title={Balancing}]
2514
2515The \type {node.direct.vbalance} function will either disappear or get accompanied
2516by related helpers (mirroring primitives); it depends on what \CONTEXT\ needs.
2517
2518Updating marks is done with the following set of helpers, that just call the code
2519that does the same before handing over content to the output routine:
2520
2521\starttyping[option=LUA]
2522function nodes.direct.updatetopmarks ( )
2523 return <t:boolean> done
2524end
2525
2526function nodes.direct.updatefirstmarks ( )
2527 return <t:boolean> done
2528end
2529
2530function nodes.direct.updatefirstandbotmark ( <t:direct> box )
2531 no return value
2532end
2533
2534function nodes.direct.updatemarks ( <t:direct> box )
2535 return <t:boolean> done
2536end
2537\stoptyping
2538
2539\stopsubsection
2540
2541\startsubsection[title={\SYNCTEX}]
2542
2543You can set and query the \SYNCTEX\ fields, a file number aka tag and a line
2544number, for a \type {glue}, \type {kern}, \type {hlist}, \type {vlist}, \type
2545{rule} and \type {math} nodes as well as \type {glyph} nodes (although this last
2546one is not used in native \SYNCTEX).
2547
2548\starttyping[option=LUA]
2549function node.direct.setsynctexfields ( <t:integer> fileid, <t:integer> line )
2550 no return values
2551end
2552
2553function node.direct.getsynctexfields ( <t:direct> n )
2554 return
2555 <t:integer>, fileid
2556 <t:integer> line
2557end
2558\stoptyping
2559
2560Of course you need to know what youre doing as no checking on sane values takes
2561place. Also, the \SYNCTEX\ interpreter used in editors is rather peculiar and has
2562some assumptions (heuristics) and there are different incompatible versions
2563floating around. Even more important to notice is that the engine doesnt do
2564anything with this so support is upto \LUA.
2565
2566\stopsubsection
2567
2568\startsubsection[title={Two access models}]
2569
2570Deep down in \TEX\ a node has a number which is a numeric entry in a memory
2571table. In fact, this model, where \TEX\ manages memory is real fast and one of
2572the reasons why plugging in callbacks that operate on nodes is quite fast too.
2573Each node gets a number that is in fact an index in the memory table and that
2574number often is reported when you print node related information. You go from
2575user data nodes and there numeric references and back with:
2576
2577\starttyping[option=LUA]
2578function node.todirect ( <t:node> n) return <t:direct> end
2579function node.tonode ( <t:direct> d) return <t:node> end
2580\stoptyping
2581
2582The user data model is rather robust as it is a virtual interface with some
2583additional checking while the more direct access which uses the node numbers
2584directly. However, even with user data you can get into troubles when you free
2585nodes that are no longer allocated or mess up lists. If you apply \type
2586{tostring} to a node you see its internal (direct) number and id.
2587
2588The userdata model provides key based access while the direct model always
2589accesses fields via functions:
2590
2591\starttyping[option=LUA]
2592local c = nodeobject.char
2593local c = getfield(nodenumber,"char")
2594\stoptyping
2595
2596If you use the direct model, even if you know that you deal with numbers, you
2597should not depend on that property but treat it as an abstraction just like
2598traditional nodes. In fact, the fact that we use a simple basic datatype has the
2599penalty that less checking can be done, but less checking is also the reason why
2600its somewhat faster. An important aspect is that one cannot mix both methods,
2601but you can cast both models. So, multiplying a node number makes no sense.
2602
2603So our advice is: use the indexed (table) approach when possible and investigate
2604the direct one when speed might be a real issue. For that reason \LUATEX\ also
2605provide the \type {get*} and \type {set*} functions in the top level node
2606namespace. There is a limited set of getters. When implementing this direct
2607approach the regular index by key variant was also optimized, so direct access
2608only makes sense when nodes are accessed millions of times (which happens in some
2609font processing for instance).
2610
2611Were talking mostly of getters because setters are less important. Documents
2612have not that many content related nodes and setting many thousands of properties
2613is hardly a burden contrary to millions of consultations.
2614
2615Normally you will access nodes like this:
2616
2617\starttyping
2618local next = current.next
2619if next then
2620 do something
2621end
2622\stoptyping
2623
2624Here \type {next} is not a real field, but a virtual one. Accessing it results in
2625a metatable method being called. In practice it boils down to looking up the node
2626type and based on the node type checking for the field name. In a worst case you
2627have a node type that sits at the end of the lookup list and a field that is last
2628in the lookup chain. However, in successive versions of \LUATEX\ these lookups
2629have been optimized and the most frequently accessed nodes and fields have a
2630higher priority.
2631
2632In the direct namespace there are more helpers and most of them are accompanied
2633by setters. The getters and setters are clever enough to see what node is meant.
2634We dont deal with whatsit nodes: their fields are always accessed by name. It
2635doesnt make sense to add getters for all fields, we just identifier the most
2636likely candidates. In complex documents, many node and fields types never get
2637seen, or seen only a few times, but for instance glyphs are candidates for such
2638optimization.
2639
2640In previous sections we only show the functions in the \type {node.direct}
2641namespace. The following functions are available in both \type {node} and \type
2642{node.direct}:
2643
2644\startthreerows
2645\startluacode
2646 context.starttabulate { "|T|" }
2647 for k, v in table.sortedhash(node.direct) do
2648 if type(v) == "function" and node[k] then
2649 context.NC() context(k)
2650 context.NC() context.NR()
2651 end
2652 end
2653 context.stoptabulate()
2654\stopluacode
2655\stopthreerows
2656
2657In \CONTEXT\ these are duplicated in \type {nodes.nuts} so that is the reference.
2658Quite some functions gets mapped onto the \type {nodes} namespace. In addition we
2659emulate some userdata functions and add some of our own. We show them here because
2660this manual takes \CONTEXT\ as reference.
2661
2662\starttworows
2663\startluacode
2664 context.starttabulate { "|T|c|c|" }
2665 context.BC() context("node.direct")
2666 context.BC() context("node")
2667 context.BC() context("nodes")
2668 context.NC() context.NR()
2669 context.NC()
2670 context.NC()
2671 context.NC()
2672 context.NC() context.NR()
2673
2674 for k, v in table.sortedhash(node.direct) do
2675 if type(v) == "function" then
2676 context.NC() context(k)
2677 context.NC() if node [k] then context("$⋆$") end
2678 context.NC() if nodes[k] then context("$⋆$") end
2679 context.NC() context.NR()
2680 end
2681 end
2682 context.stoptabulate()
2683\stopluacode
2684\stoptworows
2685
2686The following functions are in the \CONTEXT\ \type {nodes} namespace but dont
2687come from the library. Again, we show them here because \CONTEXT\ is the
2688reference.
2689
2690\starttworows
2691\startluacode
2692 context.starttabulate { "|T|c|c|" }
2693 context.BC() context("nodes")
2694 context.BC() context("nodes.nuts")
2695 context.BC() context("node")
2696 context.NC() context.NR()
2697 context.NC()
2698 context.NC()
2699 context.NC()
2700 context.NC() context.NR()
2701 for k, v in table.sortedhash(nodes) do
2702 if type(v) == "function" and not node.direct[k] then
2703 context.NC() context(k)
2704 context.NC() if nodes.nuts[k] then context("$⋆$") end
2705 context.NC() if node [k] then context("$⋆$") end
2706 context.NC() context.NR()
2707 end
2708 end
2709 context.stoptabulate()
2710\stopluacode
2711\stoptworows
2712
2713We have quite some helpers and some accept different node types. Here is the
2714repertoire:
2715
2716\startluacode
2717 context.starttabulate { "|lT|plT|" }
2718 for name, usage in table.sortedhash(moduledata.repertoire) do
2719 context.NC() context(name)
2720 context.NC() context("% t",table.sortedkeys(usage))
2721 context.NC() context.NR()
2722 end
2723 context.stoptabulate()
2724\stopluacode
2725
2726\stopsubsection
2727
2728\startsubsection[title={Properties}]
2729
2730Attributes are a convenient way to relate extra information to a node. You can
2731assign them at the \TEX\ end as well as at the \LUA\ end and consult them at the
2732\LUA\ end. One big advantage is that they obey grouping. They are linked lists
2733and normally checking for them is pretty efficient, even if you use a lot of
2734them. A macro package has to provide some way to manage these attributes at the
2735\TEX\ end because otherwise clashes in their usage can occur.
2736
2737Each node also can have a properties table and you can assign values to this
2738table using the \type {setproperty} function and get properties using the \type
2739{getproperty} function. Managing properties is way more demanding than managing
2740attributes.
2741
2742Take the following example:
2743
2744\starttyping
2745\directlua {
2746 local n = node.new("glyph")
2747
2748 node.setproperty(n,"foo")
2749 print(node.getproperty(n))
2750
2751 node.setproperty(n,"bar")
2752 print(node.getproperty(n))
2753
2754 node.free(n)
2755}
2756\stoptyping
2757
2758This will print \type {foo} and \type {bar} which in itself is not that useful
2759when multiple mechanisms want to use this feature. A variant is:
2760
2761\starttyping
2762\directlua {
2763 local n = node.new("glyph")
2764
2765 node.setproperty(n,{ one = "foo", two = "bar" })
2766 print(node.getproperty(n).one)
2767 print(node.getproperty(n).two)
2768
2769 node.free(n)
2770}
2771\stoptyping
2772
2773This time we store two properties with the node. It really makes sense to have a
2774table as property because that way we can store more. But in order for that to
2775work well you need to do it this way:
2776
2777\starttyping
2778\directlua {
2779 local n = node.new("glyph")
2780
2781 local t = node.getproperty(n)
2782
2783 if not t then
2784 t = { }
2785 node.setproperty(n,t)
2786 end
2787 t.one = "foo"
2788 t.two = "bar"
2789
2790 print(node.getproperty(n).one)
2791 print(node.getproperty(n).two)
2792
2793 node.free(n)
2794}
2795\stoptyping
2796
2797Here our own properties will not overwrite other users properties unless of
2798course they use the same keys. So, eventually you will end up with something:
2799
2800\starttyping
2801\directlua {
2802 local n = node.new("glyph")
2803
2804 local t = node.getproperty(n)
2805
2806 if not t then
2807 t = { }
2808 node.setproperty(n,t)
2809 end
2810 t.myself = { one = "foo", two = "bar" }
2811
2812 print(node.getproperty(n).myself.one)
2813 print(node.getproperty(n).myself.two)
2814
2815 node.free(n)
2816}
2817\stoptyping
2818
2819This assumes that only you use \type {myself} as subtable. The possibilities are
2820endless but care is needed. For instance, the generic font handler that ships
2821with \CONTEXT\ uses the \type {injections} subtable and you should not mess with
2822that one!
2823
2824There are a few helper functions that you normally should not touch as user: \typ
2825{getpropertiestable} and will give the table that stores properties (using
2826direct entries) and you can best not mess too much with that one either because
2827\LUAMETATEX\ itself will make sure that entries related to nodes will get wiped when
2828nodes get freed, so that the \LUA\ garbage collector can do its job. In fact, the
2829main reason why we have this mechanism is that it saves the user (or macro
2830package) some work. One can easily write a property mechanism in \LUA\ where
2831after a shipout properties gets cleaned up but its not entirely trivial to make
2832sure that with each freed node also its properties get freed, due to the fact
2833that there can be nodes left over for a next page. And having a callback bound to
2834the node deallocator would add way to much overhead.
2835
2836When we copy a node list that has a table as property, there are several
2837possibilities: we do the same as a new node, we copy the entry to the table in
2838properties (a reference), we do a deep copy of a table in the properties, we
2839create a new table and give it the original one as a metatable. After some
2840experiments (that also included timing) with these scenarios we decided that a
2841deep copy made no sense, nor did nilling. In the end both the shallow copy and
2842the metatable variant were both ok, although the second one is slower. The most
2843important aspect to keep in mind is that references to other nodes in properties
2844no longer can be valid for that copy. We could use two tables (one unique and one
2845shared) or metatables but that only complicates matters.
2846
2847When defining a new node, we could already allocate a table but it is rather easy
2848to do that at the lua end e.g.\ using a metatable \type {index} method. That
2849way it is under macro package control. When deleting a node, we could keep the
2850slot (e.g. setting it to false) but it could make memory consumption raise
2851unneeded when we have temporary large node lists and after that only small lists.
2852Both are not done because in the end this is what happens now: when a node is
2853copied, and it has a table as property, the new node will share that table. The
2854copy gets its own table with the original table as metatable.
2855
2856A few more experiments were done. For instance: copy attributes to the properties
2857so that we have fast access at the \LUA\ end. In the end the overhead is not
2858compensated by speed and convenience, in fact, attributes are not that slow when
2859it comes to accessing them. So this was rejected.
2860
2861Another experiment concerned a bitset in the node but again the gain compared to
2862attributes could be neglected and given the small amount of available bits it also
2863demands a pretty strong agreement over what bit represents what, and this is
2864unlikely to succeed in the \TEX\ community. It doesnt pay off.
2865
2866Just in case one wonders why properties make sense: it is not so much speed that
2867we gain, but more convenience: storing all kinds of (temporary) data in attributes
2868is no fun and this mechanism makes sure that properties are cleaned up when a
2869node is freed. Also, the advantage of a more or less global properties table is
2870that we stay at the \LUA\ end. An alternative is to store a reference in the node
2871itself but that is complicated by the fact that the register has some limitations
2872(no numeric keys) and we also dont want to mess with it too much.
2873
2874\stopsubsection
2875
2876\stopsection
2877
2878\stopdocument
2879 |