Source Browser ?

1/*
2    See license.txt in the root of this project.
3*/
4
5# ifndef LMT_TEXNODES_H
6# define LMT_TEXNODES_H
7
8/*tex
9
10    We can probably ditch |volatile| so that the compiler can optimize access a bit better. We
11    only need to make sure that we create nodes before we use their pointers. So, beware: a
12    newnode has to go via an intermediate variable because the |varmem| array can have been be
13    reallocated. I need to (re)check all cases! In case of a copy we use a intermediate volatile
14    variable.
15
16    Anyway, we now have only a few |quarterwords| in use, most noticeably the type and subtype.
17    Eventually I might go for a consistent
18
19    type      subtype
20    prev      next
21    attribute data
22    etc
23
24    model. Or maybe even just a flat list, no need for memoryword, but just all halfwords. However,
25    it will demand all kind of tiny adaptations and we don't gain much. We'd also loose some charm
26    of traditional \TEX. Also, we now have a double glue related field and that would then become
27    a float. So, not now.
28
29    There are a few more node types than in standard \TEX, but less than we have in e.g.\ \PDFTEX\
30    or stock \LUATEX. For instance margin nodes are now just kern nodes, some whatits are first
31    class nodes and we have only one generic whatsit left. We also have more subtypes which makes
32    a more detailed tracking of where nodes come from possible. Other nodes, like the |inserting|
33    and |split_up| nodes are ot both |inserting| but with a subtype because the register index is
34    no longer the subtype.
35
36    Not all nodes can end up in a node list. Some are used for housekeeping (stack, expressions,
37    conditional nesting, etc.) or show up in the process of breaking paragraphs into lines. When
38    we talk of nodes with users in the perspective of \TEX\ we normally refer to the ones in
39    horizontal and vertical lists or math lists, not to those more obscure housekeeping nodes. It
40    just happens that they share the same memory model and management.
41
42    A complication is that some nodes have pointers that themselves point to a (often smaller)
43    node but use the same accessors. This means that (1) their layout should be the same with
44    respect to the pointer, which happens with span nodes or (2) that there is some offset in play,
45    which happens with ins pointers and break nodes that are embedded in a disc node.
46
47    Now that we no longer have variable nodes, we can consider a different allocation model, like
48    a chain of malloced nodes but on the other hand storing them might be more work. We also cannot
49    longer share the accessors so again more work is needed. But ... maybe attributes might end up
50    as allocated lists some day, but that also demands storage. The current memory management is
51    very efficient and we don't gain anything with redoing that, apart maybe from nodes becoming
52    structs. Even then we will have an array of pointers instead of what we have now, but without
53    the jumps by side in the indices. So, given the constraints of offsets and overlap it makes no
54    sense to waste time on this.
55
56    Instead of |var_mem| we use |nodes| and related names. This better complements the additional
57    variables that we have for dynamic management. Some more names have been changed (also in order
58    to avoid side effect in syntax highlighting). Too common names also result in too many matches
59    when searching the source tree.
60
61    Soo, eventually most fields now have the type of the node in their name, which makes it more
62    clear what they are. As mentioned, it makes the syntax highlighted source look better as some
63    generic names are used elsewhere too. Another reason is that we have more fields in nodes and
64    when browsing the source it helps to know that a |width| is actually the |glue_amount| which
65    when we go down is actually a height anyway. It also makes it possible at some point to make
66    some nodes smaller when we don't need these \quote {shared by name} fields. We also need this
67    transition in order to get better interfacing to the \LUA\ end, one reason being that we need
68    to distinguish between fields that overlap (as in lists, unset nodes and and alignment
69    records).
70
71    Todo: all subtype related constants will become |_subtype| so that also means a couple more
72    _code ones for commands. It's all about consistency but that will happen stepwise. A typical
73    rainy day with some newly acquired music in background kind of activity:
74
75    - discretionary
76    - adjust
77    - noad
78    - fence
79    - radical
80    - boundary
81
82    Before we went on with \LUAMETATEX\ already quite a bit was added to \LUATEX\ (granularity in
83    subtypes, more of them, etc.) but here we add more. Some day I might make a list of difference
84    just for historic reasons. Don't assume equivalence: some in \LUATEX\ was sort of experimental
85    and improved over time and maybe should not have been added to \LUATEX\ at all and been delayed
86    for \LUAMETATEX\ instead (as it's probably not used), so that core \LUATEX\ stayed closer the
87    other engines.
88
89*/
90
91typedef enum node_types {
92    hlist_node,
93    vlist_node,
94    rule_node,
95    insert_node,
96    mark_node,
97    adjust_node,
98    boundary_node,
99    disc_node,
100    whatsit_node,
101    /*tex The last_preceding_break_node: */
102    par_node,
103    dir_node,
104    /*tex The last_non_discardable_node: */
105    math_node,
106    glue_node,
107    kern_node,
108    penalty_node,
109    style_node,
110    choice_node,
111    parameter_node,
112    simple_noad,
113    radical_noad,
114    fraction_noad,
115    accent_noad,
116    fence_noad,
117    math_char_node,
118    math_text_char_node,
119    sub_box_node,
120    sub_mlist_node,
121    delimiter_node,
122    glyph_node,
123    /*tex This was the last node with attributes, except unset nodes. */
124    unset_node,
125    specification_node,
126    align_record_node,
127    attribute_node,
128    glue_spec_node,
129    temp_node,
130    split_node,
131    /*tex The next set of nodes is invisible from the \LUA\ (but nesting nodes can show up). */
132    expression_node,
133    lmtx_expression_node,
134    rpn_expression_node,
135    loop_state_node,
136    math_spec_node,
137    font_spec_node,
138    nesting_node,
139    span_node,
140    align_stack_node,
141 // noad_state_node,
142    if_node,
143    /*tex These two are active nodes. */
144    unhyphenated_node,
145    hyphenated_node,
146    /*tex This one can also be in the active list. */
147    delta_node,
148    /*tex While this is an indirect one carrying data. */
149    passive_node,
150} node_types;
151
152# define first_nodetype hlist_node
153# define last_nodetype  passive_node
154
155 /*tex The largest node is now |par_node| which need 32 slots. */
156
157# define max_chain_size       40
158
159# define unknown_node_type    -1
160# define unknown_node_subtype -1
161
162/* Todo: [type] [subtype|size] [index] -> nodes : advantage is no holes in node id's */
163
164typedef struct node_memory_state_info {
165    memoryword  *nodes;
166 // memoryword  *volatile nodes;
167    char        *nodesizes;
168    halfword     free_chain[max_chain_size];
169    memory_data  nodes_data;
170    int          reserved; /*tex There are some predefined nodes. */
171    int          padding;
172    int          node_properties_id;
173    int          lua_properties_level;
174    halfword     attribute_cache;
175    halfword     max_used_attribute;
176    int          node_properties_table_size;
177} node_memory_state_info;
178
179extern node_memory_state_info lmt_node_memory_state;
180
181typedef enum field_types {
182    nil_field,
183    integer_field,
184    dimension_field,
185    glue_field,
186    number_field,
187    string_field,
188    boolean_field,
189    function_field,
190    node_field,
191    node_list_field,
192    token_field,
193    token_list_field,
194    attribute_field,
195} field_types;
196
197extern halfword tex_get_node                    (int size);
198extern void     tex_free_node                   (halfword p, int size);
199extern void     tex_dump_node_mem               (dumpstream f);
200extern void     tex_undump_node_mem             (dumpstream f);
201extern int      tex_used_node_count             (void);
202extern int      tex_free_node_count             (void);
203extern void     tex_initialize_node_mem         (void);
204extern void     tex_initialize_nodes            (void);
205
206extern void     lmt_nodelib_initialize          (void); /* name ? */
207
208extern void     tex_dump_specification_data     (dumpstream f);
209extern void     tex_undump_specification_data   (dumpstream f);
210
211/*tex
212
213    Most fields are integers (halfwords) that get aliased to |vinfo| and |vlink| for traditional
214    reasons. The |vlink| name is actually representing a next pointer. Only the type and subtype
215    remain quarterwords, the rest are just halfwords which wastes space for directions,
216    orientation, glue orders and glue signs but so be it.
217
218    A memory word has two 32 bit integers so 8 bytes. A glueratio is a double which is 8 bytes so
219    there we waste some space. There is actually no need now to pack (node) data in pairs so maybe
220    some day I'll change that. When we make glue ration a float again we can go flat (and with most
221    node fields now being fully qualified that is easier).
222
223    The first memoryword contains the |type| and |subtype| that are both 16 bit integers (unsigned)
224    as well as the |vlink| (next) pointer. After that comes the word that keeps the |attr| and
225    |alink| (prev) fields. Instead of the link names we use more meaningful ones. The |next|, |prev|
226    and |attr| fields all are halfwords representing a node index.
227
228    The |node_size| field is used in managing freed nodes (mostly as a check) and it overwrites the
229    |type| and |subtype| fields. Actually we could just use the type or subtype but as the size is
230    small but on the other hand using an int here makes sense.
231
232    half0 | quart0 quart1 | vinfo | size | type subtype
233    half1 |               | vlink
234
235    The |tlink| and |rlink| fields are used in disc nodes as tail and replace pointers (again
236    halfwords). We no longer need |rlink| as it's equivalent to |alink| (the prev pointer). The
237    |tlink| fields is used for links to the tail of a list. These indirect macros are somewhat
238    complicating matters. Again these have been renamed.
239
240    We used to have |alink(a)| being |vlink(a,1)| but that has (after a few years) been replaced by
241    |node_prev| because is cleaner. Keep in mind that in \LUATEX\ we use double linked node lists. So,
242    we now only have some \quote {hard coded} pointers to the memory array in this file, not in the
243    files that use the node fields. However, for the next two paragraphs to be true, I need to find
244    a solution for the insert_ptr first because that is an index.
245
246    Now, a logical question is: should we stick to the link and info model for nodes? One reason
247    is that we share the model with tokens. A less relevant reason is that the glue is stored in 8
248    bytes but that can be reverted to 4 bytes if needed. So, indeed at some point we might see a 32
249    bit wide array show up here as we're now more or less prepared for that. It will bump the direct
250    node numbers but that should work out okay. So, in the end, after stepwise abstraction we now
251    have field definitions that use a base and offset e.g. |vlink(a,3)| instead of |vlink(a+3)|.
252    Also, we have many more fields and using meaningful names quickly started to make sense.
253
254    Once all is stable I will play with |var_mem| being an array of pointers and |malloc| the
255    smaller memoryword arrays (per node). This might lead to (not always) smaller memory footprint:
256    we have one pointer per node (but only that array gets preallocated) but we need less memory in
257    total, unless we use many nodes. Anyway, we keep the  indirect model (which might add overhead,
258    but that can be compensated by \CPU\ caches) because using a numeric node pointer is more
259    efficient and quite handy. If we would go completely struct the source would change so much that
260    we loose the charm of \TEX\ and documentation and there is no gain in it. Also, using halfword
261    indices (but then to pointers) for nodes has the huge advantage that it is fast in \LUA\ (always
262    a bottleneck) and these node indices can (and have to) be stored in tokens. One nice side effect
263    would be that we have node indices in a sequence (without the current jumps due to node size
264    offset, which in turn gives more room for nodes references in tokens).
265
266    In spite of all extensions we hope the spirit of how \TEX\ does it is still very visible.
267
268*/
269
270# define mvalue(a,b)  lmt_node_memory_state.nodes[a+b].P
271# define lvalue(a,b)  lmt_node_memory_state.nodes[a+b].L
272# define dvalue(a,b)  lmt_node_memory_state.nodes[a+b].D
273
274# define vinfo(a,b)   lmt_node_memory_state.nodes[a+b].half0
275# define vlink(a,b)   lmt_node_memory_state.nodes[a+b].half1
276
277# define vinfo0(a,b)  lmt_node_memory_state.nodes[a+b].quart00
278# define vinfo1(a,b)  lmt_node_memory_state.nodes[a+b].quart01
279# define vlink0(a,b)  lmt_node_memory_state.nodes[a+b].quart10
280# define vlink1(a,b)  lmt_node_memory_state.nodes[a+b].quart11
281
282# define vinfo0s(a,b)  lmt_node_memory_state.nodes[a+b].short00
283# define vinfo1s(a,b)  lmt_node_memory_state.nodes[a+b].short01
284# define vlink0s(a,b)  lmt_node_memory_state.nodes[a+b].short10
285# define vlink1s(a,b)  lmt_node_memory_state.nodes[a+b].short11
286
287# define vinfo00(a,b) lmt_node_memory_state.nodes[a+b].single00
288# define vinfo01(a,b) lmt_node_memory_state.nodes[a+b].single01
289# define vinfo02(a,b) lmt_node_memory_state.nodes[a+b].single02
290# define vinfo03(a,b) lmt_node_memory_state.nodes[a+b].single03
291# define vlink00(a,b) lmt_node_memory_state.nodes[a+b].single10
292# define vlink01(a,b) lmt_node_memory_state.nodes[a+b].single11
293# define vlink02(a,b) lmt_node_memory_state.nodes[a+b].single12
294# define vlink03(a,b) lmt_node_memory_state.nodes[a+b].single13
295
296/*tex
297    We have some shared field names. Some day the subtypes will get meaningful names dependent on
298    the node type, if only because some already have. We used to have
299
300    \starttyping
301    # define type(a)         vinfo0(a,0)
302    # define subtype(a)      vinfo1(a,0)
303    # define node_size(a)    vinfo(a,0)
304    \stoptyping
305
306    but we dropped the size mechanism and made most field shortcuts verbose in order to be able to
307    use variable names with the same name combined with proper syntax highlighting etc. It also
308    gives less noise when we search in the whole source tree. More later.
309*/
310
311# define node_type(a)    vinfo0(a,0)
312# define node_subtype(a) vinfo1(a,0)
313
314# define node_next(a) vlink(a,0)
315# define node_prev(a) vlink(a,1)
316# define node_attr(a) vinfo(a,1)
317
318# define node_head(a) vlink(a,0) /*tex the head |hlink(a)| aka |vlink(a)| of a disc sublist */
319# define node_tail(a) vinfo(a,1) /*tex the tail |tlink(a)| aka |vinfo(a)|, overlaps with |node_attr()| */
320
321/*tex
322
323    The dimension fields shared their locations which made for sometimes more compact code but
324    in the end the number of placxes where it really saved code were limited. Also, compilers will
325    do their job and deal with common code. So, these are now replaced by more meaningful names:
326
327    \starttyping
328    # define width(a)  vlink(a,2)
329    # define depth(a)  vlink(a,3)
330    # define height(a) vlink(a,4)
331    \stoptyping
332
333    Inserts use a trick. The insert pointers directly point into a node at the place where the list
334    starts which is why |list_ptr| has to overlap with |node_next|! I have looked into changign this
335    but it doesn't pay off and it's best to stay close to the original. A side effect is that some
336    fields in insert nodes are sort of impossible (for now).
337
338    \starttyping
339    # define box_list_ptr(a) vlink(a,5) // We need to use the same field as |node_next|.
340    # define insert_list(a) (a + 5) // This kind of makes a virtual node: start at list.
341    \stoptyping
342
343    Beware of the fact that for instance alignments use some fields for other purposes, like:
344    |u_part(a)|, |v_part(a)|, |span_ptr(a)|, etc. and assume the rest of the fields to overlap
345    with list nodes. So, we cannot simply reshuffle positions!
346
347    In the original \TEX\ source (and also in \LUATEX) there are a couple of offsets used. Most
348    noticeably is the |list_offset| but in 2.0618 the related trickery was replaced by using
349    |list_ptr| and using the fact that we have a doubel linked list. The four fields are in
350    successive memory words and that means that we can use |node_next| in a field pointed to
351    by |list_offset| (because actually we then have the list pointer!). This makes for simple
352    loops in original \TEX. The dimension offsets are used to set fields in boxed but we already
353    abstracted that to proper field names; these were for instance used in alignment nodes that
354    have mostly the same properties as a box node.
355
356    \starttyping
357    # define width_offset  2
358    # define depth_offset  3
359    # define height_offset 4
360    # define list_offset   5
361    \stoptyping
362
363    These abstractions mean that we now have nodes, fields and offsets all abstracted in such a way
364    that all definitions and trickery in in this file. Of course I could have messed up.
365
366*/
367
368/*tex
369
370    Syntex supports demands some extra fields in nodes that makes it possible to output location as
371    well as file/line information for viewer-editor synchronization. The ideas is quite okay but
372    unfortunately the implementation of the library is rather bound to the way e.g. \LATEX\ typesets
373    documents. Synctex has always been problematic when it comes to \CONTEXT. There is for instance
374    no control over filenames and discussions around some limitations (and possible features) in the
375    \PDFTEX\ and early \LUATEX\ times never resulted in fixing that (setting filenames, adding some
376    additional synchronization points, etc). All that was supposed to happen deep down in the library
377    and was not considered to be dealt with by a macro package. For instance multiple loading of the
378    same file (metapost runs or smaple files) was a problem as was the need to block access to files
379    in tds (like styles). We also needed binding to for instance elements in an \XML\ file where line
380    numbers are sort of special and out of sync with inclusion. I guess we were ahead of the pack
381    because after nearly two decades of \LUATEX\ there is some discussion about this.
382
383    Anyway, for the reasons mentioned \LUATEX\ offers some setters that overload the engine ones and
384    that permits \CONTEXT\ to implement its own variant. However, in \LUAMETATEX\ setting tags and
385    lines from \LUA\ is now the only way to support \SYNCTEX\ because the library is absent: we just
386    have some extra fields in some nodes. In \LUAMETATEX\ only glyph and list nodes have these fields
387    as it makes no sense to have them elsewhere: macro packages can add glue and kerns and rules and
388    \unknown\ all over the place and adding file state info there only makes things confusing and
389    working less well. This is what the mode parameter can handle in \LUATEX\ and in \LUAMETATEX\ we
390    have no control other than what the macro package provides. The engine can set the file and line
391    fields or it can be done from \LUA. Automatic setting is controlled by |input_file_state.mode|.
392
393    As a side note: the fact that a viewer needs to embed the library is also a limitation. Calling
394    out to an external program that analyzes the file and gives back the filename and line is more
395    flexible and robust. Because we have such an analyzer in \MKIV\ it was no big deal to add a few
396    lines so that the \TEX shop environment could use that script/method (bidirectional); hopefully
397    other viewers and editors will follow.
398
399*/
400
401/*
402
403    Temporary nodes are really special head node pointers that only need links. They ensure that
404    there is at least one node in a list.
405
406*/
407
408# define temp_node_size 2
409
410/*tex
411
412    In \LUATEX\ we have attribute list nodes and attribute nodes that are (anyway) of the same
413    size. In the end I decided to combine them into one node with a subtype. That also helps
414    diagnose issues. It is one of the few nodes now that has fields depending on the subtype
415    but these nodes are not really user ones anyway.
416
417    We could move index to the subtype and then have an extra data field but I'll delay that till
418    it's really needed. Chains are often short and using arrays doesn't bring any gain nor does
419    using some btree approach.
420
421*/
422
423# define attribute_node_size 2
424# define attribute_unset(a)  vinfo(a,1)
425# define attribute_index(a)  vinfo0(a,1)
426# define attribute_detail(a) vinfo1(a,1)
427# define attribute_count(a)  vlink(a,1)  /*tex the reference count */
428# define attribute_value(a)  vlink(a,1)
429
430typedef enum attribute_subtypes {
431    attribute_list_subtype,
432    attribute_value_subtype,
433} attribute_subtypes;
434
435# define last_attribute_subtype attribute_value_subtype
436
437/*tex
438    Penalties have only one primitive so we don't have |_code| here, also because it would conflict
439    with arguments.
440*/
441
442# define penalty_node_size     5
443# define penalty_amount(a)     vlink(a,2)
444# define penalty_options(a)    vinfo(a,2)
445# define penalty_tnuoma(a)     vlink(a,3)
446# define penalty_used(a)       vinfo(a,3) /* used internally */
447# define penalty_belongs_to(a) vlink(a,4) /* used internally */
448# define penalty_reserved(a)   vinfo(a,4) /* used internally */
449
450static inline void tex_add_penalty_option    (halfword a, halfword r) { penalty_options(a) |= r; }
451static inline void tex_remove_penalty_option (halfword a, halfword r) { penalty_options(a) &= ~r; }
452static inline int  tex_has_penalty_option    (halfword a, halfword r) { return (penalty_options(a) & r) == r; }
453
454typedef enum penalty_option_codes {
455    penalty_option_normal        = 0x0000,
456    penalty_option_math_forward  = 0x0001,
457    penalty_option_math_backward = 0x0002,
458    penalty_option_orphaned      = 0x0004,
459    penalty_option_widowed       = 0x0008,
460    penalty_option_clubbed       = 0x0010,
461    penalty_option_toddlered     = 0x0020,
462    penalty_option_widow         = 0x0040,
463    penalty_option_club          = 0x0080,
464    penalty_option_broken        = 0x0100,
465    penalty_option_shaping       = 0x0200,
466    penalty_option_double        = 0x0400,
467    penalty_option_double_used   = 0x0800,
468    penalty_option_factor_used   = 0x1000,
469    penalty_option_end_of_par    = 0x2000,
470    penalty_option_in_insert     = 0x4000,
471    penalty_option_final_balance = 0x8000,
472} penalty_option_codes;
473
474typedef enum penalty_subtypes {
475    user_penalty_subtype,
476    linebreak_penalty_subtype, /*tex includes widow, club, broken etc. */
477    line_penalty_subtype,
478    word_penalty_subtype,
479    orphan_penalty_subtype,
480    toddler_penalty_subtype,
481    single_line_penalty_subtype,
482    final_penalty_subtype,
483    math_pre_penalty_subtype,
484    math_post_penalty_subtype,
485    before_display_penalty_subtype,
486    after_display_penalty_subtype,
487    equation_number_penalty_subtype,
488    discretionary_penalty_subtype,      /* not (yet) used */
489} penalty_subtypes;
490
491# define last_penalty_subtype discretionary_penalty_subtype
492
493/*tex
494    We have plenty of glue variables and in the node lists most are also flagged. There is no
495    one|-|to|-|one correspondence between the codes (in tokens) and subtypes (in nodes) as listed
496    below, but they come close. The special math related glues and inserts now have nicer numbers.
497*/
498
499typedef enum glue_subtypes {
500    user_skip_glue,
501    line_skip_glue,
502    baseline_skip_glue,
503    par_skip_glue,
504    above_display_skip_glue,
505    below_display_skip_glue,
506    above_display_short_skip_glue,
507    below_display_short_skip_glue,
508    left_skip_glue,
509    right_skip_glue,
510    top_skip_glue,
511    bottom_skip_glue,
512    split_top_skip_glue,
513    tab_skip_glue,
514    space_skip_glue,
515    xspace_skip_glue,
516    zero_space_skip_glue,
517    par_fill_left_skip_glue,
518    par_fill_right_skip_glue,
519    par_init_left_skip_glue,
520    par_init_right_skip_glue,
521    indent_skip_glue,
522    left_hang_skip_glue,
523    right_hang_skip_glue,
524    correction_skip_glue,
525    inter_math_skip_glue,
526    ignored_glue,           /*tex |subtype| for cases where we ignore zero glue (alignments) */
527    page_glue,              /*tex |subtype| used in the page builder */
528    /*tex math */
529    math_skip_glue,
530    thin_mu_skip_glue,
531    med_mu_skip_glue,
532    thick_mu_skip_glue,
533    /*tex more math */
534    conditional_math_glue,  /*tex special |subtype| to suppress glue in the next node */ /* no need for jump */
535    rulebased_math_glue,
536    mu_glue,                /*tex |subtype| for math glue */
537    /*tex leaders (glue with list) */
538    a_leaders,              /*tex |subtype| for aligned leaders */
539    c_leaders,              /*tex |subtype| for centered leaders */
540    x_leaders,              /*tex |subtype| for expanded leaders */
541    g_leaders,              /*tex |subtype| for global (page) leaders */
542    u_leaders,
543} glue_subtypes;
544
545# define last_glue_subtype u_leaders
546
547typedef enum skip_glue_codes_alias {
548    par_fill_skip_glue = par_fill_right_skip_glue,
549} skip_glue_codes_alias;
550
551# define is_leader(a) (node_subtype(a) >= a_leaders)
552
553# define glue_node_size        8
554# define glue_spec_size        5
555# define glue_options(a)       vinfo(a,2)
556# define glue_amount(a)        vlink(a,2)
557# define glue_stretch(a)       vinfo(a,3)
558# define glue_shrink(a)        vlink(a,3)
559# define glue_stretch_order(a) vinfo(a,4)
560# define glue_shrink_order(a)  vlink(a,4)
561# define glue_font(a)          vinfo(a,5) /* not in spec */ /* when inter_math_skip_glue: parameter */
562# define glue_data(a)          vlink(a,5) /* not in spec */
563# define glue_leader_ptr(a)    vinfo(a,6) /* not in spec */
564# define glue_callback(a)      vlink(a,6) /* not in spec */
565# define glue_belongs_to(a)    vinfo(a,7) /* not in spec */
566# define glue_reserved(a)      vlink(a,7) /* not in spec */
567
568typedef enum glue_option_codes {
569    glue_option_normal            = 0x0000,
570    glue_option_no_auto_break     = 0x0001, /* math related */
571    glue_option_has_factor        = 0x0002,
572    glue_option_is_limited        = 0x0004,
573    glue_option_limit             = 0x0008,
574    glue_option_u_leaders_line    = 0x0010,
575    glue_option_set_discardable   = 0x0020,
576    glue_option_reset_discardable = 0x0040,
577    glue_option_non_discardable   = 0x0080,
578    glue_option_in_insert         = 0x0100,
579} glue_option_codes;
580
581static inline void tex_add_glue_option    (halfword a, halfword r) { glue_options(a) |= r; }
582static inline void tex_remove_glue_option (halfword a, halfword r) { glue_options(a) &= ~r; }
583static inline int  tex_has_glue_option    (halfword a, halfword r) { return (glue_options(a) & r) == r; }
584
585typedef enum math_subtypes {
586    begin_inline_math,
587    end_inline_math,
588    begin_broken_math,
589    end_broken_math,
590} math_subtypes;
591
592# define last_math_subtype end_broken_math
593
594/*tex
595    Math nodes (currently) partially overlap with glue because they also have a glue property.
596*/
597
598# define math_node_size        7
599# define math_surround(a)      vinfo(a,2)
600# define math_amount(a)        vlink(a,2)
601# define math_stretch(a)       vinfo(a,3)
602# define math_shrink(a)        vlink(a,3)
603# define math_stretch_order(a) vinfo(a,4)
604# define math_shrink_order(a)  vlink(a,4)
605# define math_penalty(a)       vinfo(a,5)
606# define math_options(a)       vlink(a,5)
607# define math_tolerance(a)     vinfo(a,6)
608# define math_pre_tolerance(a) vlink(a,6)
609
610static inline void tex_add_math_option    (halfword a, halfword r) { math_options(a) |= r; }
611static inline void tex_remove_math_option (halfword a, halfword r) { math_options(a) &= ~r; }
612static inline int  tex_has_math_option    (halfword a, halfword r) { return (math_options(a) & r) == r; }
613
614typedef enum math_option_codes {
615    math_option_normal   = 0x0000,
616    math_option_short    = 0x0001,
617    math_option_orphaned = 0x0002,
618    math_option_display  = 0x0004,
619    math_option_cramped  = 0x0008,
620} math_option_codes;
621
622/*tex Here are some (inline) helpers. We need specific ones for math glue. */
623
624static inline int tex_glue_is_zero(halfword g)
625{
626    return (! g) || ((glue_amount(g) == 0) && (glue_stretch(g) == 0) && (glue_shrink(g) == 0));
627}
628
629static inline int tex_math_glue_is_zero(halfword g)
630{
631    return (! g) || ((math_amount(g) == 0) && (math_stretch(g) == 0) && (math_shrink(g) == 0));
632}
633
634static inline int tex_same_glue(halfword a, halfword b)
635{
636    return
637        (a == b) /* same glue specs or both zero */
638     || (a && b && glue_amount(a)        == glue_amount(b)
639                && glue_stretch(a)       == glue_stretch(b)
640                && glue_shrink(a)        == glue_shrink(b)
641                && glue_stretch_order(a) == glue_stretch_order(b)
642                && glue_shrink_order(a)  == glue_shrink_order(b)
643        )
644    ;
645}
646
647static inline void tex_reset_glue_to_zero(halfword target)
648{
649    if (target) {
650        glue_amount(target) = 0;
651        glue_stretch(target) = 0;
652        glue_shrink(target) = 0;
653        glue_stretch_order(target) = 0;
654        glue_shrink_order(target) = 0;
655    }
656}
657
658static inline void tex_reset_math_glue_to_zero(halfword target)
659{
660    if (target) {
661        math_amount(target) = 0;
662        math_stretch(target) = 0;
663        math_shrink(target) = 0;
664        math_stretch_order(target) = 0;
665        math_shrink_order(target) = 0;
666    }
667}
668
669static inline void tex_copy_glue_values(halfword target, halfword source)
670{
671    if (source) {
672        glue_amount(target) = glue_amount(source);
673        glue_stretch(target) = glue_stretch(source);
674        glue_shrink(target) = glue_shrink(source);
675        glue_stretch_order(target) = glue_stretch_order(source);
676        glue_shrink_order(target) = glue_shrink_order(source);
677    } else {
678        glue_amount(target) = 0;
679        glue_stretch(target) = 0;
680        glue_shrink(target) = 0;
681        glue_stretch_order(target) = 0;
682        glue_shrink_order(target) = 0;
683    }
684}
685
686static inline int tex_is_par_init_glue(halfword n)
687{
688    switch (node_subtype(n)) {
689        case indent_skip_glue:
690        case par_init_left_skip_glue:
691        case par_init_right_skip_glue:
692            return 1;
693        default:
694            return 0;
695    }
696}
697
698/*tex
699    Kern nodes are relatively simple. Instead of |width| we use |kern_amount| which  makes more
700    sense: we can go left, right, up or down. Margin kerns have been dropped and are now just a
701    special subtype of regular kerns.
702*/
703
704typedef enum kern_subtypes {
705    explicit_kern_subtype,      /*tex from |\kern| */
706    accent_kern_subtype,        /*tex from accents */
707    font_kern_subtype,
708    italic_kern_subtype,        /*tex from |\/| */
709    left_margin_kern_subtype,
710    right_margin_kern_subtype,
711    left_correction_kern_subtype,
712    right_correction_kern_subtype,
713    space_font_kern_subtype,    /*tex for tracing only */
714    explicit_math_kern_subtype,
715    math_shape_kern_subtype,
716    left_math_slack_kern_subtype,
717    right_math_slack_kern_subtype,
718    horizontal_math_kern_subtype,
719    vertical_math_kern_subtype,
720} kern_subtypes;
721
722# define last_kern_subtype vertical_math_kern_subtype
723
724# define font_related_kern(s) (s >= font_kern_subtype && s <= space_font_kern_subtype)
725
726# define kern_node_size     4
727# define kern_amount(a)     vlink(a,2) 
728# define kern_expansion(a)  vinfo(a,2) /*tex expansion factor (hz) */
729# define kern_options(a)    vlink(a,3)
730# define kern_belongs_to(a) vinfo(a,3)
731
732typedef enum kern_option_codes {
733    kern_option_normal    = 0x0000,
734    kern_option_in_insert = 0x0001,
735} kern_option_codes;
736
737// static inline void tex_add_kern_option    (halfword a, halfword r) { kern_options(a) |= r; }
738// static inline void tex_remove_kern_option (halfword a, halfword r) { kern_options(a) &= ~r; }
739// static inline int  tex_has_kern_option    (halfword a, halfword r) { return (kern_options(a) & r) == r; }
740
741/*tex
742
743    Disc nodes are complicated: they have three embedded nesting nodes to which the |pre_break|,
744    |post_break| and |no_break| fields point. In there we find a head pointer (|vlink| aka |hlink|)
745    and tail pointer (|tlink|). The |alink| pointer is used in the base mode font machinery and is
746    not really a prev pointer. We have to make sure it gets nilled when we communicate with \LUA.
747
748    The no-, pre-, and postbreak fields point to nesting nodes that are part of the disc node (three
749    times two memorywords). Sometimes these nodes are actually used, for instance when a temp node
750    is expected at the head of a list. The layout is:
751
752    \starttyping
753    [ type+subtype + next      ]
754    [ attr         + prev      ]
755    [ penalty      + nobreak   ]
756    [ prebreak     + postbreak ]
757    [ type+subtype next/hlink  ] (nesting node prebreak)
758    [ tlink        prev        ]
759    [ type+subtype next/hlink  ] (nesting node postbreak)
760    [ tlink        prev        ]
761    [ type+subtype next/hlink  ] (nesting node nobreak)
762    [ tlink        prev        ]
763    \stoptyping
764
765    Another reason why we need the indirect apoproach is that we can set the fields to |null| which
766    is better than point to a nest node with no following up.
767
768*/
769
770/*tex
771    Among the dropped nodes (\LUATEX\ has them) are movements nodes (used in the \DVI\ backend)
772    and variable nodes (replaced by specification nodes).
773
774    Nesting nodes are really simple and just use the common type, subtype and next fields so they
775    have no dedicated fields. They can be part of another node type (like disc nodes).
776*/
777
778# define nesting_node_size 2
779
780typedef enum nesting_subtypes {
781    pre_break_code,
782    post_break_code,
783    no_break_code,
784    insert_head_code,
785    unset_nesting_code,
786} nesting_subtypes;
787
788# define last_nesting_subtype unset_nesting_code
789
790/*tex Here the codes in commands and subtypes are in sync. */
791
792typedef enum discretionary_subtypes {
793    normal_discretionary_code,
794    explicit_discretionary_code,
795    automatic_discretionary_code,
796    mathematics_discretionary_code,
797    syllable_discretionary_code,
798} discretionary_subtypes;
799
800# define last_discretionary_subtype syllable_discretionary_code
801# define last_discretionary_code    automatic_discretionary_code
802
803typedef enum disc_option_codes {
804    disc_option_normal_word               = 0x00000000,
805    disc_option_pre_word                  = 0x00000001,
806    disc_option_post_word                 = 0x00000002,
807    disc_option_orphaned                  = 0x00000004,
808    disc_option_prefer_break              = 0x00000010,
809    disc_option_prefer_nobreak            = 0x00000020,
810    /* */
811    disc_option_no_italic_correction      = 0x00000040,
812    disc_option_no_zero_italic_correction = 0x00000080,
813    /*tex We permit user options. */
814    disc_option_user_first                = 0x00010000,
815    disc_option_user_last                 = 0x40000000,
816    /*tex So watch out: these are subsets! */
817    disc_option_valid                     = 0x4FFF0FFF,
818} disc_option_codes;
819
820# define first_disc_option disc_option_prefer_break
821# define last_disc_option  disc_option_prefer_nobreak
822
823# define disc_node_size       13
824# define disc_no_break(a)     vlink(a,2) /* beware: vinfo is used for type/subtype */
825# define disc_pre_break(a)    vlink(a,3) /* beware: vinfo is used for type/subtype */
826# define disc_post_break(a)   vlink(a,4) /* beware: vinfo is used for type/subtype */
827/*       disc_no_break_node   5  6 */    /* this is a nesting node of size 2 */
828/*       disc_pre_break_node  7  8 */    /* this is a nesting node of size 2 */
829/*       disc_post_break_node 9 10 */    /* this is a nesting node of size 2 */
830# define disc_penalty(a)      vinfo(a,11)
831# define disc_options(a)      vlink(a,11)
832# define disc_class(a)        vinfo(a,12)
833# define disc_orphaned(a)     vlink(a,12)
834
835# define set_disc_penalty(a,b) disc_penalty(a) = b
836# define set_disc_class(a,b)   disc_class(a) = b
837# define set_disc_options(a,b) disc_options(a) = b
838# define set_disc_option(a,b)  disc_options(a) |= b
839
840# define has_disc_option(a,b) ((disc_options(a) & b) == b)
841
842# define unset_disc_class -1
843
844static inline void tex_add_disc_option    (halfword a, halfword r) { disc_options(a) |= r; }
845static inline void tex_remove_disc_option (halfword a, halfword r) { disc_options(a) &= ~r; }
846static inline int  tex_has_disc_option    (halfword a, halfword r) { return (disc_options(a) & r) == r; }
847
848/*tex
849    These are pseudo nodes inside a node. We used to reference them by |*_break_head| but now call
850    just call them nodes so that we can use head and tail instead of hlink and tlink.
851*/
852
853# define disc_pre_break_node(a)  (a+5)
854# define disc_post_break_node(a) (a+7)
855# define disc_no_break_node(a)   (a+9)
856
857# define disc_pre_break_head(a)  node_head(disc_pre_break_node(a))
858# define disc_post_break_head(a) node_head(disc_post_break_node(a))
859# define disc_no_break_head(a)   node_head(disc_no_break_node(a))
860
861# define disc_pre_break_tail(a)  node_tail(disc_pre_break_node(a))
862# define disc_post_break_tail(a) node_tail(disc_post_break_node(a))
863# define disc_no_break_tail(a)   node_tail(disc_no_break_node(a))
864
865extern void     tex_set_disc_field          (halfword target, halfword location, halfword source);
866extern void     tex_check_disc_field        (halfword target);
867extern void     tex_set_discpart            (halfword d, halfword h, halfword t, halfword code);
868extern halfword tex_flatten_discretionaries (halfword head, int *count, int nest);
869extern void     tex_soften_hyphens          (halfword head, int *found, int *replaced);
870
871typedef enum uleader_locations {
872    uleader_filtered_hpack,
873    uleader_lua,
874    uleader_before_vpack,
875    uleader_after_vpack,
876    uleader_post_linebreak,
877} uleader_locations;
878
879extern int      tex_flatten_leaders         (halfword box, int grp, int just_pack, int location, int checkline);
880extern halfword tex_harden_spaces           (halfword head, halfword tolerance, int *count);
881
882/*tex
883    Lists need a rather large node, also because the have quite some extra possibilities, like the
884    orientation features. We can put the dir with the orientation but it becomes messy in casting
885    that way. Also, memory is not really a constraint and for a cpu cache we're better off this
886    way.
887
888    In the original setup the unset and align_record nodes have overlapping fields. This has the
889    side effect that when we access the alternates from \LUA\ that they can have weird values
890    unless we reset them. Even then, it can be that we actually want to use those other fields
891    somehow. For that reason it's better to waste a few more slots and play safe. We can now
892    actually explore table cells with offsets if we want.
893
894    Beware: in alignments
895
896    \startitemize[packed]
897    \startitem align record nodes become unset nodes \stopitem
898    \startitem unset nodes become hlist or vlist nodes \stopitem
899    \stopitemize
900*/
901
902/* we could support usertypes: > 255  */
903
904typedef enum list_subtypes {
905    unknown_list,
906    line_list,                 /*tex paragraph lines */
907    hbox_list,                 /*tex |\hbox| */
908    indent_list,               /*tex indentation box */
909    container_list,            /*tex container box */
910    align_row_list,            /*tex row from a |\halign| or |\valign| */
911    align_cell_list,           /*tex cell from a |\halign| or |\valign| */
912    equation_list,             /*tex display equation */
913    equation_number_list,      /*tex display equation number */
914    math_list_list,
915    math_char_list,
916    math_pack_list,
917    math_h_extensible_list,
918    math_v_extensible_list,
919    math_h_delimiter_list,
920    math_v_delimiter_list,
921    math_over_delimiter_list,
922    math_under_delimiter_list,
923    math_numerator_list,
924    math_denominator_list,
925    math_modifier_list,
926    math_fraction_list,
927    math_nucleus_list,
928    math_sup_list,
929    math_sub_list,
930    math_prime_list,
931    math_pre_post_sup_list,
932    math_pre_post_sub_list,
933    math_degree_list,
934    math_scripts_list,
935    math_over_list,
936    math_under_list,
937    math_accent_list,
938    math_radical_list,
939    math_fence_list,
940    math_rule_list,
941    math_ghost_list,
942    math_text_list,
943    insert_result_list,
944    local_list,
945    local_left_list,
946    local_right_list,
947    local_middle_list,
948    balance_slot_list,
949    balance_list,
950    spacing_list,
951} list_subtypes;
952
953# define last_list_subtype    spacing_list
954# define noad_class_list_base 0x0100
955
956typedef enum list_anchors {
957    left_origin_anchor    = 0x001,
958    left_height_anchor    = 0x002,
959    left_depth_anchor     = 0x003,
960    right_origin_anchor   = 0x004,
961    right_height_anchor   = 0x005,
962    right_depth_anchor    = 0x006,
963    center_origin_anchor  = 0x007,
964    center_height_anchor  = 0x008,
965    center_depth_anchor   = 0x009,
966    halfway_total_anchor  = 0x00A,
967    halfway_height_anchor = 0x00B,
968    halfway_depth_anchor  = 0x00C,
969    halfway_left_anchor   = 0x00D,
970    halfway_right_anchor  = 0x00E,
971} list_anchors;
972
973typedef enum list_signs {
974    negate_x_anchor = 0x100,
975    negate_y_anchor = 0x200,
976} list_signs;
977
978typedef enum list_geometries {
979    no_geometry          = 0x0,
980    offset_geometry      = 0x1,
981    orientation_geometry = 0x2,
982    anchor_geometry      = 0x4,
983} list_geometries;
984
985typedef enum list_balance_states { 
986    balance_state_inserts  = 0x01,
987    balance_state_discards = 0x02,
988    balance_state_uinserts = 0x04,
989} list_balance_states;
990
991# define box_node_size        17
992# define box_width(a)         vlink(a,2)
993# define box_w_offset(a)      vinfo(a,2)
994# define box_depth(a)         vlink(a,3)
995# define box_d_offset(a)      vinfo(a,3)
996# define box_height(a)        vlink(a,4)
997# define box_h_offset(a)      vinfo(a,4)
998# define box_list(a)          vlink(a,5)   /* 5 = list_offset */
999# define box_shift_amount(a)  vinfo(a,5)
1000# define box_glue_order(a)    vlink(a,6)
1001# define box_glue_sign(a)     vinfo01(a,6)
1002# define box_balance_state(a) vinfo02(a,6)
1003# define box_reserved_1(a)    vinfo03(a,6) /* can be used */
1004# define box_reserved_2(a)    vinfo04(a,6) /* can be used */
1005# define box_glue_set(a)      dvalue(a,7)  /* So we reserve a whole memory word! */
1006# define box_dir(a)           vlink00(a,8) /* We could encode it as geometry but not now. */
1007# define box_package_state(a) vlink01(a,8)
1008# define box_options(a)       vlink02(a,8)
1009# define box_geometry(a)      vlink03(a,8)
1010# define box_orientation(a)   vinfo(a,8)   /* Also used for size in alignments. */
1011# define box_x_offset(a)      vlink(a,9)
1012# define box_y_offset(a)      vinfo(a,9)
1013# define box_pre_migrated(a)  vlink(a,10)
1014# define box_post_migrated(a) vinfo(a,10)
1015# define box_pre_adjusted(a)  vlink(a,11)
1016# define box_post_adjusted(a) vinfo(a,11)
1017# define box_source_anchor(a) vlink(a,12)
1018# define box_target_anchor(a) vinfo(a,12)
1019# define box_anchor(a)        vlink(a,13)
1020# define box_index(a)         vinfo(a,13)
1021# define box_except(a)        vlink(a,14)
1022# define box_exdepth(a)       vinfo(a,14)
1023# define box_discardable(a)   vlink(a,15)  /* internal usage */
1024# define box_reserved(a)      vinfo(a,15)  /* can be used */
1025# define box_input_file(a)    vlink(a,16)  /* can be used */
1026# define box_input_line(a)    vinfo(a,16)
1027
1028# define box_total(a) (box_height(a) + box_depth(a)) /* Here we add, with glyphs we maximize. */
1029
1030static inline void tex_set_box_geometry   (halfword b, halfword g) { box_geometry(b) |= (singleword) g; }
1031static inline void tex_unset_box_geometry (halfword b, halfword g) { box_geometry(b) &= (singleword) ~g; }
1032static inline int  tex_has_geometry       (halfword g, halfword f) { return ((singleword) g & (singleword) f) == (singleword) f; }
1033static inline int  tex_has_box_geometry   (halfword b, halfword g) { return (box_geometry(b) & (singleword) g) == (singleword) g; }
1034
1035/*tex These are used in reporting states in Lua. */
1036
1037typedef enum box_pre_post_states {
1038    has_pre_adjust    = 0x1,
1039    has_post_adjust   = 0x2,
1040    has_pre_migrated  = 0x4,
1041    has_post_migrated = 0x8,
1042} box_pre_post_states;
1043
1044typedef enum box_insert_states {
1045    has_inserts              = 0x1,
1046    has_inserts_with_content = 0x2,
1047    has_inserts_with_height  = 0x4,
1048} box_insert_states;
1049
1050/*tex These three make the bitset in |box_package_state|. */
1051
1052typedef enum package_states {
1053    unknown_package_state = 0x00,
1054    hbox_package_state    = 0x01,
1055    vbox_package_state    = 0x02,
1056    vtop_package_state    = 0x03, /* == 0x01 + 0x02 */
1057    dbox_package_state    = 0x04,
1058    /* maybe vcenter */
1059} package_states;
1060
1061typedef enum package_dimension_states {
1062    package_dimension_not_set  = 0x00,
1063    package_dimension_size_set = 0x10, /* used in in alignments */
1064} package_dimension_states;
1065
1066typedef enum package_leader_states {
1067    package_u_leader_not_set  = 0x00,
1068    package_u_leader_set      = 0x20,
1069    package_u_leader_delayed  = 0x40,
1070    package_u_leader_found    = 0x80,
1071} package_leader_states;
1072
1073# define set_box_package_state(p,s) box_package_state(p) |= s
1074# define has_box_package_state(p,s) ((box_package_state(p) & s) == s)
1075# define is_box_package_state(p,s)  ((p & s) == s)
1076
1077typedef enum box_option_flags { 
1078    box_option_no_math_axis = 0x01,
1079    box_option_discardable  = 0x02,
1080 // box_option_synchronize  = 0x08,
1081} box_option_flags;
1082
1083static inline void tex_add_box_option    (halfword a, halfword r) { box_options(a) |= r; }
1084static inline void tex_remove_box_option (halfword a, halfword r) { box_options(a) &= ~r; }
1085static inline int  tex_has_box_option    (halfword a, halfword r) { return (box_options(a) & r) == r; }
1086
1087/*tex
1088    These |unset| nodes have the same layout as list nodes and at some point become an |hlist| or
1089    |vlist| node.
1090*/
1091
1092# define unset_node_size     box_node_size
1093# define box_glue_stretch(a) box_w_offset(a)
1094# define box_glue_shrink(a)  box_h_offset(a)
1095# define box_span_count(a)   box_d_offset(a)
1096# define box_size(a)         box_orientation(a)
1097
1098/*tex
1099    The |align record| nodes have the same layout as list nodes and at some point become an |unset|
1100    node.
1101*/
1102
1103# define align_record_size          box_node_size
1104# define align_record_span_ptr(a)   box_w_offset(a)    /*tex A column spanning list */
1105# define align_record_cmd(a)        box_h_offset(a)    /*tex Info to remember during template. */
1106# define align_record_chr(a)        box_d_offset(a)    /*tex Info to remember during template. */
1107# define align_record_pre_part(a)   box_x_offset(a)    /*tex The pointer to |u_j| token list. */
1108# define align_record_post_part(a)  box_y_offset(a)    /*tex The pointer to |v_j| token list. */
1109# define align_record_dimension(a)  box_orientation(a) /*tex Optionally enforced width. */
1110# define align_record_pre_local(a)  box_pre_migrated(a)
1111# define align_record_post_local(a) box_post_migrated(a)
1112
1113/*tex
1114   Span nodes are tricky in the sense that their |span_link| actually has to sit in the same slot
1115   as |align_record_span_ptr| because we need the initial location to be the same. This is why we
1116   renamed this field to |span_ptr|. Moving it to another spot than in \LUATEX\ also opens the
1117   possibility for attributes to cells.
1118*/
1119
1120# define span_node_size 3
1121# define span_span(a)   vinfo(a,1)
1122# define span_unused(a) vlink(a,1)
1123# define span_width(a)  vlink(a,2)  /* overlaps with |box_width(a)|. */
1124# define span_ptr(a)    vinfo(a,2)  /* overlaps with |box_w_offset(a)| and align_record_span_ptr(a). */
1125
1126/*tex
1127    Here the subtypes and command codes partly overlay. We actually hav eonly avery few left because
1128    it's mostly a backend feature now.
1129*/
1130
1131typedef enum rule_subtypes {
1132    normal_rule_subtype,
1133    empty_rule_subtype,
1134    strut_rule_subtype,
1135    virtual_rule_subtype,
1136    /* */
1137    outline_rule_subtype,
1138    user_rule_subtype,
1139    math_over_rule_subtype,
1140    math_under_rule_subtype,
1141    math_fraction_rule_subtype,
1142    math_radical_rule_subtype,
1143    box_rule_subtype,
1144    image_rule_subtype,
1145    spacing_rule_subtype,
1146} rule_subtypes;
1147
1148typedef enum rule_codes {
1149    normal_rule_code,
1150    empty_rule_code,
1151    strut_rule_code,
1152    virtual_rule_code,
1153} rule_codes;
1154
1155typedef enum rule_option_codes {
1156    rule_option_horizontal  = 0x01, /* maybe we want both at some time */
1157    rule_option_vertical    = 0x02, /* when none is set we have a math rule */
1158    rule_option_thickness   = 0x04, /* future */
1159    rule_option_running     = 0x08,
1160    rule_option_discardable = 0x10,
1161    rule_option_valid       = 0x1F,
1162} rule_option_codes;
1163
1164# define last_rule_subtype spacing_rule_subtype
1165# define first_rule_code   normal_rule_code
1166# define last_rule_code    strut_rule_code
1167
1168# define rule_node_size      9
1169# define rule_width(a)       vlink(a,2)
1170# define rule_x_offset(a)    vinfo(a,2)
1171# define rule_depth(a)       vlink(a,3)
1172# define rule_y_offset(a)    vinfo(a,3)
1173# define rule_height(a)      vlink(a,4)
1174# define rule_data(a)        vinfo(a,4)   /* used for linewidth */
1175# define rule_options(a)     vlink(a,5)
1176# define rule_thickness(a)   vinfo(a,5)   /* future see data */
1177# define rule_left(a)        vinfo(a,6)   /* depends on subtype */
1178# define rule_right(a)       vlink(a,6)   /* depends on subtype */
1179# define rule_extra_1(a)     vinfo(a,7)   /* depends on subtype */
1180# define rule_extra_2(a)     vlink(a,7)   /* depends on subtype */
1181# define rule_discardable(a) vinfo(a,8)   /* internal usage */
1182# define rule_reserved(a)    vlink(a,8) 
1183
1184# define rule_line_on         rule_extra_1    /* for user rules */
1185# define rule_line_off        rule_extra_2    /* for user rules */
1186
1187# define rule_strut_font      rule_extra_1    /* for strut rules */
1188# define rule_strut_character rule_extra_2    /* for strut rules */
1189
1190# define rule_virtual_width   rule_left
1191# define rule_virtual_height  rule_right
1192# define rule_virtual_depth   rule_extra_1
1193# define rule_virtual_unused  rule_extra_2
1194
1195# define rule_total(a) (rule_height(a) + rule_depth(a))
1196
1197# define set_rule_options(a,b)  rule_options(a) |= b
1198
1199/*tex
1200
1201    Originally glyph nodes had a |lig_ptr| but storing components makes not that much sense so we
1202    dropped that. The free slot is now used for a state field. We already had a data field that
1203    took another free slot and that behaves like an attribute. The glyph data field can be set at
1204    the \TEX\ end, the state field is only accessible in \LUA. At the same time we reshuffled the
1205    fields a bit so that the most accessed fields are close together.
1206
1207    The \LUATEX\ engine dropped the language node and moved that feature to the glyph nodes. In
1208    addition to the language more properties could be set but they were all packed into one
1209    halfword. In \LUAMETATEX\ we waste a few more bytes and keep the language separate but we
1210    still pack a few properties.
1211
1212    In \TEX\ we have character nodes and glyph nodes, but here we only have one type. The subtype
1213    can be used to indicate if we have ligatures but in \LUATEX\ for various reasons we don't follow
1214    the integrated approach that \TEX\ has: we have callbacks for hyphenation, ligature building,
1215    kerning etc.\ which demands separation, but more important is that we want to use \LUA\ to deal
1216    with modern fonts. The components field that is still present in \LUATEX\ is gone because it
1217    serves no purpose. We don't need to reassemble and when dealing with \OPENTYPE\ fonts we loose
1218    information in successive steps anyway.
1219
1220    This also makes that the subtype is now only used to flag if glyphs have been processed. The
1221    macro package can decide what additional properties get stored in this field.
1222
1223    We used to have this:
1224
1225    \starttyping
1226    static inline void protect_glyph      (halfword a) { quarterword s = node_subtype(a) ; if (s <= 256) { node_subtype(a) = s == 1 ? 256 : 256 + s; } }
1227    static inline void unprotect_glyph    (halfword a) { quarterword s = node_subtype(a) ; if (s >  256) { node_subtype(a) = s - 256; } }
1228    static inline int  is_protected_glyph (halfword a) { return node_subtype(a) >= 256; }
1229    \stoptyping
1230
1231    These were also dropped:
1232
1233    \starttyping
1234    # define is_character(p)        (((node_subtype(p)) & glyph_character) == glyph_character)
1235    # define is_ligature(p)         (((node_subtype(p)) & glyph_ligature ) == glyph_ligature )
1236    # define is_simple_character(p) (is_character(p) && ! is_ligature(p))
1237    # define set_is_glyph(p)         node_subtype(p) = (quarterword) (node_subtype(p) & ~glyph_character)
1238    \stoptyping
1239
1240*/
1241
1242/*tex
1243
1244    Putting |width|, |height| and |depth| in a glyph has some advantages, for instance when we
1245    fetch them in the builder, packer, \LUA\ interface, but it also has a disadvantage: we need to
1246    have more complex copying of glyph nodes. For instance, when we copy glyphs in the open type
1247    handler (e.g. for multiples) we also copy the fields. But then when we set a character, we also
1248    would have to set the dimensions. Okay, some helper could do that (or a flag in setchar). It's
1249    anyway not something to do in a hurry. An |x_extra| field is something different: combined with
1250    setting |x_offset| that could replace font kerns: |x_advance = width + x_offset + x_extra|.
1251
1252*/
1253
1254# define glyph_node_size     14
1255# define glyph_character(a)  vinfo(a,2)
1256# define glyph_font(a)       vlink(a,2)   /*tex can be quarterword */
1257# define glyph_data(a)       vinfo(a,3)   /*tex handy in context */
1258# define glyph_state(a)      vlink(a,3)   /*tex handy in context */
1259# define glyph_language(a)   vinfo0(a,4)
1260# define glyph_script(a)     vinfo1(a,4)
1261# define glyph_control(a)    vlink0(a,4)  /*tex we store 0xXXXX in the |\cccode| */
1262# define glyph_disccode(a)   vlink1(a,4)  /*tex can be smaller */
1263//define glyph_disccode(a)   vlink02(a,4) 
1264//define glyph_reserved_1(a) vlink03(a,4) 
1265# define glyph_options(a)    vinfo(a,5)
1266# define glyph_hyphenate(a)  vlink(a,5)
1267# define glyph_protected(a)  vinfo00(a,6)
1268# define glyph_lhmin(a)      vinfo01(a,6)
1269# define glyph_rhmin(a)      vinfo02(a,6)
1270# define glyph_discpart(a)   vinfo03(a,6) 
1271# define glyph_expansion(a)  vlink(a,6)
1272# define glyph_x_scale(a)    vinfo(a,7)
1273# define glyph_y_scale(a)    vlink(a,7)
1274# define glyph_scale(a)      vinfo(a,8)
1275# define glyph_raise(a)      vlink(a,8)
1276# define glyph_left(a)       vinfo(a,9)
1277# define glyph_right(a)      vlink(a,9)
1278# define glyph_x_offset(a)   vinfo(a,10)
1279# define glyph_y_offset(a)   vlink(a,10)
1280# define glyph_weight(a)     vinfo(a,11)
1281# define glyph_slant(a)      vlink(a,11)
1282# define glyph_properties(a) vinfo0(a,12)  /*tex for math */
1283# define glyph_group(a)      vinfo1(a,12)  /*tex for math */
1284# define glyph_index(a)      vlink(a,12)   /*tex for math */
1285# define glyph_input_file(a) vinfo(a,13)
1286# define glyph_input_line(a) vlink(a,13)
1287
1288# define get_glyph_data(a)      ((halfword) glyph_data(a))
1289# define get_glyph_state(a)     ((halfword) glyph_state(a))
1290# define get_glyph_language(a)  ((quarterword) glyph_language(a))
1291# define get_glyph_script(a)    ((quarterword) glyph_script(a))
1292# define get_glyph_control(a)   ((quarterword) glyph_control(a))
1293# define get_glyph_disccode(a)  ((quarterword) glyph_disccode(a))
1294# define get_glyph_x_scale(a)   ((halfword) glyph_x_scale(a))
1295# define get_glyph_y_scale(a)   ((halfword) glyph_y_scale(a))
1296# define get_glyph_scale(a)     ((halfword) glyph_scale(a))
1297# define get_glyph_raise(a)     ((halfword) glyph_raise(a))
1298# define get_glyph_lhmin(a)     ((halfword) glyph_lhmin(a))
1299# define get_glyph_rhmin(a)     ((halfword) glyph_rhmin(a))
1300# define get_glyph_left(a)      ((halfword) glyph_left(a))
1301# define get_glyph_right(a)     ((halfword) glyph_right(a))
1302# define get_glyph_hyphenate(a) ((halfword) glyph_hyphenate(a))
1303# define get_glyph_options(a)   ((halfword) glyph_options(a))
1304# define get_glyph_discpart(a)  ((halfword)   (glyph_discpart(a)       & 0xF))
1305# define get_glyph_discafter(a) ((halfword) ( (glyph_discpart(a) >> 4) & 0xF))
1306
1307# define set_glyph_data(a,b)      glyph_data(a) = b
1308# define set_glyph_state(a,b)     glyph_state(a) = b
1309# define set_glyph_language(a,b)  glyph_language(a) = (quarterword) b
1310# define set_glyph_script(a,b)    glyph_script(a) = (quarterword) b
1311# define set_glyph_control(a,b)   glyph_control(a) = (quarterword) b
1312# define set_glyph_disccode(a,b)  glyph_disccode(a) = (quarterword) b
1313# define set_glyph_x_scale(a,b)   glyph_x_scale(a) = b
1314# define set_glyph_y_scale(a,b)   glyph_y_scale(a) = b
1315# define set_glyph_x_offset(a,b)  glyph_x_offset(a) = b
1316# define set_glyph_y_offset(a,b)  glyph_y_offset(a) = b
1317# define set_glyph_slant(a,b)     glyph_slant(a) = b
1318# define set_glyph_weight(a,b)    glyph_weight(a) = b
1319# define set_glyph_scale(a,b)     glyph_scale(a) = b
1320# define set_glyph_raise(a,b)     glyph_raise(a) = b
1321# define set_glyph_left(a,b)      glyph_left(a) = b
1322# define set_glyph_right(a,b)     glyph_right(a) = b
1323# define set_glyph_lhmin(a,b)     glyph_lhmin(a) = ((singleword) (b))
1324# define set_glyph_rhmin(a,b)     glyph_rhmin(a) = ((singleword) (b))
1325# define set_glyph_hyphenate(a,b) glyph_hyphenate(a) = ((halfword) (b))
1326# define set_glyph_options(a,b)   glyph_options(a) = ((halfword) (b))
1327# define set_glyph_discpart(a,b)  glyph_discpart(a) = (glyph_discpart(a) | (singleword)  ((b) & 0xF)      )
1328# define set_glyph_discafter(a,b) glyph_discpart(a) = (glyph_discpart(a) | (singleword) (((b) & 0xF) << 4))
1329
1330# define get_glyph_dohyph(a) (hyphenation_permitted(glyph_hyphenate(a), syllable_hyphenation_mode ) || hyphenation_permitted(glyph_hyphenate(a), force_handler_hyphenation_mode))
1331# define get_glyph_uchyph(a) (hyphenation_permitted(glyph_hyphenate(a), uppercase_hyphenation_mode) || hyphenation_permitted(glyph_hyphenate(a), force_handler_hyphenation_mode))
1332
1333/*       set_glyph_dohyph(a,b) glyph_hyphenate(a) = ((halfword) flip_hyphenation_mode(glyph_hyphenate(a),syllable_hyphenation_mode)) */
1334# define set_glyph_uchyph(a,b) glyph_hyphenate(a) = ((halfword) flip_hyphenation_mode(glyph_hyphenate(a),uppercase_hyphenation_mode))
1335
1336typedef enum glyph_subtypes {
1337    /* initial value: */
1338    glyph_unset_subtype,
1339    /* traditional text: */
1340    glyph_character_subtype,
1341    glyph_ligature_subtype,
1342    /* special math */
1343    glyph_math_delimiter_subtype,
1344    glyph_math_extensible_subtype,
1345    /* engine math, class driven */
1346    glyph_math_ordinary_subtype,
1347    glyph_math_operator_subtype,
1348    glyph_math_binary_subtype,
1349    glyph_math_relation_subtype,
1350    glyph_math_open_subtype,
1351    glyph_math_close_subtype,
1352    glyph_math_punctuation_subtype,
1353    glyph_math_variable_subtype,
1354    glyph_math_active_subtype,
1355    glyph_math_inner_subtype,
1356    glyph_math_under_subtype,
1357    glyph_math_over_subtype,
1358    glyph_math_fraction_subtype,
1359    glyph_math_radical_subtype,
1360    glyph_math_middle_subtype,
1361    glyph_math_prime_subtype,
1362    glyph_math_accent_subtype,
1363    glyph_math_fenced_subtype,
1364    glyph_math_ghost_subtype,
1365    /* bogus subtype */
1366    glyph_math_vcenter_subtype,
1367    /* extra math, user classes, set but anonymous */
1368    glyph_math_extra_subtype = 31,
1369} glyph_subtypes;
1370
1371# define glyph_node_is_text(g) (node_subtype(g) <= glyph_ligature_subtype)
1372
1373# define last_glyph_subtype glyph_math_accent_subtype
1374
1375/* todo: make this a bitset so that we can also register breakpoints */
1376
1377typedef enum glyph_discpart_codes {
1378    glyph_discpart_unset   = 0x0,
1379    glyph_discpart_pre     = 0x1,
1380    glyph_discpart_post    = 0x2,
1381    glyph_discpart_replace = 0x3,
1382    glyph_discpart_always  = 0x4,
1383} glyph_discpart_codes;
1384
1385typedef enum glyph_disc_codes {
1386    glyph_disc_normal      = 0x1, /* disc subtype + 1 */
1387    glyph_disc_explicit    = 0x2,
1388    glyph_disc_automatic   = 0x3,
1389    glyph_disc_mathematics = 0x4,
1390    glyph_disc_syllable    = 0x5,
1391} glyph_disc_codes;
1392
1393typedef enum glyph_option_codes {
1394    /*tex These are part of the defaults (all): */
1395    glyph_option_normal_glyph              = 0x00000000,
1396    glyph_option_no_left_ligature          = 0x00000001,
1397    glyph_option_no_right_ligature         = 0x00000002,
1398    glyph_option_no_left_kern              = 0x00000004,
1399    glyph_option_no_right_kern             = 0x00000008,
1400    glyph_option_no_expansion              = 0x00000010,
1401    glyph_option_no_protrusion             = 0x00000020,
1402    glyph_option_no_italic_correction      = 0x00000040,
1403    glyph_option_no_zero_italic_correction = 0x00000080,
1404    /* */
1405    glyph_option_apply_x_offset            = 0x00000100,
1406    glyph_option_apply_y_offset            = 0x00000200,
1407    /* These are only meant for math characters: */
1408    glyph_option_math_discretionary        = 0x00000400,
1409    glyph_option_math_italics_too          = 0x00000800,
1410    glyph_option_math_artifact             = 0x00001000,
1411    /* */
1412    glyph_option_weight_less               = 0x00002000,
1413    glyph_option_space_factor_overload     = 0x00004000,
1414    /* */
1415    glyph_option_check_toddler             = 0x00008000,
1416    glyph_option_check_twin                = 0x00010000,
1417    glyph_option_is_toddler                = 0x00020000,
1418    /* */
1419    glyph_option_is_continuation           = 0x00040000,
1420    /*tex We permit user options. */
1421    glyph_option_user_first                = 0x00100000,
1422    glyph_option_user_last                 = 0x40000000,
1423    /*tex These ranges can change! */
1424    glyph_option_system                    = 0x0001FFFF,
1425    glyph_option_user                      = 0x4FF00000,
1426    /*tex Keep this in sync with the above! */
1427    glyph_option_valid                     = glyph_option_system
1428                                           | glyph_option_user,
1429} glyph_option_codes;
1430
1431typedef enum auto_discretionary_codes {
1432    auto_discretionary_normal = 0x0001, /* turn glyphs into discretionary with three similar components */
1433    auto_discretionary_italic = 0x0002, /* also include italic correction when present */
1434} auto_discretionary_codes;
1435
1436static inline void tex_add_glyph_option    (halfword a, halfword r) { glyph_options(a) |= r; }
1437static inline void tex_remove_glyph_option (halfword a, halfword r) { glyph_options(a) &= ~r; }
1438static inline int  tex_has_glyph_option    (halfword a, halfword r) { return (glyph_options(a) & r) == r; }
1439
1440/*tex
1441    As we have a small field available for protection we no longer need to pack the protection
1442    state in the subtype. We can now basically use the subtype for anything we want (as long as it
1443    stays within the range |0x0000-0xFFFF|.
1444*/
1445
1446/* static inline void tex_protect_glyph      (halfword a) {        node_subtype(a) |= (quarterword) 0x8000; } */
1447/* static inline void tex_unprotect_glyph    (halfword a) {        node_subtype(a) &= (quarterword) 0x7FFF; } */
1448/* static inline int  tex_is_protected_glyph (halfword a) { return node_subtype(a) >= (quarterword) 0x8000; } */
1449/* static inline int  tex_subtype_of_glyph   (halfword a) { return node_subtype(a) &  (quarterword) 0x7FFF; } */
1450
1451typedef enum glyph_protection_codes {
1452    glyph_unprotected_code    = 0x0,
1453    glyph_protected_text_code = 0x1,
1454    glyph_protected_math_code = 0x2,
1455} glyph_protection_codes;
1456
1457/*tex
1458    Next come some very specialized nodes types. First the marks. They just register a token list.
1459*/
1460
1461# define mark_node_size 3
1462# define mark_ptr(a)    vlink(a,2)
1463# define mark_index(a)  vinfo(a,2)
1464
1465typedef enum mark_codes {
1466    set_mark_value_code,
1467    reset_mark_value_code,
1468} mark_codes;
1469
1470# define last_mark_subtype reset_mark_value_code
1471
1472/*tex
1473    The (not really used in \CONTEXT) |\vadjust| nodes are also small. The codes and subtypes
1474    overlap.
1475*/
1476
1477typedef enum adjust_subtypes {
1478    pre_adjust_code,
1479    post_adjust_code,
1480    local_adjust_code,
1481} adjust_subtypes;
1482
1483typedef enum adjust_options {
1484    adjust_option_none         = 0x00,
1485    adjust_option_before       = 0x01,
1486    adjust_option_baseline     = 0x02,
1487    adjust_option_depth_before = 0x04,
1488    adjust_option_depth_after  = 0x08,
1489    adjust_option_depth_check  = 0x10,
1490    adjust_option_depth_last   = 0x20,
1491    adjust_option_except       = 0x40,
1492} adjust_options;
1493
1494# define last_adjust_subtype local_adjust_code
1495
1496# define adjust_node_size       5
1497# define adjust_list(a)         vlink(a,2)
1498# define adjust_options(a)      vinfo(a,2)
1499# define adjust_index(a)        vlink(a,3)
1500# define adjust_except(a)       vinfo(a,3)
1501# define adjust_depth_before(a) vlink(a,4)
1502# define adjust_depth_after(a)  vinfo(a,4)
1503
1504# define has_adjust_option(p,o) ((adjust_options(p) & o) == o)
1505
1506/*tex
1507    Inserts are more complicated. The |ins| node stores an insert in the list while |inserting|
1508    nodes keep track of where to break the page so that they (hopefully) stay with the text. As
1509    already mentioned, the insert node is tricky in the sense that it uses an offset to an
1510    embedded (fake) node. That node acts as start of a next chain. Making that more transparent
1511    would demand some changes that I'm not willing to make right now (and maybe never).
1512*/
1513
1514typedef enum insert_options {
1515    insert_option_in_insert = 0x01,
1516} insert_options;
1517
1518# define insert_node_size       9          /*tex Can become 1 smaller or we can have insert_index instead of subtype. */
1519# define insert_index(a)        vinfo(a,2) /*tex The |width| is not used. */
1520# define insert_float_cost(a)   vlink(a,2)
1521# define insert_identifier(a)   vinfo(a,3) /*tex aka: |insert_data| but that macro messes up a field */
1522# define insert_max_depth(a)    vlink(a,3)
1523# define insert_options(a)      vinfo(a,4) /* */
1524# define insert_total_height(a) vlink(a,4) /*tex The sum of height and depth, i.e. total. */
1525# define insert_list(a)         vinfo(a,5) /*tex Is alias for |node_next|. */
1526# define insert_split_top(a)    vlink(a,5)
1527# define insert_line_height(a)  vinfo(a,6)
1528# define insert_line_depth(a)   vlink(a,6)
1529# define insert_stretch(a)      vinfo(a,7)
1530# define insert_shrink(a)       vlink(a,7)
1531# define insert_callback(a)     vinfo(a,8)
1532# define insert_belongs_to(a)   vlink(a,8)
1533
1534# define insert_first_box(a)    (a + 5)    /*tex A fake node where box_list_ptr becomes a next field. */
1535
1536# define split_node_size        5
1537# define split_insert_index(a)  vinfo(a,2) /*tex Same slot! */
1538# define split_broken(a)        vlink(a,2) /*tex An insertion for this class will break here if anywhere. */
1539# define split_broken_insert(a) vinfo(a,3) /*tex This insertion might break at |broken_ptr|. */
1540# define split_last_insert(a)   vlink(a,3) /*tex The most recent insertion for this |subtype|. */
1541# define split_best_insert(a)   vinfo(a,4) /*tex The optimum most recent insertion. */
1542# define split_height(a)        vlink(a,4) /*tex Aka |height(a) = vlink(a,4)| */ /* todo */
1543
1544typedef enum split_subtypes {
1545    normal_split_subtype,
1546    insert_split_subtype,
1547} split_subtypes;
1548
1549# define last_split_subtype insert_split_subtype
1550
1551/*tex
1552    It's now time for some Some handy shortcuts. These are used when determining proper break points
1553    and|/|or the beginning or end of words.
1554*/
1555
1556# define last_preceding_break_node whatsit_node
1557# define last_non_discardable_node dir_node
1558# define last_node_with_attributes glyph_node
1559# define last_complex_node         align_record_node
1560# define max_node_type             passive_node
1561
1562# define precedes_break(a)  (node_type(a) <= last_preceding_break_node)
1563# define precedes_kern(a)   ((node_type(a) == kern_node) && (node_subtype(a) == font_kern_subtype || node_subtype(a) == accent_kern_subtype || node_subtype(a) == math_shape_kern_subtype))
1564# define precedes_dir(a)    ((node_type(a) == dir_node) && normalize_line_mode_option(break_after_dir_mode))
1565# define non_discardable(a) (node_type(a) <= last_non_discardable_node)
1566
1567static inline int tex_nodetype_is_complex     (halfword t) { return t <= last_complex_node; }
1568static inline int tex_nodetype_has_attributes (halfword t) { return t <= last_node_with_attributes; }
1569static inline int tex_nodetype_has_subtype    (halfword t) { return t != glue_spec_node && t != math_spec_node && t != font_spec_node; }
1570static inline int tex_nodetype_has_prev       (halfword t) { return t != glue_spec_node && t != math_spec_node && t != font_spec_node && t != attribute_node; }
1571static inline int tex_nodetype_has_next       (halfword t) { return t != glue_spec_node && t != math_spec_node && t != font_spec_node; }
1572static inline int tex_nodetype_is_valid       (halfword t) { return (t >= 0) && (t <= max_node_type); }
1573static inline int tex_nodetype_is_visible     (halfword t) { return (t >= 0) && (t <= max_node_type) && lmt_interface.node_data[t].visible; }
1574
1575/*tex
1576    This is a bit weird place to define them but anyway. In the meantime in \LUAMETATEX\ we no
1577    longer have the option to report the codes used in \ETEX. We have different nodes so it makes
1578    no sense to complicate matters (although earlier version of \LUAMETATEX\ has this organized
1579    quite well \unknown\ just an example of cleaning up, wondering about the use and then dropping
1580    it.
1581*/
1582
1583# define get_node_size(i) (lmt_interface.node_data[i].size)
1584# define get_node_name(i) (lmt_interface.node_data[i].name)
1585/*       get_etex_code(i) (lmt_interface.node_data[i].etex) */
1586
1587/*tex
1588    Although expressions could use some dedicated data structure, currently they are implemented
1589    using a linked list. This means that only memory is the limitation for recursion but I might
1590    as well go for a dedicated structure some day, just for the fun of implementing it. It is
1591    probably also more efficient. The current approach is inherited from \ETEX. The stack is only
1592    used when we have expressions between parenthesis.
1593*/
1594
1595/*tex Traditional \ETEX\ expression: */
1596
1597# define expression_node_size     3
1598# define expression_type(a)       vinfo00(a,1) /* overloads attr */ /*tex one of the value levels */
1599# define expression_state(a)      vinfo01(a,1) /* overloads attr */
1600# define expression_result(a)     vinfo02(a,1) /* overloads attr */
1601# define expression_reserved(a)   vinfo03(a,1) /* overloads attr */ /*tex not used */
1602# define expression_expression(a) vlink(a,1)   /* overloads prev */ /*tex saved expression so far */
1603# define expression_term(a)       vlink(a,2)   /*tex saved term so far */
1604# define expression_numerator(a)  vinfo(a,2)   /*tex saved numerator */
1605
1606/*tex Extended traditional expression: */
1607
1608# define lmtx_expression_node_size          6
1609# define lmtx_expression_type(a)            vinfo00(a,1) /* overloads attr */ /*tex one of the value levels, not used */
1610# define lmtx_expression_state(a)           vinfo01(a,1) /* overloads attr */
1611# define lmtx_expression_result(a)          vinfo02(a,1) /* overloads attr */
1612# define lmtx_expression_negate(a)          vinfo03(a,1) /* overloads attr */ /* bitset */
1613# define lmtx_expression_type_expression(a) vlink(a,1)   /* overloads prev */
1614# define lmtx_expression_type_term(a)       vinfo(a,2)
1615# define lmtx_expression_type_numerator(a)  vlink(a,2)
1616# define lmtx_expression_expression(a)      lvalue(a,3) 
1617# define lmtx_expression_term(a)            lvalue(a,4) 
1618# define lmtx_expression_numerator(a)       lvalue(a,5) 
1619
1620/*tex RPN stacked expression: */
1621
1622# define rpn_expression_node_size 3
1623# define rpn_expression_type(a)   vinfo00(a,1) /* overloads attr */ /*tex not used */
1624# define rpn_expression_state(a)  vinfo01(a,1) /* overloads attr */ /*tex not used */
1625# define rpn_expression_result(a) vinfo02(a,1) /* overloads attr */ /*tex not used */
1626# define rpn_expression_negate(a) vinfo03(a,1) /* overloads attr */ /*tex bitset   */
1627# define rpn_expression_entry(a)  lvalue(a,2) 
1628
1629/*tex
1630    Why not.
1631*/
1632
1633# define loop_state_node_size 2
1634# define loop_state_count(a)  vinfo(a,1) /* instead if node_attr */
1635
1636/*tex
1637    This is a node that stores a font state. In principle we can do without but for tracing it
1638    really helps to have this compound element because it is more compact. We could have gone
1639    numeric and use the sparse array approach but then we'd have to add a 4 int store which is more
1640    code and also makes save and restore more complex.
1641*/
1642
1643typedef enum fontspec_states {
1644    font_spec_scale_set   = 0x0001,
1645    font_spec_x_scale_set = 0x0002,
1646    font_spec_y_scale_set = 0x0004,
1647    font_spec_slant_set   = 0x0008,
1648    font_spec_weight_set  = 0x0010,
1649    font_spec_all_set     = 0x001F,
1650} font_spec_states;
1651
1652# define font_spec_node_size     5           /* we can be smaller: no attr and no prev */
1653# define font_spec_state(a)      vinfo(a,1)  /* slot of node_attr */
1654# define font_spec_identifier(a) vinfo(a,2)
1655# define font_spec_scale(a)      vlink(a,2)
1656# define font_spec_x_scale(a)    vinfo(a,3)
1657# define font_spec_y_scale(a)    vlink(a,3)
1658# define font_spec_slant(a)      vinfo(a,4)
1659# define font_spec_weight(a)     vlink(a,4)
1660
1661# define font_spec_property_is_set(a,b) ((font_spec_state(a) & b) == b)
1662
1663static inline int tex_same_fontspec(halfword a, halfword b)
1664{
1665    return
1666        (a == b)
1667     || (a && b && font_spec_state(a)      == font_spec_state(b)
1668                && font_spec_identifier(a) == font_spec_identifier(b)
1669                && font_spec_scale(a)      == font_spec_scale(b)
1670                && font_spec_x_scale(a)    == font_spec_x_scale(b)
1671                && font_spec_y_scale(a)    == font_spec_y_scale(b)
1672                && font_spec_slant(a)      == font_spec_slant(b)
1673                && font_spec_weight(a)     == font_spec_weight(b)
1674        )
1675    ;
1676}
1677
1678/*tex
1679    At the cost of some more memory we now use a mode for storage. This not only overcomes the
1680    \UNICODE\ limitation but also permits storing more in the future.
1681*/
1682
1683# define math_spec_node_size     3
1684# define math_spec_class(a)      vinfo00(a,1)  /* attr */
1685# define math_spec_family(a)     vinfo01(a,1)
1686# define math_spec_character(a)  vlink(a,1)    /* prev */
1687# define math_spec_properties(a) vinfo0(a,2)
1688# define math_spec_group(a)      vinfo1(a,2)
1689# define math_spec_index(a)      vlink(a,2)
1690
1691# define math_spec_value(a)     (((math_spec_class(a) & 0x3F) << 12) + ((math_spec_family(a) & 0x3F) << 8) + (math_spec_character(a) & 0xFF))
1692
1693static inline int tex_same_mathspec(halfword a, halfword b)
1694{
1695    return
1696        (a == b)
1697     || (a && b && math_spec_class(a)      == math_spec_class(b)
1698                && math_spec_family(a)     == math_spec_family(b)
1699                && math_spec_character(a)  == math_spec_character(b)
1700                && math_spec_properties(a) == math_spec_properties(b)
1701                && math_spec_group(a)      == math_spec_group(b)
1702                && math_spec_index(a)      == math_spec_index(b)
1703        )
1704    ;
1705}
1706
1707/*tex
1708    Here are some more stack related nodes.
1709*/
1710
1711# define align_stack_node_size                16
1712# define align_stack_align_ptr(a)             vinfo(a,1)
1713# define align_stack_cur_align(a)             vlink(a,1)
1714# define align_stack_preamble(a)              vinfo(a,2)
1715# define align_stack_cur_span(a)              vlink(a,2)
1716# define align_stack_cur_loop(a)              vinfo(a,3)
1717# define align_stack_wrap_source(a)           vlink(a,3)
1718# define align_stack_align_state(a)           vinfo(a,4)
1719# define align_stack_no_align_level(a)        vlink(a,4)
1720# define align_stack_cur_post_adjust_head(a)  vinfo(a,5)
1721# define align_stack_cur_post_adjust_tail(a)  vlink(a,5)
1722# define align_stack_cur_pre_adjust_head(a)   vinfo(a,6)
1723# define align_stack_cur_pre_adjust_tail(a)   vlink(a,6)
1724# define align_stack_cur_post_migrate_head(a) vinfo(a,7)
1725# define align_stack_cur_post_migrate_tail(a) vlink(a,7)
1726# define align_stack_cur_pre_migrate_head(a)  vinfo(a,8)
1727# define align_stack_cur_pre_migrate_tail(a)  vlink(a,8)
1728# define align_stack_options(a)               vinfo(a,9)
1729# define align_stack_attr_list(a)             vlink(a,9)
1730# define align_stack_callback(a)              vinfo(a,10)
1731# define align_stack_data(a)                  vlink(a,10) /* maybe */
1732
1733# define align_stack_row_attrlist(a)          vinfo(a,11)
1734# define align_stack_row_orientation(a)       vlink(a,11)
1735# define align_stack_row_yoffset(a)           vinfo(a,12)
1736# define align_stack_row_xoffset(a)           vlink(a,12)
1737# define align_stack_row_ymove(a)             vinfo(a,13)
1738# define align_stack_row_xmove(a)             vlink(a,13)
1739# define align_stack_row_shift(a)             vinfo(a,14)
1740# define align_stack_row_source(a)            vlink(a,14)
1741# define align_stack_row_target(a)            vinfo(a,15)
1742# define align_stack_row_anchor(a)            vlink(a,15)
1743
1744/*tex
1745    If nodes are for nesting conditionals. We have more state information that in (for instance)
1746    \LUATEX\ because we have more tracing and more test variants.
1747*/
1748
1749# define if_node_size           3             /*tex we can use prev now */
1750# define if_limit_type(a)       vinfo0(a,1)   /*tex overlaps with node_attr */
1751# define if_limit_subtype(a)    vinfo1(a,1)   /*tex overlaps with node_attr */
1752# define if_limit_unless(a)     vinfo00(a,2)
1753# define if_limit_step(a)       vinfo01(a,2)
1754# define if_limit_stepunless(a) vinfo02(a,2)
1755# define if_limit_unused(a)     vinfo03(a,2)
1756# define if_limit_line(a)       vlink(a,2)
1757
1758/*tex
1759    Now come some rather special ones. For instance par shapes and file cq.\ line related nodes
1760    were variable nodes. Thsi was dropped and replaced by a more generic specficiation node type.
1761    In principle we can use that for more purposes.
1762
1763    We use a bit of abstraction as preparation for different allocations. Dynamic allocation makes
1764    it possible to get rid of variable nodes but it is slower.
1765
1766    Because this node has no links we can use the next field as counter. The subtype is just for
1767    diagnostics. This node is special in the sense that it has a real pointer. Such nodes will not
1768    be stored in the format file. Because there is a pointer field we have some extra accessors.
1769
1770    Todo: we also need to catch the fact that we can run out of memory but in practice that will
1771    not happen soon, for instance because we seldom use parshapes. And in the meantime the pseudo
1772    file related nodes are gone anyway because all file IO has been delegated to \LUA\ now.
1773*/
1774
1775# define specification_node_size     4
1776# define specification_count(a)      vlink(a,0)      /* next */
1777# define specification_options(a)    vinfo(a,1)
1778# define specification_size(a)       vlink(a,1)      /* allocated size */
1779# define specification_pointer(a)    (mvalue(a,2))
1780# define specification_anything_1(a) vinfo(a,3)
1781# define specification_anything_2(a) vlink(a,3)
1782
1783/*tex
1784    We now define some math related nodes (and noads) and start with style and choice nodes. Style
1785    nodes can be smaller, the information is encoded in |subtype|, but choice nodes are on-the-spot
1786    converted to style nodes with slack. The advantage is that we don't run into issues when a choice
1787    node is the first node in which case we would have to adapt head pointers (read: feed them back
1788    into the calling routines). So, we keep this as it is now.
1789
1790    Parameter nodes started out as an experiment. We could actually use the same mechanism as
1791    attributes but (1) we don't want attribute nodes in the list, it is very math specific and (3)
1792    we don't need to be real fast here.
1793
1794    Maybe these three can be merged into one type but on the other hand they are part of the \TEX\
1795    legacy and well documented so \unknown for now we keep it as-is. In the meantime we are no
1796    longer casting choices to styles.
1797
1798*/
1799
1800# define style_node_size               3
1801# define style_style                   node_subtype
1802# define style_scale(a)                vinfo(a,2)
1803# define style_reserved(a)             vlink(a,2)
1804
1805# define choice_node_size              5
1806//define choice_style                  node_subtype
1807# define choice_display_mlist(a)       vinfo(a,2) /*tex mlist to be used in display style or pre_break */
1808# define choice_text_mlist(a)          vlink(a,2) /*tex mlist to be used in text style or post_break */
1809# define choice_script_mlist(a)        vinfo(a,3) /*tex mlist to be used in script style or no_break */
1810# define choice_script_script_mlist(a) vlink(a,3) /*tex mlist to be used in scriptscript style */
1811# define choice_class(a)               vinfo(a,4) /*tex we could abuse the script script field */
1812# define choice_unused(a)              vlink(a,4)
1813
1814# define choice_pre_break              choice_display_mlist
1815# define choice_post_break             choice_text_mlist
1816# define choice_no_break               choice_script_mlist
1817
1818# define parameter_node_size           3
1819# define parameter_style               node_subtype
1820# define parameter_name(a)             vinfo(a,2)
1821# define parameter_value(a)            vlink(a,2)
1822
1823typedef enum simple_choice_subtypes {
1824    normal_choice_subtype,
1825    discretionary_choice_subtype,
1826} simple_choice_subtypes;
1827
1828# define last_choice_subtype discretionary_choice_subtype
1829
1830/*tex
1831    Because noad types get changed when processing we need to make sure some if the node sizes
1832    match and that we don't share slots with different properties.
1833
1834    First come the regular noads. The generic noad has the same size and similar fields as a fence
1835    noad, and their types get swapped a few times.
1836
1837    We accept a little waste of space in order to get nicer code. After all, math is not that
1838    demanding. Although delimiter, accent, fraction and radical share the same structure we do use
1839    specific field names because of clarity. Not all fields are used always.
1840
1841    \starttabulate[|l|l|l|l|l|l|]
1842    \FL
1843    \BC            \BC noad       \BC accent            \BC fraction         \BC radical          \NC fence           \NC \NR
1844    \ML
1845    \NC vlink  2   \NC new_hlist  \NC                   \NC                  \NC                  \NC                 \NC \NR
1846    \ML
1847    \NC vinfo  2   \NC nucleus    \NC                   \NC                  \NC                  \NC                 \NC \NR
1848    \NC vlink  3   \NC supscr     \NC                   \NC                  \NC                  \NC                 \NC \NR
1849    \NC vinfo  3   \NC subscr     \NC                   \NC                  \NC                  \NC                 \NC \NR
1850    \NC vlink  4   \NC supprescr  \NC                   \NC                  \NC                  \NC                 \NC \NR
1851    \NC vinfo  4   \NC subprescr  \NC                   \NC                  \NC                  \NC                 \NC \NR
1852    \ML
1853    \NC vlink  5   \NC italic     \NC                   \NC                  \NC                  \NC                 \NC \NR
1854    \NC vinfo  5   \NC width      \NC                   \NC                  \NC                  \NC                 \NC \NR
1855    \NC vlink  6   \NC height     \NC                   \NC                  \NC                  \NC                 \NC \NR
1856    \NC vinfo  6   \NC depth      \NC                   \NC                  \NC                  \NC                 \NC \NR
1857    \ML
1858    \NC vlink  7   \NC options    \NC                   \NC                  \NC                  \NC                 \NC \NR
1859    \NC vinfo  7   \NC style      \NC                   \NC                  \NC                  \NC                 \NC \NR
1860    \NC vlink  8   \NC family     \NC                   \NC                  \NC                  \NC                 \NC \NR
1861    \NC vinfo  8   \NC class      \NC                   \NC                  \NC                  \NC                 \NC \NR
1862    \NC vlink  9   \NC source     \NC                   \NC                  \NC                  \NC                 \NC \NR
1863    \NC vinfo  9   \NC prime      \NC                   \NC                  \NC                  \NC                 \NC \NR
1864    \NC vlink 10   \NC leftslack  \NC                   \NC                  \NC                  \NC                 \NC \NR
1865    \NC vinfo 10   \NC rightslack \NC                   \NC                  \NC                  \NC                 \NC \NR
1866    \ML
1867    \NC vlink 11   \NC extra_1    \NC top_character     \NC rule_thickness   \NC degree           \NC delimiter       \NC \NR
1868    \NC vinfo 11   \NC extra_2    \NC bot_character     \NC left_delimiter   \NC left_delimiter   \NC source          \NC \NR
1869    \NC vlink 12   \NC extra_3    \NC overlay_character \NC right_delimiter  \NC right_delimiter  \NC topdelimiter    \NC \NR
1870    \NC vinfo 12   \NC extra_4    \NC fraction          \NC middle_delimiter \NC size             \NC bottomdelimiter \NC \NR
1871    \NC vlink 13   \NC extra_5    \NC topovershoot      \NC h_factor         \NC height           \NC topovershoot    \NC \NR
1872    \NC vinfo 13   \NC extra_6    \NC botovershoot      \NC v_factor         \NC depth            \NC botovershoot    \NC \NR
1873    \NC vlink 14   \NC extra_7    \NC                   \NC numerator        \NC top_delimiter    \NC delimitervariant\NC \NR
1874    \NC vinfo 14   \NC extra_8    \NC                   \NC denominator      \NC bottom_delimiter \NC                 \NC \NR
1875    \LL
1876    \stoptabulate
1877
1878    We can use smaller variables for style and class and then have one field available for other
1879    usage so no need to grow.
1880
1881    As with other nodes, not all fields are used and|/|or can be set at the tex end but they are
1882    available for usage at the \LUA\ end. Some have been used for experiments and stay around.
1883
1884*/
1885
1886//define noad_state_node_size      6
1887//define noad_state_topright(a)    vlink(a,2)
1888//define noad_state_bottomright(a) vinfo(a,2)
1889//define noad_state_topleft(a)     vlink(a,3)
1890//define noad_state_bottomleft(a)  vinfo(a,3)
1891//define noad_state_height(a)      vlink(a,4)
1892//define noad_state_depth(a)       vinfo(a,4)
1893//define noad_state_toptotal(a)    vlink(a,5)
1894//define noad_state_bottomtotal(a) vinfo(a,5)
1895
1896# define noad_size            18
1897# define noad_new_hlist(a)    vlink(a,2)    /*tex the translation of an mlist; a bit confusing name */
1898# define noad_nucleus(a)      vinfo(a,2)
1899# define noad_supscr(a)       vlink(a,3)
1900# define noad_subscr(a)       vinfo(a,3)
1901# define noad_supprescr(a)    vlink(a,4)
1902# define noad_subprescr(a)    vinfo(a,4)
1903# define noad_italic(a)       vlink(a,5)    /*tex Sometimes used, might become more. */
1904# define noad_width(a)        vinfo(a,5)
1905# define noad_height(a)       vlink(a,6)
1906# define noad_depth(a)        vinfo(a,6)
1907//define noad_options(a)      vlink(a,7)
1908//define noad_style(a)        vinfo00(a,7)
1909//define noad_family(a)       vinfo01(a,7)
1910//define noad_script_state(a) vinfo02(a,7)
1911//define noad_analyzed(a)     vinfo03(a,7)  /*tex used for experiments */
1912//define noad_state(a)        vlink(a,8)    /*tex this might replace */
1913# define noad_options(a)      lvalue(a,7)   /*tex 64 bit fullword */
1914# define noad_style(a)        vlink00(a,8)
1915# define noad_family(a)       vlink01(a,8)
1916# define noad_script_state(a) vlink02(a,8)
1917# define noad_analyzed(a)     vlink03(a,8)  /*tex used for experiments */
1918# define noad_class_main(a)   vinfo00(a,8)
1919# define noad_class_left(a)   vinfo01(a,8)
1920# define noad_class_right(a)  vinfo02(a,8)
1921# define noad_script_order(a) vinfo03(a,8)
1922# define noad_source(a)       vlink(a,9)
1923# define noad_prime(a)        vinfo(a,9)
1924# define noad_left_slack(a)   vlink(a,10)
1925# define noad_right_slack(a)  vinfo(a,10)
1926# define noad_subshift(a)     vinfo(a,11) /* continuation */
1927# define noad_supshift(a)     vlink(a,11) /* continuation */
1928# define noad_primeshift(a)   vlink(a,12) /* continuation */
1929# define noad_script_kern(a)  vinfo(a,12) /* continuation */
1930# define noad_extra_attr(a)   vlink(a,13)
1931# define noad_reserved(a)     vinfo(a,13)
1932# define noad_extra_1(a)      vlink(a,14)
1933# define noad_extra_2(a)      vinfo(a,14)
1934# define noad_extra_3(a)      vlink(a,15)
1935# define noad_extra_4(a)      vinfo(a,15)
1936# define noad_extra_5(a)      vlink(a,16)
1937# define noad_extra_6(a)      vinfo(a,16)
1938# define noad_extra_7(a)      vlink(a,17)
1939# define noad_extra_70(a)     vlink0(a,17)
1940# define noad_extra_71(a)     vlink1(a,17)
1941# define noad_extra_8(a)      vinfo(a,17)
1942# define noad_extra_80(a)     vinfo0(a,17)
1943# define noad_extra_81(a)     vinfo1(a,17)
1944
1945# define noad_total(a) (noad_height(a) + noad_depth(a))
1946
1947# define noad_has_postscripts(a)       (noad_subscr(a) || noad_supscr(a))
1948# define noad_has_prescripts(a)        (noad_subprescr(a) || noad_supprescr(a))
1949# define noad_has_scripts(a)           (noad_has_postscripts(a) || noad_has_prescripts(a) || noad_prime(a))
1950# define noad_has_following_scripts(a) (noad_subscr(a) || noad_supscr(a) || noad_prime(a))
1951# define noad_has_superscripts(a)      (noad_supprescr(a) || noad_supscr(a) || noad_prime(a))
1952# define noad_has_subscripts(a)        (noad_subprescr(a) || noad_subscr(a))
1953
1954# define noad_has_scriptstate(a,s)     ((noad_script_state(a) & s) == s)
1955
1956# define unset_noad_class 0xFE
1957
1958typedef enum noad_script_states {
1959    post_super_script_state = 0x01,
1960    post_sub_script_state   = 0x02,
1961    pre_super_script_state  = 0x04,
1962    pre_sub_script_state    = 0x08,
1963    prime_script_state      = 0x10,
1964} noad_script_states;
1965
1966typedef enum noad_script_locations {
1967    prime_unknown_location,
1968    prime_at_begin_location,
1969    prime_above_sub_location,
1970    prime_at_end_location,
1971} noad_prime_locations;
1972
1973typedef enum noad_script_order {
1974    script_unknown_first,
1975    script_primescript_first,
1976    script_subscript_first,
1977    script_superscript_first,
1978} noad_script_order;
1979
1980typedef struct noad_classes {
1981    singleword main;
1982    singleword left;
1983    singleword right;
1984} noad_classes;
1985
1986static inline void tex_reset_noad_classes(halfword n)
1987{
1988    noad_class_main(n)  = (singleword) unset_noad_class;
1989    noad_class_left(n)  = (singleword) unset_noad_class;
1990    noad_class_right(n) = (singleword) unset_noad_class;
1991    noad_analyzed(n)    = (singleword) unset_noad_class;
1992}
1993
1994static inline void tex_set_noad_classes(halfword n, halfword c)
1995{
1996    noad_class_main(n)  = (singleword) (c & 0xFF);
1997    noad_class_left(n)  = (singleword) (c & 0xFF);
1998    noad_class_right(n) = (singleword) (c & 0xFF);
1999}
2000
2001# define set_noad_main_class(n,c)  noad_class_main(n)  = (singleword) (c & 0xFF)
2002# define set_noad_left_class(n,c)  noad_class_left(n)  = (singleword) (c & 0xFF)
2003# define set_noad_right_class(n,c) noad_class_right(n) = (singleword) (c & 0xFF)
2004
2005# define get_noad_main_class(n)  (noad_class_main(n))
2006# define get_noad_left_class(n)  (noad_class_left(n))
2007# define get_noad_right_class(n) (noad_class_right(n))
2008
2009# define set_noad_style(n,s)  noad_style(n)  = (singleword) (s & 0xFF)
2010# define set_noad_family(n,f) noad_family(n) = (singleword) (f & 0xFF)
2011
2012/*tex
2013    Options are something \LUATEX\ and in \LUAMETATEX\ we added some more. When we have dimensions
2014    then we obey |axis| and otherwise |noaxis|. This might evolve a bit over time. These options
2015    currently are on the same spot but we pretend they aren't so we have dedicated accessors. This
2016    also makes clear what noads have what options.
2017
2018    If we run out of options we can combine some, like auto.
2019*/
2020
2021// # if (defined(_MSC_VER) && ! defined(__MINGW32__))
2022// typedef enum noad_options : unsigned __int64 {
2023// # else
2024typedef enum noad_options {
2025// # endif
2026    noad_option_axis                       = 0x00000001,
2027    noad_option_no_axis                    = 0x00000002,
2028    noad_option_exact                      = 0x00000004,
2029    noad_option_left                       = 0x00000008, /* align option for overflown under/over */ /* used ? */
2030    noad_option_middle                     = 0x00000010, /* idem */
2031    noad_option_right                      = 0x00000020, /* idem */
2032    noad_option_adapt_to_left_size         = 0x00000040, /* old trickery, might go away but kind of fun */
2033    noad_option_adapt_to_right_size        = 0x00000080, /* idem */
2034    noad_option_no_sub_script              = 0x00000100,
2035    noad_option_no_super_script            = 0x00000200,
2036    noad_option_no_sub_pre_script          = 0x00000400,
2037    noad_option_no_super_pre_script        = 0x00000800,
2038    noad_option_no_script                  = 0x00001000,
2039    noad_option_no_overflow                = 0x00002000, /* keep (middle) extensible widthin target size */
2040    noad_option_void                       = 0x00004000, /* wipe and set width to zero */
2041    noad_option_phantom                    = 0x00008000, /* wipe */
2042    noad_option_openup_height              = 0x00010000,
2043    noad_option_openup_depth               = 0x00020000,
2044    noad_option_limits                     = 0x00040000, /* traditional modifier */
2045    noad_option_no_limits                  = 0x00080000, /* idem */
2046    noad_option_prefer_font_thickness      = 0x00100000,
2047    noad_option_no_ruling                  = 0x00200000,
2048    noad_option_indexed_sub_script         = 0x00400000,
2049    noad_option_indexed_super_script       = 0x00800000,
2050    noad_option_indexed_sub_pre_script     = 0x01000000,
2051    noad_option_indexed_super_pre_script   = 0x02000000,
2052    noad_option_unpack_list                = 0x04000000,
2053    noad_option_no_check                   = 0x08000000, /* don't check for missing end fence */
2054    noad_option_auto                       = 0x10000000,
2055    noad_option_unroll_list                = 0x20000000,
2056    noad_option_followed_by_space          = 0x40000000,
2057    noad_option_proportional               = 0x80000000,
2058    /*tex Watch out: the following options exceed halfword: |noad_options| are |long long|. */
2059} noad_options;
2060
2061/*tex The Microsoft compiler truncates to int, so: */
2062
2063# define noad_option_source_on_nucleus          (uint64_t) 0x0000000100000000
2064# define noad_option_fixed_super_or_sub_script  (uint64_t) 0x0000000200000000
2065# define noad_option_fixed_super_and_sub_script (uint64_t) 0x0000000400000000
2066# define noad_option_auto_base                  (uint64_t) 0x0000000800000000
2067# define noad_option_stretch                    (uint64_t) 0x0000001000000000
2068# define noad_option_shrink                     (uint64_t) 0x0000002000000000
2069# define noad_option_center                     (uint64_t) 0x0000004000000000
2070# define noad_option_scale                      (uint64_t) 0x0000008000000000
2071# define noad_option_keep_base                  (uint64_t) 0x0000010000000000
2072# define noad_option_single                     (uint64_t) 0x0000020000000000
2073# define noad_option_no_rule                    (uint64_t) 0x0000040000000000
2074# define noad_option_auto_middle                (uint64_t) 0x0000080000000000
2075# define noad_option_reflected                  (uint64_t) 0x0000100000000000
2076# define noad_option_continuation               (uint64_t) 0x0000200000000000 /* relates to script continuation */
2077# define noad_option_inherit_class              (uint64_t) 0x0000400000000000 /* idem */
2078# define noad_option_discard_shape_kern         (uint64_t) 0x0000800000000000 /* idem */
2079# define noad_option_realign_scripts            (uint64_t) 0x0001000000000000 /* idem */
2080# define noad_option_ignore_empty_sub_script    (uint64_t) 0x0002000000000000
2081# define noad_option_ignore_empty_super_script  (uint64_t) 0x0004000000000000
2082# define noad_option_ignore_empty_prime_script  (uint64_t) 0x0008000000000000
2083# define noad_option_continuation_head          (uint64_t) 0x0010000000000000 /* relates to script continuation */
2084# define noad_option_continuation_kernel        (uint64_t) 0x0020000000000000 /* relates to script continuation */
2085# define noad_option_reorder_pre_scripts        (uint64_t) 0x0040000000000000 /* relates to script continuation */
2086# define noad_option_ignore                     (uint64_t) 0x0080000000000000 /* whatever fence */
2087# define noad_option_no_more_scripts            (uint64_t) 0x0100000000000000
2088# define noad_option_carry_over_classes         (uint64_t) 0x0200000000000000
2089# define noad_option_use_callback               (uint64_t) 0x0400000000000000
2090
2091# define has_option(a,b)   (((a) & (b)) == (b))
2092# define unset_option(a,b) ((a) & ~(b))
2093
2094static inline void tex_add_noad_option    (halfword a, uint64_t r) { noad_options(a) |= r; }
2095static inline void tex_remove_noad_option (halfword a, uint64_t r) { noad_options(a) &= ~r; }
2096static inline int  tex_has_noad_option    (halfword a, uint64_t r) { return (noad_options(a) & r) == r; }
2097
2098static inline int has_noad_no_script_option(halfword n, halfword option)
2099{
2100    switch (node_type(n)) {
2101        case simple_noad:
2102        case accent_noad:
2103        case radical_noad:
2104        case fence_noad:
2105        case fraction_noad:
2106            return has_option(noad_options(n), option) || has_option(noad_options(n), noad_option_no_script);
2107    }
2108    return 0;
2109}
2110
2111# define has_noad_option_nosubscript(a)    has_noad_no_script_option(a, noad_option_no_sub_script)
2112# define has_noad_option_nosupscript(a)    has_noad_no_script_option(a, noad_option_no_super_script)
2113# define has_noad_option_nosubprescript(a) has_noad_no_script_option(a, noad_option_no_sub_pre_script)
2114# define has_noad_option_nosupprescript(a) has_noad_no_script_option(a, noad_option_no_super_pre_script)
2115
2116# define has_noad_option_indexedsubscript(a)           (has_option(noad_options(a), noad_option_indexed_sub_script))
2117# define has_noad_option_indexedsupscript(a)           (has_option(noad_options(a), noad_option_indexed_super_script))
2118# define has_noad_option_indexedsubprescript(a)        (has_option(noad_options(a), noad_option_indexed_sub_pre_script))
2119# define has_noad_option_indexedsupprescript(a)        (has_option(noad_options(a), noad_option_indexed_super_pre_script))
2120# define has_noad_option_axis(a)                       (has_option(noad_options(a), noad_option_axis))
2121# define has_noad_option_exact(a)                      (has_option(noad_options(a), noad_option_exact))
2122# define has_noad_option_noaxis(a)                     (has_option(noad_options(a), noad_option_no_axis))
2123# define has_noad_option_openupheight(a)               (has_option(noad_options(a), noad_option_openup_height))
2124# define has_noad_option_openupdepth(a)                (has_option(noad_options(a), noad_option_openup_depth))
2125# define has_noad_option_adapttoleft(a)                (has_option(noad_options(a), noad_option_adapt_to_left_size))
2126# define has_noad_option_adapttoright(a)               (has_option(noad_options(a), noad_option_adapt_to_right_size))
2127# define has_noad_option_limits(a)                     (has_option(noad_options(a), noad_option_limits))
2128# define has_noad_option_nolimits(a)                   (has_option(noad_options(a), noad_option_no_limits))
2129# define has_noad_option_nooverflow(a)                 (has_option(noad_options(a), noad_option_no_overflow))
2130# define has_noad_option_preferfontthickness(a)        (has_option(noad_options(a), noad_option_prefer_font_thickness))
2131# define has_noad_option_noruling(a)                   (has_option(noad_options(a), noad_option_no_ruling))
2132# define has_noad_option_unpacklist(a)                 (has_option(noad_options(a), noad_option_unpack_list))
2133# define has_noad_option_nocheck(a)                    (has_option(noad_options(a), noad_option_no_check))
2134# define has_noad_option_exact(a)                      (has_option(noad_options(a), noad_option_exact))
2135# define has_noad_option_left(a)                       (has_option(noad_options(a), noad_option_left))
2136# define has_noad_option_middle(a)                     (has_option(noad_options(a), noad_option_middle))
2137# define has_noad_option_right(a)                      (has_option(noad_options(a), noad_option_right))
2138# define has_noad_option_auto(a)                       (has_option(noad_options(a), noad_option_auto))
2139# define has_noad_option_phantom(a)                    (has_option(noad_options(a), noad_option_phantom))
2140# define has_noad_option_void(a)                       (has_option(noad_options(a), noad_option_void))
2141# define has_noad_option_unrolllist(a)                 (has_option(noad_options(a), noad_option_unroll_list))
2142# define has_noad_option_followedbyspace(a)            (has_option(noad_options(a), noad_option_followed_by_space))
2143# define has_noad_option_proportional(a)               (has_option(noad_options(a), noad_option_proportional))
2144# define has_noad_option_center(a)                     (has_option(noad_options(a), noad_option_center))
2145# define has_noad_option_source_on_nucleus(a)          (has_option(noad_options(a), noad_option_source_on_nucleus))
2146# define has_noad_option_fixed_super_or_sub_script(a)  (has_option(noad_options(a), noad_option_fixed_super_or_sub_script))
2147# define has_noad_option_fixed_super_and_sub_script(a) (has_option(noad_options(a), noad_option_fixed_super_and_sub_script))
2148# define has_noad_option_stretch(a)                    (has_option(noad_options(a), noad_option_stretch))
2149# define has_noad_option_shrink(a)                     (has_option(noad_options(a), noad_option_shrink))
2150# define has_noad_option_auto_base(a)                  (has_option(noad_options(a), noad_option_auto_base))
2151# define has_noad_option_scale(a)                      (has_option(noad_options(a), noad_option_scale))
2152# define has_noad_option_keep_base(a)                  (has_option(noad_options(a), noad_option_keep_base))
2153# define has_noad_option_single(a)                     (has_option(noad_options(a), noad_option_single))
2154# define has_noad_option_norule(a)                     (has_option(noad_options(a), noad_option_no_rule))
2155# define has_noad_option_auto_middle(a)                (has_option(noad_options(a), noad_option_auto_middle))
2156# define has_noad_option_reflected(a)                  (has_option(noad_options(a), noad_option_reflected))
2157# define has_noad_option_continuation(a)               (has_option(noad_options(a), noad_option_continuation))
2158# define has_noad_option_inherit_class(a)              (has_option(noad_options(a), noad_option_inherit_class))
2159# define has_noad_option_discard_shape_kern(a)         (has_option(noad_options(a), noad_option_discard_shape_kern))
2160# define has_noad_option_realign_scripts(a)            (has_option(noad_options(a), noad_option_realign_scripts))
2161# define has_noad_option_ignore_empty_sub_script(a)    (has_option(noad_options(a), noad_option_ignore_empty_sub_script))
2162# define has_noad_option_ignore_empty_super_script(a)  (has_option(noad_options(a), noad_option_ignore_empty_super_script))
2163# define has_noad_option_ignore_empty_prime_script(a)  (has_option(noad_options(a), noad_option_ignore_empty_prime_script))
2164# define has_noad_option_continuation_head(a)          (has_option(noad_options(a), noad_option_continuation_head))
2165# define has_noad_option_continuation_kernel(a)        (has_option(noad_options(a), noad_option_continuation_kernel))
2166# define has_noad_option_reorder_pre_scripts(a)        (has_option(noad_options(a), noad_option_reorder_pre_scripts))
2167# define has_noad_option_ignore(a)                     (has_option(noad_options(a), noad_option_ignore))
2168# define has_noad_option_no_more_scripts(a)            (has_option(noad_options(a), noad_option_no_more_scripts))
2169# define has_noad_option_carry_over_classes(a)         (has_option(noad_options(a), noad_option_carry_over_classes))
2170# define has_noad_option_use_callback(a)               (has_option(noad_options(a), noad_option_use_callback))
2171
2172typedef enum double_atom_options {
2173    inherit_class_double_atom_option      = 0x01,
2174    discard_shape_kern_double_atom_option = 0x02,
2175    realign_scripts_double_atom_option    = 0x04,
2176    reorder_double_pre_script_atom_option = 0x08,
2177} double_atom_options;
2178
2179/*tex
2180    In the meantime the codes and subtypes are in sync. The variable component does not really
2181    become a subtype.
2182*/
2183
2184typedef enum simple_noad_subtypes {
2185    ordinary_noad_subtype,
2186    operator_noad_subtype,
2187    binary_noad_subtype,
2188    relation_noad_subtype,
2189    open_noad_subtype,
2190    close_noad_subtype,
2191    punctuation_noad_subtype,
2192    variable_noad_subtype,    /* we want to run in parallel */
2193    active_noad_subtype,      /* we want to run in parallel */
2194    inner_noad_subtype,
2195    under_noad_subtype,
2196    over_noad_subtype,
2197    fraction_noad_subtype,
2198    radical_noad_subtype,
2199    middle_noad_subtype,
2200    prime_noad_subtype,
2201    accent_noad_subtype,
2202    fenced_noad_subtype,
2203    ghost_noad_subtype,
2204    vcenter_noad_subtype,
2205} simple_noad_subtypes;
2206
2207# define last_noad_type    vcenter_noad_subtype
2208# define last_noad_subtype vcenter_noad_subtype
2209
2210typedef enum math_component_types {
2211    math_component_ordinary_code,
2212    math_component_operator_code,
2213    math_component_binary_code,
2214    math_component_relation_code,
2215    math_component_open_code,
2216    math_component_close_code,
2217    math_component_punctuation_code,
2218    math_component_variable_code,
2219    math_component_inner_code,
2220    math_component_under_code,
2221    math_component_over_code,
2222    math_component_fraction_code,
2223    math_component_radical_code,
2224    math_component_middle_code,
2225    math_component_prime_code,
2226    math_component_accent_code,
2227    math_component_fenced_code,
2228    math_component_ghost_code,
2229    math_component_atom_code,
2230} math_component_types;
2231
2232# define first_math_component_type math_component_ordinary_code
2233# define last_math_component_type  math_component_accent_code
2234
2235/*tex
2236    When I added adapt options, the |math_limits_cmd| became |math_modifier_cmd| just because it
2237    nicely fits in there.
2238*/
2239
2240typedef enum math_modifier_types {
2241    display_limits_modifier_code,
2242    limits_modifier_code,
2243    no_limits_modifier_code,
2244    adapt_to_left_modifier_code,
2245    adapt_to_right_modifier_code,
2246    axis_modifier_code,
2247    no_axis_modifier_code,
2248    phantom_modifier_code,
2249    void_modifier_code,
2250    source_modifier_code,
2251    openup_height_modifier_code,
2252    openup_depth_modifier_code,
2253} math_modifier_types;
2254
2255# define first_math_modifier_code display_limits_modifier_code
2256# define last_math_modifier_code  openup_depth_modifier_code
2257
2258/*tex accent noads: todo, left and right offsets and options */
2259
2260# define accent_noad_size        noad_size
2261# define accent_top_character    noad_extra_1 /*tex the |top_accent_chr| field of an accent noad */
2262# define accent_bottom_character noad_extra_2 /*tex the |bot_accent_chr| field of an accent noad */
2263# define accent_middle_character noad_extra_3 /*tex the |overlay_accent_chr| field of an accent noad */
2264# define accent_fraction         noad_extra_4
2265# define accent_top_overshoot    noad_extra_5
2266# define accent_bot_overshoot    noad_extra_6
2267
2268typedef enum math_accent_subtypes {
2269    /* we can be more specific: todo fixedoverlay_accent_subtype */
2270    bothflexible_accent_subtype,
2271    fixedtop_accent_subtype,
2272    fixedbottom_accent_subtype,
2273    fixedboth_accent_subtype,
2274} math_accent_subtypes;
2275
2276# define last_accent_subtype fixedboth_accent_subtype
2277
2278/*tex
2279    With these left and right fencing noads we have a historical mix of |fence| and |delimiter| (and
2280    |shield|) naming which for now we keep. It gets swapped with the generic noad, so size matters.
2281 */
2282
2283# define fence_noad_size         noad_size
2284# define fence_delimiter         noad_extra_1
2285# define fence_nesting_factor    noad_extra_2
2286# define fence_delimiter_top     noad_extra_3
2287# define fence_delimiter_bottom  noad_extra_4
2288# define fence_top_overshoot     noad_extra_5
2289# define fence_bottom_overshoot  noad_extra_6
2290# define fence_delimiter_variant noad_extra_7
2291
2292typedef enum fence_subtypes {
2293    unset_fence_side,
2294    left_fence_side,
2295    middle_fence_side,
2296    right_fence_side,
2297    left_operator_side,
2298    no_fence_side,
2299    extended_left_fence_side,
2300    extended_middle_fence_side,
2301    extended_right_fence_side,
2302} fence_subtypes;
2303
2304# define last_fence_subtype extended_right_fence_side
2305# define first_fence_code   left_fence_side
2306# define last_fence_code    extended_right_fence_side
2307
2308/*tex
2309    Fraction noads are generic in the sense that they are also used for non|-|fractions, not that
2310    it matters much. We keep them as they are in \TEX\ but have more fields.
2311
2312    We put the numerator and denomerator in script fields so there can be no such direct scripts
2313    attached. Because we have prescripts we can used these fields and limit this handicap a bit but
2314    if we ever overcome this (at the cost of more fields in these similar noads) we need to adapt
2315    the error message for double scripts in |tex_run_math_script|.
2316
2317*/
2318
2319# define fraction_noad_size        noad_size
2320# define fraction_rule_thickness   noad_extra_1
2321# define fraction_left_delimiter   noad_extra_2
2322# define fraction_right_delimiter  noad_extra_3
2323# define fraction_middle_delimiter noad_extra_4
2324# define fraction_h_factor         noad_extra_5
2325# define fraction_v_factor         noad_extra_6
2326# define fraction_numerator        noad_extra_7
2327# define fraction_denominator      noad_extra_8
2328
2329typedef enum fraction_subtypes {
2330    over_fraction_subtype,
2331    atop_fraction_subtype,
2332    above_fraction_subtype,
2333    skewed_fraction_subtype,
2334    stretched_fraction_subtype,
2335} fraction_subtypes;
2336
2337# define valid_fraction_subtype(s) (s >= over_fraction_subtype && s <= stretched_fraction_subtype)
2338
2339# define last_fraction_subtype stretched_fraction_subtype
2340
2341/*tex
2342    Radical noads are like fraction noads, but they only store a |left_delimiter|. They are also
2343    used for extensibles (over, under, etc) so the name is is somewhat confusing.
2344*/
2345
2346# define radical_noad_size        noad_size
2347# define radical_degree           noad_extra_1
2348# define radical_left_delimiter   noad_extra_2
2349# define radical_right_delimiter  noad_extra_3
2350# define radical_size             noad_extra_4
2351# define radical_height           noad_extra_5
2352# define radical_depth            noad_extra_6
2353# define radical_top_delimiter    noad_extra_7
2354# define radical_bottom_delimiter noad_extra_8
2355
2356typedef enum radical_subtypes {
2357    normal_radical_subtype,
2358    radical_radical_subtype,
2359    root_radical_subtype,
2360    rooted_radical_subtype,
2361    under_delimiter_radical_subtype,
2362    over_delimiter_radical_subtype,
2363    delimiter_under_radical_subtype,
2364    delimiter_over_radical_subtype,
2365    delimited_radical_subtype,
2366    h_extensible_radical_subtype,
2367} radical_subtypes;
2368
2369# define last_radical_subtype h_extensible_radical_subtype
2370# define last_radical_code    h_extensible_radical_subtype
2371
2372/*tex
2373    Again a very simple small node: it represents a math character so naturally it has a family.
2374    It can be turned list. These are subnodes. When an extra options field gets added, the
2375    overlapping character and list fields can be split, so then we also have the origin saved.
2376
2377    The following nodes are kernel nodes: |math_char_node|, |math_text_char_node|, |sub_box_node|
2378    and |sub_mlist_node|. Characters eventually becomes wrapped in a list.
2379*/
2380
2381typedef enum math_kernel_options {
2382    math_kernel_no_italic_correction = 0x0001,
2383    math_kernel_no_left_pair_kern    = 0x0002,
2384    math_kernel_no_right_pair_kern   = 0x0004,
2385    math_kernel_auto_discretionary   = 0x0008,
2386    math_kernel_full_discretionary   = 0x0010,
2387    math_kernel_ignored_character    = 0x0020,
2388    math_kernel_is_large_operator    = 0x0040,
2389    math_kernel_has_italic_shape     = 0x0080,
2390} math_kernel_options;
2391
2392# define math_kernel_node_size     5
2393# define kernel_math_family(a)     vinfo(a,2)   /* can be vinfo00 */
2394# define kernel_math_character(a)  vlink(a,2)
2395# define kernel_math_options(a)    vinfo(a,3)
2396# define kernel_math_list(a)       vlink(a,3)
2397# define kernel_math_properties(a) vinfo0(a,4)  /* for characters */
2398# define kernel_math_group(a)      vinfo1(a,4)  /* for characters */
2399# define kernel_math_index(a)      vlink(a,4)   /* for characters */
2400
2401/* delimiters can be old and new as type */
2402
2403# define kernel_math_small_family     vinfo00(a,2)
2404# define kernel_math_large_family     vinfo01(a,2)
2405# define kernel_math_small_character  vlink(a,2)
2406# define kernel_math_large_character  vinfo02(a,2)
2407
2408# define math_kernel_node_has_option(a,b) ((kernel_math_options(a) & b) == b)
2409# define math_kernel_node_set_option(a,b) kernel_math_options(a) = (kernel_math_options(a) | b)
2410
2411/*tex
2412    This is also a subnode, this time for a delimiter field. The large family field is only used
2413    in traditional \TEX\ fonts where a base character can come from one font, and the extensible
2414    from another, but in \OPENTYPE\ math font that doesn't happen.
2415*/
2416
2417# define math_delimiter_node_size     5
2418# define delimiter_small_family(a)    vinfo(a,2)   /*tex |family| for small delimiter */
2419# define delimiter_small_character(a) vlink(a,2)   /*tex |character| for small delimiter */
2420# define delimiter_large_family(a)    vinfo(a,3)   /*tex |family| for large delimiter */
2421# define delimiter_large_character(a) vlink(a,3)   /*tex |character| for large delimiter */
2422# define delimiter_math_properties(a) vinfo0(a,4)  /* for characters */
2423# define delimiter_math_group(a)      vinfo1(a,4)  /* for characters */
2424# define delimiter_math_index(a)      vlink(a,4)   /* for characters */
2425
2426/*tex
2427    Before we come to the by now rather large local par node we define some small ones. The
2428    boundary nodes are an extended version of the original ones. The direction nodes are
2429    a simplified version of what \OMEGA\ has as whatsit. In \LUATEX\ it became a first class
2430    citizen and in \LUAMETATEX\ we cleaned it up.
2431*/
2432
2433typedef enum boundary_subtypes {
2434    cancel_boundary,
2435    user_boundary,
2436    protrusion_boundary, /* 1=left, 2=right, 3=both */
2437    word_boundary,
2438    page_boundary,
2439    math_boundary,
2440    optional_boundary,
2441    lua_boundary,
2442    par_boundary,
2443    adjust_boundary,
2444    balance_boundary,
2445} boundary_subtypes;
2446
2447typedef enum protrusion_boundary_options {
2448    protrusion_skip_none,
2449    protrusion_skip_next,
2450    protrusion_skip_previous,
2451    protrusion_skip_both,
2452} protrusion_boundary_options;
2453
2454typedef enum math_boundary_options {
2455    /* default multiplier 1000 */
2456    begin_math_implicit_boundary,
2457    end_math_implicit_boundary,
2458    /* given multiplier */
2459    begin_math_explicit_boundary,
2460    end_math_explicit_boundary,
2461} math_boundary_options;
2462
2463# define last_boundary_subtype word_boundary
2464# define last_boundary_code    optional_boundary
2465
2466# define boundary_node_size   3
2467# define boundary_data(a)     vinfo(a,2)
2468# define boundary_reserved(a) vlink(a,2) // maybe level
2469
2470typedef enum dir_subtypes {
2471    normal_dir_subtype,
2472    cancel_dir_subtype,
2473} dir_subtypes;
2474
2475# define last_dir_subtype cancel_dir_subtype
2476
2477# define dir_node_size    3
2478# define dir_direction(a) vinfo(a,2)
2479# define dir_level(a)     vlink(a,2)
2480
2481/*tex
2482    Local par nodes come from \OMEGA\ and store the direction as well as local boxes. In \LUATEX
2483    we use a leaner direction model and in \LUAMETATEX\ we only kept the two directions that just
2484    work. In the end it is the backend that deals with these properties. The frontend just keeps
2485    a little track of them.
2486
2487    However, in \LUAMETATEX\ we can also store the paragraph state in this node. That way we no
2488    longer have the issue that properties are lost when a group ends before a |\par| is triggered.
2489    This is probably a feature that only makes sense in \CONTEXT\ which is why I made sure that
2490    there is not much overhead. In the first version one could control each variable, but as we
2491    ran out of bits in the end was done per group of variables. However, when I really need more
2492    detail I might go for a 64 bit field instead. After all we have that possibility in memory
2493    words.
2494
2495    These local par nodes can actually end up in the middle of lines  as they can be used to change
2496    the left and right box as well as inject penalties. For that reason they now have a proper
2497    subtype so that the initial and successive instances can be recognized.
2498 */
2499
2500typedef enum par_categories {
2501    par_none_category                = 0x00000000,
2502    par_hsize_category               = 0x00000001, // \hsize
2503    par_skip_category                = 0x00000002, // \leftskip \rightskip
2504    par_hang_category                = 0x00000004, // \hangindent \hangafter
2505    par_indent_category              = 0x00000008, // \parindent
2506    par_par_fill_category            = 0x00000010, // \parfillskip \parfillleftskip
2507    par_adjust_category              = 0x00000020, // \adjustspacing
2508    par_protrude_category            = 0x00000040, // \protrudechars
2509    par_tolerance_category           = 0x00000080, // \tolerance \pretolerance
2510    par_stretch_category             = 0x00000100, // \emergcystretch
2511    par_looseness_category           = 0x00000200, // \looseness
2512    par_last_line_category           = 0x00000400, // \lastlinefit
2513    par_line_penalty_category        = 0x00000800, // \linepenalty \interlinepenalty \interlinepenalties
2514    par_club_penalty_category        = 0x00001000, // \clubpenalty \clubpenalties
2515    par_widow_penalty_category       = 0x00002000, // \widowpenalty \widowpenalties
2516    par_display_penalty_category     = 0x00004000, // \displaypenalty \displaypenalties
2517    par_broken_penalty_category      = 0x00008000, // \brokenpenalty
2518    par_demerits_category            = 0x00010000, // \doublehyphendemerits \finalhyphendemerits \adjdemerits
2519    par_shape_category               = 0x00020000, // \parshape
2520    par_line_category                = 0x00040000, // \baselineskip \lineskip \lineskiplimit
2521    par_hyphenation_category         = 0x00080000, // \hyphenationmode
2522    par_shaping_penalty_category     = 0x00100000, // \shapingpenaltiesmode
2523    par_orphan_penalty_category      = 0x00200000, // \orphanpenalties
2524    par_toddler_penalty_category     = 0x00400000, //
2525    par_emergency_category           = 0x00800000, // \emergencyleftskip \emergencyrightskip \emergencyextrastretch
2526    par_par_passes_category          = 0x01000000, //
2527    par_single_line_penalty_category = 0x02000000, //
2528    par_hyphen_penalty_category      = 0x04000000, //
2529    par_ex_hyphen_penalty_category   = 0x08000000, //
2530    par_line_break_checks_category   = 0x10000000, //
2531    par_twin_demerits_category       = 0x20000000, //
2532    par_fitness_classes_category     = 0x40000000, //
2533    par_all_category                 = 0x7FFFFFFF, //
2534} par_categories;
2535
2536typedef enum par_codes {                   /* extrahyphenpenalty : in parpass     */
2537    par_none_code,                         /* hyphenpenalty      : why not stored */
2538    par_hsize_code,
2539    par_left_skip_code,
2540    par_right_skip_code,
2541    par_hang_indent_code,
2542    par_hang_after_code,
2543    par_par_indent_code,
2544    par_par_fill_left_skip_code,
2545    par_par_fill_right_skip_code,
2546    par_par_init_left_skip_code,
2547    par_par_init_right_skip_code,
2548    par_emergency_left_skip_code,
2549    par_emergency_right_skip_code,
2550    par_adjust_spacing_code,               /* parpass */
2551    par_protrude_chars_code,
2552    par_pre_tolerance_code,
2553    par_tolerance_code,                    /* parpass */
2554    par_emergency_stretch_code,            /* parpass */ /* dimension */
2555    par_looseness_code,                    /* parpass */
2556    par_last_line_fit_code,
2557    par_line_penalty_code,                 /* parpass */
2558    par_inter_line_penalty_code,
2559    par_club_penalty_code,
2560    par_widow_penalty_code,
2561    par_display_widow_penalty_code,
2562    par_left_twin_demerits_code,           /* parpass */
2563    par_right_twin_demerits_code,          /* parpass */
2564    par_broken_penalty_code,
2565    par_adj_demerits_code,                 /* parpass */
2566    par_double_hyphen_demerits_code,       /* parpass */
2567    par_final_hyphen_demerits_code,        /* parpass */
2568    par_par_shape_code,
2569    par_inter_line_penalties_code,
2570    par_club_penalties_code,
2571    par_widow_penalties_code,
2572    par_display_widow_penalties_code,
2573    par_broken_penalties_code,
2574    par_orphan_penalties_code,
2575    par_toddler_penalties_code,
2576    par_fitness_classes_code,              /* parpass */
2577    par_adjacent_demerits_code,            /* parpass */
2578    par_orphan_line_factors_code,          /* parpass */
2579    par_baseline_skip_code,
2580    par_line_skip_code,
2581    par_line_skip_limit_code,
2582    par_adjust_spacing_step_code,          /* parpass */
2583    par_adjust_spacing_shrink_code,        /* parpass */
2584    par_adjust_spacing_stretch_code,       /* parpass */
2585    par_hyphenation_mode_code,
2586    par_shaping_penalties_mode_code,
2587    par_shaping_penalty_code,
2588    par_emergency_extra_stretch_code,
2589    par_par_passes_code,
2590    par_line_break_checks_code,
2591    par_single_line_penalty_code,
2592    par_hyphen_penalty_code,
2593    par_ex_hyphen_penalty_code,
2594    par_n_of_codes,
2595} par_codes;
2596
2597static int par_category_to_codes[par_n_of_codes] = { /* explicit size is check */
2598    par_none_category,                // par_none_code
2599    par_hsize_category,               // par_hsize_code
2600    par_skip_category,                // par_left_skip_code
2601    par_skip_category,                // par_right_skip_code
2602    par_hang_category,                // par_hang_indent_code
2603    par_hang_category,                // par_hang_after_code
2604    par_indent_category,              // par_par_indent_code
2605    par_par_fill_category,            // par_par_fill_left_skip_code
2606    par_par_fill_category,            // par_par_fill_right_skip_code
2607    par_par_fill_category,            // par_par_init_left_skip_code
2608    par_par_fill_category,            // par_par_init_right_skip_code
2609    par_emergency_category,           // par_emergency_left_skip_code
2610    par_emergency_category,           // par_emergency_right_skip_code
2611    par_adjust_category,              // par_adjust_spacing_code
2612    par_protrude_category,            // par_protrude_chars_code
2613    par_tolerance_category,           // par_pre_tolerance_code
2614    par_tolerance_category,           // par_tolerance_code
2615    par_stretch_category,             // par_emergency_stretch_code
2616    par_looseness_category,           // par_looseness_code
2617    par_last_line_category,           // par_last_line_fit_code
2618    par_line_penalty_category,        // par_line_penalty_code
2619    par_line_penalty_category,        // par_inter_line_penalty_code
2620    par_club_penalty_category,        // par_club_penalty_code
2621    par_widow_penalty_category,       // par_widow_penalty_code
2622    par_display_penalty_category,     // par_display_widow_penalty_code
2623    par_twin_demerits_category,       // par_left_twin_demerits_code
2624    par_twin_demerits_category,       // par_right_twin_demerits_code
2625    par_broken_penalty_category,      // par_broken_penalty_code
2626    par_demerits_category,            // par_adj_demerits_code
2627    par_demerits_category,            // par_double_hyphen_demerits_code
2628    par_demerits_category,            // par_final_hyphen_demerits_code
2629    par_shape_category,               // par_par_shape_code
2630    par_line_penalty_category,        // par_inter_line_penalties_code
2631    par_club_penalty_category,        // par_club_penalties_code
2632    par_widow_penalty_category,       // par_widow_penalties_code
2633    par_display_penalty_category,     // par_display_widow_penalties_code
2634    par_broken_penalty_category,      // par_broken_penalties_code
2635    par_orphan_penalty_category,      // par_orphan_penalties_code
2636    par_toddler_penalty_category,     // par_toddler_penalties_code
2637    par_fitness_classes_category,     // par_fitness_classes_code
2638    par_demerits_category,            // par_adjacent_demerits_code
2639    par_orphan_penalty_category,      // par_orphan_line_factors_code
2640    par_line_category,                // par_baseline_skip_code
2641    par_line_category,                // par_line_skip_code
2642    par_line_category,                // par_line_skip_limit_code
2643    par_adjust_category,              // par_adjust_spacing_step_code
2644    par_adjust_category,              // par_adjust_spacing_shrink_code
2645    par_adjust_category,              // par_adjust_spacing_stretch_code
2646    par_hyphenation_category,         // par_hyphenation_mode_code
2647    par_shaping_penalty_category,     // par_shaping_penalties_mode_code
2648    par_shaping_penalty_category,     // par_shaping_penalty_code
2649    par_emergency_category,           // par_emergency_extra_stretch_code
2650    par_par_passes_category,          // par_par_passes_code
2651    par_line_break_checks_category,   // par_line_break_checks_code
2652    par_single_line_penalty_category, // par_single_line_penalty_code
2653    par_hyphen_penalty_category,      // par_hyphen_penalty_code
2654    par_ex_hyphen_penalty_category,   // par_ex_hyphen_penalty_code
2655};
2656
2657
2658typedef enum par_options {      
2659    par_option_synchronize = 0x01,
2660} par_options;
2661
2662/*tex Make sure that |max_chain_size| is large enough to have this huge node! */
2663
2664# define par_node_size                   35 // todo: less because we can pack some
2665
2666# define par_dir(a)                      vlink00(a, 2)
2667# define par_options(a)                  vlink01(a, 2)
2668# define par_box_left(a)                 vinfo(a, 2)
2669# define par_box_left_width(a)           vlink(a, 3)
2670# define par_box_right(a)                vinfo(a, 3)
2671# define par_box_right_width(a)          vlink(a, 4)
2672# define par_box_middle(a)               vinfo(a, 4)
2673# define par_par_passes(a)               vlink(a, 5)
2674# define par_line_break_checks(a)        vinfo(a, 5)
2675# define par_state(a)                    vlink(a, 6)
2676# define par_prev_graf(a)                vinfo(a, 6) /*tex A bit of a joke but maybe handy indeed. */
2677# define par_hsize(a)                    vlink(a, 7)
2678# define par_left_skip(a)                vinfo(a, 7)
2679# define par_right_skip(a)               vlink(a, 8)
2680# define par_hang_indent(a)              vinfo(a, 8)
2681# define par_hang_after(a)               vlink(a, 9)
2682# define par_par_indent(a)               vinfo(a, 9)
2683# define par_par_fill_left_skip(a)       vlink(a,10)
2684# define par_par_fill_right_skip(a)      vinfo(a,10)
2685# define par_adjust_spacing(a)           vlink(a,11) /* can be single */
2686# define par_protrude_chars(a)           vinfo(a,11) /* can be single */
2687# define par_pre_tolerance(a)            vlink(a,12)
2688# define par_tolerance(a)                vinfo(a,12)
2689# define par_emergency_stretch(a)        vlink(a,13)
2690# define par_looseness(a)                vinfo(a,13) /* can be less */
2691# define par_last_line_fit(a)            vlink(a,14) /* can be less */
2692# define par_line_penalty(a)             vinfo(a,14)
2693# define par_inter_line_penalty(a)       vlink(a,15)
2694# define par_club_penalty(a)             vinfo(a,15)
2695# define par_widow_penalty(a)            vlink(a,16)
2696# define par_display_widow_penalty(a)    vinfo(a,16)
2697# define par_orphan_penalty(a)           vlink(a,17)
2698# define par_toddler_penalties(a)        vinfo(a,17)
2699# define par_broken_penalty(a)           vlink(a,18)
2700# define par_adj_demerits(a)             vinfo(a,18)
2701# define par_double_hyphen_demerits(a)   vlink(a,19)
2702# define par_final_hyphen_demerits(a)    vinfo(a,19)
2703# define par_par_shape(a)                vlink(a,20)
2704# define par_inter_line_penalties(a)     vinfo(a,20)
2705# define par_club_penalties(a)           vlink(a,21)
2706# define par_widow_penalties(a)          vinfo(a,21)
2707# define par_display_widow_penalties(a)  vlink(a,22)
2708# define par_broken_penalties(a)         vinfo(a,22)
2709# define par_orphan_penalties(a)         vlink(a,23)
2710# define par_single_line_penalty(a)      vinfo(a,23)
2711# define par_baseline_skip(a)            vlink(a,24)
2712# define par_line_skip(a)                vinfo(a,24)
2713# define par_line_skip_limit(a)          vlink(a,25)
2714# define par_adjust_spacing_step(a)      vinfo(a,25)
2715# define par_adjust_spacing_shrink(a)    vlink(a,26)
2716# define par_adjust_spacing_stretch(a)   vinfo(a,26)
2717# define par_end_par_tokens(a)           vlink(a,27)
2718# define par_hyphenation_mode(a)         vinfo(a,27) /* can be single */
2719# define par_shaping_penalties_mode(a)   vlink(a,28) /* can be single */
2720# define par_shaping_penalty(a)          vinfo(a,28)
2721# define par_par_init_left_skip(a)       vlink(a,29)
2722# define par_par_init_right_skip(a)      vinfo(a,29)
2723# define par_emergency_left_skip(a)      vlink(a,30)
2724# define par_emergency_right_skip(a)     vinfo(a,30)
2725# define par_emergency_extra_stretch(a)  vlink(a,31)
2726# define par_fitness_classes(a)          vinfo(a,31)
2727# define par_adjacent_demerits(a)        vlink(a,32)
2728# define par_hyphen_penalty(a)           vinfo(a,32)
2729# define par_ex_hyphen_penalty(a)        vlink(a,33)
2730# define par_left_twin_demerits(a)       vinfo(a,33)
2731# define par_right_twin_demerits(a)      vlink(a,34)
2732# define par_orphan_line_factors(a)      vinfo(a,34)
2733
2734/*
2735    At some point we will have this (array with double values), depends on the outcome of an
2736    experiment but I want to reserve this. We then also patch |texlocalboxes.c| line 295+.
2737*/
2738
2739// define par_lousyness(a)               vinfo(a,34)
2740// define par_reserved(a)                vlink(a,34)
2741
2742typedef enum par_subtypes {
2743    vmode_par_par_subtype,
2744    local_box_par_subtype,
2745    hmode_par_par_subtype,
2746    parameter_par_subtype,
2747    local_break_par_subtype,
2748    math_par_subtype,       /* not used yet */
2749} par_subtypes;
2750
2751# define last_par_subtype math_par_subtype
2752
2753static inline int tex_is_start_of_par_node(halfword n)
2754{
2755    return ( n && (node_type(n) == par_node) && (node_subtype(n) == vmode_par_par_subtype || node_subtype(n) == hmode_par_par_subtype) );
2756}
2757
2758extern halfword    tex_get_par_par          (halfword p, halfword what);
2759extern void        tex_set_par_par          (halfword p, halfword what, halfword v, int force);
2760extern void        tex_snapshot_par         (halfword p, halfword what);
2761extern halfword    tex_find_par_par         (halfword head);
2762/*     halfword    tex_internal_to_par_code (halfword cmd, halfword index); */
2763extern void        tex_update_par_par       (halfword cmd, halfword index);
2764
2765static inline int  tex_par_state_is_set     (halfword p, halfword what)     { return (par_state(p) & par_category_to_codes[what]) == par_category_to_codes[what]; }
2766static inline void tex_set_par_state        (halfword p, halfword what)     { par_state(p) |= par_category_to_codes[what]; }
2767static inline int  tex_par_to_be_set        (halfword state, halfword what) { return (state & par_category_to_codes[what]) == par_category_to_codes[what]; }
2768
2769/*tex
2770    Because whatsits are used by the backend (or callbacks in the frontend) we do provide this node.
2771    It only has the basic properties: subtype, attribute, prev link and  next link. User nodes have
2772    been dropped because one can use whatsits to achieve the same. We also don't standardize the
2773    subtypes as it's very macro package specific what they do. So, only a size here:
2774*/
2775
2776# define whatsit_node_size 2
2777
2778/*tex
2779    Active and passive nodes are used in the par builder. There is plenty of comments in the code
2780    that explains them (although it's not that trivial I guess). Delta nodes just store the
2781    progression in widths, stretch and shrink: they are copies of arrays. Originally they just used
2782    offsets:
2783
2784    \starttyping
2785    # define delta_node_size    10
2786    # define delta_field(a,n)   node_next(a + n)
2787    \stoptyping
2788
2789    But that wasted 9 halfs for storing the 9 fields. So, next I played with this:
2790
2791    \starttyping
2792    # define delta_field_1(d) (delta_field(d,1)) // or: vinfo(d,1)
2793    # define delta_field_2(d) (delta_field(d,2)) // or: vlink(d,1)
2794    ...
2795    # define delta_field_9(d) (delta_field(d,9)) // or: vinfo(d,5)
2796    \stoptyping
2797
2798    But soon after that more meaningfull names were introduced, simply because in the code where they
2799    are used also verbose names showed up.
2800
2801    The active node is actually a |hyphenated_node| or an |unhyphenated_node| but for now we keep
2802    the \TEX\ lingua. We could probably turn the type into a subtype and moev fitness to another
2803    spot.
2804*/
2805
2806/*tex
2807    We can use vinfo(a,2) for fitness instead the subtype field.  But then we also need to set
2808    it explicitly because now that happens in the allocator.
2809
2810    The subtype uisage is somewhat confusing. Maybe we need less in the active node.
2811*/
2812
2813# define active_node_size                  6            /*tex |hyphenated_node| or |unhyphenated_node| */
2814//define active_fitness                    node_subtype /*tex |very_loose_fit..tight_fit| on final line for this break */
2815# define active_fitness(a)                 vinfo1(a,0)
2816# define active_break_node(a)              vlink(a,1)   /*tex pointer to the corresponding passive node */
2817# define active_line_number(a)             vinfo(a,1)   /*tex line that begins at this breakpoint */
2818# define active_total_demerits(a)          vlink(a,2)   /*tex the quantity that \TEX\ minimizes */
2819# define active_line_width(a)              vinfo(a,2)
2820# define active_glue(a)                    vlink(a,3)   /*tex corresponding glue stretch or shrink */
2821# define active_short(a)                   vinfo(a,3)   /*tex |shortfall| of this line */
2822# define active_quality(a)                 vlink(a,4)   /* last line related, normally we can use the passive one */
2823# define active_deficiency(a)              vinfo(a,4)   /* last line related, normally we can use the passive one */
2824# define active_n_of_fitness_classes(a)    vlink(a,5)
2825# define active_reserved(a)                vinfo(a,5)
2826
2827# define active_page_number(a)             vinfo(a,1)
2828# define active_page_height(a)             vinfo(a,2)
2829
2830# define passive_node_size                 12
2831# define passive_fitness(a)                vinfo1(a,0)
2832# define passive_cur_break(a)              vlink(a,1)   /*tex in passive node, points to position of this breakpoint */
2833# define passive_prev_break(a)             vinfo(a,1)   /*tex points to passive node that should precede this one */
2834# define passive_interline_penalty(a)      vlink(a,2)
2835# define passive_broken_penalty(a)         vinfo(a,2)
2836# define passive_left_box(a)               vlink(a,3)
2837# define passive_left_box_width(a)         vinfo(a,3)
2838# define passive_last_left_box(a)          vlink(a,4)
2839# define passive_last_left_box_width(a)    vinfo(a,4)
2840# define passive_right_box(a)              vlink(a,5)
2841# define passive_right_box_width(a)        vinfo(a,5)
2842# define passive_serial(a)                 vlink(a,6)   /*tex serial number for symbolic identification (pass) */
2843# define passive_middle_box(a)             vinfo(a,6)
2844# define passive_quality(a)                vlink(a,7)
2845# define passive_deficiency(a)             vinfo(a,7)
2846# define passive_demerits(a)               vlink(a,8)
2847# define passive_par_node(a)               vinfo(a,8)   /*tex experiment */
2848# define passive_badness(a)                vlink(a,9)
2849# define passive_n_of_fitness_classes(a)   vinfo(a,9)
2850# define passive_ref_count(a)              vlink(a,10)
2851# define passive_reserved(a)               vinfo(a,10)
2852# define passive_factor(a)                 dvalue(a,11)
2853
2854# define delta_node_size                   6
2855# define delta_field_total_glue(d)         vinfo(d,1)
2856# define delta_field_total_stretch(d)      vlink(d,2)
2857# define delta_field_total_shrink(d)       vinfo(d,2)
2858# define delta_field_total_fi_amount(d)    vlink(d,3)
2859# define delta_field_total_fil_amount(d)   vinfo(d,3)
2860# define delta_field_total_fill_amount(d)  vlink(d,4)
2861# define delta_field_total_filll_amount(d) vinfo(d,4)
2862# define delta_field_font_shrink(d)        vlink(d,5)
2863# define delta_field_font_stretch(d)       vinfo(d,5)
2864
2865/*tex
2866    Again we now have some helpers. We have a double linked list so here we go:
2867*/
2868
2869static inline void tex_couple_nodes(int a, int b)
2870{
2871    node_next(a) = b;
2872    node_prev(b) = a;
2873}
2874
2875static inline void tex_try_couple_nodes(int a, int b)
2876{
2877    if (b) {
2878        if (a) {
2879            node_next(a) = b;
2880        }
2881        node_prev(b) = a;
2882    } else if (a) {
2883        node_next(a) = null;
2884   }
2885}
2886
2887static inline void tex_uncouple_node(int a)
2888{
2889    node_next(a) = null;
2890    node_prev(a) = null;
2891}
2892
2893static inline halfword tex_head_of_node_list(halfword n)
2894{
2895    while (node_prev(n)) {
2896        n = node_prev(n);
2897    }
2898    return n;
2899}
2900
2901static inline halfword tex_tail_of_node_list(halfword n)
2902{
2903    while (node_next(n)) {
2904        n = node_next(n);
2905    }
2906    return n;
2907}
2908
2909/*tex
2910    Attribute management is kind of complicated. They are stored in a sorted linked list and we
2911    try to share these for successive nodes. In \LUATEX\ a state is kept and reset frequently but
2912    in \LUAMETATEX\ we try to be more clever, for instance we keep track of grouping. This comes
2913    as some overhead but saves reconstructing (often the same) list. It also saves memory.
2914*/
2915
2916# define attribute_cache_disabled max_halfword
2917# define current_attribute_state  lmt_node_memory_state.attribute_cache
2918
2919extern halfword tex_copy_attribute_list        (halfword attr);
2920extern halfword tex_copy_attribute_list_set    (halfword attr, int index, int value);
2921extern halfword tex_patch_attribute_list       (halfword attr, int index, int value);
2922extern halfword tex_merge_attribute_list       (halfword first, halfword second);
2923extern void     tex_dereference_attribute_list (halfword attr);
2924extern void     tex_build_attribute_list       (halfword target);
2925extern halfword tex_current_attribute_list     (void);
2926extern int      tex_unset_attribute            (halfword target, int index, int value);
2927extern void     tex_unset_attributes           (halfword first, halfword last, int index);
2928extern void     tex_set_attribute              (halfword target, int index, int value);
2929extern int      tex_has_attribute              (halfword target, int index, int value);
2930
2931extern void     tex_reset_node_properties      (halfword target);
2932
2933# define get_attribute_list(target) \
2934    node_attr(target)
2935
2936/*
2937# define add_attribute_reference(a) do { \
2938    if (a && a != attribute_cache_disabled) { \
2939        ++attribute_count(a); \
2940    } \
2941} while (0)
2942*/
2943
2944static inline void add_attribute_reference(halfword a)
2945{
2946    if (a && a != attribute_cache_disabled) {
2947        ++attribute_count(a);
2948    }
2949}
2950
2951/*
2952# define delete_attribute_reference(a) do { \
2953    if (a && a != attribute_cache_disabled) { \
2954        tex_dereference_attribute_list(a); \
2955    } \
2956} while (0)
2957*/
2958
2959static inline void delete_attribute_reference(halfword a)
2960{
2961    if (a && a != attribute_cache_disabled) {
2962        tex_dereference_attribute_list(a);
2963    }
2964}
2965
2966# define remove_attribute_list(target) do { \
2967    halfword old_a = node_attr(target); \
2968    delete_attribute_reference(old_a); \
2969    node_attr(target) = null; \
2970} while (0)
2971
2972/*
2973static inline void remove_attribute_list(halfword target)
2974{
2975    halfword a_old = node_attr(target);
2976    if (a_old && a_old != attribute_cache_disabled) {
2977        dereference_attribute_list(a_old);
2978    }
2979    node_attr(target) = null;
2980}
2981*/
2982
2983/* This can be dangerous: */
2984
2985# define wipe_attribute_list_only(target) \
2986    node_attr(target) = null;
2987
2988/*tex
2989    Better is to add a ref before we remove one because there's the danger of premature freeing
2990    otherwise.
2991*/
2992
2993static inline void tex_attach_attribute_list_copy(halfword target, halfword source)
2994{
2995    if (source) {
2996        halfword a_new = node_attr(source);
2997        halfword a_old = node_attr(target);
2998        node_attr(target) = a_new;
2999        add_attribute_reference(a_new);
3000        delete_attribute_reference(a_old);
3001    }
3002}
3003
3004static inline void tex_attach_attribute_list_attribute(halfword target, halfword a_new)
3005{
3006    if (a_new) {
3007        halfword a_old = node_attr(target);
3008        if (a_old != a_new) {
3009            node_attr(target) = a_new;
3010            add_attribute_reference(a_new);
3011            delete_attribute_reference(a_old);
3012        }
3013    }
3014}
3015
3016# define attach_current_attribute_list tex_build_attribute_list /* (target) */
3017
3018# define set_current_attribute_state(v) do { \
3019      current_attribute_state = v; \
3020} while (0)
3021
3022/*
3023# define change_attribute_register(a,id,value) do { \
3024    if (eq_value(id) != value) { \
3025        if (is_global(a)) { \
3026            int i; \
3027            for (i = (lmt_save_state.save_stack_data.ptr - 1); i >= 0; i--) { \
3028                if (save_type(i) == attribute_list_save_type) { \
3029                    delete_attribute_reference(save_value(i)); \
3030                    save_value(i) = attribute_cache_disabled; \
3031                } \
3032            } \
3033        } else { \
3034            delete_attribute_reference(current_attribute_state); \
3035        } \
3036        set_current_attribute_state(attribute_cache_disabled); \
3037    } \
3038} while (0)
3039*/
3040
3041extern void tex_change_attribute_register(halfword a, halfword id, halfword value);
3042
3043/*tex
3044    We now arrive at some functions that report the nodes to users. The subtype information that
3045    is used in the \LUA\ interface is stored alongside.
3046*/
3047
3048extern void        tex_print_short_node_contents         (halfword n);
3049extern const char *tex_aux_subtype_str                   (halfword n);
3050extern void        tex_show_node_list                    (halfword n, int threshold, int max);
3051extern halfword    tex_actual_box_width                  (halfword r, scaled base_width);
3052extern void        tex_print_name                        (halfword p, const char *what);
3053extern void        tex_print_node_list                   (halfword n, const char *what, int threshold, int max);
3054/*     void        tex_print_node_and_details            (halfword p); */
3055/*     void        tex_print_subtype_and_attributes_info (halfword p, halfword s, node_info *data); */
3056extern void        tex_print_extended_subtype            (halfword p, quarterword s);
3057extern void        tex_aux_show_dictionary               (halfword p, halfword properties, halfword group, halfword index, halfword font, halfword character);
3058
3059/*tex
3060    Basic node management:
3061*/
3062
3063extern halfword tex_new_node        (quarterword i, quarterword j);
3064extern void     tex_flush_node_list (halfword n);
3065extern void     tex_flush_node      (halfword n);
3066extern halfword tex_copy_node_list  (halfword n, halfword e);
3067extern halfword tex_copy_node       (halfword n);
3068extern halfword tex_copy_node_only  (halfword n);
3069/*     halfword tex_fix_node_list   (halfword n); */
3070
3071/*tex
3072    We already defined glue and gluespec node but here are some of the properties
3073    that they have. Again a few helpers.
3074*/
3075
3076typedef enum glue_orders {
3077    normal_glue_order,
3078    fi_glue_order,
3079    fil_glue_order,
3080    fill_glue_order,
3081    filll_glue_order
3082} glue_orders;
3083
3084typedef enum glue_amounts {
3085    /* we waste slot zero, we padd anyway */
3086    total_advance_amount = 1, // 1 //
3087    total_stretch_amount = 2, // 3 //
3088    total_fi_amount      = 3, // 4 //
3089    total_fil_amount     = 4, // 5 //
3090    total_fill_amount    = 5, // 6 //
3091    total_filll_amount   = 6, // 7 //
3092    total_shrink_amount  = 7, // 2 //
3093    font_stretch_amount  = 8, // 8 //
3094    font_shrink_amount   = 9, // 9 //
3095    /* */
3096    max_height_amount    = 10,
3097    max_depth_amount     = 11,
3098    /* */
3099    n_of_glue_amounts    = 12,
3100} glue_amounts;
3101
3102# define min_glue_order normal_glue_order
3103# define max_glue_order filll_glue_order
3104
3105typedef enum glue_signs {
3106    normal_glue_sign,
3107    stretching_glue_sign,
3108    shrinking_glue_sign
3109} glue_signs;
3110
3111# define min_glue_sign normal_glue_sign
3112# define max_glue_sign shrinking_glue_sign
3113
3114# define normal_glue_multiplier 0.0
3115
3116static inline singleword tex_checked_glue_sign  (halfword sign)  { return ((sign  < min_glue_sign ) || (sign  > max_glue_sign )) ? normal_glue_sign  : (singleword) sign ; }
3117static inline halfword   tex_checked_glue_order (halfword order) { return ((order < min_glue_order) || (order > max_glue_order)) ? normal_glue_order : order; }
3118
3119/*tex
3120    These are reserved nodes that sit at the start of main memory. We could actually just allocate
3121    them, but then we also need to set some when we start up. Now they are just saved in the format
3122    file. In \TEX\ these nodes were shared as much as possible (using a reference count) but here
3123    we just use copies.
3124
3125    Below we start at |zero_glue| which in our case is just 0, or |null| in \TEX\ speak. After these
3126    reserved nodes the memory used for whatever nodes are needed takes off.
3127
3128    Changing this to real nodes makes sense but is also tricky due to initializations ... some day
3129    (we need to store stuff in the states then and these are not saved!).
3130
3131    The first five could actualy be replaced by assignments because we don't share them as \TEX\
3132    does which makes it easier the just change them (we don't need to save memory here). The other
3133    ones could be nodes that get initialized at startup and be put in the structs that need them,
3134    but for now we keep it as-it-is.
3135
3136*/
3137
3138# define fi_glue           (zero_glue         + glue_spec_size)
3139# define fi_l_glue         (fi_glue           + glue_spec_size)
3140# define fi_ll_glue        (fi_l_glue         + glue_spec_size)
3141# define fi_ss_glue        (fi_ll_glue        + glue_spec_size)
3142# define fi_l_neg_glue     (fi_ss_glue        + glue_spec_size)
3143
3144# define page_insert_head  (fi_l_neg_glue     + glue_spec_size)
3145# define contribute_head   (page_insert_head  + split_node_size) /*tex This was temp_node_size but we assign more. */
3146# define page_head         (contribute_head   + temp_node_size)
3147# define temp_head         (page_head         + glue_node_size)  /*tex It gets a glue type assigned. */
3148# define hold_head         (temp_head         + temp_node_size)
3149# define post_adjust_head  (hold_head         + temp_node_size)
3150# define pre_adjust_head   (post_adjust_head  + temp_node_size)
3151# define post_migrate_head (pre_adjust_head   + temp_node_size)
3152# define pre_migrate_head  (post_migrate_head + temp_node_size)
3153# define align_head        (pre_migrate_head  + temp_node_size)
3154# define active_head       (align_head        + temp_node_size)
3155# define end_span          (active_head       + active_node_size)
3156# define begin_period      (end_span          + span_node_size)  /*tex Used to mark begin of word in hjn. */
3157# define end_period        (begin_period      + glyph_node_size) /*tex Used to mark end of word in hjn. */
3158
3159# define last_reserved     (end_period        + glyph_node_size - 1)
3160
3161/*tex More helpers! */
3162
3163extern int       tex_list_has_glyph       (halfword list);
3164
3165extern halfword  tex_new_null_box_node    (quarterword type, quarterword subtype);
3166extern halfword  tex_new_rule_node        (quarterword subtype);
3167extern halfword  tex_new_glyph_node       (quarterword subtype, halfword fnt, halfword chr, halfword parent); /*tex afterwards: when we mess around */
3168extern halfword  tex_new_char_node        (quarterword subtype, halfword fnt, halfword chr, int all);         /*tex as we go: in maincontrol */
3169extern halfword  tex_new_text_glyph       (halfword fnt, halfword chr);
3170extern halfword  tex_new_disc_node        (quarterword subtype);
3171extern halfword  tex_new_glue_spec_node   (halfword param);
3172extern halfword  tex_new_param_glue_node  (quarterword param, quarterword subtype);
3173extern halfword  tex_new_glue_node        (halfword qlue, quarterword subtype);
3174extern halfword  tex_new_kern_node        (scaled width, quarterword subtype);
3175extern halfword  tex_new_penalty_node     (halfword penalty, quarterword subtype);
3176extern halfword  tex_new_par_node         (quarterword subtype);
3177
3178extern halfword  tex_new_temp_node        (void);
3179
3180extern scaled    tex_glyph_width          (halfword p); /* x/y scaled */
3181extern scaled    tex_glyph_height         (halfword p); /* x/y scaled */
3182extern scaled    tex_glyph_depth          (halfword p); /* x/y scaled */
3183extern scaled    tex_glyph_total          (halfword p); /* x/y scaled */
3184extern scaledwhd tex_glyph_dimensions     (halfword p); /* x/y scaled */
3185extern int       tex_glyph_has_dimensions (halfword p); /* x/y scaled */
3186extern scaled    tex_glyph_width_ex       (halfword p); /* x/y scaled, expansion included */
3187extern scaledwhd tex_glyph_dimensions_ex  (halfword p); /* x/y scaled, expansion included */
3188
3189extern halfword  tex_kern_dimension       (halfword p);
3190extern halfword  tex_kern_dimension_ex    (halfword p); /* expansion included */
3191
3192extern scaled    tex_effective_glue       (halfword parent, halfword glue);
3193
3194extern scaledwhd tex_pack_dimensions      (halfword p);
3195
3196extern halfword  tex_list_node_mem_usage  (void);
3197extern halfword  tex_reversed_node_list   (halfword list);
3198extern int       tex_n_of_used_nodes      (int counts[]);
3199
3200# define _valid_node_(p) ((p > lmt_node_memory_state.reserved) && (p < lmt_node_memory_state.nodes_data.allocated) && (lmt_node_memory_state.nodesizes[p] > 0))
3201
3202static inline int tex_valid_node(halfword n)
3203{
3204    return n && _valid_node_(n) ? n : null;
3205}
3206
3207/*tex This is a bit strange place but better than a macro elsewhere: */
3208
3209static inline int tex_math_skip_boundary(halfword n)
3210{
3211    return (n && node_type(n) == glue_node
3212              && (node_subtype(n) == space_skip_glue  ||
3213                  node_subtype(n) == xspace_skip_glue ||
3214                  node_subtype(n) == zero_space_skip_glue));
3215}
3216
3217typedef enum special_node_list_types { /* not in sycn with the above .. maybe add bogus ones */
3218    page_insert_list_type,
3219    contribute_list_type,
3220    page_list_type,
3221    temp_list_type,
3222    hold_list_type,
3223    post_adjust_list_type,
3224    pre_adjust_list_type,
3225    post_migrate_list_type,
3226    pre_migrate_list_type,
3227    align_list_type,
3228    /* in different spot */
3229    page_discards_list_type,
3230    split_discards_list_type,
3231 // best_page_break_type
3232} special_node_list_types;
3233
3234extern int      tex_is_special_node_list  (halfword n, int *istail);
3235extern halfword tex_get_special_node_list (special_node_list_types list, halfword *tail);
3236extern void     tex_set_special_node_list (special_node_list_types list, halfword head);
3237
3238# endif
3239
3240