Source Browser ?

1/*
2    See license.txt in the root of this project.
3*/
4
5# ifndef LMT_TEXNODES_H
6# define LMT_TEXNODES_H
7
8/*tex
9
10    We can probably ditch |volatile| so that the compiler can optimize access a bit better. We
11    only need to make sure that we create nodes before we use their pointers. So, beware: a
12    newnode has to go via an intermediate variable because the |varmem| array can have been be
13    reallocated. I need to (re)check all cases! In case of a copy we use a intermediate volatile
14    variable.
15
16    Anyway, we now have only a few |quarterwords| in use, most noticeably the type and subtype.
17    Eventually I might go for a consistent
18
19    type      subtype
20    prev      next
21    attribute data
22    etc
23
24    model. Or maybe even just a flat list, no need for memoryword, but just all halfwords. However,
25    it will demand all kind of tiny adaptations and we don't gain much. We'd also loose some charm
26    of traditional \TEX. Also, we now have a double glue related field and that would then become
27    a float. So, not now.
28
29    There are a few more node types than in standard \TEX, but less than we have in e.g.\ \PDFTEX\
30    or stock \LUATEX. For instance margin nodes are now just kern nodes, some whatits are first
31    class nodes and we have only one generic whatsit left. We also have more subtypes which makes
32    a more detailed tracking of where nodes come from possible. Other nodes, like the |inserting|
33    and |split_up| nodes are ot both |inserting| but with a subtype because the register index is
34    no longer the subtype.
35
36    Not all nodes can end up in a node list. Some are used for housekeeping (stack, expressions,
37    conditional nesting, etc.) or show up in the process of breaking paragraphs into lines. When
38    we talk of nodes with users in the perspective of \TEX\ we normally refer to the ones in
39    horizontal and vertical lists or math lists, not to those more obscure housekeeping nodes. It
40    just happens that they share the same memory model and management.
41
42    A complication is that some nodes have pointers that themselves point to a (often smaller)
43    node but use the same accessors. This means that (1) their layout should be the same with
44    respect to the pointer, which happens with span nodes or (2) that there is some offset in play,
45    which happens with ins pointers and break nodes that are embedded in a disc node.
46
47    Now that we no longer have variable nodes, we can consider a different allocation model, like
48    a chain of malloced nodes but on the other hand storing them might be more work. We also cannot
49    longer share the accessors so again more work is needed. But ... maybe attributes might end up
50    as allocated lists some day, but that also demands storage. The current memory management is
51    very efficient and we don't gain anything with redoing that, apart maybe from nodes becoming
52    structs. Even then we will have an array of pointers instead of what we have now, but without
53    the jumps by side in the indices. So, given the constraints of offsets and overlap it makes no
54    sense to waste time on this.
55
56    Instead of |var_mem| we use |nodes| and related names. This better complements the additional
57    variables that we have for dynamic management. Some more names have been changed (also in order
58    to avoid side effect in syntax highlighting). Too common names also result in too many matches
59    when searching the source tree.
60
61    Soo, eventually most fields now have the type of the node in their name, which makes it more
62    clear what they are. As mentioned, it makes the syntax highlighted source look better as some
63    generic names are used elsewhere too. Another reason is that we have more fields in nodes and
64    when browsing the source it helps to know that a |width| is actually the |glue_amount| which
65    when we go down is actually a height anyway. It also makes it possible at some point to make
66    some nodes smaller when we don't need these \quote {shared by name} fields. We also need this
67    transition in order to get better interfacing to the \LUA\ end, one reason being that we need
68    to distinguish between fields that overlap (as in lists, unset nodes and and alignment
69    records).
70
71    Todo: all subtype related constants will become |_subtype| so that also means a couple more
72    _code ones for commands. It's all about consistency but that will happen stepwise. A typical
73    rainy day with some newly acquired music in background kind of activity:
74
75    - discretionary
76    - adjust
77    - noad
78    - fence
79    - radical
80    - boundary
81
82    Before we went on with \LUAMETATEX\ already quite a bit was added to \LUATEX\ (granularity in 
83    subtypes, more of them, etc.) but here we add more. Some day I might make a list of difference
84    just for historic reasons. Don't assume equivalence: some in \LUATEX\ was sort of experimental 
85    and improved over time and maybe should not have been added to \LUATEX\ at all and been delayed
86    for \LUAMETATEX\ instead (as it's probably not used), so that core \LUATEX\ stayed closer the 
87    other engines. 
88
89*/
90
91typedef enum node_types {
92    hlist_node,
93    vlist_node,
94    rule_node,
95    insert_node,
96    mark_node,
97    adjust_node,
98    boundary_node,
99    disc_node,
100    whatsit_node,
101    /*tex The last_preceding_break_node: */
102    par_node,
103    dir_node,
104    /*tex The last_non_discardable_node: */
105    math_node,
106    glue_node,
107    kern_node,
108    penalty_node,
109    style_node,
110    choice_node,
111    parameter_node,
112    simple_noad,
113    radical_noad,
114    fraction_noad,
115    accent_noad,
116    fence_noad,
117    math_char_node,      
118    math_text_char_node,
119    sub_box_node,
120    sub_mlist_node,
121    delimiter_node,
122    glyph_node,
123    /*tex This was the last node with attributes, except unset nodes. */
124    unset_node,
125    specification_node,
126    align_record_node,
127    attribute_node,
128    glue_spec_node,
129    temp_node,
130    split_node,
131    /*tex The next set of nodes is invisible from the \LUA\ (but nesting nodes can show up). */
132    expression_node,
133    loop_state_node,
134    math_spec_node,
135    font_spec_node,
136    nesting_node,
137    span_node,
138    align_stack_node,
139 // noad_state_node,
140    if_node,
141    /*tex These two are active nodes. */
142    unhyphenated_node, 
143    hyphenated_node,   
144    /*tex This one can also be in the active list. */
145    delta_node,
146    /*tex While this is an indirect one carrying data. */
147    passive_node,
148} node_types;
149
150 /*tex The largest node is now |par_node| which need 30 slots. */
151
152# define max_chain_size  32
153
154# define unknown_node_type    -1
155# define unknown_node_subtype -1
156
157/* Todo: [type] [subtype|size] [index] -> nodes : advantage is no holes in node id's */
158
159typedef struct node_memory_state_info {
160    memoryword  *nodes;
161 // memoryword  *volatile nodes;
162    char        *nodesizes;
163    halfword     free_chain[max_chain_size];
164    memory_data  nodes_data;
165    memory_data  extra_data;
166    int          reserved; /*tex There are some predefined nodes. */
167    int          padding;
168    int          node_properties_id;
169    int          lua_properties_level;
170    halfword     attribute_cache;
171    halfword     max_used_attribute;
172    int          node_properties_table_size;
173} node_memory_state_info;
174
175extern node_memory_state_info lmt_node_memory_state;
176
177typedef enum field_types {
178    nil_field,
179    integer_field,
180    dimension_field,
181    glue_field,
182    number_field,
183    string_field,
184    boolean_field,
185    function_field,
186    node_field,
187    node_list_field,
188    token_field,
189    token_list_field,
190    attribute_field,
191} field_types;
192
193extern halfword tex_get_node            (int size);
194extern void     tex_free_node           (halfword p, int size);
195extern void     tex_dump_node_mem       (dumpstream f);
196extern void     tex_undump_node_mem     (dumpstream f);
197extern int      tex_used_node_count     (void);
198extern int      tex_free_node_count     (void);
199extern void     tex_initialize_node_mem (void);
200extern void     tex_initialize_nodes    (void);
201
202extern void     lmt_nodelib_initialize  (void); /* name ? */
203
204/*tex
205
206    Most fields are integers (halfwords) that get aliased to |vinfo| and |vlink| for traditional
207    reasons. The |vlink| name is actually representing a next pointer. Only the type and subtype
208    remain quarterwords, the rest are just halfwords which wastes space for directions,
209    orientation, glue orders and glue signs but so be it.
210
211    A memory word has two 32 bit integers so 8 bytes. A glueratio is a double which is 8 bytes so
212    there we waste some space. There is actually no need now to pack (node) data in pairs so maybe
213    some day I'll change that. When we make glue ration a float again we can go flat (and with most
214    node fields now being fully qualified that is easier).
215
216    The first memoryword contains the |type| and |subtype| that are both 16 bit integers (unsigned)
217    as well as the |vlink| (next) pointer. After that comes the word that keeps the |attr| and
218    |alink| (prev) fields. Instead of the link names we use more meaningful ones. The |next|, |prev|
219    and |attr| fields all are halfwords representing a node index.
220
221    The |node_size| field is used in managing freed nodes (mostly as a check) and it overwrites the
222    |type| and |subtype| fields. Actually we could just use the type or subtype but as the size is
223    small but on the other hand using an int here makes sense.
224
225    half0 | quart0 quart1 | vinfo | size | type subtype
226    half1 |               | vlink
227
228    The |tlink| and |rlink| fields are used in disc nodes as tail and replace pointers (again
229    halfwords). We no longer need |rlink| as it's equivalent to |alink| (the prev pointer). The
230    |tlink| fields is used for links to the tail of a list. These indirect macros are somewhat
231    complicating matters. Again these have been renamed.
232
233    We used to have |alink(a)| being |vlink(a,1)| but that has (after a few years) been replaced by
234    |node_prev| because is cleaner. Keep in mind that in \LUATEX\ we use double linked node lists. So,
235    we now only have some \quote {hard coded} pointers to the memory array in this file, not in the
236    files that use the node fields. However, for the next two paragraphs to be true, I need to find
237    a solution for the insert_ptr first because that is an index.
238
239    Now, a logical question is: should we stick to the link and info model for nodes? One reason
240    is that we share the model with tokens. A less relevant reason is that the glue is stored in 8
241    bytes but that can be reverted to 4 bytes if needed. So, indeed at some point we might see a 32
242    bit wide array show up here as we're now more or less prepared for that. It will bump the direct
243    node numbers but that should work out okay. So, in the end, after stepwise abstraction we now
244    have field definitions that use a base and offset e.g. |vlink(a,3)| instead of |vlink(a+3)|.
245    Also, we have many more fields and using meaningful names quickly started to make sense.
246
247    Once all is stable I will play with |var_mem| being an array of pointers and |malloc| the
248    smaller memoryword arrays (per node). This might lead to (not always) smaller memory footprint:
249    we have one pointer per node (but only that array gets preallocated) but we need less memory in
250    total, unless we use many nodes. Anyway, we keep the  indirect model (which might add overhead,
251    but that can be compensated by \CPU\ caches) because using a numeric node pointer is more
252    efficient and quite handy. If we would go completely struct the source would change so much that
253    we loose the charm of \TEX\ and documentation and there is no gain in it. Also, using halfword
254    indices (but then to pointers) for nodes has the huge advantage that it is fast in \LUA\ (always
255    a bottleneck) and these node indices can (and have to) be stored in tokens. One nice side effect
256    would be that we have node indices in a sequence (without the current jumps due to node size
257    offset, which in turn gives more room for nodes references in tokens).
258
259    In spite of all extensions we hope the spirit of how \TEX\ does it is still very visible.
260
261*/
262
263# define mvalue(a,b)  lmt_node_memory_state.nodes[a+b].P
264# define lvalue(a,b)  lmt_node_memory_state.nodes[a+b].L
265# define dvalue(a,b)  lmt_node_memory_state.nodes[a+b].D
266
267# define vinfo(a,b)   lmt_node_memory_state.nodes[a+b].half0
268# define vlink(a,b)   lmt_node_memory_state.nodes[a+b].half1
269
270# define vinfo0(a,b)  lmt_node_memory_state.nodes[a+b].quart00
271# define vinfo1(a,b)  lmt_node_memory_state.nodes[a+b].quart01
272# define vlink0(a,b)  lmt_node_memory_state.nodes[a+b].quart10
273# define vlink1(a,b)  lmt_node_memory_state.nodes[a+b].quart11
274
275# define vinfo00(a,b) lmt_node_memory_state.nodes[a+b].single00
276# define vinfo01(a,b) lmt_node_memory_state.nodes[a+b].single01
277# define vinfo02(a,b) lmt_node_memory_state.nodes[a+b].single02
278# define vinfo03(a,b) lmt_node_memory_state.nodes[a+b].single03
279# define vlink00(a,b) lmt_node_memory_state.nodes[a+b].single10
280# define vlink01(a,b) lmt_node_memory_state.nodes[a+b].single11
281# define vlink02(a,b) lmt_node_memory_state.nodes[a+b].single12
282# define vlink03(a,b) lmt_node_memory_state.nodes[a+b].single13
283
284/*tex
285    We have some shared field names. Some day the subtypes will get meaningful names dependent on
286    the node type, if only because some already have. We used to have
287
288    \starttyping
289    # define type(a)         vinfo0(a,0)
290    # define subtype(a)      vinfo1(a,0)
291    # define node_size(a)    vinfo(a,0)
292    \stoptyping
293
294    but we dropped the size mechanism and made most field shortcuts verbose in order to be able to
295    use variable names with the same name combined with proper syntax highlighting etc. It also
296    gives less noise when we search in the whole source tree. More later.
297*/
298
299# define node_type(a)    vinfo0(a,0)
300# define node_subtype(a) vinfo1(a,0)
301
302# define node_next(a) vlink(a,0)
303# define node_prev(a) vlink(a,1)
304# define node_attr(a) vinfo(a,1)
305
306# define node_head(a) vlink(a,0) /*tex the head |hlink(a)| aka |vlink(a)| of a disc sublist */
307# define node_tail(a) vinfo(a,1) /*tex the tail |tlink(a)| aka |vinfo(a)|, overlaps with |node_attr()| */
308
309/*tex
310
311    The dimension fields shared their locations which made for sometimes more compact code but
312    in the end the number of placxes where it really saved code were limited. Also, compilers will
313    do their job and deal with common code. So, these are now replaced by more meaningful names:
314
315    \starttyping
316    # define width(a)  vlink(a,2)
317    # define depth(a)  vlink(a,3)
318    # define height(a) vlink(a,4)
319    \stoptyping
320
321    Inserts use a trick. The insert pointers directly point into a node at the place where the list
322    starts which is why |list_ptr| has to overlap with |node_next|! I have looked into changign this
323    but it doesn't pay off and it's best to stay close to the original. A side effect is that some
324    fields in insert nodes are sort of impossible (for now).
325
326    \starttyping
327    # define box_list_ptr(a) vlink(a,5) // We need to use the same field as |node_next|.
328    # define insert_list(a) (a + 5) // This kind of makes a virtual node: start at list.
329    \stoptyping
330
331    Beware of the fact that for instance alignments use some fields for other purposes, like:
332    |u_part(a)|, |v_part(a)|, |span_ptr(a)|, etc. and assume the rest of the fields to overlap
333    with list nodes. So, we cannot simply reshuffle positions!
334
335    In the original \TEX\ source (and also in \LUATEX) there are a couple of offsets used. Most
336    noticeably is the |list_offset| but in 2.0618 the related trickery was replaced by using
337    |list_ptr| and using the fact that we have a doubel linked list. The four fields are in
338    successive memory words and that means that we can use |node_next| in a field pointed to
339    by |list_offset| (because actually we then have the list pointer!). This makes for simple
340    loops in original \TEX. The dimension offsets are used to set fields in boxed but we already
341    abstracted that to proper field names; these were for instance used in alignment nodes that
342    have mostly the same properties as a box node.
343
344    \starttyping
345    # define width_offset  2
346    # define depth_offset  3
347    # define height_offset 4
348    # define list_offset   5
349    \stoptyping
350
351    These abstractions mean that we now have nodes, fields and offsets all abstracted in such a way
352    that all definitions and trickery in in this file. Of course I could have messed up.
353
354*/
355
356/*tex
357
358    Syntex supports demands some extra fields in nodes that makes it possible to output location as
359    well as file/line information for viewer-editor synchronization. The ideas is quite okay but
360    unfortunately the implementation of the library is rather bound to the way e.g. \LATEX\ typesets
361    documents. Synctex has always been problematic when it comes to \CONTEXT. There is for instance
362    no control over filenames and discussions around some limitations (and possible features) in the
363    \PDFTEX\ and early \LUATEX\ times never resulted in fixing that (setting filenames, adding some
364    additional synchronization points, etc). All that was supposed to happen deep down in the library
365    and was not considered to be dealt with by a macro package. For instance multiple loading of the
366    same file (metapost runs or smaple files) was a problem as was the need to block access to files
367    in tds (like styles). We also needed binding to for instance elements in an \XML\ file where line
368    numbers are sort of special and out of sync with inclusion. I guess we were ahead of the pack
369    because after nearly two decades of \LUATEX\ there is some discussion about this.
370
371    Anyway, for the reasons mentioned \LUATEX\ offers some setters that overload the engine ones and
372    that permits \CONTEXT\ to implement its own variant. However, in \LUAMETATEX\ setting tags and
373    lines from \LUA\ is now the only way to support \SYNCTEX\ because the library is absent: we just
374    have some extra fields in some nodes. In \LUAMETATEX\ only glyph and list nodes have these fields
375    as it makes no sense to have them elsewhere: macro packages can add glue and kerns and rules and
376    \unknown\ all over the place and adding file state info there only makes things confusing and
377    working less well. This is what the mode parameter can handle in \LUATEX\ and in \LUAMETATEX\ it
378    only supports the modes 1 and 3.
379
380    As a side note: the fact that a viewer needs to embed the library is also a limitation. Calling
381    out to an external program that analyzes the file and gives back the filename and line is more
382    flexible and robust. Because we have such an analyzer in \MKIV\ it was no big deal to add a few
383    lines so that the \TEX shop environment could use that script/method (bidirectional); hopefully
384    other viewers and editors will follow.
385
386    So, compared to \LUATEX\ less nodes have the extra fields (which saves memory) and therefore
387    less has to be set. Because there is no library at all, writing a synctex file is up to some
388    additional \LUA\ code, but that was already the case in \MKIV\ anyway. We might at some point
389    change the field names to \quote {file} and \quote {line} and remove interface options that
390    have no use any more. We also moved to a more generic naming of (input related) fields.
391
392*/
393
394/*
395
396    Temporary nodes are really special head node pointers that only need links. They ensure that
397    there is at least one node in a list.
398
399*/
400
401# define temp_node_size 2
402
403/*tex
404
405    In \LUATEX\ we have attribute list nodes and attribute nodes that are (anyway) of the same
406    size. In the end I decided to combine them into one node with a subtype. That also helps
407    diagnose issues. It is one of the few nodes now that has fields depending on the subtype
408    but these nodes are not really user ones anyway.
409
410    We could move index to the subtype and then have an extra data field but I'll delay that till
411    it's really needed. Chains are often short and using arrays doesn't bring any gain nor does 
412    using some btree approach. 
413
414*/
415
416# define attribute_node_size 2
417# define attribute_unset(a)  vinfo(a,1)
418# define attribute_index(a)  vinfo0(a,1) 
419# define attribute_detail(a) vinfo1(a,1) 
420# define attribute_count(a)  vlink(a,1)  /*tex the reference count */
421# define attribute_value(a)  vlink(a,1)
422
423typedef enum attribute_subtypes {
424    attribute_list_subtype,
425    attribute_value_subtype,
426} attribute_subtypes;
427
428# define last_attribute_subtype attribute_value_subtype
429
430/*tex
431    Penalties have only one primitive so we don't have |_code| here, also because it would conflict
432    with arguments.
433*/
434
435# define penalty_node_size  3
436# define penalty_amount(a)  vlink(a,2)
437# define penalty_options(a) vinfo(a,2)
438
439static inline void tex_add_penalty_option    (halfword a, halfword r) { penalty_options(a) |= r; }
440static inline void tex_remove_penalty_option (halfword a, halfword r) { penalty_options(a) &= ~(r | penalty_options(a)); }
441static inline int  tex_has_penalty_option    (halfword a, halfword r) { return (penalty_options(a) & r) == r; }
442
443typedef enum penalty_option_codes {
444    penalty_option_normal        = 0x0000,
445    penalty_option_math_forward  = 0x0001,
446    penalty_option_math_backward = 0x0002,
447    penalty_option_orphaned      = 0x0004,
448    penalty_option_widowed       = 0x0008,
449    penalty_option_clubbed       = 0x0010,
450} penalty_option_codes; 
451
452typedef enum penalty_subtypes {
453    user_penalty_subtype,
454    linebreak_penalty_subtype, /*tex includes widow, club, broken etc. */
455    line_penalty_subtype,
456    word_penalty_subtype,
457    orphan_penalty_subtype,
458    single_line_penalty_subtype,
459    final_penalty_subtype,
460    math_pre_penalty_subtype,
461    math_post_penalty_subtype,
462    before_display_penalty_subtype,
463    after_display_penalty_subtype,
464    equation_number_penalty_subtype,
465} penalty_subtypes;
466
467# define last_penalty_subtype equation_number_penalty_subtype
468
469/*tex
470    We have plenty of glue variables and in the node lists most are also flagged. There is no
471    one|-|to|-|one correspondence between the codes (in tokens) and subtypes (in nodes) as listed
472    below, but they come close. The special math related glues and inserts now have nicer numbers.
473*/
474
475typedef enum glue_subtypes {
476    user_skip_glue,
477    line_skip_glue,
478    baseline_skip_glue,
479    par_skip_glue,
480    above_display_skip_glue,
481    below_display_skip_glue,
482    above_display_short_skip_glue,
483    below_display_short_skip_glue,
484    left_skip_glue,
485    right_skip_glue,
486    top_skip_glue,
487    split_top_skip_glue,
488    tab_skip_glue,
489    space_skip_glue,
490    xspace_skip_glue,
491    zero_space_skip_glue,
492    par_fill_left_skip_glue,
493    par_fill_right_skip_glue,
494    par_init_left_skip_glue,
495    par_init_right_skip_glue,
496    indent_skip_glue,
497    left_hang_skip_glue,
498    right_hang_skip_glue,
499    correction_skip_glue,
500    inter_math_skip_glue,
501    ignored_glue,           /*tex |subtype| for cases where we ignore zero glue (alignments) */
502    page_glue,              /*tex |subtype| used in the page builder */
503    /*tex math */
504    math_skip_glue,
505    thin_mu_skip_glue,
506    med_mu_skip_glue,
507    thick_mu_skip_glue,
508    /*tex more math */
509    conditional_math_glue,  /*tex special |subtype| to suppress glue in the next node */ /* no need for jump */
510    rulebased_math_glue,
511    mu_glue,                /*tex |subtype| for math glue */
512    /*tex leaders (glue with list) */
513    a_leaders,              /*tex |subtype| for aligned leaders */
514    c_leaders,              /*tex |subtype| for centered leaders */
515    x_leaders,              /*tex |subtype| for expanded leaders */
516    g_leaders,              /*tex |subtype| for global (page) leaders */
517    u_leaders,
518} glue_subtypes;
519
520# define last_glue_subtype u_leaders
521
522typedef enum skip_glue_codes_alias {
523    par_fill_skip_glue = par_fill_right_skip_glue,
524} skip_glue_codes_alias;
525
526# define is_leader(a) (node_subtype(a) >= a_leaders)
527
528# define glue_node_size        7
529# define glue_spec_size        5
530# define glue_options(a)       vinfo(a,2)
531# define glue_amount(a)        vlink(a,2)
532# define glue_shrink(a)        vinfo(a,3)
533# define glue_stretch(a)       vlink(a,3)
534# define glue_stretch_order(a) vinfo(a,4)
535# define glue_shrink_order(a)  vlink(a,4)
536# define glue_font(a)          vinfo(a,5) /* not in spec */ /* when inter_math_skip_glue: parameter */
537# define glue_data(a)          vlink(a,5) /* not in spec */
538# define glue_leader_ptr(a)    vinfo(a,6) /* not in spec */
539# define glue_callback(a)      vlink(a,6) /* not in spec */
540
541static inline void tex_add_glue_option    (halfword a, halfword r) { glue_options(a) |= r; }
542static inline void tex_remove_glue_option (halfword a, halfword r) { glue_options(a) &= ~(r | glue_options(a)); }
543static inline int  tex_has_glue_option    (halfword a, halfword r) { return (glue_options(a) & r) == r; }
544
545typedef enum glue_option_codes {
546    glue_option_normal        = 0x0000,
547    glue_option_no_auto_break = 0x0001,
548    glue_option_has_factor    = 0x0002,
549    glue_option_is_limited    = 0x0004,
550    glue_option_limit         = 0x0008,
551} glue_option_codes; 
552
553typedef enum math_subtypes {
554    begin_inline_math,
555    end_inline_math
556} math_subtypes;
557
558# define last_math_subtype end_inline_math
559
560/*tex
561    Math nodes (currently) partially overlap with glue because they also have a glue property.
562*/
563
564# define math_node_size        7
565# define math_surround(a)      vinfo(a,2)
566# define math_amount(a)        vlink(a,2)
567# define math_shrink(a)        vinfo(a,3)
568# define math_stretch(a)       vlink(a,3)
569# define math_stretch_order(a) vinfo(a,4)
570# define math_shrink_order(a)  vlink(a,4)
571# define math_penalty(a)       vinfo(a,5)
572# define math_options(a)       vlink(a,5) 
573# define math_tolerance(a)     vinfo(a,6) 
574# define math_pre_tolerance(a) vlink(a,6) 
575
576static inline void tex_add_math_option    (halfword a, halfword r) { math_options(a) |= r; }
577static inline void tex_remove_math_option (halfword a, halfword r) { math_options(a) &= ~(r | math_options(a)); }
578static inline int  tex_has_math_option    (halfword a, halfword r) { return (math_options(a) & r) == r; }
579
580typedef enum math_option_codes {
581    math_option_normal   = 0x0000,
582    math_option_short    = 0x0001,
583    math_option_orphaned = 0x0002,
584} math_option_codes; 
585
586/*tex Here are some (inline) helpers. We need specific ones for math glue. */
587
588static inline int tex_glue_is_zero(halfword g)
589{
590    return (! g) || ((glue_amount(g) == 0) && (glue_stretch(g) == 0) && (glue_shrink(g) == 0));
591}
592
593static inline int tex_math_glue_is_zero(halfword g)
594{
595    return (! g) || ((math_amount(g) == 0) && (math_stretch(g) == 0) && (math_shrink(g) == 0));
596}
597
598static inline int tex_same_glue(halfword a, halfword b)
599{
600    return
601        (a == b) /* same glue specs or both zero */
602     || (a && b && glue_amount(a)        == glue_amount(b)
603                && glue_stretch(a)       == glue_stretch(b)
604                && glue_shrink(a)        == glue_shrink(b)
605                && glue_stretch_order(a) == glue_stretch_order(b)
606                && glue_shrink_order(a)  == glue_shrink_order(b)
607        )
608    ;
609}
610
611static inline void tex_reset_glue_to_zero(halfword target)
612{
613    if (target) {
614        glue_amount(target) = 0;
615        glue_stretch(target) = 0;
616        glue_shrink(target) = 0;
617        glue_stretch_order(target) = 0;
618        glue_shrink_order(target) = 0;
619    }
620}
621
622static inline void tex_reset_math_glue_to_zero(halfword target)
623{
624    if (target) {
625        math_amount(target) = 0;
626        math_stretch(target) = 0;
627        math_shrink(target) = 0;
628        math_stretch_order(target) = 0;
629        math_shrink_order(target) = 0;
630    }
631}
632
633static inline void tex_copy_glue_values(halfword target, halfword source)
634{
635    if (source) {
636        glue_amount(target) = glue_amount(source);
637        glue_stretch(target) = glue_stretch(source);
638        glue_shrink(target) = glue_shrink(source);
639        glue_stretch_order(target) = glue_stretch_order(source);
640        glue_shrink_order(target) = glue_shrink_order(source);
641    } else {
642        glue_amount(target) = 0;
643        glue_stretch(target) = 0;
644        glue_shrink(target) = 0;
645        glue_stretch_order(target) = 0;
646        glue_shrink_order(target) = 0;
647    }
648}
649
650static inline int tex_is_par_init_glue(halfword n)
651{
652    switch (node_subtype(n)) {
653        case indent_skip_glue:
654        case par_init_left_skip_glue:
655        case par_init_right_skip_glue:
656            return 1;
657        default:
658            return 0;
659    }
660}
661
662/*tex
663    Kern nodes are relatively simple. Instead of |width| we use |kern_amount| which  makes more
664    sense: we can go left, right, up or down. Margin kerns have been dropped and are now just a
665    special subtype of regular kerns.
666*/
667
668typedef enum kern_subtypes {
669    explicit_kern_subtype,      /*tex from |\kern| */
670    accent_kern_subtype,        /*tex from accents */
671    font_kern_subtype,
672    italic_kern_subtype,        /*tex from |\/| */
673    left_margin_kern_subtype,
674    right_margin_kern_subtype,
675    left_correction_kern_subtype,
676    right_correction_kern_subtype,
677    space_font_kern_subtype,    /*tex for tracing only */
678    explicit_math_kern_subtype,
679    math_shape_kern_subtype,
680    horizontal_math_kern_subtype,
681    vertical_math_kern_subtype,
682} kern_subtypes;
683
684# define last_kern_subtype vertical_math_kern_subtype
685
686# define font_related_kern(s) (s >= font_kern_subtype && s <= space_font_kern_subtype) 
687
688# define kern_node_size       3
689# define kern_amount(a)       vlink(a,2) /*tex aka |width = vlink(a,2)| */
690# define kern_expansion(a)    vinfo(a,2) /*tex expansion factor (hz) */
691
692/*tex
693
694    Disc nodes are complicated: they have three embedded nesting nodes to which the |pre_break|,
695    |post_break| and |no_break| fields point. In there we find a head pointer (|vlink| aka |hlink|)
696    and tail pointer (|tlink|). The |alink| pointer is used in the base mode font machinery and is
697    not really a prev pointer. We have to make sure it gets nilled when we communicate with \LUA.
698
699    The no-, pre-, and postbreak fields point to nesting nodes that are part of the disc node (three
700    times two memorywords). Sometimes these nodes are actually used, for instance when a temp node
701    is expected at the head of a list. The layout is:
702
703    \starttyping
704    [ type+subtype + next      ]
705    [ attr         + prev      ]
706    [ penalty      + nobreak   ]
707    [ prebreak     + postbreak ]
708    [ type+subtype next/hlink  ] (nesting node prebreak)
709    [ tlink        prev        ]
710    [ type+subtype next/hlink  ] (nesting node postbreak)
711    [ tlink        prev        ]
712    [ type+subtype next/hlink  ] (nesting node nobreak)
713    [ tlink        prev        ]
714    \stoptyping
715
716    Another reason why we need the indirect apoproach is that we can set the fields to |null| which
717    is better than point to a nest node with no following up.
718
719*/
720
721/*tex
722    Among the dropped nodes (\LUATEX\ has them) are movements nodes (used in the \DVI\ backend)
723    and variable nodes (replaced by specification nodes).
724
725    Nesting nodes are really simple and just use the common type, subtype and next fields so they
726    have no dedicated fields. They can be part of another node type (like disc nodes).
727*/
728
729# define nesting_node_size 2
730
731typedef enum nesting_subtypes {
732    pre_break_code,
733    post_break_code,
734    no_break_code,
735    insert_head_code,
736    unset_nesting_code,
737} nesting_subtypes;
738
739# define last_nesting_subtype unset_nesting_code
740
741/*tex Here the codes in commands and subtypes are in sync. */
742
743typedef enum discretionary_subtypes {
744    normal_discretionary_code,
745    explicit_discretionary_code,
746    automatic_discretionary_code,
747    mathematics_discretionary_code,
748    syllable_discretionary_code,
749} discretionary_subtypes;
750
751# define last_discretionary_subtype syllable_discretionary_code
752# define last_discretionary_code    automatic_discretionary_code
753
754typedef enum disc_options {
755    disc_option_normal_word               = 0x00000000,
756    disc_option_pre_word                  = 0x00000001,
757    disc_option_post_word                 = 0x00000002,
758    disc_option_orphaned                  = 0x00000004,
759    disc_option_prefer_break              = 0x00000010,
760    disc_option_prefer_nobreak            = 0x00000020,
761    /* */
762    disc_option_no_italic_correction      = 0x00000040,
763    disc_option_no_zero_italic_correction = 0x00000080,
764    /*tex We permit user options. */
765    disc_option_user_first                = 0x00010000,
766    disc_option_user_last                 = 0x40000000,
767    /*tex So watch out: these are subsets! */
768    disc_option_valid                     = 0x4FFF0FFF,
769} disc_options;
770
771# define first_disc_option disc_option_prefer_break
772# define last_disc_option  disc_option_prefer_nobreak
773
774# define disc_node_size       13
775# define disc_no_break(a)     vlink(a,2) /* beware: vinfo is used for type/subtype */
776# define disc_pre_break(a)    vlink(a,3) /* beware: vinfo is used for type/subtype */
777# define disc_post_break(a)   vlink(a,4) /* beware: vinfo is used for type/subtype */
778/*       disc_no_break_node   5  6 */    /* this is a nesting node of size 2 */
779/*       disc_pre_break_node  7  8 */    /* this is a nesting node of size 2 */
780/*       disc_post_break_node 9 10 */    /* this is a nesting node of size 2 */
781# define disc_penalty(a)      vinfo(a,11)
782# define disc_options(a)      vlink(a,11)
783# define disc_class(a)        vinfo(a,12)
784# define disc_orphaned(a)     vlink(a,12)
785
786# define set_disc_penalty(a,b) disc_penalty(a) = b
787# define set_disc_class(a,b)   disc_class(a) = b
788# define set_disc_options(a,b) disc_options(a) = b
789# define set_disc_option(a,b)  disc_options(a) |= b
790
791# define has_disc_option(a,b) ((disc_options(a) & b) == b)
792
793# define unset_disc_class -1
794
795static inline void tex_add_disc_option    (halfword a, halfword r) { disc_options(a) |= r; }
796static inline void tex_remove_disc_option (halfword a, halfword r) { disc_options(a) &= ~(r | disc_options(a)); }
797static inline int  tex_has_disc_option    (halfword a, halfword r) { return (disc_options(a) & r) == r; }
798
799/*tex
800    These are pseudo nodes inside a node. We used to reference them by |*_break_head| but now call
801    just call them nodes so that we can use head and tail instead of hlink and tlink.
802*/
803
804# define disc_pre_break_node(a)   (a+5)
805# define disc_post_break_node(a)  (a+7)
806# define disc_no_break_node(a)    (a+9)
807
808# define disc_pre_break_head(a)  node_head(disc_pre_break_node(a))
809# define disc_post_break_head(a) node_head(disc_post_break_node(a))
810# define disc_no_break_head(a)   node_head(disc_no_break_node(a))
811
812# define disc_pre_break_tail(a)  node_tail(disc_pre_break_node(a))
813# define disc_post_break_tail(a) node_tail(disc_post_break_node(a))
814# define disc_no_break_tail(a)   node_tail(disc_no_break_node(a))
815
816extern void     tex_set_disc_field          (halfword target, halfword location, halfword source);
817extern void     tex_check_disc_field        (halfword target);
818extern void     tex_set_discpart            (halfword d, halfword h, halfword t, halfword code);
819extern halfword tex_flatten_discretionaries (halfword head, int *count, int nest);
820extern int      tex_flatten_leaders         (halfword box, int grp, int just_pack);
821extern void     tex_soften_hyphens          (halfword head, int *found, int *replaced);
822extern halfword tex_harden_spaces           (halfword head, halfword tolerance, int *count);
823
824/*tex
825    Lists need a rather large node, also because the have quite some extra possibilities, like the
826    orientation features. We can put the dir with the orientation but it becomes messy in casting
827    that way. Also, memory is not really a constraint and for a cpu cache we're better off this
828    way.
829
830    In the original setup the unset and align_record nodes have overlapping fields. This has the
831    side effect that when we access the alternates from \LUA\ that they can have weird values
832    unless we reset them. Even then, it can be that we actually want to use those other fields
833    somehow. For that reason it's better to waste a few more slots and play safe. We can now
834    actually explore table cells with offsets if we want.
835
836    Beware: in alignments
837
838    \startitemize[packed]
839    \startitem align record nodes become unset nodes \stopitem
840    \startitem unset nodes become hlist or vlist nodes \stopitem
841    \stopitemize
842*/
843
844typedef enum list_subtypes {
845    unknown_list,
846    line_list,                 /*tex paragraph lines */
847    hbox_list,                 /*tex |\hbox| */
848    indent_list,               /*tex indentation box */
849    container_list,            /*tex container box */
850    align_row_list,            /*tex row from a |\halign| or |\valign| */
851    align_cell_list,           /*tex cell from a |\halign| or |\valign| */
852    equation_list,             /*tex display equation */
853    equation_number_list,      /*tex display equation number */
854    math_list_list,
855    math_char_list,
856    math_pack_list,
857    math_h_extensible_list,
858    math_v_extensible_list,
859    math_h_delimiter_list,
860    math_v_delimiter_list,
861    math_over_delimiter_list,
862    math_under_delimiter_list,
863    math_numerator_list,
864    math_denominator_list,
865    math_modifier_list,
866    math_fraction_list,
867    math_nucleus_list,
868    math_sup_list,
869    math_sub_list,
870    math_pre_post_list,
871    math_degree_list,
872    math_scripts_list,
873    math_over_list,
874    math_under_list,
875    math_accent_list,
876    math_radical_list,
877    math_fence_list,
878    math_rule_list,
879    math_ghost_list,
880    insert_result_list,
881    local_list,
882    local_left_list,
883    local_right_list,
884    local_middle_list,
885} list_subtypes ;
886
887# define last_list_subtype     local_middle_list
888# define noad_class_list_base  0x0100
889
890typedef enum list_anchors {
891    left_origin_anchor    = 0x001,
892    left_height_anchor    = 0x002,
893    left_depth_anchor     = 0x003,
894    right_origin_anchor   = 0x004,
895    right_height_anchor   = 0x005,
896    right_depth_anchor    = 0x006,
897    center_origin_anchor  = 0x007,
898    center_height_anchor  = 0x008,
899    center_depth_anchor   = 0x009,
900    halfway_total_anchor  = 0x00A,
901    halfway_height_anchor = 0x00B,
902    halfway_depth_anchor  = 0x00C,
903    halfway_left_anchor   = 0x00D,
904    halfway_right_anchor  = 0x00E,
905} list_anchors;
906
907typedef enum list_signs {
908    negate_x_anchor = 0x100,
909    negate_y_anchor = 0x200,
910} list_signs;
911
912typedef enum list_geometries {
913    no_geometry          = 0x0,
914    offset_geometry      = 0x1,
915    orientation_geometry = 0x2,
916    anchor_geometry      = 0x4,
917} list_geometries;
918
919# define box_node_size          15
920# define box_width(a)           vlink(a,2)
921# define box_w_offset(a)        vinfo(a,2)
922# define box_depth(a)           vlink(a,3)
923# define box_d_offset(a)        vinfo(a,3)
924# define box_height(a)          vlink(a,4)
925# define box_h_offset(a)        vinfo(a,4)
926# define box_list(a)            vlink(a,5)    /* 5 = list_offset */
927# define box_shift_amount(a)    vinfo(a,5)
928# define box_glue_order(a)      vlink(a,6)    /* quarterword (?) */
929# define box_glue_sign(a)       vinfo(a,6)    /* quarterword (!) */
930//define box_glue_order(a)      vlink00(a,6)  
931//define box_glue_sign(a)       vlink01(a,6)  
932//define box_glue_reserved_1(a) vlink02(a,6)  
933//define box_glue_reserved_2(a) vlink03(a,6)  
934# define box_glue_set(a)        dvalue(a,7)   /* So we reserve a whole memory word! */
935# define box_dir(a)             vlink00(a,8)  /* We could encode it as geomtry but not now. */
936# define box_package_state(a)   vlink01(a,8)
937# define box_axis(a)            vlink02(a,8)
938# define box_geometry(a)        vlink03(a,8)
939# define box_orientation(a)     vinfo(a,8)    /* Also used for size in alignments. */
940# define box_x_offset(a)        vlink(a,9)
941# define box_y_offset(a)        vinfo(a,9)
942# define box_pre_migrated(a)    vlink(a,10)
943# define box_post_migrated(a)   vinfo(a,10)
944# define box_pre_adjusted(a)    vlink(a,11)
945# define box_post_adjusted(a)   vinfo(a,11)
946# define box_source_anchor(a)   vlink(a,12)
947# define box_target_anchor(a)   vinfo(a,12)
948# define box_anchor(a)          vlink(a,13)
949# define box_index(a)           vinfo(a,13)
950# define box_input_file(a)      vlink(a,14) /* aka box_synctex_tag  */
951# define box_input_line(a)      vinfo(a,14) /* aka box_synctex_line */
952
953# define box_total(a) (box_height(a) + box_depth(a)) /* Here we add, with glyphs we maximize. */ 
954
955static inline void tex_set_box_geometry   (halfword b, halfword g) { box_geometry(b) |= (singleword) (g); }
956/*     static void tex_unset_box_geometry (halfword b, halfword g) { box_geometry(b) &= (singleword) ~((singleword) (g) | box_geometry(b)); } */
957static inline void tex_unset_box_geometry (halfword b, halfword g) { box_geometry(b) &= (singleword) (~g); }
958static inline int  tex_has_geometry       (halfword g, halfword f) { return ((singleword) (g) & (singleword) (f)) == (singleword) (f); }
959static inline int  tex_has_box_geometry   (halfword b, halfword g) { return (box_geometry(b) & (singleword) (g)) == (singleword) (g); }
960
961typedef enum box_pre_post_states { 
962    has_pre_adjust    = 0x1,
963    has_post_adjust   = 0x2,
964    has_pre_migrated  = 0x4,
965    has_post_migrated = 0x8,
966} box_pre_post_states;
967
968typedef enum package_states {
969    unknown_package_state = 0x00,
970    hbox_package_state    = 0x01,
971    vbox_package_state    = 0x02,
972    vtop_package_state    = 0x03,/* == 0x01 + 0x02 */
973    dbox_package_state    = 0x04,
974    /* maybe vcenter */
975} package_states;
976
977typedef enum package_dimension_states {
978    package_dimension_not_set  = 0x00,
979    package_dimension_size_set = 0x10, /* used in in alignments */
980} package_dimension_states;
981
982typedef enum package_leader_states {
983    package_u_leader_not_set  = 0x00,
984    package_u_leader_set      = 0x20,
985    package_u_leader_delayed  = 0x40,
986    package_u_leader_found    = 0x80,
987} package_leader_states;
988
989# define set_box_package_state(p,s) box_package_state(p) |= s
990# define has_box_package_state(p,s) ((box_package_state(p) & s) == s)
991# define is_box_package_state(p,s)  ((p & s) == s)
992
993typedef enum list_axis { /* or maybe math states */
994    no_math_axis = 0x01,
995} list_axis;
996
997# define has_box_axis(p,s) ((box_axis(p) & s) == s)
998# define set_box_axis(p,s) box_axis(p) |= (s & 0xFF)
999
1000/*tex
1001    These |unset| nodes have the same layout as list nodes and at some point become an |hlist| or
1002    |vlist| node.
1003*/
1004
1005# define unset_node_size     box_node_size
1006# define box_glue_stretch(a) box_w_offset(a)
1007# define box_glue_shrink(a)  box_h_offset(a)
1008# define box_span_count(a)   box_d_offset(a)
1009# define box_size(a)         box_orientation(a)
1010
1011/*tex
1012    The |align record| nodes have the same layout as list nodes and at some point become an |unset|
1013    node.
1014*/
1015
1016# define align_record_size         box_node_size
1017# define align_record_span_ptr(a)  box_w_offset(a)    /*tex A column spanning list */
1018# define align_record_cmd(a)       box_h_offset(a)    /*tex Info to remember during template. */
1019# define align_record_chr(a)       box_d_offset(a)    /*tex Info to remember during template. */
1020# define align_record_pre_part(a)  box_x_offset(a)    /*tex The pointer to |u_j| token list. */
1021# define align_record_post_part(a) box_y_offset(a)    /*tex The pointer to |v_j| token list. */
1022# define align_record_dimension(a) box_orientation(a) /*tex Optionally enforced width. */
1023
1024/*tex
1025   Span nodes are tricky in the sense that their |span_link| actually has to sit in the same slot
1026   as |align_record_span_ptr| because we need the initial location to be the same. This is why we
1027   renamed this field to |span_ptr|. Moving it to another spot than in \LUATEX\ also opens the
1028   possibility for attributes to cells.
1029*/
1030
1031# define span_node_size 3
1032# define span_span(a)   vinfo(a,1)
1033# define span_unused(a) vlink(a,1)
1034# define span_width(a)  vlink(a,2)  /* overlaps with |box_width(a)|. */
1035# define span_ptr(a)    vinfo(a,2)  /* overlaps with |box_w_offset(a)| and align_record_span_ptr(a). */
1036
1037/*tex
1038    Here the subtypes and command codes partly overlay. We actually hav eonly avery few left because
1039    it's mostly a backend feature now.
1040*/
1041
1042typedef enum rule_subtypes {
1043    normal_rule_subtype,
1044    empty_rule_subtype,
1045    strut_rule_subtype,
1046    outline_rule_subtype,
1047    virtual_rule_subtype,
1048    user_rule_subtype,
1049    math_over_rule_subtype,
1050    math_under_rule_subtype,
1051    math_fraction_rule_subtype,
1052    math_radical_rule_subtype,
1053    box_rule_subtype,
1054    image_rule_subtype,
1055} rule_subtypes;
1056
1057typedef enum rule_codes {
1058    normal_rule_code,
1059    empty_rule_code,
1060    virtual_rule_code,
1061    strut_rule_code
1062} rule_codes;
1063
1064# define last_rule_subtype image_rule_subtype
1065# define first_rule_code   normal_rule_code
1066# define last_rule_code    strut_rule_code
1067
1068# define rule_node_size    7
1069# define rule_width(a)     vlink(a,2)
1070# define rule_x_offset(a)  vinfo(a,2)
1071# define rule_depth(a)     vlink(a,3)
1072# define rule_y_offset(a)  vinfo(a,3)
1073# define rule_height(a)    vlink(a,4)
1074# define rule_data(a)      vinfo(a,4)
1075# define rule_left(a)      vinfo(a,5) /* depends on subtype */ 
1076# define rule_right(a)     vlink(a,5) /* depends on subtype */ 
1077# define rule_extra_1(a)   vinfo(a,6) /* depends on subtype */ 
1078# define rule_extra_2(a)   vlink(a,6) /* depends on subtype */ 
1079
1080# define rule_strut_font      rule_extra_1
1081# define rule_strut_character rule_extra_2
1082
1083# define rule_virtual_width   rule_left
1084# define rule_virtual_height  rule_right
1085# define rule_virtual_depth   rule_extra_1
1086# define rule_virtual_unused  rule_extra_2
1087
1088# define rule_total(a) (rule_height(a) + rule_depth(a))
1089
1090/*tex
1091
1092    Originally glyph nodes had a |lig_ptr| but storing components makes not that much sense so we
1093    dropped that. The free slot is now used for a state field. We already had a data field that
1094    took another free slot and that behaves like an attribute. The glyph data field can be set at
1095    the \TEX\ end, the state field is only accessible in \LUA. At the same time we reshuffled the
1096    fields a bit so that the most accessed fields are close together.
1097
1098    The \LUATEX\ engine dropped the language node and moved that feature to the glyph nodes. In
1099    addition to the language more properties could be set but they were all packed into one
1100    halfword. In \LUAMETATEX\ we waste a few more bytes and keep the language separate but we
1101    still pack a few properties.
1102
1103    In \TEX\ we have character nodes and glyph nodes, but here we only have one type. The subtype
1104    can be used to indicate if we have ligatures but in \LUATEX\ for various reasons we don't follow
1105    the integrated approach that \TEX\ has: we have callbacks for hyphenation, ligature building,
1106    kerning etc.\ which demands separation, but more important is that we want to use \LUA\ to deal
1107    with modern fonts. The components field that is still present in \LUATEX\ is gone because it
1108    serves no purpose. We don't need to reassemble and when dealing with \OPENTYPE\ fonts we loose
1109    information in successive steps anyway.
1110
1111    This also makes that the subtype is now only used to flag if glyphs have been processed. The
1112    macro package can decide what additional properties get stored in this field.
1113
1114    We used to have this:
1115
1116    \starttyping
1117    static inline void protect_glyph      (halfword a) { quarterword s = node_subtype(a) ; if (s <= 256) { node_subtype(a) = s == 1 ? 256 : 256 + s; } }
1118    static inline void unprotect_glyph    (halfword a) { quarterword s = node_subtype(a) ; if (s >  256) { node_subtype(a) = s - 256; } }
1119    static inline int  is_protected_glyph (halfword a) { return node_subtype(a) >= 256; }
1120    \stoptyping
1121
1122    These were also dropped:
1123
1124    \starttyping
1125    # define is_character(p)        (((node_subtype(p)) & glyph_character) == glyph_character)
1126    # define is_ligature(p)         (((node_subtype(p)) & glyph_ligature ) == glyph_ligature )
1127    # define is_simple_character(p) (is_character(p) && ! is_ligature(p))
1128    # define set_is_glyph(p)         node_subtype(p) = (quarterword) (node_subtype(p) & ~glyph_character)
1129    \stoptyping
1130
1131*/
1132
1133/*tex
1134
1135    Putting |width|, |height| and |depth| in a glyph has some advantages, for instance when we
1136    fetch them in the builder, packer, \LUA\ interface, but it also has a disadvantage: we need to
1137    have more complex copying of glyph nodes. For instance, when we copy glyphs in the open type
1138    handler (e.g. for multiples) we also copy the fields. But then when we set a character, we also
1139    would have to set the dimensions. Okay, some helper could do that (or a flag in setchar). It's
1140    anyway not something to do in a hurry. An |x_extra| field is something different: combined with
1141    setting |x_offset| that could replace font kerns: |x_advance = width + x_offset + x_extra|.
1142
1143*/
1144
1145# define glyph_node_size     14
1146# define glyph_character(a)  vinfo(a,2)
1147# define glyph_font(a)       vlink(a,2)
1148# define glyph_data(a)       vinfo(a,3)   /*tex We had that unused, so now it's like an attribute. */
1149# define glyph_state(a)      vlink(a,3)   /*tex A user field (can be handy in \LUA). */
1150# define glyph_language(a)   vinfo(a,4)
1151# define glyph_script(a)     vlink(a,4)
1152# define glyph_options(a)    vinfo(a,5)
1153# define glyph_hyphenate(a)  vlink(a,5)
1154# define glyph_protected(a)  vinfo00(a,6)
1155# define glyph_lhmin(a)      vinfo01(a,6)
1156# define glyph_rhmin(a)      vinfo02(a,6)
1157# define glyph_discpart(a)   vinfo03(a,6)
1158# define glyph_expansion(a)  vlink(a,6)
1159# define glyph_x_scale(a)    vinfo(a,7)
1160# define glyph_y_scale(a)    vlink(a,7)
1161# define glyph_scale(a)      vinfo(a,8)
1162# define glyph_raise(a)      vlink(a,8)
1163# define glyph_left(a)       vinfo(a,9)
1164# define glyph_right(a)      vlink(a,9)
1165# define glyph_x_offset(a)   vinfo(a,10)
1166# define glyph_y_offset(a)   vlink(a,10)
1167# define glyph_properties(a) vinfo0(a,11)
1168# define glyph_group(a)      vinfo1(a,11)
1169# define glyph_index(a)      vlink(a,11)
1170# define glyph_weight(a)     vinfo(a,12)        
1171# define glyph_slant(a)      vlink(a,12)  
1172# define glyph_input_file(a) vinfo(a,13) 
1173# define glyph_input_line(a) vlink(a,13) 
1174
1175# define get_glyph_data(a)      ((halfword) glyph_data(a))
1176# define get_glyph_state(a)     ((halfword) glyph_state(a))
1177# define get_glyph_language(a)  ((halfword) glyph_language(a))
1178# define get_glyph_script(a)    ((halfword) glyph_script(a))
1179# define get_glyph_x_scale(a)   ((halfword) glyph_x_scale(a))
1180# define get_glyph_y_scale(a)   ((halfword) glyph_y_scale(a))
1181# define get_glyph_scale(a)     ((halfword) glyph_scale(a))
1182# define get_glyph_raise(a)     ((halfword) glyph_raise(a))
1183# define get_glyph_lhmin(a)     ((halfword) glyph_lhmin(a))
1184# define get_glyph_rhmin(a)     ((halfword) glyph_rhmin(a))
1185# define get_glyph_left(a)      ((halfword) glyph_left(a))
1186# define get_glyph_right(a)     ((halfword) glyph_right(a))
1187# define get_glyph_hyphenate(a) ((halfword) glyph_hyphenate(a))
1188# define get_glyph_options(a)   ((halfword) glyph_options(a))
1189# define get_glyph_dohyph(a)    (hyphenation_permitted(glyph_hyphenate(a), syllable_hyphenation_mode ) || hyphenation_permitted(glyph_hyphenate(a), force_handler_hyphenation_mode))
1190# define get_glyph_uchyph(a)    (hyphenation_permitted(glyph_hyphenate(a), uppercase_hyphenation_mode) || hyphenation_permitted(glyph_hyphenate(a), force_handler_hyphenation_mode))
1191
1192# define set_glyph_data(a,b)      glyph_data(a) = b
1193# define set_glyph_state(a,b)     glyph_state(a) = b
1194# define set_glyph_language(a,b)  glyph_language(a) = b
1195# define set_glyph_script(a,b)    glyph_script(a) = b
1196# define set_glyph_x_scale(a,b)   glyph_x_scale(a) = b
1197# define set_glyph_y_scale(a,b)   glyph_y_scale(a) = b
1198# define set_glyph_x_offset(a,b)  glyph_x_offset(a) = b
1199# define set_glyph_y_offset(a,b)  glyph_y_offset(a) = b
1200# define set_glyph_slant(a,b)     glyph_slant(a) = b
1201# define set_glyph_weight(a,b)    glyph_weight(a) = b
1202# define set_glyph_scale(a,b)     glyph_scale(a) = b
1203# define set_glyph_raise(a,b)     glyph_raise(a) = b
1204# define set_glyph_left(a,b)      glyph_left(a) = b
1205# define set_glyph_right(a,b)     glyph_right(a) = b
1206# define set_glyph_lhmin(a,b)     glyph_lhmin(a) = (singleword) b
1207# define set_glyph_rhmin(a,b)     glyph_rhmin(a) = (singleword) b
1208# define set_glyph_hyphenate(a,b) glyph_hyphenate(a) = ((halfword) b)
1209# define set_glyph_options(a,b)   glyph_options(a) = ((halfword) b)
1210/*       set_glyph_dohyph(a,b)    glyph_hyphenate(a) = ((halfword) flip_hyphenation_mode(glyph_hyphenate(a),syllable_hyphenation_mode)) */
1211# define set_glyph_uchyph(a,b)    glyph_hyphenate(a) = ((halfword) flip_hyphenation_mode(glyph_hyphenate(a),uppercase_hyphenation_mode))
1212# define set_glyph_discpart(a,b)  glyph_discpart(a) = (singleword) (b)
1213# define get_glyph_discpart(a)    ((halfword) glyph_discpart(a))
1214
1215typedef enum glyph_subtypes {
1216    /* initial value: */
1217    glyph_unset_subtype,
1218    /* traditional text: */
1219    glyph_character_subtype,
1220    glyph_ligature_subtype,
1221    /* special math */
1222    glyph_math_delimiter_subtype,
1223    glyph_math_extensible_subtype,
1224    /* engine math, class driven */
1225    glyph_math_ordinary_subtype,
1226    glyph_math_operator_subtype,
1227    glyph_math_binary_subtype,
1228    glyph_math_relation_subtype,
1229    glyph_math_open_subtype,
1230    glyph_math_close_subtype,
1231    glyph_math_punctuation_subtype,
1232    glyph_math_variable_subtype,
1233    glyph_math_active_subtype,
1234    glyph_math_inner_subtype,
1235    glyph_math_under_subtype,
1236    glyph_math_over_subtype,
1237    glyph_math_fraction_subtype,
1238    glyph_math_radical_subtype,
1239    glyph_math_middle_subtype,
1240    glyph_math_accent_subtype,
1241    glyph_math_fenced_subtype,
1242    glyph_math_ghost_subtype,
1243    /* bogus subtype */
1244    glyph_math_vcenter_subtype,
1245    /* extra math, user classes, set but anonymous */
1246    glyph_math_extra_subtype = 31,
1247} glyph_subtypes;
1248
1249# define last_glyph_subtype glyph_math_accent_subtype
1250
1251typedef enum glyph_hstate_codes {
1252    glyph_discpart_unset,
1253    glyph_discpart_pre,
1254    glyph_discpart_post,
1255    glyph_discpart_replace,
1256    glyph_discpart_always,
1257} glyph_hstate_codes;
1258
1259typedef enum glyph_option_codes {
1260    /*tex These are part of the defaults (all): */
1261    glyph_option_normal_glyph              = 0x00000000,
1262    glyph_option_no_left_ligature          = 0x00000001,
1263    glyph_option_no_right_ligature         = 0x00000002,
1264    glyph_option_no_left_kern              = 0x00000004,
1265    glyph_option_no_right_kern             = 0x00000008,
1266    glyph_option_no_expansion              = 0x00000010,
1267    glyph_option_no_protrusion             = 0x00000020,
1268    glyph_option_no_italic_correction      = 0x00000040,
1269    glyph_option_no_zero_italic_correction = 0x00000080,
1270    /* */
1271    glyph_option_apply_x_offset            = 0x00000100,
1272    glyph_option_apply_y_offset            = 0x00000200,
1273    /* These are only meant for math characters: */
1274    glyph_option_math_discretionary        = 0x00000400,
1275    glyph_option_math_italics_too          = 0x00000800,
1276    /*tex We permit user options. */
1277    glyph_option_user_first                = 0x00010000,
1278    glyph_option_user_last                 = 0x40000000,
1279    /*tex So watch out: these are subsets! */
1280    glyph_option_all                       = 0x000003FF,
1281    glyph_option_valid                     = 0x4FFF0FFF,
1282} glyph_option_codes;
1283
1284
1285typedef enum auto_discretionary_codes {
1286    auto_discretionary_normal = 0x0001, /* turn glyphs into discretionary with three similar components */
1287    auto_discretionary_italic = 0x0002, /* also include italic correction when present */
1288} auto_discretionary_codes;
1289
1290static inline void tex_add_glyph_option    (halfword a, halfword r) { glyph_options(a) |= r; }
1291static inline void tex_remove_glyph_option (halfword a, halfword r) { glyph_options(a) &= ~(r | glyph_options(a)); }
1292static inline int  tex_has_glyph_option    (halfword a, halfword r) { return (glyph_options(a) & r) == r; }
1293
1294/*tex
1295    As we have a small field available for protection we no longer need to pack the protection
1296    state in the subtype. We can now basically use the subtype for anything we want (as long as it
1297    stays within the range |0x0000-0xFFFF|.
1298*/
1299
1300/* static inline void tex_protect_glyph      (halfword a) {        node_subtype(a) |= (quarterword) 0x8000; } */
1301/* static inline void tex_unprotect_glyph    (halfword a) {        node_subtype(a) &= (quarterword) 0x7FFF; } */
1302/* static inline int  tex_is_protected_glyph (halfword a) { return node_subtype(a) >= (quarterword) 0x8000; } */
1303/* static inline int  tex_subtype_of_glyph   (halfword a) { return node_subtype(a) &  (quarterword) 0x7FFF; } */
1304
1305typedef enum glyph_protection_codes {
1306    glyph_unprotected_code    = 0x0,
1307    glyph_protected_text_code = 0x1,
1308    glyph_protected_math_code = 0x2,
1309} glyph_protection_codes;
1310
1311/*tex
1312    Next come some very specialized nodes types. First the marks. They just register a token list.
1313*/
1314
1315# define mark_node_size 3
1316# define mark_ptr(a)    vlink(a,2)
1317# define mark_index(a)  vinfo(a,2)
1318
1319typedef enum mark_codes {
1320    set_mark_value_code,
1321    reset_mark_value_code,
1322} mark_codes;
1323
1324# define last_mark_subtype reset_mark_value_code
1325
1326/*tex
1327    The (not really used in \CONTEXT) |\vadjust| nodes are also small. The codes and subtypes
1328    overlap.
1329*/
1330
1331typedef enum adjust_subtypes {
1332    pre_adjust_code,
1333    post_adjust_code,
1334    local_adjust_code,
1335} adjust_subtypes;
1336
1337typedef enum adjust_options {
1338    adjust_option_none         = 0x00,
1339    adjust_option_before       = 0x01,
1340    adjust_option_baseline     = 0x02,
1341    adjust_option_depth_before = 0x04,
1342    adjust_option_depth_after  = 0x08,
1343    adjust_option_depth_check  = 0x10,
1344    adjust_option_depth_last   = 0x20,
1345} adjust_options;
1346
1347# define last_adjust_subtype local_adjust_code
1348
1349# define adjust_node_size       5
1350# define adjust_list(a)         vlink(a,2)
1351# define adjust_options(a)      vinfo(a,2)
1352# define adjust_index(a)        vlink(a,3)
1353# define adjust_reserved(a)     vinfo(a,3)
1354# define adjust_depth_before(a) vlink(a,4)
1355# define adjust_depth_after(a)  vinfo(a,4)
1356
1357# define has_adjust_option(p,o) ((adjust_options(p) & o) == o)
1358
1359/*tex
1360    Inserts are more complicated. The |ins| node stores an insert in the list while |inserting|
1361    nodes keep track of where to break the page so that they (hopefully) stay with the text. As
1362    already mentioned, the insert node is tricky in the sense that it uses an offset to an
1363    embedded (fake) node. That node acts as start of a next chain. Making that more transparent
1364    would demand some changes that I'm not willing to make right now (and maybe never).
1365*/
1366
1367# define insert_node_size       6          /* can become 1 smaller or we can have insert_index instead of subtype */
1368# define insert_index(a)        vinfo(a,2) /* width is not used */
1369# define insert_float_cost(a)   vlink(a,2)
1370# define insert_max_depth(a)    vlink(a,3)
1371# define insert_total_height(a) vlink(a,4) /* the sum of height and depth, i.e. total */
1372# define insert_list(a)         vinfo(a,5) /* is alias for |node_next|*/
1373# define insert_split_top(a)    vlink(a,5) /* a state variable */
1374
1375# define insert_first_box(a)    (a + 5)    /*tex A fake node where box_list_ptr becomes a next field. */
1376
1377# define split_node_size        5          /*tex Can become a |split_up_node|. */
1378# define split_insert_index(a)  vinfo(a,2) /*tex Same slot! */
1379# define split_broken(a)        vlink(a,2) /*tex An insertion for this class will break here if anywhere. */
1380# define split_broken_insert(a) vinfo(a,3) /*tex This insertion might break at |broken_ptr|. */
1381# define split_last_insert(a)   vlink(a,3) /*tex The most recent insertion for this |subtype|. */
1382# define split_best_insert(a)   vinfo(a,4) /*tex The optimum most recent insertion. */
1383# define split_height(a)        vlink(a,4) /*tex Aka |height(a) = vlink(a,4)| */ /* todo */
1384
1385typedef enum split_subtypes {
1386    normal_split_subtype,
1387    insert_split_subtype,
1388} split_subtypes;
1389
1390# define last_split_subtype insert_split_subtype
1391
1392/*tex
1393    It's now time for some Some handy shortcuts. These are used when determining proper break points
1394    and|/|or the beginning or end of words.
1395*/
1396
1397# define last_preceding_break_node whatsit_node
1398# define last_non_discardable_node dir_node
1399# define last_node_with_attributes glyph_node
1400# define last_complex_node         align_record_node
1401# define max_node_type             passive_node
1402
1403# define precedes_break(a)  (node_type(a) <= last_preceding_break_node)
1404# define precedes_kern(a)   ((node_type(a) == kern_node) && (node_subtype(a) == font_kern_subtype || node_subtype(a) == accent_kern_subtype || node_subtype(a) == math_shape_kern_subtype))
1405# define precedes_dir(a)    ((node_type(a) == dir_node) && normalize_line_mode_permitted(normalize_line_mode_par,break_after_dir_mode))
1406# define non_discardable(a) (node_type(a) <= last_non_discardable_node)
1407
1408static inline int tex_nodetype_is_complex     (halfword t) { return t <= last_complex_node; }
1409static inline int tex_nodetype_has_attributes (halfword t) { return t <= last_node_with_attributes; }
1410static inline int tex_nodetype_has_subtype    (halfword t) { return t != glue_spec_node && t != math_spec_node && t != font_spec_node; }
1411static inline int tex_nodetype_has_prev       (halfword t) { return t != glue_spec_node && t != math_spec_node && t != font_spec_node && t != attribute_node; }
1412static inline int tex_nodetype_has_next       (halfword t) { return t != glue_spec_node && t != math_spec_node && t != font_spec_node; }
1413static inline int tex_nodetype_is_visible     (halfword t) { return (t >= 0) && (t <= max_node_type) && lmt_interface.node_data[t].visible; }
1414
1415/*tex
1416    This is a bit weird place to define them but anyway. In the meantime in \LUAMETATEX\ we no
1417    longer have the option to report the codes used in \ETEX. We have different nodes so it makes
1418    no sense to complicate matters (although earlier version of \LUAMETATEX\ has this organized
1419    quite well \unknown\ just an example of cleaning up, wondering about the use and then dropping
1420    it.
1421*/
1422
1423# define get_node_size(i) (lmt_interface.node_data[i].size)
1424# define get_node_name(i) (lmt_interface.node_data[i].name)
1425/*       get_etex_code(i) (lmt_interface.node_data[i].etex) */
1426
1427/*tex
1428    Although expressions could use some dedicated data structure, currently they are implemented
1429    using a linked list. This means that only memory is the limitation for recursion but I might
1430    as well go for a dedicated structure some day, just for the fun of implementing it. It is
1431    probably also more efficient. The current approach is inherited from \ETEX. The stack is only
1432    used when we have expressions between parenthesis.
1433*/
1434
1435# define expression_node_size     3
1436# define expression_type(a)       vinfo00(a,1)   /*tex one of the value levels */
1437# define expression_state(a)      vinfo01(a,1)
1438# define expression_result(a)     vinfo02(a,1)
1439# define expression_unused(a)     vinfo03(a,1)
1440# define expression_expression(a) vlink(a,1)     /*tex saved expression so far */
1441# define expression_term(a)       vlink(a,2)     /*tex saved term so far */
1442# define expression_numerator(a)  vinfo(a,2)     /*tex saved numerator */
1443
1444/*tex 
1445    Why not. 
1446*/
1447
1448# define loop_state_node_size 2
1449# define loop_state_count(a)  vinfo(a,1) /* instead if node_attr */
1450
1451/*tex
1452    To be decided: go double 
1453*/
1454
1455# define expression_entry(a)      lvalue(a,2)
1456
1457/*tex
1458    This is a node that stores a font state. In principle we can do without but for tracing it
1459    really helps to have this compound element because it is more compact. We could have gone
1460    numeric and use the sparse array approach but then we'd have to add a 4 int store which is more
1461    code and also makes save and restore more complex.
1462*/
1463
1464# define font_spec_node_size     5           /* we can be smaller: no attr and no prev */
1465# define font_spec_identifier(a) vinfo(a,2)
1466# define font_spec_scale(a)      vlink(a,2)
1467# define font_spec_x_scale(a)    vinfo(a,3)
1468# define font_spec_y_scale(a)    vlink(a,3)
1469# define font_spec_slant(a)      vinfo(a,4)
1470# define font_spec_weight(a)     vlink(a,4)
1471
1472static inline int tex_same_fontspec(halfword a, halfword b)
1473{
1474    return
1475        (a == b)
1476     || (a && b && font_spec_identifier(a) == font_spec_identifier(b)
1477                && font_spec_scale(a)      == font_spec_scale(b)
1478                && font_spec_x_scale(a)    == font_spec_x_scale(b)
1479                && font_spec_y_scale(a)    == font_spec_y_scale(b)
1480                && font_spec_slant(a)      == font_spec_slant(b)
1481                && font_spec_weight(a)     == font_spec_weight(b)
1482        )
1483    ;
1484}
1485
1486/*tex
1487    At the cost of some more memory we now use a mode for storage. This not only overcomes the
1488    \UNICODE\ limitation but also permits storing more in the future.
1489*/
1490
1491# define math_spec_node_size     3
1492# define math_spec_class(a)      vinfo00(a,1)  /* attr */
1493# define math_spec_family(a)     vinfo01(a,1)
1494# define math_spec_character(a)  vlink(a,1)    /* prev */
1495# define math_spec_properties(a) vinfo0(a,2)
1496# define math_spec_group(a)      vinfo1(a,2)
1497# define math_spec_index(a)      vlink(a,2)
1498
1499# define math_spec_value(a)     (((math_spec_class(a) & 0x3F) << 12) + ((math_spec_family(a) & 0x3F) << 8) + (math_spec_character(a) & 0xFF))
1500
1501static inline int tex_same_mathspec(halfword a, halfword b)
1502{
1503    return
1504        (a == b)
1505     || (a && b && math_spec_class(a)      == math_spec_class(b)
1506                && math_spec_family(a)     == math_spec_family(b)
1507                && math_spec_character(a)  == math_spec_character(b)
1508                && math_spec_properties(a) == math_spec_properties(b)
1509                && math_spec_group(a)      == math_spec_group(b)
1510                && math_spec_index(a)      == math_spec_index(b)
1511        )
1512    ;
1513}
1514
1515/*tex
1516    Here are some more stack related nodes.
1517*/
1518
1519# define align_stack_node_size                 16
1520# define align_stack_align_ptr(a)              vinfo(a,1)
1521# define align_stack_cur_align(a)              vlink(a,1)
1522# define align_stack_preamble(a)               vinfo(a,2)
1523# define align_stack_cur_span(a)               vlink(a,2)
1524# define align_stack_cur_loop(a)               vinfo(a,3)
1525# define align_stack_wrap_source(a)            vlink(a,3)
1526# define align_stack_align_state(a)            vinfo(a,4)
1527# define align_stack_no_align_level(a)         vlink(a,4)
1528# define align_stack_cur_post_adjust_head(a)   vinfo(a,5)
1529# define align_stack_cur_post_adjust_tail(a)   vlink(a,5)
1530# define align_stack_cur_pre_adjust_head(a)    vinfo(a,6)
1531# define align_stack_cur_pre_adjust_tail(a)    vlink(a,6)
1532# define align_stack_cur_post_migrate_head(a)  vinfo(a,7)
1533# define align_stack_cur_post_migrate_tail(a)  vlink(a,7)
1534# define align_stack_cur_pre_migrate_head(a)   vinfo(a,8)
1535# define align_stack_cur_pre_migrate_tail(a)   vlink(a,8)
1536# define align_stack_options(a)                vinfo(a,9)
1537# define align_stack_attr_list(a)              vlink(a,9)
1538# define align_stack_callback(a)               vinfo(a,10)
1539# define align_stack_data(a)                   vlink(a,10) /* maybe */
1540
1541# define align_stack_row_attrlist(a)           vinfo(a,11) 
1542# define align_stack_row_orientation(a)        vlink(a,11)
1543# define align_stack_row_yoffset(a)            vinfo(a,12)
1544# define align_stack_row_xoffset(a)            vlink(a,12)
1545# define align_stack_row_ymove(a)              vinfo(a,13)
1546# define align_stack_row_xmove(a)              vlink(a,13)
1547# define align_stack_row_shift(a)              vinfo(a,14)
1548# define align_stack_row_source(a)             vlink(a,14)
1549# define align_stack_row_target(a)             vinfo(a,15)
1550# define align_stack_row_anchor(a)             vlink(a,15)
1551
1552/*tex
1553    If nodes are for nesting conditionals. We have more state information that in (for instance)
1554    \LUATEX\ because we have more tracing and more test variants.
1555*/
1556
1557# define if_node_size            3             /*tex we can use prev now */
1558# define if_limit_type(a)        vinfo0(a,1)   /*tex overlaps with node_attr */
1559# define if_limit_subtype(a)     vinfo1(a,1)   /*tex overlaps with node_attr */
1560# define if_limit_unless(a)      vinfo00(a,2)
1561# define if_limit_step(a)        vinfo01(a,2)
1562# define if_limit_stepunless(a)  vinfo02(a,2)
1563# define if_limit_unused(a)      vinfo03(a,2)
1564# define if_limit_line(a)        vlink(a,2)
1565
1566/*tex
1567    Now come some rather special ones. For instance par shapes and file cq.\ line related nodes
1568    were variable nodes. Thsi was dropped and replaced by a more generic specficiation node type.
1569    In principle we can use that for more purposes.
1570
1571    We use a bit of abstraction as preparation for different allocations. Dynamic allocation makes
1572    it possible to get rid of variable nodes but it is slower.
1573
1574    Because this node has no links we can use the next field as counter. The subtype is just for
1575    diagnostics. This node is special in the sense that it has a real pointer. Such nodes will not
1576    be stored in the format file. Because there is a pointer field we have some extra accessors.
1577
1578    Todo: we also need to catch the fact that we can run out of memory but in practice that will
1579    not happen soon, for instance because we seldom use parshapes. And in the meantime the pseudo
1580    file related nodes are gone anyway because all file IO has been delegated to \LUA\ now.
1581*/
1582
1583# define specification_node_size  3
1584# define specification_count(a)   vlink(a,0)
1585# define specification_options(a) vinfo(a,1)
1586# define specification_unused(a)  vlink(a,1)
1587# define specification_pointer(a) (mvalue(a,2))
1588
1589typedef enum specification_options {
1590    specification_option_repeat = 0x0001,
1591} specifications_options;
1592
1593# define specification_index(a,n) ((memoryword *) specification_pointer(a))[n - 1]
1594
1595# define specification_repeat(a)  ((specification_options(a) & specification_option_repeat) == specification_option_repeat)
1596
1597# define specification_n(a,n)     (specification_repeat(a) ? ((n - 1) % specification_count(a) + 1) : (n > specification_count(a) ? specification_count(a) : n))
1598
1599/* interesting: 1Kb smaller bin: */
1600
1601// static inline halfword specification_n(halfword a, halfword n) { return specification_repeat(a) ? ((n - 1) % specification_count(a) + 1) : (n > specification_count(a) ? specification_count(a) : n); }
1602
1603extern void            tex_null_specification_list     (halfword a);
1604extern void            tex_new_specification_list      (halfword a, halfword n, halfword o);
1605extern void            tex_dispose_specification_list  (halfword a);
1606extern void            tex_copy_specification_list     (halfword a, halfword b);
1607extern void            tex_shift_specification_list    (halfword a, int n, int rotate);
1608
1609static inline int      tex_get_specification_count     (halfword a)                         { return a ? specification_count(a) : 0; }
1610static inline void     tex_set_specification_option    (halfword a, int o)                  { specification_options(a) |= o; }
1611
1612static inline halfword tex_get_specification_indent    (halfword a, halfword n)             { return specification_index(a,specification_n(a,n)).half0; }
1613static inline halfword tex_get_specification_width     (halfword a, halfword n)             { return specification_index(a,specification_n(a,n)).half1; }
1614static inline halfword tex_get_specification_penalty   (halfword a, halfword n)             { return specification_index(a,specification_n(a,n)).half0; }
1615
1616static inline void     tex_set_specification_indent    (halfword a, halfword n, halfword v) { specification_index(a,n).half0 = v; }
1617static inline void     tex_set_specification_width     (halfword a, halfword n, halfword v) { specification_index(a,n).half1 = v; }
1618static inline void     tex_set_specification_penalty   (halfword a, halfword n, halfword v) { specification_index(a,n).half0 = v; }
1619
1620# define par_passes_size 11
1621
1622# define par_passes_slot(n,m) ((n-1)*par_passes_size+m)
1623
1624typedef enum passes_features { 
1625    passes_quit_pass          = 0x0001,
1626    passes_skip_pass          = 0x0002,
1627    passes_optional_set       = 0x0004,
1628    passes_callback_set       = 0x0008,
1629    passes_orphan_penalty_set = 0x0010,
1630    passes_if_adjust_spacing  = 0x0020,
1631} passes_features;
1632
1633typedef enum passes_classes { 
1634    very_loose_class     = 0x0001,
1635    loose_fit_class      = 0x0002,     
1636    semi_loose_fit_class = 0x0004,
1637    decent_fit_class     = 0x0008,    
1638    semi_tight_fit_class = 0x0010,
1639    tight_fit_class      = 0x0020,     
1640} passes_classes;
1641
1642// todo: optimize order 
1643
1644static inline void     tex_set_passes_threshold            (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 1)).half0 = v; }
1645static inline void     tex_set_passes_badness              (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 1)).half1 = v; }
1646static inline void     tex_set_passes_features             (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 2)).quart00 |= (v & 0xFFFF); }
1647static inline void     tex_set_passes_classes              (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 2)).quart01 |= (v & 0xFFFF); }
1648static inline void     tex_set_passes_tolerance            (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 2)).half1 = v; }
1649static inline void     tex_set_passes_linepenalty          (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 3)).half0 = v; }
1650static inline void     tex_set_passes_extrahyphenpenalty   (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 3)).half1 = v; }
1651static inline void     tex_set_passes_doublehyphendemerits (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 4)).half0 = v; }
1652static inline void     tex_set_passes_finalhyphendemerits  (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 4)).half1 = v; }
1653static inline void     tex_set_passes_adjdemerits          (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 5)).half0 = v; }
1654static inline void     tex_set_passes_linebreakcriterion   (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 5)).half1 = v; }
1655static inline void     tex_set_passes_emergencystretch     (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 6)).half0 = v; }
1656static inline void     tex_set_passes_looseness            (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 6)).half1 = v; }
1657static inline void     tex_set_passes_adjustspacingstep    (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 7)).half0 = v; }
1658static inline void     tex_set_passes_adjustspacingshrink  (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 7)).half1 = v; }
1659static inline void     tex_set_passes_adjustspacingstretch (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 8)).half0 = v; }
1660static inline void     tex_set_passes_adjustspacing        (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 8)).half1 = v; }
1661static inline void     tex_set_passes_identifier           (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 9)).half0 = v; }
1662static inline void     tex_set_passes_optional             (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n, 9)).half1 = v; }
1663static inline void     tex_set_passes_callback             (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n,10)).half0 = v; }
1664static inline void     tex_set_passes_orphanpenalty        (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n,10)).half1 = v; }
1665static inline void     tex_set_passes_doubleadjdemerits    (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n,11)).half0 = v; }
1666static inline void     tex_set_passes_reserved             (halfword a, halfword n, halfword v) { specification_index(a,par_passes_slot(n,11)).half1 = v; } /* available */
1667
1668static inline halfword tex_get_passes_threshold            (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 1)).half0; }
1669static inline halfword tex_get_passes_badness              (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 1)).half1; }
1670static inline halfword tex_get_passes_features             (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 2)).quart00; }
1671static inline halfword tex_get_passes_classes              (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 2)).quart01; }
1672static inline halfword tex_get_passes_tolerance            (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 2)).half1; }
1673static inline halfword tex_get_passes_linepenalty          (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 3)).half0; }
1674static inline halfword tex_get_passes_extrahyphenpenalty   (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 3)).half1; }
1675static inline halfword tex_get_passes_doublehyphendemerits (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 4)).half0; }
1676static inline halfword tex_get_passes_finalhyphendemerits  (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 4)).half1; }
1677static inline halfword tex_get_passes_adjdemerits          (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 5)).half0; }
1678static inline halfword tex_get_passes_linebreakcriterion   (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 5)).half1; }
1679static inline halfword tex_get_passes_emergencystretch     (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 6)).half0; }
1680static inline halfword tex_get_passes_looseness            (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 6)).half1; }
1681static inline halfword tex_get_passes_adjustspacingstep    (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 7)).half0; }
1682static inline halfword tex_get_passes_adjustspacingshrink  (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 7)).half1; }
1683static inline halfword tex_get_passes_adjustspacingstretch (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 8)).half0; }
1684static inline halfword tex_get_passes_adjustspacing        (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 8)).half1; }
1685static inline halfword tex_get_passes_identifier           (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 9)).half0; }
1686static inline halfword tex_get_passes_optional             (halfword a, halfword n) { return specification_index(a,par_passes_slot(n, 9)).half1; }
1687static inline halfword tex_get_passes_callback             (halfword a, halfword n) { return specification_index(a,par_passes_slot(n,10)).half0; }
1688static inline halfword tex_get_passes_orphanpenalty        (halfword a, halfword n) { return specification_index(a,par_passes_slot(n,10)).half1; }
1689static inline halfword tex_get_passes_doubleadjdemerits    (halfword a, halfword n) { return specification_index(a,par_passes_slot(n,11)).half0; }
1690static inline halfword tex_get_passes_reserved             (halfword a, halfword n) { return specification_index(a,par_passes_slot(n,11)).half1; } /* available */
1691
1692extern        halfword tex_new_specification_node      (halfword n, quarterword s, halfword options);
1693extern        void     tex_dispose_specification_nodes (void);
1694
1695/*tex
1696    We now define some math related nodes (and noads) and start with style and choice nodes. Style
1697    nodes can be smaller, the information is encoded in |subtype|, but choice nodes are on-the-spot
1698    converted to style nodes with slack. The advantage is that we don't run into issues when a choice
1699    node is the first node in which case we would have to adapt head pointers (read: feed them back
1700    into the calling routines). So, we keep this as it is now.
1701
1702    Parameter nodes started out as an experiment. We could actually use the same mechanism as
1703    attributes but (1) we don't want attribute nodes in the list, it is very math specific and (3)
1704    we don't need to be real fast here.
1705
1706    Maybe these three can be merged into one type but on the other hand they are part of the \TEX\
1707    legacy and well documented so \unknown for now we keep it as-is. In the meantime we are no 
1708    longer casting choices to styles. 
1709
1710*/
1711
1712# define style_node_size               3
1713# define style_style                   node_subtype
1714# define style_scale(a)                vinfo(a,2)
1715# define style_reserved(a)             vlink(a,2)
1716
1717# define choice_node_size              5
1718//define choice_style                  node_subtype
1719# define choice_display_mlist(a)       vinfo(a,2) /*tex mlist to be used in display style or pre_break */
1720# define choice_text_mlist(a)          vlink(a,2) /*tex mlist to be used in text style or post_break */
1721# define choice_script_mlist(a)        vinfo(a,3) /*tex mlist to be used in script style or no_break */
1722# define choice_script_script_mlist(a) vlink(a,3) /*tex mlist to be used in scriptscript style */
1723# define choice_class(a)               vinfo(a,4) /*tex we could abuse the script script field */
1724# define choice_unused(a)              vlink(a,4)
1725
1726# define choice_pre_break              choice_display_mlist
1727# define choice_post_break             choice_text_mlist          
1728# define choice_no_break               choice_script_mlist
1729
1730# define parameter_node_size           3
1731# define parameter_style               node_subtype
1732# define parameter_name(a)             vinfo(a,2)
1733# define parameter_value(a)            vlink(a,2)
1734
1735typedef enum simple_choice_subtypes {
1736    normal_choice_subtype,
1737    discretionary_choice_subtype,
1738} simple_choice_subtypes; 
1739
1740# define last_choice_subtype discretionary_choice_subtype
1741
1742/*tex
1743    Because noad types get changed when processing we need to make sure some if the node sizes
1744    match and that we don't share slots with different properties.
1745
1746    First come the regular noads. The generic noad has the same size and similar fields as a fence
1747    noad, and their types get swapped a few times.
1748
1749    We accept a little waste of space in order to get nicer code. After all, math is not that
1750    demanding. Although delimiter, accent, fraction and radical share the same structure we do use
1751    specific field names because of clarity. Not all fields are used always.
1752
1753    \starttabulate[|l|l|l|l|l|l|]
1754    \FL
1755    \BC            \BC noad       \BC accent            \BC fraction         \BC radical          \NC fence        \NC \NR
1756    \ML                                                                                           
1757    \NC vlink  2   \NC new_hlist  \NC                   \NC                  \NC                  \NC              \NC \NR
1758    \ML                                                                                           
1759    \NC vinfo  2   \NC nucleus    \NC                   \NC                  \NC                  \NC              \NC \NR
1760    \NC vlink  3   \NC supscr     \NC                   \NC                  \NC                  \NC              \NC \NR
1761    \NC vinfo  3   \NC subscr     \NC                   \NC                  \NC                  \NC              \NC \NR
1762    \NC vlink  4   \NC supprescr  \NC                   \NC                  \NC                  \NC              \NC \NR
1763    \NC vinfo  4   \NC subprescr  \NC                   \NC                  \NC                  \NC              \NC \NR
1764    \ML                                                                                           
1765    \NC vlink  5   \NC italic     \NC                   \NC                  \NC                  \NC              \NC \NR
1766    \NC vinfo  5   \NC width      \NC                   \NC                  \NC                  \NC              \NC \NR
1767    \NC vlink  6   \NC height     \NC                   \NC                  \NC                  \NC              \NC \NR
1768    \NC vinfo  6   \NC depth      \NC                   \NC                  \NC                  \NC              \NC \NR
1769    \ML                                                                                           
1770    \NC vlink  7   \NC options    \NC                   \NC                  \NC                  \NC              \NC \NR
1771    \NC vinfo  7   \NC style      \NC                   \NC                  \NC                  \NC              \NC \NR
1772    \NC vlink  8   \NC family     \NC                   \NC                  \NC                  \NC              \NC \NR
1773    \NC vinfo  8   \NC class      \NC                   \NC                  \NC                  \NC              \NC \NR
1774    \NC vlink  9   \NC source     \NC                   \NC                  \NC                  \NC              \NC \NR
1775    \NC vinfo  9   \NC prime      \NC                   \NC                  \NC                  \NC              \NC \NR
1776    \NC vlink 10   \NC leftslack  \NC                   \NC                  \NC                  \NC              \NC \NR
1777    \NC vinfo 10   \NC rightslack \NC                   \NC                  \NC                  \NC              \NC \NR
1778    \ML                                                                                           
1779    \NC vlink 11   \NC extra_1    \NC top_character     \NC rule_thickness   \NC degree           \NC list         \NC \NR
1780    \NC vinfo 11   \NC extra_2    \NC bot_character     \NC left_delimiter   \NC left_delimiter   \NC source       \NC \NR
1781    \NC vlink 12   \NC extra_3    \NC overlay_character \NC right_delimiter  \NC right_delimiter  \NC top          \NC \NR
1782    \NC vinfo 12   \NC extra_4    \NC fraction          \NC middle_delimiter \NC size             \NC bottom       \NC \NR
1783    \NC vlink 13   \NC extra_5    \NC topovershoot      \NC h_factor         \NC height           \NC topovershoot \NC \NR
1784    \NC vinfo 13   \NC extra_6    \NC botovershoot      \NC v_factor         \NC depth            \NC botovershoot \NC \NR
1785    \NC vlink 14   \NC extra_7    \NC                   \NC numerator        \NC top_delimiter    \NC              \NC \NR
1786    \NC vinfo 14   \NC extra_8    \NC                   \NC denominator      \NC bottom_delimiter \NC              \NC \NR
1787    \LL
1788    \stoptabulate
1789
1790    We can use smaller variables for style and class and then have one field available for other 
1791    usage so no need to grow.
1792
1793    As with other nodes, not all fields are used and|/|or can be set at the tex end but they are 
1794    available for usage at the \LUA\ end. Some have been used for experiments and stay around. 
1795
1796*/
1797
1798//define noad_state_node_size      6
1799//define noad_state_topright(a)    vlink(a,2)
1800//define noad_state_bottomright(a) vinfo(a,2)
1801//define noad_state_topleft(a)     vlink(a,3)
1802//define noad_state_bottomleft(a)  vinfo(a,3)
1803//define noad_state_height(a)      vlink(a,4)
1804//define noad_state_depth(a)       vinfo(a,4)
1805//define noad_state_toptotal(a)    vlink(a,5)
1806//define noad_state_bottomtotal(a) vinfo(a,5)
1807
1808# define noad_size            15
1809# define noad_new_hlist(a)    vlink(a,2)    /*tex the translation of an mlist; a bit confusing name */
1810# define noad_nucleus(a)      vinfo(a,2)
1811# define noad_supscr(a)       vlink(a,3)
1812# define noad_subscr(a)       vinfo(a,3)
1813# define noad_supprescr(a)    vlink(a,4)
1814# define noad_subprescr(a)    vinfo(a,4)
1815# define noad_italic(a)       vlink(a,5)    /*tex Sometimes used, might become more. */
1816# define noad_width(a)        vinfo(a,5)
1817# define noad_height(a)       vlink(a,6)
1818# define noad_depth(a)        vinfo(a,6)
1819//define noad_options(a)      vlink(a,7)
1820//define noad_style(a)        vinfo00(a,7)
1821//define noad_family(a)       vinfo01(a,7)
1822//define noad_script_state(a) vinfo02(a,7)
1823//define noad_analyzed(a)     vinfo03(a,7)  /*tex used for experiments */
1824//define noad_state(a)        vlink(a,8)    /*tex this might replace */
1825# define noad_options(a)      lvalue(a,7)   /*tex 64 bit fullword */
1826# define noad_style(a)        vlink00(a,8)
1827# define noad_family(a)       vlink01(a,8)
1828# define noad_script_state(a) vlink02(a,8)
1829# define noad_analyzed(a)     vlink03(a,8)  /*tex used for experiments */
1830# define noad_class_main(a)   vinfo00(a,8)
1831# define noad_class_left(a)   vinfo01(a,8)
1832# define noad_class_right(a)  vinfo02(a,8)
1833# define noad_script_order(a) vinfo03(a,8)
1834# define noad_source(a)       vlink(a,9)
1835# define noad_prime(a)        vinfo(a,9)
1836# define noad_left_slack(a)   vlink(a,10)
1837# define noad_right_slack(a)  vinfo(a,10)
1838# define noad_extra_1(a)      vlink(a,11)
1839# define noad_extra_2(a)      vinfo(a,11)
1840# define noad_extra_3(a)      vlink(a,12)
1841# define noad_extra_4(a)      vinfo(a,12)
1842# define noad_extra_5(a)      vlink(a,13)
1843# define noad_extra_6(a)      vinfo(a,13)
1844# define noad_extra_7(a)      vlink(a,14)
1845# define noad_extra_8(a)      vinfo(a,14)
1846
1847# define noad_total(a) (noad_height(a) + noad_depth(a))
1848
1849# define noad_has_postscripts(a)       (noad_subscr(a) || noad_supscr(a))
1850# define noad_has_prescripts(a)        (noad_subprescr(a) || noad_supprescr(a))
1851# define noad_has_scripts(a)           (noad_has_postscripts(a) || noad_has_prescripts(a) || noad_prime(a))
1852# define noad_has_following_scripts(a) (noad_subscr(a) || noad_supscr(a) || noad_prime(a))
1853# define noad_has_superscripts(a)      (noad_supprescr(a) || noad_supscr(a) || noad_prime(a))
1854# define noad_has_subscripts(a)        (noad_subprescr(a) || noad_subscr(a))
1855
1856# define noad_has_scriptstate(a,s)     ((noad_script_state(a) & s) == s)
1857
1858# define unset_noad_class 0xFE
1859
1860typedef enum noad_script_states {
1861    post_super_script_state = 0x01,
1862    post_sub_script_state   = 0x02,
1863    pre_super_script_state  = 0x04,
1864    pre_sub_script_state    = 0x08,
1865    prime_script_state      = 0x10,
1866} noad_script_states;
1867
1868typedef enum noad_script_locations {
1869    prime_unknown_location,
1870    prime_at_begin_location,
1871    prime_above_sub_location,
1872    prime_at_end_location,
1873} noad_prime_locations;
1874
1875typedef enum noad_script_order {
1876    script_unknown_first,
1877    script_primescript_first,
1878    script_subscript_first,
1879    script_superscript_first,
1880} noad_script_order;
1881
1882typedef struct noad_classes {
1883    singleword main;
1884    singleword left;
1885    singleword right;
1886} noad_classes;
1887
1888# define reset_noad_classes(n) do { \
1889   noad_class_main(n)  = (singleword) unset_noad_class; \
1890   noad_class_left(n)  = (singleword) unset_noad_class; \
1891   noad_class_right(n) = (singleword) unset_noad_class; \
1892   noad_analyzed(n)    = (singleword) unset_noad_class; \
1893} while (0);
1894
1895# define set_noad_classes(n,c) do { \
1896   noad_class_main(n)  = (singleword) (c & 0xFF); \
1897   noad_class_left(n)  = (singleword) (c & 0xFF); \
1898   noad_class_right(n) = (singleword) (c & 0xFF); \
1899} while (0);
1900
1901# define set_noad_main_class(n,c)  noad_class_main(n)  = (singleword) (c & 0xFF)
1902# define set_noad_left_class(n,c)  noad_class_left(n)  = (singleword) (c & 0xFF)
1903# define set_noad_right_class(n,c) noad_class_right(n) = (singleword) (c & 0xFF)
1904
1905# define get_noad_main_class(n)  (noad_class_main(n))
1906# define get_noad_left_class(n)  (noad_class_left(n))
1907# define get_noad_right_class(n) (noad_class_right(n))
1908
1909# define set_noad_style(n,s)  noad_style(n)  = (singleword) (s & 0xFF)
1910# define set_noad_family(n,f) noad_family(n) = (singleword) (f & 0xFF)
1911
1912/*tex
1913    Options are something \LUATEX\ and in \LUAMETEX\ we added some more. When we have dimensions
1914    then we obey |axis| and otherwise |noaxis|. This might evolve a bit over time. These options
1915    currently are on the same spot but we pretend they aren't so we have dedicated accessors. This
1916    also makes clear what noads have what options.
1917
1918    If we run out of options we can combine some, like auto.
1919*/
1920
1921// # if (defined(_MSC_VER) && ! defined(__MINGW32__))
1922// typedef enum noad_options : unsigned __int64 {
1923// # else 
1924typedef enum noad_options {
1925// # endif 
1926    noad_option_axis                       = 0x00000001,
1927    noad_option_no_axis                    = 0x00000002,
1928    noad_option_exact                      = 0x00000004,
1929    noad_option_left                       = 0x00000008, /* align option for overflown under/over */ /* used ? */
1930    noad_option_middle                     = 0x00000010, /* idem */
1931    noad_option_right                      = 0x00000020, /* idem */
1932    noad_option_adapt_to_left_size         = 0x00000040, /* old trickery, might go away but kind of fun */
1933    noad_option_adapt_to_right_size        = 0x00000080, /* idem */
1934    noad_option_no_sub_script              = 0x00000100,
1935    noad_option_no_super_script            = 0x00000200,
1936    noad_option_no_sub_pre_script          = 0x00000400,
1937    noad_option_no_super_pre_script        = 0x00000800,
1938    noad_option_no_script                  = 0x00001000,
1939    noad_option_no_overflow                = 0x00002000, /* keep (middle) extensible widthin target size */
1940    noad_option_void                       = 0x00004000, /* wipe and set width to zero */
1941    noad_option_phantom                    = 0x00008000, /* wipe */
1942    noad_option_openup_height              = 0x00010000,
1943    noad_option_openup_depth               = 0x00020000,
1944    noad_option_limits                     = 0x00040000, /* traditional modifier */
1945    noad_option_no_limits                  = 0x00080000, /* idem */
1946    noad_option_prefer_font_thickness      = 0x00100000,
1947    noad_option_no_ruling                  = 0x00200000,
1948    noad_option_shifted_sub_script         = 0x00400000,
1949    noad_option_shifted_super_script       = 0x00800000,
1950    noad_option_shifted_sub_pre_script     = 0x01000000,
1951    noad_option_shifted_super_pre_script   = 0x02000000,
1952    noad_option_unpack_list                = 0x04000000,
1953    noad_option_no_check                   = 0x08000000, /* don't check for missing end fence */
1954    noad_option_auto                       = 0x10000000,
1955    noad_option_unroll_list                = 0x20000000,
1956    noad_option_followed_by_space          = 0x40000000,
1957    noad_option_proportional               = 0x80000000,
1958    /*tex Watch out: the following options exceed halfword: |noad_options| are |long long|. */
1959} noad_options;
1960
1961/*tex The Microsoft compiler truncates to int, so: */
1962
1963# define noad_option_source_on_nucleus          (uint64_t) 0x000100000000
1964# define noad_option_fixed_super_or_sub_script  (uint64_t) 0x000200000000
1965# define noad_option_fixed_super_and_sub_script (uint64_t) 0x000400000000
1966# define noad_option_auto_base                  (uint64_t) 0x000800000000
1967# define noad_option_stretch                    (uint64_t) 0x001000000000
1968# define noad_option_shrink                     (uint64_t) 0x002000000000
1969# define noad_option_center                     (uint64_t) 0x004000000000
1970# define noad_option_scale                      (uint64_t) 0x008000000000
1971# define noad_option_keep_base                  (uint64_t) 0x010000000000
1972# define noad_option_single                     (uint64_t) 0x020000000000
1973# define noad_option_no_rule                    (uint64_t) 0x040000000000
1974# define noad_option_auto_middle                (uint64_t) 0x080000000000
1975# define noad_option_reflected                  (uint64_t) 0x100000000000
1976
1977# define has_option(a,b)     (((a) & (b)) == (b))
1978# define unset_option(a,b)   ((a) & ~(b))
1979
1980static inline void tex_add_noad_option    (halfword a, uint64_t r) { noad_options(a) |= r; }
1981static inline void tex_remove_noad_option (halfword a, uint64_t r) { noad_options(a) &= ~(r | noad_options(a)); }
1982static inline int  tex_has_noad_option    (halfword a, uint64_t r) { return (noad_options(a) & r) == r; }
1983
1984static inline int has_noad_no_script_option(halfword n, halfword option)
1985{
1986    switch (node_type(n)) {
1987        case simple_noad:
1988        case accent_noad:
1989        case radical_noad:
1990        case fence_noad:
1991        case fraction_noad:
1992            return has_option(noad_options(n), option) || has_option(noad_options(n), noad_option_no_script);
1993    }
1994    return 0;
1995}
1996
1997# define has_noad_option_nosubscript(a)    has_noad_no_script_option(a, noad_option_no_sub_script)
1998# define has_noad_option_nosupscript(a)    has_noad_no_script_option(a, noad_option_no_super_script)
1999# define has_noad_option_nosubprescript(a) has_noad_no_script_option(a, noad_option_no_sub_pre_script)
2000# define has_noad_option_nosupprescript(a) has_noad_no_script_option(a, noad_option_no_super_pre_script)
2001
2002# define has_noad_option_shiftedsubscript(a)            (has_option(noad_options(a), noad_option_shifted_sub_script))
2003# define has_noad_option_shiftedsupscript(a)            (has_option(noad_options(a), noad_option_shifted_super_script))
2004# define has_noad_option_shiftedsubprescript(a)         (has_option(noad_options(a), noad_option_shifted_sub_pre_script))
2005# define has_noad_option_shiftedsupprescript(a)         (has_option(noad_options(a), noad_option_shifted_super_pre_script))
2006# define has_noad_option_axis(a)                        (has_option(noad_options(a), noad_option_axis))
2007# define has_noad_option_exact(a)                       (has_option(noad_options(a), noad_option_exact))
2008# define has_noad_option_noaxis(a)                      (has_option(noad_options(a), noad_option_no_axis))
2009# define has_noad_option_openupheight(a)                (has_option(noad_options(a), noad_option_openup_height))
2010# define has_noad_option_openupdepth(a)                 (has_option(noad_options(a), noad_option_openup_depth))
2011# define has_noad_option_adapttoleft(a)                 (has_option(noad_options(a), noad_option_adapt_to_left_size))
2012# define has_noad_option_adapttoright(a)                (has_option(noad_options(a), noad_option_adapt_to_right_size))
2013# define has_noad_option_limits(a)                      (has_option(noad_options(a), noad_option_limits))
2014# define has_noad_option_nolimits(a)                    (has_option(noad_options(a), noad_option_no_limits))
2015# define has_noad_option_nooverflow(a)                  (has_option(noad_options(a), noad_option_no_overflow))
2016# define has_noad_option_preferfontthickness(a)         (has_option(noad_options(a), noad_option_prefer_font_thickness))
2017# define has_noad_option_noruling(a)                    (has_option(noad_options(a), noad_option_no_ruling))
2018# define has_noad_option_unpacklist(a)                  (has_option(noad_options(a), noad_option_unpack_list))
2019# define has_noad_option_nocheck(a)                     (has_option(noad_options(a), noad_option_no_check))
2020# define has_noad_option_exact(a)                       (has_option(noad_options(a), noad_option_exact))
2021# define has_noad_option_left(a)                        (has_option(noad_options(a), noad_option_left))
2022# define has_noad_option_middle(a)                      (has_option(noad_options(a), noad_option_middle))
2023# define has_noad_option_right(a)                       (has_option(noad_options(a), noad_option_right))
2024# define has_noad_option_auto(a)                        (has_option(noad_options(a), noad_option_auto))
2025# define has_noad_option_phantom(a)                     (has_option(noad_options(a), noad_option_phantom))
2026# define has_noad_option_void(a)                        (has_option(noad_options(a), noad_option_void))
2027# define has_noad_option_unrolllist(a)                  (has_option(noad_options(a), noad_option_unroll_list))
2028# define has_noad_option_followedbyspace(a)             (has_option(noad_options(a), noad_option_followed_by_space))
2029# define has_noad_option_proportional(a)                (has_option(noad_options(a), noad_option_proportional))
2030# define has_noad_option_center(a)                      (has_option(noad_options(a), noad_option_center))
2031# define has_noad_option_source_on_nucleus(a)           (has_option(noad_options(a), noad_option_source_on_nucleus))
2032# define has_noad_option_fixed_super_or_sub_script(a)   (has_option(noad_options(a), noad_option_fixed_super_or_sub_script))
2033# define has_noad_option_fixed_super_and_sub_script(a)  (has_option(noad_options(a), noad_option_fixed_super_and_sub_script))
2034# define has_noad_option_stretch(a)                     (has_option(noad_options(a), noad_option_stretch))
2035# define has_noad_option_shrink(a)                      (has_option(noad_options(a), noad_option_shrink))
2036# define has_noad_option_auto_base(a)                   (has_option(noad_options(a), noad_option_auto_base))
2037# define has_noad_option_scale(a)                       (has_option(noad_options(a), noad_option_scale))
2038# define has_noad_option_keep_base(a)                   (has_option(noad_options(a), noad_option_keep_base))
2039# define has_noad_option_single(a)                      (has_option(noad_options(a), noad_option_single))
2040# define has_noad_option_norule(a)                      (has_option(noad_options(a), noad_option_no_rule))
2041# define has_noad_option_auto_middle(a)                 (has_option(noad_options(a), noad_option_auto_middle))
2042# define has_noad_option_reflected(a)                   (has_option(noad_options(a), noad_option_reflected))
2043
2044/*tex
2045    In the meantime the codes and subtypes are in sync. The variable component does not really
2046    become a subtype.
2047*/
2048
2049typedef enum simple_noad_subtypes {
2050    ordinary_noad_subtype,
2051    operator_noad_subtype,
2052    binary_noad_subtype,
2053    relation_noad_subtype,
2054    open_noad_subtype,
2055    close_noad_subtype,
2056    punctuation_noad_subtype,
2057    variable_noad_subtype,    /* we want to run in parallel */
2058    active_noad_subtype,      /* we want to run in parallel */
2059    inner_noad_subtype,
2060    under_noad_subtype,
2061    over_noad_subtype,
2062    fraction_noad_subtype,
2063    radical_noad_subtype,
2064    middle_noad_subtype,
2065    accent_noad_subtype,
2066    fenced_noad_subtype,
2067    ghost_noad_subtype,
2068    vcenter_noad_subtype,
2069} simple_noad_subtypes;
2070
2071# define last_noad_type    vcenter_noad_subtype
2072# define last_noad_subtype vcenter_noad_subtype
2073
2074typedef enum math_component_types {
2075    math_component_ordinary_code,
2076    math_component_operator_code,
2077    math_component_binary_code,
2078    math_component_relation_code,
2079    math_component_open_code,
2080    math_component_close_code,
2081    math_component_punctuation_code,
2082    math_component_variable_code,
2083    math_component_inner_code,
2084    math_component_under_code,
2085    math_component_over_code,
2086    math_component_fraction_code,
2087    math_component_radical_code,
2088    math_component_middle_code,
2089    math_component_accent_code,
2090    math_component_fenced_code,
2091    math_component_ghost_code,
2092    math_component_atom_code,
2093} math_component_types;
2094
2095# define first_math_component_type math_component_ordinary_code
2096# define last_math_component_type  math_component_accent_code
2097
2098/*tex
2099    When I added adapt options, the |math_limits_cmd| became |math_modifier_cmd| just because it
2100    nicely fits in there.
2101*/
2102
2103typedef enum math_modifier_types {
2104    display_limits_modifier_code,
2105    limits_modifier_code,
2106    no_limits_modifier_code,
2107    adapt_to_left_modifier_code,
2108    adapt_to_right_modifier_code,
2109    axis_modifier_code,
2110    no_axis_modifier_code,
2111    phantom_modifier_code,
2112    void_modifier_code,
2113    source_modifier_code,
2114    openup_height_modifier_code,
2115    openup_depth_modifier_code,
2116} math_modifier_types;
2117
2118# define first_math_modifier_code display_limits_modifier_code
2119# define last_math_modifier_code  openup_depth_modifier_code
2120
2121/*tex accent noads: todo, left and right offsets and options */
2122
2123# define accent_noad_size        noad_size
2124# define accent_top_character    noad_extra_1 /*tex the |top_accent_chr| field of an accent noad */
2125# define accent_bottom_character noad_extra_2 /*tex the |bot_accent_chr| field of an accent noad */
2126# define accent_middle_character noad_extra_3 /*tex the |overlay_accent_chr| field of an accent noad */
2127# define accent_fraction         noad_extra_4
2128# define accent_top_overshoot    noad_extra_5
2129# define accent_bot_overshoot    noad_extra_6
2130
2131typedef enum math_accent_subtypes { 
2132    /* we can be more specific: todo fixedoverlay_accent_subtype */
2133    bothflexible_accent_subtype,
2134    fixedtop_accent_subtype,
2135    fixedbottom_accent_subtype,
2136    fixedboth_accent_subtype,
2137} math_accent_subtypes;
2138
2139# define last_accent_subtype fixedboth_accent_subtype
2140
2141/*tex
2142    With these left and right fencing noads we have a historical mix of |fence| and |delimiter| (and
2143    |shield|) naming which for now we keep. It gets swapped with the generic noad, so size matters.
2144 */
2145
2146# define fence_noad_size        noad_size
2147# define fence_delimiter_list   noad_extra_1    // not really a list
2148# define fence_nesting_factor   noad_extra_2
2149# define fence_delimiter_top    noad_extra_3
2150# define fence_delimiter_bottom noad_extra_4
2151# define fence_top_overshoot    noad_extra_5
2152# define fence_bottom_overshoot noad_extra_6
2153
2154typedef enum fence_subtypes {
2155    unset_fence_side,
2156    left_fence_side,
2157    middle_fence_side,
2158    right_fence_side,
2159    left_operator_side,
2160    no_fence_side,
2161    extended_left_fence_side,
2162    extended_middle_fence_side,
2163    extended_right_fence_side,
2164} fence_subtypes;
2165
2166# define last_fence_subtype extended_right_fence_side
2167# define first_fence_code   left_fence_side
2168# define last_fence_code    extended_right_fence_side
2169
2170/*tex
2171    Fraction noads are generic in the sense that they are also used for non|-|fractions, not that
2172    it matters much. We keep them as they are in \TEX\ but have more fields.
2173
2174    We put the numerator and denomerator in script fields so there can be no such direct scripts
2175    attached. Because we have prescripts we can used these fields and limit this handicap a bit but
2176    if we ever overcome this (at the cost of more fields in these similar noads) we need to adapt
2177    the error message for double scripts in |tex_run_math_script|.
2178
2179*/
2180
2181# define fraction_noad_size        noad_size
2182# define fraction_rule_thickness   noad_extra_1
2183# define fraction_left_delimiter   noad_extra_2
2184# define fraction_right_delimiter  noad_extra_3
2185# define fraction_middle_delimiter noad_extra_4
2186# define fraction_h_factor         noad_extra_5
2187# define fraction_v_factor         noad_extra_6
2188# define fraction_numerator        noad_extra_7
2189# define fraction_denominator      noad_extra_8
2190
2191typedef enum fraction_subtypes {
2192    over_fraction_subtype,
2193    atop_fraction_subtype, 
2194    above_fraction_subtype, 
2195    skewed_fraction_subtype,
2196    stretched_fraction_subtype,
2197} fraction_subtypes;
2198
2199# define valid_fraction_subtype(s) (s >= over_fraction_subtype && s <= stretched_fraction_subtype) 
2200
2201/*tex
2202    Radical noads are like fraction noads, but they only store a |left_delimiter|. They are also
2203    used for extensibles (over, under, etc) so the name is is somewhat confusing.
2204*/
2205
2206# define radical_noad_size        noad_size
2207# define radical_degree           noad_extra_1
2208# define radical_left_delimiter   noad_extra_2
2209# define radical_right_delimiter  noad_extra_3
2210# define radical_size             noad_extra_4
2211# define radical_height           noad_extra_5
2212# define radical_depth            noad_extra_6
2213# define radical_top_delimiter    noad_extra_7
2214# define radical_bottom_delimiter noad_extra_8
2215
2216typedef enum radical_subtypes {
2217    normal_radical_subtype,
2218    radical_radical_subtype,
2219    root_radical_subtype,
2220    rooted_radical_subtype,
2221    under_delimiter_radical_subtype,
2222    over_delimiter_radical_subtype,
2223    delimiter_under_radical_subtype,
2224    delimiter_over_radical_subtype,
2225    delimited_radical_subtype,
2226    h_extensible_radical_subtype,
2227} radical_subtypes;
2228
2229# define last_radical_subtype h_extensible_radical_subtype
2230# define last_radical_code    h_extensible_radical_subtype
2231
2232/*tex
2233    Again a very simple small node: it represents a math character so naturally it has a family.
2234    It can be turned list. These are subnodes. When an extra options field gets added, the
2235    overlapping character and list fields can be split, so then we also have the origin saved.
2236
2237    The following nodes are kernel nodes: |math_char_node|, |math_text_char_node|, |sub_box_node|
2238    and |sub_mlist_node|. Characters eventually becomes wrapped in a list. 
2239*/
2240
2241typedef enum math_kernel_options {
2242    math_kernel_no_italic_correction = 0x0001,
2243    math_kernel_no_left_pair_kern    = 0x0002,
2244    math_kernel_no_right_pair_kern   = 0x0004,
2245    math_kernel_auto_discretionary   = 0x0008,
2246    math_kernel_full_discretionary   = 0x0010,
2247    math_kernel_ignored_character    = 0x0020,
2248    math_kernel_is_large_operator    = 0x0040,
2249    math_kernel_has_italic_shape     = 0x0080,
2250} math_kernel_options;
2251
2252# define math_kernel_node_size     5
2253# define kernel_math_family(a)     vinfo(a,2)
2254# define kernel_math_character(a)  vlink(a,2)
2255# define kernel_math_options(a)    vinfo(a,3)
2256# define kernel_math_list(a)       vlink(a,3)
2257# define kernel_math_properties(a) vinfo0(a,4)  /* for characters */
2258# define kernel_math_group(a)      vinfo1(a,4)  /* for characters */
2259# define kernel_math_index(a)      vlink(a,4)   /* for characters */
2260
2261# define math_kernel_node_has_option(a,b) ((kernel_math_options(a) & b) == b)
2262# define math_kernel_node_set_option(a,b) kernel_math_options(a) = (kernel_math_options(a) | b)
2263
2264/*tex
2265    This is also a subnode, this time for a delimiter field. The large family field is only used
2266    in traditional \TEX\ fonts where a base character can come from one font, and the extensible
2267    from another, but in \OPENTYPE\ math font that doesn't happen.
2268*/
2269    
2270/* It could be: */
2271
2272// # define math_delimiter_node_size     4
2273// # define delimiter_small_family(a)    vinfo00(a,2)  
2274// # define delimiter_large_family(a)    vinfo01(a,2)  
2275// # define delimiter_reserved_1         vinfo02(a,2)  
2276// # define delimiter_reserved_2         vinfo03(a,2)  
2277// # define delimiter_reserved_3         vlink(a,2)  
2278// # define delimiter_small_character(a) vinfo(a,3)
2279// # define delimiter_large_character(a) vlink(a,3)
2280
2281/* And some day (we then even assume traditionally to be mapped onto wide): */
2282
2283// # define math_delimiter_node_size 3
2284// # define delimiter_family(a)      vinfo00(a,2)  
2285// # define delimiter_reserved_1     vinfo01(a,2)  
2286// # define delimiter_reserved_2     vinfo02(a,2)  
2287// # define delimiter_reserved_3     vinfo03(a,2)  
2288// # define delimiter_character(a)   vlink(a,2)
2289
2290# define math_delimiter_node_size     4
2291# define delimiter_small_family(a)    vinfo(a,2) /*tex |family| for small delimiter */
2292# define delimiter_small_character(a) vlink(a,2) /*tex |character| for small delimiter */
2293# define delimiter_large_family(a)    vinfo(a,3) /*tex |family| for large delimiter */
2294# define delimiter_large_character(a) vlink(a,3) /*tex |character| for large delimiter */
2295
2296/*tex
2297    Before we come to the by now rather large local par node we define some small ones. The
2298    boundary nodes are an extended version of the original ones. The direction nodes are
2299    a simplified version of what \OMEGA\ has as whatsit. In \LUATEX\ it became a first class
2300    citizen and in \LUAMETATEX\ we cleaned it up.
2301*/
2302
2303typedef enum boundary_subtypes {
2304    cancel_boundary,
2305    user_boundary,
2306    protrusion_boundary, /* 1=left, 2=right, 3=both */
2307    word_boundary,
2308    page_boundary,
2309    math_boundary,
2310    optional_boundary,
2311    par_boundary,
2312} boundary_subtypes;
2313
2314typedef enum protrusion_boundary_options {
2315    protrusion_skip_none,
2316    protrusion_skip_next,
2317    protrusion_skip_previous,
2318    protrusion_skip_both,
2319} protrusion_boundary_options;
2320
2321# define last_boundary_subtype word_boundary
2322# define last_boundary_code    optional_boundary
2323
2324# define boundary_node_size   3
2325# define boundary_data(a)     vinfo(a,2)
2326# define boundary_reserved(a) vlink(a,2) // maybe level
2327
2328typedef enum dir_subtypes {
2329    normal_dir_subtype,
2330    cancel_dir_subtype,
2331} dir_subtypes;
2332
2333# define last_dir_subtype cancel_dir_subtype
2334
2335# define dir_node_size    3
2336# define dir_direction(a) vinfo(a,2)
2337# define dir_level(a)     vlink(a,2)
2338
2339/*tex
2340    Local par nodes come from \OMEGA\ and store the direction as well as local boxes. In \LUATEX
2341    we use a leaner direction model and in \LUAMETATEX\ we only kept the two directions that just
2342    work. In the end it is the backend that deals with these properties. The frontend just keeps
2343    a little track of them.
2344
2345    However, in \LUAMETATEX\ we can also store the paragraph state in this node. That way we no
2346    longer have the issue that properties are lost when a group ends before a |\par| is triggered.
2347    This is probably a feature that only makes sense in \CONTEXT\ which is why I made sure that
2348    there is not much overhead. In the first version one could control each variable, but as we
2349    ran out of bits in the end was done per group of variables. However, when I really need more
2350    detail I might go for a 64 bit field instead. After all we have that possibility in memory
2351    words.
2352
2353    These local par nodes can actually end up in the middle of lines  as they can be used to change
2354    the left and right box as well as inject penalties. For that reason they now have a proper
2355    subtype so that the initial and successive instances can be recognized.
2356 */
2357
2358typedef enum par_codes {
2359    par_none_code,
2360    par_hsize_code,
2361    par_left_skip_code,
2362    par_right_skip_code,
2363    par_hang_indent_code,
2364    par_hang_after_code,
2365    par_par_indent_code,
2366    par_par_fill_left_skip_code,
2367    par_par_fill_right_skip_code,
2368    par_par_init_left_skip_code,
2369    par_par_init_right_skip_code,
2370    par_emergency_left_skip_code,
2371    par_emergency_right_skip_code,
2372    par_adjust_spacing_code,
2373    par_protrude_chars_code,
2374    par_pre_tolerance_code,
2375    par_tolerance_code,
2376    par_emergency_stretch_code,
2377    par_looseness_code,
2378    par_last_line_fit_code,
2379    par_line_penalty_code,
2380    par_inter_line_penalty_code,
2381    par_club_penalty_code,
2382    par_widow_penalty_code,
2383    par_display_widow_penalty_code,
2384    par_orphan_penalty_code,
2385    par_broken_penalty_code,
2386    par_adj_demerits_code,
2387    par_double_adj_demerits_code,
2388    par_double_hyphen_demerits_code,
2389    par_final_hyphen_demerits_code,
2390    par_par_shape_code,
2391    par_inter_line_penalties_code,
2392    par_club_penalties_code,
2393    par_widow_penalties_code,
2394    par_display_widow_penalties_code,
2395    par_orphan_penalties_code,
2396    par_baseline_skip_code,
2397    par_line_skip_code,
2398    par_line_skip_limit_code,
2399    par_adjust_spacing_step_code,
2400    par_adjust_spacing_shrink_code,
2401    par_adjust_spacing_stretch_code,
2402    par_hyphenation_mode_code,
2403    par_shaping_penalties_mode_code,
2404    par_shaping_penalty_code,
2405    par_emergency_extra_stretch_code,
2406    par_par_passes_code,
2407    par_single_line_penalty_code,
2408} par_codes;
2409
2410typedef enum par_categories {
2411    par_none_category                    = 0x00000000,
2412    par_hsize_category                   = 0x00000001, // \hsize
2413    par_skip_category                    = 0x00000002, // \leftskip \rightskip
2414    par_hang_category                    = 0x00000004, // \hangindent \hangafter
2415    par_indent_category                  = 0x00000008, // \parindent
2416    par_par_fill_category                = 0x00000010, // \parfillskip \parfillleftskip
2417    par_adjust_category                  = 0x00000020, // \adjustspacing
2418    par_protrude_category                = 0x00000040, // \protrudechars
2419    par_tolerance_category               = 0x00000080, // \tolerance \pretolerance
2420    par_stretch_category                 = 0x00000100, // \emergcystretch
2421    par_looseness_category               = 0x00000200, // \looseness
2422    par_last_line_category               = 0x00000400, // \lastlinefit
2423    par_line_penalty_category            = 0x00000800, // \linepenalty \interlinepenalty \interlinepenalties
2424    par_club_penalty_category            = 0x00001000, // \clubpenalty \clubpenalties
2425    par_widow_penalty_category           = 0x00002000, // \widowpenalty \widowpenalties
2426    par_display_penalty_category         = 0x00004000, // \displaypenalty \displaypenalties
2427    par_broken_penalty_category          = 0x00008000, // \brokenpenalty
2428    par_demerits_category                = 0x00010000, // \doublehyphendemerits \finalhyphendemerits \adjdemerits
2429    par_shape_category                   = 0x00020000, // \parshape
2430    par_line_category                    = 0x00040000, // \baselineskip \lineskip \lineskiplimit
2431    par_hyphenation_category             = 0x00080000, // \Hyphenationmode
2432    par_shaping_penalty_category         = 0x00100000, // \shapingpenaltiesmode
2433    par_orphan_penalty_category          = 0x00200000, // \orphanpenalties
2434    par_emergency_category               = 0x00400000, // \emergencyleftskip \emergencyrightskip \emergencyextrastretch
2435    par_par_passes_category              = 0x00800000,
2436    par_par_single_line_penalty_category = 0x01000000,
2437    par_all_category                     = 0x7FFFFFFF, //
2438} par_categories;
2439
2440static int par_category_to_codes[] = {
2441    par_none_category,
2442    par_hsize_category,           // par_hsize_code
2443    par_skip_category,            // par_left_skip_code
2444    par_skip_category,            // par_right_skip_code
2445    par_hang_category,            // par_hang_indent_code
2446    par_hang_category,            // par_hang_after_code
2447    par_indent_category,          // par_par_indent_code
2448    par_par_fill_category,        // par_par_fill_skip_code
2449    par_par_fill_category,        // par_par_fill_left_skip_code
2450    par_par_fill_category,        // par_par_init_skip_code
2451    par_par_fill_category,        // par_par_init_skip_code
2452    par_emergency_category,       // par_par_emergency_left_skip
2453    par_emergency_category,       // par_par_emergency_right_skip
2454    par_adjust_category,          // par_adjust_spacing_code
2455    par_protrude_category,        // par_protrude_chars_code
2456    par_tolerance_category,       // par_pre_tolerance_code
2457    par_tolerance_category,       // par_tolerance_code
2458    par_stretch_category,         // par_emergency_stretch_code
2459    par_looseness_category,       // par_looseness_code
2460    par_last_line_category,       // par_last_line_fit_code
2461    par_line_penalty_category,    // par_line_penalty_code
2462    par_line_penalty_category,    // par_inter_line_penalty_code
2463    par_club_penalty_category,    // par_club_penalty_code
2464    par_widow_penalty_category,   // par_widow_penalty_code
2465    par_display_penalty_category, // par_display_widow_penalty_code
2466    par_orphan_penalty_category,  // par_orphan_penalty_code
2467    par_broken_penalty_category,  // par_broken_penalty_code
2468    par_demerits_category,        // par_adj_demerits_code
2469    par_demerits_category,        // par_double_adj_demerits_code
2470    par_demerits_category,        // par_double_hyphen_demerits_code
2471    par_demerits_category,        // par_final_hyphen_demerits_code
2472    par_shape_category,           // par_par_shape_code
2473    par_line_penalty_category,    // par_inter_line_penalties_code
2474    par_club_penalty_category,    // par_club_penalties_code
2475    par_widow_penalty_category,   // par_widow_penalties_code
2476    par_display_penalty_category, // par_display_widow_penalties_code
2477    par_orphan_penalty_category,  // par_orphan_penalties_code
2478    par_line_category,            // par_baseline_skip_code
2479    par_line_category,            // par_line_skip_code
2480    par_line_category,            // par_line_skip_limit_code
2481    par_adjust_category,          // par_adjust_spacing_step_code
2482    par_adjust_category,          // par_adjust_spacing_shrink_code
2483    par_adjust_category,          // par_adjust_spacing_stretch_code
2484    par_hyphenation_category,     // par_hyphenation_mode_code
2485    par_shaping_penalty_category, // par_shaping_penalties_mode_code
2486    par_shaping_penalty_category, // par_shaping_penalty_code
2487    par_emergency_category,
2488    par_par_passes_category,
2489    par_par_single_line_penalty_category,
2490};
2491
2492
2493# define par_node_size                  31          /*tex Make sure thet |max_chain_size| is large enough! */
2494# define par_single_line_penalty(a)     vinfo(a, 2) 
2495# define par_prev_graf(a)               vlink(a, 2) /*tex A bit of a joke but maybe handy indeed. */
2496# define par_box_left(a)                vinfo(a, 3)
2497# define par_box_left_width(a)          vlink(a, 3)
2498# define par_box_right(a)               vinfo(a, 4)
2499# define par_box_right_width(a)         vlink(a, 4)
2500# define par_box_middle(a)              vinfo(a, 5) /* no width here */
2501# define par_dir(a)                     vlink(a, 5)
2502# define par_state(a)                   vinfo(a, 6)
2503# define par_hsize(a)                   vlink(a, 6)
2504# define par_left_skip(a)               vinfo(a, 7)
2505# define par_right_skip(a)              vlink(a, 7)
2506# define par_hang_indent(a)             vinfo(a, 8)
2507# define par_hang_after(a)              vlink(a, 8)
2508# define par_par_indent(a)              vinfo(a, 9)
2509# define par_par_fill_left_skip(a)      vlink(a, 9)
2510# define par_par_fill_right_skip(a)     vinfo(a,10)
2511# define par_adjust_spacing(a)          vlink(a,10)
2512# define par_protrude_chars(a)          vinfo(a,11)
2513# define par_pre_tolerance(a)           vlink(a,11)
2514# define par_tolerance(a)               vinfo(a,12)
2515# define par_emergency_stretch(a)       vlink(a,12)
2516# define par_looseness(a)               vinfo(a,13)
2517# define par_last_line_fit(a)           vlink(a,13)
2518# define par_line_penalty(a)            vinfo(a,14)
2519# define par_inter_line_penalty(a)      vlink(a,14) /* */
2520# define par_club_penalty(a)            vinfo(a,15)
2521# define par_widow_penalty(a)           vlink(a,15)
2522# define par_display_widow_penalty(a)   vinfo(a,16)
2523# define par_orphan_penalty(a)          vlink(a,16)
2524# define par_broken_penalty(a)          vinfo(a,17) /* */
2525# define par_adj_demerits(a)            vlink(a,17)
2526# define par_double_hyphen_demerits(a)  vinfo(a,18)
2527# define par_final_hyphen_demerits(a)   vlink(a,18)
2528# define par_par_shape(a)               vinfo(a,19)
2529# define par_inter_line_penalties(a)    vlink(a,19)
2530# define par_club_penalties(a)          vinfo(a,20)
2531# define par_widow_penalties(a)         vlink(a,20)
2532# define par_display_widow_penalties(a) vinfo(a,21)
2533# define par_orphan_penalties(a)        vlink(a,21)
2534# define par_baseline_skip(a)           vinfo(a,22)
2535# define par_line_skip(a)               vlink(a,22)
2536# define par_line_skip_limit(a)         vinfo(a,23)
2537# define par_adjust_spacing_step(a)     vlink(a,23)
2538# define par_adjust_spacing_shrink(a)   vinfo(a,24)
2539# define par_adjust_spacing_stretch(a)  vlink(a,24)
2540# define par_end_par_tokens(a)          vinfo(a,25)
2541# define par_hyphenation_mode(a)        vlink(a,25)
2542# define par_shaping_penalties_mode(a)  vinfo(a,26)
2543# define par_shaping_penalty(a)         vlink(a,26)
2544# define par_par_init_left_skip(a)      vinfo(a,27)
2545# define par_par_init_right_skip(a)     vlink(a,27) 
2546# define par_emergency_left_skip(a)     vinfo(a,28)
2547# define par_emergency_right_skip(a)    vlink(a,28) 
2548# define par_emergency_extra_stretch(a) vinfo(a,29) 
2549# define par_par_passes(a)              vlink(a,29) 
2550# define par_double_adj_demerits(a)     vinfo(a,30)
2551# define par_reserved(a)                vlink(a,30)
2552
2553/*
2554    At some point we will have this (array with double values), depends on the outcome of an  
2555    experiment but I want to reserve this. We then also patch |texlocalboxes.c| line 295+. 
2556*/
2557
2558// define par_lousyness(a)              vinfo(a,2) /* par_penalty_interline */
2559// define par_reserved(a)               vlink(a,2) /* par_penalty_broken */
2560
2561typedef enum par_subtypes {
2562    vmode_par_par_subtype,
2563    local_box_par_subtype,
2564    hmode_par_par_subtype,
2565    parameter_par_subtype,
2566    math_par_subtype,       /* not used yet */
2567} par_subtypes;
2568
2569# define last_par_subtype math_par_subtype
2570
2571static inline int tex_is_start_of_par_node(halfword n)
2572{
2573    return ( n && (node_type(n) == par_node) && (node_subtype(n) == vmode_par_par_subtype || node_subtype(n) == hmode_par_par_subtype) );
2574}
2575
2576extern halfword    tex_get_par_par          (halfword p, halfword what);
2577extern void        tex_set_par_par          (halfword p, halfword what, halfword v, int force);
2578extern void        tex_snapshot_par         (halfword p, halfword what);
2579extern halfword    tex_find_par_par         (halfword head);
2580/*     halfword    tex_internal_to_par_code (halfword cmd, halfword index); */
2581extern void        tex_update_par_par       (halfword cmd, halfword index);
2582
2583static inline int  tex_par_state_is_set     (halfword p, halfword what)     { return (par_state(p) & par_category_to_codes[what]) == par_category_to_codes[what]; }
2584static inline void tex_set_par_state        (halfword p, halfword what)     { par_state(p) |= par_category_to_codes[what]; }
2585static inline int  tex_par_to_be_set        (halfword state, halfword what) { return (state & par_category_to_codes[what]) == par_category_to_codes[what]; }
2586
2587/*tex
2588    Because whatsits are used by the backend (or callbacks in the frontend) we do provide this node.
2589    It only has the basic properties: subtype, attribute, prev link and  next link. User nodes have
2590    been dropped because one can use whatsits to achieve the same. We also don't standardize the
2591    subtypes as it's very macro package specific what they do. So, only a size here:
2592*/
2593
2594# define whatsit_node_size 2
2595
2596/*tex
2597    Active and passive nodes are used in the par builder. There is plenty of comments in the code
2598    that explains them (although it's not that trivial I guess). Delta nodes just store the
2599    progression in widths, stretch and shrink: they are copies of arrays. Originally they just used
2600    offsets:
2601
2602    \starttyping
2603    # define delta_node_size    10
2604    # define delta_field(a,n)   node_next(a + n)
2605    \stoptyping
2606
2607    But that wasted 9 halfs for storing the 9 fields. So, next I played with this:
2608
2609    \starttyping
2610    # define delta_field_1(d) (delta_field(d,1)) // or: vinfo(d,1)
2611    # define delta_field_2(d) (delta_field(d,2)) // or: vlink(d,1)
2612    ...
2613    # define delta_field_9(d) (delta_field(d,9)) // or: vinfo(d,5)
2614    \stoptyping
2615
2616    But soon after that more meaningfull names were introduced, simply because in the code where they
2617    are used also verbose names showed up.
2618
2619    The active node is actually a |hyphenated_node| or an |unhyphenated_node| but for now we keep
2620    the \TEX\ lingua. We could probably turn the type into a subtype and moev fitness to another
2621    spot.
2622*/
2623
2624/*tex 
2625    We can use vinfo(a,2) for fitness instead the subtype field.  But then we also need to set 
2626    it explicitly because now that happens in the allocator.
2627*/
2628
2629# define active_node_size                  6            /*tex |hyphenated_node| or |unhyphenated_node| */
2630# define active_fitness                    node_subtype /*tex |very_loose_fit..tight_fit| on final line for this break */
2631# define active_break_node(a)              vlink(a,1)   /*tex pointer to the corresponding passive node */
2632# define active_line_number(a)             vinfo(a,1)   /*tex line that begins at this breakpoint */
2633# define active_total_demerits(a)          vlink(a,2)   /*tex the quantity that \TEX\ minimizes */
2634# define active_line_width(a)              vinfo(a,2)
2635# define active_glue(a)                    vlink(a,3)   /*tex corresponding glue stretch or shrink */
2636# define active_short(a)                   vinfo(a,3)   /*tex |shortfall| of this line */
2637# define active_quality(a)                 vlink(a,4)   /* probably we can use the passive one */
2638# define active_deficiency(a)              vinfo(a,4)   /* probably we can use the passive one */
2639# define active_badness(a)                 vlink(a,5)   /* probably we can use the passive one */
2640# define active_unused(a)                  vinfo(a,5)   /* probably we can use the passive one */
2641
2642# define passive_node_size                 9 
2643# define passive_cur_break(a)              vlink(a,1)   /*tex in passive node, points to position of this breakpoint */
2644# define passive_prev_break(a)             vinfo(a,1)   /*tex points to passive node that should precede this one */
2645# define passive_pen_inter(a)              vinfo(a,2)
2646# define passive_pen_broken(a)             vlink(a,2)
2647# define passive_left_box(a)               vlink(a,3)
2648# define passive_left_box_width(a)         vinfo(a,3)
2649# define passive_last_left_box(a)          vlink(a,4)
2650# define passive_last_left_box_width(a)    vinfo(a,4)
2651# define passive_right_box(a)              vlink(a,5)
2652# define passive_right_box_width(a)        vinfo(a,5)
2653# define passive_serial(a)                 vlink(a,6)   /*tex serial number for symbolic identification (pass) */
2654# define passive_middle_box(a)             vinfo(a,6)
2655# define passive_quality(a)                vlink(a,7) 
2656# define passive_deficiency(a)             vinfo(a,7)
2657# define passive_badness(a)                vlink(a,8) 
2658# define passive_unused(a)                 vinfo(a,8) 
2659
2660# define delta_node_size                   6
2661# define delta_field_total_glue(d)         vinfo(d,1)
2662# define delta_field_total_shrink(d)       vinfo(d,2)
2663# define delta_field_total_stretch(d)      vlink(d,2)
2664# define delta_field_total_fi_amount(d)    vinfo(d,3)
2665# define delta_field_total_fil_amount(d)   vlink(d,3)
2666# define delta_field_total_fill_amount(d)  vinfo(d,4)
2667# define delta_field_total_filll_amount(d) vlink(d,4)
2668# define delta_field_font_shrink(d)        vinfo(d,5)
2669# define delta_field_font_stretch(d)       vlink(d,5)
2670
2671/*tex
2672    Again we now have some helpers. We have a double linked list so here we go:
2673*/
2674
2675static inline void tex_couple_nodes(int a, int b)
2676{
2677    node_next(a) = b;
2678    node_prev(b) = a;
2679}
2680
2681static inline void tex_try_couple_nodes(int a, int b)
2682{
2683    if (b) {
2684        if (a) {
2685            node_next(a) = b;
2686        }
2687        node_prev(b) = a;
2688    } else if (a) {
2689        node_next(a) = null;
2690   }
2691}
2692
2693static inline void tex_uncouple_node(int a)
2694{
2695    node_next(a) = null;
2696    node_prev(a) = null;
2697}
2698
2699static inline halfword tex_head_of_node_list(halfword n)
2700{
2701    while (node_prev(n)) {
2702        n = node_prev(n);
2703    }
2704    return n;
2705}
2706
2707static inline halfword tex_tail_of_node_list(halfword n)
2708{
2709    while (node_next(n)) {
2710        n = node_next(n);
2711    }
2712    return n;
2713}
2714
2715/*tex
2716    Attribute management is kind of complicated. They are stored in a sorted linked list and we
2717    try to share these for successive nodes. In \LUATEX\ a state is kept and reset frequently but
2718    in \LUAMETATEX\ we try to be more clever, for instance we keep track of grouping. This comes
2719    as some overhead but saves reconstructing (often the same) list. It also saves memory.
2720*/
2721
2722# define attribute_cache_disabled max_halfword
2723# define current_attribute_state  lmt_node_memory_state.attribute_cache
2724
2725extern halfword tex_copy_attribute_list        (halfword attr);
2726extern halfword tex_copy_attribute_list_set    (halfword attr, int index, int value);
2727extern halfword tex_patch_attribute_list       (halfword attr, int index, int value);
2728extern void     tex_dereference_attribute_list (halfword attr);
2729extern void     tex_build_attribute_list       (halfword target);
2730extern halfword tex_current_attribute_list     (void);
2731extern int      tex_unset_attribute            (halfword target, int index, int value);
2732extern void     tex_unset_attributes           (halfword first, halfword last, int index);
2733extern void     tex_set_attribute              (halfword target, int index, int value);
2734extern int      tex_has_attribute              (halfword target, int index, int value);
2735
2736extern void     tex_reset_node_properties      (halfword target);
2737
2738# define get_attribute_list(target) \
2739    node_attr(target)
2740
2741/*
2742# define add_attribute_reference(a) do { \
2743    if (a && a != attribute_cache_disabled) { \
2744        ++attribute_count(a); \
2745    } \
2746} while (0)
2747*/
2748
2749inline static void add_attribute_reference(halfword a) 
2750{
2751    if (a && a != attribute_cache_disabled) { 
2752        ++attribute_count(a); 
2753    } 
2754}
2755
2756/*
2757# define delete_attribute_reference(a) do { \
2758    if (a && a != attribute_cache_disabled) { \
2759        tex_dereference_attribute_list(a); \
2760    } \
2761} while (0)
2762*/
2763
2764inline static void delete_attribute_reference(halfword a)
2765{
2766    if (a && a != attribute_cache_disabled) { 
2767        tex_dereference_attribute_list(a); 
2768    } 
2769}
2770
2771
2772# define remove_attribute_list(target) do { \
2773    halfword old_a = node_attr(target); \
2774    delete_attribute_reference(old_a); \
2775    node_attr(target) = null; \
2776} while (0)
2777
2778/*
2779static inline void remove_attribute_list(halfword target)
2780{
2781    halfword a_old = node_attr(target);
2782    if (a_old && a_old != attribute_cache_disabled) {
2783        dereference_attribute_list(a_old);
2784    }
2785    node_attr(target) = null;
2786}
2787*/
2788
2789/* This can be dangerous: */
2790
2791# define wipe_attribute_list_only(target) \
2792    node_attr(target) = null;
2793
2794/*tex
2795    Better is to add a ref before we remove one because there's the danger of premature freeing
2796    otherwise.
2797*/
2798
2799static inline void tex_attach_attribute_list_copy(halfword target, halfword source)
2800{
2801    halfword a_new = node_attr(source);
2802    halfword a_old = node_attr(target);
2803    node_attr(target) = a_new;
2804    add_attribute_reference(a_new);
2805    delete_attribute_reference(a_old);
2806}
2807
2808static inline void tex_attach_attribute_list_attribute(halfword target, halfword a_new)
2809{
2810    halfword a_old = node_attr(target);
2811    if (a_old != a_new) {
2812        node_attr(target) = a_new;
2813        add_attribute_reference(a_new);
2814        delete_attribute_reference(a_old);
2815    }
2816}
2817
2818# define attach_current_attribute_list tex_build_attribute_list /* (target) */
2819
2820# define set_current_attribute_state(v) do { \
2821      current_attribute_state = v; \
2822} while (0)
2823
2824/*
2825# define change_attribute_register(a,id,value) do { \
2826    if (eq_value(id) != value) { \
2827        if (is_global(a)) { \
2828            int i; \
2829            for (i = (lmt_save_state.save_stack_data.ptr - 1); i >= 0; i--) { \
2830                if (save_type(i) == attribute_list_save_type) { \
2831                    delete_attribute_reference(save_value(i)); \
2832                    save_value(i) = attribute_cache_disabled; \
2833                } \
2834            } \
2835        } else { \
2836            delete_attribute_reference(current_attribute_state); \
2837        } \
2838        set_current_attribute_state(attribute_cache_disabled); \
2839    } \
2840} while (0)
2841*/
2842
2843extern void tex_change_attribute_register(halfword a, halfword id, halfword value);
2844
2845/*tex
2846    We now arrive at some functions that report the nodes to users. The subtype information that
2847    is used in the \LUA\ interface is stored alongside.
2848*/
2849
2850extern void        tex_print_short_node_contents         (halfword n);
2851extern const char *tex_aux_subtype_str                   (halfword n);
2852extern void        tex_show_node_list                    (halfword n, int threshold, int max);
2853extern halfword    tex_actual_box_width                  (halfword r, scaled base_width);
2854extern void        tex_print_name                        (halfword p, const char *what);
2855extern void        tex_print_node_list                   (halfword n, const char *what, int threshold, int max);
2856/*     void        tex_print_node_and_details            (halfword p); */
2857/*     void        tex_print_subtype_and_attributes_info (halfword p, halfword s, node_info *data); */
2858extern void        tex_print_extended_subtype            (halfword p, quarterword s);
2859extern void        tex_aux_show_dictionary               (halfword p, halfword properties, halfword group, halfword index, halfword font, halfword character);
2860
2861/*tex 
2862    Basic node management:
2863*/
2864
2865extern halfword tex_new_node        (quarterword i, quarterword j);
2866extern void     tex_flush_node_list (halfword n);
2867extern void     tex_flush_node      (halfword n);
2868extern halfword tex_copy_node_list  (halfword n, halfword e);
2869extern halfword tex_copy_node       (halfword n);
2870extern halfword tex_copy_node_only  (halfword n);
2871/*     halfword tex_fix_node_list   (halfword n); */
2872
2873/*tex
2874    We already defined glue and gluespec node but here are some of the properties
2875    that they have. Again a few helpers.
2876*/
2877
2878typedef enum glue_orders {
2879    normal_glue_order,
2880    fi_glue_order,
2881    fil_glue_order,
2882    fill_glue_order,
2883    filll_glue_order
2884} glue_orders;
2885
2886typedef enum glue_amounts {
2887    /* we waste slot zero, we padd anyway */
2888    total_advance_amount = 1, // 1 //
2889    total_stretch_amount = 2, // 3 //
2890    total_fi_amount      = 3, // 4 //
2891    total_fil_amount     = 4, // 5 //
2892    total_fill_amount    = 5, // 6 //
2893    total_filll_amount   = 6, // 7 //
2894    total_shrink_amount  = 7, // 2 //
2895    font_stretch_amount  = 8, // 8 //
2896    font_shrink_amount   = 9, // 9 //
2897    /* */
2898    max_height_amount    = 10,
2899    max_depth_amount     = 11,
2900    /* */
2901    n_of_glue_amounts    = 12,
2902} glue_amounts;
2903
2904# define min_glue_order normal_glue_order
2905# define max_glue_order filll_glue_order
2906
2907typedef enum glue_signs {
2908    normal_glue_sign,
2909    stretching_glue_sign,
2910    shrinking_glue_sign
2911} glue_signs;
2912
2913# define min_glue_sign normal_glue_sign
2914# define max_glue_sign shrinking_glue_sign
2915
2916# define normal_glue_multiplier 0.0
2917
2918static inline halfword tex_checked_glue_sign  (halfword sign)  { return ((sign  < min_glue_sign ) || (sign  > max_glue_sign )) ? normal_glue_sign  : sign ; }
2919static inline halfword tex_checked_glue_order (halfword order) { return ((order < min_glue_order) || (order > max_glue_order)) ? normal_glue_order : order; }
2920
2921/*tex
2922    These are reserved nodes that sit at the start of main memory. We could actually just allocate
2923    them, but then we also need to set some when we start up. Now they are just saved in the format
2924    file. In \TEX\ these nodes were shared as much as possible (using a reference count) but here
2925    we just use copies.
2926
2927    Below we start at |zero_glue| which in our case is just 0, or |null| in \TEX\ speak. After these
2928    reserved nodes the memory used for whatever nodes are needed takes off.
2929
2930    Changing this to real nodes makes sense but is also tricky due to initializations ... some day
2931    (we need to store stuff in the states then and these are not saved!).
2932
2933    The first five could actualy be replaced by assignments because we don't share them as \TEX\ 
2934    does which makes it easier the just change them (we don't need to save memory here). The other
2935    ones could be nodes that get initialized at startup and be put in the structs that need them,
2936    but for now we keep it as-it-is.
2937
2938*/
2939
2940# define fi_glue           (zero_glue         + glue_spec_size) 
2941# define fi_l_glue         (fi_glue           + glue_spec_size)
2942# define fi_ll_glue        (fi_l_glue         + glue_spec_size)
2943# define fi_ss_glue        (fi_ll_glue        + glue_spec_size)
2944# define fi_l_neg_glue     (fi_ss_glue        + glue_spec_size)
2945
2946# define page_insert_head  (fi_l_neg_glue     + glue_spec_size)
2947# define contribute_head   (page_insert_head  + split_node_size) /*tex This was temp_node_size but we assign more. */
2948# define page_head         (contribute_head   + temp_node_size)
2949# define temp_head         (page_head         + glue_node_size)  /*tex It gets a glue type assigned. */
2950# define hold_head         (temp_head         + temp_node_size)
2951# define post_adjust_head  (hold_head         + temp_node_size)
2952# define pre_adjust_head   (post_adjust_head  + temp_node_size)
2953# define post_migrate_head (pre_adjust_head   + temp_node_size)
2954# define pre_migrate_head  (post_migrate_head + temp_node_size)
2955# define align_head        (pre_migrate_head  + temp_node_size)
2956# define active_head       (align_head        + temp_node_size)
2957# define end_span          (active_head       + active_node_size)
2958# define begin_period      (end_span          + span_node_size)  /*tex Used to mark begin of word in hjn. */
2959# define end_period        (begin_period      + glyph_node_size) /*tex Used to mark end of word in hjn. */
2960
2961# define last_reserved     (end_period        + glyph_node_size - 1)
2962
2963/*tex More helpers! */
2964
2965extern int       tex_list_has_glyph       (halfword list);
2966
2967extern halfword  tex_new_null_box_node    (quarterword type, quarterword subtype);
2968extern halfword  tex_new_rule_node        (quarterword subtype);
2969extern halfword  tex_new_glyph_node       (quarterword subtype, halfword fnt, halfword chr, halfword parent); /*tex afterwards: when we mess around */
2970extern halfword  tex_new_char_node        (quarterword subtype, halfword fnt, halfword chr, int all);         /*tex as we go: in maincontrol */
2971extern halfword  tex_new_text_glyph       (halfword fnt, halfword chr);
2972extern halfword  tex_new_disc_node        (quarterword subtype);
2973extern halfword  tex_new_glue_spec_node   (halfword param);
2974extern halfword  tex_new_param_glue_node  (quarterword param, quarterword subtype);
2975extern halfword  tex_new_glue_node        (halfword qlue, quarterword subtype);
2976extern halfword  tex_new_kern_node        (scaled width, quarterword subtype);
2977extern halfword  tex_new_penalty_node     (halfword penalty, quarterword subtype);
2978extern halfword  tex_new_par_node         (quarterword mode);
2979
2980extern halfword  tex_new_temp_node        (void);
2981
2982extern scaled    tex_glyph_width          (halfword p); /* x/y scaled */
2983extern scaled    tex_glyph_height         (halfword p); /* x/y scaled */
2984extern scaled    tex_glyph_depth          (halfword p); /* x/y scaled */
2985extern scaled    tex_glyph_total          (halfword p); /* x/y scaled */
2986extern scaledwhd tex_glyph_dimensions     (halfword p); /* x/y scaled */
2987extern int       tex_glyph_has_dimensions (halfword p); /* x/y scaled */
2988extern scaled    tex_glyph_width_ex       (halfword p); /* x/y scaled, expansion included */
2989extern scaledwhd tex_glyph_dimensions_ex  (halfword p); /* x/y scaled, expansion included */
2990
2991extern halfword  tex_kern_dimension       (halfword p);
2992extern halfword  tex_kern_dimension_ex    (halfword p); /* expansion included */
2993
2994extern scaled    tex_effective_glue       (halfword parent, halfword glue);
2995
2996extern scaledwhd tex_pack_dimensions      (halfword p);
2997
2998extern halfword  tex_list_node_mem_usage  (void);
2999extern halfword  tex_reversed_node_list   (halfword list);
3000extern int       tex_n_of_used_nodes      (int counts[]);
3001
3002# define _valid_node_(p) ((p > lmt_node_memory_state.reserved) && (p < lmt_node_memory_state.nodes_data.allocated) && (lmt_node_memory_state.nodesizes[p] > 0))
3003
3004static inline int tex_valid_node(halfword n)
3005{
3006    return n && _valid_node_(n) ? n : null;
3007}
3008
3009/*tex This is a bit strange place but better than a macro elsewhere: */
3010
3011static inline int tex_math_skip_boundary(halfword n)
3012{
3013    return (n && node_type(n) == glue_node
3014              && (node_subtype(n) == space_skip_glue  ||
3015                  node_subtype(n) == xspace_skip_glue ||
3016                  node_subtype(n) == zero_space_skip_glue));
3017}
3018
3019typedef enum special_node_list_types { /* not in sycn with the above .. maybe add bogus ones */
3020    page_insert_list_type,
3021    contribute_list_type,
3022    page_list_type,
3023    temp_list_type,
3024    hold_list_type,
3025    post_adjust_list_type,
3026    pre_adjust_list_type,
3027    post_migrate_list_type,
3028    pre_migrate_list_type,
3029    align_list_type,
3030    /* in different spot */
3031    page_discards_list_type,
3032    split_discards_list_type,
3033 // best_page_break_type
3034} special_node_list_types;
3035
3036extern int      tex_is_special_node_list  (halfword n, int *istail);
3037extern halfword tex_get_special_node_list (special_node_list_types list, halfword *tail);
3038extern void     tex_set_special_node_list (special_node_list_types list, halfword head);
3039
3040# endif
3041
3042