Source Browser ?

1/*
2    See license.txt in the root of this project.
3*/
4
5# include "luametatex.h"
6
7/*tex
8
9    TODO: Check |[get|set]field| for recent (new) fields.
10    TODO: Maybe we can lock glue nodes so that a calback can't mess with them.
11
12*/
13
14/*tex
15
16    We come now to what is probably the most interesting algorithm of \TEX: the mechanism for
17    choosing the \quote {best possible} breakpoints that yield the individual lines of a paragraph.
18    \TEX's line-breaking algorithm takes a given horizontal list and converts it to a sequence of
19    boxes that are appended to the current vertical list. In the course of doing this, it creates
20    a special data structure containing three kinds of records that are not used elsewhere in
21    \TEX. Such nodes are created while a paragraph is being processed, and they are destroyed
22    afterwards; thus, the other parts of \TEX\ do not need to know anything about how line-breaking
23    is done.
24
25    The method used here is based on an approach devised by Michael F. Plass and the author in 1977,
26    subsequently generalized and improved by the same two people in 1980. A detailed discussion
27    appears in {\sl SOFTWARE---Practice \AM\ Experience \bf11} (1981), 1119--1184, where it is
28    shown that the line-breaking problem can be regarded as a special case of the problem of
29    computing the shortest path in an acyclic network. The cited paper includes numerous examples
30    and describes the history of line breaking as it has been practiced by printers through the
31    ages. The present implementation adds two new ideas to the algorithm of 1980: Memory space
32    requirements are considerably reduced by using smaller records for inactive nodes than for
33    active ones, and arithmetic overflow is avoided by using \quote {delta distances} instead of
34    keeping track of the total distance from the beginning of the paragraph to the current point.
35
36    The |line_break| procedure should be invoked only in horizontal mode; it leaves that mode and
37    places its output into the current vlist of the enclosing vertical mode (or internal vertical
38    mode). There is one explicit parameter: |d| is true for partial paragraphs preceding display
39    math mode; in this case the amount of additional penalty inserted before the final line is
40    |display_widow_penalty| instead of |widow_penalty|.
41
42    There are also a number of implicit parameters: The hlist to be broken starts at |node_next
43    (head)|, and it is nonempty. The value of |prev_graf| in the enclosing semantic level tells
44    where the paragraph should begin in the sequence of line numbers, in case hanging indentation
45    or |\parshape| are in use; |prev_graf| is zero unless this paragraph is being continued after a
46    displayed formula. Other implicit parameters, such as the |par_shape_ptr| and various penalties
47    to use for hyphenation, etc., appear in |eqtb|.
48
49    After |line_break| has acted, it will have updated the current vlist and the value of
50    |prev_graf|. Furthermore, the global variable |just_box| will point to the final box created
51    by |line_break|, so that the width of this line can be ascertained when it is necessary to
52    decide whether to use |above_display_skip| or |above_display_short_skip| before a displayed
53    formula.
54
55    We have an additional parameter |\parfillleftskip| and below we cheat a bit. We add two glue
56    nodes so that the par builder will work the same and doesn't need to be adapted, but when we're
57    done we move the leftbound node to the beginning of the (last) line.
58
59    Remark for myself: the \LUA\ variant that I use(d) for playing around occasionally is no longer
60    in sync with the code here so it's abandoned.
61
62    I played a bit with prerolling: make a copy, run the par builder, afterwards collect the
63    result in a box that then can be consulted: wd, ht, dp, quality, hyphens, and especially
64    shape fitting (which was the reason, because |\hangafter| assumes lines and esp with math a
65    line is somewhat unpredictable so we get bad fitting). In the end I decided that it was kind
66    of useless because of the unlikely usage scenario. Of course it can be done in \LUA\ but we
67    don't want the associated performance hit (management overhead) and dealing with (progressive)
68    solutions oscillating is also an issue. In the meantime we got par passes so prerolling no
69    longer is on the agenda.
70
71*/
72
73/*tex
74
75    A in-depth explanation abotu multiple par passes can be found in articles and wrapups by Mikael
76    and Hans. Here is enough to know that as an alternative to the (upto) three passes that regular
77    \TEX\ has, we can have many more. When no criteria are set the first two passes are regular
78    passes, otherwise we only have what are called \quote {specification} passes.
79
80    Apart from more passes, we also provide ways to set up more than five badness (fit) classes and
81    use different demerits per distance between classes. There is also support for exceptions, more
82    controlled looseness, etc. Introspective callbacks can be used to get more insight in the way
83    the ending does all this.
84
85    Although default behavior is (of course) present, the code below has way more lines than the
86    original but one should be able to recognize plenty of Knuthian code (and comments, as these
87    still apply).
88
89*/
90
91typedef enum linebreak_states {
92    linebreak_no_pass,
93    linebreak_first_pass,
94    linebreak_second_pass,
95    linebreak_final_pass,
96    linebreak_specification_pass,
97} linebreak_states;
98
99linebreak_state_info lmt_linebreak_state = {
100    .just_box                     = 0,
101    .last_line_fill               = 0,
102    .no_shrink_error_yet          = 0,
103    .callback_id                  = 0,
104    .threshold                    = 0,
105    .adjust_spacing               = 0,
106    .adjust_spacing_step          = 0,
107    .adjust_spacing_shrink        = 0,
108    .adjust_spacing_stretch       = 0,
109    .current_font_step            = 0,
110    .passive                      = 0,
111    .printed_node                 = 0,
112    .serial_number                = 0,
113    .active_width                 = { 0 },
114    .background                   = { 0 },
115    .break_width                  = { 0 },
116    .internal_interline_penalty   = 0,
117    .internal_broken_penalty      = 0,
118    .internal_left_box            = null,
119    .internal_left_box_width      = 0,
120    .internal_left_box_init       = 0,
121    .internal_left_box_width_init = 0,
122    .internal_right_box           = null,
123    .internal_right_box_width     = 0,
124    .internal_middle_box          = null,
125    .disc_width                   = { 0 },
126    .minimal_demerits             = { 0 },
127    .minimum_demerits             = 0,
128    .easy_line                    = 0,
129    .last_special_line            = 0,
130    .first_width                  = 0,
131    .second_width                 = 0,
132    .first_indent                 = 0,
133    .second_indent                = 0,
134    .best_bet                     = 0,
135    .fewest_demerits              = 0,
136    .best_line                    = 0,
137    .actual_looseness             = 0,
138    .do_last_line_fit             = 0,
139    .fill_width                   = { 0 },
140    .dir_ptr                      = 0,
141    .warned                       = 0,
142    .calling_back                 = 0,
143    .saved_threshold              = 0,
144    .global_threshold             = 0,
145    .line_break_dir               = 0,
146    .checked_expansion            = -1,
147    .passes                       = { { 0, 0, 0, 0, 0, 0, 0, 0 } },
148    .n_of_left_twins              = 0,
149    .n_of_right_twins             = 0,
150    .n_of_double_twins            = 0,
151    .internal_par_node            = null,
152    .current_line_number          = 0,
153    .has_orphans                  = 0,
154    .has_toddlers                 = 0,
155};
156
157/*tex
158    We could use a bit larger array and glue_orders where normal starts at 0 so we then need a larger
159    array. Let's not do that now.
160*/
161
162typedef enum fill_orders {
163    fi_order    = 0,
164    fil_order   = 1,
165    fill_order  = 2,
166    filll_order = 3,
167} fill_orders;
168
169/*tex
170
171    The |just_box| variable has the |hlist_node| for the last line of the new paragraph. In it's
172    complete form, |line_break| is a rather lengthy procedure --- sort of a small world unto itself
173    --- we must build it up little by little. Below you see only the general outline. The main task
174    performed here is to move the list from |head| to |temp_head| and go into the enclosing semantic
175    level. We also append the |\parfillskip| glue to the end of the paragraph, removing a space (or
176    other glue node) if it was there, since spaces usually precede blank lines and instances of
177    |$$|. The |par_fill_skip| is preceded by an infinite penalty, so it will never be considered as
178    a potential breakpoint.
179
180 */
181
182void tex_line_break_prepare(
183    halfword par,
184    halfword *tail,
185    halfword *parinit_left_skip_glue,
186    halfword *parinit_right_skip_glue,
187    halfword *parfill_left_skip_glue,
188    halfword *parfill_right_skip_glue,
189    halfword *final_line_penalty
190)
191{
192    /* too much testing of next .. it needs checking anyway */
193    if (node_type(par) == par_node) { /* maybe check for h|v subtype */
194        if (tracing_linebreak_lists) {
195            tex_begin_diagnostic();
196            tex_print_format("[linebreak: prepare, before]");
197            tex_show_box(par);
198            tex_end_diagnostic();
199        }
200        *tail = *tail ? *tail : tex_tail_of_node_list(par);
201        *final_line_penalty = tex_new_penalty_node(infinite_penalty, line_penalty_subtype);
202        *parfill_left_skip_glue = tex_new_glue_node(tex_get_par_par(par, par_par_fill_left_skip_code), par_fill_left_skip_glue);
203        *parfill_right_skip_glue = tex_new_glue_node(tex_get_par_par(par, par_par_fill_right_skip_code), par_fill_right_skip_glue);
204        *parinit_left_skip_glue = null;
205        *parinit_right_skip_glue = null;
206        while (par != *tail && node_type(*tail) == glue_node && ! tex_is_par_init_glue(*tail)) {
207            halfword prev = node_prev(*tail);
208            node_next(prev) = null;
209            tex_flush_node(*tail);
210            *tail = prev;
211            if (! normalize_par_mode_option(remove_trailing_spaces_mode)) {
212                break;
213            }
214        }
215        tex_add_penalty_option(*final_line_penalty, penalty_option_end_of_par);
216        tex_attach_attribute_list_copy(*final_line_penalty, par);
217        tex_attach_attribute_list_copy(*parfill_left_skip_glue, par);
218        tex_attach_attribute_list_copy(*parfill_right_skip_glue, par);
219        tex_try_couple_nodes(*tail, *final_line_penalty);
220        tex_try_couple_nodes(*final_line_penalty, *parfill_left_skip_glue);
221        tex_try_couple_nodes(*parfill_left_skip_glue, *parfill_right_skip_glue);
222        *tail = *parfill_right_skip_glue;
223        if (node_next(par)) {
224            halfword p = par;
225            halfword n = node_next(par);
226            while (node_next(p) && node_type(node_next(p)) == dir_node) {
227                p = node_next(p);
228            }
229            while (n) {
230                if (node_type(n) == glue_node && node_subtype(n) == indent_skip_glue) {
231                    *parinit_left_skip_glue = tex_new_glue_node(tex_get_par_par(par, par_par_init_left_skip_code), par_init_left_skip_glue);
232                    *parinit_right_skip_glue = tex_new_glue_node(tex_get_par_par(par, par_par_init_right_skip_code), par_init_right_skip_glue);
233                    tex_attach_attribute_list_copy(*parinit_left_skip_glue, par);
234                    tex_attach_attribute_list_copy(*parinit_right_skip_glue, par);
235                    tex_try_couple_nodes(*parinit_right_skip_glue, n);
236                    tex_try_couple_nodes(*parinit_left_skip_glue, *parinit_right_skip_glue);
237                 // tex_try_couple_nodes(par, *parinit_left_skip_glue);
238                    tex_try_couple_nodes(p, *parinit_left_skip_glue);
239                    break;
240                } else {
241                    n = node_next(n); /* sort of weird and tricky */
242                }
243            }
244        }
245        if (tracing_linebreak_lists) {
246            tex_begin_diagnostic();
247            tex_print_format("[linebreak: prepare, after]");
248            tex_show_box(par);
249            tex_end_diagnostic();
250        }
251    }
252}
253
254void tex_line_break(int group_context, int par_context, int display_math)
255{
256    halfword head = node_next(cur_list.head);
257    /*tex There should be a local par node at the beginning! */
258    if (node_type(head) == par_node) { /* maybe check for h|v subtype */
259        /*tex We need this for over- or underfull box messages. */
260        halfword tail = cur_list.tail;
261        lmt_packaging_state.pack_begin_line = cur_list.mode_line;
262        node_prev(head) = null;
263        /*tex Hyphenate, driven by callback or fallback to normal \TEX. */
264        if (tex_list_has_glyph(head)) {
265            tex_handle_hyphenation(head, tail);
266            head = tex_handle_glyphrun(head, group_context, par_dir(head));
267            tail = tex_tail_of_node_list(head);
268            tex_try_couple_nodes(cur_list.head, head);
269            cur_list.tail = tail;
270        }
271        /*tex We remove (only one) trailing glue node, when present. */
272     // if (head != tail && node_type(tail) == glue_node && ! tex_is_par_init_glue(tail)) {
273     //     halfword prev = node_prev(tail);
274     //     node_next(prev) = null;
275     //     tex_flush_node(tail);
276     //     cur_list.tail = prev;
277     // }
278        node_next(temp_head) = head;
279        /*tex There should be a local par node at the beginning! */
280        if (node_type(head) == par_node) { /* maybe check for h|v subtype */
281            /*tex
282                The tail thing is a bit weird here as it's not the tail. One day I will look into
283                this. One complication is that we have the normal break routing or a callback that
284                replaces it but that callback can call the normal routine itself with specific
285                parameters set.
286            */
287            halfword start_of_par;
288            halfword par = head;
289            halfword parinit_left_skip_glue = null;
290            halfword parinit_right_skip_glue = null;
291            halfword parfill_left_skip_glue = null;
292            halfword parfill_right_skip_glue = null;
293            halfword final_line_penalty = null;
294            tex_line_break_prepare(par, &tail, &parinit_left_skip_glue, &parinit_right_skip_glue, &parfill_left_skip_glue, &parfill_right_skip_glue, &final_line_penalty);
295            cur_list.tail = tail;
296            /*tex
297                We start with a prepared list. If you mess with that the linebreak routine might not
298                work well especially if the pointers are messed up. So be it.
299            */
300            lmt_node_filter_callback(pre_linebreak_filter_callback, group_context, temp_head, &(cur_list.tail));
301            /*tex
302                We assume that the list is still okay.
303            */
304            lmt_linebreak_state.last_line_fill = cur_list.tail;
305            tex_pop_nest();
306            start_of_par = cur_list.tail;
307            lmt_linebreak_state.calling_back = 1;
308            if (lmt_linebreak_callback(temp_head, display_math, &(cur_list.tail))) {
309                /*tex
310                    When we end up here we have a prepared list so we need to make sure that when
311                    the callback usaes that list with the built in break routine we don't do that
312                    twice. One should work on copies! Afterwards we need to find the correct value
313                    for the |just_box|.
314                */
315                halfword box_search = cur_list.tail;
316                lmt_linebreak_state.just_box  = null;
317                if (box_search) {
318                    do {
319                        if (node_type(box_search) == hlist_node) {
320                           lmt_linebreak_state.just_box = box_search;
321                        }
322                        box_search = node_next(box_search);
323                    } while (box_search);
324                }
325                if (! lmt_linebreak_state.just_box) {
326                    tex_handle_error(
327                        succumb_error_type,
328                        "Invalid linebreak_filter",
329                        "A linebreaking routine should return a non-empty list of nodes and at least one\n"
330                        "of those has to be a \\hbox. Sorry, I cannot recover from this."
331                    );
332                }
333            } else {
334                line_break_properties properties = {
335                    .initial_par             = par,
336                    .group_context           = group_context,
337                    .par_context             = par_context,
338                    .tracing_paragraphs      = tracing_paragraphs_par,
339                    .tracing_fitness         = tracing_fitness_par,
340                    .tracing_passes          = tracing_passes_par,
341                    .tracing_toddlers        = tracing_toddlers_par,
342                    .tracing_orphans         = tracing_orphans_par,
343                    .paragraph_dir           = par_dir(par),
344                    .paragraph_options       = par_options(par),
345                    .parfill_left_skip       = parfill_left_skip_glue,
346                    .parfill_right_skip      = parfill_right_skip_glue,
347                    .parinit_left_skip       = parinit_left_skip_glue,
348                    .parinit_right_skip      = parinit_right_skip_glue,
349                    .tolerance               = tex_get_par_par(par, par_tolerance_code),
350                    .emergency_stretch       = tex_get_par_par(par, par_emergency_stretch_code),
351                    .emergency_original      = 0, /*tex This one is set afterwards. */
352                    .looseness               = tex_get_par_par(par, par_looseness_code),
353                    .adjust_spacing          = tex_get_par_par(par, par_adjust_spacing_code),
354                    .protrude_chars          = tex_get_par_par(par, par_protrude_chars_code),
355                    .adj_demerits            = tex_get_par_par(par, par_adj_demerits_code),
356                    .line_penalty            = tex_get_par_par(par, par_line_penalty_code),
357                    .last_line_fit           = tex_get_par_par(par, par_last_line_fit_code),
358                    .double_hyphen_demerits  = tex_get_par_par(par, par_double_hyphen_demerits_code),
359                    .final_hyphen_demerits   = tex_get_par_par(par, par_final_hyphen_demerits_code),
360                    .hsize                   = tex_get_par_par(par, par_hsize_code),
361                    .left_skip               = tex_get_par_par(par, par_left_skip_code),
362                    .right_skip              = tex_get_par_par(par, par_right_skip_code),
363                    .emergency_left_skip     = tex_get_par_par(par, par_emergency_left_skip_code),
364                    .emergency_right_skip    = tex_get_par_par(par, par_emergency_right_skip_code),
365                    .pretolerance            = tex_get_par_par(par, par_pre_tolerance_code),
366                    .hang_indent             = tex_get_par_par(par, par_hang_indent_code),
367                    .hang_after              = tex_get_par_par(par, par_hang_after_code),
368                    .par_shape               = tex_get_par_par(par, par_par_shape_code),
369                    .inter_line_penalty      = tex_get_par_par(par, par_inter_line_penalty_code),
370                    .inter_line_penalties    = tex_get_par_par(par, par_inter_line_penalties_code),
371                    .club_penalty            = tex_get_par_par(par, par_club_penalty_code),
372                    .club_penalties          = tex_get_par_par(par, par_club_penalties_code),
373                    .widow_penalty           = tex_get_par_par(par, par_widow_penalty_code),
374                    .widow_penalties         = tex_get_par_par(par, par_widow_penalties_code),
375                    .display_widow_penalty   = tex_get_par_par(par, par_display_widow_penalty_code),
376                    .display_widow_penalties = tex_get_par_par(par, par_display_widow_penalties_code),
377                    .broken_penalties        = tex_get_par_par(par, par_broken_penalties_code),
378                    .toddler_penalties       = tex_get_par_par(par, par_toddler_penalties_code),
379                    .left_twin_demerits      = tex_get_par_par(par, par_left_twin_demerits_code),
380                    .right_twin_demerits     = tex_get_par_par(par, par_right_twin_demerits_code),
381                    .single_line_penalty     = tex_get_par_par(par, par_single_line_penalty_code),
382                    .hyphen_penalty          = tex_get_par_par(par, par_hyphen_penalty_code),
383                    .ex_hyphen_penalty       = tex_get_par_par(par, par_ex_hyphen_penalty_code),
384                    .fitness_classes         = tex_get_par_par(par, par_fitness_classes_code),
385                    .adjacent_demerits       = tex_get_par_par(par, par_adjacent_demerits_code),
386                    .orphan_line_factors     = tex_get_par_par(par, par_orphan_line_factors_code),
387                    .orphan_penalties        = tex_get_par_par(par, par_orphan_penalties_code),
388                    .broken_penalty          = tex_get_par_par(par, par_broken_penalty_code),
389                    .baseline_skip           = tex_get_par_par(par, par_baseline_skip_code),
390                    .line_skip               = tex_get_par_par(par, par_line_skip_code),
391                    .line_skip_limit         = tex_get_par_par(par, par_line_skip_limit_code),
392                    .adjust_spacing_step     = tex_get_par_par(par, par_adjust_spacing_step_code),
393                    .adjust_spacing_shrink   = tex_get_par_par(par, par_adjust_spacing_shrink_code),
394                    .adjust_spacing_stretch  = tex_get_par_par(par, par_adjust_spacing_stretch_code),
395                    .hyphenation_mode        = tex_get_par_par(par, par_hyphenation_mode_code),
396                    .shaping_penalties_mode  = tex_get_par_par(par, par_shaping_penalties_mode_code),
397                    .shaping_penalty         = tex_get_par_par(par, par_shaping_penalty_code),
398                    .emergency_extra_stretch = tex_get_par_par(par, par_emergency_extra_stretch_code),
399                    .par_passes              = line_break_passes_par > 0 ? tex_get_par_par(par, par_par_passes_code) : 0,
400                    .line_break_checks       = tex_get_par_par(par, par_line_break_checks_code),
401                    .extra_hyphen_penalty    = 0,
402                    .line_break_optional     = line_break_optional_par, /* hm, why different than above */
403                    .math_penalty_factor     = 0,
404                    .sf_factor               = 0,
405                    .sf_stretch_factor       = 0,
406                    .max_adj_demerits        = 0,
407                };
408             /* properties.emergency_original = properties.emergency_stretch; */
409                tex_do_line_break(&properties);
410                /*tex
411                    We assume that the list is still okay when we do some post line break stuff.
412                */
413            }
414            lmt_linebreak_state.calling_back = 0;
415            lmt_node_filter_callback(post_linebreak_filter_callback, group_context, start_of_par, &(cur_list.tail));
416            lmt_packaging_state.pack_begin_line = 0;
417            return;
418        }
419    }
420    tex_confusion("missing local par node");
421}
422
423/*tex
424
425    Glue nodes in a horizontal list that is being paragraphed are not supposed to include \quote
426    {infinite} shrinkability; that is why the algorithm maintains four registers for stretching but
427    only one for shrinking. If the user tries to introduce infinite shrinkability, the shrinkability
428    will be reset to finite and an error message will be issued. A boolean variable
429    |no_shrink_error_yet| prevents this error message from appearing more than once per paragraph.
430
431    Beware, this does an in-place fix to the glue (which can be a register!). As we store glues a
432    bit different we do a different fix here.
433
434*/
435
436static scaled tex_aux_checked_shrink(halfword p)
437{
438    if (glue_shrink(p) && glue_shrink_order(p) != normal_glue_order) {
439        if (lmt_linebreak_state.no_shrink_error_yet) {
440            lmt_linebreak_state.no_shrink_error_yet = 0;
441            tex_handle_error(
442                normal_error_type,
443                "Infinite glue shrinkage found in a paragraph",
444                "The paragraph just ended includes some glue that has infinite shrinkability,\n"
445                "e.g., '\\hskip 0pt minus 1fil'. Such glue doesn't belong there---it allows a\n"
446                "paragraph of any length to fit on one line. But it's safe to proceed, since the\n"
447                "offensive shrinkability has been made finite."
448            );
449        }
450        glue_shrink_order(p) = normal_glue_order;
451    }
452    return glue_shrink(p);
453}
454
455/*tex
456
457    A pointer variable |cur_p| runs through the given horizontal list as we look for breakpoints.
458    This variable is global, since it is used both by |line_break| and by its subprocedure
459    |try_break|.
460
461    Another global variable called |threshold| is used to determine the feasibility of individual
462    lines: breakpoints are feasible if there is a way to reach them without creating lines whose
463    badness exceeds |threshold|. (The badness is compared to |threshold| before penalties are
464    added, so that penalty values do not affect the feasibility of breakpoints, except that no
465    break is allowed when the penalty is 10000 or more.) If |threshold| is 10000 or more, all
466    legal breaks are considered feasible, since the |badness| function specified above never
467    returns a value greater than~10000.
468
469    Up to three passes might be made through the paragraph in an attempt to find at least one set
470    of feasible breakpoints. On the first pass, we have |threshold = pretolerance| and |second_pass
471    = final_pass = false|. If this pass fails to find a feasible solution, |threshold| is set to
472    |tolerance|, |second_pass| is set |true|, and an attempt is made to hyphenate as many words as
473    possible. If that fails too, we add |emergency_stretch| to the background stretchability and
474    set |final_pass = true|.
475
476    |second_pass| is this our second attempt to break this paragraph and |final_path| our final
477    attempt to break this paragraph while |threshold| is the maximum badness on feasible lines.
478
479    The maximum fill level for |hlist_stack|. Maybe good if larger than |2 * max_quarterword|, so
480    that box nesting level would overflow first. The stack for |find_protchar_left()| and
481    |find_protchar_right()|; |hlist_stack_level| is the fill level for |hlist_stack|
482
483*/
484
485# define max_hlist_stack 512
486
487/* We can optimize this when we have a global setting. */
488
489static inline int  tex_has_glyph_expansion(halfword a)
490{
491    return
492        ! ((glyph_options(a) & glyph_option_no_expansion) == glyph_option_no_expansion)
493        && has_font_text_control(glyph_font(a), text_control_expansion);
494}
495
496/*tex
497
498    Search left to right from list head |l|, returns 1st non-skipable item:
499
500*/
501
502static halfword tex_aux_find_protchar_left(halfword l, int d)
503{
504    int done = 0 ;
505    halfword initial = l;
506    while (node_next(l) && node_type(l) == hlist_node && tex_zero_box_dimensions(l) && ! box_list(l)) {
507        /*tex For paragraph start with |\parindent = 0pt| or any empty hbox. */
508        l = node_next(l);
509        done = 1 ;
510    }
511    if (! done && node_type(l) == par_node) { /* maybe check for h|v subtype */
512        l = node_next(l);
513        done = 1 ;
514    }
515    if (! done && d) {
516        while (node_next(l) && ! (node_type(l) == glyph_node || non_discardable(l))) {
517            /*tex standard discardables at line break, \TEX book, p 95 */
518            l = node_next(l);
519        }
520    }
521    if (node_type(l) != glyph_node) {
522        halfword t;
523        int run = 1;
524        halfword hlist_stack[max_hlist_stack];
525        int hlist_stack_level = 0;
526        do {
527            t = l;
528            while (run && node_type(l) == hlist_node && box_list(l)) {
529                if (hlist_stack_level >= max_hlist_stack) {
530                 /* return tex_normal_error("push_node", "stack overflow"); */
531                    return initial;
532                } else {
533                    hlist_stack[hlist_stack_level++] = l;
534                }
535                l = box_list(l);
536            }
537            while (run && tex_protrusion_skipable(l)) {
538                while (! node_next(l) && hlist_stack_level > 0) {
539                    /*tex Don't visit this node again. */
540                    if (hlist_stack_level <= 0) {
541                        /*tex This can point to some bug. */
542                     /* return tex_normal_error("pop_node", "stack underflow (internal error)"); */
543                        return initial;
544                    } else {
545                        l = hlist_stack[--hlist_stack_level];
546                    }
547                    run = 0;
548                }
549                if (node_next(l) && node_type(l) == boundary_node && node_subtype(l) == protrusion_boundary && (boundary_data(l) == protrusion_skip_next || boundary_data(l) == protrusion_skip_both)) {
550                    /*tex Skip next node. */
551                    l = node_next(l);
552                }
553                if (node_next(l)) {
554                    l = node_next(l);
555                } else if (hlist_stack_level == 0) {
556                    run = 0;
557                }
558            }
559        } while (t != l);
560    }
561    return l;
562}
563
564/*tex
565
566    Search right to left from list tail |r| to head |l|, returns 1st non-skipable item.
567
568*/
569
570static halfword tex_aux_find_protchar_right(halfword l, halfword r)
571{
572    if (r) {
573        halfword t;
574        int run = 1;
575        halfword initial = r;
576        halfword hlist_stack[max_hlist_stack];
577        int hlist_stack_level = 0;
578        do {
579            t = r;
580            while (run && node_type(r) == hlist_node && box_list(r)) {
581                if (hlist_stack_level >= max_hlist_stack) {
582                 /* tex_normal_error("push_node", "stack overflow"); */
583                    return initial;
584                } else {
585                    hlist_stack[hlist_stack_level++] = l;
586                }
587                if (hlist_stack_level >= max_hlist_stack) {
588                 /* tex_normal_error("push_node", "stack overflow"); */
589                    return initial;
590                } else {
591                    hlist_stack[hlist_stack_level++] = r;
592                }
593                l = box_list(r);
594                r = l;
595                while (node_next(r)) {
596                    halfword s = r;
597                    r = node_next(r);
598                    node_prev(r) = s;
599                }
600            }
601            while (run && tex_protrusion_skipable(r)) {
602                while (r == l && hlist_stack_level > 0) {
603                    /*tex Don't visit this node again. */
604                    if (hlist_stack_level <= 0) {
605                        /*tex This can point to some bug. */
606                        /* return tex_normal_error("pop_node", "stack underflow (internal error)"); */
607                        return initial;
608                    } else {
609                        r = hlist_stack[--hlist_stack_level];
610                    }
611
612                    if (hlist_stack_level <= 0) {
613                        /*tex This can point to some bug. */
614                     /* return tex_normal_error("pop_node", "stack underflow (internal error)"); */
615                        return initial;
616                    } else {
617                        l = hlist_stack[--hlist_stack_level];
618                    }
619                }
620                if ((r != l) && r) {
621                    if (node_prev(r) && node_type(r) == boundary_node && node_subtype(r) == protrusion_boundary && (boundary_data(r) == protrusion_skip_previous || boundary_data(r) == protrusion_skip_both)) {
622                        /*tex Skip next node. */
623                        r = node_prev(r);
624                    }
625                    if (node_prev(r)) {
626                        r = node_prev(r);
627                    } else {
628                        /*tex This is the input: |\leavevmode \penalty -10000 \penalty -10000| */
629                        run = 0;
630                    }
631                } else if (r == l && hlist_stack_level == 0) {
632                    run = 0;
633                }
634            }
635        } while (t != r);
636    }
637    return r;
638}
639
640/*tex
641
642    The algorithm essentially determines the best possible way to achieve each feasible combination
643    of position, line, and fitness. Thus, it answers questions like, \quotation {What is the best
644    way to break the opening part of the paragraph so that the fourth line is a tight line ending at
645    such-and-such a place?} However, the fact that all lines are to be the same length after a
646    certain point makes it possible to regard all sufficiently large line numbers as equivalent, when
647    the looseness parameter is zero, and this makes it possible for the algorithm to save space and
648    time.
649
650    An \quote {active node} and a \quote {passive node} are created in |mem| for each feasible
651    breakpoint that needs to be considered. Active nodes are three words long and passive nodes
652    are two words long. We need active nodes only for breakpoints near the place in the
653    paragraph that is currently being examined, so they are recycled within a comparatively short
654    time after they are created.
655
656    An active node for a given breakpoint contains six fields:
657
658    \startitemize[n]
659
660        \startitem
661            |vlink| points to the next node in the list of active nodes; the last active node has
662            |vlink=active|.
663        \stopitem
664
665        \startitem
666            |break_node| points to the passive node associated with this breakpoint.
667        \stopitem
668
669        \startitem
670            |line_number| is the number of the line that follows this breakpoint.
671        \stopitem
672
673        \startitem
674            |fitness| is the fitness classification of the line ending at this breakpoint.
675        \stopitem
676
677        \startitem
678            |type| is either |hyphenated_node| or |unhyphenated_node|, depending on whether this
679            breakpoint is a |disc_node|.
680        \stopitem
681
682        \startitem
683            |total_demerits| is the minimum possible sum of demerits over all lines leading from
684            the beginning of the paragraph to this breakpoint.
685        \stopitem
686
687    \stopitemize
688
689    The value of |node_next(active)| points to the first active node on a vlinked list of all currently
690    active nodes. This list is in order by |line_number|, except that nodes with |line_number >
691    easy_line| may be in any order relative to each other.
692
693*/
694
695void tex_initialize_active(void)
696{
697    node_type(active_head) = hyphenated_node;
698    active_line_number(active_head) = max_halfword;
699    /*tex
700        The |subtype| is actually the |fitness|. It is set with |new_node| to one of the fitness
701        values.
702    */
703    active_fitness(active_head) = default_fitness;
704    active_n_of_fitness_classes(active_head) = 5; /* the minimum */
705}
706
707/*tex
708
709    The passive node for a given breakpoint contains eight fields:
710
711    \startitemize
712
713        \startitem
714            |vlink| points to the passive node created just before this one, if any, otherwise it
715            is |null|.
716        \stopitem
717
718        \startitem
719            |cur_break| points to the position of this breakpoint in the horizontal list for the
720            paragraph being broken.
721        \stopitem
722
723        \startitem
724            |prev_break| points to the passive node that should precede this one in an optimal path
725            to this breakpoint.
726        \stopitem
727
728        \startitem
729            |serial| is equal to |n| if this passive node is the |n|th one created during the
730            current pass. (This field is used only when printing out detailed statistics about the
731            line-breaking calculations.)
732        \stopitem
733
734        \startitem
735            |passive_interline_penalty| holds the current |localinterlinepenalty|
736        \stopitem
737
738        \startitem
739            |passive_broken_penalty| holds the current |localbrokenpenalty|
740        \stopitem
741
742    \stopitemize
743
744    There is a global variable called |passive| that points to the most recently created passive
745    node. Another global variable, |printed_node|, is used to help print out the paragraph when
746    detailed information about the line-breaking computation is being displayed.
747
748    The most recent node on passive list, the most recent node that has been printed, and the number
749    of passive nodes allocated on this pass, is registered in the passive field.
750
751    The active list also contains \quote {delta} nodes that help the algorithm compute the badness
752    of individual lines. Such nodes appear only between two active nodes, and they have |type =
753    delta_node|. If |p| and |r| are active nodes and if |q| is a delta node between them, so that
754    |vlink (p) = q| and |vlink (q) = r|, then |q| tells the space difference between lines in the
755    horizontal list that start after breakpoint |p| and lines that start after breakpoint |r|. In
756    other words, if we know the length of the line that starts after |p| and ends at our current
757    position, then the corresponding length of the line that starts after |r| is obtained by adding
758    the amounts in node~|q|. A delta node contains seven scaled numbers, since it must record the
759    net change in glue stretchability with respect to all orders of infinity. The natural width
760    difference appears in |mem[q+1].sc|; the stretch differences in units of pt, sfi, fil, fill,
761    and filll appear in |mem[q + 2 .. q + 6].sc|; and the shrink difference appears in |mem[q +
762    7].sc|. The |subtype| field of a delta node is not used.
763
764    {\em NB: Actually, we have more fields now.}
765
766    As the algorithm runs, it maintains a set of seven delta-like registers for the length of the
767    line following the first active breakpoint to the current position in the given hlist. When it
768    makes a pass through the active list, it also maintains a similar set of seven registers for
769    the length following the active breakpoint of current interest. A third set holds the length
770    of an empty line (namely, the sum of |\leftskip| and |\rightskip|); and a fourth set is used
771    to create new delta nodes.
772
773    When we pass a delta node we want to do operations like:
774
775    \starttyping
776    for k := 1 to 7 do
777        cur_active_width[k] := cur_active_width[k] + mem[q+k].sc|};
778    \stoptyping
779
780    and we want to do this without the overhead of |for| loops so we use update macros.
781
782    |active_width| is he distance from first active node to~|cur_p|, |background| the length of an
783    \quote {empty} line, and |break_width| the length being computed after current break.
784
785    We make |auto_breaking| accessible out of |line_break|.
786
787    Let's state the principles of the delta nodes more precisely and concisely, so that the
788    following programs will be less obscure. For each legal breakpoint~|p| in the paragraph, we
789    define two quantities $\alpha(p)$ and $\beta(p)$ such that the length of material in a line
790    from breakpoint~|p| to breakpoint~|q| is $\gamma+\beta(q)-\alpha(p)$, for some fixed $\gamma$.
791    Intuitively, $\alpha(p)$ and $\beta(q)$ are the total length of material from the beginning
792    of the paragraph to a point after a break at |p| and to a point before a break at |q|; and
793    $\gamma$ is the width of an empty line, namely the length contributed by |\leftskip| and
794    |\rightskip|.
795
796    Suppose, for example, that the paragraph consists entirely of alternating boxes and glue
797    skips; let the boxes have widths $x_1\ldots x_n$ and let the skips have widths $y_1\ldots
798    y_n$, so that the paragraph can be represented by $x_1y_1\ldots x_ny_n$. Let $p_i$ be the
799    legal breakpoint at $y_i$; then $\alpha(p_i) = x_1 + y_1 + \cdots + x_i + y_i$, and $\beta
800    (p_i) = x_1 + y_1 + \cdots + x_i$. To check this, note that the length of material from
801    $p_2$ to $p_5$, say, is $\gamma + x_3 + y_3 + x_4 + y_4 + x_5 = \gamma + \beta (p_5) -
802    \alpha (p_2)$.
803
804    The quantities $\alpha$, $\beta$, $\gamma$ involve glue stretchability and shrinkability as
805    well as a natural width. If we were to compute $\alpha(p)$ and $\beta(p)$ for each |p|, we
806    would need multiple precision arithmetic, and the multiprecise numbers would have to be kept
807    in the active nodes. \TeX\ avoids this problem by working entirely with relative differences
808    or \quote {deltas}. Suppose, for example, that the active list contains $a_1\,\delta_1\,a_2\,
809    \delta_2\,a_3$, where the |a|'s are active breakpoints and the $\delta$'s are delta nodes.
810    Then $\delta_1 = \alpha(a_1) - \alpha(a_2)$ and $\delta_2 = \alpha(a_2) - \alpha(a_3)$. If the
811    line breaking algorithm is currently positioned at some other breakpoint |p|, the |active_width|
812    array contains the value $\gamma  +\beta(p) - \alpha(a_1)$. If we are scanning through the list
813    of active nodes and considering a tentative line that runs from $a_2$ to~|p|, say, the
814    |cur_active_width| array will contain the value $\gamma + \beta(p) - \alpha(a_2)$. Thus, when we
815    move from $a_2$ to $a_3$, we want to add $\alpha(a_2) - \alpha(a_3)$ to |cur_active_width|; and
816    this is just $\delta_2$, which appears in the active list between $a_2$ and $a_3$. The
817    |background| array contains $\gamma$. The |break_width| array will be used to calculate values
818    of new delta nodes when the active list is being updated.
819
820    The heart of the line-breaking procedure is |try_break|, a subroutine that tests if the current
821    breakpoint |cur_p| is feasible, by running through the active list to see what lines of text
822    can be made from active nodes to~|cur_p|. If feasible breaks are possible, new break nodes are
823    created. If |cur_p| is too far from an active node, that node is deactivated.
824
825    The parameter |pi| to |try_break| is the penalty associated with a break at |cur_p|; we have
826    |pi = eject_penalty| if the break is forced, and |pi = inf_penalty| if the break is illegal.
827
828    The other parameter, |break_type|, is set to |hyphenated_node| or |unhyphenated_node|, depending
829    on whether or not the current break is at a |disc_node|. The end of a paragraph is also regarded
830    as |hyphenated_node|; this case is distinguishable by the condition |cur_p = null|.
831
832    \startlines
833    |internal_interline_penalty|   running |\localinterlinepenalty|
834    |internal_broken_penalty|      running |\localbrokenpenalty|
835    |internal_left_box|            running |\localleftbox|
836    |internal_left_box_width|      running |\localleftbox|
837    |init_internal_left_box|       running |\localleftbox|
838    |init_internal_left_box_width| running |\localleftbox| width
839    |internal_right_box|           running |\localrightbox|
840    |internal_right_box_width|     running |\localrightbox| width
841    |disc_width|                   the length of discretionary material preceding a break
842    \stoplines
843
844    As we consider various ways to end a line at |cur_p|, in a given line number class, we keep
845    track of the best total demerits known, in an array with one entry for each of the fitness
846    classifications. For example, |minimal_demerits [tight_fit]| contains the fewest total
847    demerits of feasible line breaks ending at |cur_p| with a |tight_fit| line; |best_place
848    [tight_fit]| points to the passive node for the break before |cur_p| that achieves such an
849    optimum; and |best_pl_line[tight_fit]| is the |line_number| field in the active node
850    corresponding to |best_place [tight_fit]|. When no feasible break sequence is known, the
851    |minimal_demerits| entries will be equal to |awful_bad|, which is $2^{30}-1$. Another variable,
852    |minimum_demerits|, keeps track of the smallest value in the |minimal_demerits| array.
853
854    The length of lines depends on whether the user has specified |\parshape| or |\hangindent|. If
855    |par_shape_ptr| is not null, it points to a $(2n+1)$-word record in |mem|, where the |vinfo|
856    in the first word contains the value of |n|, and the other $2n$ words contain the left margins
857    and line lengths for the first |n| lines of the paragraph; the specifications for line |n|
858    apply to all subsequent lines. If |par_shape_ptr = null|, the shape of the paragraph depends on
859    the value of |n = hang_after|; if |n >= 0|, hanging indentation takes place on lines |n + 1|,
860    |n + 2|, \dots, otherwise it takes place on lines 1, \dots, $\vert n\vert$. When hanging
861    indentation is active, the left margin is |hang_indent|, if |hang_indent >= 0|, else it is 0;
862    the line length is $|hsize|-\vert|hang_indent|\vert$. The normal setting is |par_shape_ptr =
863    null|, |hang_after = 1|, and |hang_indent = 0|. Note that if |hang_indent = 0|, the value of
864    |hang_after| is irrelevant.
865
866    Some more variables and remarks:
867
868    line numbers |> easy_line| are equivalent in break nodes
869
870    line numbers |> last_special_line| all have the same width
871
872    |first_width| is the width of all lines |<= last_special_line|, if no |\parshape| has been
873    specified
874
875    |second_width| is the width of all lines |> last_special_line|
876
877    |first_indent| is the left margin to go with |first_width|
878
879    |second_indent| s the left margin to go with |second_width|
880
881    |best_bet| indicated the passive node and its predecessors
882
883    |fewest_demerits| are the demerits associated with |best_bet|
884
885    |best_line| is the line number following the last line of the new paragraph
886
887    |actual_looseness| is the difference between |line_number (best_bet)| and the optimum
888    |best_line|
889
890    |line_diff| is the difference between the current line number and the optimum |best_line|
891
892    \TEX\ makes use of the fact that |hlist_node|, |vlist_node|, |rule_node|, |insert_node|,
893    |mark_node|, |adjust_node|, |disc_node|, |whatsit_node|, and |math_node| are at the low end of
894    the type codes, by permitting a break at glue in a list if and only if the |type| of the
895    previous node is less  than |math_node|. Furthermore, a node is discarded after a break if its
896    type is |math_node| or~more.
897
898*/
899
900static void tex_aux_clean_up_the_memory(void)
901{
902    halfword q = node_next(active_head);
903    while (q != active_head) {
904        halfword p = node_next(q);
905     // tex_flush_node(q);
906        tex_free_node(q, get_node_size(node_type(q))); // less overhead & testing
907        q = p;
908    }
909    node_next(active_head) = null;
910    q = lmt_linebreak_state.passive;
911    while (q) {
912        halfword p = node_next(q);
913     // tex_flush_node(q);
914     // tex_free_node(q, get_node_size(node_type(q))); // less overhead & testing
915        tex_free_node(q, passive_node_size); // less overhead & testing
916        q = p;
917    }
918    lmt_linebreak_state.passive = null;
919}
920
921/*tex
922    Instead of macros we use inline functions. Nowadays compilers generate code that is quite
923    similar as when we use macros (and sometimes even better).
924*/
925
926static inline void tex_aux_add_disc_source_to_target(halfword adjust_spacing, scaled target[], const scaled source[])
927{
928    target[total_advance_amount] += source[total_advance_amount];
929    if (adjust_spacing) {
930        target[font_stretch_amount] += source[font_stretch_amount];
931        target[font_shrink_amount]  += source[font_shrink_amount];
932    }
933}
934
935static inline void tex_aux_sub_disc_target_from_source(halfword adjust_spacing, scaled target[], const scaled source[])
936{
937    target[total_advance_amount] -= source[total_advance_amount];
938    if (adjust_spacing) {
939        target[font_stretch_amount] -= source[font_stretch_amount];
940        target[font_shrink_amount]  -= source[font_shrink_amount];
941    }
942}
943
944static inline void tex_aux_reset_disc_target(halfword adjust_spacing, scaled *target)
945{
946    target[total_advance_amount] = 0;
947    if (adjust_spacing) {
948        target[font_stretch_amount] = 0;
949        target[font_shrink_amount]  = 0;
950    }
951}
952
953/* A memcopy for the whole array is probably more efficient. */
954
955static inline void tex_aux_set_target_to_source(halfword adjust_spacing, scaled target[], const scaled source[])
956{
957 // memcpy(&target[total_glue_amount], &source[total_glue_amount], font_shrink_amount * sizeof(halfword));
958    for (int i = total_advance_amount; i <= total_shrink_amount; i++) {
959        target[i] = source[i];
960    }
961    if (adjust_spacing) {
962        target[font_stretch_amount] = source[font_stretch_amount];
963        target[font_shrink_amount]  = source[font_shrink_amount];
964    }
965}
966
967/*
968    These delta nodes use an offset and as a result we waste half of the memory words. So, by not
969    using an offset but just named fields, we can save 4 memory words (32 bytes) per delta node. So,
970    instead of this:
971
972    \starttyping
973    inline void add_to_target_from_delta(halfword adjust_spacing, scaled *target, halfword delta)
974    {
975        for (int i = total_glue_amount; i <= total_shrink_amount; i++) {
976            target[i] += delta_field(delta, i);
977        }
978        if (adjust_spacing) {
979            target[font_stretch_amount] += delta_field(delta, font_stretch_amount);
980            target[font_shrink_amount]  += delta_field(delta, font_shrink_amount);
981        }
982    }
983    \stoptyping
984
985    We use the more verbose variants and let the compiler optimize the lot.
986
987*/
988
989static inline void tex_aux_add_to_target_from_delta(halfword adjust_spacing, scaled target[], halfword delta)
990{
991    target[total_advance_amount] += delta_field_total_glue(delta);
992    target[total_stretch_amount] += delta_field_total_stretch(delta);
993    target[total_fi_amount]      += delta_field_total_fi_amount(delta);
994    target[total_fil_amount]     += delta_field_total_fil_amount(delta);
995    target[total_fill_amount]    += delta_field_total_fill_amount(delta);
996    target[total_filll_amount]   += delta_field_total_filll_amount(delta);
997    target[total_shrink_amount]  += delta_field_total_shrink(delta);
998    if (adjust_spacing) {
999        target[font_stretch_amount] += delta_field_font_stretch(delta);
1000        target[font_shrink_amount]  += delta_field_font_shrink(delta);
1001    }
1002}
1003
1004static inline void tex_aux_sub_delta_from_target(halfword adjust_spacing, scaled target[], halfword delta)
1005{
1006    target[total_advance_amount] -= delta_field_total_glue(delta);
1007    target[total_stretch_amount] -= delta_field_total_stretch(delta);
1008    target[total_fi_amount]      -= delta_field_total_fi_amount(delta);
1009    target[total_fil_amount]     -= delta_field_total_fil_amount(delta);
1010    target[total_fill_amount]    -= delta_field_total_fill_amount(delta);
1011    target[total_filll_amount]   -= delta_field_total_filll_amount(delta);
1012    target[total_shrink_amount]  -= delta_field_total_shrink(delta);
1013    if (adjust_spacing) {
1014        target[font_stretch_amount] -= delta_field_font_stretch(delta);
1015        target[font_shrink_amount]  -= delta_field_font_shrink(delta);
1016    }
1017}
1018
1019static inline void tex_aux_add_to_delta_from_delta(halfword adjust_spacing, halfword target, halfword source)
1020{
1021    delta_field_total_glue(target)         += delta_field_total_glue(source);
1022    delta_field_total_stretch(target)      += delta_field_total_stretch(source);
1023    delta_field_total_fi_amount(target)    += delta_field_total_fi_amount(source);
1024    delta_field_total_fil_amount(target)   += delta_field_total_fil_amount(source);
1025    delta_field_total_fill_amount(target)  += delta_field_total_fill_amount(source);
1026    delta_field_total_filll_amount(target) += delta_field_total_filll_amount(source);
1027    delta_field_total_shrink(target)       += delta_field_total_shrink(source);
1028    if (adjust_spacing) {
1029        delta_field_font_stretch(target) += delta_field_font_stretch(source);
1030        delta_field_font_shrink(target)  += delta_field_font_shrink(source);
1031    }
1032}
1033
1034static inline void tex_aux_set_delta_from_difference(halfword adjust_spacing, halfword delta, const scaled source_1[], const scaled source_2[])
1035{
1036    delta_field_total_glue(delta)         = (source_1[total_advance_amount] - source_2[total_advance_amount]);
1037    delta_field_total_stretch(delta)      = (source_1[total_stretch_amount] - source_2[total_stretch_amount]);
1038    delta_field_total_fi_amount(delta)    = (source_1[total_fi_amount]      - source_2[total_fi_amount]);
1039    delta_field_total_fil_amount(delta)   = (source_1[total_fil_amount]     - source_2[total_fil_amount]);
1040    delta_field_total_fill_amount(delta)  = (source_1[total_fill_amount]    - source_2[total_fill_amount]);
1041    delta_field_total_filll_amount(delta) = (source_1[total_filll_amount]   - source_2[total_filll_amount]);
1042    delta_field_total_shrink(delta)       = (source_1[total_shrink_amount]  - source_2[total_shrink_amount]);
1043    if (adjust_spacing) {
1044        delta_field_font_stretch(delta) = (source_1[font_stretch_amount] - source_2[font_stretch_amount]);
1045        delta_field_font_shrink(delta)  = (source_1[font_shrink_amount]  - source_2[font_shrink_amount]);
1046    }
1047}
1048
1049static inline void tex_aux_add_delta_from_difference(halfword adjust_spacing, halfword delta, const scaled source_1[], const scaled source_2[])
1050{
1051    delta_field_total_glue(delta)         += (source_1[total_advance_amount] - source_2[total_advance_amount]);
1052    delta_field_total_stretch(delta)      += (source_1[total_stretch_amount] - source_2[total_stretch_amount]);
1053    delta_field_total_fi_amount(delta)    += (source_1[total_fi_amount]      - source_2[total_fi_amount]);
1054    delta_field_total_fil_amount(delta)   += (source_1[total_fil_amount]     - source_2[total_fil_amount]);
1055    delta_field_total_fill_amount(delta)  += (source_1[total_fill_amount]    - source_2[total_fill_amount]);
1056    delta_field_total_filll_amount(delta) += (source_1[total_filll_amount]   - source_2[total_filll_amount]);
1057    delta_field_total_shrink(delta)       += (source_1[total_shrink_amount]  - source_2[total_shrink_amount]);
1058    if (adjust_spacing) {
1059        delta_field_font_stretch(delta) += (source_1[font_stretch_amount] - source_2[font_stretch_amount]);
1060        delta_field_font_shrink(delta)  += (source_1[font_shrink_amount]  - source_2[font_shrink_amount]);
1061    }
1062}
1063
1064/*tex
1065
1066    This function is used to add the width of a list of nodes (from a discretionary) to one of the
1067    width arrays. Replacement texts and discretionary texts are supposed to contain only character
1068    nodes, kern nodes, and box or rule nodes.
1069
1070    From now on we just ignore \quite {invalid} nodes. If any such node influences the width, so be
1071    it.
1072
1073    \starttyping
1074    static void bad_node_in_disc_error(halfword p)
1075    {
1076        tex_formatted_error(
1077            "linebreak",
1078            "invalid node with type %s found in discretionary",
1079            node_data[node_type(p)].name
1080        );
1081    }
1082    \stoptyping
1083*/
1084
1085static inline scaled tex_aux_applied_amount(halfword n, halfword factor)
1086{
1087    return factor && glue_amount(n) ? tex_xn_over_d(glue_amount(n), factor, scaling_factor) : glue_amount(n);
1088}
1089
1090static inline scaled tex_aux_applied_stretch(halfword n, halfword factor)
1091{
1092    return factor && glue_stretch(n) ? tex_xn_over_d(glue_stretch(n), factor, scaling_factor) : glue_stretch(n);
1093}
1094
1095static void tex_aux_add_to_widths(halfword s, int adjust_spacing, int adjust_spacing_step, halfword sf_factor, halfword sf_stretch_factor, scaled widths[])
1096{
1097    /* todo only check_expand_pars once per font (or don't check) */
1098    while (s) {
1099        switch (node_type(s)) {
1100            case glyph_node:
1101                widths[total_advance_amount] += tex_glyph_width(s);
1102                if (adjust_spacing && adjust_spacing_step > 0 && tex_has_glyph_expansion(s)) {
1103                    lmt_packaging_state.previous_char_ptr = s;
1104                    widths[font_stretch_amount] += tex_char_stretch(s);
1105                    widths[font_shrink_amount] += tex_char_shrink(s);
1106                };
1107                break;
1108            case hlist_node:
1109            case vlist_node:
1110                widths[total_advance_amount] += box_width(s);
1111                break;
1112            case rule_node:
1113                widths[total_advance_amount] += rule_width(s);
1114                break;
1115            case glue_node:
1116                if (tex_has_glue_option(s, glue_option_has_factor)) {
1117                    widths[total_advance_amount] += tex_aux_applied_amount(s, sf_factor);
1118                    widths[total_stretch_amount + glue_stretch_order(s)] += tex_aux_applied_stretch(s, sf_stretch_factor);
1119                } else {
1120                    widths[total_advance_amount] += glue_amount(s);
1121                    widths[total_stretch_amount + glue_stretch_order(s)] += glue_stretch(s);
1122                }
1123                widths[total_shrink_amount] += glue_shrink(s);
1124                break;
1125            case kern_node:
1126                widths[total_advance_amount] += kern_amount(s);
1127                if (adjust_spacing == adjust_spacing_full && node_subtype(s) == font_kern_subtype) {
1128                    halfword n = node_prev(s);
1129                    if (n && node_type(n) == glyph_node && ! tex_has_glyph_option(node_next(s), glyph_option_no_expansion)) {
1130                        widths[font_stretch_amount] += tex_kern_stretch(s);
1131                        widths[font_shrink_amount] += tex_kern_shrink(s);
1132                    }
1133                }
1134                break;
1135            case disc_node:
1136                break;
1137            default:
1138             /* bad_node_in_disc_error(s); */
1139                break;
1140        }
1141        s = node_next(s);
1142    }
1143}
1144
1145/*tex
1146
1147    This function is used to substract the width of a list of nodes (from a discretionary) from one
1148    of the width arrays. It is used only once, but deserves it own function because of orthogonality
1149    with the |add_to_widths| function.
1150
1151*/
1152
1153static void tex_aux_sub_from_widths(halfword s, int adjust_spacing, int adjust_spacing_step, halfword sf_factor, halfword sf_stretch_factor, scaled widths[])
1154{
1155    while (s) {
1156        /*tex Subtract the width of node |s| from |break_width|; */
1157        switch (node_type(s)) {
1158            case glyph_node:
1159                widths[total_advance_amount] -= tex_glyph_width(s);
1160                if (adjust_spacing && adjust_spacing_step > 0 && tex_has_glyph_expansion(s)) {
1161                    lmt_packaging_state.previous_char_ptr = s;
1162                    widths[font_stretch_amount] -= tex_char_stretch(s);
1163                    widths[font_shrink_amount] -= tex_char_shrink(s);
1164                }
1165                break;
1166            case hlist_node:
1167            case vlist_node:
1168                widths[total_advance_amount] -= box_width(s);
1169                break;
1170            case rule_node:
1171                widths[total_advance_amount] -= rule_width(s);
1172                break;
1173            case glue_node:
1174                if (tex_has_glue_option(s, glue_option_has_factor)) {
1175                    widths[total_advance_amount] -= tex_aux_applied_amount(s, sf_factor);
1176                    widths[total_stretch_amount + glue_stretch_order(s)] -= tex_aux_applied_stretch(s, sf_stretch_factor);
1177                } else {
1178                    widths[total_advance_amount] -= glue_amount(s);
1179                    widths[total_stretch_amount + glue_stretch_order(s)] -= glue_stretch(s);
1180                }
1181                widths[total_shrink_amount] -= glue_shrink(s);
1182                break;
1183            case kern_node:
1184                widths[total_advance_amount] -= kern_amount(s);
1185                if (adjust_spacing == adjust_spacing_full && node_subtype(s) == font_kern_subtype) {
1186                    halfword n = node_prev(s);
1187                    if (n && node_type(n) == glyph_node && ! tex_has_glyph_option(node_next(s), glyph_option_no_expansion)) {
1188                        widths[font_stretch_amount] -= tex_kern_stretch(s);
1189                        widths[font_shrink_amount] -= tex_kern_shrink(s);
1190                    }
1191                }
1192                break;
1193            case disc_node:
1194                break;
1195            default:
1196             /* bad_node_in_disc_error(s); */
1197                break;
1198        }
1199        s = node_next(s);
1200    }
1201}
1202
1203/*tex
1204
1205    When we insert a new active node for a break at |cur_p|, suppose this new node is to be placed
1206    just before active node |a|; then we essentially want to insert $\delta\,|cur_p|\,\delta ^
1207    \prime$ before |a|, where $\delta = \alpha (a) - \alpha (|cur_p|)$ and $\delta ^ \prime =
1208    \alpha (|cur_p|) - \alpha (a)$ in the notation explained above. The |cur_active_width| array
1209    now holds $\gamma + \beta (|cur_p|) - \alpha (a)$; so $\delta$ can be obtained by subtracting
1210    |cur_active_width| from the quantity $\gamma + \beta (|cur_p|) - \alpha (|cur_p|)$. The latter
1211    quantity can be regarded as the length of a line from |cur_p| to |cur_p|; we call it the
1212    |break_width| at |cur_p|.
1213
1214    The |break_width| is usually negative, since it consists of the background (which is normally
1215    zero) minus the width of nodes following~|cur_p| that are eliminated after a break. If, for
1216    example, node |cur_p| is a glue node, the width of this glue is subtracted from the background;
1217    and we also look ahead to eliminate all subsequent glue and penalty and kern and math nodes,
1218    subtracting their widths as well.
1219
1220    Kern nodes do not disappear at a line break unless they are |explicit|.
1221
1222*/
1223
1224static void tex_aux_compute_break_width(int break_type, int adjust_spacing, int adjust_spacing_step, halfword sf_factor, halfword sf_stretch_factor, halfword p)
1225{
1226    /*tex
1227
1228        Glue and other whitespace to be skipped after a break; used if unhyphenated, or |post_break
1229        = null|.
1230
1231    */
1232    halfword s = p;
1233    if (p) {
1234        /*tex TTP has a curious |break_type > unhyphenated_node| here, so delta and passive also are valid. */
1235        switch (break_type) {
1236            case hyphenated_node:
1237            case delta_node:
1238            case passive_node:
1239                /*tex
1240
1241                    Compute the discretionary |break_width| values. When |p| is a discretionary
1242                    break, the length of a line \quotation {from |p| to |p|} has to be defined
1243                    properly so that the other calculations work out. Suppose that the pre-break
1244                    text at |p| has length $l_0$, the post-break text has length $l_1$, and the
1245                    replacement text has length |l|. Suppose also that |q| is the node following
1246                    the replacement text. Then length of a line from |p| to |q| will be computed as
1247                    $\gamma + \beta (q) - \alpha (|p|)$, where $\beta (q) = \beta (|p|) - l_0 + l$.
1248                    The actual length will be the background plus $l_1$, so the length from |p| to
1249                    |p| should be $\gamma + l_0 + l_1 - l$. If the post-break text of the
1250                    discretionary is empty, a break may also discard~|q|; in that unusual case we
1251                    subtract the length of~|q| and any other nodes that will be discarded after the
1252                    discretionary break.
1253
1254                    The value of $l_0$ need not be computed, since |line_break| will put it into the
1255                    global variable |disc_width| before calling |try_break|. In case of nested
1256                    discretionaries, we always follow the no-break path, as we are talking about the
1257                    breaking on {\it this} position.
1258
1259                */
1260                if (node_type(p) == disc_node) {
1261                    tex_aux_sub_from_widths(disc_no_break_head(p), adjust_spacing, adjust_spacing_step, sf_factor, sf_stretch_factor, lmt_linebreak_state.break_width);
1262                    tex_aux_add_to_widths(disc_post_break_head(p), adjust_spacing, adjust_spacing_step, sf_factor, sf_stretch_factor, lmt_linebreak_state.break_width);
1263                    tex_aux_add_disc_source_to_target(adjust_spacing, lmt_linebreak_state.break_width, lmt_linebreak_state.disc_width);
1264                    if (disc_post_break_head(p)) {
1265                        s = null;
1266                    } else {
1267                        /*tex no |post_break|: skip any whitespace following */
1268                        s = node_next(p);
1269                    }
1270                } else { 
1271                    tex_confusion("line breaking 3");
1272                }
1273                break;
1274        }
1275    }
1276    while (s) {
1277        switch (node_type(s)) {
1278            case glue_node:
1279                /*tex Subtract glue from |break_width|; */
1280                if (tex_has_glue_option(s, glue_option_has_factor)) {
1281                    lmt_linebreak_state.break_width[total_advance_amount] -= tex_aux_applied_amount(s, sf_factor);
1282                    lmt_linebreak_state.break_width[total_stretch_amount + glue_stretch_order(s)] -= tex_aux_applied_stretch(s, sf_stretch_factor);
1283                } else {
1284                    lmt_linebreak_state.break_width[total_advance_amount] -= glue_amount(s);
1285                    lmt_linebreak_state.break_width[total_stretch_amount + glue_stretch_order(s)] -= glue_stretch(s);
1286                }
1287                lmt_linebreak_state.break_width[total_shrink_amount] -= glue_shrink(s);
1288                break;
1289            case penalty_node:
1290                break;
1291            case kern_node:
1292                if (node_subtype(s) == explicit_kern_subtype || node_subtype(s) == italic_kern_subtype) {
1293                    lmt_linebreak_state.break_width[total_advance_amount] -= kern_amount(s);
1294                    break;
1295                } else {
1296                    return;
1297                }
1298            case math_node:
1299                if (tex_math_glue_is_zero(s)) {
1300                    lmt_linebreak_state.break_width[total_advance_amount] -= math_surround(s);
1301                } else {
1302                    lmt_linebreak_state.break_width[total_advance_amount] -= math_amount(s);
1303                    lmt_linebreak_state.break_width[total_stretch_amount + math_stretch_order(s)] -= math_stretch(s);
1304                    lmt_linebreak_state.break_width[total_shrink_amount] -= math_shrink(s);
1305                }
1306                break;
1307            default:
1308                return;
1309        };
1310        s = node_next(s);
1311    }
1312}
1313
1314/*tex
1315
1316    I changed the report into a table but in the end decided to stick to passing arguments mainly
1317    because it is more convenient and efficient at the \LUA\ end. I also played with the following
1318    but in the end decided against it as it complicates the \LUA\ end and these callbacks will
1319    not be used beyond tracing and experiments so performance is not an issue. Also, stacking them
1320    will likely create an interference mess. So I payed the price of a few hours wasteful coding.
1321
1322    \starttyping
1323    # define line_break_callback_step(checks,context) (! (checks >> 16) || ((checks >> 16) & (1 << context)))
1324    # define line_break_callback_checks(checks) (checks & 0xFFFF)
1325
1326    static void tex_aux_line_break_callback_initialize(int callback_id, halfword checks, int subpasses)
1327    {
1328      if (line_break_callback_step(checks, initialize_line_break_context)) {
1329            lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "ddd->",
1330                initialize_line_break_context,
1331                line_break_callback_checks(checks),
1332                subpasses
1333            );
1334        }
1335    }
1336    \stoptyping
1337
1338*/
1339
1340static void tex_aux_line_break_callback_initialize(int callback_id, halfword checks, int subpasses)
1341{
1342    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "ddd->",
1343        initialize_line_break_context,
1344        checks,
1345        subpasses
1346    );
1347}
1348
1349static void tex_aux_line_break_callback_start(int callback_id, halfword checks, int pass, int subpass, int classes, int decent)
1350{
1351    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "dddddd->",
1352        start_line_break_context,
1353        checks,
1354        pass,
1355        subpass,
1356        classes,
1357        decent
1358    );
1359}
1360
1361static void tex_aux_line_break_callback_stop(int callback_id, halfword checks)
1362{
1363    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "ddd->",
1364        stop_line_break_context,
1365        checks,
1366        lmt_linebreak_state.fewest_demerits
1367    );
1368}
1369
1370static void tex_aux_line_break_callback_collect(int callback_id, halfword checks)
1371{
1372    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "dd->",
1373        collect_line_break_context,
1374        checks
1375    );
1376}
1377
1378static void tex_aux_line_break_callback_line(int callback_id, halfword checks, int line, halfword passive)
1379{
1380    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "ddNdddddd->",
1381        line_line_break_context,
1382        checks,
1383        lmt_linebreak_state.just_box,
1384        lmt_packaging_state.last_badness,
1385        lmt_packaging_state.last_overshoot,
1386        lmt_packaging_state.total_shrink[normal_glue_order],
1387        lmt_packaging_state.total_stretch[normal_glue_order],
1388        line,
1389        passive_serial(passive)
1390    );
1391}
1392
1393static void tex_aux_line_break_callback_delete(int callback_id, halfword checks, halfword passive)
1394{
1395    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "dddd->",
1396        delete_line_break_context,
1397        checks,
1398        passive_serial(passive),
1399        passive_ref_count(passive)
1400    );
1401}
1402
1403static void tex_aux_line_break_callback_wrapup(int callback_id, halfword checks)
1404{
1405    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "dddd->",
1406        wrapup_line_break_context,
1407        checks,
1408        lmt_linebreak_state.fewest_demerits,
1409        lmt_linebreak_state.actual_looseness
1410    );
1411}
1412
1413static halfword tex_aux_line_break_callback_report(int callback_id, halfword checks, int pass, halfword subpass, halfword active, halfword passive)
1414{
1415    halfword demerits = active_total_demerits(active);
1416    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "ddddddddddddNddd->r",
1417        report_line_break_context,
1418        checks,
1419        pass,
1420        subpass,
1421        passive_serial(passive),
1422        passive_prev_break(passive) ? passive_serial(passive_prev_break(passive)) : 0,
1423        active_line_number(active) - 1,
1424        node_type(active),
1425        active_fitness(active) + 1,            /* we offset the class */
1426        passive_n_of_fitness_classes(passive), /* also in passive  */
1427        passive_badness(passive),
1428        demerits,
1429        passive_cur_break(passive),
1430        active_short(active),
1431        active_glue(active),
1432        active_line_width(active),
1433        &demerits  /* optionally changed */
1434    );
1435    return demerits;
1436}
1437
1438static void tex_aux_line_break_callback_list(int callback_id, halfword checks, halfword passive)
1439{
1440    lmt_run_callback(lmt_lua_state.lua_instance, callback_id, "dddd->",
1441        list_line_break_context,
1442        checks,
1443        passive_serial(passive),
1444        passive_ref_count(passive)
1445    );
1446}
1447
1448/* */
1449
1450void tex_aux_print_break_node(halfword active, halfword passive, int do_last_line_fit)
1451{
1452    /*tex Print a symbolic description of the new break node. */
1453    tex_print_format(
1454        "[break: serial %i, line %i, class %i of %i, %sdemerits %i, ",
1455        passive_serial(passive),
1456        active_line_number(active) - 1,
1457        active_fitness(active), // + 1 here too?
1458        active_n_of_fitness_classes(active),
1459        node_type(active) == hyphenated_node ? " hyphenated, " : "",
1460        active_total_demerits(active)
1461    );
1462    if (do_last_line_fit) {
1463        /*tex Print additional data in the new active node. */
1464        tex_print_format(
1465            " short %p, %s %p, ",
1466            active_short(active),
1467            passive_cur_break(passive) ? "glue" : "active",
1468            active_glue(active)
1469        );
1470    }
1471    tex_print_format(
1472        "previous %i]",
1473        passive_prev_break(passive) ? passive_serial(passive_prev_break(passive)) : null
1474    );
1475}
1476
1477void tex_aux_print_feasible_break(halfword current, halfword breakpoint, halfword badness, halfword penalty, halfword d, halfword artificial_demerits, halfword fit_class, halfword printed_node)
1478{
1479    /*tex Print a symbolic description of this feasible break. */
1480    if (printed_node != current) {
1481        /*tex Print the list between |printed_node| and |cur_p|, then set |printed_node := cur_p|. */
1482     // tex_print_nlp();
1483        tex_print_format("%l[break: stripe] ");
1484        if (current) {
1485            halfword save_link = node_next(current);
1486            node_next(current) = null;
1487            tex_short_display(node_next(lmt_linebreak_state.printed_node));
1488            node_next(current) = save_link;
1489        } else {
1490            tex_short_display(node_next(lmt_linebreak_state.printed_node));
1491        }
1492        lmt_linebreak_state.printed_node = current;
1493    }
1494    tex_print_format(
1495        "%l[break: feasible, trigger '%s', serial %i, badness %B, penalty %i, demerits %B, fit class %i]",
1496        lmt_interface.node_data[current ? node_type(current) : par_node].name, /* weird */
1497        active_break_node(breakpoint) ? passive_serial(active_break_node(breakpoint)) : 0,
1498        badness,
1499        penalty,
1500        artificial_demerits ? awful_bad : d,
1501        fit_class
1502    );
1503}
1504
1505/*tex We implement this one later on. */
1506
1507/*
1508    The only reason why we still have line_break_dir is because we have some experimental protrusion
1509    trickery depending on it. The post linebreak function is here and not in its own module because
1510    it is called from the line break routine.
1511*/
1512
1513static void tex_aux_post_line_break (
1514    const line_break_properties *properties,
1515    halfword                     line_break_dir,
1516    int                          callback_id,
1517    halfword                     checks,
1518    int                          state
1519);
1520
1521/*tex
1522
1523    The next subroutine is used to compute the badness of glue, when a total |t| is supposed to be
1524    made from amounts that sum to~|s|. According to {\em The \TEX book}, the badness of this
1525    situation is $100(t/s)^3$; however, badness is simply a heuristic, so we need not squeeze out
1526    the last drop of accuracy when computing it. All we really want is an approximation that has
1527    similar properties.
1528
1529    The actual method used to compute the badness is easier to read from the program than to
1530    describe in words. It produces an integer value that is a reasonably close approximation to
1531    $100(t/s)^3$, and all implementations of \TEX\ should use precisely this method. Any badness of
1532    $2^{13}$ or more is treated as infinitely bad, and represented by 10000.
1533
1534    It is not difficult to prove that |badness (t + 1, s) >= badness (t, s) >= badness (t, s + 1)|
1535    The badness function defined here is capable of computing at most 1095 distinct values, but
1536    that is plenty.
1537
1538    A core aspect of the linebreak algorithm is the calculation of badness. The formula currently
1539    used has evolved with the tex versions before Don Knuth settled on this approach. And I (HH)
1540    admit that I see no real reason to change something here. The only possible extension could
1541    be changing the hardcoded |loose_criterion| of 99 and |decent_criterion| of 12. These could
1542    become parameters instead. When looking at the code you will notice a loop that runs from
1543    |very_loose_fit| to |tight_fit| with the following four steps:
1544
1545    \starttyping
1546    very_loose_fit  loose_fit  decent_fit  tight_fit
1547    \stoptyping
1548
1549    where we have only |loose_fit| and |decent_fit| with associated criteria later on. So, as an
1550    experiment I decided to add two steps in between.
1551
1552    \starttyping
1553    very_loose_fit  semi_loose_fit  loose_fit  decent_fit  semi_tight_fit  tight_fit
1554    \stoptyping
1555
1556    Watch how we keep the assymetrical nature of this sequence: there is basicaly one tight
1557    step less than loose steps. Adding these steps took hardly any code so it was a cheap
1558    experiment. However, the result is not that spectacular: I'm pretty sure that users will
1559    not be able to choose consistently what result looks better, but who knows. For the moment
1560    I keep it, if only to be able to discuss it as useless extension. Configuring the value s
1561    is done with |\linebreakcriterion| which gets split into 4 parts (2 bytes per criterion).
1562
1563    It is probably hard to explain to users what a different setting does and although one can
1564    force different output in narrow raggedright text it would probbably enough to just make
1565
1566    the |decent_criterion| configureable. Anyway, because we're talking heuristics and pretty
1567    good estimates from Don Knuth here, it would be pretentious to suggest that I really did
1568    research this fuzzy topic (if it was worth the effort at all).
1569
1570    Here |large_width_excess| is 110.32996pt while |small_stretchability| equals 25.38295pt.
1571
1572*/
1573
1574/*tex
1575
1576    Around 2023-05-24 Mikael Sundqvist and I did numerous tests with the badness function below in
1577    comparison with the variant mentioned in Digital Typography (DEK) and we observed that indeed
1578    both functions behave pretty close (emulations with lua, mathematica etc). In practice one can
1579    get different badness values (especially low numbers). We ran some test on documents and on
1580    hundreds of pages one can get a few different decisions. The main reason for looking into this
1581    was that we were exploring a bit more visual approach to deciding on what penalties to use in
1582    the math inter-atom spacing in \CONTEXT\ (where we use a more granular class model). In the end
1583    the magic criteria became even more magic (and impressive). BTW, indeed we could get these 1095
1584    different badness cases with as maximum calculated one 8189.
1585
1586*/
1587
1588halfword tex_badness(scaled t, scaled s)
1589{
1590    /*tex Approximation to $\alpha t/s$, where $\alpha^3 \approx 100 \cdot 2^{18}$ */
1591
1592    if (t == 0) {
1593        return 0;
1594    } else if (s <= 0) {
1595        return infinite_bad;
1596    } else {
1597        /*tex $297^3 = 99.94 \times 2^{18}$ */
1598        if (t <= large_width_excess) {
1599            t = (t * 297) / s;  /* clipping by integer division */
1600        } else if (s >= small_stretchability) {
1601            t = t / (s / 297);  /* clipping by integer division */
1602        } else {
1603            /*tex
1604                When we end up here |t| is pretty large so we can as well save a test and return
1605                immediately. (HH & MS: we tested this while cheating a bit because this function
1606                is seldom entered with values that make us end up here.)
1607            */
1608            return infinite_bad;
1609        }
1610        if (t > 1290) {
1611            /*tex As $1290^3 < 2^{31} < 1291^3$ we catch an overflow here. */ /* actually badness 8189 */
1612            return infinite_bad;
1613        } else {
1614            /*tex 297*297*297 == 26198073 / 100 => 261981 */
1615            /*tex This is $t^3 / 2^{18}$, rounded to the nearest integer */
1616         // return (t * t * t + 0400000) / 01000000; /* 0400000/01000000 == 1/2 */
1617         // return (t * t * t + 0x20000) /  0x40000;
1618            return (t * t * t +  131072) /   262144;
1619        }
1620    }
1621}
1622
1623/*tex
1624
1625    Regular \TEX\ has (basically) four classes: tight, decent, loose, very loose but one can also think
1626    of it as five ranges. In \LUAMETATEX\ we have generalized this so that we can have more granularity.
1627
1628*/
1629
1630void tex_check_fitness_classes(halfword fitnessclasses)
1631{
1632    if (! fitnessclasses) {
1633        tex_normal_error("linebreak", "unknown fitnessclasses");
1634        return;
1635    } else { 
1636        halfword max = tex_get_specification_count(fitnessclasses);
1637        halfword med = 0;
1638        if (max >= max_n_of_fitness_values) {
1639            tex_normal_error("linebreak", "too many fitnessclasses");
1640            return;
1641        }
1642        if (max < 3) {
1643            tex_normal_error("linebreak", "less than three fitnessclasses");
1644            return;
1645        }
1646        for (med = 1; med <= max; (med)++) {
1647            if (tex_get_specification_fitness_class(fitnessclasses, med) == 0) {
1648                break;
1649            }
1650        }
1651        if ((med <= 1) || (med == max)) {
1652            tex_normal_error("linebreak", "invalid decent slot in fitnessclasses");
1653        } else {
1654            tex_set_specification_decent(fitnessclasses, med);
1655        }
1656    }
1657}
1658
1659static inline halfword tex_max_fitness(halfword fitnessclasses)
1660{
1661    return tex_get_specification_count(fitnessclasses);
1662}
1663
1664static inline halfword tex_med_fitness(halfword fitnessclasses)
1665{
1666    return tex_get_specification_decent(fitnessclasses);
1667}
1668
1669static inline halfword tex_get_demerits(const line_break_properties *properties, halfword distance, halfword start, halfword stop)
1670{
1671    if (distance) {
1672        halfword demerits = properties->adjacent_demerits;
1673        if (demerits && tex_get_specification_count(demerits)) {
1674            if (start > stop) {
1675                return tex_get_specification_adjacent_u(demerits, distance);
1676            } else {
1677                return tex_get_specification_adjacent_d(demerits, distance);
1678            }
1679        } else if (distance > 1) {
1680            /*tex
1681                Traditional \TEX\ only adds these demerits when the distance is more than one. We're
1682                not sure about the rationale but it kin dof makes sense not to punish a tight vs decent
1683                line because after all shrink is limited and normally reasonable (in spaces) whiel on
1684                the other hand loose can get out hand, so here a distance more than one is kind of bad
1685                while loose vs very loose is probably already bad enough.
1686            */
1687            return properties->adj_demerits;
1688        }
1689    }
1690    return 0;
1691}
1692
1693/*tex
1694    Watch out: here we map from indices to inbetween (zero based) categories.
1695*/
1696
1697static inline halfword tex_normalized_loose_badness(halfword b, halfword fitnessclasses)
1698{
1699    halfword med = tex_get_specification_decent(fitnessclasses);
1700    for (halfword c = med - 1; c >= 1; c--) {
1701        if (b <= tex_get_specification_fitness_class(fitnessclasses, c)) {
1702            return c;
1703        }
1704    }
1705    return 0;
1706}
1707
1708static inline halfword tex_normalized_tight_badness(halfword b, halfword fitnessclasses)
1709{
1710    halfword max = tex_get_specification_count(fitnessclasses);
1711    halfword med = tex_get_specification_decent(fitnessclasses);
1712    for (halfword c = med + 1; c <= max; c++) {
1713        if (b <= tex_get_specification_fitness_class(fitnessclasses, c)) {
1714            return c - 2;
1715        }
1716    }
1717    return max - 1;
1718}
1719
1720halfword tex_default_fitness_classes(void) {
1721    halfword n = tex_new_specification_node(5, fitness_classes_code, 0);
1722    tex_set_specification_fitness_class(n, 1, 99);
1723    tex_set_specification_fitness_class(n, 2, 12);
1724    tex_set_specification_fitness_class(n, 3, 0);
1725    tex_set_specification_fitness_class(n, 4, 12);
1726    tex_set_specification_fitness_class(n, 5, 99);
1727    tex_check_fitness_classes(n);
1728    return n;
1729}
1730
1731static void tex_check_protrusion_shortfall(halfword breakpoint, halfword first, halfword current, halfword *shortfall)
1732{
1733    // if (line_break_dir == dir_righttoleft) {
1734    //     /*tex Not now, we need to keep more track. */
1735    // } else {
1736        halfword other = null;
1737        halfword left = active_break_node(breakpoint) ? passive_cur_break(active_break_node(breakpoint)) : first; /* nasty */
1738        if (current) {
1739            other = node_prev(current);
1740            if (node_next(other) != current) {
1741                tex_normal_error("linebreak", "the node list is messed up");
1742            }
1743        }
1744        /*tex
1745
1746            The last characters (hyphenation character) if these two list should always be
1747            the same anyway, so we just look at |pre_break|. Let's look at the right margin
1748            first.
1749
1750        */
1751        if (current && node_type(current) == disc_node && disc_pre_break_head(current)) {
1752            /*tex
1753                A |disc_node| with non-empty |pre_break|, protrude the last char of
1754                |pre_break|:
1755            */
1756            other = disc_pre_break_tail(current);
1757        } else {
1758            other = tex_aux_find_protchar_right(left, other);
1759        }
1760        if (other && node_type(other) == glyph_node) {
1761            *shortfall += tex_char_protrusion(other, right_margin_kern_subtype);
1762        }
1763        /*tex now the left margin */
1764        if (left && node_type(left) == disc_node && disc_post_break_head(left)) {
1765            /*tex The first char could be a disc! Protrude the first char. */
1766            other = disc_post_break_head(left);
1767        } else {
1768            other = tex_aux_find_protchar_left(left, 1);
1769        }
1770        if (other && node_type(other) == glyph_node) {
1771            *shortfall += tex_char_protrusion(other, left_margin_kern_subtype);
1772        }
1773    // }
1774}
1775
1776static void tex_aux_set_quality(halfword active, halfword passive, scaled shrt, scaled glue, scaled width, halfword badness)
1777{
1778    halfword quality = 0;
1779    halfword deficiency = 0;
1780    active_short(active) = shrt;
1781    active_glue(active) = glue;
1782    active_line_width(active) = width;
1783    if (shrt < 0) {
1784        shrt = -shrt;
1785        if (shrt > glue) {
1786            quality = par_is_overfull;
1787            deficiency = shrt - glue;
1788        }
1789    } else if (shrt > 0) {
1790        if (shrt > glue) {
1791            quality = par_is_underfull;
1792            deficiency = shrt - glue;
1793        }
1794    }
1795    passive_quality(passive) = quality;
1796    passive_deficiency(passive) = deficiency;
1797    passive_demerits(passive) = active_total_demerits(active); /* ... */
1798    passive_badness(passive) = badness;
1799    active_quality(active) = quality;
1800    active_deficiency(active) = deficiency;
1801 }
1802
1803/*tex
1804
1805    The twin detection is a follow up on a discussion between Dedier Verna, Mikael Sundqvist and
1806    Hans Hagen. We have been playing with features like this in ConTeXt when we extended the par
1807    builder but that uses a callback that is meant for tracing (read: articles that we write on
1808    this topic). When Dedier wondered if it could be an engine feature we pickup up this thread.
1809
1810    In a 2024 article for the TUG meeting Dedier describes some performance measurements with
1811    his (lisp based) framework but we already know that the par builder is not that critical
1812    in the whole picture. Here we distinguish between a left and right edge twin (repeated word)
1813    and we also look into discretionaries because we can assume that more complex opentype
1814    features can give more complex discretionaries than a single hyphen. One can condider a
1815    configurable size of comparison.
1816
1817    Using a Lua approach is quite flexible and permits nice tracing but, as said, it abuses a
1818    callback that, due to the many different invocations is not the best candidate for this. We
1819    could add another callback but that is overkill. Instead, part of the Lua code has been turned
1820    into a native feature so that we can do both: native twin checking as well as tracing of the
1821    break routine (which we need for testing par passes) but also exploring more features using
1822    that callback.
1823
1824    The reference implementation is still done in Lua where we then also have twin tracing. In
1825    principle that one is fast enough. The native C implementation works sligthly different but
1826    is still based on the Lua code. We also have some constraints, like the maximum size of a
1827    snippet.
1828
1829    In both cases (Lua and C) the overhead is rather small because we only look at a small set
1830    of breakpoints. In Lua we gain performance by caching, in C by limiting the snippet. We
1831    can squeeze out some more performance if needed by immediately comparing the second snippet
1832    with the first one. Contrary to the Lua variant, the C implementation checks for a glyph
1833    option being set. Because it has to fit in the rest we carry the |\lefttwindemerits| and
1834    |\righttwindemerits| in the par node and also can set it in the (optional extra) par passes.
1835
1836    Maybe (as in the \LUA) variant, also support |twinslimit|.
1837
1838*/
1839
1840static int tex_aux_get_before(halfword breakpoint, int snippet[])
1841{
1842    halfword current = null;
1843    int count = 0;
1844    halfword font = 0;
1845    quarterword prop = 0;
1846    switch (node_type(breakpoint)) {
1847        case glue_node:
1848            {
1849                current = node_prev(breakpoint);
1850                if (node_type(current) == glyph_node && tex_has_glyph_option(current, glyph_option_check_twin)) {
1851                    font = glyph_font(current);
1852                    prop = glyph_control(current);
1853                    snippet[count++] = font;
1854                    snippet[count++] = glyph_character(current);
1855                    if (count == max_twin_snippet) {
1856                        return 0;
1857                    }
1858                    current = node_prev(current);
1859                } else {
1860                    return 0;
1861                }
1862            }
1863            break;
1864        case disc_node:
1865            {
1866                halfword r = disc_pre_break_tail(breakpoint);
1867                current = breakpoint;
1868                while (r) {
1869                    if (node_type(r) == glyph_node) {
1870                        if (! tex_has_glyph_option(r, glyph_option_check_twin)) {
1871                            return 0;
1872                        } else if (! font) {
1873                            font = glyph_font(r);
1874                            prop = glyph_control(r);
1875                            snippet[count++] = font;
1876                        }
1877                        snippet[count++] = glyph_character(r);
1878                        if (count == max_twin_snippet) {
1879                            return 0;
1880                        }
1881                    }
1882                    r = node_prev(r);
1883                }
1884                current = node_prev(current);
1885            }
1886            break;
1887        default:
1888            return 0;
1889    }
1890    if (current && font) {
1891        while (current) {
1892            switch (node_type(current)) {
1893                case glyph_node:
1894                    if (! tex_has_glyph_option(current, glyph_option_check_twin)) {
1895                        return 0;
1896                    } else if (glyph_font(current) == font) {
1897                        snippet[count++] = glyph_character(current);
1898                        if (count == max_twin_snippet) {
1899                            return 0;
1900                        } else {
1901                            break;
1902                        }
1903                    } else {
1904                        return 0;
1905                    }
1906                case disc_node:
1907                    {
1908                        halfword r = disc_no_break_tail(current);
1909                        while (r) {
1910                            switch (node_type(r)) {
1911                                case glyph_node:
1912                                    if (! tex_has_glyph_option(r, glyph_option_check_twin)) {
1913                                        return 0;
1914                                    } else if (glyph_font(r) == font) {
1915                                        snippet[count++] = glyph_character(r);
1916                                        if (count == max_twin_snippet) {
1917                                         // return count;
1918                                            return 0;
1919                                        } else {
1920                                            break;
1921                                        }
1922                                    } else {
1923                                        return 0;
1924                                    }
1925                                case kern_node:
1926                                    if (node_subtype(r) == font_kern_subtype) {
1927                                        break;
1928                                    } else {
1929                                        return 0;
1930                                    }
1931                                default:
1932                                    return 0;
1933                            }
1934                            r = node_prev(r);
1935                        }
1936                        break;
1937                    }
1938                case kern_node:
1939                    if (node_subtype(current) == font_kern_subtype) {
1940                        break;
1941                    } else {
1942                        return 0;
1943                    }
1944                case glue_node:
1945                    {
1946                        halfword prev = node_prev(current);
1947                        if (prev && ((node_type(prev) == glyph_node && tex_has_glyph_option(prev, glyph_option_check_twin)) || node_type(prev) == disc_node)) {
1948                            /* Just a test for an article, we need a field in the glyph node if we go this route. */
1949                            if (count > 1 && prop && has_character_control(prop, ignore_twin_character_control_code)) {
1950                                for (int i = 2; i < count; i++) {
1951                                    snippet[i-1] = snippet[i];
1952                                }
1953                                --count;
1954                            }
1955                            return count;
1956                        }
1957                    }
1958                    return 0;
1959                default:
1960                    return 0;
1961            }
1962            current = node_prev(current);
1963        }
1964    }
1965    return 0;
1966}
1967
1968static int tex_aux_get_after(halfword breakpoint, int snippet[])
1969{
1970    halfword current = null;
1971    int count = 0;
1972    halfword font = 0;
1973    quarterword prop = 0;
1974    switch (node_type(breakpoint)) {
1975        case glue_node:
1976            {
1977                current = node_next(breakpoint);
1978                if (node_type(current) == glyph_node && tex_has_glyph_option(current, glyph_option_check_twin)) {
1979                    font = glyph_font(current);
1980                    prop = glyph_control(current);
1981                    snippet[count++] = font;
1982                    snippet[count++] = glyph_character(current);
1983                    if (count == max_twin_snippet) {
1984                        return 0;
1985                    }
1986                    current = node_next(current);
1987                } else {
1988                    return 0;
1989                }
1990            }
1991            break;
1992        case disc_node:
1993            {
1994                halfword r = disc_post_break_head(breakpoint);
1995                current = breakpoint;
1996                while (r) {
1997                    if (node_type(r) == glyph_node) {
1998                        if (! tex_has_glyph_option(r, glyph_option_check_twin)) {
1999                            return 0;
2000                        } else if (! font) {
2001                            font = glyph_font(r);
2002                            prop = glyph_control(r);
2003                            snippet[count++] = font;
2004                        }
2005                        snippet[count++] = glyph_character(r);
2006                        if (count == max_twin_snippet) {
2007                            return 0;
2008                        }
2009                    }
2010                    r = node_next(r);
2011                }
2012                current = node_next(current);
2013            }
2014            break;
2015        default:
2016            return 0;
2017    }
2018    if (current && font) {
2019        while (current) {
2020            switch (node_type(current)) {
2021                case glyph_node:
2022                    if (! tex_has_glyph_option(current, glyph_option_check_twin)) {
2023                        return 0;
2024                    } else if (glyph_font(current) == font) {
2025                        snippet[count++] = glyph_character(current);
2026                        if (count == max_twin_snippet) {
2027                            return 0;
2028                        } else {
2029                            prop = glyph_control(current);
2030                            break;
2031                        }
2032                    } else {
2033                        return 0;
2034                    }
2035                case disc_node:
2036                    {
2037                        halfword r = disc_no_break_head(current);
2038                        while (r) {
2039                            switch (node_type(r)) {
2040                                case glyph_node:
2041                                    if (! tex_has_glyph_option(r, glyph_option_check_twin)) {
2042                                        return 0;
2043                                    } else if (glyph_font(r) == font) {
2044                                        snippet[count++] = glyph_character(r);
2045                                        if (count == max_twin_snippet) {
2046                                            return 0;
2047                                        } else {
2048                                            prop = glyph_control(r);
2049                                            break;
2050                                        }
2051                                    } else {
2052                                        return 0;
2053                                    }
2054                                case kern_node:
2055                                    if (node_subtype(r) == font_kern_subtype) {
2056                                        break;
2057                                    } else {
2058                                        return 0;
2059                                    }
2060                                default:
2061                                    return 0;
2062                            }
2063                            r = node_next(r);
2064                        }
2065                        break;
2066                    }
2067                case kern_node:
2068                    if (node_subtype(current) == font_kern_subtype) {
2069                        break;
2070                    } else {
2071                        return 0;
2072                    }
2073                case glue_node:
2074                    {
2075                        halfword next = node_prev(current);
2076                        if (next && ((node_type(next) == glyph_node && tex_has_glyph_option(next, glyph_option_check_twin)) || node_type(next) == disc_node)) {
2077                            /* Just a test for an article, we need a field in the glyph node if we go this route. */
2078                            if (count > 1 && has_character_control(prop, ignore_twin_character_control_code)) {
2079                                --count;
2080                            }
2081                            return count;
2082                        }
2083                    }
2084                    return 0;
2085                default:
2086                    return 0;
2087            }
2088            current = node_next(current);
2089        }
2090    }
2091    return 0;
2092}
2093
2094static int tex_aux_same_snippet(int snippetone[], int snippettwo[], int nsone, int nstwo)
2095{
2096    if (nsone > 0 && nsone == nstwo) {
2097        for (int i = 0; i < nsone; i++) {
2098            if (snippetone[i] != snippettwo[i]) {
2099                return 0;
2100            }
2101        }
2102        return 1;
2103    } else {
2104        return 0;
2105    }
2106}
2107
2108/* */
2109
2110static scaled tex_aux_try_break(
2111    const line_break_properties *properties,
2112    halfword penalty,
2113    halfword break_type,
2114    halfword first_p,
2115    halfword cur_p,
2116    int callback_id,
2117    halfword checks,
2118    int pass,
2119    int subpass,
2120    int artificial
2121)
2122{
2123    /*tex stays a step behind |r| */
2124    halfword previous = active_head;
2125    /*tex a step behind |prev_r|, if |type(prev_r) = delta_node| */
2126    halfword before_previous = null;
2127    /*tex distance from current active node */
2128    scaled current_active_width[n_of_glue_amounts] = { 0 };
2129    /*tex
2130        These status arrays are global to the main loop and will be initialized as we go.
2131    */
2132    halfword best_place      [max_n_of_fitness_values] = { 0 };
2133    halfword best_place_line [max_n_of_fitness_values] = { 0 };
2134    scaled   best_place_short[max_n_of_fitness_values] = { 0 };
2135    scaled   best_place_glue [max_n_of_fitness_values] = { 0 };
2136    /*tex badness of test line */
2137    halfword badness = 0;
2138    halfword prev_badness = 0;
2139    /*tex demerits of test line */
2140    int demerits = 0;
2141    /*tex glue stretch or shrink of test line, adjustment for last line */
2142    scaled glue = 0;
2143    /*tex used in badness calculations */
2144    scaled shortfall = 0;
2145    /*tex maximum line number in current equivalence class of lines */
2146    halfword old_line = 0;
2147    /*tex have we found a feasible break at |cur_p|? */
2148    bool no_break_yet = true;
2149    /*tex should node |r| remain in the active list? */
2150    int current_stays_active;
2151    /*tex possible fitness class of test line */
2152    halfword fit_class;
2153    /*tex has |d| been forced to zero? */
2154    int artificial_demerits;
2155    /*tex the current line will be justified to this width */
2156    scaled line_width = 0;
2157    /*tex line number of current active node */
2158    halfword line = 0;
2159    /* */
2160    int twins = properties->left_twin_demerits > 0 || properties->right_twin_demerits > 0;
2161    halfword sf_factor = properties->sf_factor;
2162    halfword sf_stretch_factor = properties->sf_stretch_factor;
2163    /*tex
2164        We have added an extra category, just as experiment. In practice there is very little
2165        to gain here as it becomes kind of fuzzy and DEK values are quite okay.
2166    */
2167    /*tex Make sure that |pi| is in the proper range; */
2168    if (penalty >= infinite_penalty) {
2169        /*tex this breakpoint is inhibited by infinite penalty */
2170        return shortfall;
2171    } else if (penalty <= -infinite_penalty) {
2172        /*tex this breakpoint will be forced */
2173        penalty = eject_penalty; /* bad name here */
2174    }
2175    /*tex Consider a demerit for two lines with stretch/shrink based on expansion. */
2176    tex_aux_set_target_to_source(properties->adjust_spacing, current_active_width, lmt_linebreak_state.active_width);
2177    while (1) {
2178        /*tex Here |r| runs through the active list: */
2179        halfword current = node_next(previous);
2180        /*tex
2181
2182            If node |r| is of type |delta_node|, update |cur_active_width|, set |prev_r| and
2183            |prev_prev_r|, then |goto continue|. The following code uses the fact that |type
2184            (active) <> delta_node|.
2185
2186        */
2187        if (node_type(current) == delta_node) {
2188            tex_aux_add_to_target_from_delta(properties->adjust_spacing, current_active_width, current);
2189            before_previous = previous;
2190            previous = current;
2191            continue;
2192        } else {
2193            /*tex We have an |unhyphenated_node| or |hyphenated_node|. */
2194        }
2195        /*tex
2196
2197            If a line number class has ended, create new active nodes for the best feasible breaks
2198            in that class; then |return| if |r = active|, otherwise compute the new |line_width|.
2199
2200            The first part of the following code is part of \TEX's inner loop, so we don't want to
2201            waste any time. The current active node, namely node |r|, contains the line number that
2202            will be considered next. At the end of the list we have arranged the data structure so
2203            that |r = active| and |line_number (active) > old_l|.
2204
2205        */
2206        lmt_linebreak_state.current_line_number = line; /* we could just use this variable */
2207        line = active_line_number(current);
2208        if (line > old_line) {
2209            /*tex Now we are no longer in the inner loop (well ...). */
2210            if ((lmt_linebreak_state.minimum_demerits < awful_bad) && ((old_line != lmt_linebreak_state.easy_line) || (current == active_head))) {
2211                int s_before[max_twin_snippet];
2212                int s_after[max_twin_snippet];
2213                int n_before = -1;
2214                int n_after = -1;
2215                /*tex
2216
2217                    Create new active nodes for the best feasible breaks just found. It is not
2218                    necessary to create new active nodes having |minimal_demerits| greater than
2219                    |linebreak_state.minimum_demerits + abs (adj_demerits)|, since such active
2220                    nodes will never be chosen in the final paragraph breaks. This observation
2221                    allows us to omit a substantial number of feasible breakpoints from further
2222                    consideration.
2223
2224                */
2225                if (line > lmt_linebreak_state.easy_line) {
2226                    line_width = lmt_linebreak_state.second_width;
2227                } else if (line > lmt_linebreak_state.last_special_line) {
2228                    line_width = lmt_linebreak_state.second_width;
2229                } else if (properties->par_shape) {
2230                    line_width = tex_get_specification_width(properties->par_shape, line);
2231                } else {
2232                    line_width = lmt_linebreak_state.first_width;
2233                }
2234                if (no_break_yet) {
2235                    /*tex
2236
2237                        If we have a |hyphenated_node|, |delta_node| or |passive_node| the |cur_p| 
2238                        has to be a disc node! Look at the next emergency adaptions, we've actually 
2239                        set them elsewhere. So, when we end up here we have to be pretty sure that
2240                        these threesome are okay as there was no test for node type!
2241
2242                    */
2243                    no_break_yet = false;
2244                    if (lmt_linebreak_state.emergency_percentage) {
2245                        scaled stretch = tex_xn_over_d(line_width, lmt_linebreak_state.emergency_percentage, scaling_factor);
2246                        lmt_linebreak_state.background[total_stretch_amount] -= lmt_linebreak_state.emergency_amount;
2247                        lmt_linebreak_state.background[total_stretch_amount] += stretch;
2248                        lmt_linebreak_state.emergency_amount = stretch;
2249                    }
2250                    if (lmt_linebreak_state.emergency_width_extra) {
2251                        scaled extra = tex_xn_over_d(line_width, lmt_linebreak_state.emergency_width_extra, scaling_factor);
2252                        lmt_linebreak_state.background[total_advance_amount] -= lmt_linebreak_state.emergency_width_amount;
2253                        lmt_linebreak_state.background[total_advance_amount] += extra;
2254                        lmt_linebreak_state.emergency_width_amount = extra;
2255                    }
2256                    if (lmt_linebreak_state.emergency_left_extra) {
2257                        scaled extra = tex_xn_over_d(line_width, lmt_linebreak_state.emergency_left_extra, scaling_factor);
2258                        lmt_linebreak_state.background[total_advance_amount] -= lmt_linebreak_state.emergency_left_amount;
2259                        lmt_linebreak_state.background[total_advance_amount] += extra;
2260                        lmt_linebreak_state.emergency_left_amount = extra;
2261                    }
2262                    if (lmt_linebreak_state.emergency_right_extra) {
2263                        scaled extra = tex_xn_over_d(line_width, lmt_linebreak_state.emergency_right_extra, scaling_factor);
2264                        lmt_linebreak_state.background[total_advance_amount] -= lmt_linebreak_state.emergency_right_amount;
2265                        lmt_linebreak_state.background[total_advance_amount] += extra;
2266                        lmt_linebreak_state.emergency_right_amount = extra;
2267                    }
2268                    tex_aux_set_target_to_source(properties->adjust_spacing, lmt_linebreak_state.break_width, lmt_linebreak_state.background);
2269                    tex_aux_compute_break_width(break_type, properties->adjust_spacing, properties->adjust_spacing_step, sf_factor, sf_stretch_factor, cur_p);
2270                }
2271                /*tex
2272
2273                    Insert a delta node to prepare for breaks at |cur_p|. We use the fact that
2274                    |type (active) <> delta_node|.
2275
2276                */
2277                if (node_type(previous) == delta_node) {
2278                    /*tex modify an existing delta node */
2279                    tex_aux_add_delta_from_difference(properties->adjust_spacing, previous, lmt_linebreak_state.break_width, current_active_width);
2280                } else if (previous == active_head) {
2281                    /*tex no delta node needed at the beginning */
2282                    tex_aux_set_target_to_source(properties->adjust_spacing, lmt_linebreak_state.active_width, lmt_linebreak_state.break_width);
2283                } else {
2284                    halfword q = tex_new_node(delta_node, default_fitness); /* class and classes zero */
2285                    node_next(q) = current;
2286                    tex_aux_set_delta_from_difference(properties->adjust_spacing, q, lmt_linebreak_state.break_width, current_active_width);
2287                    node_next(previous) = q;
2288                    before_previous = previous;
2289                    previous = q;
2290                }
2291                if (properties->max_adj_demerits >= awful_bad - lmt_linebreak_state.minimum_demerits) {
2292                    lmt_linebreak_state.minimum_demerits = awful_bad - 1;
2293                } else {
2294                    lmt_linebreak_state.minimum_demerits += properties->max_adj_demerits;
2295                }
2296                for (halfword fit_class = default_fitness; fit_class <= tex_max_fitness(properties->fitness_classes); fit_class++) {
2297                    if (lmt_linebreak_state.minimal_demerits[fit_class] <= lmt_linebreak_state.minimum_demerits) {
2298                        /*tex
2299
2300                            Insert a new active node from |best_place [fit_class]| to |cur_p|. When
2301                            we create an active node, we also create the corresponding passive node.
2302                            In the passive node we also keep track of the subparagraph penalties.
2303
2304                        */
2305                        halfword passive = tex_new_node(passive_node, (quarterword) fit_class);
2306                        halfword active = tex_new_node((quarterword) break_type, (quarterword) fit_class);
2307                        halfword prev_break = best_place[fit_class];
2308                        /*tex Initialize the passive node: */
2309                        active_n_of_fitness_classes(active) = tex_max_fitness(properties->fitness_classes);
2310                        passive_n_of_fitness_classes(passive) = tex_max_fitness(properties->fitness_classes);
2311                        passive_cur_break(passive) = cur_p;
2312                        passive_serial(passive) = ++lmt_linebreak_state.serial_number;
2313                        passive_ref_count(passive) = 1;
2314                        passive_prev_break(passive) = prev_break;
2315                        passive_interline_penalty(passive) = lmt_linebreak_state.internal_interline_penalty;
2316                        passive_broken_penalty(passive) = lmt_linebreak_state.internal_broken_penalty;
2317                        passive_par_node(passive) = lmt_linebreak_state.internal_par_node;
2318                        passive_last_left_box(passive) = lmt_linebreak_state.internal_left_box;
2319                        passive_last_left_box_width(passive) = lmt_linebreak_state.internal_left_box_width;
2320                        if (prev_break) {
2321                            passive_left_box(passive) = passive_last_left_box(prev_break);
2322                            passive_left_box_width(passive) = passive_last_left_box_width(prev_break);
2323                            passive_ref_count(prev_break) += 1;
2324                        } else {
2325                            passive_left_box(passive) = lmt_linebreak_state.internal_left_box_init;
2326                            passive_left_box_width(passive) = lmt_linebreak_state.internal_left_box_width_init;
2327                        }
2328                        passive_right_box(passive) = lmt_linebreak_state.internal_right_box;
2329                        passive_right_box_width(passive) = lmt_linebreak_state.internal_right_box_width;
2330                        passive_middle_box(passive) = lmt_linebreak_state.internal_middle_box;
2331                        /*tex Initialize the active node: */
2332                        active_break_node(active) = passive;
2333                        active_line_number(active) = best_place_line[fit_class] + 1;
2334                        active_total_demerits(active) = lmt_linebreak_state.minimal_demerits[fit_class];
2335                        /*tex Store additional data in the new active node. */
2336                        tex_aux_set_quality(active, passive, best_place_short[fit_class], best_place_glue[fit_class], line_width, prev_badness);
2337                        /*tex Append the passive node. */
2338                        node_next(passive) = lmt_linebreak_state.passive;
2339                        lmt_linebreak_state.passive = passive;
2340                        /*tex Append the active node. */
2341                        node_next(active) = current;
2342                        node_next(previous) = active;
2343                        previous = active;
2344                        /*tex
2345                            Because we have a loop it can be that we have the same |cur_p| a few
2346                            times but it happens not that often. Nevertheless we do cache its
2347                            before and after snippets.
2348
2349                            We could check the second snippet immediately but the savings can be
2350                            neglected as we seldom enter this branch. It also measn two helpers
2351                            with the checking on also having to track the size. Messy and ugly
2352                            code for no gain so we just grab the whole second snippet and then
2353                            compare fast.
2354
2355                            Maybe take the largest demerits when we have two? Needs some discussion.
2356                        */
2357                        if (twins && cur_p && prev_break && passive_cur_break(prev_break)) {
2358                            int count = 0;
2359                            int snippet[max_twin_snippet];
2360                            if (properties->right_twin_demerits) {
2361                                if (n_before < 0) {
2362                                    n_before = tex_aux_get_before(cur_p, s_before);
2363                                }
2364                                if (n_before) {
2365                                    int n = tex_aux_get_before(passive_cur_break(prev_break), snippet);
2366                                    if (tex_aux_same_snippet(s_before, snippet, n_before, n)) {
2367                                        if (properties->tracing_paragraphs > 1) {
2368                                            tex_begin_diagnostic();
2369                                            tex_print_format("[linebreak: bumping demerits %i by right twin demerits %i]", active_total_demerits(active), properties->right_twin_demerits);
2370                                            tex_end_diagnostic();
2371                                        }
2372                                        active_total_demerits(active) += properties->right_twin_demerits;
2373                                        ++lmt_linebreak_state.n_of_right_twins;
2374                                        ++count;
2375                                    }
2376                                }
2377                            }
2378                            if (properties->left_twin_demerits) {
2379                                if (n_after < 0) {
2380                                    n_after = tex_aux_get_after(cur_p, s_after);
2381                                }
2382                                if (n_after) {
2383                                    int n = tex_aux_get_after(passive_cur_break(prev_break), snippet);
2384                                    if (tex_aux_same_snippet(s_after, snippet, n_after, n)) {
2385                                        if (properties->tracing_paragraphs > 1) {
2386                                            tex_begin_diagnostic();
2387                                            tex_print_format("[linebreak: bumping demerits %i by left twin demerits %i]", active_total_demerits(active), properties->left_twin_demerits);
2388                                            tex_end_diagnostic();
2389                                        }
2390                                        active_total_demerits(active) += properties->left_twin_demerits;
2391                                        ++lmt_linebreak_state.n_of_left_twins;
2392                                        ++count;
2393                                    }
2394                                }
2395                            }
2396                            if (count > 1) {
2397                                ++lmt_linebreak_state.n_of_double_twins;
2398                            }
2399                        }
2400                        /* */
2401                        if (callback_id) {
2402                            active_total_demerits(active) = tex_aux_line_break_callback_report(callback_id, checks, pass, subpass, active, passive);
2403                        }
2404                        if (properties->tracing_paragraphs > 0) {
2405                            tex_begin_diagnostic();
2406                            tex_aux_print_break_node(active, passive, lmt_linebreak_state.do_last_line_fit);
2407                            tex_end_diagnostic();
2408                        }
2409                    }
2410                    lmt_linebreak_state.minimal_demerits[fit_class] = awful_bad;
2411                }
2412                lmt_linebreak_state.minimum_demerits = awful_bad;
2413                /*tex
2414
2415                    Insert a delta node to prepare for the next active node. When the following
2416                    code is performed, we will have just inserted at least one active node before
2417                    |r|, so |type (prev_r) <> delta_node|.
2418
2419                */
2420                if (current != active_head) {
2421                    halfword delta = tex_new_node(delta_node, default_fitness);
2422                    node_next(delta) = current;
2423                    tex_aux_set_delta_from_difference(properties->adjust_spacing, delta, current_active_width, lmt_linebreak_state.break_width);
2424                    node_next(previous) = delta;
2425                    before_previous = previous;
2426                    previous = delta;
2427                }
2428            }
2429            /*tex
2430
2431                Quit on an active node, otherwise compute the new line width. When we come to the
2432                following code, we have just encountered the first active node~|r| whose
2433                |line_number| field contains |l|. Thus we want to compute the length of the
2434                $l\mskip1mu$th line of the current paragraph. Furthermore, we want to set |old_l|
2435                to the last number in the class of line numbers equivalent to~|l|.
2436
2437            */
2438            /* line_width already has been calculated */
2439            if (line > lmt_linebreak_state.easy_line) {
2440                old_line = max_halfword - 1;
2441                line_width = lmt_linebreak_state.second_width;
2442            } else {
2443                old_line = line;
2444                if (line > lmt_linebreak_state.last_special_line) {
2445                    line_width = lmt_linebreak_state.second_width;
2446                } else if (properties->par_shape) {
2447                    line_width = tex_get_specification_width(properties->par_shape, line);
2448                } else {
2449                    line_width = lmt_linebreak_state.first_width;
2450                }
2451            }
2452            if (current == active_head) {
2453                shortfall = line_width - current_active_width[total_advance_amount];
2454                return shortfall;
2455            }
2456        }
2457        /*tex
2458
2459            If a line number class has ended, create new active nodes for the best feasible breaks
2460            in that class; then |return| if |r = active|, otherwise compute the new |line_width|.
2461
2462            Consider the demerits for a line from |r| to |cur_p|; deactivate node |r| if it should
2463            no longer be active; then |goto continue| if a line from |r| to |cur_p| is infeasible,
2464            otherwise record a new feasible break.
2465
2466        */
2467        artificial_demerits = 0;
2468        shortfall = line_width - current_active_width[total_advance_amount];
2469        if (active_break_node(current)) {
2470            shortfall -= passive_last_left_box_width(active_break_node(current));
2471        } else {
2472            shortfall -= lmt_linebreak_state.internal_left_box_width_init;
2473        }
2474        shortfall -= lmt_linebreak_state.internal_right_box_width;
2475        if (properties->protrude_chars) {
2476            tex_check_protrusion_shortfall(current, first_p, cur_p, &shortfall);
2477        }
2478        /*tex
2479            The only reason why we have a shared ratio is that we need to calculate the shortfall
2480            for a line with mixed fonts. However in \LUAMETATEX\ we can mix and have no instances
2481            so ... The division by 2 comes from \PDFTEX\ where it isn't explained but it seems to
2482            work ok. Removing gives more overfull boxes and a different value gives different
2483            results.
2484        */
2485        if (shortfall > 0) {
2486            halfword total_stretch = current_active_width[font_stretch_amount];
2487            if (total_stretch > 0) {
2488                if (total_stretch > shortfall) {
2489                    shortfall  = total_stretch / 2;
2490                } else {
2491                    shortfall -= total_stretch;
2492                }
2493            }
2494        } else if (shortfall < 0) {
2495            halfword total_shrink = current_active_width[font_shrink_amount];
2496            if (total_shrink > 0) {
2497                if (total_shrink > -shortfall) {
2498                    shortfall  = - total_shrink / 2;
2499                } else {
2500                    shortfall += total_shrink;
2501                }
2502            }
2503        }
2504        if (shortfall > 0) {
2505            /*tex
2506
2507                Set the value of |b| to the badness for stretching the line, and compute the
2508                corresponding |fit_class|. When a line must stretch, the available stretchability
2509                can be found in the subarray |cur_active_width [2 .. 6]|, in units of points, sfi,
2510                fil, fill and filll.
2511
2512                The present section is part of \TEX's inner loop, and it is most often performed
2513                when the badness is infinite; therefore it is worth while to make a quick test for
2514                large width excess and small stretchability, before calling the |badness| subroutine.
2515
2516            */
2517            if (current_active_width[total_fi_amount]   || current_active_width[total_fil_amount] ||
2518                current_active_width[total_fill_amount] || current_active_width[total_filll_amount]) {
2519                if (lmt_linebreak_state.do_last_line_fit) {
2520                    if (! cur_p) {
2521                        /*tex
2522
2523                            The last line of a paragraph. Perform computations for last line and
2524                            |goto found|. Here we compute the adjustment |g| and badness |b| for a
2525                            line from |r| to the end of the paragraph. When any of the criteria for
2526                            adjustment is violated we fall through to the normal algorithm. The last
2527                            line must be too short, and have infinite stretch entirely due to
2528                            |par_fill_skip|.
2529
2530                        */
2531                        if (active_short(current) == 0 || active_glue(current) <= 0) {
2532                            /*tex
2533
2534                                Previous line was neither stretched nor shrunk, or was infinitely
2535                                bad.
2536
2537                            */
2538                            goto NOT_FOUND;
2539                        }
2540                        if (current_active_width[total_fi_amount]   != lmt_linebreak_state.fill_width[fi_order]   || current_active_width[total_fil_amount]   != lmt_linebreak_state.fill_width[fil_order] ||
2541                            current_active_width[total_fill_amount] != lmt_linebreak_state.fill_width[fill_order] || current_active_width[total_filll_amount] != lmt_linebreak_state.fill_width[filll_order]) {
2542                            /*tex
2543                                Infinite stretch of this line not entirely due to |par_fill_skip|.
2544                            */
2545                            goto NOT_FOUND;
2546                        }
2547                        if (active_short(current) > 0) {
2548                            glue = current_active_width[total_stretch_amount];
2549                        } else {
2550                            glue = current_active_width[total_shrink_amount];
2551                        }
2552                        if (glue <= 0) {
2553                            /*tex No finite stretch and no shrink. */
2554                            goto NOT_FOUND;
2555                        }
2556                        lmt_scanner_state.arithmic_error = 0;
2557                        glue = tex_fract(glue, active_short(current), active_glue(current), max_dimension);
2558                        if (properties->last_line_fit < 1000) {
2559                            glue = tex_fract(glue, properties->last_line_fit, 1000, max_dimension);
2560                        }
2561                        if (lmt_scanner_state.arithmic_error) {
2562                            glue = (active_short(current) > 0) ? max_dimension : -max_dimension;
2563                        }
2564                        if (glue > 0) {
2565                            /*tex
2566
2567                                Set the value of |b| to the badness of the last line for stretching,
2568                                compute the corresponding |fit_class, and |goto found|. These
2569                                badness computations are rather similar to those of the standard
2570                                algorithm, with the adjustment amount |g| replacing the |shortfall|.
2571
2572                            */
2573                            if (glue > shortfall) {
2574                                glue = shortfall;
2575                            }
2576                            if (glue > large_width_excess && (current_active_width[total_stretch_amount] < small_stretchability)) {
2577                                badness = infinite_bad;
2578                                fit_class = default_fitness;
2579                            } else {
2580                                badness = tex_badness(glue, current_active_width[total_stretch_amount]);
2581                                fit_class = tex_normalized_loose_badness(badness, properties->fitness_classes);
2582                            }
2583                            goto FOUND;
2584                        } else if (glue < 0) {
2585                            /*tex
2586
2587                                Set the value of |b| to the badness of the last line for shrinking,
2588                                compute the corresponding |fit_class, and |goto found||.
2589
2590                            */
2591                            if (-glue > current_active_width[total_shrink_amount]) {
2592                                glue = -current_active_width[total_shrink_amount];
2593                            }
2594                            badness = tex_badness(-glue, current_active_width[total_shrink_amount]);
2595                            fit_class = tex_normalized_tight_badness(badness, properties->fitness_classes);
2596                            goto FOUND;
2597                        }
2598                    }
2599                  NOT_FOUND:
2600                    shortfall = 0;
2601                }
2602                badness = 0;
2603                /*tex Infinite stretch. */
2604                fit_class = tex_get_specification_decent(properties->fitness_classes) - 1;
2605            } else if (shortfall > large_width_excess && current_active_width[total_stretch_amount] < small_stretchability) {
2606                badness = infinite_bad;
2607                fit_class = default_fitness;
2608            } else {
2609                badness = tex_badness(shortfall, current_active_width[total_stretch_amount]);
2610                fit_class = tex_normalized_loose_badness(badness, properties->fitness_classes);
2611            }
2612        } else {
2613            /*tex
2614
2615                Set the value of |b| to the badness for shrinking the line, and compute the
2616                corresponding |fit_class|. Shrinkability is never infinite in a paragraph; we
2617                can shrink the line from |r| to |cur_p| by at most |cur_active_width
2618                [total_shrink_amount]|.
2619
2620            */
2621            if (-shortfall > current_active_width[total_shrink_amount]) {
2622                badness = infinite_bad + 1;
2623            } else {
2624                badness = tex_badness(-shortfall, current_active_width[total_shrink_amount]);
2625        }
2626            fit_class = tex_normalized_tight_badness(badness, properties->fitness_classes);
2627        }
2628        if (1) { // lmt_linebreak_state.do_last_line_fit) {
2629            /*tex Adjust the additional data for last line; */
2630            if (! cur_p) {
2631                shortfall = 0;
2632                glue = 0;
2633            } else if (shortfall > 0) {
2634                glue = current_active_width[total_stretch_amount];
2635            } else if (shortfall < 0) {
2636                glue = current_active_width[total_shrink_amount];
2637            } else {
2638                glue = 0;
2639            }
2640        } else {
2641            /* Can we get here at all? */
2642        }
2643      FOUND:
2644        if ((badness > infinite_bad) || (penalty == eject_penalty)) {
2645            /*tex
2646
2647                Prepare to deactivate node~|r|, and |goto deactivate| unless there is a reason to
2648                consider lines of text from |r| to |cur_p|. During the final pass, we dare not
2649                lose all active nodes, lest we lose touch with the line breaks already found. The
2650                code shown here makes sure that such a catastrophe does not happen, by permitting
2651                overfull boxes as a last resort. This particular part of \TEX\ was a source of
2652                several subtle bugs before the correct program logic was finally discovered; readers
2653                who seek to improve \TEX\ should therefore think thrice before daring to make any
2654                changes here.
2655
2656                This last remark applies to HH & MS and it also makes par passes (1 & 2) hard to act
2657                compatible because we lack a criterium.
2658
2659            */
2660
2661            if (artificial && (lmt_linebreak_state.minimum_demerits == awful_bad) && (node_next(current) == active_head) && (previous == active_head)) {
2662                /*tex Set demerits zero, this break is forced. */
2663                artificial_demerits = 1;
2664            } else if (badness > lmt_linebreak_state.threshold) {
2665                goto DEACTIVATE;
2666            }
2667            current_stays_active = 0;
2668        } else {
2669            previous = current;
2670            if (badness > lmt_linebreak_state.threshold) {
2671                continue;
2672            } else {
2673                current_stays_active = 1;
2674            }
2675        }
2676        /*tex
2677
2678            Record a new feasible break. When we get to this part of the code, the line from |r| to
2679            |cur_p| is feasible, its badness is~|b|, and its fitness classification is |fit_class|.
2680            We don't want to make an active node for this break yet, but we will compute the total
2681            demerits and record them in the |minimal_demerits| array, if such a break is the current
2682            champion among all ways to get to |cur_p| in a given line-number class and fitness class.
2683
2684        */
2685        if (artificial_demerits) {
2686            demerits = 0;
2687        } else {
2688            /*tex Compute the demerits, |d|, from |r| to |cur_p|. */
2689            int fit_current = (halfword) active_fitness(current);
2690            int distance = abs(fit_class - fit_current);
2691            demerits = badness + properties->line_penalty;
2692            if (abs(demerits) >= infinite_bad) {
2693                demerits = extremely_deplorable;
2694            } else {
2695                demerits = demerits * demerits;
2696            }
2697            if (penalty) {
2698                if (penalty > 0) {
2699                    demerits += (penalty * penalty);
2700                } else if (penalty > eject_penalty) {
2701                    demerits -= (penalty * penalty);
2702                }
2703            }
2704            /*tex
2705                If we have a hyphen penalty of 50 and add 5000 to the demerits in case of double
2706                hyphens, we do that only once, so for triple we get:
2707
2708                \starttyping
2709                xxxxxxxxxxxxx- @ xxxxxxxxxxxxxxxx  (d + 2500)
2710                xxxxxxxxxxxxx- @ xxxxxxxxxxxxxxxx  (d + 2500) + 5000
2711                xxxxxxxxxxxxx- @ xxxxxxxxxxxxxxxx  (d + 2500) + 5000
2712                \stoptyping
2713
2714                We considered an array or a multiplier but when one has more than two in a row it
2715                is already quite penalized and one has to go very tolerant and stretch so then the
2716                solution space gets better and the threesome likely goes away.
2717            */
2718            if (break_type == hyphenated_node && node_type(current) == hyphenated_node) {
2719                if (cur_p) {
2720                    demerits += properties->double_hyphen_demerits;
2721                } else {
2722                    demerits += properties->final_hyphen_demerits;
2723                }
2724            }
2725            /*tex
2726                Here |fitness| is just the subtype, so we could have put the cast in the macro
2727                instead: |#define fitness (n) ((halfword) (subtype (n))|. We need to cast because
2728                some compilers (versions or whatever) get confused by the type of (unsigned) integer
2729                used.
2730            */
2731            demerits += tex_get_demerits(properties, distance, fit_current, fit_class);
2732        }
2733        prev_badness = badness;
2734        if (properties->tracing_paragraphs > 0) {
2735         // tex_begin_diagnostic();
2736            tex_aux_print_feasible_break(cur_p, current, badness, penalty, demerits, artificial_demerits, fit_class, lmt_linebreak_state.printed_node);
2737         // tex_end_diagnostic();
2738        }
2739        /*tex This is the minimum total demerits from the beginning to |cur_p| via |r|. */
2740        demerits += active_total_demerits(current);
2741        if (demerits <= lmt_linebreak_state.minimal_demerits[fit_class]) {
2742            lmt_linebreak_state.minimal_demerits[fit_class] = demerits;
2743            best_place[fit_class] = active_break_node(current);
2744            best_place_line[fit_class] = line;
2745            /*tex
2746                Store additional data for this feasible break. For each feasible break we record
2747                the shortfall and glue stretch or shrink (or adjustment). Contrary to \ETEX\ we
2748                do this always, also when we have no last line fit set.
2749            */
2750            best_place_short[fit_class] = shortfall;
2751            best_place_glue[fit_class] = glue;
2752            if (demerits < lmt_linebreak_state.minimum_demerits) {
2753                lmt_linebreak_state.minimum_demerits = demerits;
2754            }
2755        }
2756        /*tex Record a new feasible break. */
2757        if (current_stays_active) {
2758            /*tex |prev_r| has been set to |r|. */
2759            continue;
2760        }
2761      DEACTIVATE:
2762        /*tex
2763
2764            Deactivate node |r|. When an active node disappears, we must delete an adjacent delta
2765            node if the active node was at the beginning or the end of the active list, or if it
2766            was surrounded by delta nodes. We also must preserve the property that |cur_active_width|
2767            represents the length of material from |vlink (prev_r)| to~|cur_p|.
2768
2769        */
2770        {
2771            halfword passive = active_break_node(current);
2772            node_next(previous) = node_next(current);
2773            if (passive) {
2774                passive_ref_count(passive) -= 1;
2775                if (callback_id) {
2776                    /*tex Not that usefull, basically every passive is touched. */
2777                    switch (node_type(current)) {
2778                        case unhyphenated_node:
2779                        case hyphenated_node:
2780                            tex_aux_line_break_callback_delete(callback_id, checks, passive);
2781                            break;
2782                    //  case delta_node:
2783                    //      break;
2784                    }
2785                }
2786            }
2787         // tex_flush_node(current);
2788            tex_free_node(current, get_node_size(node_type(current))); // less overhead & testing
2789        }
2790        if (previous == active_head) {
2791            /*tex
2792
2793                Update the active widths, since the first active node has been deleted. The following
2794                code uses the fact that |type (active) <> delta_node|. If the active list has just
2795                become empty, we do not need to update the |active_width| array, since it will be
2796                initialized when an active node is next inserted.
2797
2798            */
2799            current = node_next(active_head);
2800            if (node_type(current) == delta_node) {
2801                tex_aux_add_to_target_from_delta(properties->adjust_spacing, lmt_linebreak_state.active_width, current);
2802                tex_aux_set_target_to_source(properties->adjust_spacing, current_active_width, lmt_linebreak_state.active_width);
2803                node_next(active_head) = node_next(current);
2804             // tex_flush_node(current);
2805                tex_free_node(current, delta_node_size); // less overhead & testing
2806            }
2807        } else if (node_type(previous) == delta_node) {
2808            current = node_next(previous);
2809            if (current == active_head) {
2810                tex_aux_sub_delta_from_target(properties->adjust_spacing, current_active_width, previous);
2811                node_next(before_previous) = active_head;
2812             // tex_flush_node(previous);
2813                tex_free_node(previous, delta_node_size); // less overhead & testing
2814                previous = before_previous;
2815            } else if (node_type(current) == delta_node) {
2816                tex_aux_add_to_target_from_delta(properties->adjust_spacing, current_active_width, current);
2817                tex_aux_add_to_delta_from_delta(properties->adjust_spacing, previous, current);
2818                node_next(previous) = node_next(current);
2819             // tex_flush_node(current);
2820                tex_free_node(current, delta_node_size); // less overhead & testing
2821           }
2822        }
2823    }
2824    /* We never end up here. */
2825    return shortfall;
2826}
2827
2828
2829
2830static halfword tex_aux_inject_orphan_penalty(const line_break_properties *properties, halfword current, halfword amount, int orphaned)
2831{
2832    halfword prev = node_prev(current);
2833    if (prev && node_type(prev) != penalty_node) {
2834        halfword penalty = tex_new_penalty_node(amount, orphan_penalty_subtype);
2835        tex_couple_nodes(prev, penalty);
2836        tex_couple_nodes(penalty, current);
2837        if (properties->tracing_paragraphs > 1 || properties->tracing_orphans) {
2838            tex_begin_diagnostic();
2839            tex_print_format("[linebreak: adding orphanpenalty %i]\n", amount);
2840            tex_end_diagnostic();
2841        }
2842        if (orphaned) {
2843            tex_add_penalty_option(penalty, penalty_option_orphaned);
2844        }
2845        lmt_linebreak_state.has_orphans = 1;
2846        return prev;
2847    } else {
2848        return current;
2849    }
2850}
2851
2852static halfword tex_aux_inject_toddler_penalty(const line_break_properties *properties, halfword current, halfword amount, halfword tnuoma, int toddlered, int duplex)
2853{
2854    halfword next = node_next(current);
2855    halfword prev = node_prev(current);
2856    halfword penalty = null;
2857    halfword nepalty = null;
2858    /* what if we already have a penalty there, we need to take the max */
2859    if (duplex) {
2860        penalty = tex_new_penalty_node(amount, toddler_penalty_subtype);
2861        nepalty = tex_new_penalty_node(tnuoma, toddler_penalty_subtype);
2862        tex_couple_nodes(current, penalty);
2863        tex_couple_nodes(penalty, next);
2864        if (node_type(prev) == glue_node && node_prev(prev)) {
2865            tex_couple_nodes(node_prev(prev), nepalty);
2866            tex_couple_nodes(nepalty, prev);
2867        } else {
2868            tex_couple_nodes(prev, nepalty);
2869            tex_couple_nodes(nepalty, current);
2870        }
2871        if (toddlered) {
2872            tex_add_penalty_option(penalty, penalty_option_toddlered);
2873            tex_add_penalty_option(nepalty, penalty_option_toddlered);
2874        }
2875    } else {
2876        penalty = tex_new_penalty_node(amount, toddler_penalty_subtype);
2877        tex_couple_nodes(penalty, next);
2878        tex_couple_nodes(current, penalty);
2879        if (toddlered) {
2880            tex_add_penalty_option(penalty, penalty_option_toddlered);
2881        }
2882    }
2883    if (properties->tracing_paragraphs > 1 || properties->tracing_toddlers) {
2884        tex_begin_diagnostic();
2885        tex_print_format("[linebreak: adding toddlerpenalty %i %i]\n", tnuoma, amount);
2886        tex_end_diagnostic();
2887    }
2888    lmt_linebreak_state.has_toddlers = 1;
2889    return penalty;
2890}
2891
2892static inline int tex_aux_valid_glue_break(halfword p)
2893{
2894    halfword prv = node_prev(p);
2895    return (prv && prv != temp_head && (node_type(prv) == glyph_node || precedes_break(prv) || precedes_kern(prv) || precedes_dir(prv)));
2896}
2897
2898static inline halfword tex_aux_upcoming_math_penalty(halfword p, halfword factor) {
2899    halfword n = node_next(p);
2900    if (n && node_type(n) == math_node && node_subtype(n) == begin_inline_math) {
2901        return factor ? tex_xn_over_d(math_penalty(n), factor, scaling_factor) : math_penalty(n);
2902    } else {
2903        return 0;
2904    }
2905}
2906
2907/*tex
2908
2909    I played a bit with a height driven hanging indentation. One can store |cur_p| in the active
2910    node and progressively calculate the height + depth and then act on that but in the end
2911    interline space, adjustsm etc. also have to be taken into account and that all happens later
2912    so in the end it makes no sense. There are valdi reasons why \TEX\ can't do some things
2913    reliable: user demands are unpredictable.
2914
2915*/
2916
2917/*tex
2918
2919    Here we pickup the line number from |prev_graf| which relates to display math inside a
2920    paragraph. A display formula is then considered to span three lines. Of course this also
2921    assume a constant baseline distance with lines heigths not exceeding that amount. It also
2922    assumes that the shape and hang are not reset. We check the prevgraf for a large value
2923    because when we're close to |max_integer| we can wrap around due to addition beyond that
2924    and negative values has side effects (see musings-sideffects) but it's optional so that we
2925    can actually use these side effects.
2926
2927*/
2928
2929# define max_prev_graf (max_integer/2)
2930
2931static inline int tex_aux_short_math(halfword m) 
2932{
2933    return m && node_subtype(m) == begin_inline_math && math_penalty(m) > 0 && tex_has_math_option(m, math_option_short);
2934}
2935
2936static inline void tex_aux_adapt_short_math_penalty(halfword m, halfword p1, halfword p2, int orphaned) 
2937{
2938    if (p1 > math_penalty(m)) {
2939        math_penalty(m) = p1;
2940        if (orphaned) {
2941            tex_add_math_option(m, math_option_orphaned);
2942        }
2943    }
2944    if (p2 > math_penalty(m)) {
2945        math_penalty(m) = p2;
2946        if (orphaned) {
2947            tex_add_math_option(m, math_option_orphaned);
2948        }
2949    }
2950}
2951
2952static inline halfword tex_aux_backtrack_over_math(halfword m)
2953{
2954    if (node_subtype(m) == end_inline_math) {
2955        do {
2956            m = node_prev(m);
2957        } while (m && node_type(m) != math_node);
2958    }
2959    return m;
2960}
2961
2962static inline int tex_aux_emergency(const line_break_properties *properties)
2963{
2964    if (properties->emergency_stretch > 0) {
2965        return 1;
2966    } else if (properties->emergency_extra_stretch > 0) {
2967        return 1;
2968    } else if (! (tex_glue_is_zero(properties->emergency_left_skip) && glue_stretch_order(properties->emergency_left_skip) == normal_glue_order && glue_shrink_order(properties->emergency_left_skip) == normal_glue_order)) {
2969        return 1;
2970    } else if (! (tex_glue_is_zero(properties->emergency_right_skip) && glue_stretch_order(properties->emergency_right_skip) == normal_glue_order && glue_shrink_order(properties->emergency_right_skip) == normal_glue_order)) {
2971        return 1;
2972    } else {
2973        return 0;
2974    }
2975}
2976
2977static inline int tex_aux_emergency_skip(halfword s)
2978{
2979    return ! tex_glue_is_zero(s) && glue_stretch_order(s) == normal_glue_order && glue_shrink_order(s) == normal_glue_order;
2980}
2981
2982static scaled tex_check_linebreak_quality(scaled shortfall, scaled *overfull, scaled *underfull, halfword *verdict, halfword *classified)
2983{
2984    halfword active = active_break_node(lmt_linebreak_state.best_bet);
2985    halfword passive = passive_prev_break(active);
2986    int result = 1;
2987    /* last line ... */
2988    switch (active_quality(active)) {
2989        case par_is_overfull:
2990            *overfull = active_deficiency(active);
2991            *underfull = 0;
2992            break;
2993        case par_is_underfull:
2994            *overfull = 0;
2995            *underfull = active_deficiency(active);
2996            break;
2997        default:
2998            *overfull = 0;
2999            *underfull = 0;
3000            break;
3001    }
3002    *verdict = active_total_demerits(active);
3003    *classified |= (1 << active_fitness(active));
3004    /* previous lines */
3005    if (passive) {
3006        while (passive) {
3007            switch (passive_quality(passive)) {
3008                case par_is_overfull:
3009                    if (passive_deficiency(passive) > *overfull) {
3010                        *overfull = passive_deficiency(passive);
3011                    }
3012                    break;
3013                case par_is_underfull:
3014                    if (passive_deficiency(passive) > *underfull) {
3015                        *underfull = passive_deficiency(passive);
3016                    }
3017                    break;
3018                default:
3019                    /* not in tex */
3020                    break;
3021            }
3022            // *classified |= classification[node_subtype(q)];
3023            *classified |= (1 << passive_fitness(passive));
3024            if (passive_demerits(passive) > *verdict) {
3025                *verdict = passive_demerits(passive);
3026            }
3027            passive = passive_prev_break(passive);
3028        }
3029    } else {
3030        /*tex
3031            Likely a single overfull line, but is this always okay? When the tolerance is too
3032            low an emergency pass is needed then but in our case we want to enter par passes. We
3033            might need an option to disable this branch when we go for par passes only in order
3034            to be compatible with regular TeX.
3035        */
3036        if (passive_demerits(active) > *verdict) {
3037            *verdict = passive_demerits(active);
3038            result = 2;
3039        }
3040        if (-shortfall > *overfull) {
3041            *overfull = -shortfall;
3042            result = 2;
3043        }
3044    }
3045    if (*verdict > infinite_bad) {
3046        *verdict = infinite_bad;
3047    }
3048    return result;
3049}
3050
3051static void tex_aux_quality_callback(
3052    int callback_id, halfword par,
3053    int id, int pass, int subpass, int subpasses, int state,
3054    halfword overfull, halfword underfull, halfword verdict, halfword classified
3055)
3056{
3057    lmt_run_callback(
3058        lmt_lua_state.lua_instance, callback_id, "Ndddddddddd->N",
3059        par, id, pass, subpass, subpasses, state,
3060        overfull, underfull, verdict, classified,
3061        lmt_packaging_state.pack_begin_line,
3062        &lmt_linebreak_state.inject_after_par
3063    );
3064}
3065
3066/*
3067    The orphan penalty injection is something new. It works backward so the first penalty in
3068    the list is injected first. If there is a penalty before a space we skip that space and
3069    also skip a penalty in the list.
3070*/
3071
3072static void tex_aux_remove_special_penalties(line_break_properties *properties)
3073{
3074    halfword current = node_prev(properties->parfill_right_skip);
3075    while (current) {
3076        switch (node_type(current)) {
3077            case glue_node:
3078            case penalty_node:
3079                current = node_prev(current);
3080                break;
3081            default:
3082                goto CHECK;
3083        }
3084    }
3085  CHECK:
3086    while (current) {
3087        halfword prev = node_prev(current);
3088        switch (node_type(current)) {
3089            case penalty_node:
3090                switch (node_subtype(current)) {
3091                    case orphan_penalty_subtype:
3092                        if (tex_has_penalty_option(current, penalty_option_orphaned)) {
3093                            tex_try_couple_nodes(prev, node_next(current));
3094                            tex_flush_node(current);
3095                        }
3096                        break;
3097                    case toddler_penalty_subtype:
3098                        if (tex_has_penalty_option(current, penalty_option_toddlered)) {
3099                            tex_try_couple_nodes(prev, node_next(current));
3100                            tex_flush_node(current);
3101                        }
3102                        break;
3103                }
3104                break;
3105            case math_node:
3106                 current = tex_aux_backtrack_over_math(current);
3107                 if (tex_aux_short_math(current)) {
3108                     if (tex_has_math_option(current, math_option_orphaned)) {
3109                        math_penalty(current) = 0;
3110                        tex_remove_math_option(current, math_option_orphaned);
3111                     }
3112                 }
3113                return;
3114            case disc_node:
3115                if (tex_has_disc_option(current, disc_option_orphaned)) {
3116                    disc_orphaned(current) = 0;
3117                    tex_remove_disc_option(current, disc_option_orphaned);
3118                }
3119                break;
3120        }
3121        current = prev;
3122    }
3123    lmt_linebreak_state.has_orphans = 0;
3124    lmt_linebreak_state.has_toddlers = 0;
3125}
3126
3127static void tex_aux_apply_special_penalties(const line_break_properties *properties, halfword current, int state)
3128{
3129    if (paragraph_has_math(state)) {
3130        halfword factor = properties->math_penalty_factor; /* maybe also use penalty_used for this one but no room in math_node */
3131        if (factor) {
3132            while (current) {
3133                switch (node_type(current)) {
3134                    case penalty_node:
3135                        switch (node_subtype(current)) {
3136                            case math_pre_penalty_subtype:
3137                            case math_post_penalty_subtype:
3138                                if (penalty_amount(current)) {
3139                                    penalty_amount(current) = tex_xn_over_d(penalty_amount(current), factor, scaling_factor);
3140                                }
3141                                break;
3142                            case orphan_penalty_subtype:
3143                            case toddler_penalty_subtype:
3144                                if (tex_has_penalty_option(current, penalty_option_factor_used)) {
3145                                    penalty_amount(current) = penalty_used(current);
3146                                }
3147                                break;
3148                        }
3149                        break;
3150                    case math_node:
3151                        if (math_penalty(current)) {
3152                            math_penalty(current) = tex_xn_over_d(math_penalty(current), factor, scaling_factor);
3153                        }
3154                        break;
3155                }
3156                current = node_next(current);
3157            }
3158        }
3159    } else if (lmt_linebreak_state.has_toddlers || lmt_linebreak_state.has_orphans) {
3160        while (current) {
3161            if (node_type(current) == penalty_node) {
3162                switch (node_subtype(current)) {
3163                    case orphan_penalty_subtype:
3164                    case toddler_penalty_subtype:
3165                        if (tex_has_penalty_option(current, penalty_option_factor_used)) {
3166                            penalty_amount(current) = penalty_used(current);
3167                        }
3168                        break;
3169                }
3170            }
3171            current = node_next(current);
3172        }
3173    }
3174}
3175
3176static void tex_aux_apply_special_factors(const line_break_properties *properties, halfword current, int state)
3177{
3178    if (paragraph_has_factor(state) && (properties->sf_factor || properties->sf_stretch_factor)) {
3179        while (current) {
3180            if (node_type(current) == glue_node && tex_has_glue_option(current, glue_option_has_factor)) {
3181                glue_amount(current) = tex_aux_applied_amount(current, properties->sf_factor);
3182                glue_stretch(current) = tex_aux_applied_stretch(current, properties->sf_stretch_factor);
3183            }
3184            current = node_next(current);
3185        }
3186    }
3187}
3188
3189/*tex
3190    We could act upon a callback if needed. We could also have a flag in the hc property
3191    but then we also need to carry that in glyph options (glyph_option_text or so). One could
3192    also argue for uppercase only in which case we shopuld have a uppercase flag in the glyph
3193    state.
3194
3195    An alternative is to delay this till we check a break and add to the demerits as we do with
3196    twins but this is also a bit of a demo of how to do this.
3197*/
3198
3199static void tex_aux_fix_toddler_penalties(const line_break_properties *properties, halfword duplex, halfword head, halfword tail, int trace, int count)
3200{
3201    halfword done = 1;
3202    halfword start = tail;
3203    tail = node_prev(start);
3204    while (tail) {
3205        if (node_type(tail) == glyph_node && tex_has_glyph_option(tail, glyph_option_is_toddler)) {
3206            halfword prev = 0;
3207            halfword next = 0;
3208            halfword left = null;
3209            halfword right = null;
3210            if (duplex) {
3211                prev = node_prev(tail);
3212                next = node_next(tail);
3213                if (node_type(prev) == glue_node && node_prev(prev)) {
3214                    prev = node_prev(prev);
3215                }
3216                left = tex_get_specification_penalty(properties->toddler_penalties, ++done);
3217                right = tex_get_specification_nepalty(properties->toddler_penalties, done);
3218                penalty_amount(prev) = left;
3219                penalty_amount(next) = right;
3220                if (trace) {
3221                    tex_begin_diagnostic();
3222                    tex_print_format("[linebreak: toddler penalties %i %i fixed]\n", left, right);
3223                    tex_end_diagnostic();
3224                }
3225            } else {
3226                next = node_next(tail);
3227                right = tex_get_specification_penalty(properties->toddler_penalties, ++done);
3228                penalty_amount(next) = right;
3229                if (trace) {
3230                    tex_begin_diagnostic();
3231                    tex_print_format("[linebreak: toddler penalty %i fixed]\n", right);
3232                    tex_end_diagnostic();
3233                }
3234            }
3235        }
3236        if (tail == head) {
3237            break;
3238        } else {
3239            tail = node_prev(tail);
3240        }
3241    }
3242    /* plenty of safeguards here */
3243    tail = start;
3244    while (tail) {
3245        if (node_type(tail) == glyph_node && tex_has_glyph_option(tail, glyph_option_is_toddler)) {
3246            halfword next = node_next(tail);
3247            if (next && node_type(next) == penalty_node) {
3248                if (! penalty_amount(next)) {
3249                    tex_couple_nodes(node_prev(next), node_next(next));
3250                    tex_flush_node(next);
3251                }
3252            }
3253            if (duplex) {
3254                halfword prev = node_prev(tail);
3255                if (prev) {
3256                    halfword left = 0;
3257                    if (node_type(prev) == glue_node && node_prev(prev)) {
3258                        prev = node_prev(prev);
3259                    }
3260                    if (prev && node_type(prev) == penalty_node) {
3261                        if (--count > 0) {
3262                            penalty_amount(prev) = 0;
3263                        } else {
3264                            left = penalty_amount(prev);
3265                        }
3266                        if (! left) {
3267                            tex_couple_nodes(node_prev(prev), node_next(prev));
3268                            tex_flush_node(prev);
3269                        }
3270                    }
3271                }
3272            }
3273        }
3274        if (tail == head) {
3275            break;
3276        } else {
3277            tail = node_prev(tail);
3278        }
3279    }
3280}
3281
3282/* we already have these somewhere */
3283
3284static inline int tex_aux_prev_penalty_found(halfword current)
3285{
3286    halfword prev = node_prev(current);
3287    if (prev) {
3288        switch (node_type(prev)) {
3289            case glue_node:
3290                prev = node_prev(prev);
3291                return prev && node_type(prev) == penalty_node && node_subtype(prev) != toddler_penalty_subtype;
3292            case penalty_node:
3293                return node_subtype(prev) != toddler_penalty_subtype;
3294            default:
3295                return 0;
3296        }
3297    } else {
3298        return 0;
3299    }
3300}
3301
3302static inline int tex_aux_next_penalty_found(halfword current)
3303{
3304    halfword next = node_next(current);
3305    if (next) {
3306        switch (node_type(next)) {
3307            case glue_node:
3308                next = node_prev(next);
3309                return next && node_type(next) == penalty_node && node_subtype(next) != toddler_penalty_subtype;
3310            case penalty_node:
3311                return node_subtype(next) != toddler_penalty_subtype;
3312            default:
3313                return 0;
3314        }
3315    } else {
3316        return 0;
3317    }
3318}
3319
3320static void tex_aux_set_toddler_penalties(const line_break_properties *properties, int toddlered)
3321{
3322    if (properties->toddler_penalties) {
3323        int found = lmt_linebreak_state.has_toddlers;
3324        if (found == 0) {
3325            /* mark toddlers */
3326            halfword current = node_next(properties->parinit_left_skip);
3327            halfword count = 0;
3328            halfword done = 0;
3329            halfword glyph = null;
3330            halfword mathlevel = 0;
3331            while (current) {
3332                switch (node_type(current)) {
3333                    case glue_node:
3334                        if (! mathlevel) {
3335                            switch (node_subtype(current)) {
3336                                case space_skip_glue:
3337                                case xspace_skip_glue:
3338                                case zero_space_skip_glue:
3339                                    if (glyph && done && count == 1) {
3340                                        tex_add_glyph_option(glyph, glyph_option_is_toddler);
3341                                        found += 1;
3342                                    }
3343                                    done = 1;
3344                                    break;
3345                            }
3346                            count = 0;
3347                        } else {
3348                            /*tex Not needed but shows the issue. */
3349                            count += 2;
3350                        }
3351                        break;
3352                    case glyph_node:
3353                        if (! mathlevel) {
3354                            glyph = current;
3355                            if (glyph_node_is_text(current) && tex_has_glyph_option(current, glyph_option_check_toddler)) {
3356                                ++count;
3357                            }
3358                        } else {
3359                            /*tex Not needed but shows the issue. */
3360                            count += 2;
3361                        }
3362                        break;
3363                    case math_node:
3364                        /* todo use skip over math helper */
3365                        switch (node_subtype(current)) {
3366                            case begin_inline_math:
3367                                ++mathlevel;
3368                                break;
3369                            case end_inline_math:
3370                                --mathlevel;
3371                                break;
3372                        }
3373                        /*tex Bump more so that we have word e.g. $x$, and the comma is no toddler. */
3374                        count += 2;
3375                        break;
3376                    default:
3377                        /* tex We have to make sure that we don't end up with one glyph bound to something. */
3378                        count += 2;
3379                        break;
3380                }
3381                current = node_next(current);
3382            }
3383            lmt_linebreak_state.has_toddlers = found ? 1 : -1;
3384        }
3385        if (found > 0) {
3386            halfword current = node_next(properties->parinit_left_skip);
3387            halfword head = null;
3388            halfword tail = null;
3389            halfword count = 0;
3390            halfword multiples = 0;
3391            halfword duplex = tex_has_specification_option(properties->toddler_penalties, specification_option_double);
3392            int trace = properties->tracing_paragraphs > 1 || properties->tracing_toddlers;
3393            if (trace) {
3394                tex_begin_diagnostic();
3395                tex_print_format("[linebreak: %i toddlers found]\n", found);
3396                tex_end_diagnostic();
3397            }
3398            while (current) {
3399                if (node_type(current) == glyph_node) {
3400                    if (tex_has_glyph_option(current, glyph_option_is_toddler)) {
3401                        if (tex_aux_prev_penalty_found(current) || tex_aux_next_penalty_found(current)) {
3402                            /*tex
3403                                We need to avoid interference with e.g. math penalties, so
3404                                have to quit when we run into some bounding penalty, which can
3405                                be separated by some glue.
3406                            */
3407                            count = 0;
3408                        } else {
3409                            halfword amount = 0;
3410                            halfword tnuoma = 0;
3411                            if (duplex) {
3412                                amount = tex_get_specification_nepalty(properties->toddler_penalties, 1);
3413                                tnuoma = tex_get_specification_penalty(properties->toddler_penalties, 1);
3414                            } else {
3415                                amount = tex_get_specification_penalty(properties->toddler_penalties, 1);
3416                            }
3417                            tex_aux_inject_toddler_penalty(properties, current, amount, tnuoma, toddlered, duplex);
3418                            if (! count) {
3419                                head = current;
3420                            }
3421                            tail = current;
3422                            count++;
3423                        }
3424                    } else if (count > 1) {
3425                        tex_aux_fix_toddler_penalties(properties, duplex, head, tail, trace, count);
3426                        multiples++;
3427                        count = 0;
3428                    } else {
3429                        count = 0;
3430                    }
3431                }
3432                current = node_next(current);
3433            }
3434            if (count > 1) {
3435                tex_aux_fix_toddler_penalties(properties, duplex, head, tail, trace, count);
3436                multiples++;
3437            }
3438            if (trace) {
3439                tex_begin_diagnostic();
3440                tex_print_format("[linebreak: %i multiple toddlers found]\n", multiples);
3441                tex_end_diagnostic();
3442            }
3443        }
3444    }
3445}
3446
3447static void tex_aux_check_competing_penalties(const line_break_properties *properties, halfword current, halfword penalty, int orphaned)
3448{
3449    if (penalty > penalty_amount(current)) {
3450        if (properties->tracing_paragraphs > 1 || properties->tracing_orphans) {
3451            tex_begin_diagnostic();
3452            tex_print_format("[linebreak: orphanpenalty %i wins over toddlerpenalty %i]\n", penalty, penalty_amount(current));
3453            tex_end_diagnostic();
3454        }
3455        penalty_amount(current) = penalty;
3456    } else {
3457        if (properties->tracing_paragraphs > 1 || properties->tracing_toddlers) {
3458            tex_begin_diagnostic();
3459            tex_print_format("[linebreak: toddlerpenalty %i wins over orphanpenalty %i]\n", penalty_amount(current), penalty);
3460            tex_end_diagnostic();
3461        }
3462    }
3463    if (orphaned && tex_has_penalty_option(current, penalty_option_toddlered)) {
3464        tex_add_penalty_option(current, penalty_option_orphaned);
3465    }
3466}
3467
3468static void tex_aux_set_orphan_penalties(const line_break_properties *properties, int orphaned)
3469{
3470    if (properties->orphan_penalties || short_inline_orphan_penalty_par) {
3471        halfword current = node_prev(properties->parfill_right_skip);
3472        if (current) {
3473            int n = properties->orphan_penalties ? specification_count(properties->orphan_penalties) : 0;
3474            /*tex Skip over trailing glue and penalties. */
3475            while (current) {
3476                switch (node_type(current)) {
3477                    case glue_node:
3478                    case penalty_node:
3479                        current = node_prev(current);
3480                        break;
3481                    default:
3482                        goto INJECT;
3483                }
3484            }
3485          INJECT:
3486            if (n) {
3487                /*tex
3488                    Inject specified penalties before spaces. When we see a math node with a penalty
3489                    set then we take the max and jump over a (preceding) skip. Maybe at some point
3490                    the |short_inline_orphan_penalty_par| value will also move into the par state.
3491                */
3492                int skip = 0;
3493                halfword i = 0;
3494                while (current) {
3495                    switch (node_type(current)) {
3496                        case glue_node:
3497                            switch (node_subtype(current)) {
3498                                case space_skip_glue:
3499                                case xspace_skip_glue:
3500                                case zero_space_skip_glue:
3501                                    if (skip) {
3502                                        skip = 0;
3503                                    } else {
3504                                        current = tex_aux_inject_orphan_penalty(properties, current, tex_get_specification_penalty(properties->orphan_penalties, ++i), 0);
3505                                    }
3506                                    if (i == n) {
3507                                        return;
3508                                    } else {
3509                                        break;
3510                                    }
3511                            }
3512                            break;
3513                        case math_node:
3514                            current = tex_aux_backtrack_over_math(current);
3515                            if (tex_aux_short_math(current)) {
3516                                halfword penalty = tex_get_specification_penalty(properties->orphan_penalties, ++i);
3517                                tex_aux_adapt_short_math_penalty(current, short_inline_orphan_penalty_par, penalty, 0);
3518                                if (i == n) {
3519                                    return;
3520                                } else {
3521                                    skip = 1;
3522                                }
3523                            } else {
3524                                return;
3525                            }
3526                            break;
3527                        case disc_node:
3528                            skip = 0;
3529                            if (i < n) {
3530                                disc_orphaned(current) = tex_get_specification_penalty(properties->orphan_penalties, i + 1);
3531                                if (orphaned) {
3532                                    tex_add_disc_option(current, disc_option_orphaned);
3533                                }
3534                            }
3535                            break;
3536                        case penalty_node:
3537                            if (node_subtype(current) == toddler_penalty_subtype) {
3538                                halfword penalty = tex_get_specification_penalty(properties->orphan_penalties, ++i);
3539                                tex_aux_check_competing_penalties(properties, current, penalty, orphaned);
3540                            }
3541                            break;
3542                        default:
3543                            skip = 0;
3544                            break;
3545                    }
3546                    current = node_prev(current);
3547                }
3548            } else if (short_inline_orphan_penalty_par) {
3549                /*tex
3550                    Short formulas at the end of a line are normally not followed by something other
3551                    than punctuation. In practice one can set this penalty to e.g. a relatively low
3552                    200 to get the desired effect. We definitely quit on a space and take for granted
3553                    what comes before we see the formula.
3554                */
3555                while (current) {
3556                    switch (node_type(current)) {
3557                        case glue_node:
3558                            switch (node_subtype(current)) {
3559                                case space_skip_glue:
3560                                case xspace_skip_glue:
3561                                case zero_space_skip_glue:
3562                                    return;
3563                            }
3564                            break;
3565                        case math_node:
3566                            current = tex_aux_backtrack_over_math(current);
3567                            if (tex_aux_short_math(current)) {
3568                                tex_aux_adapt_short_math_penalty(current, short_inline_orphan_penalty_par, 0, 0);
3569                            }
3570                            return;
3571
3572                    }
3573                    current = node_prev(current);
3574                }
3575            }
3576        }
3577    }
3578}
3579
3580static inline int tex_aux_has_expansion(void) /* could be part of this identify pass over the list */
3581{
3582    if (lmt_linebreak_state.checked_expansion == -1) {
3583        halfword current = node_next(temp_head);
3584        while (current) {
3585            if (node_type(current) == glyph_node && has_font_text_control(glyph_font(current), text_control_expansion)) {
3586                lmt_linebreak_state.checked_expansion = 1;
3587                break;
3588            } else {
3589                current = node_next(current);
3590            }
3591        }
3592        lmt_linebreak_state.checked_expansion = 0;
3593    }
3594    return lmt_linebreak_state.checked_expansion;
3595}
3596
3597static inline void tex_aux_set_initial_active(const line_break_properties *properties)
3598{
3599    halfword initial = tex_new_node(unhyphenated_node, (quarterword) tex_get_specification_decent(properties->fitness_classes) - 1);
3600    node_next(initial) = active_head;
3601    active_break_node(initial) = null;
3602    active_line_number(initial) = cur_list.prev_graf + 1;
3603    active_total_demerits(initial) = 0; // default
3604    active_short(initial) = 0;          // default
3605    active_glue(initial) = 0;           // default
3606    active_line_width(initial) = 0;     // default
3607    node_next(active_head) = initial;
3608}
3609
3610static inline void tex_aux_set_local_par_state(halfword current)
3611{
3612    if (current && node_type(current) == par_node) {
3613        switch (node_subtype(current)) {
3614            case vmode_par_par_subtype:
3615            case hmode_par_par_subtype:
3616                /*tex We'd better be at the start here. */
3617                node_prev(current) = temp_head;
3618            break;
3619        }
3620        /* */
3621        lmt_linebreak_state.internal_interline_penalty = tex_get_local_interline_penalty(current);
3622        lmt_linebreak_state.internal_broken_penalty = tex_get_local_broken_penalty(current);
3623        lmt_linebreak_state.internal_par_node = current;
3624        /* */
3625        lmt_linebreak_state.internal_left_box_init = par_box_left(current);
3626        lmt_linebreak_state.internal_left_box_width_init = tex_get_local_left_width(current);
3627        lmt_linebreak_state.internal_right_box = par_box_right(current);
3628        lmt_linebreak_state.internal_right_box_width = tex_get_local_right_width(current);
3629        lmt_linebreak_state.internal_middle_box = par_box_middle(current);
3630    } else {
3631        lmt_linebreak_state.internal_interline_penalty = 0;
3632        lmt_linebreak_state.internal_broken_penalty = 0;
3633        lmt_linebreak_state.internal_par_node = null;
3634        lmt_linebreak_state.internal_left_box_init = null;
3635        lmt_linebreak_state.internal_left_box_width_init = 0;
3636        lmt_linebreak_state.internal_right_box = null;
3637        lmt_linebreak_state.internal_right_box_width = 0;
3638        lmt_linebreak_state.internal_middle_box = null;
3639    }
3640    lmt_linebreak_state.internal_left_box = lmt_linebreak_state.internal_left_box_init;
3641    lmt_linebreak_state.internal_left_box_width = lmt_linebreak_state.internal_left_box_width_init;
3642}
3643
3644static inline void tex_aux_set_adjust_spacing(line_break_properties *properties)
3645{
3646    if (properties->adjust_spacing) {
3647        lmt_linebreak_state.adjust_spacing = properties->adjust_spacing;
3648        if (properties->adjust_spacing_step > 0) {
3649            lmt_linebreak_state.adjust_spacing_step = properties->adjust_spacing_step;
3650            lmt_linebreak_state.adjust_spacing_shrink = properties->adjust_spacing_shrink;
3651            lmt_linebreak_state.adjust_spacing_stretch = properties->adjust_spacing_stretch;
3652        } else {
3653            lmt_linebreak_state.adjust_spacing_step = 0;
3654            lmt_linebreak_state.adjust_spacing_shrink = 0;
3655            lmt_linebreak_state.adjust_spacing_stretch = 0;
3656        }
3657        properties->adjust_spacing = tex_checked_font_adjust(
3658            properties->adjust_spacing,
3659            lmt_linebreak_state.adjust_spacing_step,
3660            lmt_linebreak_state.adjust_spacing_shrink,
3661            lmt_linebreak_state.adjust_spacing_stretch
3662        );
3663    } else {
3664        lmt_linebreak_state.adjust_spacing_step = 0;
3665        lmt_linebreak_state.adjust_spacing_shrink = 0;
3666        lmt_linebreak_state.adjust_spacing_stretch = 0;
3667        properties->adjust_spacing = adjust_spacing_off;
3668    }
3669    lmt_linebreak_state.current_font_step = -1;
3670}
3671
3672static inline void tex_aux_set_looseness(const line_break_properties *properties)
3673{
3674    lmt_linebreak_state.actual_looseness = 0;
3675    if (properties->looseness == 0) {
3676        lmt_linebreak_state.easy_line = lmt_linebreak_state.last_special_line;
3677    } else {
3678        lmt_linebreak_state.easy_line = max_halfword;
3679    }
3680}
3681
3682static inline void tex_aux_set_adjacent_demerits(line_break_properties *properties)
3683{
3684    if (properties->adjacent_demerits) {
3685        properties->adj_demerits = 0;
3686        properties->max_adj_demerits = specification_adjacent_max(properties->adjacent_demerits);
3687        if (! properties->max_adj_demerits) {
3688            /*tex Otherwise we loose solutions. */
3689            properties->max_adj_demerits = emergency_adj_demerits;
3690        }
3691    } else {
3692        properties->max_adj_demerits = properties->adj_demerits;
3693    }
3694}
3695
3696static int tex_aux_set_sub_pass_parameters(
3697    line_break_properties *properties,
3698    halfword               passes,
3699    int                    subpass,
3700    halfword               first,
3701    int                    details,
3702    halfword               features,
3703    halfword               overfull,
3704    halfword               underfull,
3705    halfword               verdict,
3706    halfword               classified,
3707    halfword               threshold,
3708    halfword               demerits,
3709    halfword               classes
3710) {
3711    int success = 0;
3712    uint64_t okay = tex_get_passes_okay(passes, subpass);
3713    /*tex
3714        One of the more important properties:
3715    */
3716    if (okay & passes_tolerance_okay) {
3717        properties->tolerance = tex_get_passes_tolerance(passes, subpass);
3718    }
3719    lmt_linebreak_state.threshold = properties->tolerance;
3720    lmt_linebreak_state.global_threshold = lmt_linebreak_state.threshold;
3721    /*tex
3722        The basics: tolerance, hyphenation and emergencystretch.
3723    */
3724    if (okay & passes_basics_okay) {
3725        if (okay & passes_hyphenation_okay) {
3726            lmt_linebreak_state.force_check_hyphenation = tex_get_passes_hyphenation(passes, subpass) > 0 ? 1 : 0;
3727        }
3728        if (okay & passes_emergencyfactor_okay) {
3729            lmt_linebreak_state.emergency_factor = tex_get_passes_emergencyfactor(passes, subpass);
3730        }
3731        if (okay & passes_emergencypercentage_okay) {
3732            lmt_linebreak_state.emergency_percentage = tex_get_passes_emergencypercentage(passes, subpass);
3733        }
3734        if (okay & passes_emergencywidthextra_okay) {
3735            lmt_linebreak_state.emergency_width_extra = tex_get_passes_emergencywidthextra(passes, subpass);
3736        }
3737        if (okay & passes_emergencyleftextra_okay) {
3738            lmt_linebreak_state.emergency_left_extra = tex_get_passes_emergencyleftextra(passes, subpass);
3739        }
3740        if (okay & passes_emergencyleftextra_okay) {
3741            lmt_linebreak_state.emergency_right_extra = tex_get_passes_emergencyrightextra(passes, subpass);
3742        }
3743    }
3744    /*tex
3745        We also need to handle the current document emergency stretch.
3746    */
3747    if (okay & passes_emergencystretch_okay) {
3748        halfword v = tex_get_passes_emergencystretch(passes, subpass);
3749        if (v) {
3750            properties->emergency_stretch = v;
3751            properties->emergency_original = v; /* ! */
3752        } else {
3753            properties->emergency_stretch = properties->emergency_original;
3754        }
3755    } else {
3756        properties->emergency_stretch = properties->emergency_original;
3757    }
3758    if (lmt_linebreak_state.emergency_factor) {
3759        properties->emergency_stretch = tex_xn_over_d(properties->emergency_original, lmt_linebreak_state.emergency_factor, scaling_factor);
3760    } else {
3761        properties->emergency_stretch = 0;
3762    }
3763    lmt_linebreak_state.background[total_stretch_amount] -= lmt_linebreak_state.extra_background_stretch;
3764    lmt_linebreak_state.extra_background_stretch = properties->emergency_stretch;
3765    lmt_linebreak_state.background[total_stretch_amount] += properties->emergency_stretch;
3766    /*tex
3767        The additional settings, they are seldom set so we try to avoid this branch:
3768    */
3769    if (okay & passes_additional_okay) {
3770        if (okay & passes_linepenalty_okay) {
3771            properties->line_penalty = tex_get_passes_linepenalty(passes, subpass);
3772        }
3773        if (okay & passes_lefttwindemerits_okay) {
3774            properties->left_twin_demerits = tex_get_passes_lefttwindemerits(passes, subpass);
3775        }
3776        if (okay & passes_righttwindemerits_okay) {
3777            properties->right_twin_demerits = tex_get_passes_righttwindemerits(passes, subpass);
3778        }
3779        if (okay & passes_extrahyphenpenalty_okay) {
3780            properties->extra_hyphen_penalty = tex_get_passes_extrahyphenpenalty(passes, subpass);
3781        }
3782        if (okay & passes_doublehyphendemerits_okay) {
3783            properties->double_hyphen_demerits = tex_get_passes_doublehyphendemerits(passes, subpass);
3784        }
3785        if (okay & passes_finalhyphendemerits_okay) {
3786            properties->final_hyphen_demerits = tex_get_passes_finalhyphendemerits(passes, subpass);
3787        }
3788        if (okay & passes_adjdemerits_okay) {
3789            properties->adj_demerits = tex_get_passes_adjdemerits(passes, subpass);
3790            properties->adjacent_demerits = tex_get_passes_adjacentdemerits(passes, subpass);
3791            tex_aux_set_adjacent_demerits(properties);
3792        }
3793        if (okay & passes_orphanlinefactors_okay) {
3794            properties->orphan_line_factors = tex_get_passes_orphanlinefactors(passes, subpass);
3795        }
3796        if (okay & passes_orphanpenalties_okay) {
3797            properties->orphan_penalties = tex_get_passes_orphanpenalties(passes, subpass);
3798        }
3799        if (okay & passes_toddlerpenalties_okay) {
3800            properties->toddler_penalties = tex_get_passes_toddlerpenalties(passes, subpass);
3801        }
3802        if (okay & passes_fitnessclasses_okay) { /* currenty also syncs with adj */
3803            if (tex_get_passes_fitnessclasses(passes, subpass)) { /* for now */
3804                properties->fitness_classes = tex_get_passes_fitnessclasses(passes, subpass);
3805            }
3806        }
3807        if (okay & passes_linebreakchecks_okay) {
3808            properties->line_break_checks = tex_get_passes_linebreakchecks(passes, subpass);
3809        }
3810        if (okay & passes_linebreakoptional_okay) {
3811            properties->line_break_optional = tex_get_passes_linebreakoptional(passes, subpass);
3812        }
3813    }
3814    /*tex
3815        This one is kind of special (the values are checked in maincontrol):
3816    */
3817    if (okay & passes_mathpenaltyfactor_okay) {
3818        properties->math_penalty_factor = tex_get_passes_mathpenaltyfactor(passes, subpass);
3819    }
3820    /*tex
3821        Kind of neat:
3822    */
3823    if (okay & passes_looseness_okay) {
3824        properties->looseness = tex_get_passes_looseness(passes, subpass);
3825        tex_aux_set_looseness(properties);
3826    }
3827    /*tex
3828        Kind of experimental:
3829    */
3830    if (okay & passes_sffactor_okay) {
3831        properties->sf_factor = tex_get_passes_sffactor(passes, subpass);
3832    }
3833    if (okay & passes_sfstretchfactor_okay) {
3834        properties->sf_stretch_factor = tex_get_passes_sfstretchfactor(passes, subpass);
3835    }
3836    /*tex
3837        Expansion (aka hz):
3838    */
3839    if (okay & passes_expansion_okay) {
3840        if (okay & passes_adjustspacingstep_okay) {
3841            properties->adjust_spacing_step = tex_get_passes_adjustspacingstep(passes, subpass);
3842        }
3843        if (okay & passes_adjustspacingshrink_okay) {
3844            properties->adjust_spacing_shrink = tex_get_passes_adjustspacingshrink(passes, subpass);
3845        }
3846        if (okay & passes_adjustspacingstretch_okay) {
3847            properties->adjust_spacing_stretch = tex_get_passes_adjustspacingstretch(passes, subpass);
3848        }
3849        if (okay & passes_adjustspacing_okay) {
3850            properties->adjust_spacing = tex_get_passes_adjustspacing(passes, subpass);
3851        }
3852    }
3853    tex_aux_set_adjust_spacing(properties);
3854    /*tex
3855        Callbacks: 0=quit, 1=once, 2=repeat
3856    */
3857    if (okay & passes_callback_okay) {
3858        halfword callback = tex_get_passes_callback(passes, subpass);
3859        halfword id = passes_identifier(passes);
3860        int repeat = 0;
3861        success = lmt_par_pass_callback(
3862            first,
3863            properties, id, subpass, callback, features,
3864            overfull, underfull, verdict, classified,
3865            threshold, demerits, classes, &repeat
3866        );
3867        if (repeat) {
3868            subpass -= 1;
3869        }
3870    } else {
3871        success = 1;
3872    }
3873    /*tex
3874        We need to handle some special penalties:
3875    */
3876    tex_aux_remove_special_penalties(properties);
3877    /* todo : set plural in par pass */
3878    if ((okay & passes_orphanpenalty_okay) || (okay & passes_orphanpenalties_okay)) {
3879        tex_aux_set_orphan_penalties(properties, 1);
3880    }
3881    if (okay & passes_toddlerpenalties_okay) {
3882        tex_aux_set_toddler_penalties(properties, 1);
3883    }
3884    /* */
3885    if (details) {
3886
3887        # define is_okay(a) ((okay & a) == a ? ">" : " ")
3888
3889        tex_begin_diagnostic();
3890        tex_print_format("[linebreak: values used in subpass %i]\n", subpass);
3891        tex_print_str("  --------------------------------\n");
3892        tex_print_format("  use criteria          %s\n", subpass >= passes_first_final(passes) ? "true" : "false");
3893        if (features & passes_test_set) {
3894            tex_print_str("  --------------------------------\n");
3895            if (features & passes_if_text)              { tex_print_str("  if text              true\n"); }
3896            if (features & passes_if_math)              { tex_print_str("  if math              true\n"); }
3897            if (features & passes_if_glue)              { tex_print_str("  if glue              true\n"); }
3898            if (features & passes_if_space_factor)      { tex_print_str("  if space factor      true\n"); }
3899            if (features & passes_if_adjust_spacing)    { tex_print_str("  if adjust spacing    true\n"); }
3900            if (features & passes_if_emergency_stretch) { tex_print_str("  if emergency stretch true\n"); }
3901            if (features & passes_if_looseness)         { tex_print_str("  if looseness         true\n"); }
3902            if (features & passes_unless_math)          { tex_print_str("  unless math          true\n"); }
3903        }
3904        tex_print_str("  --------------------------------\n");
3905        tex_print_format("%s threshold            %p\n", is_okay(passes_threshold_okay), tex_get_passes_threshold(passes, subpass));
3906        tex_print_format("%s demerits             %i\n", is_okay(passes_demerits_okay), tex_get_passes_demerits(passes, subpass));
3907        tex_print_format("%s classes              %X\n", is_okay(passes_classes_okay), tex_get_passes_classes(passes, subpass));
3908        tex_print_str("  --------------------------------\n");
3909        tex_print_format("%s tolerance            %i\n", is_okay(passes_tolerance_okay), properties->tolerance);
3910        tex_print_format("%s hyphenation          %s\n", is_okay(passes_hyphenation_okay), lmt_linebreak_state.force_check_hyphenation ? "true": "false");
3911        tex_print_format("%s looseness            %i\n", is_okay(passes_looseness_okay), properties->looseness);
3912        tex_print_str("  --------------------------------\n");
3913        tex_print_format("%s adjdemerits          %i\n", is_okay(passes_adjdemerits_okay), properties->adj_demerits);
3914        tex_print_format("%s adjacentdemerits     %i",   is_okay(passes_adjdemerits_okay), tex_get_specification_count(properties->adjacent_demerits));
3915        if (tex_get_specification_count(properties->adjacent_demerits) > 0) {
3916            if (specification_size(properties->adjacent_demerits)) {
3917                for (halfword c = 1; c <= tex_get_specification_count(properties->adjacent_demerits); c++) {
3918                    tex_print_format(" [%i %i]",
3919                        tex_get_specification_adjacent_u(properties->adjacent_demerits, c),
3920                        tex_get_specification_adjacent_d(properties->adjacent_demerits, c)
3921                    );
3922                }
3923            } else {
3924                tex_print_format(" [0 0] [%i %i]",
3925                    specification_adjacent_adj(properties->adjacent_demerits),
3926                    specification_adjacent_adj(properties->adjacent_demerits)
3927                );
3928            }
3929            tex_print_format(" max: %i",
3930                specification_adjacent_max(properties->adjacent_demerits)
3931            );
3932        }
3933        tex_print_str("\n");
3934        tex_print_format("%s fitnessclasses       %i",   is_okay(passes_fitnessclasses_okay), tex_get_specification_count(properties->fitness_classes));
3935        if (tex_get_specification_count(properties->fitness_classes) > 0) {
3936            tex_print_str(" [");
3937            for (halfword c = 1; c <= tex_get_specification_count(properties->fitness_classes); c++) {
3938                tex_print_format(" %i",
3939                    tex_get_specification_fitness_class(properties->fitness_classes, c)
3940                );
3941            }
3942            tex_print_str(" ]");
3943        }
3944        tex_print_str("  --------------------------------\n");
3945        tex_print_format("%s emergencyoriginal    %p\n", is_okay(passes_emergencystretch_okay), properties->emergency_original);
3946        tex_print_format("%s emergencystretch     %p\n", is_okay(passes_emergencystretch_okay), properties->emergency_stretch);
3947        tex_print_format("%s emergencyfactor      %i\n", is_okay(passes_emergencyfactor_okay), tex_get_passes_emergencyfactor(passes, subpass));
3948        tex_print_format("%s emergencypercentage  %i\n", is_okay(passes_emergencypercentage_okay), lmt_linebreak_state.emergency_percentage);
3949        tex_print_format("%s emergencyleftextra   %i\n", is_okay(passes_emergencyleftextra_okay), lmt_linebreak_state.emergency_left_extra);
3950        tex_print_format("%s emergencyrightextra  %i\n", is_okay(passes_emergencyrightextra_okay), lmt_linebreak_state.emergency_right_extra);
3951        tex_print_str("  --------------------------------\n");
3952        tex_print_format("%s mathpenaltyfactor    %i\n", is_okay(passes_mathpenaltyfactor_okay),    properties->math_penalty_factor);
3953        tex_print_str("  --------------------------------\n");
3954        tex_print_format("%s sffactor             %i\n", is_okay(passes_sffactor_okay), properties->sf_factor);
3955        tex_print_format("%s sfstretchfactor      %i\n", is_okay(passes_sfstretchfactor_okay), properties->sf_stretch_factor);
3956        tex_print_str("  --------------------------------\n");
3957        tex_print_format("%s adjustspacingstep    %i\n", is_okay(passes_adjustspacingstep_okay), properties->adjust_spacing_step);
3958        tex_print_format("%s adjustspacingshrink  %i\n", is_okay(passes_adjustspacingshrink_okay), properties->adjust_spacing_shrink);
3959        tex_print_format("%s adjustspacingstretch %i\n", is_okay(passes_adjustspacingstretch_okay), properties->adjust_spacing_stretch);
3960        tex_print_str("  --------------------------------\n");
3961        tex_print_format("%s doublehyphendemerits %i\n", is_okay(passes_doublehyphendemerits_okay), properties->double_hyphen_demerits);
3962        tex_print_format("%s finalhyphendemerits  %i\n", is_okay(passes_finalhyphendemerits_okay), properties->final_hyphen_demerits);
3963        tex_print_format("%s lefttwindemerits     %i\n", is_okay(passes_lefttwindemerits_okay), properties->left_twin_demerits);
3964        tex_print_format("%s righttwindemerits    %i\n", is_okay(passes_righttwindemerits_okay), properties->right_twin_demerits);
3965        tex_print_str("  --------------------------------\n");
3966        tex_print_format("%s linepenalty          %i\n", is_okay(passes_linepenalty_okay), properties->line_penalty);
3967        tex_print_format("%s extrahyphenpenalty   %i\n", is_okay(passes_extrahyphenpenalty_okay),properties->extra_hyphen_penalty);
3968        tex_print_str("  --------------------------------\n");
3969        tex_print_format("%s toddlerpenalties     %i", is_okay(passes_toddlerpenalties_okay), tex_get_specification_count(properties->toddler_penalties));
3970        if (tex_get_specification_count(properties->toddler_penalties) > 0) {
3971            tex_print_str(" [");
3972            for (halfword c = 1; c <= tex_get_specification_count(properties->toddler_penalties); c++) {
3973                tex_print_format(" %i",
3974                    tex_get_specification_penalty(properties->toddler_penalties, c)
3975                );
3976            }
3977            tex_print_str(" ]");
3978        }
3979        tex_print_str("\n");
3980        tex_print_str("  --------------------------------\n");
3981        tex_print_format("%s orphanpenalties      %i", is_okay(passes_orphanpenalties_okay), tex_get_specification_count(properties->orphan_penalties));
3982        if (tex_get_specification_count(properties->orphan_penalties) > 0) {
3983            tex_print_str(" [");
3984            for (halfword c = 1; c <= tex_get_specification_count(properties->orphan_penalties); c++) {
3985                tex_print_format(" %i",
3986                    tex_get_specification_penalty(properties->orphan_penalties, c)
3987                );
3988            }
3989            tex_print_str(" ]");
3990        }
3991        tex_print_str("\n");
3992        tex_print_format("%s orphanlinefactors    %i",   is_okay(passes_orphanlinefactors_okay), tex_get_specification_count(properties->orphan_line_factors));
3993        if (tex_get_specification_count(properties->orphan_line_factors) > 0) {
3994            tex_print_str(" [");
3995            for (halfword c = 1; c <= tex_get_specification_count(properties->orphan_line_factors); c++) {
3996                tex_print_format(" %i",
3997                    tex_get_specification_penalty(properties->orphan_line_factors, c)
3998                );
3999            }
4000            tex_print_str(" ]");
4001        }
4002        tex_print_str("\n");
4003        tex_print_str("  --------------------------------\n");
4004        tex_print_format("%s linebreakchecks      %i\n", is_okay(passes_linebreakchecks_okay), properties->line_break_checks);
4005        tex_print_format("%s linebreakoptional    %i\n", is_okay(passes_linebreakoptional_okay), properties->line_break_optional);
4006        tex_print_str("  --------------------------------\n");
4007        tex_end_diagnostic();
4008    }
4009    return success;
4010}
4011
4012static void tex_aux_skip_message(halfword passes, int subpass, int nofsubpasses, const char *str)
4013{
4014    tex_begin_diagnostic();
4015    tex_print_format("[linebreak: id %i, subpass %i of %i, skip %s]\n",
4016        passes_identifier(passes), subpass, nofsubpasses, str
4017    );
4018    tex_end_diagnostic();
4019}
4020
4021static inline int tex_aux_next_subpass(const line_break_properties *properties, halfword passes, int subpass, int nofsubpasses, halfword state, int tracing)
4022{
4023    while (++subpass <= nofsubpasses) {
4024        halfword features = tex_get_passes_features(passes, subpass);
4025        if (features & passes_test_set) {
4026            if (features & passes_if_text) {
4027                if (! paragraph_has_text(state)) {
4028                    if (tracing) {
4029                        tex_aux_skip_message(passes, subpass, nofsubpasses, "no text");
4030                    }
4031                    continue;
4032                }
4033            }
4034            if (features & passes_if_math) {
4035                if (! paragraph_has_math(state)) {
4036                    if (tracing) {
4037                        tex_aux_skip_message(passes, subpass, nofsubpasses, "no math");
4038                    }
4039                    continue;
4040                }
4041            }
4042            if (features & passes_unless_math) {
4043                if (paragraph_has_math(state)) {
4044                    if (tracing) {
4045                        tex_aux_skip_message(passes, subpass, nofsubpasses, "do math");
4046                    }
4047                    continue;
4048                }
4049            }
4050            if (features & passes_if_glue) {
4051                if (! paragraph_has_glue(state)) {
4052                    if (tracing) {
4053                        tex_aux_skip_message(passes, subpass, nofsubpasses, "no glue");
4054                    }
4055                    continue;
4056                }
4057            }
4058            if (features & passes_if_space_factor) {
4059                if (! paragraph_has_factor(state)) {
4060                    if (tracing) {
4061                        tex_aux_skip_message(passes, subpass, nofsubpasses, "no space factor");
4062                    }
4063                    continue;
4064                }
4065            }
4066            if (features & passes_if_adjust_spacing && tex_aux_has_expansion()) {
4067                if (! paragraph_has_text(state) || ! tex_get_passes_adjustspacing(passes, subpass)) {
4068                    if (tracing) {
4069                        tex_aux_skip_message(passes, subpass, nofsubpasses, "adjust spacing");
4070                    }
4071                    continue;
4072                }
4073            }
4074            if (features & passes_if_emergency_stretch) {
4075                if (! ( (properties->emergency_original || tex_get_passes_emergencystretch(passes, subpass)) && tex_get_passes_emergencyfactor(passes, subpass) ) ) {
4076                    if (tracing) {
4077                        tex_aux_skip_message(passes, subpass, nofsubpasses, "emergency stretch");
4078                    }
4079                    continue;
4080                }
4081            }
4082            if (features & passes_if_looseness) {
4083                if (! properties->looseness) {
4084                    if (tracing) {
4085                        tex_aux_skip_message(passes, subpass, nofsubpasses, "no looseness");
4086                    }
4087                    continue;
4088                }
4089            }
4090        }
4091        return subpass;
4092    }
4093    return nofsubpasses + 1;
4094}
4095
4096static inline int tex_aux_check_sub_pass(line_break_properties *properties, halfword state, scaled shortfall, halfword passes, int subpass, int nofsubpasses, halfword first)
4097{
4098    scaled overfull = 0;
4099    scaled underfull = 0;
4100    halfword verdict = 0;
4101    halfword classified = 0;
4102    int tracing = properties->tracing_paragraphs > 0 || properties->tracing_passes > 0;
4103    int result = tex_check_linebreak_quality(shortfall, &overfull, &underfull, &verdict, &classified);
4104    if (result) {
4105        if (tracing && result > 1) {
4106            tex_begin_diagnostic();
4107            tex_print_format("[linebreak: id %i, subpass %i of %i, overfull %p, verdict %i, special case, entering subpasses]\n",
4108                passes_identifier(passes), subpass, nofsubpasses, overfull, verdict
4109            );
4110            tex_end_diagnostic();
4111        }
4112        while (subpass < nofsubpasses) {
4113            subpass = tex_aux_next_subpass(properties, passes, subpass, nofsubpasses, state, tracing);
4114            if (subpass > nofsubpasses) {
4115                return subpass;
4116            } else {
4117                halfword features = tex_get_passes_features(passes, subpass);
4118                if (features & passes_quit_pass) {
4119                    return -1;
4120                } else if (features & passes_skip_pass) {
4121                    continue;
4122                } else {
4123                    scaled threshold = tex_get_passes_threshold(passes, subpass);
4124                    halfword demerits = tex_get_passes_demerits(passes, subpass);
4125                    halfword classes = tex_get_passes_classes(passes, subpass);
4126                    int callback = features & passes_callback_set;
4127                    int success = 0;
4128                    int details = properties->tracing_passes > 1;
4129                    int retry = callback ? 1 : overfull > threshold || verdict > demerits || (classes && (classes & classified) != 0);
4130                    if (tracing) {
4131                        int id = passes_identifier(passes);
4132                        tex_begin_diagnostic();
4133                        if (callback) {
4134                            tex_print_format("[linebreak: id %i, subpass %i of %i, overfull %p, underfull %p, verdict %i, classified %x, %s]\n",
4135                                id, subpass, nofsubpasses, overfull, underfull, verdict, classified, "callback"
4136                            );
4137                        } else {
4138                            const char *action = retry ? "retry" : "skipped";
4139                            if (id < 0) {
4140                                id = -id; /* nicer for our purpose */
4141                            }
4142                            if (threshold == max_dimension) {
4143                                if (demerits == max_dimension) {
4144                                    tex_print_format("[linebreak: id %i, subpass %i of %i, overfull %p, underfull %p, verdict %i, classified %x, %s]\n",
4145                                        id, subpass, nofsubpasses, overfull, underfull, verdict, classified, action
4146                                    );
4147                                } else {
4148                                    tex_print_format("[linebreak: id %i, subpass %i of %i, overfull %p, underfull %p, verdict %i, demerits %i, classified %x, %s]\n",
4149                                        id, subpass, nofsubpasses, overfull, underfull, verdict, demerits, classified, action
4150                                    );
4151                                }
4152                            } else {
4153                                if (demerits == max_dimension) {
4154                                    tex_print_format("[linebreak: id %i, subpass %i of %i, overfull %p, underfull %p, verdict %i, threshold %p, classified %x, %s]\n",
4155                                        id, subpass, nofsubpasses, overfull, underfull, verdict, threshold, classified, action
4156                                    );
4157                                } else {
4158                                    tex_print_format("[linebreak: id %i, subpass %i of %i, overfull %p, underfull %p, verdict %i, threshold %p, demerits %i, classified %x, %s]\n",
4159                                        id, subpass, nofsubpasses, overfull, underfull, verdict, threshold, demerits, classified, action
4160                                    );
4161                                }
4162                            }
4163                        }
4164                    }
4165                    if (retry) {
4166                        success = tex_aux_set_sub_pass_parameters(
4167                            properties, passes, subpass, first,
4168                            details,
4169                            features, overfull, underfull, verdict, classified, threshold, demerits, classes
4170                        );
4171                    }
4172                    if (tracing) {
4173                        tex_end_diagnostic();
4174                    }
4175                    if (success) {
4176                        return subpass;
4177                    }
4178                }
4179            }
4180        }
4181    } else {
4182        /*tex We have a few hits in our test files. */
4183    }
4184    return 0;
4185}
4186
4187/*tex
4188
4189    To access the first node of paragraph as the first active node has |break_node = null|.
4190
4191    Determine legal breaks: As we move through the hlist, we need to keep the |active_width| array
4192    up to date, so that the badness of individual lines is readily calculated by |try_break|. It
4193    is convenient to use the short name |active_width [1]| for the component of active width that
4194    represents real width as opposed to glue.
4195
4196    Advance |cur_p| to the node following the present string of characters. The code that passes
4197    over the characters of words in a paragraph is part of \TEX's inner loop, so it has been
4198    streamlined for speed. We use the fact that |\parfillskip| glue appears at the end of each
4199    paragraph; it is therefore unnecessary to check if |vlink (cur_p) = null| when |cur_p| is a
4200    character node.
4201
4202    Advance |cur_p| to the node following the present string of characters. The code that passes
4203    over the characters of words in a paragraph is part of \TEX's inner loop, so it has been
4204    streamlined for speed. We use the fact that |\parfillskip| glue appears at the end of each
4205    paragraph; it is therefore unnecessary to check if |vlink (cur_p) = null| when |cur_p| is
4206    a character node. (This is no longer true because we've split the hyphenation and font
4207    processing steps.)
4208
4209    The code below is in the meantime a mix between good old \TEX, \ETEX\ (last line) , \OMEGA\
4210    (local boxes but redone), \PDFTEX\ (expansion and protrusion), \LUATEX\ and quite a bit of
4211    \LUAMETATEX. But the principles remain the same.
4212
4213*/
4214
4215static inline void tex_aux_wipe_optionals(const line_break_properties *properties, halfword current, int state)
4216{
4217    if (paragraph_has_optional(state)) {
4218        while (current) {
4219            if (node_type(current) == boundary_node && node_subtype(current) == optional_boundary) {
4220                if (properties->line_break_optional) {
4221                    if (! boundary_data(current)) {
4222                        current = node_next(current);
4223                        continue;
4224                    } else if ((boundary_data(current) & properties->line_break_optional) == properties->line_break_optional) {
4225                        current = node_next(current);
4226                        continue;
4227                    }
4228                }
4229                {
4230                    halfword first = current;
4231                    while (1) {
4232                        current = node_next(current);
4233                        if (! current) {
4234                            return;
4235                        } else if (node_type(current) == boundary_node && node_subtype(current) == optional_boundary && ! boundary_data(current) ) {
4236                            halfword prev = node_prev(first);
4237                            halfword next = node_next(current);
4238                            halfword wiped = first;
4239                            node_next(current) = null;
4240                            tex_try_couple_nodes(prev, next);
4241                            tex_flush_node_list(wiped);
4242                            current = prev;
4243                            break;
4244                        }
4245                    }
4246                }
4247            }
4248            current = node_next(current);
4249        }
4250    }
4251}
4252
4253static void tex_aux_show_threshold(const char *what, halfword value)
4254{
4255    tex_begin_diagnostic();
4256    tex_print_format("[linebreak: %s threshold %i]", what, value);
4257    tex_end_diagnostic();
4258}
4259
4260/*tex
4261    In most cases (90\percent\ or more) we have only one pass so then it makes sense to just use
4262    that pass and accept some redundant stat echecking later on.
4263*/
4264
4265/*
4266
4267static inline halfword tex_aux_analyze_list(halfword current)
4268{
4269    halfword state = 0;
4270    while (current) {
4271        switch (node_type(current)) {
4272            case glyph_node:
4273                state |= par_has_glyph;
4274                break;
4275            case glue_node:
4276                switch (node_subtype(current)) {
4277                    case space_skip_glue:
4278                    case xspace_skip_glue:
4279                    case zero_space_skip_glue:
4280                        state |= par_has_space;
4281                        break;
4282                    case u_leaders:
4283                        state |= par_has_uleader;
4284                        break;
4285                }
4286                state |= par_has_glue; // todo: only when stretch or shrink
4287                break;
4288            case disc_node:
4289                state |= par_has_disc;
4290                break;
4291            case math_node:
4292                if (! (tex_math_glue_is_zero(current) || tex_ignore_math_skip(current))) {
4293                    state |= par_has_glue; // todo: only when stretch or shrink
4294                }
4295                state |= par_has_math;
4296                break;
4297            case boundary_node:
4298                if (node_subtype(current) == optional_boundary) {
4299                    state |= par_has_optional;
4300                }
4301                break;
4302        }
4303        current = node_next(current);
4304    }
4305    return state;
4306}
4307*/
4308
4309static inline halfword tex_aux_break_list(const line_break_properties *properties, halfword pass, halfword subpass, halfword current, halfword first, halfword *state, int artificial)
4310{
4311    halfword callback_id = lmt_linebreak_state.callback_id;
4312    halfword checks = properties->line_break_checks;
4313    halfword sf_factor = properties->sf_factor;
4314    halfword sf_stretch_factor = properties->sf_stretch_factor;
4315    while (current && (node_next(active_head) != active_head)) { /* we check the cycle */
4316        switch (node_type(current)) {
4317            case glyph_node:
4318                /* why ex here and not in add/sub disc glyphs */
4319                lmt_linebreak_state.active_width[total_advance_amount] += tex_glyph_width(current);
4320                if (properties->adjust_spacing && properties->adjust_spacing_step > 0 && tex_has_glyph_expansion(current)) {
4321                    lmt_packaging_state.previous_char_ptr = current;
4322                    lmt_linebreak_state.active_width[font_stretch_amount] += tex_char_stretch(current);
4323                    lmt_linebreak_state.active_width[font_shrink_amount] += tex_char_shrink(current);
4324                }
4325                *state |= par_has_glyph;
4326                break;
4327            case hlist_node:
4328            case vlist_node:
4329                lmt_linebreak_state.active_width[total_advance_amount] += box_width(current);
4330                break;
4331            case rule_node:
4332                lmt_linebreak_state.active_width[total_advance_amount] += rule_width(current);
4333                break;
4334            case dir_node:
4335                /*tex Adjust the dir stack for the |line_break| routine. */
4336                lmt_linebreak_state.line_break_dir = tex_update_dir_state(current, properties->paragraph_dir);
4337                break;
4338            case par_node:
4339                /*tex Advance past a |par| node. */
4340                switch (node_subtype(current)) {
4341                    case vmode_par_par_subtype:
4342                    case hmode_par_par_subtype:
4343                        break;
4344                    case local_box_par_subtype:
4345                        break;
4346                    case parameter_par_subtype:
4347                        {
4348                            halfword t = pass == linebreak_first_pass ? tex_get_local_pre_tolerance(current) : tex_get_local_tolerance(current);
4349                            if (t == 0) {
4350                                t = lmt_linebreak_state.global_threshold;
4351                                if (properties->tracing_paragraphs > 1) {
4352                                    tex_aux_show_threshold("global", t);
4353                                }
4354                            } else {
4355                                if (properties->tracing_paragraphs > 1) {
4356                                    tex_aux_show_threshold("local", t);
4357                                }
4358                            }
4359                            lmt_linebreak_state.threshold = t;
4360                            lmt_linebreak_state.internal_interline_penalty = tex_get_local_interline_penalty(current);
4361                            lmt_linebreak_state.internal_broken_penalty = tex_get_local_broken_penalty(current);
4362                            lmt_linebreak_state.internal_par_node = current;
4363                            break;
4364                        }
4365                    case local_break_par_subtype:
4366                        /*tex
4367                            This is an experiment. We might at some point use more trickery with
4368                            these nodes.
4369                        */
4370                        tex_aux_try_break(properties, -100000, unhyphenated_node, first, current, callback_id, checks, pass, subpass, artificial);
4371                        break;
4372                }
4373                lmt_linebreak_state.internal_left_box = par_box_left(current);
4374                lmt_linebreak_state.internal_left_box_width = tex_get_local_left_width(current);
4375                lmt_linebreak_state.internal_right_box = par_box_right(current);
4376                lmt_linebreak_state.internal_right_box_width = tex_get_local_right_width(current);
4377                lmt_linebreak_state.internal_middle_box = par_box_middle(current);
4378                break;
4379            case glue_node:
4380                /*tex
4381
4382                    If node |cur_p| is a legal breakpoint, call |try_break|; then update the
4383                    active widths by including the glue in |glue_ptr (cur_p)|.
4384
4385                    When node |cur_p| is a glue node, we look at the previous to see whether
4386                    or not a breakpoint is legal at |cur_p|, as explained above.
4387
4388                    We only break after certain nodes (see texnodes.h), a font related kern
4389                    and a dir node when |\breakafterdirmode = 1|.
4390
4391                */
4392                if (tex_has_glue_option(current, glue_option_no_auto_break)) {
4393                    /*tex Glue in math is not a valid breakpoint, unless we permit it. */
4394                } else if (tex_is_par_init_glue(current)) {
4395                    /*tex Of course we don't break here. */
4396                } else if (tex_aux_valid_glue_break(current)) {
4397                    tex_aux_try_break(properties, tex_aux_upcoming_math_penalty(current, properties->math_penalty_factor), unhyphenated_node, first, current, callback_id, checks, pass, subpass, artificial);
4398                }
4399                if (tex_has_glue_option(current, glue_option_has_factor)) {
4400                    *state |= par_has_factor;
4401                    lmt_linebreak_state.active_width[total_advance_amount] += tex_aux_applied_amount(current, sf_factor);
4402                    lmt_linebreak_state.active_width[total_stretch_amount + glue_stretch_order(current)] += tex_aux_applied_stretch(current, sf_stretch_factor);
4403                } else {
4404                    lmt_linebreak_state.active_width[total_advance_amount] += glue_amount(current);
4405                    lmt_linebreak_state.active_width[total_stretch_amount + glue_stretch_order(current)] += glue_stretch(current);
4406                }
4407                lmt_linebreak_state.active_width[total_shrink_amount] += tex_aux_checked_shrink(current);
4408                switch (node_subtype(current)) {
4409                    case space_skip_glue:
4410                    case xspace_skip_glue:
4411                    case zero_space_skip_glue:
4412                        *state |= par_has_space;
4413                        break;
4414                    case u_leaders:
4415                        *state |= par_has_uleader;
4416                        break;
4417                }
4418                *state |= par_has_glue; /* todo: only when stretch or shrink */
4419                break;
4420            case kern_node:
4421                switch (node_subtype(current)) {
4422                    case explicit_kern_subtype:
4423                    case italic_kern_subtype:
4424                    case right_correction_kern_subtype:
4425                        {
4426                            /*tex There used to be a |! is_char_node(node_next(cur_p))| test here. */
4427                            /*tex We catch |\emph{test} $\hat{x}$| aka |test\kern5pt\hskip10pt$\hat{x}$|. */
4428                            halfword nxt = node_next(current);
4429                            if (nxt && node_type(nxt) == glue_node && ! tex_aux_upcoming_math_penalty(nxt, properties->math_penalty_factor) && ! tex_has_glue_option(nxt, glue_option_no_auto_break)) {
4430                                tex_aux_try_break(properties, 0, unhyphenated_node, first, current, callback_id, checks, pass, subpass, artificial);
4431                            }
4432                        }
4433                        break;
4434                 // case left_correction_kern_subtype:
4435                 //     break;
4436                    case font_kern_subtype:
4437                        if (properties->adjust_spacing == adjust_spacing_full) {
4438                            lmt_linebreak_state.active_width[font_stretch_amount] += tex_kern_stretch(current);
4439                            lmt_linebreak_state.active_width[font_shrink_amount] += tex_kern_shrink(current);
4440                        }
4441                        break;
4442                }
4443                lmt_linebreak_state.active_width[total_advance_amount] += kern_amount(current);
4444                break;
4445            case disc_node:
4446                /*tex
4447
4448                    Try to break after a discretionary fragment, then |goto done5|. The
4449                    following code knows that discretionary texts contain only character
4450                    nodes, kern nodes, box nodes, and rule nodes. This branch differs a bit
4451                    from older engines because in \LUATEX\ we already have hyphenated the list.
4452                    This means that we need to skip automatic disc nodes. Or better, we need
4453                    to treat discretionaries and explicit hyphens always, even in the first
4454                    pass.
4455
4456                    We used to have |init_disc| followed by |select disc| variants where the
4457                    |select_disc|s were handled by the leading |init_disc|. The question is: should
4458                    we bother about select nodes? Knuth indicates in the original source that only
4459                    a very few cases need hyphenation so the exceptional case of >2 char ligatures
4460                    having hyphenation points in between is rare. We'd better have proper compound
4461                    word handling. Keep in mind that these (old) init and select subtypes always
4462                    came in isolated pairs and that they only were meant for the simple (enforced)
4463                    hyphenation discretionaries.
4464
4465                    Therefore, this feature has been dropped from \LUAMETATEX. It not only makes
4466                    the code simpler, it also avoids having code on board for border cases that
4467                    even when dealt with are suboptimal. It's better to have nothing that something
4468                    fuzzy. It also makes dealing with (intermediate) node lists easier. If I want
4469                    something like this it should be okay for any situation.
4470
4471                */
4472                {
4473                    halfword replace = disc_no_break_head(current);
4474                    if (lmt_linebreak_state.force_check_hyphenation || (node_subtype(current) != syllable_discretionary_code)) {
4475                        halfword actual_penalty = disc_penalty(current) + disc_orphaned(current) + properties->extra_hyphen_penalty;
4476                        halfword pre = disc_pre_break_head(current);
4477                        tex_aux_reset_disc_target(properties->adjust_spacing, lmt_linebreak_state.disc_width);
4478                        if (pre) {
4479                            /*tex
4480                                After \OPENTYPE\ processing we can have long snippets in a disc node
4481                                and as a result we can have trailing disc nodes which also means that
4482                                we can have a replacement that is shorter than a pre snippet so then
4483                                we need bypass the pre checking. It also catches the case where we
4484                                have a pre and replace but no post that then can result in an empty
4485                                line when we have a final disc.
4486
4487                                Another more realistic example of required control is when we use
4488                                discretionaries for optional content, where we actually might want to
4489                                favour either the replacement or the prepost pair. We're still talking
4490                                or rare situations (that showed up in test loops that triggered these
4491                                border cases).
4492                            */
4493                            if (replace && node_subtype(current) != mathematics_discretionary_code) {
4494                                if (tex_has_disc_option(current, disc_option_prefer_break) || tex_has_disc_option(current, disc_option_prefer_nobreak)) {
4495                                    /*tex
4496                                        Maybe we need some more (subtype) checking but first I need a
4497                                        better (less border) use case. In this border case we forget
4498                                        about expansion.
4499
4500                                        We likely default to |prefer_nobreak| so by testing |prefer_break|
4501                                        first we let that one take control.
4502                                    */
4503                                    switch (node_type(node_next(current))) {
4504                                        case glue_node:
4505                                        case penalty_node:
4506                                        case boundary_node:
4507                                            {
4508                                                /*tex
4509                                                    Lets assume that when |wd == wp| we actually
4510                                                    want the pre text (and a possible empty post).
4511                                                */
4512                                                scaled wpre = tex_natural_hsize(pre, NULL);
4513                                                scaled wreplace = tex_natural_hsize(replace, NULL);
4514                                                if (tex_has_disc_option(current, disc_option_prefer_break)) {
4515                                                    halfword post = disc_post_break_head(current);
4516                                                    scaled wpost = post ? tex_natural_hsize(post, NULL) : 0;
4517                                                    if (wpost > 0) {
4518                                                        if (properties->tracing_paragraphs > 1) {
4519                                                            tex_begin_diagnostic();
4520                                                            tex_print_format("[linebreak: favour final prepost over replace, widths %p %p]", wpre + wpost, wreplace);
4521                                                            tex_print_str("%l[linebreak: stripe] ");
4522                                                            tex_short_display(node_next(temp_head));
4523                                                            tex_end_diagnostic();
4524                                                        }
4525                                                    } else {
4526                                                        goto REPLACEONLY;
4527                                                    }
4528                                                } else {
4529                                                    if (wreplace < wpre) {
4530                                                        if (properties->tracing_paragraphs > 1) {
4531                                                            tex_begin_diagnostic();
4532                                                            tex_print_format("[linebreak: favour final replace over pre, widths %p %p]", wreplace, wpre);
4533                                                            tex_print_str("%l[linebreak: stripe] ");
4534                                                            tex_short_display(node_next(temp_head));
4535                                                            tex_end_diagnostic();
4536                                                        }
4537                                                        goto REPLACEONLY;
4538                                                    }
4539                                                }
4540                                            }
4541                                    }
4542                                }
4543                            }
4544                            tex_aux_add_to_widths(pre, properties->adjust_spacing, properties->adjust_spacing_step, sf_factor, sf_stretch_factor, lmt_linebreak_state.disc_width);
4545                            tex_aux_add_disc_source_to_target(properties->adjust_spacing, lmt_linebreak_state.active_width, lmt_linebreak_state.disc_width);
4546                            tex_aux_try_break(properties, actual_penalty, hyphenated_node, first, current, callback_id, checks, pass, subpass, artificial);
4547                            tex_aux_sub_disc_target_from_source(properties->adjust_spacing, lmt_linebreak_state.active_width, lmt_linebreak_state.disc_width);
4548                        } else {
4549                            /*tex trivial pre-break */
4550                            tex_aux_try_break(properties, actual_penalty, hyphenated_node, first, current, callback_id, checks, pass, subpass, artificial);
4551                        }
4552                    }
4553                  REPLACEONLY:
4554                    if (replace) {
4555                        tex_aux_add_to_widths(replace, properties->adjust_spacing, properties->adjust_spacing_step, sf_factor, sf_stretch_factor, lmt_linebreak_state.active_width);
4556                    }
4557                    *state |= par_has_disc;
4558                    break;
4559                }
4560            case penalty_node:
4561                {
4562                    halfword penalty = penalty_amount(current);
4563                    switch (node_subtype(current)) {
4564                        /* maybe more */
4565                        case math_pre_penalty_subtype:
4566                        case math_post_penalty_subtype:
4567                            if (properties->math_penalty_factor) {
4568                                penalty = tex_xn_over_d(penalty, properties->math_penalty_factor, scaling_factor);
4569                            }
4570                            break;
4571                    }
4572                    if (properties->orphan_line_factors && node_subtype(current) == orphan_penalty_subtype) {
4573                        if (lmt_linebreak_state.current_line_number == max_halfword) {
4574                              /*tex This is kind of weird. */
4575                        } else {
4576                            int factor = tex_get_specification_penalty(properties->orphan_line_factors, lmt_linebreak_state.current_line_number);
4577                            /* todo tracing */
4578                            penalty = tex_xn_over_d(penalty, factor, scaling_factor);
4579                        }
4580                    }
4581                    tex_add_penalty_option(current, penalty_option_factor_used);
4582                    penalty_used(current) = penalty;
4583                    tex_aux_try_break(properties, penalty, unhyphenated_node, first, current, callback_id, checks, pass, subpass, artificial);
4584                    break;
4585                }
4586            case math_node:
4587                {
4588                    /*tex
4589                        There used to a ! is_char_node(node_next(cur_p)) test here but I'm
4590                        not sure what that is.
4591                    */
4592                    switch (node_subtype(current)) {
4593                        case begin_inline_math:
4594                            lmt_linebreak_state.saved_threshold = lmt_linebreak_state.threshold;
4595                            if (pass == linebreak_first_pass) {
4596                                if (math_pre_tolerance(current)) {
4597                                    lmt_linebreak_state.threshold = math_pre_tolerance(current);
4598                                }
4599                            } else {
4600                                if (math_tolerance(current)) {
4601                                    lmt_linebreak_state.threshold = math_tolerance(current);
4602                                }
4603                            }
4604                            if (properties->tracing_paragraphs > 1) {
4605                                tex_aux_show_threshold("math", lmt_linebreak_state.threshold);
4606                            }
4607                            break;
4608                        case end_inline_math:
4609                            /*tex
4610                                We also store the restore value in the end node but we can
4611                                actually change the tolerance mid paragraph so that value might
4612                                be wrong, which is why we save the old threshold and use that.
4613                            */
4614                            lmt_linebreak_state.threshold = lmt_linebreak_state.saved_threshold;
4615                            if (properties->tracing_paragraphs > 1) {
4616                                tex_aux_show_threshold("text", lmt_linebreak_state.threshold);
4617                            }
4618                            break;
4619                    }
4620                    // lmt_linebreak_state.auto_breaking = finishing;
4621                    if (tex_math_glue_is_zero(current) || tex_ignore_math_skip(current)) {
4622                        /*tex
4623                            When we end up here we assume |\mathsurround| but we only check for
4624                            a break when we're ending math. Maybe this is something we need to
4625                            open up. The math specific penalty only kicks in when we break.
4626                        */
4627                        switch (node_subtype(current)) {
4628                            case begin_inline_math:
4629                                /*tex This one is a lookahead penalty and handled in glue. */
4630                                break;
4631                            case end_inline_math:
4632                                {
4633                                    halfword penalty = math_penalty(current);
4634                                    if (node_type(node_next(current)) == glue_node) {
4635                                        if (properties->math_penalty_factor) {
4636                                            penalty = tex_xn_over_d(penalty, properties->math_penalty_factor, scaling_factor);
4637                                        }
4638                                        tex_aux_try_break(properties, penalty, unhyphenated_node, first, current, callback_id, checks, pass, subpass, artificial);
4639                                    }
4640                                }
4641                                break;
4642                        }
4643                        lmt_linebreak_state.active_width[total_advance_amount] += math_surround(current);
4644                    } else {
4645                        /*tex
4646                            This one does quite some testing, is that still needed?
4647                        */
4648                        switch (node_subtype(current)) {
4649                            case begin_inline_math:
4650                                /*tex This one is a lookahead penalty and handled in glue. */
4651                                break;
4652                            case end_inline_math:
4653                                {
4654                                    halfword penalty = math_penalty(current);
4655                                    if (tex_aux_valid_glue_break(current)) {
4656                                        if (properties->math_penalty_factor) {
4657                                            penalty = tex_xn_over_d(penalty, properties->math_penalty_factor, scaling_factor);
4658                                        }
4659                                        tex_aux_try_break(properties, penalty, unhyphenated_node, first, current, callback_id, checks, pass, subpass, artificial);
4660                                    }
4661                                }
4662                                break;
4663                        }
4664                        lmt_linebreak_state.active_width[total_advance_amount] += math_amount(current);
4665                        lmt_linebreak_state.active_width[total_stretch_amount + math_stretch_order(current)] += math_stretch(current);
4666                        lmt_linebreak_state.active_width[total_shrink_amount] += tex_aux_checked_shrink(current);
4667                        *state |= par_has_glue; /* todo: only when stretch or shrink */
4668                    }
4669                    *state |= par_has_math;
4670                    break;
4671                }
4672            case boundary_node:
4673                if (node_subtype(current) == optional_boundary) {
4674                    *state |= par_has_optional;
4675                    if (properties->line_break_optional) {
4676                        if (! boundary_data(current)) {
4677                         // printf("END PROCESS OPTIONAL\n");
4678                            break;
4679                        } else if ((boundary_data(current) & properties->line_break_optional) == properties->line_break_optional) {
4680                            /* use this optional */
4681                         // printf("BEGIN PROCESS OPTIONAL\n");
4682                            break;
4683                        }
4684                    }
4685                 // printf("BEGIN IGNORE OPTIONAL\n");
4686                    while (1) {
4687                        current = node_next(current);
4688                        if (! current) {
4689                            /* actually this is an error */
4690                            return null;
4691                        } else if (node_type(current) == boundary_node && node_subtype(current) == optional_boundary && ! boundary_data(current)) {
4692                         // printf("END IGNORE OPTIONAL\n");
4693                            break;
4694                        }
4695                    }
4696                }
4697                break;
4698            case whatsit_node:
4699            case mark_node:
4700            case insert_node:
4701            case adjust_node:
4702                /*tex Advance past these nodes in the |line_break| loop. */
4703                break;
4704            default:
4705                tex_formatted_error("parbuilder", "weird node %d in paragraph", node_type(current));
4706        }
4707        current = node_next(current);
4708    }
4709    return current;
4710}
4711
4712static void tex_aux_report_fitness_classes(const line_break_properties *properties, int pass, int subpass)
4713{
4714    tex_begin_diagnostic();
4715    tex_print_format("[linebreak: fitnessclasses, pass %i, subpass %i]\n", pass, subpass);
4716    for (halfword c = 1; c <= tex_get_specification_count(properties->fitness_classes); c++) {
4717        tex_print_format("%l  %i : %i\n", c,
4718            tex_get_specification_fitness_class(properties->fitness_classes, c)
4719        );
4720    }
4721    tex_end_diagnostic();
4722}
4723
4724static void tex_aux_report_adjacent_demerits(const line_break_properties *properties, int pass, int subpass)
4725{
4726    tex_begin_diagnostic();
4727    tex_print_format("[linebreak: adjacentdemerits, pass %i, subpass %i]\n", pass, subpass);
4728    for (halfword c = 1; c <= tex_get_specification_count(properties->adjacent_demerits); c++) {
4729        tex_print_format("%l  %i : %i %i\n", c,
4730            tex_get_specification_adjacent_u(properties->adjacent_demerits, c),
4731            tex_get_specification_adjacent_d(properties->adjacent_demerits, c)
4732        );
4733    }
4734    tex_end_diagnostic();
4735}
4736
4737static void tex_aux_fix_prev_graf(void)
4738{
4739    /*tex Fix a buglet that probably is a feature. */
4740    if ((cur_list.prev_graf > max_prev_graf || cur_list.prev_graf < 0) && normalize_par_mode_option(limit_prev_graf_mode)) {
4741        tex_formatted_warning("tex", "clipping prev_graf %i to %i", cur_list.prev_graf, max_prev_graf);
4742        cur_list.prev_graf = max_prev_graf;
4743    }
4744}
4745
4746/*tex
4747    We compute the values of |easy_line| and the other local variables relating to line length when
4748    the |line_break| procedure is initializing itself.
4749*/
4750
4751static void tex_aux_set_indentation(const line_break_properties *properties)
4752{
4753    if (properties->par_shape) {
4754        int n = specification_count(properties->par_shape);
4755        if (n > 0) {
4756            if (specification_repeat(properties->par_shape)) {
4757                lmt_linebreak_state.last_special_line = max_halfword;
4758            } else {
4759                lmt_linebreak_state.last_special_line = n - 1;
4760            }
4761            lmt_linebreak_state.second_indent = tex_get_specification_indent(properties->par_shape, n);
4762            lmt_linebreak_state.second_width = tex_get_specification_width(properties->par_shape, n);
4763            lmt_linebreak_state.second_indent = swap_parshape_indent(properties->paragraph_dir, lmt_linebreak_state.second_indent, lmt_linebreak_state.second_width);
4764        } else {
4765            lmt_linebreak_state.last_special_line = 0; 
4766            lmt_linebreak_state.second_width = properties->hsize;
4767            lmt_linebreak_state.second_indent = 0;
4768        }
4769    } else if (properties->hang_indent == 0) {
4770        lmt_linebreak_state.last_special_line = 0;
4771        lmt_linebreak_state.second_width = properties->hsize;
4772        lmt_linebreak_state.second_indent = 0;
4773    } else {
4774        halfword used_hang_indent = swap_hang_indent(properties->paragraph_dir, properties->hang_indent);
4775        /*tex
4776
4777            Set line length parameters in preparation for hanging indentation. We compute the
4778            values of |easy_line| and the other local variables relating to line length when the
4779            |line_break| procedure is initializing itself.
4780
4781        */
4782        lmt_linebreak_state.last_special_line = abs(properties->hang_after);
4783        if (properties->hang_after < 0) {
4784            lmt_linebreak_state.first_width = properties->hsize - abs(used_hang_indent);
4785            if (used_hang_indent >= 0) {
4786                lmt_linebreak_state.first_indent = used_hang_indent;
4787            } else {
4788                lmt_linebreak_state.first_indent = 0;
4789            }
4790            lmt_linebreak_state.second_width = properties->hsize;
4791            lmt_linebreak_state.second_indent = 0;
4792        } else {
4793            lmt_linebreak_state.first_width = properties->hsize;
4794            lmt_linebreak_state.first_indent = 0;
4795            lmt_linebreak_state.second_width = properties->hsize - abs(used_hang_indent);
4796            if (used_hang_indent >= 0) {
4797                lmt_linebreak_state.second_indent = used_hang_indent;
4798            } else {
4799                lmt_linebreak_state.second_indent = 0;
4800            }
4801        }
4802    }
4803}
4804
4805/*tex
4806
4807    Check for special treatment of last line of paragraph. The new algorithm for the last line
4808    requires that the stretchability |par_fill_skip| is infinite and the stretchability of
4809    |left_skip| plus |right_skip| is finite.
4810
4811*/
4812
4813static void tex_aux_set_last_line_fit(const line_break_properties *properties)
4814{
4815    lmt_linebreak_state.do_last_line_fit = 0;
4816    if (properties->last_line_fit > 0) {
4817        halfword q = lmt_linebreak_state.last_line_fill;
4818        if (glue_stretch(q) > 0 && glue_stretch_order(q) > normal_glue_order) {
4819            if (lmt_linebreak_state.background[total_fi_amount] == 0 && lmt_linebreak_state.background[total_fil_amount] == 0 &&
4820                lmt_linebreak_state.background[total_fill_amount] == 0 && lmt_linebreak_state.background[total_filll_amount] == 0) {
4821                lmt_linebreak_state.do_last_line_fit = 1;
4822                lmt_linebreak_state.fill_width[fi_order] = 0;
4823                lmt_linebreak_state.fill_width[fil_order] = 0;
4824                lmt_linebreak_state.fill_width[fill_order] = 0;
4825                lmt_linebreak_state.fill_width[filll_order] = 0;
4826                lmt_linebreak_state.fill_width[glue_stretch_order(q) - fi_glue_order] = glue_stretch(q);
4827            }
4828        }
4829    }
4830}
4831
4832static void tex_aux_apply_last_line_fit(void)
4833{
4834    if (lmt_linebreak_state.do_last_line_fit) {
4835        /*tex
4836            Adjust the final line of the paragraph; here we either reset |do_last_line_fit| or
4837            adjust the |par_fill_skip| glue.
4838        */
4839        if (active_short(lmt_linebreak_state.best_bet) == 0) {
4840            lmt_linebreak_state.do_last_line_fit = 0;
4841        } else {
4842            glue_amount(lmt_linebreak_state.last_line_fill) += (active_short(lmt_linebreak_state.best_bet) - active_glue(lmt_linebreak_state.best_bet));
4843            glue_stretch(lmt_linebreak_state.last_line_fill) = 0;
4844        }
4845    }
4846}
4847
4848static void tex_aux_set_both_skips(const line_break_properties *properties)
4849{
4850    halfword l = properties->left_skip;
4851    halfword r = properties->right_skip;
4852    lmt_linebreak_state.background[total_advance_amount] = glue_amount(l) + glue_amount(r);
4853    lmt_linebreak_state.background[total_stretch_amount] = 0;
4854    lmt_linebreak_state.background[total_fi_amount] = 0;
4855    lmt_linebreak_state.background[total_fil_amount] = 0;
4856    lmt_linebreak_state.background[total_fill_amount] = 0;
4857    lmt_linebreak_state.background[total_filll_amount] = 0;
4858    lmt_linebreak_state.background[total_stretch_amount + glue_stretch_order(l)] = glue_stretch(l);
4859    lmt_linebreak_state.background[total_stretch_amount + glue_stretch_order(r)] += glue_stretch(r);
4860    lmt_linebreak_state.background[total_shrink_amount] = tex_aux_checked_shrink(l) + tex_aux_checked_shrink(r);
4861}
4862
4863static void tex_aux_set_adjust_spacing_state(void)
4864{
4865    lmt_linebreak_state.background[font_stretch_amount] = 0;
4866    lmt_linebreak_state.background[font_shrink_amount] = 0;
4867    lmt_linebreak_state.current_font_step = -1;
4868    lmt_packaging_state.previous_char_ptr = null;
4869}
4870
4871static void tex_aux_set_extra_stretch(line_break_properties *properties)
4872{
4873    halfword el = properties->emergency_left_skip;
4874    halfword er = properties->emergency_right_skip;
4875    lmt_linebreak_state.background[total_stretch_amount] += properties->emergency_extra_stretch;
4876    if (tex_aux_emergency_skip(el)) {
4877        lmt_linebreak_state.emergency_left_skip = tex_copy_node(properties->left_skip);
4878        properties->left_skip = lmt_linebreak_state.emergency_left_skip;
4879        glue_amount(properties->left_skip) += glue_amount(el);
4880        glue_stretch(properties->left_skip) += glue_stretch(el);
4881        glue_shrink(properties->left_skip) += glue_shrink(el);
4882        lmt_linebreak_state.background[total_advance_amount] += glue_amount(el);
4883        lmt_linebreak_state.background[total_stretch_amount] += glue_stretch(el);
4884        lmt_linebreak_state.background[total_shrink_amount] += glue_shrink(el);
4885    }
4886    if (tex_aux_emergency_skip(er)) {
4887        lmt_linebreak_state.emergency_right_skip = tex_copy_node(properties->right_skip);
4888        properties->right_skip = lmt_linebreak_state.emergency_right_skip;
4889        glue_amount(properties->right_skip) += glue_amount(er);
4890        glue_stretch(properties->right_skip) += glue_stretch(er);
4891        glue_shrink(properties->right_skip) += glue_shrink(er);
4892        lmt_linebreak_state.background[total_advance_amount] += glue_amount(er);
4893        lmt_linebreak_state.background[total_stretch_amount] += glue_stretch(er);
4894        lmt_linebreak_state.background[total_shrink_amount] += glue_shrink(er);
4895    }
4896}
4897
4898/*tex 
4899    Find an active node with fewest demerits. 
4900*/
4901
4902static int tex_aux_quit_linebreak(const line_break_properties *properties, int pass)
4903{
4904    if (properties->looseness == 0) {
4905        return 1;
4906    } else {
4907        /*tex
4908
4909            Find the best active node for the desired looseness. The adjustment for a
4910            desired looseness is a slightly more complicated version of the loop just
4911            considered. Note that if a paragraph is broken into segments by displayed
4912            equations, each segment will be subject to the looseness calculation,
4913            independently of the other segments.
4914
4915            Using local variables makes it more readable.
4916
4917        */
4918        halfword r = node_next(active_head);
4919        halfword actual_looseness = 0;
4920        halfword best_line = lmt_linebreak_state.best_line;
4921        int verdict = 0;
4922        int tracing = tracing_looseness_par;
4923        if (tracing) {
4924            tex_begin_diagnostic();
4925            tex_print_format("[looseness: pass %i, lines %i, looseness %i]\n", pass, best_line - 1, properties->looseness);
4926        }
4927        do {
4928            if (node_type(r) != delta_node) {
4929                halfword line_number = active_line_number(r);
4930                halfword line_difference = line_number - best_line;
4931                halfword total_demerits = active_total_demerits(r);
4932                if ((line_difference < actual_looseness && properties->looseness <= line_difference) || (line_difference > actual_looseness && properties->looseness >= line_difference)) {
4933                    lmt_linebreak_state.best_bet = r;
4934                    actual_looseness = line_difference;
4935                    lmt_linebreak_state.fewest_demerits = total_demerits;
4936                    if (tracing) {
4937                        tex_print_format("%l[looseness: pass %i, line %i, difference %i, demerits %i, %s optimal]", pass, line_number - 1, line_difference, total_demerits, "sub");
4938                    }
4939                } else if (line_difference == actual_looseness && total_demerits < lmt_linebreak_state.fewest_demerits) {
4940                    lmt_linebreak_state.best_bet = r;
4941                    lmt_linebreak_state.fewest_demerits = total_demerits;
4942                    if (tracing) {
4943                        tex_print_format("%l[looseness: pass %i, line %i, difference %i, demerits %i, %s optimal]", pass, line_number - 1, line_difference, total_demerits, "more");
4944                    }
4945                } else {
4946                    if (tracing) {
4947                        tex_print_format("%l[looseness: pass %i, line %i, difference %i, demerits %i, %s optimal]", pass, line_number - 1, line_difference, total_demerits, "not");
4948                    }
4949                }
4950            }
4951            r = node_next(r);
4952        } while (r != active_head);
4953        lmt_linebreak_state.actual_looseness = actual_looseness;
4954        lmt_linebreak_state.best_line = active_line_number(lmt_linebreak_state.best_bet);
4955        verdict = actual_looseness == properties->looseness;
4956        if (tracing) {
4957            /*tex
4958                The finally reported demerits in the tracker lack the linepenalty squared etc. so
4959                we cannot really compare these values.
4960            */
4961            tex_print_format("%l[looseness: pass %i, looseness %i, line %i, demerits %i, %s]\n", pass, actual_looseness, lmt_linebreak_state.best_line - 1, lmt_linebreak_state.fewest_demerits, verdict ? "success" : "failure");
4962            tex_end_diagnostic();
4963        }
4964        return verdict || pass >= linebreak_final_pass;
4965    }
4966}
4967
4968static void tex_aux_find_best_bet(void)
4969{
4970    halfword r = node_next(active_head);
4971    lmt_linebreak_state.fewest_demerits = awful_bad;
4972    do {
4973        if ((node_type(r) != delta_node) && (active_total_demerits(r) < lmt_linebreak_state.fewest_demerits)) {
4974            lmt_linebreak_state.fewest_demerits = active_total_demerits(r);
4975            lmt_linebreak_state.best_bet = r;
4976        }
4977        r = node_next(r);
4978    } while (r != active_head);
4979    lmt_linebreak_state.best_line = active_line_number(lmt_linebreak_state.best_bet);
4980}
4981
4982void tex_do_line_break(line_break_properties *properties)
4983{
4984 // halfword passes = line_break_passes_par > 0 ? properties->par_passes : 0; /* We could test this earlier. */
4985    halfword passes = properties->par_passes;
4986    int subpasses = passes ? tex_get_specification_count(passes) : 0;
4987    int subpass = -2;
4988    int pass = linebreak_no_pass;
4989    halfword first = node_next(temp_head);
4990    int state = 0;
4991    lmt_linebreak_state.passes[properties->par_context].n_of_break_calls++;
4992    /*tex
4993        We do some preparations first. This concern the node list that we are going to break into
4994        lines.
4995    */
4996    tex_aux_fix_prev_graf();
4997    /*tex
4998        Next we check properties. These mostly come from the initial par node.
4999    */
5000    properties->emergency_original = properties->emergency_stretch;
5001    /*tex
5002        We start with all kind of initializations. By wrapping in a record we don't need to pass
5003        them around.
5004    */
5005    lmt_linebreak_state.force_check_hyphenation = hyphenation_permitted(properties->hyphenation_mode, force_check_hyphenation_mode);
5006    lmt_linebreak_state.callback_id = properties->line_break_checks ? lmt_callback_defined(line_break_callback) : 0;
5007    lmt_linebreak_state.fewest_demerits = 0;
5008    lmt_linebreak_state.checked_expansion = -1;
5009    lmt_linebreak_state.no_shrink_error_yet = 1;
5010    lmt_linebreak_state.extra_background_stretch = 0;
5011    lmt_linebreak_state.emergency_left_skip = null;
5012    lmt_linebreak_state.emergency_right_skip = null;
5013    lmt_linebreak_state.emergency_amount = 0;
5014    lmt_linebreak_state.emergency_factor = scaling_factor;
5015    lmt_linebreak_state.emergency_percentage = 0;
5016    lmt_linebreak_state.emergency_width_amount = 0;
5017    lmt_linebreak_state.emergency_width_extra = 0;
5018    lmt_linebreak_state.emergency_left_amount = 0;
5019    lmt_linebreak_state.emergency_left_extra = 0;
5020    lmt_linebreak_state.emergency_right_amount = 0;
5021    lmt_linebreak_state.emergency_right_extra = 0;
5022    lmt_linebreak_state.has_orphans = 0;
5023    lmt_linebreak_state.has_toddlers = 0;
5024
5025 // for (int i = 0; i < n_of_glue_amounts; i++) {
5026 //     lmt_linebreak_state.active_width[i] = 0;
5027 //     lmt_linebreak_state.background[i] = 0;
5028 //     lmt_linebreak_state.break_width[i] = 0;
5029 //     lmt_linebreak_state.minimal_demerits[i] = 0;
5030 // }
5031
5032    /* Needs checking: shouldn't this be done every pass? */
5033
5034    lmt_linebreak_state.minimum_demerits = awful_bad;
5035    for (int i = 0; i < max_n_of_fitness_values; i++) {
5036        lmt_linebreak_state.minimal_demerits[i] = awful_bad;
5037    }
5038
5039    lmt_linebreak_state.line_break_dir = properties->paragraph_dir;
5040    if (lmt_linebreak_state.dir_ptr) {
5041        tex_flush_node_list(lmt_linebreak_state.dir_ptr);
5042        lmt_linebreak_state.dir_ptr = null;
5043    }
5044    /*tex
5045        This is a bit terrible hack butif we want to inject something it has to be done after we
5046        are done with the left and right protrusion as we are redetecing in the post line break
5047        routine too.
5048    */
5049    if (lmt_linebreak_state.inject_after_par) {
5050        /*tex This should not happen. */
5051        tex_flush_node(lmt_linebreak_state.inject_after_par);
5052    }
5053    lmt_linebreak_state.inject_after_par = null;
5054    /* */
5055    tex_aux_set_adjacent_demerits(properties);
5056    tex_aux_set_adjust_spacing(properties);
5057    tex_aux_set_orphan_penalties(properties, 0);
5058    tex_aux_set_toddler_penalties(properties, 0);
5059    tex_aux_set_indentation(properties);
5060    tex_aux_set_looseness(properties);
5061    tex_aux_set_both_skips(properties);
5062    tex_aux_set_adjust_spacing_state();
5063    tex_aux_set_last_line_fit(properties);
5064    /*tex
5065        Here we start doing the real work: find optimal breakpoints. We have an initial pass
5066        (pretolerance), when needed a second one (tolerance) and when we're still not done we
5067        do a third pass when emergency stretch is set. However, in \LUAMETATEX\ we can also
5068        have additional so called par passes replacing that one.
5069
5070        first pass  : pretolerance without hyphenation
5071        second pass : tolerance with hyphenation
5072        final pass  : tolerance with hyphenation and emergencystretch and looseness
5073    */
5074    lmt_linebreak_state.threshold = properties->pretolerance;
5075    if (properties->tracing_paragraphs > 1) {
5076        tex_begin_diagnostic();
5077        tex_print_str("[linebreak: original] ");
5078        tex_short_display(first);
5079        tex_end_diagnostic();
5080    }
5081    if (subpasses) {
5082        pass = linebreak_specification_pass;
5083        lmt_linebreak_state.threshold = properties->pretolerance; /* or tolerance */
5084        if (properties->tracing_paragraphs > 0 || properties->tracing_passes > 0) {
5085            if (specification_presets(passes)) {
5086                tex_begin_diagnostic();
5087                tex_print_str("[linebreak: specification presets]");
5088                tex_end_diagnostic();
5089            }
5090        }
5091        if (specification_presets(passes)) {
5092            subpass = 1;
5093        }
5094    } else if (properties->pretolerance >= 0) {
5095        pass = linebreak_first_pass;
5096        lmt_linebreak_state.threshold = properties->pretolerance;
5097    } else {
5098        pass = linebreak_second_pass;
5099        lmt_linebreak_state.threshold = properties->tolerance;
5100    }
5101    lmt_linebreak_state.global_threshold = lmt_linebreak_state.threshold;
5102    if (lmt_linebreak_state.callback_id) {
5103        tex_aux_line_break_callback_initialize(lmt_linebreak_state.callback_id, properties->line_break_checks, subpasses);
5104    }
5105    /*tex
5106        The main loop starts here. We set |current| to the start if the paragraph and the break
5107        routine returns either |null| or some place in the list that needs attention.
5108    */
5109 /* state = tex_aux_analyze_list(first); */
5110    while (1) {
5111        halfword current = first;
5112        int artificial = 0;
5113        switch (pass) {
5114            case linebreak_no_pass:
5115                goto DONE;
5116            case linebreak_first_pass:
5117                if (properties->tracing_paragraphs > 0 || properties->tracing_passes > 0) {
5118                    tex_begin_diagnostic();
5119                    tex_print_format("[linebreak: first pass, used tolerance %i]", lmt_linebreak_state.threshold);
5120                    // tex_end_diagnostic();
5121                }
5122                lmt_linebreak_state.passes[properties->par_context].n_of_first_passes++;
5123                break;
5124            case linebreak_second_pass:
5125                if (tex_aux_emergency(properties)) {
5126                    lmt_linebreak_state.passes[properties->par_context].n_of_second_passes++;
5127                    if (properties->tracing_paragraphs > 0 || properties->tracing_passes > 0) {
5128                        tex_begin_diagnostic();
5129                        tex_print_format("[linebreak: second pass, used tolerance %i]", lmt_linebreak_state.threshold);
5130                        // tex_end_diagnostic();
5131                    }
5132                    lmt_linebreak_state.force_check_hyphenation = 1;
5133                    break;
5134                } else {
5135                    pass = linebreak_final_pass;
5136                    /* fall through */
5137                }
5138            case linebreak_final_pass:
5139                lmt_linebreak_state.passes[properties->par_context].n_of_final_passes++;
5140                if (properties->tracing_paragraphs > 0 || properties->tracing_passes > 0) {
5141                    tex_begin_diagnostic();
5142                    tex_print_format("[linebreak: final pass, used tolerance %i, used emergency stretch %p]", lmt_linebreak_state.threshold, properties->emergency_stretch);
5143                    // tex_end_diagnostic();
5144                }
5145                lmt_linebreak_state.force_check_hyphenation = 1;
5146                lmt_linebreak_state.background[total_stretch_amount] += properties->emergency_stretch;
5147                tex_aux_set_extra_stretch(properties);
5148                break;
5149            case linebreak_specification_pass:
5150                if (specification_presets(passes)) {
5151                    if (subpass <= passes_first_final(passes)) {
5152                        tex_aux_set_sub_pass_parameters(
5153                            properties, passes, subpass,
5154                            first,
5155                            properties->tracing_passes > 1,
5156                            tex_get_passes_features(passes,subpass),
5157                            0, 0, 0, 0, 0, 0, 0
5158                        );
5159                        lmt_linebreak_state.passes[properties->par_context].n_of_specification_passes++;
5160                    }
5161                } else {
5162                    switch (subpass) {
5163                        case -2:
5164                            lmt_linebreak_state.threshold = properties->pretolerance;
5165                            lmt_linebreak_state.force_check_hyphenation = 0;
5166                            subpass = -1;
5167                            break;
5168                        case -1:
5169                            lmt_linebreak_state.threshold = properties->tolerance;
5170                            lmt_linebreak_state.force_check_hyphenation = 1;
5171                            subpass = 0;
5172                            break;
5173                        default:
5174                            lmt_linebreak_state.force_check_hyphenation = 1;
5175                            break;
5176                    }
5177                }
5178                if (properties->tracing_paragraphs > 0 || properties->tracing_passes > 0) {
5179                    tex_begin_diagnostic();
5180                    tex_print_format("[linebreak: specification subpass %i]\n", subpass);
5181                }
5182                lmt_linebreak_state.passes[properties->par_context].n_of_sub_passes++;
5183                break;
5184        }
5185        lmt_linebreak_state.saved_threshold = 0;
5186        if (lmt_linebreak_state.threshold > infinite_bad) {
5187            lmt_linebreak_state.threshold = infinite_bad; /* we can move this check to where threshold is set */
5188        }
5189        lmt_linebreak_state.global_threshold = lmt_linebreak_state.threshold;
5190        if (properties->tracing_fitness && properties->fitness_classes) {
5191            tex_aux_report_fitness_classes(properties, pass, subpass);
5192            tex_aux_report_adjacent_demerits(properties, pass, subpass);
5193        }
5194        if (lmt_linebreak_state.callback_id) {
5195            tex_aux_line_break_callback_start(lmt_linebreak_state.callback_id, properties->line_break_checks, pass, subpass,
5196                tex_max_fitness(properties->fitness_classes), tex_med_fitness(properties->fitness_classes));
5197        }
5198        /*tex
5199            Create an active breakpoint representing the beginning of the paragraph. After
5200            doing this we have created a cycle.
5201        */
5202        tex_aux_set_initial_active(properties);
5203        /*tex
5204            We now initialize the arrays that will be used in the calculations. We start fresh
5205            each pass.
5206        */
5207
5208        {
5209            halfword line = 1;
5210            scaled line_width;
5211
5212            lmt_linebreak_state.current_line_number = line; /* we could just use this variable */
5213            if (line > lmt_linebreak_state.easy_line) {
5214                line_width = lmt_linebreak_state.second_width;
5215            } else if (line > lmt_linebreak_state.last_special_line) {
5216                line_width = lmt_linebreak_state.second_width;
5217            } else if (properties->par_shape) {
5218                line_width = tex_get_specification_width(properties->par_shape, line);
5219            } else {
5220                line_width = lmt_linebreak_state.first_width;
5221            }
5222            lmt_linebreak_state.background[total_stretch_amount] -= lmt_linebreak_state.emergency_amount;
5223            if (lmt_linebreak_state.emergency_percentage) {
5224                scaled stretch = tex_xn_over_d(line_width, lmt_linebreak_state.emergency_percentage, scaling_factor);
5225                lmt_linebreak_state.background[total_stretch_amount] += stretch;
5226                lmt_linebreak_state.emergency_amount = stretch;
5227            } else {
5228                lmt_linebreak_state.emergency_amount = 0;
5229            }
5230            lmt_linebreak_state.background[total_advance_amount] -= lmt_linebreak_state.emergency_width_amount;
5231            if (lmt_linebreak_state.emergency_width_extra) {
5232                scaled extra = tex_xn_over_d(line_width, lmt_linebreak_state.emergency_width_extra, scaling_factor);
5233                lmt_linebreak_state.background[total_advance_amount] += extra;
5234                lmt_linebreak_state.emergency_width_amount = extra;
5235            } else {
5236                lmt_linebreak_state.emergency_width_amount = 0;
5237            }
5238
5239            lmt_linebreak_state.background[total_advance_amount] -= lmt_linebreak_state.emergency_left_amount;
5240            if (lmt_linebreak_state.emergency_left_extra) {
5241                scaled extra = tex_xn_over_d(line_width, lmt_linebreak_state.emergency_left_extra, scaling_factor);
5242                lmt_linebreak_state.background[total_advance_amount] += extra;
5243                lmt_linebreak_state.emergency_left_amount = extra;
5244            } else {
5245                lmt_linebreak_state.emergency_left_amount = 0;
5246            }
5247            lmt_linebreak_state.background[total_advance_amount] -= lmt_linebreak_state.emergency_right_amount;
5248            if (lmt_linebreak_state.emergency_right_extra) {
5249                scaled extra = tex_xn_over_d(line_width, lmt_linebreak_state.emergency_right_extra, scaling_factor);
5250                lmt_linebreak_state.background[total_advance_amount] += extra;
5251                lmt_linebreak_state.emergency_right_amount = extra;
5252            } else {
5253                lmt_linebreak_state.emergency_right_amount = 0;
5254            }
5255        }
5256        tex_aux_set_target_to_source(properties->adjust_spacing, lmt_linebreak_state.active_width, lmt_linebreak_state.background);
5257        lmt_linebreak_state.passive = null;
5258        lmt_linebreak_state.printed_node = temp_head;
5259        lmt_linebreak_state.serial_number = 0;
5260        lmt_print_state.font_in_short_display = null_font;
5261        lmt_packaging_state.previous_char_ptr = null;
5262        /*tex
5263            We have local boxes and (later) some possible mid paragraph overloads that we need to
5264            initialize.
5265        */
5266        tex_aux_set_local_par_state(current);
5267        /*tex
5268
5269            As we move through the hlist, we need to keep the |active_width| array up to date, so
5270            that the badness of individual lines is readily calculated by |try_break|. It is
5271            convenient to use the short name |active_width [1]| for the component of active width
5272            that represents real width as opposed to glue.
5273
5274        */
5275        switch (pass) {
5276            case linebreak_final_pass:
5277                artificial = 1;
5278                break;
5279            case linebreak_specification_pass:
5280                artificial = (subpass >= passes_first_final(passes)) || (subpass == subpasses);
5281                break;
5282            default:
5283                artificial = 0;
5284                break;
5285        }
5286        current = tex_aux_break_list(properties, pass, subpass, current, first, &state, artificial);
5287        if (! current) {
5288            /*tex
5289
5290                Try the final line break at the end of the paragraph, and |goto done| if the desired
5291                breakpoints have been found.
5292
5293                The forced line break at the paragraph's end will reduce the list of breakpoints so
5294                that all active nodes represent breaks at |cur_p = null|. On the first pass, we
5295                insist on finding an active node that has the correct \quote {looseness.} On the
5296                final pass, there will be at least one active node, and we will match the desired
5297                looseness as well as we can.
5298
5299                The global variable |best_bet| will be set to the active node for the best way to
5300                break the paragraph, and a few other variables are used to help determine what is
5301                best.
5302
5303            */ /* why hyphenated node here: */
5304            scaled shortfall = tex_aux_try_break(properties, eject_penalty, hyphenated_node, first, current, lmt_linebreak_state.callback_id, properties->line_break_checks, pass, subpass, artificial);
5305            if (node_next(active_head) != active_head) {
5306                /*tex Find an active node with fewest demerits. */
5307                tex_aux_find_best_bet();
5308                if (pass == linebreak_specification_pass) {
5309                    /*tex This is where sub passes differ: we do a check. */
5310                    if (subpass < 0) {
5311                        goto HERE;
5312                    } else if (subpass < passes_first_final(passes)) {
5313                        goto DONE;
5314                    } else if (subpass < subpasses) {
5315                        int found = tex_aux_check_sub_pass(properties, state, shortfall, passes, subpass, subpasses, first);
5316                        if (found > 0) {
5317                            subpass = found;
5318                            goto HERE;
5319                        } else if (found < 0) {
5320                            goto DONE;
5321                        }
5322                    } else {
5323                        /* continue */
5324                    }
5325                }
5326                if (tex_aux_quit_linebreak(properties, pass)) {
5327                    goto DONE;
5328                }
5329            }
5330        }
5331        if (subpass <= passes_first_final(passes)) {
5332            ++subpass;
5333        }
5334      HERE:
5335        if (properties->tracing_paragraphs > 0 || properties->tracing_passes > 0) {
5336            tex_end_diagnostic(); // see above
5337        }
5338        /*tex Clean up the memory by removing the break nodes. */
5339        tex_aux_clean_up_the_memory();
5340        switch (pass) {
5341            case linebreak_no_pass:
5342                /* just to be sure */
5343                goto DONE;
5344            case linebreak_first_pass:
5345                lmt_linebreak_state.threshold = properties->tolerance;
5346                lmt_linebreak_state.global_threshold = lmt_linebreak_state.threshold;
5347                pass = linebreak_second_pass;
5348                break;
5349            case linebreak_second_pass:
5350                pass = linebreak_final_pass;
5351                break;
5352            case linebreak_final_pass:
5353                pass = linebreak_no_pass;
5354                break;
5355            case linebreak_specification_pass:
5356                break;
5357        }
5358        if (lmt_linebreak_state.callback_id) {
5359            tex_aux_line_break_callback_stop(lmt_linebreak_state.callback_id, properties->line_break_checks);
5360        }
5361    }
5362    goto INDEED;
5363  DONE:
5364    if (lmt_linebreak_state.callback_id) {
5365        tex_aux_line_break_callback_stop(lmt_linebreak_state.callback_id, properties->line_break_checks);
5366    }
5367    if (properties->tracing_paragraphs > 0 || properties->tracing_passes > 0) {
5368        tex_end_diagnostic(); // see above
5369    }
5370  INDEED:
5371    tex_aux_apply_last_line_fit();
5372    tex_aux_wipe_optionals(properties, first, state);
5373    tex_aux_apply_special_penalties(properties, first, state);
5374    tex_aux_apply_special_factors(properties, first, state);
5375    tex_flush_node_list(lmt_linebreak_state.dir_ptr);
5376    lmt_linebreak_state.dir_ptr = null;
5377    {
5378        int callback_id = lmt_callback_defined(linebreak_quality_callback);
5379        if (callback_id) {
5380            halfword overfull = 0;
5381            halfword underfull = 0;
5382            halfword verdict = 0;
5383            halfword classified = 0;
5384            tex_check_linebreak_quality(0, &overfull, &underfull, &verdict, &classified);
5385            tex_aux_quality_callback(callback_id, first,
5386                passes ? passes_identifier(passes) : 0, pass, subpass, subpasses, state,
5387                overfull, underfull, verdict, classified
5388            );
5389        }
5390    }
5391    /*tex
5392        Here is a bonus warning because, after all, a user wants something to happen:
5393    */
5394    if (properties->looseness && (! tracing_looseness_par) && (properties->looseness != lmt_linebreak_state.actual_looseness)) {
5395        tex_print_nlp();
5396        tex_print_format("%l[looseness: line %i, requested %i, actual %i]\n", lmt_linebreak_state.best_line - 1, properties->looseness, lmt_linebreak_state.actual_looseness);
5397    }
5398    /*tex
5399        Break the paragraph at the chosen breakpoints. Once the best sequence of breakpoints has been
5400        found (hurray), we call on the procedure |post_line_break| to finish the remainder of the
5401        work. By introducing this subprocedure, we are able to keep |line_break| from getting
5402        extremely long. The first thing |ext_post_line_break| does is reset |dir_ptr|.
5403
5404        Here we still have a temp node as head.
5405    */
5406    tex_aux_post_line_break(properties, lmt_linebreak_state.line_break_dir, lmt_linebreak_state.callback_id, properties->line_break_checks, state);
5407    /*tex
5408        Clean up memory by removing the break nodes (maybe: |tex_flush_node_list(cur_p)|).
5409    */
5410    if (lmt_linebreak_state.emergency_left_skip) {
5411        tex_flush_node(lmt_linebreak_state.emergency_left_skip);
5412     // lmt_linebreak_state.emergency_left_skip = null;
5413    }
5414    if (lmt_linebreak_state.emergency_right_skip) {
5415        tex_flush_node(lmt_linebreak_state.emergency_right_skip);
5416     // lmt_linebreak_state.emergency_right_skip = null;
5417    }
5418    tex_aux_clean_up_the_memory();
5419    if (lmt_linebreak_state.callback_id) {
5420        tex_aux_line_break_callback_wrapup(lmt_linebreak_state.callback_id, properties->line_break_checks);
5421    }
5422}
5423
5424void tex_get_linebreak_info(int *f, int *a)
5425{
5426    *f = lmt_linebreak_state.fewest_demerits;
5427    *a = lmt_linebreak_state.actual_looseness;
5428}
5429
5430/*tex
5431
5432    So far we have gotten a little way into the |line_break| routine, having covered its important
5433    |try_break| subroutine. Now let's consider the rest of the process.
5434
5435    The main loop of |line_break| traverses the given hlist, starting at |vlink (temp_head)|, and
5436    calls |try_break| at each legal breakpoint. A variable called |auto_breaking| is set to true
5437    except within math formulas, since glue nodes are not legal breakpoints when they appear in
5438    formulas.
5439
5440    The current node of interest in the hlist is pointed to by |cur_p|. Another variable, |prev_p|,
5441    is usually one step behind |cur_p|, but the real meaning of |prev_p| is this: If |type (cur_p)
5442    = glue_node| then |cur_p| is a legal breakpoint if and only if |auto_breaking| is true and
5443    |prev_p| does not point to a glue node, penalty node, explicit kern node, or math node.
5444
5445    The total number of lines that will be set by |post_line_break| is |best_line - prev_graf - 1|.
5446    The last breakpoint is specified by |break_node (best_bet)|, and this passive node points to
5447    the other breakpoints via the |prev_break| links. The finishing-up phase starts by linking the
5448    relevant passive nodes in forward order, changing |prev_break| to |next_break|. (The
5449    |next_break| fields actually reside in the same memory space as the |prev_break| fields did,
5450    but we give them a new name because of their new significance.) Then the lines are justified,
5451    one by one.
5452
5453    The |post_line_break| must also keep an dir stack, so that it can output end direction
5454    instructions at the ends of lines and begin direction instructions at the beginnings of lines.
5455
5456*/
5457
5458/*tex The new name for |prev_break| after links are reversed: */
5459
5460# define passive_next_break passive_prev_break
5461
5462/*tex The |int|s are actually |halfword|s or |scaled|s. */
5463
5464void tex_get_line_content_range(halfword head, halfword tail, halfword *first, halfword *last)
5465{
5466    halfword current = head;
5467    *first = head;
5468    *last = tail;
5469    while (current) {
5470        if (node_type(current) == glue_node) {
5471            switch (node_subtype(current)) {
5472                case left_skip_glue:
5473                case par_fill_left_skip_glue:
5474                case par_init_left_skip_glue:
5475                case indent_skip_glue:
5476                case left_hang_skip_glue:
5477                    *first = current;
5478                    current = node_next(current);
5479                    break;
5480                default:
5481                    current = null;
5482                    break;
5483            }
5484        } else {
5485            current = null;
5486        }
5487    }
5488    current = tail;
5489    while (current) {
5490        if (node_type(current) == glue_node) {
5491            switch (node_subtype(current)) {
5492                case right_skip_glue:
5493                case par_fill_right_skip_glue:
5494                case par_init_right_skip_glue:
5495                case right_hang_skip_glue:
5496                    *last = current;
5497                    current = node_prev(current);
5498                    break;
5499                default:
5500                    current = null;
5501                    break;
5502            }
5503        } else {
5504            current = null;
5505        }
5506    }
5507}
5508
5509static halfword checked_penalty(long long p)
5510{
5511    return p > max_integer ? infinite_penalty : p < min_integer ? -infinite_penalty : (halfword) p;
5512}
5513
5514static void tex_aux_trace_penalty(const char *what, int line, int index, long long penalty, long long total)
5515{
5516    if (tracing_penalties_par > 0) {
5517        tex_begin_diagnostic();
5518        tex_print_format("[linebreak: %s penalty, line %i, index %i, delta %i, total %i]", what, line, index, checked_penalty(penalty), checked_penalty(total));
5519        tex_end_diagnostic();
5520    }
5521}
5522
5523static void tex_aux_post_line_break(const line_break_properties *properties, halfword line_break_dir, int callback_id, halfword checks, int state)
5524{
5525    /*tex temporary registers for list manipulation */
5526    halfword q, r;
5527    halfword ls = null;
5528    halfword rs = null;
5529    /*tex was a break at glue? */
5530    int glue_break;
5531    /*tex are we in some shape */
5532    int shaping = 0;
5533    /*tex was the current break at a discretionary node? */
5534    int disc_break;
5535    /*tex and did it have a nonempty post-break part? */
5536    int post_disc_break;
5537    /*tex width of line number |cur_line| */
5538    scaled cur_width;
5539    /*tex left margin of line number |cur_line| */
5540    scaled cur_indent;
5541    /*tex |cur_p| will become the first breakpoint; */
5542    halfword cur_p = null;
5543    /*tex the current line number being justified */
5544    halfword cur_line;
5545    /*tex this saves calculations: */
5546    int last_line = 0;
5547    int first_line = 0;
5548    int math_nesting = 0;
5549    int math_attr = null;
5550    /*tex the current direction: */
5551    lmt_linebreak_state.dir_ptr = cur_list.direction_stack;
5552    /*tex
5553        Reverse the links of the relevant passive nodes, setting |cur_p| to the first breakpoint.
5554        The job of reversing links in a list is conveniently regarded as the job of taking items
5555        off one stack and putting them on another. In this case we take them off a stack pointed
5556        to by |q| and having |prev_break| fields; we put them on a stack pointed to by |cur_p|
5557        and having |next_break| fields. Node |r| is the passive node being moved from stack to
5558        stack.
5559    */
5560    if (callback_id) {
5561        tex_aux_line_break_callback_collect(callback_id, checks);
5562    }
5563    q = active_break_node(lmt_linebreak_state.best_bet);
5564    do {
5565        r = q;
5566        q = passive_prev_break(q);
5567        passive_next_break(r) = cur_p;
5568        cur_p = r;
5569    } while (q);
5570    if (callback_id) {
5571        halfword p = cur_p;
5572        while (p) {
5573            tex_aux_line_break_callback_list(callback_id, checks, p);
5574            p = passive_next_break(p);
5575        }
5576    }
5577    /*tex Temporary: */
5578    if (properties->tracing_passes > 0) {
5579        halfword passive = cur_p;
5580        tex_begin_diagnostic();
5581        tex_print_str("[linebreak: (class demerits deficiency)");
5582        while (passive) {
5583            tex_print_format(" (%i %B %i %p)", (1 << passive_fitness(passive)),  passive_badness(passive), passive_demerits(passive), passive_deficiency(passive));
5584            passive = passive_prev_break(passive);
5585        }
5586        tex_print_str("]");
5587        tex_end_diagnostic();
5588    }
5589    /*tex |prevgraf + 1| */
5590    cur_line = cur_list.prev_graf + 1;
5591    do {
5592        /*tex
5593            Justify the line ending at breakpoint |cur_p|, and append it to the current vertical
5594            list, together with associated penalties and other insertions.
5595
5596            The current line to be justified appears in a horizontal list starting at |vlink
5597            (temp_head)| and ending at |cur_break (cur_p)|. If |cur_break (cur_p)| is a glue node,
5598            we reset the glue to equal the |right_skip| glue; otherwise we append the |right_skip|
5599            glue at the right. If |cur_break (cur_p)| is a discretionary node, we modify the list
5600            so that the discretionary break is compulsory, and we set |disc_break| to |true|. We
5601            also append the |left_skip| glue at the left of the line, unless it is zero.
5602        */
5603        /*tex
5604            We want to get rid of it.
5605        */
5606        halfword cur_disc = null;
5607        /*tex
5608            Local left and right boxes come from \OMEGA\ but have been adapted and extended.
5609        */
5610        halfword leftbox = null;
5611        halfword rightbox = null;
5612        halfword middlebox = null;
5613        if (lmt_linebreak_state.dir_ptr) {
5614            /*tex Insert dir nodes at the beginning of the current line. */
5615            for (halfword q = lmt_linebreak_state.dir_ptr; q; q = node_next(q)) {
5616                halfword tmp = tex_new_dir(normal_dir_subtype, dir_direction(q));
5617                halfword nxt = node_next(temp_head);
5618                tex_attach_attribute_list_copy(tmp, nxt ? nxt : temp_head);
5619                tex_couple_nodes(temp_head, tmp);
5620                /*tex |\break\par| */
5621                tex_try_couple_nodes(tmp, nxt);
5622            }
5623            tex_flush_node_list(lmt_linebreak_state.dir_ptr);
5624            lmt_linebreak_state.dir_ptr = null;
5625        }
5626        /*tex
5627            Modify the end of the line to reflect the nature of the break and to include
5628            |\rightskip|; also set the proper value of |disc_break|. At the end of the following
5629            code, |q| will point to the final node on the list about to be justified. In the
5630            meanwhile |r| will point to the node we will use to insert end-of-line stuff after.
5631            |q == null| means we use the final position of |r|.
5632        */
5633        /*tex begin mathskip code */
5634        q = temp_head;
5635        while (q) {
5636            switch (node_type(q)) {
5637                case glyph_node:
5638                    goto DONE;
5639                case hlist_node:
5640                    if (node_subtype(q) == indent_list) {
5641                        break;
5642                    } else {
5643                        goto DONE;
5644                    }
5645                case glue_node:
5646                    if (tex_is_par_init_glue(q)) {
5647                        break;
5648                    } else {
5649                        goto DONE;
5650                    }
5651                case kern_node:
5652                    if (node_subtype(q) == explicit_kern_subtype || node_subtype(q) == italic_kern_subtype) {
5653                        break;
5654                    } else {
5655                        goto DONE;
5656                    }
5657                case math_node:
5658                    math_surround(q) = 0;
5659                    tex_reset_math_glue_to_zero(q);
5660                    goto DONE;
5661                default:
5662                    if (non_discardable(q)) {
5663                        goto DONE;
5664                    } else {
5665                        break;
5666                    }
5667            }
5668            q = node_next(q);
5669        }
5670      DONE:
5671        /*tex end mathskip code */
5672        r = passive_cur_break(cur_p);
5673        q = null;
5674        disc_break = 0;
5675        post_disc_break = 0;
5676        glue_break = 0;
5677        if (r) {
5678            switch (node_type(r)) {
5679                case glue_node:
5680                    tex_copy_glue_values(r, properties->right_skip);
5681                    node_subtype(r) = right_skip_glue;
5682                    glue_break = 1;
5683                    /*tex |q| refers to the last node of the line */
5684                    q = r;
5685                    rs = q;
5686                    r = node_prev(r);
5687                    /*tex |r| refers to the node after which the dir nodes should be closed */
5688                    break;
5689                case disc_node:
5690                    {
5691                        halfword prv = node_prev(r);
5692                        halfword nxt = node_next(r);
5693                        halfword h = disc_no_break_head(r);
5694                        if (h) {
5695                            tex_flush_node_list(h);
5696                            disc_no_break_head(r) = null;
5697                            disc_no_break_tail(r) = null;
5698                        }
5699                        h = disc_pre_break_head(r);
5700                        if (h) {
5701                            halfword t = disc_pre_break_tail(r);
5702                            tex_set_discpart(r, h, t, glyph_discpart_pre);
5703                            tex_couple_nodes(prv, h);
5704                            tex_couple_nodes(t, r);
5705                            disc_pre_break_head(r) = null;
5706                            disc_pre_break_tail(r) = null;
5707                        }
5708                        h = disc_post_break_head(r);
5709                        if (h) {
5710                            halfword t = disc_post_break_tail(r);
5711                            tex_set_discpart(r, h, t, glyph_discpart_post);
5712                            tex_couple_nodes(r, h);
5713                            tex_couple_nodes(t, nxt);
5714                            disc_post_break_head(r) = null;
5715                            disc_post_break_tail(r) = null;
5716                            post_disc_break = 1;
5717                        }
5718                        cur_disc = r;
5719                        disc_break = 1;
5720                    }
5721                    break;
5722                case kern_node:
5723                    if (node_subtype(r) != right_correction_kern_subtype && node_subtype(r) != left_correction_kern_subtype) {
5724                        kern_amount(r) = 0;
5725                    }
5726                    break;
5727                case math_node :
5728                    math_surround(r) = 0;
5729                    tex_reset_math_glue_to_zero(r);
5730                    break;
5731            }
5732        } else {
5733            /*tex Again a tail run ... maybe combine. */
5734         // for (r = temp_head; node_next(r); r = node_next(r));
5735            r = tex_tail_of_node_list(temp_head);
5736            /*tex Now we're at the end. */
5737            if (r == properties->parfill_right_skip) {
5738                /*tex This should almost always be true... */
5739                q = r;
5740                /*tex |q| refers to the last node of the line (and paragraph) */
5741                r = node_prev(r);
5742            }
5743            /*tex |r| refers to the node after which the dir nodes should be closed */
5744        }
5745        /*tex Adjust the dir stack based on dir nodes in this line. */
5746        line_break_dir = tex_sanitize_dir_state(node_next(temp_head), passive_cur_break(cur_p), properties->paragraph_dir);
5747        /*tex Insert dir nodes at the end of the current line. */
5748        r = tex_complement_dir_state(r);
5749        /*tex
5750            Modify the end of the line to reflect the nature of the break and to include |\rightskip|;
5751            also set the proper value of |disc_break|; Also put the |\leftskip| glue at the left and
5752            detach this line.
5753
5754            The following code begins with |q| at the end of the list to be justified. It ends with
5755            |q| at the beginning of that list, and with |node_next(temp_head)| pointing to the remainder
5756            of the paragraph, if any.
5757
5758            Now [q] refers to the last node on the line and therefore the rightmost breakpoint. The
5759            only exception is the case of a discretionary break with non-empty |pre_break|, then
5760            |q| s been changed to the last node of the |pre_break| list. If the par ends with a
5761            |\break| command, the last line is utterly empty. That is the case of |q == temp_head|.
5762
5763            This code needs to be cleaned up as we now have protrusion and boxes at the edges to
5764            deal with. Old hybrid code.
5765        */
5766        leftbox = tex_use_local_boxes(passive_left_box(cur_p), local_left_box_code);
5767        rightbox = tex_use_local_boxes(passive_right_box(cur_p), local_right_box_code);
5768        middlebox = tex_use_local_boxes(passive_middle_box(cur_p), local_middle_box_code);
5769        /*tex
5770            First we append the right box. It is part of the content so inside the skips.
5771        */
5772        if (rightbox) {
5773             halfword nxt = node_next(r);
5774             tex_couple_nodes(r, rightbox);
5775             tex_try_couple_nodes(rightbox, nxt);
5776             r = rightbox;
5777        }
5778        if (middlebox) {
5779             /*tex
5780                These middle boxes might become more advanced as we can process them by a pass over
5781                the line so that we retain the spot but then, we also loose that with left and right,
5782                so why bother. It would also complicate uniqueness.
5783             */
5784             halfword nxt = node_next(r);
5785             tex_couple_nodes(r, middlebox);
5786             tex_try_couple_nodes(middlebox, nxt);
5787             r = middlebox;
5788        }
5789        if (! q) {
5790            q = r;
5791        }
5792        if (q != temp_head && properties->protrude_chars) {
5793            if (line_break_dir == dir_righttoleft && properties->protrude_chars == protrude_chars_advanced) {
5794                halfword p = q;
5795                halfword l = null;
5796                /*tex Backtrack over the last zero glues and dirs. */
5797                while (p) {
5798                    switch (node_type(p)) {
5799                        case dir_node:
5800                            if (node_subtype(p) != cancel_dir_subtype) {
5801                                goto DONE1;
5802                            } else {
5803                                break;
5804                            }
5805                        case glue_node:
5806                            if (glue_amount(p)) {
5807                                goto DONE3;
5808                            } else {
5809                                break;
5810                            }
5811                        case glyph_node:
5812                            goto DONE1;
5813                        default:
5814                            goto DONE3;
5815                    }
5816                    p = node_prev(p);
5817                }
5818              DONE1:
5819                /*tex When |p| is non zero we have something. */
5820                while (p) {
5821                    switch (node_type(p)) {
5822                        case glyph_node:
5823                            l = p ;
5824                            break;
5825                        case glue_node:
5826                            if (glue_amount(p)) {
5827                                l = null;
5828                            }
5829                            break;
5830                        case dir_node:
5831                            if (dir_direction(p) != dir_righttoleft) {
5832                                goto DONE3;
5833                            } else {
5834                                goto DONE2;
5835                            }
5836                        case par_node:
5837                            goto DONE2;
5838                        case temp_node:
5839                            /*tex Go on. */
5840                            break;
5841                        default:
5842                            l = null;
5843                            break;
5844                    }
5845                    p = node_prev(p);
5846                }
5847              DONE2:
5848                /*tex Time for action. */
5849                if (l && p) {
5850                    scaled w = tex_char_protrusion(l, right_margin_kern_subtype);
5851                    halfword k = tex_new_kern_node(-w, right_margin_kern_subtype);
5852                    tex_attach_attribute_list_copy(k, l);
5853                    tex_couple_nodes(p, k);
5854                    tex_couple_nodes(k, l);
5855                }
5856            } else {
5857                scaled w = 0;
5858                halfword p, ptmp;
5859                if (disc_break && (node_type(q) == glyph_node || node_type(q) != disc_node)) {
5860                    /*tex |q| is reset to the last node of |pre_break| */
5861                    p = q;
5862                } else {
5863                    /*tex get |node_next(p) = q| */
5864                    p = node_prev(q);
5865                }
5866                ptmp = p;
5867                p = tex_aux_find_protchar_right(node_next(temp_head), p);
5868                w = tex_char_protrusion(p, right_margin_kern_subtype);
5869                if (w && lmt_packaging_state.last_rightmost_char) {
5870                    /*tex we have found a marginal kern, append it after |ptmp| */
5871                    halfword k = tex_new_kern_node(-w, right_margin_kern_subtype);
5872                    tex_attach_attribute_list_copy(k, p);
5873                    tex_try_couple_nodes(k, node_next(ptmp));
5874                    tex_couple_nodes(ptmp, k);
5875                    if (ptmp == q) {
5876                        q = node_next(q);
5877                    }
5878                }
5879            }
5880        }
5881      DONE3:
5882        /*tex
5883            If |q| was not a breakpoint at glue and has been reset to |rightskip| then we append
5884            |rightskip| after |q| now?
5885        */
5886        if (glue_break) {
5887            /*tex A rightskip has already been added. Maybe check it! */
5888            rs = q;
5889        } else {
5890            /*tex We add one, even when zero. */
5891            rs = tex_new_glue_node(properties->right_skip ? properties->right_skip : zero_glue, right_skip_glue);
5892            tex_attach_attribute_list_copy(rs, q); /* or next of it? or q */
5893            tex_try_couple_nodes(rs, node_next(q));
5894            tex_couple_nodes(q, rs);
5895            q = rs;
5896        }
5897        /*tex
5898            More preparations.
5899        */
5900        r = node_next(q);
5901        node_next(q) = null;
5902        q = node_next(temp_head);
5903        tex_try_couple_nodes(temp_head, r);
5904        /*tex
5905            Now we prepend the left box. It is part of the content so inside the skips.
5906        */
5907        if (leftbox) {
5908             halfword nxt = node_next(q);
5909             tex_couple_nodes(leftbox, q);
5910             q = leftbox;
5911             if (nxt && (cur_line == cur_list.prev_graf + 1) && (node_type(nxt) == hlist_node) && ! box_list(nxt)) {
5912                 /* what is special about an empty hbox, needs checking */
5913                 q = node_next(q);
5914                 tex_try_couple_nodes(leftbox, node_next(nxt));
5915                 tex_try_couple_nodes(nxt, leftbox);
5916             }
5917        }
5918        /*tex
5919            At this point |q| is the leftmost node; all discardable nodes have been discarded.
5920        */
5921        if (properties->protrude_chars) {
5922            if (line_break_dir == dir_righttoleft && properties->protrude_chars == protrude_chars_advanced) {
5923                halfword p = tex_aux_find_protchar_left(q, 0);
5924                halfword w = tex_char_protrusion(p, left_margin_kern_subtype);
5925                if (w && lmt_packaging_state.last_leftmost_char) {
5926                    halfword k = tex_new_kern_node(-w, left_margin_kern_subtype);
5927                    tex_attach_attribute_list_copy(k, p);
5928                    tex_couple_nodes(k, q);
5929                    q = k;
5930                }
5931            } else {
5932                halfword p = tex_aux_find_protchar_left(q, 0);
5933                halfword w = tex_char_protrusion(p, left_margin_kern_subtype);
5934                if (w && lmt_packaging_state.last_leftmost_char) {
5935                    halfword k = tex_new_kern_node(-w, left_margin_kern_subtype);
5936                    tex_attach_attribute_list_copy(k, p);
5937                    if (node_type(q) == par_node) {
5938                        tex_couple_nodes(k, node_next(q));
5939                        tex_couple_nodes(q, k);
5940                    } else {
5941                        tex_couple_nodes(k, q);
5942                        q = k;
5943                    }
5944                }
5945            }
5946        }
5947        /*tex
5948            Fix a possible mess up.
5949        */
5950        if (node_type(q) == par_node ) {
5951            if (! tex_is_start_of_par_node(q)) {
5952                node_subtype(q) = hmode_par_par_subtype;
5953            }
5954            if (lmt_linebreak_state.inject_after_par) {
5955                tex_couple_nodes(lmt_linebreak_state.inject_after_par, node_next(q));
5956                tex_couple_nodes(q, lmt_linebreak_state.inject_after_par);
5957                lmt_linebreak_state.inject_after_par = null;
5958            }
5959        }
5960        /*tex
5961            Put the |\leftskip| glue at the left and detach this line. Call the packaging
5962            subroutine, setting |just_box| to the justified box. Now|q| points to the hlist that
5963            represents the current line of the paragraph. We need to compute the appropriate line
5964            width, pack the line into a box of this size, and shift the box by the appropriate
5965            amount of indentation. In \LUAMETATEX\ we always add the leftskip.
5966        */
5967        ls = tex_new_glue_node(properties->left_skip, left_skip_glue);
5968        tex_attach_attribute_list_copy(ls, q);
5969        tex_couple_nodes(ls, q);
5970        q = ls;
5971        /*tex
5972            We have these |par| nodes that, when we have callbacks, kind of polute the list. Let's
5973            get rid of them now. We could have done this in previous loops but for the sake of
5974            clearity we do it here. That way we keep the existing code as it is in older engines.
5975            Okay, I might collapse it eventually. This is code that has been prototyped using \LUA.
5976        */
5977        if (cur_line > lmt_linebreak_state.last_special_line) { //  && (! (properties->par_shape && specification_repeat(properties->par_shape)))) {
5978            cur_width = lmt_linebreak_state.second_width;
5979            cur_indent = lmt_linebreak_state.second_indent;
5980        } else if (properties->par_shape) {
5981            if (specification_count(properties->par_shape)) {
5982                cur_indent = tex_get_specification_indent(properties->par_shape, cur_line);
5983                cur_width = tex_get_specification_width(properties->par_shape, cur_line);
5984                cur_indent = swap_parshape_indent(properties->paragraph_dir, cur_indent, cur_width);
5985            } else {
5986                cur_width = lmt_linebreak_state.first_width;
5987                cur_indent = lmt_linebreak_state.first_indent;
5988            }
5989        } else {
5990            cur_width = lmt_linebreak_state.first_width;
5991            cur_indent = lmt_linebreak_state.first_indent;
5992        }
5993        /*tex
5994            When we have a left hang, the width is the (hsize-hang) and there is a shift if hang
5995            applied. The overall linewidth is hsize. When we vbox the result, we get a box with
5996            width hsize.
5997
5998            When we have a right hang, the width is the (hsize-hang) and therefore we end up with
5999            a box that is less that the hsize. When we vbox the result, we get a box with width
6000            hsize minus the hang, so definitely not consistent with the previous case.
6001
6002            In both cases we can consider the hang to be at the edge, simply because the whole lot
6003            gets packaged and then shift gets applied. Although, for practical reasons we could
6004            decide to put it after the left and before the right skips, which actually opens up
6005            some options.
6006
6007            Anyway, after a period of nasty heuristics we can now do a better job because we still
6008            have the information that we started with.
6009
6010        */
6011        first_line = rs && (cur_line == 1) && properties->parinit_left_skip && properties->parinit_right_skip;
6012        last_line = ls && (cur_line + 1 == lmt_linebreak_state.best_line) && properties->parfill_left_skip && properties->parfill_right_skip;
6013        if (first_line) {
6014            halfword n = node_next(properties->parinit_left_skip);
6015            while (n) {
6016                if (n == properties->parinit_right_skip) {
6017                    tex_couple_nodes(node_prev(n), node_next(n));
6018                    tex_couple_nodes(node_prev(rs), n);
6019                    tex_couple_nodes(n, rs);
6020                    break;
6021                } else {
6022                    n = node_next(n);
6023                }
6024            }
6025            if (! n && normalize_line_mode_par) {
6026                /*tex For the moment: */
6027                tex_normal_warning("tex", "right parinit skip is gone");
6028            }
6029        }
6030        if (last_line) {
6031            halfword n = node_prev(properties->parfill_right_skip);
6032            while (n) {
6033                if (n == properties->parfill_left_skip) {
6034                    tex_couple_nodes(node_prev(n), node_next(n));
6035                    tex_couple_nodes(n, node_next(ls));
6036                    tex_couple_nodes(ls, n);
6037                    break;
6038                } else {
6039                    n = node_prev(n);
6040                }
6041            }
6042            if (! n && normalize_line_mode_par) {
6043                /*tex For the moment: */
6044                tex_normal_warning("tex", "left parfill skip is gone");
6045            }
6046            if (first_line && node_next(properties->parfill_right_skip) == properties->parinit_right_skip) {
6047                halfword p = node_prev(properties->parfill_right_skip);
6048                halfword n = node_next(properties->parinit_right_skip);
6049                tex_couple_nodes(p, properties->parinit_right_skip);
6050                tex_couple_nodes(properties->parfill_right_skip, n);
6051                tex_couple_nodes(properties->parinit_right_skip, properties->parfill_right_skip);
6052            }
6053        }
6054        if (lmt_linebreak_state.emergency_left_amount) {
6055            if (ls) {
6056                glue_amount(ls) += lmt_linebreak_state.emergency_left_amount;
6057            } else {
6058                /* error */
6059            }
6060        }
6061        if (lmt_linebreak_state.emergency_right_amount) {
6062            if (rs) {
6063                glue_amount(rs) += lmt_linebreak_state.emergency_right_amount;
6064            } else {
6065                /* error */
6066            }
6067        }
6068        /*tex Some housekeeping. */
6069        lmt_packaging_state.post_adjust_tail = post_adjust_head;
6070        lmt_packaging_state.pre_adjust_tail = pre_adjust_head;
6071        lmt_packaging_state.post_migrate_tail = post_migrate_head;
6072        lmt_packaging_state.pre_migrate_tail = pre_migrate_head;
6073        /*tex   
6074            A bonus feature. However, we still have them in hboxes, som maybe we also 
6075            need to move then there. And also set a flag in the box options in order
6076            to avoid redundant operations. 
6077        */
6078        if (normalize_line_mode_option(flatten_discretionaries_mode)) {
6079            int count = 0;
6080            q = tex_flatten_discretionaries(q, &count, 0); /* there is no need to nest */
6081            cur_disc = null;
6082            if (properties->tracing_paragraphs > 1) {
6083                tex_begin_diagnostic();
6084                tex_print_format("[linebreak: flatten, line %i, count %i]", cur_line, count);
6085                tex_end_diagnostic();
6086            }
6087        }
6088        /*tex Finally we pack the lot. */
6089        shaping = 0;
6090        if (normalize_line_mode_option(normalize_line_mode)) {
6091            halfword head = q;
6092            halfword tail = rs ? rs : head;
6093            halfword lefthang = 0;
6094            halfword righthang = 0;
6095            // we already have the tail somewhere
6096            while (node_next(tail)) {
6097                tail = node_next(tail);
6098            }
6099            if (properties->par_shape) {
6100                int n = specification_count(properties->par_shape);
6101                if (n > 0) {
6102                    if (specification_repeat(properties->par_shape)) {
6103                        n = cur_line;
6104                    } else {
6105                        n = cur_line > n ? n : cur_line;
6106                    }
6107                    lefthang = tex_get_specification_indent(properties->par_shape, n);
6108                    righthang = properties->hsize - lefthang - tex_get_specification_width(properties->par_shape, n);
6109                 // lefthang = swap_parshape_indent(paragraph_dir, lefthang, width); // or so
6110                }
6111            } else if (properties->hang_after) {
6112                if (properties->hang_after > 0 && cur_line > properties->hang_after) {
6113                    if (properties->hang_indent < 0) {
6114                        righthang = -properties->hang_indent;
6115                    }
6116                    if (properties->hang_indent > 0) {
6117                        lefthang = properties->hang_indent;
6118                    }
6119                } else if (properties->hang_after < 0 && cur_line <= -properties->hang_after) {
6120                    if (properties->hang_indent < 0) {
6121                        righthang = -properties->hang_indent;
6122                    }
6123                    if (properties->hang_indent > 0) {
6124                        lefthang = properties->hang_indent;
6125                    }
6126                }
6127            }
6128            shaping = (lefthang || righthang);
6129            lmt_linebreak_state.just_box = tex_hpack(head, cur_width, properties->adjust_spacing ? packing_linebreak : packing_exactly, (singleword) properties->paragraph_dir, holding_none_option, box_limit_line);
6130         // attach_attribute_list_copy(linebreak_state.just_box, properties->initial_par);
6131// if (cur_line == 1 && (properties->paragraph_options & par_option_synchronize)) {
6132//     tex_add_box_option(lmt_linebreak_state.just_box, box_option_synchronize);
6133// }
6134            if (node_type(tail) != glue_node || node_subtype(tail) != right_skip_glue) {
6135                halfword rs = tex_new_glue_node((properties->right_skip ? properties->right_skip : zero_glue), right_skip_glue);
6136                tex_attach_attribute_list_copy(rs, tail);
6137                tex_try_couple_nodes(rs, node_next(q));
6138                tex_couple_nodes(tail, rs);
6139                tail = rs;
6140            }
6141            {
6142                halfword lh = tex_new_glue_node(zero_glue, left_hang_skip_glue);
6143                halfword rh = tex_new_glue_node(zero_glue, right_hang_skip_glue);
6144                glue_amount(lh) = lefthang;
6145                glue_amount(rh) = righthang;
6146                tex_attach_attribute_list_copy(lh, head);
6147                tex_attach_attribute_list_copy(rh, tail);
6148                tex_try_couple_nodes(lh, head);
6149                tex_try_couple_nodes(tail, rh);
6150                head = lh;
6151                tail = rh;
6152            }
6153            /*tex
6154                This is kind of special. Instead of using |cur_width| also on an overfull box as well
6155                as shifts, we want \quote {real} dimensions. A disadvantage is that we need to adapt
6156                analyzers that assume this correction not being there (unpack and repack). So we have
6157                a flag to control it.
6158            */
6159            if (normalize_line_mode_option(clip_width_mode)) {
6160                if (lmt_packaging_state.last_overshoot) {
6161                    halfword g = tex_new_glue_node(zero_glue, correction_skip_glue);
6162                    glue_amount(g) =  -lmt_packaging_state.last_overshoot;
6163                    tex_attach_attribute_list_copy(g, rs);
6164                    tex_try_couple_nodes(node_prev(rs), g);
6165                    tex_try_couple_nodes(g, rs);
6166                }
6167                box_width(lmt_linebreak_state.just_box) = properties->hsize;
6168            }
6169            if (paragraph_has_math(state) && normalize_line_mode_option(balance_inline_math_mode)) {
6170                halfword current = head;
6171                int begin_needed = 0;
6172                int end_needed = 0;
6173                int inmath = 0;
6174                while (current && current != tail) {
6175                    if (node_type(current) == math_node) {
6176                        switch (node_subtype(current)) {
6177                            case begin_inline_math:
6178                                inmath = 1;
6179                                math_attr = current;
6180                                break;
6181                            case end_inline_math:
6182                                if (inmath) {
6183                                    inmath = 0;
6184                                } else {
6185                                    begin_needed = 1;
6186                                }
6187                                math_nesting = 0;
6188                                break;
6189                            default:
6190                                /* error */
6191                                break;
6192                        }
6193                    }
6194                    current = node_next(current);
6195                }
6196                if (inmath) {
6197                    end_needed = 1;
6198                    math_nesting = 1;
6199                } else if (math_nesting) {
6200                    if (! last_line) {
6201                        begin_needed = 1;
6202                    }
6203                    end_needed = 1;
6204                }
6205                if (begin_needed || end_needed) {
6206                    halfword first = null;
6207                    halfword last = null;
6208                    tex_get_line_content_range(head, tail, &first, &last);
6209                    if (begin_needed && first) {
6210                        halfword m = tex_new_node(math_node, begin_broken_math);
6211                        tex_attach_attribute_list_copy(m, math_attr);
6212                        tex_try_couple_nodes(m, node_next(first));
6213                        tex_couple_nodes(first, m);
6214                    }
6215                    if (end_needed && last) {
6216                        halfword m = tex_new_node(math_node, end_broken_math);
6217                        tex_attach_attribute_list_copy(m, math_attr);
6218                        tex_try_couple_nodes(node_prev(last), m);
6219                        tex_couple_nodes(m, last);
6220                    }
6221                }
6222            }
6223            box_list(lmt_linebreak_state.just_box) = head;
6224            q = head;
6225            /*tex So only callback when we normalize. */
6226            if (leftbox || rightbox || middlebox) {
6227                halfword linebox = lmt_linebreak_state.just_box;
6228                lmt_local_box_callback(
6229                    linebox, leftbox, rightbox, middlebox, cur_line,
6230                    tex_effective_glue(linebox, properties->left_skip),
6231                    tex_effective_glue(linebox, properties->right_skip),
6232                    lefthang, righthang, cur_indent,
6233                    (first_line && properties->parinit_left_skip) ? tex_effective_glue(linebox, properties->parinit_left_skip) : null,
6234                    (first_line && properties->parinit_right_skip) ? tex_effective_glue(linebox, properties->parinit_right_skip) : null,
6235                    (last_line && properties->parfill_left_skip) ? tex_effective_glue(linebox, properties->parfill_left_skip) : null,
6236                    (last_line && properties->parfill_right_skip) ? tex_effective_glue(linebox, properties->parfill_right_skip) : null,
6237                    lmt_packaging_state.last_overshoot
6238                );
6239            }
6240        } else {
6241            /*tex Here we can have a right skip way to the right due to an overshoot! */
6242            lmt_linebreak_state.just_box = tex_hpack(q, cur_width, properties->adjust_spacing ? packing_linebreak : packing_exactly, (singleword) properties->paragraph_dir, holding_none_option, box_limit_line);
6243         // attach_attribute_list_copy(linebreak_state.just_box, properties->initial_par);
6244            box_shift_amount(lmt_linebreak_state.just_box) = cur_indent;
6245        }
6246        /*tex Call the packaging subroutine, setting |just_box| to the justified box. */
6247        if (has_box_package_state(lmt_linebreak_state.just_box, package_u_leader_found) && ! has_box_package_state(lmt_linebreak_state.just_box, package_u_leader_delayed)) {
6248            tex_flatten_leaders(lmt_linebreak_state.just_box, cur_group, 0, uleader_post_linebreak, 1);
6249        }
6250        node_subtype(lmt_linebreak_state.just_box) = line_list;
6251        if (callback_id) {
6252            tex_aux_line_break_callback_line(callback_id, checks, cur_line, cur_p);
6253        }
6254        /*tex Pending content (callback). */
6255        if (node_next(contribute_head)) {
6256            if (! lmt_page_builder_state.output_active) {
6257                lmt_append_line_filter_callback(pre_box_append_line_context, 0);
6258            }
6259        }
6260        /* Pre-adjust content (no callback). */
6261        if (pre_adjust_head != lmt_packaging_state.pre_adjust_tail) {
6262            tex_inject_adjust_list(pre_adjust_head, 1, lmt_linebreak_state.just_box, properties);
6263        }
6264        lmt_packaging_state.pre_adjust_tail = null;
6265        /* Pre-migrate content (callback). */
6266        if (pre_migrate_head != lmt_packaging_state.pre_migrate_tail) {
6267            tex_append_list(pre_migrate_head, lmt_packaging_state.pre_migrate_tail);
6268            if (! lmt_page_builder_state.output_active) {
6269                lmt_append_line_filter_callback(pre_migrate_append_line_context, 0);
6270            }
6271        }
6272        lmt_packaging_state.pre_migrate_tail = null;
6273        if (cur_line == 1 && lmt_linebreak_state.best_line == 2 && properties->single_line_penalty) {
6274        // if (cur_line == 1 && lmt_linebreak_state.best_line == 2 && single_line_penalty_par) {
6275            halfword r = tex_new_penalty_node(properties->single_line_penalty, single_line_penalty_subtype);
6276        //  halfword r = tex_new_penalty_node(single_line_penalty_par, single_line_penalty_subtype);
6277            tex_couple_nodes(cur_list.tail, r);
6278            cur_list.tail = r;
6279        }
6280        /* Line content (callback). */
6281        tex_append_to_vlist(lmt_linebreak_state.just_box, lua_key_index(post_linebreak), properties);
6282        if (! lmt_page_builder_state.output_active) {
6283            /* Here we could use the par specific baselineskip and lineskip. */
6284            lmt_append_line_filter_callback(box_append_line_context, 0);
6285        }
6286        /* Post-migrate content (callback). */
6287        if (post_migrate_head != lmt_packaging_state.post_migrate_tail) {
6288            tex_append_list(post_migrate_head, lmt_packaging_state.post_migrate_tail);
6289            if (! lmt_page_builder_state.output_active) {
6290                lmt_append_line_filter_callback(post_migrate_append_line_context, 0);
6291            }
6292        }
6293        lmt_packaging_state.post_migrate_tail = null;
6294        /* Post-adjust content (callback). */
6295        if (post_adjust_head != lmt_packaging_state.post_adjust_tail) {
6296            tex_inject_adjust_list(post_adjust_head, 1, null, properties);
6297        }
6298        if (lmt_packaging_state.except) {
6299            box_exdepth(lmt_linebreak_state.just_box) = lmt_packaging_state.except;
6300        }
6301        lmt_packaging_state.post_adjust_tail = null;
6302        lmt_packaging_state.except = 0;
6303        /*tex
6304            Append the new box to the current vertical list, followed by the list of special nodes
6305            taken out of the box by the packager. Append a penalty node, if a nonzero penalty is
6306            appropriate. Penalties between the lines of a paragraph come from club and widow lines,
6307            from the |inter_line_penalty| parameter, and from lines that end at discretionary breaks.
6308            Breaking between lines of a two-line paragraph gets both club-line and widow-line
6309            penalties. The local variable |pen| will be set to the sum of all relevant penalties for
6310            the current line, except that the final line is never penalized.
6311        */
6312        if (cur_line + 1 != lmt_linebreak_state.best_line) {
6313            /*tex
6314                When we end up here we have multiple lines so we need to add penalties between them
6315                according to (several) specifications.
6316            */
6317            long long pen = 0;
6318            long long nep = 0;
6319            halfword spm = properties->shaping_penalties_mode;
6320            halfword option = 0;
6321            halfword penclub = 0;
6322            halfword penwidow = 0;
6323            halfword nepclub = 0;
6324            halfword nepwidow = 0;
6325            int largest = 0;
6326            if (! spm) {
6327                shaping = 0;
6328            }
6329            if (tracing_penalties_par > 1) {
6330                tex_begin_diagnostic();
6331                tex_print_format("[linebreak: penalty, line %i, best line %i, prevgraf %i, mode %x (i=%i c=%i w=%i b=%i)]",
6332                    cur_line, lmt_linebreak_state.best_line, cur_list.prev_graf, spm,
6333                    is_shaping_penalties_mode(spm, inter_line_penalty_shaping),
6334                    is_shaping_penalties_mode(spm, club_penalty_shaping),
6335                    is_shaping_penalties_mode(spm, widow_penalty_shaping),
6336                    is_shaping_penalties_mode(spm, broken_penalty_shaping)
6337                );
6338                tex_end_diagnostic();
6339            }
6340            if (! (shaping && is_shaping_penalties_mode(spm, inter_line_penalty_shaping))) {
6341                halfword penalty = 0;
6342                halfword index = 0;
6343                if (passive_interline_penalty(cur_p)) {
6344                    penalty = passive_interline_penalty(cur_p);
6345                } else {
6346                    halfword specification = properties->inter_line_penalties;
6347                    if (specification) {
6348                        index = cur_line;
6349                        penalty = tex_get_specification_penalty(specification, index);
6350                    } else {
6351                        penalty = properties->inter_line_penalty;
6352                    }
6353                }
6354                if (penalty) {
6355                    pen += penalty;
6356                    nep += penalty;
6357                    tex_aux_trace_penalty("interline", cur_line, index, penalty, pen);
6358                }
6359            }
6360            if (! (shaping && is_shaping_penalties_mode(spm, club_penalty_shaping))) {
6361                halfword penalty = 0;
6362                halfword nepalty = 0;
6363                halfword specification = properties->club_penalties;
6364                halfword index = 0;
6365                if (specification) {
6366                    index = cur_line - cur_list.prev_graf;
6367                    penalty = tex_get_specification_penalty(specification, index);
6368                    if (specification_double(specification)) {
6369                        nepalty = tex_get_specification_nepalty(specification, index);
6370                        option |= penalty_option_double;
6371                    } else {
6372                        nepalty = penalty;
6373                    }
6374                    if (specification_largest(specification)) {
6375                        penclub = penalty;
6376                        nepclub = nepalty;
6377                        largest |= 1;
6378                    }
6379                } else if (cur_line == cur_list.prev_graf + 1) {
6380                    /*tex prevgraf */
6381                    penalty = properties->club_penalty;
6382                    nepalty = penalty;
6383                }
6384                if (nepalty) {
6385                    nep += nepalty;
6386                    tex_aux_trace_penalty("club l", cur_line, index, nepalty, nep);
6387                    if (nep >= infinite_penalty) {
6388                        option |= penalty_option_clubbed;
6389                    }
6390                    option |= penalty_option_club;
6391                }
6392                if (penalty) {
6393                    pen += penalty;
6394                    tex_aux_trace_penalty("club r", cur_line, index, penalty, pen);
6395                    if (pen >= infinite_penalty) {
6396                        option |= penalty_option_clubbed;
6397                    }
6398                    option |= penalty_option_club;
6399                }
6400            }
6401            if (! (shaping && is_shaping_penalties_mode(spm, widow_penalty_shaping))) {
6402                halfword penalty = 0;
6403                halfword nepalty = 0;
6404                halfword specification = properties->par_context == math_par_context ? properties->display_widow_penalties : properties->widow_penalties;
6405                int index = 0;
6406                if (specification) {
6407                    index = lmt_linebreak_state.best_line - cur_line - 1;
6408                    penalty = tex_get_specification_penalty(specification, index);
6409                    if (specification_double(specification)) {
6410                        nepalty = tex_get_specification_nepalty(specification, index);
6411                        option |= penalty_option_double;
6412                    } else {
6413                        nepalty = penalty;
6414                    }
6415                    if (specification_largest(specification)) {
6416                        penwidow = penalty;
6417                        nepwidow = nepalty;
6418                        largest |= 2;
6419                    }
6420                    } else if (cur_line + 2 == lmt_linebreak_state.best_line) {
6421                    penalty = properties->par_context == math_par_context ? properties->display_widow_penalty : properties->widow_penalty;
6422                    nepalty = penalty;
6423                }
6424                if (nepalty) {
6425                    nep += nepalty;
6426                    tex_aux_trace_penalty("widow l", cur_line, index, nepalty, nep);
6427                    if (nep >= infinite_penalty) {
6428                        option |= penalty_option_widowed;
6429                    }
6430                    option |= penalty_option_widow;
6431                }
6432                if (penalty) {
6433                    pen += penalty;
6434                    tex_aux_trace_penalty("widow r", cur_line, index, penalty, pen);
6435                    if (pen >= infinite_penalty) {
6436                        option |= penalty_option_widowed;
6437                    }
6438                    option |= penalty_option_widow;
6439                }
6440            }
6441            if (disc_break && ! (shaping && is_shaping_penalties_mode(spm, broken_penalty_shaping))) {
6442                halfword penalty = 0;
6443                halfword nepalty = 0;
6444                halfword index = 0;
6445                if (passive_broken_penalty(cur_p)) {
6446                    penalty = passive_broken_penalty(cur_p);
6447                    nepalty = penalty;
6448                } else {
6449                    halfword specification = properties->broken_penalties;
6450                    if (specification) {
6451                        index = 1;
6452                        penalty = tex_get_specification_penalty(specification, index);
6453                        if (specification_double(specification)) {
6454                            nepalty = tex_get_specification_nepalty(specification, index);
6455                            option |= penalty_option_double;
6456                        } else {
6457                            nepalty = penalty;
6458                        }
6459                    } else {
6460                        penalty = properties->broken_penalty;
6461                        nepalty = penalty;
6462                    }
6463                }
6464                if (nepalty) {
6465                    nep += nepalty;
6466                    tex_aux_trace_penalty("broken l", cur_line, index, nepalty, nep);
6467                    option |= penalty_option_broken;
6468                }
6469                if (penalty) {
6470                    pen += penalty;
6471                    tex_aux_trace_penalty("broken r", cur_line, index, penalty, pen);
6472                    option |= penalty_option_broken;
6473                }
6474            }
6475            if (shaping && ! pen) {
6476                pen = properties->shaping_penalty;
6477                nep = pen;
6478                if (pen) {
6479                    tex_aux_trace_penalty("shaping", cur_line, 0, pen, pen);
6480                    option |= penalty_option_shaping;
6481                }
6482            }
6483            if (pen || nep) {
6484                /* experiment: both largest flags have to be set */
6485                if (largest == 3) {
6486                    if (penclub > penwidow) {
6487                        pen -= penwidow;
6488                        tex_aux_trace_penalty("discard widow r", cur_line, 0, -penwidow, pen);
6489                    } else if (penclub < penwidow) {
6490                        pen -= penclub;
6491                        tex_aux_trace_penalty("discard club r", cur_line, 0, -penclub, pen);
6492                    }
6493                    if (nepclub > nepwidow) {
6494                        nep -= nepwidow;
6495                        tex_aux_trace_penalty("discard window l", cur_line, 0, -nepwidow, nep);
6496                    } else if (nepclub < nepwidow) {
6497                        nep -= nepclub;
6498                        tex_aux_trace_penalty("discard club l", cur_line, 0, -nepclub, nep);
6499                    }
6500                }
6501                /* */
6502                r = tex_new_penalty_node(checked_penalty(pen), linebreak_penalty_subtype);
6503                penalty_tnuoma(r) = checked_penalty(nep);
6504                tex_add_penalty_option(r, option);
6505                tex_couple_nodes(cur_list.tail, r);
6506                cur_list.tail = r;
6507            }
6508        }
6509        /*tex
6510            Append a penalty node, if a nonzero penalty is appropriate. Justify the line ending at
6511            breakpoint |cur_p|, and append it to the current vertical list, together with associated
6512            penalties and other insertions.
6513        */
6514        ++cur_line;
6515        cur_p = passive_next_break(cur_p);
6516        if (cur_p && ! post_disc_break) {
6517            /*tex
6518                Prune unwanted nodes at the beginning of the next line. Glue and penalty and kern
6519                and math nodes are deleted at the beginning of a line, except in the anomalous case
6520                that the node to be deleted is actually one of the chosen breakpoints. Otherwise
6521                the pruning done here is designed to match the lookahead computation in
6522                |try_break|, where the |break_width| values are computed for non-discretionary
6523                breakpoints.
6524            */
6525            r = temp_head;
6526            /*tex
6527                Normally we have a matching math open and math close node but when we cross a line
6528                the open node is removed, including any glue or penalties following it. This is
6529                however not that nice for callbacks that rely on symmetry. Of course this only
6530                counts for one liners, as we can still have only a begin or end node on a line. The
6531                end_of_math lua helper is made robust against this although there you should be
6532                aware of the fact that one can end up in the middle of math in callbacks that don't
6533                work on whole paragraphs, but at least this branch makes sure that some proper
6534                analysis is possible. (todo: check if math glyphs have the subtype marked done).
6535            */
6536            /*tex Suboptimal but not critical. Todo.*/
6537            while (1) {
6538                q = node_next(r);
6539                if (node_type(q) == math_node) {
6540                    if (node_subtype(q) == begin_inline_math) {
6541                        /*tex We keep it for tracing. */
6542                        break;
6543                    } else {
6544                        /*tex begin mathskip code */
6545                        math_surround(q) = 0 ;
6546                        tex_reset_math_glue_to_zero(q);
6547                        /*tex end mathskip code */
6548                    }
6549                }
6550                if (q == passive_cur_break(cur_p)) {
6551                    break;
6552                } else if (node_type(q) == glyph_node) {
6553                    break;
6554                } else if (node_type(q) == glue_node && (node_subtype(q) == par_fill_left_skip_glue || node_subtype(q) == par_init_left_skip_glue)) {
6555                    /*tex Keep it. Can be tricky after a |\break| with no follow up (loops). */
6556                    break;
6557                } else if (node_type(q) == par_node && node_subtype(q) == local_box_par_subtype) {
6558                    /*tex Weird, in the middle somewhere .. these local penalties do this. */
6559                    break; /* if not we leak, so maybe this needs more testing */
6560                } else if (non_discardable(q)) {
6561                    break;
6562                } else if (node_type(q) == kern_node && ! (node_subtype(q) == explicit_kern_subtype || node_subtype(q) == italic_kern_subtype)) {
6563                    break;
6564                }
6565                r = q;
6566            }
6567            if (r != temp_head) {
6568                node_next(r) = null;
6569                tex_flush_node_list(node_next(temp_head));
6570                tex_try_couple_nodes(temp_head, q);
6571            }
6572        }
6573        if (cur_disc) {
6574            tex_try_couple_nodes(node_prev(cur_disc),node_next(cur_disc));
6575            tex_flush_node(cur_disc);
6576        }
6577        /* We can clean up the par nodes. */
6578    } while (cur_p);
6579    if (cur_line != lmt_linebreak_state.best_line) {
6580        tex_begin_diagnostic();
6581        tex_print_format("[linebreak: dubious situation, current line %i is not best line %i]", cur_line, lmt_linebreak_state.best_line);
6582        tex_end_diagnostic();
6583    //  tex_confusion("line breaking 1");
6584    } else if (node_next(temp_head)) {
6585        tex_confusion("line breaking 2");
6586    }
6587    /*tex |prevgraf| etc */
6588    cur_list.prev_graf = lmt_linebreak_state.best_line - 1;
6589    cur_list.direction_stack = lmt_linebreak_state.dir_ptr;
6590    lmt_linebreak_state.dir_ptr = null;
6591}
6592
6593halfword tex_wipe_margin_kerns(halfword head)
6594{
6595    /*tex We assume that head is a temp node or at least can be skipped (for now). */
6596    halfword tail = head;
6597    while (1) {
6598        halfword next = node_next(tail);
6599        if (next) {
6600            if (node_type(next) == kern_node && (node_subtype(next) == left_margin_kern_subtype || node_subtype(next) == right_margin_kern_subtype)) {
6601                tex_try_couple_nodes(tail, node_next(next));
6602                tex_flush_node(next);
6603            } else {
6604                tail = next;
6605            }
6606        } else {
6607            return tail;
6608        }
6609    }
6610}
6611