1
2
3\environment luametatexstyle
4
5\startcomponent luametatexbuilding
6
7\startchapter[reference=building,title={Boxes, paragraphs and pages}]
8
9\startsection[title={Introduction}]
10
11\topicindex {building}
12\topicindex {pages}
13\topicindex {paragraphs}
14\topicindex {marks}
15\topicindex {inserts}
16
17There are some enhancements that relate to the way paragraphs and pages are
18built. In this chapter we will cover those. There can be a bit of overlap with
19other chapters. These enhancements are still somewhat experimental.
20
21\stopsection
22
23\startsection[title=Directions]
24
25\topicindex {\OMEGA}
26\topicindex {\ALEPH}
27\topicindex {directions}
28
29\startsubsection[title={Two directions}]
30
31The directional model in \LUAMETATEX\ is a simplified version the the model used
32in \LUATEX. In fact, not much is happening at all: we only register a change in
33direction.
34
35\stopsubsection
36
37\startsubsection[title={How it works}]
38
39The approach is that we try to make node lists balanced but also try to avoid
40some side effects. What happens is quite intuitive if we forget about spaces
41(turned into glue) but even there what happens makes sense if you look at it in
42detail. However that logic makes ingroup switching kind of useless when no
43properly nested grouping is used: switching from right to left several times
44nested, results in spacing ending up after each other due to nested mirroring. Of
45course a sane macro package will manage this for the user but here we are
46discussing the low level injection of directional information.
47
48This is what happens:
49
50\starttyping
51\textdirection 1 nur {\textdirection 0 run \textdirection 1 NUR} nur
52\stoptyping
53
54This becomes stepwise:
55
56\startnarrower
57\starttyping
58injected: [push 1]nur {[push 0]run [push 1]NUR} nur
59balanced: [push 1]nur {[push 0]run [pop 0][push 1]NUR[pop 1]} nur[pop 0]
60result : run {RUNrun } run
61\stoptyping
62\stopnarrower
63
64And this:
65
66\starttyping
67\textdirection 1 nur {nur \textdirection 0 run \textdirection 1 NUR} nur
68\stoptyping
69
70becomes:
71
72\startnarrower
73\starttyping
74injected: [TRT]nur {nur [TLT]run [TRT]NUR} nur
75balanced: [TRT]nur {nur [TLT]run [TLT][TRT]NUR[TRT]} nur[TRT]
76result : run {run RUNrun } run
77\stoptyping
78\stopnarrower
79
80Now, in the following examples watch where we put the braces:
81
82\startbuffer
83\textdirection 1 nur {{\textdirection 0 run} {\textdirection 1 NUR}} nur
84\stopbuffer
85
86\typebuffer
87
88This becomes:
89
90\startnarrower
91\getbuffer
92\stopnarrower
93
94Compare this to:
95
96\startbuffer
97\textdirection 1 nur {{\textdirection 0 run }{\textdirection 1 NUR}} nur
98\stopbuffer
99
100\typebuffer
101
102Which renders as:
103
104\startnarrower
105\getbuffer
106\stopnarrower
107
108So how do we deal with the next?
109
110\startbuffer
111\def\ltr{\textdirection 0\relax}
112\def\rtl{\textdirection 1\relax}
113
114run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}
115run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}
116\stopbuffer
117
118\typebuffer
119
120It gets typeset as:
121
122\startnarrower
123\startlines
124\getbuffer
125\stoplines
126\stopnarrower
127
128We could define the two helpers to look back, pick up a skip, remove it and
129inject it after the dir node. But that way we loose the subtype information that
130for some applications can be handy to be kept asis. This is why we now have a
131variant of \prm {textdirection} which injects the balanced node before the skip.
132Instead of the previous definition we can use:
133
134\startbuffer[def]
135\def\ltr{\linedirection 0\relax}
136\def\rtl{\linedirection 1\relax}
137\stopbuffer
138
139\typebuffer[def]
140
141and this time:
142
143\startbuffer[txt]
144run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}
145run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}
146\stopbuffer
147
148\typebuffer[txt]
149
150comes out as a properly spaced:
151
152\startnarrower
153\startlines
154\getbuffer[def,txt]
155\stoplines
156\stopnarrower
157
158Anything more complex that this, like combination of skips and penalties, or
159kerns, should be handled in the input or macro package because there is no way we
160can predict the expected behaviour. In fact, the \prm {linedirection} is just a
161convenience extra which could also have been implemented using node list parsing.
162
163Directions are complicated by the fact that they often need to work over groups
164so a separate grouping related stack is used. A side effect is that there can be
165paragraphs with only a local par node followed by direction synchronization
166nodes. Paragraphs like that are seen as empty paragraphs and therefore ignored.
167Because \prm {noindent} doesnt inject anything but a \prm {indent} injects
168an box, paragraphs with only an indent and directions are handles and paragraphs
169with content. When indentation is normalized a paragraph with an indentation
170skip is seen as content.
171
172\stopsubsection
173
174\startsubsection[title={Normalizing lines}]
175
176The original \TEX\ machinery was never meant to be opened up. As a consequence a
177constructed line can have different layouts. There can be left andor right
178skips and hanging indentation or parshape can result in a shift and adapted
179width. In \LUATEX\ glue got subtypes so we can recognize the left, right and
180parfill skips, but still there is no hundred percent certainty about the shape.
181
182In \LUAMETATEX\ lines can be normalized. This is optional because we want to
183preserve the original (for comparison) and is controlled by \prm
184{normalizelinemode}. That variable actually drives some more. An earlier version
185provided a few more granular options (for instance: does a leftskip comes before
186or after a left hanging indentation) but in the end that was dropped. Because
187this normalization only is seen at the \LUA\ end there is no need to go into much
188detail here.
189
190At this moment a line has this pattern: left parfill, left hang, left skip,
191indentation, content, right hang, right skip, right parfill. Of course the
192indentation and fill skips are not present in every line.
193
194Control over normalization happens via the mentioned mode variable and here is
195what the engine provides right now. We use a bitmap:
196
197\starttabulate[ll]
198\DB value \BC reported \NC \NR
199\TB
200\NC \type{0x0001} \NC normalize line as described above \NC \NR
201\NC \type{0x0002} \NC use a skip for parindent instead of a box \NC \NR
202\NC \type{0x0004} \NC swap hangindent in l2r mode \NC \NR
203\NC \type{0x0008} \NC swap parshape in l2r mode \NC \NR
204\NC \type{0x0010} \NC put breaks after dir in l2r mode \NC \NR
205\NC \type{0x0020} \NC remove margin kerns (\PDFTEX\ leftover) \NC \NR
206\NC \type{0x0040} \NC if needed clip width and use correction kern \NC \NR
207\LL
208\stoptabulate
209
210Setting the bit enables the related normalization. More features might be added
211in future releases.
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312\stopsubsection
313
314\startsubsection[title=Orientations]
315
316\topicindex {boxesorientations}
317
318As mentioned, the difference with \LUATEX\ is that we only have numeric
319directions and that there are only two: lefttoright (\type {0}) and
320righttoleft (\type {1}). The direction of a box is set with \type
321{direction}.
322
323In addition to that boxes can now have an \type {orientation} keyword followed by
324optional \type {xoffset} andor \type {yoffset} keywords. The offsets dont
325have consequences for the dimensions. The alternatives \type {xmove} and \type
326{ymove} on the contrary are reflected in the dimensions. Just play with them. The
327offsets and moves only are accepted when there is also an orientation, so no time
328is wasted on testing for these rarely used keywords. There are related primitives
329\type {\box...} that set these properties.
330
331As these are experimental it will not be explained here (yet). They are covered
332in the descriptions of the development of \LUAMETATEX: articles andor
333documents in the \CONTEXT\ distribution. For now it is enough to know that the
334orientation can be up, down, left or right (rotated) and that it has some
335anchoring variants. Combined with the offsets this permits macro writers to
336provide solutions for topdown and bottomup writing directions, something
337that is rather macro package specific and used for scripts that need
338manipulations anyway. The \quote {old} vertical directions were never okay and
339therefore not used.
340
341There are a couple of properties in boxes that you can set and query but that
342only really take effect when the backend supports them. When usage on \CONTEXT\
343shows that ist okay, they will become official, so we just mention them: \prm
344{boxdirection}, \prm {boxattribute}, \prm {boxorientation}, \prm {boxxoffset},
345\prm {boxyoffset}, \prm {boxxmove}, \prm {boxymove} and \prm {boxtotal}.
346
347{\em This is still somewhat experimental and will be documented in more detail
348when Ive used it more in \CONTEXT\ and the specification is frozen. This might
349take some time (and user input).}
350
351\stopsubsection
352
353\stopsection
354
355\startsection[title={Boxes, rules and leaders}]
356
357\startsubsection[title={\prm {outputbox}}]
358
359\topicindex {output}
360
361This integer parameter allows you to alter the number of the box that will be
362used to store the page sent to the output routine. Its default value is 255, and
363the acceptable range is from 0 to 65535.
364
365\startsyntax
366\outputbox = 12345
367\stopsyntax
368
369\stopsubsection
370
371\startsubsection[title={\prm {hrule}, \prm {vrule}, \prm {srule}, \prm {nohrule}, \prm {novrule},
372\prm {virtualhrule} and \prm {virtualvrule}}]
373
374\topicindex {rules}
375
376Both rule drawing commands take an optional \type {xoffset} and \type {yoffset}
377parameter. The displacement is virtual and not taken into account when the
378dimensions are calculated. A rule is specified in the usual way:
379
380\obeydepth
381
382\startbuffer
383\blue \vrule
384 height 2ex depth 1ex width 10cm
385\relax
386\stopbuffer
387
388\startlinecorrection
389\getbuffer
390\stoplinecorrection
391
392There is however a catch. The keyword scanners in \LUAMETATEX\ are implemented
393slightly different. When \TEX\ scans a keyword it will (case insensitive) scan
394for a whole keyword. So, it scans for \type {height} and when it doesnt find it
395it will scan for \type {depth} etc. When it does find a keyword in this case it
396expects a dimension next. When that criterion is not met it will issue an error
397message.
398
399In order to avoid look ahead failures like that it is recommended to end the
400specification with \type {\relax}. A glue specification is an other example where
401a \type {\relax} makes sense when look ahead issues are expected and actually
402there in traditional scanning the order of keywords can also matter. In any case,
403when no valid keyword is seen the characters scanned so far are pushed back in
404the input.
405
406The main reason for using an adapted scanner is that we always permit repetition
407(consistency) and accept an arbitrary order. Because we have more keywords to
408process the scanner quits at a partial failure. This prevents some push back and
409also gives an earlier warning. Interesting is that some \CONTEXT\ users ran into
410error messages due to a missing \type {\relax} and found out that their style has
411a potential flaw with respect to look ahead. One can be lucky for years.
412
413Back to rules, there are some extra keywords, two deal with an offset, and four
414provide margins. The margins are a bit special because \type {left} and \type
415{top} are the same as are \type {right} and \type {bottom}. They influence the
416edges and these depend on it being a horizontal or vertical rule.
417
418\obeydepth
419
420\startbuffer
421\blue \vrule
422 height 2.0ex depth 1.0ex width 10cm
423\relax
424\white \vrule
425 height 1.0ex depth 0.5ex width 9cm
426 xoffset 9.5cm yoffset .25ex
427\relax
428\blue \vrule
429 height .5ex depth 0.25ex width 8cm
430 xoffset 18cm yoffset .375ex top 1pt
431\relax
432\stopbuffer
433
434\startlinecorrection
435\getbuffer
436\stoplinecorrection
437
438Two new primitives were introduced: \prm {nohrule} and \prm {novrule}. These can
439be used to reserve space. This is often more efficient than creating an empty box
440with fake dimensions. Of course this assumes that the backend implements them
441being invisible but still taking space.
442
443An \prm {srule} is sort of special. In text mode it is just a convenience (we
444could do without it for ages) but in math mode it comes in handy when we want to
445enforce consistency. \footnote {In \CONTEXT\ there is a lot of focus on
446consistent vertical spacing, something that doesnt naturally comes with \TEX\
447(you have to pay attention!) and therefore for decades now you can find plenty of
448documents with bad spacing of a nature that has seem to have become accepted as
449quality. This probably makes these \prm {srule}s one of the few primitives that
450actually targets at \CONTEXT.}
451
452As with all rules, the backend will makes rules span the width or height and
453depth of the encapsulating box. An \prm {srule} is just a \prm {vrule} but is set
454up such that it can adapt itself:
455
456\startbuffer
457\hbox to 3cm {x\leaders\hrule\hfil x}
458\hbox{x \vrule width 4cm \relax x}
459\hbox{x \srule width 4cm \relax x}
460\hbox{x \srule font \font char ( width 4cm \relax x}
461\hbox{$x \srule fam \fam char ( width 4cm \relax x$}
462
463
464\stopbuffer
465
466\typebuffer
467
468You can hard code the height and depth or get it from a fontfamilycharacter
469combination. This is especially important in math mode where then can adapt to
470(stylistic) circumstances.
471
472\startlines
473\showboxes\getbuffer
474\stoplines
475
476Because this kind of rules has a dedicated subtype you can intercept it in the backend
477if needed. The two virtual variants are special in the way that they are like normal
478rules but take no space. Can you figure out how to get this?
479
480\startlinecorrection[blank]
481\dontleavehmode \hbox{
482 \hbox{\green before}
483 {\darkblue \virtualvrule width 40pt height 2pt depth 4pt xoffset 20pt\relax}
484 \hbox{\red after}
485}
486\stoplinecorrection
487
488
489
490
491
492
493
494\stopsubsection
495
496\startsubsection[title={\prm {vsplit}, \prm {tsplit} and \prm {dsplit}}]
497
498\topicindex {splitting}
499
500The \prm {vsplit} primitive has to be followed by a specification of the required
501height. As alternative for the \type {to} keyword you can use \type {upto} to get
502a split of the given size but result has the natural dimensions then.
503
504\starttyping
505\vsplit 123 to 10cm
506\vsplit 123 upto 10cm
507\stoptyping
508
509The two alternative primitives return a \prm {vtop} or \prm {dbox} instead of a
510\prm {vbox}. All three accept the \type {attr} keyword as boxes do.
511
512\stopsubsection
513
514\startsubsection[title={\prm {boxxoffset}, \prm {boxyoffset}, \prm {boxxmove}, \prm {boxymove},
515\prm{boxorientation} and \prm{boxgeometry}}]
516
517This repertoire of primitives can be used to do relative positioning. The offsets
518are virtual while the moves adapt the dimensions. The orientation bitset can be
519used to rotate the box over 90, 180 and 270 degrees. It also influences the
520corner, midpoint or baseline.
521
522{\em There is information in the \CONTEXT\ low level manuals and in due time I
523will add a few examples here. This feature needs support in the backend when used
524(as in \CONTEXT) so it might influence performance.}
525
526\stopsubsection
527
528\startsubsection[title={\prm {boxtotal}}]
529
530The \prm {boxtotal} primitive returns the sum of the height and depth and is less
531useful as setter: it just sets the height and depth to half of the given value.
532
533\stopsubsection
534
535\startsubsection[title={\prm {boxshift}}]
536
537In traditional \TEX\ a box has height, depth, width and a shift where the later
538relates to \prm {raise}, \prm {lower}, \prm {moveleft} and \prm {moveright}. This
539primitive can be used to query and set this property.
540
541\startbuffer
542\setbox0\hbox{test test test}
543\setbox2\hbox{test test test} \boxshift2 10pt
544\ruledhbox{x \raise10pt\box0\ x}
545\ruledhbox{x \box2\ x}
546\stopbuffer
547
548\typebuffer
549
550\stopsubsection
551
552\startsubsection[title={\prm {boxanchor}, \prm {boxanchors}, \prm {boxsource} and \prm {boxtarget}}]
553
554{\em These are experimental.}
555
556\stopsubsection
557
558\startsubsection[title={\prm {boxfreeze}, \prm {boxadapt} and \prm {boxrepack}}]
559
560\topicindex {boxespostprocessing}
561
562This operation will freeze the glue in the given box, something that normally is
563delayed and delegated to the backend.
564
565\startbuffer
566\setbox 0 \hbox to 5cm {\hss test}
567\setbox 2 \hbox to 5cm {\hss test}
568\boxfreeze 2 0
569\ruledhbox{\unhbox 0}
570\ruledhbox{\unhbox 2}
571\stopbuffer
572
573\typebuffer
574
575The second parameter to \prm {boxfreeze} determines recursion. Here we just
576freeze the outer level:
577
578\getbuffer
579
580Repacking will take the content of an existing box and add or subtract from it:
581
582\startbuffer
583\setbox 0 \hbox {test test test}
584\setbox 2 \hbox {\red test test test} \boxrepack0 .2em
585\setbox 4 \hbox {\green test test test} \boxrepack0 .2em
586\ruledhbox{\box0} \vskip\lineheight
587\ruledhbox{\box0} \vskip\lineheight
588\ruledhbox{\box0}
589\stopbuffer
590
591\typebuffer
592
593\getbuffer
594
595We can use this primitive to check the natural dimensions:
596
597\startbuffer
598\setbox 0 \hbox spread 10pt {test test test}
599\ruledhbox{\box0} (\the\boxrepack0,\the\wd0)
600\stopbuffer
601
602\typebuffer
603
604\getbuffer
605
606Adapting will recalculate the dimensions with a scale factor for the glue:
607
608\startbuffer
609\setbox 0 \hbox {test test test}
610\setbox 2 \hbox {\red test test test} \boxadapt 0 200
611\setbox 4 \hbox {\blue test test test} \boxadapt 0 200
612\ruledhbox{\box0} \vskip\lineheight
613\ruledhbox{\box0} \vskip\lineheight
614\ruledhbox{\box0}
615\stopbuffer
616
617\typebuffer
618
619\getbuffer
620
621\stopsubsection
622
623\startsubsection[title={\prm {boxvadjust}}]
624
625This primitive binds a \prm {vadjust} to a box and therefore also accepts the
626\type {pre} and \type {post} keywords which means that you can prepend and append
627as the box itself gets flushed.
628
629\stopsubsection
630
631\startsubsection[title={Overshooting dimensions}]
632
633\topicindex {boxesoverfull}
634
635The \prm {overshoot} primitive reports the most recent amount of overshoot when a
636box is packages. It relates to overfull boxes and the then set \prm {badness} of
6371000000.
638
639\startbuffer
640\hbox to 2cm {does it fit} \the\overshoot
641\hbox to 2cm {does it fit in here} \the\overshoot
642\hbox to 2cm {how much does fit in here} \the\overshoot
643\stopbuffer
644
645\typebuffer
646
647This global state variables reports a dimension:
648
649\startlines
650\getbuffer
651\stoplines
652
653\stopsubsection
654
655\startsubsection[title={Images and reused box objects},reference=sec:imagesandforms]
656
657\topicindex {images}
658
659In original \TEX\ image support is dealt with via specials. Its not a native
660feature of the engine. All that \TEX\ cares about is dimensions, so in practice
661that meant: using a box with known dimensions that wraps a special that instructs
662the backend to include an image. The wrapping is needed because a special itself
663is a whatsit and as such has no dimensions.
664
665In \PDFTEX\ a special whatsit for images was introduced and that one {\em has}
666dimensions. As a consequence, in several places where the engine deals with the
667dimensions of nodes, it now has to check the details of whatsits. By inheriting
668code from \PDFTEX, the \LUATEX\ engine also had that property. However, at some
669point this approach was abandoned and a more natural trick was used: images (and
670box resources) became a special kind of rules, and as rules already have
671dimensions, the code could be simplified.
672
673When direction nodes and (formerly local) par nodes also became first class
674nodes, whatsits again became just that: nodes representing whatever you want, but
675without dimensions, and therefore they could again be ignored when dimensions
676mattered. And, because images were disguised as rules, as mentioned, their
677dimensions automatically were taken into account. This separation between front
678and backend cleaned up the code base already quite a bit.
679
680In \LUAMETATEX\ we still have the image specific subtypes for rules, but the
681engine never looks at subtypes of rules. That was up to the backend. This means
682that image support is not present in \LUAMETATEX. When an image specification was
683parsed the special properties, like the filename, or additional attributes, were
684stored in the backend and all that \LUATEX\ does is registering a reference to an
685images specification in the rule node. But, having no backend means nothing is
686stored, which in turn would make the image inclusion primitives kind of weird.
687
688Therefore you need to realize that contrary to \LUATEX, {\em in \LUAMETATEX\
689support for images and box reuse is not built in}! However, we can assume that
690an implementation uses rules in a similar fashion as \LUATEX\ does. So, you can
691still consider images and box reuse to be core concepts. Here we just mention the
692primitives that \LUATEX\ provides. They are not available in the engine but can
693of course be implemented in \LUA.
694
695\starttabulate[lp]
696\DB command \BC explanation \NC \NR
697\TB
698\NC \tex {saveboxresource} \NC save the box as an object to be included later \NC \NR
699\NC \tex {saveimageresource} \NC save the image as an object to be included later \NC \NR
700\NC \tex {useboxresource} \NC include the saved box object here (by index) \NC \NR
701\NC \tex {useimageresource} \NC include the saved image object here (by index) \NC \NR
702\NC \tex {lastsavedboxresourceindex} \NC the index of the last saved box object \NC \NR
703\NC \tex {lastsavedimageresourceindex} \NC the index of the last saved image object \NC \NR
704\NC \tex {lastsavedimageresourcepages} \NC the number of pages in the last saved image object \NC \NR
705\LL
706\stoptabulate
707
708An implementation probably should accept the usual optional dimension parameters
709for \type {\use...resource} in the same format as for rules. With images, these
710dimensions are then used instead of the ones given to \tex {useimageresource} but
711the original dimensions are not overwritten, so that a \tex {useimageresource}
712without dimensions still provides the image with dimensions defined by \tex
713{saveimageresource}. These optional parameters are not implemented for \tex
714{saveboxresource}.
715
716\starttyping
717\useimageresource width 20mm height 10mm depth 5mm \lastsavedimageresourceindex
718\useboxresource width 20mm height 10mm depth 5mm \lastsavedboxresourceindex
719\stoptyping
720
721Examples or optional entries are \type {attr} and \type {resources} that accept a
722token list, and the \type {type} key. When set to nonzero the \type {Type}
723entry is omitted. A value of 1 or 3 still writes a \type {BBox}, while 2 or 3
724will write a \type {Matrix}. But, as said: this is entirely up to the backend.
725Generic macro packages (like \type {tikz}) can use these assumed primitives so
726one can best provide them. It is probably, for historic reasons, the only more or
727less standardized image inclusion interface one can expect to work in all macro
728packages.
729
730\stopsubsection
731
732\startsubsection[title={\prm {dbox}}]
733
734This primitive is a variant on \prm {vbox} in the sense that when it gets
735appended to a vertical list the height of the topmost line or rule as well as the
736depth of the box are taken into account when interline space is calculated.
737
738\stopsubsection
739
740
741\startsubsection[title={\prm {hpack}, \prm {vpack}, \prm {tpack} and \prm {dpack}}]
742
743\topicindex {packing}
744
745These three primitives are the equivalents of \prm {hbox}, \prm {vbox}, \prm
746{vtop} and \prm {dbox} but they dont trigger the packaging related callbacks.
747Of course one never know if content needs a treatment so using them should be
748done with care. Apart from accepting more keywords (and therefore options) the
749normal box behave the same as before.
750
751\stopsubsection
752
753\startsubsection[title={\prm {vcenter}}]
754
755The \prm {vcenter} builder also works in text mode.
756
757\stopsubsection
758
759\startsubsection[title={\prm {unhpack}, \prm {unvpack}}]
760
761\topicindex {unpacking}
762
763These two are somewhat experimental. They ignore the accumulated pre and
764postmigrated material bound to a box. I needed it for some experiment so the
765functionality might change when I really need it.
766
767\stopsubsection
768
769\startsubsection[title={\prm {gleaders} and \prm {uleaders}},reference=sec:gleaders]
770
771\topicindex {leaders}
772
773This type of leaders is anchored to the origin of the box to be shipped out. So
774they are like normal \prm {leaders} in that they align nicely, except that the
775alignment is based on the {\it largest\/} enclosing box instead of the {\it
776smallest\/}. The \type {g} stresses this global nature. The \prm {uleaders} are
777used for flexible boxes and are discussed elsewhere.
778
779\stopsubsection
780
781\stopsection
782
783\startsection[title={Paragraphs}]
784
785\startsubsection[title=Freezing]
786
787In \LUAMETATEX\ we store quite some properties with a paragraph. Where in traditional
788\TEX\ the properties that are set when the paragraph broken into lines are used, here
789we can freeze them.
790
791{\em At some point this section will describe \prm {autoparagraphmode}, \prm
792{everybeforepar}, \prm {snapshotpar}, \prm {wrapuppar}, etc. For the moment the
793manuals that come with \CONTEXT\ have to do.}
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849\stopsubsection
850
851\startsubsection[title=Penalties]
852
853In addition to the penalties introduced in \ETEX, we also provide \prm
854{orphanpenalty} and \prm {orphanpenalties}. When were shaping a paragraph
855an additional \prm {shapingpenalty} can be injected. This penalty gets
856injected instead of the usual penalties when the following bits are set in
857\prm {shapingpenaltiesmode}:
858
859\starttabulate[llp]
860\DB value \BC ignored \NC \NR
861\TB
862\NC \type {0x01} \NC interlinepenalty \NC \NR
863\NC \type {0x02} \NC widowpenalty \NC \NR
864\NC \type {0x04} \NC clubpenalty \NC \NR
865\NC \type {0x08} \NC brokenpenalty \NC \NR
866\LL
867\stoptabulate
868
869When none of these is set the shaping penalty will be added. That way one can
870prevent a page break inside a shape.
871
872\stopsubsection
873
874\startsubsection[title=Criteria]
875
876The linebreak algorithm uses some heuristics for determining the badness of a
877line. In most cases that works quite well. Of course one can run into a bad
878result when one has a large document of weird (extreme) constraints and it can be
879tempting to mess around with parameters which then of course can lead to bad
880results in other places. A solution is is to locally tweak penalties or looseness
881but one can also just accept the occasional less optimal result (after all there
882are plenty occasions to make a document look bad otherwise so best focus on the
883average first). That said, it is tempting to see if changing the hard codes
884criteria makes a difference. Experiments with this demonstrated the usual: when
885asked what looks best contradictions mix with expectations and being triggered by
886events that one related to \TEX, like successive hyphenated lines.
887
888The \prm {linebreakcriterion} parameter can be set to a value made from four bytes. Were
889not going to explain the magic numbers because they come from and are discussed in original
890\TEX. It is enough to know that we have four criteria:
891
892\starttabulate[llp]
893\DB magic \BC bound to \NC bytes \NC \NR
894\TB
895\NC 12 \NC semi tight \NC 0x7F...... \NC \NR
896\NC 12 \NC decent \NC 0x..7F.... \NC \NR
897\NC 12 \NC semi loose \NC 0x....7F.. \NC \NR
898\NC 99 \NC loose \NC 0x......7F \NC \NR
899\LL
900\stoptabulate
901
902These four values can be changed according to the above pattern and are limited
903to the range 1\endash127 which is plenty especially when one keeps in mind that
904the actual useful values sit around the 12 anyway. Values outside the range (and
905therefore an allover zero assignment) makes the defaults kick in.
906
907The original decisions are made in the following way:
908
909\starttyping
910function loose(badness)
911 if badness > loosecriterion then
912 return veryloosefit
913 elseif badness > decentcriterion then
914 return loosefit
915 else {
916 return decentfit
917 end
918end
919
920function tight(badness)
921 if badness > decentcriterion then
922 return tightfit
923 else {
924 return decentfit
925 end
926end
927\stoptyping
928
929while in \LUAMETATEX\ we use (again in \LUA speak):
930
931\starttyping
932function loose(badness)
933 if badness > loose then
934 return veryloosefit
935 elseif badness > semiloose then
936 return semiloosefit
937 elseif badness > decent then
938 return loosefit
939 else
940 return decentfit
941 end
942end
943
944function tight(badness)
945 if badness > semitight then
946 return semitightfit
947 else if badness > decent then
948 return tightfit
949 else
950 return decentfit
951 end
952end
953\stoptyping
954
955So we have a few more steps to play with. But dont be disappointed when it
956doesnt work out as you expect. Don Knuth did a good job on the heuristics and
957after many decades there is no real need to change something. Consider it a
958playground.
959
960The parameter \prm {ignoredepthcriterion} is set to 1000pt at startup and is a
961special signal for \prm {prevdepth}. You can change the value locally for
962educational purposes but best not mess with this standard value in production
963code unless you want special effects.
964
965\stopsubsection
966
967\stopsection
968
969\startsection[title={Inserts}]
970
971Inserts are tightly integrated into the page builder. Depending on penalties and
972available space they end up on the same page as were they got injected or they
973move to following pages, either or not split.
974
975In traditional \TEX\ inserts are controlled by registers. A quadruple of box,
976skip, dimen and count registers with the same number acts as an insert class.
977Details can be found in the \TEX book. A side effect of this is that we only have
978these four properties bound to class, other properties of inserts are driven by
979shared parameters. Another side effect is that register management has to make
980sure that these foursome get \quote {allocates} as set and not clashes with other
981register allocations.
982
983In \LUAMETATEX\ you can set the \prm {insertmode} to a non zero value in which case
984inserts are not using the register pool but have their own (global) resources. For
985now this is mode driven (for compatibility reasons) and once set or when an
986insert has been accessed, this mode is frozen, so this parameter can be set
987very early in the macro package loading process.
988
989
990\starttabulate[llp]
991\DB primitive \BC traditional \BC explanation \NC \NR
992\TB
993\NC \prm {insertdistance} \NC skip \NC the space before the first instance (on a page) \NC \NR
994\NC \prm {insertmultiplier} \NC count \NC a factor that is used to calculate the height used \NC \NR
995\NC \prm {insertlimit} \NC dimen \NC the maximum amount of space on a page to be taken \NC \NR
996\NC \prm {insertpenalty} \NC \prm {insertpenalties} \NC the floating penalty (used when set) \NC \NR
997\NC \prm {insertmaxdepth} \NC \prm {maxdepth} \NC the maximum split depth (used when set) \NC \NR
998\NC \prm {insertstorage} \NC \NC signals that the insert has to be stored for later \NC \NR
999\NC \prm {insertheight} \NC \prm {ht} box index \NC the accumulated height of the inserts so far \NC \NR
1000\NC \prm {insertdepth} \NC \prm {dp} box index \NC the current depth of the inserts so far \NC \NR
1001\NC \prm {insertwidth} \NC \prm {wd} box index \NC the width of the inserts \NC \NR
1002\NC \prm {insertbox} \NC box index \NC the boxed content \NC \NR
1003\NC \prm {insertcopy} \NC box index \NC a copy of the boxed content \NC \NR
1004\NC \prm {insertunbox} \NC box index \NC the unboxed content \NC \NR
1005\NC \prm {insertuncopy} \NC box index \NC a copy of the unboxed content \NC \NR
1006\NC \prm {insertuncopy} \NC box index \NC a copy of the unboxed content \NC \NR
1007\NC \prm {insertprogress} \NC box index \NC the currently accumulated height \NC \NR
1008\LL
1009\stoptabulate
1010
1011These primitives takes an insert class number. The \prm {insertpenalties}
1012primitives is unchanged, as is the \LUATEX\ \prm {insertheights} one. When \prm
1013{insertstoring} is set 1, all inserts that have their storage flag set will be
1014saved. Think of a multi column setup where inserts have to end up in the last
1015column. If there are three columns, the first two will store inserts. Then when
1016the last column is dealt with \prm {insertstoring} can be set to 2 and that will
1017signal the builder that we will inject the inserts. In both cases, the value of
1018this register will be set to zero so that it doesnt influence further
1019processing. You can use \prm {ifinsert} to check if an insert box is void. More
1020details about these (probably experimental for a while) features can be found in
1021documents that come with \CONTEXT.
1022
1023A limitation of inserts is that when they are buried too deep, a property they
1024share with inserts, they become invisible This can be dealt with by the migration
1025feature described in an upcoming section.
1026
1027The \LUAMETATEX\ engine has some tracing built in that is enabled by setting \prm
1028{tracinginserts} to a positive value.
1029
1030\stopsection
1031
1032\startsection[title={Marks}]
1033
1034\topicindex {marks}
1035
1036Marks are kind of signal nodes in the list that refer to stored token lists. When
1037a page has been split off and is handed over to the output routine these signals
1038are resolved into first, top and bottom mark references that can (for instance)
1039be used for running headers.
1040
1041In \ETEX\ the standard \TEX\ primitives \prm {mark}, \prm {firstmark}, \prm
1042{topmark}, \prm {botmark}, \prm {splitfirstmark} and \prm {splitbotmark} have
1043been extended with plural forms that accent a number before the token list. That
1044number indicates a mark class.
1045
1046In addition to the mark fetch commands, we also have access to the last set
1047mark in the given class with \prm {currentmarks}:
1048
1049\startsyntax
1050\currentmarks <16bit number>
1051\stopsyntax
1052
1053A problem with marks is that one cannot really reset them. Mark states are kept
1054in the node lists and only periodically the state is snapshot into the global
1055state variables. The \LUATEX\ engine can reset these global states with \prm
1056{clearmarks} but thats only half a solution. In \LUAMETATEX\ we have \prm
1057{flushmarks} which, like \prm {marks}, puts a node in the list that does a reset.
1058This permits implementing controlled resets of specific marks at the cost of a
1059possible interfering mode, but that can normally be dealt with rather well.
1060
1061The \prm {clearmarks} primitive complements the \ETEX\ mark primitives and clears
1062a mark class completely, resetting all three connected mark texts to empty. It is
1063an immediate command (no synchronization node is used).
1064
1065\startsyntax
1066\clearmarks <16bit number>
1067\stopsyntax
1068
1069The \prm {flushmarks} variant is delayed but puts a (mark) node in the list as
1070signal (we could have gone for a keyword to \prm {marks} instead).
1071
1072\startsyntax
1073\flushmarks <16bit number>
1074\stopsyntax
1075
1076Another problem with marks is that when they are buried too deep, a property they
1077share with inserts, they become invisible. This can be dealt with by the
1078migration feature described in the next section.
1079
1080The \LUAMETATEX\ engine has some tracing built in that is enabled by setting \prm
1081{tracingmarks} to a positive value. When set to1 the page builder shows the set
1082values, and when set to a higher value details about collecting them are shown.
1083
1084\stopsection
1085
1086\startsection[title={Adjusts}]
1087
1088The \prm {vadjust} primitive injects something in the vertical list after the
1089line where it ends up. In \PDFTEX\ the \type {pre} keyword was added so that one
1090could force something before a previous line (actually this was something that we
1091needed in \CONTEXT\ \MKII). The \LUAMETATEX\ engine also supports the \type {post}
1092keyword.
1093
1094We support a few more keywords: \type {before} will prepend the adjustment to the
1095already given one, and \type {after} will append it. The \type {index} keyword
1096expects an integer and relates that to the current adjustment. This index is
1097passed to an (optional) callback when the adjustment is finally moved to the
1098vertical list. That move is actually delayed because like inserts and marks these
1099(vertical) adjustments can migrate to the \quote {outer} vertical level.
1100
1101The main reason for the index having no influence on the order is that this
1102primitive already could be used multiple times and order is determined by usage.
1103\footnote {Under consideration is to let the callback mess with the flushing
1104order.}
1105
1106The \LUAMETATEX\ engine has some tracing built in that is enabled by setting \prm
1107{tracingadjusts} to a positive value. Currently there is not that much tracing
1108which is why the value has to be at least 2 in order to be compatible with other
1109(detailed) tracers.
1110
1111\stopsection
1112
1113\startsection[title={Migration}]
1114
1115There are a few injected node types that are used to track information: marks,
1116inserts and adjusts (see previous sections). Marks are token lists that can be
1117used to register states like section numbers and titles they are synchronized in
1118the page builder when a page is shipped out. Inserts are node lists that get
1119rendered and relate to specific locations and these are flushed with the main
1120vertical list which also means that in calculating page breaks they need to be
1121taken into account. An Adjust is material that gets injected before or after a
1122line. Strictly spoke local boxes also in this repertoire but they are dealt with
1123in the par builder.
1124
1125A new primitive \prm {automigrationmode} can be used to let deeply burried marks
1126and inserts bubble up to the outer level.
1127
1128\starttabulate[cp]
1129\DB value \BC explanation \NC \NR
1130\TB
1131\NC \the\markautomigrationcode \NC migrate marks in the par builder \NC \NR
1132\NC \the\insertautomigrationcode \NC migrate inserts in the par builder \NC \NR
1133\NC \the\adjustautomigrationcode \NC migrate adjusts in the par builder \NC \NR
1134\NC \the\preautomigrationcode \NC migrate prebox material in the page builder \NC \NR
1135\NC \the\postautomigrationcode \NC migrate postbox material in the page builder \NC \NR
1136\LL
1137\stoptabulate
1138
1139If you want to migrate marks and inserts you need to set all these flags. Migrated
1140marks and inserts end up as postbox properties and will be handled in the page
1141builder as such. At the \LUA\ end you can add pre and postbox material too.
1142
1143The primitive register \prm {holdingmigrations} is a bitset that can be used to temporarily
1144disable migrations. It is a generalization of \prm {holdinginserts}.
1145
1146\starttabulate[cTp]
1147\DB value \BC explanation \NC \NR
1148\TB
1149\NC 0x01 \NC marks \NC \NR
1150\NC 0x02 \NC inserts \NC \NR
1151\NC 0x04 \NC adjusts \NC \NR
1152\LL
1153\stoptabulate
1154
1155Migrates material is bound to boxes so boxed material gets unboxed it is taken
1156into account, but you should be aware of potential side effects. But then, marks,
1157inserts and adjusts always demanded care.
1158
1159\stopsection
1160
1161\startsection[title={Pages}]
1162
1163The page builder can be triggered by (for instance) a penalty but you can also
1164use \prm {pageboundary}. This will trigger the page builder but not leave
1165anything behind.
1166
1167{\em In due time we will discuss \prm {pagevsize}, \prm {pageextragoal} and \prm
1168{lastpageextra} but for now we treat them as very experimental and they will be
1169tested in \CONTEXT, also in discussion with users.}
1170
1171\stopsection
1172
1173\startsection[title={Paragraphs}]
1174
1175The numeric primitive \prm {lastparcontext} inspector reports the current context
1176in which a paragraph triggering commands happened. The numbers can be queried
1177with \type {tex.getparcontextvalues()} and currently are: \showvaluelist
1178{tex.getparcontextvalues()}. As with the other \type {\last...} primitives this
1179variable is global.
1180
1181Traditional \TEX\ has the \prm {parfillskip} parameter that determines the way
1182the last line is filled. In \LUAMETATEX\ we also have \prm {parfillleftskip}. The
1183counterparts for the first line are \prm {parinitleftskip} and \prm
1184{parinitrightskip}. We also have \prm {parfillrightskip} as consistency alias.
1185
1186\startbuffer
1187\leftskip 2em
1188\rightskip \leftskip
1189\parfillskip \zeropoint plus 1 fill
1190\parfillleftskip \parfillskip
1191\parinitleftskip \parfillleftskip
1192\parinitrightskip\parfillleftskip
1193\input ward
1194\stopbuffer
1195
1196\typebuffer This results in: \par \start \em \getbuffer \par \stop
1197
1198An additional tracing primitive \prm {tracingfullboxes} reports details about the
1199encountered overfull boxes. This can be rather verbose!
1200
1201Normally \TEX\ will insert an empty hbox when paragraph indentation is requested
1202but when the second bit in \prm {normalizelinemode} has been set \LUAMETATEX\
1203will in a glue node instead. You can zero the set value with \prm {undent} unless
1204of course some more has been inserted already.
1205
1206\startbuffer
1207\parinitleftskip1cm \parindent 1cm \indent test \par
1208\parinitleftskip1cm \parindent 1cm \undent test \par
1209\parinitleftskip1cm \parindent 1cm \indent \undent test \par
1210\parinitleftskip1cm \parindent 1cm \indent \strut \undent test \par
1211\stopbuffer
1212
1213\typebuffer \startpacked \getbuffer \stoppacked
1214
1215By setting \prm {tracingpenalties} to a positive value penalties related to
1216windows, clubs, lines etc.\ get reported to the output channels.
1217
1218\stopsection
1219
1220\startsection[title={Local boxes}]
1221
1222As far as I know the \OMEGA\ALEPH\ local box mechanism is mostly in those
1223engines in order to support repetitive quotes. In \LUATEX\ this mechanism has
1224been made more robust and in \LUAMETATEX\ it became more tightly integrated in
1225the paragraph properties. In order for it to be more generic and useful, it got
1226more features. For instance it is a bit painful to manage with respect to
1227grouping (which is a reason why its not that much used). The most interesting
1228property is that the dimensions are taken into account when a paragraph is
1229broken into lines.
1230
1231There are three commands: \prm {localleftbox}, \prm {localrightbox} and the
1232\LUAMETATEX\ specific \prm {localmiddlebox} which is basically a right box but
1233when we pass these boxes to a callback they can be distinguished (we could have
1234used the index but this was a cheap extra signal so we keep it).
1235
1236These commands take optional keywords. The \type {index} keyword has to be
1237followed by an integer. This index determines the order which doesnt introduce a
1238significant compatibility issue: local boxes are hardly used and originally had
1239only one instance.
1240
1241The \type {par} keyword forces the box to be added to the current paragraph head.
1242This permits setting them when a paragraph has already started. The
1243implementation of these boxes is done via so called (local) paragraph nodes and
1244there is one at the start of each paragraph.
1245
1246The \type {local} keyword tells this mechanism not to update the registers that
1247keep these boxes. In that case a next paragraph will start fresh. The \type
1248{keep} option will do the opposite and retain the box after a group ends.
1249
1250The commands: \prm {localleftboxbox}, \prm {localrightboxbox} and \prm
1251{localmiddleboxbox} return a copy of the current related register content.
1252
1253\stopsection
1254
1255\startsection[title={Leaders}]
1256
1257Leaders are flexible content that are basically just seen as glue and it is up to
1258the backend to apply the effective glue to the result as seen in the backend
1259(like a rule of box). This means that the frontend doesnt do anything with the
1260fact that we have a regular \prm {leaders}, a \prm {gleaders}, \prm {xleaders} or
1261\prm {cleaders}. The \prm {uleaders} that has been added in \LUAMETATEX\ is just
1262that: an extra leader category. The main difference is that the width of the
1263given box is added to the glue. That way we create a stretchable box.
1264
1265\startbuffer
1266\unexpandedloop 1 30 1 {x \hbox{1 2 3} x }
1267\unexpandedloop 1 30 1 {x {\uleaders \hbox{1 2 3}\hskip 0pt plus 10pt minus 10pt\relax} x }
1268\unexpandedloop 1 30 1 {x {\uleaders \hbox{1 2 3}\hskip 0pt plus \interwordstretch minus \interwordshrink} x }
1269\unexpandedloop 1 30 1 {x {\uleaders \hbox{1 2 3}\hskip 0pt plus 2\interwordstretch minus 2\interwordshrink} x }
1270\stopbuffer
1271
1272\typebuffer
1273
1274Here are some examples:
1275
1276\startlines
1277\getbuffer
1278\stoplines
1279
1280So the flexibility fo the box plays a role in the line break calculations. But in
1281the end the backend has to do the work.
1282
1283\startbuffer[a]
1284{\green \hrule width \hsize} \par \vskip2pt
1285\vbox to 40pt {
1286 {\red\hrule width \hsize} \par \vskip2pt
1287 \vbox {
1288 \vskip2pt {\blue\hrule width \hsize} \par
1289 \vskip 10pt plus 10pt minus 10pt
1290 {\blue\hrule width \hsize} \par \vskip2pt
1291 }
1292 \vskip2pt {\red\hrule width \hsize} \par
1293}
1294\vskip2pt {\green \hrule width \hsize} \par
1295\stopbuffer
1296
1297\startbuffer[b]
1298{\green \hrule width \hsize} \par \vskip2pt
1299\vbox to 40pt {
1300 {\red\hrule width \hsize} \par \vskip2pt
1301 \uleaders\vbox {
1302 \vskip2pt {\blue\hrule width \hsize} \par
1303 \vskip 10pt plus 10pt minus 10pt
1304 {\blue\hrule width \hsize} \par \vskip2pt
1305 }\vskip 0pt plus 10pt minus 10pt
1306 \vskip2pt {\red\hrule width \hsize} \par
1307}
1308\vskip2pt {\green \hrule width \hsize} \par
1309\stopbuffer
1310
1311\typebuffer[a]
1312
1313with
1314
1315\typebuffer[b]
1316
1317In the first case we get the this:
1318
1319\startlinecorrection
1320\getbuffer[a]
1321\stoplinecorrection
1322
1323but with \prm {uleaders} we get:
1324
1325\startlinecorrection
1326\normalizeparmode\zerocount
1327\getbuffer[b]
1328\stoplinecorrection
1329
1330or this:
1331
1332\startlinecorrection
1333\normalizeparmode"FF
1334\getbuffer[b]
1335\stoplinecorrection
1336
1337In the second case we flatten the leaders in the engine by setting the second bit
1338in the \prm {normalizeparmode} parameter (\type {0x2}). We actually do the same
1339with \prm {normalizelinemode} where bit 10 is set (\type {0x200}). The \type
1340{delay} keyword can be passed with a box to prevent flattening. If we dont do
1341this in the engine, the backend has to take care of it. In principle this permits
1342implementing variants in a macro package. Eventually there will be plenty examples in
1343the \CONTEXT\ code base and documentation. Till then, consider this experimental.
1344
1345\stopsection
1346
1347\startsection[title=Alignments]
1348
1349The primitive \prm {alignmark} duplicates the functionality of \type {#} inside
1350alignment preambles, while \prm {aligntab} duplicates the functionality of \type
1351{}. The \prm {aligncontent} primitive directly refers to an entry so that one
1352does not get repeated.
1353
1354Alignments can be traced with \prm {tracingalignments}. When set to1 basics
1355usage is shown, for instance of \prm {noalign} but more interesting is2 or more:
1356you then get the preambles reported.
1357
1358The \prm {halign} (tested) and \prm {valign} (yet untested) primitives accept a
1359few keywords in addition to \type {to} and \type {spread}:
1360
1361\starttabulate[lp]
1362\DB keyword \BC explanation \NC \NR
1363\TB
1364\NC \type {attr} \NC set the given attribute to the given value \NC \NR
1365\NC \type {callback} \NC trigger the \type {alignmentfilter} callback \NC \NR
1366\NC \type {discard} \NC discard zero \prm {tabskip}s \NC \NR
1367\NC \type {noskips} \NC dont even process zero \prm {tabskip}s \NC \NR
1368\NC \type {reverse} \NC reverse the final rows \NC \NR
1369\LL
1370\stoptabulate
1371
1372In the preamble the \prm {tabsize} primitive can be used to set the width of a
1373column. By doing so one can avoid using a box in the preamble which, combined
1374with the sparse tabskip features, is a bit easier on memory when you produce
1375tables that span hundreds of pages and have a dozen columns.
1376
1377The \prm {everytab} complements the \prm {everycr} token register but is sort of
1378experimental as it might become more selective and powerful some day.
1379
1380The two primitives \prm {alignmentcellsource} and \prm {alignmentwrapsource} that
1381associate a source id (integer) to the current cell and row (line). Sources and
1382targets are experimental and are being explored in \CONTEXT\ so well see where
1383that ends up in.
1384
1385{\em todo: callbacks}
1386
1387\stopsection
1388
1389\stopchapter
1390
1391\stopcomponent
1392 |