lowlevel-paragraphs.tex /size: 52 Kb    last modification: 2025-02-21 11:03
1% language=us runpath=texruns:manuals/lowlevel
2
3\environment lowlevel-style
4
5\startdocument
6  [title=paragraphs,
7   color=middlecyan]
8
9\startsectionlevel[title=Introduction]
10
11This manual is mostly discussing a few low level wrappers around low level \TEX\
12features. Its writing is triggered by an update to the \METAFUN\ and \LUAMETAFUN\
13manuals where we mess a bit with shapes. It gave a good reason to also cover some
14more paragraph related topics but it might take a while to complete. Remind me if
15you feel that takes too much time.
16
17Because paragraphs and their construction are rather central to \TEX, you can
18imagine that the engine exposes dealing with them. This happens via commands
19(primitives) but only when it's robust. Then there are callbacks, and some
20provide detailed information about what we're dealing with. However, intercepting
21node lists can already be hairy and we do that a lot in \CONTEXT. Intercepting
22and tweaking paragraph properties is even more tricky, which is why we try to
23avoid that in the core. But \unknown\ in the following sections you will see that
24there are actually a couple of mechanism that do so. Often new features like this
25are built in stepwise and enabled locally for a while and when they seem okay
26they get enabled by default. \footnote {For this we have \type
27{\enableexperiments} which one can use in \type {cont-loc.mkxl} or \type
28{cont-exp.mkxl}, files that are loaded runtime when on the system. When you use
29them, make sure they don't interfere; they are not part of the updates, contrary
30to \type {cont-new.mkxl}.}
31
32\stopsectionlevel
33
34\startsectionlevel[title=Paragraphs]
35
36Before we demonstrate some trickery, let's see what a paragraph is. Normally a
37document source is formatted like this:
38
39\starttyping[option=TEX]
40some text (line 1)
41some text (line 2)
42
43some more test (line 1)
44some more test (line 2)
45\stoptyping
46
47There are two blocks of text here separated by an empty line and they become two
48paragraphs. Unless configured otherwise an empty line is an indication that we
49end a paragraph. You can also explicitly do that:
50
51\starttyping[option=TEX]
52some text (line 1)
53some text (line 2)
54\par
55some more test (line 1)
56some more test (line 2)
57\stoptyping
58
59When \TEX\ starts a paragraph, it actually also does something think of:
60
61\starttyping[option=TEX]
62[\the\everypar]some text      (line 1) some text      (line 2) \par
63[\the\everypar]some more test (line 1) some more test (line 2) \par
64\stoptyping
65
66or more accurate:
67
68\starttyping[option=TEX]
69[\the\everypar]some text      some text      \par
70[\the\everypar]some more test some more test \par
71\stoptyping
72
73because the end|-|of|-|line character has become a space. As mentioned,
74an empty line is actually the end of a paragraph. But in \LUAMETATEX\
75we can cheat a bit. If we have this:
76
77\startbuffer
78line 1
79
80line 2
81\stopbuffer
82
83\typebuffer[option=TEX]
84
85We can do this (watch how we need to permit overloading a primitive when we have
86enabled \type {\overloadmode}):
87
88\startbuffer
89\pushoverloadmode
90\def\linepar{\removeunwantedspaces !\ignorespaces}
91\popoverloadmode
92line 1
93
94line 2
95\stopbuffer
96
97\typebuffer[option=TEX]
98
99This comes out as:
100
101\start \getbuffer \stop
102
103I admit that since it got added (as part of some cleanup halfway the overhaul of
104the engine) I never saw a reason to use it, but it is a cheap feature. The \type
105{\linepar} primitive is undefined (\type {\undefined}) by default so no user sees
106it anyway. Just don't use it unless maybe for some pseudo database trickery (I
107considered using it for the database module but it is not needed). In a similar
108fashion, just don't redefine \type {\par}: it's asking for troubles and \quote
109{not done} in \CONTEXT\ anyway.
110
111Back to reality. In \LUATEX\ we get a node list that starts with a so called
112\type {localpar} node and ends with a \type {\parfillskip}. The first node is
113prepended automatically. That list travels through the system: hyphenation,
114applying font properties, break the effectively one line into lines, wrap them
115and add them to a vertical list, etc. Each stage can be intercepted via
116callbacks.
117
118When the paragraph is broken into lines hanging indentation or a so called par
119shape can be applied, and we will see more of that later, here we talk \type
120{\par} and show another \LUAMETATEX\ trick:
121
122\startbuffer
123\def\foo{{\bf test:} \ignorepars}
124
125\foo
126
127line
128\stopbuffer
129
130\typebuffer[option=TEX]
131
132The macro typesets some text and then skips to the next paragraph:
133
134\start \getbuffer \stop
135
136Think of this primitive as being a more powerful variant of \type
137{\ignorespaces}. This leaves one aspect: how do we start a paragraph. Technically
138we need to force \TEX\ into so called horizontal mode. When you look at plain
139\TEX\ documents you will notice commands like \type {\noindent} and \type
140{\indent}. In \CONTEXT\ we have more high level variants, for instance we have
141\type {\noindentation}.
142
143A robust way to make sure that you get in horizontal mode is using \type
144{\dontleavehmode} which is a wink to \type {\leavevmode}, a command that you
145should never use in \CONTEXT, so when you come from plain or \LATEX, it's one of
146the commands you should wipe from your memory.
147
148When \TEX\ starts with a paragraph the \type {\everypar} token list is expanded
149and again this is a primitive you should not mess with yourself unless in very
150controlled situations. If you change its content, you're on your own with respect
151to interferences and side effects.
152
153One of the things that \TEX\ does in injecting the indentation. Even when there
154is none, it gets added, not as skip but as an empty horizontal box of a certain
155width. This is easier on the engine when it constructs the paragraph from the one
156liner: starting with a skip demands a bit more testing in the process (a nice
157trick so to say). However, in \CONTEXT\ we enable the \LUAMETATEX\ feature that
158does use a skip instead of a box. It's part of the normalization that is
159discussed later. Instead of checking for a box with property indent, we check for
160a skip with such property. This is often easier and cleaner.
161
162A bit off topic is the fact that in traditional \TEX\ empty lines or \type {\par}
163primitives can trigger an error. This has to do with the fact that the program
164evolved in a time where paper terminals were used and runtime could be excessive.
165So, in order to catch a possible missing brace, a concept of \type {\long}
166macros, permitting \type {\par} or equivalents in arguments, was introduced as
167well as not permitting them in for instance display math. In \CONTEXT\ \MKII\
168most macros that could be sensitive for this were defined as \type {\long} so
169that users never had to bother about it and probably were not even aware of it.
170Right from the start in \LUATEX\ these error|-|triggers could be disabled which
171of course we enable in \CONTEXT\ and in \LUAMETATEX\ these features have been
172removed altogether. I don't think users will complain about this.
173
174If you want to enforce a newline but not a new paragraph you can use the \type
175{\crlf} command. When used on its own it will produce an empty line. Don't use
176this to create whitespace between lines.
177
178If you want to do something after so called par tokens are seen you can do this:
179
180\startbuffer
181\def\foo{{\bf >>>> }}
182\expandafterpars\foo
183
184this is a new paragraph ...
185
186\expandafterpars\foo
187\par\par\par\par
188this is a new paragraph ...
189\stopbuffer
190
191\typebuffer[option=TEX]
192
193This not to be confused with \type {\everypar} which is a token list that \TEX\
194itself injects before each paragraph (also nested ones).
195
196\getbuffer
197
198This is typically a primitive that will only be used in macros. You can actually
199program it using macros: pickup a token, check and push it back when it's not a
200par equivalent token. The primitive is is just nicer (and easier on the log when
201tracing is enabled).
202
203\stopsectionlevel
204
205\startsectionlevel[title=Properties]
206
207A paragraph is just a collection of lines that result from one input line that
208got broken. This process of breaking into lines is influenced by quite some
209parameters. In traditional \TEX\ and also in \LUAMETATEX\ by default the values
210that are in effect when the end of the paragraph is met are used. So, when you
211change them in a group and then ends the paragraph after the group, the values
212you've set in the group are not used.
213
214However, in \LUAMETATEX\ we can optionally store them with the paragraph. When
215that happens the values current at the start are frozen. You can still overload
216them but that has to be done explicitly then. The advantage is that grouping no
217longer interferes with the line break algorithm. The magic primitive is \type
218{\snapshotpar} which takes a number made from categories mentioned below:
219
220\startluacode
221    local specifications = tex.getpardataspecifications()
222    local categories = tex.getfrozenparvalues()
223 -- context.starttabulate { "|l|l|l|l|" }
224    context.starttabulate { "|l|l|l|    " }
225    context.BC() context("variable")
226    context.BC() context("category")
227    context.BC() context("code")
228    context.NC() context.NR()
229    for i=1,#specifications do
230        local s = specifications[i]
231        local n = s[3]
232        local c = s[4]
233        context.NC() context.formatted.type("\\%s",n)
234        context.NC() context(categories[c])
235        context.NC() context("0x%08X",c)
236     -- context.NC() context.formatted.type("\\%sfrozensparcode",categories[c])
237        context.NC() context.NR()
238    end
239    context.stoptabulate()
240\stopluacode
241
242As you can see here, there are more paragraph related parameters than in for
243instance \PDFTEX\ and \LUATEX\ and these are (to be) explained in the
244\LUAMETATEX\ manual. You can imagine that keeping this around with the paragraph
245adds some extra overhead to the machinery but most users won't notice that
246because is is compensated by gains elsewhere.
247
248This is pretty low level and there are a bunch of helpers that support this but
249these are not really user level macros. As with everything \TEX\ you can mess
250around as much as you like, and the code gives plenty of examples but when you do
251this, you're on your own because it can interfere with \CONTEXT\ core
252functionality.
253
254In \LMTX\ taking these snapshots is turned on by default and because it thereby
255fundamentally influences the par builder, users can run into compatibility issues
256but in practice there has been no complaints (and this feature has been in use
257quite a while before this document was written). One reason for users not
258noticing is that one of the big benefits is probably handled by tricks mentioned
259on the mailing list. Imagine that you have this:
260
261\starttyping[option=TEX]
262{\bf watch out:} here is some text
263\stoptyping
264
265In this small example the result will be as expected. But what if something magic
266with the start of a paragraph is done? Like this:
267
268\starttyping[option=TEX]
269\placefigure[left]{A cow!}{\externalfigure[cow.pdf]}
270
271{\bf watch out:} here is some text ... of course much more is needed to
272    get a flow around the figure!
273\stoptyping
274
275The figure will hang at the left side of the paragraph but it is put there when
276the text starts and that happens inside the bold group. It means that the
277properties we set in order to get the shape around the figure are lost as soon as
278we're at \quote{\type {here is some text}} and definitely is wrong when the
279paragraph ends and the par builder has to use them to get the shape right. We get
280text overlapping the figure. A trick to overcome this is:
281
282\starttyping[option=TEX]
283\dontleavehmode {\bf watch out:} here is some text ... of course much
284    more is needed to get a flow around the figure!
285\stoptyping
286
287where the first macro makes sure we already start a paragraph before the group is
288entered (using a \type {\strut} also works). It's not nice and I bet users have
289been bitten by this and by now know the tricks. But, with snapshots such fuzzy
290hacks are not needed any more! The same is true with this:
291
292\starttyping[option=TEX]
293{\leftskip 1em some text \par}
294\stoptyping
295
296where we had to explicitly end the paragraph inside the group in order to retain
297the skip. I suppose that users normally use the high level environments so they
298never had to worry about this. It's also why users probably won't notice that
299this new mechanism has been active for a while. Actually, when you now change a
300parameter inside the paragraph its new value will not be applied (unless you
301prefix it with \type {\frozen} or snapshot it) but no one did that anyway.
302
303\stopsectionlevel
304
305\startsectionlevel[title=Wrapping up]
306
307In \CONTEXT\ \LMTX\ we have a mechanism to exercise macros (or content) before a
308paragraph ends. This is implemented using the \type {\wrapuppar} primitive. The
309to be wrapped up material is bound to the current paragraph which in order to
310get this done has to be started when this primitive is used.
311
312Although the high level interface has been around for a while it still needs a
313bit more testing (read: use cases are needed). In the few cases where we already
314use it application can be different because again it relates to snapshots. This
315because in the past we had to use tricks that also influenced the user interface
316of some macros (which made them less natural as one would expect). So the
317question is: where do we apply it in old mechanisms and where not.
318
319{\em todo: accumulation, interference, where applied, limitations}
320
321% \vbox   {vbox    : \wrapuppar{1}test\par x\wrapuppar{2}test}\blank
322% \vtop   {vtop    : \wrapuppar{1}test\par x\wrapuppar{2}test}\blank
323% \vcenter{vcenter : \wrapuppar{1}test\par x\wrapuppar{2}test}\blank
324% $$x = \vcenter{vcenter : \wrapuppar{1}test\par x\wrapuppar{2}test}$$\blank
325% x\vadjust{vadjust : \wrapuppar{1}test\par x\wrapuppar{2}test}x\blank
326
327% \starttext
328% \starttabulate[|||]
329%     \NC test \NC test \NC \NR
330%     \NC test \NC \hbox{\hbox{\hbox{\vadjust pre {\kern-12pt}}}}test \NC \NR
331%     \NC test \NC \hbox{\hbox{\hbox{\vadjust     {\kern-12pt}}}}test \NC \NR
332%     \NC test \NC test \NC \NR
333% \stoptabulate
334% \starttabulate[||p|]
335%     \NC test \NC test \NC \NR
336%     \NC test \NC \hbox{\vadjust{\kern-22pt}}test \NC \NR
337%     \NC test \NC test \NC \NR
338% \stoptabulate
339% \stoptext
340
341\stopsectionlevel
342
343\startsectionlevel[title=Hanging]
344
345There are two mechanisms for getting a specific paragraph shape: rectangular
346hanging and arbitrary shapes. Both mechanisms work top|-|down. The first
347mechanism uses a combination of \type {\hangafter} and \type {\hangindent}, and
348the second one depends on \type {\parshape}. In this section we discuss the
349rectangular one.
350
351\startbuffer[demo-5]
352\hangafter  4 \hangindent  4cm \samplefile{tufte} \page
353\hangafter -4 \hangindent  4cm \samplefile{tufte} \page
354\hangafter  4 \hangindent -4cm \samplefile{tufte} \page
355\hangafter -4 \hangindent -4cm \samplefile{tufte} \page
356\stopbuffer
357
358\typebuffer[demo-5][option=TEX]
359
360As you can see in \in {figure} [fig:hang], the four cases are driven by the sign
361of the values. If you want to hang into the margin you need to use different
362tricks, like messing with the \type {\leftskip}, \type {\rightskip} or \type
363{\parindent} parameters (which then of course can interfere with other mechanisms
364uses at the same time).
365
366\startplacefigure[title=Hanging indentation,reference=fig:hang]
367\startcombination[nx=2,ny=2]
368    {\typesetbuffer[demo-5][page=1,width=.4\textwidth,frame=on]} {\type{\hangafter +4 \hangindent +4cm}}
369    {\typesetbuffer[demo-5][page=2,width=.4\textwidth,frame=on]} {\type{\hangafter -4 \hangindent +4cm}}
370    {\typesetbuffer[demo-5][page=3,width=.4\textwidth,frame=on]} {\type{\hangafter +4 \hangindent -4cm}}
371    {\typesetbuffer[demo-5][page=4,width=.4\textwidth,frame=on]} {\type{\hangafter -4 \hangindent -4cm}}
372\stopcombination
373\stopplacefigure
374
375\stopsectionlevel
376
377\startsectionlevel[title=Shapes]
378
379In \CONTEXT\ we don't use \type {\parshape} a lot. It is used in for instance
380side floats but even there not in all cases. It's more meant for special
381applications. This means that in \MKII\ and \MKIV\ we don't have some high level
382interface. However, when \METAFUN\ got upgraded to \LUAMETAFUN, and the manual
383also needed an update, one of the examples in that manual that used shapes also
384got done differently (read: nicer). And that triggered the arrival of a new low
385level shape mechanism.
386
387One important property of the \type {\parshape} mechanism is that it works per
388paragraph. You define a shape in terms of a left margin and width of a line. The
389shape has a fixed number of such pairs and when there is more content, the last
390one is used for the rest of the lines. When the paragraph is finished, the shape
391is forgotten. \footnote {Not discussed here is a variant that might end up in
392\LUAMETATEX\ that works with the progression, i.e.\ takes the height of the
393content so far into account. This is somewhat tricky because for that to work
394vertical skips need to be frozen, which is no real big deal but has to be done
395careful in the code.}
396
397The high level interface is a follow up on the example in the \METAFUN\ manual and
398uses shapes that carry over to the next paragraph. In addition we can cycle over
399a shape. In this interface shapes are defined using keyword. Here are some
400examples:
401
402\starttyping[option=TEX]
403\startparagraphshape[test]
404    left 1mm right 1mm
405    left 5mm right 5mm
406\stopparagraphshape
407\stoptyping
408
409This shape has only two entries so the first line will have a 1mm margin while
410later lines will get 5mm margins. This translates into a \type {\parshape} like:
411
412\starttyping[option=TEX]
413\parshape 2
414    1mm \dimexpr\hsize-1mm\relax
415    5mm \dimexpr\hsize-5mm\relax
416\stoptyping
417
418Watch the number \type {2}: it tells how many specification lines follow. As you
419see, we need to calculate the width.
420
421\starttyping[option=TEX]
422\startparagraphshape[test]
423    left 1mm right 1mm
424    left 5mm right 5mm
425    repeat
426\stopparagraphshape
427\stoptyping
428
429This variant will alternate between 1mm and 5mm margins. The repeating feature is
430translated as follows. Maybe at some point I will introduce a few more options.
431
432\starttyping[option=TEX]
433\parshape 2 options 1
434    1mm \dimexpr\hsize-1mm\relax
435    5mm \dimexpr\hsize-5mm\relax
436\stoptyping
437
438A shape can have some repetition, and we can save keystrokes by copying the last
439entry. The resulting \type {\parshape} becomes rather long.
440
441\starttyping[option=TEX]
442\startparagraphshape[test]
443    left 1mm right 1mm
444    left 2mm right 2mm
445    left 3mm right 3mm
446    copy 8
447    left 4mm right 4mm
448    left 5mm right 5mm
449    left 5mm hsize 10cm
450\stopparagraphshape
451\stoptyping
452
453Also watch the \type {hsize} keyword: we don't calculate the hsize from the \type
454{left} and \type {right} values but explicitly set it.
455
456\starttyping[option=TEX]
457\startparagraphshape[test]
458    left 1mm right 1mm
459    right 3mm
460    left 5mm right 5mm
461    repeat
462\stopparagraphshape
463\stoptyping
464
465When a \type {right} keywords comes first the \type {left} is assumed to be zero.
466In the examples that follow we will use a couple of definitions:
467
468\startbuffer[setup-0]
469\startparagraphshape[test]
470    both 1mm both 2mm both 3mm both 4mm both 5mm both 6mm
471    both 7mm both 6mm both 5mm both 4mm both 3mm both 2mm
472\stopparagraphshape
473\stopbuffer
474
475\startbuffer[setup-0-repeat]
476\startparagraphshape[test-repeat]
477    both 1mm both 2mm both 3mm both 4mm both 5mm both 6mm
478    both 7mm both 6mm both 5mm both 4mm both 3mm both 2mm
479    repeat
480\stopparagraphshape
481\stopbuffer
482
483\typebuffer[setup-0,setup-0-repeat][option=TEX]
484
485The last one could also be defines as:
486
487\starttyping[option=TEX]
488\startparagraphshape[test-repeat]
489    \rawparagraphshape{test} repeat
490\stopparagraphshape
491\stoptyping
492
493In the previous code we already introduced the \type {repeat} option. This will
494make the shape repeat at the engine level when the shape runs out of specified
495lines. In the application of a shape definition we can specify a \type {method}
496to be used and that determine if the next paragraph will start where we left off
497and discard afterwards (\type {shift}) or that we move the discarded lines up
498front so that we never run out of lines (\type {cycle}). It sounds complicated
499but just keep in mind that \type {repeat} is part of the \type {\parshape} and
500act within a paragraph while \type {shift} and \type {cycle} are applied when a
501new paragraph is started.
502
503\startbuffer[demo-1]
504\startshapedparagraph[list=test]
505    \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par}
506\stopshapedparagraph
507\stopbuffer
508
509\startbuffer[demo-1-repeat]
510\startshapedparagraph[list=test-repeat]
511    \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par}
512\stopshapedparagraph
513\stopbuffer
514
515In \in {figure} [fig:shape:discard] you see the following applied:
516
517\typebuffer[demo-1,demo-1-repeat][option=TEX]
518
519\startplacefigure[title=Discarded shaping,reference=fig:shape:discard]
520\startcombination[nx=2,ny=2]
521    {\typesetbuffer[setup-0,demo-1]       [page=1,width=.4\textwidth,frame=on]} {discard, finite shape,    page 1}
522    {\typesetbuffer[setup-0,demo-1]       [page=2,width=.4\textwidth,frame=on]} {discard, finite shape,    page 2}
523    {\typesetbuffer[setup-0,demo-1-repeat][page=1,width=.4\textwidth,frame=on]} {discard, repeat in shape, page 1}
524    {\typesetbuffer[setup-0,demo-1-repeat][page=2,width=.4\textwidth,frame=on]} {discard, repeat in shape, page 2}
525\stopcombination
526\stopplacefigure
527
528In \in {figure} [fig:shape:shift] we use this instead:
529
530\startbuffer[demo-2]
531\startshapedparagraph[list=test,method=shift]
532    \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par}
533\stopshapedparagraph
534\stopbuffer
535
536\startbuffer[demo-2-shift]
537\startshapedparagraph[list=test-repeat,method=shift]
538    \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par}
539\stopshapedparagraph
540\stopbuffer
541
542\typebuffer[demo-2,demo-2-repeat][option=TEX]
543
544\startplacefigure[title=Shifted shaping,reference=fig:shape:shift]
545\startcombination[nx=2,ny=2]
546    {\typesetbuffer[setup-0,demo-2][page=1,width=.4\textwidth,frame=on]}              {shift, finite shape,    page 1}
547    {\typesetbuffer[setup-0,demo-2][page=2,width=.4\textwidth,frame=on]}              {shift, finite shape,    page 2}
548    {\typesetbuffer[setup-0-repeat,demo-2-shift][page=1,width=.4\textwidth,frame=on]} {shift, repeat in shape, page 1}
549    {\typesetbuffer[setup-0-repeat,demo-2-shift][page=2,width=.4\textwidth,frame=on]} {shift, repeat in shape, page 2}
550\stopcombination
551\stopplacefigure
552
553Finally, in \in {figure} [fig:shape:cycle] we use:
554
555\startbuffer[demo-3]
556\startshapedparagraph[list=test,method=cycle]
557    \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par}
558\stopshapedparagraph
559\stopbuffer
560
561\startbuffer[demo-3-cycle]
562\startshapedparagraph[list=test-repeat,method=cycle]
563    \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par}
564\stopshapedparagraph
565\stopbuffer
566
567\typebuffer[demo-3,demo-3-repeat][option=TEX]
568
569\startplacefigure[title=Cycled shaping,reference=fig:shape:cycle]
570\startcombination[nx=2,ny=2]
571    {\typesetbuffer[setup-0,demo-3][page=1,width=.4\textwidth,frame=on]}              {cycle, finite shape,    page 1}
572    {\typesetbuffer[setup-0,demo-3][page=2,width=.4\textwidth,frame=on]}              {cycle, finite shape,    page 2}
573    {\typesetbuffer[setup-0-repeat,demo-3-cycle][page=1,width=.4\textwidth,frame=on]} {cycle, repeat in shape, page 1}
574    {\typesetbuffer[setup-0-repeat,demo-3-cycle][page=2,width=.4\textwidth,frame=on]} {cycle, repeat in shape, page 2}
575\stopcombination
576\stopplacefigure
577
578These examples are probably too small to see the details but you can run them
579yourself or zoom in on the details. In the margin we show the values used. Here
580is a simple example of (non) poetry. There are other environments that can be
581used instead but this makes a good example anyway.
582
583\startbuffer
584\startparagraphshape[test]
585    left 0em right 0em
586    left 1em right 0em
587    repeat
588\stopparagraphshape
589
590\startshapedparagraph[list=test,method=cycle]
591    verse line 1.1\crlf verse line 2.1\crlf
592    verse line 3.1\crlf verse line 4.1\par
593    verse line 1.2\crlf verse line 2.2\crlf
594    verse line 3.2\crlf verse line 4.2\crlf
595    verse line 5.2\crlf verse line 6.2\par
596\stopshapedparagraph
597\stopbuffer
598
599\typebuffer[option=TEX]
600
601\start \getbuffer \stop
602
603Because the idea for this feature originates in \METAFUN, we will now kick in
604some \METAPOST. The following code creates a shape for a circle. We use a
6052mm offset here:
606
607\startbuffer
608\startuseMPgraphic{circle}
609    path p ; p := fullcircle scaled TextWidth ;
610    build_parshape(p,
611        2mm, 0, 0,
612        LineHeight, StrutHeight, StrutDepth, StrutHeight
613    ) ;
614\stopuseMPgraphic
615\stopbuffer
616
617\typebuffer[option=TEX]
618
619\start \getbuffer \stop
620
621We plug this into the already described macros:
622
623\startbuffer
624\startshapedparagraph[mp=circle]%
625    \setupalign[verytolerant,stretch,last]%
626    \samplefile{tufte}
627    \samplefile{tufte}
628\stopshapedparagraph
629\stopbuffer
630
631\typebuffer[option=TEX]
632
633And get ourself a circular shape. Watch out, at this moment the shape environment
634does not add grouping so when as in this case you change the alignment it can
635influence the document.
636
637\start \getbuffer \stop
638
639\startbuffer[framed]
640\framed[align=normal,width=\textwidth,offset=2mm,strut=no]\bgroup
641    \getbuffer
642\egroup
643\stopbuffer
644
645Assuming that the shape definition above is in a buffer we can do this:
646
647\typebuffer[option=TEX]
648
649The result is shown in \in {figure} [fig:shape:circle]. Because all action
650happens in the framed environment, we can also use this definition:
651
652\starttyping[option=TEX]
653\startuseMPgraphic{circle}
654    path p ; p := fullcircle scaled \the\dimexpr\framedwidth+\framedoffset*2\relax ;
655    build_parshape(p,
656        \framedoffset, 0, 0,
657        LineHeight, StrutHeight, StrutDepth, StrutHeight
658    ) ;
659    draw p ;
660\stopuseMPgraphic
661\stoptyping
662
663\startplacefigure[title=A framed circular shape,reference=fig:shape:circle]
664    \getbuffer[framed]
665\stopplacefigure
666
667A mechanism like this is often never completely automatic in the sense that you
668need to keep an eye on the results. Depending on user demands more features can
669be added. With weird shapes you might want to set up the alignment to be \type
670{tolerant} and have some \type {stretch}.
671
672The interface described in the \METAFUN\ manual is pretty old, the time stamp of
673the original code is mid 2000, but the principles didn't change. The examples in
674\type {meta-imp-txt.mkxl} can now be written as:
675
676\startuseMPgraphic{test 1}
677  begingroup ;
678    save p ; path p ; p := fullcircle scaled 6cm ;
679    lmt_parshape [
680        path        = p,
681        offset      = BodyFontSize/2,
682        dx          = 0,           % default
683        dy          = 0,           % default
684        lineheight  = LineHeight,  % default
685        strutheight = StrutHeight, % default
686        strutdepth  = StrutDepth,  % default
687        topskip     = StrutHeight, % default
688    ] ;
689    draw p withpen pencircle scaled 1pt ;
690  endgroup ;
691\stopuseMPgraphic
692
693\startuseMPgraphic{test 2}
694  begingroup ;
695    save p ; path p ; p := fullsquare rotated 45 scaled 5cm ;
696    lmt_parshape [
697        path   = p,
698        offset = BodyFontSize/2,
699        trace  = true,
700    ] ;
701    draw p withpen pencircle scaled 1pt ;
702  endgroup ;
703\stopuseMPgraphic
704
705\startuseMPgraphic{test 3}
706  begingroup ;
707    save w, h, p ; path p ; w := h := 6cm ;
708    p := (.5w,h) -- (   0,  h) -- (0,0) -- (w,0) &
709         (  w,0) .. (.75w,.5h) .. (w,h) &  (w,h) -- cycle ;
710    lmt_parshape [
711        path   = p,
712        offset = BodyFontSize/2,
713    ] ;
714    draw p withpen pencircle scaled 1pt ;
715  endgroup ;
716\stopuseMPgraphic
717
718\startuseMPgraphic{test 4}
719  begingroup ;
720    save d, p, q ; path p, q ; d := BodyFontSize/2;
721    vardef shape(expr w, h, o) =
722        (o,o) -- (w-o,o) & (w-o,o) .. (.75w-o,.5h) ..
723        (w-2o,h-o) & (w-2o,h-o) -- (o,h-o) -- cycle
724    enddef ;
725    p := shape(6cm, 6cm, d) ; q := shape(6cm, 6cm, 0) ;
726    lmt_parshape [
727        path       = p,
728        offsetpath = q,
729        dx         = d,
730        dy         = d,
731        trace      = true,
732    ] ;
733    draw q withpen pencircle scaled 1pt ;
734  endgroup ;
735\stopuseMPgraphic
736
737\defineoverlay[test 1][\useMPgraphic{test 1}]
738\defineoverlay[test 2][\useMPgraphic{test 2}]
739\defineoverlay[test 3][\useMPgraphic{test 3}]
740\defineoverlay[test 4][\useMPgraphic{test 4}]
741
742\startbuffer
743  \startshapetext[test 1,test 2,test 3,test 4]
744    \setupalign[verytolerant,stretch,normal]%
745    \samplefile{douglas} % Douglas R. Hofstadter
746  \stopshapetext
747  \startcombination[2*2]
748    {\framed[offset=overlay,frame=off,background=test 1]{\getshapetext}}
749        {test 1}
750    {\framed[offset=overlay,frame=off,background=test 2]{\getshapetext}}
751        {test 2}
752    {\framed[offset=overlay,frame=off,background=test 3]{\getshapetext}}
753        {test 3}
754    {\framed[offset=overlay,frame=off,background=test 4]{\getshapetext}}
755        {test 4}
756  \stopcombination
757\stopbuffer
758
759\typebuffer[option=TEX]
760
761In \in {figure} [fig:shapes:chain] we see the result. Watch how for two shapes
762we have enabled tracing. Of course you need to tweak till all fits well but we're
763talking of special situations anyway.
764
765\startplacefigure[Title=Multiple shapes,reference=fig:shapes:chain]
766    \getbuffer
767\stopplacefigure
768
769Here is a bit more extreme example. Again we use a circle:
770
771\startbuffer
772\startuseMPgraphic{circle}
773    lmt_parshape [
774        path       = fullcircle scaled 136mm,
775        offset     = 2mm,
776        bottomskip = - 1.5LineHeight,
777    ] ;
778\stopuseMPgraphic
779\stopbuffer
780
781\typebuffer[option=TEX]
782
783But we output a longer text:
784
785\startbuffer
786\startshapedparagraph[mp=circle,repeat=yes,method=cycle]%
787    \setupalign[verytolerant,stretch,last]\dontcomplain
788    {\darkred     \samplefile{tufte}}\par
789    {\darkgreen   \samplefile{tufte}}\par
790    {\darkblue    \samplefile{tufte}}\par
791    {\darkcyan    \samplefile{tufte}}\par
792    {\darkmagenta \samplefile{tufte}}\par
793\stopshapedparagraph
794\stopbuffer
795
796\typebuffer[option=TEX]
797
798We get a multi|-|page shape:
799
800\start \getbuffer \stop
801
802Compare this with:
803
804\startbuffer
805\startshapedparagraph[mp=circle,repeat=yes,method=cycle]%
806    \setupalign[verytolerant,stretch,last]\dontcomplain
807    {\darkred     \samplefile{tufte}}
808    {\darkgreen   \samplefile{tufte}}
809    {\darkblue    \samplefile{tufte}}
810    {\darkcyan    \samplefile{tufte}}
811    {\darkmagenta \samplefile{tufte}}
812\stopshapedparagraph
813\stopbuffer
814
815\typebuffer[option=TEX]
816
817Which gives:
818
819\start \getbuffer \stop
820
821Here the \type {bottomskip} takes care of subtle rounding issues as well as
822discarding the last line in the shape so that we get nicer continuation. There is
823no full automated solution for all you can come up with.
824
825Mixing a \METAPOST\ specification into a regular one is also possible. The next
826example demonstrates this as well as the option to remove some lines from a
827specification:
828
829\starttyping[option=TEX]
830\startparagraphshape[test]
831    left 0em right 0em
832    left 1em right 0em
833    metapost {circle}
834    delete 3
835    metapost {circle,circle,circle}
836    delete 7
837    metapost {circle}
838    repeat
839\stopparagraphshape
840\stoptyping
841
842You can combine a shape with narrowing a paragraph. Watch the \type {absolute}
843keyword in the next code. The result is shown in \in {figure} [fig:shape:skips].
844
845\startbuffer[demo-4]
846\startuseMPgraphic{circle}
847    lmt_parshape [
848        path       = fullcircle scaled TextWidth,
849        bottomskip = - 1.5LineHeight,
850    ] ;
851\stopuseMPgraphic
852
853\startparagraphshape[test-1]
854    metapost {circle} repeat
855\stopparagraphshape
856
857\startparagraphshape[test-2]
858    absolute left metapost {circle} repeat
859\stopparagraphshape
860
861\startparagraphshape[test-3]
862    absolute right metapost {circle} repeat
863\stopparagraphshape
864
865\startparagraphshape[test-4]
866    absolute both metapost {circle} repeat
867\stopparagraphshape
868
869\showframe
870
871\startnarrower[4*left,2*right]
872    \startshapedparagraph[list=test-1,repeat=yes,method=repeat]%
873        \setupalign[verytolerant,stretch,last]\dontcomplain
874        \dorecurse{3}{\samplefile{thuan}}
875    \stopshapedparagraph
876    \page
877    \startshapedparagraph[list=test-2,repeat=yes,method=repeat]%
878        \setupalign[verytolerant,stretch,last]\dontcomplain
879        \dorecurse{3}{\samplefile{thuan}}
880    \stopshapedparagraph
881    \page
882    \startshapedparagraph[list=test-3,repeat=yes,method=repeat]%
883        \setupalign[verytolerant,stretch,last]\dontcomplain
884        \dorecurse{3}{\samplefile{thuan}}
885    \stopshapedparagraph
886    \page
887    \startshapedparagraph[list=test-4,repeat=yes,method=repeat]%
888        \setupalign[verytolerant,stretch,last]\dontcomplain
889        \dorecurse{3}{\samplefile{thuan}}
890    \stopshapedparagraph
891\stopnarrower
892\stopbuffer
893
894\typebuffer[demo-4][option=TEX]
895
896\startplacefigure[title=Skip compensation,reference=fig:shape:skips]
897\startcombination[nx=2,ny=2]
898    {\typesetbuffer[demo-4][page=1,width=.4\textwidth,frame=on]} {test 1}
899    {\typesetbuffer[demo-4][page=2,width=.4\textwidth,frame=on]} {test 2, left}
900    {\typesetbuffer[demo-4][page=3,width=.4\textwidth,frame=on]} {test 3, right}
901    {\typesetbuffer[demo-4][page=4,width=.4\textwidth,frame=on]} {test 4, both}
902\stopcombination
903\stopplacefigure
904
905The shape mechanism has a few more tricks but these are really meant for usage
906in specific situations, where one knows what one deals with. The following
907examples are visualized in \in {figure} [fig:flow].
908
909\startbuffer[jano]
910\useMPlibrary[dum]
911\usemodule[article-basics]
912
913\startbuffer
914    \externalfigure[dummy][width=6cm]
915\stopbuffer
916
917\startshapedparagraph[text=\getbuffer]
918    \dorecurse{3}{\samplefile{ward}\par}
919\stopshapedparagraph
920
921\page
922
923\startshapedparagraph[text=\getbuffer,distance=1em]
924    \dorecurse{3}{\samplefile{ward}\par}
925\stopshapedparagraph
926
927\page
928
929\startshapedparagraph[text=\getbuffer,distance=1em,
930        hoffset=-2em]
931    \dorecurse{3}{\samplefile{ward}\par}
932\stopshapedparagraph
933
934\page
935
936\startshapedparagraph[text=\getbuffer,distance=1em,
937        voffset=-2ex,hoffset=-2em]
938    \dorecurse{3}{\samplefile{ward}\par}
939\stopshapedparagraph
940
941\page
942
943\startshapedparagraph[text=\getbuffer,distance=1em,
944        voffset=-2ex,hoffset=-2em,lines=1]
945    \dorecurse{3}{\samplefile{ward}\par}
946\stopshapedparagraph
947
948\page
949
950\startshapedparagraph[width=4cm,lines=4]
951    \dorecurse{3}{\samplefile{ward}\par}
952\stopshapedparagraph
953\stopbuffer
954
955\typebuffer[jano]
956
957\startplacefigure[title={Flow around something},reference=fig:flow]
958    \startcombination[nx=3,ny=2]
959        {\typesetbuffer[jano][page=1,frame=on,width=\measure{combination}]}{}
960        {\typesetbuffer[jano][page=2,frame=on,width=\measure{combination}]}{}
961        {\typesetbuffer[jano][page=3,frame=on,width=\measure{combination}]}{}
962        {\typesetbuffer[jano][page=4,frame=on,width=\measure{combination}]}{}
963        {\typesetbuffer[jano][page=5,frame=on,width=\measure{combination}]}{}
964        {\typesetbuffer[jano][page=6,frame=on,width=\measure{combination}]}{}
965    \stopcombination
966\stopplacefigure
967
968\stopsectionlevel
969
970\startsectionlevel[title=Modes]
971
972% \ruledvbox{1\ifhmode\writestatus{!}{HMODE 1}\fi}                               % hsize
973% \ruledvbox{\hbox{\strut 2}\ifhmode\writestatus{!}{HMODE 2}\fi}                 % fit
974% \ruledvbox{\hbox{\strut 3}\hbox{\strut 3}\ifhmode\writestatus{!}{HMODE 3}\fi}  % fit
975% \ruledvbox{\hbox{\strut 4}4\ifhmode\writestatus{!}{HMODE 4}\fi}                % hsize
976% \ruledvbox{\hbox{\strut 5}5\hbox{\strut 5}\ifhmode\writestatus{!}{HMODE 5}\fi} % hsize
977% \ruledvbox{6\hbox{\strut 6}\ifhmode\writestatus{!}{HMODE 6}\fi}                % hsize
978
979{\em todo: some of the side effects of so called modes}
980
981\stopsectionlevel
982
983\startsectionlevel[title=Leaders]
984
985Leaders are a basic feature that users probably never run into directly. They
986repeat content till it fits the specified width which can be stretched out. The
987content is typeset once and it is the backend that does the real work of
988repetition.
989
990\startbuffer
991\strut\leaders \hbox{!}\hfill\strut
992\strut\xleaders\hbox{!}\hfill\strut
993\strut\cleaders\hbox{!}\hfill\strut
994\strut\gleaders\hbox{!}\hfill\strut
995\stopbuffer
996
997\typebuffer
998
999Here \type {\leaders} starts at the left edge and are repeats the box as long as
1000it fits, \type {\xleaders} spreads till the edges and \type {\cleaders} centers
1001the lot. The \type {\gleaders} primitive (which is not in orginal \TEX) takes the
1002outer box as reference and further behaves like \type {\cleaders}.
1003
1004\startlines \showmakeup[line] \getbuffer \stoplines
1005
1006The leader primitives take box or rule but in \LUAMETATEX\ a glyph can also be
1007specified, which saves wrapping in a box.
1008
1009\startbuffer
1010\ruledvbox \bgroup \hsize 10cm
1011    \strut\cleaders\hbox{!}\hfill\strut
1012\egroup
1013
1014\ruledvbox \bgroup \hsize 10cm
1015    \strut\cleaders\hrule\hfill\strut
1016\egroup
1017
1018\ruledvbox \bgroup \hsize 10cm
1019    \strut\cleaders\glyph`!\hfill\strut
1020\egroup
1021\stopbuffer
1022
1023\typebuffer
1024
1025\getbuffer
1026
1027The \LUAMETATEX\ engine also introduced \type {\uleaders}
1028
1029\definecolor[tred]  [r=.6,a=1,t=.5]
1030\definecolor[tgreen][g=.6,a=1,t=.5]
1031\definecolor[tblue] [b=.6,a=1,t=.5]
1032
1033\startbuffer[one]
1034    x xx xxx xxxx
1035    \ruledhbox{L\hss R}\space
1036    x xx xxx xxxx
1037\stopbuffer
1038
1039\startbuffer[two]
1040    x xx xxx xxxx
1041    \uleaders\backgroundhbox[gray]{L\hss R}\hskip\zeropoint plus 100pt\relax\space
1042    x xx xxx xxxx
1043\stopbuffer
1044
1045\startbuffer[three]
1046    x xx xxx xxxx
1047    \uleaders\ruledhbox{L\hss R}\hskip\zeropoint plus 100pt\relax\space
1048    x xx xxx xxxx
1049\stopbuffer
1050
1051We show three boxes, a regular one first (red):
1052
1053\typebuffer[one]
1054
1055The second one (blue) is also a box but one that stretches upto 100pt and is in a
1056later stage, when the paragraph has been built, is repackaged to the effective
1057width. The third example (green) leaves out the background.
1058
1059\startlinecorrection
1060\startoverlay
1061    {\vbox{\color[tgreen]{\small\dorecurse {20} {\getbuffer[three]}}}}
1062    {\vbox{\color[tblue] {\small\dorecurse {20} {\getbuffer  [two]}}}}
1063    {\vbox{\color[tred]  {\small\dorecurse {20} {\getbuffer  [one]}}}}
1064\stopoverlay
1065\stoplinecorrection
1066
1067In \CONTEXT\ we have wrapped this feature in the adaptive box mechanism, so here
1068a few a few examples:
1069
1070\setupexternalfigures[location={default,local,global}]
1071
1072\startbuffer
1073\startsetups adaptive:test:a
1074    \setbox\usedadaptivebox\vbox to \usedadaptivetotal \bgroup
1075        \externalfigure
1076          [cow.pdf]
1077          [width=\framedmaxwidth,
1078           frame=on,
1079           height=\usedadaptivetotal]%
1080   \egroup
1081\stopsetups
1082
1083\startsetups adaptive:test:b
1084    \setbox\usedadaptivebox\vbox to \usedadaptivetotal \bgroup
1085        \externalfigure
1086          [cow.pdf]
1087          [width=\usedadaptivewidth,
1088           frame=on,
1089           height=\usedadaptivetotal]%
1090   \egroup
1091\stopsetups
1092\stopbuffer
1093
1094\typebuffer \getbuffer
1095
1096We use this as follows (see \in {figure} [fig:adaptive] for the result):
1097
1098\startbuffer
1099\framed[height=18cm,align=middle,adaptive=yes,top=,bottom=] {%
1100    \begstrut \samplefile{tufte} \endstrut
1101    \par
1102    \adaptivevbox
1103      [strut=yes,setups=adaptive:test:a]
1104      {\showstruts\strut\hsize5cm\hss}%
1105    \par
1106    \adaptivevbox
1107      [strut=yes,setups=adaptive:test:b]
1108      {\showstruts\strut\hsize5cm\hss}%
1109    \par
1110    \begstrut \samplefile{tufte} \endstrut
1111}
1112\stopbuffer
1113
1114\typebuffer
1115
1116\startplacefigure[reference=fig:adaptive]
1117    \getbuffer
1118\stopplacefigure
1119
1120Here is one that you can test yourself:
1121
1122\starttyping
1123\startsetups adaptive:test
1124    \setbox\usedadaptivebox\vbox to \usedadaptivetotal \bgroup
1125        \externalfigure
1126          [cow.pdf]
1127          [width=\usedadaptivewidth,
1128           height=\usedadaptivetotal]%
1129   \egroup
1130\stopsetups
1131
1132\ruledvbox to \textheight {
1133    \par \begstrut \samplefile{tufte} \endstrut \par
1134    \adaptivevbox[strut=yes,setups=adaptive:test]{\hsize\textwidth\hss}
1135    \par \begstrut \samplefile{tufte} \endstrut
1136}
1137\stoptyping
1138
1139The next example comes from the test suite (where it runs over many pages in
1140order to illustrate the idea):
1141
1142\startbuffer
1143\startMPdefinitions
1144    def TickTock =
1145        interim linecap := squared;
1146        save p ; path p ;
1147        p := fullsquare xysized(AdaptiveWidth,.9(AdaptiveHeight+AdaptiveDepth)) ;
1148        fill p withcolor AdaptiveColor ;
1149        draw bottomboundary (p enlarged (-AdaptiveThickness) )
1150            withdashes (3*AdaptiveThickness)
1151            withpen pencircle scaled AdaptiveThickness
1152            withcolor white ;
1153    enddef ;
1154\stopMPdefinitions
1155
1156\startsetups adaptive:test
1157    \setbox\usedadaptivebox\hbox
1158        to         \usedadaptivewidth
1159        yoffset -.9\usedadaptivedepth
1160    \bgroup
1161        \hss
1162        \startMPcode
1163            TickTock ;
1164        \stopMPcode
1165        \hss
1166   \egroup
1167\stopsetups
1168
1169\definecolor[adaptive:tick][.25(blue,green)]
1170\definecolor[adaptive:tock][.75(blue,green)]
1171
1172\defineadaptive
1173  [tick]
1174  [setups=adaptive:test,
1175   color=adaptive:tick,
1176   foregroundcolor=white,
1177   foregroundstyle=\infofont,
1178   strut=yes]
1179
1180\defineadaptive
1181  [tock]
1182  [tick]
1183  [color=adaptive:tock]
1184
1185\dostepwiserecurse{8}{12}{1}{%
1186    \dostepwiserecurse{5}{15}{1}{%
1187        this~#1.##1 is~#1.##1 test~#1.##1
1188        \ifodd##1\relax
1189            \adaptivebox[tick]{\hss tick #1.##1\hss}
1190        \else
1191            \adaptivebox[tock]{\hss tock #1.##1\hss}
1192        \fi
1193    }
1194}
1195\stopbuffer
1196
1197\typebuffer \getbuffer
1198
1199In the next example the graphics adapt to the available space:
1200
1201\startbuffer
1202\startsetups adaptive:test
1203    \setbox\usedadaptivebox\hbox
1204        to       \usedadaptivewidth
1205        yoffset -\usedadaptivedepth
1206    \bgroup
1207        \externalfigure
1208          [cow.pdf]
1209          [width=\usedadaptivewidth,
1210           height=\dimexpr\usedadaptivetotal\relax]%
1211    \egroup
1212\stopsetups
1213
1214\dostepwiserecurse{1}{50}{1}{%
1215    this~#1 is~#1 test~#1
1216    {\adaptivebox[strut=yes,setups=adaptive:test]{}}
1217}
1218\stopbuffer
1219
1220\typebuffer \getbuffer
1221
1222\stopsectionlevel
1223
1224\startsectionlevel[title=Prevdepth]
1225
1226The depth of a box is normally positive but rules can have a negative depth in
1227order to get a rule above the baseline. When \TEX\ was written the assumption was
1228that a negative depth of more than 1000 point made no sense at all. The last
1229depth on a vertical list is registered in the \type {\prevdepth} variable. This
1230is basically a reference into the current list. In order to illustrate some
1231interesting side effects of setting this \type {\prevdepth} and especially when
1232we set it to $\tf -1000pt$. In order to illustrate this this special value can be set
1233to a different value in \LUAMETATEX. However, as dealing with the property is
1234somewhat special in the engine you should not set it unless you know that the
1235macro package is ware of it.
1236
1237\startbuffer
1238line 1\par line 2 \par \nointerlineskip line 3 \par
1239\stopbuffer
1240
1241\typebuffer
1242
1243Assuming that we haven't set any inter paragraph spacing this gives:
1244
1245\startlinecorrection
1246\ruledvbox{\setupwhitespace[none]\showmakeup[line]\getbuffer}
1247\stoplinecorrection
1248
1249Here \type {\nointerlineskip} is (normally) defined as:
1250
1251\starttyping
1252\prevdepth-1000pt
1253\stoptyping
1254
1255although in \CONTEXT\ we use \typ {\ignoredepthcriterion} instead of the hard
1256coded dimension. We now give a more extensive example:
1257
1258\startbuffer[definition-1]
1259\def\PrevTest#1%
1260  {\setbox0\ruledhbox{\strut$\tf#1$}%
1261   \dp0=#1
1262   \vbox\bgroup\hsize4em
1263     FIRST\par
1264     \unhbox0\par
1265     LAST\par
1266   \egroup}
1267\stopbuffer
1268
1269\startbuffer[definition-2]
1270\def\PrevTest#1%
1271  {\setbox0\ruledhbox{\strut$\tf#1$}%
1272   \dp0=#1
1273   \vbox\bgroup
1274     \ruledhbox{FIRST}\par
1275     \box0\par
1276     \ruledhbox{LAST}\par
1277   \egroup}
1278\stopbuffer
1279
1280\startbuffer[example]
1281\ruledhbox \bgroup
1282    \PrevTest{-10.0pt}\quad
1283    \PrevTest{-20.0pt}\quad
1284    \PrevTest{-49.9pt}\quad
1285    \PrevTest{-50.0pt}\quad
1286    \PrevTest{-50.1pt}\quad
1287    \PrevTest{-60.0pt}\quad
1288    \PrevTest{-80.0pt}%
1289\egroup
1290\stopbuffer
1291
1292\typebuffer[example][option=TEX]
1293
1294In this example we set \typ {\ignoredepthcriterion} to $\tf -50.0pt$ instead of the
1295normal $\tf -1000pt$. The helper is defined as:
1296
1297\typebuffer[definition-1][option=TEX]
1298
1299or
1300
1301\typebuffer[definition-2][option=TEX]
1302
1303The result is shown in \in {figures} [fig:prevdepth-1] \in {and}
1304[fig:prevdepth-2]. The first case is what we normally have in text and we haven't
1305set \type {\prevdepth} explicitly between lines so \TEX\ will just look at the
1306depth of the lines. In the second case the depth is ignored when less than the
1307criterion which is why, when we set the depth of the box to a negative value we
1308get somewhat interesting skips.
1309
1310\startplacefigure[reference=fig:prevdepth-1]
1311    \showmakeup[line]
1312    \ignoredepthcriterion-50pt
1313    \setupwhitespace[none]
1314    \getbuffer[definition-1,example]
1315\stopplacefigure
1316
1317\startplacefigure[reference=fig:prevdepth-2]
1318    \showmakeup[line]
1319    \ignoredepthcriterion-50pt
1320    \setupwhitespace[none]
1321    \getbuffer[definition-2,example]
1322    \blank[5*line]
1323\stopplacefigure
1324
1325I'm sure one can use this effect otherwise than intended but I doubt is any user
1326is willing to do this but the fact that we can lower the criterion makes for nice
1327experiments. Just for the record, in \in {figure} [fig:prevdepth-3] you see what
1328we get with positive values:
1329
1330\startbuffer[example]
1331\ruledhbox \bgroup
1332    \PrevTest{10.0pt}\quad
1333    \PrevTest{20.0pt}\quad
1334    \PrevTest{49.9pt}\quad
1335    \PrevTest{50.0pt}\quad
1336    \PrevTest{50.1pt}\quad
1337    \PrevTest{60.0pt}\quad
1338    \PrevTest{80.0pt}%
1339\egroup
1340\stopbuffer
1341
1342\typebuffer[example][option=TEX]
1343
1344\startplacefigure[reference=fig:prevdepth-3]
1345    \showmakeup[line]
1346    \ignoredepthcriterion50pt
1347    \setupwhitespace[none]
1348    \getbuffer[definition-2,example]
1349\stopplacefigure
1350
1351Watch the interline skip kicking in when we make the depth larger than in
1352\typ {\ignoredepthcriterion} being $\tf 50pt$.
1353
1354\stopsectionlevel
1355
1356\startsectionlevel[title=Normalization]
1357
1358{\em todo: users don't need to bother about this but it might be interesting anyway}
1359
1360\stopsectionlevel
1361
1362\startsectionlevel[title=Dirty tricks]
1363
1364{\em todo: explain example for combining paragraphs}
1365
1366% test\wrapuppar{\crlf\ignorepars}
1367%
1368% test
1369%
1370% test\wrapuppar{\ignorepars}
1371%
1372% test
1373%
1374% test\wrapuppar{\removeunwantedspaces\ignorepars}
1375%
1376% test
1377
1378\stopsectionlevel
1379
1380\startsectionlevel[title=Penalties]
1381
1382In \in {figiure} [fig:penaltylists] we demonstrate the (accumulated) effect of a
1383few penalty arrays that you can set. Keep in mind that these are low level
1384(primitive) commands that can interfere with other mechanisms. The
1385interline penalties are applied to the current paragraph in the same was as \typ
1386{\looseness} is. This makes sense because using the club and widow penalties is
1387more predictable. You have to zoom in to see how the penalties add up. As with
1388\typ {\looseness} the \typ {\interlinepenalties} get reset before the paragraph,
1389which is shown in the bottom cells of this figure.
1390
1391\startbuffer
1392\interlinepenalty 0
1393\clubpenalty      0
1394\widowpenalty     0
1395\orphanpenalty    0
1396\shapingpenalty   0
1397
1398\clubpenalties      5   1000   2000   3000   4000   5000 % 6 -> 0
1399\widowpenalties     5     10     20     30     40     50 % 6 -> 0
1400\orphanpenalties    5      1      2      3      4      5 % 6 -> 0
1401\interlinepenalties 5 100000 200000 300000 400000 500000 % 6 -> 0
1402\stopbuffer
1403
1404\typebuffer
1405
1406It actually makes sense to explicitly zero the last entry because as you can see
1407in the figure the last one gets used when we run out of entries.
1408
1409\startplacefigure[title=Penalty lists,reference=fig:penaltylists]
1410\startcombination[nx=2,ny=2]
1411  {\ruledvbox\bgroup
1412     \hsize.45tw
1413     \showmakeup[penalty]
1414     \getbuffer
1415     \samplefile{ward} \par
1416     \samplefile{ward} \par
1417   \egroup} {\type{\normalizeparmode 8}}
1418  {\ruledvbox\bgroup
1419     \hsize.45tw
1420     \bitwiseflip\normalizeparmode\pluseight
1421     \showmakeup[penalty]
1422     \getbuffer
1423     \samplefile{ward} \par
1424     \samplefile{ward} \par
1425   \egroup} {\type{\normalizeparmode 8}}
1426  {\ruledvbox\bgroup
1427     \hsize.45tw
1428     \showmakeup[penalty]
1429     \getbuffer
1430
1431     \samplefile{ward} \par
1432     \samplefile{ward} \par
1433   \egroup} {newline and \type{\normalizeparmode 8}}
1434  {\ruledvbox\bgroup
1435     \hsize.45tw
1436     \bitwiseflip\normalizeparmode\pluseight
1437     \showmakeup[penalty]
1438     \getbuffer
1439
1440     \samplefile{ward} \par
1441     \samplefile{ward} \par
1442   \egroup} {newline and \type{\normalizeparmode 8}}
1443\stopcombination
1444\stopplacefigure
1445
1446Can you guess what the next specification does?
1447
1448\starttyping
1449\widowpenalties 3 options \largestspecificationoptioncode 3000 2000 1000
1450\clubpenalties  3 options \largestspecificationoptioncode   30   20   10
1451\stoptyping
1452
1453\stopsectionlevel
1454
1455\startsectionlevel[title=Par passes]
1456
1457Everything comes together in what we call par passes. Before we explain them first
1458something about a feature that makes setting up for instance \type {\widowpenalties}
1459easier. Here are a few definitions:
1460
1461\starttyping
1462\specificationdef\strictwidowpenalties      \widowpenalties  \plusthree
1463    \maxcount \maxcount \zerocount \relax
1464\specificationdef\strictwidowpenaltiestwo   \widowpenalties  \plustwo
1465    \maxcount \zerocount \relax
1466\specificationdef\strictwidowpenaltiesthree \widowpenalties  \plusthree
1467    \maxcount \maxcount \zerocount \relax
1468\specificationdef\strictwidowpenaltiesfour  \widowpenalties  \plusfour
1469    \maxcount \maxcount \maxcount \zerocount \relax
1470\stoptyping
1471
1472These are defined in the core and hooked into the alignment interface:
1473
1474\starttyping
1475\installaligncommand{strictwidows}  {\strictwidowpenalties     }
1476\installaligncommand{strictwidows:2}{\strictwidowpenaltiestwo  }
1477\installaligncommand{strictwidows:3}{\strictwidowpenaltiesthree}
1478\installaligncommand{strictwidows:4}{\strictwidowpenaltiesfour }
1479\stoptyping
1480
1481We also have four such \quote {strict} definitions for club but only one for
1482broken penalties. For orphan penalties we have four \quote {less} orphan
1483penalties but for widow, club and broken we have only one. So we end up with \typ
1484{lessorphans}, \typ {lessorphans:2}, \typ {lessorphans:3}, \typ {lessorphans:4},
1485\typ {defaultwidows}, \typ {defaultclubs}, \typ {defaultbroken}, \typ
1486{strictwidows}, \typ {strictwidows:2}, \typ {strictwidows:3}, \typ
1487{strictwidows:4}, \typ {strictclubs}, \typ {strictclubs:2}, \typ {strictclubs:3},
1488\typ {strictclubs:4} and \typ {strictbroken}.
1489
1490You can also use \type {\specificationdef} for other constructs that have this
1491multiple variable setup. Now to par passes. This is a mechanism unique to
1492\LUAMETATEX\ that permits more than the usual upto three paragraph break steps:
1493pretolerance, tolerance and emergency. How this works is explained in detail in
1494the paragraphs chapter in the \type {beyond} document.
1495
1496{\em todo: copy some from article when published}
1497
1498\stopsectionlevel
1499
1500\stopdocument
1501
1502% I rewarded myself after writing a section by watching the video "Final Thing On
1503% My Mind", The Pineapple Thief, Live, 2020, the usual perfect GH performance,
1504% wondering if live would turn to normal so that we could go to such concerts once
1505% again given successive covids. Writing manuals can do with a distraction.
1506%
1507% Gavin Harrison: Soundcheck, Drummerworld Jan 27, 2021 ... I wish I could make
1508% something called a check into pefect solo. Okay, another section and I'll check
1509% out the latest Simon Phillips and other favourite dummer uploads.
1510
1511% todo:
1512
1513% \everyhbox \everyvbox : useless unless one resets
1514% \parattr
1515% \snapshotpar
1516% \wrapuppar
1517
1518% \normalizelinemode0
1519% x\ruledhbox{\parindent 1cm\indent x \indent x}
1520% \normalizelinemode \parindentskipnormalizecode
1521% x\ruledhbox{\parindent 1cm\indent x \indent x}
1522
1523% \starttext
1524%     \tracingoutput1 \tracingonline1
1525%     \pretolerance9000 test \pretolerance 8000 test \par
1526%     \pretolerance9000 test \pretolerance 7000 \updateparagraphproperties test \par
1527%     \pretolerance9000 test \pretolerance 6000 \snapshotpar\tolerancefrozenparcode test \par
1528%     \pretolerance9000 test {\pretolerance5000 \snapshotpar\ntolerancefrozenparcode}test \par
1529% \stoptext
1530
1531% \par[newgraf][16=1,17=1], .... pretolerance 9000, ....
1532% \par[newgraf][16=1,17=1], .... pretolerance 7000, ....
1533% \par[newgraf][16=1,17=1], .... pretolerance 6000, ....
1534
1535% \parfillleftskip
1536
1537% todo:
1538
1539% \starttext
1540%
1541% \showframe
1542%
1543% % this is the current default behaviour; we have a topskip with quite some stretch
1544% % and that stretch is taken into account when tex calculates the badness; watch
1545% % the next page
1546%
1547% \dorecurse{4}{
1548%     \bgroup
1549%         \shapingpenaltiesmode\zerocount
1550%         \showmakeup[penalty]
1551%         \placefigure[left]{}{}
1552%         \input tufte \par
1553%     \egroup
1554% }
1555%
1556% \page EMPTY \page
1557%
1558% % the ntop is new and when >= 0 it is a multiplier for the number of lines
1559% % stretch, here we have one line; watch the next page
1560%
1561% \dorecurse{4}{
1562%     \bgroup
1563%         \shapingpenaltiesmode\zerocount
1564%         \setupinterlinespace[ntop=1]
1565%         \showmakeup[penalty]
1566%         \placefigure[left]{}{}
1567%         \input tufte \par
1568%     \egroup
1569% }
1570%
1571% \page EMPTY \page
1572%
1573% % these shaping parameters will discourage a linebreak within a shape;
1574% % the placement macro checked available space but that doesn't mean that
1575% % there can be no linebreak
1576%
1577% \dorecurse{4}{
1578%     \bgroup
1579%         \showmakeup[penalty]
1580%         \shapingpenalty\plushundred
1581%         \shapingpenaltiesmode"FF\relax
1582%         \placefigure[left]{}{}
1583%         \input tufte \par
1584%     \egroup
1585% }
1586%
1587% \page EMPTY \page
1588%
1589% % the next pages show the effect of changed topskip on page breaks with
1590% % footnotes
1591%
1592% \dorecurse{10}{
1593%     [#1] \input tufte \footnote{\input{tufte}} \par
1594% }
1595%
1596% \page EMPTY \page
1597%
1598% \setupinterlinespace[ntop=2]
1599%
1600% \dorecurse{10}{
1601%     [#1] \input tufte \footnote{\input{tufte}} \par
1602% }
1603%
1604% \page EMPTY \page
1605%
1606% \setupinterlinespace[ntop=0]
1607%
1608% \dorecurse{10}{
1609%     [#1] \input tufte \footnote{\input{tufte}} \par
1610% }
1611%
1612% \stoptext
1613
1614