hybrid-grouping.tex /size: 15 Kb    last modification: 2020-07-01 14:35
1\startcomponent hybrid-grouping
2
3\startbuffer[MyAbstract]
4\StartAbstract
5    In this article I will discuss a few things that are hard to do in
6    traditional \TEX, but reasonable well in \LUATEX.
7\StopAbstract
8\stopbuffer
9
10\doifmodeelse {tugboat} {
11    \usemodule[tug-01,abr-01]
12    \setvariables
13      [tugboat]
14      [columns=yes]
15    \setvariables
16      [tugboat]
17      [year=2010,
18       volume=99,
19       number=9,
20       page=99]
21    \setvariables
22      [tugboat]
23      [title=Grouping,
24       subtitle=A few things you can do with LUATEX,
25       keywords=,
26       author=Hans Hagen,
27       address=PRAGMA ADE\\Ridderstraat 27\\8061GH Hasselt NL,
28       email=pragma@xs4all.nl]
29    %
30    % we use a buffer as abstract themselves are buffers and
31    % inside macros we loose line endings and such
32    \getbuffer[MyAbstract]
33    %
34    \StartArticle
35} {
36    \environment hybrid-environment
37    \startchapter[title={Grouping}]
38}
39
40\setupbars[rulethickness=.15] % nicer
41
42\startsection [title={Variants}]
43
44After using \TEX\ for a while you get accustomed to one of its interesting
45concepts: grouping. Programming languages like \PASCAL\ and \MODULA\ have
46keywords \type {begin} and \type {end}. So, one can say:
47
48\starttyping
49if test then begin
50    print_bold("test 1")
51    print_bold("test 2")
52end
53\stoptyping
54
55Other languages provide a syntax like:
56
57\starttyping
58if test {
59    print_bold("test 1")
60    print_bold("test 2")
61}
62\stoptyping
63
64So, in those languages the \type {begin} and \type {end} and|/|or the curly
65braces define a \quote {group} of statements. In \TEX\ on the other hand we have:
66
67\starttyping
68test \begingroup \bf test \endgroup test
69\stoptyping
70
71Here the second \type {test} comes out in a bold font and the switch to bold
72(basically a different font is selected) is reverted after the group is closed.
73So, in \TEX\ grouping deals with scope and not with grouping things together.
74
75In other languages it depends on the language of locally defined variables are
76visible afterwards but in \TEX\ they're really local unless a \type {\global}
77prefix (or one of the shortcuts) is used.
78
79In languages like \LUA\ we have constructs like:
80
81\starttyping
82for i=1,100 do
83    local j = i + 20
84    ...
85end
86\stoptyping
87
88Here \type {j} is visible after the loop ends unless prefixed by \type {local}.
89Yet another example is \METAPOST:
90
91\starttyping
92begingroup ;
93    save n ; numeric n ; n := 10 ;
94    ...
95endgroup ;
96\stoptyping
97
98Here all variables are global unless they are explicitly saved inside a group.
99This makes perfect sense as the resulting graphic also has a global (accumulated)
100property. In practice one rarely needs grouping, contrary to \TEX\ where one
101really wants to keep changes local, if only because document content is so
102unpredictable that one never knows when some change in state happens.
103
104In principle it is possible to carry over information across a group boundary.
105Consider this somewhat unrealistic example:
106
107\starttyping
108\begingroup
109    \leftskip 10pt
110    \begingroup
111        ....
112        \advance\leftskip 10pt
113        ....
114    \endgroup
115\endgroup
116\stoptyping
117
118How do we carry the advanced leftskip over the group boundary without using a
119global assignment which could have more drastic side effects? Here is the trick:
120
121\starttyping
122\begingroup
123    \leftskip 10pt
124    \begingroup
125        ....
126        \advance\leftskip 10pt
127        ....
128        \expandafter
129    \endgroup
130    \expandafter \leftskip \the\leftskip
131\endgroup
132\stoptyping
133
134This is typical the kind of code that gives new users the creeps but normally
135they never have to do that kind of coding. Also, that kind of tricks assumes that
136one knows how many groups are involved.
137
138\stopsection
139
140\startsection [title={Implication}]
141
142What does this all have to do with \LUATEX\ and \MKIV ? The user interface of
143\CONTEXT\ provide lots of commands like:
144
145\starttyping
146\setupthis[style=bold]
147\setupthat[color=green]
148\stoptyping
149
150Most of them obey grouping. However, consider a situation where we use \LUA\ code
151to deal with some aspect of typesetting, for instance numbering lines or adding
152ornamental elements to the text. In \CONTEXT\ we flag such actions with
153attributes and often the real action takes place a bit later, for instance when a
154paragraph or page becomes available.
155
156A comparable pure \TEX\ example is the following:
157
158\starttyping
159{test test \bf test \leftskip10pt test}
160\stoptyping
161
162Here the switch to bold happens as expected but no leftskip of 10pt is applied.
163This is because the set value is already forgotten when the paragraph is actually
164typeset. So in fact we'd need:
165
166\starttyping
167{test test \bf test \leftskip10pt test \par}
168\stoptyping
169
170Now, say that we have:
171
172\starttyping
173{test test test \setupflag[option=1] \flagnexttext test}
174\stoptyping
175
176We flag some text (using an attribute) and expect it to get a treatment where
177option~1 is used. However, the real action might take place when \TEX\ deals with
178the paragraph or page and by that time the specific option is already forgotten
179or it might have gotten another value. So, the rather natural \TEX\ grouping does
180not work out that well in a hybrid situation.
181
182As the user interface assumes a consistent behaviour we cannot simply make these
183settings global even if this makes much sense in practice. One solution is to
184carry the information with the flagged text i.e.\ associate it somehow in the
185attribute's value. Of course, as we never know in advance when this information
186is used, this might result in quite some states being stored persistently.
187
188A side effect of this \quote {problem} is that new commands might get suboptimal
189user interfaces (especially inheritance or cloning of constructs) that are
190somewhat driven by these \quote {limitations}. Of course we may wonder if the end
191user will notice this.
192
193To summarize this far, we have three sorts of grouping to deal with:
194
195\startitemize[item]
196
197\startitem
198    \TEX's normal grouping model limits its scope to the local situation and
199    normally has only direct and local consequences. We cannot carry information
200    over groups.
201\stopitem
202
203\startitem
204    Some of \TEX's properties are applied later, for instance when a paragraph or
205    page is typeset and in order to make \quote {local} changes effective, the
206    user needs to add explicit paragraph ending commands (like \type {\par} or
207    \type {\page}).
208\stopitem
209
210\startitem
211    Features dealt with asynchronously by \LUA\ are at that time unaware of
212    grouping and variables set that were active at the time the feature was
213    triggered so there we need to make sure that our settings travel with the
214    feature. There is not much that a user can do about it as this kind of
215    management has to be done by the feature itself.
216\stopitem
217
218\stopitemize
219
220\stopsection
221
222It is the third case that we will give an example of in the next section. We
223leave it up to the user if it gets noticed on the user interface.
224
225\startsection [title={An example}]
226
227A group of commands that has been reimplemented using a hybrid solution is
228underlining or more generic: bars. Just take a look at the following examples and
229try to get an idea on how to deal with grouping. Keep in mind that:
230
231\startitemize[packed]
232\startitem
233    Colors are attributes and are resolved in the backend, so way after the
234    paragraph has been typeset.
235\stopitem
236\startitem
237    Overstrike is also handled by an attribute and gets applied in the backend as
238    well, before colors are applied.
239\stopitem
240\startitem
241    Nested overstrikes might have different settings.
242\stopitem
243\startitem
244    An overstrike rule either inherits from the text or has its own color
245    setting.
246\stopitem
247\stopitemize
248
249First an example where we inherit color from the text:
250
251\startbuffer
252\definecolor[myblue][b=.75]
253\definebar[myoverstrike][overstrike][color=]
254
255Test \myoverstrike{%
256    Test \myoverstrike{\myblue
257        Test \myoverstrike{Test}
258        Test}
259    Test}
260Test
261\stopbuffer
262
263\typebuffer \getbuffer
264
265Because color is also implemented using attributes and processed later on we can
266access that information when we deal with the bar.
267
268The following example has its own color setting:
269
270\startbuffer
271\definecolor[myblue][b=.75]
272\definecolor[myred] [r=.75]
273\definebar[myoverstrike][overstrike][color=myred]
274
275Test \myoverstrike{%
276    Test \myoverstrike{\myblue
277        Test \myoverstrike{Test}
278        Test}
279    Test}
280Test
281\stopbuffer
282
283\typebuffer \getbuffer
284
285See how we can color the levels differently:
286
287\startbuffer
288\definecolor[myblue] [b=.75]
289\definecolor[myred]  [r=.75]
290\definecolor[mygreen][g=.75]
291
292\definebar[myoverstrike:1][overstrike][color=myblue]
293\definebar[myoverstrike:2][overstrike][color=myred]
294\definebar[myoverstrike:3][overstrike][color=mygreen]
295
296Test \myoverstrike{%
297    Test \myoverstrike{%
298        Test \myoverstrike{Test}
299        Test}
300    Test}
301Test
302\stopbuffer
303
304\typebuffer \getbuffer
305
306Watch this:
307
308\startbuffer
309\definecolor[myblue] [b=.75]
310\definecolor[myred]  [r=.75]
311\definecolor[mygreen][g=.75]
312
313\definebar[myoverstrike][overstrike][max=1,dy=0,offset=.5]
314\definebar[myoverstrike:1][myoverstrike][color=myblue]
315\definebar[myoverstrike:2][myoverstrike][color=myred]
316\definebar[myoverstrike:3][myoverstrike][color=mygreen]
317
318Test \myoverstrike{%
319    Test \myoverstrike{%
320        Test \myoverstrike{Test}
321        Test}
322    Test}
323Test
324\stopbuffer
325
326\typebuffer \getbuffer
327
328It this the perfect user interface? Probably not, but at least it keeps the
329implementation quite simple.
330
331The behaviour of the \MKIV\ implementation is roughly the same as in \MKII,
332although now we specify the dimensions and placement in terms of the ratio of the
333x-height of the current font.
334
335\startbuffer
336Test \overstrike{Test \overstrike{Test \overstrike{Test} Test} Test} Test \blank
337Test \underbar  {Test \underbar  {Test \underbar  {Test} Test} Test} Test \blank
338Test \overbar   {Test \overbar   {Test \overbar   {Test} Test} Test} Test \blank
339Test \underbar  {Test \overbar   {Test \overstrike{Test} Test} Test} Test \blank
340\stopbuffer
341
342\typebuffer \getbuffer
343
344As an extra this mechanism can also provide simple backgrounds. The normal
345background mechanism uses \METAPOST\ and the advantage is that we can use
346arbitrary shapes but it also carries some limitations. When the development of
347\LUATEX\ is a bit further along the road I will add the possibility to use
348\METAPOST\ shapes in this mechanism.
349
350Before we come to backgrounds, first take a look at these examples:
351
352\startbuffer
353\startbar[underbar]  \input zapf \stopbar \blank
354\startbar[underbars] \input zapf \stopbar \blank
355\stopbuffer
356
357\typebuffer \getbuffer
358
359First notice that it is no problem to span multiple lines and that hyphenation is
360not influenced at all. Second you can see that continuous rules are also
361possible. From such a continuous rule to a background is a small step:
362
363\startbuffer
364\definebar
365  [backbar]
366  [offset=1.5,rulethickness=2.8,color=blue,
367   continue=yes,order=background]
368
369\definebar
370  [forebar]
371  [offset=1.5,rulethickness=2.8,color=blue,
372   continue=yes,order=foreground]
373\stopbuffer
374
375\typebuffer \getbuffer
376
377The following example code looks messy but this has to do with the fact that we
378want properly spaced sample injection.
379
380\startbuffer
381from here
382    \startcolor[white]%
383        \startbar[backbar]%
384            \input zapf
385            \removeunwantedspaces
386        \stopbar
387    \stopcolor
388\space till here
389\blank
390from here
391    \startbar[forebar]%
392        \input zapf
393        \removeunwantedspaces
394    \stopbar
395\space till here
396\stopbuffer
397
398\typebuffer \getbuffer
399
400Watch how we can use the order to hide content. By default rules are drawn on top
401of the text.
402
403Nice effects can be accomplished with transparencies:
404
405\startbuffer
406\definecolor [tblue] [b=.5,t=.25,a=1]
407\setupbars [backbar] [color=tblue]
408\setupbars [forebar] [color=tblue]
409\stopbuffer
410
411\typebuffer \getbuffer
412
413We use as example:
414
415\startbuffer[sample]
416from here {\white \backbar{test test}
417    \backbar {nested nested} \backbar{also also}} till here
418from here {\white \backbar{test test
419    \backbar {nested nested}          also also}} till here
420from here {\white \backbar{test test
421    \backbar {nested nested}          also also}} till here
422\stopbuffer
423
424\typebuffer[sample] \getbuffer[sample]
425
426The darker nested variant is just the result of two transparent bars on top of
427each other. We can limit stacking, for instance:
428
429\startbuffer
430\setupbars[backbar][max=1]
431\setupbars[forebar][max=1]
432\stopbuffer
433
434\typebuffer \getbuffer
435
436This gives
437
438\getbuffer[sample]
439
440There are currently some limitations that are mostly due to the fact that we use
441only one attribute for this feature and a change in value triggers another
442handling. So, we have no real nesting here.
443
444The default commands are defined as follows:
445
446\starttyping
447\definebar[overstrike]  [method=0,dy= 0.4,offset= 0.5]
448\definebar[underbar]    [method=1,dy=-0.4,offset=-0.3]
449\definebar[overbar]     [method=1,dy= 0.4,offset= 1.8]
450
451\definebar[overstrikes] [overstrike] [continue=yes]
452\definebar[underbars]   [underbar]   [continue=yes]
453\definebar[overbars]    [overbar]    [continue=yes]
454\stoptyping
455
456As the implementation is rather non|-|intrusive you can use bars
457almost everywhere. You can underbar a whole document but equally
458well you can stick to fooling around with for instance formulas.
459
460\startbuffer
461\definecolor [tred]   [r=.5,t=.25,a=1]
462\definecolor [tgreen] [g=.5,t=.25,a=1]
463\definecolor [tblue]  [b=.5,t=.25,a=1]
464
465\definebar [mathred]   [backbar] [color=tred]
466\definebar [mathgreen] [backbar] [color=tgreen]
467\definebar [mathblue]  [backbar] [color=tblue]
468
469\startformula
470    \mathred{e} = \mathgreen{\white mc} ^ {\mathblue{\white e}}
471\stopformula
472\stopbuffer
473
474\typebuffer
475
476We get:
477
478\getbuffer
479
480We started this chapter with some words on grouping. In the examples you see no
481difference between adding bars and for instance applying color. However you need
482to keep in mind that this is only because behind the screens we keep the current
483settings along with the attribute. In practice this is only noticeable when you
484do lots of (local) changes to the settings. Take:
485
486\starttyping
487{test test test \setupbars[color=red] \underbar{test} test}
488\stoptyping
489
490This results in a local change in settings, which in turn will associate a new
491attribute to \type {\underbar}. So, in fact the following underbar becomes a
492different one than previous underbars. When the page is prepared, the unique
493attribute value will relate to those settings. Of course there are more
494mechanisms where such associations take place.
495
496\stopsection
497
498\startsection [title={More to come}]
499
500Is this all there is? No, as usual the underlying mechanisms can be used for
501other purposes as well. Take for instance inline notes:
502
503\startbuffer
504According to the wikipedia this is the longest English word:
505pneumonoultramicroscopicsilicovolcanoconiosis~\shiftup {other long
506words are pseudopseudohypoparathyroidism and
507flocci­nauci­nihili­pili­fication}. Of course in languages like Dutch and
508German we can make arbitrary long words by pasting words together.
509\stopbuffer
510
511\typebuffer
512
513This will produce:
514
515\getbuffer
516
517I wonder when users really start using such features.
518
519\stopsection
520
521\startsection [title={Summary}]
522
523Although under the hood the \MKIV\ bar commands are quite different from their
524\MKII\ counterparts users probably won't notice much difference at first sight.
525However, the new implementation does not interfere with the par builder and other
526mechanisms. Plus, it is configurable and it offers more functionality. However,
527as it is processed rather delayed, side effects might occur that are not
528foreseen.
529
530So, if you ever notice such unexpected side effects, you know where it might
531result from: what you asked for is processed much later and by then the
532circumstances might have changed. If you suspect that it relates to grouping
533there is a simple remedy: define a new bar command in the document preamble
534instead of changing properties mid|-|document. After all, you are supposed to
535separate rendering and content in the first place.
536
537\stopsection
538
539\doifmodeelse {tugboat} {
540    \StopArticle
541} {
542    \stopchapter
543}
544
545\stopcomponent
546