Source Browser ?

1% language=us runpath=texruns:manuals/ontarget
2
3\startcomponent ontarget-makesnosense
4
5\environment ontarget-style
6
7\startchapter[title={Not all makes sense}]
8
9The development of \CONTEXT\ is to a large extend driven by users with a wide
10variety of background and usage. I can safely say that much time spent on
11\CONTEXT\ qualifies as hobby (or maybe even more by curiosity). Of course I do
12use it myself but personally I never make advanced documents. I'm not a writer,
13nor an artist, nor a typesetter. I do like challenges so that's why we get
14mechanisms that can do tricky things and some stay sort of hidden because the
15practical usage is limited, although you will be surprised to see what users find
16in the source and use anyway. My colleague uses \CONTEXT\ for large scale, mostly
17complex and demanding \XML\ documents where one source is rendered in different
18ways with different parts used. Many features in \CONTEXT\ relate to workflows.
19
20I like to visualize things so that's part of the development cycle. I never start
21from some \quote {typographical} point of view, if only because in my experience
22much design is arbitrary and personal. The output should look okay on the
23average, and on reasonable simple documents there should be no need for manual
24intervention. I am quite willing to accept an occasional less optimal looking
25page and don't loose sleep over it. A next time, when a sentence gets added, it
26might be better and the problem can be moved further down the pages. Also, given
27what one runs into nowadays the average job that \TEX\ does is pretty good (but
28users can of course mess up). It is boundary conditions that determine in what
29direction a style or solution goes. The more abstract one argues about
30typesetting and possible solutions, the less interested I often become simply
31because there are no perfect solutions for every case. There are always those
32last few \percent\ points that need manual intervention or some trickery and most
33users get that. It is also what makes using \TEX\ fun.
34
35As mentioned, the \TEX\ engine does a pretty good job on average but that didn't
36prevent me from extending it: the mix of \TEX, \METAPOST\ and \LUA\ is even more
37fun. But what is the development agenda there? Again, it is very much driven by
38what users want me to solve, but there's also the curiosity element. A recent
39example of extending is the math sub system. It was already made more
40configurable and some features where added but now it is really flexible. This
41was doable because the heuristics in the engine are clear. It was could be done
42because I had a dedicated partner in this journey. \footnote {In another chapter
43I summarize what Mikael Sundqvist and I did in this context.} Other parts are
44more difficult but have nevertheless been extended, to mention a few: alignments,
45par building and page building. However the last two use some heuristics that are
46hard to make more flexible. For instance the badness calculation combined with
47the loop that tries to find breakpoints is already quite good and the somewhat
48special values involved in the calculations have been optimized stepwise by Don
49Knuth during the development of \TEX.
50
51Does that mean that one cannot add some options to influence that tuning? For sure
52one can. The source has this comment:
53
54\startquotation
55    When looking for optimal line breaks, \TEX\ creates a \quote {break node} for
56    each break that is {\em feasible}, in the sense that there is a way to end a line
57    at the given place without requiring any line to stretch more than a given
58    tolerance. A break node is characterized by three things: the position of the
59    break (which is a pointer to a \type {glue_node}, \type {math_node}, \type
60    {penalty_node}, or \type {disc_node}); the ordinal number of the line that will
61    follow this breakpoint; and the fitness classification of the line that has just
62    ended, i.e., \type {tight_fit}, \type {decent_fit}, \type {loose_fit}, or \type
63    {very_loose_fit}.
64\stopquotation
65
66The book {\TEX\ by Topic} (by Eijkhout) gives a good explanation of the way lines
67are broken so there is no need to go into detail here. The code involved is not
68that trivial anyway. The criteria for deciding what is bad are as follows:
69
70\starttabulate[|c|c|c|]
71\FL
72\BC verdict    \BC effect  \BC badness \NC \NR
73\ML
74\NC very loose \NC stretch \NC >= 100  \NC \NR
75\NC loose      \NC stretch \NC >= 13   \NC \NR
76\NC decent     \NC         \NC <= 12   \NC \NR
77\NC tight      \NC shrink  \NC >= 13   \NC \NR
78\LL
79\stoptabulate
80
81When the difference between two lines is more than one, they are considered to be
82visually incompatible. Then, if the badness of any line exceeds \type
83{pretolerance} a second pass is triggered, When \type {pretolerance} is negative
84the first pass is skipped. When the badness of any line exceeds \type {tolerance}
85a third pass is triggered and \type {emergencystretch} is used to make things fit.
86
87Where in traditional \TEX\ a lot of parsing, hyphenation, font handling and par
88building is combined, in \LUAMETATEX\ we always work with completely hyphenated
89and font readied lists. In traditional \TEX\ the first pass works on
90the original non|-|hyphenated lists.
91
92In the source there is an old note that one day I will play with a plugged in
93badness calculation but it also says that there might be a performance impact as
94well as all kind of unforeseen side effects because \TEX\ makes sure that the
95heuristics lead to values that don't result in overflow and such.
96
97Another note concerns more fitness values. Doing that will increase the runtime a
98little but on a modern machine that is not really an issue. Shortly after I
99upgraded my laptop to a somewhat newer one I decided to play with this and
100therefore any performance hit would go unnoticed anyway. The following snippet
101from the source shows the idea:
102
103\starttyping
104typedef enum fitness_value {
105    very_loose_fit, /*tex lines stretching more than their stretchability */
106    loose_fit,      /*tex lines stretching 0.5 to 1.0 of their stretchability */
107    semi_loose_fit,
108    decent_fit,     /*tex for all other lines */
109    semi_tight_fit,
110    tight_fit,      /*tex lines shrinking 0.5 to 1.0 of their shrinkability */
111    n_of_finess_values
112} fitness_value;
113\stoptyping
114
115This means that when we loop over \type {very_loose_fit} upto \type {tight_fit}
116we have two more classes to take into account: the semi ones. Playing with that
117and associating them with magic numbers quickly learned that we enter the area of
118\quote {random improvements}. You can render variants and because some will look
119better and others worse one can argue for any case. And as usual, once a user
120(unaware of what we are doing) looks at it, things like successive hyphens, wider
121spaces, rivers and such are seen as the main difference. Of course spacing is the
122direct result of this kind of messing, but because the effects are actually
123mostly noticeable on non|-|justified texts it then is the end|-|of|-|line spacing
124that influences the verdict. \footnote {When \HZ\ showed up in \PDFTEX\ we did
125experiments with random samples of its usage and \TEX ies at user group meetings
126and the results were such that one could only draw the conclusion that on the
127average a user has no clue if something is good or bad for what reason. The
128strong emphasis in the \TEX\ community on hyphenation makes that an eye|-|catching
129criterium. So having two in a successive lines even when there is really no
130better solution is what draws the attention and users then tend to think that
131what a survey is about is \quotation {The quality of hyphenation related to
132breaking paragraphs into lines.}}
133
134In the end this kind of extensions make little sense. One can of course play
135science and introduce all kind of imaginary cases where it might work but that is
136why I started this summary by explaining what drives developments: users and
137constraints. Playing science for the sake of it is pseudo science. And, as with
138much science related to typesetting (probably with the exception of Don's work)
139most has therefore little practical value.
140
141So, do we keep this feature or not? We actually do, if only to be able to
142demonstrate the fuzziness of this. We have an undocumented magic parameter:
143
144\starttyping
145\linebreakcriterium"0C0C0C63
146\stoptyping
147
148Actually the value is zero but when one of the four byte pairs is zero it will
149default to \type {"0C} (\number"0C) or \type {"63} (\number"63). The values
150concern \type {semitight}, \type {decent}, \type {semiloose}, and \type {loose}.
151After some trial and error I got to the examples on the next two pages. You need
152to zoom in to see the differences (the black one is the original). In setting used are:
153
154\starttabulate[|c|c|l|]
155\BC   \BC \type {\hsize} \BC \type {\setupalign}       \NC \NR
156\NC 1 \NC 12em           \NC normal, stretch, tolerant \NC \NR
157\NC 2 \NC 18em           \NC flushleft                 \NC \NR
158\stoptabulate
159
160As mentioned, one can look at specific expected properties and draw conclusions
161but when \TEX\ cannot find a good solution using its default, it is unlikely that
162alternative settings help you out, unless you do that on a per|-|paragraph basis.
163
164% run in default font and layout
165
166\startbuffer[lbc]
167\def\TestA#1#2#3%
168  {\switchtobodyfont[10pt]%
169   \ruledvtop
170     {\hsize#1\relax
171      \begingroup
172      \setupalign[#2]%
173      \linebreakcriterium"#3\relax
174      \samplefile{tufte}\par
175      \endgroup
176      \hpack to \hsize{\hss\infofont\setstrut\strut\black\string\linebreakcriterium="#3\hss}}}
177
178\def\TestB#1#2%
179  {\startTEXpage[offset=4dk]
180     \hbox \bgroup
181         {\black  \TestA{#1}{#2}{00000000}}\kern1ex
182         {\red    \TestA{#1}{#2}{00001C00}}\kern1ex
183         {\green  \TestA{#1}{#2}{00002C00}}\kern1ex
184         {\blue   \TestA{#1}{#2}{00003C00}}\kern1ex
185         {\cyan   \TestA{#1}{#2}{00004C00}}\kern1ex
186         {\magenta\TestA{#1}{#2}{00005C00}}
187     \egroup
188     \vskip1ex
189     \hbox \bgroup
190         \startoverlay {\TestA{#1}{#2}{00000000}}                                    \stopoverlay \kern1ex
191         \startoverlay {\TestA{#1}{#2}{00000000}} {\red    \TestA{#1}{#2}{00001C00}} \stopoverlay \kern1ex
192         \startoverlay {\TestA{#1}{#2}{00000000}} {\green  \TestA{#1}{#2}{00002C00}} \stopoverlay \kern1ex
193         \startoverlay {\TestA{#1}{#2}{00000000}} {\blue   \TestA{#1}{#2}{00003C00}} \stopoverlay \kern1ex
194         \startoverlay {\TestA{#1}{#2}{00000000}} {\cyan   \TestA{#1}{#2}{00004C00}} \stopoverlay \kern1ex
195         \startoverlay {\TestA{#1}{#2}{00000000}} {\magenta\TestA{#1}{#2}{00005C00}} \stopoverlay
196    \egroup
197   \stopTEXpage}
198
199\TestB{12em}{normal,stretch,tolerant}
200\TestB{18em}{flushleft}
201\stopbuffer
202
203\startpagemakeup[pagestate=start,doublesided=no,page=no]
204    \centerbox {\typesetbuffer[lbc][page=1,frame=on,height=\dimexpr\textwidth-2ex\relax,orientation=90]}
205\stoppagemakeup
206\startpagemakeup[pagestate=start,doublesided=no,page=no]
207    \centerbox {\typesetbuffer[lbc][page=2,frame=on,width=\dimexpr\textheight-2ex\relax,orientation=90]}
208\stoppagemakeup
209
210\stopchapter
211
212\stopcomponent
213
214