Source Browser ?

1% language=us runpath=texruns:manuals/musings
2
3% \showfontkerns
4
5\startcomponent musings-manuals
6
7\environment musings-style
8
9\ifdefined\startnarrowtyping \else
10    \definetyping[narrowtyping][style=\ttx]
11\fi
12
13\startchapter[title={About what \CONTEXT\ isn't}]
14
15\startsection[title={Introduction}]
16
17It really puzzles me why, when someone someplace asks if \CONTEXT\ is suitable
18for her or is his needs, there are answers like: \quotation {You need to think of
19\CONTEXT\ as being kind of plain \TEX: you have to define everything yourself.}
20That answer probably stems from the fact that for \LATEX\ you load some style
21that defines a lot, which you then might need to undefine or redefine, but that's
22not part of the answer.
23
24In the following sections I will go into a bit more detail of what plain \TEX\ is
25and how it influences macro packages, especially \CONTEXT . I'm sure I have
26discussed this before so consider this another go at it.
27
28The \type {plain.tex} file start with the line:
29
30\starttyping
31% This is the plain TeX format that's described in The TeXbook.
32\stoptyping
33
34A few lines later we read:
35
36\starttyping
37% And don't modify the file under any circumstances.
38\stoptyping
39
40So, this format related to the \TEX\ reference. It serves as a template for what
41is called a macro package. Here I will not go into the details of macro programming
42but an occasional snippet of code can be illustrative.
43
44\stopsection
45
46\startsection[title={Getting started}]
47
48The first code we see in the plain file is:
49
50\startnarrowtyping
51\catcode`\{=1 % left brace is begin-group character
52\catcode`\}=2 % right brace is end-group character
53\catcode`\$=3 % dollar sign is math shift
54\catcode`\&=4 % ampersand is alignment tab
55\catcode`\#=6 % hash mark is macro parameter character
56\catcode`\^=7 \catcode`\^^K=7 % circumflex and uparrow are for superscripts
57\catcode`\_=8 \catcode`\^^A=8 % underline and downarrow are for subscripts
58\catcode`\^^I=10                          % ascii tab is a blank space
59\chardef\active=13    \catcode`\~=\active % tilde is active
60\catcode`\^^L=\active \outer\def^^L{\par} % ascii form-feed is "\outer\par"
61\stopnarrowtyping
62
63Assigning catcodes to the braces and hash are needed in order to make it possible
64to define macros. The dollar is set to enter math mode and the ampersand becomes
65a separator in tables. The superscript and subscript also relate to math. Nothing
66demands these bindings but they are widely accepted. In this respect \CONTEXT\ is
67indeed like plain.
68
69The tab is made equivalent to a space and a tilde is made active which means that
70later on we need to give it some meaning. It is quite normal to make that an
71unbreakable space, and one with the width of a digit when we're doing tables.
72Now, nothing demands that we have to assume \ASCII\ input but for practical
73reasons the formfeed character is made equivalent to a \type {\par}.
74
75Now what do these \type {^^K} and similar triplets represent? The \type {^^A}
76represents character zero and normally all these control characters below decimal
7732 (space) are special. The \type {^^I} is the \ASCII\ tab character, and \type
78{^^L} the formfeed. But, the ones referred to as uparrow and downarrow in the
79comments have only meaning on certain keyboards. So these are typical definitions
80that only made sense for Don Knuth at that time and are not relevant in other
81macro packages that aim at standardized input media.
82
83\startnarrowtyping
84% We had to define the \catcodes right away, before the message line, since
85% \message uses the { and } characters. When INITEX (the TeX initializer) starts
86% up, it has defined the following \catcode values:
87%
88% \catcode`\^^@=9  % ascii null is ignored
89% \catcode`\^^M=5  % ascii return is end-line
90% \catcode`\\=0    % backslash is TeX escape character
91% \catcode`\%=14   % percent sign is comment character
92% \catcode`\ =10   % ascii space is blank space
93% \catcode`\^^?=15 % ascii delete is invalid
94% \catcode`\A=11 ... \catcode`\Z=11 % uppercase letters
95% \catcode`\a=11 ... \catcode`\z=11 % lowercase letters
96% all others are type 12 (other)
97\stopnarrowtyping
98
99The comments above speak for themselves. Changing catcodes is one way to adapt
100interpretation. For instance, in verbatim mode most catcodes can best be made
101letter or other. In \CONTEXT\ we always had so called catcode regimes: for
102defining macros, for normal text, for \XML, for verbatim, etc. In \MKIV\ this
103mechanism was adapted to the new catcode table mechanism available in that
104engine. It was one of the first things we added to \LUATEX. So, again, although
105we follow some standards (expectations) \CONTEXT\ differs from plain.
106
107\startnarrowtyping
108% We make @ signs act like letters, temporarily, to avoid conflict between user
109% names and internal control sequences of plain format.
110
111\catcode`@=11
112\stopnarrowtyping
113
114In \CONTEXT\ we went a step further and when defining macros also adapted
115the catcode of \type {!} and \type {?} and later in \MKIV\ \type {_}. When
116we're in unprotected mode this applies. In addition to regular text
117input math is dealt with:
118
119\startnarrowtyping
120% INITEX sets up \mathcode x=x, for x=0..255, except that
121%
122% \mathcode x=x+"7100, for x = `A to `Z and `a to `z;
123% \mathcode x=x+"7000, for x = `0 to `9.
124
125% The following changes define internal codes as recommended in Appendix C of
126% The TeXbook:
127
128\mathcode`\^^@="2201 % \cdot
129\mathcode`\^^A="3223 % \downarrow
130\mathcode`\^^B="010B % \alpha
131\mathcode`\^^C="010C % \beta
132....................
133\mathcode`\|="026A
134\mathcode`\}="5267
135\mathcode`\^^?="1273 % \smallint
136\stopnarrowtyping
137
138Here we see another set of definitions but the alphabetic ones are not defined in
139\CONTEXT, they are again bindings to the authors special keyboard.
140
141\startnarrowtyping
142% INITEX sets \sfcode x=1000 for all x, except that \sfcode`X=999 for uppercase
143% letters. The following changes are needed:
144
145\sfcode`\)=0 \sfcode`\'=0 \sfcode`\]=0
146
147% The \nonfrenchspacing macro will make further changes to \sfcode values.
148\stopnarrowtyping
149
150Definitions like this depend on the language. Because original \TEX\ was mostly
151meant for typesetting English, these things are hard coded. In \CONTEXT\ such
152definitions relate to languages.
153
154I show these definitions because they also illustrate what \TEX\ is about:
155typesetting math:
156
157\startnarrowtyping
158% Finally, INITEX sets all \delcode values to -1, except \delcode`.=0
159
160\delcode`\(="028300
161\delcode`\)="029301
162\delcode`\[="05B302
163\delcode`\]="05D303
164\delcode`\<="26830A
165\delcode`\>="26930B
166\delcode`\/="02F30E
167\delcode`\|="26A30C
168\delcode`\\="26E30F
169
170% N.B. { and } should NOT get delcodes; otherwise parameter grouping fails!
171\stopnarrowtyping
172
173Watch the last comment. One of the complications of \TEX\ is that because some
174characters have special meanings, we also need to deal with exceptions. It also
175means that arbitrary input is not possible. For instance, unless the percent
176character is made a letter, everything following it till the end of a line will
177be discarded. This is an areas where macro packages can differ but in \MKII\ we
178followed these rules. In \MKIV\ we made what we called \type {\nonknuthmode}
179default which means that ampersands are just that and scripts are only special in
180math (there was also \type {\donknuthmode}). So, \CONTEXT\ is not like plain
181there.
182
183\stopsection
184
185\startsection[title=Housekeeping]
186
187The next section defines some numeric shortcuts. Here the fact is used that a
188defined symbolic character can act as counter value. When the number is larger
189than 255 a math character is to be used. In \LUATEX, which is a \UNICODE\ engine
190character codes can be much larger.
191
192\startnarrowtyping
193% To make the plain macros more efficient in time and space, several constant
194% values are declared here as control sequences. If they were changed, anything
195% could happen; so they are private symbols.
196
197\chardef\@ne=1
198\chardef\tw@=2
199\chardef\thr@@=3
200\chardef\sixt@@n=16
201\chardef\@cclv=255
202\mathchardef\@cclvi=256
203\mathchardef\@m=1000
204\mathchardef\@M=10000
205\mathchardef\@MM=20000
206\stopnarrowtyping
207
208In \CONTEXT\ we still support these shortcuts but never use them ourselves. We
209have plenty more variables and constants and nowadays always use verbose names.
210(There was indeed a time when each extra characters depleted string memory more
211and more so then using short command names made sense.) The comment is right that
212using such variables is more efficient, for instance once loaded a macro is a
213sequence of tokens, so \type {\@one} takes one memory slot. In the case of the
214first three the saving is zero and even interpreting a single character token
215\type {3} is not less efficient than \type {\thr@@}, but in the case of \type
216{\@cclv} the three tokens \type {255} take more memory and also trigger the number
217scanner which is much slower than simply taking the meaning of the \type
218{\chardef}'d token. However, the \CONTEXT\ variable \type {\plusone} is as
219efficient as the \type {\@ne} and it looks prettier in code too (and I'm very
220sensitive for that). So, here \CONTEXT\ is definitely different!
221
222It makes no sense to show the next section here: it deals with managing
223registers, like counters and dimensions and token lists. Traditional \TEX\ has
224255 registers per category. Associating a control sequence (name) with a specific
225counter is done with \type {\countdef} but I don't think that you will find a
226macro package that expects a user to use that primitive. Instead it will provide
227a \type {\newcount} macro. So yes, here \CONTEXT\ is like plain.
228
229Understanding these macros is a test case for understanding \TEX. Take the
230following snippet:
231
232\startnarrowtyping
233\let\newtoks=\relax % we do this to allow plain.tex to be read in twice
234\outer\def\newhelp#1#2{\newtoks#1#1\expandafter{\csname#2\endcsname}}
235\outer\def\newtoks{\alloc@5\toks\toksdef\@cclvi}
236\stopnarrowtyping
237
238The \type {\outer} prefix flags macros as to be used at the outermost level and
239because the \type {\newtoks} is in the macro body of \type {\newtoks} it has to
240be relaxed first. Don't worry if you don't get it. In \CONTEXT\ we have no outer
241macros so the definitions differ there.
242
243The plain format assumes that the first 10 registers are used for scratch
244purposes, so best also assume this to be the case in other macro packages. There
245is no need for \CONTEXT\ to differ from plain here. The definitions of box
246registers and inserts are special: there is no \type {\boxdef} and inserts use
247multiple registers. Especially the allocation of inserts is macro package
248specific. Anyway, \CONTEXT\ users never see such details because inserts are used
249as building blocks deep down.
250
251Right after defining the allocators some more constants are defined:
252
253\startnarrowtyping
254% Here are some examples of allocation.
255
256\newdimen\maxdimen \maxdimen=16383.99999pt % the largest legal <dimen>
257\stopnarrowtyping
258
259We do have that one, as it's again a standard but we do have more such constants.
260This definition is kind of interesting as it assumes knowledge about what is
261acceptable for \TEX\ as dimension:
262
263\startbuffer
264{\dimen0=16383.99999pt \the\dimen0 \quad \number\dimen0}
265{\dimen0=16383.99998pt \the\dimen0 \quad \number\dimen0}
266\stopbuffer
267
268\typebuffer
269
270\startlines
271\getbuffer
272\stoplines
273
274Indeed it is the largest legal dimension but the real largest one is slightly
275less. We could also have said the following, which also indicates what the
276maximum cardinal is:
277
278\startnarrowtyping
279\newdimen\maxdimen \maxdimen=1073741823sp
280\stopnarrowtyping
281
282We dropped some of the others defined in plain. So, \CONTEXT\ is a bit like plain
283but differs substantially. In fact, \MKII\ already used a different allocator
284implementation and \MKIV\ is even more different. We also have more \type {\new}
285things.
286
287The \type {\newif} definition also differs. Now that definition is quite special
288in plain \TEX, so if you want a challenge, look it up. It defines three macros as
289the comment says:
290
291\startnarrowtyping
292% For example, \newif\iffoo creates \footrue, \foofalse to go with \iffoo.
293\stopnarrowtyping
294
295The \type {\iffoo} is either equivalent to \type {\iftrue} or \type {\iffalse}
296because that is what \TEX\ needs to see in order to be able to skip nested
297conditional branches. In \CONTEXT\ we have so called conditionals, which are more
298efficient. So, yes, you will find such defined ifs in the \CONTEXT\ source but
299way less than you'd expect in such a large macro package: \CONTEXT\ code doesn't
300look much like plain code I fear.
301
302\stopsection
303
304\startsection[title=Parameters]
305
306A next stage sets the internal parameters:
307
308\startnarrowtyping
309% All of TeX's numeric parameters are listed here, but the code is commented out
310% if no special value needs to be set. INITEX makes all parameters zero except
311% where noted.
312\stopnarrowtyping
313
314We use different values for many of them. The reason is that the plain \TEX\ format
315is set up for a 10 point Computer Modern font system, and for a particular kind
316of layout, so we use different values for:
317
318\startnarrowtyping
319\hsize=6.5in
320\vsize=8.9in
321\maxdepth=4pt
322\stopnarrowtyping
323
324and
325
326\startnarrowtyping
327\abovedisplayskip=12pt plus 3pt minus 9pt
328\abovedisplayshortskip=0pt plus 3pt
329\belowdisplayskip=12pt plus 3pt minus 9pt
330\belowdisplayshortskip=7pt plus 3pt minus 4pt
331\stopnarrowtyping
332
333No, here \CONTEXT\ is not like plain. But, there is one aspect that we do inherit and
334that is the ratio. Here a 10 point relates to 12 point and this 1.2 factor is carried
335over in some defaults in \CONTEXT. So, in the end we're a bit like plain.
336
337After setting up the internal quantities plain does this:
338
339% We also define special registers that function like parameters:
340
341\startnarrowtyping
342\newskip\smallskipamount \smallskipamount=3pt plus 1pt minus 1pt
343\newskip\medskipamount \medskipamount=6pt plus 2pt minus 2pt
344\newskip\bigskipamount \bigskipamount=12pt plus 4pt minus 4pt
345\newskip\normalbaselineskip \normalbaselineskip=12pt
346\newskip\normallineskip \normallineskip=1pt
347\newdimen\normallineskiplimit \normallineskiplimit=0pt
348\newdimen\jot \jot=3pt
349\newcount\interdisplaylinepenalty \interdisplaylinepenalty=100
350\newcount\interfootnotelinepenalty \interfootnotelinepenalty=100
351\stopnarrowtyping
352
353The first three as well as the following three related variables are not internal
354quantities but preallocated registers. These are not used in the engine but in
355macros. In \CONTEXT\ we do provide them but the first three are never used that
356way. The last three are not defined at all. So, \CONTEXT\ provides a bit what
357plain provides, just in case.
358
359\stopsection
360
361\startsection[title=Fonts]
362
363The font section is quite interesting. I assume that one reason why some want to
364warn users against using \CONTEXT\ is because it supports some of the font
365switching commands found in plain. We had no reasons to come up with different ones
366but they do different things anyway, for instance adapting to situations. So, in
367\CONTEXT\ you will not find the plain definitions:
368
369\startnarrowtyping
370\font\tenrm=cmr10 % roman text
371\font\preloaded=cmr9
372\font\preloaded=cmr8
373\font\sevenrm=cmr7
374\font\preloaded=cmr6
375\font\fiverm=cmr5
376\stopnarrowtyping
377
378There is another thing going on here. Some fonts are defined \type {\preloaded}. So,
379\type {cmr9} is defined, and then \type {cmr8} and \type {cmr6}. But they all use the
380same name. Later on we see:
381
382\startnarrowtyping
383\let\preloaded=\undefined % preloaded fonts must be declared anew later.
384\stopnarrowtyping
385
386If you never ran into the relevant part of the \TEX\ book or read the program
387source of \TEX, you won't realize that preloading means that it stays in memory
388which in turn means that when it gets (re)defined later, the font data doesn't
389come from disk. In fact, as the plain format is normally dumped for faster reload
390later on, the font data is also retained. So, preloading is a speed up hack. In
391\CONTEXT\ font loading has always been delayed till the moment a font is really
392used. This permits plenty of definitions and gives less memory usage. Of course
393we do reuse fonts once loaded. All this, plus the fact that we have a a system of
394related sizes, collections of families, support multiple font encodings
395alongside, collect definitions in so called typescript, etc. makes that the
396\CONTEXT\ font subsystem is far from what plain provides. Only some of the
397command stick, like \type {\rm} and \type {\bf}.
398
399The same is true for math fonts, where we can have different math font setups in
400one document. Definitely in \MKII\ times, we also had to work around limitations
401in the number of available math families, which again complicated the code. In
402\MKIV\ things are even more different, one can even consider the implementation
403somewhat alien for a standard macro package, but that's for another article (if
404at all).
405
406\stopsection
407
408\startsection[title=Macros]
409
410Of course \CONTEXT\ comes with macros, but these are organized in setups,
411environments, instances, etc. The whole process and setup is keyword driven. Out
412of the box all things work: nothing needs to be loaded. If you want it different,
413you change some settings, but you don't need to load something. Maybe that last
414aspect is what is meant with \CONTEXT\ being like plain: you don't (normally)
415load extra stuff. You just adapt the system to your needs. So there we proudly
416follow up on plain \TEX.
417
418In the plain macro section we find definitions like:
419
420\startnarrowtyping
421\def\frenchspacing{\sfcode`\.\@m \sfcode`\?\@m \sfcode`\!\@m
422  \sfcode`\:\@m \sfcode`\;\@m \sfcode`\,\@m}
423\def\nonfrenchspacing{\sfcode`\.3000\sfcode`\?3000\sfcode`\!3000%
424  \sfcode`\:2000\sfcode`\;1500\sfcode`\,1250 }
425\stopnarrowtyping
426
427and:
428
429\startnarrowtyping
430\def\space{ }
431\def\empty{}
432\def\null{\hbox{}}
433
434\let\bgroup={
435\let\egroup=}
436\stopnarrowtyping
437
438and:
439
440\startnarrowtyping
441\def\nointerlineskip{\prevdepth-1000\p@}
442\def\offinterlineskip{\baselineskip-1000\p@
443  \lineskip\z@ \lineskiplimit\maxdimen}
444\stopnarrowtyping
445
446Indeed we also provide these, but apart from the two grouping related aliases
447their implementation is different in \CONTEXT. There is no need to reinvent
448names.
449
450For a while we kept (and did in \MKII) some of the plain helper macros, for
451instance those that deal with tabs, but we have several more extensive table
452models that are normally used. We always had our own code for float placement,
453and we also have more options there. Footnotes are supported but again we have
454multiple classes, placements, options, etc. Idem for itemized lists, one of the
455oldest mechanisms in \CONTEXT. We don't have \type {\beginsection} but of course
456we do have sectioning commands, and have no \type {\proclaim} but provide lots of
457descriptive alternatives, so many that I forgot about most of them by now (so
458plain is a winner in terms of knowing a macro package inside out).
459
460The fact that we use tables, floats and footnotes indeed makes \CONTEXT\ to act
461like plain, but that's then also true for other macro packages. A fact is that
462plain sets the standard for how to think about these matters! The same is true
463for naming characters:
464
465\startnarrowtyping
466\chardef\%=`\%
467\chardef\&=`\&
468\chardef\#=`\#
469\chardef\$=`\$
470\chardef\ss="19
471\chardef\ae="1A
472\chardef\oe="1B
473\chardef\o="1C
474\chardef\AE="1D
475\chardef\OE="1E
476\chardef\O="1F
477\chardef\i="10 \chardef\j="11 % dotless letters
478\stopnarrowtyping
479
480But we have many more and understandable the numbers are different in \CONTEXT\
481because we use different font (encodings). Their implementation is more adaptive.
482The same is true for accented characters:
483
484\startnarrowtyping
485\def\`#1{{\accent18 #1}}
486\def\'#1{{\accent19 #1}}
487\stopnarrowtyping
488
489The definitions in \MKII\ are different (in most cases we use native glyphs) and
490in \MKIV\ we use \UNICODE\ anyway. I think that the \type {\accent} command is
491only used in a few exceptional cases (like very limited fonts) in \MKII\ and never
492in \MKIV. The implementation of for instance accents (and other pasted together
493symbols) in math is also quite different.
494
495There are also definitions that seem to be commonly used in macro packages but
496that we never use in \CONTEXT\ because they interfere badly with all kind of
497other mechanisms, so you will find no usage of
498
499\startnarrowtyping
500\def\leavevmode{\unhbox\voidb@x} % begins a paragraph, if necessary
501\stopnarrowtyping
502
503in \CONTEXT. In order to stress that we provide \type {\dontleavehmode}, a wink
504to not using the one above.
505
506The macro section ends with lots of math definitions. Most of the names used are
507kind of standard so again here \CONTEXT\ is like plain, but the implementation
508can differ as does the level of control.
509
510\stopsection
511
512\startsection[title=Output]
513
514Once a page is ready it gets wrapped up and shipped out. Here \CONTEXT\ is very
515different from plain. The amount of code in plain is not that large but the
516possibilities aren't either, which is exactly what the objectives demand: a
517simple (example) format that can be described in the \TEX book. But, as with
518other aspects of plain, it steered the way macro packages started out as it
519showed the way. As did many examples in the \TEX\ book.
520
521\stopsection
522
523\startsection[title=Hyphenation]
524
525As an afterthought, the plain format ends with loading hyphenation patterns, that
526is the English ones. That said it will be clear that \CONTEXT\ is not like plain:
527we support many languages, and the subsystem deals with labels, specific
528typesetting properties, etc.\ too.
529
530\startnarrowtyping
531\lefthyphenmin=2 \righthyphenmin=3 % disallow x- or -xx breaks
532\input hyphen
533\stopnarrowtyping
534
535We don't even use these patterns as we switched to \UTF\ long ago (also in \MKII)
536if only because we had to deal with a mix of font encodings. But we did preload the
537lot there. In \MKIV\ again things are much different.
538
539\stopsection
540
541\startsection[title=Conclusion]
542
543The plain format does (and provides) what it is supposed to do. It is a showcase
544of possibilities and part of the specification. In that respect it's nice that
545\CONTEXT\ is considered to be like plain. But if it wasn't more, there was no
546reason for its existence. Like more assumptions about \CONTEXT\ it demonstrates
547that those coming up with answers and remarks like that probably missed something
548in assessing \CONTEXT. Just let users find out themselves what suits best (and
549for some that actually might be plain \TEX).
550
551Let me make on thing clear. If you look at the documents that describe the
552development of \TEX, \METAFONT\ and the related fonts, you can only awe at what
553was done on hardware that doesn't come close to what we hold now in the palm of
554our hand. And it was done in a relative short time span. The fact that plain
555\TEX\ ran on it the way it did is amazing. Anyone who considers criticizing \TEX\
556and plain \TEX\ should think (more than) twice.
557
558\stopsection
559
560\stopchapter
561
562\stopcomponent
563