1
2
3\usemodule[presentboring,abbreviationslogos]
4
5\startdocument
6 [title={LUAMETATEX},
7 banner={where do we stand},
8 location={context\enspace {\bf 2020}\enspace meeting}]
9
10\starttitle[title=When it started]
11
12\startitemize
13
14\startitem
15 About three years ago the idea came up to go this route.
16\stopitem
17\startitem
18 At the 2018 meeting it was first mentioned and those present were okay with
19 it.
20\stopitem
21\startitem
22 Early 2019 the first beta release took place.
23\stopitem
24\startitem
25 At the 2019 meeting the first more official version was presented.
26\stopitem
27\startitem
28 Around the 2020 meeting we have more or less arrived at what I had in mind.
29\stopitem
30\startitem
31 At the 2021 meeting I expect the code to be stable and repositories to be set
32 up.
33\stopitem
34\startitem
35 At the 2022 meeting we can make the official transition from \MKIV\ to \LMTX.
36\stopitem
37\startitem
38 Some new options are only enabled in my local \type {contexp.tex} file.
39\stopitem
40\startitem
41 Knowing that Wolfgang keeps an eye on all those changes makes me more daring.
42\stopitem
43\startitem
44 We aim to get less (but more efficient) macro code that on the average looks
45 better.
46\stopitem
47
48\stopitemize
49
50\stoptitle
51
52\starttitle[title=Why it started]
53
54\startitemize
55
56\startitem
57 There was an increasing pressure for a stable \LUATEX.
58\stopitem
59\startitem
60 There should be no more changes to the interfaces, no more extensions.
61\stopitem
62\startitem
63 One can run into interesting comments on the web (as usual), like
64 \startitemize[packed]
65 \startitem The \LUATEX\ program has \quote {many bugs}. \stopitem
66 \startitem The \LUATEX\ manual is bad. \stopitem
67 \startitem The \LUATEX\ program is too slow to be useful. \stopitem
68 \startitem The \LUATEX\ program will never end up in distributions. \stopitem
69 \startitem The \LUATEX\ project is funded and developed in a commercial setting. \stopitem
70 \stopitemize
71\stopitem
72\startitem
73 I wont comment on how I read these (demotivating) comments because \unknown
74\stopitem
75\startitem
76 \unknown\ it anyway often says more about the writer (attitudes) than about
77 \LUATEX.
78\stopitem
79\startitem
80 I also looks like (non \CONTEXT) users are charmed by \LUATEX, and the more
81 they code, the more we need to freeze.
82\stopitem
83\startitem
84 So, hopefully, the \LUAMETATEX\ development does not interfere badly with
85 developments outside the \CONTEXT\ community.
86\stopitem
87
88\stopitemize
89
90\stoptitle
91
92\starttitle[title=The development]
93
94The summary on the next pages is partial. More can be found in articles and
95documents that come with the distribution.
96
97\startitemize
98
99\startitem
100 \LUATEX\ started out as \CWEB\ code \unknown\ that eventually became just
101 \CCODE\ \unknown\ which in \LUAMETATEX\ has been detached from the (complex)
102 infrastructure.
103\stopitem
104\startitem
105 The basic idea is to only keep the core of \TEX, but for instance font
106 loading, file handling and the backend are gone.
107\stopitem
108\startitem
109 As a consequence the code has been reorganized (shuffled around).
110\stopitem
111\startitem
112 I experimented a lot without bothering about usage elsewhere and I like the
113 result so far.
114\stopitem
115\startitem
116 The \CONTEXT\ distribution will at some point ship with the source.
117\stopitem
118
119\stopitemize
120
121\starttitle[title=File handling]
122
123\startitemize
124
125\startitem
126 All file handling goes via \LUA, also read and write related primitives.
127\stopitem
128\startitem
129 The same is true for terminal (console) handling.
130\stopitem
131\startitem
132 Part of that (the writing) was actually kind of extension code in \TEX\ and
133 partly a system dependency.
134\stopitem
135\startitem
136 The \ETEX\ pseudo file \type {\scantokens} primitive uses the same mechanism
137 as \LUA\ does.
138\stopitem
139
140\stopitemize
141
142\stoptitle
143
144\starttitle[title=The macro machinery]
145
146\startitemize
147
148\startitem
149 There are extensions to the way macro arguments are handled (less clumsy
150 macros).
151\stopitem
152\startitem
153 There are extra if tests (makes for nicer macros).
154\stopitem
155\startitem
156 Else branches in conditions can be collapsed using \type {\orelse} and \type
157 {\orunless} which gives cleaner low level code.
158\stopitem
159\startitem
160 Tracing gives more detail about node properties and also shows attributes.
161\stopitem
162\startitem
163 Some new data carriers have been added that can be played with from \LUA .
164\stopitem
165\startitem
166 Macros can efficiently be frozen (new) and protected (redone) and the
167 concepts \quote {long} and \type {outer} are gone. \footnote {In \CONTEXT\
168 macros were always \type {\long} and never \type {\outer}. Most commands were
169 unexpandable (also in \MKII, pre \ETEX). So, users wont notice this.}
170\stopitem
171\startitem
172 Saving and restoring is somewhat more efficient (partly a side effect of
173 wider memory).
174\stopitem
175
176\stopitemize
177
178\stoptitle
179
180\starttitle[title=Language]
181
182\startitemize
183
184\startitem
185 Language control settings now use less parameters but bit sets instead.
186\stopitem
187\startitem
188 Only basic parameters are stored in the format file now.
189\stopitem
190\startitem
191 There are all kind of small improvements.
192\stopitem
193
194\stopitemize
195
196\stoptitle
197
198\starttitle[title=Typesetting]
199
200\startitemize
201
202\startitem
203 Attributes (the lists and states) are implemented more efficiently.
204\stopitem
205\startitem
206 The paragraph state is stored with the paragraph.
207\stopitem
208\startitem
209 Paragraphs can be normalized and options are now set with bit sets.
210\stopitem
211\startitem
212 Boxes carry orientation related information (offsets, rotation, etc).
213\stopitem
214\startitem
215 Some nodes carry more information.
216\stopitem
217\startitem
218 Directions are mostly gone (its up to the backend).
219\stopitem
220\startitem
221 Migrated content is optionally kept with boxes.
222\stopitem
223
224\stopitemize
225
226\stoptitle
227
228\starttitle[title=Math]
229
230\startitemize
231
232\startitem
233 Some math concepts have been extended (like prescripts and some more
234 control over styles).
235\stopitem
236\startitem
237 There are plenty of new control details.
238\stopitem
239\startitem
240 The math parameter settings obey grouping in a math list.
241\stopitem
242\startitem
243 We can have math in discretionaries in text and more advanced discretionaries
244 in math as well.
245\stopitem
246
247\stopitemize
248
249\stoptitle
250
251\starttitle[title=Fonts]
252
253\startitemize
254
255\startitem
256 Font specification information no longer uses the string pool (which saves a
257 lot).
258\stopitem
259\startitem
260 Of course we still have the basic font handler.
261\stopitem
262\startitem
263 We only store what is needed for traditional \TEX\ font handling.
264\stopitem
265\startitem
266 Virtual fonts are even more virtual (also a backend thing) so we can have
267 more features.
268\stopitem
269
270\stopitemize
271
272\stoptitle
273
274\starttitle[title=The code]
275
276\startitemize
277
278\startitem
279 Artifacts from \PASCAL\ and \CWEB\ have been removed.
280\stopitem
281\startitem
282 Languages, fonts, marks etc are no longer \quote {register} based.
283\stopitem
284\startitem
285 The token interface is more abstract and no longer presents strange numbers.
286\stopitem
287\startitem
288 Some internals have been reconstructed because of cleaner \LUA\ interfacing.
289\stopitem
290\startitem
291 A side effect of this is better abstraction of the equivalent ranges.
292\stopitem
293\startitem
294 The code has been made more abstract (and looks easier in e.g. Visual Studio).
295\stopitem
296\startitem
297 The compile farm is used to check if compilation works out of the box.
298\stopitem
299\startitem
300 Compilation is fast and easy, otherwise this project was not possible.
301\stopitem
302\startitem
303 Readability of the code is constantly improved (the usual: has to look okay
304 in my editor).
305\stopitem
306\startitem
307 The code has been made mostly independent of specific operating system needs.
308\stopitem
309\startitem
310 Wide characters are dealt with in Windows interfaces.
311\stopitem
312
313\stopitemize
314
315\stoptitle
316
317\starttitle[title=Libraries]
318
319\startitemize
320
321\startitem
322 We really want to stay lean and mean: the engine is also a \LUA\ engine.
323\stopitem
324\startitem
325 All code is included, a few libraries are used, but these are small, old and
326 stable.
327\stopitem
328\startitem
329 In addition some helper libraries are made (including pplib by Pawel).
330\stopitem
331\startitem
332 What we ship is what you get: \CONTEXT\ will not depend on more than that.
333\stopitem
334\startitem
335 If something is updated (at all) the differences are checked first.
336\stopitem
337
338\stopitemize
339
340\stoptitle
341
342\starttitle[title=The \LUA\ engine]
343
344\startitemize
345
346\startitem
347 We use the latest (even alpha) \LUA\ (5.4) because \LUAMETATEX\ is a good
348 test.
349\stopitem
350\startitem
351 There is no support for \LUAJIT\ and the \FFI\ interface is gone.
352\stopitem
353\startitem
354 There is a limited set of libraries that we support but no code is (and will
355 be) included.
356\stopitem
357\startitem
358 There are less callbacks (because we only have a frontend).
359\stopitem
360\startitem
361 There are more token scanners and some options have been added.
362\stopitem
363
364\stopitemize
365
366\stoptitle
367
368\starttitle[title=Efficiency]
369
370\startitemize
371
372\startitem
373 We benefit some more from the wider memory words (some constructs could go).
374\stopitem
375\startitem
376 The format file is smaller and not longer compressed.
377\stopitem
378\startitem
379 Memory management is now mostly dynamic and usage is much lower.
380\stopitem
381\startitem
382 There are more statistics (also as side effect of memory management).
383\stopitem
384\startitem
385 Dumping the format has been made a bit more robust and is faster.
386\stopitem
387\startitem
388 The core engine performs a bit better (machines dont get that much faster).
389\stopitem
390\startitem
391 We want to be prepared for future architectures.
392\stopitem
393\startitem
394 We manage to keep the binary way below 3 MB.
395\stopitem
396\startitem
397 The lot runs quite well on e.g.\ a Raspberry Pi 4.
398\stopitem
399
400\stopitemize
401
402\starttitle[title=Upgraded \METAPOST]
403
404\startitemize
405
406\startitem
407 All (eight bit) font stuff has been stripped from the \METAPOST\ library.
408\stopitem
409\startitem
410 The library no longer has a \POSTSCRIPT\ backend.
411\stopitem
412\startitem
413 The library provides scanners that make extensions possible.
414\stopitem
415\startitem
416 All file \IO\ goes via \LUA.
417\stopitem
418\startitem
419 There are a few additions like prepostscripts for clip and bounding boxes.
420\stopitem
421\stopitemize
422
423\stoptitle
424
425\starttitle[title=Praise for the users]
426
427\startitemize
428
429\startitem
430 Much has been done and I probably forget to mention a lot.
431\stopitem
432\startitem
433 The number of bugs is relative small compared to what gets changed and added.
434\stopitem
435\startitem
436 The test suite gets ran very often, also to check if performance is okay.
437\stopitem
438\startitem
439 I could only do this because the \CONTEXT\ users are so tolerant.
440\stopitem
441\startitem
442 Some seem to constantly check for updates so they help with fast testing.
443\stopitem
444\startitem
445 The \CONTEXT\ code base gets stepwise adapted (split files) which again
446 forces users to test.
447\stopitem
448\startitem
449 It takes a lot of time because we take small steps in order not to mess up.
450\stopitem
451\startitem
452 I would not do it without the positive attribute of the \CONTEXT\ users.
453\stopitem
454\startitem
455 Its all about motivation and I thank the \CONTEXT\ users for providing this
456 friendly and noncompetitive bubble!
457\stopitem
458
459\stopitemize
460
461\starttitle[title=Todo]
462
463\startitemize
464
465\startitem
466 Maybe add some more sanity checks in order to catch errors intruded by
467 callbacks. Maybe add some more tracing too.
468\stopitem
469\startitem
470 Explore variants, like having registers in dedicated eqtb tables so that we
471 can allocate them dynamically (mostly for the fun of doing it).
472\stopitem
473\startitem
474 Add some more documentation (read: addition cq.\ remarks about where the
475 original documentation no longer applies, but we have years for doing that).
476\stopitem
477\startitem
478 Update the manual (which is done occasionally in batch based on printouts;
479 there is no real need to hurry because we still experiment).
480\stopitem
481\startitem
482 Apply some of the new stuff in \LMTX. Take up some challenges.
483\stopitem
484\startitem
485 Wrap up new functionality (once its stable) in articles and other documents.
486\stopitem
487
488\stopitemize
489
490\stoptitle
491
492\starttitle[title=And \LUATEX ?]
493
494\startitemize
495
496\startitem
497 Of course \LUATEX\ will be maintained! After all, \MKIV\ needs it and it
498 serves as reference for the frontend rendering and backend generation
499 when were messing with \LUAMETATEX.
500\stopitem
501\startitem
502 It is used by \LATEX\ and there are now also plain inspired packages. Because
503 there are spinoffs (\LATEX\ has settled on a version with builtin font
504 processing) we cannot change much.
505\stopitem
506\startitem
507 And \LUATEX\ being nicely integrated into \TEXLIVE\ is another argument for
508 not touching it too much.
509\stopitem
510\startitem
511 I have no clue of \LUATEX\ usage but that fact alone already makes an
512 argument for being even more careful. Its bad advertisement for \TEX\ when
513 users who use the low level interfaces get confronted with conceptual
514 changes.
515\stopitem
516\startitem
517 So in the end not much will be back ported to \LUATEX: at some point the code
518 base became too different and its the price paid for the stability demand.
519 That way we cannot introduce new bugs either. It also doesnt pay of.
520\stopitem
521\startitem
522 But, a few nonintrusive things might actually trickle into it in due time,
523 also out of self interest: it might help to share code between \MKIV\ and
524 \LMTX.
525\stopitem
526
527\stopitemize
528
529\stoptitle
530
531\stopdocument
532 |
