1
2
3\startcomponent followinguplua
4
5\environment followingupstyle
6
7\startchapter[title={\LUA}]
8
9\startsection[title={Move to 5.4}]
10
11Another experiment concerned testing \LUA\ 5.4 which looks like a minor update in
12terms of new functionality but has some consequences. By now the old module model
13is even more deprecated and compatibility mode no longer makes much sense. As a
14consequence we now need to adapt the way libraries are loaded (and we use global
15ones) and a few other low level calls had to be adapted. This is no real issue
16and once that was done, I found out that the bit32 module was even more obsolete
17so I decided to get rid of it. We already have a bit32 replacement in \CONTEXT\
18so I had to enable that. As \CONTEXT\ doesnt need compatibility mode it was no
19problem to drop that too.
20
21The biggest changes in 5.4 are under the hood: some optimized byte code and a new
22generational garbage collector. I did a few runs and a 12.4 seconds run on the
23manual now dropped to around 12.1 and given that we spend (probably) more than
24half the time in \LUA\ that means some 5\% gain in performance. This is still
25more than the 9.6 seconds that \LUAJIT\ needs but it looks like every \LUA\
26release gains a bit and Im pretty sure that there is more to gain. \footnote {In
27the meantime there are experiments in 5.4 with \type {<const>} directives which
28might have advantages.}
29
30An interesting experiment was to disable the automatic string to number
31conversion when a number is expected but a string is needed. So far I only had to
32adapt two lines of code in the in the meantime considerable amount of \LUA\ code
33that comes with \CONTEXT.
34
35\stopsection
36
37\startsection[title={No more \LUAJIT}]
38
39One thing I had to consider was the future of \LUAJIT. This project is sort of
40stalled and will not follow \LUA\ development. Now, to some extend we can deal
41with this but with the faster \LUA\ 5.4 around the corner, the limitations of
42\LUAJIT\ with respect to loading large tables, as well as the fact that we need a
43patched hash function to get an advantage over regular \LUA\ anyway, it makes
44sense to drop it in \LUAMETATEX. After discussing this with Alan, who crunched
45numbers in order to make impressive graphics with \METAPOST, we came to the
46conclusion that we should not overestimate the benefits. There is still a gain
47but removing the need to support both could also makes it possible to improve
48existing code (although one should not expect too much from that; its more a
49matter of convenience for me). Also, for as long as have \LUAJITTEX\ that is
50still an option when one has to squeeze out every second.
51
52A valid question is if ditching \LUAJIT\ will harm users. The answer to this
53depends on the kind of documents that you process. Given decent programming, you
54can gain quite a bit of runtime, but on the average the difference is not that
55large. There is for instance always the overhead of callbacks and crossing the so
56called \CCODE\ boundary that has an impact.
57
58\stopsection
59
60\startsection[title={Performance}]
61
62At the time of writing this Thomas Schmitz was wondering if there was a
63significant difference in runtime between the table mechanisms and especially
64natural tables and extreme tables. Some test demonstrated that extreme tables
65were best for his case. That case concerned generating about 400 pages of tables
66from \XML\ files, including some juggling of data in \LUA. The bottleneck in that
67document can be roughly simulated with the following test. We assume one pass
68over the table but in practice there are upto four, but only the last one has
69frames. So, the test concerns 80.000 (400 pages with 40 rows of 5 columns) calls
70to \type {\framed}.
71
72
73
74\starttyping
751 \hpack{\framed {oeps}}
762 \hpack{\framed[frame=off] {oeps}}
773 \setupframed[frame=off] \hpack{\framed {oeps}}
784 \hpack{\framed[frame=on] {oeps}}
795 \setupframed[frame=on] \hpack{\framed {oeps}}
806 \hpack{\framed[frame=closed]{oeps}}
817 \setupframed[frame=closed] \hpack{\framed {oeps}}
82\stoptyping
83
84\starttabulate[cccc]
85\HL
86\BC sample \BC luatex mkiv \BC luajittex mkiv \BC luametatex lmtx \BC \NR
87\HL
88\NC 1 \NC 17.3 \NC 16.8 \NC 13.5 \NC \NR
89\NC 2 \NC 17.8 \NC 17.2 \NC 14.0 \NC \NR
90\NC 3 \NC 17.3 \NC 16.8 \NC 13.3 \NC \NR
91\NC 4 \NC 17.9 \NC 17.4 \NC 13.7 \NC \NR
92\NC 5 \NC 17.4 \NC 17.1 \NC 13.3 \NC \NR
93\NC 6 \NC 17.4 \NC 16.8 \NC 12.9 \NC \NR
94\NC 7 \NC 16.4 \NC 16.0 \NC 12.6 \NC \NR
95\HL
96\stoptabulate
97
98Even if we add the usual .1 second interval around these values it will be clear
99that we gain enough not to worry about the loss of \LUAJIT, also because the gain
100is not in the \LUA\ part only. A nice consequence of this is that when we replace
101the \CPUs in a server with low power ones that perform 25\% less, we can
102compensate that by using \LMTX. \footnote {Theres still room for improvement,
103because mid July 2019 were at 12.9, 13.2, 12.9, 13.5, 13.0, 12.5 and 12.2
104seconds or less. But dont expect too many miracles.}
105
106When wrapping this up, the \LUATEX\ manual processed with \LMTX\ took slightly
107less than 11.9 seconds, compared to a normal run of 12.6 seconds, so were
108gaining some there too. And just after I wrote this we went down to 11.7 seconds
109by (as experiment) changing the \LUA\ virtual machine dispatcher, so there is
110still some to gain. In the energy saving fitlet with small amd processor
111processing the manual with stock \LUATEX\ takes about 37 seconds, but 33.5 with
112\LMTX\ so here also were not off worse.
113
114\stopsection
115
116\startsection[title={Modules}]
117
118Right from the start \LUATEX\ had some extra libraries linked in: \type {md5}
119(for hashing), \type {lfs} (for accessing file properties), \type {slunicode}
120(for basic \UTF\ handling), \type {gzip} and \type {zlib} (for zipping files and
121streams), \type {zip} (for accessing zip files) and \type {socket} (for
122communicating other than with files).
123
124In \LUAMETATEX\ the not so useful \type {slunicode} library was removed pretty
125early but the others stayed around. The more backend specific \type {img} and
126\type {pdf} libraries went away too, as did the (already not used) \type
127{fontloader} library. The \type {kpse} library is also gone, as we do those
128things in \LUA. The \type {epdf} library was kept. A couple of libraries were
129added, like \type {sha2}, \type {basexx}, and \type {flate}, plus a few handy
130helper libraries that are still experimental and therefore not mentioned here.
131
132The \type {flate} library is also an experiment but will replace the \type {gzip}
133and \type {zlib} libraries. Currently these use \type {libz} but \type
134{libdeflate} will be the low level replacement once it support streams and is
135already used for \type {flate}. The \type {md5} library has been redone using
136utility code \type {pplib}, as \type {sha2} does. The type {basexx} library also
137falls back on utility code form \type {pplib} (that code is actually
138independent).
139
140The \type {lfs} code has been replaced by a variant that omits features not
141common to the platforms and with a iterator that permits much faster directory
142scans and has a few more helpers. It is not compatible but we kept the name
143because of legacy usage. I might strip the socket code to what is actually used,
144but on the other hand: dont touch what works well. The original code doesnt
145change that much anyway.
146
147\stopsection
148
149\stopchapter
150
151\stopcomponent
152 |