Source Browser ?

1% language=us runpath=texruns:manuals/luametatex
2
3\environment luametatex-style
4
5\startcomponent luametatex-differences
6
7\startchapter[reference=differences,title={Differences with \LUATEX}]
8
9As \LUAMETATEX\ is a leaner and meaner \LUATEX. This means that substantial parts and
10dependencies are gone: quite some font code, all backend code with related frontend
11code and of course image and font inclusion. There is also new functionality which
12makes for less lean but in the end we still have less, also in terms of dependencies.
13This chapter will discuss what is gone. We start with the primitives that were dropped.
14
15\starttabulate[|l|pl|]
16\BC fonts       \NC \type {\letterspacefont}
17                    \type {\copyfont}
18                    \type {\expandglyphsinfont}
19                    \type {\ignoreligaturesinfont}
20                    \type {\tagcode}
21                    \type {\leftghost}
22                    \type {\rightghost}
23                \NC \NR
24\BC backend     \NC \type {\dviextension}
25                    \type {\dvivariable }
26                    \type {\dvifeedback}
27                    \type {\pdfextension}
28                    \type {\pdfvariable }
29                    \type {\pdffeedback}
30                    \type {\dviextension}
31                    \type {\draftmode}
32                    \type {\outputmode}
33                \NC \NR
34\BC dimensions  \NC \type {\pageleftoffset}
35                    \type {\pagerightoffset}
36                    \type {\pagetopoffset}
37                    \type {\pagebottomoffset}
38                    \type {\pageheight}
39                    \type {\pagewidth}
40                \NC \NR
41\BC resources   \NC \type {\saveboxresource}
42                    \type {\useboxresource}
43                    \type {\lastsavedboxresourceindex}
44                    \type {\saveimageresource}
45                    \type {\useimageresource}
46                    \type {\lastsavedimageresourceindex}
47                    \type {\lastsavedimageresourcepages}
48                \NC \NR
49\BC positioning \NC \type {\savepos}
50                    \type {\lastxpos}
51                    \type {\lastypos}
52                \NC \NR
53\BC directions  \NC \type {\textdir}
54                    \type {\linedir}
55                    \type {\mathdir}
56                    \type {\pardir}
57                    \type {\pagedir}
58                    \type {\bodydir}
59                    \type {\pagedirection}
60                    \type {\bodydirection}
61                \NC \NR
62\BC randomizer  \NC \type {\randomseed}
63                    \type {\setrandomseed}
64                    \type {\normaldeviate}
65                    \type {\uniformdeviate}
66                \NC \NR
67\BC utilities   \NC \type {\synctex}
68                \NC \NR
69\BC extensions  \NC \type {\latelua}
70                    \type {\lateluafunction}
71                    \type {\openout}
72                    \type {\write}
73                    \type {\closeout}
74                    \type {\openin}
75                    \type {\read}
76                    \type {\readline}
77                    \type {\closein}
78                    \type {\ifeof}
79                \NC \NR
80\BC control     \NC \type {\suppressfontnotfounderror}
81                    \type {\suppresslongerror}
82                    \type {\suppressprimitiveerror}
83                    \type {\suppressmathparerror}
84                    \type {\suppressifcsnameerror}
85                    \type {\suppressoutererror}
86                    \type {\mathoption}
87                \NC \NR
88\BC system      \NC \type {\primitive}
89                    \type {\ifprimitive}
90                    \type {\formatname}
91                \NC \NR
92\BC ignored     \NC \type {\long}
93                    \type {\outer}
94                    \type {\mag}
95                \NC \NR
96\stoptabulate
97
98The resources and positioning primitives are actually useful but can be defined
99as macros that (via \LUA) inject nodes in the input that suit the macro package
100and backend. The three||letter direction primitives are gone and the numeric
101variants are now leading. There is no need for page and body related directions
102and they don't work well in \LUATEX\ anyway. We only have two directions left.
103Because we can hook in \LUA\ functions that get information about what is expected
104(consumer or provider) there are plenty possibilities for adding functionality
105using this scripting langauge.
106
107The primitive related extensions were not that useful and reliable so they have
108been removed. There are some new variants that will be discussed later. The \type
109{\outer} and \type {\long} prefixes are gone as they don't make much sense
110nowadays and them becoming dummies opened the way to something new: control
111sequence properties that permit protection against as well as controlled
112overloading of definitions. I don't think that (\CONTEXT) users will notice these
113prefixes being gone. The definition and parsing related \type {\suppress..}
114features are now default and can't be changed so related primitives are gone.
115
116The \type {\shipout} primitive does no ship out but just erases the content of
117the box unless of course that has happened already in another way. A macro
118package should implement its own backend and related shipout. Talking of backend,
119the extension primitives that relate to backends can be implemented as part of a
120backend design using generic whatsits. There is only one type of whatsit now. In
121fact we're now closer to original \TEX\ with respect to the extensions.
122
123The \type {img} library has been removed as it's rather bound to the backend. The
124\type {slunicode} library is also gone. There are some helpers in the string
125library that can be used instead and one can write additional \LUA\ code if
126needed. There is no longer a \type {pdf} backend library but we have an up to
127date \PDF\ parsing library on board.
128
129In the \type {node}, \type {tex} and \type {status} library we no longer have
130helpers and variables that relate to the backend. The \LUAMETATEX\ engine is in
131principle \DVI\ and \PDF\ unaware. There are, as mentioned, only generic whatsit
132nodes that can be used for some management related tasks. For instance you can
133use them to implement user nodes. More extensive status information is provided
134in the overhauled status library. All libraries have additional functionality and
135names of functions have been normalized (for as far as possible).
136
137The margin kern nodes are gone and we now use regular kern nodes for them. As a
138consequence there are two extra subtypes indicating the injected left or right
139kern. The glyph field served no real purpose so there was no reason for a special
140kind of node.
141
142The \KPSE\ library is no longer built|-|in, but one can use an external \KPSE\
143library, assuming that it is present on the system, because the engine has a so
144called optional library interface to it. Because there is no backend, quite some
145file related callbacks could go away. The following file related callbacks
146remained (till now):
147
148\starttyping
149find_write_file find_format_file open_data_file
150\stoptyping
151
152The callbacks related to errors are changed:
153
154\starttyping
155intercept_tex_error intercept_lua_error
156show_error_message show_warning_message
157\stoptyping
158
159There is a hook that gets called when one of the fundamental memory structures
160gets reallocated.
161
162\starttyping
163trace_memory
164\stoptyping
165
166When you use the overload protect mechanisms, a callback can be plugged in to handle
167exceptions:
168
169\starttyping
170handle_overload
171\stoptyping
172
173The (job) management hooks are kept:
174
175\starttyping
176process_jobname
177start_run stop_run wrapup_run
178pre_dump
179start_file stop_file
180\stoptyping
181
182Because we use a more generic whatsit model, there is a new callback:
183
184\starttyping
185show_whatsit
186\stoptyping
187
188Because tracing boxes now reports a lot more information, we have a plug in for
189detail:
190
191\starttyping
192get_attribute
193\stoptyping
194
195Being the core of extensibility, the typesetting callbacks of course stayed. This
196is what we ended up with:
197
198\startalign[flushleft,nothyphenated]
199\tt \cldcontext{table.concat(table.sortedkeys(callbacks.list), ", ")}
200\stopalign
201
202As in \LUATEX\ font loading happens with the following callback. This time it
203really needs to be set because there is no built|-|in font loader.
204
205\starttyping
206define_font
207\stoptyping
208
209There are all kinds of subtle differences in the implementation, for instance we
210no longer intercept \type {*} and \type {&} as these were already replaced long
211ago in \TEX\ engines by command line options. Talking of options, only a few are
212left. All input goes via \LUA, even the console. One can program a terminal if
213needed.
214
215We took our time for reaching a stable state in \LUATEX. Among the reasons is the
216fact that most was experimented with in \CONTEXT, which we can adapt to the
217engine as we go. It took many years to decide what to keep and how to do things.
218Of course there are places when things can be improved but that most likely only
219happens in \LUAMETATEX. Contrary to what is sometimes suggested, the
220\LUATEX|-|\CONTEXT\ \MKIV\ combination (assuming matched versions) has been quite
221stable. It made no sense otherwise. Most \CONTEXT\ functionality didn't change
222much at the user level. Of course there have been issues, as is natural with
223everything new and beta, but we have a fast update cycle.
224
225The same is true for \LUAMETATEX\ and \CONTEXT\ \LMTX: it can be used for
226production as usual and in practice \CONTEXT\ users tend to use the beta
227releases, which proves this. Of course, if you use low level features that are
228experimental you're on your own. Also, as with \LUATEX\ it might take many years
229before a long term stable is defined. The good news is that, when the source code
230has become part of the \CONTEXT\ distribution, there is always a properly
231working, more or less long term stable, snapshot.
232
233The error reporting subsystem has been redone quite a bit but is still
234fundamentally the same. We don't really assume interactive usage but if someone
235uses it, it might be noticed that it is not possible to backtrack or inject
236something. Of course it is no big deal to implement all that in \LUA\ if needed.
237It removes a system dependency and makes for a bit cleaner code. In \CONTEXT\ we
238quit on an error simply because one has to fix source anyway and runs are fast
239enough. Logging provides more detail and new primitives can be used to prevent
240clutter in tracing (the more complex a macro package becomes, the more extreme
241tracing becomes).
242
243There are new primitives as well as some extensions to existing primitive
244functionality. These are described in following chapters but there might be
245hidden treasures in the binary. If you locate them, don't automatically assume
246them to stay, some might be part of experiments! There are for instance a few
247csname related definers, we have integer and dimension constants, the macro
248argument parser can be brought in tolerant mode, the repertoire of conditionals
249has been extended, some internals can be controlled (think of normalization of
250lines, hyphenation etc.), and macros can be protected against user overload. Not
251all is discussed in detail in this manual but there are introductions in the
252\CONTEXT\ distribution that explain them. But the \TEX\ kernel is of course
253omnipresent.
254
255\startluacode
256
257local luametatex = tex.primitives()
258local luatex     = table.load("luatex-primitives.lua")
259
260if not luatex then
261    local tex = "\\starttext \\ctxlua {table.save(tex.jobname .. '.lua',tex.primitives())} \\stoptext"
262
263    io.savedata("luatex-primitives.tex",    tex)
264
265    os.execute("context --luatex --once luatex-primitives")
266
267    luatex = table.load("luatex-primitives.lua")
268end
269
270
271if luatex and luametatex then
272
273    luatex     = table.tohash(luatex)
274    luametatex = table.tohash(luametatex)
275
276 -- context.page()
277
278    context("The following primitives are available in \\LUATEX\\ but not in \\LUAMETATEX.  ")
279    context("Some of these are emulated in \\CONTEXT.")
280
281    context.blank()
282    context.startcolumns { n = 2 }
283        for k, v in table.sortedhash(luatex) do
284            if not luametatex[k] then
285                context.type(k)
286                context.crlf()
287            end
288        end
289    context.stopcolumns()
290    context.blank()
291
292 -- context.page()
293
294    context("The following primitives are available in \\LUAMETATEX\\ only. ")
295    context("At some point in time some might be added to \\LUATEX.")
296
297    context.blank()
298    context.startcolumns { n = 2 }
299        for k, v in table.sortedhash(luametatex) do
300            if not luatex[k] then
301                context.type(k)
302                context.crlf()
303            end
304        end
305    context.stopcolumns()
306    context.blank()
307
308end
309
310\stopluacode
311
312As part of a bit more consistency some function names also changed. Names with an
313\type {_} got that removed (as that was the minority). It's easy to provide a
314back mapping if needed (just alias the functions).
315
316{\em Todo: only mention the \LUATEX\ ones.}
317
318\starttabulate[|l|l|l|l|]
319\DB library  \BC old name          \BC new name         \BC comment \NC \NR
320\TB
321\NC language \NC clear_patterns    \NC clearpatterns    \NC \NR
322\NC          \NC clear_hyphenation \NC clearhyphenation \NC \NR
323\NC mplib    \NC italcor           \NC italic           \NC \NR
324\NC          \NC pen_info          \NC peninfo          \NC \NR
325\NC          \NC solve_path        \NC solvepath        \NC \NR
326\NC texio    \NC write_nl          \NC writenl          \NC old name stays \NC \NR
327\NC node     \NC protect_glyph     \NC protectglyph     \NC \NR
328\NC          \NC protect_glyphs    \NC protectglyphs    \NC \NR
329\NC          \NC unprotect_glyph   \NC unprotectglyph   \NC \NR
330\NC          \NC unprotect_glyphs  \NC unprotectglyphs  \NC \NR
331\NC          \NC end_of_math       \NC endofmath        \NC \NR
332\NC          \NC mlist_to_hlist    \NC mlisttohlist     \NC \NR
333\NC          \NC effective_glue    \NC effectiveglue    \NC \NR
334\NC          \NC has_glyph         \NC hasglyph         \NC \NR
335\NC          \NC first_glyph       \NC firstglyph       \NC \NR
336\NC          \NC has_field         \NC hasfield         \NC \NR
337\NC          \NC copy_list         \NC copylist         \NC \NR
338\NC          \NC flush_node        \NC flushnode        \NC \NR
339\NC          \NC flush_list        \NC flushlist        \NC \NR
340\NC          \NC insert_before     \NC insertbefore     \NC \NR
341\NC          \NC insert_after      \NC insertafter      \NC \NR
342\NC          \NC last_node         \NC lastnode         \NC \NR
343\NC          \NC is_zero_glue      \NC iszeroglue       \NC \NR
344\NC          \NC make_extensible   \NC makeextensible   \NC \NR
345\NC          \NC uses_font         \NC usesfont         \NC \NR
346\NC          \NC is_char           \NC ischar           \NC \NR
347\NC          \NC is_direct         \NC isdirect         \NC \NR
348\NC          \NC is_glyph          \NC isglyph          \NC \NR
349\NC          \NC is_node           \NC isnode           \NC \NR
350\NC token    \NC scan_keyword      \NC scankeyword      \NC \NR
351\NC          \NC scan_keywordcs    \NC scankeywordcs    \NC \NR
352\NC          \NC scan_int          \NC scanint          \NC \NR
353\NC          \NC scan_real         \NC scanreal         \NC \NR
354\NC          \NC scan_float        \NC scanfloat        \NC \NR
355\NC          \NC scan_dimen        \NC scandimen        \NC \NR
356\NC          \NC scan_glue         \NC scanglue         \NC \NR
357\NC          \NC scan_toks         \NC scantoks         \NC \NR
358\NC          \NC scan_code         \NC scancode         \NC \NR
359\NC          \NC scan_string       \NC scanstring       \NC \NR
360\NC          \NC scan_argument     \NC scanargument     \NC \NR
361\NC          \NC scan_word         \NC scanword         \NC \NR
362\NC          \NC scan_csname       \NC scancsname       \NC \NR
363\NC          \NC scan_list         \NC scanlist         \NC \NR
364\NC          \NC scan_box          \NC scanbox          \NC \NR
365\LL
366\stoptabulate
367
368It's all part of trying to make the code base consistent but it is sometimes a
369bit annoying. However, that's why we develop this engine independent of the
370\LUATEX\ code base. It's anyway a change that has been on my todo list for quite
371a while because those inconsistencies annoyed me.
372
373\stopchapter
374
375\stopcomponent
376
377