1
2
3\environment luametatexstyle
4
5\startdocument[title=\LUA]
6
7\startsection[title={Introduction}]
8
9In this chapter aspects of the \LUA\ interfaces will be discusses. The \type
10{lua} module described here is rather low level and probably not of much interest
11to the average user as its functions are meant to be used in higher level
12interfaces.
13
14\stopsection
15
16\startsection[title={Initialization}]
17
18\startsubsection[title={A bare bone engine}]
19
20When the \LUAMETATEX\ program launches it will not do much useful. You can
21compare it to computer hardware without (high level) operating system with a
22\TEX\ kernel being the bios. It can interpret \TEX\ code but for typesetting you
23need a reasonable setup. You also need to load fonts, and for output you need a
24backend, and both can be implemented in \LUA. If you dont like that and want to
25get up and running immediately, you will be more happy with \LUATEX, \PDFTEX\ or
26\XETEX, combined with your favorite macro package.
27
28If you just want to play around you can install the \CONTEXT\ distribution which
29(including manuals and some fonts) is tiny compared to a full \TEXLIVE\
30installation and can be run alongside it without problems. If there are issues
31you can go to the usual \CONTEXT\ support platforms and seek help where you can
32find the people who made \LUATEX\ and \LUAMETATEX.
33
34If you use the engine as \TEX\ interpreter you need to set up a few characters.
35Of course one can wonder why this is the case, but lets consider this to be
36educational of nature as it right from the start forces one to wonder what
37category codes are.
38
39\starttyping
40\catcode\{=1 \catcode\}=2 \catcode\#=6
41\stoptyping
42
43After that you can start defining macros. Contrary to \LUATEX\ the \LUAMETATEX\
44engine initializes all the primitives but it will quit when the minimal set of
45callback is not initialized, like a logger. The lack of font loader and backend
46makes that it is not usable for loading an arbitrary macro package that doesnt
47set up these components. There is simply no argument for starting in in original
48\TEX\ mode without \ETEX\ extensions and such.
49
50\stopsubsection
51
52\startsubsection[title={\LUAMETATEX\ as a \LUA\ interpreter}]
53
54Although \LUAMETATEX\ is primarily meant as a \TEX\ engine, it can also serve as
55a stand alone \LUA\ interpreter and there are two ways to make \LUAMETATEX\
56behave like one. The first method uses the command line option \type {luaonly}
57followed by a filename. The second is more automatic: if the only nonoption
58argument (file) on the command line has the extension \type {lmt} or \type {lua}.
59The \type {luc} extension has been dropped because bytecode compiled files are
60not portable and one can always load them indirectly. The \type {lmt} suffix is
61more \CONTEXT\ specific and makes it possible to have files for \LUATEX\ and
62\LUAMETATEX\ alongside.
63
64In interpreter mode, the program will set \LUAs \type {arg[0]} to the found
65script name, pushing preceding options in negative values and the rest of the
66command line in the positive values, just like the \LUA\ interpreter does. The
67program will exit immediately after executing the specified \LUA\ script and is
68thereby effectively just a somewhat bulky stand alone \LUA\ interpreter with a
69bunch of extra preloaded libraries. But we still wanted and managed to keep the
70binary small, somewhere around 3MB, which is okay for a script engine.
71
72When no argument is given, \LUAMETATEX\ will look for a \LUA\ file with the same
73name as the binary and run that one when present. This makes it possible to use
74the engine as a stub. For instance, in \CONTEXT\ a symlink from \type {mtxrun} to
75type {luametatex} will run the \type {mtxrun.lmt} or \type {mtxrun.lua} script
76when present in the same path as the binary itself. As mentioned before first
77checking for (\CONTEXT) \type {lmt} files permits different files for different
78engines in the same path.
79
80\stopsubsection
81
82\startsubsection[title={Other commandline processing}]
83
84When the \LUAMETATEX\ executable starts, it looks for the \type {lua} command
85line option. If there is no such option, the command line is interpreted in a
86similar fashion as the other \TEX\ engines. All options are accepted but only
87some are understood by \LUAMETATEX\ itself:
88
89\starttabulate[lp]
90\FL
91\BC commandline argument \BC explanation \NC \NR
92\TL
93\NC \type{credits} \NC display credits and exit \NC \NR
94\NC \type{fmt=FORMAT} \NC load the format file \type {FORMAT} \NC\NR
95\NC \type{help} \NC display help and exit \NC\NR
96\NC \type{ini} \NC be \type {iniluatex}, for dumping formats \NC\NR
97\NC \type{jobname=STRING} \NC set the job name to \type {STRING} \NC \NR
98\NC \type{lua=FILE} \NC load and execute a \LUA\ initialization script \NC\NR
99\NC \type{version} \NC display version and exit \NC \NR
100\NC \type{permitloadlib} \NC permits loading of external libraries \NC \NR
101\LL
102\stoptabulate
103
104There are less options than with \LUATEX, because one has to deal with them in
105\LUA\ anyway. So for instance there are no options to enter a safer mode or
106control executing programs because this can easily be achieved with a startup
107\LUA\ script, which can interpret whatever options got passed.
108
109Next the initialization script is loaded and executed. From within the script,
110the entire command line is available in the \LUA\ table \type {arg}, beginning
111with \type {arg[0]}, containing the name of the executable. As consequence
112warnings about unrecognized options are suppressed. Command line processing
113happens very early on. So early, in fact, that none of \TEXs initializations
114have taken place yet. The \LUA\ libraries that dont deal with \TEX\ are
115initialized rather soon so you have these available.
116
117\LUAMETATEX\ allows some of the command line options to be overridden by reading
118values from the \type {texconfig} table at the end of script execution (see the
119description of the \type {texconfig} table later on in this document for more
120details on which ones exactly). The value to use for \type {\jobname} is decided
121as follows:
122
123\startitemize
124\startitem
125 If \type {jobname} is given on the command line, its argument will be the
126 value for \type {\jobname}, without any changes. The argument will not be
127 used for actual input so it need not exist. The \type {jobname} switch only
128 controls the \type {\jobname} setting.
129\stopitem
130\startitem
131 Otherwise, \type {\jobname} will be the name of the first file that is read
132 from the file system, with any path components and the last extension (the
133 part following the last \type {.}) stripped off.
134\stopitem
135\startitem
136 There is an exception to the previous point: if the command line goes into
137 interactive mode (by starting with a command) and there are no files input
138 via \type {\everyjob} either, then the \type {\jobname} is set to \type
139 {texput} as a last resort.
140\stopitem
141\stopitemize
142
143So lets summarize this. The handling of what is called job name is a bit complex.
144There can be explicit names set on the command line but when not set they can be
145taken from the \type {texconfig} table.
146
147\starttabulate[lTTT]
148\NC startup filename \NC lua \NC a \LUA\ file \NC \NC \NR
149\NC startup jobname \NC jobname \NC a \TEX\ tex \NC texconfig.jobname \NC \NR
150\NC startup dumpname \NC fmt \NC a format file \NC texconfig.formatname \NC \NR
151\stoptabulate
152
153These names are initialized according to \type {luaonly} or the first filename
154seen in the list of options. Special treatment of \type {} and \type {*} as well
155as interactive startup is gone but we still enter \TEX\ via an forced \type {\input}
156into the input buffer. \footnote {This might change at some point into an explicit
157loading triggered via \LUA.}
158
159When we are in \TEX\ mode at some point the engine needs a filename, for instance
160for opening a log file. At that moment the set jobname becomes the internal one
161and when it has not been set which internalized to jobname but when not set
162becomes \type {texput}. When you see a \type {texput.log} file someplace on your
163system it normally indicates a bad run.
164
165
166
167
168
169The command line option \type{permitloadlib} has to be given when you load
170external libraries via \LUA. Although you could manage this via \LUA\ itself in a
171startup script, the reason for having this as option is the wish for security (at
172some point that became a demand for \LUATEX), so this might give an extra feeling
173of protection.
174
175\stopsubsection
176
177\stopsection
178
179\startsection[title={\LUA\ behaviour}]
180
181\startsubsection[title={The \LUA\ version}]
182
183We currently use \LUA\ version \luaversion\ and will follow developments of the
184language but normally with some delay. Therefore the user needs to keep an eye on
185(subtle) differences in successive versions of the language. Here are a few
186examples.
187
188\LUA s \type {tostring} function (and \type {string.format}) may return values in
189scientific notation, thereby confusing the \TEX\ end of things when it is used as
190the righthand side of an assignment to a \type {\dimen} or \type {\count}. The
191output of these serializers also depend on the \LUA\ version, so in \LUA\ 5.3 you
192can get different output than from 5.2. It is best not to depend the automatic
193cast from string to number and vise versa as this can change in future versions.
194
195When \LUA\ introduced bitwise operators, instead of providing functions in the
196\type {bit32} library, we wanted to use these. The solution in \CONTEXT\ was to
197implement a macro subsystem (kind of like what \CCODE\ does) and replace the
198function calls by native bitwise operations. However, because \LUAJITTEX\ didnt
199evolve we dropped that and when we split the code base between \MKIV\ and \MKXL\
200we went native bitwise. The \type {bit32} library is still there but implemented
201in \LUA\ instead.
202
203\stopsubsection
204
205\startsubsection[title={Locales}]
206
207In stock \LUA, many things depend on the current locale. In \LUAMETATEX, we cant
208do that, because it makes documents nonportable. While \LUAMETATEX\ is running
209if forces the following locale settings:
210
211\starttyping
212LCCTYPE=C
213LCCOLLATE=C
214LCNUMERIC=C
215\stoptyping
216
217There is no way to change that as it would interfere badly with the often
218language specific conversions needed at the \TEX\ end.
219
220\stopsubsection
221
222\stopsection
223
224\startsection[title={\LUA\ modules}]
225
226Of course the regular \LUA\ modules are present. In addition we provide the \type
227{lpeg} library by Roberto Ierusalimschy, This library is not \UNICODEaware,
228but interprets strings on a byteperbyte basis. This mainly means that \type
229{lpeg.S} cannot be used with \UTF8 characters that need more than one byte, and
230thus \type {lpeg.S} will look for one of those two bytes when matching, not the
231combination of the two. The same is true for \type {lpeg.R}, although the latter
232will display an error message if used with multibyte characters. Therefore \type
233{lpeg.R(aä)} results in the message \type {bad argument #1 to R (range must
234have two characters)}, since to \type {lpeg}, \type {ä} is two characters
235(bytes), so \type {aä} totals three. In practice this is no real issue and with
236some care you can deal with \UNICODE\ just fine.
237
238There are some more libraries present. For instance we embed \type {luasocket}
239but contrary to \LUATEX\ dont embed the related \LUA\ code but some patched and
240extended variant. The \type {luafilesystem} module has been replaced by a more
241efficient one that also deals with the \MSWINDOWS\ file and environment
242properties better (\UNICODE\ support in \MSWINDOWS\ dates from before \UTF8
243became dominant so we need to deal with wide \UNICODE16). We dont have a
244\UNICODE\ library because we always did conversions in \LUA, but there are some
245helpers in the \type {string} library, which makes sense because \LUA\ itself is
246now also becoming \UNICODE\ aware.
247
248There are more extensive math libraries and there are libraries that deal with
249encryption and compression. There are also some optional libraries that we do
250interface but that are loaded on demand. The interfaces are as minimal as can be
251because we so much in \LUA, which also means that one can tune behavior to usage
252better.
253
254\stopsection
255
256\startsection[title={Files}]
257
258\startsubsection[title={File syntax}]
259
260\LUAMETATEX\ will accept a braced argument as a file name:
261
262\starttyping
263\input {plain}
264\openin 0 {plain}
265\stoptyping
266
267This allows for embedded spaces, without the need for double quotes. Macro
268expansion takes place inside the argument. Keep in mind that as side effect of
269delegating \IO\ to \LUA\ the \tex {openin} primitive is not provided by the
270engine and has to be implemented by the macro package. This also means that the
271limit on the number of open files is not enforced by the engine.
272
273\stopsubsection
274
275\startsubsection[title={Writing to file}]
276
277Writing to a file in \TEX\ has two forms: delayed and immediate. Delayed writing
278means that the to be written text is anchored in the node list and flushed by the
279backend. As all \IO\ is delegated to \LUA, this also means that it has to deal
280with distinction. In \LUATEX\ the number of open files was already bumped to 127,
281but in \LUAMETATEX\ it depends on the macro package. The special meaning of
282channel 18 was already dropped in \LUATEX\ because we have \type {os.execute}.
283
284\stopsubsection
285
286\stopsection
287
288\startsection[title={Testing}]
289
290For development reasons you can influence the used startup date and time. By
291setting the \type {starttime} variable in the \type {texconfig} table; as with
292other variables we use the internal name there. When Universal Time is needed,
293set the entry \type {useutctime} in the \type {texconfig} table.
294
295In \CONTEXT\ we provide the command line argument \type {nodates} that does
296a bit more than disabling dates; it avoids time dependent information in the
297output file for instance.
298
299\stopsection
300
301\startsection[title={Helpers}]
302
303\startsubsection[title=Basics]
304
305The \type {lua} library is relatively small and only provides a few functions.
306There are many more helpers but these are organized in specific modules for file
307io, handling strings, and manipulating table.
308
309The \LUA\ interpreter is stack bases and when you put a lot of values on the stack
310it can overflow. However, if that is the case youre probably doing something wrong.
311The next function returns the current top and is mainly there for development reasons.
312
313\starttyping[option=LUA]
314function lua.getstacktop ( )
315 return <t:integer>
316end
317\stoptyping
318
319The next example:
320
321\startbuffer
322\startluacode
323context(lua.getstacktop())
324context(lua.getstacktop(1,2,3))
325context(lua.getstacktop(1,2,3,4,5,6))
326\stopluacode
327\stopbuffer
328
329\typebuffer
330
331typesets: \inlinebuffer, so were fine.
332
333\startbuffer
334\startluacode
335context(lua.getstacktop(unpack(token.getprimitives())))
336\stopluacode
337\stopbuffer
338
339\typebuffer
340
341But even this one os okay: \inlinebuffer, because some thousand plus entries is not
342bothering the engine. Of course it makes little sense because now one has to loop
343over the arguments.
344
345
346
347
348
349
350
351The engines exit code can be set with:
352
353\starttyping[option=LUA]
354function lua.setexitcode ( <t:integer> )
355 no return values
356end
357\stoptyping
358
359and queried with:
360
361\starttyping[option=LUA]
362function lua.getexitcode ( )
363 return <t:integer>
364end
365\stoptyping
366
367The name of the startup file, in our case \quote {\tttf \cldcontext
368{file.basename (lua.getstartupfile ())}} with the path part stripped, can be
369fetched with:
370
371\starttyping[option=LUA]
372function lua.getstartupfile ( )
373 return <t:string>
374end
375\stoptyping
376
377The current \LUA\ version, as reported by the next helper, is \cldcontext {lua.getversion ()}.
378
379\starttyping[option=LUA]
380function lua.getversion ( )
381 return todo
382end
383\stoptyping
384
385We provide high resolution timers so that we can more reliable do performance
386tests when needed and for that we have time related helpers. The \type
387{getruntime} function returns the time passed since startup. The \type
388{getcurrenttime} does what its name says. Just play with them to see how it pays
389off. The \type {getpreciseticks} returns a number that can be used later, after a
390similar call, to get a difference. The \type {getpreciseseconds} function gets
391such a tick (delta) as argument and returns the number of seconds. Ticks can
392differ per operating system, but one always creates a reference first and then
393use deltas to this reference.
394
395\stopsubsection
396
397\startsubsection[title=Timers]
398
399\starttyping[option=LUA]
400function lua.getruntime ( )
401 return <t:number> actually an integer
402end
403\stoptyping
404
405\starttyping[option=LUA]
406function lua.getcurrenttime ( )
407 return <t:number> actually an integer
408end
409\stoptyping
410
411\starttyping[option=LUA]
412function lua.getpreciseticks ( )
413 return <t:number> actually an integer
414end
415\stoptyping
416
417\starttyping[option=LUA]
418function lua.getpreciseseconds ( <t:number> ticks )
419 return <t:number>
420end
421\stoptyping
422
423There is a little bit of duplication in the timers; here is what they report at
424this stage of the current run:
425
426\starttabulate[lTlTl]
427 \FL
428 \BC library \NC function \BC result \NC \NR
429 \TL
430 \NC lua \NC getruntime \NC \cldcontext{lua.getruntime()} \NC \NR
431 \NC \NC getcurrenttime \NC \cldcontext{lua.getcurrenttime()} \NC \NR
432 \NC \NC getpreciseticks \NC \cldcontext{lua.getpreciseticks()} \NC \NR
433 \NC \NC getpreciseseconds \NC \cldcontext{lua.getpreciseseconds(lua.getpreciseticks())} \NC \NR
434 \ML
435 \NC os \NC clock \NC \cldcontext{os.clock()} \NC \NR
436 \NC \NC time \NC \cldcontext{os.time()} \NC \NR
437 \NC \NC gettimeofday \NC \cldcontext{os.gettimeofday()} \NC \NR
438 \LL
439\stoptabulate
440
441\stopsubsection
442
443\startsubsection[title=Bytecode registers]
444
445\LUA\ registers can be used to store \LUA\ code chunks. The accepted values for
446assignments are functions and \type {nil}. Likewise, the retrieved value is
447either a function or \type {nil}.
448
449\starttyping[option=LUA]
450function lua.setbytecode (
451 <t:integer> register,
452 <t:function> loader,
453 <t:boolean> strip
454)
455 no return values
456end
457\stoptyping
458
459An example of a valid call is \type {lua.setbytecode(5,loadfile("foo.lua"))}. The complement
460of this helper is:
461
462\starttyping[option=LUA]
463function lua.getbytecode ( <t:integer> register )
464 retrurn <t:bytecode>
465end
466\stoptyping
467
468The codes are stored in the virtual table \type {lua.bytecode}. The contents of
469this array is stored inside the format file as actual \LUA\ bytecode, so it can
470also be used to preload \LUA\ code. The function must not contain any upvalues.
471The associated function calls are:
472
473\starttyping[option=LUA]
474function lua.callbytecode ( <t:integer> register )
475 <t:boolean> success
476end
477\stoptyping
478
479Note that the path of the file is stored in the \LUA\ bytecode to be used in
480stack backtraces and therefore dumped into the format file if the above code is
481used in \INITEX. If it contains private information, i.e. the user name, this
482information is then contained in the format file as well. This should be kept in
483mind when preloading files into a bytecode register in \INITEX.
484
485\stopsubsection
486
487\startsubsection[title=Tables]
488
489You can preallocate tables with these two helpers. The first one preallocates the
490given amount of hash entries and index entries. The \type {newindex} function
491create an indexed table with default values:
492
493\starttyping[option=LUA]
494function lua.newtable ( <t:integer> hashsize, <t:integer> indexsize )
495 return <t:table>
496end
497
498function lua.newindex ( <t:integer> size, <t:whatever> default )
499 return <t:table>
500end
501\stoptyping
502
503\stopsubsection
504
505\startsubsection[title=Nibbles]
506
507Nibbles are half bytes so they run from \type {0x0} upto \type {0xF}. When we
508needed this for math state fields, the helpers made it here.
509
510\starttyping[option=LUA]
511function lua.setnibble ( <t:integer> original, <t:integer> position, <t:integer> value )
512 return <t:integer>
513end
514\stoptyping
515
516\starttyping[option=LUA]
517function lua.getnibble ( <t:integer> original, <t:integer> position )
518 return <t:integer>
519end
520\stoptyping
521
522\starttyping[option=LUA]
523function lua.addnibble ( <t:integer> original, <t:integer> position, <t:integer> value )
524 return <t:integer>
525end
526\stoptyping
527
528\starttyping[option=LUA]
529function lua.subnibble ( <t:integer> original, <t:integer> position, <t:integer> value )
530 return <t:integer>
531end
532\stoptyping
533
534Here a a few examples (positions go from right to left and start at one):
535
536\protected\def\Test#1#2{\NC #2 \NC \cldcontext{"0x\letterpercent#1X",#2} \NC \NR}
537
538\starttabulate[lTr]
539\Test{04}{lua.setnibble(0x0000,2,0x1)}
540\Test{04}{lua.setnibble(0x0000,4,0x7)}
541\Test{01}{lua.getnibble(0x1234,2)}
542\Test{01}{lua.getnibble(0x1234,4)}
543\Test{04}{lua.addnibble(0x0000,2)}
544\Test{04}{lua.addnibble(0x0030,2)}
545\Test{04}{lua.subnibble(0x00F0,2)}
546\Test{04}{lua.subnibble(0x0080,2)}
547\stoptabulate
548
549\stopsubsection
550
551\startsubsection[title=Functions]
552
553The functions table stores functions by index. The index can be used with the
554primitives that call functions by index. In order to prevent interferences a
555macro package should provide some interface to the function call mechanisms, just
556like it does with registers.
557
558\starttyping[option=LUA]
559function lua.getfunctionstable ( )
560 return <t:table>
561end
562\stoptyping
563
564\stopsubsection
565
566\startsubsection[title=Tracing]
567
568The engine also includes the serializer from the \type {luac} program, just
569because it can be interesting to see what \LUA\ does with your code.
570
571\starttyping[option=LUA]
572function luac.print ( <t:string> bytecode, <t:boolean> detailed )
573 nothing to return
574end
575\stoptyping
576
577\stopsubsection
578
579\stopsection
580
581\stopdocument
582
583 |