% language=us runpath=texruns:manuals/luametatex \environment luametatex-style \startcomponent luametatex-tex \startchapter[reference=tex,title={The \TEX\ related libraries}] \startsection[title={The \type {lua} library}][library=lua] \startsubsection[title={Version information}] \topicindex{libraries+\type{lua}} \topicindex{getversion} \topicindex{getstartupfile} \libindex{getversion} \libindex{getstartupfile} This version of the used \LUA\ interpreter (currently {\tttf \cldcontext {lua.getversion()}}) can be queried with: \starttyping v = lua.getversion() \stoptyping The name of used startup file, if at all, is returned by: \starttyping s = lua.getstartupfile() \stoptyping For this document the reported value is: \blank {\ttx \cldcontext {lua.getstartupfile()}} \blank \stopsubsection \startsubsection[title={Table allocators}] \topicindex{tables} \libindex{newtable} \libindex{newindex} Sometimes performance (and memory usage) can benefit a little from it preallocating a table with \type {newtable}: \starttyping t = lua.newtable(100,5000) \stoptyping This preallocates 100 hash entries and 5000 index entries. The \type {newindex} function create an indexed table with preset values: \starttyping
t = lua.newindex(2500,true) \stoptyping \stopsubsection \startsubsection[title={Bytecode registers}] \topicindex{bytecodes} \topicindex{registers+bytecodes} \libindex{bytecode} \libindex{setbytecode} \libindex{getbytecode} \libindex{callbytecode} \LUA\ registers can be used to store \LUA\ code chunks. The accepted values for assignments are functions and \type {nil}. Likewise, the retrieved value is either a function or \type {nil}. \starttyping lua.bytecode[ n] = f f = lua.bytecode[ n] % -- f() \stoptyping The contents of the \type {lua.bytecode} array is stored inside the format file as actual \LUA\ bytecode, so it can also be used to preload \LUA\ code. The function must not contain any upvalues. The associated function calls are: \startfunctioncall lua.setbytecode( n, f) f = lua.getbytecode( n) \stopfunctioncall Note: Since a \LUA\ file loaded using \type {loadfile(filename)} is essentially an anonymous function, a complete file can be stored in a bytecode register like this: \startfunctioncall lua.setbytecode(n,loadfile(filename)) \stopfunctioncall Now all definitions (functions, variables) contained in the file can be created by executing this bytecode register: \startfunctioncall lua.callbytecode(n) \stopfunctioncall Note that the path of the file is stored in the \LUA\ bytecode to be used in stack backtraces and therefore dumped into the format file if the above code is used in \INITEX. If it contains private information, i.e. the user name, this information is then contained in the format file as well. This should be kept in mind when preloading files into a bytecode register in \INITEX. \stopsubsection \startsubsection[title={Introspection}] \libindex{getstacktop} \libindex{getruntime} \libindex{getcurrenttime} \libindex{getpreciseticks} \libindex{getpreciseseconds} The \type {getstacktop} function return a number indicating how full the \LUA\ stack is. This function only makes sense as breakpoint when checking some mechanism going haywire. There are four time related helpers. The \type {getruntime} function returns the time passed since startup. The \type {getcurrenttime} does what its name says. Just play with them to see how it pays off. The \type {getpreciseticks} returns a number that can be used later, after a similar call, to get a difference. The \type {getpreciseseconds} function gets such a tick (delta) as argument and returns the number of seconds. Ticks can differ per operating system, but one always creates a reference first and then deltas to this reference. \stopsubsection \stopsection \startsection[title={The \type {status} library}][library=status] \topicindex{libraries+\type{status}} \libindex{list} \libindex{resetmessages} \libindex{setexitcode} This contains a number of run|-|time configuration items that you may find useful in message reporting, as well as an iterator function that gets all of the names and values as a table. \startfunctioncall
info = status.list() \stopfunctioncall The keys in the table are the known items, the value is the current value. There are toplevel items and items that are tables with subentries. The current list is: \startluacode local list = status.list() context.starttabulate { "|Tw(10em)|Tp|" } context.DB() context("toplevel statistics") context.BC() context.NC() context.NR() context.TB() for k, v in table.sortedhash(list) do if type(v) ~= "table" then context.NC() context(k) context.NC() context(tostring(v)) context.NC() context.NR() end end context.LL() context.stoptabulate() for k, v in table.sortedhash(list) do if type(v) == "table" then context.starttabulate { "|Tw(10em)|Tp|" } context.DB() context(k ..".*") context.BC() context.NC() context.NR() context.TB() for k, v in table.sortedhash(v) do context.NC() context(k) context.NC() context(v == "" and "unset" or tostring(v)) context.NC() context.NR() end context.LL() context.stoptabulate() end end \stopluacode There are also getters for the subtables. The whole repertoire of functions in the \type {status} table is: {\tttf \cldcontext {table . concat ( table . sortedkeys (status), ", ")}}. The error and warning messages can be wiped with the \type {resetmessages} function. The states in subtables relate to memory management and are mostly there for development purposes. The \type {getconstants} query gives back a table with all kind of internal quantities and again these are only relevant for diagnostic and development purposes. Many are good old \TEX\ constants that are describes in the original documentation of the source but some are definitely \LUAMETATEX\ specific. \startluacode context.starttabulate { "|Tw(15em)|Tp|" } context.DB() context("constants.*") context.BC() context.NC() context.NR() context.TB() for k, v in table.sortedhash(status.getconstants()) do if type(v) ~= "table" then context.NC() context(k) context.NC() context(tostring(v)) context.NC() context.NR() end end context.LL() context.stoptabulate() \stopluacode Most variables speak for themselves, some are more obscure. For instance the \type {run_state} variable indicates what the engine is doing: \starttabulate[|l|l|p|] \DB n \NC meaning \NC explanation \NC \NR \TB \NC 0 \NC initializing \NC \type {--ini} mnode \NC \NR \NC 1 \NC updating \NC relates to \prm {overloadmode} \NC \NR \NC 2 \NC production \NC a regular (format driven) run \NC \NR \LL \stoptabulate \stopsection \startsection[title={The \type {tex} library}][library=tex] \startsubsection[title={Introduction}] \topicindex{libraries+\type{tex}} The \type {tex} table contains a large list of virtual internal \TEX\ parameters that are partially writable. The designation \quote {virtual} means that these items are not properly defined in \LUA, but are only front\-ends that are handled by a metatable that operates on the actual \TEX\ values. As a result, most of the \LUA\ table operators (like \type {pairs} and \type {#}) do not work on such items. At the moment, it is possible to access almost every parameter that you can use after \prm {the}, is a single token or is sort of special in \TEX. This excludes parameters that need extra arguments, like \type {\the\scriptfont}. The subset comprising simple integer and dimension registers are writable as well as readable (like \prm {tracingcommands} and \prm {parindent}). \stopsubsection \startsubsection[title={Internal parameter values, \type {set} and \type {get}}] \topicindex{parameters+internal} \libindex{set} \libindex{get} For all the parameters in this section, it is possible to access them directly using their names as index in the \type {tex} table, or by using one of the functions \type {tex.get} and \type {tex.set}. The exact parameters and return values differ depending on the actual parameter, and so does whether \type {tex.set} has any effect. For the parameters that {\em can} be set, it is possible to use \type {global} as the first argument to \type {tex.set}; this makes the assignment global instead of local. \startfunctioncall tex.set (["global",] n, ...) ... = tex.get ( n) \stopfunctioncall Glue is kind of special because there are five values involved. The return value is a \nod {glue_spec} node but when you pass \type {false} as last argument to \type {tex.get} you get the width of the glue and when you pass \type {true} you get all five values. Otherwise you get a node which is a copy of the internal value so you are responsible for its freeing at the \LUA\ end. When you set a glue quantity you can either pass a \nod {glue_spec} or upto five numbers. Beware: as with regular \LUA\ tables you can add values to the \type {tex} table. So, the following is valid: \starttyping tex.foo = 123 \stoptyping When you access a \TEX\ parameter a look up takes place. For read||only variables that means that you will get something back, but when you set them you create a new entry in the table thereby making the original invisible. There are a few special cases that we make an exception for: \type {prevdepth}, \type {prevgraf} and \type {spacefactor}. These normally are accessed via the \type {tex.nest} table: \starttyping tex.nest[tex.nest.ptr].prevdepth = p tex.nest[tex.nest.ptr].spacefactor = s \stoptyping However, the following also works: \starttyping tex.prevdepth = p tex.spacefactor = s \stoptyping Keep in mind that when you mess with node lists directly at the \LUA\ end you might need to update the top of the nesting stack's \type {prevdepth} explicitly as there is no way \LUATEX\ can guess your intentions. By using the accessor in the \type {tex} tables, you get and set the values at the top of the nesting stack. \subsubsection{Integer parameters} \topicindex{parameters+integer} The integer parameters accept and return \LUA\ integers. In some cases the values are checked, trigger other settings or result in some immediate change of behaviour: \ctxlua {document.filteredprimitives ("internal_int")}. Some integer parameters are read only, because they are actually referring not to some internal integer register but to an engine property: \typ {deadcycles}, \typ {insertpenalties}, \typ {parshape}, \typ {interlinepenalties}, \typ {clubpenalties}, \typ {widowpenalties}, \typ {displaywidowpenalties}, \typ {prevgraf} and \typ {spacefactor}. \subsubsection{Dimension parameters} \topicindex{parameters+dimension} The dimension parameters accept \LUA\ numbers (signifying scaled points) or strings (with included dimension). The result is always a number in scaled points. These are read|-|write: \ctxlua {document.filteredprimitives ("internal_dimen")}. These are read|-|only: \typ {pagedepth}, \typ {pagefilllstretch}, \typ {pagefillstretch}, \typ {pagefilstretch}, \typ {pagegoal}, \typ {pageshrink}, \typ {pagestretch} and \typ {pagetotal}. \subsubsection{Direction parameters} \topicindex{parameters+direction} The direction states can be queried with: \typ {gettextdir}, \typ {getlinedir}, \typ {getmathdir} and \typ {getpardir}. You can set them with \typ {settextdir}, \typ {setlinedir}, \typ {setmathdir} and \typ {setpardir}, commands that accept a number. You can also set these parameters as table key|/|values: \typ {textdirection}, \typ {linedirection}, \typ {mathdirection} and \typ {pardirection}, so the next code sets the text direction to \typ {r2l}: \starttyping tex.textdirection = 1 \stoptyping \subsubsection{Glue parameters} \topicindex{parameters+glue} The internal glue parameters accept and return a userdata object that represents a \nod {glue_spec} node: \ctxlua {document.filteredprimitives ("internal_glue")}. \subsubsection{Muglue parameters} \topicindex{parameters+muglue} All muglue parameters are to be used read|-|only and return a \LUA\ string \ctxlua {document.filteredprimitives ("internal_mu_glue")}. \subsubsection{Tokenlist parameters} \topicindex{parameters+tokens} The tokenlist parameters accept and return \LUA\ strings. \LUA\ strings are converted to and from token lists using \prm {the} \prm {toks} style expansion: all category codes are either space (10) or other (12). It follows that assigning to some of these, like \quote {tex.output}, is actually useless, but it feels bad to make exceptions in view of a coming extension that will accept full|-|blown token strings. Here is the lot: \ctxlua {document.filteredprimitives ("internal_toks")}. \stopsubsection \startsubsection[title={Convert commands}] \topicindex{convert commands} All \quote {convert} commands are read|-|only and return a \LUA\ string. The supported commands at this moment are: \ctxlua {document.filteredprimitives ("convert")}. You will get an error message if an operation is not (yet) permitted. Some take an string or number argument, just like at the \TEX\ end some extra input is expected. \stopsubsection \startsubsection[title={Item commands}] \topicindex{last items} All so called \quote {item} commands are read|-|only and return a number. The complete list of these commands is: \ctxlua {document.filteredprimitives ("some_item")}. No all are currently supported but eventually that might be the case. Like the lists in previous sections, there are differences between \LUATEX\ and \LUAMETATEX, where some commands are organized differently in order to provide a consistent \LUA\ interface. \stopsubsection \startsubsection[title={Accessing registers: \type {set*}, \type {get*} and \type {is*}}] \topicindex{attributes} \topicindex{registers} \libindex{attribute} \libindex{setattribute} \libindex{getattribute} \libindex{isattribute} \libindex{count} \libindex{setcount} \libindex{getcount} \libindex{iscount} \libindex{dimen} \libindex{setdimen} \libindex{getdimen} \libindex{isdimen} \libindex{skip} \libindex{setskip} \libindex{getskip} \libindex{isskip} \libindex{muskip} \libindex{setmuskip} \libindex{getmuskip} \libindex{ismuskip} \libindex{glue} \libindex{setglue} \libindex{getglue} \libindex{isglue} \libindex{muglue} \libindex{setmuglue} \libindex{getmuglue} \libindex{ismuglue} \libindex{toks} \libindex{settoks} \libindex{gettoks} \libindex{istoks} \libindex{box} \libindex{setbox} \libindex{getbox} \libindex{isbox} \libindex{scantoks} \libindex{getmark} \TEX's attributes (\prm {attribute}), counters (\prm {count}), dimensions (\prm {dimen}), skips (\prm {skip}, \prm {muskip}) and token (\prm {toks}) registers can be accessed and written to using two times five virtual sub|-|tables of the \type {tex} table: \startthreecolumns \starttyping tex.attribute tex.count tex.dimen tex.skip tex.glue tex.muskip tex.muglue tex.toks \stoptyping \stopthreecolumns It is possible to use the names of relevant \prm {attributedef}, \prm {countdef}, \prm {dimendef}, \prm {skipdef}, or \prm {toksdef} control sequences as indices to these tables: \starttyping tex.count.scratchcounter = 0 enormous = tex.dimen['maxdimen'] \stoptyping In this case, \LUATEX\ looks up the value for you on the fly. You have to use a valid \prm {countdef} (or \prm {attributedef}, or \prm {dimendef}, or \prm {skipdef}, or \prm {toksdef}), anything else will generate an error (the intent is to eventually also allow \type {} and even macros that expand into a number). \startitemize \startitem The count registers accept and return \LUA\ numbers. \stopitem \startitem The dimension registers accept \LUA\ numbers (in scaled points) or strings (with an included absolute dimension; \type {em} and \type {ex} and \type {px} are forbidden). The result is always a number in scaled points. \stopitem \startitem The token registers accept and return \LUA\ strings. \LUA\ strings are converted to and from token lists using \prm {the} \prm {toks} style expansion: all category codes are either space (10) or other (12). \stopitem \startitem The skip registers accept and return \nod {glue_spec} userdata node objects (see the description of the node interface elsewhere in this manual). \stopitem \startitem The glue registers are just skip registers but instead of userdata are verbose. \stopitem \startitem Like the counts, the attribute registers accept and return \LUA\ numbers. \stopitem \stopitemize As an alternative to array addressing, there are also accessor functions defined for all cases, for example, here is the set of possibilities for \prm {skip} registers: \startfunctioncall tex.setskip (["global",] n, s) tex.setskip (["global",] s, s) s = tex.getskip ( n) s = tex.getskip ( s) \stopfunctioncall We have similar setters for \type {count}, \type {dimen}, \type {muskip}, and \type {toks}. Counters and dimen are represented by numbers, skips and muskips by nodes, and toks by strings. Again the glue variants are not using the \nod {glue-spec} userdata nodes. The \type {setglue} function accepts upto five arguments: width, stretch, shrink, stretch order and shrink order. Non|-|numeric values set the property to zero. The \type {getglue} function reports all five properties, unless the second argument is \type {false} in which case only the width is returned. Here is an example using a threesome: \startfunctioncall local d = tex.getdimen("foo") if tex.isdimen("oof") then tex.setdimen("oof",d) end \stopfunctioncall There are six extra skip (glue) related helpers: \startfunctioncall tex.setglue (["global"], n, width, stretch, shrink, stretch_order, shrink_order) tex.setglue (["global"], s, width, stretch, shrink, stretch_order, shrink_order) width, stretch, shrink, stretch_order, shrink_order = tex.getglue ( n) width, stretch, shrink, stretch_order, shrink_order = tex.getglue ( s) \stopfunctioncall The other two are \type {tex.setmuglue} and \type {tex.getmuglue}. There are such helpers for \type {dimen}, \type {count}, \type {skip}, \type {muskip}, \type {box} and \type {attribute} registers but the glue ones are special because they have to deal with more properties. As with the general \type {get} and \type {set} function discussed before, for the skip registers \type {getskip} returns a node and \type {getglue} returns numbers, while \type {setskip} accepts a node and \type {setglue} expects upto 5 numbers. Again, when you pass \type {false} as second argument to \type {getglue} you only get the width returned. The same is true for the \type {mu} variants \type {getmuskip}, \type {setmuskip}, \type {getmuskip} and\type {setmuskip}. For tokens registers we have an alternative where a catcode table is specified: \startfunctioncall tex.scantoks(0,3,"$e=mc^2$") tex.scantoks("global",0,3,"$\int\limits^1_2$") \stopfunctioncall In the function-based interface, it is possible to define values globally by using the string \type {global} as the first function argument. There is a dedicated getter for marks: \type {getmark} that takes two arguments. The first argument is one of \type {top}, \type {bottom}, \type {first}, \type {splitbottom} or \type {splitfirst}, and the second argument is a marks class number. When no arguments are given the current maximum number of classes is returned. When \type {tex.gettoks} gets an extra argument \type {true} it will return a table with userdata tokens. \stopsubsection \startsubsection[title={Character code registers: \type {[get|set]*code[s]}}] \topicindex{characters+codes} \libindex{lccode} \libindex{setlccode} \libindex{getlccode} \libindex{uccode} \libindex{setuccode} \libindex{getuccode} \libindex{sfcode} \libindex{setsfcode} \libindex{getsfcode} \libindex{catcode} \libindex{setcatcode} \libindex{getcatcode} \libindex{mathcode} \libindex{setmathcode} \libindex{getmathcode} \libindex{delcode} \libindex{setdelcode} \libindex{getdelcode} \libindex{setdelcodes} \libindex{getdelcodes} \libindex{setmathcodes} \libindex{getmathcodes} \TEX's character code tables (\prm {lccode}, \prm {uccode}, \prm {sfcode}, \prm {catcode}, \prm {mathcode}, \prm {delcode}) can be accessed and written to using six virtual subtables of the \type {tex} table \startthreecolumns \starttyping tex.lccode tex.uccode tex.sfcode tex.catcode tex.mathcode tex.delcode \stoptyping \stopthreecolumns The function call interfaces are roughly as above, but there are a few twists. \type {sfcode}s are the simple ones: \startfunctioncall tex.setsfcode (["global",] n, s) s = tex.getsfcode ( n) \stopfunctioncall The function call interface for \type {lccode} and \type {uccode} additionally allows you to set the associated sibling at the same time: \startfunctioncall tex.setlccode (["global"], n, lc) tex.setlccode (["global"], n, lc, uc) lc = tex.getlccode ( n) tex.setuccode (["global"], n, uc) tex.setuccode (["global"], n, uc, lc) uc = tex.getuccode ( n) \stopfunctioncall The function call interface for \type {catcode} also allows you to specify a category table to use on assignment or on query (default in both cases is the current one): \startfunctioncall tex.setcatcode (["global"], n, c) tex.setcatcode (["global"], cattable, n, c) lc = tex.getcatcode ( n) lc = tex.getcatcode ( cattable, n) \stopfunctioncall The interfaces for \type {delcode} and \type {mathcode} use small array tables to set and retrieve values: \startfunctioncall tex.setmathcode (["global"], n,
mval )
mval = tex.getmathcode ( n) tex.setdelcode (["global"], n,
dval )
dval = tex.getdelcode ( n) \stopfunctioncall Where the table for \type {mathcode} is an array of 3 numbers, like this: \starttyping { class, family, character } \stoptyping And the table for \type {delcode} is an array with 4 numbers, like this: \starttyping { small_fam, small_char, large_fam, large_char } \stoptyping You can also avoid the table: \startfunctioncall tex.setmathcode (["global"], n, class, family, character) class, family, char = tex.getmathcodes ( n) tex.setdelcode (["global"], n, smallfam, smallchar, largefam, largechar) smallfam, smallchar, largefam, largechar = tex.getdelcodes ( n) \stopfunctioncall Normally, the third and fourth values in a delimiter code assignment will be zero according to \prm {Udelcode} usage, but the returned table can have values there (if the delimiter code was set using \prm {delcode}, for example). Unset \type {delcode}'s can be recognized because \type {dval[1]} is $-1$. \stopsubsection \startsubsection[title={Box registers: \type {[get|set]box}}] \topicindex{registers} \topicindex{boxes} \libindex{box} \libindex{setbox} \libindex{getbox} It is possible to set and query actual boxes, coming for instance from \prm {hbox}, \prm {vbox} or \prm {vtop}, using the node interface as defined in the \type {node} library: \starttyping tex.box \stoptyping for array access, or \starttyping tex.setbox(["global",] n, s) tex.setbox(["global",] cs, s) n = tex.getbox( n) n = tex.getbox( cs) \stoptyping for function|-|based access. In the function-based interface, it is possible to define values globally by using the string \type {global} as the first function argument. Be warned that an assignment like \starttyping tex.box[0] = tex.box[2] \stoptyping does not copy the node list, it just duplicates a node pointer. If \type {\box2} will be cleared by \TEX\ commands later on, the contents of \type {\box0} becomes invalid as well. To prevent this from happening, always use \type {node.copy_list} unless you are assigning to a temporary variable: \starttyping tex.box[0] = node.copy_list(tex.box[2]) \stoptyping \stopsubsection \startsubsection[title={\type {triggerbuildpage}}] \topicindex{pages} \libindex{triggerbuildpage} You should not expect to much from the \type {triggerbuildpage} helpers because often \TEX\ doesn't do much if it thinks nothing has to be done, but it might be useful for some applications. It just does as it says it calls the internal function that build a page, given that there is something to build. \stopsubsection \startsubsection[title={\type {splitbox}}] \topicindex{boxes+split} \libindex{splitbox} You can split a box: \starttyping local vlist = tex.splitbox(n,height,mode) \stoptyping The remainder is kept in the original box and a packaged vlist is returned. This operation is comparable to the \prm {vsplit} operation. The mode can be \type {additional} or \type {exactly} and concerns the split off box. \stopsubsection \startsubsection[title={Accessing math parameters: \type {[get|set]math}}] \topicindex{math+parameters} \topicindex{parameters+math} \libindex{setmath} \libindex{getmath} It is possible to set and query the internal math parameters using: \startfunctioncall tex.setmath(["global",] n, t, n) n = tex.getmath( n, t) \stopfunctioncall As before an optional first parameter \type {global} indicates a global assignment. The first string is the parameter name minus the leading \quote {Umath}, and the second string is the style name minus the trailing \quote {style}. Just to be complete, the values for the math parameter name are: \starttyping quad axis operatorsize overbarkern overbarrule overbarvgap underbarkern underbarrule underbarvgap radicalkern radicalrule radicalvgap radicaldegreebefore radicaldegreeafter radicaldegreeraise stackvgap stacknumup stackdenomdown fractionrule fractionnumvgap fractionnumup fractiondenomvgap fractiondenomdown fractiondelsize limitabovevgap limitabovebgap limitabovekern limitbelowvgap limitbelowbgap limitbelowkern underdelimitervgap underdelimiterbgap overdelimitervgap overdelimiterbgap subshiftdrop supshiftdrop subshiftdown subsupshiftdown subtopmax supshiftup supbottommin supsubbottommax subsupvgap spaceafterscript connectoroverlapmin ordordspacing ordopspacing ordbinspacing ordrelspacing ordopenspacing ordclosespacing ordpunctspacing ordinnerspacing opordspacing opopspacing opbinspacing oprelspacing opopenspacing opclosespacing oppunctspacing opinnerspacing binordspacing binopspacing binbinspacing binrelspacing binopenspacing binclosespacing binpunctspacing bininnerspacing relordspacing relopspacing relbinspacing relrelspacing relopenspacing relclosespacing relpunctspacing relinnerspacing openordspacing openopspacing openbinspacing openrelspacing openopenspacing openclosespacing openpunctspacing openinnerspacing closeordspacing closeopspacing closebinspacing closerelspacing closeopenspacing closeclosespacing closepunctspacing closeinnerspacing punctordspacing punctopspacing punctbinspacing punctrelspacing punctopenspacing punctclosespacing punctpunctspacing punctinnerspacing innerordspacing inneropspacing innerbinspacing innerrelspacing inneropenspacing innerclosespacing innerpunctspacing innerinnerspacing \stoptyping The values for the style parameter are: \starttyping display crampeddisplay text crampedtext script crampedscript scriptscript crampedscriptscript \stoptyping The value is either a number (representing a dimension or number) or a glue spec node representing a muskip for \type {ordordspacing} and similar spacing parameters. \stopsubsection \startsubsection[title={Special list heads: \type {[get|set]list}}] \topicindex{lists} \libindex{lists} \libindex{setlist} \libindex{getlist} The virtual table \type {tex.lists} contains the set of internal registers that keep track of building page lists. \starttabulate[|l|p|] \DB field \BC explanation \NC \NR \TB \NC \type{pageinserthead} \NC circular list of pending insertions \NC \NR \NC \type{contributehead} \NC the recent contributions \NC \NR \NC \type{pagehead} \NC the current page content \NC \NR %NC \type{temphead} \NC \NC \NR \NC \type{holdhead} \NC used for held-over items for next page \NC \NR \NC \type{postadjusthead} \NC head of the (pending) post adjustments \NC \NR \NC \type{preadjusthead} \NC head of the (pending) pre adjustments \NC \NR \NC \type{postmigratehead} \NC head of the (pending) post migrations \NC \NR \NC \type{premigratehead} \NC head of the (pending) pre migrations \NC \NR %NC \type{alignhead} \NC \NC \NR \NC \type{pagediscardshead} \NC head of the discarded items of a page break \NC \NR \NC \type{splitdiscardshead} \NC head of the discarded items in a vsplit \NC \NR \LL \stoptabulate The getter and setter functions are \type {getlist} and \type {setlist}. You have to be careful with what you set as \TEX\ can have expectations with regards to how a list is constructed or in what state it is. \stopsubsection \startsubsection[title={Semantic nest levels: \type {getnest} and \type {ptr}}] \topicindex{nesting} \libindex{nest} \libindex{ptr} %libindex{setnest} % only a message \libindex{getnest} The virtual table \type {nest} contains the currently active semantic nesting state. It has two main parts: a zero-based array of userdata for the semantic nest itself, and the numerical value \type {ptr}, which gives the highest available index. Neither the array items in \type {nest[]} nor \type {ptr} can be assigned to (as this would confuse the typesetting engine beyond repair), but you can assign to the individual values inside the array items, e.g.\ \type {tex.nest[tex.nest.ptr].prevdepth}. \type {tex.nest[tex.nest.ptr]} is the current nest state, \type {nest[0]} the outermost (main vertical list) level. The getter function is \type {getnest}. You can pass a number (which gives you a list), nothing or \type {top}, which returns the topmost list, or the string \type {ptr} which gives you the index of the topmost list. The known fields are: \starttabulate[|l|l|l|p|] \DB key \BC type \BC modes \BC explanation \NC \NR \TB \NC \type{mode} \NC number \NC all \NC the meaning of these numbers depends on the engine and sometimes even the version; you can use \typ {tex.getmodevalues()} to get the mapping: positive values signal vertical, horizontal and math mode, while negative values indicate inner and inline variants \NC \NR \NC \type{modeline} \NC number \NC all \NC source input line where this mode was entered in, negative inside the output routine \NC \NR \NC \type{head} \NC node \NC all \NC the head of the current list \NC \NR \NC \type{tail} \NC node \NC all \NC the tail of the current list \NC \NR \NC \type{prevgraf} \NC number \NC vmode \NC number of lines in the previous paragraph \NC \NR \NC \type{prevdepth} \NC number \NC vmode \NC depth of the previous paragraph \NC \NR \NC \type{spacefactor} \NC number \NC hmode \NC the current space factor \NC \NR \NC \type{direction} \NC node \NC hmode \NC stack used for temporary storage by the line break algorithm \NC \NR \NC \type{noad} \NC node \NC mmode \NC used for temporary storage of a pending fraction numerator, for \prm {over} etc. \NC \NR \NC \type{delimiter} \NC node \NC mmode \NC used for temporary storage of the previous math delimiter, for \prm {middle} \NC \NR \NC \type{mathdir} \NC boolean \NC mmode \NC true when during math processing the \prm {mathdirection} is not the same as the surrounding \prm {textdirection} \NC \NR \NC \type{mathstyle} \NC number \NC mmode \NC the current \prm {mathstyle} \NC \NR \LL \stoptabulate When a second string argument is given to the \type {getnest}, the value with that name is returned. Of course the level must be valid. When \type {setnest} gets a third argument that value is assigned to the field given as second argument. \stopsubsection \startsubsection[reference=sec:luaprint,title={Print functions}] \topicindex{printing} The \type {tex} table also contains the three print functions that are the major interface from \LUA\ scripting to \TEX. The arguments to these three functions are all stored in an in|-|memory virtual file that is fed to the \TEX\ scanner as the result of the expansion of \prm {directlua}. The total amount of returnable text from a \prm {directlua} command is only limited by available system \RAM. However, each separate printed string has to fit completely in \TEX's input buffer. The result of using these functions from inside callbacks is undefined at the moment. \subsubsection{\type {print}} \libindex{print} \startfunctioncall tex.print( s, ...) tex.print( n, s, ...) tex.print(
t) tex.print( n,
t) \stopfunctioncall Each string argument is treated by \TEX\ as a separate input line. If there is a table argument instead of a list of strings, this has to be a consecutive array of strings to print (the first non-string value will stop the printing process). The optional parameter can be used to print the strings using the catcode regime defined by \prm {catcodetable}~\type {n}. If \type {n} is $-1$, the currently active catcode regime is used. If \type {n} is $-2$, the resulting catcodes are the result of \prm {the} \prm {toks}: all category codes are 12 (other) except for the space character, that has category code 10 (space). Otherwise, if \type {n} is not a valid catcode table, then it is ignored, and the currently active catcode regime is used instead. The very last string of the very last \type {tex.print} command in a \prm {directlua} will not have the \prm {endlinechar} appended, all others do. \subsubsection{\type {sprint}} \libindex{sprint} \startfunctioncall tex.sprint( s, ...) tex.sprint( n, s, ...) tex.sprint(
t) tex.sprint( n,
t) \stopfunctioncall Each string argument is treated by \TEX\ as a special kind of input line that makes it suitable for use as a partial line input mechanism: \startitemize[packed] \startitem \TEX\ does not switch to the \quote {new line} state, so that leading spaces are not ignored. \stopitem \startitem No \prm {endlinechar} is inserted. \stopitem \startitem Trailing spaces are not removed. Note that this does not prevent \TEX\ itself from eating spaces as result of interpreting the line. For example, in \starttyping before\directlua{tex.sprint("\\relax")tex.sprint(" in between")}after \stoptyping the space before \type {in between} will be gobbled as a result of the \quote {normal} scanning of \prm {relax}. \stopitem \stopitemize If there is a table argument instead of a list of strings, this has to be a consecutive array of strings to print (the first non-string value will stop the printing process). The optional argument sets the catcode regime, as with \type {tex.print}. This influences the string arguments (or numbers turned into strings). Although this needs to be used with care, you can also pass token or node userdata objects. These get injected into the stream. Tokens had best be valid tokens, while nodes need to be around when they get injected. Therefore it is important to realize the following: \startitemize \startitem When you inject a token, you need to pass a valid token userdata object. This object will be collected by \LUA\ when it no longer is referenced. When it gets printed to \TEX\ the token itself gets copied so there is no interference with the \LUA\ garbage collection. You manage the object yourself. Because tokens are actually just numbers, there is no real extra overhead at the \TEX\ end. \stopitem \startitem When you inject a node, you need to pass a valid node userdata object. The node related to the object will not be collected by \LUA\ when it no longer is referenced. It lives on at the \TEX\ end in its own memory space. When it gets printed to \TEX\ the node reference is used assuming that node stays around. There is no \LUA\ garbage collection involved. Again, you manage the object yourself. The node itself is freed when \TEX\ is done with it. \stopitem \stopitemize If you consider the last remark you might realize that we have a problem when a printed mix of strings, tokens and nodes is reused. Inside \TEX\ the sequence becomes a linked list of input buffers. So, \type {"123"} or \type {"\foo{123}"} gets read and parsed on the fly, while \typ {} already is tokenized and effectively is a token list now. A \typ {} is also tokenized into a token list but it has a reference to a real node. Normally this goes fine. But now assume that you store the whole lot in a macro: in that case the tokenized node can be flushed many times. But, after the first such flush the node is used and its memory freed. You can prevent this by using copies which is controlled by setting \prm {luacopyinputnodes} to a non|-|zero value. This is one of these fuzzy areas you have to live with if you really mess with these low level issues. \subsubsection{\type {tprint}} \libindex{tprint} \startfunctioncall tex.tprint({ n, s, ...}, {...}) \stopfunctioncall This function is basically a shortcut for repeated calls to \type {tex.sprint( n, s, ...)}, once for each of the supplied argument tables. \subsubsection{\type {cprint}} \libindex{cprint} This function takes a number indicating the to be used catcode, plus either a table of strings or an argument list of strings that will be pushed into the input stream. \startfunctioncall tex.cprint( 1," 1: $&{\\foo}") tex.print("\\par") -- a lot of \bgroup s tex.cprint( 2," 2: $&{\\foo}") tex.print("\\par") -- matching \egroup s tex.cprint( 9," 9: $&{\\foo}") tex.print("\\par") -- all get ignored tex.cprint(10,"10: $&{\\foo}") tex.print("\\par") -- all become spaces tex.cprint(11,"11: $&{\\foo}") tex.print("\\par") -- letters tex.cprint(12,"12: $&{\\foo}") tex.print("\\par") -- other characters tex.cprint(14,"12: $&{\\foo}") tex.print("\\par") -- comment triggers \stopfunctioncall % \subsubsection{\type {write}, \type {twrite}, \type {nwrite}} \subsubsection{\type {write}} \libindex{write} % \libindex{twrite} % \libindex{nwrite} \startfunctioncall tex.write( s, ...) tex.write(
t) \stopfunctioncall Each string argument is treated by \TEX\ as a special kind of input line that makes it suitable for use as a quick way to dump information: \startitemize \item All catcodes on that line are either \quote{space} (for '~') or \quote {character} (for all others). \item There is no \prm {endlinechar} appended. \stopitemize If there is a table argument instead of a list of strings, this has to be a consecutive array of strings to print (the first non-string value will stop the printing process). % The functions \type {twrite} and \type {nwrite} can be used to write a token or % node back to \TEX\, possibly intermixed with regular strings that will be % tokenized. You have to make sure that you pass the right data because sometimes % \TEX\ has expectations that need to be met. \stopsubsection \startsubsection[title={Helper functions}] \subsubsection{\type {round}} \topicindex {helpers} \libindex{round} \startfunctioncall n = tex.round( o) \stopfunctioncall Rounds \LUA\ number \type {o}, and returns a number that is in the range of a valid \TEX\ register value. If the number starts out of range, it generates a \quote {number too big} error as well. \subsubsection{\type {scale}} \libindex{scale} \startfunctioncall n = tex.scale( o, delta)
n = tex.scale(table o, delta) \stopfunctioncall Multiplies the \LUA\ numbers \type {o} and \nod {delta}, and returns a rounded number that is in the range of a valid \TEX\ register value. In the table version, it creates a copy of the table with all numeric top||level values scaled in that manner. If the multiplied number(s) are of range, it generates \quote{number too big} error(s) as well. Note: the precision of the output of this function will depend on your computer's architecture and operating system, so use with care! An interface to \LUATEX's internal, 100\% portable scale function will be added at a later date. \subsubsection{\type {number} and \type {romannumeral}} \libindex{number} \libindex{romannumeral} These are the companions to the primitives \prm {number} and \prm {romannumeral}. They can be used like: \startfunctioncall tex.print(tex.romannumeral(123)) \stopfunctioncall \subsubsection{\type {fontidentifier} and \type {fontname}} \libindex{fontidentifier} \libindex{fontname} The first one returns the name only, the second one reports the size too. \startfunctioncall tex.print(tex.fontname(1)) tex.print(tex.fontidentifier(1)) \stopfunctioncall \subsubsection{\type {sp}} \libindex{sp} \startfunctioncall n = tex.sp( o) n = tex.sp( s) \stopfunctioncall Converts the number \type {o} or a string \type {s} that represents an explicit dimension into an integer number of scaled points. For parsing the string, the same scanning and conversion rules are used that \LUATEX\ would use if it was scanning a dimension specifier in its \TEX|-|like input language (this includes generating errors for bad values), expect for the following: \startitemize[n] \startitem only explicit values are allowed, control sequences are not handled \stopitem \startitem infinite dimension units (\type {fil...}) are forbidden \stopitem \startitem \type {mu} units do not generate an error (but may not be useful either) \stopitem \stopitemize \subsubsection{\type {tex.getlinenumber} and \type {tex.setlinenumber}} \libindex{getlinenumber} \libindex{setlinenumber} You can mess with the current line number: \startfunctioncall local n = tex.getlinenumber() tex.setlinenumber(n+10) \stopfunctioncall which can be shortcut to: \startfunctioncall tex.setlinenumber(10,true) \stopfunctioncall This might be handy when you have a callback that reads numbers from a file and combines them in one line (in which case an error message probably has to refer to the original line). Interference with \TEX's internal handling of numbers is of course possible. \subsubsection{\type {error}, \type {show_context} and \type {gethelptext}} \topicindex{errors} \libindex{error} \libindex{show_context} \libindex{gethelptext} \startfunctioncall tex.error( s) tex.error( s,
help) s = tex.gethelptext() \stopfunctioncall This creates an error somewhat like the combination of \prm {errhelp} and \prm {errmessage} would. During this error, deletions are disabled. The array part of the \type {help} table has to contain strings, one for each line of error help. In case of an error the \type {show_context} function will show the current context where we're at (in the expansion). \subsubsection{\type {getfamilyoffont}} \libindex {getfamilyoffont} When you pass a proper family identifier the next helper will return the font currently associated with it. \startfunctioncall id = font.getfamilyoffont( fam) \stopfunctioncall \subsubsection{\type {[set|get]interaction}} \libindex{setinteraction} \libindex{getinteraction} The engine can be in one of four modes: \starttabulate[|lT|l|pl|] \DB value \NC mode \BC meaning \NC \NR \TB \NC 0 \NC batch \NC omits all stops and omits terminal output \NC \NR \NC 1 \NC nonstop \NC omits all stops \NC \NR \NC 2 \NC scroll \NC omits error stops \NC \NR \NC 3 \NC errorstop \NC stops at every opportunity to interact \NC \NR \LL \stoptabulate The mode can be queried and set with: \startfunctioncall i = tex.getinteraction() tex.setinteraction( i) \stopfunctioncall \subsubsection{\type {runtoks} and \type {quittoks}} Because of the fact that \TEX\ is in a complex dance of expanding, dealing with fonts, typesetting paragraphs, messing around with boxes, building pages, and so on, you cannot easily run a nested \TEX\ run (read nested main loop). However, there is an option to force a local run with \type {runtoks}. The content of the given token list register gets expanded locally after which we return to where we triggered this expansion, at the \LUA\ end. Instead a function can get passed that does some work. You have to make sure that at the end \TEX\ is in a sane state and this is not always trivial. A more complex mechanism would complicate \TEX\ itself (and probably also harm performance) so this simple local expansion loop has to do. \startfunctioncall tex.runtoks() tex.runtoks() tex.runtoks() tex.runtoks() \stopfunctioncall When the \prm {tracingnesting} parameter is set to a value larger than~2 some information is reported about the state of the local loop. The return value indicates an error: \starttabulate[|lT|pl|] \DB value \NC meaning \NC \NR \TB \NC 0 \NC no error \NC \NR \NC 1 \NC bad register number \NC \NR \NC 2 \NC unknown macro or register name \NC \NR \NC 3 \NC macro is unsuitable for runtoks (has arguments) \NC \NR \LL \stoptabulate This function has two optional arguments in case a token register is passed: \startfunctioncall tex.runtoks(,force,grouped,obeymode) \stopfunctioncall Inside for instance an \type {\edef} the \type {runtoks} function behaves (at least tries to) like it were an \type {\the}. This prevents unwanted side effects: normally in such an definition tokens remain tokens and (for instance) characters don't become nodes. With the second argument you can force the local main loop, no matter what. The third argument adds a level of grouping. The last argument tells the scanner to stay in the current mode. You can quit the local loop with \type {\endlocalcontrol} or from the \LUA\ end with \type {tex.quittoks}. In that case you end one level up! Of course in the end that can mean that you arrive at the main level in which case an extra end will trigger a redundancy warning (not an abort!). \subsubsection{\type {forcehmode}} \libindex{forcehmode} An example of a (possible error triggering) complication is that \TEX\ expects to be in some state, say horizontal mode, and you have to make sure it is when you start feeding back something from \LUA\ into \TEX. Normally a user will not run into issues but when you start writing tokens or nodes or have a nested run there can be situations that you need to run \type {forcehmode}. There is no recipe for this and intercepting possible cases would weaken \LUATEX's flexibility. \subsubsection{\type {hashtokens}} \libindex{hashtokens} \topicindex{hash} \startfunctioncall for i,v in pairs (tex.hashtokens()) do ... end \stopfunctioncall Returns a list of names. This can be useful for debugging, but note that this also reports control sequences that may be unreachable at this moment due to local redefinitions: it is strictly a dump of the hash table. You can use \type {token.create} to inspect properties, for instance when the \type {command} key in a created table equals \type {123}, you have the \type {cmdname} value \type {undefined_cs}. \subsubsection{\type {definefont}} \topicindex{fonts+defining} \libindex{definefont} \startfunctioncall tex.definefont( csname, fontid) tex.definefont( global, csname, fontid) \stopfunctioncall Associates \type {csname} with the internal font number \type {fontid}. The definition is global if (and only if) \type {global} is specified and true (the setting of \type {globaldefs} is not taken into account). \stopsubsection \startsubsection[reference=luaprimitives,title={Functions for dealing with primitives}] \subsubsection{\type {enableprimitives}} \libindex{enableprimitives} \topicindex{initialization} \topicindex{primitives} \startfunctioncall tex.enableprimitives( prefix,
primitive names) \stopfunctioncall This function accepts a prefix string and an array of primitive names. For each combination of \quote {prefix} and \quote {name}, the \type {tex.enableprimitives} first verifies that \quote {name} is an actual primitive (it must be returned by one of the \type {tex.extraprimitives} calls explained below, or part of \TEX82, or \prm {directlua}). If it is not, \type {tex.enableprimitives} does nothing and skips to the next pair. But if it is, then it will construct a csname variable by concatenating the \quote {prefix} and \quote {name}, unless the \quote {prefix} is already the actual prefix of \quote {name}. In the latter case, it will discard the \quote {prefix}, and just use \quote {name}. Then it will check for the existence of the constructed csname. If the csname is currently undefined (note: that is not the same as \prm {relax}), it will globally define the csname to have the meaning: run code belonging to the primitive \quote {name}. If for some reason the csname is already defined, it does nothing and tries the next pair. An example: \starttyping tex.enableprimitives('LuaTeX', {'formatname'}) \stoptyping will define \type {\LuaTeXformatname} with the same intrinsic meaning as the documented primitive \prm {formatname}, provided that the control sequences \type {\LuaTeXformatname} is currently undefined. When \LUATEX\ is run with \type {--ini} only the \TEX82 primitives and \prm {directlua} are available, so no extra primitives {\bf at all}. If you want to have all the new functionality available using their default names, as it is now, you will have to add \starttyping \ifx\directlua\undefined \else \directlua {tex.enableprimitives('',tex.extraprimitives ())} \fi \stoptyping near the beginning of your format generation file. Or you can choose different prefixes for different subsets, as you see fit. Calling some form of \type {tex.enableprimitives} is highly important though, because if you do not, you will end up with a \TEX82-lookalike that can run \LUA\ code but not do much else. The defined csnames are (of course) saved in the format and will be available at runtime. \subsubsection{\type {extraprimitives}} \libindex{extraprimitives} \startfunctioncall
t = tex.extraprimitives( s, ...) \stopfunctioncall This function returns a list of the primitives that originate from the engine(s) given by the requested string value(s). The possible values and their (current) return values are given in the following table. In addition the somewhat special primitives \quote{\tex{ }}, \quote{\tex {/}} and \quote{\type {-}} are defined. \starttabulate[|l|pl|] \DB name \BC values \NC \NR \TB \NC tex \NC \ctxlua{document.showprimitives('tex') } \NC \NR \NC core \NC \ctxlua{document.showprimitives('core') } \NC \NR \NC etex \NC \ctxlua{document.showprimitives('etex') } \NC \NR \NC luatex \NC \ctxlua{document.showprimitives('luatex') } \NC \NR \LL \stoptabulate Note that \type {luatex} does not contain \type {directlua}, as that is considered to be a core primitive, along with all the \TEX82 primitives, so it is part of the list that is returned from \type {'core'}. Running \type {tex.extraprimitives} will give you the complete list of primitives \type {-ini} startup. It is exactly equivalent to \type {tex.extraprimitives("etex","luatex")}. \subsubsection{\type {primitives}} \libindex{primitives} \startfunctioncall
t = tex.primitives() \stopfunctioncall This function returns a list of all primitives that \LUATEX\ knows about. \stopsubsection \startsubsection[title={Core functionality interfaces}] \subsubsection{\type {badness}} \libindex{badness} \startfunctioncall b = tex.badness( t, s) \stopfunctioncall This helper function is useful during linebreak calculations. \type {t} and \type {s} are scaled values; the function returns the badness for when total \type {t} is supposed to be made from amounts that sum to \type {s}. The returned number is a reasonable approximation of \mathematics {100(t/s)^3}; \subsubsection{\type {tex.resetparagraph}} \topicindex {paragraphs+reset} \libindex{resetparagraph} This function resets the parameters that \TEX\ normally resets when a new paragraph is seen. \subsubsection{\type {linebreak}} \topicindex {linebreaks} \libindex{linebreak} \startfunctioncall local nodelist,
info = tex.linebreak( listhead,
parameters) \stopfunctioncall The understood parameters are as follows: \starttabulate[|l|l|p|] \DB name \BC type \BC explanation \NC \NR \TB \NC \type{pardir} \NC string \NC \NC \NR \NC \type{pretolerance} \NC number \NC \NC \NR \NC \type{tracingparagraphs} \NC number \NC \NC \NR \NC \type{tolerance} \NC number \NC \NC \NR \NC \type{looseness} \NC number \NC \NC \NR \NC \type{hyphenpenalty} \NC number \NC \NC \NR \NC \type{exhyphenpenalty} \NC number \NC \NC \NR \NC \type{pdfadjustspacing} \NC number \NC \NC \NR \NC \type{adjdemerits} \NC number \NC \NC \NR \NC \type{protrudechars} \NC number \NC \NC \NR \NC \type{linepenalty} \NC number \NC \NC \NR \NC \type{lastlinefit} \NC number \NC \NC \NR \NC \type{doublehyphendemerits} \NC number \NC \NC \NR \NC \type{finalhyphendemerits} \NC number \NC \NC \NR \NC \type{hangafter} \NC number \NC \NC \NR \NC \type{interlinepenalty} \NC number or table \NC if a table, then it is an array like \prm {interlinepenalties} \NC \NR \NC \type{clubpenalty} \NC number or table \NC if a table, then it is an array like \prm {clubpenalties} \NC \NR \NC \type{widowpenalty} \NC number or table \NC if a table, then it is an array like \prm {widowpenalties} \NC \NR \NC \type{brokenpenalty} \NC number \NC \NC \NR \NC \type{emergencystretch} \NC number \NC in scaled points \NC \NR \NC \type{hangindent} \NC number \NC in scaled points \NC \NR \NC \type{hsize} \NC number \NC in scaled points \NC \NR \NC \type{leftskip} \NC glue_spec node \NC \NC \NR \NC \type{rightskip} \NC glue_spec node \NC \NC \NR \NC \type{parshape} \NC table \NC \NC \NR \LL \stoptabulate Note that there is no interface for \prm {displaywidowpenalties}, you have to pass the right choice for \type {widowpenalties} yourself. It is your own job to make sure that \type {listhead} is a proper paragraph list: this function does not add any nodes to it. To be exact, if you want to replace the core line breaking, you may have to do the following (when you are not actually working in the \cbk {pre_linebreak_filter} or \cbk {linebreak_filter} callbacks, or when the original list starting at listhead was generated in horizontal mode): \startitemize \startitem add an \quote {indent box} and perhaps a \nod {par} node at the start (only if you need them) \stopitem \startitem replace any found final glue by an infinite penalty (or add such a penalty, if the last node is not a glue) \stopitem \startitem add a glue node for the \prm {parfillskip} after that penalty node \stopitem \startitem make sure all the \type {prev} pointers are OK \stopitem \stopitemize The result is a node list, it still needs to be vpacked if you want to assign it to a \prm {vbox}. The returned \type {info} table contains four values that are all numbers: \starttabulate[|l|p|] \DB name \BC explanation \NC \NR \TB \NC prevdepth \NC depth of the last line in the broken paragraph \NC \NR \NC prevgraf \NC number of lines in the broken paragraph \NC \NR \NC looseness \NC the actual looseness value in the broken paragraph \NC \NR \NC demerits \NC the total demerits of the chosen solution \NC \NR \LL \stoptabulate Note there are a few things you cannot interface using this function: You cannot influence font expansion other than via \type {pdfadjustspacing}, because the settings for that take place elsewhere. The same is true for hbadness and hfuzz etc. All these are in the \type {hpack} routine, and that fetches its own variables via globals. \subsubsection{\type {shipout}} \topicindex {shipout} \libindex{shipout} \startfunctioncall tex.shipout( n) \stopfunctioncall Ships out box number \type {n} to the output file, and clears the box register. \subsubsection{\type {getpagestate}} \topicindex {pages} \libindex{getpagestate} This helper reports the current page state: \type {empty}, \type {box_there} or \type {inserts_only} as integer value. \subsubsection{\type {getlocallevel}} \topicindex {nesting} \libindex{getlocallevel} This integer reports the current level of the local loop. It's only useful for debugging and the (relative state) numbers can change with the implementation. \stopsubsection \startsubsection[reference=synctex,title={Functions related to synctex}] \topicindex {synctex} \libindex{setsynctexmode} \libindex{getsynctexmode} \libindex{setsynctexnofiles} \libindex{setsynctextag} \libindex{getsynctextag} \libindex{forcesynctextag} \libindex{setsynctexline} \libindex{getsynctexline} \libindex{forcesynctexline} The next helpers only make sense when you implement your own synctex logic. Keep in mind that the library used in editors assumes a certain logic and is geared for plain and \LATEX, so after a decade users expect a certain behaviour. \starttabulate[|l|p|] \DB name \BC explanation \NC \NR \TB \NC \type{setsynctexmode} \NC \type {0} is the default and used normal synctex logic, \type {1} uses the values set by the next helpers while \type {2} also sets these for glyph nodes; \type{3} sets glyphs and glue and \type {4} sets only glyphs \NC \NR \NC \type{setsynctextag} \NC set the current tag (file) value (obeys save stack) \NC \NR \NC \type{setsynctexline} \NC set the current line value (obeys save stack) \NC \NR \NC \type{setsynctexnofiles} \NC disable synctex file logging \NC \NR \NC \type{getsynctexmode} \NC returns the current mode (for values see above) \NC \NR \NC \type{getsynctextag} \NC get the currently set value of tag (file) \NC \NR \NC \type{getsynctexline} \NC get the currently set value of line \NC \NR \NC \type{forcesynctextag} \NC overload the tag (file) value (\type {0} resets) \NC \NR \NC \type{forcesynctexline} \NC overload the line value (\type {0} resets) \NC \NR \LL \stoptabulate The last one is somewhat special. Due to the way files are registered in \SYNCTEX\ we need to explicitly disable that feature if we provide our own alternative if we want to avoid that overhead. Passing a value of 1 disables registering. \stopsubsection \stopsection \startsection[title={The \type {texconfig} table},reference=texconfig][library=texconfig] \topicindex{libraries+\type{texconfig}} \topicindex {configuration} This is a table that is created empty. A startup \LUA\ script could fill this table with a number of settings that are read out by the executable after loading and executing the startup file. Watch out: some keys are different from \LUATEX, which is a side effect of a more granular and dynamic memory management. \starttabulate[|l|l|l|l|] \DB key \BC type \BC default \BC comment \NC \NR \TB \NC \type{buffersize} \NC number/table \NC 1000000 \NC input buffer bytes \NC \NR \NC \type{filesize} \NC number/table \NC 1000 \NC max number of open files \NC \NR \NC \type{fontsize} \NC number/table \NC 250 \NC number of permitted fonts \NC \NR \NC \type{hashsize} \NC number/table \NC 150000 \NC number of hash entries \NC \NR \NC \type{inputsize} \NC number/table \NC 10000 \NC maximum input stack \NC \NR \NC \type{languagesize} \NC number/table \NC 250 \NC number of permitted languages \NC \NR \NC \type{marksize} \NC number/table \NC 50 \NC number of mark classes \NC \NR \NC \type{nestsize} \NC number/table \NC 1000 \NC max depth of nesting \NC \NR \NC \type{nodesize} \NC number/table \NC 1000000 \NC max node memory (various size) \NC \NR \NC \type{parametersize} \NC number/table \NC 20000 \NC max size of parameter stack \NC \NR \NC \type{poolsize} \NC number/table \NC 10000000 \NC max number of string bytes \NC \NR \NC \type{savesize} \NC number/table \NC 100000 \NC mas size of save stack \NC \NR \NC \type{stringsize} \NC number/table \NC 150000 \NC max number of strings \NC \NR \NC \type{tokensize} \NC number/table \NC 1000000 \NC max token memory \NC \NR \ML \NC \type{expandsize} \NC number/table \NC 10000 \NC max expansion nesting \NC \NR \NC \type{propertiessize} \NC number \NC 0 \NC initial size of node properties table \NC \NR \NC \type{functionsize} \NC number \NC 0 \NC initial size of \LUA\ functions table \NC \NR \NC \type{errorlinesize} \NC number \NC 79 \NC how much or an error is shown \NC \NR \NC \type{halferrorlinesize} \NC number \NC 50 \NC idem \NC \NR \ML \NC \type{formatname} \NC string \NC \NC \NC \NR \NC \type{jobname} \NC string \NC \NC \NC \NR \ML \NC \type{starttime} \NC number \NC \NC for testing only \NC \NR \NC \type{useutctime} \NC number \NC \NC for testing only \NC \NR \NC \type{permitloadlib} \NC number \NC \NC for testing only \NC \NR \LL \stoptabulate If no format name or jobname is given on the command line, the related keys will be tested first instead of simply quitting. The statistics library has methods for tracking down how much memory is available and has been configured. The size parameters take a number (for the maximum allocated size) or a table with three possible keys: \type {size}, \type {plus} (for extra size) and step for the increment when more memory is needed. They all start out with a hard coded minimum and also have an hard coded maximum, the the configured size sits somewhere between these. \stopsection \startsection[title={The \type {texio} library}][library=texio] \topicindex{libraries+\type{texio}} \topicindex{\IO} This library takes care of the low|-|level I/O interface: writing to the log file and|/|or console. \startsubsection[title={\type {write} and \type {writeselector}}] \libindex{write} \libindex{writeselector} \startfunctioncall texio.write( target, s, ...) texio.write( s, ...) texio.writeselector( s, ...) \stopfunctioncall Without the \type {target} argument, writes all given strings to the same location(s) \TEX\ writes messages to at this moment. If \prm {batchmode} is in effect, it writes only to the log, otherwise it writes to the log and the terminal. The optional \type {target} can be one of \type {terminal}, \type {logfile} or \type {terminal_and_logfile}. Note: If several strings are given, and if the first of these strings is or might be one of the targets above, the \type {target} must be specified explicitly to prevent \LUA\ from interpreting the first string as the target. \stopsubsection \startsubsection[title={\type {writenl} and \type {writeselectornl}}] \libindex{writenl} \libindex{writeselectornl} \startfunctioncall texio.writenl( target, s, ...) texio.writenl( s, ...) texio.writeselectornl( target, ...) \stopfunctioncall This function behaves like \type {texio.write}, but makes sure that the given strings will appear at the beginning of a new line. You can pass a single empty string if you only want to move to the next line. The selector variants always expect a selector, so there is no misunderstanding if \type {logfile} is a string or selector. \stopsubsection \startsubsection[title={\type {setescape}}] \libindex{setescape} You can disable \type {^^} escaping of control characters by passing a value of zero. \stopsubsection \startsubsection[title={\type {closeinput}}] \libindex{closeinput} This function should be used with care. It acts as \prm {endinput} but at the \LUA\ end. You can use it to (sort of) force a jump back to \TEX. Normally a \LUA\ call will just collect prints and at the end bump an input level and flush these prints. This function can help you stay at the current level but you need to know what you're doing (or more precise: what \TEX\ is doing with input). \stopsubsection \stopsection \startsection[title={The \type {token} library}][library=token] \startsubsection[title={The scanner}] \topicindex{libraries+\type{token}} \topicindex{tokens} \libindex{scankeyword} \libindex{scankeywordcs} \libindex{scanint} \libindex{scanreal} \libindex{scanfloat} \libindex{scandimen} \libindex{scanglue} \libindex{scantoks} \libindex{scancode} \libindex{scanstring} \libindex{scanargument} \libindex{scanword} \libindex{scancsname} \libindex{scanlist} The token library provides means to intercept the input and deal with it at the \LUA\ level. The library provides a basic scanner infrastructure that can be used to write macros that accept a wide range of arguments. This interface is on purpose kept general and as performance is quite okay so one can build additional parsers without too much overhead. It's up to macro package writers to see how they can benefit from this as the main principle behind \LUATEX\ is to provide a minimal set of tools and no solutions. The scanner functions are probably the most intriguing. \starttabulate[|l|l|p|] \DB function \BC argument \BC result \NC \NR \TB \NC \type{scankeyword} \NC string \NC returns true if the given keyword is gobbled; as with the regular \TEX\ keyword scanner this is case insensitive (and \ASCII\ based) \NC \NR \NC \type{scankeywordcs} \NC string \NC returns true if the given keyword is gobbled; this variant is case sensitive and also suitable for \UTF8 \NC \NR \NC \type{scanint} \NC \NC returns an integer \NC \NR \NC \type{scanreal} \NC \NC returns a number from e.g.\ \type {1}, \type {1.1}, \type {.1} with optional collapsed signs \NC \NR \NC \type{scanfloat} \NC \NC returns a number from e.g.\ \type {1}, \type {1.1}, \type {.1}, \type {1.1E10}, , \type {.1e-10} with optional collapsed signs \NC \NR \NC \type{scandimen} \NC infinity, mu-units \NC returns a number representing a dimension or two numbers being the filler and order \NC \NR \NC \type{scanglue} \NC mu-units \NC returns a glue spec node \NC \NR \NC \type{scantoks} \NC definer, expand \NC returns a table of tokens \NC \NR \NC \type{scancode} \NC bitset \NC returns a character if its category is in the given bitset (representing catcodes) \NC \NR \NC \type{scanstring} \NC \NC returns a string given between \type {{}}, as \type {\macro} or as sequence of characters with catcode 11 or 12 \NC \NR \NC \type{scanargument} \NC \NC this one is simular to \type {scanstring} but also accepts a \type {\cs} (which then get expanded) \NC \NR \NC \type{scanword} \NC \NC returns a sequence of characters with catcode 11 or 12 as string \NC \NR \NC \type{scancsname} \NC \NC returns \type {foo} after scanning \type {\foo} \NC \NR \NC \type{scanlist} \NC \NC picks up a box specification and returns a \type {[h|v]list} node \NC \NR \LL \stoptabulate The integer, dimension and glue scanners take an extra optional argument that signals that en optional equal is permitted. The scanners can be considered stable apart from the one scanning for a token. The \type {scancode} function takes an optional number, the \type {scankeyword} function a normal \LUA\ string. The \type {infinity} boolean signals that we also permit \type {fill} as dimension and the \type {mu-units} flags the scanner that we expect math units. When scanning tokens we can indicate that we are defining a macro, in which case the result will also provide information about what arguments are expected and in the result this is separated from the meaning by a separator token. The \type {expand} flag determines if the list will be expanded. The \type {scanargument} function expands the given argument. When a braced argument is scanned, expansion can be prohibited by passing \type {false} (default is \type {true}). In case of a control sequence passing \type {false} will result in a one|-|level expansion (the meaning of the macro). The string scanner scans for something between curly braces and expands on the way, or when it sees a control sequence it will return its meaning. Otherwise it will scan characters with catcode \type {letter} or \type {other}. So, given the following definition: \startbuffer \def\oof{oof} \def\foo{foo-\oof} \stopbuffer \typebuffer \getbuffer we get: \starttabulate[|l|Tl|l|] \DB name \BC result \NC \NR \TB \NC \type {\directlua{token.scanstring()}{foo}} \NC \directlua{context("{\\red\\type {"..token.scanstring().."}}")} {foo} \NC full expansion \NC \NR \NC \type {\directlua{token.scanstring()}foo} \NC \directlua{context("{\\red\\type {"..token.scanstring().."}}")} foo \NC letters and others \NC \NR \NC \type {\directlua{token.scanstring()}\foo} \NC \directlua{context("{\\red\\type {"..token.scanstring().."}}")}\foo \NC meaning \NC \NR \LL \stoptabulate The \type {\foo} case only gives the meaning, but one can pass an already expanded definition (\prm {edef}'d). In the case of the braced variant one can of course use the \prm {detokenize} and \prm {unexpanded} primitives since there we do expand. The \type {scanword} scanner can be used to implement for instance a number scanner. An optional boolean argument can signal that a trailing space or \type {\relax} should be gobbled: \starttyping function token.scannumber(base) return tonumber(token.scanword(),base) end \stoptyping This scanner accepts any valid \LUA\ number so it is a way to pick up floats in the input. You can use the \LUA\ interface as follows: \starttyping \directlua { function mymacro(n) ... end } \def\mymacro#1{% \directlua { mymacro(\number\dimexpr#1) }% } \mymacro{12pt} \mymacro{\dimen0} \stoptyping You can also do this: \starttyping \directlua { function mymacro() local d = token.scandimen() ... end } \def\mymacro{% \directlua { mymacro() }% } \mymacro 12pt \mymacro \dimen0 \stoptyping It is quite clear from looking at the code what the first method needs as argument(s). For the second method you need to look at the \LUA\ code to see what gets picked up. Instead of passing from \TEX\ to \LUA\ we let \LUA\ fetch from the input stream. In the first case the input is tokenized and then turned into a string, then it is passed to \LUA\ where it gets interpreted. In the second case only a function call gets interpreted but then the input is picked up by explicitly calling the scanner functions. These return proper \LUA\ variables so no further conversion has to be done. This is more efficient but in practice (given what \TEX\ has to do) this effect should not be overestimated. For numbers and dimensions it saves a bit but for passing strings conversion to and from tokens has to be done anyway (although we can probably speed up the process in later versions if needed). \stopsubsection \startsubsection[title={Picking up one token}] \libindex {scannext} \libindex {scannextexpanded} \libindex {skipnext} \libindex {skipnextexpanded} \libindex {peeknext} \libindex {peeknextexpanded} \libindex {scantoken} \libindex {expand} The scanners look for a sequence. When you want to pick up one token from the input you use \type {scannext}. This creates a token with the (low level) properties as discussed next. This token is just the next one. If you want to enforce expansion first you can use \type {scantoken} or the \type {_expanded} variants. Internally tokens are characterized by a number that packs a lot of information. In order to access the bits of information a token is wrapped in a userdata object. The \type {expand} function will trigger expansion of the next token in the input. This can be quite unpredictable but when you call it you probably know enough about \TEX\ not to be too worried about that. It basically is a call to the internal expand related function. \starttabulate[|lT|p|] \DB name \BC explanation \NC \NR \TB \NC scannext \NC get the next token \NC \NR \NC scannextexpanded \NC get the next expanded token \NC \NR \NC skipnext \NC skip the next token \NC \NR \NC skipnextexpanded \NC skip the next expanded token \NC \NR \NC peeknext \NC get the next token and put it back in the input \NC \NR \NC peeknextexpanded \NC get the next expanded token and put it back in the input \NC \NR \LL \stoptabulate The peek function accept a boolean argument that triggers skipping spaces and alike. \stopsubsection \startsubsection[title={Creating tokens}] \libindex{create} \libindex{new} \libindex{is_defined} \libindex{is_token} \libindex{biggest_char} \libindex{commands} \libindex{command_id} \libindex{getcommand} \libindex{getcmdname} \libindex{getcsname} \libindex{getid} \libindex{getactive} \libindex{getexpandable} \libindex{getprotected} \libindex{getmode} \libindex{getindex} \libindex{gettok} \libindex{getfrozen} \libindex{getuser} \libindex{scannext} The creator function can be used as follows: \starttyping local t = token.create("relax") \stoptyping This gives back a token object that has the properties of the \prm {relax} primitive. The possible properties of tokens are: \starttabulate[|l|p|] \DB name \BC explanation \NC \NR \TB \NC \type {command} \NC a number representing the internal command number \NC \NR \NC \type {cmdname} \NC the type of the command (for instance the catcode in case of a character or the classifier that determines the internal treatment) \NC \NR \NC \type {csname} \NC the associated control sequence (if applicable) \NC \NR \NC \type {id} \NC the unique id of the token \NC \NR \NC \type {tok} \NC the full token number as stored in \TEX \NC \NR \NC \type {active} \NC a boolean indicating the active state of the token \NC \NR \NC \type {expandable} \NC a boolean indicating if the token (macro) is expandable \NC \NR \NC \type {protected} \NC a boolean indicating if the token (macro) is protected \NC \NR \NC \type {frozen} \NC a boolean indicating if the token is a frozen command \NC \NR \NC \type {user} \NC a boolean indicating if the token is a user defined command \NC \NR \NC \type {index} \NC a number that indicated the subcommand; differs per command \NC \NR \LL \stoptabulate Alternatively you can use a getter \type {get} to access a property of a token. The numbers that represent a catcode are the same as in \TEX\ itself, so using this information assumes that you know a bit about \TEX's internals. The other numbers and names are used consistently but are not frozen. So, when you use them for comparing you can best query a known primitive or character first to see the values. You can ask for a list of commands: \starttyping local t = token.commands() \stoptyping The id of a token class can be queried as follows: \starttyping local id = token.command_id("math_shift") \stoptyping If you really know what you're doing you can create character tokens by not passing a string but a number: \starttyping local letter_x = token.create(string.byte("x")) local other_x = token.create(string.byte("x"),12) \stoptyping Passing weird numbers can give side effects so don't expect too much help with that. As said, you need to know what you're doing. The best way to explore the way these internals work is to just look at how primitives or macros or \prm {chardef}'d commands are tokenized. Just create a known one and inspect its fields. A variant that ignores the current catcode table is: \starttyping local whatever = token.new(123,12) \stoptyping You can test if a control sequence is defined with \type {is_defined}, which accepts a string and returns a boolean: \starttyping local okay = token.is_defined("foo") \stoptyping The largest character possible is returned by \type {biggest_char}, just in case you need to know that boundary condition. \stopsubsection \startsubsection[title={Macros}] \topicindex {macros} \libindex{setmacro} \libindex{getmacro} \libindex{getmeaning} \libindex{setchar} \libindex{setlua} \libindex{getfunctionstable} \libindex{pushmacro} \libindex{popmacro} The \type {set_macro} function can get upto 4 arguments: \starttyping set_macro("csname","content") set_macro("csname","content","global") set_macro("csname") \stoptyping You can pass a catcodetable identifier as first argument: \starttyping set_macro(catcodetable,"csname","content") set_macro(catcodetable,"csname","content","global") set_macro(catcodetable,"csname") \stoptyping The results are like: \starttyping \def\csname{content} \gdef\csname{content} \def\csname{} \stoptyping The \type {getmacro} function can be used to get the content of a macro while the \type {getmeaning} function gives the meaning including the argument specification (as usual in \TEX\ separated by \type {->}). The \type {set_char} function can be used to do a \prm {chardef} at the \LUA\ end, where invalid assignments are silently ignored: \starttyping set_char("csname",number) set_char("csname",number,"global") \stoptyping A special one is the following: \starttyping set_lua("mycode",id) set_lua("mycode",id,"global","protected") \stoptyping This creates a token that refers to a \LUA\ function with an entry in the table that you can access with \type {lua.getfunctions_table}. It is the companion to \prm {luadef}. When the first (and only) argument is true the size will preset to the value of \type {texconfig.function_size}. The \type {pushmacro} and \type {popmacro} function are very experimental and can be used to get and set an existing macro. The push call returns a user data object and the pop takes such a userdata object. These object have no accessors and are to be seen as abstractions. \stopsubsection \startsubsection[title={Pushing back}] \libindex{scannext} \libindex{putnext} There is a (for now) experimental putter: \starttyping local t1 = token.scannext() local t2 = token.scannext() local t3 = token.scannext() local t4 = token.scannext() -- watch out, we flush in sequence token.putnext { t1, t2 } -- but this one gets pushed in front token.putnext ( t3, t4 ) \stoptyping When we scan \type {wxyz!} we get \type {yzwx!} back. The argument is either a table with tokens or a list of tokens. The \type {token.expand} function will trigger expansion but what happens really depends on what you're doing where. This putter is actually a bit more flexible because the following input also works out okay: \startbuffer \def\foo#1{[#1]} \directlua { local list = { 101, 102, 103, token.create("foo"), "{abracadabra}" } token.putnext("(the)") token.putnext(list) token.putnext("(order)") token.putnext(unpack(list)) token.putnext("(is reversed)") } \stopbuffer \typebuffer We get this: \blank {\tt \inlinebuffer} \blank So, strings get converted to individual tokens according to the current catcode regime and numbers become characters also according to this regime. \stopsubsection \startsubsection[title={Nota bene}] When scanning for the next token you need to keep in mind that we're not scanning like \TEX\ does: expanding, changing modes and doing things as it goes. When we scan with \LUA\ we just pick up tokens. Say that we have: \pushmacro\oof \let\oof\undefined \starttyping \oof \stoptyping but \type {\oof} is undefined. Normally \TEX\ will then issue an error message. However, when we have: \starttyping \def\foo{\oof} \stoptyping We get no error, unless we expand \type {\foo} while \type {\oof} is still undefined. What happens is that as soon as \TEX\ sees an undefined macro it will create a hash entry and when later it gets defined that entry will be reused. So, \type {\oof} really exists but can be in an undefined state. \startbuffer[demo] oof : \directlua{tex.print(token.scancsname())}\oof foo : \directlua{tex.print(token.scancsname())}\foo myfirstoof : \directlua{tex.print(token.scancsname())}\myfirstoof \stopbuffer \startlines \getbuffer[demo] \stoplines This was entered as: \typebuffer[demo] The reason that you see \type {oof} reported and not \type {myfirstoof} is that \type {\oof} was already used in a previous paragraph. If we now say: \startbuffer \def\foo{} \stopbuffer \typebuffer \getbuffer we get: \startlines \getbuffer[demo] \stoplines And if we say \startbuffer \def\foo{\oof} \stopbuffer \typebuffer \getbuffer we get: \startlines \getbuffer[demo] \stoplines When scanning from \LUA\ we are not in a mode that defines (undefined) macros at all. There we just get the real primitive undefined macro token. \startbuffer \directlua{local t = token.scannext() tex.print(t.id.." "..t.tok)}\myfirstoof \directlua{local t = token.scannext() tex.print(t.id.." "..t.tok)}\mysecondoof \directlua{local t = token.scannext() tex.print(t.id.." "..t.tok)}\mythirdoof \stopbuffer \startlines \getbuffer \stoplines This was generated with: \typebuffer So, we do get a unique token because after all we need some kind of \LUA\ object that can be used and garbage collected, but it is basically the same one, representing an undefined control sequence. \popmacro\oof \stopsubsection \stopsection \stopchapter \stopcomponent