% language=us runpath=texruns:manuals/followingup \startcomponent followingup-stubs \environment followingup-style \startchapter[title={Stubs}] \startsection[title={Bare bone}] The most barebone way to process a \CONTEXT\ file is something like: \starttyping luametatex --fmt="/luametatex/cont-en" --lua="/luametatex/cont-en.lui" --jobname="article" "cont-yes.mkiv" \stoptyping We pas extra options, like: \starttyping --c:autopdf --c:currentrun=1 --c:fulljobname="./article.tex" --c:input="./article.tex" --c:kindofrun=1 --c:maxnofruns=9 --c:texmfbinpath="c:/data/develop/tex-context/tex/texmf-win64/bin" \stoptyping but for what we are going to discuss here it doesn't really matter. The main point is that we use a \LUA\ startup file. That one has a minimal amount of code so that the format can be loaded as we like it. For instance we need to start up with initial memory settings. The file \type {cont-yes} sets up the way processing content happens. This can be the \type {jobname} file but also something different. It is enough to know that this startup is quite controlled. I will explore a different approach to format loading but for now this is how it goes. After al, we need to be compatible with \LUATEX\ and normal \MKIV\ runs, at least for now. \stopsection \startsection[title={Management (some history)}] In \CONTEXT\ we always had a script: \type {texexec}, originally a \MODULA2 program, later a \PERL\ script, then a \RUBY\ script but now we have \type {mtxrun}, a \LUA\ script. All take care of making sure that the file is processed enough times to get the cross references, tables of contents, indexes, multi|-|pass data stable. It also makes it possible to avoid using these special binaries (or links) that trick the engine into thinking it is bound to a format: we never had \type {pdfcontext} or \type {luacontext}, just one \type {context}. Actually, because we have multiple user interfaces, we would have needed many stubs instead. Getting this approach accepted was not easy but in the meantime I've seen management scripts for other packages being mentioned occasionally. The same is true for scripts: for a long time \CONTEXT\ came with quite some scripts but when an average \TEX\ distribution started growing, including many other scripts, we abandoned this approach and stuck to one management script that also launched auxiliary scripts. That way we could be sure that there were no clashes in names. If you look at a full \TEX\ installation you see many stubs to scripts and more keep coming. How that can work out well without unexpected side effects (name clashes) is not entirely clear to me, as a modern computer can have large bin paths. Just imagine that all large programs (or ecosystems) would introduce hundreds of new \quote {binaries}. Anyway, in the end a \CONTEXT\ installation using \MKIV\ only needs \type {mtxrun} and as bonus \type {context}. The above call is triggered by: \starttyping mtxrun --autogenerate --script context --autopdf article.tex \stoptyping from the editor. Here we create formats when none is found, and start or activate the \PDF\ viewer afterwards, so more minimal is: \starttyping mtxrun --script context article.tex \stoptyping Normally there is also a \type {context} stub so this also works: \starttyping context article.tex \stoptyping \stopsection \startsection[title={The launch process (more history)}] In \MKII, when we use \PDFTEX, the actual launch of these script is somewhat complex and a bit different per platform. But, on all platforms \KPSE\ does the lookup of the script. Already long ago I found out that this startup overhead could amount to seconds on a complete \TEX Live installation (imagine running over a network) which is why eventually we came up with the minimals. The reason is that the file databases have to be loaded: first for looking up, then for the stub that also needs that information and finally by the actual program. There were no \SSD's then. The first hurdle we took was to combine the lookup and the runner. Of course this is sort of out of our control because an installer can decide to still use a lookup approach but at least on \MSWINDOWS\ this was achieved quite easy. Sort of: \starttyping texexex -> [lookup] --> texexec.pl -> [lookup] -> pdftex + formats -> [lookup] -> processing \stoptyping The first lookup can be avoided by some fast relative lookup, but for more complex management the second one is always there. Over time this mechanism became more sophisticated, for instance we use caching, could work over sockets using a \KPSE\ server, etc. When \LUATEX\ came around, it was already decided early that it also would serve as script engine for the \CONTEXT\ runner, this time \type {mtxrun}. The way this works differs per platform. On \WINDOWS\ there is a small binary, say \type {runner.exe}. It gets two copies: \type {mtxrun.exe} and \type {context.exe}. If you find more copies on your system, something might be wrong with your installation. \starttyping mtxrun.exe -> loads mtxrun.lua in same path context.exe -> idem but runs with --script=context \stoptyping The \type {mtxrun.lua} script will load its file database which is very efficient and fast. It will then load the given script and execute it. In the case of \type {context.exe} the \type {mtx-context.lua} script is loaded, which lives in the normal place in the \TEX\ tree (alongside other scripts). So, a minimal amount of programs and scripts is then: \starttyping texmf-win64/bin/luatex.exe texmf-win64/bin/mtxrun.exe texmf-win64/bin/mtxrun.lua texmf-win64/bin/context.exe \stoptyping with (we also need to font manager): \starttyping texmf-context/scripts/context/lua/mtx-context.lua texmf-context/scripts/context/lua/mtx-fonts.lua \stoptyping But \unknown\ there is a catch here: \LUATEX\ has to be started in script mode in order to process \type {mtxrun}. So, in fact we see this in distributions. \starttyping texmf-win64/bin/luatex.exe texmf-win64/bin/texlua.exe texmf-win64/bin/mtxrun.exe texmf-win64/bin/mtxrun.lua texmf-win64/bin/context.exe \stoptyping The \type {texlua} program is just a copy of \type {luatex} that by its name knows that is is supposed to run scripts and not process \TEX\ files. The setup can be different using dynamic libraries (more files but a shared engine part) but the principles are the same. Nowadays the stub doesn't need the \type {texlua.exe} binary any more, so this is the real setup: \starttyping texmf-win64/bin/luatex.exe large program texmf-win64/bin/mtxrun.exe small program texmf-win64/bin/mtxrun.lua large lua file texmf-win64/bin/context.exe small program \stoptyping Just for the record: we cannot really use batch files here because we need to know the original command, and when run from a script that is normally not known. It works to some extend but for instance when started indirectly from an editor it can fail, depending on how that editor is calling programs. Therefore the stub is the most robust method. On a \UNIX\ system the situation differs: \starttyping texmf-linux-64/bin/luatex large program texmf-linux-64/bin/texlua symlink to luatex texmf-linux-64/bin/mtxrun large lua file texmf-linux-64/bin/context shell script that starts mtxrun \stoptyping Here \type {mtxrun.lua} is renamed to \type {mtxrun} with a shebang line that triggers loading by \type {texlua} which is a symlink to \type {luatex} because shebang lines don't support the \type {--texlua} argument. As on windows, this is not really pretty. \stopsection \startsection[title={The \LMTX\ way (the present)}] Now when we move to \LMTX\ we need to make sure that the method that we choose is acceptable for distributions but also nicely consistent over platforms. We only have one binary \type {luametatex} with all messy logic removed and no second face like \type {metaluatex}. When it is copied to another instance (or linked) it will load the script with its own name when it finds one. So on \WINDOWS\ we now have: \starttyping texmf-win64/bin/luametatex.exe medium program texmf-win64/bin/mtxrun.exe copy (or link) of luametatex texmf-win64/bin/mtxrun.lua large lua file texmf-win64/bin/context.exe copy (or link) of luametatex texmf-win64/bin/context.lua small lua file \stoptyping and in \UNIX: \starttyping texmf-linux-64/bin/luametatex mediumprogram texmf-linux-64/bin/mtxrun copy (or link) of luametatex texmf-linux-64/bin/mtxrun.lua large lua file texmf-linux-64/bin/context copy (or link) of luametatex texmf-linux-64/bin/context.lua small lua file \stoptyping So, \type {luametatex[.exe]}, \type {mtxrun[.exe]} and \type {context[.exe]} are all the same. On both platforms there is \type {mtxrun.lua} (with suffix) and on both we also use the same runner approach. The \type {context.lua} script is really small and just sets the script command line argument before loading \type {mtxrun.lua} from the same path. In the case of copied binaries: keep in mind that the three copies together are not (much) larger than the \type {luatex} and \type {texlua} pair (especially when you take additional libraries into account). The disadvantage of using copies is that one can forget to copy with an update, but the fact that one can use them might be easier for installers. It's up to those who create the installers. One complication is that the \type {mtxrun.lua} script has to deal with the old and the new setup. But, when we release we will assume that one used either \LUATEX\ or \LUAMETATEX, not some mix. As \type {mtxrun} and \type {context} know what got it started they will then trigger the right engine, unless one passes \typ {--engine=luatex}. In that case the \LUAMETATEX\ launcher will trigger a \LUATEX\ run. But a mixed installation is unlikely to happen. \stopsection \startsection[title={Why not \unknown}] Technically we could use one call for both the runner and \TEX\ processor but when multiple runs are needed this would demand an internal engine reset as well as macro package reset while keeping some (multi|-|pass) data around. A way in|-|between could be to spawn the next run. In the end the gain would be minimal (we have now .2 seconds overhead per total run, which can trigger multiple passes, due to the management script, to basically we can neglect it. (Triggering the viewer takes more time.) \stopsection \stopchapter \stopcomponent