Source Browser ?

1% language=us runpath=texruns:manuals/followingup
2
3\startcomponent followingup-mp
4
5\environment followingup-style
6
7\startchapter[title={\METAPOST}]
8
9\startsection[title={Introduction}]
10
11Relatively late in the followup I started wondering about what to do with \MPLIB.
12Alan Braslau is working on the \type {luapost} module and we discuss handy
13extensions written in \LUA\ and \METAPOST\ code but who knows what more is
14needed. Some ideas were put on delay but it looked like a good moment to pick up
15on them. One problem is that when we play with the \MPLIB\ code itself in
16\LUAMETATEX, the question is how to keep in sync with the official library. In
17this chapter I'll discuss both: keeping up with the official code, and keeping
18ahead with ideas.
19
20\stopsection
21
22\startsection[title={The code base}]
23
24The \MPLIB\ code is written in \CWEB\ and lives in files with the suffix \type
25{w}. These files need to be converted to \type {c} and \type {h} files, something
26that is done with the \type {ctangle} program. To avoid that dependency I just
27took the \CCODE\ files from \LUATEX, but I had to apply a few patches (to get rid
28of dependencies). Now, it is a fact that \METAPOST\ doesn't really develop fast and
29in principle a diff could identify the changes easily. So, why shouldn't I also
30start experimenting with \MPLIB\ itself in the follow up? It's easy to merge
31future changes (in both directions).
32
33The first thing I wrote was a \type {w-to-c} script. This was not that hard given
34that I already had written lexers. After a first prototype worked out well, I
35redid the code a bit (so that in the future I can also implement support for
36change files for instance). A complication was that I found out that the regular
37\CWEB\ converter messes around a bit with the code. So, I had to write another
38script to mimmick that to the level that I could compare the results. For
39example, spaces are removed before and after operators and all leading space gets
40removed too. When I got the same output I could get rid of that code and output
41what I want. For instance I'd like to keep the spacing the same because compilers
42can warn about some issues, like missing \type {;} and misleading indentation in
43simple \type {if} and \type {while} constructs where braces are omitted.
44\footnote {This is no problem in for instance \PASCAL\ where we always have a
45\type {begin} and \type {end}.} One can argue that this is not important, but if
46not, then why enable warnings at all. I had to fix half a dozen places in the
47\type {w} file to make the compiler happy, so the price was small.
48
49Once I had a more or less instantaneous conversion \footnote {Conversion of the
50\type {w} files involved took just over half a second at that time, currently it
51takes just over a quarter of a second, on a relatively old machine that is.} I
52got the same feeling as with the rest of the code: experimenting became
53convenient due to the fast edit|-|compile cycle. So, with al this covered I could
54do what I always had wanted to do: remove traces of the backends (including the
55full \POSTSCRIPT\ one), because they are actually to be plug|-|ins, and also get
56rid of internal font handling, which is bound to \TYPEONE\ (rendering) and small
57size \TFM\ (generating). With respect to that export: I wonder if anyone used
58that these days because even the Gust font project always had their own tool
59chain alongside \METAPOST. I could also void the hacks needed to trick the
60library in not being dependent of \type {png.h} and \type {zlib.h} headers, for
61which I had to use dummies. \footnote {The converter can load a file with patches
62to be applied but by now there are no patches.}
63
64It took a few days scripting the converter (most time went into getting identical
65output in order to check the converter which was later dropped), a few days
66stripping unused code, another day cleaning up the remaining code and then I
67could start playing with some new extensions. The binary has shrunk with 200KB
68and the whole \LUAMETATEX\ code base in compressed \type {tar.xz} format is now
69below 1.8MB while before it was above 2MB. Not that it matters much, but it was
70an nice side effect. \footnote {Size matters as we want to code to end up in the
71\CONTEXT\ distribution. It might grow a bit as side effect of adding some more
72features to \MPLIB.}
73
74What new extensions would show up was still open. Because Alan and I play with
75scanners it made sense to look into that. Error handling and logging has also
76been on my radar for a while. In the process some more code might be dropped, but
77actually the current version is still useable as library for a stand alone
78program, given that one reconstructs the \POSTSCRIPT\ driver from the dropped
79code (not that much work). Some configuration options are missing then but that
80could be provided as extensions (after all we can have change files.) On the
81other hand, wrapping code in \CONTEXT, like:
82
83\starttyping
84\starttext
85\startMPpage
86    ........
87\stopMPpage
88\startMPpage
89    ........
90\stopMPpage
91\stoptext
92\stoptyping
93
94will give a \PDF\ file that can be converted to all kinds of formats, and the
95advantage is that one has full font support. There is already a script in the
96distribution that does this anyway.
97
98\stopsection
99
100\startsection[title={Communication}]
101
102The first experiment concerns a change in the interfacing between the \METAPOST\
103and \LUA\ end. In the original library all file \IO\ is handled by the library
104itself. The filenames can be resolved via a callback. Once an instance is
105initialized, snippets of code are passed to the instance via the \type {execute}
106call. Log, terminal and error information is collected and returned as part of
107the return value (a table). This means that reporting back to the user has a
108delay: it can be shown {\em after} all code in the buffer has been processed. The
109code given as argument to \type {execute} is passed to the engine as (fake)
110terminal input, which nicely fits in the concept of interactive input, which
111already is part of the \METAPOST\ concept.
112
113In our follow up variant all file \IO\ goes via \LUA. This means that we have a
114bit more control over matters. In \CONTEXT\ we now can use the usual file
115handling code. One defines an \type {open_file} callback that returns a table
116with possible methods \type {close}, \type {reader} and \type {writer}, as in
117similar \LUATEX\ callbacks. A special file, with the name \type {terminal} is
118used for terminal communication. Now, when the \type {execute} command is
119handled, the string that gets passed ends up in the terminal, so the file handler
120has to deal with it: the string gets written to the handle, and the handle has to
121return it as lines on request. In \CONTEXT\ we directly feed the to be executed
122code into the terminal cache.
123
124It's all experimental and subject to changes but as we keep \CONTEXT\ \LMTX\ and
125\LUAMETATEX\ in sync, this is no problem. Users will not use these low level
126interfaces directly. It might take a few years to settle on this.
127
128The reports that come from the \METAPOST\ engine are now passed on to the \type
129{run_logger} callback. That one gets a target and a string passed. Where the
130original library can output stuff twice, once for the log and once for the
131console, in the new situation it gets output once, with the target being
132terminal, log file or both. The nice thing about this callback is that there is no
133delay: the messages come as the code is processed.
134
135We combine this logging with the new \type {halt_on_error} flag, which makes the
136engine abort after one error. This mechanism will be improved as we go. The
137interaction option \type {silent} hides some of the less useful messages.
138
139The overall efficiency of the library doesn't suffer from these changes, and in
140some cases it can perform even better. Anyhow, the user experience is much better
141with synchronous reports.
142
143Although not strictly related to \IO, we already has extended the library with
144the option to support \UTF-8, which is handy for special symbols, as for instance
145used in the \type {luapost} library.
146
147\stopsection
148
149\startsection[title={Scanning}]
150
151Another extension is more fundamental in the sense that it can affect the way
152users see \METAFUN: extending the user interface. It is again an example of why
153is having an independent code base has benefits: we can do such experiments for a
154long time, before we decide that (and how) it can end up in the parent (of course
155the same is true for the mentioned \IO\ features). I will not discuss these
156features here. For now it is enough to know that it gets applied in \CONTEXT\ and
157will provide a convenient additional interface. Once it is stable I'll wrap it up
158in writing.
159
160\stopsection
161
162\stopchapter
163
164\stopcomponent
165