1
2
3
4
5
6\environment mkenvironment
7
8\startcomponent mkpunk
9
10\page[right] \start
11
12
13
14
15
16
17
18
19\usemodule[m][punk]
20\usetypescript[punk]
21\switchtobodyfont[punk,12pt]
22
23\StartRandomPunk
24
25\definesymbol[1][]
26\setupsorting[logo][style=]
27\setupcapitals[title=no]
28\setuptype[style=\tf]
29\setuptyping[style=\tf]
30\logo[METAPOST] {MetaPost}
31\logo[METAFONT] {MetaFont}
32
33\chapter{How to convince Don and Hermann to use \LUATEX}
34
35{\em The code shown here should look a bit different in versions
36of \MKIV\ after March 2011. This is because the font system was
37cleaned up and upgraded. The prinicples remain the same. You can
38have a look at \type {mpunk.mkiv} in the \CONTEXT\ distribution.}
39
40Odds are pretty low that Don Knuth will use \LUATEX\ for
41typesetting the next update of his opus magnum, and odds are even
42lower that Hermann Zapf will use \MPLIB\ for Melior Nova. However,
43the next example of combining \METAFONT\ and \TEX\ may draw their
44interest in this new variant: \METATEX.
45
46The font used here is called \quote {punk} and is designed by
47Donald Knuth. There is a note in the file that says: \quotation
48{Font inspired by Gerard and Marjan Ungers lectures, February
491985}. If you didnt notice it yet: punk is a random font.
50
51You may wonder why we started looking into this masterpiece of
52font design. Well, there are a few reasons:
53
54\startitemize
55
56\item We always liked this font, but after the rise of outline
57 fonts it was not a natural candidate for using in
58 documents. Fun is always a good motive.
59
60\item For many years we have been suggesting that special glyphs
61 andor aspects of typesetting could be realized by runtime
62 generation of graphics, and we need this testbed for the
63 Oriental \TEX\ project: Idris needs stretchable interglyph
64 connections.
65
66\item Taco likes using tricky \METAPOST\ backgrounds for his
67 presentations that demonstrate this programming language.
68
69\item Hartmut loves to tweak the backend and runtime font generation
70 will demand some extensions to the font inclusion and literal
71 handlers.
72
73\item Because Hans attends many \TEX\ conferences together with Volker
74 Schaa, he has promised him to avoid repeating talk and
75 presentation layouts, and so a new presentation style was needed.
76
77\stopitemize
78
79To this we can add an already mentioned motivation: convince Don and
80Hermann to use \LUATEX\ \unknown\ who knows. And, if that fails, maybe
81they can team up for an extensions to this font: more style variants,
82proper math and the full range of \UNICODE\ glyphs.
83
84The punk font is written in \METAFONT\ and there are multiple
85sources. These are merged into one file which is to be processed
86using the \type {mfplain} format. Definitions of characters in
87this font look like:
88
89\starttyping
90beginpunkchar ("A",13,1,2) ;
91 z1 = pp(1.5u,0) ; z2 = (.5w,1.1h) ; z3 = pp(w1.5u,0) ;
92 pd z1 ; pd z3 ; draw z1 z2 z3 ;
93 z4 = pp .3[z1,z2] ; z5 = pp .3[z3,z2] ;
94 pd z4 ; pd z5 ; draw z4 z5;
95endchar ;
96\stoptyping
97
98When \TEX\ needs a font, i.e.\ when we have something like this:
99
100\starttyping
101\font\somefont=whatever at 12pt
102\stoptyping
103
104in \CONTEXT\ control is delegated to a font loader written in
105\LUA\ that is hooked into \TEX. This loader interprets the name
106and if needed filters the specification from it. Think of this:
107
108\starttyping
109\font\somefont=whatever*smallcaps at 16pt
110\stoptyping
111
112This means: load font \type {whatever} and enable the smallcaps features.
113However this mechanism is mostly geared towards \TYPEONE\ and \OPENTYPE\
114fonts. But punk is neither: its a \METAFONT, and we need to treat it as
115such. We will use \LUATEXs powerful virtual font technology
116because that way we can smuggle the proper shapes in the final
117file. And \unknown\ no bitmaps and no funny encoding.
118
119In \CONTEXT\ \MKIV\ there is a preliminary virtual font definition
120mechanism. There is no advanced \TEX\ interface yet so we need to do it in
121\LUA. Fortunately we do have access to this from the font mechanism:
122
123\starttyping
124\font\somefont=mypunk@punk at 20pt
125\stoptyping
126
127This is a rather valid directive to create a font that internally
128will be called \type {mypunk}. For this the virtual font creation
129command \type {punk} will be used, and in a moment we will see what
130this triggers.
131
132Of course, users will never see such low level definitions. They will
133use proper typescript, which set up a whole font system. For instance,
134in this document we use:
135
136\typebuffer[fontdefinition]
137
138Now, using punk in inself is not that much of a challenge, but how about
139using multiple instances of this font and then typeset the text chosing
140variants of a glyph at random. Of course this will have some tradeoff in
141terms of runtime. In this document we use punk as the bodyfont and
142therefore it comes in several sizes. On Hanss laptop generating the
143glyphs takes a while:
144
145\starttyping
1467500 glyphs, 12.887 seconds runtime, 581 glyphssecond
147\stoptyping
148
149Fortunately \MKIV\ provides a caching mechanism so once the fonts
150are generated, a next run will be more comfortable. This time we
151get reported:
152
153\starttyping
1540.187 seconds, 60 instances, 320.856 instancessecond
155\stoptyping
156
157which is not that bad for loading 60 files of 5 megabytes \PDF\
158literals each. The reason why the files are large is that although
159these glyphs look simple, in fact they are rather complex: each
160glyph at least one paths and several knots, and since a special
161pen is used, conversion results in a larger than normal description
162of a shape.
163
164Since we use the standard converter from \METAPOST\ to \PDF, we
165can gain some generation time by using a dedicated converter for
166glyphs. Eventually the \MPLIB\ library may even provide a proper
167charstring generator so that we can construct real fonts at
168runtime.
169
170So, how does this work behind the screens? Because we can use some
171of the mechanisms already present in \CONTEXT\ it is not even that complex.
172
173\startitemize
174
175\item The \type {punk} directive tells \CONTEXT\ to create a virtual
176 font. Such a font can be made out of real fonts; we use this
177 for instance in the font feature \type {combine}, where we
178 add virtually composed characters that are missing by combining
179 characters present. However, here we have no real font.
180
181\item And so this virtual font is not build on top of an existing font, but
182 spawns a \MPLIB\ process that will build the font, unless it is
183 present in the cache on disk. The shapes are converted to \PDF\ literals
184 and for each character a proper definition table is made.
185
186\item In total 10 such fonts are made, but only one is returned to the
187 font callback that asked us to provide the font. The list of
188 the alternatives is stored in the \LUA\ table that represents
189 the font and kept at the \LUA\ end. So, for each size used,
190 a unique set of 10 variants is generated.
191
192\item The randomizer operates on the node list. Instead of using a
193 dedicated mechanism for this, we hijack one of the attribute values
194 of the case swapper already present in \MKIV. After that we can selectively
195 turn on and off the randomizer.
196
197\item At some point \TEX\ will hand over the node lists to \CONTEXT. At
198 that moment a lot of things can happen to the list, and one of
199 them is a sequence of character handlers, of which the mentioned case
200 handler is one. The handler sweeps over the nodelist
201 and for each glyph node triggers a function that is bound to the
202 attribute value.
203
204\item This function is rather trivial: it looks at the font id of the
205 glyph, and resolves it to the font table. If that table has a
206 list of alternatives, it will randomly choose one and assign it to
207 the font attribute of the glyph. Thats all.
208
209\item Eventually the backend routines will inject the \PDF\ literals that
210 were collected in the commands table of the virtual glyph.
211
212\stopitemize
213
214It will not come as a surprise that our resulting file is larger
215than what we get when using traditional outline fonts or just one
216instance of punk. However, this is just an experiment, and
217eventually a proper font constructor will be provided, so that the
218glyph drawing is delegated to the font renderer. An intermediate
219optimization can be to use so called \PDF\ xforms, but a properly
220runtime generated font is best because then we can search in the
221file too.
222
223Because by now reading the punk font should go fluently we can now
224move on to the code. We already have a \type {fonts} namespace,
225which we now extend with an \METAPOST\ sub namespace:
226
227\starttyping
228fonts.mp = fonts.mp or { }
229\stoptyping
230
231We set a version number and define a cache on disk. When the number changes
232fonts stored in the cache will be regenerated when needed. The
233\type {containers} module provides the relevant function.
234
235\starttyping
236fonts.mp.version = 1.01
237fonts.mp.cache = containers.define("fonts", "mp", fonts.mp.version, true)
238\stoptyping
239
240We already have a \type {metapost} namespace, and within it we define a
241sub namespace:
242
243\starttyping
244metapost.characters = metapost.characters or { }
245\stoptyping
246
247Now were ready for the real action: we define a dedicated flusher
248that will be passed to the \METAPOST\ converter. A next version of
249\MPLIB\ will provide the \TFM\ font information which gives better
250glyph dimensions, plus additional kerning information. All this code
251is defined in a closure (\type {do ... end}) which
252nicely hides the local variables.
253
254\starttyping
255local characters, descriptions = { }, { }
256local factor, total, variants = 100, 0, 0
257local l, n, w, h, d = { }, 0, 0, 0, 0
258
259local flusher = {
260 startfigure = function(chrnum,llx,lly,urx,ury)
261 l, n = { }, chrnum
262 w, h, d = urx llx, ury, lly
263 total = total 1
264 end,
265 flushfigure = function(t)
266 for i=1, #t do
267 l[#l1] = t[i]
268 end
269 end,
270 stopfigure = function()
271 local cd = characters.data[n]
272 descriptions[n] = {
273 unicode = n,
274 name = cd and cd.adobename,
275 width = w*100,
276 height = h*100,
277 depth = d*100,
278 }
279 characters[i] = {
280 commands = {
281 { "special", "pdf: " .. table.concat(l," ") },
282 }
283 }
284 end
285}
286\stoptyping
287
288In the normal converter, the start and stop function do the
289packaging in a box. The flush function is called when literals
290need to be flushed. This threesome does as much as collecting
291glyph information in the \type {list} table. Intermediate literals
292are stored in the \type {l} table. Each glyph has a description and
293(in this case) one command that defines the virtual shape. The name
294is picked up from the character data table that is present in \MKIV.
295
296As told before we generate multiple instances per requested font
297and here is how it happens. We initialize the \type {mfplain}
298format and reset it afterwards. The punk definition file is
299adapted for multiple runs. Scaling happens here because later on
300the scaler has no knowledge about what is present in the commands.
301We use a few helpers for processing the \METAPOST\ code and format
302the final font table in a way \CONTEXT\ \MKIV\ likes. Currently
303the parameters (font dimensions) are rather hard coded, but this
304will change when \MPLIB\ can provide them.
305
306\starttyping
307function metapost.characters.process(mpxformat, name, instances, scalefactor)
308 statistics.starttiming(metapost.characters)
309 scalefactor = scalefactor or 1
310 instances = instances or 10
311 local fontname = file.removesuffix(file.basename(name))
312 local hash = file.robustname(string.format(
313 "
314 local lists = containers.read(fonts.mp.cache, hash)
315 if not lists then
316 statistics.starttiming(flusher)
317 local data = io.loaddata(resolvers.findfile(name))
318 metapost.reset(mpxformat)
319 lists = { }
320 for i=1,instances do
321 characters, descriptions = { }
322 metapost.process(
323 mpxformat,
324 {
325 "randomseed := " .. i*10 .. ";",
326 "scalefactor := " .. scalefactor .. " ;",
327 data
328 },
329 false,
330 flusher
331 )
332 lists[#lists1] = {
333 designsize = 655360,
334 name = string.format("
335 parameters = {
336 slant = 0,
337 space = 333 * scalefactor,
338 spacestretch = 166.5 * scalefactor,
339 spaceshrink = 111 * scalefactor,
340 xheight = 431 * scalefactor,
341 quad = 1000 * scalefactor,
342 extraspace = 0
343 },
344 ["type"] = "virtual",
345 characters = characters,
346 descriptions = descriptions,
347 }
348 end
349 metapost.reset(mpxformat) saves memory
350 lists = containers.write(fonts.mp.cache, hash, lists)
351 statistics.stoptiming(flusher)
352 end
353 variants = variants #lists
354 statistics.stoptiming(metapost.characters)
355 return lists
356end
357\stoptyping
358
359Were not yet there. This was just a font generator that returns
360a list of fonts defined in a format liked by \MKIV\ and not that
361far from what \TEX\ wants back from us. Next we define the
362main definition function, the one that is called when the font
363is defined as virtual font. The special number \type {1000}
364tells the scaler to honour the designsize, which boils down to
365no scaling, but just copying to the final table that is passed
366to \TEX. The \type {define} function returns an id which we will
367use later.
368
369The scaler uses the \type {descriptions} to add dimensions (and other data
370needed) in the \type {characters} table. This is something \MKIV\ specific.
371
372\starttyping
373function fonts.handlers.vf.combiner.commands.metafont(g,v)
374 local size = g.specification.size
375 local data = metapost.characters.process(v[2],v[3],v[4],size655360)
376 local list, t = { }, { }
377 for d=1,#data do
378 t = data[d]
379 t = fonts.constructors.scale(t, 1000)
380 t.id = font.define(t)
381 list[#list1] = t.id
382 end
383 for k, v in pairs(t) do
384 g[k] = v kind of replace, when not present, make nil
385 end
386 g.variants = list
387end
388\stoptyping
389
390We hook this into the \CONTEXT\ font handler and from now on
391the \type {@punk} is recognized:
392
393\starttyping
394fonts.definers.methods.install( "punk", { { "metafont", "mfplain", "punkfont.mp", 10 } } )
395\stoptyping
396
397Now that we can define the font, we need to deal with
398the randomizer. This is optional fun. The mentioned case swappers
399are implemented in the \type {cases} namespace:
400
401\starttyping
402local fontdata = fonts.hashes.identifiers
403
404cases.actions[99] = function(current)
405 local c = current.char
406 local used = fontdata[current.font].variants
407 if used then
408 local f = math.random(1,#used)
409 current.font = used[f]
410 return current, true
411 else
412 return current, false
413 end
414end
415\stoptyping
416
417This function is called in one of the passes over the node
418list. Thanks to this framework we dont need that much code.
419We didnt show two statistics functions. They are the reason why
420we keep track of the total number of glyphs defined.
421
422This leaves us defining the interface, so here we go:
423
424\starttyping
425\def\StartRandomPunk{\begingroup\setcharactercasing[99]}
426\def\StopRandomPunk {\endgroup}
427\stoptyping
428
429The set command just sets the attribute that we associated
430with casing (one of the many attributes). The number 99 is
431rather arbitrary.
432
433If you follow the development of \LUATEX\ and \MKIV\ (we do talks at
434conferences, keep track of the development history in \type {mk.pdf},
435and report on the \CONTEXT\ mailing list) you will have noticed that
436we often use somewhat extreme examples to explore and test the
437functionality and this is no exception. As usual it helped us to improve
438the code and extend our todo list. Can the previous code convince
439the grand wizards to start using \LUATEX ? Probably not. Lets
440anyway hope that they will put the addition of punk math to their todo
441list. In the meantime weve already started adding missing characters:
442
443\startlinecorrection[blank]
444 \hbox to \hsize \bgroup \hss
445 \dorecurse{6}{\hbox{\char123\enspace\char39\enspace\char92\enspace\char34\enspace\char125}\quad}\unskip
446 \hss \egroup
447\stoplinecorrection
448
449Also, because we can be sure that Mojca Miklavecs first test will
450be if her favourite characters \color [mkcolor] {\ccaron}, \color
451[mkcolor] {\scaron} and \color [mkcolor] {\zcaron} are supported,
452we made sure that we composed those accented characters as well.
453\footnote {This is accomplished by adding \type
454{composecharacters(t)} at an undisclosed location in
455the previous code.}
456
457\StopRandomPunk \page[right] \stop
458
459\stopcomponent
460 |