Source Browser ?

1% language=us
2
3% todo: callback: stream, llx, lly, urx, ury, wd, lsb
4% add glyphs runtime
5% create whole cff so that we can go from mp
6
7\startcomponent onandon-variable
8
9\environment onandon-environment
10
11\startchapter[title=Variable fonts]
12
13\startsubject[title=Introduction]
14
15History shows the tendency to recycle ideas. Often, quite some effort is made by
16historians to figure out what really happened, not just long ago, when nothing
17was written down and we have to do with stories or pictures at most, but also in
18recent times. Descriptions can be conflicting, puzzling, incomplete, partially
19lost, biased, \unknown
20
21Just as language was invented (or evolved) several times, so were scripts. The
22same might be true for rendering scripts on a medium. Semaphores came and went
23within decades, and how many people know now that they existed and that encryption
24was involved? Are the old printing presses truly the old ones, or are older
25examples simply gone? One of the nice aspects of the internet is that one can now
26more easily discover similar solutions to the same problem but with a different
27(and independent) origin.
28
29So, how about this \quotation {next big thing} in font technology: variable fonts?
30In this case, history shows that it's not that new. For most \TEX\ users, the
31names \METAFONT\ and \METAPOST\ will ring bells. They have a very well-documented
32history, so there is not much left to speculation. There are articles, books,
33pictures, examples, sources, and more around for decades. So, the ability to
34change the appearance of a glyph in a font depending on some parameters is not
35new. What probably {\em is} new is that creating variable fonts is done in the
36natural environment, where fonts are designed: an interactive program. The
37\METAFONT\ toolkit demands quite some insight into programming shapes in such a way
38that one can change look and feel depending on parameters. There are not that
39many metafonts made, and one reason is that making them requires a certain mind-
40and skillset. On the other hand, faster computers, interactive programs,
41evolving web technologies, where real-time rendering and therefore more or less
42real-time tweaking of fonts is a realistic option, all play a role in acceptance.
43
44But do interactive font design programs make this easier? You still need to
45translate ideas into usable beautiful fonts. Taking the common shapes of glyphs,
46defining extremes and letting a program calculate some interpolations will not
47always give good results. It's like morphing a picture of your baby's face into
48yours of old age or that of your grandparent: not all intermediate results will
49look great. It's good to notice that variable fonts are a revival of existing
50techniques and ideas used in, for instance, multiple master fonts. The details
51might matter even more as they can now be exaggerated when transformations are
52applied.
53
54There is currently (March 2017) not much information about these fonts, so what I
55say next may be partially wrong or at least different from what is intended. The
56perspective will be one of a \TEX\ user and coder. Whatever you think of them,
57these fonts will be out there, and, for sure, there will be nice examples
58circulating soon. And so, when I ran into a few experimental fonts with
59\POSTSCRIPT\ and \TRUETYPE\ outlines, I decided to have a look at what is inside.
60After all, because it's visual, it's also fun to play with. Let's stress that, at
61the moment of this writing, I only have a few simple fonts available, fonts that
62are designed for testing and not for usage. Some recommended tables were missing
63and no complex \OPENTYPE\ features were used in these fonts.
64
65\stopsubject
66
67\startsubject[title=The specification]
68
69I'm not that good at reading specifications, first of all because I quickly fall
70asleep with such documents, but mostly because I prefer reading other stuff
71(I do have lots of books waiting to be read). I'm also someone who has to play
72with something in order to understand it: trial and error is my modus operandi.
73Eventually, it's my intended usage that drives the interface and that is when
74everything comes together.
75
76Exploring this technology comes down to: locate a font, get the \OPENTYPE\ 1.8
77specification from the \MICROSOFT\ website, and try to figure out what is in the
78font. When I had a rough idea, the next step was to get to the shapes and see if
79I could manipulate them. Of course, it helped that we can already load fonts and
80play with shapes in \CONTEXT using \METAPOST. I didn't have to install and learn
81other programs. Once I could render them, in this case by creating a virtual font
82with inline \PDF\ literals, a next step was to apply variation. Then came the
83first experiments with a possible user interface. Seeing more variation then
84drove the exploration of additional properties needed for typesetting, like
85features.
86
87The main extension to the data packaged in a font file concerns the (to be
88discussed) axis along which variable fonts operate and deltas to be applied to
89coordinates. The \type {gdef} table has been extended and contains information
90that is used in \type {gpos} features. There are new \type {hvar}, \type {vvar}
91and \type {mvar} tables that influence the horizontal, vertical, and general font
92dimensions. The \type {gvar} table is used for \TRUETYPE\ variants, while the
93\type {cff2} table replaces the \type {cff} table for \OPENTYPE\ \POSTSCRIPT\
94outlines. The \type {avar} and \type {stat} tables contain some
95metainformation about the axes of variations.
96
97It must be said that, because this is a new technology, the information in the
98standard is not always easy to understand. The fact that we have two rendering
99techniques, \POSTSCRIPT\ \type {cff} and \TRUETYPE\ \type {ttf}, also means that
100we have different information and perspectives. But this situation is not much
101different from \OPENTYPE\ standards a few years ago: it takes time, but, in the
102end, I will get there. And, after all, users also complain about the lack of
103documentation for \CONTEXT, so who am I to complain? In fact, it will be those
104\CONTEXT\ users who will provide feedback and make the implementation better in
105the~end.
106
107\stopsubject
108
109\startsubject[title=Loading]
110
111Before we discuss some details, it will be useful to summarize what the font
112loader does when a user requests a font at a certain size and with specific
113features enabled. When a font is used for the first time, its binary format is
114converted into a form that makes it suitable for use in \CONTEXT\ and therefore
115in \LUATEX. This conversion involves collecting the properties of the font as a
116whole (official names, general dimensions like x-height and em-width, etc.), of
117glyphs (dimensions, \UNICODE\ properties, optional math properties), and all
118kinds of information that relate to (contextual) replacements of glyphs (small
119caps, oldstyle, scripts like Arabic) and positioning (kerning, anchoring marks,
120etc.). In the \CONTEXT\ font loader, this conversion is done in \LUA.
121
122The result is stored in a condensed format in a cache, and, the next time the font
123is needed, it loads in an instant. In the cached version, the dimensions are
124untouched, so a font at different sizes has just one copy in the cache. Often, a
125font is needed at several sizes, and for each size, we create a copy with scaled
126glyph dimensions. The feature-related dimensions (kerning, anchoring, etc.)\ are
127shared and scaled when needed. This happens when sequences of characters in the
128node list get converted into sequences of glyphs. We could do the same with glyph
129dimensions, but one reason for having a scaled copy is that this copy can also
130contain virtual glyphs, and these have to be scaled beforehand. In practice, there
131are several layers of caching in order to keep the memory footprint within
132reasonable bounds. \footnote {In retrospect, one can wonder if that makes sense;
133just look at how much memory a browser uses when it has been open for some time.
134In the beginning of \LUATEX, users wondered about caching fonts, but again, just
135look at what amounts browsers cache: it gets pretty close to the average amount
136of writes that a \SSD\ can handle per day within its guarantee.}
137
138When the font is actually used, interaction between characters is resolved using
139the feature|-|related information. When, for instance, two characters need to be
140kerned, a lookup results in the injection of a kern, scaled from general
141dimensions to the current size of the font.
142
143When the outlines of glyphs are needed in \METAFUN, the font is also converted
144from its binary form to something in \LUA, but this time we filter the shapes.
145For a \type {cff}, this comes down to interpreting the \type {charstrings} and
146reducing the complexity to \type {moveto}, \type {lineto}, and \type {curveto}
147operators. In the process, subroutines are inlined. The result is something that
148\METAPOST\ is happy with but that also can be turned into a piece of a \PDF.
149
150We now come to what a variable font actually is: a basic design which is
151transformed along one or more axes. A simple example is wider shapes:
152
153\startlinecorrection
154\startMPcode
155for i=1 upto 4 :
156    fill fullsquare xyscaled (i*.5cm,1cm) shifted (i*2.5*cm,0) withcolor "darkgray";
157    fill fullcircle xyscaled (2.5mm,2.5mm) shifted (i*2.5*cm,0) withcolor "lightgray" ;
158endfor ;
159\stopMPcode
160\stoplinecorrection
161
162We can also go taller and retain the width:
163
164\startlinecorrection
165\startMPcode
166for i=1 upto 4 :
167    fill fullsquare xyscaled (1cm,i*.5cm) shifted (i*2.5*cm,0) withcolor "darkgray";
168    fill fullcircle xyscaled (2.5mm,2.5mm) shifted (i*2.5*cm,0) withcolor "lightgray" ;
169endfor ;
170\stopMPcode
171\stoplinecorrection
172
173Here we have linear scaling, but glyphs are not normally done that way. There
174are font collections out there with lots of intermediate variants (say from light
175to heavy) and it's more profitable to sell each variant independently. However,
176there is often some logic behind it, probably supported by programs that
177designers use, so why not build that logic into the font and have one file that
178represents many intermediate forms. In fact, once we have multiple axes, even
179when the designer has clear ideas of the intended usage, nothing will prevent
180users from tinkering with the axis properties in ways that will fulfil their
181demands (and hurt the designer's eyes). I will not discuss that dilemma here.
182
183When a variable font follows the route described above, we face a problem. When
184you load a \TRUETYPE\ font, it will just work. The glyphs are packaged in the
185same format as static fonts. However, a variable font has axes, and, on each
186axis, a value can be set. Each axis has a minimum, a maximum, and a default. It
187can be that the default instance also assumes some transformations are applied.
188The standard recommends adding tables to describe these things, but the fonts
189that I played with each lacked such tables. So that leaves some guesswork. But
190still, just loading a \TRUETYPE\ font gives some sort of outcome, although the
191dimensions (widths) might be weird due to the lack of a (default) axis being
192applied.
193
194An \OPENTYPE\ font with \POSTSCRIPT\ outlines is different: the internal \type
195{cff} format has been upgraded to \type {cff2}, which on the one hand is less
196complicated, but on the other hand has a few new operators, which results
197in programs that have not been adapted complaining or simply crashing on them.
198
199One could argue that a font is just a resource and that one only has to pass it
200along, but that's not what works well in practice. Take \LUATEX. We can, of
201course, load the font and apply axis vales, so that we can process the document
202as we normally do. But, at some point, we have to create a \PDF\ file. We can
203simply embed the \TRUETYPE\ files, but no axis values are applied. This is
204because, even if we add the relevant information, there is no way in the current
205\PDF\ formats to deal with it. For that, we should be able to pass all relevant
206axis|-|related information as well as specify what values to use along these
207axes. And, for \TRUETYPE\ fonts, this information is not part of the shape
208description so then we need to filter and pass more. An \OPENTYPE\ \POSTSCRIPT\
209font is much cleaner, because there we have the information needed to transform
210the shape mostly in the glyph description. There, we only need to carry some
211extra information on how to apply these so|-|called blend values. The
212region|/|axis model used there only demands passing a relatively simple table
213(stripped down to what we need). But, as said above, \type {cff2} is not
214backward-compatible, so a viewer will (currently) simply not show anything.
215
216Recalling how we load fonts, how does that change with variable changes? If we
217have two characters with glyphs that get transformed and that have a kern between
218them, the kern may or may not transform. So, when we choose values on an axis,
219then not only glyph properties change but also relations. We can no longer share
220positional information and scale afterwards, because each instance can have
221different values to start with. We could carry all that information around and
222apply it at runtime, but, because we're typesetting documents with a static design,
223it's more convenient to just apply it once and create an instance. We can use the
224same caching as mentioned before, but each chosen instance (provided by the font
225or made up by user specifications) is kept in the cache. As a consequence, using
226a variable font has no overhead, apart from initial caching.
227
228So, having dealt with that, how do we proceed? Processing a font is not different
229from what we already had. However, I would not be surprised if users are not
230always satisfied with, for instance, kerning, because in such fonts, a lot of care
231has to be given to this by the designer. Of course, I can imagine that programs
232used to create fonts deal with this, but even then, there is a visual aspect to
233it, too. The good news is that in \CONTEXT\ we can manipulate features, so, in
234theory, one can create a so|-|called font goodie file for a specific instance.
235
236\stopsubject
237
238\startsubject[title=Shapes]
239
240For \OPENTYPE\ \POSTSCRIPT\ shapes, we always have to do a dummy rendering in
241order to get the right bounding box information. For \TRUETYPE, this information
242is already present but not when we use a variable instance, so I had to do a bit
243of coding for that. Here we face a problem. For \TEX, we need the width, height
244and depth of a glyph. Consider the following case:
245
246\startlinecorrection
247\startMPcode
248path p ; p := fullcircle xysized (3cm,2cm) ;
249fill p
250    withcolor "lightgray" ;
251draw boundingbox currentpicture
252    withpen pencircle scaled .5mm
253    withcolor "darkgray" ;
254setbounds currentpicture to p ;
255draw boundingbox currentpicture
256    leftenlarged 2mm rightenlarged 5mm ;
257\stopMPcode
258\stoplinecorrection
259
260The shape has a bounding box that fits the shape. However, its left corner is not
261at the origin. So, when we calculate a tight bounding box, we cannot use it for
262actually positioning the glyph. We do use it (for horizontal scripts) to get the
263height and depth, but for the width, we depend on an explicit value. In \OPENTYPE\
264\POSTSCRIPT, we have the width available, and how the shape is positioned relative
265to the origin doesn't much matter. In a \TRUETYPE\ shape, a bounding box is part
266of the specification, as is the width, but for a variable font, one has to use
267so|-|called phantom points to recalculate the width, and the test fonts I had were
268not suitable for investigating this.
269
270At any rate, once I could generate documents with typeset text using variable
271fonts, it was time to start thinking about a user interface. A variable font
272can have predefined instances, but, of course, a user also wants to mess with axis
273values. Take one of the test fonts: Adobe Variable Font Prototype. It has several
274instances:
275
276\unexpanded\def\SampleFont#1#2#3%
277 {\NC #2
278  \NC \definedfont[name:#1#3*default]It looks like this!
279      \normalexpanded{\noexpand\NC\currentfontinstancespec}
280  \NC \NR}
281
282\starttabulate[|||T|]
283\SampleFont {adobevariablefontprototype} {extralight}            {extralight}
284\SampleFont {adobevariablefontprototype} {light}                 {light}
285\SampleFont {adobevariablefontprototype} {regular}               {regular}
286\SampleFont {adobevariablefontprototype} {semibold}              {semibold}
287\SampleFont {adobevariablefontprototype} {bold}                  {bold}
288\SampleFont {adobevariablefontprototype} {black high contrast}   {blackhighcontrast}
289\SampleFont {adobevariablefontprototype} {black medium contrast} {blackmediumcontrast}
290\SampleFont {adobevariablefontprototype} {black}                 {black}
291\stoptabulate
292
293Such an instance is accessed with:
294
295\starttyping
296\definefont
297  [MyLightFont]
298  [name:adobevariablefontprototypelight*default]
299\stoptyping
300
301The Avenir Next variable demo font (currently) provides:
302
303\starttabulate[|||T|]
304\SampleFont {avenirnextvariable} {regular}           {regular}
305\SampleFont {avenirnextvariable} {medium}            {medium}
306\SampleFont {avenirnextvariable} {bold}              {bold}
307\SampleFont {avenirnextvariable} {heavy}             {heavy}
308\SampleFont {avenirnextvariable} {condensed}         {condensed}
309\SampleFont {avenirnextvariable} {medium condensed}  {mediumcondensed}
310\SampleFont {avenirnextvariable} {bold condensed}    {boldcondensed}
311\SampleFont {avenirnextvariable} {heavy condensed}   {heavycondensed}
312\stoptabulate
313
314Before we continue, I will show a few examples of variable shapes. Here, we use some
315\METAFUN\ magic. Just take these definitions for granted.
316
317\startbuffer[a]
318\startMPcode
319  draw outlinetext.b
320    ("\definedfont[name:adobevariablefontprototypeextralight]foo@bar")
321    (withcolor "gray")
322    (withcolor red withpen pencircle scaled 1/10)
323    xsized .45TextWidth ;
324\stopMPcode
325\stopbuffer
326
327\startbuffer[b]
328\startMPcode
329  draw outlinetext.b
330    ("\definedfont[name:adobevariablefontprototypelight]foo@bar")
331    (withcolor "gray")
332    (withcolor red withpen pencircle scaled 1/10)
333    xsized .45TextWidth ;
334\stopMPcode
335\stopbuffer
336
337\startbuffer[c]
338\startMPcode
339  draw outlinetext.b
340    ("\definedfont[name:adobevariablefontprototypebold]foo@bar")
341    (withcolor "gray")
342    (withcolor red withpen pencircle scaled 1/10)
343    xsized .45TextWidth ;
344\stopMPcode
345\stopbuffer
346
347\startbuffer[d]
348\startMPcode
349  draw outlinetext.b
350    ("\definefontfeature[whatever][axis={weight:350}]%
351      \definedfont[name:adobevariablefontprototype*whatever]foo@bar")
352    (withcolor "gray")
353    (withcolor red withpen pencircle scaled 1/10)
354    xsized .45TextWidth ;
355\stopMPcode
356\stopbuffer
357
358\typebuffer[a,b,c,d]
359
360The results are shown in \in {figure} [fig:whatever:1]. What we see here is that
361as long as we fill the shape everything will look as expected, but only using an
362outline won't. The crucial (control) points are moved to different locations
363and, as a result, they can end up inside the shape. Giving up outlines is the price
364we evidently need to pay. Of course this is not unique for variable fonts,
365although, in practice, static fonts behave better. To some extent, we're back to
366where we were with \METAFONT\ and (for instance) Computer Modern: because these
367originate in bitmaps (and probably use similar design logic) we also can have
368overlap and bits and pieces pasted together and no one will notice that. The
369first outline variants of Computer Modern also had such artifacts, while in the
370static Latin Modern successors, outlines were cleaned up.
371
372\startplacefigure[title=Four variants,reference=fig:whatever:1]
373    \startcombination[2*2]
374        {\getbuffer[a]} {a}
375        {\getbuffer[b]} {b}
376        {\getbuffer[c]} {d}
377        {\getbuffer[d]} {c}
378    \stopcombination
379\stopplacefigure
380
381The fact that we need to preprocess an instance, but that we only know how to do
382that after we have retrieved information about axis values from the font means
383that the font handler has to be adapted to keep caching correct. Another
384definition is:
385
386\starttyping
387\definefontfeature
388  [lightdefault]
389  [default]
390  [axis={weight:230,contrast:50}]
391
392\definefont
393  [MyLightFont]
394  [name:adobevariablefontprototype*lightdefault]
395\stoptyping
396
397Here, the complication is that where normally features are dealt with after
398loading, the axis feature is part of the preparation (and caching). If you want
399the virtual font solution, you can do this:
400
401\starttyping
402\definefontfeature
403  [inlinelightdefault]
404  [default]
405  [axis={weight:230,contrast:50},
406   variableshapes=yes]
407
408\definefont
409  [MyLightFont]
410  [name:adobevariablefontprototype*inlinelightdefault]
411\stoptyping
412
413When playing with these fonts, it was hard to see if loading was done right. For
414instance, not all values make sense. It is beyond the scope of this article, but
415axes like weight, width, contrast, and italic values get applied differently to
416so|-|called regions (subspaces). So, say that we have an $x$ coordinate with the
417value $50$. This value can be adapted in, for instance, four subspaces (regions),
418so we actually get:
419
420\startformula
421    x^\prime = x
422             + s_1 \times x_1
423             + s_2 \times x_2
424             + s_3 \times x_3
425             + s_4 \times x_4
426\stopformula
427
428The (here four) scale factors $s_n$ are determined by the axis value. Each axis
429has some rules about how to map the values $230$ for weight and $50$ for contrast
430to such a factor. Each region has its own translation from axis values to these
431factors. The deltas $x_1,\dots,x_4$ are provided by the font. In a
432\POSTSCRIPT|-|based font, we find sequences like:
433
434\starttyping
4351 <setvstore>
436120 [10 -30 40 -60] 1 <blend> ... <operator>
437100 120 [10 -30 40 -60] [30 -10 -30 20] 2 <blend> .. <operator>
438\stoptyping
439
440A store refers to a region specification. From there the factors are calculated
441using the chosen values on the axis. The deltas are part of the glyphs
442specification. Officially, there can be multiple region specifications, but how
443likely it is that they will be used in real fonts is an open question.
444
445In \TRUETYPE\ fonts, the deltas are not in the glyph specification but in a
446dedicated \type {gvar} table.
447
448\starttyping
449apply x deltas [10 -30 40 -60] to x 120
450apply y deltas [30 -10 -30 20] to y 100
451\stoptyping
452
453Here, the deltas come from tables outside the glyph specification and their
454application is triggered by a combination of axis values and regions.
455
456The following two examples use Avenir Next Variable and demonstrate that kerning
457is adapted to the variant.
458
459\startbuffer
460\definefontfeature
461  [default:shaped]
462  [default]
463  [axis={width:10}]
464
465\definefont
466  [SomeFont]
467  [file:avenirnextvariable*default:shaped]
468\stopbuffer
469
470\typebuffer \getbuffer
471
472\start \showglyphs \showfontkerns \SomeFont \input zapf \wordright{Hermann Zapf}\par \stop
473
474\startbuffer
475\definefontfeature
476  [default:shaped]
477  [default]
478  [axis={width:100}]
479
480\definefont
481  [SomeFont]
482  [file:avenirnextvariable*default:shaped]
483\stopbuffer
484
485\typebuffer \getbuffer
486
487\start \showglyphs \showfontkerns \SomeFont \input zapf \wordright{Hermann Zapf}\par \stop
488
489\stopsubject
490
491\startsubject[title=Embedding]
492
493Once we're done typesetting and a \PDF\ file has to be created, there are three
494possible routes:
495
496\startitemize
497    \startitem
498        We can embed the shapes as \PDF\ images (inline literal) using virtual
499        font technology. We cannot use so|-|called xforms here, because we want to
500        support color selectively in text.
501    \stopitem
502    \startitem
503        We can wait till the \PDF\ format supports such fonts, which might
504        happen, but even then we might be stuck for years with viewers getting
505        there. Also, documents need to be printed, and when printer support might
506        arrive is another unknown.
507    \stopitem
508    \startitem
509        We can embed a regular font with shapes that match the chosen values on the
510        axis. This solution is way more efficient than the first.
511    \stopitem
512\stopitemize
513
514Once I could interpret the right information in the font, the first route was the
515way to go. A side effect of having a converter for both outline types meant that
516it was trivial to create a virtual font at runtime. This option will stay in
517\CONTEXT\ as a pseudo|-|feature \type {variableshapes}.
518
519When trying to support variable fonts, I tried to limit the impact on the backend
520code. Also, processing features and such was not touched. The inclusion of the
521right shapes is done via a callback that requests the blob to be injected in the
522\type {cff} or \type {glyf} table. When implementing this, I actually found out
523that the \LUATEX\ backend also does some juggling of charstrings to inline
524subroutines. In retrospect, I could have learned a few tricks faster by looking
525at that code, but I never realized that it was there. Looking at the code again,
526it strikes me that the whole inclusion could be done with \LUA\ code, and, some
527day, I will give that a try.
528
529\stopsubject
530
531\startsubject[title=Conclusion]
532
533When I first heard about variable fonts, I was confident that when they showed
534up, they could be supported. Of course, a specimen was needed to prove this. A
535first implementation demonstrates that, indeed, it's no big deal to let \CONTEXT\
536with \LUATEX\ handle such fonts. Of course, we need to fill in some gaps, which
537can be done once we have complete fonts. And then, of course, users will demand
538more control. In the meantime, the helper script that deals with identifying
539fonts by name has been extended, and the relevant code has been added to the
540distribution. At some point, the \CONTEXT\ Garden will provide the \LUATEX\
541binary that has the callback.
542
543I end on a warning note. On the one hand, this technology looks promising, but on
544the other hand, one can easily get lost. Most such fonts probably operate over a
545well|-|defined domain of values, but, even then, one should be aware of complex
546interactions with features like positioning or replacements. Not all combinations
547can be tested. It's probably best to stick with fonts that have all the relevant
548tables and don't depend on properties of a specific rendering technology.
549
550Although support is now present in the core of \CONTEXT, the official release
551will happen at the \CONTEXT\ meeting in 2017. By then, I hope to have tested more
552fonts. Maybe the interface will also have been extended by then, because, after
553all, \TEX\ is about control.
554
555\stopsubject
556
557\stopchapter
558
559\stopcomponent
560