1
2
3\usetypescriptfile[typehusayni]
4
5\startcomponent hybridgoodies
6
7\environment hybridenvironment
8
9
10
11\definefontfeature
12 [husayninone]
13 [analyze=yes,mode=node,
14 language=dflt,script=arab,
15 ccmp=yes]
16
17\definefontfeature
18 [husaynidefault]
19 [analyze=yes,mode=node,
20 language=dflt,script=arab,
21 ccmp=yes,init=yes,medi=yes,fina=yes,
22 rlig=yes,calt=yes,salt=yes,anum=yes,
23 kern=yes,curs=yes,mark=yes,mkmk=yes,
24 ss01=yes,ss03=yes,ss10=yes,ss12=yes,ss15=yes,ss16=yes,
25 ss19=yes,ss24=yes,ss25=yes,ss26=yes,ss27=yes,ss31=yes,
26 ss34=yes,ss35=yes,ss36=yes,ss37=yes,ss38=yes,ss41=yes,
27 ss43=yes]
28
29\definefontfeature
30 [husaynifirstorder]
31 [script=arab,ss01=yes,ss03=yes,ss05=yes,
32 ss10=yes,ss12=yes,ss15=yes,ss16=yes,ss19=yes,ss24=yes,
33 ss25=yes,ss26=yes,ss27=yes,ss31=yes,ss34=yes,ss35=yes,
34 ss36=yes,ss37=yes,ss38=yes,ss41=yes,ss42=yes,ss43=yes]
35
36\definefontfeature
37 [husaynistackjiimmultilevel]
38 [script=arab,ss05=yes]
39
40\definefontfeature
41 [husayniminimalstretching]
42 [script=arab,
43 ss05=yes,ss09=yes,ss06=yes,ss13=yes,ss17=yes,ss40=yes,
44 js11=yes,js14=yes,js16=yes]
45
46\definefontfeature
47 [husaynimaximalstretching]
48 [script=arab,
49 ss05=yes,ss09=yes,ss06=yes,ss13=yes,ss17=yes,ss40=yes,
50 js13=yes,js14=yes,js16=yes]
51
52\definefontfeature
53 [husaynichophaa]
54 [script=arab,
55 ss05=yes,ss09=yes,ss06=yes,ss13=yes,ss17=yes,ss54=yes]
56
57\definefontfeature
58 [husayninormal]
59 [goodies=husayni,
60 featureset=default]
61
62\definefont[ArabicFontNone][husayni*husayninone at 40pt]
63\definefont[ArabicFontFull][husayni*husaynidefault at 40pt]
64
65\startchapter[title={Font Goodies}]
66
67\startsection[title={Introduction}]
68
69The Oriental \TEX\ project is one of the first and more ambitious users of
70\LUATEX. A major undertaking in this project is the making of a rather full
71features and complex font for typesetting Arabic. As the following text will show
72some Arabic, you might get the impression that Im an expert but be warned that
73Im far from that. But as Idris compensates this quite well the team has a lot of
74fun in figuring out how to achieve our goals using \OPENTYPE\ technology in
75combination with \LUATEX\ and \MKIV. A nice side effect of this is that we end up
76with some neat tricks in the \CONTEXT\ core.
77
78Before we come to some of these goodies, an example of Arabic is given that
79relates quite well to the project. It was first used at the euro\TEX\ 2009
80meeting. Take the following 6 shapes:
81
82\starttabulate[cccccc]
83\NC \ArabicFontFull ل \NC \ArabicFontFull و \NC \ArabicFontFull ا \NC \ArabicFontFull ت \NC \ArabicFontFull ي \NC \ArabicFontFull خ \NC \NR
84\NC \type{l} \NC \type{w} \NC \type{ā} \NC \type{t} \NC \type{ī} \NC \type{kh} \NC \NR
85\stoptabulate
86
87With these we can make the name \LUATEX\ and as we use a nice script we can
88forget about the loweredE. Putting these characters in sequence is not enough as
89Arabic typesetting has to mimick the subtle aspects of scribes.
90
91In Latin scripts we have mostly onetoone and manytoone substitutions.
92These can happen in sequence which in in practice boils down to multiple passes
93over the stream of characters. In this process sometimes surrounding characters
94(or shapes) play a role, for instance ligatures are not always wanted and their
95coming into existence might depend on neighbouring characters. In some cases
96glyphs have to be (re)positioned relative to each other. While in Latin scripts
97the number of substitutions and positioning is not that large but in advanced
98Arabic fonts it can be pretty extensive.
99
100With \OPENTYPE\ we have some machinery available, so we try to put as much logic
101in the font as possible. However, in addition we have some dedicated optimizing
102routines. The whole process is split into a couple if stages.
103
104The so called FirstOrder Analysis puts a given character into isolated,
105initial, middle, or final state. Next, the SecondOrder Analysis looks at the
106characters and relates this state to what characters precede or succeed it. Based
107on that state we do character substitutions. There can be multiple analysis and
108replacements in sequence. We can do some simple aesthetic stretching and
109additional related replacements. We need to attach identity marks and vowels in
110proper but nice looking places. In most cases were then done. Contrary to other
111fonts we dont use many ligatures but compose characters.
112
113The previous steps already give reasonable results and implementing it also
114nicely went along with the development of \LUATEX\ and \CONTEXT\ \MKIV. Currently
115were working on extending and perfecting the font to support what we call
116ThirdOrder Contextual Analysis. This boils down to an interplay between the
117paragraph builder and additional font features. In order to get pleasing spacing
118we apply further substitutions, this time with wider or narrower shapes. When
119this is done we need to reattach identity marks and vowels. Optionally we can
120apply \HZ\ like stretching as a finishing touch but so far we didnt follow that
121route yet.
122
123So, lets see how we can typeset the word \LUATEX\ in Arabic using some of these
124techniques.
125
126\startlines
127no order (kh ī t ā w [u] l)\hfilll {\righttoleft\ArabicFontNone لُواتيخ}
128first order \hfilll {\subff{husaynifirstorder}\righttoleft\ArabicFontFull لُواتيخ}
129second order \hfilll {\righttoleft\ArabicFontFull لُواتيخ}
130second order (Jiimstacking) \hfilll {\addff{husaynistackjiimmultilevel}\righttoleft\ArabicFontFull لُواتيخ}
131minimal stretching \hfilll {\addff{husayniminimalstretching}\righttoleft\ArabicFontFull لُواتيخ}
132maximal stretching (level 3) \hfilll {\addff{husaynimaximalstretching}\righttoleft\ArabicFontFull لُواتيخ}
133chopped letter khaa (for e.g.\ underlining) \hfilll {\addff{husaynichophaa}\righttoleft\ArabicFontFull لُواتيخ}
134\stoplines
135
136As said, this font is quite complex in the sense that it has many features and
137associated lookups. In addition to the usual features we have stylistic and
138justification variants. As these are not standardized (after all, each font can
139have its own look and feel and associated treatments) we store some information
140in the goodies files that ship with this font.
141
142\startbuffer[stylistics]
143\startluacode
144 local goodies = fonts.goodies.load("husayni")
145 local stylistics = goodies and goodies.stylistics
146 if stylistics then
147 local col, row, type = context.NC, context.NR, context.type
148 context.starttabulate { "|l|pl|" }
149 col() context("feature") col() context("meaning") col() row()
150 for feature, meaning in table.sortedpairs(stylistics) do
151 col() type(feature) col() type(meaning) col() row()
152 end
153 context.stoptabulate()
154 end
155\stopluacode
156\stopbuffer
157
158\getbuffer[stylistics]
159
160It is highly unlikely that a user will remember all these features, which is why
161there will be a bunch of predefined combinations. These are internalized as
162follows:
163
164\startbuffer[featuresets]
165\startluacode
166 local goodies = fonts.goodies.load("husayni")
167 local featuresets = goodies and goodies.featuresets
168 if featuresets then
169 local col, row, type = context.NC, context.NR, context.type
170 context.starttabulate { "|l|pl|" }
171 col() context("featureset") col() context("definitions") col() row()
172 for featureset, definitions in table.sortedpairs(featuresets) do
173 col() type(featureset) col()
174 for k, v in table.sortedpairs(definitions) do
175 type(string.format("%s=%s",k,tostring(v)))
176 context.quad()
177 end
178 col() row()
179 end
180 context.stoptabulate()
181 end
182\stopluacode
183\stopbuffer
184
185\getbuffer[featuresets]
186
187\stopsection
188
189\startsection[title={Color}]
190
191One of the objectives of the oriental \TEX\ project is to bring color to typeset
192Arabic. When Idris started making samples with much manual intervention it was
193about time to figure out if it could be supported by a bit of \LUA\ code.
194
195As the colorization concerns classes of glyphs (like vowels) this is something
196that can best be done after all esthetics have been sorted out. Because things
197like coloring are not part of font technology and because we dont want to misuse
198the \OPENTYPE\ feature mechanisms for that, the solution lays in an extra file
199that describes these goodies.
200
201\startbuffer[goodies1]
202\definefontfeature
203 [husaynicolored]
204 [goodies=husayni,
205 colorscheme=default,
206 featureset=default]
207\stopbuffer
208
209\startbuffer[goodies2]
210\start
211 \definedfont[husayni*husaynicolored at 72pt]
212 \righttoleft
213 \resetfontcolorscheme لُواتيخ ألف ليلة وليلة \par
214 \setfontcolorscheme [1]لُواتيخ ألف ليلة وليلة \crlf
215 \setfontcolorscheme [2]لُواتيخ ألف ليلة وليلة \crlf
216\stop
217\stopbuffer
218
219\getbuffer[goodies1,goodies2]
220
221The second and third of these three lines have colored vowels and identity marks.
222So how did we get the colors? There are actually two mechanisms involved in this:
223
224\startitemize[packed]
225\startitem we need to associate colorschemes with classed of glyphs \stopitem
226\startitem we need to be able to turn on and off coloring \stopitem
227\stopitemize
228
229The first is done by loading goodies and selecting a colorscheme:
230
231\typebuffer[goodies1]
232
233Turning on and off coloring is done with two commands (we might provide a proper
234environment for this) as shown in:
235
236\typebuffer[goodies2]
237
238If you look closely at the feature definition youll notice that we also choose a
239default featureset. For most (latin) fonts the regular feature definitions are
240convenient, but for fonts that are used for Arabic there are preferred
241combinations of features as there can be many.
242
243Currently the font we use here has the following colorschemes:
244
245\startbuffer[colorschemes]
246\startluacode
247 local goodies = fonts.goodies.load("husayni")
248 local colorschemes = goodies and goodies.colorschemes
249 if colorschemes then
250 local col, row, type = context.NC, context.NR, context.type
251 context.starttabulate { "|l|pl|" }
252 col() context("colorscheme") col() context("numbers") col() row()
253 for colorscheme, numbers in table.sortedpairs(colorschemes) do
254 col() type(colorscheme) col()
255 for i=1,#numbers do
256 type(i)
257 context.quad()
258 end
259 col() row()
260 end
261 context.stoptabulate()
262 end
263\stopluacode
264\stopbuffer
265
266\getbuffer[colorschemes]
267
268\stopsection
269
270\startsection[title={The goodies file}]
271
272In principle a goodies files can contain anuy data that makes sense but in order
273to be useable some entries have a prescribed structure. A goodies file looks as
274follows:
275
276\starttyping
277return {
278 name = "husayni",
279 version = "1.00",
280 comment = "Goodies that complement the Husayni font by Idris Samawi Hamid.",
281 author = "Idris Samawi Hamid and Hans Hagen",
282 featuresets = {
283 default = {
284 key = value, <table>, ...
285 },
286 ...
287 },
288 stylistics = {
289 key = value, ...
290 },
291 colorschemes = {
292 default = {
293 [1] = {
294 "glypha.one", "glyphb.one", ...
295 },
296 ...
297 }
298 }
299}
300\stoptyping
301
302We already saw the list of special features and these are defined in the \type
303{stylistics} stable. In this document, that list was typeset using the following
304(hybrid) code:
305
306\typebuffer[stylistics]
307
308The table with colorscheme that we showed is generated with:
309
310\getbuffer[colorschemes]
311
312In a similar fashion we typeset the featuresets:
313
314\typebuffer[featuresets]
315
316The unprocessed \type {featuresets} table can contain one or more
317named sets and each set can be a mixture of tables and key value
318pairs. Say that we have:
319
320\starttyping
321 default = {
322 kern = "yes", { ss01 = "yes" }, { ss02 = "yes" }, "mark"
323 }
324\stoptyping
325
326Given the previous definition, the order of processing is as follows.
327
328\startitemize[packed,n]
329\startitem \type {{ ss01 = "yes" }} \stopitem
330\startitem \type {{ ss02 = "yes" }} \stopitem
331\startitem \type {mark} (set to \type {"yes"}) \stopitem
332\startitem \type {kern = "yes"} \stopitem
333\stopitemize
334
335So, first we process the indexed part if the list, and next the hash. Already set
336values are not set again. The advantage of using a \LUA\ table is that you can
337simplify definitions. Before we return the table we can define local variables,
338like:
339
340\starttyping
341local one = { ss01 = "yes" }
342local two = { ss02 = "yes" }
343local pos = { kern = "yes", mark = "yes" }
344\stoptyping
345
346and use them in:
347
348\starttyping
349default = {
350 one, two, pos
351}
352\stoptyping
353
354That way we we can conveniently define all kind of interesting combinations
355without the need for many repetitive entries.
356
357The \type {colorsets} table has named subtables that are (currently) indexed by
358number. Each number is associated with a color (at the \TEX\ end) and is coupled
359to a list of glyphs. As you can see here, we use the name of the glyph. We prefer
360this over an index (that can change during development of the font). We cannot
361use \UNICODE\ points as many such glyphs are just variants and have no unique
362code.
363
364\stopsection
365
366\startsection[title={Optimizing Arabic}]
367
368\usemodule[abr01,narrowtt]
369
370\enabletrackers[fonts.goodies,nodes.optimizer]
371
372The ultimate goal of the Oriental \TEX\ project is to improve the look and feel
373of a paragraph. Because \TEX\ does a pretty good job on breaking the paragraph
374into lines, and because complicating the paragraph builder is not a good idea, we
375finally settled on improving the lines that result from the par builder. This
376approach is rather close to what scribes do and the advanced Husayni font
377provides features that support this.
378
379In principle the current optimizer can replace character expansion but that would
380slow down considerably. Also, for that we first have to clean up the experimental
381\LUA\ based par builder.
382
383After several iterations the following approach was chosen.
384
385\startitemize
386
387\startitem
388 We typeset the paragraph with an optimal feature set. In our case this is
389 \type {husaynidefault}.
390\stopitem
391
392\startitem
393 Next we define two sets of additional features: one that we can apply to
394 shrink words, and one that does the opposite.
395\stopitem
396
397\startitem
398 When the line has a badness we dont like, we either stepwise shrink words or
399 stretch them, depending on how bad things are.
400\stopitem
401
402\stopitemize
403
404The set that takes care of shrinking is defined as:
405
406\starttyping
407\definefontfeature
408 [shrink]
409 [husaynidefault]
410 [flts=yes,js17=yes,ss05=yes,ss11=yes,ss06=yes,ss09=yes]
411\stoptyping
412
413Stretch has a few more variants:
414
415\starttyping
416\definefontfeature
417 [minimalstretching]
418 [husaynidefault]
419 [js11=yes,js03=yes]
420\definefontfeature
421 [mediumstretching]
422 [husaynidefault]
423 [js12=yes,js05=yes]
424\definefontfeature
425 [maximalstretching]
426 [husaynidefault]
427 [js13=yes,js05=yes,js09=yes]
428\definefontfeature
429 [wideall]
430 [husaynidefault]
431 [js11=yes,js12=yes,js13=yes,js05=yes,js09=yes]
432\stoptyping
433
434Next we define a font solution:
435
436\starttyping
437\definefontsolution
438 [FancyHusayni]
439 [goodies=husayni,
440 less=shrink,
441 more={minimalstretching,mediumstretching,maximalstretching,wideall}]
442\stoptyping
443
444Because these featuresets relate quite closely to the font design we dont use
445this way if defining but put the definitions in the goodies file:
446
447\startntyping
448 .....
449 featuresets = { here we dont have references to featuresets
450 default = {
451 default,
452 },
453 minimalstretching = {
454 default, js11 = yes, js03 = yes,
455 },
456 mediumstretching = {
457 default, js12=yes, js05=yes,
458 },
459 maximalstretching= {
460 default, js13 = yes, js05 = yes, js09 = yes,
461 },
462 wideall = {
463 default, js11 = yes, js12 = yes, js13 = yes, js05 = yes, js09 = yes,
464 },
465 shrink = {
466 default, flts = yes, js17 = yes, ss05 = yes, ss11 = yes, ss06 = yes, ss09 = yes,
467 },
468 },
469 solutions = { here we have references to featuresets, so we use strings!
470 experimental = {
471 less = { "shrink" },
472 more = { "minimalstretching", "mediumstretching", "maximalstretching", "wideall" },
473 },
474 },
475 .....
476\stopntyping
477
478Now the definition looks much simpler:
479
480\startbuffer
481\definefontsolution
482 [FancyHusayni]
483 [goodies=husayni,
484 solution=experimental]
485\stopbuffer
486
487
488
489\typebuffer \getbuffer
490
491{\em I want some funny text (complete with translation). Actually I want all
492examples translated.}
493
494\startbuffer[sample]
495قد صعدنا
496ذرى الحقائق بأقدام النبوة و الولاية و نورنا
497سبع طبقات أعلام الفتوى بالهداية فنحن ليوث
498الوغى و غيوث الندى و طعان العدى و فينا السيف و
499القلم في العاجل و لواء الحمد
500و الحوض في الآجل و أسباطنا حلفاء
501الدين و خلفاء النبيين و مصابيح الأمم و مفاتيح
502الكرم فالكليم ألبس حلة الاصطفاء لما عهدنا
503منه الوفاء و روح القدس في جنان الصاقورة ذاق من
504حدائقنا الباكورة و شيعتنا الفئة الناجية و
505الفرقة الزاكية و صاروا لنا ردءا و صونا و على
506الظلمة ألبا و عونا و سينفجر لهم ينابيع
507الحيوان بعد لظى النيران لتمام آل حم و طه و
508الطواسين من السنين و هذا الكتاب درة من درر
509الرحمة و قطرة من بحر الحكمة و كتب الحسن بن
510علي العسكري في سنة أربع و خمسين و مائتين
511\stopbuffer
512
513\startbuffer
514\definedfont[husayni*husaynidefault at 24pt]
515
516\expanded{\setuplocalinterlinespace[line=\the\dimexpr2\lineheight]}
517\setfontsolution[FancyHusayni]
518\enabletrackers[builders.paragraphs.solutions.splitters.colors]
519\righttoleft \getbuffer[sample] \par
520\disabletrackers[builders.paragraphs.solutions.splitters.colors]
521\resetfontsolution
522\stopbuffer
523
524In the following example the yellow words are stretched and the green ones are
525shrunken.\footnote {Make sure that the paragraph is finished (for instance using
526\type {\par} before resetting it.)}
527
528\typebuffer
529
530\start \getbuffer \stop
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545This mechanism is somewhat experimental as is the (user) interface. It is also
546rather slow compared to normal processing. There is room for improvement but I
547will do that when other components are more stable so that simple variants (that
548we can use here) can be derived.
549
550When criterium0 used above is changed into for instance5 processing is faster.
551When you enable a preroll processing is more time consuming. Examples of settings
552are:
553
554\starttyping
555\setupfontsolutions[method={preroll,normal},criterium=2]
556\setupfontsolutions[method={preroll,random},criterium=5]
557\setupfontsolutions[method=reverse,criterium=8]
558\setupfontsolutions[method=random,criterium=2]
559\stoptyping
560
561Using a preroll is slower because it first tries all variants and then settles
562for the best; otherwise we process the first till the last solution till the
563criterium is satisfied.
564
565
566
567
568\stopsection
569
570\startsection[title={Protrusion and expansion}]
571
572There are two entries in the goodies file that relate to advanced parbuilding:
573\type {protrusions} and \type {expansions}.
574
575\starttyping
576protrusions = {
577 vectors = {
578 pure = {
579 [0x002C] = { 0, 1 }, comma
580 [0x002E] = { 0, 1 }, period
581 .....
582 }
583 }
584}
585\stoptyping
586
587These vectors are similar to the ones defined globally but the vectors defined in
588a goodie file are taken instead when present.
589
590\stopsection
591
592\startsection[title={Filenames and properties}]
593
594As filenames and properties of fonts are somewhat of an inconsistent mess, we can
595use the goodies to provide more information:
596
597\starttyping
598files = {
599 name = "antykwapoltawskiego", shared
600 list = {
601 ["AntPoltLtCondRegular.otf"] = {
602 name = "antykwapoltawskiego",
603 style = "regular",
604 weight = "light",
605 width = "condensed",
606 },
607 .....
608 }
609 }
610}
611\stoptyping
612
613Internally this will become a lookup tree so that we can have a predictable
614specifier:
615
616\starttyping
617\definefont[MyFontA][antykwapoltawskiegobolditalic]
618\definefont[MyFontB][antykwapoltawskiegonormalitaliccondensed]
619\definefont[MyFontC][antykwapoltawskiegolightregularsemicondensed]
620\stoptyping
621
622Of course one needs to load the goodies. One way to force that is:
623
624\starttyping
625\loadfontgoodies[antykwapoltawskiego]
626\stoptyping
627
628The Antykwa Poltawskiego family is rather large and provides all kind of
629combinations.
630
631\startbuffer
632\usemodule[fontsgoodies]
633\showfontgoodiesfiles[name=antykwapoltawskiego]
634\stopbuffer
635
636\startpacked
637\getbuffer
638\stoppacked
639
640This list is generated with:
641
642\typebuffer
643
644\stopsection
645
646\stopchapter
647
648\stopcomponent
649 |