x-asciimath.mkiv /size: 18 Kb    last modification: 2021-10-28 13:51
1%D \module
2%D   [       file=x-asciimath,
3%D        version=2014.06.01, % 2006.04.24, % 1999.11.06,
4%D          title=\CONTEXT\ Modules,
5%D       subtitle=AsciiMath,
6%D         author=Hans Hagen,
7%D           date=\currentdate,
8%D      copyright={PRAGMA ADE \& \CONTEXT\ Development Team}]
9%C
10%C This module is part of the \CONTEXT\ macro||package and is
11%C therefore copyrighted by \PRAGMA. See mreadme.pdf for
12%C details.
13
14\registerctxluafile{x-asciimath}{}
15
16%D When the Math4All project started, we immediately started using content \MATHML.
17%D Because in school math there is often a reference to calculator input, we also
18%D provided what we called \quote {calcmath}: a predictable expression based way
19%D entering math. At some point \OPENMATH\ was also used but that was later
20%D abandoned because editing is more cumbersome.
21%D
22%D Due to limitations in the web variant (which is independent of rendering for
23%D paper but often determines the coding of document, not seldom for the worse) the
24%D switch was made to presentational \MATHML. But even that proved to be too complex
25%D for rendering on the web, so it got converted to so called \ASCIIMATH\ which
26%D can be rendered using some \JAVASCRIPT\ magic. However, all the formulas (and
27%D we're talking of tens of thousands of them) were very precisely coded by the main
28%D author. Because in intermediate stages of the editing (by additional authors) a
29%D mixture of \MATHML\ and \ASCIIMATH\ was used, we wrote the first version of this
30%D module. As reference we took \url
31%D {http://www1.chapman.edu/~jipsen/mathml/asciimath.html} and. The idea was to
32%D stick to \MATHML\ as reference and if needed use \ASCIIMATH\ as annotation.
33%D
34%D Eventually we ended up with supporting several math encodings in \CONTEXT\ that
35%D could be used mixed: content \MATHML\ (preferred), presentation \MATHML\ (often
36%D messy), \OPENMATH\ (somewhat minimalistic) calcmath (handy for students who are
37%D accustomed to calculators), \ASCIIMATH\ (to make web support easier) and of
38%D course \TEX.
39%D
40%D The first version had some limitations as we didn't want to support all quirks of
41%D \ASCIIMATH\ and also because I was not really in the mood to write a complex parser
42%D when a bit of sane coding can work equally well. Some comments from that version:
43%D
44%D \startnarrower
45%D \startitemize
46%D \item We support only the syntactically clear variants and as long as lpeg does
47%D       not support left recursion this is as far as we want to go.
48%D \item The parser is rather insensitive for spaces but yet the advice is to avoid
49%D       weird coding like \type {d/dxf(x)} but use \type {d/dx f(x)} instead. After
50%D       all we're not in a compact coding cq.\ parser challenge.
51%D \item We also don't support the somewhat confusing \type {sqrt sqrt 2} nor \type
52%D       {root3x} (although the second one kind of works). A bit of defensive coding
53%D       does not hurt.
54%D \item We can process \type {a/b/c/d} but it's not compatible with the default
55%D       behaviour of \ASCIIMATH. Use grouping instead. Yes, we do support the somewhat
56%D       nonstandard grouping token mix.
57%D \item You should use explicit \type {text(..)} directives as one can never be sure
58%D       what is a reserved word and  not.
59%D \stopitemize
60%D
61%D Actually, as the only parsing sensitive elements of \TEX\ are fractions (\type {\over}
62%D and friends, a restricted use of \TEX\ coding is probably as comprehensive and
63%D parsable. The webpage with examples served as starting point so anything beyond
64%D what can be found there isn't supported.
65%D \stopnarrower
66%D
67%D Then in 2014 something bad happened. Following the fashion of minimal encoding
68%D (which of course means messy encoding of complex cases and which can make authors
69%D sloppy too) the web based support workflow of the mentioned project ran into some
70%D limitations and magically one day all carefully coded \MATHML\ was converted into
71%D \ASCIIMATH. As there was no way to recover the original thousands of files and
72%D tens of thousands of formulas we were suddenly stuck with \ASCIIMATH. Because the
73%D conversion had be done automagically, we also saw numerous errors and were forced
74%D to come up with some methods to check formulas. Because \MATHML\ poses some
75%D restrictions it has predictable rendering; \ASCIIMATH\ on the other hand enforces
76%D no structure. Also, because \MATHML\ has to be valid \XML\ it always processes.
77%D Of course, during the decade that the project had run we also had to built in
78%D some catches for abuse but at least we had a relatively stable and configurable
79%D subsystem. So, in order to deal with less predictable cases as well as extensive
80%D checking, a new \ASCIIMATH\ parser was written, one that could also be used to
81%D trace bad coding.
82%D
83%D Because the formal description is incomplete, and because some links to resources
84%D are broken, and because some testing on the web showed that sequences of characters
85%D are interpreted that were not mentioned anywhere (visible), and because we noticed
86%D that the parser was dangerously tolerant, the new code is quite different from the
87%D old code.
88%D
89%D One need to keep in mind that because spaces are optional, the only robust way to
90%D edit \ASCIIMATH\ is to use a \WYSIWYG\ editor and hope that the parser doesn't
91%D change ever. Keys are picked up from spaceless sequences and when not recognized
92%D a (sequence) of characters is considered to be variables. So, \type {xsqrtx} is
93%D valid and renders as \type {$x\sqrt{x}$}, \type {xx} becomes \type {×} (times)
94%D but \type {ac} becomes \type {$a c$} (a times c). We're lucky that \type {AC} is
95%D not turned into Alternating Current, but who knows what happens a few years from
96%D now. So, we do support this spaceless mess, but users are warned: best use a
97%D spacy sequence. The extra amount of spaces (at one byte each) an author has to
98%D include in his|/|her active writing time probably stays below the size of one
99%D holiday picture. Another complication is that numbers (in Dutch) use commas instead
100%D of periods, but vectors use commas as well. We also hav esome different names for
101%D functions which then can conflict with the expectations about collapsed variables.
102%D
103%D It must be noted that simplified encodings (that seem to be the fashion today)
104%D can demand from applications to apply fuzzy logic to make something work out
105%D well. Because we have sequential data that gets rendered, sometimes wrong input
106%D gets obscured simply by the rendering: like the comma's in numbers as well as
107%D for separators (depending on space usage), or plain wrong symbols that somehow
108%D get a representation anyway. This in itself is more a side effect of trying to
109%D use the simplified encoding without applying rules (in the input) or to use it
110%D beyong its intended usage, which then of course can lead to adapted parsers and
111%D catches that themselves trigger further abuse. Imagine that instead of developing
112%D new cars, planes, space ships, mobile phones, computers we would have adapted
113%D horse cars, kites, firework, old fashioned phones and mechanical calculators in a
114%D similar way: patch upon patch of traditional means for sure would not have
115%D worked. So, when you use \ASCIIMATH\ best check immediately how it gets rendered
116%D in the browser as well as on paper. And be prepared to check the more complex
117%D code in the future again. We don't offer any guarantees but of course will try to
118%D keep up.
119%D
120%D In retrospect I sometimes wonder if the energy put into constantly adapting to
121%D the fashion of the day pays off. Probably not. It definitely doesn't pay of.
122%D
123%D More complex crap:
124%D
125%D 1: $x + \stackrel{comment}{\stackrel{\utfchar{"23DE}}{yyyyyyyy}} = y$ \blank
126%D 2: \asciimath{x + stackrel{\utfchar{"23DE}}{yyyyyyyy} = y} \blank
127%D 3: \asciimath{x + stackrel{yyyyyyyy}{\utfchar{"23DE}} = y} \blank
128%D 4: \asciimath{x + stackrel{"comment"}{stackrel{\utfchar{"23DE}}{yyyyyyyy}} = y} \blank
129
130\usemodule[mathml-basics]
131
132\startmodule[asciimath]
133
134\unprotect
135
136\writestatus{asciimath}{beware, this is an experimental (m4all only) module}
137
138%D Hacks:
139
140\unexpanded\def\asciimathoptext        #1{\ifmmode\mathoptext{#1}\else#1\fi}
141\unexpanded\def\asciimathoptexttraced  #1{\ifmmode\mathoptext{\color[darkgreen]{#1}}\else\color[darkgreen]{#1}\fi}
142\unexpanded\def\asciimathstackrel    #1#2{\mathematics{\mathop{\math_relax_limits\mover{#2}{#1}}}}
143\unexpanded\def\asciimathroot        #1#2{\sqrt[#1]{#2}}
144\unexpanded\def\asciimathsqrt          #1{\sqrt{#1}}
145
146%D The core commands:
147
148% if we need to set
149
150\installsetuponlycommandhandler {asciimath} {asciimath}
151
152\appendtoks
153    \ctxlua{moduledata.asciimath.setup {
154        splitmethod = "\asciimathparameter\c!splitmethod",
155        separator   = "\asciimathparameter\c!separator",
156        symbol      = "\asciimathparameter\c!symbol",
157    }}%
158\to \everysetupasciimath
159
160\newtoks\everyasciimath
161
162% \appendtoks
163%     \ignorediscretionaries
164% \to \everyasciimath
165
166\appendtoks
167    \enableautofences
168    \enableautofencemode
169\to \everyasciimath
170
171\unexpanded\def\asciimath
172  {\doifnextoptionalelse\asciimath_yes\asciimath_nop}
173
174\def\asciimath_yes[#1]#2%
175  {\mathematics
176    [#1]%
177    {\the\everyasciimath
178     \clf_justasciimath{\detokenize\expandafter{\normalexpanded{#2}}}}}
179
180\def\asciimath_nop#1%
181  {\mathematics
182    {\the\everyasciimath
183     \clf_justasciimath{\detokenize\expandafter{\normalexpanded{#1}}}}}
184
185% \unexpanded\def\xmlasciimath
186%   {\clf_xmlasciimath}
187
188\unexpanded\def\xmlasciimath
189  {\doifnextoptionalelse\xmlasciimath_yes\xmlasciimath_nop}
190
191\def\xmlasciimath_yes[#1]#2%
192  {\mathematics
193    [#1]%
194    {\the\everyasciimath
195     \clf_xmlasciimath{#2}}}
196
197\def\xmlasciimath_nop#1%
198  {\mathematics
199    {\the\everyasciimath
200     \clf_xmlasciimath{#1}}}
201
202\unexpanded\def\ctxmoduleasciimath#1%
203  {\ctxlua{moduledata.asciimath.#1}}
204
205%D Some tracing commands. Using tex commands is 10\% slower that directly piping
206%D from \LUA, but this is non|-|critical code.
207
208\unexpanded\def\ShowAsciiMathLoad  [#1]{\ctxlua{moduledata.asciimath.show.load("#1")}}
209\unexpanded\def\ShowAsciiMathIgnore[#1]{\ctxlua{moduledata.asciimath.show.ignore("#1")}}
210\unexpanded\def\ShowAsciiMathXML   #1#2{\ctxlua{moduledata.asciimath.show.filter("#1","#2")}}
211\unexpanded\def\ShowAsciiMathStats     {\ctxlua{moduledata.asciimath.show.statistics()}}
212\unexpanded\def\ShowAsciiMathMax       {\ctxlua{moduledata.asciimath.show.max()}}
213
214\unexpanded\def\ShowAsciiMathResult#1%
215  {\begingroup
216     \blank
217     % if we are in vmode, we don't get positions i.e. a smaller tuc file
218     \inleft{\ttbf#1\hfill\ctxlua{moduledata.asciimath.show.count(#1,true)}}%
219     \dontleavehmode
220     \begingroup
221       \ttbf
222       \ctxlua{moduledata.asciimath.show.files(#1)}
223     \endgroup
224     \blank[medium,samepage]
225     \startcolor[darkblue]
226     \ctxlua{moduledata.asciimath.show.input(#1,true)}
227     \stopcolor
228     \blank[medium,samepage]
229     \doifmode{asciimath:show:dirty} {
230        \dorecurse{\ctxlua{moduledata.asciimath.show.nofdirty(#1)}} {
231          \ctxlua{moduledata.asciimath.show.dirty(\recurselevel,true)}
232          \blank[medium,samepage]
233        }
234     }
235     \ctxlua{moduledata.asciimath.show.result(#1)}
236     \blank
237   \endgroup}
238
239\unexpanded\def\ShowAsciiMathStart
240  {\begingroup
241   \let\asciimathoptext\asciimathoptexttraced
242   \setuptyping[\v!buffer][\c!before=,\c!after=]
243   \setupmargindata[\v!left][\c!style=]}
244
245\unexpanded\def\ShowAsciiMathStop
246  {\endgroup}
247
248\unexpanded\def\ShowAsciiMath
249  {\dodoubleempty\doShowAsciiMath}
250
251\unexpanded\def\doShowAsciiMath[#1][#2]%
252  {\iffirstargument
253     \ShowAsciiMathStart
254     \ShowAsciiMathLoad[#1]
255     \ifsecondargument
256       \ShowAsciiMathIgnore[#2]
257     \fi
258     \dorecurse{\ShowAsciiMathMax}{\ShowAsciiMathResult\recurselevel}
259     \page
260     \ShowAsciiMathStats
261     \ShowAsciiMathStop
262  \fi}
263
264\unexpanded\def\xmlShowAsciiMath#1#2%
265  {\iffirstargument
266     \ShowAsciiMathStart
267     \ShowAsciiMathXML{#1}{#2}%
268     \dorecurse{\ShowAsciiMathMax}{\ShowAsciiMathResult\recurselevel}
269     \page
270     \ShowAsciiMathStats
271     \ShowAsciiMathStop
272  \fi}
273
274\unexpanded\def\ShowAsciiMathSave
275  {\dosingleempty\doShowAsciiMathSave}
276
277\unexpanded\def\doShowAsciiMathSave[#1]%
278  {\ctxlua{moduledata.asciimath.show.save("#1")}}
279
280\protect
281
282\startsetups asciimath:layout
283
284    \setupbodyfont
285    % [pagella,10pt]
286      [dejavu,10pt]
287
288    \setuplayout
289      [backspace=35mm,
290       leftmargin=20mm,
291       rightmargindistance=0pt,
292       leftmargindistance=5mm,
293       cutspace=1cm,
294       topspace=1cm,
295       bottomspace=1cm,
296       width=middle,
297       height=middle,
298       header=0cm,
299       footer=1cm]
300
301    \setupheadertexts
302      []
303
304    \setupfootertexts
305      [\currentdate][\pagenumber]
306
307    \setupalign
308      [flushleft,verytolerant,stretch]
309
310    \dontcomplain
311
312\stopsetups
313
314\stopmodule
315
316\continueifinputfile{x-asciimath.mkiv}
317
318%D This will become an extra.
319
320\starttext
321
322\setups[asciimath:layout]
323
324% \enabletrackers[modules.asciimath.mapping]
325% \enabletrackers[modules.asciimath.detail]
326
327% \starttext
328%     \enablemode[asciimath:show:dirty]
329%     \ShowAsciiMath[e:/temporary/asciimath/*.xml]
330%   % \ShowAsciiMathSave[e:/temporary/asciimath/asciimath.lua]
331% \stoptext
332
333\subject{Some tests}
334
335% \unexpanded\def\MyAsciiMath#1{\startformula\asciimath{#1}\stopformula}
336%
337% \startlines
338% \MyAsciiMath{x^2 / 10 // z_12^34 / 20}
339% \MyAsciiMath{{:{:x^2:} / 10:} // {:{:z_12^34 :} / 20:}}
340% \MyAsciiMath{x^2+y_1+z_12^34}
341% \MyAsciiMath{sin^-1(x)}
342% \MyAsciiMath{d/dx f(x)=lim_(h->0) (f(x+h)-f(x))/h}
343% \MyAsciiMath{f(x)=sum_(n=0)^oo(f^((n))(a))/(n!)(x-a)^n}
344% \MyAsciiMath{int_0^1 f(x)dx}
345% \MyAsciiMath{int^1_0 f(x)dx}
346% \MyAsciiMath{a//b}
347% \MyAsciiMath{a//\alpha}
348% \MyAsciiMath{(a/b)/(d/c)}
349% \MyAsciiMath{((a*b))/(d/c)}
350% \MyAsciiMath{[[a,b],[c,d]]((n),(k))}
351% \MyAsciiMath{1/x={(1,text{if } x!=0),(text{undefined},if x=0):}}
352% \MyAsciiMath{{ (1,2), (x,(x + text(x))) }}
353% \MyAsciiMath{{(1,2),(x,(x+text(x))),(x,text(x))}}
354% \MyAsciiMath{{(1,2),(x,(x+text(x))),(x,x text(x))}}
355% \MyAsciiMath{{(1,2/2),(x,(x+x^22+sqrt(xx))),(x,x text(xyz))}}
356% \MyAsciiMath{{(1,2/2),(x,(x+x^22+sqrt(xx))),(x,text(xyz)+1+text(hans))}}
357% \MyAsciiMath{<<a,b>> text{and} {:(x,y),(u,v):}}
358% \MyAsciiMath{(a,b] = {x text(in) RR | a < x <= b}}
359% \MyAsciiMath{a/b / c/d = (a * d) / (b * d) / (b * c) / (b * d) = (a * d) / (b * c)}
360% \MyAsciiMath{ (a/b) // (c/d) =  ( (a * d) / (b * d) ) // ( (b * c) / (b * d) ) = (a * d) / (b * c)}
361% \MyAsciiMath{sin(x+1)_3^2/b / c/d}
362% \MyAsciiMath{{:{:sin(x+1)_3^2:}/b:} / {:c/d:}}
363% \MyAsciiMath{cos(a) + sin(x+1)_3^2/b / c/d = (a * d) / (b * d) / (b * c) / (b * d) = (a * d) / (b * c)}
364% \MyAsciiMath{S_(11)}
365% \MyAsciiMath{f(x)}
366% \MyAsciiMath{sin(x)}
367% \MyAsciiMath{sin(x+1)}
368% \MyAsciiMath{sin^-1(x)}
369% \MyAsciiMath{sin(2x)}
370% \MyAsciiMath{a_2^2}
371% \MyAsciiMath{( (S_(11),S_(12),S_(1n)),(vdots,ddots,vdots),(S_(m1),S_(m2),S_(mn)) ]}
372% \MyAsciiMath{frac a b}
373% \MyAsciiMath{sin(x)/2 // cos(x)/pi}
374% \MyAsciiMath{a/13 // c/d}
375% \MyAsciiMath{a/b // c/d}
376% \MyAsciiMath{x}
377% \MyAsciiMath{x^2}
378% \MyAsciiMath{sqrt x}
379% \MyAsciiMath{sqrt (x)}
380% \MyAsciiMath{root 2 x}
381% \MyAsciiMath{x+x}
382% \MyAsciiMath{x/3}
383% \MyAsciiMath{x^2 / 10}
384% \MyAsciiMath{x^2 / 10 // z_12^34 / 20}
385% \MyAsciiMath{a^23}
386% \MyAsciiMath{a^{:b^23:}+3x}
387% \MyAsciiMath{a/b / c/d}
388% \MyAsciiMath{sin(x)/b / c/d}
389% \MyAsciiMath{sin(x)/b // c/d}
390% \MyAsciiMath{a/b / c/d = (a * d) / (b * d) / (b * c) / (b * d) = (a * d) / (b * c) }
391% \MyAsciiMath{{:{:x^2:} / 10:} // {:{:z_12^34 :} / 20:}}
392% \MyAsciiMath{x^2+y_1+z_12^34}
393% \MyAsciiMath{sin^-1(x)}
394% \MyAsciiMath{d/dx f(x)=lim_(h->0) (f(x+h)-f(x))/h}
395% \MyAsciiMath{f(x)=sum_(n=0)^oo(f^((n))(a))/(n!)(x-a)^n}
396% \MyAsciiMath{int_0^1 f(x)dx}
397% \MyAsciiMath{int^1_0 f(x)dx}
398% \MyAsciiMath{2x}
399% \MyAsciiMath{a//b}
400% \MyAsciiMath{a//\alpha}
401% \MyAsciiMath{(a/b)/(d/c)}
402% \MyAsciiMath{((a*b))/(d/c)}
403% \MyAsciiMath{[[a,b],[c,d]]((n),(k))}
404% \MyAsciiMath{1/x={(1,text{if } x!=0),(text{undefined},if x=0):}}
405% \MyAsciiMath{{ (1,2), (x,(x + text(x))) }}
406% \MyAsciiMath{{(1,2),(x,(x+text(x))),(x,text(x))}}
407% \MyAsciiMath{{(1,2),(x,(x+text(x))),(x,x text(x))}}
408% \MyAsciiMath{{(1,2/2),(x,(x+x^22+sqrt(xx))),(x,x text(xyz))}}
409% \MyAsciiMath{{(1,2/2),(x,(x+x^22+sqrt(xx))),(x,text(xyz)+1+text(hans))}}
410% \MyAsciiMath{<<a,b>> text{and} {:(x,y),(u,v):}}
411% \MyAsciiMath{(a,b] = {x text(in) RR | a < x <= b}}
412% \MyAsciiMath{x^-2}
413% \MyAsciiMath{x^2(x-1/16)=0}
414% \MyAsciiMath{y= ((1/4)) ^x}
415% \MyAsciiMath{log (0,002) / (log(1/4))}
416% \MyAsciiMath{x=ax+b \ oeps}
417% \MyAsciiMath{x=\ ^ (1/4) log(x)}
418% \MyAsciiMath{x=\ ^ (1/4) log(0 ,002 )= log(0,002) / (log(1/4))}
419% \MyAsciiMath{x^ (-1 1/2) =1/x^ (1 1/2)=1/ (x^1*x^ (1/2)) =1/ (xsqrt(x))}
420% \MyAsciiMath{x^2(10 -x)&gt;2 x^2}
421% \MyAsciiMath{x^4&gt;x}
422% \stoplines
423
424% \setupasciimath[splitmethod=3,symbol={{,}}]
425%
426% \startlines
427% \asciimath{sqrt 1}
428% \asciimath{sqrt 1.2}
429% \asciimath{sqrt 1.2}
430% \asciimath{1}
431% \asciimath{12}
432% \asciimath{123}
433% \asciimath{1234}
434% \asciimath{12345}
435% \asciimath{123456}
436% \asciimath{1234567}
437% \asciimath{12345678}
438% \asciimath{123456789}
439% \asciimath{1.1}
440% \asciimath{12.12}
441% \asciimath{1234.123}
442% \asciimath{1234.1234}
443% \asciimath{12345.1234}
444% \asciimath{1234.12345}
445% \asciimath{12345.12345}
446% \asciimath{123456.123456}
447% \asciimath{1234567.1234567}
448% \asciimath{12345678.12345678}
449% \asciimath{123456789.123456789}
450% \asciimath{0.1234}
451% \asciimath{1234.0}
452% \asciimath{1234.00}
453% \asciimath{0.123456789}
454% \stoplines
455
456% \definemixedcolumns[asciimath][n=3,balance=yes]
457%
458% \startluacode
459%     local asciimath = moduledata.asciimath
460%     local variables = { "w", "x", "y", "z", "p", "q", "r" }
461%     local constants = { "a", "b", "c" }
462%     local functions = { "g", "h", "i" }
463%     local iterators = { "i", "j", "k" }
464%     local vectors   = { "A", "B", "C", "D", "E", "P", "Q", "R" }
465%     local reserved  = { }
466%     local reserved  = {
467%      -- "vdots","ddots","oint",
468%         "grad", "prod", "prop", "sube", "supe", "sum",
469%         "vvv", "nnn", "uuu", "sub", "sup",
470%         "iff", "int", "del",
471%         "sinh", "cosh", "tanh", "sin", "cos", "tan", "csc", "sec", "cot",
472%         "atan", "asin", "acos", "arctan", "arcsin", "arccos",
473%         "log", "ln", "det", "lim", "mod", "gcd", -- "lcm",
474%         "min", "max",
475%         "xx", "in", "ox", "vv", "nn", "uu", "oo",  "bb",
476%         "not", "and", "or", "if",
477%         "AA", "EE", "TT",
478%         "sqrt", "root", "frac", "stackrel",
479%         "hat", "overbar", "underline", "vec",
480%         "dx", "dy", "dz",
481%     }
482%     for c=1,#constants do
483%         for r=1,#reserved do
484%             context.startmixedcolumns { "asciimath" }
485%             for v1=1,#variables do
486%                 for v2=1,#variables do
487%                     local str = constants[c] .. variables[v1] .. reserved[r] .. variables[v2]
488%                     context.type(str)
489%                     context.quad()
490%                     commands.asciimath(str)
491%                     context.par()
492%                 end
493%             end
494%             context.stopmixedcolumns()
495%             context.blank()
496%         end
497%     end
498% \stopluacode
499
500\stoptext
501