Source Browser ?

1% language=us
2
3\environment mk-environment
4
5\usemodule[timing]
6
7\startcomponent mk-memory
8
9\chapter{Collecting garbage}
10
11We use the \type {mk.tex} document for testing and because it keeps track of
12how \LUATEX\ evolves. As a result it has some uncommon characteristics. For
13instance, you can see increments in memory usage at points where we
14load fonts: the chapters on Zapfino, Arabic and CJK (unfinished). This memory
15is not freed because the font memory is used permanently. In the following
16graphic, the red line is the memory consumption of \LUATEX\ for the current
17version of \type {mk.tex}. The blue line is the runtime per page.
18
19\ShowNamedUsage{mk-luatex-progress}{luastate_bytes}{elapsed_time}
20
21At the moment of writing this Taco has optimized the \LUATEX\ code base and
22I have added dynamic feature support to the \MKIV\ and optimized much of the
23critical \LUA\ code. At the time of writing this (December 23, 2007),
24\type {mk.tex} counted 142 pages. Our rather aggressive optimizations brought
25down runtime from about 29 seconds to under 16 seconds. By sharing as much font
26data as possible at the \LUA\ end (at the cost of a more complex implementation)
27the memory consumption of huge fonts was brought down to a level where
28a somewhat \quote {older} computer with 512 MB memory could also cope with
29\MKIV. Keep in mind that some fonts are just real big. Eventually we may decide
30to use a more compact table model for passing \OPENTYPE\ fonts to \LUA, but this
31will not happen in 2007.
32
33The following tests show when \LUA's garbage collector becomes active. The blue
34spike shows that some extra time is spent on this initially. After that garbage
35more garbage is collected, which makes the time spent per page slightly higher.
36
37\starttyping
38\usemodule[timing] \starttext \dorecurse{2000}{
39    \input tufte \par \input tufte \par \input tufte \page
40} \stoptext
41\stoptyping
42
43\ShowNamedUsage{mk-timing-1-luatex-progress}{luastate_bytes}{elapsed_time}
44
45The maximum memory footprint is somewhat misleading because \LUA\ reserves more
46than needed. As discussed in an earlier chapter, it is possible to tweak to control
47memory management somewhat, but eventually we decided that it does not make much sense
48to divert from the default settings.
49
50\starttyping
51\usemodule[timing] \starttext \dorecurse{2000}{
52    \input tufte \par \input tufte \par \input tufte \par
53} \stoptext
54\stoptyping
55
56\ShowNamedUsage{mk-timing-2-luatex-progress}{luastate_bytes}{elapsed_time}
57
58The last example of this set does not load files, but stores the text in
59a macro. This is faster, although not that mich because the operating system
60caches the file and there is not \UTF\ collapsing needed for this file.
61
62\starttyping
63\usemodule[timing] \starttext \dorecurse{2000}{
64    \tufte \par \tufte \par \tufte \par
65} \stoptext
66\stoptyping
67
68\ShowNamedUsage{mk-timing-3-luatex-progress}{luastate_bytes}{elapsed_time}
69
70There are subtle differences in memory usage between the examples and eventually
71test like these will permit us to optimize the code even further. For the record:
72the first test runs in 39.5 seconds, the second on in 36.5 seconds and the last one
73only takes 31.5 seconds (all in batch mode).
74
75Keep in mind that these quotes in \type {tufte.tex} are just test samples, and
76not that realistic in everyday documents. On the other hand, these tests involve
77the usual font loading, node processing, attribute handling etc. They provide a
78decent baseline.
79
80Another document that we use for testing functionality and performance is the
81reference manual. The preliminary beta~2 version gives the following statistics.
82
83\ShowNamedUsage{luatexref-t-luatex-progress-runtime}{luastate_bytes}{elapsed_time}
84
85The previous graphic shows the statistics of a run with runtime \METAPOST\ graphics
86enabled. This means that, because each pagenumber comes with a graphic, for each
87page \METAPOST\ is called. The speed of this call is heavily influenced by the
88\METAPOST\ startup time, which in turn (in a windows platform) is influences by the
89initialization time of the \KPSE\ library. Technically the call time can near zero
90but this demands sharing libraries and databases. Anyhow, we're moving towards an
91embedded \METAPOST\ library anyway, and the next graphic shows what will happen then.
92Here we run \CONTEXT\ in delayed \METAPOST\ mode: graphics are collected and processed
93between runs. Where the runtime variant takes some 45 seconds processing time, the
94intermediate versions takes 15.
95
96\ShowNamedUsage{luatexref-t-luatex-progress-intermediate}{luastate_bytes}{elapsed_time}
97
98In the \type {mk.tex} document we use \TYPEONE\ fonts for the main body of the text and
99load some (huge) \OPENTYPE\ fonts later on. Here we use \OPENTYPE\ fonts exclusively and
100since \CONTEXT\ loads fonts only when needed, you see several spikes in the time per page
101bars and memory consumption quickly becomes stable. Interesting is that contrary to the
102\type {tufte.tex} samples, memory usage is quite stable. Here we don't have a memory
103sawtooth and no garbage collection spikes.
104
105The previous graphics combine \LUA\ memory consumption with time spent per page. The
106following graphics show variants of this. The graphics concern this document
107(\type{mk.tex}). Again, the blue lines represent the runtime per page.
108
109\ShowMemoryUsage{mk-luatex-progress}
110
111In \LUATEX\ node memory management is rewritten. Contrary to what you may expect, node memory
112consumption is not that large. Pages seldom contain more than 5000 nodes, although
113extensive use of attributes can easily duplicate this. Node usage in this documents is
114as follows.
115
116\ShowNodeUsage{mk-luatex-progress}
117
118If node memory usage stays high, i.e.\ is not reclaimed, this can be an indication of
119a memory leak. In the December 2007 beta version there is such a leak in math subformulas,
120something that will be resolved when math node processing is opened up. The current \MKIV\
121code cleans up most of its temporary data. We do so, because it permits us to keep an eye
122on unwanted memory leaks. When writing this chapter, some of the peaks in the graphics
123coincided with peaks in the runtime per page, which is no surprise.
124
125If you want to run such tests yourself, you need to load a module at startup:
126
127\starttyping
128\usemodule[timing]
129\stoptyping
130
131The graphics can be generated with:
132
133\starttyping
134\ShowUsage      {optional filename}
135\ShowNamedUsage {optional filename}{red graphic}{blue graphic}
136\ShowMemoryUsage{optional filename}
137\ShowNodeUsage  {optional filename}
138\stoptyping
139
140(This interface may change.)
141
142\stopcomponent
143