Field Specialization: Just another JIT?
In an earlier blog post, we discussed how field specialization creates optimized code at DBMS runtime. There is another widely used technology, Just-in-Time (JIT) compilation, that also performs code optimization at runtime. This blog entry looks at how these two approaches to runtime optimization resemble, and differ from, each other.
JIT compilers are used alongside interpreters, for languages like Java, JavaScript, and Python, to speed up interpreted byte code. The primary objective here is to remove the overhead of bytecode interpretation by compiling the bytecode into native code at runtime. Along the way, JIT compilers will also do other optimizations, e.g., register allocation, to remove inefficiencies arising out of the original bytecode representation of the program. Figure 1 illustrates the flow from source to execution in the presence of JIT compilation.
Figure 1: JIT Compilation
JIT compilation and field specialization share some characteristics arising out of the fact that they both perform optimizations and generate code at runtime. For example, both contain mechanisms to create code dynamically, link this code into the rest of the program (which requires some of the functionality of a linker), transfer control to the newly created code, and maybe garbage-collect the dynamically created code when appropriate. Once we go beyond this sort of dynamic code generation infrastructure, however, the two turn out to be quite different.
Field specialization differs from JIT compilation in two fundamental ways. First and most important, while JIT compilation is applied to interpreted bytecode, field specialization is applied to native code. The primary objective of field specialization is to speed up software -- including code that may have already been extensively optimized by the compiler -- by taking advantage of runtime invariants that are unavailable at compile time. Since JIT compilation is applied only to the code that is interpreted, it does nothing to speed up the interpreter itself. Field specialization can speed up compiled and optimized code.
A second big difference between field specialization and JIT compilation is in when the analysis to perform the optimization is performed, and consequently, the kinds of optimizations available. In the JIT compilation model, we have an interpreter plus some unknown bytecode programs that have to be optimized. It therefore requires additional runtime infrastructure to monitor the bytecode programs as they execute, find hot spots, and identify optimization opportunities. Since the cost of execution monitoring and code optimization count against the benefits of optimization, JIT compilation typically restricts itself to simple and lightweight optimizations (though some JIT compilers may, in some situations, apply medium-weight optimizations). By contrast, the field specialization model assumes that the program being specialized (i.e., the DBMS) is known ahead of time and so hot spots and applicable optimizations can be determined ahead of time, and so those activities don't incur a runtime cost. This allows field specialization to apply heavier-weight analyses and optimizations that would be too expensive to be practical in a JIT compiler.
A third difference between these two optimization approaches is that in modern JIT compilers, the optimizations typically progress through several levels of sophistication, while in field specialization the optimizations are applied in a single step. As mentioned above, the JIT compiler doesn’t have advance knowledge of the execution behavior of the program it’s compiling, and so has no way to tell, ahead of time, whether a particular method will be invoked only a few times and so is only worth the cost of a few lightweight optimizations of limited benefit, or whether it will be executed a large number of times and so is worth a much larger optimization effort. JIT compilers deal with this problem by moving the code it is optimizing through a number of different optimization levels, with the higher levels corresponding to higher-cost higher-benefit optimizations. This problem doesn’t arise in field specialization since the execution characteristics of the code being specialized are known ahead of time. As a result, field specialization does not need the additional runtime machinery (and cost!) to keep track of execution counts and optimization levels in JIT compilers.
Figure 2 illustrates the flow from source to execution in the presence of field specialization.
Figure 2: Field Specialization
Notice, in the first figure, how much larger and more complex the runtime environment needs to be in the case of JIT compilation as compared to that for field specialization in the second figure. In field specialization, much of the machinery needed for runtime code specialization is moved to the (static) field-specializing compiler, which makes it possible to have a much simpler, more lightweight, and more efficient runtime system.