Chapter 3. Efficiency

FIXME: The material in the CMU CL manual about getting good performance from the compiler should be reviewed, reformatted in DocBook, lightly edited for SBCL, and substituted into this manual. In the meantime, the original CMU CL manual is still 95+% correct for the SBCL version of the Python compiler. See the sections

• Advanced Compiler Use and Efficiency Hints

• Advanced Compiler Introduction

• More About Types in Python

• Type Inference

• Source Optimization

• Tail Recursion

• Local Call

• Block Compilation

• Inline Expansion

• Object Representation

• Numbers

• General Efficiency Hints

• Efficiency Notes

Besides this information from the CMU CL manual, there are a few other points to keep in mind.

• The CMU CL manual doesn't seem to state it explicitly, but Python has a mental block about type inference when assignment is involved. Python is very aggressive and clever about inferring the types of values bound with let, let*, inline function call, and so forth. However, it's much more passive and dumb about inferring the types of values assigned with setq, setf, and friends. It would be nice to fix this, but in the meantime don't expect that just because it's very smart about types in most respects it will be smart about types involved in assignments. (This doesn't affect its ability to benefit from explicit type declarations involving the assigned variables, only its ability to get by without explicit type declarations.)

• Since the time the CMU CL manual was written, CMU CL (and thus SBCL) has gotten a generational garbage collector. This means that there are some efficiency implications of various patterns of memory usage which aren't discussed in the CMU CL manual. (Some new material should be written about this.)

• SBCL has some important known efficiency problems. Perhaps the most important are

  • There is no support for the ANSI dynamic-extent declaration, not even for closures or &rest lists.

  • The garbage collector is not particularly efficient.

  • Various aspects of the PCL implementation of CLOS are more inefficient than necessary.

Finally, note that Common Lisp defines many constructs which, in the infamous phrase, "could be compiled efficiently by a sufficiently smart compiler". The phrase is infamous because making a compiler which actually is sufficiently smart to find all these optimizations systematically is well beyond the state of the art of current compiler technology. Instead, they're optimized on a case-by-case basis by hand-written code, or not optimized at all if the appropriate case hasn't been hand-coded. Some cases where no such hand-coding has been done as of SBCL version 0.6.3 include

• (reduce #'f x) where the type of x is known at compile time

• various bit vector operations, e.g. (position 0 some-bit-vector)

If your system's performance is suffering because of some construct which could in principle be compiled efficiently, but which the SBCL compiler can't in practice compile efficiently, consider writing a patch to the compiler and submitting it for inclusion in the main sources. Such code is often reasonably straightforward to write; search the sources for the string "deftransform" to find many examples (some straightforward, some less so).