Subject: Re: Storing macro dependencies in the in-memory function object (was Re: When to recompile/reeval?) From: Erik Naggum <erik@naggum.no> Date: 20 Oct 2002 13:51:01 +0000 Newsgroups: comp.lang.lisp Message-ID: <3244110661811327@naggum.no> * Tim Bradshaw | Lisp applications which take a long time to recompile should, I guess, | either be doing something very complex in the way of really hairy | macros, say, or be huge. Just to clarify: More than 100 ms is a long time if you have to do it in order to test-run a macro. That is sufficient to interrupt your flow of thinking and working. When I develop with Allegro CL and Emacs, I tell it not to compile with (setq fi:lisp-evals-always-compile nil) on the Emacs side, and I happily go about M-C-x'ing definitions, run stuff in the background, and test code. Code that matures is saved, compiled, and loaded, but code in development generally runs interpreted. I really like that Allegro CL makes this so streamlined and painless. | My system of ~22,000 lines takes 13 seconds to build from cold, or ~26 | seconds to build + dump an image. And I think Unix `ls´ takes a long time to run on directories with 1000 files so Emacs `dired´ is something other than instantaneous. | Sometimes if I change a major macro I just blow everything away and | rebuild, because I know my system declarations have bugs. Precisely. And you cannot do this every time you change a macro. I have always developed mini-languages, even when I worked mainly in C, and macros give me the opportunity to reprogram things and have code that reasons about the code I write. Macros do a lot of the work in my code. When I want to change something important, it is always in the macros, and minor changes can have major effects. Neat protocol descriptions are usually reworked by macros to produce reems of code that is extremely hard to follow, but very efficient. Perhaps I "overuse" macros, but the ability to run interpreted correctly is essential. If macros were harder to use, or function redefinition did not work, I would not be able to write this kind of code at all. What I would have done then is hard to tell, but that I would have ended up with less intelligent and messier code is virtually guaranteed. E.g., I have been working on and off on a regular expression machine that does as much as possible at compile-time, such as maintain multiple paths through the graph in parallel (breadth-first) instead of backtracking and wasting lots of time if the order of alternatives is not optimal. This came up when I sat down to reimplement the Common Lisp reader: if I could compute the integer value while reading a potential number digit by digit and make other decisions along the way, I could exercise the cache more fully and read only once from uncached memory. Writing code that does this manually is very, very hard, especially if you want to get it right. Writing macros that arrange multiple execution paths in parallel is not easy, but at least it is much easier than to write what they produce. | It annoys me that no such tool exists though, especially as one of the | things the system above does (or did) was to construct these kinds of | dependency relationships for C++/CORBA systems automatically... This may be related to the curse of macros: You cannot in general expect to understand fully what some code would compile to without being the compiler. In inferior languages, the code you write is probably the code the machine will run. In Common Lisp with lots of macros, the code you write is only data for lots of other programs before it becomes code for the compiler to arrange for the machine to run. It takes some getting used to. -- Erik Naggum, Oslo, Norway Act from reason, and failure makes you rethink and study harder. Act from faith, and failure makes you blame someone and push harder.