Subject: Re: Writing shared libraries using Lisp
From: rpw3@rpw3.org (Rob Warnock)
Date: Wed, 13 Jul 2005 04:25:54 -0500
Newsgroups: comp.lang.lisp
Message-ID: <0rSdnV-BTum_Q0nfRVn-jA@speakeasy.net>
Ulrich Hobelmann  <u.hobelmann@web.de> wrote:
+---------------
| Edi Weitz wrote:
| > Did you see the CMUCL startup time in Rob's posting?
| 
| Yes, way shorter than the ECL time.  But assuming that he wanted 
| to use ECL for some reason, not CMUCL, a server is the solution to 
| the startup problem.
+---------------

Actually, I *do* use CMUCL in "server mode" for all my heavy web
lifting [via a mod_lisp-like C-based CGI, which could trivially
be replaced with the real mod_lisp].

But I also like to use CL as a generic scripting language, whether
for CGI or miscellaneous "shell scripting". And in both those
applications, one wants simple scripts to run in negligible human
time, preferably under 100ms [which to my taste is a "just-noticable
difference" (1 JND) in response time]. Perl does that; /bin/sh does
that; and so do both CLISP & CMUCL.

Background note: For some time, I was using both CLISP & CMUCL for
"scripting", since while I wanted the performance of a native-compiled
implementation I was concerned about the startup time with CMUCL's
"large image". But to my surprise, I found that for small scripts
CMUCL's startup time was slightly *less* that CLISP's on all of the
platforms I use [various Linuxes and FreeBSDs on P3s, Athlons, and
Opterons (in 32-bit mode)]. So for any operating system which caches
mmap'd files in the page pool [which includes all the ones I use],
the "large image" issue is now a non-issue for me.

+---------------
| In this thread (or some other thread the last couple of days) 
| people were also discussing Lisp image sizes, and that some don't 
| like the idea of having a Lisp image around for every single shell 
| script.  Small ECL executable files are a solution to this at 
| least, but with the startup drawback.
+---------------

Hunh?!? As I said a couple of weeks before this, you don't *need*
a separate "Lisp image around for every single shell script"!!
Just pre-load all of your heavily-used libraries, packages, modules,
and whatever into *one* Lisp image, installed as the default, which
gets shared by all the "scripts" [interpreted Lisp] that reference it!
The one common image gets shared/cached, and all your scripts use the
same one.

In my experience, you can throw a *lot* of additional packages
into a CMUCL image before it gets much bigger than the base image. ;-}
E.g., all of the CGI, SQL, HTML, and HTTP forms stuff I use adds
less than 10% to the size of a base CMUCL image.

Again, it doesn't matter whether it's a "libecl.so" or a "lisp.core",
if it is mmap'd into memory [MAP_PRIVATE, of course], then the file,
whether DSO, DLL, or "image", will get cached *and* shared, so the
size of the library or the image in the filesystem is pretty much a
non-problem.

No, the problem I was talking about is that one implementation takes
*way* longer than a human JND time (770%) to start up, while two others
take *way* less than a human JND time to start up (15-20%). So which
ones are people likely to want to use for interactive "shell scripting"
on a daily basis? The one that is clearly making you wait or the ones
that are "instantaneous", like most of the other shell commands you use?

+---------------
| If you have a virtual binary directory, which is just a Lisp 
| server that loads your scripts, you don't have startup, and you 
| only need one image.
+---------------

Uh... How do you *securely* give firewalled access to dozens of users
on a system who want to use Lisp for their *own* "shell scripts"?
Run a server per user? No thanks...


-Rob

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607