Subject: Re: Upper limits of CL
From: rpw3@rigden.engr.sgi.com (Rob Warnock)
Date: 20 Jun 2002 10:22:56 GMT
Newsgroups: comp.lang.lisp
Message-ID: <aesae0$2j36i$1@fido.engr.sgi.com>
Erik Naggum  <erik@naggum.net> wrote:
+---------------
| Besides, if you really need gigabyte upon gigabyte of memory
| and the hardware to utilize it, the only thing between you and
| satisfying that need has been money for at least 10 years.
+---------------

Exactly so, see:

	<URL:http://www.sgi.com/origin/3000/>
	<URL:http://www.sgi.com/products/storage/9400.html>

With adequate cash, you can buy a COTS system (do, please!! -- and list
me as a referral!) with 512 CPUs [single system image, not a cluster]
with 1 TB of main RAM [flat address space -- each CPU sees all of it]
and dozens of terabytes of disk. Oh, and measured filesystem read rates
as high as 7.5 GB/s on a single Unix file descriptor.

Unfortunately(?), applications that make effective *use* of all that
horsepower at once in a single program tend to be written in Fortran
(usually with OpenMP or MPI), not Common Lisp.

+---------------
| So pardon my cynical twist, but what are you doing with that
| 20,000x20,000 double-precision floating point matrix you say
| you need to invert _today_?
+---------------

People who buy our systems invert, multiply, add, and diagonalize
such matrices all the time, sometimes even larger ones. Oil & gas
exploration, weather prediction, drug modelling, simulated crash
analysis, computational fludic analysis (CFD) [simulated wind
tunnels], etc., etc., can all make use of that scale of math.

+---------------
| And if you _are_ doing serious stuff of this magnitude, why do you even
| bother with run-of-the-mill Common Lisp implementations on stock hardware?
| Implement your own goddamn Common Lisp system optimized for your hardware
| and your domain-specific language and other needs.
+---------------

I'd agree, in the case of a single-CPU problem. But isn't using
Common Lisp effectively on multiple processors still considered
a "hard" problem? [I'm referring to running a "single program"
that uses large numbers of CPUS, not a large number of separate
communicating Unix processes -- that's easy.] Allocation & garbage
collection alone would seem to be... "interesting".

[If anyone knows of an implementation that can run the mutator
on >25 CPUs while running the garbage collector on, say, ~5 more
CPUs, I'd be very interested in hearing about it...]


-Rob

-----
Rob Warnock, 30-3-510		<rpw3@sgi.com>
SGI Network Engineering		<http://www.rpw3.org/>
1600 Amphitheatre Pkwy.		Phone: 650-933-1673
Mountain View, CA  94043	PP-ASEL-IA

[Note: aaanalyst@sgi.com and zedwatch@sgi.com aren't for humans ]