Jochen Schmidt  <jsc@dataheaven.de> wrote:
+---------------
| This might be a really dumb question but is it possible in any system to 
| allocate a huge uninitialized array (bigger than e. g. physical+swap) and
| use it as a sparse array? I could imagine that the mmu and the 
| memory-management-system of the OS could manage this in similar terms
| like sparse-files are handled in UNIX-Filesystems.
+---------------

In SGI's "Irix" version of Unix, you can dynamically configure
the system with some amount of "virtual swap space", to overcommit
the physically-available swap area. The resulting "logical swap"
space is the sum of vswap & phtsical swap. This is obviously dangerous
(as it can cause deadlock), as warned in the "swap(1)" & "swapctl(2)"
man pages, but it's sometimes is necessary to allow a huge program to
fork when all it's going to do is immediately exec a smaller program,
or allow a program that uses memory sparsely [and does not use the
technique shown later] to run without requiring a lot of swap space
that is not really needed.

Given a enough vswap, one could "allocate" a huge amount of virtual
space in a user program by mmap'ing /dev/zero (which is implicitly
MAP_PRIVATE), and succeed, though "logical swap space" will still
be allocated. The pages in the resulting user virtual address range
will not be initialized or swapped unless they are actually touched.

A better way (if you control the program's source) is to explicitly
defer the allocation of swap space by mmap'ing /dev/zero with the
MAP_AUTORESRV flag. From "mmap(2)":

    The MAP_AUTORESRV flag causes logical swap space to be automatically
    reserved as each page is first referenced with a store operation instead
    of when the mapping is created.  When this flag is used, no logical swap
    space is reserved when the mapping is created.  Therefore, the system
    cannot guarantee that space will be available when needed.  If all the
    logical swap space has been taken by other processes when a page in a
    MAP_AUTORESRV mapping is first stored to, then the process will be sent
    SIGBUS.

Now I don't know if any Lisp implementation for Irix actually uses this 
feature to support sparseness, but it's there (and I know that certain C
programs depend on it)...


-Rob

-----
Rob Warnock, 31-2-510		rpw3@sgi.com
SGI Network Engineering		http://reality.sgi.com/rpw3/
1600 Amphitheatre Pkwy.		Phone: 650-933-1673
Mountain View, CA  94043	PP-ASEL-IA