Tim Bradshaw <tfb+google@tfeb.org> wrote:
+---------------
| rpw3@rpw3.org (Rob Warnock) wrote:
| > So, yes, I guess I'm disagreeing with you, Tim.
|
| And that's because it turns out I was wrong, even for Solaris! Sorry!
| Solaris computes the load average the way it has always been done (at
| least back to BSD 4?), which is based on number of processes in the
| run queue (so the way you think it should be done). Linux (it seems)
| uses some function of run queue & blocked for I/O queue. So you're
| right, and I was wrong.
+---------------
Not a problem. The only reason I was being pretty stubborn about it
was that Linux is the *only* place I've ever seen these silly 100+
load averages on essentially idle systems!!
+---------------
| What led me astray was that for a machine which is paging heavily,
| then processes are in the run queue when in fact they are waiting for
| disk (because they're waiting for memory which has been paged out).
| It's *this* case where you can get huge load averages but tiny CPU
| utilization. Obviously that doesn't happen as much as it did, but I
| remember we used to see this a lot for Lisps in GC on the BSD machines
| where I first saw Unix (30 or 30MB images on a machine with 2MB of
| real memory, which did very well except during GC).
+---------------
Hmmm... O.k., I suppose counting processes in the "Page-In" wait queue
[as TOPS-10 called it] as part of "load" isn't all that unreasonable.
I'll have to go look at the current FreeBSD code again and see if they
still do that.
+---------------
| And now I think I also remember what lead me even further astray. If a
| process uses mmap to map a large file into its address apce, and then
| accesses the resulting array, what does that count as? If you're
| naive it looks like paging and therefore this process looks like it's
| runnable. But actually it is blocked for I/O: in particular it's not
| starving for memory. Except, maybe it is, if the file it is paging
| from is its own executable's text segment.
+---------------
Yeah, that one's a bit borderline, particularly since with the
proliferation of software that's based on some language VM
[hint, hint] the definition of "executable" is a bit broad.
+---------------
| (This next bit is based on memory and may be wrong in places.)
| Solaris uses mapped files pervasively, to the extent that normal memory
| allocation is treated as mapping a file, which file happens to sit in
| a special filesystem which is backed by swap. For a long time this
| gave rise to all sorts of problems, one of which was that systems
| which were actually I/O bound could show up as having high load averages.
+---------------
Ah, yezz! Now that you mention it, I do seem to recall something
about that with some version of Sun systems.
Anyway, whether processes waiting to be re-paged-in get counted in
"load" or not, I think we can all probably agree that current Linux
does it wrong when an idle system has a 100+ load! ;-}
-Rob
-----
Rob Warnock <rpw3@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607