Subject: Re: Mars Rover failure
From: rpw3@rpw3.org (Rob Warnock)
Date: Thu, 29 Jan 2004 05:39:51 -0600
Newsgroups: comp.lang.lisp
Message-ID: <kAKdnXS5dIuabIXdXTWc-w@speakeasy.net>
Paul Tarvydas  <tarvydas@allstream.net> wrote:
+---------------
| http://space.com/missionlaunches/spirit_relay_040124.html
| "There is a growing feeling here that the robot may have been taxed too
| much that is, too many "do this, do that" instructions were sent. That
| multi-tasking could have sparked the problem, according to sources here.
+---------------

More recent news:

	<URL:http://spaceflightnow.com/mars/mera/040126spirit.html>

suggests that it was a matter of the flash (EEROM) filesystem metadata
(or maybe filesystem cache?) overflowing RAM, due to the length of time
that the system had been running and the amount of data captured:

	It is now believed that the rover's flash memory had become
	so full of files that the craft couldn't manage all of the
	information stored aboard. Spirit bogged down because it didn't
	have enough random access memory, or RAM, to handle the current
	amount of files in the flash -- including data recorded during
	its cruise from Earth to Mars and the 18 days of operations on
	the red planet's surface. 
	...
	And in that we realized that we had this reset problem. Based
	on just kind of the hunch of our lead software architect, he
	believed that the problem was probably associated with the mounting
	of flash and initialization. There is a hardware command that
	we can send that bypasses the software where we can actually
	tell the hardware to not allow us to mount flash on initialization.
	When we the next day actually sent the command to do that, software
	initialized normally and was behaving like the software that we
	had always known. It was a fantastic moment. 
	...
	Right now, our most likely candidate for the issue has been
	narrowed down a little bit. It is really an issue with the
	file system in flash. Essentially, the amount of space required
	in RAM to manage all of the files we have in flash is apparently
	more than we initially anticipated.
	...
	Tomorrow, we are might try to access flash and do a little bit
	of a health check on it. The next day we might try to delete
	some files to see if our hunch is correct that it's really due
	to the number of files that we are trying to manage on the flash
	file system.
	...


-Rob

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607