Artem Baguinski <artm@v2.nl> wrote:
+---------------
| then i've found that useful-function is buggy, what do i do now?
| change the definition in repl window? or in a text window and then
| eval-last-defun and try that out?
+---------------
Yes, but as you've guessed, that probably won't get you very excited.
+---------------
| i guess what i'm looking for is a description of other peoples
| everyday lisp programming routines...
+---------------
O.k., here's where it really wins for me [and I'm sorry for the long
wind-up, but as is typically the case, the advantages don't really show
in the simple cases]:
I'm developing some Lisp-based middleware for a web application -- that is,
there's an Apache web server and a PostgreSQL database and a CMUCL image
in the middle that gets requests from Apache[1] and pokes around in the
database and eventually spits out some HTML via Apache to the browser user.
A typical development session has the following windows open[2]:
- A Netscape browser window.
- An "xterm"[3] to the REPL[4] of the middleware.
- One or more editor windows into the application's Lisp code.
- [Sometimes] Another "xterm" open running "psql", the command-line
interface to the PostgreSQL database.
Now you might not want to do exactly the following in production,
because running the ASDF[5] "LOAD-OP" does add a few milliseconds
to the access even if it has nothing to do, but during development
the following is *terribly* convenient:
% cd /usr/local/apache/htdocs/some/path
% grep -i lisp .htaccess
# sock.cgi is the little CGI trampoline that connects to the Lisp process.
Action lisp-handled-pages /sock.cgi
AddHandler lisp-handled-pages .lhp
% cat foo.lhp
;;; Boilerplate for using ASDF from LHP page.
(flet ((this-page (http-request)
(asdf:operate 'asdf:load-op "foo") ; Ensure up-to-date each time.
;; Must explicitly INTERN, since package may not exist before now.
(funcall (intern "MAIN" "FOO") http-request)))
(lhp-set-page-function #'this-page))
%
Don't worry about all that LHP stuff (it's part of the infrastructure of
the persistent Lisp server), except to note that the very first time the
page <URL://http:my.domain/some/path/foo.lhp> gets accessed, that little
bit of Lisp code gets LOADed and the function #'THIS-PAGE gets registered
as the handler for that URL. Then #'THIS-PAGE will be called, which will
compile & load all of the files in the "FOO" subsystem (via the ASDF call),
and then finally will call FOO::MAIN (which should "do stuff" based on the
data in the HTTP request block and/or the POST data, and emit some HTML).
Now here's the development cycle:
1. Using the Netscape window, access [or later, "Reload"] the page.
(If nothing has changed, this will just call FOO::MAIN.)
2. In the REPL window, note the logging of the web request, and the
results of the ASDF:OPERATE (if any). In the Netscape window, see
if you got the correct HTML results displayed.
3. If there were compile errors, fix them in an editor window, "Save"
the file [but stay in the editor], and go back to #1, which will
automatically (thanks to ASDF) re-do the compile & load (but of only
the sources which changed and the modules that depended on those).
4. If there were logic errors [that is, the output to the browser wasn't
correct], you can edit the source, save it, and go back to #1, which
will recompile whatever's needed and try the access again.
As far as *finding* your problem, you can:
1. Use the REPL window to poke at bits at the last web request, e.g.:
app_srv> (http-request-method *req0*)
"GET"
app_srv> (http-request-self *req0*)
"/~rpw3/foo.lhp"
app_srv> (http-request-xpath *req0*)
"/u/rpw3/public_html/foo.lhp"
app_srv> (let ((h (http-request-handler *req0*)))
(uri-handler-function h))
#<Closure Over ORG.RPW3.CGI.LHP::CACHED-LHP-PAGE-FUNC {48335039}>
app_srv>
2. You can trace/untrace one or more routines within the app, then hit
"Reload" on the browser and watch the call graph flow.
3. You can call the lower-level functions of the application directly
from the REPL, and look at the results. [It is helpful to have a
wrapper function that will "clone" an existing HTTP-request block
and call its handler function, but with output to the REPL stream.]
4. You can hand-code some debugging functions [in another editor window]
and LOAD them into the image, then edit the app's code to call them
at strategic points.
5. If you're getting unexpected conditions being thrown, you can cause
a backtrace to come out on the REPL, and either choose to enter the
debugger or not. [The wrapper mentioned in #3 is helpful here, too.]
And so on, the usual stuff.
The point is that the it gives you "fingers" into almost every aspect
of the running system. The time around the "try/oops!/debug/fix" loop --
just minutes, sometimes just seconds! -- is *VERY* much shorter than
if you had to do a traditional "make" and restart the whole middleware
server each time. I find it *tremendously* productive!
[And all the while, the part of the web site you're *not* working on can
continue to be accessed without interruption, if you need to do that...]
-Rob
p.s. Oh, and did I mention that all of the HTML gets generated using
Lisp macros[6]? Displaying the results of a database query in an HTML
table is really easy when you can intermingle your Lisp code and HTML
using a single syntax (S-exprs) for both:
(let ((column-names (first results))
(rows (rest results))
(stream (http-request-stream request)))
(with-html-output (s stream t)
(:table (:border 1 :cellspacing 0 :cellpadding 1) ; make compact
(:tr ()
(dolist (name column-names)
(htm (:th (:nowrap) name))))
(dolist (row rows)
(htm (:tr ()
(dolist (item row)
(htm (:td (:nowrap) (esc item))))))))))
==========
Footnotes:
[1] Not exactly the same as "mod_lisp" <URL:http://www.cliki.net/mod_lisp>,
but close: a *tiny* C-coded CGI trampoline connects to the CMUCL image
via a Unix-domain socket, then speaks a "mod_lisp"-like protocol to it.
[2] When I was just starting out with using Common Lisp in web apps it was
using classical CGI, in which a whole fresh CLISP or CMUCL image would
get fork/exec'd for each page hit. In that case, I found it useful to
have another "xterm" running "tail -f /usr/local/apache/logs/error_log"
to see the error messages in realtime. Now, though, with condition
handlers wrapped around the HTTP request handling code, you can see
any problems directly on the main REPL window (or in the dribble log
"detachtty"[3] provides).
[3] Well, the "xterm" is *actually* running "attachtty" connected to a
"detachtty" <URL:http://www.cliki.net/detachtty> that started the
Lisp image at boot time, giving REPL access to the detached daemon.
Note that accesses to the server also get logged there, e.g.:
; Feb 24 08:29:45.11 cgi-sock[723]: GET "/lhp/clhs"
; Feb 24 08:29:53.69 cgi-sock[724]: GET "/lhp/clhs?state=quick&pat=db"
and that "detachtty" supports a dribble log file, so that you don't
miss stuff that happens when you weren't watching. [The script I use to
run "attachtty" does a "tail -30" of the dribble file before exec'ing
"attachtty", which provides easy access to some recent history.]
[4] The server supports additional REPLs with a "telnet /path/to/.sock.repl",
which connects via a different Unix-domain socket to the CMUCL image
and starts up another top-level REPL thread, but in practice I've
found I don't really use those much -- the single "attachtty/detachtty"
is usually enough.
[5] <URL:http://www.cliki.net/asdf>
<URL:http://www.cliki.net/ASDF%20System>
[6] <URL:http://www.cliki.net/htout>
-----
Rob Warnock <rpw3@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607