Subject: Re: Keywords and CL-WHO
From: rpw3@rpw3.org (Rob Warnock)
Date: Thu, 21 Sep 2006 06:00:30 -0500
Newsgroups: comp.lang.lisp
Message-ID: <dfSdnW0ndM5T7Y_YnZ2dnUVZ_rednZ2d@speakeasy.net>
Dustin Withers <fadeddata@gmail.com> wrote:
+---------------
| I have a list representation of a purchase order (specifically an ASC
| X12 850).  All the segments and elements are broken into lists of
| lists.  This works well for parsing but I'm running into a problem
| turning the the first string in an element into a keyword (with the :
| on the front) that can be accepted by CL-WHO:WITH-HTML-OUTPUT.
+---------------

Short answer: Ain't gonna work. [Not as stated, at least.]

Medium answer: You're going about it the wrong way, but since you
seem already prepared to write code transform a parsed XML tree
into keywords [wrong approach] you should also be up to the task
of writing the code needed to transform a parsed XML tree directly
into strings of HTML output [right approach].

Much longer answer:
It appears that you are confusing macroexpansion time [which is
either part of compile-time or evaluation-time (for interpreted
code)] with run-time -- when the Common Lisp code actually executes.

CL-WHO:WITH-HTML-OUTPUT is a *macro* which is intended to be used
on literal [constant] source code forms at macroexpansion time, not
on dynamically-built structures at run-time. It does not evaluate its
&BODY argument; it *rewrites* it into Lisp code which the CL system
then compiles (or evaluates) in the usual fashion. Since a list that
begins with a keyword is not legal CL source code, WITH-HTML-OUTPUT
borrows that illegal format to use for itself as a marker that rewriting
is needed. Any literal input list or sublist *not* beginning with a
keyword (or a sublist beginning with a keyword) is passed straight
through to the CL system unmodified.

+---------------
| (cl-who:with-html-output (*standard-output* nil :prologue t)
| 	   (:html (:body "test")))
| 
| Produces:
| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
| "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
| <html><body>test</body></html>
+---------------

Right. But now let's look at *how* that happened:

    > (macroexpand
       '(cl-who:with-html-output (*standard-output* nil :prologue t)
 	  (:html (:body "test"))))

    (LET ((*STANDARD-OUTPUT* *STANDARD-OUTPUT*))
      (PROGN
	(WRITE-STRING
	 "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\"><html><body>test</body></html>"
	 *STANDARD-OUTPUT*)))
    T
    > 

That is, then entire (:HTML (:BODY "TEST")) subform got transformed --
at macroexpansion time!! -- into a single literal string. But as noted
above, if there are forms *not* beginning with keywords, they're just
passed through, permitting the intermingling of Lisp code with template
writing, which can be very useful, e.g.:

    > (import '(cl-who:htm cl-who:fmt)) ; for brevity

    T
    > (macroexpand
       '(cl-who:with-html-output (*standard-output* nil :prologue t)
 	  (:html
	    (:body
	      (loop for i from 1 to 5 do
		(htm "test #" (fmt "~d" i) (:br)))))))

    (LET ((*STANDARD-OUTPUT* *STANDARD-OUTPUT*))
      (PROGN
	(WRITE-STRING
	 "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\"><html><body>"
	 *STANDARD-OUTPUT*)
	(LOOP FOR I FROM 1 TO 5
	      DO (PROGN
		   (WRITE-STRING "test #" *STANDARD-OUTPUT*)
		   (FORMAT *STANDARD-OUTPUT* "~d" I)
		   (WRITE-STRING "<br />" *STANDARD-OUTPUT*)))
	(WRITE-STRING "</body></html>" *STANDARD-OUTPUT*)))
    T
    > 

Again, after the macro expands, all of the original (illegal) "keyword
forms" are gone, transformed into a legal CL program form. And then
when you *execute* that form [newlines added to output for clarity]:

    > (progn (eval *) (values))
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
      "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html><body> test #1<br />test #2<br />test #3<br />test #4<br />
    test #5<br /></body></html>
    > 

So to repeat, any CL code in the WITH-HTML-OUTPUT &BODY does run
at run-time, but *all* of the "keyword" stuff was removed [well,
transformed] by the macro at macroexpansion time.

And in any case, if your CL code is compiled there's no
WITH-HTML-OUTPUT left at run-time to pass your document to.

+---------------
| So knowing the little about keywords that I do I thought this would work:
| 
| (cl-who:with-html-output (*standard-output* nil :prologue t)
| 	   ((intern "FOO" :keyword) (:body "test")))
+---------------

Before trying to evaluate this, let's macroexpand it first:

    > (macroexpand
       '(cl-who:with-html-output (*standard-output* nil :prologue t)
 	  ((intern "FOO" :keyword) (:body "test"))))

    (LET ((*STANDARD-OUTPUT* *STANDARD-OUTPUT*))
      (PROGN
	(WRITE-STRING
	 "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">"
	 *STANDARD-OUTPUT*)
	((INTERN "FOO" :KEYWORD) (:BODY "test"))))
    T
    > 

Oops! Look at that form after the WRITE-STRING, which CL-WHO *didn't*
rewrite, since it didn't start with a keyword:

    ((INTERN "FOO" :KEYWORD) (:BODY "test"))

That's not legal Common Lisp, on several counts. No wonder your CL
implementation complained about it.

+---------------
| Sorry for my ignorance but can anyone explain where my thinking is
| wrong? I'd like to produce an XML document of the data that I have.
+---------------

One naive way would be to try to call the internal routine
CL-WHO::TREE-TO-COMMANDS at run-time with your munged-up
parsed XML tree, and then EVAL that, e.g.:

    (eval (cl-who::tree-to-commands (replace-tags-with-keywords my-tree)
				    *standard-output*
				    cl-who::*prologue*))

In the immortal words of a disgraced politician: "But that would
be wrong." Much better would be to simply walk the parsed XML tree
yourself, writing the output as text -- in essence, "unparsing" the
ML document. That is, instead of writing a REPLACE-TAGS-WITH-KEYWORDS
routine [which you were going to write anyway, yes?], write a
WRITE-TREE-AS-XML-TEXT routine. Hint: The code for the two is
*very* similar...


-Rob

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607