Subject: Re: strings and characters From: Erik Naggum <erik@naggum.no> Date: 2000/03/16 Newsgroups: comp.lang.lisp Message-ID: <3162184639382952@naggum.no> * Tim Bradshaw <tfb@cley.com> | The particular thing I don't understand is what type a literal string | has. It looks at first sight as if it should be something capable of | holding any CHARACTER, but I'm not really sure if that's right. It looks | to me as if it might be possible read things such that it's OK to return | something that can only hold a subtype of CHARACTER in some cases. strings _always_ contain a subtype of character. e.g., an implementation that supports bits will have to discard them from strings. the only array type that can contain all character objects has element-type t. | I'm actually more concerned with the flip side of this -- if almost all | the time I get some `good' subtype of CHARACTER (probably BASE-CHAR?) | but sometimes I get some ginormous multibyte unicode thing or something, | because I need to be able I have to deal with some C code which is | blithely assuming that unsigned chars are just small integers and strings | are arrays of small integers and so on in the usual C way, and I'm not | sure that I can trust my strings to be the same as its strings. this is not a string issue, it's an FFI issue. if you tell your FFI that you want to ship a string to a C function, it should do the conversion for you if it needs to be performed. if you can't trust your FFI to do the necessary conversions, you need a better FFI. | I realise that people who care about character issues are probably | laughing at me at this point, but my main aim is to keep everything as | simple as I can, and especially I don't want to have to keep copying my | strings into arrays of small integers (which I was doing at one point, | but it's too hairy). if you worry about these things, your life is already _way_ more complex than it needs to be. a string is a string. each element of the string is a character. stop worrying beyond this point. C and Common Lisp agree on this fundamental belief, believe it or not. your _quality_ Common Lisp implementation will ensure that whatever invariants are maintained in _each_ environment. | The practical question I guess is -- are there any implementations which | do currently have really big characters in strings? yes, and not only that -- it's vitally important that strings take up no more space than they need. a system that doesn't support both base-string (of base-char) and string (of extended-char) when it attempts to support Unicode will fail in the market -- Europe and the U.S. simply can't tolerate the huge growth in memory consumption from wantonly using twice as much as you need. Unicode even comes with a very intelligent compression technique because people realize that it's a waste of space to use 16 bits and more for characters in a given character set group. | I know there's an international Allegro, so those might have horrors in | them. sure, but in the same vein, it might also have responsible, intelligent people behind it, not neurotics who fail to realize that customers have requirements that _must_ be resolved. Allegro CL's international version deals very well with conversion between the native system strings and its internal strings. I know -- not only do I run the International version in a test environment that needs wide characters _internally_, the test environment can't handle Unicode or anything else wide at all, and it's never been a problem. incidentally, I don't see this as any different from whether you have a simple-base-string, a simple-string, a base-string, or a string. if you _have_ to worry, you should be the vendor or implementor of strings, not the user. if you are the user and worry, you either have a problem that you need to take up with your friendly programmer-savvy shrink, or you call your vendor and ask for support. I don't see this as any different from whether an array has a fill-pointer or not, either. if you hand it to your friendly FFI and you worry about the length of the array with or without fill-pointer, you're simply worrying too much, or you have a bug that needs to be fixed. "might have horrors"! what's next? monster strings under your bed? #:Erik