« prev · plain text · next »       2002 · Erik Naggum comp.lang.lisp archive       search · random

Subject: Re: Why I can't use Lisp.
From: Erik Naggum <erik@naggum.no>
Date: 03 Aug 2002 04:11:51 +0000
Newsgroups: comp.lang.lisp
Message-ID: <3237336711612014@naggum.no>

  The most important, if not the only, difference between the Common Lisp world
  and that of most other languages is that it is /perceived/ to be impossible to
  get anything done with the free Common Lisp systems; you cannot even /learn/
  Common LIsp using the free Common Lisp systems; not even the Trial Edition of
  Allegro CL or the Personal Edition of LispWorks will satisfy the "newbie" who
  wants to use Common Lisp.

  I have no idea where this /perception/ comes from or how it originated, but it
  is apparently deeply ingrained in people who know nothing about Common Lisp
  and refuse to learn or listen.  The amazingly stupid people who remain newbies
  for their entires lives seem to be upset about something related to commercial
  systems, but I actually wonder what it might be.  I think it /might/ be envy.

  Do you /need/ the power of the expensive commercial systems to use Common Lisp?
  No.  What is it that people /miss/ when they claim that they cannot use Common
  Lisp for their applications while they /could/ use just about any other language?
  I actually do not know, nor do I know where to start to understand this issue.

  Allow me to illustrate why I fail to understand this issue with a past project
  of mine.  I developed a system for real-time distribution of financial news
  for a wire service using Common Lisp that depended on multiprocessing support
  in the Common Lisp environment, in particular the ability to listen for
  activity on more than one socket.  We chose a commercial Common Lisp.  The
  reasons why we made this choice should be illumiating.

  The real-time nature of the application means that it was not CPU-intensive,
  and thus did not need preemptive multitasking between the processes.  What
  made it so convenient to use multi-threading in the Common Lisp environment
  was the shared memory.  It would have been possible to use the file system to
  share information between processes and to use a control connection to each
  process and then to implement a broker for messages between the processes.  It
  would be a different design that might have required more time to implement,
  but it would most certainly be doable.  The application would need to be
  implemented as an event dispatcher in each process, and it would have to be
  taught how to listen for activity on several sockets (as in the select system
  call) if the Common Lisp system did not already know how to do that.  I
  estimate that this complexity would have meant an additional month of
  development time, but it would also have meant that we would have needed more
  powerful hardware, since the memory requirements would be much higher and the
  processing power to deal with the interprocess communication (IPC) would be
  significantly higher.

  However, my system was to replace an overly complex system that basically did
  a tremendous amount of IPC and which had serious problems and shortcomings
  because of the constraints of the IPC methodology used, such as starting and
  stopping parts of the system independently.  (You would think not, but it was
  written by an idiot.)  It would have been a hard sell if I had argued that I
  would replace the existing mess with another mess of approximately the same
  magnitude.  So what I sold the project on was in fact that I could make all
  this a lot simpler in Common Lisp and thus remove the manual monitoring
  required by the frequently failing system.  In this regard, it was indeed the
  benefits of the commercial Common Lisp environment that made the project.
  This was a highly commercial project, with high risk and high benefits.  It
  had to succeed in a hostile environment,I would have to bring all the clients
  of the news wire on board with a new communication protocol to solve major
  problems in the previous design, and I needed all the confidence that money
  could buy -- in this case, it was for sale in the form of a commercial Common
  Lisp system.

  The commercial Common Lisp development environment was dwarfed the other costs
  involved.  (We were much less happy with the "runtime" licensing, but the time
  and money I spent on that was still worth the cost compared to reimplementing
  it with a lesser environment, again mostly because of the confidence required
  in the multi-threading support.)  /However/, had this /not/ been a project to
  salvage a wire service that would in all likelihood have faced survival issues
  if it had continued to crash and involve lots of manpower to keep it running,
  and I had been hired to do the original project, instead, I could easily have
  taken a chance on a "free" environment first, and in case of problems or costs
  or failure, have purchased "safety" in a supported commercial environment.

  I included this fairly long exposition of this project to show that I chose a
  commercial Common Lisp environment with commercial values in mind.  I would
  have to implement some of the things I would have gotten with that commercial
  environment if I used a free environment, and I would have to make myself an
  expert in the performance of IPC and socket communication and streams in
  Common Lisp, and many other internal details.  I have become that expert over
  time, but I was not ready to do it at that time.  So another commercial value
  is development and testing time before deployment.  I wager that these are
  /not/ issues for the free environments, /nor/ of their usability.  If you /can/
  afford to develop this for yourself, you should do it.  If you can /neither/
  afford to develop it yourself /nor/ purchase it from someone who has, you had
  better realize that you should not take on the project.

  What /is/ missing in the perception of the immature and inexperienced Common
  Lisp programmers is that /even/ if you had to do the internal stuff you would
  have to do in any other programming language and /even/ if this non-portable
  and implementation-dependent, you are /less/ system-dependent and /less/
  forced to work at the internal /level/.  This is hard to empathize enough to
  those who have made up their mind that they should get everything for free
  before they can even /start/ to think about using Common Lisp, but Common Lisp
  offers so much more convenience in the higher-level logic of the application
  and so much better ways to think about the system-dependency issues and the
  native operating system support that it is not worth the effort for anyone to
  sit down and "standardize" the missing features.  It is not the same level of
  pain and suffering it is in other languages.  Too many programmers would want
  a different take on the too many operating systems that might offer them that
  it would not make sense to "standardize" a huge number of features.

  It would, in fact, be detrimental to many development efforts to have to rely
  on many software packages from independent sources.  The reason many of the
  other programming languages do all these things is that they are single-vendor
  and single-implementation and can very easily merge independent projects into
  the common fold and people are willing to hand over copyrights and ownership
  to this single vendor.  This is utterly infeasible in the multi-vendor, multi-
  implementation world of Common Lisp where some people actually make money from
  supplying just these kinds of tools.  In fact, many programmers are annoyed
  that a vendor has chosen to join the "open source" community to undermine
  their own customers' ability to make money on similar efforts, and because the
  supposed "open source" is so heavily dependent on internal coding practices.

  It may be difficult for outsiders and newcomers to understand this, but the
  commercial /success/ of Common Lisp has made it less appealing for people to
  give stuff away.  It is /because/ Common Lisp is so eminently able to make money
  for those who use it that programmers and managers are reluctant to give away
  the genius and hard work behind the kind of quality and solid design that we
  are used to in the Common Lisp world.  It is because of the high level of pain
  and suffering to do relatively simple things that people share the burden in
  other languages -- the comparable implementation in Common Lisp is far simpler
  and more elegant.  On the other hand, Common Lisp programmers are finicky lot
  and woudl most probably want a different take on the same features from the
  operating system or other libraries -- satisfying them all would be an additional
  cost that would be hard to recover.  Common Lisp is on a level /between/ the
  "free" languages that are quite painful to use and the commercial success of a
  mass-market language where you can make money on third-party libraries.

  Those who are new to Common Lisp seldom realize that it is such a different
  language from the rest of the crop -- it is almost a different crop.  This may
  sound arrogant to people who have not used Common Lisp for real projects.  I
  may sound hopelessly out of touch with other languages and their advantages to
  those who come from environments where the cost of learning everything you
  need is much higher than the cost of implementing it in Common Lisp.  Those
  who succeed with Common Lisp are usually very bright people with experience in
  solving hard problems by making them simpler in surprising ways.  To many of
  them, the boneheaded brute force of many other languages are /distasteful/,
  not features.  Simplicity of implementation is generally not a value, because
  the elegance of use is what matters to users.  Common Lisp programmers are far
  more /practical/ than the average lot, who are often more self-indulgent and
  childishly delighted that anyone uses their code or rewards them for their
  hacks or compact form of expression.  Common Lisp has, in essence, a whole
  different value system.

  I have said that Common Lisp is the language you graduate into.  Common Lisp
  is what you get to use when you have been a good programmer for a while and
  grew tired of all the cruft.  Common Lisp programmers usually know dozens of
  languages and are not afraid to use them when practical, but because the cost
  of using a specialized language can sometimes be very high, such as when you
  have to keep track of which particular version of "language" and libraries and
  system libraries and kernels and whatnot in order to keep your system running.
  Knowing lots of languages also means that you are able to integrate them with
  ease and use particular libraries from particular languages more easily because
  you have a deeper understanding of the issues involved and a much broader
  knowledge of your actual needs.  Instead of being blinded by the glitz and the
  hype factors, Common Lisp programmers are generally more /practical/ and
  solution-oriented than the random youngsters who want attention and try to get
  it by giving away their time to strangers.

  However, if you are the perennial newbie, you will never understand this.  All
  you see is that your perennial newbiehood is rejected and your opinions are
  not validated by group hugs and cheers.  Neither are you able to write code
  that anyone will applaud, so being a newbie in Common Lisp is not the kind of
  "fun" it is to be a newbie in Python or Perl or whatever, where everybody is
  no more experienced than you are and you can impress just about anyone if you
  are above average.  Common Lisp requires more of you, but when you have grown
  up to meet those requirements, it is also extremely rewarding.  If you remain
  on the unmet side of those requirements, your task is to meet them, not to
  sulk over the rejection or wax apologetical about your lacking prowess: Your
  task should simply be to realize that it takes more than you got, then either
  decide to return later when you have what it takes, or decide to take on the
  task of learning what it takes.  If you hang around feeling inferior, you will
  do nobody any good, especially not yourself.  Despite the nervousness of some
  Germans, I consider it proper to make it clear to people that they should make
  this choice, but for some reason, some people prefer to sulk and whine than
  sit down and figure out what it takes to become a member in good standing.

  Common Lisp is not a "free" language in that it takes no effort to learn.
  Common Lisp is not a "free" language in that you can as easily discard it.
  Common Lisp is not a "free" language in that you will be unchanged by it.
  Common Lisp is not an investment-free language, but neither it is profit-free.

-- 
Erik Naggum, Oslo, Norway

Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.