Saturday, November 25, 2006

getting back my office

There hasn't been much Lisp activity lately as I have been taking advantage of some free time, courtesy of a little grandparent babysitting, to work on the attic. I am pretty much done with the drywall, and am working on trimming out the new windows. The new knee wall is going to be lined with bookshelves, and the far end will be the new location of my home office. It is still pretty slow-going, and I expect it will be a couple weeks before I have it knocked out. We still haven't even picked out paint colors, or flooring. If I was hip, I would go for some industrial job, but I'm not hip, so it will probably be whatever laminate is on clearance at Lowe's.


Wednesday, November 22, 2006

Rolling Tire Change

When I was younger I remember seeing a car on a TV show that some guy had tricked out with everything you could imagine (in the early 80s at least). The really cool part was when he showed how you could actually change a tire while the car was still moving. This worked by dropping a fifth wheel down, lifting up the offending tire, and extending running boards on the side of the car on which a person could work while changing the tire.

In my last post I mentioned finding an issue in clsql where it wasn't handling backslashes properly because it didn't have a database object with which to determine how to handle the backslashes. I filled Jasko in on the details on Monday, and complained that this meant putting in a fix into the clsql tree, and having to deal with all the tracking issues involved. At that point, he just looked at me weird and said - why don't you just redefine the function in our code and be done with it?


So, in our db package there is now a little extra function:

(defun clsql-sys::sql-output
       (sql-expr &optional (database clsql:*default-database*))
  (progv '(clsql-sys::*sql-stream*)
  (clsql-sys::output-sql sql-expr database)
  (clsql-sys::get-output-stream-string clsql-sys::*sql-stream*)))

What is even cooler is that I could write the new function, and then load it into the running server without ever requiring a restart. We could replace a defective function in a third-party library on-the-fly without any downtime. People delivering code in a more traditional distribution format wouldn't get much use from this kind of ability, the implications for software-as-a-service are pretty amazing (to me at least). While the choice of Lisp as our implementation language, especially given my own level of understanding, seemed a little outside the mainstream, we have benefited many times over from that decision.


Sunday, November 19, 2006

debugging in lisp

Tim and I installed an update last night, and also used to opportunity while we were bringing the server down and back up to bring the back-end up in slime so we could take a look at some conditions that were being logged during operation. Normally, we use detachtty to run both the web-facing component (which we call nexus), as well as the agent-facing component (called archon). This is useful for starting up a long running process, and then coming back to it, but we miss out on the convenience of slime. (I need to find some information on how to get the convenience of remote slime while still being able to detach/re-attach. Any pointers?). In any case, we were getting a simple-error that:

simple-error in message-handler:
There is no applicable method for the generic function
when called with arguments (NIL).

At first glance, it looks like we are missing a database connection. Maybe we are losing it? But during the course of tracking down the cause, we figure out that it only happens when we try to do a select on the database looking for software called "Part Control System (C:\\Pc\\)".

If I were a real Lispnik, at this point I would launch into a great example about how you can use the power of Lisp and the slime development environment to solve a problem like this very quickly. Unfortunately, I'm not. Instead, I floundered around all day following one bad assumption with an even worse hunch after another.

Gee. it must be an issue with the backslashes, let me try a zillion combinations of escaping and unescaping them. Or, maybe it is that database connection, let me try creating one locally and seeing if I can use that. Hmm, oh, wait...

Now, if I had just used the tools I had, and walked through the problem one step at a time, I would have found the answer much sooner. It turns out the original error is entirely correct. If I had taken the effort to go through the backtrace one function at a time, as I finally did, I would have found that one of the functions was defined as:

(defun sql-output (sql-expr &optional database)

and the function that calls it doesn't actually pass in a database! It turns out that the code path that required the database was only triggered if there were backslashes in the sql expression. I've got an email in to the clsql list to see if this is a real bug, or just a figment of my imagination. I do know that setting a default value for the &optional database to *default-database* fixed the problem for me.

This is not to say that slime, the debugger and inspection tools are not powerful. Once I actually started using them, I was able to find out what was wrong pretty quickly. It was a trepidation toward diving into the clsql code itself that caused me so much wasted effort. The backtrace showed me where the error was, I had the code on my machine—heck, hitting v on the line in the backtrace will take you right there—and I avoided it by trying to cargo-cult my way out of the problem. In the end, the clsql code in question was not even that complex. I see the old-timers on #lisp respond to queries all the time to the effect: "what is the backtrace telling you?" Maybe we are just so new at it that we haven't come to trust our tools yet, or we come from a background with tools that weren't trustworthy (C++ template errors anyone?). One thing I can take comfort in: there will be no end of opportunities to get more familiar with debugging in Lisp.