Thursday, August 03, 2006

Lisp is changing my habits

Back when we were doing serious heavy-lifting on the agent for Manage most of the subversion wrangling inevitably fell to Jasko. He very quickly realized that his fellow team members were not doing the best of jobs putting meaningful comments with our source code repository commits. Often times he would end up wading through half-a-dozen revision numbers with absolutely no comments, trying to figure out which one he needed. He very kindly reminded us to make sure and include meaningful messages with all of our commits, and we (mostly) complied.

To make matters worse, I would often find myself with a working copy littered with many unrelated changes. Instead of breaking them up after-the-fact, they would often go into the repository with a comment listing all of the changes. At least there were comments! Of course, this made rollbacks and merges difficult if you were later only concerned with an individual change that occurred in a single revision.

Obviously the main culprit of my bad source code control habits is simply procrastination. I would let the commit tasks build up until someone needed the changes I had been making. "Hey Tim, you ever get the whazits name working?" someone would IM. "Oh, yeah. Let me commit that," followed by a dump of everything I had been working on for the last couple days. I think one factor that contributed subconsciously to commit delinquency was the long code-build-test cycle that our agent required. A full build of the agent on a dual Xeon workstation takes 7-8 minutes. This stretched out the whole code-build-test cycle enough that we all avoided anything that would require us to rebuild the entire project.

It hit me a few weeks ago that I was making commits on a much more fine-grained scale. I was committing almost every little change. While I would like to believe that I am improving on my very bad procrastination streak, I'm starting to wonder if it isn't something more subtle. With my current Lisp setup (SBCL, Slime and Emacs) I can now make a small change in a function, hit Ctrl-c Ctrl-c in slime, and in seconds determine if my changes were for the better. If it knocks out a task, I can commit, and mark it off my todo. While the REPL has obviously-cool implications that can make for strong advocacy fodder, I wonder how much it is the subtle changes in habit over the long term that make Lisp's allure hard to express.

Tuesday, August 01, 2006

what is up with all the streams?

Is creating a new stream package a rite-of-passage for Lispers?

jsnell on #lisp pointed out the obvious to me this evening: to make a stream capable of output, use :output t, and to make it capable of input, use :input t. I figured all out on my own (and expect a lolli-pop later) that to make it binary you need :element-type :default. Alltogether it was:

(let*
((client-stream (make-socket-stream client-socket
:output t
:input t
:element-type :default))

Strangely, I had these same streams working unencumbered by the demands of a voratious OpenSSL implementation working fine. This was all to get something that CL+SSL would accept as a reasonable facimile of a stream. At this point, you have to flip things back around so you can go back to character-based operations when OpenSSL has finished reconstituting the bits from gibberish to meaningful words.

(cl+ssl:make-ssl-server-stream client-stream
:external-format :iso8859-1
:key ... :cert ... )

I very cleverly stuck Jasko with the hard part of getting the agent-side (read C++) SSL code working today (He's young and strong - I know he'll survive) In fact, not only did he survive, he beat me to getting his working against the openssl test server before I could get archon accepting connections with its certificate, We now have the two peices to the point where we can point them at each other and see if everything works.

We'll try not to cross the streams.

Sunday, July 30, 2006

Some things I learned this weekend

First, while I have known it for a while, it just doesn't get prettier on closer examination—OpenSSL is one convoluted library. I was attempting to roll my own OpenSSL interface using SBCL's sb-alien ffi hooks. While I was making progress, I suddenly came to the part where I needed to deal with x509_store_st structures, and decided there had to be an easier way. I originally looked at CL+SSL but abandonded it on initial inspection because I thought it would be too difficult to get it working with our epoll code. Once I hit the wall that was x509_store_st (for good reading check out "Secure Programming Cookbook for C and C++" Section 10.5) I decided to take a closer look at the CL+SSL code. It turns out that you can get at the underlying socket, even though it is wrapped under many stream classes. I should be able to use this socket to connect the CL+SSL streams into the epoll event notification structure I have set up.

Second, while the time spent on the OpenSSL stuff was a bust, I did learn more about how to declare complex structures in sb-alien. Things that might be obvious to others, but weren't to me:

void is ony valid as a return type for a function. The function "void foo(void*)" as a function pointer in a structure becomes:

(define-alien-type nil
(struct foo
(bar (* (function void (* t))))))

The first void is retained as it is the return void. The void in the arg list gets turned into a t.

Along the function definition line, it also doesn't work to include the names of the function's paramenters

(foo (* (function int (bar int) (baz int)))) ; does not work
(foo (* (function int int int)))

This may have been mentioned, but I couldn't seem to get scructures to nest without declaring the entire inside structure in-place. What was

(define-alien-type nil
(struct baz
(some int)
(other int)))

(define-alien-type nil
(struct foo
(bar (struct baz))))

Will not work with a complaint about unable to determine storage space needed for baz, In this case, the only recourse is to put baz into bar lock-stock-and-barrel:

(define-alien-type nil
(struct foo
(bar (struct baz
(some int)
(other int)))