Red Rome Logic

Xach has shamed me into posting a blog entry

2007-02-05T21:56:00.000-05:00

Life has been decidedly non-lispy lately, so I haven't had much to update in that area. I did a very quick reflector for a new addition to the Paragent.com service, but that was a very small bit of lisp coding that really took advantage of networking code I already had in place for Archon.

We rolled out some new features for Paragent.com including: Google searches right in the interface, hotfix lookups from microsoft's knowledge base, as well as license key reports.

Otherwise, it has really been a mix of decidedly non-programming activities that have been frittering away my time, including making a bunch of sales calls last month.

I have also been working on getting ready for our first ever Paragent.com trade show appearance. We are going to be in Las Vegas February 18th-21st at the Venetian for the 11th Annual International IT Service Management Conference & Exhibition. We decided rather last-minute to attend, so we have been pulling stuff together as quickly as possible. We don't want to blow too much money to attend, but we don't want our booth to look like a Jr. High science fair project either. We have also been getting schooled in the fine art of monopoly power. If working with the people that put on trade shows isn't actually working with the mob, it has to be a fair simulacrum. You get to answer questions like: Do you want to pay double to keep us from turning off the power in your booth outside of show hours? Do you want the $450 4 day, single computer wireless access, or the $2000 capped ethernet connection? Oh, how about the $62 dollars per day to empty your trash can (said trashcan having been rented from them, mind you). Just read the Grapes of Wrath, it is a much cheaper experience. I must blog more on this fun when we get back.

Oh, I've also been getting back into a little Cocoa programming. Leopard is going to have the new Objective-C 2.0. Improvements include new foreach syntax (even though I thought the NSEnumerator pattern not really painful), class properties, some other minor additions, and the biggie: a generational garbage collector (queue fanfare). What isn't getting into obj-c 2.0? Closures.

I am still of the opinion that Objective-C and Cocoa are the best gui development environment I have ever used. On the other hand, it is one thing to come to Cocoa from C++ or Java, and another to come back to it from Lisp. I have been looking into the Cocoa bridges for Lisp, but there isn't one available yet for my Lisp of choice. I've been doing a little digging to see how hard creating a bridge would be, but programming in Cocoa is just as much about the tools like Interface Builder as it is the available widgets. I don't even know enough at this point to carry on a conversation about it, but I am looking. The first step may in fact be an XCode lisp plugin.

A sobering look at alternative energy

2007-01-17T16:03:00.000-05:00

I think I am going to keep this article in mind whenever I am thinking of alternative energy. I tend to find things like fuel-cells, solar-panels and the like pretty cool—not because I am an environmentalist, but because I am a geek. Then you read something that puts you back down on terra firma with a hard thump.

Can this possibly be right?

2007-01-15T17:00:00.000-05:00

According to this article, Sarbanes-Oxley Act is apparently requiring Apple to charge $4.99 for unlocking the 802.11n support in computers that have been shipping with the functionality disabled. I don't know if S-O is just the latest boogeyman, but I hear it come up at least twice a week in some negative context regarding its unintended impact on business. If S-O is so sweeping, broad and ill-defined that a company like Apple has to start charging customers for something they would otherwise give them for free, it ceases to provide any meaningful positive impact.

[UPDATE]
Jeff Harrell does a much better job over at Shape Of Days.

Blu-Ray Just Lost

2007-01-13T10:12:00.000-05:00

This article at Ars indicates the "adult entertainment industry" is backing the HD-DVD standard. Apparently Sony doesn't want porn on their precious format. Sometimes you just have to shake your head. I believe the 3.5in disk was a Sony product, but apart from that now-obsolete media, Sony has a pretty abysmal record when it comes to getting consumers to buy. Beta, Mini-disk, memory-sticks, Blu-ray, and probably many more. [note: I believe beta is still used in some professional settings, but I could be wrong]. At some point wouldn't you just give up?

Not Likely

2007-01-13T09:34:00.000-05:00

According to this article, the Apple, Inc. board may be "forced" to fire Jobs as CEO. This is not the first article to suggest such an outcome of the backdating scandal. What I find interesting is that none of the articles I have read explain why exactly the board would be forced into such an action. The board is responsible to the shareholders. I think it is pretty clear that the majority of Apple stockholders want Jobs at the helm. Unless there were some legal limit to Jobs remaining as CEO, the board doesn't have to do jack—backdating or no.

iPhone

2007-01-10T10:03:00.000-05:00

So, got your iPhone on order yet? That is one sweet, sweet device. Unfortunately, there is some grumbling that folks are being told at MacWorld that Apple is not going to allow 3rd party apps. That is just dumb, dumb, dumb. I can imagine their arguments for it, and I think Apple will be printing money with these things, but for my part, it would kill much of my enthusiasm for the device. What is the whole point of promoting OS X on the device if they are going to hobble it? The people that would care about OS X on the phone want 3rd party apps, and the people that don't want 3rd party apps could care less about what OS is being used. Just doesn't make sense to me. That being said, there is no official word from Apple, the phone is not even out for another 6 months, and the real story may be different. Lets hope so.

Overheard on #lisp

2006-12-14T16:31:00.000-05:00

[4:25pm] Xach: sorry, i want santa to think about any details i didn't already specify
[4:26pm] froydnj: Xach: pffft, santa needs a list!
[4:27pm] Jasko: He's consing a list and checking it twice
[4:34pm] froydnj: Jasko: Santa Claus is coming to the heap?
[4:38pm] Jasko: froydnj: He knows if you've been badp

Time to panic

2006-12-09T17:15:00.001-05:00

I was having a string of kernel panics, and was not sure what was causing it. There seemed to be a correlation with using Parallels and World of Warcraft. My initial thought was that perhaps I had a bad stick of memory that was being exposed under high memory loads. I upgraded my mini when I first got it to 2GB of ram. It was a very stressful upgrade, and I was dreading the thought of cracking open the little bugger again if it was in fact the memory. Thankfully, I found the following commands to tell OSX how much memory to use.sudo nvram boot-args="maxmem=1024"

To set it back, use:

sudo nvram boot-args=""
I also found reference to a panic.log file during my google trolling, so I dug around, and found that it did record the KPs. I grabbed the first line from the last one, put it into google, and came back with the diagnosis that there was a good chance it might be the airport card.

It turns out that a) I just recently started using the airport connection on the mini because I had to move my setup down to a spare bedroom while I work on my home office in the attic where my hard-wired ports are. b) The only reason I was running Parallels was so my son could stream Noggin videos online, which only runs on windows. So the kernel panics were always happening during high network usage.

I guess this is motivation to get the darn attic finished.

Someone is ready for Christmas

2006-12-02T23:28:00.000-05:00

Ian asks us every day whether it is Christmas yet. Every. Day. Kristin and I just pray he has the same tenacity when it comes to doing his homework ten years from now. Otherwise, a slow day today. Did a little trim work around the windows in the attic, and grabbed a couple gallons of primer to start painting.
Not much progress on the rendering front. The AA vector drawing I have in place right now is just way to basic. This is getting a little repetitive, but once again, Xach has come to the rescue with a good link. Hopefully more to show tomorrow.

bezier progress

2006-12-02T09:39:00.000-05:00

Xach's comment referring me to a paper at the Anti-Grain Geometry site on estimating bezier curves using line segments was spot on. I had to replace the AA line-drawing algorithm to use fractional pixel locations. As you can see from the above, the line segments do a pretty good approximation. There are still some issues with how the individual line-segments meet up that can be seen in the image. I think this is because when two lines meet at a pixel location, the last line to be drawn overwrites the pixel color instead of adding its contribution. This code will also give me an excuse to finally dig into CL optimizations like type declarations and inlining. From some debugging output, I know that some of the code is actually using rationals. I'm almost sure that can't be good for speed

Brute Force Bezier Curves

2006-11-29T22:19:00.000-05:00

I think in order to render TrueType fonts you first have to be able to render Bezier curves. Above is the brute-force method I hacked together for my in-progress rendering engine in lisp. I still don't know how to anti-alias them. Once I get the curves looking good, then I can integrate Xach's zfb-ttf library to get the curves for different True-Type fonts. From the little I've read there is more to decent font rendering than just following the control points. I've seen references to hinting, which I think are instructions for tweaking the font rendering especially at very small sizes.

Lines

2006-11-27T13:11:00.000-05:00

In C, I'm used to swapping values with a temporary:
int x = 1; int y = 2; int tmp = x; x = y; y = tmp;
In Lisp, you can do the same thing using psetf:
(let ((x 1) (y 2)) (psetf x y y x))
Sweet.

I'm doing more math-wise in Lisp as I am playing around with a little bit of rendering. psetf is an example of one of those small corner-case functions that greatly simplify coding in Lisp. Going through the kind of "and here are all the NUMBER functions..." reference doesn't penetrate my thick skull. Necessity is the mother of memorization (or something like that). I am moving forward on the working assumption that no matter what it is I am coding, there is probably a simpler way to do it. This isn't necessarily always the case. There are a few areas where Lisp seems overly verbose (multiple return values in a let comes to mind); however, it is a pretty good chance that if it looks ugly, Lisp has a better way of doing it. The picture above is my anti-aliased line rendering courtesy of Xach's Salza package and example png.lisp code. I'm rolling my own as a hobby/exercise more than anything else. I've got some particular image rendering goals, so this is probably not going to be that generic. I don't care about rotating bitmaps, color correction, or any of that image manipulation stuff. What I do want to do is render nice anti-aliased curves, text and gradients.

[UPDATE]
And even when I've found a simpler way, I am probably still missing something. Thas on #lisp pointed out that the simple swap above could be done with (rotatef x y) much more concisely. In my case, I am actually performing several swaps at once based on the slope of the line:
(when (minusp dy) (psetf x1 x2 x2 x1 y1 y2 y2 y1 dx (- x1 x2) dy (- y1 y2)))
But, for the simple swap, rotatef is your man.

Lisp at Microsoft

2006-11-27T10:49:00.000-05:00

It turns out the garbage collector for Microsoft's CLR was originally written in Common Lisp.

getting back my office

2006-11-25T20:48:00.000-05:00

There hasn't been much Lisp activity lately as I have been taking advantage of some free time, courtesy of a little grandparent babysitting, to work on the attic. I am pretty much done with the drywall, and am working on trimming out the new windows. The new knee wall is going to be lined with bookshelves, and the far end will be the new location of my home office. It is still pretty slow-going, and I expect it will be a couple weeks before I have it knocked out. We still haven't even picked out paint colors, or flooring. If I was hip, I would go for some industrial job, but I'm not hip, so it will probably be whatever laminate is on clearance at Lowe's.

Rolling Tire Change

2006-11-22T20:59:00.000-05:00

When I was younger I remember seeing a car on a TV show that some guy had tricked out with everything you could imagine (in the early 80s at least). The really cool part was when he showed how you could actually change a tire while the car was still moving. This worked by dropping a fifth wheel down, lifting up the offending tire, and extending running boards on the side of the car on which a person could work while changing the tire.

In my last post I mentioned finding an issue in clsql where it wasn't handling backslashes properly because it didn't have a database object with which to determine how to handle the backslashes. I filled Jasko in on the details on Monday, and complained that this meant putting in a fix into the clsql tree, and having to deal with all the tracking issues involved. At that point, he just looked at me weird and said - why don't you just redefine the function in our code and be done with it?

blink

So, in our db package there is now a little extra function:
(defun clsql-sys::sql-output (sql-expr &optional (database clsql:*default-database*)) (progv '(clsql-sys::*sql-stream*) `(,(clsql-sys::make-string-output-stream)) (clsql-sys::output-sql sql-expr database) (clsql-sys::get-output-stream-string clsql-sys::*sql-stream*)))
What is even cooler is that I could write the new function, and then load it into the running server without ever requiring a restart. We could replace a defective function in a third-party library on-the-fly without any downtime. People delivering code in a more traditional distribution format wouldn't get much use from this kind of ability, the implications for software-as-a-service are pretty amazing (to me at least). While the choice of Lisp as our implementation language, especially given my own level of understanding, seemed a little outside the mainstream, we have benefited many times over from that decision.

debugging in lisp

2006-11-19T23:01:00.000-05:00

Tim and I installed an update last night, and also used to opportunity while we were bringing the server down and back up to bring the back-end up in slime so we could take a look at some conditions that were being logged during operation. Normally, we use detachtty to run both the web-facing component (which we call nexus), as well as the agent-facing component (called archon). This is useful for starting up a long running process, and then coming back to it, but we miss out on the convenience of slime. (I need to find some information on how to get the convenience of remote slime while still being able to detach/re-attach. Any pointers?). In any case, we were getting a simple-error that:
simple-error in message-handler: There is no applicable method for the generic function #<STANDARD-GENERIC-FUNCTION CLSQL-SYS:DATABASE-TYPE (2)> when called with arguments (NIL).
At first glance, it looks like we are missing a database connection. Maybe we are losing it? But during the course of tracking down the cause, we figure out that it only happens when we try to do a select on the database looking for software called "Part Control System (C:\\Pc\\)".

If I were a real Lispnik, at this point I would launch into a great example about how you can use the power of Lisp and the slime development environment to solve a problem like this very quickly. Unfortunately, I'm not. Instead, I floundered around all day following one bad assumption with an even worse hunch after another.

Gee. it must be an issue with the backslashes, let me try a zillion combinations of escaping and unescaping them. Or, maybe it is that database connection, let me try creating one locally and seeing if I can use that. Hmm, oh, wait...

Now, if I had just used the tools I had, and walked through the problem one step at a time, I would have found the answer much sooner. It turns out the original error is entirely correct. If I had taken the effort to go through the backtrace one function at a time, as I finally did, I would have found that one of the functions was defined as:

(defun sql-output (sql-expr &optional database) ...
and the function that calls it doesn't actually pass in a database! It turns out that the code path that required the database was only triggered if there were backslashes in the sql expression. I've got an email in to the clsql list to see if this is a real bug, or just a figment of my imagination. I do know that setting a default value for the &optional database to *default-database* fixed the problem for me.

This is not to say that slime, the debugger and inspection tools are not powerful. Once I actually started using them, I was able to find out what was wrong pretty quickly. It was a trepidation toward diving into the clsql code itself that caused me so much wasted effort. The backtrace showed me where the error was, I had the code on my machine—heck, hitting v on the line in the backtrace will take you right there—and I avoided it by trying to cargo-cult my way out of the problem. In the end, the clsql code in question was not even that complex. I see the old-timers on #lisp respond to queries all the time to the effect: "what is the backtrace telling you?" Maybe we are just so new at it that we haven't come to trust our tools yet, or we come from a background with tools that weren't trustworthy (C++ template errors anyone?). One thing I can take comfort in: there will be no end of opportunities to get more familiar with debugging in Lisp.

SBCL + UCW Installation Roadmap

2006-11-17T21:30:00.000-05:00

The following is a rundown of the procedure we use to get our appliance up and running with SBCL and UCW on a stock minimal install of Fedora Core 6 x86. You can see the actual application in action if you go to Paragent.com and click on demo.

Preping the OS

After getting the initial system up and running, be sure to get the latest updates by running:
# yum -y install emacs # yum -y upgrade
This will download and install any updates since the ISO images were created.

Next, you will want to make sure that the firewall is set up correctly
# system-config-securitylevel-tui
choose ssh, http, https if need be, and any other ports you might need. You will also want to make sure that SELinux is disabled. Otherwise, it interferes with the apache/mod_lisp communication with the lisp process. You can do this by going to /etc/selinux/config and setting
SELINUX=disabled
There may be a safer way to accomplish this, but I didn't feel like mucking about with SELinux. If anyone knows the proper way to set up the SELinux permissions, let me know.

SBCL

Next, you need to get sbcl installed. we typically use the latest version of sbcl from cvs. However, in order to build sbcl, you will need an initial copy that we can use to bootstrap the build process. As usual, yum is the easiest way to go here.
# yum -y install sbcl
Now, you can go about getting sbcl and building it. In this example, we are putting all of our lisp related items into /lisp
# mkdir /lisp # cd /lisp # cvs -d:pserver:anonymous@sbcl.cvs.sourceforge.net:/cvsroot/sbcl login Cvs password: [return] # cvs -z3 -d:pserver:anonymous@sbcl.cvs.sourceforge.net:/cvsroot/sbcl co -P sbcl # cd sbcl # emacs -nw customize-target-features.lisp
This last command is to create a file called customize-target-features.lisp that will tell the sbcl build process what extra features we would like. In this case, it is only threading support we are interested in. Put the following code into the file and save:
;; add threading support (lambda (list) (cons :sb-thread list))
Now, we are ready to build sbcl:
# sh make.sh
and install it (assuming you have the priviledges here to do so):
# sh install.sh
now that we have a local version, get rid of the sbcl with which we bootstrapped the whole operation.
# yum remove sbcl
NOTE: for some reason, even though the new one is in the /usr/local/bin dir, which is in our path, it is still looking for it in /usr/bin. if you just type sbcl at the prompt without the absolute path, it won't find it. must be caching it somewhere.

[UPDATE] Zach pointed out in the comments that
hash -r
Will clear the path cache

ASDF-BINARY-LOCATIONS

Next, you will want to set up asdf-binary-locations. You can use asdf-install to get this package. In order to install it system-wide, you will want to be running as a user that has permissions to write to the sbcl site-systems directory (i.e. root)
# /usr/local/bin/sbcl * (require 'asdf) * (require 'asdf-install) * (asdf-install:install 'asdf-binary-locations)
select 1 'System-wide' for where to install, and hit zero when the GPG error pops up

SPLIT-SEQUENCE

One handy library that gets used a bit is split sequence, so we go ahead and install it at this point as well.
* (asdf-install:install 'split-sequence)
again, select 1 for system-wide install and then 0 to skip gpg check.

you need to make sure that sbcl stuff doesn't get caught up in the asdf-binary-locations tom-foolery, so create a ~/.sbclrc file and put the following into it:
;;; -*- Lisp -*- (require :asdf) (require :asdf-binary-locations) (defvar asdf::*source-to-target-mappings* '(("/usr/local/lib/sbcl/" nil)))

SLIME

The next thing you will want to install is slime:
# cd /lisp # cvs -d :pserver:anonymous@common-lisp.net:/project/slime/cvsroot login Cvs password: anonymous # cvs -z3 -d :pserver:anonymous@common-lisp.net:/project/slime/cvsroot co -P slime
To use slime, you will need to edit your .emacs file:
; .emacs setup for slime (setq inferior-lisp-program "/usr/local/bin/sbcl") (add-to-list 'load-path "/lisp/slime") (require 'slime) (slime-setup)

MySQL

We happen to use mysql as our back-end database, so we install it now if it isn't already on the system.
# yum install mysql # yum install mysql-server # yum install mysql-devel # /etc/init.d/mysqld start
to get mysqld to restart on reboot properly, issue a
# chkconfig mysqld on

CLSQL

Now we want to install clsql. It has an additional dependency of uffi that we will also install at this time.
# cd /lisp # wget http://files.b9.com/uffi/uffi-1.5.17.tar.gz # wget http://www.paragent.com/downloads/clsql-3.6.3.tar.gz # tar zxf uffi-1.5.17.tar.gz # tar zxf clsql-3.6.3.tar.gz # cd /usr/local/lib/sbcl/site-systems # ln -s /lisp/uffi-1.5.17/uffi.asd . # ln -s /lisp/clsql-3.6.3/clsql.asd . # ln -s /lisp/clsql-3.6.3/clsql-mysql.asd . # ln -s /lisp/clsql-3.6.3/clsql-uffi.asd . # cd /lisp/clsql-3.6.3/db-mysql/ # make clean # make

Additional Dependencies

Depending on what you are going to do, you probably don't need all of these, but I have included them here to show some extra things that you might want.

CL-BASE64

This is a dependency of cl-smtp, the email library we use.
# cd /lisp # wget http://files.b9.com/cl-base64/cl-base64-latest.tar.gz # tar zxf cl-base64-latest.tar.gz # cd /usr/local/lib/sbcl/site-systems # ln -s /lisp/cl-base64-3.3.2/cl-base64.asd .

CL-SMTP

cl-smtp allows us to send out email using normal mail servers. It even supports authentication to the server, so if your mail provider requires you to log in to send mail, this still works. Very handy for sending out automated announcements, etc.
# cd /lisp # wget http://common-lisp.net/project/cl-smtp/cl-smtp.tar.gz # tar zxf cl-smtp.tar.gz # cd /usr/local/lib/sbcl/site-systems # ln -s /lisp/cl-smtp/cl-smtp.asd .

Ironclad

If you are going to be doing any encryption type stuff, ironclad is a useful library. It is available via asdf-install, so from slime, you can:
CL-USER> (require 'asdf-install) CL-USER> (asdf-install:install 'ironclad)
Remember that you need to have permission to write to your sbcl site-systems directory when you start up emacs if you want to be able to do a system-wide install.

UCW

This is the biggie. Luckily, they have packaged up everything you need in a handy boxset.
# cd /lisp # wget http://common-lisp.net/project/ucw/ucw-boxset.tar.gz # tar zxf ucw-boxset.tar.gz

Your Code Here

This is where you would put your code. Hopefully this will turn into a link to a ucw tutorial at some point.

Create MySQL Database

You might want to make some changes to you mysql configuration. A couple changes we make are in /etc/my.cnf. Add the following two lines to [mysqld] config:
lower_case_table_names=1 set-variable=max_connections=500

mod_lisp

You will next want to get mod_lisp2 and set it up (this is assuming apache 2.x). It does take a little bit of fiddling.
# cd /lisp # mkdir mod_lisp # cd mod_lisp # wget http://www.fractalconcept.com:8000/public/open-source/mod_lisp/mod_lisp2.c
At the moment this file needs to be edited to take into account newer version of apache. The code you need to change is line 152-3 of mod_lisp2.c:
#if (APR_MAJOR_VERSION==1 && APR_MINOR_VERSION==2 && APR_PATCH_VERSION==2)
needs to be changed to accept anything over 1.2.2:
#if (APR_MAJOR_VERSION==1 && APR_MINOR_VERSION==2 && APR_PATCH_VERSION>=2)
Now you can install the apache development tools if you don't have them, and build the library:
# yum install httpd-devel # apxs -ica mod_lisp2.c
Add the following in /etc/httpd/conf/httpd.conf
# # mod_lisp2 configuration # <LocationMatch "/.*\.ucw"> LispServer 127.0.0.1 3001 ucw SetHandler lisp-handler </LocationMatch> <LocationMatch "/.*\.csv"> LispServer 127.0.0.1 3001 ucw SetHandler lisp-handler </LocationMatch>
Now, restart the apache server
# apachectl restart # chkconfig httpd on
You will now want to make sure ucw is using mod_lisp. edit /lisp/ucw-boxset/ucw_dev/src/vars.lisp
;;; `ucw:create-server' variables (defvar *ucw-backend-type* :mod-lisp) (defvar *ucw-backend-host* "127.0.0.1") (defvar *ucw-backend-port* 3001) (defvar *ucw-server-class* 'standard-server)

detachtty

One useful program included in the ucw-boxset is detachtty, which allows you to spin off your lisp process and then come back to it later.
# cd /lisp/ucw-boxset/dependencies/detachtty # make # make install

quote of the day

2006-11-15T07:54:00.000-05:00

Overheard on c.l.l: >> Live anywhere else in the world and the internet is a useful resource. You can for instance find things about where you live by using a search engine. When I lived in cambridge I could type things like "fish and chip shops cambridge" or "curtains cambridge" and get useful results. If I do the same in Limerick all I get is ribald poetry. The only way of finding anything quickly is the phone book. >try googling: limerick curtains -"there -once -was" should eliminate most of the ribald poetry ;^) Wow, that mostly works, thanks.

at your service

2006-11-14T22:37:00.000-05:00

We had a couple off site trips today. One was a planned install, the other was to go check out a server that was having problems, and didn't yet have remote access setup. We quickly figured out what was causing the web server grief, which we happened to have a fix for, in addition to a couple additional fixes I had brought with me on a thumb drive. That is when we decided to do some extreme programming on-the-spot. The customer had a feature request to collect some information about the services that were installed on their monitored machines. We reloaded the web and back-end services in slime, and started hacking away with the customer watching over our shoulders. Jasko and I tag teamed on the keyboard, and in about 30 minutes had it knocked out and running.

There are a couple takeaways for me from the experience.

First: emacs. despite the fact that Jasko prefers Eclipse to emacs, we had a fully functioning development environment running on a crappy VGA terminal window with only a keyboard. emacs, M-x slime and we were off and running. Sure, having more code visible on screen would have been very nice, but it was perfectly usable.

Second: macros. A major aspect of why it took us so little time to add the new functionality was that the hard work had been done once with the creation of macros, and we just drafted behind. The entire effort in adding the data collection to the back-end was three lines:(defpropquery query-services "service" ("name" "description") 'services "service" (:name :description))
What you don't see is the 20-line macro that does all the magic in the background. It takes those three lines, and creates the code to query the remote agent, collect up the results, update the database, see if there were any changes to the properties, and log any events that might have occurred. The same thing was true in the database code, as well as the ucw components for the front-end.

Lemonodor fame awaits

2006-11-09T21:22:00.000-05:00

Jasko has done it - he has achieved Lemonodor Fame for his Cusp plugin for Eclipse! While I use emacs+slime myself, Tim uses Cusp day in and day out, so it is definitely a case of eating your own dogfood with him. There are more good things to come.

A positive side-effect is the boost in traffic we are seeing at Paragent. Chees called me up when he saw the spike in google analytics. The cusp page is the #1 hit on our site. Congratulations!

I put in a rolling fix to the server today.
# svn update properties.lisp # attachtty /var/run/archon-socket * (load "/lisp/repos/archon/properties")
And the fix was smoothly inserted into a running server. Man I love Lisp.

elephants, openssl and clsql

2006-11-08T22:36:00.000-05:00

My son has told us he wants to be an elephant when he grows up. It doesn't take much imagination to guess what he was for halloween.

Thankfully, we have finally burned through the three bags of halloween candy at work. I can go through a bag of candy like, well... like something that can eat an entire bag of candy in a couple days. There is NO MORE to eat.

Dustin added a monitor for the backend server this evening, so I can now get paged in the middle of the night when it goes down. If you haven't used it before, the openssl command-line tool comes in handy when you need to test ssl code. you can use the s_client and s_server options to make connections.
$ openssl s_client -connect 10.0.0.10:7777
In our case, Dustin just wrapped a call to openssl s_client and checked to make sure the server responds with an identification query as it should.

I frittered away the day it seems otherwise. I am still trying to work out what is wrong with our clsql code. Version 3.6.3 worked fine for us, but we are now getting a bunch of errors with 3.7.7 about [= being an unknown variable. Apparently there has been a change in the way that enable-sql-reader-syntax works, and our code is no longer doing the right thing to enable the use of the [ and ] macros.

[UPDATE]
While something changed to break our code with clsql, Marcus Pearce was kind enough to send along some pointers on how to get it working again. It may be that we were simply using the ENABLE-SQL-READER-SYNTAX incorrectly, and were lucky enough to have it working. We were able to reduce our issue down to a simple test-case, so we might hear back more on that front. In the mean time, I have used #.(clsql:locally-enable-sql-reader-syntax) and #.(clsql:restore-sql-reader-syntax-state) as suggested by Marcus at the beginning and end of all the files that use the brackets. It is not nearly as convenient as defining it in one location and having it work everywhere, but it works, and is backward compatible with the 3.6.3 version of clsql.

dependencies

2006-11-06T22:42:00.000-05:00

I spent most of today replicating our hosted web application on a new server, and actually documenting every step. It is amazing the way these things accumulate dependencies. There are things I never remember installing in the first place. It is like dependencies are their own form of self-replicating organism. CL-BASE64, where did that come from? There was even a well timed comment today on #lisp that ucw required half of the cl.net libraries to run. I thought - "If you also want to use clsql it requires the other half of cl.net."

It came to mind today that while in most languges there is a drive among adopters to want to interface it with existing libraries, in Lisp there really seems to be a desire to implement libraries. Lispniks seem to have a preference for seeing a solution coded in Lisp itself. It is not that Lisp doesn't have an easy way to talk to foreign libraries. If anything, it suffers from too many (sb-alien, sb-grovel, uffi, cffi, etc). People just enjoy writing stuff in Lisp itself.

It seems like there are one or two conversations a week explaining to a newcomer to Lisp that "in Lisp" means literally just that - the cl-foobar library doesn't call to the foobar command-line tool, or link with libfoobar.so. It is a native implementation of foobar in 100% Lisp. Come to think of it, more often than not, Xach is involved on one side of the conversation. I think the old-timers take great pleasure from the obvious shock to people that have been taught the rigid low-level compiled versus high-level dynamic view of the programming language spectrum. Heck, that seems to be the entire point behind Practical Common Lisp.

Okay, now where did that dependency come from...

Has Linux stagnated as much as Windows?

2006-11-04T22:18:00.000-05:00

I used to really be into Linux. I even did embedded Linux development for little MPC860-based routers. Did all the custom kernel compiling stuff, etc. In fact, that is how I got my first Mac - it had the same PPC chip we were going to use in a new router we were working on. That little beast is still in daily use in my household for web browsing and email checking. It is a 500MHz G4 PowerBook - back when they made them out of TITANIUM kids! For a long time, it ran Yellow Dog Linux. Then OS X came out, and over time the Linux partition fell into disuse, and eventual elimination. But, I still have a soft spot in my heart for Linux.

I've been getting back into Linux a bit more lately as we have been using it for our web-based software. We got in a couple IBM xSeries servers yesterday, and Dustin and I spent some time today (Dustin much more than me) working on getting Linux installed. Inevitably, we ran into an issue with the adaptec SATA RAID controller. Imagine that. More to the point, this was with the brand-new Fedora Core 6. If you go through a normal installation process, anaconda just hangs. None of this was necessarily unexpected. I've dealt with driver issues with Linux before. I've even written drivers myself. But that was five years ago. I guess I just expected there to be some improvement. From what I can tell, there hasn't been any real fundamental improvement in Linux's real usability. Heck, just try installing Firefox 2 on a standard Fedora Core 5 install.

The dangers of auto-complete

2006-11-03T22:38:00.000-05:00

I ordered two IBM xSeries 306m servers yesterday with overnight shipping so we could put an appliance version of our Lisp-based desktop management service together. In the process of putting in the order, Safari's autocompletion kicked in with my home address information. In my haste, I changed the number, but not the street name. When I checked the package tracking this afternoon, it was marked as "Delivery Attempted" which caused the office air to be filled with many unprintable invectives, as I was certain that no such delivery attempt had been made. On further inspection, I realized my mistake, and called the delivery company. Thankfully, the truck was still on its route, the kind lady assisting me got in touch with the driver, and they came by the right address 30 minutes later. Kudos to DHL for their service.

Proving once again that no project is complete without at least three trips to the store*, it turns out the server didn't come with the rails needed to mount any drives, which have to be ordered seperately at $30 a piece. rasberries to IBM for not including them. We should be able to temporarily wedge the drives in place while we do the install until the rails arrive on Monday or Tuesday.

It is going to be interesting to see what we can do with the appliance model. At the moment, we don't really use any cores from (save-lisp-and-die). Since everything is so long-running, I just have a lisp file that loads everything we need. Since we are going to have many identical implementations out in the field, using a core file might be able to save us from having to get all of the dependencies in place for a full compile. I'm not entirely sure at this point whether we would still need to have the original source or fasls lying around somewhere.

* In my case, the store it typically the local Lowes home supply megamart.

unplugged

2006-11-02T22:15:00.000-05:00

I took a couple days off at the beginning of this week to work on the attic. It took most of Monday to get it cleared, and then Tuesday I started the process of piecing it back together. Slowly. All the bad drywall is out, and the new drywall is up, taped and mudded once. I started on the knee wall tonight. I'm just working in sections, bringing up the 2x4s, hammering it together, and rough fitting it in place. Hopefully the weekend will give me a good block of time to get the major construction stuff out of the way. We are still trying to decide how we are going to finish it. It is mostly a question of where the bookshelves are going. We have lots of books.

My chance for getting anything done with the attic may be hindered by a couple servers we ordered that will hopefully come in tomorrow. I need to get the memory and drives installed, put linux on them, and then install Archon. These are going to be an appliance-based version of our hosted desktop management service at Paragent.com for a couple customers that do not have external internet access and need something on site.

Since my home-office had to be swept aside for the sake of progress, I only have the old PowerBook handy for my net-addiction. Even though I can pull the laptop out from under the couch and check my email and browse the web at will, it is just not the same as having a computer that is constantly on and connected to IRC and IM. I know I'm missing out on something...

Microsoft's Motivations

2006-09-23T21:30:00.000-04:00

I thought I would install IE7 today on my home computer to have a look-see at how it would handle our new site. At present, IE6 is managing its normal Cuisinart job on it. In the process of trying to download and install IE7, I was asked no less than three times to verify my Windows XP install with Microsoft Genuine Advantage. All of which passed with flying colors, but for some reason couldn't be remembered.

I get the definite impression that IE7 is just a vehicle for Microsoft to scan for illegal copies of Windows. That is probably what green-lighted the whole project. If some exec at MS hadn't realized they could use the promise of IE7 as the carrot, and Genuine Advantage as the stick, IE7 would probably have ended up as a Vista-only feature. Imagine if every time you tried driving your car you got pulled over because the dealer kept reporting it as possibly stolen. Clicking through pages of Microsoft Genuine Advantage crap is a complete waste of time.

Of course, so is rambling on a blog...

Lisp is changing my habits

2006-08-03T22:39:00.000-04:00

Back when we were doing serious heavy-lifting on the agent for Manage most of the subversion wrangling inevitably fell to Jasko. He very quickly realized that his fellow team members were not doing the best of jobs putting meaningful comments with our source code repository commits. Often times he would end up wading through half-a-dozen revision numbers with absolutely no comments, trying to figure out which one he needed. He very kindly reminded us to make sure and include meaningful messages with all of our commits, and we (mostly) complied.

To make matters worse, I would often find myself with a working copy littered with many unrelated changes. Instead of breaking them up after-the-fact, they would often go into the repository with a comment listing all of the changes. At least there were comments! Of course, this made rollbacks and merges difficult if you were later only concerned with an individual change that occurred in a single revision.

Obviously the main culprit of my bad source code control habits is simply procrastination. I would let the commit tasks build up until someone needed the changes I had been making. "Hey Tim, you ever get the whazits name working?" someone would IM. "Oh, yeah. Let me commit that," followed by a dump of everything I had been working on for the last couple days. I think one factor that contributed subconsciously to commit delinquency was the long code-build-test cycle that our agent required. A full build of the agent on a dual Xeon workstation takes 7-8 minutes. This stretched out the whole code-build-test cycle enough that we all avoided anything that would require us to rebuild the entire project.

It hit me a few weeks ago that I was making commits on a much more fine-grained scale. I was committing almost every little change. While I would like to believe that I am improving on my very bad procrastination streak, I'm starting to wonder if it isn't something more subtle. With my current Lisp setup (SBCL, Slime and Emacs) I can now make a small change in a function, hit Ctrl-c Ctrl-c in slime, and in seconds determine if my changes were for the better. If it knocks out a task, I can commit, and mark it off my todo. While the REPL has obviously-cool implications that can make for strong advocacy fodder, I wonder how much it is the subtle changes in habit over the long term that make Lisp's allure hard to express.

what is up with all the streams?

2006-08-01T22:51:00.000-04:00

Is creating a new stream package a rite-of-passage for Lispers?

jsnell on #lisp pointed out the obvious to me this evening: to make a stream capable of output, use :output t, and to make it capable of input, use :input t. I figured all out on my own (and expect a lolli-pop later) that to make it binary you need :element-type :default. Alltogether it was:

(let*
((client-stream (make-socket-stream client-socket
:output t
:input t
:element-type :default))

Strangely, I had these same streams working unencumbered by the demands of a voratious OpenSSL implementation working fine. This was all to get something that CL+SSL would accept as a reasonable facimile of a stream. At this point, you have to flip things back around so you can go back to character-based operations when OpenSSL has finished reconstituting the bits from gibberish to meaningful words.

(cl+ssl:make-ssl-server-stream client-stream
:external-format :iso8859-1
:key ... :cert ... )

I very cleverly stuck Jasko with the hard part of getting the agent-side (read C++) SSL code working today (He's young and strong - I know he'll survive) In fact, not only did he survive, he beat me to getting his working against the openssl test server before I could get archon accepting connections with its certificate, We now have the two peices to the point where we can point them at each other and see if everything works.

We'll try not to cross the streams.

Some things I learned this weekend

2006-07-30T23:41:00.000-04:00

First, while I have known it for a while, it just doesn't get prettier on closer examination—OpenSSL is one convoluted library. I was attempting to roll my own OpenSSL interface using SBCL's sb-alien ffi hooks. While I was making progress, I suddenly came to the part where I needed to deal with x509_store_st structures, and decided there had to be an easier way. I originally looked at CL+SSL but abandonded it on initial inspection because I thought it would be too difficult to get it working with our epoll code. Once I hit the wall that was x509_store_st (for good reading check out "Secure Programming Cookbook for C and C++" Section 10.5) I decided to take a closer look at the CL+SSL code. It turns out that you can get at the underlying socket, even though it is wrapped under many stream classes. I should be able to use this socket to connect the CL+SSL streams into the epoll event notification structure I have set up.

Second, while the time spent on the OpenSSL stuff was a bust, I did learn more about how to declare complex structures in sb-alien. Things that might be obvious to others, but weren't to me:

void is ony valid as a return type for a function. The function "void foo(void*)" as a function pointer in a structure becomes:

(define-alien-type nil
(struct foo
(bar (* (function void (* t))))))

The first void is retained as it is the return void. The void in the arg list gets turned into a t.

Along the function definition line, it also doesn't work to include the names of the function's paramenters

(foo (* (function int (bar int) (baz int)))) ; does not work
(foo (* (function int int int)))

This may have been mentioned, but I couldn't seem to get scructures to nest without declaring the entire inside structure in-place. What was

(define-alien-type nil
(struct baz
(some int)
(other int)))

(define-alien-type nil
(struct foo
(bar (struct baz))))

Will not work with a complaint about unable to determine storage space needed for baz, In this case, the only recourse is to put baz into bar lock-stock-and-barrel:

(define-alien-type nil
(struct foo
(bar (struct baz
(some int)
(other int)))

while ( still_stuck_on_c ) { ... }

2006-07-27T23:21:00.000-04:00

Obviously I'm not the only one that keeps reaching for the while loop. This from CL+SSL:

;;; Misc
;;;
(defmacro while (cond &body body)
`(do () ((not ,cond)) ,@body))

Closures

2006-07-27T19:27:00.000-04:00

As a novice in the ways of Lisp I'm constantly being reminded just how cool it is. In this particular case, I was working on some code for sending messages to our agent, and needed to provide a sync version of an async send-message. We do something similar in C++ and ended up with a special synchronous handler class, and about half a page of code. I was approaching the same issue in our new Lisp backend with some trepidation since I knew I was going to need to use some mutexes and condition variables, which I had never used in Lisp before. Including the time it took to actually read the docs and get reminders about how Lisp works from Jasko, I had something working in a matter of minutes:

(defmethod send-message-sync ((client client) command)
(let ((condition (make-waitqueue))
(lock (make-mutex :name "sync lock"))
(result nil))
(with-mutex (lock)
(send-message client command
#'(lambda (obj &rest args)
(with-mutex (lock)
(setf result args)
(condition-notify condition))))
(condition-wait condition lock))
result))

That little lambda tucked away in there closes around a lock, condition variable and result—all waiting for our eventual message return. It is not that I can pass the value of result into the lambda that is so powerful. But rather, when the lambda assigns a value to result and fires off the condition variable in whatever far-off place it ends up, back in the thread where I started and am patiently waiting my result gets the new value.

When you read about closures in texts on Lisp they seem so pedestrian. "Yeah, but I can do that with a functor in C++, pointers, references, blah, blah, blah." Then, you actually use them to solve a real problem, and the lightbulb goes off.

Remember the firewall

2006-07-25T14:09:00.000-04:00

If you are doing network coding it is very important to remember that you have a firewall turned on. I would like to say I have learned my lesson, but I'm almost sure I've been here before. Sheesh.

Fedora-on-Parallels

2006-07-24T23:45:00.000-04:00

I installed Fedora on Parallels on my mini at home so I could work on the networking parts of our server that were going to be very closely tied to the platform. Unfortunately, BSD and Linux have gone divergent paths when it comes to highly scalable networking APIs. Linux has epoll, and BSD kqueues. Since Rackspace supports Linux, epoll it is. I installed Fedora Core 5 on a Parallels virtual machine, and it seemed pretty nice.

Pretty nice that is except for the slight mouse lag, which started to get very annoying. Alas, there are no Parallel Tools for Linux yet as there are in Windows, which fixes the same issue when running XP as the guest operating system. My solution is to run X11 on the mini itself, and run:

ssh -X virtual-machine-ip-address

This works great, as long as you don't want to resize your emacs window. As soon as you do, you get a nice:

X protocol error: BadWindow (invalid Window parameter) ...

Google being the all-knowing, all-seeing eye, told me to add a -Y to the ssh command, and all was well:

ssh -X -Y virtual-machine-ip-address

Senior Software Developer

2006-07-24T19:34:00.000-04:00

Paragent is looking for a Senior Software Developer to help deliver our next product over the web. We are seeking an individual with experience developing large-scale, secure hosted applications using Lisp. Responsibilities will include development and maintenance of all server-side applications and infrastructure. In addition, this position will provide database maintenance, backup, load-balancing expertise and server configuration.

Requirements:
Bachelor's Degree
3+ years experience in developing hosted applications

Skills:
Good written and verbal communication skills
Web site scalability
Load balancing
MySQL clusters
Encryption (OpenSSL)
Linux System Administration
Backup and Data-retention
Languages: Lisp, SQL, C/C++
SBCL, UCW development experience a plus

Lead Support Engineer

2006-07-24T19:32:00.000-04:00

Do you have a passion for supporting customers? As a Lead Support Engineer at Paragent, you will be responsible for helping current and potential customers through training, installation and long-term support. Additional responsibilities will include the development and documentation of support procedures, as well as the participation in new product testing. You will work closely with both the development and sales teams to lay the groundwork for new product and feature rollouts. This position requires someone with excellent verbal and written communication skill with an outgoing, friendly and patient personality.

Requirements:
Bachelor's degree
3+ years of experience in a support or QA position

Skills:
Windows, Linux
Active Directory
Large-scale package rollouts
Scripting Languages (Perl, Python, etc.)
Unix administration and C++/Lisp programming a plus

Win32 Sockets II

2006-05-24T13:57:00.000-04:00

Just when you thought it was safe to go back in the water...

Here are a patch and file that reworks the sb-bsd-sockets win32 implementation to use sb-grovel instead of the hand-coded alien garbage I had written before. On the plus side, this means the compiler doesn't yell at us every time we touch a socket. On the down side, the sb-bsd-sockets.asd is a bit more convoluted to address the whole fd->handle wrapping that needs to be done, as well as making sure we load the winsock library before we do our groveling. There may very well be a better way to organize the win32 specific changes, so any suggestions would be welcome.

the cluster [193.5]

2006-05-01T14:20:00.001-04:00

yes, again with sockets [192.0]

2006-04-05T21:32:00.000-04:00

Looks like rudi was able to get the socket changes into the development cvs tree after an extended outage at SourceForge. What is more, Xof was able to get slime up an running on windows with it. There is always the chance that there is some tweak on your local machine that you made that you will never remember that makes it work for you and no-one else. Getting independent confirmation from a clean checkout of the tree is just confirmation that you aren't just imagining things.

Xof was kind enough to send along a tarball of the latest dev tree, since SF.net isn't going to reconnect the anon cvs side until Friday (go figure).

Apple posted a tool for loading XP on their machines. It even got front-page coverage on FoxNews and BBC. I was going to try it on the mini, but am not sure it is worth loosing the disk space for an environment that is two ctrl-presses away with the KVM switch.

Otherwise, no major news.

win32 socket commits [194.5]

2006-03-29T21:52:00.000-05:00

My IRC icon was bouncing in the dock this morning when I got into work, which means someone referred to my nick. I assumed it must have been from conversations the night before, but upon scrolling back realized it was actually rudi talking about committing my win32 socket patch into the repository. Happy, Happy, Joy, Joy. How many thousands of lines of code have I committed to Paragent's repository that are in a shipping product? Hell, I'm the boss, it has to be accepted. There is a certain warm-and-fuzzy feeling when you are able to contribute something that others find of value.

I didn't get into open-source because of all the free-as-in-speech fist-pounding. I enjoy it because I get the chance to interact with some very smart people. I hate to say it, but when I see that a piece of open source code has a GPL license it turns me off. Not because I think "gee, we can't rip this off like BSD or MIT licensed code," but because I assume that the original creator has some anti-corporate axe to grind. The only real argument that I see for a GPL style license is that it allows the powers-that-be to force companies to provide their enhancements to the code base to everyone else--when they get caught. While there have been a few cases of companies being called out on their GPL violations, I would like to know how many of those have resulted in some stunning contribution back to the source tree when they finally complied. My bet is that it has never happened.

An open source software project is not like some Wikipedia that can be edited by anyone at will. Any software project worth stealing is going to be complex. This requires the coordination of some smart people to organize, filter and commit changes. Programmers that want to contribute have to minimally participate in the community. You have to want to do this. Forcing companies to make source code available for applications that anyone can get the code for off of source-forge is just petty.

This reminds me of a parable:

"For the kingdom of heaven is like a landowner who went out early in the morning to hire men to work in his vineyard. He agreed to pay them a denarius for the day and sent them into his vineyard. About the third hour he went out and saw others standing in the marketplace doing nothing. He told them, 'You also go and work in my vineyard, and I will pay you whatever is right.' So they went. "He went out again about the sixth hour and the ninth hour and did the same thing. About the eleventh hour he went out and found still others standing around. He asked them, 'Why have you been standing here all day long doing nothing?' 'Because no one has hired us,' they answered. He said to them, 'You also go and work in my vineyard. When evening came, the owner of the vineyard said to his foreman, 'Call the workers and pay them their wages, beginning with the last ones hired and going on to the first.' The workers who were hired about the eleventh hour came and each received a denarius. So when those came who were hired first, they expected to receive more. But each one of them also received a denarius. When they received it, they began to grumble against the landowner. 'These men who were hired last worked only one hour,' they said, 'and you have made them equal to us who have borne the burden of the work and the heat of the day.' But he answered one of them, 'Friend, I am not being unfair to you. Didn't you agree to work for a denarius? Take your pay and go. I want to give the man who was hired last the same as I gave you. Don't I have the right to do what I want with my own money? Or are you envious because I am generous?'

The parable is actually about the last being first/first being last/everyone is equal, but that is not what I am interested in. It is also a telling commentary on how people can convince themselves that they have been "wronged" because someone got a better deal. The people who contribute to open-source projects do so because they want to, expecting no monetary reward. I certainly don't expect to be rewarded because I contributed some small bit of socket code to a Lisp project. So now some corporation comes along and uses said software. Have I lost anything? Have they taken away that warm-fuzzy-feeling, or the sense of camaraderie that I feel for everyone trying to make a better free Lisp? I contributed the code without any expectation of compensation. Why should I be jealous of the fact that someone else has found a way to make a buck off of it. There is already a perfectly viable method for making sure that individuals are rewarded "fairly" for their work - it is known as "closed-source."

Less politics, more coding.

Sheesh - how did this turn into a GPL rant? ... sorry about that.

SBCL devs gettin bank [194.5]

2006-03-27T19:57:00.000-05:00

An employee of ITA Software let it be known that his company might be willing to provide a little incentive to speed up the compile times of SBCL in the way of cold hard cash. So far there haven't been any public takers, but sbcl-devel is a pretty low-traffic list. Also, it could very well be that conversations have been taken offline. I found this post interesting because I have been contemplating a similar move to get work done on the win32-side of the house. Bringing the win32 port to parity with Linux would make life much easier for us. There are also some very specific things that are not high-priority items such as executable packaging and tree-shaking that it would be nice to have done. The economics of this are very interesting. We can either fork out serious money and just get a commercial implementation such as LispWorks or Allegro, or we can pony up bounties for people that are already inclined to do the work to bump it up in their prioritization. The benefits going forward in good-will alone make sense, and the bounties would probably cost less in the long run.

Our experience with Microsoft support has definitely opened my eyes to the economics of coding. Sometimes it is simply more cost effective to pay someone else to do the work. I think that all good programmers like to think that they can get anything working. Calling for reinforcements only makes sense if you do it soon enough to make up for the wasted time that would have otherwise been poured down a black hole. A Microsoft incident costs $245. Depending on what you pay your programmers your break-even point will be different, but if it is something you think will take more than two or three days to figure out, you will "save money" by having a Microsoft programmer figure it out for you. I had to put save money in quotes just then because most likely the work you are displacing to your support structure is being done by a full-time employee, and therefore you will be paying them regardless. In fact, you are not saving money in a way that can be tallied up very easily. The real cost savings comes from how much it is worth for your company to ship its product two weeks or even two days earlier. For it really to be cost effective, you probably need to be saving a week or so per call, and there has to be some definite benefit to getting your product out sooner. If you need that product for revenue, it makes all the sense in the world.

The trick is knowing when to throw in the towel early enough on an intractable problem to get the most bang-for-the-buck. Of course, these aren't double-blind experiments here. We only know how long something is going to take to fix after we've fixed it. This is probably a good reason to keep your bug-tracking items up-to-date with estimates of how long it will take to fix. After time, you can get a sense for whether you are a good estimator of bug difficulty. Use the bug-tracking system as training. Once you have developed the skills to accurately estimate what bugs will take the longest to fix, you could do a simple sort of all your current issues, take the ones that have long estimates, and figure out if it is something you could have done by bringing in the big-guns.

a Mini update [196.5]

2006-03-26T15:06:00.000-05:00

hmm, it has been a couple of weeks since I have posted anything. Typical.

The win32/sbcl work has been in a two steps forward, one step back kind of mode. I had all but two contribs building, and was even getting asdf-install worked out when I decided to update my tree from CVS. While necessary, it pretty much borked my tree. I had to do a clean checkout, and then move everything back over by hand to figure out where the breakage occured. Sockets are working again, but I seem to have lost the asdf-install build. sigh. Since those aren't strictly necessary for what we need in the agent, I am not going to worry about it.

Jasko got his MacBook Pro this week, and we spent a few days messing around with it. Tim got Windows XP installed on it. Hopefully someone will work out the video driver stuff. The XP install took several reformats to get right. Unfortunately, it does appear to have the infamous "whine." Tim kept complaining about the fan noise, so I suggested the mirror widget trick, and that seemed to solve it. I am assuming this is something Apple can fix in a software update. The laptop is otherwise pretty sweet. I can't wait to get one! We had fun playing with the photobooth software and making funny pictures of ourselves. Tim ended up with a great alien shot.

The rest of the laptop install was getting sbcl/slime/emacs/mysql/clsql/ucw up and running. The sbcl port is still in a, ahem "fluid" state right now. It mostly works, but seems to periodically fall over. slyrus and nfroyd from #lisp are working hard at getting the last issues worked out with the new lutexes. While sbcl has a great threading implementation on Linux, its dependency on the linux-only futex has created problems porting it to other platforms. The x86/darwin threading port is the main front in attacking this problem with the creation of a lisp futex - lutex. While our production servers will all be linux, we really need threads for the agent running on a variety of platforms. Getting lutexs working reliably appears to be the bottleneck for that happening. Once lutexes on x86/darwin are reliable, it can be used to get threads working on ppc/darwin and win32.

What this is all leading up to is the news that I bought a Mac Mini yesterday. In fact, there were a number of small excuses for getting one: a) I want to take my G5 back to work so I can get back to a Mac as my primary environment. b) I want to help get lutexes working to make sure Jasko is able to do lisp work under OS X on his laptop without going crazy. c) lutexes are the gatekeeper to being able to use a single platform to deploy our agent across linux/os x/win32. d) I was jealous of Tim's new laptop.

I got the core duo with 512MB of ram. It is a pretty sweet little machine, and should do a pretty good job. My only gripe so far is that I am getting the beachball of death a little too often when I switch between apps, but I am doing a lot of heavy lifting. My assumption at this point is that the 512MB of memory is borderline, and that I am seeing paging activilty as I bring different apps into the foreground. I am going to save up my pennies and buy some more ram for it in a month or two. I assume that most people aren't doing multiple sbcl builds while they are also installing applications, browsing the web, and generally mucking about.

What this thing really needs is a new monitor. I have my eye on the Apple 23". I need to set a suitable goal for getting that thing. Maybe when I reach my target weight...

socket() vs. WSASocket()

2006-03-14T22:36:00.000-05:00

It turns out there are more differences in socket() and WSASocket() than just the number of arguments they take. In working on the sbcl/win32 socket port, Alastair Bridgewater pointed out that there were some Unix'y assumptions about socket handles being equivalent to file descriptors. The rest of sbcl expects to be able to _read and _write with them. On windows, you have to convert between SOCKETs and FDs using _open_osfhandle and _get_osfhandle. The interesting lesson is that in order to do anything with the file descriptor you get, the original SOCKET has to be generated with WSASocket(), NOT just socket(). Spent a fair amount of time today figuring that one out. The good news is, we now have slime running on win32/sbcl:

on WSAWaitForMultipleObjects

2006-03-03T23:00:00.000-05:00

As we found out today, there are only so many objects that you can wait on in this function. That number is 64. If you try to wait on more than that, very bad things happen. In our case, our listener thread blew up. We were trying to handle more than 64 simultaneous non-blocking connection requests that had been accepted. The fix was a simple matter of making the events over 64 wait in a queue until we can handle them. We will check to make sure this behavior still works for us on the cluster before we roll it into the app.

While there was not a lot of visible changes this week, Tim and I actually fixed some persistent bugs (the above, an SQL injection issue that had been causing caching issues, an Installed Software problem, and the whole Updater not rebooting mess) as well as figured out some interesting new things (like extracting license keys from the registry). It actually felt pretty productive.

more sb-win32-sockets

2006-03-03T22:51:00.000-05:00

I've now cleared most of the WARNINGs and STYLE-WARNINGs from the win32 port of sb-bsd-sockets. You can download the reults here. I'm still having fits with a piece of code that initializes the windows sockets subsystem.

For some unknown reason, on windows, you have to run WSAStartup() in your application before you make any calls to networking APIs, or Windows gives you the cold shoulder. Unix isn't so demanding, so I'm hard-pressed to figure out why Windows has to be different. In any case, it is, and so I have to deal with it. It was suggested that I call the function in the initialization of a special.

In Lisp-speak, this is a dynamic, as opposed to lexical variable. The code looks something like:

(defvar *wsa-is-initialized* (wsa-startup ...))

Of course, this doesn't work for me. wsa-startup is in the package sb-bsd-sockets-internal, while the code you are seeing is in the sb-bsd-sockets package. While the same file is sprinkled with internal socket functions, my wsa-startup functions get a nasty ERROR from the compiler. I am assuming that this has to do with the whole eval-when business that I must claim I have absolutely no clue what it is really doing.

[UPDATE]
It was a package export issue. I wasn't exporting the wsa-startup symbol from the internals package. This was confusing because I thought I was doing the right thing by prefacing it with win32sockint:wsa-startup. It turns out this only works for exported symbols. For unexported symbols, use two colons. win32sockint::wsa-startup. Thanks go to the #lisp crowd for the help on that one. At least you CAN get to the symbol. I was becoming more discontent with C++'s whole public/protected/private pseudo-encapsulation, and Lisp has only solidified that view.

sb-win32-sockets compiling

2006-02-25T23:03:00.000-05:00

I've been able to get all of the sb-bsd-sockets code (ignoring tests.lisp for now) compiling under win32. That means it all works right? Well, this only gets me to the point of having it compile without falling over. Now I need to go back and put in all the alien function definitions that I know are missing.

It is at least something.

You can see the snapshot of this writing here

End of an Era

2006-02-22T22:33:00.000-05:00

The 80's are officially over. In highschool, I remember visiting the high-end AV stores with their plush theatre rooms. These were the days before plasma screens, and the state of the art was the Laser Disk. An LP-sized monstrosity that seemed to be always demoed using one movie - Top Gun. Ohh, how I loved that movie. I just knew I could bag all the chicks if I only had a Laser Disk and a copy of Top Gun. For that reason, this story made me very sad today.

Feed Reader Update

2006-02-18T23:19:00.000-05:00

Well, progress is being made on the rss aggregator front, which means little to no work on the Lisp front. At this point, the basic infrastructure for downloading feeds periodically and putting them in a context menu in the notification tray works. I even have a snazy animation when processing feeds. It is actually a functional aggregator, as long as you are willing to hack the registry every time you want to add a new feed. Obviously this won't work. I still have to add the ability to easily add/edit/remove feeds. Oh, and parse atom feeds as well. One sticky issue I have run into is HTTP 302 redirects. Jasko was able to break the RSS processing with the first two feeds he gave me. The first because it had an XML comment node where I didn't expect (easy fix) and the other caused a 302 redirect. Blech. That one I am still working on.

Speaking of Jasko, he has been cranking on some very cool Lisp tool stuff that is going to make our lives (and I would imagine a number of other non-old-school Lispers) much better. Speaking of making life easier, SBCL can now create stand-alone executables.

Hopefully I can get back to sb-bsd-sockets hacking on Windows once I get this reader squared away. Things are falling into place.

Right Pew, Wrong Church

2006-02-12T22:22:00.000-05:00

this post does a very good job of covering some of the shifts in what is considered good programming practice in C++. I noticed the shift to non-member functions over the last couple of years, and generic programming has obviously been taking over for some time. I now completely understand why lispniks are so bitter and cynical. What the C++ gurus are coming around to is the fact that single-dispatch message-passing OO is an inferior model to a generic function OO - the same OO style of CLOS - the Common Lisp Object System.

Windows Feed Aggregators

2006-02-12T11:05:00.000-05:00

After careful consideration, I have come to the decision that RSS/Atom readers on Windows all suck. Actually, SharpReader isn't too bad, but it necessitates a 30MB .NET runtime download if you don't already have it. Based on our experience deploying Paragent Manage in the wild, .NET just isn't widely deployed even now.

Why all this interest in an RSS aggregator now? We are evaluating some future development using a web-based model, and want to allow our users to get timely information through RSS feeds. I wanted to see if there were some good RSS readers that our users could use. All of them are just overkill for our needs:

A small, self-contained executable that runs in the system tray
An icon that changes to indicate the severity of new items
a context menu that lists the new items
pop toast on new entries

That is pretty much it. All the readers we know about use a three-pane email style interface that allow for reading all entries within the reader itself. For showing alerts, this is overkill. Once we have these features, we are not far from my personal ideal news reader. Add the following, and I would use for all my feed needs:

a simple interface for managing feeds
a very simple display of all the current items with their descriptions

So, I have been working on a new aggregator. Watch this space...

Knowing When to Ignore Your Users

2006-02-09T17:55:00.000-05:00

I like reading Joel on Software. We use FogBugz at work, and use many of his suggestions on specifications, painless schedules, etc. However, when I read this I just said "huh?"

"Understand enough about time zones so I can enter a flight. Flights from New Zealand to Los Angeles arrive before they departed. It's confusing but it's true and if I can't enter them properly on my calendar I'm back to typing itineraries in Word."

So, apparently Mr. Spolsky is making so many flights between New Zealand and Los Angeles that not being able to record the departure and arrival times properly makes these systems unusable. Not only that, but he must also believe that a sizable enough portion of the potential users of these sites will also be in this quandary. I know that Peter Jackson has made New Zealand popular, but let's keep this in perspective.

Users will always have a laundry list of "must have" features that they are asking you for. The trick is knowing when to listen, and when to ignore them. Maybe if Mr. Spolsky were a paying user of one of these services, they might think of accommodating him...

Paragent Manage 2.1 Release

2006-02-08T00:28:00.000-05:00

We got Paragent Manage 2.1 out the door Monday, new downloader and all. For those not familiar with Paragent, we have been working on a p2p development platform that combines a traditional distributed hash table overlay network with a functional-based scripting language. Originally, we were going to try and market the development platform, but soon realized that was a pipe dream, so we decided to see what kind of application we could make with it ourselves. Our first product is a desktop management application that works without a central server. Most p2p apps we have seen out there have either tried to legitimize file-sharing in some way, or provide an easier-to-use communication medium. We like the idea of using p2p's ad hoc, decentralized nature to help provide business applications that have traditionally been a real PITA to deploy. Download the trial, and tell us what you think, good or bad.

sb-win32-sockets

2006-02-04T10:59:00.000-05:00

I've been working very slowly on getting sb-bsd-sockets working on the sbcl/win32 port. It actually isn't too bad. The biggest hurdle is my own lack of understanding with how sb-alien works. sb-alien is the foreign function interface for sbcl that allows talking with external C libraries. In this case, I am making calls to ws2_32.dll - the winsock2 implementation on Windows XP and the like. I've been slowly building up the definitions for the alien interface for all of the underlying functions like socket, bind, accept, recv and the like. I was able to send my first message from an OS X box to the win32 sbcl instance last night. To give you an idea how rough it is right now, I was looking at what I received one character at a time with (deref (deref buf 0) 0).

getting ready for 2.1

2006-02-01T22:48:00.000-05:00

Looks like we finally worked through the final downloader issues, and the new agent has been running pretty smoothly on the cluster for over two weeks. Tomorrow we should run final regression tests on the new build, and release 2.1. We are not making any radical changes, and many of the usability issues that we would like to clean up will need a much larger rewrite. That will have to wait. For now, it polishes the rough edges, and makes the out-of-box experience a little nicer.

Check Paragent's web site in a couple days, and grab a copy.

New Downloader Almost Complete

2006-01-30T00:17:00.000-05:00

I have been working on a little downloader application for our software. You download this 200K app, and it walks you through checking for .net, downloading and installing that if necessary, then downloading and installing the other parts of the application. In our case, this is the Paragent Framework (our agent) and Paragent Manage, the GUI console. This was all raw win32 GDI type stuff. I'm pretty happy with how it turned out. I need to go back and review all of the corner cases to make sure we are doing the right thing for the user.

What is means is little to no lisp work this weekend. I was able to start making foreign function calls to the ws2_32 library today (winsock2). I got as far as initializing with WSAStartup, and creating a socket. I knew I was really talking to the library because I was getting real winsock error codes back. When you see 10092, you know you can only be talking to Windows...

I should be able to take a stab at sb-grovel now that I have at least a rough understanding of sb-alien. I need sb-grovel because there are still some things I don't understand about creating unions with the sb-alien commands. You can't get very far (read: bind) before you need to use in_addr, which is a union. I think I missed the class on why anyone ever bothers with unions. In any case, crafting the right combination of struct and unions specifiers was beyond my keen. sb-grovel is SUPPOSED to do all that for me.

double buffering

2006-01-26T11:14:00.000-05:00

Now, for something completely different (i.e. NOT lisp). I'm working on a new mini installer, and wanted to create a small ~200KB stand-alone executable that could be quickly downloaded, and then used to download and install the application itself. For windows, if you want to be small, portable and pretty, you are basically talking about some hard-core Win32 GDI hacking. No GDI+, no MFC, and certainly no .NET WinForms.

To make things look good, everthing is pretty much painted by hand. This means I have to deal with issues of flickering, and the way to do that is through double-buffering. In GID, you do that by creating a memory contect, and drawing into THAT context behind the scenes. When you are ready to show the results to the world, you then BitBlt the final image onto the screen.

hdc = BeginPaint(hWnd, &ps);

memDC = CreateCompatibleDC(hdc);
memBM = CreateCompatibleBitmap(hdc, 512, 316);
SelectObject(memDC, memBM);

Draw(memDC);
BitBlt(hdc, 0, 0, 512, 316, memDC, 0, 0, SRCCOPY);

DeleteObject(memBM);
DeleteObject(memDC);

EndPaint(hWnd, &ps);

Note, just creating a compatible DC is not enough. This creates a 1x1 drawing area. You actually have to select a bitmap INTO it to size it. You can imagine how long it took me to figure this little gem out. Also, note that you are not creating the compatible bitmap using the new memory device context, but using the screen dc because that is what you want your bitmap to be compatible with. Once you have your memory device context all set up, you can pass it to all of your drawing functions like it was the screen, and then you are ready to go, just copy it in one BitBlt. No more flicker.

custom setf forms

2006-01-24T23:04:00.000-05:00

Norvig presents several ways of creating custom setf forms, but none of them as elegant as Grahams:

(defun get-route (id table)
...)

(defun (setf get-route) (rt id table)
...)

The other options are something like the macro defsetf, or even more complicated beasts. I ran across the above need in code I was writing, and happened to be reading Norvig. He shows the ugly (to my mind) way of doing it, and I had to pull out my ANSI Common Lisp by Graham to find the pattern that had been tickling the back of my head.

There is an annoying downside to the setf form in something like our routing table when it comes to testing. Our routes are objects with several properties, including id, address, proximity, etc. I have a convenience function called make-random-route, that generates a route for me with all the slots, um well randomized (I'm figure you could have guessed that). Previously, I could insert a test route with (insert-route (make-random-route)). I wanted to use the setf form, since that seemed to follow lisp conventions better. This means that to do the same means

(let ((rt (make-random-route))
(setf (get-route (id rt)) rt))

Practically speaking, in the real world, we would be creating a route object before we would ever consider inserting it, so the let form is not really an additional burden. I just happened to run into it because of the testing.

apply'ing a list

2006-01-23T21:57:00.000-05:00

I was wanting to create a convenience function for a class, and wanted to pass along &key parameters to the make-instance function. I couldn't figure out how to use a &rest argument, which puts everything in a list, and I need to get the keywords. My first pass was to use a macro:

(defmacro make-routing-table (&rest args)
`(make-instance 'routing-table ,@args))

worked, but ugly... much better is:

(defun make-routing-table (&rest args)
(apply #'make-instance 'routing-table args))

this from norvig.

Using let outside functions

2006-01-21T21:55:00.000-05:00

I was writing defvars for functions that needed to share state information.

(defvar *splitlevel* 0)

(defun foo ()
(do something with *splitlevel*))

(defun bar ()
(do something else with *splitlevel*))

I've now found the use of let to wrap up several functions at once:

(let ((splitlevel 0))
(defun foo ()
(do something with splitlevel))

(defun bar ()
(do something else with splitlevel)))

&optional labels

2006-01-20T23:13:00.000-05:00

I'm trying to document different lisp coding conventions as I run across them. For instance, I was writing recursive functions like:

(defun foo (x &optional (y 0))
(progn ...
(foo x (1+ y))))

This just felt wrong - very wrong. I'm trying to make it more convenient for the caller (me in some future coding life, like say 60 minutes later) that doesn't care about properly initializing the accumulator or index. If since come across the following form, which just feels more comfortable to me:

(defun foo (x)
(labels ((find-foo (x y)
....
(foo x (1+ y)) ... )
(find-foo x 0)))

It is more verbose, I know, but somehow that (defun foo (x) ... ) just seems so much cleaner. This is of course predicated on the notion that you will NEVER want to call foo with an intermediate y value directly.

Piling On

2006-01-16T21:18:00.000-05:00

I just love an underdog, what can I say? Give me a new technology outside of the mainstream, and like some geeky magpie, I just can't resist the temptation of a shiney new toy. Linux, BeOS, OS X... I've had love affairs with them all. It turns out my latest love affair is not so new and shiney, but like the NBC tagline: "If you haven't seen it, It's new to YOU."

Yep, I'm really getting into Lisp.

On Apple's (not so) Mighty Mouse

2005-08-05T20:51:00.000-04:00

I got a chance to stop by the Apple store yesterday and check out their new-fangled two button cum trackball cum squeeze button contraption they call the Mighty Mouse. I was prepared to be blown away. I was expecting a well crafted input device that felt solid in the hand, with assertive input response and tactile feedback. What I got was an ergonomic nightmare that felt no more solid than any number of substantially cheaper alternatives.

First, lets be entirely clear. This is not a two button mouse. It is a single button mouse, with the same scissor action of Apple's old mouse. Instead of having two physical buttons, there is just the entire-mouse-as-a-button implementation. What Apple has done is to add sensors that tell which side of the front of the mouse your fingers are touching when you press that single button. If there is a finger on the left when the button is clicked, it is a left click. If there is a finger on the right - and only the right, it is a right click. This last point is very important. To get a right click to register, your index finger can't be anywhere near the surface of the left side of the mouse. What this means in practice is that instead of pressing the right mouse button, the real action is to lift your index finger, then press the right side of the mouse with your middle finger. This formula, while aesthetically appealing, is a complete ergonomic failure. They have made right clicking twice as hard with the mouse. More importantly, they have made you THINK about using the mouse. If you make a "right" click, and have accidentally left your index finger too close to the surface, you get a left click instead. The result is that you find the computer not doing what you expect. Apple has broken the cardinal rule of interface design with this mouse.

Second, lets talk about the wheel. This is sad to say, but at this point, I still have no idea whether that little nipple on the front of the mouse actually moves. When I started using the first mouse, I thought I had gotten a dud. I was convinced that the little ball was broken and wouldn't roll. But sure enough, the web page was moving up and down (I can be slow like that some times). At this point I thought that maybe it was just sensing my finger moving over the surface of the ball. Once I starting using it though, I had the sensation that it was moving. I'm not sure if it was my brain tricking me into thinking it was moving because I was getting feedback from the screen or not. Like I said - I still don't know if that sucker actually moves. Once again, the mouse was disconnecting me from what I was trying to get done on the screen. As far as the fact that it is a ball that lets you move sideways, I found it to be less than appealing. Our fingers just aren't designed to have as much control in the yaw axis like that, and the range of motion is so small, that you spend a lot of effort wagging your finger back and forth. That is my other beef with the ball - it is too small to really use effectively, there is a very small amount of physical range of motion to translation to scrolls on a giant display.

Third, and finally, the squeeze buttons. I'm sure you can guess by now where I am going with this. a) they are positioned in such a way that you have to work to squeeze them. b) there is no physical feedback, so I found myself giving the little sucker a death-grip each time. Again, the physical device was making me work too hard to get the result on screen that I want.

The frustrating part of this mouse is that it almost feels right. In fact, I really worked hard to like this thing. I wanted to throw $50 at Apple. At the end of the day though, it was a death by a thousand cuts. As I went through a number of tasks on the display machines, I kept being jarred out of the flow by little problems. My right clicks wouldn't register at completely random times. I had to work to make fine scroll adjustments. I had to reposition my had every time I wanted to squeeze. Over time, I knew there was no way I could use this mouse. the $19 Microsoft Optical mouse is still the best thing going.

If statements inside out

2005-07-25T11:19:00.000-04:00

NOTE TO SELF: I am doing if statements in reverse. Instead of

bool error = someFunc();
if(!error) {
// do the next thing
} else {
// do error corrective actions
}

I should be doing

bool error = someFunc();
if(error) {
// try to fix or bail
}

// go on

This makes much more sense to me, and I'm not sure why I was doing it the other way before. I guess it is the easier way to think about it initially. The original way I was handling errors meant that a) I was moving the error handling away from the place where I actually caught the error, and b) was nesting things. a lot. The second form means that the normal flow of execution stays at the top level. This means that the eye can follow it much easier. Also, the error blocks are indented, and noticably different. You can't stick with the "single return point" mantra, but I've never done that anyway (it has never made sense to me), so nothing changes for me.

Apple -> Intel

2005-07-16T23:52:00.000-04:00

I have been following as much commentary on the Apple to Intel story as I can stomach. There appears to be one common thread among all of the punditry, and that is the search for the reason Apple made the switch. The latest I read was rehashing early ideas that the move was based on the need for Apple to get on the Intel-cum-Microsoft-backed DRM bandwagon known at Trusted Computer Group (TCG). Apple just doesn't stike me as a company that would make a major processor transition just for the sake of compatiblility. Lets face it, if there were enough motivation to do it, it would have come long before now.

Of all the stuff I've heard, the most likely scenario in my mind is:

a) Steve Jobs had planned to ditch PPC as soon as he came on board, but realized that he had to get a certain percentage of existing Mac users transitioned to NextStep...I mean OS X...before he could make the switch. Jobs had already mentally switch from the PPC back when he was running NeXT, so it was a no brainer when he took the helm at Apple.

b) There were a whole host of issues that just built up to make the move necessary and or compelling. The hardware R&D costs of building PPC systems, suppliers that couldn't feed them fast enough chips, Intel dumping a bucket-load of co-marketing cash their way, an operating system that was agnostic to processor type, The popularity of the iPod to give them the financial capability to weather the transition, The desire to get volume discounts on xScale chips for their media devices, the pressure to adopt more effective DRM... the list goes on and on.

All of these at one time or other over the last couple months have been presented as the reason Apple made the move. More than likely, all of them combined were the critical mass necessary to start the chain reaction.

WSAEWOULDBLOCK

2005-06-28T00:47:00.000-04:00

I've been working on the new networking code for some time now, fighting a number of integration issues with the new remote routing infrastructure. While extremely powerful, it is also very complex to debug in a systematic way.

My illness didn't help any on the progress front. So anyway, today I finally got a chance to spend some good time going over the code. During one of my breaks, I happened to read this post over at Project Aardvark, which I have been following (yes, I am a Joel on Software fanboy).

It turns out, I wasn't checking for WSAEWOULDBLOCK either. Finding the problem still took some serious effort. I'll explain more tomorrow. Right now, I am going to bed.

2005-06-26T23:47:00.000-04:00

Bronchitis is no fun. Spent Thursday night - ALL Thursday night, vomiting. I am only now feeling myself.

What this really means is the loss of 3-4 days of productive work which I can ill afford. I guess it just means a few more late nights.

Hopefully, I will have more to talk about this week.

Question - does the CLOSE_WAIT state suck, or is it just me. As far as I can tell, it means waiting however long your OS has set the timeout before you can relaunch your application.

2005-06-19T10:52:00.000-04:00

I'm on a number of mailing lists that I use to keep myself up to speed on different technologies. At any point in time, I am probably subscribed to between 5 and 10 lists. The Apple PPC to Intel transition has amplified a problem that has been bugging me lately: Why on God's green earth can these people not take 60 seconds to google their questions or statements of fact?

One poster to the macox-talk list had a list of about 5 technical 'issues' he had with Intel chips. In 5 minutes I had either found solid counter arguments, or completely refuted what he had to say (such as no 64-bit intels, no Intel chips supporting NX flag, big-endian is right, little-endian is wrong)

Just today, they have been talking about file system support, and whether the x86 version of OS X would be able to support HFS+. If they had taken 60 seconds to use that little textbox right in their Safari window, they could have found that the x86 version of Darwin, upon which OS X is based, already supports HFS+. Sheesh.

I make a point now of always providing a hyperlink to the actual data that I have found from Google. I think I am going to start including the actual link to the Google results as well, to reinforce the point.

At least Tim and Eric know to hit Google before asking me anything, thank God.

2005-06-15T23:28:00.000-04:00

Still don't get the point with Dashboard. Can't remember to bring the damn thing up. Maybe it is just something to get used to. It is not like I use expose that much either. I guess that is what I get for moving between macs and window boxen during the day. Instead of being good at either, I end up with some LCD of their interface functionality.

I must say that using Windows is just PAINFUL once you have gotten used to a Mac. Are there issues with OS X? Sure - you can find any number of rants about how Apple has thrown out many established UI axioms for the sake of a pretty demo (the Dock for one). Still, these are rants coming from people with a long history of using operating systems with good UI. Coming from Windows, they are all insignificant.

What really blows me away is just how colossally crappy application interfaces are in Windows. I'm not talking about the default Widgets. I'm talking about basic aesthetics of the layout, and what people do with those Widgets. I don't believe that Windows programers don't care about these things. We certanly do.

2005-06-14T20:50:00.000-04:00

While using my Dual 2.0 GHz G5 PowerMac yesterday, I was distrurbed to notice that 1GB of ram was missing, and I could no longer import video from my camera. Reseating the cards fixed both problems - wierd.

My Brother-in-Law turned me onto the RSS Screen Saver in Tiger. Too sweet.

I'm starting to get mixed feelings about Dashboard. Why do they all have to be in their own "space" where I cannot mix them with normal windows. Many of the Widgets seem to make sense when they are a part of your visual field all the time. What good is a nitification widget when you have to click something to see that its state has changed? That would be like having an alarm clock that you have to keep checking to see if it has gone off. I like the widget organization, but let me change whether they appear on the regular screen. You know - like ... shhh ... (konfabulator) ...

2005-06-13T21:36:00.000-04:00

Well, it turns out my source third-hand was on the money, and it survived the rumor chain until it got to me. I think the fact that the secret was just that for as long as it did pretty amazing. On the other hand, the WSJ did break the story a while back. Too bad everyone decided to ignore them. It jsut seemed so absurd.

The forums are still cranking on the news, with some pretty crazy responses for die-hard mac fans. Its not like the PowerPC was even created by Apple. Why all the loyalty? Most Mac-heads wouldn't even be able to tell you what "endian" means. Sheesh.

It has gotten me motivated to get my PowerMac re-integrated into my workflow. I've been punching away at Windows code for so long, I haven't even had the damn thing turned on. Thankfully, the new kernel code makes the platform specific code needed even smaller. It should be pretty easy to get it up and running.

2005-06-04T17:32:00.000-04:00

Apple is switching to Intel. That is the word on the street as of Saturday afternoon. Monday we are supposed to know for sure.

Most people seem to be concerned about what the switch to a new processor will do for developers. From the point of view of a developer that works on cross platform applications, the fact that there will be a different processor underneath the OS X operating system really doesn't amount to much. Any developer worth their salt knows about endianness issues, since we've had to deal with those in any networked application. We just don't do that much low-level work any more. The OS is supposed to abstract all that stuff, right? I do feel sorry for all the in-house AltiVec experts out there. At least there should be plently of work in the console business.

I would love to see Apple switch to x86. It makes many more interesting things possible. But frankly, Apple putting an x86 in their boxes wouldn't really change my world, but here is what would:

A) Apple provides a version of OS X for non-Apple hardware. There was recently a reference to the fact that Steve Jobs wants to overtake Windows market share. If he is serious, he has to know this isn't going to happen all through Apple hardware. Throw an Apple supported version of WINE in there, and people will have a choice in operating systems independent of the applications they want to run. Will there be compatibility issues? Sure. But nothing worse than what Longhorn will create. If Apple comes out with a PC version of OS X, we will have a level of operating system competion the likes of which haven't been seen since OS/2, or before.

B) In addition to their own x86 OS X, Apple delivers the Cocoa framework for Windows and Linux. As a developer, things would get really interesting. I believe that Microsoft has put themselves in a very critical position right now vis-a-vis frameworks, and the developer mindshare. Longhorn is late. Way late. People that have bought into C# and WinForms, are ALREADY staring down the barrel of obsolecense with Avalon. Given the choice, I think developers will flock to Cocoa. Apple (ala NeXT) did it once with OpenStep. They can do it again. Internally, this seems like it would make sense for Apple as well. I am not entirely clear what applications in their iLife/iWork sweet are Cocoa/Carbon, but having a single framework to deliver both OS X and Windows applications would have to cut costs.

If Apple only announces they are putting x86 chips in their boxes, I'll still be happy Monday afternoon. If they come out with a Windows Cocoa framework, I will be extatic.

2005-05-26T23:19:00.000-04:00

I have a bone to pick with Computer Science departments. To put it in context, we have spent an inordinate amount of time at Paragent out interviewing soon-to-be graduates of undergraduate CS programs. The number of students goes well over 100. I'm sure that you might be expecting me to complain about the fact that none (or very few) of these programmers have real software development skills. Actually, that is okay with me. Hell, I still don't have many of those skills. I don't expect them to have those skills as undergradutes. The people that love to program will have picked up many of these skills on their own.

As an aside - it is amazing what you can tell from an interview. When you are asked if you have any programming projects outside of school, don't tell me "My classes give me enough programming to do." If you don't enjoy programming enough to want to do it on your own, you will never enjoy doing it for work, and will never do what it takes to be a great programmer.

Back to the main point of the rant: My beef is with the fact that these people THINK they are ready to be employed as software developers. I don't blame them, I blame the CS departments. Ask a CS professor why they don't teach good debugging, source code control or real language skills, and they will tell you that it is the Computer SCIENCE department. What we are asking for is more professional skills that don't belong there. That's fine, someone just forgot to tell the students. The fact is, 99.9% of all CS majors go into it to get jobs in programming. The profs know this, and are selling them a false bill of goods.

2005-05-14T21:25:00.000-04:00

Is it me, or does there seem to be something missing in the whole XBox 360 launch? Like perhaps. games. I've not been following it that closely, but from a cursory glance at the reviews, I haven't seen a single reference to any of the games that will be coming out for the 360. Seems kind of important. Of course, apparently they haven't even decided yet if it will be backwards compatible. Strange.

2005-05-14T10:50:00.000-04:00

Just as in the fashion industry, there are fads in programming. What was once in vogue eventually is thrown away as unproductive, too confusing, or just plain un-cool. Unlike the fashion industry, old styles don't go out with a whimper, they often go out with a bang. This is usually in the form of some vitriolic screed by the proponents of the "One New Way." Sometimes the object of derision is a big thing - like pointers, and sometimes it is a small thing - like operator overloading. A classic example of this is white space. The parallel with the fashion industry could not be more simple, since white space is entirely about how code looks. The "white space matters" paradigm was in fashion, out of fashion, and now, with Python, back in fashion. Sheesh.

Sometimes, something comes along to remind you why a feature like operator overloading, while out of fashion, is still darn cool. While complaints against operator overloading are justified in many instances, in a domain specific area such as parsing, where there are well known forms (EBNF), operator overloading can be simply sublime. The Spirit framework is just such a case. Instead of thinking of it as C++ - I think it is much better to think of it as a way to easily embed a domain specific language (DSL) right within your C++ code. If you do any C++ coding and parsing, you owe it to yourself to check it out.

2005-05-12T00:11:00.000-04:00

Lesson for the day - No matter how fancy-dancy your programming language, it still must insterface with the real world if it is going to be of any use to anyone. That is when all the nice little abstractions start falling apart. Joel Spolsky calls this "The Law of Leaky Abstractions."

My developers love C#. They want to do everything in C# because they see it as so much easier than C or C++. Case in point - networking. C# has all these nice networking classes NetworkStreams, BinaryReader, etc. that make it soooo easy to create little networking applications. The problem is, as advanced as C# is, it cannot ignore the reality that is network programming. Things break down, packets are lost, there are ghosts in the machine.

In particular, just because you send 1024 bytes using

writer.Write(buff, 0, length);

it doesn't mean that

reader.Read(buff, 0, length);

on the other end is going to always give you 1024 bytes in one go. BinaryReader::Read has a return value that you cannot ignore - it is the actual amount of data it was able to read. Basic stuff, but we tend to forget the basic stuff when we think our programming lanuage is so much easier to use. I fell into the trap myself, and spent an hour trying to help debug the above networking code, assuming that C# was just taking care of all the little niggling bits. At one point, I was convinced the bug was in the BinaryReader class. New languages don't eliminate the need to remember the most basic cenventions, such as: Don't ignore return values!

2005-05-11T10:14:00.000-04:00

One of our goals at Paragent is to try and create a community around our products. In part, this means creating products that have the features that are compelling, and benefit from community involvement. Konfabulator is a great example of software that enables the development of a community. It is a part of our development strategy to add these kinds of features to viewport, and I'll have more to say about that in the future.

There is another way to create a community around a company or product, and that is through transparency--make people feel like they are "inside" the process, so that they take ownership of the product. At Paragent, we do this in many ways, including focus groups, early adopter programs, etc.

This blog is another. While I first started (and stopped) writing this blog a long time ago, for entirely different reasons, I hope to start using it as a means of connecting the inside of Paragent to our existing and potential customers. Hopefully this will allow people to see what kind of a company we are, and what crazy ideas we have for the future. Stay tuned...

2003-03-05T19:36:00.000-05:00

Holy cow. Has it really been a month? Shit. and I am still on the same damn chapter. Now it is going to be 1500/day or I am sunk.

2003-02-04T00:43:00.000-05:00

long-time no-blog

$1 up in cards tonight. It was a close call on the +/- column. I was actually down most of the evening. I realize I really need to get a job. I sweat the $10 I have in the game. That should be chump change, and I treat it like the last dollar to my name. I feel like it is really K's money more than mine.

1000 words/day is my goal. So far I have not written 1. I need to resynch some stuff in Scriptorium WRT how new nodes are created, and I can plunge headlong into it.

2003-02-01T11:03:00.000-05:00

Columbia Lost - God speed...

2003-01-28T23:55:00.000-05:00

Must resist temptation to add fancy features.... I've got some neat ideas for a commandLine. Primarily, there should be an easier way to navigate around documents. Raskin's THE interface is IMNSHO not really that new. Frankly, it sounds like he is arguing for a move to an emacs or vi interface. Pretty funny actually. Scriptorium is going to have a command-line mode for better navigation, search/find, scripting, etc.

Hmm, stumbled across Charlie Rose. This should be good...well, it was going well until kinsley opened his trap.

Anyway, back to interfaces. Ideally, you should be able to pretty easily move around a document and do everything you would like without ever leaving the keyboard. A keyboard is designed for two hands. Moving one over to a mouse (or down to a trackpad) is just bad news.

On RSS Feeds:
some active debate on whether RSS feeds should be integrated into a browser. Part of the problem is the different ways people use feeds. Some sites have entire messages (like blogs) while others just have abstracts of articles.

RSS (and the crop of tools that have been developed to support it) are really great for blogging. Otherwise, I am just using it to know when a website has been updated, and what the new info might be. It would be nice if Safari could simply use rss to let me know when a website has been updated.

My biggest problem frankly? it takes up a spot on my dock for an application that is only SUPPORTING Safari.

2003-01-28T00:37:00.000-05:00

It turns out the Ars forums have a way of sending you IM or email when a topic you are interested in is updated. At least in theory. I haven't gotten it to work yet :)

175 feels like the limit of an infinite series at this point.

Thinking of a DTD -> ObjectiveC -> XML class generator. What is even cooler, I think you should be able to do this on the fly. I am still working out how to add instance variables to a class. The biggest problem is that I REALLY don't know XML that well. Ohh, I know all the basic tags, etc. etc. but I don't know what is possible: namespaces, includes, schemas, etc. As always, getting something basic working is pretty simple. Making it complete is MUCH harder. I have a feeling that I might be duplicating some of the EOF stuff if I dive into it. Ultimately, this could replace the current Scriptorium XML back-end. However, that is a HUGE side project that I frankly don't have the time for.

Node insertion is looking pretty good. The trick to rapid application development like this is to continually be refactoring how you see things working. I have fiddled with the APIs many times as I try creating something that looks good. I don't get too tied to an implementation, and am willing to pull out large chunks of code if there appears to be a cleaner way of doing things.

Don seemed genuinely impressed with Script. I think he sees the chance to mold the perfect editor.

I have decided to put a calculator in the interface. I am tired of having to pull one up...

2003-01-26T23:32:00.000-05:00

Davit Hyatt (one of the persons behind Safari) has discovered NetNewsWire. What is the biggest advantage of rss? productivity. plain and simple. Instead of going through all 100 or so web sites every 30 minutes to make sure I have fed my addiction for news, I can just glance down at my NetNewsWire icon, right-click, and see what has changed.

Anybody remember the name of that application that was all about push? PointCast she was called. It was going to be the new rage.

I just wish some of the forums I am on were able to use something like rss. I hate having to check to see if there are any messages, especially now that I am actually posting.

2003-01-26T11:11:00.000-05:00

man, I keep forgetting to make entries. I guess my life is too boring to keep up with a blog...

179.0 4lbs to go. (sad that I am looking back at my entry for the 21st, and it was 180.0)

Finally got am e-mail from my editor Viki, asking when the book would be done. Looks like April is going to be the month. She seemed okay with it, although I feel pretty damn bad about letting it slip. However, I am really motivated to get Scriptorium to a point that I can use it on a day-to-day basis.

I am thinking a new look at the outline is in order. I have two chapters completely done, and they come out to 100 pages using the default DocBook output. That is a 1/4 of the original booklength goal! I had divided it into ~13 chapters, which would mean the MS will be pushing 700 pages. I need to get a better handle on the organization as well. I was originally going to submit chapters one at a time, but am thinking that it might be best if I get most of the book on disk, and then have the flexibility to move parts around to make the most sense.

2003-01-21T23:11:00.000-05:00

180.0 -- 5 lbs. to go. I can almost smell it! It is a little depressing to look in the mirror though. I'm afraid there is going to be a little excercise between me and a decent body. Well, I knew 175 was really at the upper end of my ideal range anyway.

I have FINALLY gotten the new element preferences panel working in Scriptorium. There are some VERY weird errors that crop up when you don't get your retention strategy correct. Because of the multithreaded nature of the Obj-C runtime, bad accesses are difficult to trace. At one point, I had a valid pointer, but a null isa! Lesson learned?: don't return autoreleased objects from copyOnZone:

2003-01-21T00:52:00.000-05:00

Cards tonight. We tried out Jeff's new tourney format. Had loads of fun. Came in second. We then played for another couple hours. I think I was $15 ahead when the night was over. Note to self---stick to cards, no craps.

2003-01-20T12:54:00.000-05:00

180.5--sooo, close.

2003-01-20T01:22:00.000-05:00

seems like I am time-shifting forward. I don't seem to get around to my entry until past midnight, so it is posted as the next day.

I am starting to spot a trend in my development. Today, I started messing around with how the ElementTagCells, simplifying how to create new cell types as Scriptorium progresses. Of course, this means I jacked with the preferences stuff, which I most certainly did NOT want to mess with. instead, here I am at 1am, still not finished with my tag preferences completed. However, I am much happier with the way the new preference pane is going to work. The fact that visible document structure (like tags, content, etc) does not have a one-to-one correspondence with tree nodes causes me a little consternation. I am also still trying to figure out how I want to do comments. Hold on... I think I just had an idea. (). Comments are weird. They are like really big tags.... same goes for CDATA.

Met Jeff and Eric at Scotty's to talk about the poker tourney. E. asked about Hospital work. It would be nice to get a paying gig. I am tired of being on a shoestring. Damn this PhD thing....

2003-01-19T01:22:00.000-05:00

Got the sinks installed in the bathroom vanity today. Kristin was pretty excited to see me get off my ass and actually get some work done on the house.

Progress on Scriptorium continues a pace. I got comment/entity and CDATA tags looking a little better today, and got the selection code up and running. There seem to be an infinite number of ways to approach editing XML. I am basically just trying to implement what *I* think is the most intuitive. I just have to hope that other people are looking for the same thing. In particular, deciding how much free-form editing, versus DTD constraints to allow always gives me pause. With proper selection semantics working (if you select the end-tags for a node, you select the whole node), I should be able to guarantee that users are working with self-contained sub-trees. This is going to come in handy with selection replacements, copy/paste, etc.

The real trick is going to be determining when to check for validity when editing inside a document. For example, cutting out a subtree to replace it will invalidate the document, but the user is obviously GOING to fix it. Current thinking: tags go RED when there is an error in a node (with tooltips, etc. to indicate the prob). Use a background thread to validate the document as changes are made, updating what parts of the document are and are not valid. As users bring the sections into validity, the tags will change from red to their normal color.

2003-01-17T12:15:00.000-05:00

The Anti-Bush rhetoric has reached a fever pitch in the department. I think the discovery of the chemical warheads in Iraq delivered an unexpected reality check. Up to this point, the left could always hope the whole thing would blow over, that nothing would be found, and that the US would bow to liberal pressure to back down.

2003-01-16T09:14:00.000-05:00

If Bush's advisors were really evil, they would release their smoking gun on Iraq tomorrow, just as the peace rally in DC is about to get underway...

Friday, January 17, 2003 9:56:56 AM
wow, not two hours after I posted that comment, they found empty chemical warheads.

I am a little late on finishing my entry for yesterday. I am still trying to get into this blogging crap, but am too lazy to keep up with it.

Tolkien:
In a letter to Christopher, Tolkien talks about man's machinations, and how they are proof of the Fall. Man continuously strives for labour saving devices (the wheel, levers, pulleys, the internal combustion engine, TNT, etc.) to give himself more leasure, and all are put to the uses of war.

2003-01-15T21:33:00.000-05:00

I wonder if the file format for Keynote holds any indications it might be used for a broader application suite. Rumors about that the GoBe team may be working at Apple on an Office killer. One of GoBe's features was its integrated file format (one file format for documents, spreadsheets and presentations). The Keynote extention is apxl (or something like that), NOT something that is more fitting (like .key or .knt). APXL is a general enough extention that it could be used for any office suite applications. There is also some consernation that the file format is not more SVG-like. This could explain why Apple chose a unique XML schema--perhaps we are not seeing it all...

2003-01-14T23:01:00.000-05:00

Scriptorium: Again, frames, borders, views, flipping, and all that jazz is just too much for me to get my head around. However, the autocomplete feature is now working!

Otherwise, a pretty boring day.

2003-01-14T01:59:00.000-05:00

Found Ring! It was out in the front yard. Amazing. Lost for three days.

Normal day otherwise. worked on the autocomplete window in Scriptorium. I just can't seem to get around all the damn coordinate crap. frustrating.

Poker night at our place. Had a good time. Ran until about midnight, then we went and had homebrew in Eric's freezing garage until 1am.. (he was, we assume, asleep at the time, as he had skipped cards).

2003-01-12T13:52:00.000-05:00

Working on the bathroom remodel. It is like the Bataan death march. I'm hoping to get the new vanity in place today.

Bad news from the 10th - lost my wedding ring. I think I flushed it down a toilet. I have been on a low-carb diet, and my ring had become pretty loose. I'm pretty bumed.

Making progress on Scriptorium. New use of attachments seems to be working well. I like the look of the rounded tags. I am now reimplementing the autocompete window to match the new look.

2003-01-10T09:20:00.000-05:00

I tried Blogger with OmniWeb AGES ago, and got nothing, but this seems to be working. Who knows, maybe I will actually keep up with a blog this time around...