Artful Code

With version 9.3 of newLISP on the cusp of release, I thought it apropos to discuss some of the improvements to the interpreter. While the buzz is all about FOOP (functional object-oriented programming), I am more interested in some of the new functions supporting nested association lists.

Improved association list support

This version has a new set of functions for managing nested association lists. > Nested association lists are frequently the result from converting XML into Lisp s-expressions. XML has established itself as a format for data interchange on the internet. The new functions permit access, modification and deletion of associations in nested association lists. These functions also work together with FOOP, which represents objects as associations.

This is an important update. newLISP does not have true hash tables; contexts can serve the same purpose, but they can be clumsy and use a lot of memory. The are optimal for large data sets, such as a series of several hundred thousand key-value pairs.

newLISP is, in any case, optimized to traverse lists, and association lists are an excellent choice for small to moderate string-indexed data sets.
set-assoc, which replaces replace-assoc, supports string indexing of lists in a manner similar to implicit indexing. For example:

(set 'foo-list '(("foo" "bar") ("baz" "bat")))
(set-assoc (foo-list "foo") '("foo" "another value"))
(set-assoc (foo-list "baz") '("foo" "yet another value"))

This works like the older replace-assoc function in previous versions. The difference is that set-assoc can handle nested lists, such as those generated by xml-parse:

((set 'foo-list
    '(("root" ("address" ("name" "Bob") ("email" "")))))
(set-assoc (foo-list "root" "address" "name") '("name" "Robert"))

set-assoc‘s fraternal twin, assoc-set, works much the same but returns only the value of the replaced association, rather than the entire list. Both are destructive functions. Similarly, pop-assoc removes an association from the list.

More importantly to me are ref-set, set-ref, and set-ref-all. These facilitate list-modification via comparison functions such as match and unify. For example, if we have:

((set 'foo '(root (address (user-name "Bob") (email ""))))

…and we want to get the name and email, we might use match:

((match '(root (address (user-name ?) (email ?))) foo)
; => ("Bob" "")

Now, we can use match to set the data as well:

((set-ref (foo '(user-name ?)) '(user-name "John") match)
; => (root (address (user-name "John") (email "")))

ref-set does the same thing but returns the expression matched.
set-ref-all will continue recursing through the list and setting all matches. In all of these functions, the symbol $0 holds the matched expression and may be used in the replacement.

So, using our example list above, we could write a function to set a user’s data non-destructively (since variables are passed by value to functions):

((define (set-user-data lst key value)
    (set-ref (lst (list key '?)) (list key value) match))
 
(set-user-data foo 'name "Jeff")
; => (root (address (name "Jeff") (email "")))
 
foo
; => (root (address (name "John") (email "")))

It’s easy to see how these may be used to build more complex mechanisms for working with nested structures.

pack and dostring

One change that also interests me is that pack can now accept lists as arguments. The documentation is not yet available for this change (it’s not in the 9.27 pre-release docs), but it promises to be an excellent way of pickling data for any purpose.

Another nice change to see is dostring, which iterates quickly over the characters of a string.
count has also been sped up on unix-like systems. Both of these are welcome changes (and, incidentally, ones that I requested).