Artful Code

Multi-processing in newLISP can be a chore. All processes must be directly known to any others that will be accessing shared memory, which must be synchronized using semaphores. A new development release of newLISP (9.3.7) will change all that by adding Cilk-style primitives.

The Cilk project is a parallel programming language based on C. Its goal is to radically simplify concurrency in C applications. Cilk applications are first compiled to C. Cilk adds simple concurrency syntax and a scheduler to determine whether or not to launch a new thread or simply execute the code block. Here is a sample from the Cilk website:

cilk int fib (int n)
{
    if (n > 2) return n;
    else
    {
        int x, y;
 
        x = spawn fib (n-1);
        y = spawn fib (n-2);
 
        sync;
 
        return (x+y);
    }
}

The next version of newLISP will have the spawn, sync, and abort functions. Here is the same function, written in newLISP:

(define (fib n)
  (if (> n 2) 1
    (begin
      (spawn 'x (fib (- n 1)))
      (spawn 'y (fib (- n 2)))
      (sync 10000)
      (+ x y))))

There are a few notable differences.
spawn not only spawns a new process, but it also sets the target symbol to be assigned when sync is run.
sync itself has a millisecond timeout. There is currently no scheduler and sync cannot block indefinitely, but hopefully these will be added in the full release.

spawn returns the pid of the launched child process, which may be cancelled later using (abort pid). On its own, abort cancels all child processes.

sync causes the current process to block until all children have completed or the timeout (in milliseconds) is reached. An interesting addition is that sync can also be used as a predicate to determine if all child processes have quit:

(until (sync 10000)
    (println "Waiting... (foot-tapping)")
    (sleep 100))

By itself, it returns a list of pids of child processes:

(spawn 'foo (+ 1 1))
(spawn 'bar (+ 2 2))
(sync) ; =>(12345 12346)
(sync 100) ; => true
foo ; => 2
bar ; => 4

Underneath the hood, the new functionality is not using Cilk – it simply aims to add similar behavior to newLISP. Because of newLISP’s small size (~500 KB in memory), forked processes (in the typical sense, not the Erlang one) are cheap. With sync functionality, newLISP will have a real edge on languages using OS threads.

In other news

Another recent development release of newLISP (9.3.5) added another important feature – the ability to pre-compile regular expressions:

(set 're (regex-comp "(foo|bar)"))
(regex re "raising the bar" 0x10000)

The new regular expression int-optin, 0x10000, alerts the regex function to expect a precompiled pattern. This works in all regex functions.