Artful Code

Among the various methods of extending Python about which I have written, Python 2.5′s ctypes module is possibly the easiest.

Libraries written in many languages beyond C, including ECL, may be compiled into a shared library or DLL. Many other languages, like Lua and newLISP, include simple DLLs of their own to embed their interpreters in an application.

ctypes allows you to wrap a shared library in pure Python. Here is a quick example to get you started. newLISP exports a simple function, newlispEvalStr, which accepts a string of newLISP code and returns the result of the code’s evaluation as a string. To compile newLISP as a shared library, use the PLATFORM_lib variant of your system’s make target (quick note: make install does not install the shared library; it must be manually copied to your path). The helper function, find_library (located in ctypes.util), gives you a platform-independent way of finding the needed path string for a library.

from ctypes import *
from ctypes.util import find_library
 
lib_path = find_library('newlisp')
newlisp = CDLL(lib_path)

CDLL extends LoadLibrary, the primary utility class in ctypes. In Windows, there are also OleDLL and WinDll. All of these release the global interpreter lock when a foreign function is entered and reclaim it when the function call completes. There is also a PyDLL class that does not release the GIL; its main purpose is the provide direct access to the Python C API from within your Python application.

newlispEvalStr is accessed as a method of the CDLL instance. As with any low level interface, it is important to know the library you are using. Accessing invalid function names can lead to unexpected results or system instability.

nl_eval = newlisp.newlispEvalStr

nl_eval is now an instance of the class _FuncPtr. To be safe, we want to make sure that only strings get passed to the function and to access the returned data as a string.
ctypes provides the convenience properties argtypes and restype for this purpose.
argtypes is set to lists of argument types (using the c-mapped types here).
restype is a single c type:

nl_eval.argtypes = [c_char_p]
nl_eval.restype = c_char_p

We are now ready to evaluate newLISP code:

nl_eval('(+ 2 2)') # => '4'
nl_eval("(apply + '(4 3 2 5 4))") # => '18'

A more general function can be defined to coerce the desired result type after nl_eval returns:
<

def newlisp_eval(code, return_type=str):
    return return_type(nl_eval(code))
 
newlisp_eval('(+ 2 2)', int) # => 4

What’s even better is that because the GIL is released and the library (in this instance) is not directly modifying any Python data, we can make effective use of threading.

Finally, here is a simple module putting all this together:

from ctypes import *
from ctypes.util import find_library
 
class LibraryNotFound(Exception):
    pass
 
class newLISP(object):
    def __init__(self):
        found = find_library('newlisp')
        if found is None:
            raise LibraryNotFound()
        else:
            self.lib = CDLL(found)
            self._eval = self.lib.newlispEvalStr
            self._eval.argtypes = [c_char_p]
            self._eval.restype = c_char_p
 
    def eval(self, code, return_type=str):
        code = c_char_p(code)
        result = self._eval(code)
        if result:
            result = result.strip()
            return return_type(result)
        else:
            return None

Links:

• Python ctypes documentation