This is by no means an authoritative discussion about SIP. Rather, it is a chronicle of the adventures and misadventures of a bumbling newbie trying to learn to use a great tool with little documentation. Some references that are essential in conjunction to this include:
For the convenience of the reader, I've included a concise summary of the sip and python API items used here.
sip has changed from version to version. I'm using python 2.2 and sip 3.0pre7. Depending on the verion you're using, sip will behave a little differently. The most notable addition to the new version of sip is support for classes that are wrapped inside namespaces.
I'm currently working on C++ software, and want to create an interface to my libraries in a more user-friendly programming language. I make use of Qt, so the fact that Sip is committed to supporting Qt is a big plus from my point of view.
Here we present a simple example of sip: an implementation of the C++ string class. There are two main aspects to implementing the bindings: writing the .sip file that the sip preprocesses uses to generate bindings, and automating the build process. First, let's create a sip file. Creating this file is pretty simple. A pretty html file of the code is available here, the sip file itself is here . First, we need to declare the name of our python module:
%ModuleThen we include the declarations given in the header file:
%HeaderCode #include <string.h> %EndThen we declare the interface. This is largely uneventful.
namespace std { class string { public: string(); string(const char*); string(const std::string&); bool empty(); int length(); int size(); /* blah blah blah */
The only comments worth making here are:
One could leave it at that and just declare those methods. However, there are some other things one may want a string class to do ...
First, let's implement __str__. We start with some code to declare the method:
PyMethod __str__ %MemberCodeFirst, we declare some variables. ptr is a pointer to the underlying C++ object, and s is a temp to hold the return value of the c_str() method that we use to extract a C-style string.
const char* s; std::string* ptr;
Note that we don't have to declare the parameters, because sip declares them for us when it processes the file. The parameters are a0, which is the calling object, and a1 which is the index (type int). We need to recover the C++ pointer from the PyObject pointer a0.
ptr = (std::string*)sipGetCppPtr((sipThisType *) a0, sipClass_std_string ); if (ptr == NULL) return NULL;
The function sipGetCppPtr retrieves the this pointer from a python object. The first argument is the object, the second argument is the base class (more precisely, it's a pointer to an object that represents the class we want to convert to) Once we've extracted the generic pointer, we cast it to type std::string*. After performing all of those casts, we need to check that the conversions were succesful. sipGetCppPtr returns NULL on failure, so if the pointer is null, we abort, and return a NULL pointer. Otherwise, we have obtained a pointer to our instance and may proceed further.
Having obtained ptr, we obtain the char* pointer from that, using c_str(), and convert to a python string using the PyString_FromString function in the python API. See section 7.3.1 the python API reference, which comes with the python distribution for more information about this function.
Next, we'd like to implement [] notation. This is done by implementing the __getitem__ method. Start by declaring the method:
PySequenceMethod __getitem__ %MemberCode
Now for the C++ member code. First, we declare and extract the this pointer as per the previous example. Then we need to check that the requested index is not out of bounds it is important to do this, uncaught C++ exceptions will cause python to abort !. So it's necessary to either prevent C++ exceptions from occuring, or to trap them and propogate python exceptions.
if (a1 >= ptr->length()) { /* Python API Reference, Ch 4 */ PyErr_SetString ( PyExc_IndexError ,"string index out of range" ); return NULL; }The PyErr_SetString function is part of the python API and is explained in the API reference. What these lines of code do is raise an exception if the argument is out of bounds. Otherwise, we may return a value:
return Py_BuildValue("c", ptr->at(a1));
Py_BuildValue is documented in the Extending and Embedding Python, 1.3, 1.7 P8-11.
So the problem we now have is to compile our code. Here's what works for me: I have a
The top level Makefile is pretty simple, it just runs sip and then builds in the sub-directory.
SIP=/usr/local/bin/sip sip: $(OBJS) $(SIP) -s ".cc" -c sipcode string.sip cd sipcode && make
sip will generate the following files in this example:
Note the way the namespace is munged into the names. Now the Makefile to build this in the sipcode directory looks like this:
module=String class=stdstring objs=$(module)cmodule.o sip$(module)$(class).o PYTHON_INCLUDES=-I/usr/include/python2.2 SIP_INCLUDES=-I/usr/local/include/sip %.o: %.cc $(CXX) $(CXXFLAGS) -c -I.. -I. $(PYTHON_INCLUDES) $(SIP_INCLUDES) $< all: libs libs: sip$(module)Version.h $(objs) $(CXX) -shared -L/usr/local/lib/ -lsip -o lib$(module)cmodule.so *.o sip$(module)Version.h: $(CXX) -o sip_helper sip_helper.cc ./sip_helper > sip$(module)Version.h clean: rm -f *.o *.so *.cc *.h sip_helper *.py *.pyc
And that's it! Now we've got a String module that compiles. Installing it into the proper directories is left as an exercise to the reader, (OK, I admit it, I don't know how to do it yet)
An example that gives us a chance to play with more operators, and other finer points of sip, is that of implementing a fraction data type. Consider the fraction class with the declaration frac.h ( download code ) and source code frac.cc ( download source ). This class was written for completeness, and economy of the coders time (-; so a lot of operators are implemented in terms of others. To implement a python version of this, we redeclare the member functions, excluding operators:
Some comments about the class:
declaring the basic interface is straightforward:
class Fraction { public: Fraction(int , int=1 ); Fraction(const Fraction&); int numerator() const; int denominator() const; /* we'll put more code here later */ }; int gcd(int, int);The interesting part is declaring the operators. Implementing the arithmatic binary operators +,-,*,/ involves essentially the same code, and most of the code just does error conversion and type checking. It would be nice to break this boilerplate code off into its own function. The strategy we use is to have a function that takes a function pointer as an argument. The function pointer is to a function that invokes one of the operators +,-,*,/.
%HeaderCode #include <frac.h> typedef Fraction* (*binary_fraction_op_t) (Fraction*,Fraction*); PyObject* BinaryOp(PyObject* a0, PyObject* a1, binary_fraction_op_t op); Fraction* plus (Fraction* x, Fraction* y); Fraction* minus (Fraction* x, Fraction* y); Fraction* mult (Fraction* x, Fraction* y); Fraction* div (Fraction* x, Fraction* y); %End
Then we need to implement the body of these functions. The operator functions are very simple, they just perform the operation. Note that we use pointers all the time, because all our Fraction objects are heap allocated.
%C++Code Fraction* plus (Fraction* x, Fraction* y) { return new Fraction(*x + *y); } Fraction* minus (Fraction* x, Fraction* y) { return new Fraction(*x - *y); } Fraction* mult (Fraction* x, Fraction* y) { return new Fraction(*x * *y); } Fraction* div (Fraction* x, Fraction* y) { return new Fraction(*x / *y); } PyObject* BinaryOp(PyObject* a0, PyObject* a1, binary_fraction_op_t op) { Fraction *ptr1, *ptr2; PyObject* res; /** Extract operands */ ptr1 = (Fraction*) sipGetCppPtr((sipThisType*) a0, sipClass_Fraction); if (ptr1 == NULL) return NULL; ptr2 = (Fraction*) sipGetCppPtr((sipThisType*) a1, sipClass_Fraction); if (ptr2 == NULL) return NULL; /** * op() returns a dynamically allocated pointer. Calling sipNewCppToSelf() * with flags SIP_PY_OWNED | SIP_SIMPLE creates a python object that is * owned by python, with a reference count of 1. */ return sipNewCppToSelf ( op(ptr1, ptr2), sipClass_Fraction, SIP_PY_OWNED | SIP_SIMPLE ); } %End
This makes implementing arithmatic the operators simpler. Note that all the binary arithmatic operators return type PyObject*, and take two arguments of type PyObject*. The functions that implement these operators are documented in the API reference, section 6.2, the number protocol. For example, the __add__ method corresponds with the function PyNumber_Add. While the function signatures are obvious for a lot of the data types, some of them like __coerce__ require one to read the documentation to understand how to implement them.
So here's the code to implement our operations:
PyNumberMethod __add__ %MemberCode return BinaryOp (a0,a1,plus); %End PyNumberMethod __sub__ %MemberCode return BinaryOp (a0,a1,minus); %End PyNumberMethod __mul__ %MemberCode return BinaryOp (a0,a1,minus); %End PyNumberMethod __div__ %MemberCode return BinaryOp (a0,a1,minus); %EndWe'd also like to permit explicit conversion to floating point numbers. We do this by implementing __float__. Note the python API function Py_BuildValue.
PyNumberMethod __float__ %MemberCode Fraction* ptr; ptr = (Fraction*) sipGetCppPtr((sipThisType*) a0, sipClass_Fraction); if (ptr == NULL) return NULL; double x = (double)ptr->numerator() / ptr->denominator(); return Py_BuildValue ( "d", x ); %End
We're almost done. A desirable feature to make our fraction data type better interoperate with pythons numerical types would be an implementation of the "type promotion" feature, the __coerce__ method. Implementing this is a little tricky. Referring to the python API documentation on the Number Protocol, 6.2 P29, we see that:
int PyNumber_Coerce(PyObject **p1, PyObject **p2)
This function takes the addresses of two variables of type PyObject*. It the objects pointed to by *p1 and *p2 have the same type, increment their reference count and return 0 (success). If the objects can be converted to a common numeric type, replace *p1 and *p2 by their converted value (with 'new' reference counts), and return 0. If no conversion is possible, or if some other error occurs, return -1 (failure) and don't increment the reference counts. The call PyNumber_Coerce(&o1, & o2) is equivalent to the python statement 'o1,o2 = coerce(o1,o2)'
So, it's a good thing we read the documents. While one might reasonably guess the meaning of the return code, and deduce that the arguments are supposed to be overwritten with the return values, the reference counts are a potential trap. If we didn't read the document, our code would have segfaulted (like mine did when I was learning this!)
The basic aim of the coerce method then is to end up with two objects of the same type, and we know that the first is a fraction. We start by declaring some variables:
PyNumberMethod __coerce__ %MemberCode Fraction* ptr; long i; bool success = false;
ptr is the first argument, after conversion. i is a temp used to handle conversions from pythons integral data types. success is a status flag. We begin by recovering ptr in the usual way.
ptr = (Fraction*) sipGetCppPtr((sipThisType*) *a0, sipClass_Fraction); if (ptr == NULL) return -1;Then we check to see if both objects are of the same type. We already know that *a0 is of type Fraction, we need to perform a check for *a1. Calling sipGetCppPtr as we did for *a0 is unsafe, becasuse we know nothing about the type of *a1. So we use a special function to check the type of *a1, sipIsSubClassInstance().
int sipIsSubClassInstance(PyObject *inst, PyObject *baseclass
This function returns a true value if inst is an object whose class is some derived type of baseclass
if (sipIsSubClassInstance(*a1, sipClass_Fraction)) { /* API guide, 3.1 */ Py_XINCREF(*a0); Py_XINCREF(*a1); return 0; }
Note that this works as long as *a1 is an object of some derived class of Fraction
If the arguments are not both fractions, we need to check for integral types, and convert. We do this using the checking functions PyLong_Check() and PyInt_Check() and the conversions PyLong_AsLong() and PyInt_AsLong() (again, described in 6.1, API ref). Note that if these conversions are succesful, we increment the reference count of *a0. This has the same effect as returning a copy of *a0 with a reference count of 1.
if ( PyLong_Check(*a1) ) { success = true; i = PyLong_AsLong (*a1); } else if ( PyInt_Check(*a1) ) { success = true; i = PyInt_AsLong (*a1); } if (success) { Py_XINCREF(*a0); *a1 = sipNewCppToSelf ( new Fraction(i,1), sipClass_Fraction SIP_PY_OWNED | SIP_SIMPLE ); return 0; } else { return -1; }
And that's it. We now have a complete sip file (download). An interesting exercise would be to implement other operators, and/or imeplement __coerce__ for floating point data types, but this is sufficient to get the reader started.
const void *sipGetCppPtr (sipThisType*,PyObject *); PyObject *sipMapCppToSelf (const void *,PyObject *); PyObject *sipNewCppToSelf (const void *,PyObject *); int sipIsSubClassInstance (PyObject *,PyObject *);
PyObject *sipNewCppToSelf (const void * object,PyObject * class, int flags);
Convert a C++ object to a Python object. This function is used to return values from functions. For the flags, you will nearly always want to use SIP_SIMPLE | SIP_PY_OWNED. This means that Python is responsible for managing the object. I am still not clear what SIP_SIMPLE means, but it has something to do with the internal representation of the object.
PyObject *sipMapCppToSelf (const void * object,PyObject * class);
Convert a C++ object to a Python object. This function is used to convert C++ objects to Python objects. C++ bears responsibility for deallocating the memory (so use sipNewCppToSelf to construct return values)
const void* sipGetCppPtr(sipThisType* object, PyObject* class);
Returns C++ class pointer. This function is unsafe, in that the way it's usually used involves a cast from a generic PyObject pointer to a sipThisType pointer. So it should only be used if the argument is known to be a sip object.
int sipIsSubClassInstance (PyObject * object,PyObject * class);
Check to see if object belongs to class or some subclass of class. This is useful for type checking, if you don't know anything about an objects type.
These functions are all documented in the API reference, but are listed here for convenience. This is not meant to be comprehensive (for that, there's the Python API reference).
int PyInt_Check(PyObject* o); int PyLong_Check(PyObject* o); int PyFloat_Check(PyObject* o);Returns true value if o is respectively a python int, long or float object.
int PyInt_AsLong(PyObject* o); long PyInt_AsLong(PyObject* o); longlong PyInt_AsLongLong(PyObject* o); double PyLong_AsDouble(PyObject* o); double PyFloat_AsDouble(PyObject* o);
Convert python objects. Several conversions are offered for Pythons long data type, because it is an arbitrary precision type. Conversions involving long raise OverflowError if unsuccesful.
Overlading numeric operators amounts to partially implementing pythons number protocol described in the API reference. In particular, one implements functions that implement the functionality documented in the API reference. The python functions delegate to the method table built by sip, so sip__add__classname does the work when __add__ (or PyNumber_Add, which has the same effect) is called. There are a lot of arithmatic operators, and they're all documented in the API reference. Here, we present the ones used in the examples.
PyObject* PyNumber_Add(PyObject* left, PyObject* right); PyObject* PyNumber_Subtract(PyObject* left, PyObject* right); PyObject* PyNumber_Multiply(PyObject* left, PyObject* right); PyObject* PyNumber_Divide(PyObject* left, PyObject* right);
Respectively add, subtract, multiply, and divide two python objects. Same as (respectively) the methods __add__, __sub__, __mul__, __div__ in Python. Methods should return NULL if unsuccesful. Python takes care of type checking and promotion to make sure both operands are of the same type.
This is a little complex, so I will quote the API reference verbatim:
int PyNumber_Coerce(PyObject **p1, PyObject **p2)
This function takes the addresses of two variables of type
PyObject*. It the objects pointed to by *p1
and *p2 have the same type, increment their reference count
and return 0 (success). If the objects
can be converted to a common numeric type, replace *p1 and
*p2 by their converted value (with 'new' reference counts),
and return 0.
If no conversion is possible, or if some other error occurs,
return -1 (failure) and don't increment the
reference counts. The call PyNumber_Coerce(&o1& , o2)
is equivalent to the python statement 'o1,o2 = coerce(o1,o2)'
PyObject* PyNumber_Int(PyObject*); PyObject* PyNumber_Long(PyObject*); PyObject* PyNumber_Float(PyObject*);
implement the float(), int() and long() operators respectively.
PyObject* Py_BuildValue(char* format, ... );
Construct a python object from C variables. Return NULL on failure. The specific rules re format are quite long and described in the Extending and Embedding guide.
PyErr_SetString(PyObject* exception, char* message);Exceptions behave in a way that may seem strange to a C++ programmer. One throws an exception by "setting" an exception flag. The exception objects are defined in python (See API reference, P16 4.1 for a complete list). One usually throws a standard exception type.
One should very rarely have to use any of these when writing sip bindings. They are needed occasionaly to implement a python method, eg __coerce__.
void Py_XINCREF(PyObject* o); void Py_XDECREF(PyObject* o);
Respectively increment the reference count of o. If o is a null pointer, this has no effect.