Get best of the week

Creating Python bindings for C / C ++ libraries using SIP. Part 2

In the first part of the article, we examined the basics of working with the SIP utility designed to create Python bindings for libraries written in C and C ++. We looked at the basic files that you need to create to work with SIP and started looking at directives and annotations. So far, we have done the binding for a simple library written in C. In this part, we will figure out how to do the binding for a C ++ library that contains classes. Using the example of this library, we will see what techniques can be useful when working with an object-oriented library, and at the same time we will deal with new directives and annotations for us.

All examples for the article are available in the github repository at: https://github.com/Jenyay/sip-examples .

Making a binding for a library in C ++


The following example, which we will consider, is in the pyfoo_cpp_01 folder .

First, create a library for which we will do the binding. The library will still reside in the foo folder and contain one class - Foo . The header file foo.h with the declaration of this class is as follows:

#ifndef FOO_LIB
#define FOO_LIB

class Foo {
    private:
        int _int_val;
        char* _string_val;
    public:
        Foo(int int_val, const char* string_val);
        virtual ~Foo();

        void set_int_val(int val);
        int get_int_val();

        void set_string_val(const char* val);
        char* get_string_val();
};

#endif

This is a simple class with two getters and setters that sets and returns values ​​of type int and char * . The implementation of the class is as follows:

#include <string.h>

#include "foo.h"

Foo::Foo(int int_val, const char* string_val): _int_val(int_val) {
    _string_val = nullptr;
    set_string_val(string_val);
}

Foo::~Foo(){
    delete[] _string_val;
    _string_val = nullptr;
}

void Foo::set_int_val(int val) {
    _int_val = val;
}

int Foo::get_int_val() {
    return _int_val;
}

void Foo::set_string_val(const char* val) {
    if (_string_val != nullptr) {
        delete[] _string_val;
    }

    auto count = strlen(val) + 1;
    _string_val = new char[count];
    strcpy(_string_val, val);
}

char* Foo::get_string_val() {
    return _string_val;
}

To test the library's functionality, the foo folder also contains the main.cpp file using the Foo class :

#include <iostream>

#include "foo.h"

using std::cout;
using std::endl;

int main(int argc, char* argv[]) {
    auto foo = Foo(10, "Hello");
    cout << "int_val: " << foo.get_int_val() << endl;
    cout << "string_val: " << foo.get_string_val() << endl;

    foo.set_int_val(0);
    foo.set_string_val("Hello world!");

    cout << "int_val: " << foo.get_int_val() << endl;
    cout << "string_val: " << foo.get_string_val() << endl;
}

To build the foo library, use the following Makefile :

CC=g++
CFLAGS=-c -fPIC
DIR_OUT=bin

all: main

main: main.o libfoo.a
    $(CC) $(DIR_OUT)/main.o -L$(DIR_OUT) -lfoo -o $(DIR_OUT)/main

main.o: makedir main.cpp
    $(CC) $(CFLAGS) main.cpp -o $(DIR_OUT)/main.o

libfoo.a: makedir foo.cpp
    $(CC) $(CFLAGS) foo.cpp -o $(DIR_OUT)/foo.o
    ar rcs $(DIR_OUT)/libfoo.a $(DIR_OUT)/foo.o

makedir:
    mkdir -p $(DIR_OUT)

clean:
    rm -rf $(DIR_OUT)/*

The difference from the Makefile in the previous examples, in addition to changing the compiler from gcc to g ++ , is that another option -fPIC was added for compilation , which tells the compiler to place the code in the library in a certain way (the so-called "position-independent code"). Since this article is not about compilers, we will not examine in more detail what this parameter does and why it is needed.

Let's start tying for this library. The pyproject.toml and project.py files are almost unchanged from the previous examples. Here's what the pyproject.toml file now looks like :

[build-system]
requires = ["sip >=5, <6"]
build-backend = "sipbuild.api"

[tool.sip.metadata]
name = "pyfoocpp"
version = "0.1"
license = "MIT"

[tool.sip.bindings.pyfoocpp]
headers = ["foo.h"]
libraries = ["foo"]

Now our examples written in C ++ will be packaged in the pyfoocpp Python package , this is perhaps the only noticeable change in this file.

The project.py file remains the same as in the pyfoo_c_04 example :

import os
import subprocess

from sipbuild import Project

class FooProject(Project):
    def _build_foo(self):
        cwd = os.path.abspath('foo')
        subprocess.run(['make'], cwd=cwd, capture_output=True, check=True)

    def build(self):
        self._build_foo()
        super().build()

    def build_sdist(self, sdist_directory):
        self._build_foo()
        return super().build_sdist(sdist_directory)

    def build_wheel(self, wheel_directory):
        self._build_foo()
        return super().build_wheel(wheel_directory)

    def install(self):
        self._build_foo()
        super().install()

But the pyfoocpp.sip file we will consider in more detail. Let me remind you that this file describes the interface for the future Python module: what it should include, what the class interface should look like, etc. The .sip file is not required to repeat the library header file, although they will have a lot in common. Inside this class, new methods may be added that were not in the original class. Those. the interface described in the .sip file can tailor library classes to the principles accepted in Python, if necessary. In the pyfoocpp.sip file we will see new directives for us.

First, let's see what this file contains:

%Module(name=foocpp, language="C++")
%DefaultEncoding "UTF-8"

class Foo {
    %TypeHeaderCode
    #include <foo.h>
    %End

    public:
        Foo(int, const char*);

        void set_int_val(int);
        int get_int_val();

        void set_string_val(const char*);
        char* get_string_val();
};

The first lines should already be clear to us from the previous examples. In the % Module directive, we indicate the name of the Python module that will be created (that is, to use this module, we will need to use the import foocpp or from foocpp import ... commands . In the same directive, we indicate that we now have the language - C ++. The % DefaultEncoding directive sets the encoding that will be used to convert the Python string to char , const char , char * and const char * types .

Then the interface declaration of the Foo class follows . Immediately after the declaration of the Foo classthere is still not used the % TypeHeaderCode directive , which ends with the % End directive . The % TypeHeaderCode directive must contain code that declares the interface of the C ++ class for which the wrapper is being created. As a rule, in this directive it is enough to include the header file with the class declaration.

After that, the class methods that will be converted to the methods of the Foo class for the Python language are listed . It is important to note that at this point we only declare public methods that will be accessible from the Foo class in Python (since there are no private and protected members in Python). Since we used the % DefaultEncoding directive at the very beginning, then in methods that take arguments of type const char * , you can not use the Encoding annotation to specify the encoding for converting these parameters to Python strings and vice versa.

Now we just need to compile the pyfoocpp Python package and test it. But before assembling a full-fledged wheel package, let's use the sip-build command and see what source files SIP will create for later compilation, and try to find in them something similar to the class that will be created in Python code. To do this, the above sip-build command must be called in the pyfoo_cpp_01 folder . As a result, the build folder will be created. with the following contents:

build
└── foocpp
    ├── apiversions.c
    ├── array.c
    ├── array.h
    ├── bool.cpp
    ├── build
    │   └── temp.linux-x86_64-3.8
    │       ├── apiversions.o
    │       ├── array.o
    │       ├── bool.o
    │       ├── descriptors.o
    │       ├── int_convertors.o
    │       ├── objmap.o
    │       ├── qtlib.o
    │       ├── sipfoocppcmodule.o
    │       ├── sipfoocppFoo.o
    │       ├── siplib.o
    │       ├── threads.o
    │       └── voidptr.o
    ├── descriptors.c
    ├── foocpp.cpython-38-x86_64-linux-gnu.so
    ├── int_convertors.c
    ├── objmap.c
    ├── qtlib.c
    ├── sipAPIfoocpp.h
    ├── sipfoocppcmodule.cpp
    ├── sipfoocppFoo.cpp
    ├── sip.h
    ├── sipint.h
    ├── siplib.c
    ├── threads.c
    └── voidptr.c


As an additional task, carefully consider the sipfoocppFoo.cpp file (we will not discuss it in detail in this article):

/*
 * Interface wrapper code.
 *
 * Generated by SIP 5.1.1
 */

#include "sipAPIfoocpp.h"

#line 6 "/home/jenyay/temp/2/pyfoocpp.sip"
    #include <foo.h>
#line 12 "/home/jenyay/temp/2/build/foocpp/sipfoocppFoo.cpp"

PyDoc_STRVAR(doc_Foo_set_int_val, "set_int_val(self, int)");

extern "C" {static PyObject *meth_Foo_set_int_val(PyObject *, PyObject *);}
static PyObject *meth_Foo_set_int_val(PyObject *sipSelf, PyObject *sipArgs)
{
    PyObject *sipParseErr = SIP_NULLPTR;

    {
        int a0;
         ::Foo *sipCpp;

        if (sipParseArgs(&sipParseErr, sipArgs, "Bi", &sipSelf, sipType_Foo, &sipCpp, &a0))
        {
            sipCpp->set_int_val(a0);

            Py_INCREF(Py_None);
            return Py_None;
        }
    }

    /* Raise an exception if the arguments couldn't be parsed. */
    sipNoMethod(sipParseErr, sipName_Foo, sipName_set_int_val, doc_Foo_set_int_val);

    return SIP_NULLPTR;
}

PyDoc_STRVAR(doc_Foo_get_int_val, "get_int_val(self) -> int");

extern "C" {static PyObject *meth_Foo_get_int_val(PyObject *, PyObject *);}
static PyObject *meth_Foo_get_int_val(PyObject *sipSelf, PyObject *sipArgs)
{
    PyObject *sipParseErr = SIP_NULLPTR;

    {
         ::Foo *sipCpp;

        if (sipParseArgs(&sipParseErr, sipArgs, "B", &sipSelf, sipType_Foo, &sipCpp))
        {
            int sipRes;

            sipRes = sipCpp->get_int_val();

            return PyLong_FromLong(sipRes);
        }
    }

    /* Raise an exception if the arguments couldn't be parsed. */
    sipNoMethod(sipParseErr, sipName_Foo, sipName_get_int_val, doc_Foo_get_int_val);

    return SIP_NULLPTR;
}

PyDoc_STRVAR(doc_Foo_set_string_val, "set_string_val(self, str)");

extern "C" {static PyObject *meth_Foo_set_string_val(PyObject *, PyObject *);}
static PyObject *meth_Foo_set_string_val(PyObject *sipSelf, PyObject *sipArgs)
{
    PyObject *sipParseErr = SIP_NULLPTR;

    {
        const char* a0;
        PyObject *a0Keep;
         ::Foo *sipCpp;

        if (sipParseArgs(&sipParseErr, sipArgs, "BA8", &sipSelf, sipType_Foo, &sipCpp, &a0Keep, &a0))
        {
            sipCpp->set_string_val(a0);
            Py_DECREF(a0Keep);

            Py_INCREF(Py_None);
            return Py_None;
        }
    }

    /* Raise an exception if the arguments couldn't be parsed. */
    sipNoMethod(sipParseErr, sipName_Foo, sipName_set_string_val, doc_Foo_set_string_val);

    return SIP_NULLPTR;
}

PyDoc_STRVAR(doc_Foo_get_string_val, "get_string_val(self) -> str");

extern "C" {static PyObject *meth_Foo_get_string_val(PyObject *, PyObject *);}
static PyObject *meth_Foo_get_string_val(PyObject *sipSelf, PyObject *sipArgs)
{
    PyObject *sipParseErr = SIP_NULLPTR;

    {
         ::Foo *sipCpp;

        if (sipParseArgs(&sipParseErr, sipArgs, "B", &sipSelf, sipType_Foo, &sipCpp))
        {
            char*sipRes;

            sipRes = sipCpp->get_string_val();

            if (sipRes == SIP_NULLPTR)
            {
                Py_INCREF(Py_None);
                return Py_None;
            }

            return PyUnicode_FromString(sipRes);
        }
    }

    /* Raise an exception if the arguments couldn't be parsed. */
    sipNoMethod(sipParseErr, sipName_Foo, sipName_get_string_val, doc_Foo_get_string_val);

    return SIP_NULLPTR;
}

/* Call the instance's destructor. */
extern "C" {static void release_Foo(void *, int);}
static void release_Foo(void *sipCppV, int)
{
    delete reinterpret_cast< ::Foo *>(sipCppV);
}

extern "C" {static void dealloc_Foo(sipSimpleWrapper *);}
static void dealloc_Foo(sipSimpleWrapper *sipSelf)
{
    if (sipIsOwnedByPython(sipSelf))
    {
        release_Foo(sipGetAddress(sipSelf), 0);
    }
}

extern "C" {static void *init_type_Foo(sipSimpleWrapper *, PyObject *, 
                 PyObject *, PyObject **, PyObject **, PyObject **);}
static void *init_type_Foo(sipSimpleWrapper *, PyObject *sipArgs, PyObject *sipKwds,
                                   PyObject **sipUnused, PyObject **, PyObject **sipParseErr)
{
     ::Foo *sipCpp = SIP_NULLPTR;

    {
        int a0;
        const char* a1;
        PyObject *a1Keep;

        if (sipParseKwdArgs(sipParseErr, sipArgs, sipKwds, SIP_NULLPTR, sipUnused, "iA8", &a0, &a1Keep, &a1))
        {
            sipCpp = new  ::Foo(a0,a1);
            Py_DECREF(a1Keep);

            return sipCpp;
        }
    }

    {
        const  ::Foo* a0;

        if (sipParseKwdArgs(sipParseErr, sipArgs, sipKwds, SIP_NULLPTR, sipUnused, "J9", sipType_Foo, &a0))
        {
            sipCpp = new  ::Foo(*a0);

            return sipCpp;
        }
    }

    return SIP_NULLPTR;
}

static PyMethodDef methods_Foo[] = {
    {sipName_get_int_val, meth_Foo_get_int_val, METH_VARARGS, doc_Foo_get_int_val},
    {sipName_get_string_val, meth_Foo_get_string_val, METH_VARARGS, doc_Foo_get_string_val},
    {sipName_set_int_val, meth_Foo_set_int_val, METH_VARARGS, doc_Foo_set_int_val},
    {sipName_set_string_val, meth_Foo_set_string_val, METH_VARARGS, doc_Foo_set_string_val}
};

PyDoc_STRVAR(doc_Foo, "\1Foo(int, str)\n"
"Foo(Foo)");

sipClassTypeDef sipTypeDef_foocpp_Foo = {
    {
        -1,
        SIP_NULLPTR,
        SIP_NULLPTR,
        SIP_TYPE_CLASS,
        sipNameNr_Foo,
        SIP_NULLPTR,
        SIP_NULLPTR
    },
    {
        sipNameNr_Foo,
        {0, 0, 1},
        4, methods_Foo,
        0, SIP_NULLPTR,
        0, SIP_NULLPTR,
        {SIP_NULLPTR, SIP_NULLPTR, SIP_NULLPTR, SIP_NULLPTR, SIP_NULLPTR,
         SIP_NULLPTR, SIP_NULLPTR, SIP_NULLPTR, SIP_NULLPTR, SIP_NULLPTR},
    },
    doc_Foo,
    -1,
    -1,
    SIP_NULLPTR,
    SIP_NULLPTR,
    init_type_Foo,
    SIP_NULLPTR,
    SIP_NULLPTR,
    SIP_NULLPTR,
    SIP_NULLPTR,
    dealloc_Foo,
    SIP_NULLPTR,
    SIP_NULLPTR,
    SIP_NULLPTR,
    release_Foo,
    SIP_NULLPTR,
    SIP_NULLPTR,
    SIP_NULLPTR,
    SIP_NULLPTR,
    SIP_NULLPTR,
    SIP_NULLPTR,
    SIP_NULLPTR
};

Now build the package using the sip-wheel command . After executing this command, if all goes well, a pyfoocpp-0.1-cp38-cp38-manylinux1_x86_64.whl file or with a similar name will be created . Install it using the pip install --user pyfoocpp-0.1-cp38-cp38-manylinux1_x86_64.whl command and run the Python interpreter to verify:

>>> from foocpp import Foo
>>> x = Foo(10, 'Hello')

>>> x.get_int_val()
10

>>> x.get_string_val()
'Hello'

>>> x.set_int_val(50)
>>> x.set_string_val('')

>>> x.get_int_val()
50

>>> x.get_string_val()
''

Works! Thus, we just made a Python module with a binding for a class in C ++. Further we will bring beauty to this class and add various amenities.

Add properties


Classes created using SIP are not required to exactly repeat the C ++ class interface. For example, in our Foo class, there are two getters and two setters, which can clearly be combined into a property to make the class more “Python”. Adding properties using sip is easy enough as it is done, an example in the pyfoo_cpp_02 folder shows .

This example is similar to the previous one, the main difference is in the pyfoocpp.sip file , which now looks like this:

%Module(name=foocpp, language="C++")
%DefaultEncoding "UTF-8"

class Foo {
    %TypeHeaderCode
    #include <foo.h>
    %End

    public:
        Foo(int, const char*);

        void set_int_val(int);
        int get_int_val();
        %Property(name=int_val, get=get_int_val, set=set_int_val)

        void set_string_val(const char*);
        char* get_string_val();
        %Property(name=string_val, get=get_string_val, set=set_string_val)
};

As you can see, everything is quite simple. To add a property, the % Property directive is intended , which has two required parameters: name to specify the name of the property, and get to specify a method that returns a value (getter). There may not be a setter, but if the property also needs to be assigned values, the setter method is specified as the value of the set parameter . In our example, properties are created in a fairly straightforward manner, since there are already functions that work as getters and setters.

We can only collect the package using the sip-wheel command , install it, after that we will check the operation of the properties in the python interpreter command mode:

>>> from foocpp import Foo
>>> x = Foo(10, "Hello")
>>> x.int_val
10

>>> x.string_val
'Hello'

>>> x.int_val = 50
>>> x.string_val = ''

>>> x.get_int_val()
50

>>> x.get_string_val()
''

As you can see from the example of using the Foo class , the int_val and string_val properties work both for reading and writing.

Add documentation lines


We will continue to improve our Foo class . The following example, which is located in the pyfoo_cpp_03 folder , shows how to add documentation lines (docstring) to various elements of a class. This example is based on the previous one, and the main change in it concerns the pyfoocpp.sip file . Here are its contents:

%Module(name=foocpp, language="C++")
%DefaultEncoding "UTF-8"

class Foo {
%Docstring
Class example from C++ library
%End

    %TypeHeaderCode
    #include <foo.h>
    %End

    public:
        Foo(int, const char*);

        void set_int_val(int);
        %Docstring(format="deindented", signature="prepended")
            Set integer value
        %End

        int get_int_val();
        %Docstring(format="deindented", signature="prepended")
            Return integer value
        %End

        %Property(name=int_val, get=get_int_val, set=set_int_val)
        {
            %Docstring "deindented"
                The property for integer value
            %End
        };

        void set_string_val(const char*);
        %Docstring(format="deindented", signature="appended")
            Set string value
        %End

        char* get_string_val();
        %Docstring(format="deindented", signature="appended")
            Return string value
        %End

        %Property(name=string_val, get=get_string_val, set=set_string_val)
        {
            %Docstring "deindented"
                The property for string value
            %End
        };
};

As you already understood, in order to add documentation lines to any element of the class, you need to use the % Docstring directive . This example shows several ways to use this directive. For a better understanding of this example, let's immediately compile the pyfoocpp package using the sip-wheel command , install it, and we will sequentially figure out which parameter of this directive affects what, considering the resulting documentation lines in Python command mode. Let me remind you that documentation lines are stored as members of the __doc__ objects to which these lines belong.

The first line of documentation is for the Foo class.. As you can see, all documentation lines are located between the % Docstring and % End directives . Lines 5-7 of this example do not use any additional parameters of the % Docstring directive , so the documentation line will be written to the Foo class as is. That is why there are no indentation in lines 5-7, otherwise indents in front of the documentation line would also fall into Foo .__ doc__. We will make sure that the Foo class really contains the line of documentation that we introduced:

>>> from foocpp import Foo
>>> Foo.__doc__
'Class example from C++ library'

The following % Docstring directive , located on lines 17-19, uses two parameters at once. The format parameter can take one of two values: “raw” or “deindented”. In the first case, the documentation lines are saved as they are written, and in the second case, the initial space characters (but not tabs) are deleted. The default value for the case if the format parameter is not specified can be set using the % DefaultDocstringFormat directive (we will consider it a bit later), and if it is not specified, then it is assumed that format = "raw" .

In addition to the specified documentation lines, SIP adds a description of its signature (what types of variables are expected at the input and what type the function returns) to the lines of documentation of functions. The signature parameter indicates where to put such a signature: before the specified line of documentation ( signature = "prepended" ), after it ( signature = "appended" ) or not add the signature ( signature = "discarded" ).

Our example sets the signature = "prepended" parameter for the get_int_val and set_int_val functions , as well as signature = "appended" for the get_string_val and set_string_val functions. The format = "deindented" parameter was also added to remove spaces at the beginning of the documentation line. Let's check how these parameters work in Python:

>>> Foo.get_int_val.__doc__
'get_int_val(self) -> int\nReturn integer value'

>>> Foo.set_int_val.__doc__
'set_int_val(self, int)\nSet integer value'

>>> Foo.get_string_val.__doc__
'Return string value\nget_string_val(self) -> str'

>>> Foo.set_string_val.__doc__
'Set string value\nset_string_val(self, str)'

As you can see, using the signature parameter of the % Docstring directive, you can change the position of the function signature description in the documentation line.

Now consider adding a documentation line to properties. Note that in this case, the % Docstring ... % End directives are enclosed in braces after the% Property directive. This recording format is described in the documentation for the % Property directive .

Also notice how we specify the % Docstring directive parameter . Such a format for writing directives is possible if we set only the first parameter of the directive (in this case, the format parameter) Thus, in this example, three methods of using directives are used at once.

Make sure that the documentation line for the properties is set:

>>> Foo.int_val.__doc__
'The property for integer value'

>>> Foo.string_val.__doc__
'The property for string value'

>>> help(Foo)
Help on class Foo in module foocpp:

class Foo(sip.wrapper)
 |  Class example from C++ library
 |  
 |  Method resolution order:
 |      Foo
 |      sip.wrapper
 |      sip.simplewrapper
 |      builtins.object
 |  
 |  Methods defined here:
 |  
 |  get_int_val(...)
 |      get_int_val(self) -> int
 |      Return integer value
 |  
 |  get_string_val(...)
 |      Return string value
 |      get_string_val(self) -> str
 |  
 |  set_int_val(...)
 |      set_int_val(self, int)
 |      Set integer value
 |  
 |  set_string_val(...)
 |      Set string value
 |      set_string_val(self, str)
...


Let's simplify this example by setting the default values ​​for the format and signature parameters using the % DefaultDocstringFormat and % DefaultDocstringSignature directives . The use of these directives is shown in the example from the pyfoo_cpp_04 folder . The pyfoocpp.sip file in this example contains the following code:

%Module(name=foocpp, language="C++")
%DefaultEncoding "UTF-8"
%DefaultDocstringFormat "deindented"
%DefaultDocstringSignature "prepended"

class Foo {
    %Docstring
    Class example from C++ library
    %End

    %TypeHeaderCode
    #include <foo.h>
    %End

    public:
        Foo(int, const char*);

        void set_int_val(int);
        %Docstring
            Set integer value
        %End

        int get_int_val();
        %Docstring
            Return integer value
        %End

        %Property(name=int_val, get=get_int_val, set=set_int_val)
        {
            %Docstring
                The property for integer value
            %End
        };

        void set_string_val(const char*);
        %Docstring
            Set string value
        %End

        char* get_string_val();
        %Docstring
            Return string value
        %End

        %Property(name=string_val, get=get_string_val, set=set_string_val)
        {
            %Docstring
                The property for string value
            %End
        };
};

At the beginning of the file, the lines % DefaultDocstringFormat "deindented" and % DefaultDocstringSignature "prepended" were added , and then all the parameters from the % Docstring directive were removed.

After assembling and installing this example, we can see what the description of the Foo class looks like now , which the help (Foo) command displays :

>>> from foocpp import Foo
>>> help(Foo)

class Foo(sip.wrapper)
 |  Class example from C++ library
 |  
 |  Method resolution order:
 |      Foo
 |      sip.wrapper
 |      sip.simplewrapper
 |      builtins.object
 |  
 |  Methods defined here:
 |  
 |  get_int_val(...)
 |      get_int_val(self) -> int
 |      Return integer value
 |  
 |  get_string_val(...)
 |      get_string_val(self) -> str
 |      Return string value
 |  
 |  set_int_val(...)
 |      set_int_val(self, int)
 |      Set integer value
 |  
 |  set_string_val(...)
 |      set_string_val(self, str)
 |      Set string value
...

Everything looks quite neat and of the same type.

Rename classes and methods


As we already said, the interface provided by the Python bindings does not have to match the interface that the C / C ++ library provides. We added properties to classes above, and now we’ll look at one more technique that can be useful if conflicts of class names or functions arise, for example, if a function name matches some Python keyword. To do this, you can rename classes, functions, exceptions, and other entities.

To rename an entity, the PyName annotation is used , the value of which needs to be assigned a new entity name. The work with the PyName annotation is shown in the example from the pyfoo_cpp_05 folder . This example is based on the previous example.pyfoo_cpp_04 and differs from it by the pyfoocpp.sip file , the contents of which now look like this:

%Module(name=foocpp, language="C++")
%DefaultEncoding "UTF-8"
%DefaultDocstringFormat "deindented"
%DefaultDocstringSignature "prepended"

class Foo /PyName=Bar/ {
    %Docstring
    Class example from C++ library
    %End

    %TypeHeaderCode
    #include <foo.h>
    %End

    public:
        Foo(int, const char*);

        void set_int_val(int) /PyName=set_integer_value/;
        %Docstring
            Set integer value
        %End

        int get_int_val() /PyName=get_integer_value/;
        %Docstring
            Return integer value
        %End

        %Property(name=int_val, get=get_integer_value, set=set_integer_value)
        {
            %Docstring
                The property for integer value
            %End
        };

        void set_string_val(const char*) /PyName=set_string_value/;
        %Docstring
            Set string value
        %End

        char* get_string_val() /PyName=get_string_value/;
        %Docstring
            Return string value
        %End

        %Property(name=string_val, get=get_string_value, set=set_string_value)
        {
            %Docstring
                The property for string value
            %End
        };
};

In this example, we renamed the Foo class to the Bar class , and also assigned other names to all methods using the PyName annotation . I think that everything here is quite simple and clear, the only thing worth paying attention to is the creation of properties. In the % Property directive, the get and set parameters must specify the names of the methods, as they will be called in the Python class, and not the names that they were originally called to C ++ code.

Compile the example, install it, and see how this class will look in Python:

>>> from foocpp import Bar
>>> help(Bar)

Help on class Bar in module foocpp:

class Bar(sip.wrapper)
 |  Class example from C++ library
 |  
 |  Method resolution order:
 |      Bar
 |      sip.wrapper
 |      sip.simplewrapper
 |      builtins.object
 |  
 |  Methods defined here:
 |  
 |  get_integer_value(...)
 |      get_integer_value(self) -> int
 |      Return integer value
 |  
 |  get_string_value(...)
 |      get_string_value(self) -> str
 |      Return string value
 |  
 |  set_integer_value(...)
 |      set_integer_value(self, int)
 |      Set integer value
 |  
 |  set_string_value(...)
 |      set_string_value(self, str)
 |      Set string value
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  int_val
 |      The property for integer value
 |  
 |  string_val
 |      The property for string value
 |  
 |  ----------------------------------------------------------------------
...

It worked! We managed to rename the class itself and its methods.

Sometimes libraries use the agreement that the names of all classes begin with a prefix, for example, with the letter “Q” in Qt or “wx” in wxWidgets. If in your Python binding you want to rename all classes, getting rid of such prefixes, then in order not to set the PyName annotation for each class, you can use the % AutoPyName directive . We will not consider this directive in this article, we will only say that the % AutoPyName directive should be located inside the % Module directive and restrict ourselves to an example from the documentation:

%Module PyQt5.QtCore
{
    %AutoPyName(remove_leading="Q")
}

Add type conversion


Example using the std :: wstring class


So far, we have looked at functions and classes that have worked with simple types like int and char * . For these types, SIP automatically created a converter to and from Python classes. In the following example, which is located in the pyfoo_cpp_06 folder , we will consider the case when class methods accept and return more complex objects, for example, strings from STL. To simplify the example and not complicate the conversion of bytes to Unicode and vice versa, the std :: wstring string class will be used in this example . The idea of ​​this example is to show how you can manually set the rules for converting C ++ classes to and from Python classes.

For this example, we will change the Foo class from the foo library. Now the class definition will look like this (file foo.h ):

#ifndef FOO_LIB
#define FOO_LIB

#include <string>

using std::wstring;

class Foo {
    private:
        int _int_val;
        wstring _string_val;
    public:
        Foo(int int_val, wstring string_val);

        void set_int_val(int val);
        int get_int_val();

        void set_string_val(wstring val);
        wstring get_string_val();
};

#endif

The implementation of the Foo class in the foo.cpp file :

#include <string>

#include "foo.h"

using std::wstring;

Foo::Foo(int int_val, wstring string_val):
    _int_val(int_val), _string_val(string_val) {}

void Foo::set_int_val(int val) {
    _int_val = val;
}

int Foo::get_int_val() {
    return _int_val;
}

void Foo::set_string_val(wstring val) {
    _string_val = val;
}

wstring Foo::get_string_val() {
    return _string_val;
}

And the main.cpp file for checking the library's functionality:

#include <iostream>

#include "foo.h"

using std::cout;
using std::endl;

int main(int argc, char* argv[]) {
    auto foo = Foo(10, L"Hello");
    cout << L"int_val: " << foo.get_int_val() << endl;
    cout << L"string_val: " << foo.get_string_val().c_str() << endl;

    foo.set_int_val(0);
    foo.set_string_val(L"Hello world!");

    cout << L"int_val: " << foo.get_int_val() << endl;
    cout << L"string_val: " << foo.get_string_val().c_str() << endl;
}

Files foo.h , foo.cpp and main.cpp , as before, are located in the foo folder . The makefile and library build process has not changed. There are also no significant changes to the pyproject.toml and project.py files .

But the pyfoocpp.sip file has become noticeably more complicated:

%Module(name=foocpp, language="C++")
%DefaultEncoding "UTF-8"

class Foo {
    %TypeHeaderCode
    #include <foo.h>
    %End

    public:
        Foo(int, std::wstring);

        void set_int_val(int);
        int get_int_val();
        %Property(name=int_val, get=get_int_val, set=set_int_val)

        void set_string_val(std::wstring);
        std::wstring get_string_val();
        %Property(name=string_val, get=get_string_val, set=set_string_val)
};

%MappedType std::wstring
{
%TypeHeaderCode
#include <string>
%End

%ConvertFromTypeCode
    // Convert an std::wstring to a Python (Unicode) string
    PyObject* newstring;
    newstring = PyUnicode_FromWideChar(sipCpp->data(), -1);
    return newstring;
%End

%ConvertToTypeCode
    // Convert a Python (Unicode) string to an std::wstring
    if (sipIsErr == NULL) {
        return PyUnicode_Check(sipPy);
    }
    if (PyUnicode_Check(sipPy)) {
        *sipCppPtr = new std::wstring(PyUnicode_AS_UNICODE(sipPy));
        return 1;
    }
    return 0;
%End
};

For illustrative purposes, the pyfoocpp.sip file does not add documentation lines. If we left only the declaration of the Foo class in the pyfoocpp.sip file without the subsequent % MappedType directive , then we would get the following error during the build process:

$ sip-wheel

These bindings will be built: pyfoocpp.
Generating the pyfoocpp bindings...
sip-wheel: std::wstring is undefined

We need to explicitly describe how an object of type std :: wstring will be converted to some Python object, and also describe the inverse transformation. To describe the conversion, we will need to work at a fairly low level in the C language and use the Python / C API . Since the Python / C API is a big topic, worthy of even a separate article, but a book, in this section we will consider only those functions that are used in the example, without going into too much detail.

To declare conversions from C ++ objects to Python and vice versa, the % MappedType directive is intended , inside which there may be three other directives: % TypeHeaderCode , % ConvertToTypeCode and % ConvertFromTypeCode. After the % MappedType expression, you need to specify the type for which converters will be created. In our case, the directive begins with the expression % MappedType std :: wstring . We have already met the % TypeHeaderCode

directive in the section Making a binding for a library in C ++ . Let me remind you that this directive is intended to declare the types used or to include the header files in which they are declared. In this example , the header file string , where the class std :: string is declared, is connected inside the % TypeHeaderCode directive . Now we need to describe the transformations



% ConvertFromTypeCode. Converting C ++ Objects to Python


We start by converting std :: wstring objects to the Python class str . This conversion in the example is as follows:

%ConvertFromTypeCode
    // Convert an std::wstring to a Python (Unicode) string
    PyObject* newstring;
    newstring = PyUnicode_FromWideChar(sipCpp->data(), -1);
    return newstring;
%End

Inside this directive, we have the sipCpp variable - a pointer to an object from C ++ code, by which we need to create a Python object and return the created object from the directive using the return statement . In this case, the sipCpp variable is of type std :: wstring * . To create the str class , use the PyUnicode_FromWideChar function from the Python / C API. This function accepts an array (pointer) of type const wchar_t * w as the first parameter , and the size of this array as the second parameter. If you pass the value -1 as the second parameter, then the PyUnicode_FromWideChar function itself will calculate the length using the functionwcslen .

To get the wchar_t * array, use the data method from the std :: wstring class .

The PyUnicode_FromWideChar function returns a pointer to PyObject or NULL in case of an error. PyObject is any Python object, in this case it will be the str class . In the Python / C API, work with objects usually occurs through PyObject * pointers , so in this case, we return the PyObject * pointer from the % ConvertFromTypeCode directive .

% ConvertToTypeCode. Convert Python objects to C ++


The inverse conversion from a Python object (essentially from PyObject * ) to the std :: wstring class is described in the % ConvertToTypeCode directive . In the pyfoo_cpp_06 example , the conversion is as follows:

%ConvertToTypeCode
    // Convert a Python (Unicode) string to an std::wstring
    if (sipIsErr == NULL) {
        return PyUnicode_Check(sipPy);
    }
    if (PyUnicode_Check(sipPy)) {
        *sipCppPtr = new std::wstring(PyUnicode_AS_UNICODE(sipPy));
        return 1;
    }
    return 0;
%End

The code of the % ConvertToTypeCode directive looks more complicated, because during the conversion process it is called several times for different purposes. Inside the % ConvertToTypeCode directive, SIP creates several variables that we can (or should) use.

One of these variables, PyObject * sipPy, is a Python object that you need to create an instance of the std :: wstring class in this case . The result will need to be written to another variable - sipCppPtr is a double pointer to the created object, i.e. in our case, this variable will be of type std :: wstring ** .

Another % ConvertToTypeCode created inside the directivethe variable is int * sipIsErr . If the value of this variable is NULL , then the % ConvertToTypeCode directive is called only to check if type conversion is possible. In this case, we are not obliged to carry out the transformation, but only need to check whether it is possible in principle. If possible, then the directive must return a non-zero value, otherwise, if conversion is not possible, they must return 0. If this pointer is not NULL , then you need to perform the conversion, and if an error occurs during the conversion, the integer error code can be saved into this variable (given that this variable is a pointer to int * ).

In this example, to verify that sipPy is a unicode string ( str class ), the PyUnicode_Check macro is used , which takes an argument of type PyObject * if the passed argument is a unicode string or a class derived from it.

Conversion to a C ++ object is performed using the string * sipCppPtr = new std :: wstring (PyUnicode_AS_UNICODE (sipPy)); . This calls the PyUnicode_AS_UNICODE macro from the Python / C API, which returns an array of type Py_UNICODE * , which is equivalent to wchar_t * . This array is passed to the constructor of the std :: wstring class. As mentioned above, the result is stored in the sipCppPtr variable .

At the moment, the PyUnicode_AS_UNICODE directive is deprecated and it is recommended to use other macros, but this macro is used to simplify the example.

If the conversion was successful, the % ConvertToTypeCode directive should return a non-zero value (in this case 1), and in case of an error it should return 0.

Check


We described the conversion of type std :: wstring to str and vice versa, now we can make sure that the package is successfully built and the binding works as it should. To build, call sip-wheel , then install the package using pip and check the operability in Python command mode:

>>> from foocpp import Foo
>>> x = Foo(10, 'Hello')

>>> x.string_val
'Hello'

>>> x.string_val = ''
>>> x.string_val
''

>>> x.get_string_val()
''

As you can see, everything works, there are no problems with the Russian language either, i.e. Unicode string conversions performed correctly.

Conclusion


In this article, we covered the basics of using SIP to create Python bindings for libraries written in C and C ++. First (in the first part ) we created a simple library in C and figured out the files that need to be created to work with SIP. The pyproject.toml file contains information about the package (name, version number, license, and paths to header and object files). Using the project.py file, you can influence the process of building the Python package, for example, start building the C / C ++ library or allow the user to specify the location of the header and object files of the library.

In the * .sip filedescribes the interface of the Python module listing the functions and classes that will be contained in the module. Directives and annotations are used to describe the interface in the * .sip file . The Python class interface does not have to match the C ++ class interface. For example, you can add properties to classes using the % Property directive , rename entities using the / PyName / annotation , and add documentation lines using the % Docstring directive .

Elementary types like int , char , char *etc. SIP automatically converts to similar Python classes, but if you need to perform a more complex conversion, you need to program it yourself inside the % MappedType directive using the Python / C API. Conversion from the Python class to C ++ must be done in the % ConvertToTypeCode nested directive . Converting from a C ++ type to a Python class must be done in the % ConvertFromTypeCode nested directive .

Some directives like % DefaultEncoding , % DefaultDocstringFormat and % DefaultDocstringSignature are auxiliary and allow you to set default values ​​for cases when some annotation parameters are not set explicitly.

In this article, we examined only the basic and simplest directives and annotations, but many of them were ignored. For example, there are directives for managing GIL, for creating new Python exceptions, for manually managing memory and garbage collectors, for tweaking classes for different operating systems, and many others that can be useful when creating complex C / C ++ library bindings. We also circumvented the issue of building packages for different operating systems, limiting ourselves to building under Linux using gcc / g ++ compilers.

References


More articles:


All Articles