Multiple inheritance support

README.md

@@ -45,10 +45,12 @@ pybind11 can map the following core C++ features to Python
 - Arbitrary exception types
 - Enumerations
 - Callbacks
+- Iterators and ranges
 - Custom operators
+- Single and multiple inheritance
 - STL data structures
 - Iterators and ranges
-- Smart pointers with reference counting like `std::shared_ptr`
+- Smart pointers with reference counting like ``std::shared_ptr``
 - Internal references with correct reference counting
 - C++ classes with virtual (and pure virtual) methods can be extended in Python
 

docs/advanced.rst

@@ -217,6 +217,8 @@ The following interactive session shows how to call them from Python.
     that demonstrates how to work with callbacks and anonymous functions in
     more detail.
 
+.. _overriding_virtuals:
+
 Overriding virtual functions in Python
 ======================================
 
@@ -2151,3 +2153,45 @@ type is explicitly allowed.
             };
         }
     };
+
+Multiple Inheritance
+====================
+
+pybind11 can create bindings for types that derive from multiple base types
+(aka. *multiple inheritance*). To do so, specify all bases in the template
+arguments of the ``class_`` declaration:
+
+.. code-block:: cpp
+
+    py::class_<MyType, BaseType1, BaseType2, BaseType3>(m, "MyType")
+       ...
+
+The base types can be specified in arbitrary order, and they can even be
+interspersed with alias types and holder types (discussed earlier in this
+document)---pybind11 will automatically find out which is which. The only
+requirement is that the first template argument is the type to be declared.
+
+There are two caveats regarding the implementation of this feature:
+
+1. When only one base type is specified for a C++ type that actually has
+   multiple bases, pybind11 will assume that it does not participate in
+   multiple inheritance, which can lead to undefined behavior. In such cases,
+   add the tag ``multiple_inheritance``:
+
+    .. code-block:: cpp
+
+        py::class_<MyType, BaseType2>(m, "MyType", py::multiple_inheritance());
+
+   The tag is redundant and does not need to be specified when multiple base
+   types are listed.
+
+2. As was previously discussed in the section on :ref:`overriding_virtuals`, it
+   is easy to create Python types that derive from C++ classes. It is even
+   possible to make use of multiple inheritance to declare a Python class which
+   has e.g. a C++ and a Python class as bases. However, any attempt to create a
+   type that has *two or more* C++ classes in its hierarchy of base types will
+   fail with a fatal error message: ``TypeError: multiple bases have instance
+   lay-out conflict``. Core Python types that are implemented in C (e.g.
+   ``dict``, ``list``, ``Exception``, etc.) also fall under this combination
+   and cannot be combined with C++ types bound using pybind11 via multiple
+   inheritance.

docs/intro.rst

@@ -35,12 +35,14 @@ The following core C++ features can be mapped to Python
 - Instance methods and static methods
 - Overloaded functions
 - Instance attributes and static attributes
-- Exceptions
+- Arbitrary exception types
 - Enumerations
-- Iterators and ranges
 - Callbacks
+- Iterators and ranges
 - Custom operators
+- Single and multiple inheritance
 - STL data structures
+- Iterators and ranges
 - Smart pointers with reference counting like ``std::shared_ptr``
 - Internal references with correct reference counting
 - C++ classes with virtual (and pure virtual) methods can be extended in Python

docs/limitations.rst

@@ -9,15 +9,12 @@ certain limitations:
   values. This means that some additional care is needed to avoid bugs that
   would be caught by the type checker in a traditional C++ program.
 
-- Multiple inheritance relationships on the C++ side cannot be mapped to
-  Python.
-
 - The NumPy interface ``pybind11::array`` greatly simplifies accessing
   numerical data from C++ (and vice versa), but it's not a full-blown array
   class like ``Eigen::Array`` or ``boost.multi_array``.
 
-All of these features could be implemented but would lead to a significant
-increase in complexity. I've decided to draw the line here to keep this project
-simple and compact. Users who absolutely require these features are encouraged
-to fork pybind11.
+These features could be implemented but would lead to a significant increase in
+complexity. I've decided to draw the line here to keep this project simple and
+compact. Users who absolutely require these features are encouraged to fork
+pybind11.
 

include/pybind11/attr.h

@@ -41,6 +41,9 @@ template <typename T> struct base {
 /// Keep patient alive while nurse lives
 template <int Nurse, int Patient> struct keep_alive { };
 
+/// Annotation indicating that a class is involved in a multiple inheritance relationship
+struct multiple_inheritance { };
+
 NAMESPACE_BEGIN(detail)
 /* Forward declarations */
 enum op_id : int;
@@ -127,6 +130,8 @@ struct function_record {
 
 /// Special data structure which (temporarily) holds metadata about a bound class
 struct type_record {
+    PYBIND11_NOINLINE type_record() { }
+
     /// Handle to the parent scope
     handle scope;
 
@@ -148,21 +153,36 @@ struct type_record {
     /// Function pointer to class_<..>::dealloc
     void (*dealloc)(PyObject *) = nullptr;
 
-    // Pointer to RTTI type_info data structure of base class
-    const std::type_info *base_type = nullptr;
-
-    /// OR: Python handle to base class
-    handle base_handle;
+    /// List of base classes of the newly created type
+    list bases;
 
     /// Optional docstring
     const char *doc = nullptr;
+
+    /// Multiple inheritance marker
+    bool multiple_inheritance = false;
+
+    PYBIND11_NOINLINE void add_base(const std::type_info *base, void *(*caster)(void *)) {
+        auto base_info = detail::get_type_info(*base, false);
+        if (!base_info) {
+            std::string tname(base->name());
+            detail::clean_type_id(tname);
+            pybind11_fail("generic_type: type \"" + std::string(name) +
+                          "\" referenced unknown base type \"" + tname + "\"");
+        }
+
+        bases.append((PyObject *) base_info->type);
+
+        if (caster)
+            base_info->implicit_casts.push_back(std::make_pair(type, caster));
+    }
 };
 
 /**
  * Partial template specializations to process custom attributes provided to
  * cpp_function_ and class_. These are either used to initialize the respective
- * fields in the type_record and function_record data structures or executed
- * at runtime to deal with custom call policies (e.g. keep_alive).
+ * fields in the type_record and function_record data structures or executed at
+ * runtime to deal with custom call policies (e.g. keep_alive).
  */
 template <typename T, typename SFINAE = void> struct process_attribute;
 
@@ -257,13 +277,19 @@ template <> struct process_attribute<arg_v> : process_attribute_default<arg_v> {
 /// Process a parent class attribute
 template <typename T>
 struct process_attribute<T, typename std::enable_if<std::is_base_of<handle, T>::value>::type> : process_attribute_default<handle> {
-    static void init(const handle &h, type_record *r) { r->base_handle = h; }
+    static void init(const handle &h, type_record *r) { r->bases.append(h); }
 };
 
-/// Process a parent class attribute
+/// Process a parent class attribute (deprecated, does not support multiple inheritance)
 template <typename T>
 struct process_attribute<base<T>> : process_attribute_default<base<T>> {
-    static void init(const base<T> &, type_record *r) { r->base_type = &typeid(T); }
+    static void init(const base<T> &, type_record *r) { r->add_base(&typeid(T), nullptr); }
+};
+
+/// Process a multiple inheritance attribute
+template <>
+struct process_attribute<multiple_inheritance> : process_attribute_default<multiple_inheritance> {
+    static void init(const multiple_inheritance &, type_record *r) { r->multiple_inheritance = true; }
 };
 
 /***

include/pybind11/common.h

@@ -89,6 +89,7 @@
 #include <unordered_map>
 #include <memory>
 #include <typeindex>
+#include <type_traits>
 
 #if PY_MAJOR_VERSION >= 3 /// Compatibility macros for various Python versions
 #define PYBIND11_INSTANCE_METHOD_NEW(ptr, class_) PyInstanceMethod_New(ptr)

include/pybind11/operators.h

@@ -10,7 +10,6 @@
 #pragma once
 
 #include "pybind11.h"
-#include <type_traits>
 
 #if defined(__clang__) && !defined(__INTEL_COMPILER)
 #  pragma clang diagnostic ignored "-Wunsequenced" // multiple unsequenced modifications to 'self' (when using def(py::self OP Type()))

include/pybind11/pybind11.h

@@ -17,6 +17,7 @@
 #  pragma warning(disable: 4512) // warning C4512: Assignment operator was implicitly defined as deleted
 #  pragma warning(disable: 4800) // warning C4800: 'int': forcing value to bool 'true' or 'false' (performance warning)
 #  pragma warning(disable: 4996) // warning C4996: The POSIX name for this item is deprecated. Instead, use the ISO C and C++ conformant name
+#  pragma warning(disable: 4702) // warning C4702: unreachable code
 #elif defined(__INTEL_COMPILER)
 #  pragma warning(push)
 #  pragma warning(disable: 186)   // pointless comparison of unsigned integer with zero
@@ -576,18 +577,6 @@ public:
     PYBIND11_OBJECT_DEFAULT(generic_type, object, PyType_Check)
 protected:
     void initialize(type_record *rec) {
-        if (rec->base_type) {
-            if (rec->base_handle)
-                pybind11_fail("generic_type: specified base type multiple times!");
-            rec->base_handle = detail::get_type_handle(*(rec->base_type));
-            if (!rec->base_handle) {
-                std::string tname(rec->base_type->name());
-                detail::clean_type_id(tname);
-                pybind11_fail("generic_type: type \"" + std::string(rec->name) +
-                              "\" referenced unknown base type \"" + tname + "\"");
-            }
-        }
-
         auto &internals = get_internals();
         auto tindex = std::type_index(*(rec->type));
 
@@ -617,6 +606,12 @@ protected:
             ht_qualname = name;
         }
 #endif
+
+        size_t num_bases = rec->bases.size();
+        tuple bases(num_bases);
+        for (size_t i = 0; i < num_bases; ++i)
+            bases[i] = rec->bases[i];
+
         std::string full_name = (scope_module ? ((std::string) scope_module.str() + "." + rec->name)
                                               : std::string(rec->name));
 
@@ -629,6 +624,11 @@ protected:
             memcpy((void *) tp_doc, rec->doc, size);
         }
 
+        /* Danger zone: from now (and until PyType_Ready), make sure to
+           issue no Python C API calls which could potentially invoke the
+           garbage collector (the GC will call type_traverse(), which will in
+           turn find the newly constructed type in an invalid state) */
+
         object type_holder(PyType_Type.tp_alloc(&PyType_Type, 0), false);
         auto type = (PyHeapTypeObject*) type_holder.ptr();
 
@@ -646,8 +646,12 @@ protected:
         /* Basic type attributes */
         type->ht_type.tp_name = strdup(full_name.c_str());
         type->ht_type.tp_basicsize = (ssize_t) rec->instance_size;
-        type->ht_type.tp_base = (PyTypeObject *) rec->base_handle.ptr();
-        rec->base_handle.inc_ref();
+
+        if (num_bases > 0) {
+            type->ht_type.tp_base = (PyTypeObject *) ((object) bases[0]).inc_ref().ptr();
+            type->ht_type.tp_bases = bases.release().ptr();
+            rec->multiple_inheritance |= num_bases > 1;
+        }
 
         type->ht_name = name.release().ptr();
 
@@ -689,9 +693,23 @@ protected:
         if (rec->scope)
             rec->scope.attr(handle(type->ht_name)) = *this;
 
+        if (rec->multiple_inheritance)
+            mark_parents_nonsimple(&type->ht_type);
+
         type_holder.release();
     }
 
+    /// Helper function which tags all parents of a type using mult. inheritance
+    void mark_parents_nonsimple(PyTypeObject *value) {
+        tuple t(value->tp_bases, true);
+        for (handle h : t) {
+            auto tinfo2 = get_type_info((PyTypeObject *) h.ptr());
+            if (tinfo2)
+                tinfo2->simple_type = false;
+            mark_parents_nonsimple((PyTypeObject *) h.ptr());
+        }
+    }
+
     /// Allocate a metaclass on demand (for static properties)
     handle metaclass() {
         auto &ht_type = ((PyHeapTypeObject *) m_ptr)->ht_type;
@@ -811,31 +829,18 @@ protected:
     static void releasebuffer(PyObject *, Py_buffer *view) { delete (buffer_info *) view->internal; }
 };
 
-template <template<typename> class Predicate, typename... BaseTypes> struct class_selector;
-template <template<typename> class Predicate, typename Base, typename... Bases>
-struct class_selector<Predicate, Base, Bases...> {
-    static inline void set_bases(detail::type_record &record) {
-        if (Predicate<Base>::value) record.base_type = &typeid(Base);
-        else class_selector<Predicate, Bases...>::set_bases(record);
-    }
-};
-template <template<typename> class Predicate>
-struct class_selector<Predicate> {
-    static inline void set_bases(detail::type_record &) {}
-};
-
 NAMESPACE_END(detail)
 
 template <typename type_, typename... options>
 class class_ : public detail::generic_type {
     template <typename T> using is_holder = detail::is_holder_type<type_, T>;
     template <typename T> using is_subtype = detail::bool_constant<std::is_base_of<type_, T>::value && !std::is_same<T, type_>::value>;
-    template <typename T> using is_base_class = detail::bool_constant<std::is_base_of<T, type_>::value && !std::is_same<T, type_>::value>;
+    template <typename T> using is_base = detail::bool_constant<std::is_base_of<T, type_>::value && !std::is_same<T, type_>::value>;
     template <typename T> using is_valid_class_option =
         detail::bool_constant<
             is_holder<T>::value ||
             is_subtype<T>::value ||
-            is_base_class<T>::value
+            is_base<T>::value
         >;
 
 public:
@@ -848,9 +853,6 @@ public:
     static_assert(detail::all_of_t<is_valid_class_option, options...>::value,
             "Unknown/invalid class_ template parameters provided");
 
-    static_assert(detail::count_t<is_base_class, options...>::value <= 1,
-            "Invalid class_ base types: multiple inheritance is not supported");
-
     PYBIND11_OBJECT(class_, detail::generic_type, PyType_Check)
 
     template <typename... Extra>
@@ -864,7 +866,9 @@ public:
         record.init_holder = init_holder;
         record.dealloc = dealloc;
 
-        detail::class_selector<is_base_class, options...>::set_bases(record);
+        /* Register base classes specified via template arguments to class_, if any */
+        bool unused[] = { (add_base<options>(record), false)... };
+        (void) unused;
 
         /* Process optional arguments, if any */
         detail::process_attributes<Extra...>::init(extra..., &record);
@@ -877,6 +881,16 @@ public:
         }
     }
 
+    template <typename Base, std::enable_if_t<is_base<Base>::value, int> = 0>
+    static void add_base(detail::type_record &rec) {
+        rec.add_base(&typeid(Base), [](void *src) -> void * {
+            return static_cast<Base *>(reinterpret_cast<type *>(src));
+        });
+    }
+
+    template <typename Base, std::enable_if_t<!is_base<Base>::value, int> = 0>
+    static void add_base(detail::type_record &) { }
+
     template <typename Func, typename... Extra>
     class_ &def(const char *name_, Func&& f, const Extra&... extra) {
         cpp_function cf(std::forward<Func>(f), name(name_),
@@ -1198,7 +1212,7 @@ template <return_value_policy Policy = return_value_policy::reference_internal,
 iterator make_iterator(Iterator first, Sentinel last, Extra &&... extra) {
     typedef detail::iterator_state<Iterator, Sentinel, false, Policy> state;
 
-    if (!detail::get_type_info(typeid(state))) {
+    if (!detail::get_type_info(typeid(state), false)) {
         class_<state>(handle(), "iterator")
             .def("__iter__", [](state &s) -> state& { return s; })
             .def("__next__", [](state &s) -> ValueType {
@@ -1223,7 +1237,7 @@ template <return_value_policy Policy = return_value_policy::reference_internal,
 iterator make_key_iterator(Iterator first, Sentinel last, Extra &&... extra) {
     typedef detail::iterator_state<Iterator, Sentinel, true, Policy> state;
 
-    if (!detail::get_type_info(typeid(state))) {
+    if (!detail::get_type_info(typeid(state), false)) {
         class_<state>(handle(), "iterator")
             .def("__iter__", [](state &s) -> state& { return s; })
             .def("__next__", [](state &s) -> KeyType {

include/pybind11/pytypes.h

@@ -612,7 +612,7 @@ public:
     }
     size_t size() const { return (size_t) PyList_Size(m_ptr); }
     detail::list_accessor operator[](size_t index) const { return detail::list_accessor(*this, index); }
-    void append(const object &object) const { PyList_Append(m_ptr, object.ptr()); }
+    void append(handle h) const { PyList_Append(m_ptr, h.ptr()); }
 };
 
 class args : public tuple { PYBIND11_OBJECT_DEFAULT(args, tuple, PyTuple_Check) };

include/pybind11/stl_bind.h

@@ -12,8 +12,6 @@
 #include "common.h"
 #include "operators.h"
 
-#include <type_traits>
-#include <utility>
 #include <algorithm>
 #include <sstream>
 

tests/CMakeLists.txt

@@ -27,9 +27,9 @@ set(PYBIND11_TEST_FILES
   test_pickling.cpp
   test_python_types.cpp
   test_sequences_and_iterators.cpp
-  test_smart_ptr.cpp
   test_stl_binders.cpp
   test_virtual_functions.cpp
+  test_multiple_inheritance.cpp
 )
 
 string(REPLACE ".cpp" ".py" PYBIND11_PYTEST_FILES "${PYBIND11_TEST_FILES}")

tests/test_inheritance.cpp

@@ -61,7 +61,7 @@ test_initializer inheritance([](py::module &m) {
         .def(py::init<std::string>());
 
     /* Another way of declaring a subclass relationship: reference parent's C++ type */
-    py::class_<Rabbit>(m, "Rabbit", py::base<Pet>())
+    py::class_<Rabbit, Pet>(m, "Rabbit")
         .def(py::init<std::string>());
 
     /* And another: list parent in class template arguments */

tests/test_multiple_inheritance.cpp

+/*
+    tests/test_multiple_inheritance.cpp -- multiple inheritance,
+    implicit MI casts
+
+    Copyright (c) 2016 Wenzel Jakob <wenzel.jakob@epfl.ch>
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#include "pybind11_tests.h"
+
+PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
+
+struct Base1 {
+    Base1(int i) : i(i) { }
+    int foo() { return i; }
+    int i;
+};
+
+struct Base2 {
+    Base2(int i) : i(i) { }
+    int bar() { return i; }
+    int i;
+};
+
+struct Base12 : Base1, Base2 {
+    Base12(int i, int j) : Base1(i), Base2(j) { }
+};
+
+struct MIType : Base12 {
+    MIType(int i, int j) : Base12(i, j) { }
+};
+
+test_initializer multiple_inheritance([](py::module &m) {
+    py::class_<Base1>(m, "Base1")
+        .def(py::init<int>())
+        .def("foo", &Base1::foo);
+
+    py::class_<Base2>(m, "Base2")
+        .def(py::init<int>())
+        .def("bar", &Base2::bar);
+
+    py::class_<Base12, Base1, Base2>(m, "Base12");
+
+    py::class_<MIType, Base12>(m, "MIType")
+        .def(py::init<int, int>());
+});
+
+/* Test the case where not all base classes are specified,
+   and where pybind11 requires the py::multiple_inheritance
+   flag to perform proper casting between types */
+
+struct Base1a {
+    Base1a(int i) : i(i) { }
+    int foo() { return i; }
+    int i;
+};
+
+struct Base2a {
+    Base2a(int i) : i(i) { }
+    int bar() { return i; }
+    int i;
+};
+
+struct Base12a : Base1a, Base2a {
+    Base12a(int i, int j) : Base1a(i), Base2a(j) { }
+};
+
+test_initializer multiple_inheritance_nonexplicit([](py::module &m) {
+    py::class_<Base1a, std::shared_ptr<Base1a>>(m, "Base1a")
+        .def(py::init<int>())
+        .def("foo", &Base1a::foo);
+
+    py::class_<Base2a, std::shared_ptr<Base2a>>(m, "Base2a")
+        .def(py::init<int>())
+        .def("bar", &Base2a::bar);
+
+    py::class_<Base12a, /* Base1 missing */ Base2a,
+               std::shared_ptr<Base12a>>(m, "Base12a", py::multiple_inheritance())
+        .def(py::init<int, int>());
+
+    m.def("bar_base2a", [](Base2a *b) { return b->bar(); });
+    m.def("bar_base2a_sharedptr", [](std::shared_ptr<Base2a> b) { return b->bar(); });
+});

tests/test_multiple_inheritance.py

+import pytest
+
+
+def test_multiple_inheritance_cpp(msg):
+    from pybind11_tests import MIType
+
+    mt = MIType(3, 4)
+
+    assert mt.foo() == 3
+    assert mt.bar() == 4
+
+
+def test_multiple_inheritance_mix1(msg):
+    from pybind11_tests import Base2
+
+    class Base1:
+        def __init__(self, i):
+            self.i = i
+
+        def foo(self):
+            return self.i
+
+    class MITypePy(Base1, Base2):
+        def __init__(self, i, j):
+            Base1.__init__(self, i)
+            Base2.__init__(self, j)
+
+    mt = MITypePy(3, 4)
+
+    assert mt.foo() == 3
+    assert mt.bar() == 4
+
+
+def test_multiple_inheritance_mix2(msg):
+    from pybind11_tests import Base1
+
+    class Base2:
+        def __init__(self, i):
+            self.i = i
+
+        def bar(self):
+            return self.i
+
+    class MITypePy(Base1, Base2):
+        def __init__(self, i, j):
+            Base1.__init__(self, i)
+            Base2.__init__(self, j)
+
+    mt = MITypePy(3, 4)
+
+    assert mt.foo() == 3
+    assert mt.bar() == 4
+
+
+def test_multiple_inheritance_virtbase(msg):
+    from pybind11_tests import Base12a, bar_base2a, bar_base2a_sharedptr
+
+    class MITypePy(Base12a):
+        def __init__(self, i, j):
+            Base12a.__init__(self, i, j)
+
+    mt = MITypePy(3, 4)
+    assert mt.bar() == 4
+    assert bar_base2a(mt) == 4
+    assert bar_base2a_sharedptr(mt) == 4