Add new options and docs for pybind11_add_module

See the documentation for a description of the options.

Add new options and docs for pybind11_add_module
See the documentation for a description of the options.
0cbec5c9 · Dean Moldovan · Wenzel Jakob · b0f3885c · 0cbec5c9 · 0cbec5c9
Commit 0cbec5c9 authored Dec 16, 2016 by Dean Moldovan Committed by Wenzel Jakob Dec 19, 2016
Showing with 145 additions and 67 deletions

docs/compiling.rst
+93 -33

tests/test_cmake_build/installed_function/CMakeLists.txt
+1 -1

tests/test_cmake_build/subdirectory_function/CMakeLists.txt
+1 -1

tools/pybind11Tools.cmake
+50 -32

No files found.
--- a/docs/compiling.rst
+++ b/docs/compiling.rst
@@ -39,30 +39,88 @@ extension module can be created with just a few lines of code:
 This assumes that the pybind11 repository is located in a subdirectory named
 :file:`pybind11` and that the code is located in a file named :file:`example.cpp`.
-The CMake command ``add_subdirectory`` will import a function with the signature
+The CMake command ``add_subdirectory`` will import the pybind11 project which
-``pybind11_add_module(<name> source1 [source2 ...])``. It will take care of all
+provides the ``pybind11_add_module`` function. It will take care of all the
-the details needed to build a Python extension module on any platform.
+details needed to build a Python extension module on any platform.
-The target Python version can be selected by setting the ``PYBIND11_PYTHON_VERSION``
-variable before adding the pybind11 subdirectory. Alternatively, an exact Python
-installation can be specified by setting ``PYTHON_EXECUTABLE``.
 A working sample project, including a way to invoke CMake from :file:`setup.py` for
 PyPI integration, can be found in the [cmake_example]_  repository.
 .. [cmake_example] https://github.com/pybind/cmake_example
-For CMake-based projects that don't include the pybind11
+pybind11_add_module
-repository internally, an external installation can be detected
+-------------------
-through `find_package(pybind11 ... CONFIG ...)`. See the `Config file
-<https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in>`_
+To ease the creation of Python extension modules, pybind11 provides a CMake
-docstring for details of relevant CMake variables.
+function with the following signature:
+.. code-block:: cmake
+    pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
+                        [NO_EXTRAS] [THIN_LTO] source1 [source2 ...])
+This function behaves very much like CMake's builtin ``add_library`` (in fact,
+it's a wrapper function around that command). It will add a library target
+called ``<name>`` to be built from the listed source files. In addition, it
+will take care of all the Python-specific compiler and linker flags as well
+as the OS- and Python-version-specific file extension. The produced target
+``<name>`` can be further manipulated with regular CMake commands.
+``MODULE`` or ``SHARED`` may be given to specify the type of library. If no
+type is given, ``MODULE`` is used by default which ensures the creation of a
+Python-exclusive module. Specifying ``SHARED`` will create a more traditional
+dynamic library which can also be linked from elsewhere. ``EXCLUDE_FROM_ALL``
+removes this target from the default build (see CMake docs for details).
+Since pybind11 is a template library, ``pybind11_add_module`` adds compiler
+flags to ensure high quality code generation without bloat arising from long
+symbol names and duplication of code in different translation units. The
+additional flags enable LTO (Link Time Optimization), set default visibility
+to *hidden* and strip unneeded symbols. See the :ref:`FAQ entry <faq:symhidden>`
+for a more detailed explanation. These optimizations are never applied in
+``Debug`` mode. If ``NO_EXTRAS`` is given, they will always be disabled, even
+in ``Release`` mode. However, this will result in code bloat and is generally
+not recommended.
+As stated above, LTO is enabled by default. Some newer compilers also support
+different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause
+the function to prefer this flavor if available. The function falls back to
+regular LTO if ``-flto=thin`` is not available.
+.. _ThinLTO: http://clang.llvm.org/docs/ThinLTO.html
+Configuration variables
+-----------------------
+By default, pybind11 will compile modules with the latest C++ standard
+available on the target compiler. To override this, the standard flag can
+be given explicitly in ``PYBIND11_CPP_STANDARD``:
+.. code-block:: cmake
+    set(PYBIND11_CPP_STANDARD -std=c++11)
+    add_subdirectory(pybind11)  # or find_package(pybind11)
+Note that this and all other configuration variables must be set **before** the
+call to ``add_subdiretory`` or ``find_package``. The variables can also be set
+when calling CMake from the command line using the ``-D<variable>=<value>`` flag.
+The target Python version can be selected by setting ``PYBIND11_PYTHON_VERSION``
+or an exact Python installation can be specified with ``PYTHON_EXECUTABLE``.
+For example:
-Once detected, and after setting any variables to guide Python and
+.. code-block:: bash
-C++ standard detection, the aforementioned ``pybind11_add_module``
-wrapper to ``add_library`` can be employed as described above (after
+    cmake -DPYBIND11_PYTHON_VERSION=3.6 ..
-``include(pybind11Tools)``). This procedure is available when using CMake
+    # or
->= 2.8.12. A working example can be found at [test_installed_module]_ .
+    cmake -DPYTHON_EXECUTABLE=path/to/python ..
+find_package vs. add_subdirectory
+---------------------------------
+For CMake-based projects that don't include the pybind11 repository internally,
+an external installation can be detected through ``find_package(pybind11)``.
+See the `Config file`_ docstring for details of relevant CMake variables.
 .. code-block:: cmake
@@ -72,27 +130,32 @@ wrapper to ``add_library`` can be employed as described above (after
    find_package(pybind11 REQUIRED)
    pybind11_add_module(example example.cpp)
-.. [test_installed_module] https://github.com/pybind/pybind11/blob/master/tests/test_installed_module/CMakeLists.txt
+Once detected, the aforementioned ``pybind11_add_module`` can be employed as
+before. The function usage and configuration variables are identical no matter
+if pybind11 is added as a subdirectory or found as an installed package. You
+can refer to the same [cmake_example]_ repository for a full sample project
+-- just swap out ``add_subdirectory`` for ``find_package``.
+.. _Config file: https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in
+Advanced: interface library target
+----------------------------------
-When using a version of CMake greater than 3.0, pybind11 can
+When using a version of CMake greater than 3.0, pybind11 can additionally
-additionally be used as a special *interface library* following the
+be used as a special *interface library* . The target ``pybind11::pybind11``
-call to ``find_package``. CMake variables to guide Python and C++
+is available with pybind11 headers, Python headers and libraries as needed,
-standard detection should be set *before* ``find_package``. When
+and C++ compile definitions attached. This target is suitable for linking
-``find_package`` returns, the target ``pybind11::pybind11`` is
+to an independently constructed (through ``add_library``, not
-available with pybind11 headers, Python headers and libraries as
+``pybind11_add_module``) target in the consuming project.
-needed, and C++ compile definitions attached. This target is suitable
-for linking to an independently constructed (through ``add_library``,
-not ``pybind11_add_module``) target in the consuming project. A working
-example can be found at [test_installed_target]_ .
 .. code-block:: cmake
    cmake_minimum_required(VERSION 3.0)
    project(example)
-    add_library(example MODULE main.cpp)
+    find_package(pybind11 REQUIRED)  # or add_subdirectory(pybind11)
-    find_package(pybind11 REQUIRED)
+    add_library(example MODULE main.cpp)
    target_link_libraries(example PRIVATE pybind11::pybind11)
    set_target_properties(example PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}"
                                             SUFFIX "${PYTHON_MODULE_EXTENSION}")
@@ -111,6 +174,3 @@ example can be found at [test_installed_target]_ .
    (``-fvisibility=hidden``) and .OBJ files with many sections on Visual Studio
    (``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an
    explanation on why these are needed.
-.. [test_installed_target] https://github.com/pybind/pybind11/blob/master/tests/test_installed_target/CMakeLists.txt
--- a/tests/test_cmake_build/installed_function/CMakeLists.txt
+++ b/tests/test_cmake_build/installed_function/CMakeLists.txt
@@ -6,7 +6,7 @@ set(CMAKE_MODULE_PATH "")
 find_package(pybind11 CONFIG REQUIRED)
 message(STATUS "Found pybind11 v${pybind11_VERSION}: ${pybind11_INCLUDE_DIRS}")
-pybind11_add_module(test_cmake_build SHARED ../main.cpp)
+pybind11_add_module(test_cmake_build SHARED NO_EXTRAS ../main.cpp)
 add_custom_target(check ${CMAKE_COMMAND} -E env PYTHONPATH=$<TARGET_FILE_DIR:test_cmake_build>
                  ${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/../test.py ${PROJECT_NAME})
--- a/tests/test_cmake_build/subdirectory_function/CMakeLists.txt
+++ b/tests/test_cmake_build/subdirectory_function/CMakeLists.txt
@@ -2,7 +2,7 @@ cmake_minimum_required(VERSION 2.8.12)
 project(test_subdirectory_module CXX)
 add_subdirectory(${PYBIND11_PROJECT_DIR} pybind11)
-pybind11_add_module(test_cmake_build ../main.cpp)
+pybind11_add_module(test_cmake_build THIN_LTO ../main.cpp)
 add_custom_target(check ${CMAKE_COMMAND} -E env PYTHONPATH=$<TARGET_FILE_DIR:test_cmake_build>
                  ${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/../test.py ${PROJECT_NAME})
--- a/tools/pybind11Tools.cmake
+++ b/tools/pybind11Tools.cmake
@@ -14,6 +14,7 @@ set(Python_ADDITIONAL_VERSIONS 3.7 3.6 3.5 3.4)
 find_package(PythonLibsNew ${PYBIND11_PYTHON_VERSION} REQUIRED)
 include(CheckCXXCompilerFlag)
+include(CMakeParseArguments)
 function(select_cxx_standard)
  if(NOT MSVC AND NOT PYBIND11_CPP_STANDARD)
@@ -33,29 +34,48 @@ function(select_cxx_standard)
  endif()
 endfunction()
+# Internal: find the appropriate LTO flag for this compiler
+macro(_pybind11_find_lto_flag output_var prefer_thin_lto)
+  if(${prefer_thin_lto})
+    # Check for ThinLTO support (Clang)
+    check_cxx_compiler_flag("-flto=thin" HAS_THIN_LTO_FLAG)
+    set(${output_var} $<${HAS_THIN_LTO_FLAG}:-flto=thin>)
+  endif()
+  if(NOT ${prefer_thin_lto} OR NOT HAS_THIN_LTO_FLAG)
+    if(NOT CMAKE_CXX_COMPILER_ID MATCHES "Intel")
+      # Check for Link Time Optimization support (GCC/Clang)
+      check_cxx_compiler_flag("-flto" HAS_LTO_FLAG)
+      set(${output_var} $<${HAS_LTO_FLAG}:-flto>)
+    else()
+      # Intel equivalent to LTO is called IPO
+      check_cxx_compiler_flag("-ipo" HAS_IPO_FLAG)
+      set(${output_var} $<${HAS_IPO_FLAG}:-ipo>)
+    endif()
+  endif()
+endmacro()
 # Build a Python extension module:
-# pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL] source1 [source2 ...])
+# pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
+#                     [NO_EXTRAS] [THIN_LTO] source1 [source2 ...])
 #
 function(pybind11_add_module target_name)
-  set(lib_type "MODULE")
+  set(options MODULE SHARED EXCLUDE_FROM_ALL NO_EXTRAS THIN_LTO)
-  set(do_lto True)
+  cmake_parse_arguments(ARG "${options}" "" "" ${ARGN})
-  set(exclude_from_all "")
-  set(sources "")
+  if(ARG_MODULE AND ARG_SHARED)
+    message(FATAL_ERROR "Can't be both MODULE and SHARED")
-  set(_args_to_try "${ARGN}")
+  elseif(ARG_SHARED)
-  foreach(_ex_arg IN LISTS _args_to_try)
+    set(lib_type SHARED)
-    if(${_ex_arg} STREQUAL "MODULE")
+  else()
-      set(lib_type "MODULE")
+    set(lib_type MODULE)
-    elseif(${_ex_arg} STREQUAL "SHARED")
+  endif()
-      set(lib_type "SHARED")
-    elseif(${_ex_arg} STREQUAL "EXCLUDE_FROM_ALL")
-      set(exclude_from_all "EXCLUDE_FROM_ALL")
-    else()
-      list(APPEND sources "${_ex_arg}")
-    endif()
-  endforeach()
-  add_library(${target_name} ${lib_type} ${exclude_from_all} ${sources})
+  if(ARG_EXCLUDE_FROM_ALL)
+    set(exclude_from_all EXCLUDE_FROM_ALL)
+  endif()
+  add_library(${target_name} ${lib_type} ${exclude_from_all} ${ARG_UNPARSED_ARGUMENTS})
  target_include_directories(${target_name}
    PRIVATE ${PYBIND11_INCLUDE_DIR}  # from project CMakeLists.txt
@@ -86,7 +106,7 @@ function(pybind11_add_module target_name)
    target_link_libraries(${target_name} PRIVATE "-undefined dynamic_lookup")
-    if(${lib_type} STREQUAL "SHARED")
+    if(ARG_SHARED)
      # Suppress CMake >= 3.0 warning for shared libraries
      set_target_properties(${target_name} PROPERTIES MACOSX_RPATH ON)
    endif()
@@ -96,23 +116,21 @@ function(pybind11_add_module target_name)
  if(NOT MSVC)
    # Make sure C++11/14 are enabled
    target_compile_options(${target_name} PUBLIC ${PYBIND11_CPP_STANDARD})
+  endif()
+  if(ARG_NO_EXTRAS)
+    return()
+  endif()
+  if(NOT MSVC)
    # Enable link time optimization and set the default symbol
    # visibility to hidden (very important to obtain small binaries)
    string(TOUPPER "${CMAKE_BUILD_TYPE}" U_CMAKE_BUILD_TYPE)
    if (NOT ${U_CMAKE_BUILD_TYPE} MATCHES DEBUG)
-      # Check for Link Time Optimization support (GCC/Clang)
+      # Link Time Optimization
-      check_cxx_compiler_flag("-flto" HAS_LTO_FLAG)
+      if(NOT CYGWIN)
-      if(HAS_LTO_FLAG AND NOT CYGWIN)
+        _pybind11_find_lto_flag(lto_flag ARG_THIN_LTO)
-        target_compile_options(${target_name} PRIVATE -flto)
+        target_compile_options(${target_name} PRIVATE ${lto_flag})
-      endif()
-      # Intel equivalent to LTO is called IPO
-      if(CMAKE_CXX_COMPILER_ID MATCHES "Intel")
-        check_cxx_compiler_flag("-ipo" HAS_IPO_FLAG)
-        if(HAS_IPO_FLAG)
-          target_compile_options(${target_name} PRIVATE -ipo)
-        endif()
      endif()
      # Default symbol visibility