fix: ipo should be off for debug or relwithdebinfo (#2590)

* fix: ipo should be off for debug or relwithdebinfo

Closes #2587

* docs: slightly more detailed about IPO

* Update pybind11Common.cmake

docs/compiling.rst

@@ -264,8 +264,15 @@ As stated above, LTO is enabled by default. Some newer compilers also support
 different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause
 different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause
 the function to prefer this flavor if available. The function falls back to
 the function to prefer this flavor if available. The function falls back to
 regular LTO if ``-flto=thin`` is not available. If
 regular LTO if ``-flto=thin`` is not available. If
-``CMAKE_INTERPROCEDURAL_OPTIMIZATION`` is set (either ON or OFF), then that
-will be respected instead of the built-in flag search.
+``CMAKE_INTERPROCEDURAL_OPTIMIZATION`` is set (either ``ON`` or ``OFF``), then
+that will be respected instead of the built-in flag search.
+
+.. note::
+
+   If you want to set the property form on targets or the
+   ``CMAKE_INTERPROCEDURAL_OPTIMIZATION_<CONFIG>`` versions of this, you should
+   still use ``set(CMAKE_INTERPROCEDURAL_OPTIMIZATION OFF)`` (otherwise a
+   no-op) to disable pybind11's ipo flags.
 
 
 The ``OPT_SIZE`` flag enables size-based optimization equivalent to the
 The ``OPT_SIZE`` flag enables size-based optimization equivalent to the
 standard ``/Os`` or ``-Os`` compiler flags and the ``MinSizeRel`` build type,
 standard ``/Os`` or ``-Os`` compiler flags and the ``MinSizeRel`` build type,

tools/pybind11Common.cmake

@@ -327,7 +327,9 @@ function(_pybind11_generate_lto target prefer_thin_lto)
 
 
   # Enable LTO flags if found, except for Debug builds
   # Enable LTO flags if found, except for Debug builds
   if(PYBIND11_LTO_CXX_FLAGS)
   if(PYBIND11_LTO_CXX_FLAGS)
-    set(not_debug "$<NOT:$<CONFIG:Debug>>")
+    # CONFIG takes multiple values in CMake 3.19+, until then we have to use OR
+    set(is_debug "$<OR:$<CONFIG:Debug>,$<CONFIG:RelWithDebInfo>>")
+    set(not_debug "$<NOT:${is_debug}>")
     set(cxx_lang "$<COMPILE_LANGUAGE:CXX>")
     set(cxx_lang "$<COMPILE_LANGUAGE:CXX>")
     if(MSVC AND CMAKE_VERSION VERSION_LESS 3.11)
     if(MSVC AND CMAKE_VERSION VERSION_LESS 3.11)
       set(genex "${not_debug}")
       set(genex "${not_debug}")