c – CMake：修改共享库时的冗余链接

2023年12月25日 234次阅读

我在一个由几十个共享库组成的项目上工作,每个库都有许多相关的单元测试.许多库也依赖于其他库,因为某些特定功能的库将使用来自一个更常见的库的代码.最后当然还有依赖于libs的生产可执行文件.

毫无疑问,某些核心公共库的API(头文件)的更改应该触发几乎整个系统的主要重新编译.但通常只有实现中的变化,并且编译的唯一文件是修改后的.cxx,理论上只需要链接修改后的lib – 由于动态链接,不需要重新链接任何其他内容.但是CMake继续前进并且无论如何都要这样做：在重新链接lib后,它重新链接与该lib相关的所有单元测试.然后,它重新链接该lib的依赖关系树中的所有lib以及所有单元测试.最后,它重新链接生产可执行文件.由于项目的规模,这需要很多宝贵的时间.

我已经使用基于this最小示例的简单项目重现了这种行为(为简洁而删除了注释,并且lib已更改为共享).我的系统是英特尔PC上的Ubuntu 16,我的CMake版本是3.5.1.

从空目录开始并创建这些文件：

的CMakeLists.txt

cmake_minimum_required (VERSION 2.8.11)
project (HELLO)
add_subdirectory (Hello)
add_subdirectory (Demo)

演示/的CMakeLists.txt

add_executable (helloDemo demo.cxx)
target_link_libraries (helloDemo LINK_PUBLIC Hello)

演示/ demo.cxx

#include "hello.h"
int main() { hello(); }

你好/的CMakeLists.txt

add_library (Hello SHARED hello.cxx)
target_include_directories (Hello PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

你好/ hello.h

void hello();

你好/ hello.cxx

#include <stdio.h>
void hello() { printf("hello!\n"); }

现在运行命令：

mkdir build
cd build
cmake ../
make

您现在可以执行Demo / helloDemo并查看hello！.

现在,触摸Hello / hello.cxx并再次制作.您将看到helloDemo可执行文件已重新链接(“链接CXX可执行文件helloDemo”).即使修改了hello.cxx以打印不同的字符串,重新链接的可执行文件仍然是二进制相同的,所以实际上重新连接是不必要的.

有没有办法防止这些冗余的构建操作？

最佳答案 摘要

>下面没有适当的解决方案
>修补CMake会产生可行的解决方案;但这些变化很可能会引入错误.
> Bazel没有相同的问题(经过测试),并且对于您的特定用例可能会明显更快.

旅程

使用Ninja生成器,生成的build.ninja文件(运行cmake -G Ninja ..)具有以下部分.本节清楚地显示了错误：CMake在Hello / libHello.dylib上添加了一个隐式依赖,但是Order-Only-Dependency就足够了.

接下来是完整部分,但请阅读下面的说明,请滚动到右侧：

#############################################
# Link the executable Demo/helloDemo

build Demo/helloDemo: CXX_EXECUTABLE_LINKER__helloDemo Demo/CMakeFiles/helloDemo.dir/demo.cxx.o | Hello/libHello.dylib || Hello/libHello.dylib
  LINK_LIBRARIES = -Wl,-rpath,/Users/myuser/devel/misc/stackoverflow/q50084885/ninja/Hello     Hello/libHello.dylib
  OBJECT_DIR = Demo/CMakeFiles/helloDemo.dir
  POST_BUILD = :
  PRE_LINK = :
  TARGET_FILE = Demo/helloDemo
  TARGET_PDB = helloDemo.dbg

我在macOS上,因为Linux将* .dylib全部读为* .so.

注意第一个非注释行：
build Demo / helloDemo：…. Ninja-grammar如下：
build< output>：< rule> <输入> | || <为了仅-先决条件>

<规则>是CXX_EXECUTABLE_LINKER_helloDemo,而Hello / libHelly.dylib既是隐式输入,也是仅限订单的先决条件.

手动编辑生成的build.ninja并删除隐式输入,但不是仅订单的先决条件,修复了问题！

修补CMake

使用以下修补程序修补v3.11.1(对于此特定示例).但是,它没有深入了解CMake源代码并且单元测试失败. (其中一个失败的测试是BuildDepends,只有补丁失败才会失败！)

diff --git a/Source/cmNinjaTargetGenerator.cxx b/Source/cmNinjaTargetGenerator.cxx
index f4faf47a2..bdbf6b948 100644
--- a/Source/cmNinjaTargetGenerator.cxx
+++ b/Source/cmNinjaTargetGenerator.cxx
@@ -239,7 +239,8 @@ cmNinjaDeps cmNinjaTargetGenerator::ComputeLinkDeps() const
 {
   // Static libraries never depend on other targets for linking.
   if (this->GeneratorTarget->GetType() == cmStateEnums::STATIC_LIBRARY ||
-      this->GeneratorTarget->GetType() == cmStateEnums::OBJECT_LIBRARY) {
+      this->GeneratorTarget->GetType() == cmStateEnums::OBJECT_LIBRARY ||
+      this->GeneratorTarget->GetType() == cmStateEnums::EXECUTABLE ) {
     return cmNinjaDeps();
   }

这个补丁导致生成的代码,完全相同的改变,我说可以手动完成.

所以,这似乎有效.

进一步参考的踪迹

试图摆脱构建顺序依赖

这里的问题类似于我们的问题：当编译目标X时,对象O和库依赖关系L,在编译对象O之前不需要等待L来构建.

> https://cmake.org/Bug/view.php?id=14726#c35023
> https://cmake.org/Bug/view.php?id=13799

target_link_libraries adds transitive link dependencies and build order dependencies between the target and its dependencies. In many (if not most) cases the build order dependency is not necessary and results in an overlay constrained dependency graph, which limits parallel execution of the build.

[Brad King]: FYI, the reason we have these dependencies by default is because the build rules for a library may have custom commands to generate headers or sources that are then used by a target that links to it. This applies even to non-linking targets like static libraries. Furthermore there is no separation of target-level ordering dependencies between compilation and link steps within a single target. Therefore any target that links (shared libs and exes) must have an order dependency on its link dependencies.

Of course if a project author wants to take responsibility I see no reason not to have an option to skip such dependencies, at least for static libraries. One approach is to add a target property to override the target-level ordering dependencies. This way one could make a static library depend either on nothing or on a subset of its implementation dependencies:

> https://cmake.org/pipermail/cmake-developers/2014-June/010708.html

My point is that there is no reason to wait building b.cc.o and prog.cc.o;
they can be built at the same time as a.cc.o .

Hence I wonder why libA.so is added as a order-only dependency to b.cc.o
when CMake processes this?

进一步的参考

> https://gitlab.kitware.com/cmake/cmake/issues/17666
> https://cmake.org/pipermail/cmake-developers/2016-March/028012.html
也许是一个相关的补丁,但从2016年3月开始.

点赞