14.4 使用ThreadSaniiser向CDash报告数据争用

NOTE:此示例代码可以在 https://github.com/dev-cafe/cmake-cookbook/tree/v1.0/chapter-14/recipe-03 中找到,其中包含一个C++示例。该示例在CMake 3.5版(或更高版本)中是有效的,并且已经在GNU/Linux、macOS和Windows上进行过测试。

在这个示例中,我们将重用前一个示例中的方法,但是使用ThreadSanitizer或TSan,结合CTest和CDash,来检查数据竞争,并将它们报告给CDash。ThreadSanitizer的文档可以在网上找到,https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual

准备工作

这个示例中,我们将使用以下示例代码(example.cpp):

  1. #include <chrono>
  2. #include <iostream>
  3. #include <thread>
  4. static const int num_threads = 16;
  5. void increase(int i, int &s) {
  6. std::this_thread::sleep_for(std::chrono::seconds(1));
  7. std::cout << "thread " << i << " increases " << s++ << std::endl;
  8. }
  9. int main() {
  10. std::thread t[num_threads];
  11. int s = 0;
  12. // start threads
  13. for (auto i = 0; i < num_threads; i++) {
  14. t[i] = std::thread(increase, i, std::ref(s));
  15. }
  16. // join threads with main thread
  17. for (auto i = 0; i < num_threads; i++) {
  18. t[i].join();
  19. }
  20. std::cout << "final s: " << s << std::endl;
  21. return 0;
  22. }

这个示例代码中,我们启动16个线程,每个线程都调用increase函数。increase函数休眠1s,然后打印并递增一个整数s。我们预计此示例代码将显示数据竞争,因为所有线程读取和修改相同的地址,而不需要任何显式同步或协调。换句话说,我们期望在代码末尾打印的最终s,每次的结果都不同。代码有bug,我们将尝试在ThreadSanitizer的帮助下识别数据竞争。如果不运行ThreadSanitizer,我们可能不会看到代码有任何问题:

  1. $ ./example
  2. thread thread 0 increases 01 increases 1
  3. thread 9 increases 2
  4. thread 4 increases 3
  5. thread 10 increases 4
  6. thread 2 increases 5
  7. thread 3 increases 6
  8. thread 13 increases 7
  9. thread thread 7 increases 8
  10. thread 14 increases 9
  11. thread 8 increases 10
  12. thread 12 increases 11
  13. thread 15 increases 12
  14. thread 11 increases 13
  15. 5 increases 14
  16. thread 6 increases 15
  17. final s: 16

具体实施

  1. 文件CMakeLists.txt首先定义一个受支持的最低版本、项目名称、受支持的语言。在本例中,定义了C++11标准项目:

    1. cmake_minimum_required(VERSION 3.5 FATAL_ERROR)
    2. project(recipe-04 LANGUAGES CXX)
    3. set(CMAKE_CXX_STANDARD 11)
    4. set(CMAKE_CXX_EXTENSIONS OFF)
    5. set(CMAKE_CXX_STANDARD_REQUIRED ON)
  2. 接下来,找到线程库,定义可执行文件,并将其链接到线程库:

    1. find_package(Threads REQUIRED)
    2. add_executable(example example.cpp)
    3. target_link_libraries(example
    4. PUBLIC
    5. Threads::Threads
    6. )
  3. 然后,提供编译选项和代码,并链接到ThreadSanitizer:

    1. option(ENABLE_TSAN "Enable ThreadSanitizer" OFF)
    2. if(ENABLE_TSAN)
    3. if(CMAKE_CXX_COMPILER_ID MATCHES GNU)
    4. message(STATUS "ThreadSanitizer enabled")
    5. target_compile_options(example
    6. PUBLIC
    7. -g -O1 -fsanitize=thread -fno-omit-frame-pointer -fPIC
    8. )
    9. target_link_libraries(example
    10. PUBLIC
    11. tsan
    12. )
    13. else()
    14. message(WARNING "ThreadSanitizer not supported for this compiler")
    15. endif()
    16. endif()
  4. 最后,编译测试用例:

    1. enable_testing()
    2. # allow to report to a cdash dashboard
    3. include(CTest)
    4. add_test(
    5. NAME
    6. example
    7. COMMAND
    8. $<TARGET_FILE:example>
    9. )
  5. CTestConfig.cmake没有变化:

    1. set(CTEST_DROP_METHOD "http")
    2. set(CTEST_DROP_SITE "my.cdash.org")
    3. set(CTEST_DROP_LOCATION "/submit.php?project=cmake-cookbook")
    4. set(CTEST_DROP_SITE_CDASH TRUE)
  6. dashboard.cmake需要为TSan进行简单修改:

    1. set(CTEST_PROJECT_NAME "example")
    2. cmake_host_system_information(RESULT _site QUERY HOSTNAME)
    3. set(CTEST_SITE ${_site})
    4. set(CTEST_BUILD_NAME "${CMAKE_SYSTEM_NAME}-${CMAKE_HOST_SYSTEM_PROCESSOR}")
    5. set(CTEST_SOURCE_DIRECTORY "${CTEST_SCRIPT_DIRECTORY}")
    6. set(CTEST_BINARY_DIRECTORY "${CTEST_SCRIPT_DIRECTORY}/build")
    7. include(ProcessorCount)
    8. ProcessorCount(N)
    9. if(NOT N EQUAL 0)
    10. set(CTEST_BUILD_FLAGS -j${N})
    11. set(ctest_test_args ${ctest_test_args} PARALLEL_LEVEL ${N})
    12. endif()
    13. ctest_start(Experimental)
    14. ctest_configure(
    15. OPTIONS
    16. -DENABLE_TSAN:BOOL=ON
    17. )
    18. ctest_build()
    19. ctest_test()
    20. set(CTEST_MEMORYCHECK_TYPE "ThreadSanitizer")
    21. ctest_memcheck()
    22. ctest_submit()
  7. 让我们以这个例子为例。通过CTEST_CMAKE_GENERATOR选项来设置生成器:

    1. $ ctest -S dashboard.cmake -D CTEST_CMAKE_GENERATOR="Unix Makefiles"
    2. Each . represents 1024 bytes of output
    3. . Size of output: 0K
    4. Each symbol represents 1024 bytes of output.
    5. '!' represents an error and '*' a warning.
    6. . Size of output: 0K
  8. 在面板上,我们将看到以下内容:

    14.4 使用ThreadSaniiser向CDash报告数据争用 - 图1

  9. 我们可以看到更详细的动态分析:

    14.4 使用ThreadSaniiser向CDash报告数据争用 - 图2

工作原理

该示例CMakeLists.txt的核心部分:

  1. option(ENABLE_TSAN "Enable ThreadSanitizer" OFF)
  2. if(ENABLE_TSAN)
  3. if(CMAKE_CXX_COMPILER_ID MATCHES GNU)
  4. message(STATUS "ThreadSanitizer enabled")
  5. target_compile_options(example
  6. PUBLIC
  7. -g -O1 -fsanitize=thread -fno-omit-frame-pointer -fPIC
  8. )
  9. target_link_libraries(example
  10. PUBLIC
  11. tsan
  12. )
  13. else()
  14. message(WARNING "ThreadSanitizer not supported for this compiler")
  15. endif()
  16. endif()

dashboard.cmake也需要更新:

  1. # ...
  2. ctest_start(Experimental)
  3. ctest_configure(
  4. OPTIONS
  5. -DENABLE_TSAN:BOOL=ON
  6. )
  7. ctest_build()
  8. ctest_test()
  9. set(CTEST_MEMORYCHECK_TYPE "ThreadSanitizer")
  10. ctest_memcheck()
  11. ctest_submit()

和上一个示例一样,我们也可以在本地查看ThreadSanitizer的输出:

  1. $ mkdir -p build
  2. $ cd build
  3. $ cmake -DENABLE_TSAN=ON ..
  4. $ cmake --build .
  5. $ cmake --build . --target test
  6. Start 1: example
  7. 1/1 Test #1: example ..........................***Failed 1.07 sec
  8. 0% tests passed, 1 tests failed out of 1
  9. $ ./build/example
  10. thread 0 increases 0
  11. ==================
  12. WARNING: ThreadSanitizer: data race (pid=24563)
  13. ... lots of output ...
  14. SUMMARY: ThreadSanitizer: data race /home/user/cmake-recipes/chapter-14/recipe-04/cxx-example/example

更多信息

对使用OpenMP的应用TSan是很常见的,但是请注意,在某些情况下,OpenMP会在TSan下生成误检的结果。对于Clang编译器,一个解决方案是用-DLIBOMP_TSAN_SUPPORT=TRUE重新编译编译器本身及其libomp。通常,以合理的方式使用TSan可能需要重新编译整个工具堆栈,以避免误报。在使用pybind11的C++项目的情况,我们可能需要重新编译Python,并启用TSan来获得有意义的东西。或者,Python绑定可以通过使用TSan抑制而被排除在外,如 https://github.com/google/sanitizers/wiki/threadsanitizersuppression 。例如:如果一个动态库同时被一个经过TSan的二进制文件和一个Python插件调用,那么这种情况可能是不可能使用TSan。

下面的博客文章讨论了如何添加对动态分析工具的支持:https://blog.kitware.com/ctest-cdash-add-support-for-new-dynamic-analysis-tools/