CMake庫搜尋函數居然不搜尋LD_LIBRARY_PATH

2022-07-30 12:00:12
摘要: 本文通過編譯後執行找不到庫檔案的問題引入,首先分析了find_package(JNI)的工作流程,而後針對cmake不搜尋LD_LIBRARY_PATH的問題,提出了一種通用的解決辦法。

本文分享自華為雲社群《CMake庫搜尋函數居然不搜尋LD_LIBRARY_PATH? 由編譯工具使用體驗而引發的思考》,作者: 蜉蝣與海 。

最近產品要使用JNI技術,CMake編譯C++程式碼時需要對外連線libjvm.so庫。程式碼編譯倒是正常,系統中也有libjvm.so, 然而使用時卻報瞭如下異常:

error while loading shared libraries: libjvm.so: cannot open shared object file: No such file or directory

這個報錯表示,作業系統並沒有找到libjvm.so, 我們的作業系統是從LD_LIBRARY_PATH中搜尋這些動態連結庫,很顯然目前libjvm.so並不在這個目錄下。

問題的解決倒是簡單,直接在LD_LIBRARY_PATH里加入libjvm.so的庫即可。但是這卻引發了我的思考:

為什麼構建時可以找到libjvm.so, 執行時卻找不到呢?

這個問題的回答,既可以有簡明扼要版解釋,又可以刨根問底深挖。

先來看簡明扼要版解釋:

程式碼的CMakeList中使用了下列語句,在編譯過程中尋找並連結libjvm.so,這個搜尋方式和作業系統的搜尋方式不同:

find_package(JNI)
get_filename_component(JVM_LIB_PATH ${JAVA_JVM_LIBRARY} DIRECTORY)
get_filename_component(JAVA_LIB_PATH ${JVM_LIB_PATH} DIRECTORY)
link_directories(${JVM_LIB_PATH} ${JAVA_LIB_PATH})
set_target_properties(${NAME} PROPERTIES LINK_FLAGS "-ljvm")

其中find_package(JNI)會搜尋libjvm.so可能存在的路徑,通過get_filename_component來獲得libjvm.so的資料夾,並把這個資料夾設為預設搜尋庫路徑。而後set_target_properties會進行連結工作。

這個答案只能告訴我們「是什麼」,但是作為一隻程式猿,還要了解「為什麼」,這裡引申幾個問題討論:

  1. find_package(JNI)的工作過程是怎樣的?為什麼LD_LIBRARY_PATH裡沒找到的依賴庫,cmake可以找到
  2. cmake的庫搜尋函數find_library會搜尋LD_LIBRARY_PATH嗎,如果不會,可以通過設定來搜尋LD_LIBRARY_PATH嗎?

問題一:find_package(JNI)的工作過程是怎樣的

為了方便開發者參照外部包,cmake官方預定義了許多尋找依賴包的Module, 他們儲存在cmake的/share/-cmake-<version>/Modules目錄下。每個以Find<LibraryName>.cmake命名的檔案都可以幫我們找到一個包[1]。在本地計算機執行以下指令,即可找到find_package(JNI)使用的指令碼檔案。

find / -name FindJNI.cmake

開啟自己的cmake對應的FindJNI檔案,可以看到密密麻麻的註釋和指令碼,通過閱讀這些指令碼,我們得以得知FindJNI是如何工作的。

分析問題前,先看問題帶來的結果,檔案最上方註釋有如下說明:

This module sets the following result variables:
``JNI_INCLUDE_DIRS``
 the include dirs to use
``JNI_LIBRARIES``
  the libraries to use (JAWT and JVM)
``JNI_FOUND``
  TRUE if JNI headers and libraries were found.
Cache Variables
^^^^^^^^^^^^^^^
The following cache variables are also available to set or use:
``JAVA_AWT_LIBRARY``
  the path to the Java AWT Native Interface (JAWT) library
``JAVA_JVM_LIBRARY``
  the path to the Java Virtual Machine (JVM) library
``JAVA_INCLUDE_PATH``
 the include path to jni.h
``JAVA_INCLUDE_PATH2``
 the include path to jni_md.h and jniport.h
``JAVA_AWT_INCLUDE_PATH``
 the include path to jawt.h

這段程式碼錶明,執行find_package(JNI)之後,會有一系列變數被設定,其中包括表示JNI是否被找到的變數JNI_FOUND,以及表示libjvm.so的變數JAVA_JVM_LIBRARY。這些變數在設定之後,通過FindPackageHandleStandardArgs匯出,返回撥用處,FindPackageHandleStandardArgs是cmake專門用來匯出變數的宏[2]:

include(${CMAKE_CURRENT_LIST_DIR}/FindPackageHandleStandardArgs.cmake)
FIND_PACKAGE_HANDLE_STANDARD_ARGS(JNI  DEFAULT_MSG  JAVA_AWT_LIBRARY
                                                    JAVA_JVM_LIBRARY
                                                    JAVA_INCLUDE_PATH
                                                    JAVA_INCLUDE_PATH2
                                                    JAVA_AWT_INCLUDE_PATH)

在檔案中定位JAVA_JVM_LIBRARY, 可以追蹤到下述程式碼片段:

foreach(search ${_JNI_SEARCHES})
 find_library(JAVA_JVM_LIBRARY ${_JNI_${search}_JVM})
 find_library(JAVA_AWT_LIBRARY ${_JNI_${search}_JAWT})
 if(JAVA_JVM_LIBRARY)
 break()
 endif()
endforeach()

由此可知,JAVA_JVM_LIBRARY這個變數,是通過逐個搜尋${_JNI_${search}_JVM}裡的資料夾進而確定JAVA_JVM_LIBRARY的。而${_JNI_${search}_JVM}相關的定義語句如圖:

set(_JNI_FRAMEWORK_JVM NAMES JavaVM)
set(_JNI_NORMAL_JVM
  NAMES jvm
  PATHS ${JAVA_JVM_LIBRARY_DIRECTORIES}
  )

其中JAVA_JVM_LIBRARY_DIRECTORIES中涉及了大量可能的libjvm.so存在的路徑。

set(JAVA_JVM_LIBRARY_DIRECTORIES)
foreach(dir ${JAVA_AWT_LIBRARY_DIRECTORIES})
 list(APPEND JAVA_JVM_LIBRARY_DIRECTORIES
 "${dir}"
 "${dir}/client"
 "${dir}/server"
 # IBM SDK, Java Technology Edition, specific paths
 "${dir}/j9vm"
 "${dir}/default"
    )
endforeach()
set(JAVA_AWT_LIBRARY_DIRECTORIES)
if(_JAVA_HOME)
  JAVA_APPEND_LIBRARY_DIRECTORIES(JAVA_AWT_LIBRARY_DIRECTORIES
 ${_JAVA_HOME}/jre/lib/{libarch}
 ${_JAVA_HOME}/jre/lib
 ${_JAVA_HOME}/lib/{libarch}
 ${_JAVA_HOME}/lib
 ${_JAVA_HOME}
    )
endif()
JAVA_APPEND_LIBRARY_DIRECTORIES(JAVA_AWT_LIBRARY_DIRECTORIES
 ${_JNI_JAVA_AWT_LIBRARY_TRIES}
  )
foreach(_java_dir IN LISTS _JNI_JAVA_DIRECTORIES_BASE)
 list(APPEND _JNI_JAVA_AWT_LIBRARY_TRIES
 ${_java_dir}/jre/lib/{libarch}
 ${_java_dir}/jre/lib
 ${_java_dir}/lib/{libarch}
 ${_java_dir}/lib
 ${_java_dir}
  )
 list(APPEND _JNI_JAVA_INCLUDE_TRIES
 ${_java_dir}/include
  )
endforeach()

如上圖所示,變數依賴順序如下:

JAVA_JVM_LIBRARY_DIRECTORIES => JAVA_AWT_LIBRARY_DIRECTORIES => _JNI_JAVA_AWT_LIBRARY_TRIES & _JAVA_HOME => _JNI_JAVA_DIRECTORIES_BASE

最終發現JAVA_JVM_LIBRARY_DIRECTORIES變數的值,是由JAVA_HOME變數的值和_JNI_JAVA_DIRECTORIES_BASE變數的值共同決定的。而JNI_JAVA_DIRECTORY_BASE預置了大量預定義路徑:

set(_JNI_JAVA_DIRECTORIES_BASE
  /usr/lib/jvm/java
  /usr/lib/java
  /usr/lib/jvm
  /usr/local/lib/java
  /usr/local/share/java
  /usr/lib/j2sdk1.4-sun
  /usr/lib/j2sdk1.5-sun
  /opt/sun-jdk-1.5.0.04
  /usr/lib/jvm/java-6-sun
  /usr/lib/jvm/java-1.5.0-sun
  /usr/lib/jvm/java-6-sun-1.6.0.00       # can this one be removed according to #8821 ? Alex
  /usr/lib/jvm/java-6-openjdk
  /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0        # fedora
 # Debian specific paths for default JVM
  /usr/lib/jvm/default-java
 # Arch Linux specific paths for default JVM
  /usr/lib/jvm/default
 # Ubuntu specific paths for default JVM
  /usr/lib/jvm/java-11-openjdk-{libarch}    # Ubuntu 18.04 LTS
  /usr/lib/jvm/java-8-openjdk-{libarch}    # Ubuntu 15.10
  /usr/lib/jvm/java-7-openjdk-{libarch}    # Ubuntu 15.10
  /usr/lib/jvm/java-6-openjdk-{libarch}    # Ubuntu 15.10
 # OpenBSD specific paths for default JVM
  /usr/local/jdk-1.7.0
  /usr/local/jre-1.7.0
  /usr/local/jdk-1.6.0
  /usr/local/jre-1.6.0
 # SuSE specific paths for default JVM
  /usr/lib64/jvm/java
  /usr/lib64/jvm/jre
  )

通過以上分析可以看出,JAVA_JVM_LIBRARY的搜尋,依賴JAVA_HOME和大量預定義路徑。

問題二:cmake庫搜尋函數find_library會搜尋LD_LIBRARY_PATH嗎

通過閱讀Does CMake's find_library search LD_LIBRARY_PATH可以知道,find_library預設不搜尋LD_LIBRARY_PATH, 並且網上也找不到讓cmake搜尋LD_LIBRARY_PATH的文章。

那cmake能搜尋LD_LIBRARY_PATH嗎?

答案是可以的,通過cmake獲取LD_LIBRARY_PATH環境變數,並轉為cmake可理解的list格式,而後注入find_library即可,程式碼如下:

string(REPLACE ":" ";" RUNTIME_PATH "$ENV{LD_LIBRARY_PATH}")
find_library(JVM_API NAMES jvm HINTS ${RUNTIME_PATH})
if (JVM_API STREQUAL "JVM_API-NOTFOUND")
 message(WARNING "found libjvm.so only in ${JAVA_JVM_LIBRARY} but not in LD_LIBRARY_PATH. environment variable LD_LIBRARY_PATH must include its' directory.")
endif()

如果希望找不到這個庫時編譯失敗,可以將WARNING改為fatal_error, 程式碼如下:

string(REPLACE ":" ";" RUNTIME_PATH "$ENV{LD_LIBRARY_PATH}")
find_library(JVM_API NAMES jvm HINTS ${RUNTIME_PATH})
if (JVM_API STREQUAL "JVM_API-NOTFOUND")
 message(FATAL_ERROR "found libjvm.so only in ${JAVA_JVM_LIBRARY} but not in LD_LIBRARY_PATH. environment variable LD_LIBRARY_PATH must include its' directory.")
endif()

小結

本文通過編譯後執行找不到庫檔案的問題引入,首先分析了find_package(JNI)的工作流程,而後針對cmake不搜尋LD_LIBRARY_PATH的問題,提出了一種通用的解決辦法。

參考文獻:

[1] Cmake之深入理解find_package()的用法:https://zhuanlan.zhihu.com/p/97369704?utm_source=wechat_session

[2] Cmake中find_package命令的搜尋模式之模組模式(Module mode):https://www.jianshu.com/p/f983a90bcf91

[3]Does CMake's find_library search LD_LIBRARY_PATH?:https://stackoverflow.com/questions/41566316/does-cmakes-find-library-search-ld-library-path

 

點選關注,第一時間瞭解華為雲新鮮技術~