摘要: 本文通過編譯後執行找不到庫檔案的問題引入,首先分析了find_package(JNI)的工作流程,而後針對cmake不搜尋LD_LIBRARY_PATH的問題,提出了一種通用的解決辦法。
本文分享自華為雲社群《CMake庫搜尋函數居然不搜尋LD_LIBRARY_PATH? 由編譯工具使用體驗而引發的思考》,作者: 蜉蝣與海 。
最近產品要使用JNI技術,CMake編譯C++程式碼時需要對外連線libjvm.so庫。程式碼編譯倒是正常,系統中也有libjvm.so, 然而使用時卻報瞭如下異常:
error while loading shared libraries: libjvm.so: cannot open shared object file: No such file or directory
這個報錯表示,作業系統並沒有找到libjvm.so, 我們的作業系統是從LD_LIBRARY_PATH中搜尋這些動態連結庫,很顯然目前libjvm.so並不在這個目錄下。
問題的解決倒是簡單,直接在LD_LIBRARY_PATH里加入libjvm.so的庫即可。但是這卻引發了我的思考:
這個問題的回答,既可以有簡明扼要版解釋,又可以刨根問底深挖。
先來看簡明扼要版解釋:
程式碼的CMakeList中使用了下列語句,在編譯過程中尋找並連結libjvm.so,這個搜尋方式和作業系統的搜尋方式不同:
find_package(JNI) get_filename_component(JVM_LIB_PATH ${JAVA_JVM_LIBRARY} DIRECTORY) get_filename_component(JAVA_LIB_PATH ${JVM_LIB_PATH} DIRECTORY) link_directories(${JVM_LIB_PATH} ${JAVA_LIB_PATH}) set_target_properties(${NAME} PROPERTIES LINK_FLAGS "-ljvm")
其中find_package(JNI)會搜尋libjvm.so可能存在的路徑,通過get_filename_component來獲得libjvm.so的資料夾,並把這個資料夾設為預設搜尋庫路徑。而後set_target_properties會進行連結工作。
這個答案只能告訴我們「是什麼」,但是作為一隻程式猿,還要了解「為什麼」,這裡引申幾個問題討論:
為了方便開發者參照外部包,cmake官方預定義了許多尋找依賴包的Module, 他們儲存在cmake的/share/-cmake-<version>/Modules目錄下。每個以Find<LibraryName>.cmake命名的檔案都可以幫我們找到一個包[1]。在本地計算機執行以下指令,即可找到find_package(JNI)使用的指令碼檔案。
find / -name FindJNI.cmake
開啟自己的cmake對應的FindJNI檔案,可以看到密密麻麻的註釋和指令碼,通過閱讀這些指令碼,我們得以得知FindJNI是如何工作的。
分析問題前,先看問題帶來的結果,檔案最上方註釋有如下說明:
This module sets the following result variables: ``JNI_INCLUDE_DIRS`` the include dirs to use ``JNI_LIBRARIES`` the libraries to use (JAWT and JVM) ``JNI_FOUND`` TRUE if JNI headers and libraries were found. Cache Variables ^^^^^^^^^^^^^^^ The following cache variables are also available to set or use: ``JAVA_AWT_LIBRARY`` the path to the Java AWT Native Interface (JAWT) library ``JAVA_JVM_LIBRARY`` the path to the Java Virtual Machine (JVM) library ``JAVA_INCLUDE_PATH`` the include path to jni.h ``JAVA_INCLUDE_PATH2`` the include path to jni_md.h and jniport.h ``JAVA_AWT_INCLUDE_PATH`` the include path to jawt.h
這段程式碼錶明,執行find_package(JNI)之後,會有一系列變數被設定,其中包括表示JNI是否被找到的變數JNI_FOUND,以及表示libjvm.so的變數JAVA_JVM_LIBRARY。這些變數在設定之後,通過FindPackageHandleStandardArgs匯出,返回撥用處,FindPackageHandleStandardArgs是cmake專門用來匯出變數的宏[2]:
include(${CMAKE_CURRENT_LIST_DIR}/FindPackageHandleStandardArgs.cmake)
FIND_PACKAGE_HANDLE_STANDARD_ARGS(JNI DEFAULT_MSG JAVA_AWT_LIBRARY
JAVA_JVM_LIBRARY
JAVA_INCLUDE_PATH
JAVA_INCLUDE_PATH2
JAVA_AWT_INCLUDE_PATH)
在檔案中定位JAVA_JVM_LIBRARY, 可以追蹤到下述程式碼片段:
foreach(search ${_JNI_SEARCHES}) find_library(JAVA_JVM_LIBRARY ${_JNI_${search}_JVM}) find_library(JAVA_AWT_LIBRARY ${_JNI_${search}_JAWT}) if(JAVA_JVM_LIBRARY) break() endif() endforeach()
由此可知,JAVA_JVM_LIBRARY這個變數,是通過逐個搜尋${_JNI_${search}_JVM}裡的資料夾進而確定JAVA_JVM_LIBRARY的。而${_JNI_${search}_JVM}相關的定義語句如圖:
set(_JNI_FRAMEWORK_JVM NAMES JavaVM) set(_JNI_NORMAL_JVM NAMES jvm PATHS ${JAVA_JVM_LIBRARY_DIRECTORIES} )
其中JAVA_JVM_LIBRARY_DIRECTORIES中涉及了大量可能的libjvm.so存在的路徑。
set(JAVA_JVM_LIBRARY_DIRECTORIES) foreach(dir ${JAVA_AWT_LIBRARY_DIRECTORIES}) list(APPEND JAVA_JVM_LIBRARY_DIRECTORIES "${dir}" "${dir}/client" "${dir}/server" # IBM SDK, Java Technology Edition, specific paths "${dir}/j9vm" "${dir}/default" ) endforeach() set(JAVA_AWT_LIBRARY_DIRECTORIES) if(_JAVA_HOME) JAVA_APPEND_LIBRARY_DIRECTORIES(JAVA_AWT_LIBRARY_DIRECTORIES ${_JAVA_HOME}/jre/lib/{libarch} ${_JAVA_HOME}/jre/lib ${_JAVA_HOME}/lib/{libarch} ${_JAVA_HOME}/lib ${_JAVA_HOME} ) endif() JAVA_APPEND_LIBRARY_DIRECTORIES(JAVA_AWT_LIBRARY_DIRECTORIES ${_JNI_JAVA_AWT_LIBRARY_TRIES} ) foreach(_java_dir IN LISTS _JNI_JAVA_DIRECTORIES_BASE) list(APPEND _JNI_JAVA_AWT_LIBRARY_TRIES ${_java_dir}/jre/lib/{libarch} ${_java_dir}/jre/lib ${_java_dir}/lib/{libarch} ${_java_dir}/lib ${_java_dir} ) list(APPEND _JNI_JAVA_INCLUDE_TRIES ${_java_dir}/include ) endforeach()
如上圖所示,變數依賴順序如下:
JAVA_JVM_LIBRARY_DIRECTORIES => JAVA_AWT_LIBRARY_DIRECTORIES => _JNI_JAVA_AWT_LIBRARY_TRIES & _JAVA_HOME => _JNI_JAVA_DIRECTORIES_BASE
最終發現JAVA_JVM_LIBRARY_DIRECTORIES變數的值,是由JAVA_HOME變數的值和_JNI_JAVA_DIRECTORIES_BASE變數的值共同決定的。而JNI_JAVA_DIRECTORY_BASE預置了大量預定義路徑:
set(_JNI_JAVA_DIRECTORIES_BASE /usr/lib/jvm/java /usr/lib/java /usr/lib/jvm /usr/local/lib/java /usr/local/share/java /usr/lib/j2sdk1.4-sun /usr/lib/j2sdk1.5-sun /opt/sun-jdk-1.5.0.04 /usr/lib/jvm/java-6-sun /usr/lib/jvm/java-1.5.0-sun /usr/lib/jvm/java-6-sun-1.6.0.00 # can this one be removed according to #8821 ? Alex /usr/lib/jvm/java-6-openjdk /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0 # fedora # Debian specific paths for default JVM /usr/lib/jvm/default-java # Arch Linux specific paths for default JVM /usr/lib/jvm/default # Ubuntu specific paths for default JVM /usr/lib/jvm/java-11-openjdk-{libarch} # Ubuntu 18.04 LTS /usr/lib/jvm/java-8-openjdk-{libarch} # Ubuntu 15.10 /usr/lib/jvm/java-7-openjdk-{libarch} # Ubuntu 15.10 /usr/lib/jvm/java-6-openjdk-{libarch} # Ubuntu 15.10 # OpenBSD specific paths for default JVM /usr/local/jdk-1.7.0 /usr/local/jre-1.7.0 /usr/local/jdk-1.6.0 /usr/local/jre-1.6.0 # SuSE specific paths for default JVM /usr/lib64/jvm/java /usr/lib64/jvm/jre )
通過以上分析可以看出,JAVA_JVM_LIBRARY的搜尋,依賴JAVA_HOME和大量預定義路徑。
通過閱讀Does CMake's find_library search LD_LIBRARY_PATH可以知道,find_library預設不搜尋LD_LIBRARY_PATH, 並且網上也找不到讓cmake搜尋LD_LIBRARY_PATH的文章。
答案是可以的,通過cmake獲取LD_LIBRARY_PATH環境變數,並轉為cmake可理解的list格式,而後注入find_library即可,程式碼如下:
string(REPLACE ":" ";" RUNTIME_PATH "$ENV{LD_LIBRARY_PATH}") find_library(JVM_API NAMES jvm HINTS ${RUNTIME_PATH}) if (JVM_API STREQUAL "JVM_API-NOTFOUND") message(WARNING "found libjvm.so only in ${JAVA_JVM_LIBRARY} but not in LD_LIBRARY_PATH. environment variable LD_LIBRARY_PATH must include its' directory.") endif()
如果希望找不到這個庫時編譯失敗,可以將WARNING改為fatal_error, 程式碼如下:
string(REPLACE ":" ";" RUNTIME_PATH "$ENV{LD_LIBRARY_PATH}") find_library(JVM_API NAMES jvm HINTS ${RUNTIME_PATH}) if (JVM_API STREQUAL "JVM_API-NOTFOUND") message(FATAL_ERROR "found libjvm.so only in ${JAVA_JVM_LIBRARY} but not in LD_LIBRARY_PATH. environment variable LD_LIBRARY_PATH must include its' directory.") endif()
本文通過編譯後執行找不到庫檔案的問題引入,首先分析了find_package(JNI)的工作流程,而後針對cmake不搜尋LD_LIBRARY_PATH的問題,提出了一種通用的解決辦法。
[1] Cmake之深入理解find_package()的用法:https://zhuanlan.zhihu.com/p/97369704?utm_source=wechat_session
[2] Cmake中find_package命令的搜尋模式之模組模式(Module mode):https://www.jianshu.com/p/f983a90bcf91
[3]Does CMake's find_library search LD_LIBRARY_PATH?:https://stackoverflow.com/questions/41566316/does-cmakes-find-library-search-ld-library-path