INTEL MKL: undefined symbol

Usability
1

I installed the latest Version of INTEL MKL and compiled the latest version of OGS with OGS_USE_MKL. On execution I obtain the error

/home/dominik/OGS/release/bin/ogs: symbol lookup error: /opt/intel/oneapi/mkl/2022.0.2/lib/intel64/libmkl_intel_lp64.so: undefined symbol: mkl_blas_dgemm

Are there other users with the same problem?

2

Dear @dominik-kern

I am learning how to compile OGS under Ubuntu 20.04 LTS (I use WSL2), and I indeed need the MKL library to enable PardisoLU liner solver. Would you mind sharing your steps or instructions about how to install Intel MKL library and how to compile OGS under Ubuntu 20.04 LTS with OGS_USE_MKL=ON?

Btw, I would like to inquire whether MKL library is free or open-source to download and install?

Thanks for your time!
Rui

3

Hi @dominik-kern,
I guess this might be somehow related to PardisoLU fails with unspecific message - #13 by joergbuchwald
Does that also happen if you delete the MKL paths for all BLAS variables when configuring?
Did you set the path variables via setvars source /opt/intel/oneapi/setvars.sh intel64?

4

Hi @Rui_Feng,
MKL does not cost anything for private/academic use but it is not open source.

  • You can download it here: https://www.intel.com/content/www/us/en/develop/documentation/get-started-with-mkl-for-dpcpp/top.html
  • The installer is just a bash script with GUI so you need to make it executable befro running.
  • The OneAPI thing is much bigger than what you actually need. So, you can select before install what you want (which is basically the IntelMKL) and deselect the rest.
  • After installing you need to set the environment variables as it is usually installed under /opt.
    This can be done by something like source /opt/intel/oneapi/setvars.sh intel64. You can put it in bashrc/zshrc such that you donā€™t need to do that every time.
  • To configure OGS with MKL, you can do something like cmake -DMKL_DIR="/opt/intel/oneapi/mkl/2021.4.0" -DOGS_USE_MKL="ON" -DCMAKE_BUILD_TYPE="Release" -G Ninja .. (replace MKL_DIR by the actual MKL_DIR and ... by the OGS source dir.
  • If you did not do the source command before, you can compile OGS directly, otherwise you need to deselect MKL for BLAS in order to not run into problems as state above. This can be done by ccmake .., then you can toggle into advanced mode by typing t. There you can delete the strings for all BLAS related variables. After that you can re-run configure by typing c once or twice until it is possible to type g for generating. After compilation Pardiso should work as Linear solver.
1 Like
5

It seems not really open-source in terms of license, but it is freely available.
I downloaded from
https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html
and then followed the instrutions
https://www.intel.com/content/www/us/en/develop/documentation/installation-guide-for-intel-oneapi-toolkits-linux/top/installation/install-using-package-managers/apt.html
to install IntelĀ® oneAPI Base Toolkit.

1 Like
6

Dear @joergbuchwald

Thanks for your instructions very much! In step 1-4, I think I follow your instruction very exactly and correctly, and it works well in step 1-4! But in step 5, one problem occurs, which is

when I executed this command cmake -DMKL_DIR="/opt/intel/oneapi/mkl/2022.0.2" -DOGS_USE_MKL="ON" -DCMAKE_BUILD_TYPE="Release" -G Ninja /home/ubuntu_2004tls/OGS6_linux/ogs, an error occured and the configuring was incomplete:

CMake Error at scripts/cmake/CMakeSetup.cmake:13 (CPMAddPackage):
  Unknown CMake command "CPMAddPackage".
Call Stack (most recent call first):
  CMakeLists.txt:13 (include)

CMake Error
CMake Error3840Ɨ2400 436 KB

Do you have some ideas why did this error occur? How can I solve it?

Looking forward to hearing from you!

Best regards,
Rui

7

Dear @joergbuchwald

I tried another time to compile OGS6 using this ccmake command in ogs source code directory: ccmake -S . --preset=release. I also deleted all strings with BLAS related variables in toggle t. It showed configuration done, but when I found /build/release/bin directory, it only contained OpenGeoSysCND.xsd OpenGeoSysGLI.xsd OpenGeoSysNum.xsd OpenGeoSysProject.xsd OpenGeoSysSTN.xsd. This output really made me confusing because I think it should be like in this video at 3:13, https://www.youtube.com/watch?v=GL5sugIyHEk&t=197s. Would you mind giving me some advice? Thank you very much!

Btw, one good news is that I can switch OGS_USE_MKL to ON during ccmake in this new try.

OS: Ubuntu 20.04LTS under WSL2

wrong bin
wrong bin2922Ɨ1077 468 KB

8

If configure with cmake/ccmake is done, you need to build ogs as shown in the videoor on this webpage: Build. So with ā€˜makeā€™ or ā€˜ninjaā€™ depending on how configured it. I think the preset uses ninja, the command I suggested above as well.

9

Dear @joergbuchwald

Thanks for your prompt reply! I built ogs using ninja -j 2, but it still didnā€™t work.

I used this command to configure ogs: ccmake -S . --preset=release
and this command to build ogs: ninja -j 2. The error shown below:

ninja error
ninja error1920Ɨ1200 311 KB

10

which compiler version do you use? I think you need at least gcc-10 for this option. It should be part of ubuntu-20.04 (sudo apt install gcc-10 g++-10)

11 
(base) ubuntu_2004tls@DESKTOP-GDOK97I:~$ gcc --version
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

It is gcc 9.3.0.

Actually, I followed this guide to Set Up Prerequisites. In this guide, Set Up Prerequisites told me that ā€œI need to have at least gcc 9.1ā€ :thinking:

12

Sure, you can also use an older version, but then you canā€™t use tigerlake/native for -march.
This refers to optimizations for your micro architecture. It is not a general OGS setting.

13

Dear @joergbuchwald

Thanks for your valuable reply very much! I updated gcc version to 10.3, but it still had errors. So, I summarize my questions to a new post, CMake error during configuring OGS6.

If possible, I hope you could give me some advice in your spare time. Thanks!

Best regards,
Rui

14

I am still having the same error, can you tell me what you (or automatically) entered into the following line (-t advanced mode)?

 BLAS_guide_LIBRARY               BLAS_guide_LIBRARY-NOTFOUND

Or even better all BLAS variables, I guess they should be set to the files inside /opt/intel/oneapi/....

15

Just to add to this. Until I fetched some recent changes today PardisoLU was working fine for me. What worked for me until now was to delete the paths in the BLAS variables in ccmake and rebuilding it.
If I donā€™t do this deletion on configuration I also get a undefined symbol error.
Unfortunately now my solver just fails where before the fetch it was working fineā€¦

16

Thank you, so I will give up for today. If somebody succeeds to compile the recent version of OGS with MKL, please let me know about the lines

BLAS_guide_LIBRARY               BLAS_guide_LIBRARY-NOTFOUND
BLAS_mkl_intel_LIBRARY           BLAS_mkl_intel_LIBRARY-NOTFOUND

The remaining BLAS variables I figured out so far, at least believe so.

17

Okay so I reset my branch to the state of 2 days ago and as of the commit 6fd4aa9daa7054c779c1fde61aa8e8b3fdb8124c PardisoLU is running fine with the configuration process described above. I should mention I didnā€™t install MKL via the link @joergbuchwald provided but just via sudo apt-get install intel-mkl
My variables with this working state look like this:

 BLAS_iomp5_LIBRARY               /usr/lib/x86_64-linux-gnu/libiomp5.so        
 BLAS_mkl_core_LIBRARY            /usr/lib/x86_64-linux-gnu/libmkl_core.so     
 BLAS_mkl_intel_LIBRARY           BLAS_mkl_intel_LIBRARY-NOTFOUND
 BLAS_mkl_intel_lp64_LIBRARY      /usr/lib/x86_64-linux-gnu/libmkl_intel_lp64.s
 BLAS_mkl_intel_thread_LIBRARY    /usr/lib/x86_64-linux-gnu/libmkl_intel_thread
 MKL_DIR                          MKL_DIR-NOTFOUND
 MKL_INCLUDE_DIR                  /usr/include/mkl
 MKL_LIB_CORE                     /usr/lib/x86_64-linux-gnu/libmkl_core.so     
 MKL_LIB_INTEL                    /usr/lib/x86_64-linux-gnu/libmkl_intel_lp64.s
 MKL_LIB_THREAD                   /usr/lib/x86_64-linux-gnu/libmkl_gnu_thread.s

Now when going to the current master branch, rebuilding and running a sim I get:
ogs: symbol lookup error: /lib/x86_64-linux-gnu/libmkl_intel_lp64.so: undefined symbol: mkl_blas_dsyrk
But the cmake vars above havenā€™t changed. And after cleaning my build dir and repeating the process it seems that the vars canā€™t be found as my vars suggest:

 BLAS_guide_LIBRARY               BLAS_guide_LIBRARY-NOTFOUND
 BLAS_iomp5_LIBRARY
 BLAS_mkl_LIBRARY                 BLAS_mkl_LIBRARY-NOTFOUND
 BLAS_mkl_core_LIBRARY
 BLAS_mkl_em64t_LIBRARY           BLAS_mkl_em64t_LIBRARY-NOTFOUND
 BLAS_mkl_ia32_LIBRARY            BLAS_mkl_ia32_LIBRARY-NOTFOUND
 BLAS_mkl_intel_LIBRARY           BLAS_mkl_intel_LIBRARY-NOTFOUND
 BLAS_mkl_intel_lp64_LIBRARY
 BLAS_mkl_intel_thread_LIBRARY    
 BLAS_mkl_rt_LIBRARY              /usr/lib/x86_64-linux-gnu/libmkl_rt.so

Interestingly there are now more BLAS variables??
And unsurprisingly PardisoLU again doesnā€™t work.
Maybe someone can find out the significant difference since the commit I used previously, which caused this change in behaviourā€¦

19

I was also not very successfull with MKL provided by my distribution. I would recommend deleting intel-mkl and install MKL directly from intel.

20

These are correct. My experience is that using MKL BLAS is not working in OGS.

21

Which BLAS version do you use then and how do you install it?