INTEL MKL: undefined symbol

Hi @Rui_Feng,
MKL does not cost anything for private/academic use but it is not open source.

  • You can download it here: Get Started with Intel® oneAPI Math Kernel Library
  • The installer is just a bash script with GUI so you need to make it executable befro running.
  • The OneAPI thing is much bigger than what you actually need. So, you can select before install what you want (which is basically the IntelMKL) and deselect the rest.
  • After installing you need to set the environment variables as it is usually installed under /opt.
    This can be done by something like source /opt/intel/oneapi/setvars.sh intel64. You can put it in bashrc/zshrc such that you don’t need to do that every time.
  • To configure OGS with MKL, you can do something like cmake -DMKL_DIR="/opt/intel/oneapi/mkl/2021.4.0" -DOGS_USE_MKL="ON" -DCMAKE_BUILD_TYPE="Release" -G Ninja .. (replace MKL_DIR by the actual MKL_DIR and ... by the OGS source dir.
  • If you did not do the source command before, you can compile OGS directly, otherwise you need to deselect MKL for BLAS in order to not run into problems as state above. This can be done by ccmake .., then you can toggle into advanced mode by typing t. There you can delete the strings for all BLAS related variables. After that you can re-run configure by typing c once or twice until it is possible to type g for generating. After compilation Pardiso should work as Linear solver.
1 Like

It seems not really open-source in terms of license, but it is freely available.
I downloaded from

and then followed the instrutions

to install Intel® oneAPI Base Toolkit.

1 Like

Dear @joergbuchwald

Thanks for your instructions very much! In step 1-4, I think I follow your instruction very exactly and correctly, and it works well in step 1-4! But in step 5, one problem occurs, which is

when I executed this command cmake -DMKL_DIR="/opt/intel/oneapi/mkl/2022.0.2" -DOGS_USE_MKL="ON" -DCMAKE_BUILD_TYPE="Release" -G Ninja /home/ubuntu_2004tls/OGS6_linux/ogs, an error occured and the configuring was incomplete:

CMake Error at scripts/cmake/CMakeSetup.cmake:13 (CPMAddPackage):
  Unknown CMake command "CPMAddPackage".
Call Stack (most recent call first):
  CMakeLists.txt:13 (include)

Do you have some ideas why did this error occur? How can I solve it?

Looking forward to hearing from you!

Best regards,
Rui

Dear @joergbuchwald

I tried another time to compile OGS6 using this ccmake command in ogs source code directory: ccmake -S . --preset=release. I also deleted all strings with BLAS related variables in toggle t. It showed configuration done, but when I found /build/release/bin directory, it only contained OpenGeoSysCND.xsd OpenGeoSysGLI.xsd OpenGeoSysNum.xsd OpenGeoSysProject.xsd OpenGeoSysSTN.xsd. This output really made me confusing because I think it should be like in this video at 3:13, "OpenGeoSys Workflow Tutorial: Solving - YouTube. Would you mind giving me some advice? Thank you very much!

Btw, one good news is that I can switch OGS_USE_MKL to ON during ccmake in this new try.

OS: Ubuntu 20.04LTS under WSL2

If configure with cmake/ccmake is done, you need to build ogs as shown in the videoor on this webpage: Build. So with ‘make’ or ‘ninja’ depending on how configured it. I think the preset uses ninja, the command I suggested above as well.

Dear @joergbuchwald

Thanks for your prompt reply! I built ogs using ninja -j 2, but it still didn’t work.

I used this command to configure ogs: ccmake -S . --preset=release
and this command to build ogs: ninja -j 2. The error shown below:

which compiler version do you use? I think you need at least gcc-10 for this option. It should be part of ubuntu-20.04 (sudo apt install gcc-10 g++-10)

(base) ubuntu_2004tls@DESKTOP-GDOK97I:~$ gcc --version
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

It is gcc 9.3.0.

Actually, I followed this guide to Set Up Prerequisites. In this guide, Set Up Prerequisites told me that “I need to have at least gcc 9.1” :thinking:

Sure, you can also use an older version, but then you can’t use tigerlake/native for -march.
This refers to optimizations for your micro architecture. It is not a general OGS setting.

Dear @joergbuchwald

Thanks for your valuable reply very much! I updated gcc version to 10.3, but it still had errors. So, I summarize my questions to a new post, CMake error during configuring OGS6.

If possible, I hope you could give me some advice in your spare time. Thanks!

Best regards,
Rui

I am still having the same error, can you tell me what you (or automatically) entered into the following line (-t advanced mode)?

 BLAS_guide_LIBRARY               BLAS_guide_LIBRARY-NOTFOUND

Or even better all BLAS variables, I guess they should be set to the files inside /opt/intel/oneapi/....

Just to add to this. Until I fetched some recent changes today PardisoLU was working fine for me. What worked for me until now was to delete the paths in the BLAS variables in ccmake and rebuilding it.
If I don’t do this deletion on configuration I also get a undefined symbol error.
Unfortunately now my solver just fails where before the fetch it was working fine…

Thank you, so I will give up for today. If somebody succeeds to compile the recent version of OGS with MKL, please let me know about the lines

BLAS_guide_LIBRARY               BLAS_guide_LIBRARY-NOTFOUND
BLAS_mkl_intel_LIBRARY           BLAS_mkl_intel_LIBRARY-NOTFOUND

The remaining BLAS variables I figured out so far, at least believe so.

Okay so I reset my branch to the state of 2 days ago and as of the commit 6fd4aa9daa7054c779c1fde61aa8e8b3fdb8124c PardisoLU is running fine with the configuration process described above. I should mention I didn’t install MKL via the link @joergbuchwald provided but just via sudo apt-get install intel-mkl
My variables with this working state look like this:

 BLAS_iomp5_LIBRARY               /usr/lib/x86_64-linux-gnu/libiomp5.so        
 BLAS_mkl_core_LIBRARY            /usr/lib/x86_64-linux-gnu/libmkl_core.so     
 BLAS_mkl_intel_LIBRARY           BLAS_mkl_intel_LIBRARY-NOTFOUND
 BLAS_mkl_intel_lp64_LIBRARY      /usr/lib/x86_64-linux-gnu/libmkl_intel_lp64.s
 BLAS_mkl_intel_thread_LIBRARY    /usr/lib/x86_64-linux-gnu/libmkl_intel_thread
 MKL_DIR                          MKL_DIR-NOTFOUND
 MKL_INCLUDE_DIR                  /usr/include/mkl
 MKL_LIB_CORE                     /usr/lib/x86_64-linux-gnu/libmkl_core.so     
 MKL_LIB_INTEL                    /usr/lib/x86_64-linux-gnu/libmkl_intel_lp64.s
 MKL_LIB_THREAD                   /usr/lib/x86_64-linux-gnu/libmkl_gnu_thread.s 

Now when going to the current master branch, rebuilding and running a sim I get:
ogs: symbol lookup error: /lib/x86_64-linux-gnu/libmkl_intel_lp64.so: undefined symbol: mkl_blas_dsyrk
But the cmake vars above haven’t changed. And after cleaning my build dir and repeating the process it seems that the vars can’t be found as my vars suggest:

 BLAS_guide_LIBRARY               BLAS_guide_LIBRARY-NOTFOUND
 BLAS_iomp5_LIBRARY
 BLAS_mkl_LIBRARY                 BLAS_mkl_LIBRARY-NOTFOUND
 BLAS_mkl_core_LIBRARY
 BLAS_mkl_em64t_LIBRARY           BLAS_mkl_em64t_LIBRARY-NOTFOUND
 BLAS_mkl_ia32_LIBRARY            BLAS_mkl_ia32_LIBRARY-NOTFOUND
 BLAS_mkl_intel_LIBRARY           BLAS_mkl_intel_LIBRARY-NOTFOUND
 BLAS_mkl_intel_lp64_LIBRARY
 BLAS_mkl_intel_thread_LIBRARY    
 BLAS_mkl_rt_LIBRARY              /usr/lib/x86_64-linux-gnu/libmkl_rt.so

Interestingly there are now more BLAS variables??
And unsurprisingly PardisoLU again doesn’t work.
Maybe someone can find out the significant difference since the commit I used previously, which caused this change in behaviour…

I was also not very successfull with MKL provided by my distribution. I would recommend deleting intel-mkl and install MKL directly from intel.

These are correct. My experience is that using MKL BLAS is not working in OGS.

Which BLAS version do you use then and how do you install it?

actually I don’t use a separate BLAS. I’m not sure what happens internally. I guess Eigen has its own BLAS.

Anything Jörg recommend so far is correct, the missing puzzle piece in my case was to remove the old MKL

sudo apt remove intel-mkl

and possibly autoremove, so that OGS solely relies on the latest MKL from Intel.

Another useful hint by Aqeel was to use the clang compiler, alternatively, thanks to Jörg, you may select another linker https://gitlab.opengeosys.org/ogs/ogs/-/commit/02967311512e5b9e2e747cd2b6832e17a54140f3

It seems to be that this commit is really breaking things with some configurations.
I opened an issue: 02967311512e5b9e2e747cd2b6832e17a54140f3 causes undefinded symbol error on some machines when compiled intel MKL (#3259) · Issues · ogs / ogs · GitLab
There, I also propose another fix.