the project file and input mesh look almost good, I had to tweak few things a little.
First the mesh had a “hull” out of triangles which had to be removed. There are several ways, one of them would be to use a utility in OGS:
removeMeshElements -i /tmp/injectionmesh1.vtu -o /tmp/injectionmesh2.vtu -t tri
(Replace the mesh name in the prj afterwards.)
Another thing is the relation between permeability and time values. For the set permeability 1e-20 it would take much longer than one second for pressure to propagate. I replaced the permeability as shown, not sure where the embedded fracture permeability would be used in the setup:
and end time to 864000s with delta t 86400s. This produces some observable variations of the pressure field over time. This relation must be chosen carefully.
NB:
Biot coefficient should be in the range [porosity, 1]
I switched to PardisoLU solver, otherwise the simulation takes ages. A smaller mesh for initial experiments is recommended.
after spending some time working with your suggestions, I have a couple follow up questions for you. First, I removed my mesh hull using the removeMeshElements utility as you suggested however running my project file with the new mesh causes failure with the following error message:
Because of this, I decided to implement the rest of your suggestions while maintaining my original mesh. This allowed for visible changes to the pressure field over time as intended which I am very pleased with. This being said, the displacement still appears identical at 1s and 8640000s (after checking at your suggested t_end, I added another order order of magnitude to confirm that there would not be any change even very far into the future).
If you or anyone who reads this could provide some guidance into why the new mesh might be failing and why there is no change to the displacement I would very much appreciate it.
there is no displacement because the Biot coefficient is zero. Setting it to 1 enables the pressure-displacement coupling and the displacement is evolving together with the pressure over time.
Why the linear solver failed is difficult to say—please post the complete log file.
for now I have decided to ignore my previous solver error as I am planning on using Pardiso from now on, due to your recommendation. I hope that once Pardiso is up and running on my machine, this issue will not exist anymore. In order to compile OGS with MKL I have tried the following command:
Unfortunately, this produces the following output:
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Generator: Ninja
CMake Deprecation Warning at scripts/cmake/json-cmake/JSONParser.cmake:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
Call Stack (most recent call first):
scripts/cmake/Versions.cmake:1 (include)
CMakeLists.txt:82 (include)
-- Found Git: /usr/bin/git (found version "2.34.1")
-- OGS VERSION: 6.4.4-1517-g29e4abfe (reported by git)
-- Found Python: /usr/bin/python3.10 (found suitable version "3.10.12", required range is "3.6...<3.12") found components: Interpreter Development.Module Development.Embed
-- Using GNU gold linker. (GNU gold (GNU Binutils for Ubuntu 2.38) 1.16
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
)
-- Number of (logical) cores: 16, Free memory: 56163 MB
-- Job pool light_tasks using 16 cores.
-- Job pool heavy_tasks using 9 cores.
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) (Required is at least version "1.9.2")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Could NOT find TIFF (missing: TIFF_LIBRARY TIFF_INCLUDE_DIR)
-- Could NOT find GEOTIFF (missing: libgeotiff_LIBRARY libgeotiff_INCLUDE_DIR xtiff_INCLUDE_DIR TIFF_FOUND)
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5") found components: C CXX
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_intel_lp64.so;/opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_intel_thread.so;/opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_core.so;/opt/intel/oneapi/compiler/2023.2.0/linux/compiler/lib/intel64_lin/libiomp5.so;-lm;-ldl
-- Looking for cheev_
-- Looking for cheev_ - found
-- Found LAPACK: /opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_intel_lp64.so;/opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_intel_thread.so;/opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_core.so;/opt/intel/oneapi/compiler/2023.2.0/linux/compiler/lib/intel64_lin/libiomp5.so;-lm;-ldl;-lm;-ldl
CMake Error at scripts/cmake/FindMKL.cmake:80 (message):
Could not find MKL: please set environment or CMake variable MKLROOT or
MKL_DIR!
Call Stack (most recent call first):
scripts/cmake/Find.cmake:97 (find_package)
CMakeLists.txt:91 (include)
-- Configuring incomplete, errors occurred!
I have looked into this issue and found that you were able to help someone with a similar issue here: compiling OGS with MKL (#3155) · Issues · ogs / ogs · GitLab. Unfortunately, I after reading it, I am still unsuccessful. Any guidance on this would be appreciated.
I would recommend not to set MKL_DIR with cmake, but to execute the setvars script prior to the cmake command: source /opt/intel/oneapi/setvars.sh. Make sure that the build directory is empty. The setvars command is also needed before executing ogs, so it makes sense add this line to your .bashrc/zshrc.
I have added: source /opt/intel/oneapi/setvars.sh to my .bashrc and run: cmake -DOGS_USE_MKL="ON" -DCMAKE_BUILD_TYPE="Release" -G Ninja /home/liam/Downloads/ogsbuildtake2/ogs with an empty build directory. Unfortunately, the same error persists. I am fairly certain that the ‘setvars.sh’ is executing properly as every time that I open a new terminal session I am met with:
I then tried adding source /opt/intel/oneapi/mkl/latest/env/vars.sh to my .bashrc instead but this still gives the same result. Is it possible that I am missing something else? Should I be modifying my cmake command?
Hmokay, I have to say that there are several things that are a bit strange here.
The unspecific linear solver failure message suggests that OGS was compiled with MKL. So it ‘worked’ somehow earlier. There were just the linking issues with libgomp and libiomp5, I assume.
Before the CMAKE MKL error there is the BLAS/LAPACK message that lists basically all MKL shared libraries, so it obviously finds them.
source /opt/intel/oneapi/setvars.sh should be enough. Setting /opt/intel/oneapi/mkl/latest/env/vars.sh is probably not a good idea because then libiomp5 might not be found.
I suspect that maybe multiple MKL versions might interact (Some that came with the distribution), although not sure though because I can’t find different paths above. I could also think of some incompatibilities with the installed version?
You could try setting the corresponding paths manually via a gui like ccmake or maybe set MKL_ROOT instead of MKL_DIR (however, I’m pretty sure that these are not needed if the libraries are sourced corectly). @bilke do you have an idea what might have happened here?
Hi there, unfortunately it seems that the include/mkl_cblas.h file does not exist anywhere in my MKL installation tree. Perhaps I have incorrectly installed it?
Hi Joerg, I apologize for the confusion. The linear solver issue occurred before I changed to the Pardiso solver. So no, the OGS compilation with MKL did not work previously.
I have changed line 70 of the FindMKL.cmake file and unfortunately the same error persists. For more clarity, here is the cmake command that I have been using: cmake -DMKL_DIR="/opt/intel/oneapi/mkl/latest" -DOGS_USE_MKL="ON" -DCMAKE_BUILD_TYPE="Release" -G Ninja /home/liam/Downloads/ogsbuildtake2/ogs/
and the output that this command results in:
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Generator: Ninja
CMake Deprecation Warning at scripts/cmake/json-cmake/JSONParser.cmake:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
Call Stack (most recent call first):
scripts/cmake/Versions.cmake:1 (include)
CMakeLists.txt:82 (include)
-- Found Git: /usr/bin/git (found version "2.34.1")
-- OGS VERSION: 6.4.4-1517-g29e4abfe.dirty (reported by git)
-- Found Python: /usr/bin/python3.10 (found suitable version "3.10.12", required range is "3.6...<3.12") found components: Interpreter Development.Module Development.Embed
-- Using GNU gold linker. (GNU gold (GNU Binutils for Ubuntu 2.38) 1.16
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
)
-- Number of (logical) cores: 16, Free memory: 58152 MB
-- Job pool light_tasks using 16 cores.
-- Job pool heavy_tasks using 9 cores.
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) (Required is at least version "1.9.2")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Could NOT find TIFF (missing: TIFF_LIBRARY TIFF_INCLUDE_DIR)
-- Could NOT find GEOTIFF (missing: libgeotiff_LIBRARY libgeotiff_INCLUDE_DIR xtiff_INCLUDE_DIR TIFF_FOUND)
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5") found components: C CXX
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_intel_lp64.so;/opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_intel_thread.so;/opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_core.so;/opt/intel/oneapi/compiler/2023.2.0/linux/compiler/lib/intel64_lin/libiomp5.so;-lm;-ldl
-- Looking for cheev_
-- Looking for cheev_ - found
-- Found LAPACK: /opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_intel_lp64.so;/opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_intel_thread.so;/opt/intel/oneapi/mkl/2023.2.0/lib/intel64/libmkl_core.so;/opt/intel/oneapi/compiler/2023.2.0/linux/compiler/lib/intel64_lin/libiomp5.so;-lm;-ldl;-lm;-ldl
CMake Error at scripts/cmake/FindMKL.cmake:80 (message):
Could not find MKL: please set environment or CMake variable MKLROOT or
MKL_DIR!
Call Stack (most recent call first):
scripts/cmake/Find.cmake:97 (find_package)
CMakeLists.txt:91 (include)
-- Configuring incomplete, errors occurred!
No. In fact there is not any “include” folder located in /opt/intel/oneapi/mkl/latest/. This directory contains 4 folders labelled: “env”, “lib”, “licensing” and “modulefiles”.
Then there is something wrong with your installation. Maybe you only installed the runtime components but you also need the development components.
How did you install mkl? If you install it with your system package manager there is maybe a package called mkl-devel or mkl-dev which you may need to install.
There was no mkl-devel or mkl-dev package for Ubuntu but I was able to install the libmkl-dev package. Now I have the mkl.h file as well as a mkl_blas.h file however they are both located at /usr/include rather than /opt/intel/oneapi/mkl/latest/include/. I have again tried the same cmake command from earlier with the same result. Perhaps I need to change the DMKL_DIR path in my command?
