Bug #1913

Thread count issue on Windows on modern CPU

Added by pospelov almost 3 years ago. Updated over 2 years ago.

Status:ResolvedStart date:01 Dec 2017
Priority:HighDue date:
Assignee:-% Done:

0%

Category:-
Target version:Sprint 36

Description

Arthur reporting
https://github.com/scgmlz/BornAgain/issues/322

From my own observation, my home PC based on WIndows 10 + AMD Ryzen (8 core, 16 threads) behaves very similar to Arhur's.
In my case:
  • 8 threads selected leads to 50% CPU load
  • Any other selection (i.e. 2 threads, 5 threads, 15 threads and max 16 threads) lead to only 1 thread busy and CPU at 6%

Main question - how to debug this if we do not have any modern CPU in the group?

Capture.PNG - compiler flags on buildserver (13.6 KB) herck, 19 Feb 2018 16:53

History

#1 Updated by herck over 2 years ago

  • Assignee set to herck

#2 Updated by herck over 2 years ago

Created a simple program for testing CPU load when using a variable number of threads:
https://github.com/waltervh/multithreading.git

#3 Updated by herck over 2 years ago

  • Assignee deleted (herck)

#4 Updated by herck over 2 years ago

#5 Updated by pospelov over 2 years ago

Compiler and linker flags at Gennady's home PC (Ryzen 8 cores)

Multithreading output


  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.12.25827\bin\HostX86\x64\CL.exe /c /IC
  :\opt\multithreading /nologo /W3 /WX- /diagnostics:classic /O2 /Ob2 /D WIN32 /D _WINDOWS /D NDEBUG /D "CMAKE_INTDIR=\
  "Release\"" /D _MBCS /Gm- /EHsc /MD /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR /Fo"MultiThreading.dir\Re
  lease\\" /Fd"MultiThreading.dir\Release\vc141.pdb" /Gd /TP /errorReport:queue C:\opt\multithreading\main.cpp C:\opt\m
  ultithreading\threadfunctions.cpp
  main.cpp
  threadfunctions.cpp
  Generating Code...
Link:
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.12.25827\bin\HostX86\x64\link.exe /ERR
  ORREPORT:QUEUE /OUT:"C:\opt\multithreading\build\Release\MultiThreading.exe" /INCREMENTAL:NO /NOLOGO kernel32.lib use
  r32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIF
  ESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:"C:/opt/multithreading/build/Release/MultiThreading.
  pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/opt/multithreading/build/Release/MultiThreading.l
  ib" /MACHINE:X64  /machine:x64 MultiThreading.dir\Release\main.obj
  MultiThreading.dir\Release\threadfunctions.obj
  MultiThreading.vcxproj -> C:\opt\multithreading\build\Release\MultiThreading.exe

BornAgain compilation

  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.12.25827\bin\HostX86\x64\link.exe /ERR
  ORREPORT:QUEUE /OUT:"C:\opt\build\bin\TestGUI.exe" /INCREMENTAL:NO /NOLOGO ..\..\..\GUI\coregui\Release\BornAgainGUI.
  lib "..\..\..\ThirdParty\common\gtest\gtest-1.8.0\Release\gtest.lib" C:\Qt\5.10.1\msvc2017_64\lib\Qt5Designer.lib C:\
  Qt\5.10.1\msvc2017_64\lib\Qt5Network.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Test.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Xml
  .lib "..\..\..\ThirdParty\GUI\qt-manhattan-style\Release\ManhattanStyle.lib" ..\..\..\ThirdParty\GUI\qcustomplot\Rele
  ase\qcustomplot.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5PrintSupport.lib ..\..\..\Core\Release\BornAgainCore.lib ..\..\..
  \Fit\Release\BornAgainFit.lib ..\..\..\ThirdParty\Fit\RootMinimizers\Release\RootMinimizers.lib C:\opt\local_x64\lib\
  libtiff.lib "C:\opt\local_x64\lib\boost_date_time-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_chrono-vc141-mt-1_
  65_1.lib" "C:\opt\local_x64\lib\boost_program_options-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_iostreams-vc14
  1-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_system-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_filesystem-vc141
  -mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_regex-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_zlib-vc141-mt-1_65
  _1.lib" "C:\opt\local_x64\lib\boost_bzip2-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\libfftw3-3.lib" C:\Anaconda\libs
  \python36.lib C:\opt\local_x64\lib\gsl.lib C:\opt\local_x64\lib\cblas.lib ..\..\..\ThirdParty\Core\Faddeeva\Release\F
  addeeva.lib ..\..\..\GUI\ba3d\ba3d\Release\ba3d.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5OpenGL.lib C:\Qt\5.10.1\msvc2017_
  64\lib\Qt5Widgets.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Gui.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Core.lib kernel32.lib u
  ser32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MAN
  IFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:"C:/opt/build/bin/TestGUI.pdb" /SUBSYSTEM:CONSOLE
  /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/opt/build/Tests/UnitTests/GUI/Release/TestGUI.lib" /MACHINE:X64  /machine
  :x64 TestGUI.dir\Release\Comparators.obj

Link:
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.12.25827\bin\HostX86\x64\link.exe /ERR
  ORREPORT:QUEUE /OUT:"C:\opt\build\bin\BornAgain.exe" /INCREMENTAL:NO /NOLOGO ..\coregui\Release\BornAgainGUI.lib "..\
  ..\ThirdParty\GUI\qt-manhattan-style\Release\ManhattanStyle.lib" ..\..\ThirdParty\GUI\qcustomplot\Release\qcustomplot
  .lib ..\..\Core\Release\BornAgainCore.lib ..\..\Fit\Release\BornAgainFit.lib ..\..\ThirdParty\Fit\RootMinimizers\Rele
  ase\RootMinimizers.lib C:\opt\local_x64\lib\libtiff.lib "C:\opt\local_x64\lib\boost_date_time-vc141-mt-1_65_1.lib" "C
  :\opt\local_x64\lib\boost_chrono-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_program_options-vc141-mt-1_65_1.lib
  " "C:\opt\local_x64\lib\boost_iostreams-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_system-vc141-mt-1_65_1.lib" 
  "C:\opt\local_x64\lib\boost_filesystem-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_regex-vc141-mt-1_65_1.lib" "C
  :\opt\local_x64\lib\boost_zlib-vc141-mt-1_65_1.lib" "C:\opt\local_x64\lib\boost_bzip2-vc141-mt-1_65_1.lib" "C:\opt\lo
  cal_x64\lib\libfftw3-3.lib" C:\Anaconda\libs\python36.lib C:\opt\local_x64\lib\gsl.lib C:\opt\local_x64\lib\cblas.lib
   ..\..\ThirdParty\Core\Faddeeva\Release\Faddeeva.lib ..\ba3d\ba3d\Release\ba3d.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Op
  enGL.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Designer.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Xml.lib C:\Qt\5.10.1\msvc2017_6
  4\lib\Qt5PrintSupport.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Widgets.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Gui.lib C:\Qt\5
  .10.1\msvc2017_64\lib\Qt5Network.lib C:\Qt\5.10.1\msvc2017_64\lib\Qt5Core.lib C:\Qt\5.10.1\msvc2017_64\lib\qtmain.lib
   kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib
   /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:"C:/opt/build/bin/BornAgain.pdb" /S
  UBSYSTEM:WINDOWS /TLBID:1 /ENTRY:"mainCRTStartup" /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/opt/build/GUI/main/Release/BornA
  gain.lib" /MACHINE:X64  /machine:x64 BornAgain.dir\Release\bornagain.res
  BornAgain.dir\Release\main.obj
  BornAgain.dir\Release\appoptions.obj
  BornAgain.dir\Release\mocs_compilation.obj
  BornAgain.vcxproj -> C:\opt\build\bin\BornAgain.exe

#6 Updated by pospelov over 2 years ago

Current behavior:

  • Threads from 1-9 behaves as expected, with CPU load from 6% to 60%
  • Threads 10-16 - CPU load at 6%

Ideas:

  • Run Python example and check CPU load

Can't do it, crash on numpy library load

  • Run GUI in Visual Studio debugger to see actual number of used threads.

Can't compile in Debug mode, no boost libraries in debug.

  • Can't it be, that we have some kind of thread lock?

#7 Updated by pospelov over 2 years ago

  • Status changed from Sprint to Resolved

Also available in: Atom PDF