BornAgain

In the view of coming cleanup of ./App folder we have to create new machinery to be able to measure performance changes.

Minimal goal

• reproduce functionality of current App/TestPerformance

Suggestions/ideas:

• It has to be a python script in FunctionalTest/TestPyCore
• It has to rely mostly on SimulationRegistry/SampleBuilderFactory to construct set of standard geometries
• As possible candidates for testing geometries see App/TestPerformance
• Three general cases should be tested
  • Test of core "calculations": very large detector, test runs once.

i.e. how fast our form factors, functions, algorithms are

• Test of core "overheads": standard detector (100x100), test runs 100 times

i.e. how much overhead we have to form intensity data. Important for real time simulations in GUI.

• Test of overhead on python-C++ boundary crossing

Custom form factor in python

Questions:

• Do we need actually dedicated performance test?

Can't we just rely on execution time of our current functional tests?

We could introduce a longer version of "make check" where every test will be run 5 times. So instead of 20 sec it will take 100 sec so the measurements will be more reliable.

• How much geometries should we test?

Shell we test performance of every form factor and every interference function?

• How to access system information (CPU/Memory) usage in platform independent way.

Not very important, since tests will be run mostly on Linux

• How to measure performance reliably

warming up, throttling issues, CPU time or Wall time, single threaded or multi-threaded

• How to attach performance test to buildbot

Can we reliably measure the performance by running test in container?

• Should we include also test for memory leakages, memory allocations/deallocations.

For the moment, before every release we check for leakages (and unnecessary allocation/deallocations) using valgrind and MacOS/Insturment.