PythonAPI: redesing Py++ boost::python API code generation and installation
|Status:||Archived||Start date:||30 Mar 2013|
|Target version:||Sprint 12|
- find the reason of huge memory consumption during automatic boost python code generation (pygccxml)
#4 Updated by pospelov about 7 years ago
- Tracker changed from Bug to Refactoring
- Subject changed from PythonAPI: find the reason of huge memory consumption during autmatic boost python code generation (pygccxml) to PythonAPI: redesing Py++ boost::python API code generation and installation
- Description updated (diff)
- The main problem for boost/Py++ is the presence of pure virtual functions.
Ideally I would like to expose to python only methods which are necessary for sample creation, running the simulation and retrieving the results.
However, If I'm excluding all unnecessary methods from Py++ consideration, including pure virtual methods, than Py++ produces the boost::python code which is not compilable.
The problem is in a way boost implements for Python mechanism of function overloading. According to the boost we have to provide lots of additional wrapping classes (it's where we use Py++ to generates them).
PyObject is constructed from our class under consideration in a way:
boost::class_<Wrapper_of_Base<Base> > x;
Here 'Base' is our class, 'Wrapper_Of_Base' is Py++ generated wrapper which contains overloading magic, boost::class_<Wrapper_Of_BAse> is our future PyObject
If I exclude in Py++ virtual methods for Base class, than generated code of Wrapper_of_Base will not have anything about these virtual methods too.
As a result, compiler complains that we are trying instantiate template from a class without implementing pure virtual methods.
That means, that we have to expose all pure virtual methods (and, in principle, we have to expose all types in method's arguments).
virtual void getInterference(IInterference *interference) = 0;
(both, getInterference method and IInterference class should be exposed)
That's a lot.
#5 Updated by pospelov about 7 years ago
PythonAPI generation has been fully redesigned.
Now it resides in dev-tools/python-bindings
See instruction how to use hereList of significant changes
- now regeneration of the whole PythonAPI takes 20 sec of CPU and 200 Mb of memory instead of 5 min and 4Gb of memory
- code generation works under Linux too
- installation of generated code into BornAgain source tree is more gentle. There is no replacement of unchanged files so recompilation is faster.
- as a side ane very pleasant effect - shared_ptr are working from python
- so no technical reasons to get rid from PTransform3D and shared_ptr to make PythonAPI working
- with remainder, that we have to set global policy about shared_ptr in BornAgain in the view of multi threading...
- ...and make lots of tests before actually starting to modify interfaces
- All Python, C++ functional tests are working