Move to Jugit, gitlab-ci, scgbuild, local Windows
|Status:||Rejected||Start date:||22 May 2020|
- Compilation on free cloud services is unacceptably slow:
- Queueing times are often of the order of hours, and unpredictable.
- When a PR is finally ready for merging, a rebase on develop may be needed. Then the waiting starts over again.
- We need CI for more target system:
- We need CI for Mac.
- Under Linux, we want to cover different distributions, two different compilers (clang and gcc), and different compiler versions.
- We need to shut down the old apps server; so we need a replacement for Redmine anyway.
- When team members (especially myself) work on different projects, they ideally want to use the same tools; at least, the want tools of comparable quality.
- Repository with public issue tracker on Jugit
- Internal issue tracker in project ba-intern, already on Jugit
- Integration tests controlled by gitlab-ci
- Tests under different Linux distributions and versions in docker containers on scgbuild
- Windows and Mac tests on local machines.
To JuGit discussion by Gennady Pospelov¶
These are my thoughts related to the migration of BornAgain repository
from GitHub to JuGit.
Let me first set the ground for a discussion: what do we want to achieve?
Better Continuos Integration¶
Continuous integration serves two main goals.
Support for release procedure
The main goal of CI is to make the release easy. The release procedure should be not more than changing the version number, followed by a push to the release branch. All the rest, including upload of installers to a website, should be done automatically. Project [easyDiffraction](https://github.com/easyDiffraction/easyDiffraction) by Thomas Rod group has almost this state (see `.travis` section and sophisticated `Scripts` directory in this repository). Release section of `easyDiffraction` is filled automatically, developers only change label `Draft` to the `Release` in GitHub UI, when ready.
Support for pull requests
The second main goal of CI is to provide fast validation of pull requests. Ideally, pull requests should be built in parallel, each developer should have the possibility to build the code for all supported platforms on his own before providing a pull request.
What should be the CI's capabilities?
In my opinion, with the proper planning phase, proper test-driven-development, and proper balance `features/refactoring` a single developer can produce, on average, one pull request per day (with average size 20 commits, 20 files changed). Also, in my opinion, a single developer is comfortable in reviewing a single pull request of a similar size per day. Thus, CI for a team of 3 people should be capable of swallowing 3 pull requests within reasonable (30 min for 3 requests (?), to be defined) time.
GitHub with Actions system allows us to achieve that.
CI with GitHub Actions¶
+ Free build server for Linux, Windows, and Mac.
+ Parallel builds are allowed (4-20 parallel builds can serve parallel pull requests).
+ It is possible to mask subdirectories not to trigger build (i.e. Documentation).
+ Caching of build environment (build time decreases 35min first build, down to 10 min following builds).
+ Trivial to set up CI for own development branch.
+ Isolated build environment, no build server is required.
+ [Marketplace](https://github.com/marketplace?type=actions) with community solutions (see for example [Release Draftter](https://github.com/marketplace/actions/release-drafter)).
+ Native integration with GitHub.
+ Solution is scalable (switch from Actions back to Appveyor, or to AWS, or connect with own build server like Mantid).
+ [GitHub has self-hosted runners](https://help.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners)
We have switched already to new GitHub Actions instead of Travis/Appveyor. Now collecting statistics.
Issues with JuGit¶
GitLab runner is not everything
GitLab has convenient GitLab runner and integration with UI. However, it is not a build server. All configurations still have to be done, all Build environment (docker containers, Windows PC) have to be supported. The existence of a runner is nice but makes not much difference concerning the amount of work to setup CI properly. The necessity to support the build server and provide software updates (i.e. Python, Qt) and docker container updates has to be added to the overall price.
Current plans of migrations to GitLab say not much about Mac OS
Where we are going to build it? What will be the CI performance? How we are going to follow XCode/OS upgrades?
Current plans of migrations to GitLab say nothing about Windows environment isolation
One of the biggest problems with BornAgain was the interference with Steca Windows builds. How do we provide isolation of build environment and safety from any perturbation in other projects?
JuGit is community unfriendly
Access to JuGit is non-trivial. Migration to it will ruin our (starting) social activity. When Andrew Nelson mentions one of us in his [reflectivity](https://github.com/reflectivity) we see it. We can mention him back in one of our issues, and he will be notified.
It is not GitHub
Working with a large Open Source project, developed GitHub profile, reach daily contribution activity is getting more and more important nowadays while looking for a job. I'm sure, our @anikhalder, @Entkenntnis, and others are pretty happy to have BornAgain in the list of their repositories. I would be
sorry to lose the most mature project of my life from my GitHub statistics.
GitHub can do everything what JuGit can. JuGit can't make everything that is possible on GitHub.
Instead of conclusion¶
BornAgain is our most advanced project which is in the most advanced state. All other projects of our group cannot present such a level of multi-platform support, any close test coverage, user support, and comparable level of CI. Such an advanced state was achieved by many years of hard work for many people. Continuous integration in BornAgain should be improved and go to the next level, however, it should be done in an evolutionary manner. Basically, everything is working already, the price was already paid. Why break the working thing?
In this issue you[Joachim] rise 3 different and, to my opinion, independent problems:
- Slow CI
- Change from mono-repository to multi-repository pattern
- Moving to jugit
Below I will address these issues one by one.
1. Slow CI
Here I agree to Gennady: slow CI problem can be easily solved by use of the github actions.
- It supports both, self-hosted and github-hosted runners. Thus, use of our build-server with github actions is also possible, although github-hosted runners allow for more benefits.
- Github-hosted runners allow for up to 5 simultaneous builds for mac os x and for up to 20 simultaneous builds for other systems. Thus, no queuing even for several PRs submitted simultaneously.
- The intelligent caching machinery of github-hosted runners essentially accelerates builds: although the very first build may take about 30 minutes, the subsequent builds finish in about 5-10 minutes.
- It is easy to add some directories to "ignore-paths" and, for example, do not run the full sequence of CI builds if PR concerns just some changes in the manual.
- Besides virtual environments, github actions also support docker containers. Thus, it is easy to integrate builds on various linux systems in BornAgain CI.
- All configuration files are written in YAML format and stored in the git repository under the version control. This means that even if some changes break the CI, they can be easily reverted.
Regarding time investment. I've spent one full working day to understand how the github actions work and have created drafts of working CI for ubuntu and mac os x and a half-working one for windows. I estimate, that in a few more working days one can create a fully-working CI workflow on the base of the github actions including publication of nightly builds and even automated publication of release builds.
2. Change from mono-repository to multi-repository pattern
To my knowledge, both patterns make sense and the choice for the particular project should be thoroughly thought through. Switching the pattern is usually a lot of work and will require a lot of time (especially on side of Windows and MacOS installers). It will also cause delays in development and missing the deadlines.
Generally, the multi-repository pattern works well for projects consisting of many weakly-connected modules and with large development teams. For smaller projects with smaller development teams usually mono-repository pattern works better.
What exactly do you want to achieve by splitting BornAgain to Fit, PyFit, Core, PyCore, GUI?
If libMVVM is completely independent on BornAgain, then it can be treated as any other external dependency (like boost, eigen, fftw, etc). Thus, it is not required to split the repository to add this new dependency.
What exactly do you mean under PyCore? If it is Python API, then I would not move it to the separate repository, since most of changes in the core will require also changes in the Python API. This will be much easier to handle having them both in one repository. The same is for other parts which need to be changed to follow changes in the core.
Imho, if the aim is only to build different parts separately, it is enough to revise the cmake build machinery to fulfill it. Since libMVVM is already in the other repository, it can be handled as any other external dependency and does not require splitting of BornAgain.
3. Moving to Jugit
Here I am fully agree to Gennady. To my opinion, both, Jugit and Github are quite powerful version control systems and the use of one or the other is mainly a matter of personal taste. However, moving of the repository with an established over several years history from one system to the other is a major step and has to be thoroughly thought through, should bring significant benefits and should be agreed to the all developers.
Presently, I do not see any benefits from moving BornAgain from Github to Jugit. However, I see the following disadvantages:
- this will destroy the workflow of other developers and will cause delays in development and missing the deadlines. For this reason, priority of this step has to be agreed to ESS, since they expect delivery of particular functionality by September.
- jugit requires a Jülich email address to access the repository. I have an experience that even colleagues from other Helmholtz centers were not able to login to jugit. Presently we do not have any external developers and can probably organize access for our students. However, as far as I remember, ESS had an interest to contribute to BornAgain at some point. Thus, they should also know about this issue and agree to that before moving the repository.
- BornAgain is a community project and is used not only in Jülich, but has a broad user community worldwide. Presently, users can easily register at Github and create issues with questions and bug reports. Other users can benefit from reading of our answers. By moving the repository, the created knowledge database will be lost. The active user community will be lost as well, since most of our users will not be able to contribute issues any more.
- BornAgain is significantly larger than Steca or other projects of our group presently hosted on jugit. Thus, moving CI to the scgbuild will not solve "the slow CI" problem. Even worse, it will highly probably cause additional problems especially for mac and windows builds. For example, CI on scgbuild, old mac mini and old windows laptop would not allow for multiple simultaneous builds and can cause interference with other projects of our group.