Ready for some practical tricks on how to use the build system? Create custom builds, deterministic builds, reproduce specific builds, and more!
This is a follow-up to the build system overview, if you haven’t, read that first!
Below are some examples of use cases, and if you want to cross-reference the build system diagrams, the services interacting from this point are:
- GitHub (d2-ci)
- GitHub (dhis2)
What can we do with it?
The primary use cases which are useful to be aware of are:
- Answer questions about a build
- Create custom builds
- Control when to fetch new apps
Answer questions about a build
What are the bundled applications in a build?
dhis-web-apps module serves all the apps which were bundled into the
WAR at build time and includes the time the WAR-file was assembled and from
which commit it was created from.
Example build information
Apps * dhis-web-app-management * dhis-web-cache-cleaner * dhis-web-capture * dhis-web-visualizer * dhis-web-core-resource * dhis-web-dashboard * dhis-web-data-administration * dhis-web-data-visualizer * dhis-web-data-quality * dhis-web-datastore * dhis-web-event-capture * dhis-web-event-reports * dhis-web-event-visualizer * dhis-web-importexport * dhis-web-interpretation * dhis-web-maintenance * dhis-web-mapping * dhis-web-maps * dhis-web-menu-management * dhis-web-messaging * dhis-web-pivot * dhis-web-scheduler * dhis-web-settings * dhis-web-tracker-capture * dhis-web-translations * dhis-web-usage-analytics * dhis-web-user * dhis-web-user-profile Mon Mar 25 2019 15:37:12 GMT+0000 (UTC) 790c8930767b13e24cf63c6949cba62b0d56d889
Exactly what versions of the apps does a build contain?
To get a list of applications along with a reference to the exact build of the
application you can query the
dhis-web-apps module for the
[ "https://github.com/d2-ci/app-management-app#97862203ef37d8a69d51b8eb556a5958148e6b72", "https://github.com/d2-ci/cache-cleaner-app#1e6ce2230b6f4e00402d41057a9054d23e0029b3", "https://github.com/d2-ci/capture-app#05ed5635654f6542637dcf04ec332467c9d32066", "https://github.com/d2-ci/charts-app#ceab8f259e812b4683bdc871e4bce85fb9632331", "https://github.com/d2-ci/core-resource-app#0d6099a0b3e88f03735d7284b662cbeb7ed0aae5", "https://github.com/d2-ci/dashboards-app#1eb94a4df386b3c695df1dc458d761b29c33c3c5", "https://github.com/d2-ci/data-administration-app#8c6aeb2552143a868e883218fde416e2b3306832", "https://github.com/d2-ci/data-visualizer-app#ecaf7dbd6a906b79461d945c3ba70fe346de2979", "https://github.com/d2-ci/data-quality-app#3166be606054bb7f5a979c041048450d97e1e6fe", "https://github.com/d2-ci/datastore-app#8f38acb9cf3f97e13310c21ccf4bc4a585fedab5", "https://github.com/d2-ci/event-capture-app#889463e56df7072399d410474f9292aa69c3f2d1", "https://github.com/d2-ci/event-reports-app#31ac64d73063c33b501961d0373a0e754532c3ec", "https://github.com/d2-ci/event-charts-app#cc9d4364ad84abbcd2a64a0cbf99382f10bc9ca8", "https://github.com/d2-ci/import-export-app#ebc5a7c0c92890433880f82cdacb28b8c78fcf87", "https://github.com/d2-ci/interpretation-app#fd0693d79cede942a4d6f61a47f0138ae816ee3f", "https://github.com/d2-ci/maintenance-app#2d1b789c709344398fc28731189e4bef7865db7b", "https://github.com/d2-ci/gis-app#f84373dc97b441a1e32d9a68307631fe6bfc99e4", "https://github.com/d2-ci/maps-app#5840ec14e92e5f1cf6d54136da114a8844fe6318", "https://github.com/d2-ci/menu-management-app#6f218875dcfd199d9ff40ff105e6de5012599b3a", "https://github.com/d2-ci/messaging-app#632212396dd5d58600748c95cbec6ae836b9837e", "https://github.com/d2-ci/pivot-tables-app#8afe54cc833370aa55f3bc14e3e82f4f6616fad0", "https://github.com/d2-ci/scheduler-app#8ac36309919f9cfd8bda5c2610c36802a3229f49", "https://github.com/d2-ci/settings-app#5f1b31099754395885c1adc650fbb3bd8eeae521", "https://github.com/d2-ci/tracker-capture-app#40717db9fbb6bdc9a006a148114fb764951a4e7c", "https://github.com/d2-ci/translations-app#5612f5392b1ae0134b0024604f0d33bf726a5e63", "https://github.com/d2-ci/usage-analytics-app#23effbc623c62c346dc75d18648162d788cea30d", "https://github.com/d2-ci/user-app#94d08dca9ab1dccb1f0dc031d5e3de79e17748f6", "https://github.com/d2-ci/user-profile-app#0671f19a4cedf95a19c873cc014f08af3d17f73f" ]
Each of the
refspec points to the d2-ci repo of the
application which stores all the
build artifacts for apps:
E.g. for the Data Quality App the build included is: https://github.com/d2-ci/data-quality-app/commits/3166be606054bb7f5a979c041048450d97e1e6fe
What is the source commit for a given application build?
apps-bundle.json file references the commit of the build artifact. This
is useful for referencing specific builds, but as a developer you might be more
interested in the source code commit from which the build was created.
Each application exposes a
BUILD_INFO file which is available through the
The commit hash in
BUILD_INFO points to the source repo in the DHIS2 organisation:
Fri Nov 9 23:19:00 UTC 2018 fb596620b6e5a6d5af730c793baffd423ecb23d8
Create custom builds
To change which apps and respective versions of the apps are bundled, a single file needs to be modified in dhis2-core:
This is a JSON file which contains an array with Git repositories. The
build scripts for
dhis-web-apps then use Git to clone the respective
element in the list.
The Git repos we use to bundle applications is a build artifact repository. For every app repo on the github.com/dhis2 org, we have a build repo on github.com/d2-ci to store the build artifacts for each build (one per commit).
https://is used for the apps to avoid the build environment having a git+ssh set up. If you know that you can clone from a
git://URL then that is fine to use as well.
For each of the referenced Git URLs it is possible to specify what tree-ish should be used for the build. The syntax for that is:
- Can be a commit:
- Or a branch:
- Or a tag:
Some example use cases below:
Replicate a specific build
Say that you are running 2.31.0 in production today, and you know that the applications in that version are good. Then version 2.31.1 is released and the pre-built WAR-file comes with application updates which you do not want. This might be due to your test procedures of having to do a full regression test on all updated applications.
So you decide that you want to keep the applications versions in their known good state and see if you can update only the core from 2.31.0 to 2.31.1.
First you download the https://play.dhis2.org/2.31.0/dhis-web-apps/apps-bundle.json file, which includes the exact build artifact refspecs for the applications which were bundled with the WAR-file when it was originally built.
Then checkout the branch for 2.31.1 in the
dhis2-core repository and put
the contents of
dhis2-core/dhis-2/dhis-web/dhis-web-apps to bundle the specified application
Execute the build and you will get a 2.31.1 build with the applications you had in your 2.31.0 instance.
Customize your build
Remember the syntax outlined above:
Here is a deeper dive into how to use the part following the hash (
Default: use master branch
#<treeish> is left out, then the default behaviour is to fetch the
latest build from the
[ "https://github.com/d2-ci/dashboards-app", ]
#master to the app URL to do so explicitly.
[ "https://github.com/d2-ci/dashboards-app#master", ]
Use arbitrary branches
To test the latest build from a branch append it to the URL after the hash
[ "https://github.com/d2-ci/dashboards-app#v31", ]
This allows your build to track the latest build from a specific branch, and is what we do for our development environments, for example. It is also handy when you are working on a feature branch and want to be able to deploy it for testing/verification/sharing, etc.
Lock to a specific commit
Sometimes there is a need to lock the core to track a specific commit, either to reproduce a build or to lock the build to a known good commit while a broken application is being fixed to stop the broken build from being put into a production build.
Any tree-ish will do after the hash (
#), so a commit SHA is also fine:
[ "https://github.com/d2-ci/dashboards-app#5afaf70e8b7e427bc064fa025610eea2c0e195e5", ]
Lock to a tag
A tag also works as a tree-ish, so the formal DHIS2 releases track an application tag, so the 2.31.0 version would track the 2.31.1 tags of the applications.
[ "https://github.com/d2-ci/dashboards-app#2.31.1", ]
Control when to fetch new apps
It is important to note that we try to align as close as possible with
how Maven works. When the
dhis-web-apps module is built it pulls in
all the web applications defined in
apps-to-bundle.json into the
dhis-web-apps/target directory. This is the default behavior to ensure
that the build system is built using the correct artifacts.
This allows Maven to have control over when the directory is due for a clean up.
This also means that if you use
mvn clean install, the
dhis-web-apps/target directory is going to be removed and all apps
are refreshed from d2-ci!
It is optimized to perform as shallow a clone as is possible, but in some cases, it needs to fetch additional commits to find a specific one.
In general, using a branch or tag is fast, using a specific commit is slow.
In offline scenarios, there are two strategies to avoid downloading apps, depending on your needs.
Strategy One: Use
mvn install if possible
If it is possible to do an incremental build, the scripts in
dhis-web-apps will first check if the apps that are necessary are in
dhis-web-apps/target, and if they are, will use those for your bundle.
It is the
clean that causes the
target directory to be wiped, so if
you avoid that, you will reuse the cached apps.
Strategy Two: Use the ignore module flag:
Sometimes it is unavoidable that you must use
mvn clean install, and
if you still do not want to refresh the apps there is an alternative.
For example to do a
clean install in
dhis-web, except for
mvn clean install -Pdev -f dhis-web/pom.xml -pl !dhis-web-apps
By now you should have an idea of how the build system operates and some tricks you can pull off with it.
Want a minimal
dhis2-core build? Remove all the apps from the
Want to share a single app with a feature branch? Create a build with a single app referencing that feature branch!
And so on.
As a final note, this build system is available for DHIS2 from 2.29 going forward.
Hope this was helpful to read and reach out if you have any questions about it.