Opengapps.org’s daily builds are generated by our own buildbot and served via GitHub’s releases feature to the end-users. The buildbot is hosted on mfonville’s server and pulls in all the latest Open GApps git commits at midnight UTC.
Trigger
After having pulled in all the latest commits the buildbot uses the report_sources function to generate a hash for each architecture. The buildbot checks if the hash has changed, indicating that there have been significant updates for the architecture. If these exist, the buildbot will start to create new builds.
Build
The builds are forged at the server and are build using some specific characteristics. All builds by the buildbot are signed with a specific OpenGApps.org signature. During the build process a buildlog is generated, that specifies all APKs and libraries used, which can be downloaded by the users when they click the version information button at opengapps.org. All also builds have their md5 checksum calculated, also downloadable by the users, that can be used by the recovery to verify the integrity of the Open GApps file downloaded. The buildbot also generates a generic sources report that lists all APKs that were that day available in the repository for that architecture and Android version.
Upload
The builds and other files are then uploaded to GitHub. This process is automated, but not as smooth as we would like. GitHub’s servers regularly give an error 502 during the upload process. Which GitHub still has not been able to resolve (even though multiple users are writing to them about it for a year). Such an error results in an incomplete upload and blocks when retrying to upload a file with the same file name. Thus the buildbot must detect which uploads did fail and remove those from GitHub. This was something we had to do manually in the past. But luckily, we were able to find an automated solution recently. It was necessary to list the assets per release id to list these incomplete uploads, while successful uploads where listed in the generic list of releases. Also we had to work around GitHub’s API pagination because it was initially not showing all uploaded assets.
Download
GitHub’s API is accessible with javascript’s JSON, a feature that we use for opengapps.org When you visit the site and select an architecture, an AJAX request is send to the GitHub API to query for the latest release of the respective architecture. Basic information about the selected package is also directly fetched using this query and is shown in the dialog. The download button directly forwards to GitHub’s release infrastructure that hosts the file.
Issues
One current issue is that GitHub’s latest version detection mechanism is unfortunately not without faults. If multiple releases/tags are based on the same git commit, GitHub prefers to mark the lowest version number as the latest. This is of course illogical and we’ve written to them about it a couple of times, but without any solution yet. Luckily this doesn’t impact the more popular architectures, that almost always have new commits and thus no shared tags, but sometimes x86_64 doesn’t show the factual latest release. You can always click at older versions to open the GitHub releases page to review if there are newer releases.