Good news for those who are still using Android Lollipop and do like Google Calculator!
AOSP app replacements by Google
Google started in recent years to replace various apps that used to be completely in the AOSP project with their own proprietary spins on Nexus and Android One devices. This started with Lollipop (5.X) for Google’s WebView, Google Contacts, Google Dialer, Google Clock and Google’s NFC Tag apps. With Android Marshmallow (6.0) this was also extended to the Google Calculator app.
Google Calculator minimal API level
When the first spins of Google Calculator were released with the Nexus devices of 2015, the APK for Google Calculator were compiled to demand at minimum API level 23, that means Android 6.0. So Open GApps could only offer to install the Google Calculator in the 6.0 packages. Last week Google released an updated version of the APK in preparation for Android Nougat (7.0) and at the same time decided to lower the requirement to API level 21. We don’t know why they lowered the requirement, because in general compiling for a lower API level means that the code has to include more compatibility-hacks. But we tested this new APK on Android (5.X) devices and it works without problems. That is why you can now find Google Calculator also in the (5.0) and (5.1) Open GApps packages (mini and larger).
Because of health issues development on the scripts has been a bit slow recently, sorry for that. Nevertheless it was time to blog, so today a small update and a link to the Pokémon Go APK.
Build system overhaul
I have still have some major architectural overhauls for the Open GApps build system queued, but I need more time to fix some last issues with it. When these changes are done, the build system will be much faster and use less disk space during builds, and prepares the way to multi-architecture/dynamic packages.
Moar and moar APKs
As could be read in our previous blogpost our APKCrawler project got some big updates and we are now crawling more intensively than ever before for new APKs. x86_64 tend to also be much more complete now. Also we updated the playstorecrawler to be able to find updates for the Play Store itself directly, a mechanism that is a bit different than regular APKs.
What is up with TVStock?
We are still busy in the process of preparing a package for Android TV. It is not there yet (and first the build system overhaul has to be done). So no ETA yet.
GPG signed commits
Recently I switched to a new GPG key, because my old one was from 2005 and could not be considered safe anymore because of the low amount of bits. Created a new one now and applied the necessary good practice (masterkey offline, various subkeys for daily use). Still in the process of the switch and propagation of the new key. But I hope to include verified commits into our development workflow soon.
Pokémon Go
Pokémon Go is the biggest application hit ever, it seems. Even the devs are messing around with it (we can tell you it uses Google’s protobuf)! Unfortunately eligibility is geo-location restricted, but using APKCrawler we could get the APK straight from the Play Store! You can download it here There is only an arm version at the moment, so those who have x86 devices might encounter low performance (or it not working at all). Let’s hope Nintendo releases a multi-architecture version soon!
Notice the Pokemon Go APK has been updated several times already and we will try to keep track of the most recent version
Today a blog article with an interview with Nick Buse, long term contributor to Open GApps and the originator of the APKCrawler project.
When did you start the APKCrawler project?
First commit was on August 5, 2015, but some prototyping began in July 2015
Why did you start the project?
Between the start of the Open GApps project in March 2015 and August 2015 mfonville had compiled a long list of sites to check for new and updated APK files. When I first saw mfonville’s keyboard (below), I knew something needed to be done to automate the process.
What was the first crawler you have written?
The obvious choice then (and still is now) is APKMirror.com. The have the most complete and accessible list of variants (cpu, dpi, sdk) for all APKs. This is probably due to their crowd sourced upload functionality.
What is the most useful crawler?
The AptoideCrawler. It is the #1 source of APKs that we find and then re-contribute to APKMirror! A close second is the new PlayStoreCrawler that mfonville and therealssj have been working on. There is no more reliable source of APKs than the Google Play Store!
What are the benefits?
mfonville has been using the same keyboard for 6 months now, so that is a huge cost savings for him! Seriously though, any software person knows the benefits of automation. Now we can focus family and friends!
Any challenges or issues?
When a site does not provide a usable API, we rely on the Python module BeautifulSoup to parse the site’s DOM to get the APK’s information as well and download the APK itself. The obvious drawback is when sites go through a redesign. Even with an API in place they break from time to time. One of Aptoide’s official API functions is already broken for months now. That is when I found an undocumented API and rewrote the crawler to be much more efficient. As of today it still kicks ass!
Where do we get more info about this cool thing of yours?
Check out our GitHub repository for the latest development (contributions are welcome!)
Opengapps.org’s daily builds are generated by our own buildbot and served via GitHub’s releases feature to the end-users. The buildbot is hosted on mfonville’s server and pulls in all the latest Open GApps git commits at midnight UTC.
Trigger
After having pulled in all the latest commits the buildbot uses the report_sources function to generate a hash for each architecture. The buildbot checks if the hash has changed, indicating that there have been significant updates for the architecture. If these exist, the buildbot will start to create new builds.
Build
The builds are forged at the server and are build using some specific characteristics. All builds by the buildbot are signed with a specific OpenGApps.org signature. During the build process a buildlog is generated, that specifies all APKs and libraries used, which can be downloaded by the users when they click the version information button at opengapps.org. All also builds have their md5 checksum calculated, also downloadable by the users, that can be used by the recovery to verify the integrity of the Open GApps file downloaded. The buildbot also generates a generic sources report that lists all APKs that were that day available in the repository for that architecture and Android version.
Upload
The builds and other files are then uploaded to GitHub. This process is automated, but not as smooth as we would like. GitHub’s servers regularly give an error 502 during the upload process. Which GitHub still has not been able to resolve (even though multiple users are writing to them about it for a year). Such an error results in an incomplete upload and blocks when retrying to upload a file with the same file name. Thus the buildbot must detect which uploads did fail and remove those from GitHub. This was something we had to do manually in the past. But luckily, we were able to find an automated solution recently. It was necessary to list the assets per release id to list these incomplete uploads, while successful uploads where listed in the generic list of releases. Also we had to work around GitHub’s API pagination because it was initially not showing all uploaded assets.
Download
GitHub’s API is accessible with javascript’s JSON, a feature that we use for opengapps.org When you visit the site and select an architecture, an AJAX request is send to the GitHub API to query for the latest release of the respective architecture. Basic information about the selected package is also directly fetched using this query and is shown in the dialog. The download button directly forwards to GitHub’s release infrastructure that hosts the file.
Issues
One current issue is that GitHub’s latest version detection mechanism is unfortunately not without faults. If multiple releases/tags are based on the same git commit, GitHub prefers to mark the lowest version number as the latest. This is of course illogical and we’ve written to them about it a couple of times, but without any solution yet. Luckily this doesn’t impact the more popular architectures, that almost always have new commits and thus no shared tags, but sometimes x86_64 doesn’t show the factual latest release. You can always click at older versions to open the GitHub releases page to review if there are newer releases.
Google enabled now the Google Dialer for many more phones in the Play Store so we will be abandon the whitelist that was mentioned in the initial Google Dialer blogpost. This means that from now on, Google Dialer framework will be installed by default on all phones. The Dialer can subsequently be installed from the Play Store, or it can directly be installed if using the Stock, Super or Aroma package.
You still need to set Google Dialer as the default Phone app manually, if you remove or disable the stock/AOSP dialer application, to prevent the phone from rebooting when making or receiving a phone call.
Instructions how to set the Google Dialer as default phone app
If you also want Google Dialer to be able to adjust your ringtone settings, you need to grant it access to your system settings. Otherwise you will receive a toast with a message it is not allowed to change those.
Instructions how to set grant Google Dialer permission to access system settings
The advisory to regularly flash an updated GApps package to update DialerFramework remains intact. It can only receive security updates if you update your Open GApps installation and not via the Play Store. Make sure to update your Open GApps installation at least once a month, after Google’s monthly security updates are added to the Open GApps repository.