Finally a post about Marshmallow!
No builds yet
Most important information first: we don’t provide any stable builds for Marshmallow yet, in the cause of that is because Marshmallow poses a couple of challenges that have not been properly resolved yet. We only want to offer pre-built Open GApps packages when these issues are resolved.
So, what are these challenges you ask? We will highlight a few:
- Marshmallow comes with several new core applications. Our scripts had to be updated to package and install these applications too. This has been done and implemented already in our code.
- Google’s Marshmallow images have for some applications no pre-extracted libraries. They read the libraries from within the APK. For some reason AOSP ROMs do not always accept libraries this way, and still demand pre-extracted libraries. We still don’t know why they differ in behaviour, but this is something that should be resolved by the ROMs and consistent behaviour should be defined.
- Google finally fixed the security mechanisms for apps on
/system/partition and demands all APKs to be properly signed and verified. This does mean that we cannot remove the libraries from the APK, or it would not pass this verification anymore. In combination with the previous point, this would mean that all libraries would take twice the system space, if they would have to be both within the APK next to being pre-extracted.
- Only applications that are signed with the platform key are automatically granted all permissions on
/system/. All other applications have to request these permissions. On Google’s stock images this is no problem, because both the platform as the Google Apps are signed by Google. But on AOSP ROMs, the ROM is not signed by Google, resulting in that the Google Apps don’t get their permissions automatically assigned.
- Which leads to our following point, that the core Google Apps assume that they have already been granted all permissions, and if they don’t have the permissions, they just crash and give constant FCs on the screen while the device is booting.
- The only ‘solution’ to work around this problem for now, is not bundling some core applications like SetupWizard. Which is something that in our opinion is not good enough of a workaround for pre-built packages to the public.
- The real solution would be to patches in the AOSP ROMs to also automatically grant permissions to Google Apps too, or e.g. use an init script that can interact with the Android Shell to perform the granting of the permissions automatically.
- But we reasoned that we are not alone in this problem. So we are waiting for some of the ‘good’ vendors that play nice with open source, like Sony, to release the source code of their changes to AOSP and to check how they chose to work around this problem. Because also they don’t have access to Google’s platform key, but they do have to bundle these Google Apps with their ROM.
The very nice thing about these problems though, is that it gives more unity to the GApps devs community and most of us are working together and sharing our knowledge and progress. So kudos to dankoman, benzo and others for their efforts.
What, not a post about Marshmallow? We know that all of you want to hear about Marshmallow and GApps support for it, and yes, we are busy with it. But it is not the time to blog about it yet. So instead, today a small post about one of the unique features of Open GApps: full multi DPI support.
Multi DPI support was already available in PA GApps, but only for a very limited set of applications: Google Play Services and Google Play Games. But throughout time, Google has released more applications that have DPI-specific versions. First, many organizations and packagers were unaware of this trend. E.g. APK-mirror would show you the very latest APK of an app, but unknowingly of its DPI. Luckily they also did spot our pioneering work on aggregating as many DPI-variations of each APK as possible, so nowadays you can find the DPI in the title of many of the APKs posted (and they also followed suit with our nodpi classification ;-) ). Many of the more rare DPI-specific versions are now regularly contributed by the Open GApps packagers.
Best APK for your device
Having these DPI-variations of each APK posed an issue for GApps packagers. Which version to bundle? Some APKs have a universal DPI version (but not all!), meaning that they contain the graphic resources for all DPI variations. Those are practical to package, because they are suitable for all devices. But having a lower version number then the DPI-specific versions, most users will receive an update to their DPI-specific version via the Play Store. The Play Store saves the APK on the
/data/ partition, and the advantage of having the GApp installed on the
/system/ partition in the first place is gone. Also, this slows down the device’s boot process, because more APKs have to be processed by the Android package manager. Another solution for a GApps packager would be to bundle the highest DPI version. That version will have the highest version number, so users won’t get DPI-specific upgrades from the Play Store offered. Problem is though, that the resources are not optimized for the device. This can lead to distorted graphics, but also slows down the Android Launcher, that has to resize the icons, the app itself, that has to resize all the graphics, and leads to increased CPU and/or GPU use, leading to more heat and lower battery life. Though impact in practice is not that high, knowing that it can be better is for people that are looking for the best ROM also a reason to look for the best GApps implementation concerning this issue.
Package them all
That is why Open GApps bundles all DPI-specific versions (if available) in its package. Open GApps tries to install the best DPI package for your system. The exact matching if possible, and otherwise it falls back on a universal version or as last resort a version close to your DPI. This does mean that the Open GApps package became huge. E.g. PA GApps was already one of the largest GApps packages around with stock’ being more than 500MiB. When we started to add DPI-specific content the stock package started to grow out over more than 1GiB and the latest Open GApps stock release contains more than 2GiB of APKs.
But luckily we were able to reduce the size of the package to around 500MiB again by using smart compression techniques. All APKs of the same application are stored within a common XZ-archive. The XZ-compression is able to detect similarities between these APKs. Since the graphics resources tend to only take up to 10% of the APK, and the other 90% is the same, a very high compression can be achieved. For each extra DPI-specific variation, the XZ-archive only needs a few 100KiB more. To unpack these XZ archives, a XZ-archiver is necessary. The latest TWRP recoveries already contain these, and for as long as CWM is still around, we have bundled an XZ-decompressor within the package. Decompressing the XZ archives takes a bit more time, that is why OpenGApps installation takes quite long, but you’ll know that afterwards you’ll have the fastest GApps around!
OpenGApps is a relatively young project. If you check our domain information, you’ll see we registered opengapps.org at June 27th 2015, so that is only 3 months ago! But the project has been received quite well by the community, and each day we keep attracting more people to our website, and more stargazers at GitHub.
Currently we have around 12.000 daily visitors at opengapps.org, in the weekends even more. Around 45% of those go directly to our website, so probably our loyal group of flashaholics who bookmarked the website :-) The other visitors are referred to us from a varied set of Android related websites. As to be expected, XDA Forums is the largest, but also many non-English forums and various ROM hosting websites refer their visitors to Open GApps.
Our number of daily downloads is even a little bit higher than the amount of visitors on opengapps.org, because some users (e.g. those using Pushbullet) directly browse to our releases section on GitHub, without ever going through our website.
We gathered some download statistics over the period of 17th of July to the 23rd of September and created some graphs to give more insight into which packages are most popular.
First we can have a look at the most popular platforms. Unsurprisingly ARM is the most popular, taking around 91.3% market share. But is interesting to see that ARM64 is already 6.1%, while the amount of 64 bit devices is relatively limited. With the new set of high end Android devices to be announced this fall we expect this number to increase.
Another interesting graph is the set of Android versions that are chosen. 5.1 is clearly dominating, with most custom ROMs being based on the latest Lollipop release. What surprised us though, is the amount of 5.0 downloads. With 6.7% the market share is not large, but with Google itself having released 5.1 very quickly after 5.0, to fix some serious issues, it is surprising to see that any ROM is still 5.0 based. Especially considering that CyanogenMod’s 12 (=5.0) branch did also never really mature before switching to 12.1 (=5.1) and e.g. lacks important (security) fixes.
Our final graph is the set of variants chosen. The discontinued fornexus variant is left out of the graph since its functionality is now embedded into the other variants (and it had less than 1% anyway). As you can see the stock package is the most popular, probably helped by being the default choice when visiting opengapps.org without any preset parameters. But all packages have quite some serious share, which shows that there is demand for all the flavours. Not shown in the graph, but important to notice, is that the variant preference varies by Android version. E.g. the devices with Android 4.4 tend to have a small
/system/ partition, resulting in the smaller packages being more popular. While for 5.1 the larger packages like stock and aroma are popular choices.
In normal ‘educated’ computing practice, no one in their sane mind would download a random executable-file from some unknown person on the internet, grant it all administrator rights to your system and happily keep using the device for banking and other confidential and trusted purposes.
But still, this is what almost everybody with a Custom ROM does… First they download a ROM created by somebody on XDA and put it on their device. Then they download a GApps package, probably also from XDA, and again they just install it on their device. Then they set up their Google account, install their banking app etcetera. No one seems to worry about possible security vulnerabilities in that process, let alone possible malicious intents…
Luckily, for the ROM there are already relatively good solutions in place. Most ROM developers publicize their development work in a Git repository already. One can review their code, clone the repository and compile their own ROM from these sources. Creating their own safe-to-install version, that can be trusted not to have malicious content (only if you have the time to check all the code, though).
With the GApps, that is more difficult. First of all, the GApps themselves are not open source, neither will they probably ever be. But at least most people have good faith in Google and trust them not put some malicious code within their applications. But still, how could one know that an app in a GApps package is really from Google?
System APK Verification
First the bad news: The Android platform itself will not help you in verifying an APK that is going to be installed on
/system/. None of the applications that is installed on
/system/ will be verified for having a correct signature. That is why one should always be very careful when flashing any ZIP package from recovery.
Open Source GApps Packaging
So how could one generate a trusted GApps package? Well, key is that the generation process of the GApps package has to be open and reproducible, which is a key aspect of our project. Because only if you can perform (or fully check) all exact steps yourself, one can trust the outcome. We at Open GApps care about this and therefore trying to give you the tools to do so. In the meantime, we also try to improve our own build process to have more safeguards for the integrity of our packages. We try to achieve this by integrating more automated checks for certificate and integrity checking of the APKs that are used in our source repositories and GApps packages.
As second the good news: a simple cookbook to a safe GApps Package:
- Download a trusted Google Nexus Image and extract a Google APK from it, or download a trusted Google Application from the Google Play Store.
META-INF/CERT.RSAfrom the APK, extract the key that is used for this certificate (e.g. command:
openssl pkcs7 -inform DER -print_certs -text) and store it in a keystore with the
- Download the Open GApps source from GitHub and review the code. Also review the code that is used from third-party tools, like the SignApk.jar and if necessary compile them yourself or choose not to use them.
- Adapt the Open GApps build process to only let it use the keystore you created earlier.
- Check if the certificates of the APKs that will be used to build your GApps package all match the keystore you created earlier.
- Run the
makecommand and create your own trusted GApps.
By completing these steps, you have your own trusted GApps package (as long as you trust Google).
We are still busy trying to improve in this field, if you are interested, please take a look at our tracking issue #92
OpenGApps.org currently enjoys a nice Material Design Lite layout. But it did take some effort to get there…
Material Design Lite
The moment Google released their new Material Design Lite Project and mfonville and raulx222 got directly interested, the toolset was simple and it allowed to adapt a simple HTML page into a full Material Design compliant website with near-native look and feel for a webapp easily.
We decided to go live at one moment though, because it was good enough of an improvement over the initial site and we have received a lot of positive feedback.
We will keep adding small improvements to the site, especially when Material Design Lite gets updated. We especially look forward to their 1.1 update which is planned for October, which will include even more Material Design components like Snackbars and Toasts, which we plan to use for e.g. the obligatory cookies warning.
I hope you guys like the website and we have a small tip for you: If you open the website in Chrome on your Android Device, you can add the website to your homescreen. When doing so, the website will behave completely like a native Material Design Android app. In the future we will expand the features of this app, when technically possible (there are still some GitHub limitations) with e.g. push messages for updates.