We sometimes get questions asking about mirrors or people uploading the OpenGApps.org pre-built packages to mirrors, which violates our license terms. In this post we will try to give some insight behind the rationale of the no-mirror policy of the OpenGApps.org pre-built packages:
Technical merit
The very first reason it technical: we want users to use the latest, greatest and most up-to-date version. The rationale behind the project was to be able to do automated builds to achieve this goal and it would be weird to forfeit this. To make this easy, we made it possible to hotlink and automatically download the latest versions for e.g. ROM devs their OPs on XDA. This eliminates the need for mirrors which would be outdated within one day.
Support and educate users
The second reason is support and documentation. If just uploading the package to a filehost it isn’t accompanied by any documentation or access to a support forum. If offered via OpenGApps.org there are direct links to the Wiki, XDA Forum, the source code and existing bug reports. This helps users find their way and to benefit most from the various features offered by the package.
Fostering community
Another very import reason is fostering a community. Open source does not mean “just download and ignore where it comes from”. For open source projects it is critical that people are aware of where their software originates from, the rationale behind it, how to contribute and how to get involved. If users of the software can’t get acquainted with the project, the project won’t be able to attract new potential volunteers and would have a large chance to die. OpenGApps.org is a HUB for the users of Open GApps, that enables them to get acquainted with the project and support it.
Inclusive to all
Also this policy benefits the people who are using less popular architectures, apis and variants. The amount of ARM 5.1 and 6.0 downloads each day is huge, but some people use less popular builds, like 4.4 x86, that only has a few downloads each day. To be able to compile builds every day for everyone instead of just the popular few packages, the buildbot compiles and uploads more than 20GiB of new packages every day. That is not without costs and the ads and donations on opengapps.org support us to do this. If you download an ARM 6.0 pico on our website and you see an advertisement, you are i.e. funding the 4.4 x86 users’ download.
Acknowledge, credit and motivation
Another aspect of open source projects and voluntary work is that it is important to give credit where it is due and respecting people’s efforts: volunteering projects can only strive because of the many people involved that contribute with their knowledge and skills. If filehosts offer the downloads without any reference to the context to the people who made it possible, it neglects this important aspect of keeping people motivated to put in the many hours they do. That is why direct access to the e.g. the source code commits is important.
Trust and security
Trustworthiness, the packages from OpenGApps.org are signed with a custom certificate, to ensure people that they were built by us. People can choose to trust OpenGApps.org’s packages to be signed with this certificate had no changes compared to the GitHub source code and don’t contain APKs that were tampered with.
What if downloads are slow?
Sometimes people experience slow downloads from OpenGApps.org. The OpenGApps.org packages are hosted on GitHub Releases, which is hosted on the Amazon Web Services Cloud. Most of the time and in most places of the world their service is good and reliable, but for some users it is sometimes slow. If it is slow, you can write to GitHub to let them ask Amazon for a solution.
Compile; don’t copy!
If you want to upload your own Open GApps variant, you are free to do so. The source code of Open GApps is freely available and everybody can compile and upload their own builds and variants, which you can distribute under various terms (as long as in compliance with the Open GApps license). Please just don’t use copy-paste the pre-built ones from OpenGApps.org.
With Android Marshmallow Google released a new version “3” of their Google Camera app. This version of camera comes pre-bundled with most of their stock Marshmallow ROMs.
Marshmallow
The updated application APK states that it requires SDK level 23 (which means Marshmallow 6.0) as minimal android level for installation. This result in the Open GApps package-building scripts to automatically pick the v3 of the Google Camera when creating installable zips for 6.0. Because the APK is deemed compatible.
Incompatibilities
Unfortunately many people did experience issues with Google Camera v3 on Marshmallow when recording video. Apparently with the introduction of the new Google Camera v3 it assumes certain device and firmware capabilities that many devices and ROMs cannot offer. This posed a problem for the Open GApps packages. E.g. if we would bundle the v2 version on Marshmallow (which would need an override in the package-building scripts), the people with a v3 compatible device would directly get an (unexpected) update from the Play store offered. But if we bundled v3, many people would experience video recording problems.
We did need to find a solution for this problem, that would provide Google Camera v2 for devices that were not v3-compatible, and v3 for the compatible ones.
Determining compatibility indicator(s)
First challenge is: which devices are actually v3-compatible? We looked at the Android Source code and Android Camera documentation, but any version information seems to be only available within the OS itself. E.g. the Google Play Services is able to assess and communicate the device’s information and capabilities to the Play Store. Based on that information it can decide to offer the device the Google Camera v3 update or not. But we, from the device’s recovery, have no access to this information (yet). We looked at the camera libraries in the ROM and the symbols it exposes, but we have not found any reliable indicator. So we are forced to work with a white-list with v3 compatible devices. If you know your device is compatible, but it isn’t on the white-list yet, please let us know or create a pull request with the devicename added.
Including new and legacy Google Camera
The second challenge: how to include both the v2 and v3 Google Camera and let only the compatible version be installed. For this we added functionality to the inc.buildhelper.sh that allows to set a maximum SDK level when looking for an APK to package. After that we created an extra entry in the inc.buildtarget.sh script called cameragooglelegacy with a maximum SDK level of 22. Next step was to add “cameragooglelegacy” as extra package-building rule for the 6.0 stock and super packages. In the update-binary installer script there are some logic checks that decides if “cameragoogle” should be installed or not (based on keywords used and your device’s prerequisites). An extra piece of logic was added, that if cameragoogle is going to be installed, but your device is not on the white-list, it will use cameragooglelegacy instead.
Adding the extra Camera Google v2 to the 6.0 packages takes around 20MiB extra. So the download is a bit larger than before. Let’s hope that when Android N is released this situation to be improved, so that this trick won’t be necessary anymore.
If you do have any ideas how to figure out if a device is Google Camera v3 compatible from the recovery without using a white-list, please let us know in the comments or on the XDA forum!
Happy Open GApps user? Vote for us at ProductHunt!
The title might sound cryptic to most readers, but for most GApps developers 2016 did start with a lot of bug reports from CM Recovery users. Reports with errors like “I got error 255” while we don’t have such error code specified ourselves, or “mount failed with error 1”. Cause of all this trouble is a new command line tool called Toybox and the decision of Cyanogenmod that the Christmas holidays were the perfect moment to embed it in their recovery instead of Busybox.
Most users will probably wonder: what is Toybox and why did Cyanogenmod switch to it and what was Busybox in the first place?
Busybox
Let’s start with the last question first: what is Busybox? Busybox is a GPL-licensed effort that offers many Unix tools (commands that offer functionality like “copy this file”, “search for this string”, “mount this partition”) in a single small binary and already exists since 1999. The concept of it is great, it offers a lot of functionality, comparable to an operating system, but within a small 2MiB binary. As such Busybox is perfect for use in embedded software or, more applicable to our situation, the Android recovery. That is why TWRP, CWM and until recently CM Recovery were all using Busybox as core provider of functionality. It is a powerful tool that enabled us to use advanced logic within the Open GApps installer.
Toybox
But what is Toybox? Toybox’s goal and functionality are meant to be just like Busybox and it is even being developed by the very same person that maintained Busybox for many years (Rob Landley). But where Busybox is licensed as GPLv2, Toybox is using the BSD license. Because Google has chosen for a policy to not include any GPL in Android’s user space, it disqualified Busybox to be included in Android’s upstream AOSP recovery repository. But Toybox does have a compatible license which enables it to be upstreamed into AOSP. This is in general a very positive development, because a more powerful recovery in AOSP is very welcome.
Toybox is not yet finished though (see the current status here). While the original recovery in the AOSP project had barely any capabilities, for them the adaptation of the still under-development Toybox is a huge step forward. But the alternative recoveries were using the powerful Busybox. CM Recovery decided to switch from Busybox to Toybox. But because Toybox still does miss some essential commands that we rely on in the Open GApps installer (unzip, awk, xz) that are not implemented (yet). This breaks the Open GApps installer on CM Recoveries that are bundled with the CM13 nightlies.
CM Recovery
Why did Cyanogenmod switch? We don’t know. Of course it convenient to stay as close as possible to upstream as possible, but in our opinion Cyanogenmod has switched way too early from Busybox to Toybox for their recovery. As Open GApps we are very unhappy with the timing of their decision. We believe that at least the posix subset should have been fully implemented in Toybox before they should have switched. As Open GApps we now have the choice whether
- we will rewrite our code to the limited Toybox capabilities (if possible at all),
- defy CM Recovery as supportable effort,
- bundle our own Busybox in the installer,
- hope that Toybox will develop the missing commands at lightspeed,
- or that Cyanogenmod reconsiders their decision.
UPDATE: we went with option 3
AROMA Open Gapps (previously known as AROMA LP-GAPPS) was created at the beginning of 2015. What triggered its creation, you ask?
AROMA LP-GAPPS
From a simple discussion between two friends about how awesome it would be, if you could only install those GApps that you really want in a simple manner. Because the standard packages (nano, pico, mini, stock, etc.) didn’t satisfy our needs. This sparked the idea for AROMA LP-GAPPS and raulx222 started to develop a config for AROMA-Installer and made a package based on PA GApps.
Joining Open GApps
On 1st of June, LP-GAPPS was further integrated with the Open GApps project, a new GApps project that used the same (keyword) install mechanism from the, at that point discontinued, PA GApps. This was a new beginning for LP-GAPPS and the best thing that happened to the project.
The main benefit of integration with the Open GApps project is that the AROMA packages were now build everyday containing the latest GApps, compared to the old situation where the packages were the AROMA-scripts had to be repackaged manually every release.
AROMA Installer
Now let’s talk more about AROMA Installer, an essential ingredient for the project. What is AROMA Installer?
“AROMA” was taken from Bahasa Indonesia (Indonesian Language) and it means “Scent”, but it is also an abbreviation of “AMARULLZ ANDROID ROM MANIFESTATION”. It is an advanced update-binary for Android Recovery. It contains many features, like Wizard Installation, Touch User Interface (AROMA UI), Customizable Packages, System Inspecting, Themeable, and User Interactive. - amarullz (Ahmad Amarullah)
AROMA Installer was created somewhere in 2012. This piece of software got very popular very fast and many ROMs developers were using it and still use it, because it is simply unique and very powerful. It does let users customize their installations based on their preferences through a very attractive GUI within the recovery environment.
AROMA maintenance
Somewhere in 2013 the development did almost stop completely, but AROMA Installer still works on most devices with ARM architecture.
Since then the smartphone industry also developed devices with other architectures, like x86, which are not compatible with the current AROMA Installer. But even not all ARM devices are compatible with AROMA Installer, because of various issues. But we believe these could very possibly be fixed through further development.
Lib AROMA
Since 2013 amarullz started to develop his “aromalib” which is the core component of AROMA Installer. So there’s a newer, modified aromalib of which amarullz confirmed that it would be used in the next update of AROMA Installer. In May 2015 amarullz uploaded a video with a test application of libaroma on TWRP, you can see the video here. But development of aromalib seems to go slow, which worries us as AROMA Installer users.
All we can do is to encourage amarullz to keep working on his powerful aromalib and help him to release a new update! If you appreciate his work and want to encourage further AROMA Installer development you can donate to amarullz and you can check out his other work.
AROMA Installer future
We hope that we can receive an updated AROMA Installer in the near future. But of course we did not stop our own development and hacking around AROMA’s current limitations. Sometimes we faced issues when implementing new features, because they did not work properly in the aroma package. But we managed to fix them! For example we were obliged to compile and bundle our own xzdec (for decompressing xz files on ARM devices) in order to decompress xz files within AROMA Installer. Without our own xzdec tool, using recovery’s built-in decompression tool, the AROMA Installer would failing during decompression, probably because of memory issues.
AROMA Open GApps future
In the future we are planning to make the user interface of the aroma package more dynamic per Android version. E.g. not all keywords are available in the various aroma packages. With the new Super package we did expand our keywords to cover the newest app additions. But the aroma package for 4.4 and 5.0 are still based on the stock package, because super is only available for 5.1 and 6.0. The current aroma config is still shared between these various versions and does contain all the keywords, even when they are not available. When we will make the interface more dynamic, the “unavailable keywords” can be hidden on Android versions where they are not applicable.