Digging around in AOSP is a great way to make new discoveries about Android, and this time we've come across something rather hilarious. For some time, users have reported that the technology website TheVerge.com provided slow performance on mobile devices.
Now to their credit, their website performance has improved over time in my experience. Plus, it's not as if other sites (including our own) don't have issues we can strive to work on, but nevertheless I found it quite amusing that in its official set of workload benchmarks, Google decided to use The Verge in their testing.
Android Workload Automation
Workload Automation (WA) is a framework developed by ARM for collecting performance data on Android devices by executing a suite of many repeatable workloads. Google benchmarks performance on their devices by doing many of these workload tests and collecting a summary of power usage, which they then import into a spreadsheet to see how their optimizations have improved performance over time. The company picks and chooses which apps to include in its test suite, but in general they limit themselves to most of the popular Google apps. That's the gist of how it works, but we'll show the evidence from source code and describe the test suites in more detail so you can get a better picture of what automated tests Google does to measure performance.
Within AOSP, there is a directory dedicated to the workload automation tests. The apps that are used for testing are defined in defs.sh, and generally fall under one of two categories: default, pre-installed Google App or third-party web browser. One benchmark app stands out from the rest, and it's com.BrueComputing.SunTemple/com.epicgames.ue4.GameActivity which I assume refers to the BrueBench ST Reviewer benchmark which is based on the Unreal Engine 4.
# default activities. Can dynamically generate with -g. gmailActivity='http://ift.tt/1jbuT1y' clockActivity='com.google.android.deskclock/com.android.deskclock.DeskClock' hangoutsActivity='http://ift.tt/1rktmqC' chromeActivity='com.android.chrome/_not_used' contactsActivity='com.google.android.contacts/com.android.contacts.activities.PeopleActivity' youtubeActivity='com.google.android.youtube/com.google.android.apps.youtube.app.WatchWhileActivity' cameraActivity='com.google.android.GoogleCamera/com.android.camera.CameraActivity' playActivity='com.android.vending/com.google.android.finsky.activities.MainActivity' feedlyActivity='com.devhd.feedly/com.devhd.feedly.Main' photosActivity='com.google.android.apps.photos/com.google.android.apps.photos.home.HomeActivity' mapsActivity='http://ift.tt/1vjKlL1' calendarActivity='com.google.android.calendar/com.android.calendar.AllInOneActivity' earthActivity='com.google.earth/com.google.earth.EarthActivity' calculatorActivity='com.google.android.calculator/com.android.calculator2.Calculator' calculatorLActivity='com.android.calculator2/com.android.calculator2.Calculator' sheetsActivity='com.google.android.apps.docs.editors.sheets/com.google.android.apps.docs.app.NewMainProxyActivity' docsActivity='http://ift.tt/2cEZB09' operaActivity='com.opera.mini.native/com.opera.mini.android.Browser' firefoxActivity='org.mozilla.firefox/org.mozilla.firefox.App' suntempleActivity='com.BrueComputing.SunTemple/com.epicgames.ue4.GameActivity' homeActivity='com.google.android.googlequicksearchbox/com.google.android.launcher.GEL' These activities are launched via the ADB command line with the following Systrace options to measure app performance:
dflttracecategories="gfx input view am rs power sched freq idle load memreclaim" The Chrome app in particular is launched with a flag to load The Verge:
As for why the test seems to differ for the volantis (Nexus 9), I'm not exactly sure. Anyways, as for what tests this Chrome-activity-with-The-Verge actually goes through, we can determine by looking at the source code of the workload automation tests.
Test Suites
First up, there's the systemapps.sh test, which Google states works as such:
# Script to start a set of apps in order and then in each iteration # switch the focus to each one. For each iteration, the time to start # the app is reported as measured using atrace events and via am ThisTime. # The output also reports if applications are restarted (eg, killed by # LMK since previous iteration) or if there were any direct reclaim # events. Next, there's the recentfling.sh test, which works like this:
# Script to start a set of apps, switch to recents and fling it back and forth. # For each iteration, Total frames and janky frames are reported. And then there's chromefling.sh, which tests Chrome's performance rather simply:
# Script to start 3 chrome tabs, fling each of them, repeat # For each iteration, Total frames and janky frames are reported. Another amusing test in the Workload Automation suite, although unrelated to The Verge, is the youtube.sh performance test which measures UI jank
# # Script to play a john oliver youtube video N times. # For each iteration, Total frames and janky frames are reported. # # Options are described below. # iterations=10 app=youtube searchText="last week tonight with john oliver: online harassment" vidMinutes=15 Finally, each of these tests are used to measure real world power usage by cycling through them for a certain amount of time, as defined in pwrtest.sh:
# Script to gather perf and perf/watt data for several workloads # # Setup: # # - device connected to monsoon with USB passthrough enabled # - network enabled (baseline will be measured and subtracted # from results) (network needed for chrome, youtube tests) # - the device is rebooted after each test (can be inhibited # with "-r 0") # # Default behavior is to run each of the known workloads for # 30 minutes gathering both performance and power data. Google can then collect these data using pwrsummary.sh and import them into a spreadsheet:
# print summary of output generated by pwrtest.sh # # default results directories are <device>-<date>[-experiment]. By default # match any device and the year 201*. # # Examples: # # - show output for all bullhead tests in july 2015: # ./pwrsummary.sh -r "bh-201507*" # # - generate CSV file for import into spreadsheet: # ./pwrsummary.sh -o csv # These are all fairly common real-world UI performance tests, not unlike the kinds you would see in our own testing. It appears that the change to load The Verge's homepage when opening Chrome was rather recent, as before last year Google would only open a new tab in Chrome during these tests. A change made in May 28th, 2015 introduced the use of The Verge when testing Chrome, however. As amusing as it is that Google is using The Verge of all places when performing Workload Automation testing, keep in mind that this doesn't mean The Verge is the worst offender out there for web performance.
Far from it, in fact, as many other webpages suffer from mediocre performance thanks to the proliferation of more and more ads to compensate for the rise of ad-blockers. Indeed, it's most likely that the decision to use The Verge was simply one out of convenience, given how tech savvy the average Googler is and the inside joke among many enthusiasts regarding The Verge's webpage performance.
Google Apps on Android are a love-hate affair. Average consumers find the pre-loaded set of Google Apps to fit within their requirements enough to not worry about alternatives. But on the other hand, the apps themselves are not the best examples of how to create guidelines, or how to make apps either.
The previous statement becomes clearer when you expand your scope to look at Google Apps on other ecosystems, like iOS or on desktop platforms like Chrome OS. Apps made by Google themselves on these platforms feature a different design, and often offer functionality that the Android app gets days, weeks and months later (looking at you, Hangouts). Some Android apps, like Youtube, are missing useful features present on desktop. So we ask you,
What Features do Google Apps on Android sorely lack, when compared to the same set of Google Apps on other platforms? Which was the most frustrating discrepancy you noticed when you switched from using Google Apps on Android to other platforms?
Let us know in the comments below!
Samsung is satisfied with the progress they've made in the mobile VR space thanks to the Gear VR for the Galaxy S and Galaxy Note series. However, they do realize that the display technology isn't quite there yet and this is why they've been holding back on releasing a standalone VR headset. The company also says VR is at the peak of its hype phase, and they want to wait and see if the market matures.
In case you don't remember, Samsung confirmed they were working on a standalone VR headset back in April of this year. As with most products that big companies work on, not all of them will see the light of day. A lot of them are worked on to help progress the technology to a level where consumers will he happy with it. This is what we're currently seeing at Samsung with a VR headset that doesn't require a smartphone.
Samsung believes display technology needs to advance to at least twice the pixel density that we have in smartphones today. So it looks like the company is waiting and seeing how the experience of a standalone VR headset will be with Ultra HD display panels. Samsung's President & Chief Strategy Officer, Young Sohn, says this could be an incentive for the company to advance the technology faster, but it would cost them at least $5 to $10 billion to do so and develop a 10K mobile display.
As as we mentioned earlier, they just don't know if the VR market is worth that type of investment right now. They are willing to do so if the customers prove they are willing to buy into this technology, but they are worried the market will stagnate after the hype dies down. Sohn says it's "a bit of a chicken and egg problem right now," and would rather just wait and see how the market evolves before taking a big leap like that.
Following up on the launch of the Xiaomi Mi 5s in China, XDA sub forums for the device have been created! Talk and discuss all about the phone with other forum members!
A while back Miles did an in-depth review of the FreedomOS ROM on the OnePlus 3. If you didn't see that video, you can check it out here. In this video, we will follow up with this ROM and see how it has progressed so far.
One of the biggest differences if that everything is starting to look much more like HydrogenOS. Things like the power menu and the notification center have all been modified for a more Hydrogren type look.
A new feature that has been added to this ROM is the ability to choose between the Layers and the Substratum theme engines. Substratum is fairly new, but it's really special so check it out!
The advanced settings menu has been ported in from HydrogenOS. This menu lets you schedule power on/off, toggle user enjoy plan, and customize how you clear your apps.
Another change is the ability to access the system theme settings through the display menu. Choose between light, dark or default themes.
That's about it for changes in the FreedomOS ROM. If you want to get this ROM for yourself, check out this thread here.
In an attempt to limit the number of employees the company has to lay off in the long run, the CEO of BlackBerry has officially announced they will no longer be manufacturing smartphones anymore. This is a move that we've seen coming for a long time. John Chen has even alluded to this multiple times in the past, but had remained optimistic and hoping the BlackBerry Priv could turn things around for them.
It's still unclear if the company will drop their BlackBerry OS entirely, but they did follow through with previous reports that said they would focus entirely on Android this year, and they are reportedly focusing on the software-side of things. And this is a place where the company has done a job good, too. BlackBerry has often rolled out security updates faster than the rest of the competition, and has at times even beat Google to the punch. Making a profit on hardware just hasn't been where BlackBerry has excelled.
""We have decided to discontinue all the handset hardware development, only hardware""
We've seen multiple reports of the BlackBerry Priv not being able to sell as many units as expected. So it makes sense for John Chen to announce the change the company is making this week. He said they will still continue with development and manufacturing for a little while longer, but they want to be out it completely by the end of their fiscal year (which is February 28th). Chen says the company will be able to save a lot of money since they won't have to carry inventory and won't have to manage as much equipment.
We've already seen this shift recently with the BlackBerry DTEK50 and the upcoming DTEK60. BlackBerry will license their technology and brand to other companies so they can develop and manufacture the smartphones. They've already signed a deal with a telecom joint venture in Indonesia who will manufacture, distribute and promote BlackBerry devices within the country. Chen confirms these will also be running their secure versions of the Android OS.
