Benchmark Report: Android 12 vs Android 13 using the viSer Test Automation Solution

Latest Testing News Update

Date: 20th October 2022

Android 13 - There are some interesting changes

The Android 13 update is finally here after a relatively short beta testing program. The latest version will be available in devices gradually, but we have already seen a number of supported devices.

You can view the list of available devices here.

At SmartViser we understand the importance and additional workload these upgrades bring both manufacturers but also for operators. We understand that:

• Manufacturers need to ensure that the new OS will not have any significant impact on the performance of the device and cause any unnecessary regression issues.

• Operators from the other side also carry out significant testing to ensure their end-users continue to get quality devices with superior Quality of Service and Experience in their network.

Our viSer test automation solution is compatible with any device with any OS version making it the ideal tool to run benchmark reports, especially when new software or a new OS is available. There is no need to change any of the test scripts or test plans when testing different devices or OS versions.

For this Benchmark Report, we have used the Google Pixel 5 device. They have been tested on Orange France and SFR networks. One device had the Android 12 OS and the other was upgraded to the Android 13 OS.

So what did we discover?

FILE TRANSFER THROUGHPUTS

HTTP Downloads and Uploads

In terms of the File Transfer throughputs, we see a marginal difference between the two versions with Android 12 performing better both on the HTTP download and upload.

Image from Viser Web Services Analytics Studio

FTP Downloads and Uploads

On the FTP Side again in the downloads, the Android 12 performs better than the Android 13 version, but we saw a difference in the upload speeds with the Android 13 having a better performance there.

CALL PERFORMANCE

Performed 50 VoLTE calls on Orange Network. Both versions performed very well and had no call drops or failures.

Both versions performed very similarly in the call establishment time and the average time is almost identical.

MESSAGING SMS/MMS

Tested on 25 SMS messages on a 4G network

Both versions performed similarly regarding the time to send an SMS. The Android 13 on average had a longer time during the testing we performed but at a very similar level to Android 12.

Viser Web Services offer an additional timeline for each SMS time to send metrics.

For MMS the testing was performed 25 times. All tests were successful and again the marginal difference between the two OS.

WEB LOADING

For web browsing testing. Testing was performed in the 4G area for around 30min.

All test cases were automated using the same web pages on each device.

The average loading time for both OS versions Android 12 and Android 13 was very similar.

BATTERY LIFE

Finally, we come to the crucial battery life performance. We have performed a number of activities as an end user will do. Both batteries have been used previously and the capacity is below 90%.

In terms of performance.

• Pixel 5 with Android 12 lasted 13h 33min 12s

• Pixel 5 with Android 13 lasted 13h 16min 21s

• No impact on the battery performance has been observed with the two different OS.

Conclusion

We are very pleased to report that during our benchmark testing we have not found a big gap in terms of performance between the two Android versions. It is very important to test thoroughly during major OS upgrades to identify and fix any issues prior to release in the market.

Upgrading from Android 12 to Android 13 for the Pixel 5 it did not have any flaws in terms of performance. This is not always the case as in the past we have observed some regression in terms of battery performance but not this time. On the other side, there is no major improvement in the 5 areas we have benchmarked.

This was not the main focus for Android as one of the key features the Android 13 brings is mainly around customisation, App language choice, opt-in notifications and battery management.

We are happy to arrange a Free Trial of viSer, our Test Automation Solution where device manufacturers can test any device without any limitations on the number of tests or test devices required. Check how Android 13 will impact the performance of your devices.

Equally, if you are a Mobile Network Operator with an increased testing workload due to the Android 13 speak to us today on how test automation can help you with the validation and benchmark,