Lessons Learned regarding upgrading to Robolectric 3.3.2

Abstract

Robolectric is a unit test framework which allows you to easily unit test your Android app’s classes which are tied to the Android SDK such as activities, fragments or views.

Using Robolectric can really facilitate developer’s life because Robolectric makes testing such type of classes convenient by rewriting Android SDK classes as they’re being loaded and making it possible for them to run on a regular JVM.

This article discusses the advantages and the dis-advantages of upgrading to Robolectric 3.3.2 from Robolectric 3.0 with mentioning lessons learned during upgrade on a Gradle build system.

Robolectric 3.3.2 Incentives

Robolectric 3.3.2 has many advantages from the Robolectric 3.0 for example:

• It has a better Integration with PowerMock:
  Some of the known issues of Robolectric with PowerMock are fixed:
  For example, NullPointerException when running with RoboelectricTestRunner is fixed:
  https://github.com/robolectric/robolectric/issues/2749
  Other issues are fixed as shown below:
  https://github.com/robolectric/robolectric/pull/2952
  https://github.com/robolectric/robolectric/issues/2944
• It works peacefully with Mockito 2.x mocking of finals:
  The issue link is here, Mockito mocking of finals was broken and it is fixed in 3.3 release:
  https://github.com/robolectric/robolectric/issues/2677
• It has memory leakage fixes when testing activities:
  If you are still using Robolectric 3.0, your activities test may suffer from a memory leakage which can cause your build server to run out of memory, thanks to Robolectric team, this issue is fixed from 3.2.2:
  https://github.com/robolectric/robolectric/issues/2068
• It has RobolectricTestRunner which now supports running tests against multiple SDKs:
  This is important if you want to run for example a unit test class on an SDK version that does not match the targetSdkVersion specified in your manifest.
• Allows overriding the assets directory in Gradle builds:
  An important defect is fixed here:
  https://github.com/robolectric/robolectric/pull/2939
• Other Important bug fixes:
  For example, Robolectric 3.3.2 supports mocking Resource array which was not supported in Robolectric 3.0:
  https://github.com/robolectric/robolectric/issues/2064

Lessons Learned

1. Revise your old Timeout @Rule

You may find the following exception when you migrate your old Robolectric unit tests

java.lang.RuntimeException: Can’t create handler inside thread that has not called Looper.prepare()

After investigating the reason behind this exception, I found that tests who have this exception are failing because of an old @Rule Timeout which were not used properly, so removing these @Rule Timeout can make tests pass successfully.

2. Create a base Robolectric TestCase class for all Robolectric Tests

To avoid the fixing common errors/issues per every Robolectric test such as:

android.content.res.Resources$NotFoundException: Unable to find resource ID #0x7f10000b in packages [android, org.robolectric.default]

I recommend to have a base Robolectric TestCase class for all Robolectric tests which contains Configuration constants and other configurations if necessary for all Robolectric unit test classes as shown in the diagram below.

This base use case can be simply implemented like this.

So any extending Robolectric test class will not have to annotate test classes with @RunWith(RobolectricTestRunner.class) or even care about any other common configuration.

It can also override setUpTest() to initialize its test without having to use @Before for every test class.

3. Resolve conflicts

You may have some build conflicts after the upgrade, for example, if you are already using Guava in your app (with a different version), you may see a build conflict, to avoid it just make sure to exclude it from your Gradle configuration.

4. Do not forget to avoid depending on deprecated classes to keep your tests up to date

Do not depend on deprecated classes to avoid paying bigger migration bills in the future.

Such as migrating old org.robolectric.util.ActivityController to org.robolectric.android.controller.ActivityController.

You can find the definitive list in:
http://robolectric.org/migrating/

5. Do not expect faster tests execution using Robolectric 3.3.2

I already reported this issue in the project:
https://github.com/robolectric/robolectric/issues/3056

Unfortunately, Robolectric 3.3.2 has a slight performance degradation from Robolectric 3.0, around 10% performance degradation.

For example, running the following gradle command on hundreds of tests:

./gradlew :myproject:clean :myproject:test

First Run: Performance of the test degrades 12.5%:

• With Robolectric 3.3.2: Total time to run tests: 7 mins 50.634 secs (470 seconds).
• With Robolectric 3.0: Total time to run tests: 6 mins 58.988 secs (418 seconds).

Second Run: Performance of the test degrades 9%:

• With Robolectric 3.3.2: Total time to run tests: 7 mins 28.919 secs (449 seconds).
• With Robolectric 3.0: Total time to run tests: 6 mins 52.602 secs (412 seconds).

6. Revise your Old Custom Runner if applicable

Note that Custom Robolectric Runner API are unfortunately not 100% compatible from 3.0 to 3.3.2 as reported by myself here as an issue in Robolectric project:
https://github.com/robolectric/robolectric/issues/3073

So my advice here is to really revise if you still need to have a custom Robolectric runner for your Robolectric tests since you may not if you apply point #2.

This is all about my Robolectric 3.3.2 upgrade learned lessons.