Library does not work on Android b/c of Base64

[Former user]

Unfortunately, you cannot simply drop in the JAR in an Android app because Android uses an older implementation of Apache Commons Base64. In order to support Android developers moving forward, it may be prudent to create a branch for Android builds only. Thoughts?

[Vladimir Dzhuvinov]

Thank you for posting this issue.

We at NimbusDS do not have experience with the Android platform, so I would like to ask you a few questions in order to find a suitable resolution:

• Which commons codec version does Android come with?

• Is it possible to drop in the commons codec JAR that the JOSE+JWT lib depends on?

[Aby Das]

Thanks for the reply and sorry for not replying earlier. I've answered your questions inline:

Which commons codec version does Android come with? -- Based on several blog posts, it appears that it's <= 1.4. Reference: http://stackoverflow.com/questions/2047706/apache-commons-codec-with-android-could-not-find-method

https://github.com/Mashape/unirest-java/issues/1

Is it possible to drop in the commons codec JAR that the JOSE+JWT lib depends on? -- The last github page sort of answers that question. I don't think that there's anyway to tell the Android runtime to "select x.y version of the codecs library."

Also to experiment with, I did replace the Base64 functions call with the ones available in Android and it worked.

Thanks!

[Aby Das]

p.s., I think it still allowed me to post the first issue even when I wasn't signed in (therefore the Anonymous name)

[Vladimir Dzhuvinov]

Thanks Aby for working through this. I'll see that we make the library generally compatible with Android. For that it looks like the

String encodedString = new String(Base64.encodeBase64('string to encode'));
String safeString = encodedString.replace('+','-').replace('/','_');

solution would be the easiest to implement.

Is that what you did when you mentioned replacing the Base64 methods?

[Aby Das]

Thanks for getting back. I believe the method you mentioned above uses the Apache Commons Code function. The exact function I'm using is: android.util.Base64.encode/decode - which is using Apache Commons Codec underneath the covers (I believe the Base64.encode and decode function is just a wrapper of the Apache Commons Codec Base64 implementation underneath). These Android functions take two parameters, one being the byte array you want to encode/decode and base64 flags which I'm manually specifying. These flags are: NO_PADDING + NO_WRAP + URL_SAFE which are equivalent to the replace function you mentioned above.

So from what I understand Base64.encodeBase64 will not work as Android will not know which version of the Commons Codec to choose from (inbuilt vs. the one provided in the Commons Codec JAR). If this is the case, we might have to provide two base64 "strategies," one platform specific (Android, BlackBerry) and one server specific (that uses the latest Commons Codec JAR).

Thanks!

Thoughts?

[James Holland]

The issues on Android are not limited to just Apache Commons Codec but also the BouncyCastle version bundled with Android is almost useless. See the issues at http://rtyley.github.io/spongycastle/. I was in the process of forking and patching to SpongyCastle and http://migbase64.sourceforge.net/ but have had to postpone.

[A Das]

I changed the Base64 implementation to make it compatible with what Android currently has, ran the unit tests and it works. Haven't run into any issues lately.

[Vladimir Dzhuvinov]

Hi James,

I'm sorry that we're of little help with this issue, as we don't really have any mobile developers in our team. But if you guys manage to create some kind of a patch or publish a workaround, I will be happy to include that in the code / docs for other developers in your situation.

[A Das]

Thanks Vladimir.

Android BC does have limitations. (e.g. PBKDF2 with HMAC SHA256 is not supported). But it does support the crypto algs for this project. Plus in devices >= 4.2 they have moved over the SecureRandom function to use openSSL. See: http://stackoverflow.com/questions/13383006/encryption-error-on-android-4-2

[A Das]

See also: http://android-developers.blogspot.ca/2013/08/some-securerandom-thoughts.html

[Vladimir Dzhuvinov]

Thanks, I read the articles. It looks like developers need to copy with significant API instability when coding for Android :)

I've had some thoughts about switching the base64 codec to migbase64 and created a ticket for that if someone feels like working on that: https://bitbucket.org/nimbusds/nimbus-jose-jwt/issue/63/consider-switching-base64-codec-to-http

The com.nimbusds.jose.util.Base64URL class nicely encapsulates all encode and decode operation so a switch from Apache Commons Codec to the Mig library should not affect the JOSE API and client code at all.

[James Holland]

Hi Vladimir,

don't be sorry you have been a great help with producing this! The issue is that on Android people will use this project in conjunction with others that may require PBKDF2 with HMAC SHA256.

The only thing I did was change the usage over from BouncyCastle to the SpongyCastle version by changing the package names and provider.

Ideally, abstracting the usage of the crypto provider would be best, so we could plug in other providers. I haven't got that far, as all I've done is modified for SpongyCastle. Not sure what the best route to fork and add SpongyCastle, or add to this project the abstraction?

As for your proposal for the base64 switch these look good, but again, abstraction would be better. Happy to help with all of this.

[Vladimir Dzhuvinov]

Hi James,

Thanks for getting back.

I did a quick grep to check where the library has hardwired dependencies on BouncyCastle:

grep bouncycastle -R src/main/*

These are the classes:

• src/main/java/com/nimbusds/jose/crypto/AESCBC.java
• src/main/java/com/nimbusds/jose/crypto/AESGCM.java
• src/main/java/com/nimbusds/jose/crypto/AES.java
• src/main/java/com/nimbusds/jose/crypto/ECDSAParameters.java
• src/main/java/com/nimbusds/jose/crypto/ECDSAProvider.java
• src/main/java/com/nimbusds/jose/crypto/ECDSASigner.java
• src/main/java/com/nimbusds/jose/crypto/ECDSAVerifier.java
• src/main/java/com/nimbusds/jose/crypto/RSA_OAEP.java
• src/main/java/com/nimbusds/jose/jwk/ECKey.java

Unfortunately the entire EC JWS and large portions of the RSA JWE encryption cannot be done through the JCE provider interface (Java 6) and requires direct calls to classes in the BC library. One solution is to raise the Java requirement to version 7, which I know provides an API for EC signatures and probably extended support RSA encryption.

I need to study this more carefully because there are a lot of things to consider. I'm a bit pressed for time at the moment and cannot say how long this would take.

Adding a BASE64 SPI or provider would be somewhat of an overkill, I think just pasting Mig's code would be simpler, to implement and then for developers to use too.

BTW, yesterday there was an important bugfix release (2.19.1) which fixed a bad memory leak that affected RSA encryption and was caused by incorrect loading of the BC provider.

[James Holland]

Hi Vladimir,

agreed most of these can wait and yes a BASE64 SPI may be over kill.

I think keeping the Java 6 compatible would be a better option than moving directly to Java 7 as I think this opens the SDK up to a lot more users, as we don't know what the Android JRE compatibility is with Java 7, but i suspect that it is not. Probably the reason they bundle BouncyCastle.

Also are the JCIP annotations just for information, as I can't see a direct usage? Probably me missing the obvious.

As for 2.19.1 yes did see that and we're using that :-)

[Vladimir Dzhuvinov]

Hi James,

Over here we use the 7 JVM but keep the code compatible with 6.

Last night I studied the JCA interfaces and the BC spec in more depth and it now looks like the JWE RSA encryption with GCM and CBC can actually be abstracted entirely through the JCA provider interfaces. I don't know if this is going to affect performance, compared to the current direct use of the "lightweight" BC crypto classes.

The JCIP tags are indeed informational, for now. I've heard of tools that can use them to validate the concurrency of the code (but I haven't applied them to the code yet).

[Cedric Staub]

Hi Vladimir,

There shouldn't be performance issues with using the Java JCA interfaces directly as opposed to the BouncyCastle interfaces. The advantage of using the JCA interfaces on the other hand is that it would allow for automatic pluggable support of Hardware Security Modules. That is, some algorithms could be automatically backed by a hardware instead of a software implementation. This can be a big win for security/performance (especially the RSA case is interesting here). On Android, that stuff would also start going through the native provider (conscrypt) which is backed by OpenSSL and is probably faster than BC's Java implementation.

Cheers, Cedric

[Vladimir Dzhuvinov]

Hi Cendric and thanks for commenting.

I didn't know about the conscrypt provider on Android, that looks like another good reason to switch entirely to the JCA interfaces. Right now we're in the process of completing our OpenID Connect server product and once it's released I intend to get back to closing this issue.

[Vladimir Dzhuvinov]

Hi guys,

I just pushed a new release to Maven Central (under version 2.24) which incorporates a built-in base 64 codec (taken from the migbase64 project and modified for JOSE). The Apache Commons Codec dependency is gone now, so Android users should rejoice :)

[A Das]

Thanks Vladimir. I'm sure many will find this useful.