Exception handlers often broken in converted Android code
input:
java -jar procyon-decompiler-0.5.15.jar f.class > 123
output:
// java.lang.IllegalStateException: Expression is linked from several locations: Label_0243: // at com.strobel.decompiler.ast.Error.expressionLinkedFromMultipleLocations(Error.java:27) // at com.strobel.decompiler.ast.AstOptimizer.mergeDisparateObjectInitializations(AstOptimizer.java:2386) // at com.strobel.decompiler.ast.AstOptimizer.optimize(AstOptimizer.java:214) // at com.strobel.decompiler.ast.AstOptimizer.optimize(AstOptimizer.java:42) // at com.strobel.decompiler.languages.java.ast.AstMethodBodyBuilder.createMethodBody(AstMethodBodyBuilder.java:214) // at com.strobel.decompiler.languages.java.ast.AstMethodBodyBuilder.createMethodBody(AstMethodBodyBuilder.java:100) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createMethodBody(AstBuilder.java:668) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createMethod(AstBuilder.java:567) // at com.strobel.decompiler.languages.java.ast.AstBuilder.addTypeMembers(AstBuilder.java:493) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createTypeCore(AstBuilder.java:460) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createTypeNoCache(AstBuilder.java:140) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createType(AstBuilder.java:129) // at com.strobel.decompiler.languages.java.ast.AstBuilder.addType(AstBuilder.java:105) // at com.strobel.decompiler.languages.java.JavaLanguage.decompileType(JavaLanguage.java:55) // at com.strobel.decompiler.DecompilerDriver.decompileType(DecompilerDriver.java:270) // at com.strobel.decompiler.DecompilerDriver.main(DecompilerDriver.java:114) //
Comments (27)
-
-
I get this a lot too but only for dex2jar produced code. Just to check though I used another tool, Dare which again converts a dex into class files. Exactly the same result from procyon, Expression is linked from several locations so the code seems to be valid jvm code as far as I can tell. Would be nice to have this work, really nice but other Java decompilers are failing at this, including cfr.
-
yeah, Marius, you got it right, the java classes were produced by the d2j but, isn't it a beauty of the reality to break the rules? ie to do things that others takes as rule "just not possible because not possible?" :)
-
Oh it is, it is. De-compiling closed source Android apps to see how they work is so much fun. I played around with whatsapp quite a bit so I can make an open source version.
Maybe Mike can help us out here, I've tried looking into the code but it's not so easy to determine what's wrong.
-
my words was not about 3rd party closed app, but to the fact that other decompilers can't do it - that is the aim, the app was taken as a random one.......
-
Oh I see now. I'm interested in some particular ones but that's for Mike to decide whether he wants to beat his co-worker that's developing cfr :)
-
Looks like dex2jar produces some really nasty exception handler tables. This definitely isn't going to be an easy fix.
-
Mike I've attached the class file produced by dex2jar and dare for a case where procyon fails in a similar way. Maybe the one produced by Dare is better.
Couldn't attach here for some reason so: http://www62.zippyshare.com/v/1976085/file.html http://www62.zippyshare.com/v/27137416/file.html
Also here's a idea which won't fix the issue entirely but it would be good enough for me. If a class method can't be decoded and it has some try/catch blocks go back and decode it but ignore all the try/catch and finally.
If this is not too hard to do it's good enough for me.
I can then add the try/catch code myself if needed but I'd much prefer this to no method decoding at all.
-
@mike Probably a good idea is to report to dex2jar team your concerns regarding exc handlers?
-
Alex if dare produces similar results ( and it seems it does ) I'm not sure what dex2jar can do about it. Dex2jar is also not a team, just like Mike it's more of a one man show :). Mike when you have the time could you look if the code produced by Dare is in any way better. Dare even has a flag that's supposed to make it easier for decompilers and I've used it. From what I could tell the dare code does look a bit better.
-
-
assigned issue to
- changed component to CompilerTools
- changed title to Exception handlers often broken in converted Android code
More in Issue
#120. -
assigned issue to
-
Issue
#120was marked as a duplicate of this issue. -
Marius, may I suggest using Krakatau instead? It's designed handle stuff like this just fine.
-
Uncaffeinated I did try Krakatau but didn't have much luck with it either. I find it harder to use also. Took me a while to figure out it wants all the classes referenced by the ones to decompile so I had to add the android.jar to the path. Also wasn't clear what the path separator was so as to add multiple jars.
This seems like a big difference, Mike can procyon use extra objects in the class path to provide better results ? Like in this case would having the android framework jar help since a lot of classes import classes from there ?
Anyway the more the better. Often one compiler fails and another succeeds and I'm sure Mike will get around to fixing this, I only wish commercial software would have bug fixing like procyon does.
Right now though with the dex2jar generated jar I'm afraid Krakatau behaves even worse than procyon, it just hangs. I'll report the issue.
-
If you found a bug or have suggestions for where the documentation should be improved, feel free to report it over at https://github.com/Storyyeller/Krakatau. It definitely shouldn't be just hanging.
-
OK so after testing Krakatau it seems dare actually produces better code from an Android dex or so it seems. Below are 2 jars from Dare and dex2jar made from the same APK:
http://www25.zippyshare.com/v/73555638/file.html
http://www25.zippyshare.com/v/34367401/file.html
These work with procyon but some methods give the errors above ( the duplicate label and the index problem ) . Duplicate label is also a problem for other decompilers but the index one is not.
-
This seems like a big difference, Mike can procyon use extra objects in the class path to provide better results ? Like in this case would having the android framework jar help since a lot of classes import classes from there?
Procyon will do its best to try to resolve dependencies with whatever information it has available. This includes probing the system and bootstrap classpaths (including your
CLASSPATH
environment variable). It also maintains a memory of where it has found classes before, and may probe the file system according to package structure. For instance, if you were to runprocyon ./C.class
, andC.class
is the typea.b.C
, Procyon will remember that directory as a known location for thea.b
package. Then, if it needs to find the classa.X
, it'll try searching one level up in the file system.Procyon will always attempt to locate referenced classes as they are needed, but it should be able to produce reasonable output even if it fails to load every other class. In theory, the only classes it must be able to resolve are:
- The class you are trying to decompile;
- The class
java.lang.Object
.
You will get the best results when all referenced classes can be resolved. You may see artifacts due to failed class resolution, most notably:
- There may be redundant casts at method call sites if method overload resolution cannot be performed (e.g., because the target class could not be loaded).
- Missing inner classes (especially anonymous inner classes) will result in some artifacts being left behind.
- Variable types may not be as specific as they could be. If Procyon has to figure out a common supertype for two classes, but it can't resolve all the base classes and interfaces, it may end up picking
java.lang.Object
.
But in the case of the errors here, they all seem to be related to the exception handler structure. The DEX to JAR converters seem to produce a lot of overlapping handlers, which I don't handle very well at the moment.
-
I have tried both d2j and dare to reproduce the class files and I also get this error. perhaps I could attach my sample class file produced by dare and you could check if it does the same thing as d2j does.
dare - https://dl.dropboxusercontent.com/u/16422464/classes/dare/DatabaseHelper.class
d2j - https://dl.dropboxusercontent.com/u/16422464/classes/d2j/DatabaseHelper.class
-
Here is some of my research TryCatchTest.zip
I took TryCatchTest.java from Krakatau and made the following:
- Unobfuscated: javac->dx->dex2jar->procyon
- Obfuscated: javac->zkm->dx->dex2jar->procyon
Both provides huge exceptional tables. Even without obfuscation Procyon produces damaged sources (while fernflower and jd-gui does not at all)
-
hi guys, I am the author of dex2jar, here is why, the exception table is huge.
the following code is valid in dex, from issue 109 of dex2jar https://code.google.com/p/dex2jar/issues/detail?id=109
L0: //... L1: //... L4: //... return a2; L2: // ... L5: // ... L6: return a2; L0 ~ L1 > L2 L4 ~ L5 > L6
because dalvik has different control flow to jvm, only a throwable op goes to the exception handler. but in jvm, any op in the exception table goes to the exception handler.
so, in jvm, a path (L0,L2,L6,return a2) is created, and cause a validationException mention that a2 is not initialized.
to fix this, i cut the exception table into pieces. Its a simple but stupid change.
any suggestion?
-
Interesting. Krakatau effectively does the same thing internally in the hopes of simplifying the code. But if the input can't easily be expressed with Java control flow, then the output will be really messy no matter what you do.
-
Hi pxb1988. First of all thank you for dex2jar, together with Procyon or Krakatau it proved very useful in understanding how some Android apps work. I'm not the most qualified to comment here due to my limited understanding of this but what I did notice is that Dare seems to take a slightly different approach to it. Not sure if it's better, if it's correct but certainly seems different. There are some attached files from both Dare and D2J.
-
Hi Mike
seen this issue on the latest .25 build:
// Exceptions: // Try Handler // Start End Start End Type // ----- ----- ----- ----- --------------------- // 91 117 128 721 Ljava/lang/Exception; // 91 117 318 328 Any // 117 125 128 721 Ljava/lang/Exception; // 117 125 318 328 Any // 130 189 318 328 Any // 195 200 701 712 Ljava/lang/Exception; // 195 200 318 328 Any // 223 249 128 721 Ljava/lang/Exception; // 223 249 318 328 Any // 249 275 128 721 Ljava/lang/Exception; // 249 275 318 328 Any // 275 307 128 721 Ljava/lang/Exception; // 275 307 318 328 Any // 307 315 128 721 Ljava/lang/Exception; // 307 315 318 328 Any // 328 355 128 721 Ljava/lang/Exception; // 328 355 318 328 Any // 363 397 128 721 Ljava/lang/Exception; // 363 397 318 328 Any // 400 427 128 721 Ljava/lang/Exception; // 400 427 318 328 Any // 435 451 128 721 Ljava/lang/Exception; // 435 451 318 328 Any // 451 478 128 721 Ljava/lang/Exception; // 451 478 318 328 Any // 478 574 128 721 Ljava/lang/Exception; // 478 574 318 328 Any // 574 582 128 721 Ljava/lang/Exception; // 574 582 318 328 Any // 585 598 128 721 Ljava/lang/Exception; // 585 598 318 328 Any // 601 628 128 721 Ljava/lang/Exception; // 601 628 318 328 Any // 636 643 128 721 Ljava/lang/Exception; // 636 643 318 328 Any // 646 674 128 721 Ljava/lang/Exception; // 646 674 318 328 Any // 677 683 128 721 Ljava/lang/Exception; // 677 683 318 328 Any // 683 693 128 721 Ljava/lang/Exception; // 683 693 318 328 Any // 703 709 318 328 Any // 712 718 318 328 Any // // The error that occurred was: // // java.lang.IllegalStateException: Expression is linked from several locations: Label_0200: // at com.strobel.decompiler.ast.Error.expressionLinkedFromMultipleLocations(Error.java:27) // at com.strobel.decompiler.ast.AstOptimizer.mergeDisparateObjectInitializations(AstOptimizer.java:2592) // at com.strobel.decompiler.ast.AstOptimizer.optimize(AstOptimizer.java:235) // at com.strobel.decompiler.ast.AstOptimizer.optimize(AstOptimizer.java:42) // at com.strobel.decompiler.languages.java.ast.AstMethodBodyBuilder.createMethodBody(AstMethodBodyBuilder.java:214) // at com.strobel.decompiler.languages.java.ast.AstMethodBodyBuilder.createMethodBody(AstMethodBodyBuilder.java:99) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createMethodBody(AstBuilder.java:756) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createMethod(AstBuilder.java:654) // at com.strobel.decompiler.languages.java.ast.AstBuilder.addTypeMembers(AstBuilder.java:531) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createTypeCore(AstBuilder.java:498) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createTypeNoCache(AstBuilder.java:140) // at com.strobel.decompiler.languages.java.ast.AstBuilder.createType(AstBuilder.java:129) // at com.strobel.decompiler.languages.java.ast.AstBuilder.addType(AstBuilder.java:104) // at com.strobel.decompiler.languages.java.JavaLanguage.decompileType(JavaLanguage.java:59) // at com.strobel.decompiler.DecompilerDriver.decompileType(DecompilerDriver.java:301) // at com.strobel.decompiler.DecompilerDriver.main(DecompilerDriver.java:127) // throw new IllegalStateException("An error occurred while decompiling this method."); }
I guess its same but in a case you'd like to take a look I could PM you the class file Alex
-
I ran into this problem with Android classes as well. I think I have traced it to ensureDesiredProtectedRanges (called by pruneExceptionHandlers() in compilertools.com.strobel.decompiler.ast.AstBuilder)
(The code below has some of my comments and debug statements in it. Note that I commented out the 'handlers.set()' call to test the 'fix'.) This is complicated by the fact that I don't really understand that 'ensureDesiredProtectedRanges' is supposed to do. The effect is that ensureDesiredProtectedRanges is messing up the length of the try block by extending it to the instruction before the catch block.
Commenting out the handlers.set() fixes the 'Expression is linked from several locations' problem for me (not the IndexOutOfRange exception version of the problem).
private void ensureDesiredProtectedRanges() { final List<ExceptionHandler> handlers = _exceptionHandlers;
for (int i = 0; i < handlers.size(); i++) { final ExceptionHandler handler = handlers.get(i); final InstructionBlock tryBlock = handler.getTryBlock(); final List<ExceptionHandler> siblings = findHandlers(tryBlock, handlers); // List of handlers with same tryBlock final ExceptionHandler firstSibling = first(siblings); final InstructionBlock firstHandler = firstSibling.getHandlerBlock(); final Instruction desiredEndTry = firstHandler.getFirstInstruction().getPrevious(); for (int j = 0; j < siblings.size(); j++) { ExceptionHandler sibling = siblings.get(j); if (handler.getTryBlock().getLastInstruction() != desiredEndTry) { // Last instruction of exHndlr try block is not the firstInstruction of the firstHandler final int index = handlers.indexOf(sibling); System.out.println(" eDPR "+handler+" tryEnd != "+desiredEndTry //RAF +"\n "+j+"/"+handlers.size()+" hndlrs, "+index+"/"+siblings.size()); org.ribo.jDecomp.rLog.append("eDPR "+this._context.getCurrentMethod().getName() +" "+_context.getCurrentType().getName()+" "+sibling+"\n"); if (sibling.isCatch()) { // This is a try/catch (not try/finally) //RAF //handlers.set( // index, // ExceptionHandler.createCatch( // new InstructionBlock( // tryBlock.getFirstInstruction(), // desiredEndTry // ), // sibling.getHandlerBlock(), // sibling.getCatchType() // ) //); } else { ...
What fix to the list of exceptions is this method supposed to do? Is it supposed to merge two handlers or adjust the range of the try block? In any event I'm suspicious of the:
desiredEndTry = firstHandler.getFirstInstruction().getPrevious();
statement. desiredEndTry is an instruction preceding the catch handler code block -- changing the end of the try code block to be something before the catch block code. In the Android code I'm decompiling, the catch block doesn't directly follow the try block code. This the try block ranges of code of two different try blocks to overlap.
So, what is ensureDesiredProtectedRanges intended to fix in the list of exception handlers? -- or what assumptions is it making about the relationships between trys and catches?
I've instrumented the decompiler code so I can see what is going on, so I can test things out.
-
I think the best answer here is to just use an android decompiler instead of dex2jar + a java decompiler. Converting to a jar first will always be a lossy process.
-
I'm not so sure using dex2jar or similar + java decompiler can't have perfect or almost perfect results. Of course the Java decompiler probably has to do some tricks it wouldn't normally do but it should work.
The problem with using an android to java decompiler is that I don't know of one that's free or at least reasonably priced. JEB costs too much money for me. I'd like it open source or maybe I'd pay like $100 it were really, really good.
-
I haven't really investigated Android compilers since I was never interested in Android, but if worst comes to worst, you could always just modify Krakatau to read dex files. Sadly, I'm probably the best person to do that but I don't have the time or interest any more.
- Log in to comment
I don't suppose you would happen to have the original, unobfuscated class, would you? Or the original source? :)