Dexpatcher injection breaks dex2oat verification

[shomykohai]

Hi @Lanchon ,
I recently got reports for haruka not being able to apply signature spoofing patch.
My module uses dex2oat on the device with specific flags to avoid compiling oat files that might make the device bootloop.

Unfortunately, it seems like on some roms (mainly A15), dex2oat fails with hard verification errors.
I checked everything I could to conclude it to being dexpatcher's fault

Here are the logs
shomykohai/haruka#3 (comment)

The injection itself works flawlessly, but the written dex files contain issues that make them invalid and prone to bootloops.

That said, I was wondering if you have any idea what the specific issue could be, and if you will continue development on dexpatcher, because the tool itself is really handy and powerful and enjoyable for stuff like this, and it would be sad if I had to switch tool 😅

Hope that a solution can be found, and thanks again for your amazing tool!

[Lanchon]

hey!

saw your msg on xda. yep, i sort of lost interest in dexpatcher for a while now. it was all caused by horrible behavior on the part of the gradle team, plus noncompliance on their part to their own stated policies for a period of i think over 2 years, that demotivated me enough to stop working on it. (it does not impact your use case, but gradle was the android build system to target if dexpatcher was going to be easy to use to patch apps.)

anyway...

dexpatcher uses dexlib to parse and write dex files. if dexlib interface was a bit better designed or maintained, dexpatcher should just be able to just ride on newer dexlibs without issue. or at least dexlib could have been designed to break clients such as dxp at compile time if its api was semantically broken.

but this was never the case. dxp would compile and run fine on newer dexlibs, all the while making "mistakes" during its process because of obscure semantic changes in dexlib. i pushed some commits into dexlib and had lengthy exchanges with the author to help stabilize its api, and in this regard it got better with time. but for every new lib release, i still had to go though each and every commit in dexlib to analyze its possible impact on dxp's logic.

fast forward to today, and a lot has happened...

last thing i knew, a couple years ago JF stopped maintaining dexlib. dexlib was important for google: google's implementation for reading and writing dex was d8/r8 (by the author of the v8 javascript VM in chrome). d8/v8 was an implementation based on the design docs of dalvik. but it is very hard to verify the correctness of such code if you do not have anything to check it against. google's solution was to test d8/r8 via interop with dexlib, an independent implementation of the specs, hoping that deviation from the specs in those two implementations would not be correlated and thus flagged as issues during testing.

this worked well until dexlib was abandoned. at this point, someone at google forked dexlib and started moving it forward to match android releases.

you can look for that dexlib fork and check if it is still kept up to date and where. post your findings here.

without an updated dexlib, dxp will not be able to read and write dex while conserving all the metadata required for newer android systems.

assuming there is an updated dexlib, just swapping it from under dxp can very possibly work. (but keep in mind that runtime or even compile time compatibility of a newer dexlib does not guarantee correctness of dxp output... but it should mostly work... i guess.)

but unfortunately i think that the dexlib fork changed the package name (to something withing com.google or com.android i think, maybe somewhere within AOSP). so swapping dexlib requires more than a line of gradle to override the old lib: either you search and replace the package name in dxp sources, or rewrite its *.class files to do the same. (but you should do the former to at least verify compile time correctness.)

so a plausible roadmap would be:

• join hands in prayer for the existence of an up-to-date dexlib fork.
• find it.
• mod dxp to use it.
• pray even more for it not to break semantics of dxp.

if you get the ball rolling with this i might be able to help.

thanks!

[shomykohai]

Hi @Lanchon, thanks for your detailed response.

I looked up for the google fork of dexlib2, and I found it to be hosted here:
https://github.com/google/smali

Furthermore, it is currently being used in dex2jar fork, which is another tool I use in haruka for converting compiled patches into dex files, and the tool works fine, as well as APKTool.

The fork itself seem to have not so frequent updates (still maintained of course), meaning it can surely be used without needing to check so frequently.

I can surely try replacing dexpatcher's internal to use the new google fork, which as I said is used on many reliable tools (even ReVanced uses it for the patcher), and hopefully got it to compile!
My only hope is that the API didn't change much between the dexlib dxp 1.8.0 uses and google's fork

Edit:
Interestingly enough, ReVanced also forked multidexlib2 to point to google's fork of dexlib2, so I guess 20% of the job is already done using that!

[Lanchon]

thanks for the investigation! i should look into all of this at some point.

btw, i remember google made a horrible change to dex where they added a flag to framework members to denote whether they should be visible from apps or not, or something like that. AFAIR, the horrible part was that this ad-hoc flag did not have java language counterpart (such as an annotation) and was injected into the dexing process from another source, such as a text file. this did not apply to apps, only the framework, and thus i didn't concern myself much with this horrible hack. but if you wanted to edit the framework (as you do) you did need to handle it somehow. google might have replaced this quick and dirty junk with some proper solution, or they might been stacking up more of this kind of crap into a big stinking pile, IDK.

that said, it would be a reasonable subproject to bring dexptacher-tool up to speed with modern android. and yeah, that would need to start with multidexlib. oh, and i had completely forgotten that thing existed, lol

[shomykohai]

thanks for the investigation! i should look into all of this at some point.

btw, i remember google made a horrible change to dex where they added a flag to framework members to denote whether they should be visible from apps or not, or something like that. AFAIR, the horrible part was that this ad-hoc flag did not have java language counterpart (such as an annotation) and was injected into the dexing process from another source, such as a text file. this did not apply to apps, only the framework, and thus i didn't concern myself much with this horrible hack. but if you wanted to edit the framework (as you do) you did need to handle it somehow. google might have replaced this quick and dirty junk with some proper solution, or they might been stacking up more of this kind of crap into a big stinking pile, IDK.

that said, it would be a reasonable subproject to bring dexptacher-tool up to speed with modern android. and yeah, that would need to start with multidexlib. oh, and i had completely forgotten that thing existed, lol

Hi, sorry for the delay.
Fortunately, multidexlib has been forked by Revanced as I said, and considering how ReVanced is used by many people without any issues, I'd say it's pretty stable as a fork.

I tried bringing the project to newer versions of gradle and so on, but not having worked with gradle myself, it didn't went that well, and feels harder than porting dexlib itself.

I could help only after the build system itself is fixed first I'd say 😅