Running DexPatcher-tool on-device (Android and TWRP)

[ShapeShifter499]

There might be a case where there is a critical upgrade for a bug but you don't have a computer around so you have to chose between a OS update or MicroG services support. The question is, if we have root why can't we patch these files on the device itself?

[Lanchon]

hi,

it's entirely possible to patch in-device. in fact, the patcher could use the 'addon.d' mechanism to patch automatically after every rom flash, just like gapps self install after every rom flash.

the issue is that haystack is not a project i care about or want to invest time in, it's just a dexpatcher demo.

the problem with this approach

if you ask me, patching roms like this shouldn't be needed. projects like cyanogenmod should go extint. CM has a horrible track record of choosing google, their marketability, an app developers over CM users. they seemed to believe android had to be "fair" to developers and "play nice" in the ecosystem. they did horrible stuff like this:

• crippled CM one day, in preparation for their conversion to Cyanogen Inc, by removing all code that was GPL or had a non-proprietizable license; CM lost important features to users because of this.
• rejected patches that lied to apps to protect user's privacy. for instance, lying about location, contacts, accounts, phone number, ESN, IMEI, AndroidID and the list goes on forever.
• rejected OpenPDroid, another set of privacy oriented patches with more or less the same aim.
• rejected signature spoofing for microG.

and many, many more. the rationale is always fairness to app developers or some other equally stupid excuse, as if app developers owned your phone and your phone should be poised to help them. even as if app developers could be trusted!

the reality is that they rejected every change that would make CM not aligned with Google's idea of what android should be, a necessary step to be able to license Google's Play Store and other proprietary software from them.

what is needed is a community that cares about freedom (it doesn't exist, they only care about immediate features) and defends themselves from malevolent dictators such as steve kondik. the followup for cyanogen, call it X Android, should have these characteristics:

• no copyright assignment agreement enforced to accept submissions. XA.org must NOT be able to relicense XA, because it must not own XA; the people, the contributors, must own XA.
• all additions to the codebase must be accepted in a set of predefined FLOSS licences. by default the GPL v3 licence should be used for contributions, unless the code already has another FLOSS license. this means that XA will not be proprietizable by any company: nobody will be able to take features of XA and bundle them in a proprietary software phone. if they want XA features, they'll have to make the device and all its OS free.

these safeguards ensures that the commercial value of XA is close zero, and precludes business-oriented decisions that hurt XA users.

back to haystack and dexpatcher

dexpatcher has a million ways to improve. i have so many feature's i'd like to implement in my head. unfortunately my time is limited, and there seems to be very few people interested in dxp anyway; and that demotivates me.

one issue with dxp is that it was not designed to handle various codebase versions to patch on. my use case for dxp is this:

• i patch an app.
• if the app gets an update, maybe i'll update my project and rebuild. dxp does a great job of alerting me of any inconsistencies with the new codebase, so i can update my patch to be compatible with it. (however, it could still improve some more).

dxp was never designed for the following case:

• build a patch that applies cleanly and safely not to one, but to various codebase versions.

in haystack the corollary of this is seen as the individual "hook" patches that adapt haystack to each OS version.

so if i ever implement dexpatcher changes i have in my head to support "multi-targeting", haystack could be unified into a single patch that applies cleanly to all OS versions.

in that situation, making an in-device patcher would be much simpler. i still would be hesitant to use my time for that goal, but maybe...

some final notes

keep in mind that patching takes time. on average phones (slow), it wouldn't surprise me if patching took around 5 minutes, even after all the optimization work i put into dxp.

odexed roms still cannot be patched, so haystack is not an ideal solution. i've heard that Android 7 is mostly non pre-odexed. finally!!!! that odex crap retired from the platform!! this change, if confirmed, would increase the value of haystack.

patching is trivial to do in-device on android, because android can readily run java bytecode (via dalvik bytecode conversion), but not so easy to do in recovery. to patch in recovery (ideal) you need a java runtime (JRE) that works in recovery. such VMs probably exist but one has to be built for the environment and each supported architecture. these VMs tend to be simple implementations and thus very basic in their optimization strategies (read: bad performance). so the 5 minute patching could well turn into 15 minutes in these low quality VM implementations.

of these VMs, i favor "avian" for its build simplicity. i tested dexpatcher on amd64 avian (works just fine) and benchmarked it, and was pleasantly surprised to find good performance: not that far from the openjdk jre numbers.

[Lanchon]

if you want to make a solution yourself for current haystack, you can.

dexpatcher runs just fine in the dalvikvm. take a look here to see how:
http://stackoverflow.com/questions/10199863/how-to-execute-the-dex-file-in-android-with-command

[Nanolx]

I worked something out. In

https://github.com/Nanolx/NanoMod

you'll now find the sub directory framework-patcher this is indeed an on-device framework-patcher. You can use it in two ways:

• flashable zip
• manually run commands

to create a flashable zip issue mod.sh patcher which will create the zip from the repo and flash as usual

manual instructions:

adb push framework-pacher/dexpatcher /tmp/dexpatcher/
adb shell
/tmp/dexpatcher/framework-patcher.sh

current issues:

• when flashing as zip framework-patcher and dexpatcher output is not shown
• only tested on one device currently (Galaxy Tab 4)
• may not work on x86/x86_64 (because I have not expanded LD_LIBRARY_PATH)
  • who uses that anyway?

This is the first try. I hope it works for most out-of-the-box if not, provide me /tmp/recovery.log and more infos at the NanoMod repo.

[Nanolx]

Ah... about the speed: on my Galaxy Tab 4 that process takes about 3 - 4 Minutes.

Edit: on my Moto X Play it also takes about 3 Minutes. Quite speedy this dexpatcher thingy. 🥇

Another note: the framework-patcher is able to install the patched services.jar into three places:

• NanoMod Magisk Module (if it's installed)
• NanoMod microG Module (if it's installed)
• directly into /system/framework (fallback)

[ShapeShifter499]

@Nanolx Does this mean I can go down to having to only flash four things whenever LineageOS (or other rom) has an update all without a computer?

For example
The rom update zip
Magisk
The NanoMod Module
The zip that initiates the dexpatcher

[Nanolx]

Almost:

• ROM
• Magisk
• Patcher

NanoMod stays intact until you wipe /data (regardless of Magisk re-install).

That is, if the patcher zip works for you, too. If that's the case: no more pc needed.

I uploaded a pre-built patcher zip to the NanoMod afh folder, btw.

[ShapeShifter499]

@Nanolx thank you! I'll give it a try on the next LineageOS release since I just got everything setup. Maybe next week though since their "nightly" builds are more like weekly builds.

[Nanolx]

I pushed an updated version to github which fixes several issues and uploaded a new zip to https://www.androidfilehost.com/?a=show&w=files&flid=150729 (it's the "NanoMod-patcher-YYYYMMDD.zip" file)

[Nanolx]

@Lanchon I noticed the following regarding dalvikvm/dexpatcher on device:

• some devices need to boot atleast once into the ROM after full wipe, else dalvikvm fails to initialize
  • happens on Galaxy S6, but not on Galaxy Tab 4 or Moto X Play
• some devices don't need to boot into the ROM once after full wipe, but they take awfully long to patch

Since my technical knowledge about dalvik is ... not so much, maybe you can sched some light into this?

I will try with avian and see how that works out.

Edit: updated post.

[Lanchon]

@Nanolx,

you'll now find the sub directory framework-patcher this is indeed an on-device framework-patcher. You can use it in two ways:

flashable zip

dalvikvm is not available in recovery. so what is your zip doing? does it schedules dexpatcher to run on next boot? or are you trying to setup an environment in recovery that is able to run dalvikvm?

current issues:

when flashing as zip framework-patcher and dexpatcher output is not shown

hehe, use Flashize for that!

some devices need to boot atleast once into the ROM after full wipe, else dalvikvm fails to initialize
happens on Galaxy S6, but not on Galaxy Tab 4 or Moto X Play
some devices don't need to boot into the ROM once after full wipe, but they take awfully long to patch

well firstly i don't know how you are running dalvikvm. i also don't know which version of android you are testing on. finally i don't know dalvikvm well either lol.

i believe the dalvik vm does not exist anymore since Android 5 and ART. so dalvikvm must be an interface to ART. it looks like ART in Android 6 is unable to run dalvik code directly and hence the long first boot time and apk install time, in which ART is compiling everything to native code. so there is no more VM? i don't know. maybe the VM is kept for compatibility with dynamically loaded bytecode.

so what happens when you invoke dalvikvm in Android 6? who knows! maybe the bytecode is compiled ahead-of-time into a temporary or cached OAT file. or maybe the bytecode is interpreted by the vestigial VM. or maybe it is also JIT'ed, but i think JIT is not present in Android 6.

by your account, it looks like in some devices the framework (including java.* packages) might not be oated, which would happen in first boot, and either dalvikvm fails to run or interprets everything, slowing to a crawl. or maybe it is oated, but dalvikvm can't find the oats because it's not configured properly yet. or maybe dalvikvm always compiles code passed to it Android 6, but it is unable to compile it before first boot and fails or runs dexpatcher terribly slow.

in Android 7, it's even more loose, because we know it has a working interpreter, plus JIT, plus AOT compiler. so at least one should expect dalvikvm not to fully interpret dexpatcher.

i can say one thing: if you run dex2oat on dexpatcher before running it should run much faster than what you've seen!! or not, lol, who knows!

whatever you find out, do tell us :)

I will try with avian and see how that works out.

avian's classpath is unable to run dexpatcher. it is very incomplete and dexpatcher wasn't written to support that scaled-down runtime.

but... avian-pack does! and, considering that it's a simple VM, it even runs at an acceptable speed; i've tried it. unfortunately avian-pack -supposedly arm compatible- is distributed pre-built only for intel, which is a drag. i gave a try to building it for arm but the process failed and i didn't have time to continue. supposedly you can build a completely statically linked fat binary with even the android classpath in it: a complete Java VM in a single file that you can use anywhere, including in recovery. and if you succeed building a static avian-pack for arm, i'll give you a big wet kiss.

[Nanolx]

@Lanchon Thanks!

My zip mounts /system and runs the dalvikvm command from the ROM, on clean flash it takes ages - as you said because everything is still in unprepared state - on an used ROM it's pretty fast (only 3 - 4 Minutes on mid-range devices like Galaxy Tab 4 or Moto X Play).

I didn't know about Flashize, but I managed to wrap around the shell output like

print () {
  print_stuff_and_catch_error
}

command() {
  do_stuff || on_error
}

(command | print) || exit

I haven't tested with dex2oat, will check that.

The build-process for avian-pack currently runs, dunno what comes out.

Edit: you're right the build process is a mess:

• uses bashisms but does not enforce /bin/bash
• sets CC="gcc -fPic" on linux so the configure scripts looks for "/usr/bin/gcc -fPIC"
• re-defines _POSIX_C_SOURCE
• undefs GNU_SOURCE but does not redefine thus missing definitions
• many other errors

[Nanolx]

I made avian-pack finally compile on my linux-x86_64 box, but not (yet) an cross-compiled one for use on android.

• patch for avian-pack
• patch for avian sub-module
  • apply after make git-refresh

In case you're interested.

[Lanchon]

definitely use flashize! it takes a shell script and outputs a flashable zip. output from the shell script gets displayed in recovery. there's also optional output logging, zip extraction, and setting up environment (eg, binaries for the script, such as busybox).

also take a look at this:
https://justanapplication.wordpress.com/2014/09/30/building-the-android-runtime-art-for-mac-os-x-part-eight-testing-testing/
it seems art is a higher-level interface to the runtime than dalvikvm. but now that you are working on avian-pack, this is irrelevant.