Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] apktool cannot recompile obfuscated apks with shrinkResources = true after updating Gradle #3534

Open
jcrabanal opened this issue Mar 8, 2024 · 13 comments
Open

[BUG] apktool cannot recompile obfuscated apks with shrinkResources = true after updating Gradle #3534

jcrabanal opened this issue Mar 8, 2024 · 13 comments
Assignees
@iBotPeaches

Comments

@jcrabanal
Copy link

jcrabanal commented Mar 8, 2024 •
edited

Information

  1. Apktool Version: 2.9.3 (latest)
  2. Operating System: Windows
  3. APK From?: My own compiled APK
  4. Java Version: Several versions, not relevant I guess

I'm getting errors while decompiling -> recompiling my own apk. The issue started happening after updating Android Studio and the gradle plugin.

It only happens when I compile my app with obfuscation enabled and "shrinkResources true" in my build.gradle file. If I compile an apk with "shrinkResources false", apktool is able to decompile/recompile the apk.

Stacktrace/Logcat

I'm getting a few of those errors:

temp_a711ab5f-4bc1-4ff0-84f4-f5198dc96737\temp\res\values\styles.xml:1552: error: expected enum but got (raw string)

Here is the referenced part of styles.xml file of the apk that breaks apktool:

    <style name="Base.Widget.MaterialComponents.TextInputLayout" parent="@style/Widget.Design.TextInputLayout">
        <item name="android:textColorHint">@color/mtrl_indicator_text_color</item>
        <item name="boxBackgroundColor">@null</item>
        <item name="boxBackgroundMode">2</item>
        <item name="boxCollapsedPaddingTop">0.0dip</item>
        <item name="boxStrokeColor">@color/mtrl_outlined_stroke_color</item>
        <item name="boxStrokeErrorColor">@color/mtrl_error</item>
        <item name="boxStrokeWidth">@dimen/mtrl_textinput_box_stroke_width_default</item>
        <item name="boxStrokeWidthFocused">@dimen/mtrl_textinput_box_stroke_width_focused</item>
        <item name="counterOverflowTextAppearance">?textAppearanceCaption</item>
        <item name="counterOverflowTextColor">@color/mtrl_error</item>
        <item name="counterTextAppearance">?textAppearanceCaption</item>
        <item name="counterTextColor">@color/mtrl_indicator_text_color</item>
        <item name="endIconTint">@color/mtrl_outlined_icon_tint</item>
        <item name="enforceMaterialTheme">true</item>
        <item name="enforceTextAppearance">true</item>
        <item name="errorIconDrawable">@drawable/mtrl_ic_error</item>
        <item name="errorIconTint">@color/mtrl_error</item>
        <item name="errorTextAppearance">?textAppearanceCaption</item>
        <item name="errorTextColor">@color/mtrl_error</item>
        <item name="helperTextTextAppearance">?textAppearanceCaption</item>
        <item name="helperTextTextColor">@color/mtrl_indicator_text_color</item>
        <item name="hintTextAppearance">?textAppearanceCaption</item>
        <item name="hintTextColor">?colorPrimary</item>
        <item name="placeholderTextAppearance">?textAppearanceSubtitle1</item>
        <item name="placeholderTextColor">@color/mtrl_indicator_text_color</item>
        <item name="prefixTextAppearance">?textAppearanceSubtitle1</item>
        <item name="prefixTextColor">@color/mtrl_indicator_text_color</item>
        <item name="shapeAppearance">?shapeAppearanceSmallComponent</item>
        <item name="shapeAppearanceOverlay">@null</item>
        <item name="startIconTint">@color/mtrl_outlined_icon_tint</item>
        <item name="suffixTextAppearance">?textAppearanceSubtitle1</item>
        <item name="suffixTextColor">@color/mtrl_indicator_text_color</item>
    </style>

And here is the styles.xml in the apk that apktool recompiles OK:

    <style name="Base.Widget.MaterialComponents.TextInputLayout" parent="@style/Widget.Design.TextInputLayout">
        <item name="android:textColorHint">@color/mtrl_indicator_text_color</item>
        <item name="boxBackgroundColor">@null</item>
        <item name="boxBackgroundMode">outline</item>
        <item name="boxCollapsedPaddingTop">0.0dip</item>
        <item name="boxStrokeColor">@color/mtrl_outlined_stroke_color</item>
        <item name="boxStrokeErrorColor">@color/mtrl_error</item>
        <item name="boxStrokeWidth">@dimen/mtrl_textinput_box_stroke_width_default</item>
        <item name="boxStrokeWidthFocused">@dimen/mtrl_textinput_box_stroke_width_focused</item>
        <item name="counterOverflowTextAppearance">?textAppearanceCaption</item>
        <item name="counterOverflowTextColor">@color/mtrl_error</item>
        <item name="counterTextAppearance">?textAppearanceCaption</item>
        <item name="counterTextColor">@color/mtrl_indicator_text_color</item>
        <item name="endIconTint">@color/mtrl_outlined_icon_tint</item>
        <item name="enforceMaterialTheme">true</item>
        <item name="enforceTextAppearance">true</item>
        <item name="errorIconDrawable">@drawable/mtrl_ic_error</item>
        <item name="errorIconTint">@color/mtrl_error</item>
        <item name="errorTextAppearance">?textAppearanceCaption</item>
        <item name="errorTextColor">@color/mtrl_error</item>
        <item name="helperTextTextAppearance">?textAppearanceCaption</item>
        <item name="helperTextTextColor">@color/mtrl_indicator_text_color</item>
        <item name="hintTextAppearance">?textAppearanceCaption</item>
        <item name="hintTextColor">?colorPrimary</item>
        <item name="placeholderTextAppearance">?textAppearanceSubtitle1</item>
        <item name="placeholderTextColor">@color/mtrl_indicator_text_color</item>
        <item name="prefixTextAppearance">?textAppearanceSubtitle1</item>
        <item name="prefixTextColor">@color/mtrl_indicator_text_color</item>
        <item name="shapeAppearance">?shapeAppearanceSmallComponent</item>
        <item name="shapeAppearanceOverlay">@null</item>
        <item name="startIconTint">@color/mtrl_outlined_icon_tint</item>
        <item name="suffixTextAppearance">?textAppearanceSubtitle1</item>
        <item name="suffixTextColor">@color/mtrl_indicator_text_color</item>
    </style>

The line the stack trace is referencing to (1552) is this one:

<item name="boxBackgroundMode">2</item>

Apparently R8, the obfuscator, replaced an enum with its literal value. My humble guess is that apktool might not need to be that strict while checking this.

Questions to ask before submission

  1. Have you tried apktool d, apktool b without changing anything?: Yes
@jcrabanal jcrabanal changed the title [BUG] [BUG] apktool cannot recompile obfuscated apks with shrinkResources = true after updating Gradle Mar 8, 2024
@iBotPeaches
Copy link
Owner

Do you have a smaller apk sample to work from?

@jcrabanal
Copy link
Author

The sample projects of Android Studio seem to do the trick. Here you have.

Both APKs here decompile OK with "java -jar apktool_2.9.3.jar d [file]", but only shrinkResources_false.apk can be recompiled.

ApkToolTest.zip

@iBotPeaches iBotPeaches self-assigned this Mar 17, 2024
@jcrabanal
Copy link
Author

Setting android.enableNewResourceShrinker.preciseShrinking=false in my gradle.properties file and enabling shrinkResources = true seems to make it work again (decompile & recompile)

https://developer.android.com/build/releases/gradle-plugin#resource-shrinking

@iBotPeaches
Copy link
Owner

I made a sample that leveraged that feature when investigating another bug - maybe some overlap with this. Especially if this is the feature that automatically strips namespace context from resources.

@dbgp
Copy link

dbgp commented Apr 12, 2024

maybe this is the same condition.

➜  apktool java -jar apktool2.9.3.jar b -o venus.apk m171
I: Using Apktool 2.9.3
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes6 folder into classes6.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes3 folder into classes3.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes4 folder into classes4.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes5 folder into classes5.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes2 folder into classes2.dex...
I: Checking whether resources has changed...
I: Building resources...
W: /Users/dbgp/Downloads/apktool/m171/res/values/styles.xml:6670: error: expected enum but got (raw string) uniform.
W: /Users/dbgp/Downloads/apktool/m171/res/values/styles.xml:6684: error: expected enum but got (raw string) uniform.
W: error: failed linking references.
brut.androlib.exceptions.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [/var/folders/jm/1g3t6w6n5692dnhz5jk0gph40000gn/T/brut_util_Jar_56673902650333380682946784968269564306.tmp, link, -o, /var/folders/jm/1g3t6w6n5692dnhz5jk0gph40000gn/T/APKTOOL8336347317507460300.tmp, --package-id, 127, --min-sdk-version, 24, --target-sdk-version, 33, --rename-manifest-package, one.mixin.messenger.venus, --rename-instrumentation-target-package, one.mixin.messenger.venus, --version-code, 1070100, --version-name, 1.7.1, --no-auto-version, --no-version-vectors, --no-version-transitions, --no-resource-deduping, --allow-reserved-package-id, --no-compile-sdk-metadata, --warn-manifest-validation, -e, /var/folders/jm/1g3t6w6n5692dnhz5jk0gph40000gn/T/APKTOOL5073417883081723115.tmp, -0, arsc, -I, /Users/dbgp/Library/apktool/framework/1.apk, --manifest, /Users/dbgp/Downloads/apktool/m171/AndroidManifest.xml, /Users/dbgp/Downloads/apktool/m171/build/resources.zip]

related two lines:

<style name="Widget.MaterialComponents.MaterialCalendar.HeaderSelection" parent="@style/Widget.AppCompat.TextView">
        <item name="android:textAppearance">?textAppearanceHeadline4</item>
        <item name="android:textColor">?colorOnPrimary</item>
        <item name="android:ellipsize">end</item>
        <item name="android:layout_gravity">start|center|top</item>
        <item name="android:layout_width">fill_parent</item>
        <item name="android:layout_height">wrap_content</item>
        <item name="android:maxLines">@integer/mtrl_calendar_selection_text_lines</item>
        <item name="autoSizeMaxTextSize">34.0sp</item>
        <item name="autoSizeMinTextSize">2.0sp</item>
        <item name="autoSizeTextType">uniform</item> <!-- this is line 6670 -->
    </style>

<style name="Widget.MaterialComponents.MaterialCalendar.HeaderTitle" parent="@style/Widget.AppCompat.TextView">
        <item name="android:textAppearance">?textAppearanceOverline</item>
        <item name="android:textColor">?colorOnPrimary</item>
        <item name="android:ellipsize">end</item>
        <item name="android:maxLines">1</item>
        <item name="autoSizeMaxTextSize">10.0sp</item>
        <item name="autoSizeMinTextSize">2.0sp</item>
        <item name="autoSizeTextType">uniform</item>  <!-- this is line 6684 -->
    </style>

after decode , this item(autoSizeTextType) value is 1, the same error rised while re-building. I change its value to uniform manually according to this link, but this error still happen.

@iBotPeaches iBotPeaches mentioned this issue Apr 22, 2024
Closed
@inotia00 inotia00 mentioned this issue May 1, 2024
Open
3 tasks
@rikumi
Copy link

rikumi commented May 11, 2024

Maybe apktool could detect those values by simply looking for their attribute names in the attrs.xml file and create pseudo keys for them there. For example:

<!-- styles.xml -->
<item name="someAttribute">1</item>

<!-- attrs.xml -->
<attr name="someAttribute">
    <enum name="anotherKey" value="0" />
</attr>

can be turned into:

<!-- styles.xml -->
<item name="someAttribute">apktool_generatedStubEnumKey_1</item>

<!-- attrs.xml -->
<attr name="someAttribute">
    <enum name="anotherKey" value="0" />
    <enum name="apktool_generatedStubEnumKey_1" value="1" />
</attr>

@iBotPeaches
Copy link
Owner

That is one possibility, but Apktool during processing may not have enough information at time of each AXML file disassembly to know at that exact moment if the scalar should be a enum or not.

I've been experimenting with relaxing the aapt2 build requirement to see if can survive with the scalar instead of re-making the enum.

https://github.com/iBotPeaches/platform_frameworks_base/tree/apktool-2.10.x

@kefir500 kefir500 mentioned this issue May 17, 2024
Closed
@cyberboh cyberboh mentioned this issue May 19, 2024
Merged
@iBotPeaches iBotPeaches mentioned this issue May 26, 2024
Closed
@HassanMirza01
Copy link

HassanMirza01 commented May 27, 2024 •
edited

i have successfully recompiled apks which were compiled with shrinkResources = true
i have added some attrs and replaced all matching lines in res/layout/*.xml files, like that

  1. Added attributes to attrs.xml file:
    <attr name="layout_constraintEnd_toEndOf" format="reference">
    <enum name="parent" value="0" />
    </attr>

  2. Changed app:layout_constraintEnd_toEndOf="0" to app:layout_constraintEnd_toEndOf="parent" and that built a fully working apk, tried 3 apks and all are working totally fine.

Conclusion: If we can include all attrs of such type from google's source or a generic file by default in each decompile and these values get auto detected or changed in res/layout and res/xml folders ?

@jcrabanal
Copy link
Author

If you set android.enableNewResourceShrinker.preciseShrinking=false in your gradle.properties, then you can enable shrinkResources = true again and produce APKs that apktool can process, but gradle shows this message...

"The option setting 'android.enableNewResourceShrinker.preciseShrinking=false' is deprecated.
The current default is 'true'.
It will be removed in version 9.0 of the Android Gradle plugin."

We may not be able to do this anymore in the future. Any update on this issue?

@iBotPeaches
Copy link
Owner

We may not be able to do this anymore in the future. Any update on this issue?

Not at this time. Changes occurred in AOSP and I can no longer build aapt2 and its stretched past my knowledge. So I'm blocked on everything until I learn more on how to resolve that.

context: https://groups.google.com/g/android-building/c/TvKKsSLSlPY

@HassanMirza01
Copy link

We may not be able to do this anymore in the future. Any update on this issue?

Not at this time. Changes occurred in AOSP and I can no longer build aapt2 and its stretched past my knowledge. So I'm blocked on everything until I learn more on how to resolve that.

context: https://groups.google.com/g/android-building/c/TvKKsSLSlPY

i used "lunch aosp_arm64-trunk_staging-eng" and i can build aapt and aapt2 fine with latest repo sync which removed lunch menu.

@iBotPeaches
Copy link
Owner

i used "lunch aosp_arm64-trunk_staging-eng" and i can build aapt and aapt2 fine with latest repo sync which removed lunch menu.

thanks. I'll give that a shot tonight.

@HassanMirza01
Copy link

image

this version it compiled,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@iBotPeaches iBotPeaches

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@iBotPeaches @rikumi @HassanMirza01 @jcrabanal @dbgp