Repository navigation
Extract r8 rules based on version - #511
snazhmudinov wants to merge 16 commits into
Conversation
Add runfiles dependency and data attribute to both AAR and JAR proguard extractor targets, and read the R8 version from r8.version at runtime.
Thread r8_version from main() through _Main and into all proguard_extractor_lib functions. No behavioral change yet.
ExtractEmbeddedProguardFromJar now matches r8-from-X-upto-Y directories against the actual R8 version and only extracts rules whose version range covers the current R8 version. Falls back to legacy META-INF/proguard/ rules when no targeted range matches.
Update ExtractEmbeddedProguardFromAar to prioritize version-targeted R8 rules from classes.jar, falling back to proguard.txt at the AAR root when no targeted range matches. Remove unused ExtractR8Rules and ExtractEmbeddedProguardFromAarLegacy functions.
| outs = [ | ||
| "r8.version", | ||
| ], | ||
| cmd = "$(location :r8) --version | awk -F' ' '{ print $$2 }' >$(OUTS)", |
There was a problem hiding this comment.
This isn't very robust - it assumes the version format is R8 8.9.35 (build ...) (which is the case today, but can change in the future).
There was a problem hiding this comment.
Maybe could use the r8-version.properties resource?
e.g.
unzip -p $(location r8_jar) r8-version.properties | sed -En 's/^version.version=(.+)$/\1/p'There was a problem hiding this comment.
The result looks a bit odd to me:
version.sha=afec818d6658b003ba394e10107b7dfc565dd94b
releaser=go/r8bot (luci-r8-custom-ci-archive-0-stqp)
version-file.version.code=1
There was a problem hiding this comment.
Ah I see. In my case there was also a version.version property key with the specific version. I suppose when it's built from an untagged version that doesn't exist.
damianw
left a comment
There was a problem hiding this comment.
Amazing!
Fwiw this doesn't fully solve #491. Beyond respecting the R8 version, the prioritization of rules vs proguard/proguard.txt are also part of that issue. Solving that would require extracting the rules from individual jars instead of a single deploy jar.
|
Specifically, this is where the rules are currently being extracted. It can't really be done here, because all the resources are in one jar. So applying logic for "if has |
Ah that makes sense. I tried to run the extractor on jars, which get merged into deploy.jar. Commit. |
|
Great! That all makes sense to me. |
Depending on @bazel_tools//tools/python/runfiles puts the bazel_tools
repository root on the Python path, and its tools package shadows this
repository's tools package when rules_android is used as an external
repository. The extractors then fail with "No module named 'tools.android'".
Rlocation("rules_android/...") also misses the canonical rules_android+
directory under Bzlmod.
Read r8.version from next to the extractor script instead, falling back to
the runfiles manifest when there is no runfiles tree.
Extracting from each runtime JAR puts one --input_jars flag per JAR on the
command line. Large apps have more than 10,000 runtime JARs, which exceeds
the OS argument length limit ("Argument list too long"). Pass the flags in
an absl --flagfile instead.
Match AGP's FilterShrinkerRulesTransform and AarTransform: - Accept every directory form: r8, r8-from-<X>, r8-upto-<Y> and r8-from-<X>-upto-<Y>. Only the two-sided form matched, so libraries such as kotlinx-coroutines-android (r8-from-1.6.0, r8-upto-3.0.0) fell back to their legacy rules, which keep AndroidDispatcherFactory. - Match directory names case-insensitively, accept versions with a suffix such as 8.0.0-alpha01 or 9.5.2-dev, and treat a bound that can't be parsed as no constraint instead of failing. - If a JAR, or an AAR's classes.jar, has any R8-targeted rules, use only the matching ones and skip the legacy rules even if none match.
aar_import only passes --extract_r8_rules when --aar_import_extract_r8_rules is set (off by default), but the extractor stopped reading it, so every AAR's classes.jar rules were used and the flag did nothing. Only consider classes.jar rules when it is set again, and use proguard.txt otherwise, as before.
|
I tried this out on a large app with R8 9.5.28 and found a few issues. I've opened fixes as PRs against your branch, so you can take whichever you want:
The three PRs merge cleanly in any order. With all three, our app's coroutines dispatcher is constructed directly (no ServiceLoader), and the DEX is slightly smaller than with our previous |
Read the R8 version without the runfiles library
Pass embedded proguard extractor inputs in a param file
Select embedded R8 rules the way AGP does
|
|
||
| public class R8Version { | ||
| public static void main(String[] args) { | ||
| System.out.println(com.android.tools.r8.Version.LABEL); |
There was a problem hiding this comment.
I am really not a fan of specializing behavior based on the R8 version. It's unclear to me why this needs to happen, both from the linked issue and the body of the PR.
There was a problem hiding this comment.
Agreed, rules_android shouldn't need to know R8 versions at all. R8 already picks META-INF/com.android.tools/r8-from-*-upto-* rules against its own version: it does that for each program input (EmbeddedRulesExtractor), and it ships the same logic as a command-line tool, com.android.tools.r8.ExtractR8Rules.
The bug in #491 is that the choice is made per library. If a JAR has rules for the running R8, its legacy META-INF/proguard rules are ignored. Because rules_android gives R8 one merged deploy JAR, that choice is made for the whole app: once any library ships R8-targeted rules, every other library's legacy rules are dropped.
I've opened a draft: #585 as an alternative. When r8_extract_embedded_proguard_specs is on, it runs ExtractR8Rules from the toolchain's R8 on each runtime JAR.
Addresses #491.
Additional info used: https://developer.android.com/topic/performance/app-optimization/library-optimization#support-different