Skip to content

Assembly fails for version >= 0.30.0 with "deduplicate: different file contents found in the following" error #257

@jfuentescpp

Description

@jfuentescpp

In order to reproduce:

build.sbt

name := "assembly_conflict"
version := "0.0.1"
scalaVersion := "2.12.11"

libraryDependencies ++= List(
  "com.lightstep.tracer" % "lightstep-tracer-jre" % "0.30.1",
  "com.lightstep.tracer" % "tracer-grpc" % "0.30.0",
  "io.grpc" % "grpc-netty" % "1.23.0",
  "io.netty" % "netty-tcnative-boringssl-static" % "2.0.25.Final",
  "org.slf4j" % "slf4j-simple" % "1.7.25"
)

project/plugins.sbt

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.15.0")

project/build.properties

sbt.version=1.1.1

then sbt assembly give erros:

[error] 13 errors were encountered during merge
[error] java.lang.RuntimeException: deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-buffer/jars/netty-buffer-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-http/jars/netty-codec-http-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-http2/jars/netty-codec-http2-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-socks/jars/netty-codec-socks-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec/jars/netty-codec-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-common/jars/netty-common-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-handler-proxy/jars/netty-handler-proxy-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-handler/jars/netty-handler-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-resolver/jars/netty-resolver-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-tcnative-boringssl-static/jars/netty-tcnative-boringssl-static-2.0.25.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-transport/jars/netty-transport-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Collector$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Collector$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Collector.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Collector.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$2.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$2.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$Builder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$Builder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$ValueCase.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$ValueCase.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValueOrBuilder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValueOrBuilder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter$Builder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter$Builder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/ReporterOrBuilder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/ReporterOrBuilder.class
[error]         at sbtassembly.Assembly$.applyStrategies(Assembly.scala:161)
[error]         at sbtassembly.Assembly$.x$1$lzycompute$1(Assembly.scala:43)
[error]         at sbtassembly.Assembly$.x$1$1(Assembly.scala:41)
[error]         at sbtassembly.Assembly$.stratMapping$lzycompute$1(Assembly.scala:41)
[error]         at sbtassembly.Assembly$.stratMapping$1(Assembly.scala:41)
[error]         at sbtassembly.Assembly$.inputs$lzycompute$1(Assembly.scala:86)
[error]         at sbtassembly.Assembly$.inputs$1(Assembly.scala:76)
[error]         at sbtassembly.Assembly$.apply(Assembly.scala:103)
[error]         at sbtassembly.Assembly$.$anonfun$assemblyTask$1(Assembly.scala:278)
[error]         at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error]         at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:39)
[error]         at sbt.std.Transform$$anon$4.work(System.scala:66)
[error]         at sbt.Execute.$anonfun$submit$2(Execute.scala:262)
[error]         at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]         at sbt.Execute.work(Execute.scala:271)
[error]         at sbt.Execute.$anonfun$submit$1(Execute.scala:262)
[error]         at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:174)
[error]         at sbt.CompletionService$$anon$2.call(CompletionService.scala:36)
[error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error]         at java.lang.Thread.run(Thread.java:748)
[error] (assembly) deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-buffer/jars/netty-buffer-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-http/jars/netty-codec-http-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-http2/jars/netty-codec-http2-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-socks/jars/netty-codec-socks-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec/jars/netty-codec-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-common/jars/netty-common-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-handler-proxy/jars/netty-handler-proxy-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-handler/jars/netty-handler-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-resolver/jars/netty-resolver-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-tcnative-boringssl-static/jars/netty-tcnative-boringssl-static-2.0.25.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-transport/jars/netty-transport-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Collector$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Collector$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Collector.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Collector.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$2.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$2.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$Builder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$Builder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$ValueCase.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$ValueCase.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValueOrBuilder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValueOrBuilder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter$Builder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter$Builder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/ReporterOrBuilder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/ReporterOrBuilder.class
[error] Total time: 8 s, completed Aug 12, 2020 12:58:41 PM

Issue can be worked around with something like code below. It worked fine before 0.30.0 release.

assemblyMergeStrategy in assembly := {
  case PathList("com", "lightstep", "tracer", "grpc", xs @ _*) =>
    MergeStrategy.first
  case x => MergeStrategy.defaultMergeStrategy(x)
}

Checking the dependency tree, the conflict is from libraries that lightstep-tracer-jre depends:
Screenshot 2020-08-12 at 13 09 13

Why is this error happening? Content of the source code seems quite similar but I didn't properly check. Some difference in the .class?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions