both org.scala-lang and org.typelevel versions of scala-library are sometimes pulled in

[tpolecat]

Via @nigredo-tori here is a reproduction that shows both libs being pulled in.

tmp$ mkdir tls-test
tmp$ cd !$
cd tls-test
tls-test$ cat - > build.sbt
scalaVersion in ThisBuild := "2.12.3"
libraryDependencies += "org.http4s" %% "http4s-dsl" % "0.17.0-M3"
tls-test$ mkdir project
tls-test$ cat - > project/build.properties
sbt.version=0.13.16
tls-test$ sbt 'show managedClasspath' | grep 'scala-library'
[info] Resolving org.scala-lang#scala-library;2.10.6 ...
[info] Resolving org.scala-lang#scala-library;2.10.6 ...
[info] Resolving org.typelevel#scala-library;2.12.1 ...
[info] * Attributed(/Users/rnorris/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.3.jar)
[info] * Attributed(/Users/rnorris/.ivy2/cache/org.typelevel/scala-library/jars/scala-library-2.12.1.jar)
tls-test$ 

Adding an exclusion "fixes" the problem, however one could imagine cases where this wouldn't work if the libs diverge.

tls-test$ cat - >> build.sbt 
excludeDependencies += "org.typelevel" % "scala-library"
tls-test$ sbt 'show managedClasspath' | grep 'scala-library'
[info] Resolving org.scala-lang#scala-library;2.12.3 ...
[info] * Attributed(/Users/rnorris/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.3.jar)
tls-test$ 

Can we review what the desired/expected behavior is here? @eed3si9n has offered to help us track this down. As a lead-in perhaps this observation would be of use.

[rossabaker]

if the libs diverge

My understanding of the compatibility guarantee is that this should not happen: all code compiled with TLS should link with the LBS runtime, and vice versa. Am I reading too much into that?

I've received a few complaints from https + sbt-assembly users. Perhaps the right thing to do for library authors using TLS is to post process the POM to reference LBS?

[nigredo-tori]

all code compiled with TLS should link with the LBS runtime

I assume you mean "of the same version" here. E.g. code compiled against TLS 2.12.3 should work with LBS 2.12.3. However, once SBT sees the two as separate libraries, the versions can diverge. In my case I had TLS 2.12.1 with LBS 2.12.3. Here excluding TLS works, since it's replaced with a backwards compatible version. If, however, I had TLS 2.12.3 and LBS 2.12.1, this would have had the opposite effect (making the crash deterministic).

[rossabaker]

I also thought that scala-library was bidirectionally compatible across patch releases since 2.10. I would rewrite the POM from TLS-2.12.3 to LBS-2.12.3, so we'd be fine, but I'm also surprised that patch versions matter.

[tpolecat]

Re: divergence I think we're seeing it. LBS added Properties.coloredOutputEnabled and TLS added ValueOf, so if you need both you're out of luck. If I'm reading the smoke correctly.

[milessabin]

There is a TLS version with both.

[tpolecat]

So which library should people be getting? Should sbt always prefer TLS?

[milessabin]

As far as I can tell the problem here is the mix of 2.12.1 and 2.12.3. You should always use the most recent. If you're using ValueOf then TLS-4 for 2.12.3 is what you need. But if you're not using ValueOf then LBS 2.12.3 should be fine.

[tpolecat]

How should sbt resolve this? Find the most recent version and select TLS for it? So given LBS 2.12.3 and TLS 2.12.1 it should go for TLS 2.12.3? What if it doesn't exist yet?

[eed3si9n]

Would it make sense to put the diverged bits to another library that gets added in addition to the org.scala-lang/Lightbend standard library?

[milessabin]

I think that if http4s is publishing binaries built with TLS they really need to republish for the most recent TLS version. I can commit to getting a TLS version out within a single digit number of days of the corresponding LBS release.

[tpolecat]

I think this diminishes the appeal of TLS quite a bit. The promise has been that you can use it without any extra requirements for you or for your downstream users. What do we think of Eugene's idea? If TLS didn't publish scala-library at all then we might have an easier path.

[milessabin]

Is http4s using ValueOf?

[milessabin]

Looking at http4s master it seems not. In fact it looks as though the only TLS feature it's using is -Ypartial-unification which is included in 2.12.x anyway. So I'm not quite sure I understand why it's being published for 2.12.x with a TLS dependency at all.

[rossabaker]

http4s is publishing with the latest TLS version, and does not use ValueOf. We adopted TLS early to help test it, with the understanding that it was transparent to downstream users. We cross build for 2.11, so we still need partial unification by some means.

I like Eugene's idea for resolving the scala-library conflict. However, I expect a TLS-compiled library would still have a declared org.typelevel runtime dependency on that extras library, even if that dependency is not used. So it would resolve the conflict but still not achieve transparency.

[milessabin]

Then I'm confused ... the http4s artefact being pulled in in the example at the top of this ticket is relative to 2.12.1 ... is there a later published artefact that is built against 2.12.3 which should be used instead?