Problems with Private Repositories

[guv]

For my private Nexus repository I use the following setup in project.clj:

 :repositories ^:replace [["releases" {:url "https://MYREPOSITORY.TLD/content/repositories/releases"}]
                           ["snapshots" {:url "https://MYREPOSITORY.TLD/content/repositories/snapshots"}]]

This works like a charm in Leiningen (Leiningen 2.5.0 on Java 1.8.0_40 64-Bit Server VM) when I also provide my certificate's public key since the server uses a self-signed certificate. This is done in profiles.clj as follows:

{:user {:certificates ["/home/MYUSER/.lein/MYREPOSITORY.pem"]}}

It is also possible to specify the :certificates settings in the actual project.clj.

However, CCW couldn't handle this pre 0.32 and also does not handle this well in 0.33.

With the :certificates settings in profiles.clj I get the following error:

Could not transfer artifact lein-pprint:lein-pprint:pom:1.1.2 from/to releases (https://MYREPOSITORY.TLD/content/repositories/releases): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

With the :certificates settings in project.clj the following error is reported:

java.lang.IllegalArgumentException: No matching field found: getConnectionManager for class org.apache.maven.wagon.providers.http.HttpWagon
 at clojure.lang.Reflector.getInstanceField (Reflector.java:271)
    clojure.lang.Reflector.invokeNoArgInstanceMember (Reflector.java:315)
    leiningen.core.ssl/fn (ssl.clj:91)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invoke (core.clj:624)
    clojure.core$memoize$fn__5097.doInvoke (core.clj:5846)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.lang.Var.invoke (Var.java:379)
    leiningen.core.project$load_certificates.invoke (project.clj:764)
    leiningen.core.project$init_project.invoke (project.clj:878)
    leiningen.core.project$read.invoke (project.clj:952)
    leiningen.core.project$read.invoke (project.clj:953)
    leiningen.core.main$_main$fn__6160.invoke (main.clj:388)
    leiningen.core.main$_main.doInvoke (main.clj:385)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invoke (core.clj:624)
    ccw.leiningen.main$_main.doInvoke (main.clj:66)
    clojure.lang.RestFn.invoke (RestFn.java:3894)
    clojure.lang.Var.invoke (Var.java:676)
    clojure.lang.AFn.applyToHelper (AFn.java:403)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.core$apply.invoke (core.clj:624)
    clojure.main$main_opt.invoke (main.clj:315)
    clojure.main$main.doInvoke (main.clj:420)
    clojure.lang.RestFn.invoke (RestFn.java:3894)
    clojure.lang.Var.invoke (Var.java:676)
    clojure.lang.AFn.applyToHelper (AFn.java:403)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.main.main (main.java:37)

Both cases work for lein deps and lein repl on command line.

The show stopper here is that although all dependencies are available in the local .m2 folder, CCW tries to download them anyway and no REPL is started, which makes me unable to get any work done with CCW 0.33 on these projects. :(

[guv]

If anyone else is troubled by this, you can downgrade to 0.31.1 where you will be able to start up a REPL despite the dependency problems.

[laurentpetit]

I will take a serious look at it. Maybe it is the "trigger issue" that I've been waiting for a while to help me decide it's time to remove leiningen-core and its dependencies from ccw's classpath and only talk to leiningen via its real true stable interface : the command line when started as an external process.

[laurentpetit]

Hello,

I can see 2 kinds of problems here:

• First one, CCW is unable to properly compute the Leiningen Classpath Container. Am I right? Thus in Eclipse build path, you see no classpath container.
• Second one, you are probably trying to start your project with the option of not launching leiningen, but rather use the Eclipse project's build path? If I am right, then it is indeed a consequence of the first problem. A workaround, before I can release a fix to this, would be to go to the Clojure preferences , General, and check the box to use leiningen to start your project REPL, for some time.

[guv]

You are wrong there. I use the "Launch Leiningen Projects with Leiningen" option since it has been available.
I switched that option off for a test: The REPL starts up but the dependencies are missing (ClassNotFoundException etc).
Reactivating the leiningen startup leads to the exceptions reported above.

Hence, I'll keep 0.31.1 until that is fixed.

[laurentpetit]

Ok, it's a shame. I would really have expected the REPL scenario to work.
Do you know how it would be possible for me to reproduce the issue so that I understand it better? Do I need to create my own private repository? Is that a lot of work? How do I get started?

[laurentpetit]

Can you provide as much info as possible on the issue with starting the leiningen REPL from CCW? Can you go to the Debug perspective and look what the java command line is in the process properties? Do you see something interesting in the workspace's .metadata/.log file?

[guv]

Sorry for the late response. Sonatype Nexus [1] has good documentation [2] for installation and management.
I am using Apache with a self-signed certificate in front of Nexus [3].

I am not sure whether it is worthwile to setup a private repository to fix the issue in CCW compared to just use Leiningen. Remember that Leiningen handles this setup out-of-the-box provided that :repositories and :certificates options are provided.

Apart from the above exceptions there was nothing related in the log file.

The older issue #708 also refers to CCWs hand-tailored usage of Leiningen.

[1] http://www.sonatype.org/nexus/go/
[2] http://books.sonatype.com/nexus-book/reference/install.html
[3] http://books.sonatype.com/nexus-book/reference/install-sect-proxy.html

[laurentpetit]

ok, getting to the point where I have an almost operational nexus with an nginx https reverse proxy in front of it :-)

[laurentpetit]

My configuration:

• nginx listens to https on port 443 and fowards to nexus on port 8081
• ~/.lein/profiles.clj is set correctly with {:user {:certificates ["/usr/local/etc/nginx/nginx.crt"]}}
• my test project.clj contains repositories ^:replace [["releases" {:url "https://localhost/nexus/content/repositories/central"}] ["snapshots" {:url "https://localhost/nexus/content/repositories/clojars"}]])

Now I try leiningen on the command line.

With leiningen 2.5.0, works like a charm:

% cd test
test % lein deps :tree
warn
 [clojure-complete "0.2.3" :scope "test" :exclusions [[org.clojure/clojure]]]
 [org.clojure/clojure "1.6.0"]
 [org.clojure/tools.nrepl "0.2.6" :scope "test" :exclusions [[org.clojure/clojure]]]
test %

With leiningen 2.5.1, there's a problem:

lein version
java.lang.IllegalArgumentException: No matching field found: getConnectionManager for class org.apache.maven.wagon.providers.http.HttpWagon
 at clojure.lang.Reflector.getInstanceField (Reflector.java:271)
    clojure.lang.Reflector.invokeNoArgInstanceMember (Reflector.java:315)
    leiningen.core.ssl/fn (ssl.clj:91)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invoke (core.clj:624)
    clojure.core$memoize$fn__5097.doInvoke (core.clj:5846)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.lang.Var.invoke (Var.java:379)
    leiningen.core.project$load_certificates.invoke (project.clj:764)
    leiningen.core.project$activate_middleware.invoke (project.clj:771)
    leiningen.core.project$init_project.invoke (project.clj:878)
    leiningen.core.project$init_project.invoke (project.clj:885)
    leiningen.core.main$default_project.invoke (main.clj:376)
    leiningen.core.main$_main$fn__6160.invoke (main.clj:389)
    leiningen.core.main$_main.doInvoke (main.clj:385)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.lang.Var.invoke (Var.java:379)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.core$apply.invoke (core.clj:624)
    clojure.main$main_opt.invoke (main.clj:315)
    clojure.main$main.doInvoke (main.clj:420)
    clojure.lang.RestFn.invoke (RestFn.java:436)
    clojure.lang.Var.invoke (Var.java:388)
    clojure.lang.AFn.applyToHelper (AFn.java:160)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.main.main (main.java:37)

This problem disappears in 2.5.1 if I remove the ~/.lein/profiles.clj (thus the certificate):

lein version
Leiningen 2.5.1 on Java 1.8.0_45 Java HotSpot(TM) 64-Bit Server VM

But then when I invoke lein deps :tree in my test, project, there will be a crash, as expected:

cd test
test % lein deps :tree
Could not transfer artifact org.clojure:clojure:pom:1.6.0 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Could not transfer artifact org.clojure:tools.nrepl:pom:0.2.6 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Could not transfer artifact clojure-complete:clojure-complete:pom:0.2.3 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
test %

So it seems that Leiningen 2.5.1 has a regression. And CCW has started embedding 2.5.1 a few releases ago.

Now I also did the test with Leiningen 2.5.2, and the error with PKIX certificates comes back:

lein version
Leiningen 2.5.2 on Java 1.8.0_45 Java HotSpot(TM) 64-Bit Server VM
tmp test % cd test
test % lein deps :tree
Could not transfer artifact org.clojure:clojure:pom:1.6.0 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Could not transfer artifact org.clojure:tools.nrepl:pom:0.2.10 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Could not transfer artifact clojure-complete:clojure-complete:pom:0.2.3 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.

So I tried to also place a profiles.clj inside my project directory, and here's the result (not working again):

ein deps :tree
(WARNING: user-level profile defined in project files.)
Could not transfer artifact org.clojure:clojure:pom:1.6.0 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Could not transfer artifact org.clojure:tools.nrepl:pom:0.2.10 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Could not transfer artifact clojure-complete:clojure-complete:pom:0.2.3 from/to releases (https://localhost/nexus/content/repositories/central): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.

[laurentpetit]

tldr: it seems that only leiningen 2.5.0 works correctly with private proxy and self signed certificates. It's not (entirely) a CCW issue ...

So I will continue my tests with Leiningen 2.5.0 embedded in CCW, and try to fix things for this version

[laurentpetit]

OK, so here's the trick to have 0.33.0 let you open a repl, or use Alt + L L to launch arbitrary commands: Go to the Preferences > Clojure > General and find the 2.5.0 leiningen jar in your computer, e.g. /Users/laurentpetit/.lein/self-installs/leiningen-2.5.0-standalone.jar.

CCW will then use this jar for external commands and launching a repl with leiningen.
It will still have problems for when it uses its embedded leiningen (for the project build path, and when invoking the wizard to create a new project).

[laurentpetit]

I'm considering moving back the embedded leiningen to 2.5.0 ...

Will you also please file an issue about this to Leiningen, since it will affect you on the command line as soon as you upgrade leiningen?