t/packlist.t test fails on clean systems

[sjn]

Hi!

t/packlist.t fails on systems where I haven't upgraded List::MoreUtils (meaning, ExtUtils::Install hasn't added any .packlist file)

Here's the output of 'perl -Mblib t/packlist.t':

not ok 1 - Found packlist for List::MoreUtils
#   Failed test 'Found packlist for List::MoreUtils'
#   at t/packlist.t line 14.
ok 2 - Returns false on not installed
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
ERROR: no packlist file found: ''
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# Tests were run but no plan was declared and done_testing() was not seen.

-- Salve

[sjn]

Maybe prepare the test environment by setting up an install tree (e.g. with local::lib) in /tmp and then direct Module::Build::CleanInstall's attention there? Depending on a particular system setup for your tests is going to break in all kinds of interesting ways, so you might as well set up the test environment in such a way that you can test your module in a predictable way.

[jberger]

While you do offer an interesting suggestion (local::lib), I do have List::MoreUtils listed as a build_requires prereq https://github.com/jberger/Module-Build-CleanInstall/blob/master/Build.PL#L19 . Part of the reason not to have a clean test environment is to see how and when such things do fail in the real world. In this case, since its a declared dependency (and on one of the most common modules), I think this is probably acceptable loss. But I will leave it open for a bit to think about it.

[sjn]

I have List::MoreUtils installed with my system Perl (Ubuntu 12.10; Perl 5 version 14.2; clean install). My (meaning; Ubuntu's) List::MoreUtils version is 0.33. The newest released version in CPAN is 0.33. The system package containing List::MoreUtils doesn't install any .packlist files - in fact those files are removed explicitly so that the users of a system can see which modules they have successfully installed.

My (automated) build system now fails to install Module::Build::CleanInstall, meaning my (automated) install of Pinto fails. A bit annoying, especially when thinking about the amount of people who are trying to install Pinto these days (Pinto was featured on the FLOSS weekly podcast a few weeks ago).

Re: testing "how things are in the real world": That's fine, and I'd have no problems helping you doing this by running you test - if it can be automated (eg. your tests giving you useful information when being reported with Test::Reporter), and if this doesn't affect the build, test and install process in a pointless negative way.

Others might disagree with me though. :)

On that note, tests are good for detecting bugs and regressions and environment assumptions in your own code, but perhaps not well suited for "figuring out how and when such things fail in the real world". "The real world" is a messy place, and if you're going to learn about it by publishing failing tests, you'll just end up alienating lots of people (and with this, make people not use your modules, and hence limit your ability to learn anything in this way anyway.)

So, please consider this: A test failure communicates that there is something wrong somewhere, and that the user installing your software (e.g. me) should spend some attention and time looking into it and perhaps communicate the failure to the author(s). If the test failures won't be fixed (meaning they're deemd as an "acceptable loss"), the you're just giving a signal that other people should be careful with what you publish. If that's your intention, then do tell! I'm sure lots of people would love to know - myself including. :)

Cheers! :)

-- Salve

[sjn]

BTW, it seems to me that "test_requires" is better for describing your dependency to List::MoreUtils than "build_requires". Although strictly speaking, you need that module to have been manually installed (something I think is not possible to declare inside Build.PL).

Hope you find a solution to this soon! :)

[jberger]

If the system installed List::MoreUtils doesn't install the .packlist, then a) it is both broken and b) Module::Build::CleanInstall won't work as it depends on the .packlist files.

M::B::CI has a perfect test record: http://www.cpantesters.org/distro/M/Module-Build-CleanInstall.html#Module-Build-CleanInstall-0.05 so I don't think anything in its Build.PL needs to be changed. "Acceptable loss" probably isn't the phrase I mean, what I mean is, because I test a real-installed module, now I'm aware of the problem with (broken) package manager versions of List::MoreUtils. I'm not sure that there's much I can do since if you are correct, M::B::CI won't function on such systems.

[jberger]

Also, I don't see how Pinto factors in to this. Can you explain. It doesn't look like Pinto depends on M::B::CI.

[jberger]

NM. Metacpan's list doesn't show the dependency, but Pinto is using it directly.

[sjn]

Pinto 0.068 lists M::B::CI as a build dependency: https://metacpan.org/source/THALJEF/Pinto-0.068/Build.PL#L10

AFAIK, both Redhat (RPM) and Debian (DPKG) -based systems have a long tradition of removing .packlist files, and there's probably some good reason for it (here's "real world" you were looking for ;)).

[jberger]

Ok here's the latest from IRC.

There is a problem with testing List::MoreUtils since a vendor-package doesn't come with a packlist. In principle I'm ok not necessarily testing this behavior, but I have been comfortable with that test before now.

There is a deeper problem with multiple copies of modules in @inc.

if @inc is like ('path/a', 'path/b', 'path/c'), and the module you are uinstalling is under 'path/c', and you then install the updated version under 'path/a', it might break other perl apps\scripts that doesnt have the same inc as you had when uninstalling a module

This is a problem, but there are several suggestions.

1. require that the module only install into path/c
2. don't uninstall from path/c. only uninstall from the same path as the installation target
3. die in crappy circumstances
4. do what it does currently
5. deprecate MBCI

I think that 2 is reasonable, if it can be done correctly, however, this would leave a file in path/c which the loader would find, which is unexpected under the concept of MBCI. This perhaps suggests that MBCI cannot work as hoped and should be deprecated.

Thoughts?

[sjn]

MBCI has some broken assumptions about how things get installed. The way things get installed are also broken (but a known problem, with efforts to fix it being done at the QA Hackathon in Lancaster and in the CPAN toolchain gang).

Right now, 2) is probably a good idea. I'd also warn about any brokeness (e.g. that some files were not uninstalled).

Simplest option? 5)

[jberger]

The problem about the loader finding files later in @INC only occurs for Perl files in a dist. As MBCI was created more for cleaning static/non-perl files in webapps, to completely deprecate is throwing the baby out with the bathwater. I will attempt to do option (2) when I get a chance. As I have said on IRC, my doctoral defense is in a week and that take priority.

[sjn]

Ok.

So the intention of this module is closer to managing/uninstalling stuff installed with File::ShareDir?

[thaljef]

Another (probably better) way to install Pinto is:

curl -L http://getpinto.stratopan.com | bash

That installs Pinto as a standalone application in ~/opt/local/pinto It also does not run the tests.

[sjn]

@jberger - Any news on this? :)

@thaljef - I'm a big fan of running tests, since it gives me information on whether or not the author cares if his code continues to work in different environments. Being able to detect regressions (or faulty assumptions, for that matter) kinda depends on having a base-line of successful tests to compare against. If I wanted to silence any feedback, I'd use cpanm's --notest flag. :)

[jberger]

I'm inclined to give @thaljef comaint on this. I created MBCI as a toy project to solve a simple problem. Now the problem doesn't seem so simple and is also being used in projects that depend on it in a deeper way that I can possibly understand. Would you want it @thaljef?