Standardize +requirements+ field from the specification

[strzibny]

Currently, they are no standards regarding the definition of external dependencies for gems. We have requirements field in the specification that is quite benevolent and as such does not allow to write useful third-party tooling to handle the problem of installing proper header files/libraries on targeted systems.

They are projects like https://github.com/voxik/gem-nice-install that tries to work around it by creating their own specification file, but that's a path that does not scale well.

What I would like to see is a standardize requirements field that would provide a structured information about external system dependencies. Note that on RubyGems part this can be implemented as a guideline, proper installation of those dependencies can be left to external tooling such as gem-nice-install.

My idea is to leverage the requirements field and make it a JSON schema of external dependencies such as:

{ 
  "ruby": {
    "libxml2": {
      "fedora": {
        "22": [
          "libxml2-devel"
        ],
        "default": [
          "libxml2-devel"
        ]
      },
      "ubuntu": {
        "14.04": [
          "libxml2-dev"
        ]
      }
    }
  },
  "java": {

  }
}

The highest level would represent platforms as known from RubyGems (ruby, java, x86-mingw32, ...), then there will be a list of dependencies (official names) with definition of operating systems and their versions which would at last specify the package/port name for a given OS version.

OS identifiers would have to be standardize ideally so they remain the same across all released gems:

fedora, rhel,
opensuse, sles,
ubuntu, debian,
arch,
coreos,
darwin,
gentoo,
nixos,
solaris,
freebsd, openbsd, netbsd,
darwin,
windows

RubyGems part would be to ensure that the requirements field is a valid JSON in the given format with valid identifiers issuing a warning on gem build if not.

Please note that even if one dependency or two will be specified by the author, it might still be enough information on downloading a proper dependency on other systems (Fedora -devel vs Debian -dev sub-packages).

This standardization could enable many Ruby developers to create proper tooling and make installing gems with extensions finally as easy as installing those without. It will also enable us to improve gem-to-rpm conversions (and conversions to other formats) automaticaly with proper dependencies stated. It's something I really miss in current RubyGems.

@drbrain @tenderlove

[jrafanie]

I ❤ this idea if rubygems created the specification and validated it only as described. Gem authors could choose to use it if desired and external tooling could be written to install the dependencies.

[indirect]

This sounds good to me, at least in the abstract. :) I would be interested in helping and/or coordinating work on this.

On Fri, Jun 26, 2015 at 4:54 PM, Joe Rafaniello notifications@github.com
wrote:

I ❤ this idea if rubygems created the specification and validated it only as described. Gem authors could choose to use it if desired and external tooling could be written to install the dependencies.

Reply to this email directly or view it on GitHub:
#1296 (comment)

[mkristian]

FYI how the requirements field is used by jruby:

https://github.com/lookout/leafy/blob/master/leafy-rack/leafy-rack.gemspec#L19

to declare jar dependencies. which produces through ruby-maven such dependency graphs
https://gist.github.com/mkristian/c47dd73a14c9225728b6

the java section could take those jar dependencies - maybe if as formal json as well.

[voxik]

OS identifiers would have to be standardize ideally so they remain the same across all released gems:

We should probably use ID form os-release [1], where supported.

http://www.freedesktop.org/software/systemd/man/os-release.html#ID=

[strzibny]

@headius Could you please let us know if we could do something on JRuby site to avoid a conflict of this. I would like to move this forward.

[voxik]

Just FYI, Python guys are working on something similar, so it might be interesting to keep compatibility on some level?

pypa/interoperability-peps#30

[hsbt]

I and @kou discussed this feature.

rubygems already have msys2_mingw_dependencies feature for installing external dependencies like libssl.

@kou propose expand these feature to yum/dnf and apt and etc for Linux distribution.

[headius]

I never saw this the first time through, but @mkristian covered our equivalent feature for JRuby gems with Maven-source jar dependencies. This unsurprisingly uses the jar-dependencies library we ship with JRuby to supplement RubyGems logic.

JRuby usually considers our "OS" to be Java, and so os-release would look like java-19...

HOWEVER... we have long wished we could track both the "java" platform and the native host platform, so that JRuby gems could ship an OS-specific prebuilt binary but still install on JRuby (currently because we chose "java" as our platform there's no way to also differentiate on native OS).

We would hope that any requirements specification would provide a generic way for different dependencies to be managed and installed. We could potentially eliminate our jar-dependencies library or roll it into RubyGems rather than doing it a custom post-install hooks. It would also allow us to fail installation of a package if the dependencies could not be retrieved, where currently I think it leaves the library partially installed.

[kou]

Thanks for sharing a JRuby side opinion.

I'll care about JRuby too for this feature.