RuboCop dependency issues via measure tester gem

[DavidGoldwasser]

Issue overview

Impacting URBANopt and may impact other downstream internal and external tools as well.

Current Behavior

Email thread for background

From Nicholas:
I think it works because there were a few gems that we build for packaging, and I am pretty sure that JSON was one of those. I don’t know why there is a just version of rubocop though.

If we move measure tester’s JSON dependency to be a development dependency, then it will just mean that windows users will have to install DevKit, which we didn’t want to do. That way a measure developer could still run tests without having to deal with the nightmare of DevKit (but maybe that has improved).

From Katherine

When we run a UO project, we have a Gemfile in the project directly and the OpenStudio CLI installs that Gemfile. This is done via OpenStudio’s ruby when users use the UO installer which packages OS and Ruby…

The Gemfile lists a bunch of the OpenStudio measure gems, which all depend on OpenStudio extension gem.

OpenStudio extension gem has a dependency on OpenStudio Measure tester gem.

Measure-tester-gem in turn depends on robocop 1.50. Rubocop has a json dependency that has a native gem extension and must be compiled.

This famously does not work with OpenStudio unless we package OpenStudio with the gem ahead of time.

I am curious to understand how OpenStudio is packaging openstudio-measure-tester without issues. And why Julien made its’ own fork of robocop:

https://github.com/NREL/openstudio-gems/commit/0e84b9a21307c7e513a71ca95487a1e7c1d27d27

This seemed to fix the issue with measure tester gem’s json dependency(ies). But it’s not working outside the box for the UO workflow.

I found the following workaround for the UO workflow:

https://github.com/NREL/OpenStudio-measure-tester-gem/tree/remove-json-gem

This makes the 3 rubocop dependencies development dependencies, so they will not be installed at runtime when we run the UO workflow. I am not 100% on the implications of this change for OpenStudio though, and for the OpenStudio CLI update_measures call.

Long, Nicholas, what are your thoughts here? How did this ever work without shipping the native extension json with OpenStudio??

Expected Behavior

Steps to Reproduce

Require these gems from OpenStudio's Ruby
https://github.com/NREL/OpenStudio-measure-tester-gem/blob/develop/openstudio_measure_tester.gemspec#L37-L39

Possible Solution

Details

Environment

Some additional details about your environment for this issue (if relevant):

• Platform (Operating system, version): I believe it impacts all platforms
• Version of OpenStudio (if using an intermediate build, include SHA): At least OS 3.8 and 3.9

Context

[DavidGoldwasser]

Maybe move style and rubo to separate style testing gem.

[DavidGoldwasser]

Are there other ways to support SketchUp Plugin

[jmarrec]

What I answered:

I had to fork rubocop to remove json has a runtime dependency, or it ends up in the bundle, which we embed in the CLI.
Except we already have json native from ruby source itself in the ruby 3.2.2 we embed into the CLI, and that is a compatible version (cf rubocop/rubocop@v1.50.0...jmarrec:rubocop:1.50.0-no_json). And since having two copies of json (ruby's + the bundled one) was creating issue, I just removed it.

The native json ext is here, in source: https://github.com/ruby/ruby/tree/v3_2_2/ext/json

openstudio measure --run_tests <path> definitely uses rubocop, and that shows on the dashboard. I'd be perfectly happy personally if we remove that.

I sound like a broken record, but ruby's build system (low-level, not the language itself) and the packaging on top of it strange (why would it download standard library gems is a question that comes to mind...). Combine that with the fact that we do all kinds of shenanigans like compiling ruby and the native gem exts we want with MSVC so we have binary compatibility for the embedded stuff, and you end up in this situation. We've talked about trying to stop embedding the gems (and just carry folders around) before, like we do with python, but we've not started trying.

I'm not 100% sure I'm following the issue you're having with UO exactly. I'm not familiar with the project and I see you have multiple repos so I'm not sure where I'd go look.

Is strapping the same gem 'rubocop', :github => 'jmarrec/rubocop', :ref => '1.50.0-no_json' in its Gemfile something you've tried already?

Another thing to consider is while openstudio-extension-gem used to actually carry Ruby files that would do stuff (core/os_lib_xxx.rb stuff for eg), this is no longer the case: they've all been moved to C++ or to openstudio-standards.

• Removing model methods and model method spec tests openstudio-extension-gem#184

The openstudio-extension gem is pretty much nothing but Rake Tasks and helpers to run these Rake tasks now, so that one could be moved to be a development_dependency altogether, in all of the gem repos that use it.

I hope this helps a bit,

[kbenne]

Here is what I have tested so far and what I've noticed. All of the bundle commands are using the version that is provided by conan during the OpenStudio configure / dev setup. This is a convenient way for me to make sure that the version of ruby/bundler aligns with what is provided by OpenStudio. This test is on Ubuntu.

1. Create a new ruby based test project

mkdir testrb
cd testrb
bundle init

2. Add openstudio-common-measures to the newly created Gemfile

echo "gem 'openstudio-common-measures'" >> Gemfile   

3. Install the gems into a new directory

bundle install --path gems

4. Create a test ruby file, main.rb, containing this content

require 'rubocop'

puts "Rubocop version: #{Gem.loaded_specs['rubocop'].version}"
puts "Rubocop path: #{Gem.loaded_specs['rubocop'].full_gem_path}"
puts "JSON version: #{Gem.loaded_specs['json'].version}"
puts "JSON path: #{Gem.loaded_specs['json'].full_gem_path}"

5. Run main.rb using the openstudio cli. (first without any bundle options)

openstudio execute_ruby_script main.rb

The output is...

Rubocop version: 1.50.0
Rubocop path: :/ruby/3.2.0/gems/rubocop-1.50.0
JSON version: 2.6.3
JSON path: :/ruby/gems/3.2.0/specifications/gems/json-2.6.3

which means Rubocop and json were correctly found from the openstudio install
6. Run main.rb using bundle options

openstudio  --bundle ./Gemfile --bundle_path ./gems execute_ruby_script main.rb

The output is...

Warn: Bundle activated but ENV['BUNDLE_WITHOUT'] is not set
Info: Setting BUNDLE_WITHOUT to 'test'
[ruby] <1> Error activating gem :/ruby/3.2.0/specifications/parallel-1.19.1.gemspec
[ruby] <1> Error activating gem :/ruby/3.2.0/specifications/rchardet-1.8.0.gemspec
[ruby] <1> Error activating gem :/ruby/3.2.0/specifications/multipart-post-2.4.1.gemspec
[ruby] <1> Error activating gem :/ruby/3.2.0/specifications/openstudio-extension-0.8.2.gemspec
[ruby] <1> Error activating gem :/ruby/3.2.0/specifications/osut-0.5.0.gemspec
[ruby] <1> Error activating gem :/ruby/3.2.0/specifications/tbd-3.4.3.gemspec
[ruby] <1> Error activating gem :/ruby/3.2.0/specifications/openstudio-standards-0.7.0.gemspec
Rubocop version: 1.50.0
Rubocop path: /home/kbenne/Development/OpenStudio2/build/testrb/gems/ruby/3.2.0/gems/rubocop-1.50.0
JSON version: 2.6.3
JSON path: :/ruby/gems/3.2.0/specifications/gems/json-2.6.3

I think the activation errors can be ignored because these gems are provided by the bundle, however they are tacky. Nevertheless, it looks like the system is working how I would have guessed. That is the bundled rubocop is loaded from my computer's filesystem. json was preloaded by openstudio from the embedded system. I think this is all good. maybe?

Are there additional tests that I can do to uncover the issue? Barring any additional tests I can add to this, I'm going to move on to other platforms. Mac first. Then, shudder, Windows if I need to.

One more thing... Notably, the version of json that came in during the bundle process was version 2.10.

➜  testrb git:(develop) find ./gems -name json   
./gems/ruby/3.2.0/gems/json-2.10.2/lib/json

[kbenne]

Alright I did the same steps on my Mac, but in this case I used the same tools (bundle, openstudio) that come with this URBANopt installer and I got the same results. Meaning I couldn't replicate an issue.

Then, @kflemin shared the mkmf.log from her failed attempt, and we realized that the issue stems from a failed bundle update command that is within the URBANOpt code. In her case, the required json library failed to build because of a missing dependency (libgmp).

Stepping back, it seems awkward to expect end users to have a suitable development environment setup to build native gems like json on the fly, so we think the correct solution is to make these native gem requirements a dev dependency after all. Note, that rubocop, json, etc are still available when using openstudio CLI since they are packaged in with OS. That is to say, end users still have access to these libraries if they need them, we just probably shouldn't expect them to build their copy on the fly on their end user computer.

I think @kflemin can link to a specific change that probably resolves this issue and once confirmed, I think we can close this issue.