Skip to content

🚨 [security] Update nokogiri: 1.10.10 → 1.11.5 (minor)#351

Closed
depfu[bot] wants to merge 1 commit intodevelopfrom
depfu/update/nokogiri-1.11.5
Closed

🚨 [security] Update nokogiri: 1.10.10 → 1.11.5 (minor)#351
depfu[bot] wants to merge 1 commit intodevelopfrom
depfu/update/nokogiri-1.11.5

Conversation

@depfu
Copy link
Contributor

@depfu depfu bot commented May 20, 2021

🚨 Your current dependencies have known security vulnerabilities 🚨

This dependency update fixes known security vulnerabilities. Please see the details below and assess their impact carefully. We recommend to merge and deploy this as soon as possible!

Here is everything you need to know about this update. Please take a good look at what changed and the test results before merging this pull request.

What changed?

↗️ nokogiri (indirect, 1.10.10 → 1.11.5) · Repo · Changelog

Security Advisories 🚨

🚨 Update packaged dependency libxml2 from 2.9.10 to 2.9.12

Summary

Nokogiri v1.11.4 updates the vendored libxml2 from v2.9.10 to v2.9.12 which addresses:

Note that two additional CVEs were addressed upstream but are not relevant to this release. CVE-2021-3516 via xmllint is not present in Nokogiri, and CVE-2020-7595 has been patched in Nokogiri since v1.10.8 (see #1992).

Please note that this advisory only applies to the CRuby implementation of Nokogiri < 1.11.4, and only if the packaged version of libxml2 is being used. If you've overridden defaults at installation time to use system libraries instead of packaged libraries, you should instead pay attention to your distro's libxml2 release announcements.

Mitigation

Upgrade to Nokogiri >= 1.11.4.

Impact

I've done a brief analysis of the published CVEs that are addressed in this upstream release. The libxml2 maintainers have not released a canonical set of CVEs, and so this list is pieced together from secondary sources and may be incomplete.

All information below is sourced from security.archlinux.org, which appears to have the most up-to-date information as of this analysis.

CVE-2019-20388

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2020-7595

This has been patched in Nokogiri since v1.10.8 (see #1992).

CVE-2020-24977

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2021-3516

Verified that the fix commit first appears in v2.9.11. This vector does not exist within Nokogiri, which does not ship xmllint.

CVE-2021-3517

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2021-3518

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2021-3537

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4.

CVE-2021-3541

Verified that the fix commit first appears in v2.9.11. It seems possible that this issue would be present in programs using Nokogiri < v1.11.4, however Nokogiri's default parse options prevent the attack from succeeding (it is necessary to opt into DTDLOAD which is off by default).

For more details supporting this analysis of this CVE, please visit #2233.

🚨 Nokogiri::XML::Schema trusts input by default, exposing risk of an XXE vulnerability

Description

In Nokogiri versions <= 1.11.0.rc3, XML Schemas parsed by Nokogiri::XML::Schema
are trusted by default, allowing external resources to be accessed over the
network, potentially enabling XXE or SSRF attacks.

This behavior is counter to
the security policy followed by Nokogiri maintainers, which is to treat all input
as untrusted by default whenever possible.

Please note that this security
fix was pushed into a new minor version, 1.11.x, rather than a patch release to
the 1.10.x branch, because it is a breaking change for some schemas and the risk
was assessed to be "Low Severity".

Affected Versions

Nokogiri <= 1.10.10 as well as prereleases 1.11.0.rc1, 1.11.0.rc2, and 1.11.0.rc3

Mitigation

There are no known workarounds for affected versions. Upgrade to Nokogiri
1.11.0.rc4 or later.

If, after upgrading to 1.11.0.rc4 or later, you wish
to re-enable network access for resolution of external resources (i.e., return to
the previous behavior):

  1. Ensure the input is trusted. Do not enable this option
    for untrusted input.
  2. When invoking the Nokogiri::XML::Schema constructor,
    pass as the second parameter an instance of Nokogiri::XML::ParseOptions with the
    NONET flag turned off.

So if your previous code was:

# in v1.11.0.rc3 and earlier, this call allows resources to be accessed over the network
# but in v1.11.0.rc4 and later, this call will disallow network access for external resources
schema = Nokogiri::XML::Schema.new(schema)

# in v1.11.0.rc4 and later, the following is equivalent to the code above

# (the second parameter is optional, and this demonstrates its default value)

schema = Nokogiri::XML::Schema.new(schema, Nokogiri::XML::ParseOptions::DEFAULT_SCHEMA)

Then you can add the second parameter to indicate that the input is trusted by changing it to:

# in v1.11.0.rc3 and earlier, this would raise an ArgumentError
# but in v1.11.0.rc4 and later, this allows resources to be accessed over the network
schema = Nokogiri::XML::Schema.new(trusted_schema, Nokogiri::XML::ParseOptions.new.nononet)
Release Notes

1.11.4

1.11.4 / 2021-05-14

Security

[CRuby] Vendored libxml2 upgraded to v2.9.12 which addresses:

Note that two additional CVEs were addressed upstream but are not relevant to this release. CVE-2021-3516 via xmllint is not present in Nokogiri, and CVE-2020-7595 has been patched in Nokogiri since v1.10.8 (see #1992).

Please see nokogiri/GHSA-7rrm-v45f-jp64 or #2233 for a more complete analysis of these CVEs and patches.

Dependencies

  • [CRuby] vendored libxml2 is updated from 2.9.10 to 2.9.12. (Note that 2.9.11 was skipped because it was superseded by 2.9.12 a few hours after its release.)

1.11.3

1.11.3 / 2021-04-07

Fixed

  • [CRuby] Passing non-Node objects to Document#root= now raises an ArgumentError exception. Previously this likely segfaulted. [#1900]
  • [JRuby] Passing non-Node objects to Document#root= now raises an ArgumentError exception. Previously this raised a TypeError exception.
  • [CRuby] arm64/aarch64 systems (like Apple's M1) can now compile libxml2 and libxslt from source (though we continue to strongly advise users to install the native gems for the best possible experience)

1.11.2

1.11.2 / 2021-03-11

Fixed

  • [CRuby] NodeSet may now safely contain Node objects from multiple documents. Previously the GC lifecycle of the parent Document objects could lead to nodes being GCed while still in scope. [#1952]
  • [CRuby] Patch libxml2 to avoid "huge input lookup" errors on large CDATA elements. (See upstream GNOME/libxml2#200 and GNOME/libxml2!100.) [#2132].
  • [CRuby+Windows] Enable Nokogumbo (and other downstream gems) to compile and link against nokogiri.so by including LDFLAGS in Nokogiri::VERSION_INFO. [#2167]
  • [CRuby] {XML,HTML}::Document.parse now invokes #initialize exactly once. Previously #initialize was invoked twice on each object.
  • [JRuby] {XML,HTML}::Document.parse now invokes #initialize exactly once. Previously #initialize was not called, which was a problem for subclassing such as done by Loofah.

Improved

  • Reduce the number of object allocations needed when parsing an HTML::DocumentFragment. [#2087] (Thanks, @ashmaroli!)
  • [JRuby] Update the algorithm used to calculate Node#line to be wrong less-often. The underlying parser, Xerces, does not track line numbers, and so we've always used a hacky solution for this method. [#1223, #2177]
  • Introduce --enable-system-libraries and --disable-system-libraries flags to extconf.rb. These flags provide the same functionality as --use-system-libraries and the NOKOGIRI_USE_SYSTEM_LIBRARIES environment variable, but are more idiomatic. [#2193] (Thanks, @eregon!)
  • [TruffleRuby] --disable-static is now the default on TruffleRuby when the packaged libraries are used. This is more flexible and compiles faster. (Note, though, that the default on TR is still to use system libraries.) [#2191, #2193] (Thanks, @eregon!)

Changed

  • Nokogiri::XML::Path is now a Module (previously it has been a Class). It has been acting solely as a Module since v1.0.0. See 8461c74.

1.11.1

v1.11.1 / 2021-01-06

Fixed

  • [CRuby] If libxml-ruby is loaded before nokogiri, the SAX and Push parsers no longer call libxml-ruby's handlers. Instead, they defensively override the libxml2 global handler before parsing. [#2168]

SHA-256 Checksums of published gems

a41091292992cb99be1b53927e1de4abe5912742ded956b0ba3383ce4f29711c  nokogiri-1.11.1-arm64-darwin.gem
d44fccb8475394eb71f29dfa7bb3ac32ee50795972c4557ffe54122ce486479d  nokogiri-1.11.1-java.gem
f760285e3db732ee0d6e06370f89407f656d5181a55329271760e82658b4c3fc  nokogiri-1.11.1-x64-mingw32.gem
dd48343bc4628936d371ba7256c4f74513b6fa642e553ad7401ce0d9b8d26e1f  nokogiri-1.11.1-x86-linux.gem
7f49138821d714fe2c5d040dda4af24199ae207960bf6aad4a61483f896bb046  nokogiri-1.11.1-x86-mingw32.gem
5c26111f7f26831508cc5234e273afd93f43fbbfd0dcae5394490038b88d28e7  nokogiri-1.11.1-x86_64-darwin.gem
c3617c0680af1dd9fda5c0fd7d72a0da68b422c0c0b4cebcd7c45ff5082ea6d2  nokogiri-1.11.1-x86_64-linux.gem
42c2a54dd3ef03ef2543177bee3b5308313214e99f0d1aa85f984324329e5caa  nokogiri-1.11.1.gem

1.11.0

v1.11.0 / 2021-01-03

Notes

Faster, more reliable installation: Native Gems for Linux and OSX/Darwin

"Native gems" contain pre-compiled libraries for a specific machine architecture. On supported platforms, this removes the need for compiling the C extension and the packaged libraries. This results in much faster installation and more reliable installation, which as you probably know are the biggest headaches for Nokogiri users.

We've been shipping native Windows gems since 2009, but starting in v1.11.0 we are also shipping native gems for these platforms:

  • Linux: x86-linux and x86_64-linux -- including musl platforms like alpine
  • OSX/Darwin: x86_64-darwin and arm64-darwin

We'd appreciate your thoughts and feedback on this work at #2075.

Dependencies

Ruby

This release introduces support for Ruby 2.7 and 3.0 in the precompiled native gems.

This release ends support for:

Gems

  • Explicitly add racc as a runtime dependency. [#1988] (Thanks, @voxik!)
  • [MRI] Upgrade mini_portile2 dependency from ~> 2.4.0 to ~> 2.5.0 [#2005] (Thanks, @alejandroperea!)

Security

See note below about CVE-2020-26247 in the "Changed" subsection entitled "XML::Schema parsing treats input as untrusted by default".

Added

  • Add Node methods for manipulating "keyword attributes" (for example, class and rel): #kwattr_values, #kwattr_add, #kwattr_append, and #kwattr_remove. [#2000]
  • Add support for CSS queries a:has(> b), a:has(~ b), and a:has(+ b). [#688] (Thanks, @jonathanhefner!)
  • Add Node#value? to better match expected semantics of a Hash-like object. [#1838, #1840] (Thanks, @MatzFan!)
  • [CRuby] Add Nokogiri::XML::Node#line= for use by downstream libs like nokogumbo. [#1918] (Thanks, @stevecheckoway!)
  • nokogiri.gemspec is back after a 10-year hiatus. We still prefer you use the official releases, but master is pretty stable these days, and YOLO.

Performance

  • [CRuby] The CSS ~= operator and class selector . are about 2x faster. [#2137, #2135]
  • [CRuby] Patch libxml2 to call strlen from xmlStrlen rather than the naive implementation, because strlen is generally optimized for the architecture. [#2144] (Thanks, @ilyazub!)
  • Improve performance of some namespace operations. [#1916] (Thanks, @ashmaroli!)
  • Remove unnecessary array allocations from Node serialization methods [#1911] (Thanks, @ashmaroli!)
  • Avoid creation of unnecessary zero-length String objects. [#1970] (Thanks, @ashmaroli!)
  • Always compile libxml2 and libxslt with '-O2' [#2022, #2100] (Thanks, @ilyazub!)
  • [JRuby] Lots of code cleanup and performance improvements. [#1934] (Thanks, @kares!)
  • [CRuby] RelaxNG.from_document no longer leaks memory. [#2114]

Improved

  • [CRuby] Handle incorrectly-closed HTML comments as WHATWG recommends for browsers. [#2058] (Thanks to HackerOne user mayflower for reporting this!)
  • {HTML,XML}::Document#parse now accept Pathname objects. Previously this worked only if the referenced file was less than 4096 bytes long; longer files resulted in undefined behavior because the read method would be repeatedly invoked. [#1821, #2110] (Thanks, @doriantaylor and @phokz!)
  • [CRuby] Nokogumbo builds faster because it can now use header files provided by Nokogiri. [#1788] (Thanks, @stevecheckoway!)
  • Add frozen_string_literal: true magic comment to all lib files. [#1745] (Thanks, @oniofchaos!)
  • [JRuby] Clean up deprecated calls into JRuby. [#2027] (Thanks, @headius!)

Fixed

  • HTML Parsing in "strict" mode (i.e., the RECOVER parse option not set) now correctly raises a XML::SyntaxError exception. Previously the value of the RECOVER bit was being ignored by CRuby and was misinterpreted by JRuby. [#2130]
  • The CSS ~= operator now correctly handles non-space whitespace in the class attribute. commit e45dedd
  • The switch to turn off the CSS-to-XPath cache is now thread-local, rather than being shared mutable state. [#1935]
  • The Node methods add_previous_sibling, previous=, before, add_next_sibling, next=, after, replace, and swap now correctly use their parent as the context node for parsing markup. These methods now also raise a RuntimeError if they are called on a node with no parent. [nokogumbo#160]
  • [JRuby] XML::Schema XSD validation errors are captured in XML::Schema#errors. These errors were previously ignored.
  • [JRuby] Standardize reading from IO like objects, including StringIO. [#1888, #1897]
  • [JRuby] Fix how custom XPath function namespaces are inferred to be less naive. [#1890, #2148]
  • [JRuby] Clarify exception message when custom XPath functions can't be resolved.
  • [JRuby] Comparison of Node to Document with Node#<=> now matches CRuby/libxml2 behavior.
  • [CRuby] Syntax errors are now correctly captured in Document#errors for short HTML documents. Previously the SAX parser used for encoding detection was clobbering libxml2's global error handler.
  • [CRuby] Fixed installation on AIX with respect to vasprintf. [#1908]
  • [CRuby] On some platforms, avoid symbol name collision with glibc's canonicalize. [#2105]
  • [Windows Visual C++] Fixed compiler warnings and errors. [#2061, #2068]
  • [CRuby] Fixed Nokogumbo integration which broke in the v1.11.0 release candidates. [#1788] (Thanks, @stevecheckoway!)
  • [JRuby] Fixed document encoding regression in v1.11.0 release candidates. [#2080, #2083] (Thanks, @thbar!)

Removed

  • The internal method Nokogiri::CSS::Parser.cache_on= has been removed. Use .set_cache if you need to muck with the cache internals.
  • The class method Nokogiri::CSS::Parser.parse has been removed. This was originally deprecated in 2009 in 13db61b. Use Nokogiri::CSS.parse instead.

Changed

XML::Schema input is now "untrusted" by default

Address CVE-2020-26247.

In Nokogiri versions <= 1.11.0.rc3, XML Schemas parsed by Nokogiri::XML::Schema were trusted by default, allowing external resources to be accessed over the network, potentially enabling XXE or SSRF attacks.

This behavior is counter to the security policy intended by Nokogiri maintainers, which is to treat all input as untrusted by default whenever possible.

Please note that this security fix was pushed into a new minor version, 1.11.x, rather than a patch release to the 1.10.x branch, because it is a breaking change for some schemas and the risk was assessed to be "Low Severity".

More information and instructions for enabling "trusted input" behavior in v1.11.0.rc4 and later is available at the public advisory.

HTML parser now obeys the strict or norecover parsing option

(Also noted above in the "Fixed" section) HTML Parsing in "strict" mode (i.e., the RECOVER parse option not set) now correctly raises a XML::SyntaxError exception. Previously the value of the RECOVER bit was being ignored by CRuby and was misinterpreted by JRuby.

If you're using the default parser options, you will be unaffected by this fix. If you're passing strict or norecover to your HTML parser call, you may be surprised to see that the parser now fails to recover and raises a XML::SyntaxError exception. Given the number of HTML documents on the internet that libxml2 would consider to be ill-formed, this is probably not what you want, and you can omit setting that parse option to restore the behavior that you have been relying upon.

Apologies to anyone inconvenienced by this breaking bugfix being present in a minor release, but I felt it was appropriate to introduce this fix because it's straightforward to fix any code that has been relying on this buggy behavior.

VersionInfo, the output of nokogiri -v, and related constants

This release changes the metadata provided in Nokogiri::VersionInfo which also affects the output of nokogiri -v. Some related constants have also been changed. If you're using VersionInfo programmatically, or relying on constants related to underlying library versions, please read the detailed changes for Nokogiri::VersionInfo at #2139 and accept our apologies for the inconvenience.

SHA-256 Checksums of published gems

17ed2567bf76319075b4a6a7258d1a4c9e2661fca933b03e037d79ae2b9910d0  nokogiri-1.11.0.gem
2f0149c735b0672c49171b18467ce25fd323a8e608c9e6b76e2b2fa28e7f66ee  nokogiri-1.11.0-java.gem
2f249be8cc705f9e899c07225fcbe18f4f7dea220a59eb5fa82461979991082e  nokogiri-1.11.0-x64-mingw32.gem
9e219401dc3f93abf09166d12ed99c8310fcaf8c56a99d64ff93d8b5f0604e91  nokogiri-1.11.0-x86-mingw32.gem
bda2a9c9debf51da7011830c7f2dc5771c122ebcf0fc2dd2c4ba4fc95b5c38f2  nokogiri-1.11.0-x86-linux.gem
d500c3202e2514b32f4b02049d9193aa825ae3e9442c9cad2d235446c3e17d8d  nokogiri-1.11.0-x86_64-linux.gem
3a613188e3b76d593b04e0ddcc46f44c288b13f80b32ce83957356f50e22f9ee  nokogiri-1.11.0-arm64-darwin.gem
b8f9b826d09494b20b30ecd048f5eb2827dccd85b77abeb8baf1f610e5ed28ed  nokogiri-1.11.0-x86_64-darwin.gem

Does any of this look wrong? Please let us know.

Commits

See the full diff on Github. The new version differs by more commits than we can show here.

↗️ mini_portile2 (indirect, 2.4.0 → 2.5.1) · Repo · Changelog

Release Notes

2.5.1

2.5.1 / 2021-04-28

Dependencies

This release ends support for ruby < 2.3.0. If you're on 2.2.x or earlier, we strongly suggest that you find the time to upgrade, because official support for Ruby 2.2 ended on 2018-03-31.

Enhancements

  • MiniPortile.execute now takes an optional :env hash, which is merged into the environment variables for the subprocess. Likely this is only useful for specialized use cases. [#99]
  • Experimental support for cmake-based projects extended to Windows. (Thanks, @larskanis!)

2.5.0

2.5.0 / 2020-02-24

Enhancements

  • When verifying GPG signatures, remove all imported pubkeys from keyring [#90] (Thanks, @hanazuki!)

Does any of this look wrong? Please let us know.

Commits

See the full diff on Github. The new version differs by 43 commits:

🆕 racc (added, 1.5.2)

Depfu Status

Depfu will automatically keep this PR conflict-free, as long as you don't add any commits to this branch yourself. You can also trigger a rebase manually by commenting with @depfu rebase.

All Depfu comment commands
@​depfu rebase
Rebases against your default branch and redoes this update
@​depfu recreate
Recreates this PR, overwriting any edits that you've made to it
@​depfu merge
Merges this PR once your tests are passing and conflicts are resolved
@​depfu close
Closes this PR and deletes the branch
@​depfu reopen
Restores the branch and reopens this PR (if it's closed)
@​depfu pause
Ignores all future updates for this dependency and closes this PR
@​depfu pause [minor|major]
Ignores all future minor/major updates for this dependency and closes this PR
@​depfu resume
Future versions of this dependency will create PRs again (leaves this PR as is)

@depfu depfu bot added the depfu label May 20, 2021
@depfu depfu bot mentioned this pull request May 20, 2021
Closed
@depfu
Copy link
Contributor Author

depfu bot commented Sep 27, 2021

Closed in favor of #360.

@depfu depfu bot closed this Sep 27, 2021
@depfu depfu bot deleted the depfu/update/nokogiri-1.11.5 branch September 27, 2021 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

depfu

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

0 participants