remove dependency on org.reflections by jsvd · Pull Request #18295 · elastic/logstash

jsvd · 2025-10-08T11:48:50Z

This removes the annoying log info:

[2025-10-14T16:46:35,085][INFO ][org.reflections.Reflections] Reflections took 44 ms to scan 1 urls, producing 149 keys and 523 values

Given we only use reflectiong to grab locally installed java plugin classes by annotation, we implement here a tiny scanner to do the same, and then we can remove the dependency.

exhaustive test suite run: https://buildkite.com/elastic/logstash-exhaustive-tests-pipeline/builds/2701/steps/canvas

github-actions · 2025-10-08T11:48:59Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

mergify · 2025-10-08T11:49:27Z

This pull request does not have a backport label. Could you fix it @jsvd? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
If no backport is necessary, please add the backport-skip label

Copilot

Pull Request Overview

This PR removes the dependency on org.reflections library and replaces it with a custom lightweight package scanner. The change eliminates verbose logging from the reflections library while maintaining the same plugin discovery functionality.

Key Changes:

Implemented a custom PackageScanner class to scan for annotated plugin classes
Refactored PluginRegistry to use the new scanner instead of the reflections library
Removed the org.reflections:reflections dependency from build configuration and licensing files

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java`	New custom scanner implementation for finding annotated classes in packages
`logstash-core/src/main/java/org/logstash/plugins/discovery/PluginRegistry.java`	Updated to use `PackageScanner` instead of reflections library, simplified plugin discovery logic
`logstash-core/src/test/java/org/logstash/plugins/discovery/PluginRegistryTest.java`	New test suite verifying plugin discovery works for built-in plugins
`logstash-core/build.gradle`	Removed `org.reflections:reflections` dependency
`tools/dependencies-report/src/main/resources/licenseMapping.csv`	Removed license mapping entry for reflections
`tools/dependencies-report/src/main/resources/notices/org.reflections!reflections-NOTICE.txt`	Removed license notice file
`NOTICE.TXT`	Removed reflections library attribution

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java

logstash-core/src/main/java/org/logstash/plugins/discovery/PluginRegistry.java

andsel

Left a couple of notes

logstash-core/src/main/java/org/logstash/plugins/discovery/PluginRegistry.java

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

logstash-core/src/main/java/org/logstash/plugins/discovery/PluginRegistry.java

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java

Copilot · 2025-12-19T11:43:51Z

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java

+                if ("file".equals(protocol)) {
+                    scanDirectory(resource, basePackage, classNames);
+                } else if ("jar".equals(protocol)) {
+                    scanJar(resource, resourcePath, classNames);
+                } else if (resource.openConnection() instanceof JarURLConnection) {
+                    scanJar(resource, resourcePath, classNames);
+                }


The protocol check at lines 64-67 calls resource.openConnection() potentially multiple times (once at line 66 to check the protocol, then again at line 96 inside scanJar). This creates unnecessary connection overhead and could lead to resource leaks. Consider checking the connection type once and reusing it.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-19T15:46:09Z

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java

+                    URLConnection connection = resource.openConnection();
+                    if (connection instanceof JarURLConnection) {
+                        scanJar((JarURLConnection) connection, resourcePath, classNames);
+                    }


The URLConnection opened on line 67 is not explicitly closed. While JarURLConnection is handled with try-with-resources in scanJar, if the connection is not a JarURLConnection but is some other type of URLConnection that requires cleanup, it may lead to a resource leak. Consider closing the connection in a finally block or using try-with-resources if the connection type is uncertain.

scanJar cover with try-with-resources just the JarFile obtained from connection.getJarFile() not the URLConnection itself.

@jsvd maybe we could use a try-with-resources when we get the connection at resource.openConnection();

Suggested change

URLConnection connection = resource.openConnection();

if (connection instanceof JarURLConnection) {

scanJar((JarURLConnection) connection, resourcePath, classNames);

}

try (URLConnection connection = resource.openConnection()) {

if (connection instanceof JarURLConnection) {

scanJar((JarURLConnection) connection, resourcePath, classNames);

}

}

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java

andsel

Left a couple of suggestion to cover the issue that Copilot partially surfaced, and an readability improvement of the class names scanning.

andsel · 2025-12-22T08:17:38Z

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java

+                    URLConnection connection = resource.openConnection();
+                    if (connection instanceof JarURLConnection) {
+                        scanJar((JarURLConnection) connection, resourcePath, classNames);
+                    }


scanJar cover with try-with-resources just the JarFile obtained from connection.getJarFile() not the URLConnection itself.

@jsvd maybe we could use a try-with-resources when we get the connection at resource.openConnection();

Suggested change

URLConnection connection = resource.openConnection();

if (connection instanceof JarURLConnection) {

scanJar((JarURLConnection) connection, resourcePath, classNames);

}

try (URLConnection connection = resource.openConnection()) {

if (connection instanceof JarURLConnection) {

scanJar((JarURLConnection) connection, resourcePath, classNames);

}

}

andsel · 2025-12-22T08:33:15Z

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java

+        Set<Class<?>> result = new HashSet<>();
+        for (String className : classNames) {
+            if (className.contains("$")) {
+                continue;
+            }
+            try {
+                Class<?> candidate = Class.forName(className, false, loader);
+                if (candidate.isAnnotationPresent(annotation)) {
+                    result.add(candidate);
+                }
+            } catch (ClassNotFoundException | LinkageError e) {
+                throw new IllegalStateException("Unable to load class discovered during scanning: " + className, e);
+            }
+        }
+        return result;


Given that the size of classNames is more than some tens, I think we could use stream syntax to separate looping from logic to be applied:

Suggested change

Set<Class<?>> result = new HashSet<>();

for (String className : classNames) {

if (className.contains("$")) {

continue;

}

try {

Class<?> candidate = Class.forName(className, false, loader);

if (candidate.isAnnotationPresent(annotation)) {

result.add(candidate);

}

} catch (ClassNotFoundException | LinkageError e) {

throw new IllegalStateException("Unable to load class discovered during scanning: " + className, e);

}

}

return result;

Set<Class<?>> result =classNames.stream()

.filter(PackageScanner::isInnerClass)

.map(className -> loadClass(loader, className))

.filter(cls -> cls.isAnnotationPresent(annotation))

.collect(Collectors.toSet());

where the isInenrClass and loadClass methods are:

private static Class<?> loadClass(ClassLoader loader, String className) { try { return Class.forName(className, false, loader); } catch (ClassNotFoundException | LinkageError e) { throw new IllegalStateException("Unable to load class discovered during scanning: " + className, e); } } private static boolean isInnerClass(String name) { return name.contains("$"); }

Fix URLConnection resource leak by checking URL protocol directly instead of opening a connection. Refactor scanForAnnotation into stream pipeline with helper methods and log-and-skip unloadable classes instead of throwing. Add dedicated PackageScannerTest and remove stale @NotThreadSafe annotations.

elasticmachine · 2026-02-06T14:50:02Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 164230d

Failed CI Steps

History

💛 Build #4006 was flaky c18eefb
💚 Build #4004 succeeded d2c7b47
💛 Build #4002 was flaky 9c4aaa6
💛 Build #3677 was flaky 26942ce

andsel

LGTM

jsvd changed the title ~~attempt at removing dependency on org.reflections~~ remove dependency on org.reflections Oct 14, 2025

jsvd marked this pull request as ready for review October 14, 2025 15:56

jsvd requested a review from Copilot October 24, 2025 10:41

Copilot AI reviewed Oct 24, 2025

View reviewed changes

andsel requested changes Dec 1, 2025

View reviewed changes

logstash-core/src/main/java/org/logstash/plugins/discovery/PluginRegistry.java Outdated Show resolved Hide resolved

logstash-core/src/main/java/org/logstash/plugins/discovery/PackageScanner.java Outdated Show resolved Hide resolved

jsvd force-pushed the remove_reflections_library branch from 26942ce to 9c4aaa6 Compare December 19, 2025 11:01

jsvd requested review from andsel and Copilot December 19, 2025 11:39

Copilot started reviewing on behalf of jsvd December 19, 2025 11:39 View session

Copilot AI reviewed Dec 19, 2025

View reviewed changes

jsvd force-pushed the remove_reflections_library branch from d2c7b47 to c18eefb Compare December 19, 2025 15:07

jsvd requested a review from Copilot December 19, 2025 15:42

Copilot started reviewing on behalf of jsvd December 19, 2025 15:43 View session

Copilot AI reviewed Dec 19, 2025

View reviewed changes

andsel requested changes Dec 22, 2025

View reviewed changes

jsvd added 4 commits February 6, 2026 13:47

attempt at removing dependency on org.reflections

adf2bf4

address review comments

8f2731d

address review comments

758cb8e

jsvd force-pushed the remove_reflections_library branch from c18eefb to 164230d Compare February 6, 2026 14:16

jsvd requested a review from andsel February 6, 2026 14:16

andsel approved these changes Feb 6, 2026

View reviewed changes

jsvd merged commit 41cb3b7 into elastic:main Feb 6, 2026
12 checks passed

jsvd deleted the remove_reflections_library branch February 6, 2026 15:16

Conversation

jsvd commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 8, 2025

🤖 GitHub comments

Uh oh!

mergify bot commented Oct 8, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andsel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

andsel Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andsel left a comment

Choose a reason for hiding this comment

Uh oh!

andsel Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

andsel Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Feb 6, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

History

Uh oh!

andsel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jsvd commented Oct 8, 2025 •

edited

Loading