-
Notifications
You must be signed in to change notification settings - Fork 526
[Nginx Ingress Controller Logs] added missing filestream fingerprint settings #16913
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Added configuration options for fingerprinting in the manifest.
Added configuration options for fingerprinting in the manifest.
Add fingerprint settings for container-logs in version 1.14.0.
Added fingerprint settings for container-logs in version 1.14.0.
|
I have tested the build locally and all work fine. Way to test is:Prerequisites to have a k8s cluster running and also install the elastic-package
elastic-package build
Build the package
2026/01/12 13:51:56 INFO License text found in "/Users/andreasgkizas/integrations/integrations/LICENSE.txt" will be included in package
README.md file rendered: /Users/andreasgkizas/integrations/integrations/packages/nginx_ingress_controller/docs/README.md
Package built: /Users/andreasgkizas/integrations/integrations/build/packages/nginx_ingress_controller-1.14.0.zip
Done
See below the relevant text-boxes that appear in version 1.14.0 of the integration FYI there is an warning that appears by default in the logs {"log.level":"warn","@timestamp":"2026-01-12T10:49:27.159Z","message":"1 file is too small to be ingested, files need to be at least 1024 in size for ingestion to start. To change this behaviour set 'prospector.scanner.fingerprint.length' and 'prospector.scanner.fingerprint.offset'. Enable debug logging to see all file names.","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-default","type":"filestream"},"log":{"source":"filestream-default"},"ecs.version":"1.6.0","log.logger":"input.scanner","log.origin":{"file.line":423,"file.name":"filestream/fswatch.go","function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*fileScanner).GetFiles"},"service.name":"filebeat","filestream_id":"nginx-ingress-controller-access-7a902c467ad8c61e88a0f9dc70af800e3c3cb2b3b9d9d31b2c41b102f527bc8a","ecs.version":"1.6.0"}You need to update example prospector.scanner.fingerprint.length: 70 in the box that appears above. |
gizas
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
@andrewkroh have u seen this behaviour again regarding Does it have sth to do with the fork and/or with the note above: |

Proposed commit message
This commit should fix a problem we detected using this integration. It seems to stop scraping nginx-ingress-controller logs from Kubernetes clusters with elastic-agents installed. Unfortunately I do not have a lot of knowledge about creating/maintaining integrations and currently have no option to test this change somehow, but by comparing the Kubernetes integration package files with this integration I think I figured out what's missing so that the agent's Filebeat detects container-log rotations correctly and continues to scrape the needed logs.
I think only the fingerprint settings similarly like in the Kubernetes integration are missing in the access- and error-log files - like in
https://github.com/elastic/integrations/blob/main/packages/kubernetes/data_stream/container_logs/manifest.yml#L90-L106
and
https://github.com/elastic/integrations/blob/main/packages/kubernetes/data_stream/container_logs/agent/stream/stream.yml.hbs#L20-L26
Checklist
changelog.ymlfile.