container_name returns <unkonwn> #11
Description
Hello,
I have a problem using your plugin with an EFK stack.
Every thing seems OK at startup, the conf is exactly the same as yours, I can see everything up, logs are forwarded to elasticsearch and Kibana retrieves them.
But my container_name remains at value.
I checked your code and did not notice anything where you get container_name.
One thing i noticed is that i have to use sudo to open config.v2.json. Maybe your plugin does not have read access on it ? (see the CONTAINER FOLDER RIGHTS part).
Can you have a look please, and give me some tips to debug your pluign (i am a Java developer...)
CONF
log_level debug@log_name e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b
@timestamp October 18th 2016, 16:31:06.000
t_id AVfYNAJ2sUxcrcA5hnKg
t_index logstash-2016.10.18
_score
t_type fluentd
tcontainer_id e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b
tcontainer_name
tlog �[2m2016-10-18 14:31:06.083�[0;39m �[32m INFO�[0;39m �[35m7�[0;39m �[2m---�[0;39m �[2m[ main]�[0;39m �[36mo.a.camel.spring.SpringCamelContext �[0;39m �[2m:�[0;39m StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
tstream stdout
FLUENTD
fluentd_2 | 2016-10-18T14:29:57.190742756Z Fluentd waithing TCP connection :...
fluentd_2 | 2016-10-18T14:29:58.194248978Z Start fluentd
fluentd_2 | 2016-10-18T14:29:58.461565888Z 2016-10-18 14:29:58 +0000 [info]: fluent/supervisor.rb:471:read_config: reading config file path="/fluentd/etc/fluent.conf"
fluentd_2 | 2016-10-18T14:29:58.464357222Z 2016-10-18 14:29:58 +0000 [info]: fluent/supervisor.rb:337:supervise: starting fluentd-0.12.29
fluentd_2 | 2016-10-18T14:29:58.543209510Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-mixin-config-placeholders' version '0.4.0'
fluentd_2 | 2016-10-18T14:29:58.543257242Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-docker-format' version '0.2.3'
fluentd_2 | 2016-10-18T14:29:58.543317650Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-elasticsearch' version '1.7.0'
fluentd_2 | 2016-10-18T14:29:58.543408305Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-s3' version '0.7.1'
fluentd_2 | 2016-10-18T14:29:58.543525763Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluentd' version '0.12.29'
fluentd_2 | 2016-10-18T14:29:58.543638143Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluentd' version '0.10.61'
fluentd_2 | 2016-10-18T14:29:58.543915201Z 2016-10-18 14:29:58 +0000 [info]: fluent/agent.rb:129:add_match: adding match pattern="docker.var.lib.docker.containers...log" type="docker_format"
fluentd_2 | 2016-10-18T14:29:58.546189569Z 2016-10-18 14:29:58 +0000 [info]: fluent/agent.rb:129:add_match: adding match pattern="" type="copy"
fluentd_2 | 2016-10-18T14:29:58.546478263Z 2016-10-18 14:29:58 +0000 [debug]: plugin/out_copy.rb:44:block in configure: adding store type="elasticsearch"
fluentd_2 | 2016-10-18T14:29:58.654633862Z 2016-10-18 14:29:58 +0000 [debug]: plugin/out_copy.rb:44:block in configure: adding store type="s3"
fluentd_2 | 2016-10-18T14:29:58.706809710Z 2016-10-18 14:29:58 +0000 [info]: fluent/root_agent.rb:147:add_source: adding source type="tail"
fluentd_2 | 2016-10-18T14:29:58.717655521Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:133:configure: using configuration file:
fluentd_2 | 2016-10-18T14:29:58.717667174Z
fluentd_2 | 2016-10-18T14:29:58.717670825Z log_level debug
fluentd_2 | 2016-10-18T14:29:58.717673963Z
fluentd_2 | 2016-10-18T14:29:58.717677094Z
fluentd_2 | 2016-10-18T14:29:58.717680053Z type tail
fluentd_2 | 2016-10-18T14:29:58.717683043Z path /var/lib/docker/containers//-json.log
fluentd_2 | 2016-10-18T14:29:58.717686178Z pos_file fluentd-docker.pos
fluentd_2 | 2016-10-18T14:29:58.717689108Z time_format %Y-%m-%dT%H:%M:%S
fluentd_2 | 2016-10-18T14:29:58.717692020Z tag docker.*
fluentd_2 | 2016-10-18T14:29:58.717694865Z format json
fluentd_2 | 2016-10-18T14:29:58.717697657Z
fluentd_2 | 2016-10-18T14:29:58.717700640Z <match docker.var.lib.docker.containers...log>
fluentd_2 | 2016-10-18T14:29:58.717703595Z type docker_format
fluentd_2 | 2016-10-18T14:29:58.717706470Z container_id ${tag_parts[5]}
fluentd_2 | 2016-10-18T14:29:58.717718503Z tag docker.${name}
fluentd_2 | 2016-10-18T14:29:58.717724682Z docker_containers_path /var/lib/docker/containers
fluentd_2 | 2016-10-18T14:29:58.717727907Z
fluentd_2 | 2016-10-18T14:29:58.717730840Z <match docker.
fluentd_2 | 2016-10-18T14:29:58.717733834Z @type copy
fluentd_2 | 2016-10-18T14:29:58.717736595Z
fluentd_2 | 2016-10-18T14:29:58.717739495Z @type elasticsearch
fluentd_2 | 2016-10-18T14:29:58.717742347Z host elasticsearch
fluentd_2 | 2016-10-18T14:29:58.717745172Z port 9200
fluentd_2 | 2016-10-18T14:29:58.717747949Z include_tag_key true
fluentd_2 | 2016-10-18T14:29:58.717750819Z tag_key @log_name
fluentd_2 | 2016-10-18T14:29:58.717753656Z logstash_format true
fluentd_2 | 2016-10-18T14:29:58.717756494Z flush_interval 5s
fluentd_2 | 2016-10-18T14:29:58.717759227Z
fluentd_2 | 2016-10-18T14:29:58.717762106Z
fluentd_2 | 2016-10-18T14:29:58.717764943Z @type s3
fluentd_2 | 2016-10-18T14:29:58.717767631Z aws_key_id xxxxxx
fluentd_2 | 2016-10-18T14:29:58.717770376Z aws_sec_key xxxxxx
fluentd_2 | 2016-10-18T14:29:58.717773109Z s3_bucket fluentd
fluentd_2 | 2016-10-18T14:29:58.717775893Z s3_region eu-central-1
fluentd_2 | 2016-10-18T14:29:58.717778631Z path logs/
fluentd_2 | 2016-10-18T14:29:58.717781389Z buffer_path /var/log/fluent/s3
fluentd_2 | 2016-10-18T14:29:58.717784193Z time_slice_format %Y%m%d%H
fluentd_2 | 2016-10-18T14:29:58.717786958Z time_slice_wait 5s
fluentd_2 | 2016-10-18T14:29:58.717789730Z utc
fluentd_2 | 2016-10-18T14:29:58.717793708Z buffer_chunk_limit 256m
fluentd_2 | 2016-10-18T14:29:58.717796698Z
fluentd_2 | 2016-10-18T14:29:58.717799677Z
fluentd_2 | 2016-10-18T14:29:58.717802648Z
fluentd_2 | 2016-10-18T14:29:59.612179183Z 2016-10-18 14:29:59 +0000 [info]: plugin/in_tail.rb:557:initialize: following tail of /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b-json.log
CONTAINER FOLDER RIGHTS
[...]$ sudo ls -al /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b
total 4644
drwx------ 3 root root 4096 Oct 18 14:30 .
drwx------ 12 root root 20480 Oct 18 14:31 ..
-rw-rw-rw- 1 root root 9708 Oct 18 14:30 config.v2.json
-rw-r----- 1 root root 4692269 Oct 18 14:31 e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b-json.log
-rw-rw-rw- 1 root root 1282 Oct 18 14:30 hostconfig.json
-rw-r--r-- 1 root root 13 Oct 18 14:30 hostname
-rw-r--r-- 1 root root 174 Oct 18 14:30 hosts
-rw-r--r-- 1 root root 109 Oct 18 14:30 resolv.conf
-rw-r--r-- 1 root root 71 Oct 18 14:30 resolv.conf.hash
drwxrwxrwt 2 root root 40 Oct 18 14:30 shm
CONFIG.V2.JSON
[...]$ sudo cat /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/config.v2.json
{"State":{"Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"RemovalInProgress":false,"Dead":false,"Pid":6820,"StartedAt":"2016-10-18T14:30:40.006984652Z","FinishedAt":"2016-10-18T14:30:35.585791768Z","Health":null},"ID":"e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b","Created":"2016-10-17T17:42:42.568174797Z","Managed":false,"Path":"/bin/sh", ........,"Config":{"Hostname":"e47bced8e497","Domainname":"","User":"",
[...]
,"LogPath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b-json.log","Name":"/r-stack_my-container","Driver":"overlay","MountLabel":"","ProcessLabel":"","RestartCount":0,"HasBeenStartedBefore":false,"HasBeenManuallyStopped":false,"MountPoints":{"/tmp":{"Source":"","Destination":"/tmp","RW":true,"Name":"3f92ecaad901e05cc4ce24f998a4605a2861590ff22f3924abe594e79c8b8262","Driver":"local","Relabel":"","Propagation":"","Named":false,"ID":"13a83ceabb05a0128342ba0b52db7443de61bd03ed95a36ee9356c66c658af72"}},"AppArmorProfile":"","HostnamePath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/hostname","HostsPath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/hosts","ShmPath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/shm","ResolvConfPath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/resolv.conf","SeccompProfile":"","NoNewPrivileges":false}
CAT
[...]$ cat /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b
cat: can't open '/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b': Permission denied
WHEREAS
sudo cat /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b-json.
shows file content