Duplicate declaration prevents installation

[kn0ll]

when trying to apply the configuration to a client:

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: File[/var/lib/elasticsearch] is already declared in file /etc/puppet/modules/elasticsearch/manifests/init.pp at line 111; cannot redeclare at /etc/puppet/modules/elasticsearch/manifests/init.pp:118 on node search0

i'm assuming this is because $esDataPath and /var/lib/${esBasename} evaluate to the same path, hence a redeclaration.

am i doing something wrong? thanks!

[BadSector82]

I am experiencing the same issue, did you ever resolve it?

[kn0ll]

@BadSector82 i resolved this by simply commenting out the offending duplicate declarations in init.pp. unless i'm missing something it seems straight forward- in that, even the comments suggest they're doing the same exact thing.

iirc, once i commented out this declaration, the same error arose in other locations for other directories- all of which i just commented out as well. after removing all the duplicates (which you can find simply by trancing the error each time you run it), the install seemed to work fine!

[shengjie-min]

can you guys post a list of duplicates, I am commenting out the duplicates as well. Does it make sense to submit a fix for this?

[BadSector82]

Hi Shenjie,

In the end, I had to change a couple of things in the init.pp script to make it work in our environment. I'm posting our entire script - it should make it easier to spot the differences. The amended sections are marked with comments.

# Class: elasticsearch
#
# This class installs Elasticsearch
#
# Usage:
# include elasticsearch

class elasticsearch($version = "0.15.2", $xmx = "2048m") {
      $esBasename       = "elasticsearch"
      $esName           = "${esBasename}-${version}"
      $esFile           = "${esName}.tar.gz"
      $esServiceName    = "${esBasename}-servicewrapper"
      $esServiceFile    = "${esServiceName}.tar.gz"
      $esPath           = "${ebs1}/usr/local/${esName}"
      $esPathLink       = "/usr/local/${esBasename}"
      $esDataPath       = "${ebs1}/var/lib/${esBasename}"
      $esLibPath        = "${esDataPath}"
      $esLogPath        = "${ebs1}/var/log/${esBasename}"
      $esXms            = "256m"
      $esXmx            = "${xmx}"
      $cluster          = "${esBasename}"
      $esTCPPortRange   = "9300-9399"
      $esHTTPPortRange  = "9200-9299"
      $esUlimitNofile   = "32000"
      $esUlimitMemlock  = "unlimited"
      $esPidpath        = "/var/run"
      $esPidfile        = "${esPidpath}/${esBasename}.pid"
      $esJarfile        = "${esName}.jar"

      # Ensure the elasticsearch user is present
      user { "$esBasename":
               ensure => "present",
               comment => "Elasticsearch user created by puppet",
               managehome => true,
               shell   => "/bin/false",          
      #Removed by chris         require => [Package["sun-java6-jre"], lvmconfig[$ebs1]],
               uid => 901
     }

     file { "/etc/security/limits.d/${esBasename}.conf":
            content => template("elasticsearch/elasticsearch.limits.conf.erb"),                                                                                                    
            ensure => present,
            owner => root,
            group => root,
     }

#     file { "/etc/init/${esBasename}.conf":
#          content => template("elasticsearch/upstart.elasticsearch.conf.erb"),
#          ensure => present,
#          owner => root,
#          group => root,
#          mode => 644
#     }

     exec { "mkdir-ebs-mongohome":
          path => "/bin:/usr/bin",
          command => "mkdir -p $ebs1/usr/local",
          before => File["$esPath"],
          require => user["$esBasename"]
     }    

     # Make sure we have the application path
     file { "$esPath":
             ensure     => directory,
             require    => User["$esBasename"],
             owner      => "$esBasename",
             group      => "$esBasename", 
             recurse    => true
      }

      # Temp location
      file { "/tmp/$esFile":
             source  => "puppet:///elasticsearch/$esFile",
             require => File["$esPath"],
             owner => "$esBasename"
      }

      # Remove old files and copy in latest
      exec { "elasticsearch-package":
             path => "/bin:/usr/bin",
             command => "mkdir -p $esPath && tar -xzf /tmp/$esFile -C /tmp && sudo -u$esBasename cp -rf /tmp/$esName/. $esPath/. && rm -rf /tmp/$esBasename*", 
             unless  => "test -f $esPath/bin/elasticsearch",
             require => File["/tmp/$esFile"],
             notify => Service["$esBasename"],
      }

      ## Note: this is a bit hackish, need to stop the old elasticsearch when upgrading
      exec { "stop-elasticsearch-version-change":
           path => "/bin:/usr/bin",    #added by chris
           command => "service elasticsearch stop",
           unless => "ps aux | grep ${esName} | grep -v grep",
           onlyif => "ps aux | grep ${esBasename} | grep -v grep",
           require => Exec["elasticsearch-package"],
           notify => Service["$esBasename"]
      }

      # Create link to /usr/local/<esBasename> which will be the current version
      file { "$esPathLink":
           ensure => link,
           target => "$esPath",
           require => Exec["stop-elasticsearch-version-change"] 

      }

      # Ensure the data path is created
      file { "$esDataPath":
           ensure => directory,
           owner  => "$esBasename",
           group  => "$esBasename",
           require => Exec["elasticsearch-package"],
           recurse => true           
      }

      # Ensure the data path is created
      #Removed by chris
      #file { "/var/lib/${esBasename}":
      #     ensure => link,
      #     target => "${esDataPath}",
      #     require => File["$esDataPath"],
      #}


      # Ensure the link to the data path is set
      file { "$esPath/data":
           ensure => link,
           force => true,
           target => "$esDataPath",
           require => File["$esDataPath"]
      }

      # Symlink config to /etc
      file { "/etc/$esBasename":
             ensure => link,
             target => "$esPathLink/config",
             require => Exec["elasticsearch-package"],
      }

      # Apply config template for search
      file { "$esPath/config/elasticsearch.yml":
             content => template("elasticsearch/elasticsearch.yml.erb"),
             require => File["/etc/$esBasename"]      
      }

      # Stage the Service Package
      file { "/tmp/$esServiceFile":
           source => "puppet:///elasticsearch/$esServiceFile",
            require => Exec["elasticsearch-package"]
      }

      # Move the service wrapper into place
      exec { "elasticsearch-service":
             path => "/bin:/usr/bin",
             unless => "test -d $esPath/bin/service/lib",
             command => "tar -xzf /tmp/$esServiceFile -C /tmp && mv /tmp/$esServiceName/service $esPath/bin && rm /tmp/$esServiceFile",
             require => [file["/tmp/$esServiceFile"], user["$esBasename"]]
      }

      # Ensure the service is present
      file { "$esPath/bin/service":
           ensure => directory,
           owner  => elasticsearch,
           group  => elasticsearch,
           recurse => true,
           require => Exec["elasticsearch-service"]
      }

      # Set the service config settings
      file { "$esPath/bin/service/elasticsearch.conf":
             content => template("elasticsearch/elasticsearch.conf.erb"),
             require => file["$esPath/bin/service"]
      }

      # Add customized startup script (see: http://www.elasticsearch.org/tutorials/2011/02/22/running-elasticsearch-as-a-non-root-user.html)
      file { "$esPath/bin/service/elasticsearch":
             source => "puppet:///elasticsearch/elasticsearch",
             require => file["$esPath/bin/service"]
      }

      # Create startup script
      file { "/etc/init.d/elasticsearch":
             ensure => link,
             target => "$esPath/bin/service/./elasticsearch",
             require => [Exec["stop-elasticsearch-version-change"], File["$esPath/bin/service/elasticsearch"]]
      }

      # Ensure logging directory
      file { "$esLogPath":
           owner     => "$esBasename",
           group     => "$esBasename",
           ensure    => directory,
           recurse   => true,
           require   => exec["elasticsearch-package"],
      }

      # Ensure logging link is in place
      #Removed by chris
      #file { "/var/log/$esBasename":
      #     ensure => link,
      #     target => "$esLogPath",
      #     require => [File["${esLogPath}"], File["/etc/init.d/$esBasename"]]
      #}


      file { "$esPath/logs":
           ensure => link,
           target => "/var/log/$esBasename",
           force => true,
           require => File["/var/log/$esBasename"]
      }

      # Ensure the service is running
      service { "$esBasename":
            enable => true,
            ensure => running,
            hasrestart => true,
            require => File["$esPath/logs"]
      }

}

You can then kick it off with something like this:

class elasticsearch::wrapper  {
  import "elasticsearch/init.pp"
     class { elasticsearch: version => "0.19.10" }
 }

Hope this helps!

[danieljamesscott]

As far as I can tell, it's because '$ebs1' is undefined. Anyone know what this means and what it should be?

[lrhazi]

Yeah, what is $ebs1 and also what is lvmconfig[$ebs1] ?

[hunt3r]

hey guys, I have to apologize for this component... When I wrote this thing I was just trying to get something working for our needs, and open sourced it based on the fact that at the time, there was no puppet modules for elastic search, hence writing our own. I was not necessarily concerned with creating an abstracted component.

If I remember, $ebs1 was a reference to the Elastic Block Storage device path on the EC2 instance we would be installing to. All our DB servers had two EBS drives if I remember. Visibiz was forced to shut down operations in late 2011 so I don't remember exactly. Do what you will with this, I am no longer maintaining it. I believe there are several forks from this that may be a better and more abstracted solution at this point though, so look around.

[fredstluka]

Hi Chris,

Yeah, I've seen a dozen or so forks since you created this.
Good job!

Working with Matt, I hear... Cool! You guys will get along great.
I always though he'd be a great addition to Visibiz someday. I
worked with him for about 6 years when he was in grade school
and high school, teaching him computer stuff one evening per
week at my sister's house.

--Fred

Fred Stluka -- mailto:fred@bristle.com -- http://bristle.com/~fred/
Bristle Software, Inc -- http://bristle.com -- Glad to be of service!

Open Source: Without walls and fences, we need no Windows or Gates.

On 2/8/13 5:13 PM, hunt3r wrote:

hey guys, I have to apologize for this component... When I wrote this
thing I was just trying to get something working for our needs, and
open sourced it based on the fact that at the time, there was no
puppet modules for elastic search, hence writing our own. I was not
necessarily concerned with creating an abstracted component.

If I remember, $ebs1 was a reference to the Elastic Block Storage
device path on the EC2 instance we would be installing to. All our DB
servers had two EBS drives if I remember. Visibiz was forced to shut
down operations in late 2011 so I don't remember exactly. Do what you
will with this, I am no longer maintaining it. I believe there are
several forks from this that may be a better and more abstracted
solution at this point though, so look around.

—
Reply to this email directly or view it on GitHub
#2 (comment)..