While developing my Meteor application I have started to require running my Mongo process separate from the process that Meteor fires up when it is run. Because I have a port open to my running app on a separate computer, I put the DB on the computer servicing the port. For performance reasons you will want to be running a replica set so that DB change events can be fired by OPLOG tailing. OPLOG tailing is much more responsive than polling for changes. For development you only really need a single instance replica set and that is what I describe here.
There are a couple of steps to setting this up:
- My Mongo config options - this will tell give you a set of config options that are necessary for setting up this DB.
- Start the database.
- Initialize replica set. - to get the replica set working you need to do an
rs.initiate(). - Intialize the admin DB and your DB.
- Set up an oplog reader user in your mongo DB.
- Set up environment variables and run meteor.
Note: using mongo shell seemed to be better than using Robomongo. The errors produced are better.
1. My Mongo config options
I am finding that using mongo configuration files is really the way to go for me. So here is my example config file and I will describe what each section means:
A quick warning about mongo config files, they use YAML as their file type. YAML is an indentation standard and it can have trouble with tabs v. spaces and I was unsure if/where quoting was used. This YAML linter is a good tool to use to make sure your file is formatted correctly.
On Windows:
systemLog:
destination: file
path: c:\<DB>\<DBNAME_LOG>\<DBNAME>.log
storage:
dbPath: c:\<DB>\<DBNAME>
net:
bindIp: <DBHOST>
port: 27017
replication:
replSetName: rs0
oplogSizeMB: 8
This is pretty simple it specifies that I am logging to a file and if I log to a file I also am required to specify a path.
Here I am simply specifying the path to where the DB is on disk.
net:
This one is important and valuable. If you want your mongo DB to be able to connect using your LAN you are required to enter the bindIP option correctly. This address must be accessible from any client server that accesses the DB. I specify the LAN address of the computer running Mongo ( <DBHOST>). Each of my server host machines have access to <DBHOST>. Now other computers on my LAN can access this DB. That way I can develop my Meteor app on one machine and host my mongoDB on another machine.
The port is simply the port to connect to Mongo. I use the default 27017.
Meteor supports reading the Mongo OpLog for observe operations. This provides a much more cpu efficient and is way faster than polling.
There are a number of "how tos" out there, but none that specifically focus on setting up a "single" replica. They all seem to set up a replica set of at least 3 processes to initialize the opLog. It is much simpler to set up a single replica.
This is the only option that is required to set up the replica set. You just have to name it.
I was groveling through the meteor code and found that they set up an oplogSizeMB of 8 MB for development so I followed their lead.
Optional: If you want to access this DB from a separate computer you will need to open up the firewall. See instructions here.
You want to set up a Windows service. I followed the instructions here. Note: if you want to change config settings you just need to change the config file, stop the service and start it again. There is a restart option on the service in the Task Manager -> Services tab.
On Lunix based machines you need only kick off mongod with the --fork flag. Like this:
mongod --fork --config <DB>/<db>.cfg
If you want to change config settings on an OSX machine you will need to kill this processes and restart it.
Connect to your database (you can use mongo shell, I have been using robomongo). Run the rs.initiate() command:
rs.initiate({
"_id" : "rs0",
"members" : [
{ "_id": 0, "host" : "<DBHOST>:27017" }
]
})
This sets 2 key variables:
It is important that these 2 variables are set properly, you can confirm their settings with a rs.conf() command.
That's it! Now you should be able to examine collections on the DB.
To properly set up a user on your DB, you will need to add the user to the admin DB. But first the admin DB has to be initialized. Execute these mongo shell commands:
use admin # Sets your current DB to admin
db.foo.insert({foo: "foo"}) # This will insert a collection named foo with a document
# that has one property foo set to foo
use <DBNAME> # Set your current DB to <DBNAME>
db.foo.insert({foo: "foo"}) # insert a dummy collection
db.show # In addition to local you should now see admin and DBNAME databases
All they do is toss some data into the DB so that the DB initializes. You can remove the foo collections once the DBs are operational with some real data.
In order for Meteor to utilize the oplogs it needs to have access to the logs. While connected to your database you can grant access by creating a user in the DB that has oplog read access using db.createUser:
use admin
db.createUser({ user: "logreader",
pwd: "password",
roles: ["read"]
})
Make sure you add the user to the admin DB. use admin then create the user.
Other useful functions:
db.getUser("username")
db.changeUserPassword("username", "newPassword")
db.dropUser("username")
Finally, we can set up the environment variables required by Meteor:
MONGO_URL=mongodb://<DBHOST>:27017/<DBNAME>?replicaSet=rs0&connectTimeoutMS=60000&socketTimeoutMS=60000
MONGO_OPLOG_URL=mongodb://logreader:password@<DBHOST>:27017/local?authSource=admin
The MONGO_URL environment variable contains some arguments:
replicaSet=rs0: this replica set name matches the one set on rs.initiate().connectionTimeoutMS=60000: Healthy 60 second connection timeoutsocketTimeoutMS=60000: Healthy 60 second socket timeout
The MONGO_OPLOG_URL sets up the authenticated oplog reader URL.
Set these environment variables in your shell and you can run meteor and your database will be running where you specified.