Setting Up a Second Graylog2 Server Node

Technical Context: Ubuntu 14.04, first Graylog2 IP: 11.11.11.11, second Graylog2 IP: 22.22.22.22

1. Install Graylog2

Instructions here.

(Note that the installing the Graylog web interface, graylog-web, is optional).

2. MongoDB

If your MongoDB instance already runs on a seperate machine from any of your Graylog, all you have to do is adjust your firewall rules for that machine (if any exists) to allow the IP address of the new Graylog2 server node to connect to port 27017 (or whatever custom port you’ve defined for your MongoDB instance).

Otherwise

If your MongoDB instance lives on the same machine as an existing Graylog2 node, that means your current configuration (/etc/mongod.conf) will look something like this (it should, or you’re in big trouble):

#port = 27017

# Listen to local interface only. Comment out to listen on all interfaces.
bind_ip = 127.0.0.1

This means that your MongoDB instance is only accessible to other processes running on the same machine. If so, you may or may not have authentication set up on your MongoDB instance - it doesn’t really matter.

You will need to change your MongoDB configuration to listen on a publicly accessible interface. Change bind_ip by either commenting it out, or changing it to 0.0.0.0.

Now that your MongoDB instance is publicly accessible, we’re going to have to take necessary security measures.

MongoDB authentication

Here, I’ll cover authentication in MongoDB very quickly. Open a MongoDB shell, make sure that you’re using the correct database, then create a new user with read and write privileges:

$ mongo
> use graylog2
> db.createUser({ user:"graylogusername", pwd:"graylogpassword", roles:[{role: "readWrite", db:"graylog2"}] })

Once that’s done, we can tell Graylog2 to use these credentials when connecting to MongoDB. In recent versions of Graylog2, the MongoDB connection is recommended to be specified using MongoDB connection string URI format, which may look something like this:

mongodb_uri = mongodb://graylogusername:graylogpassword@127.0.0.1:27017/graylog2

Firewall

After setting up authentication, you’d also want to set up appropriate firewall policies. Specifically, you should allow only the second Graylog2 server node to connect to MongoDB. I wrote a comprehensive guide to using APF and BFD here, which you should read. The APF rule for allowing 22.22.22.22 to connect to port 27017 looks like this:

# from the other graylog node to access MongoDB
tcp:in:d=27017:s=22.22.22.22

3. Graylog2

Most of these instructions come straight from the official docs:

Change is_master to false:

is_master = false

Copy the password_secret from the existing Graylog2 server node:

password_secret = KlU1JJYpKeJq9oy5JsWKSA8sf8aJ8anNnisNs1fWEWjAAq7bI246K42idz79r10E5Z1klrGAhtl1Af2fUp4NxNRAAk31lvVX

Change the MongoDB connection credentials (see above).

Change the Elasticsearch settings to match your first Graylog2 server node’s (most importantly, the elasticsearch_discovery_zen_ping_unicast_hosts setting, which tells Graylog2 which Elasticsearch nodes to connect to)

4. Graylog2 Web Interface

The web interface runs independently of any Graylog2 server nodes, so all we have to do now is inform it about the additional node that we’re adding¹:

$ vim graylog-web-interface.conf

If you were previously running the web interface on the same machine as an existing Graylog server node, then you’d see

graylog2-server.uris="http://127.0.0.1:12900/

which you can append to, like so:

graylog2-server.uris="http://127.0.0.1:12900/,http://22.22.22.22:12900/"

(In case you were wondering, yes, you can run multiple web interfaces for failover purposes, but I’m guessing the web interface is for internal consumption only so this may be overkill.)

Footnotes