July 26, 2017

What is Kadira you ask? Kadira is a performance monitoring tool for Meteor. “Wait, what’s Meteor?”, I’m so glad you asked! Meteor is a development ecosystem that we use to build products such as Happy Boards. We rely on Kadira heavily for monitoring things such as memory and CPU usage as well as error logging.

As of May 15th, 2017, Kadira shut down its services leaving a lot of people looking for alternatives. Unfortunately, there are none that work with Meteor as seamless as Kadira, mainly because Kadira was built specifically for Meteor. Luckily, the Kadira source code was released as open source on GitHub! However, people are still having troubles getting their own instances of Kadira running so here I’m going to show you what worked for me.

In this article, we’re going to be using Digital Ocean for our server hosting, and Compose for hosting our database. We’re also going to be using Cloudflare for our domain management so that we can take advantage of their Free One-Click SSL.

 

Initial Setup

So, let’s get started!

 

Creating The Server

First, log into your Digital Ocean account and create a new droplet. You’ll want to set the operating system to Ubuntu v16.04.2 x64 and choose a droplet size that has at least 4GB of memory.

Screen Shot 2017-07-21 at 10.14.22 AM

After that is finished, jot down the IP address that was assigned to your freshly created server and head on over to your Cloudflare account.

 

Setting Up The Domains

You’re going to need to create DNS entries for two domains or sub-domains (or combination of the two) that both point to your server’s IP address.

So, for this article, the IP address for my server is 138.197.78.175 and I’m going to be creating kadira.gethappyboards.com and kadira-engine.gethappyboards.com that both point to that IP address.

Screen Shot 2017-07-21 at 10.47.48 AM

Note: It shouldn’t matter if the SSL setting for the domain is set to “Flexible” or “Full” just as long as it’s not set to “Off”.

After you’re finished with that, the real fun begins.

 

Preparing The Server

Open up your favorite terminal application and SSH into your server as root.

Screen Shot 2017-07-21 at 11.06.48 AM

Creating The User

Create a new user called kadira (Be sure to write down the password you give it).

adduser kadira

Give it root privileges.

usermod -aG sudo kadira

Log in as kadira.

su - kadira

Make sure it has root privileges.

sudo ls -la /root

 

Installing Dependencies

Next, we’re going to need to install some dependencies. Run these commands:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list
sudo apt-get update
sudo apt-get install -y mongodb-org nano nginx nodejs npm python ufw
curl https://install.meteor.com/ | sh

The packages you just installed are:

  • Meteor – Development ecosystem.
  • MongoDB – Document oriented NoSQL database program.
  • Nano – Terminal text-editor.
  • NGINX – High-performance HTTP server.
  • Node.js – Javascript runtime.
  • NPM – Package manager for Node.js.
  • Python – Programming language package.
  • UFW – Firewall configuration tool.

For some reason, the nodejs package gets installed to your bin as nodejs when packages that depend on it expect it to be just node. So what we’re going to do is create a symlink:

sudo ln -s /usr/bin/nodejs /usr/bin/node

Let’s make sure that did what we want it to:

node --version

It should have showed you which node version is installed.

 

Configuring The Firewall

We need to configure UFW so that everything runs smoothly:

sudo ufw allow OpenSSH
sudo ufw allow 80
sudo ufw allow 443
sudo ufw allow 4000
sudo ufw allow 11011
sudo ufw enable
sudo ufw status

Here is what you just did:

  • Allowed OpenSSH so that we can get back into the server via SSH after the firewall is enabled.
  • Allowed port 80 so that services can use HTTP.
  • Allowed port 443 so that services can use HTTPS.
  • Allowed port 4000 for the Kadira UI.
  • Allowed port 11011 for the Kadira Engine.
  • Enabled the firewall.
  • Checked the status of the firewall.

After that, you should see something like this:

Screen Shot 2017-07-21 at 12.09.27 PM

Configuring NGINX

We need to create a server-block for Kadira so that the server knows exactly what it needs to do when starting up.

First, delete the default server-block:

sudo rm /etc/nginx/sites-enabled/default

Then, create one specifically for Kadira:

sudo nano /etc/nginx/sites-available/kadira

Nano will then open and present you with a blank file. You’re going to want to copy and paste the following into there of course replacing both server_name domains to the ones we set up in Cloudflare earlier:

server_tokens off;

map $http_upgrade $connection_upgrade {
 default upgrade;
 '' close;
}

# Kadira UI
server {
 listen 80;
 listen 443;
 server_name kadira.gethappyboards.com;

 location / {
   proxy_pass http://127.0.0.1:4000;
   proxy_http_version 1.1;
   proxy_set_header Upgrade $http_upgrade;
   proxy_set_header Connection $connection_upgrade;
   proxy_set_header X-Forwarded-For $remote_addr;
   proxy_set_header Host $host;
 }
}

# Kadira Engine
server {
 listen 80;
 listen 443;
 server_name kadira-engine.gethappyboards.com;

 location / {
   proxy_pass http://127.0.0.1:11011;
   proxy_http_version 1.1;
   proxy_set_header Upgrade $http_upgrade;
   proxy_set_header Connection $connection_upgrade;
   proxy_set_header X-Forwarded-For $remote_addr;
   proxy_set_header Host $host;
 }
}

After you’ve done that, save your changes by pressing Control+X on your keyboard, then Y, and then Enter to confirm your changes.

We then need to enable our newly created server-block and test it:

sudo ln -s /etc/nginx/sites-available/kadira /etc/nginx/sites-enabled/
sudo nginx -t

You should get a response like this:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

At this point, if everything has been going smoothly, go ahead and restart nginx:

sudo systemctl restart nginx

 

Setting Up The Databases

Now you’re going to want to log into your Compose account and create a MongoDB deployment if you don’t have one already. Inside of that deployment you’re going to create two databases, kadira-apps, and kadira-data. You’re also going to want to create a user for each database. (I made the usernames and passwords the same for both for the sake of simplicity.)

While you’re in there, go get your Replica Set URIs for both of your databases. You can find these by clicking into each database and clicking the Admin tab on the left-hand side. For example, mine looks like this:

mongodb://<user>:<password>@candidate.51.mongolayer.com:10113,candidate.50.mongolayer.com:10369/kadira-apps?replicaSet=set-54e751a22c37aa0f6400209a

Save both of these somewhere because you’re going to need them later.

Take your kadira-data URI and put it in front of the mongo command. For example, here’s mine:

mongo mongodb://kadira:[email protected]:10113,candidate.50.mongolayer.com:10369/kadira-data?replicaSet=set-55e152292e37af9f6130129a

You will then be put into a sub-terminal. Run each one of these commands one-by-one:

db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'1min',provider:'methods',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'1min',provider:'errors',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'1min',provider:'pubsub',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'1min',provider:'system',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'3hour',provider:'methods',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'3hour',provider:'errors',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'3hour',provider:'pubsub',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'3hour',provider:'system',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'30min',provider:'methods',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'30min',provider:'errors',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'30min',provider:'pubsub',shard:"one"}})
db.mapReduceProfileConfig.insert({lastTime: new Date(), _id:{profile:'30min',provider:'system',shard:"one"}})

After each command, you should get a response like this:

WriteResult({ “nInserted” : 1 })

And after you’re done entering all of those, run this:

exit

Your databases are now all ready to rock-and-roll.

 

Installing Kadira

Up until now we have only been making preparations for Kadira and now we finally get to install!

cd ~
git clone https://github.com/lampe/kadira-server.git

 

Configuration

We then need to do a little bit of configuration:

cd kadira-server
nano init-shell.sh

Replace the APP_MONGO_URL and DATA_MONGO_URL strings with their appropriate URIs that we saved before.

Since we’re using Compose and they only provide an Oplog if you pay extra, I just went ahead and commented the APP_MONGO_OPLOG_URL line out by placing a # symbol at the very beginning of the line and didn’t run into any issues because of it.

Also, if you would like to be able to use the CPU Profiler feature of Kadira, you will need to set up an Amazon S3 Bucket and put the credentials in the AWS_DEFAULT_REGION, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY lines however this isn’t necessary for basic functionality.

After you’re done with all of that you can go ahead and save your changes by once again pressing Control+X, then Y, and then Enter.

You’ll then want to make that file executable:

chmod +x init-shell.sh

 

We’re now going to configure the five individual pieces of Kadira.

 

Kadira Engine

Get into the kadira-engine folder, install its dependencies, and source the config file we set up above:

cd kadira-engine
npm install
source ../init-shell.sh

Then we want to make the run.sh file executable and test it out:

chmod +x run.sh
./run.sh

You should have gotten a response like this:

starting apm-engine on port 11011
Initialized Mongo Sharded Cluster for shards: one
DDONE

That tells you that the engine is set up correctly, so you can go ahead and press Control+C to stop the engine because we still need to set it up to run on its own.

Create a systemd service file:

sudo nano /etc/systemd/system/kadira-engine.service

and place the following into that file:

[Unit]
Description=kadira-engine

[Service]
Type=simple
User=kadira
WorkingDirectory=/home/kadira/kadira-server
ExecStart=/bin/bash -c "source init-shell.sh; cd kadira-engine; ./run.sh"
Restart=on-abort

[Install]
WantedBy=multi-user.target

Save your changes by pressing Control+X, then Y, and then Enter. And then start the service and test it:

systemctl start kadira-engine.service
systemctl status kadira-engine.service

It will prompt you for the password you set for your Kadira user. After that you should have got a response like this:

● kadira-engine.service – kadira-engine
Loaded: loaded (/etc/systemd/system/kadira-engine.service; disabled; vendor preset: enabled)
Active: active (running) since Fri 2017-07-21 21:54:13 UTC; 7s ago
Main PID: 25549 (bash)
Tasks: 11
Memory: 35.5M
CPU: 1.300s
CGroup: /system.slice/kadira-engine.service
├─25549 /bin/bash -c source init-shell.sh; cd kadira-engine; ./run.sh
├─25552 /bin/bash -c source init-shell.sh; cd kadira-engine; ./run.sh
└─25553 node server.js

That means everything is set up correctly so we can go ahead and enable the service:

systemctl enable kadira-engine.service

You’ll once again be prompted for your password. Then we just want to get back into the main server directory:

cd ../

 

The steps for the next four pieces are very similar to this one so I’m not going to go into as much detail for those.

 

Kadira RMA

Run these commands:

cd kadira-rma
npm install
sudo nano /etc/systemd/system/kadira-rma.service

Paste the following into the text editor:

[Unit]
Description=kadira-rma

[Service]
Type=simple
User=kadira
WorkingDirectory=/home/kadira/kadira-server
ExecStart=/bin/bash -c "source init-shell.sh; cd kadira-rma; ./run.sh"
Restart=on-abort

[Install]
WantedBy=multi-user.target

Save your changes and then start the service and check that it’s working okay:

systemctl start kadira-rma.service
systemctl status kadira-rma.service

If everything seems fine, enable the service:

systemctl enable kadira-rma.service

Get back to the main server directory:

cd ../

 

 

Kadira UI

Run these commands:

cd kadira-ui
npm install
sudo nano /etc/systemd/system/kadira-ui.service

Paste the following into the text editor:

[Unit]
Description=kadira-ui

[Service]
Type=simple
User=kadira
WorkingDirectory=/home/kadira/kadira-server
ExecStart=/bin/bash -c "source init-shell.sh; cd kadira-ui; ./run.sh"
Restart=on-abort

[Install]
WantedBy=multi-user.target

Save your changes and then start the service and check that it’s working okay:

systemctl start kadira-ui.service
systemctl status kadira-ui.service

If everything seems fine, enable the service:

systemctl enable kadira-ui.service

Get back to the main server directory:

cd ../

 

 

Kadira API

Run these commands:

cd kadira-api
npm install
sudo nano /etc/systemd/system/kadira-api.service

Paste the following into the text editor:

[Unit]
Description=kadira-api

[Service]
Type=simple
User=kadira
WorkingDirectory=/home/kadira/kadira-server
ExecStart=/bin/bash -c "source init-shell.sh; cd kadira-api; ./run.sh"
Restart=on-abort

[Install]
WantedBy=multi-user.target

Save your changes and then start the service and check that it’s working okay:

systemctl start kadira-api.service
systemctl status kadira-api.service

If everything seems fine, enable the service:

systemctl enable kadira-api.service

Get back to the main server directory:

cd ../

 

 

Kadira Alertsman

Run these commands:

cd kadira-alertsman
npm install
sudo nano /etc/systemd/system/kadira-alertsman.service

Paste the following into the text editor:

[Unit]
Description=kadira-alertsman

[Service]
Type=simple
User=kadira
WorkingDirectory=/home/kadira/kadira-server
ExecStart=/bin/bash -c "source init-shell.sh; cd kadira-alertsman; ./run.sh"
Restart=on-abort

[Install]
WantedBy=multi-user.target

Save your changes and then start the service and check that it’s working okay:

systemctl start kadira-alertsman.service
systemctl status kadira-alertsman.service

If everything seems fine, enable the service:

systemctl enable kadira-alertsman.service

Get back to the main server directory:

cd ../

 

Setting Up Your Kadira Account

At this point you’re basically all up and running! The last thing we need to do is set up the account we’re going to use on Kadira. Run these two commands:

cd kadira-ui
meteor shell

You will then be put into a sub-terminal. Run the following command after replacing the strings with your desired credentials:

Accounts.createUser({username:'happymedium', email:'[email protected]', password:'kadira123'})

You should get a response with your newly created account ID. Exit the sub-terminal by running:

.exit

At this point, we’re done with the terminal so you can go ahead and exit your SSH session and close your terminal application by running the exit command three times:

exit
exit
exit

Go ahead and visit the main URL that you set up for Kadira before, for example, mine was kadira.gethappyboards.com and log in using the account you just created. If all went well you should be presented with the My Apps screen:
Screen Shot 2017-07-22 at 11.39.02 AM

 

You’ll want to go through the next steps for each Meteor application that you would like to use Kadira with.

 

Setting Up Your App

Go to the My Apps page of Kadira and create an app for your Meteor app that you’re wanting to use Kadira with (leave it on the “Free Plan” for now, we’ll change that later). You should then be presented with your App ID and App Secret. Take both of those as well as the kadira-engine URL we set up before and put them in the settings.json file of your Meteor app like this:

"kadira": {
 "appId": "w7jLHndFR2jHDaAxM",
 "appSecret": "b9021540-2685-436d-b70f-23e05e13b379",
 "options": {
   "endpoint": "https://kadira-engine.gethappyboards.com"
 }
}

Congratulations! Your Meteor app is now connected to Kadira!

 

Upgrading Your App’s Plan

To get the full potential of Kadira, we’re going to upgrade our app to the paid “Business” plan of Kadira.

Log into your Compose account and go to the apps collection inside of your kadira-apps database. Find your app by the appId.

Change the plan key value from “free” to “business” and the pricingType key value from “free” to “paid”.

For example, mine looked like this when all was said and done:

{
 _id: "w7jLHndFR2jHDaAxM",
 name: "happy-boards-production",
 created: ISODate("2017-07-22T16:42:37.104Z"),
 owner: "Ldava8jf5HgiTg7Bt",
 secret: "b9021540-2685-436d-b70f-23e05e13b379",
 plan: "business",
 shard: "one",
 subShard: NumberLong("62"),
 pricingType: "paid"
}

And that’s it! Your app is all connected to Kadira with a plan that they used to charge $360 per month for!

 

Screen Shot 2017-07-22 at 12.04.58 PM

 

Conclusion

Kadira is such a powerful tool for Meteor apps and it’s a shame that they decided to quit offering its services (at least to anybody not using Meteor’s Paid Hosting service). Luckily, they at least released the source code for the community. However, they are not offering any support for it at all so it’s up to the community to help each other out!

Speaking of helping each other out, the biggest help for me was Michael Lazarski’s original post describing how to get Kadira set up on an Ubuntu server. His post is great but I found that I was left to figure out a lot of things on my own so hopefully my post helps people who felt the same.

I also wanted to note that this doesn’t get the e-commerce part of Kadira running nor am I sure if this setup could handle a large amount of users. We’re just using this setup for internal use.

Happy Hacking!