Today we're going to deploy a Phoenix application to DigitalOcean using the gatling tool from HashRocket. In the process, we'll configure distillery to generate a release.

Gatling is extremely cool. It takes some setting up, but you end up with a deployment process that feels like heroku, but builds proper erlang releases. Let's get started

Project

We're building a tool to help us organize the content in our Slack. It uses hedwig to connect to our Slack and listens for posts that have a tag, then stores them. It's still very early, but we wanted to deploy it so we could easily hit the API, and it seemed like an excellent thing to write an episode around.

We're starting with the phoenix_slack_posting repo tagged before this episode .

Distillery Release

First, we need to set up our project to build a release with Distillery. We'll add the dependency: 

vim mix.exs
defmodule SlackPosting.Mixfile do
  # ...
  defp deps do
    [
      # ...
      {:distillery, "~> 1.5", runtime: false}
    ]
  end
  # ...
end
mix deps.get

We also need to tweak the production configuration a bit: 

vim config/prod.exs
# ...
# We'll set up phoenix to use an env var to figure out what port to host on
config :slack_posting, SlackPostingWeb.Endpoint,
  http: [port: {:system, "PORT"}], # Use an ENV var for our port
  url: [host: "localhost", port: {:system, "PORT"}], # This is critical for ensuring web-sockets properly authorize.
  cache_static_manifest: "priv/static/cache_manifest.json",
  # We want phoenix to run our server when launched as an otp app
  server: true,
  # Let's expose the version of our app in this config
  version: Application.spec(:slack_posting, :vsn)

# We also need to tell phoenix to serve its endpoints for the OTP release
config :phoenix, serve_endpoints: true

# Let's configure the database with env vars
# I already have these set in my environment
# NOTE: This will mean that these variables are compiled in at compile time -
# we'll be compiling on the server, so it will work fine. Just keep it in mind.
# More details here: https://elixirforum.com/t/what-is-the-difference-between-using-system-port-and-system-get-env-port-in-deployment/1975/4
config :slack_posting, SlackPosting.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: System.get_env("DB_USERNAME"),
  password: System.get_env("DB_PASSWORD"),
  database: System.get_env("DB_DATABASE"),
  hostname: System.get_env("DB_HOSTNAME"),
  pool_size: 20
# ...
# remove prod.secrets.exs bits, we don't use that

Then we build our release: 

mix release.init
MIX_ENV=prod mix release --env=prod
==> Assembling release..
==> Building release slack_posting:0.0.1 using environment dev
==> One or more direct or transitive dependencies are missing from
    :applications or :included_applications, they will not be included
    in the release:

    :exactor

    This can cause your application to fail at runtime. If you are sure
    that this is not an issue, you may ignore this warning.

==> You have set dev_mode to true, skipping archival phase
==> Release successfully built!
    You can run it in one of the following ways:
      Interactive: _build/dev/rel/slack_posting/bin/slack_posting console
      Foreground: _build/dev/rel/slack_posting/bin/slack_posting foreground
      Daemon: _build/dev/rel/slack_posting/bin/slack_posting start

It built our release, but it's missing exactor , which ex_admin depends on. Let's add it to our extra_applications list: 

vim mix.exs
defmodule SlackPosting.Mixfile do
  # ...
  def application do
    [
      # ...
      extra_applications: [
        # ...
        :exactor
      ]
    ]
  end
  # ...
end

Build the release again: 

MIX_ENV=prod mix release --env=prod

Now when we run the app, the Phoenix application will be running on the port specified in our PORT environment variable, which for me is 4000. 

_build/prod/rel/slack_posting/bin/slack_posting console

We need to build our assets - this is an API but it has an ExAdmin interface, so we have some to build via brunch: 

cd assets
./node_modules/.bin/brunch b
cd ..
MIX_ENV=prod mix phx.digest
MIX_ENV=prod mix release --env=prod

Now if we run it, we can visit the app: 

_build/prod/rel/slack_posting/bin/slack_posting console
# visit <localhost:4000> and it's running.

That's got our release running locally.

Deploying with gatling

Now that our distillery release is out of the way, we can deploy with gatling ! First, I'll provision a new DigitalOcean droplet:

provisioning a droplet

I just want the cheapest box they have, in the New York datacenter. I'll do that, and give it my public SSH key so I can get into the droplet.

we have a machine

Awesome. We have some metal to run our app on. Let's ssh in and install Elixir, nginx, and the gatling archive. I'll set up an ssh config for this host, so I don't have to type the IP all the time: 

vim ~/.ssh/config
Host phoenix-slack-posting
  Hostname SOME.IP.WE.GOT
  User root

Now I can ssh into the box as root and I don't have to remember that that's the user for this IP. If you don't use the ssh config file you totally should. 

ssh phoenix-slack-posting

And I'm in! We'll install gatling 's dependencies:

Elixir

We probably want Erlang and Elixir on the box: 

# We'll add the erlang solutions apt repository
wget https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb && dpkg -i erlang-solutions_1.0_all.deb
# Then refresh our list of packages
apt update
# I'm going to install some very specific versions of erlang and elixir because
# of an issue with gatling that I ran across
# https://github.com/hashrocket/gatling/issues/53
apt-cache showpkg esl-erlang | more
apt-cache showpkg elixir | more
# note the available versions above, use pre-20 and pre-1.5 sadly
apt install -y esl-erlang=1:19.3.6
apt install -y elixir=1.4.5-1
# hold the versions to avoid auto-update
apt-mark hold esl-erlang elixir

nginx and git

gatling wants nginx and git installed, and what gatling wants gatling gets. 

apt install -y nginx git

JavaScript things

We will build the assets on the box. Consequently, this box needs node. We're also going to include build tools on the box. Don't do this on an important deployment. 

curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -
apt install -y nodejs build-essential

Databases are good

We'll run a database on this box and talk to it. Let's install postgres: 

apt install -y postgresql

Let's add a database user, named deploy: 

sudo -u postgres createuser -s -P deploy

We'll enter the password deploypassword here. We'll also create a database for the user: 

sudo -u postgres createdb deploy

The gatling archive

Then we need to install gatling on the box itself. We probably want a user. We'll make a deploy user: 

adduser deploy

We want deploy to be able to sudo without requiring a password, so we'll do that: 

visudo -f /etc/sudoers.d/deploy

Put this in there: 

deploy  ALL=(ALL) NOPASSWD:ALL

Then we'll become deploy and use him from now on. 

su deploy
cd
mkdir ~/.ssh
# We'll set up an authorized key so we can log in as this user without a
# password in the future.
vim ~/.ssh/authorized_keys
# copy in my key

And we'll install the gatling mix archive so we can use it as this user: 

mix archive.install https://github.com/hashrocket/gatling_archives/raw/master/gatling.ez

We'll install hex and rebar: 

mix local.hex --force
mix local.rebar --force

Now we want gatling to be ready for our project. We use mix gatling.load with our application name as the argument. 

mix gatling.load slack_posting

This creates a git repository on our server. Now we'll go back into our project and add a domains file that lists the domains our API will respond on: 

vim domains
phoenix-slack-posting-test.dailydrip.com

And add it to git: 

git add .
git commit -m"added gatling deployment"

I'll add an A record for this subdomain quickly off-screen.

We also need to have gatling build the assets before deployment. We can use its lifecycle hooks to do this. We make a deploy.exs file in the root of our project: 

vim deploy.exs
defmodule SlackPosting.DeployCallbacks do
  import Gatling.Bash

  def before_mix_digest(env) do
    bash("mkdir", ~w[-p priv/static], cd: env.build_dir)
    bash("npm", ~w[install], cd: env.build_dir <> "/assets")
    bash("npm", ~w[run deploy], cd: env.build_dir <> "/assets")
  end
end

We can also run migrations before upgrading our app. To do this, we add another file in the root of our project: 

vim upgrade.exs
defmodule SlackPosting.UpgradeCallbacks do
  import Gatling.Bash

  def before_mix_digest(env) do
    bash("npm", ~w[install], cd: env.build_dir <> "/assets")
    bash("npm", ~w[run deploy], cd: env.build_dir <> "/assets")
  end

  def before_upgrade_service(env) do
    bash("mix", ~w[ecto.migrate], cd: env.build_dir)
  end
end

And commit them: 

git add .
git commit -m"Add deploy, upgrade scripts for gatling"

Now we can push the app to the box with gatling for the first time: 

git remote add production deploy@phoenix-slack-posting:slack_posting
git push production feature/gatling:master
# In this case, I'm pushing my local feature branch to the remote's master
# normally you would just do:
#
# git push production master

We need to be running production mode, so we'll tweak the server's environment. Now's a good time to set the env vars we prepared for configuration earlier as well: 

ssh phoenix-slack-posting
vim /etc/environment
# ...
MIX_ENV=prod
DB_USERNAME=deploy
DB_PASSWORD=deploypassword
DB_DATABASE=phoenix_slack_posting_production
DB_HOSTNAME=localhost
SLACK_TOKEN="this_is_secret"

We'll reboot for good measure. Our environment will now have these variables set. 

reboot
# make tea
ssh deploy@phoenix-slack-posting
echo $DB_USERNAME
#=> deploy

Now we'll do our initial deployment. On the droplet: 

sudo --preserve-env mix gatling.deploy slack_posting

That's deployed our app. However, as it stands it doesn't seem to create my database, and I can't quite figure out why. Regardless, we can do it ourselves: 

cd ~/slack_posting
PORT=4000 mix ecto.setup

Now we can view the app at the domain:

http://phoenix-slack-posting-test.dailydrip.com

Let's make a small tweak to the homepage, and see upgrades work: 

vim lib/slack_posting_web/templates/page/index.html.eex
<div class="jumbotron">
  <h2>DailyDrip Phoenix Slack Posting API</h2>
</div>

We also have to bump our version in order for an upgrade to happen: 

vim mix.exs
defmodule SlackPosting.Mixfile do
  # ...
  def project do
    [
      # ...
      version: "0.0.2",
      # ...
    ]
  end
  # ...
end
git add .
git commit -m"Update homepage"
git push production feature/gatling:master

And we can see gatling building our release and upgrading it in place. If we visit it, we'll see the new code was hot upgraded and our changes took effect.

Summary

In today's episode, we added a distillery configuration to build releases for our app, set up a DigitalOcean droplet from scratch, and configured gatling deployments. Now we can deploy our app, with hot upgrades, as easily as if we were pushing to Heroku. See you soon!

Resources