When someone posts a link to something, it would be nice to present the link as a rich bit of data where possible. Slack does this for messages. The standard for doing this sort of thing is called oEmbed . There's an Elixir library for extracting data from oEmbed enabled links, called (interestingly enough) elixir-oembed . Let's use it in Firestorm!

Project

We're starting with the dailydrip/firestorm repo tagged before this episode .

Let's bring in the library and play with it in the REPL a bit: 

vim mix.exs
defmodule FirestormWeb.Mixfile do
  # ...
  defp deps do
    [
      # ...
      {:oembed, "~> 0.1.1"},
      # ...
    ]
  end
  # ...
end

And we'll install it: 

mix deps.get

OK, so we have the package. What can we do with it? 

iex -S mix

First, we'll try the example from the README file: 

{:ok, result} = OEmbed.for("https://www.youtube.com/watch?v=dQw4w9WgXcQ")

Let's look at the result: 

%OEmbed.Video{author_name: "RickAstleyVEVO",
 author_url: "https://www.youtube.com/user/RickAstleyVEVO", cache_age: nil,
 height: 270,
 html: "<iframe width=\"480\" height=\"270\" src=\"https://www.youtube.com/embed/dQw4w9WgXcQ?feature=oembed\" frameborder=\"0\" allowfullscreen></iframe>",
 provider_name: "YouTube", provider_url: "https://www.youtube.com/",
 thumbnail_height: 360,
 thumbnail_url: "https://i.ytimg.com/vi/dQw4w9WgXcQ/hqdefault.jpg",
 thumbnail_width: 480, title: "Rick Astley - Never Gonna Give You Up",
 type: "video", version: "1.0", width: 480}

Here we can see we have an OEmbed.Video which has some html to be used. If we want to use this in Firestorm , we'll need to extract all of the URLs from a given post, get their oembed information, and then show it. We'll want to fetch this, but not for every request. We'll likely end up using a LRU cache to store our cached oEmbed data eventually, but not today.

Let's write a test: 

vim test/unit/forums/oembed_extractor_test.exs
defmodule FirestormWeb.Forums.OembedExtractorTest do
  use ExUnit.Case

  alias FirestormWeb.Forums.OembedExtractor

  test "returns a list of oembed data for a given block of text, by extracting links" do
    url = "https://www.youtube.com/watch?v=H686MDn4Lo8"

    example_text = """
    This is a cool video, check it out: #{url}
    """

    empire_city_elixir_conf_vid =
      %OEmbed.Video{author_name: "Empire City Elixir Conference",
        author_url: "https://www.youtube.com/channel/UCIYiFWyuEytDzyju6uXW40Q",
        cache_age: nil, height: 270,
        html: "<iframe width=\"480\" height=\"270\" src=\"https://www.youtube.com/embed/H686MDn4Lo8?feature=oembed\" frameborder=\"0\" allowfullscreen></iframe>",
        provider_name: "YouTube", provider_url: "https://www.youtube.com/",
        thumbnail_height: 360,
        thumbnail_url: "https://i.ytimg.com/vi/H686MDn4Lo8/hqdefault.jpg",
        thumbnail_width: 480, title: "Real World Elixir Deployment // Pete Gamache",
        type: "video", version: "1.0", width: 480}

    assert [{url, empire_city_elixir_conf_vid}] == OembedExtractor.get_embeds(example_text)
  end
end

This defines what we want to do - get back a list of the URLs and their corresponding oEmbed data. Next, let's write the OembedExtractor so the test passes: 

defmodule FirestormWeb.Forums.OembedExtractor do
  # We're going to match all of the URLs.
  @url_regex ~r(https?://[^ $\n]*)

  def get_embeds(body) do
    # For every URL, we'll spin out a new task that will ultimately return a
    # 2-tuple containing the url in question and the oembed result for it.
    body
    |> get_urls_from_string()
    |> Task.async_stream(fn url -> {url, OEmbed.for(url)} end)
    # If OEmbed.for failed for the url, we'll just filter it out.
    |> Enum.filter(&successful_oembed?/1)
    # Then we have this awkward pattern match that turns the result into what we
    # want to return.
    |> Enum.map(fn {:ok, {url, {:ok, a}}} -> {url, a} end)
  end

  # We add a basic function to filter out failed embeds
  defp successful_oembed?({:ok, {url, {:ok, _data}}}), do: true
  defp successful_oembed?(x) do
    false
  end

  @doc """
  Gathers anything in the string that looks like a link into a list of links.
  """
  def get_urls_from_string(string) do
    Regex.scan(@url_regex, string)
    |> Enum.map(&hd/1)
  end
end

Now we can get the embeds for all of the URLs in any block of text. We'd like to be able to put this list of embeds onto a post. We'll add a virtual field for that: 

defmodule FirestormWeb.Forums.Post do
  # ...
  schema "forums_posts" do
    # ...
    field :oembeds, :any, virtual: true
    # ...
  end
end

Next, let's add a test that verifies that when someone posts an oembed-able link we see the oembed for it: 

defmodule FirestormWeb.Feature.ThreadsTest do
  # ...
  test "thread posts have oembeds rendered", %{session: session} do
    import Page.Thread.Show

    {:ok, [elixir]} = create_categories(["Elixir"])
    {:ok, user} = Forums.create_user(%{username: "knewter", email: "josh@smoothterminal.com", name: "Josh Adams"})
    {:ok, otp_is_cool} = Forums.create_thread(elixir, user, %{title: "Thread with oEmbed in the first post", body: "This is a cool video, check it out: https://www.youtube.com/watch?v=H686MDn4Lo8"})

    session
    |> log_in_as(user)
    |> visit(category_thread_path(FirestormWeb.Web.Endpoint, :show, elixir, otp_is_cool))
    |> assert_has(oembed_for("https://www.youtube.com/watch?v=H686MDn4Lo8"))
  end
  # ...
end
defmodule Page.Thread.Show do
  # ...
  def oembed_for(url) do
    # FIXME: It would be ideal to do this:
    # css(".oembed-for[data-oembed-url='#{url}']")
    # but it failed for dumb reasons so we're just doing this for now.
    css(".oembed-for")
  end
  # ...
end

The test will fail. Next, let's define a means to decorate a given post with its oembed data: 

vim lib/firestorm_web/forums/forums.ex
# ...
  def decorate_post_oembeds(%Post{} = post) do
    oembeds =
      post.body
      |> OembedExtractor.get_embeds()

    %Post{ post | oembeds: oembeds }
  end
# ...

Now that we can get the oembed data onto a post, let's wire this up in the ThreadController and show them in the template: 

  def show(conn, %{"id" => id}, category) do
    # ...
    all_posts =
      thread.posts
      |> Enum.map(&Forums.decorate_post_oembeds/1)

    [first_post | posts] = all_posts
    # ...
  end

We'll update the template - sadly I still have _first_post.html.eex and _post.html.eex so we'll put this in both of them: 

<div class="post-item" id="post-<%= @post.id %>">
  <div class="item-metadata">
    <!-- ... -->
  </div>

  <div class="body">
    <%= markdown(@post.body) %>
  </div>

  <%= for {url, oembed} <- @post.oembeds do %>
    <div class="oembed-for" data-oembed-url="<%= url %>">
      <%= render_oembed(oembed) %>
    </div>
  <% end %>
  <!-- ... -->

We also need to include this in _post.html.eex for both Threads and Users.

We'll define this render_oembed function: 

vim lib/firestorm_web/web/views/oembed_helpers.ex

For Photo , we'll just render an image tag. Otherwise, we'll extract the html version for Video and Rich and mark it safe to render: 

defmodule FirestormWeb.Web.OembedHelpers do
  def render_oembed(%OEmbed.Photo{url: url}) do
    Phoenix.HTML.Tag.img_tag(url)
  end
  def render_oembed(%OEmbed.Video{html: html}) do
    Phoenix.HTML.raw(html)
  end
  def render_oembed(%OEmbed.Rich{html: html}) do
    Phoenix.HTML.raw(html)
  end
  def render_oembed(_), do: nil
end

We want this available everywhere, so we'll include it in web.ex 

defmodule FirestormWeb.Web do
  # ...
  def view do
    quote do
      # ...
      import FirestormWeb.Web.OembedHelpers
      # ...
    end
  end

Now if we run the tests, they pass. We can also create a new post that includes an oEmbed link, and see it working in the template.

Summary

In today's episode, we saw how to add oEmbed support to the Firestorm Forum using elixir-oembed . I hope you enjoyed it. See you soon!

Resources