HTTP Client

How do we know the server is running? Let’s create a client with http4s to try our service.

A recap of the dependencies for this example, in case you skipped the service example. Ensure you have the following dependencies in your build.sbt:

scalaVersion := "2.11.8" // Also supports 2.10.x and 2.12.x

val http4sVersion = "0.19.0"

// Only necessary for SNAPSHOT releases
resolvers += Resolver.sonatypeRepo("snapshots")

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-dsl" % http4sVersion,
  "org.http4s" %% "http4s-blaze-server" % http4sVersion,
  "org.http4s" %% "http4s-blaze-client" % http4sVersion
)

Then we create the service again so tut picks it up:

import cats.effect._
import org.http4s._
import org.http4s.dsl.io._
import org.http4s.server.blaze._

Blaze needs a [ConcurrentEffect] instance, which is derived from [ContextShift]. The following lines are not necessary if you are in an [IOApp]:

import scala.concurrent.ExecutionContext.global
implicit val cs: ContextShift[IO] = IO.contextShift(global)
implicit val timer: Timer[IO] = IO.timer(global)

Finish setting up our server:

val routes = HttpRoutes.of[IO] {
  case GET -> Root / "hello" / name =>
    Ok(s"Hello, $name.")
}
// routes: org.http4s.HttpRoutes[cats.effect.IO] = Kleisli(org.http4s.HttpRoutes$$$Lambda$49618/1801422598@55cb93cb)

val server = BlazeBuilder[IO].bindHttp(8080, "localhost").mountService(routes, "/").resource
// server: cats.effect.Resource[cats.effect.IO,org.http4s.server.Server[cats.effect.IO]] = Allocate(<function1>)

We’ll start the server in the background. The IO.never keeps it running until we cancel the fiber.

val fiber = server.use(_ => IO.never).start.unsafeRunSync()
// fiber: cats.effect.Fiber[cats.effect.IO,Nothing] = Tuple(IO$221518362,IO$447539972)

Creating the client

A good default choice is the BlazeClientBuilder. The BlazeClientBuilder maintains a connection pool and speaks HTTP 1.x.

import org.http4s.client.blaze._
import org.http4s.client._
import scala.concurrent.ExecutionContext.Implicits.global

BlazeClientBuilder[IO](global).resource.use { client =>
  // use `client` here and return an `IO`.
  // the client will be acquired and shut down
  // automatically each time the `IO` is run.
  IO.unit
}
// res0: cats.effect.IO[Unit] = IO$1243994591

For the remainder of this tut, we’ll use an alternate client backend built on the standard java.net library client. Unlike the blaze client, it does not need to be shut down. Like the blaze-client, and any other http4s backend, it presents the exact same Client interface!

It uses blocking IO and is less suited for production, but it is highly useful in a REPL:

import scala.concurrent.ExecutionContext
import java.util.concurrent._

val blockingEC = ExecutionContext.fromExecutorService(Executors.newFixedThreadPool(5))
val httpClient: Client[IO] = JavaNetClientBuilder(blockingEC).create

Describing a call

To execute a GET request, we can call expect with the type we expect and the URI we want:

val helloJames = httpClient.expect[String]("http://localhost:8080/hello/James")
// helloJames: cats.effect.IO[String] = IO$995670882

Note that we don’t have any output yet. We have a IO[String], to represent the asynchronous nature of a client request.

Furthermore, we haven’t even executed the request yet. A significant difference between a IO and a scala.concurrent.Future is that a Future starts running immediately on its implicit execution context, whereas a IO runs when it’s told. Executing a request is an example of a side effect. In functional programming, we prefer to build a description of the program we’re going to run, and defer its side effects to the end.

Let’s describe how we’re going to greet a collection of people in parallel:

import cats._, cats.effect._, cats.implicits._
import org.http4s.Uri
import scala.concurrent.ExecutionContext.Implicits.global

def hello(name: String): IO[String] = {
  val target = Uri.uri("http://localhost:8080/hello/") / name
  httpClient.expect[String](target)
}
// hello: (name: String)cats.effect.IO[String]

val people = Vector("Michael", "Jessica", "Ashley", "Christopher")
// people: scala.collection.immutable.Vector[String] = Vector(Michael, Jessica, Ashley, Christopher)

val greetingList = people.parTraverse(hello)
// greetingList: cats.effect.IO[scala.collection.immutable.Vector[String]] = IO$214288430

Observe how simply we could combine a single F[String] returned by hello into a scatter-gather to return a F[List[String]].

Making the call

It is best to run your F “at the end of the world.” The “end of the world” varies by context:

• In a command line app, it’s your main method.
• In an HttpApp[F], an F[Response[F]] is returned to be run by the server.
• Here in the REPL, the last line is the end of the world. Here we go:

val greetingsStringEffect = greetingList.map(_.mkString("\n"))
// greetingsStringEffect: cats.effect.IO[String] = <function1>

greetingsStringEffect.unsafeRunSync
// res1: String =
// Hello, Michael.
// Hello, Jessica.
// Hello, Ashley.
// Hello, Christopher.

Constructing a URI

Before you can make a call, you’ll need a Uri to represent the endpoint you want to access.

There are a number of ways to construct a Uri.

If you have a literal string, you can use Uri.uri(...):

Uri.uri("https://my-awesome-service.com/foo/bar?wow=yeah")
// res2: org.http4s.Uri = https://my-awesome-service.com/foo/bar?wow=yeah

This only works with literal strings because it uses a macro to validate the URI format at compile-time.

Otherwise, you’ll need to use Uri.fromString(...) and handle the case where validation fails:

val validUri = "https://my-awesome-service.com/foo/bar?wow=yeah"
// validUri: String = https://my-awesome-service.com/foo/bar?wow=yeah

val invalidUri = "yeah whatever"
// invalidUri: String = yeah whatever

val uri: Either[ParseFailure, Uri] = Uri.fromString(validUri)
// uri: Either[org.http4s.ParseFailure,org.http4s.Uri] = Right(https://my-awesome-service.com/foo/bar?wow=yeah)

val parseFailure: Either[ParseFailure, Uri] = Uri.fromString(invalidUri)
// parseFailure: Either[org.http4s.ParseFailure,org.http4s.Uri] =
// Left(org.http4s.ParseFailure: Invalid URI: Invalid input ' ', expected '+', '.', ':', SubDelims, '-', Unreserved, Digit, Alpha or PctEncoded (line 1, column 5):
// yeah whatever
//     ^)

You can also build up a URI incrementally, e.g.:

val baseUri = Uri.uri("http://foo.com")
// baseUri: org.http4s.Uri = http://foo.com

val withPath = baseUri.withPath("/bar/baz")
// withPath: org.http4s.Uri = http://foo.com/bar/baz

val withQuery = withPath.withQueryParam("hello", "world")
// withQuery: org.http4s.Uri = http://foo.com/bar/baz?hello=world

Middleware

Like the server middleware, the client middleware is a wrapper around a Client that provides a means of accessing or manipulating Requests and Responses being sent.

Included Middleware

Http4s includes some middleware Out of the Box in the org.http4s.client.middleware package. These include:

• Following of redirect responses (Follow Redirect)
• Retrying of requests (Retry)
• Metrics gathering (Metrics)
• Logging of requests (Request Logger)
• Logging of responses (Response Logger)
• Logging of requests and responses (Logger)

Metrics Middleware

Apart from the middleware mentioned in the previous section. There is, as well, Out of the Box middleware for Dropwizard and Prometheus metrics

Dropwizard Metrics Middleware

To make use of this metrics middleware the following dependencies are needed:

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-client" % http4sVersion,
  "org.http4s" %% "http4s-dropwizard-metrics" % http4sVersion
)

We can create a middleware that registers metrics prefixed with a provided prefix like this.

import org.http4s.client.middleware.Metrics
import org.http4s.metrics.dropwizard.Dropwizard
import com.codahale.metrics.SharedMetricRegistries

implicit val clock = Clock.create[IO]
// clock: cats.effect.Clock[cats.effect.IO] = cats.effect.Clock$$anon$1@24de5855

val registry = SharedMetricRegistries.getOrCreate("default")
// registry: com.codahale.metrics.MetricRegistry = com.codahale.metrics.MetricRegistry@55baa55f

val requestMethodClassifier = (r: Request[IO]) => Some(r.method.toString.toLowerCase)
// requestMethodClassifier: org.http4s.Request[cats.effect.IO] => Some[String] = $$Lambda$50178/1853173888@1ce1fb1

val meteredClient = Metrics[IO](Dropwizard(registry, "prefix"), requestMethodClassifier)(httpClient)
// meteredClient: org.http4s.client.Client[cats.effect.IO] = org.http4s.client.Client$$anon$1@b423f80

A classifier is just a function Request[F] => Option[String] that allows to add a subprefix to every metric based on the Request

Prometheus Metrics Middleware

To make use of this metrics middleware the following dependencies are needed:

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-client" % http4sVersion,
  "org.http4s" %% "http4s-prometheus-metrics" % http4sVersion
)

We can create a middleware that registers metrics prefixed with a provided prefix like this.

import org.http4s.client.middleware.Metrics
import org.http4s.metrics.prometheus.Prometheus
import io.prometheus.client.CollectorRegistry

implicit val clock = Clock.create[IO]
// clock: cats.effect.Clock[cats.effect.IO] = cats.effect.Clock$$anon$1@5392ea8f

val registry = new CollectorRegistry()
// registry: io.prometheus.client.CollectorRegistry = io.prometheus.client.CollectorRegistry@29fa0347

val requestMethodClassifier = (r: Request[IO]) => Some(r.method.toString.toLowerCase)
// requestMethodClassifier: org.http4s.Request[cats.effect.IO] => Some[String] = $$Lambda$50181/760838283@3d14c69f

val meteredClient = Metrics[IO](Prometheus(registry, "prefix"), requestMethodClassifier)(httpClient)
// meteredClient: org.http4s.client.Client[cats.effect.IO] = org.http4s.client.Client$$anon$1@29a587a4

A classifier is just a function Request[F] => Option[String] that allows to add a label to every metric based on the Request

Examples

Send a GET request, treating the response as a string

You can send a GET by calling the expect method on the client, passing a Uri:

httpClient.expect[String](Uri.uri("https://google.com/"))
// res3: cats.effect.IO[String] = IO$1298298998

If you need to do something more complicated like setting request headers, you can build up a request object and pass that to expect:

import org.http4s.client.dsl.io._
import org.http4s.headers._
import org.http4s.MediaType

val request = GET(
  Uri.uri("https://my-lovely-api.com/"),
  Authorization(Credentials.Token(AuthScheme.Bearer, "open sesame")),
  Accept(MediaType.application.json)
)
// request: cats.effect.IO[org.http4s.Request[cats.effect.IO]] = IO(Request(method=GET, uri=https://my-lovely-api.com/, headers=Headers(Authorization: <REDACTED>, Accept: application/json)))

httpClient.expect[String](request)
// res4: cats.effect.IO[String] = IO$543865265

Post a form, decoding the JSON response to a case class

case class AuthResponse(access_token: String)
// defined class AuthResponse

// See the JSON page for details on how to define this
implicit val authResponseEntityDecoder: EntityDecoder[IO, AuthResponse] = null
// authResponseEntityDecoder: org.http4s.EntityDecoder[cats.effect.IO,AuthResponse] = null

val postRequest = POST(
  Uri.uri("https://my-lovely-api.com/oauth2/token"),
  UrlForm(
    "grant_type" -> "client_credentials",
    "client_id" -> "my-awesome-client",
    "client_secret" -> "s3cr3t"
  )
)
// postRequest: cats.effect.IO[org.http4s.Request[cats.effect.IO]] = IO(Request(method=POST, uri=https://my-lovely-api.com/oauth2/token, headers=Headers(Content-Type: application/x-www-form-urlencoded; charset=UTF-8, Content-Length: 78)))

httpClient.expect[AuthResponse](postRequest)
// res6: cats.effect.IO[AuthResponse] = IO$176576098

Calls to a JSON API

Take a look at json.

Body decoding / encoding

The reusable way to decode/encode a request is to write a custom EntityDecoder and EntityEncoder. For that topic, take a look at entity.

If you prefer a more fine-grained approach, some of the methods take a Response[F] => F[A] argument, such as fetch or get, which lets you add a function which includes the decoding functionality, but ignores the media type.

client.fetch(req) {
  case Status.Successful(r) => r.attemptAs[A].leftMap(_.message).value
  case r => r.as[String]
    .map(b => Left(s"Request $req failed with status ${r.status.code} and body $b"))
}

However, your function has to consume the body before the returned F exits. Don’t do this:

// will come back to haunt you
client.get[EntityBody]("some-url")(response => response.body)

Passing it to a EntityDecoder is safe.

client.get[T]("some-url")(response => jsonOf(response.body))