HTTP Client

How do we know the server is running? Let’s create a client with http4s to try our service.

A recap of the dependencies for this example, in case you skipped the service example. Ensure you have the following dependencies in your build.sbt:

scalaVersion := "2.11.8" // Also supports 2.10.x and 2.12.x

val http4sVersion = "0.18.11"

// Only necessary for SNAPSHOT releases
resolvers += Resolver.sonatypeRepo("snapshots")

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-dsl" % http4sVersion,
  "org.http4s" %% "http4s-blaze-server" % http4sVersion,
  "org.http4s" %% "http4s-blaze-client" % http4sVersion
)

Then we create the service again so tut picks it up:

import cats.effect._
// import cats.effect._

import org.http4s._
// import org.http4s._

import org.http4s.dsl.io._
// import org.http4s.dsl.io._

import org.http4s.server.blaze._
// import org.http4s.server.blaze._

val service = HttpService[IO] {
  case GET -> Root / "hello" / name =>
    Ok(s"Hello, $name.")
}
// service: org.http4s.HttpService[cats.effect.IO] = Kleisli(org.http4s.HttpService$$$Lambda$41763/1389501468@3ca55513)

val builder = BlazeBuilder[IO].bindHttp(8080, "localhost").mountService(service, "/").start
// builder: cats.effect.IO[org.http4s.server.Server[cats.effect.IO]] = IO$1916273097

val server = builder.unsafeRunSync
// server: org.http4s.server.Server[cats.effect.IO] = BlazeServer(/127.0.0.1:8080)

Creating the client

A good default choice is the Http1Client. The Http1Client maintains a connection pool and speaks HTTP 1.x.

Note: In production code you would want to use Http1Client.stream[F[_]: Effect]: Stream[F, Http1Client] to safely acquire and release resources. In the documentation we are forced to use .unsafeRunSync to create the client.

import org.http4s.client.blaze._
// import org.http4s.client.blaze._

val httpClient = Http1Client[IO]().unsafeRunSync
// httpClient: org.http4s.client.Client[cats.effect.IO] = Client(Kleisli(org.http4s.client.blaze.BlazeClient$$$Lambda$41792/1501125643@6eb36339),IO$1224263808)

Describing a call

To execute a GET request, we can call expect with the type we expect and the URI we want:

val helloJames = httpClient.expect[String]("http://localhost:8080/hello/James")
// helloJames: cats.effect.IO[String] = IO$1582682898

Note that we don’t have any output yet. We have a IO[String], to represent the asynchronous nature of a client request.

Furthermore, we haven’t even executed the request yet. A significant difference between a IO and a scala.concurrent.Future is that a Future starts running immediately on its implicit execution context, whereas a IO runs when it’s told. Executing a request is an example of a side effect. In functional programming, we prefer to build a description of the program we’re going to run, and defer its side effects to the end.

Let’s describe how we’re going to greet a collection of people in parallel:

import cats._, cats.effect._, cats.implicits._
// import cats._
// import cats.effect._
// import cats.implicits._

import org.http4s.Uri
// import org.http4s.Uri

import scala.concurrent.ExecutionContext.Implicits.global
// import scala.concurrent.ExecutionContext.Implicits.global

def hello(name: String): IO[String] = {
  val target = Uri.uri("http://localhost:8080/hello/") / name
  httpClient.expect[String](target)
}
// hello: (name: String)cats.effect.IO[String]

val people = Vector("Michael", "Jessica", "Ashley", "Christopher")
// people: scala.collection.immutable.Vector[String] = Vector(Michael, Jessica, Ashley, Christopher)

val greetingList = fs2.async.parallelTraverse(people)(hello)
// greetingList: cats.effect.IO[scala.collection.immutable.Vector[String]] = IO$812487393

Observe how simply we could combine a single F[String] returned by hello into a scatter-gather to return a F[List[String]].

Making the call

It is best to run your F “at the end of the world.” The “end of the world” varies by context:

  • In a command line app, it’s your main method.
  • In an HttpService[F], an F[Response[F]] is returned to be run by the server.
  • Here in the REPL, the last line is the end of the world. Here we go:
val greetingsStringEffect = greetingList.map(_.mkString("\n"))
// greetingsStringEffect: cats.effect.IO[String] = <function1>

greetingsStringEffect.unsafeRunSync
// res0: String =
// Hello, Michael.
// Hello, Jessica.
// Hello, Ashley.
// Hello, Christopher.

Constructing a URI

Before you can make a call, you’ll need a Uri to represent the endpoint you want to access.

There are a number of ways to construct a Uri.

If you have a literal string, you can use Uri.uri(...):

Uri.uri("https://my-awesome-service.com/foo/bar?wow=yeah")
// res1: org.http4s.Uri = https://my-awesome-service.com/foo/bar?wow=yeah

This only works with literal strings because it uses a macro to validate the URI format at compile-time.

Otherwise, you’ll need to use Uri.fromString(...) and handle the case where validation fails:

val validUri = "https://my-awesome-service.com/foo/bar?wow=yeah"
// validUri: String = https://my-awesome-service.com/foo/bar?wow=yeah

val invalidUri = "yeah whatever"
// invalidUri: String = yeah whatever

val uri: Either[ParseFailure, Uri] = Uri.fromString(validUri)
// uri: Either[org.http4s.ParseFailure,org.http4s.Uri] = Right(https://my-awesome-service.com/foo/bar?wow=yeah)

val parseFailure: Either[ParseFailure, Uri] = Uri.fromString(invalidUri)
// parseFailure: Either[org.http4s.ParseFailure,org.http4s.Uri] =
// Left(org.http4s.ParseFailure: Invalid URI: Invalid input ' ', expected '+', '.', ':', SubDelims, '-', Unreserved, Digit, Alpha or PctEncoded (line 1, column 5):
// yeah whatever
//     ^)

You can also build up a URI incrementally, e.g.:

val baseUri = Uri.uri("http://foo.com")
// baseUri: org.http4s.Uri = http://foo.com

val withPath = baseUri.withPath("/bar/baz")
// withPath: org.http4s.Uri = http://foo.com/bar/baz

val withQuery = withPath.withQueryParam("hello", "world")
// withQuery: org.http4s.Uri = http://foo.com/bar/baz?hello=world

Examples

Send a GET request, treating the response as a string

You can send a GET by calling the expect method on the client, passing a Uri:

httpClient.expect[String](Uri.uri("https://google.com/"))
// res2: cats.effect.IO[String] = IO$2086667411

If you need to do something more complicated like setting request headers, you can build up a request object and pass that to expect:

import org.http4s.client.dsl.io._
// import org.http4s.client.dsl.io._

import org.http4s.headers._
// import org.http4s.headers._

import org.http4s.MediaType._
// import org.http4s.MediaType._

val request = GET(
  Uri.uri("https://my-lovely-api.com/"),
  Authorization(Credentials.Token(AuthScheme.Bearer, "open sesame")),
  Accept(`application/json`)
)
// request: cats.effect.IO[org.http4s.Request[cats.effect.IO]] = IO(Request(method=GET, uri=https://my-lovely-api.com/, headers=Headers(Authorization: <REDACTED>, Accept: application/json)))

httpClient.expect[String](request)
// res3: cats.effect.IO[String] = IO$1973008038

Post a form, decoding the JSON response to a case class

case class AuthResponse(access_token: String)
// defined class AuthResponse

// See the JSON page for details on how to define this
implicit val authResponseEntityDecoder: EntityDecoder[IO, AuthResponse] = null
// authResponseEntityDecoder: org.http4s.EntityDecoder[cats.effect.IO,AuthResponse] = null

val postRequest = POST(
  Uri.uri("https://my-lovely-api.com/oauth2/token"),
  UrlForm(
    "grant_type" -> "client_credentials",
    "client_id" -> "my-awesome-client",
    "client_secret" -> "s3cr3t"
  )
)
// postRequest: cats.effect.IO[org.http4s.Request[cats.effect.IO]] = IO$1130835185

httpClient.expect[AuthResponse](postRequest)
// res5: cats.effect.IO[AuthResponse] = IO$1174295271

Cleaning up

Our client consumes system resources. Let’s clean up after ourselves by shutting it down:

httpClient.shutdownNow()

If the client is created using HttpClient.stream[F](), it will be shut down when the resulting stream finishes.

Calls to a JSON API

Take a look at json.

Body decoding / encoding

The reusable way to decode/encode a request is to write a custom EntityDecoder and EntityEncoder. For that topic, take a look at entity.

If you prefer a more fine-grained approach, some of the methods take a Response[F] => F[A] argument, such as fetch or get, which lets you add a function which includes the decoding functionality, but ignores the media type.

client.fetch(req) {
  case Status.Successful(r) => r.attemptAs[A].leftMap(_.message).value
  case r => r.as[String]
    .map(b => Left(s"Request $req failed with status ${r.status.code} and body $b"))
}

However, your function has to consume the body before the returned F exits. Don’t do this:

// will come back to haunt you
client.get[EntityBody]("some-url")(response => response.body)

Passing it to a EntityDecoder is safe.

client.get[T]("some-url")(response => jsonOf(response.body))