HTTP Client

How do we know the server is running? Let's create a client with http4s to try our service.

A recap of the dependencies for this example, in case you skipped the service example. Ensure you have the following dependencies in your build.sbt:

scalaVersion := "2.13.4" // Also supports 2.11.x and 2.12.x

val http4sVersion = "0.22.15"

// Only necessary for SNAPSHOT releases
resolvers += Resolver.sonatypeRepo("snapshots")

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-dsl" % http4sVersion,
  "org.http4s" %% "http4s-blaze-server" % http4sVersion,
  "org.http4s" %% "http4s-blaze-client" % http4sVersion
)

Then we create the service again so mdoc picks it up: >

import cats.effect._
import org.http4s._
import org.http4s.dsl.io._
import org.http4s.implicits._
import org.http4s.blaze.server._

Blaze needs a [ConcurrentEffect] instance, which is derived from [ContextShift]. The following lines are not necessary if you are in an [IOApp]:

import scala.concurrent.ExecutionContext.global
implicit val cs: ContextShift[IO] = IO.contextShift(global)
implicit val timer: Timer[IO] = IO.timer(global)

Finish setting up our server:

import scala.concurrent.ExecutionContext.global

val app = HttpRoutes.of[IO] {
  case GET -> Root / "hello" / name =>
    Ok(s"Hello, $name.")
}.orNotFound
// app: cats.data.Kleisli[IO[A], Request[IO], Response[IO[A]]] = Kleisli(
//   run = org.http4s.syntax.KleisliResponseOps$$Lambda$21998/101692964@55a5b2d3
// )

val server = BlazeServerBuilder[IO](global)
  .bindHttp(8080, "localhost")
  .withHttpApp(app)
  .resource
// server: Resource[IO, org.http4s.server.Server] = Bind(
//   source = Allocate(
//     resource = Map(
//       source = Delay(
//         thunk = org.http4s.blazecore.package$$$Lambda$22002/1745261408@3e5f54d8,
//         trace = StackTrace(
//           stackTrace = List(
//             cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//             cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//             cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//             cats.effect.IO$.delay(IO.scala:1176),
//             cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:919),
//             cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:865),
//             org.http4s.blazecore.package$.tickWheelResource(package.scala:26),
//             org.http4s.blaze.server.BlazeServerBuilder.resource(BlazeServerBuilder.scala:363),
//             repl.MdocSession$App.<init>(client.md:51),
//             repl.MdocSession$.app(client.md:3),
//             mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//             scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//             scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//             scala.Console$.withErr(Console.scala:193),
//             mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//             scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//             scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//             scala.Console$.withOut(Console.scala:164),
//             mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.scala:88),
//             mdoc.internal.markdown.MarkdownBuilder$.$anonfun$buildDocument$2(MarkdownBuilder.scala:47),
//             mdoc.internal.markdown.MarkdownBuilder$$anon$1.run(MarkdownBuilder.scala:103)
//           )
//         )
//       ),
//       f = cats.effect.Resource$$$Lambda$22004/715616620@72f256f3,
//       trace = StackTrace(
//         stackTrace = List(
//           cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//           cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//           cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//           cats.effect.IO.map(IO.scala:106),
//           cats.effect.IOLowPriorityInstances$IOEffect.map(IO.scala:872),
//           cats.effect.IOLowPriorityInstances$IOEffect.map(IO.scala:865),
// ...

We'll start the server in the background. The IO.never keeps it running until we cancel the fiber.

val fiber = server.use(_ => IO.never).start.unsafeRunSync()
// fiber: Fiber[IO, Nothing] = Tuple(
//   join = Bind(
//     source = Async(
//       k = cats.effect.internals.IOAsync$$$Lambda$21728/1386801979@5a0ffcf8,
//       trampolineAfter = false,
//       trace = StackTrace(
//         stackTrace = List(
//           cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//           cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//           cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//           cats.effect.internals.IOAsync$.apply(IOAsync.scala:30),
//           cats.effect.IO$.async(IO.scala:1284),
//           cats.effect.internals.IOFromFuture$.apply(IOFromFuture.scala:37),
//           cats.effect.internals.IOStart$.fiber(IOStart.scala:48),
//           cats.effect.internals.IOStart$.$anonfun$apply$1(IOStart.scala:42),
//           cats.effect.internals.IOStart$.$anonfun$apply$1$adapted(IOStart.scala:29),
//           cats.effect.internals.IORunLoop$RestartCallback.start(IORunLoop.scala:464),
//           cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:165),
//           cats.effect.internals.IORunLoop$.$anonfun$suspendAsync$1(IORunLoop.scala:338),
//           cats.effect.internals.IORunLoop$.$anonfun$suspendAsync$1$adapted(IORunLoop.scala:337),
//           cats.effect.internals.IORunLoop$RestartCallback.start(IORunLoop.scala:464),
//           cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:165),
//           cats.effect.internals.IORunLoop$.start(IORunLoop.scala:38),
//           cats.effect.IO.unsafeRunAsync(IO.scala:274),
//           cats.effect.internals.IOPlatform$.unsafeResync(IOPlatform.scala:39),
//           cats.effect.IO.unsafeRunTimed(IO.scala:342),
//           cats.effect.IO.unsafeRunSync(IO.scala:256),
//           repl.MdocSession$App.<init>(client.md:59),
//           repl.MdocSession$.app(client.md:3),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withErr(Console.scala:193),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withOut(Console.scala:164),
//           mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.sca...

Creating the client

A good default choice is the BlazeClientBuilder. The BlazeClientBuilder maintains a connection pool and speaks HTTP 1.x.

import org.http4s.blaze.client._
import org.http4s.client._
import scala.concurrent.ExecutionContext.global

BlazeClientBuilder[IO](global).resource.use { client =>
  // use `client` here and return an `IO`.
  // the client will be acquired and shut down
  // automatically each time the `IO` is run.
  IO.unit
}

For the remainder of this tutorial, we'll use an alternate client backend built on the standard java.net library client. Unlike the blaze client, it does not need to be shut down. Like the blaze-client, and any other http4s backend, it presents the exact same Client interface!

It uses blocking IO and is less suited for production, but it is highly useful in a REPL:

import cats.effect.Blocker
import java.util.concurrent._

val blockingPool = Executors.newFixedThreadPool(5)
val blocker = Blocker.liftExecutorService(blockingPool)
val httpClient: Client[IO] = JavaNetClientBuilder[IO](blocker).create

Describing a call

To execute a GET request, we can call expect with the type we expect and the URI we want:

val helloJames = httpClient.expect[String]("http://localhost:8080/hello/James")
// helloJames: IO[String] = Bind(
//   source = Map(
//     source = Delay(
//       thunk = org.http4s.client.JavaNetClientBuilder$$Lambda$22179/1373300398@215fad7a,
//       trace = StackTrace(
//         stackTrace = List(
//           cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//           cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//           cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//           cats.effect.IO$.delay(IO.scala:1176),
//           cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:919),
//           cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:865),
//           org.http4s.client.JavaNetClientBuilder.$anonfun$create$1(JavaNetClientBuilder.scala:146),
//           org.http4s.client.Client$$anon$1.run(Client.scala:201),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:97),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:123),
//           org.http4s.client.DefaultClient.$anonfun$expectOr$6(DefaultClient.scala:135),
//           scala.util.Either.fold(Either.scala:189),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:135),
//           org.http4s.client.DefaultClient.expect(DefaultClient.scala:142),
//           repl.MdocSession$App.<init>(client.md:107),
//           repl.MdocSession$.app(client.md:3),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withErr(Console.scala:193),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withOut(Console.scala:164),
//           mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.scala:88),
//           mdoc.internal.markdown.MarkdownBuilder$.$anonfun$buildDocument$2(MarkdownBuilder.scala:47),
//           mdoc.internal.markdown.MarkdownBuilder$$anon$1.run(MarkdownBuilder.scala:103)
//         )
//       )
//     ),
//     f = cats.effect.Resource$$$Lambda$22042/1447373468@4126d95,
//     trace = StackTrace(
//       stackTrace = List(
// ...

Note that we don't have any output yet. We have a IO[String], to represent the asynchronous nature of a client request.

Furthermore, we haven't even executed the request yet. A significant difference between a IO and a scala.concurrent.Future is that a Future starts running immediately on its implicit execution context, whereas a IO runs when it's told. Executing a request is an example of a side effect. In functional programming, we prefer to build a description of the program we're going to run, and defer its side effects to the end.

Let's describe how we're going to greet a collection of people in parallel:

import cats._, cats.effect._, cats.implicits._
import org.http4s.Uri

def hello(name: String): IO[String] = {
  val target = uri"http://localhost:8080/hello/" / name
  httpClient.expect[String](target)
}

val people = Vector("Michael", "Jessica", "Ashley", "Christopher")
// people: Vector[String] = Vector(
//   "Michael",
//   "Jessica",
//   "Ashley",
//   "Christopher"
// )

val greetingList = people.parTraverse(hello)
// greetingList: IO[Vector[String]] = Map(
//   source = Map(
//     source = Async(k = <function3>, trampolineAfter = true, trace = null),
//     f = cats.data.Chain$$$Lambda$21824/387456616@57e074d7,
//     trace = StackTrace(
//       stackTrace = List(
//         cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//         cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//         cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//         cats.effect.IO.map(IO.scala:106),
//         cats.effect.IOParallelNewtype$$anon$2.map(IO.scala:845),
//         cats.data.Chain$.$anonfun$traverseViaChain$5(Chain.scala:798),
//         cats.Eval.$anonfun$map$1(Eval.scala:57),
//         cats.Eval$.loop$1(Eval.scala:338),
//         cats.Eval$.cats$Eval$$evaluate(Eval.scala:363),
//         cats.Eval$FlatMap.value(Eval.scala:284),
//         cats.data.Chain$.traverseViaChain(Chain.scala:817),
//         cats.instances.VectorInstances$$anon$1.traverse(vector.scala:100),
//         cats.instances.VectorInstances$$anon$1.traverse(vector.scala:15),
//         cats.Parallel$.parTraverse(Parallel.scala:215),
//         cats.syntax.ParallelTraversableOps1$.parTraverse$extension(parallel.scala:143),
//         repl.MdocSession$App.<init>(client.md:131),
//         repl.MdocSession$.app(client.md:3),
//         mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//         scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//         scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//         scala.Console$.withErr(Console.scala:193),
//         mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//         scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//         scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//         scala.Console$.withOut(Console.scala:164),
//         mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.scala:88),
//         mdoc.internal.markdown.MarkdownBuilder$.$anonfun$buildDocument$2(MarkdownBuilder.scala:47),
//         mdoc.internal.markdown.MarkdownBuilder$$anon$1.run(MarkdownBuilder.scala:103)
//       )
//     )
//   ),
//   f = cats.instances.VectorInstances$$anon$1$$Lambda$22185/259851629@1a28a6d6,
//   trace = StackTrace(
//     stackTrace = List(
//       cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//       cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//       cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
// ...

Observe how simply we could combine a single F[String] returned by hello into a scatter-gather to return a F[List[String]].

Making the call

It is best to run your F "at the end of the world." The "end of the world" varies by context:

• In a command line app, it's your main method.
• In an HttpApp[F], an F[Response[F]] is returned to be run by the server.
• Here in the REPL, the last line is the end of the world. Here we go:

val greetingsStringEffect = greetingList.map(_.mkString("\n"))
// greetingsStringEffect: IO[String] = Map(
//   source = Map(
//     source = Map(
//       source = Async(k = <function3>, trampolineAfter = true, trace = null),
//       f = cats.data.Chain$$$Lambda$21824/387456616@57e074d7,
//       trace = StackTrace(
//         stackTrace = List(
//           cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//           cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//           cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//           cats.effect.IO.map(IO.scala:106),
//           cats.effect.IOParallelNewtype$$anon$2.map(IO.scala:845),
//           cats.data.Chain$.$anonfun$traverseViaChain$5(Chain.scala:798),
//           cats.Eval.$anonfun$map$1(Eval.scala:57),
//           cats.Eval$.loop$1(Eval.scala:338),
//           cats.Eval$.cats$Eval$$evaluate(Eval.scala:363),
//           cats.Eval$FlatMap.value(Eval.scala:284),
//           cats.data.Chain$.traverseViaChain(Chain.scala:817),
//           cats.instances.VectorInstances$$anon$1.traverse(vector.scala:100),
//           cats.instances.VectorInstances$$anon$1.traverse(vector.scala:15),
//           cats.Parallel$.parTraverse(Parallel.scala:215),
//           cats.syntax.ParallelTraversableOps1$.parTraverse$extension(parallel.scala:143),
//           repl.MdocSession$App.<init>(client.md:131),
//           repl.MdocSession$.app(client.md:3),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withErr(Console.scala:193),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withOut(Console.scala:164),
//           mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.scala:88),
//           mdoc.internal.markdown.MarkdownBuilder$.$anonfun$buildDocument$2(MarkdownBuilder.scala:47),
//           mdoc.internal.markdown.MarkdownBuilder$$anon$1.run(MarkdownBuilder.scala:103)
//         )
//       )
//     ),
//     f = cats.instances.VectorInstances$$anon$1$$Lambda$22185/259851629@1a28a6d6,
//     trace = StackTrace(
//       stackTrace = List(
// ...
greetingsStringEffect.unsafeRunSync()
// res1: String = """Hello, Michael.
// Hello, Jessica.
// Hello, Ashley.
// Hello, Christopher."""

Constructing a URI

Before you can make a call, you'll need a Uri to represent the endpoint you want to access.

There are a number of ways to construct a Uri.

If you have a literal string, you can use uri"...":

uri"https://my-awesome-service.com/foo/bar?wow=yeah"
// res2: Uri = Uri(
//   scheme = Some(value = Scheme(https)),
//   authority = Some(
//     value = Authority(
//       userInfo = None,
//       host = RegName(host = my-awesome-service.com),
//       port = None
//     )
//   ),
//   path = /foo/bar,
//   query = wow=yeah,
//   fragment = None
// )

This only works with literal strings because it uses a macro to validate the URI format at compile-time.

Otherwise, you'll need to use Uri.fromString(...) and handle the case where validation fails:

val validUri = "https://my-awesome-service.com/foo/bar?wow=yeah"
// validUri: String = "https://my-awesome-service.com/foo/bar?wow=yeah"
val invalidUri = "yeah whatever"
// invalidUri: String = "yeah whatever"

val uri: Either[ParseFailure, Uri] = Uri.fromString(validUri)
// uri: Either[ParseFailure, Uri] = Right(
//   value = Uri(
//     scheme = Some(value = Scheme(https)),
//     authority = Some(
//       value = Authority(
//         userInfo = None,
//         host = RegName(host = my-awesome-service.com),
//         port = None
//       )
//     ),
//     path = /foo/bar,
//     query = wow=yeah,
//     fragment = None
//   )
// )

val parseFailure: Either[ParseFailure, Uri] = Uri.fromString(invalidUri)
// parseFailure: Either[ParseFailure, Uri] = Left(
//   value = ParseFailure(
//     sanitized = "Invalid URI",
//     details = "Error(4,NonEmptyList(EndOfString(4,13)))"
//   )
// )

You can also build up a URI incrementally, e.g.:

val baseUri: Uri = uri"http://foo.com"
// baseUri: Uri = Uri(
//   scheme = Some(value = Scheme(http)),
//   authority = Some(
//     value = Authority(
//       userInfo = None,
//       host = RegName(host = foo.com),
//       port = None
//     )
//   ),
//   path = ,
//   query = ,
//   fragment = None
// )
val withPath: Uri = baseUri.withPath(path"/bar/baz")
// withPath: Uri = Uri(
//   scheme = Some(value = Scheme(http)),
//   authority = Some(
//     value = Authority(
//       userInfo = None,
//       host = RegName(host = foo.com),
//       port = None
//     )
//   ),
//   path = /bar/baz,
//   query = ,
//   fragment = None
// )
val withQuery: Uri = withPath.withQueryParam("hello", "world")
// withQuery: Uri = Uri(
//   scheme = Some(value = Scheme(http)),
//   authority = Some(
//     value = Authority(
//       userInfo = None,
//       host = RegName(host = foo.com),
//       port = None
//     )
//   ),
//   path = /bar/baz,
//   query = hello=world,
//   fragment = None
// )

Middleware

Like the server middleware, the client middleware is a wrapper around a Client that provides a means of accessing or manipulating Requests and Responses being sent.

Consider functions from Int to String. We could create a wrapper over functions of this type, which would take anInt => Stringand return anInt => String`.

Such a wrapper could make the result inspect its input, do something to it, and call the original function with that input (or even another one). Then it could look at the response and also make some actions based on it.

An example wrapper could look something like this:

def mid(f: Int => String): Int => String = in => {
  // here, `in` is the input originally passed to the function
  // we can decide to pass it to `f`, or modify it first. We'll change it for the example.
  val resultOfF = f(in + 1)

  // Now, `resultOfF` is the result of the function applied with the modified result.
  // We can return it verbatim or _also_ modify it first! We could even ignore it.
  // Here, we'll use both results - the one we got from the original call (f(in)) and the customized one (f(in + 1)).
  s"${f(in)} is the original result, but $resultOfF's input was modified!"
}

If we were to wrap a simple function, say, one returning the String representation of a number:

val f1: Int => String = _.toString
// f1: Int => String = <function1>

// Here, we're applying our wrapper to `f1`. Notice that this is still a function.
val f2: Int => String = mid(f1)
// f2: Int => String = <function1>

f1(10)
// res3: String = "10"
f2(10)
// res4: String = "10 is the original result, but 11's input was modified!"

We would see how it's changing the result of the f1 function by giving it another input.

This wrapper could be considered a middleware over functions from Int to String. Now consider a simplified definition of Client[F] - it boils down to a single abstract method:

trait Client[F[_]] {
  def run(request: Request[F]): Resource[F, Response[F]]
}

Knowing this, we could say a Client[F] is equivalent to a function from Request[F] to Resource[F, Response[F]]. In fact, given a client, we could call client.run _ to get that function.

A client middleware follows the same idea as our original middleware did: it takes a Client (which is a function) and returns another Client (which is also a function).

It can see the input Request[F] that we pass to the client when we call it, it can modify that request, pass it to the underlying client (or any other client, really!), and do all sorts of other things, including effects - all it has to do is return a Resource[F, Response[F]].

The real definition of Client is a little more complicated because there's several more abstract methods. If you want to implement a client using just a function (for example, to make a middleware), consider using Client.apply.

A simple middleware, which would add a constant header to every request and response, could look like this:

import org.typelevel.ci.CIString

def addTestHeader[F[_]: Concurrent](underlying: Client[F]): Client[F] = Client[F] { req =>
  underlying
    .run(
      req.withHeaders(Header.Raw(CIString("X-Test-Request"), "test"))
    )
    .map(
      _.withHeaders(Header.Raw(CIString("X-Test-Response"), "test"))
    )
}

As the caller of the client you would get from this, you would see the extra header in the response. Similarly, every service called by the client would see an extra header in the requests.

Included Middleware

Http4s includes some middleware Out of the Box in the org.http4s.client.middleware package. These include:

• Following of redirect responses (Follow Redirect)
• Retrying of requests (Retry)
• Metrics gathering (Metrics)
• Logging of requests (Request Logger)
• Logging of responses (Response Logger)
• Logging of requests and responses (Logger)

Metrics Middleware

Apart from the middleware mentioned in the previous section. There is, as well, Out of the Box middleware for Dropwizard and Prometheus metrics

Dropwizard Metrics Middleware

To make use of this metrics middleware the following dependencies are needed:

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-client" % http4sVersion,
  "org.http4s" %% "http4s-dropwizard-metrics" % http4sVersion
)

We can create a middleware that registers metrics prefixed with a provided prefix like this.

import org.http4s.client.middleware.Metrics
import org.http4s.metrics.dropwizard.Dropwizard
import com.codahale.metrics.SharedMetricRegistries

implicit val clock = Clock.create[IO]
// clock: Clock[IO] = cats.effect.Clock$$anon$1@67f9e2e5
val registry = SharedMetricRegistries.getOrCreate("default")
// registry: com.codahale.metrics.MetricRegistry = com.codahale.metrics.MetricRegistry@2a8ab1ba
val requestMethodClassifier = (r: Request[IO]) => Some(r.method.toString.toLowerCase)
// requestMethodClassifier: Request[IO] => Some[String] = <function1>

val meteredClient =
  Metrics[IO](Dropwizard(registry, "prefix"), requestMethodClassifier)(httpClient)
// meteredClient: Client[IO] = org.http4s.client.Client$$anon$1@27b5dca5

A classifier is just a function Request[F] => Option[String] that allows to add a subprefix to every metric based on the Request

Prometheus Metrics Middleware

To make use of this metrics middleware the following dependencies are needed:

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-client" % http4sVersion,
  "org.http4s" %% "http4s-prometheus-metrics" % http4sVersion
)

We can create a middleware that registers metrics prefixed with a provided prefix like this.

import cats.effect.{Resource, IO}
import org.http4s.client.middleware.Metrics
import org.http4s.metrics.prometheus.Prometheus

implicit val clock = Clock.create[IO]
// clock: Clock[IO] = cats.effect.Clock$$anon$1@7dc0466a
val requestMethodClassifier = (r: Request[IO]) => Some(r.method.toString.toLowerCase)
// requestMethodClassifier: Request[IO] => Some[String] = <function1>

val meteredClient: Resource[IO, Client[IO]] =
  for {
    registry <- Prometheus.collectorRegistry[IO]
    metrics <- Prometheus.metricsOps[IO](registry, "prefix")
  } yield Metrics[IO](metrics, requestMethodClassifier)(httpClient)
// meteredClient: Resource[IO, Client[IO]] = Bind(
//   source = Allocate(
//     resource = Map(
//       source = Map(
//         source = Delay(
//           thunk = org.http4s.metrics.prometheus.Prometheus$$$Lambda$22661/1538836295@55866dfd,
//           trace = StackTrace(
//             stackTrace = List(
//               cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//               cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//               cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//               cats.effect.IO$.delay(IO.scala:1176),
//               cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:919),
//               cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:865),
//               org.http4s.metrics.prometheus.Prometheus$.collectorRegistry(Prometheus.scala:84),
//               repl.MdocSession$App.<init>(client.md:280),
//               repl.MdocSession$.app(client.md:3),
//               mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//               scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//               scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//               scala.Console$.withErr(Console.scala:193),
//               mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//               scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//               scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//               scala.Console$.withOut(Console.scala:164),
//               mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.scala:88),
//               mdoc.internal.markdown.MarkdownBuilder$.$anonfun$buildDocument$2(MarkdownBuilder.scala:47),
//               mdoc.internal.markdown.MarkdownBuilder$$anon$1.run(MarkdownBuilder.scala:103)
//             )
//           )
//         ),
//         f = cats.effect.Resource$$$Lambda$22040/906641353@2afe1196,
//         trace = StackTrace(
//           stackTrace = List(
//             cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//             cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//             cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//             cats.effect.IO.map(IO.scala:106),
// ...

A classifier is just a function Request[F] => Option[String] that allows to add a label to every metric based on the Request

Examples

Send a GET request, treating the response as a string

You can send a GET by calling the expect method on the client, passing a Uri:

httpClient.expect[String](uri"https://google.com/")
// res5: IO[String] = Bind(
//   source = Map(
//     source = Delay(
//       thunk = org.http4s.client.JavaNetClientBuilder$$Lambda$22179/1373300398@2327a098,
//       trace = StackTrace(
//         stackTrace = List(
//           cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//           cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//           cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//           cats.effect.IO$.delay(IO.scala:1176),
//           cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:919),
//           cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:865),
//           org.http4s.client.JavaNetClientBuilder.$anonfun$create$1(JavaNetClientBuilder.scala:146),
//           org.http4s.client.Client$$anon$1.run(Client.scala:201),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:97),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:123),
//           org.http4s.client.DefaultClient.$anonfun$expectOr$6(DefaultClient.scala:135),
//           scala.util.Either.fold(Either.scala:189),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:135),
//           org.http4s.client.DefaultClient.expect(DefaultClient.scala:142),
//           repl.MdocSession$App.<init>(client.md:107),
//           repl.MdocSession$.app(client.md:3),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withErr(Console.scala:193),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withOut(Console.scala:164),
//           mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.scala:88),
//           mdoc.internal.markdown.MarkdownBuilder$.$anonfun$buildDocument$2(MarkdownBuilder.scala:47),
//           mdoc.internal.markdown.MarkdownBuilder$$anon$1.run(MarkdownBuilder.scala:103)
//         )
//       )
//     ),
//     f = cats.effect.Resource$$$Lambda$22042/1447373468@5cab4180,
//     trace = StackTrace(
//       stackTrace = List(
// ...

If you need to do something more complicated like setting request headers, you can build up a request object and pass that to expect:

import org.http4s.client.dsl.io._
import org.http4s.headers._
import org.http4s.MediaType
import org.http4s.Method._

val request = GET(
  uri"https://my-lovely-api.com/",
  Authorization(Credentials.Token(AuthScheme.Bearer, "open sesame")),
  Accept(MediaType.application.json)
)
// request: Request[IO] = (
//    = GET,
//    = Uri(
//     scheme = Some(value = Scheme(https)),
//     authority = Some(
//       value = Authority(
//         userInfo = None,
//         host = RegName(host = my-lovely-api.com),
//         port = None
//       )
//     ),
//     path = /,
//     query = ,
//     fragment = None
//   ),
//    = HttpVersion(major = 1, minor = 1),
//    = Headers(Authorization: Bearer open sesame, Accept: application/json),
//    = Stream(..),
//    = org.typelevel.vault.Vault@781044f0
// )

httpClient.expect[String](request)
// res6: IO[String] = Bind(
//   source = Map(
//     source = Delay(
//       thunk = org.http4s.client.JavaNetClientBuilder$$Lambda$22179/1373300398@631c82d1,
//       trace = StackTrace(
//         stackTrace = List(
//           cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//           cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//           cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//           cats.effect.IO$.delay(IO.scala:1176),
//           cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:919),
//           cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:865),
//           org.http4s.client.JavaNetClientBuilder.$anonfun$create$1(JavaNetClientBuilder.scala:146),
//           org.http4s.client.Client$$anon$1.run(Client.scala:201),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:97),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:123),
//           org.http4s.client.DefaultClient.$anonfun$expectOr$6(DefaultClient.scala:135),
//           scala.util.Either.fold(Either.scala:189),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:135),
//           org.http4s.client.DefaultClient.expect(DefaultClient.scala:142),
//           repl.MdocSession$App.<init>(client.md:107),
//           repl.MdocSession$.app(client.md:3),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withErr(Console.scala:193),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withOut(Console.scala:164),
//           mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.scala:88),
//           mdoc.internal.markdown.MarkdownBuilder$.$anonfun$buildDocument$2(MarkdownBuilder.scala:47),
//           mdoc.internal.markdown.MarkdownBuilder$$anon$1.run(MarkdownBuilder.scala:103)
//         )
//       )
//     ),
//     f = cats.effect.Resource$$$Lambda$22042/1447373468@6407802e,
//     trace = StackTrace(
//       stackTrace = List(
// ...

Post a form, decoding the JSON response to a case class

import org.http4s.circe._
import io.circe.generic.auto._

case class AuthResponse(access_token: String)

implicit val authResponseEntityDecoder: EntityDecoder[IO, AuthResponse] =
  jsonOf[IO, AuthResponse]
// authResponseEntityDecoder: EntityDecoder[IO, AuthResponse] = org.http4s.EntityDecoder$$anon$2@6d67a61d

val postRequest = POST(
  UrlForm(
    "grant_type" -> "client_credentials",
    "client_id" -> "my-awesome-client",
    "client_secret" -> "s3cr3t"
  ),
  uri"https://my-lovely-api.com/oauth2/token"
)
// postRequest: Request[IO] = (
//    = POST,
//    = Uri(
//     scheme = Some(value = Scheme(https)),
//     authority = Some(
//       value = Authority(
//         userInfo = None,
//         host = RegName(host = my-lovely-api.com),
//         port = None
//       )
//     ),
//     path = /oauth2/token,
//     query = ,
//     fragment = None
//   ),
//    = HttpVersion(major = 1, minor = 1),
//    = Headers(Content-Type: application/x-www-form-urlencoded; charset=UTF-8, Content-Length: 78),
//    = Stream(..),
//    = org.typelevel.vault.Vault@6aeabb6a
// )

httpClient.expect[AuthResponse](postRequest)
// res7: IO[AuthResponse] = Bind(
//   source = Map(
//     source = Delay(
//       thunk = org.http4s.client.JavaNetClientBuilder$$Lambda$22179/1373300398@323440e9,
//       trace = StackTrace(
//         stackTrace = List(
//           cats.effect.internals.IOTracing$.buildFrame(IOTracing.scala:48),
//           cats.effect.internals.IOTracing$.buildCachedFrame(IOTracing.scala:39),
//           cats.effect.internals.IOTracing$.cached(IOTracing.scala:34),
//           cats.effect.IO$.delay(IO.scala:1176),
//           cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:919),
//           cats.effect.IOLowPriorityInstances$IOEffect.delay(IO.scala:865),
//           org.http4s.client.JavaNetClientBuilder.$anonfun$create$1(JavaNetClientBuilder.scala:146),
//           org.http4s.client.Client$$anon$1.run(Client.scala:201),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:97),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:123),
//           org.http4s.client.DefaultClient.$anonfun$expectOr$6(DefaultClient.scala:135),
//           scala.util.Either.fold(Either.scala:189),
//           org.http4s.client.DefaultClient.expectOr(DefaultClient.scala:135),
//           org.http4s.client.DefaultClient.expect(DefaultClient.scala:142),
//           repl.MdocSession$App.<init>(client.md:107),
//           repl.MdocSession$.app(client.md:3),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$2(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withErr(Console.scala:193),
//           mdoc.internal.document.DocumentBuilder$$doc$.$anonfun$build$1(DocumentBuilder.scala:89),
//           scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18),
//           scala.util.DynamicVariable.withValue(DynamicVariable.scala:59),
//           scala.Console$.withOut(Console.scala:164),
//           mdoc.internal.document.DocumentBuilder$$doc$.build(DocumentBuilder.scala:88),
//           mdoc.internal.markdown.MarkdownBuilder$.$anonfun$buildDocument$2(MarkdownBuilder.scala:47),
//           mdoc.internal.markdown.MarkdownBuilder$$anon$1.run(MarkdownBuilder.scala:103)
//         )
//       )
//     ),
//     f = cats.effect.Resource$$$Lambda$22042/1447373468@77e56544,
//     trace = StackTrace(
//       stackTrace = List(
// ...

Calls to a JSON API

Take a look at json.

Body decoding / encoding

The reusable way to decode/encode a request is to write a custom EntityDecoder and EntityEncoder. For that topic, take a look at entity.

If you prefer a more fine-grained approach, some of the methods take a Response[F] => F[A] argument, such as run or get, which lets you add a function which includes the decoding functionality, but ignores the media type.

client.run(req).use {
  case Status.Successful(r) => r.attemptAs[A].leftMap(_.message).value
  case r => r.as[String]
    .map(b => Left(s"Request $req failed with status ${r.status.code} and body $b"))
}

However, your function has to consume the body before the returned F exits. Don't do this:

// will come back to haunt you
client.get[EntityBody]("some-url")(response => response.body)

Passing it to a EntityDecoder is safe.

client.get[T]("some-url")(response => jsonOf(response.body))