The http4s DSL

Recall from earlier that an HttpService[F] is just a type alias for Kleisli[F, Request[F], Response[F]]. This provides a minimal foundation for declaring services and executing them on blaze or a servlet container. While this foundation is composeable, it is not highly productive. Most service authors will seek a higher level DSL.

Add the http4s-dsl to your build

One option is the http4s-dsl. It is officially supported by the http4s team, but kept separate from core in order to encourage multiple approaches for different needs.

This tutorial assumes that http4s-dsl is on your classpath. Add the following to your build.sbt:

libraryDependencies ++= Seq(
  "org.http4s" %% "http4s-dsl" % http4sVersion,
)

All we need is a REPL to follow along at home:

$ sbt console

The simplest service

We’ll need the following imports to get started:

import cats.effect._
import org.http4s._, org.http4s.dsl.io._

The central concept of http4s-dsl is pattern matching. An HttpService[F] is declared as a simple series of case statements. Each case statement attempts to match and optionally extract from an incoming Request[F]. The code associated with the first matching case is used to generate a F[Response[F]].

The simplest case statement matches all requests without extracting anything. The right hand side of the request must return a F[Response[F]].

In the following we use cats.effect.IO as the effect type F.

val service = HttpService[IO] {
  case _ =>
    IO(Response(Status.Ok))
}
// service: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@3ce2452b
// )

Testing the Service

One beautiful thing about the HttpService[F] model is that we don’t need a server to test our route. We can construct our own request and experiment directly in the REPL.

val getRoot = Request[IO](Method.GET, uri("/"))
// getRoot: Request[IO] = Request(
//   Method("GET"),
//   Uri(None, None, "/", Query(), None),
//   HttpVersion(1, 1),
//   Headers(),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

val io = service.orNotFound.run(getRoot)
// io: IO[Response[IO]] = Map(
//   Delay(<function0>),
//   scala.Function1$$Lambda$12648/922425644@7dc333d,
//   1
// )

Where is our Response[F]? It hasn’t been created yet. We wrapped it in an IO. In a real service, generating a Response[F] is likely to be an asynchronous operation with side effects, such as invoking another web service or querying a database, or maybe both. Operating in a F gives us control over the sequencing of operations and lets us reason about our code like good functional programmers. It is the HttpService[F]’s job to describe the task, and the server’s job to run it.

But here in the REPL, it’s up to us to run it:

val response = io.unsafeRunSync
// response: Response[IO] = Response(
//   Status(200),
//   HttpVersion(1, 1),
//   Headers(),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

Cool.

Generating responses

We’ll circle back to more sophisticated pattern matching of requests, but it will be a tedious affair until we learn a more succinct way of generating F[Response]s.

Status codes

http4s-dsl provides a shortcut to create an F[Response] by applying a status code:

val okIo = Ok()
// okIo: IO[Response[IO]] = Pure(
//   Response(
//     Status(200),
//     HttpVersion(1, 1),
//     Headers(Content-Length(0L)),
//     Stream(..),
//     org.http4s.AttributeMap@8cf85da
//   )
// )
val ok = okIo.unsafeRunSync
// ok: Response[IO] = Response(
//   Status(200),
//   HttpVersion(1, 1),
//   Headers(Content-Length(0L)),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

This simple Ok() expression succinctly says what we mean in a service:

HttpService[IO] {
  case _ => Ok()
}.orNotFound.run(getRoot).unsafeRunSync
// res0: Response[IO] = Response(
//   Status(200),
//   HttpVersion(1, 1),
//   Headers(Content-Length(0L)),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

This syntax works for other status codes as well. In our example, we don’t return a body, so a 204 No Content would be a more appropriate response:

import cats.implicits._

HttpService[IO] {
  case _ => NoContent()
}.orNotFound.run(getRoot).unsafeRunSync
// res1: Response[IO] = Response(
//   Status(204),
//   HttpVersion(1, 1),
//   Headers(),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

Headers

http4s adds a minimum set of headers depending on the response, e.g:

Ok("Ok response.").unsafeRunSync.headers
// res2: Headers = Headers(
//   Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//   Content-Length(12L)
// )

Extra headers can be added using putHeaders, for example to specify cache policies:

import org.http4s.headers.`Cache-Control`
import org.http4s.CacheDirective.`no-cache`
import cats.data.NonEmptyList

Ok("Ok response.", `Cache-Control`(NonEmptyList(`no-cache`(), Nil))).unsafeRunSync.headers
// res3: Headers = Headers(
//   Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//   Cache-Control(NonEmptyList(no-cache(List()), List())),
//   Content-Length(12L)
// )

http4s defines all the well known headers directly, but sometimes you need to define custom headers, typically prefixed by an X-. In simple cases you can construct a Header instance by hand

Ok("Ok response.", Header("X-Auth-Token", "value")).unsafeRunSync.headers
// res4: Headers = Headers(
//   Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//   Raw(X-Auth-Token, "value"),
//   Content-Length(12L)
// )

Cookies

http4s has special support for Cookie headers using the Cookie type to add and invalidate cookies. Adding a cookie will generate the correct Set-Cookie header:

Ok("Ok response.").map(_.addCookie(Cookie("foo", "bar"))).unsafeRunSync.headers
// res5: Headers = Headers(
//   Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//   Content-Length(12L),
//   Set-Cookie(Cookie("foo", "bar", None, None, None, None, false, false, None))
// )

Cookie can be further customized to set, e.g., expiration, the secure flag, httpOnly, flag, etc

Ok("Ok response.").map(_.addCookie(Cookie("foo", "bar", expires = Some(HttpDate.now), httpOnly = true, secure = true))).unsafeRunSync.headers
// res6: Headers = Headers(
//   Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//   Content-Length(12L),
//   Set-Cookie(
//     Cookie(
//       "foo",
//       "bar",
//       Some(Tue, 04 Jan 2022 21:16:30 GMT),
//       None,
//       None,
//       None,
//       true,
//       true,
//       None
//     )
//   )
// )

To request a cookie to be removed on the client, you need to set the cookie value to empty. http4s can do that with removeCookie

Ok("Ok response.").map(_.removeCookie("foo")).unsafeRunSync.headers
// res7: Headers = Headers(
//   Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//   Content-Length(12L),
//   Set-Cookie(
//     Cookie(
//       "foo",
//       "",
//       Some(Thu, 01 Jan 1970 00:00:00 GMT),
//       Some(0L),
//       None,
//       None,
//       false,
//       false,
//       None
//     )
//   )
// )

Responding with a body

Simple bodies

Most status codes take an argument as a body. In http4s, Request[F] and Response[F] bodies are represented as a fs2.Stream[F, ByteVector]. It’s also considered good HTTP manners to provide a Content-Type and, where known in advance, Content-Length header in one’s responses.

All of this hassle is neatly handled by http4s’ EntityEncoders. We’ll cover these in more depth in another tut. The important point for now is that a response body can be generated for any type with an implicit EntityEncoder in scope. http4s provides several out of the box:

Ok("Received request.").unsafeRunSync
// res8: Response[IO] = Response(
//   Status(200),
//   HttpVersion(1, 1),
//   Headers(
//     Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//     Content-Length(17L)
//   ),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

import java.nio.charset.StandardCharsets.UTF_8
Ok("binary".getBytes(UTF_8)).unsafeRunSync
// res9: Response[IO] = Response(
//   Status(200),
//   HttpVersion(1, 1),
//   Headers(
//     Content-Type(MediaType(application/octet-stream), None),
//     Content-Length(6L)
//   ),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

Per the HTTP specification, some status codes don’t support a body. http4s prevents such nonsense at compile time:

NoContent("does not compile")
// error: type mismatch;
//  found   : String("does not compile")
//  required: org.http4s.Header
// NoContent("does not compile")
//           ^^^^^^^^^^^^^^^^^^

Asynchronous responses

While http4s prefers F[_]: Effect, you may be working with libraries that use standard library [Future]s. Some relevant imports:

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

You can respond with a Future of any type that has an EntityEncoder by lifting it into IO or any F[_] that suspends future. Note: unlike IO, wrapping a side effect in Future does not suspend it, and the resulting expression would still be side effectful, unless we wrap it in IO.

val ioResp1 = Ok(IO.fromFuture(IO(Future {
  println("I run when the future is constructed.")
  "Greetings from the future!"
})))
// ioResp1: IO[Response[IO]] = Bind(
//   Bind(
//     Bind(Delay(<function0>), cats.effect.IO$$$Lambda$13604/1369914522@66e6927b),
//     org.http4s.EntityEncoderInstances$$anon$8$$Lambda$13605/471163159@7f10b377
//   ),
//   org.http4s.dsl.impl.EntityResponseGenerator$$Lambda$12628/727887815@6176a697
// )
ioResp1.unsafeRunSync
// I run when the future is constructed.
// res11: Response[IO] = Response(
//   Status(200),
//   HttpVersion(1, 1),
//   Headers(
//     Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//     Content-Length(26L)
//   ),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

As good functional programmers who like to delay our side effects, we of course prefer to operate in [F]s:

val ioResp2 = Ok(IO {
  println("I run when the IO is run.")
  "Mission accomplished!"
})
// ioResp2: IO[Response[IO]] = Bind(
//   Bind(
//     Delay(<function0>),
//     org.http4s.EntityEncoderInstances$$anon$8$$Lambda$13605/471163159@7fab4c89
//   ),
//   org.http4s.dsl.impl.EntityResponseGenerator$$Lambda$12628/727887815@2d8c7c4d
// )
ioResp2.unsafeRunSync
// I run when the IO is run.
// res12: Response[IO] = Response(
//   Status(200),
//   HttpVersion(1, 1),
//   Headers(
//     Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//     Content-Length(21L)
//   ),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

Note that in both cases, a Content-Length header is calculated. http4s waits for the Future or F to complete before wrapping it in its HTTP envelope, and thus has what it needs to calculate a Content-Length.

Streaming bodies

Streaming bodies are supported by returning a fs2.Stream. Like Futures and IOs, the stream may be of any type that has an EntityEncoder.

An intro to Stream is out of scope, but we can glimpse the power here. This stream emits the elapsed time every 100 milliseconds for one second:

val drip: fs2.Stream[IO, String] =
  fs2.Scheduler[IO](2).flatMap { s =>
    import scala.concurrent.duration._
    s.awakeEvery[IO](100.millis).map(_.toString).take(10)
  }
// drip: fs2.Stream[IO, String] = Stream(..)

We can see it for ourselves in the REPL:

val dripOutIO = drip.through(fs2.text.lines).through(_.evalMap(s => {IO{println(s); s}})).compile.drain
// dripOutIO: IO[Unit] = Bind(
//   Delay(fs2.internal.Algebra$$$Lambda$12658/991270135@577df980),
//   fs2.internal.Algebra$$$Lambda$12659/2026159217@36bafe6
// )
dripOutIO.unsafeRunSync
// 114167858 nanoseconds208974353 nanoseconds308128670 nanoseconds407810159 nanoseconds509068989 nanoseconds609626119 nanoseconds709757445 nanoseconds809470413 nanoseconds910159931 nanoseconds1005840032 nanoseconds

When wrapped in a Response[F], http4s will flush each chunk of a Stream as they are emitted. Note that a stream’s length can’t generally be anticipated before it runs, so this triggers chunked transfer encoding:

Ok(drip).unsafeRunSync
// res14: Response[IO] = Response(
//   Status(200),
//   HttpVersion(1, 1),
//   Headers(
//     Content-Type(MediaType(text/plain), Some(Charset(UTF-8))),
//     Transfer-Encoding(NonEmptyList(TransferCoding(chunked), List()))
//   ),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

Matching and extracting requests

A Request is a regular case class - you can destructure it to extract its values. By extension, you can also match/case it with different possible destructurings. To build these different extractors, you can make use of the DSL.

The -> object

More often, you extract the Request into a HTTP Method and path info via the -> object. On the left side is the method, and on the right side, the path info. The following matches a request to GET /hello:

HttpService[IO] {
  case GET -> Root / "hello" => Ok("hello")
}
// res15: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@26816406
// )

Path info

Path matching is done on the request’s pathInfo. Path info is the request’s URI’s path after the following:

• the mount point of the service
• the prefix, if the service is composed with a Router
• the prefix, if the service is rewritten with TranslateUri

Matching on request.pathInfo instead of request.uri.path allows multiple services to be composed without rewriting all the path matchers.

Matching paths

A request to the root of the service is matched with the Root extractor. Root consumes the leading slash of the path info. The following matches requests to GET /:

HttpService[IO] {
  case GET -> Root => Ok("root")
}
// res16: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@68a11a8f
// )

We usually match paths in a left-associative manner with Root and /. Each "/" after the initial slash delimits a path segment, and is represented in the DSL with the ‘/’ extractor. Segments can be matched as literals or made available through standard Scala pattern matching. For example, the following service responds with “Hello, Alice!” to GET /hello/Alice:

HttpService[IO] {
  case GET -> Root / "hello" / name => Ok(s"Hello, $name!")
}
// res17: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@50a85fe2
// )

The above assumes only one path segment after "hello", and would not match GET /hello/Alice/Bob. To match to an arbitrary depth, we need a right-associative /: extractor. In this case, there is no Root, and the final pattern is a Path of the remaining segments. This would say "Hello, Alice and Bob!"

HttpService[IO] {
  case GET -> "hello" /: rest => Ok(s"""Hello, ${rest.toList.mkString(" and ")}!""")
}
// res18: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@ee2ce8e
// )

To match a file extension on a segment, use the ~ extractor:

HttpService[IO] {
  case GET -> Root / file ~ "json" => Ok(s"""{"response": "You asked for $file"}""")
}
// res19: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@236e2a00
// )

Handling path parameters

Path params can be extracted and converted to a specific type but are Strings by default. There are numeric extractors provided in the form of IntVar and LongVar.

def getUserName(userId: Int): IO[String] = ???

val usersService = HttpService[IO] {
  case GET -> Root / "users" / IntVar(userId) =>
    Ok(getUserName(userId))
}
// usersService: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@437b2980
// )

If you want to extract a variable of type T, you can provide a custom extractor object which implements def unapply(str: String): Option[T], similar to the way in which IntVar does it.

import cats.implicits._
import java.time.LocalDate
import scala.util.Try

object LocalDateVar {
  def unapply(str: String): Option[LocalDate] = {
    if (!str.isEmpty)
      Try(LocalDate.parse(str)).toOption
    else
      None
  }
}

def getTemperatureForecast(date: LocalDate): IO[Double] = IO(42.23)

val dailyWeatherService = HttpService[IO] {
  case GET -> Root / "weather" / "temperature" / LocalDateVar(localDate) =>
    Ok(getTemperatureForecast(localDate).map(s"The temperature on $localDate will be: " + _))
}
// dailyWeatherService: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@51be2942
// )

val requestWeather = Request[IO](method = GET, uri = Uri.uri("/weather/temperature/2016-11-05"))
// requestWeather: Request[IO] = Request(
//   Method("GET"),
//   Uri(None, None, "/weather/temperature/2016-11-05", Query(), None),
//   HttpVersion(1, 1),
//   Headers(),
//   Stream(..),
//   org.http4s.AttributeMap@8cf85da
// )

println(dailyWeatherService.orNotFound.run(requestWeather).unsafeRunSync)
// Response(status=200, headers=Headers(Content-Type: text/plain; charset=UTF-8, Content-Length: 44))

Handling query parameters

A query parameter needs to have a QueryParamDecoderMatcher provided to extract it. In order for the QueryParamDecoderMatcher to work there needs to be an implicit QueryParamDecoder[T] in scope. QueryParamDecoders for simple types can be found in the QueryParamDecoder object. There are also QueryParamDecoderMatchers available which can be used to return optional or validated parameter values.

In the example below we’re finding query params named country and year and then parsing them as a String and java.time.Year.

import cats.effect._
import org.http4s._, org.http4s.dsl.io._
import java.time.Year
import cats.data.ValidatedNel

object CountryQueryParamMatcher extends QueryParamDecoderMatcher[String]("country")

implicit val yearQueryParamDecoder: QueryParamDecoder[Year] =
  QueryParamDecoder[Int].map(Year.of)
// yearQueryParamDecoder: QueryParamDecoder[Year] = org.http4s.QueryParamDecoder$$anon$8@49b00b45

object YearQueryParamMatcher extends QueryParamDecoderMatcher[Year]("year")

def getAverageTemperatureForCountryAndYear(country: String, year: Year): IO[Double] = ???

val averageTemperatureService = HttpService[IO] {
  case GET -> Root / "weather" / "temperature" :? CountryQueryParamMatcher(country) +& YearQueryParamMatcher(year)  =>
    Ok(getAverageTemperatureForCountryAndYear(country, year).map(s"Average temperature for $country in $year was: " + _))
}
// averageTemperatureService: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@1b5c0659
// )

Optional query parameters

To accept a optional query parameter a OptionalQueryParamDecoderMatcher can be used.

import cats.effect._
import org.http4s._, org.http4s.dsl.io._
import java.time.Year

implicit val yearQueryParamDecoder: QueryParamDecoder[Year] =
  QueryParamDecoder[Int].map(Year.of)
// yearQueryParamDecoder: QueryParamDecoder[Year] = org.http4s.QueryParamDecoder$$anon$8@1711b9b

object OptionalYearQueryParamMatcher extends OptionalQueryParamDecoderMatcher[Year]("year")

def getAverageTemperatureForCurrentYear: IO[String] = ???
def getAverageTemperatureForYear(y: Year): IO[String] = ???

val service2 = HttpService[IO] {
  case GET -> Root / "temperature" :? OptionalYearQueryParamMatcher(maybeYear) =>
    maybeYear match {
      case None =>
        Ok(getAverageTemperatureForCurrentYear)
      case Some(year) =>
        Ok(getAverageTemperatureForYear(year))
    }
}
// service2: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@2125d6ce
// )

Missing required query parameters

A request with a missing required query parameter will fall through to the following case statements and may eventually return a 404. To provide contextual error handling, optional query parameters or fallback routes can be used.

Invalid query parameter handling

To validate query parsing you can use ValidatingQueryParamDecoderMatcher which returns a ParseFailure if the parameter cannot be decoded. Be careful not to return the raw invalid value in a BadRequest because it could be used for Cross Site Scripting attacks.

import cats.effect._
import cats.implicits._
import org.http4s._, org.http4s.dsl.io._
import java.time.Year

def getAverageTemperatureForYear(y: Year): IO[String] = ???

implicit val yearQueryParamDecoder: QueryParamDecoder[Year] =
  QueryParamDecoder[Int].map(Year.of)
// yearQueryParamDecoder: QueryParamDecoder[Year] = org.http4s.QueryParamDecoder$$anon$8@68860b8c

object YearQueryParamMatcher extends ValidatingQueryParamDecoderMatcher[Year]("year")

val service = HttpService[IO] {
  case GET -> Root / "temperature" :? YearQueryParamMatcher(yearValidated) =>
    yearValidated.fold(
      parseFailures => BadRequest("unable to parse argument year"),
      year => Ok(getAverageTemperatureForYear(year))
    )
}
// service: HttpService[IO] = Kleisli(
//   org.http4s.HttpService$$$Lambda$12903/708292025@1313b60b
// )