API edge cases revisited

For pipeline steps, nesting is often better than chaining.

In my last post, I talked about the difficulty of chaining together steps in an API and explored a way to construct an API pipeline that was more flexible to change.

However, it has two problems. 1) There is a lot of boilerplate, especially the error branches. 2) It looks horrible.

So, I experimented and found another way that seems much better. I actually discovered this idea when working with Elm. However, I couldn't implement it in quite the same way. The syntax I was going for looked something like this:

AsyncResult.retn request
|> AsyncResult.bind (fun request -> getHandler request.Path
|> AsyncResult.bind (fun (handlerPath, handle) -> getUser request.User
))

However, this does not compile in F# because the parser doesn't like the spacing. Also the parenthesis become a bit bothersome over time. So then I tried using the inline operator for bind (>>=), and eventually I stumbled upon a style that I found amenable. Here is the code for my query API pipeline. Look how fun it is.

let (>>=) x f =
    AsyncResult.bind f x

let run connectionString (request:Request) (readJsonFunc:ReadJsonFunc) =
    let correlationId = Guid.NewGuid()
    let log = Logger.log correlationId
    let readJson = fun _ -> readJsonFunc.Invoke() |> Async.AwaitTask
 
    AsyncResult.retn request
    >>= fun request ->
        log <| RequestStarted (RequestLogEntry.fromRequest request)
        getHandler request.Path
 
    >>= fun (handlerPath, handle) ->
        log <| HandlerFound handlerPath
        getUser request.User
 
    >>= fun user ->
        log <| UserFound (user.Identity.Name)
        authorize handlerPath user
 
    >>= fun claim ->
        log <| OperationAuthorized claim
        getJson readJson ()
 
    >>= fun json ->
        log <| JsonLoaded json
        let jsonRequest = JsonRequest.mk user json
        handle connectionString jsonRequest
 
    >>= fun (resultJson, count) ->
        log <| QueryFinished count
        toJsonResponse resultJson
 
    |> AsyncResult.teeError (ErrorEncountered >> log)
    |> AsyncResult.either id Responder.toErrorResponse
    |> Async.tee (ResponseCreated >> log)
    |> Async.StartAsTask

This style is a vast improvement in readability as well as (lack of) boilerplate. Now each step is actually nested, but F# lets me write them without nested indentation.

The primary value proposition of nested binds (e.g. x >>= fun x' -> f1 x' >>= f2) instead of chain binds (e.g. x >>= f1 >>= f2) is the easy access to all previous step results. For example, handle is defined in the 2nd step but is used in the 5th step. Notice that I could easily swap steps 2 and 3 without affecting any subsequent steps. (Select/cut getHandler down to thru the log statement, and paste it below the UserFound log statement. No other refactoring needed!)

If I were to do chaining, Steps 3 and 4 would have to carry handle through their code into their output so that Step 5 has access to it. This creates coupling between steps, as well as extra data structures (tuple or record passed between steps) that need maintenance when steps change.

I think the next thing this needs is inlining the logging. But for now, I'm pretty happy with it.

For reference, here is the old version of the code which enumerates every branch explicitly. (Also uses nesting instead of chaining.)

let run connectionString (request:Request) (readJsonFunc:ReadJsonFunc) =
    let correlationId = Guid.NewGuid()
    let log = Logger.log correlationId
    let readJson = fun _ -> readJsonFunc.Invoke() |> Async.AwaitTask
    let logErr = tee (ErrorEncountered >> log) >> Responder.toErrorResponse
    let logResponse = ResponseCreated >> log
    async {
        log <| RequestStarted (RequestLogEntry.fromRequest request)
        match getHandler request.Path with
        | Error err ->
            return logErr err
 
        | Ok (handlerPath, handle) ->
        log <| HandlerFound handlerPath
        match getUser request.User with
        | Error err ->
            return logErr err
 
        | Ok user ->
        log <| UserFound (user.Identity.Name)
        match authorize handlerPath user with
        | Error err ->
            return logErr err
 
        | Ok claim ->
        log <| OperationAuthorized claim
        let! jsonResult = getJson readJson ()
        match jsonResult with
        | Error err ->
            return logErr err
 
        | Ok json ->
        log <| JsonLoaded json
        let jsonRequest = JsonRequest.mk user json
        let! outEventsResult = handle connectionString jsonRequest
        match outEventsResult with
        | Error err ->
            return logErr err
 
        | Ok (resultJson, count) ->
            log <| QueryFinished count
            return Responder.toJsonResponse resultJson
 
    }
    |> Async.tee logResponse

    |> Async.StartAsTask