F# / .NET: Gotcha's

The dreaded System.Net.WebException: The remote server returned an error mess...

If you've ever used the WebClient to target RESTful API's in .NET, you probably hit this error at least once:

System.Net.WebException: The remote server returned an error: {Some Error}.
   at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
   at System.Net.WebClient.DownloadString(Uri address)
   at System.Net.WebClient.DownloadString(String address)

Scenario: you're just chugging along, working on some stuff, then an exception comes in and BAM, messes up your whole day.

I had this happen to a coworker yesterday. He was working on something and kept wondering why the endpoint was giving him a 404: Not Found in a WebException. He'd never really hit this before, because this was the first time he needed to consume a RESTful API in .NET.

What's happening here?

When the .NET WebClient sends a request, it's backed by a WebRequest, which happens to have someone's "good" idea: when we hit a non-200 status code, throw an exception. That makes sense, right? If we hit a 404, it makes sense to throw an exception, right?

Sure, in 1990. It's 2019 though, we use the 404 to mean something. Some genius out there said "hey, when building RESTful API's, just send an HTTP status code with your 'thing', we already have a list of them." The problem is that up, down, left and right people were treating 300, 400, and 500 status codes as actual errors, and things would break. Well, .NET is no exception.

So how do we fix it?

Yes, this is the easy part. We have to use a try / catch or try / with, depending on the language (OH NO!).

Effectively, the following code will throw a System.Net.WebException: The remote server returned an error: (404) Not Found. on line 7, when called on line 11. We'll fix that.

open System.Net
let urlBase = "https://httpstat.us/"

let getCode code =
    use wc = new WebClient()
    wc.Headers.Add(HttpRequestHeader.Accept, "application/json")
    code |> sprintf "%s%i" urlBase |> wc.DownloadString

200 |> getCode |> printfn "%s"
202 |> getCode |> printfn "%s"
404 |> getCode |> printfn "%s"

So, what do we do?

Easy, catch the WebException and read the response:

open System.IO
open System.Net
let urlBase = "https://httpstat.us/"

let getCode code =
    try
        use wc = new WebClient()
        wc.Headers.Add(HttpRequestHeader.Accept, "application/json")
        code |> sprintf "%s%i" urlBase |> wc.DownloadString
    with :? WebException as ex ->
        use sr = new StreamReader(ex.Response.GetResponseStream())
        sr.ReadToEnd()

200 |> getCode |> printfn "%s"
202 |> getCode |> printfn "%s"
404 |> getCode |> printfn "%s"

This isn't perfect, we'll catch all WebException issues, so we want one more "guard" clause:

open System.IO
open System.Net
let urlBase = "https://httpstat.us/"

let getCode code =
    try
        use wc = new WebClient()
        wc.Headers.Add(HttpRequestHeader.Accept, "application/json")
        code |> sprintf "%s%i" urlBase |> wc.DownloadString
    with :? WebException as ex when ex.Status = WebExceptionStatus.ProtocolError ->
        use sr = new StreamReader(ex.Response.GetResponseStream())
        sr.ReadToEnd()

200 |> getCode |> printfn "%s"
202 |> getCode |> printfn "%s"
404 |> getCode |> printfn "%s"

Bam. Now we only catch ProtocolError exceptions, like 400, 500, etc.

F# / .NET: Gotcha's

This is another annoying "feature" in .NET. For some reasons, TimeSpan.ToString with a custom format string does all sorts of wonky stuff, particularly it will give you the "Invalid Format String" error on any unescaped character that isn't a valid format specifier.

Basically, take this list: Custom TimeSpan Format Strings and treat it literally.

Want to add a space between days and hours/minutes/seconds? Escape it: d\ hh\:mm\:ss. Yes, even spaces.

F# / .NET: Gotcha's

This is one of the biggest, most painful errors I've ever seen in my life. There's absolutely no direction this error can take you.

Here's the thing: GDI+ doesn't really report any errors it encounters down the stack properly. They're all "Generic Errors".

So, some things to look for:

• Is the resource disposed? If you have a Using / using / use, and you try to access the resource outside of that scope, you'll probably get this error.
• Calling to .Save? There's a whole list of stuff for that. If you get this error when calling .Save on an Image / Bitmap / etc., you might want to check any and all of the following:
  • Does the directory exist? GDI+ won't create it.
  • Do you have permission to the directory / file? GDI+ won't tell you if that's the problem.
  • Did you dispose of the MemoryStream? That'll be a generic error. You'll want to make sure you save before disposal.

There are many, many more, and I'll be updating this list as time permits.

F# / .NET: Gotcha's

If you are a user of Giraffe with ASP.NET Core and F#, you might have come across the following error:

FS0708: This control construct may only be used if the computation expression builder defines a 'Bind' method

This happens if you try to use a let! expression inside a task computation expression:

task {
    let! x = ...Async()

The problem here is that task as a computation expression doesn't define a "Bind" method, from what I can tell. There is, however, a quick workaround:

open FSharp.Control.Tasks.V2.ContextInsensitive

For some reason, this open fixes it. I assume that it allows the async computation-expression version of the Bind to be used, so that the code above works.

There's been discussion around this, from what I can tell, but it hasn't really resolved this issue, not sure if it's a version thing, or entirely unrelated: Q: remove async await bind from task {}?.

The Giraffe folks do a great job, so I want to make sure you can work around this particular issue if and when you encounter it.

Recently I've been digging more-and-more into F#, so I want to start putting together a list of things that I occasionally (or regularly) run into that new users of the language (or even those who are substantially experienced) might not have a great time with.

I'll be notating if it's a general .NET "gotcha", or an F#-specific "gotcha."

F#:

• Giraffe Task "FS0708"

.NET:

• A Generic Error Occurred in GDI+
• TimeSpan: Invalid Format String
• System.Net.WebException: The remote server returned an error

If you have something you want to see in the list, please let me know either via Twitter or as a comment. I'll try to check back here regularly to keep this list as up-to-date as possible.