F# / .NET: Gotcha's
The dreaded System.Net.WebException: The remote server returned an error
mess...
If you've ever used the WebClient
to target RESTful API's in .NET, you probably hit this error at least once:
System.Net.WebException: The remote server returned an error: {Some Error}.
at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
at System.Net.WebClient.DownloadString(Uri address)
at System.Net.WebClient.DownloadString(String address)
Scenario: you're just chugging along, working on some stuff, then an exception comes in and BAM, messes up your whole day.
I had this happen to a coworker yesterday. He was working on something and kept wondering why the endpoint was giving him a 404: Not Found
in a WebException
. He'd never really hit this before, because this was the first time he needed to consume a RESTful API in .NET.
What's happening here?
When the .NET WebClient
sends a request, it's backed by a WebRequest
, which happens to have someone's "good" idea: when we hit a non-200 status code, throw an exception. That makes sense, right? If we hit a 404, it makes sense to throw an exception, right?
Sure, in 1990. It's 2019 though, we use the 404 to mean something. Some genius out there said "hey, when building RESTful API's, just send an HTTP status code with your 'thing', we already have a list of them." The problem is that up, down, left and right people were treating 300
, 400
, and 500
status codes as actual errors, and things would break. Well, .NET is no exception.
So how do we fix it?
Yes, this is the easy part. We have to use a try
/ catch
or try
/ with
, depending on the language (OH NO!).
Effectively, the following code will throw a System.Net.WebException: The remote server returned an error: (404) Not Found.
on line 7, when called on line 11. We'll fix that.
open System.Net
let urlBase = "https://httpstat.us/"
let getCode code =
use wc = new WebClient()
wc.Headers.Add(HttpRequestHeader.Accept, "application/json")
code |> sprintf "%s%i" urlBase |> wc.DownloadString
200 |> getCode |> printfn "%s"
202 |> getCode |> printfn "%s"
404 |> getCode |> printfn "%s"
So, what do we do?
Easy, catch the WebException
and read the response:
open System.IO
open System.Net
let urlBase = "https://httpstat.us/"
let getCode code =
try
use wc = new WebClient()
wc.Headers.Add(HttpRequestHeader.Accept, "application/json")
code |> sprintf "%s%i" urlBase |> wc.DownloadString
with :? WebException as ex ->
use sr = new StreamReader(ex.Response.GetResponseStream())
sr.ReadToEnd()
200 |> getCode |> printfn "%s"
202 |> getCode |> printfn "%s"
404 |> getCode |> printfn "%s"
This isn't perfect, we'll catch all WebException
issues, so we want one more "guard" clause:
open System.IO
open System.Net
let urlBase = "https://httpstat.us/"
let getCode code =
try
use wc = new WebClient()
wc.Headers.Add(HttpRequestHeader.Accept, "application/json")
code |> sprintf "%s%i" urlBase |> wc.DownloadString
with :? WebException as ex when ex.Status = WebExceptionStatus.ProtocolError ->
use sr = new StreamReader(ex.Response.GetResponseStream())
sr.ReadToEnd()
200 |> getCode |> printfn "%s"
202 |> getCode |> printfn "%s"
404 |> getCode |> printfn "%s"
Bam. Now we only catch ProtocolError
exceptions, like 400
, 500
, etc.