Quicker and Easier headers with HTTParty

You may remember last month I showed how to retrieve just the headers in an HTTP request using the HTTParty ruby gem.

At that time, there were a couple of extra options that needed to specified in the case of redirects, otherwise on each subsequent redirect, the HEAD request would be converted to a normal GET request. This wasn’t ideal because the reason I was doing HEAD requests in the first place was to save the time it would take to retrieve the entire page and instead just find out whether the request was successful. It also didn’t make much common sense.

So I changed the gem. It now follows redirects and maintains the HEAD request across all redirects.

This is one of the reasons I really enjoy working in code. There’s this great tool that works extremely well in most cases, but there’s something slightly off about one small part of it. Rather than making a support request to some big conglomeration that may or may not read it and may or may not fix the issue, I can just jump in, make the change, and hopefully help improve the functionality for myself and, in the process, for countless others.

Quick and easy HTTP headers with HTTParty

UPDATE: I changed the gem. It now follows redirects and maintains the HEAD request across all redirects.

I have a method that checks URLs submitted by the user to ensure they are valid. I don’t want to scrape the page to make sure any specific text is on there because the URLs can go almost anywhere, so I basically only care if the URL returns an HTTP 200 status.

I’m using the excellent HTTParty Ruby gem to make my external requests, and it offers a handy .head method that only returns the website headers for the request:

HTTPart.head(external_url)

However, I found out that if the requested site redirects, then subsequent requests get changed to GET, which isn’t ideal since the GET request will download the entire page’s content rather than just the headers, and is understandably slower.

Thankfully, there are a couple options that can be passed to the .head method which takes care of this:

HTTParty.head(external_url_with_redirects,
              maintain_method_across_redirects: true,
              follow_redirects: true)

Now, I get just the headers, and since all I care about is whether it was a successful request, I’m able to call .success? in my method and use the return value.

There are a few external HTTP gems out there - Net:HTTP (of which HTTParty is a wrapper), RestClient, etc - but HTTParty is my favorite so far, and easiest to work with. Check it out if you haven’t already.