URL Encoding and Silly Safari

I have a web app that also acts as an API for an iOS, and I ran into a tricky situation. One of my models on the web app takes a URL as input, and it validates it using the validates_url gem. However, some URLs when visited from within the iOS app would give a Webkit error and show as invalid.

After some head scratching, it turned out that those URLs had encoding issues that were handled fine by all modern browsers except Safari, the default browser on iOS. For example, something with curly brackets like "http://www.example.com?ID={12345}" would work in most browser but not in Safari/iOS.

To get around this, in addition to using validates_url, I had to escape each URL during validation using URI.escape(url). The above URL would then become "http://www.example.com?ID={12345}". (Notice that the left curly bracket changed to %7B, the right turned to %7D - check out this HTML URL Encoding Reference for details)

But wait, there’s more!

It turns out that on subsequent saves, the URL would be escaped again, and it wouldn’t recognize the %7B and %7D as encoding, it would just see the % sign as an invalid character needing escape. So, on the second save, the same URL would be transformed again into "http://www.example.com?ID=%7B12345%7D". Notice the “25” inserted in there twice?

The fix ended up being something that reads very strange and seems unnecessary, but that works perfectly:

URI.escape(URI.unescape(url))

So, if a URL is already escaped, it unescapes it first, then re-escapes it. If the URL is not escaped, then unescaping it does nothing, then it gets escaped as it should.

If there’s a better way to do this, I haven’t found it, and am open to suggestions.

Refactoring using guard clauses

This post is almost purely about code style, but it’s an approach to coding and refactoring that I just found and really enjoy for its simplicity and readability.

The secret sauce to this approach is called a guard clause.

Say I have a method that I’m writing where I want to return one thing is a condition is true, another thing if it’s false. The way I used to write the code is exactly how I wrote that sentence:

def do_something_conditionally(fact)
    if fact == true
        fact
    else
        false
    end
end

Pretty standard, right? Now check out the same example with a guard clause:

def do_something_conditionally(fact)
    return false unless fact == true
    fact
end

We saved a couple lines of code, which is nice (although I could’ve saved more turning the conditional into ternary (fact == true ? fact : false)). However, the big gain from doing it this way is that it’s easier to read. Disagree?

It may help to have a more complex example, perhaps using nested conditionals:

def account_type
    if has_problems?
        type = "problems"
        if is_past_due?
            type = "past due"
            if is_weeks_past_due?
                type = "delinquent"
            end
        end
    else
        type = "normal"
    end
    type
end

…and now with guard clauses:

def account_type
    return "delinquent" if is_weeks_past_due?
    return "past due" if is_past_due?
    return "problems" if has_problems?
    "normal"
end

We’ve saved more lines of code, this time something we could not have easily change to ternary, and it’s harder to argue that this isn’t more readable now.

If all of that isn’t convincing enough, check out the quasi-official Ruby Style Guide, because it also recommends using guard clauses when possible.

Asynchronous processing pitfalls and ActiveRecord locking

My app is humming along, but there are a few sections that have to do a lot of heavy lifting, whether that be image processing, updating several different related objects, or looping through all of a has_many association to manipulate every record.

These types of things are perfect places to push some logic to a background process. In other words, make it asynchronous. So, if a user clicks a button to update several objects, unless the user needs to see those updates completed immediately, then a tool like Sidekiq can help run some expensive logic as a background process and allow the user to continue surfing the site while that process runs.

There are plenty of tutorials about how to set this up, so that’s not what I want to talk about here.

One of the potential pitfalls of processing things asynchronously, particularly if the processing involves updating database records, is multiple records being updated at the same time. For example, one user updates a Project in a background process, and another user updates the same Project somewhere else in the app that doesn’t use a background process. We want to last update to be the one that sticks, but if both updates are happening at the same time, things could get hairy.

This is the dreaded “race conditions” issue, and luckily there’s a pretty simple fix.

On ActiveRecord - and I believe this will work with any SQL database but definitely at least on Postgres and MySQL - I can lock a specific row while editing, then unlock it afterwards, so that each edit takes place in its own little bubble.

Account.transaction do
  acc = Account.lock.find(id)
  acc.balance -= 25
  acc.save!
end

This finds an account and locks it, then updates the balance, saves, and unlocks.

This is really important when dealing with numbers, money, and balances. Let me share a parable to explain:

There is an Account with $100 balance. One user in that account makes a transaction for$75.

A split second later, another user in that account makes a transaction for $90.

What should happen is the second user’s transaction should be rejected for lack of funds.

What actually happens without locking is that for each transaction, the first step is that the server runs account = Account.find(id) to retrieve the account, and since this is happening before either transaction has finished, the balance on the account in both cases is $100. So, after the first user's transaction, the balance is updated to$25 and the account is saved. But on the second user’s transaction, the account object that was pulled from the database still shows a balance of $100, so the server deducts the$90 charge from the balance and saves it, thereby allowing the transaction to go through and setting the saved account balance as $10.

Is your head spinning yet?

When we lock the account per transaction, we avoid this issue by only allowing the account to be edited on transaction at a time.

It’s a cool concept but also can be tricky to wrap your head around. Here’s a blog post I borrowed heavily from for my post. Or check out the Rails API docs.

Turbocharged custom Rails / ActiveRecord validations

My code just got a lot less janky when dealing with start and end times. Before, when I wanted to validate that a user-submitted end time came after a start time, I was doing something like this:

validate :start_time_is_before_end_time
def start_time_is_before_end_time
    unless start_time.to_i < end_time.to_i
        errors.add(:end_time, 'must come after start time')
    end
end

Converting the values to epoch time and comparing isn’t terrible, but I just found a cool way to extract this out to a custom validator to make it simpler and more readable:

# app/models/model.rb
require "active_model/sequential_validator"
validates :start_time, :end_time, sequential: true

# lib/active_model/sequential_validator.rb
class SequentialValidator < ActiveModel::Validator
    def validate record
        values = options[:attributes].map do |attribute|
           record.send(attribute)
        end.compact
        if values.sort != values
            record.errors.add options[:attributes].last, "cannot be before #{options[:attributes].first}"
        end
    end
end

True, the actual validator code is not as easy to read I don’t think, but it takes some of the burden out of the model, and it also allows reusing the sequential validation in other models.

(Thanks to botandrose and the calagator project for this cool piece of code.)

More readable for ActiveRecord statements using Placeholder Conditions

Here’s another readability hack for Ruby ActiveRecord code. It’s called “Placeholder Conditions.”

This:

where("start_time < :time", time: Time.now)

works the same as this

where("start_time < ?", Time.now)

The second is more terse, but if you are inserting a lot of values into the WHERE statement, then the first is nice for readability.

Source: Ruby on Rails Guides

Indifferent Access in Ruby Hashes (with ActiveSupport)

I kept seeing this “Hash with indifferent access” in open source projects, but never knew quite what it was. Turns out it’s something I use in Rails all of the time without knowing, particularly in dealing with params. It allows me to access values in a hash using either string or symbolized keys.

For example:

params['name'] == params[:name]
# => true

However, a normal Hash in Ruby does not allow this:

hash = { 'name' => 'Fred' }
hash['name']
# => 'Fred'
hash[:name]
# => nil
hash['name'] == hash[:name]
# => false

We can fix this by making this into a hash with indifferent access:

new_hash = hash.with_indifferent_access
new_hash['name'] == new_hash[:name]
# => true

The hash can also be defined/initialized with indifferent access, either by calling that method on the initial hash assignment, or by creating it using ActiveSupport (which is what the with_indifferent_access method does under the hood anyway):

hash = ActiveSupport::HashWithIndifferentAccess.new(name: 'Fred')
hash['name'] == hash[:name]
# => true

Note: When working in straight Ruby, this will not work as it’s using ActiveSupport. One would likely need to run gem install active_support and require it properly.

Source: apidock and RoR api

More readable value change testing in rspec

Some simple but effective rspec syntax that I just learned:

expect{Object.method}.to change{Object.count}.from(0).to(1)

I think this is fairly straightforward, but essentially it just lets you test for value changes. This is basically just a clearer way of writing expectations and would replace something like this:

expect(Object.count).to eq(0)
Object.method
expect(Object.count).to eq(1)

Check the rspec docs for more details.