Simple IRB formatting trick

If you execute IRB with the command line flag –noprompt you will enter in to an IRB session with none of the extra characters showing up on the left side of the terminal. This is great if you want to experiment with code and then use the mouse to copy & paste from it. Like, for example, in a code blog like the one you’re reading…

(Thanks to 6ftdan for this cool tip.)

Converting unix timestamps in Ruby

Unix timestamps are common to see in other languages, and also common to see in API responses. Converting a unix timestamp to DateTime in Ruby is dead simple:

unix_timestamp = Time.now.to_i
# => 1453667949
DateTime.strptime(unix_timestamp.to_s, "%s").to_s
# => "2016-01-24T20:39:09+00:00" 

Let’s say your unix timestamp includes milliseconds:

unix_timestamp = (Time.now.to_f * 1000).to_i
# => 1453667949151
DateTime.strptime(unix_timestamp.to_s, "%Q").to_s
# => "2016-01-24T20:39:09+00:00" 


DateTime.strptime will allow you convert just about any formatted time string, see the docs for examples.

Notice how both DateTime strings are the same? DateTime converted to string drops any millisecond data, but keep it as a DateTime object and you can run strftime and display it in any format you like:

DateTime.strptime(unix_timestamp.to_s, "%Q").strftime('%D %T:%L')
# => "01/24/16 20:39:09:151"

Arbitrary SQL ordering

I ran across an issue the other day where I had 4 records from the database that were being displayed in order of creation date, but I wanted them displayed in an arbitrary order - not alphabetical by name or title, or chronological, or sorted by id.

Before, I probably would have used Ruby to loop through the records and build an array by comparing ids. Now, though, I know better than to use Ruby for this type of thing, because it’s faster and better for memory to do it in SQL if possible (which I learned the hard way).

Turns out, querying using SQL looks a lot like what I would have written in Ruby anyhow:

SELECT * FROM foo ORDER BY
CASE
  WHEN id=67 THEN 1
  WHEN id=23 THEN 2
  WHEN id=1362 THEN 3
  WHEN id=24 THEN 4
END

Pretty cool! Thanks to cpjolicoeur on Github for this one.

Migrating data in Rails using Rails migrations

Put this one in the “should’ve been obvious” column…

For an embarassingly long while, when I had a feature ready for deploy that required production data be modified for a whole bunch of records – for example, downcasing all name attributes – I would write a one-off rake task to update those records, then delete that task.

Not only is this bad code since it doesn’t keep a record of what happened, it’s also tedious because it’s an extra step that must be taken by whoever deploys the code.

Luckily there’s a better way - just put the updating login in a migration!

Example:

class DowncaseName < ActiveRecord::Migration
  def up
    User.where.not(name: nil).find_each do |user|
       user.update!(name: user.name.downcase)
    end
  end
end

This example isn’t great because 1. there’s almost certainly a way to do this with pure SQL without instantiating every user record, and 2. it’s not reversible. But since I’m lazy and since it illustrates the point of the post, I’m leaving it.

No return value in rails console

A quick tip when working in rails console. If you're testing commands to run and don't want the console to output the return value, just append a semicolon to your ruby code.

  User.all
  # => [#<user:0x007fd72a408630> {:id => 1, :name => "Mr. T", ...}, tons and tons of other records]
  User.all;
  # => nothing!

OK, so that's a contrived example, but this really comes in handy

• when creating records, or 
• if you need to update several records in one shot and don't want to see the return values, or
• POSTing to an external API and avoiding all the cruft that comes in the response

Classy Ruby enumerating

Given two users:

User.create(first_name: "Joe", last_name: "Louis")
#=> User id: 1, first_name: "Joe", last_name: "Louis"

User.create(first_name: "Muhammed", last_name: "Ali")
#=> User id: 2, first_name: "Muhammed", last_name: "Ali"

Say you want to get an array of arrays, with each individual array containing two elements: the first and last name, and the id. You could do something like this, which is perfectly valid but also gross to read:

User.pluck(:id, :first_name, :last_name).map do |user|
  id = user[0]
  first = user[1]
  last = user[2]
  ["#{first} #{last}", id]
end

However, it’s much more readable to use tuple syntax:

User.pluck(:id, :first_name, :last_name).map do |(id, first, last)|
  ["#{first} #{last}", id]
end
#=> [
      ["Joe Louis", 1],
      ["Muhammed Ali", 2]
    ]

Funky Ruby Class Method Syntax

I’ve run across this syntax a few different times but never grasped how it worked, and I guess never cared enough to look it up. Turns out it’s dead simple.

This:

class Person  
  def self.species
    "Homo Sapien"
  end
end

is equivalent to this:

class Person  
  class << self
    def species
      "Homo Sapien"
    end
  end
end

Thanks to Yehuda Katz for finally clearing that up for me.

Literally the coolest way to build an array

One of Ruby’s literal syntax shorthands, %i essentially loops over each word, converts to a symbol and adds it to an array. For example:

%i(dog cat mouse)
# => [:dog, :cat, :mouse]

Caveat: This syntax was added in Ruby 2.0, so don’t go trying it in any previous version.

Familiarize quickly in new code with this trick

Here’s a quick and nifty little trick that comes in extremely handy in a lot of situations, but particularly when dealing with an unfamiliar codebase, such as at a new job or working with a previously unused external gem.

In the rails console, enter:

ls Model 

and it will show all the class and instance methods, as well as associations and a bunch of other cool stuff for that model.

I’ve begun using this in place of these two:

Model.methods
Model.new.methods

Not only because it’s simpler with less typing but it else ends up being much easier to read.

count vs size on ActiveRecord select queries

I stumbled on this bit of weirdness when using the SQL query User.select(:id, :name, :email).count. It threw a PG::UndefinedFunction error. Using User.select(:id, :name, :email).size worked instead.

Out of curiosity, I did some digging. According the the ActiveRecord docs,

not all valid select expressions are valid count expressions. The
specifics differ between databases. In invalid cases, an error from
the database is thrown.

I also stumbled on an issue on rails Github issues page about this, and a very helpful comment from rafaelfranca explained that I can also use count(:all).

Intro to SSH for relative newbies

I have an app running on an Amazon EC2 instance, and it requires an SSH key to access it. Previously, I had to access it by navigating to the folder containing my private key pem file, then running ssh -p 2222 -i private_key.pem -A ubuntu@www.example.com.

Let’s break this down a bit.

• -p 2222 - the port that my server runs on
• -i private_key.pem - use an Identity File (SSH key) and specify the name
• -A - use ForwardAgent to allow my public keys to pass through to the AWS server (important if I want to be able to access Github or some other service while SSH-ed into the server)
• ubuntu@www.example.com - username (ubuntu) and server HostName (www.example.com)

This is all fine and dandy, but it’s a pain to remember. So, my next step to simplify this was setting up a terminal alias in ~/.bash_aliases

alias my-server-ssh="cd ~ && ssh -p 2222 -i private_key.pem -A ubuntu@www.example.com"

This lets me simply run my-server-ssh from whatever directory I’m in on the terminal and automatically login to my server. However, once I have a couple more servers to juggle, it starts to get troublesome keeping track of these things. It’s also problematic if I want to use a tool like Capistrano for deployment.

Here’s where setting up the ~/.ssh/config file comes in handy. I opened this file in my text editor, and entered this info:

Host my-server
    HostName www.example.com
    Port 2222
    User ubuntu
    IdentityFile "~/.ssh/private_key.pem"
    ForwardAgent yes

Once I save, I can now use my computer’s built in SSH manager to access my server by running the command ssh my-server. Then, in a tool like Capistrano, I can plug this line into my config file:

server 'my-server', roles: %w{app web}

It will read my ~/.ssh/config file, find the configuration for host my-server and SSH in.

–

Big ups to Nerderati for the helpful explanation on SSH config.

Object overload! Careful of instantiating objects in Ruby

Wowzers, I just tackled a major memory issue in an app I’ve been developing, and it was nearly all due to rampant over-instantiation of ActiveRecord objects.

Here’s the TL;DR: Use SQL queries whenever possible to grab info from the database, rather than using Ruby code.

Now for the longer version. First, I knew I had a memory issue because my server was crashing and it was showing up the logs saying that memory could not be allocated. But I had to find out what the issue was first.

Oink to the rescue. It’s a handy little Ruby gem that will log how much memory and how many ActiveRecord objects are instantiated for each request.

I installed Oink, ran through a few requests that seemed kinda slow, and lo and behold, I found the problem: One of my requests was generating 1500 objects!

Here’s what was happening. I was looping through an array of IDs, and for each one I was calling

account.users.order(:id).select{ |user| user.id == id }.first

Don’t laugh. I have a baby at home, I was sleep deprived, I don’t know why I did it this way. But yes, this is instantiating all User objects in an account, and it’s doing it every single time it iterates through the loop. Hence, 1500 objects instantiated per request.

This was easily solved by a simple find_by_id with the id from the array.

That took me down to only one instance of each User object, which is still not fantastic. So, I ended up using one of my favorite ARel methods, pluck, to get just the attributes of the users that I need without actually creating the User objects.

That’s the ticket! From 1500 User objects to 0, this should significantly help with speed and memory usage. I ended up combing through the rest of the app and finding a few other spots where object instantiation was a bit out of control, though not nearly as bad as that one.

It became a kind of game - and in the end, I’d say I won.

Thanks to Engine Yard for the great tips. Lots more in their linked post.

URL Encoding and Silly Safari

I have a web app that also acts as an API for an iOS, and I ran into a tricky situation. One of my models on the web app takes a URL as input, and it validates it using the validates_url gem. However, some URLs when visited from within the iOS app would give a Webkit error and show as invalid.

After some head scratching, it turned out that those URLs had encoding issues that were handled fine by all modern browsers except Safari, the default browser on iOS. For example, something with curly brackets like "http://www.example.com?ID={12345}" would work in most browser but not in Safari/iOS.

To get around this, in addition to using validates_url, I had to escape each URL during validation using URI.escape(url). The above URL would then become "http://www.example.com?ID={12345}". (Notice that the left curly bracket changed to %7B, the right turned to %7D - check out this HTML URL Encoding Reference for details)

But wait, there’s more!

It turns out that on subsequent saves, the URL would be escaped again, and it wouldn’t recognize the %7B and %7D as encoding, it would just see the % sign as an invalid character needing escape. So, on the second save, the same URL would be transformed again into "http://www.example.com?ID=%7B12345%7D". Notice the “25” inserted in there twice?

The fix ended up being something that reads very strange and seems unnecessary, but that works perfectly:

URI.escape(URI.unescape(url))

So, if a URL is already escaped, it unescapes it first, then re-escapes it. If the URL is not escaped, then unescaping it does nothing, then it gets escaped as it should.

If there’s a better way to do this, I haven’t found it, and am open to suggestions.

Refactoring using guard clauses

This post is almost purely about code style, but it’s an approach to coding and refactoring that I just found and really enjoy for its simplicity and readability.

The secret sauce to this approach is called a guard clause.

Say I have a method that I’m writing where I want to return one thing is a condition is true, another thing if it’s false. The way I used to write the code is exactly how I wrote that sentence:

def do_something_conditionally(fact)
    if fact == true
        fact
    else
        false
    end
end

Pretty standard, right? Now check out the same example with a guard clause:

def do_something_conditionally(fact)
    return false unless fact == true
    fact
end

We saved a couple lines of code, which is nice (although I could’ve saved more turning the conditional into ternary (fact == true ? fact : false)). However, the big gain from doing it this way is that it’s easier to read. Disagree?

It may help to have a more complex example, perhaps using nested conditionals:

def account_type
    if has_problems?
        type = "problems"
        if is_past_due?
            type = "past due"
            if is_weeks_past_due?
                type = "delinquent"
            end
        end
    else
        type = "normal"
    end
    type
end

…and now with guard clauses:

def account_type
    return "delinquent" if is_weeks_past_due?
    return "past due" if is_past_due?
    return "problems" if has_problems?
    "normal"
end

We’ve saved more lines of code, this time something we could not have easily change to ternary, and it’s harder to argue that this isn’t more readable now.

If all of that isn’t convincing enough, check out the quasi-official Ruby Style Guide, because it also recommends using guard clauses when possible.

Asynchronous processing pitfalls and ActiveRecord locking

My app is humming along, but there are a few sections that have to do a lot of heavy lifting, whether that be image processing, updating several different related objects, or looping through all of a has_many association to manipulate every record.

These types of things are perfect places to push some logic to a background process. In other words, make it asynchronous. So, if a user clicks a button to update several objects, unless the user needs to see those updates completed immediately, then a tool like Sidekiq can help run some expensive logic as a background process and allow the user to continue surfing the site while that process runs.

There are plenty of tutorials about how to set this up, so that’s not what I want to talk about here.

One of the potential pitfalls of processing things asynchronously, particularly if the processing involves updating database records, is multiple records being updated at the same time. For example, one user updates a Project in a background process, and another user updates the same Project somewhere else in the app that doesn’t use a background process. We want to last update to be the one that sticks, but if both updates are happening at the same time, things could get hairy.

This is the dreaded “race conditions” issue, and luckily there’s a pretty simple fix.

On ActiveRecord - and I believe this will work with any SQL database but definitely at least on Postgres and MySQL - I can lock a specific row while editing, then unlock it afterwards, so that each edit takes place in its own little bubble.

Account.transaction do
  acc = Account.lock.find(id)
  acc.balance -= 25
  acc.save!
end

This finds an account and locks it, then updates the balance, saves, and unlocks.

This is really important when dealing with numbers, money, and balances. Let me share a parable to explain:

There is an Account with $100 balance. One user in that account makes a transaction for$75.

A split second later, another user in that account makes a transaction for $90.

What should happen is the second user’s transaction should be rejected for lack of funds.

What actually happens without locking is that for each transaction, the first step is that the server runs account = Account.find(id) to retrieve the account, and since this is happening before either transaction has finished, the balance on the account in both cases is $100. So, after the first user's transaction, the balance is updated to$25 and the account is saved. But on the second user’s transaction, the account object that was pulled from the database still shows a balance of $100, so the server deducts the$90 charge from the balance and saves it, thereby allowing the transaction to go through and setting the saved account balance as $10.

Is your head spinning yet?

When we lock the account per transaction, we avoid this issue by only allowing the account to be edited on transaction at a time.

It’s a cool concept but also can be tricky to wrap your head around. Here’s a blog post I borrowed heavily from for my post. Or check out the Rails API docs.

Turbocharged custom Rails / ActiveRecord validations

My code just got a lot less janky when dealing with start and end times. Before, when I wanted to validate that a user-submitted end time came after a start time, I was doing something like this:

validate :start_time_is_before_end_time
def start_time_is_before_end_time
    unless start_time.to_i < end_time.to_i
        errors.add(:end_time, 'must come after start time')
    end
end

Converting the values to epoch time and comparing isn’t terrible, but I just found a cool way to extract this out to a custom validator to make it simpler and more readable:

# app/models/model.rb
require "active_model/sequential_validator"
validates :start_time, :end_time, sequential: true

# lib/active_model/sequential_validator.rb
class SequentialValidator < ActiveModel::Validator
    def validate record
        values = options[:attributes].map do |attribute|
           record.send(attribute)
        end.compact
        if values.sort != values
            record.errors.add options[:attributes].last, "cannot be before #{options[:attributes].first}"
        end
    end
end

True, the actual validator code is not as easy to read I don’t think, but it takes some of the burden out of the model, and it also allows reusing the sequential validation in other models.

(Thanks to botandrose and the calagator project for this cool piece of code.)

More readable for ActiveRecord statements using Placeholder Conditions

Here’s another readability hack for Ruby ActiveRecord code. It’s called “Placeholder Conditions.”

This:

where("start_time < :time", time: Time.now)

works the same as this

where("start_time < ?", Time.now)

The second is more terse, but if you are inserting a lot of values into the WHERE statement, then the first is nice for readability.

Source: Ruby on Rails Guides

Indifferent Access in Ruby Hashes (with ActiveSupport)

I kept seeing this “Hash with indifferent access” in open source projects, but never knew quite what it was. Turns out it’s something I use in Rails all of the time without knowing, particularly in dealing with params. It allows me to access values in a hash using either string or symbolized keys.

For example:

params['name'] == params[:name]
# => true

However, a normal Hash in Ruby does not allow this:

hash = { 'name' => 'Fred' }
hash['name']
# => 'Fred'
hash[:name]
# => nil
hash['name'] == hash[:name]
# => false

We can fix this by making this into a hash with indifferent access:

new_hash = hash.with_indifferent_access
new_hash['name'] == new_hash[:name]
# => true

The hash can also be defined/initialized with indifferent access, either by calling that method on the initial hash assignment, or by creating it using ActiveSupport (which is what the with_indifferent_access method does under the hood anyway):

hash = ActiveSupport::HashWithIndifferentAccess.new(name: 'Fred')
hash['name'] == hash[:name]
# => true

Note: When working in straight Ruby, this will not work as it’s using ActiveSupport. One would likely need to run gem install active_support and require it properly.

Source: apidock and RoR api

More readable value change testing in rspec

Some simple but effective rspec syntax that I just learned:

expect{Object.method}.to change{Object.count}.from(0).to(1)

I think this is fairly straightforward, but essentially it just lets you test for value changes. This is basically just a clearer way of writing expectations and would replace something like this:

expect(Object.count).to eq(0)
Object.method
expect(Object.count).to eq(1)

Check the rspec docs for more details.

Quicker and Easier headers with HTTParty

You may remember last month I showed how to retrieve just the headers in an HTTP request using the HTTParty ruby gem.

At that time, there were a couple of extra options that needed to specified in the case of redirects, otherwise on each subsequent redirect, the HEAD request would be converted to a normal GET request. This wasn’t ideal because the reason I was doing HEAD requests in the first place was to save the time it would take to retrieve the entire page and instead just find out whether the request was successful. It also didn’t make much common sense.

So I changed the gem. It now follows redirects and maintains the HEAD request across all redirects.

This is one of the reasons I really enjoy working in code. There’s this great tool that works extremely well in most cases, but there’s something slightly off about one small part of it. Rather than making a support request to some big conglomeration that may or may not read it and may or may not fix the issue, I can just jump in, make the change, and hopefully help improve the functionality for myself and, in the process, for countless others.