This past weekend, I deployed a ton of under-the-hood updates to Cushion. In that list was the re-enabling of Skylight, a Ruby profiler for requests and database queries. This gem has helped us pinpoint the issues behind slow requests in the past, but once we got everything in good shape, we disabled it to save money rather than keep it running when we weren’t actively utilizing it.

Now that I’ve started modernizing Cushion, I wanted to turn Skylight back on, but it’s been a while. There have been about three major version releases between the gem Cushion was using and the most recent one. Sure, I could’ve kept it simple and re-enabled Skylight with the old gem, but the new version actually includes a few features I’ve been eager to use—environments, which lets you specify whether a server is development, review-#{n}, staging, or production, and support for background jobs, which allows you to profile the performance of your workers.

Like with every dependency bump, I triggered an update with my fingers crossed, hoping it wouldn’t complain about conflicts with other dependencies. Alas, the latest Skylight version requires a newer version of ActiveSupport. This was worrisome because ActiveSupport tends to patch Ruby code to provide its extensions, so plenty of behaviors could change without any reference to “ActiveSupport”. Luckily, I have solid code coverage with my backend tests, so it should be able to catch anything unexpected.

The unit tests did catch two issues—neither of which were obvious upon seeing the initial errors. The first error said that the database query for getting time-tracking entries no longer returned anything. Fantastic. The second error triggered VCR, indicating that the OAuth refresh token tests for Harvest authorization were trying to hit a different URL. Wat?

After possibly hours of inching closer and closer to the first issue, I pinpointed it. Cushion’s database query for time-tracking entries previously included Sequel’s .split_symbols feature, which let you say :table__column to output to table.column in the query, but for some reason, updating ActiveSupport changed this behavior to result in table__column. That’s why the query didn’t return any entries. To fix it, I simply updated the code to Sequel’s new API for this, Sequel[:table][:column]. Part of me definitely feels like I should just be writing raw SQL queries, but that’s outside of the given task’s scope.

The second error involving the different OAuth URL was also strange because there’s no immediate reason why updating ActiveSupport would change a URL. Once again, inching closer and closer to the issue by running the test and finding its source, I narrowed in on the culprit. For Cushion’s OAuth refresh token code, I grab the client options from the proper OAuth strategy. Unfortunately, the OAuth strategy options use string-based keys while OAuth2::Client requires symbol-based keys. Without going on too much of a tangent, supporting both string-based and symbol-based keys in a hash is probably one of the most aggravating parts of Ruby and has led to the smh-inducing ActiveSupport::HashWithIndifferentAccess class.

In any case, to accommodate this in the past, I used symbolize_keys, which turns string-based keys into symbol-based keys, but strangely, the ActiveSupport update broken this implementation. Luckily, it didn’t take me too long to realize I could no longer call symbolize_keys on the OmniAuth::Strategy::Options instances themselves. Instead, I had to first convert them to a hash. This makes sense, but why did it work before and why did the update change that? At this point, I’ve had enough and don’t need to know the reasons, so I decided to move on.

Fortunately, these were the only two hiccups that came up when updating ActiveSupport and they only cost a few hours of my life. Now I have Skylight running across all of Cushion’s environments, including its worker dynos, which handle all the heavy lifting, like integrations, backups, etc. It feels great to have insight into performance again, so I can keep a close eye on any changes going forward.