We’re currently in the process of extracting Cushion’s client-facing invoice page as its own server. I’ll explain exactly why in another post, but we’re taking this opportunity to improve and modernize our web infrastructure on a smaller scale before migrating our entire stack all at once.

On the main app, we use AWS Cloudfront as a CDN for our assets (scripts, images, stylesheets, etc), so they’re cached globally and load as fast as possible from the nearest “edge location”. Our current build process on Codeship includes three steps to put our assets online—one to compile the assets, another to upload the assets to AWS S3, and a third step to commit our asset manifest to the codebase. An asset manifest keeps track of the asset filenames because they include randomly generated suffixes for cache-busting (e.g. app.js becomes app-123456789.js). If you glossed over that last sentence because you’re still irked that we commit our manifest to the codebase, I’m right there with you.

In our new build process on Heroku CI, we compile our assets in Heroku’s “postbuild” hook, which occurs right after installing dependencies. Instead of uploading the assets to S3, we leave them in Heroku’s file system. We can do this because of Cloudfront. I didn’t realize this several years ago, but we can actually point Cloudfront at our entire server, then point our domain at Cloudfront instead of only using it for the assets. Even reading the Nuxt.js documentation on the publicPath setting, I was under the impressions that we needed upload our assets to S3 first, but luckily, we don’t. Everything works as-is.

The one caveat to this approach is that, by default, Cloudfront will cache everything, including your app’s server-side rendered pages. For our use-case, we don’t want to cache the invoice pages too heavily in the event that the user needs to make a quick change. Fortunately, Cloudfront lets us specify “behaviors” to handle specific routes differently from others. We created behaviors for both the API and the app pages, then changed “Cache Based on Selected Request Headers” to “All”. This tells Cloudfront to cache requests using headers like cache-control. Now, when a user needs to tweak an invoice, the change is reflected right away instead of remaining cached by Cloudfront until eternity.

Another issue we came across with this approach is that Heroku returns a 404 unless you set your app’s domain to the domain you’re viewing, which at this time would be the Cloudfront URL. Since we’ll eventually use a custom domain, this will fix itself upon adding the custom domain to Heroku.

To get our custom domain to load from Cloudfront, we created a CNAME for our app and pointed it to the Cloudfront URL. We also specified our domain in Cloudfront’s “Alternate Domain Names” field. Our Cloudfront origin then points to the original Heroku app URL, which displays the app using our custom domain.

In the main app, believe it or not, we still pay for a wildcard certificate and manually renew it each year—the “uphill both ways” for devs. This has cursed us with years of scrutiny from the HN community whenever our running costs page makes the rounds. Since originally launching Cushion, however, we’ve been graced with several free SSL services, like Let’s Encrypt, Heroku ACM, and AWS Certificate Manager. In the past, we had to hold off because our in.vc domain name had special circumstances, but now with our invoice server, we’re able to make the switch.

The in.vc domain currently masks the main app, and in order to use the root domain in.vc (instead of www.in.vc), we needed to hack Heroku’s SSL Endpoint add-on in order to get everything working properly. Because of this, we couldn’t take advantage of Heroku’s free auto-renewing SSL. Now that the invoice server will run as its own Heroku app, we can easily set up SSL without any hacks—and we can save a ton of money.

By switching to a free SSL service, like AWS Certificate Manager, and migrating in.vc’s DNS from DNSimple to AWS Route 53, we’ll actually save around $400/year. Then, once we fully switch from Codeship to Heroku CI, we’ll save another $450/year.

I’m thrilled that we were able to make these improvements because it keeps our setup fresh while lifting the weight of an aging build process off our shoulders. I always hear horror stories about web services that are paralyzed by an ancient infrastructure. Fortunately, Cushion is young enough that we can afford to improve ours, which will continue to save us time, money, and stress.