Our blog

Most of our attention regarding websites is around creating new ones, but we cannot escape the reality of having to deal with retiring old or existing websites as well. After facing this scenario a few times at Miranj we built a tool to handle the end-of-life stage for websites. This tool attempts to crawl all publicly accessible URLs for a site and create a static archive, much like the Internet Archive’s Wayback Machine. It has a few built-in smarts to detect and capture the many different ways assets are referenced in modern markup. It generates a folder structure that retains clean URLs, as well as intelligently handles URLs with query parameters.

While the tool is not yet publicly available, we have been using it internally and some of the static archives generated by the tool of erstwhile CMS-based sites have been running smoothly in the wild for years.

email hidden; JavaScript is required if you’re interested in archiving a content-heavy site of your own. Always happy to help or exchange ideas around digital archival¹.

In the vast, multi-layered subject area of web performance, server response time is an important metric. From a CMS standpoint, however, it is one of the most significant. Best practise recommends a 200ms or lower time-to-first-byte. For medium to high-traffic sites, server load is another vital statistic.

We were invited to talk about our learnings from optimising and hosting high-traffic sites such as Guiding Tech at the sixth edition of the Bangalore Site Speed Meetup.

Bangalore Site Speed Meetup 6^th Event on YouTube.

In this talk we go over caching as a broad performance strategy, before dive into micro-caching as a specific approach to handle loads of 10 to 100 concurrent requests per second. We cover the filter, storage, and invalidation implementations¹ of this caching strategy in Nginx. The talk concludes by comparing metrics of our caching strategy against a target of achieving sub-200ms TTFB response times for all visitors.

This talk is a newer revision of the Fortifying Craft CMS for High Traffic talk from 2019² which, as it says on the tin, was specifically tailored for Craft CMS based workflows. In this edition we focus on just micro-caching as a strategy and look at it independent of any specific CMS running at the application layer.

As a web studio, we have always been focused on our core craft of designing and developing websites. Hosting would often be an afterthought and we’d typically suggest clients to go for a shared web host. After all, shared web hosting was inexpensive and did not require much technical oversight. We would guide our clients through the purchase process, deploy our code and bring our engagement to a closure.

This approach worked fine for a few years, but over time we started noticing several drawbacks:

1. Server administration is a bit of a blind spot for clients. Some of them have even suffered website data loss because they overlooked renewal reminders.
2. Our faith in shared hosting was depleting. Long support wait times, poor performance, being unable to reach our server while some other site on the same shared-host was experiencing a DDoS attack, etc. were frequently souring our experience. Further, the lack of control over server configuration severely limited our ability to install tools or fine-tune the server to meet modern performance benchmarks.
3. Modern VPS providers were steadily decreasing prices while matching or eclipsing shared host offerings. They provided better access to hardware and high control over the software.
4. Hosting technologies have become more complex in the last decade. It’s no longer just about the hardware (and bandwidth) specifications advertised by the web hosting services. A modern hosting strategy needs a holistically approach considering several aspects such as SSL renewals, reliable outgoing emails, caching, CDN, backups, software updates, and more.

By 2018 these challenges had become important enough for us to actively seek alternatives. We came across many different approaches — unmanaged VPS servers, managed servers, app hosting solutions, etc. However, none struck the right balance between —

• Extending full server control
• Ease of server management
• Reliable, high-frequency backups
• Costs

Eventually, we decided to get our hands dirty. Based on our experience of setting up the server architecture for Guiding Tech (which receives lots of traffic and high rate of updates), we slowly put together a hosting solution which features:

• High-performance LEMP stack with FastCGI micro-caching
• Automated server-side image compression and optimisation
• Multi-tiered backup strategy — server snapshots, local backups and offsite backups
• Automated monitoring of uptime and server vitals

We launched in 2018, and in the two years since we’ve been providing a highly performant and reliable hosting service to our clients based on the above architecture. Under the hood, we use Digital Ocean VPS nodes and reinforce the software to deal with heavy loads and traffic bursts. Digital Ocean’s developer-friendly infrastructure (and their community documentation) has played an important role in our journey to offer high-quality website hosting. Through this post, I’m happy to also share that we’ve recently joined the Digital Ocean Solutions Partner Program. This brings us even closer to the Digital Ocean community. 

If you’d like to learn more about how we’ve scaled inexpensive VPS hardware to serve hundreds of requests per second, check out Prateek’s talk at Dot All 2019. If you’d like to discuss more, email hidden; JavaScript is required.