Our blog

The Pixel & Tonic team were kind enough to invite me back on the Dot All stage¹ to share some of our learnings with a room full of people in Barcelona. This time, I spoke about leveraging some of the power and tools that Craft already ships with — filters, functions, collections, helpers, services, caches, and much more — right from within Twig.

The talk covers three broad areas:

• We start by looking at the benefits of leveraging Craft (and Yii) APIs in general and the ways in which that makes us more efficient developers, and our code more reliable and performant.

• Having established the why, we move on to examining commonly occurring scenarios during the development lifecycle and identifying patterns where we can start falling back to and integrating more of the API.

• In the third and final part, we look at ways to identify the appropriate Twig filters and functions, or Craft helpers, services, and components that can be plugged in these scenarios.

Demystifying the Craft API for Twig Developers by Prateek Rungta on Vimeo.

In order to demonstrate some of these more richly with relatable examples, I created a bare-bones site for a fictional Dot All Music Festival. The code for this site is up on GitHub.

Photographs from the event. 

This talk is based upon my own developer journey using Craft CMS over the years. I also drew from the experience of witnessing first-hand the individual development journeys of my teammates, and spotting common patterns between their paths and my own.

I hope this helps you discover and use more of the powerful tools available to use inside a Craft CMS Twig template file. email hidden; JavaScript is required if you found any of the techniques useful or discovered something new that you could use.

Craft CMS continues to be an indispensable part of our toolbox for building websites here at Miranj. It has gotten even better earlier this year when a host of new content modelling options were added with the Craft 5 release and we’re loving the extra flexibility in creating our data models.

Aside from using Craft to power our projects, Miranj has also released a few utilitarian Craft plugins over the years. We were quick to update most of them with Craft 5 support immediately after the public release, and we have finally gotten around to updating the last (and most complex) plugin as well:

• Contact Form Tuner
• Cryptographer
• Obfuscator
• Router
• Sheets to Tables

In addition to these plugins, we also maintain a public Craft CMS starter project which has also been upgraded with Craft 5 support. Hope you find these useful!

We make no secret of our love for Craft CMS here at Miranj. While we rarely prize a tool or tech over the problem it helps us solve, we greatly value the efficiency of a good tool, and Craft CMS continues to shine on that front. One such aspect is the ease with which one can tap into and extend the core functionality of the CMS. There’s a thriving community of plugins that enhance, boost, and add to the already elaborate set of core features offered by Craft. While we depend on a fair few of them for most of our projects, we have also released some plugins of our own:

• Contact Form Tuner
• Cryptographer
• Obfuscator
• Router
• Sheets to Tables

And as of this week, all five of our plugins have been updated to work with the latest release of Craft 4. Some of these plugins (Router, Obfuscator and Cryptographer) date back all the way to 2015 and Craft 2, and we are glad to continue supporting them and making them available for the community to use on their projects as well.

In the vast, multi-layered subject area of web performance, server response time is an important metric. From a CMS standpoint however, it is one of the most significant. Best practise recommends a 200ms or lower time-to-first-byte (TTFB). That is the time in which a CMS has to figure out what to do with the request, parse templates, query the database, render the HTML, capture it all and send it back as the response.

Craft CMS — our preferred CMS for content-heavy websites here at Miranj — is quite fast out of the box, but as our pages and content models grow in complexity so does the server response time. Add traffic to that mix and it can quickly lead to poor TTFBs and slow overall experience.

In this talk we go through a multi-tiered caching strategy using Craft and Nginx that enables a single VPS to consistently deliver sub-200ms response times, even while handling loads of 10 to 100 concurrent requests per second. It covers our learnings from optimising a low-powered server to handle millions of visitors each month on the Guiding Tech website project. We achieve this by caching the website at two places — at the web server level using Nginx’s FastCGI micro-caching, and at the CMS level using flag-based template caches in Craft CMS. We also factor in real-world edge cases such as bypassing the cache, delivering variations to different visitors, etc, that are necessary to account for in a robust, production-ready system.

Fortifying Craft for High Traffic with Prateek Rungta from Craft CMS on Vimeo.

We also put out a sample Nginx config for the micro-caching strategy discussed in the slides.

This talk was prepared for and delivered at Dot All 2019 in Montréal, Canada. This was my first time speaking at a conference outside India, but any nervousness I carried on stage was quickly dispelled by the warm engagement and wonderful conversations with the Craft community.

Photographs courtesy Pixel & Tonic

I’m extremely grateful to the folks at Pixel & Tonic for extending me an opportunity to present to an international audience, and for placing their trust in us a second time after Souvik’s talk the previous year at Dot All 2018, Berlin.

I email hidden; JavaScript is required about your mileage from adopting any of the caching strategies mentioned in the talk, or any alternate approaches you have implemented to optimise Craft CMS for heavy loads and high traffic.

Back around 2013 with every passing project, we were trying to push ourselves to break down websites into small, atomic information pieces, almost in the object-oriented way. The more we pushed ourselves the more we got disappointed by the website tools (and paradigms) that surrounded us. On the one hand, we felt constrained by WordPress’ pervasive Pages and Posts paradigm. It was (and still remains) so ubiquitous that everyone seems comfortable in imagining websites as a collection of hierarchical pages and an accompanying blog. On the other hand, some tools (like Jekyll) supported atomic content pieces by combining data formats like YAML and Markdown, but fell short in extending a reasonable authoring experience. We had reached a point of dreaming up our own custom CMS architecture but stopped short of implementing it since that would be a significant deviation from our core work i.e. designing and developing content-driven websites.

An year later, while working on the Quicksand website, we were desperately looking for a reliable Content Management System (CMS) that could complement our approach while delivering a good authoring experience. That’s when we stumbled upon Craft — an un-opinionated, content-first CMS. It touted several good features but there were a few noteworthy ones which caught our immediate attention:

Flexible Content Modelling

This one hit the nail right on the head. Craft allowed us to deconstruct content into small pieces and build websites bottom-up. We even learned that our object-oriented approach was called Content Modelling. It freed us from existing content paradigms and allowed us to architect the website content uniquely as per project needs.

Relational Fields

Every independent piece of content modelled inside Craft can link to any piece of content across the website. Relationships are extremely powerful because not only do they help create cross-references and allow people to navigate to related information, but they greatly cut down redundant data capture. Shared pieces of information can now be independently created once and simply linked/​reused in several related places.

Matrix Field

This was around the time when design systems were maturing and the Matrix field complemented a modular component-based page design. It enabled the creation of re-usable blocks/​patterns which could be included multiple times on a page. Further, the blocks could be moved up or down to re-order content within a page.

Live Preview

It’s very reassuring to be able to review how the content will show up on the website before it’s published. And it’s even more powerful if you can see a live preview as-you-type. That’s exactly what Craft shipped and it reminded us of the experimental interfaces demonstrated by Bret Victor in his talk Inventing on Principle.

Clean Separation of Concerns in Code

Developers crave to focus on one thing at a time – data, business logic or templating. An architecture that cleanly separates these three layers makes the code more resilient, error-free and secure. Not to mention it’s also far more enjoyable to work with.

Security

The Craft team has never beaten around the bush about their security measures. They’d been following all the good practices in the book, published zero-day fixes and diligently updated vulnerability databases. We were re-assured that Craft’s security was taken very seriously by its makers.

One-Time License and Developer Support

Craft was not a free CMS (unlike many popular alternatives) but their pricing was very compelling — reasonable for the value it delivered, affordable for a small business and importantly a one-time fee for perpetual use. We saw this as good thing because it assured us that the product had a sound business model and that they’d likely be around for a long time to come. Also, who doesn’t like developer support from the makers of the product.

Zero-byte (empty) output for fresh CMS installation

If this one sounds trivial, trust us, it’s a profound change. When a CMS does not impose a starter theme but instead comes with zero front-end code, it puts the control right back in the hands of the designer and the developer. The CMS landscape had many firmly established players (WordPress, Drupal and the likes) but Craft was a breath of fresh air in this regard.

Not only did Craft check most of our requirements but it also raised our curiosity and excitement. However, like with any new software, we approached our first few Craft projects with extreme caution. It took us about a year to grow confident in Craft’s versatility, and then we were hooked! Quicksand was our first Craft project and it continues to be running stably today.

Fast forward to late 2018. The Craft team rolled out the Craft Partner Network to bring together agencies from around the globe with proven expertise in Craft. With over 4 years of Craft CMS experience (at that time), our team satisfied the review criteria and we became an official Craft CMS Partner — the first agency from India, and among the earliest set of members from the Asia Pacific region.

Over the years Craft has enabled Miranj to take on several information-dense projects with formidable IA challenges. Craft’s feature set has been steadily expanding and today we’re using it to power some fairly complex use-cases. We leveraged its multi-lingual and multi-site capabilities in MIIT and Forech. In fact, in the latter project we hooked up Google Translate to pull in automated translations for French, German and Spanish (thanks to Craft’s extensible architecture). Our client IndiaBioscience is making great use of the multi-user capability with user groups and fine-grained permissions system to collectively manage their large website. We used Craft’s powerful Twig templates to enable Guiding Tech to syndicate content with external services. Craft also has a growing Plugin Store where one can find many useful extensions for the CMS, including a few that we‘ve built and published. And perhaps the most important of all (and dare I say — our favourite feature of Craft CMS) is the thriving community that is growing steadily in India and around the world. The close-knit community is connected over Discord, Stack Exchange, an annual conference (Dot All) and the Craft Partner Network.

With over six years of Craft experience, we are proud to be at the forefront of its adoption in India and the neighbouring geographies. During this journey we’ve participated in every Dot All Conference and have been invited to present our techniques at the 2018 and 2019 editions. Needless to say, Craft CMS has been a great ally in facilitating our purpose and we couldn’t be happier with our decision to adopt this technology.

If you’d like to learn more about Craft CMS and how it compares with some of the other CMSes out there, join me on June 6^th (Saturday) at Content Web — a series of online discussions on content publishing, web design, web development and the business around it.

2017 was a year of some pretty significant shifts inside Miranj. The changes were so fundamental that they affected almost everything we did. The initial results were encouraging, but only now — two years into the experiment — can we fully appreciate the impact. Our first six years were about finding our feet, about survival. We are now starting to find our voice. Here’s 2018, in review.

We worked on eight different projects of varying sizes in 2018. Each project had its own set of challenges, but there were some in particular that stood out.

High-Traffic

We’ve always strived to build performant websites, but we’ve never really had to deal with heavy loads. Performance takes a whole new meaning when a site is accessed by 50 to 100 people every second. This is the challenge that Guiding Tech came to us with. It is amongst the top 11k websites on the entire internet, and its traffic far outpaced any other project we had worked on so far. Guiding Tech gave us the opportunity to double down on and really push the limits of server-side optimisation. We learned a lot about Nginx, caching, concurrency, and were able to make major strides in handling massive loads on low-powered hardware.

We have been able to take the optimisations from this project and apply them far and wide. We also shared some of our learnings with the community at ReactFoo Delhi 2018, and have another talk on performance coming up this September.

@ReactFoo

Languages

There had been a growing itch to work on multi-lingual and non-English projects. I found it more than a little ironical that in a country with 22 official languages and only about 10% English speakers, all websites we were building were in English. 2018, finally, brought the opportunity to break out of the English bubble. We did a Burmese edition site for Myanmar Institute of Information Technology (also our first university website, yay!), and French, Spanish, and German editions for conveyor belt manufacturers Forech. The latter even involved some fancy translations and content-sync on the backend via the Google Translate API.

Five new languages in 2018, but none native to the country we operate out of. We’re still looking for that elusive Indian language web project.

Hosting

We have been believers in separation-of-concerns and anti-vendor-lock-in ever since we started Miranj. This has manifested itself in many different ways — making all content client-editable without developer intervention, sharing the source code along with instructions to host the website anywhere, maintaining full transparency about our collaborators, etc. This philosophy also made us averse to ever hosting a client’s website. We firmly believed that the website and its content were the client’s intellectual property, and should be controlled by them. We would instead always ask clients to sign up directly with a web host, and then deploy the website on those accounts.

This arrangement worked well initially but we started noticing some patterns over time.

• Deployments were getting more complex. Earlier it meant simply uploading a bunch of files to a folder on a fully-managed, shared web host. But now we were dealing with DNS management, CDNs, image processing tools, image optimisation tools, web server level configurations, automated backups, VPS management, security protocols, and more. A lot of different moving parts needed to come together to host a website.
• Software updates to the underlying CMS and plugins. We would want all our deployed sites to pull down these updates, but we were no longer actively engaged on the project, and applying minor point updates felt like too small a quantum of work to re-engage. Yet this was not a trivial task either, because sometimes a minor update could also end up breaking something major or having unintended consequences.
• Critical security fixes. This is the sort of thing we did not account for in our naivety and youth because it felt so rare, but soon realised is actually not that uncommon. Vulnerabilities such as Heartbleed and critical security updates such as Craft 2.6.2982 made us realise that we did not have a contingency plan for delivering these security updates to our clients. We took evasive action in these specific instances, but realised the need for a better fundamental approach.
• We were managing a lot of servers anyway. Our clients would always trust us with access to their servers. They would rarely revoke access at the end of our engagement, and we would often find ourselves fixing things on their servers, either of our own accord, or when they would bring something to our notice. This was all done pro-bono on a goodwill basis, outside any formal engagement.

At some point of time, we realised that while we did not offer hosting as a service, we performed a lot of what it entailed anyway. We also realised that ownership of content does not have to translate into ownership of infrastructure. Or rather, that Miranj can own the infrastructure, but still ensure that clients own all content and IP.¹ Once we crossed that threshold, we found that there were a lot more benefits we could offer to our clients if we formalised this engagement, such as:

• Fully managed infrastructure – SSL, CDN etc. standard and baked in
• Monitoring
• Security best practises
• Performance best practises
• Critical updates
• Robust, regular, automated onsite and offsite backups
• Automatically propagating any fixes for issues discovered across all sites we were managing

We signed our first batch of hosting clients in 2018, and we are grateful to them for placing their trust in us.

📦 Craft 2 Packages

Sometime during early 2018 we learned about a way to use Composer to install Craft 2 plugins.² We were already managing a bunch of different Craft 2 sites, with at least four (dev × 2, staging, production) deployments per site, and adding new ones every couple months. The tedium of manual plugin installation had slowly but surely added up. So we jumped on the opportunity to automate plugin installation and have it under version control.

We adopted Derrick Grigg​’s described method for two ongoing projects. Deployments became smoother. But the manual steps involved in the initial plugin install continued to feel like a chore, more-so now that one piece of the puzzle had been solved. One thing led to another, we went down a few internet rabbit holes, conducted many failed experiments, and finally realised that we could, probably, make plugin installs slightly easier. All we had to do was:

• Create a private Packagist server
• Maintain a human-editable list of Craft plugins
• Figure out ways to make plugins with different folder structures Composer-installable by issuing the standard composer require <vendor>/<plugin> command
• Put in a process to auto-update the Packagist server whenever the plugin list is updated
• Allow the community to contribute and grow the repository of plugins

It was slightly insane, and I’m sure we lost more time doing this than we can ever make up in gained plugin installation efficiency, but we did it anyway. We launched craft2​pack​ages​.miranj​.in and managed to fully automate plugin installation.

My only regret is that we didn’t learn of this possibility sooner. Everything about Craft 2 Packages – inception, experimentation, and release – happened in May. Craft 3 was already out by then (released in April 2018) and most developers were in the process of switching over. While we are committed to keeping this server running for at least the next 10 years, we realise most Craft 2 projects are probably in maintenance mode now and will not go in for major changes. I feel like the community could’ve really benefitted from this had it been around when Craft 2 usage was at its peak.

Dot All

We fell in love with Craft CMS when we built our first Craft powered website in 2014. We’ve built on it extensively in the following years. Craft also has a thriving developer community, largely centred in North America, Europe and Australia. Our interactions with this community were limited to Slack messages, at best. Then in 2017 they announced their own conference, Dot All. We decided to take the plunge and meet the community. Souvik travelled all the way to Portland, USA to attend the conference and put faces to all the Slack handles, GitHub handles, Stack Overflow users, and Andrew Welches (there had to be more than one). It was a wonderful experience and he had a great time.

That was 2017. Here’s Souvik narrating what happened the following year, in 2018 —

Our Craft implementations had matured significantly, especially our homegrown template architecture and code organisation which had been evolving over the last 3 – 4 years. I felt that it would be a valuable technique for other Craft developers and went on to propose a talk for Dot All 2018. A few weeks down the line, the Craft team invited me to take the stage at Dot All in Berlin. This was my first international speaking opportunity — undoubtedly, a pretty significant milestone. I felt the pressure while preparing for the talk and encountered frequent moments of excitement and butterflies-in-stomach along the way. The entire process took about two months of exploration, iterations and internal discussions before the final slides started taking shape a few weeks before the conference.

Prateek and I flew into Berlin a day ahead of the 3‑day conference. Although this was our first visit to Berlin (and Germany) we never felt lost (or alone) thanks to the constant stream of suggestions and recommendations on the Dot All Slack. We kept bumping into fellow attendees everywhere — in the hotel, at the venue, at nearby bars and eateries. It was a lot of fun.

My talk was scheduled for the third (and final) day of the conference. I kept working on my slides from the sidelines and the tweaks continued until about half hour before the presentation. I took the stage right after lunch, slightly nervous but definitely excited. The presentation went very well without any significant hiccups. It was the best version of all the rehearsals I’d done with Prateek and I was quite pleased. More importantly, it was received very well by fellow attendees. Many found it helpful, some had questions, a bunch of them gave us good feedback and asked us to share our code for reference and their understanding.

@DotAllConf

The overall experience was a growing one. We managed to shun our imposter syndrome, present in an unfamiliar culture (and geography) and hold our nerves throughout the experience. It was deeply fulfilling to engage with the Craft community and share our work and experience with everyone.

Lows

While we had a lot of highs, not everything went great, and it would be dishonest if we did not mention the lows in an annual review.

The first setback was the closing down of Design Fabric. The website had undergone a redesign earlier in the year and we took that opportunity to work on its performance as well. Using a bunch of different techniques we were able to make the media heavy Design Fabric, already one of our fastest shipping websites, even faster. We were proudly showing it off to clients and our peers. However the project was taken down shortly after the founder of the publication was accused as a part of the #MeToo movement.

The other major setback was in December. Three weeks into what was meant to be a much longer relationship, a project we were working on was brought to an abrupt halt. We had failed to resolve differences with the client or find common grounds on which to proceed. Having been in business for eight years now, we have encountered a healthy diversity of opinions and personalities. We would often take pride in our willingness and ability to use reasoning and discussion to resolve differences between teams, bridge gaps and move a project forward. This experience reminded us that we still have much to learn. This was the first time we found ourselves at a stalemate.

Recap

Running a business is no easy feat — between chasing new leads, managing ongoing projects, looking after our hosted websites, handling support for older projects, accounting and regulatory compliances, running an office, keeping up with the industry, learning new skills, and doing the actual design and development work to build websites, our team of two rarely finds time to pause and reflect. This review lets us do that; take stock of how far we’ve come, and appreciate how far we have to go.

2018 in Numbers

• Undertook 8 client projects
• Collaborated on 3 projects
• Delivered 5 talks
• Hosted 1 event
• Shipped work in 5 languages
• Plugins: 2 new releases, 1 update release, 1 new Packagist repository
• Worked with/​in 3 countries
• Worked with 4 non-profit organisations
• Cups of coffee consumed: don’t know (we don’t keep track)
• 8 co-workers
• 11 new office plants
• 1 person got married

See you next year!

Back in 2014 we were looking for a Content Management System (CMS) that could fit our concept of a website constructed from a collection of atomic pieces of content. Our search led us to Craft CMS – it was still a fairly new CMS back then but it looked promising. We initially adopted Craft for a project where the website was moved from WordPress. In the next project the move was from Drupal. Both implementations were a learning experience but the outcomes reinforced that we made a good choice. The latter has now grown to be our largest Craft implementation.

Craft CMS makes no assumption about the content. It frees our thinking from the shackles of the WordPress content model (Posts and Pages) while at the same time does not weigh us down under Drupal-like technical baggage. Being able to model content from scratch (almost treating the content model as data) and not being held back by any starter theme helps unlock creative freedom. As a result, we can hand-craft a website’s information architecture, user interface and user experience without the CMS acting a hurdle (or barrier).

Our relationship with Craft strengthened over time and 3 years since our discovery, in 2017, I found myself attending two conferences in the US: Peers (Craft CMS was a sponsor) and Dot All – the first official Craft CMS conference. Little did I know that the very next year I’d be taking the stage at the Craft CMS conference.

Any system that offers high flexibility, in turn, demands a disciplined approach – and that applies to Craft as well. With each passing project executed in Craft, we inched towards a disciplined approach for content modelling and developed a modular templating architecture that can be used in a large number of use cases, especially sites that contain a lot of information. I presented this architecture at Dot All 2018, in Berlin.

The talk walks through how our imagination of a website has evolved over the years and goes on to outline our current approach and utilises these fundamental principles—

• modelling content and giving them a structure (objects)
• separating content (objects) from presentation (views)
• defining presentations (views) based on a clear purpose
• ensuring presentations (views) are evolvable in order to serve a different (or greater) purpose

I further go on to explain the implementation that uses a Routers-Views-Components-Layouts paradigm. It’s amazing how we’ve been able to simplify large and complex websites using this approach – making our work more flexible, scalable and maintainable.

Here’s a video of the full talk —

Architecting a Content Website with Souvik Das Gupta from Craft CMS on Vimeo.

It’s always great to meet the Pixel and Tonic team, and the Craft CMS community who we mostly know through Slack Discord. I got great feedback on the talk, and based on Andrew​’s recommendation went on to convert Craft’s Happy Lager demo site into our templating approach. It’s called Happier Lager and should be a good resource to observe the contrast.

Photographs courtesy Dot All.

It was my first time in Berlin (and Germany) and after the conference, I took some time out to visit a few other cities, which included Munich and the popular Oktoberfest 🍺.

If you use Craft and have any feedback or questions, email hidden; JavaScript is required. I’d also be keen to hear from you if you’re tackling similar challenges and the techniques you’ve adopted.