Statamic at SPIEGEL Scale (Part 2)

There is no such thing as a one-size-fits-all solution

There is no one perfect platform or architecture for every website. Just like a wrench will not fix all broken things in your home, fried chicken is not suitable for all meals, and the color hot pink (probably) does not belong on all walls. Your content is unique. Your company is unique. You are unique.

Over the last seven years we’ve taken a hard and opinionated stance on how content should be structured. We don’t believe in the WordPress-style approach of jamming all content into a database as a giant blob of text. Instead, we highly encourage and enable a unique, bespoke content model for every website.

Titles, subtitles, images, content blocks, summaries, excerpts, video embeds, code snippets, related content, and on and on — we believe it should all be broken up, structured, and stored in an object model instead of one big ol’ blob. This approach enables you to adapt and stay flexible in how you want to present and organize your content for the world. This separation of content and presentation has many, many benefits.

Flat files make the very best permanence layer.

You cannot beat the stability and flexibility of storing data in flat files. They can be backed up, compressed easily, are highly portable, can be version controlled, accessed on any platform, and manipulated in batch by any number of external tools and languages. The only real downside to flat files is performance. Reading, writing, parsing, indexing, and manipulating thousands or millions of files is not a small task, especially if done on the fly while rendering a dynamic request. That’s literally why databases were invented more than 50 years ago.

Here Statamic has been poised, at the crossroads between flexibility and performance, not willing to sacrifice one for the other. So we blazed a new trail into uncharted territory.

What if you didn’t have to choose between flat files and databases? What if you didn’t have to pigeonhole your entire site into a relational database to handle a large dataset with a good control panel? What if you wanted to version control some of your content and configuration but have millions of entries to manage? What if you wanted to use No-DB, or run a headless CMS on your own server?

Statamic 3’s cornerstone feature is an adaptable architecture

We thought those very same things and SPIEGEL helped us realize we already had most of the answer.

Statamic 3 has an end-to-end abstraction layer that allows you completely swap out the storage mechanism in a “driver” like fashion, allowing you to connect to just about anything as a data source. Let’s unpack this with some examples.

If you have a lot of small bits of relational data — like an ecommerce system with thousands of products, manufacturers, parts, and variants, you likey want to use a relational database like MySQL or Postgres handle the complex many-to-many relationships. Fantastic, now you can with Statamic. Grab an Eloquent driver, tweak it to your needs, map a fieldset (aka blueprint in v3) to match your table schema, and you’re off and running.

Perhaps you have a huge content store of articles, like SPIEGEL. Write a MongoDB, ElasticSearch, or Firebase driver to your needs.

The control panel will pull your data, populating the control panel as usual through the driver’s query builder, and push it back out as you desire. Essentially, the entire control panel only cares about JSON-in and JSON-out.

If you want to use Statamic’s front-end engine (Antlers), our tags take advantage of the same abstractions and will work exactly the same way you’re used to with Statamic v2. Swap drivers, populate your data store (whatever that is), and your {{ collection:articles }} tag keeps doing its job.

Don’t want to use Antlers? You’re already in a Laravel app — write a few controllers & routes and render your site with Blade. Or Twig. Or push it into a Vue or React SPA. With Statamic 3 you can go full headless mode and take whatever approach you need.

Still flat file by default

Out of the box, Statamic 3 will behave in the same way you may love today, storing its content in YAML front-loaded Markdown files and consuming data from the Stache compiled from them. It works better than ever and you can worry about scaling later. This abstraction is ready for you if and when you need it.

So how does SPIEGEL do it?

Curious about how SPIEGEL’s stack works? Here’s their high level approach, keeping in mind this is running on v2. Upgrading to v3 streamlines even more of their stack.

• A huge flat file content store organized with each entry inside its own subdirectory (you can’t have more than 10k files inside a single directory, an operating system limitation)
• The content store is using a cloud storage solution.
• Statamic’s control panel pushes entries into the content store, and then into an ElasticSearch instance via message queue.
• ElasticSearch runs as a content API with blazing fast response times and is consumed by the control panel and front-end
• The front-end is built in Go.
• User accounts are connected with an Office 365 Active Directory integration with OAuth.

Statamic, now in many-sizes-fits-most

Statamic is no longer a one-size-fits all solution. It adapts to many sizes, making it a great fit for an increasingly wider set of challenges. It can be a standalone control panel, a headless CMS, a backend for your Laravel application, a simple markdown editor for your company blog, and many other things in between.

Statamic 3 isn’t finished yet, but we’re happy to start talking to teams interested in running larger sites with Statamic. André Basse from SPIEGEL has even volunteered to be available to share their team’s experience with Statamic. I’m happy to put you in touch if you’re seriously interested (email me at jack@statamic.com for an introduction).

Statamic can be a standalone control panel, a headless CMS, a backend for your Laravel application, a simple markdown editor for your company blog, and many other things in between.

Our next milestone is Alpha 2, which will be an invite-only small group of folks who are very experienced with v2. If you’ve never used Statamic before, Alpha 2 is not for you. Tire-kickers can jump into the open beta when that’s ready.

We encourage you to jump into the #v3 room in our Discord chat server to continue this conversation and explore the future of possibilities. We’re excited to see where things grow from here, and hope you are too. 😁