Autonomic + Collective Tools

Just under a year ago, Collective Tools approached us to ask for support with their infrastructure. Here’s our write-up of this extremely radical anti-competitive partnership so far 🌠

Collective Tools is a multi-stakeholder cooperative, based in Sweden, who provide email and cloud (file/calendar/contact-sharing) services to their user members.

As well as these services, Collective Tools users gain membership in a user forum, and the right to attend annual members’ meetings and have a voice in the direction of the project.

At the time of contacting us, Collective Tools was facing a very uncertain future: their email provider had significantly jacked up prices, and maintenance was becoming increasingly time-consuming – making the difficult job of competing with the likes of Google and Microsoft even harder.

While there’s a lot of overlap between the services that collective.tools offers and our own services as Autonomic (both cooperative.computer, and our bespoke tool hosting for groups), we think it’s both politically and practically important for the world to have as many cooperative digital tool options as possible.

We believe that solidarity and cooperation are our only hope in against the surveillance capitalism tech giants – some of the largest and most influential companies in history.

So, we geared up for a 3-part revolutionary overhaul:

Switching from Ansible to Co-op Cloud for application deployment and maintenance.
Migrating Collective Tools’ apps to a unified dedicated server.
Migrating from a corporate provider to self-hosted email with Mailu.

Ansible to Co-op Cloud

We’ll be writing another post soon about our long journey with Ansible, and why we moved away from it for application maintenance. For the moment, we’ll just say that we spent a lot of time trying to achieve our dreams of a configuration-as-code utopia using it, and we eventually moved away – to a series of lightweight bash scripts for server provisioning, a Python tool for server maintenance, and Co-op Cloud for application deployment and maintenance.

The main advantage, which is very relevant for collective.tools, is that work on the application configurations is collectivised and commoned between members of the Co-op Cloud Federation. That means that collective.tools members benefit from performance improvements, security hardening, and updates, and that all improvements made to the application “recipes” for collective.tools are likewise released in a public, libre, and easily-to-use format. You can see the recipes – which can also be used without Co-op Cloud’s command-line tool, as raw Docker Swarm deployments – for the key apps which collective.tools uses, here:

Unified dedicated server

Collective.tools’ previous approach was to create a separate Hetzner VPS for each group member. While this has a lot of benefits in flexible scaling, and a high level of security isolation between instances, hosting costs were racking up quickly, and the technical overhead of managing a growing number of servers was becoming non-trivial.

Despite putting “cloud” in the name of Co-op Cloud, we have a lot of critiques of the concept of “cloud”, especially the way that it abstracts the material reality of whose computer your stuff is running on, where it’s located, and how it’s managed. So, as well as being cheaper and easier to manage for collective.tools, we’re also happy that moving to a single dedicated server has made a potential migration to a different host in the future easier to reason about, and made the total hardware needs of the collective.tools platform clearer.

Self-hosted email

As we mentioned in our “Infinite Migrations” write-up, we’ve been working with the Mailu mail-server platform for a couple of years now, and collective.tools’ request came conveniently after we’d determined that our platform was solid enough to be offering to paying clients.

Although running an email server has a reputation of being Very Difficult™, Mailu automates enough of the hard parts to free up time for us to handle the rest. As well as lowering costs, running a mailserver provides more functionality than all the corporate email hosts that we’ve found, including:

automation,
advanced filters and auto-replies,
fine-tuned spam filtering,
integration with mailing list and other email software.

Despite a relatively large number of accounts to move across, some lightweight automation using the Mailu API made it pretty quick to set up all the new accounts, andimapsync again made the mailbox migrations smooth.

Lastly, moving across also involved updating the custom Nextcloud app which collective.tools uses to automatically set up mailboxes for Nextcloud users https://gitlab.com/collective.tools/ctautomailhooks – which are released as open source to the community; useful for anyone else using Mailu in combination with Nextcloud.

Conclusion

We’re very happy to be working with collective.tools; their service has been a big inspiration for our own offerings, and doing migration and hosting work at this scale has helped improve our processes and our platform a lot. We’re hopeful that some of the bits we developed for the migration are useful to other folks (they certainly have been to us!)

We’d encourage anyone looking for reliable, ethical online services to check out Collective.tools, and we at Autonomic would also be very happy to hear from anyone who’s looking for similar help making self-hosted infrastructure cheaper and easier to manage.