Digital infrastructure
Make the right platform choices. Automate what you can. Prepare for failure.
Navigating the digital realm can be challenging sometimes. We rely on so many tools and so much infrastructure. How can we tame this beast, make sure it serves us, and keep it from messing up our lives?
My current formula is, more or less:
- Don't let the tool choose you
Ignore the fancy features; shop for what you actually need - Don't believe the hype
New tech === new problems - Check development activity and community involvement
- Understand the motivations and (commercial) interests of the developer(s)
- Anticipate your next moves if the software were to disappear / be assimilated by Big Tech
- Server setup and maintenance
- Software installation and configuration
- Code and content deployments
- Build and regression tests
- Monitoring and error reporting
- Automatically sync your devices / servers
- Sane local + external backup strategy
- Version history of important files
- Encrypt your data to prevent theft and spying
- Practical access management (don't lock yourself out)
Getting this right does take continuous effort, but in the end it gives me great personal freedom and peace of mind.
If that's something you need as well, then you might be interested in setting up a server in a day together. If you're just curious why I bother with all this, then read on.
Server in a day
Reclaim your privacy, personal freedom and peace of mind.
If you want to host your own software or website, then let's set up a server together.
The objective is to work out your preferences, create a playbook to automate configuration and set up the server.
After our session, you'll be able to manage (or recreate) the server from your local computer. How much software we'll be able to install will depend on their complexity and system requirements. Recurring costs for hosting and maintenance depend on software and provider chosen.
Software choices
For a number of reasons, I moved away from many cloud services over the years, and adopted a number of self-hosted alternatives (see Transitions below). This doesn't always make things easier (and sometimes not even cheaper), but the gains in freedom alone I feel have been well worth it. That's also not to say that paid / closed source software is automatically bad or evil (on the contrary: I'm happy to pay for good, trustworthy software), but merely that setting up your infrastructure exactly how you need it to work will often lead you to free software choices. Free as in speech that is, and most of the time also free as in beer.
Pricing traps
Most online services are designed as a funnel. They lure you in with attractive free tiers, but the features you really need are in the Pro subscription. Sure, in the beginning a few of these 3$ per month subscriptions don't really break the bank, but they can start adding up after a while if you keep signing up for other services.
Plus: there's no telling when and how these services will change their business models in the future. Evernote once limited its sync option to 2 devices overnight. Trello limited the use of "powerups" to a meager 1 after being acquired by Atlassian. Hardly empowering.
Technical limitations
Next to being hidden behind a paid tier, some options might simply not be available for a given service. If you're a slightly more advanced user, then chances are that you're outside of the scope of many services. They prefer to serve simplicity, but what you're looking for is flexibility.
One good example is interactions with other platforms. A lot of time, this is either a paid option, or a non-existent one. Such a feature might also not align with the business interests of some services.
Service interruptions
And then there's the catch that all of these services require an internet connection, which for me wasn't a certainty any longer after leaving NL. I needed my files and tools to be available offline, as much as possible.
The online service itself can also fail. We've seen some major platform outages over the years already. Because access to these platforms is centralized, a technical glitch on their side can severely disrupt your daily operations. Although these services obviously have failsafes for that in place... To you, they are still a single point of failure.
Freedom and privacy concerns
It's often said that you pay with your privacy for a lot of "free" online services. In response, many people reply along the lines of "but that's OK, I have nothing to hide". Sure, if you're not a political dissident, then it might not seem so troubling to share your private data with the big tech giants, and thereby the worlds governments through backdoors and selling of data. But by giving our consent for goverments and business to spy on us, we allow ourselves to be manipulated by them.
The way most data is used nowadays, is to influence our decision making in order to make more money. As long as we're behaving like good consumers, the system works seemingly flawlessly. Digital life is easy. But is it really serving our best interests? Did we really gain anything from scrolling down our Facebook timeline again (ever wonder why it keeps on scrolling?) or could we have spent that time better?
Political forces use this influencing power too, to nudge our thoughts into the direction of certain ideologies (and Big Tech happily takes their money for it). Once we're inside an echo chamber of messaging that aligns with our beliefs, these beliefs get amplified. We start defending them when "attacked" by unbelievers, or even end up doing the attacking ourselves. The nuances slowly disappear from the debate, and we become polarized and biased.
This cognitive bias is again amplified by the fact that different viewpoints are deliberately hidden from our timelines and search results. Sometimes by algorithms deciding what's best for us, but also because opposing views are quickly labeled nowadays as "disinformation" or even "conspiracy theory". Now, there absolutely is a ton of rubbish and insanity floating around online. Let me be clear on that. But that is being used as an excuse to suppress inconvenient opposition to the mainstream narratives. Take the lab leak story during the pandemic. Consistently labeled as fake in the beginning, but as it gained in function credibility, it was eventually pickup by mainstream media and higher political echelons. Regardless if it's true or not, that is a remarkable twist. How much more information is currently being similarly twisted? Or is not reaching us at all? Shouldn't we have access to all the information available, and be allowed to make up our own minds without algorithmic interference?
For this, and all the other reasons, I decided to opt-out.
We don’t have to ask for our privacy, we can take it back.
Transitions
Below are a few digital migrations that I went through. To avoid becoming repetitive: I can recommend all the mentioned apps, except the ones I moved away from (unless stated otherwise). And in most cases, the service being replaced can be viewed as synonymous to a lot of similar cloud services.
Yeah, for all the reasons mentioned above. Proton is a great example of a paid online service still being a trustworthy alternative. The mobile app and web interface work well, and a lot of their software is open source. Calendar and Files functionalities are still young, but the basics are there. Go on, migrate. You won't regret it. And I'm not being paid to say that ;)
Nextcloud mostly serves as a Google Docs / Google Drive alternative for me. Sharing files with other people. But it offers much more: calendar, tasks, mail, chat.. Nextcloud is not without its quirks, but given the amount of integrations available under one (private) roof, you should definitely consider it of you're in the process of degoogling.
More and more alternatives are popping up for Google Analytics. Matomo has been around for a while and offers a very decent, hassle free experience. Smooth updates, easy to navigate, clear descriptions within the UI. Good software.
And recently Plausible popped up as a more light-weight alternative. Has all the basic features, and none of the bloat.
Another Google giant that many of us can't live without. And this one is hard to work around. Not everyone can write mapping software, let alone host map data as a service. Thankfully, alternatives exist. Leaflet is an open source Javascript library for interacting with maps, people all over the world provide geographical data through the Open Street Maps project (OSM), and Mapbox is a commercial provider for the map tiles (the actual chartography / satellite imagery needed for displaying the map; other providers available). Implementation has a learning curve, but for me, it is an enjoyable one. I'm really learning how to work with geospatial data, instead of just how to work with Google Maps.
Or any other cloud storage service. Their business model is obvious: make people depend on it, and then start charging when the free storage runs out. Nothing wrong with that, but there are alternatives which provide more freedom and privacy. For me, Syncthing stands out as the Suiss army knife of file synchronization. It's reliable, well maintained and straight-forward to operate. It does 1 thing, and does it very well.
I used it for a long time, and still collaborate with a few people in Trello. But after it was bought by Atlassian, it limited the amount of "power-ups" to a measly 1. I did not appreciate this rude interruption of my workflow and found an open souce alternative in Kanboard. Although much less visually appealing than Trello, Kanboard offers a purist, distraction free kanban experience. I also needed the database to be readable offline, so I could access my tasks without an internet connection. The biggest downsides of Kanboard are that syncing the database is rather impossible if you're accessing it with multiple users, and that it lacks a good mobile app. But overall, it served me well for many years, and still does today.
I used to be a big fan of Evernote and was a paid user for some time. I moved away because the quality of the apps was getting worse, there was no Linux app, no Markdown support, and I wanted my notes to be actual files on my computer.
This led me to TagSpaces, and various Markdown editors (Epsilon is good on Android). I ended up with Obsidian for note taking, and Syncthing for, well, syncing.
I have to say, I don't really hold any grudges against Mailchimp. Their free tier is generous. Their functionality is great. Their documentation is too. And technically, email can be really challenging to work with. So if it works for you, mainly in terms of pricing, then stick to it!
But if you want cheap.. Like: really cheap... Then take a look at Sendy. Unlike Mailchimp (or any other mailing platform), this software is a one time purchase, with the option to upgrade for a small fee (every 2 years or so). You hook this up to Amazons Simple Email Service (SES), which allows you to send tens of thousands of mails for pennies (works great for transactional mails too btw, through an SMTP relay). Templating in Sendy is horrible compared to other platforms, so you'd probably want to achieve that some other way.
For anyone experienced with Zapier, this may sound familiar: the functionality is great, their pricing is not. To have more than a few automations zapping around, you pay big bucks. This service is clearly not aimed at us small fry.
What a joy it was then, when I discovered n8n. This is short for Nodemation, an open source automation tool based on NodeJS. Install it yourself, or use the hosted version, and hook up some of the 200+ available integrations with one another. But the best part is that, if you know a little Javascript, you can customize your own tasks as well. Capture data from any webhook, evaluate it, and send it to any desired end point. n8n is exactly what we always wanted Zapier to be.
When I was introduced to Airtable in a project, I was impressed. But again... One feature that was crucial to our project, was hidden behind a steep paywal. Which for us, as an NGO, was a no-go.
This was my first encounter with "no code database" software. And of course, there are many other tools out there providing similar functionality. Some are open source, and a few are really good. NocoDB is one of those, gaining a lot of traction recently.
This roughly translates to my web development journey. First, just upload your files via FTP (gone are those days). Then came remote server access with a command prompt, which would let you tweak the server also (and mess things up). Then the bundling of these commands into shell scripts, so you didn't have to remember each command (and mess things up as much). Then came the realisation that shell scripts are instruments of the devil, only leading to things blowing up bigger, followed by the embracing of a tool that sane internet people use to protect themselves from themselves (Ansible). Now I can safely deploy anything, at any time. Even while having that victory beer already.
This is something I'm still exploring, but Matrix is an important technology to mention. It has the potential to take back control of our communications from Big Tech, without getting siloed into yet another fancy app. Matrix is a decentralised technology, meaning anyone can host their own instance on a server and control who has access to it. Data can then be federated between servers, if desired, turning it into a worldwide network. I suggest you read up on it first on their website, and then try out one of the compatible clients for yourself. The most mature and full-featured client is Element, which shares many similarities with Slack. Except, of course, a 10.000 message limit.
I always loved using Pocket, and I can still recommend it. It's under the wings of Mozilla now, which upholds much higher privacy standards than most Big Tech counterparts. Pocket is a read-later app for web content, or a "save button for the internet", as they call it. Very convenient, and it even worked on my E-book reader.
So when I found Wallabag, it had big shoes to fill. But Wallabag has pretty much all the same features, a mobile app, and an import function for Pocket articles. Great developer and community support too. So.. There you go. Shoes filled, by a kangaroo!