Categories
LostInThought/SlightRanting

Nextcloud as NAS replacement

Today for the second time in a few weeks I noticed my computer’s fans were at high speed when I had no other app that one for remoting into somewhere else. Immediately I knew this must be Nextcloud doing its CPU-heavy syncing which I confirmed by opening Activity Monitor on my Mac.

Nextcloud for some reason trips up and deletes all of my files without warning. I know there’s a setting to actually warn if I’m syncing more than X at a time, but when many of the files I frequently use are more than X, and every other files fits under X, there’s no point checking it up.

Luckily, in addition to syncing to Nextcloud, I’m also using a Synology unit which has a version explorer that allows me to recover deleted and/or modified files. This has saved me for the second time this month and has just made me think twice about continuing using Nextcloud.

Here’s why:

Synology’s DSM has access over NFS, SMBv1-v3, AFP, in addition to Nextcloud’s complicated WebDAV link, Synology’s is much easier to remember. Windows computers in the domain will even automatically mount users’ home folders in A:\ and a shared resource on B:, both on the Synology unit use a single address and it becomes available on the web GUI, and in several other places, including Nextcloud itself. While true, you can access the system hosting Nextcloud over any of these protocols, there’s no telling what’s going to happen if you do, and you manage to do it correctly over the right shares, over the right permissions, over the right everything. There are too many variables and not enough documentation.

Synology’s client syncs fast. My personal files is not that big, maybe 15GB. Synology Drive takes a few minutes to complete synchronization. Nextcloud takes days.

As far as I know, Nextcloud has no way to recover deleted files.

Nextcloud has an impressive set of apps that keeps growing fast, but so does Synology. It’s not as impressive as Nextcloud’s but somehow they are feel much cohesive, faster and reliable. Case in point: my Nextcloud database is hosted on a Synology unit.

I’d like to point out many more but frankly there’s only one that I think matters the most and I think is the only consideration those looking into Nextcloud should take: Synology’s DiskStation Manager handles disks, Nextcloud is unaware of disks.

As such, if you want to keep your data on redundant storage (I’m not talking backup, just redundancy) you must know about Linux disk administration, and be proficient at managing things like LVM and whatnot which I think are horribly complicated things to manage. Even is much easier, and I’m saying this as some who only has a vague idea about ZFS. All of this makes Nextcloud better suited to run as a VM, not as an independent system. Running Nextcloud on a VM lets you have redundancy and HA at the hypervisor level, assuming you have the tools. Is isn’t cheap, but it’s easier than the first-party option.

Running Nextcloud as a VM comes with many of the same issues too. First of all, unless it’s run as a Docker instance in a RAID-aware system that has easy disk administration, like unRAID, you will still have storage issues. For instance: being a VM means its virtual disks bust be given something like a dedicated datastore with managed redundant disks to thick provision it and be done with it. If it’s thin provisioned, it’s likely to be migrated from time to time to reallocate resources. This won’t be an easy feat because of Nextcloud very nature of storing files, disk are expected to be huge. A word of containers; containers are harder to understand, harder to manage and harder to configure than VMs. Good look deploying DNS-based LetsEncrypt on a container if you’re behind a proxy that won’t allow you to get HTTP validations.

Being a VM disk also makes up for huge potential data waste, for instance, if a Nextcloud user decided to voluntarily (versus spontaneously vanishing data) take his/her things off of Nextcloud suddenly a big chunk of space would become available on the VM but not on the storage backend because VM disks cannot be shrunk once the grow, at least not on the most popular virtualization platforms, like vSphere and Hyper-V.

Nextcloud’s data could be stored using a remotely mounted system on the Nextcloud host for Nextcloud, but that’s a lot of the same again: you must be a Linux filesystems/permissions wiz to know how to correctly do it. Perhaps be joined to a directory to have UIDs/GIDs sync, IDK.

So, assuming you’re OK with randomly losing data and having your computer partly frozen because of a sync zeroing in your lost files, and then partly frozen from the sync from reuploading your files (assuming you have a way to recover, which should’ve been Nextcloud’s role in the first place), if you have no user-friendly disk management, none of Nextcloud features matter.

Lastly there’s the workaround option: mounting external storage directly in Nextcloud. The best method is probably SMB: if you sign in directly on Nextcloud, it’ll use Kerberos –provided it’s configured correctly– to authenticate with storage server which in turn do its own authentication to let you users in so they won’t need a password. That’s best case scenario though. If you’re using some form of Single Sign-On like ADFS, authentication claims don’t get mapped into Kerberos tickets, so you’ll have to reauthenticate to the external storage. You can let your users do it own their own and save the passwords for later, but when it’s time to change passwords they wouldn’t know what to do. Finally WebDAV. Very slow. The real question here would be: if you’re using another server to store stuff that probably has tricks of its own, Synology’s ffice (or whatever it’s called) for instance is a very robust suite, what’s the point of Nextcloud?

So, what are the options. First Nextcloud Enterprise.

My first thought, but, it starts at EUR1900. It’s astronomically high and you must have at least 50 users which I do not, even SMBs don’t. You can get a NAS unit for that that comes with inclusive support for the life of the unit. A EUR1900 NAS unit can definitely support more than 50 users and can even do virtualization and not only support domain-joining but being an actual Active Directory domain controller—not LDAP, full-fledged Active Directory. And you’d still be left with about EUR1200-1400 to get some disks and a couple of cache NVMe sticks to put in your unit. For EUR1900 you could even get a rack-mounted unit and worry about the disks later since it will be able to rebalance them on the fly as you need to add space. I’m speaking of Synology here because it’s the only widely available NAS products I have first hand experience, but I hear offerings from brands such as QNAP and whatever-Asus-NAS-line-is-called can offer more flexibility, though not as robust support.

Fun fact: middle tier and up Synology products can run Docker and host websites complete with reverse-proxy both serving HTTP/2 sites, so technically you can run Nextcloud on DiskStation Manager–twice.

Secondly, what exactly are they willing to support? Nextcloud is something you install on your choice Linux variant, so will they walk you through every possible setting? Or will they do it remotely like the people at Synology does?

I once had an issue with a Synology unit (power outage at the wrong moment) were the partition tables got screw up. It was only a wrong move before having complete data loss. I got in contact with Synology’s through the built-in tools (because the system still managed to boot) and remotely they recovered access to the data, mounting temporarily the filesystems so I could copy files off the unit and rebuild the array.

You can even activate an agent that will facilitate them tunneling in with a temporary key you give them if you don’t know your way around your firewall. Unlike Nextcloud, Synology’s tools are not open source, well… most aren’t. But they are very upfront and transparent about what their goals are and they are generally trustworthy. They also have a steep entrance price.

I’ve also come across a few instances where I have noticed documentation for Nextcloud purposely hidden, yet the open source work is still taken from user input. Configuration of SAML is one of these, another is High Availability and/or Clustering. As an open source organization, hiding away documentation to me seems shady. Highly. But, I want to believe in Nextcloud so I’ll leave it at that.

What if Nextcloud was distributed with an OS? I’m not saying to lock it down and scare away developers and innovation, nothing like that. I’m saying distribute it in such a way that Nextcloud isn’t just a web app but disk-aware system, that has guidelines for something like a superocc (from the file occ, for lack of a more imaginative name) a daemon or framework or whatever you call it that dictates/manages how the things should behave reconfiguring/fixing common areas essential for the system’s proper function and scalability either up (e.g; running bare-metal, adding a disk to its array and have Nextcloud manage it and/or create/rebalance disk array) or down (e.g; running virtual, letting the hypervisor know it’s using so much disk space and it’s OK to shrink, perhaps work in tandem with hypervisor to redistribute blocks if needed) and how easy this should be for non-tech savvy users who have never touched a CLI.

So here’s an idea. How about having an entrance fee that not a yearly EUR1900. I’m not saying start charging per feature (even those thought to be only needed in the biggest of the Enterprise sector, like SAML, HA, branding/theming) or to keep our private clouds “with the latest updates”, two of the most cringeworthy statements that public cloud providers seemingly think are their selling points when it’s usually the least appealing thing they could advertise and something that could definitely harm open source contributions to Nextcloud; becoming controlled, specially remotely by forcing updates. I’m talking about about having low entrance fee, something symbolic that maybe would be worthless by the count of one user alone, but if all of us contribute to it, it becomes… IDK, how big the reported installed base is, millions?

Probably enough to have Nextcloud delivered as a whole package that’s installable bare-metal, as a VM or (by having “superocc” guidelines) ported into another base distribution that still lets your users locally or lets Nextcloud Support remote in to fix things easily because they will always know what to expect.

You’d still have Nextcloud Enterprise Support, hopefully at a much more palatable price and it would be more like a mission critical scenario, as for the rest of us have (low) incident-based pricing. Since Nextcloud systems would be deployed to specifications, it would be trivial to support them and hence maintain a low pricing, it could even be automated.

By keeping a symbolic price, guidelines for system recovery, an open base system (where users can dnfapt or xyz at their leisure) users wouldn’t feel alienated and stop contributing to Nextcloud and to keep pricing low have superocc warn the users or the aspiring developer/super tinkerer that continuing doing a certain modification would break superocc guidelines resulting on its inability to recover therefrom and impeding the Nextcloud instance to continue to receive support.

You could still post in an article ways to recover manually, and have superocc go through a checklist of all its needed permissions and files so not all is ended for the user. Both parties win: you don’t lose completely revenue from low-cost incidents, and the users may feel 1, a little empowered fixing something on their own yet not too much and 2; in gratitude towards Nextcloud supporting them. I know I’ve felt this several times using Synology’s support and each time I can feel strengthens my brand loyalty for them when they make what I deem impossible happen.

I think this would allow users to continue to trust Nextcloud as all of these preventions would work in-system yet they would not block the user to fuck up the system if he/she chooses to ignore a warning. The system, despite being delivered as a whole by Nextcloud, would be still the users’ to tinker with.

Nextcloud could make a little money from everyone without appearing as an untrustworthy motherfucker company to the little users and the developers, possibly delivering even superior support to enterprises to which the same support tools would apply. Since this imaginary superocc would be like the configuration mastermind it could be made not only system-aware but also cluster-aware and ease support in the Enterprise sector…or home users with one too many arrays.

The one thing I have not thought through is how this could be charged, I certainly wouldn’t recommend activation nor per-instance pricing as I mentioned earlier, many rely on hypervisor-based HA in the absence of native clustering documentation, even if our instances’ user count is 3 with 1 of those 3 being the local admin account. Although if it’s truly a symbolic pricing, this should not matter.