Tue 26 January 2021
tags: update lab nextcloud
Before I begin, I want to point out that Nextcloud itself has not been the issue with 99% of my problems with the install. As it turns out, most of my issues were because of oversights on my part or suspected hardware failure. I haven't rectified the hardware issue yet, but I've hopefully learned from those oversights, as you'll see later on. Again, this blog isn't a critique of specific products, just a record of what I've tried and how I've failed.
For some reason, it took me months to work up the courage to actually get started on the Nextcloud server, and then it took me another month to fix it after it broke, and then... Let's just say that I procrastinated on this for no good reason, shall we? I ended up pulling out my ECS Liva X2, a mini-PC with a Celeron and 2GB of memory, for the project. This is actually the same computer I used for my DNS/DHCP server a while back before that completely failed on me. It's been sitting in its box ever since, and I figured that a seldom-used web-based storage platform was a decent use for something with its specs. In fact, my Liva is the first thing I got for my lab over 4 years ago, and Nextcloud was the original use for it. However, I made several mistakes along the way that would add up, and the hardware choice was one of them.
The initial installation went smoothly. I installed Debian 10 with LVM partitioning, set up Apache with a web cert, and installed Nextcloud using their command line instructions. I knew the 32GB of storage wouldn't last long, but I wasn't concerned - I have two 1 TB HDDs that I got for the original Nextcloud server, although this time I wouldn't be running them in RAID 1. I did make a small mistake here, which I thought would be bigger. I mounted the drive with an fstab entry, which was fine. However, then I tried to use it as the standard data directory for Nextcloud. When I did so, I messed up something with the config, and so it just wouldn't use the drive at all. On top of that, for some reason I only had about 100 MB of storage available on the 32 GB eMMC drive. I figured I had misconfigured Nextcloud, so I resolved to start over.
The second time around, I kept the Debian and Apache installations and simply wiped out Nextcloud and re-installed it. I had taken long enough that version 20 had come out, so I also got a nice upgrade. After dropping the old database info from MariaDB, I once again followed the command line installation instructions provided by Nextcloud. However, this time I simply enabled the "External Storage" module within Nextcloud and connected my 1TB drive as a local drive. It was a bit cruder, but it also went smoother - until I once again ran out of storage on the eMMC drive. What gives? At first I re-attempted what I had done before, which actually kind of worked. But then disaster struck.
Going back a bit in time, the primary reason that my DNS server failed a year and a half ago is that the config file for BIND just... disappeared. The file itself existed, but there was no data in it. I got frustrated and returned to using my router's built-in DNS and DHCP setup instead. This should have been a warning. The exact same thing happened to my Nextcloud setup. The core config file just got wiped clean, and with no backup yet I couldn't restore it. I couldn't find the reason for the file being cleaned out like that, so at this point I'm chalking it up to hardware failure. Somewhere, there is a (or several) bad block on the unit, and it loses whatever is stored there. Maybe that's incorrect, but I'm admittedly a bit tired of fiddling with the Liva.
However, during all of that analysis I found my second large error - my LVM partition setup. When I told Debian to install with LVM partitions, I hadn't thought about the partition sizes. I just let it automatically allocate space as it thought it needed. Because of this, the /var partition was tiny, and since that's where I had my Nextcloud setup, it ran out of space pretty quickly. I have temporarily resolved this by plugging in a 128 GB flash drive and expanding the logical volume, but I don't want to rely on that. Hence, other hardware.
Since I don't want to trust the internal storage of the Liva any longer, I'll most likely be selling it off for cheap. It would work fine as a HTPC or maybe just a hobby computer, but not for anything mission-critical. Nextcloud will be moved to a VM on my ESXi host instead - it's still as unused as it was in my last post. I'll also spin up a second server to use as an Nginx reverse proxy, since I have a few other web-based services I'd like to test out. After that, I'll look to expand my lab by using the ThinkCentre Tiny that I have, and then my Asus VivoPC will be last (if it's used at all).
For now, future plans are thus:
Re-do Nextcloud from the ground up on a ESXi VM
Set up an Nginx reverse proxy on ESXi
Upgrade my ThinkCentre M72e for use as a second ESXi host
Actually set up infrastructure backbone (DNS, DHCP, PXE, etc)
Set up a small git server
I'm hoping it all works out a little bit better this time, but we'll see!
Also, a small life update: I got a cat. Her name is Eowyn .