SQL Woes

For the most part, managing my web server is pretty straightforward, especially because I don’t really get a ton of traffic. Its mostly just keeping things up to date through standard channels.

Occasionally I have a bit of a brain fart moment. I recently was doing regular Linux updates on the server. I noticed a message I had seen before about some packages being held back. Occasionally I will go through and update these, because I am not real sure why they are being held back, but don’t really see any reason they should be.

Then MySQL broke.

So I went digging in some logs and searching for solutions, and decided I needed to roll back the version. Following a guide I found, I discovered… I had done this before, which I now vaguely remebered. Because the old .deb file was still there from last time I broke it.

Anyway, this didn’t fix it, MySQL still was not launching.

I decided that maybe it was time to just switch to MariaDB, which I believe is the spiritual successor to MySQL. And the process was simple enough, I would not even need to dump my Databases. So I uninstalled MySQL, installed MariaDB and… It worked!

Then it stopped working.

I restarted the SQL service and it worked!

Then it…. Stopped working… Again…

So I checked logs again and corrected some issues there and again it worked, then a half hour or so later it stopped working.

One thing I had come across in troubleshooting the original MySQL issue was that there was a command, mysql_upgrade that needed to be run to change how some tables are configured. I couldn’t do that before because I couldn’t even get MySQL to run. But I could get MariaDB to run at least for a bit, and had successfully gotten this upgrade command to run.

So I decided to, once again, try MySQL again, so I uninstalled MariaDB, and purged everything out, rebooting a few times to be sure. And MySQL would not even install anymore, so more purging, this time, including the Databases.

One thing I was glad I had decided to do, “Just in Case” when MariaDB was “working” was dump the databases out with backups. I was glad I did at this point. So with absolutely everything purged, MySQL installed and was working.

I set about recreating the databases from the dumps, and while I was at it updated all the passwords, since I had to recreate the user accounts used by WordPress anyway.

And now everything is working smoothly again.

A couple of links that were actually helpful in solving my problem.



Fixing Cron Not Executing

Recently I encountered an issue I hadn’t run into before. Specifically, my Cron Jobs were not running. Everything seemed correct and I could manually run the commands at the CLI. I’ve had some issue before with getting things to run because I wasn’t using the complete path for programs but this seemed to be something different.

The problem I found was that the root password needed to be changed. Running the following:

sudo  grep CRON /var/log/syslog

Would output a long list of the same issue repeating over and over.

May 27 10:30:01 Webserver CRON[12943]: Authentication token is no longer valid; new one required
May 27 10:39:01 Webserver CRON[12978]: Authentication token is no longer valid; new one required
May 27 10:39:01 Webserver CRON[12977]: Authentication token is no longer valid; new one required
May 27 10:40:01 Webserver CRON[13049]: Authentication token is no longer valid; new one required

When running the following command:

 sudo chage -l root

Would output something like:

Password expires               : never
Password inactive              : never
Account expires                : never

Which suggests the root password has never changed. So I ran the following command:

sudo passwd root

And set a new root password (which was the same as the old root password) and suddenly everything started working again. It felt like a really odd issue, especially considering I didn’t actually change the password, and as far as I could tell I had a root password. Plus the password wasn’t set to expire at all.

Anyway, I wrapped it off by doing an (optional) truncation of the system log. Since the file had become unwieldingly huge with the following.

sudo truncate -s 0 /var/log/syslog