Category Archives: SystemD

Linux, MariaDB, MySQL, SystemD

MariaDB 10.1 and SystemD start issue

I’ve always compiled my software on servers from source. Call me a perfectionist / idiot, whatever, but I do it for several reasons, including making sure I’ve got the latest features and security patches, as well as being able to tailor the software I use on servers so it’s a minimal install of the software, still providing what I require.

Anyways, I’ve just found a quirk with getting MariaDB to starting up. As far as I could tell from the provided systemd .service file, it was being run by the user that owned the data directory I’d configured at compile time, this directory was specified correctly in the my.cnf, and when I tried to run the service manually it worked. but something was blocking it when it came to running it via systemctl.

I was receiving the following messages:

Sep 25 22:37:58 centos7 systemd: Starting MariaDB database server...
Sep 25 22:37:58 centos7 mysqld: 2016-09-25 22:37:58 139993404594304 [Note] /usr/local/bin/mysqld (mysqld 10.1.17-MariaDB) starting as process 7978 ...
Sep 25 22:37:58 centos7 mysqld: 2016-09-25 22:37:58 139993404594304 [Warning] Can't create test file /usr/local/mariadb/data/centos7.lower-test
Sep 25 22:37:58 centos7 mysqld: 2016-09-25 22:37:58 139993404594304 [ERROR] mysqld: File './mysql-bin.index' not found (Errcode: 30 "Read-only file system")
Sep 25 22:37:58 centos7 mysqld: 2016-09-25 22:37:58 139993404594304 [ERROR] Aborting
Sep 25 22:37:58 centos7 systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE
Sep 25 22:37:58 centos7 systemd: Failed to start MariaDB database server.
Sep 25 22:37:58 centos7 systemd: Unit mariadb.service entered failed state.
Sep 25 22:37:58 centos7 systemd: mariadb.service failed.

I was inspecting the provided mariadb.service file, trying to spot anything that might indicate why the user couldn’t write to a folder it owned, when I came across this:

# Prevent writes to /usr, /boot, and /etc
ProtectSystem=full

So there you have it. I’d configured it to have the data directory under the mariadb program directory under /usr. MariaDB was allowing me to configure a data directory at compile time that it knew it wouldn’t be able to boot the daemon and write to (if I used systemd, which, lets face it, is becoming the norm, at-least on RedHat based distros). There’s a lot of guides on the net that as an example have the data directory created under /usr/local/mysql/data. This obviously isn’t an excuse. Using Javascript as an example there’s a lot of bad guides on the internet that produce quick hacks rather than quality long term code.

I actually both understand and agree with the systemd setting especially when you take into account the Filesystem Hierarchy Standard (FHS)

Possible Solutions:

  • Move the data directory to somewhere else. According to the FHS this should lie under /var/lib/mariadb
  • Set the ProtectSystem value to false

I’d recommend the former

Fedora, Fedora 15, Linux, MongoDB, RedHat, Shell Scripting, SystemD

Installing MongoDB on a Linux distro using SystemD instead of inittab

1 Comment

While trying to get MongoDB working on a Fedora 15 test server I found that there was no way of installing the latest version with SystemD control due to a pre-existing known bug.

To Mongo’s credit there is a file provided in the source under rpm/init.d-mongod, however when it comes to this being used by SystemD, well it just errors.

This was frustrating, but as I was doing it on a VM, I did what any logical person would do. Installed Mongo from the package manager, took a copy of the required SystemD files, reverted to a snapshot, then installed the latest version of Mongo from source.

After this I adapted the mongod.service file to use the correct locations, ensured all the required directories, files and users were present on the system then started the service et voila, working latest version of MongoDB on Fedora 15 with SystemD

the two required files are:
/lib/systemd/system/mongod.service

[Unit]
Description=High-performance, schema-free document-oriented database
After=syslog.target network.target
 
[Service]
Type=forking
User=mongod
Group=mongod
PIDFile=/var/run/mongodb/mongod.pid
EnvironmentFile=/etc/sysconfig/mongod
ExecStart=/usr/local/bin/mongod $OPTIONS run
 
[Install]
WantedBy=multi-user.target

/etc/sysconfig/mongod

Or you can just use the RedHat distro based install script I created:

#!/bin/sh

# MongoDB Version
MONGODB_VER='2.2.2'

# Get all the dependencies up to date
yum -y update
yum -y install scons gcc-c++ glibc-devel

# Get the source
cd /usr/local/src/
wget http://downloads.mongodb.org/src/mongodb-src-r$MONGODB_VER.tar.gz
tar xfz mongodb-src-r$MONGODB_VER.tar.gz
cd mongodb-src-r$MONGODB_VER

# Compile and Install
scons all
scons install

# Create the SystemD dependant files
echo '[Unit]
Description=High-performance, schema-free document-oriented database
After=syslog.target network.target
 
[Service]
Type=forking
User=mongod
Group=mongod
PIDFile=/var/run/mongodb/mongod.pid
EnvironmentFile=/etc/sysconfig/mongod
ExecStart=/usr/local/bin/mongod $OPTIONS run
 
[Install]
WantedBy=multi-user.target' > /lib/systemd/system/mongod.service

echo 'OPTIONS="--quiet -f /etc/mongod.conf"' > /etc/sysconfig/mongod

# Setup the required user and group
useradd -r -U mongod

# Setup the required directories
mkdir -p /var/run/mongodb/
mkdir -p /var/log/mongo/
mkdir -p /var/lib/mongo/
chown mongod:mongod /var/run/mongodb/
chown mongod:mongod /var/log/mongo/
chown mongod:mongod /var/lib/mongo/
chmod 0755 /var/log/mongo/
chmod 0755 /var/run/mongodb/
chmod 0755 /var/lib/mongo

# Start the new service and enable it on boot
systemctl --system daemon-reload
systemctl start mongod.service
systemctl enable mongod.service