MySQL can use any file system to store its data. Using the Direct Access NFS features of MapR to provide the storage for MySQL delivers several advantages over the basic local file system configuration:
- The metrics database can utilize the capacity of the MapR cluster, not just the local Linux file systems. Of course, the administrative capabilities of MapR still enable rational limits to be placed on the size of the metrics database.
- The database will be automatically replicated, providing a more reliable storage solution in the event of an individual disk drive failure.
- In the event of node failure, the MySQL data is still available. Another node can be easily configured to re-host the metrics service.
Assumptions and PrerequisitesIf the MySQL server is not yet installed, you’ll need to identify the server you wish to use and install the proper packages (
mysql-clienton Ubuntu systems,
mysqlon RedHat/CentOS systems). MapR software does not have to be installed on the server running MySQL, although having the basic
mapr-clientpackages will simplify some of the following configuration steps. You must have a copy of the
/opt/mapr/bin/setup.sqlscript (whether from the
mapr-*packages installed on the MySQL server node or copied from another node in the cluster).
soci-mysqlpackage from the EPEL Repository as well; you’ll need to configure the EPEL repository specification on your server before attempting the mysql-server installation.
Setting Up the Metrics DatabaseNow we’re ready to set up MySQL. The basic details of setting up the metrics database can be found in Setting up the MapR Metrics Database in the MapR Documentation.
We will make the following changes to the setup procedure in order to store MySQL data on the MapR cluster.
- Create a new volume in the MapR cluster to store the metrics data
- Ensure that the volume is mounted to the MySQL server system via NFS. (This step has been completed above, and we assume the cluster is mounted at
- Configure the MySQL instance to use that location for its data.
Creating a volume is simple using the following
maprclicommands. Ideally, the following commands should be executed on the server that has the MySQL software installed so as to properly identify the user
mysqlto the MapR cluster.
If you have a running MySQL instance, you’ll need to shut it down to make the necessary configuration changes. Any metrics data for MapR jobs that run during this brief shutdown will not be logged.
maprcli volume create -name mapr.mysql -user mysql:fc \ -path /var/mapr/mysql -createparent true -topology / maprcli acl edit -type volume -name mapr.mysql -user mysql:fc
/etc/mysql/my.cnfon Ubuntu or
/etc/my.cnfon RedHat/CentOS) and update the
datadirspecification to match the path specified for the MySQL data volume above. Replace
<cluster>with the name of your cluster.
Set the ownership of that directory to match the default data location. Failing to do this will result in a failure to launch the database.
datadir = /mapr/<cluster>/var/mapr/mysql
If your server had already instantiated a MySQL database, you’ll want to move those data files from the local file system to the new /mapr location. A simple
chown --reference=/var/lib/mysql /mapr/<cluster>/var/mapr/mysql
cp -rpcommand will do. For example:
Otherwise, you’ll need to initialize the new directory for the database instance with the command
cp -rp /var/lib/mysql/* /mapr/<cluster>/var/mapr/mysql
The database can be started with the command
service mysqld start
$MAPR_HOME/bin/setup.sqlscript. Those steps are the same, whether the data is stored on a local file system or stored on the cluster.
And you’re done! If everything went successfully, you can confirm that the MapR Control System GUI can access the Metrics database and that any MySQL data you copied onto the cluster is recognized.
Improving High Availability of the Metrics DatabaseSo what happens if the MySQL server fails? How will you recover so that MapR Metrics can continue logging data? As a best practice, you can save the MySQL configuration file
my.cnfto the cluster as well. In the event that the server node running MySQL fails, you can recover the metrics functionality by doing the following.
- Configure a new server with the MySQL server software.
- Copy the archived
my.cnffile into place on the new server.
- NFS-mount the cluster file system on that new server.
- Start up the database instance.
- Reconfigure the cluster nodes with the new metrics specification and restart the
hoststatsprocess on those nodes.
$MAPR_HOME/server/configure.sh -R -d <new_MySQL_srvr>:3306 cd $MAPR_HOME/initscripts maprcli node services -name hoststats -action restart -nodes `hostname`
/mapr/<cluster>/var/mapr/configuration/default/confand updating the
metrics-hostfield. Changes to the
metrics-hostfield will be pulled down to the cluster nodes automatically within a few minutes. You will still need to restart the
hoststatsdaemon once the configuration files have been downloaded to the cluster nodes.