This page provides information about managing nodes in the cluster, including the following topics:

Viewing a List of Nodes

You can view all nodes using the node list command, or view them in the MapR Control System using the following procedure.

To view all nodes using the MapR Control System:

• In the Navigation pane, expand the Cluster group and click the Nodes view.

Adding a Node

To Add Nodes to a Cluster

1. PREPARE all nodes, making sure they meet the hardware, software, and configuration requirements.
2. PLAN which services to run on the new nodes.
3. INSTALL MapR Software:
   • On all new nodes, ADD the MapR Repository.
   • On each new node, INSTALL the planned MapR services.
   • On all new nodes, RUN configure.sh.
   • On all new nodes, FORMAT disks for use by MapR.
   • If any configuration files on your existing cluster's nodes have been modified (for example, warden.conf or mapred-site.xml), replace the default configuration files on all new nodes with the appropriate modified files.
4. Start ZooKeeper on all new nodes that have ZooKeeper installed:

   service mapr-zookeeper start

5. Start the warden on all new nodes:

   service mapr-warden start

6. If any of the new nodes are CLDB and/or ZooKeeper nodes, RUN configure.sh on all new and existing nodes in the cluster, specifying all CLDB and ZooKeeper nodes.
7. SET UP node topology for the new nodes.
8. On any new nodes running NFS, SET UP NFS for HA.

Managing Services

You can manage node services using the node services command, or in the MapR Control System using the following procedure.

To manage node services using the MapR Control System:

1. In the Navigation pane, expand the Cluster group and click the Nodes view.
2. Select the checkbox beside the node or nodes you wish to remove.
3. Click the Manage Services button to display the Manage Node Services dialog.
4. For each service you wish to start or stop, select the appropriate option from the corresponding drop-down menu.
5. Click Change Node to start and stop the services according to your selections.

You can also display the Manage Node Services dialog by clicking Manage Services in the Node Properties view.

Formatting Disks on a Node

The disksetup script is used to format disks for use by the MapR cluster. Create a text file /tmp/disks.txt listing the disks and partitions for use by MapR on the node. Each line lists either a single disk or all applicable partitions on a single disk. When listing multiple partitions on a line, separate by spaces. For example:

/dev/sdb
/dev/sdc1 /dev/sdc2 /dev/sdc4
/dev/sdd

Later, when you run disksetup to format the disks, specify the disks.txt file. For example:

/opt/mapr/server/disksetup -F /tmp/disks.txt

The script disksetup removes all data from the specified disks. Make sure you specify the disks correctly, and that any data you wish to keep has been backed up elsewhere.

If you are re-using a node that was used previously in another cluster, it is important to format the disks to remove any traces of data from the old cluster.

Removing a Node

You can remove a node using the node remove command, or in the MapR Control System using the following procedure. Removing a node detaches the node from the cluster, but does not remove the MapR software from the cluster.

To remove a node using the MapR Control System:

Before you start, drain the node of data by moving the node to the /decommissioned physical topology. All the data on a node in the /decommissioned topology is migrated to volumes and nodes in the /data topology.

Run the following command to check if a given volume is present on the node:

maprcli dump volumenodes -volumename <volume> -json | grep <ip:port>

Run this command for each non-local volume in your cluster to verify that the node being removed is not storing any volume data.

1. In the Navigation pane, expand the Cluster group and click the Nodes view.
2. Select the checkbox beside the node or nodes you wish to remove.
3. Click Manage Services and stop all services on the node.
4. Wait 5 minutes. The Remove button becomes active.
5. Click the Remove button to display the Remove Node dialog.
6. Click Remove Node to remove the node.

If you are using Ganglia, restart all gmeta and gmon daemons in the cluster. See Ganglia.

You can also remove a node by clicking Remove Node in the Node Properties view.

Decommissioning a Node

Use the following procedures to remove a node and uninstall the MapR software. This procedure detaches the node from the cluster and removes the MapR packages, log files, and configuration files, but does not format the disks.

Before Decommissioning a Node Make sure any data on the node is replicated and any needed services are running elsewhere. If the node you are decommissioning runs a critical service such as CLDB or ZooKeeper, verify that enough instances of that service are running on other nodes in the cluster. See Planning the Deployment for recommendations on service assignment to nodes.

To decommission a node permanently:

Before you start, drain the node of data by moving the node to the /decommissioned physical topology. All the data on a node in the /decommissioned topology is migrated to volumes and nodes in the /data topology.

Run the following command to check if a given volume is present on the node:

maprcli dump volumenodes -volumename <volume> -json | grep <ip:port>

Run this command for each non-local volume in your cluster to verify that the node being decommissioned is not storing any volume data.

1. Change to the root user (or use sudo for the following commands).
2. Stop the Warden:
   service mapr-warden stop
3. If ZooKeeper is installed on the node, stop it:
   service mapr-zookeeper stop
4. Determine which MapR packages are installed on the node:
   • dpkg --list | grep mapr (Ubuntu)
   • rpm -qa | grep mapr (Red Hat or CentOS)
5. Remove the packages by issuing the appropriate command for the operating system, followed by the list of services. Examples:
   • apt-get purge mapr-core mapr-cldb mapr-fileserver (Ubuntu)
   • yum erase mapr-core mapr-cldb mapr-fileserver (Red Hat or CentOS)
6. Remove the /opt/mapr directory to remove any instances of hostid, hostname, zkdata, and zookeeper left behind by the package manager.
7. Remove any MapR cores in the /opt/cores directory.
8. If the node you have decommissioned is a CLDB node or a ZooKeeper node, then run configure.sh on all other nodes in the cluster (see Configuring a Node).

If you are using Ganglia, restart all gmeta and gmon daemons in the cluster. See Ganglia.

Reconfiguring a Node

You can add, upgrade, or remove services on a node to perform a manual software upgrade or to change the roles a node serves. There are four steps to this procedure:

• Stopping the Node
• Formatting the Disks (optional)
• Installing or Removing Software or Hardware
• Configuring the Node
• Starting the Node

This procedure is designed to make changes to existing MapR software on a machine that has already been set up as a MapR cluster node. If you need to install software for the first time on a machine to create a new node, please see Adding a Node instead.

Stopping a Node

1. Change to the root user (or use sudo for the following commands).
2. Stop the Warden:
   service mapr-warden stop
3. If ZooKeeper is installed on the node, stop it:
   service mapr-zookeeper stop

Installing or Removing Software or Hardware

Before installing or removing software or hardware, stop the node using the procedure described in Stopping the Node.

Once the node is stopped, you can add, upgrade or remove software or hardware. At some point in time after adding or removing services, it is recommended to restart the warden, to re-optimize memory allocation among all the services on the node. It is not crucial to perform this step immediately; you can restart the warden at a time when the cluster is not busy.

To add or remove individual MapR packages, use the standard package management commands for your Linux distribution:

• apt-get (Ubuntu)
• yum (Red Hat or CentOS)

For information about the packages to install, see Planning the Deployment.

You can add or remove services from a node after it has been deployed in a cluster. This process involves installing or uninstalling packages on the node, and then updating the cluster to recognize the new role for this node.

Adding a service to an existing node:

The process of adding a service to a node is similar to the initial installation process for nodes. For further detail see Installing MapR Software.

1. Install the package(s) corresponding to the new role(s) using apt-get or yum.
2. Run configure.sh with a list of the CLDB nodes and ZooKeeper nodes in the cluster.
3. If you added the CLDB or ZooKeeper role, you must run configure.sh -R on all other nodes in the cluster.
4. If you added the fileserver role, run disksetup to format and prepare disks for use as storage.
5. Restart the warden

   % service mapr-warden restart
   

When the warden restarts, it picks up the new configuration and starts the new services, making them active in the cluster.

Removing a service from an existing node:

1. Stop the service you want to remove from the MapR Control System (MCS) or with the maprcli command-line tool. The following example stops the HBase master service:

   % maprcli node services -hbmaster stop -nodes mapr-node1
   

2. Purge the service packages with the apt-get, yum, or zypper commands, as suitable for your operating system.
3. Run the configure.sh script with the -R option.
4. When you remove the CLDB or ZooKeeper role from a node, run configure.sh -R on all nodes in the cluster.

Setting Up a Node

Formatting the Disks

The disksetup script is used to format disks for use by the MapR cluster. Create a text file /tmp/disks.txt listing the disks and partitions for use by MapR on the node. Each line lists either a single disk or all applicable partitions on a single disk. When listing multiple partitions on a line, separate by spaces. For example:

/dev/sdb
/dev/sdc1 /dev/sdc2 /dev/sdc4
/dev/sdd

Later, when you run disksetup to format the disks, specify the disks.txt file. For example:

/opt/mapr/server/disksetup -F /tmp/disks.txt

The script disksetup removes all data from the specified disks. Make sure you specify the disks correctly, and that any data you wish to keep has been backed up elsewhere.

If you are re-using a node that was used previously in another cluster, it is important to format the disks to remove any traces of data from the old cluster.

Configuring the Node

The script configure.sh configures a node to be part of a MapR cluster, or modifies services running on an existing node in the cluster. The script creates (or updates) configuration files related to the cluster and the services running on the node. Before performing this step, make sure you have a list of the hostnames of the CLDB and ZooKeeper nodes. You can optionally specify the ports for the CLDB and ZooKeeper nodes as well. If you do not specify them, the default ports are:

• CLDB – 7222
• ZooKeeper – 5181

The script configure.sh takes an optional cluster name and log file, and comma-separated lists of CLDB and ZooKeeper host names or IP addresses (and optionally ports), using the following syntax:

/opt/mapr/server/configure.sh -C <host>[:<port>][,<host>[:<port>]...] -Z <host>[:<port>][,<host>[:<port>]...] [-L <logfile>][-N <cluster name>]

Example:

/opt/mapr/server/configure.sh -C r1n1.sj.us:7222,r3n1.sj.us:7222,r5n1.sj.us:7222 -Z r1n1.sj.us:5181,r2n1.sj.us:5181,r3n1.sj.us:5181,r4n1.sj.us:5181,r1n1.sj.us5:5181 -N MyCluster

Starting the Node

1. If ZooKeeper is installed on the node, start it:
   service mapr-zookeeper start
2. Start the Warden:
   service mapr-warden start

Renaming a Node

To rename a node:

1. Stop the warden on the node. Example:

   service mapr-warden stop

2. If the node is a ZooKeeper node, stop ZooKeeper on the node. Example:

   service mapr-zookeeper stop

3. Rename the host:
   • On Red Hat or CentOS, edit the HOSTNAME parameter in the file /etc/sysconfig/network and restart the xinetd service or reboot the node.
   • On Ubuntu, change the old hostname to the new hostname in the /etc/hostname and /etc/hosts files.
4. If the node is a ZooKeeper node or a CLDB node, run configure.sh with a list of CLDB and ZooKeeper nodes. See configure.sh.
5. If the node is a ZooKeeper node, start ZooKeeper on the node. Example:

   service mapr-zookeeper start

6. Start the warden on the node. Example:

   service mapr-warden start

Maintenance Mode for Nodes

You can place a node into a maintenance mode for a specified timeout duration. For the duration of the timeout, the cluster's CLDB does not consider this node's data as lost and does not trigger a resync of the data on this node.

To put a node into maintenance mode, use the following command:

maprcli node maintenance -timeoutminutes <minutes>

Specify a timeout in minutes with the -timeoutminutes option.

To take a node out of maintenance mode before the timeout expires, follow this process:

1. From a terminal, use the following command:

   maprcli node maintenance -timeoutminutes 0
   

2. Restart the Warden:

   service mapr-warden restart
   

Limitations A node that is running both the CLDB and MFS services cannot be put into maintenance mode. You can shut down the CLDB service on the node provide it is a secondary CLDB node or High Availability for CLDB is enabled.