Spectrum Scale uses several logical networks for its communication.
We use the term logical network to make clear that these are functions that may or may not use different physical networks, depending on configuration.
Daemon interface and network
The Spectrum Scale daemon, mmfsd, runs on every active Spectrum Scale node, listening on port 1191 on the network interface designated to be its daemon interface. We will call the network to which all the daemon interfaces are connected the daemon network.
The port used for daemon communication can be changed from 1191 to some other value by changing the tscTcpPort parameter. If this is done, any remote cluster mounts will need to specify the tcpPort value to the mmremotecluster command, along with the usual contact nodes. The mmsdrservPort parameter typically be changed as well, to match tscTcpPort.
Some administrative commands require additional ports to receive responses. By default, ephemeral ports are used. But the range to be used can be explicitly set with the tscCmdPortRange parameter.
Within a Spectrum Scale cluster, network traffic must be able to flow between the daemon network interfaces of any two cluster nodes. The subnets setting can be used to choose another network after initial communication has been established.
Administrative interface and network
Every Spectrum Scale nodes has one of its network interfaces designated to be the administrative interface. The network to which the administrative interfaces are connected will be called the administrative network.
Some Spectrum Scale commands require operations outside the scope of the Spectrum Scale daemon — such as the mmaddnode command (to configure a node as part of the cluster) or the mmstartup command (to start up the Spectrum Scale daemon). These operations are conveyed using the ssh protocol, and the administrative network carries this ssh traffic. Historically all Spectrum Scale nodes needed to be able to connect to all other nodes in the cluster using the ssh protocol, and many people still believe that is true. However, generally Spectrum Scale is configured in a mode whereby only some nodes (those on which administrative commands will be issued) need to be able to initiate ssh connections to other nodes.
By default, the administrative interface of a node is the same as its daemon interface.
In many cases this is acceptable, but if necessary the administrative network can be a separate network, set using:
mmchnode --admin-interface=HOSTNAME -N NODENAME
The question then arises: since that administrative network can be made a separate network, should it be separate? This network is only lightly used, so separating the administrative network will not noticeably improve the performance or stability of the cluster. However, the security organizations of some enterprises may require that all systems listen for incoming ssh connections only on a particular (secured) network, which might not be the best network for data traffic. In these cases, it makes sense to change the administrative network to comply with the requirements of the security organization.
Parallel File Systems 