EMC Monitoring: PowerMax VMAX VPLEX ECS XtremIO PowerStore

Dell EMC² monitoring

PowerMax / VMAX
PowerStore
ECS
DataDomain / PowerProtect DD
PowerScale / Isilon
ME4/5 (WBI)
ME4/5 (SMI-S)
PowerFlex / ScaleIO
SC series (Compellent)
XtremIO
VPLEX
Metro node
Unity
VNX / CLARiiON
VNX file / Celerra
VNXe

XtremIO monitoring is supported by our first generation tool STOR2RRD only so far.

Do you want to let us know your priorities? Vote for your missing features and device support. More votes, bigger priority for us.
List of not supported technologies yet

VPLEX monitoring is supported by our first generation tool STOR2RRD only so far.

Do you want to let us know your priorities? Vote for your missing features and device support. More votes, bigger priority for us.
List of not supported technologies yet

Metro node monitoring is supported by our first generation tool STOR2RRD only so far.

Do you want to let us know your priorities? Vote for your missing features and device support. More votes, bigger priority for us.
List of not supported technologies yet

Unity monitoring is supported by our first generation tool STOR2RRD only so far.

Do you want to let us know your priorities? Vote for your missing features and device support. More votes, bigger priority for us.
List of not supported technologies yet

VNX monitoring is supported by our first generation tool STOR2RRD only so far.

Do you want to let us know your priorities? Vote for your missing features and device support. More votes, bigger priority for us.
List of not supported technologies yet

VNX file monitoring is supported by our first generation tool STOR2RRD only so far.

Do you want to let us know your priorities? Vote for your missing features and device support. More votes, bigger priority for us.
List of not supported technologies yet

VNXe monitoring is supported by our first generation tool STOR2RRD only so far.

Do you want to let us know your priorities? Vote for your missing features and device support. More votes, bigger priority for us.
List of not supported technologies yet

Source of performance data are not storage devices themself but REST API of one of these:

Dell Storage Enterprise Manager
Dell Storage Manager Data Collector

Connection directly to the storage will not work, use one of above options.

User rights options

Create user xormon under admin role or use already existing admin account.

Node (controller): CPU, IO, data, response time
Port: IO, data, response time

Create user xormon under Reporter role and assign him rights to all or specific:
- volume folders
- server folders
- disk folders
In this case will be reported utilization only for assigned folders and their items.

Storage connectivity

Install the Enterprise Manager or DSM Data Collector and attach there your Compellent boxes.
Create user xormon on the Enterprise Manager or DSM Data Collector under admin role.
Unfortunatelly admin role is only the role which can get all performance data from the storage.
If you use web tool Unisphere central for SC series then registration there is not enough.
You have to register storage for user in Enterprise Manager or DSM Data Collector.
Allow access from the XorMon NG host to the Enterprise Manager on port 3033 (HTTPS).
Set "Report gathering settings" of IO usage to 5 minutes.

XorMon NG storage configuration

Add storage into configuration from the UI:
XorMon NG UI ➡ Settings icon ➡ Device ➡ Storage ➡ New ➡ Vendor:device ➡ Dell SC series
On the picture you can see 3 compellent storage in the Enterprise Manager.

Configure every storage you want to monitor, use same storage name alias as you see in the Enterprise Manager.
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI

Storage agent uses eithre the EMC Unisphere as data source, or the storage itself (that method works at least on 5978_711/0365 storage firmware and newer, most probaly even on olders)
EMC Unispheres 1.6.x, 8.x, 9.x and 10.x are supported.
Read why TDEVs are not monitored.

Prerequisites

Allow access from the XorMon NG host to the EMC Unisphere or to the storage itself depends on mode you use on port 8443 (TCP).

EMC Configuration

Unisphere (scenario 1)
- Create user xormon under "PerfMonitor" role on Unisphere (example uses 'stor2rrd' user but use 'xormon' preferable).
- Allow statistics logging on the EMC Unisphere
  UI ➡ Performance ➡ Settings ➡ System Registrations ➡ Diagnostics ➡ On (Diagnostics interval 5 minutes)
  
  EMC Unisphere 1.6
  
  EMC Unisphere 8+
Direct storage attachment (scenario 2)
- Connect on one of both PMAX e-management port and on the wheel on the top right corner
- Click “users and groups”, then “local users”, and then the “create” button
- Enter: The account name, description and Password 2 times (warning: both PMAX and XorMon NG have forbidden special characters and they are not the same)
- Choose the “perfmonitor” role
- Keep right access as “read only”

XorMon NG storage configuration

Add storage into configuration from the UI, fill in Symmetrix ID for both scenarios
XorMon NG UI: Settings icon ➡ Device ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC VMAX
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI

Storage agent uses EMC Data Domain CLI accessed via ssh to get all necessary data from the storage.
EMC DataDomain firmware versions 6.x and 7.x are supported.
EMC DataDomain firmware versions 5.5 is supported only in STOR2RRD.
EMC DataDomain 6.0.2.50+ and 6.1.2.40+ (6.2 is not affected) do not provide disk stats community.emc.com/docs/DOC-78887

Prerequisites

Allow access from the XorMon NG host to the EMC storage on port 22 (TCP).
Create user xormon under user role on the storage (example uses 'stor2rrd' user but use 'xormon' preferable)

Exchange ssh keys
Print public key:

$ cat .ssh/id_rsa.pub
  ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPpQlm0Z/x1X4txd2GgZgvg1Fm7QHeusQ6LqBp/Sb1doAoiDmXErk+lNY27aJAqb4YiGnqejioqCJ9N93NiKpffxH+L3

Log on to the storage and add above key:

$ ssh <data domain host> -l xormon
  xormon@data_domain_host> adminaccess add ssh-keys
    Enter the key and then press Control-D, or press Control-C to cancel.
    ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPpQlm0Z/x1X4txd2GgZgvg1Fm7QHeusQ6LqBp/Sb1doAoiDmXErk+lNY27aJAqb4YiGnqejioqCJ9N93NiKpffxH+L3

Test connection:

$ ssh <data domain host> -l xormon
  EMC Data Domain Virtual Edition
  Welcome to Data Domain OS 6.1.0.11-571028

XorMon NG storage configuration

Add storage into configuration from the UI:
XorMon NG UI: Settings icon ➡ Device ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC DataDomain
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI

Troubleshooting

DD refuses some ssh sessions, you can find in XorMon NG log (~/xormon-ng/server-nest/log/datadomain) this:

Error: Unable to connect to remote machine (dd01) : No such file or directory
Error: Error command: ssh login failed  unable to load Perl module IO::Pty: Can't locate IO/Pty.pm ...

Increase DataDomain parameter login-max-active, xormon ssh sessions are in conjuction with normal backup sessions.

sysadmin@dd(active:0)# adminaccess option show login-max-active.
  Adminaccess option "login-max-active" set to "150".
sysadmin@dd(active:0)# adminaccess option set login-max-active 500
  Adminaccess option "login-max-active" set to "500".

Isilon v7.x, v8.x and v9.x firmwares are supported.
When you use Isilon v9.x then just use one Management IP Address from the Cluster (do not use direct Node's IPs).
The product uses EMC Isilon REST API interface natively provided by the storage to get all necessary data.

Storage connectivity

Allow access from the XorMon NG host to the EMC storage on port 8080.

Create user xormon on the storage with new read-only role called xormon.

$ ssh -l admin 192.168.1.1
isilon01-1% isi auth users create xormon --enabled yes --password-expires no --password XXXXXXX
isilon01-1% isi auth roles create --name xormon --description "Read-only role for XorMon NG"
isi auth roles modify xormon   --add-user=xormon --add-priv-ro=ISI_PRIV_EVENT --add-priv-ro=ISI_PRIV_LOGIN_PAPI --add-priv-ro=ISI_PRIV_STATISTICS --add-priv-ro=ISI_PRIV_SMARTPOOLS --add-priv-ro=ISI_PRIV_DEVICES --add-priv-ro=ISI_PRIV_QUOTA

XorMon NG storage configuration

Add storage into configuration from the UI:
XorMon NG UI: Settings icon ➡ Device ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC Isilon
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI

The product uses EMC ME4 SMI-S and WBI API interfaces natively provided by the storage.

Note: storage firmware GT280R010-01 does not provide data via SMI-S (up to GT280R009-02 it is fine), it is a confirmed issue in the storage firmware (SR #14853424). Upgrade to GT280R011-01 or use WBI access method.

Storage connectivity

Allow access from the XorMon NG host to the storage on ports 80 (WBI) and 5988 (SMI-S).
If you use https and secure SMI-S then test ports 443 and 5989 instead.
If you use storage cmd line configuration like in examples below (you can use the storage UI instead) then port 22 (ssh) must be open.

Make sure that secure SMI-S (port 5989) and WBI HTTPS (port 443) protocols are enabled on the storage

CMD line:

$ ssh -l manage <storage_hostname>
# show protocols
  Service and Security Protocols
  ------------------------------
  Web Browser Interface (HTTP): Enabled
  Secure Web Browser Interface (HTTPS): Enabled
  Command Line Interface (Telnet): Disabled
  Secure Command Line Interface (SSH): Enabled
  Storage Management Initiative Specification (SMI-S): Enabled
  Unsecure Storage Management Initiative Specification (SMI-S 5988): Disabled
  ...

If secure SMI-S or WBI HTTPS are not enabled:

# set protocols smis enabled
# set protocols https enabled

Create user xormon on the storage with monitor and standard roles, allow SMI-S and WBI.
- CMD line method:
  Adjust password to your preferred one from used Test1234.
```
$ ssh -l manage <storage_hostname>
# create user interfaces smis,wbi locale English password Test1234 roles monitor,standard xormon
```
- UI:
  
  (example uses 'stor2rrd' user but use 'xormon' preferable)
Note for newer firmwares role "standard" is necessary for WBI access.

XorMon NG storage configuration

Add storage into configuration from the UI:
XorMon NG UI: Settings icon ➡ Device ➡Storage ➡ New ➡ Vendor:device ➡ Dell EMC ➡ PowerVault ME4 ➡ SMI-S

If you set https in above storage properties then this must be enabled on the storage:

$ ssh -l manage <storage_hostname>
# show protocols
  ...
  Secure Web Browser Interface (HTTPS): Enabled
  Storage Management Initiative Specification (SMI-S): Enabled
  ...

If you use http then this:

  Web Browser Interface (HTTP): Enabled
  Unsecure Storage Management Initiative Specification (SMI-S 5988): Enabled

Do not use shell special characters like @:;#!?|$*[]\{}`"'& in the password, use rather .+-=_% instead.
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI

Data is obtained from the EMC ECS Rest API, from VDC nodes.

Allow access from the XorMon NG host to each EMC ECS VDC node on port 4443
Create Storage Management user "xormon" with System Monitor role on each monitored VDC.
Add storage into configuration from the UI:
XorMon NG UI: Settings icon ➡ Device ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC : Elastic Cloud Storage

The addition of a VDC to ECS configuration is intended as choosing specific VDC you want to display in Stor2rrd GUI.
To monitor your chosen VDC, just set one or two nodes which are management EndPoint for a given VDC.
No need to add every node in VDC
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI

The product uses REST API natively provided by EMC PowerStore storage.

Note the issue with PowerStore firmware 3.0.0.1, upgrade to 3.2.1.0.

Storage connectivity

Allow access from the XorMon NG host to the EMC storage on port 443 (https)
Create a new user in PowerStore Manager UI with Operator role
See this: www.delltechnologies.com/en-us/collaterals/unauth/white-papers/products/storage/h18150-dell-emc-powerstore-manager-overview.pdf
(page 40: it is sufficient to give only Operator privileges to an account which is only responsible for monitoring instead of giving full privileges with the Administrator role)

XorMon NG storage configuration

Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ EMC PowerStore
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI

XorMon NG obtains preformance and configuration data from the PowerFlex Gateway using REST API.

Storage connectivity

Allow access from the XorMon NG host to the PowerFlex Gateway on port 443
Create a new user with the 'Monitor' role
- Login to the primary MDM server as 'root'
- Login to PowerFlex SCLI as administrator
```
scli --login --username admin 
```
- Create new user User will be created with a temporary password which needs to be changed during the first login
```
scli --add_user --username xormon --user_role Monitor 
```
- Login to PowerFlex SCLI with the new user credentials and set new password
```
scli --login --username xormon
scli --set_password 
```

XorMon NG storage configuration

Add storage to the configuration:
XorMon NG UI: Settings icon ➡ Device ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC : PowerFlex
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI

The product uses storage WBI API interfaces natively provided by the storage.
WBI access must be used on new storage models which do not support SMI-S protocol.

Storage connectivity

Allow access from the XorMon NG host to the storage on ports 443.

Make sure that secure and WBI HTTPS (port 443) protocol is enabled on the storage

CMD line:

$ ssh -l manage <storage_hostname>
# show protocols
  Service and Security Protocols
  ------------------------------
  ...
  Secure Web Browser Interface (HTTPS): Enabled
  ...

If secure WBI HTTPS is not enabled:

# set protocols https enabled

Create user xormon on the storage with monitor and standard roles, allow WBI.
- CMD line method:
  Adjust password to your preferred one from used Test1234.
```
$ ssh -l manage <storage_hostname>
# create user interfaces wbi locale English password Test1234 roles monitor,standard xormon
```
- UI:
  
  (example uses 'stor2rrd' user but use 'xormon' preferable)
Note for some newer firmwares role "standard" is necessary for WBI access.

XorMon NG storage configuration

Add storage into configuration from the UI:
XorMon NG UI: Settings icon ➡ Device ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC ➡ PowerVault ME4 ➡ WBI

If you set https in above storage properties then this must be enabled on the storage:
```
$ ssh -l manage <storage_hostname>
# show protocols
  ...
  Secure Web Browser Interface (HTTPS): Enabled
  ...
```
If you use http then this:
```
  Web Browser Interface (HTTP): Enabled
```
Run "Test" for configured storage device, it must show "OK"
Wait about 1 hour, then reload the web browser, you should see it in XorMon NG UI