SMcli issue
Hello.
I have stor2rrd setup with ~50 different NetApp storages registered.
Every 1-2 weeks SMcli gets corrupted here and I have to clean up it completely and install again.
Vendor told me that SMcli can't work the way then it runs in 30-50 instances at a time. Most probably, it works with files with no any locks system, so at some moment (pretty rare in other circumstances) two SMcli processes can modify set of files in wrong sequence.
After that SMcli returns error 12 every time.
I tried steps from your wiki, it doesn't help.
I have these recommendations from NetApp. If you want to implement them or design you own queue/lock system over SMcli - it would be great.
For now I'm thinking about some kind of wrapper over SMcli with internal queueing.
NetApp options:
I have stor2rrd setup with ~50 different NetApp storages registered.
Every 1-2 weeks SMcli gets corrupted here and I have to clean up it completely and install again.
Vendor told me that SMcli can't work the way then it runs in 30-50 instances at a time. Most probably, it works with files with no any locks system, so at some moment (pretty rare in other circumstances) two SMcli processes can modify set of files in wrong sequence.
After that SMcli returns error 12 every time.
I tried steps from your wiki, it doesn't help.
I have these recommendations from NetApp. If you want to implement them or design you own queue/lock system over SMcli - it would be great.
For now I'm thinking about some kind of wrapper over SMcli with internal queueing.
NetApp options:
- Split the SMcli requests into batches on separated systems (as we discussed already )
- Restrict the job so you are polling only one array at a time but every 5 seconds so its serial not parallel
- Lastly we have a new product called SANtricity Web Services which uses REST API calls is designed for this type of job. Consider moving to this product for the new array types to achieve your aims. https://mysupport.netapp.com/site/products/all/details/eseries-webservices/downloads-tab/download/62313/4.1/downloads, https://mysupport.netapp.com/documentation/productlibrary/index.html?productID=62313, https://exchange.nagios.org/directory/Plugins/Hardware/Storage-Systems/SAN-and-NAS/NetApp/NetApp-SANtricity-Plug-2Din-for-Nagios/details
- In fact you can use test REST API’s directly to the arrays web server
And:
- create additional SMcli scripts pointing to a different data dir. The SMcli reads and writes multiple files in the data dir
- Create new data dir. The default data dir is /var/opt/SM. I went with the pattern /var/opt/SM/N where N is 1 to 50.
- Then copy the SMcli to SMcliN. Modify the SMcliN file and change the “-Ddevmgr.datadir=/var/opt/SM” to “-Ddevmgr.datadir=/var/opt/SM/N”.
- If they wanted to keep using the “-n” option with the CLI (specify array by name) they would either need to copy the emwdata_v04.bin file from /var/opt/SM to /var/opt/SM/N or add the array or arrays they want to use with the SMcliN command (SMcliN -A Address1 Address2).
- If they used the IP or domain name addresses of the storage arrays in the CLI command, they wouldn’t need to do this last step: SMcliN 10.0.0.1 10.0.0.2 -c "show storageArray healthStatus;"
- Any number of these could be created. Each one would use less than 100KB. This is the quick approach to see if it would resolve the issue.
- If it does, this could be made such that a new SMcli script is modified to parse the -n parameter for the array name, look for the existence of a /var/opt/SM/ARRAY_NAME directory and create it if it doesn’t exist and copy the emwdata_v04.bin file, and then set the devmgr.datadir path to this directory.
Comments
-
Hi Sergey,
you can use REST API instead of SMcli. It is supported since stor2rrd version 2.80.
Refer it here: https://www.stor2rrd.com/NetApp-monitoring.php#Eseries -
Great! Thank you.
Howdy, Stranger!
Categories
- 1.6K All Categories
- 48 XORMON NG
- 25 XORMON
- 153 LPAR2RRD
- 13 VMware
- 16 IBM i
- 2 oVirt / RHV
- 4 MS Windows and Hyper-V
- Solaris / OracleVM
- XenServer / Citrix
- Nutanix
- 7 Database
- 2 Cloud
- 10 Kubernetes / OpenShift / Docker
- 124 STOR2RRD
- 19 SAN
- 7 LAN
- 17 IBM
- 3 EMC
- 12 Hitachi
- 5 NetApp
- 15 HPE
- Lenovo
- 1 Huawei
- 2 Dell
- Fujitsu
- 2 DataCore
- INFINIDAT
- 3 Pure Storage
- Oracle