Http timeout with DS6200

kuznetsovmyu

Hello,
I stopped receiving IO data from my Lenovo DS6200. Error log contains 

Thu Jan  9 12:54:52 2020: Thu Jan  9 12:54:52 2020 command: /usr/bin/wbemcli -nl -t -noverify ei 'http://stor2rrd:Test_1234@10.200.8.59:5988/root/smis:CIM_BlockStorageStatisticalData' timed out after: 700 seconds /var/lib/stor2rrd/bin/dothillperf.pl:2446 : Interrupted system call
Thu Jan  9 13:06:32 2020: Thu Jan  9 13:06:32 2020 command: /usr/bin/wbemcli -nl -t -noverify ei 'http://stor2rrd:Test_1234@10.200.8.59:5988/root/smis:CIM_BlockStorageStatisticalData' timed out after: 700 seconds /var/lib/stor2rrd/bin/dothillperf.pl:2446 : Interrupted system call
Thu Jan  9 13:18:12 2020: Thu Jan  9 13:18:12 2020 command: /usr/bin/wbemcli -nl -t -noverify ei 'http://stor2rrd:Test_1234@10.200.8.59:5988/root/smis:CIM_BlockStorageStatisticalData' timed out after: 700 seconds /var/lib/stor2rrd/bin/dothillperf.pl:2446 : Interrupted system call

But sometimes data comes in. I changed polling period from 5 to 10 minutes, it didn't help.

Pavel

Hi,

send us logs.

Any change in yoru environment on 16th of Dec? I see that before it worked. Any storage firmware upgrade etc?

Note a short problem description in the text field of the upload form.

cd /home/stor2rrd/stor2rrd # or where is your STOR2RRD working dir

tar cvhf logs.tar logs tmp/*txt

gzip -9 logs.tar

Send us logs.tar.gz via https://upload.stor2rrd.com

kuznetsovmyu

Hi Pavel,
I sent you logs. I have no information about any changes on the storage on 15th or 16th Dec.

NoOne

Hi there.

We have the same problem with Lenovo DS6200.

A lot of performance gaps, error messages in email:

STOR2RRD Data Health Status event for device Lenovo-DS6200-1 Storage does not provide performance statistics longer than 6 hours.

As I think it started to happen in December too. It might be connected with the firmware update.

But after that we had some more firmware updates, but the problem is still there.

And we have Lenovo DS2200 with the same firmware level, and no problem.

STOR2RRD is 2.80.

I have uploaded logs.

Thank you.

Pavel

Hi,

try this:

1. wait for time period when there is no data coming

2. run this cmd (change IP/username/password)

time /usr/bin/wbemcli -nl -t -noverify ei 'https://stor2rrdro:XXXXX@10.11.12.156:5989/root/dhs:CIM_BlockStorageStatisticalData'

Does it provide any output?

How long it takes? (you get time stats once cmd finishes)

NoOne

Unfortunately, I have this result:

$ time /usr/bin/wbemcli -nl -t -noverify ei 'https://monitoring:***@10.***:5989/root/smis:CIM_BlockStorageStatisticalData'
*
* /usr/bin/wbemcli: Http Exception: Server returned nothing (no headers, no data)
*

real    1m10.434s
user    0m0.069s
sys     0m0.115s

And in the same time DS2200 returns good result in two seconds. Does it mean, that there are some issues with the storage array?

Pavel

Hi,

We already seen that, some storage timing our or return this.
Try to upgrade firmware, it helped in some cases.
Raise a call with the storage vendor, send output from above cmd from working and not working storage (which are ideally in same subnet to avoid firewall issues)

NoOne

Hi.

We opened an issue ticket with the vendor. As I understand, they confirmed the problem and escalated it to developers.

Unfortunately, the deadlines for solving the problem have not been set.

So, will be waiting for their response.