integration of SAN fails

shimaore

Hello All,

I am trying to configure stor2rrd and sannav .

I am getting this error

API host network connection:OK

TCP connection to x.x.x.x is OK.

Crontab test:Warning

WARNING: SANNAV agent is not scheduled in crontab
Put there this line as per documentation (under user: stor2rrd) :
crontab -e
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /stor2rrd/stor2rrd/load_sannavperf.sh > /stor2rrd/stor2rrd/logs/load_sannavperf.out 2>&1

Ignore it if you have just done in past 5 - 10 minutes

KAFKA network connection (usually on the same machine as STOR2RRD):OK

TCP connection to x.x.x.x:8081 is OK.

TCP connection to x.x.x.x :9092 is OK.

API authorization:OK

Switches : 63
Fabrics  : 36

Kafka data test:NOK

Kafka consumer daemon is not running (sannavperf.py 172.26.42.65).
Try this connection test again after 10 minutes.

Karel

Hi shimaore,

it looks like the Kafka consumer is failing. Can you send us logs?

Note a short problem description in the text field of the upload form.

cd /home/stor2rrd/stor2rrd # or where is your STOR2RRD working dir
tar cvhf logs.tar logs tmp/*txt
gzip -9 logs.tar

Send us logs.tar.gz via https://upload.stor2rrd.com

shimaore

Thanks Karel,

I just uploaded the logs

Karel

Hi all,

Kafka consumer daemon didn't work because there were old versions of librdkafka and kafkian.

The issue was resolved after reinstalling librdkafka and kafian to the latest versions.

You can identify this issue by this error in the kafka consumer error log:

cd /home/stor2rrd/stor2rrd # or where is your STOR2RRD working dir
cat logs/error.log-kafka-consumer-<Kafka hostname or IP>

Traceback (most recent call last):
 File "/stor2rrd/stor2rrd/bin/sannavperf.py", line 51, in <module>
    value_deserializer=AvroDeserializer(schema_registry_url=f'http://{ip}:{schema_port}'),
 File "/usr/lib64/python3.6/site-packages/kafkian/consumer.py", line 76, in __init__
  self._consumer_impl = self._init_consumer_impl(config)
 File "/usr/lib64/python3.6/site-packages/kafkian/consumer.py", line 81, in _init_consumer_impl
  return ConfluentConsumer(config, logger=logging.getLogger("librdkafka.consumer"))
cimpl.KafkaException: KafkaError{code=_INVALID_ARG,val=-186,str="Property "throttle_cb" must be set through dedicated .._set_..() function"}
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
 File "/usr/lib64/python3.6/site-packages/kafkian/consumer.py", line 117, in _close
self._consumer_impl.close()
AttributeError: 'Consumer' object has no attribute '_consumer_impl'

Here are versions we're using in our labs:

rpm -qa | grep -i kafka

librdkafka1-1.9.0-1.cflt.el7.x86_64

librdkafka-devel-1.9.0-1.cflt.el7.x86_64

pip3 show kafkian

Name: kafkian

Version: 0.13.0

Summary: Opinionated Kafka Python client on top of Confluent python library

Home-page: https://github.com/saabeilin/kafkian

Author: Sergei Beilin

Author-email: saabeilin@gmail.com

License: Apache 2.0

Location: /home/stor2rrd/.local/lib/python3.6/site-packages

Requires: avro-python3, fastavro, confluent-kafka

pip3 show avro-python3

Name: avro-python3

Version: 1.10.2

Summary: Avro is a serialization and RPC framework.

Home-page: https://avro.apache.org/

Author: Apache Avro

Author-email: dev@avro.apache.org

License: Apache License 2.0

Location: /home/stor2rrd/.local/lib/python3.6/site-packages

Requires: 

pip3 show fastavro

Name: fastavro

Version: 1.4.7

Summary: Fast read/write of AVRO files

Home-page: https://github.com/fastavro/fastavro

Author: Miki Tebeka

Author-email: miki.tebeka@gmail.com

License: MIT

Location: /home/stor2rrd/.local/lib/python3.6/site-packages

Requires:

pip3 show confluent-kafka

Name: confluent-kafka

Version: 1.8.2

Summary: Confluent's Python client for Apache Kafka

Home-page: https://github.com/confluentinc/confluent-kafka-python

Author: Confluent Inc

Author-email: support@confluent.io

License: UNKNOWN

Location: /home/stor2rrd/.local/lib/python3.6/site-packages

Requires: