7.20 Fails to complete the load_vmware.sh

MLavigne

Upgraded a functional running 7.10 version (I think. It was the Xormon required level ) to 7.20.  In the VMWare section I see the VCSA and the storage but there is nothing for the ESX systems.  I had bookmarked graphs for the Dashbboard and they will appears (with that looks like current data) but clicking on the graph does not take me to the page (as it looks like it doesn't exist).  

start of load_vmware.sh:
[lpar2rrd@ant-nagios lpar2rrd]$ ./load_vmware.sh
jobs -l vmware :  
[1]+ 30078 Running                 $BINDIR/vmware_run.sh &
vmware timeout : timeout has been set to 3550
Parallelization: ESXi:10 Cluster:60 Datastore:200
LPAR2RRD free version 7.20
Vmware    start: antvcsa|antvcsa.mysystem.com|lpar2rrd@vsphere.local Fri Jun 11 11:42:45 2021 PID 30085 ScriptSize 318203
Perl version   : 5.016003
SDK            : antvcsa(43068d5d-a970-430a-b4f9-5f6635ca7ce0), vSphere SDK for Perl version-a: 6.7.0
system      UTC: antvcsa, VirtualCenter VMware vCenter Server 7.0.2 build-17958471 UTC:2021-06-11T15:42:45.950501Z : 1623426165
vC DB config   : antvcsa, Perf.Stats.MaxCollectionThreads:2  VirtualCenter.MaxDBConnection:50  config.vpxd.stats.maxQueryMetrics: not detected
hist_interval0 : antvcsa, 1 for 'Past day' for VI_SERVER antvcsa.nya.ibm.com enabled:1
Hosts number   : antvcsa 2 $PARALLELIZATION 10


POST VMWare output:
--------------------------------- line_count 55
date end       : antvcsa.mysystem.com Fri Jun 11 11:42:52 2021
vmware finish  : status 0
vmware timeout : expired after 3550, killing : kill -TERM -30078
Vcleaning procs: 30078
Vcleaning done : 30078
/var/www/html/lpar2rrd/lpar2rrd/bin/vmware_push_perf_data_to_rrd.sh: line 23: /vmware_push_perf_data_to_rrd.sh.pid: Permission denied
Updating       : start updating rrdfiles with vcenter data in background
reducing       : vm names : start Fri Jun 11 11:42:58 2021
reducing       : vm names : stop Fri Jun 11 11:42:58 2021 lines read:59, lines written:59
installing WWW : install-html.sh vmware
Host identif   :   
timestamp      : Fri Jun 11 11:42:58 EDT 2021  
Menu           : no menu refresh for vmware
vmware-menu2db.pl : _DB Inserting VCENTER : antvcsa
vmware-menu2db.pl : _DB Inserting DATACENTER : datastore_IBM Datacenter  
vmware-menu2db.pl : _DB Inserting DATASTORE : Beast_Local_Datastore  
vmware-menu2db.pl : _DB Inserting DATASTORE : NFE_IBMV7K1_Datastore  
vmware-menu2db.pl : _DB Inserting DATASTORE : NFE_IBMV7K1_Datastore2  
vmware-menu2db.pl : _DB Inserting DATASTORE : NFE_IBMV7K2_Datastore1  
vmware-menu2db.pl : _DB Inserting DATASTORE : NFE_IBMV7K2_Datastore2  
vmware-menu2db.pl : _DB Inserting DATASTORE : Halfpint_Local_Datastore  
heatmap        : start Fri Jun 11 11:42:59 2021
An error occured in lpar2rrd, check /var/www/html/lpar2rrd/lpar2rrd/logs/error.log-vmware and output of load.sh

$ tail -1 /var/www/html/lpar2rrd/lpar2rrd/logs/error.log-vmware

nohup: ignoring input
date end all VM: Fri Jun 11 11:42:59 EDT 2021

Stumped and would like to not be missing data but any thoughts would be appreciated. 

Matthew

MLavigne

So I had a stored clone of this system and it is evidencing the same issue on 7.10.  I did upgrade my VCSA to 7.02 due to the known security issues.  Do I need to upgrade the SDK to match the vcsa version?

MLavigne

So upgrade the SDK to 7.0 also and still nothing showing anything other then the data stores.  I have dropped vcsa and readded it back and still do not get anything.   Would like to see if anyone has any ideas. 

Jindra

trying discover this err

/var/www/html/lpar2rrd/lpar2rrd/bin/vmware_push_perf_data_to_rrd.sh: line 23: /vmware_push_perf_data_to_rrd.sh.pid: Permission denied

pls output of these cmnds:

ls -l /var/www/html/lpar2rrd/lpar2rrd/tmp/*pid

find /var/www/html/lpar2rrd/lpar2rrd/ -exec ls -ld {} \; | grep root

MLavigne

Results:
[lpar2rrd@ant-nagios ~]$ ls -l /var/www/html/lpar2rrd/lpar2rrd/tmp/*pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:30 /var/www/html/lpar2rrd/lpar2rrd/tmp/custom.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_aws.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_azure.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_gcloud.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_hyperv.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_kubernetes.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_nutanix.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_openshift.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:35 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_oracledb.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_oraclevm.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_ovirt.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:35 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_postgres.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_proxmox.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:30 /var/www/html/lpar2rrd/lpar2rrd/tmp/load.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:30 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_vmware.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:20 /var/www/html/lpar2rrd/lpar2rrd/tmp/load_xenserver.sh.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 11 09:03 /var/www/html/lpar2rrd/lpar2rrd/tmp/lpar2rrd-daemon.pid -rw-r--r-- 1 lpar2rrd lpar2rrd 5 Jun 15 05:30 /var/www/html/lpar2rrd/lpar2rrd/tmp/vmware_push_perf_data_to_rrd.sh.pid [lpar2rrd@ant-nagios ~]$ find /var/www/html/lpar2rrd/lpar2rrd/ -exec ls -ld {} \; | grep root -rw-rw-r-- 1 root root 44 Mar 11 09:16 /var/www/html/lpar2rrd/lpar2rrd/etc/web_config/.htusers.cfg

So nothing really outrageous in that bunch. 

MLavigne

As expected there is no data in xormon either. (Finally logged into that to verify if there is anything there either).   It looks based on what I have favorited on the Dashboard that there is data getting loaded just don't have a way to look at the individual esx systems.  I am thinking that this may be related to updating the vcsa to 7.0