Oracle VM - script crash during work

brykfusitos

Hello,

I added 2 Oracle VM Manager for monitoring in lpar2rrd and i don't have data's in lpar2rrd.

in logs i saw errors  like this :

"Use of uninitialized value $RRDp::error_mode in string eq at /usr/lib64/perl5/vendor_perl/RRDp.pm line 168.
Thu Jul 23 08:10:49 2020: RRDp::create read error : /home/lpar2rrd/lpar2rrd/data/OracleVM/vm/0004fb0000060000e91ccaf6a806a049/disk-/dev/mapper/36005076802810d105000000000000082.rrd : ERROR: creating '/home/lpar2rrd/lpar2rrd/da
ta/OracleVM/vm/0004fb0000060000e91ccaf6a806a049/disk-/dev/mapper/36005076802810d105000000000000082.rrd': No such file or directory at /home/lpar2rrd/lpar2rrd/bin/Xorux_lib.pm line 88 : No such file or directory
Thu Jul 23 08:10:49 2020: Unable to create /home/lpar2rrd/lpar2rrd/data/OracleVM/vm/0004fb0000060000e91ccaf6a806a049/disk-/dev/mapper/36005076802810d105000000000000082.rrd : at /home/lpar2rrd/lpar2rrd/bin/OracleVmLoadDataModul
e.pm: line 140 : No such file or directory"

after about 50 errors scripts ends.

before these errors i see one information more:

Updating server net RRD      : /home/lpar2rrd/lpar2rrd/data/OracleVM/server/e2bcdc7c921311e5846dd67c1c6b0b2b/lan-eth12.rrd
orvm-json2rrd.pl : status 0, Thu Jul 23 08:30:47 CEST 2020
/home/lpar2rrd/lpar2rrd/load_oraclevm.sh: line 191:  3013 Alarm clock             wait_time $timeout

I see in browser configuration which servers, vms are discovered but its don't have data and graphs, heatmap, memory etc.

version: OVM Manager are : 3.4.2.1384 and 3.4.6.2622

successfully request - https://server2:port/ovm/core/wsapi/rest/Statistic/Server/8f:da:75:85:e0:16:11:e2:8d:4a:6c:ae:8b:73:c2:64/DISK_READ/range?startTime=1e/wsapi/rest/Statistic/Vm/0004fb0000060000d9733aea35ee5001/NETWORK_RECEIVED/range?startTime=1595484002000&endTime=1595485202000

successfully request - https://server1:port/ovm/core/wsapi/rest/Statistic/Vm/0004fb0000060000281dd7b566791f39/CPU_COUNT/range?startTime=1595484601000&endTime=est - https://server2:port/ovm/core/wsapi/rest/Statistic/Vm/0004fb00000600008694f3c56931dbce/DISK_WRITE/range?startTime=1595484601000&endTime=1595485801000

successfully request - https://server1:port/ovm/core/wsapi/rest/Statistic/Vm/0004fb0000060000281dd7b566791f39/CPU_COUNT/range?startTime=1595482801000&endTime=000919e97600a4e59ce/DISK_READ/range?startTime=1595482801000&endTime=1595484001000

server1, server2 and port - is only word because i can't show you my server name and port

Pavel

ls -la /home/lpar2rrd/lpar2rrd/data/OracleVM

ls -l /home/lpar2rrd/lpar2rrd/data/OracleVM/vm

ls -l /home/lpar2rrd/lpar2rrd/data/OracleVM/OracleVM

brykfusitos

Hi,

[lpar2rrd@c003s3537 lpar2rrd]$ ls -la /home/lpar2rrd/lpar2rrd/data/OracleVM
total 192
drwxrwxrwx   7 lpar2rrd lpar2rrd   4096 Jul 21 14:01 .
drwxrwxrwx 210 lpar2rrd lpar2rrd  24576 Jul 20 09:32 ..
-rw-r--r--   1 lpar2rrd lpar2rrd 142163 Jul 23 14:01 conf.json
drwxr-xr-x   2 lpar2rrd lpar2rrd   4096 Jul 23 14:10 iostats
drwxr-xr-x   2 lpar2rrd lpar2rrd   4096 Jul 21 13:29 metadata
drwxr-xr-x  11 lpar2rrd lpar2rrd   4096 Jul 22 10:21 server
drwxr-xr-x  60 lpar2rrd lpar2rrd   4096 Jul 21 15:10 vm
drwxr-xr-x   2 lpar2rrd lpar2rrd   4096 Jul 23 14:10 xml

[lpar2rrd@c003s3537 lpar2rrd]$ ls -la /home/lpar2rrd/lpar2rrd/data/OracleVM/vm
total 240
drwxr-xr-x 60 lpar2rrd lpar2rrd 4096 Jul 21 15:10 .
drwxrwxrwx  7 lpar2rrd lpar2rrd 4096 Jul 21 14:01 ..
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb000006000000c7c6384e7dc100
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb00000600000562776726f03edd
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb000006000005758b857b870ebb
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb000006000009590ba7eda7d953
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600000b84b9ad7df46cdf
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 13:30 0004fb00000600000ea5ee6d0778634b
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600001117405c8c24d4c6
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb00000600001b365e725e4764fc
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 16:50 0004fb000006000023ff5310fd8ebe5f
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 13:30 0004fb0000060000281dd7b566791f39
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb00000600002a6f9b6a0432ec2c
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600002d2f91f0b9231801
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600003390f58b7667b003
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600004151a0ccf381acb9
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb0000060000439383adad75a216
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb00000600004624b0f2f6971931
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600004b68db53edae6285
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600004d5693cd443267f3
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 16:50 0004fb00000600005a23b1fdc440c964
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb000006000060ddd373b3e1e2cc
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb000006000062635d6718eee2da
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000695510ec61e4466a
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb00000600006ac3b33bcb5f3b91
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600006c704308a48fbd62
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb000006000074e9121b231f9aca
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb000006000074f60012feb3b80a
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 13:30 0004fb0000060000756db25cb6805821
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb00000600007b0a3a252f53ec33
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600007e7a9b902435af7e
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb000006000081bbeb892c4b6a7d
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600008694f3c56931dbce
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 16:50 0004fb000006000090e4008f02984935
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000919e97600a4e59ce
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 16:50 0004fb0000060000938459bba97062bd
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000942f273d08c67311
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb000006000098a38acb4af2df86
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600009c06f4340ef52c44
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb00000600009e10636316f853b3
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 15:10 0004fb0000060000a5d127529acd298a
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000ae1a9bbc38624baf
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000b20a78cb765d6e02
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb0000060000bc76015822219e67
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000ce57d32cd4ba8c0e
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb0000060000d9733aea35ee5001
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000db50071f70a462fc
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb0000060000dc3f8aa7e59c4928
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000dd9456984bb67132
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000dde76949b078717a
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 14:01 0004fb0000060000df0859b13d266a89
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000df4fbab9f674631a
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000e00ab2c6ab2c79ef
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 16:50 0004fb0000060000e1ab2438e1da9802
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 13:30 0004fb0000060000e91ccaf6a806a049
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000f299cb3a70efd082
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000f45921f4cc1861cb
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 21 16:50 0004fb0000060000f466694012990c52
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000fc0f188803ee94c3
drwxr-xr-x  2 lpar2rrd lpar2rrd 4096 Jul 22 10:21 0004fb0000060000ff1446d9264ad3dc

[lpar2rrd@c003s3537 lpar2rrd]$ ls -la /home/lpar2rrd/lpar2rrd/data/OracleVM/OracleVM
ls: cannot access /home/lpar2rrd/lpar2rrd/data/OracleVM/OracleVM: No such file or directory

JiriT

Hi,

can you send us logs please?

tar cvhf logs.tar logs tmp/*txt tmp/*json
gzip -9 logs.tar

Send us logs.tar.gz via https://upload.lpar2rrd.com

brykfusitos

Hello,

Communicating with company information security on this and I can't do it because in that files has too much data about companies IT configurations.

That is a security policy in the company

JiriT

Hi,

ok so send us this output pls.
cd /home/lpar2rrd/lpar2rrd ### or where is you working dir

1) ls -l tmp/menu*
2) ls -l data/OracleVM/vm/*

brykfusitos

Hello

1):
[lpar2rrd@c003s3537 lpar2rrd]$ ls -l tmp/menu*

-rw-r--r-- 1 lpar2rrd lpar2rrd  25051 Jul 27 12:02 tmp/menu_oraclevm.json

-rw-r--r-- 1 lpar2rrd lpar2rrd 246451 Aug 10 00:02 tmp/menu_ovirt.json

-rwxrwxrwx 1 lpar2rrd lpar2rrd  35786 Aug 10 00:02 tmp/menu_power.txt

-rwxrwxrwx 1 lpar2rrd lpar2rrd 798399 Aug 10 00:02 tmp/menu.txt

-rwxrwxrwx 1 lpar2rrd lpar2rrd 497029 Aug 10 00:01 tmp/menu_vms_pl.txt

-rwxrwxrwx 1 lpar2rrd lpar2rrd 265584 Aug 10 00:01 tmp/menu_vmware_pl.txt

-rwxrwxrwx 1 lpar2rrd lpar2rrd 762613 Aug 10 00:01 tmp/menu_vmware.txt

2) i attach file

brykfusitos

Hi,
I still don't have any data:

One ovm sometimes show sometimes don't show
and one ovm although it has been added to the configuration, it is not displayed in the menu
Can you help me?

JiriT

Hi,

please upgrade your LPAR2RRD on this version: https://www.lpar2rrd.com/download/lpar2rrd-6.21-13.tar

# su - lpar2rrd (if the tool is running under lpar2rrd user)
$ tar xvf lpar2rrd-6.XX.tar
$ cd lpar2rrd-6.XX
$ sh ./update.sh

Build a new GUI

$ cd /home/lpar2rrd/lpar2rrd
$ ./load.sh | tee logs/load.out-initial

brykfusitos

Ok i will update today and verify

I want to ask why the data collection time range is the same time point as below:
range?startTime=1598352002000&endTime=1598353202000
maybe that is why my one of ovm show torn data

brykfusitos

Hello,
from update to v6.21-13 i don't have new data in that OVM in which i had.

brykfusitos

HI,
I have re-verified my data collection for OVMs today.
After almost 20 hours from updating to the latest version of the application as there was no data for OVM that did not collect data, so they do not collect it.


And I have checked another OVM that was collecting data before updating and I can see new data



Unfortunately, in my case, the update did not bring the expected results and some of the OVMs still do not collect data

JiriT

Hi,

please send us these outputs.

1) ls -l data/OracleVM/vm/*/*
2) ls -l data/OracleVM/server/*/*
3) ls -l tmp/menu*

brykfusitos

Hi,
I sent You file lpar2rrd_OVM.tar.gz via https://upload.lpar2rrd.com and I leave comment said it was for you JiriT and to that subject

brykfusitos

and i see now in lpar2rrd in overlap OracleVm:

ERROR          : wrong parameter: errlogoraclevm : name=errlogoraclevm&gui=&_=1598519816369 : 

JiriT

Hi,

thanks for upload your logs...

I can see that some servers and vms have stopped updating.

Please can you send me this file logs/error.log-oraclevm?

brykfusitos

In this file I only have an entry as below repeatedly duplicated

Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/orvm-api2json.pl line 631

and nothing more

brykfusitos

but in file /var/tmp/lpar2rrd-realt-error.log i see data like  bellow:

Use of uninitialized value $server_pool_url in concatenation (.) or string at /home/lpar2rrd/lpar2rrd/bin/lpar-search.pl line 776.
Use of uninitialized value $vm_label in pattern match (m//) at /home/lpar2rrd/lpar2rrd/bin/lpar-search.pl line 766.
Use of uninitialized value $vm_label in pattern match (m//) at /home/lpar2rrd/lpar2rrd/bin/lpar-search.pl line 766.

and:

Fri Aug 28 07:38:56 2020: wrong parameter: errlogoraclevm : name=errlogoraclevm&gui=&_=1598519816369 : 
[Fri Aug 28 08:36:17 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1010.
[Fri Aug 28 08:36:17 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1011.
[Fri Aug 28 08:36:17 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1013.
[Fri Aug 28 08:36:18 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1010.
[Fri Aug 28 08:36:18 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1011.
[Fri Aug 28 08:36:18 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1013.
Fri Aug 28 10:08:00 2020: wrong parameter: errlogoraclevm : name=errlogoraclevm&gui=&_=1598519816419 : 
[Fri Aug 28 10:49:18 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1010.
[Fri Aug 28 10:49:18 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1011.
[Fri Aug 28 10:49:18 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1013.
[Fri Aug 28 10:49:19 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1010.
[Fri Aug 28 10:49:19 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1011.
[Fri Aug 28 10:49:19 2020] genjson.pl: Use of uninitialized value $vc in hash element at /home/lpar2rrd/lpar2rrd/bin/genjson.pl line 1013.