lpar2rrd data after migration

master07

Hi there,

I have copied the lpar2rrd/data directory to a new lpar2rrd instance on another linux server and since there no new lpar2rrd agent data is collected. I have changed all crontab entries for all clients, but for me it looks like that the lpar2rrd-daemon.pl don't start the lpar2rrd service on tcp port 8162 correct.

In the lsof -i command I can see that the lpar2rrd service is on LISTEN, but a telnet from a client to the server don't work:

lsof -i

perl       5809  xormon    3u  IPv4 85639007      0t0  TCP *:lpar2rrd (LISTEN)

$ grep lpar2rrd /etc/services
lpar2rrd           8162/tcp     # LPAR2RRD client server communication  [LPAR2RRD_project] [Pavel_Hampl]

$ telnet <lpar2rrd_ip> 8162                         
Trying...

Thanks for your support!

master07

here is the latest load.out log output:

$ tail -30 load.out
/home/xormon/lpar2rrd/data/m9b/hmc0001b/lnx0256b.rrm 1545138044 1513602045 1545138045
/home/xormon/lpar2rrd/data/m9b/hmc0001b/lnx0277b.rrm 1545138044 1513602045 1545138045
/home/xormon/lpar2rrd/data/m9b/hmc0001b/lnx0278b.rrm 1545138044 1513602045 1545138045
/home/xormon/lpar2rrd/data/m9b/hmc0001b/lnx0279b.rrm 1545138044 1513602045 1545138045
/home/xormon/lpar2rrd/data/m9b/hmc0001b/lnx0280b.rrm 1545138044 1513602045 1545138045
/home/xormon/lpar2rrd/data/m9b/hmc0001b/lnx0319b.rrm 1545138044 1513602045 1545138045
/home/xormon/lpar2rrd/data/m9b/hmc0001b/vio01m9b.rrm 1545138043 1513602045 1545138045
/home/xormon/lpar2rrd/data/m9b/hmc0001b/vio02m9b.rrm 1545138043 1513602045 1545138045
date end       : hmc0001b Tue Dec 18 14:00:45 2018
LPAR2RRD custom free version 5.07
No custom groups configured
rmc check      : not this time 18 == 18
lpar_check     : not this time 18 == 18
lpars_count    : not this time 18 == 18
trim logs      : not this time 18 == 18
installing WWW : install-html.sh power
Host identif   : Linux lnx0006a 4.12.14-25.25-default #1 SMP Thu Oct 25 16:07:27 UTC 2018 (d2d8b17) x86_64 x86_64 x86_64 GNU/Linux a8c02d79
timestamp      : Tue Dec 18 14:00:46 CET 2018
AS400 cleaning :
clean all /JOB :
config global  : creating global config
config global  : creating CSV global config
command find_active_lpar.pl: OK
favourites     : no defined yet, placing default
heatmap        : start Tue Dec 18 14:03:00 2018
heatmap        : (Power) set cpu utilization for 338 lpars
heatmap        : (Power) set cpu utilization for 34 pools
heatmap        : (Power) set memory utilization for 317 lpars
heatmap        : end Tue Dec 18 14:03:04 2018
date end all   : Tue Dec 18 14:03:04 CET 2018

Pavel

Hi,

on the server:

netstat -rn | grep 8162

ps -ef| grep lpar2rrd-daemon

I think it is running, but just make sure.

As per telnet test it seem to ne network access issue (blocked of firewalls)

master07

Hi Pavel,

see the lsof -i output beore. the firewall department told me the the lpar2rrd server is not responding.

here is ps -efa output:

$ ps -efa | grep lpar2rrd-daemon
xormon    2324     1  0 14:05 pts/2    00:00:01 /usr/bin/perl -w /home/xormon/lpar2rrd/bin/lpar2rrd-daemon.pl
xormon   15941  2324 63 15:04 pts/2    00:00:10 /usr/bin/perl -w /home/xormon/lpar2rrd/bin/lpar2rrd-daemon.pl
ruxi010  16742 15982  0 15:05 pts/4    00:00:00 grep --color=auto lpar2rrd-daemon
xormon   28577  2324 66 15:00 pts/2    00:03:21 /usr/bin/perl -w /home/xormon/lpar2rrd/bin/lpar2rrd-daemon.pl
xormon   28579  2324 67 15:00 pts/2    00:03:23 /usr/bin/perl -w /home/xormon/lpar2rrd/bin/lpar2rrd-daemon.pl
xormon   30205  2324 66 15:00 pts/2    00:03:09 /usr/bin/perl -w /home/xormon/lpar2rrd/bin/lpar2rrd-daemon.pl
xormon   31575  2324 59 15:00 pts/2    00:02:28 /usr/bin/perl -w /home/xormon/lpar2rrd/bin/lpar2rrd-daemon.pl

Pavel

it looks good!

It does not responding before all 5 process are serving clients.

perhaps a lot of data to load (all missing data from all agents)

It might take longer time, not sure how much agents do you have and how long was outage (migration window)

either wait till it finishes or you can increase of number processes on the server side (in the daemon) to speed it up.

su - lpar2rrd
cd /home/lpar2rrd/lpar2rrd
echo "LPAR2RRD_FORK_MAX=20" >> etc/.magic
echo "export LPAR2RRD_FORK_MAX" >> etc/.magic
./load.sh daemon_stop
./load.sh daemon

master07

Hi Pavel,

thanks for the response. in the log/daemon.out file I got now following messages:

....

Wait for chld  : 0 : 49 : 25245 : Wed Dec 19 09:46:08 2018
could not prepare filename uuid.txt in /home/xormon/lpar2rrd/data/Linux/no_hmc/lnx0223a/JOB/11081.cfg
could not prepare filename uuid.txt in /home/xormon/lpar2rrd/data/Linux/no_hmc/lnx0223a/JOB/21749.cfg
could not prepare filename uuid.txt in /home/xormon/lpar2rrd/data/Linux/no_hmc/lnx0223a/JOB/21681.cfg
could not prepare filename uuid.txt in /home/xormon/lpar2rrd/data/Linux/no_hmc/lnx0223a/JOB/21703.cfg

...

should now the lpar2rrd clients send there data to the new lpar2rrd server that are missing? and should i see after a couple of hours that the graphics close the missing time windows between the migration window?

thanks

master07

no new data, since the data migration, for example the client lnx0223a, is shown in the lpar2rrd gui. I have migrated the data on 13.12.2018

how can i check if the new data is collected by the new lpar2rrd server installation and why the graphics in the lpar2rrd gui doesn't show the new collected data (see the screenshot attached)

Pavel

you should files with actual timespamp

find data -hname \*mmm ls -l {} \;| grep "Dec 19"

master07

I have found a couple of files, but still no current memory graphic is shown.
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:03 ./Linux--unknown/no_hmc/lnx0223a--NMON--/cpu.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:03 ./Linux--unknown/no_hmc/lnx0223a--NMON--/lan-vnet45.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:03 ./Linux--unknown/no_hmc/lnx0223a--NMON--/lan-vnet0.mmm
-rw-r--r-x 1 xormon xormon 5738368 Dec 19 12:03 ./Linux--unknown/no_hmc/lnx0223a--NMON--/mem.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:03 ./Linux--unknown/no_hmc/lnx0223a--NMON--/lan-vnet9.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:03 ./Linux--unknown/no_hmc/lnx0223a--NMON--/lan-vnet61.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:03 ./Linux--unknown/no_hmc/lnx0223a--NMON--/lan-vnet40.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:55 ./Linux--unknown/no_hmc/lnx0223a/cpu.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:55 ./Linux--unknown/no_hmc/lnx0223a/san-host0.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:55 ./Linux--unknown/no_hmc/lnx0223a/lan-eth1.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:55 ./Linux--unknown/no_hmc/lnx0223a/pgs.mmm
-rw-r--r-x 1 xormon xormon 5738368 Dec 19 12:55 ./Linux--unknown/no_hmc/lnx0223a/mem.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:55 ./Linux--unknown/no_hmc/lnx0223a/lan-br_vl2200.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:55 ./Linux--unknown/no_hmc/lnx0223a/lan-eth0.mmm
-rw-r--r-x 1 xormon xormon 3825840 Dec 19 12:55 ./Linux--unknown/no_hmc/lnx0223a/san-host1.mmm

In the screenshot I see "Updated: Fri Dec 14 01:55:01 2018", maybe that's the problem?!?

master07

Hi Pavel,
the graphics are now generated correctly, after the initial data collect job was finished last night.

I would like to take this opportunity to say thank you for your excellent support this year. It encourages us in the decision for lpar2rrd.