Cannot setup VSP G400

vadim

I'm trying to setup Hitachi VSP G400 and cannot complete configuration of CCI.

config

[lpar2rrd@ctc-xorux ~]$ cat /etc/horcm1.conf
HORCM_MON
localhost 11001 1000 3000
HORCM_CMD
\\.\IPCMD-172.19.15.195-31001

error:

[lpar2rrd@ctc-xorux ~]$ /HORCM/usr/bin/horcmstart.sh 1
starting HORCM inst 1
HORCM inst 1 has failed to start.

log:

[lpar2rrd@ctc-xorux ~]$ cat /HORCM/log1/curlog/horcm_ctc-xorux.log
*****************************************************************************
- HORCM STARTUP LOG - Mon Jun 27 12:49:19 2016
*****************************************************************************
12:49:19-2159d-00013903- horcmgr started on Mon Jun 27 12:49:19 2016
12:49:19-2196e-00013903- execvp() horcmd_01 using /etc/horcmgr [CWD=/]
12:49:19-224c7-00013904- Fibre address conversion TBL has been set to '3'

P.P. : RAID Manager for Linux/x64
Model : RAID-Manager/Linux/x64
Ver&Rev: 01-35-03/05
Release: Production(GA)

ALL Rights Reserved, Copyright (c) 1998-2015, Hitachi, Ltd.

HORCM(ctc-xorux 13904) started by lpar2rrd (1005) on Mon Jun 27 12:49:19 2016

12:49:19-23d75-00013904- horcmd_01 started on Mon Jun 27 12:49:19 2016
slmcmrhdr size = 56 bytes
rqry_test size = 288 bytes
rpprc_cmd size = 688 bytes
rread_cfg size = 208 bytes
rwait_evt size = 184 bytes
12:49:19-24d3f-00013904- ******** starts Loading library for IPv6 *******
[ AF_INET6 = 10, AI_PASSIVE = 1 ]
getaddrinfo() : Linked on itself
inet_pton() : Linked on itself
inet_ntop() : Linked on itself
12:49:19-25ab4-00013904- ******** finished Loading library **************
12:49:19-25daa-00013904- [horcmcfgrdf] access(conf_file) OK.
12:49:19-26086-00013904- [horcmcfgrdf] access(check) OK.
12:49:19-263b3-00013904- [horcmcfgrdf] open(conf_file) OK.
12:49:19-26677-00013904- Cannot find HORCM_MON config data, so that HORCM set default values.
12:49:19-268cd-00013904- [horcmcfgetent] HORCM_MON set the following values.[NONE][31002][1000][3000]
12:49:19-26b3b-00013904- [horcmcfgetent] fseek(top) OK.
12:49:19-26e69-00013904- ERROR: Reserved string is used as groupname, or extra character(s) exist(s) behind reserved string.: line 1
12:49:19-270f1-00013904- HORCM_MON
12:49:19-2a0f5-00013904- grpname:[localhost] ipaddr:[0.0.42.249] port:[cadlock2] are registered.
12:49:19-2a30f-00013904- ERROR: Reserved string is used as groupname, or extra character(s) exist(s) behind reserved string.: line 3
12:49:19-2a946-00013904- HORCM_CMD
12:49:19-2ab0c-00013904- ERROR: No ipaddr line exists in HORCM_INST:line 4
12:49:19-2acdc-00013904- \\.\IPCMD-172.19.15.195-31001
12:49:19-2aea6-00013904- [horcmcfgetent] read(conf_file) OK.
12:49:19-2b07a-00013904- [horcmcfgrdf] close(conf_file) OK.
12:49:19-2b287-00013904- ERROR:horcm_cfg_create
12:49:21-e6959-00013903- horcmgr:Failed to connect to HORCM.

Please help.

Pavel

Hi Vadims,

this come strange to me
12:49:19-268cd-00013904- [horcmcfgetent] HORCM_MON set the following values.[NONE][31002][1000][3000]
--> it says 31002, but in config file I see 31001:
\\.\IPCMD-172.19.15.195-31001

This is also strange (IP, and port cadlock2 is 1000)
12:49:19-2a0f5-00013904- grpname:[localhost] ipaddr:[0.0.42.249] port:[cadlock2] are registered.

ps -ef | grep horcm

Assure that everythig is owner by stor2rrd
# chown -R stor2rrd /HORCM/log* /HORCM/.uds /HORCM/usr
# chmod 755 /HORCM/usr/bin /HORCM/usr/bin/* /HORCM/log /HORCM/etc/horcmgr /HORCM/etc/*conf /HORCM/.uds/ /HORCM/.uds/*

Send us logs:
truss of log.out /HORCM/usr/bin/horcmstart.sh 1

vadim

In case of virtual appliance - should I use lpar2rrd user for chown?

Pavel

yes, there is no stor2rrd account

vadim

ps:

[root@ctc-xorux ~]# ps -ef | grep horcm
root 6558 6534 0 15:32 pts/0 00:00:00 grep --color=auto horcm

Same errors in log after chown and chmod:

15:43:58-2b14a-00011838- [horcmcfgrdf] access(conf_file) OK.
15:43:58-2b3d0-00011838- [horcmcfgrdf] access(check) OK.
15:43:58-2b672-00011838- [horcmcfgrdf] open(conf_file) OK.
15:43:58-2b9de-00011838- Cannot find HORCM_MON config data, so that HORCM set default values.
15:43:58-2bc5c-00011838- [horcmcfgetent] HORCM_MON set the following values.[NONE][31002][1000][3000]
15:43:58-2be9d-00011838- [horcmcfgetent] fseek(top) OK.
15:43:58-2c0dd-00011838- ERROR: Reserved string is used as groupname, or extra character(s) exist(s) behind reserved string.: line 1
15:43:58-2c302-00011838- HORCM_MON
15:43:58-2e723-00011838- grpname:[localhost] ipaddr:[0.0.42.249] port:[cadlock2] are registered.
15:43:58-2ea09-00011838- ERROR: Reserved string is used as groupname, or extra character(s) exist(s) behind reserved string.: line 3
15:43:58-2ec86-00011838- HORCM_CMD
15:43:58-2eee4-00011838- ERROR: No ipaddr line exists in HORCM_INST:line 4
15:43:58-2f14a-00011838- \\.\IPCMD-172.19.15.195-31001
15:43:58-2f39c-00011838- [horcmcfgetent] read(conf_file) OK.
15:43:58-2f614-00011838- [horcmcfgrdf] close(conf_file) OK.
15:43:58-2f882-00011838- ERROR:horcm_cfg_create
15:44:00-ec5b8-00011837- horcmgr:Failed to connect to HORCM.

vadim

Should I provide more logs / info?

Pavel

This line confuses me, no explanation
15:43:58-2e723-00011838- grpname:[localhost] ipaddr:[0.0.42.249] port:[cadlock2] are registered.

netstat -an | egrep "1000|11001"

send us truss (strace if it is on Linux)
truss -f log.out /HORCM/usr/bin/horcmstart.sh 1

vadim

netstat shows nothing about 1000 and 11001.
strace output attached.

Pavel

Do you use right IP in horcm.conf file?
horcm1.conf should have storage node IP (not SVC). And in storage-list.cfg SVC address is needed

Pavel

telnet 172.19.15.195 31001

vadim

Oh, this way it works. It was not obvious for me from installation guide - which IP where to use.
Thanks a lot!

Pavel

Great. It was something we were not aware until today as well
I have just updated the install docu.

muhammed_jabri

Please suggest i am getting this error while setting up vsp g600.

[lpar2rrd@xorux /]$ su root

Password:

[root@xorux /]# su stor2rrd

bash-4.2$ /HORCM/usr/bin/horcmstart.sh 1

starting HORCM inst 1

HORCM inst 1 has failed to start.

bash-4.2$ /HORCM/usr/bin/horcmstart.sh 1 2

starting HORCM inst 1

HORCM inst 1 has failed to start.

bash-4.2$ /HORCM/usr/bin/horcmstart.sh 2

starting HORCM inst 2

HORCM inst 2 has failed to start.

bash-4.2$ ps -ef |grep horcm

stor2rrd 12181 12167  0 09:09 pts/0    00:00:00 grep horcm

stor2rrd 21283     1  0 Aug27 ?        00:00:18 horcmd_01

stor2rrd 21289     1  0 Aug27 ?        00:00:13 horcmd_02

bash-4.2$ raidcom -login stor2rrd hello12345 -I1

COMMAND ERROR : EUserId for HORC[1] : stor2rrd (1007) Sun Sep 9 09:10:12 2018

CMDLINE : raidcom -login stor2rrd ***** -I1

*********** SYSTEM ERROR ***********

P.P. : RAID Manager for Linux

Model : RAID-Manager/Linux

Ver&Rev: 01-41-03/05

Release: Production(GA)

Host: xorux

EUserId: stor2rrd (1007)

Process: 13764

[System Call Error]

SysCall: bind

Errorno: 2 (No such file or directory)

ErrInfo: Internal Error

ErrTime: Sun Sep 9 09:10:12 2018

SrcFile: shorcmc.c

SrcLine: 744

09:10:12-81ef8-00013764- 13764:HORCM death detected

09:10:15-52263-00013764- ERROR:cm_open[scmclcon() timeout_err]

09:10:15-52299-00013764- ERROR:horcm_lep_create

09:10:15-522b3-00013764- [raidcom][exit(251)]

[EX_ATTHOR] Can't be attached to HORC manager

[Cause ]:Couldn't connect with the HORC manager.

[Action]:Please check if HORC manager is running or if HORCMINST is set correctly.

raidcom: [EX_ATTHOR] Can't be attached to HORC manager

bash-4.2$ raidqry -l -I1

COMMAND ERROR  : EUserId for HORC[1] : stor2rrd (1007)  Sun Sep  9 09:30:30 2018

CMDLINE : raidqry -l -I1

*********** SYSTEM ERROR ***********

P.P.   : RAID Manager for Linux

Model  : RAID-Manager/Linux

Ver&Rev: 01-41-03/05

Release: Production(GA)

   Host: xorux

EUserId: stor2rrd (1007)

Process: 21769

[System Call Error]

SysCall: bind

Errorno: 2 (No such file or directory)

ErrInfo: Internal Error

ErrTime: Sun Sep  9 09:30:30 2018

SrcFile: shorcmc.c

SrcLine: 744

09:30:30-887ec-00021769- 21769:HORCM death detected

09:30:33-58e58-00021769- ERROR:cm_open[scmclcon() timeout_err]

09:30:33-58ead-00021769- ERROR:horcm_lep_create

09:30:33-58edd-00021769- [raidqry][exit(251)]

[EX_ATTHOR] Can't be attached to HORC manager

[Cause ]:Couldn't connect with the HORC manager.

[Action]:Please check if HORC manager is running or if HORCMINST is set correctly.

raidqry: [EX_ATTHOR] Can't be attached to HORC manager

Refer to the command log(/HORCM/log1/horcc_xorux.log) for details.