Copy Services Error

dickinsonzach
dickinsonzach
in IBM
How do I go about reporting a bug?  We upgraded to 2.40 and received many of these:

STOR2RRD Copy Services Error for device gcdelta7kmy
name:source_vdisk_name:target_vdisk_name:group_name:status:progress:start_time

fcmap0:Genisys_iASP1_1:vdisk0::stopped:0:

The devices/relationship called out are the background Flash Copies when you have a Remote Copy setup between IBM V7000.  They go into a stopped state when not replicating.

So while true, it should be called out as an error.

Thank you, Zach.
«1

Comments

  • Pavel
    Pavel
    Hi,

    do you say that stopped status is ok in this case?
  • dickinsonzach
    dickinsonzach
    It CAN be OK.  If you create a Global Mirror with Change Volumes.  It will create a Thin Volume where the changes are placed.  Then every 5 minutes (default), the changes are replicated from the change volume.  If there are no changes being made or very little, the relationship between the replicated volume and the Change Volume can be stopped.

    On the back end, the relationship is a Flash Copy.  Watching it in real time the Flash Copies will go from active, to stopped, to active, etc...

    Perhaps there needs to be an exception for Flash Copies that exist because they are part of a Remote Copy?
  • Pavel
    Pavel
    is that related only to Global Mirror, right? Then stopped does nt meanerror status. That is clear. Metro mirror should not stop, right?

    What about flash copy generally? That is your case, righ? Shall we behave same? i.e. stopped == ok


  • dickinsonzach
    dickinsonzach
    I cannot say for Metro Mirror.  The Global Mirror should not be stopped.



    What can be stopped is the Change Copy Volume in the FlashCopy Mapping.


    Does that make sense?  Thank you, Zach.
  • dickinsonzach
    dickinsonzach
    If there are any logs I can provide, please let me know.  Or I do have the ability to clone our existing instance and troubleshoot.  Thank you, Zach.
  • Pavel
    Pavel
    we got other user complaining same with : GM with Change Volume

    it is not about logs, it is about how to handle states (i.e. ok/warning/error)

    ok, we will not threat it as error in this situations.
    we make a fix during tomorrow and let you know.

  • dickinsonzach
    dickinsonzach
    Wow!  Thank you!
  • Pavel
    Pavel
    looking into that but got a problem:

    For flash copy we have these statuses:
    * idle_or_copied … normal  (green)
    * preparing … warning  (orange) 
    * prepared … warning  (orange) 
    * copying … normal  (green)
    * stopped … error  (red)
    * suspended … warning  (orange) 
    * stopping … warning  (orange) 

    just *stopped status is red.
    If we change it to orange for example then flash copy will never be red.

    here is data example what we get from storage:

    FC Map Level Configuration
    ------------------------
    id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_id,group_name,status,progress,copy_rate,clean_progress,incremental,partner_FC_id,partner_FC_name,restoring,start_time,rc_controlled
    1,fcmap1,107,aix_rootvg-migrace,108,aix_rootvg-migrace_01,,,copying,22,0,100,off,,,no,170612150923,no
    2,fcmap2,102,aix1_rootvg-migrace,104,aix1_rootvg-migrace_1,,,copying,3,0,100,off,,,no,170616132736,no

    Any idea?



  • dickinsonzach
    dickinsonzach
    Good morning, specifically for us we do not use Flash Copy for anything other than Remote Copy.  If we had a way to disable a check, that works for us.

    Two ideas:  1.)By default when creating a Remote Copy the Flash Copies get created with the name vdisk#  so vdisk0, vdisk1, etc...  You could ignore Flash Copies that have vdisk# as the target volume.  2.)Can you check to see if a Flash Copy relationship volume is also in a Remote Copy?  3.)OK, three ideas.  Combine 1 and 2.  Ignore if the target is vdisk# AND the relationship exists as a Flash Copy AND a Remote Copy.

    I realize there are scenarios that could be overlooked.  Thank you, Zach.
  • dickinsonzach
    dickinsonzach
    Is there an update on this?  We will not be able to upgrade until this is resolved.  Thank you, Zach.
  • Pavel
    Pavel
    sorry, no update yet.
    it sounds quite complex, we are waiting for an anwer of our expert.
    we will let you know during this week.
  • dickinsonzach
    dickinsonzach
    Sounds good.  Thank you, Zach.
  • Pavel
    Pavel
    Try this fix for that.
    Solution is exclude flashes created by global mirroring.
    They do not have to be monitored. If anything is wrong with them then global mirror reports problem which we present.


    Gunzip it and copy to /home/stor2rrd/stor2rrd/bin (755, stor2rrd owner)
    -rwxr-xr-x 1 stor2rrd stor2rrd 93479 Oct  3 13:57 bin/svcconfig.pl

    If your web browser gunzips it automatically then just rename it: mv svcconfig.pl.gz svcconfig.pl
    Assure that file size is same as above listed one


  • dickinsonzach
    dickinsonzach
    1.)Ran the upgrade
    2.)Copied the provided file
    3.)Rebuilt the GUI:  ./load.sh | tee logs/load.out-initial

    At the end of rebuilding the GUI, I got a number of the following errors:
    Use of uninitialized value $p_id in substitution (s///) at bin/Storcfg2html.pm line 482, <FCFGR> line 127.

    Thoughts?  Thank you, Zach.
  • Pavel
    Pavel
    ignore that.
    have disappeared remote flashes from global copy pairs?
    It could take 10 - 20 minutes.

  • dickinsonzach
    dickinsonzach
    I will ignore the error from the GUI refresh.

    As for the Remote Flashs:

    STOR2RRD Copy Services Error for device gcmilton7kmg
    name:source_vdisk_name:target_vdisk_name:group_name:status:progress:start_time

     fcmap238:GC_Milton_MG_GCI51001-iASP_FD_RC_43_Aux:GC_Milton_MG_MiltonFC-iASP_FD_FC_43:MILTONFC:stopped:0:

    We got one alert, but it had an entry for each Remote Flash.



    Thank you, Zach.
  • Pavel
    Pavel

    are global remote flashes still listed in tmp/snapshotting-status/<storage name>.txt ?

  • dickinsonzach
    dickinsonzach
    I have attached the two files in that directory.  They are Remote Partners.  What's interesting is that we 2 other arrays that have a Remote Partnership and they do not appear.  Thank you, Zach.
  • Pavel
    Pavel
    ok, then it works partially right?
    on some storages it work and do not some others.

    Could you send us data from storage where it still not working?
    cd /home/stor2rrd/stor2rrd

    3 files : ls -l tmp/<storage name>*txt

    Thanks
  • dickinsonzach
    dickinsonzach
    Both Remote Copies are not working as I would expect.

    The relationship between GCDelta7KMY and GCMilton7KMG is reporting in the GUI, but kicks an unnecessary error each night.

    The relationship between GCDeltta7KKA and GCMilton7K7W is not reporting in the GUI.

    I have uploaded the requested files for all 4 arrays.  Thank you, Zach.
  • dickinsonzach
    dickinsonzach
    Good morning, any other thoughts on this subject?  Thank you, Zach.
  • Pavel
    Pavel
    Hi, working on it :)
    It is quite complicated to cover all statuses which might appear in copy services.

    the first issue has been resolved in previous patch but there were 2 independed issues to be fixed.
    We will provide you a fix during today, we already know what need to be done.
  • dickinsonzach
    dickinsonzach
    Sounds good.  Thank you for the quick reply, Zach.
  • Pavel
    Pavel
    Try this cumulative patch since 2.40:

    Download link: http://download.stor2rrd.com/patch/stor2rrd-2.40-03.tar

    su - stor2rrd
    tar xvf stor2rrd-2.40-03.tar
    cd stor2rrd-2.40-03
    ./install.sh

    cd /home/stor2rrd/stor2rrd
    ./load.sh

    refresh the GUI when it finishes (Ctrl-F5)


  • dickinsonzach
    dickinsonzach
    I will install the update and let you know how it goes.  Thank you, Zach.
  • dickinsonzach
    dickinsonzach
    I still get:

    Use of uninitialized value $p_id in substitution (s///) at bin/Storcfg2html.pm line 482, <FCFGR> line 127.

    after re-doing the GUI, but everything else looks good.

    Thank you, Zach.
  • Pavel
    Pavel
    ignore that
  • dickinsonzach
    dickinsonzach
    I will ignore that.

    It's been over a month, it did pop up about 2 hours ago:

    STOR2RRD Copy Services Error for device gcmilton7kmg
    name:source_vdisk_name:target_vdisk_name:group_name:status:progress:start_time

     fcmap204:GC_Milton_MG_GCI51001-iASP_FD_RC_9_Aux:GC_Milton_MG_MiltonFC-iASP_FD_FC_9:MILTONFC:stopped:0

    Seems to be green now though.  I will post if it happens again.  Thank you, Zach.
  • dickinsonzach
    dickinsonzach
    Good morning, we are seeing STOR2RRD Copy Services Error for device gcmilton7kmg...

    fcmap196:GC_Milton_MG_GCI51001-iASP_FD_RC_1_Aux:GC_Milton_MG_MiltonFC-iASP_FD_FC_1:MILTONFC:stopped:0...

    Every other day or so and it's showing as an error and paging out.  Is there a way to ignore this error?  Or set to warning so it e-mails, but does not page?

    Thank you, Zach.
  • Pavel
    Pavel
    Hi,

    is that still the same issue with global mirror?
    We do not have evidency of such issues since the last fixes month ago (a fix above in nthe thread).
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Categories

In this Discussion