OpenShift - Growth of folder /home/lpar2rrd/lpar2rrd/data/OpenShift

Hi,

Since I have added the OpenShift platform to the LPAR2RRD, the growth of the / filesystem is out of control (111GB in just 2 weeks).

The folder with the most occupied space is the following: /home/lpar2rrd/lpar2rrd/data/OpenShift

Is this occupation normal? How can we solve it?

Best regards,

Comments

  • Hi,

    RRD allocates the total capacity at creation, so there will be no increase in size. The only thing that could increase the size are new pods and containers for monitoring.

    But we did a fix to better clean up the data of old pods and containers.
    https://www.lpar2rrd.com/download/lpar2rrd-7.10-8.tar

    It is now possible to skip container monitoring and monitor only nodes and pods.



  • Hi,

    We have updated lpar2rrd to 7.10.8, but now, no performance statistics are collected for OpenShift.

    The following error appears:

    $ tail -24 /home/lpar2rrd/lpar2rrd/logs/error.log-openshift

    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Openshift.pm line 70.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Openshift.pm line 70.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Openshift.pm line 70.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Openshift.pm line 70.
    Thu Apr  1 12:05:22 2021: Empty last.json : Bad file descriptor
    Use of uninitialized value $RRDp::error_mode in string eq at /usr/lib64/perl5/vendor_perl/RRDp.pm line 168.
    Thu Apr  1 12:05:22 2021: Failed during read last time /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd: ERROR: mmaping file '/home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd': Invalid argument at /home/lpar2rrd/lpar2rrd/bin/KubernetesLoadDataModule.pm line 37.
     /home/lpar2rrd/lpar2rrd/bin/KubernetesLoadDataModule.pm:41 at /home/lpar2rrd/lpar2rrd/bin/KubernetesLoadDataModule.pm line 41.
    Can't use string ("1") as a SCALAR ref while "strict refs" in use at /home/lpar2rrd/lpar2rrd/bin/KubernetesLoadDataModule.pm line 145.
    malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/lpar2rrd/lpar2rrd/bin/openshift-background.pl line 43.
    date end all   : Thu Apr  1 12:05:22 CEST 2021


    Can you help me?

    Best regards,



  • ls -l /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd
    rrdtool info /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd



  • Hi Pavel,

    This is de output:

    [lpar2rrd@lpar2rrd ~]$ ls -l /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd
    -rw-r--r-- 1 lpar2rrd lpar2rrd 0 Apr  1 04:19 /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd

    [lpar2rrd@lpar2rrd ~]$ rrdtool info /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd
    ERROR: mmaping file '/home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd': Invalid argument


  • remove the file, it has 0 size, perhaps you run out of the space in the FS

    it should resolve it if there is no other 0 size files

    ls -lR /home/lpar2rrd/lpar2rrd/data/Openshift/*/*| grep " 0 "

Sign In or Register to comment.