OpenShift - Growth of folder /home/lpar2rrd/lpar2rrd/data/OpenShift

aixtor
aixtor
in LPAR2RRD
Hi,

Since I have added the OpenShift platform to the LPAR2RRD, the growth of the / filesystem is out of control (111GB in just 2 weeks).

The folder with the most occupied space is the following: /home/lpar2rrd/lpar2rrd/data/OpenShift

Is this occupation normal? How can we solve it?

Best regards,

Comments

  • Matej
    Matej
    Hi,

    RRD allocates the total capacity at creation, so there will be no increase in size. The only thing that could increase the size are new pods and containers for monitoring.

    But we did a fix to better clean up the data of old pods and containers.
    https://www.lpar2rrd.com/download/lpar2rrd-7.10-8.tar

    It is now possible to skip container monitoring and monitor only nodes and pods.



  • aixtor
    aixtor
    Hi,

    We have updated lpar2rrd to 7.10.8, but now, no performance statistics are collected for OpenShift.

    The following error appears:

    $ tail -24 /home/lpar2rrd/lpar2rrd/logs/error.log-openshift

    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Openshift.pm line 70.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Openshift.pm line 70.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Kubernetes.pm line 517.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Openshift.pm line 70.
    Unrecognized LWP::UserAgent options: ssl_opts at /home/lpar2rrd/lpar2rrd/bin/Openshift.pm line 70.
    Thu Apr  1 12:05:22 2021: Empty last.json : Bad file descriptor
    Use of uninitialized value $RRDp::error_mode in string eq at /usr/lib64/perl5/vendor_perl/RRDp.pm line 168.
    Thu Apr  1 12:05:22 2021: Failed during read last time /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd: ERROR: mmaping file '/home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd': Invalid argument at /home/lpar2rrd/lpar2rrd/bin/KubernetesLoadDataModule.pm line 37.
     /home/lpar2rrd/lpar2rrd/bin/KubernetesLoadDataModule.pm:41 at /home/lpar2rrd/lpar2rrd/bin/KubernetesLoadDataModule.pm line 41.
    Can't use string ("1") as a SCALAR ref while "strict refs" in use at /home/lpar2rrd/lpar2rrd/bin/KubernetesLoadDataModule.pm line 145.
    malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/lpar2rrd/lpar2rrd/bin/openshift-background.pl line 43.
    date end all   : Thu Apr  1 12:05:22 CEST 2021


    Can you help me?

    Best regards,



  • Pavel
    Pavel
    ls -l /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd
    rrdtool info /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd



  • aixtor
    aixtor
    Hi Pavel,

    This is de output:

    [lpar2rrd@lpar2rrd ~]$ ls -l /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd
    -rw-r--r-- 1 lpar2rrd lpar2rrd 0 Apr  1 04:19 /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd

    [lpar2rrd@lpar2rrd ~]$ rrdtool info /home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd
    ERROR: mmaping file '/home/lpar2rrd/lpar2rrd/data/Openshift/Container/service-ca-operator-6845757879-45wb8--operator.rrd': Invalid argument


  • Pavel
    Pavel
    remove the file, it has 0 size, perhaps you run out of the space in the FS

    it should resolve it if there is no other 0 size files

    ls -lR /home/lpar2rrd/lpar2rrd/data/Openshift/*/*| grep " 0 "

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Categories

In this Discussion