Memory Consumption Linux without filesystem cache/buffers

Hi,

on Linux the lpar2rrd memory value shows the free memory from the /usr/bin/free command:

$ free -m
             total       used       free     shared    buffers     cached
Mem:        386750     385031       1719        268     218095       6884
-/+ buffers/cache:     160051     226699
Swap:        12287          1      12286



The System is just fine, because we have about 226699 MB free Memory (included Buffers and Cached)

According to this https://www.linuxatemyram.com/
the free memory should include the buffers and cache values.

What do you think, is it possible to change that free memory query in the lpar2rrd client?!?

Thanks!




Comments

  • Hi,

    as per me FS cache is "cached".
    man free --> ... buffers used by the kernel ...

    what is exactly wrong? We do not count with bufferes


  • Hi Pavel,

    the point is, that the "used capacity" in the free command includes also the buffers and cached space used. Cached Memory is good so we can reduce disk I/O but that also means that if an application needs more memory it uses the amount of memory from the cache/bufferd capacity and/or from the free space. If the application needs any more memory it got only a problem when the free capacity and the cache/buffered capacity is very low. in lpar2rrd the used memory included that cache/buffered capacity which is usable for the application and in our case it looks like that the application got a problem because there is not enough free space, which is wrong because the memory is in cache or buffered.

    So my point is, is it possible to include the buffered and cache amount of memory in the diagram. I think it would useful for a lot of other customers too, because the cache amount is a very important point when memory consumption is displayed.

    I hope you understand now my concerns and sorry for the bad english.

    Thanks.

  • Hi,

    cache is included as FS cache.
    Buffers is not included as some OS do not provide separate value for it.
  • master07
    edited December 2017
    Hi Pavel,

    how lpar2rrd get the FS cache amount? Maybe we understand then better the values.

    thanks.
  • It should be "cached" value, but it does not pass on you example.
    What OS agent version do you use?
    rpm -qa| grep lpa2rrrd
  • master07
    edited December 2017
    lpar2rrd-agent-5.05-4.noarch
    lpar2rrd server: 5.05
    oslevel: SUSE Linux Enterprise Server 12 SP2

  • this output:
    free -m
    tail -1 /var/tmp/lpar2rrd*txt
  • root@lnxa:~
    $ tail -n1  /var/tmp/lpar2rrd-agent-*txt
    ==> /var/tmp/lpar2rrd-agent-192.168.121.16-root-ps_job.txt <==
    32685,root,/usr/sbin/cmirrord,00:00:00,36312,1868

    ==> /var/tmp/lpar2rrd-agent-192.168.121.16-root.txt <==
    Linux:lnxa:DE3D0F39-6BDC-E14E-86D3-6F15148C296C:1513169761:Wed Dec 13 13:56:01 2017 version 5.05-4:|::::mem:::396032824:279519816:116513008:-1:278267276:1252540:pgs:::26:14902:12287.99609375:0.4:::lan:br_vl2200:192.168.200.98:21226648447:1648610098:::::lan:eth0:192.168.121.38:13306815309:6994387349:::::lan:eth1:10.1.139.38:5266449026:5960974201:::::cpu:::0:4:8:0:::san:host0::178378008092:678286952492:406820676:227729985:::san:host1::178287079316:665119639264:400087655:226763279::

    ==> /var/tmp/lpar2rrd-agent-nmon-192.168.121.16-root-time_file.txt <==
    SRV_OS-like-Linux_LPR_lnxa_TIME_1513169110_XOR_lnxa_171213_0000.nmon
    root@lnxa:~
    $ free -m
                 total       used       free     shared    buffers     cached
    Mem:        386750     273015     113734        356     109541       3628
    -/+ buffers/cache:     159845     226905
    Swap:        12287         54      12233
    root@lnxa:~
    $


  • ok, I have just reminded looking in the code that we have do not use free cmd anymore.
    we use this, pls provide output:
    cat /proc/meminfo


  • Hi Pavel,

    okay, then this makes sense because in the /proc/meminfo it looks like this:
    $ cat /proc/meminfo 
    MemTotal:       792390272 kB
    MemFree:         2447200 kB
    MemAvailable:   467149264 kB
    Buffers:        461943416 kB
    Cached:          1816108 kB
    SwapCached:       108816 kB
    Active:         543459932 kB
    Inactive:       235166272 kB

    But the reason for my concerns are still the same. The MemAvailable is about 445GB but in the diagram above it looks like we got a problem and run out of memory.

  • I do not understand that.
    free -m shows total memory  386GB and /proc/meminfo  456G

  • I'm really sorry. I have posted some data from the wrong linux server. Here is the output from the correct server:

    $ free -m
                 total       used       free     shared    buffers     cached
    Mem:        773818     770715       3102       1010     450406       8593
    -/+ buffers/cache:     311715     462103
    Swap:        24607       2219      22388

    and the cat /proc/meminfo output is the same as above:
    $ cat /proc/meminfo
    MemTotal:       792390272 kB
    MemFree:         3175696 kB
    MemAvailable:   467037064 kB
    Buffers:        461214000 kB
    Cached:          1842952 kB
    SwapCached:       106356 kB
    Active:         527484560 kB
    Inactive:       250532152 kB
    Active(anon):   289523484 kB
    Inactive(anon): 26487796 kB
    Active(file):   237961076 kB
    Inactive(file): 224044356 kB

    sorry for the confusion

  • psl attache screenshot from the GUI of that lpar as well to see it, I suppose it is different lpar than from example on the start of that thread.
  • sure, here is the diagram screenshot from the memory consuption, that confuses us:

  • ok, you can see difference.
    1. we and /proc/meminfo says that FS cache is
    Cached:          1842952 kB
    2. free cmd says: 8593MB

    What is right and why?

  • it's difficult to say.
    in my view it would be nice to now what is the amount of buffer and what is the memory available like in the /proc/meminfo
    MemAvailable:   467037064 kB

    I think it describes our point of view in the Linus Torvalds git commit as best:

    https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=34e431b0ae398fc54ea69ff85ec700722c9da773


    +MemAvailable: An estimate of how much memory is available for starting new
    +              applications, without swapping. Calculated from MemFree,
    +              SReclaimable, the size of the file LRU lists, and the low
    +              watermarks in each zone.
    +              The estimate takes into account that the system needs some
    +              page cache to function well, and that not all reclaimable
    +              slab will be reclaimable, due to items being in use. The
    +              impact of those factors will vary from system to system.



  • Upgrade to this version, we have added buffers to cached, it shoudl be better now

    http://www.lpar2rrd.com/download/lpar2rrd-agent-5.05-5.noarch.rpm

  • Hi Pavel,

    after I have upgrade yesterday at about 16:00, it looks much better:


    could you please provide us the rpms also for Power on Linux, AIX and so on.


    Thank you!

  • there is no change on AIX.
    That rpm will work even on Power Linux.
  • it has been released even on usuall download page: http://www.lpar2rrd.com/download.htm

  • thanks a lot
Sign In or Register to comment.