Memory Consumption Linux without filesystem cache/buffers

master07

Hi,

on Linux the lpar2rrd memory value shows the free memory from the /usr/bin/free command:

$ free -m

             total       used       free     shared    buffers     cached

Mem:        386750     385031       1719        268     218095       6884

-/+ buffers/cache:     160051     226699

Swap:        12287          1      12286

The System is just fine, because we have about 226699 MB free Memory (included Buffers and Cached)

According to this https://www.linuxatemyram.com/
the free memory should include the buffers and cache values.

What do you think, is it possible to change that free memory query in the lpar2rrd client?!?

Thanks!

Pavel

Hi,

as per me FS cache is "cached".
man free --> ... buffers used by the kernel ...

what is exactly wrong? We do not count with bufferes

master07

Hi Pavel,

the point is, that the "used capacity" in the free command includes also the buffers and cached space used. Cached Memory is good so we can reduce disk I/O but that also means that if an application needs more memory it uses the amount of memory from the cache/bufferd capacity and/or from the free space. If the application needs any more memory it got only a problem when the free capacity and the cache/buffered capacity is very low. in lpar2rrd the used memory included that cache/buffered capacity which is usable for the application and in our case it looks like that the application got a problem because there is not enough free space, which is wrong because the memory is in cache or buffered.

So my point is, is it possible to include the buffered and cache amount of memory in the diagram. I think it would useful for a lot of other customers too, because the cache amount is a very important point when memory consumption is displayed.

I hope you understand now my concerns and sorry for the bad english.

Thanks.

Pavel

Hi,

cache is included as FS cache.
Buffers is not included as some OS do not provide separate value for it.

master07

Hi Pavel,

how lpar2rrd get the FS cache amount? Maybe we understand then better the values.

thanks.

Pavel

It should be "cached" value, but it does not pass on you example.
What OS agent version do you use?
rpm -qa| grep lpa2rrrd

master07

lpar2rrd-agent-5.05-4.noarch
lpar2rrd server: 5.05
oslevel: SUSE Linux Enterprise Server 12 SP2

Pavel

this output:
free -m
tail -1 /var/tmp/lpar2rrd*txt

master07

root@lnxa:~
$ tail -n1  /var/tmp/lpar2rrd-agent-*txt
==> /var/tmp/lpar2rrd-agent-192.168.121.16-root-ps_job.txt <==
32685,root,/usr/sbin/cmirrord,00:00:00,36312,1868

==> /var/tmp/lpar2rrd-agent-192.168.121.16-root.txt <==
Linux:lnxa:DE3D0F39-6BDC-E14E-86D3-6F15148C296C:1513169761:Wed Dec 13 13:56:01 2017 version 5.05-4:|::::mem:::396032824:279519816:116513008278267276:1252540:pgs:::26:14902:12287.99609375:0.4:::lan:br_vl2200:192.168.200.98:21226648447:1648610098:::::lan:eth0:192.168.121.38:13306815309:6994387349:::::lan:eth1:10.1.139.38:5266449026:5960974201:::::cpu:::0:4:8:0:::san:host0::178378008092:678286952492:406820676:227729985:::san:host1::178287079316:665119639264:400087655:226763279::

==> /var/tmp/lpar2rrd-agent-nmon-192.168.121.16-root-time_file.txt <==
SRV_OS-like-Linux_LPR_lnxa_TIME_1513169110_XOR_lnxa_171213_0000.nmon
root@lnxa:~
$ free -m
             total       used       free     shared    buffers     cached
Mem:        386750     273015     113734        356     109541       3628
-/+ buffers/cache:     159845     226905
Swap:        12287         54      12233
root@lnxa:~
$

Pavel

ok, I have just reminded looking in the code that we have do not use free cmd anymore.
we use this, pls provide output:
cat /proc/meminfo

master07

Hi Pavel,

okay, then this makes sense because in the /proc/meminfo it looks like this:

$ cat /proc/meminfo 

MemTotal:       792390272 kB

MemFree:         2447200 kB

MemAvailable:   467149264 kB

Buffers:        461943416 kB

Cached:          1816108 kB

SwapCached:       108816 kB

Active:         543459932 kB

Inactive:       235166272 kB

But the reason for my concerns are still the same. The MemAvailable is about 445GB but in the diagram above it looks like we got a problem and run out of memory.

Pavel

I do not understand that.
free -m shows total memory  386GB and /proc/meminfo  456G

master07

I'm really sorry. I have posted some data from the wrong linux server. Here is the output from the correct server:

$ free -m
             total       used       free     shared    buffers     cached
Mem:        773818     770715       3102       1010     450406       8593
-/+ buffers/cache:     311715     462103
Swap:        24607       2219      22388

and the cat /proc/meminfo output is the same as above:

$ cat /proc/meminfo
MemTotal:       792390272 kB
MemFree:         3175696 kB
MemAvailable:   467037064 kB
Buffers:        461214000 kB
Cached:          1842952 kB
SwapCached:       106356 kB
Active:         527484560 kB
Inactive:       250532152 kB
Active(anon):   289523484 kB
Inactive(anon): 26487796 kB
Active(file):   237961076 kB
Inactive(file): 224044356 kB

sorry for the confusion

Pavel

psl attache screenshot from the GUI of that lpar as well to see it, I suppose it is different lpar than from example on the start of that thread.

master07

sure, here is the diagram screenshot from the memory consuption, that confuses us:

Pavel

ok, you can see difference.
1. we and /proc/meminfo says that FS cache is
Cached:          1842952 kB
2. free cmd says: 8593MB

What is right and why?

master07

it's difficult to say.
in my view it would be nice to now what is the amount of buffer and what is the memory available like in the /proc/meminfo

MemAvailable:   467037064 kB

I think it describes our point of view in the Linus Torvalds git commit as best:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=34e431b0ae398fc54ea69ff85ec700722c9da773

+MemAvailable: An estimate of how much memory is available for starting new

+              applications, without swapping. Calculated from MemFree,

+              SReclaimable, the size of the file LRU lists, and the low

+              watermarks in each zone.

+              The estimate takes into account that the system needs some

+              page cache to function well, and that not all reclaimable

+              slab will be reclaimable, due to items being in use. The

+              impact of those factors will vary from system to system.

Pavel

Upgrade to this version, we have added buffers to cached, it shoudl be better now

http://www.lpar2rrd.com/download/lpar2rrd-agent-5.05-5.noarch.rpm

master07

Hi Pavel,

after I have upgrade yesterday at about 16:00, it looks much better:

could you please provide us the rpms also for Power on Linux, AIX and so on.

Thank you!

Pavel

there is no change on AIX.
That rpm will work even on Power Linux.

Pavel

it has been released even on usuall download page: http://www.lpar2rrd.com/download.htm

master07

thanks a lot