Demande #4740
closedRevoir le paramégrage RAM de la vm agir
0%
Description
Cela fait maintenant une dizaine de jours que le nouveau paramétrage du swap est en place (#4664).
Pourtant il semble que la vm agir continue de swapper plus qu'attendu :
cpm@ocmstar (00:57:04) ~/…/admin/scripts 5 > ./swapshow.sh Swap Swap RAM Total Used Total (Mo) (Mo) % (Mo) agir 951 698 73 % 1238
Demande : analyser pourquoi ça swap et proposer une solution.
(le retour à 100% à la fin du graphique correspond à un reboot)
Files
Updated by Christian P. Momon about 4 years ago
Voyons ce qui encombre le swap :
(April) root@agir:/etc/gitea[master]# psswap |head mysqld 644 174312 kB systemd-journal 21118 62876 kB gitea 16083 61956 kB
J'en conclue que quelque chose prend de la place en mémoire qui pousse tout ça dans le swap.
En trifouillant top (touche 'f' pour ajouter les colonnes SWAP et USED, touche 'e' pour modifier l'unité) :
Tasks: 113 total, 2 running, 104 sleeping, 0 stopped, 7 zombie %Cpu(s): 0.7 us, 0.3 sy, 0.2 ni, 97.8 id, 1.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 1239.0 total, 144.7 free, 818.3 used, 275.9 buff/cache MiB Swap: 952.0 total, 618.8 free, 333.2 used. 355.2 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND SWAP USED 16083 gitea 20 0 2837.8m 272.3m 13.4m S 1.0 22.0 49:46.10 gitea 60.3m 332.6m 17893 www-data 20 0 518.5m 172.1m 8.7m S 0.0 13.9 1:26.46 ruby 4.9m 177.0m 12549 www-data 20 0 387.0m 130.8m 6.0m S 0.0 10.6 0:00.36 ruby 0.0m 130.8m 12529 www-data 20 0 255.2m 117.0m 17.5m S 0.0 9.4 0:02.40 ruby 0.0m 117.0m 644 mysql 20 0 1573.9m 101.6m 0.0m S 0.0 8.2 175:39.76 mysqld 162.8m 264.4m 17778 root 20 0 1064.2m 8.3m 3.8m S 0.0 0.7 0:13.15 PassengerAgent 1.5m 9.8m 9450 postfix 20 0 42.2m 7.3m 6.5m S 0.0 0.6 0:00.01 pickup 0.0m 7.3m
Rien de vraiment anormal, 332 +177 + 130 +117 +264 + 10 + 7 = 1037, ce qui doit pouvoir le faire avec 1239M de RAM.
Updated by Christian P. Momon about 4 years ago
En cherchant d'éventuel OOM Kill, je suis tombé sur :
(April) root@agir:/etc/gitea[master]# zgrep "Out of memory" /var/log/syslog* /var/log/syslog.141.gz:Apr 28 19:11:23 agir kernel: [590937.072410] Out of memory: Kill process 497 (gitea) score 610 or sacrifice child /var/log/syslog.119.gz:May 20 18:09:47 agir kernel: [881642.376108] Out of memory: Kill process 31993 (gitea) score 563 or sacrifice child /var/log/syslog.119.gz:May 20 18:10:16 agir kernel: [881671.395342] Out of memory: Kill process 31993 (gitea) score 563 or sacrifice child /var/log/syslog.106.gz:Jun 2 23:22:45 agir kernel: [2023599.778117] Out of memory: Kill process 3914 (gitea) score 499 or sacrifice child /var/log/syslog.92.gz: Jun 16 03:56:48 agir kernel: [427874.838786] Out of memory: Kill process 505 (gitea) score 531 or sacrifice child /var/log/syslog.87.gz: Jun 21 19:45:55 agir kernel: [916814.154429] Out of memory: Kill process 3118 (gitea) score 577 or sacrifice child /var/log/syslog.87.gz: Jun 21 19:48:09 agir kernel: [916947.348482] Out of memory: Kill process 3809 (gitea) score 501 or sacrifice child /var/log/syslog.87.gz: Jun 21 19:49:10 agir kernel: [917008.585841] Out of memory: Kill process 3831 (gitea) score 532 or sacrifice child /var/log/syslog.87.gz: Jun 21 19:50:05 agir kernel: [917061.859641] Out of memory: Kill process 3950 (gitea) score 567 or sacrifice child /var/log/syslog.87.gz: Jun 21 19:52:56 agir kernel: [917235.274212] Out of memory: Kill process 4427 (gitea) score 605 or sacrifice child /var/log/syslog.87.gz: Jun 21 19:55:49 agir kernel: [917407.711219] Out of memory: Kill process 5068 (gitea) score 542 or sacrifice child /var/log/syslog.87.gz: Jun 21 19:56:59 agir kernel: [917477.853177] Out of memory: Kill process 5601 (gitea) score 522 or sacrifice child /var/log/syslog.87.gz: Jun 21 19:58:29 agir kernel: [917567.928418] Out of memory: Kill process 5726 (gitea) score 478 or sacrifice child /var/log/syslog.87.gz: Jun 21 20:01:19 agir kernel: [917737.661714] Out of memory: Kill process 5933 (gitea) score 643 or sacrifice child /var/log/syslog.75.gz: Jul 3 19:53:17 agir kernel: [1954035.411466] Out of memory: Kill process 23099 (gitea) score 523 or sacrifice child /var/log/syslog.75.gz: Jul 3 19:53:27 agir kernel: [1954045.916354] Out of memory: Kill process 23099 (gitea) score 574 or sacrifice child /var/log/syslog.64.gz: Jul 14 09:22:55 agir kernel: [2866596.812501] Out of memory: Kill process 9354 (gitea) score 583 or sacrifice child /var/log/syslog.60.gz: Jul 18 06:30:09 agir kernel: [3201824.544533] Out of memory: Kill process 10481 (gitea) score 536 or sacrifice child /var/log/syslog.60.gz: Jul 18 18:55:03 agir kernel: [3246517.817505] Out of memory: Kill process 31311 (gitea) score 575 or sacrifice child /var/log/syslog.48.gz: Jul 30 04:42:46 agir kernel: [350795.729807] Out of memory: Kill process 503 (gitea) score 558 or sacrifice child /var/log/syslog.47.gz: Jul 31 22:08:57 agir kernel: [499964.704627] Out of memory: Kill process 13868 (gitea) score 587 or sacrifice child /var/log/syslog.42.gz: Aug 5 01:30:15 agir gitea[523]: fatal: Out of memory, malloc failed (tried to allocate 73394818 bytes) /var/log/syslog.40.gz: Aug 7 21:44:59 agir gitea[523]: fatal: Out of memory, malloc failed (tried to allocate 82456820 bytes) /var/log/syslog.40.gz: Aug 7 21:45:27 agir kernel: [501712.547840] Out of memory: Kill process 523 (gitea) score 607 or sacrifice child /var/log/syslog.28.gz: Aug 19 12:25:13 agir kernel: [1504881.119011] Out of memory: Kill process 17230 (gitea) score 524 or sacrifice child /var/log/syslog.28.gz: Aug 19 22:19:46 agir kernel: [1540553.717669] Out of memory: Kill process 17230 (gitea) score 520 or sacrifice child /var/log/syslog.27.gz: Aug 20 22:01:26 agir kernel: [1625843.628131] Out of memory: Kill process 15393 (gitea) score 570 or sacrifice child /var/log/syslog.23.gz: Aug 24 18:41:12 agir kernel: [1959432.888457] Out of memory: Kill process 14576 (gitea) score 499 or sacrifice child /var/log/syslog.19.gz: Aug 28 22:15:45 agir gitea[14576]: fatal: Out of memory, malloc failed (tried to allocate 82456820 bytes) /var/log/syslog.19.gz: Aug 28 22:16:31 agir kernel: [2317945.059618] Out of memory: Kill process 14576 (gitea) score 581 or sacrifice child /var/log/syslog.18.gz: Aug 29 20:55:12 agir kernel: [2399464.460851] Out of memory: Kill process 11642 (gitea) score 512 or sacrifice child /var/log/syslog.18.gz: Aug 29 20:55:24 agir kernel: [2399476.968879] Out of memory: Kill process 11642 (gitea) score 515 or sacrifice child /var/log/syslog.17.gz: Aug 30 07:45:03 agir kernel: [2438454.611207] Out of memory: Kill process 5911 (gitea) score 644 or sacrifice child /var/log/syslog.16.gz: Aug 31 04:01:37 agir kernel: [2511447.910023] Out of memory: Kill process 11931 (gitea) score 595 or sacrifice child /var/log/syslog.16.gz: Aug 31 19:45:04 agir kernel: [2568053.705629] Out of memory: Kill process 20039 (gitea) score 514 or sacrifice child /var/log/syslog.13.gz: Sep 3 01:46:17 agir kernel: [2762523.856579] Out of memory: Kill process 25278 (gitea) score 470 or sacrifice child /var/log/syslog.13.gz: Sep 3 01:47:07 agir kernel: [2762574.296537] Out of memory: Kill process 25278 (gitea) score 540 or sacrifice child /var/log/syslog.10.gz: Sep 6 17:37:08 agir gitea[5528]: error: Out of memory, malloc failed (tried to allocate 37095436 bytes) /var/log/syslog.10.gz: Sep 6 17:37:10 agir kernel: [3078770.453659] Out of memory: Kill process 5528 (gitea) score 479 or sacrifice child /var/log/syslog.6.gz: Sep 10 09:34:26 agir kernel: [3395399.447465] Out of memory: Kill process 20601 (gitea) score 650 or sacrifice child /var/log/syslog.4.gz: Sep 12 07:10:57 agir kernel: [3559589.503606] Out of memory: Kill process 10598 (gitea) score 534 or sacrifice child /var/log/syslog.4.gz: Sep 12 07:11:12 agir kernel: [3559604.615636] Out of memory: Kill process 10598 (gitea) score 595 or sacrifice child /var/log/syslog.4.gz: Sep 12 16:37:41 agir gitea[16727]: fatal: Out of memory, malloc failed (tried to allocate 82456820 bytes) /var/log/syslog.4.gz: Sep 12 16:38:00 agir kernel: [3593611.511849] Out of memory: Kill process 16727 (gitea) score 592 or sacrifice child /var/log/syslog.4.gz: Sep 12 22:15:12 agir kernel: [3613843.209350] Out of memory: Kill process 12159 (gitea) score 532 or sacrifice child /var/log/syslog: Sep 16 00:50:51 agir kernel: [3882378.501710] Out of memory: Kill process 16083 (gitea) score 476 or sacrifice child /var/log/syslog: Sep 16 00:52:35 agir kernel: [3882481.931408] Out of memory: Kill process 16083 (gitea) score 571 or sacrifice child
Donc il semble que fréquement (plusieurs fois par mois), cette vm manque de RAM.
Question : qui est le glouton ?
Updated by Christian P. Momon about 4 years ago
- Status changed from Nouveau to En cours de traitement
- Assignee set to Christian P. Momon
Pour information, pour Gitea il est recommandé 1Go de RAM. Comme d'autres applications tournent sur cette vm, on peut penser légitime d'augmenter un peu la RAM.
Proposition :- augmenter la RAM de 512 Mo.
Vos avis ?
Updated by Christian P. Momon about 4 years ago
- File swapagir.jpg swapagir.jpg added
- Description updated (diff)
Updated by Quentin Gibeaux about 4 years ago
- Target version changed from Septembre 2020 to Octobre 2020
Updated by Christian P. Momon about 4 years ago
Pendant la réunion de sprint, la question du total de mémoire utilisée sur le cluster a été posée.
Pour y répondre, création du script memkvm.sh
dans le dépôt git admin.
Avant :
cpm@ocmstar (04:03:29) ~/Dossiers/April/Adminsys/git/admin/scripts 196 > ./memkvm.sh KVM RAM adl 2304 Mo calamus agir 1280 Mo calamus bots 512 Mo calamus candidatsfr 1024 Mo calamus lamp 2048 Mo calamus pad 1024 Mo calamus pouet 2048 Mo calamus republique-numerique 512 Mo calamus scm 512 Mo calamus spip 1024 Mo calamus webchat 512 Mo calamus ====== Total 12800 Mo KVM RAM admin 1024 Mo virola bastion 768 Mo virola cms-dev 1024 Mo virola dns 1024 Mo virola drupal6 4096 Mo virola dtc 1024 Mo virola mail 2048 Mo virola mumble 512 Mo virola sympa 4096 Mo virola ====== Total 15616 Mo ====== TOTAL 28416 Mo
Updated by Christian P. Momon about 4 years ago
- Status changed from En cours de traitement to Résolu
Fait.
Après :
cpm@ocmstar (04:20:18) ~/Dossiers/April/Adminsys/git/admin/scripts 202 > ./memkvm.sh KVM RAM adl 2304 Mo calamus bots 512 Mo calamus candidatsfr 1024 Mo calamus lamp 2048 Mo calamus pad 1024 Mo calamus pouet 2048 Mo calamus republique-numerique 512 Mo calamus scm 512 Mo calamus spip 1024 Mo calamus webchat 512 Mo calamus agir 1792 Mo calamus ====== Total 13312 Mo KVM RAM admin 1024 Mo virola bastion 768 Mo virola cms-dev 1024 Mo virola dns 1024 Mo virola drupal6 4096 Mo virola dtc 1024 Mo virola mail 2048 Mo virola mumble 512 Mo virola sympa 4096 Mo virola ====== Total 15616 Mo ====== TOTAL 28928 Mo
Updated by Quentin Gibeaux about 4 years ago
- Status changed from Résolu to Fermé