Anomalie #6013
https://wiki.april.org/ en panne
Début:
07/11/2022
Echéance:
% réalisé:
0%
Temps estimé:
Difficulté:
2 Facile
Description
Ce matin, après plusieurs erreurs de connexion 50x, https://wiki.april.org/ a fini pour afficher l'erreur :
MediaWiki internal error.
Original exception: [38f05b5bde8d7fa4163550b3] 2022-11-07 09:02:40: Fatal exception of type "Wikimedia\Rdbms\DBConnectionError"
Exception caught inside exception handler.
Set $wgShowExceptionDetails = true; at the bottom of LocalSettings.php to show detailed debugging information.
La partie entre crochets changeait à chaque fois que j'essayais de charger la page https://wiki.april.org/.
Historique
Mis à jour par Pierre-Louis Bonicoli il y a plus d'un an
Les logs apache (/var/log/apache2/wiki.april.org/wiki.april.org-access.log.2.gz) indiquent que l'erreur (erreur HTTP 500) a commencé a 07/Nov/2022:10:02:26 +0100 et s'est terminée vers 07/Nov/2022:10:06:57.
Le service mariadb fournissant la base de données pour le wiki a été redémarré à ce moment:
Oct 27 00:08:59 lamp /etc/mysql/debian-start[800]: Upgrading MySQL tables if necessary. Nov 07 10:02:40 lamp systemd[1]: mariadb.service: A process of this unit has been killed by the OOM killer. Nov 07 10:02:40 lamp systemd[1]: mariadb.service: Main process exited, code=killed, status=9/KILL Nov 07 10:02:40 lamp systemd[1]: mariadb.service: Failed with result 'oom-kill'. Nov 07 10:02:41 lamp systemd[1]: mariadb.service: Consumed 1h 31min 54.586s CPU time. Nov 07 10:06:53 lamp systemd[1]: Starting MariaDB 10.5.15 database server... Nov 07 10:06:54 lamp mariadbd[962402]: 2022-11-07 10:06:54 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.15-MariaDB-0+deb11u1) starting as process 962402 ...
Les logs systèmes:
Nov 07 10:02:24 lamp kernel: apache2 invoked oom-killer: gfp_mask=0x100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0 Nov 07 10:02:41 lamp kernel: CPU: 3 PID: 961925 Comm: apache2 Not tainted 5.10.0-19-amd64 #1 Debian 5.10.149-2 Nov 07 10:02:41 lamp kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014 [...] ov 07 10:02:41 lamp kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Nov 07 10:02:41 lamp kernel: 57751 total pagecache pages Nov 07 10:02:41 lamp kernel: 28467 pages in swap cache Nov 07 10:02:41 lamp kernel: Swap cache stats: add 670045, delete 641580, find 4796484/4872632 Nov 07 10:02:41 lamp kernel: Free swap = 0kB Nov 07 10:02:41 lamp kernel: Total swap = 974844kB Nov 07 10:02:41 lamp kernel: 524150 pages RAM Nov 07 10:02:41 lamp kernel: 0 pages HighMem/MovableOnly Nov 07 10:02:41 lamp kernel: 16657 pages reserved Nov 07 10:02:41 lamp kernel: 0 pages hwpoisoned Nov 07 10:02:41 lamp kernel: Tasks state (memory values in pages): Nov 07 10:02:41 lamp kernel: [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name Nov 07 10:02:41 lamp kernel: [ 266] 0 266 5563 28 69632 416 -1000 systemd-udevd Nov 07 10:02:41 lamp kernel: [ 532] 0 532 604 0 45056 31 0 acpid Nov 07 10:02:41 lamp kernel: [ 534] 0 534 937 24 49152 39 0 cron Nov 07 10:02:41 lamp kernel: [ 535] 105 535 1998 67 57344 88 -900 dbus-daemon Nov 07 10:02:41 lamp kernel: [ 542] 0 542 55239 104 77824 354 0 rsyslogd Nov 07 10:02:41 lamp kernel: [ 549] 0 549 3437 110 61440 126 0 systemd-logind Nov 07 10:02:41 lamp kernel: [ 563] 0 563 1541 0 45056 33 0 agetty Nov 07 10:02:41 lamp kernel: [ 592] 108 592 18625 39 57344 136 0 ntpd Nov 07 10:02:41 lamp kernel: [ 637] 111 637 677053 67298 2138112 157828 0 mariadbd Nov 07 10:02:41 lamp kernel: [ 685] 0 685 7508 268 90112 1522 0 unattended-upgr Nov 07 10:02:41 lamp kernel: [ 757] 0 757 10013 29 73728 133 0 master Nov 07 10:02:41 lamp kernel: [ 759] 109 759 10052 54 61440 125 0 qmgr Nov 07 10:02:41 lamp kernel: [ 811] 110 811 98623 7 135168 453 0 icinga2 Nov 07 10:02:41 lamp kernel: [ 910] 110 910 412900 1085 241664 2729 0 icinga2 Nov 07 10:02:41 lamp kernel: [ 919] 110 919 98623 26 126976 436 0 icinga2 Nov 07 10:02:41 lamp kernel: [ 248387] 0 248387 3339 16 61440 228 -1000 sshd Nov 07 10:02:41 lamp kernel: [ 833348] 0 833348 15169 107 155648 118 -250 systemd-journal Nov 07 10:02:41 lamp kernel: [ 833348] 0 833348 15169 107 155648 118 -250 systemd-journal Nov 07 10:02:41 lamp kernel: [ 920792] 0 920792 166087 5500 258048 1369 0 apache2 Nov 07 10:02:41 lamp kernel: [ 920793] 33 920793 12978 314 77824 525 0 apache2 Nov 07 10:02:41 lamp kernel: [ 956680] 109 956680 10019 31 73728 123 0 pickup Nov 07 10:02:41 lamp kernel: [ 961460] 33 961460 172071 22474 491520 4470 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961576] 33 961576 172061 20921 483328 6246 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961638] 33 961638 189817 27698 565248 6615 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961640] 33 961640 185774 28811 524288 3367 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961667] 33 961667 170049 20861 454656 4492 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961674] 33 961674 171932 19215 479232 6027 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961783] 33 961783 169351 18199 413696 1528 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961790] 33 961790 168893 16715 417792 3383 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961812] 33 961812 185763 23520 499712 3370 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961814] 33 961814 185747 30582 532480 3871 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961881] 33 961881 168709 10091 348160 2597 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961882] 33 961882 169263 12861 364544 1705 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961883] 33 961883 185487 21095 458752 1809 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961884] 33 961884 168268 11316 356352 2092 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961885] 33 961885 185649 23443 479232 2095 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961886] 33 961886 168283 12004 344064 1489 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961887] 33 961887 168735 12246 352256 1588 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961895] 33 961895 168680 10082 348160 2582 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961896] 33 961896 168069 8999 360448 1823 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961897] 33 961897 168777 12210 344064 1454 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961914] 33 961914 185476 20689 454656 1804 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961915] 33 961915 185476 24046 483328 1763 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961916] 33 961916 185478 21090 466944 2544 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961922] 33 961922 168080 5292 323584 1823 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961923] 33 961923 185476 18274 442368 2027 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961924] 33 961924 185476 25022 491520 1862 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961925] 33 961925 185476 21879 466944 1849 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961926] 33 961926 185476 21588 471040 2341 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961927] 33 961927 185476 21287 462848 1867 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961928] 33 961928 185478 20813 462848 2308 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961929] 33 961929 168715 10964 344064 2265 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961930] 33 961930 168712 10114 339968 2612 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961931] 33 961931 170563 13003 360448 3040 0 apache2 Nov 07 10:02:41 lamp kernel: [ 961932] 33 961932 168670 10103 348160 2552 0 apache2 [...] Nov 07 10:02:41 lamp kernel: Out of memory: Killed process 637 (mariadbd) total-vm:2708212kB, anon-rss:269192kB, file-rss:0kB, shmem-rss:0kB, UID:111 pgtables:2088kB oom_sc> Nov 07 10:02:41 lamp kernel: oom_reaper: reaped process 637 (mariadbd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB Nov 07 10:02:40 lamp systemd[1]: mariadb.service: A process of this unit has been killed by the OOM killer. Nov 07 10:02:34 lamp sshd[962059]: Connection from 172.16.0.1 port 41376 on 172.16.0.11 port 22 rdomain "" Nov 07 10:02:41 lamp icinga2[910]: [2022-11-07 10:02:29 +0100] information/RemoteCheckQueue: items: 0, rate: 0/s (12/min 60/5min 180/15min); Nov 07 10:02:41 lamp icinga2[910]: [2022-11-07 10:02:39 +0100] information/RemoteCheckQueue: items: 0, rate: 0/s (12/min 60/5min 180/15min); Nov 07 10:02:41 lamp icinga2[910]: [2022-11-07 10:02:40 +0100] information/Application: We jumped forward in time: 18.6335 seconds Nov 07 10:02:40 lamp systemd[1]: mariadb.service: Main process exited, code=killed, status=9/KILL Nov 07 10:02:34 lamp sshd[962059]: Connection closed by 172.16.0.1 port 41376 [preauth] Nov 07 10:02:40 lamp systemd[1]: mariadb.service: Failed with result 'oom-kill'. Nov 07 10:02:41 lamp systemd[1]: mariadb.service: Consumed 1h 31min 54.586s CPU time.