Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7798

mysql.server init script can't stop mysqld when WSREP is turned off

        Details

          Description

          If WSREP is turned off, the init script can't stop mysqld.

          We can start the server, then turn off WSREP:

          [gmontee@localhost ~]$ sudo service mysql bootstrap
Bootstrapping the cluster.. Starting MySQL....             [  OK  ]
[gmontee@localhost ~]$ mysql -u root --execute="SET GLOBAL wsrep_on=OFF;"

          Then try to stop the server, but it fails:

          [gmontee@localhost ~]$ sudo service mysql stop
Shutting down MySQL..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................[FAILED]..........................................

          The process is still running:

          [gmontee@localhost ~]$ ps -elf | grep "mysqld"
4 S root      6905     1  0  80   0 -  2868 wait   11:17 pts/1    00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/localhost.localdomain.pid --wsrep-new-cluster
4 S mysql     7168  6905  0  80   0 - 230398 futex_ 11:17 pts/1   00:00:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --wsrep-provider=/usr/lib64/galera/libgalera_smm.so --wsrep-new-cluster --log-error=/var/lib/mysql/localhost.localdomain.err --pid-file=/var/lib/mysql/localhost.localdomain.pid --wsrep_start_position=5edc131e-ae0b-11e4-85d1-c2c0512e9e7e:4
0 S gmontee   9202  3464  0  80   0 - 25814 pipe_w 11:33 pts/1    00:00:00 grep mysqld

          However, the server isn't usable:

          [gmontee@localhost ~]$ mysql -u root
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)

          The error log:

          [gmontee@localhost ~]$ sudo tail -n 20 /var/lib/mysql/localhost.localdomain.err
150318 11:17:54 InnoDB: Completed initialization of buffer pool
150318 11:17:54 InnoDB: highest supported file format is Barracuda.
150318 11:17:54  InnoDB: Waiting for the background threads to start
150318 11:17:55 Percona XtraDB (http://www.percona.com) 5.5.40-MariaDB-36.1 started; log sequence number 1604754
150318 11:17:55 [Note] Plugin 'FEEDBACK' is disabled.
150318 11:17:55 [Note] Server socket created on IP: '0.0.0.0'.
150318 11:17:55 [Note] Event Scheduler: Loaded 0 events
150318 11:17:55 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.41-MariaDB-wsrep'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server, wsrep_25.11.r4026
150318 11:17:55 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
150318 11:17:55 [Note] WSREP: REPL Protocols: 5 (3, 1)
150318 11:17:55 [Note] WSREP: Service thread queue flushed.
150318 11:17:55 [Note] WSREP: Assign initial position for certification: 4, protocol version: 3
150318 11:17:55 [Note] WSREP: Service thread queue flushed.
150318 11:17:55 [Note] WSREP: Synchronized with group, ready for connections
150318 11:17:55 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
150318 11:18:10 [Note] /usr/sbin/mysqld: Normal shutdown

150318 11:18:10 [Note] WSREP: rollbacker thread exiting
150318 11:18:10 [Note] Event Scheduler: Purging the queue. 0 events

          I have to kill -9 the server to stop it:

          [gmontee@localhost ~]$ sudo kill -9 7168
[gmontee@localhost ~]$ ps -elf | grep "mysqld"
0 S gmontee   9255  3464  0  80   0 - 25813 pipe_w 11:36 pts/1    00:00:00 grep mysqld

          If I turn WSREP back on before attempting to shut down the server, everything works normally:

          [gmontee@localhost ~]$ sudo service mysql bootstrap
Bootstrapping the cluster.. Starting MySQL....             [  OK  ]
[gmontee@localhost ~]$ mysql -u root --execute="SET GLOBAL wsrep_on=OFF;"
[gmontee@localhost ~]$ mysql -u root --execute="SET GLOBAL wsrep_on=ON;"
[gmontee@localhost ~]$ sudo service mysql stop
Shutting down MySQL...                                     [  OK  ]

            Gliffy Diagrams

              Attachments

                Activity

                Ascending order - Click to sort in descending order
                Hide
                Permalink
                GeoffMontee Geoff Montee added a comment -

                Attempting to stop the server via mysqladmin has the same effect:

                [gmontee@localhost ~]$ sudo service mysql bootstrap
Bootstrapping the cluster.. Starting MySQL....             [  OK  ]
[gmontee@localhost ~]$ mysql -u root --execute="SET GLOBAL wsrep_on=OFF;"
[gmontee@localhost ~]$ mysqladmin -u root shutdown
[gmontee@localhost ~]$ ps -elf | grep "mysqld"
4 S root     10122     1  0  80   0 -  2868 wait   11:58 pts/1    00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/localhost.localdomain.pid --wsrep-new-cluster
4 S mysql    10385 10122  0  80   0 - 230472 futex_ 11:58 pts/1   00:00:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --wsrep-provider=/usr/lib64/galera/libgalera_smm.so --wsrep-new-cluster --log-error=/var/lib/mysql/localhost.localdomain.err --pid-file=/var/lib/mysql/localhost.localdomain.pid --wsrep_start_position=5edc131e-ae0b-11e4-85d1-c2c0512e9e7e:4
0 S gmontee  10446  3464  0  80   0 - 25814 pipe_w 11:58 pts/1    00:00:00 grep mysqld
[gmontee@localhost ~]$ sudo tail -n 20 /var/lib/mysql/localhost.localdomain.err
150318 11:58:32 InnoDB: Completed initialization of buffer pool
150318 11:58:32 InnoDB: highest supported file format is Barracuda.
150318 11:58:32  InnoDB: Waiting for the background threads to start
150318 11:58:33 Percona XtraDB (http://www.percona.com) 5.5.40-MariaDB-36.1 started; log sequence number 1605062
150318 11:58:33 [Note] Plugin 'FEEDBACK' is disabled.
150318 11:58:33 [Note] Server socket created on IP: '0.0.0.0'.
150318 11:58:33 [Note] Event Scheduler: Loaded 0 events
150318 11:58:33 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.41-MariaDB-wsrep'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server, wsrep_25.11.r4026
150318 11:58:33 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
150318 11:58:33 [Note] WSREP: REPL Protocols: 5 (3, 1)
150318 11:58:33 [Note] WSREP: Service thread queue flushed.
150318 11:58:33 [Note] WSREP: Assign initial position for certification: 4, protocol version: 3
150318 11:58:33 [Note] WSREP: Service thread queue flushed.
150318 11:58:33 [Note] WSREP: Synchronized with group, ready for connections
150318 11:58:33 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
150318 11:58:47 [Note] /usr/sbin/mysqld: Normal shutdown

150318 11:58:47 [Note] WSREP: rollbacker thread exiting
150318 11:58:47 [Note] Event Scheduler: Purging the queue. 0 events
[gmontee@localhost ~]$ mysql -u root
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
                Show
                GeoffMontee Geoff Montee added a comment - Attempting to stop the server via mysqladmin has the same effect: [gmontee@localhost ~]$ sudo service mysql bootstrap Bootstrapping the cluster.. Starting MySQL.... [ OK ] [gmontee@localhost ~]$ mysql -u root --execute= "SET GLOBAL wsrep_on=OFF;" [gmontee@localhost ~]$ mysqladmin -u root shutdown [gmontee@localhost ~]$ ps -elf | grep "mysqld" 4 S root 10122 1 0 80 0 - 2868 wait 11:58 pts/1 00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/ var /lib/mysql --pid-file=/ var /lib/mysql/localhost.localdomain.pid --wsrep- new -cluster 4 S mysql 10385 10122 0 80 0 - 230472 futex_ 11:58 pts/1 00:00:00 /usr/sbin/mysqld --basedir=/usr --datadir=/ var /lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --wsrep-provider=/usr/lib64/galera/libgalera_smm.so --wsrep- new -cluster --log-error=/ var /lib/mysql/localhost.localdomain.err --pid-file=/ var /lib/mysql/localhost.localdomain.pid --wsrep_start_position=5edc131e-ae0b-11e4-85d1-c2c0512e9e7e:4 0 S gmontee 10446 3464 0 80 0 - 25814 pipe_w 11:58 pts/1 00:00:00 grep mysqld [gmontee@localhost ~]$ sudo tail -n 20 / var /lib/mysql/localhost.localdomain.err 150318 11:58:32 InnoDB: Completed initialization of buffer pool 150318 11:58:32 InnoDB: highest supported file format is Barracuda. 150318 11:58:32 InnoDB: Waiting for the background threads to start 150318 11:58:33 Percona XtraDB (http: //www.percona.com) 5.5.40-MariaDB-36.1 started; log sequence number 1605062 150318 11:58:33 [Note] Plugin 'FEEDBACK' is disabled. 150318 11:58:33 [Note] Server socket created on IP: '0.0.0.0'. 150318 11:58:33 [Note] Event Scheduler: Loaded 0 events 150318 11:58:33 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.41-MariaDB-wsrep' socket: '/ var /lib/mysql/mysql.sock' port: 3306 MariaDB Server, wsrep_25.11.r4026 150318 11:58:33 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 150318 11:58:33 [Note] WSREP: REPL Protocols: 5 (3, 1) 150318 11:58:33 [Note] WSREP: Service thread queue flushed. 150318 11:58:33 [Note] WSREP: Assign initial position for certification: 4, protocol version: 3 150318 11:58:33 [Note] WSREP: Service thread queue flushed. 150318 11:58:33 [Note] WSREP: Synchronized with group, ready for connections 150318 11:58:33 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 150318 11:58:47 [Note] /usr/sbin/mysqld: Normal shutdown 150318 11:58:47 [Note] WSREP: rollbacker thread exiting 150318 11:58:47 [Note] Event Scheduler: Purging the queue. 0 events [gmontee@localhost ~]$ mysql -u root ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/ var /lib/mysql/mysql.sock' (2)
                Hide
                Permalink
                nirbhay_c Nirbhay Choubey added a comment -

                http://lists.askmonty.org/pipermail/commits/2015-March/007661.html

                Show
                nirbhay_c Nirbhay Choubey added a comment - http://lists.askmonty.org/pipermail/commits/2015-March/007661.html
                Hide
                Permalink
                jplindst Jan Lindström added a comment -

                ok to push.

                Show
                jplindst Jan Lindström added a comment - ok to push.
                Hide
                Permalink
                nirbhay_c Nirbhay Choubey added a comment -

                https://github.com/MariaDB/server/commit/6a204546ef91aa90dedb6b47353bbb568f0531b0

                Show
                nirbhay_c Nirbhay Choubey added a comment - https://github.com/MariaDB/server/commit/6a204546ef91aa90dedb6b47353bbb568f0531b0

                  People

                  • Assignee:
                    nirbhay_c Nirbhay Choubey
                    Reporter:
                    GeoffMontee Geoff Montee
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    3 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: