Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-5289

master server starts slave parallel threads

    Details

      Description

      When you run this command in a slave
      set global slave_parallel_threads=10;
      you get a rightful error

      ERROR 1198 (HY000): This operation cannot be performed as you have a running slave ''; run STOP SLAVE '' first
      

      However, if you run the same command in a master, the statement is accepted, and the thread started.

      master [localhost] {msandbox} ((none)) > select ID, DB, state, time_ms, memory_used from information_schema .PROCESSLIST where USER='system user';
      Empty set (0.01 sec)
      
      master [localhost] {msandbox} ((none)) > set global slave_parallel_threads=10;
      Query OK, 0 rows affected (0.00 sec)
      
      master [localhost] {msandbox} ((none)) > select ID, DB, state, time_ms, memory_used from information_schema .PROCESSLIST where USER='system user';
      +----+------+----------------------------------+----------+-------------+
      | ID | DB   | state                            | time_ms  | memory_used |
      +----+------+----------------------------------+----------+-------------+
      | 47 | NULL | Waiting for work from SQL thread | 2207.683 |       34704 |
      | 46 | NULL | Waiting for work from SQL thread | 2207.686 |       34704 |
      | 45 | NULL | Waiting for work from SQL thread | 2207.722 |       34704 |
      | 44 | NULL | Waiting for work from SQL thread | 2207.723 |       34704 |
      | 43 | NULL | Waiting for work from SQL thread | 2207.754 |       34704 |
      | 42 | NULL | Waiting for work from SQL thread | 2207.757 |       34704 |
      | 41 | NULL | Waiting for work from SQL thread | 2207.765 |       34704 |
      | 40 | NULL | Waiting for work from SQL thread | 2207.801 |       34704 |
      | 39 | NULL | Waiting for work from SQL thread | 2207.843 |       34704 |
      | 38 | NULL | Waiting for work from SQL thread | 2207.851 |       34704 |
      +----+------+----------------------------------+----------+-------------+
      10 rows in set (0.01 sec)
      

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            datacharmer Giuseppe Maxia added a comment -

            I agree that MySQL 5.6 behavior is quite different.
            What I wanted to stress is that the server should not start any task related to slave role unless it is configured as such.
            This, is also how Tungsten Replicator works. You can configure a service to run parallel threads, but the threads are only enabled if the service role changes to 'slave'. When a slave is promoted to master, it shelves its additional threads and resumes working in a single thread.
            I would expect the same thing from any server. Tasks that are only valid for a slave should not be enabled by default when the service is running as master.

            Show
            datacharmer Giuseppe Maxia added a comment - I agree that MySQL 5.6 behavior is quite different. What I wanted to stress is that the server should not start any task related to slave role unless it is configured as such. This, is also how Tungsten Replicator works. You can configure a service to run parallel threads, but the threads are only enabled if the service role changes to 'slave'. When a slave is promoted to master, it shelves its additional threads and resumes working in a single thread. I would expect the same thing from any server. Tasks that are only valid for a slave should not be enabled by default when the service is running as master.
            Hide
            knielsen Kristian Nielsen added a comment -

            Right, so the pool of threads could magically spring into life upon the first CHANGE MASTER statement, and be teared down when the last slave is removed by RESET SLAVE ALL.

            But I do not have bandwidth to work on that kind of stuff...

            Show
            knielsen Kristian Nielsen added a comment - Right, so the pool of threads could magically spring into life upon the first CHANGE MASTER statement, and be teared down when the last slave is removed by RESET SLAVE ALL. But I do not have bandwidth to work on that kind of stuff...
            Hide
            knielsen Kristian Nielsen added a comment - - edited

            I changed my mind a bit, would be nice to not have the threads started unnecessarily, even if they are not harmful as such. So re-opening.

            Show
            knielsen Kristian Nielsen added a comment - - edited I changed my mind a bit, would be nice to not have the threads started unnecessarily, even if they are not harmful as such. So re-opening.
            Hide
            knielsen Kristian Nielsen added a comment -

            Idea for how to implement this in a way that works with the current code without introducing extra locking overhead:

            1. Do not start the worker threads in the pool at server start.

            2. In start_slave_threads(), first startup the worker threads, if we did not already.

            3. At the end of handle_slave_sql(), if we are shutting down the last SQL driver thread, shut down the pool of worker threads also.

            And do the similar thing for the slave background thread also. This has to be started briefly during server startup (to load mysql.gtid_slave_pos), but we can at that point stop it again, and then start and stop it along with the worker threads in the pool.

            Show
            knielsen Kristian Nielsen added a comment - Idea for how to implement this in a way that works with the current code without introducing extra locking overhead: 1. Do not start the worker threads in the pool at server start. 2. In start_slave_threads(), first startup the worker threads, if we did not already. 3. At the end of handle_slave_sql(), if we are shutting down the last SQL driver thread, shut down the pool of worker threads also. And do the similar thing for the slave background thread also. This has to be started briefly during server startup (to load mysql.gtid_slave_pos), but we can at that point stop it again, and then start and stop it along with the worker threads in the pool.
            Hide
            knielsen Kristian Nielsen added a comment -

            Pushed to 10.0.18:

            http://lists.askmonty.org/pipermail/commits/2015-March/007562.html

            With this patch, parallel replication worker threads are not spawned until
            needed (when an SQL thread is started), and they will be de-spawned if all SQL
            threads are stopped.

            Thanks, Giuseppe, for reporting this!

            Show
            knielsen Kristian Nielsen added a comment - Pushed to 10.0.18: http://lists.askmonty.org/pipermail/commits/2015-March/007562.html With this patch, parallel replication worker threads are not spawned until needed (when an SQL thread is started), and they will be de-spawned if all SQL threads are stopped. Thanks, Giuseppe, for reporting this!

              People

              • Assignee:
                knielsen Kristian Nielsen
                Reporter:
                datacharmer Giuseppe Maxia
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: