Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7004

Merge scalability fixes from 10.0-power

    Details

    • Type: Task
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Fix Version/s: 10.1.2
    • Component/s: OTHER
    • Labels:
      None

      Description

      Please review scalability fixes from 10.0-power. Since fixes are ongoing, better to review them at some point close to 10.1 feature freeze.

      At the moment of writing in certain cases 10.0-power shows about 60% better throughput compared to plain 10.0 (17kTPS -> 27kTPS) on Power8.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

              Hide
              svoj Sergey Vojtovich added a comment -

              Raising priority to blocker to make sure these patches will be pushed into the next 10.1 release.

              Show
              svoj Sergey Vojtovich added a comment - Raising priority to blocker to make sure these patches will be pushed into the next 10.1 release.
              Hide
              svoj Sergey Vojtovich added a comment -

              Pushed most of the fixes to 10.1.2:

              commit 78564373fee5e6cccf144b11bc60b0876b4bbd0b
              Author: Monty <monty@mariadb.org>
              Date:   Fri Aug 29 14:07:43 2014 +0300
              
                  my_alloc.c
                  - Changed 0x%lx -> %p
                  array.c:
                  - Static (preallocated) buffer can now be anywhere
                  my_sys.h
                  - Define MY_INIT_BUFFER_USED
                  sql_delete.cc & sql_lex.cc
                  - Use memroot when allocating classes (avoids call to current_thd)
                  sql_explain.h:
                  - Use preallocated buffers
                  sql_explain.cc:
                  - Use preallocated buffers and memroot
                  sql_select.cc:
                  - Use multi_alloc_root() instead of many alloc_root()
                  - Update calls to Explain
              
              commit 3392278c869a9594f13a0b956d34efb600ab0ed0
              Author: Sergey Vojtovich <svoj@mariadb.org>
              Date:   Thu Dec 4 17:44:46 2014 +0400
              
                  MDEV-7004 - Merge scalability fixes from 10.0-power
              
                  Preallocate dynamic array and bitmap on mem_root to avoid expensive malloc.
                  This reduces number of allocations from 39 to 31 per OLTP RO transaction.
              
              commit eaa8c154e81cb3a9a2623f53482fe2bbd9ffa459
              Author: Sergey Vojtovich <svoj@mariadb.org>
              Date:   Thu Dec 4 17:42:32 2014 +0400
              
                  MDEV-7004 - Merge scalability fixes from 10.0-power
              
                  Preallocate dynamic array on THD mem_root to avoid expensive malloc.
              
              commit 070a6e7a890e5a73eb38920657c060605e520b55
              Author: Monty <monty@mariadb.org>
              Date:   Fri Aug 29 14:09:51 2014 +0300
              
                  - Changed default values of query_prealloc_size and query_alloc_block_size
                  so that a simple query with one join would not have to call my_malloc.
                  - Allow lower limites for query_prealloc_size for testing.
                  - Fixed wrong initialization of trans_alloc_block_size
              
              commit 9127784d5ce845c9d66a34eee371cbaf82468d89
              Author: Sergey Vojtovich <svoj@mariadb.org>
              Date:   Thu Dec 4 17:35:55 2014 +0400
              
                  Cherry pick dynamic array changes from commit:
              
                  commit 85fd3d901311688e18ffce92ffc78129e5625791
                  Author: Monty <monty@mariadb.org>
                  Date:   Fri Aug 29 14:07:43 2014 +0300
              
                      my_alloc.c
                      - Changed 0x%lx -> %p
                      array.c:
                      - Static (preallocated) buffer can now be anywhere
                      my_sys.h
                      - Define MY_INIT_BUFFER_USED
                      sql_delete.cc & sql_lex.cc
                      - Use memroot when allocating classes (avoids call to current_thd)
                      sql_explain.h:
                      - Use preallocated buffers
                      sql_explain.cc:
                      - Use preallocated buffers and memroot
                      sql_select.cc:
                      - Use multi_alloc_root() instead of many alloc_root()
                      - Update calls to Explain
              
              commit 974808772b63563d031f687fd0fd158f83eb94aa
              Author: Sergey Vojtovich <svoj@mariadb.org>
              Date:   Tue Dec 2 15:03:35 2014 +0400
              
                  MDEV-7004 - Merge scalability fixes from 10.0-power
              
                  All callers of open_cached_file() use 2 characters prefix. Allocating memory for
                  such short string is an overkill. Store it on IO_CACHE structure instead.
              
                  All callers of open_cached_file() use mysql_tmpdir as dir. No need to allocate
                  memory for it since it is constant and available till server shutdown.
              
                  This reduces number of allocations from 31 to 27 per OLTP RO transaction.
              
              commit 9e9f1da0d2dd5d9de9094d20d0b39e71ec4ce479
              Author: Sergey Vojtovich <svoj@mariadb.org>
              Date:   Tue Dec 2 14:59:01 2014 +0400
              
                  MDEV-7004 - Merge scalability fixes from 10.0-power
              
                  Remove call to deprecated set_thread_state. It is noop anyway, but generates
                  function call independently of performance schema state. According to perf
                  this saves ~0.2% of execution time.
              
              commit b4ec230917a50cbc1e406f5175189c8359bcb9a0
              Author: Sergey Vojtovich <svoj@mariadb.org>
              Date:   Tue Dec 2 14:54:30 2014 +0400
              
                  MDEV-7004 - Merge scalability fixes from 10.0-power
              
                  Preallocate locks on THD mem_root to avoid expensive malloc.
              
              commit 9bc5cec0f115d7d0c277a49b91b96109560280f4
              Author: Sergey Vojtovich <svoj@mariadb.org>
              Date:   Tue Dec 2 14:50:18 2014 +0400
              
                  MDEV-7004 - Merge scalability fixes from 10.0-power
              
                  Preallocate locks on THD mem_root to avoid expensive malloc.
              
              Show
              svoj Sergey Vojtovich added a comment - Pushed most of the fixes to 10.1.2: commit 78564373fee5e6cccf144b11bc60b0876b4bbd0b Author: Monty <monty@mariadb.org> Date: Fri Aug 29 14:07:43 2014 +0300 my_alloc.c - Changed 0x%lx -> %p array.c: - Static (preallocated) buffer can now be anywhere my_sys.h - Define MY_INIT_BUFFER_USED sql_delete.cc & sql_lex.cc - Use memroot when allocating classes (avoids call to current_thd) sql_explain.h: - Use preallocated buffers sql_explain.cc: - Use preallocated buffers and memroot sql_select.cc: - Use multi_alloc_root() instead of many alloc_root() - Update calls to Explain commit 3392278c869a9594f13a0b956d34efb600ab0ed0 Author: Sergey Vojtovich <svoj@mariadb.org> Date: Thu Dec 4 17:44:46 2014 +0400 MDEV-7004 - Merge scalability fixes from 10.0-power Preallocate dynamic array and bitmap on mem_root to avoid expensive malloc. This reduces number of allocations from 39 to 31 per OLTP RO transaction. commit eaa8c154e81cb3a9a2623f53482fe2bbd9ffa459 Author: Sergey Vojtovich <svoj@mariadb.org> Date: Thu Dec 4 17:42:32 2014 +0400 MDEV-7004 - Merge scalability fixes from 10.0-power Preallocate dynamic array on THD mem_root to avoid expensive malloc. commit 070a6e7a890e5a73eb38920657c060605e520b55 Author: Monty <monty@mariadb.org> Date: Fri Aug 29 14:09:51 2014 +0300 - Changed default values of query_prealloc_size and query_alloc_block_size so that a simple query with one join would not have to call my_malloc. - Allow lower limites for query_prealloc_size for testing. - Fixed wrong initialization of trans_alloc_block_size commit 9127784d5ce845c9d66a34eee371cbaf82468d89 Author: Sergey Vojtovich <svoj@mariadb.org> Date: Thu Dec 4 17:35:55 2014 +0400 Cherry pick dynamic array changes from commit: commit 85fd3d901311688e18ffce92ffc78129e5625791 Author: Monty <monty@mariadb.org> Date: Fri Aug 29 14:07:43 2014 +0300 my_alloc.c - Changed 0x%lx -> %p array.c: - Static (preallocated) buffer can now be anywhere my_sys.h - Define MY_INIT_BUFFER_USED sql_delete.cc & sql_lex.cc - Use memroot when allocating classes (avoids call to current_thd) sql_explain.h: - Use preallocated buffers sql_explain.cc: - Use preallocated buffers and memroot sql_select.cc: - Use multi_alloc_root() instead of many alloc_root() - Update calls to Explain commit 974808772b63563d031f687fd0fd158f83eb94aa Author: Sergey Vojtovich <svoj@mariadb.org> Date: Tue Dec 2 15:03:35 2014 +0400 MDEV-7004 - Merge scalability fixes from 10.0-power All callers of open_cached_file() use 2 characters prefix. Allocating memory for such short string is an overkill. Store it on IO_CACHE structure instead. All callers of open_cached_file() use mysql_tmpdir as dir. No need to allocate memory for it since it is constant and available till server shutdown. This reduces number of allocations from 31 to 27 per OLTP RO transaction. commit 9e9f1da0d2dd5d9de9094d20d0b39e71ec4ce479 Author: Sergey Vojtovich <svoj@mariadb.org> Date: Tue Dec 2 14:59:01 2014 +0400 MDEV-7004 - Merge scalability fixes from 10.0-power Remove call to deprecated set_thread_state. It is noop anyway, but generates function call independently of performance schema state. According to perf this saves ~0.2% of execution time. commit b4ec230917a50cbc1e406f5175189c8359bcb9a0 Author: Sergey Vojtovich <svoj@mariadb.org> Date: Tue Dec 2 14:54:30 2014 +0400 MDEV-7004 - Merge scalability fixes from 10.0-power Preallocate locks on THD mem_root to avoid expensive malloc. commit 9bc5cec0f115d7d0c277a49b91b96109560280f4 Author: Sergey Vojtovich <svoj@mariadb.org> Date: Tue Dec 2 14:50:18 2014 +0400 MDEV-7004 - Merge scalability fixes from 10.0-power Preallocate locks on THD mem_root to avoid expensive malloc.

                People

                • Assignee:
                  svoj Sergey Vojtovich
                  Reporter:
                  svoj Sergey Vojtovich
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: