Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-2754

LP:923772 - performance regression in mariadb-5.3 with many threads

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      sysbench OLTP benchmark with 64 threads yields 50% or fewer operations per second with MariaDB 5.3.3-rc compared to MariaDB 5.2.10.

      I.e. on a machine with 24 cores we get 125K ops/sec with 5.2.10 but only 64K ops/sec with 5.3.3. With more cores the result gets worse. I.e. 32 cores give 175K vs. 39K.

      sysbench parameters: --test=oltp --num-threads=64 --max-time=60 --max-requests=0 --oltp-table-size=10000000 --db-driver=mysql --mysql-table-engine=innodb

      my.cnf:

      skip-external-locking
      bind-address = 127.0.0.1
      key_buffer = 16M
      max_allowed_packet = 16M
      default-storage-engine = INNODB
      innodb_buffer_pool_size = 8G
      innodb_additional_mem_pool_size = 64M
      innodb_log_buffer_size = 128M
      innodb_thread_concurrency = 0
      innodb_flush_log_at_trx_commit = 0
      innodb_locks_unsafe_for_binlog = 1
      innodb_data_file_path = ibdata1:11G;ibdata2:11G:autoextend
      innodb_log_file_size = 1333M
      innodb_log_files_in_group = 3
      innodb_file_io_threads = 4
      innodb_max_dirty_pages_pct = 90
      innodb_lock_wait_timeout = 120
      back_log = 50
      max_connections = 100
      table_open_cache = 2048
      max_heap_table_size = 64M
      sort_buffer_size = 8M
      join_buffer_size = 8M
      thread_cache_size = 8
      query_cache_size = 64M
      query_cache_limit = 2M
      thread_stack = 192K
      transaction_isolation = REPEATABLE-READ
      tmp_table_size = 64M
      long_query_time = 2

      PMP shows significant mutex contention in the Query_cache::try_lock() method. If the query cache is disabled, 5.3.3 performance gets dramatically better. Still 5.2.10 is about 20% faster. There seems to be extra contention inside InnoDB, caused by make_join_statistics(). The same call stack can be seen in 5.2.10, but less often.

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            axel Axel Schwenke added a comment -

            stack dumps from PMP for 5.2.10 and 5.3.3 with and w/o QC
            LPexportBug923772_stacks.tar.gz

            Show
            axel Axel Schwenke added a comment - stack dumps from PMP for 5.2.10 and 5.3.3 with and w/o QC LPexportBug923772_stacks.tar.gz
            Hide
            axel Axel Schwenke added a comment -

            Re: performance regression in mariadb-5.3 with many threads

            Show
            axel Axel Schwenke added a comment - Re: performance regression in mariadb-5.3 with many threads
            Hide
            ratzpo Rasmus Johansson added a comment -

            Launchpad bug id: 923772

            Show
            ratzpo Rasmus Johansson added a comment - Launchpad bug id: 923772

              People

              • Assignee:
                sanja Oleksandr Byelkin
                Reporter:
                axel Axel Schwenke
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: