Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7274

Server crashes in free () from /usr/lib/libjemalloc.so.1 on client connect

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 10.1.2
    • Fix Version/s: 10.1
    • Labels:
      None
    • Environment:
      Ubuntu Precise 64bit libjemalloc 2.2.5-1, cmake . -DCMAKE_BUILD_TYPE=Debug

      Description

      The crash appeared on 10.1 tree since this revision (as soon as 'since' makes sense on git log --topo-order)

      commit 853077ad7e81be1ade20b4beab1b95d5766d87b1 bf3b4a23f75de50e0f1ab4a562e5801dabc7305b 2b5db1d5bcd7b46b654d59a07fc119ef6a6b8651
      Merge: bf3b4a2 2b5db1d
      Author: Sergei Golubchik <serg@mariadb.org>
      Date:   Tue Dec 2 22:25:16 2014 +0100
      
          Merge branch '10.0' into bb-10.1-merge
      

      The crash only happens if TokuDB is enabled.

      Out of two machines I tried (Precise and Wheezy), it happens on Precise but not on Wheezy; maybe because the versions of libjemalloc* are different there, or could be some other reason.

      # ATTENTION: Run with --mysqld=--plugin-load=ha_tokudb
      
      --connect (con1,localhost,root,,)
      
      SELECT CASE WHEN table_schema = 'information_schema' THEN 'INFORMATION_SCHEMA' ELSE table_schema END, table_name, CASE WHEN table_type = 'BASE TABLE' THEN 'table' WHEN table_type = 'VIEW' THEN 'view' WHEN table_type = 'SYSTEM VIEW' then 'view' ELSE 'misc' END, column_name, CASE WHEN column_key = 'PRI' THEN 'primary' WHEN column_key = 'MUL' THEN 'indexed' WHEN column_key = 'UNI' THEN 'indexed' ELSE 'ordinary' END FROM information_schema.tables INNER JOIN information_schema.columns USING(table_schema, table_name) WHERE table_name <> 'DUMMY';
      
      --disconnect con1
      
      --connect (con2,localhost,root,,)
      select 1;
      --disconnect con2
      
      Stack trace from 10.1 d1522af72dad1965b8a8a37415545014ba743f49
      #3  <signal handler called>
      #4  0x00007f0a2730ae1b in free () from /usr/lib/libjemalloc.so.1
      #5  0x00007f0a2667c3b9 in pthread_create@@GLIBC_2.2.5 () from /lib/x86_64-linux-gnu/libpthread.so.0
      #6  0x00007f0a28a081ff in spawn_thread_v1 (key=6, thread=0x7f0a1e4e4a10, attr=0x7f0a2943bc60 <connection_attrib>, start_routine=0x7f0a283355a3 <handle_one_connection(void*)>, arg=0x7f0a1e4e1070) at 10.1/storage/perfschema/pfs.cc:1910
      #7  0x00007f0a2814098c in inline_mysql_thread_create (key=6, thread=0x7f0a1e4e4a10, attr=0x7f0a2943bc60 <connection_attrib>, start_routine=0x7f0a283355a3 <handle_one_connection(void*)>, arg=0x7f0a1e4e1070) at 10.1/include/mysql/psi/mysql_thread.h:1255
      #8  0x00007f0a2814af1b in create_thread_to_handle_connection (thd=0x7f0a1e4e1070) at 10.1/sql/mysqld.cc:6175
      #9  0x00007f0a2814b3d8 in create_new_thread (thd=0x7f0a1e4e1070) at 10.1/sql/mysqld.cc:6268
      #10 0x00007f0a2814bdea in handle_connections_sockets () at 10.1/sql/mysqld.cc:6556
      #11 0x00007f0a2814a6ef in mysqld_main (argc=120, argv=0x7f0a24c545e8) at 10.1/sql/mysqld.cc:5777
      #12 0x00007f0a281400b8 in main (argc=7, argv=0x7fff6567cae8) at 10.1/sql/main.cc:25
      

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            serg Sergei Golubchik added a comment -

            Yes, I believe I've seen it. This is why our Precise builder has jemalloc 3.6.0. I've even written in KB

            jemalloc 2.2.5 seems to be buggy, causes crashes in the mysql-test (in the jemalloc code, red-black tree handling)

            In 10.0 we've linked with jemalloc statically, in 10.1 we link with it dynamically (MDEV-6895). I suppose that's why 10.0 binaries work on Precise (using jemalloc 3.6.0 they were linked with) and 10.1 binaries crash (using system jemalloc 2.2.5).

            We either need to link with jemalloc statically on Precise, or use a versioned jemalloc dependency in .deb files.

            Show
            serg Sergei Golubchik added a comment - Yes, I believe I've seen it. This is why our Precise builder has jemalloc 3.6.0. I've even written in KB jemalloc 2.2.5 seems to be buggy, causes crashes in the mysql-test (in the jemalloc code, red-black tree handling) In 10.0 we've linked with jemalloc statically, in 10.1 we link with it dynamically ( MDEV-6895 ). I suppose that's why 10.0 binaries work on Precise (using jemalloc 3.6.0 they were linked with) and 10.1 binaries crash (using system jemalloc 2.2.5). We either need to link with jemalloc statically on Precise, or use a versioned jemalloc dependency in .deb files.
            Hide
            serg Sergei Golubchik added a comment -

            It has just occurred to me, do you mean server crashes in buildbot? Or in your own Precise VM where you've built MariaDB with system jemalloc 2.2.5?

            If the latter, the fix could be — don't use jemalloc if it's older than 2.3

            Show
            serg Sergei Golubchik added a comment - It has just occurred to me, do you mean server crashes in buildbot ? Or in your own Precise VM where you've built MariaDB with system jemalloc 2.2.5? If the latter, the fix could be — don't use jemalloc if it's older than 2.3
            Hide
            elenst Elena Stepanova added a comment -

            Neither. I mean perro. I suppose we should just upgrade jemalloc there.

            Show
            elenst Elena Stepanova added a comment - Neither. I mean perro. I suppose we should just upgrade jemalloc there.

              People

              • Assignee:
                serg Sergei Golubchik
                Reporter:
                elenst Elena Stepanova
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: