Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6799

Crash in field_conv, memcpy_field_possible

    Details

    • Type: Bug
    • Status: Stalled
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 10.0.13
    • Fix Version/s: 10.0
    • Labels:
    • Sprint:
      10.0.21

      Description

      Crash with the following stack trace happened at a customer:

      > mysqld.exe!Field_string::type() Line 2239 + 0x15 bytes	C++
      mysqld.exe!memcpy_field_possible(Field * to, Field * from) Line 840 + 0x10 bytes	C++
      mysqld.exe!field_conv(Field * to, Field * from) Line 865 + 0x5 bytes	C++
      mysqld.exe!save_field_in_field(Field * from, bool * null_value, Field * to, bool no_conversions) Line 5978	C++
      mysqld.exe!sp_eval_expr(THD * thd, Field * result_field, Item * * expr_item_ptr) Line 434	C++
      mysqld.exe!sp_rcontext::set_variable(THD * thd, Field * field, Item * * value) Line 377 + 0xe bytes	C++
      mysqld.exe!sp_cursor::Select_fetch_into_spvars::send_data(List<Item> & items) Line 553 + 0x25 bytes	C++
      mysqld.exe!Materialized_cursor::fetch(unsigned long num_rows) Line 339 + 0x10 bytes	C++
      mysqld.exe!sp_cursor::fetch(THD * thd, List<sp_variable> * vars) Line 510	C++
      mysqld.exe!sp_instr_cfetch::execute(THD * thd, unsigned int * nextp) Line 3864 + 0xe bytes	C++
      mysqld.exe!sp_head::execute(THD * thd, bool merge_da_on_success) Line 1366 + 0x11 bytes	C++
      mysqld.exe!sp_head::execute_procedure(THD * thd, List<Item> * args) Line 2152 + 0xe bytes	C++
      mysqld.exe!mysql_execute_command(THD * thd) Line 4688 + 0x13 bytes	C++
      mysqld.exe!mysql_parse(THD * thd, char * rawbuf, unsigned int length, Parser_state * parser_state) Line 6416	C++
      mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length) Line 1310	C++
      mysqld.exe!do_command(THD * thd) Line 1011	C++
      mysqld.exe!threadpool_process_request(THD * thd) Line 233 + 0x8 bytes	C++
      mysqld.exe!io_completion_callback(_TP_CALLBACK_INSTANCE * instance, void * context, void * overlapped, unsigned long io_result, unsigned __int64 nbytes, _TP_IO * io) Line 568 + 0x17 bytes	C++
      kernel32.dll!BasepTpIoCallback() + 0x52 bytes
      ntdll.dll!TppIopExecuteCallback() + 0x1ba bytes
      ntdll.dll!TppWorkerThread() + 0x33f bytes
      kernel32.dll!BaseThreadInitThunk() + 0xd bytes
      ntdll.dll!RtlUserThreadStart() + 0x21 bytes
      

      We were unable to get a small repeatable testcase, but under customer' load the crash is reproducible.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

              Hide
              psergey Sergei Petrunia added a comment - - edited

              Some details from my attempt to investigate

              So, in memcpy_field_possible() we crash at this line:

              const enum_field_types to_type= from->type();

              Let's assume for the sake of investigation that from=NULL (or an invalid pointer)
              the caller function is:

              int field_conv(Field *to,Field *from)

              keep the assumption, from=NULL. The caller is:

              static int save_field_in_field(Field *from, bool *null_value,
              Field *to, bool no_conversions)

              which has:

              res= field_conv(to, from);

              however, above that, the function has:

              if (from->is_null())
              {
              ...

              if we had from==NULL here, we would have crashed earlier.

              Show
              psergey Sergei Petrunia added a comment - - edited Some details from my attempt to investigate So, in memcpy_field_possible() we crash at this line: const enum_field_types to_type= from->type(); Let's assume for the sake of investigation that from=NULL (or an invalid pointer) the caller function is: int field_conv(Field *to,Field *from) keep the assumption, from=NULL. The caller is: static int save_field_in_field(Field *from, bool *null_value, Field *to, bool no_conversions) which has: res= field_conv(to, from); however, above that, the function has: if (from->is_null()) { ... if we had from==NULL here, we would have crashed earlier.
              Hide
              psergey Sergei Petrunia added a comment -
               class Field_string :public Field_longstr {
               ...
                 enum_field_types type() const
                 {
                   return ((can_alter_field_type && orig_table &&
                            orig_table->s->db_create_options & HA_OPTION_PACK_RECORD &&
                     field_length >= 4) &&
                           orig_table->s->frm_version < FRM_VER_TRUE_VARCHAR ?
                    MYSQL_TYPE_VAR_STRING : MYSQL_TYPE_STRING);
               ^ we crash here
              

              it seems, the function is called for valid Field objects: at the start of memcpy_field_possible(), there are these calls:

                 const enum_field_types to_real_type= to->real_type();
                 const enum_field_types from_real_type= from->real_type();
              

              and we didn't crash there. real_type() have trivial

              {return $type; }

              implementations, so the fact that they succeeded only means that field object is valid.

              The call to from->type() crashes. I assume that orig_table has invalid value.

              It was probably invalid in 5.5 also. But, before the fix for MDEV-4309 the call to from->type() was at the end of if-condition, so execution simply didn't reach it.

              MDEV-4309 moved the call to start of the function, and now it crashes.

              Show
              psergey Sergei Petrunia added a comment - class Field_string :public Field_longstr { ... enum_field_types type() const { return ((can_alter_field_type && orig_table && orig_table->s->db_create_options & HA_OPTION_PACK_RECORD && field_length >= 4) && orig_table->s->frm_version < FRM_VER_TRUE_VARCHAR ? MYSQL_TYPE_VAR_STRING : MYSQL_TYPE_STRING); ^ we crash here it seems, the function is called for valid Field objects: at the start of memcpy_field_possible(), there are these calls: const enum_field_types to_real_type= to->real_type(); const enum_field_types from_real_type= from->real_type(); and we didn't crash there. real_type() have trivial {return $type; } implementations, so the fact that they succeeded only means that field object is valid. The call to from->type() crashes. I assume that orig_table has invalid value. It was probably invalid in 5.5 also. But, before the fix for MDEV-4309 the call to from->type() was at the end of if-condition, so execution simply didn't reach it. MDEV-4309 moved the call to start of the function, and now it crashes.
              Show
              psergey Sergei Petrunia added a comment - The patch is at https://code.launchpad.net/~maria-captains/maria/10.0-csc8034-r1
              Hide
              psergey Sergei Petrunia added a comment -

              We've got a script that allows to repeat the problem: see CSC#8034, stress_minimal.sql.

              I was using 10.0.13, compiled with "cmake . ; make " and I got:

              ./mysqld(my_print_stacktrace+0x24)[0xac2584]
              ./mysqld(handle_fatal_signal+0x3d8)[0x6ecf68]
              /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7f17fa45acb0]
              ./mysqld(_ZNK12Field_string4typeEv+0x1a)[0x6e605a]
              ./mysqld(_Z21memcpy_field_possibleP5FieldS0_+0x4c)[0x6e8f2c]
              ./mysqld(_Z10field_convP5FieldS0_+0x19)[0x6e9449]
              ./mysqld[0x6fe6c9]
              ./mysqld(_Z12sp_eval_exprP3THDP5FieldPP4Item+0x7e)[0x7d9b1e]
              ./mysqld(_ZN11sp_rcontext12set_variableEP3THDP5FieldPP4Item+0x17)[0x7e0cd7]
              ./mysqld(_ZN9sp_cursor24Select_fetch_into_spvars9send_dataER4ListI4ItemE+0x56)[0x7e0d56]
              ./mysqld(_ZN19Materialized_cursor5fetchEm+0x4c)[0x7ec1dc]
              ./mysqld(_ZN9sp_cursor5fetchEP3THDP4ListI11sp_variableE+0x4f)[0x7e110f]
              ./mysqld(_ZN15sp_instr_cfetch7executeEP3THDPj+0x2f)[0x7d7a3f]
              ./mysqld(_ZN7sp_head7executeEP3THDb+0x754)[0x7dac04]
              ./mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x581)[0x7dbf61]
              ./mysqld(_Z21mysql_execute_commandP3THD+0x3178)[0x5b5038]
              ./mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x169)[0x5b9bc9]
              ./mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x16f9)[0x5bb3a9]
              ./mysqld(_Z24do_handle_one_connectionP3THD+0x1e4)[0x668ae4]
              ./mysqld(handle_one_connection+0x40)[0x668b70]
              /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f17fa452e9a]
              /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f17f93463fd]
              
              Show
              psergey Sergei Petrunia added a comment - We've got a script that allows to repeat the problem: see CSC#8034, stress_minimal.sql. I was using 10.0.13, compiled with "cmake . ; make " and I got: ./mysqld(my_print_stacktrace+0x24)[0xac2584] ./mysqld(handle_fatal_signal+0x3d8)[0x6ecf68] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7f17fa45acb0] ./mysqld(_ZNK12Field_string4typeEv+0x1a)[0x6e605a] ./mysqld(_Z21memcpy_field_possibleP5FieldS0_+0x4c)[0x6e8f2c] ./mysqld(_Z10field_convP5FieldS0_+0x19)[0x6e9449] ./mysqld[0x6fe6c9] ./mysqld(_Z12sp_eval_exprP3THDP5FieldPP4Item+0x7e)[0x7d9b1e] ./mysqld(_ZN11sp_rcontext12set_variableEP3THDP5FieldPP4Item+0x17)[0x7e0cd7] ./mysqld(_ZN9sp_cursor24Select_fetch_into_spvars9send_dataER4ListI4ItemE+0x56)[0x7e0d56] ./mysqld(_ZN19Materialized_cursor5fetchEm+0x4c)[0x7ec1dc] ./mysqld(_ZN9sp_cursor5fetchEP3THDP4ListI11sp_variableE+0x4f)[0x7e110f] ./mysqld(_ZN15sp_instr_cfetch7executeEP3THDPj+0x2f)[0x7d7a3f] ./mysqld(_ZN7sp_head7executeEP3THDb+0x754)[0x7dac04] ./mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x581)[0x7dbf61] ./mysqld(_Z21mysql_execute_commandP3THD+0x3178)[0x5b5038] ./mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x169)[0x5b9bc9] ./mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x16f9)[0x5bb3a9] ./mysqld(_Z24do_handle_one_connectionP3THD+0x1e4)[0x668ae4] ./mysqld(handle_one_connection+0x40)[0x668b70] /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f17fa452e9a] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f17f93463fd]
              Hide
              Lawrin Novitsky added a comment -

              Hi,
              I've seen identical crash call stack that people has got win 10.0.20

              mysqld.exe!Field_string::type() Line 2279 + 0x15 bytes C++
              > mysqld.exe!memcpy_field_possible(Field * to, Field * from) Line 865 + 0xe9 bytes C++
              mysqld.exe!field_conv(Field * to, Field * from) Line 873 + 0x5 bytes C++
              mysqld.exe!save_field_in_field(Field * from, bool * null_value, Field * to, bool no_conversions) Line 5919 C++
              mysqld.exe!sp_eval_expr(THD * thd, Field * result_field, Item * * expr_item_ptr) Line 434 C++
              mysqld.exe!sp_rcontext::set_variable(THD * thd, Field * field, Item * * value) Line 378 + 0xe bytes C++
              mysqld.exe!sp_cursor::Select_fetch_into_spvars::send_data(List<Item> & items) Line 554 + 0x25 bytes C++
              mysqld.exe!Materialized_cursor::fetch(unsigned long num_rows) Line 344 + 0x10 bytes C++
              mysqld.exe!sp_cursor::fetch(THD * thd, List<sp_variable> * vars) Line 511 C++
              mysqld.exe!sp_instr_cfetch::execute(THD * thd, unsigned int * nextp) Line 3874 + 0xe bytes C++
              mysqld.exe!sp_head::execute(THD * thd, bool merge_da_on_success) Line 1371 + 0x11 bytes C++
              mysqld.exe!sp_head::execute_procedure(THD * thd, List<Item> * args) Line 2159 + 0xe bytes C++
              mysqld.exe!mysql_execute_command(THD * thd) Line 4701 + 0x13 bytes C++
              mysqld.exe!mysql_parse(THD * thd, char * rawbuf, unsigned int length, Parser_state * parser_state) Line 6534 C++
              mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length) Line 1311 C++
              mysqld.exe!do_command(THD * thd) Line 1007 C++
              mysqld.exe!threadpool_process_request(THD * thd) Line 233 + 0x8 bytes C++
              mysqld.exe!io_completion_callback(_TP_CALLBACK_INSTANCE * instance, void * context, void * overlapped, unsigned long io_result, unsigned __int64 nbytes, _TP_IO * io) Line 568 + 0x17 bytes C++
              kernel32.dll!00000000773b32e2()
              [Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]
              00000007f9a3fa80()
              0000000806034c30()
              ntdll.dll!00000000774afbf0()
              00000007768b8c80()

              Might be that bug needs to be re-open

              Show
              Lawrin Novitsky added a comment - Hi, I've seen identical crash call stack that people has got win 10.0.20 mysqld.exe!Field_string::type() Line 2279 + 0x15 bytes C++ > mysqld.exe!memcpy_field_possible(Field * to, Field * from) Line 865 + 0xe9 bytes C++ mysqld.exe!field_conv(Field * to, Field * from) Line 873 + 0x5 bytes C++ mysqld.exe!save_field_in_field(Field * from, bool * null_value, Field * to, bool no_conversions) Line 5919 C++ mysqld.exe!sp_eval_expr(THD * thd, Field * result_field, Item * * expr_item_ptr) Line 434 C++ mysqld.exe!sp_rcontext::set_variable(THD * thd, Field * field, Item * * value) Line 378 + 0xe bytes C++ mysqld.exe!sp_cursor::Select_fetch_into_spvars::send_data(List<Item> & items) Line 554 + 0x25 bytes C++ mysqld.exe!Materialized_cursor::fetch(unsigned long num_rows) Line 344 + 0x10 bytes C++ mysqld.exe!sp_cursor::fetch(THD * thd, List<sp_variable> * vars) Line 511 C++ mysqld.exe!sp_instr_cfetch::execute(THD * thd, unsigned int * nextp) Line 3874 + 0xe bytes C++ mysqld.exe!sp_head::execute(THD * thd, bool merge_da_on_success) Line 1371 + 0x11 bytes C++ mysqld.exe!sp_head::execute_procedure(THD * thd, List<Item> * args) Line 2159 + 0xe bytes C++ mysqld.exe!mysql_execute_command(THD * thd) Line 4701 + 0x13 bytes C++ mysqld.exe!mysql_parse(THD * thd, char * rawbuf, unsigned int length, Parser_state * parser_state) Line 6534 C++ mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length) Line 1311 C++ mysqld.exe!do_command(THD * thd) Line 1007 C++ mysqld.exe!threadpool_process_request(THD * thd) Line 233 + 0x8 bytes C++ mysqld.exe!io_completion_callback(_TP_CALLBACK_INSTANCE * instance, void * context, void * overlapped, unsigned long io_result, unsigned __int64 nbytes, _TP_IO * io) Line 568 + 0x17 bytes C++ kernel32.dll!00000000773b32e2() [Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll] 00000007f9a3fa80() 0000000806034c30() ntdll.dll!00000000774afbf0() 00000007768b8c80() Might be that bug needs to be re-open
              Hide
              psergey Sergei Petrunia added a comment -

              Reopening due to reports that the crash is still observable.

              Show
              psergey Sergei Petrunia added a comment - Reopening due to reports that the crash is still observable.
              Hide
              psergey Sergei Petrunia added a comment -

              Lawrin Novitsky , can you post the stack trace here? I have succeded in loading the minidump into WinDbg but couldn't get any further..

              The reason for asking is that I've now got a patch for MDEV-7040.

              Show
              psergey Sergei Petrunia added a comment - Lawrin Novitsky , can you post the stack trace here? I have succeded in loading the minidump into WinDbg but couldn't get any further.. The reason for asking is that I've now got a patch for MDEV-7040 .
              Hide
              psergey Sergei Petrunia added a comment -

              Ok it was a misunderstanding, the stack trace is as posted above in the comment dated 2015-07-06 17:23. This is supposed to be fixed by MDEV-7040.

              Show
              psergey Sergei Petrunia added a comment - Ok it was a misunderstanding, the stack trace is as posted above in the comment dated 2015-07-06 17:23. This is supposed to be fixed by MDEV-7040 .

                People

                • Assignee:
                  psergey Sergei Petrunia
                  Reporter:
                  psergey Sergei Petrunia
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:

                    Agile