Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Incomplete
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: None
-
Labels:
Description
We upgrade from 5.2.5-MariaDB-mariadb99-log to 5.5.25-MariaDB-log on a Slave. The master is 5.2.5-MariaDB-mariadb99-log.
After execute "start slave;", the slave crash.
The binlog execute until reaches a set of a user-defined variable:
# at 10826 #120816 11:02:20 server id 2101 end_log_pos 845941273 Query thread_id=468834444 exec_time=0 error_code=0 SET TIMESTAMP=1345129340/*!*/; BEGIN /*!*/; # at 10905 #120816 11:02:20 server id 2101 end_log_pos 845941320 User_var SET @`lastId`:=7521389/*!*/;
After recover, it continuous crash when must execute a set of a user-defined variable:
120816 12:56:39 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 5.5.25-MariaDB-log key_buffer_size=134217728 read_buffer_size=2097152 max_used_connections=1 max_threads=1002 thread_count=1 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4249215 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x2aaafc016470 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41cba0b8 thread_stack 0x48000 ??:0(my_print_stacktrace)[0xa8783e] ??:0(handle_fatal_signal)[0x6d048c] ??:0(??)[0x3e8c40eb70] ??:0(THD::cleanup_after_query())[0x54a441] ??:0(Query_log_event::do_apply_event(Relay_log_info const*, char const*, unsigned int))[0x7a7bb9] ??:0(apply_event_and_update_pos(Log_event*, THD*, Relay_log_info*))[0x50ca52] ??:0(exec_relay_log_event(THD*, Relay_log_info*))[0x514710] ??:0(handle_slave_sql)[0x515a24] ??:0(??)[0x3e8c40673d] ??:0(??)[0x3e8bcd44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x2aaafc01c2f3): is an invalid pointer Connection ID (thread ID): 4 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash.
Gliffy Diagrams
Attachments
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
Re: MariaDB Crash executing user-defined variables on binlog
I'm continuing the debugging:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x40820940 (LWP 17747)]
0x0000000000ab6074 in strnmov ()
(gdb) bt
#0 0x0000000000ab6074 in strnmov ()
#1 0x0000000000532035 in create_table_def_key(THD*, char*, TABLE_LIST const*, bool) ()
#2 0x000000000053ae88 in open_table(THD*, TABLE_LIST*, st_mem_root*, Open_table_context*) ()
#3 0x000000000053d89e in open_tables(THD*, TABLE_LIST*, unsigned int, unsigned int, Prelocking_strategy*) ()
#4 0x000000000053e5ce in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) ()
#5 0x000000000056a19a in mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) ()
#6 0x00000000005827dd in mysql_execute_command(THD*) ()
#7 0x0000000000583a41 in mysql_parse(THD*, char*, unsigned int, Parser_state*) ()
#8 0x00000000007a7bb9 in Query_log_event::do_apply_event(Relay_log_info const*, char const*, unsigned int) ()
#9 0x000000000050ca52 in apply_event_and_update_pos(Log_event*, THD*, Relay_log_info*) ()
#10 0x0000000000514710 in exec_relay_log_event(THD*, Relay_log_info*) ()
#11 0x0000000000515a24 in handle_slave_sql ()
#12 0x00000031b800677d in start_thread () from /lib64/libpthread.so.0
#13 0x00000031b78d33ed in clone () from /lib64/libc.so.6
Re: MariaDB Crash executing user-defined variables on binlog
Hi,
Sorry, I don't quite understand what you meant by
> We re kick the db with 5.2.5 and decide to upgrade again to 5.5.25 and fails again setting a user variables.
> We change the config files to the minimal configuration and it didn't work. It keeps crashing.
But you already had 5.5.25 on the slave, which was crashing. So what changed? You upgraded the master and it started crashing?
The variables you attached – are they from the master or from the slave?
Can you attach another set (if these were from the master, then attach from the slave, and vice versa)?
You are saying that you can easily reproduce the crash using the test you provided, but the error log seems to be from your non-simpified production scenario. Could you please:
- start fresh servers on new clean datadirs;
- enable general log and error log;
- reproduce the problem;
- pack the datadirs and the logs and upload them along with cnf files and server startup command lines to ftp://ftp.askmonty.org/ (the datadirs should contain binlogs and the data itself; since you're not using InnoDB tables in the scenario, you can drop InnoDB files so the directories should be tiny).
Thank you.
Re: MariaDB Crash executing user-defined variables on binlog
Answers:
1- But you already had 5.5.25 on the slave, which was crashing. So what changed? You upgraded the master and it started crashing?
We delete the slave with 5.5.25. We started again and it didn't work.
2- The variables you attached – are they from the master or from the slave?
They are from the slave.
3- Can you attach another set (if these were from the master, then attach from the slave, and vice versa)?
No, it is not necessary because we suspend the project.
We think and we are pretty sure that MariaDB 5.5.25 and our version of CentOS 5 can not work together. We are planning to upgrade our OS. Do you need the execution of "rpm -qa" to keep analyzing?
Re: MariaDB Crash executing user-defined variables on binlog
Hi David,
No, thanks, rpm -qa won't be necessary at this point.
Please let us know whether the upgrade made the problem disappear.
Meanwhile, we will watch similar issues, if we get some more cases, we might be able to chase down the root of the problem. I also believe there was a somewhat similar bug report in the upstream, if it is so indeed, we might be able to see it in the bug system after the fix is released.
Launchpad bug id: 1037711
Re: MariaDB Crash executing user-defined variables on binlog
I confirm this on versions:
5.2.5-MariaDB-mariadb99-log
5.3.2-MariaDB-beta-mariadb102-log
Steps to reproduce:
1- Install on DB-MASTER 5.2.5-MariaDB-mariadb99-log or 5.3.2-MariaDB-beta-mariadb102-log
2- Install on DB-SLAVE 5.5.25-MariaDB-log
3- Replicate DB-SLAVE from DB-MASTER
4- create a table
5- Insert data like this:
insert into DB.a values (1);
insert into DB.a values (2);
insert into DB.a values (3);
set @ant=4;
insert into DB.a values (@ant);
6- Replication fails and mysqld crashes