Details
Critical
Description
histogram_size=255 optimizer_use_condition_selectivity=5 use_stat_tables='preferably' optimizer_selectivity_sampling_limit=97 query_cache_type=1 optimizer_switch='index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=on,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=on,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=off,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on' mrr_buffer_size=32M join_buffer_space_limit=32M join_cache_level=5 tmp_table_size=448M max_heap_table_size=448M sort_buffer_size=32M join_buffer_size=32M
Using either tokudb or inndob, I have at query that can reliably trigger:
stack_bottom = 0x7f91b7f27e10 thread_stack 0x48000 mysys/stacktrace.c:247(my_print_stacktrace)[0x7f91b7997efb] sql/signal_handler.cc:153(handle_fatal_signal)[0x7f91b752ff05] /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x7f91b60c1340] sql/opt_range_mrr.cc:100(step_down_to)[0x7f91b731fb49] sql/opt_range_mrr.cc:196(sel_arg_range_seq_next(void*, st_key_multi_range*))[0x7f91b732303b] sql/opt_range.cc:3426(records_in_column_ranges)[0x7f91b732405d]
I have rebuilt the table several times, including rounds of optimize & analyze table, anayzle table persistent for all, etc.
CREATE TABLE `f` ( `i` int(10) unsigned NOT NULL AUTO_INCREMENT, `u` varchar(2048) NOT NULL, `n` varchar(2048) NOT NULL, `e` tinyint(1) unsigned NOT NULL DEFAULT '1', `d` tinyint(1) unsigned NOT NULL, `c` datetime DEFAULT NULL, `p` int(10) unsigned NOT NULL, `g` tinyint(1) unsigned NOT NULL DEFAULT '0', `ur` varchar(2048) NOT NULL, `r` bigint(20) unsigned DEFAULT NULL, `cl` tinyint(1) unsigned NOT NULL DEFAULT '0', PRIMARY KEY (`i`), UNIQUE KEY (`i`,`d`,`p`), KEY (`p`), KEY (`d`,`p`), KEY (`e`,`d`,`p`), KEY (`r`) ) ENGINE=tokudb DEFAULT CHARSET=utf8 /* `compression`=tokudb_quicklz */; MariaDB [test]> SELECT f.i, f.u, f.n, f.e, f.d, f.c, f.p, f.g, f.ur, f.r, f.cl FROM f WHERE (f.d = 0 AND f.p = '1' AND f.i != '-1' AND f.n = 'some text' ); ERROR 2013 (HY000): Lost connection to MySQL server during query
Gliffy Diagrams
Attachments
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
In create_key_parts_for_pseudo_indexes() we remember field->key_length() for every used field. For "`n` varchar(2048) NOT NULL" we get 6144 (2048 * mbmaxlen of UTF-8, which is 3).
Later we do step_down_to()/SEL_ARG::store_max(), which does memcpy(*max_key,max_value,length). IIUC max_key was allocated as: RANGE_OPT_PARAM::max_key[MAX_KEY_LENGTH+MAX_FIELD_WIDTH], in other words 3072 + 255 * 3 + 1= 3838.
So we actually do memcpy(max_key[3838], max_value, 6144). This statement effectively breaks stack trace, crash happens later as an outcome of this overwrite.
I was wondering if partition pruning is affected by this problem. It turns out, no. partition_info::check_partition_field_length() disallows creation of partitioning schemes which have partitioning columns occupy more than MAX_KEY_LENGTH when encoded with KeyTupleFormat.
Looking through opt_range.cc code, RANGE_OPT_PARAM::min_key and max-key seem to be the only places where the code implies that key images do not exceed MAX_KEY_LENGTH in size.
The steps are:
- get_mm_leaf() stores field value in table->record[0] (check value->save_in_field_no_warnings() calls)
- Then, SEL_ARG object is created. It has key part values, for which buffer of appropriate size (key_part->store_length) is allocated on MEM_ROOT.
- Then, we call sel_arg_range_seq_init/sel_arg_range_seq_next and this is where it attempts to store the key value in RANGE_OPT_PARAM::min_key/max_key.
Buffer overrun seems to only be possible in #3.
Following Sergey Vojtovich 's advice, debugged a few queries. new alloc_root calls do not cause calls to malloc()
The patch had a typo which caused MDEV-8826. This is now fixed and pushed in both 10.0 and 10.1
Dan Vande More
Thanks for the report and the test case, and also for providing your mysqld options right away, it helped to save time.
The crash is reproducible on 10.0 and 10.1 with InnoDB or TokuDB; not reproducible with MyISAM.
I am getting a broken (or incomplete) stack trace both on 10.0 and 10.1, on two different machines, although it looks somewhat different. As I understand, it was broken for Dan Vande More as well.
Examples:
Apart from fixing the crash, it is also very important to investigate what's going on with the stack trace. As discussed, assigning to Sergey Vojtovich for a preliminary analysis.