About filesort_debug.test failure.
We get error 1041 instead of 1038. In symbols, that is:

#define ER_OUT_OF_SORTMEMORY 1038
#define ER_OUT_OF_RESOURCES 1041

The difference comes from the following. mysql-5.6 has these lines in my_malloc():

DBUG_EXECUTE_IF("simulate_out_of_memory",
DBUG_SET("-d,simulate_out_of_memory");

added by alexey.kopytov@sun.com-20100521112348-rz0zm53wbo6g2aqr,
Bug #42064: low memory crash when importing hex strings, in
Item_hex_string::Item_hex_string

10.0-serg doesn't have them. This causes "simulate_out_of_memory" condition to remain switched on, which causes malloc failure when we're trying to allocate a memory for the error message. Which, in turn, produces a ER_OUT_OF_RESOURCES error here:

#0 sql_alloc_error_handler () at /home/psergey/dev2/10.0-serg/sql/thr_malloc.cc:49
#1 0x0000000000e6d30e in alloc_root (mem_root=0x2ed2648, length=392) at /home/psergey/dev2/10.0-serg/mysys/my_alloc.c:203
#2 0x00000000005887b1 in Sql_alloc::operator new (size=392, mem_root=0x2ed2648) at /home/psergey/dev2/10.0-serg/sql/sql_list.h:43
#3 0x000000000060ba70 in Warning_info::push_warning (this=0x2ed2648, thd=0x2ecd610, sql_errno=1038, sqlstate=0xf083db "HY001", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fffe4160bc0 "Out of sort memory, consider increasing server sort buffer size") at /home/psergey/dev2/10.0-serg/sql/sql_error.cc:692
#4 0x000000000060032f in Diagnostics_area::push_warning (this=0x2ed2420, thd=0x2ecd610, sql_errno=1038, sqlstate=0xf083db "HY001", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fffe4160bc0 "Out of sort memory, consider increasing server sort buffer size") at /home/psergey/dev2/10.0-serg/sql/sql_error.h:807
#5 0x00000000005f3ea2 in THD::raise_condition (this=0x2ecd610, sql_errno=1038, sqlstate=0xf083db "HY001", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fffe4160bc0 "Out of sort memory, consider increasing server sort buffer size") at /home/psergey/dev2/10.0-serg/sql/sql_class.cc:1213
#6 0x000000000057f29d in my_message_sql (error=1038, str=0x7fffe4160bc0 "Out of sort memory, consider increasing server sort buffer size", MyFlags=4166) at /home/psergey/dev2/10.0-serg/sql/mysqld.cc:3371
#7 0x0000000000e70bf2 in my_error (nr=1038, MyFlags=4166) at /home/psergey/dev2/10.0-serg/mysys/my_error.c:125
#8 0x00000000007fa578 in filesort (thd=0x2ecd610, table=0x7fffa804cb30, sortorder=0x7fffa8016760, s_length=2, select=0x7fffa8016568, max_rows=18446744073709551615, sort_positions=false, examined_rows=0x7fffe41614a0, found_rows=0x7fffe4161498) at /home/psergey/dev2/10.0-serg/sql/filesort.cc:278
.

I've tried to figure out which changeset exchanged the order of push_warning() and set_error_status() calls in THD::raise_condition(). It was this merge changeset (and the change was introduced by the merge cset itself, not by one of the merged-in csets) :

revno: 3774 [merge]
revision-id: sergii@pisem.net-20130721143919-7cltcw2l9g29f983
parent: sergii@pisem.net-20130718144657-oi7ql96c29knkqqi
parent: sergii@pisem.net-20130717165112-i9klgxk4enpvc09a
committer: Sergei Golubchik <sergii@pisem.net>
branch nick: 10.0
timestamp: Sun 2013-07-21 16:39:19 +0200
message:
10.0-monty merge
...

Hi Sergei,

yes, that's what I was afraid of. I guess it is just merge mistake.

Regards,
Sergey

TDC_RT_REMOVE_NOT_OWN_KEEP_SHARE is not same as TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE. Comment in sql_table.cc says:

Storage engine has requested exclusive lock only for prepare phase
and we are not under LOCK TABLES.
Don't mark TABLE_SHARE as old in this case, as this won't allow opening
of table by other threads during main phase of in-place ALTER TABLE.

At this moment we hold exclusive metadata lock, all we should do is purge unused TABLE objects.

HA_READ_OUT_OF_SYNC was introduced with the undermentioned revision. Yet the
sole purpose of this flag is to disable RBR updates batching.

Originally it was named HA_STORES_NO_ROWS, which I find applicable to blackhole only.
I suggested two alternatives: HA_EMPTY or HA_READ_OUT_OF_SYNC.
Meaning of HA_EMPTY is the same as HA_STORES_NO_ROWS.

But HA_READ_OUT_OF_SYNC is a bit more generic: the idea is that storage engine doesn't
guarantee that table updates will be [immediately] visible.

E.g. abstract storage engine may consume N rows [on slave], but return single aggregated result row.
HA_READ_OUT_OF_SYNC is applicable to this case as well, whereas HA_STORES_NO_ROWS is
controversial.

revno: 3763 [merge]
committer: Rohit Kalhans <rohit.kalhans@oracle.com>
branch nick: mysql-trunk
timestamp: Wed 2012-05-02 17:34:42 +0530
message:
WL#5597: Using batch operations when there is no index in RBR

CONTEXT
-------

With RBR, if a table has no indexes, the slave does a full table scan for each
row changed (i.e. updated or deleted). This can be extremely time consuming
when there is a high number of rows to be updated.

SOLUTION
---------

When there is a table without a PK or an INDEX, create a temporary in-memory
index (e.g. in-memory hash table) and store the rows to be update in it. Then for
each row in the table, check if the row exists in the hash table. If there is a
match, do the operation, i.e. update or delete.

In 10.0 TDC_RT_REMOVE_NOT_OWN is almost unused in favor of TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE.

But technically I failed to understand it's purpose: wait_while_table_is_used() upgrades shared meta-data lock to exclusive and no other thread is permitted to access this table. Why do we need extra protection mechanism in table definition cache?

If TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE serves a purpose indeed, we need to restore it.

CREATE TABLE t1(a INT, b VARCHAR(255));
ALTER TABLE t1 DROP COLUMN b;

Is the above alter good for in-place algorithm?
If it is, what row format shall t1 have (static or dynamic)?

If it is not allowed or (allowed and format is static): remove juggling with HA_OPTION_PACK_RECORD in mysql_create_frm_image().
else: create .frm after we know for sure which algorithm we will be using, disable juggling with copy algorithm.

ps_ddl.test doesn't contain a test case for revision alexander.nozdrin@oracle.com-20111219114211-49pqi0wfs9p4o9yi
Bug#11748352 - 36002: PREPARED STATEMENTS: IF A VIEW USED IN A STATEMENT IS REPLACED, BAD DATA.

Verified that the problem is applicable to our code, test case from the above revision fails, original test case from BUG#36002 fails.
Also verified that with fully applied changeset test cases succeed.

Merge pushed, remaining items will be moved to another post-10.0.4 task.