Create a table like this
Then run the following query to crash the server:
The query will enter the optimizing state and start consuming memory (and swap, if applicable) until the server runs out of memory and mysqld is killed.
Note that it does not matter whether the table is an InnoDB or MyISAM table. MySQL 5.5.29 does not exhibit this behavior.