1.) There's nothing else in the logs except notes that IPs cannot be resolved.
2.) About 70 qps.
3.) After the first crash i found this in logs:
Nov 13 11:43:09 server01 mysqld: 141113 11:43:09 InnoDB: Error: table `<censored>`.`serial_menu` does not exist in the InnoDB internal
Nov 13 11:43:09 server01 mysqld: InnoDB: data dictionary though MySQL is trying to drop it.
Nov 13 11:43:09 server01 mysqld: InnoDB: Have you copied the .frm file of the table to the
Nov 13 11:43:09 server01 mysqld: InnoDB: MySQL database directory from another database?
Nov 13 11:43:09 server01 mysqld: InnoDB: You can look for further help from
Nov 13 11:43:09 server01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.5/en/innodb-troubleshooting.html
Nov 13 11:43:09 server01 mysqld: 141113 11:43:09 InnoDB: Error creating file './<censored>/serial_menu.ibd'.
Nov 13 11:43:09 server01 mysqld: 141113 11:43:09 InnoDB: Operating system error number 17 in a file operation.
Nov 13 11:43:09 server01 mysqld: InnoDB: Error number 17 means 'File exists'.
Nov 13 11:43:09 server01 mysqld: InnoDB: Some operating system error numbers are described at
Nov 13 11:43:09 server01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html
Nov 13 11:43:09 server01 mysqld: InnoDB: The file already exists though the corresponding table did not
Nov 13 11:43:09 server01 mysqld: InnoDB: exist in the InnoDB data dictionary. Have you moved InnoDB
Nov 13 11:43:09 server01 mysqld: InnoDB: .ibd files around without using the SQL commands
Nov 13 11:43:09 server01 mysqld: InnoDB: DISCARD TABLESPACE and IMPORT TABLESPACE, or did
Nov 13 11:43:09 server01 mysqld: InnoDB: mysqld crash in the middle of CREATE TABLE? You can
Nov 13 11:43:09 server01 mysqld: InnoDB: resolve the problem by removing the file './<censored>/serial_menu.ibd'
Nov 13 11:43:09 server01 mysqld: InnoDB: under the 'datadir' of MySQL.
This was probably the table which cased the crash. This is the current table structure (but i cannot tell if it's the same as it was while server crashed):
CREATE TABLE `serial_menu` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`serial_id` int(11) NOT NULL,
`name` varchar(30) NOT NULL,
`slug` varchar(50) NOT NULL,
`page_id` int(10) NOT NULL,
`visible` int(1) NOT NULL DEFAULT '1',
`default` int(1) NOT NULL DEFAULT '0',
`order` int(5) NOT NULL,
PRIMARY KEY (`id`),
KEY `serial_id` (`serial_id`),
KEY `page_id` (`page_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
Here is also other structure of a table from different database but with, probably, the same tables:
CREATE TABLE `serial_menu` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`serial_id` int(11) NOT NULL,
`name` varchar(30) NOT NULL,
`slug` varchar(50) NOT NULL,
`page_id` int(10) NOT NULL,
`visible` int(1) NOT NULL DEFAULT '1',
`default` int(1) NOT NULL DEFAULT '0',
`order` int(5) NOT NULL,
PRIMARY KEY (`id`),
KEY `serial_id` (`serial_id`),
KEY `page_id` (`page_id`),
CONSTRAINT `serial_menu_ibfk_1` FOREIGN KEY (`serial_id`) REFERENCES `serials` (`id`) ON DELETE CASCADE,
CONSTRAINT `serial_menu_ibfk_2` FOREIGN KEY (`page_id`) REFERENCES `serial_pages` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8
Hi,
First, could you please include a bigger part of the error logs, not starting from 'signal 11', but preferably starting from the server startup and up to the restart? (In your case the first message after startup is "[Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.").
Second, how high is the load on the server? Would it be possible to turn on general logging for one session, to find which query it actually crashes on?
Third, while the first stack trace is non-informative at all, the second one shows that the crash happened upon table creation. Does table creation happen often on that server? Do you, by any chance, know what and how was created during this time? There were 68 threads running at the moment, so I understand it's unlikely that you know what was executed at the moment, but just in case...
Thanks.