Elena Stepanova added a comment - 2015-09-03 15:12 Hi Daniel, Do you know whether it's already been filed upstream? If not, are you planning to do so? You must have a really huge transaction to have this binlog position, given that the max value for max_binlog_size is ~18 times less...

Daniel Black added a comment - 2015-09-03 15:41 haven't looked upstream yet. log-slave-update and a start slave sql_thread until relay_log .... Seems the limit got exceeded | max_binlog_size | 104857600 | -rw-rw---- 1 mysql mysql 107362623 Aug 28 13:13 mysql-bin.000057 -rw-rw---- 1 mysql mysql 104866589 Aug 28 13:13 mysql-bin.000058 -rw-rw---- 1 mysql mysql 105813602 Aug 28 13:14 mysql-bin.000059 -rw-rw---- 1 mysql mysql 105470582 Aug 28 13:14 mysql-bin.000060 -rw-rw---- 1 mysql mysql 35721262759 Sep 3 12:30 mysql-bin.000061 -rw-rw---- 1 mysql mysql 1952 Aug 28 13:14 mysql-bin.index mysql]# tail mysql-bin.index /var/lib/mysql/mysql-bin.000052 /var/lib/mysql/mysql-bin.000053 /var/lib/mysql/mysql-bin.000054 /var/lib/mysql/mysql-bin.000055 /var/lib/mysql/mysql-bin.000056 /var/lib/mysql/mysql-bin.000057 /var/lib/mysql/mysql-bin.000058 /var/lib/mysql/mysql-bin.000059 /var/lib/mysql/mysql-bin.000060 /var/lib/mysql/mysql-bin.000061

Elena Stepanova added a comment - 2015-09-03 15:52 Is it possible that there was such a huge transaction? My point is, maybe we have a bug around max_binlog_size (regardless the initial mysqlbinlog issue).

Daniel Black added a comment - 2015-09-04 00:32 no, it was mixed replication and lots of small transactions. Yes it looks like a max_binlog_size bug. start slave sql_thread until relay_log=.... is what was used on this node. I was replaying a few days of relay logs into galera as a replication slave as a test.

Daniel Black added a comment - 2015-09-04 01:50 Elena Stepanova did you want a separate bug for the max_binlog_size being exceeded?

Elena Stepanova added a comment - 2015-09-04 02:03 Daniel Black , if you don't mind filing it, it would be nice, thanks. I was going to try to reproduce it, but I can easily get distracted and forget, with a separate bug it has much better chances.

Elena Stepanova added a comment - 2015-09-04 17:02 Back to the initial problem, apparently it's not mysqlbinlog-specific, but the protocol limitation (for COM_BINLOG_DUMP). It looks even worse with change master to... : MariaDB [test]> change master to master_log_pos = 4294967296, master_log_file='wheezy-64-bin.000001'; Query OK, 0 rows affected (0.44 sec) MariaDB [test]> start slave; Query OK, 0 rows affected (0.00 sec) No warnings, no nothing, but Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position; the first event 'wheezy-64-bin.000001' at 0, the last event read from 'wheezy-64-bin.000001' at 4, the last b Note at 0 . That's how master interpreted the request: 13 Query SELECT binlog_gtid_pos('wheezy-64-bin.000001',4294967296) 13 Binlog Dump Log: 'wheezy-64-bin.000001' Pos: 0 Compare: MariaDB [test]> change master to master_log_pos = 4294967295, master_log_file='wheezy-64-bin.000001'; Query OK, 0 rows affected (0.33 sec) MariaDB [test]> start slave; Query OK, 0 rows affected (0.00 sec) Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position; the first event 'wheezy-64-bin.000001' at 4294967295, the last event read from 'wheezy-64-bin.000001' at 4, the last byte read from 'wheezy-64-bin.000001' at 4.' 14 Query SELECT binlog_gtid_pos('wheezy-64-bin.000001',4294967295) 14 Binlog Dump Log: 'wheezy-64-bin.000001' Pos: 4294967295

Elena Stepanova added a comment - 2015-09-04 18:11 - edited All in all, I suppose the fix might be quite intrusive. Also, I don't know if it makes any sense to re-report it upstream, most likely they'll suggest not to have such big binary logs...