Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 10.0.20-galera
-
Fix Version/s: None
-
Component/s: Galera, Replication, wsrep
-
Labels:
Description
If a MariaDB Galera Cluster 10.0 cluster is configured as a slave of MariaDB 10.0 using GTID, GTIDs from the master do not propagate to the WSREP replication stream. This makes it difficult to use master_use_gtid=current_pos if you ever need to switch slaves, since the non-slave nodes in the cluster won't be aware of the master's GTIDs.
Let's say we have a replication topology like this:
| MariaDB 10.0 | --- master_use_gtid=current_pos ---> | MariaDB Galera Cluster 10.0 (galera1) | <--- WSREP ---> | MariaDB Galera Cluster 10.0 (galera2) |
On the master:
MariaDB [tmp]> show global variables like 'server_id'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | server_id | 2 | +---------------+-------+ 1 row in set (0.00 sec) MariaDB [tmp]> show global variables like 'gtid_domain_id'; +----------------+-------+ | Variable_name | Value | +----------------+-------+ | gtid_domain_id | 22 | +----------------+-------+ 1 row in set (0.00 sec) MariaDB [tmp]> show global variables like 'gtid_current_pos'; +------------------+-------+ | Variable_name | Value | +------------------+-------+ | gtid_current_pos | | +------------------+-------+ 1 row in set (0.00 sec)
On galera1:
MariaDB [(none)]> show global variables like 'server_id';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id | 1 |
+---------------+-------+
1 row in set (0.00 sec)
MariaDB [(none)]> show global variables like 'gtid_domain_id';
+----------------+-------+
| Variable_name | Value |
+----------------+-------+
| gtid_domain_id | 21 |
+----------------+-------+
1 row in set (0.00 sec)
MariaDB [(none)]> show global variables like 'gtid_current_pos';
+------------------+--------+
| Variable_name | Value |
+------------------+--------+
| gtid_current_pos | 21-1-3 |
+------------------+--------+
1 row in set (0.00 sec)
MariaDB [(none)]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.1.33
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mariadb-bin.000001
Read_Master_Log_Pos: 314
Relay_Log_File: localhost-relay-bin.000002
Relay_Log_Pos: 603
Relay_Master_Log_File: mariadb-bin.000001
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 314
Relay_Log_Space: 904
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 2
Master_SSL_Crl:
Master_SSL_Crlpath:
Using_Gtid: Current_Pos
Gtid_IO_Pos: 21-1-3
1 row in set (0.00 sec)
On galera2:
MariaDB [(none)]> show global variables like 'server_id'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | server_id | 1 | +---------------+-------+ 1 row in set (0.00 sec) MariaDB [(none)]> show global variables like 'gtid_domain_id'; +----------------+-------+ | Variable_name | Value | +----------------+-------+ | gtid_domain_id | 21 | +----------------+-------+ 1 row in set (0.00 sec) MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+---------+ | Variable_name | Value | +------------------+---------+ | gtid_current_pos | 21-1-10 | +------------------+---------+ 1 row in set (0.00 sec)
Now let's insert some data on the master to see how it affects gtid_current_pos:
MariaDB [tmp]> INSERT INTO tmp.test_table VALUES (1, 'str'); Query OK, 1 row affected (0.00 sec) MariaDB [tmp]> show global variables like 'gtid_current_pos'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | gtid_current_pos | 22-2-1 | +------------------+--------+
On galera1:
MariaDB [(none)]> show global variables like 'gtid_current_pos';
+------------------+---------------+
| Variable_name | Value |
+------------------+---------------+
| gtid_current_pos | 22-2-1,21-1-3 |
+------------------+---------------+
1 row in set (0.00 sec)
MariaDB [(none)]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.1.33
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mariadb-bin.000001
Read_Master_Log_Pos: 485
Relay_Log_File: localhost-relay-bin.000002
Relay_Log_Pos: 774
Relay_Master_Log_File: mariadb-bin.000001
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 485
Relay_Log_Space: 1075
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 2
Master_SSL_Crl:
Master_SSL_Crlpath:
Using_Gtid: Current_Pos
Gtid_IO_Pos: 21-1-3,22-2-1
1 row in set (0.00 sec)
On galera2 the value is now empty:
MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+-------+ | Variable_name | Value | +------------------+-------+ | gtid_current_pos | | +------------------+-------+ 1 row in set (0.00 sec)
Gliffy Diagrams
Attachments
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
This might only happen if binlog_format on the MariaDB 10.0 master is set to STATEMENT.
If it's set to ROW (which is required for Galera Cluster anyway), the GTIDs in the replication events seem to get passed along, and gtid_current_pos is updated.
This might be working as expected.
This does also appear to happen in some instances when binlog_format is set to ROW.
It looks like the way to reproduce this with binlog_format set to ROW is by setting slave_parallel_threads on the slave. Is it required for Galera to set slave_parallel_threads to 0?
e.g. on the master:
MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | gtid_current_pos | 22-2-3 | +------------------+--------+ 1 row in set (0.00 sec) MariaDB [(none)]> INSERT INTO tmp.test_table VALUES (4, 'str'); Query OK, 1 row affected (0.05 sec) MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | gtid_current_pos | 22-2-4 | +------------------+--------+ 1 row in set (0.00 sec)
On galera1:
MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+---------------+ | Variable_name | Value | +------------------+---------------+ | gtid_current_pos | 22-2-4,21-1-3 | +------------------+---------------+ 1 row in set (0.00 sec) MariaDB [(none)]> show global variables like 'slave_parallel_threads'; +------------------------+-------+ | Variable_name | Value | +------------------------+-------+ | slave_parallel_threads | 2 | +------------------------+-------+ 1 row in set (0.00 sec)
On galera2:
MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+-------+ | Variable_name | Value | +------------------+-------+ | gtid_current_pos | | +------------------+-------+ 1 row in set (0.00 sec)
master binlog:
#150714 11:00:01 server id 2 end_log_pos 520 GTID 22-2-4
/*!100001 SET @@session.gtid_seq_no=4*//*!*/;
BEGIN
/*!*/;
# at 520
# at 571
#150714 11:00:01 server id 2 end_log_pos 571 Table_map: `tmp`.`test_table` mapped to number 70
#150714 11:00:01 server id 2 end_log_pos 609 Write_rows: table id 70 flags: STMT_END_F
### INSERT INTO `tmp`.`test_table`
### SET
### @1=4
### @2='str'
# at 609
#150714 11:00:01 server id 2 end_log_pos 636 Xid = 38
COMMIT/*!*/;
galera1 binlog:
#150714 11:00:01 server id 2 end_log_pos 382 GTID 22-2-4 /*!100001 SET @@session.gtid_domain_id=22*//*!*/; /*!100001 SET @@session.server_id=2*//*!*/; /*!100001 SET @@session.gtid_seq_no=4*//*!*/; BEGIN /*!*/; # at 382 # at 433 #150714 11:00:01 server id 2 end_log_pos 433 Table_map: `tmp`.`test_table` mapped to number 70 #150714 11:00:01 server id 2 end_log_pos 471 Write_rows: table id 70 flags: STMT_END_F ### INSERT INTO `tmp`.`test_table` ### SET ### @1=4 ### @2='str' # at 471 #150714 11:00:01 server id 2 end_log_pos 498 Xid = 24 COMMIT/*!*/;
galera2 binlog:
#150714 11:00:01 server id 2 end_log_pos 398 GTID 21-2-14 /*!100001 SET @@session.gtid_domain_id=21*//*!*/; /*!100001 SET @@session.server_id=2*//*!*/; /*!100001 SET @@session.gtid_seq_no=14*//*!*/; BEGIN /*!*/; # at 398 # at 449 #150714 11:00:01 server id 2 end_log_pos 449 Table_map: `tmp`.`test_table` mapped to number 70 #150714 11:00:01 server id 2 end_log_pos 487 Write_rows: table id 70 flags: STMT_END_F ### INSERT INTO `tmp`.`test_table` ### SET ### @1=4 ### @2='str' # at 487 #150714 11:00:01 server id 2 end_log_pos 514 Xid = 24 COMMIT/*!*/;
Same issue with:
binlog_format=ROW
slave_parallel_threads=0
wsrep_slave_threads=2
and:
binlog_format=ROW
slave_parallel_threads=0
wsrep_slave_threads=1
I was able to see the GTID propagate to galera2 yesterday with the second set of values, but I haven't been able to make that happen again today. There might be some other interaction going on here.
This problem might be unrelated to the replication thread.
Leaving out the master server completely, I inserted a value directly into galera1. Galera2 assigned the transaction a different GTID.
galera1:
MariaDB [(none)]> INSERT INTO tmp.test_table VALUES (12, 'str'); Query OK, 1 row affected (0.01 sec) MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+----------------+ | Variable_name | Value | +------------------+----------------+ | gtid_current_pos | 22-2-11,21-1-1 | +------------------+----------------+ 1 row in set (0.01 sec)
galera2:
MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | gtid_current_pos | 21-1-2 | +------------------+--------+ 1 row in set (0.00 sec)
galera1 binlog:
#150714 12:08:59 server id 1 end_log_pos 520 GTID 21-1-1 /*!100001 SET @@session.gtid_domain_id=21*//*!*/; /*!100001 SET @@session.server_id=1*//*!*/; /*!100001 SET @@session.gtid_seq_no=1*//*!*/; BEGIN /*!*/; # at 520 # at 571 #150714 12:08:59 server id 1 end_log_pos 571 Table_map: `tmp`.`test_table` mapped to number 70 #150714 12:08:59 server id 1 end_log_pos 609 Write_rows: table id 70 flags: STMT_END_F ### INSERT INTO `tmp`.`test_table` ### SET ### @1=12 ### @2='str' # at 609 #150714 12:08:59 server id 1 end_log_pos 636 Xid = 31 COMMIT/*!*/;
galera2 binlog:
#150714 12:08:59 server id 1 end_log_pos 506 GTID 21-1-2 /*!100001 SET @@session.server_id=1*//*!*/; /*!100001 SET @@session.gtid_seq_no=2*//*!*/; BEGIN /*!*/; # at 506 # at 557 #150714 12:08:59 server id 1 end_log_pos 557 Table_map: `tmp`.`test_table` mapped to number 70 #150714 12:08:59 server id 1 end_log_pos 595 Write_rows: table id 70 flags: STMT_END_F ### INSERT INTO `tmp`.`test_table` ### SET ### @1=12 ### @2='str' # at 595 #150714 12:08:59 server id 1 end_log_pos 622 Xid = 31 COMMIT/*!*/;
Also, galera1 binlog:
galera2 binlog: