PATCH: no slave left behind

Description

patch has been ported to 10.1
you can use it under new (also called "3-clause") BSD license
i have done internal benchmarks...but you are most welcome to do some too.

DESCRIPTION:
no slave left behind

this patch implements master throttling based on slave lag,
aka no slave left behind. the core feature works as follows
1) the semi-sync-reply is ammended to also report back SQL-thread
position (aka exec position)
2) transactions are not removed from the "active-transaction-list"
in the semi-sync-master plugin until atleast one slave has reported
that it has executed this transaction. the slave lag can then
be estimated by calculating how long the oldest transaction has been
lingering in the active-transaction-list.
3) client-threads are forced to wait before commit until slave lag
has decreased to acceptable value.

the following variables are introduced on master:

  • rpl_semi_sync_master_max_slave_lag (global)

  • rpl_semi_sync_master_slave_lag_wait_timeout (session)


the following status variables are introduced on master:

  • rpl_semi_sync_master_slave_lag_wait_sessions

  • rpl_semi_sync_master_estimated_slave_lag

  • rpl_semi_sync_master_trx_slave_lag_wait_time

  • rpl_semi_sync_master_trx_slave_lag_wait_num

  • rpl_semi_sync_master_avg_trx_slave_lag_wait_time


the following variables are introduced on slave:

  • rpl_semi_sync_slave_lag_enabled (global)


in addition to this, 2 optimizations that decreases overhead of semi-sync
is introduced.
1) the idea of this is that if when a slave should send and transaction,
it checks if it should be semi-synced, but rather
than semi-sync:ing each transaction (which is done currently) the code
will skip semi-syncing transaction if there already is newer transactions
committed. But, since this can mean that semi-syncing is delayed indefinitely
a cap is set using 2 new master variables:

  • rpl_semi_sync_master_max_unacked_event_bytes (global)

  • rpl_semi_sync_master_max_unacked_event_count (global)
    2) rpl_semi_sync_master_group_commit which makes the semi-sync
    plugin only semi-sync the last transaction in a group commit.

Environment

None

Status

Assignee

Kristian Nielsen

Reporter

Jonas Oreland

Labels

None

External issue ID

None

External issue ID

None

Time tracking

8h

Fix versions

Priority

Major