Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7258

[PATCH] Read free replication for TokuDB

    Details

      Description

      Read free replication avoids unnecessary row reads on the replication slave when executing write, update, or delete events. This can significantly increase the throughput of a replication slave that was previously I/O bound.

      The feature is described here:
      https://github.com/Tokutek/tokudb-engine/wiki/Read-Free-Replication-with-TokuDB

      The benefit is described here:
      http://www.tokutek.com/2014/09/tokudb-v7-5-read-free-replication-the-benchmark/

      A patch for MariaDB 10.0.15:
      https://github.com/Tokutek/mariadb-10.0/commit/9992195def7e739b4034902b478ee1b448b0e21a

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            elenst Elena Stepanova added a comment -

            I've set the target version to 10.0 because that's what the patch is for; not sure however if it can go to the stable release.

            Show
            elenst Elena Stepanova added a comment - I've set the target version to 10.0 because that's what the patch is for; not sure however if it can go to the stable release.
            Hide
            yoshinori.matsunobu@gmail.com Yoshinori Matsunobu added a comment - - edited

            I'm interested in this feature but have a question about handler api changes (https://github.com/Tokutek/mariadb-10.0/commit/9992195def7e739b4034902b478ee1b448b0e21a#diff-8c78bfc758be72b7df2306f1d9984c47R12217).

            Why are below functions needed?

            + virtual void rpl_before_write_rows() {
            + }
            + virtual void rpl_after_write_rows() {
            + }
            + virtual void rpl_before_delete_rows() {
            + }
            + virtual void rpl_after_delete_rows() {
            + }
            + virtual void rpl_before_update_rows() {
            + }
            + virtual void rpl_after_update_rows() {
            + }
            

            I assume what needs to be done is skipping reads for update/delete row events at Rows_log_event::find_row(), because BI (before image) exists.
            Instead of just calling rpl_lookup_rows(), how about checking binlog event types here like this?
            current patch:

            + if (!table->file->rpl_lookup_rows())
            +   DBUG_RETURN(0);
            

            proposed patch:

            + uint event_type= this->get_type_code();
            +  if ((event_type == UPDATE_ROWS_EVENT || event_type == DELETE_ROWS_EVENT) &&
            +    !table->file->rpl_lookup_rows())
            + DBUG_RETURN(0);
            

            (and removing all rpl_before/after_write/update/delete* handler apis)

            Show
            yoshinori.matsunobu@gmail.com Yoshinori Matsunobu added a comment - - edited I'm interested in this feature but have a question about handler api changes ( https://github.com/Tokutek/mariadb-10.0/commit/9992195def7e739b4034902b478ee1b448b0e21a#diff-8c78bfc758be72b7df2306f1d9984c47R12217 ). Why are below functions needed? + virtual void rpl_before_write_rows() { + } + virtual void rpl_after_write_rows() { + } + virtual void rpl_before_delete_rows() { + } + virtual void rpl_after_delete_rows() { + } + virtual void rpl_before_update_rows() { + } + virtual void rpl_after_update_rows() { + } I assume what needs to be done is skipping reads for update/delete row events at Rows_log_event::find_row(), because BI (before image) exists. Instead of just calling rpl_lookup_rows(), how about checking binlog event types here like this? current patch: + if (!table->file->rpl_lookup_rows()) + DBUG_RETURN(0); proposed patch: + uint event_type= this->get_type_code(); + if ((event_type == UPDATE_ROWS_EVENT || event_type == DELETE_ROWS_EVENT) && + !table->file->rpl_lookup_rows()) + DBUG_RETURN(0); (and removing all rpl_before/after_write/update/delete* handler apis)
            Hide
            prohaska7 Rich Prohaska added a comment -

            The replication slave does extra reads in the replication event execution layer and in the storage engine. The reads in the replication layer are there I assume to detect table mismatch between the master and the slave. We think that this check has too much overhead. The reads in the storage engine for write, delete, and update replication events occur to enforce any unique key constraints.

            We added the rpl_lookup_rows handler function to avoid the reads in the replication layer.

            We added the other handler functions to avoid extra reads in the storage engine. We wanted to know that the replication layer is executing a write rows, delete rows, or update rows replication event.

            I suspect that the storage engine handler could maintain a one shot flag which gets set in the rpl_lookup_rows function and used in subsequent calls to the handler write_row, delete_row, and update_row functions. This could eliminate the need for these extra handler functions.

            Show
            prohaska7 Rich Prohaska added a comment - The replication slave does extra reads in the replication event execution layer and in the storage engine. The reads in the replication layer are there I assume to detect table mismatch between the master and the slave. We think that this check has too much overhead. The reads in the storage engine for write, delete, and update replication events occur to enforce any unique key constraints. We added the rpl_lookup_rows handler function to avoid the reads in the replication layer. We added the other handler functions to avoid extra reads in the storage engine. We wanted to know that the replication layer is executing a write rows, delete rows, or update rows replication event. I suspect that the storage engine handler could maintain a one shot flag which gets set in the rpl_lookup_rows function and used in subsequent calls to the handler write_row, delete_row, and update_row functions. This could eliminate the need for these extra handler functions.
            Hide
            serg Sergei Golubchik added a comment -

            What is rpl_lookup_rows() supposed to do? I will need to document all API functions.

            A one shot flag is a good idea. But as far as I can see rpl_lookup_rows() isn't called for Write_rows_log_event. Or is it?

            Show
            serg Sergei Golubchik added a comment - What is rpl_lookup_rows() supposed to do? I will need to document all API functions. A one shot flag is a good idea. But as far as I can see rpl_lookup_rows() isn't called for Write_rows_log_event. Or is it?
            Hide
            prohaska7 Rich Prohaska added a comment -
            Show
            prohaska7 Rich Prohaska added a comment - I added comments to the code on my 10.0-rfr branch. See https://github.com/Tokutek/mariadb-server/commit/0e5f4a2bd740502814bdfc652bdd3d4ec08112ef

              People

              • Assignee:
                serg Sergei Golubchik
                Reporter:
                prohaska7 Rich Prohaska
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: