Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-5146

Bulk loads into partitioned table not working.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.5.33a-galera
    • Fix Version/s: 5.5.46-galera, 10.0.22-galera
    • Component/s: wsrep
    • Labels:
      None
    • Environment:
      Centos 6.3 64 bit.

      Description

      We have:
      wsrep_load_data_splitting=on
      in our server.cnf file. Bulk loads into a partition table do not get replicated to the other nodes.

      Log file shows:

      131007 10:38:54 [Warning] WSREP: SQL statement was ineffective: LOAD DATA INFILE '/workspace3/tempspace/OnTargetPublish.load_file.24194.20131007103841' IGNORE
      INTO TABLE TRIGGERED_SENDS.LOG_BASED_SENDS_UNIV
      FIELDS TERMINATED BY '  '
      LINES TERMINATED BY '\n'
      (
              EMAIL_KEY,
              SEND_ID,
              @START_DTTM,
              @END_DTTM,
              ACTIVE_IND,
              CHR_DATA_OBJ
      )
      SET START_DTTM = STR_TO_DATE(@START_DTTM,'%Y%m%d%H%i%s'),
      END_DTTM = STR_TO_DATE(@END_DTTM,'%Y%m%d%H%i%s'),
      EVENT_DTTM = NULL,
      TO_DB_DTTM = NOW(),
      UPDT_DB_DTTM = NOW()
       => Skipping replication
      

      I think that may be normal - we want the records replicated, but not the sql statement itself.

      The table is partitoned on SEND_ID.
      I'm not sure if it is the partition table that is causing the issue or something else.

      Let me know if you need anything else to debug or have anything else you want us to try.
      Thanks.

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            elenst Elena Stepanova added a comment -

            Hi Matthew,

            Could you please provide the full cnf file(s)?
            Thanks.

            Show
            elenst Elena Stepanova added a comment - Hi Matthew, Could you please provide the full cnf file(s)? Thanks.
            Hide
            matt.wheeler Matthew Wheeler added a comment -

            from the node that it is submitted to - 4c-maria-01:

             
            [server]
            
            [mysqld]
            socket          = /database/mysql.sock
            port            = 3306
            datadir         = /database/data
            tmpdir          = /database/tmp/
            innodb_log_group_home_dir = /database/logFiles
            innodb_data_home_dir = /database/data
            
            # logs and replication
            log-bin=mysql-bin
            binlog-format=ROW
            
            server-id=175
            
            max_binlog_cache_size=1024G
            
            max_connections=255
            max_connect_errors=10000
            max_allowed_packet=128M
            
            # Galera Settings
            wsrep_provider=/usr/lib64/galera/libgalera_smm.so
            wsrep_cluster_address=gcomm://10.200.0.11
            wsrep_cluster_name='client01_cluster'
            wsrep_node_name='4c-maria-01'
            wsrep_slave_threads=24
            wsrep_retry_autocommit=10
            wsrep_sst_method=xtrabackup
            wsrep_sst_auth=galera:somepassword
            wsrep_load_data_splitting=on
            #wsrep_notify_cmd=/bin/wsrep_notify.sh
            
            # innodb (xtradb) settings
            default_storage_engine=InnoDB
            innodb_file_per_table
            innodb_file_format=barracuda
            innodb_log_file_size=2000M
            innodb_log_files_in_group=2
            innodb_flush_log_at_trx_commit=2
            innodb_autoinc_lock_mode=2
            innodb_locks_unsafe_for_binlog=1
            
            # For zfs we cannot use O_DIRECT
            #innodb_flush_method=O_DIRECT
            
            # For zfs native aio fails on tmpfs
            innodb_use_native_aio=FALSE
            
            # zfs prevents partial writes already.  No need to have MariaDB issue them.
            innodb_doublewrite=0
            
            innodb_io_capacity=1400
            innodb_thread_concurrency=0
            innodb_read_io_threads=24
            innodb_write_io_threads=24
            innodb_purge_threads=1
            
            innodb_buffer_pool_size=44G
            innodb_buffer_pool_instances=8
            innodb_additional_mem_pool_size=20M
            
            [mariadb-5.5]
            thread_handling=pool-of-threads
            

            and the second node - 4c-maria-02:

             
            [server]
            
            [mysqld]
            socket          = /database/mysql.sock
            port            = 3306
            datadir         = /database/data
            tmpdir          = /database/tmp/
            innodb_log_group_home_dir = /database/logFiles
            innodb_data_home_dir = /database/data
            
            # logs and replication
            log-bin=mysql-bin
            binlog-format=ROW
            
            server-id=176
            
            max_binlog_cache_size=1024G
            
            max_connections=255
            max_connect_errors=10000
            max_allowed_packet=128M
            
            # Galera Settings
            wsrep_provider=/usr/lib64/galera/libgalera_smm.so
            wsrep_cluster_address=gcomm://10.200.0.10
            wsrep_cluster_name='client01_cluster'
            wsrep_node_name='4c-maria-02'
            wsrep_slave_threads=24
            wsrep_retry_autocommit=10
            wsrep_sst_method=xtrabackup
            wsrep_sst_auth=galera:somepassword
            wsrep_load_data_splitting=on
            #wsrep_notify_cmd=/bin/wsrep_notify.sh
            # Slave Settings
            wsrep_provider_options="gcs.fc_limit=256;gcs.fc_factor=0.9;gcs.fc_master_slave=yes"
            
            # innodb (xtradb) settings
            default_storage_engine=InnoDB
            innodb_file_per_table
            innodb_file_format=barracuda
            innodb_log_file_size=2000M
            innodb_log_files_in_group=2
            innodb_flush_log_at_trx_commit=2
            innodb_autoinc_lock_mode=2
            innodb_locks_unsafe_for_binlog=1
            
            # For zfs we cannot use O_DIRECT
            #innodb_flush_method=O_DIRECT
            
            # For zfs native aio fails on tmpfs
            innodb_use_native_aio=FALSE
            
            # zfs prevents partial writes already.  No need to have MariaDB issue them.
            innodb_doublewrite=0
            
            innodb_io_capacity=1400
            innodb_thread_concurrency=0
            innodb_read_io_threads=24
            innodb_write_io_threads=24
            innodb_purge_threads=1
            
            innodb_buffer_pool_size=44G
            innodb_buffer_pool_instances=8
            innodb_additional_mem_pool_size=20M
            
            [mariadb-5.5]
            thread_handling=pool-of-threads
            
            Show
            matt.wheeler Matthew Wheeler added a comment - from the node that it is submitted to - 4c-maria-01: [server] [mysqld] socket = /database/mysql.sock port = 3306 datadir = /database/data tmpdir = /database/tmp/ innodb_log_group_home_dir = /database/logFiles innodb_data_home_dir = /database/data # logs and replication log-bin=mysql-bin binlog-format=ROW server-id=175 max_binlog_cache_size=1024G max_connections=255 max_connect_errors=10000 max_allowed_packet=128M # Galera Settings wsrep_provider=/usr/lib64/galera/libgalera_smm.so wsrep_cluster_address=gcomm://10.200.0.11 wsrep_cluster_name='client01_cluster' wsrep_node_name='4c-maria-01' wsrep_slave_threads=24 wsrep_retry_autocommit=10 wsrep_sst_method=xtrabackup wsrep_sst_auth=galera:somepassword wsrep_load_data_splitting=on #wsrep_notify_cmd=/bin/wsrep_notify.sh # innodb (xtradb) settings default_storage_engine=InnoDB innodb_file_per_table innodb_file_format=barracuda innodb_log_file_size=2000M innodb_log_files_in_group=2 innodb_flush_log_at_trx_commit=2 innodb_autoinc_lock_mode=2 innodb_locks_unsafe_for_binlog=1 # For zfs we cannot use O_DIRECT #innodb_flush_method=O_DIRECT # For zfs native aio fails on tmpfs innodb_use_native_aio=FALSE # zfs prevents partial writes already. No need to have MariaDB issue them. innodb_doublewrite=0 innodb_io_capacity=1400 innodb_thread_concurrency=0 innodb_read_io_threads=24 innodb_write_io_threads=24 innodb_purge_threads=1 innodb_buffer_pool_size=44G innodb_buffer_pool_instances=8 innodb_additional_mem_pool_size=20M [mariadb-5.5] thread_handling=pool-of-threads and the second node - 4c-maria-02: [server] [mysqld] socket = /database/mysql.sock port = 3306 datadir = /database/data tmpdir = /database/tmp/ innodb_log_group_home_dir = /database/logFiles innodb_data_home_dir = /database/data # logs and replication log-bin=mysql-bin binlog-format=ROW server-id=176 max_binlog_cache_size=1024G max_connections=255 max_connect_errors=10000 max_allowed_packet=128M # Galera Settings wsrep_provider=/usr/lib64/galera/libgalera_smm.so wsrep_cluster_address=gcomm://10.200.0.10 wsrep_cluster_name='client01_cluster' wsrep_node_name='4c-maria-02' wsrep_slave_threads=24 wsrep_retry_autocommit=10 wsrep_sst_method=xtrabackup wsrep_sst_auth=galera:somepassword wsrep_load_data_splitting=on #wsrep_notify_cmd=/bin/wsrep_notify.sh # Slave Settings wsrep_provider_options="gcs.fc_limit=256;gcs.fc_factor=0.9;gcs.fc_master_slave=yes" # innodb (xtradb) settings default_storage_engine=InnoDB innodb_file_per_table innodb_file_format=barracuda innodb_log_file_size=2000M innodb_log_files_in_group=2 innodb_flush_log_at_trx_commit=2 innodb_autoinc_lock_mode=2 innodb_locks_unsafe_for_binlog=1 # For zfs we cannot use O_DIRECT #innodb_flush_method=O_DIRECT # For zfs native aio fails on tmpfs innodb_use_native_aio=FALSE # zfs prevents partial writes already. No need to have MariaDB issue them. innodb_doublewrite=0 innodb_io_capacity=1400 innodb_thread_concurrency=0 innodb_read_io_threads=24 innodb_write_io_threads=24 innodb_purge_threads=1 innodb_buffer_pool_size=44G innodb_buffer_pool_instances=8 innodb_additional_mem_pool_size=20M [mariadb-5.5] thread_handling=pool-of-threads
            Show
            nirbhay_c Nirbhay Choubey added a comment - http://lists.askmonty.org/pipermail/commits/2015-August/008271.html
            Hide
            jplindst Jan Lindström added a comment -

            ok, to push.

            Show
            jplindst Jan Lindström added a comment - ok, to push.
            Show
            nirbhay_c Nirbhay Choubey added a comment - https://github.com/MariaDB/server/commit/4ee28865f67c980848bb62f0009440be73ebee7c

              People

              • Assignee:
                nirbhay_c Nirbhay Choubey
                Reporter:
                matt.wheeler Matthew Wheeler
              • Votes:
                2 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 hour
                  1h