Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8720

Binary logs deleted after machine restarts

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 10.0.19-galera
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      I'm trying to set up replication of mariadb cluster. I have set up slave to replicate from correct binlog filename/position and everything works fine, but if I restart that machine (master), old binary log is "gone", "show binary logs" doesn't display it anymore (while physically it still exists on disk) and thus replication fails with error:

      Error reading packet from server: Could not find first log file name in binary log index file ( server_errno=1236)

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            elenst Elena Stepanova added a comment -

            Was it a one-time thing, or are you able to reproduce it?

            If you can reproduce it, please

            • run SHOW BINARY LOGS;
            • run SHOW MASTER STATUS;
            • run ls -l <your binlog location>/*
            • restart the server;
            • run all the above again;
            • paste the output.

            Please also attach your cnf file from master.

            Show
            elenst Elena Stepanova added a comment - Was it a one-time thing, or are you able to reproduce it? If you can reproduce it, please run SHOW BINARY LOGS ; run SHOW MASTER STATUS ; run ls -l <your binlog location>/* restart the server; run all the above again; paste the output. Please also attach your cnf file from master.
            Hide
            jbaramidze Zhani Baramidze added a comment - - edited

            happens all the time:

            MariaDB [dbname]> show binary logs;
            +---------------+-----------+
            | Log_name      | File_size |
            +---------------+-----------+
            | binlog.000008 |       309 |
            +---------------+-----------+
            
            MariaDB [dbname]> show master status;
            +---------------+----------+--------------+------------------+
            | File          | Position | Binlog_Do_DB | Binlog_Ignore_DB |
            +---------------+----------+--------------+------------------+
            | binlog.000008 |      309 |              |                  |
            +---------------+----------+--------------+------------------+
            
            root@fd17ddaaef90:/var/lib/mysql# ls -l
            total 243816
            -rw-rw---- 1 mysql mysql     16384 Sep  1 14:29 aria_log.00000001
            -rw-rw---- 1 mysql mysql        52 Sep  1 14:29 aria_log_control
            -rw-rw---- 1 mysql mysql       358 Sep  1 14:23 backup-my.cnf
            -rw-rw---- 1 mysql mysql       375 Sep  1 14:23 binlog.000006
            -rw-rw---- 1 mysql mysql       328 Sep  1 14:29 binlog.000007
            -rw-rw---- 1 mysql mysql       309 Sep  1 14:30 binlog.000008
            -rw-rw---- 1 mysql mysql        16 Sep  1 14:23 binlog.index
            drwx------ 2 mysql mysql      4096 Sep  1 14:23 dbname
            -rw------- 1 mysql mysql 134219048 Sep  1 14:30 galera.cache
            -rw-rw---- 1 mysql mysql       104 Sep  1 14:30 grastate.dat
            -rw-rw---- 1 mysql mysql       218 Sep  1 14:30 gvwstate.dat
            -rw-rw---- 1 mysql mysql  50331648 Sep  1 14:30 ib_logfile0
            -rw-rw---- 1 mysql mysql  50331648 Sep  1 14:23 ib_logfile1
            -rw-rw---- 1 mysql mysql  12582912 Sep  1 14:30 ibdata1
            -rw-rw---- 1 mysql mysql      6159 Sep  1 12:11 innobackup.backup.log
            -rw-rw---- 1 mysql mysql      5507 Sep  1 14:23 innobackup.prepare.log
            -rw-rw---- 1 mysql mysql         0 Sep  1 14:23 multi-master.info
            drwx------ 2 mysql mysql      4096 Sep  1 14:23 mysql
            drwx------ 2 mysql mysql      4096 Sep  1 14:23 performance_schema
            drwx------ 2 mysql mysql      4096 Sep  1 14:23 turn
            -rw-rw---- 1 mysql mysql        25 Sep  1 14:23 xtrabackup_binlog_info
            -rw-rw---- 1 mysql mysql        20 Sep  1 14:23 xtrabackup_binlog_pos_innodb
            -rw-rw---- 1 mysql mysql        89 Sep  1 14:23 xtrabackup_checkpoints
            -rw-rw---- 1 mysql mysql        40 Sep  1 14:23 xtrabackup_galera_info
            -rw-rw---- 1 mysql mysql       775 Sep  1 14:23 xtrabackup_info
            -rw-rw---- 1 mysql mysql   2097152 Sep  1 14:23 xtrabackup_logfile
            

            after restart:

            MariaDB [dbname]> show binary logs;
            +---------------+-----------+
            | Log_name      | File_size |
            +---------------+-----------+
            | binlog.000009 |       309 |
            +---------------+-----------+
            
            MariaDB [dbname]> show master status;
            +---------------+----------+--------------+------------------+
            | File          | Position | Binlog_Do_DB | Binlog_Ignore_DB |
            +---------------+----------+--------------+------------------+
            | binlog.000009 |      309 |              |                  |
            +---------------+----------+--------------+------------------+
            
            root@9b14c52a45c8:/var/lib/mysql# ls -l
            total 243820
            -rw-rw---- 1 mysql mysql     16384 Sep  1 14:33 aria_log.00000001
            -rw-rw---- 1 mysql mysql        52 Sep  1 14:33 aria_log_control
            -rw-rw---- 1 mysql mysql       358 Sep  1 14:23 backup-my.cnf
            -rw-rw---- 1 mysql mysql       375 Sep  1 14:23 binlog.000006
            -rw-rw---- 1 mysql mysql       328 Sep  1 14:29 binlog.000007
            -rw-rw---- 1 mysql mysql       328 Sep  1 14:33 binlog.000008
            -rw-rw---- 1 mysql mysql       309 Sep  1 14:33 binlog.000009
            -rw-rw---- 1 mysql mysql        16 Sep  1 14:23 binlog.index
            drwx------ 2 mysql mysql      4096 Sep  1 14:23 dbname
            -rw------- 1 mysql mysql 134219048 Sep  1 14:33 galera.cache
            -rw-rw---- 1 mysql mysql       104 Sep  1 14:33 grastate.dat
            -rw-rw---- 1 mysql mysql       218 Sep  1 14:33 gvwstate.dat
            -rw-rw---- 1 mysql mysql  50331648 Sep  1 14:33 ib_logfile0
            -rw-rw---- 1 mysql mysql  50331648 Sep  1 14:23 ib_logfile1
            -rw-rw---- 1 mysql mysql  12582912 Sep  1 14:33 ibdata1
            -rw-rw---- 1 mysql mysql      6159 Sep  1 12:11 innobackup.backup.log
            -rw-rw---- 1 mysql mysql      5507 Sep  1 14:23 innobackup.prepare.log
            -rw-rw---- 1 mysql mysql         0 Sep  1 14:23 multi-master.info
            drwx------ 2 mysql mysql      4096 Sep  1 14:23 mysql
            drwx------ 2 mysql mysql      4096 Sep  1 14:23 performance_schema
            drwx------ 2 mysql mysql      4096 Sep  1 14:23 turn
            -rw-rw---- 1 mysql mysql        25 Sep  1 14:23 xtrabackup_binlog_info
            -rw-rw---- 1 mysql mysql        20 Sep  1 14:23 xtrabackup_binlog_pos_innodb
            -rw-rw---- 1 mysql mysql        89 Sep  1 14:23 xtrabackup_checkpoints
            -rw-rw---- 1 mysql mysql        40 Sep  1 14:23 xtrabackup_galera_info
            -rw-rw---- 1 mysql mysql       775 Sep  1 14:23 xtrabackup_info
            -rw-rw---- 1 mysql mysql   2097152 Sep  1 14:23 xtrabackup_logfile
            
            Show
            jbaramidze Zhani Baramidze added a comment - - edited happens all the time: MariaDB [dbname]> show binary logs; +---------------+-----------+ | Log_name | File_size | +---------------+-----------+ | binlog.000008 | 309 | +---------------+-----------+ MariaDB [dbname]> show master status; +---------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +---------------+----------+--------------+------------------+ | binlog.000008 | 309 | | | +---------------+----------+--------------+------------------+ root@fd17ddaaef90:/var/lib/mysql# ls -l total 243816 -rw-rw---- 1 mysql mysql 16384 Sep 1 14:29 aria_log.00000001 -rw-rw---- 1 mysql mysql 52 Sep 1 14:29 aria_log_control -rw-rw---- 1 mysql mysql 358 Sep 1 14:23 backup-my.cnf -rw-rw---- 1 mysql mysql 375 Sep 1 14:23 binlog.000006 -rw-rw---- 1 mysql mysql 328 Sep 1 14:29 binlog.000007 -rw-rw---- 1 mysql mysql 309 Sep 1 14:30 binlog.000008 -rw-rw---- 1 mysql mysql 16 Sep 1 14:23 binlog.index drwx------ 2 mysql mysql 4096 Sep 1 14:23 dbname -rw------- 1 mysql mysql 134219048 Sep 1 14:30 galera.cache -rw-rw---- 1 mysql mysql 104 Sep 1 14:30 grastate.dat -rw-rw---- 1 mysql mysql 218 Sep 1 14:30 gvwstate.dat -rw-rw---- 1 mysql mysql 50331648 Sep 1 14:30 ib_logfile0 -rw-rw---- 1 mysql mysql 50331648 Sep 1 14:23 ib_logfile1 -rw-rw---- 1 mysql mysql 12582912 Sep 1 14:30 ibdata1 -rw-rw---- 1 mysql mysql 6159 Sep 1 12:11 innobackup.backup.log -rw-rw---- 1 mysql mysql 5507 Sep 1 14:23 innobackup.prepare.log -rw-rw---- 1 mysql mysql 0 Sep 1 14:23 multi-master.info drwx------ 2 mysql mysql 4096 Sep 1 14:23 mysql drwx------ 2 mysql mysql 4096 Sep 1 14:23 performance_schema drwx------ 2 mysql mysql 4096 Sep 1 14:23 turn -rw-rw---- 1 mysql mysql 25 Sep 1 14:23 xtrabackup_binlog_info -rw-rw---- 1 mysql mysql 20 Sep 1 14:23 xtrabackup_binlog_pos_innodb -rw-rw---- 1 mysql mysql 89 Sep 1 14:23 xtrabackup_checkpoints -rw-rw---- 1 mysql mysql 40 Sep 1 14:23 xtrabackup_galera_info -rw-rw---- 1 mysql mysql 775 Sep 1 14:23 xtrabackup_info -rw-rw---- 1 mysql mysql 2097152 Sep 1 14:23 xtrabackup_logfile after restart: MariaDB [dbname]> show binary logs; +---------------+-----------+ | Log_name | File_size | +---------------+-----------+ | binlog.000009 | 309 | +---------------+-----------+ MariaDB [dbname]> show master status; +---------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +---------------+----------+--------------+------------------+ | binlog.000009 | 309 | | | +---------------+----------+--------------+------------------+ root@9b14c52a45c8:/var/lib/mysql# ls -l total 243820 -rw-rw---- 1 mysql mysql 16384 Sep 1 14:33 aria_log.00000001 -rw-rw---- 1 mysql mysql 52 Sep 1 14:33 aria_log_control -rw-rw---- 1 mysql mysql 358 Sep 1 14:23 backup-my.cnf -rw-rw---- 1 mysql mysql 375 Sep 1 14:23 binlog.000006 -rw-rw---- 1 mysql mysql 328 Sep 1 14:29 binlog.000007 -rw-rw---- 1 mysql mysql 328 Sep 1 14:33 binlog.000008 -rw-rw---- 1 mysql mysql 309 Sep 1 14:33 binlog.000009 -rw-rw---- 1 mysql mysql 16 Sep 1 14:23 binlog.index drwx------ 2 mysql mysql 4096 Sep 1 14:23 dbname -rw------- 1 mysql mysql 134219048 Sep 1 14:33 galera.cache -rw-rw---- 1 mysql mysql 104 Sep 1 14:33 grastate.dat -rw-rw---- 1 mysql mysql 218 Sep 1 14:33 gvwstate.dat -rw-rw---- 1 mysql mysql 50331648 Sep 1 14:33 ib_logfile0 -rw-rw---- 1 mysql mysql 50331648 Sep 1 14:23 ib_logfile1 -rw-rw---- 1 mysql mysql 12582912 Sep 1 14:33 ibdata1 -rw-rw---- 1 mysql mysql 6159 Sep 1 12:11 innobackup.backup.log -rw-rw---- 1 mysql mysql 5507 Sep 1 14:23 innobackup.prepare.log -rw-rw---- 1 mysql mysql 0 Sep 1 14:23 multi-master.info drwx------ 2 mysql mysql 4096 Sep 1 14:23 mysql drwx------ 2 mysql mysql 4096 Sep 1 14:23 performance_schema drwx------ 2 mysql mysql 4096 Sep 1 14:23 turn -rw-rw---- 1 mysql mysql 25 Sep 1 14:23 xtrabackup_binlog_info -rw-rw---- 1 mysql mysql 20 Sep 1 14:23 xtrabackup_binlog_pos_innodb -rw-rw---- 1 mysql mysql 89 Sep 1 14:23 xtrabackup_checkpoints -rw-rw---- 1 mysql mysql 40 Sep 1 14:23 xtrabackup_galera_info -rw-rw---- 1 mysql mysql 775 Sep 1 14:23 xtrabackup_info -rw-rw---- 1 mysql mysql 2097152 Sep 1 14:23 xtrabackup_logfile
            Hide
            jbaramidze Zhani Baramidze added a comment -

            master, running in cluster, started as:
            mysqld --log-bin=binlog --skip-name-resolve --log-slave-updates=ON
            I'll attach my.cnf

            Show
            jbaramidze Zhani Baramidze added a comment - master, running in cluster, started as: mysqld --log-bin=binlog --skip-name-resolve --log-slave-updates=ON I'll attach my.cnf
            Hide
            jbaramidze Zhani Baramidze added a comment -

            slave is trying to continue from 'binlog.000011', position 628, while in 0011 I have:

            1. at 628
              #150901 15:38:13 server id 1 end_log_pos 647 Stop
              DELIMITER ;
            2. End of log file
              ROLLBACK /* added by mysqlbinlog */;
              /!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE/;
              /!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0/;

            and show binarry logs doesn't show 0011 anymore

            Show
            jbaramidze Zhani Baramidze added a comment - slave is trying to continue from 'binlog.000011', position 628, while in 0011 I have: at 628 #150901 15:38:13 server id 1 end_log_pos 647 Stop DELIMITER ; End of log file ROLLBACK /* added by mysqlbinlog */; / !50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE /; / !50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0 /; and show binarry logs doesn't show 0011 anymore

              People

              • Assignee:
                nirbhay_c Nirbhay Choubey
                Reporter:
                jbaramidze Zhani Baramidze
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: