Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6372

mysqldump crashes the server sometimes

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 10.0.11
    • Fix Version/s: N/A
    • Component/s: OTHER
    • Labels:
      None
    • Environment:
      Windows Server 2012 R2

      Description

      Just recently upgraded from 5.x to 10.0.11 and noticed that the mariadb service has started crashing about once a week ever since. Looking at the timestamp in the .err file, I'm almost certain it has something to do the scheduled daily backup at 5:30.

      The relevant part from the log file:

      2014-06-21 05:31:56 de0  InnoDB: Operating system error number 6 in a file operation.
      InnoDB: Some operating system error numbers are described at
      InnoDB: http://dev.mysql.com/doc/refman/5.6/en/operating-system-error-codes.html
      140621  5:31:56 [ERROR] InnoDB: File (unknown): 'read' returned OS error 106. Cannot continue operation
      140621  6:49:27 [Note] InnoDB: Using mutexes to ref count buffer pool pages
      140621  6:49:27 [Note] InnoDB: The InnoDB memory heap is disabled
      140621  6:49:27 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
      140621  6:49:27 [Note] InnoDB: Compressed tables use zlib 1.2.3
      140621  6:49:27 [Note] InnoDB: Not using CPU crc32 instructions
      140621  6:49:27 [Note] InnoDB: Initializing buffer pool, size = 3.0G
      140621  6:49:27 [Note] InnoDB: Completed initialization of buffer pool
      140621  6:49:27 [Note] InnoDB: Highest supported file format is Barracuda.
      140621  6:49:27 [Note] InnoDB: Log scan progressed past the checkpoint lsn 548267694925
      140621  6:49:27 [Note] InnoDB: Database was not shutdown normally!
      140621  6:49:27 [Note] InnoDB: Starting crash recovery.
      140621  6:49:27 [Note] InnoDB: Reading tablespace information from the .ibd files...
      140621  6:49:32 [Note] InnoDB: Restoring possible half-written data pages 
      140621  6:49:32 [Note] InnoDB: from the doublewrite buffer...
      InnoDB: Doing recovery: scanned up to log sequence number 548267802672
      140621  6:49:35 [Note] InnoDB: Starting an apply batch of log records to the database...
      InnoDB: Progress in percent: 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
      InnoDB: Apply batch completed
      140621  6:49:36 [Note] InnoDB: 128 rollback segment(s) are active.
      140621  6:49:36 [Note] InnoDB: Waiting for purge to start
      140621  6:49:36 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.17-65.0 started; log sequence number 548267802672
      140621  6:49:36 [Note] Plugin 'FEEDBACK' is disabled.
      140621  6:49:36 [Note] Server socket created on IP: '::'.
      140621  6:49:38 [ERROR] mysqld.exe: Table '.\mysql\event' is marked as crashed and should be repaired
      140621  6:49:38 [Warning] Checking table:   '.\mysql\event'
      140621  6:49:38 [ERROR] mysql.event: 1 client is using or hasn't closed the table properly
      140621  6:49:38 [Note] Event Scheduler: Loaded 7 events
      140621  6:49:38 [Note] Event Scheduler: scheduler thread started with id 1
      140621  6:49:38 [Note] C:\Program Files\MariaDB 10.0\bin\mysqld.exe: ready for connections.
      Version: '10.0.11-MariaDB-log'  socket: ''  port: 3306  mariadb.org binary distribution
      

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            elenst Elena Stepanova added a comment -

            Hi,

            System error 6 is "No such device or address". Can it be that some of your tables have a tablespace in the location that is not available?

            Would it be possible to enable general_log for one time, so that we know which exact query/table causes the trouble?
            To do that, run set global general_log_file=<location of your choice> }} and {{set global general_log=1. After the server crashes, please upload the general log and the error log to ftp.askmonty.org/private
            If the general log contains any private info, it can be obfuscated; in fact, only the last part of it is needed, around the time when the crash occurs.
            From the error log, please extract the entire session, from server startup and up to and including the crash and server restart.

            Thanks.

            Show
            elenst Elena Stepanova added a comment - Hi, System error 6 is "No such device or address". Can it be that some of your tables have a tablespace in the location that is not available? Would it be possible to enable general_log for one time, so that we know which exact query/table causes the trouble? To do that, run set global general_log_file=<location of your choice> }} and {{set global general_log=1 . After the server crashes, please upload the general log and the error log to ftp.askmonty.org/private If the general log contains any private info, it can be obfuscated; in fact, only the last part of it is needed, around the time when the crash occurs. From the error log, please extract the entire session, from server startup and up to and including the crash and server restart. Thanks.
            Hide
            gabest gabest added a comment - - edited

            No crash since last time, but I am enabling the general log just in case it happens again. Had 3 crashes in the first month, then nothing for two weeks.

            The tables were separated from one large ibdata file into individual table files during the upgrade. I did nothing special to set their location.

            Show
            gabest gabest added a comment - - edited No crash since last time, but I am enabling the general log just in case it happens again. Had 3 crashes in the first month, then nothing for two weeks. The tables were separated from one large ibdata file into individual table files during the upgrade. I did nothing special to set their location.
            Hide
            elenst Elena Stepanova added a comment -

            Closing for now as "Cannot reproduce".
            If the problem happens again, please add a comment and the issue will be re-opened.

            Show
            elenst Elena Stepanova added a comment - Closing for now as "Cannot reproduce". If the problem happens again, please add a comment and the issue will be re-opened.

              People

              • Assignee:
                Unassigned
                Reporter:
                gabest gabest
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: