Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-2818

LP:451080 - Uninitialised memory write in XTDatabaseLog::xlog_append

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Minor
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: None
        • Component/s: None
        • Labels:

          Description

          valgrind catch uninitialised memory write in pbxt.alias (for example, the error mentioned in many other cases), stack is following:

          ==9024== Thread 4:
          ==9024== Syscall param pwrite64(buf) points to uninitialised byte(s)
          ==9024== at 0x504F3A8: (within /lib64/libpthread-2.9.so)
          ==9024== by 0xA0E46C: xt_pwrite_file(XTOpenFile*, long, unsigned long, void*, XTIOStats*, XTThread*) (filesys_xt.cc:848)
          ==9024== by 0x9F140C: XTDatabaseLog::xlog_append(XTThread*, unsigned long, unsigned char*, unsigned long, unsigned char*, int, unsigned int*, long*) (xactlog_xt.cc:1111)
          ==9024== by 0x9F21D4: xt_xlog_log_data(XTThread*, unsigned long, XTXactLogBuffer*, int) (xactlog_xt.cc:1487)
          ==9024== by 0x9EA751: xt_xn_log_tab_id(XTThread*, unsigned int) (xaction_xt.cc:1471)
          ==9024== by 0x9DBF12: xt_create_table(XTThread*, XTPathStr*, XTDictionary*) (table_xt.cc:1502)
          ==9024== by 0x9AB32A: ha_pbxt::create(char const*, st_table*, st_ha_create_information*) (ha_pbxt.cc:5086)
          ==9024== by 0x7A4B26: handler::ha_create(char const*, st_table*, st_ha_create_information*) (handler.cc:3376)
          ==9024== by 0x7A7C19: ha_create_table(THD*, char const*, char const*, char const*, st_ha_create_information*, bool) (handler.cc:3587)
          ==9024== by 0x75875B: rea_create_table(THD*, char const*, char const*, char const*, st_ha_create_information*, List<Create_field>&, unsigned int, st_key*, handler*) (unireg.cc:416)
          ==9024== by 0x7C61BE: mysql_create_table_no_lock(THD*, char const*, char const*, st_ha_create_information*, Alter_info*, bool, unsigned int) (sql_table.cc:3853)
          ==9024== by 0x7C658F: mysql_create_table(THD*, char const*, char const*, st_ha_create_information*, Alter_info*, bool, unsigned int) (sql_table.cc:3960)
          ==9024== by 0x67C4AA: mysql_execute_command(THD*) (sql_parse.cc:2732)
          ==9024== by 0x683ECE: mysql_parse(THD*, char const*, unsigned int, char const**) (sql_parse.cc:5979)
          ==9024== by 0x684CD8: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1223)
          ==9024== by 0x68602C: do_command(THD*) (sql_parse.cc:862)
          ==9024== Address 0xf096292 is 50 bytes inside a block of size 1,049,088 alloc'd
          ==9024== at 0x4C24CFE: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
          ==9024== by 0x9B9A51: xt_malloc(XTThread*, unsigned long) (memory_xt.cc:101)
          ==9024== by 0x9F4055: XTDatabaseLog::xlog_setup(XTThread*, XTDatabase*, long, unsigned long, int) (xactlog_xt.cc:639)
          ==9024== by 0x9EB76C: xt_xn_init_db(XTThread*, XTDatabase*) (xaction_xt.cc:1103)
          ==9024== by 0x9FD727: xt_get_database(XTThread*, char*, int) (database_xt.cc:469)
          ==9024== by 0x9FD96A: xt_open_database(XTThread*, char*, int) (database_xt.cc:625)
          ==9024== by 0x9C362A: xn_xres_run_recovery_thread(XTThread*) (restart_xt.cc:3206)
          ==9024== by 0x9E32A0: thr_main (thread_xt.cc:1022)
          ==9024== by 0x5048016: start_thread (in /lib64/libpthread-2.9.so)
          ==9024== by 0x602248C: clone (in /lib64/libc-2.9.so)
          ==9024==
          ==9024== Syscall param pwrite64(buf) points to uninitialised byte(s)
          ==9024== at 0x504F3A8: (within /lib64/libpthread-2.9.so)
          ==9024== by 0xA0E46C: xt_pwrite_file(XTOpenFile*, long, unsigned long, void*, XTIOStats*, XTThread*) (filesys_xt.cc:848)
          ==9024== by 0x9F140C: XTDatabaseLog::xlog_append(XTThread*, unsigned long, unsigned char*, unsigned long, unsigned char*, int, unsigned int*, long*) (xactlog_xt.cc:1111)
          ==9024== by 0x9F223E: XTDatabaseLog::xlog_flush(XTThread*) (xactlog_xt.cc:729)
          ==9024== by 0x9D5002: xt_sync_flush_table(XTThread*, XTOpenTable*) (table_xt.cc:2170)
          ==9024== by 0x9FC4C7: db_lock_table_pool(XTThread*, XTDatabase*, unsigned int, int, int) (database_xt.cc:830)
          ==9024== by 0x9FC8E1: xt_db_lock_table_pool_by_name(XTThread*, XTDatabase*, XTPathStr*, int, int, int, int, XTTable**) (database_xt.cc:901)
          ==9024== by 0x9D62F8: tab_lock_table(XTThread*, XTPathStr*, int, int, int, XTTable**) (table_xt.cc:1259)
          ==9024== by 0x9D7E73: xt_drop_table(XTThread*, XTPathStr*, int) (table_xt.cc:1627)
          ==9024== by 0x9AE8A7: ha_pbxt::delete_table(char const*) (ha_pbxt.cc:4759)
          ==9024== by 0x7A4B92: handler::ha_delete_table(char const*) (handler.cc:3346)
          ==9024== by 0x7AA132: ha_delete_table(THD*, handlerton*, char const*, char const*, char const*, bool) (handler.cc:1966)
          ==9024== by 0x7CA94A: mysql_rm_table_part2(THD*, TABLE_LIST*, bool, bool, bool, bool) (sql_table.cc:1976)
          ==9024== by 0x7CAE68: mysql_rm_table(THD*, TABLE_LIST*, char, char) (sql_table.cc:1749)
          ==9024== by 0x67E890: mysql_execute_command(THD*) (sql_parse.cc:3398)
          ==9024== by 0x683ECE: mysql_parse(THD*, char const*, unsigned int, char const**) (sql_parse.cc:5979)
          ==9024== Address 0xf096414 is 436 bytes inside a block of size 1,049,088 alloc'd
          ==9024== at 0x4C24CFE: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
          ==9024== by 0x9B9A51: xt_malloc(XTThread*, unsigned long) (memory_xt.cc:101)
          ==9024== by 0x9F4055: XTDatabaseLog::xlog_setup(XTThread*, XTDatabase*, long, unsigned long, int) (xactlog_xt.cc:639)
          ==9024== by 0x9EB76C: xt_xn_init_db(XTThread*, XTDatabase*) (xaction_xt.cc:1103)
          ==9024== by 0x9FD727: xt_get_database(XTThread*, char*, int) (database_xt.cc:469)
          ==9024== by 0x9FD96A: xt_open_database(XTThread*, char*, int) (database_xt.cc:625)
          ==9024== by 0x9C362A: xn_xres_run_recovery_thread(XTThread*) (restart_xt.cc:3206)
          ==9024== by 0x9E32A0: thr_main (thread_xt.cc:1022)
          ==9024== by 0x5048016: start_thread (in /lib64/libpthread-2.9.so)
          ==9024== by 0x602248C: clone (in /lib64/libc-2.9.so)

          Many other cases can be found in:
          http://askmonty.org/buildbot/builders/gentoo-amd64-sanja/builds/4/steps/test_1/logs/mysqld.1.err.1
          http://askmonty.org/buildbot/builders/gentoo-amd64-sanja/builds/4/steps/test_1/logs/mysqld.1.err.2
          http://askmonty.org/buildbot/builders/gentoo-amd64-sanja/builds/4/steps/test_1/logs/mysqld.1.err.3
          http://askmonty.org/buildbot/builders/gentoo-amd64-sanja/builds/4/steps/test_1/logs/mysqld.1.err.4

          Can be repeated if run pbxt test suite under valgrind (valgrind build (one of BUILD/compile*valgrind* ) and --valgrind parameter of mysql-test-run)

            Gliffy Diagrams

              Attachments

                Activity

                Ascending order - Click to sort in descending order
                Hide
                Permalink
                vladimirkolesnikov Vladimir Kolesnikov added a comment -

                Re: Uninitialised memory write in XTDatabaseLog::xlog_append
                Hi Sanja,

                I have seen this case before and my analysis showed that the uninitialized bytes are at the tail of 512-byte blocks. This is not a bug. This happens because we write data block-by-block and the last block is likely to be only partially filled with data. If you have evidence of any real probem with this please let me know, otherwise I will close the bug.

                BR,
                Vladimir

                Show
                vladimirkolesnikov Vladimir Kolesnikov added a comment - Re: Uninitialised memory write in XTDatabaseLog::xlog_append Hi Sanja, I have seen this case before and my analysis showed that the uninitialized bytes are at the tail of 512-byte blocks. This is not a bug. This happens because we write data block-by-block and the last block is likely to be only partially filled with data. If you have evidence of any real probem with this please let me know, otherwise I will close the bug. BR, Vladimir
                Hide
                Permalink
                mdcallag Mark Callaghan added a comment -

                Re: Uninitialised memory write in XTDatabaseLog::xlog_append
                Other code in MySQL avoids warnings like this by fully writing blocks when HAVE_purify is defined. Can the same be done here? I prefer that approach over adding more valgrind suppressions.

                Show
                mdcallag Mark Callaghan added a comment - Re: Uninitialised memory write in XTDatabaseLog::xlog_append Other code in MySQL avoids warnings like this by fully writing blocks when HAVE_purify is defined. Can the same be done here? I prefer that approach over adding more valgrind suppressions.
                Hide
                Permalink
                vladimirkolesnikov Vladimir Kolesnikov added a comment -

                Re: Uninitialised memory write in XTDatabaseLog::xlog_append
                Mark,

                I think this is possible... Thanks for pointing the right macro to use...

                Show
                vladimirkolesnikov Vladimir Kolesnikov added a comment - Re: Uninitialised memory write in XTDatabaseLog::xlog_append Mark, I think this is possible... Thanks for pointing the right macro to use...
                Hide
                Permalink
                monty Michael Widenius added a comment -

                [Bug 451080] Re: Uninitialised memory write in XTDatabaseLog::xlog_append

                Hi!

                >>>>> "Vladimir" == Vladimir Kolesnikov <vladimir@primebase.org> writes:

                Vladimir> Hi Sanja,
                Vladimir> I have seen this case before and my analysis showed that the
                Vladimir> uninitialized bytes are at the tail of 512-byte blocks. This is not a
                Vladimir> bug. This happens because we write data block-by-block and the last
                Vladimir> block is likely to be only partially filled with data. If you have
                Vladimir> evidence of any real probem with this please let me know, otherwise I
                Vladimir> will close the bug.

                We would prefer to not have this happen when compiled with HAVE_valgrind.
                For example, if we ignore these, we will also miss bugs when you
                compile uninitialized bytes into the middle of a block.

                For example, in MyISAM we handle it when writing a block:

                #ifdef HAVE_valgrind

                { length=mi_getint(buff); bzero((uchar*) buff+length,keyinfo->block_length-length); length=keyinfo->block_length; }

                #endif

                In other words, we explicitly fill the not used parts with 0 to ensure
                that valgrind doesn't complain.

                HAVE_valgrind is only defined in debug builds, so there is no speed
                penally for production systems.

                Regards,
                Monty

                Show
                monty Michael Widenius added a comment - [Bug 451080] Re: Uninitialised memory write in XTDatabaseLog::xlog_append Hi! >>>>> "Vladimir" == Vladimir Kolesnikov <vladimir@primebase.org> writes: Vladimir> Hi Sanja, Vladimir> I have seen this case before and my analysis showed that the Vladimir> uninitialized bytes are at the tail of 512-byte blocks. This is not a Vladimir> bug. This happens because we write data block-by-block and the last Vladimir> block is likely to be only partially filled with data. If you have Vladimir> evidence of any real probem with this please let me know, otherwise I Vladimir> will close the bug. We would prefer to not have this happen when compiled with HAVE_valgrind. For example, if we ignore these, we will also miss bugs when you compile uninitialized bytes into the middle of a block. For example, in MyISAM we handle it when writing a block: #ifdef HAVE_valgrind { length=mi_getint(buff); bzero((uchar*) buff+length,keyinfo->block_length-length); length=keyinfo->block_length; } #endif In other words, we explicitly fill the not used parts with 0 to ensure that valgrind doesn't complain. HAVE_valgrind is only defined in debug builds, so there is no speed penally for production systems. Regards, Monty
                Hide
                Permalink
                vladimirkolesnikov Vladimir Kolesnikov added a comment -

                Re: Uninitialised memory write in XTDatabaseLog::xlog_append
                Hi Monty,

                thanks for the input. I have added the code, it solves the problem. Will push it tomorrow.

                Show
                vladimirkolesnikov Vladimir Kolesnikov added a comment - Re: Uninitialised memory write in XTDatabaseLog::xlog_append Hi Monty, thanks for the input. I have added the code, it solves the problem. Will push it tomorrow.
                Hide
                Permalink
                hakanküçükyılmaz Hakan Küçükyılmaz added a comment -

                Re: Uninitialised memory write in XTDatabaseLog::xlog_append
                Vladimir, I guess this fix is already released. Can you please point me to the bzr patch, so I can check it in our sources?

                Thanks

                Show
                hakanküçükyılmaz Hakan Küçükyılmaz added a comment - Re: Uninitialised memory write in XTDatabaseLog::xlog_append Vladimir, I guess this fix is already released. Can you please point me to the bzr patch, so I can check it in our sources? Thanks
                Hide
                Permalink
                vladimirkolesnikov Vladimir Kolesnikov added a comment -

                Re: Uninitialised memory write in XTDatabaseLog::xlog_append
                hi Hakan!

                the fix is here:

                lp:~vkolesnikov/pbxt/pbxt-bug-451080

                revisions 718 and 719.

                BR

                Show
                vladimirkolesnikov Vladimir Kolesnikov added a comment - Re: Uninitialised memory write in XTDatabaseLog::xlog_append hi Hakan! the fix is here: lp:~vkolesnikov/pbxt/pbxt-bug-451080 revisions 718 and 719. BR
                Hide
                Permalink
                ratzpo Rasmus Johansson added a comment -

                Launchpad bug id: 451080

                Show
                ratzpo Rasmus Johansson added a comment - Launchpad bug id: 451080

                  People

                  • Assignee:
                    Unassigned
                    Reporter:
                    sanja Oleksandr Byelkin
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    0 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: