Details
Minor
Description
I built the server with openSSL (WITH_SSL=system instead of bundled) and started getting valgrind warnings on server shutdown:
perl ./mtr alias --valgrind-mysqld
==29019== 24 bytes in 1 blocks are still reachable in loss record 1 of 600 ==29019== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==29019== by 0x5DCE7F3: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0) ==29019== by 0x5E4928A: lh_insert (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0) ==29019== by 0x5DD149B: OBJ_NAME_add (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0) ==29019== by 0x5E53714: EVP_add_cipher (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0) ==29019== by 0x5B54690: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0) ==29019== by 0xEBE487: check_ssl_init (viosslfactories.c:155) ==29019== by 0xEBE5AA: new_VioSSLFd (viosslfactories.c:189) ==29019== by 0xEBEB66: new_VioSSLAcceptorFd (viosslfactories.c:336) ==29019== by 0x5841CF: init_ssl() (mysqld.cc:4412) ==29019== by 0x585888: mysqld_main(int, char**) (mysqld.cc:5224) ==29019== by 0x57C4C3: main (main.cc:25) ==29019== ==29019== 24 bytes in 1 blocks are still reachable in loss record 2 of 600 ==29019== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==29019== by 0x5DCE7F3: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0) ==29019== by 0x5E4928A: lh_insert (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0) ==29019== by 0x5DD149B: OBJ_NAME_add (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0) ==29019== by 0x5B54690: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0) ==29019== by 0xEBE487: check_ssl_init (viosslfactories.c:155) ==29019== by 0xEBE5AA: new_VioSSLFd (viosslfactories.c:189) ==29019== by 0xEBEB66: new_VioSSLAcceptorFd (viosslfactories.c:336) ==29019== by 0x5841CF: init_ssl() (mysqld.cc:4412) ==29019== by 0x585888: mysqld_main(int, char**) (mysqld.cc:5224) ==29019== by 0x57C4C3: main (main.cc:25) ==29019== ...
I'm not sure whether they're essential or not, and if they are, whether it's our fault, maybe it's a bad valgrind version or a 3rd-party library bug.
ldd ../sql/mysqld linux-vdso.so.1 => (0x00007fff7578d000) libodbc.so.1 => /usr/lib/x86_64-linux-gnu/libodbc.so.1 (0x00007f1731474000) libaio.so.1 => /lib/x86_64-linux-gnu/libaio.so.1 (0x00007f1731272000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f173105a000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1730e52000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1730c35000) libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f17309fb000) libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f173079d000) libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f17303d5000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f17301d0000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f172fed0000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f172fbd4000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f172f9bd000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f172f5fe000) libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f172f3f4000) /lib64/ld-linux-x86-64.so.2 (0x00007f17316ed000)
valgrind --version valgrind-3.7.0
Gliffy Diagrams
Attachments
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
The problem is repeatable with:
- 5.5 bzr tree built with -DWITH_SSL=system
- 5.3 bzr tree built with ./BUILD/compile-pentium-debug-openssl
- 5.1 bzr tree built with ./BUILD/compile-pentium-debug-openssl
Still reproducible with openssl-1.0.1e-4.fc17.x86_64.
Fixed in 5.1, 5.2, 5.3, 5.5, 10.0-base.
The change in vio_end() made here http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/2502.565.54 causes mysqltest_embedded to hang on exit. So in 10.0.5 tarball testing of embedded server doesn't work at all when it's compiled with openssl.
I think the problem is that mysql_server_end() in case of compilation with embedded server calls both end_embedded_server() (which calls clean_up() which calls vio_end() ) and vio_end(). But apparently ERR_remove_state(0) can be called only once. On the second call it hangs inside openssl trying to lock the mutex.
Hi Alexander,
Re-opening to draw your attention to the above. I think for the sake of changelog history and such, it will be better to open another bug report and close this one again, but I'll leave it to you to decide.
Pavel, can you please post how-to-repeat instructions?
I can't repeat the hang on my FC18 64bit box.
I tried both debug and release builds:
cmake . -DCMAKE_INSTALL_PREFIX=/opt/mariadb-10.0 -DWITH_SSL=system -DWITH_EMBEDDED_SERVER=yes
and
cmake . -DCMAKE_INSTALL_PREFIX=/opt/mariadb-10.0 -DWITH_SSL=system -DWITH_EMBEDDED_SERVER=yes -DCMAKE_BUILD_TYPE=Debug
It does not hang.
Note, I can confirm that there is a problem in the code with vio_end() executed twice.
But it would be nice to repeat the hang anyway.
Thanks.
Hm... Sorry, it turned out that I was linking with a locally built openssl library that included some hacks which were actually causing the hang. I can't reproduce with Ubuntu's standard openssl library either.
Thanks for checking! I opened a separate issue for this:
https://mariadb.atlassian.net/browse/MDEV-5311
The double call for vio_end() still needs to be fixed.
The same problem is observed with: