Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-567

Wrong result from a query with correlated subquery if ICP is allowed

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The following test case produces a wrong result set in MariaDB 5.3:

      CREATE TABLE t1 (a int, b int, INDEX idx(a));
      INSERT INTO t1 VALUES (9,0), (7,1), (1,9), (7,3), (2,1);
      
      CREATE TABLE t2 (a int, b int, INDEX idx(a));
      INSERT INTO t2 VALUES (2,1), (6,4), (7,6), (9,4);
      
      CREATE TABLE t3 (a int, b int);
      INSERT INTO t3 VALUES (1,0), (1,1), (1,3);
      
      SELECT * FROM t3
        WHERE a = (SELECT COUNT(DISTINCT t2.b) FROM t1, t2
                      WHERE t1.a = t2.a AND t2.a BETWEEN 7 AND 9
                            AND t3.b = t1.b
                    GROUP BY t1.b);
      

      The returned result for the above query is:

      MariaDB [test]> SELECT * FROM t3
          ->   WHERE a = (SELECT COUNT(DISTINCT t2.b) FROM t1, t2
          ->                 WHERE t1.a = t2.a AND t2.a BETWEEN 7 AND 9
          ->                       AND t3.b = t1.b
          ->               GROUP BY t1.b);
      +------+------+
      | a    | b    |
      +------+------+
      |    1 |    0 |
      |    1 |    3 |
      +------+------+
      

      The EXPLAIN output for this query is:

       
      MariaDB [test]> EXPLAIN
          -> SELECT * FROM t3
          ->   WHERE a = (SELECT COUNT(DISTINCT t2.b) FROM t1, t2
          ->                 WHERE t1.a = t2.a AND t2.a BETWEEN 7 AND 9
          ->                       AND t3.b = t1.b
          ->               GROUP BY t1.b);
      +----+--------------------+-------+-------+---------------+------+---------+-----------+------+----------------------------------------------------+
      | id | select_type        | table | type  | possible_keys | key  | key_len | ref       | rows | Extra                                              |
      +----+--------------------+-------+-------+---------------+------+---------+-----------+------+----------------------------------------------------+
      |  1 | PRIMARY            | t3    | ALL   | NULL          | NULL | NULL    | NULL      |    3 | Using where                                        |
      |  2 | DEPENDENT SUBQUERY | t1    | range | idx           | idx  | 5       | NULL      |    2 | Using index condition; Using where; Using filesort |
      |  2 | DEPENDENT SUBQUERY | t2    | ref   | idx           | idx  | 5       | test.t1.a |    2 | Using index condition                              |
      +----+--------------------+-------+-------+---------------+------+---------+-----------+------+----------------------------------------------------+
      3 rows in set (0.00 sec)
      

      If ICP is turned off the result returned by the query is correct:

      MariaDB [test]> set optimizer_switch='index_condition_pushdown=off';
      Query OK, 0 rows affected (0.00 sec)
      
      MariaDB [test]> SELECT * FROM t3
          ->   WHERE a = (SELECT COUNT(DISTINCT t2.b) FROM t1, t2
          ->                 WHERE t1.a = t2.a AND t2.a BETWEEN 7 AND 9
          ->                       AND t3.b = t1.b
          ->               GROUP BY t1.b);
      +------+------+
      | a    | b    |
      +------+------+
      |    1 |    0 |
      |    1 |    1 |
      |    1 |    3 |
      +------+------+
      3 rows in set (0.00 sec)
      

      The above test case is a simplified version of the test case for Oracle's bug#12667154 that can be found in http://lists.mysql.com/commits/143149
      The fix itself (without any test case) was pulled into MariaDB 5.5.
      The validity of the fix should be re-checked.

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            pomyk Patryk Pomykalski added a comment -

            The fix introduced other bug: MDEV-536

            Show
            pomyk Patryk Pomykalski added a comment - The fix introduced other bug: MDEV-536
            Hide
            psergey Sergei Petrunia added a comment -

            Fix backported and pushed into 5.3

            Show
            psergey Sergei Petrunia added a comment - Fix backported and pushed into 5.3
            Hide
            psergey Sergei Petrunia added a comment -

            Sorry, somehow missed comment by Patryk.

            Note that lp:~maria-captains/maria/5.5-show-explain, which is based on 5.5.27 and has alternative fix developed for use with SHOW EXPLAIN, doesn't produce a wrong result.

            Show
            psergey Sergei Petrunia added a comment - Sorry, somehow missed comment by Patryk. Note that lp:~maria-captains/maria/5.5-show-explain, which is based on 5.5.27 and has alternative fix developed for use with SHOW EXPLAIN, doesn't produce a wrong result.
            Hide
            psergey Sergei Petrunia added a comment -

            have made a patch that takes out Olav's fix
            (revid: olav.sandstaa@oracle.com-20120516074923-vd0dhp183vqcp2ql )
            and instead adds my fix ( revid:
            psergey@askmonty.org-20120719115219-212cxmm6qvf0wlrb, branch: 5.5-show-explain,
            then 10.0)

            As a result of that, I've got a patch that can handle subqueries with pushed
            index conditions. Testcase for MDEV-567 passes with it. However, it doesn't
            work with MRR+ICP. I don't have a ready testcase for this, but looking at the
            code I see that it won't work. These lines in create_sort_index():

            <quote>
            if (table->file->inited == handler::INDEX)

            { // Save index #, save index condition join->pre_sort_index= table->file->active_index; join->pre_sort_idx_pushed_cond= table->file->pushed_idx_cond; // no need to save key_read? err= table->file->ha_index_end(); }

            </quote>

            will not function correctly when using DS-MRR. With DS-MRR,
            table->file->inited==handler::RND (DS-MRR uses secondary handler object for
            index scans and primary handler object for fetching fows).

            Show
            psergey Sergei Petrunia added a comment - have made a patch that takes out Olav's fix (revid: olav.sandstaa@oracle.com-20120516074923-vd0dhp183vqcp2ql ) and instead adds my fix ( revid: psergey@askmonty.org-20120719115219-212cxmm6qvf0wlrb, branch: 5.5-show-explain, then 10.0) As a result of that, I've got a patch that can handle subqueries with pushed index conditions. Testcase for MDEV-567 passes with it. However, it doesn't work with MRR+ICP. I don't have a ready testcase for this, but looking at the code I see that it won't work. These lines in create_sort_index(): <quote> if (table->file->inited == handler::INDEX) { // Save index #, save index condition join->pre_sort_index= table->file->active_index; join->pre_sort_idx_pushed_cond= table->file->pushed_idx_cond; // no need to save key_read? err= table->file->ha_index_end(); } </quote> will not function correctly when using DS-MRR. With DS-MRR, table->file->inited==handler::RND (DS-MRR uses secondary handler object for index scans and primary handler object for fetching fows).
            Hide
            psergey Sergei Petrunia added a comment -

            Finished, pushed the fix into 5.5

            Show
            psergey Sergei Petrunia added a comment - Finished, pushed the fix into 5.5
            Hide
            igor Igor Babaev added a comment -

            Sergey,

            Do you plan to back-port the fix into 5.3:? The bug was reported against 5.3.

            Show
            igor Igor Babaev added a comment - Sergey, Do you plan to back-port the fix into 5.3:? The bug was reported against 5.3.
            Hide
            elenst Elena Stepanova added a comment -

            The initial reported problem was fixed in 5.3.10 by the backport of the Oracle fix:

            revno: 3586
            revision-id: psergey@askmonty.org-20121010052122-1uua5ogs8pyoanbf
            parent: sanja@montyprogram.com-20121005092655-1xfcw36gsdyhawu5
            committer: Sergey Petrunya <psergey@askmonty.org>
            branch nick: 5.3
            timestamp: Wed 2012-10-10 09:21:22 +0400
            message:
              Backport of: olav.sandstaa@oracle.com-20120516074923-vd0dhp183vqcp2ql
              .. into MariaDB 5.3
              
              Fix for Bug#12667154 SAME QUERY EXEC AS WHERE SUBQ GIVES DIFFERENT
            ...
            

            So, the test case from the description does not fail anymore.
            The fix introduced the bug MDEV-614/MDEV-536. This other bug was only fixed in 5.5 and up, and is still reproducible on 5.3. If it's necessary to fix it in 5.3, MDEV-614 can be re-opened, but I suppose it makes no sense to keep this one open any longer, so I close it as fixed.

            Show
            elenst Elena Stepanova added a comment - The initial reported problem was fixed in 5.3.10 by the backport of the Oracle fix: revno: 3586 revision-id: psergey@askmonty.org-20121010052122-1uua5ogs8pyoanbf parent: sanja@montyprogram.com-20121005092655-1xfcw36gsdyhawu5 committer: Sergey Petrunya <psergey@askmonty.org> branch nick: 5.3 timestamp: Wed 2012-10-10 09:21:22 +0400 message: Backport of: olav.sandstaa@oracle.com-20120516074923-vd0dhp183vqcp2ql .. into MariaDB 5.3 Fix for Bug#12667154 SAME QUERY EXEC AS WHERE SUBQ GIVES DIFFERENT ... So, the test case from the description does not fail anymore. The fix introduced the bug MDEV-614 / MDEV-536 . This other bug was only fixed in 5.5 and up, and is still reproducible on 5.3. If it's necessary to fix it in 5.3, MDEV-614 can be re-opened, but I suppose it makes no sense to keep this one open any longer, so I close it as fixed.

              People

              • Assignee:
                psergey Sergei Petrunia
                Reporter:
                igor Igor Babaev
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 hour, 30 minutes
                  1h 30m