Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-4903

slow query using IN () on primary key with 5 columns

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 10.0.3
    • Fix Version/s: 10.1
    • Component/s: None
    • Labels:

      Description

      TESTED WITH 10.0.1 AND 10.0.3

      running a query using IN () take a table scan
      while using OR = OR = OR = .... use index

      the problem in this case that's the "nf_serie" column don't have a value of "ecf1", in this case this query will return "always false"
      the table is optimized via OPTIMIZE TABLE

      i will append table definitions and query next:

      explain extended query:

      select `19_org`.`nf_arquivo_nfe`.`emitente_tipo` AS `emitente_tipo`,`19_org`.`nf_arquivo_nfe`.`emitente_id` AS `emitente_id`,`19_org`.`nf_arquivo_nfe`.`emitente_propriedade` AS `emitente_propriedade`,`19_org`.`nf_arquivo_nfe`.`nf` AS `nf`,`19_org`.`nf_arquivo_nfe`.`nf_serie` AS `nf_serie`,`19_org`.`nf_arquivo_nfe`.`arquivo` AS `arquivo`,`19_org`.`nf_arquivo_nfe`.`nome_arquivo` AS `nome_arquivo` 
      from `19_org`.`nf_arquivo_nfe` 
      where 
      ((`19_org`.`nf_arquivo_nfe`.`emitente_tipo`,`19_org`.`nf_arquivo_nfe`.`emitente_id`,`19_org`.`nf_arquivo_nfe`.`emitente_propriedade`,`19_org`.`nf_arquivo_nfe`.`nf_serie`,`19_org`.`nf_arquivo_nfe`.`nf`) in 
      (<cache>(('j','1','0','ecf1','-2')),<cache>(('j','1','0','ecf1','-1'))))
      
      explain
      select emitente_tipo, emitente_id, emitente_propriedade, nf, nf_serie,arquivo,nome_arquivo
       from nf_arquivo_nfe
       WHERE 
       (emitente_tipo, emitente_id, emitente_propriedade, nf_serie, nf) IN 
       (('j','1','0','ecf1',"-2"),('j','1','0','ecf1',"-1")) 
      
      id select_type table type possible_keys key key_len ref rows Extra
      1 SIMPLE nf_arquivo_nfe ALL         633786 Using where


      "optimized" version

      explain extended query:

      select `19_org`.`nf_arquivo_nfe`.`emitente_tipo` AS `emitente_tipo`,`19_org`.`nf_arquivo_nfe`.`emitente_id` AS `emitente_id`,`19_org`.`nf_arquivo_nfe`.`emitente_propriedade` AS `emitente_propriedade`,`19_org`.`nf_arquivo_nfe`.`nf` AS `nf`,`19_org`.`nf_arquivo_nfe`.`nf_serie` AS `nf_serie`,`19_org`.`nf_arquivo_nfe`.`arquivo` AS `arquivo`,`19_org`.`nf_arquivo_nfe`.`nome_arquivo` AS `nome_arquivo` 
      from `19_org`.`nf_arquivo_nfe` 
      where 
      (
      	(
      		(`19_org`.`nf_arquivo_nfe`.`nf_serie` = 'ecf1') and 
      		(`19_org`.`nf_arquivo_nfe`.`emitente_tipo` = 'j') and 
      		(`19_org`.`nf_arquivo_nfe`.`emitente_id` = '1') and 
      		(`19_org`.`nf_arquivo_nfe`.`emitente_propriedade` = '0') and 
      		(`19_org`.`nf_arquivo_nfe`.`nf` = '-2')
      	) or (
      		(`19_org`.`nf_arquivo_nfe`.`nf_serie` = 'ecf1') and 
      		(`19_org`.`nf_arquivo_nfe`.`emitente_tipo` = 'j') and 
      		(`19_org`.`nf_arquivo_nfe`.`emitente_id` = '1') and 
      		(`19_org`.`nf_arquivo_nfe`.`emitente_propriedade` = '0') and 
      		(`19_org`.`nf_arquivo_nfe`.`nf` = '-1')
      	)
      )
      
      explain
      select emitente_tipo, emitente_id, emitente_propriedade, nf, nf_serie,arquivo,nome_arquivo
       from nf_arquivo_nfe
       WHERE 
       (emitente_tipo, emitente_id, emitente_propriedade, nf_serie, nf)=
       ('j','1','0','ecf1',"-2") OR 
       (emitente_tipo, emitente_id, emitente_propriedade, nf_serie, nf)=
       ('j','1','0','ecf1',"-1")
      
      id select_type table type possible_keys key key_len ref rows Extra
      1 SIMPLE nf_arquivo_nfe ref PRIMARY PRIMARY 14 const,const,const,const 1 Using index condition; Using where

      create table:

      CREATE TABLE `nf_arquivo_nfe` (
        `emitente_tipo` enum('f','j') NOT NULL DEFAULT 'f',
        `emitente_id` int(10) unsigned NOT NULL DEFAULT '0',
        `emitente_propriedade` int(10) unsigned NOT NULL DEFAULT '0',
        `nf_serie` char(5) NOT NULL DEFAULT '',
        `nf` bigint(20) unsigned NOT NULL DEFAULT '0',
        `id_arquivo` int(10) unsigned NOT NULL DEFAULT '0',
        `tipo_arquivo` varchar(50) NOT NULL,
        `mime_arquivo` varchar(255) NOT NULL DEFAULT '',
        `nome_arquivo` varchar(255) NOT NULL,
        `arquivo` longblob NOT NULL,
        `data_arquivo` double unsigned NOT NULL DEFAULT '0',
        `usuario` varchar(50) NOT NULL DEFAULT '',
        `md5_arquivo` varchar(50) NOT NULL,
        `tipo_ambiente` tinyint(3) unsigned NOT NULL DEFAULT '0',
        `tamanho_arquivo` bigint(20) unsigned NOT NULL DEFAULT '0',
        PRIMARY KEY (`emitente_tipo`,`emitente_id`,`emitente_propriedade`,`nf_serie`,`nf`,`id_arquivo`)
      ) ENGINE=Aria DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
      

      table informations:

      select emitente_tipo,emitente_id,emitente_propriedade,nf_serie,count(*)
      from nf_arquivo_nfe
      group by emitente_tipo,emitente_id,emitente_propriedade,nf_serie
      
      emitente_tipo emitente_id emitente_propriedade nf_serie count(*)
        0 0   715
      j 1 0 1a 21
      j 1 0 1nf 57831
      j 1 0 1nf2 244437
      j 1 0 1nfe 144572
      j 1 0 2nfe 26884
      j 1 0 3nfe 158280
      j 1 0 4nfe 1044
      j 111091 0 1 1
      j 111130 0 1 1

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            rspadim roberto spadim added a comment -

            hi guys, any idea how to optimize it? one solution could be a query rewrite, anyother?

            Show
            rspadim roberto spadim added a comment - hi guys, any idea how to optimize it? one solution could be a query rewrite, anyother?
            Hide
            psergey Sergei Petrunia added a comment -

            This could be optimized. Some more details are here:
            https://lists.launchpad.net/maria-developers/msg07413.html

            will discuss in optimizer team meeting.

            Show
            psergey Sergei Petrunia added a comment - This could be optimized. Some more details are here: https://lists.launchpad.net/maria-developers/msg07413.html will discuss in optimizer team meeting.
            Hide
            psergey Sergei Petrunia added a comment -

            .. discussed. Decided that we can backport this feature from MySQL 5.7 to MariaDB 10.1

            Show
            psergey Sergei Petrunia added a comment - .. discussed. Decided that we can backport this feature from MySQL 5.7 to MariaDB 10.1
            Hide
            rspadim roberto spadim added a comment - - edited

            nice
            there's another feature request, about type specific checks
            for example enum(), when "where enum_field='value out of enum declaration' ", this could return a impossible where even without index (MDEV-4419) , samething for "where unsigned_number_value<0"
            any idea if we could execute a 'query rewrite' inside optimizer, and with this rewrite we got the "impossible where" optimization?

            Show
            rspadim roberto spadim added a comment - - edited nice there's another feature request, about type specific checks for example enum(), when "where enum_field='value out of enum declaration' ", this could return a impossible where even without index ( MDEV-4419 ) , samething for "where unsigned_number_value<0" any idea if we could execute a 'query rewrite' inside optimizer, and with this rewrite we got the "impossible where" optimization?

              People

              • Assignee:
                psergey Sergei Petrunia
                Reporter:
                rspadim roberto spadim
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: