Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7650

COLUMN_CHECK() does not always detect data truncation

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 10.0.16
    • Fix Version/s: 10.1, 10.0
    • Component/s: Dynamic Columns
    • Labels:
    • Environment:
      Server version: 10.0.16-MariaDB Homebrew
      Darwin fsb 13.4.0 Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64 x86_64

      Description

      Bug is fully described in the following test script:

      select @@max_allowed_packet;
      # 1073741824
      
      select column_check(column_create('a', 1, 'b', repeat('1234567890', 6554)));
      # 1
      
      set @txt = (select repeat('1234567890', 6554));
      select column_check(column_create('a', 1, 'txt', @txt));
      # 1
      
      select column_check(column_create('a', 1, 'css', @txt));
      # 1
      
      create database if not exists dyncol_test;
      use dyncol_test;
      drop table if exists t;
      create table t (a int, b varchar(255), dcols blob);
      
      insert into t (a, b, dcols) values (1, 'two', column_create('txt', @txt));
      select column_check(dcols),
          substring(column_get(dcols, 'txt' as char), 1, 40),
          substring(column_json(dcols), 1, 40) from t;
      # 1   1234567890123456789012345678901234567890    {"txt":"12345678901234567890123456789012
      
      delete from t;
      insert into t (a, b, dcols) values (1, 'two', column_create('one', 1, 'txt', @txt));
      select column_check(dcols),
          substring(column_get(dcols, 'txt' as char), 1, 40),
          substring(column_json(dcols), 1, 40) from t;
      # 1   1234567890123456789012345678901234567890    {"one":1,"txt":"123456789012345678901234
      
      delete from t;
      insert into t (a, b, dcols) values (1, 'two', column_create('css', @txt));
      select column_check(dcols),
          substring(column_get(dcols, 'css' as char), 1, 40),
          substring(column_json(dcols), 1, 40) from t;
      # 1   1234567890123456789012345678901234567890    {"css":"12345678901234567890123456789012
      
      delete from t;
      insert into t (a, b, dcols) values (1, 'two', column_create('one', 1, 'css', @txt));
      select column_check(dcols),
          substring(column_get(dcols, 'css' as char), 1, 40),
          substring(column_json(dcols), 1, 40) from t;
      # ERROR 1919 (HY000) at line 31: Encountered illegal format of dynamic column string
      
      # Looks like the conditions for the error are:
      # 1. dynamic column blob size greater than 64kB
      # 2. dynamic column blob is saved to a table
      # 3. dynamic column blob contains more than one dynamic columns
      # 4. the dynamic column with the long value is named 'css' and not 'txt' !!!
      

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            elenst Elena Stepanova added a comment -

            Thanks for the report and the test case.

            Show
            elenst Elena Stepanova added a comment - Thanks for the report and the test case.
            Hide
            thefsb Tom added a comment - - edited

            Dynamic columns are packed into the blob in alphabetical order.

            create database if not exists dyncol_test;
            use dyncol_test;
            drop table if exists t;
            create table t (id int, dcols blob);
            
            set @txt = repeat('1234567890', 6551);
            insert into t (id, dcols) values 
                (1, column_create('aaa', 1, 'aab', @txt)),
                (2, column_create('aac', 1, 'aab', @txt)),
                (3, column_create('aaa', 1, 'aab', concat(@txt, '1234567890'))),
                (4, column_create('aac', 1, 'aab', concat(@txt, '1234567890')));
            
            select id, column_check(dcols) from t;
            >>>
            id	column_check(dcols)
            1	1
            2	1
            3	1
            4	0
            
            Show
            thefsb Tom added a comment - - edited Dynamic columns are packed into the blob in alphabetical order. create database if not exists dyncol_test; use dyncol_test; drop table if exists t; create table t (id int , dcols blob); set @txt = repeat('1234567890', 6551); insert into t (id, dcols) values (1, column_create('aaa', 1, 'aab', @txt)), (2, column_create('aac', 1, 'aab', @txt)), (3, column_create('aaa', 1, 'aab', concat(@txt, '1234567890'))), (4, column_create('aac', 1, 'aab', concat(@txt, '1234567890'))); select id, column_check(dcols) from t; >>> id column_check(dcols) 1 1 2 1 3 1 4 0
            Hide
            elenst Elena Stepanova added a comment -

            Right. Long ago there was a discussion about consequences of data truncation in dynamic columns (which happens here, all INSERTs produce the warning); but I can't find or remember what the outcome of that discussion was, so I leave it to Sanja to comment, he will remember for sure.

            Show
            elenst Elena Stepanova added a comment - Right. Long ago there was a discussion about consequences of data truncation in dynamic columns (which happens here, all INSERTs produce the warning); but I can't find or remember what the outcome of that discussion was, so I leave it to Sanja to comment, he will remember for sure.
            Hide
            serg Sergei Golubchik added a comment -

            A clearer test case:

            MariaDB [test]> select column_check(left(column_create('one', 1, 'txt', @txt), 65535));
            +-----------------------------------------------------------------+
            | column_check(left(column_create('one', 1, 'txt', @txt), 65535)) |
            +-----------------------------------------------------------------+
            |                                                               1 |
            +-----------------------------------------------------------------+
            1 row in set (0.00 sec)
            
            MariaDB [test]> select column_check(left(column_create('one', 1, 'css', @txt), 65535));
            +-----------------------------------------------------------------+
            | column_check(left(column_create('one', 1, 'css', @txt), 65535)) |
            +-----------------------------------------------------------------+
            |                                                               0 |
            +-----------------------------------------------------------------+
            1 row in set (0.00 sec)
            
            Show
            serg Sergei Golubchik added a comment - A clearer test case: MariaDB [test]> select column_check(left(column_create('one', 1, 'txt', @txt), 65535)); +-----------------------------------------------------------------+ | column_check(left(column_create('one', 1, 'txt', @txt), 65535)) | +-----------------------------------------------------------------+ | 1 | +-----------------------------------------------------------------+ 1 row in set (0.00 sec) MariaDB [test]> select column_check(left(column_create('one', 1, 'css', @txt), 65535)); +-----------------------------------------------------------------+ | column_check(left(column_create('one', 1, 'css', @txt), 65535)) | +-----------------------------------------------------------------+ | 0 | +-----------------------------------------------------------------+ 1 row in set (0.00 sec)
            Hide
            thefsb Tom added a comment -

            I did not know that blobs were restricted to such a small size.

            This was therefore my foolish error. Caught out again by the crazy selection of MySQL types.

            I do not know how to delete or close this issue.

            Show
            thefsb Tom added a comment - I did not know that blobs were restricted to such a small size. This was therefore my foolish error. Caught out again by the crazy selection of MySQL types. I do not know how to delete or close this issue.
            Hide
            serg Sergei Golubchik added a comment -

            While we can close it, I'd suggest to keep it as a COLUMN_CHECK bug. I'll change the subject to better reflect what this bug is about.

            Show
            serg Sergei Golubchik added a comment - While we can close it, I'd suggest to keep it as a COLUMN_CHECK bug. I'll change the subject to better reflect what this bug is about.

              People

              • Assignee:
                sanja Oleksandr Byelkin
                Reporter:
                thefsb Tom
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: