Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-3841

LevelDB storage engine

        Details

        • Type: Task
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Fix Version/s: None
        • Component/s: None
        • Labels:
          None

          Description

          Implement a LevelDB storage engine. Basic feature list:

          • single-statement transactions
          • secondary indexes
          • HANDLER implementation with extensions to support atomic multi-put (kind of like multi-statement transactions)
          • binlog XA on the master to be crash safe
          • crash-proof slave replication state
          • (almost) non blocking schema change
          • full test coverage via mysql-test-run
          • hot backup
          • possible options to have LevelDB instance per mysqld, per schema or per table

          The spec is being worked on here: https://kb.askmonty.org/en/leveldb-storage-engine

            Gliffy Diagrams

              Attachments

                Issue Links

                includes

                Task - A task that needs to be done. MDEV-4110 LevelDB: Support AUTO_INCREMENT

                • Major - Major loss of function.
                • Closed

                Task - A task that needs to be done. MDEV-4122 LevelDB: store key/index numbers, not names.

                • Major - Major loss of function.
                • Closed

                Task - A task that needs to be done. MDEV-4201 LevelDB Storage Engine MS2

                • Major - Major loss of function.
                • Closed
                1.
                		 Server crashes on creating a table with timestamp column Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 15 minutes
                 
                2.
                		 Assertion `kdef' fails in Primary_key_comparator::Compare on inserting data into a table previously altered and converted into LevelDB Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 10 minutes
                 
                3.
                		 Assertion `slice->size() == table->s->reclength' fails in ha_leveldb::convert_record_from_storage_format(le veldb::Slice*) on accessing a table after ALTER Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 30 minutes
                 
                4.
                		 Server crashes on running DISCARD TABLESPACE on a LevelDB table Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 15 minutes
                 
                5.
                		 Assertion `tablename[0] == '.' && tablename[1] == '/'' fails in ha_leveldb::delete_table(const char*) on creating a TEMPORARY LevelDB table Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 20 minutes
                 
                6.
                		 SELECT with ORDER BY causes "ERROR 1030 (HY000): Got error 122 from storage engine" Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 15 minutes
                 
                7.
                		 JOIN or WHERE conditions involving keys on LevelDB tables don't work Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 2 hours, 30 minutes
                 
                8.
                		 Assertion `!pk_descr' fails in ha_leveldb::open on adding partitions to a partitioned LevelDB table Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 10 minutes
                 
                9.
                		 [recreate] Server hangs or assertion `slice->size() == table->s->reclength' fails on a table created instead of a dropped one Technical task Closed Sergei Petrunia  
                 
                10.
                		 [recreate?] Assorted server crashes on a scenario involving creating and populating a LevelDB table Technical task Closed Sergei Petrunia  
                 
                11.
                		 [recreate] Server hangs on creating sort index in a re-created LevelDB table Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 15 minutes
                 
                12.
                		 UPDATE produces a wrong result while modifying a PK on a LevelDB table Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 1 hour, 30 minutes
                 
                13.
                		 Queries where execution plans on LevelDB don't use PK while on MyISAM they do Technical task Closed Sergei Petrunia  
                 
                14.
                		 A set of assorted crashes on inserting a row into a LevelDB table (does not involve drop table) Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 4 hours
                 
                15.
                		 Session value of leveldb_lock_wait_timeout is ignored Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 1 hour
                 
                16.
                		 LevelDB: SELECT produces different results inside a transaction (read is not repeatable) Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 20 minutes
                 
                17.
                		 LevelDB: INSERT .. ON DUPLICATE KEY UPDATE does not work, produces ER_DUP_KEY Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 4 hours
                 
                18.
                		 LevelDB: REPLACE doesn't work, produces ER_DUP_KEY Technical task Closed Sergei Petrunia  
                 
                19.
                		 LevelDB: SELECT ... FOR UPDATE does not work for gaps Technical task Closed Sergei Petrunia  
                 
                20.
                		 LevelDB: SELECT .. FOR UPDATE locks a row outside of transaction and prevents UPDATE from the same connection Technical task Closed Sergei Petrunia  
                 
                21.
                		 Server crashes in Primary_key_comparator::get_hashnr on INSERT into a LevelDB table with varbinary PK Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 15 minutes
                 
                22.
                		 LevelDB (Feature request): Produce ER_DUP_ENTRY instead of ER_DUP_KEY Technical task Closed Sergei Petrunia  
                 
                23.
                		 LevelDB: UPDATE or DELETE with ORDER BY locks itself and ends with ER_LOCK_WAIT_TIMEOUT Technical task Closed Sergei Petrunia  
                 
                24.
                		 LevelDB: Multi-table DELETE locks itself and ends with ER_LOCK_WAIT_TIMEOUT Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 15 minutes
                 
                25.
                		 LevelDB: Assertion `0' fails in Protocol::end_statement() on multi-table INSERT IGNORE Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 3 hours
                 
                26.
                		 LevelDB (Feature request): Add --leveldb option to enable/disable the engine Technical task Closed Sergei Petrunia  
                 
                27.
                		 LevelDB: DELETE hangs in state System lock on concurrent DELETE / INSERT flow Technical task Closed Sergei Petrunia  
                 
                28.
                		 LevelDB: Reading by PK prefix does not work (fails with "Table storage engine doesn't have this option") Technical task Closed Sergei Petrunia  
                 
                29.
                		 LevelDB: UPDATE/DELETE by a multi-part PK does not work (fails with "Table storage engine doesn't have this option") Technical task Closed Sergei Petrunia  
                 
                30.
                		 LevelDB: query waiting for a lock cannot be killed until query timeout exceeded Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 3 hours
                 
                31.
                		 LevelDB: Assertion `! trx->batch' fails in Ldb_transaction* get_or_create_trx(THD*) on SELECT after rolling back a transaction which didn't make changes Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 2 hours
                 
                32.
                		 LevelDB: Changes from an interrupted query are still applied Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 30 minutes
                 
                33.
                		 LevelDB works incorrectly with query cache Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 1 hour
                 
                34.
                		 LevelDB: ER_KEY_NOT_FOUND (Can't find record) Technical task Closed Sergei Petrunia  
                 
                35.
                		 LevelDB: Assertion `0' fails in ha_leveldb::index_read_map on range select with ORDER BY .. DESC Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 7 hours
                 
                36.
                		 LevelDB: Wrong result (duplicate row) on select with range access by primary key Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 30 minutes
                 
                37.
                		 Wrong result (missing rows) on select with secondary keys and index_merge Technical task Closed Sergei Petrunia  
                 
                38.
                		 LevelDB throws error 122 on an attempt to create a table with unique index Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 30 minutes
                 
                39.
                		 LevelDB: Wrong result on IN subquery with index lookup Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 30 minutes
                 
                40.
                		 LevelDB: Wrong result on range condition and aggregate function with two keys on a field Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 30 minutes
                 
                41.
                		 LevelDB does not allow a query with multi-part pk and index and ORDER BY .. DEC Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 1 hour, 30 minutes
                 
                42.
                		 LevelDB: Extensive memory usage on relatively small tables, valgrind warnings "blocks are definitely lost" Technical task Closed Sergei Petrunia  
                 
                43.
                		 LevelDB: Wrong result (duplicate rows) on range access with secondary key and ORDER BY DESC Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 45 minutes
                 
                44.
                		 LevelDB: Assertion `in_table(pa, a_len)' fails in LDBSE_KEYDEF::cmp_full_keys with a multi-part key and ORDER BY .. DESC Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 45 minutes
                 
                45.
                		 LevelDB: IN subquery by secondary key with NULL among values returns true instead of NULL Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 20 minutes
                 
                46.
                		 LevelDB: Wrong result on SELECT and ER_KEY_NOT_FOUND on DELETE with search by NULL-able secondary key after updating PK Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 1 hour, 45 minutes
                 
                47.
                		 LevelDB: indexes on text/blob fields are not allowed Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 15 minutes
                 
                48.
                		 LevelDB: Wrong results with index and range access after INSERT IGNORE or REPLACE Technical task Closed Sergei Petrunia  
                 
                49.
                		 LevelDB: Autoincrement is not increased on inserting an explicit value Technical task Closed Sergei Petrunia

                100%
                Original Estimate - Not Specified Original Estimate - Not Specified
                Time Spent - 3 hours, 30 minutes
                 
                50.
                		 LevelDB (Feature request): Improve output of SHOW TABLE STATUS Technical task Closed Sergei Petrunia  
                 
                51.
                		 LevelDB: Assertion `pb_end - pb >= LDBSE_KEYDEF::INDEX_NUMBER_SIZE' fails in Primary_key_comparator::Compare Technical task Closed Sergei Petrunia  
                 

                  Activity

                  Ascending order - Click to sort in descending order
                  10 older comments
                  Hide
                  Permalink
                  psergey Sergei Petrunia added a comment -

                  Got UPDATE and DELETE commands to work for simple examples. (They don't work for complex examples because right now all changes are applied immediately)

                  Show
                  psergey Sergei Petrunia added a comment - Got UPDATE and DELETE commands to work for simple examples. (They don't work for complex examples because right now all changes are applied immediately)
                  Hide
                  Permalink
                  zshao@fb.com Zheng Shao added a comment -

                  Hi Sergei, how's going so far?

                  Show
                  zshao@fb.com Zheng Shao added a comment - Hi Sergei, how's going so far?
                  Hide
                  Permalink
                  psergey Sergei Petrunia added a comment -

                  I'm still working on implementing row-locking system and hooking it to ha_leveldb. (More details in the email)

                  Show
                  psergey Sergei Petrunia added a comment - I'm still working on implementing row-locking system and hooking it to ha_leveldb. (More details in the email)
                  Hide
                  Permalink
                  arissety3 Aris Setyawan added a comment -

                  Hi Sergei,

                  In which version of MariaDB, LevelDB storage engine will be released?
                  What is the current status?

                  Show
                  arissety3 Aris Setyawan added a comment - Hi Sergei, In which version of MariaDB, LevelDB storage engine will be released? What is the current status?
                  Hide
                  Permalink
                  psergey Sergei Petrunia added a comment -

                  Hi Aris,

                  Current status is that a tree (based on mysql-5.6) with the engine is available. The engine has a number of known limitations that make it difficult for it to compete against a more feature-complete engine like InnoDB. Right now, I cannot say which version of MariaDB the engine will be pushed to.

                  Show
                  psergey Sergei Petrunia added a comment - Hi Aris, Current status is that a tree (based on mysql-5.6) with the engine is available. The engine has a number of known limitations that make it difficult for it to compete against a more feature-complete engine like InnoDB. Right now, I cannot say which version of MariaDB the engine will be pushed to.

                    People

                    • Assignee:
                      psergey Sergei Petrunia
                      Reporter:
                      psergey Sergei Petrunia
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      7 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved:

                        Time Tracking

                        Estimated:
                        Original Estimate - Not Specified
                        Not Specified
                        Remaining:
                        Remaining Estimate - 0 minutes
                        0m
                        Logged:
                        Time Spent - 6 weeks, 7 hours, 51 minutes
                        6w 7h 51m