Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6778

Include number of concurrent queries while insert into query cache

    Details

    • Type: Task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Fix Version/s: 10.1
    • Component/s: Query Cache
    • Labels:
      None

      Description

      when insertint at query cache we can have competitiors with the same query
      the number of competitors could be used with query cache qc_plugin (mdev-4682)
      instead of *10 as a high period category, we could use number of competitors + 1

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

              Hide
              serg Sergei Golubchik added a comment -

              what do you mean?

              Show
              serg Sergei Golubchik added a comment - what do you mean?
              Hide
              rspadim roberto spadim added a comment -

              hi sergei, sorry the delay, i'm talking about same query being inserted twice:
              https://github.com/MariaDB/server/blob/10.1/sql/sql_cache.cc#L1521

              show be nice have a status variable (global) to count how many refused queries we have and how many was 'concurrent insert into query cache'
              at the qc_info plugin, it's something to tell developer/dba that some query have a 'concurrent insert' or not (must check if this part is or isn't easy to include, but that's the main idea of qc_info part here)

              Show
              rspadim roberto spadim added a comment - hi sergei, sorry the delay, i'm talking about same query being inserted twice: https://github.com/MariaDB/server/blob/10.1/sql/sql_cache.cc#L1521 show be nice have a status variable (global) to count how many refused queries we have and how many was 'concurrent insert into query cache' at the qc_info plugin, it's something to tell developer/dba that some query have a 'concurrent insert' or not (must check if this part is or isn't easy to include, but that's the main idea of qc_info part here)
              Hide
              serg Sergei Golubchik added a comment -

              Ah, I see, thanks. More questions:

              • What did you mean by "*10" or "+1" ? What did you want to use this number (*10 or +1) for?
              • If the user will know the number of "refused" queries, what can he do with it? Why is it useful? For example, the number of pages removed from key cache can useful — if it's very high, a user may want to increase the size of the key cache. But how can one use the number of "refused" queries?
              Show
              serg Sergei Golubchik added a comment - Ah, I see, thanks. More questions: What did you mean by "*10" or "+1" ? What did you want to use this number (*10 or +1) for? If the user will know the number of "refused" queries, what can he do with it? Why is it useful? For example, the number of pages removed from key cache can useful — if it's very high, a user may want to increase the size of the key cache. But how can one use the number of "refused" queries?
              Hide
              rspadim roberto spadim added a comment -

              Hi
              10 and competitors+1
              At query cache plugin with statistcs what i compute is time between one hit and other hit at the same query (the period)
              If the period is lower than query execution time the hit is considered as low (query cache speedup query) but if period is higher we could have two others types of hit
              One hit very strange(outlier) and a interesting hit (high), it's interesting cause we avoid disk i/o, and i consider hit as 'high' when period is higher than query expend time, and lower than 1 seconds (maybe could use slow query time) and 10 times the query time, check that 10 is a constant, i'm considering include the competitors column to check if the 10 number is something related to the number of queries running at the same time of query insert, why?
              When we insert the query expend time could be (that i want check) proporcional to numbers of concurrent query cache inserts, if it's we have a better high period definition instead of a fixed number

              About the global status that's insteresting to check about refuse indormation, if we have a very high qps trying to insert at query cache, or if we have a very bad hit rate or low memory
              The refused without information about why it was refused could be interpreted as bad queries inside cache and/or many concurrent threads running the same query, that's what i'm trying to separe, cause comcurrent thread with aame query is common with a query that could return faster with query cache result (low period) but should not occur (or maybe less) with a high/outlier period query

              Show
              rspadim roberto spadim added a comment - Hi 10 and competitors+1 At query cache plugin with statistcs what i compute is time between one hit and other hit at the same query (the period) If the period is lower than query execution time the hit is considered as low (query cache speedup query) but if period is higher we could have two others types of hit One hit very strange(outlier) and a interesting hit (high), it's interesting cause we avoid disk i/o, and i consider hit as 'high' when period is higher than query expend time, and lower than 1 seconds (maybe could use slow query time) and 10 times the query time, check that 10 is a constant, i'm considering include the competitors column to check if the 10 number is something related to the number of queries running at the same time of query insert, why? When we insert the query expend time could be (that i want check) proporcional to numbers of concurrent query cache inserts, if it's we have a better high period definition instead of a fixed number About the global status that's insteresting to check about refuse indormation, if we have a very high qps trying to insert at query cache, or if we have a very bad hit rate or low memory The refused without information about why it was refused could be interpreted as bad queries inside cache and/or many concurrent threads running the same query, that's what i'm trying to separe, cause comcurrent thread with aame query is common with a query that could return faster with query cache result (low period) but should not occur (or maybe less) with a high/outlier period query
              Hide
              rspadim roberto spadim added a comment - - edited

              This mdev is just an improvement of MDEV-4682
              MDEV-4682 can be used without implementing this one

              Show
              rspadim roberto spadim added a comment - - edited This mdev is just an improvement of MDEV-4682 MDEV-4682 can be used without implementing this one
              Hide
              rspadim roberto spadim added a comment - - edited

              About the "*10" can be found as QUERY_CACHE_QC_INFO_PERIOD_OUTLIER

              I included a comment at github now (commit 79102cd425 at github rspadim/server branch qc_info_statistics)

              Sql_cache.h

              +      if (hit_period<qc_info_select_expend_time) {
              263	+        /* lower than expend time hit period */
              264	+        qc_info_freq_low_hits         += 1;
              265	+        qc_info_freq_low_hits_time    += (qc_info_now.val - hit_time);
              266	+      } else if (
              267	+          hit_period>(qc_info_select_expend_time * QUERY_CACHE_QC_INFO_PERIOD_OUTLIER) &&
              268	+          hit_period>QUERY_CACHE_QC_INFO_PERIOD_OUTLIER_FIXED)
              269	+      {
              270	+        /* outlier */
              271	+        qc_info_freq_high_hits_outlier+= 1;
              272	+      } else {
              273	+        /* higher than expend time hit period, lower than outlier */
              274	+        qc_info_freq_high_hits        += 1;
              275	+        qc_info_freq_high_hits_time   += (qc_info_now.val - hit_time);
              276	+      }
              
              Show
              rspadim roberto spadim added a comment - - edited About the "*10" can be found as QUERY_CACHE_QC_INFO_PERIOD_OUTLIER I included a comment at github now (commit 79102cd425 at github rspadim/server branch qc_info_statistics) Sql_cache.h + if (hit_period<qc_info_select_expend_time) { 263 + /* lower than expend time hit period */ 264 + qc_info_freq_low_hits += 1; 265 + qc_info_freq_low_hits_time += (qc_info_now.val - hit_time); 266 + } else if ( 267 + hit_period>(qc_info_select_expend_time * QUERY_CACHE_QC_INFO_PERIOD_OUTLIER) && 268 + hit_period>QUERY_CACHE_QC_INFO_PERIOD_OUTLIER_FIXED) 269 + { 270 + /* outlier */ 271 + qc_info_freq_high_hits_outlier+= 1; 272 + } else { 273 + /* higher than expend time hit period, lower than outlier */ 274 + qc_info_freq_high_hits += 1; 275 + qc_info_freq_high_hits_time += (qc_info_now.val - hit_time); 276 + }

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  rspadim roberto spadim
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated: