Understanding Oracle SQL Plan Management SPM – Part 3

This is the third post of SQL Plan Management. In the previous post, Understanding SQL Plan Management– Part 2, I described the three main components of SPM, this time I want to show you how to manually capture plans from the cursor cache. In other words, manually capture plans that reside in memory.

In order to manually capture plans, I’m considering that automatic capture is disabled or OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES is set to FALSE. Oracle provides a function named DBMS_SPM.load_plans_from_cursor_cache to manual capture plans from cursor cache.

DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE (

   sql_id            IN  VARCHAR2,

   plan_hash_value   IN  NUMBER   := NULL,

   sql_text          IN  CLOB,

   fixed             IN  VARCHAR2 := 'NO',

   enabled           IN  VARCHAR2 := 'YES')

 RETURN PLS_INTEGER;

The common use for this function, is to specify the sql_id of the statement we want to capture. We also can specify the plan_hash_value or the sql_text. Two more attributes can be set within this function, fixed is ‘NO’ by default and enabled is ‘YES’ by default. If you want to load plans, but you don’t want the CBO to make use if them; set the attribute enabled to ‘NO’.

In the below example I’m capturing all the SQL plans for sql_id 4c372tsuhtunm and printing the number of plans loaded:

SQL> SET SERVEROUTPUT ON

DECLARE

  l_plans_loaded  PLS_INTEGER;

BEGIN

  l_plans_loaded := DBMS_SPM.load_plans_from_cursor_cache(

    sql_id => '4c372tsuhtunm');

  DBMS_OUTPUT.put_line('Plans Loaded: ' || l_plans_loaded);

END;

/SQL>   2    3    4    5    6    7    8    9

Plans Loaded: 8

PL/SQL procedure successfully completed.

To verify the SQL baselines created for this sql_id, I just need to query DBA_SQL_PLAN_BASELINES view:

SQL> select SIGNATURE,SQL_HANDLE,PLAN_NAME,ORIGIN,ENABLED,ACCEPTED,FIXED,REPRODUCED from dba_sql_plan_baselines;

            SIGNATURE SQL_HANDLE            PLAN_NAME                      ORIGIN         ENABLED ACCEPTED FIXED REPRODUCED

--------------------- --------------------- ------------------------------ -------------- ------- -------- ----- ----------

  5535156277878016190 SQL_4cd0d34ee73148be  SQL_PLAN_4tn6m9vmm2k5ye8c8b02d MANUAL-LOAD    YES     YES      NO    YES

  5535156277878016190 SQL_4cd0d34ee73148be  SQL_PLAN_4tn6m9vmm2k5yc568a49d MANUAL-LOAD    YES     YES      NO    YES

  5535156277878016190 SQL_4cd0d34ee73148be  SQL_PLAN_4tn6m9vmm2k5ybff74238 MANUAL-LOAD    YES     YES      NO    YES

  5535156277878016190 SQL_4cd0d34ee73148be  SQL_PLAN_4tn6m9vmm2k5y0310173f MANUAL-LOAD    YES     YES      NO    YES

  5535156277878016190 SQL_4cd0d34ee73148be  SQL_PLAN_4tn6m9vmm2k5y8e42f3cc MANUAL-LOAD    YES     YES      NO    YES

  5535156277878016190 SQL_4cd0d34ee73148be  SQL_PLAN_4tn6m9vmm2k5y25375ef9 MANUAL-LOAD    YES     YES      NO    YES

  5535156277878016190 SQL_4cd0d34ee73148be  SQL_PLAN_4tn6m9vmm2k5y9de69d5d MANUAL-LOAD    YES     YES      NO    YES

  5535156277878016190 SQL_4cd0d34ee73148be  SQL_PLAN_4tn6m9vmm2k5yb76f0084 MANUAL-LOAD    YES     YES      NO    YES

8 rows selected.

Look at the data for the ORIGIN column, it shows from where the plans were loaded, either from manual load or auto capture.

Another method is to manual load plans from SQL Tuning Sets (STS). The STS may contain plans that are not present in memory, like plans in the AWR repository. This method is very useful when you want to create baselines of plans that were created by the CBO few days ago and are not in the cursor cache at this time. Keep in mind that STS requires a special license in order to be used. Oracle provides the function DBMS_SPM.load_plans_from_sqlset to accomplish this task.

DBMS_SPM.LOAD_PLANS_FROM_SQLSET (

   sqlset_name      IN  VARCHAR2,

   sqlset_owner     IN  VARCHAR2 := NULL,

   basic_filter     IN  VARCHAR2 := NULL,

   fixed            IN  VARCHAR2 := 'NO',

   enabled          IN  VARCHAR2 := 'YES'

   commit_rows      IN  NUMBER   := 1000)

RETURN PLS_INTEGER;

You just need to specify the sqlset_name and the sqlset_owner in order to load the plans contained in the STS. It also has the default parameters fixed, set to ‘NO’ and enabled, set to ‘YES’. Two interesting attributes can be set within this function. The basic_filter attribute allows you to select only the plans that meet this filter criteria, in other words, is like a ‘where’ in a query.

basic_filter => 'sql_text like ''select /*LOAD_STS*/%''' orbasic_filter => 'sql_id="4c372tsuhtunm"'

The commit_rows attribute, allows you to commit after the value specified, which is 1000 by default. Let’s assume that inside this STS we have 30000 plans and we want to load all of them, SPM will commit every 1000 plans to help to reduce the undo log.

Keep in mind that once you manually load plans for a specific SQL (signature), the CBO will continue capturing plans for it, even if the automatic capture is disabled. Those automatic captured plans will not be used by the CBO until you verify and evolve them.

In the next post, I’m going to show you how to evolve plans in 11g and 12c versions.

Thanks,

Alfredo

Understanding Oracle SQL Plan Management SPM – Part 2

In my previous post Understanding SQL Plan Management – Part 1, I tried to cover some basic concepts of SQL Plan Management (SPM). This post will show you the main components of SPM and how they work in the different versions of Oracle.   

SQL plan management framework has three main components, plan capture, plan selection and plan evolution. These components allow administrators choose which plans should be executed by the database. Plan capture is the process of loading execution plans from different sources into the SQL management base. Plan selection is the process the database follows in order to choose the best plan available after considering many factors including SQL baselines. Plan evolution is the process of making baselines executable after verifying if any performance improvement is available.

Plan capture can be done in two forms, automatic and manual. Automatic plan capture takes place when the database initialization parameter OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES is set to TRUE. Keep in mind that the default value for this parameter is FALSE. If automatic capture is enabled, the CBO will log the SQL signature of any SQL statement executed for the very first time in the database. When the SQL statement is executed for the second time it will recognize it as a repeatable statement in the database and SQL baselines are automatically created and marked as “ACCEPTED” for each repeatable SQL statement. Be careful with this feature because the baseline will be created using the execution plan of the second execution whether is optimal or not. If the CBO produces a better execution plan after the baseline is created, the plan will be stored in the SMB but not used until the plan is evolved.

Manual plan capture is the most common method to capture plans. It can be done in conjunction or instead of automatic capture for a single SQL statement or a group of statements. Plans can be manually loaded by using either DBMS_SPM package or Oracle Enterprise Manager (OEM) from four different sources. Manual loaded plans are automatically “ACCEPTED” and the CBO will continue to load plans automatically for this plans even if automatically plan capture is disabled. Plans automatically loaded after the manual load are marked as not “ACCEPTED”. You can load plans from four sources. From cursor cache, which are the active plans in memory. From SQL tuning sets, which require the SQL tuning pack or real application testing license. From stored outlines and from another Oracle database system using datapump. In order to transfer baselines from one database system to another, you must pack, load and unpack the baseline by following the below steps:

1. On the original system, create a staging table using the DBMS_SPM.CREATE_STGTAB_BASELINE procedure.

2. Pack the SQL plan baselines you want to export from the SQL management base into the staging table using the DBMS_SPM.PACK_STGTAB_BASELINE function.

3. Export the staging table into a flat file using the export command or Oracle Data Pump.

4. Transfer this flat file to the target system.

5. Import the staging table from the flat file using the import command or Oracle Data Pump.

6. Unpack the SQL plan baselines from the staging table into the SQL management base on the target system using the DBMS_SPM.UNPACK_STGTAB_BASELINE function.

The CBO calculates the execution plan every time the SQL statement is parsed (compiled) and then proceeds to execute the statement. If the OPTIMIZER_USE_SQL_PLAN_BASELINES is set to TRUE, the CBO will check if a SQL baseline exists before the plan is executed. If there is no SQL signature that matches the parsed SQL statement then the CBO will log this signature in the statement log if the automatic capture is enabled or just proceeds to execute the statement if disabled. If a signature matches the statement and a baseline doesn’t exists, it will proceed to create a plan for verification but if a baseline already exists then the baseline will be executed. If more than one “ACCEPTED” plan exists in the baseline, the CBO costs each plan for the given SQL statement and picks the one with the lowest cost. If a baseline is marked as “FIXED”, the CBO picks the lowest cost fixed plan unless all fixed plans are marked as non-reproducible.

Plan evolution, in simple terms is the process to “ACCEPT” or “REJECT” plans after verifying that performance is better or not compared with the current plan. Plan evolution as plan capture can be done automatic and manual. Automatic plan evolution is managed by the SQL tuning advisor task in 11g versions, while is managed by the SPM Evolve Advisor in 12c.

Adaptive SQL Plan Management  is one of the new features of Oracle 12c. Adaptive SQL Plan Management, is just the new automatic evolve task SYS_AUTO_SPM_EVOLVE_TASK that runs in the nightly maintenance window and is enabled by default. This task ranks all unaccepted plans and runs the evolve process for them. If the plan performs 1.5x times better than the current plan in the SQL plan baseline, then the plan is automatically accepted and becomes usable by the optimizer. This ratio defined by the hidden parameter _PLAN_VERIFY_IMPROVEMENT_MARGIN and can be modified. After the evolve task is complete, a persistent report is generated with details on how the non-accepted plan perform compared to the accepted plan performance. Administrators can go back and check what plans were evolved to any point in time.

      

Manual plan evolution is bit different in every version of the database. In 11g release 1, is being controlled using the DBMS_SPM.ALTER_SQL_PLAN_BASELINE function and by changing the attribute_name ‘ACCEPTED’ to ‘YES’. Is the administrator’s responsibility verify that the plan is performing better than the current plan. In 11g release 2, the DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE function helps the adminstrator verify if the plan performs better than the current plan and if it does, the plan is marked as “ACCEPTED”. A new SPM Evolve Advisor API is available in 12c version. Create task, execute task and report the evolve taks are part of the three step process of evolving plans. Unaccepted plans are not manually evolved when using the SPM Evolve Advisor, therfore the plan must be manually accepted using DBMS_SPM.ACCEPT_SQL_PLAN_BASELINE function.   

In the next post I will show you how to use the different DBMS_SPM functions to capture and evolve plans.

Thanks,

Alfredo

Understanding Oracle SQL Plan Management SPM – Part 1

This time I want to talk about SQL Plan Management, explain how this feature works and how to make the best use of it. As is a very extensive topic I will split it in three posts.

Introduction

The Oracle Cost Based Optimizer (CBO) introduced in Oracle version 7. It determines the most efficient way to execute SQL statements after considering several factors such as database initialization parameters, optimizer statistics and many others like bind peeking, cardinality feedback, etc.

The CBO’s goal is to produce an optimal execution plan for the SQL statements executed in the database system. This is the reason why we want the CBO to be flexible enough to produce the best execution plans for the SQL statements. We must not forget that DB systems are dynamic enough to have data distribution changes, new Indexes may be added and initialization parameters could be modified, leading the CBO to produce multiple execution plans for the same SQL statement. This is an expected behavior of the CBO and we should be glad that is smart enough to produce optimal plans most of the times during the hard parsing, however sometimes it may also produce sub-optimal plans under special circumstances.

Here’s where plan stability tools come on the scene. Tools like hints, outlines, SQL Plan Management and custom SQL Profiles help the DBA and developers to allow only optimal plans be executed, hence avoiding severe performance problems within the database.

SQL Plan Management (SPM) is a new feature of Oracle 11g. SPM provides a framework for plan stability and control by ensuring that only selected plans are executed. If new plans are created by the CBO, they will not be executed until they are verified by the database or by the administrator and marked as accepted.

SPM controls the execution plans by using three control flags. The “ENABLED” flag that accepts two values YES & NO controls if the plan is available or not for the CBO to be considered. If the “ACCEPTED” flag is set to YES and “ENABLED” is set to YES, then the CBO will execute the plan. If “ACCEPTED” is set to NO, the plan should be verified by the database system or the administrator to check if it has comparable or better performance than the current plan. This process of making not “ACCEPTED” plans into “ACCEPTED” is called plan evolution. The last control flag name is “FIXED” and can only be set to YES if the plan is “ENABLED” and “ACCEPTED”, if set to YES plans have priority over “ACCEPTED” plans just because they are not expected to change.

SPM uses a mechanism called SQL plan baseline (baseline). Baseline is a group or set of “ACCEPTED” plans the CBO is allowed to use for a particular SQL statement. These baselines and the not “ACCEPTED” plans are stored into the SQL plan history. All baselines and the plan history are logically stored into the SQL management base (SMB) in the data dictionary. The SQL management base stores the statement log, plan histories, SQL plan baselines, and SQL profiles. Oracle provides information about SQL baselines in the DBA_SQL_PLAN_BASELINES and v$SQL system views. Database administrators can query information about what SQL baselines exists, their status and origin.

More information about DBA_SQL_PLAN_BASELINES can be found here https://docs.oracle.com/database/121/REFRN/refrn23714.htm#REFRN23714.

Oracle matches SQL statements with SQL baselines by using signatures. SQL signature is a unique identifier created from the normalized SQL text, uncased and whitespaces removed. SQL signatures ensure that the same SQL statement is always having the same SQL signature independent of the upper/lower case or spaces. SQL signatures are stored into the SQL management log in the SMB. If a SQL statement has its signature stored in the SMB, means to be a repeatable statement and a baseline will be created during the next execution.

SQL baselines also have three status flags. The first flag “REPRODUCED” is automatically set to YES when the CBO is able to reproduce the plan for the given SQL statement and NO when not possible, like when an Index is dropped. The second flag “AUTOPURGE” is user modifiable, if set to YES the plan will be purged when not used and reaches the SPM plan retention limit parameter. The last status flag “REJECTED” is set to YES when “ACCEPTED” is set to NO and the plan was verified (has LAST_VERIFIED), or “ENABLED” is set to no in 11gR2 and 12c versions.

SQL plan management is controlled by two initialization parameters. When the optimizer_use_sql_plan_baselines parameter is set to TRUE (default), the CBO will make use of the baselines created and stored in the SMB. The second parameter optimizer_capture_sql_plan_baselines is set to FALSE by default and controls if the database will automatically capture the SQL signature in the SQL management log and automatically create a SQL baseline on the second execution of the SQL statement. As a best practice and personal recommendation, don’t set optimizer_capture_sql_plan_baselines to TRUE. If set to TRUE, any baseline that is automatically created is also set to “ACCEPTED”, this means that the second execution of a SQL statement will always be used by the CBO even if there’s a better execution plan available. 

In the next post I’m going to cover how to capture and evolve SQL baselines in 11g and 12c version, stay tuned!

Thanks,

Alfredo

Why you should attend to IOUG Collaborate 2015?

Today I want to share a personal story about IOUG’s Collaborate, a great conference that supports products from Engineered Systems, Oracle Databases to Fusion Middleware and Siebel. Last year was my first time as a speaker and was one of kind experience as I was able to share my knowledge in Oracle Database performance tuning with professionals that work on a daily basis with Oracle technologies.

This is what makes Collaborate great, professionals sharing their experiences to other professionals. You can be sure that the guys who are speaking at Collaborate really know what they are talking about, that’s a guarantee. Apart of education, networking is also great at Collaborate, so you can meet IT professionals, Oracle Product Managers and technology gurus (you may already seen their blogs).

This year Collaborate will be greater than ever, just imagine to learn from Tom Kyte, Craig Shallahamer and Kellyn Pot’Vin-Gorman, you don’t want to miss the opportunity!

If you want to be part of this big IT event, want to learn great stuff that will help you with your daily work and want to network with great people, you should register right away!

More information available at http://collaborate.ioug.org/

BTW. If you attend the Collaborate 2015, be sure to stop by my sessions about Oracle DB Performance and Oracle Enterprise Manager

http://coll15.mapyourshow.com/6_0/sessions/session-details.cfm?ScheduleID=3431

http://coll15.mapyourshow.com/6_0/sessions/session-details.cfm?ScheduleID=4503

Oracle Enterprise Manager at Collaborate 2015 https://lnkd.in/e3G6ZzK

Thanks,

Alfredo