How to check SAP Performance | Quick checks

We already shared SAP Performance Tuning in very detailed steps in the post SAP Performance Tuning. Now we are sharing the quick checks. I hope it will help you also.

Yes, there are many things to check and lot of things to be considered before we are going ahead with tuning part. In this post I would like to share a quick protocol on SAP Performance issues and troubleshooting.

Short introduction on basic components which we are going to use commonly. 

Response Time: Time taken by the user request from dispatcher to GUI

Wait Time: Work processes request is waiting in dispatcher Queue

Roll –In: User context data moves from the roll area to work process

Roll –Out: process of user context data from the work process to Roll area

Roll –wait Time: Time b/w roll in and roll out

Load Time: Time of Loading the report from the database to buffer

DB request Time: Time of retrieving/insert the report from/to the data base

CPU time: Time of executing report logic in CPU

Processing Time: Report or program executing time

Below is the summary as how to check the SAP performance issues:

 Standard threshold values for better SAP Performance

Component	Threshold value
Response Time	< 1200 ms
Wait Time	10% of Average Response Time
Roll –wait Time	200 ms
Load Time	10% of Average Response Time
DB request Time	40% of Average Response Time – Wait Time
CPU time	40% of Average Response Time – Wait Time
processing Time	Not much larger than CPU Time

 SAP Performance Problems and Possible Reasons

Problem	Possible Reason
Large Wait Time	Problem with work process sizing
Large Roll –wait Time	Can be a problem with network and RFC connections
Large Load Time	Program (PXA) and Screen (CUA) buffers are too small
Large DB request Time	Can be a problem with Expensive SQL statements, missing indexes ,CPU/Memory bottle necks and DB server
Large CPU time	Can be a problem with processing large tables and frequently accessing R/3 buffers
Large processing Time	CPU bottle necks, Net work problem and common problems

• If single user or single transaction getting this (large response time) performance problem, we need to tune the program which is causing large response time.                
• From ST03, the average response time for dialog-step should not be more than 1000 ms or 1200 ms.                                                                                                                 
• We can Check for the top programs which are causing high data base request time and programs with high CPU time, mainly we concentrate on customized programs             

• If the average DB request time is more than 500 ms, It can problem with missing indexes on tables and expensive SQL statements                                                            
• We can check for missing indexes in DB02 (detail analysis-missing indexes), if we find any missing indexes then we have to re-create the indexes for proper fields.
• And also Check the expensive SQL statements in St04,which are taking High disk reads(> 5% of total physical reads), High buffer gets/misses(> 5% of total reads) monitoring (detailed analysis menu-SQL request tab),if we got any expensive SQL statement ,we need to optimize that SQl statement by ABAP‘ers                                        
• If the CPU time is greater than 400 ms, It can problem with wrong logic of the ABAP program
• We can analyze the time consuming transaction using STAT (ie.,ABAP: RSSTAT30) and request ABAPers to make correction of wrong logic or reprogram.
• If all transactions are getting the (large response time) performance problem we need to analyze the programs which are causing large Wait Time, large roll- wait time, and large load time, large DB request time, large CPU time and large processing time.

If the average wait time is more than 50 ms

• It can problem with long running jobs.
• We can identify the long running jobs by using SM50 or SM66 and tune that long running job or terminate them.
• Work processes are blocked by long running reports, this problem occurring frequently we need to increase the work processes.
• The average number of free work processes available in the system should be sufficiently to ensure that user‘s queries are processed without delay
• If user starts the programs such as reports with very long response times, the affected work processes may be occupied for several minutes. This means that number of remaining work processes is not sufficient to process the queries of other users, which could cause wait times.

If the average DB request time is more than 500 ms

• It can be a problem with missing indexes on tables and expensive SQL statements
• We can check for missing indexes in DB02, if we find any missing indexes and create the indexes for proper fields.
• Also Check the expensive SQL statements in St04,which are taking High disk reads, High buffer gets/misses monitoring (detailed analysis menu-SQL request tab),if we got any expensive SQL statement ,we need to optimize that SQL statement by ABAP‘ers.
• Check and update CBO statistics (DB21).

If the CPU time is greater than 400 ms

• It can be a problem with wrong logic of the ABAP program.
• We can analyze the time-consuming transaction using STAT (ie.,ABAP: RSSTAT30). Than request ABAPers to make correction of wrong logic or reprogram.

If the Load time is greater than 10 ms

• It can be a problem with Program (PXA) and Screen (CUA) buffers are too small
• From ST02, We can check for the buffer quality ,it should be more than 90%,If it is less than 90%
• We can check for swaps on buffer, swaps should be in red colour and not more than 5000 swaps day, if we getting more swaps on buffers
• We need to increase program and screen buffer sizes: Program (PXA) buffer: This buffer stores the compiled executable version of the ABAP programs, also known as program loads. abap/buffersize. Screen (CUA) buffer: This buffer stores menu data, buttons and related SAPGui functionality-rsdb/cua/buffersize)

If the Roll wait time is greater than 200 ms

• There could be a problem with RFC connections or network communication
• From ST03 -We can check the average RFC + CPIC time this time indicates the time for establishing an RFC connection.
• This average RFC time should be less than 10 ms.

If the Processing time is greater than CPU time

• Can be a problem with CPU bottle necks and network problems
• From ST06, we can check the CPU idle %, if high CPU loads (the CPU Idle average per hour is less than 40%, if it is more than 40% can be a problem with logic of the current executing program.
• Then we can analyze the program which is causing top CPU TIME by using STAT (ie.,ABAP: RSSTAT30).
• And also we can analyze the Top CPU processes( ST06)
• If work processes causing high CPU load, We can analyze the time-consuming transaction using STAT (ie.,ABAP: RSSTAT30).If external processes causing high CPU load, we need to terminate the external processes
• We can check the LAN communication from ST06 (detail analysis menu –check by ping tab).

SAP Performance Tuning | How to tune sap performance

SAP Performance Tuning

SAP Performance Tuning is the major activity in the SAP as a BASIS admin, because systematic, proactive performance optimization increases the benefits of your R/3 system and reduces the cost of ownership. The consequences of poor performance tuning may be additional work, production delays, and financial loss. In this post we will describe as how to identify and solve the performance problems in SAP system.

Guidelines for SAP Performance Tuning

          Analyze carefully: Before you change anything, analyze your system carefully. Determine the areas of your R/3 system in which performance problems occur. Changes should be performed only in these areas. Keep a record of your analysis data and main thing is that change as little as possible but as much as necessary.

           Make no changes without analysis: You should never change parameters or take other tuning measures without first performing an analysis. Change suggestions should be implemented only in conjunction with analysis and verification. So blindly applying suggestions from any source can make professional negligence.

            Verify: Once you have made the changes, perform another analysis. Verify whether the changes have provided the desired results. Keep a record of the data from the verification analysis.

            Take small steps: As the term tune implies, the success of an optimization depends on the right amount. Therefore you must not make too many changes at once. Only in this way can you maintain an overview of the tuning measures and verify which of them were successful.

            Every guideline has exceptions: In some situations you may require a different approach than the guideline values or rules of provided by any source. Despite taking every precaution, a recommendation may have negative results. Therefore it is crucial that you perform a verification of every tuning measure taken. This will help you determine why any rule does not suit for a particular situation.

-------------------------------------------------------------------------------------------------------------------------

     Before you going ahead with performance tuning first you need to understand how the communication is happening during running the system between three layers, the diagram is simple flow chart is simple to describe this scenario.

Tuning Categories

Performance Tuning we can categorize into t types.

          1) Technical tuning 

          2) Application tuning.

Technical tuning:

• Configuring all the components of the R/3 system, so that load placed on the system by users can be optimally processed and does not cause performance bottlenecks.                                       
• The components for technical tuning are the operating system, the database, the R/3 work processes, R/3 buffers memory management and the net work.                                                             
• Technical tuning is necessary for every R/3 system. In the technical tuning generally we can check the parameters of work processes, memory allocation, logon groups configuration and load distribution.

Application Tuning:

• Which deals with the programs of R/3 application modules. The main focus is on verifying the necessity and efficiency of processes in applications and minimizing the use of resources such as main memory, CPU, network transfers, and hard disk access.                                                      
• Application tuning typically results in more effective use of R/3 transactions or improved performance of customer developed ABAP programs.                                                                        
• Application tuning is necessary for every R/3 system. In application tuning, the amount of work required increases, with the size of the installation, as reflected in the data volume, the number of users, and the number of customer developed programs and customer modifications to standard SAP objects.

Verifying Tuning Results:

executing the action list, you should reanalyze the R/3 system to verify the success or failure of performance optimization. If desired improvements are not achieved or only partially achieved, another optimization cycle is required. After large scale performance tuning, new problems sometimes occur, so that also need to be solved.

Performance monitor for R/3 applications:

Basis tuning tasks:

1) Optimize System Parameters

• R3 parameters (for example: Memory Management)                                                                          
• Database parameters (for example: Database buffer sizes)                                                                 
• Operating system and network parameters

2) Optimize Database Disk layout through I/O balancing
3) Optimization of Workload distribution

• Number of work processes, Background scheduling, log-on groups

4) Verify hardware sizing by detecting hardware bottlenecks

Traces with ST01:

       a)  In ST01 we can do traces with Analysis

       b)  Trace for Authorization check

       c)  Trace for Kernel functions

        d) Trace for DB access

        e) Trace for Table buffer

        f) Trace for RFC calls

            So that we can find out : by TRACE ON and OFF

Memory Tuning in ST02:

            The buffer tune summary transaction displays the R/3 buffer performance statistics. It is used to tune buffer parameters of R/3 and, to a lesser degree, the R/3 database and operating system.

            The buffer is important because significant buffer swapping reduces performance. Look under Swaps for red entries. Regularly check these entries to establish trends and get a feel of the buffer behavior.

             In the Command field, enter transaction ST02 and choose Enter (or choose Tools Administration, then Monitor > Performance, and then Setup/Buffers > Buffers).

             Based on the buffer hit ratio and swaps In the St02 you can get the relevant data as which parameters needs to be checked for further analysis and performance tuning.

The two important things to review:

Hit Ratio: for which the target value is 95 percent and higher. Soon after starting the system, this value is typically low, because certain buffers are empty.  The hit ratio will increase as the system is used and the buffers are loaded. It usually takes a day to load the buffers that are normally used.

Swaps: for which the target value is less than 1,000. Swaps occur when the necessary data is not in the buffer. The system has to retrieve the data from the database. The swap value is reset to zero (0) when the system is restarted.

Buffer swaps may be due to

• Buffer too small, out of space,
• Out of buffer directory entries
• Fragmentation in buffer, particularly the program buffer
• If program buffer exceeds 10,000, it should be investigated
• Generic table buffer
• Sometimes very large buffered objects should be un buffered (over 2MB)

Work load analysis with ST03:

         1)   Workload analysis is used to determine system performance. St03 is most important transaction in SAP performance tuning.
          2)  Statistics should be checked, and trends should be recorded to get a “feel” for the system’s behavior and performance. Understanding the system when it is running well will help determine what changes need to be made when it is not.
          3)   Check Response times
          4)   Choose Detailed analysis menu from toolbar.
          5)   Choose One recent period from Performance history, Global.
          6)   We can choose a time period for comparison: Today, Previous days, This week, Previous weeks, This month, Previous months.
          7)   Choose Dialog at the bottom of the screen
          8)   Choose Transaction profile from the toolbar
          9)   Main menu dialog step average time should be very short, If it is not, this could indicate a dialog instance bottleneck
Key dialog response times should be less than 1-2 seconds per step.

Key dialog transaction step time components:

• Wait time should be very short
• DB time should be much less than CPU time.
• Compare the servers
• Choose Detailed analysis menu from toolbar
• Choose Compare all servers from Performance history, Global.
• This indicates average performance across all instances
• If there is a performance problem specific to an instance, Avg wait time and CPU time will be the best indicators.

Database Performance Analysis:

Oracle data buffer checks:

• Check If there are more than a few million reads, the buffer quality should be > 98%

• < 98% could indicate buffer too small or expensive SQL statements.

• Buffer waits should be < 5% of reads 5% could indicate I/O bottleneck.

Oracle call statistics checks

• Reads per user call should be < 40
• 40 could indicate expensive SQL.
• User calls / recursive calls should be>5
• < 5 could indicate that the Shared Pool may be too small.

Oracle call statistics checks

• Reads per user call should be < 40
• 40 could indicate expensive SQL.
• User calls / recursive calls should be > 5
• < 5 could indicate that the Shared Pool may be too small.

Oracle shared Pool statistics checks

• DD cache quality should be > 90%
• < 90% could indicate that the Shared Pool is too small
• SQL Area get ratio, pin ratio should be > 90%
• < 90% could indicate that the Shared Pool is too small

Check the Performance in ST05:

Check database SQLLook for statements with

• More than 1,000-10,000 buffer gets / record (most optimization potential)
• Buffer gets / total reads since start in more than 5% of the total
• More than a few executions
• Larger fraction of reads from disk than other statements
• Double click on line to get more information
• Once displayed, the “Explain” option can be selected for further analysis

Expensive SQL statement Means

• From user point of view: Lengthy response time of transaction using the statement
• From the system point of view: Many data blocks are scanned to find the selected records

Consequences of Expensive SQL statement

• Database is busy reading with many data blocks
• High CPU load on Database Server
• Work process is block by report & wait time for other processes
• Many blocks are displaced from Database buffer
• Cache hit rate for other SQL statement suffers
• Expensive SQL statement reduces the performance of the entire R3 System.

Detecting Expensive SQL statements:

• What to detect: Reports or transactions where database request time is a large fraction of response time
• SQL statement with a high number of buffer gets

For each Expensive SQL Statement find out:

• Table Name
• Where Clause
• Indexes used
• Name of the report or transaction containing the statement

Operating System Monitor in ST06:

• CPU utilization and system load                                                                                           
• Check idle %
• Check load average
• Load average should be > 1.5
• Physical and virtual memory
• Swap statistics
• Slowest disk
• LAN statistics

Analyzing the hardware bottle neck:

From a user point of view:

• Lengthy response time for transactions

From the system point of view

• CPU Utilization is 100 %
• High average number of work process are waiting for CPU resources
• High paging rates
• Lengthy disk response times
• Lengthy network response times

Reasons

• Incorrect sizing of physical main memory, CPU, etc…
• Incorrect workload distribution
• Expensive Programs (SQL Programs, External Programs)
• Incorrect disk layout or slow disks
• Incorrect network topology

We can also check the ST07 to monitor user distribution based on the application components.

How to check SAP Performance | Quick checks