Recent Blog Entries

Monday, 20 August 2012

AIA/SOA Trips & Tricks (4) : How to Save AIA/BPEL 11g Execution Time Statistics Programmatically in a File

Accessing and saving statistics is quite different in SOA 11g – this is done through JXM MBeans and not anymore by calling a BPEL API.

The following example shows how to retrieve the execution time statistics for all BPEL components deployed to one SOA server.

The example output is:

FOUND 15
Time    BPEL Name    Count    Min    Avg    Max
11:48:19    ProcessFOBillingAccountListRespOSMCFSCommsJMSProducer    6    326    2568.6666666666665    3068
11:48:19    UpdateSalesOrderSiebelCommsProvABCSImplProcess    6    1482    1821.5    2236
11:48:19    CommsProcessFulfillmentOrderBillingAccountListEBF    6    16590    22458.5    29167
11:48:19    ProcessFulfillmentOrderBillingResponseOSMCFSCommsJMSProducer    6    28    166.5    842
11:48:19    AIAAsyncErrorHandlingBPELProcess    4    1459    1758.5    2065
11:48:19    ProcessFulfillmentOrderBillingBRMCommsProvABCSImplProcess    6    1805    2462.8333333333335    4031
11:48:19    QueryCustomerPartyListSiebelProvABCSImplV2    10    640    2639.8    11079
11:48:19    AIASessionPoolManager    20    13    96.0    1344
11:48:19    ProcessSalesOrderFulfillmentOSMCFSCommsJMSProducer    10    94    562.9    1930
11:48:19    ProcessFulfillmentOrderBillingBRMCommsAddSubProcessProcess    6    773    1211.0    1577
11:48:19    SyncCustomerPartyListBRMCommsProvABCSImpl    10    323    2956.0    4045
11:48:19    TestOrderOrchestrationEBF    6    39979    46680.166666666664    52206
11:48:19    ProcessSalesOrderFulfillmentSiebelCommsReqABCSImplProcess    10    1125    2247.1    6522
11:48:19    CommsProcessBillingAccountListEBF    10    7342    12365.5    22876
11:48:19    AIAReadJMSNotificationProcess    4    9    54.5    124

You can easily paste the output in Excel to display charts like:

image

image

You also can periodically retrieve the statistics to determine if there is any performance degrade for some BPEL processes over time.

Lets see how the JMX API is used to achieve this:

First we need to establish a connection to the MBean server – for this we use the same method as we did in our JMXClient:

public static void initConnection(String hostname, String portString,
                                  String username,
                                  String password) throws IOException,
                                                          MalformedURLException {
    String protocol = "iiop";

    Integer portInteger = Integer.valueOf(portString);
    int port = portInteger.intValue();
    String jndiroot = "/jndi/";
    String mserver = "weblogic.management.mbeanservers.domainruntime";

    JMXServiceURL serviceURL =
        new JMXServiceURL(protocol, hostname, port, jndiroot + mserver);

    Hashtable h = new Hashtable();
    h.put(Context.SECURITY_PRINCIPAL, username);
    h.put(Context.SECURITY_CREDENTIALS, password);
    h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES,
          "weblogic.management.remote");
    // Wait timeout 60 seconds
    h.put("jmx.remote.x.request.waiting.timeout", new Long(60000));
    connector = JMXConnectorFactory.connect(serviceURL, h);
    connection = connector.getMBeanServerConnection();
}

After that we retrieve all Mbeans which have the same pattern:

String mBeanName =
    "oracle.dms:Location=" + servername + ",soainfra_composite_label=*,type=soainfra_component,soainfra_component_type=bpel,soainfra_composite=*,soainfra_composite_revision=*,soainfra_domain=default,name=*";

Set<ObjectInstance> mbeans =
    connection.queryMBeans(new ObjectName(mBeanName), null);
System.out.println("FOUND " + mbeans.size());

This matches the display in Enterprise Manager “System MBean Browser”:

EM2

Now, we can query each MBean for the attributes

  • Name
  • successfulInstanceProcessingTime_completed
  • successfulInstanceProcessingTime_minTime
  • successfulInstanceProcessingTime_avg
  • successfulInstanceProcessingTime_maxTime

That’s it!

You can find the complete JDeveloper project here.

The same statistics can of course be retrieved as well programmatically for composites (services) and references.

Friday, 17 August 2012

AIA/SOA Trips & Tricks (3) : How to Save BPEL 10g Statistics Programmatically in a File

In Oracle BPEL Process Manager, statistics about the average execution times of BPEL processes can be displayed on the BPEL Console:

image

It is very important to monitor these values to be able to react on abnormal high execution times – especially for sync processes – to prevent from JTA transaction timeouts, for example.

While this is possible with EM Grid Control and SOA Management Pack, there is no out of the box solution without EM GC  for saving these statistics periodically to a file.

For this purpose, I have written two Java classes – one for retrieving the statistics via the BPEL API and one for resetting the statistics values.

The first makes use of of the following API calls

Statistics[] asyncStats = domain.getAsyncStatistics();
System.out.println("===> Async BPEL Domain Statistics:");
for (int i=0;i<asyncStats.length;i++)
    System.out.println(asyncStats[i]);

Statistics[] syncStats = domain.getSyncStatistics();
System.out.println("===> Sync BPEL Domain Statistics:");
for (int i=0;i<syncStats.length;i++)               
    System.out.println(syncStats[i]);

Statistics[] requestStats = domain.getRequestStatistics();
System.out.println("===> BPEL Engine Requests Statistics:");
for (int i=0;i<requestStats.length;i++)
    System.out.println(requestStats[i]);

The JDeveloper project with both classes can be downloaded from here.

To adapt it to your environment, you need to change the connection URL and credentials in file classes/context.properties. Also you need to have the OC4J libraries listed in getBPELStats.sh in a local directory (copy these from the server).

Sample output:  (format is:  minimum < “Process Name”  ~ average  < maximum time is milliseconds)

GetBPELStats for SOA10g
Date: 17.08.2012 11:41:41:336
===> Successfully connected to domain default
===> Async BPEL Domain Statistics:
3158 < OrderEnqueue 1.0 ~ 3158.0 < 3158 based on 1 stats
===> Sync BPEL Domain Statistics:
781 < SyncCustomerPartyListBRMCommsProvABCSImpl 1.0 ~ 972.9 < 1261 based on 10 stats
18468 < CommsProcessFulfillmentOrderBillingAccountListEBF 1.0 ~ 18995.5 < 19315 based on 10 stats
14181 < CommsProcessBillingAccountListEBF 1.0 ~ 14589.9 < 14881 based on 10 stats
1461 < QueryCustomerPartyListSiebelProvABCSImplV2 1.0 ~ 1908.2 < 2605 based on 10 stats
1346 < ProcessFulfillmentOrderBillingResponseOSMCFSCommsJMSProducer 1.0 ~ 1504.3 < 1792 based on 10 stats
7255 < ProcessSalesOrderFulfillmentSiebelCommsReqABCSImpl 1.0 ~ 7543.5 < 7704 based on 10 stats
5 < AIASessionPoolManager 1.0 ~ 716.3 < 2162 based on 20 stats
18692 < ProcessFulfillmentOrderBillingBRMCommsProvABCSImpl 1.0 ~ 21442.1 < 22780 based on 10 stats
108791 < TestOrderOrchestrationEBF 1.0 ~ 109057.7 < 109601 based on 10 stats
17035 < UpdateSalesOrderSiebelCommsProvABCSImpl 1.0 ~ 18881.2 < 20064 based on 10 stats
3403 < ProcessFulfillmentOrderBillingBRMCommsAddSubProcess 1.0 ~ 3973.8 < 4550 based on 10 stats
1816 < ProcessSalesOrderFulfillmentOSMCFSCommsJMSProducer 1.0 ~ 2263.8 < 2448 based on 10 stats
2426 < ProcessFOBillingAccountListRespOSMCFSCommsJMSProducer 1.0 ~ 2784.2 < 3002 based on 10 stats
===> BPEL Engine Requests Statistics:
0 < eng-composite-request.eng-single-request.handle-workitem.do-perform.actual-perform.InvokePCM_OP_CUST_COMMIT_CUSTOMER.sensor-send-activity-data ~ 0.0 < 0 based on 20 stats
8 < eng-composite-request.eng-single-request.handle-workitem.do-perform.actual-perform.Invoke_UpdateSO.invoke-service.invoke.actual-invoke.wsif-invoke.wsif-remote-call.prepare-call.populate-message.deep-copy ~ 14.2 < 20 based on 10 stats
0 < eng-composite-request.glue-requests-store.cube-instance-save-datastore.ci-store.at-store.at-db-insert.to-byte-array ~ 0.05 < 2 based on 71 stats
0 < eng-composite-request.eng-single-request.handle-workitem.do-perform.actual-perform.TwoPhaseMode_to_Three.sensor-send-variable-data ~ 0.0 < 0 based on 10 stats
1 < eng-composite-request.eng-single-request.load-instance-for-workitem.load-instance.actual-instance-load.cube-instance-load-datastore.ci-load.ci-load-instance ~ 4.2 < 71 based on 40 stats
4 < eng-composite-request.eng-single-request.handle-workitem.do-perform.actual-perform.ds-subscribe ~ 12.17 < 44 based on 40 stats

Tuesday, 7 August 2012

AIA Tips & Tricks Series (2): Preventing from Exceptions with Very Large Audit Trails in AIA/SOA 11g

In very complex flows, audit trail or flow trace sizes can grow above the limit where EM console will display this correctly with the default settings. The reason is to prevent from generating huge audit trail objects in memory and thus potentially causing OOM errors – therefore the standard limit is set to 1 MB.

You will see the following exception in a case where the audit trail exceeds this limit:

Exception occured while retrieving the Flowtrace XML for the Composite Instance; ECID: 9c37958941ffc184:-3503c179:138e19ea68a:-8000-0000000000002ee6
java.rmi.RemoteException: EJB Exception: ; nested exception is: 
   java.lang.RuntimeException: oracle.soa.management.facade.DataSetTooLargeException: Requested audit trail size is larger than threshold 1048576 chars
   at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:237)
   …

If this is not a design or implementation error (for example an indefinite loop), then you can try to increase this audit trail limit:

instanceTrackingAuditTrailThreshold

You can modify it

in the System MBean Browser at the following location:

image

See also http://docs.oracle.com/cd/E25178_01/core.1111/e10108/others.htm Chapter “11.2.3 instanceTrackingAuditTrailThreshold” for details.