Propagation Timeout Exception

Problem: While running propagation on WebLogic Portal, it is timing out after 1 Hour/ 3600 seconds.

Error Stack Trace:

Caused By: javax.ejb.EJBException: nested exception is: javax.ejb.EJBException: SQL Error while performing get operation on portlet instance: SQL [select DEF.MARKUP_DEFINITION_ID, DEF.IS_FORKABLE, INST.INTERSECTION_ID, DEF.PORTLET_LABEL, INST.PORTLET_LABEL, DEF.CONTENT_URI, DEF.BACKING_FILE, DEF.IS_PUBLIC, INST.PORTLET_DEFINITION_ID, INST.THEME_DEFINITION_ID, INST.TITLE_BAR_ORIENTATION, INST.DEFAULT_MINIMIZED, INST.INSTANCE_TYPE, INST.PARENT_PORTLET_INSTANCE_ID, DEF.FORK_RENDER, DEF.IS_CACHEABLE, DEF.CACHE_EXPIRES, DEF.PORTLET_FILE, DEF.IS_PORTLET_FILE_DELETED, DEF.HELP_URI, DEF.EDIT_URI, DEF.WEBAPP_NAME, INST.CREATION_DATE, INST.MODIFIED_DATE, DEF.CREATION_DATE, DEF.MODIFIED_DATE, PPV.PRODUCER_HANDLE, PPV.PORTLET_HANDLE, PPV.PORTLET_STATE_CHANGE, PPV.REQUIRES_URL_TEMPLATES, PPV.TEMPLATES_STORED_IN_SESSION, PPV.PORTLET_STATE, PPV.IS_PRODUCER_OFFERED, INST.TITLE_KEY from PF_PORTLET_DEFINITION DEF, PF_PORTLET_INSTANCE INST left outer join PF_PROXY_PORTLET_INSTANCE_V PPV on PPV.PORTLET_INSTANCE_ID = INST.PORTLET_INSTANCE_ID where INST.PORTLET_INSTANCE_ID = ? and DEF.PORTLET_DEFINITION_ID = INST.PORTLET_DEFINITION_ID] portletInstanceId [1,456,041] database error code [0] SQL state [null].; nested exception is: java.sql.SQLException: The transaction is no longer active - status: 'Marked rollback. [Reason=weblogic.transaction.internal.TimedOutException: Transaction timed out after 36002 seconds 
BEA1-5C8E244BF578F5B2D7B1]'. No further JDBC access is allowed within this transaction.
at com.bea.netuix.application.manager.persistence.jdbc.ExportImportManagerImpl.updateDesktopView(ExportImportManagerImpl.java:144)
at com.bea.netuix.application.manager.persistence.jdbc.ExportImportManager_7mrwg_EOImpl.updateDesktopView(ExportImportManager_7mrwg_EOImpl.java:366)
at com.bea.propagation.netuix.online.util.PortalXIPHelper.importDesktop(PortalXIPHelper.java:292)
at com.bea.propagation.netuix.online.util.PortalXIPHelper.addOrUpdateDesktopDefinitionInternal(PortalXIPHelper.java:254)
at com.bea.propagation.netuix.online.util.PortalXIPHelper.addDesktopDefinition(PortalXIPHelper.java:193)
at com.bea.propagation.netuix.online.util.PortalXIPHelper.updateDesktopDefinition_Standard(PortalXIPHelper.java:211)
at com.bea.propagation.netuix.online.updater.PortalDesktopDefnNodeUpdater.updateNode(PortalDesktopDefnNodeUpdater.java:123)
at com.bea.propagation.framework.tool.resolver.TreeResolver.processUpdate(TreeResolver.java:868)
at com.bea.propagation.framework.tool.resolver.TreeResolver.processElectionsIterative(TreeResolver.java:509)
at com.bea.propagation.framework.tool.resolver.TreeResolver.processElections(TreeResolver.java:408)
at com.bea.propagation.framework.tool.resolver.TreeResolver.resolveTree(TreeResolver.java:317)
at com.bea.propagation.framework.tool.servlet.InventoryManagementServlet.commitOperation(InventoryManagementServlet.java:816)
at com.bea.propagation.framework.tool.servlet.InventoryManagementServlet.doOperation(InventoryManagementServlet.java:301)
at com.bea.propagation.framework.tool.servlet.InventoryManagementServlet.doPost(InventoryManagementServlet.java:201)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:821)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:57)
at com.bea.p13n.servlets.PortalServletFilter.doFilter(PortalServletFilter.java:338)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:57)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3592)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2202)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2108)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1432)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)


Main cause: 
Since the propagation is running more than one hour (predefined timeout limit), it is getting timeout after one hour and transactions are being rolled back.

Other possible reasons:
Propagating between two domains which are separated by a firewall, could result in increased time for transactions going over the firewall. Firewall could consume as much as 400% time of actual propagation transactions depending upon the network configurations in enterprise environments. 

Proposed solutions:
Solution 1. Scope the propagation and/or use policy file for propagation to complete within 1 Hour.
Solution 2. Remove the firewall between the propagation domains (source and target), this may or may not save significant amount of time during propagation transactions.
Solution 3. Increase the transaction time out for propagation itself - this requires little more work, details to follow in upcoming posts.

These are the few possible solutions, there could be more like changing the way propagation works e.g. change the proliferation mode to ASYNC or OFF or so.

Default Transaction Attribute for EJBs

Default transaction attributes differ from container to container.
In Weblogic Server, default transaction attribute for MDB is NotSupported however for all other EJBs is Supports for version 10.0 onwards - if no attribute value is specified for trans-attribute in the ejb deployment descriptor.

Container-Managed Transaction Elements in ejb-jar.xml (WebLogic Server 10.x)

Element: transaction-type
Description: Allowable values are Bean or Container.
Default: None, EJB 2.x requires this attribute to be specified.

Element: trans-attribute

Description: 

Specifies how the container manages the transaction boundaries when delegating a method invocation to an enterprise bean's business method. Allowable values are:

• NotSupported
With the NotSupported value, when an entity bean runs in an unspecified transaction, if a transaction exists, the EJB container suspends the transaction; when an entity bean runs in an unspecified transaction, and no transaction exists, the EJB container takes no action.
• Supports
With the Supports value, when an entity bean runs in an unspecified transaction, if a transaction exists, the EJB container uses the current transaction; when an entity bean runs in an unspecified transaction, and no transaction exists, the EJB container takes no action.
• Required
• RequiresNew
• Mandatory
• Never
With the Never value, when an entity bean runs in an unspecified transaction, if a transaction exists, the EJB container throws an exception; when an entity bean runs in an unspecified transaction, and no transaction exists, the EJB container takes no action.

Note: In in pre-9.0 releases of WebLogic Server, the EJB container would start a new transaction when no transaction existed and the value of trans-attribute was NotSupported, Supports, and Never. Set entity-always-uses-transaction inweblogic-ejb-jar.xml to True if you want the EJB container to behave as it did in pre-9.0 releasesof WebLogic Server and create a new transaction.

Because clients do not provide a transaction context for calls to an MDB, MDBs that use container-managed transactions must have trans-attribute of Required.

Default: If not specified, the EJB container issues a warning, and usesNotSupported for MDBs and Supportsfor other types of EJBs.

Element: transaction-scope

Description: 

This optional element specifies whether an enterprise bean requires distributed transactions for its methods or whether the local transaction optimization may be used.

Allowable values are Local and Distributed.

Default: If not specified, the container assumes that distributed transactions must be used.

Reference: http://docs.oracle.com/cd/E11035_01/wls100/ejb/implementing.html#wp1150418

EJB Client generation

EJB Client generation using workspace IDE

- Right click the ejb project – > Properties -> Weblogic EJB – > Jar Settings EJB Client Jar: enter the name of the client jar
- To build the client jar export the project as “EJB Jar File”, both the jars will be created, client as well as ejb jar.

EJB Client generation using ANT

- Add this @JarSettings(ejbClientJar = “xyzManagementClient.jar”) to the bean
- add desired EJBC flags directly to the build script where “weblogic.ejbc” is executed.