Openbravo Issue Tracking System - Openbravo ERP
View Issue Details
0029708Openbravo ERP02. Master data managementpublic2015-04-27 16:272015-06-06 16:53
egoitz 
umartirena 
immediatemajoralways
closedfixed 
5
 
 
dmiguelez
Core
No
0029708: The costing background process is very slow
Executing the costing background process on an environment with more than 1.500.000 transactions to be processed takes long
-Execute the process on an environment with more than 1.500.000 transactions where the process has never been executed before.

*Right now the commmit is being done for every iteration of the loop.
Doing the commit for every x iterations reduces the time.

No tags attached.
related to defect 0029739 closed Triage Omni WMS Costing Background process takes most of the time of execution on a system 
causes defect 0030201 closed dmiguelez Costing Background Process does not process Transactions if there are more than 1000 in Skip Status 
Issue History
2015-04-27 16:27egoitzNew Issue
2015-04-27 16:27egoitzAssigned To => Triage Finance
2015-04-27 16:27egoitzModules => Core
2015-04-27 16:27egoitzTriggers an Emergency Pack => No
2015-04-27 16:28egoitzResolution time => 1431640800
2015-04-29 19:45VictorVillarIssue Monitored: VictorVillar
2015-04-30 13:25ioritzCiaRelationship addedhas duplicate 0029739
2015-04-30 13:26egoitzPriorityurgent => immediate
2015-04-30 13:30ioritzCiaRelationship deletedhas duplicate 0029739
2015-04-30 13:30ioritzCiaRelationship addedrelated to 0029739
2015-06-03 17:30umartirenaAssigned ToTriage Finance => umartirena
2015-06-03 17:30umartirenaStatusnew => scheduled
2015-06-04 17:47hgbotCheckin
2015-06-04 17:47hgbotNote Added: 0078093
2015-06-04 17:47hgbotStatusscheduled => resolved
2015-06-04 17:47hgbotResolutionopen => fixed
2015-06-04 17:47hgbotFixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/654d4ad1872f1468265ac30de77d055fcd63abde [^]
2015-06-04 18:04dmiguelezReview Assigned To => dmiguelez
2015-06-04 18:04dmiguelezNote Added: 0078096
2015-06-04 18:04dmiguelezStatusresolved => closed
2015-06-06 16:53hudsonbotCheckin
2015-06-06 16:53hudsonbotNote Added: 0078127
2015-06-18 10:11dmitry_mezentsevRelationship addedcauses 0030201

Notes
(0078093)
hgbot   
2015-06-04 17:47   
Repository: erp/devel/pi
Changeset: 654d4ad1872f1468265ac30de77d055fcd63abde
Author: Unai Martirena <unai.martirena <at> openbravo.com>
Date: Tue Jun 02 09:12:55 2015 +0200
URL: http://code.openbravo.com/erp/devel/pi/rev/654d4ad1872f1468265ac30de77d055fcd63abde [^]

Fixes bug 29708, Fixes bug 29936: Improve performance and memory problems

2 Problems are happening in Costing Background process:

1) Java Heap error: This problem happens because getTransactions method can load on memory a lot of objects and even an ScrollableResult is being used the following sentence 'OBDal.getInstance().getConnection().setHoldability(ResultSet.HOLD_CURSORS_OVER_COMMIT)' prevents in fact to scroll properly the ScrollableResult. To avoid this issue the query of getTransactions method has been changed to return only 1000 records. So, instead of calling just once this method it is done once each 1000 records.

Also instead of returning a list of 1000 MaterialTransaction objects, that can be a relatively big amount of objects to load in memory, only the Ids are returned and each id the it is instanced on each iteration.

2) Performance problems: The process has a performance issue because it does a commit on each iteration of the list. Now, this has been changed to do commit every 1000 records, and in case that an exception is raised on a certain iteration a new method has been added in catch clause to calculate the cost of the transactions that have been rolledback but which there were properly calculated.

---
M src/org/openbravo/costing/CostingBackground.java
---
(0078096)
dmiguelez   
2015-06-04 18:04   
Code Review + Testing Ok
(0078127)
hudsonbot   
2015-06-06 16:53   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/7419f65cdeb3 [^]
Maturity status: Test