Openbravo Issue Tracking System - Openbravo ERP
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0029708Openbravo ERP02. Master data managementpublic2015-04-27 16:272015-06-06 16:53
Reporteregoitz 
Assigned Toumartirena 
PriorityimmediateSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in Version 
Merge Request Status
Review Assigned Todmiguelez
OBNetwork customerOBPS
Web browser
ModulesCore
Support ticket35231
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary0029708: The costing background process is very slow
DescriptionExecuting the costing background process on an environment with more than 1.500.000 transactions to be processed takes long
Steps To Reproduce-Execute the process on an environment with more than 1.500.000 transactions where the process has never been executed before.

*Right now the commmit is being done for every iteration of the loop.
Doing the commit for every x iterations reduces the time.

Proposed Solution
Additional Information
TagsNo tags attached.
Relationships
related to defect 0029739 closed Triage Omni WMS Costing Background process takes most of the time of execution on a system 
causes defect 0030201 closed dmiguelez Costing Background Process does not process Transactions if there are more than 1000 in Skip Status 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2015-04-27 16:27egoitzNew Issue
2015-04-27 16:27egoitzAssigned To => Triage Finance
2015-04-27 16:27egoitzOBNetwork customer => Yes
2015-04-27 16:27egoitzModules => Core
2015-04-27 16:27egoitzTriggers an Emergency Pack => No
2015-04-27 16:28egoitzSupport ticket => 35231
2015-04-27 16:28egoitzResolution time => 1431640800
2015-04-29 19:45VictorVillarIssue Monitored: VictorVillar
2015-04-30 13:25ioritzCiaRelationship addedhas duplicate 0029739
2015-04-30 13:26egoitzPriorityurgent => immediate
2015-04-30 13:30ioritzCiaRelationship deletedhas duplicate 0029739
2015-04-30 13:30ioritzCiaRelationship addedrelated to 0029739
2015-06-03 17:30umartirenaAssigned ToTriage Finance => umartirena
2015-06-03 17:30umartirenaStatusnew => scheduled
2015-06-04 17:47hgbotCheckin
2015-06-04 17:47hgbotNote Added: 0078093
2015-06-04 17:47hgbotStatusscheduled => resolved
2015-06-04 17:47hgbotResolutionopen => fixed
2015-06-04 17:47hgbotFixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/654d4ad1872f1468265ac30de77d055fcd63abde [^]
2015-06-04 18:04dmiguelezReview Assigned To => dmiguelez
2015-06-04 18:04dmiguelezNote Added: 0078096
2015-06-04 18:04dmiguelezStatusresolved => closed
2015-06-06 16:53hudsonbotCheckin
2015-06-06 16:53hudsonbotNote Added: 0078127
2015-06-18 10:11dmitry_mezentsevRelationship addedcauses 0030201

Notes
(0078093)
hgbot   
2015-06-04 17:47   
Repository: erp/devel/pi
Changeset: 654d4ad1872f1468265ac30de77d055fcd63abde
Author: Unai Martirena <unai.martirena <at> openbravo.com>
Date: Tue Jun 02 09:12:55 2015 +0200
URL: http://code.openbravo.com/erp/devel/pi/rev/654d4ad1872f1468265ac30de77d055fcd63abde [^]

Fixes bug 29708, Fixes bug 29936: Improve performance and memory problems

2 Problems are happening in Costing Background process:

1) Java Heap error: This problem happens because getTransactions method can load on memory a lot of objects and even an ScrollableResult is being used the following sentence 'OBDal.getInstance().getConnection().setHoldability(ResultSet.HOLD_CURSORS_OVER_COMMIT)' prevents in fact to scroll properly the ScrollableResult. To avoid this issue the query of getTransactions method has been changed to return only 1000 records. So, instead of calling just once this method it is done once each 1000 records.

Also instead of returning a list of 1000 MaterialTransaction objects, that can be a relatively big amount of objects to load in memory, only the Ids are returned and each id the it is instanced on each iteration.

2) Performance problems: The process has a performance issue because it does a commit on each iteration of the list. Now, this has been changed to do commit every 1000 records, and in case that an exception is raised on a certain iteration a new method has been added in catch clause to calculate the cost of the transactions that have been rolledback but which there were properly calculated.

---
M src/org/openbravo/costing/CostingBackground.java
---
(0078096)
dmiguelez   
2015-06-04 18:04   
Code Review + Testing Ok
(0078127)
hudsonbot   
2015-06-06 16:53   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/7419f65cdeb3 [^]
Maturity status: Test