Openbravo Issue Tracking System - Openbravo ERP
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0031580Openbravo ERPZ. Otherspublic2015-11-25 13:422016-03-17 10:55
Reportermaite 
Assigned Tovmromanos 
PriorityimmediateSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in Version 
Merge Request Status
Review Assigned Tovmromanos
OBNetwork customer
Web browser
ModulesCore
Support ticket
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary0031580: Java Heap Space error when posting payment with huge amount of details
DescriptionJava Heap Space error when posting payment with huge amount of details
Steps To ReproduceIn case of having Payment documents with huge number of lines (<= 101.698 lines), java heap space error is thrown when posting the document

NOTE: I can provide environment where problem is reproducible
Proposed SolutionProblem seems to be in "loadLinesFieldProvider" method because huge list of FIN_PaymentDetail objects is being loaded in following line:

List<FIN_PaymentDetail> paymentDetails = FIN_Utility.getOrderedPaymentDetailList(payment);

Proposed solution will be not to load this info as FIN_PaymentDetail objects but just handle it as ID (and after that instance objects when required)
Additional Information
TagsNo tags attached.
Relationships
causes defect 0032203 closed vmromanos Not possible to post a remittance as well as the settlement of that remittance. 
causes defect 0033508 closed markmm82 Cannot post a Cash VAT payment that pays more than one invoice, Process failed during execution error is shown 
causes defect 0033688 closed rqueralta Cannot post a financial account transaction if the invoice is partially paid by using credit 
Attached Filesdiff 31580.diff (71,897) 2015-12-29 10:55
https://issues.openbravo.com/file_download.php?file_id=8872&type=bug
diff 31580_cr.diff (2,008) 2015-12-29 11:34
https://issues.openbravo.com/file_download.php?file_id=8873&type=bug
png with_last_fix_after_reopening_issue_post_payment_and_transaction.png (36,579) 2016-01-21 17:07
https://issues.openbravo.com/file_download.php?file_id=8958&type=bug
png
Issue History
Date ModifiedUsernameFieldChange
2015-11-25 13:42maiteNew Issue
2015-11-25 13:42maiteAssigned To => Triage Finance
2015-11-25 13:42maiteModules => Core
2015-11-25 13:42maiteResolution time => 1448578800
2015-11-25 13:42maiteTriggers an Emergency Pack => No
2015-11-25 13:43maiteIssue Monitored: networkb
2015-11-27 12:35aferrazAssigned ToTriage Finance => aferraz
2015-11-27 12:36aferrazStatusnew => scheduled
2015-12-29 10:55vmromanosFile Added: 31580.diff
2015-12-29 11:34vmromanosFile Added: 31580_cr.diff
2015-12-30 11:34vmromanosTarget Version3.0PR16Q1 =>
2015-12-31 16:38vmromanosNote Added: 0083117
2016-01-11 19:05hgbotCheckin
2016-01-11 19:05hgbotNote Added: 0083202
2016-01-11 19:05hgbotStatusscheduled => resolved
2016-01-11 19:05hgbotResolutionopen => fixed
2016-01-11 19:05hgbotFixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/217fb3a426febf71995fa1394dac159afa49b44f [^]
2016-01-11 19:05hgbotCheckin
2016-01-11 19:05hgbotNote Added: 0083203
2016-01-11 19:05hgbotFixed in SCM revisionhttp://code.openbravo.com/erp/devel/pi/rev/217fb3a426febf71995fa1394dac159afa49b44f [^] => http://code.openbravo.com/erp/devel/pi/rev/1817418c55aff648d979a7ca7c3d56a90ecab721 [^]
2016-01-11 19:05vmromanosReview Assigned To => vmromanos
2016-01-11 19:05vmromanosNote Added: 0083204
2016-01-11 19:05vmromanosStatusresolved => closed
2016-01-19 14:00maiteNote Added: 0083398
2016-01-19 14:00maiteStatusclosed => new
2016-01-19 14:00maiteResolutionfixed => open
2016-01-20 13:25vmromanosAssigned Toaferraz => vmromanos
2016-01-20 13:25vmromanosStatusnew => acknowledged
2016-01-21 10:19vmromanosStatusacknowledged => scheduled
2016-01-21 17:07vmromanosFile Added: with_last_fix_after_reopening_issue_post_payment_and_transaction.png
2016-01-25 09:15hgbotCheckin
2016-01-25 09:15hgbotNote Added: 0083575
2016-01-25 09:16vmromanosStatusscheduled => resolved
2016-01-25 09:16vmromanosFixed in SCM revisionhttp://code.openbravo.com/erp/devel/pi/rev/1817418c55aff648d979a7ca7c3d56a90ecab721 [^] => http://code.openbravo.com/erp/devel/pi/rev/7dafd0844a3dcfa21a67cca627e283e2ba7ba76b [^]
2016-01-25 09:16vmromanosResolutionopen => fixed
2016-01-25 09:19vmromanosNote Added: 0083576
2016-01-25 09:19vmromanosStatusresolved => closed
2016-02-11 13:07vmromanosRelationship addedcauses 0032203
2016-03-17 10:53hudsonbotCheckin
2016-03-17 10:53hudsonbotNote Added: 0085028
2016-03-17 10:53hudsonbotCheckin
2016-03-17 10:53hudsonbotNote Added: 0085029
2016-03-17 10:55hudsonbotCheckin
2016-03-17 10:55hudsonbotNote Added: 0085077
2016-07-22 14:26aferrazRelationship addedcauses 0033508
2016-08-12 11:41aferrazRelationship addedcauses 0033688

Notes
(0083117)
vmromanos   
2015-12-31 16:38   
Patches provided to be tested by reporter in customer's environment
(0083202)
hgbot   
2016-01-11 19:05   
Repository: erp/devel/pi
Changeset: 217fb3a426febf71995fa1394dac159afa49b44f
Author: Alvaro Ferraz <alvaro.ferraz <at> openbravo.com>
Date: Tue Dec 01 16:44:30 2015 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/217fb3a426febf71995fa1394dac159afa49b44f [^]

Fixes issue 31580: Java Heap Space error when posting payment with many details

Add a new FIN_Utility.getOrderedPaymentDetailList method and deprecate the old one, to return a List<String> instead of a List<FIN_PaymentDetail> and avoid loading all the entities in memory.
Adapt loadLines methods in DocFINFinAccTransaction, DocFINPayment and DocFINReconciliation to the new FIN_Utility.getOrderedPaymentDetailList method.
Create a new getPaymentDetailWriteOffAndAmount method in AcctServer to adapt it to new parameters.

---
M modules/org.openbravo.advpaymentmngt/src/org/openbravo/advpaymentmngt/utility/FIN_Utility.java
M src/org/openbravo/erpCommon/ad_forms/AcctServer.java
M src/org/openbravo/erpCommon/ad_forms/DocFINFinAccTransaction.java
M src/org/openbravo/erpCommon/ad_forms/DocFINPayment.java
M src/org/openbravo/erpCommon/ad_forms/DocFINReconciliation.java
---
(0083203)
hgbot   
2016-01-11 19:05   
Repository: erp/devel/pi
Changeset: 1817418c55aff648d979a7ca7c3d56a90ecab721
Author: Víctor Martínez Romanos <victor.martinez <at> openbravo.com>
Date: Tue Dec 29 11:32:34 2015 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/1817418c55aff648d979a7ca7c3d56a90ecab721 [^]

Fixed bug 31580: code review improvements
Properly passing parameter to the HQL query
Force to check client+org when using admin mode

---
M modules/org.openbravo.advpaymentmngt/src/org/openbravo/advpaymentmngt/utility/FIN_Utility.java
---
(0083204)
vmromanos   
2016-01-11 19:05   
Code review + testing OK

Pending to be confirmed by customer
(0083398)
maite   
2016-01-19 14:00   
I still obtain Java Heap Space error with changes applied. Error is obtained after 3 hours of execution for a single FIN_Payment document and using following tomcat configuration: -Xms2048M -Xmx2048M -XX:MaxPermSize=256M

I can provide environment where problem is still reproducible
(0083575)
hgbot   
2016-01-25 09:15   
Repository: erp/devel/pi
Changeset: 7dafd0844a3dcfa21a67cca627e283e2ba7ba76b
Author: Víctor Martínez Romanos <victor.martinez <at> openbravo.com>
Date: Thu Jan 21 16:52:52 2016 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/7dafd0844a3dcfa21a67cca627e283e2ba7ba76b [^]

Related to issue 31580: Reduced Java Heap Usage even more

DocFINFinAccTransaction and DocFINPayment:
+ Avoid storing useless objects like ps and pd, and replaced by String Ids, so we can safely clear the Hibernate session
+ Save cInvoiceId when loading lines in field provider so it's available latter on when loading the document lines, thus we avoid an extra query to the database to get the invoice from the payment detail
+ Forces a session clear every 100 records when working with the payment details. This is only possible because DAL objects are not stored anywhere.

DocLine_FINFinAccTransaction and DocLine_FINPayment:
+ Declare a String invoiceId parameter instead of using an Invoice object. This improves the memory consumption and allows to manage external session clears when required.
+ Declare as deprecated all the methods that used the invoice object and adapt the getInvoice() to be backward compatible (returns the invoice object is previously declared in the object, or a new object based on the invoiceId)
+ For DocLine_FINPayment only, adapted the Order object usage following the same logic.

FIN_Utility.getOrderedPaymentDetailList() query uses an inner join to the FPSD because having a linked FPSD is a requirement in the accounting classes.


Note that DocFINReconciliation class should be also adapted following the same logics described above. However, this class would require deeper refactor mainly because of the attribute Set<FIN_FinaccTransaction> transactionsToBePosted, that avoids safely session clears.

---
M modules/org.openbravo.advpaymentmngt/src/org/openbravo/advpaymentmngt/utility/FIN_Utility.java
M src/org/openbravo/erpCommon/ad_forms/DocFINFinAccTransaction.java
M src/org/openbravo/erpCommon/ad_forms/DocFINPayment.java
M src/org/openbravo/erpCommon/ad_forms/DocLine_FINFinAccTransaction.java
M src/org/openbravo/erpCommon/ad_forms/DocLine_FINPayment.java
---
(0083576)
vmromanos   
2016-01-25 09:19   
With the last changeset the memory consumption is reduced around 1.5GB.
See attached with_last_fix_after_reopening_issue_post_payment_and_transaction.png for details
(0085028)
hudsonbot   
2016-03-17 10:53   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/b22fb0500156 [^]
Maturity status: Test
(0085029)
hudsonbot   
2016-03-17 10:53   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/b22fb0500156 [^]
Maturity status: Test
(0085077)
hudsonbot   
2016-03-17 10:55   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/b22fb0500156 [^]
Maturity status: Test