Anonymous | Login
Project:
RSS
  
News | My View | View Issues | Roadmap | Summary

View Issue Details[ Jump to Notes ] [ Issue History ] [ Print ]
ID
0031580
TypeCategorySeverityReproducibilityDate SubmittedLast Update
defect[Openbravo ERP] Z. Othersmajoralways2015-11-25 13:422016-03-17 10:55
ReportermaiteView Statuspublic 
Assigned Tovmromanos 
PriorityimmediateResolutionfixedFixed in Version
StatusclosedFix in branchFixed in SCM revision7dafd0844a3d
ProjectionnoneETAnoneTarget Version
OSAnyDatabaseAnyJava version
OS VersionDatabase versionAnt version
Product VersionSCM revision 
Merge Request Status
Review Assigned Tovmromanos
OBNetwork customerOBPS
Web browser
ModulesCore
Support ticket39391
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary

0031580: Java Heap Space error when posting payment with huge amount of details

DescriptionJava Heap Space error when posting payment with huge amount of details
Steps To ReproduceIn case of having Payment documents with huge number of lines (<= 101.698 lines), java heap space error is thrown when posting the document

NOTE: I can provide environment where problem is reproducible
Proposed SolutionProblem seems to be in "loadLinesFieldProvider" method because huge list of FIN_PaymentDetail objects is being loaded in following line:

List<FIN_PaymentDetail> paymentDetails = FIN_Utility.getOrderedPaymentDetailList(payment);

Proposed solution will be not to load this info as FIN_PaymentDetail objects but just handle it as ID (and after that instance objects when required)
TagsNo tags attached.
Attached Filesdiff file icon 31580.diff [^] (71,897 bytes) 2015-12-29 10:55 [Show Content]
diff file icon 31580_cr.diff [^] (2,008 bytes) 2015-12-29 11:34 [Show Content]
png file icon with_last_fix_after_reopening_issue_post_payment_and_transaction.png [^] (36,579 bytes) 2016-01-21 17:07

- Relationships [ Relation Graph ] [ Dependency Graph ]
causes defect 0032203 closedvmromanos Not possible to post a remittance as well as the settlement of that remittance. 
causes defect 0033508 closedmarkmm82 Cannot post a Cash VAT payment that pays more than one invoice, Process failed during execution error is shown 
causes defect 0033688 closedrqueralta Cannot post a financial account transaction if the invoice is partially paid by using credit 

-  Notes
(0083117)
vmromanos (viewer)
2015-12-31 16:38

 Patches provided to be tested by reporter in customer's environment
(0083202)
hgbot (developer)
2016-01-11 19:05

 Repository: erp/devel/pi
Changeset: 217fb3a426febf71995fa1394dac159afa49b44f
Author: Alvaro Ferraz <alvaro.ferraz <at> openbravo.com>
Date: Tue Dec 01 16:44:30 2015 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/217fb3a426febf71995fa1394dac159afa49b44f [^]

Fixes issue 31580: Java Heap Space error when posting payment with many details

Add a new FIN_Utility.getOrderedPaymentDetailList method and deprecate the old one, to return a List<String> instead of a List<FIN_PaymentDetail> and avoid loading all the entities in memory.
Adapt loadLines methods in DocFINFinAccTransaction, DocFINPayment and DocFINReconciliation to the new FIN_Utility.getOrderedPaymentDetailList method.
Create a new getPaymentDetailWriteOffAndAmount method in AcctServer to adapt it to new parameters.

---
M modules/org.openbravo.advpaymentmngt/src/org/openbravo/advpaymentmngt/utility/FIN_Utility.java
M src/org/openbravo/erpCommon/ad_forms/AcctServer.java
M src/org/openbravo/erpCommon/ad_forms/DocFINFinAccTransaction.java
M src/org/openbravo/erpCommon/ad_forms/DocFINPayment.java
M src/org/openbravo/erpCommon/ad_forms/DocFINReconciliation.java
---
(0083203)
hgbot (developer)
2016-01-11 19:05

 Repository: erp/devel/pi
Changeset: 1817418c55aff648d979a7ca7c3d56a90ecab721
Author: Víctor Martínez Romanos <victor.martinez <at> openbravo.com>
Date: Tue Dec 29 11:32:34 2015 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/1817418c55aff648d979a7ca7c3d56a90ecab721 [^]

Fixed bug 31580: code review improvements
Properly passing parameter to the HQL query
Force to check client+org when using admin mode

---
M modules/org.openbravo.advpaymentmngt/src/org/openbravo/advpaymentmngt/utility/FIN_Utility.java
---
(0083204)
vmromanos (viewer)
2016-01-11 19:05

 Code review + testing OK

Pending to be confirmed by customer
(0083398)
maite (viewer)
2016-01-19 14:00

 I still obtain Java Heap Space error with changes applied. Error is obtained after 3 hours of execution for a single FIN_Payment document and using following tomcat configuration: -Xms2048M -Xmx2048M -XX:MaxPermSize=256M

I can provide environment where problem is still reproducible
(0083575)
hgbot (developer)
2016-01-25 09:15

 Repository: erp/devel/pi
Changeset: 7dafd0844a3dcfa21a67cca627e283e2ba7ba76b
Author: Víctor Martínez Romanos <victor.martinez <at> openbravo.com>
Date: Thu Jan 21 16:52:52 2016 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/7dafd0844a3dcfa21a67cca627e283e2ba7ba76b [^]

Related to issue 31580: Reduced Java Heap Usage even more

DocFINFinAccTransaction and DocFINPayment:
+ Avoid storing useless objects like ps and pd, and replaced by String Ids, so we can safely clear the Hibernate session
+ Save cInvoiceId when loading lines in field provider so it's available latter on when loading the document lines, thus we avoid an extra query to the database to get the invoice from the payment detail
+ Forces a session clear every 100 records when working with the payment details. This is only possible because DAL objects are not stored anywhere.

DocLine_FINFinAccTransaction and DocLine_FINPayment:
+ Declare a String invoiceId parameter instead of using an Invoice object. This improves the memory consumption and allows to manage external session clears when required.
+ Declare as deprecated all the methods that used the invoice object and adapt the getInvoice() to be backward compatible (returns the invoice object is previously declared in the object, or a new object based on the invoiceId)
+ For DocLine_FINPayment only, adapted the Order object usage following the same logic.

FIN_Utility.getOrderedPaymentDetailList() query uses an inner join to the FPSD because having a linked FPSD is a requirement in the accounting classes.


Note that DocFINReconciliation class should be also adapted following the same logics described above. However, this class would require deeper refactor mainly because of the attribute Set<FIN_FinaccTransaction> transactionsToBePosted, that avoids safely session clears.

---
M modules/org.openbravo.advpaymentmngt/src/org/openbravo/advpaymentmngt/utility/FIN_Utility.java
M src/org/openbravo/erpCommon/ad_forms/DocFINFinAccTransaction.java
M src/org/openbravo/erpCommon/ad_forms/DocFINPayment.java
M src/org/openbravo/erpCommon/ad_forms/DocLine_FINFinAccTransaction.java
M src/org/openbravo/erpCommon/ad_forms/DocLine_FINPayment.java
---
(0083576)
vmromanos (viewer)
2016-01-25 09:19

 With the last changeset the memory consumption is reduced around 1.5GB.
See attached with_last_fix_after_reopening_issue_post_payment_and_transaction.png for details
(0085028)
hudsonbot (viewer)
2016-03-17 10:53

 A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/b22fb0500156 [^]
Maturity status: Test
(0085029)
hudsonbot (viewer)
2016-03-17 10:53

 A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/b22fb0500156 [^]
Maturity status: Test
(0085077)
hudsonbot (viewer)
2016-03-17 10:55

 A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/b22fb0500156 [^]
Maturity status: Test

- Issue History
Date Modified Username Field Change
2015-11-25 13:42 maite New Issue
2015-11-25 13:42 maite Assigned To => Triage Finance
2015-11-25 13:42 maite OBNetwork customer => Yes
2015-11-25 13:42 maite Modules => Core
2015-11-25 13:42 maite Support ticket => 39391
2015-11-25 13:42 maite Resolution time => 1448578800
2015-11-25 13:42 maite Triggers an Emergency Pack => No
2015-11-25 13:43 maite Issue Monitored: networkb
2015-11-27 12:35 aferraz Assigned To Triage Finance => aferraz
2015-11-27 12:36 aferraz Status new => scheduled
2015-12-29 10:55 vmromanos File Added: 31580.diff
2015-12-29 11:34 vmromanos File Added: 31580_cr.diff
2015-12-30 11:34 vmromanos Target Version 3.0PR16Q1 =>
2015-12-31 16:38 vmromanos Note Added: 0083117
2016-01-11 19:05 hgbot Checkin
2016-01-11 19:05 hgbot Note Added: 0083202
2016-01-11 19:05 hgbot Status scheduled => resolved
2016-01-11 19:05 hgbot Resolution open => fixed
2016-01-11 19:05 hgbot Fixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/217fb3a426febf71995fa1394dac159afa49b44f [^]
2016-01-11 19:05 hgbot Checkin
2016-01-11 19:05 hgbot Note Added: 0083203
2016-01-11 19:05 hgbot Fixed in SCM revision http://code.openbravo.com/erp/devel/pi/rev/217fb3a426febf71995fa1394dac159afa49b44f [^] => http://code.openbravo.com/erp/devel/pi/rev/1817418c55aff648d979a7ca7c3d56a90ecab721 [^]
2016-01-11 19:05 vmromanos Review Assigned To => vmromanos
2016-01-11 19:05 vmromanos Note Added: 0083204
2016-01-11 19:05 vmromanos Status resolved => closed
2016-01-19 14:00 maite Note Added: 0083398
2016-01-19 14:00 maite Status closed => new
2016-01-19 14:00 maite Resolution fixed => open
2016-01-20 13:25 vmromanos Assigned To aferraz => vmromanos
2016-01-20 13:25 vmromanos Status new => acknowledged
2016-01-21 10:19 vmromanos Status acknowledged => scheduled
2016-01-21 17:07 vmromanos File Added: with_last_fix_after_reopening_issue_post_payment_and_transaction.png
2016-01-25 09:15 hgbot Checkin
2016-01-25 09:15 hgbot Note Added: 0083575
2016-01-25 09:16 vmromanos Status scheduled => resolved
2016-01-25 09:16 vmromanos Fixed in SCM revision http://code.openbravo.com/erp/devel/pi/rev/1817418c55aff648d979a7ca7c3d56a90ecab721 [^] => http://code.openbravo.com/erp/devel/pi/rev/7dafd0844a3dcfa21a67cca627e283e2ba7ba76b [^]
2016-01-25 09:16 vmromanos Resolution open => fixed
2016-01-25 09:19 vmromanos Note Added: 0083576
2016-01-25 09:19 vmromanos Status resolved => closed
2016-02-11 13:07 vmromanos Relationship added causes 0032203
2016-03-17 10:53 hudsonbot Checkin
2016-03-17 10:53 hudsonbot Note Added: 0085028
2016-03-17 10:53 hudsonbot Checkin
2016-03-17 10:53 hudsonbot Note Added: 0085029
2016-03-17 10:55 hudsonbot Checkin
2016-03-17 10:55 hudsonbot Note Added: 0085077
2016-07-22 14:26 aferraz Relationship added causes 0033508
2016-08-12 11:41 aferraz Relationship added causes 0033688

Copyright © 2000 - 2009 MantisBT Group
Powered by Mantis Bugtracker