Anonymous | Login
Project:
RSS
  
News | My View | View Issues | Roadmap | Summary

View Issue Details[ Jump to Notes ] [ Issue History ] [ Print ]
ID
0036067
TypeCategorySeverityReproducibilityDate SubmittedLast Update
design defect[Openbravo ERP] Z. Othersmajorhave not tried2017-05-23 08:332017-09-22 15:14
ReporteralostaleView Statuspublic 
Assigned Tomarkmm82 
PrioritynormalResolutionfixedFixed in Version3.0PR17Q4
StatusclosedFix in branchFixed in SCM revision2ab8016a2353
ProjectionnoneETAnoneTarget Version
OSAnyDatabaseAnyJava version
OS VersionDatabase versionAnt version
Product VersionSCM revision 
Review Assigned Toaferraz
Web browser
ModulesCore
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary

0036067: old callouts should be refactored to implement SimpleCallout

DescriptionCallouts that do not extend SimpleCallout are defined as Servlets (as this is how 2.50 technology worked), they executed as follows:

1. A fake request is created to invoke them as a Servlet
2. That request is used to write the response in as a String representing JavaSript code
3. That string is evaluated, parsed and executed as JavaScript in server
4. To finally set the values it requires

In addition to all the overhead involved in each of the steps described above, step 3 generates bytecodes from JavaScript that are stored in PermGen memory space. This causes excessive PermGen consumption as well as it trigger Major GC Collections even with a low heap use.

All these steps are avoided for SimpleCallout since PR16Q4 0032366
Steps To ReproduceTo emulate it:

1. Apply attached diff that executes step 3 defined in description 2K times per callout
2. Limit PermGen (ie. 160MB) and monitor JVM
3. Open Sales Order window and click on new (this executes SL_Order_DocType and SL_Order_PriceList callouts)
  -> Check JVM graphs during execution (see attached image):
      * PermGen rapidly increases
      * This is caused to a big number of classes loaded
      * Finally, major GC collection cycles are triggered

To detect wrong callouts, apply detect-callouts.diff and execute ClassLoaderTest.

Attached the list of the 94 cases obtained in current pi.

Proposed SolutionAll callouts should be refactored to implement SimpleCallout, this is specially important for those ones that are often executed.
TagsPerformance
Attached Filesdiff file icon fic-execute-many-callouts.diff [^] (1,175 bytes) 2017-05-23 08:34 [Show Content]
diff file icon detect-callouts.diff [^] (2,134 bytes) 2017-05-23 08:34 [Show Content]
png file icon Selection_186.png [^] (23,493 bytes) 2017-05-23 08:34


? file icon incorrect-callouts [^] (13,396 bytes) 2017-05-23 08:38 [Show Content]

- Relationships [ Relation Graph ] [ Dependency Graph ]
related to design defect 00323663.0PR16Q4 closedinigosanchez callout invocation ineficiencies 
related to defect 0036192 closedplatform Not possible to overwrite the result of a callout from your own callout using a requestdispatcher an a decorator in Tomcat 7 
related to defect 0036559 closedalostale Old callouts should be refactored to implement SimpleCallout 
related to defect 0036280 closedAtulOpenbravo Several unused + probably unusable callouts exists and are candidates to be deleted 
depends on defect 0036617 closedaferraz API change: functional callouts now implement SimpleCallout 
related to defect 0035635 closedAtulOpenbravo Performance should be improved when select a business partner in a new sales invoice 
related to defect 0036228 closedalostale SimpleCallout used as ConnectionProvider borrows 2 connections in same thread 
related to defect 0036302 closedplatform Display logic in fields is not working in callouts using SimpleCallout 
related to defect 0036392 closedcollazoandy4 Avoid casting to Object when adding numeric results from a SimpleCallout 
related to defect 0036621 newTriage Omni OMS It is possible to use UOM conversions of other clients in some callouts. 
related to defect 0036590 closedmarkmm82 Callouts retrieve wrong quantity when Format.xml file is set with too many decimals 
related to design defect 0040535 closedalostale old callouts should use OBScriptEngine to evaluate response 
related to design defect 0044795 newTriage Omni OMS Tracking issue to refactor remaining old callouts to extend SimpleCallout 
causes defect 0037676 closeddmiguelez "Project Phase" field does not show any value (although Project is selected) in Expense Sheet window 
causes defect 0038639 closedAtulOpenbravo Debit and credit amounts are wrongly updated in G/L Journal lines with different currency than general ledger currency 
causes defect 0039608 closedAtulOpenbravo Requisition window - columns are not displayed when selecting the product 

-  Notes
(0097323)
AtulOpenbravo (developer)
2017-06-12 16:23

 Test Plan SE_Invoice_Project

- Login as F&B Internation Group Admin
- Create a Payment Term with Search Key = "36067", Name = "Test 36067 Payment Term".
- Create a Payment Method with name = "Test 36067 Payment Method"
- Add Payment Method to Financial Account "Cuenta de Banco".
- Create a Service Project
    - Search Key: Test Project 36067
    - Name : Test Project 36067
    - Payment Method: Test 36067 Payment Method
    - Payment Term : Test 36067 Payment Term
- Create a Sales Invoice.
- Realize that Payment Term and Payment Method is not blank.
- Select Project in Dimension section as "Test Project 36067".
- Realize that Payment Term and Payment Method are not updated.
- Set Project as blank.
- Set Payment Term and Payment Method as blank.
- Reset project to "Test Project 36067".
- Realize that following are set
    - Payment Method: Test 36067 Payment Method
    - Payment Term : Test 36067 Payment Term

Above test plan does not apply to Purchase Invoice.
(0097744)
markmm82 (developer)
2017-06-28 21:13
edited on: 2017-06-28 21:21

 Test Plan - SL_JournalLineAmt


Test Plan 1 - G/L Journal Window -> Header Tab -> Lines Tab
1. Go to G/L Journal window and create a new one.
2. Go to Header tab and create a new one.
3. Go to Lines tab and create a new one. Define a Debit amount = 100.
Notice following amounts are updated:
    General Ledger Currency Debit = 100.
    Credit amount = 0.
    General Ledger Currency Credit = 0.
4. Now change the Credit amount and set to 500.
Notice following amounts are updated:
   General Ledger Currency Debit = 0.
   Debit amount = 0.
   General Ledger Currency Credit = 500.
5. Now change the Debit amount and set to 800.
Notice following amounts are updated:
   General Ledger Currency Debit = 800.
   Notice Credit amount = 0.
   General Ledger Currency Credit = 0.

Test Plan 2 - Simple G/L Journal -> Lines
1. Go to G/L Journal window and create a new one.
2. Go to Lines tab and create a new one. Define a Debit amount = 100.
Notice following amounts are updated:
    General Ledger Currency Debit = 100.
    Credit amount = 0.
    General Ledger Currency Credit = 0.
3. Now change the Credit amount and set to 500.
Notice following amounts are updated:
   General Ledger Currency Debit = 0.
   Debit amount = 0.
   General Ledger Currency Credit = 500.
4. Now change the Debit amount and set to 800.
Notice following amounts are updated:
   General Ledger Currency Debit = 800.
   Notice Credit amount = 0.
   General Ledger Currency Credit = 0.
(0098412)
hgbot (developer)
2017-08-08 08:34

 Repository: erp/devel/api-checks
Changeset: 9428f36c95c9418a98ef7bcfc5f1a753d93ef8b7
Author: Alvaro Ferraz <alvaro.ferraz <at> openbravo.com>
Date: Tue Aug 08 08:32:43 2017 +0200
URL: http://code.openbravo.com/erp/devel/api-checks/rev/9428f36c95c9418a98ef7bcfc5f1a753d93ef8b7 [^]

Fixes issue 36617: API Change for issue 36067

---
M java/reference/java.japi.gz
---
(0098414)
hgbot (developer)
2017-08-08 08:36

 Repository: erp/devel/pi
Changeset: 2ab8016a23535c8182170c2dc4f991c2dc5786a9
Author: Mark <markmm82 <at> gmail.com>
Date: Fri Aug 04 19:25:27 2017 -0400
URL: http://code.openbravo.com/erp/devel/pi/rev/2ab8016a23535c8182170c2dc4f991c2dc5786a9 [^]

Fixes issue 36067: Improve callouts performance extending from SimpleCallout

Refactorized some old callouts extending HttpSecureAppServlet to use SimpleCallout and improve performance.

---
M src/org/openbravo/erpCommon/ad_callouts/SE_Expense_Amount.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Expense_BP_Project.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Expense_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SE_ExternalPOS_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_FinancialAccount_BSAccounts.java
M src/org/openbravo/erpCommon/ad_callouts/SE_InOut_DocType.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Invoice_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Invoice_Project.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Order_Project.java
M src/org/openbravo/erpCommon/ad_callouts/SE_PeriodNo.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Period_Control.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Pricelist_Active.java
M src/org/openbravo/erpCommon/ad_callouts/SE_ProjectLine_Value.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Project_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Proposal_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Supplier_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Taxes_Active.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Asset_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Assets.java
M src/org/openbravo/erpCommon/ad_callouts/SL_BankDebt_Amount.java
M src/org/openbravo/erpCommon/ad_callouts/SL_BankStmt_Amount.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Budget_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_CCPMeasure_Group.java
M src/org/openbravo/erpCommon/ad_callouts/SL_CashJournal_Amounts.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Charge.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Conversion_Rate.java
M src/org/openbravo/erpCommon/ad_callouts/SL_CostcenterEmployee_SalaryCategory.java
M src/org/openbravo/erpCommon/ad_callouts/SL_GlobalUse_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_GlobalUse_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_InOutLine_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_InOut_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SL_InOut_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Internal_Consumption_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Internal_Consumption_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Inventory_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Inventory_Locator.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Inventory_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_InvoiceTax_Amt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Amt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Charge_Tax.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_DocType.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Glitem.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_PriceList.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_JournalLineAmt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Journal_Period.java
M src/org/openbravo/erpCommon/ad_callouts/SL_LegalEntity_Acct.java
M src/org/openbravo/erpCommon/ad_callouts/SL_MachineCost.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Movement_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Movement_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Amt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Charge_Tax.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_DocType.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_PriceList.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Tax.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Organization_Info.java
M src/org/openbravo/erpCommon/ad_callouts/SL_PC_Case_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_ProductPrice_PriceListVersion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Product_Type.java
M src/org/openbravo/erpCommon/ad_callouts/SL_ProductionPlan_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_ProductionPlan_WRPhase.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Production_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Production_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Project_Margin.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Project_Planned.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Project_PriceList.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Project_Type.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Proposal_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_RequisitionLine_Amt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_RequisitionLine_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_RequisitionLine_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_RequisitionOrder_Order.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Requisition_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Requisition_BPartner_PriceList.java
M src/org/openbravo/erpCommon/ad_callouts/SL_ScheduledMaintenance_Maintenance.java
M src/org/openbravo/erpCommon/ad_callouts/SL_SequenceProduct_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_SequenceProduct_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_User_Name.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WRPhaseProduct_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WRPhase_Sequence.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WorkRequirement_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WorkRequirement_ProcessPlan.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WorkRequirement_StartDate.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Year_Validation.java
---
(0098416)
aferraz (manager)
2017-08-08 09:34

 Code review OK

https://docs.google.com/spreadsheets/d/1rW7KRBcHMWOKx5fBkcRVgWNihXVYNGFpZevd6IE41LE/edit?usp=sharing [^]
(0099299)
hudsonbot (developer)
2017-09-21 16:49

 A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/9750b78d3e5c [^]
Maturity status: Test

- Issue History
Date Modified Username Field Change
2017-05-23 08:33 alostale New Issue
2017-05-23 08:33 alostale Assigned To => Triage Finance
2017-05-23 08:33 alostale Modules => Core
2017-05-23 08:33 alostale Triggers an Emergency Pack => No
2017-05-23 08:34 alostale File Added: fic-execute-many-callouts.diff
2017-05-23 08:34 alostale File Added: detect-callouts.diff
2017-05-23 08:34 alostale File Added: Selection_186.png
2017-05-23 08:35 alostale Steps to Reproduce Updated View Revisions
2017-05-23 08:35 alostale Tag Attached: Performance
2017-05-23 08:37 alostale Steps to Reproduce Updated View Revisions
2017-05-23 08:38 alostale File Added: incorrect-callouts
2017-05-23 08:39 alostale Steps to Reproduce Updated View Revisions
2017-05-29 13:20 shuehner Relationship added related to 0032366
2017-05-29 13:21 shuehner Issue Monitored: shuehner
2017-06-07 10:47 AtulOpenbravo Status new => scheduled
2017-06-07 10:47 AtulOpenbravo Assigned To Triage Finance => AtulOpenbravo
2017-06-08 09:49 alostale Relationship added related to 0036192
2017-06-09 14:17 aferraz Relationship added related to 0035635
2017-06-12 13:54 alostale Relationship added related to 0036228
2017-06-12 16:23 AtulOpenbravo Note Added: 0097323
2017-06-21 11:02 aferraz Relationship added related to 0036302
2017-06-28 21:13 markmm82 Note Added: 0097744
2017-06-28 21:21 markmm82 Note Edited: 0097744 View Revisions
2017-07-10 13:03 aferraz Relationship added related to 0036392
2017-08-01 09:03 aferraz Relationship added related to 0036559
2017-08-07 09:46 aferraz Relationship added depends on 0036617
2017-08-07 15:48 markmm82 Relationship added blocks 0036621
2017-08-07 15:52 markmm82 Relationship deleted blocks 0036621
2017-08-07 15:52 markmm82 Relationship added related to 0036621
2017-08-08 08:31 aferraz Assigned To AtulOpenbravo => markmm82
2017-08-08 08:34 hgbot Checkin
2017-08-08 08:34 hgbot Note Added: 0098412
2017-08-08 08:36 hgbot Checkin
2017-08-08 08:36 hgbot Note Added: 0098414
2017-08-08 08:36 hgbot Status scheduled => resolved
2017-08-08 08:36 hgbot Resolution open => fixed
2017-08-08 08:36 hgbot Fixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/2ab8016a23535c8182170c2dc4f991c2dc5786a9 [^]
2017-08-08 09:34 aferraz Review Assigned To => aferraz
2017-08-08 09:34 aferraz Note Added: 0098416
2017-08-08 09:34 aferraz Status resolved => closed
2017-08-08 09:34 aferraz Fixed in Version => 3.0PR17Q4
2017-08-18 12:57 inigosanchez Relationship added related to 0036590
2017-09-21 16:49 hudsonbot Checkin
2017-09-21 16:49 hudsonbot Note Added: 0099299
2017-09-22 15:14 aferraz Relationship added related to 0036280
2018-01-24 16:55 dmiguelez Relationship added causes 0037676
2018-05-28 11:24 ngarcia Relationship added causes 0038639
2018-11-12 09:56 dmitry_mezentsev Relationship added causes 0039608
2019-04-05 10:14 alostale Relationship added related to 0040535
2020-08-12 16:54 shuehner Relationship added related to 0044795

Copyright © 2000 - 2009 MantisBT Group
Powered by Mantis Bugtracker