Openbravo Issue Tracking System - Openbravo ERP
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0036067Openbravo ERPZ. Otherspublic2017-05-23 08:332017-09-22 15:14
Reporteralostale 
Assigned Tomarkmm82 
PrioritynormalSeveritymajorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in Version3.0PR17Q4 
Merge Request Status
Review Assigned Toaferraz
OBNetwork customer
Web browser
ModulesCore
Support ticket
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary0036067: old callouts should be refactored to implement SimpleCallout
DescriptionCallouts that do not extend SimpleCallout are defined as Servlets (as this is how 2.50 technology worked), they executed as follows:

1. A fake request is created to invoke them as a Servlet
2. That request is used to write the response in as a String representing JavaSript code
3. That string is evaluated, parsed and executed as JavaScript in server
4. To finally set the values it requires

In addition to all the overhead involved in each of the steps described above, step 3 generates bytecodes from JavaScript that are stored in PermGen memory space. This causes excessive PermGen consumption as well as it trigger Major GC Collections even with a low heap use.

All these steps are avoided for SimpleCallout since PR16Q4 0032366
Steps To ReproduceTo emulate it:

1. Apply attached diff that executes step 3 defined in description 2K times per callout
2. Limit PermGen (ie. 160MB) and monitor JVM
3. Open Sales Order window and click on new (this executes SL_Order_DocType and SL_Order_PriceList callouts)
  -> Check JVM graphs during execution (see attached image):
      * PermGen rapidly increases
      * This is caused to a big number of classes loaded
      * Finally, major GC collection cycles are triggered

To detect wrong callouts, apply detect-callouts.diff and execute ClassLoaderTest.

Attached the list of the 94 cases obtained in current pi.

Proposed SolutionAll callouts should be refactored to implement SimpleCallout, this is specially important for those ones that are often executed.
Additional Information
TagsPerformance
Relationships
related to design defect 00323663.0PR16Q4 closed inigosanchez callout invocation ineficiencies 
related to defect 0036192 closed platform Not possible to overwrite the result of a callout from your own callout using a requestdispatcher an a decorator in Tomcat 7 
related to defect 0036559 closed alostale Old callouts should be refactored to implement SimpleCallout 
related to defect 0036280 closed AtulOpenbravo Several unused + probably unusable callouts exists and are candidates to be deleted 
depends on defect 0036617 closed aferraz API change: functional callouts now implement SimpleCallout 
related to defect 0035635 closed AtulOpenbravo Performance should be improved when select a business partner in a new sales invoice 
related to defect 0036228 closed alostale SimpleCallout used as ConnectionProvider borrows 2 connections in same thread 
related to defect 0036302 closed platform Display logic in fields is not working in callouts using SimpleCallout 
related to defect 0036392 closed collazoandy4 Avoid casting to Object when adding numeric results from a SimpleCallout 
related to defect 0036621 new Triage Omni OMS It is possible to use UOM conversions of other clients in some callouts. 
related to defect 0036590 closed markmm82 Callouts retrieve wrong quantity when Format.xml file is set with too many decimals 
related to design defect 0040535 closed alostale old callouts should use OBScriptEngine to evaluate response 
related to design defect 0044795 new Triage Omni OMS Tracking issue to refactor remaining old callouts to extend SimpleCallout 
causes defect 0037676 closed dmiguelez "Project Phase" field does not show any value (although Project is selected) in Expense Sheet window 
causes defect 0038639 closed AtulOpenbravo Debit and credit amounts are wrongly updated in G/L Journal lines with different currency than general ledger currency 
causes defect 0039608 closed AtulOpenbravo Requisition window - columns are not displayed when selecting the product 
Attached Filesdiff fic-execute-many-callouts.diff (1,175) 2017-05-23 08:34
https://issues.openbravo.com/file_download.php?file_id=10779&type=bug
diff detect-callouts.diff (2,134) 2017-05-23 08:34
https://issues.openbravo.com/file_download.php?file_id=10780&type=bug
png Selection_186.png (23,493) 2017-05-23 08:34
https://issues.openbravo.com/file_download.php?file_id=10781&type=bug
png

? incorrect-callouts (13,396) 2017-05-23 08:38
https://issues.openbravo.com/file_download.php?file_id=10782&type=bug
Issue History
Date ModifiedUsernameFieldChange
2017-05-23 08:33alostaleNew Issue
2017-05-23 08:33alostaleAssigned To => Triage Finance
2017-05-23 08:33alostaleModules => Core
2017-05-23 08:33alostaleTriggers an Emergency Pack => No
2017-05-23 08:34alostaleFile Added: fic-execute-many-callouts.diff
2017-05-23 08:34alostaleFile Added: detect-callouts.diff
2017-05-23 08:34alostaleFile Added: Selection_186.png
2017-05-23 08:35alostaleSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=15198#r15198
2017-05-23 08:35alostaleTag Attached: Performance
2017-05-23 08:37alostaleSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=15199#r15199
2017-05-23 08:38alostaleFile Added: incorrect-callouts
2017-05-23 08:39alostaleSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=15200#r15200
2017-05-29 13:20shuehnerRelationship addedrelated to 0032366
2017-05-29 13:21shuehnerIssue Monitored: shuehner
2017-06-07 10:47AtulOpenbravoStatusnew => scheduled
2017-06-07 10:47AtulOpenbravoAssigned ToTriage Finance => AtulOpenbravo
2017-06-08 09:49alostaleRelationship addedrelated to 0036192
2017-06-09 14:17aferrazRelationship addedrelated to 0035635
2017-06-12 13:54alostaleRelationship addedrelated to 0036228
2017-06-12 16:23AtulOpenbravoNote Added: 0097323
2017-06-21 11:02aferrazRelationship addedrelated to 0036302
2017-06-28 21:13markmm82Note Added: 0097744
2017-06-28 21:21markmm82Note Edited: 0097744bug_revision_view_page.php?bugnote_id=0097744#r15440
2017-07-10 13:03aferrazRelationship addedrelated to 0036392
2017-08-01 09:03aferrazRelationship addedrelated to 0036559
2017-08-07 09:46aferrazRelationship addeddepends on 0036617
2017-08-07 15:48markmm82Relationship addedblocks 0036621
2017-08-07 15:52markmm82Relationship deletedblocks 0036621
2017-08-07 15:52markmm82Relationship addedrelated to 0036621
2017-08-08 08:31aferrazAssigned ToAtulOpenbravo => markmm82
2017-08-08 08:34hgbotCheckin
2017-08-08 08:34hgbotNote Added: 0098412
2017-08-08 08:36hgbotCheckin
2017-08-08 08:36hgbotNote Added: 0098414
2017-08-08 08:36hgbotStatusscheduled => resolved
2017-08-08 08:36hgbotResolutionopen => fixed
2017-08-08 08:36hgbotFixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/2ab8016a23535c8182170c2dc4f991c2dc5786a9 [^]
2017-08-08 09:34aferrazReview Assigned To => aferraz
2017-08-08 09:34aferrazNote Added: 0098416
2017-08-08 09:34aferrazStatusresolved => closed
2017-08-08 09:34aferrazFixed in Version => 3.0PR17Q4
2017-08-18 12:57inigosanchezRelationship addedrelated to 0036590
2017-09-21 16:49hudsonbotCheckin
2017-09-21 16:49hudsonbotNote Added: 0099299
2017-09-22 15:14aferrazRelationship addedrelated to 0036280
2018-01-24 16:55dmiguelezRelationship addedcauses 0037676
2018-05-28 11:24ngarciaRelationship addedcauses 0038639
2018-11-12 09:56dmitry_mezentsevRelationship addedcauses 0039608
2019-04-05 10:14alostaleRelationship addedrelated to 0040535
2020-08-12 16:54shuehnerRelationship addedrelated to 0044795

Notes
(0097323)
AtulOpenbravo   
2017-06-12 16:23   
Test Plan SE_Invoice_Project

- Login as F&B Internation Group Admin
- Create a Payment Term with Search Key = "36067", Name = "Test 36067 Payment Term".
- Create a Payment Method with name = "Test 36067 Payment Method"
- Add Payment Method to Financial Account "Cuenta de Banco".
- Create a Service Project
    - Search Key: Test Project 36067
    - Name : Test Project 36067
    - Payment Method: Test 36067 Payment Method
    - Payment Term : Test 36067 Payment Term
- Create a Sales Invoice.
- Realize that Payment Term and Payment Method is not blank.
- Select Project in Dimension section as "Test Project 36067".
- Realize that Payment Term and Payment Method are not updated.
- Set Project as blank.
- Set Payment Term and Payment Method as blank.
- Reset project to "Test Project 36067".
- Realize that following are set
    - Payment Method: Test 36067 Payment Method
    - Payment Term : Test 36067 Payment Term

Above test plan does not apply to Purchase Invoice.
(0097744)
markmm82   
2017-06-28 21:13   
(edited on: 2017-06-28 21:21)
Test Plan - SL_JournalLineAmt


Test Plan 1 - G/L Journal Window -> Header Tab -> Lines Tab
1. Go to G/L Journal window and create a new one.
2. Go to Header tab and create a new one.
3. Go to Lines tab and create a new one. Define a Debit amount = 100.
Notice following amounts are updated:
    General Ledger Currency Debit = 100.
    Credit amount = 0.
    General Ledger Currency Credit = 0.
4. Now change the Credit amount and set to 500.
Notice following amounts are updated:
   General Ledger Currency Debit = 0.
   Debit amount = 0.
   General Ledger Currency Credit = 500.
5. Now change the Debit amount and set to 800.
Notice following amounts are updated:
   General Ledger Currency Debit = 800.
   Notice Credit amount = 0.
   General Ledger Currency Credit = 0.

Test Plan 2 - Simple G/L Journal -> Lines
1. Go to G/L Journal window and create a new one.
2. Go to Lines tab and create a new one. Define a Debit amount = 100.
Notice following amounts are updated:
    General Ledger Currency Debit = 100.
    Credit amount = 0.
    General Ledger Currency Credit = 0.
3. Now change the Credit amount and set to 500.
Notice following amounts are updated:
   General Ledger Currency Debit = 0.
   Debit amount = 0.
   General Ledger Currency Credit = 500.
4. Now change the Debit amount and set to 800.
Notice following amounts are updated:
   General Ledger Currency Debit = 800.
   Notice Credit amount = 0.
   General Ledger Currency Credit = 0.
(0098412)
hgbot   
2017-08-08 08:34   
Repository: erp/devel/api-checks
Changeset: 9428f36c95c9418a98ef7bcfc5f1a753d93ef8b7
Author: Alvaro Ferraz <alvaro.ferraz <at> openbravo.com>
Date: Tue Aug 08 08:32:43 2017 +0200
URL: http://code.openbravo.com/erp/devel/api-checks/rev/9428f36c95c9418a98ef7bcfc5f1a753d93ef8b7 [^]

Fixes issue 36617: API Change for issue 36067

---
M java/reference/java.japi.gz
---
(0098414)
hgbot   
2017-08-08 08:36   
Repository: erp/devel/pi
Changeset: 2ab8016a23535c8182170c2dc4f991c2dc5786a9
Author: Mark <markmm82 <at> gmail.com>
Date: Fri Aug 04 19:25:27 2017 -0400
URL: http://code.openbravo.com/erp/devel/pi/rev/2ab8016a23535c8182170c2dc4f991c2dc5786a9 [^]

Fixes issue 36067: Improve callouts performance extending from SimpleCallout

Refactorized some old callouts extending HttpSecureAppServlet to use SimpleCallout and improve performance.

---
M src/org/openbravo/erpCommon/ad_callouts/SE_Expense_Amount.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Expense_BP_Project.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Expense_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SE_ExternalPOS_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_FinancialAccount_BSAccounts.java
M src/org/openbravo/erpCommon/ad_callouts/SE_InOut_DocType.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Invoice_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Invoice_Project.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Order_Project.java
M src/org/openbravo/erpCommon/ad_callouts/SE_PeriodNo.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Period_Control.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Pricelist_Active.java
M src/org/openbravo/erpCommon/ad_callouts/SE_ProjectLine_Value.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Project_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Proposal_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Supplier_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SE_Taxes_Active.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Asset_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Assets.java
M src/org/openbravo/erpCommon/ad_callouts/SL_BankDebt_Amount.java
M src/org/openbravo/erpCommon/ad_callouts/SL_BankStmt_Amount.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Budget_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_CCPMeasure_Group.java
M src/org/openbravo/erpCommon/ad_callouts/SL_CashJournal_Amounts.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Charge.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Conversion_Rate.java
M src/org/openbravo/erpCommon/ad_callouts/SL_CostcenterEmployee_SalaryCategory.java
M src/org/openbravo/erpCommon/ad_callouts/SL_GlobalUse_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_GlobalUse_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_InOutLine_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_InOut_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SL_InOut_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Internal_Consumption_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Internal_Consumption_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Inventory_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Inventory_Locator.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Inventory_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_InvoiceTax_Amt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Amt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Charge_Tax.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_DocType.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Glitem.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_PriceList.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Invoice_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_JournalLineAmt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Journal_Period.java
M src/org/openbravo/erpCommon/ad_callouts/SL_LegalEntity_Acct.java
M src/org/openbravo/erpCommon/ad_callouts/SL_MachineCost.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Movement_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Movement_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Amt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Charge_Tax.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_DocType.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_PriceList.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Order_Tax.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Organization_Info.java
M src/org/openbravo/erpCommon/ad_callouts/SL_PC_Case_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_ProductPrice_PriceListVersion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Product_Type.java
M src/org/openbravo/erpCommon/ad_callouts/SL_ProductionPlan_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_ProductionPlan_WRPhase.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Production_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Production_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Project_Margin.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Project_Planned.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Project_PriceList.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Project_Type.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Proposal_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_RequisitionLine_Amt.java
M src/org/openbravo/erpCommon/ad_callouts/SL_RequisitionLine_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_RequisitionLine_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_RequisitionOrder_Order.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Requisition_BPartner.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Requisition_BPartner_PriceList.java
M src/org/openbravo/erpCommon/ad_callouts/SL_ScheduledMaintenance_Maintenance.java
M src/org/openbravo/erpCommon/ad_callouts/SL_SequenceProduct_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_SequenceProduct_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_User_Name.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WRPhaseProduct_Product.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WRPhase_Sequence.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WorkRequirement_Conversion.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WorkRequirement_ProcessPlan.java
M src/org/openbravo/erpCommon/ad_callouts/SL_WorkRequirement_StartDate.java
M src/org/openbravo/erpCommon/ad_callouts/SL_Year_Validation.java
---
(0098416)
aferraz   
2017-08-08 09:34   
Code review OK

https://docs.google.com/spreadsheets/d/1rW7KRBcHMWOKx5fBkcRVgWNihXVYNGFpZevd6IE41LE/edit?usp=sharing [^]
(0099299)
hudsonbot   
2017-09-21 16:49   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/9750b78d3e5c [^]
Maturity status: Test