0038205: Cost Adjustment Process very slow - Openbravo Issue Tracking System

Openbravo Issue Tracking System - Openbravo ERP

View Issue Details

Project

Category

View Status

Date Submitted

Last Update

0038205

Openbravo ERP

I. Performance

public

2018-03-22 18:13

2018-05-09 19:30

Reporter

gorkaion

Assigned To

collazoandy4

Priority

urgent

Severity

major

Reproducibility

always

Status

closed

Resolution

fixed

Platform

OS Version

Product Version

Target Version

Fixed in Version

3.0PR18Q3

Merge Request Status

Review Assigned To

dmiguelez

OBNetwork customer

Web browser

Modules

Core

Support ticket

Regression level

Regression date

Regression introduced in release

Regression introduced by commit

Triggers an Emergency Pack

Summary

0038205: Cost Adjustment Process very slow

Description

A Cost Adjustment on an environment with 16k transactions and adjusting 11.5k takes almost 10h to complete.

After a review in YourKit there are 3 areas of improvement.
1. Method that inserts m_transaction_cost records.
2. Logic to calculate new LineNo of Cost Adjustment Line.
3. Inserts in Cost Adjustment Line. (already reported to platform in a separate issue 0038197 )

Steps To Reproduce

On an environment with around 16k transactions using manufacturing.
Execute a Cost Adjustment on one of the oldest transactions of a raw material.
This adjustment will also impact all items produced with that raw material resulting in a cost adjustment including many m_transactions, and much more adjustment lines.

Proposed Solution

Attached a patch extracted from a customer test instance with a tentative fix for issues 1st and 2nd, and other changes to improve performance. This patch does changes in following core files reducing the time to 2.5h:

AverageCostAdjustment:
- Logger using wrong class.
- Remove not needed save() from updated objects.
CostAdjustmentProcess:
- Remove not needed save() from updated objects.
- generateTransactionCosts():
-- Do the flush() only once at the end.
-- Evict the trx object on each iteration as it is not modified. Or instead of loading the full trx change to a query retrieving only the required properties.
-- Even better change the method to use a unique insert statement.
CostAdjustementUtils:
- getNewLineNo() method consumes too much time when many lines are added on a single adjustment.
-- Change order and get first by a get max(line) -> when there is no index on the order column it performs better.
-- Add a new insertCostAdjustmentLine() with a lineNo parameter so it is not required to call the getNewLineNo() method.
CostingAlgorithmAdjustmentImp:
- Add new field nextLineNo to store last LineNo used to use new API in CostAdjustmentUtils and avoid the query to get the max lineno.

With the provided patch the flush() still takes almost 1/3 of the total time. Other changes that could be considered are:

in any class involved:
- Review if there are more flush() and save() that can be removed.
- Review if it is possible to do a query to get a single property instead of the full BaseOBObject to reduce memory usage and fasten the flushes when the BaseOBObject is not expected to be updated.

AverageCostAdjustment:
- 2 flushes required when updating m_costing to avoid the trigger that disables the ability to modify a permanent costing.
-- Consider a massive update of the m_costing to set the flag to false and another one at the end to set it back to true to avoid 2 flushes on each m_costing update. Or disable the trigger when the CostAdjustmentProcess is running.

Additional Information

Tags

No tags attached.

Relationships

related to	defect	0038197		acknowledged	Triage Platform Base	TreeTablesEventHandler performance issue on processes with lots of inserts
related to	defect	0038206		closed	collazoandy4	Cost Adjustment Process generates too much lines