Openbravo Issue Tracking System - Openbravo ERP
View Issue Details
0037324Openbravo ERPA. Platformpublic2017-11-20 11:592018-01-03 12:39
alostale 
alostale 
urgentmajorhave not tried
closedfixed 
5
 
3.0PR18Q1 
caristu
Core
No
0037324: slow login having many preferences
In a system with some thousands of preferences, depending on how they are distributed, login (both in back-office and POS) can last up to some seconds.

Two different cases detected with different configurations:

== Case 1: many visible preferences ==
Thousands of preferences visible from the client/org/role that is performing the login. Testing this case with 8K preferences defined at system level, login requests took up to 40s.

This case, even possible, it's a bit hypothetical, not that probable to occur in a real instance.

== Case 2: preferences defined in readable org, but not visible due to "visible at org" field ==
Having thousands of preferences defined on an organization that is in the parent tree of the one that is used for current login, login time takes up to extra 2s even those preferences are set as not visible through the "visible at org" field.

This case can occur for example in a real instance with many (in the steps emulating 500) organizations all of them children of a common parent. Preferences are defined in parent one, and visibility limited of each of the children.
Following cases are reproducible in an instance with Openbravo for Retail installed.

1. case 1
1.1. Create 8K preferences at System level, visible from all roles:
  insert into ad_preference
    (ad_preference_id,  ad_client_id,      ad_org_id,
     createdby,         updatedby,         attribute,
     value,              ispropertylist)
    select get_uuid(),  '0',     '0',
           '0',         '0',     'visible-all-'||s, 
           'v:'||s,     'N'
      from generate_series(1,8000) s;

1.2. Login WebPOS
    -> It takes up to 40s to complete, note most of this time is spent at Preferneces.getPreferenceFromList (see Selection_230.png)

2. case 2
2.1. for each preference defined in WebPOS module, insert up to 500 instances, all of them with ad_org_id being in a common parent org (ie in The White Valley Group), but limiting visibility though visibleat_org_id to some no org not visible in login (ie. Central Mountains Store)

insert into ad_preference
            (ad_preference_id,  ad_client_id,      ad_org_id,         
             createdby,          updatedby,         property,         
             value,              ispropertylist,
             visibleat_org_id)
       slect get_uuid(), 
            '39363B0921BB4293B48383844325E84C', -- Client: The White Valley Group
            '67839EEFA49E44AC969BD60093FCC899', -- Org: The White Valley Group
            '0', '0',  value,
            'v:'||s, 'Y',
            '3B187EC130A549A7A9388F8060EF156D' -- visible at org: Central Mountains Store
       from ad_ref_list, generate_series(1,500) s
      where ad_module_id ='FF808181326CC34901326D53DBCF0018' 
        and ad_reference_id ='A26BA480E2014707B47257024C3CBFF7' -- all POS preferences     

2.2 login WebPOS with 'Demo user' on its default 'Vallblanca' organization, which cannot see 'Central Mountains Store'
  -> login request take up to 4s, in this case, time is spent in loading DAL objects in memory
After fixing both cases, time drops to less than 200ms for 1st case and less than 100ms for 2nd one.
Performance
related to feature request 0035553 closed caristu Improve scalability of the login process 
depends on defect 0037112 closed alostale incorrect HQL to retrieve preferences 
related to feature request 0037329 closed alostale make a more generic Utility.arrayListToString 
png Selection_230.png (38,360) 2017-11-20 12:35
https://issues.openbravo.com/file_download.php?file_id=11303&type=bug
png

png Selection_231.png (73,305) 2017-11-20 12:50
https://issues.openbravo.com/file_download.php?file_id=11304&type=bug
png
Issue History
2017-11-20 11:59alostaleNew Issue
2017-11-20 11:59alostaleAssigned To => platform
2017-11-20 11:59alostaleModules => Core
2017-11-20 11:59alostaleTriggers an Emergency Pack => No
2017-11-20 12:00alostaleSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=16313#r16313
2017-11-20 12:08alostaleSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=16314#r16314
2017-11-20 12:10alostaleSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=16315#r16315
2017-11-20 12:10alostaleTag Attached: Performance
2017-11-20 12:10alostaleAssigned Toplatform => alostale
2017-11-20 12:12alostaleRelationship addeddepends on 0037112
2017-11-20 12:13alostaleRelationship addedrelated to 0035553
2017-11-20 12:35alostaleFile Added: Selection_230.png
2017-11-20 12:37alostaleSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=16316#r16316
2017-11-20 12:50alostaleFile Added: Selection_231.png
2017-11-20 12:51alostaleSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=16317#r16317
2017-11-20 13:00alostaleReview Assigned To => caristu
2017-11-20 13:00alostalePrioritynormal => urgent
2017-11-20 13:00alostaleDescription Updatedbug_revision_view_page.php?rev_id=16319#r16319
2017-11-20 14:42alostaleRelationship addedrelated to 0037329
2017-11-21 09:58hgbotCheckin
2017-11-21 09:58hgbotNote Added: 0100637
2017-11-21 09:58hgbotStatusnew => resolved
2017-11-21 09:58hgbotResolutionopen => fixed
2017-11-21 09:58hgbotFixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/f422e1ebac8e738f9855d871d63d3727d4bd349d [^]
2017-11-21 09:58hgbotCheckin
2017-11-21 09:58hgbotNote Added: 0100638
2017-11-21 09:58hgbotFixed in SCM revisionhttp://code.openbravo.com/erp/devel/pi/rev/f422e1ebac8e738f9855d871d63d3727d4bd349d [^] => http://code.openbravo.com/erp/devel/pi/rev/b817e5607d8b45760927890f6f3e3558028c1c1c [^]
2017-11-21 10:07alostaleNote Added: 0100641
2017-11-21 10:28alostaleProposed Solution updated
2017-11-29 11:06caristuNote Added: 0100808
2017-11-29 11:06caristuStatusresolved => closed
2017-11-29 11:06caristuFixed in Version => 3.0PR18Q1
2018-01-03 12:39hudsonbotCheckin
2018-01-03 12:39hudsonbotNote Added: 0101444
2018-01-03 12:39hudsonbotCheckin
2018-01-03 12:39hudsonbotNote Added: 0101445

Notes
(0100637)
hgbot   
2017-11-21 09:58   
Repository: erp/devel/pi
Changeset: f422e1ebac8e738f9855d871d63d3727d4bd349d
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Mon Nov 20 13:11:31 2017 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/f422e1ebac8e738f9855d871d63d3727d4bd349d [^]

fixed bug 37324: slow login having many preferences - case 1

  Retrieving all preferences for current role/org was slow having thousands of
  visible preferences because of how duplicate preferences with different visibility
  were checked. This check was performed by sequentially scanning a list of already
  saved preferences to verify whether current one was already in the list.

  It has now fixed by keeping a temporary map of preferences where the key defines
  if preference is the same or not, in this way it is not necessary to iterate
  over all the entries each time.

---
M src/org/openbravo/erpCommon/businessUtility/Preferences.java
---
(0100638)
hgbot   
2017-11-21 09:58   
Repository: erp/devel/pi
Changeset: b817e5607d8b45760927890f6f3e3558028c1c1c
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Mon Nov 20 13:24:45 2017 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/b817e5607d8b45760927890f6f3e3558028c1c1c [^]

fixed bug 37324: slow login having many preferences - case 2

  When there were thousands of preferences with ad_org visible from current org
  but not visible through visible at org field, loading all preferences was slow.

  The problem was, all those preferences were loaded as DAL objects to filter
  out later in memory those that were not visible.

  This gets fixed by filtering out directly in the query so that no unneeded rows
  are retrived from DB to instantiate DAL objects that later will be discarded
  immediatelly.

---
M src/org/openbravo/erpCommon/businessUtility/Preferences.java
---
(0100641)
alostale   
2017-11-21 10:07   
Related changesets not detected automatically:

changeset: 33029:27c3e6dd6f54
user: Asier Lostalé <asier.lostale@openbravo.com>
date: Tue Nov 21 09:50:10 2017 +0100
summary: related to 37324: renamed PreferenceTest methods

changeset: 33030:f930ff85c698
tag: tip
user: Asier Lostalé <asier.lostale@openbravo.com>
date: Tue Nov 21 09:53:27 2017 +0100
summary: related to 37324: added test case for getAllPreferences
(0100808)
caristu   
2017-11-29 11:06   
Code reviewed + tested OK.
(0101444)
hudsonbot   
2018-01-03 12:39   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/c81e0d3cbab5 [^]
Maturity status: Test
(0101445)
hudsonbot   
2018-01-03 12:39   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/c81e0d3cbab5 [^]
Maturity status: Test