Openbravo Issue Tracking System - Retail Modules
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0041027Retail ModulesWeb POSpublic2019-06-04 18:502019-11-04 11:22
Reportergorka_gil 
Assigned Togorka_gil 
PrioritynormalSeveritymajorReproducibilityN/A
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in VersionRR19Q4 
Merge Request Status
Review Assigned Tomarvintm
OBNetwork customer
Support ticket
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary0041027: Optimize masterdata incremental refresh process
DescriptionCurrent masterdata incremental refresh process, blocks the ui during the process that takes some seconds.

The current process is:
- block ui with a show loading
- request one mastertdata model and insert into the db
- continue in the same way with the rest of models one by one
Steps To Reproduce-
Proposed SolutionThe proposal is:
- request all the models in background, without block the ui, saving in the ram
- when all data ready, block the ui and save into db

Since the amount of data can be very big for the ram, idea is to try in this way, but once passed a limit, go back to the current incremental refresh
Additional Information
TagsNo tags attached.
Relationships
related to defect 0040846 new Retail Add popup in case of Timeout when loading a Mobile service 
related to feature request 0041832 new Retail Wrong message is shown "Master data refresh will happen after this ticket is closed" 
related to defect 0041588 closed gorka_gil Messages in web pos ui, don't concatenate 
related to defect 0041831 closed ranjith_qualiantech_com Incremental Refresh is not happening immediately if Time to Show incremental Refresh Popup = 0 
related to defect 0048198 closed shuehner Delete module script from issue 41027 added in 19Q4 (it never worked) 
related to defect 0048422 closed Retail SCO is locked during incremental refresh 
Attached Filesdiff 41027-incrementalRefresh-mobilecore-18q3-newFormatter.diff (26,751) 2019-08-27 11:29
https://issues.openbravo.com/file_download.php?file_id=13266&type=bug
diff 41027-incrementalRefresh-posterminal-18q3-newFormatter.diff (6,531) 2019-08-27 11:29
https://issues.openbravo.com/file_download.php?file_id=13267&type=bug
diff mobCore_OldFormatter_1.diff (9,127) 2019-08-30 15:12
https://issues.openbravo.com/file_download.php?file_id=13275&type=bug
diff mobCore_OldFormatter_2.diff (32,907) 2019-08-30 15:12
https://issues.openbravo.com/file_download.php?file_id=13276&type=bug
diff mobCore_OldFormatter_3.diff (759) 2019-08-30 15:12
https://issues.openbravo.com/file_download.php?file_id=13277&type=bug
diff posterminal_OldFormatter.diff (6,524) 2019-08-30 15:13
https://issues.openbravo.com/file_download.php?file_id=13278&type=bug
diff 41027-incrementalRefresh-mobilecore-18q3-newFormatter-simpleQuotes.diff (26,630) 2019-09-06 11:49
https://issues.openbravo.com/file_download.php?file_id=13292&type=bug
diff 41027-incrementalRefresh-posterminal-18q3-newFormatter-simpleQuotes.diff (6,518) 2019-09-06 11:49
https://issues.openbravo.com/file_download.php?file_id=13293&type=bug
Issue History
Date ModifiedUsernameFieldChange
2019-06-04 18:50gorka_gilNew Issue
2019-06-04 18:50gorka_gilAssigned To => gorka_gil
2019-06-04 18:50gorka_gilTriggers an Emergency Pack => No
2019-06-04 18:55gorka_gilStatusnew => scheduled
2019-06-05 13:30gorka_gilRelationship addedrelated to 0040846
2019-07-22 16:46rafarodaIssue Monitored: rafaroda
2019-08-07 13:57gorka_gilRelationship addedrelated to 0041588
2019-08-08 16:41hgbotCheckin
2019-08-08 16:41hgbotNote Added: 0113910
2019-08-08 16:41hgbotCheckin
2019-08-08 16:41hgbotNote Added: 0113911
2019-08-08 16:41hgbotCheckin
2019-08-08 16:41hgbotNote Added: 0113913
2019-08-08 16:41hgbotCheckin
2019-08-08 16:41hgbotNote Added: 0113914
2019-08-08 16:41hgbotStatusscheduled => resolved
2019-08-08 16:41hgbotResolutionopen => fixed
2019-08-08 16:41hgbotFixed in SCM revision => http://code.openbravo.com/erp/pmods/org.openbravo.mobile.core/rev/86b8435e3609dc11c611a20a563becfd68611616 [^]
2019-08-08 16:41hgbotCheckin
2019-08-08 16:41hgbotNote Added: 0113915
2019-08-16 08:09hgbotCheckin
2019-08-16 08:09hgbotNote Added: 0114027
2019-08-16 08:10marvintmReview Assigned To => marvintm
2019-08-16 08:10marvintmStatusresolved => closed
2019-08-16 08:10marvintmFixed in Version => RR19Q4
2019-08-27 11:29gorka_gilFile Added: 41027-incrementalRefresh-mobilecore-18q3-newFormatter.diff
2019-08-27 11:29gorka_gilFile Added: 41027-incrementalRefresh-posterminal-18q3-newFormatter.diff
2019-08-30 15:11samuel_nicuesaFile Added: mobCore_OldFormater_1.diff
2019-08-30 15:11samuel_nicuesaFile Added: mobCore_OldFormater_2.diff
2019-08-30 15:11samuel_nicuesaFile Deleted: mobCore_OldFormater_1.diff
2019-08-30 15:11samuel_nicuesaFile Deleted: mobCore_OldFormater_2.diff
2019-08-30 15:12samuel_nicuesaFile Added: mobCore_OldFormatter_1.diff
2019-08-30 15:12samuel_nicuesaFile Added: mobCore_OldFormatter_2.diff
2019-08-30 15:12samuel_nicuesaFile Added: mobCore_OldFormatter_3.diff
2019-08-30 15:13samuel_nicuesaFile Added: posterminal_OldFormatter.diff
2019-09-06 11:07gorka_gilFile Added: 41027-incrementalRefresh-mobilecore-18q3-newFormatter-simpleQuotes.diff
2019-09-06 11:07gorka_gilFile Added: 41027-incrementalRefresh-posterminal-18q3-newFormatter-simpleQuotes.diff
2019-09-06 11:48gorka_gilFile Deleted: 41027-incrementalRefresh-mobilecore-18q3-newFormatter-simpleQuotes.diff
2019-09-06 11:48gorka_gilFile Deleted: 41027-incrementalRefresh-posterminal-18q3-newFormatter-simpleQuotes.diff
2019-09-06 11:49gorka_gilFile Added: 41027-incrementalRefresh-mobilecore-18q3-newFormatter-simpleQuotes.diff
2019-09-06 11:49gorka_gilFile Added: 41027-incrementalRefresh-posterminal-18q3-newFormatter-simpleQuotes.diff
2019-09-20 10:20gorka_gilRelationship addedrelated to 0041831
2019-11-04 11:22rafarodaRelationship addedrelated to 0041832
2021-12-07 11:52shuehnerRelationship addedrelated to 0048198
2022-01-19 18:17gorka_gilRelationship addedrelated to 0048422

Notes
(0113910)
hgbot   
2019-08-08 16:41   
Repository: tools/automation/pi-mobile
Changeset: 56e61a198f644b6b867f172d681f49b9a2df17e6
Author: Gorka Gil <gorka.gil <at> openbravo.com>
Date: Wed Jul 31 19:28:15 2019 +0200
URL: http://code.openbravo.com/tools/automation/pi-mobile/rev/56e61a198f644b6b867f172d681f49b9a2df17e6 [^]

Related to issue 41027: added masterdata refresh tests and fix old ones

Create new tests for play with the preference of masterdata background limit.

Old tests are checking that the popup of masterdata appears, changed them to check the lastTimestamp to check that the update has happen

Removes the statis waits of 2 mins, and call directly to the functions that are executed after the first timeout happends, and when
the second timeout happen.

Added tests to wait the timers to do the refresh masterdata, but only 2 and without higvol, so uses as less as possible time.

Added api to work with the refresh masterdata to mobileCoreTerminalHelper

Added api to set and remove preferences.

---
M src-test/org/openbravo/test/mobile/retail/extmodules/selenium/terminals/WebPOSExtModulesLoyaltyTerminalHelper.java
M src-test/org/openbravo/test/mobile/retail/extmodules/selenium/tests/cashup/I38194_ResfreshMasterData.java
M src-test/org/openbravo/test/mobile/retail/extmodules/selenium/tests/selfcheckout/I38194_SCORefreshMasterData.java
M src-test/org/openbravo/test/mobile/retail/mobilecore/selenium/terminals/MobileCoreTerminalHelper.java
M src-test/org/openbravo/test/mobile/retail/pack/selenium/terminals/WebPOSTerminalHelper.java
M src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/masterDataRefresh/I38194_ResfreshMasterDataCashManagement.java
M src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/masterDataRefresh/I38194_ResfreshMasterDataCashUp.java
M src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/masterDataRefresh/I38194_ResfreshMasterDataPointOfSale.java
M src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/orderselector/VerifyOrderSelector.java
M src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/system/I30382_MasterDataUpdatedWithIncrementalRefresh.java
M src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/system/I34868_ResetCacheModelAfterIncrementalRefresh.java
A src-test/org/openbravo/test/mobile/retail/extmodules/selenium/tests/selfcheckout/BaseSCOMasterDataRefreshTests.java
A src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/masterDataRefresh/BaseMasterDataRefreshTests.java
A src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/masterDataRefresh/I41027_ResfreshMasterDataInBackground.java
A src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/masterDataRefresh/ResfreshMasterDataOnTimer.java
---
(0113911)
hgbot   
2019-08-08 16:41   
Repository: tools/automation/pi-mobile
Changeset: 970006e75a7a5389bcf1afdd68206e81ac700824
Author: Gorka Gil <gorka.gil <at> openbravo.com>
Date: Thu Aug 08 16:40:22 2019 +0200
URL: http://code.openbravo.com/tools/automation/pi-mobile/rev/970006e75a7a5389bcf1afdd68206e81ac700824 [^]

Related to issue 41027: disable masterdata background tests till they are stable

---
M src-test/org/openbravo/test/mobile/retail/pack/selenium/tests/masterDataRefresh/I41027_ResfreshMasterDataInBackground.java
---
(0113913)
hgbot   
2019-08-08 16:41   
Repository: erp/pmods/org.openbravo.mobile.core
Changeset: 74cff7c7a4469593b37761ca694e2977c3a141ae
Author: Gorka Gil <gorka.gil <at> openbravo.com>
Date: Tue Aug 06 15:09:17 2019 +0200
URL: http://code.openbravo.com/erp/pmods/org.openbravo.mobile.core/rev/74cff7c7a4469593b37761ca694e2977c3a141ae [^]

Related to issue 41027: Create modulescript to mantain refresh on login in existing instances

---
A src-util/modulescript/src/org/openbravo/mobile/core/modulescript/MantainMasterdataRefreshBehaviour41027.java
A src-util/modulescript/src/org/openbravo/mobile/core/modulescript/MantainMasterdataRefreshBehaviour41027_Data.xsql
A src-util/modulescript/src/src/org/openbravo/mobile/core/modulescript/MantainMasterdataRefreshBehaviour41027Data.java
---
(0113914)
hgbot   
2019-08-08 16:41   
Repository: erp/pmods/org.openbravo.mobile.core
Changeset: 86b8435e3609dc11c611a20a563becfd68611616
Author: Gorka Gil <gorka.gil <at> openbravo.com>
Date: Wed Jul 31 17:55:38 2019 +0200
URL: http://code.openbravo.com/erp/pmods/org.openbravo.mobile.core/rev/86b8435e3609dc11c611a20a563becfd68611616 [^]

Fixes issue 41027: Implement masterdata refresh in background

- Set as default the preference that skip the refresh in logins and F5.
Also created a modulescript to mantain old behaviour in current instances.

- split the code inside handleIncremental into 2 functions:
  - doModelRequest
  - doModelDataSave

- Also added 2 new functions:
  - doModelReadFromRAM
  - doModelDataSaveInRAM
These functions save in ram the request and read from ram
to pass to the save in db. In ram is saved all the needed things
so the save code in db is exactly the same as before.


Changes in behaviour:

There is a timeout by default 60 mins, that calls to OB.UTIL.loadModelsIncFunc()
- old: it saves in local storage that is needed to do the refresh
- new: start the request of models in background

When a ticket is closed, or when the other timeout (by default 5 mins) finish,
or when there is navigation to another window, it calls to
OB.UTIL.checkRefreshMasterData():
- old: a popup with a countdwon of 3 secs starts and if not cancelled a foreground
refresh starts
- new: starts a foreground save from ram to db


Others:

- The backgorund refresh if reach a limit, launches the old refresh (foreground one).
In this case is saved in localstorage to avoid any other background refresh till a
foreground one has happen.

The models loaded by the backgorund request don't update the timestamp used to know
if a model has been updated, so if the 10th model exceeds the limit and normal one is used
the first nine models are requested again and no data is lost.

- Since now the request and save are independent, there are protections to not start
the save in db before the the requests has finish. Also to protect that requests, save
and foreground refresh don't execute in parallel. This protections are saved in ram
so a F5 will delete them, intended to not let the refresh blocked if presed F5 in the midle
of process.

- If no update in any model, it will do nothing, not even show the scrim of showloading.
Also if a model don't have updates, it is skipped.
Added a message at the end of the process to know that has finish, since skipped models
don't show a line in the console.

- Reduced the console messages, since the console messages makes the process much slower.

- Add missing return in triggerReady first if, so don't call twice to sucess callback
(dalLoadModelsCallback).

---
M src-db/database/sourcedata/AD_MESSAGE.xml
M src-db/database/sourcedata/AD_PREFERENCE.xml
M src-db/database/sourcedata/AD_REF_LIST.xml
M web/org.openbravo.mobile.core/source/data/ob-dal.js
M web/org.openbravo.mobile.core/source/data/ob-datasource.js
M web/org.openbravo.mobile.core/source/model/ob-terminal-model.js
---
(0113915)
hgbot   
2019-08-08 16:41   
Repository: erp/pmods/org.openbravo.retail.posterminal
Changeset: 8a8262d9cfe88ad8193b5a1d3468cc8e19419e4f
Author: Gorka Gil <gorka.gil <at> openbravo.com>
Date: Wed Jul 31 17:54:03 2019 +0200
URL: http://code.openbravo.com/erp/pmods/org.openbravo.retail.posterminal/rev/8a8262d9cfe88ad8193b5a1d3468cc8e19419e4f [^]

Related to issue 41027: Implement masterdata refresh in background

---
M web/org.openbravo.retail.posterminal/js/utils/ob-utilitiesuipos.js
---
(0114027)
hgbot   
2019-08-16 08:09   
Repository: erp/pmods/org.openbravo.mobile.core
Changeset: a77df10e67664a12795082c652f20e4b53ff7918
Author: Antonio Moreno Perez <antonio.moreno <at> openbravo.com>
Date: Fri Aug 16 08:09:13 2019 +0200
URL: http://code.openbravo.com/erp/pmods/org.openbravo.mobile.core/rev/a77df10e67664a12795082c652f20e4b53ff7918 [^]

Related to issue 41027. Show which model failed in case there was a failure.

---
M web/org.openbravo.mobile.core/source/data/ob-datasource.js
---