Anonymous | Login
Project:
RSS
  
News | My View | View Issues | Roadmap | Summary

View Issue Details[ Jump to Notes ] [ Issue History ] [ Print ]
ID
0052614
TypeCategorySeverityReproducibilityDate SubmittedLast Update
feature request[Openbravo ERP] A. Platformmajorhave not tried2023-05-30 18:292023-10-26 18:06
ReportercbernerView Statuspublic 
Assigned Tocberner 
PrioritynormalResolutionfixedFixed in VersionPR23Q4
StatusclosedFix in branchFixed in SCM revision
ProjectionnoneETAnoneTarget Version
OSAnyDatabaseAnyJava version
OS VersionDatabase versionAnt version
Product VersionSCM revision 
Review Assigned To
Web browser
ModulesCore
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary

0052614: There should be a mechanism to have non-blocking async EDL Requests

DescriptionCurrently all async EDL requests block at least a Import Entry thread and a DB connection, to better use the resources, the generated IE should unblock the thread until the request or processing finishes, and when finished, it should mark the corresponding DB records with the proper status.

A functional and technical design have been prepared for this project:
- Functional design: https://docs.google.com/document/d/1J4pViMYbAqsCJwURGtG3zfub2pttz62n63EVj-sWgcE/edit# [^]
- Technical design: https://docs.google.com/document/d/11IStWu--gxJlyVio3rKT7mfMgM01WSYmn6nkSfajSNY/edit# [^]
Steps To Reproduce-
TagsNo tags attached.
Attached Files

- Relationships [ Relation Graph ] [ Dependency Graph ]
related to defect 0053816 closedcaristu Modules EDL process is duplicating request lines 
causes defect 0053639 closedcberner Openbravo ERP EDL in initial or processing state indefinitely when role has no access to Import Entry table 
causes defect 0053742 closedcberner Openbravo ERP NullPointerException thrown in EDL in unhandled exception cases 
causes defect 0053774 closedcaristu Openbravo ERP RO pool is not closed by IE threads 
causes defect 0053784 closedcberner Openbravo ERP failed non-blocking EDL requests remain in Initial status 
causes backport 0053749PR23Q4 closedcberner Openbravo ERP EDL in initial or processing state indefinitely when role has no access to Import Entry table 

-  Notes
(0151165)
hgbot (developer)
2023-06-14 17:06

 Merge Request created: https://gitlab.com/openbravo/product/openbravo/-/merge_requests/912 [^]
(0151378)
hgbot (developer)
2023-06-20 14:28

 Merge Request created: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration/-/merge_requests/27 [^]
(0153250)
hgbot (developer)
2023-08-04 15:11

 Merge Request created: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/merge_requests/169 [^]
(0153739)
hgbot (developer)
2023-08-21 15:30

 Directly closing issue as related merge request is already approved.

Repository: https://gitlab.com/openbravo/product/openbravo [^]
Changeset: 03fa452e48c60b06e2f1a70f29a093bbf07004d0
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 15:27:39
URL: https://gitlab.com/openbravo/product/openbravo/-/commit/03fa452e48c60b06e2f1a70f29a093bbf07004d0 [^]

Fixes FR-52614: Support non-blocking Import Entries and EDLs

Adds support for a new type of import entry and infrastructure for
external systems to handle non-blocking processing. The non-blocking
processing is performed through CompletableFuture's API, which allows
executing methods after the CompletableFuture has been resolved.

---
A src/org/openbravo/service/NonBlockingExecutorServiceProvider.java
A src/org/openbravo/service/importprocess/NonBlockingImportEntryProcessRunnable.java
M src-db/database/model/tables/C_IMPORT_ENTRY.xml
M src-db/database/model/tables/C_IMPORT_ENTRY_ARCHIVE.xml
M src-db/database/sourcedata/AD_COLUMN.xml
M src-db/database/sourcedata/AD_ELEMENT.xml
M src-db/database/sourcedata/AD_FIELD.xml
M src/org/openbravo/service/externalsystem/http/HttpExternalSystem.java
M src/org/openbravo/service/importprocess/ImportEntryBuilder.java
M src/org/openbravo/service/importprocess/ImportEntryManager.java
M src/org/openbravo/service/importprocess/ImportEntryProcessor.java
---
(0153740)
hgbot (developer)
2023-08-21 15:30

 Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration [^]
Changeset: 3814f42dd715096570c6d519a63e77fb9ebb2738
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 15:22:39
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration/-/commit/3814f42dd715096570c6d519a63e77fb9ebb2738 [^]

Related to FR-52614: Support non-blocking EDLs

Adds support for a new type of EDL, non-blocking, which allows not
awaiting the response of some EDLs, like for example external http
requests. This allows processing asynchronously while also freeing
resources, both DB and CPU wise.

---
A src-test/org/openbravo/externaldata/integration/process/AsynchronousProcessorTest.java
A src/org/openbravo/externaldata/integration/process/NonBlockingAfterBatch.java
A src/org/openbravo/externaldata/integration/process/NonBlockingBeforeBatch.java
M src/org/openbravo/externaldata/integration/process/AsynchronousProcessor.java
M src/org/openbravo/externaldata/integration/process/EDLImportEntryProcessor.java
M src/org/openbravo/externaldata/integration/process/ItemProcessor.java
M src/org/openbravo/externaldata/integration/process/ProcessRequest.java
---
(0153741)
hgbot (developer)
2023-08-21 15:30

 Merge request merged: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration/-/merge_requests/27 [^]
(0153742)
hgbot (developer)
2023-08-21 15:30

 Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.api [^]
Changeset: a91a4d01a195b12ea951ab1ea80f4edc6396b150
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 15:25:52
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/commit/a91a4d01a195b12ea951ab1ea80f4edc6396b150 [^]

Related to FR-52614: Switch Push API to the new non-blocking EDL API

Push API was modified to use the non-blocking EDL API, allowing it to
perform external http requests without awaiting the response of each of
them.

---
M src/org/openbravo/api/edl/ApiEventItemProcessor.java
---
(0153743)
hgbot (developer)
2023-08-21 15:30

 Merge request merged: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/merge_requests/169 [^]
(0153744)
hgbot (developer)
2023-08-21 15:30

 Merge request merged: https://gitlab.com/openbravo/product/openbravo/-/merge_requests/912 [^]
(0153745)
hgbot (developer)
2023-08-21 16:01

 Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration [^]
Changeset: 1ae6da586ab4a7c25fb0ebadfc50cb316e43625c
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 15:59:53
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration/-/commit/1ae6da586ab4a7c25fb0ebadfc50cb316e43625c [^]

Related to FR-52614: Fix eclipse warning on ItemProcessor

---
M src/org/openbravo/externaldata/integration/process/ItemProcessor.java
---
(0153750)
hgbot (developer)
2023-08-21 16:28

 Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.api [^]
Changeset: 36cc31575a39bcb1906b2998779565dcab309303
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 16:28:09
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/commit/36cc31575a39bcb1906b2998779565dcab309303 [^]

Revert "Related to FR-52614: Switch Push API to the new non-blocking EDL API"

This reverts commit a91a4d01a195b12ea951ab1ea80f4edc6396b150, reversing
changes made to aa47d8388ec87b666f8e9cbbe74fa504980d8d08.

This is reverted due to the APIEventsItemProcessorTest not passing in
green, it is missing some mocks and changes related with the
non-blocking API modification.

---
M src/org/openbravo/api/edl/ApiEventItemProcessor.java
---
(0153751)
hgbot (developer)
2023-08-21 17:06

 Merge Request created: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/merge_requests/178 [^]
(0153791)
hgbot (developer)
2023-08-22 13:58

 Merge request merged: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/merge_requests/178 [^]
(0153792)
hgbot (developer)
2023-08-22 13:58

 Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.api [^]
Changeset: 0606f993d0671e3a2597df60ba1a5b9115f1e436
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 22-08-2023 11:58:52
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/commit/0606f993d0671e3a2597df60ba1a5b9115f1e436 [^]

Related to FR-52614: Switch Push API to the new non-blocking EDL API

This also fixes the ApiEventItemProcessorTest to properly mock
non-blocking EDL infrastructure and usage.

---
M src-test/org/openbravo/api/edl/ApiEventItemProcessorTest.java
M src/org/openbravo/api/edl/ApiEventItemProcessor.java
---

- Issue History
Date Modified Username Field Change
2023-05-30 18:29 cberner New Issue
2023-05-30 18:29 cberner Assigned To => cberner
2023-05-30 18:29 cberner Modules => Core
2023-05-30 18:29 cberner Triggers an Emergency Pack => No
2023-06-14 17:06 hgbot Note Added: 0151165
2023-06-20 14:28 hgbot Note Added: 0151378
2023-08-04 15:11 hgbot Note Added: 0153250
2023-08-21 15:30 hgbot Resolution open => fixed
2023-08-21 15:30 hgbot Status new => closed
2023-08-21 15:30 hgbot Fixed in Version => PR23Q4
2023-08-21 15:30 hgbot Note Added: 0153739
2023-08-21 15:30 hgbot Note Added: 0153740
2023-08-21 15:30 hgbot Note Added: 0153741
2023-08-21 15:30 hgbot Note Added: 0153742
2023-08-21 15:30 hgbot Note Added: 0153743
2023-08-21 15:30 hgbot Note Added: 0153744
2023-08-21 16:01 hgbot Note Added: 0153745
2023-08-21 16:28 hgbot Note Added: 0153750
2023-08-21 17:06 hgbot Note Added: 0153751
2023-08-22 13:58 hgbot Note Added: 0153791
2023-08-22 13:58 hgbot Note Added: 0153792
2023-10-24 16:14 cberner Relationship added causes 0053639
2023-10-24 16:14 cberner Relationship added causes 0053742
2023-10-26 17:17 rafaroda Relationship added related to 0053774
2023-10-26 18:05 caristu Relationship deleted related to 0053774
2023-10-26 18:06 caristu Relationship added causes 0053774
2023-10-27 10:11 alostale Relationship added causes 0053784
2023-10-27 10:26 AugustoMauch Relationship added causes 0053749
2023-11-10 09:11 caristu Relationship added related to 0053816
2023-11-10 09:13 caristu Relationship deleted related to 0053816
2023-11-10 09:14 caristu Relationship added related to 0053816

Copyright © 2000 - 2009 MantisBT Group
Powered by Mantis Bugtracker