Anonymous | Login
Project:
RSS
  
News | My View | View Issues | Roadmap | Summary

View Issue Details[ Jump to Notes ] [ Issue History ] [ Print ]
ID
0045213
TypeCategorySeverityReproducibilityDate SubmittedLast Update
defect[Modules] Integrations with Azure Blob Storageminorhave not tried2020-10-13 12:172020-10-20 07:41
ReporteralostaleView Statuspublic 
Assigned Tocberner 
PrioritynormalResolutionfixedFixed in Version
StatusclosedFix in branchFixed in SCM revision
ProjectionnoneETAnoneTarget Version
OSAnyDatabaseAnyJava version
OS VersionDatabase versionAnt version
Product VersionSCM revision 
Regression date
Regression introduced by commit
Regression level
Review Assigned To
Regression introduced in release
Summary

0045213: Azure: OOM migrating attachements if missconfigured

DescriptionHaving a missconfiguration in Azure Storage module and trying to migrate thousands of attachments results in an OOM.

The problem occurs when trying to append to the process log the errors received for each of the attachments that is tried to be migrated.
Steps To ReproduceIn an instance with thousands of attachments stored in file system:
1. Configure Azure incorrectly
2. Schedule migration process:

2020-10-08 19:05:12,753 [DefaultQuartzScheduler_Worker-2] ERROR org.quartz.core.ErrorLogger - Job (OB_QUARTZ_GROUP.82ACCBF860F94F688604B7917F3A6907 threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception.
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz-2.3.2.jar:?]
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) [quartz-2.3.2.jar:?]
Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:3745) ~[?:?]
        at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172) ~[?:?]
        at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538) ~[?:?]
        at java.lang.StringBuilder.append(StringBuilder.java:174) ~[?:?]
        at org.openbravo.scheduling.ProcessLogger.log(ProcessLogger.java:45) ~[classes/:?]
        at org.openbravo.service.integration.azure.storage.process.MigrateProcess.migrateAttachments(MigrateProcess.java:169) ~[classes/:?]
        at org.openbravo.service.integration.azure.storage.process.MigrateProcess.doExecute(MigrateProcess.java:110) ~[classes/:?]
        at org.openbravo.service.db.DalBaseProcess.execute(DalBaseProcess.java:85) ~[classes/:?]
        at org.openbravo.scheduling.DefaultJob.execute(DefaultJob.java:65) ~[classes/:?]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:202) ~[quartz-2.3.2.jar:?]
        ... 1 more
Proposed SolutionBefore starting the batch process, ensure configuration is valid and fail properly in case it is not.
TagsNo tags attached.
Attached Files

- Relationships [ Relation Graph ] [ Dependency Graph ]
related to defect 0045246 newTriage Platform Conn OOM exception if misconfigured or files fail to upload consistently 

-  Notes
(0123680)
hgbot (developer)
2020-10-14 15:07

 Merge Request created: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage/-/merge_requests/4 [^]
(0123784)
hgbot (developer)
2020-10-20 07:41

 Directly closing issue as related merge request is already approved.

Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage [^]
Changeset: f5f52378edd222f27ba661c075c74e0eee341e46
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 2020-10-16T11:16:36+02:00
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage/-/commit/f5f52378edd222f27ba661c075c74e0eee341e46 [^]

Fixes ISSUE-45213: OOM migrating attachments if misconfigured

An OutOfMemory exception is thrown if the azure configuration is not
correct when executing MigrateProcess. This was happening because of an
infinite loop when processing batches of files for upload.
It has been fixed by setting a threshold of consecutive errors allowed,
if this threshold is exceeded, MigrateProcess fails. The threshold has
been set to 300.
Note that the errors must be consecutive, if a eventually suceeds
to upload, the current errors counter is reset.

---
M src/org/openbravo/service/integration/azure/storage/process/MigrateProcess.java
---
(0123785)
hgbot (developer)
2020-10-20 07:41

 Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage [^]
Changeset: b32c3084d5b4ff17d86f10a64690f71f2280b0a0
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 2020-10-19T15:17:14+02:00
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage/-/commit/b32c3084d5b4ff17d86f10a64690f71f2280b0a0 [^]

Related to ISSUE-45213: Add logger for Openbravo on MigrateProcess process

Previously MigrateProcess was only logging to the process logger, this
resulted in it not showing its current status until the process ended.
Adding also an log4j logger, allows to have logs in Tomcat and see the
current status of the Migration process or if there's any error or warning.

---
M src/org/openbravo/service/integration/azure/storage/process/MigrateProcess.java
---
(0123786)
hgbot (developer)
2020-10-20 07:41

 Merge request merged: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage/-/merge_requests/4 [^]

- Issue History
Date Modified Username Field Change
2020-10-13 12:17 alostale New Issue
2020-10-13 12:17 alostale Assigned To => platform
2020-10-13 12:17 alostale Modules => Core
2020-10-13 12:17 alostale Triggers an Emergency Pack => No
2020-10-13 12:24 alostale Project Openbravo ERP => Modules
2020-10-13 12:25 alostale Category A. Platform => Integrations with Azure Blob Storage
2020-10-14 15:07 hgbot Note Added: 0123680
2020-10-14 17:07 cberner Assigned To platform => cberner
2020-10-19 12:41 cberner Relationship added related to 0045246
2020-10-20 07:41 hgbot Resolution open => fixed
2020-10-20 07:41 hgbot Status new => closed
2020-10-20 07:41 hgbot Note Added: 0123784
2020-10-20 07:41 hgbot Note Added: 0123785
2020-10-20 07:41 hgbot Note Added: 0123786

Copyright © 2000 - 2009 MantisBT Group
Powered by Mantis Bugtracker