[midPoint-git] [Evolveum/midpoint] fcff84: Change Set<Task> to List<Task> to lower contention
mederly
noreply at github.com
Wed Feb 13 18:39:41 CET 2019
Branch: refs/heads/tmp/synchronized-running-tasks
Home: https://github.com/Evolveum/midpoint
Commit: fcff849fc3a1b6845ebbe5e40d3b19e543d4ee5b
https://github.com/Evolveum/midpoint/commit/fcff849fc3a1b6845ebbe5e40d3b19e543d4ee5b
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2019-02-12 (Tue, 12 Feb 2019)
Changed paths:
M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/ExecutionManager.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/LocalNodeManager.java
Log Message:
-----------
Change Set<Task> to List<Task> to lower contention
Lightweight asynchronous tasks are shared objects that hold their
data in thread-unsafe prism objects. This is to be fixed somehow.
Using HashSet<Task> structures makes this problem even worse because
hashCode/equals are called repetitively in various threads that
access those sets. So here we replace sets with lists to treat
at least some of the concurrency issues (MID-5136).
Commit: f8ea63965656b12ce1dc9712e6aa12a67a1b17d4
https://github.com/Evolveum/midpoint/commit/f8ea63965656b12ce1dc9712e6aa12a67a1b17d4
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2019-02-13 (Wed, 13 Feb 2019)
Changed paths:
M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StatisticsCollector.java
M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccCertTimedActionTriggerHandler.java
M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignCreationTaskHandler.java
M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignReiterationTriggerHandler.java
M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCloseStageApproachingTriggerHandler.java
M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCloseStageTriggerHandler.java
M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationClosingTaskHandler.java
M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationRemediationTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/CleanUpTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/ShadowRefreshTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ExecuteDeltasTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ModelOperationTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/importer/ImportAccountsFromResourceTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/importer/ImportObjectsFromFileTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckResultHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckResultHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/AssignmentEvaluator.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/Clockwork.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/OperationalDataManager.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ExecutionContext.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/IterativeScriptExecutionTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ScriptExecutionTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/ExecuteChangesTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/FocusValidityScannerTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/LiveSyncTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/RecomputeTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/ReconciliationTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizeAccountResultHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/CompletedTaskCleanupTriggerHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/RecomputeTriggerHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerScannerTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AbstractScannerResultHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AbstractScannerTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AuditReindexTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/DeleteTaskHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ModelImplUtils.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ReindexResultHandler.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ReindexTaskHandler.java
M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/manual/TestSemiManualGroupingProposed.java
M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/util/MockTaskHandler.java
M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/util/MockTriggerHandler.java
M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportCreateTaskHandler.java
M model/report-impl/src/test/java/com/evolveum/midpoint/report/TestReportJasper.java
M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processes/common/WfTimedActionTriggerHandler.java
M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processes/itemApproval/MidpointUtil.java
M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPrepareChildOperationTaskHandler.java
M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPrepareRootOperationTaskHandler.java
M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPropagateTaskObjectReferenceTaskHandler.java
M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfProcessInstanceShadowTaskHandler.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ShadowCache.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationResultHandler.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationTaskHandler.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationResultHandler.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationTaskHandler.java
M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeResultHandler.java
M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeTaskHandler.java
M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/TaskHandlerUtil.java
M repo/repo-sql-impl-test/src/test/java/com/evolveum/midpoint/repo/sql/util/SimpleTaskAdapter.java
M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightTaskHandler.java
A repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningTask.java
M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskHandler.java
M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/WorkBucketAwareTaskHandler.java
A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/InternalTaskInterface.java
A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImplUtil.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/ExecutionManager.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/StalledTasksWatcher.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/TaskSynchronizer.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/JdbcPingTaskHandler.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/NoOpTaskHandler.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/PartitioningTaskHandler.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WaitForSubtasksByPollingTaskHandler.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WaitForTasksTaskHandler.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersCreationTaskHandler.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersRestartTaskHandler.java
A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/WorkStateManager.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/workers/WorkersManager.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/AbstractTaskManagerTest.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/CleanupTest.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockCycleTaskHandler.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockLongTaskHandler.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockParallelTaskHandler.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockSingleTaskHandler.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockWorkBucketsTaskHandler.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestQuartzTaskManagerContract.java
M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestWorkDistribution.java
M repo/task-quartz-impl/src/test/resources/basic/task-100LightweightSubtasks.xml
Log Message:
-----------
Make running tasks thread-safe by synchronization
This is an experimental commit trying to resolve
ConcurrentModificationExceptions (e.g. MID-5111, MID-5113, MID-4088,
MID-3954, MID-5131, MID-5135).
However, the solution is not optimal:
1) synchronization covers actions that could take a long time
(repository updates), blocking other unrelated actions (e.g. retrieving
task state information from the task object)
2) there's a lot of delegation code, making RunningTaskQuartzImpl ugly
Compare: https://github.com/Evolveum/midpoint/compare/fcff849fc3a1%5E...f8ea63965656
More information about the midPoint-svn
mailing list