[midPoint-git] [Evolveum/midpoint] fcff84: Change Set<Task> to List<Task> to lower contention

mederly noreply at github.com
Wed Feb 13 18:39:41 CET 2019


  Branch: refs/heads/tmp/synchronized-running-tasks
  Home:   https://github.com/Evolveum/midpoint
  Commit: fcff849fc3a1b6845ebbe5e40d3b19e543d4ee5b
      https://github.com/Evolveum/midpoint/commit/fcff849fc3a1b6845ebbe5e40d3b19e543d4ee5b
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-12 (Tue, 12 Feb 2019)

  Changed paths:
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/ExecutionManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/LocalNodeManager.java

  Log Message:
  -----------
  Change Set<Task> to List<Task> to lower contention

Lightweight asynchronous tasks are shared objects that hold their
data in thread-unsafe prism objects. This is to be fixed somehow.

Using HashSet<Task> structures makes this problem even worse because
hashCode/equals are called repetitively in various threads that
access those sets. So here we replace sets with lists to treat
at least some of the concurrency issues (MID-5136).


  Commit: f8ea63965656b12ce1dc9712e6aa12a67a1b17d4
      https://github.com/Evolveum/midpoint/commit/f8ea63965656b12ce1dc9712e6aa12a67a1b17d4
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-13 (Wed, 13 Feb 2019)

  Changed paths:
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StatisticsCollector.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccCertTimedActionTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignCreationTaskHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignReiterationTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCloseStageApproachingTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCloseStageTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationClosingTaskHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationRemediationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/CleanUpTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/ShadowRefreshTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ExecuteDeltasTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ModelOperationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/importer/ImportAccountsFromResourceTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/importer/ImportObjectsFromFileTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/AssignmentEvaluator.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/Clockwork.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/OperationalDataManager.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ExecutionContext.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/IterativeScriptExecutionTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ScriptExecutionTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/ExecuteChangesTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/FocusValidityScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/LiveSyncTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/RecomputeTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/ReconciliationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizeAccountResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/CompletedTaskCleanupTriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/RecomputeTriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AbstractScannerResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AbstractScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AuditReindexTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/DeleteTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ModelImplUtils.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ReindexResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ReindexTaskHandler.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/manual/TestSemiManualGroupingProposed.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/util/MockTaskHandler.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/util/MockTriggerHandler.java
    M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportCreateTaskHandler.java
    M model/report-impl/src/test/java/com/evolveum/midpoint/report/TestReportJasper.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processes/common/WfTimedActionTriggerHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processes/itemApproval/MidpointUtil.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPrepareChildOperationTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPrepareRootOperationTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPropagateTaskObjectReferenceTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfProcessInstanceShadowTaskHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ShadowCache.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationResultHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationTaskHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationResultHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationTaskHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeResultHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeTaskHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/TaskHandlerUtil.java
    M repo/repo-sql-impl-test/src/test/java/com/evolveum/midpoint/repo/sql/util/SimpleTaskAdapter.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightTaskHandler.java
    A repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningTask.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskHandler.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/WorkBucketAwareTaskHandler.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/InternalTaskInterface.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImplUtil.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/ExecutionManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/StalledTasksWatcher.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/TaskSynchronizer.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/JdbcPingTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/NoOpTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/PartitioningTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WaitForSubtasksByPollingTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WaitForTasksTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersCreationTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersRestartTaskHandler.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/WorkStateManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/workers/WorkersManager.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/AbstractTaskManagerTest.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/CleanupTest.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockCycleTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockLongTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockParallelTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockSingleTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockWorkBucketsTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestQuartzTaskManagerContract.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestWorkDistribution.java
    M repo/task-quartz-impl/src/test/resources/basic/task-100LightweightSubtasks.xml

  Log Message:
  -----------
  Make running tasks thread-safe by synchronization

This is an experimental commit trying to resolve
ConcurrentModificationExceptions (e.g. MID-5111, MID-5113, MID-4088,
MID-3954, MID-5131, MID-5135).

However, the solution is not optimal:
1) synchronization covers actions that could take a long time
(repository updates), blocking other unrelated actions (e.g. retrieving
task state information from the task object)
2) there's a lot of delegation code, making RunningTaskQuartzImpl ugly


Compare: https://github.com/Evolveum/midpoint/compare/fcff849fc3a1%5E...f8ea63965656


More information about the midPoint-svn mailing list