[midPoint-git] [Evolveum/midpoint] db9db4: Change Set<Task> to List<Task> to lower contention

mederly noreply at github.com
Fri Feb 15 16:58:56 CET 2019 

  Branch: refs/heads/master
  Home:   https://github.com/Evolveum/midpoint
  Commit: db9db4effef612d2bfc64496a5d7c349e196e65f
      https://github.com/Evolveum/midpoint/commit/db9db4effef612d2bfc64496a5d7c349e196e65f
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-15 (Fri, 15 Feb 2019)

  Changed paths:
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/ExecutionManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/LocalNodeManager.java

  Log Message:
  -----------
  Change Set<Task> to List<Task> to lower contention

Lightweight asynchronous tasks are shared objects that hold their
data in thread-unsafe prism objects. This is to be fixed somehow.

Using HashSet<Task> structures makes this problem even worse because
hashCode/equals are called repetitively in various threads that
access those sets. So here we replace sets with lists to treat
at least some of the concurrency issues (MID-5136).


  Commit: d9e4d229dd6dd62dab8c463ed7260f036d1d9191
      https://github.com/Evolveum/midpoint/commit/d9e4d229dd6dd62dab8c463ed7260f036d1d9191
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-15 (Fri, 15 Feb 2019)

  Changed paths:
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/admin/configuration/PageAdminConfiguration.java
    M infra/prism-api/src/main/java/com/evolveum/midpoint/prism/xml/XmlTypeConverter.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StatisticsCollector.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccCertTimedActionTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignCreationTaskHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignReiterationTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCloseStageApproachingTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCloseStageTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationClosingTaskHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationRemediationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/ModelConstants.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/CleanUpTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/ShadowRefreshTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ExecuteDeltasTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ModelOperationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/importer/ImportAccountsFromResourceTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/importer/ImportObjectsFromFileTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/AssignmentEvaluator.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/Clockwork.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/OperationalDataManager.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ExecutionContext.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/IterativeScriptExecutionTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ScriptExecutionTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/ExecuteChangesTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/FocusValidityScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/LiveSyncTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/RecomputeTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/ReconciliationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizationContext.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizeAccountResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/CompletedTaskCleanupTriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/RecomputeTriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AbstractScannerResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AbstractScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AuditReindexTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/DeleteTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ModelImplUtils.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ReindexResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ReindexTaskHandler.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/manual/TestSemiManualGroupingProposed.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/util/MockTaskHandler.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/util/MockTriggerHandler.java
    M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportCreateTaskHandler.java
    M model/report-impl/src/test/java/com/evolveum/midpoint/report/TestReportJasper.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processes/common/WfTimedActionTriggerHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processes/itemApproval/MidpointUtil.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/general/GeneralChangeProcessor.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPrepareChildOperationTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPrepareRootOperationTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPropagateTaskObjectReferenceTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfProcessInstanceShadowTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfTask.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfTaskController.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfTaskUtil.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ProvisioningServiceImpl.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ShadowCache.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationResultHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationTaskHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationResultHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationTaskHandler.java
    M provisioning/provisioning-impl/src/test/java/com/evolveum/midpoint/provisioning/impl/mock/SynchronizationServiceMock.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeResultHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeTaskHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/TaskHandlerUtil.java
    M repo/repo-sql-impl-test/src/test/java/com/evolveum/midpoint/repo/sql/util/SimpleTaskAdapter.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightTaskHandler.java
    A repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningTask.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskHandler.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/WorkBucketAwareTaskHandler.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/InternalTaskInterface.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImplUtil.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/ExecutionManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/StalledTasksWatcher.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/TaskSynchronizer.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/JdbcPingTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/NoOpTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/PartitioningTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WaitForSubtasksByPollingTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WaitForTasksTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersCreationTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersRestartTaskHandler.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/WorkStateManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/workers/WorkersManager.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/AbstractTaskManagerTest.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/CleanupTest.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockCycleTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockLongTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockParallelTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockSingleTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockWorkBucketsTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestQuartzTaskManagerContract.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestWorkDistribution.java
    M repo/task-quartz-impl/src/test/resources/basic/task-100LightweightSubtasks.xml

  Log Message:
  -----------
  Make running tasks thread-safe by synchronization

This is to resolve ConcurrentModificationExceptions (e.g. MID-5111,
MID-5113, MID-4088, MID-3954, MID-5131, MID-5135).

Features:
1) Methods of Task implementation are synchronized on various
objects to ensure thread safety
2) RunningTask as a subtype of Task provides even more safety
by cloning returned objects where appropriate

Although the implementation is not 100% safe yet, it should be well
enough for current state of midPoint code base.


  Commit: 2140665202bd586df90b371644468205086ef18b
      https://github.com/Evolveum/midpoint/commit/2140665202bd586df90b371644468205086ef18b
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-15 (Fri, 15 Feb 2019)

  Changed paths:
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java

  Log Message:
  -----------
  Fix memory leak in tests


  Commit: a559b2e659d8f2282ee3a9107597461fcb0d57ea
      https://github.com/Evolveum/midpoint/commit/a559b2e659d8f2282ee3a9107597461fcb0d57ea
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-15 (Fri, 15 Feb 2019)

  Changed paths:
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeResultHandler.java

  Log Message:
  -----------
  Fix compilation issues after merging


  Commit: 17c5f9f3795530313f6c6136b0fbe22a1939917e
      https://github.com/Evolveum/midpoint/commit/17c5f9f3795530313f6c6136b0fbe22a1939917e
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-15 (Fri, 15 Feb 2019)

  Changed paths:
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/component/assignment/AssignmentEditorPanel.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/component/form/DropDownFormGroup.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/component/input/RelationDropDownChoicePanel.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/component/wizard/Wizard.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/component/wizard/WizardSteps.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/self/AbstractShoppingCartTabPanel.html
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/self/AbstractShoppingCartTabPanel.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/self/PageAssignmentShoppingCart.html
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/self/PageAssignmentShoppingCart.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/self/PageAssignmentsList.java
    M gui/admin-gui/src/main/resources/localization/Midpoint.properties
    M infra/schema/src/main/resources/localization/schema.properties
    M testing/schrodingertest/src/test/java/com/evolveum/midpoint/testing/schrodinger/TestBase.java
    A testing/schrodingertest/src/test/java/com/evolveum/midpoint/testing/schrodinger/scenarios/AssignmentArchetypeTest.java
    A testing/schrodingertest/src/test/java/com/evolveum/midpoint/testing/schrodinger/scenarios/ObjectListArchetypeTests.java
    M testing/schrodingertest/src/test/java/com/evolveum/midpoint/testing/schrodinger/scenarios/PostAuthenticationTests.java
    A testing/schrodingertest/src/test/resources/configuration/objects/archetypes/archetype-business-role.xml
    A testing/schrodingertest/src/test/resources/configuration/objects/archetypes/archetype-employee.xml
    M testing/schrodingertest/testng-integration-schrodinger.xml
    R tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/FocusSetAssignmentsModal.java
    R tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/FocusSetProjectionModal.java
    R tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/common/ConfirmationModal.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/common/DropDown.java
    R tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/common/ModalBox.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/common/PrismForm.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/common/table/Table.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/configuration/AdminGuiTab.java
    A tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/configuration/ObjectCollectionViewsPanel.java
    A tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/configuration/RoleManagementTab.java
    A tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/modal/ConfirmationModal.java
    A tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/modal/FocusSetAssignmentsModal.java
    A tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/modal/FocusSetProjectionModal.java
    A tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/modal/ModalBox.java
    A tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/modal/ObjectBrowserModal.java
    A tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/modal/ObjectBrowserModalTable.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/resource/ResourceShadowTableCog.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/resource/TestConnectionModal.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/user/UserAssignmentsTab.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/user/UserProjectionsCog.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/user/UsersPageTable.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/component/user/UsersTableDropDown.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/page/BasicPage.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/page/configuration/AboutPage.java
    M tools/schrodinger/src/main/java/com/evolveum/midpoint/schrodinger/page/configuration/SystemPage.java

  Log Message:
  -----------
  Merge remote-tracking branch 'origin/master'


Compare: https://github.com/Evolveum/midpoint/compare/bc0266dfe018...17c5f9f37955

More information about the midPoint-svn mailing list