[midPoint-git] [Evolveum/midpoint] e63c0e: Optimize work allocation (round 1)

[Pavol Mederly]

  Branch: refs/heads/master
  Home:   https://github.com/Evolveum/midpoint
  Commit: e63c0ec58232c8d7bff21e7c1df1cb5af9cd648c
      https://github.com/Evolveum/midpoint/commit/e63c0ec58232c8d7bff21e7c1df1cb5af9cd648c
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2018-04-16 (Mon, 16 Apr 2018)

  Changed paths:
    M infra/schema/src/main/resources/xml/ns/public/common/common-core-3.xsd
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/BaseWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/ExplicitWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/NumericWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/SingleNullWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/StringWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestWorkDistribution.java
    A repo/task-quartz-impl/src/test/resources/work/task-107-0.xml
    M repo/task-quartz-impl/src/test/resources/work/task-130-c.xml

  Log Message:
  -----------
  Optimize work allocation (round 1)

Here we optimize creation of new work buckets, to be processed
in batches (instead of one-by-one). Also, when allocating existing
buckets we select them randomly, not getting always the first one.

This helps a bit: although it does not prevent collisions on coordinator
task updates, precondition violations in WorkStateManager are greatly
avoided.

TODO:
- maintain a reasonably long list of ready buckets (for randomization
  to work well)
- delegate buckets in batches (not only create them) - this will avoid
  contention on the coordinator task even more


  Commit: 424fde5057878f1d92381a67f6bb8b3a814162a4
      https://github.com/Evolveum/midpoint/commit/424fde5057878f1d92381a67f6bb8b3a814162a4
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2018-04-30 (Mon, 30 Apr 2018)

  Changed paths:
    M infra/schema/src/main/resources/xml/ns/public/common/common-core-3.xsd
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/BaseWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/ExplicitWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/NumericWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/SingleNullWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/StringWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestWorkDistribution.java
    A repo/task-quartz-impl/src/test/resources/work/task-107-0.xml
    M repo/task-quartz-impl/src/test/resources/work/task-130-c.xml

  Log Message:
  -----------
  Merge branch 'feature/optimizing-work-allocation'


  Commit: d4f42619bcd88823d118d71f838eeeef83f15ecb
      https://github.com/Evolveum/midpoint/commit/d4f42619bcd88823d118d71f838eeeef83f15ecb
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2018-05-02 (Wed, 02 May 2018)

  Changed paths:
    M infra/schema/src/main/resources/xml/ns/public/common/common-core-3.xsd
    M infra/util/src/main/java/com/evolveum/midpoint/util/backoff/ExponentialBackoffComputer.java
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/SqlBaseService.java
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/helpers/BaseHelper.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerConfiguration.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/WorkStateManager.java
    A samples/tasks/multinode-task-2-customized-allocation.xml

  Log Message:
  -----------
  Make work allocation parameters task-configurable

So these parameters can be configured per task in
workManagement/buckets/allocation section.

BEWARE: global configuration parameter
taskManager.workAllocationRetryInterval was changed to
workAllocationRetryIntervalBase.


Compare: https://github.com/Evolveum/midpoint/compare/8759a8055fb9...d4f42619bcd8
      **NOTE:** This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.