[midPoint-git] [Evolveum/midpoint] 71bc1f: Improve task performance reporting

[mederly]

  Branch: refs/heads/tmp/fixing-progress
  Home:   https://github.com/Evolveum/midpoint
  Commit: 71bc1f684e174d3ac953f8891d917f157c1b73ab
      https://github.com/Evolveum/midpoint/commit/71bc1f684e174d3ac953f8891d917f157c1b73ab
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2021-03-19 (Fri, 19 Mar 2021)

  Changed paths:
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/admin/server/dto/TaskIterativeProgressType.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/IterativeOperationCollector.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/IterativeOperationStartInfo.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/IterativeTaskInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/IterativeTaskInformationPrinter.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskOperationStatsUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskPartPerformanceInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/WallClockTimeComputer.java
    M infra/schema/src/main/resources/xml/ns/public/common/common-core-3.xsd
    M infra/util/src/main/java/com/evolveum/midpoint/util/MiscUtil.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/recon/ReconciliationTaskFirstPartExecution.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/recon/ReconciliationTaskSecondPartExecution.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/recon/ReconciliationTaskThirdPartExecution.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/TestLiveSyncTaskMechanics.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/TestProgressReporting.java
    A model/model-intest/src/test/resources/reporting/task-reconcile-dummy-0t-nb-np.xml
    A model/model-intest/src/test/resources/reporting/task-reconcile-dummy-2t-nb-np.xml
    R model/model-intest/src/test/resources/reporting/task-reconcile-dummy-interrupted.xml
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractIterativeTaskPartExecution.java
    A repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/CurrentBucketStatistics.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/ItemProcessingGatekeeper.java
    R repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/ItemProcessingStatistics.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/TaskHandlerUtil.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/test/NullTaskImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningLightweightTaskImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/run/BucketAwareHandlerExecution.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java

  Log Message:
  -----------
  Improve task performance reporting

1. Fixed statistics multiplication on bucket iteration. (Also fixed
zeroing statistics on LAT deletion.)
2. Added comprehensive statistics on item and bucket completion.
3. Fixed task part start time recording.
4. Unified in-memory and in-repo processing time spent: changed
from long to double.
5. Improved consistency of bucket-related and part-related statistics:
fixed unimportant but visually appealing details like minor differences
due to different places where time is measured.

Unrelated changes:
- Removed useless synchronization clauses in Statistics class. Instead
we use more appropriate volatile clauses on relevant objects (that are
synchronized on their own) instead.

Still work in progress.


  Commit: 20374ee59300cd461b370e23e954edbdaf7e4e08
      https://github.com/Evolveum/midpoint/commit/20374ee59300cd461b370e23e954edbdaf7e4e08
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2021-03-19 (Fri, 19 Mar 2021)

  Changed paths:
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/PartitioningTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersCreationTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/AbstractTaskManagerTest.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestPartitioning.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestWorkersManagement.java

  Log Message:
  -----------
  Stop setting progress for aux tasks

Worker creation and partitioning task handlers no longer increase
progress on their execution. It currently interferes with the progress
reporting feature.

Adapted known tests that relied on this. (Maybe some others
are still to be changed.)


  Commit: 350e5870813819cb731ad1f009bd1cd567cfae7e
      https://github.com/Evolveum/midpoint/commit/350e5870813819cb731ad1f009bd1cd567cfae7e
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2021-03-19 (Fri, 19 Mar 2021)

  Changed paths:
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskPartPerformanceInformation.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskPerformanceInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskProgressUtil.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/TestProgressReporting.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/StructuredTaskProgressAsserter.java

  Log Message:
  -----------
  Provide TaskPerformanceInformation

This is very quick-and-dirty attempt to provide easy access to the
task performance information.


Compare: https://github.com/Evolveum/midpoint/compare/703b5ac1d1b0...350e58708138