[midPoint-git] [Evolveum/midpoint] cb2f09: Improve performance reporting
mederly
noreply at github.com
Thu Feb 20 13:26:33 CET 2025
Branch: refs/heads/support-4.8
Home: https://github.com/Evolveum/midpoint
Commit: cb2f095faa746710fd23d773ce1e6072c42d0459
https://github.com/Evolveum/midpoint/commit/cb2f095faa746710fd23d773ce1e6072c42d0459
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2025-02-13 (Thu, 13 Feb 2025)
Changed paths:
M infra/schema/src/main/java/com/evolveum/midpoint/schema/result/OperationConstants.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/result/OperationResult.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/AbstractStatisticsPrinter.java
A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/BasicComponentStructure.java
A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ComponentsPerformanceComputer.java
A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ComponentsPerformanceInformationPrinter.java
A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ComponentsPerformanceInformationUtil.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/OperationsPerformanceInformationPrinter.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/OperationsPerformanceInformationUtil.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/RepositoryPerformanceInformationPrinter.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/traces/OpType.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskOperationStatsUtil.java
M infra/schema/src/main/resources/xml/ns/public/common/common-core-3.xsd
M infra/schema/src/main/resources/xml/ns/public/common/common-tasks-3.xsd
M infra/test-util/src/main/java/com/evolveum/midpoint/test/util/TestReportUtil.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ProvisioningServiceImpl.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/resourceobjects/ResourceObjectSearchOperation.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/resources/ConnectorManager.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowGetOperation.java
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/TestSystemPerformance.java
Log Message:
-----------
Improve performance reporting
1. "Own time" was added to individual operations performance
information.
2. Based on this information, basic component-level reporting was
introduced. See BasicComponentStructure class.
3. Some bugs in operation result handling in connector discovery methods
(in provisioning) were fixed. They led to obviously wrong (negative)
own times for the respective operations.
4. Operation name for ResourceObjectSearchOperation#handleObjectFound
was fixed, to be correctly categorized as belonging to provisioning.
Work in progress. To be tested.
Commit: 4cd68fa98435704bd47aa342c34e64c7eb5ffa30
https://github.com/Evolveum/midpoint/commit/4cd68fa98435704bd47aa342c34e64c7eb5ffa30
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2025-02-18 (Tue, 18 Feb 2025)
Changed paths:
M infra/schema/src/main/java/com/evolveum/midpoint/schema/ObjectHandler.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/ResultHandler.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/result/OperationResult.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/BasicComponentStructure.java
M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ComponentsPerformanceInformationUtil.java
M infra/test-util/src/main/java/com/evolveum/midpoint/test/util/OperationResultAssert.java
M model/model-api/src/main/java/com/evolveum/midpoint/model/api/hooks/HookRegistry.java
M model/model-common/src/main/java/com/evolveum/midpoint/model/common/mapping/metadata/MetadataItemProcessingSpecImpl.java
M model/model-common/src/main/java/com/evolveum/midpoint/model/common/stringpolicy/AbstractValuePolicyOriginResolver.java
M model/model-common/src/main/java/com/evolveum/midpoint/model/common/util/ObjectTemplateIncludeProcessor.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/ModelObjectResolver.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ModelController.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/LensOwnerResolver.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/projector/focus/AutoAssignMappingCollector.java
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/projector/policy/scriptExecutor/LinkSourceFinder.java
M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/misc/AbstractMiscTest.java
M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/misc/TestMisc.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/operations/ProvisioningSearchLikeOperation.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/resourceobjects/ResourceObjectSearchOperation.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowSearchLikeOperation.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowsFacade.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/manager/ShadowFinder.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/task/MultiPropagationActivityRun.java
M provisioning/ucf-api/src/main/java/com/evolveum/midpoint/provisioning/ucf/api/ConnectorInstance.java
M provisioning/ucf-impl-connid/src/main/java/com/evolveum/midpoint/provisioning/ucf/impl/connid/ConnectorInstanceConnIdImpl.java
M provisioning/ucf-impl-connid/src/main/java/com/evolveum/midpoint/provisioning/ucf/impl/connid/SearchExecutor.java
M repo/repo-api/src/main/java/com/evolveum/midpoint/repo/api/RepositoryService.java
M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/RepositoryCache.java
M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/SearchOpHandler.java
M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/RepoObjectResolver.java
M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/activity/run/processing/ContainerableProcessingRequest.java
M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/activity/run/processing/ItemProcessingGatekeeper.java
M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/activity/run/processing/ProcessingCoordinator.java
M repo/repo-sqale/src/main/java/com/evolveum/midpoint/repo/sqale/SqaleRepositoryService.java
M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/SqlRepositoryServiceImpl.java
M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/helpers/ObjectRetriever.java
M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/AbstractIntegrationTest.java
A repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/ComponentsPerformanceInformationAsserter.java
A repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/OperationsPerformanceInformationAsserter.java
M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/quartz/TaskSynchronizer.java
Log Message:
-----------
Fix performance reporting by components
This commit fixes the reporting of performance by components, especially
concentrating on handling objects found in the iterative search methods.
See also the docs added to OperationResult (principle #4) and
ResultHandler#providingOwnOperationResult.
Work in progress.
Related to MID-10446.
Commit: e4072dab1a5d121ce03778b3aa2a621ed50c110f
https://github.com/Evolveum/midpoint/commit/e4072dab1a5d121ce03778b3aa2a621ed50c110f
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2025-02-18 (Tue, 18 Feb 2025)
Changed paths:
M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ModelController.java
M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/misc/TestMisc.java
M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/TestImportRecon.java
A model/model-intest/src/test/resources/misc/resource-dummy-perf.xml
M provisioning/provisioning-api/src/main/java/com/evolveum/midpoint/provisioning/api/ProvisioningService.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ProvisioningServiceImpl.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/resourceobjects/ResourceObjectConverter.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/resourceobjects/ResourceObjectSearchOperation.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowSearchLikeOperation.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowsFacade.java
M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/manager/ShadowFinder.java
M provisioning/ucf-impl-connid/src/main/java/com/evolveum/midpoint/provisioning/ucf/impl/connid/ConnectorInstanceConnIdImpl.java
M provisioning/ucf-impl-connid/src/main/java/com/evolveum/midpoint/provisioning/ucf/impl/connid/SearchExecutor.java
M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/RepositoryCache.java
M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/GetObjectOpHandler.java
M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/OperationsPerformanceInformationAsserter.java
Log Message:
-----------
Add more performance reporting tests
Now we test also searchObjectsIteratively and getObject operations
that are related to shadows (both fetch and noFetch mode).
Also, this commit fixes failing TestImportRecon by removing
assertions on operation results that were out of date (and not
strictly necessary).
Related to MID-10446.
Commit: 40dd84193fd84624c1e4f93d2a18caa868e366bd
https://github.com/Evolveum/midpoint/commit/40dd84193fd84624c1e4f93d2a18caa868e366bd
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2025-02-19 (Wed, 19 Feb 2025)
Changed paths:
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/TestSystemPerformance.java
Log Message:
-----------
Add dumping representative shadows in perf. test
Commit: 981e38f526836f673804d7f25491e7fe548a2931
https://github.com/Evolveum/midpoint/commit/981e38f526836f673804d7f25491e7fe548a2931
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2025-02-19 (Wed, 19 Feb 2025)
Changed paths:
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/ImportConfiguration.java
A testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/RandomSource.java
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/ReconciliationConfiguration.java
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/RolesConfiguration.java
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/SourceInitializer.java
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/SourcesConfiguration.java
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/TargetsConfiguration.java
Log Message:
-----------
Make TestSystemPerformance less random
In order to compare performance between various runs more exactly,
this commit reduces the variability of generated data.
Previously, the number of operations varied mainly because the number
of technical roles (and therefore, entitlements) could be different
amount of overlap between randomly-generated and randomly-assigned
business roles.
Now we reduce the randomness by using a generator with a fixed seed.
Commit: f9826c355b1da41c01bbd7db1e892bf8406cf1aa
https://github.com/Evolveum/midpoint/commit/f9826c355b1da41c01bbd7db1e892bf8406cf1aa
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2025-02-19 (Wed, 19 Feb 2025)
Changed paths:
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/SourcesConfiguration.java
M testing/story/src/test/java/com/evolveum/midpoint/testing/story/sysperf/TargetsConfiguration.java
Log Message:
-----------
Fix compilation errors
(Occurred during cherry-picking changes from master.)
Commit: 01c88db0ba9d9fc9da2b017268bf9356cc7cf4e1
https://github.com/Evolveum/midpoint/commit/01c88db0ba9d9fc9da2b017268bf9356cc7cf4e1
Author: Pavol Mederly <mederly at evolveum.com>
Date: 2025-02-20 (Thu, 20 Feb 2025)
Changed paths:
M docs/deployment/ninja/command/export-mining.adoc
M docs/roles-policies/mining/anonymous-data-export/index.adoc
M release-notes.adoc
Log Message:
-----------
Merge remote-tracking branch 'origin/support-4.8' into support-4.8
Compare: https://github.com/Evolveum/midpoint/compare/c70ad73dd6f2...01c88db0ba9d
To unsubscribe from these emails, change your notification settings at https://github.com/Evolveum/midpoint/settings/notifications
More information about the midPoint-svn
mailing list