[midPoint-git] [Evolveum/midpoint] b321f1: Improve the repository cache

[mederly]

  Branch: refs/heads/feature/no-metadata-repo-cache
  Home:   https://github.com/Evolveum/midpoint
  Commit: b321f109078e60a8e3ffa599d2c5e64829d12161
      https://github.com/Evolveum/midpoint/commit/b321f109078e60a8e3ffa599d2c5e64829d12161
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2025-01-27 (Mon, 27 Jan 2025)

  Changed paths:
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/GetOperationOptions.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/SearchResultList.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/SearchResultMetadata.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/constants/ObjectTypes.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/global/AbstractGlobalCache.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/global/GlobalCacheObjectValue.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/global/GlobalCacheQueryValue.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/global/GlobalQueryCache.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/BaseOpHandler.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/CacheAccessInfo.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/CacheSetAccessInfo.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/CacheSetAccessInfoFactory.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/CacheUpdater.java
    A repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/CacheUseMode.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/CachedOpExecution.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/CachedOpHandler.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/CachingResultHandler.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/GetObjectOpExecution.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/GetObjectOpHandler.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/GetVersionOpExecution.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/GetVersionOpHandler.java
    R repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/PassReason.java
    A repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/RecordingResultHandler.java
    R repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/ReportingResultHandler.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/SearchOpExecution.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/handlers/SearchOpHandler.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/invalidation/ChangeDescription.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/invalidation/Invalidator.java
    A repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/local/LocalCacheObjectValue.java
    A repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/local/LocalCacheQueryValue.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/local/LocalObjectCache.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/local/LocalQueryCache.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/local/QueryKey.java
    A repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/local/SingleTypeQueryKey.java
    A repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/values/CachedObjectValue.java
    A repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/values/CachedQueryValue.java
    R repo/repo-cache/src/test/java/com/evolveum/midpoint/repo/cache/TestRepositoryCache.java

  Log Message:
  -----------
  Improve the repository cache

This commit aims to improve repo cache functionality and performance:

1. Allowing to return data that were requested with "exclude" retrieval
option (if they are present in the cache). This is needed e.g. when
serving queries from AssignmentTargetSearch (full exclusion).

2. Allowing to return data that were requested with "include" retrieval
option (if they are present in the cache, and do not contain any
incomplete items). This implements MID-6003.

3. Splitting query cache into per-type caches, aiming to more efficient
invalidation: we no longer need to iterate through all queries, but only
to ones for relevant types (supertypes + given type + subtypes).

The exclude/include items handling (plus existing zero-staleness option
handling) was implemented by replacing PassReason with CacheUseMode,
facilitating separate "can use cached data" and "can update cache"
conditions.

Work in progress.


  Commit: 83169d49d42e4d39be6ecca395c2d5cc07b6ab4a
      https://github.com/Evolveum/midpoint/commit/83169d49d42e4d39be6ecca395c2d5cc07b6ab4a
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2025-01-27 (Mon, 27 Jan 2025)

  Changed paths:
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/AbstractLazilyInitializableShadowedEntity.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/AssociationsHelper.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowAcquisition.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowPostProcessor.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowedChange.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/manager/ShadowFinder.java

  Log Message:
  -----------
  Avoid extra SQL access for embedded shadows

Embedded shadows (used for associations and reference attributes)
do not need "operationExecution" information. So, we exclude it
from the respective repo search operations.

(The back side is that the repository cache is avoided. But we assume
that shadows are usually not cached anyway.)

This should improve performance of provisioning "get" and "search"
operations when many entitlements are present.


Compare: https://github.com/Evolveum/midpoint/compare/b321f109078e%5E...83169d49d42e

To unsubscribe from these emails, change your notification settings at https://github.com/Evolveum/midpoint/settings/notifications