Not able to create sessions cache

[samyakjain224]

Hi
I am integrating keycloak with infinispan, but when i am creating sessions cache on infinispan i am getting following error.

i am using following values.yaml

#Build configuration
images:
  server: quay.io/infinispan/server:latest
  initContainer: registry.access.redhat.com/ubi8-micro
#Deployment configuration
deploy:
  infinispan:
    cacheContainer:
      distributedCacheConfiguration:
        name: "sessions-cfg"
        mode: "SYNC"
        statistics: "true"
        locking:
          acquire-timeout: "0"
              mediaType: "application/x-jboss-marshalling"



  #Add a user with full security authorization.
  security:
    batch: "user create myuser -p qwer1234 -g admin"
  #Create a cluster with two pods.
  replicas: 2

getting the similar error when i an doing it with cache definations

Build configuration
images:
  server: quay.io/infinispan/server:latest
  initContainer: registry.access.redhat.com/ubi8-micro
#Deployment configuration
deploy:
  infinispan:
    cacheContainer:
      name: "keycloak"
      statistics: "true"
      caches:
        work:
          distributedCache:
            mode: "SYNC"
            statistics: "true"
            encoding:
              mediaType: "application/x-jboss-marshalling"
        sessions:
          distributedCache:
            mode: "SYNC"
            statistics: "true"
            encoding:
              mediaType: "application/x-jboss-marshalling"
        authenticationSessions:
          distributedCache:
            mode: "SYNC"
            statistics: "true"
            encoding:
              mediaType: "application/x-jboss-marshalling"
        offlineSessions:
          distributedCache:
            mode: "SYNC"
            statistics: "true"
            encoding:
              mediaType: "application/x-jboss-marshalling"
        clientSessions:
          distributedCache:
            mode: "SYNC"
            statistics: "true"
            encoding:
              mediaType: "application/x-jboss-marshalling"
        offlineClientSessions:
          distributedCache:
            mode: "SYNC"
            statistics: "true"
            encoding:
              mediaType: "application/x-jboss-marshalling"
        loginFailures:
          distributedCache:
            mode: "SYNC"
            statistics: "true"
            encoding:
              mediaType: "application/x-jboss-marshalling"
        actionTokens:
          distributedCache:
            mode: "SYNC"
            statistics: "true"
            encoding:
              mediaType: "application/x-jboss-marshalling"



  #Add a user with full security authorization.
  security:
    batch: "user create myuser -p qwer1234 -g admin"
  #Create a cluster with two pods.
  replicas: 2

in this case getting following error
�[1;31m2023-08-02 19:35:11,859 FATAL (main) [org.infinispan.SERVER] ISPN080028: Infinispan Server failed to start org.infinispan.manager.EmbeddedCacheManagerStartupException: ISPN000573: Cannot recreate persisted configuration for cache 'sessions' because configuration Configuration{simpleCache=false, clustering=[[mode=DIST_SYNC, remote-timeout=15000, invalidation-batch-size=128, bias-acquisition=ON_WRITE, bias-lifespan=300000], hash=[consistent-hash-factory=null, owners=2, segments=256, capacity-factor=1.0, key-partitioner=HashFunctionPartitioner{hashFunction=MurmurHash3, ns=256}], l1=[enabled=false, invalidation-threshold=0, l1-lifespan=600000, l1-cleanup-interval=60000], state-transfer=[enabled=true, timeout=240000, chunk-size=512, await-initial-transfer=true], partition-handling=[when-split=ALLOW_READ_WRITES, merge-policy=NONE]], customInterceptors=[interceptors=[]], encoding=[[media-type=application/x-protostream], key=[media-type=null], value=[media-type=null]], expiration=[lifespan=-1, max-idle=-1, reaperEnabled=true, interval=60000, touch=SYNC], query=[properties={}, default-max-results=100, hit-count-accuracy=10000], indexing=[properties={}, index=null, auto-config=false, key-transformers={}, indexed-entities=[], enabled=false, storage=filesystem, startup-mode=NONE, path=null], reader=[refresh-interval=0], writer=[[thread-pool-size=1, queue-count=1, queue-size=null, commit-interval=null, ram-buffer-size=null, max-buffered-entries=null, low-level-trace=false], index-merge=[max-entries=null, factor=null, min-size=null, max-size=null, max-forced-size=null, calibrate-by-deletes=null]], invocationBatching=[enabled=false], locking=[concurrency-level=32, isolation=REPEATABLE_READ, acquire-timeout=10000, striping=false], memory=[storage=HEAP, max-size=null, max-count=-1, when-full=NONE], modules={}, persistence=[passivation=false, availability-interval=1000, connection-attempts=10, connection-interval=50], stores=[], security=[authorization=[enabled=true, roles=[observer]]], sites=[[merge-policy=org.infinispan.xsite.spi.DefaultXSiteEntryMergePolicy@49153009, max-cleanup-delay=30000, tombstone-map-size=512000], backups=[], backup-for=[remote-cache=null, remote-site=null]], statistics=[statistics=true, statistics-available=true], transaction=[[auto-commit=true, stop-timeout=30000, locking=OPTIMISTIC, transaction-manager-lookup=org.infinispan.transaction.lookup.GenericTransactionManagerLookup@a1b7549, transaction-synchronization-registry-lookup=null, mode=NON_TRANSACTIONAL, synchronization=false, single-phase-auto-commit=false, reaper-interval=30000, complete-timeout=60000, notifications=true], recovery=[enabled=false, recovery-cache=__recoveryInfoCacheName__]], unsafe=[unreliable-return-values=false], template=false} is incompatible with the existing configuration Configuration{simpleCache=false, clustering=[[mode=DIST_SYNC, remote-timeout=17500, invalidation-batch-size=128, bias-acquisition=ON_WRITE, bias-lifespan=300000], hash=[consistent-hash-factory=null, owners=2, segments=256, capacity-factor=1.0, key-partitioner=HashFunctionPartitioner{hashFunction=MurmurHash3, ns=256}], l1=[enabled=false, invalidation-threshold=0, l1-lifespan=600000, l1-cleanup-interval=60000], state-transfer=[enabled=true, timeout=60000, chunk-size=512, await-initial-transfer=true], partition-handling=[when-split=ALLOW_READ_WRITES, merge-policy=NONE]], customInterceptors=[interceptors=[]], encoding=[[media-type=application/x-protostream], key=[media-type=null], value=[media-type=null]], expiration=[lifespan=-1, max-idle=-1, reaperEnabled=true, interval=60000, touch=SYNC], query=[properties={}, default-max-results=100, hit-count-accuracy=10000], indexing=[properties={}, index=null, auto-config=false, key-transformers={}, indexed-entities=[], enabled=false, storage=filesystem, startup-mode=NONE, path=null], reader=[refresh-interval=0], writer=[[thread-pool-size=1, queue-count=1, queue-size=null, commit-interval=null, ram-buffer-size=null, max-buffered-entries=null, low-level-trace=false], index-merge=[max-entries=null, factor=null, min-size=null, max-size=null, max-forced-size=null, calibrate-by-deletes=null]], invocationBatching=[enabled=false], locking=[concurrency-level=1000, isolation=REPEATABLE_READ, acquire-timeout=15000, striping=false], memory=[storage=HEAP, max-size=null, max-count=-1, when-full=NONE], modules={}, persistence=[passivation=false, availability-interval=1000, connection-attempts=10, connection-interval=50], stores=[], security=[authorization=[enabled=false, roles=[]]], sites=[[merge-policy=org.infinispan.xsite.spi.DefaultXSiteEntryMergePolicy@49153009, max-cleanup-delay=30000, tombstone-map-size=512000], backups=[], backup-for=[remote-cache=null, remote-site=null]], statistics=[statistics=true, statistics-available=true], transaction=[[auto-commit=true, stop-timeout=30000, locking=OPTIMISTIC, transaction-manager-lookup=org.infinispan.transaction.lookup.GenericTransactionManagerLookup@a1b7549, transaction-synchronization-registry-lookup=null, mode=NON_TRANSACTIONAL, synchronization=false, single-phase-auto-commit=false, reaper-interval=30000, complete-timeout=60000, notifications=true], recovery=[enabled=false, recovery-cache=__recoveryInfoCacheName__]], unsafe=[unreliable-return-values=false], template=false} at org.infinispan.manager.DefaultCacheManager.internalStart(DefaultCacheManager.java:781) at org.infinispan.manager.DefaultCacheManager.start(DefaultCacheManager.java:746) at org.infinispan.server.SecurityActions.lambda$startCacheManager$1(SecurityActions.java:68) at org.infinispan.security.Security.doPrivileged(Security.java:56) at org.infinispan.server.SecurityActions.doPrivileged(SecurityActions.java:40) at org.infinispan.server.SecurityActions.startCacheManager(SecurityActions.java:71) at org.infinispan.server.Server.run(Server.java:408) at org.infinispan.server.Bootstrap.runInternal(Bootstrap.java:173) at org.infinispan.server.tool.Main.run(Main.java:98) at org.infinispan.server.Bootstrap.main(Bootstrap.java:56) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.infinispan.server.loader.Loader.run(Loader.java:106) at org.infinispan.server.loader.Loader.main(Loader.java:51)

i am deploying this on my local machine using minikube using following command

 helm install infinispan openshift-helm-charts/infinispan-infinispan --values values.yaml

can anyone help me what i am doing wrong.?

[ryanemerson]

In the second example you have defined the "sessions" cache declaratively, so it's created on server startup, which is why it's not possible to create it again via the Infinispan console.

The Infinispan helm chart uses a StatefulSet to manage the deployment. I suggest that you make sure that all associated PVC instances are removed between deployment attempls. Old configuration state may be loaded if you have created multiple deployments with the same name and not cleaned up old PVCs between removal/installation.

[samyakjain224]

HI thanks @ryanemerson , this seems to resolved the issue, i have some old PVC which was causing the issue and they were not displaying on UI so hence the difficulty to identify it.

After removing the old PVC sessions caches are getting created but not now i am getting following error

• when i am using UI to see the cached data it keeps loading , same thing is happening for infinispan APIs.

• 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling. PVC has been created i have verified it on cluster

• Startup probe failed: Get "http://172.17.0.5:11222/rest/v2/cache-managers/default/health/status": dial tcp 172.17.0.5:11222: connect: connection refused

seems default health status not working i am getting HEALTHY status when i am using following url
rest/v2/cache-managers/keycloak/health/status
this is causing only one node to come up.

thanks again.

[ryanemerson]

• 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available:

The PVC instance have been created, but you don't have an PersistentVolume (PV) available for the PVC to be bound to:

https://stackoverflow.com/questions/74741993/0-1-nodes-are-available-1-pod-has-unbound-immediate-persistentvolumeclaims

[samyakjain224]

ok cool thanks, Just one more thing any idea about 2nd issue i am getting i just changed the cacheContainer name this seems to the reason for health check to fail how can i fix it

[ryanemerson]

Two choices:

1. Change the name of your cacheContainer back to "default"
2. Update the probe settings in the helm templates: https://github.com/infinispan/infinispan-helm-charts/blob/main/templates/statefulset.yaml#L85-L115

[samyakjain224]

@ryanemerson ok ,thanks a lot..!!

[samyakjain224]

@ryanemerson i am using the following values.yml , but when infinispna cluster is restarting all the cache data is deleted, as per documentation ephemeral flag is set to false by default. i can also see that infinispan pod is binded to PV , why on restart infinispan not able to get on the previous state.

`images:
server: quay.io/infinispan/server:latest
initContainer: registry.access.redhat.com/ubi8-micro
deploy:
infinispan:
cacheContainer:
name: "keycloak"
statistics: "true"
caches:
work:
replicatedCache:
mode: "SYNC"
statistics: "true"
encoding:
mediaType: "application/x-jboss-marshalling"
sessions:
distributedCache:
mode: "SYNC"
statistics: "true"
encoding:
mediaType: "application/x-jboss-marshalling"
authenticationSessions:
distributedCache:
mode: "SYNC"
statistics: "true"
encoding:
mediaType: "application/x-jboss-marshalling"
offlineSessions:
distributedCache:
mode: "SYNC"
statistics: "true"
encoding:
mediaType: "application/x-jboss-marshalling"
clientSessions:
distributedCache:
mode: "SYNC"
statistics: "true"
encoding:
mediaType: "application/x-jboss-marshalling"
offlineClientSessions:
distributedCache:
mode: "SYNC"
statistics: "true"
encoding:
mediaType: "application/x-jboss-marshalling"
loginFailures:
distributedCache:
mode: "SYNC"
statistics: "true"
encoding:
mediaType: "application/x-jboss-marshalling"
actionTokens:
distributedCache:
mode: "SYNC"
statistics: "true"
owners: "2"
encoding:
mediaType: "application/x-jboss-marshalling"

replicas: 2
security:
authentication: true
secretName: infinispan-credentials
`

could you please help here.?

[ryanemerson]

@samyakjain224 You haven't defined persistence on your caches, which means there content is stored in-memory only.

Infinispan supports different persistence integrations, most notably file-based and JDBC. File-based stores are the simplest to use, as a store is just created per pod, however in the event of pod restart stale values for a given key may be reloaded from the store if the key was updated on a live pod. JDBC stores workaround the aforementioned limitation by using shared=true which means that a single DB table is used to persist updates from all Infinispan pods.

If the limitations of file-based stores are acceptable, you can update your cache configurations to include the following:

sessions:
  distributedCache:
    mode: "SYNC"
    statistics: "true"
    encoding:
      mediaType: "application/x-jboss-marshalling"
   persistence:
     fileStore: ~

For more information and additional configuration options, please see the Infinispan Docs.

[samyakjain224]

ok thanks