Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Peeking from compressed files throws an Exception #5

Open
gixxi opened this issue Jul 7, 2021 · 5 comments
Open

Peeking from compressed files throws an Exception #5

gixxi opened this issue Jul 7, 2021 · 5 comments

Comments

@gixxi
Copy link

gixxi commented Jul 7, 2021

Hi there,

we implement a WAL (write ahead log) using your library. we are facing the issue, that peeking from a queue fails with an exception when the head is within a compressed queue file.
@hewrin
Copy link

hewrin commented Jul 8, 2021

Below is the error that we receive


2. Unhandled clojure.lang.Compiler$CompilerException
   Error compiling test/lambdaroyal/memory/eviction/test_wal_log_basics.clj at (54:1)

             Compiler.java: 7526  clojure.lang.Compiler/load
                      REPL:    1  user/eval19660
                      REPL:    1  user/eval19660
             Compiler.java: 7062  clojure.lang.Compiler/eval
             Compiler.java: 7025  clojure.lang.Compiler/eval
                  core.clj: 3206  clojure.core/eval
                  core.clj: 3202  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  657  clojure.core/apply
                  core.clj: 1965  clojure.core/with-bindings*
                  core.clj: 1965  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  243  clojure.main/repl/read-eval-print/fn
                  main.clj:  243  clojure.main/repl/read-eval-print
                  main.clj:  261  clojure.main/repl/fn
                  main.clj:  261  clojure.main/repl
                  main.clj:  177  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  748  java.lang.Thread/run

1. Caused by java.lang.IndexOutOfBoundsException
   Reader index error. index(8) < 0 || index(8)+length(682) >
   writerIndex(512)

        BytesAbstract.java:  494  io.appulse.utils.BytesAbstract/checkReaderBounds
      BytesFixedArray.java:  181  io.appulse.utils.BytesFixedArray/readBytes
            BytesPool.java:  326  io.appulse.utils.BytesPool$PooledBytes/readBytes
        BytesAbstract.java:  369  io.appulse.utils.BytesAbstract/readBytes
            BytesPool.java:  195  io.appulse.utils.BytesPool$PooledBytes/readBytes
        BytesAbstract.java:  363  io.appulse.utils.BytesAbstract/readBytes
            BytesPool.java:  195  io.appulse.utils.BytesPool$PooledBytes/readBytes
         Deserializer.java:  156  org.infobip.lib.popout.Deserializer$StringDeserializer/deserialize
         Deserializer.java:  151  org.infobip.lib.popout.Deserializer$StringDeserializer/deserialize
    ItemSerialization.java:   51  org.infobip.lib.popout.synced.ItemSerialization/deserialize
      SyncedFileQueue.java:  111  org.infobip.lib.popout.synced.SyncedFileQueue/lambda$peek$2
   ReadWriteBytesPool.java:   66  org.infobip.lib.popout.ReadWriteBytesPool/borrow
      SyncedFileQueue.java:  108  org.infobip.lib.popout.synced.SyncedFileQueue/peek
NativeMethodAccessorImpl.java:   -2  sun.reflect.NativeMethodAccessorImpl/invoke0
NativeMethodAccessorImpl.java:   62  sun.reflect.NativeMethodAccessorImpl/invoke
DelegatingMethodAccessorImpl.java:   43  sun.reflect.DelegatingMethodAccessorImpl/invoke
               Method.java:  498  java.lang.reflect.Method/invoke
            Reflector.java:   93  clojure.lang.Reflector/invokeMatchingMethod
            Reflector.java:  313  clojure.lang.Reflector/invokeNoArgInstanceMember
   test_wal_log_basics.clj:   54  lambdaroyal.memory.eviction.test-wal-log-basics/eval19676
   test_wal_log_basics.clj:   54  lambdaroyal.memory.eviction.test-wal-log-basics/eval19676
             Compiler.java: 7062  clojure.lang.Compiler/eval
             Compiler.java: 7514  clojure.lang.Compiler/load
                      REPL:    1  user/eval19660
                      REPL:    1  user/eval19660
             Compiler.java: 7062  clojure.lang.Compiler/eval
             Compiler.java: 7025  clojure.lang.Compiler/eval
                  core.clj: 3206  clojure.core/eval
                  core.clj: 3202  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  657  clojure.core/apply
                  core.clj: 1965  clojure.core/with-bindings*
                  core.clj: 1965  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  243  clojure.main/repl/read-eval-print/fn
                  main.clj:  243  clojure.main/repl/read-eval-print
                  main.clj:  261  clojure.main/repl/fn
                  main.clj:  261  clojure.main/repl
                  main.clj:  177  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  748  java.lang.Thread/run

@xxlabaza
Copy link
Collaborator

xxlabaza commented Jul 8, 2021

hi! can you please provide a code example? I'm going to reproduce it in the tests and fix the issue

@hewrin
Copy link

hewrin commented Sep 10, 2021

Hi, here is our code example

https://gist.github.com/hewrin/cdc4ab885f0e7bc09716f4e5291a115f

Attached also are the wal files that cause the issue.

wal2.zip

Thanks in advance

@gixxi
Copy link
Author

gixxi commented Nov 11, 2021

Hi Artem,

are you able to reproduce the issue, can we somehow support you providing a bugfix.

thanks in advance & kind regards

@aerovulpe
Copy link

I also had this issue. Flushing the queue before calling peek resolved it for me. I hope this helps anyone coming across this in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@xxlabaza @gixxi @aerovulpe @hewrin