Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BatchedFileQueue does not persist all data on close #8

Open
ozivotsky opened this issue Sep 27, 2022 · 0 comments
Open

BatchedFileQueue does not persist all data on close #8

ozivotsky opened this issue Sep 27, 2022 · 0 comments

Comments

@ozivotsky
Copy link

BatchedFileQueue implementation does not persist head (reading cache) on flush and does not persist it even on close. The head of the queue is kept only in a memory and data are lost when the program exit.

JUnit test simulating the bug:

class BatchedFileQueueTests {

  @BeforeEach
  void beforeEach () {
    clearTestFiles();
  }

  @AfterEach
  void afterEach () {
    clearTestFiles();
  }

  @Test
  void close () {
    val builder = FileQueue.<Character>batched()
        .name("batched-queue-close")
        .restoreFromDisk(true)
        .folder(FOLDER)
        .serializer(Serializer.CHARACTER)
        .deserializer(Deserializer.CHARACTER)
        .wal(WalFilesConfig.builder()
            .maxCount(Integer.MAX_VALUE)
            .build())
        .batchSize(1000);

    val chars = RandomString.make(2000).toCharArray();
    try (val queue = builder.build()) {
      for (val character : chars) {
        queue.add(character);
      }
      val ch = queue.poll();
      queue.add(ch);

      assertThat(queue.size()).isEqualTo(chars.length);
    }

    try (val queue = builder.build()) {
      // AssertionFailedError:
      // Expecting: <1001> to be equal to: <2000> but was not.
      //  -  999 characters were lost !
      assertThat(queue.size()).isEqualTo(chars.length);
    }
  }
}

Suggested solution: flush also BatchedFileQueue.head in close() method the same way as the tail is flushed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant