Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] Gh page #234

Merged
merged 17 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 17 additions & 23 deletions .github/workflows/pages.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,13 @@ on:
# Runs on pushes targeting the default branch
push:
branches: [ $default-branch ]
paths:
- "docs/**"
pull_request:
branches: [ 'main', '2.x' ]
types: [ opened ]
paths:
- "docs/**"

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
Expand All @@ -33,6 +37,9 @@ jobs:
# Build and Deploy job
build_and_deploy:
runs-on: ubuntu-latest
defaults:
run:
working-directory: docs
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -42,33 +49,20 @@ jobs:
ruby-version: '3.1' # Not needed with a .ruby-version file
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
cache-version: 0 # Increment this number if you need to re-download cached gems
working-directory: '${{ github.workspace }}/docs'
- name: Setup Pages
id: pages
uses: actions/configure-pages@v4
- name: Bundle install in docs folder
run: |
cd docs
bundle install
run: bundle install
- name: Build Jekyll site
run: |
cd docs
bundle exec jekyll build --baseurl /rqueue
mkdir -p _site # Ensure _site directory exists
run: bundle exec jekyll build --baseurl "${{ steps.pages.outputs.base_path }}"
env:
JEKYLL_ENV: production
- name: Verify _site directory
run: |
cd docs
ls -l _site
- name: Upload artifact
# Automatically uploads an artifact from the './_site' directory by default
uses: actions/upload-artifact@v2
uses: actions/upload-pages-artifact@v2
with:
name: Jekyll site
path: docs/_site
path: 'docs/_site/'
- name: Deploy to GitHub Pages
uses: JamesIves/[email protected] # Replace with the appropriate deploy action
with:
branch: $default-branch
folder: docs/_site
cname: sonus21.github.io
env:
JEKYLL_ENV: production
BASE_URL: /rqueue
id: deployment
uses: actions/deploy-pages@v3
1 change: 1 addition & 0 deletions docs/.ruby-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.1.0
1 change: 1 addition & 0 deletions docs/Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ GEM

PLATFORMS
arm64-darwin-21
arm64-darwin-23
x86_64-linux

DEPENDENCIES
Expand Down
2 changes: 1 addition & 1 deletion docs/_includes/footer_custom.html
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Copyright &copy; 2019-{{ "now" | date: "%Y" }} Sonu Kumar. Distributed by an <a href="https://github.com/sonus21/rqueue/tree/main/LICENSE">Apache 2.0 license.</a>
Copyright &copy; 2019-{{ "now" | date: "%Y" }} Sonu Kumar. Distributed by an <a href="https://github.com/sonus21/rqueue/tree/main/LICENSE">Apache 2.0 license.</a> Build on {{ "now" | date:"%Y-%m-%d %H:%M" }}
7 changes: 0 additions & 7 deletions docs/configuration/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,6 @@ configuration or directly in the code.

{: .fs-6 .fw-300 }

## Table of contents

{: .no_toc .text-delta }

1. TOC
{:toc}

---
Apart from the basic configuration, Rqueue can be heavily customized, such as adjusting the number
of tasks executed concurrently. Additional configurations can be provided using
Expand Down
6 changes: 4 additions & 2 deletions docs/configuration/retry-and-backoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,9 @@ the `SimpleRqueueListenerContainerFactory`. The default back-off time is 5 secon

In scenarios where you don't want Rqueue to retry failures, you can handle this in two ways:

1. **Using `RqueueListener` Annotation**: Add exceptions to the `doNotRetry` list within
- **Using `RqueueListener` Annotation**: Add exceptions to the `doNotRetry` list within
the `RqueueListener` annotation. This instructs Rqueue not to retry for specific exceptions.

```java

public class MessageListener{
Expand All @@ -30,7 +31,8 @@ public class MessageListener{

}
```
2. **Returning `-2` from Execution Backoff Method**: Alternatively, you can return `-2` from the

- **Returning `-1` from Execution Backoff Method**: Alternatively, you can return `-1` from the
execution backoff method. This signals Rqueue to stop any further retry attempts for the failed
message.

Expand Down
4 changes: 4 additions & 0 deletions docs/dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,17 +84,21 @@ prefix: [http://localhost:8080/my-application/rqueue](http://localhost:8080/my-a
### Dashboard Screenshots

#### Latency Graph
<br/>

[![Latency Graph](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/stats-graph.png)](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/stats-graph.png)

#### Queue Statistics
<br/>

[![Queue Statistics](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/queues.png)](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/queues.png)

#### Tasks Waiting for Execution
<br/>

[![Tasks Waiting for Execution](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/queue-explore.png)](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/queue-explore.png)

#### Running Tasks
<br/>

[![Running Tasks](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/running-tasks.png)](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/running-tasks.png)
3 changes: 2 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ frameworks.
* **Queue Priority**: Supports both group-level and sub-queue level priorities.
* **Long Execution Jobs**: Check-in mechanism for long-running jobs.
* **Execution Backoff**: Supports exponential and fixed backoff strategies.
* **Do not retry**: Supports do not retry exceptions.
* **Do not retry**: Supports do not retry strategy.
* **Middleware**: Allows integration of middleware to intercept messages before processing.
* **Callbacks**: Supports callbacks for handling dead letter queues and discarding messages.
* **Events**: Provides bootstrap and task execution events.
Expand Down Expand Up @@ -304,6 +304,7 @@ the [GitHub repository][Rqueue repo] or contact the maintainers directly.
{: .fs-5 }

Rqueue is licensed under the Apache License 2.0. See the [LICENSE](https://github.com/sonus21/rqueue/blob/master/LICENSE) file for more details.

[Rqueue Docs]: https://github.com/sonus21/rqueue/wiki
[Boot Maven Central]: https://search.maven.org/artifact/com.github.sonus21/rqueue-spring-boot-starter
[Maven Central]: https://search.maven.org/artifact/com.github.sonus21/rqueue-spring
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,6 @@ void handle(JobImpl job,
case IGNORED:
handleIgnoredMessage(job, failureCount);
break;
case FAILED_IGNORED:
handleFailedIgnoredMessage(job, failureCount);
break;
case OLD_MESSAGE:
handleOldMessage(job, job.getRqueueMessage());
break;
Expand Down Expand Up @@ -283,7 +280,7 @@ private int getMaxRetryCount(RqueueMessage rqueueMessage, QueueDetail queueDetai

private void handleFailure(JobImpl job, int failureCount, Throwable throwable) {
if (job.getQueueDetail().isDoNotRetryError(throwable)) {
handleFailedIgnoredMessage(job, failureCount);
handleRetryExceededMessage(job, failureCount, throwable);
} else {
int maxRetryCount = getMaxRetryCount(job.getRqueueMessage(), job.getQueueDetail());
if (failureCount < maxRetryCount) {
Expand All @@ -292,8 +289,6 @@ private void handleFailure(JobImpl job, int failureCount, Throwable throwable) {
throwable);
if (delay == TaskExecutionBackOff.STOP) {
handleRetryExceededMessage(job, failureCount, throwable);
} else if (delay == TaskExecutionBackOff.DO_NOT_RETRY) {
handleFailedIgnoredMessage(job, failureCount);
} else {
parkMessageForRetry(job, null, failureCount, delay);
}
Expand All @@ -312,14 +307,4 @@ private void handleIgnoredMessage(JobImpl job, int failureCount) {
job.getQueueDetail().getName());
deleteMessage(job, MessageStatus.IGNORED, failureCount);
}

private void handleFailedIgnoredMessage(JobImpl job, int failureCount) {
log(
Level.DEBUG,
"Message {} failed & ignored, Queue: {}",
null,
job.getRqueueMessage(),
job.getQueueDetail().getName());
deleteMessage(job, MessageStatus.DISCARDED, failureCount);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -286,15 +286,10 @@ private boolean shouldRetry(long maxProcessingTime, int retryCount, int failureC
boolean doNoRetry = queueDetail.isDoNotRetryError(error);
// it should not be retried based on the exception list
if (doNoRetry) {
status = ExecutionStatus.FAILED_IGNORED;
return false;
}
// check if this should not be retried based on the backoff
long backOff = postProcessingHandler.backOff(rqueueMessage, userMessage, failureCount, error);
if (backOff == TaskExecutionBackOff.DO_NOT_RETRY) {
status = ExecutionStatus.FAILED_IGNORED;
return false;
}
return backOff != TaskExecutionBackOff.STOP;
}
return false;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ public enum ExecutionStatus {
THROTTLED,
DELETED,
FAILED,
FAILED_IGNORED,
IGNORED,
OLD_MESSAGE,
QUEUE_INACTIVE,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,6 @@ public interface TaskExecutionBackOff {
* should not be retried further.
*/
long STOP = -1;
/**
* Return this value, so that it will not retry post-failure, and it won't move to DLQ as well.
*/
long DO_NOT_RETRY = -2;

/**
* Return the number of milliseconds to wait for the same message to be consumed.
* <p>Return {@value #STOP} to indicate that no further enqueue should be made for the message.
Expand All @@ -44,7 +39,6 @@ public interface TaskExecutionBackOff {

/**
* Return the number of milliseconds to wait for the same message to be consumed.
* <p>Return {@value #DO_NOT_RETRY} to indicate that no further retry should be made</p>
* <p>Return {@value #STOP} to indicate message should be moved to DLQ if DLQ is set</p>
*
* @param message message that's fetched
Expand Down
Loading