Filter Support in Subscription (rest-api)

[gauravAshok]

https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html demonstrates a decent policy that the queueing server can provide.

Taking the above as inspiration and what we already have, lets go with:

The filters again will be defined only on headers. Not the body.
(Though body can also be used, if schema support is added, and there is a way to achieve partial & efficient deserialization)

We will assume headers to follow the java type Map<String, String>?.
(technically, headers follow MultiMap<String, String> but that complication can be tackled separately and later below)

Meaning of filter match?

If the filter condition matches, then only will the message be delivered to the endpoint.

What does a filter expression look like?

The filter expression can be seen in the form of
(a & b & c & .....), with optional negation of the whole expression.

where, a & b & c means 3 "conditions" which must all match.

A condition can be nested.
So, a filter may look like (a & (b & c) & d). But we can see, that it is equivalent to a & b & c & d making nesting not that useful. Nesting will be useful to represent "or" logic in one of the conditions.
Like, a & (b | c) & d, meaning I want to receive message matching condition a & b & d Or a & c & d.

We can choose to make the nesting arbitrarily deep like a & (b | (c & (e | f)) & g), but it arguably makes less maintainable for the user & more prone to logic errors. It is also to be seen how useful would this be if we support it. Complexity wise it shouldn't be too hard to implement, but we can start with just one level of nesting that allows us to incorporate "or" logic in our expressions.

So, we shall support the expressions like (a & (b | c) & (d | e) & (g | f | h) & x & ....)
A condition, will be defined as a pair containing key and its possible values or a predicate.

Condition Struct

{
    "key": "{key_name}",
    "op": "operator_name",
    "value": "operator args / value"
}

a null op, is equivalent to "in" operator.

operators that we can support: "startWith", "endsWith", "contains", "in", "exists". The equality operator can be simulated using in operator where the value is a array of length 1.

Examples

{
 "key" : "name",
 "op": "startsWith",
 "value": "my_prefix"
}

{
 "key" : "name",
 "op": "endsWith",
 "value": "my_suffix"
}

{
 "key" : "name",
 "op": "contains",
 "value": "some_part of the string"
}

{
 "key" : "name",
 "op": "in",
 "value": ["first_option", "second_option"]
}

{
 "key" : "name",
 "op": "exists"
}

Expression Struct

{
    "op": {binary operator: like AND, OR, NOT},
    "exp": [expression | condition]
}

Expression is a list of sub expressions or conditions along with a binary operator. If NOT, then the exp can only have a 1 sub expression or 1 condition.

[kmrdhruv]

Instead of restricting the nesting to one level, let the implementation be open for multi level nesting. Restrictions can be added via config at parsing level to restrict the depth of nesting.

[gauravAshok]

Instead of restricting the nesting to one level, let the implementation be open for multi level nesting. Restrictions can be added via config at parsing level to restrict the depth of nesting.

Yes, the model is not restrictive, and will be able to express most expectations.

[kmrdhruv]

for condition struct, can we evaluate "exists" operator as well i.e.checks for presence of header, not its value.

[gauravAshok]

for condition struct, can we evaluate "exists" operator as well i.e.checks for presence of header, not its value.

Added example for it.

[AayuStark007]

I'm assuming, that the exp field in Expression Struct can contain a list of Expression Structs or Condition Structs. Is that correct?

[gaurav-ashok]

I'm assuming, that the exp field in Expression Struct can contain a list of Expression Structs or Condition Structs. Is that correct?

Yes. or the mix of both. to represent a scenario of "exp": (C1 AND C2) OR C3. where "C1 AND C2" itself is a expression containing 2 conditions. C3 is a standalone condition.