Add pagination to event queries

[Pascualex]

Preflight Checklist

• I have searched the issue tracker for an issue that matches the one I want to file, without success.

Problem Description

When retrieving a list of events trough the API you need to specify a limit for the number of elements returned. This can work fine for very constrained queries, but doesn't scale to high numbers of results.

Proposed Solution

Add pagination parameters to the query (e.g. page-size and page) to be able to retrieve specific parts of the query result.

Alternatives Considered

With the current API, it should be possible to implement infinite scrolling by querying for the events older than the last event retrieved. However, this seems hacky and doesn't support other types of pagination (like jumping directly to page n).

Additional Information

Although the existing /query endpoint is a great way to power visual dashboards, we believe that being able to properly navigate disaggregated usage records is equally important and currently lacking.

[hekike]

@Pascualex, thanks for opening the issue. We are planning to improve pagination for the events endpoint. Initially, this endpoint was meant to be used for debugging only and to show the last couple of events.

What is your use-case for getting more historical events?

[Pascualex]

Hi @hekike, sorry for the late response. We were evaluating using OpenMeter as the main source of usage record data in our console, which is available to our customers. This meant trying to include not only information oriented to understand billing, but also to understand API usage and the obtained results in general. With that in mind, we wanted to make navigation and exploration of these records flexible and pagination needed for that.

Since this doesn't seem to be the focus of OpenMeter, we have for now discarded it to fulfill that role. However, we are still considering it for its main metering use case of powering billing and limiting usage through entitlements.

[hekike]

Hi @Pascualex, we added pagination for the raw event API to OpenMeter Cloud but didn't bring the change back yet to the OSS API as it requires a breaking change. If it helps, we can prioritize it and introduce pagination to OSS API.

Would pagination be enough or do you need to filter events by arbitary properties?