Message Count widget display is inconsistent for search range All Time and default interval Auto

[damianharouff]

When selecting All Time for a given Search (some of which default to All Time, e.g. Show Received Messages for an input), the Message Count widget display is inconsistent when left on the default Group By Interval = Auto.

This is easiest reproduced via Show Received Messages for an input, which defaults to All Time and Message Count Interval = Auto.

Example 1: Show Received Messages for an input: One large graph bar spanning the entire date range and message count:

Example 2: Show Received Messages for an input: Nothing displayed in the Message Count widget:

Example 3: Show Received Messages for an input: One large graph bar spanning the entire date range and message count:

Example 4: Show Received Messages for an input: Nothing displayed in the Message Count widget:

Example 5: Default search, all time, all streams: One large graph bar spanning the entire date range and message count:

In all examples, clicking the edit pencil icon for the Message Count widget, unchecking Auto for Interval -> Group By, then specifying an interval produces a plausible display. Example 5 above, with the Message Count widget modified to 1 Day:

Noted via HS-1480713908: Graylog 5.0.2+59d96f8 on (Eclipse Adoptium 17.0.5 on Linux 4.18.0-425.10.1.el8_7.x86_64)
Confirmed in Graylog 5.0.5+d61a926 (Eclipse Adoptium 17.0.6 on Linux 5.10.0-21-amd64)
Confirmed in Graylog 5.1.0-SNAPSHOT+24c5e8d (Eclipse Adoptium 17.0.6 on Linux 5.10.165-143.735.amzn2.x86_64)

[luk-kaminski]

@damianharouff , could you please confirm that it happens only for relatively new environments, with data that are not older than 1 month?

(The problem seems to be in a fact that "All time" means from 1970 till now in GL. For those 53 years GL picks the longest acceptable interval of 1 month. In relatively new installations you probably have only data for up to 1 month, and for some inputs no data at all. That's why you get either 1 bucket or no bucket at all, resulting in 1 or 0 bars on the chart. In order to improve it, we would probably need to figure out the timestamp of the oldest message for the query, either by introducing additional query to OS/ES in the backend, or maintaining some kind of Periodical that collects those timestamps... so that we do not start with 1970...)

[damianharouff]

@luk-kaminski In my lab, picked one of my inputs that writes into indexes with a 90 day retention, seems to produce a plausible display:

This is Graylog 5.1.3+a017005. Indexes for this input are a mix of several different retention periods and methods.

I've been in customer environments where indexes range calculations are wrong, and some indexes will show ranges like "messages from 53 years ago to a few seconds ago". I just checked all of my lab and personal install indexes and they all have the correct ranges on them; might that be a contributor to this situation?

[luk-kaminski]

@damianharouff -> "messages from 53 years ago to a few seconds ago" seems odd, but your screenshot in the issue have correct "All time" to "Now" ranges, and still you end up with either no or only 1 bucket.

So the main contributor to a situation still seems to be combination of:

• long time range,
• no data at all or data present only for the small part of time.

FYI - #15940 fixes the problem, but introduces a small side effect that we do not know if is acceptable or not. Discussion is ongoing. We will return to you when we know more.