Custom Retention of metrics

[rajeshksv]

Hi,

We use OpenTSDB as multi tenant (1000+ tenants) metric store and Grafana for visualisation. Some tenants need retention of 1 month and some need 2 months and so on. Due to storage restrictions, we need to delete data post retention period. Can we have custom (tiered ?) retention of metrics in Opentsdb ?

(Since retention is not uniform across all tenants, setting TTL on HBase Table won't solve our problem)

[rajeshksv]

Bump

[johann8384]

There isn't a way to do this built in, as far as I know.

[rajeshksv]

If it's not there, we are interested in building the feature into OpenTSDB. Let us know your thoughts if this use case aligns with your roadmap.

1. User sets "TTL meta tag" while emitting metric. Ex: __TTL__ = 3600
2. OpenTSDB parses TTL meta tag and sets TTL at HBase row level. This overrides HBase table level TTL (if any)

Couple of notes

• Typically, "User" would be Metrics Agent or Telemetry Pipeline. They set TTL based on some policy. Apps can also set it while emitting metric directly but may be tedious to do so.
• Seems like AsyncHBase client doesn't support setting TTL directly unlike official HBase Java client. So this has to be patched as well (which passes TTL as attribute in the reqeust)
• Meta tags will be stripped off from user payload before writing to HBase.

Effort wise, it doesn't seem very complex to do. Let us know your thoughts.

[rajeshksv]

Bump

[johann8384]

It sounds like a good idea to me.