persist starlark state for more starlark types

[gazpachoking]

Use Case

#15170 added the ability for the starlark processor to persist the global state to the statefile between runs. This is very nice, but it can't store lists, dicts, tuples, or metrics to the state file which severely limits its usefulness. If you try any more complex type you get an error like invalid starlark type *starlark.Metric when it tries to write the state file.

My desired use case is similar to the example for comparing a metric to the previous one. This doesn't work because it stores both a dict and a metric in the global state.

Expected behavior

Starlark standard types (lists, dicts, tuples, and maybe also metrics) can be persisted to the statefile.

Actual behavior

Error when persisting the plugin states.

Additional info

No response

[srebhan]

Next steps: implement a function to convert starlark values to go-values and vice versa.

[knollet]

Hi, the statefile saves json, and you can in fact save lists, dicts, strings, numbers, booleans and Nonetype (None, null). (I will call this "simple-type")

To allow for storing arbitrary types could be a problem, because not every object can be stored (Resource handles of any kind (open files, handles for secrets, etc) come to mind).

I would recommend to implement "simple-type"-serializers and deserializers (marshallers and unmarshallers) for every type (or Metric at least) which should be storable.

So have something like
Metric.serialize() -> dict # {"name": ..., "tags": { ... }, "fields": { ... }, "time": ...}
and
Metric.__init__(marsh: dict) -> Metric

Also: state could have an interface which denies everything which isn't a "simple-type", so it would error out before it tries to write it to the statefile.

[srebhan]

The metric should be serialized as gob and encoded as base64 to be able to distinct integer/float types which is not possible with pure JSON.