Why does Mongo.Collection.dump/1 always call filter_nils ?

[rvnash]

In order to save a document represented by an Elixir struct, it must be past through dump/1 to strip out the struct key/values. That seems ok. But then why does that function always strip out all the keys where the value is nil? It seems like it shouldn't do this, or at least it should be optional.

[zookzook]

Filtering makes the documents smaller and thus consumes less space on the server. In each document, the keys are stored; if they have no value, it would make sense not to store them unnecessarily. The smaller the document the faster the network communication. What advantage would provide the storing attributes with nil?

[rvnash]

In my application there is another piece of software written in NodeJS that is accessing the same MongoDB. That application uses the existence of a key in the document as a flag to do something. I can't change that application. So I would really like to have control over the keys in the document, and not have my driver strip them out unconditionally. I understand the reasoning here. But it seems the driver is making a decision about the data which should really be with the client. It sure would be nice if it were at least an option. Thanks for answering!

[zookzook]

Interesting use case. I need some time to check how it could work to be optional. I use macros to create very fast load and dump functions.

[zookzook]

As a workaround, you can add the removed attributes after the dump function.

[rvnash]

Yeah, that would work. What I did do is just created a simple version of your dump macros as a straight forward function. Here is the code if it is of any use to you.

  @doc """
  dump function that does not strip nils
  """
  def mdump(%{__struct__: Date} = data), do: data
  def mdump(%{__struct__: DateTime} = data), do: data
  def mdump(%{__struct__: NaiveDateTime} = data), do: data
  def mdump(%{__struct__: Time} = data), do: data
  def mdump(%{__struct__: BSON.ObjectId} = data), do: data
  def mdump(%{__struct__: BSON.Binary} = data), do: data
  def mdump(%{__struct__: BSON.Regex} = data), do: data
  def mdump(%{__struct__: BSON.JavaScript} = data), do: data
  def mdump(%{__struct__: BSON.Timestamp} = data), do: data
  def mdump(%{__struct__: BSON.LongNumber} = data), do: data

  def mdump(%{__struct__: _} = struct) do
    mdump(Map.from_struct(struct))
  end

  def mdump(list) when is_list(list) do
    Enum.map(list, fn x -> mdump(x) end)
  end

  def mdump(map) when is_map(map) do
    Map.to_list(map)
    |> Enum.map(fn {k, v} -> {k, mdump(v)} end)
    |> Map.new()
  end

  def mdump(data), do: data