StructuredIO

StructuredIO resembles Elixir’s IO module. The difference is that whereas IO exposes a freeform stream of bytes or lines of data, StructuredIO guarantees that only complete data elements are returned from its reader functions. This simplifies your application logic with respect to fragmentary input.

There are two main features of this library:

1. It provides a stateful process with a writer function for writing binary data. (No big deal since IO gives you that.)
2. It provides a variety of reader functions for conditional reading according to a specified data structure. Virtually any wire format can be specified, including binary encodings, nested and flat markup, and delimited data. If a complete data element has not (yet) been written to the process, nothing is read.

See what’s changed lately by reading the project history.

Usage

Here’s a contrived example that shows how to write and read structured data using the StructuredIO.write and .read_* functions. This example depicts Unicode data, but binary data of any kind can be written and read, too. See the API reference for detailed examples.

iex> {:ok,
...>  structured_io} = StructuredIO.start_link(:unicode)

Now we have a running StructuredIO process that expects properly encoded Unicode data.

iex> StructuredIO.write structured_io,
...>                    "  <p>foo</p"
:ok

We’ve written some markup to the process. Note that the <p> element is preceded by whitespace and is not properly closed.

iex> StructuredIO.read_across structured_io,
...>                          "<p>",
...>                          "</p>"
""

No <p> element is read because the available data in the process doesn’t begin with a <p>.

iex> StructuredIO.read_to structured_io,
...>                      "<p>"
"  "
iex> StructuredIO.read_across structured_io,
...>                          "<p>",
...>                          "</p>"
""

We managed to get past the whitespace, but no <p> element is read because the available data in the process doesn’t contain a complete element.

iex> StructuredIO.write structured_io,
...>                    "><hr /><p>bar</p>"
:ok

Now the first element is properly closed, and a second complete element has been written to the process.

iex> StructuredIO.read_across structured_io,
...>                          "<p>",
...>                          "</p>"
"<p>foo</p>"
iex> StructuredIO.read_through structured_io,
...>                           "<hr />"
"<hr />"
iex> StructuredIO.read_between structured_io,
...>                           "<p>",
...>                           "</p>"
"bar"

We’ve read one element at a time from the available data in the process. The read operations demonstrate both seeking and skipping.

iex> StructuredIO.read_across structured_io,
...>                          "<p>",
...>                          "</p>"
""

No more elements can be read unless more data is written to the process.

iex> collector = StructuredIO.collect(structured_io)
iex> ["<p>baz</p>",
...>  "<p>qux</p>",
...>  "<p>quux</p>"]
...> |> Enum.into(collector)
iex> structured_io
...> |> StructuredIO.enumerate_with(:read_between,
...>                                "<p>",
...>                                "</p>")
...> |> Enum.map(&String.upcase/1)
["BAZ",
 "QUX",
 "QUUX"]

The StructuredIO.collect function returns a struct that implements Elixir’s Collectable protocol, which lets you pipe data into the process instead of performing individual write operations. Likewise, the StructuredIO.enumerate_with function returns a struct that implements Elixir’s Enumerable protocol, which lets you pipe data elements out of the process instead of performing individual read operations.

iex> StructuredIO.stop structured_io
:ok

Don’t forget to stop the process when you’re finished with it.

You’ll find more detailed examples in the documentation for the StructuredIO module.

Installation

Install the Hex package by adding :structured_io to the list of dependencies in your project’s mix.exs file:

# mix.exs

# ...
def deps do
  [
    {:structured_io, "~> 1.5.0"}
  ]
end
# ...

Contributing

To submit a patch to the project:

1. Fork the official repository.
2. Create your feature branch: git checkout -b my-new-feature.
3. Commit your changes: git commit -am 'Add some feature'.
4. Push to the branch: git push origin my-new-feature.
5. Create a new pull request.

After cloning the repository, mix deps.get to install dependencies. Then mix test to run the tests. You can also iex -S mix to get an interactive prompt that will allow you to experiment. To build this package, mix hex.build.

To release a new version:

1. Update the project history in History.md, and then commit.
2. Update the version number in mix.exs respecting Semantic Versioning, update the “Installation” section of this readme to reference the new version, and then commit.
3. Build and publish the Hex package with mix hex.publish.
4. Tag with a name like vMAJOR.MINOR.PATCH corresponding to the new version, and then push commits and tags.

License

Released under the MIT License.