This feature is disabled by default and so does not change any
behaviour for new or existing converted models; enabling this feature
will require changes to the converter to set the correct op flags.

Currently, binary convolutions expect full precision input, bitpack
that input, perform the computation, and then write full-precision
output.

This feature allows a binary convolutions to instead perform
bitpacking inside the kernel and write (8-bit) bitpacked output, and
for the subsequent binary convolution to skip the usual initial
bitpacking step and read the already-bitpacked input directly.
Doing this significantly reduces the number of reads and writes to
memory and the the overall memory footprint of the model inference.

Support is added only for the C++ kernels (x86 and Arm32). Support
for the optimised assembly kernels (Arm64) requires additional
future work.

…d-x86

…nel.

This reverts commit 51b14d1.

Co-Authored-By: Adam Hillier <[email protected]>

Co-Authored-By: Adam Hillier <[email protected]>