PCA-B-Stream: Byte Stream Representation of Piecewise-constant Array

Documentation

The documentation is available below.

Installation

A Web app of PCA-B-Stream might be available on PythonAnywhere.

This project is published on the Python Package Index (PyPI) at: _. It should be installable from Python distribution platforms or Integrated Development Environments (IDEs). Otherwise, it can be installed from a command console using pip:

Brief Description

In a Few Words

The PCA-B-Stream project allows to generate a printable byte stream representation of a piecewise-constant Numpy array, and to re-create the array from the byte stream, similarly to what is available as part of the COCO API.

Illustration

In Python:

>>> import pca_b_stream as pcas
>>> import numpy as nmpy
>>> # --- Array creation
>>> array = nmpy.zeros((10, 10), dtype=nmpy.uint8)
>>> array[1, 1] = 1
>>> # --- Array -> Byte stream -> Array
>>> stream = pcas.PCA2BStream(array)
>>> decoding = pcas.BStream2PCA(stream)
>>> # --- Check and print
>>> assert nmpy.array_equal(decoding, array)
>>> print(stream)
b'FnmHoFain+3jtU'

From command line:

pca2bstream some_image_file           # Prints the corresponding byte stream
bstream2pca a_byte_stream a_filename  # Creates an image from the byte stream and stores it

Motivations

The motivations for developing an alternative to existing solutions are:

• Arrays can be of any dimension (i.e., not just 2-dimensional),
• Their dtype can be of boolean, integer, or float types,
• They can contain more than 2 distinct values (i.e., non-binary arrays) as long as the values are integers (potentially stored in a floating-point format though),
• The byte stream representation is self-contained; In particular, there is no need to keep track of the array shape externally,
• The byte stream representation contains everything needed to re-create the array exactly as it was instantiated (dtype, endianness, C or Fortran ordering); See note though.

Documentation

Functions

The pca_b_stream module defines the following functions:

• PCA2BStream

  • Generates the byte stream representation of an array; Does not check the array validity (see PCArrayIssues)
  • Input: a Numpy ndarray
  • Output: an object of type bytes

• BStream2PCA

  • Re-creates the array from its bytes stream representation; Does not check the stream format validity
  • Input/Output: input and output of PCA2BStream swapped

• PCArrayIssues

  • Checks whether an array is a valid input for stream representation generation; It is meant to be used before calling PCA2BStream
  • Input: a Numpy ndarray
  • Output: a tuple issues in str format. The tuple is empty if the array is valid.
  • Additional information about what are valid piecewise-constant arrays here is provided in the section "Motivations".

• BStreamDetails

  • Extract details from a byte stream representation; See section "Byte Stream Format"

  • Inputs:

    • a byte stream generated by PCA2BStream

    • details: a string where each character corresponds to a detail to extract, or "+" to extract all of the available details; Default: "+"; Available details are:

      • c: compression indicator

      • d: array dimension

      • l: array lengths per dimension

      • t: dtype type code; See:

      • T: dtype name; Translated from "t" by Numpy.sctypeDict

      • o: enumeration order; See: , ?_CONTIGUOUS

      • e: endianness (or byte order); See:

    • should_print: a boolean to instruct whether the extracted details should be printed to console; Defaults: False

    • should_return: a boolean to instruct whether the extracted details should be returned (see Outputs); Defaults: True

  • Output: either one of:

    • None if should_return is False
    • a dictionary of all of the available details if the details parameter is "+"
    • a tuple of the requested details in the same order as in the details parameter

Command Line Scripts

The PCA-B-Stream project defines two command line scripts: pca2bstream and bstream2pca. The former takes a path to an image file as argument, and prints the corresponding byte stream (without the "b" string type prefix). The latter takes a character string and a filename as arguments, in that order, and creates an image file with this name that corresponds to the string interpreted as a byte stream. The file must not already exist.

Byte Stream Format

A byte stream is a base85-encoded stream. Once decoded, it has the following format (in lexicographical order; all characters are in bytes format):

• one character "0" or "1": indicates whether the remaining of the stream is in uncompressed or ZLIB compressed format; See note on compression; The remaining of the description applies to the stream in uncompressed format

• 3 characters "{E}{T}{O}":

  • E: endianness among "|", "<" and ">"
  • T: dtype character code among: "?" + numpy.typecodes["AllInteger"] + numpy.typecodes["Float"]
  • O: enumeration order among "C" (C-ordering) and "F" (Fortran-ordering)

• one integer for the dimension of the array (1 for vectors, 2 for matrices, 3 for volumes...)

• one integer per dimension giving the length of the array in that dimension

The remaining of the stream is the actual array content.

• If the array is not all False's or zeros:

  • one character "0" or "1": whether the first value in the array is zero (or False) or one (or True)
  • one integer for the length of the run-length representation
  • integers of the run-length representation of the array read in its proper enumeration order

• If the array is all False's or zeros:

  • one character "2"

All the integers are encoded by the unsigned LEB128 encoding using the leb128 project.

For non-boolean arrays with a maximum value of 2 or more, the content part is the concatenation of the sub-contents corresponding to each value between 1 and the maximum value in the array.

Dependencies

The development relies on several packages:

• Mandatory: dominate, flask, flask-bootstrap4, flask-session, flask-uploads, flask-wtf, imageio, leb128, numpy, si-fi-o, tqdm, wtforms
• Optional: None

The mandatory dependencies, if any, are installed automatically by pip, if they are not already, as part of the installation of PCA-B-Stream. Python distribution platforms or Integrated Development Environments (IDEs) should also take care of this. The optional dependencies, if any, must be installed independently by following the related instructions, for added functionalities of PCA-B-Stream.

Acknowledgments

The project is developed with PyCharm Community.

The code is formatted by Black, The Uncompromising Code Formatter.

The imports are ordered by isort... your imports, so you don't have to.