NDArray is short for N Dimension Array, it is a headers only library for N-Dimensions array(i.e. tensor in deep learning) expressions, such as arithmatic operations, reshape, reduction, slice and etc.

Relevant Research

In deep learning applications, tensor can be seen everywhere[7]. It is necessary to wrap its operations to lower the difficulty and cost for application developers.

Eigen[1][2] is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms. It has been applied into many open-source project, e.g. TensorFlow and etc. Its implementation is template meta programming[3] based and its expression templates allow to intelligently remove temporaries and enable lazy evaluation, when that is appropriate[4].

TACO(Tensor Algebra Compiler)[6]is a C++ library that computes tensor algebra expressions on sparse and dense tensors. It uses novel compiler techniques to get performance competitive with hand-optimized kernels in widely used libraries for both sparse tensor algebra and sparse linear algebra.

NDArray Key Features

Many C++11 new features has been applied during its implementation, e.g. Rvalue references, variadic templates, initializer lists, type traits and etc. Some of its design concept comes from C++ STL container and Eigen tensor[5].

• NDArray is versatile. It provides much flexibility, since its implementation is template meta programming based. It supports all standard numeric types, and any dimension NDArrays.
• NDArray will provide both static and dynamic containers.
• To guarantee performance, NDArray expression templates allow to intelligently remove temporaries and enable lazy evaluation. And I will try to take use of SIMD instructions for CPU, GPU or some ASIC platforms.

Reference

• [1] Eigen bitbucket repo: https://bitbucket.org/eigen/
• [2] Eigen official page: http://eigen.tuxfamily.org/
• [3] Template Meta Programming, wiki page: https://en.wikipedia.org/wiki/Template_metaprogramming
• [4] Eigen Lazy Evaluation and Aliasing, http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html
• [5] Eigen Tensor, checkout its README
• [6] TACO official page: http://tensor-compiler.org
• [7] TACO presentation: http://groups.csail.mit.edu/commit/presentations/2017/tensor-compiler.pdf

For more details about NDArray src, see its github repo and its gh-pages...

Any questions or suggestions, feel free to open an issue @here or e-mail me to lijiansong@ict.ac.cn.