In recent years, the mathematical manipulation of tensors – tensor algebra – has become crucial to not only big-data analysis but machine learning, too. Traditionally, to handle tensor algebra, mathematics software has decomposed tensor operations into their constituent parts.
So if a computation required two tensors to be multiplied and then added to a third, the software would run its standard tensor multiplication routine on the first two tensors, store the result, and then run its standard tensor addition routine.
The number of possible kernels is infinite: The kernel for adding together three tensors is different from the kernel for adding together four, and the kernel for adding three three-dimensional tensors is different from the kernel for adding three four-dimensional tensors.
Many tensor operations involve multiplying an entry from one tensor with one from another. This makes tensor manipulations much faster, but it requires the programmer to do a lot more work.
For any given operation on two tensors, Taco builds a hierarchical map that indicates, first, which paired entries from both tensors are nonzero and which entries from each tensor are paired with zeroes.