Releases: tensorly/tensorly
Release 0.8.0
We are releasing a new version of TensorLy, long in the making, with a long list of major improvements, new features, better documentation, bug fixes and overall quality of life improvements!
New features
Transparent support for einsum
There are two main ways to implement tensor algebraic methods:
- Perhaps the most common, using existing matrix based algebraic methods, which typically involves unfolding the tensor (reshaping the tensor into a matrix and permuting its dimensions)
- Directly leverage tensor contraction, e.g. through an ensue interface. This implies that the einsum actually performs tensor contraction.
We improved the tenalg backend, you can transparently dispatch all tensor algebraic operations to the backend's einsum:
import tensorly as tl
# Tensor algebra
from tensorly import tenalg
# Dispatch all operations to einsum
tenalg.set_backend('einsum')
Now all tenalg functions will call einsum under the hood!
Opt-Einsum support
In addition, for each einsum call, you can now use opt-einsum to compute a (near) optimal contraction path and cache it with just one call!
# New opt-einsum plugin
from tensorly.plugins import use_opt_einsum
# Transparently compute and cache contraction path using opt-einsum
use_opt_einsum('optimal')
Switch back to the original backend's einsum:
# New opt-einsum plugin
from tensorly.plugins import use_default_einsum
use_default_einsum()
Efficient contraction on GPU with cuQuantum
If you want to accelerate your computation, you probably want to use the GPU.
TensorLy has been supporting GPU transparently for a while, through its MXNet, CuPy, TensorFlow, PyTorch and more recently, JAX backends.
Now you can also get efficient tensor contractions on GPU using NVIDIA's cuQuantum library!
Now any function to the `tenalg` module
# New opt-einsum plugin
from tensorly.plugins import use_cuquantum
# Transparently compute and cache contraction path using opt-einsum
use_cuquantum('optimal')
# Create a new tensor on GPU
tensor = tl.randn((32, 256, 256, 3), device='cuda')
# Decompose it with CP, keep 5% of the parameters
parafac(tensor, rank=0.05, init='random', n_iter_max=10)
Similarity measure
We now provide CorrIndex, a correlation invariant index
CP-partial least square regression
This release brings a new multi-linear partial least squares regression, as first introduce by Rasmus Bro, exposed in a convenient scikit-learn-like class, CP_PLSR
Tensor-Train via orthogonal iteration
We have a new tensor decomposition tensor_train_OI
class for tensor-train decomposition via orthogonal iteration.
Unified SVD interface
We now have a unified interface for Singular Value Decomposition svd_interface
.
It has support for resolving sign indeterminacy, returning a non-negative output, missing values (masked input), and various computation methods, all in one, neat interface!
New datasets
TensorLy now includes real-world datasets well-suited for tensor analysis, that you can now directly load/download in a ready to use form!
COVID-19 Serology Dataset
Systems serology is a new technology that examines the antibodies from a patient’s serum, aiming to comprehensively profile the interactions between the antibodies and Fc receptors alongside other types of immunological and demographic data. Here, we will apply CP decomposition to a COVID-19 system serology dataset. In this dataset, serum antibodies of 438 samples collected from COVID-19 patients were systematically profiled by their binding behavior to SARS-CoV-2 (the virus that causes COVID-19) antigens and Fc receptors activities. The data is formatted in a three-mode tensor of samples, antigens, and receptors Samples are labeled by the status of the patients.
IL2
IL-2 signals through the Jak/STAT pathway and transmits a signal into immune cells by phosphorylating STAT5 (pSTAT5). When phosphorylated, STAT5 will cause various immune cell types to proliferate, and depending on whether regulatory (regulatory T cells, or Tregs) or effector cells (helper T cells, natural killer cells, and cytotoxic T cells, or Thelpers, NKs, and CD8+ cells) respond, IL-2 signaling can result in immunosuppression or immunostimulation respectively. Thus, when designing a drug meant to repress the immune system, potentially for the treatment of autoimmune diseases, IL-2 which primarily enacts a response in Tregs is desirable. Conversely, when designing a drug that is meant to stimulate the immune system, potentially for the treatment of cancer, IL-2 which primarily enacts a response in effector cells is desirable. In order to achieve either signaling bias, IL-2 variants with altered affinity for its various receptors (IL2Rα or IL2Rβ) have been designed. Furthermore IL-2 variants with multiple binding domains have been designed as multivalent IL-2 may act as a more effective therapeutic.
The data contains the responses of 8 different cell types to 13 different IL-2 mutants, at 4 different timepoints, at 12 standardized IL-2 concentrations. It is formatted as a 4th order tensor of shape (13 x 4 x 12 x 8), with dimensions representing IL-2 mutant, stimulation time, dose, and cell type respectively.
Kinetic
A Kinetic fluorescence dataset, well suited for Parafac and multi-way partial least squares regression (N-PLS).
The data is represented as a four-way data set with the modes: Concentration, excitation wavelength, emission wavelength and time.
Indian Pines
Airborne Visible / Infrared Imaging Spectrometer (AVIRIS) hyperspectral sensor data. It consists of 145 times 145 pixels and 220 spectral reflectance bands in the wavelength range 0.4–2.5 10^(-6) meters.
Black linting
We now automatically check for code formatting and the CI tests the code style against the Black styleguides.
List of merged pull requests in this release
In addition to these big features, this release also comes with a whole lot of improvements, better documentation and bug fixes!
Non-exhaustive list of changes:
- solves hals rec_error0 issue and does some pep improvements by @caglayantuna in #339
- Copy cptensor by @caglayantuna in #324
- Normalization by @caglayantuna in #281
- Add random_state for non negative PARAFAC HALS by @MarieRoald in #344
- fix tensorflow dtype issue by @caglayantuna in #340
- nn_tucker hals class, doc, api improvements by @caglayantuna in #345
- Constrained parafac example and improved docstrings by @caglayantuna in #347
- IL-2 stimulation dataset by @borcuttjahns in #348
- Adds matricize function by @JeanKossaifi in #366
- copy tucker tensor by @caglayantuna in #367
- simplex projection issue by @caglayantuna in #363
- Make
tl.shape
return tuple for PyTorch backend by @MarieRoald in #357 - Add
keepdims
totl.sum
with the PyTorch backend by @MarieRoald in #356 - Fix bug with
tl.clip
for the PyTorch and TensorFlow backends by @MarieRoald in #355 - Import COVID-19 systems serology example dataset by @cyrillustan in #359
- Add Covid example notebook by @cyrillustan in #361
- Adds exp to the backend by @JeanKossaifi in #377
- Adds digamma fun to backend by @JeanKossaifi in #378
- Adds log function to tensorly by @caglayantuna in #381
- Clip function to sparse backend with a_max=None by @caglayantuna in #379
- CorrIndex implementation for comparing decomposition outputs by @hmbaghdassarian in #364
- Adds pad_tt_rank by @JeanKossaifi in https://github.com/tensorly/t...
TensorLy Release 0.7.0
TensorLy 0.7.0 is out!
In this new version of TensorLy, the whole team has been working hard to bring you lots of improvements, from new decompositions to new functions, faster code and better documentation.
Major improvements and new features
New decompositions
We added some great new tensor decompositions, including
- Coupled Matrix-Tensor Factorisation:
[ CMTF-ALS #293 thanks to @IsabellLehmann and @aarmey ] - Tensor-Ring (e.g. MPS with periodic boundary conditions):
[ Tensor Ring implementation #229 thanks to @merajhashemi ] - Non-negative Tucker decomposition via Hierarchical ALS:
[ NN-HALS Tucker @caglayantuna @cohenjer #254 ] - A new CP decomposition that supports various constraints on each mode, including monotony, non-negativity, l1/l2 regularization, smoothness, sparsity, etc!
[ Constrained parafac #284, thanks to @caglayantuna and @cohenjer ]
Brand new features
We added a brand new tensordot
that supports batching!
[ Adding a new Batched Tensor Dot + API simplification #309 ]
Normalization for Tucker factors, #283 thanks to @caglayantuna and @cohenjer!
Added a convenient function to compute the gradient of the difference norm between a CP and dense tensor, #294, thanks to @aarmey
Backend refactoring
In an effort to make the TensorLy backend even more flexible and fast, we refactored the main backend as well as the tensor algebra backend. We make lots of small quality of life improvement in the process! In particular, reconstructing a tt-matrix is a lot more efficient now.
[ Backend refactoring : use a BackendManager class and use it directly as tensorly.backend's Module class #330, @JeanKossaifi ]
Enhancements
Improvements to Parafac2 (convergence criteria, etc) #267, thanks to @MarieRoald
HALS convergence FIX TODO, @MarieRoald and @IsabellLehmann, #271
Ensuring consistency between the object oriented API and the functional one thanks to @yngvem, #268
Added lstsq to backend, #305, thanks to @merajhashemi
Fix documentation for case insensitive clashes between the function and class: #219
Added random-seed for TT-cross, #304 thanks to @yngvem
Fix svd sign indeterminacy #216, thanks to @merajhashemi
Rewrote vonneumann_entropy to handle multidimensional tensors. #270, thanks to @taylorpatti
Adding check for all modes fixed case and if true then to just return the initialization #325, thanks to @ParvaH
We now provide a prod
function that works like math.prod for users using Python < 3.8, in tensorly.utils.prod
New backend functions
All backend now support matmul
, tensor dot
(#306), as well as sin
, cos
, flip
, argsort
, count_nonzero
, cumsum
, any
, lstsq
and trace
.
Bug Fixes
Fixed NN-Tucker hals sparsity coefficient issue, thanks to @caglayantuna #295
Fix svd for pytorch < 1.8 #312 thanks to @merajhashemi
Fix dot and matmul in PyTorch and TF #313 thanks to @merajhashemi
Fix tl.partial_unfold #315, thanks to @merajhashemi
Fixed behaviour of diag for TensorFlow backend.
Fix tl.partial_svd : now explicitly check for NaN values, #318 thanks to @merajhashemi
fix diag function for tensorflow and pytorch backends #321, thanks to @caglayantuna
Fix singular vectors to be orthonormal #320 thanks to @merajhashemi
fix active set and hals tests #323 thanks to @caglayantuna
Add test for matmul #322 thanks to @merajhashemi
Sparse backend usage fix by @caglayantuna in #280
Release 0.6.0
What's new
This version brings lots of new functionalities, improvements and fixes many small bugs and issues. We have a new theme for the TensorLy project's documentations, our new TensorLy Sphinx theme, which we've open-sourced and that you can also easily use in your own projects! We've also switched testing from Travis to Github actions and for coverage, from Coveralls to CodeCov.
New features
- Non-negative CP decomposition via Hierarchical Alternating Least Square (HALS), thanks to @caglayantuna and @cohenjer, #224
- Non-negative SVD and refactoring of non-negative CP, thanks to @aarmey, #204
- Randomized SVD thanks to @merajhashemi, #215
- Sign correction method for CP thanks to @aarmey, #226
- Support for complex tensors, thanks to @merajhashemi #213, #247
- Entropy metrics, thanks to @taylorpatti, #231
Backend refactoring
- Various SVD forms are backend-agnostic.
- Norm method is backend-agnostic.
- CuPy tests now pass thanks to @aarmey, #217
- Random number generation seed (check_random_state) was moved to the backend, and random functions now use NumPy’s global seed by default thanks to @merajhashemi, #209 #212
- Using mxnet.np interface in MXNet backend, thanks to @aarmey, #225, #207
Bug fixes and improvements
- PARAFAC2: fixed SVD init and improved naming consistency, thanks to @MarieRoald, #220
- Improved CP initialisation thanks to @caglayantuna, #230
- Improved testing for
check_tucker_rank
- Support for order-1 and 2 CP and TT/TTM tensors.
- Fixed issues in CP via robust tensor power iteration, thanks to @chrisyeh96, #244
- Added support for
tensordot
in all backends - Fix for partial_tucker when provided with mask and applied only to a subset of modes, 691f785
- Documentation and user-guide improvements, switched to new theme, e328d8e
And many other small improvements!
Release 0.5.1
CP: l2 reg
CP: sparsity
Added fixed_modes for CP and Tucker.
Masked Tucker
Sparse backend
And many small improvements and bug fixes!
Standardisation of the names:
Kruskal-tensors have been renamed cp_tensors
Matrix-product-state has now been renamed tensor-train
Rank selection: validate_cp_rank, option to set rank=‘same’ or rank=float to automatically determine the rank.