LowerTriangularMatrices

LowerTriangularMatrices is a submodule that has been developed for SpeedyWeather.jl which is technically independent (SpeedyWeather.jl however imports it and so does SpeedyTransforms) and can also be used without running simulations. It is just not put into its own respective repository.

This module defines LowerTriangularArray, a lower triangular matrix format, which in contrast to LinearAlgebra.LowerTriangular does not store the entries above the diagonal. SpeedyWeather.jl uses LowerTriangularArray which is defined as a subtype of AbstractArray to store the spherical harmonic coefficients (see Spectral packing). For 2D LowerTriangularArray the alias LowerTriangularMatrix exists. Higher dimensional LowerTriangularArray are 'batches' of 2D LowerTriangularMatrix. So, for example a $(10\times 10\times 10)$ LowerTriangularArray holds 10 LowerTriangularMatrix of size $(10\times 10)$ in one array.

LowerTriangularMatrix is actually a vector

LowerTriangularMatrix and LowerTriangularArray can in many ways be used very much like a Matrix or Array, however, because they unravel the lower triangle into a vector their dimensionality is one less than their Array counterparts. A LowerTriangularMatrix should therefore be treated as a vector rather than a matrix with some (limited) added functionality to allow for matrix-indexing (vector or flat-indexing is the default though). More details below.

Creation of `LowerTriangularArray`

A LowerTriangularMatrix and LowerTriangularArray can be created using zeros, ones, rand, or randn

julia> using SpeedyWeather.LowerTriangularMatrices
julia> L = rand(LowerTriangularMatrix{Float32}, 5, 5)15-element, 5x5 LowerTriangularMatrix{Float32}
 0.754201  0.0       0.0       0.0       0.0
 0.228015  0.963678  0.0       0.0       0.0
 0.33673   0.213394  0.357787  0.0       0.0
 0.788527  0.288331  0.309036  0.673593  0.0
 0.974242  0.362114  0.823219  0.915312  0.324043
julia> L2 = rand(LowerTriangularArray{Float32}, 5, 5, 5)15×5 LowerTriangularArray{Float32, 2, Matrix{Float32}}:
 0.460127   0.685045   0.308648   0.838044   0.742962
 0.330389   0.74337    0.246252   0.952649   0.0040158
 0.489276   0.640539   0.32824    0.251595   0.0619842
 0.826866   0.697894   0.20607    0.216961   0.996558
 0.0168188  0.963347   0.282049   0.945945   0.10861
 0.140407   0.987225   0.320094   0.668217   0.902443
 0.1006     0.572466   0.0799204  0.140812   0.637067
 0.209681   0.352899   0.482267   0.222904   0.110663
 0.700554   0.251882   0.342482   0.0417897  0.493244
 0.303527   0.24273    0.785403   0.259035   0.420696
 0.646055   0.846662   0.641894   0.96663    0.941053
 0.172105   0.645486   0.67827    0.477354   0.501657
 0.653103   0.54779    0.860599   0.421897   0.8671
 0.871523   0.0720502  0.17235    0.0219402  0.522493
 0.533395   0.508753   0.172106   0.645856   0.180213

or the undef initializer LowerTriangularMatrix{Float32}(undef, 3, 3). The element type is arbitrary though, you can use any type T too.

Note how for a matrix both the upper triangle and the lower triangle are shown in the terminal. The zeros are evident. However, for higher dimensional LowerTriangularArray we fall back to show the unravelled first two dimensions. Hence, here, the first column is the first matrix with 15 elements forming a 5x5 matrix, but the zeros are not shown.

Alternatively, it can be created through conversion from Array, which drops the upper triangle entries and sets them to zero (which are not stored however)

julia> M = rand(Float16, 3, 3)3×3 Matrix{Float16}:
 0.6475  0.5254  0.01074
 0.8267  0.248   0.5264
 0.2373  0.9307  0.659
julia> L = LowerTriangularMatrix(M)6-element, 3x3 LowerTriangularMatrix{Float16}
 0.6475  0.0     0.0
 0.8267  0.248   0.0
 0.2373  0.9307  0.659
julia> M2 = rand(Float16, 3, 3, 2)3×3×2 Array{Float16, 3}:
[:, :, 1] =
 0.3315  0.1812  0.9507
 0.5654  0.8926  0.871
 0.3877  0.945   0.8154

[:, :, 2] =
 0.496  0.94    0.6577
 0.895  0.2476  0.4688
 0.683  0.1787  0.8184
julia> L2 = LowerTriangularArray(M2)6×2 LowerTriangularArray{Float16, 2, Matrix{Float16}}:
 0.3315  0.496
 0.5654  0.895
 0.3877  0.683
 0.8926  0.2476
 0.945   0.1787
 0.8154  0.8184

Size of `LowerTriangularArray`

There are three different ways to describe the size of a LowerTriangularArray. For example with L

julia> L = rand(LowerTriangularMatrix, 5, 5)15-element, 5x5 LowerTriangularMatrix{Float64}
 0.876703  0.0       0.0        0.0       0.0
 0.68133   0.751802  0.0        0.0       0.0
 0.854972  0.180731  0.0515063  0.0       0.0
 0.362829  0.373475  0.145412   0.182639  0.0
 0.822454  0.70375   0.219726   0.634356  0.697729

we have (additional dimensions follow naturally thereafter)

1-based vector indexing (default)

julia> size(L)    # equivalently size(L, OneBased, as=Vector)(15,)

The lower triangle is unravelled hence the number of elements in the lower triangle is returned.

1-based matrix indexing

julia> size(L, as=Matrix)    # equivalently size(L, OneBased, as=Matrix)(5, 5)

If you think of a LowerTriangularMatrix as a matrix this is the most intuitive size of L, which, however, does not agree with the size of the underlying data array (hence it is not the default).

0-based matrix indexing

Because LowerTriangularArrays are used to represent the coefficients of spherical harmonics which are commonly indexed based on zero (i.e. starting with the zero mode representing the mean), we also add ZeroBased to get the corresponding size.

julia> size(L, ZeroBased, as=Matrix)(4, 4)

which is convenient if you want to know the maximum degree and order of the spherical harmonics in L. 0-based vector indexing is not implemented.

Indexing `LowerTriangularArray`

We illustrate the two types of indexing LowerTriangularArray supports.

Matrix indexing, by denoting two indices, column and row [l, m, ..]
Vector/flat indexing, by denoting a single index [lm, ..].

The matrix index works as expected

julia> L15-element, 5x5 LowerTriangularMatrix{Float64}
 0.876703  0.0       0.0        0.0       0.0
 0.68133   0.751802  0.0        0.0       0.0
 0.854972  0.180731  0.0515063  0.0       0.0
 0.362829  0.373475  0.145412   0.182639  0.0
 0.822454  0.70375   0.219726   0.634356  0.697729
julia> L[2, 2]0.7518024040006529

But the single index skips the zero entries in the upper triangle, i.e. a 2, 2 index points to the same element as the index 6

julia> L[6]0.7518024040006529

which, important, is different from single indices of an AbstractMatrix

julia> Matrix(L)[6]0.0

which would point to the first element in the upper triangle (hence zero).

In performance-critical code a single index should be used, as this directly maps to the index of the underlying data vector. The matrix index is somewhat slower as it first has to be converted to the corresponding single index.

Consequently, many loops in SpeedyWeather.jl are build with the following structure

julia> n, m = size(L, as=Matrix)(5, 5)
julia> ij = 00
julia> for j in 1:m, i in j:n
           ij += 1
           L[ij] = i+j
       end

which loops over all lower triangle entries of L::LowerTriangularArray and the single index ij is simply counted up. However, one could also use [i, j] as indices in the loop body or to perform any calculation (i+j here).

`end` doesn't work for matrix indexing

Indexing LowerTriangularMatrix and LowerTriangularArray in matrix style ([i, j]) with end doesn't work. It either returns an error or wrong results as the end is lowered by Julia to the size of the underlying flat array dimension.

The setindex! functionality of matrixes will throw a BoundsError when trying to write into the upper triangle of a LowerTriangularArray, for example

julia> L[2, 1] = 0    # valid index0
julia> L[1, 2] = 0    # invalid index in the upper triangleERROR: BoundsError: attempt to access 15-element, 5x5 LowerTriangularMatrix{Float64} at index [1, 2]

But reading from it will just return a zero

julia> L[2, 3]     # in the upper triangle0.0

Higher dimensional LowerTriangularArray can be indexed with multidimensional array indices like most other arrays types. Both the vector index and the matrix index for the lower triangle work as shown here

julia> L = rand(LowerTriangularArray{Float32}, 3, 3, 5)6×5 LowerTriangularArray{Float32, 2, Matrix{Float32}}:
 0.427862  0.960641    0.193589  0.342795   0.745776
 0.877455  0.137305    0.255006  0.155619   0.335596
 0.387818  0.0853382   0.270926  0.0257831  0.875353
 0.685461  0.676677    0.897132  0.0873436  0.997193
 0.839729  0.184804    0.684143  0.478833   0.377579
 0.414933  0.00716048  0.833742  0.120531   0.795846
julia> L[2, 1] # second lower triangle element of the first lower triangle matrix0.8774554f0
julia> L[2, 1, 1] # (2,1) element of the first lower triangle matrix0.8774554f0

The setindex! functionality follows accordingly.

Iterators

An iterator over all entries in the array can be created with eachindex

julia> L = rand(LowerTriangularArray, 5, 5, 5)15×5 LowerTriangularArray{Float64, 2, Matrix{Float64}}:
 0.180468   0.642701   0.327858   0.150903   0.659806
 0.712001   0.530599   0.764324   0.784543   0.809936
 0.956459   0.203753   0.0783505  0.994241   0.77469
 0.509646   0.361142   0.205246   0.0783278  0.136034
 0.39534    0.116564   0.288026   0.628726   0.437837
 0.388027   0.941505   0.620908   0.402577   0.705894
 0.853627   0.40361    0.523534   0.236791   0.939366
 0.854213   0.21846    0.65041    0.180958   0.278733
 0.3276     0.0848787  0.631815   0.723012   0.363702
 0.21545    0.776591   0.294322   0.928902   0.669115
 0.0185994  0.637076   0.372913   0.0469227  0.620208
 0.128412   0.577487   0.903976   0.850883   0.993418
 0.431778   0.237438   0.328974   0.0545039  0.151965
 0.328445   0.0278092  0.798808   0.849257   0.676606
 0.453082   0.856392   0.564087   0.745893   0.82973
julia> for ij in eachindex(L)
           # do something
       end
julia> eachindex(L)Base.OneTo(75)

In order to only loop over the harmonics (essentially the horizontal, ignoring other dimensions) use eachharmonic

julia> eachharmonic(L)Base.OneTo(15)

If you only want to loop over the other dimensions use eachmatrix

julia> eachmatrix(L)CartesianIndices((5,))

together they can be used as

julia> for k in eachmatrix(L)
           for lm in eachharmonic(L)
               L[lm, k]
           end
       end

Note that k is a CartesianIndex that will loop over all other dimensions, whether there's only 1 (representing a 3D variable) or 5 (representing a 6D variable with the first two dimensions being a lower triangular matrix).

Linear algebra with `LowerTriangularArray`

The LowerTriangularMatrices module's main purpose is not linear algebra, and typical matrix operations will not work with LowerTriangularMatrix because it's treated as a vector not as a matrix, meaning that the following will not work as expected

julia> L = rand(LowerTriangularMatrix{Float32}, 3, 3)6-element, 3x3 LowerTriangularMatrix{Float32}
 0.0722944  0.0        0.0
 0.174991   0.301877   0.0
 0.235292   0.0860656  0.150332
julia> L * LERROR: MethodError: no method matching *(::LowerTriangularMatrix{Float32}, ::LowerTriangularMatrix{Float32})
The function `*` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  *(::Any, ::Any, ::Any, ::Any...)
   @ Base operators.jl:596
  *(::ChainRulesCore.NoTangent, ::Any)
   @ ChainRulesCore ~/.julia/packages/ChainRulesCore/6Pucz/src/tangent_arithmetic.jl:64
  *(::Any, ::ChainRulesCore.NotImplemented)
   @ ChainRulesCore ~/.julia/packages/ChainRulesCore/6Pucz/src/tangent_arithmetic.jl:38
  ...
julia> inv(L)ERROR: MethodError: no method matching inv(::LowerTriangularMatrix{Float32})
The function `inv` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  inv(::BigFloat)
   @ Base mpfr.jl:592
  inv(::Crayons.ANSIColor)
   @ Crayons ~/.julia/packages/Crayons/u3AH8/src/crayon.jl:64
  inv(::Crayons.Crayon)
   @ Crayons ~/.julia/packages/Crayons/u3AH8/src/crayon.jl:98
  ...

And many other operations that require L to be a AbstractMatrix which it isn't. In contrast, typical vector operations like a scalar product between two "LowerTriangularMatrix" vectors does work

julia> L' * L0.21234795f0

Broadcasting with `LowerTriangularArray`

In contrast to linear algebra, many element-wise operations work as expected thanks to broadcasting, so operations that can be written in . notation whether implicit +, 2*, ... or explicitly written .+, .^, ... or via the @. macro

julia> L + L6-element, 3x3 LowerTriangularMatrix{Float32}
 0.144589  0.0       0.0
 0.349983  0.603755  0.0
 0.470584  0.172131  0.300664
julia> 2L6-element, 3x3 LowerTriangularMatrix{Float32}
 0.144589  0.0       0.0
 0.349983  0.603755  0.0
 0.470584  0.172131  0.300664
julia> @. L + 2L - 1.1*L / L^26-element, 3x3 LowerTriangularMatrix{Float64}
 -14.9987     0.0       0.0
  -5.76105   -2.73823   0.0
  -3.96916  -12.5227   -6.86613

GPU

LowerTriangularArray{T, N, ArrayType} wraps around an array of type ArrayType. If this array is a GPU array (e.g. CuArray), all operations are performed on GPU as well (work in progress). The implementation was written so that scalar indexing is avoided in almost all cases, so that GPU operation should be performant. To use LowerTriangularArray on GPU you can e.g. just adapt an existing LowerTriangularArray.

using Adapt
L = rand(LowerTriangularArray{Float32}, 5, 5, 5)
L_gpu = adapt(CuArray, L)

Function and type index

SpeedyWeather.LowerTriangularMatrices.LowerTriangularArray — Type

A lower triangular array implementation that only stores the non-zero entries explicitly. L<:AbstractArray{T,N-1} although we do allow both "flat" N-1-dimensional indexing and additional N-dimensional or "matrix-style" indexing.

Supports n-dimensional lower triangular arrays, so that for all trailing dimensions L[:, :, ..] is a matrix in lower triangular form, e.g. a (5x5x3)-LowerTriangularArray would hold 3 lower triangular matrices.