數字派 NumPy ︰陣列運算《五》

詩經‧國風‧豳風‧伐柯

伐柯如何?匪斧不克。

取妻如何?匪媒不得。

伐柯伐柯,其則不遠。

我覯之子,籩豆有踐。

 

伐『柯』柯即是製作斧柄,卻是非斧不行??將之比作娶『妻』

※《説文解字》: 柯,斧柄也。从木,可聲。

實在費疑猜??若思善用『工具』能夠創造更好的『工具』,那麼『伐柯伐柯,其則不遠。』果真『有理有則』的乎!?因此假借 SymPy 『符號矩陣』學習『矩陣』,也是『順理而行』的了?!

這裡僅以『基本矩陣』之法,求解一般 2x2 矩陣之『反矩陣』M 為例,演義這則『伐柯取妻』有趣之事︰

if

E \cdot M = (E_n \cdot \cdots E_2 \cdot E_1)  \cdot M = I

then

E = M^{-1}

M = {E_1}^{-1} \cdot {E_2}^{-1} \cdot \cdots {E_n}^{-1} = E^{-1}

望可消消暑氣也!!??

─── 《光的世界︰派生科學計算四

 

認識『初等矩陣』可得『高斯消去法』之精神,知道『陣列』的『元素乘法』易入『神經網路』世界︰

Michael Nielsen 先生此段文字是本章的重點,他先從

The Hadamard product,s \odot t

The backpropagation algorithm is based on common linear algebraic operations – things like vector addition, multiplying a vector by a matrix, and so on. But one of the operations is a little less commonly used. In particular, suppose s and t are two vectors of the same dimension. Then we use s \odot t to denote the elementwise product of the two vectors. Thus the components of s \odot t are just (s \odot t)_j = s_j t_j. As an example,

\left[\begin{array}{c} 1 \\ 2 \end{array}\right] \odot \left[\begin{array}{c} 3 \\ 4\end{array} \right] = \left[ \begin{array}{c} 1 * 3 \\ 2 * 4 \end{array} \right] = \left[ \begin{array}{c} 3 \\ 8 \end{array} \right].  \ \ \ \  (28)

This kind of elementwise multiplication is sometimes called the Hadamard product or Schur product. We’ll refer to it as the Hadamard product. Good matrix libraries usually provide fast implementations of the Hadamard product, and that comes in handy when implementing backpropagation.

───

講起,是希望藉著『記號法』,方便讀者理解『反向傳播算法』的內容。這個稱作 Hadamard product 之『對應元素乘法』

Hadamard product (matrices)

In mathematics, the Hadamard product (also known as the Schur product[1] or the entrywise product[2]) is a binary operation that takes two matrices of the same dimensions, and produces another matrix where each element ij is the product of elements ij of the original two matrices. It should not be confused with the more common matrix product. It is attributed to, and named after, either French mathematician Jacques Hadamard, or German mathematician Issai Schur.

The Hadamard product is associative and distributive, and unlike the matrix product it is also commutative.

220px-Hadamard_product_qtl1.svg

The Hadamard product operates on identically-shaped matrices and produces a third matrix of the same dimensions.

Definition

For two matrices, A, B, of the same dimension, m \times n the Hadamard product, A \circ B, is a matrix, of the same dimension as the operands, with elements given by

(A \circ B)_{i,j} = (A)_{i,j} \cdot (B)_{i,j}.

For matrices of different dimensions (m \times n and p \times q, where m \not= p or n \not= q or both) the Hadamard product is undefined.

Example

For example, the Hadamard product for a 3×3 matrix A with a 3×3 matrix B is:

 \left(\begin{array}{ccc} \mathrm{a}_{11} & \mathrm{a}_{12} & \mathrm{a}_{13}\\ \mathrm{a}_{21} & \mathrm{a}_{22} & \mathrm{a}_{23}\\ \mathrm{a}_{31} & \mathrm{a}_{32} & \mathrm{a}_{33} \end{array}\right) \circ \left(\begin{array}{ccc} \mathrm{b}_{11} & \mathrm{b}_{12} & \mathrm{b}_{13}\\ \mathrm{b}_{21} & \mathrm{b}_{22} & \mathrm{b}_{23}\\ \mathrm{b}_{31} & \mathrm{b}_{32} & \mathrm{b}_{33} \end{array}\right) = \left(\begin{array}{ccc} \mathrm{a}_{11}\, \mathrm{b}_{11} & \mathrm{a}_{12}\, \mathrm{b}_{12} & \mathrm{a}_{13}\, \mathrm{b}_{13}\\ \mathrm{a}_{21}\, \mathrm{b}_{21} & \mathrm{a}_{22}\, \mathrm{b}_{22} & \mathrm{a}_{23}\, \mathrm{b}_{23}\\ \mathrm{a}_{31}\, \mathrm{b}_{31} & \mathrm{a}_{32}\, \mathrm{b}_{32} & \mathrm{a}_{33}\, \mathrm{b}_{33} \end{array}\right)

───

明晰易解,大概無需畫蛇添足的吧!

─── 摘自《W!O+ 的《小伶鼬工坊演義》︰神經網絡【BACKPROPAGATION】四

 

藉著『工具』改善『工具』︰

Linear Algebra (scipy.linalg)

When SciPy is built using the optimized ATLAS LAPACK and BLAS libraries, it has very fast linear algebra capabilities. If you dig deep enough, all of the raw lapack and blas libraries are available for your use for even more speed. In this section, some easier-to-use interfaces to these routines are described.

All of these linear algebra routines expect an object that can be converted into a 2-dimensional array. The output of these routines is also a two-dimensional array.

scipy.linalg vs numpy.linalg

scipy.linalg contains all the functions in numpy.linalg. plus some other more advanced ones not contained in numpy.linalg

Another advantage of using scipy.linalg over numpy.linalg is that it is always compiled with BLAS/LAPACK support, while for numpy this is optional. Therefore, the scipy version might be faster depending on how numpy was installed.

Therefore, unless you don’t want to add scipy as a dependency to your numpy program, use scipy.linalg instead of numpy.linalg

numpy.matrix vs 2D numpy.ndarray

The classes that represent matrices, and basic operations such as matrix multiplications and transpose are a part of numpy. For convenience, we summarize the differences between numpy.matrix and numpy.ndarray here.

numpy.matrix is matrix class that has a more convenient interface than numpy.ndarray for matrix operations. This class supports for example MATLAB-like creation syntax via the semicolon, has matrix multiplication as default for the * operator, and contains I and members that serve as shortcuts for inverse and transpose:

 

Despite its convenience, the use of the numpy.matrix class is discouraged, since it adds nothing that cannot be accomplished with 2D numpy.ndarray objects, and may lead to a confusion of which class is being used. For example, the above code can be rewritten as:

 

或離『伐柯其則』不遠,所以才串講也◎