這樣的一個學習者將會如何建造自己知識之金字塔的呢？他會不會用『想像的實驗』去釐清『基本概念』之糾葛的呢？還是用『推導歸謬』的邏輯，去探測一個『自明假設』之深遠結論的呢？又或者會將在大自然中發現的方程式求解，然後『畫圖』與『演示』這個解之意義的呢？……

─── 《加百利之號角！！》

 

假使 AI 只是場欺人的美夢，傳說中之 Oneiroi 會穿越『象牙之門』而來？

The gates of horn and ivory are a literary image used to distinguish true dreams (corresponding to factual occurrences) from false. The phrase originated in the Greek language, in which the word for “horn” is similar to that for “fulfil” and the word for “ivory” is similar to that for “deceive”. On the basis of that play on words, true dreams are spoken of as coming through the gates of horn, false dreams as coming through those of ivory.

 

是誰拿著『犄角』！吹響了號角？！

Why this name, Keras?

Keras (κέρας) means horn in Greek. It is a reference to a literary image from ancient Greek and Latin literature, first found in the Odyssey, where dream spirits (Oneiroi, singular Oneiros) are divided between those who deceive men with false visions, who arrive to Earth through a gate of ivory, and those who announce a future that will come to pass, who arrive through a gate of horn. It’s a play on the words κέρας (horn) / κραίνω (fulfill), and ἐλέφας (ivory) / ἐλεφαίρομαι (deceive).

Keras was initially developed as part of the research effort of project ONEIROS (Open-ended Neuro-Electronic Intelligent Robot Operating System).

“Oneiroi are beyond our unravelling –who can be sure what tale they tell? Not all that men look for comes to pass. Two gates there are that give passage to fleeting Oneiroi; one is made of horn, one of ivory. The Oneiroi that pass through sawn ivory are deceitful, bearing a message that will not be fulfilled; those that come out through polished horn have truth behind them, to be accomplished for men who see them.” Homer, Odyssey 19. 562 ff (Shewring translation).

 

果真能左右夢神俄涅洛伊之行事！？

祇聞日昨紅樓方建

Theano at a Glance

Theano is a Python library that lets you to define, optimize, and evaluate mathematical expressions, especially ones with multi-dimensional arrays (numpy.ndarray). Using Theano it is possible to attain speeds rivaling hand-crafted C implementations for problems involving large amounts of data. It can also surpass C on a CPU by many orders of magnitude by taking advantage of recent GPUs.

Theano combines aspects of a computer algebra system (CAS) with aspects of an optimizing compiler. It can also generate customized C code for many mathematical operations. This combination of CAS with optimizing compilation is particularly useful for tasks in which complicated mathematical expressions are evaluated repeatedly and evaluation speed is critical. For situations where many different expressions are each evaluated once Theano can minimize the amount of compilation/analysis overhead, but still provide symbolic features such as automatic differentiation.

Theano’s compiler applies many optimizations of varying complexity to these symbolic expressions. These optimizations include, but are not limited to:

• use of GPU for computations
• constant folding
• merging of similar subgraphs, to avoid redundant calculation
• arithmetic simplification (e.g. x*y/x -> y, --x -> x)
• inserting efficient BLAS operations (e.g. GEMM) in a variety of contexts
• using memory aliasing to avoid calculation
• using inplace operations wherever it does not interfere with aliasing
• loop fusion for elementwise sub-expressions
• improvements to numerical stability (e.g. and )
• for a complete list, see Optimizations

Theano was written at the LISA lab to support rapid development of efficient machine learning algorithms. Theano is named after the Greek mathematician, who may have been Pythagoras’ wife. Theano is released under a BSD license (link).

 

今天竟已成黃葉地呦？？！！

 

 

 

 

 

 

 

 

雖說最新版的 Up-Ubuntu 突然有 HDMI Audio 之問題？

[ 12.012421] Audio Port: ASoC: no backend DAIs enabled for Audio Port

不過眼下關心的卻是『張量流』哩！樂見『TensorFlow』二十行內完成之『手寫阿拉伯數字』辨識程式，可以正確且快速執行也︰

up@up-UP-CHT01:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from tensorflow.examples.tutorials.mnist import input_data
>>> mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting MNIST_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
>>> import tensorflow as tf
>>> x = tf.placeholder(tf.float32, [None, 784])
>>> W = tf.Variable(tf.zeros([784, 10]))
>>> b = tf.Variable(tf.zeros([10]))
>>> y = tf.nn.softmax(tf.matmul(x, W) + b)
>>> y_ = tf.placeholder(tf.float32, [None, 10])
>>> cross_entropy = -tf.reduce_sum(y_*tf.log(y))
>>> train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)
>>> init = tf.initialize_all_variables()
WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/util/tf_should_use.py:118: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
>>> sess = tf.Session()
2018-02-24 19:12:57.709052: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2
>>> sess.run(init)
>>> for i in range(1000):
...     batch_xs, batch_ys = mnist.train.next_batch(100)
...     sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
... 
>>> correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
>>> accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
>>> print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))
0.9166
>>> 

 

故希望能乘

Keras: The Python Deep Learning library

You have just found Keras.

Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.

Use Keras if you need a deep learning library that:

• Allows for easy and fast prototyping (through user friendliness, modularity, and extensibility).
• Supports both convolutional networks and recurrent networks, as well as combinations of the two.
• Runs seamlessly on CPU and GPU.

Read the documentation at Keras.io.

Keras is compatible with: Python 2.7-3.6.

---

Guiding principles

• User friendliness. Keras is an API designed for human beings, not machines. It puts user experience front and center. Keras follows best practices for reducing cognitive load: it offers consistent & simple APIs, it minimizes the number of user actions required for common use cases, and it provides clear and actionable feedback upon user error.
• Modularity. A model is understood as a sequence or a graph of standalone, fully-configurable modules that can be plugged together with as little restrictions as possible. In particular, neural layers, cost functions, optimizers, initialization schemes, activation functions, regularization schemes are all standalone modules that you can combine to create new models.
• Easy extensibility. New modules are simple to add (as new classes and functions), and existing modules provide ample examples. To be able to easily create new modules allows for total expressiveness, making Keras suitable for advanced research.
• Work with Python. No separate models configuration files in a declarative format. Models are described in Python code, which is compact, easier to debug, and allows for ease of extensibility.

 

的羽翼，再顯 Michael Nielsen 先生

對一本小巧完整而且寫的好的書，又該多說些什麼的呢？於是幾經思慮，就講講過去之閱讀隨筆與念頭雜記的吧！終究面對一個既舊也新的議題，尚待火石電光激發創意和發想。也許只需一個洞見或將改變人工智慧的未來乎？？

Michael Nielsen 先生開宗明義在首章起頭

The human visual system is one of the wonders of the world. Consider the following sequence of handwritten digits:

Most people effortlessly recognize those digits as 504192. That ease is deceptive. In each hemisphere of our brain, humans have a primary visual cortex, also known as V1, containing 140 million neurons, with tens of billions of connections between them. And yet human vision involves not just V1, but an entire series of visual cortices – V2, V3, V4, and V5 – doing progressively more complex image processing. We carry in our heads a supercomputer, tuned by evolution over hundreds of millions of years, and superbly adapted to understand the visual world. Recognizing handwritten digits isn’t easy. Rather, we humans are stupendously, astoundingly good at making sense of what our eyes show us. But nearly all that work is done unconsciously . And so we don’t usually appreciate how tough a problem our visual systems solve.

The difficulty of visual pattern recognition becomes apparent if you attempt to write a computer program to recognize digits like those above. What seems easy when we do it ourselves suddenly becomes extremely difficult. Simple intuitions about how we recognize shapes – “a 9 has a loop at the top, and a vertical stroke in the bottom right” – turn out to be not so simple to express algorithmically. When you try to make such rules precise, you quickly get lost in a morass of exceptions and caveats and special cases. It seems hopeless.

Neural networks approach the problem in a different way. The idea is to take a large number of handwritten digits, known as training examples,

 

and then develop a system which can learn from those training examples. In other words, the neural network uses the examples to automatically infer rules for recognizing handwritten digits. Furthermore, by increasing the number of training examples, the network can learn more about handwriting, and so improve its accuracy. So while I’ve shown just 100 training digits above, perhaps we could build a better handwriting recognizer by using thousands or even millions or billions of training examples.

In this chapter we’ll write a computer program implementing a neural network that learns to recognize handwritten digits. The program is just 74 lines long, and uses no special neural network libraries. But this short program can recognize digits with an accuracy over 96 percent, without human intervention. Furthermore, in later chapters we’ll develop ideas which can improve accuracy to over 99 percent. In fact, the best commercial neural networks are now so good that they are used by banks to process cheques, and by post offices to recognize addresses.

We’re focusing on handwriting recognition because it’s an excellent prototype problem for learning about neural networks in general. As a prototype it hits a sweet spot: it’s challenging – it’s no small feat to recognize handwritten digits – but it’s not so difficult as to require an extremely complicated solution, or tremendous computational power. Furthermore, it’s a great way to develop more advanced techniques, such as deep learning. And so throughout the book we’ll return repeatedly to the problem of handwriting recognition. Later in the book, we’ll discuss how these ideas may be applied to other problems in computer vision, and also in speech, natural language processing, and other domains.

Of course, if the point of the chapter was only to write a computer program to recognize handwritten digits, then the chapter would be much shorter! But along the way we’ll develop many key ideas about neural networks, including two important types of artificial neuron (the perceptron and the sigmoid neuron), and the standard learning algorithm for neural networks, known as stochastic gradient descent. Throughout, I focus on explaining why things are done the way they are, and on building your neural networks intuition. That requires a lengthier discussion than if I just presented the basic mechanics of what’s going on, but it’s worth it for the deeper understanding you’ll attain. Amongst the payoffs, by the end of the chapter we’ll be in position to understand what deep learning is, and why it matters.

………

之理念矣◎

說明這本書的主旨。是用『手寫阿拉伯數字辨識』這一主題貫串『神經網絡』以及『深度學習』之點滴，希望讀者能夠藉著最少的文本一窺全豹、聞一知十。因此他盡量少用『數學』，盡可能白話描述重要的『原理』與『概念』。

─── 摘自《W!o+ 的《小伶鼬工坊演義》︰神經網絡與深度學習【發凡】》

 

 

 

 

 

 

 

 

然終究得放棄 PIXEL Desktop ，並非因 HDMI Audio 沒有聲音也。

 

主要的是 TensorFlow 出狀況哩！

(tensorflow) pi@raspberry:~  cd
(tensorflow) pi@raspberry:~ 

 

就算 Up 後，空有4G 的 RAM ，七十五倍的 CPU POWER

pi@raspberrypi:~ 

 

up@up-UP-CHT01:~

 

在 AI 教育上亦乏學習園地呦？

故而擔心行還不行呀◎

俗話說︰人心不足蛇吞象。用以表達過份貪婪。據維基百科詞條講 ，語源出自《三海經》之『巴蛇食象』︰

山海經校注·海內南經

　　(山海經第十·山海經海經新釋卷五)

15、巴蛇食象，三歲而出其骨，君子服之，無心腹之疾①。其為蛇青黃赤黑②。一曰黑蛇青首③，在犀牛西。

① 　郭璞云：“今南方(虫丹)蛇(藏經本作蟒蛇——珂)吞鹿，鹿已爛，自絞於樹腹中，骨皆穿鱗甲間出，此其類也。楚詞曰：‘有蛇吞象，厥大何如?’說者云長 千尋。”郝懿行云：“今楚詞天問作‘一蛇吞象’，與郭所引異。王逸注引此經作‘靈蛇吞象’，並與今本異也。”珂案：淮南子本經篇云：“羿斷修蛇於洞庭。” 路史後紀十以“修蛇”作“長它 ”，羅苹注云：“長它即所謂巴蛇，在江岳間。其墓今巴陵之巴丘，在州治側。江源記(即江記，六朝宋庾仲雍撰 ——珂)云：‘羿屠巴蛇於洞庭，其骨若陵，曰巴陵也。’”岳陽風土記(宋范致明撰)亦云：“今巴蛇□在州院廳側，巍然而高，草木叢翳。兼有巴蛇廟，在岳陽 門內 。”又云：“象骨山。山海經云：‘巴蛇吞象。’暴其骨於此。山旁湖謂之象骨港。”是均從此經及淮南子附會而生出之神話。然而既有冢有廟，有山有港，言之確 鑿，則知傳播於民間亦已久矣。

②　珂案：言其文采斑爛也。

③ 　珂案：海內經云：“有巴遂山，澠水出焉。又有朱卷之國。有黑蛇，青首，食象。”即此。巴，小篆作□，說文十四云：“蟲也;或曰 ：食象蛇。象 形。”則所象者，物在蛇腹彭亨之形。山海經多稱大蛇 ，如北山經云：“大咸之山，有蛇名曰長蛇，其毛如彘毫，其音如鼓柝。”北次三經云：“錞於毋逢之山，是 有大蛇，赤首白身，其音如牛，見則其邑大旱。”是可以“吞象”矣。水經注葉榆河云：“山多大蛇 ，名曰髯蛇，長十丈，圍七八尺，常在樹上伺鹿獸，鹿獸過，便低頭繞之。有頃鹿死，先濡令濕訖，便吞，頭角骨皆鑽皮出。山夷始見蛇不動時，便以大竹籤籤蛇頭 至尾，殺而食之，以為珍異。”即郭注所謂(虫丹)蛇也。

───

 

然而從《山海經》的說法『君子服之，無心腹之疾。』來看，一點也沒有貪婪的意思吧！郭璞認為『巴蛇』是『蟒蛇』一類。無怪乎『TensorFlow』能在二十行內完成『手寫阿拉伯數字』辨識程式︰

MNIST For ML Beginners

This tutorial is intended for readers who are new to both machine learning and TensorFlow. If you already know what MNIST is, and what softmax (multinomial logistic) regression is, you might prefer this faster paced tutorial. Be sure to install TensorFlow before starting either tutorial.

When one learns how to program, there’s a tradition that the first thing you do is print “Hello World.” Just like programming has Hello World, machine learning has MNIST.

MNIST is a simple computer vision dataset. It consists of images of handwritten digits like these:

It also includes labels for each image, telling us which digit it is. For example, the labels for the above images are 5, 0, 4, and 1.

In this tutorial, we’re going to train a model to look at images and predict what digits they are. Our goal isn’t to train a really elaborate model that achieves state-of-the-art performance — although we’ll give you code to do that later! — but rather to dip a toe into using TensorFlow. As such, we’re going to start with a very simple model, called a Softmax Regression.

The actual code for this tutorial is very short, and all the interesting stuff happens in just three lines. However, it is very important to understand the ideas behind it: both how TensorFlow works and the core machine learning concepts. Because of this, we are going to very carefully work through the code.

……

 

# The MNIST Data
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Implementing the Regression
import tensorflow as tf

x = tf.placeholder(tf.float32, [None, 784])

W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))

y = tf.nn.softmax(tf.matmul(x, W) + b)

# Training
y_ = tf.placeholder(tf.float32, [None, 10])

cross_entropy = -tf.reduce_sum(y_*tf.log(y))

train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)

init = tf.initialize_all_variables()

sess = tf.Session()
sess.run(init)

for i in range(1000):
  batch_xs, batch_ys = mnist.train.next_batch(100)
  sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

# Evaluating Our Model
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))

 

This should be about 91%.

Is that good? Well, not really. In fact, it’s pretty bad. This is because we’re using a very simple model. With some small changes, we can get to 97%. The best models can get to over 99.7% accuracy! (For more information, have a look at this list of results.)

What matters is that we learned from this model. Still, if you’re feeling a bit down about these results, check out the next tutorial where we do a lot better, and learn how to build more sophisticated models using TensorFlow!

─── 摘自 《W!o+ 的《小伶鼬工坊演義》︰巴蛇食象》

 

 

 

 

 

 

 

 

將要選擇什麼打造 PIXEL PC 呢？幾經考慮看上了類似樹莓派 3B 的 Up-board ︰

Welcome to the UP Board wiki!

The purpose of these wiki pages is to collect and share all useful technical information related to UP Board hardware, software, and its usage.

Feel free to contribute content and corrections which may be useful to others!

 

依著 Up Ubuntu 安裝步驟，先下載 Raspbian 映像檔。成功開機後就卡在

sudo add-apt-repository ppa:ubilinux/up

上也。即使手動加入 repo

pi@raspberry:/etc/apt/sources.list.d  

 

，或因 Ubuntu 套件相容性問題，無法安裝 Up-board kernel 哩︰

pi@raspberry:~ 

 

只好鼻子摸著嘗試自行編譯的呦︰

Compile ubilinux kernel from source

sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test

sudo apt-get update
sudo apt-get dist-upgrade -y

sudo apt-get install -y git build-essential kernel-package libncurses5-dev libssl-dev gcc-4.9 g++-4.9

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.9

Getting the source

git clone https://github.com/emutex/ubilinux-kernel.git -b upboard-4.9 linux-upboard

to get only the latest commit (faster):

git clone --depth=1 https://github.com/emutex/ubilinux-kernel.git -b upboard-4.9 linux-upboard

 

偏偏 Debian Stretch 預設程式庫中沒有 kernel-package 套件，還得在 /etc/apt/sources.list 裡添上

deb http://ftp.debian.org/debian stretch-backports main

repo 呀！

更別說一時哪尋 gcc-4.9 g++-4.9 耶？且用新版

gcc version 6.3.0 20170516 (Debian 6.3.0-18)

，死馬當活馬醫吧！？

近乎三小時的等待，終於產生了 □□.deb 檔︰

pi@raspberry:~ 

 

好佳在

sudo dpkg -i *.deb

成功乎？！

※ 有圖為證

 

 

 

 

 

 

 

 

雖說︰

春耕夏耘可得時，

秋收冬藏守其法。

大道自如，變易能通◎

這用之於『教育』之『工具』亦然也☆

故爾樹莓派之

PIXEL for PC and Mac

Our vision in establishing the Raspberry Pi Foundation was that everyone should be able to afford their own programmable general-purpose computer. The intention has always been that the Raspberry Pi should be a full-featured desktop computer at a $35 price point. In support of this, and in parallel with our hardware development efforts, we’ve made substantial investments in our software stack. These culminated in the launch of PIXEL in September 2016.

 

PIXEL represents our best guess as to what the majority of users are looking for in a desktop environment: a clean, modern user interface; a curated suite of productivity software and programming tools, both free and proprietary; and the Chromium web browser with useful plugins, including Adobe Flash, preinstalled. And all of this is built on top of Debian, providing instant access to thousands of free applications.

Put simply, it’s the GNU/Linux we would want to use.

 

Back in the summer, we asked ourselves one simple question: if we like PIXEL so much, why ask people to buy Raspberry Pi hardware in order to run it? There is a massive installed base of PC and Mac hardware out there, which can run x86 Debian just fine. Could we do something for the owners of those machines?

So, after three months of hard work from Simon and Serge, we have a Christmas treat for you: an experimental version of Debian+PIXEL for x86 platforms. Simply download the image, burn it onto a DVD or flash it onto a USB stick, and boot straight into the familiar PIXEL desktop environment on your PC or Mac. Or go out and buy this month’s issue of The MagPi magazine, in stores tomorrow, which has this rather stylish bootable DVD on the cover.

 

You’ll find all the applications you’re used to, with the exception of Minecraft and Wolfram Mathematica (we don’t have a licence to put those on any machine that’s not a Raspberry Pi). Because we’re using the venerable i386 architecture variant it should run even on vintage machines like my ThinkPad X40, provided they have at least 512MB of RAM.

 

誠無心『楚河漢界』的呦！