Michael Nielsen 先生用『小品文』essay 形式、視覺『驗證』方法、動態『操作』來展現『神經網絡』可以計算『任何函數』的能力。既說是『』品的了，還請讀者一小口一小口細嚼慢嚥也。

One of the most striking facts about neural networks is that they can compute any function at all. That is, suppose someone hands you some complicated, wiggly function, $f(x)$ $f (x)$ :

……

Summing up, a more precise statement of the universality theorem is that neural networks with a single hidden layer can be used to approximate any continuous function to any desired precision. In this chapter we’ll actually prove a slightly weaker version of this result, using two hidden layers instead of one. In the problems I’ll briefly outline how the explanation can, with a few tweaks, be adapted to give a proof which uses only a single hidden layer.

……

Conclusion

The explanation for universality we’ve discussed is certainly not a practical prescription for how to compute using neural networks! In this, it’s much like proofs of universality for NAND gates and the like. For this reason, I’ve focused mostly on trying to make the construction clear and easy to follow, and not on optimizing the details of the construction. However, you may find it a fun and instructive exercise to see if you can improve the construction.

Although the result isn’t directly useful in constructing networks, it’s important because it takes off the table the question of whether any particular function is computable using a neural network. The answer to that question is always “yes”. So the right question to ask is not whether any particular function is computable, but rather what’s a good way to compute the function.

The universality construction we’ve developed uses just two hidden layers to compute an arbitrary function. Furthermore, as we’ve discussed, it’s possible to get the same result with just a single hidden layer. Given this, you might wonder why we would ever be interested in deep networks, i.e., networks with many hidden layers. Can’t we simply replace those networks with shallow, single hidden layer networks?

Chapter acknowledgments: Thanks to Jen Dodd and Chris Olah for many discussions about universality in neural networks. My thanks, in particular, to Chris for suggesting the use of a lookup table to prove universality. The interactive visual form of the chapter is inspired by the work of people such as Mike Bostock, Amit Patel, Bret Victor, and Steven Wittens.

While in principle that’s possible, there are good practical reasons to use deep networks. As argued in Chapter 1, deep networks have a hierarchical structure which makes them particularly well adapted to learn the hierarchies of knowledge that seem to be useful in solving real-world problems. Put more concretely, when attacking problems such as image recognition, it helps to use a system that understands not just individual pixels, but also increasingly more complex concepts: from edges to simple geometric shapes, all the way up through complex, multi-object scenes. In later chapters, we’ll see evidence suggesting that deep networks do a better job than shallow networks at learning such hierarchies of knowledge. To sum up: universality tells us that neural networks can compute any function; and empirical evidence suggests that deep networks are the networks best adapted to learn the functions useful in solving many real-world problems.

───

所謂『universality』哲學上譯作『共相』

共相（英語：Universal），又譯為普遍性，是一哲學與佛學常用詞彙，但兩者意思不盡相同。在形上學中，共相是指在個別物體中所擁有的共通特性。舉例來說，在房間中，存在兩張個別的綠色椅子，在這兩張椅子中，都存在共通的「椅子」特質，以及「綠色」的性質。各別的椅子，稱為殊相，而這兩個各別椅子中共通的特質，被稱為共相。

……

須慎思明辨古今東西有

共相之爭

在中世紀，哲學家曾因共相是否真實存在開始了共相之爭，而當中有三種不同的主流見解：

唯實論：觀念世界比感官世界更早出現，因此共相早於事物。
唯名論：和唯實論抱持的觀點相反，認為共相後於事物。
概念論：綜合了唯名論和唯實論兩個學說，提出共相存在於事物當中。

彼得·阿伯拉的哲學名聲與其指稱理論（doctrine of name）或唯名論（nominalism）有關。彼得·阿伯拉認為字詞只是名稱，即「能指」（signifiers）。並非所有的字詞都能指稱實體。彼得·阿伯拉特別指出，這個問題可上溯至柏拉圖與亞里斯多德的討論，例如用來指稱類別、性質與理想類型的字詞。problem of universals在於，這些用來指稱類別、性質與理想類型的字詞，實際上指稱的是不是實在的實體。

有些邏輯學家－－被稱為「唯實論者」（realist）－－堅持類別、性質與理想類型這些特有的實體的確存在，其他邏輯學家－－被稱為「概念論者」（conceptualist）－－則堅持，共相只存在於心靈中。彼得·阿伯拉採取激進的觀點，認為除了個別事物之外，任何事物均不存在。他反對共相存在，並且否認唯實論者的觀點，後者主張事物擁有本質，而本質可讓事物如其所是。彼得·阿伯拉則認為，世上並不存在本質，字詞以共相來誤導我們的思考，但是共相並非實在之物；當我們使用語言時，字詞只是我們假設的建構物。

彼得·阿伯拉運用字詞與實在之間的嚴格區分來詮釋《聖經》，他認為宗教權威之間的明顯衝突，可藉由觀察他們如何以不同方式使用同一字詞，輕易獲得解決。彼得·阿伯拉是第一位以現代意義使用「神學」一詞的哲學家，他用神學來指稱針對宗教神祕所進行的理性調查。彼得·阿伯拉重拾已經持續千年的論辯，主張應以理性來審視啟示，凡是未經理性辯護的信仰，都只能算是意見。由於彼得·阿伯拉相信理性能洞察宗教真理，因此他堅持古希臘人對於基督教教義有著值得讚賞的貢獻，即使他們僅僅約略瞥見了三位一體的本質。

「共相之爭」爭論的重點，在於「知識」與「本體」間互動的課題。原來，柏拉圖的「觀念」、「感官」的二元世界，以「觀念界」的先天性，作為「感官」後天「知識」的基準，因此認為「知識」先於「感官」事物，這就形成傾向柏拉圖思想的「實在論」（Realismus），甚至發展成「極端實在論」（Exaggerate Realism），此派主張「共相先於事物」（Universale ante rem）；而亞里斯多德的學說，則以為「知識」當始自「感官」經驗，而「觀念」則為「知識」後天所獲得，偏向亞里斯多德的學者主張「共相後於事物」（Universale post rem）。這種你來我往的辯論，到後來各說各話，一直到「大學」成立，學術漸上軌道後，才出現折衷派的「共相在事物中」（Universale in re）；算是表面上平息了「共相之爭」。

「共相」問題，基本上是蘇格拉底的「概念」（eidos, concept），柏拉圖的「觀念」（idea），亞里斯多德的「實體」（ousia, substance），是在檢證吾人的「思想」如何把握「存在」的「知識論」根本課題。「先物共相」（共相先於事物）是柏拉圖的主張，「後物共相」（共相後於事物）是亞里斯多德跟隨乃師蘇格拉底的學說，而「在物共相」（共相在事物之中）則是綜合和折衷二者所形成。當然，這種折衷也就是中世紀「共相之爭」的「知識論」成果之一。

───

須詳參細究其與『圖零完備性』之關聯︰

Turing completeness

In computability theory, a system of data-manipulation rules (such as a computer’s instruction set, a programming language, or a cellular automaton) is said to be Turing complete or computationally universal if it can be used to simulate any single-taped Turing machine. The concept is named after English mathematician Alan Turing. A classic example is lambda calculus.

A closely related concept is that of Turing equivalence – two computers P and Q are called equivalent if P can simulate Q and Q can simulate P. The Church–Turing thesis conjectures that any function whose values can be computed by an algorithm can be computed by a Turing machine, and therefore that if any real-world computer can be simulated by a Turing machine, it is Turing equivalent to a Turing machine. A Universal Turing machine can be used to simulate any Turing machine and by extension the computational aspects of any possible real-world computer.^{[NB 1]}

To show that something is Turing complete, it is enough to show that it can be used to simulate some Turing complete system. For example, an imperative language is Turing complete if it has conditional branching (e.g., “if” and “goto” statements, or a “branch if zero” instruction. See OISC) and the ability to change an arbitrary amount of memory (e.g., the ability to maintain an arbitrary number of variables). Since this is almost always the case, most (if not all) imperative languages are Turing complete if the limitations of finite memory are ignored.

……

Non-mathematical usage

In colloquial usage, the terms “Turing complete” or “Turing equivalent” are used to mean that any real-world general-purpose computer or computer language can approximately simulate the computational aspects of any other real-world general-purpose computer or computer language.

Real computers constructed so far are essentially similar to a single-tape Turing machine; thus the associated mathematics can apply by abstracting their operation far enough. However, real computers have limited physical resources, so they are only linear bounded automaton complete. In contrast, a universal computer is defined as a device with a Turing complete instruction set, infinite memory, and infinite available time.

───

或可借著

紙張、鉛筆和橡皮擦