有興趣研究『自修改程式碼』

Self-modifying code

In computer science, self-modifying code is code that alters its own instructions while it is executing – usually to reduce the instruction path length and improve performance or simply to reduce otherwise repetitively similar code, thus simplifying maintenance. Self-modification is an alternative to the method of “flag setting” and conditional program branching, used primarily to reduce the number of times a condition needs to be tested. The term is usually only applied to code where the self-modification is intentional, not in situations where code accidentally modifies itself due to an error such as a buffer overflow.

The method is frequently used for conditionally invoking test/debugging code without requiring additional computational overhead for every input/output cycle.

The modifications may be performed:

• only during initialization – based on input parameters (when the process is more commonly described as software ‘configuration‘ and is somewhat analogous, in hardware terms, to setting jumpers for printed circuit boards). Alteration of program entry pointers is an equivalent indirect method of self-modification, but requiring the co-existence of one or more alternative instruction paths, increasing the program size.
• throughout execution (‘on-the-fly’) – based on particular program states that have been reached during the execution

In either case, the modifications may be performed directly to the machine code instructions themselves, by overlaying new instructions over the existing ones (for example: altering a compare and branch to an unconditional branch or alternatively a ‘NOP‘).

In the IBM/360 and Z/Architecture instruction set, an EXECUTE (EX) instruction logically overlays the second byte of its target instruction with the low-order 8 bits of register 1. This provides the effect of self-modification although the actual instruction in storage is not altered.

Application in low and high level languages

Self-modification can be accomplished in a variety of ways depending upon the programming language and its support for pointers and/or access to dynamic compiler or interpreter ‘engines’:

• overlay of existing instructions (or parts of instructions such as opcode, register, flags or address) or
• direct creation of whole instructions or sequences of instructions in memory
• creating or modification of source code statements followed by a ‘mini compile’ or a dynamic interpretation (see eval statement)
• creating an entire program dynamically and then executing it

 

的人，當然不能錯過『紅男爵』也。

PyCQA/redbaron

Bottom-up approach to refactoring in python http://redbaron.pycqa.org/

Introduction

RedBaron is a python library and tool powerful enough to be used into IPython solely that intent to make the process of writing code that modify source code as easy and as simple as possible. That include writing custom refactoring, generic refactoring, tools, IDE or directly modifying you source code into IPython with a higher and more powerful abstraction than the advanced texts modification tools that you find in advanced text editors and IDE.

RedBaron guaranteed you that it will only modify your code where you ask him to. To achieve this, it is based on Baron a lossless AST for Python that guarantees the operation ast_to_code(code_to_ast(source_code)) == source_code. (Baron’s AST is called an FST, a Full Syntax Tree).

RedBaron API and feel is heavily inspired by BeautifulSoup. It tries to be simple and intuitive and that once you’ve get the basics principles, you are good without reading the doc for 80% of your operations.

For now, RedBaron can be considered in alpha, the core is quite stable but it is not battle tested yet and is still a bit rough.Feedback and contribution are very welcome.

The public documented API on the other side is guaranteed to be retro-compatible and won’t break until 2.0 (if breaking is needed at that point). There might be the only exception that if you directly call specific nodes constructors with FST that this API change, but this is not documented and simply horribly unpracticable, so I’m expecting no one to do that.

 

此處特別以樹莓派來談安裝︰

sudo pip3 install redbaron[pygments]

pi@raspberrypi:~ 

 

是說︰用 Raspberry Pi 跑 JupyterLab 沒問題呦☆

 

所以借 ROCK64 講『機器學習』，蓋許多訓練及測試之『資料集』常太大， Raspberry Pi 1G RAM 恐不夠矣。

既然 RedBaron 的文件和範例皆完整︰

Learn how to use RedBaron

This tutorial guides you through the big principles of RedBaron and highlights the most useful helpers and tricks. It is more or less a lighter version of the already existing documentation.

A reminder before starting:

• RedBaron doesn’t do static analysis and will never do (but it’s very likely that it will be combined with tools that do it, like astroid or rope, to bring static analysis into RedBaron or easy source code modification in the others)

The structure of this tutorial is similar to the documentation’s:

• basic principles and how to use it in a shell
• how to query the tree
• how to modify the tree
• how to play with list of things
• miscellaneous but useful stuff

from redbaron import RedBaron

red = RedBaron("code source as a string")
red.dumps()

Input and output with the source code in a file:

from redbaron import RedBaron

with open("code.py", "r") as source_code:
    red = RedBaron(source_code.read())

with open("code.py", "w") as source_code:
    source_code.write(red.dumps())

Now that you know how to load your code into RedBaron, let’s talk about its principles:

• RedBaron represents the source code as a tree. This is because when you are writing source code (of any classical language), you are actually writing a tree structure in the source file.
• For example: in 1 + 2 the top node is +, the left one is 1 and the right one is 2.
• In (1 + 2) + 3 the top node is, again, +, but the left one is actually (1 + 2) which is again another + node! This structure is a tree.
• The classical approach to handle such a structure is to use an Abstract Syntax Tree (AST) (it is used by compilers and interpreters like cpython).
• RedBaron, by relying on Baron, uses a Full Syntax Tree (FST). It’s like an AST except it keeps every information, included formatting, and is then a lossless representation of the source code. Under the hood, the FST produced by Baron is in JSON and has been thought to be read and used by humans (although not as easily as RedBaron).
• So, when BeautifulSoup wraps the HTML datastructure into objects, RedBaron does the same thing for the FST datastructure and provides a nice way to interact with the source code.

Example of an AST for some language that looks like Go:

While you don’t have to do that to use RedBaron on a daily basis, seeing the produced FST can help your understand RedBaron better (every key that has “_formatting” in its name is formatting related):

In [1]: import json

In [2]: red = RedBaron("1+2")

In [3]: print(json.dumps(red.fst(), indent=4))  # json.dumps is used for pretty printing
[
    {
        "first_formatting": [], 
        "value": "+", 
        "second_formatting": [], 
        "second": {
            "section": "number", 
            "type": "int", 
            "value": "2"
        }, 
        "type": "binary_operator", 
        "first": {
            "section": "number", 
            "type": "int", 
            "value": "1"
        }
    }
]

 

動手就開始哩◎