每日彙整: 2019-03-10

樹莓派、樹莓派之學習、樹莓派之教育

Rock It 《ML》JupyterLab 【丁】Code《七》語義【四】II

或許我們應該換個視角，聽聽 AST 男爵 Baron 對於 FST 的說法︰

Welcome to Baron’s documentation!

Introduction

Baron is a Full Syntax Tree (FST) for Python. It represents source code as a structured tree, easily parsable by a computer. By opposition to an Abstract Syntax Tree (AST) which drops syntax information in the process of its creation (like empty lines, comments, formatting), a FST keeps everything and guarantees the operation fst_to_code(code_to_fst(source_code)) == source_code.

……

 

了解『充分語法樹』的重要性︰

If you want to understand why this is important, read this:https://github.com/PyCQA/baron#why-is-this-important

Why is this important?

The usage of a FST might not be obvious at first sight so let’s consider a series of problems to illustrate it. Let’s say that you want to write a program that will:

  • rename a variable in a source file… without clashing with things that are not a variable (example: stuff inside a string)
  • inline a function/method
  • extract a function/method from a series of line of code
  • split a class into several classes
  • split a file into several modules
  • convert your whole code base from one ORM to another
  • do custom refactoring operation not implemented by IDE/rope
  • implement the class browser of smalltalk for python (the whole one where you can edit the code of the methods, not just showing code)

It is very likely that you will end up with the awkward feeling of writing clumpsy weak code that is very likely to break because you didn’t thought about all the annoying special cases and the formatting keeps bothering you. You may end up playing with ast.py until you realize that it removes too much information to be suitable for those situations. You will probably ditch this task as simply too complicated and really not worth the effort. You are missing a good abstraction that will take care of all of the code structure and formatting for you so you can concentrate on your task.

The FST tries to be this abstraction. With it you can now work on a tree which represents your code with its formatting. Moreover, since it is the exact representation of your code, modifying it and converting it back to a string will give you back your code only modified where you have modified the tree.

Said in another way, what I’m trying to achieve with Baron is a paradigm change in which writing code that will modify code is now a realist task that is worth the price (I’m not saying a simple task, but a realistic one: it’s still a complex task).

Other

Having a FST (or at least a good abstraction build on it) also makes it easier to do code generation and code analysis while those two operations are already quite feasible (using ast.py and a templating engine for example).

Some technical details

Baron produces a FST in the form of JSON (and by JSON I mean Python lists and dicts that can be dumped into JSON) for maximum interoperability.

Baron FST is quite similar to Python AST with some modifications to be more intuitive to humans, since Python AST has been made for CPython interpreter.

Since playing directly with JSON is a bit raw I’m going to build an abstraction on top of it that will looks like BeautifulSoup/jQuery.

 

是否想要玩玩看呢？

 

千萬別驚訝 AH HA ，src \ \neq \ source\_code 呦★

不過緣起於最後一行，作者少打了\n耳☺

如此也知『充分語法樹』之生不容易哩☆