Get best of the week

Tree - AST tunggal untuk memerintah semua

Halo, nama saya Dmitry Karlovsky dan saya ... membedah sepeda ... off-road ... melawan angin ... menanjak ... bermain ski. Dan hari ini saya mengundang Anda untuk ikut serta dalam dan melintasi format data teks dan bersama-sama merancang format yang ideal.


Saya sudah membicarakannya 5 tahun yang lalu, yang menyebabkan perdebatan sengit, yang memerlukan perubahan kecil dalam sintaksis. Karena itu, izinkan saya memberi tahu Anda dari awal apa itu saat ini.


 \ 
 \PiterJS #47
 2020-05-20

Ini adalah versi teks lanjutan dari kinerja eponymous di PiterJS # 47 . Anda dapat membacanya sebagai artikel , atau membukanya di antarmuka presentasi , atau menonton video .


Rencana


  • Menganalisis format data teks populer
  • Dari awal, kembangkan format baru tanpa cacat
  • Tunjukkan contoh penerapan format baru

Format


Kami akan membandingkan 5 format.


Format
XML
Json
Yaml
Tol
Pohon

Tentang tiga pertama tidak hanya mendengar tuli. Tetapi dua yang terakhir bagi banyak orang adalah kuda hitam. Nah, tidak ada apa-apa, hari ini saya akan menjelaskan mereka.


XML


XML — , " ". , , -.


<!DOCTYPE svg
    PUBLIC "-//W3C//DTD SVG 1.1//EN"
    "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"
>
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg">
    <circle r="30" cx="50" cy="50" fill="orange" />
</svg>

JSON


XML — JSON.


{
    "name": "example",
    "version": "1.0.0",
    "description": "example package",
    "main": "index.js",
    "repository": "https://example.org",
    "author": "anonymous",
    "license": "MIT"
}

, , , , .


YAML


- YAML JSON.


Date: 2001-11-23 15:03:17 -5
User: ed
Fatal:
  Unknown variable "bar"
Where:
  file: TopClass.py
  line: 23
  code: |
    x = MoreObject("345\n")

.


TOML


TOML . , , .


[servers]

[servers.alpha]
ip = "10.0.0.1"
dc = "eqdc10"

[servers.beta]
ip = "10.0.0.2"
dc = "eqdc10"

, , INI-, JSON. .


Tree


, , Tree, .


spoiler


. .


  • ?
  • ?

, .


XML


XML .


  • NodeList
  • Element Node (<br/>)
  • Attribute Node (tabindex="1")
  • Text Node (Hello, World!)
  • CDATA Node (<![CDATA[ ... ]]>)
  • Processing Instruction Node (<? ... ?>)
  • Comment Node (<!-- ... -->)
  • Document Node
  • Document Type Node (<!DOCTYPE html>)

XML


, : , . , XML , . , : .


<panel>
    <head> ?</head>
    <body>
        <button></button>
        <button></button>
    </body>
</panel>

panel — , body — , . , .


XML


, XML , .


<xsl:stylesheet
    version="1.0"
    xmlns="http://www.w3.org/1999/xhtml"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:template match="/">
        <html>
            <head>
                <link rel="stylesheet" href="web.css" />
            </head>
            <body>
                <xsl:apply-templates select="*" />
            </body>
        </html>
    </xsl:template>

</xsl:stylesheet>

— , .


JSON


JSON , . .


  • Null
  • Boolean
  • Number
  • String
  • Array
  • Dictionary

JSON


, . , . , .


{
    "foo": 777,
    "bar": 666
}

?


[
    [ "foo" , 777 ],
    [ "bar" , 666 ]
]

. - .


JSON


JSON , - , . MongoDB, , JSON .


{
    "$or": [
        {
            "sex": "female",
            "age": { "$gt": 16 },
        },
        {
            "hobby": {
                "$regex": "\\b(?:java|type)script\\b"
            }
        }
    ]
}

, OR AND . , "", "" " ". , , JSON , "$regexp", .


YAML


YAML JSON. .


  • !!null
  • !!bool
  • !!int
  • !!float
  • !!str
  • !!timestamp
  • !!seq
  • !!map
  • Anchor & Alias
  • Document
  • TypeTags

YAML


YAML , .


--- !!omap
- foo: 777
- bar: 666

" -" OrderedMap ( ).


TOML


TOML JSON, . , , , .


  • Boolean
  • Integer
  • Float
  • String
  • DateTime
  • Array
  • Dictionary

, JSON.


Tree


, . . , , . . .


  • Struct Node
  • Data Node

, . , .



, . , , , .


XMLJSONYAMLTOMLTree
+-+-+
90302109010

YAML. . , . , JSON , 30 .


, , .



: , , , , .


. .


XMLJSONYAMLTOMLTree
--+++

XML


XML , . , , , , .


, !
 ?
       ?

<message>
    <greeting>
        , <a href="http://example.org/user/alice"></a>!
    </greeting>
    <body>
        <s> ?</s><br/>
             
        <time datetime="1979-10-14T12:00:00.001-04:00"></time>
        ?
    </body>
</message>

JSON


XML , JSON, , . , .


{ "greetings": ", !\n ?\n      ?\n" }


. YAML .


XMLJSONYAMLTOMLTree
++-++

YAML


a: true # boolean
b: tru  # string
c: (-:  # error
d: :-)  # string

YAML .



— . . .



XML


XML — , .


foo > 0 && foo < 10

- , , .


`foo &gt; 0 &amp;&amp; foo &lt; 10`

JSON


JSON, . - VSCode, , JSON , .


/"[\s\S]*"/

, . , - .


"\"[\\s\\S]*\""

YAML


YAML , .


  • 5
  • 4

, YAML .


Tree

— . . , , , .



. .



XML


<users>
    <user>
        <name>Alice</name>
        <age>20</age>
    </user>
</users>

XML, , .


<!-- 13% less -->
<users><user><name>Alice</name><age>20</age></user></users>

JSON


{
    "users": [
        {
            "name": "Alice",
            "age": 20
        }
    ]
}

JSON , — .


// 30% less
{"users":[{"name":"Alice","age":20}]}

Tree

— , .



XMLJSONYAMLTOMLTree
195%140%125%110%100%
170%101%---

.


, , , , . , , , .



— , , .


  • ?
  • 2 4 ?
  • ?
  • ?
  • /?
  • //?

, . , , , . , - .



, .


, JSON, , . . . .


serialization:    foo\bar    =>  "foo\\bar"

parsing:         "foo\\bar"  =>   foo\bar

, , . , .



. , JSON, , - , , . , . , . .


XMLJSONYAMLTOMLTree
+---+
----+
----+

XML- , — . , , . - , .


, , . . , AST .



, , , . , . , . .


XMLJSONYAMLTOMLTree
--+++

, . , , — . , , , — .


, . , XML SAX , , : - , , - . JSON . , .


, . , , . , . .


Tree


, , .



tree-


, "house". ?


house

.


tree-


, ?


house
roof
wall
door
window
floor

.


tree-


?


house
    roof
    wall
    door
    window
    floor

. Python — , .


tree-


.


house
    roof
    wall
        door
        window
            glass
    floor


, - - .


street
    house
        wall
            door
            window

, .


street house wall
    window
    door

.



, - , - .


\  \(^_^)/

. . , , , . — , — .



? : .


\
    \ ‍
    \     ‍
    \          ‍

.



, . , .


user
    name \Jin
    age \35
    hobby
        \kendo 
        \dance 
        \role play 
            default

, . 2 4 .



, . , .


XMLXHTML, SVG, XSLT, ...
JSONJSON Schema, json:api, ...
YAMLyaml.org/type
TOML-
Treexml.tree, json.tree, view.tree, ...

— , .


tree.


grammar.tree


grammar.tree — . , tree.


tree .is .optional .list_of line

line .is .sequence
    .optional indent
    .optional nodes
    new_line

nodes .is .sequence
    .optional .list_of struct
    .optional data
    .with_delimiter space

struct .is .list_of .byte
    .except special

data .is .sequence
    data_prefix
    .optional .list_of .byte
        .except new_line

special .is .any_of
    new_line
    data_prefix
    indent
    space

new_line .is .byte \0A
indent .is .list_of .byte \09
data_prefix .is .byte \5C
space .is .list_of .byte \20

, , .


: tree — , — , . .


grammar.tree vs EBNF


grammar.tree , , , — , , , .


tree .is .optional .list_of line

line .is .sequence
    .optional indent
    .optional nodes
    new_line

nodes .is .sequence
    .optional .list_of struct
    .optional data
    .with_delimiter space

tree = { line };

line = [ indent ],
    [ nodes ],
    new_line;

nodes = data |
    struct,
    { space , struct },
    [ space , data ];

xml.tree vs XML


xml.tree — XML tree . XML. , XML xml.tree.


! doctype html
html
    meta @ charset \utf-8
    link
        @ href \web.css
        @ rel \stylesheet
    script @ src \web.js
    body
        h1 \Procter & Gamble

<!doctype html>
<html>

    <meta charset="utf-8" />
    <link href="web.css" rel="stylesheet" />
    <script src="web.js"></script>

    <body>
        <h1>Procter &amp; Gamble</div>
    </body>

</html>

IDE, XML xml.tree , XML. XML , , , markdown.


json.tree vs JSON


json.tree — json.


* user *
    name \Jin
    age 35
    hobby /
        \kendo 
        \dance 
    home \C:\users\jin\

{
    "user": {
        "name": "Jin",
        "age": 35,
        "hobby": [
            "kendo ",
            "dance ",
        ],
        "home": "C:\\users\\jin\\"
    }
}

2 — .


json.tree


, XML Tree , . , json, tree . , , . ?


*
    # \If disabled will be used platform specific delimiters
    # \CRLN on windows and LN on others
    unix_delimiters true

tree , .


{
    "unix_delimiters#1": "If disabled will be used platform specific delimiters",
    "unix_delimiters#2": "CRLN on windows and LN on others",
    "unix_delimiters": true,
}

JSON , - .


view.tree vs TypeScript


view.tree — $mol.


$my_details $mol_view
    sub /
        <= Pager $mol_paginator
            value?val <=> page?val 0

, . , view.tree json.tree , , JSON .


TypeScript . , .


class $my_details extends $mol_view {

    sub() { return [ this.Pager() ] }

    @ $mol_mem Pager() {
        const Pager = new $mol_paginator
        Pager.value = val => this.page( val )
        return Pager
    }

    @ $mol_mem page( val = 0 ) {
        return val
    }

}

API


, API .


API
XMLXHTML, SVG, XSLT, ...DOM, SAX, AST
JSONJSON Schema, json:api, ...Native, AST
YAMLyaml.org/typeNative, AST
TOML-Native, AST
Treexml.tree, json.tree, ...AST

XML, , DOM, SAX. , , , . JSON , — . . . , API.


JSON AST


JSON ASTExplorer.


{
  "user": {}
}

{
    "type" : "Object",
    "children" : [
        {
            "type" : "Property",
            "key" : {
                "type": "Identifier",
                "value": "user"
            }
            "value": {
                "type": "Object",
                "children": []
            }
        }
    ]
}

, AST . JSON AST. .


AST Tree


tree .


user
    name \Jin
    age 35
    hobby
        \kendo 
        \dance 
        \role play

AST.


user
    name \Jin
    age 35
    hobby
        \kendo 
        \dance 
        \role play

, - . . , , , tree — AST.


Tree


TypeScript .


interface $mol_tree2 {
    type: string
    value: string
    kids: $mol_tree2[]
    span: $mol_span
}

Span — .


interface $mol_span {
    uri: string
    row: number
    col: number
    length: number
}

Tree


, . , , span . .


interface $mol_tree2 {
    struct : ( type , kids )=> $mol_tree2
    data : ( value , kids )=> $mol_tree2
    list : ( kids )=> $mol_tree2
    clone : ( kids )=> $mol_tree2
}

Tree


, , — , .


const config_path = './config.tree'
const config_text = fs.readFileSync( config_path )
const config = $mol_tree2.fromString( config_text , config_path )
//  server auth
//      login \root
//      password \qwerty

const password = config.select( 'server' , 'auth' , 'password' , '' )

if( !auth( password.text() ) ) {
    // AuthError: Wrong password
    // \default
    // ./config.tree#5:3-11
    throw password.error( 'Wrong password' , AuthError )
}

Tree


— , "auth" "credentials". :


//  server credentials
//      login \root
//      password \qwerty
const new_config = config.list(
    input.hack({

        'auth' : ( tree , context )=> [
            tree.struct( 'credentials' , tree.hack( context ) ),
        ] ,

        '' : ( tree , context )=> [
            tree.clone( tree.hack( context ) ),
        ] ,

    })
)
fs.writeFileSync( config_path , new_config )

, tree , , AST.




, . , .




, .



XMLJSONYAMLTOMLTree
195%140%125%110%100%
90302109010
++-++
--+++
----+
----+
--+++
+-+-+
+++--


, tree .


  • LISP-
  • AST

sql.tree —


MongoDB? SQL:


select
    from $users
    fetch
        @name
        @phone
        @photo *
            @uri
            @width
            @height
    where or
        and
            @sex = female
            @age > 16
        @hobby ~ \\b(?:java|type)script\b

— , SQL. , "", "" " ". , tree, .


select
    from $users
    fetch *
    where @hobby ~ 
        word-edge
        or
            \java
            \type
        \script
        word-edge

domain.tree —


. .


hyoo_api_person
    descr \  
    inherit hyoo_api_entity
    field
        id
            descr \  
            example \person=jin
            key unique
            type text
            edit author
        avatar
            descr \  
            type list hyoo_api_image
            edit author
        mail
            descr \ 
            type set hyoo_api_mail

API, ACL, .



. — , . , , , — .



access.log.tree —


, , ?


193.34.12.132 - - [2011-10-20T12:46:08+04:00] GET /nin-jin/slides/edit/master/t
ree/readme.md HTTP/1.1 200 4435
193.34.12.132 - - [2011-10-20T12:46:09+04:00] GET /nin-jin/slides/edit/master/t
ree/readme.html HTTP/1.1 404 4435

access
    ip \193.34.12.132
    time \2011-10-20T12:46:08+04:00
    method \GET
    uri \/nin-jin/slides/edit/master/tree/readme.md
    protocol \HTTP/1.1
    response \200
    size \4435

tree-tools — CLI


, . , , , .


> cat access.log.tree | pick ip time method uri | table

\193.34.12.132  2011-10-20T12:46:08+04:00   GET /index.html
\193.34.12.132  2011-10-20T12:46:10+04:00   GET /index.css
\193.34.12.132  2011-10-20T12:46:20+04:00   GET /index.js

> cat access.log.tree | filter time >= 2019-09 | pick ip uri | table

\193.34.12.132  /index.html
\193.34.12.132  /index.css
\193.34.12.132  /index.js

, - . , - . , , .


tree


tree, , . JSON XML, — , . , - , .


> git log

commit
    message \$mol_style: TS@3.9 compatibility
    sha \b1a8f07c839604d0d34430a186246f0c1f71e628
    date \2020-05-15T23:24:32+0300
    author \nin-jin <sairi-na-tenshi@ya.ru>
commit
    message \$mol_regexp: concurent parse ability
    sha \be1abfa50542728dd5c156517ea31f469e7fb4d4
    date \2020-05-15T23:03:30+0300
    author \nin-jin <nin-jin@ya.ru>

> git log | pick date message | table

\2020-05-15T23:24:32+0300   $mol_style: TS@3.9 compatibility
\2020-05-15T23:03:30+0300   $mol_regexp: concurent parse ability

WAT


WebAssembly — , . , s-expressions.


(func $fact (param $x i64) (result i64)
    (if $x (result i64) 
      (i64.eqz
        (local.get $x)) 
      (then
        (i64.const 1))
      (else
        (i64.mul
          (local.get $x)
          (call $fact      
            (i64.sub
              (local.get $x)
              (i64.const 1)))))))

. , .


wasm.tree —


wasm.tree .


func
    name $fact
    param $x i64
    result i64
    body switch
        test i64.eqz local.get $x
        then i64.const 1
        else i64.mul
            local.get $x
            call $fact i64.sub
                local.get $x
                64.const 1

- bin.tree, .


00
61
73
6d
01
00
00
00
.
.
.

- - — WAT2.0. WebAssembly — .


jack.tree — LISP


. -, . , , .


jack
    import wasm
    tree func $fact
        > $x #8
        < #8 switch
            test is-zero $x
            then #8 1
            else mul
                $x
                $fact sub
                    $x
                    #8 1

, AST , wasm-. , , , , tree , . .


$mol_jack


LLVM


wasm , , .


compile pipelines:

                jack.tree => wasm.tree =============> bin.tree
                jack.tree => wasm.tree => arm.tree => bin.tree
any-dsl.tree => jack.tree => wasm.tree => arm.tree => bin.tree

, , , .


optimization midlewares:

jack.tree => jack.tree
wasm.tree => wasm.tree
arm.tree => arm.tree

, , , . AST tree .


, , LLVM.


AST


, , . Tree — AST. , TypeScript .


code =(P)=> loader =(P)=> compiler =(SP)=> bundler =(SP)=> terser =(S)=> bundle

P - Parse
S - Serialize

AST, , . AST, .


code =(P)=> loader =====> compiler ======> bundler ======> terser =(S)=> bundle

( ), tree AST , .


,


. . , , . , .


More articles:


All Articles