Inside the Python virtual machine. Part 1

Introduction

Python and CPython are used as synonyms in this text, but any mention of Python means CPython (the python version implemented in C). Other implementations include PyPy (python implemented in a limited subset of Python), Jython (implementation on the Java Virtual Machine), etc.

1. Initialization - this step involves setting up the various data structures required by the python process. Most likely this will happen when the program is executed in non-interactive mode through the shell of the interpreter.
2. — , : , , .
3. — .

This book uses the Python 3 version.

30,000-foot view

This article assumes you are using a Unix-based operating system, so some features may vary on Windows.

 1     typedef struct _is {
 2 
 3         struct _is *next;
 4         struct _ts *tstate_head;
 5 
 6         PyObject *modules;
 7         PyObject *modules_by_index;
 8         PyObject *sysdict;
 9         PyObject *builtins;
10         PyObject *importlib;
11 
12         PyObject *codec_search_path;
13         PyObject *codec_search_cache;
14         PyObject *codec_error_registry;
15         int codecs_initialized;
16         int fscodec_initialized;
17 
18         PyObject *builtins_copy;
19     } PyInterpreterState;

1. The * next field is a reference to another instance of the interpreter, since several Python interpreters can exist within the same process.
2. The * tstate_head field indicates the main thread of execution (if the program is multi-threaded, then the interpreter is common for all threads created by the program). We will discuss this in more detail shortly.
3. modules, modules_by_index, sysdict, builtins and importlib speak for themselves. All of them are defined as instances of PyObject , which is the root type for all objects in the Python virtual machine. Python objects will be discussed in more detail in the following chapters.
4. The fields related to codec * contain information that helps with downloading encodings. This is very important for decoding bytes.

 1     typedef struct _ts {
 2         struct _ts *prev;
 3         struct _ts *next;
 4         PyInterpreterState *interp;
 5 
 6         struct _frame *frame;
 7         int recursion_depth;
 8         char overflowed; 
 9                         
10         char recursion_critical; 
11         int tracing;
12         int use_tracing;
13 
14         Py_tracefunc c_profilefunc;
15         Py_tracefunc c_tracefunc;
16         PyObject *c_profileobj;
17         PyObject *c_traceobj;
18 
19         PyObject *curexc_type;
20         PyObject *curexc_value;
21         PyObject *curexc_traceback;
22 
23         PyObject *exc_type;
24         PyObject *exc_value;
25         PyObject *exc_traceback;
26 
27         PyObject *dict;  /* Stores per-thread state */
28         int gilstate_counter;
29 
30         ... 
31     } PyThreadState;

, Py_INCREF Py_DECREF. , . CPython : , , Py_INCREF. , , Py_DECREF.

1         >>> def square(x):
2         ...     return x*x
3         ... 
4 
5         >>> dis(square)
6         2           0 LOAD_FAST                0 (x)
7                     2 LOAD_FAST                0 (x)
8                     4 BINARY_MULTIPLY     
9                     6 RETURN_VALUE        

• Where did the values ​​loaded by the LOAD_FAST statement come from?
• Where do the arguments come from, which are used as part of the instructions?
• How are nested function and method calls handled?
• How does the interpreter loop handle exceptions?