Get best of the week

The list of changes in D 2.092. Borrowing Borrowing

From the translator. I usually don’t translate such articles, because I don’t see anything interesting in them. Here, two small, but most important innovations - support for direct linking of C ++ libraries and the “borrowed” system of ownership and borrowing from Rust (the second is the DFA swallow, the first came out in GCC10 just a couple of days ago, and Nim also thinks about it like that) with a description. Actually, I did not translate any tedious lists, and the links lead to the original.

Previous version of the list 2.0.91.1 (eng)
Where to download

2.092.0 came out with 15 important changes and 44 bug fixes and improvements. Many thanks to the 47 contributors who made this release possible.

Compiler changes


  1. The -revert = import and -transition = checkimports command line switches removed
  2. Added support for coding (mangling) of C ++ GNU ABI names
  3. Constructors and destructors for modules that are not extern (D) are marked obsolete
  4. DIP25 Violations Marked By Default Obsolete
  5. Prototype Ownership and Borrowing System for Pointers
  6. Added -preview = in option that turns the storage class in into scope const
  7. Printf and scanf parameters are now checked for compliance with format specifiers
  8. The environment variable SOURCE_DATE_EPOCH is now supported.

Runtime changes


  1. TypeInfo_Class/TypeInfo_Interface.isBaseOf C#/Java isAssignableFrom
  2. core.memory.pageSize minimumPageSize


  1. Date.isoWeekYear Date.fromISOWeek std.datetime.date
  2. std.xml
  3. std.digest.digest

Dub


  2. Dub lint --report-file

A complete list of corrections and improvements (Eng.)

Note.I have paws, therefore, it’s not possible to set up correct intra-page links from the table of contents to disclose the topics in the original, sorry = (

Compiler changes


  1. The command line switches -revert = import and -transition = checkimports have been deleted.

    These keys did not do anything, and for some time they were marked obsolete. The compiler will no longer recognize them.
  2. () C++ GNU ABI

    GNU ABI C++11 GCC 5.1. D C++, DMD UDA ( ) gnuAbiTag, core.attribute object ( ). ABI , , ++ . , std::string C++11 (DMD -extern-std={c++11,c++14,c++17}).

    :

    extern(C++):
@gnuAbiTag("tagOnStruct")
struct MyStruct {}
@gnuAbiTag("Multiple", "Tags", "On", "Function")
MyStruct func();

    gnuAbiTag. ( ). UDA -extern-std=c++11. (-extern-std=c++98) UDA . UDA extern(C++) .
  3. , extern(D) .

    ( ) , extern(D), . , , / , , 479, 479, .

    , , .
  4. DIP25 .

    DIP25 2.067.0, , -preview=dip25. , , DIP1000.

    , , -preview=dip25, -preview=dip25. ( ).

    DIP25 , @ safe . , ref , return , .

    struct Foo
{
    int x;
    // returning `this.x` escapes a reference to parameter `this`, perhaps annotate with `return`
    ref int method() /* return */ { return this.x; }
}
// returning `v` escapes a reference to parameter `v`, perhaps annotate with `return`
ref int identity(/* return */ ref int v) { return v; }

    return . DIP25 .
  5. -preview=in in scope const.

    const scope, in . , in (. inout ) .

    -preview=in, :

    void fun(in int x);
void fun(const int x);

    -preview=in, :

    void fun(in int x);
void fun(scope const int x);
  6. printf scanf

    C99 7.19.6.1 printf 7.19.6.2 scanf.

    printf , . , .

    scanf .

    :

    1. ,
    2. ,
    6. — , s
    8. C99

    -, .

    .

    , printf/scanf, :

    printf("%k\n", value);  // error: non-Standard format k
const format = "%k\n";
printf(format.ptr, value);  // no error

    — . ,

    string s;
printf("%.*s\n", s.length, s.ptr);
printf("%d\n", s.sizeof);
ulong u;
scanf("%lld%*c\n", &u);

    :

    string s;
printf("%.*s\n", cast(int) s.length, s.ptr);
printf("%zd\n", s.sizeof);
ulong u;
scanf("%llu%*c\n", &u);

    printf scanf pragma(printf) printf pragma(scanf) scanf.

    , :

    1. extern (C ) extern (C++)
    2. const(char)*
    3. … -v , va_list ( «v» printf scanf)
    .

    «v»- .
  7. SOURCE_DATE_EPOCH

    SOURCE_DATE_EPOCH. UNIX ( 1970-01-01 00:00:00), . DMD __DATE__, __TIME__ __TIMESTAMP__ .



The ownership / borrowing system (also known as OB) for pointers ensures that dereferenced pointers point to a valid memory object.

Scope of the prototype OB system


This is a prototype of the OB system, adapted for D. Initially, it is intended only for pointers, and not for dynamic arrays, class references, references or pointer fields in aggregates. Adding support of this kind will complicate the implementation, but does not change its essence, therefore, it is postponed for later. RAII objects can safely manage their own memory, so they are not covered by OB. Regardless of the method, whether the pointer allocates memory using GC or some other allocator, this is not important for OB, they do not differ and are processed identically.

The OB system is active only in functions annotated with the @ live attribute. It is applied after semantic processing solely as a check for violation of the rules of organic substances. New syntax is not added. No changes are made to the generated code. If @ live functions call functions other than @ live , it is expected that these called functions will be an @ live- compatible interface, although it is not tested. If non- @ live functions call @ live functions , the arguments passed are expected to follow @ live conventions .

The OB system detects errors:

  • dereferencing pointers that are in an invalid state
  • more than one active pointer to a mutated memory object.

It will not detect attempts to dereference a null pointer or, potentially, a null pointer. This does not work, since there is currently no method for annotating a type as a non-nullable pointer.

The basic principle of OB


OB design follows from the following principle:

For each memory object, there can be exactly one mutating pointer to it, or several non-mutating (read-only) pointers.

Design


A single mutating pointer is called the "owner" of the memory object. He transitively owns a memory object and all the objects in memory accessible from it (i.e. the graph of objects). Since it is the only pointer to this memory object, it can safely manage the memory (change its shape, allocate, free and resize) without knocking the soil out under the feet of any other pointers (mutating or not) that may be on it (graph memory) indicate.

If there are several pointers to a graph of memory objects that are read-only, then they can safely read from it without worrying about sudden changes in the graph of objects in memory.

The rest of the technology’s details relate to how pointers become read-only and invalid, and how the Core Principle of OB is constantly maintained.

Tracked pointers
Only those pointers that are declared in the @ live function as this , function parameters, or local variables are tracked . Variables from other functions are not tracked, even @ live , since the analysis of interactions with other functions completely depends only on the signature of this function, and not on its internals. Parameters that are const are not tracked.

Pointer states
Each pointer is in one of the following states:

Undefined
The pointer is in an invalid state. Dereferencing such a pointer is a mistake.
Owner The
owner is the only pointer to a graph of objects in memory. Usually a pointer - Owner does not have a scope attribute . If the pointer with the scope attribute is initialized with an expression that is not derived from the tracked pointer, then it becomes the Owner.
If the Owner pointer is assigned to another pointer - the Owner, then the first goes into the Undefined state.
Borrowed
A borrowed pointer is a pointer that temporarily becomes the only pointer to a graph of memory objects. He passes into this state through assignment from the pointer - the Owner, while the Owner passes into the state of the Borrower until the use of the borrowed pointer is completed.
A borrowed pointer must have a scope attribute and be a pointer to a mutable object.
Read-
Only The Read- Only Index is assigned by the Owner. As long as the Read-Only pointer is alive, only Read-Only pointers can still be assigned from this Owner. The read-only pointer must have a scope attribute , and it must also not be a pointer to a mutable object.

Lifetimes

The lifetime of the "Borrowed" or "Read-only" indicator starts from the moment it is first dereferenced (and not from the moment it is initialized or assigned a value) and ends when the value is dereferenced last.

It is also known as Non-Lexical Lifetimes .

State transitions for pointers

A pointer changes its state when one of these operations is performed:

  • memory is allocated for it (for example, a local variable on the stack), which places the pointer in an undefined state
  • initialization (considered as assignment)
  • assignment - the source and target pointers change state depending on what states they are in, as well as on their types and storage classes
  • out ( ), , .
  • ref , , .
  • , .
  • - ()
  • , , .
  • , ,
  • , ,
  • ,


As a prototype, there are many aspects that have not yet been considered, and will not be until the prototype shows that this is a workable solution.

Mistakes

Expect many mistakes. Please report them to bugzilla and tag with the keyword “ob”. There is no need to report other restrictions that are already listed here.

References to classes and associative arrays are not tracked.
It is assumed that they are controlled by the GC.

Borrowing and reading from non-Owners

Owners are monitored for leaks, but not other indicators. Borrowers are considered Owners if they are not initialized with a pointer.

@live void uhoh()
{
    scope p = malloc();  // p is considered an Owner
    scope const pc = malloc(); // pc is not considered an Owner
} // dangling pointer pc is not detected on exit

It would seem that it makes no sense to mark pointers such as scope , perhaps this can be fixed by treating this as an error.

Pointers are read / written by nested functions.
Not tracked.

@live void ohno()
{
    auto p = malloc();

    void sneaky() { free(p); }

    sneaky();
    free(p);  // double free not detected
}

Exceptions

Analysis assumes that exceptions are not thrown.

@live void leaky()
{
    auto p = malloc();
    pitcher();  // throws exception, p leaks
    free(p);
}

One solution is to use scope (exit) :

@live void waterTight()
{
    auto p = malloc();
    scope(exit) free(p);
    pitcher();
}

either use RAII objects or only call nothrow functions.

Lazy parameters

Not taken into account.

Quadratic complexity The

analysis demonstrates quadratic complexity, try to keep @ live functions small.

Mixing different memory groups

Combining different memory pools:

void* xmalloc(size_t);
void xfree(void*);

void* ymalloc(size_t);
void yfree(void*);

auto p = xmalloc(20);
yfree(p);  // should call xfree() instead

not detected.

This can be circumvented by using pools specific to a particular type:

U* umalloc();
void ufree(U*);

V* vmalloc();
void vfree(V*);

auto p = umalloc();
vfree(p);  // type mismatch

and possibly disallow implicit conversions to void * in @ live functions .

Arguments of a variadic function

Arguments to a variadic function (for example, printf) are considered consumed. While this is safe, it does not seem to be very practical and will likely require a review.

Runtime changes


  1. TypeInfo_Class/TypeInfo_Interface.isBaseOf C#/Java isAssignableFrom.

    TypeInfo_Class.isBaseOf true, , , , . isBaseOf, isAssignableFrom, , opAssign D — . TypeInfo_Interface.isBaseOf , TypeInfo_Class, TypeInfo_Interface.
  2. core.memory.pageSize minimumPageSize.

    pageSize .

    import core.memory : pageSize;
ubyte[] buffer = new ubyte[pageSize];

    minimumPageSize .

    , . , . pageSize, .

    , , , : ubyte[minimumPageSize].

    import core.memory : minimumPageSize;
ubyte[minimumPageSize] buffer;



  1. Date.isoWeekYear Date.fromISOWeek std.datetime.date

    ISO 8601 , Date and DateTime.

    ISO , . Date.fromISOWeek(2020, 11, DayOfWeek.mon) Date(2020, 3, 9).

    ISO , () .isoWeekYear Date ISO. , .isoWeekAndYear , .
  2. std.xml .

    std.xml . , , UndeaD. xml dub- dxml.
  3. std.digest.digest .

    2.076.1 . std.digest .

    std.digest.digest 2.101.

Dub


  1. .

    Posix , .dub. dub (, .swap.file.d), . , dub . , dub.

    , .
  2. Dub lint --report-file.

    Dub lint --report-file: dub lint --report-file report.json

More articles:


All Articles