Get best of the week

Porting APIs to TypeScript as a Problem Solver

The React frontend of the Execute Program has been converted from JavaScript to TypeScript. But the backend, written in Ruby, did not touch. However, the problems associated with this backend made the project developers think about switching from Ruby to TypeScript. The translation of the material that we are publishing today is devoted to the story about porting the Execute Program backend from Ruby to TypeScript, and what problems this helped solve.



Using the Ruby backend, we sometimes forget that some API property stores an array of strings, not a simple string. Sometimes we changed an API fragment that was accessed in different places, but forgot to update the code in one of these places. These are the usual problems of a dynamic language that are characteristic of any system whose code is not 100% covered by tests. (This, although less common, happens when the code is fully covered by tests.)

At the same time, these problems have disappeared from the frontend since we switched it to TypeScript. I have more experience in server programming than in client, but, despite this, I made more mistakes when working with the backend, and not with the frontend. All this indicated that the backend should also be converted to TypeScript.

I ported the backend from Ruby to TypeScript in March 2019 in about 2 weeks. And everything worked as it should! We deployed a new code in production on April 14, 2019. It was a beta version available to a limited number of users. After that, nothing broke. Users did not even notice anything. Here is a graph illustrating the state of our codebase before and immediately after the transition. The x-axis represents the time (in days), the y-axis represents the number of lines of code.


Translating the frontend from JavaScript to TypeScript, and translating the backend from Ruby to TypeScript

During the porting process, I wrote a large amount of auxiliary code. So, we have our own tool for running tests with a volume of 200 lines. We have a 120-line library for working with the database, as well as a larger routing library for the API, linking the front-end and back-end code.

In our own infrastructure, the most interesting thing to talk about is the router. It is a wrapper for Express, ensuring the correct application of the types that are used in both client and server code. This means that when one part of the API changes, the other does not even compile without making changes to it to eliminate the differences.

Here is a backend handler that returns a list of blog posts. This is one of the simplest similar code fragments in the system:

router.handleGet(api.blog, async () => {
  return {
    posts: blog.posts,
  }
})

If we change the key name poststo blogPosts, we get a compilation error, the text of which is shown below (here, for brevity, information about the types of objects is omitted.)

Property 'posts' is missing in type '...' but required in type '...'.

Each endpoint is defined by a view object api.someNameHere. This object is shared by the client and server. Note that types are not directly mentioned in the handler declaration. They are all inferred from the argument api.blog.

This approach works for simple endpoints, such as the endpoint described above blog. But it is suitable for more complex endpoints. For example, an endpoint API for working with lessons has a deeply nested key of a logical type .lesson.steps[index].isInteractive. Thanks to all this, it is now impossible to make the following mistakes:

  • If we try to access isinteractivethe client, or try to return such a key from the server, the code will not compile. The key name should look like isInteractive, with a capital I.
  • isInteractive — .
  • isInteractive number, , , .
  • API, , isInteractive — , , , , , , , .

Note that all this includes code generation. This is done using io-ts and a couple of hundred lines of code from our own router.

Declaring API types requires additional work, but the work is simple. When changing the structure of the API, we need to know how the structure of the code changes. We make changes to the API declarations, and then the compiler points us to all the places where the code needs to be fixed.

It is difficult to appreciate the importance of these mechanisms until you use them for a while. We can move large objects from one place in the API to another, rename the keys, we can split large objects into parts, merge small objects into one object, split or merge whole endpoints. And we can do all this without worrying about the fact that we forgot to make the appropriate changes to the client or server code.

Here is a real example. I recently spent about 20 hours on four days off redesigning the API Execute Program . The whole structure of the API has changed. When comparing the new client and server code with the old, tens of thousands of line changes were recorded. I redesigned the server-side routing code (like the abovehandleGet) I rewrote all type declarations for the API, making many of them huge structural changes. And, in addition, I rewrote all parts of the client in which the changed APIs were called. During this work, 246 of the 292 source files were changed.

In most of this work, I relied only on a type system. In the last hour of this 20-hour case, I started running tests, which, for the most part, ended successfully. At the very end, we made a full run of tests and found three small errors.

These were all logical errors: conditions that accidentally led the program to the wrong place. Typically, a type system does not help in finding such errors. It took several minutes to fix these errors. This redesigned API was deployed a few months ago. When you read something onour site - it is this API that issues relevant materials.

This does not mean that the static type system guarantees that the code will always be correct. This system does not allow to do without tests. But it greatly simplifies refactoring.

I’ll tell you about automatic code generation. Namely, we use schemats to generate type definitions from the structure of our database. The system connects to the Postgres database, analyzes the column types and writes the corresponding TypeScript type definitions to the regular file .d.tsused by the application.

A file with database schema types is kept up to date by our migration script every time it is launched. Due to this, we do not have to manually support these types. Models use definitions of database types to ensure that application code correctly accesses everything stored in the database. There are no missing tables, no missing columns, or entries nullin non-supporting columns null. We remember to process correctly nullin columns supporting null. And all this is statically checked at compile time.

All this together creates a reliable statically typed chain of information transfer, extending from the database to the properties of the React components in the frontend:

  • , ( API) , .
  • API , API, ( ) .
  • React- , API, .

While working on this material, I could not recall a single case of inconsistency in the code associated with the API that passed the compilation. We did not have production failures that arose because the client and server code related to the API had different ideas about the data form. And all this is not the result of automated testing. We, for the API itself, do not write tests.

This puts us in an extremely pleasant position: we can concentrate on the most important parts of the application. I spend very little time doing type conversions. Much less than I spent identifying the causes of confusing errors that penetrated through layers of code written in Ruby or JavaScript, and then caused strange exceptions somewhere very far from the source of the error.

This is how the project looks after translating the backend to TypeScript. As you can see, a lot of code has been written since the transition. We had enough time to evaluate the consequences of the decision.


TypeScript is used on the frontend and backend of the project.

Here we have not raised the usual question for such publications, which is to achieve the same results not through typing, but through the use of tests. Such results cannot be achieved using only tests. We, quite possibly, will talk more about this.

Dear readers! Did you translate projects written in other languages ​​into TypeScript?

More articles:


All Articles