👩🏾‍🎤 🧟 🕶️ Personalized interface. Part 2. Automatic navigation settings 👶 🤚🏿 👩🏻‍✈️

The automatic configuration of interfaces depending on the nature of the user’s work is a difficult task, but it seems feasible. Today I will tell you why our Android team began developing a personalized interface, what data we obtained during the experiments, what conclusions we made and how this knowledge influenced the further development of the entire product. If you missed the preface, then the first part is about the advantages and disadvantages of the idea.

Step 0. Understand what we want.

Our product, Wrike, is actively growing and entering new markets. I understand that soon the mobile application will be completely unacceptably overloaded with functionality.

It is necessary to remove all unnecessary and simplify it, and it is worth starting from two things:

On-the-go user experience features. These are cases when Wrike is fully used outside the office. For example, in the field of event management, when the main phase of the project lasts several hours: they bring food, bring props, install equipment, etc.
During ad-hoc work with the mobile application, the user is out of focus, so do not bother him with unnecessary irritants.

If the task was to simplify the application interface, then the main metric is the time until the first significant action. This metric shows how quickly the user can understand what is happening on the screen from the moment the application is launched to some meaningful action: scrolling through the tape, editing the task description or moving to the desired project.

We started with a common component that every user encounters regardless of the style of his work - navigation in the product. Almost every action in the service is logged, so we went into analytics to see which sections of the application are clicked more often and which ones less often. Tracking navigation between sections only was not enough, in any case, for part of the functionality. Some parts of the product do not carry value, if you just look at them, you need to start interacting with them. For example, the “Inbox” tab contains important messages for the user, and he just needs to scroll through the ribbon, while going to the “Favorites” tab means navigating to your favorite items. In other words, the “Favorites” section itself often does not carry value, but quick navigation from it is a significant action.

Step 1. Data collection

We have enough users of the web version of the product, so there were no problems with statistical significance. Things were more complicated with applications. The activity in them is significantly less, as well as the size of the audience. The reason is that Wrike is more often bought by companies in which employees spend most of their time in the office at the computer. For example, one of the most popular segments among our users is marketing agencies. These guys run advertising campaigns, prepare content - text and visual - and execute approvals.

First, we collected information about how different users work in the web version of the product: how often they open Calendars, work with Inbox, check for changes in the Taskbar, fill out Requests, etc. Next, we collected statistics on the work of these same people in a mobile application. The results showed that customers use almost the same sections of the service in the web version and in the application. Therefore, the hypothesis “User experience on-the-go is significantly different from the web version” has not been confirmed. However, the difference in functionality, albeit insignificant, gave rise to a new hypothesis: users do not work with some tools in the application because they are not aware of their existence.

Ultimately, we determined the meaningful action for each section, and also introduced coefficients. Firstly, data collected from mobile devices had more weight than data from the web version. Secondly, we noticed a pattern - often tools are actively used in pairs. Let's say 70% of the audience use tool A and B at the same time. So the active use of one of the tools inevitably increases the likelihood of a second in the lower navigation.

Step 2. Development

I’m a product manager in the Android team, so the development was done precisely within the framework of this application. We did not plan to make the entire application adaptive the first time. To begin with, the task was to check whether the setting of navigation categories is valuable.

The mobile platform worked well for two reasons:

, . , . satellite-. Wrike for Android .
.

The functionality itself is quite simple: there is a component in the interface of the mobile application - bottom navigation (Bottom navigation). It is needed in order to leave sections frequently visited by the user for quick access.

There is also a tab “Else” - it contains everything that did not fit in the bottom navigation.

The idea of the experiment was to remove all superfluous even from the bottom navigation and not to disturb the already defocused user. Functionality included automatic interface configuration with the ability to change navigation manually. It happened like this: when starting the application, we requested the navigation configuration from the server. Almost immediately after that, an animation started, in which it was slowly shown and described in detail about how and why the interface was changing:

We did not offer the user to try a new navigation, but posed a fact, broke the human habit. If he didn’t like something, he had to go to the settings himself and reset it or change it to a configuration convenient for him.

Step 3. Launch within the company

It was important to get detailed feedback from users who are familiar with the product. We had data on almost all users in all accounts, but did not want to take risks. The first launch was inside the company. The entire Android team participated in it, as well as five representatives among very active, active, weakly active and inactive users.

This was enough to collect the first feedback that showed obvious design errors. And also make sure that:

, ;
( ) .

We selected candidates for testing, starting from the number of sessions and the number of days of visiting the application for the last month. Quite by chance, the CEO Wrike was in the sample, which made me think. Any change in the work of the C-Level person may affect the processes and productivity of the company. So we had the opportunity to learn how quickly the CEO can adapt to changes in navigation. Although, of course, we understood that one person is not an indicator. Also, the sample included employees from different countries who occupy different positions in the company. We patched the database, and all users who had the latest version of the application automatically received the interface settings the next time they started.

For the test group, a description was prepared in advance with the details of the experiment, as well as questions that we expected to receive answers to. Some time after the start of the experiment, we sent this description to each member of the test group.

To summarize all the feedback - users were delighted that the interface was tailored to their work habits. The main thing in the new interface was not just convenience, but that the application began to reflect the nature of the user's work. Also, the participants in the test group liked the opportunity to manually change the interface. By the way, changes after manual adjustment made adjustments to the analytical model.

I believe that any configuration process, whether it be navigation or the appearance of a game character, is what the user wants to do, not should. And any investment of this kind often increases the value of the product for the user.

We have performed this procedure several times to make sure that the navigation predictions are correct and the feedback is positive. This was not superfluous: when testing on our account, we found a couple of bugs.

After we made sure that everything was in order, it was necessary to scale the solution.

Step 4. A / B to 100% of the audience

Of course, we can’t talk about statistical significance within one account. To begin with, we did A / A for the entire audience of registered users and made sure that there was no difference in the groups. Then we went to the A / B stage and asked the engineering analysts department to patch the base for each user from group B.

The following hypotheses were tested:

Better interface configuration allows you to do work faster.
Interface customization has a positive effect on retention.
Users become more loyal to the application if they can customize the interface.

Consider what was hidden behind each hypothesis and how we confirmed them.

About the speed of work

We do a work-management service, so our goal is to increase user productivity. I would say that productivity is the “amount of benefit” that a person brings in the workplace. The benefit can be expressed in different aspects, but let's not forget that our service is about work, but often not the work itself: designers draw banners and websites, developers write code, the security department does something safer. Their work is concentrated in other tools, and Wrike provides transparency of processes and allows everyone to move in one direction. Therefore, we do not have a goal to increase the length of the session, however, we are pleased to generate new content and frequent visits to the system. Our task is to make the work with the application so fast that it is not difficult for a person to switch the context to working communication and then return to their business again.

Within two weeks after the launch, we observed a drawdown in our main metric (columns above 0.0):

A few words about the graph:

Line 0.0 means group A - those users who have not had any changes in the interface. Everything above 0.0 shows an increase in time to a significant action, everything below - on the contrary, a decrease in time. Columns indicate the percentile from the fifth to ninety-fifth, respectively.

In these 2 weeks, we increased the time to the first significant action, however, there is a trend that the graph is gradually "moving" to the right. We hypothesized that a forced interface change broke user habits, and it took them some time to get used to the new changes in navigation.

Retention metrics did not sag significantly, so we decided to wait a while, and we did it for good reason. After the eighth week of the experiment, almost all users got used to the new navigation, which increased the speed of their work with the application:

In the end, we were able to reduce the time to the first significant action by about 18%.

About retention

Our faith was built on a pleasant experience with the application. We suggested that a more convenient tool would stimulate the return of the client to the system. I must say right away, retention has grown a little, but it can not be called statistically significant, so this hypothesis has not been confirmed.

Working with mobile applications, which are satellite products in relation to the main service, I came to the following conclusion: no matter how convenient any satellite is, it will always be inferior to the main version of the product. For example, if I am away from the computer for 2 hours out of 6 and I need a connection with the team, then I will use the Wrike application for these 2 hours. The rest of the time it’s more convenient to work with a large monitor and a full keyboard. In our case, this is due to the fact that it is easier to focus on work tasks at the computer.

About loyalty

A simple configuration of understandable entities was to increase a person’s connection with the application. Because when a client receives value not only from content, but also feels care, his workspace becomes special. We got a lot of positive feedback: users like to see an interface that matches their working style. So, users from group A told us that they also want such functionality, because "it looks comfortable and cool."

Step 5. B / C to Group A

Group B was found to be more successful. But we were in no hurry to complete the experiment, because with approach B we still had one expensive task in terms of support. The entire analysis of user behavior was based on the UI, which was about to change due to a general redesign of navigation. The whole company had a lot of work to do, and the script that made up the navigation for each user could be left without support for a long time.

Therefore, we divided group A into two parts: group B received the experiment described above, and group C received onboarding with a proposal to configure the configuration independently. But users rarely customize something, so only 10% of group C followed our suggestion and changed something in the navigation. Customers open the application to do what they need, and not what the service providers need. It was useful to see that those who set up the interface on their own, from the first week, increased their speed to the first significant action (like group B after the eighth). Group C showed good performance in a short time, but because of poor onboarding, engagement was low.

Step 6. Configuring navigation the first time you log in

One UX designer once told me: “Customers are divided into two types: there are those who read updates, and those who do not read. There is nothing to do with onboarding. ” Exceptions can be found in this phrase: to force the user to do something or pay him anything, but in general I agree with his statement. We understood that for a long time we would not be able to maintain the service on the current analytical model, so it was decided to do better onboarding.

We invited users to participate in the survey. The design of the survey was to be clean and clear, and the mechanics of interaction - primitive. Balancing between the development cost and the benefit for the client, we came to the following decision:

In an informal appeal, we ask the user what he uses most often and inform that we will customize the application to his needs. Of course, this looks less impressive than an animated navigation change with a loading indicator, saying that we do some magic on the server there. And it requires much more action from the user. However, through such a system, we were able to achieve a good conversion to the settings of the navigation panel.

Total

We tried and were not mistaken. In mobile applications, all customers are already using the solution. The next step is to move the web version along a similar path. Automation brought us greater success, but maintaining it was expensive, so onboarding in the form of a survey turned out to be a pretty good solution.

As I wrote in the first part, personalized interfaces are, rather, a development vector that can be beneficial from different perspectives. I don’t think that there is any one right way, but I’m sure that experiments in this direction plus automation of the setup process will give new knowledge about the industry and make complex products easier and make users happier.

Afterword

Now the project is on pause due to active work on navigation in the web version of the product. We believe that innovations can significantly affect user experience. Nevertheless, there are ideas what the next step could be - this is an automatic interface setup for those who register with mobile devices and see Wrike for the first time. Such users are not ready to perceive the entire system at a time, and therefore it is not worth dumping it on them so as not to scare them off.

For users who are already familiar with the web version of the product, the next step will be to move towards the organization of content. We know which “Requests” users use more often than others, with which “Reports” they interact regularly, which lists are viewed more often on the “Task Bar”. Ultimately, I want to do something like recent or recommended, which is often used now in B2C products.

That’s all for me. I hope this article was useful and will help apply knowledge in large and complex products.

Personalized interface. Part 2. Automatic navigation settings