Get best of the week

UX-research of RBS: our experience, mistakes and discoveries

Hey. I am Denis Elianovsky, Design Director at JTC and Lead at Opium Pro. We work in very narrow segments of the IT market related to finance and workflow. You certainly have not heard about these companies and today you will not know much about them, because this article is about UX research.

I have been designing for the past 12 years (8 of them are precisely the design of complex interfaces) and I want to tell how we conducted usability testing of the RBS application. And also about what mistakes were made and which made conclusions from this. In the process, I recommend a couple of books. All pictures are clickable.

What is RBS?It stands for Remote Banking Service. These applications are also called online banks. Surely all of you already have this on your phone. Here we are one of those who design and develop such applications.

You will probably find something useful in the article if you have already heard about UX research and would like to test your application, but do not know where to start. If you already have experience in tests, then it will be boring and uninteresting, forgive me.

Who is more convenient to watch the video, here is a recording of the speech





What are the tests?



If you ask to put a description of UX research into 23 words using three commas, one dash and one hyphen, then UX research is a quick useful action that can be done at the early stages of development, and thereby avoid mistakes that would come out in production. But the release of a banking application in production is not only a huge stress, but also a huge waste of money. UX-research allows you to test your application on real users long before you start unloading KAMAZ with money into the pockets of developers and marketers.

Of course, every designer (every good designer) tests his application before release. But usually this is done "at home." That is, it is tested on their near and dear ones, and very rarely it is somehow systematized. In this article I want to convey that in large and complex applications design testing should be regulated and integrated into the design process itself.

First, research can be quantitative and qualitative. When we talk about quantitative research, we mean audience coverage: we strive to increase the number of people we test. When we talk about quality research, we test few people, but with each of them we conduct a more detailed interview and dive deeper into each specific case.

Coordinate plane. Right - Quantitative. Left - Quality

The same coordinate plane. Above is the behavior. Bottom - Attitude


More research can be divided into behavioral and relational. When we conduct behavioral testing, we monitor what a person does with our application. In relational it is important that the person speaks about our application.

In this article, we will focus on Usability tests. They relate to behavioral quality tests. In this case, a small number of people are involved in the testing and basically what they are doing with the application is carefully studied.


The same coordinate plane. Top left circle mark


The design process is well reflected in Damien Newman's schedule. And since I said above that design and testing should be a single whole, the graph reflects both the process of creating a design and its testing. Two conclusions can be drawn from this graph. 1) Design and testing is a non-linear process. 2) And it is also an iterative process. This is obvious at first glance at the chart, isn't it? What do I mean by nonlinearity?

A tangled tangle that unravels in a flat line




Initially, we have so many different theories that we want to test through prototypes. And only towards the very end of the design process do they somehow settle down. From time to time, from iteration to iteration, testing sessions become more similar to each other and changes in design become less and less: they are deeper, more elaborate, but outwardly it is more difficult to notice them.

By calling the process iterative, I mean that you can’t test it once and stop there. Testing should be carried out regularly. It is then that it makes sense. Especially considering that the design is also changing dramatically in the process.


How to prepare for a UX study?



Fingered hands (one, two, three)


  1. Usability- . ? , . . , , ( ), User Story Map. User Story Map .
  2. . , . , . , 10–15 , 20 ( : , , ). , 5–7 -. , . , «» «», -, .
  3. . (5–7 ). , . -. , « » .

    I will immediately give an answer to the most popular question of our customers, which sounds like this: "You probably test everything on your own to ensure that the results are good?" No. Not only do we not include our results in the tests, we try not to even include people with professional deformations in the tests, i.e. those who are professionally engaged in tests for money. And also we exclude designers, programmers and others who are also subject to this professional deformation.



Let's repeat what we need for preparation to consolidate.

1. Hypothesis


The picture below shows several possible options for visualizing this process. The leftmost screen is how you can make the User Story Map by improvised means. To do this, just use paper stickers. By connecting them with strings, we show assumptions about how the user will walk around the application. The second option is the User Story Map, drawn in Miro. In fact, the same pieces of paper, only transferred to electronic form for convenience. And the third screen is a clickable prototype in Figma.

Three rectangular images








The above specific tools are indicated, but in fact there is no binding to them, and hypotheses can be built in what is convenient for you. For example, in our team there are enthusiasts who conducted all testing on pieces of paper. They also had a clickable prototype on pieces of paper - and this too can be begun.


2. Drawing up questions



Open questions. Great if they will be in the form of stories. That is, if we assume that a person must block the card to solve his problem, we don’t say “Block the card” in our forehead , we tell him a story. Ideally, history should immerse a person in himself and provoke him to respond with history too.

Example:

On the left is an example of a bad question. On the right is an example of good



3. Gathering respondents



This schedule was compiled by Jacob Nielsen back in the 1990s. Even then, UX studies were conducted. The horizontal line shows the number of people we are testing, and the vertical line shows the number of errors found. It is worth noting that after about the 5th person being tested, the schedule becomes more gentle, but what does this mean? This means that after 5 people the efficiency of switching on each new test person falls, and with it we find less and less errors. Jacob Nielsen made such a conclusion, and we fully share this conclusion.

Another graph on the coordinate plane. First, sharp growth, then smooth growth inhibition




It is more efficient to make small samples, but to do it often.

At first I wanted to advise Jacob's book, but in general it is already old. There is something better. The author continues to engage in UX research, and here is his website, there are many articles: nngroup.com


Testing process



Text: The process is recorded on video; Voicing thoughts; 5 why; Have you spawned


Damn it is important to record all the tests on video. Video is the most important artifact of tests. If you do not have a video, then we can say that you did not test.

The first one. Before starting the tests, since we are testing a mobile application, we give the phone to a person. We ask him to relax and be sure to voice everything that is happening. It is important for us to understand the subject’s train of thought.

Secondly, and we call this rule “5 why?”, in the process of passing the script, you need to provoke a person to explain any of his stop or doubt in his actions. Here such questions help a lot: “What did you expect to see on this screen?”, “Why did you click on this button?”. This is not always exactly 5 why, but the desire to ask as many questions as possible and to plunge into the person’s head as much as possible always helps.

And the third. Based on the test results, we ask, “Do you think you have completed the task? ". Moreover, not only the respondent answers this question, but also the person who is testing. Why we do this, I will tell below.

Now we pass directly to our tests.The table shows what the summary results of the research might look like. These are the real results of our first test. On the left is a list of questions and stories, followed by columns for each of the respondents. If it costs 1, it means that both the test subject and the tester considered the task completed. If 0.5 - it means that someone alone believes that the task is not completed. If 0 - both agree that the task is not completed. From this data, we can understand which scenarios we have are strong, which are weak. From the data in the table, we can conclude that, for example, we are doing well with locking the card, everyone coped. And with the transfer of money - not very good, this is what we should work on.


Table. The first column is the text of the questions. The remaining columns are numbers






We tested our mobile RBS application for individuals. In total, at the time of the creation of this report, two iterations of tests passed, in each of which 6 people were tested. Only 7 women, 5 men aged 20 to 50 years. Initially, we did not try to make a very wide selection of professions, but it turned out to be quite diverse: teachers, doctors, restaurant administrators, etc. Due to the request of our client, in the second session more people aged 40+ were included in the sample. And it was in this audience that the most errors were identified. Most often they stumbled somewhere, stopped somewhere, and they had to ask the most questions.

The above is written about the audience parameters







Test results



The results of the test from the point of view of "coped / failed": It turned out that the tested coped with 93% of the tasks. Moreover, they themselves believed that they had managed only 83%. This difference of 10% - those moments when a person went according to the desired scenario, our tester sees that he coped with the task, but the respondent is not sure about the results. And these are also problems that need to be worked on. After all, we understand that in such moments the application does not give the person the desired feedback and this needs to be fixed. On average, one session took 12 minutes. A good result, taking into account the fact that we focused on 10-15 minutes. Below is the design of the application that was used in the first iteration of the tests. Let me explain what we decided to change in it according to the test results.

Paychart, divided into 3 parts. 83%, 93% and the empty white part






Three mobile phones with an open application. The first screen is highlighted.


I will tell on behalf of the user who poked a finger into this application.

Suppose I need to pay for a mobile phone. Probably, there should be a “pay” button somewhere. I don’t find the button and am leaving to look for it in the burger menu in the upper left corner.

What is the problem? There are two “pay” buttons on the screen, no one has noticed any of them. This was in 3 cases out of 6.

Another problem. The analytics section, which we found very useful, unfortunately, users did not find as such. He only confused people.

In general, if you look globally, then this screen is overloaded, it is difficult for the user to sort through in their heads what and where it is located on it.

The second screen is the payment / transfer screen: During testing, we found that people are interested in seeing their regular payments, and they are looking for them on the payment screen. In the first version of the design, they were on the edge and partially hid by a horizontal scroll. Well, in principle, they were in a separate tab, which also made it difficult to find them. The third screen is a screen with bank products:

Three mobile phones with an open application. Highlighted second screen






Three mobile phones with an open application. The third screen is highlighted.


All the people we tested (ok, almost all) noted that this screen is useful. They knew where to call him, they often used him. Here we created a problem for ourselves by placing a call to this screen in the upper left burger menu. In the video, we noticed that when a person tries to press this button, many intercept the phone, and this is not convenient for everyone.

Now the phones have become large, they may fall out, and we decided that this is a problem for us, we will work with it. By the way, guess why the author of the article walks with a broken phone?

The following pictures show the significantly changed design of the second iteration.

Three new mobile phones with an open application. The first screen is highlighted.


As you can see, the screen has become much easier. Now, when we asked users to tell us what they see, they told us this in much the same way as we ourselves imagined. The analytics section was removed under an additional button, it now does not load space. On the payment screen, regular payments are made in such a way that they are noticeable. But people are still stuck here, which implies that we will definitely improve and facilitate it. The third screen is a screen with a list of bank products that a person has. And here we made access to it from the bottom of the phone instead of calling it from the upper left burger menu.

The same three new phones. Highlighted second screen






The same three new phones. The third screen is highlighted.


Now this object is located directly under the user's finger, and he does not need to reach for it, just swipe up or click on it to open the list of products.

Some more observations and conclusions that we made during testing. Who would know that there are left-handed people and people in the world who are simply used to holding the phone in their left hand. And they have their own characteristics of use. If an ordinary person intercepts the phone, then the lefthander does not intercept to press the burger menu button. We took note that lefties just use devices differently, and we will continue to test and find out if we could improve the experience for them. There are still people with poor eyesight.

Bar graph. 3 bars: 10%, 13%, 50%




Everyone knows about it, and everyone forgets about it (I mean designers). How can I help people with low vision? Firstly, you can increase the objects in the design. Secondly, you can increase the contrast in the design. And there is another less obvious point: you can separate information, increase the distance between blocks, which will also help people to read interfaces better.

According to the most conservative estimates that can be found on Wikipedia, we have 10% left-handed people, and 13% people with low vision. According to the immodest - left-handed people are somewhere around 15%, and people with low vision - 30%.

And some girls have long nails.These girls also use the phone differently. It’s hard for them to press something in the lower right corner, because instead of pressing with a fingertip they rest with a fingernail, and therefore they also have to somehow intercept. There are no official statistics here, but I can assume that from time to time up to 50% of the adult population of the planet may be in this situation.

In addition to the Usability tests highlighted above, there are many different ways to test UX. Among them:

  • Eyetracking
  • A / B tests
  • Online polls
  • In-depth interviews


On the very first coordinate plane there are circles with the above-mentioned test methods


The longer we spend testing, the more clearly we understand that UX testing is very close to such a science as cognitive psychology. And especially to such a concept as “cognitive distortion”.

For those who want to dig into this, I advise Daniel Caniman's book “Thinking, Fast and Slow” (Think slowly, decide quickly). She will not tell you anything about the tests, but it will provide good ground for reflection, showing how the same people can answer different questions in the same way.

Did this article help you get closer to starting testing the interfaces yourself? What turned out to be useful (if something still turned out to be), and what is inappropriate?

Thanks to everyone who helped conduct the study and prepare the report.


UX research:
JTC team - business analysis, design, UX research
Denis Krasilnikov - design
Anton Kazakov - UX research
Ekaterina Kashkovskaya - UX research
Dmitry Dobrodeyev - UX research
Irina Ponomareva - filming and editing

Preparation of the report:
JTC team - report generation
Maxim Blokhin - design by
Irina Ponomareva - filming and editing
Nadezhda Molodtsova - filming
Tatyana Kitaeva - editing

More articles:


All Articles