Observe your users' perceptions and use them for further development

Many people know the story of The Blind Men and the Elephant. The blind men examine different components of the elephant (symbol of reality and truth) and the descriptions turn out differently. It depends from which angle you look at the elephant or the so-called unknown and with which level of detail you can describe or evaluate your observation. In practice, digital applications and user behavior are ideally observed, evaluated and further developed together, and depending on where there can be improvements in the interactions, new solutions are developed, which are in turn tested with real users. How users understand and perceive an application should play a central role in product development (Human Centered Design). In science, behavior can be studied quantitatively and qualitatively. Which form of user study can be considered for improving an application depends on what kind of questions are to be clarified and what kind of problems are to be solved.


Quantitative user studies

Quantitative user studies target a range of more than 100 to 2,000 test persons, who are randomly tested with the help of a standardized questionnaire. For this purpose, well-known companies often commission market research companies to create, conduct and evaluate surveys and compile improvements for the client. These surveys can be sent out classically by mail as a questionnaire to selected households or offered digitally to a larger audience via an online questionnaire. The goal of data collection should be to clarify more general focal points or attitudes. The mass survey makes sense especially at the beginning of product development to be able to create an overview of the needs of the potential target group, based on which the basic requirements can be developed. In the case of large companies with a broad product range, customer attitudes and desires can provide a relatively meaningful picture of current development, on the basis of which strategic decisions about existence and further development can be made. In app and software development, quantitative surveys might be less meaningful because digital applications need to be generally understood by most people, yet the needs are usually too focused on a specific group of users, so it might be difficult to target suitable users in the high bandwidth. On the Internet, adblockers prevent target group-specific online surveys on websites and very few users voluntarily take the time to go through a questionnaire on their own, which is why a survey usually has to be communicated via several channels, as well as via one or more campaigns. The results are usually very vague and often more precise questions, also in connection with the personal background of the respondents, remain unanswered, which is why this form of user study is more suitable for stating trends (statistics).

This form of user study allows detailed observation with a selected user group of 10 to 20 subjects. With the help of individual interviews and / or group discussions (workshops) with a thematic focus and, if possible, open questions, acute or potential application problems can be solved in a sustainable manner. Qualitative user studies require more time in preparation, since test persons and their user behavior are considered individually, or several application scenarios can be tested here. In most cases, a trained psychologist is the interview partner in order to be able to conduct an interview as unbiased as possible. There are agencies that offer the procurement of suitable user profiles and organize the meeting for the test. The implementation of a qualitative user study can take place in a laboratory or an observation room. During the observation, various methods can be used, e.g. eye-tracking hardware such as webcam or wearable device can be used to track how the subject's eyes perceive the application. The recordings can be used to determine where attention was particularly high during use and which areas are not perceived at all or do not attract the user's attention. It can also be measured how long the user needs to complete an application. If the user needs too much time, then something seems to be wrong with the application pattern, the user seems to think too long about an interaction. Ideally, a user should think as little as possible about decisions in the execution of interactions. The next steps should be quick and straightforward to execute. Of course, the speed of the system with the loading times also plays a role in this. A qualitative user test reveals problems that had previously gone unnoticed. As a product owner, detailed weak points can suddenly be identified, which, without a user test, may sooner or later lead to users not using the application or developing detours that they find annoying or particularly time-consuming. Such negative experiences in operation are often difficult to make up for. In the worst case, the disappointed user will turn his back on the application and look for an alternative, usually without communicating his motives. The latter can be avoided by checking the user guidance with real users at an early stage.

Guerilla study

This form of user study requires a comparatively small amount of time in a rather spontaneous environment such as a café with five to ten test subjects. A guerrilla study should nevertheless be well prepared, because usable or rather evaluable results should be compiled quickly within 10 to 15 minutes, which represents a particular challenge in the study. Startup companies often make use of the guerrilla method. During the lunch break, for example, passers-by on the street or table neighbors in the cafeteria are approached with a paper prototype or click dummy to see if they would be interested in contributing to a vision of a product or a special function in a digital application. In most cases, people are flattered to be asked for their opinion or to continue to be asked open-ended questions that invite them to improve or redevelop solution paths. The small number of test persons can provide relatively early clarity about existing weak points in the considerations or in the product. In most cases, the prototype does not have to represent a great deal of complexity. The informal environment motivates the interlocutors to spontaneously explain areas in more detail. Guerilla can deliver quick results, but the interview partner should in any case have a charming and self-confident demeanor. Insecurities or nervousness on the part of the interviewer have a negative effect on a guerrilla test, since the test user will be distracted from the actual goal or will be more averse to the interview. Similar to the qualitative user study, knowledge of human nature and empathy are important in this type of interview. Knowledge of human nature for the reason that the interviewer has to recognize in a short time who would be open enough to face a user test and empathy to be able to interpret the answers and reactions of the selected person or to respond to the behavior.

User tests, whether one-time or at regular intervals, have the advantage of testing one's own assumptions with real users for their applicability and general usefulness. Priorities can be set with the help of user surveys. This avoids blindly transferring one's own approach to decision-making to one's users and, secondly, stops bad investments in the development of functions. The approaches presented in this article can be used separately or in combination to review acceptance criteria and prioritization in feature planning. Visions can usually be tested inexpensively and quickly with simple prototypes. However, products that have already been developed need more preparation for a user study; the interactive prototypes must communicate application experiences as clearly as possible.  The better a study is prepared and the goals for data collection are clearly defined, the more meaningful the results will be and the more effective concrete measures can be implemented in the long term after the test has been evaluated. A comparison of different views (real user tests) and technological intentions (digital problem solutions or innovations) can not only sustainably improve a product or service, but also the associated customer relationships.