Keywords

1 Introduction

The technological development of mobile devices has brought many advantages to the development of Mobile Augmented Reality (MAR) applications, which search and explore Points of Interest (POI). Smartphones and tablets allow users to carry a range of applications with Internet access, location and movement sensors, and sufficient computational resource for playing multimedia content.

Augmented Reality (AR) can enhance user’s daily lives activities using a hardware that people already own and at various locations, such as at work, at home, or during a trip. The AR uses technological resources aiming to expand the user’s senses in relation to a given task, generally using visual content. An example of task that can be enhanced by AR is to discover and visit nearby sights. When a device extends the user’s sense of direction highlighting POIs with virtual markers over the real sight, augmented reality helped the user to accomplish his task.

The MAR applications context presents challenges, such as: the lack of applications’ developing standards, some applications have limited flexibility withstanding the use in other areas, the demand for improvement in accuracy of location and pattern recognition technologies, shortage of available screen space to display virtual information, and the necessity of testing the acceptance of these new features [1].

This paper aims to present the results of MAR application usability evaluation, identify a list of problems in the application’s Graphical User Interface (GUI), and propose guidelines for building GUIs for MAR applications to avoid the problems found in this work.

The evaluation performed is the Think Aloud Protocol, which was conducted with 20 participants, mainly aged from 18 to 30 years, and with experience on smartphone usage. To analyze the results, the significant verbalization of participants was extracted from evaluation videos. Finally, to propose guidelines all participants’ verbalizations are transformed on usability problems and guidelines are proposed to avoid these problems.

The paper is organized according the following: Sect. 2 cover the background of MAR applications and introduces ARguide, which are the system used on tests, the Sect. 3 explains the usability evaluation used and its results, Sect. 4 presents the proposed guidelines, and Sect. 5 concludes the paper and points future works.

2 Mobile Augmented Reality and ARguide

The MAR is the employment of AR concept in mobile devices, which means increasing user perception about the real world with virtual elements over and synchronized to that real world, visualized through mobile devices [2]. Providing a good user experience is essential in this technology, which implies studies about GUI design are as important as innovative studies in this area [1, 3–5].

Among the main challenges identified in the development of MAR applications, the following can be highlighted [1, 3]:

  • Appropriate use of devices’ sensors;

  • Low accuracy of tracking technologies;

  • A range of hardware and software features and limitations;

  • User interface design variability;

  • Lack of standards adopted by application developers RAM;

  • Energy consumption;

With these challenges in mind, it is important to evaluate MAR applications, so the identification of usability problems allows proposals for improvements and evolutions for more robust and adoptable applications that directly impact the user experience when using MAR applications.

ARguide [6] is the application used in the evaluation test. ARguide is a MAR application that uses technologies such as sensors, camera, GPS, and Internet to provide an intuitive discovery of Points of Interest (POI) through three modes of operation: AR Browser, Map and QR Code Scanner. The main ARguide’s GUI is presented in Fig. 1.

Fig. 1.
figure 1

Screenshot of ARguide Application showing a POI through AR browser. The numbered areas highlight some GUI’s widgets and visual items.

Full size image

Figure 1 (1) shows two tabs used for change the exploration mode between map and AR browser. In (2) shows a virtual marker overlapped on the real POI, indicating the direction of the POI in the real environment. In (3) shows POIs radar and a slider widget to control the distance of interest. The area (4) has two buttons: clean and help. The area (5) has a button that starts the QR Code Scanner.

The user can read QR Codes with codes that is related with applications database. These codes makes possible to select a POI that is very close from others. After reading a QR Code, the application performs a POI selection automatically if the code is related to a POI, or shows a media directly to the user.

The Fig. 2 shows a screenshot of ARguide’s GUI with the map tab selected. In this tab the user can navigate through the map and identify POIs touching their graphic representation (red balloon) according to Fig. 2 (1). Another important function in this application is the route that user can draw building circuits or a path to a particular POI.

The map functionality can be very helpful when the user decide to visit a POI’s sight and want to know how he could reach the POI location. The AR Browser is complementary with map providing directions of POIs.

Fig. 2.
figure 2

Screenshot of ARguide Application showing POIs through Map tab. The numbered area (1) highlights the visual representation of a POI in the map while the area (2) highlights a circuit between POIs.

Full size image

3 Usability Evaluation

This section shows the uability evaluation setup used on ARguide to extract usability problems from the GUI and shows the results of this evalution.

3.1 Evaluation Setup

The usability evaluation chosen is the Think Aloud Protocol [7], due to its good performance in usability factors evaluation [8]. The evaluation was conducted with 20 participants, mainly aged from 18 to 30 years; almost all participants have used the Android operating system before, according to charts in Fig. 3. All participants declared having already used smartphones and their daily frequency of smartphone usage is high.

Fig. 3.
figure 3

Profile of participants in usability evaluation. Chart (a) shows the number of participants who have used the each of operating systems in smartphones and the chart (b) shows the proportion of participants in each the age range.

Full size image

The tasks were performed in a room in the university campus and the POIs refer to locations and buildings of this campus. The tasks were recorded on video and audio focusing on tablet screen used and participants’ hands. Specially, the participants’ speeches were recorded for later analysis of theirs unaware opinions triggered by the application usage.

The evaluation was conducted with six tasks and users had free time to perform them. The six tasks are listed below.

  • Task 1: Select the local of central library to know about it.

  • Task 2: Visualize a media (image or video) of conventions center.

  • Task 3: Trace a route from your position to the University Hospital.

  • Task 4: Find the university president’s building through RA Browser.

  • Task 5: Decrease the interest radius to visualize only nears POIs.

  • Task 6: Scan a QR Code, visualize the provided content and say what it is about.

Before starting tasks, the evaluation conductor explained the application’s functionality and characteristics to participants, and then each participant had 2 min to test the application by himself. After this, the conductor explained the tasks to participants and asked them to verbalize any difficulty, question or suggestion during the test performance.

To perform the tests we used a tablet model Samsung Galaxy Tab III with Dual-Core Processor of 1.5 GHz, 1.5 GB of RAM and 8 inches of screen size. The application was previously installed on this tablet and all participants used the same tablet.

3.2 Evaluation Results and Discussions

In the think aloud protocol evaluation is fundamental an analysis of the recorded video and audios of participants performing the suggested tasks. Hence, the first step into analysis of results is extract significant verbalization of participants. After obtaining all the significant verbalizations, each of these is classified according to the two following criteria: the widget or graphical component that the verbalization is related to and what is the motivation for this.

Table 1 shows the amount of verbalizations correlating the two criteria defined above. The amount of verbalizations in map and in its route functionality shows that usability problem in this function is very noticeable. Table 2 has some of the participants’ verbalizations, whereas the first column has the raw participants’ speeches associated and the second column has the speeches itself.

Table 1. Relationship between two proposed verbalization classifications and amount of verbalizations extracted from experiment performances in both classifications (the widget and motivation).
Full size table
Table 2. Some participants’ speeches obtained from evaluation and the related GUI component
Full size table

The verbalizations concerning the map details balloon are related to the fact that the balloon covers the map, and overlap some markers. Since there was no close button, users got confused. To close that balloon, the user had to click anywhere else on the map, however this interaction was not intuitive to almost all participants.

Markers on map cause 22 verbalizations from participants. Analyzing the recorded videos, it is clear that these verbalizations are often related to the absence of text labels on the markers. The markers are labeled with a number that indicates a suggested visitation sequence. However, the participants did not notice this sequence and did not want this information.

The trace route application’s functionality is the most problematic in tab map according to recorded videos. The verbalizations are usually referring to questions of where and how to trace route. To make a route in the application the only action needed is to select a POI marker and the system traces a route between your position and the clicked marker. Therefore, participants did not realize that the route was already drawn, and verbalized phrases, such: “The route is already done?!”.

Problems encountered in AR Browser are mainly related to the adjustment of radius of interest, and with the arrangement of markers on the screen. Participants did not understand how to operate the moving slider (Figs. 1, 2, 3) and its effect on the application. One possible reason is that the slider only gives feedback on the radar, which is a very small component, turning the up and down movement a non-intuitive process.

The arrangement of the markers hinders is confused when there is more than one marker in the same direction and the algorithm tries to spread them, aiming that the user can view all markers. However, this spreading does not always occur in a satisfactory way and ends up placing markers far above or far below from the horizon.

The verbalizations related to the QR Code Scanner occurred mainly for not providing appropriate feedback to users when they were performing the reading of a QR Code. The system updates the bottom bar with buttons that gives access to media content of the selected POI; however, participants did not notice these updates and therefore, did not notice what have occurred.

The chart in Fig. 4 visually shows the amount of participants’ verbalizations as the height of each bar, the rows from left to right represent each widget or graphical representation related to a verbalization and each color represents the verbalization motivation.

Fig. 4.
figure 4

Chart of the amount of verbalization encountered in Thinking Aloud Protocol usability evaluation

Full size image

Two outliers are clear in the chart: the user’s doubt on tracing a map route and the doubt about the visual representation of the markers on the map. These outliers mean that many participants commented about these problems, therefore representing the two major interface design problems on ARguide’s GUI interface.

Another clear factor in chart is that there was usage doubt for almost all widgets, highlighting the slider of radius of interest. Two components that get minor attention are the AR Browser’s radar (Figs. 1, 2, 3) and the Web Media button. This was expected for Web Media Button, because it is not mentioned explicitly on the tasks and its functionality is insignificant to accomplish the tasks, but the same cannot be said about the AR Browser’s radar. Finally, the graph shows that the participants gave suggestions for improvement during the course of the test for almost all graphical components.

4 Proposed Guidelines

To propose guidelines based on encountered problems in usability evaluation, first all 70 verbalizations were listed from the recorded participants’ performance and were translated to usability problems on ARguide interface. Next, a merge operation has been made with similar problems or with problems that may have the same solution. And after this, the resulting problems were discussed out on a brainstorm with ARguide’s development team and specialists in GUI design and usability.

After the brainstorm reunion, for each suggested solution, it was evaluated what could better satisfy the conditions reported by participants, according the analysis of the captured videos and verbalizations. At least, a summary of the encountered problems and their recommended solutions was proposed, aiming to meet the participants’ evaluation and subsequent users in general.

Table 3 shows in the first column the identified GUI problem and in the second column, the proposed guideline to mitigate the related problem. The GUI issues are in form of a heuristic, such that it can be modified and adapted to a variety of contexts.

Table 3. Found GUI issues and related proposed guidelines
Full size table

The proposed guidelines follow the same style as the GUI issue and are suggestions based on what was observed both in video recorded as in extracted verbalizations of evaluation participants. Notwithstanding, the proposed guidelines are not absolute and can be considered as a suitable approach for designing MAR applications’ GUI.

5 Conclusion and Future Works

This paper presented a MAR application that use georeferenced points to show POIs by AR Browser, map, and Scanning QR Code. Thinking Aloud Protocol usability evaluating was performed to identify possible problems with the application’s GUI, and based on participants’ verbalizations and actions in recorded tests, some guidelines were proposed to mitigate these problems.

The guidelines may be useful for designing MAR applications’ GUI and analog for applications that have innovative interfaces such as multimodal interfaces that are used for operating and navigating through POIs. The guidelines were developed in general terms and are flexible in the context of adjustments and changes that solve the problems. At least, the proposed guidelines can be used as either heuristics or checklist for a pre evaluation of RAM applications.

For further research, we plan to use other usability evaluations aiming to find other types of GUI design problems, apply this evaluation in MAR applications with multimodal interfaces, such as, voice commands and hand gesture recognition. An update in ARguide application using these guidelines will be conducted to verify the benefits of the proposed guideline, as well as the drawbacks.