Keywords

1 Introduction

Low fluid intake is an important issue in the aging population. There are many reasons why elderly people do not drink enough – for instance because the sensation of thirst decreases with age [27]. Diseases such as dementia can increase the risk. Frequently, the result is a threatening and often overlooked state of dehydration, which is one of the most common reasons for elderly people to get hospitalized [21]. Reduced body functionality and impaired cognitive skills are some of the typical symptoms. The loss of over 15% of the water in the human body can lead to death.

In retirement homes, caregivers are responsible for many elderly people at once. They have to maintain detailed protocols of everyone’s drinking habits. A system which tracks peoples’ fluid intake automatically and which reminds them to drink would be of great benefit since it would reduce the administrative workload of the caregivers.

As part of the enable project [2], we have developed a drinking platform in the form of an inconspicuous beermat [22] for elderly people. The smart drinking gadget is wirelessly connected to an application on a mobile device which records how much a person has drunk. To stimulate a healthy amount of liquid intake, the application includes three Serious Games which generate drinking reminders in specified intervals. We have evaluated the system in a retirement home with dementia patients and their caregivers. To this end, we have developed a Combined Evaluation Process consisting of a combination of several essential evaluation strategies (short-term and long-term), as well as communication guidelines, structured into four levels. The focus lay on evaluating the gadget and application for both major user groups, the elderly/dementia patients and the caregivers. We also discuss the results of the pilot study based on the drinking data protocols and on the caregivers’ opinions.

2 User-Centered Design (UCD) Approach

Applications are only useful and have a chance to be accepted when software validation aspects and design guidelines have been balanced and validated. In this respect, usefulness means that there is a benefit for the user towards achieving a particular goal. It should not be mistaken for good usability, which stands for an easy and pleasant user experience [15]. To achieve both, Holzinger and Errath [15] and researchers such as Queirós et al. [23] and Albaina et al. [7] recommend a User-Centred Design (UCD) approach, which focuses heavily on the end user. In the context of mobile app development, this approach consists of three phases [15]:

  1. 1.

    Investigation and observation of the end-user: To gain insights into the real-life workflows with the goal to detect benefiting elements when using applications.

  2. 2.

    Paper mock-ups: Going into detail for the workflow and parts of the application.

  3. 3.

    Evaluations: Testing the developed solution with cost-effective prototypes and iterative approaches to optimize the result [18]. In the process, user interactions with the application can be constantly improved [23].

3 Evaluation Strategies in the Context of Elderly and Dementia Patients

To evaluate a gadget and Serious Games for elderly people, we researched various directions of related literature to gain a better understanding of the user groups.

3.1 The Questionnaire—Evaluation Method of Choice

Stöber et al. [24] are describing guidelines for the development of questionnaires in the field of elderly people. It is important to incorporate the scientific quality criteria objectivity, reliability and validity. The questionnaire should start with a small instruction. On top of that, simple questions targeting previous experiences with the device should be incorporated. When formulating questions, two types of items are recommended: asking questions to discover issues (e.g. “How often are you using mobile devices?”) or assertions to see the reactions (e.g.“The usage of the app is fun.”). While an assertion ends with a fixed set of answers (e.g. applies, doesn’t apply), normal questions can be either open-ended or closed. Stöber et al. [24] recommend closed questions when targeting the elderly to avoid receiving potentially unfinished or too short answers. A question should only contain one topic and generalizations e.g. all, always etc. and causal subordinate clauses should be avoided. Andersen et al. [8] point out that, in contrast to interviews, questionnaires can avoid the introduction of irrelevant topics. They used a two-step approach, where they first let the person rate their extent of emotion for a given topic and then investigated with other questions, why the person rated like that. On top of that, interviews and group questionnaires for the two-step approach were used.

Several standardized questionnaires have been developed for the clinical environment. The Short Form Survey [6] specifically focuses on the measurement of the life quality of patients. The SF-36 version is designed for being used e.g. in small-scale treatment evaluations. It takes only 5–10 min to be completed and can be used to determine each individual improvement of the life quality of a person.

3.2 Evaluation and Communication Guidelines when Communicating with Dementia Patients

When Alzheimer’s disease (the most common form of dementia) progresses, communication deteriorates. Tappen et al. [25] have researched communication techniques with Alzheimer patients.

One of the most important ways of talking to patients is to use open-ended questions (asking about experiences, opinions or descriptions). This enables the Alzheimer patient to build up a relationship with the asking person and to tell more about themselves, which of course benefits the understanding of the patient’s experiences and opinions. Tappen et al. [25] could clearly identify longer answers when using open-ended or mixed questions in the form of an open-ended question followed up by a closed one (only yes or no answers). The answers were richer with important content and insights [4].

When a foundation (e.g. familiar environment) for the conversation is established, more specific questions can be asked. In general, the five Ws (Who, Where, What, When, Why) are a good start. They are easy to understand since the direction of the question mentioned at the beginning of the sentence.

Furthermore, eye contact is important to indicate interest [1]. It is difficult to ask personal relevant questions, e.g. about the patients’ illness, life in the nursing home or family members. There are a lot of traps even asking simple questions like “How are you today?” because this can lead into depressed answers about being not well or missing family members. When confusion or frustration takes over, arguing should be avoided, instead the subject should be changed to keep the person interested [4, 5].

There are multiple approaches to improve communication and to talk about feelings [25]:

  • Broad openings: Open-ended questions which encourage response. This approach counters anger, which can be experienced when asking detailed questions.

  • Speaking as equals: Being open to learn from the person with the development of a partnership in mind where both are equally valued without a hierarchy.

  • Establishing commonalities: Discussing and sharing of interests and perceptions to find commonalities.

  • Sharing of self: Talking about own feelings or relating to what the patient had contributed to the conversation.

  • Maintaining the conversation: Finding a striking topic at the beginning to start a conversation and then connecting consecutive thoughts (avoiding topic changes by the Alzheimer person).

Another helping approach could be the usage of gestures or visual cues, e.g. pointing at the toilet instead of asking [5]. These cues can also work as a memory aid. Bringing memories back, e.g. the day of the proposal or marriage, can help to start a conversation. Alzheimer patients still remember small details and come back alive [3, 4].

Overall the recommendations suggest avoiding closed questions, when trying to establish a relationship. Showing emotions is useful, but the emotions should not dictate the conversation. They are needed to let the Alzheimer person understand the feelings and possible concerns of the talking person to gain acceptance [1]. Support can be established by encouragement in verbal and nonverbal ways, reflection or summarization. But there are limitations in the research in the direction of communication with late stage Alzheimer people [1, 25].

4 Evaluation Process for Caregivers

Gaining insights from care staff can be a challenge. A lot of important details are hidden in small daily activities. For that reason, it is useful to utilize specific strategies with a focus on real-world situations and problem statements.

4.1 Naturalistic Enactment

Castro et al. [9] detected the problem with human interface evaluation for widely used techniques such as Controlled Experiments and Heuristic Evaluation. For them these approaches and their artificial environments missed the “[...] contextual conditions in which applications are used” [9, p. 371]. This problem is even more severe for mobile platforms/gadgets, which in contrast to the PC, will be used in different locations and under different conditions. There is no way around using a real-life scenario with “exposure” to technology and app content, to detect potential difficulties not only with the application, but also the devices and platforms used [9, 15].

For that reason, Castro et al. [9] used the auxiliary technique called Naturalistic Enactment to evaluate an assistive mobile application for nurses. Their approach contains two parts, first the Naturalistic Enactment. This includes an in-situ evaluation with a combination of a controlled setting and a high ecological validity, which contains the integration of real users and scenarios in their environment. Secondly, a posterior data collection and investigation. While a controlled experiment focuses on speed, efficiency and effectiveness, the Naturalistic Enactment focuses on situations occurring when the application is used in a real scenario. Castro et al. [9] had the goal to bring structure in the interaction and diagnosis between nurses and patients. To start early with an end-user focus, Eysenbach [13] conducted a survey of different features for a mobile application and the importance of these features rated by eHealth experts. This strengthened the development process and the subsequent evaluation process with a pronounced focus on real life scenarios.

4.2 Focus Interviews

Focus Interviews are another important way to gain knowledge from caregivers. Hopf [16] describes focus interviews as one of the best ways to get expert knowledge. They are much more suited for gaining insights into activity alternatives or self-interpretations than e.g. questionnaires. In questionnaires people tend to rate elements only in the middle of the rating spectrum, leading to bias and often unusable results. Also losing focus can be better avoided with an interview than with a questionnaire [8].

Hopf [16] described several different types of interviews, such as the Structure Interview which is mostly predefined by the creator of the interview or the Clinical Interview which focuses on the diagnoses and condition of a disease. Yet, the two main methods of choice are Focus interviews and Narrative Interviews. While a narrative interview gives the interviewee a lot of freedom to guide the conversation in certain directions, phases of life and memories by only asking one initial question, it is not suited to ask specific questions about a gadget or application. Focus interviews are much more useful in such a scenario. They have a predefined conversation topic with the goal to collect reactions and interpretations from the expert to a certain topic and to maximize the spread of topics in the given area of the interview to gain more insights from the interviewee.

Four criteria need to be considered:

  1. 1.

    Range: The interviewee must be able to react and to introduce new aspects.

  2. 2.

    Specificity: The interview needs to cover concrete focus points.

  3. 3.

    Depth: The interviewees should be able to set own focus points with their individual opinions and interests.

  4. 4.

    Personal context: It is important to understand from which context the interviewee answers the questions.

For Hopf [16] it is important to select qualified personnel with deep understanding of the topic of the interview.

5 A Combined Evaluation Process

We have created an evaluation process for application scenarios involving a combination of hardware (gadget) and software (Serious Games) for two major user groups, the elderly/dementia patients and the caregivers in a retirement home. It is illustrated in Fig. 1.

Fig. 1.
figure 1

The four levels of the Combined Evaluation Process

Full size image

5.1 First Level – Gaining Insights

The goal of the first level is to obtain knowledge about the patients by using a basic questionnaire [24] with the focus on demographic data, previous experiences with mobile devices and the openness in general for modern technology (open-ended questions with closed ones as follow-ups [25]). If the questioned target group faces a common disease, it is important to measure the state of the disease with e.g. the Mini Mental Status Test (MMST) [14] to gain early insights regarding possible candidates for the study.

A second questionnaire covers demographic data of the caregivers. Additionally, previous experiences with mobile devices, with respect to their work, can be included such that an introduction can be prepared for those who are not used to the technology. The main goal is to get a basic understanding of the people and their daily challenges, especially when working with e.g. dementia patients and to establish some early relationships with them. This level is inspired by the UCD approach, where the early investigation and observation of the end-user is important [15].

5.2 Second Level – A Connected World

To obtain a deeper understanding of the daily routines and challenges faced by the different target groups, it is important to see the situations as a connected flow, rather than as independent user groups and their issues. This is essential for a gadget/application that tries to connect the elderly, dementia patients and caregivers and wants to feature advantages for all the users. To measure quality of life improvements, specific questionnaires, such as the Short Form Survey [6] have been created. They are typically based on a Measurement at the beginning and a Recall at the end of the evaluation.

To replace pencil and paper for recording the drinking intake, it is important to understand opportunities and limitations of the drinking gadget and the games. We use the guidelines of Hopf [16] for a qualitative focus interviews with the caregivers. The goal of the interviews is to get early feedback (reactions and interpretations) about the gadget/application to further plan the next steps and counter potential design flaws.

5.3 Third Level – Long-Term Evaluation Strategies

After establishing the first contacts and gaining insights into the connected workflow of the user groups, we test the gadget/application in a more long-term oriented evaluation. While the First and Second Level are more a combined approach for the elderly and caregivers together (see “connected” world), the Third Level focuses heavily on the individual user groups with the help of specialized, long-term evaluation methods.

For the caregivers, we use the Naturalistic Enactment approach by Castro et al. [9]. We create use cases to better understand the influence of our app (combination of hardware and software) in a specific situations in a natural environment. This gives insights into exceptional situations, which would otherwise not occur frequently. Of course, in a long-term evaluation not all use cases have to arranged, most of the situations occur naturally, enhancing the evaluation quality. Nevertheless, with a limited time frame, it is useful to intentionally play through some situations by simulating the behavior of elderly people with experienced caregivers.

For the elderly target group, a combination of multiple group sessions and a documentation of events allows to obtain a wide range of insights:

  • Focus Group: The usage of Focus Groups is helpful because of the living arrangements of retirement homes, where elderly people come together in small groups and e.g. watch television or participate in small games.

  • Diary: Another approach is to see how the elderly react to the app on a daily basis and document their behavior and thoughts in a diary such as suggested by Albaina et al. [7]. In their post-intervention step after the intensive testing and usage of the app, they used the persuasive Technology Acceptance Model questionnaire designed for detecting the usefulness, ease-of-use and acceptance of an app.

5.4 Fourth Level – Post Assessment After the Intervention

After the intervention period, the elderly fill out the Short Form Survey (long-term strategy) again as recall. Then the score is compared to their initial one to analyze improvements for the individual sections of life quality.

The conclusive perceived user friendliness of the app (perceived ease of use PEOU) by the elderly and caregivers’ satisfaction (perceived usefulness (PU)) is evaluated by using the Technology Acceptance Model questionnaire [26].

6 Hardware

We have developed a succession of drinking gadgets for the elderly [22]. This included different design strategies, such as extending an existing cup with technology or to add features to existing smart cups. The design process went hand in hand with experimenting with different sensor strategies to measure the liquid volume in a glass or cup, such as using movable objects in the liquid, distance measurement, utilizing liquid features and weight measurement. In the process we collected important feedback and finally choose a weight measurement approach.

Fig. 2.
figure 2

Requirements for the drinking platform and additional challenges

Full size image

The final gadget imitates a traditional beermat. It fulfills three major requirements (see Fig. 2):

  • Hygiene: A drinking glass that is used on a daily basis needs to be dishwasher safe. From previous project results and experiences of e.g. Kreutzer et al. [19], we consider it important that the sensor has no contact with the cleansing liquid. Sensors inside the liquid, such as electrodes, are a risk factor for mold and deposits. Furthermore, designing a cup with integrated sensors to be dishwater safe is difficult [20].

  • Own cup: In most cases elderly people prefer their own, existing drinking cups over prescribed ones. A common reason for such preference is stigmatization, e.g. when using a plastic cup over a regular one. Therefore, we selected a design which incorporates existing drink glasses/cups for a natural solution.

  • Safety: Movable-objects in the liquid (e.g. a swimmer) are not suitable because parts could accidentally be swallowed. Furthermore, the solution should add no additional risk of dropping the glass, e.g. due to a slippery grip around the glass.

Fig. 3.
figure 3

CAD model with components

Full size image

Our chosen solution was to take a weight measuring approach with a Force Sensing Resistor (FSR) in the form of a station which looks like an inconspicuous beermat (see Fig. 3). One platform is assigned to each patient, collecting data about their liquid intake over the day. As long as the glass stays the same, the weight loss or gain corresponds to the amount of liquid drunk by the person without the need of calibration. Multiple platforms for different people can be connected to a single mobile computing device to store the data of all gadgets.

The drinking gadget addresses three additional topics:

  • Interaction: The gadget allows an interactive experience in two respects. Firstly, we integrated two LED stripes to show simplistic light patterns. For example, through a constantly visible rainbow pattern the elderly person is reminded to put the glass back onto the beermat. Secondly, the gadget incorporates a novel kind of interaction with the Serious Games, controlled by the elderly person. Drinking notifications appear, reminding the person to drink some water. The gadget then acts as control element, causing the notification to disappear when something has been drunk. The interrupted Game Flow [10] caused by the drink notifications acts as an incentive for the elderly.

  • Expandability: Based on the Arduino platform with the open nature of its Bluetooth implementation, it is possible to integrate the gadget flexibly into various serious games, i.e., it is possible to expand the functionality to novel games and apps beyond the predefined one. This issue of flexibility and changeability is often overlooked in commercially available smart cups, rendering them unadaptable to novel application ideas.

  • Daily usage: Based on previously acquired knowledge regarding the routines of the caregivers in a retirement home [11], we chose to integrate a display into the gadget which can show the caregivers status information. It can also act such that a clear association between gadgets and elderlies exists. This alleviates the need for work procedures, such as having to replace all drinking glasses on the table regularly.

In the next step we developed an application platform which builds upon the specific features of the drinking gadget.

7 Software

The management component of the application incorporates the functionality to record the liquid intake measured by a drinking gadget. Furthermore, the application was designed to give the elderly person the opportunity to play three games [11].

7.1 Serious Games

We have developed three games for dementia patients, utilizing game concepts and mechanics which focus on cognitive training:

  • Balloon game: A simplistic game focusing on reaction time. It teaches elderly persons how to interact with the touchscreen. The player is encouraged to touch balloons of various colours and sizes, causing them to explode. Aside from familiarizing the person with the touchscreen, this concept has the goal to train the reaction speed (see Fig. 4a).

  • Pairwise matching game: A game focusing on memory training. The player has to find similar pairs of items (see Fig. 4b). Three levels of difficulty have been integrated: (a) matching pictures, (b) matching textual object descriptions and (c) matching a mixture of both. The combination of text and picture types helps elderly people to train and associate different areas in the brain.

  • Crossword quiz: The player has to find answers to various questions based on a topic (see Fig. 4c). A game focusing on cognitive thinking and knowledge. Answers can be typed in with a senior-friendly keyboard [11]. This is the most complex game. If typing the answer is too difficult for the person, it may be played with support by a caregiver hired for mental training. It is also very suitable for group settings when several elderly people are sitting in the common room. The entire group can then be included in the game, increasing the collaborative experience.

Fig. 4.
figure 4

(a) Balloon game (b) Pairwise matching game (c) Crossword game

Full size image

Drinking reminders are incorporated in all three games, with the goal to motivate liquid intake. The app interacts with the smart drinking gadget via Bluetooth to transmit and receive information. The total consumed drinking volume can be calculated from the decreasing weight measurements and sent to the database for further evaluation. For this purpose, the caregivers have to create a profile for each person such that the individual drinking volume can be profiled over time.

7.2 Architecture

The system architecture is composed of three different components: the gadget, the application and the database (see Fig. 5). The database is stored at the mobile computing device. It includes information about the user in one table. A second table protocols the measured drinking volumes.

Fig. 5.
figure 5

System architecture: interaction between all components.

Full size image

The elderly people are able to drink anytime, independently of the drink reminders which appear every 5 min when playing one of the serious games. If the person has recently drunk from the cup/glass, the notification is shown only shortly. If this is not the case, the notification will block the user from playing until something has been drunk (detected by a weight difference). If the current weight increases above a certain threshold, the application concludes that somebody has refilled the drinking glass.

8 System Evaluation Method

8.1 Experiment Design and Recruitment Strategy

We conducted a small intervention study at an elderly home in Munich, Germany. For this first pilot study we selected residents (women and men) identified to have a potential danger of dehydration. Additionally, each of the subjects was required to be responsive, not acutely ill, not in the terminal phase of life (definition according to the clinical frailty scale), able to see, able to drink independently and able to provide consent by herself/himself in order to be included in the study. As the overall usefulness and usability in elderly homes was part of the research aim, experience in the use of tablet computers was no inclusion criteria.

To gain additional insights from the caregivers’ perspective, we selected a group of caregivers. It consisted of women and men who worked at the ward of the included residents. They had to provide consent and work during the time of the study intervention.

During the entire intervention, at least one researcher was present, ensuring correct execution of the study and offering support in case of technical difficulties.

8.2 Assessment

The first visit served as an information event for the participants. We provided detailed information regarding the study procedures and each participant signed written consent. We conducted the baseline assessment for the residents. It included demographic data (age and gender), the German translation [17] of the Mini Mental Status Test (MMST) [14], as well as a short questionnaire concerning each elderlie’s self-appreciation of their drinking habit as well as their self-perceived computer anxiety (CANX) (see Table 1). This part of the pilot study represents the First Level of the Combined Evaluation Process (see Fig. 1).

For the Second and Third Level, we visited the elderly home regularly during our evaluation period (depending on flexibility in the elderly home). When time of the caregivers was available, we conducted several spontaneous qualitative Focus Interviews. The drinking application and games were being used in the common room with the elderly people of the test group and their caregivers. The chosen group of participants can be seen as a small Focus Group and we wrote down notes as diary (Third Level).

At the post assessment after the intervention period (Fourth Level), the caregivers’ satisfaction and the impression on the user friendliness of the application were evaluated using the Technology Acceptance Model (TAM) [26]. This assessment focused on perceived usefulness s(PU), perceived ease of use (PEOU), computer self-efficacy (CSE), computer anxiety (CANX), perception of external control (PEC), job relevance (REL) and behavioral intention (BI), each of which were measured on a five-point Likert scale.

Table 1. Assessment instruments, target group and measurement times. 1: Conducted during the baseline assessment; 2: Conducted during the post assessment
Full size table

9 Results

9.1 Sample Description

At the beginning of the pilot study (First Level of the Combined Evaluation Process), we collected basic demographic and person specific data to gain a good understanding of our target group. A total of 5 residents (R1–R5) and 5 caregivers (C1–C5) were included. The residents’ age ranged from 82 to 95 years. Two individuals were males and three were females. All residents lived in the dementia ward of the retirement home and suffered from moderate (14–18 points) dementia (majority had Alzheimer’s disease) with only one participant scoring over 19 points (R2, mild cognitive impairment) while the lowest result was 14 points (R3). Participants had most difficulties with the questions of Part A (Orientation), Part D (Memory) and the drawing part in section D (Language). Three of the five participants (R1, R3, R5) stated, that they thought they drank enough every day and estimated the required amount to be 0.5 to 1 liter. Furthermore, two of the inhabitants stated during the CANX questions, that they were afraid of working with computers (R3, R4).

The age of the caregivers (four females, one male) ranged from 25 to 56 years. Two of them were responsible for the supervision of the residents and the preparation of the entertainment program (C1, C3), two of them were trained caregivers (C2, C5) and one of them was a caregiver in training (C4) (Table 2).

Table 2. Description of participants.
Full size table

9.2 Descriptive Analysis

Due to the small number of participants and the lack of a control group, we did not conduct a statistical analysis. Instead, we conducted a descriptive analysis to explore the results of the pilot study.

Acceptance of the Intervention by the Residents: As the residents suffered from moderate dementia, they were mostly not able to play the provided games by themselves, as they were not able to remember how to navigate through the application’s menu or how to play the different games. Nevertheless, they were often open and motivated to play the games as long as a supervisor was sitting next to them to help during difficult phases. During the final assessment all participants stated that they enjoyed the game and that they would like to play them again.

Performance on Drinking Behaviour: We used the difference between single measurements of fluid intake to determine a reliable value of the fluid intake of the participants.

As the residents required constant supervision for the gameplay and since only one to two supervisors were in the retirement home at the same time, it was not possible to gain consistent overall values of the six intervention days from each participant. Furthermore, the residents were not motivated to the same degree every day and thus sometimes did not want to play with the system at all or only for a very limited time. In order to be able to see possible effects on the drinking behaviour of a user, we excluded all sessions which took less than 15 min from the final evaluation. As R3 never wanted to play the games for longer than 15 min at a time, no fluid intake tracking data was included from R3 in the final data evaluation. Nevertheless, we were able to analyze the reaction to the games and gain feedback from R3 during the post interviews at the end of the intervention period. Overall, we included five measuring sessions in the evaluation of the pilot study, of which two sessions were conducted by R2 as shown in Fig. 6.

The participants used the system for more than 280 min in total and consumed a good 400 milliliters per hour of use with an average of 11 drinking actions. About 37 mL were consumed each time with a standard deviation of 37,65. This shows the huge range of drinking amounts, as also visualized in Fig. 7. It ranged from less than 10 mL to over 150 mL, occurring once when R5 emptied the entire glass.

Fig. 6.
figure 6

Drinking amount over time for each resident (R).

Full size image
Fig. 7.
figure 7

Collected data during the intervention session in the elderly home

Full size image

Perception of the System by the Caregivers: After the intervention period, the participating caregivers were asked to fill out the TAM [26] questionnaire (Fourth Level with focus only on the caregiver side). The scores of PU indicate that the caregivers had a positive perception of the usefulness of the app, as shown in Table 3. None rated the usefulness with a score lower than 4. The usefulness was rated even higher with only two ratings lower than five (C3, C5 - 40%) and 4.5 being the lowest value. The values for CSE were a bit lower, due to the rating of CS4 who gained an end result for CSE of 2.0, while the rest of the caretakers rated their CSE as at least 4.25. Nevertheless, none of the participants perceived any general negative feelings towards digital devices, as shown through the low CANX value. As CANX measures the amount of anxiety towards computers and similar systems, low ratings (i.e., low levels of anxiety) are perceived as positive in this study. PEC is rated on the lower positive side at 3.5 to 4.25 with half of the rating being lower than 4. Although REL was perceived quite well, C5 rated it lower than 4. Nevertheless, all caregivers would like to use the system again as shown by the very good BI ratings with 4.3 being the lowest rating and by 3 caregivers (60%) 5.0 being the maximum.

Table 3. Mean and standard deviation of the TAM variables, assessed from the caregivers. 1: Not filled by C4 and C5; 2: Not filled by C2
Full size table

10 Discussion

Through the pilot study we could collect some important insights:

  • The dementia patients needed constant supervision and did not take the initiative to play by themselves. It is uncertain, if the behavior would change, if they are left alone with the option to play the games for a more prolonged time period. Nevertheless, if given the opportunity to play with the supervisors and caregivers, the interest of them in the serious games was quite high and they enjoyed themselves in most cases.

  • As measured through the gadget, a healthy amount of around 400 milliliters per hour, equals 2 standard drinking glasses, has been drunk when playing. As trigger the drinking notifications were quite successful and the elderly responded to them reliably. No participant declined to drink when the application asked for it.

  • The gadget was accepted by the elderly and we didn’t observe any form of resentment or fear towards it. Some of them were quite interested in the blinking new technology on the table. While others simply accepted it’s existence as a type of beermat (as intended by us) to put their drinking glass on.

  • The feedback by the caregivers was positive and interest for the technology quite high. The low REL rating of C5 can be maybe pointing towards the missing digital drinking protocol. Therefore, giving us the feedback to extend the offered features for the caregivers in the future to provide advantages for both major user groups.

Overall we were pleased with our observations during the short intervention time.

11 Limitations

The pilot study included a small user group, where each individual was accompanied for around 1.5 hours per day by a supervisor. This is a typical example of a short-term evaluation. In relation to the Combined Evaluation Process, we limited the scope mainly to the first two levels. Therefore, excluding the SF Survey) and long-term oriented strategies for the caregivers from the Third Level, e.g. Naturalistic Enactment. Instead we included the post assessment (based on the Fourth Level) with the TAM questionnaire [26] to collect the caregivers’ opinions.

12 Future Work

With a successful pilot study at hand, we want to start planning for future focus points of the project. This involves further gadget and application development, as well as refining our evaluation process:

  • Gadget improvements: We plan to create a refined version of the described drinking gadget, which we will use for an extended evaluation with our project partners at FAU [22]. We will work on some design details of the overall appearance and daily usability of the gadget.

  • Drinking notification strategies: One important result of the pilot study, was the missing initiative of the elderly to interact with the application. To solve this issue, we will have to find approaches to integrate the drinking notifications in the daily life of the elderly, which doesn’t require constant supervision. We are currently working on a microcontroller gadget with an application to include notifications in the TV program of the elderly. Furthermore, a reminder application has been developed for the iPad with a specific design language for the elderly and Alzheimer patients [12].

  • Long-term evaluation with the Elderly/Alzheimer patients: This is necessary to see what aspects are changing when the elderly are not supervised. For example, how many drinking notifications will be accepted over a longer time period by using various strategies, e.g. TV, reminder app. Another aspect would be to assess the life quality of the elderly when using the gadget and application with the SF Survey. This could give insights on the benefit of the Serious Games, as well as actual health improvements on the basis of the additional water intake.

  • Caregiver features: To be able to include the caregivers as a major target group in future studies, we will consider aspects found in this evaluation, which suggest a need for a deeper focus on workflow improvements. For that reason, a digital drinking protocol to monitor the liquid intake with an informative user interface, including e.g. charts for visualization, will be implemented.

  • Long-term evaluation with the caregivers: To develop benefitting features for the caregivers, a UCD approach by constantly gathering feedback is necessary. This includes Focus Interviews and the Naturalistic Enactment strategy, which is suited to evaluate the application with the help of scenarios and use cases.

13 Conclusion

After developing a smart drinking gadget for the elderly and dementia patients [22], we focused on a mobile device application with Serious Games. Our goal is to motivate the usage of modern technology through fun engagements and to stimulate a healthy liquid intake. With a working system architecture consisting out of the gadget, drinking notifications (included in the Serious Game application) and a working online database to store the liquid intake data, we shifted our attention towards a Combined Evaluation Process. It contains best practices from the literature to evaluate both major user groups, the elderly/dementia patients and their caregivers in a retirement home. The resulting process combines typical short-term assessment approaches and long-term oriented evaluation strategies for both groups. In our pilot study we had to act opportunistic to gain first results with the limited available time and mood changes of the dementia patients. Therefore, we could only identify some basic trends. But most importantly, we could show the feasibility of combining a drinking gadget with Serious Games with a healthy amount of liquid intake per hour during the intervention time, the acceptance of the gadget and drinking notifications. This opens the way for a future extension of the gadget and application to replace handwritten drinking protocols with a smart, automated and digital solution for the caregivers.