Abstract
E-health sustainable systems can be optimized by empowering patients in self-care programs through artificial intelligence ecosystems in which both doctors and patients interact in an agile way. This work proposes agent-based simulators as a mechanism for predicting the repercussions of certain self-care programs in certain patients for finding the most appropriate ones. In order to make this easy for both doctors and patients, mobile agents are used to configure an app for each patient, and this app provides the resources to each self-care program. Mobile agents include a machine-learning module for learning which programs are the most appropriate for each patient. This approach is illustrated with two agent-based simulators for respectively reducing negative emotions such as depression and controlling heart rate variability extreme values related to stress. The resulting app was evaluated with a group of users with the Usefulness, Satisfaction and Ease of use (USE) scale and obtained 73% in usefulness, 77% in satisfaction, and 68% in ease of use. This trial is registered with According to the recommendations of the International Committee of Medical Journal Editors (ICMJE), this manuscript states that all experiments have been approved with the ethical committee CEICA from Community of Aragon (Spain) with registration number C.I.PI18/099.
1. Introduction
E-health sustainable systems can be supported by providing tools to instruct patients in conducting self-care programs [1]. Self-care behaviors are important in patients with heart failures to avoid the progression of their diseases. In a similar way, self-care programs can help in motor rehabilitation [2], as one can observe in the effects of the studies with augmented-reality and virtual-reality applications and serious games for this purpose. In the context of emotions, self-care mindfulness programs have an influence on pain relief [3] and reduction of anxiety [4]. Self-care programs can be especially useful for patients with chronic conditions for achieving sustainable primary care [5].
Mindfulness has proved to have several benefits in health indicators. Demarzo et al. [6] presented a theoretical review of the mechanisms for including mindfulness-based interventions in healthcare systems. They encouraged the development of mobile applications for providing introductory courses of mindfulness for patients. A recent study has proven that mindfulness programs reduce anxiety in students in the COVID-19 pandemic circumstances [7].
Although self-care programs are well known, the simulation of their repercussion on specific patients given their initial features remains a challenge. In this research line, our previous works about agent-based simulators (ABSs) for simulating mindfulness programs reveal the potential utility for estimating this mindfulness repercussion in patients with certain features. An ABS is a multiagent system composed of autonomous entities that simulate the interactions and evolution of several actors in an environment, usually for predicting a future outcome departing from an initial setting [8]. In particular, we developed and validated an ABS of emotions in mindfulness programs (ABSEM) [9] and an ABS of the influence of mindfulness programs on heart rate variability (HRV) called ABS-MindHeart [10]. However, these systems lacked the proper mechanism for transferring self-care programs into an app that could guide patients. It also lacked the proper machine-learning (ML) mechanism for improving the initial configuration of these programs and the supervised learning for adapting to users’ needs based on their feedback.
In this context, this work proposes the usage of ABSs in combination with ML for providing a customized experience for patients extracting knowledge from doctors or instructors through mobile agents. More concretely, ABSs allow simulating the effects of certain self-care programs in patients. Mobile agents use the selected self-care programs to instruct patients through mobile apps with resources for guiding different exercises. These mobile agents use ML for improving their programs through supervised learning from the specific patient, as well as for initial configuration learned from existing datasets. The main contribution of this work is the integration of ABSs and ML as a mechanism for improving the user experience of apps for empowering self-care programs by properly predicting the effects of different programs and guiding the user through the selected one.
The remainder of the article is organized as follows: the next section introduces the related work highlighting the gaps of the literature covered by this work. Section 3 introduces the proposed approach for using ABSs for empowering the selection of self-care programs, its transmission to a mobile app through mobile agents for guiding patients, and ML for a customized experience, illustrated with mindfulness self-care programs. Section 4 focuses on the experimentation of the novel parts of this approach discussing the most relevant findings. Section 5 mentions the main conclusions and depicts some future research lines.
2. Related Work
2.1. Mobile Agents
Mobile agents have raised several problems from the perspective of theoretical computer science. Among others, one of these problems is the partial gathering of mobile agents. In this problem, agents should move to a node and terminate so that each node has at least a certain number of agents at the end. Shibata et al. [11] proposed three algorithms for solving this problem. One algorithm is randomized, while the others are deterministic. They prove that the movement complexities of two of these algorithms are asymptotically optimal. One of these optimal solutions is randomized, while the other requires a unique identifier for each agent. The remaining algorithm is deterministic and does not require unique identifiers, but its complexity has not proven to reach asymptotic optimality. Moreover, Dereniowski et al. [12] introduced the rendezvous problem for two heterogeneous mobile agents in an undirected graph. These are required to traverse certain edges at different times. These know the whole graph and the initial positions of both agents, but each agent ignores the traversal times of the other agent. The evaluation compares the meeting time of this problem with the meeting time in the offline scenario where each agent has all the information about the other agent, such as its speed. The complexity of their solution is the optimal offline meeting time multiplied by the number of nodes. By contrast, the current approach is not aimed at solving a theoretical problem related to mobile agents. Instead, the current work is focused on simulating the specific repercussions of a practical domain-specific context of self-care programs (e.g., mindfulness programs), and the goal of the mobile agent is to customize an app with the appropriate mindfulness program after several simulations.
Mobile agents have also been used to solve the mobile user coordination problem. In this problem, users schedule and reschedule their tasks through mobile devices when they change their goals. In particular, Lin and Lin [13] formulated this problem as a multiobjective context-dependent distributed constraint optimization problem. In particular, they proposed a two-stage distributed optimality reaching approach and implemented it with a multiagent system (MAS) that integrated simulated and physical agents. They evaluate the performance of their approach with several experiments with a traveling backpacker problem. Both this work and the current one are related to mobile applications. However, that work is mainly focused on the coordination of the users, while the current one is aimed at simulating the influence of self-care programs on certain features of the practitioners.
2.2. Applications for Supporting Self-Care Programs
There are several mobile applications that support the practice of self-care programs, such as mindfulness programs. To begin with, Chittaro and Vianello [14] introduced an app for assisting mindfulness practitioners. This app is available from the main online stores. Their app has been experienced by a group of meditators with the app over a four-week period. The feedback of the participants showed their positive feelings about the usage of the app. In addition, AEON [15] is another app for supporting thought distancing in the context of mindfulness. Practitioners write their thoughts in the app. Then, the app shows their thoughts with a hand-writing appearance within the water. The app displays a smooth transition that dissolves these thoughts. This app increased the mindfulness and pleasantness of naive meditators compared to the common thought distancing technique with paper. However, these apps cannot be automatically customized from certain self-care programs. In addition, these works do not allow users to simulate the repercussions of distinct self-care programs for selecting the appropriate one and automatically adapting the app accordingly.
Moreover, other computer-based applications assist practitioners in training self-care programs. For instance, the work of Hudlicka [16] is aimed at virtually coaching the training of mindfulness. Her approach includes a conversational character that can maintain a natural-language dialog. It customizes the training plan for each user according to their responses. Nonetheless, the customization of this work is not based on the simulation of several self-care programs for estimating their repercussions in certain features of practitioners.
2.3. ABSs in Healthcare
Some ABSs have been applied in healthcare. For example, Kim and Yoon [17] presented an ABS that estimates future trends of customers and strategies of firms. They applied their approach in the healthcare industry. In addition, Cabrera et al. [18] introduced an ABS for optimizing the healthcare emergency department. Their ABS supports the decisions of emergency department managers for designing the strategies for handling emergencies when prioritizing efficiency.
Furthermore, Macklin et al. [19] used an ABS for simulating the spread of cells with DCIS (ductal carcinoma in situ), which is a precursor to breast cancer. They consider the interactions of these cells with certain membranes. Their simulations assist researchers and doctors in understanding the growth and calcifications of DCIS. They assessed their approach by analyzing the differences between the simulated outcomes and the real ones from existing mammograms.
Thereupon, several ABSs have been applied in healthcare. These ABSs assist researchers in understanding certain diseases and support decisions for emergencies and industry in healthcare. However, none of these ABSs estimates the influence of self-care programs on health indicators.
2.4. ML in Healthcare
ML has been applied for healthcare in different contexts. Although ML brings advantages in the improvement of automatic recommendations, there are several challenges around the application of ML to healthcare [20], such as finding the appropriate preprocessing, model training based on appropriate data, and the refinement of the system according to the specific clinical problem. In fact, Futoma et al. [21] identified the myth of generalisability of the application of ML to healthcare. ML is usually applied in a given context with data collected in certain circumstances and can be useful in certain domains, but it may not work when applied in different domains, contexts, or circumstances. This motivates the current approach for exploring ML in particular contexts of self-care programs.
In the context of ML applied to self-care programs, the review of Fatehi et al. [22] showed that self-care programs are widely used in the context of diabetes and indicated that ML is likely to be applied for personalized recommendations as one of the future directions. In this domain, ML has been applied in predicting type 2 diabetes through a decision tree. Prediction of specific types of diabetes is one of the first steps towards personalized self-care programs. However, these approaches miss the usage of ML for actually recommending a self-care program with an app guiding each step of the customized program, as the current work does. The aforementioned approaches are more focused on the diagnosis and its treatment rather than customized self-care programs.
ML has also been applied to estimate specific information of patients for properly applying self-care programs. For instance, ML was applied to estimate the indoor location of patients for properly applying the virtual care of patients [23]. Nevertheless, in this work, ML was not applied to directly obtain customized self-care programs based on the information of patients.
In conclusion, ML is considered to be one of the most promising lines for improving self-care although some challenges remain. In diseases such as diabetes, some ML approaches can estimate the type of diabetes for the application of the right treatment. In addition, ML can estimate some missing information such as the indoor location for properly applying virtual self-care. Nonetheless, to the best of the authors’ knowledge, there is no approach for obtaining self-care programs in the psychological field by using ML combined with ABSs integrated with proper automatic guidance through appropriate resources. This gap in the literature is covered by the current approach presented in the next section.
3. ABSs for Empowering Patients in Self-Care Programs Using Mobile Agents with ML
3.1. Overview
An overview of the current approach is illustrated in Figure 1. The current approach is illustrated with two ABSs about self-care mindfulness programs. First, mABSEM simulates the repercussion of these programs on the emotions of practitioners. Second, mABS-Heart simulates the influence of these programs on the HRV of practitioners. The former ABS was firstly developed as a desktop application. Then, the latter ABS was developed as an app and an online tool to reach a wider range of users and places. Both ABSs followed the Process for developing Efficient ABSs (PEABS) [24] using its simulation engine and framework. In particular, mABSEM used the original framework of PEABS adapted for Java, while mABS-Heart needed to use an adaptation of this process and framework for developing apps.
In the current approach, both mABSEM and mABS-Heart have been adapted to be able to export mobile agents with mindfulness programs. In order to unify the format of the mobile agents, the current approach uses a unified unrolled simple text format. In fact, in both ABSs, the instructor agent automatically records the activities that are instructed for each session of the last simulation. In this way, each instructor agent can be exported as a mobile agent in both approaches with this unified unrolled text.
From an internal working perspective, a mobile agent is a plain-text code file in a simple domain-specific language, in which each line indicates one exercise. This file is transferred to the app. The app runs this mobile agent by considering each exercise and including it in the self-care program for guiding the patient. This file could be considered software in a domain-specific language defined by us. However, this language does not provide any instruction to access the user file system or communicate to any external source. Thus, there is no way that this mobile agent can be used to install any malware program on the user device as far as the authors know.
Finally, the mobile agent can be loaded in a meditation application for supporting mindfulness. The agent customizes the meditation application in order to support the particular mindful activities in the specific sessions. There are two versions of the meditation application: the first one is a mobile application, while the second is an online tool. In this way, the mobile agent can be loaded into an app or online tool and customize any of these applications.
In the proposed approach, the steps from the user viewpoint considering the aforementioned approach and its integration with ML are as follows: (1) The user selects which skill, aspect, or feature they would like to train from the available options; (2) the user replies the corresponding surveys for identifying their initial state; (3) the program applies ABS for providing information about existing programs and their potential impact; (3) the user selects a type of program; (4) the ML approach provides an expected impact of the selected training program, and the user can optionally return to the previous step; (5) the mobile agent guides the user in the application of the intervention considering the selected program; and (6) the user can optionally provide feedback for both improving the customized recommendation of the program and decreasing the errors considering their postintervention measurements.
3.2. ML Module for Improving Self-Care Program Recommendations with Supervised Learning
General knowledge can be useful in proposing new self-care programs at the first stage. However, users, even with similar features, may have specific preferences. Thus, this approach uses supervised learning for updating the ML model when having specific feedback from a given user.
This ML model uses regression for predicting the level of patient satisfaction for a given self-care program. The input features used for training the model are the number of activities of each exercise type in the self-care programs, the total duration of the self-care programs, and the initial states of patients (e.g., anxiety level). The programs are associated with the target feature of satisfaction provided by patients for training the model.
Figure 2 shows the block diagram related to this supervised learning. The user can optionally provide feedback of a given experienced self-care program in the app, and this supervised learning will be used to update the local ML learned model, customizing the predictions considering the particular user.
In order to have a noticeable impact on the ML model, this model applies a weighting mechanism based on adding the same case from the user a certain number of times. The number of additions of the same case is calculated considering the total size of the initial dataset and the confidence of the user on their feedback.
In particular, we applied the following formula for calculating the number of times to add a case in supervision for weighting it up:where is the number of cased added for the ML, is a constant value (e.g., 0.10 standing for 10%), is the size of the original training dataset, is the confidence of the user (usually in five-point Likert scale), and and are the minimum and maximum limits of the used confidence Likert scale.
In the case of using natural-language text as input, such as the written comments of patients, this methodology proposes applying ELMo and Bidirectional Encoder Representations from Transformers (BERT). These techniques use word embedding strategies for capturing not only the words but also the semantic and syntactic similarity relations with other words.
3.3. mABSEM
Figure 3 shows the user interface of mABSEM. The “Export Instructor Mobile Agent” button allows the user to export the mobile agent and saves it on a file with the preferred name and location selected through a file chooser. In order to facilitate the exportation of the instructor mobile agent from different possible applications, the current approach has selected a simple unrolled notation text. In this way, the self-care mindfulness programs of ABSEM can be exported as shown in the current approach, and the ABS-Heart app can export self-care mindfulness programs. Notice that the instructor agents are represented with different programming languages in these two ABS. In particular, ABSEM and ABS-Heart, respectively, use Java and JavaScript.
The mobile agent is represented with an unrolled representation of the mindfulness program. It just contains several lines, and each line determines a session number and the activity type separated by a colon. If there is a session with several activities, it is represented with several lines in which the session number is repeated. If an activity takes more than one session, this activity is only associated with the first session, and it is assumed that the practitioner continues with this activity until there is another explicit activity in the program.
Figure 4 presents a relevant excerpt of the class diagram of mABSEM. The different instructor agent types can be defined by extending the abstract “Instructor Agent” class. The different instructor types (e.g., the Song class and the Wongtongkam class) only need to implement the live method. This method can access the session number by the “Get Session” method. It can apply different mindfulness practices by calling the “Apply Practice” method with values of the “Practice” enumeration.
The Instructor Agent class internally manages the internal representation of the instructor mobile agent for all its subclasses in a transparent way. In this way, developers do not need to perform any specific action for making specific instructor agents available as mobile agents.
In each simulation, the instructor agent is created, and the constructor starts with an empty representation of the schedule of the mindfulness program. Each time the “Apply Practice” method is called from any subclass, the parent class records the session number and the practice identifier name, with the particular textual notation. After the simulation, when the instructor agent is required to move, the parent class provides this schedule as the unrolled representation of the mobile instructor agent with all the necessary information about the application of practices for a given number of iterations.
3.4. mABS-Heart
The influence of self-care mindfulness programs on HRV is simulated with mABS-Heart. Figure 5 presents the most relevant parts of the interface of this simulator. As one can observe in the initial interface of Figure 5(a), this system receives input from the self-care mindfulness program, the number of practitioners, the number of iterations, and the initial average of HRV measured with the high-frequency power in the normalized unit (HF n.u.). The initial interface has buttons for, respectively, running the simulation and exporting the instructor agent as a mobile one.
(a)
(b)
(c)
This system simulates the evolution of HRV considering different influences for the particular mindful activities. Among other aspects, it takes into account the integral of the normal distribution with the mean centered in the initial mean of the group for implicitly considering aspects such as their age and their usual sport activity. This system provides the simulated HRV of all the practitioners, as well as their average and standard deviation. It also presents the evolution of the average HRV with a chart like in the execution example of Figure 5(b).
Moreover, mABS-Heart can export the instructor agent as a mobile one, with the same text notation as in the previous ABS (i.e., mABSEM). The system presents the content of the mobile agent in a text area as shown in Figure 5(c). Users can directly copy this content of the mobile agent to the meditation app, or into a text file for saving it.
In mABS-Heart, the self-care mindfulness programs are defined by extending the instructor agent and implementing its “Live” method like in the previous ABS. mABS-Heart uses JavaScript, as it is implemented for developing the app with the Cordova framework and for being used as an online tool. The implementation of the exportation is also transparent to the definition of mindfulness programs as specific types of instructor agent. The implementations of these specific types invoke a method of the parent for applying each practice. Besides simulating the particular practice, this method appends the iteration and the practice to the textual representation of the agent. After the simulation, the mobile agent is ready to be exported to the self-care Meditation app or the online tool.
3.5. Self-Care Meditation App and Online Tool
Figure 6 shows the user interface of the self-care Meditation app. In particular, the user needs to open the text file with the mobile agent and paste its content in the corresponding text area of the mediation app as shown in Figure 6(a). When the user presses the “Load Mobile Agent” button, the mobile agent is loaded. Then, the agent customizes the meditation app. Figure 6(b) shows an example of the meditation app customized for the mindfulness program of Song and Lindquist [25]. Notice that the whole program can be accessed by scrolling. One can observe all the activities of the program on the unrolled text of the mobile agent of the previous figure. The customized meditation app has one or several buttons for assisting the mindful activity (IES) of each session of the mindfulness program. Each button has the session number between parentheses and the name of the activity type. In this way, the practitioner knows which mindful activities are assigned to each session. When pressing on a button, the app shows a new page with informative and/or audiovisual content for assisting the practice of the corresponding activity type. As an example, Figure 6(c) shows the page that supports the body scan activity.
(a)
(b)
(c)
The current approach also includes the Meditation online tool, which is similar to the Meditation app. It has the same functionalities and a similar appearance. This tool also loads the specific instructor agents with the same text notation like in the app. This allows reaching a larger number of users, considering that some of them might want to use or test it without installing it.
4. Experimentation
In order to experience the proposed approach, we applied it using the dataset about the application of different mindfulness programs such as the Vipassana meditation and using the validated and online-available Kaggle repository about the Kentucky Inventory of Mindfulness Skills Responses https://www.kaggle.com/lucasgreenwell/kentucky-inventory-of-mindfulness-skills-responses(lastaccessed01/15/2021) with data from 601 participants.
In particular, we have used age, gender, observing skill, and describing skill as features of the user of the app, and we have used our approach for predicting the repercussion of the self-care mindfulness program on the acquisition of self-acceptance and situation acceptance, measured with the acceptance skill of the Kentucky inventory [26].
As ML techniques, we used Multilayer Perceptron (MLP) regressor, K-Nearest Neighbors (KNN), and decision trees. The MLP used two hidden layers of, respectively, 5 and 2 neurons and a maximum number of iterations of 100. In KNN, we used two neighbors (i.e., K = 2). We randomly divided the dataset into two separate sets for, respectively, training and testing, in which the size of the test set was 0.33 of the original dataset size. We compared the predicted values with the real values, and Figure 7 shows the boxplot of the errors for the three different techniques of MLP, KNN, and decision tree.
In this experimentation, we also applied a CNN with the Tensorflow library, with two dense layers of, respectively, activations called “ReLU” and “softmax” with a middle dropout layer with 0.3 value and the necessary flatten layers. The average error of the CNN was 2.21, which was very similar to the average error of 2.17 obtained by MLP. These results of the CNN may be explained by the fact that this particular scenario only used several one-dimensional psychological measurements of participants instead of images or time series in which CNNs have widely proven to be useful. Since the CNN did not provide an improvement in the results and required more computational capacity, we did not include this in the final system used by the users.
In the prediction results, one can observe that KNN obtained the lowest errors in this particular domain of the repercussion of self-care mindfulness programs in the Kentucky acceptance skill. Notice that the most appropriate ML technique may vary depending on the domain or the preprocessing, and we suggest prescreening in the particular domain of self-care programs.
As the difference between these two techniques was noticeable, we selected KNN as the most appropriate technique. We configured different numbers of neighbors to test which may be the best value for this parameter. Figure 8 presents a boxplot for comparing the errors of KNN when applying different configurations of numbers of neighbors. We considered the common range between one and five neighbors. In this range, as one can observe in the boxplot, the best results were obtained with three neighbors.
We also measured the execution time of the three ML techniques for both the training and the prediction, and Table 1 shows the resulting times. One can observe that KNN is the fastest in training and decision tree is the fastest in the time prediction. Thus, the decision tree would be the best option for real-time prediction if one just considered execution time.
This ML mechanism was integrated into the mobile agent so that it is firstly deployed with a KNN approach with three neighbors and the original dataset. Then, each mobile agent increased its initial dataset with the feedback from the user about the acquired Kentucky acceptance skill. The ABS used the common Vipassana mindfulness self-care program as the basis with minor modifications concerning the particular exercises guided in the specific instruction program as explained in the presentation of the current approach.
The resulting app was suggested to a group of 29 potential beginner meditators. Twelve users actually decided to participate in this study. They followed all the steps, including replying to the surveys for their initial emotional state, and they conducted the recommended program and provided their feedback.
In the supervision stage, we applied the formula presented in equation (1) with and a five-point Likert scale for confidence. Consequently, the number of cases added for each feedback was either 1, 16, 31, 46, or 61 depending on the confidence level indicated by each user, respectively, for 1 to 5 confidence values.
All the participants were asked to repeat the experience at least once considering their new emotional state and the improvement with ML. After their experience, they evaluated their experience Usefulness, Satisfaction and Ease of use (USE) questionnaire, and the average results were 73% in the dimension usefulness, 77% in satisfaction dimension, and 68% in ease of use.
5. Conclusions and Future Work
This work has presented a novel approach of ABSs with mobile agents for simulating the repercussion of self-care programs on specific patients considering their features. In this way, instructors can simulate and recommend programs through mobile agents, which will guide each user on following the program with resources for each specific exercise. In addition, these mobile agents include a ML approach for supervising the learning from each specific user, using an original dataset incrementally improved with the data from the user.
From the results about usefulness and satisfaction provided by the app users, we conclude that the integration of the ABSs and ML for predicting the repercussion of self-programs integrated in an app for the guidance through the programs has achieved a great user experience in terms of usefulness and satisfaction. As shown in this article, this approach has applicability in both the prediction of (1) reduction of negative emotions, such as anger, anxiety, and depression, and (2) HRV health indicators associated with stress. However, the experimentation of this work is now limited to the context of self-care programs based on mindfulness meditation, and we cannot extrapolate our findings to other self-care programs such as the ones that include medication.
As future work, we plan to present this approach to doctors of different fields of medicine, physiotherapy, and psychology to gain perspective on which other fields in this approach can be safely applied. Later, we plan to apply our approach to another field based on the feedback from doctors and develop an app for the selection and guidance of self-care programs. This app will have a paid option to let users be supervised by and communicate with doctors, so the app is more useful for patients and contributes to sustainable healthcare.
Data Availability
All the relevant data are included in the manuscript.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding this article.
Acknowledgments
This work was done in the context of the project “Technological Solutions for Facilitating the Mindfulness Practice: Walking Towards mHealth” with reference TEC2013-50049-EXP and supported by the Ministry of Economy and Competitiveness. The authors also acknowledge “CITIES: Efficient and Sustainable Smart Cities” (ref. 518RT0558) funded by CYTED (“Iberoamerican Program of Science and Technology for Development”) and “Collaborative Design for the Promotion of Well-Being in Inclusive Smart Cities” (TIN2017-88327-R) funded by the Spanish Council of Science, Innovation and Universities from the Spanish Government. The authors would like to acknowledge Prince Sultan University (PSU) and Smart Systems Engineering Lab for their valuable support and provision of research facilities that were essential for completing this work. Also, the authors would like to acknowledge the support of PSU for paying the Article Processing Charges of this publication.