Abstract
For smartphone service providers, it is of vital importance to recognize characteristics of customers. The process of recognizing these characteristics is generally referred to as user profile, which provides knowledge basis for business decisions, enables intelligent services, and brings unique competitiveness. As a basic component of user profile, bedtime could reflect lifestyle, health condition, and occupation of people. This paper presents a flexible algorithm named BTP (Bedtime Prediction), which is designed for predicting wake time and bedtime by analysing screen status of smartphone. BTP first collects screen status log data of user’s smartphone and conducts preprocessing with a series of auxiliary user profiles. Then, it detects and records users’ wake time and bedtime of one day by searching and combining major screen extinguish periods in the past 24 hours. Finally, BTP predicts future bedtime by matching current screen status sequence with all historical records. By applying BTP, most of night and morning scenario-based applications could provide more considerate services, rather than following fixed execution time like alarm clock. Experiments on practical applications prove that BTP can effectively predict wake time and bedtime without applying complicated machine learning algorithms or uploading data to server.
1. Introduction
From eras of keeping records by tying knots to big data, data analysis technique has been developing throughout the history of human civilization. With tide of mobile Internet, enterprises, and governments, even individuals have begun to exploit their own business to portable devices especially smartphones. At present, mobile Internet has already exceeded personal computer on number of users. To serve customers better, it is very important to identify their preferences and characteristics. Because of different life background, Internet business must take care of personalized needs. This way of identifying user characteristics is generally called user profile.
Accurate user behaviour prediction could help mobile Internet service providers optimize their business on extensive scenarios. For instance, system may preload application which has the highest probability of being used next moment to shorten users’ waiting time. Getting up and going to bed are two critical incidents that normally indicate the start and end of human schedule and intelligent service as well. Besides, sleep pattern can reflect the health condition of a person [1, 2]. It is possible to unite sleep pattern with other characteristics such as age, gender, and working time to conclude subhealth. Furthermore, applications could give customized suggestions based on these conclusions. “Pzizz” [3] helps people fall asleep and “Sleep Cycle” [4] offers waking up service. Both these “lullaby” and alarm clock apps strictly depend on manual activation and detailed settings. It is meaningful to liberate users from repeated mechanical operations and make services more natural and flexible.
Smartphone, integrated with multiple sensors and chips, becomes a primary user data acquisition platform owing to tight connection with people. Many researchers have made researches and applications based on usage records from smartphones for modelling user profile. Reference [5] applied supervised learning methods to infer user profiles, such as religion, relationship status, spoken languages, countries of interest, and whether the user is a parent of small children, by observing only a single snapshot of installed apps. Experiments on 200 smartphones show that for most traits the model can achieve over 90% precision. Reference [6] focused on analysing user profiles by mining usage information from multiple NFC-based applications, both on smartphone and server. Reference [7] firstly attempted to understand users through Service Set IDentifier (SSID) information. It also proposed a data cleaning and information enrichment framework for enabling user preference understanding based on the collected Wi-Fi logs. Reference [8] developed a method to predict the next app which improves home screen apps’ usage experience.
This paper aims at solving two tasks. The first one is to detect getting up and going to bed incidents. The second one is to predict the next wake time and bedtime.
The issue of sleep study is put forward by medical care institutes at first. Hospital apply Polysomnography (PSG) [9, 10] to diagnose sleep disorders by records brains waves, oxygen level in blood, heart rate, and breathing, as well as eye and leg movements of participants. However, this method requires professional devices and specialized sleep environment, which is infeasible for daily use.
Wearable devices are the second choice. Devices are worn on the wrist, used to record movements and estimate sleep parameters which are called wrist actigraph (ACT) [11]. Although actigraph has been doubt the validity of identifying wake or sleep from a medical point of view, wearable device with fine identification algorithm is still fit for daily use owing to its low cost and portability. Reference [12] proposed using a wrist actigraph to estimate sleep time at an early stage. Reference [13] later developed a monitor system based on it and replacing manual calculation by scoring algorithms. And [14] further improved it, making the algorithms distinguish sleep from wakefulness in approximately 88% of minutes scored. In recent years, development of integrated circuit (IC) and Internet of things (IoT) enables miniaturization of sensors. Activity trackers like Fitbit wristband and smartwatches like Apple Watch make it possible to collect and analyse wrist movement data of users. Reference [15] employed an artificial neural network to classify sleep and wakefulness based on night wrist actigraph data. Reference [16] integrated unsupervised machine learning algorithms and domain knowledge heuristics to detect sleep or wake status.
Without extra device, there are also methods that utilize smartphone as a monitor and upload data to calculate sleep durations. Reference [17] presented a model named BES inferring sleep duration by analysing smartphone usage patterns. The features include external brightness, phone usage, motion status, and background noise. The collection of these attributes requires a resident monitoring application on smartphones. Reference [18] designed and implemented an application named Toss “N” Turn that can infer sleep and sleep quality by captured sensor data including sound amplitude, acceleration, light intensity and screen proximity, running apps, battery, and screen status. This paper verifies its validity by notify users to enter ground truth every morning.
However, there are three limitations of these methods listed above:(1)Although wearable devices have developed rapidly in recent years, the market penetration rate is still limited. Worldwide shipments of wearable devices are 25.1 million, while there are 344.4 million smartphones (source: International Data Corporation, IDC). In addition, different manufactures would apply different types of sensors, and there is no unified API for developing software for all of them. This solution evidently limits intelligent services’ coverage rate.(2)Most of current methods demand transmitting data to web server and executing algorithms on server cluster. That would be involved in privacy protection issue. Uploading privacy data without users’ explicit permission would cause severe punishment by law in many countries. General Data Protection Regulation (GDPR) regulated that data transfer requires user’s consent in European Union. Besides, claiming various large amount data collection would cause disgust of users.(3)The computational resources of smartphone are strictly limited. Massive and heavy computation would cause rapid loss of power, high occupancy of memory and kernel. Each of these factors would burden the running time of device and lead to decrease of user experience. Furthermore, computing environment on smartphone does not support machine learning especially deep learning algorithms very well.
For the second task, to the best of our knowledge, previous research is rare to be find. Literatures retrieved by keywords like “sleep time prediction” always focused on how to detect user’s wake/sleep status or calculate sleep duration. In this paper, we classify them as solutions to task one, detection. Since intelligent services based on user profile and usage scenario just become popular recently, lack of study on predicting wake time and bedtime is reasonable, for example, Google released Awareness API on I/O developer conference, 2016 [19]. Awareness API focuses on enabling context-aware services by mining user habits. Alarm clock would not disturb your sleep if it learned when you fall asleep last night and next meeting. BTP takes a step forward, attempt to predict user’s future action which is a valuable and novel work.
The contributions of our work are as follows:(1)A lightweight predicting algorithm: BTP simply utilize data collected from smartphone without additional wearable devices. Screen status log data could be retrieved from smartphone operating system because screen light up and extinguish event are standardized system broadcasts. Besides, thanks to the ubiquity of smartphone, BTP can cover the widest potential user.(2)Avoid the dilemma of collecting data without breaking privacy protection regulations. BTP is also not involved with web service or cloud computing. In other words, BTP plays a role of data processor rather than controller. Users would not worry about the risk of personal information disclosure.(3)BTP consume limited resource on smartphone. It does not need special framework or extra library. The calculation would not cause any appreciable delay. By our experiments, BTP performs acceptable complexity.
2. Ideas
2.1. Difficulties
Detection and prediction process of BTP may be crushed by many incidents. It is not enough to use screen status log data only. In actual projects and researches, we met the following difficulties.
The first is abnormal behaviour. Someone lives regularly, but back home very late or stay out all night occasionally. This case should be determined as outlier in data analysis theory. BTP contains models based on pattern knowledge, so these outliers must be detected and abandoned. Figure 1 demonstrates a typical outlier.
The second is unexpected interruption. When people are sleeping, incoming calls, receiving messages, and notifications from applications may light up screen, ring, or vibrate and then cause waking up. It is also possible that people fall into deep sleep and ignore these events. But for screen status log data, a continues screen extinguish period is interrupted by a strange event. The screen states of an unexpected interruption case is shown in Figure 2.
The third is shutting down of smartphone. If smartphone is shut down, both screen status and auxiliary events are missing. Some people do not like flight mode, they shut down smartphone at bedtime and start up it when getting up. But a worse case is, user may forget to charge the battery and then causing smartphone power off automatically. Losing log data is the biggest threaten and risk of BTP. Figure 3 shows the screen states of a shutting down case.
The above three difficulties are mentioned with detection, and the fourth is about prediction. Because of different life experience, regulations of wake time and bedtime between people are customized.
Most of people works at office or factory, that is, a stable target location. They show an identifiable wake-up time and bedtime distribution during weekdays. But many of them are freestyle on weekends. Others do not work from Monday to Friday. Because of their occupational characteristics, they exhibit strange distributions, such as going to work every two days or three days’ work and one day’s rest. That leads to big error in predicting with simple time series models or trying to describe these patterns by weekday and weekend frame. Figure 4 shows the patterns of weekdays and weedends.
2.2. Solutions
Besides difficulties, we can utilize two advantages to fix and improve our model. One is alarm clock event. For most people, they go to work at weekdays. Alarm clock ringing moment is also wake time. This rule has very high confidence and we can check accelerator activated event to confirm that. Figure 5 shows screen states when alarm clock event occures.
The other is accelerator activated event. When accelerator is activated, we know that smartphone is moved and the user is awakened. Figure 6 shows screen states when accelerator activated.
We overcome and reduce the effect of the four difficulties mentioned above by the following solutions.
(1) Abnormal Behaviour. BTP supposes that people sleep at home. It abandoned sequences not at home to avoid the impact of abnormal behaviour. Figure 7 shows screen states after abandoning abnormal behaviours.
(2) Unexpected Interruption. Sleep disturbances are resolved by merging adjacent periods of screen extinguish status. Here we define central sleep point, about 3 o’clock, the sleepiest time of the day. When people are waked by interruption event near central sleep point, they are likely to remain sleep in a few minutes. But when they are less sleepy, such as 5:00, people may get up directly. Figure 8 shows the case about connections two adjacant periods.
(3) Shutting Down of Smartphone. For the users who switch on and off regularly and reasonably, BTP regard the power on and off events as wake time and bedtime. But we treat unregular power off event as outlier and abandon it. Figure 9 shows screen states when power off event occured.
(4) Customized Regulations between People. We used to think that user behaviour prediction should be based on weekday and weekend frame, but it is confused by customized weekday regulations, resulting in greater error. Then we changed our strategy and matched the current screen status sequence with other sequences of each past day. The most similar sequences represent the same behaviour pattern. Since waking up and going to bed are beginning and end of whole day activities, they are naturally reflected in this similar pattern system. Figure 10 shows two similar behaviour patterns.
To sum up, we effectively improving the accuracy and availability of screen status sequence in detecting and predicting wake-up time and bedtime by introducing a series of auxiliary events. The main process of BTP is shown in Figure 11. BTP firstly collects the user’s screen status log data and other auxiliary events and then searches and connect main screen extinguish periods to get daily detection result. Then prediction is operated by matching current screen status sequence with the historical sequences like K-Nearest Neighbour.
3. Methods
3.1. Data Structure
Firstly, we define data structure. Log data contains information about both screen status and auxiliary factor. An event consists of date, moment, and event. Here is an example of JSON string. Date: 2018-09-01, Moment: 22:39:02, Event: SON .
Different events are distinguished by event codes. Here the code SON represents screen light up. All the event codes and corresponding meanings are shown in Table 1.
For task one, detection of wake time and bedtime, we input log dataset with all types of events from a smartphone throughout many days. Here is part of it. … Date: 2018-09-01,Moment: 22:39:02,Event: SON, //Screen light on; Date: 2018-09-01,Moment: 22:45:02,Event: SOFF, //Scree extinguish; Date: 2018-09-01,Moment: 22:49:50,Event: AA, //Pick out phone; Date: 2018-09-01,Moment: 22:49:55,Event: SON, //Unlock phone; Date: 2018-09-01,Moment: 22:50:01,Event: SOFF, //Screen extinguish; Date: 2018-09-01,Moment:22:50:02,Event: POFF //Phone shutdown. ….
The output of detection is as follows: … Date: 2018-09-01,Bedtime: 22:39:02,Wake time: 6:29:59, Date: 2018-09-02,Bedtime: 23:12:45,Wake time: 6:40:04, Date: 2018-09-03,Bedtime: 22:59:12,Wake time: 6:30:09, ….
The human sleep pattern follows cycles of days generally. When discussing a single day, we must define the beginning of it. We set border point between two days at 12:00 instead of 0:00. Then we avoid interrupting continues sleeping that is also a continues screen extinguish period in our model.
For task two, prediction of wake time and bedtime, we also input log dataset as above. The output of prediction is as follows, if current time is “2018-09-10 18:00:00”, Date: 2018-09-10,Bedtime: 22:40:15,Wake time: 6:30:00.
Here wake time is referring to “2018-09-11 6:30:30” actually.
3.2. Data Collection and Preprocessing
Before running detection and prediction process, we must transform log data into several binary sequences. Considering that there are 1440 minutes in a day, we separate different events into six 1440-length sequences. For screen status, 0 represents screen extinguish and 1 represents screen light up. The sequences are shown in Table 2. For screen status and power status, we sampled the first second status of a minute. Yet for the other sequence, if there is any event occurred in the minute, the status will be 1.
Figure 12 demonstrates a timeline of a volunteer.
3.3. Method Process
BTP generally has two main phases, detection and prediction. Figure 13 shows the floatchart of detection.
In detection part, BTP follow four steps for each day to be detected.(1)Filter out period of screen extinguish status when user is not home by searching home status sequence.(2)Search screen status sequence and find out continues screen extinguish period which last over 30 minutes, then sort them in timeline.(3)For each period selected by step 2, BTP uses (1) to determine whether this period can be united with its adjacent periods; that is, if time interval between them is shorter than , we will conjunct the two adjacent sequences as one. In formula 2, is a threshold that determine whether to join two sequences, is used to adjust absolute value, is minutes from 12:00 to middle of period, is minutes from 12:00 to the time that people have deepest sleep, and is an offset value.(4)Adjust result with sequences of interruption status, alarm clock status, power status, and motion status.
Table 3 shows the pseudocode of detection.
After getting wake time and bedtime, as shown in Figure 14, predicting part has following steps:(1)Collect screen status sequence of today (from 0:00 to current time).(2)Calculate similarities between today’s screen sequence and each historical sequence using (2). Here, is screen status sequence from 0:00 to current time, is the sequence to be compared, and is minutes from 0:00 to current time.(3)Select detection result according to the most similar sequences and use (3) to generate result. is prediction result, is the sequence of the most similar sequences, and is previous detection result.
Table 4 shows the pseudocode of prediction.
4. Experimental Result
4.1. Experimental Background
The measurement criteria for detection is Mean Absolute Error (MAE), shown in (4). Here, is number of people involved in this experiment, is true value, and is the detected value of methods to be tested.The measurement criteria for prediction are RMSE, which are shown in (5). Here is number of samples. represents true value, and is the predicted value.Our dataset contains 30 volunteers’ data. They installed the same data acquisition application we distributed on their smartphones. The application has four missions: collecting log data, detecting both wake time and bedtime, uploading original data and result to server, and fetching feedback. Here we upload data for debug and sign agreement with volunteers. The client app is resident in android and collects log data of system day and night. At the end of a day, usually around 0:00, it starts our detection method to detect when the user wakes up and goes to bed. It also regularly displays questionnaires and returns feedback to server. These questionnaires only have a simple question: When did you fall asleep last night and wake up today? To ensure that the volunteers answer questions carefully, some feedback which is always confused with prediction value is filtered after checking their original log data. After completing 13 days’ continuous test, 39 volunteers accomplished experiments. But nine of them were filtered out for careless answers. During the experiment, we ran the phase one of BTP to detect wake time and bedtime and ran BES, proposed by Chen et.al. [17], to detect sleep duration. Then, we ran the phase two of BTP, moving average (MA) [20], and exponential smoothing (ES) [20] to predict future wake time and bedtime on the left 30 samples with 13 days records and then calculate RMSE.
All experiments were performed on volunteers’ Huawei P10 smartphones with EMUI 8.0 (Android 8.0), Kirin 960 (4×2.4GHz+4×1.8GHz), 4GB RAM, and 64GB ROM. We used Android Studio 2.3.3 with Android SDK 7.0 and Java 1.8 to develop the data acquisition application, which collects and uploads data to server regularly. The server for receiving data runs a CentOS 6.5 environment coupled with Intel(R) Xeon(R) CPU 4×2.5GHz, 64GB RAM, 1200GB HD@15000rpm. To compare BTP and other algorithms, they were both executed on a workstation of Think Station P300 with Windows10 64bit, Intel i7@4×3.6GHz, 16GB RAM@1600MHz, SATA3 hard disc 2TB@7200rpm.
4.2. Result of Detection
4.2.1. Comparison
The BES algorithm we implemented has a MAE of 48.19 rather than 42.5 in Chen’s paper because of different datasets. The comparison result is shown in Table 5. BTP has a much lower MAE than BES on the same practical dataset.
Table 6 further presents RMSE, Maximum Absolute Error (), Minimum Absolute Error (), and MAE of the detection part of BTP for showing its reliability. Since alarm clock rings regularly in weekday, wake time is more accurate than bedtime.
4.2.2. Parameter Discussion
There are three parameters in Eq. (1), , , and . We use control variate method to carry out parameter tuning by using MAE as evolution metric. Table 7 shows that the best is around 900. That means 3:00 is really the central point of people’s sleeping.
In Tables 8 and 9, we found that the best choice is and . The two parameters together decide how far we should join two adjacent screens extinguish period. A too small length will bring irrelevant periods and too big length will also make sleep duration over splitting.
4.2.3. Time Complexity
From Table 10, we prove that BTP and BES are both following a linear complexity. They are fast so we repeated them to record their time consuming. BTP does not need regression as BES but must endure heavier I/O consuming.
4.3. Result of Prediction
4.3.1. Comparison
We compare BTP with Moving Average (MA) and Exponential Smoothing (ES). MA and ES are two classic algorithms for time series prediction. The result is shown in Table 11. BTP outperforms MA and ES in all metrics because wake time and bedtime have periodicity and the cycles vary from person to person.
4.3.2. Parameter Discussion
We applied control variate method to identify the influences brought by different parameters in in (2) and in (3). Table 12 illustrates that, with the incensement of , the result is better. It proves that assembly learning takes advantage of robustness. But we only have records of 13 days; is limited.
Table 13 shows that accuracy increases with longer compared sequence. Here, ; the 75% length of a day strikes balance between accuracy and usability.
represents the number of historical days we use for prediction. If we have more historical samples, the model will be more robust. Here may meet an overfitting condition. The result of experiments with different are shown in Table 14.
4.3.3. Time Complexity
The time complexities of BTP, MA, and ES are . In practical, MA and ES would execute faster because the size of data is rather smaller. BTP uses historical screen status sequence for matching similar pattern. Each prediction requires traversing , and represents number of compared days. However, MA and ES only use historical wake time and bedtime. This difference is negligible when running in a smartphone instead of centralized calculation in server.
4.4. Survey
To collect feedback of volunteers, we design a survey for them. The questionnaire only has two questions:(1)Did you fell asleep in [“detected bedtime”] last night?(a)Exactly or deviation is under 15mins (get score 1)(b)Deviation is in 15-30mins (get score 0.5)(c)Deviation is more than 30mins (get score 0)(2)Did you get up in [“detected wake time”] this morning?(a)Exactly or deviation is under 15mins (get score 1)(b)Deviation is in 15-30mins (get score 0.5)(c)Deviation is more than 30mins (get score 0)
Results of our survey are shown in Table 15. It is quite relevant to MAE of algorithms.
5. Conclusions
This paper presented a bedtime predicting algorithm named BTP. The goal of BTP is to precisely detect and predict wake time and bedtime of a person by mining screen status log of smartphone. The implement of BTP is independent of any actigraph or cloud infrastructure, which makes it fit for popular application and satisfy privacy protection. Experiments on 13 days screen log data of 30 persons indicated that BTP can effectively accomplish detection and prediction tasks without introducing extra devices or uploading data. We also compared it with BES, another sleep detection algorithm, and BTP exceed BES in accuracy and time consuming. Furthermore, our work shows that BTP could be applied as a critical context-aware component to improve intelligent service on smartphone.
Data Availability
Experiments in this paper involve practical datasets. The practical dataset belongs to certain enterprise and is limited to science research and protected by confidentiality agreement. But researchers can develop their own app to record people’s behaviour to test their work.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is supported by the Projects on International Scientific and Technological Cooperation under the National Key R&D Program (NKP) no. 2016YFE0204500, the National Natural Science Foundation of China under Grants no. 61671081 and no. 61720106007, the Beijing Natural Science Foundation under Grant no. 4172042, and the Fundamental Research Funds for the Central Universities, Beijing University of Posts and Telecommunications 2017 Education and Teaching Reform Project no. 2017JY31.