Abstract
Smart environments, such as smart cities and streets, contain various heterogeneous devices and content that provide information to users and interact with each other. In a smart environment, appropriate content should be provided based on the situations of users. Additionally, when a user is in motion, it is necessary to provide content in a seamless manner without interruption. A method for systematically controlling the delivery of such content is required. Therefore, we propose a content service platform to meet the needs discussed above. The content service platform supports the delivery of content and events between different devices, as well as the control of content. Context-aware technology can also be applied to support customized content. In this paper, we present an architectural model, a contextual reasoning process, and case study on applying content service platform to a smart street environment. The proposed content service platform applied as a base model to support the provision of user-specific content in smart environments.
1. Introduction
Smart environments, such as smart cities and streets [1–4], are rapidly emerging due to popularization of smart devices, such as smart phones and tablets, as well as the development of IoT [5, 6], cloud [7, 8], and networking technologies [9].
In a smart environment, various contents must be delivered to users through natural interactions between the devices that constitute the smart environment, without being recognized by the users. For example, in the case of a smart street, there are fixed devices capable of computing and networking, such as kiosks and digital signage. When a user enters a smart street area, it is necessary to provide them with content that is specific to the smart street environment, such as information, entertainment, and advertisement, through interactions between the user’s mobile device and a fixed device.
Various studies on architecture and digital signage [10–12] for configuring and managing smart environments have been performed. Additional research on the configuration of smart environments is currently underway, but many of the studies being conducted are dependent on specific domains, such as smart homes and cities [1–4, 13].
In a smart environment, it is essential to deliver valuable service content to users. Various technologies are required to meet this need. Events and content must be transferred seamlessly between the devices in the smart environment and the mobile devices carried by users. Additionally, in a smart environment, users can move between various locations or remain in a fixed location. It is necessary to provide appropriate content services based on user tendencies, with consideration for changes in user locations, as well as the current time and surrounding environment.
Therefore, we propose a content service platform that can provide users with seamless content services, even when they leave the vicinity of the device that is providing the content in a smart environment, and recommend user-oriented content services that are suitable for the surrounding environment.
2. Basic Concepts
2.1. Web Sockets
The protocol for web sockets has been standardized by the Internet Engineering Task Force (IETF) [14] and the accompanying API has been standardized by the World Wide Web Consortium (W3C). A single TCP (Transfer Control Protocol) port is opened between the web server and web browser. It supports full-duplex communication. In this paper, we apply web socket technology to support interoperability and maintain high levels of communication performance by reducing the communication overhead when multiple content requests are generated in a short timeframe.
2.2. Ontology
OWL (Web Ontology Language) [15] is used to express computer-interpretable knowledge. The essential components of OWL are the class, attribute, and individual. A class is a unit that abstracts objects with similar characteristics. Attributes represent the relationships between the class objects that comprise an ontology. An individual is a physical representation of a physical or conceptual class entity. Additionally, the Semantic Web Rule Language (SWRL) [16] was proposed by W3C as a reasoning language to be used in conjunction with OWL. In this study, OWL and SWRL are utilized for our content recommendation model and reasoning in the content service platform, respectively.
2.3. Knowledge-Based Filtering
Knowledge-based filtering [17] techniques are used to apply the knowledge of users for recommending candidates. In other words, they are techniques that compare the details of each user’s knowledge regarding potential candidates in order to find the candidates that best meet user needs. In this study, we apply knowledge-based filtering concepts in order to recommend user-specific content service.
3. Related Work
Fernández-Montes et al. [11] presented a reference architecture for coordinating various resources in a smart environment. The first task that is necessary in a smart environment is to recognize environmental conditions (temperature changes, user detection, etc.). The proposed architecture defines collector, verifier, repairer, filter, and ontologizer phases for the recognition process. However, a high-level reference architecture for smart environments is not addressed.
Towle and Quinn [18] explicitly stated that, by modeling the information regarding user needs and recommendation items, it is possible to clarify the causal relationships between recommendations and improve accuracy. This knowledge-based filtering method was applied to book and electronic product recommendation. In order to explain the concepts of knowledge-based filtering, Burke [17] used a system that recommends restaurants by collecting attributes, such as food prices, styles, and moods, as well as a recommender.com service that uses values collected from users to recommend movies and music as examples.
She et al. [19] is a system that enables a user to use a smart device to send a digital signature to a wireless router via a nearby router. Such a system enables content to be provided to multiple smart devices. Kim et al. [20] proposed a system in which a digital signage system identifies a user and delivers content directly to the user’s mobile device.
4. Content Service Platform
In this paper, we propose a content service platform for seamlessly providing content services that are tailored to the needs of various users and change based on time and location within a smart environment. The content service platform is designed to provide user-specific content by using user information (age, gender, interests, and preferences) and supporting various interfaces and cross-platform capabilities.
4.1. Requirements and Architecture
The requirements for the content service platform proposed in this paper are defined in Table 1.
Based on the requirements outlined in Table 1, the content service platform architecture is presented in Figure 1.
The architectural components are described below. The Interface Manager layer consists of a WebSocket Interface module and a Session Manager module. The Interface Manager layer facilitates networking between user mobile devices, the sensors constituting the smart environment, various other devices, and the content platform based on the web socket protocol. The WebSocket Interface module provides a mechanism for devices and content platforms to establish direct connections, send and receive messages, and terminate connections through the web socket protocol. The Session Manager module manages web socket session information, which changes with every new connection based on the ID of the device stored in the integrated database of the content platform.
The Transfer layer consists of an Event Manager module and a Content Delivery module. The Event Manager is a content service platform in which events related to control, operation, etc. are defined. It identifies the type of a message and processes an event for the message. The message event includes a content control operation from a user’s mobile device, a content transmission event, and a content-use history event from a user’s mobile or digital device. The Content Caster accesses a resource when a request for content occurs and delivers the URL for the requested content, as well as any necessary additional information.
The Resource layer serves as a bridge between the Integrated DB layer and the other layers. It is comprised of a Content Manager module, Device Manager module, Member Manager module, and System Log Manager module. Each module accesses and operates on the integrated database directly when a request is made to read, create, modify, or delete content, devices, members, or system log information.
The Integrated DB layer contains a database that stores information about content, devices, users, and system logs.
The Situation-Aware Engine layer is comprised of six modules and infers a user’s situation to recommend content based on ontology information and reasoning rules. The File Loader module reads existing ontology files and verifies that there are no grammatical errors. The Ontology Repository module loads ontologies into memory. The Ontology Manager module reads and updates loaded ontologies and queries inference rules. The Context Filter module extracts context elements that contain contextual information from event data that is transmitted from sensors or devices in the smart environment. The Reasoning Engine module infers user situations by using ontology information and inference rules. The Content Recommendation Manager module uses the inferred situations and a knowledge-based filtering process based on user profiles to infer user content preferences and recommend the most appropriate to the user.
4.2. Message and Content Floating in the Content Service Platform
This section describes the approach that we use to provide seamless content to users, even when they move away from a content-providing device in the smart environment of the content service platform. Figure 2 presents the concepts of message and content floating.
Content floating means delivering content to a smart environment device through a web socket or receiving content from a user mobile terminal when a user is accessing content through a smart environment device and then leaving the area.
Event floating refers to controlling a smart environment device through a mobile terminal in the smart environment by using an existing touch screen. In order to control the smart environment device, the user transmits an event to the content service platform from a mobile terminal in order to execute the event input on the corresponding smart environment device.
Using this method, when a user mobile terminal is identified in a smart environment, it is connected through a web socket and given a unique session. In the content service platform, a session for the corresponding smart environment device is created and mapped to the user’s session. When data is transmitted from the content service platform to a smart environment device or user terminal, the data is delivered in the form of a JSON structure and transmitted as parsed data.
The user mobile device that is accessing content through the smart environment device transmits beacon information and a user ID to the content platform in order to create a mapping between the sessions on the mobile device and smart environment device based on the ID. When the mapping is successful, the controller transmits the result to the mobile device to allow the user to control the content on the smart environment device.
In order to identify the different types of messages in the content service platform, several message codes are defined. Based on the defined message codes, the functions of user identification (beacon identification), content plotting, and event plotting can be performed. Table 2 contains the defined message types.
User identification (beacon) messages enable web socket session mapping between smart environment devices and user terminals in the same zone by using the beacon information (UUID, MajorID, and MinorID) received from a user terminal. Content floating messages transfer the content received from a smart environment device to a user terminal or deliver the content received from a user terminal to a smart environment device. Event floating messages convey input events for manipulating a smart environment device through a user terminal.
4.3. Personalized Content Recommendation Based on User Preferences
Figure 3 illustrates the contextual reasoning process used in the content platform. After an ontology is loaded onto the memory, when the data related to an event occurrence is transmitted from a user’s mobile device to the content platform, a context factor is extracted from the corresponding data. The context factor is an element that has a significant effect on the user’s situation and the data associated with the context factor is defined as context information.
Context information is updated for the individual and for the ontology that is loaded into the content platform through a process of checking which context information is related to which class or attribute in the ontology. The user’s profile in the database is queried and the user’s preference for each type of content is calculated by using the knowledge-based filtering process in conjunction with the inferred situation. Once the preference calculation process is complete, the content with the highest preference rating is delivered to the user and the user decides whether or not to consume the recommended content.
The ontology meta-model of the smart environment is presented in Figure 4.
We identified device, membership, location, history, and content as the major superclasses. We calculate the user preference for a type of content based on the situation, which is inferred from the ontology and the user profile stored in the system. In order to determine the weight for each context element, which indicates the effect that each context element has on the user’s content selection, we conducted a survey and regression analysis of the potential users of the smart environment.
The following formula presents the equation used for calculating user content preference:where I is matching context elements with user profiles; P (preference) is user preference for single content; K is context element count; IK is preference for Kth context element; and WK is weight of Kth context element.
P denotes a user’s preference for a single type of content. In order to calculate this preference value, the content provider first queries values from the context element set for a target user when registering the content in the initial content platform. The preference rating for each context element can then be obtained by determining if the user context and profile information match (I) and multiplying the result by the weight () of each context element. Finally, if all the preferences for content types are added to each of the context elements, the user’s preferences for specific content can be calculated.
5. Case Study and Evaluation
The proposed content service platform is applied to a smart street environment that supports interaction between digital signage and the mobile devices of users on the street.
Figure 5 illustrates a scenario where users are given custom content recommendations by the content service platform of a smart street environment.
Based on the meta-model presented in Figure 4, we designed an ontology for the smart street environment. Table 3 presents the structure that maps elements of the smart street into an ontology based on the proposed ontology model.
We also defined situation and reasoning rules, as shown in Table 4, in order to infer user situations based on the ontology.
Figure 6 presents the system log for the operating process of the scenario and content platform from the reception of a user event to the recommendation of content. The user recognizes the information (UUID, MajorID, and MinorID) of the beacon located in the BEXCO and BC0001, which is the ID of the beacon in the content platform, is included in a message along with the current system time, which is transmitted to the context element extraction step. Only the context element data that affects the user’s current situation is extracted and the results are shown in the system log as Filtered Context Info. Figure 7 shows that the user visited BEXCO in the ontology by updating the domain ontology with the extracted data from the context element.
Figure 8 represents the inference of a visitConventionCenter situation, indicating that the user is in the situation defined in the ontology. In the system log, we can see which part of the ontology data is inserted into a SWRL rule and examine how reasoning results are derived.
In Figure 9, the content service platform first queries the weights of each context element corresponding to the location of the ConventionCenter in order to obtain the user’s preferences for content. Next, the profile information of the user and the target user information of the content are retrieved from the database and the two pieces of information are compared with each other. The content weighting result can be obtained from the content preference result, which is contained in the ContentPreferenceMap of the system log. The baseball game (Baseball Game), which had the highest preference rating for this user, is recommended.
Figure 10 illustrates the display screen for the baseball game content that is recommended to the user.
In order to verify the content recommendation accuracy of the content service platform, an experiment was performed in Busan Centum City and a total of 63 content recommendations were provided. 54 instances of real user-selected content made it to the top three recommendations, while nine instances did not. This means that 85.7% of the content that was actually selected by users was included in the top three recommendations. The details of the implemented content service platform with respect to smart street performance are as follows:(i)Content service platform engine processing time: <300 ms–TTA (Telecommunications Technology Association) [21] Certified Certification Test Report(ii)Number of simultaneous sessions on the content service platform: 2,000—TTA Official Test Report(iii)Content service platform reasoning accuracy: 90%(iv)Content service platform inference time: 50 ms. (v) Content service platform API Cross Platform Support Count: 15
6. Discussion
Another factor to be considered in smart environments, such as smart cities and smart streets, is the security of individual user’ private data. For example, in the EU, a regulation to protect personal user information, called the General Data Protection Regulation (GDPR) [22], has been in effect since May 2018. The GDPR strengthens the concept of the protection of personal information by mandating personal information impact assessments of personal information handlers who process personal information and protect personal information by default. It also recommends deidentification measures and encryption procedures to personal information handlers. In ISO/IEC JTC 1/SC 27/WG 5 [23], international standards related to personal information are currently being developed. This group has developed the Privacy Framework (ISO/IEC 29100) [24], the Privacy Impact Assessment (ISO/IEC 29134) [25], the Rules for the Protection of Personal Data (ISO/IEC 29151) [26], the Requirements and Directives related to the Privacy Framework (ISO/IEC 277010) [27], and international standards such as user-friendly online notice and consent (ISO/IEC 29184) [28], current smart city privacy guidelines (ISO/IEC 27570) [29], and a framework for establishing personal information deletion procedures (ISO/IEC 27555) [30]. These standards are under continuous development.
Therefore, the content service platform proposed in this paper also needs to apply a mechanism for protecting personal information in the future. In the case of Korea, service providers in national pilot smart cities, such as Sejong and Busan, delete or replace all or part of personal information in accordance with Article 37 of the privacy laws and anonymize a specific individual so that they can no longer be identified, even when combined with other information. Techniques that can be applied to perform anonymization include K-anonymity, L-diversity, and T-closeness [31].
The proposed content service platform was tested for applicability and operational feasibility by targeting users who received prior consent for the use of personal information from individuals (information subjects). Additionally, to protect personal information in a smart environment by reflecting the issues discussed in the smart city security model [32] of the domestic Ministry of Science and ICT, and the Korea Internet and Security Agency, the items that are currently included and improved as requirements of the proposed content service platform are summarized as follows:(i)Implementation of a user-friendly personal information consent interface changes the procedure and interface required to obtain consent for personal information in a user-friendly environment(ii)Monitoring and alarm reflect monitoring and alarm functions to notify and warn the information subjects periodically that personal information is being collected(iii)Anonymization of personal information introduces anonymization processes, such as anonymization techniques such as K-anonymity, L-diversity, T-closeness, data masking, and total processing(iv)Handling of personal information deletion introduces a function to delete personal information in response to a request for the deletion of personal information from a data subject(v)Network security function refers to networks in which services are provided according to network separation operated in logically/physically separated blocks and set up to operate in a secure network environment that blocks direct connections from the outside by configuring an isolation zone for external services
7. Conclusions and Future Work
We proposed a content service platform to provide content to users in smart environments. We identified the key requirements for providing content services in a smart environment and proposed an architecture to meet those requirements. Additionally, even if a device moves out of the smart environment, the content is provided seamlessly. A structure for controlling content through user terminals was also presented. Finally, we presented a mechanism for recommending content based on smart environment situations and user tendencies through an ontology. We verified the validity of the proposed mechanism by using it in a smart street in smart environment and evaluating the performance of the content service platform. Future research will apply the content service platform to various smart domains. We will also study how to intelligently recommend content based on the mining of historical data.
Data Availability
The data used to support the findings of this study are available from the authors upon reasonable request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this study.
Acknowledgments
This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) Support Program (IITP-2021-2020-0-01797) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation).