Abstract
There are millions of base stations distributed across China, each containing many support devices and monitoring sensors. Conventional base station management systems tend to be hosted in the cloud, but cloud-based systems are difficult to reprogram and performing tasks in real-time is sometimes problematic, for example, sounding a combination of alarms or executing linked tasks. To overcome these drawbacks, we propose a hybrid edge-cloud IoT base station system, called BSIS. This paper includes a theoretical mathematical model that demonstrates the dynamic characteristics of BSIS along with a formulation for implementing BSIS in practice. Embedded programmable logic controllers serve as the edge nodes; a dynamic programming method creates a seamless integration between the edge nodes and the cloud. The paper concludes with a series of comprehensive analyses on scalability, responsiveness, and reliability. These analyses indicate a possible 60% reduction in the number of alarms, an edge response time of less than 0.1s, and an average downtime ratio of 0.66%.
1. Introduction
A base station is an information exchange center for the smartphones within its coverage area. These networks of base stations are the backbone of a mobile network and, for many, that means the backbone of one’s work, life, or social sphere. A defect in any one of those base stations can mean great inconvenience for thousands of users. Typically, a base station consists of numerous devices that cooperate to ensure the base station’s reliability, while countless sensors linked to those devices constantly assess the surrounding environment. If even one parameter value exceeds its threshold, alarms begin to sound. The incident is uploaded to the cloud server as a maintenance request and, once the validity of the alarm is confirmed, maintenance personnel are called to respond to the request.
Advances in cloud-based computing and storage have contributed to the thriving success of centralized systems. Yet, no matter how advanced, the reality of managing millions of base stations and monitoring hundreds of parameters for each and every one brings some stark practical problems to the fore.(1)It is beyond the ability of any maintenance team to respond to every alarm or even respond to all priority alarms in time. In fact, many alarms are simply ignored.(2)Due to the low response times inherent to cloud-based platforms, some urgent and real-time tasks must be managed by edge nodes. Executing an action at the instant an alarm sounds is one such example.(3)The edge nodes and the central platform are not integrated seamlessly. Further, even though edge-node software can be updated remotely in some platforms, such updates can only occur en masse, and the fundamental program can only be updated not replaced entirely. However, the need to reprogram the tasks an edge node can perform is increasing, and every edge node may not necessarily need to perform the same task. Combined alarms that are based on just a few relevant parameters are a good example, as these signals could be monitored by a small proportion of the network’s nodes.
To address these problems, we propose a new architecture for base station management that combines edge and cloud computing, called BISI, which is presented in this paper. BISI(1)reduces the incidence of unnecessary alarms and improves overall system reliability(2)strengthens edge node capability to form a seamless edge-cloud system(3)increases the scalability of the system and the dynamism of the edge node’s functions
Edge computing has the advantages of reduced latency, less and lower traffic peaks, and longer node lifetimes [1, 2]. Hence, with an edge-cloud computing system, more functions can be shifted from the cloud to the edge nodes, especially real-time tasks, and those nodes can be reprogrammed dynamically. Moreover, the interactions between the cloud and the edge can be optimized.
Hence, the contributions of our study are summarized below.(1)This paper presents a theoretical mathematical model with dynamic characteristics for an IoT base station management system. Abstract definitions are provided for the cloud and edge components, the interactions between the cloud and the edge nodes, and the interactions between the edge nodes and the devices.(2)We explain how BSIS might be implemented in practice. In this paper, we use embedded programmable logic controllers (ePLCs) as the edge nodes with a three-layer software structure and a local database. The ePLCs comply with IEC 61131-3 standards [3]. With support from the cloud, the edge nodes can be programmed dynamically, can be reprogrammed, and can perform real-time tasks, such as linked tasks, ringing a combination of alarms, or filtering alarms. Therefore, the edge nodes are scalable and different edge nodes can run different numbers and types of tasks.(3)Comprehensive analyses of BISI verify its scalability, response times, and reliability and demonstrate the advances an edge-cloud system can make in base station management.
The remainder of this paper is structured as follows: Section 2 introduces the related works; Section 3 presents the architecture of the BSIS; Section 4 provides the scalability, response time, and reliability analyses; and the last section concludes our work.
2. Related Works
With IoT platforms, good application performance is heavily dependent on the system architecture. Given the millions of edge nodes in base station networks, appropriate system architecture must be chosen carefully and thoughtfully. In our context, this consideration is even more critical if BSIS is to deliver a responsive, reliable, and scalable system, as the related studies on three key aspects of BSIS illustrate below.
2.1. System Structure
Cloud platforms have become one of the mainstays in computing due to their ability to deliver elastic computing power and storage to satisfy the needs of resource-constrained end-user devices [4]. However, edge computing has its own advantages, such as reducing latency, avoiding traffic peaks, and extending the life of an edge node [1, 2]. Hence, combining edge computing with a cloud platform was inevitable and has given rise to a new paradigm [5]. Ever since researchers have sought ways to efficiently distribute computationally intensive tasks to leverage the respective advantages of each, many studies have proposed a resource management approach that focuses on a particular function, e.g., admission control, computational resource allocation, and power control [6–8]. Fu et al. [9] illustrated secure data storage and searching method to improve the efficiency and security of data storage and retrieval. Chekired et al. [10] proposed a two-priority queuing model to schedule and analyze industrial data. Suganuma et al. [11] proposed a flexible and advanced system model.
Moreover, edge-based cloud platforms have found applications in a variety of domains. For example, Jia et al. [12] introduced a manhole cover management system, which boasts good response times. Xu et al. [13] outlined a smart grid system to effectively integrate energy resources with storage devices. Pace et al.’s research [14] focuses on emerging healthcare industries. Smart transportation systems are another hot domain [15, 16].
However, applying the various solutions found in the studies mentioned above to base station management is problematic as few consider the complexity associated with scalability, reliability, and responsiveness.
Some software approaches to implementing IoT systems have also been developed, for example, multiagent systems and service-oriented architecture (SOA). In multiagent systems, every function or entity can be described as an agent and tasks are completed in the cloud through positive or negative interactions between these agents [17–19]. This approach provides a method with which to clearly describe complex systems that contain many functions and entities linked through complex relationships. Perhaps advantageously, BSIS cannot benefit from a multiagent approach because its functions and the types of entities are both limited and unambiguous.
SOA is a discrete functionality that relies on standardized interfaces to form applications. The SOA paradigm offers loose-coupling, reconfigurability, and flexibility, which allows service entities to be composed or orchestrated during runtime to create different system compositions [20]. Thus, the two main characteristics of SOA are autonomy and interoperability [21, 22]. BSIS shares one similar concept, i.e., scalability. However, SOAs cannot generally handle sensitive real-time tasks [21], and the scalability of an SOA is often limited to its upper layers.
2.2. Edge Node
An edge node is any computing or network resource on the path between a data source and the center of the cloud [5]. In the body of research on edge computing, many devices have served as edge nodes, e.g., smartphones [23], routers [24].
PLCs are common in industrial fields [25–29] because they are highly reliable and easy to program [30]. Moreover, they were standardized as early as the 1990s under IEC61131-3 [3]. The PLCopen organization continues the drive for standardization to this day in areas such as motion control [31], TC4 for integration, OPC UA and TC5 for safety, and TC6 for seamless program interaction between platforms.
PLCs can be divided into two rough groups according to their hardware architecture: embedded PLCs (ePLCs) and PC-based PLCs. PC-based PLCs offer more user-oriented tools [30], while ePLCs are cheaper and use less power. Given that user-oriented tools are not the focal point of BSIS, ePLCs are a more suitable choice. Plus, the great paradigm shift from PLCs to ePLCs is resulting in some great advances in multiprocessing [32, 33], customized compilation tools [29], support for market-friendly embedded hardware (e.g., Raspberry-Pi), and more. Some of this market-friendly embedded hardware can be used with IoT platforms [34, 35].
2.3. Base Station Management and BSIS
Despite all the advantages of PLCs mentioned above, each is based on a single PLC. But, in base station management, some emphasis needs to be placed on seamless integration between the edge and the cloud to ensure that the edge nodes are responsive, reliable, and scalable.
Responsiveness is reflected in high concurrency and the elasticity of the computing power [4]. Here, reprogramming and real-time tasks must be shifted to the edge where response times can be reduced to within one cycle (e.g., 1 ms).
Reliability means the system is trustworthy and consistently performs well. In the case of base station systems, the reliability of the interactions between the edge nodes and the cloud is important and significant. The guaranteed reliability of PLCs has been formally verified in several studies (e.g., [26, 27, 36]) and, when combined with the above-mentioned advantages, ePLCs make a very robust reliable choice as an edge node. As an added benefit, shifting critical tasks to the edge nodes can also increase safety.
Scalability is reflected in the system’s potential for expansion and how it integrates with other systems. Semantic technologies and methods, the capacity for dynamic programming, and compliance with standards must be considered to ensure good scalability. Following [37, 38], we have selected extensible markup language (XML) as the semantic interaction language due to its wide-spread use. As one of the main contributors to scalability, BSIS does support dynamic programming of the edge nodes. Lastly, BSIS complies with the EC61131-3 and PLCopen standards [3, 31].
3. BSIS: A Base Station IoT System
BSIS’s main purpose is to store real-time and historical data, manage alarms, and maintain the edge nodes. We have ignored business services like graph analysis and reporting, on-site visualization tools, and order management in this paper, and, instead, have solely focused on an academic analysis of edge node reprogramming and real-time task execution.
The underlying premise of BSIS is that every edge node can and must be able to perform many tasks. Some typical tasks are listed below; most are associated with alarms.(1)Core tasks: the basic functions every edge node must be able to perform.(2)Alarm filter (): where edge nodes read a filter file from a local database and filter the alarms according to the rules in the file. For example, if an alarm has already been signaled by another type of alarm, it is not necessary to report the current status as a separate issue.(3)Combined alarms (): where the data from several sensors is combined and assessed against an alarm threshold so as to reduce the number of alarms. The need for an alarm is determined by several sensors and a logic program.(4)Linked tasks (): where edge nodes directly perform some actions based on an input, for example, taking a photo of the environment if a smoke alarm sounds.
These last three types of tasks are all typical reprogramming and real-time tasks. AF and CA are designed to reduce the number of alarms to a practically manageable level. Linked alarms meet the requirement for real-time responses, which cloud platforms cannot provide.
3.1. The BSIS System Structure
As shown in Figure 1, the system consists of the edge, network, and cloud, which is common for IoT platform [39]. The edge and cloud are connected by a mixed 3G and 4G network.
Edges. The edge nodes are ePLCs and can collect data either directly from the sensors or from the devices using an appropriate communications protocol. The sensors could be measuring temperature, humidity, smoke, infrared light, or vibrations, and the devices could be heat exchange systems, inverters, standard or uninterruptible power systems, ammeters, door systems, ventilation systems, air conditioners, generators, or battery systems.
Cloud Server. The cloud server has basic functions that cover all four types of tasks. Each task corresponds to a unique static program.
Therefore, let BSIS be defined as a 4-tuple set :
where is the cloud server that produces requests and responds to the edge nodes, is the finite set of task programs, is the th task program, is the total number of tasks, and represents the basic tasks that should be installed in all edge nodes. is a finite set of edge nodes, is the th edge node, is the number of edge nodes, denotes that the edge node en contains the th task, and is the finite set of interaction commands. includes the edge’s request command , the cloud’s request command , the edge node’s response to the cloud , the cloud’s response to the edge node , the data collection command from the sensors and devices , and the edge’s command to the devices .
The finite set of interaction commands is the key to BSIS. It is divided into two parts: (a) the back and forth interactions between the edge nodes and the field sensors/devices ( and ); and (b) the requests made from the edge to the cloud and from the cloud to the edge ( and ) and the corresponding response messages in XML from the edge to the cloud and from the cloud to the edge ( or ). The content of and depends on the connected sensors and devices.
Table 1 lists the requests, and , and Table 2 shows an example of a data response message by an edge node.
3.2. Edge Computing System
In BSIS, the edge nodes are used to collect heterogeneous data from the devices and sensors and to interact with the cloud. As shown in Figure 2, the software architecture of the ePLCs has a three-layer software structure, which comprises the cloud layer, a flexible layer, and the communication layer.
The cloud layer is responsible for interfacing with the cloud. It contains two interfaces, which are the B interface for normal interactions and the M interface for maintenance.
The flexible layer is where nodes are reprogrammed or even changed automatically by a dynamic compilation function in the cloud. This layer mainly consists of types of applications (i.e., ). Real-time tasks also run in this layer.
The communication layer connects the devices and sensors with the edge nodes. Almost all communication protocols are supported (e.g., CAN, TCP, UDP, FTP, and Modbus/RTU), but can be trimmed or changed by the cloud server according to the configuration file.
The underlying operating system provides basic functions.
3.2.1. ePLC Resource Allocation
The RAM available to the ePLC, i.e., its resources, is defined aswhere is composed of RAM addresses with one byte as the smallest unit: refers to data cell, to the intermediate cell, to the input cell, to the output cell, to the timer cell, and to the counter cell.
3.2.2. Local Data Base
Application programs are independent of each other in the flexible layer. Every application has its own parameter area. A record of its required resources is kept in a configuration file, and a local database stores a limited amount of historical data along with the configuration file (CF), as shown in Figure 3. The CF is described as follows:where is the node information and is the set of resource constraints on , i.e., the constraint () and the constraint (). is the set of application descriptions, which contains the task ID () and the required number of resources (, , , , , ). is the set of historical data and contains historical alarms (), captured photos (), and the latest bin (). denotes the filter profile and is the set of protocol profiles, which contains the protocol IDs () and the connected device IDs (). The s are used to communicate with the connected device.
Thus, the works as follows:Here, every contains several tasks. is the input from sensors and controllers, is a condition that produces a request, i.e., is updated, is a condition that triggers the transfer of monitoring data to the cloud, and is a condition that outputs actions to a controller. As an example, if is true, the ePLC will send a request to the cloud to update its program. It is worth noting that each condition and corresponding action is independent.
3.3. Cloud Management
The cloud platform offers the conventional advantages, i.e., flexibility and elastic capacity for computation and storage. We have designed BSIS to operate on a standard platform-as-a-service (PaaS) structure. The reprogramming and real-time tasks are stored on the cloud platform, and every program has a corresponding XML file. When a request is received from an edge node, the cloud platform will respond with , then execute a compilation process and send to the edge node.
The compilation process is shown in Algorithm 1 and described in detail below. Resource assessment: Compilation terminates if the resource assessment fails. The function that determines whether a request will exceed its resource constraints is defined aswhere is the resource matrix and is the ePLC ID. The rows in are the tasks, and the columns contain the required number of resources () to complete that task. represents the resource constraints for in turn.
| ||||||||||||||||||||||||||||||||||
Link Programs. The cloud checks the program version number according to the program IDs in the request to determine whether the program is null or old. If the conditions are satisfied, the compilation flag is set to 1, and the program IDs are stored in another array . The start-resource address of every program is then calculated and stored in .
Call Compiler. If the compilation flag equals 1, the PLC compiler is called according to , .
Hence, the cloud process can be described aswhere is the compilation according to , is the condition that produced the request to , and is an option for human intervention. If is true, will not be sent to the edge node. .
3.4. The Execution Process
Equations (1), (4), and (6) define the BSIS model, and the execution process is illustrated in Figure 4. The edge nodes are crucial because reprogramming and real-time tasks are performed locally according to the input . Further, the edge nodes can control devices, such as cameras. They directly store some information, such as filtered alarms, and they interact with the cloud using and including conditions, i.e., , , and .
Specifically, is used to update the edge node’s programming. The cloud compiles the task programs according to the request using Algorithm 1, which results in () and sends to the edge node. Once the B interface in the ePLC receives , it will update its own bin file in the flexible layer and then hot restart.
4. Performance Analysis
To analyze the performance of BSIS, we constructed a proxy of a base station management system. As shown in Table 3, this BSIS model consists of five servers. The servers sit on the Ali Cloud, and a web server provides various business tasks to users. The task engine server is responsible for task management, and the MySQL server hosts the database. The MQTT server interacts with the edge nodes directly. The VPN server provides a private connection to managers and edge nodes. The ePLCs are IEC61131-3-compliant [40] and are composed of a master CPU (STM32F207VCT6) and a slave (MX283). The master CPU has a frequency of 454 MHz with 128MB RAM, and the slave CPU has 120MHZ with 128KB RAM. The local database has a capacity of 10 MB.
Figure 5 shows the relationship between the servers and edge nodes. The task engine interacts with the edge nodes through the MQTT server and, once the data is received, the task engine stores it in the MySQL database. Users are able to communicate with the task engine server or the MySQL database through the web server. Under special circumstances, managers can directly access the edge nodes through the VPN server. The web services and the VPN server are ancillary to this paper since our focus here is on reprogramming and real-time tasks. Hence, no detailed explanations of these components were given in Section 3, and they are only included in the following analyses as comparisons to the edge node response times.
To conduct our analyses, we collected alarm information from 1000 accessed ePLCs for the one-month period of January 2018. The information comprised the dispatch records of maintenance personnel to diagnose the reason for an ePLC being offline in 175 cases. We included the data in our dataset if the ePLC was offline for more than three days or experienced downtime 10 or more times in one day.
4.1. Scalability Analysis
As mentioned in Section 2, in base station management, scalability is reflected in the ability of an edge node to be reprogrammed or to perform real-time tasks. In BSIS, the reprogramming and real-time tasks mainly include CA, AF, and LT. However, operators could add any other reprogramming tasks to the cloud server.
Figure 6(a) lists the type and number of tasks in our analysis. There was no record of any AF tasks. Therefore, only CA and LT tasks are considered in these analyses. Figure 6(b) shows the percentage of normal alarms, CAs, and LT alarms based on six months worth of data at 88%, 11%, and 1%, respectively. There was only a relatively small proportion of linked tasks, but all of them are urgent and need to be performed in real-time (e.g., taking a photo after a smoke alarm).
(a)
(b)
The full name, abbreviation, and number of relevant alarms () for the CAs are listed in Table 4. We use to denote the th number of CAs and to denote the th number of relevant alarms. Note that there are different logic processes for the different types of CAs and their relevant alarms, so we assumed the alarm reduction ratio to be 50%. denotes the total number of current alarms (CT), while denotes the total number of alarms where a CA was not used (PT). The total reduction in the number of relevant alarms where a CA was not used (CARA) is denoted as . Hence, is defined aswhere the is the number of CA types that occurred.
Then, can be ascertained fromAnd CARA can be ascertained from .
Applying (7) and (8) to the number and types of CA alarms that occurred in January 2018 only shown in Figure 7(a), we find an overall 60% reduction in alarms.
(a)
(b)
4.2. Response Time Analysis
Our second analysis concerns response times. We compared the response times from the website (Web) and the mobile application (App) with the response times from the edge nodes. The results, averaged over 1000 runs, are shown in Figure 8, divided into two stages. is the response time from the edge nodes to the cloud, and is the response time from the cloud to the website or mobile application. The edge response times () fell within a span of several cycles to less than nearly 0.1 s overall because the ePLCs perform the functions locally. In contrast, the response times between the cloud and the website/app () were around 2.3 s and 3.2 s respectively. The mean response times between the cloud and the website/app () were 3.9 s and 4.3 s, respectively, largely because wired network speeds are faster than cellular network speeds.
(a)
(b)
4.3. Reliability Analysis
To conduct the reliability analysis, we measured the total downtime of the ePLCs for January 2018. As shown in Figure 9, the average downtime ratio was 0.66%. From this, we can conclude that BSIS has high reliability in terms of uptime. Further, we tracked the reasons for downtime for the 175 cases of offline ePLCs. The reasons were distilled into categories: edge node faults (ENF) 36; outdated software (LSV) 13; signal problems (SP) 10; SIM card problems (SCP) 23; wiring errors in the wireless module (WEW) 10; duplicate IDs (IR) 1; software update failures (SUF) 4; low power to the edge node (LPE) 8; restart failures (RF) 3, wireless module faults (WMF) 31; IP problems (IP) 12; fake stations (FS) 19; and “other”(O) 36. Figure 10 illustrates these problems with the corresponding ePLC downtime they caused, assuming these reasons account for all 0.66% of the downtime. This data will be used to inform future research.
5. Conclusion
In this paper, we explained the problems associated with using cloud computing as a platform for base station management systems. Our solution in response is an edge-cloud IoT computing system, called BSIS, that allows ePLCs to act as edge nodes to perform specific tasks locally. BSIS has several advantages. The edge nodes are reprogrammable and have much lower real-time response rates than a cloud server, which is particularly beneficial for sounding alarm signals and linking tasks. Further, BSIS seamlessly integrates the cloud and edge components of the system. Analyses of the scalability, responsiveness, and reliability of BSIS indicate a 60% reduction in the number of alarms, a potential edge response time of less than 0.1s, and a downtime ratio of 0.66%.
In future work, we will explore the potential of integrating artificial intelligence into the system architecture to further improve the performance of BSIS.
Data Availability
No data were used to support this study.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work is supported by a grant from the National Natural Science Foundation of China (no. U1609211) and the Science and Technology Program of Zhejiang Province (no. 2018C04001).