Abstract
For the problems of unreasonable computation offloading and uneven resource allocation in Mobile Edge Computing (MEC), this paper proposes a task offloading and resource allocation strategy based on deep learning for MEC. Firstly, in the multiuser multiserver MEC environment, a new objective function is designed by combining calculation model and communication model in the system, which can shorten the completion time of all computing tasks and minimize the energy consumption of all terminal devices under delay constraints. Then, based on the multiagent reinforcement learning system, system benefits and resource consumption are designed as rewards and losses in deep reinforcement learning. Dueling-DQN algorithm is used to solve the system problem model for obtaining resource allocation method with the highest reward. Finally, the experimental results show that when the learning rate is 0.001 and discount factor is 0.90, the performance of proposed strategy is the best. Furthermore, the proportions of reducing energy consumption and shortening completion time are 52.18% and 34.72%, respectively, which are better than other comparison strategies in terms of calculation amount and energy saving.
1. Introduction
With the rise of computing-intensive applications and explosive growth of data traffic, users’ requirements for the computing power and service quality of mobile devices are also increasing [1]. At present, cloud computing also faces many problems and challenges. Due to its resource-intensive architecture, mobile cloud computing imposes a huge additional load on the backhaul link of mobile networks [2, 3]. Thus, Mobile Edge Computing (MEC) technology is proposed, which physically integrates computing and storage resources into the edge of mobile network architecture [4, 5]. This not only effectively reduces the transmission delay but also solves the problems of high load and high delay caused by mobile cloud computing [6]. At the same time, MEC has the characteristics of distributed architecture, being at the edge of network, low latency, user location awareness, and network status awareness [7, 8]. However, deploying a large number of computing and storage devices at the edge of network for users to choose and accessing neighboring service providers for edge computing will bring a series of complexities such as access and resource allocation strategy selection, user mobility management, and computing task migration problems [9].
In order to achieve the goal of short completion time and lower terminal energy consumption under delay constraint, this paper proposes a task offloading and resource allocation strategy based on deep learning for MEC. In order to shorten the completion time of computing tasks and minimize the energy consumption of all terminal devices while satisfying delay constraints, the proposed strategy is designed in a multiuser multiserver MEC environment, combined with computing model and communication model in system. Moreover, a new objective function is designed, which uses objective optimization to further reduce energy consumption and time delay. It uses Dueling-DQN algorithm to solve the optimization model to shorten completion time and minimize energy consumption of all terminal devices while meeting the delay constraints.
The remaining chapters of this paper are arranged as follows: Section 2 introduces the relevant research work on mobile task unloading. Section 3 introduces the system model. Section 4 introduces the new computing offload method based on improved DQN. In Section 5, simulation experiments are designed to verify the performance of the proposed model. Section 6 is the conclusion.
2. Related Work
In MEC network, computation offloading may occur in three types: full offloading, partial offloading, and local processing [10]. An important research hotspot in the field of computation offloading is computation offloading decisions. Generally speaking, the offloading goal mainly focuses on minimizing the overall delay or minimizing energy consumption for user devices while meeting the minimum delay requirements [11]. Reference [12] proposed a distributed task offloading strategy for low-load base station groups in MEC environment. It selects the best MEC node offloading amount by game equation on the basis of quantifying offloading cost and delay. But it is not suitable for high-load application environments. In reference to the problem of unbalanced computing resources on the edge server in vehicle edge computing network, [13] proposed a load balancing task offloading scheme based on software-defined network. This solution can effectively reduce delay and improve the efficiency of task offloading processing. However, the processing method used has poor performance, which affects the distribution efficiency. Reference [14] used greedy selection to design a maximum energy-saving priority algorithm to achieve optimal offloading of computing tasks on mobile devices, but it does not consider the delay constraints of task offloading. Reference [15] combined long short-term memory (LSTM) and candidate network set to improve the deep reinforcement learning algorithm and used this algorithm to solve the problem of offloading dependency of multinode and mobile tasks in large-scale heterogeneous MEC. But they ignore the optimal allocation problem of computing resources.
Similar to computation offloading, resource allocation is also one of the core issues in MEC [16]. In MEC network, technologies such as content caching and ultradense deployment are introduced, and multiple resources are deployed to mobile network edge according to the specific needs of users. This can further ensure the quality of service and greatly increase system capacity [17]. However, due to objective reasons such as physical volume and power consumption, the mobile network edge has limited computing resources, storage cache capacity, and spectrum resources. How to allocate multiple resources and improve the system service efficiency has a huge effect on the improvement of MEC network system performance [18]. Reference [19] proposed a time average computation rate maximization (TACRM) algorithm, which allows joint allocation of radio resources and calculation resources. However, the overall performance and task requirements of devices were not considered comprehensively in the allocation process, and the allocation efficiency still needs to be improved. Reference [20] comprehensively considered factors such as CPU, hard disk space, and required time and distance and proposed a comprehensive utility function for MEC resource allocation to achieve the optimal allocation of resources in MEC and cloud computing. However, this function considers many factors, which will seriously affect the efficiency of allocation in real applications. Reference [21] designed a two-layer optimization method for MEC, which uses pruning candidate modes to reduce the number of unfeasible offloading decisions. Through ant colony algorithm to achieve the upper-level optimization, the resource allocation effect is better. However, the server computing resource constraints and task delay constraints are not considered, and the overall timeliness is not good. Reference [22] constructed a low-complexity advanced branch model, which can be used for resource scheduling in large-scale MEC scenarios.
Due to the lack of powerful processing algorithms, the overall efficiency and performance are not ideal. To this end, comprehensively considering the task offloading and resource allocation problem, a deep learning-based MEC task offloading and resource allocation strategy is proposed to coordinate and optimize the allocation and offloading between computing resources and computing tasks, which improves the comprehensive computing efficiency of MEC.
3. System Model
The system model is a multiuser multiserver application scenario, in which there are terminal devices and MEC servers. The base station is used to provide communication resources for user equipment. Each base station is connected to an edge computing server through optical fiber, through the wireless communication link to connect to MEC server to calculate task data of offloading terminal devices, as shown in Figure 1. It is assumed that each terminal device can perform offloading computations or local calculations for its own execution tasks. And when offloading, the task can only be offloaded to one MEC server for calculation, and each terminal device is within the range of wireless connection. However, the computing power of each MEC server is limited; it cannot accept the offloading request of each terminal at the same time.
The collection of terminal devices is , the collection of MEC server is , and the collection of all tasks is . Each terminal device has a calculation-intensive task to be processed, which specifically includes the data (code and parameters) required for computing task , the CPU workload required for computing task , and the completion time of task . The extension constraint is , namely, . The set of offloading decisions for each is .
When , and is local offloading, the rest means offloading to MEC servers.
3.1. Communication Model
In the computation offloading problem, two links are mainly studied: wireless link from terminal devices to MEC and the wired link from MEC to cloud in the core network. In the wireless link, Finite-State Markov Channel (FSMC) model based on fading characteristics is used. FSMC model has a wide range of applications in wireless networks [23, 24].
The channel is divided into nonoverlapping intervals through the division of channel-related parameter ranges, and each interval of selected parameters represents a state in FSMC model. The relevant parameter used in FSMC may be Signal-to-Noise Ratio (SNR) amplitude of received signal at the receiving end or collected energy. SNR can be selected as a parameter that composes SNR model [25]. The SNR of receiving end is divided into levels, and each level is associated with a state of Markov chain. The block fading channel is considered to be that the SNR of receiving end is a constant within a period of time but will change according to Markov transition probability between different periods. Assume that random variable is the SNR of receiving end of terminal device . That is, can be improved according to Markov chain of finite states, and all its states can be expressed as . The realization of random variable of terminal device in the time period is represented as , specifically expressed aswhere and . Let denote the probability of state transitioning from state to state in the time period . The channel state transition probability matrix of terminal device is denoted as , where , , and .
In practical applications, the transfer matrix can be observed and measured from wireless environment in the past. In addition, it is considered that exists independently for terminal device . Based on FSMC channel model, is used here to represent SNR between terminal device and MEC server . Since there is no interference between terminal devices, its channel efficiency can be expressed as . Considering that the bandwidth of MEC server is divided into , the bandwidth of each channel is . Assuming that each user is allocated at most one channel, the transmission rate from terminal device to MEC server can be expressed as
The subchannel owned by MEC server has certain restrictions on receiving ; that is, the bandwidth allocated by MEC server to all connected users cannot exceed the total bandwidth of MEC server . Besides, MEC server is limited by cache and computing capacity. On the one hand, MEC server can only handle a limited number of tasks; on the other hand, the load that MEC server can handle is also limited (such as the number of computing tasks). Therefore, some tasks will be further offloaded to the core network to be processed by the core network. Use to represent the computation offloading decision indicator, which is used to indicate the way the server provides services. Among them, means that the terminal device is processed by connected MEC server for computing tasks. And indicates that the task will be further offloaded to core network for processing by connected MEC server.
In order to further offload tasks to cloud, the wired backhaul link from MEC server to core network is considered. Assuming that the backhaul link capacity of network is (in bits per second), the backhaul link capacity allocated by MEC server is . Then, the following restrictions must be met:where is the connection between terminal device and MEC server and is the transmission rate between terminal device and MEC server .
The sum of the rates of offloading computation tasks to the terminal device of core network by MEC server cannot exceed the backhaul capacity of MEC server . And the sum of speeds of all terminal devices processing computing tasks in the cloud cannot exceed the total backhaul capacity of system.
3.2. Calculation Model
If is processed locally, use to represent the time when is executed locally, which is specifically defined aswhere workload is the total number of CPU cycles required to complete and is the local computing power of terminal device (i.e., the number of CPU cycles executed per second).
Use to represent the energy consumption of devices executed locally by , which is defined as follows:where is terminal device to calculate the energy consumption per unit of CPU cycle, .
If is processed at the edge, delay and device energy consumption under edge execution should be calculated from three parts: data upload, data processing, and data return [26]. The specific calculation is as follows.
First, terminal device uploads data to the corresponding MEC server by wireless channel. Let be the time when device uploads data, which is defined aswhere is the data size of and is data upload rate in the system model (i.e., the amount of data uploaded per second).
Then, the energy consumption of terminal device uploading data iswhere is the uplink transmission power of terminal device .
Then, MEC allocates computing resources for calculation after receiving processed data. Use to represent the time when the offloading data is calculated in MEC server, which is defined aswhere are the computing resources allocated by MEC servers for offload execution (i.e., the number of CPU cycles executed per second). When is unloaded to the local or other MEC server, is zero and serves as a constraint in the model, namely,
At this time, terminal device has no computing task and is in a waiting state and generates idle energy consumption. Suppose is the idle power of terminal device , then the idle energy consumption of terminal device under offloading computation is
Finally, MEC server returns the calculation result to terminal device . The calculation result during backhaul is small and downlink rate is high, so the time delay and energy consumption when terminal device is received are ignored. Therefore, delay under edge execution is the sum of transmission delay and the calculation delay of MEC server, namely,
The device energy consumption under edge execution is the sum of upload energy consumption of device and the idle energy consumption of device waiting for to complete calculation on MEC server, namely,
In summary, the time delay and energy consumption of the entire calculation process of task in terminal device are
Note that and should meet the following restrictions:
The time delay constraint of is that computing power is twice 1.4 GHz. is the overall computing resources of MEC server ; that is, the sum of computing resources allocated by each that is offloaded to MEC server should not exceed .
3.3. Problem Model
The purpose of this paper is to jointly optimize offloading decision-making and resource allocation scheme in the multiuser multi-MEC server scenario, considering the limited computing resources and time delay constraint of computing tasks. This allows all computing tasks to shorten the completion time and minimize energy consumption of all terminal devices while meeting the delay constraints and extend the use time of terminal devices [27, 28]. Thus, the system objective function is defined as is the ratio of completion time to the delay constraints. According to the calculation results of simulation experiment, the difference between and is a decimal order of magnitude. Therefore, to ensure that the two are of the same order of magnitude and optimized together, is multiplied by a factor of 10. The objective function minimizes the ratio of overall energy consumption of terminal devices to the task execution time and delay constraints by solving the optimal offloading decision and resource allocation plan to achieve research purpose. The overall problem model is as follows:where is the task offloading decision amount and is the calculation resource allocation amount. Constraints C1, C2, and C3 indicate that each task can only be offloaded to the local or one of MEC servers for calculation. C4 represents the constraint of task completion delay, and C5 and C6 represent the constraint that allocated computing resources should meet.
4. New Computation Offloading Method Based on Improved DQN
4.1. Multiagent Reinforcement Learning Algorithm
The multiagent reinforcement learning system is shown in Figure 2, where multiple agents act at the same time. Under the joint action, the entire system will be transferred, and each agent will be rewarded immediately [29, 30].
For multiagent reinforcement learning, it is first necessary to establish a Markov game model. Markov game can be described by a multigroup . Among them,(1) is the number of agents; that is, is the number of terminal devices. is the system state, which generally refers to the joint state of multiple agents, that is, the joint state of each agent. The terminal device shares the current load status of edge computing servers, which can be expressed as where is the load of MEC.(2) is the instant reward function of each agent. That is, in current state , after joint action taken by multiple agents, the reward is obtained in the next system state .
The reward function completely describes the relationship between multiple agents. When the reward function of each agent is the same, that is, , it means that the agent is a complete cooperative relationship. When there are only two agents and reward function is opposite, that is, , it means that the agents are in perfect competition. When the return function is between the two, it is a mixed relationship between competition and cooperation.
4.2. Problem Description and Modeling
4.2.1. Network Status
represents the network state space, where represents the network state at time period , and improvements are made in the entire time period . The network status consists of SNR of each terminal device and cache status of each MEC server. can be defined aswhere represents SNR between user terminal device and all MECs. represents the cache status of MEC servers.
4.2.2. Network Behavior
The intelligent agent needs to determine the attachment relationship between the terminal device and MEC server in each time period. That is the determination of the terminal device's computing offload, the allocation of computing resources, and the service cache policy of each MEC server. Thus, each executable action of terminal devices in the time period can be defined as follows:where represents the attachment indicator of terminal device and represents the calculation and offloading decision of terminal device .
4.2.3. Reward Function
The goal is to maximize total benefit of system, but the reward function should be set to current benefit of system. First calculate the system leased spectrum and backhaul resources and allocate them to terminal devices part of the revenue. The unit price of spectrum leased from MEC server is set to per Hz, and the unit price of backhaul link from MEC server to core network is set to per bps. Corresponding to this, the calculation data is transmitted to MEC server corresponding to terminal device and backhaul link from MEC server to the core network is used for charging. The unit price is defined as per Hz and per bps. Therefore, by summarizing this part of the income and expenditure, part of income for leased spectrum and backhaul resources obtained by terminal device can be obtained:
Then, calculate the profit obtained by terminal devices from allocating computing resources. On the one hand, when MEC side performs computing tasks, it needs to pay communication company for the loss of processing computing tasks and define the unit price of MEC server energy consumption as . On the other hand, the terminal device needs to pay a certain price for the server on MEC side, and computing resource allocated for each unit computing task is set to . Therefore, the benefit obtained by allocating computing resources to terminal device can be calculated as
The amount of computing resources allocated to each unit computing task by the above formula has a very important impact on the completion time of computing task. Thus, the service cache cost mainly includes two parts: the cost of replacing type of cache supported on MEC side, and the cost of caching specific services on MEC server. Define the unit price of replacing cache type on MEC server as for each service type, and the unit price for caching services on MEC server is per storage space. In order to increase the benefits of cache, the business type is quantified by weak backhaul from MEC server to the core network, which will be used to measure the cost of users. The benefits obtained by executing the cache service on MEC server can be expressed aswhere represents the number of nonzero elements, is an auxiliary function, and when , ; otherwise, . The instant reward is designed as the total income of MVNO of all current users of system during the time period , namely,
Here the long-term return is expressed aswhere is the discount rate of future earnings weights. When approaches 1, the system will pay more attention to long-term benefits, and when approaches 0, the system will pay more attention to short-term benefits.
4.3. Dueling-DQN
DQN is an effective reinforcement learning algorithm, which can make the agent learn good experience from the interaction with environments [31–33]. At the same time, according to DQN learning mechanism, there are improvements to DQN algorithm in different aspects. In DQN, due to the error in the Q estimated value itself, process can be seen according to the expression. It is equivalent to putting forward the largest error, which also leads to the problem of overestimation. Double-DQN is an effective improved algorithm for this problem. In Double-DQN algorithm, the update form of is changed towhere is the discount factor.
The Double-DQN algorithm takes advantage of double neural network and uses two neural networks to learn at the same time, effectively avoiding the overestimation problem caused by error amplification.
Dueling-DQN is also an improvement to DQN algorithm. Compared with previous algorithms, Dueling-DQN algorithm learns faster and has better results. Compared with DQN algorithm, Dueling-DQN retains most of the learning mechanism, and the only difference is the improvement of neural network, as shown in Figure 3.
(a)
(b)
In the traditional DQN algorithm, the output result is Q value corresponding to each action. In Dueling-DQN algorithm, the output is expressed as a combination of two parts: the value function and advantage function [34]. Among them, value function refers to the value of a certain state, and advantage function refers to the advantage obtained by each action on the state. Therefore, in Dueling-DQN algorithm, Q value problem in DQN can be reexpressed as the following form:where and are the value function and advantage function, respectively, and is the parameter of neural network convolutional layer. are the parameters of two control flow layers, respectively. The latter item of the plus sign centralizes the advantage function in order to solve the uniqueness problem of Q value.
5. Experimental Results and Analysis
The specific simulation parameters are as follows.
Assume that the computing power of each device is 1.5 GHz, the uplink transmission power is 800 mW, the idle power is 100 mW, and the upload rate is 2.5 Mb/s. M = 4 and overall computing capacity of each MEC server is 6 GHz, 5 GHz, 3 GHz, and 1 GHz, respectively. The data in task obeys uniform distribution of (600, 1200), and the unit is k bits. The workload obeys uniform distribution of (1000, 1500), and the unit is Megacycles.
For the parameters of Dueling-DQN algorithm, set the learning rate to 0.001 and discount coefficient to 0.90. The size of experience replay set is 3000, and the number of randomly sampled samples is 40.
5.1. Parameter Analysis
5.1.1. Learning Rate Analysis
The learning rate of the algorithm will have a great impact on the performance of the proposed strategy. Therefore, three different learning rates of 0.01, 0.001, and 0.0001 are selected to compare the convergence of improved DQN algorithm, as shown in Figure 4.
5.1.2. Discount Factors Analysis
Similarly, the influence of discount factor on improved DQN algorithm is shown in Figure 5, where the discount factor takes values 0.8, 0.9, and 0.95.
It can be seen from Figure 5 that as the discount factor increases, the long-term reward is continuously increasing. When is 0.95, the long-term reward is 3700 when it is stable. Because the discount factor will affect behavior selection strategy, that is to say, a larger discount factor will cause system to pay more attention to long-term benefits, and a lower discount factor will cause system to pay more attention to current benefits, a higher discount factor will often lead to greater long-term benefits. However, in actual use, using an overly high discount factor does not have corresponding benefits. This is because the system in reality is more changeable, and too much emphasis on future benefits will lead to excessive calculations and excessive losses in the system, which often requires a trade-off.
5.2. Optimization Comparison under Different Objective Functions
For multiobjective optimization problems that reduce time delay and energy consumption, the weighted sum of task execution time delay and terminal execution energy consumption is usually used as the objective function to solve problem, and the calculation is as follows:where is the weight coefficient of execution delay and is the weight coefficient of execution energy.
Comparing (22) with the proposed objective function (13) to optimize the delay and energy consumption, the number of terminal devices is 12. Considering that the goal of the proposed strategy is to shorten time delay and reduce energy consumption while satisfying the time delay constraints, therefore, the values of are, respectively, 0.8, 0.6, and 0.4, and the joint experiments of Energy Reduced Scale (ERS) and Time Reduced Scale (TRS) are carried out, as shown in Table 1.
It can be seen from Table 1 that when is 0.8 and 0.6, the control strategy pays more attention to the optimization of time delay, and when is 0.4, optimization results pay more attention to the optimization of energy consumption. However, the optimization result of the proposed objective function is the best, and ERS and TRS are 52.18% and 34.72%, respectively, which can shorten time delay and reduce energy consumption under the time delay constraints.
When computing task is 150, comparing control strategies under the four objective functions with the random offloading strategy, the results of ratio of the time delay and energy consumption reduction are shown in Table 2.
It can be seen from Table 2 that delay and energy consumption optimization effect of the proposed optimization target is better, and the reduction ratio of delay and energy consumption is 2.58% and 30.67%, respectively, because the optimization objective of the proposed strategy comprehensively considers the offloading decision and resource allocation plan of joint optimization system when the computing resources are limited and computing tasks have time delay constraints. This allows all computing tasks to shorten completion time and minimize the energy consumption of all terminal devices while meeting the delay constraints. This demonstrates the effectiveness of the proposed objective function.
5.3. Performance Comparison with Other Algorithms
In order to demonstrate the performance of the proposed strategy, compare it with [12], [19], and [14] in terms of objective function value, calculation amount, and time saving. Li and Jiang [12] proposed a distributed task offloading strategy, which selects the best MEC node offloading amount by game equation on the basis of quantifying offloading cost and delay. Reference [14] used the greedy selection algorithm to design the maximum energy-saving priority algorithm and energy priority strategy to achieve optimal offloading of computing tasks on mobile devices. Reference [19] used the time average calculation rate maximization algorithm to jointly and efficiently allocate radio resources and computing resources.
5.3.1. Algorithm Comparison under Different Cumulative Tasks
In the experiment, objective function value results of the four strategies are shown in Figure 6 for different accumulations of computing tasks.
It can be seen from Figure 6 that the value of objective function is gradually increasing with the increase of cumulative number of tasks for the four offloading strategies. However, the proposed strategy has a relatively lower objective function value than other strategies. That is, the energy consumption and delay are relatively small. For example, when the number of tasks is 180, the objective function value is only 298. Since the proposed strategy considers computation offloading and resource allocation comprehensively, improved deep learning algorithm is used for optimization, and delay and energy consumption are minimized. Reference [19] only matched computing resources but did not rationally optimize the task offloading scheme and computing resource allocation scheme, resulting in high task execution time delay and energy consumption. References [12] and [14] both used corresponding algorithms for optimization to achieve better resource allocation and task offloading. But their analysis of time delay is less, so the performance needs to be strengthened.
5.3.2. Computation Number Comparison of Offloading Tasks under Different Offloading Strategies
Under four different computation offloading strategies, the comparison results of the computing number of offloading tasks on terminal device side are shown in Figure 7. Vertical axis represents the total calculation number of tasks performed by all terminal devices to perform calculation and offloading. The calculation number of tasks is used to represent the amount of calculation services provided by MEC server. Therefore, the evaluation indicators in the figure also represent the benefits of computing terminal devices in the offloading mode.
It can be seen from Figure 7 that as time increases, computing tasks continue to increase, and the amount of task calculations also increases. However, the calculation amount of the proposed strategy is significantly better than other comparison strategies. Taking the simulation time of 140 s as an example, compared with [12], [19], and [14], the proposed strategy has increased by 11.54%, 20.83%, and 152.72%, respectively. It can be argued that the proposed strategy is the best compared to task offloading. It uses Dueling-DQN algorithm to process task offloading and resource allocation models, and its optimization performance is better than the greedy selection algorithm in [14] and the game equation model in [12].
5.3.3. Energy-Saving Comparison of per Unit Terminal Devices
Under four different computation offloading strategies, the comparison of energy consumption saved by each terminal device by computation offloading on average is shown in Figure 8. In the local calculation model, all energy consumption is generated by local calculations. In the computation offloading mode, the energy consumption is communication energy consumption caused by upload tasks. For the task of performing computation offloading, the difference between the two is energy saving.
It can be seen from Figure 8 that, compared with other comparison strategies, the proposed strategy has the largest energy-saving rate, which is close to 10 × 104 J; this also means the least energy consumption. Aiming at the overestimation problem in DQN, the proposed strategy uses Dueling-DQN algorithm for optimization. And it designs the system benefits and resource consumption as rewards and losses, which improves the efficiency and rationality of task offloading and resource allocation by optimizing problem solution. Reference [19] only used the time average calculation rate maximization algorithm to efficiently allocate computing resources. The optimization algorithm is more traditional and has poor performance. Thus, the overall energy saving is not high. Reference [12] used the game equation model to optimize task offloading strategy but does not realize the rationalization of resource allocation. Therefore, the maximum energy saving is 710 × 104 J. Reference [14] used greedy selection algorithm to design an optimal energy-saving strategy but did not consider server computing resource constraints and task delay constraints. Therefore, the overall performance is not as good as the proposed strategy.
6. Conclusion
MEC server has limited computing resources and computing task has delay constraint. How to shorten completion time and reduce terminal energy consumption under the delay constraints becomes an important research issue. To solve this problem, this paper proposes a task offloading and resource allocation strategy based on deep learning for MEC. In the multiuser multiserver MEC environment, a new objective function is designed to construct mathematical model. In combination with deep reinforcement learning, the partially improved Dueling-DQN algorithm is used to solve the optimization problem model, which can reduce the completion time of computing tasks and minimize energy consumption of all terminal devices under the delay constraints. The proposed strategy is demonstrated by experiments based on Python platform. The experimental results show that when learning rate is 0.001 and discount factor is 0.90, the energy saving is close to 10 × 104 J, which is better than other comparison strategies. In terms of calculation amount, it increased by 11.54%, 20.83%, and 152.72%, respectively.
In practice, different users have different concerns about service quality. Therefore, we can refer to the different needs of users when making computation and offloading decisions in the following research. It can assign a certain weight to the factors affecting the quality of service and combine the task priority for scheduling.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by Key Disciplines of Computer Science and Technology (2019xjzdxk1) and New Engineering Pilot Project (szxy2018xgk05).