Development of an AI-Enabled Q-Agent for Making Data Offloading Decisions in a Multi-RAT Wireless Network

<div>Accumulated payment at the end of a few randomly selected episode as a function <span class="nowrap"><svg height="9.25986pt" id="M223" style="vertical-align:-0.2455397pt" version="1.1" viewbox="-0.0498162 -9.01432 11.6478 9.25986" width="11.6478pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M810 587C770 574 761 568 725 543C654 630 540 687 408 687C217 687 69 577 69 430C69 319 160 247 276 247C369 247 511 319 511 495C511 510 512 523 509 538H490C490 354 393 277 274 277C174 277 110 344 110 422C110 559 219 655 415 655C524 655 636 602 695 519C628 463 572 392 516 303C433 170 377 97 313 57C252 89 199 109 143 109C79 109 36 87 36 54C36 15 82 -9 165 -9C222 -9 276 0 326 18C373 -1 423 -15 482 -15C645 -15 804 133 804 324C804 393 783 458 747 514C776 546 781 551 818 567L810 587ZM756 336C756 135 623 16 478 16C439 16 402 24 367 36C386 45 401 56 420 68C505 127 569 206 616 302C647 362 679 427 719 480C744 435 756 387 756 336ZM276 39C243 26 207 21 164 21C106 21 77 32 77 54C77 75 125 79 144 79C191 79 227 66 276 39Z"></path></g></svg>.</span> (a) <span class="nowrap"><svg height="9.25986pt" id="M224" style="vertical-align:-0.2455397pt" version="1.1" viewbox="-0.0498162 -9.01432 60.0634 9.25986" width="60.0634pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M810 587C770 574 761 568 725 543C654 630 540 687 408 687C217 687 69 577 69 430C69 319 160 247 276 247C369 247 511 319 511 495C511 510 512 523 509 538H490C490 354 393 277 274 277C174 277 110 344 110 422C110 559 219 655 415 655C524 655 636 602 695 519C628 463 572 392 516 303C433 170 377 97 313 57C252 89 199 109 143 109C79 109 36 87 36 54C36 15 82 -9 165 -9C222 -9 276 0 326 18C373 -1 423 -15 482 -15C645 -15 804 133 804 324C804 393 783 458 747 514C776 546 781 551 818 567L810 587ZM756 336C756 135 623 16 478 16C439 16 402 24 367 36C386 45 401 56 420 68C505 127 569 206 616 302C647 362 679 427 719 480C744 435 756 387 756 336ZM276 39C243 26 207 21 164 21C106 21 77 32 77 54C77 75 125 79 144 79C191 79 227 66 276 39Z"></path></g><g transform="matrix(.013,0,0,-0.013,15.137,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,26.4,0)"><path d="M456 178V225H360V632H320C217 496 115 347 20 206V178H280V106C280 40 276 34 189 27V0H445V27C364 34 360 39 360 106V178H456ZM280 225H82C149 335 214 431 278 520H280V225Z"></path></g><g transform="matrix(.013,0,0,-0.013,34.815,0)"><path d="M797 0V26C739 32 732 36 732 103V296C732 394 682 449 605 449C576 449 550 437 529 423C504 407 475 389 446 366C425 418 382 449 334 449C303 449 279 437 253 421C222 403 201 385 180 371V452C135 432 85 419 41 411V388C99 379 102 374 102 310V103C102 38 93 32 27 26V0H238V26C189 32 180 38 180 103V338C210 363 250 390 289 390C351 390 377 348 377 275V103C377 37 368 32 306 26V0H520V26C465 32 456 38 456 101V296C456 314 455 326 453 338C491 369 529 390 565 390C628 390 653 345 653 274V107C653 36 642 32 583 26V0H797Z"></path></g><g transform="matrix(.013,0,0,-0.013,45.462,0)"><path d="M135 536C164 536 186 560 186 587C186 617 164 639 136 639C109 639 85 617 85 587C85 560 109 536 135 536ZM252 0V26C188 32 181 38 181 106V451C138 433 90 420 39 412V388C99 379 102 374 102 312V106C102 38 95 32 32 26V0H252Z"></path></g><g transform="matrix(.013,0,0,-0.013,48.946,0)"><path d="M524 0V26C466 32 460 36 460 104V297C460 393 411 449 331 449C302 449 276 437 248 419C223 402 201 387 181 372V451C137 432 90 420 42 411V388C96 378 102 374 102 310V104C102 38 97 33 29 26V0H246V26C187 32 181 36 181 104V339C211 365 250 390 290 390C357 390 381 345 381 276V109C381 40 374 32 315 26V0H524Z"></path></g><g transform="matrix(.013,0,0,-0.013,55.809,0)"><path d="M298 36L289 62C276 55 253 45 228 45C202 45 169 60 169 141V397H276C289 405 292 426 282 437H169V574L155 576L90 509V437H45L17 408L21 397H90V107C90 28 125 -12 188 -12C198 -12 213 -8 230 1L298 36Z"></path></g></svg>.</span> (b) <span class="nowrap"><svg height="9.25986pt" id="M225" style="vertical-align:-0.2455397pt" version="1.1" viewbox="-0.0498162 -9.01432 60.0634 9.25986" width="60.0634pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M810 587C770 574 761 568 725 543C654 630 540 687 408 687C217 687 69 577 69 430C69 319 160 247 276 247C369 247 511 319 511 495C511 510 512 523 509 538H490C490 354 393 277 274 277C174 277 110 344 110 422C110 559 219 655 415 655C524 655 636 602 695 519C628 463 572 392 516 303C433 170 377 97 313 57C252 89 199 109 143 109C79 109 36 87 36 54C36 15 82 -9 165 -9C222 -9 276 0 326 18C373 -1 423 -15 482 -15C645 -15 804 133 804 324C804 393 783 458 747 514C776 546 781 551 818 567L810 587ZM756 336C756 135 623 16 478 16C439 16 402 24 367 36C386 45 401 56 420 68C505 127 569 206 616 302C647 362 679 427 719 480C744 435 756 387 756 336ZM276 39C243 26 207 21 164 21C106 21 77 32 77 54C77 75 125 79 144 79C191 79 227 66 276 39Z"></path></g><g transform="matrix(.013,0,0,-0.013,15.137,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,26.4,0)"><path d="M249 635C141 635 70 555 70 471C70 401 114 353 179 316C143 294 106 267 90 252C68 231 45 202 45 157C45 50 130 -12 237 -12C322 -12 435 52 435 169C435 256 372 304 303 343C349 374 375 398 383 407C401 429 411 458 411 487C411 569 344 635 249 635ZM238 603C285 603 337 567 337 482C337 422 310 385 276 358C205 393 145 426 145 500C145 552 179 603 238 603ZM248 20C183 20 125 70 125 163C125 218 158 268 206 300C284 261 355 217 355 143C355 66 308 20 248 20Z"></path></g><g transform="matrix(.013,0,0,-0.013,34.815,0)"><path d="M797 0V26C739 32 732 36 732 103V296C732 394 682 449 605 449C576 449 550 437 529 423C504 407 475 389 446 366C425 418 382 449 334 449C303 449 279 437 253 421C222 403 201 385 180 371V452C135 432 85 419 41 411V388C99 379 102 374 102 310V103C102 38 93 32 27 26V0H238V26C189 32 180 38 180 103V338C210 363 250 390 289 390C351 390 377 348 377 275V103C377 37 368 32 306 26V0H520V26C465 32 456 38 456 101V296C456 314 455 326 453 338C491 369 529 390 565 390C628 390 653 345 653 274V107C653 36 642 32 583 26V0H797Z"></path></g><g transform="matrix(.013,0,0,-0.013,45.462,0)"><path d="M135 536C164 536 186 560 186 587C186 617 164 639 136 639C109 639 85 617 85 587C85 560 109 536 135 536ZM252 0V26C188 32 181 38 181 106V451C138 433 90 420 39 412V388C99 379 102 374 102 312V106C102 38 95 32 32 26V0H252Z"></path></g><g transform="matrix(.013,0,0,-0.013,48.946,0)"><path d="M524 0V26C466 32 460 36 460 104V297C460 393 411 449 331 449C302 449 276 437 248 419C223 402 201 387 181 372V451C137 432 90 420 42 411V388C96 378 102 374 102 310V104C102 38 97 33 29 26V0H246V26C187 32 181 36 181 104V339C211 365 250 390 290 390C357 390 381 345 381 276V109C381 40 374 32 315 26V0H524Z"></path></g><g transform="matrix(.013,0,0,-0.013,55.809,0)"><path d="M298 36L289 62C276 55 253 45 228 45C202 45 169 60 169 141V397H276C289 405 292 426 282 437H169V574L155 576L90 509V437H45L17 408L21 397H90V107C90 28 125 -12 188 -12C198 -12 213 -8 230 1L298 36Z"></path></g></svg>.</span></div>

Journal of Computer Networks and Communications

fig5

Figure 5

Figure 5: Development of an AI-Enabled Q-Agent for Making Data Offloading Decisions in a Multi-RAT Wireless Network 