Optimal Path Planning for Wireless Power Transfer Robot Using Area Division Deep Reinforcement Learning

<div>The average rewards of <span class="nowrap"><svg height="12.4176pt" id="M289" style="vertical-align:-3.18147pt" version="1.1" viewbox="-0.0498162 -9.23613 43.4225 12.4176" width="43.4225pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M220 349V455C158 436 98 425 38 417V389C85 378 92 374 92 323V104C92 43 83 38 28 32V0H302V32C225 38 220 43 220 104V291C237 330 257 343 276 343C292 343 305 336 322 322C329 316 339 313 353 318C379 328 402 356 402 392C402 426 374 454 335 454C297 454 259 411 222 349H220Z"></path></g><g transform="matrix(.013,0,0,-0.013,5.272,0)"><path d="M400 123C365 93 327 80 288 80C216 80 157 134 153 241C251 245 374 254 392 257C414 260 420 273 421 293C421 375 351 454 258 454H257C203 454 148 427 104 383C62 341 30 278 30 202C30 87 103 -12 237 -12C279 -12 357 8 417 99L400 123ZM237 414C276 414 299 362 298 311C298 294 292 291 277 291C236 290 199 289 157 288C164 367 198 414 237 414Z"></path></g><g transform="matrix(.013,0,0,-0.013,10.974,0)"><path d="M699 442H515V410C577 401 582 389 573 351C563 308 532 208 517 161H515C476 254 437 345 401 437H362C326 342 293 250 258 163H256C237 225 217 294 197 358C186 392 192 401 250 410V442H3V410C45 405 55 395 71 347C108 235 151 112 191 -11H229C267 84 302 175 340 268C377 177 418 83 457 -11H495C545 129 587 249 623 342C644 394 651 402 699 410V442Z"></path></g><g transform="matrix(.013,0,0,-0.013,19.886,0)"><path d="M465 45L455 77C444 72 433 69 421 69C404 69 389 81 389 128V293C389 349 380 392 337 425C310 445 280 454 239 454C182 443 111 409 73 383C51 366 38 353 38 335C38 309 74 285 101 285C121 285 136 293 143 319C151 350 160 374 171 388C180 400 191 406 205 406C241 406 265 371 265 308V278C242 258 165 234 106 212C54 192 30 168 30 116C30 54 81 -12 165 -12C185 -12 232 21 270 46C274 34 281 21 295 9C308 -3 329 -12 356 -12L465 45ZM265 79C249 67 238 63 223 63C200 63 163 84 163 139C163 174 179 192 210 208C231 219 247 229 265 239V79Z"></path></g><g transform="matrix(.013,0,0,-0.013,25.967,0)"><path d="M220 349V455C158 436 98 425 38 417V389C85 378 92 374 92 323V104C92 43 83 38 28 32V0H302V32C225 38 220 43 220 104V291C237 330 257 343 276 343C292 343 305 336 322 322C329 316 339 313 353 318C379 328 402 356 402 392C402 426 374 454 335 454C297 454 259 411 222 349H220Z"></path></g><g transform="matrix(.013,0,0,-0.013,31.239,0)"><path d="M541 58L519 60C477 65 472 69 472 126V703C421 688 341 674 277 667V636C338 628 344 624 344 570V449C322 452 311 454 303 454C147 454 29 343 29 207C29 58 147 -12 233 -12C244 -12 259 -9 283 6L344 45V-13C423 10 517 22 541 26V58ZM344 80C322 66 307 64 289 64C243 64 171 111 171 234C171 386 248 412 271 412C301 412 328 395 344 360V80Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,38.376,3.132)"><path d="M389 0V32C297 38 291 46 291 118V635C234 613 175 595 109 583V556L161 554C203 552 207 547 207 497V118C207 46 201 38 110 32V0H389Z"></path></g></svg>,</span> <span class="nowrap"><svg height="12.4176pt" id="M290" style="vertical-align:-3.18147pt" version="1.1" viewbox="-0.0498162 -9.23613 43.4225 12.4176" width="43.4225pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M220 349V455C158 436 98 425 38 417V389C85 378 92 374 92 323V104C92 43 83 38 28 32V0H302V32C225 38 220 43 220 104V291C237 330 257 343 276 343C292 343 305 336 322 322C329 316 339 313 353 318C379 328 402 356 402 392C402 426 374 454 335 454C297 454 259 411 222 349H220Z"></path></g><g transform="matrix(.013,0,0,-0.013,5.272,0)"><path d="M400 123C365 93 327 80 288 80C216 80 157 134 153 241C251 245 374 254 392 257C414 260 420 273 421 293C421 375 351 454 258 454H257C203 454 148 427 104 383C62 341 30 278 30 202C30 87 103 -12 237 -12C279 -12 357 8 417 99L400 123ZM237 414C276 414 299 362 298 311C298 294 292 291 277 291C236 290 199 289 157 288C164 367 198 414 237 414Z"></path></g><g transform="matrix(.013,0,0,-0.013,10.974,0)"><path d="M699 442H515V410C577 401 582 389 573 351C563 308 532 208 517 161H515C476 254 437 345 401 437H362C326 342 293 250 258 163H256C237 225 217 294 197 358C186 392 192 401 250 410V442H3V410C45 405 55 395 71 347C108 235 151 112 191 -11H229C267 84 302 175 340 268C377 177 418 83 457 -11H495C545 129 587 249 623 342C644 394 651 402 699 410V442Z"></path></g><g transform="matrix(.013,0,0,-0.013,19.886,0)"><path d="M465 45L455 77C444 72 433 69 421 69C404 69 389 81 389 128V293C389 349 380 392 337 425C310 445 280 454 239 454C182 443 111 409 73 383C51 366 38 353 38 335C38 309 74 285 101 285C121 285 136 293 143 319C151 350 160 374 171 388C180 400 191 406 205 406C241 406 265 371 265 308V278C242 258 165 234 106 212C54 192 30 168 30 116C30 54 81 -12 165 -12C185 -12 232 21 270 46C274 34 281 21 295 9C308 -3 329 -12 356 -12L465 45ZM265 79C249 67 238 63 223 63C200 63 163 84 163 139C163 174 179 192 210 208C231 219 247 229 265 239V79Z"></path></g><g transform="matrix(.013,0,0,-0.013,25.967,0)"><path d="M220 349V455C158 436 98 425 38 417V389C85 378 92 374 92 323V104C92 43 83 38 28 32V0H302V32C225 38 220 43 220 104V291C237 330 257 343 276 343C292 343 305 336 322 322C329 316 339 313 353 318C379 328 402 356 402 392C402 426 374 454 335 454C297 454 259 411 222 349H220Z"></path></g><g transform="matrix(.013,0,0,-0.013,31.239,0)"><path d="M541 58L519 60C477 65 472 69 472 126V703C421 688 341 674 277 667V636C338 628 344 624 344 570V449C322 452 311 454 303 454C147 454 29 343 29 207C29 58 147 -12 233 -12C244 -12 259 -9 283 6L344 45V-13C423 10 517 22 541 26V58ZM344 80C322 66 307 64 289 64C243 64 171 111 171 234C171 386 248 412 271 412C301 412 328 395 344 360V80Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,38.376,3.132)"><path d="M414 144C384 79 371 75 317 75H135L276 221C367 316 408 376 408 465C408 570 327 635 237 635C179 635 131 609 100 575L42 494L67 471C94 510 138 565 205 565C277 565 321 517 321 435C321 348 258 270 195 195C146 137 88 81 33 26V0H411C423 44 433 88 446 135L414 144Z"></path></g></svg>,</span> and <svg height="12.5272pt" id="M291" style="vertical-align:-3.29107pt" version="1.1" viewbox="-0.0498162 -9.23613 43.4225 12.5272" width="43.4225pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M220 349V455C158 436 98 425 38 417V389C85 378 92 374 92 323V104C92 43 83 38 28 32V0H302V32C225 38 220 43 220 104V291C237 330 257 343 276 343C292 343 305 336 322 322C329 316 339 313 353 318C379 328 402 356 402 392C402 426 374 454 335 454C297 454 259 411 222 349H220Z"></path></g><g transform="matrix(.013,0,0,-0.013,5.272,0)"><path d="M400 123C365 93 327 80 288 80C216 80 157 134 153 241C251 245 374 254 392 257C414 260 420 273 421 293C421 375 351 454 258 454H257C203 454 148 427 104 383C62 341 30 278 30 202C30 87 103 -12 237 -12C279 -12 357 8 417 99L400 123ZM237 414C276 414 299 362 298 311C298 294 292 291 277 291C236 290 199 289 157 288C164 367 198 414 237 414Z"></path></g><g transform="matrix(.013,0,0,-0.013,10.974,0)"><path d="M699 442H515V410C577 401 582 389 573 351C563 308 532 208 517 161H515C476 254 437 345 401 437H362C326 342 293 250 258 163H256C237 225 217 294 197 358C186 392 192 401 250 410V442H3V410C45 405 55 395 71 347C108 235 151 112 191 -11H229C267 84 302 175 340 268C377 177 418 83 457 -11H495C545 129 587 249 623 342C644 394 651 402 699 410V442Z"></path></g><g transform="matrix(.013,0,0,-0.013,19.886,0)"><path d="M465 45L455 77C444 72 433 69 421 69C404 69 389 81 389 128V293C389 349 380 392 337 425C310 445 280 454 239 454C182 443 111 409 73 383C51 366 38 353 38 335C38 309 74 285 101 285C121 285 136 293 143 319C151 350 160 374 171 388C180 400 191 406 205 406C241 406 265 371 265 308V278C242 258 165 234 106 212C54 192 30 168 30 116C30 54 81 -12 165 -12C185 -12 232 21 270 46C274 34 281 21 295 9C308 -3 329 -12 356 -12L465 45ZM265 79C249 67 238 63 223 63C200 63 163 84 163 139C163 174 179 192 210 208C231 219 247 229 265 239V79Z"></path></g><g transform="matrix(.013,0,0,-0.013,25.967,0)"><path d="M220 349V455C158 436 98 425 38 417V389C85 378 92 374 92 323V104C92 43 83 38 28 32V0H302V32C225 38 220 43 220 104V291C237 330 257 343 276 343C292 343 305 336 322 322C329 316 339 313 353 318C379 328 402 356 402 392C402 426 374 454 335 454C297 454 259 411 222 349H220Z"></path></g><g transform="matrix(.013,0,0,-0.013,31.239,0)"><path d="M541 58L519 60C477 65 472 69 472 126V703C421 688 341 674 277 667V636C338 628 344 624 344 570V449C322 452 311 454 303 454C147 454 29 343 29 207C29 58 147 -12 233 -12C244 -12 259 -9 283 6L344 45V-13C423 10 517 22 541 26V58ZM344 80C322 66 307 64 289 64C243 64 171 111 171 234C171 386 248 412 271 412C301 412 328 395 344 360V80Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,38.376,3.132)"><path d="M290 377C321 398 342 415 358 430C378 450 389 473 389 502C389 578 329 635 238 635H237C184 635 137 610 109 578L64 515L88 493C112 529 154 573 208 573S303 542 303 482C303 409 233 370 141 341L149 308C165 313 190 319 215 319C272 319 341 283 341 193C342 98 292 43 222 43C163 43 122 72 96 94C88 101 79 100 70 94C61 87 47 73 46 60C44 47 48 37 62 23C76 10 118 -12 165 -12C238 -12 430 62 430 223C430 297 379 359 290 375V377Z"></path></g></svg> versus the training episodes in area II of the experimental field.</div>

Wireless Power Transfer

fig6

Figure 6

Figure 6: Optimal Path Planning for Wireless Power Transfer Robot Using Area Division Deep Reinforcement Learning 