A Multiphase Semistatic Training Method for Swarm Confrontation Using Multiagent Deep Reinforcement Learning

<table class="table-group" id="tab2"><tr><td><table class="table"><tr><td class="thead-hr" colspan="4"><hr/></td></tr><tr class="thead"><td class="align_left">Label</td><td class="align_center">Training steps (millions)</td><td class="align_center">Game statistics (win/lose)</td><td class="align_center">Winning percentage</td></tr><tr><td class="thead-hr" colspan="4"><hr/></td></tr><tr><td class="align_left">1</td><td class="align_center">106</td><td class="align_center">1373/726</td><td class="align_center"><span style="width: 31.7298ptpx;"><svg height="8.85534pt" id="M12" style="vertical-align:-0.4673595pt" version="1.1" viewbox="-0.0498162 -8.38798 31.7298 8.85534" width="31.7298pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M137 343C167 482 260 545 321 574C357 591 397 603 429 609L423 641C382 634 335 622 295 608C189 570 37 457 37 238C37 84 125 -12 242 -12C362 -12 447 89 447 209C447 311 374 393 267 393C247 393 226 386 204 376L137 343ZM227 337C318 337 361 256 361 173C361 105 336 22 258 22C176 22 126 120 126 240C126 266 127 291 132 310C155 323 189 337 227 337Z"></path></g><g transform="matrix(.013,0,0,-0.013,6.24,0)"><path d="M153 550H386L412 615L406 623H120L82 318C104 327 142 338 184 338C294 338 347 275 347 187C347 112 305 39 221 39C160 39 119 71 97 89C88 97 80 96 71 90C59 80 50 67 49 57C48 45 52 36 66 23C80 9 123 -12 169 -12C221 -11 288 15 342 59C403 109 431 165 431 225C431 308 366 395 238 395C212 395 165 379 127 364L153 550Z"></path></g><g transform="matrix(.013,0,0,-0.013,12.48,0)"><path d="M113 -12C146 -12 170 11 170 46C170 78 146 103 114 103S58 78 58 46C58 11 82 -12 113 -12Z"></path></g><g transform="matrix(.013,0,0,-0.013,15.444,0)"><path d="M456 178V225H360V632H320C217 496 115 347 20 206V178H280V106C280 40 276 34 189 27V0H445V27C364 34 360 39 360 106V178H456ZM280 225H82C149 335 214 431 278 520H280V225Z"></path></g><g transform="matrix(.013,0,0,-0.013,21.685,0)"><path d="M594 629L561 652L159 -9L190 -32L594 629ZM194 635C92 635 49 534 49 453C49 374 92 272 193 272C295 272 338 374 338 454S295 635 194 635ZM193 605C257 605 265 504 265 453C265 405 257 302 194 302C133 302 122 403 122 454S133 605 193 605ZM567 351C465 351 421 250 421 169C421 90 465 -12 566 -12S710 90 710 170S667 351 567 351ZM566 322C628 322 637 221 637 169C637 121 628 18 567 18S494 119 494 170C494 223 506 322 566 322Z"></path></g></svg></span></td></tr><tr><td class="align_left">2</td><td class="align_center">130</td><td class="align_center">1214/282</td><td class="align_center"><span style="width: 31.7298ptpx;"><svg height="8.85534pt" id="M13" style="vertical-align:-0.4673595pt" version="1.1" viewbox="-0.0498162 -8.38798 31.7298 8.85534" width="31.7298pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M249 635C141 635 70 555 70 471C70 401 114 353 179 316C143 294 106 267 90 252C68 231 45 202 45 157C45 50 130 -12 237 -12C322 -12 435 52 435 169C435 256 372 304 303 343C349 374 375 398 383 407C401 429 411 458 411 487C411 569 344 635 249 635ZM238 603C285 603 337 567 337 482C337 422 310 385 276 358C205 393 145 426 145 500C145 552 179 603 238 603ZM248 20C183 20 125 70 125 163C125 218 158 268 206 300C284 261 355 217 355 143C355 66 308 20 248 20Z"></path></g><g transform="matrix(.013,0,0,-0.013,6.24,0)"><path d="M384 0V27C293 34 287 42 287 114V635C232 613 172 594 109 583V559L157 557C201 555 205 550 205 499V114C205 42 199 34 109 27V0H384Z"></path></g><g transform="matrix(.013,0,0,-0.013,12.48,0)"><path d="M113 -12C146 -12 170 11 170 46C170 78 146 103 114 103S58 78 58 46C58 11 82 -12 113 -12Z"></path></g><g transform="matrix(.013,0,0,-0.013,15.444,0)"><path d="M384 0V27C293 34 287 42 287 114V635C232 613 172 594 109 583V559L157 557C201 555 205 550 205 499V114C205 42 199 34 109 27V0H384Z"></path></g><g transform="matrix(.013,0,0,-0.013,21.684,0)"><path d="M594 629L561 652L159 -9L190 -32L594 629ZM194 635C92 635 49 534 49 453C49 374 92 272 193 272C295 272 338 374 338 454S295 635 194 635ZM193 605C257 605 265 504 265 453C265 405 257 302 194 302C133 302 122 403 122 454S133 605 193 605ZM567 351C465 351 421 250 421 169C421 90 465 -12 566 -12S710 90 710 170S667 351 567 351ZM566 322C628 322 637 221 637 169C637 121 628 18 567 18S494 119 494 170C494 223 506 322 566 322Z"></path></g></svg></span></td></tr><tr><td class="align_left">3</td><td class="align_center">225</td><td class="align_center">1087/132</td><td class="align_center"><span style="width: 31.7298ptpx;"><svg height="8.85534pt" id="M14" style="vertical-align:-0.4673595pt" version="1.1" viewbox="-0.0498162 -8.38798 31.7298 8.85534" width="31.7298pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M249 635C141 635 70 555 70 471C70 401 114 353 179 316C143 294 106 267 90 252C68 231 45 202 45 157C45 50 130 -12 237 -12C322 -12 435 52 435 169C435 256 372 304 303 343C349 374 375 398 383 407C401 429 411 458 411 487C411 569 344 635 249 635ZM238 603C285 603 337 567 337 482C337 422 310 385 276 358C205 393 145 426 145 500C145 552 179 603 238 603ZM248 20C183 20 125 70 125 163C125 218 158 268 206 300C284 261 355 217 355 143C355 66 308 20 248 20Z"></path></g><g transform="matrix(.013,0,0,-0.013,6.24,0)"><path d="M244 635C114 635 38 519 38 422C38 317 111 240 217 240C236 240 255 244 277 256L345 292C311 140 203 39 59 15L64 -15C89 -15 150 -5 204 17C339 72 440 202 440 386C440 521 368 635 244 635ZM228 602C326 602 352 479 352 390C352 370 351 347 348 324C327 308 293 296 258 296C174 296 124 369 124 458C124 517 152 602 228 602Z"></path></g><g transform="matrix(.013,0,0,-0.013,12.481,0)"><path d="M113 -12C146 -12 170 11 170 46C170 78 146 103 114 103S58 78 58 46C58 11 82 -12 113 -12Z"></path></g><g transform="matrix(.013,0,0,-0.013,15.445,0)"><path d="M412 140C382 77 369 73 315 73H129L270 222C362 320 402 379 402 466C402 571 322 635 234 635C177 635 130 609 99 576L42 495L64 475C90 514 133 568 201 568C274 568 318 519 318 435C318 349 255 267 193 193C144 135 87 78 32 23V0H405C417 45 427 89 440 131L412 140Z"></path></g><g transform="matrix(.013,0,0,-0.013,21.685,0)"><path d="M594 629L561 652L159 -9L190 -32L594 629ZM194 635C92 635 49 534 49 453C49 374 92 272 193 272C295 272 338 374 338 454S295 635 194 635ZM193 605C257 605 265 504 265 453C265 405 257 302 194 302C133 302 122 403 122 454S133 605 193 605ZM567 351C465 351 421 250 421 169C421 90 465 -12 566 -12S710 90 710 170S667 351 567 351ZM566 322C628 322 637 221 637 169C637 121 628 18 567 18S494 119 494 170C494 223 506 322 566 322Z"></path></g></svg></span></td></tr><tr class="table-tr"><td colspan="4"><hr class="tbody-hr"/></td></tr></table></td></tr></table>

<div>The initial confrontation information of different strong teams vs. the weak team.</div>

Computational Intelligence and Neuroscience

tab2

Table 2

Table 2: A Multiphase Semistatic Training Method for Swarm Confrontation Using Multiagent Deep Reinforcement Learning 