Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes

<div>The optimal policy <svg height="13.532pt" id="M427" style="vertical-align:-3.4295pt" version="1.1" viewbox="-0.0498162 -10.1025 30.88 13.532" width="30.88pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M619 670C619 686 593 712 555 712S459 686 410 634S335 504 320 430H250L219 400L222 388H312L258 73C223 -133 201 -166 187 -180C175 -191 158 -199 140 -199C123 -199 88 -188 74 -172C68 -166 63 -164 54 -171C38 -185 23 -201 23 -215C23 -236 60 -261 93 -261C122 -261 161 -247 207 -200C268 -138 300 -71 337 94C365 220 376 277 394 387L501 399L521 430H401C432 623 464 665 501 665C524 665 544 651 567 627C577 617 583 618 592 625C601 631 619 651 619 670Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,8.352,-5.741)"><path d="M486 158C486 177 478 202 466 220C413 228 386 236 336 262C386 288 413 297 466 304C478 323 486 347 485 366C470 376 444 381 422 380C389 338 368 319 321 288C323 345 329 372 349 422C339 442 322 461 305 470C289 461 271 442 262 422C281 372 287 345 290 288C243 319 222 338 189 380C167 381 142 376 125 366C125 347 133 322 145 304C198 296 225 288 275 262C225 236 198 227 145 220C133 201 125 177 126 158C141 148 167 143 189 144C222 186 243 205 290 236C288 179 282 152 262 102C272 82 289 63 306 54C322 63 340 82 350 102C330 152 324 179 321 236C368 205 390 186 422 144C444 143 470 148 486 158Z"></path></g><g transform="matrix(.013,0,0,-0.013,14.431,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,18.929,0)"><path d="M536 404C536 423 520 448 491 448C445 448 398 404 308 283L286 338C255 416 242 448 217 448C182 448 138 402 92 341L111 321C149 368 169 378 178 378C188 378 198 363 214 320L254 212C185 117 138 65 107 65C98 65 85 69 82 75C79 82 73 86 65 86C44 86 23 60 23 39C23 7 44 -12 71 -12C119 -12 168 33 265 177L306 61C321 17 347 -12 373 -12C413 -12 465 33 507 96L491 119C459 84 432 60 413 60C395 60 378 92 358 148L321 250C341 279 369 310 389 332C417 363 439 382 456 382C466 382 475 376 481 368C486 361 492 358 496 358C513 358 536 381 536 404Z"></path></g><g transform="matrix(.013,0,0,-0.013,26.196,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> and <svg height="14.0461pt" id="M428" style="vertical-align:-3.9436pt" version="1.1" viewbox="-0.0498162 -10.1025 30.88 14.0461" width="30.88pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M619 670C619 686 593 712 555 712S459 686 410 634S335 504 320 430H250L219 400L222 388H312L258 73C223 -133 201 -166 187 -180C175 -191 158 -199 140 -199C123 -199 88 -188 74 -172C68 -166 63 -164 54 -171C38 -185 23 -201 23 -215C23 -236 60 -261 93 -261C122 -261 161 -247 207 -200C268 -138 300 -71 337 94C365 220 376 277 394 387L501 399L521 430H401C432 623 464 665 501 665C524 665 544 651 567 627C577 617 583 618 592 625C601 631 619 651 619 670Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,8.352,-5.741)"><path d="M486 158C486 177 478 202 466 220C413 228 386 236 336 262C386 288 413 297 466 304C478 323 486 347 485 366C470 376 444 381 422 380C389 338 368 319 321 288C323 345 329 372 349 422C339 442 322 461 305 470C289 461 271 442 262 422C281 372 287 345 290 288C243 319 222 338 189 380C167 381 142 376 125 366C125 347 133 322 145 304C198 296 225 288 275 262C225 236 198 227 145 220C133 201 125 177 126 158C141 148 167 143 189 144C222 186 243 205 290 236C288 179 282 152 262 102C272 82 289 63 306 54C322 63 340 82 350 102C330 152 324 179 321 236C368 205 390 186 422 144C444 143 470 148 486 158Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,6.721,3.784)"><path d="M485 418C485 434 469 451 441 451C389 451 312 387 255 335C221 304 194 278 159 242H157L259 678C264 698 264 710 255 710C242 710 183 681 96 673L90 643L128 642C165 641 171 640 162 602L24 -5L31 -12C51 -4 81 4 110 9L145 181C154 195 180 220 205 242C233 170 263 106 291 53C318 1 335 -12 360 -12C383 -12 425 0 483 82L463 105C437 79 411 57 400 57C388 57 378 68 355 110C329 156 288 244 268 298C299 333 355 376 405 376C426 376 440 373 448 368C453 365 465 368 468 373C477 386 485 405 485 418Z"></path></g><g transform="matrix(.013,0,0,-0.013,14.431,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,18.929,0)"><path d="M536 404C536 423 520 448 491 448C445 448 398 404 308 283L286 338C255 416 242 448 217 448C182 448 138 402 92 341L111 321C149 368 169 378 178 378C188 378 198 363 214 320L254 212C185 117 138 65 107 65C98 65 85 69 82 75C79 82 73 86 65 86C44 86 23 60 23 39C23 7 44 -12 71 -12C119 -12 168 33 265 177L306 61C321 17 347 -12 373 -12C413 -12 465 33 507 96L491 119C459 84 432 60 413 60C395 60 378 92 358 148L321 250C341 279 369 310 389 332C417 363 439 382 456 382C466 382 475 376 481 368C486 361 492 358 496 358C513 358 536 381 536 404Z"></path></g><g transform="matrix(.013,0,0,-0.013,26.196,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> as <span class="nowrap"><svg height="9.49473pt" id="M429" style="vertical-align:-0.2063999pt" version="1.1" viewbox="-0.0498162 -9.28833 40.3766 9.49473" width="40.3766pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M480 416C480 431 465 448 438 448C388 448 312 383 252 330C217 299 188 273 155 237H153L257 680C262 700 263 712 253 712C240 712 183 684 97 674L92 648L126 647C166 646 172 645 163 606L23 -6L29 -12C51 -5 77 2 107 8C115 62 130 128 142 180C153 193 179 220 204 241C231 170 259 106 288 54C317 0 336 -12 358 -12C381 -12 423 2 477 80L460 100C434 74 408 54 398 54C385 54 374 65 351 107C326 154 282 241 263 299C296 332 351 377 403 377C424 377 436 372 445 368C449 366 456 368 462 375C472 386 480 402 480 416Z"></path></g><g transform="matrix(.013,0,0,-0.013,10.171,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,21.434,0)"><path d="M384 0V27C293 34 287 42 287 114V635C232 613 172 594 109 583V559L157 557C201 555 205 550 205 499V114C205 42 199 34 109 27V0H384Z"></path></g><g transform="matrix(.013,0,0,-0.013,27.674,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z"></path></g><g transform="matrix(.013,0,0,-0.013,33.914,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z"></path></g></svg>.</span></div>

Discrete Dynamics in Nature and Society

fig2

Figure 2

Figure 2: Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes 