{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import numpy as np\n", "from matplotlib import pyplot as plt\n", "from __future__ import division\n", "\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Sampling methods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sampling methods allow to draw random samples from a given probability distribution. While special methods efficiently draw samples from common distribution, e.g. Gaussian, Poisson, ..., more general methods can draw samples from complicated, high-dimensional distributions:\n", "\n", "* Random number generators in standard libraries employ sampling methods\n", "* Bayesian models can often not be solved exactly as the normalization constant of the posterior cannot be computed in closed form and approximation or sampling methods are needed.\n", "\n", "Samples are generally useful, as they can be used to approximate expectations. Consider the expectation of $f(x)$ under some distribution with density $p(x)$, i.e.\n", "$$ \\mathbb{E}_p[f] = \\int f(x) p(x) dx $$\n", "Then, if samples $x_1,\\ldots,x_N$ from $p(x)$ are available the above expectation can be approximated as\n", "$$ \\mathbb{E}_p[f] \\approx \\frac{1}{N} \\sum_{i=1}^N f(x_i), $$\n", "i.e. by integrating $f$ over the empirical measure of the samples:\n", "$$ P_{N} = \\frac{1}{N} \\sum_{i=1}^N \\delta_{x_i} $$\n", "where $\\delta_x(A) = \\mathbb{1}_A(x)$ denotes the Dirac measure" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Why is Monte-Carlo sampling so popular? Note that to solve the (possibly high-dimensional) integral $\\int f(x) p(x) dx$ the most important points are where the integrand $f(x) p(x)$ take high values. Monte-Carlo sampling, by drawing from the distribution $p$, automatically focuses on point with high probability $p(x)$.\n", "\n", "As any statistical procedure, the quality of the Monte-Carlo approximation $\\hat{\\mu} = \\frac{1}{N} \\sum_{i=1}^N f(x_i)$ for $\\mathbb{E}_p[f] = \\mu$ can be evaluated in terms of\n", "\n", "* **bias**, i.e. $\\mathbb{E}_p[\\hat{\\mu}] - \\mu = 0$\n", "* and **variance**, i.e. $\\mathbb{V}ar_p[\\hat{\\mu}] = \\mathbb{E}_p[(\\hat{\\mu} - \\mu)^2]$\n", "\n", "**Exercise**: Show that Monte-Carlo estimation is unbiased.\n", "\n", "To compute the variance, denote the variance of a single sample $x$ as $\\sigma^2 = \\mathbb{E}_p[ (f(x) - \\mu)^2 ] = \\mathbb{V}ar_p[f]$. Then, since samples are independent, the variance of $\\hat{\\mu}$ is found to be\n", "$$ \\mathbb{V}ar_p[\\frac{1}{N} \\sum_{i=1}^N f(x_i)] = \\frac{1}{N^2} \\sum_{i=1}^N \\mathbb{V}ar_p[f(x_i)] = \\frac{1}{N} \\sigma^2 $$\n", "Interestingly, this variance scales as $\\frac{1}{N}$ independent of the dimension of the space. Basically, this observation is the reason why Monte-Carlo integration is effective in high-dimensions.\n", "\n", "Note: Also probabilities can be expressed as expectations, i.e.\n", "$$ Pr(X \\in A) = \\int_A p(x) dx = \\int \\mathbb{1}_A(x) p(x) dx = \\mathbb{E}_p[\\mathbb{1}_A] $$\n", "Basically, this is the reason that the histogram of the samples can be used to vizualize their underlying probability density." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Standard distribution\n", "\n", "Most basic sampling methods assume that a source for uniform random numbers is available.\n", "\n", "Let $U$ denote a uniform random variable, i.e. $u \\in [0,1]$ and $P(U \\leq u) = u$. Then, the distribution of $X = f(U)$ has the probability distribution\n", "$$ P(X \\leq x) = P(f(U) \\leq x) = P(U \\leq f^{-1}(x)) = f^{-1}(x)$$\n", "assuming that $f$ is invertible.\n", "\n", "Now, suppose we want to sample from some distribution with distribution function $P(X \\leq x) = h(x)$. How do we have to choose $f$ such that $P(X \\leq x)$ has the desired form?\n", "\n", "From the above calculation, we see that $f = h^{-1}$ achieves the desired result. Thus, the following algorithm allows to draw samples from $X$ with distribution function $h(x)$:\n", "\n", "* Draw a uniform random number $u \\in [0,1]$\n", "* $x = h^{-1}(u)$ is then a sample from the desired distribution" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The next plot illustrates this method for the standard Gaussian distribution with density\n", "$$ p(x) = \\frac{1}{\\sqrt{2 \\pi}} e^{-\\frac{1}{2} x^2} $$" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from scipy.stats import norm\n", "\n", "def phi(x):\n", " return norm.pdf(x)\n", "\n", "def Phi(x):\n", " return norm.cdf(x)\n", "\n", "def Phi_inv(x):\n", " return norm.ppf(x)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEACAYAAACuzv3DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VNXaxuHfhBAg9CK9CUgHAQGpMjQBERBUmooKePAI\nHOXwIXDwQCy0g6goKEVQSqRLUwRBCL1Lk9577xAgbX9/7IAhpExgZvaU576uXJnJrMy8Y8LjyrvX\n2htERERERERERERERERERERERERERJxuAnAO2JnEmK+BA8B2oKI7ihIREcfVxgznxIL8BWBh7O1n\ngfXuKEpERFKmMIkH+WigTZz7e4Fcri5IRERMAU54jnzAiTj3TwL5nfC8IiLiAGcEOYAt3n3DSc8r\nIiLJCHTCc5wCCsS5nz/2aw8oWrSocejQISe8nIiIXzkEFEtqgDNm5POBDrG3qwFXMVe5PFjJoUMY\nhuGzHwMGDLC8Br0/vbeUvL9btwz++MMgJMSgYUODnDkNsmQxqF3b4N13DYYNM5gxw2DDBoOzZw2i\no61/L/748wOKJhfCjszIpwJ1gByYvfABQOrYx8Zgrlh5ATgI3ALeduA5RcQCR4/C/Pnmx/r1UL48\n1K4N3btDxYqQLx/Y4jdKxeM5EuTtHBjT7XELERHXuHsXvv8eJkyAAwfgxRehWzeYNw/Sp7e6OnEG\nZ/TIBbDb7VaX4FK+/P589b0dPw5ffAHff2+nYUPo2xcaN4bUqZP/Xm/iqz+/lHDnH1FGbL9HRFzo\nxAno39+ccXfqBB98YLZMxDvZzF5XklntrOWHImKxmzfhv/+FChXM4D50CIYNU4j7A7VWRHzAsmXQ\nuTNUrw7btkGBAsl/j/gOBbmIF7t9G3r2hF9+gTFjoEkTqysSKyjIRbzU4cPw8stQogTs3AmZM1td\nkVhFPXIRL7R4MVSrBh07wtSpCnF/pxm5iJeZOBE+/BB+/hlq1bK6GvEECnIRLzJ0KHz3HYSFQalS\nVlcjnkJBLuIlBg2CyZNh7VrIm9fqasSTKMhFvMAXX8CPP8KKFZAnj9XViKdRkIt4uLFjYdQohbgk\nTlv0RTzY779Dhw6wejUUS/KM1OKrHNmirxm5iIfatQtefx1mz1aIS9K0jlzEA12+DM2amb3x2rWt\nrkY8nVorIh7GMKBFCyhaFL780upqxGpqrYh4oS++gHPnYNYsqysRb6EZuYgHWb/enI1v3AiFClld\njXgCnY9cxIuEh8Mbb5g7NxXikhKakYt4iA8+gIsXYcoUqysRT6IeuYiXWLECZs40T0crklJqrYhY\nLDzcPB3t6NGQLZvV1Yg3UpCLWGzIEHjmGXPduMijUI9cxEIHD5oXiNi2DfLnt7oa8URatSLiwQwD\nuneH3r0V4vJ4FOQiFpk3D44fN1eriDwOtVZELBAZCaVLm6enff55q6sRT6bWioiHGjsWihRRiItz\naEYu4mbXr0Px4rBoEVSoYHU14uk0IxfxQMOGQaNGCnFxHs3IRdzo/HkoVQq2boWCBa2uRryBZuQi\nHmbYMGjXTiEuzqUZuYibnD8PJUvCjh1aNy6O04xcxIMMHw5t2yrExfk0Ixdxg4sXoUQJcyt+gQJW\nVyPeRDNyEQ8xfDi0bq0QF9dwJMgbA3uBA0DvBB7PASwCtgF/AW85qzgRX3DtmrkBqE8fqysRX5Vc\nkKcCRmKGeWmgHVAq3phuwFagAmAHhqMLVojc9/335rpxXb5NXCW5wK0KHASOxt6fBrQA9sQZcwYo\nH3s7E3AJiHJeiSLeKzISRoyAuXOtrkR8WXJBng84Eef+SeDZeGPGAcuA00BGoLXTqhPxcjNmwFNP\nQaVKVlciviy5IHdkmcl/MPvjdqAosAR4GrgRf2BISMj923a7Hbvd7liVIl7IMODzz2HQIKsrEW8S\nFhZGWFhYir4nueWH1YAQzB45QF8gBhgaZ8xCYCCwJvb+H5gHRTfHey4tPxS/snQpvP8+/PUX2Ny5\n0Fd8ijOWH24GngIKA0FAG2B+vDF7gQaxt3MBJYDDKStVxPd89RX8+98KcXE9R37FmgBfYa5gGQ8M\nBrrEPjYGc/nhD0BBzP8xDAZ+SuB5NCMXv3HkCFStCseOQXCw1dWIN3NkRq6dnSIu0Ls3REebPXKR\nx6EgF7HAnTvm2Q3XroVixayuRrydtuiLWGD6dHjmGYW4uI+CXMTJRo2Crl2trkL8iYJcxIk2bTLP\nO96kidWViD9RkIs40ejR8O67kCqV1ZWIP9HBThEnuXHDPE3tvn2QK5fV1Yiv0MFOETeaMQPsdoW4\nuJ+CXMRJxo+HTp2srkL8kVorIk6wZw/UqwcnTkCgzsYvTqTWioibTJgAb76pEBdraEYu8pgiI82D\nnCtWmBdYFnEmzchF3ODXX82LRyjExSoKcpHHNGECdOxodRXiz9RaEXkM586ZM/GTJyFDBqurEV+k\n1oqIi02fDs2aKcTFWgpykccQGgqvv251FeLv1FoReUQHDkDt2mZbRcsOxVXUWhFxodBQaNtWIS7W\n06+gyCMwDJgyBaZNs7oSEc3IRR7Jhg3mqWqfecbqSkQU5CKP5N5BTps7jzKJJEIHO0VSKDIS8uWD\ndeugaFGrqxFfp4OdIi6wZIl5YWWFuHgKBblICoWGwmuvWV2FyN/UWhFJgdu3IU8e2LsXcue2uhrx\nB2qtiDjZokVQqZJCXDyLglwkBWbMgNatra5C5EFqrYg4KDzcbKscOAA5c1pdjfgLtVZEnOi336BK\nFYW4eB4FuYiD1FYRT6XWiogDbt2CvHnh4EF44gmrqxF/otaKiJMsXAjPPqsQF8+kIBdxgNoq4snU\nWhFJxs2b5rlVDh+G7Nmtrkb8jVorIk7w669QvbpCXDyXI0HeGNgLHAB6JzLGDmwF/gLCnFGYiKdQ\nW0U8XXKtlVTAPqABcArYBLQD9sQZkwVYAzQCTgI5gIsJPJdaK+J1bt40V6scPQrZslldjfgjZ7RW\nqgIHgaNAJDANaBFvTHtgNmaIQ8IhLuKVfvvNbKsoxMWTJRfk+YATce6fjP1aXE8B2YDlwGbgDadV\nJ2KxOXOgZUurqxBJWnIXX3akF5IaqATUB4KBdcB6zJ66iNe6e9dcPz58uNWViCQtuSA/BRSIc78A\nf7dQ7jmB2U65HfuxEniaBII8JCTk/m273Y7dbk9pvSJus2wZlCljnihLxF3CwsIICwtL0fckd7Az\nEPNgZ33gNLCRhw92lgRGYh7sTANsANoAu+M9lw52ild55x0oWRJ69rS6EvFnzjjYGQV0AxZjBvN0\nzBDvEvsB5tLERcAOzBAfx8MhLuJVoqNh3jz1x8U7aGenSAJWroR//Qu2bbO6EvF32tkp8oi0WkW8\nSXIHO0X8jmGYQb5ggdWViDhGM3KReLZuhcBAKFvW6kpEHKMgF4lnzhxo1Qps7jyCJPIYFOQi8fz8\ns/rj4l0U5CJx7NsHV66YVwMS8RYKcpE45syBl16CAP3LEC+iX1eROO71x0W8iTYEicQ6eRKefhrO\nnoXUqa2uRsSkDUEiKTB3Lrz4okJcvI+CXCSWdnOKt1JrRQS4dAmKFIEzZyA42OpqRP6m1oqIgxYs\ngAYNFOLinRTkIpibgLRaRbyVWivi927ehLx54fhxyJLF6mpEHqTWiogDfvsNqldXiIv3UpCL35s7\nV6tVxLuptSJ+LSICcueGXbt0kWXxTGqtiCRj+XLzAssKcfFmCnLxa9oEJL5ArRXxWzExkC+feaHl\np56yuhqRhKm1IpKE9eshe3aFuHg/Bbn4La1WEV+hIBe/ZBjqj4vvUJCLX9q1CyIjoWJFqysReXwK\ncvFL9y7pZnPn4X4RF1GQi1+6F+QivkDLD8XvHDsGlSub5x4PDLS6GpGkafmhSALmzoVmzRTi4jsU\n5OJ3tFpFfI1aK+JXLlyAYsXg7FlIl87qakSSp9aKSDwLFkDDhgpx8S0KcvEr2s0pvkitFfEbuqSb\neCO1VkTiWLRIl3QT36QgF7+hTUDiqxwJ8sbAXuAA0DuJcVWAKKCVE+oScaqICFi4EFq0sLoSEedL\nLshTASMxw7w00A4olci4ocAi3Nt3F3HIvUu65c1rdSUizpdckFcFDgJHgUhgGpDQnKY7MAu44Mzi\nRJxl9mx45RWrqxBxjeQ2KecDTsS5fxJ4NoExLYB6mO0VLU0RjxIVZfbHN260uhIR10guyB0J5a+A\nPrFjbai1Ih5m5UooVAiefNK8HxkdyZLDS1hxdAX7Lu0jPDKcHME5KJuzLE2KNaFC7gr3lnyJeIXk\ngvwUUCDO/QKYs/K4nsFsuQDkAJpgtmHmx3+ykJCQ+7ftdjt2uz1FxYo8ilmzzLbKnag7fLfpO4au\nGUrRbEVpXLQxb5R/g/RB6blw6wJbzmyh1YxW5AjOQUidEJoWb2p16eKHwsLCCAsLS9H3JDftCAT2\nAfWB08BGzAOeexIZ/wOwAPg5gce0IUjcLjoa8uWDkTO303fbK5TKUYqB9QZSLle5hMfHRLNg/wJ6\nL+1NyRwlGd98PDmCc7i5apG/OWNDUBTQDVgM7AamY4Z4l9gPEY+2Zg0EZrpA1w3PM7j+YOa3m59o\niAOkCkjFSyVfYuc/d1Iye0kqjanExlNqrotn0xZ98VmGYVC55RqOR29h0+QWFM5SOMXPMX/ffDrP\n78zklpNpVKyR84sUSYa26IvfMgyD/1v8ITtXFGfBp28+UogDNC/RnLlt59JhbgcWHVzk3CJFnERB\nLj7ps5WfMXfpWYrmzU61Co93cpUaBWowp80cOszpwLoT65xUoYjzKMjF54TuCOXH7T/S4M5o2rya\nyinPWaNADSa+NJGW01ty6PIhpzyniLOoRy4+ZcvpLTQObcyyDstpWqUsCxdC2bLOe/5RG0cxZssY\n1nVaR/qg9M57YpFEqEcufuVS+CVazWjFmBfHcPtYWYKDoUwZ577Ge1Xeo2Keiryz4B00MRFPoSAX\nn2AYBl1+6cLLpV6mValW9zcBOXuDps1mY3TT0fx1/i8mbp/o3CcXeUQKcvEJP2z7gQOXDzCo/iAM\nA2bOdN1JstKlTkdoq1B6LenF4SuHXfMiIimgIBevd+jyIXov7c1PrX4ibWBa1q+HtGnh6add95rl\ncpWjT80+vDHnDaJiolz3QiIOUJCLVzMMg3d/fZcPa3xImZxmQ3zaNGjb1vltlfh6VO9B6oDUfLPh\nG9e+kEgyFOTi1absmMLF8Iv0qN4DMM+tMmMGtGvn+tcOsAUwttlYBq4ayPFrx13/giKJUJCL17oY\nfpFeS3oxrtk4AgPME3mGhZlXASpe3D01FM9enPeffZ+uC7tqFYtYRkEuXqvn7z1pX649lfNWvv+1\ne20Vd+pdqzeHLh9i9p7Z7n1hkVjaECReafXx1bSf3Z7dXXeTISgDYF5gOU8e2LoVChZ0bz2rjq3i\ntZ9fY0/XPdooJE6lDUHik2KMGN5f9D5DGwy9H+IAv/8OpUq5P8QBaheqTc2CNRm2dpj7X1z8noJc\nvM6P234kbWBa2pZ9sIcydap7DnImZmiDoXyz8Rsd+BS3U2tFvMr1u9cpObIk89rOo0q+Kve/Hh5u\nHuTctw9y5bKuvgHLB7D/8n6mvjzVuiLEp6i1Ij5n0KpBPF/0+QdCHGDBAqha1doQB/iw5oesPr6a\nNcfXWFuI+BUFuXiNo1ePMu7PcQyqP+ihxyZPhtdft6CoeNIHpWdI/SH0WNxDyxHFbRTk4jU+XvEx\nXat0JW/GvA98/dw5WL0aWrWyqLB42pVrR1RMFLN2z7K6FPETCnLxCrsv7ObX/b/Ss3rPhx776Sdo\n0QIyZEjgGy0QYAtgSIMh9FvWj8joSKvLET+gIBev0H95f3rV6EXmtJkfemziRHjzTQuKSkLDIg0p\nmLkgE7ZOsLoU8QMKcvF4m09vZt3JdXSt2vWhx7Zvh8uXwW53f11JsdlsDGkwhE9WfkJ4ZLjV5YiP\nU5CLx+u3rB//fe6/BKcOfuixyZPhjTcgwAN/kyvnrUzNAjUZsX6E1aWIj9M6cvFoYUfD6DS/E3u6\n7iEoVdADj0VFQYEC5omySpSwpr7k7L+0n5oTarK3616yB2e3uhzxQlpHLl7NMAz6/tGXT+yfPBTi\nAEuWQKFCnhviYJ4d8eVSLzN49WCrSxEfpiAXj/XL/l+4FXGLduUS3nc/cSJ06ODmoh7BgDoD+GHb\nD5y8ftLqUsRHqbUiHinGiKHC6Ap8Vu8zmpdo/tDjFy9CsWJw5AhkzWpBgSnUZ2kfLt++zNhmY60u\nRbyMWivitab9NY30QelpVrxZgo9PmgTNm3tHiAP0rtmbOXvnsP/SfqtLER+kIBePExkdSf/l/RlU\nb9C92cgDDAPGjYN33rGguEeUNV1WelTrQf/l/a0uRXyQglw8zoStEyiStQh1n6yb4ONrYs9HVauW\nG4tygveffZ8Vx1bw55k/rS5FfIyCXDzK7cjbfLryUwbWG5jomLFjoXNnSGCy7tHSB6WnX+1+9FvW\nz+pSxMcoyMWjjNo0imfzP/vQaWrvuXIF5s/3vC35jvrHM/9g38V9rDy20upSxIcoyMVjXLtzjf+t\n+R+f1v000TGhodC4MeTI4cbCnCgoVRAf2z+m7x99dZpbcRoFuXiML9Z9wQtPvUDpJ0on+LhhwJgx\n3nWQMyHty7Xn2p1r/HrgV6tLER+hIBePcOHWBUZtGkWIPSTRMWFhEB0N9eq5rSyXSBWQioH1BtJv\nWT9ijBiryxEfoCAXjzBo1SDalW1H4SyFEx3zzTfQvbv3HeRMSPMSzQlOHcy0v6ZZXYr4AEeDvDGw\nFzgA9E7g8deA7cAOYA1Q3inViV84dvUYk3ZM4qPnPkp8zDFYscI806EvsNlsDKo3iP7L+xMRHWF1\nOeLlHAnyVMBIzDAvDbQDSsUbcxh4DjPAPwW0D1kcFrIihPcqv0euDIlfOfnbb82VKp5yFSBnqPtk\nXYpkLcL4P8dbXYp4OUf+SK0ODMAMcoA+sZ+HJDI+K7ATyB/v6zrXijxk94Xd2H+0c6D7gQSv/gMQ\nHg4FC8KGDVC0qJsLdLHNpzfTYloLDnQ/kOD51kWcda6VfMCJOPdPxn4tMZ2AhQ48rwgfLfuI3jV7\nJxriYC45rFHD90IczItP1ChQg282fGN1KeLFAh0Yk5JpdF2gI1AzoQdDQkLu37bb7dg97fpc4lYb\nTm5g0+lNhLYKTXRMTAwMHw6jRrmxMDf7tO6n1P6hNl0qdyFL2ixWlyMWCwsLIywsLEXf40hrpRoQ\nwt+tlb5ADDA03rjywM+x4w4m8Dxqrch9hmFQb1I9Xiv3Gp0rdU503Ny5MHAgbNzoG6tVEtNpXidy\nZ8jNwPqJn5pA/JOzWiubgaeAwkAQ0AaYH29MQcwQf52EQ1zkAUsOL+H0jdO8VeGtRMcYBgwdCr17\n+3aIAwywD2D0ltGcvXnW6lLECzkS5FFAN2AxsBuYDuwBusR+APTHPMj5HbAV2Oj0SsVnxBgx9Fna\nh8/qfkZgQOLdvVWr4NIlaNnSjcVZpGDmgnQo34GBKzUjl5TTFYLE7SZum8iYLWNY03FNgucbv6dp\nU2jRAv7xDzcWZ6ELty5QclRJNr+zmSezPml1OeIhHGmtKMjFrW5F3KLEyBLMaj2LavmrJTpu505o\n1AgOH4a0ad1YoMUGLB/AkatHmNRyktWliIfQpd7E4wxbO4znCj2XZIgDfPIJ9OjhXyEO0LNGTxYf\nWsz2s9utLkW8iGbk4janrp+i/Ojy/PmPPymUpVCi47ZvN09Ve+gQBPvhHpnvNn3H9F3TWf7m8iRb\nT+IfNCMXj9JvWT+6PNMlyRAHCAkxV6r4Y4iDefGJK3euMGv3LKtLES+hGbm4xZbTW3hx6ovs67aP\nTGkyJT5ui3mA88ABSJfOjQV6mLCjYbw59032dN2jrft+TjNy8QgxRgxdF3ZlYL2BSYY4QP/+0Lev\nf4c4gL2wnar5qvL52s+tLkW8gIJcXG78n+MJsAUkufkHzAtH7N5tXlhZ4POGn/P1hq85fu241aWI\nh1OQi0tdDL/IR8s/4tum3xJgS/zXLSYGevaEIUMgTRo3FujBCmUpRLeq3fhg0QdWlyIeTkEuLtVn\naR/alW1HhdwVkhw3ZQoEBUHr1m4qzEv0qdWHXRd2MW/vPKtLEQ+mg53iMutOrOOVma+w+73dSZ6m\nNjwcSpSA6dPN09XKg8KOhtFhTgd2vbeLjGkyWl2OuJkOdopl7kTdodP8Tnzx/BdJhjjA//5nBrhC\nPGH2wnYaFmnIR8sSvxSe+DfNyMUl/vPHf9h7cS+zW89OclPL/v1mgG/bBvnjX1NK7rt8+zJlvi3D\nvLbzqJqvqtXliBtpRi6W2Hx6M+O3jufbpt8mGeKGAe+9B/36KcSTky1dNr5s9CVvzX2L25G3rS5H\nPIyCXJwqIjqCt+e9zfDnh5M7Q+4kx06dChcvQvfubirOy7Up04Zyucrxnz/+Y3Up4mHUWhGn+nDJ\nh+y7tI+5beYmORu/cAHKl4c5c6Ba0ufPkjgu375M+e/KM6nlJOo9Wc/qcsQN1FoRt/r90O/8tPMn\nxjcfn2xLpUsXeP11hXhKZUuXje+bf8/b897m6p2rVpcjHkJBLk5x/tZ53pr7FpNaTiJHcI4kx06Z\nYp5L5dNP3VScj2lcrDHNijej0/xO6K9cAQW5OEGMEcNbc9/i7QpvJ/vn/vHj8O9/w+TJ/neucWca\n/vxwjl87zogNI6wuRTyAglwe2ycrPuH63euE2EOSHBcRAW3bwv/9H1RIeqOnJCNNYBpmvDKDwasH\ns/bEWqvLEYspyOWxzNs7jwlbJzCr9SxSp0qd5NhevSBHDvOzPL4nsz7J982+p82sNpy9edbqcsRC\nCnJ5ZLsv7OadBe8wu/XsZJcaTpsGv/wCEydCgH7rnKZZiWZ0rtiZFtNaEB4ZbnU5YhEtP5RHcubG\nGWpMqMHH9o/p8HSHJMdu2WJeum3JErVUXMEwDF6f8zoR0RFMf2V6kmeZFO+j5YfiEtfuXKNJaBM6\nV+ycbIgfOQLNm8O4cQpxV7HZbIxvPp4zN87Qe0lvrWTxQ5qRS4rcibpD05+aUjJ7SUa+MDLJ9eKX\nL0PNmuY2fO3edL1L4ZewT7TTpkwbPnpOJ9jyFY7MyBXk4rDbkbd5afpLZE2bldBWoaQKSJXo2KtX\noWFDsNth2DD31ejvzt48y3M/PMc/K/+THtV7WF2OOIEjQR7onlLE292OvE2LaS3IEZyDSS0nORTi\nNWuap6gV98mdITdLOyylzo91iIqJoldNLRHyB+qRS7Iu375M49DG5Eyfk0ktJxEYkPj//8+fhwYN\nzBD/8ktIovMiLlIwc0FWvb2KH7b9QJ+lfdQz9wMKcknS4SuHqTG+BlXzVk02xPfuherVoUkThbjV\n8mfKz6q3V7H86HI6zu/I3ai7VpckLqQgl0QtP7KcWhNq0b1qd4Y9PyzJZW3LlkGdOvDRR+Y5VBTi\n1ssenJ0/OvzBjbs3sE+0c/rGaatLEhdRkMtDYowYBq4cSPuf2/PjSz/StWrXRMdGR0NICLz2mnl+\n8bffdl+dkrwMQRmY+epMXnzqRaqOq8qyI8usLklcQKtW5AFHrhyh84LORERHMO3laeTLlC/xsUeg\nUyfzdmgo5MnjpiLlkSw+uJiO8zvSpkwbBtUfRNpAnbXMG2hDkDgsKiaKEetHUGVcFRoXbczyN5cn\nGuJRUTB8OFSpAo0amTs2FeKer1GxRux4dwfHrx2n4piKLD281OqSxEk0I/dzhmGw8MBCPlz6Ibkz\n5Oa7pt9RPHvxRMbCwoXQpw/kzAljxkCxYm4uWB6bYRjM3zefDxZ/QOW8lRlcfzDFsukH6am0IUgS\nZRgGvx/6ncGrB3P+1nmGNRzGC0+9kOBOTcOARYtg8GDzGptDhkCzZjqg6e1uR95m2NphfL3ha5qV\naMZHtT+iaLaiVpcl8SjI5SE37t5g5u6ZjNgwghgjhl41etG+XPsElxVeuwYzZsBXX0FgoHke8Xbt\nzNviO67cvsKIDSMYuXEktQrW4r0q79GgSAOdfMtDOCvIGwNfAamA74GhCYz5GmgChANvAVsTGKMg\nt8idqDssP7Kc0J2h/LL/F+oUrsM/K/+TRkUbPTQDDw+HP/4wD14uWgT16pnnSqlfXzNwX3cz4iZT\nd05l1KZRXL97ndZlWvNq6VeplKdSkufUEddyRpCnAvYBDYBTwCagHbAnzpgXgG6xn58FRgAJXVLX\np4M8LCwMu91udRkARMdEs+fiHlYdW8VvB38j7GgY5XKVo3Xp1rQr146c6XPeHxsRATt2wOrV8Ntv\nsHYtPPOMeSWf1q0hWzZznCe9P2fz5fcGKX9/hmGw7ew2Zu6eyczdM4mMjqRBkQbUe7IedQvXJU9G\nzzqy7es/P2eca6UqcBA4Gnt/GtCCB4O8OTAx9vYGIAuQCziXomq9nFW/TFduX2Hvxb3su7SPPRf2\nsPnMZjad2kTuDLmpXqA67cuZa8Gzps3GmTOwayPM3mvuwty0CbZvhyJFzB2ZXbrAzJmQKZPnvD93\n8OX3Bil/fzabjYp5KlIxT0UG1hvInot7WHZkGTN3z6Trwq5kSpOJSnkqUTF3RZ7O9TTFshWjSNYi\npEudznVvIgm+/vNzRHJBng84Eef+ScxZd3Jj8uNnQf64YowYIqIjuBt1lxsRN7h25xpX71zl2t3Y\nz3euce7WOU5dP83Jq2c5dfUCpy5d5e7NtBRKW548QSXJlaoKT6d+jboBRbl+Ij2nN8DY0xByGk6d\nguBgKFECSpY0P7dsac6+M2a0+t2Lp7LZbJR+ojSlnyhNt6rdiDFiOHzlMFvPbOXPM38y9s+xHLp8\niKNXj5I9ODtFshYhT4Y85Eyfk1zpc5ErQy6eCH6CTGkykSlNJjKmyUjGoIxkTJORDEEZ1Id3kuSC\n3NFeSPxpv6U9lKZdw9i0NhiwgRFbTGxFhhHna9jAMDCwmQMMm/k4RuznON8T/zniPXb30lmGT9l+\n//nN4fG/z/yIMWJiPxsYhkGMYT5oIwCbzUYAqQmISYctOgtEp4HoIGIiUxMTFUhMVCC2AIOgIIN0\n6WzkymKa6X1SAAADX0lEQVQjTWYb0ZnhVma4lBmis5jrusuWhbx5zY98+SBzZmf/lxZ/E2ALoFi2\nYhTLVoxXy7x6/+vRMdGcunGKI1eOcPbmWc7dOse5m+fYeGojF8IvcP3udW7cvcGNiBv3P9+KuEVg\nQCBBqYLuf6ROlfqB+4EBgQTYArBhMz/bbA/dP7btGCsnrkzwMVsSHYmk+v5Jfd/Ul6eSMY1nzX6S\n65FXA0IwD3gC9AViePCA52ggDLPtArAXqMPDM/KDgNY2iYikzCHgsRb6B8Y+SWEgCNgGlIo35gVg\nYeztasD6x3lBERFxviaYK1cOYs7IAbrEftwzMvbx7UAlt1YnIiIiIiKO6465dPEvEt5Y5At6Yh5H\nyGZ1IU42DPNntx34GfCVQ6eNMY/rHAB6W1yLsxUAlgO7MP/N/cvaclwiFeYGxAVWF+ICWYBZmP/u\ndpPw/hy3qwssAVLH3n/CwlpcpQCwCDiC7wV5Q/4+W+aQ2A9vlwqzJVgY8/cyoWNA3iw3UCH2dgbM\nFqkvvT+AfwOhwHyrC3GBiUDH2NuBeMjkaQZQz+oiXGwmUB7fDPK4WgJTrC7CCapj/o/3nj6xH75q\nLlDf6iKcKD+wFHOS6Gsz8szAYUcHu3M1/lPAc5irWsKAym58bXdogbkZaofVhbhBR/5eqeTNEtrM\nlviVNLxbYaAi5u5rX/El0AuzlelrngQuAD8AfwLjgODEBjv7PHZLMP+ci69f7GtlxezzVMGcoRdx\n8uu7WlLvry/wfJyveeNZhhJ7f//h7xlPPyAC+MldRbmQ757850EZMHut7wM3La7FWV4EzmP2x+3W\nluISgZgrALthnuPqK8y/FvtbWRTAb5gbhe45CGS3qBZnK4u5AepI7Eck5vlpcibxPd7oLWAN4CvX\nCKvGg62VvvjeAc/UwGLgA6sLcbJBmH9NHQHOALeASZZW5Fy5Md/bPbWAXyyq5QFdgI9jbxcHjltY\ni6v5Yo+8MebqhxxWF+JEjmx482Y2zHD70upCXKwOvtcjB1iJmZVg7rD3iJV+qYHJwE5gC77559A9\nh/G9ID8AHMP8U3Yr8K215ThNQhvefEUtzP7xNv7+uTVO8ju8Ux18c9XK05htFV9b8isiIiIiIiIi\nIiIiIiIiIiIiIiIiIiIiIiLi3/4fQB0xhRis3mMAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEACAYAAABMEua6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFyBJREFUeJzt3XuUlPV9x/E3ghSMUZsD1nuwQLwRIhoUUcMgGm7KRSCN\ntdTEXDyn1aSeNMea2LqatDk19VIltl6i4l0BEbkqIlMxCOIFFFAwCCleKkYQVFBYdvrHd9eFCOzs\nzuw8M8+8X+fMYXZ39pmvj/jxN9/n9/x+IEmSJEmSJEmSJEmSJEmSVPE6AAuBxcBy4FfJliNJKtQ+\n9X+2AxYApyVYiyRVtb2KcIzN9X+2B9oC64twTElSCxQj1Pci2i/vAnOJNowkqcLtT7RfMgnXIUlV\nq10Rj7URmA58Hcg2fLNr1665VatWFfFtJKkqrAK6NfeXCm2/dAIOqH/eETgLeGmnqlatIpfL+cjl\nuPLKKxOvoVwengvPhedizw+ga0tCudCR+sHAeOJ/DnsB9wBzCjymJKmFCg31V4ATilGIJKlwxZj9\nojxlMpmkSygbnotGnotGnovCtSnBe+Tq+0OSpDy1adMGWpDRjtQlKUUMdUlKEUNdklLEUJekFDHU\nJSlFDHVJShFDXZJSxFCXpBQx1CUpRQx1SUoRQ12SUsRQl6QUMdQlKUUMdUlKEUNdklLEUJekFDHU\nJSlFDHVJShFDXZIS9sc/wsknQ11d4ccy1CUpYY8+CkccAXsVIZENdUlK2MSJMGZMcY7V7J2qWyCX\ny+VK8DaSVHnWr4cjj4S33oJ99238fps2baAFGe1IXZISNGUKnHnmzoFeCENdkhI0YQKMHl2849l+\nkaSEbNgAXbrAm2/CF7+4889sv0hShXnsMTjjjM8HeiEKDfXDgbnAMmAp8KOCK5KkKjFxYnFbL1B4\n++Wg+sdiYF/gBWAE8OoOr7H9Ikl/YuPGmJu+di3st9/nf55U++X/iEAH+IgI80MKPKYkpd7kydCv\n364DvRDtinisLkAvYGERjylJqVNbC7/6FYwbV/xjF+tC6b7ARODHxIhdkrQb99wDhxwS89OLrRgj\n9b2BScC9wKO7ekFNTc1nzzOZDJlMpghvK0mV59NP4aqr4P77oc0OHfNsNks2my34+IVeKG0DjAfe\nBy7dzWu8UCpJ9caNg5kzYfr0Pb+upRdKCw3104CngZeBhuS+HJi1w2sMdUkCNm+Gbt0i0Hv12vNr\nWxrqhbZfnsEbmCQpL+PGwWmnNR3ohXCZAEkqgdWr4aST4Jln4Kijmn69ywRIUpnavh0uuAAuuyy/\nQC+EoS5Jrey662Kmy6W7m05SRLZfJKkVvfwyDBgAixbFioz5sv0iSWXmk09g7Fi45prmBXohHKlL\nUivI5eD886GuDh54YOcbjfKR1JRGSdIuXHUVvPEGzJ3b/EAvhKEuSUV2331w112wcCF07Fja97b9\nIklFNG8ejBoFc+bAV7/a8uN4oVSSEjZ/fgT6ffcVFuiFMNQlqQjmz4cRI+Duu+Gss5Krw1CXpALt\nGOiDBiVbi6EuSQWYMQOGDy+PQAdDXZJa7NZb4Xvfg6lTyyPQwSmNktRsuRxccQU8/HDMdunWLemK\nGhnqktQMmzbFiovvvhu99M6dk65oZ7ZfJClPy5dD795w8MFxp2i5BToY6pKUlwcfhH794PLL4eab\n4c/+LOmKds32iyTtwYcfwsUXw4IF8PjjcMIJSVe0Z47UJWk3FiyA44+H9u3hxRfLP9DBkbokfc4n\nn0BNDdx5Z7RaRo1KuqL8OVKXpB3Mnx+j81WrYteiSgp0cKQuSQB88EHMPZ80CW66CUaPTrqilnGk\nLqmq5XJw771w7LGwfTssW1a5gQ6O1CVVsZdegn/4h5jhMnkynHxy0hUVzpG6pKrz7rvw/e/D4MGx\nj+iiRekIdDDUJVWRjz6Cq6+G446DP/9zWLECfvhDaNs26cqKx1CXlHrbtsXUxO7dI8gXLYJf/xr2\n3z/pyorPnrqk1KqtjYugV18dgT5jBvTqlXRVrasYoX4HMBRYByS0K58kNaqthYceijA/6CC46y74\nxjeSrqo0mr1T9S6cDnwE3M2uQz2Xy+WK8DaStGfbtsXI/N/+LcL8yithwABoU4ykK7E2UXSzKy/G\nSH0e0KUIx5GkFtm8Ge64A669Fv7yL+G222JFxUoM80LZU5dUsd5/H37zm3j07QsPPAB9+iRdVbJK\nEuo1NTWfPc9kMmQymVK8raSUWrkSbrghQnzkSMhm4Zhjkq6qMNlslmw2W/BxivXhpAswFXvqklpJ\nLgdPPQX/+Z+xJO5FF8Hf/330ztMoyZ66JLWajz+G++6DG2+Mr3/0o9iFaJ99kq2rXBXj5qMHgPnA\nV4C1wHeLcExJVW7FCvjxj+GII2D69Bihv/JK3AFqoO9eMUbq5xXhGJLE1q3w6KNwyy2wdGmsz/Li\ni/DlLyddWeWw/SIpca+/Dr/9bdwkdMwx0S8fObJ8N3cuZ4a6pERs2QKPPAK33w7Ll8PYsfA//wNH\nHZV0ZZWtFFPznf0iCYgZLIsWxY1CEyZA797RYhk2LDZ3ViNnv0gqW2+9Fbfvjx8fffPvfhcWL4bD\nD0+6svRxpC6pVXz0UewmdM898PzzsUXcBRfEnZ/VePt+c7V0pG6oSyqabdvgySdjXvm0aXD66fA3\nfxPtlY4dk66ushjqkhJRVxd3eN5/Pzz8MHTtGlvE/dVfQefOSVdXueypSyqZXC564g8+GI8vfAHO\nOw+efTZCXckx1CXlJZeDZcti84mHHopWy7e/DVOnwle/ap+8XNh+kbRbDUE+YUK0Vj7+GL71rXj0\n7m2QtyZ76pKKIpeDJUtg0iSYODGCfMyYeJx0EuzldvUlYahLarG6urgpaNKkuMuzri6mII4e7Yg8\nKYa6pGaprYWnn4655JMnw377wbnnxqNXL4M8ac5+kdSkzZth9uxYCXHqVDjyyFg4a/bsyt85SMGR\nupRy770XNwJNmQJz58LXvw4jRsDw4bFWucqT7RdJn1m5Eh57LIL85ZfhrLMixIcOhS99KenqlA9D\nXapitbVx48/UqfHYuDFuzR82DM44Azp0SLpCNZehLlWZjRvh8ccjxGfOhMMOg3POiSA/8USnHlY6\nQ12qAq+/Hv3xadNiCuLpp0dL5eyz7Y+njaEupdDWrTBvXmy8PH06fPhhBPjQoXDmmbHmitLJUJdS\n4p13YMaMeMyZE9u7DR0ajxNOcP54tTDUpQq1fTs891yE+PTpsGYNfPObMHhwPA48MOkKlQRDXaog\n69bFRc6ZM+GJJ+DQQ2HIkAjxvn2hnbcFVj1DXSpj27fHhc2ZM+OxciX07x8hPmRIzFyRdmSoS2Xm\n3XcbR+OzZ8PBBze2VE49Fdq3T7pClTNDXUpYbW1s6zZrVgT5qlUwYAAMGhSPww9PukJVEkNdSsDa\ntTEanzUrZqp06dIY4n37wt57J12hKpWhLpXAJ5/EvPGGIH/nnZipMmhQ/HnwwUlXqLRIMtQHATcA\nbYHbgX//k58b6qpYuVzcxTlrVgT5vHnQo0eE+MCBseJh27ZJV6k0SirU2wIrgDOBt4BFwHnAqzu8\nxlBXRdm0CZ56qnE0vnVrY0tlwABXOVRpJLVJxknA74E19V8/CAxn51CXylpdHSxe3Dgaf/FF6NMn\nRuJTp8Jxx3kXpypHoaF+KLB2h6/fBE4u8JhSq1u3Lm76mTUrphsecECMxC+7DPr1c00VVa5CQz2v\nvkpNTc1nzzOZDJlMpsC3lZpn2zaYPz9G4o8/HtMN+/ePIP/lL2PWipSkbDZLNpst+DiFfqjsA9QQ\nF0sBLgfq2PliqT11JWL16sa+eDYL3btHS2XgwGivON1Q5SypC6XtiAulA4C3gefwQqkS8vHHEd4N\nQb5pU+N0w7POgs6dk65Qyl9SF0prgYuBx4mZML/Fi6QqkVwOli2LAJ81CxYujCmGAwfCww9Dz57u\n/qPq481HqigffABPPtkY5O3bN84ZP+MM+OIXk65QKg7vKFUq1dXFFMOGEH/55djCbeDAWBirWzen\nGyqdDHWlxnvvxXTDhrXGO3VqvPnnG9+ADh2SrlBqfYa6Ktbu1hpvCPIvfznpCqXSM9RVUd57r3GJ\n2ieegEMOiQB3rXEpGOoqa3V18PzzsQ/nzJnw2muxjsrgwa41Lu2Koa6ys2FDzBmfMSNG5Z07x9Zt\nQ4Y4GpeaYqgrcQ3zxqdPh2nTYMmSWEelYUNlb8WX8meoKxFbtsDcuY1BvtdeMHRoPDIZ6Ngx6Qql\nypTUHaWqQm+/HSE+dWrcln/88XD22dFmOfZY541LSXKkriblcrHe+GOPRZC/8UZc3Dz77PjTTSOk\n4rP9oqL69NNoqzQEeYcOMGwYnHNOXOR0hUOpddl+UcHWr48WypQpsXFEjx4R5E88AUcfbVtFqgSO\n1Kvc//5vhPijj8Y88v79YfjwuNB54IFJVydVL9svyksuB8uXwyOPwOTJEernnAMjRsSa4/vsk3SF\nksBQ1x7kcjEKnzQpwvyTT2DkyHicdhq0swknlR1DXTupq4s9OSdOjCDv2BFGjYJzz4UTT7Q/LpU7\nL5SK7dvhmWdgwoQI8k6dYPToWGvF+eNSdTDUK1xdXQT5ww9He+Wgg2DMmLgp6CtfSbo6SaVmqFeg\nXA4WLICHHopReefO8K1vwdNPQ/fuSVcnKUmGeoXI5WIrtwcegAcfjB75t78Nc+bEHHJJAkO97K1e\nHUF+333w8ccR5FOmQM+e9sglfZ6zX8rQ+vXRVrnnHlixInrk558Pp5wSqyBKSj+nNFa4rVtjlsr4\n8dFSGTgQxo6NBbNcZ0WqPoZ6hVq8GO68M1osRx0Ff/u3MTI/4ICkK5OUJOepV5D166NHfscd8fyC\nC+DZZ6Fr16Qrk1TpHKmXSF1dzB2//fZYCXHIELjwQjjjDPvkkj7P9kuZWrcO7roLbr01Fsv6wQ/i\noqcbS0jaE9svZSSXi7s8b745Ln6OHAn33gsnn+w0REmtq5AP/mOAZcB24ITilFPZPvoI/vu/4Wtf\nixF5nz6wZk1cCO3Tx0CX1PoKGam/AowEbilSLRVr9WoYNy7aLP36wfXXR6/cEJdUaoWE+mtFq6IC\nNbRYrrsO5s2Li54vvABduiRdmaRqZk+9mWprY43ya6+FjRvh0kujX/6FLyRdmSQ1HeqzgYN28f2f\nAVPzfZOamprPnmcyGTKZTL6/Wja2bIne+H/8Bxx6KFxxRWwD53REScWQzWbJZrMFH6cYXd+5wE+A\nF3fz84qe0rhpU8xiueGGmL1y2WXQt2/SVUlKu6SnNKbukuCGDXDjjXEB9JvfhCefhB49kq5Kkvas\nkObBSGAt0AeYDswsSkUJ27QJrr46NptYswZ+97u4pd9Al1QJvKO03ubNcNNNcQF00CD4l3+Bbt2S\nrkpStWpp+6XqL/PV1sJtt8V+ni+8EFvC3X23gS6pMlXtlMZcDqZNg5/+FA45BCZPht69k65KkgpT\nlaG+dGnML3/rrbh5aPBg7/6UlA5V1X754AO4+OK4hX/YMFiyJJbANdAlpUVVhHouF3d9HnssbN8O\nr74Kl1ziNnGS0if17Zff/x5++MMYpU+eHDcQSVJapXakvn179MtPOSVu53/uOQNdUvqlcqS+YgV8\n5zvQoQMsWODen5KqR6pG6rkc3HILnHYajB0Lc+YY6JKqS2pG6u+/D9//PvzhD7G++dFHJ12RJJVe\nKkbqCxdCr14xKn/2WQNdUvWq6JF6Lge33gr//M9w++0x91ySqlnFhvqnn8Lf/V2M0p95JtZukaRq\nV5GhvmEDjBgBnTrF7JZ99026IkkqDxXXU//DH+DUU+HEE2HCBANdknZUUaG+eHEE+kUXxY1F7g8q\nSTurmPbLSy/Faoo33QRjxiRdjSSVp4oI9YZAv/lmOPfcpKuRpPJV9g0MA12S8lfWe5S+8Ub00H/z\nGwNdUnVp6R6lZRvqH3wQKyxecknMR5ekapKqUN+2LVouPXrADTe0UlWSVMZSE+q5XExZfPttmDIF\n2rZtxcokqUy1NNTLbvbL+PEwf34szGWgS1LzlNVIfc0a6N071kHv2bN1i5KkctbSkXrZTGmsq4vd\nin76UwNdklqqbEL9+usj2H/yk6QrkaTKVRbtl6VLoX//2Bz6yCNLUJEklbkk2i+/Bl4FlgCPAPu3\n9ECXXgpXXWWgS1KhCgn1J4DjgK8BK4HLW3KQuXNh9Wr4wQ8KqESSBBQW6rOBuvrnC4HDmnuAXA5+\n/nOoqYG99y6gEkkSULwLpRcCM5r7SzNmwKZNcN55RapCkqpcUzcfzQYO2sX3fwZMrX/+c2ArcH9z\n3riuDq64An7xC28ykqRiaSrUz2ri598BhgAD9vSimpqaz55nMhkymQyTJkG7drHXqCRVu2w2Szab\nLfg4hUxpHARcC/QD/riH1+1ySmPPnnDNNTBoUAEVSFJKJbGg1+tAe2B9/dfPArtaJPdzob58OQwc\nGJtIu8+oJH1eEgt6dW/pL06YAKNHG+iSVGyJxOrEiRHqkqTiKnmov/YarF8fuxpJkoqr5KE+cSKM\nGmXrRZJaQ8mjdcIEGDOm1O8qSdWhpKG+ciWsWwd9+5byXSWpepQ01BtaL95BKkmto+Sh7qwXSWo9\nJQv1Vavg7bfh9NNL9Y6SVH1KtvPRli0xnbFXrxK8oyRVuCSWCchXk9vZSZJ2lsR2dpKkMmOoS1KK\nGOqSlCKGuiSliKEuSSliqEtSihjqkpQihrokpYihLkkpYqhLUooY6pKUIoa6JKWIoS5JKWKoS1KK\nGOqSlCKGuiSliKEuSSliqEtSihQS6r8AlgCLgTnA4UWpSJLUYoWE+jXA14DjgUeBK4tSUYpls9mk\nSygbnotGnotGnovCFRLqH+7wfF/gjwXWknr+hW3kuWjkuWjkuShcuwJ//1+BscBmoE/h5UiSCtHU\nSH028MouHufU//znwBHAXcD1rVOiJClfbYp0nCOAGUCPXfzs90DXIr2PJFWLVUC35v5SIe2X7sDr\n9c+HAy/t5nXNLkqSVHoTiVbMYmAScGCy5UiSJEnapUHAa0RL5rLdvObG+p8vAXqVqK4kNHUuzifO\nwcvA74CepSut5PL5ewHQG6gFzi1FUQnI5zxkiDbmUiBbkqqS0dS56ATMIroAS4HvlKyy0rsDeJfo\neuxOIrnZlrgg2gXYm/iXccyfvGYIcTEV4GRgQamKK7F8zsUpwP71zwdR3eei4XVPAdOAUaUqroTy\nOQ8HAMuAw+q/7lSq4kosn3NRA/yq/nkn4H0Kn35drk4ngnp3od7s3CzW2i8nEf+i1gDbgAeJi6c7\nGgaMr3++kPhL/BdFev9yks+5eBbYWP98IY3/IadNPucC4BLiGs17JaustPI5D39NXJt6s/7rtN7M\nl8+5eAfYr/75fkSo15aovlKbB2zYw8+bnZvFCvVDgbU7fP1m/feaek0awyyfc7Gj79H4f+K0yffv\nxXDgv+q/zpWgrlLL5zx0B74EzAWeJ27qS6N8zsVtwHHA20TL4celKa0sNTs3i/WRJt//EP90Xnwa\n/wNuzj9Tf+BC4NRWqiVp+ZyLG4B/qn9tG4p370Q5yec87A2cAAwA9iE+zS2gcdpwWuRzLn5GtGUy\nxD0us4l1pj7cw++kWbNys1ih/hY7r9J4OI0fI3f3msPqv5c2+ZwLiIujtxE99T19/Kpk+ZyLE4mP\n4BD908HEx/LHWr260snnPKwlWi5b6h9PE0GWtlDP51z0JZYggbgBZzVwFPEJptoklpvtiJPfBWhP\n0xdK+5Dei4P5nIsjiL5i2tfLyedc7OhO0jn7JZ/zcDTwJHEhcR/iwtmxpSuxZPI5F9fRuOrrXxCh\n/6US1ZeELuR3obTkuTkYWEGE1eX137uo/tFgXP3PlxAfNdOqqXNxO3Hx56X6x3OlLrCE8vl70SCt\noQ75nYd/JGbAvAL8qKTVlVZT56ITMJXIiVeIi8hp9QBx7WAr8WntQqo3NyVJkiRJkiRJkiRJkiRJ\nkiRJkiS1pv8Ho7nRrp5mizkAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "x = np.arange(-5, 5, 0.01)\n", "plt.figure()\n", "plt.plot(x, phi(x), 'g-')\n", "plt.plot(x, Phi(x), 'b-')\n", "\n", "u = np.arange(0, 1, 0.01)\n", "plt.figure()\n", "plt.plot(u, Phi_inv(u), 'b-');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before we show the sampling algorithm in action, we define a general interface for sampling methods" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class Sampling (object):\n", " \"\"\"\n", " Abstract base class for all sampling methods.\n", " \n", " Subclasses need to implement self.sample()\n", " \"\"\"\n", " def sample(self):\n", " pass\n", " \n", " def __str__(self):\n", " \"\"\"\n", " Default is to show class\n", " \"\"\"\n", " return str(self.__class__)\n", " \n", "class InversionSampling (Sampling):\n", " def __init__(self, h_inv):\n", " self.h_inv = h_inv\n", " \n", " def sample(self):\n", " return self.h_inv(np.random.uniform())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and an interactive plotter to show our sampler in action" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import ipywidgets\n", "from ipywidgets import widgets\n", "from IPython import display\n", "\n", "def show_sampling(sampling, plotter, N = 1000):\n", " def draw():\n", " return [sampling.sample() for _ in range(N)]\n", " \n", " samples = [draw()]\n", " fig, ax = plt.subplots()\n", " def show(num=100):\n", " plotter(plt, samples[0][:num])\n", " plt.show()\n", "\n", " # Button currently not working\n", " # button = widgets.Button(description=\"Redraw\")\n", " # display.display(button)\n", " \n", " # def click(b):\n", " # samples[0]=draw()\n", " \n", " # button.on_click(click)\n", " widgets.interact(show, num=(10,N,10))\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, a demonstration of inversion sampling to draw samples from a standard Gaussian distribution:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEACAYAAAC57G0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4FGW2x/FvCEEIBAFFdoxsCiKLIOKCtogMKAIRAUFR\n0XEYFddxd0bC3BmXO+6XcdxmQBmVRQgi6rC3IgKKsimLrLKKKDiAgCTQ94+3gU5IUt3p6n6ru3+f\n56mHXqr6HAJ98tapt6pARERERERERERERERERERERERERCRldQNWAquBB0tZ7xygAOhThm1FRCTO\n0oE1QDaQASwGmpew3ixgCscKfLjbiohIDJRzeL8DpkhvAPKBMUCvYta7A3gX2FGGbUVEJAacCnw9\nYFPI883B14qu0wv4R/B5IIJtRUQkRpwKfMDhfYDngYeC66YFl3C3FRGRGCnv8P4WoEHI8waYkXio\ndpj2C8DJQHdMSyacbWncuHFg7dq1EaQsIiLAWqBJNB9QPvgh2UAFnA+UjgSuinDbQCIYNmyY7RTC\nojzLYOPGQOCyywKBFi0CgTffDAT27j361rBhwwKBgwcDgY8+CgQ6dzbrfPaZvVyL4amfZSmUp7sI\no0vi1KIpAIYCU4HlwFhgBTAkuJRlWxHvmDoV2rUDnw+WLIFBg6By5cLrZGRAt24wYwYMHw69e8OI\nERBQF1K8zalFA/BRcAn1SgnrDg5jWxFvePNNeOAByMuDCy5wXj8tDa6+Gtq2NUX+u+/gf//XvC7i\nQU4jeAny+Xy2UwiL8gzT2LHwyCPg95da3IvNs3Fj+PhjmDnT/IKwzPrPMkzKM/68MPQItpNE4mTm\nTBg40LRczjqr7J+zcyd06gS33gpDh7qXn0gY0syeY6k1XAVeUsuGDdCxoxnBX3xx9J+3fj2cfz68\n8QZ07Rr954mESQVeJNSvv5p2zLXXwj33uPe5H38M11wDX30Fdeq497kipVCBFwn10EOwahVMnOj+\ngdFhw+Czz8ysnHI6tCWxpwIvCalq1Rrs2bPL1c/sALwHtAZ+CHk9K6s6u3fvjD5AQQFcdJHZO7j9\n9ug/T8SBCrwkJPMf173/EydwgK84m+EMYxz9i0bDtf9/y5ebvv6SJVC3rjufKVKCcAq89iUl6d3P\n3/iWZoyjX2wDtWgBv/893HlnbOOIhEkjePEcN0fw9dnEYtrQji/5juziork3ggc4cABatYLnn4fL\nL3fvc0WK0AheUt6TPMRL3FZCcY+BihXhmWfg/vtNX17EIhV4SVodmcdFfMKTPBTfwD16QM2aMHJk\nfOOKFKEWjXiOOy2aALO5hDe4gVHHXSKpUDR3WzRHLFwIvXqZaZlVqrj/+ZLy1KKRlNWZWdRhG6MZ\nZCeB9u3NtMkXXrATXwSN4MWDoh/BB/iM83mROxnDAKdosRnBgxm9d+oEa9dCVlZsYkjK0gheUlJ3\nPqIKexl73Jz3ODv9dOjSBf7+d7t5SMrSCF48J9oR/KdcwAvcxfiw5r3HcAQP5uSnSy4xo3j14sVF\nGsFLyjmfudTmeybQx3YqRosW5m5RL79sOxNJQRrBi+dEM4J/j558RHde5tZwo8V2BA+waBH07Anr\n1pnb/4m4QCN4SSnNWU4HPmcUN9pOpbC2baFpUxg/3nYmkmLCKfDdgJXAauDBYt7vBSwBFgFfAp1D\n3tsALA2+93k0iYo4uZ+/MYKhHKCS7VSOd++95gxX7a1KHDm1aNKBVUAXYAvwBTAAWBGyTmXgl+Dj\ns4A8oEnw+XqgHVDa9VjVopFCytKiqcX3rKA5jVnLLmpEEi32LRqAw4dNP/6VV9y5k5SkPDdaNB2A\nNZiReD4wBjNiD/VLyOMqwI9F83CIIRK1W3iNcfSLsLjHUbly5i5Szz5rOxNJIU4Fvh6wKeT55uBr\nRfXGjOo/AkKvlRoAZgALgVvKnqZIydIpYAiv8BK32U6ldIMGmbs+rV1rOxNJEU4FPtx910lAc+BK\nYHTI6xcAbYHuwO1Ap0gTFHHSk8lsIJultLadSukyM+GGG+DVV21nIimivMP7W4AGIc8bYEbxJZkT\n/MyTgJ+AbcHXd2B68x2C6xSSm5t79LHP58Pn8zmkJXLM7fydv5Mgt8kbMsTc+PvPf4YTTrCdjSQQ\nv9+P3++PaBun/nh5zEHWS4GtmJkwRQ+yNgbWYUb7ZwPjg69lYg7S7sEciJ0GDA/+GUoHWaWQSA6y\nnsEKZnMJDdlIPhXKEi0+B1lDXXYZDB4MAwfGN64kFTcOshYAQ4GpwHJgLKa4DwkuAH2AZZipkC8A\n1wRfr40ZrS8GFgBTOL64i0TlNl7idX5bxuJuya236sxWiQsvzHDRCF4KCXcEX4l9bKY+rVnC5kKd\nxIiixX8En58P2dkwdSq0bBnf2JI0dCarJLU+TGA+HaMo7pZkZMDNN+tgq8ScRvDiOeGO4GfSmZe4\njQlcHU20+I/gAdavhw4dYPNmHWyVMtEIXpLWaazjLJbxPlfaTqVsTjvNtGemTLGdiSQxFXhJSDcy\nircZyEESePQ7eLBuzC0xpRaNeI5Ti6Ych1jPaVzJ+y6c3GSpRQPwyy9Qv765KUidOnZykISlFo0k\npc7MYgc1vX/mqpPKleGqq+Df/7adiSQpFXhJODfxL0Yy2HYa7hg8GEaN0mWEJSbUohHPKa1Fk8Vu\nNtGARqxjJye5Ec1eiwZMYW/WDN56y8yqEQmTWjSSdHLIw4/PpeLuAWlp5gJkb75pOxNJQirwklCu\n5S3e4lrbabhr4EBzO7+CAtuZSJJRgZeEUYvvOYcvmEIP26m4q1Ejs8ycaTsTSTIq8JIw+jOWyfRk\nP5m2U3HfwIHw9tu2s5Ako4Os4jklHWRdQAf+yF+YTlc3o9k9yHrE9u1wxhmwdStU8uBNw8VzdJBV\nkkYTVtOQjcyis+1UYqNWLTjnHF26QFylAi8J4VreYiz9OeR4E7IEpjaNuEwtGvGc41s0AVZxOtfx\nb77A7bniHmnRAPz3v9CwIWzYANWr285GPE4tGkkK7fiSNAJ8wTm2U4mtE080t/ObONF2JpIkVODF\n8/oxjnH0wxs7nDGmNo24yAvfGLVopJDCLZoA62hEDnksoU0sonmnRQOwf7+5suS338Ipp9jORjzM\nrRZNN2AlsBp4sJj3ewFLMDfd/hIKTXNw2lakVO34kgLKsyTRrxwZrkqVoHt3yMuznYkkAacCnw6M\nwBTqFsAAoHmRdWYArYG2wI3AqxFsK1KqvoxnPH3xxs5mnPTtay5dIBIlpwLfAVgDbADygTGYEXuo\nX0IeVwF+jGBbkVIEQgp8rJQnLS0t5kvVqjXCT6l7d1i4EHbsiN1fW1KCU4GvB2wKeb45+FpRvYEV\nwEfAnRFuK1Kss/mKw5RjcUx670cUYPr9sV327NkVfkqVKsFvfgOTJkX9t5PU5nTWSLhHnyYFl07A\naOCMSJLIzc09+tjn8+Hz+SLZXJJUSrZnjujbF159FW65xXYm4hF+vx+/3x/RNk7fnI5ALqaPDvAw\ncBh4qpRt1mLaM03D3FazaKQQMzvgMGtoQl/Gs4izYxmN8Mcx0cWJ6P/5vn1mNs3atXDyybFLSxKW\nG7NoFmIKdTZQAegPTC6yTuOQIEe+iT+Fua1IsdqyCIBFtLWciSWZmdC1K7z3nu1MJIE5FfgCYCgw\nFVgOjMX02ocEF4A+wDLMNMkXgGscthVxlNLtmSM0m0ai5IVvj1o0UkhaWhqraUw/xsW4PQOebdEA\n7N0L9erB+vVQI4JZOJISdC0aSUhtgDQCqdueOaJKFejSRW0aKTMVePGcPsAE+uCNHUzL1KaRKHjh\nG6QWjRTydVoat/AZ8zkvDtE83KKBY22a776DatXcT0sSllo0knhWr+YkYAHn2s7EG6pUgYsvhg8/\ntJ2JJCAVePGWSZN4Dwjov+YxvXvr4mNSJvoWibfk5aFSVsSVV8L06eZSwiIRUIEX79i2DVasYLbt\nPLymZk1o0wZmzLCdiSQYFXjxjvffh+7dybedhxfl5OjiYxIxFXjxjrw802+W4/XqZX4BFhTYzkQS\niAq8eMPu3TB3LnTr5rxuKsrOhvr1zc9IJEwq8OINH34InTpB1aq2M/GunBzNppGI6EQnCVvVqjUi\nu3FFBN4BZgKvH30lXv8nPH6iU6ivv4YrroANGyDNC19dsUknOomrTHF3/25HFThAN05kMt8Tv8Ke\ngM48EzIyYPFi25lIglCBF+s6M4uvackP1LKdirelpalNIxFRgRfrejOJPHJsp5EYVOAlAl5o5KkH\nnyBMz8/df6tyHGIL9biQT1lLkyORXI9TsgTqwQMcPmwuPjZnDjRp4ry+JC314MXzzmUBO6gZUtyl\nVOXKQc+eOulJwqICL1blkKf2TKTUppEwqUUjYXO/RRNgNU2LuTWfWjSlOngQatWCFSugdm13PlMS\njlstmm7ASmA18GAx718LLAGWAnOBViHvbQi+vgj4PIxYkkJasJwM8nVrvkhVqADdu+tWfuLIaQSf\nDqwCugBbgC+AAcCKkHXOA5YD/8X8MsgFOgbfWw+0A3aWEkMj+ATh9gj+Uf5CTXZwNy8UjeRqnNLF\nK1YG4N51ZPoCg4HLi7yelVWd3btL+7pJsnBjBN8BWIMZiecDY4BeRdaZhynuAAuA+kXzcE5VUlFv\nJjGJVLm4WAFunhz2Ebu5gCyy+G+h12N1prEkJqcCXw/YFPJ8c/C1ktwMhN5bLADMABYCt5QlQUlO\nDdhINhuYQyfbqSSkvWQxh05cjm7lJyUr7/B+JPuulwA3AReEvHYBsA2oCUzH9PLnFN0wNzf36GOf\nz4fP54sgrCSiXrzHFHpwyPG/oJRkEr3pzSTGco3tVCQO/H4/fr8/om2c2icdMT31I9dwfRg4DDxV\nZL1WwMTgemtK+KxhwF7gmSKvqwefINzswc/gUl7kTiYf1/GD5OzBux/nFLazitOpxXYOcsLROPo+\npQY3evALgaZANlAB6A9MLrJOQ0xxv47CxT0TyAo+rgx0BZY5py3JrgY/0Z6FTOcy26kktB+oxde0\npDOzbKciHuVU4AuAocBUzEyZsZgZNEOCC8BjQHXgHxSeDlkb045ZjDn4OgWY5mLukqCu4ANmcin7\nybSdSsI70qYRKY4XZrioRZMg3GrRTCSHPHIYzfUlRXIlTngSt0UD0Jg1fMqF1GUrAcqhFk3q0LVo\nxHMqsY/OzGIKPWynkhTW0oQd1KQj822nIh6kAi9xdRnT+ZJ27KKG7VSShto0UhIVeIkrXfvdfXnk\nkEMeuhuWFKUCL3GTTgE9mMJ7xU6NlLJaRFtO4FdasNx2KuIxKvASNxfyKRtpyCYa2k4lyaSpTSPF\nUoGXuEmta8/EVx45KvByHE2TlLBFN00ywHpOowdT+IaWTpGiiBOpxJ4meUQ6BWynFm3YySZ9n1KC\npkmKZ7RmCQWU5xvOtJ1KUjpEeabQQ0c3pBAVeImLY+0ZL+w0JifTphE5RgVe4kL999ibRlc6AOzU\nDT/EUIGXmMtmPXXZyjzOs51KUttPprns2Acf2E5FPEIFXmKuF+8xmZ4cJt12KkkvD2CSZtOIoQIv\nMZdDntozcTIFYMYM2L/fdiriASrwElMns4M2LGYml9pOJSXsBDj7bJg+3XYq4gEq8BJTPZjCdC7j\nAJVsp5I6cnLUphFABV5iTBcXs6BXL3j/fSgosJ2JWKYCLzGTyS9cwmw+5HLbqaSWU0+FBg1g7lzb\nmYhlKvASM12ZxgLO5Weq204l9ahNI6jASwzp5CaLeveGvDzQdWlSWjgFvhuwElgNPFjM+9cCS4Cl\nwFygVQTbSpIqTz49mMJketpOJTW1bAnp6bBkie1MxCKnAp8OjMAU6hbAAKB5kXXWARdhCvv/AK9G\nsK0kqU7MYR2N2EwD26mkprQ0tWnEscB3ANYAG4B8YAwcd8G6ecB/g48XAPUj2FaSlNozHnCkTSMp\ny6nA1wM2hTzfHHytJDcDH5ZxW0kaARV4LzjvPNi2Ddats52JWFLe4f1IjtBcAtwEXBDptrm5uUcf\n+3w+fD5fBGHFa87hC/aRyXJa2E4ltaWnQ8+e8N57cM89trORKPn9fvx+f0TbOF2cuyOQi+mjAzwM\nHAaeKrJeK2BicL01EW6rOzoliHDv6PQkD1JAef7IX8saKaw47kiOOzqFxin0ffrgA3jqKfjkkzjE\nlnhy445OC4GmQDZQAegPTC6yTkNMcb+OY8U93G0l6QTowwQm0Md2IgJw6aVmJs0PP9jORCxwKvAF\nwFBgKrAcGAusAIYEF4DHgOrAP4BFwOcO20oSa80SynGYRbS1nYoAVKwIXbvClCm2MxELvHD/NLVo\nEkQ4LZo/8ycqcoAH+Fs0kRzjuCfJWzQAb78N77xjrk8jSSOcFo0KvIQtnAL/DS24iX+xgI7RRHKM\n454UKPA//wwNG8LWrVClShxykHhwowcvErbmLKcqu/nc3BlUvKJaNTj/fPjwQ+d1JamowItrjhxc\nDei/lff07QvjxtnOQuJMLRoJm1OLZhFtuJMXmcNF0UYqNY67UqBFA/DTT9CokWnTVK4chzwk1tSi\nkbhpxFpq8z1zj57nJp5y0knQsaOZFy8pQwVeXNGHCUyiN4dJt52KlKRvXxg/3nYWEkcq8OKKq3mX\nd7nadhpSmpwcmDYNfvnFdiYSJyrwErUGbKQR6/iYi22nIqU50qbRbJqUoQIvUevDBCbTkwIybKci\nTjSbJqWowEvU+jOWsfS3nYaEo3dvtWlSiAq8RCWb9TRmLbPobDsVCcfJJ8O556pNkyJU4CUq/RjH\nBPqoPZNI+vXTbJoUoROdJGzFnej0FW25h+f4GJ+bkY6LEzspcqJTqB9/hMaNzd2eMjPjkJPEgk50\nkphqxipqsZ05dLKdikRCbZqUoQIvZdafsYynr05uSkSaTZMS1KKRsBVu0QRYHrw08HzOczsSatGU\nPU5Y36cj16bZvBmysmKflrhOLRqJmbNYRib7mB/Vdd/FmpNOgosugkmTbGciMaQCL2VybO67F3YC\npUyuvdbc7UmSlhe+nWrRJIhjLZoAa2hCX8aziLNjEQm1aMoeJ+zv0y+/QP36sGoVnHJKbNMS17nV\noukGrARWAw8W8/4ZwDzgAPCHIu9tAJZS+GbckuDas5DDlNONtRNd5crQo4cOtiYxpwKfDozAFPkW\nwACgeZF1fgLuAJ4uZvsA4APagu7jliyuYQxjuAZv7ABKVAYOhLfesp2FxIhTge8ArMGMxPOBMUCv\nIuvsABYG3y+OqkASKcchBvAObzPQdirihi5dYO1aWLfOdiYSA04Fvh6wKeT55uBr4QoAMzC/AG6J\nLDXxokuZyVbqsvK4HTlJSBkZ5tIFOtialMo7vB/tUaELgG1ATWA6ppc/p+hKubm5Rx/7fD58Pl+U\nYSVWBjGaN7nedhripoED4eab4dFHIU073F7l9/vx+/0RbeP0r9kRyMX04AEeBg4DTxWz7jBgL/BM\nCZ9V0vuaRZMgqqSlsZkTaca37CCWsy40iyaaOBF/nwIBc9JTXh60aRObtMR1bsyiWQg0BbKBCkB/\nYHJJ8Yo8zwSOnCJXGegKLHOIJx52FfApF8a4uEvcpaXpYGuSCmd/rDvwPGZGzT+BJ4AhwfdeAWoD\nXwBVMaP7PZgZN6cAE4PrlQfeCm5blEbwCWJ6WhqvMZbx9ItxJI3go4lTpu/T8uXQtSt89x2k69pC\niSCcEbwXGm4q8IlgyxZ21q9PPfZxgEoxDqYCH02cMn+f2reHxx83hV48T9eiEfe8/TYTIQ7FXawZ\nPBhGjrSdhbhII3hxFghAq1Zc9PXXzEmq0W48YyXACH7nTnOwdcMGqFbN1azEfRrBizuWLIE9e/jU\ndh4SWzVqmPbMmDG2MxGXqMCLs1GjYNCguI2pxSK1aZKKWjRSul9/NVcc/Pxz0ho1IrnaGfGMlQAt\nGoBDh6BhQ5g+HVq0cC8tcZ1aNBK9SZOgdWs47TTbmUg8pKfD9ddrFJ8kVOCldK+/Dr/9re0sJJ5u\nvBH+/W/IL+n6gZIo1KKRkq1fDx06wKZNULFikXuyxpJaNGWXARRE/SlzgceBD0p4PyurOrt374w6\njpSdWjQSnZEjzSnsFSvazkTCVsCRu25Fs4zkVW4ip8T39+zZFce/k5SVRvBSvEOHIDsbPvgAWrUC\n0Ag+heJUYQ/fcSot+Zpt1C02jr63dmkEL2U3bRrUrXu0uEtq2UsWY+nPzfzTdioSBRV4Kd7rr5tr\nhEvKepnf8zteJd2Fnr7YoQIvx9u8GWbPhgEDbGciFi2lNZupz+V8aDsVKSMVeDneq6+ag6tZWc7r\nSlJ7md/ze162nYaUkQ6ySmEHD8Kpp8LMmcedyaiDrKkXpyL72UQDzuELNhB6spsOstqmg6wSuQkT\nTGHXaeqCuTz0m1zPLbxmOxUpA43gk0DVqjVcm5c8B3gWyCtxjcQbhXojVuLGacYqPuZiTuU7DnLC\n0Tj63tqlEXyKMMU9+pNbWrGYU6nPZPJLWEdS0beczjLOoj9jbaciEVKBl6Nu5++8whAOUd52KuIx\nz3EPd/M8+kWfWMIp8N2AlcBq4MFi3j8DmAccAP4Q4bbiEdXYRV/G8xq32E5FPOg/dCOTfVzEJ7ZT\nkQg4Ffh0YASmULcABgDNi6zzE3AH8HQZthWPGMIrvM+V/EAt26mIBwUoxwvcxT08ZzsViYBTge8A\nrAE2APnAGKBXkXV2AAuD70e6rXhABge5g//jmeN2wESOeZPruZBPacRa26lImJwKfD1gU8jzzcHX\nwhHNthJHA3iHbziTpbS2nYp42D4q8zq/5U5etJ2KhMnpaFo0R1TC3jY3N/foY5/Ph8/niyKsRCbA\nH3iG+/mb7UQkAYxgKEtpxWO2E0lBfr8fv98f0TZOBX4L0CDkeQPMSDwcYW8bWuAlvi5jOmkEmEZX\n26lIAthCff5DN37HO7ZTSTlFB7/Dhw933MapRbMQaApkAxWA/sDkEtYtOuE+km3Fkj/wDE9zH944\n500SwVM8yD0ABw7YTkUcOBX4AmAoMBVYDowFVgBDggtAbUyv/R7gj8BGoEop24pHtGERZ/IN76Cr\nRkr4ltKarwBGjbKciTjxwrBNlyqIUlkvAjaBq/iYi3mRu8KNVKY4kdOlCrwe5zzS+Cw7G1avhvI6\nMc4GXapAStSSZZzPZzqxScpkHphbOo4ZYzkTKY0KfIp6lL/yLPeyn0zbqUiieuQReOIJOHzYdiZS\nAhX4FHQ6K+nMLP7BrbZTkUTWpQtkZkJeydceFbtU4FPQo/yVF7mTveiOTRKFtDQYPhyGDYNDh2xn\nI8VQgU8xp7OSbvyHEQy1nYokg+7doWpV9eI9SrNokkAks2jG0ZeFtOd/y3Rxz+SaCRLfWMkX5+j3\ndvZs+N3vYPlyyMiIQ2yB8GbRqMAngXALfHu+YBK9acrqMh5cTbYiFc9YyRYnA3OqizEDeAf4p8tR\nsrKqs3v3Tpc/NTmowKeIcAv8NC7jXa7m1aPnqEUcKaw40VOBT7Q4HZnHWPrTlNUht/VzJ47qQ/E0\nD16O6sxMstnAv7jJdiqShOZzHotpw1BG2E5FQmgEnwScRvBpHGY+HXmGPzCO/tFEKjWOezSCT8Q4\np7OSOXSiOSv4iZNdi6P6UDyN4AWA6/g3AdIYT1/bqUgSW8UZjOEacsm1nYoEaQSfBEobwVdhDys5\ng6uYyOecG22kEuO4SyP4RI1Tg59YyRlczMesoIUrcVQfiqcRvPAof2U6l7lQ3EWc7eQkHueR4CWo\nxTaN4JNASSP4JqxmHudxFsv4njpuRCo2jvs0gk/kOBkc5Gtaci/P8gE9oo6j+lA8jeBTWoDnuIe/\ncb9LxV0kPPlU4DZeYgRDyeQX2+mkNBX4JNWHCTRiHc9zt+1UJAXNpAtzuYBhON9WTmJHLZokULRF\nU41dfMOZXM27zON8NyNhe/c/cWOlXpxT2M4yzqILM1hGqzLHUX0ons5kTRFFC/xr/JYDVOQO1086\n8U7xSLxYqRnnFl7lJv7F+XxGoEwNAxX4krjVg+8GrARWQ4lXqHox+P4SoG3I6xuApcAi4PMwYkmU\nLmEWXZnGIzxuOxURXue3HKQCd/O87VRSktMIPh1YBXQBtgBfAAMofPPsyzE3174cOBd4AegYfG89\n0A4o7WpBGsFH6cgIvgp7WEwb7uZ5pnBlLCLhpdFhYsVK3TinsY4FnIsPP8s5M+I4qg/Fc2ME3wFY\ngxmJ5wNjgF5F1ukJvBF8vACoBtQKzSOsbCVqL3AXs7kkRsVdpGzW04iHeYLRDCKDg7bTSSlOBb4e\nsCnk+ebga+GuE8BcSXQh6O7OsXQVE+jEHO0Kiyf9k5vZQj0e48+2U0kp5R3eD3ffqKRR+oXAVqAm\nMB3Ty58T5mdKmOoCL3EbPZnML1SxnY5IMdK4hddYRFtm0IWP8dlOKCU4FfgtQIOQ5w0wI/TS1qkf\nfA1McQfYAeRhWj7HFfjc3Nyjj30+Hz6fzyEtOSo/n3HAi9ypyxGIp22nNjfwBm8zkHZ8qRPwIuT3\n+/H7/a5+ZnlgLZANVAAWA82LrHM58GHwcUdgfvBxJhy9q3NlYC7QtZgYAYnC3XcHJkMgjUMBCMR4\nIQ4x4hknGf9O3o/zGLkBPxcF0skPY/3ywVixX7Kyqtv+NkckmHepnHrwBZgZMlOB5cBYzAyaIcEF\nTHFfhzkY+wpwW/D12pjR+mLMwdcpwDSnhCQC48bB5MlcD2WcYywSf3/hjxygIo/zSBhrFxCn+s6e\nPbtc+ht6hxdmuAR/GUlEvvwSunWDadNIO/tswvhl7gLvTcFLnFiKE+okfmQh7fkjf+EtrotZnMgk\n1pTMcKZJOvXgxYs2b4ZeveCVV6BtW+f1RTzmJ07mCj5gNpewkYbM4SLbKSUl7dcnmj17oEcPuOsu\nuOoq29mIlNlyzmQgbzOOfjTlW9vpJCW1aBLJgQNwxRXQpAm8/DKYXTTHe7K6J9nixDOW4pTkJv7J\nwzzBRXzCNurGLI4ztWjElvx86NcPataEl146WtxFEt2/uJlT+IEZdMGHnx2cYjulpKECnwgKCuD6\n682ssdG9WaFNAAAIdklEQVSjIT3ddkYirnqSh6nIAWbQhUuYzU5Osp1SUlCB97pff4VrrjF/TpgA\nGRm2MxKJiVxyqcgBpnMZ3fiPRvIuUIH3sn37ICcHsrJg0iSoUMF2RiIxlMZDPMkBKvIpF/IbprLB\ndkoJTgXeq7Ztg5494cwz4fXXobz+qSQVpJHLcH7gFObQiSswN5SQstE0SS9avBjOPRd694aRI1Xc\nJeW8xO3cy7NMB3oxyXY6CcsLUzE0TTLUmDFwxx1mpkzfvmFtommSiRBLccqiPWlMoAFvcAO55HKY\nWE4wSL5pkirwXrFvnzl5ye+HsWPh7LPD3lQFPhFiKU5Z45zC94ynL/upxI2MiuFVKJOvwKtF4wVf\nfQUdOpgi/9VXERV3kWT3A7W4lJks4FwW0ZYcJtpOKWFoBG/Tvn0wfDiMGgVPPw3XXVemE5g0gk+E\nWIrjRpyOzGM0g5hPR+7jabZT29VYiVSLNIL3qkAAJk+G1q1h40ZYuhQGDdLZqSIO5nMebVjMVuqy\njLMYyv+RToHttDzLCxUlriP4J554hiVLlsc8Tnp6GsOGPUCzZs0Kv7FwIdx3H/z4oxm1d+sWdSyN\n4BMhluK4Hac5yxnBUGqxnT/xP+SRQ3QlLflG8ClX4OvWPZ1t264HV3ftjnfCCa8wcuQ9DBgwwLzw\n6afw5JOmxz58OAwe7Nr0RxX4RIilOLGJE6A7H/FXHuUQ6fyZx5hCjzLeAEcFPhYsFPjJwOkxjZOV\nNYDX/n45/TMz4fnnYetWeOABuOEGqFjR1Vgq8IkQS3FiGSeNw1zFRB7kKarxMy9wF6O4McKb0KvA\nx0LSFfjTWcnvM3IYkrmNSq1bw223QZ8+MTthSQU+EWIpTnziBLiAudzN83RmFhO5itEMYg6dwhjV\nJ1+B10FWVwRoxRKG8xjLaMksOgMw87HH4OOPoX9/nY0qEhdpzOVC+vIuLfmalZzBCIayjkY8w71c\nwizKk287ybgJp8B3A1YCq4EHS1jnxeD7S4DQe8iFs21CqsdmBvEmI7mRjTQkjxwy2cctvEZ9NvNY\nxTbsqROrEzJExMk26vIM99GKpfRkMruozpM8xHZqMZZ+3MpLnMnXpHHYdqox4zSsTAdGAF2ALcAX\nwGRgRcg6lwNNgKbAucA/gI5hbpsAAtTme+ozmssooD0Lac9CKrGf2VzCLDrzOI+wmqZ4o+PlB3yW\ncwiHH+XpFj/ezxHs5ZnGMlqxjFb8hT9Rm238hqlcxCfcy7NUZxfz6cgigHffxf/rr/gGDIByid/g\ncCrwHYA1cPSqnWOAXhQu0j2BN4KPFwDVMFNUTgtjW0+ozF7qseXo0oh1NONbTmcVzfiWA1TkfjKp\nTl/G0p/7eJr1nIY3CnpRfvRld5Mf7+fpx/s5glfy/J46vMGNvMGNANRhKx2ZT2s+hNGj8fv9+IYM\nMbfGbNzY/NmkCTRqBHXrQp06cOKJCXHeilOBrwdsCnm+GTNKd1qnHlA3jG2jc+gQ7N9vzgjdv7/w\n4yN/7tkDu3YdXZ77+XtO4Haqk09NdlCPLWSQH1Le67GBbKbyG/6PO/iWZuyiBpAbXEQkmWyjLnlc\nRR4w7L33IDcX7r4b1q41y5o1MG8evPWWuYz31q3mLmt16pjl5JOhWjWoXt38eeTxiSdC5cpQqZJZ\nMjOPfxzjY3NOnx7uIeXofpW1b29+YIcOmT/DeXzokFlCf2jF/RCrVDE/7OrVoV49vqiYydbA9/y3\nXBY/pdVkW7kG/ExGyG/j3ZgrUB+7CnVV4MCBVVSs+GXYf6WDB7+kXLneUf1YRMSSatWgXTuzFGfv\nXlPst22Dn36Cn38+tqxbd+xxcYPP0MdgivyRJSOj8POir6Wnm9aRS+2jjsB/Qp4/zPEHS18Grgl5\nvhKoFea2YNo4AS1atGjREtGyhiiVB9YC2UAFYDHQvMg6lwMfBh93BOZHsK2IiFjUHViF+W3xcPC1\nIcHliBHB95cAZztsKyIiIiIiyeIPwGGghu1ESvA/mL2UxcBMoIHddEr0N8x01CXAROBEu+kUqy/w\nDXCIwnt9XpEIJ+n9C9gOLLOdiIMGwGzMv/fXwJ120ylRRcxU78XAcuAJu+mUKh1YBLxvO5FwNcAc\nlF2Pdwt8VsjjO4DXbSXi4DKOnaX8ZHDxmjOAZpgvvtcKfDqmrZgNZODd40edMGeOe73A1wbaBB9X\nwbRtvfjzBMgM/lkeczzxQou5lOZe4C3MyaMl8tKpWs8CD9hOwsGekMdVgB9tJeJgOhw9/3oBUN9i\nLiVZCXxrO4kShJ7gl8+xk/S8Zg6wy3YSYfge80sSYC9m77KuvXRKtS/4ZwXML/qdFnMpSX3M5JbX\nSZCLjfXCnAi11GlFD/grsBG4AW+OjIu6iWOznCQ8JZ28J9HLxux1LLCcR0nKYX4ZbcfsXcb+7kCR\new64H5wvohPPSxxOp/i7bDyKmWHTNeQ1m+cAl5TnI5h+16PB5SHMD3pw/FIrxClPMHkeBN6OV1JF\nhJOjFwVsJ5CkqgDvAndhRvJedBjTTjoRmIq5toLfYj5F9QB+wPTffXZTCU9LzG/L9cElH7NrfIrF\nnMLREHPAyKtuBOZiDhx5mRd78OGepOcF2Xi/Bw/mWMZU4G7biUTgT8B9tpMo4nHM3uV6YBvwC/Cm\n1Ywi5OWDrE1DHt8BjLaViINumBkLJ9tOJAyzgRLOB7cmkU7Sy8b7BT4NU4Ses52Ig5MxF0sEqAR8\nAlxqLx1HF+PtPeFircO7Bf5dzJdpMTAB7+5lrAa+w+zGLQJesptOsXIwI5H9mINwH9lN5ziJcJLe\nO8BW4FfMz9JWu9DJhZjWx2KO/Z+M/m7z7jsL+AqT51JMn9vLLsZhFo2IiIiIiIiIiIiIiIiIiIiI\niIiIiIiIiIiIiHX/D5f/XTeq0i4UAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sampling = InversionSampling(Phi_inv)\n", "def gauss_hist (ax, data):\n", " x = np.arange(-4,4,0.01)\n", " ax.hist(data, normed=True)\n", " ax.plot(x, phi(x), 'r-')\n", " \n", "show_sampling(sampling, plotter=gauss_hist)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Similarly, we can illustrate that expectations computed on the samples converge to the true value:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def show_expectation (sampling, f):\n", " \"\"\"\n", " Show how the expectation of f converges when increasing the number of samples\n", " \"\"\"\n", " n = np.arange(1,2500)\n", " tmp = [f(sampling.sample()) for _ in n]\n", " \n", " Ef = np.cumsum(tmp)/n\n", " plt.plot(n, Ef, 'b-')" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNW99/EPDrsCyiLIIqOAKO4aUR+39kZlUQOJ+iBR\nDMZckSsaiV64Gl9xvBojJK5xw32LYJRHRSUgio2IoqBsiggzLAKiIMgus57nj193qqene7qH6ame\nnvq+X69+VXXV6eozRfPr0786dQ6IiIiIiIiIiIiIiIiIiIiIiIiIiEgGPQ18DyxJsv8yYBGwGJgD\nHONTvUREJIPOAI4nebA/FWgTWe8PzPWjUiIiknn5JA/2sQ4A1tVtVUREpKb2yfDxrgKmZviYIiLi\nk3xSt+zPBpZirXsREalHGmfoOMcAT2A5+x8TFejRo4crKirK0NuJiARGEdCztgfJRBrnYOD/AZcD\nhckKFRUV4ZzTwzluu+22rNehvjx0LnQudC6qfwA9MhCn02rZTwTOAtoDa4HbgCaRfROAP2Gpm0cj\n20qBvpmonIiIZEY6wX5oiv2/izxERKSeynRvHElDKBTKdhXqDZ0Lj86FR+ci8xr5+F4ukn8SEZE0\nNWrUCDIQq9WyFxEJAAV7EZEAULAXEQkABXsRkQBQsBcRCQAFexGRAFCwFxEJAAV7EZEAULAXEQkA\nBXsRkQBQsBcRCQAFexGRAFCwFxEJAAV7H333XbZrICJBpWDvA+egvBwOOggaNYLCpJM3iojUDQV7\nH1x4ITSOmROsV6/s1UVEgknB3gdFRZWft2gBP/6YnbqISDClE+yfBr4HliTZfzjwMbAHuDFD9Wow\nxo+HZcu85zNnwgUXwG23wZ//nL16iUiwpDPV1RnATuB54OgE+zsA3YHBwI/APUmOE6hpCf/0Jxg+\nHHr0sOd/+xtcfjl07AjvvAP9+tn2Tz6Bvn2zVk0Rqef8nJZwNhbEk9kEzAdKa1uZhqKiAu64wwv0\nAN9/b4Ee4Oc/97affDIUF/tbPxEJnsapi0hNLV1a+fmHH8Jhh3nP8/Ksh05REfTsCc2b23MRkbqi\nC7R1YM0a6NzZArhzcNpp0KFD1XI9esDo0ba+YYO/dRSRYPG1ZV9QUPDv9VAoRCgU8vPt90ppKTRp\nUrPXXH01fPttemXHj4f77oNRo2DyZG/7t996/fJFJDjC4TDhcDjjx003lOQDb5L4Am1UAbCDBnSB\ntndvWL7ccvA1Cbp33AFz58Lbb6dXft487yKtc/DAA3DDDfCb38CzzyZ/3dix0L8/nH22t+2HH6y+\nq1bBIYfAgQemX28RqX8ydYE2nQNMBM4C2mNdMG8Dom3dCUAnYB7QGqjAAn4frAdPrJwL9tEAv2oV\n5Oen/7q77oJdu9LvWumcXaidN89e++KLXt5/zRo4+ODq6zdzJvzHf1jw/+gjmD3brgMcfTR8+mn6\n9RaR+idTwd5PLtdcckk06+5tA+fuvNO5iorKZRcudO6445ybM8fKFBTU/P28LL9zn31my0sucW7W\nLOe2bq1cdv36yuWTPXbsqHk9RKT+ADLSStYF2mqUlXnrX37prd96K+yzD3z8sbetqAgWLrSLsfGv\nrcn79elj6927W6v8lVfgrLPg7ru9chUV8L//a+vnnWfL0aNh4EBbP/ZYW/buDa1apX/9QEQaLgX7\napSUwL33WmAfPBi++KLy/unTvfXPP6+8L36IhHTk5cFrr8EHH0C7dvCzn0H79rbvzTdhyBDYvRsu\nvRQmTLDtf/+7teHvvRfeesu2/fGPsGcP3HmnPR86tOZ1EZGGxc88UOQXSe7o189azIcfbhc7o7p0\ngfXr4cwzYdYsu4h65ZXe/mXLoFMnaNOm9nXYvt2GRu7du+q+++6zC7mxVq2Crl29HkSFhXDqqfDZ\nZ8lz/yJSf2UqZ6+bqqoR7XaZn2/pknfesZTKjBmwcqV9CXzyiRfo58yxlneiwLy3Wre2x5AhluaJ\n7Z4Ze6NWVOyXEthNWz/8YGmhmvYqEpGGQy37apxxhvWoOfNMe37DDfDLX1rABy9wRr8I/Pjzli+3\nPPyBB1raJx0LFsAJJ8CYMTBuXN3WT0Qyy8+xcQKrpASaNvWe33+/F+gBnn/elkVFMHWqP3U67DC7\n2SrdQA9w/PH2ZfTYY5bLLy62i7m33qphGkSCQsG+Gqnunh02zJZFRZZqqc/OPRfatrWx9GfOhH/9\ny361zJoFTzzhlfvsM/uSE5GGRcG+GukMlfDcc7as6ZAK2TBrli2jXTSvvdZm0br6aruha9066wHU\nrJmti0jDoWBfjXSC/RVXwOrVcNJJvlSpVg4+GLZssfWnnrI0zs7Ifc59+9oduNE7hbt1s+WiRV5f\nfrX4RXKXgn014nP2yXTvnju9XA44wPL0v/2tdQ9dswamTbO/86WX7Itr40a7JnDXXXZvwYwZcMkl\n1uKPvZFMRHKHeuMkUFJiY83MmWP96Tt3znaN/DFsGFx8MQwaBM88Y18Iibz7buUJWMR+BW7ebF+g\n8e64w3pxHXVUzY/73XfWZTYon0GpSmPj1KEXX/TGlokfAydI/vu/7Ry8955z113n3A8/eOclfqye\noPvLX+y8LF7s3LBhtv7KK85t3mzrLVs69803VV+3datzzz3nXGmpc3v2OLdunXOvveZcSYlzU6d6\n5/vII731Dh2cO/FE5yZODPbnMyjI0Ng4atkn8Oab8Itf2IQjGzdmuzbZVVFhw0VEffih3X8wbJjX\n9TS2bHl5blys3hubN1t6q7zc0lk9e9pj1y7Yf39Lc0WnmOzUyVrlUffeCw8+aGMdLV1qI5P27w8j\nRthxk/nlL+Gaa+xu7t697RdVXp4d+5VX7BpL5842HtL48bDffnV6CiQLdAdtHdqzx5ZXXZXdetQH\n+8Rd1Tn9dOupc/TRNsBaXp7dNbzvvvDII3D77RYM419XUmJfArlybSOR6DhFyezaZXdWL1sGAwbY\n9Y/iYgvSjRvb9ZLYC/lPPmnLoiIb62j8eOsddeKJcOihNrDeNddAr16J74fYscPGP3r1VesyO2GC\njYt0++25fZ4l92X711Dann7aud697Se4JHb22ZWHUm7TxrmrrrL1QYOqphdOOsn2zZ2bnfrW1rx5\nVv/+/Z3r2dO5DRucW77cub/+1bnPP3fugw/SO86aNc4VF1uaZs4c5xYtykz9KiqcGz/e+/e47DLn\n/vAHex/JbWQojeOnbJ+ztF13nXO//nW2a1G/bdzo3I03WmA55BAvyFxxhS2nTatcvl8/r8y2bdUf\ne+dO55Ytq7u611RJiXPXXuvc6NH2vKwsu/Wpzu7dzr39tnPdutm5Hj7c++LdtMmuJ8yYkd06Ss2g\nYF934icskeQqKuyxcKFzp53m3D/+4dzkyXb+zj/fyjz+uD1/6y3nLr/cAs7cuc49+qht37278jFj\nfzFcdZX/f1PUwoXOPfywV5fJk7NXl5qqqLCJa044wSbAefJJ7+9o2dK5MWOcW73afqGAcwcfbL9U\niosTH0uyBwX7utOihQUo2TsVFV7LMvrIz7cW8qpVVWfTat3aufJye11pqW076yznDjrIK/P669ZT\nJWrsWOfOPde+UGbNqn2dP/3Uua++suMOG+Zc376V6/jAA7kZ9H76ybkLL/T+jldesVnQGjWq/Pf9\n139566ee6tyUKZaO697duWbNnLv4Yue+/LLq8Zcsce6Pf3Ru+3ZvW3Gxc0VFzr3xhnNr1yavW3m5\nzbgWXc/F8+sHFOzrzrHHWh5Waufdd70AsmePt33rVucWLHBu5UoL4NEy/ft76z/9ZGU3bXLuV7/y\ntj/0kHOzZ1f9wpg/v/rAEi82l/3hh4mndLzwwoYxrWNFReJzs2CBc489Zl8Azjm3dKlzL71UOfC3\naOHcyJHOXXONfUGMGWPnbuVK577+2rlOnaxc48bOdezo3MCBic9l//6WGm3RwrlDD3UuHPb2DR/u\n3OGH2z6wbqVFRc7deqtzAwbY+y9f7tyPP9r77txZP1Jpq1ZZQ2PAAEv93n67dU/eutX7/GYCPna9\nfBo4H9gIHJ2kzIPAAGA3MBxYkKBMpN71X58+1q3tyCOzXZNg+OknaNnS1o87znqkRKddjFqyxMb2\nue46b9sLL9jNTC1b2uxdYMM7HHNM9e+3YoWNHnr99XD++XaTWFmZdaO89lobRrprV5u0Pai9Wtau\ntbuqO3asvO2qq6yraOzMbFu2WHfSW26xc9uzp83iVlxs5/Xrr23eh0mTrGfS3LnWc2nDBtv37LPw\nz3/a6xYvth5LU6Z4xz/8cOvhFO/66+Gvf03vLve60LWr3XTZurV9Zt9/v/IUoG3aWB2vvLLyPBNr\n1tid6Z07wz/+YZ+3Dh2sR9tFF1UeWRf8vanqDOB4YEmS/QOB6AC/JwNzk5TL3FddHZo3z7nOnZ1b\nsSLbNQmW4mLL3af6KT91qrUQ41vcL73k3NVXW0+ZhQuTH6esLHHL8/77bb9SCdUrL7cW7MiRidM6\nNZGsdV5R4dwTT9gvw+gvwrIy57ZssYv7d99tv0YuuMBLQS1b5ty4cc5Nn167Ojln1zFeftl+bTz7\nrP0CmjTJejtFf5EMHmzLLVsqv3b9eqv37Nl2c2a0Y0KrVs797GfOHXVU1c/exRfb8fLznTvgAOdu\nuqny5xuf0zj5JA/2jwFDYp4vAzomKFf7fwUfRP8BVq3Kdk1kbwwf7v0bDhhQ9U7fTZts34YNlhKI\n3qU6dWp26it7r6LCuS++8HqARR+jR1vQffHFyum64mLnjj/ePiOvvmoNu5kzLY03eLBdgI92GgDn\nLrrI0lOJGgdjxlgKLB3ffGONicmT7Yty7FhLX5aU2JdnrO++s26z4Fzbts716uX/HbT5wJskTuO8\nCfwF+Cjy/F1gLPBZXDnnciCNE/3Zrin8clNZmd2peuON3rYHHrAbmg491G4Kg8o3KTmnf+tc9+WX\ncMQRsG0bXH65N5lQ8+Z2k2SHDpZWadLEZnpbvNi7eRIsXdWqlc3ZfMEFdoNbt242/0NUXp6lbfbZ\nxyYQqksffWTDpw8cCIMHZyaNk6lgfzcwJ/L8XWAM8HlcOXdbzJNQ5CEiIp5w5BF1uy3qxXAJ64Fu\nMc+7RrZVUZADLfu2be3bvW3bbNdEMmXTJpuc5Ysv4Kabqg7lIFKfhKjcEL49Qz87M9GyHwiMiixP\nAe6PLOPVyzTO7t320791a+vZ0bSp9RSI9g4REckmPwdCmwicBbQH1gK3AdFxDSdgPXEGAoXALuDK\n2lbKTyeeaHm8xYttJieonKcTEWkIcmqI43btYMECLyjvjc8/h+3bIRSyC3PRn/SxF+nq4Q8QEQmo\nQA5xvGULzJ+/98HeOWvJg/W2ib1RY/Vqu/o+cWKtqykiUu/k3KWqyy6Dxx/fu9eWlnrrDz5od8pG\nxygfPdpa9l261L6OIiL1Tc4F+z17bMKGvVFS4q3fcIMty8rgjTfslv0dO6yvrYhIQ5NTOftoTr1V\nK8u719TGjTbOxqGH2sw+Ud99500UXVycvbE2RETiZSpnn5PB3g6296/fuBEOPLDysXRxVkTqo0wF\n+5xK4zSOu5xcXm4TPX/5Zc2O06GDtx4dIfE3v4GXXqpd/URE6qucCvbNm3vrjRpZ8N+2DZ5+uvrX\njRrlteTPOMOWmzfbl8WiRfb82Wdh6NCMV1lEpF7ImTTO5s3Wc2btWusiGWvgQHj77eSvbdfOum2C\n9ciJ/4UgIlJfBS6NE22Rt2tXdV+q4L3vvrYcOVKBXkSCKWeC/Vdf2TIvz2ayibV6dfWv3bzZlrFD\nmoqIBEnOpHGivWXKyiwVEw7DgAGwapVdZN22LfmY5M2awaef2njX6lYpIrkkcF0v4ycV2bTJLrru\n3m0TCRQVJU7xRMe/KS/X0LYiknsCl7OPigb96HjzTZva5NHRNE+8sjLL0yvQi0iQ5UwI7NnTZqKP\nysuzVntens1YX1iY+HW6I1ZEJIeCfXm5BfZE8vNhzZrE+0pKFOxFRBpEsO/ePXmPnPHjYevWOquW\niEhOyKlgn6yPfKqWvYhI0OVMsC8rqz6Ns2pV4n3HHGPj3oiIBFm9DvZ79liLfs8e+P775MH+4IPh\n228rT04SVd2XhIhIUKQT7PsDy4AVwNgE+w8AXgMWAZ8AR2aqcu3a2SBmGzbY82RpnKZNbTz6tWur\n7qsu1y8iEhSpgn0e8BAW8PsAQ4Ej4srcAnwOHAtcATyQqcrt3g0ff+wF+YqK5GX33x/ef7/q9upy\n/SIiQZEq2PcFCoHVQCkwCRgUV+YIIBpmvwbygQ5kyPbtXnomOl9sIosXw+9+V3W70jgiIqmDfRcg\nNjmyLrIt1iLgV5H1vkB3oGtGaofNC7ttGxxySPXlYice2bHDW1caR0QEUiU40hnM5m4sdbMAWBJZ\nlicqWFBQ8O/1UChEKBRKefDt2+GEE1JXYvBgm9zk4oth8mS7u7awEObMsd46IiK5IBwOEw6HM37c\nVIPrnAIUYDl7gJuBCmBcNa9ZBRwN7Izb7s491zFiBFx0UXqVa9LE0jD/PkCKr57YUS937YJ+/eDD\nD2HMGBhXXY1FROopvwZCmw/0wvLwTYEhwJS4Mm0i+wD+E5hF1UAPwIwZ8PLL6VfugAPSLxuvf3+7\naAtK44iIpAr2ZcAoYDqwFHgZ+AoYEXmA9dJZgnXP7Af8vroDFhenX7nokMUnnQRPPZW6/MiR3vrs\n2fDWW7auSUtEJOjS6ZT4r8gj1oSY9Y+B3um+YXXDF7zxBrz3Hjz4oD1v2dKWu3fDUUelPvadd0KL\nFjYfbewImcnurhURCQrf76BNdJdr1MMPw9//XrXs+vXpjUffti3cc49dlI2VzheFiEhD5vvtRj/9\nlHxf/FDEpaXQpo2NWlmTvHu7dqkv5oqIBInvLfuPPkq+b9myys9LSmzcG9BFVhGR2qhXA6FF74CN\ntspLSxXsRUQyIWvB/vnn4bjjKm9r3tyWu3fbMrZlrzlkRUT2XtZC6PTpsGiRzSTlHJx/Pqxcafui\nM0upZS8ikhlZGw+ySRNbjh0Ll18OU6d6+7ZuhS5dlLMXEcmUrLXsY4cdjo5XH7Vrly1LS6FbN1tX\nGkdEZO/5GkJjR66MtuwBfvyxcrmdkcEWYlv28d0yRUQkfb4G+9hUTGzLfsuWyuV27rShiQE6d7Zl\np051WzcRkYbM12AfG+BjW/ZDhlQu9/XX1qpv2tTKOaecvYhIbfga7GMDfOx6vDFjLF9fXRkREUlf\nvUjjxGvWDF54wcvdi4hI7dSLNA7AfvvZ8oILbBjkUaP8q5eISEOXtWAf37I/7zxb7ruvf/UREQmK\nrAT7nj1h6dLK++6915aNaj35loiIxPP1DtposC8qskes9u1tWZupCEVEJLGspXE6dqy8r1kzW8ZO\nWzhtWt3XSUQkCLLWG6esLPG+2PliozdWiYhI7aQT7Ptjk4mvAMYm2N8emAYsBL4Ahid9s5h3iw/2\n0Vx9bLDXROEiIpmRKtjnAQ9hAb8PMBQ4Iq7MKGABcBwQAu4hybWA2KkCt21L/IatWsHIkbZ+7LEp\naiciImlJdYG2L1AIrI48nwQMAr6KKbMBOCay3hrYDMS1202qeWG/+cYu0O63HzzySIqaiYhI2lIF\n+y7A2pjn64CT48o8AcwEvgVaAf832cFSBfvocMYiIpJZqYJ9ivAMwC1Yvj4E9ABmAMcCO+ILFhYW\nxDwLRR4iIhIVDocJh8MZP26qW5hOAQqwnD3AzUAFMC6mzFTgz8CcyPP3sAu58+OO5c45x/Huu1Xf\nZP16byhjERHxNLLeK7W+3TTVBdr5QC8gH2gKDAGmxJVZBpwTWe8I9AZWJjpYsjSOAr2ISN1KlcYp\nw3rbTMd65jyFXZwdEdk/AbgLeAZYhH15jAG2VDkSiYP9nXfWvNIiIlIzfo5E484+2/HBB5Vvlkp1\n0VZEJMj8SuNklAK7iEh2+B7sY0e1fP11P99dRCS4stqy793bz3cXEQkuX4N9RUXl55pEXETEH74G\ne6g8AFrz5n6/u4hIMGU1jXPQQX6+u4hIcPk6U1VsGkc9c0RE/ON7GgfgwAOz8a4iIsHlexrntddg\n+XI/31VERHzvjdOqFbRp4+e7ioiI7qAVEQmArN5BKyIi/lDLXkQkABTsRUQCQGkcEZEAUMteRCQA\nFOxFRAJAaRwRkQBQy15EJADSCfb9gWXACmBsgv03AQsijyXYJOX7JzqQWvYiItmRKtjnAQ9hAb8P\nMBQ4Iq7M34DjI4+bgTCwNdHB1LIXEcmOVMG+L1AIrAZKgUnAoGrK/xqYmGyngr2ISHakCvZdgLUx\nz9dFtiXSEugHTE52MKVxRESyI9XkJTVpi18IfEiSFA7Axo0FPPMMzJwJoVCIUChUg8OLiDR84XCY\ncDic8eOmamefAhRgOXuwnHwFMC5B2deAl7FUTyKud2/H44/DmWfuRU1FRAKokaVDap0TSZXGmQ/0\nAvKBpsAQYEqCcm2AM4E3qjuY0jgiItmRKo1TBowCpmM9c54CvgJGRPZPiCwHR8r8VN3BdIFWRCQ7\n/Gxnu549Hc88A6ef7uO7iojkML/SOBmllr2ISHYo2IuIBIAGQhMRCQC17EVEAsDXYL9jh5/vJiIi\nUb4G+82blcYREckGX4O9iIhkh+/BXi17ERH/qWUvIhIACvYiIgGgNI6ISACoZS8iEgBq2YuIBIBa\n9iIiAaBgLyISAErjiIgEgO/BvqLC73cUEREFexGRAPA92JeX+/2OIiKSTrDvDywDVgBjk5QJAQuA\nL4BwdQdTsBcR8V/jFPvzgIeAc4D1wDxgCvBVTJn9gYeBfsA6oH11B1SwFxHxX6qWfV+gEFgNlAKT\ngEFxZX4NTMYCPcAP1R1QwV5ExH+pgn0XYG3M83WRbbF6AW2B94H5wLDqDqhgLyLiv1RpnHRmjW0C\nnAD8HGgJfAzMxXL8cQp44QX45BMIhUKEQqEaVVZEpKELh8OEw+GMHzfVLU6nAAXYRVqAm4EKYFxM\nmbFAi0g5gCeBacCrccdy4Hj9dRgUnwgSEZGEGtmdqLW+HTVVGmc+lqbJB5oCQ7ALtLHeAE7HLua2\nBE4GliY7oNI4IiL+S5XGKQNGAdOxYP4U1hNnRGT/BKxb5jRgMdbqfwIFexGResXPkWocOCZOhEsv\n9fFdRURymF9pnIxTy15ExH++B/uyMr/fUUREfA/2Rxzh9zuKiEiqC7QZtWcPNGvm5zuKiAj4fIHW\nuXTu0RIRkaicvUArIiL+U7AXEQkABXsRkQBQsBcRCQAFexGRAFCwFxEJAAV7EZEAULAXEQkABXsR\nkQBQsBcRCQAFexGRAFCwFxEJAAV7EZEASCfY98fmmV0BjE2wPwRsAxZEHrdmqnIiIpIZqcazzwMe\nAs4B1gPzgCnYpOOxZgG/yHjtREQkI1K17PsChcBqoBSYBAxKUM7PcfFFRKSGUgX7LsDamOfrItti\nOeD/AIuAqUCfjNVOREQyIlUaJ52ppT4HugG7gQHA68BhtayXiIhkUKpgvx4L5FHdsNZ9rB0x6/8C\nHgHaAlviD1ZQUPDv9VAoRCgUSr+mIiIBEA6HCYfDGT9uqlx7Y+Br4OfAt8CnwFAqX6DtCGzEfgX0\nBf4J5Cc4luagFRGpoUzNQZuqZV8GjAKmYz1znsIC/YjI/gnAxcDISNndwKW1rZSIiGSWn71o1LIX\nEamhTLXsdQetiEgAKNiLiASAgr2ISAAo2IuIBICCvYhIACjYi4gEgIK9iEgAKNiLiASAgr2ISAAo\n2IuIBICCvYhIACjYi4gEgIK9iEgAKNiLiASAgr2ISAAo2IuIBICCvYhIACjYi4gEgIK9iEgApBPs\n+wPLgBXA2GrKnYRNOv6rDNRLREQyKFWwzwMewgJ+H2AocESScuOAafg7iXlOCofD2a5CvaFz4dG5\n8OhcZF6qYN8XKARWA6XAJGBQgnLXAa8CmzJZuYZKH2SPzoVH58Kjc5F5qYJ9F2BtzPN1kW3xZQYB\nj0aeu8xUTUREMiVVsE8ncN8P/E+kbCOUxhERqXdSBeZTgAIsZw9wM1CB5eejVsYcpz2wG/hPYErc\nsQqBHrWoq4hIEBUBPev6TRpH3igfaAosJPEF2qhnUG8cEZF6p3GK/WXAKGA61uPmKeArYERk/4S6\nq5qIiIiIiNQr6d6U1ZCsBhYDC4BPI9vaAjOA5cA7wP4x5W/Gzs8y4Dzfalk3nga+B5bEbNubv/3E\nyDFWAA/UYX3rUqJzUYD1alsQeQyI2deQz0U34H3gS+AL4PrI9iB+NpKdiwJy+LORh12YzQeakDrn\n31Cswj7EscYDYyLrY4G7I+t9sPPSBDtPheT2MBZnAMdTOcDV5G+PXuz/FLvPA2AqXieBXJLoXNwG\n/CFB2YZ+LjoBx0XW9wO+xmJBED8byc5FnX426jqopHtTVkMU39PpF8BzkfXngMGR9UHAROz8rMbO\nV19y12zgx7htNfnbTwYOAlrh/Sp6PuY1uSTRuYDEveAa+rn4DgtYADuxa39dCOZnI9m5gDr8bNR1\nsE/npqyGyAHvAvOxbqgAHbGf9ESWHSPrnbHzEtUQz1FN//b47etpWOfkOmAR1uEhmrYI0rnIx37x\nfII+G/nYuZgbeV5nn426DvZBvZv2NOwfcABwLfZzPpaj+nPTkM9bqr+9oXsUOAT7Gb8BuCe71fHd\nfsBk4PfAjrh9Qfts7IcNM/N7rIVfp5+Nug7267GLEVHdqPxN1FBtiCw3Aa9haZnvsVwd2M+vjZH1\n+HPUNbKtIanJ374usr1r3PaGck424gW1J/FSdkE4F02wQP8C8HpkW1A/G9Fz8SLeucjpz0ZNb8pq\nCFpieTSAfYE52NXz8Xi9kf6HqheimmLf6kXk/pAT+VS9QFvTv/0TLC/ZiNy8CBeVT+VzcVDM+mjg\npch6Qz8XjbCc8n1x24P42Uh2LnL+szEAu9pciHUfaugOwf5hFmLdqqJ/c1ssj5+oi9kt2PlZBvTz\nraZ1YyLwLVCCXa+5kr3726NdygqBB+u81nUj/lz8FvtPvhjLy76Ol6OGhn0uTseGWlmI17WwP8H8\nbCQ6FwMZX6+QAAAAJUlEQVQI7mdDRERERERERERERERERERERERERERERERERCTz/j8fQhX+sqL4\nqQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_expectation(sampling, lambda x: x**2)\n", "plt.axhline(1.0, color='r'); # The true variance" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Exercise:** \n", "\n", "* Does your favorite library function, i.e. *numpy.random.normal*, use inversion sampling? If not, which algorithm is most used instead?\n", "* How does the Box-Muller method work?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Rejection sampling\n", "\n", "The next sampling method is less efficient, but more general in the sense that it can be used even if the normalization constant of the desired density $p(x)$ is unavailable.\n", "\n", "It is based on the idea that a sample at position $x$ should be drawn with probability $p(x)$, i.e. the height under the graph of $x$. Thus, a sample can be drawn using two random numbers as follows:\n", "\n", "* Draw a sample $x$ from some distribution $q(x)$\n", "* Draw a uniform number $u$ between 0 and $c q(x)$:\n", " - If $u < p(x)$ return the sample $x$\n", " - Else discard $x$ and repeat\n", " \n", "Here, it is assumed that samples from $q(x)$ can be drawn efficiently. Further, the scaling constant $c$ must be choosen such that $c q(x) \\geq p(x) \\quad \\forall x$. The plot below illustrates the idea for the Gaussian distribution:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEACAYAAACuzv3DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VEUXwOHfJnQTIAlICx2UIoLSa0IoAgEURZQmRYSA\nBZAm0juIFJEiRCkfSBFFei8hVAHpnSA9CKGGlj7fHzeJCSRkk+zu3XLe59mHTXb2ztlkOTs5d+4M\nCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQpjcXOAWcCKZx9sCx4DjwB7gTQvFJYQQwki1gbdIPpFX\nB3LE3m8E7LdEUEIIIVKnCMkn8oTcgOvmDUUIIURCTiY+3qfAehMfUwghhAkUIeUReV3gNNqoXAgh\nhIVkMNFx3gT80Wrk95NqULx4cXXx4kUTdSeEEA7jIlDiZQ1MUVopBKwA2gFByUZy8SJKKbu9DRs2\nTPcY5PXJa5PXZ383oHhKSdiYEfkSwAvIBVwDhgEZYx+bDQxFK6fMiv1eJFDFiOMKIYQwAWMSeesU\nHu8SexNCCKEDU89acVje3t56h2BW9vz67Pm1gbw+R2CwYF8qtt4jhBDCSAaDAVLI1aaatSKESIa7\nuzv37yc5mUuIeG5ubty7dy9Nz5URuRBmZjAYkPe+SEly7xNjRuRSIxdCCBsniVwIIWycJHIhhLBx\nksiFEBa3a9cuSpUqZbH+Tp8+TeXKlY1q27JlSzZu3GjmiExLErkQDqxIkSJky5aN7Nmz4+bmRs2a\nNZk9e7bZT87Wrl2bs2fPJopj+/btZutvyJAh9OvXz6i2AwYMYPDgwWaLxRwkkQvhwAwGA2vXriU0\nNJSrV6/yzTffMGHCBD799FOLx2GuD4+bN28SEBDAe++9Z1T7ypUrExoayt9//22WeMxBErkQAgBX\nV1eaNWvGsmXLWLBgAadOnQIgPDycvn37UrhwYfLmzUv37t0JCwsDICAgAE9PTyZPnkyePHnInz8/\n8+fPjz/m+vXrKVu2LNmzZ8fT05NJkybFP69gwYIAtG/fnqtXr9KsWTNcXV2ZOHEiTZs2Zfr06Yni\ne/PNN1m1atULcV++fBknJyf8/f0pUKAA+fPnj+8HYMuWLVSsWJFMmTIB2gJ+Hh4eHDlyBIDg4GBy\n585NYGBg/HO8vb1Zt25den+kFiOJXAiRSOXKlfH09GT37t0AfPPNNwQFBXHs2DGCgoK4ceMGI0eO\njG9/69YtQkNDCQ4O5pdffuHzzz/n4cOHAHz66afMmTOH0NBQTp06hY+Pzwv9LVy4kEKFCrF27Voe\nPXpEv3796NChA4sWLYpvc+zYMYKDg/H19U027oCAAIKCgti8eTMTJkxg27ZtAJw4cYLXX389vl3x\n4sWZMGEC7dq149mzZ3Tq1IlOnTpRp06d+DalS5fm2LFjafwJWp4kciF0ZjCY5mZK+fPn5969eyil\n8Pf3Z/LkyeTMmRMXFxcGDhzI0qVL49tmzJiRoUOH4uzsTOPGjXFxceHcuXMAZMqUiVOnThEaGkqO\nHDl46623jOq/WbNmnD9/nrg9DBYuXMjHH39MhgzJX4w+bNgwsmbNyhtvvEGnTp1YsmQJAA8fPsTF\nxSVR2y5dulCiRAmqVKnCrVu3GDNmTKLHXVxcePDggVGxWgNJ5ELoTCnT3Ezp+vXruLu7c+fOHZ4+\nfUrFihVxc3PDzc2Nxo0bc+fOnfi2Hh4eODn9l0qyZcvG48ePAfjjjz9Yv349RYoUwdvbm/37jdub\nPUuWLLRq1YqFCxeilGLp0qW0b9/+pc+JK9UAFCpUiODgYEC79P3Ro0cvtO/SpQunTp3iyy+/JGPG\njIkee/ToETlz5jQqVmsgiVwIkcjBgwcJDg6mVq1aeHh4kDVrVk6fPs39+/e5f/8+Dx48IDQ01Khj\nVapUiZUrVxISEsJ7771Hq1atkmxnSOJPig4dOvDrr7+ydetWsmXLRtWqVV/a19WrVxPdL1CgAKDV\n1s+fP5+o7ePHj+nVqxddunRh2LBhL6yFc+bMGSpUqGDUa7QGksiFcHBxs0VCQ0NZu3YtrVu3pn37\n9pQtWxYnJyc+++wzevXqRUhICAA3btxg8+bNKR43MjKSX3/9lYcPH+Ls7IyrqyvOzs5Jts2TJw/P\nbwVZvXp1DAYDffv25ZNPPkmxv9GjR/Ps2TNOnTrF/Pnz+eijjwCoX78+hw8fJiIiIr5tz549qVKl\nCnPmzMHX1xc/P79ExwoMDKRx48Yp9umIlBCOyJrf+0WKFFFZs2ZVrq6uKkeOHKpGjRpq5syZKiYm\nJr5NWFiY+vbbb1WxYsVU9uzZVenSpdWPP/6olFJqx44dqmDBgi8cc9u2bSoiIkI1atRIubm5qezZ\ns6sqVaqoPXv2JPm8VatWqUKFCqmcOXOqSZMmxX9/1KhRymAwqEuXLiX7Gi5duqQMBoPy9/dX+fPn\nV3nz5lUTJ05M1ObDDz9Uy5YtU0optXLlSuXp6anu37+vlFLq8ePHqkSJEmrx4sVKKaUOHDigKlas\nmNofZbol9z4BUiycyeqHQpiZrH6YdgsXLsTf3z/R1MDnXb58mWLFihEVFZWoVp/QmTNn6NChAwcO\nHEixz5YtW9KlSxcaNWqU5rjTIj2rH0oiF8LMJJGnzdOnT/Hx8eGLL76gXbt2ybYzJpHbAlnGVghh\nVzZt2sSrr75Kvnz5aNOmTYrtkzpZ6khkRC6EmcmIXBhDRuRCCOHAJJELIYSNk0QuhBA2ThK5EELY\nOEnkQghh4ySRCyEsKuFa5LZu9uzZ9O7dO8V24eHhlC5dOtFiY6YkiVwIBxa31Zurq2v87auvvtI7\nLF106tQpVe0jIiIYM2YM/fv3T7Ft5syZ6dy5M+PHj09reC9lTCKfC9wCTrykzTTgAnAMMG7BYSGE\n7uK2env06FH8bdq0aXqHZVHdu3ePXznx7t27dOvWjWfPnqX4vFWrVlG6dGny5ctnVD+tW7dmwYIF\nREZGpivepBiTyOcBL1t0oAlQAigJdAVmmSAuIUxOKcXBGweZsHsCH/3+ERXnVKTI1CLkn5SfEtNK\nUGtuLT5b/Rn+f/tz9eHVlA9o5+bPn0+tWrXo168f7u7uFCtWLH53+WXLlr2wK/2UKVN49913gZdv\nD/e8M2fO4O3tjZubG2+88QZr1qyJf6xjx474+fnRsGFDsmfPjre3d6Llas+ePUuDBg3w8PCgVKlS\nLF++PNnXc+/ePTp16kSBAgVwd3enRYsWAAwcOJChQ4eya9cuPv/8c3r27EnWrFmJjo6mb9++5M6d\nm+LFizNjxgycnJyIiYkBYMOGDXh5ecUff9myZRQrVix+7fMNGzaQL18+7t69C4Cnpydubm7s27fP\nuF9AKhiTyHcB91/yeHNgQez9v4CcQJ50xiWEyVwPvc6gbYMoOKUg7f9sz83HN2lasimzfGexo8MO\nDn52kA1tNzDaZzRv53ubgCsBVJxTkcr+lfH/258nEU/0fglm9bKrTg8cOECpUqW4e/cu/fv3j9+U\nuVmzZpw7d46goKD4tosXL6Zt27ZAytvDxYmMjKRZs2Y0atSIkJAQfvzxR9q2bZto/fDFixczdOhQ\n7ty5Q4UKFeL7ePLkCQ0aNKBdu3aEhISwdOlSevTowZkzZ5J8Le3btycsLIzTp09z+/Ztvv766/jX\n7+TkhFIq0dWV/v7+rFu3jqNHj3Lo0CF+//33REsBnDx5MtEWch999BE1atTgq6++4u7du3Tp0oVf\nfvkFDw+P+DZ6byFXhORLK2uAGgm+3gpUTKKdedeAFOI5Nx/dVF1Xd1Vu493Ul+u/VKdunzL6uVHR\nUWrd+XWq+ZLmKu/3edXkvZPVs8hnaYojpfc+wzHJLS0KFy6sXFxcVM6cOeNvP//8s1JKqXnz5qkS\nJUrEt33y5IkyGAzq1q1bSiml2rVrp0aOHKmUUur8+fPK1dVVPXv2TMXExKhXXnlFXbx4Mf65e/fu\nVUWLFlVKaUvYenp6KqWUCgwMVHnz5k0UU+vWrdXw4cOVUkp16NBBtW7dOv6xx48fK2dnZ3Xt2jW1\ndOlSVbt27UTP7dq1qxoxYsQLrzM4OFg5OTmpBw8evPCYn5+funLliurYsaO6c+eO6tq1q3r69Kmq\nW7eumj17dny7zZs3K4PBoKKjo5VSSpUsWVJt2rQp0bEePHigChUqpMqVK6f8/Pxe6Ktt27bxP7Pn\nJfc+wYhlbJPfAC91nl8HQBaWELqJjolmyv4pjNs9jo7lOxL0VRDuWd0Tt4mG9eth3ToICYGSJaF1\nayhfXnvc2cmZJiWb0KRkE479e4zBOwbz098/MafpHLyKeCXRa9qpYfr9dzEYDKxatSrJTZEB8ubN\nG38/W7ZsgLa7zquvvkqbNm3o06cPQ4YMYfHixbRo0YIsWbJw+/bt+O3h4iil4ksSCQUHB78wg6Vw\n4cLx27QZDAY8PT3jH3vllVdwd3cnODiYK1eu8Ndff+Hm5hb/eFRUVJKbUFy7dg13d3dy5MjxwmOz\nZv1XDfbw8GD27NkA3Lx584Xt4xJyc3N7YaekHDly0LJlS6ZMmcKKFSte6OvRo0eJ4jUVUyTyG0DC\n34Rn7PdeMHz48Pj73t7eeHt7m6B7If4TdC+Ijis7ksEpAwe6HKC4e/EX2pw/D+3bQ0wMtG0L9erB\nsWPQuDH4+sK0aZA163/ty+ctz5rWa1h5diXt/mxHs9eaMfmdyWTJkMWCr8z61K9fn5CQEI4dO8bS\npUuZOnUqALly5YrfHi6lE4H58+fn2rVr8WUNgCtXrlCqVClA+wC4du1afPvHjx9z7949ChQoQKFC\nhfDy8jJqt6KCBQty7949Hj58mGQyB5g3b16ir/Ply/fC9nEJJbWF3NGjR5k3bx5t2rThyy+/ZMOG\nDYkeP3PmDH379n1prAEBAQQEBKT0ktKkCMmXVpoA62PvVwOS2101yT8bhDCVNefWqNzf5VZT9k1R\n0THRSbY5cECpPHmUmj5dqQSb4CillHr4UKmPP1aqWjWlkvgLXCml1INnD9SHv32o3vrpLRV0N8io\nuKz5vV+kSBG1devWJB+bN2+eqlWrVqLvGQyGRCWT7t27q/r166s8efLElxyUUqpnz56qVatW6vbt\n20oppa5fvx5fhkhYWgkPD1fFihVT48ePVxEREWrHjh3K1dVVnTt3TimllVayZ8+udu/ercLDw1Wv\nXr3iYwoNDVWFCxdWCxcuVBERESoiIkIdOHBAnTlzJsnX4+vrq9q0aaPu37+vIiIi1M6dO1/6s5k1\na5YqU6aMun79urp3757y8fFRTk5O8a9zxYoVqmHDhvHtnz17psqWLat++uknFR4ersqVK6dmzpwZ\n//j169eVh4eHioiISLK/5N4nmKjCsQQIBiKAa0BnoFvsLc50IAht+uHbksiFJcXExKjRO0erApMK\nqL1X9ybb7tIlpfLmVWrlypcdSyk/P6V8fJQKD0++v2n7p6lXJ76qAi8HphifNb/347Z6c3Fxib+9\n//77Siml5s+f/0IN2snJKVEi37VrlzIYDOqLL75I1C4128OdOnVKeXl5qRw5cqiyZcuqlQl+QR07\ndlR+fn6qQYMGysXFRXl5eanLly/HP37u3Dnl6+urcufOrTw8PFS9evXUsWPHknyt9+7dUx06dFB5\n8uRRbm5u6oMPPnjpzyYqKkr17t1beXh4qGLFiqkZM2YkqpFHRESoQoUKqeDgYKWUUr169VJNmjSJ\nf/6xY8eUu7u7CgrSPvC/++471adPn2T7M3ciN5WX/tCESIvomGjVY20P9fbst9WN0BvJtnv6VKk3\n31Rq6tSUjxkVpVSTJkoNGPDydpuDNqvc3+VWv5387aXt5L2fdh07dlSDBw/WOwyl1H97gyb8y2PO\nnDmqV69eKT43LCxMlSpVSoWEhCTbJrn3CRY82SmExUVGR9JpVSeuhV5jR4cdZM+cPdm2I0bAa6+B\nMRctOjvD/PlQoQK88w7UrZt0uwbFG7C5/WZ8F/vyNPIpHSp0SNsLEclSVr4hx2effWZUu8yZMyc7\nLdIUJJELmxQdE03bFW15EvmEjW03kjVj1mTb/v03zJsHx4+DsTuC5c4Ns2eDnx+cOAGZMiXdrkLe\nCmz7ZBv1/lcPJ4MT7cu3T8OrEckxGAxWtY2bNcWSkGz1JmyOUorP1nzGlYdXWNt6LZkzZH5JW/D2\n1mapdOmS+r6aNgUvL+jX7+XtToecpv7/6jOp4SRal2ud6DHZ6k0YIz1bvUkiFzan7+a+7Lm2hy3t\nt+CSyeWlbdetg/79temFGdLw9+eFC1CtmjZlMcEFekk6efskPgt8WNpyKT5F/5uXLYlcGEP27BQO\nY8aBGWwI2sD6NutTTOLR0fDNNzBuXNqSOGgXCn3wAUyenHLbN159g2Utl/Hx7x9z8vbJtHUoRBpI\nIhc2Y8vFLYwKHMWa1mtwy5ry1XGrVkG2bNCsWfr6/fZb+OkniF376KXqFq3L1EZT8V3sy7+P/01f\nx0IYSUorwiacu3OOOvPr8FvL34y6RF4pqFED+vbVRtTp1aULFCoEQ4ca137YjmHsuLyDbZ9sI0/u\nPNy//7J154TQLvm/d+/eC9+XGrmwC4/CH1HZvzJ9a/Sly9vGnbHcswc6dIBz57TphOl18iQ0aABX\nriQ/gyWhGBVDsyXNKOlekqmNpqY/AOGwpEYubJ5SCr91ftQsWNPoJA4waRJ8/bVpkjjAG29A2bLw\n22/GtXcyOLGoxSLWnl/LouOLTBOEEMmQRC6s2s+Hf+b4reP82ORHo58THAwBAZDEInjp0rMn/PCD\nVrYxhltWN/786E96b+rN2TtnTRuMEAlIIhdW69i/x/h2+7cs/3A52TJmM/p58+bBhx+Cy8sntaSa\nry/cuQNHjhj/nHJ5yjG67mja/NGG8Khw0wYkRCxJ5MIqhUWF0WZFGyY3nEypXKWMfl5MDPzyCxh5\n5XSqODlBx47aB0VqdK3YlcI5CzNo+yDTByUEksiFlRq8fTBlc5el3ZvtUvW8bdsgRw6omNQeVSbQ\noQMsWQLhqRhcGwwGfm72M8tOLWPzxZTXzhYitSSRC6uz8/JOlpxcwkzfmale2+Lnn7WpguZaEqNI\nEW0XodWrU/c8j2wezH93Pp1XdeZB2AOzxCYcl0w/FFYlNDyU8j+VZ3rj6fi+5puq5z56BJ6e8M8/\nKV9Onx6LFsHixdpWcanVfW13IqIj+OXdX0wfmLBLMv1Q2Jy+m/tSr2i9VCdx0EbJtWqZN4kDvP8+\n7N0Lt26l/rnfNfiObZe2sSlok+kDEw5LErmwGoFXAll/YT2TGk5K0/OXLNE2UDa3bNm0GSxJ7K2b\nItfMrsxpNoeua7sSGh6a8hOEMIIkcmEVwqPC6bqmKz82/pEcWZLeHPdl7t2DXbvg3XfNEFwSWrWC\nZcvS9tyGxRvSsFhD+m/pb9qghMOSRC6swrjd4yiVqxQtSrdI0/P/+AMaNgRXVxMHlox33tGWxr15\nM23Pn9hwIqvPrWb/9eT2KhfCeJLIhe7OhJxh+oHpTG8yPc3HWLoUPv7YhEGlIEsWbVXFP/5I2/Nz\nZsnJxAYT8VvrR1RMlGmDEw5HErnQlVKKbmu7Mdx7OJ7ZPdN0jDt34NAhaNLExMGl4KOP0l5eAWhT\nrg0e2TyYcWCG6YISDkkSudDVryd+5UnkE7pX6p7mY6xdC/XrQ9bkt+00iwYNtFUR0zJ7BbRpZTOa\nzGD0rtEEPwo2bXDCoUgiF7p5FP6IAVsH8GPjH3F2SvsyhatXQ/PmJgzMSJkyaXX5tWvTfoxSuUrR\nrWI3em/qbbrAhMORRC50M2bXGOoVrUeNgjXSfIxnz7TL8n1TP+3cJJo3T/1Vns8bVHsQB24cYMel\nHaYJSjgcSeRCFxfuXuDnwz8zvv74dB1n2zaoUAFy5TJRYKnUpAns2AFPn6b9GFkzZuW7+t/Re1Nv\nomOiTReccBiSyIUuvt78Nf1r9ie/a/50HWfVKsvNHU+KmxtUqgRbt6bvOC3LtCR75uzMPTLXNIEJ\nhyKJXFjc+gvrOXfnHD2r9kzXcWJiYM0afRM5aOWVVavSdwyDwcCUd6YwNGAoD8MemiYw4TAkkQuL\nioyOpPem3kx5ZwqZM2RO17EOHNBKKsWLmyi4NGreXDvhGZ3OqkjF/BVpXKIxY3eNNU1gwmFIIhcW\n5X/Yn8I5CqdpUaznrV2rz2yV5xUrBrlzw8GD6T/WGJ8x/HLkFy7eu5j+gwmHYUwibwScBS4AA5J4\nPBewETgKnAQ6mio4YV8ehT9i5M6RTKg/wSTH27gRGjc2yaHSrXFj2GSCBQ3zueajT/U+9N3SN/0H\nEw4jpUTuDExHS+ZlgNZA6efafAEcASoA3sAkIINJoxR2YeLeiTQs3pC38r2V7mPdvg1BQVCtmgkC\nM4FGjbQPFlPoXb03h28eZvfV3aY5oLB7KSXyKkAQcBmIBJYCz59auglkj72fHbgLyOIRIpHgR8HM\nODiDUXVHmeR4mzeDjw9kzGiSw6VbrVpw6hTcvZv+Y2XJkIWR3iP5Zus3yGYswhgpJfICwLUEX1+P\n/V5C/kBZIBg4BqRvKoKwS8MDhvPpW59SOGdhkxxv40ZtFGwtMmeGOnXSPw0xTrs32/Ew/CFrzq8x\nzQGFXUupBGLMcOBbtPq4N1Ac2AKUBx4933D48OHx9729vfH29jYuSmHTToecZuXZlZz74pxJjhcT\no43Ix4wxyeFMplEjrU7+0UfpP5azkzPj642n/9b++Jb0TdcSBsK2BAQEEBAQkKrnpLRnZzVgOFqN\nHGAgEAMkPFu1HhgD7In9ehvaSdFDzx1L9ux0UM2XNMe7iDdfV//aJMc7dAg++QROnzbJ4UwmKEgb\nld+4YZrNn5VSeM33olOFTnR6q1P6Dyhskin27DwElASKAJmAj4DnV5Y4C9SPvZ8HeB34J3WhCnu1\n68ouTtw+weeVPzfZMTdu1DZ2sDYlSmjbwJ04YZrjGQwGJtSfwLCAYTyLfGaagwq7lFIij0KblbIJ\nOA0sA84A3WJvAGOBSmj18a1Af+CeOYIVtkUpxaDtgxjuNTzdF/8kZG318YRMOXsFoHrB6lTKX4kZ\nB2XNcpE8E/wBaDQprTiYzRc303NjT052P2myGu+DB1CwoDb90NLrjxtj7VqYPBm2bzfdMc+EnKHO\n/Dpc+PICObPkNN2BhU0wRWlFiDSJG42P8B5h0hN1O3ZAjRrWmcQBvL21pQOePDHdMUvnLk2z15ox\ned9k0x1U2BVJ5MIsVp9bTWR0JC3LtDTpcXfsgLp1TXpIk3Jxgbffht0mvpZnSJ0hzDw4k7tPTTBR\nXdgdSeTC5GJUDEN2DGFU3VE4GUz7FgsIsO5EDtqFSqYsrQAUdStKyzItmbh3omkPLOyCJHJhcstP\nLSdrxqw0fa2pSY8bEgJXrkDFiiY9rMnVq6dteGFqg2oPwv+wP7cep3GTUGG3JJELk4qKiWJowFBG\n1x0dd5LGZHbuhJo1IYOVr+RTtSqcPw/3TDx3q2COgrQt15YJe0yz6JiwH5LIhUktOr6IvC55qV+s\nfsqNU8kWyiqgbcpcs6YWr6kNrDWQBccWEPwo2PQHFzZLErkwmYjoCEbsHGGW0ThY/4nOhMxVXsnn\nmo9OFTrJ5hMiEUnkwmTmHpnL6x6vU7twbZMf+9Yt7dL3ChVMfmizMMcJzzgDag5gycklXH141Twd\nCJsjiVyYRHhUOGN3jWWE9wizHH/nTqhd2/rr43EqVNAuWrpxw/THzv1KbrpV7MbowNGmP7iwSZLI\nhUksOLaAMrnLUNWzqlmOb0tlFQAnJy1ec43K+9boy4ozK/jnvixrJCSRCxOIjI5k3O5xDPUaarY+\nduzQrpq0JeaqkwO4Z3WnR+UejNs1zjwdCJsiiVyk26LjiyjuVpwaBWuY5fg3b2plivLlzXJ4s/Hx\n0RK5uZYY6lWtFyvOruDKgyvm6UDYDEnkIl2iYqIYs2sMQ+oMMVsfAQHaOt/ONra3wmuvaUn8wgXz\nHN89qzvdKnZj/O7x5ulA2AxJ5CJdlpxYQoHsBfAq4mW2PmyxrALa5hI+PuaZTx7n6+pf89vp37j2\n8FrKjYXdkkQu0iw6JprRu0YztI75auNgOxcCJcXb27yJPFe2XHz61qd8t+c783UirJ4kcpFmy08v\nJ1e2XPgU9TFbHzduaDvTlytnti7MKi6Rm3Mp/j7V+/DriV/lak8HJolcpEmMimFU4CiG1hlqlqs4\n4+zYAV5e2nQ+W1S0qDb33Vx1coA8LnnoWKEjE/fIyoiOykb/ewi9rTizglcyvkLD4g3N2o8tl1VA\nq5N7eWkXNJlTvxr9WHBsAf8+/te8HQmrJIlcpFr8aNzLvKNxsL0LgZJi7jo5aGuwtHuzHd/v/d68\nHQmrJIlcpNrqc6txNjjjW9LXrP1cvQqhoVCmjFm7MTtL1MkB+tfsz9wjc7n95LZ5OxJWRxK5SBWl\nlMVG4wEBWhK01fp4nGLFtNcQFGTefjyze9L6jdayt6cDsvH/IsLS1l9YT2R0JM1fb272vuyhrAKW\nq5MDDKg1AP/D/tx5esf8nQmrIYlcGE0pxcjAkQypM8Tke3EmxVYvBEqKJerkAIVyFKJl6ZZM3T/V\n/J0JqyGJXBht88XNPI54zAdlPjB7X5cvQ1gYlC5t9q4swlJ1coCBtQcy69As7j+7b/7OhFWQRC6M\nEjcaH1x7sEVH42Yuw1tM8eLavxcvmr+vIjmL8N7r78mo3IFIIhdG2XF5B3ee3qFV2VaW6c+Oyiqg\nfSB5e1umTg7wbe1vmXFwBg/DHlqmQ6ErSeTCKCN3jmRQ7UE4O5l/CUKlbP9CoKR4eVmmTg5Q3L04\nTUo24ccDP1qmQ6ErSeQiRYFXArkeep025dpYpL9//oGoKG0ZWHtiyTo5wKDag/jhrx94FP7IMh0K\n3RiTyBsBZ4ELwIBk2ngDR4CTQIApAhPWY1TgKL6t/S0ZnCyzYaa91cfjlCgBMTHaB5UlvJ7rdRoU\na8CMgzMs06HQTUqJ3BmYjpbMywCtgefnEeQEZgDNgDeAliaOUeho77W9BN0Lov2b7S3Wpz2WVcDy\ndXLQRuUVrd0VAAAfAklEQVRT9k/hScQTy3UqLC6lRF4FCAIuA5HAUuDd59q0Af4Arsd+LVci2JFR\ngaP4puY3ZHTOaJH+lLKfC4GSYsk6OUDZV8viVdiLnw79ZLlOhcWllMgLAAm3Hrke+72ESgLuwA7g\nEGC5oZswqwM3DnDy9kk6VuhosT4vXNBGrnHT9eyNpevkAIPrDOb7fd/zNPKp5ToVFpVS0dOYt1tG\n4G2gHpAN2AfsR6upJzJ8+PD4+97e3njb0/wyOzQqcBQDag4gc4bMFuszrqxib/XxOCVLaidyL13S\n1mCxhDfzvEk1z2r4/+1Pz2o9LdOpSLOAgAACUvlnW0r/XaoBw9Fq5AADgRhgQoI2A4Csse0AfgY2\nAr8/dyylLDkMEely5OYRmi5pysWvLpIlQxaL9du6NTRoAJ07W6xLi2vTRnuNnTpZrs/DNw/TbEkz\ni/8+RfrFLk730lydUmnlEFrppAiQCfgIWP1cm1VALbQTo9mAqsDpVEcrrMqowFH0q9HPov/p4+rj\n9v6HmqXr5ABv53ubt/O9zdwjcy3bsbCIlBJ5FPAFsAktOS8DzgDdYm+gTU3cCBwH/gL8kURu047f\nOs6+6/voWrGrRfs9dw4yZ9a2R7NnetTJAYbUGcL43eMJjwq3bMfC7CxZiZTSio1otbwVVQpUoW+N\nvhbtd9Ys+OsvmD/fot1anFKQPz/s3Wv5D61Gixrxfun3Lf4hLdLOFKUV4WBO3T7Fzis78avkZ/G+\nHaGsAv/NJ9+xw/J9D/UaythdY4mMjrR858JsJJGLRMbsGkPvar1xyeRi0X7tdX2V5NSta/k6OUCN\ngjUo4V6ChccXWr5zYTaSyEW8c3fOsfWfrXxe+XOL9336NLi4QOHCFu9aF3XraiNyPaqNcaPyqJgo\ny3cuzEISuYg3dvdYvqzyJa6ZXS3et6OUVeLErbtiifXJn1encB08s3uy5MQSy3cuzEISuQAg6F4Q\n686v48uqX+rSvyOVVUCrk8eNyvUwpM4QRu8aTXRMtD4BCJOSRC4AGLdrHJ9X/pycWXJavO+YGC2R\nO9KIHPRN5D5FfciVLRe/nfpNnwCESUkiF1x+cJmV51bqdvn2yZOQMycULKhL97qJO+GpR53cYDAw\ntM5QRu8aTYyKsXwAwqQkkQvG7x6PX0U/3LO669L/jh3g46NL17oqWhQyZIDz5/Xpv2HxhrhkcmHF\nmRX6BCBMRhK5g7v28BrLTy+nd/XeusWwfbtj1cfj6F0njxuVjwocJaNyGyeJ3MFN2DOBT9/6lFzZ\ncunSf3Q0BAY6ZiIHfRM5QJOSTcjglIHV555fQknYEknkDuxG6A0Wn1hMn+p9dIvhyBHIlw/y5tUt\nBF3pte5KHIPBwJA6Qxi5cySyhIbtkkTuwCbsmUDHCh3J45JHtxgctT4ep0gRyJYNzpzRL4bmrzcn\nWkWz7sI6/YIQ6SKJ3EFdD73OouOLGFAzuf20LcNR6+MJ6V1ecTI4MaTOEEYFjpJRuY2SRO6gxu4a\nS5e3u+g6Go+MhD17HG/++PP0TuQA75d+n8cRj9l8cbO+gYg0kUTugK48uMKyU8voX7O/rnEcPKht\nd+bhoWsYuourk8foOHHEyeDE4NqDGbFzhIzKbZAkcgc0OnA0fhX9dJupEsfR6+NxChbULog6dUrf\nOFqVbcXD8IdsDNqobyAi1SSRO5iL9y7y59k/6VNDv5kqcaQ+/h9rKK84Ozkz0nskg3cMllG5jZFE\n7mBGBY7iiypf6HYVZ5ywMG03oDp1dA3DalhDIgdoUboFSilWnl2pdygiFSSRO5Dzd8+z7sI6elXr\npXco7N8PZcpAjhx6R2IdvL1h50596+Sg1cpH1R3FkB1DZGVEGyKJ3IGM2DmCXlV76bLC4fOkPp5Y\n/vyQOzccP653JNrVnq6ZXVl2apneoQgjSSJ3EKdDTrPl4ha+qvqV3qEAUh9PSt262s9FbwaDgTE+\nYxgWMEx2EbIRksgdxIidI+hbo68uu/8878kT7dL8WrX0jsS61K8PW7boHYXGp6gPBbMX5H/H/qd3\nKMIIksgdwNF/jxJ4JVCXvTiTsmcPVKgAr7yidyTWxccHdu+G8HC9I9GMqjuKkTtHEh5lJQGJZEki\ndwDfbvuWQbUH8Uom68icW7ZAgwZ6R2F93N2hdGnYu1fvSDQ1C9WkTO4y/HLkF71DESmQRG7ndl7e\nydk7Z+lasaveocSTRJ68Bg2sp7wC2qh8zK4xPIt8pnco4iUkkdsxpRTfbPuGUXVHkck5k97hAHDr\nFly5AlWq6B2JdbK2RF4xf0WqFqjKzIMz9Q5FvIQkcju2+txqnkY+pXW51nqHEm/rVm12RoYMekdi\nnapXh3Pn4O5dvSP5z6i6o5iwZwIPwh7oHYpIhiRyOxUdE823279lrM9YnAzW82vevFnKKi+TObM2\nm8capiHGKftqWZq91owJuyfoHYpIhjH/wxsBZ4ELwMsWr64MRAHvmyAukU4Ljy/EI6sHTUo20TuU\neEppZYOGDfWOxLpZW3kFYETdEcw5PIfrodf1DkUkIaVE7gxMR0vmZYDWQOlk2k0ANgIGUwYoUi8s\nKoxhAcMYV28cBoP1/DpOnYIsWaB4cb0jsW5xidya1q3yzO5J17e7MmzHML1DEUlIKZFXAYKAy0Ak\nsBR4N4l2XwK/AyGmDE6kzU+HfqJ8nvLULFRT71AS2bxZRuPGKFtWm0t+8aLekSQ2oNYA1pxfw8nb\nJ/UORTwnpUReALiW4Ovrsd97vs27wKzYr61oHOF4HoY9ZNzucYzxGaN3KC+QsopxDAbrusozTs4s\nORlYayDfbP1G71DEc1KaO2BMUp4KfBPb1sBLSivDhw+Pv+/t7Y23o+/xZQbjdo/Dt6Qv5fKU0zuU\nRMLCtKsWlyzROxLb0KAB/PkndO+udySJ9ajcg2kHprHz8k68injpHY5dCggIICAgIFXPSamAWg0Y\njlYjBxgIxKDVw+P8k+A4uYCnwGfA6ueOpWSxevO6/OAyFedU5ET3E+R3za93OIls3w6DBsG+fXpH\nYhtu3tSW+Q0Jsb6pmr8e/5VpB6ax/9P9VnUOxl7F/oxf+oNOqbRyCCgJFAEyAR/xYoIuBhSNvf0O\ndE+ijbCAgdsG0rNqT6tL4iDTDlMrXz4oVEjbfMPatC7XmsjoSH4//bveoYhYKSXyKOALYBNwGlgG\nnAG6xd6Eldh3bR+7r+6mT3X9t3BLyoYN8M47ekdhW3x9Yf16vaN4kZPBiYkNJjJg6wDCosL0Dkdg\n2amCUloxE6UUNebWwK+iHx0qdNA7nBdcv66tdnjrFjg76x2N7di9G774Ao4e1TuSpL279F2qe1bn\nm1py8tOcTFFaETZg+enlRERH0L58e71DSdL69dpoXJJ46lSrBteuwY0bekeStEkNJ/H93u+5+eim\n3qE4PEnkNi4sKowBWwcwqeEkq7oUP6F167QygUidDBm06ZobNugdSdJKuJeg81udGbR9kN6hODzr\n/J8vjPbD/h8on6c83kW89Q4lSWFh2v6cUh9PG19f7YPQWg2uM5gNQRs4FHxI71AcmiRyG3Y99DoT\n905kUsNJeoeSrMBAKFcOPDz0jsQ2vfOONnXTWnYNel72zNkZXXc0vTb2Qs6B6UcSuQ3rt6Uf3St1\np7i79S5eImWV9MmdW9s1aNcuvSNJXscKHXka+ZRlp5bpHYrDkkRuowIuB7Dv2j4G1h6odyjJUkpL\n5E2sZwFGm2St0xDjODs580OjH+i/pT9PI5/qHY5DkkRugyKjI/li/RdMfmcy2TJm0zucZJ0/r9XI\ny5fXOxLb1qSJddfJAWoXrk2tQrUYHTha71AckiRyGzTj4Azyu+anRakWeofyUnGjcbmKO33eegtC\nQ+HCBb0jeblJDSfhf9ifMyFn9A7F4UgitzH/Pv6XMbvGMK3xNKtf5+LPP+HdpBY9Fqni5ATNm8PK\nlXpH8nL5XPMxpM4QeqzvISc+LUwSuY3pv6U/nSt0plSuUnqH8lK3b8OJE1Cvnt6R2IcWLaw/kYO2\nOuLDsIcsPrFY71AciiRyG7L1n63svLKTIV5D9A4lRatXa1PnsmTROxL74OMDp0/Dv//qHcnLZXDK\nwCzfWfTb0k82a7YgSeQ24lnkM/zW+jHLdxYumVz0DidFf/6pjSKFaWTKBI0bw6pVekeSsqqeVWn+\nenMGbx+sdygOQxK5jRgVOIpK+StZ1WbKyQkN1eY9y7RD02rRQvuAtAXj6o3jjzN/cPDGQb1DcQiS\nyG3A8VvH+fnwz0xtNFXvUIyyYQPUqgXZs+sdiX1p3Bj27oWHD/WOJGVuWd34vsH3fLr6UyKiI/QO\nx+5JIrdy0THRdF3TlbH1xpLXJa/e4RhFyirm4eICXl7WP6c8TptybSiUoxDjdo3TOxS7J4ncys08\nOJPMGTLT+a3OeodilPBw2LhRmy4nTO+992ynvGIwGPip6U9MPzidk7dP6h2OXZNEbsWC7gUxYucI\n5jSdY7VL1D5v61Ztkaw8efSOxD41bw5btsBTG7kS3jO7J2N9xtJ5VWeiYqL0Dsdu2UZ2cEDRMdF0\nWtWJwXUG83qu1/UOx2hLlsDHH+sdhf3KnRuqVoW1a/WOxHhd3u6Ca2ZXpu63jXM8tkgSuZX64a8f\ncDI48VXVr/QOxWhPn2oJpmVLvSOxbx9/DEuX6h2F8QwGA/7N/Bm/ezzn7pzTOxy7JIncCp0JOcO4\n3eOY9+48mympgHYSrkoVKauYW4sWsG2bbcxeiVPMrRgjvEfQ/s/2REZH6h2O3bGdLOEgomKi6LCy\nAyO9R1LMrZje4aTK0qVSVrGEnDmhbl3buGQ/oR6Ve+CRzYNRgaP0DsXuSCK3MuN3jydHlhz4VfLT\nO5RUefhQO9Ep0w4to3Vr7XyELTEYDMxtPpc5f89h37V9eodjVySRW5G91/by44EfmffuPKtf2fB5\nK1eCtze4uekdiWNo2hT27YOQEL0jSZ18rvmY5TuL9n+251H4I73DsRuSyK3Eg7AHtF3RljlN5+CZ\n3VPvcFJNZqtY1iuvaEsg/Pab3pGkXovSLfAq7EXvTb31DsVuSCK3Akop/Nb60aREE94tZXsLeF+/\nDgcOyNrjltahAyxYoHcUaTO10VQCLgew/NRyvUOxC5LIrcC8o/M4FXKK7xt+r3coafK//8GHH0I2\n6911zi41aADBwXDSBi+adM3syrKWy/h8/ecE3QvSOxybJ4lcZ6dDTjNg6wCWfrCUrBmz6h1OqikF\n8+ZBp056R+J4nJ3hk0+0n78tqpi/IsO9h/Ph8g8JiwrTOxybJolcR6Hhoby/7H0mNphI2VfL6h1O\nmuzeDRkzalcbCsvr1AkWLYJIG52a3b1Sd173eJ1eG3vpHYpNMzaRNwLOAheAAUk83hY4BhwH9gBv\nmiQ6O6aUotOqTngX8aZjhY56h5NmcaNxG5tkYzdKloTXXrOdFRGfZzAYmNNsDtsvbZft4dLBmETu\nDExHS+ZlgNZA6efa/APUQUvgo4A5JozRLn2/93uuPbzGD41+0DuUNHv8WFuJr317vSNxbJ072255\nBSB75uws/3A5PTf25Oi/R/UOxyYZk8irAEHAZSASWAo8Pz9hHxB3wfBfgO3Nn7Og7Ze2M2nfJH5v\n9TuZM2TWO5w0W7IE6tSBvLaxTLrd+vBDbUem69f1jiTtyuctz/TG03lv6XuEPLGxyfFWwJhEXgC4\nluDr67HfS86nwPr0BGXP/rn/D21XtGXR+4solKOQ3uGkmVIwfTp8/rnekQgXF2jbFmbP1juS9Pno\njY9oU64NHy7/UNZjSSVjErlKxfHqAp1Juo7u8B6EPcB3sS9D6gyhfrH6eoeTLnv2QFgY1Lftl2E3\nPv8c/P21jT1s2Wif0bhmdqXnxp56h2JTMhjR5gZQMMHXBdFG5c97E/BHq6XfT+pAw4cPj7/v7e2N\nt7e3kWHavsjoSFotb0WDYg3oUbmH3uGkW9xo3EnmPVmFUqW0DT1+/10bndsqJ4MTi1osotov1Zhx\nYAafV3G8P/kCAgIICAhI1XOMmWuQATgH1AOCgQNoJzzPJGhTCNgOtAP2J3McpVRqBvf2QylFj3U9\nuPLwCqtbryaDkzGfn9YrOBjKloXLlyFHDr2jEXFWrYLx47U1WGzdP/f/odbcWszynWWTVzubUuy6\nSy/N1caMp6KAL4BNwGlgGVoS7xZ7AxgKuAGzgCNoyV7EmrRvEruv7WZpy6U2n8QBfvpJW1dFkrh1\nadoUbt7UlkuwdcXcirHq41V0WdOF/deTGxuKOJac/euQI/J5R+YxYucIdnfebZOLYT3v8WMoWhT2\n7tXmMAvr8sMP2gyW33/XOxLTWH9hPZ1XdSawUyCvebymdzi6MNWIXKTRyrMr+Xb7t2xuv9kukjjA\nzz9ry9VKErdOXbpAYCCcP693JKbRpGQTRvuMpvGvjQl+FKx3OFZLRuRmEnA5gFbLW7Gh7QYq5q+o\ndzgmEREBxYtrFwFVqqR3NCI5w4Zp5zH8/fWOxHTG7x7PgmML2NlxJ6++8qre4ViUjMh1su/aPlot\nb8WylsvsJokDLF6szY6QJG7dvvwS/vhDS+b24pta39CqTCvq/68+d5/e1TscqyMjchPbe20v7y19\nj/+1+B+NSjTSOxyTiYrSZqrMnAn16ukdjUhJr17a+jdTpugdiekopRiwdQDbL21n6ydbyZklp94h\nWYQxI3JJ5Ca05+oeWixrwcIWC3mnxDt6h2NSc+fCwoWwfbsskGUL/v1X++A9ehQKFky5va1QStFz\nY0/+uvEXG9puwD2ru94hmZ0kcgsKvBJIy99a2mUSDw/XVthbsgRq1NA7GmGsgQPh7l2YY2dL2Cml\n6L+lPxsvbmRL+y3kdbHvxX4kkVvIqrOr+GzNZyz5YAn1itlf3eHHH2HTJli7Vu9IRGrcu6d9AO/f\nDyVK6B2NaSmlGLNrDAuOLWBr+60UzllY75DMRhK5Bfx8+GeG7BjCmtZrqJTf/s4ChobC66/Dhg1Q\noYLe0YjUGj0aTpyAZcv0jsQ8pv01je/3fs/Gdhspk7uM3uGYhSRyM4obEfxy5Bc2tdtktxcr9O2r\njezmztU7EpEWT59C6dLavqpeXnpHYx4Ljy2kz+Y+LP5gsc0vRpcUSeRmEhYVRpfVXTh75yyrW68m\nv2t+vUMyizNntPXGT56EPHn0jkak1fLlMGoUHD4MGWx/hYgk7by8k1a/t2J03dF8VvEzvcMxKZlH\nbgY3H93Ea74XUTFRBHYKtNskrhT07AmDBkkSt3UtW0KuXNoaOfbKq4gXuzrt4ru939Fvcz+iY6L1\nDsmiZESeCgduHKDlby3pWrErg2oPivuktEtLl2r11SNHtM2VhW07eRJ8fLTfZ4GXbQtj4+4+vUur\n31sBsOSDJXZxFaiMyE1EKcXkfZNpurgp0xpPY3CdwXadxG/d0kbjc+dKErcXb7wBPXpA167aX1v2\nyiObB5vbbaa6Z3UqzqnInqt79A7JImREnoK7T+/ScVVHbj+5zdIPllLUrajeIZmVUvDBB9ql+GPH\n6h2NMKWICKhSRbvqs2NHvaMxv3Xn19F5dWf6VO9Dn+p9cHZy1jukNJGTnem0+eJmuqzuQquyrRhb\nbyyZnDPpHZLZLVqkbU7w99+Q2Xb3hRbJOHoUGjSAgwehSBG9ozG/yw8u88mfnwCw4L0FNjkQk0Se\nRqHhofTZ1IfN/2zGv5k/DYs31Dski4ibpbJ1K5Qvr3c0wlwmTdLmle/a5Rgf1tEx0UzZP4UJeyYw\nvt54Or/V2aZKo5LI02DDhQ34rfPjneLv8H3D78meObveIVnEkyfan919+kDnznpHI8wprnyWP7+2\n96qjOHHrBJ+s/ASPrB7M9J1pM9d+SCJPhUv3L9F7U29O3j7JTN+ZDjMKB4iOhlatIHt27QSnDQ1W\nRBo9fKgtRzxokGPUy+NExUTx418/MmbXGD6v/DkDaw8kS4Yseof1UjJrxQhPIp4wPGA4lf0rU6VA\nFU72OOlQSRygXz/t6s2ffpIk7ihy5IDVq2HAANi8We9oLCeDUwZ6V+/NkW5HOBlykjIzyrDs5DKs\neZBpDIcdkUdER+D/tz9jdo3Bq4gX39X/joI57Gi9TyNNnQqzZ2t7cLq56R2NsLTdu6FFC21RtLff\n1jsay9t+aTv9tvTD2eDMxAYT8SpifesYSGklCZHRkSw+sZjhO4dTKlcpxviM4e18DvgOBqZN0zYe\nCAiAwva7eJxIwYoV2hzz9esdM5nHqBiWnVzGt9u/paR7SQbVHkSdwnWs5oSoJPIEHkc85ufDPzN5\n32RKuJdguPdw6hSuo1s8epsyRVuedscOSeJC24fVzw/WrNFOejuiiOgIFh5byPg948nzSh4G1R5E\noxKNdE/oksjR5pHO+XsO/of9qVukLv1q9KNygcoWj8NaREdrM1M2btT+nJYkLuKsWwedOmkbUbz3\nnt7R6CcqJorlp5Yzbvc4olU0PSr14JPyn+Ca2VWXeBw2kUdGR7L2/Fpm/z2bQ8GHaPdmO76o8gUl\n3O1sdf1Uun8f2reHsDBtRTypiYvnHTqk1cz9/LQdhpwceDqEUoqAywHMODiD7Ze20/qN1nSt2JXy\neS17kYXDJvJOqzoRdC+Irm93pWWZlmTNmNUi/VqznTu1JP7++zBxoqyhIpJ344a2YqKrK8yfr803\nd3TXQ6/j/7c/v574lQOfHbDoXqEOm8jDo8LJnMEBLlkzwsOHMGyYdiXfL79AkyZ6RyRsQVQUjBkD\nM2fChAnwySeOPTqPo5SyeM3cYeeRSxKHyEjt4p7SpbWrNk+ckCQujJchgzYAWLcOZs2CmjVh3z69\no9Kf3ic+k2OXidyRhYVpCbxUKfj1V202gr+/trGAEKlVqZKWwLt2hdatoX59bbqqFcwkFgkYk8gb\nAWeBC8CAZNpMi338GPCWaUITxlJK2zigd2/w9ITffoN582DbNqhaVe/ohK1zctJms5w/D23aQLdu\nUK6cdjFZSIje0QlIOZE7A9PRknkZoDVQ+rk2TYASQEmgKzDLxDHahICAAIv29+yZdgJzwABt9N2k\nCbzyirY86caN2iqGpmTp12dJ9vzawHSvL1MmbUG1s2dhxgxthkuJEtqmzlOmaIOJmBiTdJUq9v77\nM0ZKibwKEARcBiKBpcC7z7VpDiyIvf8XkBNwuF0ezflmevJE+0+zYIE2B7x6da1U0r+/Nvtk8WK4\nckXbmq2omZZbtuf/LPb82sD0r89g0JL3okXablL9+8Pp09rc81y5oFkzGDlSu2L0/Hnt2gVzsvff\nnzFS2lO7AHAtwdfXgef/WE+qjSdwK93R2SGltJ1awsK0GSUPHmi3+/e1f+/e1aZ/Xb8O165p/4aE\nQMmSULastmXX2LHa1XevvKL3qxGOLksW8PXVbgA3b2rrnB89qk1dPHlS+56nJxQqBAULardXXwV3\n98Q3V1fImlW7ZcwoC7ilRkqJ3NhTGs//yHU9FTJx4n8nZBLe4MXvmapNcDCsWvXf11FREB6u3cLC\n/rsfEaG9SbNk0Vagy5lTuzAnZ07t5u6ubY5bqZL2hi9QQPs3Q0q/KSGsQL582pLIrVr9972nT7VB\nydWr//177py24ub9+9q/d+/Co0dayfDZM61EE5fUs2bVyjrOzknfgoO1zVDivk44TTLhh0Hc/aS+\nl5rHlyzRPnSsSUqfedWA4Wg1coCBQAwwIUGbn4AAtLILaCdGvXhxRB4EFE97qEII4ZAuop2HTLMM\nsQcpAmQCjpL0yc71sferAfvT06EQQgjTawycQxtRD4z9XrfYW5zpsY8fAxxwIUwhhBBCCCFsxJfA\nGeAkievs9qQP2nkEy62qYxkT0X53x4AVQA59wzEZYy54s1UFgR3AKbT/c1/pG45ZOANHgDV6B2IG\nOYHf0f7fnUYrXeuuLrAFiFt3L7eOsZhLQWAjcAn7S+QN+O+6g/GxN1vnjFYSLIL2vkzqHJAtywtU\niL3vglYitafXB/A18CuwWu9AzGAB0Dn2fgasZPD0G+CjdxBmthx4E/tM5Am1ABbpHYQJVEf74I3z\nTezNXq0E6ukdhAl5AlvRBon2NiLPAfxjbGNLLppVEqiDNqslAKhkwb4t4V20i6GO6x2IBXTmv5lK\ntiypi9kK6BSLuRVBWwfpL53jMKUpQD+0Uqa9KQqEAPOAw4A/kC25xqa+zGQL2p9zzxsU25cbWp2n\nMtoIvZiJ+ze3l72+gUDDBN+zxevSknt93/LfiGcQEAEstlRQZuQoa/i5oNVaewKPdY7FVJoCt9Hq\n4976hmIWGdBmAH4BHASmov21OFTPoAA2oF0oFCcI8NApFlN7A+0CqEuxt0i09Wle1TEmc+gI7AGy\n6ByHqVQjcWllIPZ3wjMjsAnopXcgJjYW7a+pS8BN4AnwP10jMq28aK8tTi1grU6xJNINGBF7/zXg\nqo6xmJs91sgboc1+sKeVzY254M2WGdCS2xS9AzEzL+yvRg4QiJYrQbvC3ipm+mUEFgIngL+xzz+H\n4vyD/SXyC8AVtD9ljwAz9Q3HZJK64M1e1EKrHx/lv99bo5c+wzZ5YZ+zVsqjlVXsbcqvEEIIIYQQ\nQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEI7t/1yaq9kDdVX4AAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def p(x):\n", " \"\"\"\n", " Unnormalized density of the standard Gaussian\n", " \"\"\"\n", " return np.exp(-0.5*x**2)\n", "\n", "q = norm(loc=0, scale=2)\n", "\n", "x = np.arange(-5,5,0.01)\n", "plt.plot(x, p(x), 'b-', x, 5.1*q.pdf(x), 'g-')\n", "plt.legend(['Density p(x)', 'Envelope c*q(x)']);" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class RejectionSampling (Sampling):\n", " def __init__ (self, p, q, c):\n", " \"\"\"\n", " q is assumed to support sampling q.rvs and density evaluation q.pdf\n", " \"\"\"\n", " self.p = p\n", " self.q = q\n", " self.c = c\n", " self.tries = 0\n", " self.samples = 0\n", "\n", " def sample (self):\n", " while True:\n", " self.tries += 1\n", " x = self.q.rvs()\n", " u = np.random.uniform(low=0, high=self.c*self.q.pdf(x))\n", " if u < self.p(x):\n", " self.samples += 1\n", " return x\n", " \n", " def __str__ (self):\n", " return \"Rejection sampling: %d tries for %d samples\" % (self.tries, self.samples)\n", " " ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEACAYAAACuzv3DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYFNXVx/Fvw4CogEI0giyCLHFHRIEkKq2goq8RUaIQ\nF4yACCIqJBqXyGjcMKKJYZEdRREUVNQwIAItgis7yD7sggYBYdhhpt8/7owMw/R0dXdN36ru3+d5\n6qGnuzh9GKbOnL63qi6IiIiIiIiIiIiIiIiIiIiIiIiIiLiuNbAcWAU8UszrQWAnMD9/eyJpmYmI\nSFRlgdVAHaAcsAA4u8g+QeDDpGYlIiK/KBPl9aaYQr4OOASMBdoUs1/A3bRERMSpaIW8BrCx0Neb\n8p8rLAz8DlgITALOcS07ERGJKiPK62EHMeYBtYC9wLXAB0DDBPMSERGHohXy7zFFukAtTFdeWE6h\nx1nAQKAqsL3wTvXq1QtnZ2fHmaaISNrKBuqXtEO0oZU5QAPMZGd54FaOndg8jSNj5E3zH28vsg/Z\n2dmEw2HPb3369LGeQ6rkaTvHSpWqRD1CnKpUqUrafz+Vp50NqBft5zNaR34Y6AFMwZzBMhxYBnTN\nf30w0A7olr/vXqB9XEeKiMtycnbgbHQwM38rKZbm88W7ohVyMMMlWUWeG1zo8YD8TURELIg2tJJ2\ngsGg7RQc8UOefsjRCNpOwBG/fD+VZ/Il8/NiOH+8RyQpAoEAzoZWHEVDP79ig/k5LrlWqyMXEfE5\nFXIREZ9TIRcR8TkVchERn1MhFxHxORVyERGfUyEXEfE5FXIREZ9TIRcR8TkVchERn1MhFxHxORVy\nERGfUyEXEfE5FXIREZ9TIRcR8TkVchERn1MhFxHxORVyERGfUyEXEfE5FXIREZ9TIRcR8TkVchER\nn1MhFxHxORVyERGfUyEXEfE5FXIREZ9TIRcR8TkVchERn1MhFxHxORVyERGfUyEXEfE5FXIREZ9T\nIRcR8TkVchERn3NSyFsDy4FVwCMl7HcJcBi4yYW8RETEoWiFvCzQH1PMzwE6AGdH2K8vMBkIuJmg\niIiULFohbwqsBtYBh4CxQJti9rsfGA9sdTM5ERGJLlohrwFsLPT1pvzniu7TBhiU/3XYndRERMSJ\njCivOynK/wL+lr9vgBKGVjIzM395HAwGCQaDDsKLuKsq22jDRJryDQ1YRQaHyaES33Eu07mS6VzJ\nYcrZTlPSVCgUIhQKxfR3oo1nNwcyMWPkAI8CeZjx8AJrCsU5BdgLdAE+LBIrHA6rWZfkCQQCFO5F\n6rOKv/MP2jCRqVxFiCAracgBjqMKOzifxVzPx9RhHS/TiwHcxx4qFkRDP79ig/k5LrlWRyvkGcAK\noCWwGfgGM+G5LML+I4GPgPeKeU2FXJKqoJCX4yCP8Rw96M+/eYBX6ckuTor4985mKX14it/xBV0Y\nyhRao0Iutjgp5NGGVg4DPYApmDNThmOKeNf81wcnlqJI6arOZj7gRrZyKo1YyOZjpniOtYxzaM84\nrmQaw+nEeNqVeN6tiG3JPFVQHbkkVaNAgI+pyUC68wJ/I54f9yps513+yD6mc/2ePXDCCe4nKlIC\nJx25ruyU1DRvHlOA3vTjBR4l3p5lB1VpzWS2A/zf/8GePS4mKeIOdeSSepYuhSuuoO3//scHLp0N\nW4YAuXfdBRs2QFYWlC/vSlyRaNSRS/r58UfTOf/zn3zgYtg8gGHDoGJF6NoV1JSIh6iQS+rYvx/a\ntIE77zSb28qWhTFjYNEieO459+OLxElDK5I6uneHrVvhnXcgEDjmPPLEFDr9cPNmuOQSeOMNaNnS\npfgixdPQiqSPd9+FKVPM8EeglPuT0083RfzOO81Qjohl6sjF/9atg6ZNYdIkuPjiX54utY68wJNP\nwtdfw+TJpf/LQ9KWOnJJfeEwdOkCvXodVcST4skn4eefYejQ5L6vSBHqyMXfRo6E/v1NZ5xx9IXK\npd6RA3z3HQSDMHcu1K7t0nuJHOHGvVbcpEIu7tqyBRo1gqlTzZ9FJKWQAzz7LMyaZYZ2NMQiLlMh\nF9+pXLkqOTk7HO37LubGP0+WuFcSCvmhQ3DRRdCnD7Rr59L7iRgq5OI7TrvoK5nGMDpzNss4QIVI\n0RzFcphZyXc/DIWgY0dYtkz3YxFXabJTUlJZDvMvHqQ3/Uoo4kkWDELz5tC3b9RdRdymjlw8xUlH\n3o2BtGM8LZlGyT/CSezIATZuhAsvhDlzoG5dl95X0p2GVsR3ohXyKmxnOWfRik9ZzAXRopUYK8bM\nnC0s8Y9/wJIlMG6cS+8r6U6FXHwnWiF/gUeowg66MsRJtBJjxZiZs0K+Zw80aAAffQRNmrj03pLO\nVMjFd0oq5NXYwhLOoxEL+Z6aTqJFjBVHZs6Xehs0CD74wNwyQCRBmuyUlPIYz/E6HR0WcYs6d4bs\nbJg+3XYmkibUkYunROrIz2Adc2nCWSznJ051Gq3YWHFmFtviy2PHwssvmytOdZGQJEAduaSMJ3ma\ngXSPoYhbdsst5v7okybZzkTSgDpy8ZTiOvIGrGQ2v6cBq9jJybFEOyZWApnF1pGDuS/6yy/Dl1+q\nK5e4qSOXlPAIffkP98dYxD3g5pth50749FPbmUiKU0cunlK0I6/FBubTmAasYgdVY42G1Y4c4M03\nYcgQmDnTpTwk3agjF9/rTT+G0ymOIu4R7dubpeE++8x2JpLC1JGLpxTuyE/lfyznLM7lO36gejzR\nsN6RA4wYYRZt1hCLxEEdufjag/yLsbSPs4h7yB13wMqVZvEJkVKgjlw8paAjr8xOsqnHxcxhPXXi\njYYnOnKAl16CefNMZy4SA12iL75TUMh70Y8mzOU2Eil8HirkO3fCmWfC/PlaEk5iokIuvhMIBCjL\nIbKpx81MYC6JLKjsoUIO0Lu3+bNfv8TTkbShMXLxpZt4jw3UTrCIe9ADD8CoUaY7F3GRCrl4zkO8\nwsv0sp2G+2rXhmuugWHDbGciKUZDK+Ipvw0EeJMzachK8iibYDSPDa2AOXOlbVtzd8Ry5RKPJylP\nQyviOw8B/+YBF4q4RzVpAvXqwYQJtjORFKKOXLxj3Tq21a1LHXaxm0ouBPRgRw7w/vtmwnPWLHfi\nSUpTRy7+MmAAo8ClIu5hf/gDbNhgTkUUcYGTQt4aWA6sAh4p5vU2wEJgPjAXuNK17CR97NsHo0Yx\n0HYeyZCRAd26wYABtjORFBFtaKUssAJoBXwPfAt0AJYV2udEYE/+4/OB94H6xcTS0IpENmoUvPMO\ngaws3BwO8eTQCsDWrdCwoZn0rOrTG4JJUrgxtNIUWA2sAw4BYzEdeGF7Cj2uCPwUS5IiAAwcCN27\n284ieU49FW64wdxQSyRB0Qp5DWBjoa835T9X1I2YLj0L6OlOapI2vv3WdKjXXms7k+Tq0cP8AsvN\ntZ2J+Fy0Qu70s+QHwNnAH4DRCWUk6WfgQLj3XiiboqccRnLJJaYzz8qynYn4XEaU178HahX6uham\nK4/k8/yYvwK2FX0xMzPzl8fBYJBgMOgwTUlZ27bBBx/AqlW2M7GjRw/o3x+uv952JuIRoVCIUCgU\n09+JNtmZgZnsbAlsBr7h2MnOesAaTPd+EfBu/nNFabJTjvXSS7BoEbzxBlD84svx8/BkZ4H9++GM\nM2D2bKhf3DkCku7cmOw8DPQApgBLgXGYIt41fwO4GViMOf3w30D7uDOW9JKXB4MGwX332c7EngoV\noGNHGDrUdibiY7qyU+yZPBkefxzmzAHTdaRfRw5m9aDLLoONG6F8+dJ5D/EtXdkp3jZokLkwJpDM\nfsKDGjaEc881cwUicVBHLnZ8/z2cd57pQitW/OXptOzIAcaNM8MrWqBZilBHLt71+utwyy1HFfG0\nduONsHgxrF5tOxPxIRVySb68PLO4QpcutjPxjuOO06SnxE2FXJJvxgyoXNncm1uO6NzZ3HPm4EHb\nmYjPqJBL8g0darrxdJ/kLEqTnhInTXZKcv30k7nwZe1aqFLlmJfTdrKzwLhxMGQITJtW+u8lvqDJ\nTvGe0aPNwgrFFHHBTHouWZK+tyyQuKiQS/KEw5rkjEaTnhIHFXJJnq++gsOHzVWMElnBpOeBA7Yz\nEZ9QIZfkGTrUFClNcpasYUNzsdTEibYzEZ/QZKckx65d5i5/y5fDaadF3C3tJzsLvP22WT1o6tTk\nvad4kiY7xTvefhtatiyxiEshbdvCggWwZo3tTMQHVMglOYYNM8Mq4kyFCnD77TB8uO1MxAc0tCIJ\nqVy5Kjk5O0rcpxEwETgTyHMU1YtDK+Uwt+d3R6VKVdi1a3vJOy1dCq1awYYNkBFtMS9JVRpakVJn\nini4xK0z9zGCTPKi7Ode0S0Nh4meu/Mt2i8/AM45B+rWhf/+1+V/i6QadeSSkGiTkxXYxyZq0pj5\nbKS2k4glxosxO4/GMvEcHQ+jRsH48fDxxy6+t/iJOnKxrh3j+ZpmDou4HOOPf4QvvoBNJa15LulO\nhVxKVWeGMQxNcsbtxBOhfXtzKqJIBBpakYSUNLTSkBV8RgtqsZHDlHMaMWK8OLLzaCwTz/HxMH++\nuQfLmjVQtqyLOYgfaGhFrOrEcF6nYwxFXIrVuDGceqouDpKI1JFLQiJ15OU4yEZqcSmzWE2DWCIW\nGy/O7Dway8SL6XgYPBg++QQmTHAxB/EDdeRizR/4iGWcHWMRl4g6dIDp0+HHH21nIh6kQi6lojPD\nGIpuV+uaypXhppvM6YgiRWhoRRJS3NBKbdYzlybUYiP7OT7WiMfESyA7j8Yy8WI+Hr76Cu64A1au\n1B0k04iGVsSKPzOSt+kQRxGXEjVrZu7BEgrZzkQ8RoVcXFWGXO5mhIZVSkMgAPfcY9b0FClEhVxc\ndTWf8APVWMwFtlNJTbffDllZZhFrkXwq5OIqXclZyqpUMYtXjx5tOxPxEE12SkIKT3b+mh9ZzlnU\nZgO7qRRvRLw5QemByc4CM2fCvffCd99p0jMNaLJTkqojr/MeNyVQxMWRyy6DvDxzMy0RVMjFNWEN\nqyRLIGBWWxo61HYm4hFadkRccTkzOUh5vqK57VTSQ8eO0KAB/PwznHyy7WzEMhVyccWRblxjts5k\nFIx9xm0cEKpShUGUAw65khU4XIZOPEWTnZKQQCDAyWxnLXWpz2q2cUqiEfHmBKX7k52JxmvFVF7k\nYS5iQcKxjpbARKy4TpOdkhS38RZZXOtCEZdYTKMlJ7GTJrYTEeucFvLWwHJgFfBIMa/fBiwEFgGz\nQVeDpJMuDNUkpwVhyjCcTrqGVhwNrZQFVgCtgO+Bb4EOwLJC+/wWWArsxBT9TDhm1ktDKyno4kCA\ncZxJA1YRduUDnleHQ7w3tAJQnc0soQa1yWEPFRNPC9DQire4NbTSFFgNrMPMqIwF2hTZ50tMEQf4\nGqgZQ57iY12A4XRyqYhLrLZwOp8DtzLOdipikZOjrwawsdDXm/Kfi6QTMCmRpMQndu/mj8Ao7rKd\nSVobihnekvTlpJDH8hnrCuBuih9Hl1Tzzjt8jukKxZ7JQE02cR6Lbaciljg5j/x7oFahr2thuvKi\nLsA0B62BHcUFyszM/OVxMBgkGAw6TFM8afBg9YEekAuM4G66MJQHeNV2OpKgUChEKMZ7zjuZ7MzA\nTHa2BDYD33DsZGdtYDpwO/BVhDia7Ewl8+ZB27aU3bCBPA9OAno7ltvxAtRmXQKrMh0bT8eqd7g1\n2XkY6AFMwZyZMg5TxLvmbwBPAlWAQcB8TLGXVPbaa3DPPeTZzkMA2MAZzOFibmaC7VTEAl3ZKbHb\nuRPq1IFlywhUr46XO1VvxnI7nol1ExPoyasE+SzheDpWvUNXdkrpePNNuOoqqFbNdiZSyIfcwG9Y\nQUNW2E5FkkyFXGITDpthlW7dbGciRRymHKO4i84Ms52KJJkKucRm9mw4dAh0xpEnDaMzd/IG5Tho\nOxVJIhVyic1rr5llxrTEmCdlU5/vOJc2TLSdiiSRJjvFua1boWFDyM6GqlWBo9fsdIdXJyj98+9s\nz9v8mZFcwydxx9Ox6h2a7BR3jRwJN974SxEXb3qftjRmPnVYazsVSRIVcnEmLw8GDzbDKuJpB6jA\naO6gK4NtpyJJokIuzkyZAiedBE2b2s5EHBhId+5mBBXYZzsVSQIVcnHm1VehZ09NcvpENvX5hqZ0\n4G3bqUgSaLJToluxAi6/HNavhwoVjnpJk51eiFd8rKv4hBd5mMbMJ7ZDXZOdXqLJTnFH//7Qpcsx\nRVy87VNaUYH9XMbntlORUqaOXEq2cyfUrQuLF0ONY9cTUUfuhXiRY3VnAEFC3MK7McXTseod6sgl\ncSNHwjXXFFvExfve4E6uZDo1j1rkS1KNCrlElpsL//mPmeQUX9pNJUZzB90YZDsVKUUq5BJZVpa5\n+Kd5c9uZSAIGcB+dGaZTEVOYCrlEplMOU8JqGvAtl+hUxBSmyU4p3tKl0LIlrFsHxx0XcTdNdnoh\nXvRYV/EJ/ejNBSwi+mGvyU4v0WSnFKty5aoEAoESt2HnnkufH34gUKFCifuJP0zlKvIowzVMcbB3\nRtSfD6db5cq6L08yqCNPQ9G66Gps4TvOpSEr2cYp0aKVGCuO7FyM59VYbsdzFut2RnMXo2jFNFfi\nOaPuPlHqyCUuPXmVMfzJQREXPxlLexqyksbMs52KuEwdeRoqqSOvSA5rqUtTvmEtZzqJFjFWnNm5\nGM+rsdyO5zxWb17iIuZxG2NciRedOvJEOenIM5KTivhFZ4YxjZYOi7j4zRDuYS11qc16NnCG7XTE\nJerI01CkjjyDQ2RTj5t4j7lc7DRasbESyM7FeF6N5Xa82GL15WHKcYhevOJKvJKpI0+Uk45chTwN\nRSrkt/EmdzOClkyPJVqxseLn1eKbOv/OGmxiERdQj2x+pkqp56bjPjGa7JQYhHmYF3mRh20nIqXs\ne2ryITdwHwNspyIuUSEXAK5jEmECTOEa26lIErzA3+jJq5zIbtupiAtUyAUI8yRP8wxPkNzRNrFl\nBWcxgyu4l9dspyIu0Bh5Gio6Rn4Vn/AKD3E+iwnH/Ls9dcaOkxfL7XjxxTqPxXzC1ZzJGvZzfKnl\npuM+MRojFwfC9OEpnuGJOIq4+NkSzucrmtOFobZTkQSpI09DhTvyK5jOILpxDkvJo2w80Ui1TrX0\nY7kdL/5YFzGXibShHtkcpODmaOrIvUQduUT1JE/zLI/HWcTF7+bRhEVcwF2Msp2KJEAdeRoq6Mgv\nYyYjuJuzWE5u3Bf5pmanWrqx3I6XWKzmfMk4bqUBq/K7cnXkXqKOXEoQ5hme4BmeSKCISyr4it+y\nkEZ0ZbDtVCRO6sjTUCAQoDWT6EdvzmdxgsMqqdupll4st+MlHusCFjKFa6jPavZQKeF4R6gjT5Q6\ncilWAHiOx3iCZzQ2LgAsohEzuIKevGo7FYmDOvI0dGsgQG8uoRlfk/iPQGp3qqUTy+147sRqwEq+\n4Hc0YBs/qyP3DLc68tbAcmAV8Egxr58FfAnsB3rHlqIk3aFDPAM8xnPoKk4pbBUN+YAb+avtRCRm\n0Qp5WaA/ppifA3QAzi6yzzbgfuAl17MT940cyQZgGq1sZyIe9DRP0hWozmbbqUgMohXypsBqYB1w\nCBgLtCmyz1ZgTv7r4mW7d8PTT/OY7TzEszZSm2HAszxuOxWJQbRCXgPYWOjrTfnPiR/17QstWvCN\n7TzE054FWjNZa3v6SLQTiF2dpcjMzPzlcTAYJBgMuhleSrJ+PQwcCAsWwJiS1muUdJcD9OEpXuEh\ngoTQXEpyhUIhQqFQTH8n2v9QcyATM0YO8CiQB/QtZt8+wG6gX4RYOmvFpg4d4De/gczMEhdfjl3q\nn83hfiy347mfWxkOM5/GPEUf3uPmhGLpuE+MG2etzAEaAHWA8sCtwIeR3i+29CRpvvgCZs2Cv+p8\nBHEmj7I8xCu8yMOU54DtdCSKaIX8MNADmAIsBcYBy4Cu+RtANcw4+kPAE8AGoGJpJCtxyM2Fnj3h\n+efhxBNtZyM+Mp2WLOE8evGy7VQkCl0QlOr694fx42HGDDAf0TS0Yj2W2/FKL7c6rOVbLuESvmUd\ndeOKpeM+MU6GVlTIU9nmzdCoEcycCWcfOf1fhdx2LLfjlW5uf+N5LmUW1/MxsZcMFfJE6V4r6a5X\nL7jnnqOKuEis+tGbuqylLe/bTkUiUEeeqj75BO69F5YsgRNOOOoldeS2Y7kdr/Rzu4yZvMVtnMNS\ndlMpplg67hOjoZV0tXs3XHABDBgA1157zMsq5LZjuR0vObkN5272cTw9GBBTLB33iVEhT1fdu8P+\n/TBiRLEvq5DbjuV2vOTkdhI/s5jz6cjrzOBKx7F03CfGSSHX0jCp5tNP4eOPYdEi25lIitnJydzD\nEIbTiQtYFOMQi5QmdeSpZNcuM6QyeDBcc03E3dSR247ldrzk5jaUzhwmg2685iiWjvvEaGgl3dx9\nN2RkwJAhJe6mQm47ltvxkptbJXaxmPPpxiCyuC5qLB33idHQSjoZMwZmz4a5c21nIikuh8rcwWjG\ncStNmMsWTredUtpTR54KVq2C3/0Opk6FCy+Murs6ctux3I5nJ7e/8zRBQlzF1BLWflVHnihdEJQO\nDhyA9u0hM9NRERdxy7M8ToAwj/Os7VTSnjpyv7vvPnMp/nvv/XIvlWjUkduO5XY8e7lVZzNzacLt\nvMl0WhYbS8d9YtSRp7qhQ2HaNBg1ynERF3HTFk7nNt5iDH+iLmtsp5O21JH71ezZ0LYtfP65WTAi\nBurIbcdyO5793HrwH+5hCL/lS/YcdRdrdeSJ0umHqWrjRmjWDIYNg+uinf51LBVy27HcjueF3MIM\nozNV2EE7xhP+5cO+CnmiNLSSin7+2RTvhx6Kq4iLlI4A3RnIr/kfL/Kw7WTSjgq5n+zfD23awJVX\nwl/+YjsbkaMc5Dhu4EOuJYteEZfuldKgC4L8IjcXbrsNqleHV17R5KZ40g6q0prJzOb3/EA1xthO\nKE2okPtBbi507myGVSZNgjL6ICXetYlaXEsW02jJXtvJpAlVBK/LzYVOnWDdOvjwQzjuONsZiUS1\nlHO5jkkMAnONg5QqnbXiZbm58Oc/w6ZN5ta0RVb6iZfOWrEdy+143s3tQgLMP+00s8jJzTe7EjPd\n6KZZfrZvH9x+O+zc6WoRF0mmBQCTJ5uVqnbsMEOE4joNrXjRtm3QqpUZRvnvf1XExd8uvBA++wye\nfx769AF9MnedCrnXrF4Nv/89XHopvPmmxsQlNTRsCF9+CVlZ5r75Bw7YziilqJB7yUcfmdvRPvgg\n9O2rs1Mktfz61zBjBuTkwGWXmSuUxRWqFF6Qmwt//7tZNHniRLj3XtsZiZSOE0+Ed9+FW26Bpk3N\nTd8kYTprxbbsbOjY0QyhjBkDp51W6m+ps1Zsx3I7nrdzi3jcz5hhJvQ7dIBnnoEKFVx6z9Sie614\nWThsFklu3hzatTOr+yShiIt4xhVXwMKFsH49XHwxzJtnOyPfUkduw6JFZkGIAwfMvcTPOSepb6+O\n3HYst+N5O7eox304DG+9Bb17m+78qafgpJNcen//021sLdm+fTvTihn7K5+Tw9kTJlB71iy+u/VW\n1rRs6WhCs2rVqrRsWdzqK/FRIbcdy+143s7N8XH/00/w2GPmuonnnoM77oCykdYCTR8q5JYMHjyY\nBx/sS7lyTQCoHD7EfQdXcs+hbN7PqMkz5c9lexmnpxWGycmZCBx2Ocv0KCLejOV2PG/nFvNx//XX\n5u6e27bB00/DTTel9RlcurLTknA4TCBwNRVynqYbg7iPAWRxLZfwMWsPnQmHYomWi/lvcvtAFfGo\nZs1g5kxzRegTT5ju/K9/NXNJ5crZzs6T0vfXXCn61caNvHYwxAp+w+ls5nJmchevs5Yzbacm4g+B\ngLmsf84cyMyEIUOgbl144QXYutV2dp6joRW3/PCDOX3w9dfJ2bSJl3bVY8DhSWzjlAQDl1ZHnh4f\n670Zy+143s7NteN+wQL497/h/fehRQu48064/vqUv/pZY+SlbcUKc2vZjz4yZ6K0bQsdO/La0qX0\n+ssi9u17zYU3USFPvVhux/N2bq4f9zk5MGECvPEGzJ8PV18NN9xgOviqVd19Lw9w6zzy1sByYBXw\nSIR9Xs1/fSHQ2HmKPrN+PYweDV26QIMGZsm1NWvg0UdNRz5yJASDaT0xI1LqKlWCu+6C6dNh2TJT\nyN95B+rUMfcpevRRM76+a5ftTJMm2mRnWaA/0Ar4HvgW+BBYVmif64D6QAOgGTAIaO56pkkSCoUI\ntmgBGzaYj3IF29y55rzvFi3g8svh/vvh/PMtLrkWAoKW3tupEN7PEZSn20IkLc9q1czCK506wd69\n5sZcM2eaexV9+y3Urw+NG5s7MF54ITRqBCefbLIMhQgGk5RnKYtWyJsCq4F1+V+PBdpwdCG/AXg9\n//HXwMnAacCPrmXptnDYnNq0ZYvZ1q0zl8pnZxOaNYvgnj1QufKR//w//QlefNH8UHhmrcwQ3j+o\nQ3g/R1CebgthJc8TToCWLc0GpvFauNBsCxaYrn3RInO/l/r1CeXkEGzXDurVg1q1zHq41apBxYrJ\nzz1B0Qp5DaDwLco2YbruaPvUpDQKeW6uWUl+3z6zFTwu/FxOjlnbsmDbufPI461bTeH+8Ufzn1m9\nutnOOMMU6VtugV/9yhRtXVkm4m/HHWduzNW06ZHn8vJMDcjOhn79TO2YONGswlXQ2GVkHKkNVaua\nDv7kk6FKlSOPTzoJjj/e/PI4/vijt4LnkniqZLRC7nSWomibWvzfa9bMFOOC7fDho78u6bmCrUIF\n800q+LPw4woVTCdd+JtdrdqRr0855chv3Ug36Fm6NOEiXqZMGcLhLCpX/kNCcYxwOg31iZSuMmWg\nRg2zTZ9uTm0sLBw2zd+WLWbea/v2oxvDVauONIh79x5pIItue/eaWBkZsW1lyhy9NWjg6J8VbZyg\nOZCJmfAEeBTIA/oW2uc1zGepsflfLwdacGxHvhqo5ygrEREpkI2Zh4xbRn6QOkB5zBJ8ZxfZ5zpg\nUv7j5sA99JLSAAAC3ElEQVRXibyhiIi471pgBaajfjT/ua75W4H++a8vBC5KanYiIiIiIhKb3phx\ndq9egvUPzCeLBcA0oJbddCL6J+Y00IXAe4BXT7P5I/Ad5hJVL35ac3LBm20jMHNOi20nEkUtYAbm\n/3sJ0NNuOhFVwJwqvQBYCjxvN50SlQXmAx/ZTqSwWsBkYC3eLeSVCj2+HxhmK5EoruLIlbkv5G9e\ndBbQEHOAe62Ql8UMCdYBylH8HJAXXIa5YtrrhbwacGH+44qYIVkvfj8BTsj/MwMzr3epxVxK0gt4\nC3MhZkTJvpb8ZeDhJL9nrHIKPa4I/GQrkSimYj7ZgOkualrMpSTLgZW2k4ig8AVvhzhywZvXfA7s\nsJ2EAz9gfhkC7MZ8YjzdXjol2pv/Z3nML/TtFnOJpCbmZJJheGjNzjaYi4UWJfE94/UssAHoiHc7\n3cLu5siZQ+JccRez1bCUS6qpg/kU8bXlPCIpg/ml8yPm0+JSu+kU6xXgrxxp2CJye2GJqZiPV0U9\njjnj5epCz9m81j1Sno9hxqIez9/+hvlm/jl5qR0lWp5g8jwIjElWUsVwkqcXpdjtOD2jIjAeeADT\nmXtRHmYY6CRgCuaeAiGL+RR1PfA/zPh40G4qR5yH+c23Nn87hPk4+2uLOTlRGzNp41V3AbMxkzde\n58Ux8uaYOZsCj+LdCc86eH+MHMxcwxTgQduJxODvwF9sJ1HEc5hPi2uBLcAe4A2rGRXDy5Odha+J\nvR8YbSuRKFpjzg5IdOWKZJkBNLGdRBFOLnjzijp4v5AHMMXmFduJRHEK5uZ+AMcDMwH3Vjd3Xws8\n+sl2Dd4t5OMxB8wCYALe/dSwCliP+eg1HxhoN52I2mI6i32YybAsu+kco7gL3rzmbWAzcADzvbQ1\n1BfNpZghiwUc+blsXeLfsON8YB4mz0WYcWgva0GUs1ZERERERERERERERERERERERERERERERERE\nxEf+H3Xc1MFJG4agAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Rejection sampling: 2046 tries for 1000 samples\n" ] } ], "source": [ "sampling = RejectionSampling(p=p, q=norm(loc=0, scale=2), c=5.1)\n", "\n", "show_sampling(sampling, plotter=gauss_hist)\n", "print sampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Rejection sampling tends to work well, if a distribution $q(x)$ which closely approximates $p(x)$ from above can be found. Otherwise, the area between the curves will be large leading to a high rejection rate. Especially in high dimensions this is a severe problem." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A standard example to illustrate the difficulties arising in high dimensions is a 2-dimensional Gaussian with high correlation:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/usr/lib/pymodules/python2.7/matplotlib/collections.py:608: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n", " if self._edgecolors_original != 'face':\n", "/usr/lib/pymodules/python2.7/matplotlib/collections.py:548: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n", " if self._edgecolors == 'face':\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD7CAYAAAChScXIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XVUVdn7x/H3pUNCsBV7VOzuwE5ExW5AEbCwBh0Lu3uw\nayxMRFEUFUVBDLA7MDBQEKWbe35/MN+ZceI3YSD6vNa6a6Hcs+/ZR/msh3322RuEEEIIIYQQQggh\nhBBCCCGEEEIIIYQQQgiRA6g+9QdUqVJFuXbt2qf+GCGE+NqcBqx+/5can/pTr127hqIoX+1r6tSp\n2X4O0kfpn/Qv+8/jY7+AJn+WqZ88tIUQQnw8EtpCCJGDSGh/ICsrq+w+hU/ua++j9C9n+9r793uf\n/EYkoPw8PiOEEOIfUqlU8CcZLZW2EELkIBLaQgiRg0hoCyFEDiKhLYQQOYiEthBC5CAS2kIIkYNI\naAshRA4ioS2EEDmIhLYQQuQgEtpCCJGDSGgLIUQOIqEthBA5iIS2EELkIBLaQgiRg0hoCyFEDiKh\nLYQQOYiEthBC5CAS2kIIkYNIaAshRA4ioS2EEDmIhLYQQuQgEtpCCJGDSGgLIUQOIqEthBA5iIS2\nEELkIB8a2hbAKeAWcBMY8cFnJIQQ4i+pPvD4Aj+/rgK5gEtAJ+DOb96jKIrygR8jhBDfFpVKBX+S\n0R9aab8iK7ABEsgK60If2KYQQoi/oPUR2yoOVAMufMQ2hRDiPYqicPt2FK9eJRAdnYypqR6tWpXK\n7tP6bD5WaOcC9gIjyaq43+Pu7v7L11ZWVlhZWX2kjxVCfCvu349m27brbN9+A0VRKF7cFF1dLY4e\nfYhaPeV/wwk5VkBAAAEBAX/7vo/RS23gEHAEWPon35cxbSHEf5KamoGPz31Wrw7lxo1IeveuSN++\nlalevSAqlYrz558zdKgvly45ZvepfnR/Nab9oZW2CtgA3ObPA1sIIf61yMhEVq0KYdWqUCwt8+Lo\nWJ0uXSzR1f01stLTM5kwwZ9Oncpm45l+fh9aaTcEzgDXgf+V0xOAo795j1TaQoh/JDw8lnnzgtix\n4ybdupXH1bUu5cvn/cP7FEVhxIgjhIW9w8enF5qaX98jJ5+q0g5CHtARQnygBw+imTUrEB+f+zg4\nVOPevWHky2f4l++fOzeIU6eeEBho91UG9v/nY84eEUKIf+Xhw7fMmRPIwYP3GTGiNg8fDid3bv3/\n95hNm66wdu1lzp61/9v3fo0ktIUQn93TpzFMm3YaH5/7ODvX5P79Yf8ogLduvcbEiSfx9+9PoUJG\nn+FMvzwS2kKIz+bly3hmzw7E0/MmLi41efBgOKamev/o2HXrLjF1agAnTvTH0vKP49zfCgltIcQn\nFx2dxLx5Z1m//jL29tW4c2fo/ztm/Xtz5waxdu0lAgIGUqaM+Sc80y+fhLYQ4pNJTExjxYqLLFp0\nji5dynHjhjOFCxv/4+MzM9W4uZ3A1/cBgYF2/+rYr5WEthDio8vMVLN+/WWmTTtNgwZFCQy0o1y5\nPP+qjdTUDPr08SIqKonAQDvMzQ0+0dnmLBLaQoiPRlEUfHzuM3HiSczM9Dl4sBc1a/77NeTevk3G\n1nY35ub6HDvW972Har51ciWEEB/F1auvGDnyKG/fJjNnTnPat//uP60H8vDhW9q1246NTVnmzm3x\nzc3D/jsS2kKIDxIeHsvEiSc5fjyMadOsGDSo+n8O2qCgcLp124O7exOGDKn5kc/06yChLYT4T5KS\n0lmw4CzLl1/8ZfqekZEuAIoCwVfgSCCE3ISrd6FDE1g2AXL9xaSRbduuM3q0H1u3dqZ169KfsSc5\ni4S2EOJfURSF7dtvMH78CerXt+DSJUeKFzcFICkZdhyGH7dDUgr0bAfDekP5UlCvN/RqBy3qv9+e\nWq3www/+7N59i1OnBlChQr5s6FXOIaEthPjHQkJeMHLkUdLT1ezZ04169SwAeP0GVu+ClTuhdiWY\nPxZa1AONn0dJ/M+BmQk0q/t+ewkJafTvv5/o6GRCQgbLDJF/QEJbCPG3Xr6MZ8IEf44fD2PmzGYM\nHFgVDQ0VD5/C/I2wxw+6tYbTP0G5ku8fe/8J9BsPq6b8GuKQtUiUjc1OGjSwYOfOrujoaH7WPuVU\nEtpCiL+UkpLBkiXnWLToHI6ONbh3bxhGRrrcfQTTVsKJc+DcEx4cgTy5/3j80xfQ0gFmjACb5r/+\nvb//I3r39mL6dCu54fgvSWgLIf5AURT27LmNm9sJqlYtwMWLgylZMjcPn8LQWXA0CEb1h7XTwOgv\nbiyGhUNzexhrBw62v7Y7d24Qy5dfZOdOW5o2LfH5OvWVkNAWQrzn2rWs+daxsals2mSDlVVxIqJg\n2EzY6Qsj+8GPk8A411+38ehZVmC7DcqqxCFr/NrB4SBPnsQQEjKYIkXkkfT/QmatCyEAiIpKxNHR\nh1atttG9ewVCQwdTtXpx3BZBBWvQ0YY7h2Cy8/8f2LceQOP+7wf2nTtR1K69DkNDbU6fHiiB/QEk\ntIX4xmVkqFm5MoQKFVZiYKDN3btDsbOvxbKtGpRtD9ExcOMALHaDvGb/f1vBV7Iq7Hmjfw1sL687\nNGmymbFj67Nxow16evIL/oeQqyfEN+zMmacMHepL3rwGnDjRn4oV8+N5GH5YCtUswX8jVPzun7W1\n5ygMnQlb5kCbRlkb77q5ncDL6w6HD/emVq3Cn7Yz3wgJbSG+QS9exDFhgj+nTj1hyZLW2NpacuqC\nito9sqblbZsHjWpChgLHEuB0ElxOhqOJcL441PnNdGpFgbnrsuZoH1sHVS2zpgj26LEXY2NdLl1y\nlPnXH5EMjwjxDcnIULN48TmqVFlNkSLG3L7tQqUa5ek0TMWgKTDOHs57Qv5KMOYVFH0AkyKzqrt2\nP+/uVV731/ZS02DABPA6kXVcVUs4ffoJtWqto1Wrkvj49JLA/sik0hbiGxEQ8IQRI46QP38ugoMd\nMM9nzqQVWY+duznA7iVwLRN6vICAJBhsCieLQTndrGq69wsYZw5GPz8D8/oNdHXNGuc+/RPo6ynM\nnx/M4sXn2Lq1My1blsreDn+lJLSF+MpFRiYyduwxAgKyhkKsrS3x8FQxey10bwO3feCeHnSIgPtp\nMNoMNhWGXL/5PXxBNDxIg40/L4199Q50HAr2XWCKCyQkpGLbx5uXL+MJCRmMhYVJ9nT2GyDDI0J8\npTIz1Xh4XKRChZXky2fIrVtD0TUvT+XOKo4GQcBm6DwKbONgwAvoaQwPSsNI8/cDe9VbWPEWvC1A\nXwN2H4GWg2DhOHAfBtevv6JGjbXkz2/I6dMDJbA/Mam0hfgKXbr0EienwxgYaBMQMABtw3x0GwNP\nXsCi70GvBjhFQdQrmJwXehiD1p/sV7AoGjzewuniUFADxi+GXUeybjhWKw+bN19l3LjjrFjRlp49\nK37ubn6TJLSF+IrExqYwceJJ9u69zdy5LbDpXIUZq1VsOQATBkOjrjD5LYRFgHte6GUCmn8S1ooC\nEyNhX3xWYBsmQofvISUNQnaDvk4aAwb4cvHiCwICZDnVz0mGR4T4CiiKwq5dNylffiVpaZncvDkU\nTKpS3lpFQhIc2w9XW4FNBHQ0gtuloa/pnwd2pgKOEeCfCGeLw7tHUKsHWJbKqrCjI99Qp856VCoI\nDR0sgf2ZSaUtRA4XFvYWFxdfIiLi2b27K1q5itJuKKiA7R7glw9axoBzbrhf6tfZH38mSQ29nkOi\nAv7F4aAvjJyTteNM7w6/DofMnt2MQYOq/6c9IMWH+RihvRFoD0QClT5Ce0KIfyAtLZOFC7Om2Lm5\nNaDvgLq4e2hy8BTMHgPJDaH3G+iQCddLgo72G65xj4eE8Yzn1KUOLfl1vdTIDLB5BqV1YHsemDQP\nDgVkPRVZqkga9vZHOH/+uQyHZLOPMTyyCWjzEdoRQvxDAQFPqFp1NcHBz7hwwZFcFg2o0kUTXR1Y\ntgfmV4C98eBbVGF8oYcc197CejYSzVtqUoMaVOMVr39p72oK1H4MLQxhjia0soOwZ1nj10rKK2rW\nXEdGhpqLF2U4JLt9jEo7ECj+EdoRQvyNyMhExo07zqlTj1m+vC0FS5Sl5wQVejqwfiOs1gO/eFiU\nX6FUrvucVJ0ikwzqU58+9EIbbaKJ5gA+OOEIwJ44cImAHwtAvrtQxw1cesL4wQrr119i0qRTLF3a\nmj59Kmdz7wXImLYQOUJmppq1ay8xdWoA/ftXIejcUGau1eHQEpg8Fm7XhEFx4GaosMDiAQGqkzwm\nk6ZYUR5LNH7+pTqat2xgM61oialixoRI8IyDIxZwdAu47oCf5kBNy2S6d/chLOwtgYF2lCuXJ5uv\ngPgfCW0hvnCXL0fg7HwYHR1NTpwYQOiDfNTuCbZtYOxOcI+DLoBf6UeEaJ7kBMm0oDmWlPslrAEi\nieQntmJFE0pn1sDmBSSo4WhuGDsG3sXBpb3w5OEzqlffR6dO5dixowu6uhITX5LP8q/h7u7+y9dW\nVlZYWVl9jo8VIkeLi0tl0qST7N59i9mzm1OjflWcZ6jIzIQZq8FDD26lgmfxFzzRPc4pYmmGFZWp\n9F5YAzzmCZ7soi2t0UypSs1nWVP/OkdAK3vo0RamD1ezYH4gK1eGsGZNB2xsymVTz79NAQEBBAQE\n/O37PtZ8neKAD38+e0RRFOUjfYwQXz9FUfDyuoOrqx+tW5di8tQWrPA0YMtBGD0KrtaG4GSYWfAN\nmrlO8Fz1nGZYUZ1qaPLH+XzXucEhfOlOVy7GlmLkK1iaH17tz9pJfcMMqFIqln799qOlpcGWLZ0p\nVMgoG3oufuvn6ZR/yOiPUWl7Ak0Ac+AZMIWsGSVCiH/pyZMYhg715fHjd2zd2pl3GcVpNAAa1wPH\nHbAwGZz1EuhW5BR3VbdoRAO60QUddLIeY/zNj7iCQgBnCCWUvsoA5r0qwLFE2GsG89yyhkMu7ITQ\nc7ep2cOXUaPqMm5cfTQ15Zm7L9nnmBkvlbYQfyM9PZNlyy4wd24QY8bUo0vP+oxdoMmDp2A/DTaY\nQlmdDOwLneeWVhBVqUJTmmCgTgGNn/cAS1wOhiMAyCCDgxziJRG0TO+L/XMjzDXBJQqGfA/dWsOk\nIWmMG3uUU6ee4OlpKzvLfGE+ZaUthPgAFy48Z/BgHwoWNOJM4CD2nzajQR+wHwIabrA2TWFq4btE\n6h8lnvw4ZXbAPPkYpEyG9GugZwsmP4LBUAASSWQHO9HHgFIJ9jR5oYurGaTtg4HbYf0MKJDrJXVq\ne1GvXhGuXBmCkZHu35yl+FJIpS1ENvnt4k6LF7emQMmKDJ2pomQJsHSFTRkwIm8kRXIfIV4VR8fM\n2pRM3A1Jm0HPBvR7g3ZdiK4Nup3BeBaveM12dlBeqcCFNy1Y+06DFYawajKkZ8D2+Qo7tgSzcGEw\nK1a0pUcPWZnvSyWVthBfEG/vuwwb5kvbtqU5fXYoM9fqc2YD2LnDjoKgo5PCiuIneax5nQrqOtRO\nOI1GYjvQ7wl5b4JmwayGktZnjWXncuMWt/HmII0y2zL9eRUyFZj/Gob+kLUzur1NPHb9vElKSick\nZDDFiplm6zUQ/42EthCf0fPncQwffoTbt6PYurULD6KybjR27g41nWFzupopha/yWv8EBkopxibq\nopvQB3SaQJ4Q0CqZ1ZCiQNIKSJhDpvkJjmmc4yY3qZzcn97PCtHXCFK2wg9HYc8SiHt9n1o1DzJk\nSA0mT26ClpbcbMypZHhEiM9ArVZYsyaUKVMCcHGpiU33RgybpYWigsaTYIMGOOV9ScHch9ACuqZq\nYR43FzSLg/Fc0K7+a2NKOsSNhLQzJObehafWBTQUTV5Gd8XjrSEzNMDjeyhRBDwmZbBgrj97995m\n+/YuNGpULLsugfiXZHhEiGxy61Ykgwf7AHDEbwD7AvLR2hHs3OB4BbisncTCwieI0LxLs7T8lItb\nh0qJBePloPe7tdjU0fCuG6j0eWLuyU4NXyzV1fnxWVNS1RqMvQMT5sOM4dCkyhvatd5HiRKmXL3q\nhJmZfjb0XnxsEtpCfCKpqRnMmROEh0cI06dbUbJSTXpMUFG1BrTfCtsy1EwseIVY/RNYZOaj77tL\naKUFgdFM0O8Pqt89KJN+G951RNHrTJBRB4JURyid0hmX8O/orQ9PlsNPYXBqs8KFM1do3NifWbOa\nMXiwrHv9NZHhESE+gaCgcBwdffjuO3PcZ7ZnwRYjgq9C9xmwzQw65X6JZR4fDJU0uiXcxyhpBxgO\nA8PvQcPwjw2m7IdYR1KNZrHTQJ9kJYVn0d3Y/NaUcUmwdCxYW8EEh2SGDT3Io0fv2L69iyyjmoPJ\n8IgQn0FcXCo//OCPl9cdli9vS5yGJa2dVXTqA6VdwJ8Uphc5SZTWdayTUygWvxaVbkvIewM0C8Ei\nOzi+GdbfgyJlQFFDwnRI2sir3JvYqnOfopnF2BDeAh2VJv0DYc5mWDcNjDWeULfOfrp2LY+np60s\n9PSVkn9VIT6SQ4fu4+x8mFatSnLgyFC+X6JHXCp0Xws7NRVczW+iaXSU79LV2EXvQRNd0NgEx67C\nxZ5gOwbyFIHcBSBXblC/g5h+KEoc5/Ms55TmLQokdWTEM0vsdODcVDivgnM7Mlnz40m2bbvBhg0d\nadOmdHZfCvEJSWgL8YEiIxMZNcqP8+efs2FjJy49LkFbZ+g7Bk5WgRcGb5he4DA6ykt6xl4mV+o5\nSBkLe6/BuZ5QrxO0dwKPYdDBBTwjIP0mvOlCum4LPI1rkUgUL18PYV2cKaMiYOlkGNEXbBq+oXtn\nLwoWNOLq1SHkzfsnQyviqyKhLcR/pCgK27ffYMyYY/TvX5mNns4Mn61NfgtouwX2KhmMKxJIos45\nWie/oWTCVlRKLzjcA47OyArojQ/B2BxuBkJ6ClS2guTtEOdKpJEbmwygcEZhVj5tTjktTRrvgnWB\nsH+FwrXzl7CyOsX06VY4OdWUm43fCAltIf6D8PBYnJ0P8/x5HHv29WF/UEG6j4He02FvYShj+ghX\ncx/KpMfRMnofWphAyDDYvhpqtoFV1yHPzws0XT4O8/rA95uh8A6U+CNcMJtNgFYM2nGdGRVRmmEZ\n4OkGtSuC36oERrv68OJFPEFBdpQtK7vKfEtk9ogQ/4JarbB6dShTppzC1bUuVRo0YPgsTWo1hqQ+\n8FyViKOFH5mat+gXf4vcKafhmQNsPAx6huC8DL6rkdWYosDehbB/CXy/FIosJE0jPztMW5KKMUdf\n2vI8zQirc/DTKlg2AYyU+wwe7IOdXVXc3a3Q0fnj+tni6/BXs0cktIX4h+7fj2bwYB/S0jJZuMSG\nNd55OHMZOs4CT2OFIQWvYpDrGO1So6kStw1VYgvYnQw3z8OgBdCkB/xvCCMxDpYOgleP4XtH0JnE\nK8MBbDQ0I19aA6Y+bUBHTQ3uzob0VFg7NY0Vi49x5MhDtm7tLE82fgMktIX4jzIy1CxZco55884y\naVJj8pWuzeh5GrTpCffbgL5ONDaFDmKmhNM19gR6aXFwqiHs3wntnaHHeNDP9WuDT27BTFuo3Ah6\naaIoRzlt6kKotg4Rb23ZE22B3Yuslflc+0HTKi/o38+LBg0sWLasDSYmetl3McRnI/O0hfgPbtx4\njb39QYyNdTng68i8n0x5GAA2K2G/TgajC54FvUBsEp9QNNEb1cMusOk0FHoCyy5CoVLvN3j8J1g/\nFuxGQ42dJGsVZbOJM5pKSXY87kARlR71toFnCBxYkYnfgUBsJofg4dGObt0qZMclEF8YqbSF+BNp\naZnMnRvEjz9eZOas5ijG1Zi0XEXnIXCuPpTL9Yy6+Q9gmR5B65g9aMaWgJ2a8OgeOC2DOh3ebzA1\nOWtK351gGN0TxexHwowc2K1nimZiWxa+rMKgZNgxFto1giGdo3Ec5IW5uQEbNnSUPRu/QVJpC/EP\nhYa+xN7+AEWLmrD/sBOTPIyIT4d2G8BXI5VRFv5oaF2kf+wNzBLPozrdHLx9wXoYfL8PdH+3MFP4\nHZjdHYqXg+llydTawwHTUTzXzENwRDeiUnLT5SRs9oTV7grP74TQolkA06ZZ4eJSS6byifdIaAvx\ns5SUDKZNC2DjxqssXNiKqMxK2IxU0X0M+FUG7dz3GWbmQ8PUl9R6swONx/VgkwnkjYKlF4hXGXC0\n3wDar1yJQZ6fp+Gd2ALrxkBfO6jlSbR+UzYY90I/tR6zHzWmK5o8nwgR+bOm8n0/xpt371I4e9Ze\npvKJPyXDI0IAwcHPcHA4SIUKeRk+tgNuSw3QMoIi4yBUI5GhxY5gqHGdPrGBGMZEwL5ScO06DFlC\nXPE6GBUuzJu7d1lZoQJO166Rv3QJ8BgK9y/CyIYoeX05Z+LAWV0z7r+xJSCmKO2vwfYlMGskGGXe\nxNX1KC4uNZk4sbFsUiBk9ogQfyYpKZ1Jk07i6XmTJUvbEvauPEu3QveJcLC0Que8Nyho4otN8lMs\n43ajCmkCO85C095E1erDyZmzeeDrS5G6del54AB6Jibw8ArM6QEVqkD3RyQbmPKTiRVqdUUWP21H\nPXR5vgDSE2GZWwoLZx/m2rVXbN3amRo1CmX3JRFfCBnTFuJ3zpx5ioPDQWrVKsQOLxdGL9AndxFo\nsBnOaMUy2sKHvMpdur09gu5LYEspSA0nbuhPnNq8h/uT21HX1RVtAwMib9yAzEzYtxh2zQF7G5Sq\n3tw36sU+/cLExFmzLbICvcJhqzuM7AtVijzApq0PtraWXLrkiL6+djZfEZETSKUtvjmJiWlMmODP\nvn13WLa8PddelGX1bug+FbyKqulX4BK5jfzokRhGiVgfVMfrwZFzpHYaR9CNGC6tXUuNIUOo5uCA\nz+DB6Bga0sVjKbprh0FCFDiZkWH+in2mXYjUKMmWZ53IrzZGYy08ug1r3dPZstaPo0cfsnGjDc2a\nlcjuSyK+QFJpCwEEBDzBweEgDRpYsGWPCyPn6lG4DNTeDBe03jLOwptimffpGH0A7TAz2JAbdQG4\nWnccp8YspVSrVgy5epWUd+/Y2rIl5bt1o7lNAzTcGkLzxihtr/LKyJKfcjUmLaklP76oTZ93Kna6\nQe92MML1Kf26HaBx42Jcu+YkD8qIf00qbfFNSEhIw83tOAcO3GP5CmtCHn/Hxv3QdQbsK6jGvtB5\njA1P0DfhDoXf+qM6UBUu3ORpPSeObPBC18iI1kuXUqhGDW7u2sWRYcNos2ghlRJC4dwBcKmGutRV\njpn05ba2BUde2pKampfCe+C8P6xzz8DP+yTbt99g9eoOdOxYNrsvifjCyY1I8c06efIxDg4HadKk\nGH0GtWHkXD2KV4JMB4jXicK2sDdlMh7QLnYvmjcsYHMYCaUbc/yumidBZ2m5cCEVundHnZHB8XHj\nuO/jQ/elsyngMw2KFYbeYbzLXYmNRtVIT2vE4nAruidpcsgNWtaFPi0jcBniRfnyeVm1qr2seS3+\nEQlt8c35X3V98OB9VvxozbmHpdnsDV1nwd78mTgWDsZY/yT9Em6SPyoQ1e6yKPefcbmIDSfXbKOq\nnR1NpkxBJ1cuYp4+xat3b/RMTenctRb6R1dC/3ooNc4RbNKfs3oWnH/VhfAkCyoeheN7wGNSJleD\nAlm1KpQlS1rTq1dFeVBG/GMS2uKb8r/q2sqqOL0d2jBiri6lKkOqPSTrRNKpyH4qpD+gVcxuNC+X\nhC13ibZsi8+Jh2SkptFxwwbyVawIwF1vbw4NGUI9RzvqJ51FpUoBhyQS8+djk3EjUjLrsOhpS2xS\ndTj1A1QvC0NtoxgxNGtHmXXrrOUxdPGvfcrQbgMsBTSB9cC8331fQlt8NgkJaYwff4IDB+69X13P\nhr35MhlS+CxG+icZkHCdvK8uoPIsjvr5W86btSRo4w4aTZxInREj0NDUJC0hgWNjxxLm54etax+K\nnF0DNrVRml3gqkkvjuoV49abzlyJK0W9M+CzCZaOVxN+M5iFC88xd25z7O2rSXUt/pNPFdqawD2g\nBfACCAF6AXd+8x4JbfFZ/G9mSKNGRenn2Jbhs3UpURnS7CFFN5JOhfdTMf0hLWN2ohFSGrbdIqpS\nJw4cvIq2gSHW69djViprVb7ws2fxHjCAYrVr0rpoHHqxT8BRj5Si2mw1aUm8ujrLwtvSPE2fkClQ\nIj983y+acaO8MTDQZuPGjhQrZpq9F0TkaJ9qyl9t4CHw5Oc/7wRseD+0hfik/jfv2svrDj96WHMh\n7Dt6u/1m7LpAMMb6/vSLv07e1xdRbbNA/SqKs4W7cW7JLprOmEHNIUNQaWiQkZLCycmTubFtG+2H\n96PcjS1QogpKu9fczd2V/QYlefS2I6ffWdLqInivhHmj1cQ8vUCnDoFMm2aFs3MtNDSkuhafxoeG\ndmHg2W/+/Byo84FtCvGPBQY+xc7uAPXqWfDT7qx510XLQ9VNcEUniu+L7Kd8+kNavfFEM7Q0bE0j\nqlJlvK9cRtfkHo6hoZgWLw7Ay9BQ9vfvT76y3+HkUA/DsH0w0oLU0pHsNHUiWqnC+kftaZhhiNF0\nCDcEr0UxTPx+P4oC588PonRps+y9IOKr96Gh/Y/GPdzd3X/52srKCisrqw/8WPGtS07+dc2Q5Sva\nc/1lOXq5QffpsLeQmkH5z2FkcIK+8TfIFxmMakdJ1M9ec86iB8FLd9J0xgxqDBmCSqUiIzWV09On\nc2X9elq79KPivR2o8pdHcYzlgZkVew1K8yymA8eiK9HyCngtgxkjFNIjQ+jcIYCJExsxcmRdqa7F\nBwkICCAgIOBv3/eh/8vqAu5k3YwEmACoef9mpIxpi4/q/PnnDBjgTfXqBRkysh2u8/TJVxK0nSFK\nO5oeRfZTJvMB7WJ2o3mtGGy8S3Qla7x9bqClp4/Npk3vVdcH7OzIXawoHaobkCv8IgzOT1rZVHab\ntOc1lVgdbk3tjFzcnwnGWjDNKZbJ471JTc1g0yYbWUJVfBKfakw7FPgOKA68BHqQdSNSiI8uNTWD\nqVMD2Lz5KsuWtePhu/J0Hws93GGvhZoB+UIwyXWMngl3KBR9EtXu8ih3nhJaug8BS7bQePJkag8b\n9svYdYDWxxy3AAAfiklEQVS7O1c3baK1S18q3vNEZW6JMiiRsNzl2WNYhmex7fGNqkzr6yr2LQb3\noQqq2FA6tT/F9983YMyYemhqyhKq4vP6GL/PteXXKX8bgDm/+75U2uKDXb4cQf/++ylTxpxR4zsw\nZpEhhnkg9ygI04ploMV+LNQP6BSzF627BWDtI+LLNeNAwAtS4uLptGULecpmPTr+LDiYA/b25Lcs\nS9sKmuR6cRkG5SOtTDJ7TDvwSlWJtU+tqZ5hxKM5oK/AdKc4pk/2Jj4+jc2bbbC0zJvNV0R87T7l\nglFHfn4J8dGlp2cye3YgHh4hLFrcmqiMSnQZpaLnBPAuo9AtzzVqGfvSKfEBxWMOozpYE4Kv86D2\nEA7M8aCmszONJ05EQ0uLtIQE/H/4gdt799LWqRfl7+yAAhVQHBN4mLspew2/43lsew5HVab1DRVe\ni2GKs4Jm/CVsO55izJh6jB1bXzYoENlKnogUX6ybNyMZMMCb/PkNmTS9I+OXGZGpDUXHw2WtRJyL\nHsScW/SIOYROuAGseYu6cEX8owty08ubztu2UbxJEwDuHz6M79ChlKhfh1bFktB/cxccTEktnc4e\nk7a8pjKrwztQI8OIxz9X1zOc45g2yZu4uFQ2b+5E+fJSXYvPR5ZmFTlGZqaaxYvPMX9+MLNnN0dt\nXC1rr8ZRcLQKVDK7y8DcB2iVHI5l7E5Up5rAgUASuk5h78p9aOnGM+TKFQzy5OHd48f4uboSdecO\n1kNsKXV1K5SshuL8lvtmjdn388wQv+iKtLyWNXY91UVBK+ESXayluhZfHqm0xRclLOwt/ft7o6Oj\nydyFnXBfY8LrOKjiDv6aqYwudgQ9zSsMiDmFfnQCrNeBTG2eNR3FHqcRVHNwoMmUKWSkpHB2/nxC\nPDyoN2gA9TKuoJUSCXbapBTTYqdpS6KVqqwKb0ft9Fw8mAW5NMHdMZbpkw8QHy/VtcheUmmLL5qi\nKKxeHcrkyaeYNKkx+b+rQ0dXFZ0d4FlDSDN6ytB8XtRLeUntd1tQXW8KG05Bx+FcTsyPv70THTds\noEz79lzfvh3/CRMo1qghQ2YMxuTUuqw1Q6wiuGnaDR/9Ejx+a03Au/I0vQT7l8O0YQrEhGLb8RTj\nxtVnzBiprsWXSUJbZLsXL+KwsztATEwKh47Ys2xXHq74Q9eVsE8ng++LnALtYOxjL2Mafxv21IMb\noagneeG3bjdhR7did+YM8S9fsq5WLTS0tOi2wB2LwB/h6ROYmpfEgglsM3ImXqnOmkdtaJJugNEM\neGoA+xbG4D7xAMnJ6QQG2snMEPFFk9AW2UZRFHbsuMGoUX4MG1abWlaN6OamQbOOkNsZHutF4VZo\nL5bpL2kVvRWNiJrwowaUMSZldgD77Aejzsykw9q1+I0ezZu7d2k+dTIVEi6jOjgJ+tZEqXGRUOOe\n+OsV43Z0Ry7GfEerC+C9CmaNVEiOuIhtx9NMmNAQV9e6Mu9afPFkTFtki+joJJydD3P7dhRr1nbG\n82RBDp6C7vPgJyOFUUUuotI/Qf/ERxRKPIzqrA3s3g9OS6Fpby56eHBk2DDKd+vG0zNnaDRhAjUq\n5Udr41ioWhk63yHGrBxbjGqRmlmHpU9b0Dpdl4tToVgecOv/jolu3iiKwoYNHeWpRvHFkU0QxBfD\n1/cBgwf70KNHBTr3bs7gqVpUqA7JdvBGI57+FvsxVZ7SI8Yb7URT2KQN0W9gwi5eRyeREBGBRf36\nnJ4xA11jY+r27ITOlvHw8j7Yl0Zd8jpnTHpyUdeCi69teJhQjGqnwHcLLByr5vWD88ydG8TkyY0Z\nNqy2VNfiiyShLbJdYmIaY8Ycw88vjPXrbTh7rzgenmA3A34qBAPy3cXE+ADWKW+pEL8R1fO+sMwL\nGnYFuzmgrcPeHj24tXs3E1NS0NJQwb5F4LUIrFugNAvgdS4rtuayJCOtIUufNcYmWZuAiVClNAzv\n/uaX9a7Xr7emVClZkU98uSS0RbYKDn5G//77adiwKCPHtsNllg76xlDkezirpDG22FEyNG9gH3sR\nw7SHcLIVHNgBruuhrvUv7WSkpqJOT0fn4UVYOQzy5Ye+ajLMozhs0pkwraIce2lDTGoBSh+Ck16w\nZHwmYVeCWbLkPDNmNMXRsYasyCe+eDLlT2SLtLRMpk8/zfr1l1m5sj1v1Za0GgIDR8PBKlDCKIKh\nefdSJj2epm/WoZHWHNakQuJFWB4C+Yq+155WQjSsGwu3g2GgFYqlD4+MurDLoBVpSc3weFyH7vEa\nHJoA5WqB56zXuA7PeqoyNHSw7CYjcjyptMUnc/t2FP367adAgVzMW2jDJA9DHr+EpnNgB2omFz1H\ngt4ZBiS+pFDibng1BhathEbdwG42aGn/2lhGOngvg91zoVV7aH2ZZIPc7DJpSpyqNDueW5MrwxRT\nTwgNgBU/ZHLh5GnWrr3E/PktGTCgiuzVKHIUGR4Rn42iKHh4hDBt2mlmzWqGRbnqOExW0akH3GgD\n+trxdCm8HwPlNb1iDqOtqCGwOexaCSPXQb2O7zd4xR9WjYC8BaFvPhSzU1wy7sNx3Xy8jWvL1tcV\n6RGpYvck6NYKOjV4wTAXb8qWzYOHRzvZCV3kSBLa4rN48SIOe/uDxMSksH5DZ9Z6m3PgJAxcCKsN\nYXiB+2gZe9M6TU21mGWoVHawPgxePIBJe6FgyV8be/UE1o6GR1dhYCeUcp6802/EVqOKaGRWYUV4\nKyoq+iSvhid34ccf0vHZ7c+uXbdYvrwNXbuWl+pa5FgS2uKT8/K6g7PzYYYOrUWHro3o56aBZQXQ\nc4KQjAzGFzvBG63rDEp4Ru6k/ZA8D+bOhTK1YKgH6OpnNZSSCLvnwaGV0LEvNL9OplYkx0y6cVvb\njDtvrPGPKU7HB7BjFrj0hHplHuPifJAGDYqydGlrzM0NsvdiCPGBJLTFJxMXl8qIEUcICgpny5Yu\nXHhQhNlrYehk2FYKmhlHUynfHgqoFbrE7EITTXg+HOYMgT5ToYMzqFSgVkOAJ2wcDxXrQ48CKPrb\neZyrNzsN8qOX2oAF4Q1pjRYP5kFmEiwam8LGVX4cP/6IVava0759mey+HEJ8FDJ7RHwS588/p08f\nL5o1K84xfydcZukQEw8OW2BlBswsfJ0oA1+s0vJS8d00VPr9IKIjzLYGt+1QvWVWQ9dPZ80KARgz\nHAqvJkm7PDuNR5OsKkRwhDUvUszpEgq7PGCKMxQ2vEt3G186dSrLrVsuGBnpZt+FEOIzkdAW/0lG\nhprp00+zZs0lVq1qj665JQ36QY/eWTcbg0ljYXFfojWe4JIIuRN+AJP1oNsBumuA0zKo0gwu+sLB\nFRB+Bwa4QhV/1Or1nDcZyGkdPdLi27AiohK90lQ8mQJP8sORlQnMneHLzZuReHra0rhxsey+HEJ8\nNjI8Iv61x4/f0aePF0ZGuqxZa8PSHUbsPwHDF8CiXOBoHoWZ2S6KkJtOscfQTL8Eub0hwQQeX4eH\nl2HPPDAwBiMzaG8PNZ+gZGwgIlcfthkWwDSjMqueNyefok9Bb/DzhsVuCsmvrzJ+/AkGD67O5MlN\n0NOTukN8nWRMW3wwRVHYuvU6Y8YcY/z4BrS1qUfvcSpKloCCrnAwBeYVu8JjXT86ZFai8rupqDRL\nZ1XYGrlguRP4roGxW8CyLiS8A4s7ED+RFJ26eBnXI0rDgHtvOnDgnQW934DnJGjTAJxt3zFu9EHi\n4lJZt86aatUKZvflEOKTktAWHyQ2NgUXF1+uXn3Fjh22hD7Mz/jFMGoceFeGgtpp9CzsyyuNcAak\nFcfsnRMYjgTD77NuMgI8vw93z0NDW9C4AnGjUKPBReM++GsnYZzSlLnPatFCU4N3q+HuTVg1Rc2V\ns1kLPE2Y0JCRI+vK5gTimyA3IsV/Fhz8jN6999G2bWmOnxzMqHna3A6DCRthDuCW+w06pjvRVBVg\nWJIm2vGDwWQz6LV9v6EiZaCAJsT3R0kP5bmRC556GuRRFyboRStepRrR+w5snp81je+HrhG4OB/E\n3NyACxcGyQJPQiCVtvh/ZGaqmTMniBUrLrJunTVmhcvS1w3aNAN1fziZDIuK3uSm7iFaKVbUjNuB\nKvUYmPmA1nfvN6aOhoRZkLSFhFxOeBkWI5oU3sVYszKyOAMyIXA66GrBErc0dm0JYMuW6yxY0JJ+\n/SrLQzLimyOVtvhXnj2LpU8fL7S0NAgJcWTzIWNWLgD3WbC6MJQhg/mljvFQ4x4D1R0p/G4kqDQh\nz3nQ+M2iTEoKJK6AxPlk6nXhdN4fOaf5hIKpFVj1rA7ltTTpegy27oI5o6BIrjC6dTxE/foW3Ljh\nTL58htl3EYT4AkmlLf7A2/suQ4YcwtW1Dr36N6D/eA20tcB2GkxNhqn54tA23YW+Sp/uGdXQe9sV\ndFuD8UJQ/VwHKJmQvAXip6JoV+OesT0HtB5QQF2Co69acT7RiD6vYetUaFwTxtsnMnfmMQIDn7Jq\nVXvatv3u/z9JIb5yUmmLv5WUlM6YMX74+YXh7d2DyCQL6vSEof3hVXtYnAg7iodzSXcXdahFE3VF\nNN7WAcNRYDgsqxFFgVQfiJ8IKlOicv+It04UKcpLVLHdGPOqGH20oOKPsOcBbJyp8OLBdZo1Pk7/\n/pW5dcsFQ0Od7L0QQnzBJLQFADduvKZnz31UqZKfc+eHMH2NHr5nwGMFzDGGEpmwsdQlgjROYEtn\nylIG3tYDPZuswFYyIe0ExE8DJZ4Uo8kc1dXltuoOxVKbsex5DQppamAXCps8YERfmD04mpEjDhEb\nm4qvb29q1CiU3ZdBiC+ehPY3TlEUVqy4yIwZZ1iwoCX1mlShjZOK0kVh9jYYGgM/mKgpbebHJdV9\nHHEgDz9vgqvEQ/o1iBsDyTtBIz9qQ1fO65ckQHWW0pmVuPt6BKsS9HFMhF3uoJEfzmzJYP+uYJpa\nnWfSpKx9GmUanxD/zIeEdjfAHSgH1AIuf4wTEp9PZGQiAwd6Ex2dzLlzDgTfMqNhX5g6HMKbglss\n7LVI5ZHBbl6TiROO6KMPmVGgDoc8VyBxGShxKGbHuaetyRH8MFHC0Iyxw+V1PvrqQeNtsDYIlrhB\nXr2n2Fof4rvvzLh0yVF2khHiX/qQ0L4BdAbWfKRzEZ/RkSMPcHA4iJ1dVcZ9b8XYhZoEXYa9G2Gm\nHpACZ0rGcUhrG4UpTEc6ZK3OB6BhBJlqUGlDrrG84CVH8SOeeAolt2baizJYaKtwvQcrFkCfDhD4\nUxIzp53Azy+MFSva0qlTuWztvxA51YeE9t2Pdhbis0lNzWD8+BPs23eHHTtsyVu4OPX7QM2K4LEV\n+r6BfnrgnC8CT9V26lCbxjRC9dub2Co90KlFNNH4c4pHPKZKhhUbI6oTmqKJaybsmgqH1eC3VuHq\nhWvUqXmCXr0qcuuWC8bGshqfEP+VjGl/Q+7efUPv3vsoWtSEq1ed8A3Sp9tEmD8WMqygVyRsKATl\njcLYyl6saU8lKv6hnTjiOMVpbnKLWuq6pLy1pm+0Ls65oNABmOUNM0dAPctIhjn5kpiYxpEjfahe\nXdYLEeJD/d3dn+NkDYP8/mX9ic9LfESKorBp0xUaNdqEo2MNdnj2YMxCfWaugSMbIKgOLH0LZ4pD\nMaMb7GIvPen+h8BOIIHDHGE5Hmgr2hSPG87gMCtuJesyIxx+6gcx0XBxZzpPrvrTrNlPdOtWngsX\nBklgC/GR/F2l3fJjfIi7u/svX1tZWWFlZfUxmhX/wP8WerpyJYKAgAHoGeWjQR8oXwoO7YB+b6Fw\nJlwoAWEa1/HlKA4MJD/5f2kjkUSCCCaEUKpSmRYpwxj/yoiYTJitCVtnwKrXsGM+xETco2mjo9Sp\nU5jr150oWFA21RXinwgICCAgIOBv3/cxnog8BYwFLv3F9+WJyGxy6dJLunffS6tWJVm0qDVHz2rj\nNC1r15canaHbcxiSGybmgSeqx+xiD/YM+CWw44knmPOEEEpFKlA+ozHzIk05HA8/mEKEJ6zbDW4O\n0KVpLGNGH+HOnTd4eLSjRYuSf3N2Qoj/z189Efkhk2M7A8+AusBh4MgHtCU+IrVaYcmSc7Rps515\n81qwYkUHpq3SZvQ88F0Nxu3B5hmsLgiT84KiysSHw3SkA3nJy3NesI/9LGUFKaQwSO3MnTcdqR9m\nSm4NWPwClvSFJ8/h0p5MlOiz1Km9hho1CnL9upMEthCf0IfciNz/80t8Qd69S6Z/f2+iohK5cGEQ\nhsa5aeEAujoQshuWZsCuKDhVDCroZR0TTwKRRHKDm3hzEH30qUE1Riuu+MQZUCcSaumBpzYsnAxH\nXsG6aaCvhNO+9SGKFDHm/PlBlC4tS6cK8anJ7JGvyNmz4fTp40XnzuXw8urO5duaNBkEdp3hBxdw\nfAUP0uBcCcj7m395E4zpQTfSSKcNrTDFlNOJ0Ow1ZABrzSDgJ+i9DyYMhj7tkpky+QSHDj1g6dLW\ndO1aXpZOFeIzkdD+CiiKwsKFwSxadI5166yxti7L2t0waTmsnw4trKDLc9AG/IuBwe8GxVSoqEwl\nAO6mwsBIuJYCM/OCKhgcFkDzunDVS8Hf7zpVKh/H1taS27ddMDHR++z9FeJbJqGdw0VHJ+HgcJCI\niAQuXhxM/gImDJ4CwVcgaBuYF4HmT+E7HdhYCLT+oiB+mQ7uUeAdD+PMYVwsuA2DpGTYvRhy60XR\nv/cR3r1LxsenF7VqFf68HRVCAB92I1Jks9DQl9SosZZSpXITGGiHtr4JzezgbSyc3wkmRcDqKTQy\ngJ/+IrAj0sH1FVR6BKaaEGQGD1ZAVxew7wxnfkrn8F5/GjfejLV1GS5eHCyBLUQ2ktDOgRRFYdWq\nENq1287ixa1ZtKg1l25rUqs7tGkIe5ZAjA40fgLdjGFevl/31v2f8HQYHgEVwrLmFF22gDyHoL4N\n5DKAO4cgr849qlRZxYMHb7l2zQlXV9lUV4jsJsMjOUxiYhqOjoe4dSuSoCB7ypQxZ7sPuM6FTbOg\ngxUkqKF9ONibglue948PS4P5b2BvPDiYws2ScOEMNF8AlqXg3A7Q1YhlYP+sOderVrWnVatS2dJX\nIcQfSWjnIPfvR9O1626qVi1AcLADurrajJ0P+/3h5CaoVCbrfQNfQC19+N7812MvJ8PcaDiZCE65\n4V4peHoPeo3LGk5ZOQWa1MxgwYJgliw5j6trHXbt6oqurvwXEeJLIj+ROYSX1x2cnA4xbZoVTk41\nSUpWYTsSYhMgZBeY/bws9fkkuJwCd0rB60zYFQtbY+F1Bow2hw0FITYKRk+CE+dg2jCw7wInT4ZR\nufIRLC3zcOmSI8WLyzrXQnyJJLS/cBkZaiZMOMGePbfx9e1DzZqFCH8JHYdCNcusmR06v9lScdlb\neJwO5cLgbSbYGMGcfNDMEBITYc5yWLMbnHvCPV94E/kOW1s/btyIZOnS1lhbl82+zgoh/paE9hfs\n5ct4evbci4GBNpcuOWJubsCV22A9FFz7wRi7P95gHG0O7XNBHX0opQMaKkhPhzU7YfpKaNsIrnpB\nHtN0Fi06x9Kl5xk9up4MhQiRQ8hP6RfqzJmn9Oy5FxeXWvzwQyM0NFT4BUG/8bBqCti2+vPjauln\nvSBrY/SdvjBxGZSygKNroUo5hQMH7uHqepSaNQsREjKYEiVyf76OCSE+iIT2F0ZRFBYvPsf8+cFs\n2dKJ1q1LA7DnKAybBfuXQ4Pqf9dG1nj1pGVZu4Ktnw5N68CtW5G0aXOM8PBYNm60oVmzEp+hR0KI\nj0lC+wuSlJTOwIHePH4cw4ULg365GbjdB8YthGProMrfbK148Tq4LYaXkeA+FHq0hdjYZEaODMDT\n8yaTJjXG2bkm2tqan6FHQoiPTZ6U+EI8fPiW+vU3oKenRWCg3S+Bve0gjF0Ax9f//4F99U7Wzcku\nI6F3e7h1ELq2ymTlyouUK+dBcnIGt28PZcSIOhLYQuRgUml/Afz8HtKv336mTm2Ci0utX1bM2+6T\nVTX7b4Typf/82NCbMGcdnLsK4wdlzSbR04Xjx8NwdfWjYMFcHD/ej8qV8/95A0KIHOVzrKcpO9f8\nBUVRWLLkPAsWBLNnTzcaNiz6y/cO+MMQ96zArvDd+8ep1XA8GBZthruPYKwdDOoKBvrw4EE0Y8Yc\n49atKBYubEmnTuVk2VQhcqC/2rlGQjubpKZmMGiQD7dvR7FvX/f3HmaJiIIK1nB8A9So8OsxKang\neRgWbgJtLRjZD/p0yJqnHReXyowZp9m06Spjx9Zn1Ki6MoVPiBzsr0JbfqqzwevXCdja7qZgQSMC\nA+0wMNB+7/v/197dx1ZV33Ecf8uDQtZlhIhpJhBQNIpMcC5QcY4iylOGbjIChIAtbP4BsmGTsVpS\nwOCiQcIcCctimAFltIubTIUxgqQlPMkmoYUCncAGglAcGw8yHoS2++N0UKC1dPecHs/l/UqanLa/\ne+/3l9t+es7v4XT+Yhgz/HJgH/gEfv07eH15sKHml8/DoIeCNdrV1TUsWlRGYWEJw4b1oKJiMpmZ\nGS3fKUktwtBuYdu3H2XEiCKefro3s2dn06rVtRc7/zkLK9fBqGnw1wo4dRomPAnr34S7u11uV1Ly\nD6ZNW02HDu14990x3jJVugE4PNKCVq78iNzcd1iwYBhjxvRqtF11Nfx2BbRpDd/qBT26Qqt663x2\n7/4n+flrKS+vYt68wYwcea/j1lKacUw7ZgsX/oUXX1zP8uWjycrq/H89x7FjZ5gzZx3LllWQn/8w\nU6b0pV07L5akdOSYdkxqamrJy1vN6tX72LhxInfc0fwt42fOXGD+/OA+IaNH38euXZPp1OkrEVQr\n6cvO0I7Q+fMXycl5hyNHPmPz5kl06NC8f4J78WINS5aU8cIL68jK6swHH/yQHj06RlStpCQwtCNy\n/PhZnniimMzMDFatGkf79m2bflCd2tpali+vpKBgLZmZGRQX/4D+/btEWK2kpDC0I3D06GkGD17K\noEHdmTdvcIMrRBrz/vt/p6BgLefPV/Pqq0MZMuROJxklXWJoh2z//hM89tgbTJjQm8LC71x34G7c\n+DGFhSUcOnSKOXMGMmrUfc0Ke0k3BkM7RPv3n2DgwCXk5WUxdWq/63rM1q2HmTmzlJ07P2XWrAGM\nH9/b/3guqVEu+QtJVdVp+vf/DXl5D/Hss32bbP/hh4eZObOEHTs+Zfr0/jzzzINuO5d0ieu0I3Ty\n5Dmys5fw1FP3UFg44AvbbtjwMS+9tIGysipmzHiESZMeMKwlXSOK0H4F+C7wObAPyAVONtAurUP7\n4sUahgxZSs+et7JgwbAGx7BrampZvXovc+du4sCBE0yf/jA5OX3cGCOpUVGE9uPAWqAGeLnua/kN\ntEvr0H7uuT9TWfkvVqwYS+vWV45Fnzx5jqKiChYs2MItt7QhLy+LsWO/4Zi1pCZFsSNyTb3jLcDI\nFJ4rkSorj1FcvJNduyZfCuza2lq2bj3Ca69t5a23dvHoo91ZuHA42dndXLonKWVhXZ9PBIpCeq7E\nKC+vonv3Dhw8eIpt26rYtOkgS5du58KFGnJz+7B79xRvkyopVE2F9hogs4GvFwDv1R3PIBjXXtbY\nk8yePfvScXZ2NtnZ2c2p8UurZ89ObN58iHHj3qZjx/bcf/9tLF78Pfr1u92zaknNUlpaSmlpaZPt\nUk2WHOBHwCDgXCNt0npMW5KiEMWY9lDgp8AAGg9sSVKIUjnT3gPcDPy77vPNwOQG2nmmLUnN5OYa\nSUqQxkLbBcOSlCCGtiQliKEtSQliaEtSghjakpQghrYkJYihLUkJYmhLUoIY2pKUIIa2JCWIoS1J\nCWJoS1KCGNqSlCCGtiQliKEtSQliaEtSghjakpQghrYkJYihLUkJYmhLUoIY2pKUIIa2JCWIoS1J\nCWJoS1KCGNqSlCCGtiQliKEtSQliaEtSgqQS2nOAcqAMWAt0CaUiSVKjUgntuUBvoA/wR2BWKBUl\nTGlpadwlRC7d+2j/ki3d+3e1VEL7s3rHGcCxFGtJpBvhBybd+2j/ki3d+3e1Nik+/ufAeOAMkJV6\nOZKkL9LUmfYaYEcDHyPqvj8D6AosBn4RTYmSpP+5KaTn6Qr8CejVwPf2AneG9DqSdKMoJ5gzvEIq\nwyN3AXvqjp8EtjXSrkcKryFJCsnvCYZKyoA/ALfFW44kSZJ0A0r3jTivALsJ+vg28LV4ywndKGAn\nUA18M+ZawjQUqCQY5vtZzLVE4XXgKMEVcTrqApQQ/GxWAD+Ot5z08tV6x1OBRXEVEpHHubwS5+W6\nj3RyD3A3wS9IuoR2a4JJ8m5AW4ITinvjLCgCjwAPkL6hncnliboM4G+k33t4jZa690i6b8RZA9TU\nHW8BOsdYSxQqgY/iLiJkfQlCez9wASgmmFBPJ+uB43EXEaEqgj+2AKcJrna/Hl85LSPVzTXNcaNs\nxJkIFMVdhJp0O3Cw3ueHgH4x1aLUdSO4qtgScx2RCzO01xBcrlytAHiPYCPODCCfYCNOboiv3RKa\n6h8E/fscWNZSRYXoevqXTmrjLkChySBYzfYTgjPutBZmaD9+ne2WEWzESZqm+pcDDAcGRV9KJK73\n/UsXn3DlhHgXgrNtJUtbgiXHSwluXKeQ3FXveCrwZlyFRGQowQz2rXEXErES4MG4iwhJG2AfwWX1\nzaTnRCQE/UvXicibgDfwFhqRSPeNOHuAAwS7QrcBv4q3nNB9n2D89yzB5M+qeMsJzTCCFQd7gedj\nriUKRcBh4DzB+5e0IcmmfJtgAUAZl3/3hsZakSRJkiRJkiRJkiRJkiRJkiRJkqTo/BfipyW5wqRJ\n7wAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from scipy.stats import multivariate_normal\n", "\n", "p2d = multivariate_normal(mean=[0,0], cov=[[1, 0.99],[0.99, 1]])\n", "\n", "X, Y = np.mgrid[-3:3:0.05, -3:3:0.05]\n", "XY = np.empty(X.shape + (2,))\n", "XY[:,:,0] = X; XY[:,:,1] = Y\n", "plt.contour(X, Y, p2d.pdf(XY));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The problem now is that an isotropic distribution is not a good fit since the above density has two very different length scales. Having a tight fit between the sampling and target density becomes even more important in high dimensions.\n", "\n", "For the sake of argument, consider the problem of sampling from a multi-variate Gaussian with mean zero and diagonal covariance matrix $\\sigma_p^2 \\mathbf{I}_D$ where $\\mathbf{I}_D$ denotes the $D \\times D$ identity matrix. As a sampling density, we choose another Gaussian with covariance $\\sigma_q^2 \\mathbf{I}_D$. The condition, $c q \\geq p$ now requires that $\\sigma_q^2 \\geq \\sigma_p^2$ and the optimal (smallest) value of $c$ is found to be $\\left( \\frac{\\sigma_p}{\\sigma_q} \\right)^D$. Then, the acceptance probability is given by the ratio of volumes under $p(x)$ and $c q(x)$ which is just $\\frac{1}{c}$ since both distributions are normalized. Thus, the acceptance probability vanishes exponentially in $D$.\n", "\n", "**Exercise:** Illustrate this effect when $\\sigma_q$ exceeds $\\sigma_p$ by $1 \\%$." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD9CAYAAAB3ECbVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAACZ5JREFUeJzt3U+IXdUdB/DvVO1KEIqL0Bj6NkpHKW0TaBelmCKBKJmI\nCy2CC3XVRWlXRdRFJruCm9IWoQvtRlRCa4sJSpuFk1oKQk0qVicaCyMaUWgrhUJbrb4u5plM7Js/\n7947c99v5vOBQ+57c889Px7hy+G+e95JAAAAAAAAAAAAgB1gT5LnkryS5M9JvtdvOQCsZVeSr4yO\nr0zyWpLZ/soB2N4ub9n/3VFLkn8mWUzy+dG/I18YJm+2HAZgh/ncjcnfT818+u3/e6OFQZJTSW7I\ncoB/Ypgc6XCYzbKQZH/PNVSyEJ/XJBbi85rUQrbVZ3ZofrLzT8wkYzL6M50Us3xr5BdJvp9LAxuA\nDrW9PZIkVyT5ZZLHkvx6/CkLK44HowbABX9dSP62sO5pbUN7JskjSV5N8qPVT9vfcpitMOi7gGIG\nfRdQzKDvAgoa9F3A1rp6/3L7xLmjY09re3vkG0nuSvKtJGdG7WDLa/Zk0HcBxQz6LqCYQd8FFDTo\nu4Cp1Ham/ft0d18cgHUIXIBCuvgiEoD1nJjv5DJm2gCFCG2AQoQ2QCFCG6AQoQ1QiNAGKERoAxQi\ntAEKsbgGYAKzw8ON+i3O7Z2sw4nN+cEoALaQ0AYoRGgDFCK0AQoR2gCFCG2AQoQ2QCFCG6AQi2sA\nJjDxIplP2LkGYOcR2gCFCG2AQoQ2QCFCG6AQoQ1QiNAGKERoAxRicQ0wVV4cHpu4z76ZOzahklV0\ntEimKTNtgEKENkAhQhugEKENUIjQBihEaAMUIrQBChHaAIVYXAOs79D8lg11Vw5P3Gd22GysRrvQ\nWFwDwEYJbYBCugjtR5O8l+TlDq4FwBq6CO2fJznYwXUAWEcXof18kvc7uA4A63BPG6AQoQ1QyBY9\np72w4ngwagBctDRqa5vpaLRBkuNJvjTmb8PkSEfDAFXMDidfJJPUXPCyOY4mYzK6i9sjTyT5Q5Lr\nkryV5J4OrgnAGF3cHrmzg2sAsAG+iAQoRGgDFCK0AQoR2gCFCG2AQoQ2QCF2roGdZAt3oElOb+FY\nO4eZNkAhQhugEKENUIjQBihEaAMUIrQBChHaAIUIbYBCLK6BoprsDLM412ys4e7JN7mamWm6Y9XT\nDfvtDGbaAIUIbYBChDZAIUIboBChDVCI0AYoRGgDFCK0AQqxuAY61GTBS5I8lrsa9Do2cY99ebXB\nOMnM+WGDXvONxmJtZtoAhQhtgEKENkAhQhugEKENUIjQBihEaAMUIrQBCrG4BlZzaL5Bp9ONhtr7\nncWJ+zRa8HJifvI+TBUzbYBChDZAIUIboBChDVCI0AYopIvQPpjkbJJzSe7r4HoArKJtaF+W5KdZ\nDu7rk9yZZLZtUQCM1za0v5bkjSRLST5M8mSSW1teE4BVtF1cszvJWytev53k6y2vCVNhuHtm4j4z\nc012eElmYqEMG9N2pt3sfygAjbSdaZ9PsmfF6z1Znm1/ysKK48GoAXDR0qitrW1o/zHJtVlO4XeS\nfDvLX0Z+yv6WwwBsd4NcOqE9NfastqH93yTfTfKbLD9J8kiSyX/5BoAN6eJX/p4dNQA2mRWRAIUI\nbYBChDZAIXauYdubHR5u1O/6NOg312goC2XYMDNtgEKENkAhQhugEKENUIjQBihEaAMUIrQBCvGc\nNqUcydGJ+xzOsUZj7Zt7dfJOnrdmk5lpAxQitAEKEdoAhQhtgEKENkAhQhugEKENUIjQBijE4hra\nOzQ/cZfh7plGQ83/bPI+jRbJJBbKMJXMtAEKEdoAhQhtgEKENkAhQhugEKENUIjQBihEaAMUYnEN\nrc0ePz1xn9OZbTTW0fN2k2FnM9MGKERoAxQitAEKEdoAhQhtgEKENkAhQhugEKENUIjFNVwwOzzc\nqN/i3N6J++w7cUejsXKoWTfYLsy0AQppE9q3J3klyUdJJp9qATCxNqH9cpLbkvyuo1oAWEebe9pn\nO6sCgA1xTxugkPVm2ieT7Brz/gNJjndfDgBrWS+0D3QzzMKK48GoAXDR0qitravntGfW/vP+joYB\n2K4GuXRCe2rsWeuE7ZpuS/LjJFcn+UeSM0luHnPeMDnSYhiaeHF4bOI+++Ya7AqT2BkGNsXRZExG\nt5lp/2rUANginh4BKERoAxQitAEKEdoAhQhtgEKENkAhQhugEDvXFNBkR5l9DZZNzQ5PT94pyWKb\nJVrARMy0AQoR2gCFCG2AQoQ2QCFCG6AQoQ1QiNAGKERoAxRicU0Bi3N7G/R6evJxZibvA2wtM22A\nQoQ2QCFCG6AQoQ1QiNAGKERoAxQitAEKEdoAhVhcc2i+Wb8TDftN+1jAVDPTBihEaAMUIrQBChHa\nAIUIbYBChDZAIUIboBChDVCIxTUWrgCFmGkDFCK0AQoR2gCFCG2AQoQ2QCFtQvuhJItJXkryVJKr\nOqkIgFW1Ce3fJrkhyZeTvJ7k/k4qAmBVbUL7ZJKPR8cvJLmmfTkArKWre9r3Jnmmo2sBsIr1VkSe\nTLJrzPsPJDk+On4wyQdJHu+wLgDGWC+0D6zz97uT3JLkprVPW1hxPBg1AC5aGrW1tfntkYNJfpDk\nxiT/XvvU/S2GAdgJBrl0Qntq7Flt7mn/JMmVWb6FcibJwy2uBcAGtJlpX9tZFQBsiBWRAIUIbYBC\nhDZAIUIboBChDVCI0AYoRGgDFCK0AQoR2gCFCG2AQoQ2QCFCG6CQNj8YtXGH5ic7/8SE5wPsEGba\nAIUIbYBChDZAIUIboBChfcFS3wUUs9R3AcUs9V1AQUt9FzCVhPYFS30XUMxS3wUUs9R3AQUt9V3A\nVBLaAIUIbYBCZrZgjIUkN27BOADbyakk+/suAgAAAICp9lCSxSQvJXkqyVX9ljP1bk/ySpKPkuzt\nuZZpdjDJ2STnktzXcy0VPJrkvSQv910I0+9ALj5R88NRY3VfTHJdkucitFdzWZI3kgySXJHkT0lm\n+yyogG8m+WqE9lge+bvUySQfj45fSHJNj7VUcDbJ630XMeW+luXQXkryYZInk9zaZ0EFPJ/k/b6L\nmFZCe3X3Jnmm7yIob3eSt1a8fnv0HjSyNZsgTJeTSXaNef+BJMdHxw8m+SDJ41tV1BTbyOfF6oZ9\nF8D2shND+8A6f787yS1Jbtr8UkpY7/NibeeT7Fnxek+WZ9tABw5m+WmIq/supJjnkuzru4gpdXmS\nv2T5i8jPxheRGzWILyLZgHNJ3kxyZtQe7recqXdblu/X/ivJu0me7becqXVzktey/IXk/T3XUsET\nSd5J8p8s//+6p99yAAAAAAAAAAAAAAAAgKnwPzc/FITwdTU1AAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Rejection sampling: 8889 tries for 1000 samples\n" ] } ], "source": [ "q = multivariate_normal(mean=[0,0], cov=np.eye(2))\n", "sampling = RejectionSampling(p=p2d.pdf, q=q, c=p2d.pdf([0,0])/q.pdf([0,0]))\n", "\n", "def gauss_hist2d (ax, data):\n", " x = map(lambda x: x[0], data)\n", " y = map(lambda x: x[1], data)\n", " ax.hist2d(x, y, normed=True, bins=25)\n", "\n", "show_sampling(sampling, plotter=gauss_hist2d)\n", "print sampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With a much higher rejection rate as in the 1-dimensional example above.\n", "\n", "The next example shows that rejection sampling fails, i.e. samples from the wrong distribution, if the condition $c q(x) \\geq p(x)$ is violated:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEACAYAAAC57G0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVNX5x/HPsICwFAFBkOaKIIKIFEVUkKEKBKlS7YUf\nRhFFE1FTWGOKxlhDTCwJCQiCSBFQQ3UQEUEUFnQBgQWpKqKRJmVhfn+cAYZld+/0c2fm+3697mva\nvec+LDvPnvvcc88FEREREREREREREREREREREREREZG01RVYB2wARhWz3hVAPtAvgm1FRCTBMoCN\nQBZQClgFNCpivYXAbE4l+FC3FRGROCjh8HkrTJLeAhwFJgG9ClnvPuAtYHcE24qISBw4JfhawLag\n19sD7xVcpxfw98BrfxjbiohInDgleL/D5wDPA48E1vUEllC3FRGROCnp8PkOoE7Q6zqYnniwlpjy\nC0BVoBumJBPKtlx44YX+TZs2hRGyiIgAm4D60TRQMtBIFlAa5xOlY4G+YW7rTwajR4+2HUJIFGcE\ntm71+zt39vsbN/b7x43z+/fvP/nR6NGj/f4jR/z+997z+zt0MOt89JG9WAvhqp9lMRRnbBFClcSp\nRJMPDAfmALnAZGAtMCywRLKtiHvMmQMtW4LXCzk5cPPNUK7c6euUKgVdu8L8+fD449C7N4wZA35V\nIcXdnEo0AO8FlmAvF7Hu7SFsK+IO48bBww/D9OlwzTXO63s8cMMN0Ly5SfJffQV//rN5X8SFnHrw\nEuD1em2HEBLFGaLJk+Gxx8DnKza5FxrnhRfCokWwYIH5A2GZ9Z9liBRn4rmh6xEoJ4kkyIIFMGSI\nKblcemnk7Xz/PbRtCz//OQwfHrv4RELgMUeOxeZwJXhJL1u2QOvWpgffrl307W3eDFdfDf/5D3Tp\nEn17IiFSghcJdviwKcfceCOMHBm7dhctgkGD4LPP4LzzYteuSDGU4EWCPfIIrF8P06bF/sTo6NHw\n0UdmVE4JndqS+FOCFzlh2TLo1csMhaxePfbt5+fDtdeao4N77419+yIFKMGLABw6BC1amF72wIHx\n209urqnr5+RAzZrx248IoSV4HUtK6nv6abjoIhgwIL77adwY7r4bRoyI735EQqQevKS2bdugWTP4\n9FPIyor//g4dgqZN4fnnoXv3+O9P0pZKNCI33gj16sETTyRun7NmmRO6OTlQMpSLxUXCpxKNpLel\nS+GDD0yyTaQePaBaNRg7NrH7FSlAPXhJTX4/tG8Pt94KtxecIikBVqwwo3bWr4fy5RO/f0l56sFL\n+lq4EHbtMrND2nD55WbY5Asv2Nm/COrBSyry+830ASNGwODB9uJYv97MVbNpE1SoYC8OSUnqwUt6\neu892L8/vmPeQ9GwIXTqBH/7m904JG2pBy+p55pr4P774z/uPRS5ueZcwKZNqsVLTKkHL+lnyRL4\n+mvo1892JEbjxuZuUf/4h+1IJA2pBy+ppWdP6NbNzNHuFitXmrjy8szt/0RiQD14SS+5ubB8Odx2\nm+1ITte8OTRoAFOm2I5E0kwoCb4rsA7YAIwq5PNeQA6wEvgU6BD02RZgdeCz5dEEKuLo6afNnZXK\nlrUdyZkefBCeeUY36paEcirRZADrgU7ADuATYDCwNmidcsCBwPNLgelA/cDrzUBL4Pti9qESjUTv\n66+hUSNzMrNKFdvRnOn4cVOPf/nl2NxJStJeLEo0rYCNmJ74UWASpsce7EDQ8/LAdwXjcNiHSPRe\nfdWMmnFjcgdzE5CRI+HZZ21HImnEKcHXArYFvd4eeK+g3phe/XtA8FypfmA+sAIYGnmYIsXIzzc9\n43vusR1J8W6+2dz1adMm25FImnBK8KHWTmYAjYDrgfFB718DNAe6AfcCbcMNUMTRzJlmKuDLLrMd\nSfEyM83cOK+8YjsSSRNOc5nuAOoEva6D6cUXZXGgzXOAPcCuwPu7MbX5VoF1TpOdnX3yudfrxev1\nOoQlEuRvf0ue2+QNG2YuxPrd7+Css2xHI0nE5/Ph8/nC2sapPl4Sc5K1I7ATMxKm4EnWC4E8TG+/\nBTAl8F4m5iTtPsyJ2LnA44HHYDrJKpFbu9ZcKbp1K5QubTua0HTubGa4HDLEdiSSxGJxkjUfGA7M\nAXKByZjkPiywAPQD1mCGQr4ADAq8XwPTW18FLANmc2ZyF4nOSy/BXXclT3IHcxGWrmyVBHDDCBf1\n4CUyBw9C7drmzkl16jiv7xZHj5pzBnPmQJMmtqORJKUrWSW1TZ0KrVsnV3IHM13BnXfqZKvEnXrw\nkrw6dDBDI2+4wXYk4du8GVq1gu3bdbJVIqIevKSuvDxYswauv952JJG54AJTnpk923YkksKU4CU5\n/fvfZhRKMvd+b79dN+aWuFKJRpLPsWOmBzxrlvsvbirOgQPmJHFuLpx3nu1oJMmoRCOpaeFCqFYt\nuZM7QLly0LcvvP667UgkRakHLzFXsWIV9u37IW7tT8TDEvzE606nFSpUZu/e4iZAjaEPPzRXt37+\nOXjc8HWUZBFKD94Nv1FK8CnG/OLF5/+0AnvZxtnU4zu+55y47AM8JOx30u+Hiy6CCRPMqBqREKlE\nIymnD9PxQRyTe4J5PGYCsnHjbEciKUgJXpLKjUxggu0gYm3IEHM7v/x825FIilGCl6RRna+5gk9I\nuZHj9eqZZcEC25FIilGCl6QxkMnMpCc/2Q4kHoYMgYkTbUchKUYnWSXm4nWSdRmt+DW/Zx7XxaX9\nUxJ4kvWEb76Biy+GnTvdedNwcR2dZJWUUZ8N1GUrC+lgO5T4qF4drrhCUxdITCnBS1K4kQlMZiDH\nHG9ClsRUppEYU4lGYi72JRo/62nITbzOJ7TC/NqmWIkG4McfoW5d2LIFKldO/P4lqahEIymhJZ/i\nwc8nXGE7lPg6+2xzO79p02xHIilCCV5cbwBv8iYDcMcBZ5ypTCMx5IZvjEo0KSa2JRo/edSjD9PJ\nodmJPcSw/cJYKtEA/PSTmVnyyy/h3HPtxCBJIVYlmq7AOmADMKqQz3sBOZibbn8Kpw1zcNpWpFgt\n+ZR8SpJDks8cGaqyZaFbN5g+3XYkkgKcEnwGMAaTqBsDg4FGBdaZD1wGNAduA14JY1uRYvVnClPo\njzsONhOkf38zdYFIlJwSfCtgI7AFOApMwvTYgx0Iel4e+C6MbUWK4Q9K8GmkWzdYsQJ277YdiSQ5\npwRfC9gW9Hp74L2CegNrgfeAEWFuK1KoFnzGcUqw6mTtPU2ULQvXXQczZtiORJKc01UjoZ5pmhFY\n2gLjgYvDCSI7O/vkc6/Xi9frDWdzSVFpWZ45oX9/eOUVGDrUdiTiEj6fD5/PF9Y2Tt+c1kA2po4O\n8ChwHHiqmG02YcozDULcVqNoUkxsRtH42Uh9+jOFlbQouIcYtF8ci6NoTjh40Iym2bQJqla1G4u4\nUixG0azAJOosoDQwEJhZYJ0Lg3Zy4pu4J8RtxYKKFavg8XjitsRCc1YCsJLmMWkv6WRmQpcu8Pbb\ntiORJOaU4POB4cAcIBeYjKm1DwssAP2ANZhhki8Agxy2FcvM/VL9cVyil9blmRM0mkai5IZvj0o0\nCRbPe6YG9hBl+3420IABvFlIeSYW7TtxQYkGYP9+qFULNm+GKlVsRyMuo7loJCk1YxUe/Olbnjmh\nfHno1EllGomYEry4Tj+mMpV+uOMA0zKVaSQKbvgGqUSTYG4v0ayhCUN5lY+5Ki7tO3NJiQZOlWm+\n+goqVbIdjbiISjSSdOqzgXPYwzKutB2KO5QvD+3awbvv2o5EkpASvLhKb2bwNr3w61fzlN69NfmY\nRETfInGVPkxnOn1sh+Eu118P8+aZqYRFwqAEL65Rg100Yi3v0952KO5SrRo0awbz59uORJKMEry4\nxvXM4j26cZTStkNxnz59NPmYhE0JXlyjD9OZQW/bYbhTr14waxbk59uORJKIEry4QgX2cg1L+O/J\nuenkNFlZULs2LFliOxJJIkrw4grdeZfFtGUfFW2H4l59+mg0jYRFCV5coTczVJ5xciLBu+UiLHE9\nJXixrjSH6cp/mUlP26G42yWXQKlSsGqV7UgkSSjBi3UdWMjnNOFbqtsOxd08HpVpJCxK8GJdb2bo\n4qZQKcFLGDTZWBpy02RjJTjGDmrRhg/ZRP2Ytx8ZF002VtDx42byscWLoX6oPy9JRZpsTFzvSpax\nm2phJPc0V6IE9Oypi54kJErwYpXmnomAyjQSIpVo0pB7SjROt+aLtv1IubhEA3DkCFSvDmvXQo0a\ntqMRS2JVoukKrAM2AKMK+fxGIAdYDSwBmgZ9tiXw/kpgeQj7kjTSmFxKcVS35gtX6dLQrZtu5SeO\nnBJ8BjAGk+QbA4OBRgXWyQOuxST2J4BXgj7zA16gOdAq+nAllZyae8YNB5JJpndv1eHFkVOCbwVs\nxPTEjwKTgF4F1lkK/Bh4vgyoXeBzfXulULp6NQrdupl5afbutR2JuJhTgq8FbAt6vT3wXlHuBILv\nLeYH5gMrgKGRBCipqQ5byWILi2lrO5TkVKECtG2rW/lJsUo6fB7Omab2wB3ANUHvXQPsAqoB8zC1\n/MUFN8zOzj753Ov14vV6w9itJKNevM1senDM8VdQinSiTDNokO1IJAF8Ph8+ny+sbZzKJ62BbDg5\nh+ujwHHgqQLrNQWmBdbbWERbo4H9wDMF3tcomgRzwyia+XTkRUYw84yKX2zaj47LR9Gc8M030LCh\neTzrLNvRSILFYhTNCqABkAWUBgYCMwusUxeT3G/i9OSeCVQIPC8HdAHWOIctqa4Ke7icFcyjs+1Q\nklv16tCkCSxcaDsScSmnBJ8PDAfmALnAZGAtMCywAPwWqAz8ndOHQ9bAlGNWYU6+zgbmxjB2SVI/\n4x0W0JGfyLQdSvLTaBophhtGuKhEk2C2SzTT6MN0+jCeW+LSfvSSpEQDsHEjtGkDO3eaaQwkbWgu\nGnGdshykAwuZTQ/boaSG+vWhWjX4+GPbkYgLKcFLQnVmHp/Skh+oYjuU1KEyjRRBCV4SSnO/x4Fu\n5SdFUIKXhMkgnx7M5u2IhkZKkZo3h8OHITfXdiTiMkrwkjBt+JCt1GUbdW2Hklo8HpVppFBK8JIw\nmnsmjvr0UYKXM2iYZBqyM0zSz2YuoAez+YImcWg/lpJomOQJ+fnmwqdVq6BOHdvRSAJomKS4xmXk\nkE9JvuAS26GkppIloUcPzREvp1GCl4Q4VZ5xw0FjilKZRgpQgpeEUP09Abp0geXL4fvvbUciLqEE\nL3GXxWZqspOlXGU7lNSWmQkdOsA779iORFxCCV7irhdvM5OeHCfDdiipT2UaCaIEL3F36t6rEnc9\nesD8+fDTT7YjERdQgpe4qspumrGKBXS0HUp6OOccaNEC5s2zHYm4gBK8xFUPZjOPzhyirO1Q0ofK\nNBKgBC9xpcnFLOjVC2bNMhc/SVpTgpe4yeQA7Xmfd+luO5T0cv755mrWJUtsRyKWKcFL3HRhLsu4\nkv9R2XYo6UdlGkEJXuJIFzcVrmLFKng8nrgtFStWMbNLao74tBdKgu8KrAM2AKMK+fxGIAdYDSwB\nmoaxraSokhylB7OZSU/bobjOvn0/YCZLi8+yb98P0KQJZGRATk4C/2XiNk4JPgMYg0nUjYHBQKMC\n6+QB12IS+xPAK2FsKymqLYvJox7b0cyGVng8KtOIY4JvBWwEtgBHgUlwxu14lgI/Bp4vA2qHsa2k\nKJVnXOBEmUbSllOCrwVsC3q9PfBeUe4E3o1wW0khSvAucNVVsGsX5OXZjkQsKenweThnaNoDdwDX\nhLttdnb2yederxev1xvGbsVtrgAOkkkujW2Hkt4yMqBnTzNH/MiRtqORKPl8Pnw+X1jbOE3O3RrI\nxtTRAR4FjgNPFVivKTAtsN7GMLfVHZ0SLN53dHoSD/k8xq/5Q5z2kNx3dErEHbVOxv/OO/DUU/DB\nB3Hcn9gQizs6rQAaAFlAaWAgMLPAOnUxyf0mTiX3ULeVlOOnHzCVfrYDEYCOHc1Imm+/tR2JWOCU\n4POB4cAcIBeYDKwFhgUWgN8ClYG/AyuB5Q7bSgq7jBxKACtpbjsUAShTxtwIZPZs25GIBW64f5pK\nNAkWzxLB7/gNZfg9Dyf8pt6xbT9lSjQAEyfCG2+Y+WkkZYRSolGCT0PxTDBf0Jg7WMsyJfiiW090\ngv/f/6BuXdi5E8qXj+N+JZFiUYMXCVkjcqnI3pM1OnGJSpXg6qvh3Xed15WUogQvMdOPqUylX1z7\nphKh/v3hzTdtRyEJphJNGopXiWAlzRjBiyymXVzaP0UlGoc9nBn/nj1Qr54p05QrF8d9S6KoRCMJ\nU49N1OBrlpy8zk1c5ZxzoHVrMy5e0oYSvMREP6Yyg94cJ8N2KFKU/v1hyhTbUUgCKcFLTNzAW7zF\nDbbDkOL06QNz58KBA7YjkQRRgpeo1WEr9chjEe1shyLFOVGm0WiatKEEL1Hrx1Rm0pN8StkORZxo\nNE1acZpNUsTRQCYzmsdthyEnlTwxwuIM5wCbgJoeDwcjbL1Chcrs3ft9pMFJAqkHL1HJYjMXsomF\ndLAdipyUT1G389uDn2V0pjtvFrlOSLcElKSgBC9RGcCbTKWfyjNJ5E0G0B+NpkkHutApDcXyQpvP\naM5InmMR3uA9xKz9wulCJ4c9FNv+OXzHJi7kPHbxE5kRta/vrH260Eni6iLWU51vWExb26FIGPZQ\nlWVcSXc0mibVKcFLxAYymSn018VNSWgK/RmARtOkOpVo0lBsSgR+cmnMHfyLj7mq4B5i0H5xVKJx\n2INj+1XYQx71qM129lMh7Pb1nbVPJRqJm0tZQyYH+ZjWtkORCHzPOXzAtfRmhu1QJI6U4CUiA5nM\nZAbijoNAicQEbmQIE22HIXHkhm+nSjQJFn2JwM9G6tOfKaykRWF7iLJ9JyrROOwhpPYzOcB2atOQ\n9ezm3LDa13fWvliVaLoC64ANwKhCPr8YWAocAh4q8NkWYDWn34xbktzlrOA4JXRj7SR3kHLMpodO\ntqYwpwSfAYzBJPnGwGCgUYF19gD3AX8pZHs/4AWaA62iCVTcYxCTmMQg3HEAKNGYyBBuZILtMCRO\nnBJ8K2Ajpid+FJgE9Cqwzm5gReDzwigLpJASHGMwbzCRIbZDkRiYTycuZBMXkGc7FIkDpwRfC9gW\n9Hp74L1Q+YH5mD8AQ8MLTdyoIwvYSU3WnXEgJ8kon1K8yQCdbE1RTrNJRnsm5RpgF1ANmIep5S8u\nuFJ2dvbJ516vF6/XG+VuJV5uZjzjuMV2GBJDExnCP7mTP/ArdMDtXj6fD5/PF9Y2Tv+brYFsTA0e\n4FHgOPBUIeuOBvYDzxTRVlGfaxRNgkU6iqMc+9lObS7iS4dRF+4YJRJN++kwiuYUP3nUow/TyaFZ\nSO3rO2tfLEbRrAAaAFlAaWAgMLOo/RV4nQknL5ErB3QB1jjsT1ysL9P4kDZhDqkT9/PoZGuKCuV4\nrBvwPGZEzT+BPwHDAp+9DNQAPgEqYnr3+zAjbs4FpgXWKwlMCGxbkHrwCRZpD3IunXmVoUxhgNMe\nImo/dOrBO+wh7PYbkctcunA+X4Uwt5B68G4QSg/eDQU3JfgEiyTB1GQHa7iUWuzgEGWd9hB2++FR\ngnfYQ0Ttf8LlPMYfmUcXx/b1nbVPc9FIzAxhItPoG0Jyl2Q1ltu5nbG2w5AYUg8+DYXfg/Szmqbc\ny99YzLWh7CHM9sOlHrzDHiJqvzLfk0c9stjCj1Qqtn19Z+1TD15i4jJyqMA+PqSN7VAkjn6gCnPp\nwiAm2Q5FYkQJXhzdxr8Zz8349euS8lSmSS0q0aShcEoEpTnMdmrTiuVs4YJQ9xBy+5FRicZhDxG3\nX4JjbKUunZnHWhoX2b6+s/apRCNR680McrgsjOQuyew4GYzjFvXiU4QSvBTrLl7jNe6yHYYk0L+5\njZt4nZJFzh8oyUIJXoqUxWaasYoZ9LYdiiTQlzRkMxdwHXNshyJRUoKXIt3OWCYyhMOUsR2KJNhY\nbucO/mU7DImSTrKmoVBO8pXgGFvI4me8wxqahrsHx/ajo5OsDnuIuv3y7OMrzqcJn7OLmme0r++s\nfTrJKhHrwlx2UjOC5C6pYD8VmMxA7uSftkORKKgHn4ZC6UG+RT/mcB2v8n+R7MGx/ejEu/1SQH4c\n24dk+Pk0JYfZ9OACNnPstFtHqAfvBurBS0RqsZ32vM8bDLYdiiX5mAQZryU5rOYytlOb7rxrOxSJ\nkBK8nOH/eIWJDGH/yen8JV39g7u5m3/YDkMipBJNGiquRFOKI3zF+XRkQTFXMjruocj2Y0PtJ6r9\nMvzENupwBZ8EXeymEo0bqEQjYevHVHJpHEVyl1RyiLKM4xaG8qrtUCQC6sGnoeJ68Itpw7M8yHT6\nRrOHItuPDbWfyPYvYj2LaMf5fMURzkI9eHdQD17C0pQczucrZtLTdijiIl/SkDVcykAm2w5FwqQE\nLyfdy994mWEFhsSJwHOM5AGeJ5lGAUloCb4rsA7YAIwq5POLgaXAIeChMLcVl6jED/RnCq8y1HYo\n4kL/pSuZHORaPrAdioTBKcFnAGMwiboxMBhoVGCdPcB9wF8i2FZcYhgvM4vr+ZbqtkMRF/JTghe4\nn5E8B5TE4/HEbalYsYrtf27KcErwrYCNwBbgKDAJ6FVgnd3AisDn4W4rLlCKI9zHX3nmjAMwkVPG\ncQtt+JB6cb4QbN++HxL4r0ptTgm+FrAt6PX2wHuhiGZbSaDBvMEXXMJqLrMdirjYQcrxGncxwnYg\nEjKns2nRnFEJedvs7OyTz71eL16vN4rdSnj8PMQz/JKnbQciSWAMw1nNU/yWH9nL2bbDSSs+nw+f\nzxfWNk4JfgdQJ+h1HUxPPBQhbxuc4CWxOjMPD37m0sV2KJIEdlCb/2Kms/gLv7QdTlop2Pl9/PHH\nHbdxKtGsABoAWUBpYCAws4h1Cw64D2dbseQhnuEv/AJ3XPMmyeApYCTPcRaHbIciDpwSfD4wHJgD\n5AKTgbXAsMACUANTax8J/BrYCpQvZltxiWas5BK+SONZIyUSq4HPaMFt/Nt2KOLADd02TVWQYCem\nKphKXxbRjhe5P9Z7IJkuxVf74bd/FUuYwI00YEMcLozTVAih0FQFUqQmrOFqPtKFTRKRpVzNFrIY\nxCTboUgxlODT1K/4A8/yID+RaTsUSVJ/5DEe5U94OG47FCmCEnwaagh0YCF/5+e2Q5EkNp9OHCST\nPky3HYoUQQk+Df0KeJERumOTRMnDaB7ncUZTgmO2g5FCKMGnm3Xr6Iq5YEUkWu/Rjb1UVC3epZTg\n081vfsNfgB+pZDsSSQkefsUfyCabkmdMRyW2KcGnk08+gY8+4q+245CU4qM9W6nLrfzHdihSgMbB\np5POneGGG/DcfTfJPg5b7bur/dYsZTIDacCGwG39omtfOcGZxsHLKQsWwJYtcMcdtiORFPQxV7GK\nZgxnjO1QJIh68Ong+HFo3RoeeggGDiz2ptuxofbTsf2GrGMxbWnEWvZQNar2lROcqQcvxuuvg8cD\n/fvbjkRS2HouZhKDyCbbdigSoB58qtu3Dy6+GKZNgyuvBFAPXu3Hrf0q7GEdF9OORaylccTtKyc4\nC6UHrwTvQhUrVonZbcv+hJnu8/YzPknNBKP27bf/AM/RmXn8jHcjbL8UZjLa+KhQoTJ7934ft/YT\nRQk+ScWqh12fDSzlKi5lDV9zXvAeYtJ+0dR+OrdfiiN8ThMe5FneoUfM249eahwhqAaf1vw8x0ie\n5pcFkrtIfB2lNPfwEmMYTiYHbIeT1pTgU1Q/plKPPJ7nAduhSBpaQCeWcA2jcb6tnMSPSjQuFG2J\nphI/8AWXcANvsZSrC9tDVO07U/tqH87lG9ZwKZ2Yzxqaxrz9yKVPiUYJ3oWiTfCvcheHKMN9RV50\n4o4EoPZTv/2hvMId/Iur+Qh/yAUDJfhQxKoG3xVYB2wARhWxzouBz3OA5kHvb8HcwnElsDyEfUmU\n2rOQLszlMf5oOxQRXuMujlCaB3jedihSiAxgI5CFGbu0CmhUYJ3ucHI81JXAx0GfbQaqOOzDL6cD\n/OAPeynPXv9G6vl7MNNh3cjaD31R+2r/1HIBm/zfUtXfmM9dE38qMD+n4jn14FthEvwW4CgwCehV\nYJ2ecHIauWVAJaB60OduKAOlhRe4n/dpz2yutx2KyEmbqcej/Inx3EwpjtgOJ604JfhawLag19sD\n74W6jh+YD6wA3d05nvoylbYs1qGwuNI/uZMd1OK3/M52KGmlpMPnjocAAUX10tsAO4FqwDxMLX9x\niG1KiGqyg5e4h57M5ADlbYcjUggPQ3mVlTRnPp1YhNd2QGnBKcHvAOoEva6D6aEXt07twHtgkjvA\nbmA6puRzRoLPzs4++dzr9eL1eh3CkhNKcpQ3GcCLjGA5V9oOR6RI31CDW/kPExlCSz7VBXhh8vl8\n+Hy+sLZxqo+XBNYDHTHJejkwGFgbtE53YHjgsTXwfOAxE3OSdh9QDpgLPB54DBY4XyAnhDNM8llG\nUp+N9OJtVw1DU/tqvyi/5XE6sJCOLOBYoX1MDZMMRSyGSeZjkvccIBeYjEnuwwILmBE0eZiTsS8D\n9wTer4Hpra/CnHydzZnJXaLQnzfpyUxuYVwYyV3Ert/zaw5Rhj/ymO1QUp4bRrioB19AKD34FnzK\nf+lKF+ay6rRLD0Lag2P70VH7ar945/AdK7icX/N7JnBTzNsvXvr04J1q8OJCtdjO2/RiGC9HkNxF\n7NtDVX7GO7wfuGH3Yq61HVJK0nF9kinPPmbTgxe4n+n0tR2OSMRyuYQhTORNBtCAL22Hk5JUonGh\noko0Z3GId/gZG6nP3fyDyP/73H8Ir/bTp/07+CeP8ieu5QN2UTPm7Z9JJRopwpEjR9iwYUPC93ti\nOORuqnEPL+GOv80i0fsXd3Iu3zKfTnjxsdt2QClECT5MY8eOZcSIRylTJj5jeA8f3nPGexnkM45b\n8ODnZsZznIy47FvElid5lDIcYj6daA8k/w313EEJPkxHjx7F4xnC3r1FTcUbrRcg6CYdpTnMJAZx\nFofpx1QJGtdpAAAH6klEQVTyKRWn/YrYlU02ZTjEPNbQlW/Zzbm2Q0p6OsnqYmU5yEx6cowMejOD\nQ5S1HZJIHHl4hCeZBXxIG7LYbDugpKcE71I12MUi2rGL8xjEJI5S2nZIIgngIRszM+pi2tKUHNsB\nJTUleBe6DFjGlcygN7cztojLuUVS10vcy4M8yzw604sZtsNJWsocLjOQz/grcA/P8Bb9bYcjYs0U\nBrCZC5hKP1ryKdlka4BBmNSDd4myHOQVhvIE73IdKLmLACu4giv4hHYs4l26U4NdtkNKKkrwLtCc\nz1hOKzI5SAt+wUrbAYm4yLdUpyMLWMaVrKQ5fZhmO6SkoQRvUVkO8iSjeI9uPMUobuJ19lPGdlgi\nrpNPKUbzO/ownT/zMOO5iep8bTss11OCt8LP9cwkh8uoy1aasprXuRldnSpSvI+5imasYic1WcOl\nDOevZJBvOyzXUoJPsJas4H3a80ce4z7+yhDe4NvT7lEuIsU5QHlG8WfasYg+TCeHywJlm+SfXybW\nlOAT5Bo+ZBY9mElPJnAjzVjFHLraDkskaa2lMR1ZwC95mt/wBJ9wBdczEw/HbYfmGkrwcVSKI/Rl\nKh/Qlv9wK7O4nnrk8RpDNbZdJCY8vEd3WvIpT/IIv+EJ1tOQexlDOfbbDs46Jfg4aMg6nuJhtlKX\nEbzIGIbTkPW8wjAO6ySqSMz5KcFUbqAVy7mdsXjxsZW6vMpdXMuitO3VqxsZE36aspp+TKUv06jC\n97zOTVzLB2zgItvBiaQRD0towxLacB47GcJExjCcCuxjGn2ZTQ8W2w4xgULpwXcF1gEbgFFFrPNi\n4PMcOO0ecqFsm5RqsZ2bGcdYbmMrdZlOHzI5yFBepTbbGcWfldxFLNpFTZ7hFzRlNT2ZyQ9U5kke\n4RuAAQPgpZfg88/heOr27p3G5WUA64FOwA7gE2AwsDZone7A8MDjlZj5bluHuC24/Y5Ofj98/TW+\n8ePx5uezafJkSq/OowxleJ/2LKQDC+nABhoQm2GOJ6YLjvRn4gO8Duu44Y4/PpzjjKb9aAS37yPy\nOENpPxZ8nB6jG/5/C+MjtJ9lfOOvgYddY8fCBx+Y5YcfoHVraN4cmjXDd/gw3sGDoYS7K9ixuKNT\nK2AjsCXwehLQi9OTdE/gP4Hny4BKQA3gghC2dYf9+2HHjlNLXh58+SWsX28ey5TBl5mJt39/NrRo\nwch1TVl3ZBzuHLfuI/YJKR58KM5Y8eH+GMEtcX4NcNttZgHYuRM+/hhycmD8eHw+H95hw6B+fbjw\nQvNYvz7Uqwc1a8J558HZZ4PHjd//0zkl+FrAtqDX2zG9dKd1agE1Q9g2OseOwU8/wcGD5jH4+YnH\nffvMX+jClt27TUI/ehRq1Tq1ZGXBddfBfffBRRdBlSqQnQ3Z2WwcM4bNb6zDncldRMJWsyb07WsW\nMN/1Bx6ATZvMsnEjLF0KEybArl3mD0J+vkn0550HVatCpUpQubJ5PPH87LOhXDkoW9YsmZlnPi8Z\n39OgTq2HepwUXba7/HLzAzt2zDyG8vzYMbME/9AK+yGWL29+2JUrm+TdpMmp11WrmvcqVQr5r3GJ\nEiXweGZTseJXUf2Ti3LkSB6HDsWlaREJVaVK0LKlWQqzf79J9rt2wZ498L//nVry8k49L6zzGfwc\nTJI/sZQqdfrrgu9lZJjSUYzKR62B/wa9fpQzT5b+AxgU9HodUD3EbcGUcfxatGjRoiWsZSNRKgls\nArKA0sAqoFGBdboD7waetwY+DmNbERGxqBtmNMxGTC8cYFhgOWFM4PMcoIXDtiIiIiIikioeAo4D\nVWwHUoQnMEcpq4AFQB274RTpacxw1BxgGnC23XAK1R/4AjjG6Ud9bpEMF+n9C/gGWGM7EAd1gPcx\n/9+fAyPshlOkMpih3quAXOBPdsMpVgawEphlO5BQ1cGclN2MexN8haDn9wGv2QrEQWdOXaX8ZGBx\nm4uBizBffLcl+AxMWTELKIV7zx+1xVw57vYEXwNoFnheHlO2dePPEyAz8FgScz6xjcVYivMgMAGY\nWdxKbrpU61ngYdtBONgX9Lw88J2tQBzMg5OzKy0DaluMpSjrgC9tB1GE4Av8jnLqIj23WQz8YDuI\nEHyN+SMJsB9zdFnTXjjFOhh4LI35Q/+9xViKUhszuOU1HIaouyXB98JcCLXadiAh+AOwFbgVd/aM\nC7qDU6OcJDRFXbwn0cvCHHUssxxHUUpg/hh9gzm6zLUbTqGeA34JzlNkJnI2yXmYQ7WCfoUZYdMl\n6D2bl4kWFedjmHrXrwLLI5gf9O2JC+00TnGCifMIMDFRQRUQSoxu5LcdQIoqD7wF3A+unaz9OKac\ndDYwBzO3gs9iPAX1AL7F1N+9dkMJTRPMX8vNgeUo5tD4XIsxhaIu5oSRW90GLAHXT0Dvxhp8qBfp\nuUEW7q/BgzmXMQczk16y+A3wC9tBFPBHzNHlZmAXcAAYZzWiMLn5JGuDoOf3AeNtBeKgK2bEQlXb\ngYTgfaCI68GtSaaL9LJwf4L3YJLQc7YDcVAVM1kiQFngA6CjvXActcPdR8KFysO9Cf4tzJdpFTAV\n9x5lbAC+whzGrQReshtOofpgeiI/YU7CvWc3nDMkw0V6bwA7gcOYn6WtcqGTNpjSxypO/U668YbE\nlwKfYeJcjalzu1k7HEbRiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIhY9/+5lvxyPhT/7gAAAABJ\nRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Rejection sampling: 1359 tries for 1000 samples\n" ] } ], "source": [ "sampling = RejectionSampling(p=p, q=norm(loc=0, scale=2), c=2)\n", "\n", "show_sampling(sampling, plotter=gauss_hist)\n", "print sampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Importance sampling\n", "\n", "Importance sampling tries to fix the inefficiency of rejection sampling, by assigning weights to each sample instead of rejecting. It is based on the observation that\n", "$$ \\mathbb{E}_p[f] = \\int f(x) p(x) dx = \\int f(x) \\frac{p(x)}{q(x)} q(x) dx = \\mathbb{E}_q[f \\frac{p}{q}] $$\n", "for any function $f(x)$.\n", "\n", "Thus, instead of using samples $x_1,\\ldots,x_N$ from $p(x)$ to approximate the expectation $\\mathbb{E}_p[f]$ of $f$ under $p$ as\n", "$$ \\mathbb{E}_p[f] \\approx \\frac{1}{N} \\sum_{i=1}^n f(x_i) $$\n", "we can just as well use samples from some other distribution $q(x)$ and evaluate the expectation of $f \\frac{p}{q}$ under $q$.\n", "\n", "This scheme can easily be extended when the normalization constant of $p(x)$ is not available, i.e. $p(x) = \\frac{1}{Z} \\hat{p}(x)$. Then,\n", "$$ \\mathbb{E}_p[f] = \\int \\frac{1}{Z} f(x) \\hat{p}(x) dx = \\frac{1}{Z} \\mathbb{E}_q[f \\frac{\\hat{p}}{q}] $$\n", "and since $\\int p(x) dx = 1 = \\frac{1}{Z} \\int \\frac{\\hat{p}(x)}{q(x)} q(x) dx$, $Z$ can be found via importance sampling as well:\n", "$$ Z = \\int \\frac{\\hat{p}(x)}{q(x)} q(x) dx = \\mathbb{E}_q[\\frac{\\hat{p}}{q}]$$\n", "\n", "Thus, assuming that we have samples $x_1,\\ldots,x_n$ from $q$, the expectation $\\mathbb{E}_p[f]$ can be approximated as\n", "$$ \\mathbb{E}_p[f] \\approx \\frac{\\sum_{i=1}^N f(x_i) w_i}{\\sum_{i=1}^N w_i} $$\n", "with $w_i = \\frac{\\hat{p}(x_i)}{q(x_i)}$ being the **_weight_** of sample $x_i$" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class ImportanceSampling (Sampling):\n", " def __init__(self, log_p, q):\n", " \"\"\"\n", " q is assumed to support sampling q.rvs and density evaluation q.pdf\n", " \"\"\"\n", " self.log_p = log_p\n", " self.q = q\n", " \n", " def sample (self):\n", " \"\"\"\n", " Note: Importance sampling returns a sample and its associated weight\n", " \"\"\"\n", " x = self.q.rvs()\n", " w = np.exp(self.log_p(x) - self.q.logpdf(x))\n", " return x,w" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def weighted_gauss_hist (ax, weighted_data):\n", " x = np.arange(-4,4,0.01)\n", " ax.hist(map(lambda x: x[0], weighted_data), normed=True, \\\n", " weights=map(lambda x: x[1], weighted_data))\n", " ax.plot(x, phi(x), 'r-')" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEACAYAAAC57G0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUFNXd//F3MyP7oCAoCsggSxBkkziCiPSwOYCABKMS\njTFumF8wy2N4kF8WJ7944mOiiTEkPmqMRkJkcUGC7DLtggiyIwOyDgJiBEQBRZylf3/cHmiGmem9\nb1XX53VOHbq76vb9Mmf6O7e/desWiIiIiIiIiIiIiIiIiIiIiIiIiIh4VgGwBdgGTKrluMuBMmBs\nHG1FRCTNsoDtQC5wFrAOuKSG45YCczmV4KNtKyIiKVAnwv48TJIuAUqB6cDoao67F3gROBBHWxER\nSYFICb4VsCfs+d7Qa1WPGQ08EXoejKGtiIikSKQEH4ywH+Ax4P7Qsb7QFm1bERFJkewI+/cBbcKe\nt8GMxMP1xpRfAJoDwzAlmWja0r59++COHTtiCFlERIAdQIdE3iA79Ca5QF0inyh9FvhWjG2DYjzw\nwAO2Q7DjhReCwebNg8GHHgoGjx4NBoOhn0VZWTA4Y0Yw2KZNMDhxYjBYXm43Tks8+3tRDf0sTiGK\nKkmkEk0ZMAFYCBQDM4DNwPjQFk9bkVOeegomToQlS+D++6Fx41P7srLghhtg3Tp45x247TaoqLAW\nqojbRCrRAMwPbeGerOHY70fRVsR45RX4zW+gqAg61PJNs1kzWLQIhg0zfwR+97v0xSjiYpFG8JJG\nfr/fdgjpU1wMd99tknw1yf2Mn0XDhvDyyzB7Nkyblp4YHcJTvxcR6GcRG1/kQ1IuVE4Szygrgz59\nYPx4uOuu2NquXQtDh8KqVdC2bWriE3EBn88HEXK4RvCSfo88Ysoud94Ze9teveC++8wfBg0MRGql\nEbyk186dkJcHq1fHPwIvLYWePeHBB2HMmOTGJ+IS0YzgleAlvW66CS69FH7xi8TeZ+lSuOMOU8tv\n0CA5sYm4iBK8OMvKlfCtb8HWreakaaLGjoUrroD//u/E30vEZZTgxVny8+Hmm+OrvVdn0yYYOBB2\n7Dh9/ryIB+gkqzjHsmVQUmIuVkqWrl3NH42//CV57ymSQTSCl/S49lqz3XNPct+3uNgk+V27klP2\nEXEJjeDFGdavhzVrkjt6r9Sli5lT//zzyX9vEZdTgpfUe+wxuPdeqF8/Ne//05+aPrROjchplOAl\ntQ4dMssLJOvEanUGDDBTJRcsSF0fIi6kBC+p9eyzMHIktGiRuj58PvjJT+Dxx1PXh4gL6SSrpE5F\nBXTsaBYH69MntX0dPw6tW5tav9aoEQ/QSVaxa9EiOOccczFSqjVoAOPGmW8MIgIowUsqPfOMWRLY\nl6YvinfdBX//O5SXp6c/EYdTgpfU+PRTWLwYbrwxfX326AHnn2/6FREleEmRGTPgmmtMiSad7rzT\nfHMQEZ1klRTp2xd++UsYPjy9/R4+DLm5sGcPNGmS3r5F0ihZJ1kLgC3ANmBSNftHA+uBtcBqYGDY\nvhJgQ2jfyij6kkzwwQdm3ZmhQ9Pfd9Om4PebWwGKeFykBJ8FTMEk+S7AOOCSKscsAXoAvYDbgKfC\n9gUBf2hfXsLRijtMm2ZmtGRHc0/3FBg3Dl54wU7fIg4SKcHnAdsxI/FSYDpmxB7ui7DHjYGDVfY7\noQwk6RIMwsyZ5sYetowcCe++C598Yi8GEQeIlOBbAXvCnu8NvVbVdcBmYD7wo7DXg5gR/iogxrsr\niyu9/z589RVcfrm9GBo1ghEjYNYsezGIOECkBB/t2c/ZmNLNSGBq2Ov9MOWZYcAPgf6xBiguM3Mm\nXH99+ua+10RlGhEiFUn3AW3CnrfBjOJr8lboPc8FDgH7Q68fAF7BlHzeqtqosLDw5GO/34/f748Q\nljhSMGhGzU5YunfoUPje92DvXrOEgYjLBQIBAoFATG0iDbOygQ+AQcBHmJkw4zDlmErtgZ2Y0f5l\nwKzQaw0xJ2mPAo2ARcCvQ/+G0zTJTLFxo6l/79plfwQPcOutkJcHEybYjkQk6ZIxTbIMmAAsBIqB\nGZjkPj60AYwFNmKmQv4JqDy71hIzWl8HrADmcmZyl0zilPJMpTFjNF1SPM0Jn0SN4DNFly5mLZhU\nrxwZrS+/hJYtzTeKc8+1HY1IUmk1SUmfbdvMVaR5DrrcoWFDGDQI5s61HYmIFUrwkhz//repv9dx\n2K/UmDHmjlIiHuSwT6O41pw5MGqU7SjOdO21sHSpKdeIeIwSvCTu00/NnZQGDbIdyZmaNTMXXS1c\naDsSkbRTgpfEzZ8P+fnmrkpONHIkvPaa7ShE0k4JXhLn1PJMpREjYN48cyGWiIcowUtivv7a3Hv1\n2mttR1KzDh0gJwfWrrUdiUhaKcFLYt58Ezp3NrfKc7Lhw80oXsRDlOAlMXPnmhKI040YoTq8eI4S\nvCRm4UIYNsx2FJH17w/FxXCw6u0KRDKXErzEb/duOHQIevWyHUlk9erBwIGwYIHtSETSRgle4rdw\noVmW12lXr9ZEdXjxGJd8MsWRFiyAa66xHUX0hg83f5TKymxHIpIWSvASn9JSswTA0KG2I4leq1Zm\nW7XKdiQiaaEEL/FZsQIuvtj50yOrGjoUFi+2HYVIWijBS3zcVp6pNGSIuTBLxAOU4CU+CxdCQYHt\nKGLXvz+sWwdHj9qORCTllOAldgcOwNat0Lev7Uhi17AhXHEFxHjzYhE3UoKX2C1eDH4/1K1rO5L4\nDB2qMo14QjQJvgDYAmwDJlWzfzSwHnPT7dXAwBjaihstWODO8kylIUN0olU8IdJNt7OAD4DBwD7g\nPWAcsDnsmEbAF6HH3YBXgA5RtgXddNtdgkEz1fCtt6B9e9vRxKeiwtyMe9UquOgi29GIxCUZN93O\nA7YDJUApMB0zYg/3RdjjxkDlYh/RtBW32boVsrPNFEm3qlMHBg/WKF4yXqQE3wrYE/Z8b+i1qq7D\njMznAz+Ksa24SVGRuXuTL9KXP4dTmUY8IDvC/mhrJ7NDW39gKtA5liAKCwtPPvb7/fj9/liaSzoV\nFblj9chIhgyBiRNNucYta+mIpwUCAQIxzv6KNAzrAxRiTpYCTAYqgIdrabMDU57pGGVb1eDdIhg0\ntesVKyA313Y0ievSBaZOhd69bUciErNk1OBXYRJ1LlAXuBGYU+WY9mGdXBb691CUbcVNNm8288gz\nIbmDyjSS8SIl+DJgArAQKAZmYGrt40MbwFhgI2aa5J+AmyK0FbeqrL9nivx8838SyVBOOFOmEk2c\nmjRpxtGjh9PW3yzgVeDVnKYcOfJp2vpNmcOHoW1bc5cnt160JZ6VjBKNOJhJ7sG0bD7KGUBzAnyY\n1j8qKdW0KXTsCCtX2o5EJCWU4CUqXdnE55zNXtoA2fh8vrRuTZo0S81/bOBAlWkkYynBS1TyKaKI\nyvp7Gen65lC5pexbQ36+uXGJSAZSgpeonJ7gM0j//vDee3D8uO1IRJJOCV4i8lHB1bxJAL/tUJIv\nJwe6dYPly21HIpJ0SvASUXc2cJDm7OdC26GkhqZLSoZSgpeIMrY8U0kJXjKUErxElE9RZpZnKvXr\nZ27jd+yY7UhEkkoJXmpVh3L681ZmJ/iGDeGyy2DZMtuRiCSVErzUqifr2M8F/IeWtkNJrYEDNV1S\nMo4SvNQq4+vvlVSHlwykBC+1yvj6e6U+faC4GD7/3HYkIkmjBC81yqKMq3jbGwm+Xj2T5N9803Yk\nIkmjBC81uow1fMhFHKSF7VDSQ8sWSIZRgpcaeab+Xkl1eMkwSvBSIz8Bb5RnKl1+OezcCYcO2Y5E\nJCl0ww8XMwv+p+Znl00phziXduziU86t2nPK+q3ZWZhVLFNrPvAk5g7yOZlyYxPJSLrhh8Ttm6xi\nJxdXk9xtSc8SxQEeIp97SekSxSJpogQv1fJc/T2kiHz8BGyHIZIU0ST4AmALsA2YVM3+m4H1wAZg\nGdA9bF9J6PW1gO6L5iKeq7+HrKY3bdlNcw7YDkUkYZESfBYwBZPkuwDjgEuqHLMTuBqT2H8DPBW2\nLwj4gV5AXuLhSjqcxdf0ZTlvcrXtUNKunGyW0Y8BvGE7FJGERUrwecB2zEi8FJgOjK5yzHKg8vK/\nFUDrKvudcCJXYpDHSrbSic9oajsUK1SmkUwRKcG3AvaEPd8beq0mdwDzwp4HgSXAKuCueAKU9PNq\n/b1SAD/5aD68uF92hP2xzIXLB24H+oW91g/YD7QAFmNq+W9VbVhYWHjysd/vx+/3x9CtJJufAH/k\np7bDsGYtvWjFPq9cvysuEQgECAQCMbWJVD7pAxRiavAAk4EK4OEqx3UHXg4dt72G93oAOAY8WuV1\nzYOPUyrmwdflBAdpTmv2coSza+o56f1Glt4+5zCSqcxlpn43xaGSMQ9+FdARyAXqAjcCc6occxEm\nud/C6cm9IZATetwIGApsjBy22NSHd9nMJbUkd28wZRoRd4tUoikDJgALMTNqngE2A+ND+58EfgU0\nBZ4IvVaKOTnbEpP4K/uZBixKVuCSGl6dHllVEfncaTsIkQQ5YYaLSjRxSkWJpgg/DzOJBQyrreek\n9xtZevusQzkHyabpRx/BBRekrV+RaGmpAolJfY7zTVbxNlfZDsW6CrJ4E+ANzYcX91KCl5P6spyN\ndOPYyVMn3lYEWj5YXE0JXk5S/f10AVCCF1dTgpeTvH6BU1UbwKwNv2+f7VBE4qIELwA04Et6sZZl\np12n5m1BgKuvhhgvLhFxCiV4AaAfy1hHT76kke1QnCU/XwleXEsJXgDV32uk+7SKiynBC6D6e426\ndoXPPoM9eyIfK+IwSvBCI47RnQ28w5W2Q3GeOnXA71eZRlxJCV64irdZTW++ooHtUJzJ71eZRlxJ\nCV5Uf49EJ1rFpZTgRfX3SLp0gWPHYPdu25GIxEQJ3uNyOEJXNvEufWyH4lw+n8o04kpK8B53FW+z\nkjxOUN92KM6mMo24kBK8x+VTpPp7NCpH8FraWlxECd7jBrKUpQy0HYbzde4MX38NJSW2IxGJmhK8\nh53DYTqxlZXk2Q7F+VSHFxdSgvewAbzBO1xJKXVth+IOSvDiMkrwHqbpkTGqPNGqOry4RDQJvgDY\nAmwDJlWz/2ZgPWb57GVA9xjaikWqv8eoY0coL4cdO2xHIhKVSAk+C5iCSdRdgHHAJVWO2QlcjUns\nvwGeiqGtWNKCT2jDHtZwme1Q3MPn03RJcZVICT4P2A6UAKXAdGB0lWOWA5+HHq8AWsfQVizxE+At\n+lNOtu1Q3EXLB4uLRErwrYDwdVL3hl6ryR3AvDjbShqpPBMnzYcXF4k0fIvltzgfuB1O3vMt6raF\nhYUnH/v9fvx+fwzdSjzyKeJ/ucd2GO7Tvj1kZcG2bdCpk+1oxEMCgQCBGMuDvgj7+wCFmDo6wGSg\nAni4ynHdgZdDx22PsW0wqNFQXHw+H7H9DTYuZB8b6E4LDhCMayJVfP0mxk6f1f5u3nor9OsH48en\nOR6RU8znv/YcHinBZwMfAIOAj4CVmJOlm8OOuQhYCtwCvBtjW1CCj1u8Cf5m/skYXuF6Xoq357j6\nTYyNPs8Cys549fvAEOA7KegxJ6cpR458moJ3lkwTTYKPVKIpAyYACzGzYp7BJOjKocuTwK+ApsAT\noddKMSdYa2orlqn+Hq0yqvujUsQufktfYD+Rx0ixOXo0ue8n3uaE3yaN4OMU7wh+J+0YwWtspku8\nPcfVb2Kc1GeQEnIpYAFbkj7zt4aykEgV0YzgdSWrx+SyiwYcZ7MuSUiAjyLy8ROwHYhIrZTgPebU\n8gRO+PLmXkXkk4/mw4uzKcF7jOrvyRHAHxrBq5wizqUE7ylBBrJUC4wlwYe05RiN6UKx7VBEaqQE\n7yGd2EoZ2eygve1QMkIAv8o04mhK8B4yhMW8ziBUf08OnWgVp1OC95AhLGYxQ2yHkTEq6/A+KmyH\nIlItJXiPyKaUAbzBEgbbDiVj7KUNh2lKNzbaDkWkWkrwHpHHSnbRjgOcZzuUjLKYIQxhse0wRKql\nBO8RKs+kxiKGMpRFtsMQqZYSvEcowadGEfn0ZTn1OW47FJEzKMF7QBM+pzsbeJurbIeScY5wNuvp\nQX/esh2KyBmU4D0gnyLepQ9f0cB2KBlJZRpxKiV4D1B5JrWU4MWplOA9QAk+tVbxTVqzl5bstx2K\nyGmU4DPcRezmHD5jPT1sh5KxyslmKQM1XVIcRwk+ww1hMUsYHOe9VyVaKtOIE+lTn+FUnkmPxQxh\nMEu0bIE4ihJ8BqtDOYN4XQk+DUpox1FytGyBOEo0Cb4A2AJsAyZVs78zsBz4Crivyr4SYAOwFlgZ\nd5QSlzxW8hEXso/WtkPxBJVpxGkiJfgsYAomyXcBxsEZN/M8BNwLPFJN+yDgB3oBeYkEKrEbxnzm\nM8x2GJ6xiKFcw0LbYYicFCnB5wHbMSPxUmA6MLrKMQeAVaH91dHi45YMZx7zGG47DM9YykCuYAWN\nOGY7FBEgcoJvBewJe7439Fq0gsASzB+Au2ILTRJxPh/Tnh28w5W2Q/GMY+SwgisYzBLboYgAkB1h\nf6J3FO4H7AdaAIsxtfwzFu0oLCw8+djv9+P3+xPsVgpYwBIGU8ZZtkPxlNcYwQhe41Wusx2KZJhA\nIEAgEIipTaTySR+gEFODB5gMVAAPV3PsA8Ax4NEa3qum/cFgUHemj4fP56Omv8EzuIH5DOM5vp+K\nnmvsN3Xc0WdHtlJEPq3ZS3zVSR/6PEg0zOe/9l+ySCWaVUBHIBeoC9wIzKmpvyrPGwI5oceNgKGg\nOWTpkEUZg1nCgpN/lyVdttGJL2lIT9bZDkUkYommDJgALMTMqHkG2AyMD+1/EmgJvAc0wYzuf4yZ\ncXMe8HJYP9NAc8jSoS/LKSGXj7nAdiie9BojGM481tHLdijicU6Y4aISTZxqKtH8lsmUk8UveTBV\nPVfbb2q5p88hLKKQQvrxTlx96vMg0UhGiUZcSNMj7XqDAXRlE+dy0HYo4nFK8BnGXLe6lxVcYTsU\nz/qaehSRTwELbIciHqcEn2GuZS4LKKCCLNuheFrldEkRm5TgM8x1zObVMy42lnSbx3CuYSFZlNkO\nRTxMCT6D5HCEfizT9EgH+IhWlJBLP5bZDkU8TAk+gxSwgGX04yhNbIciwGyu4zpm2w5DPEwJPoOM\n5lWVZxzkFcYwhldI//ROEUMJPkNkU8ow5jOHUbZDkZD3uZRysnRVq1ijBJ8hBvAG2+jIfi60HYqc\n5AsbxYuknxJ8hriO2czWCoaOowQvNinBZ4Qgo5ij+rsDvUsfWnCA9my3HYp4kBJ8BriMNZygHpvP\nuJui2FZBFnMYpdk0YoUSfAa4nhd5ibE4Y+04qUplGrHFCRlBq0nGyawmV8F2OvBtZrGWy9LVM25Z\n2dEJfdblBB/Tki4UR7GEs1aTlOhoNUkP6MVagvhYq7XHHetr6vEaIxjLS7ZDEY9Rgne5G5jJTG7A\nGV/GpCbTuYmbmG47DPEYJ2QFlWji5PP52EE7xvJSmu8e5N5yia0+z+Jr9nMBPVnHXtrU2qc+DxIN\nlWgyXG+gnCzW0dN2KBJBKXV5hTHcwEzboYiHKMG72LdB5RkXUZlG0i2aBF8AbAG2AZOq2d8ZWA58\nBdwXY1uJVzDIDcAsvm07EolSAD8X8aEuepK0iZTgs4ApmETdBRgHZ1xNcwi4F3gkjrYSr3fe4Tiw\nnh62I5EolZPNi1zPjcywHYp4RKQEnwdsB0qAUmA6nHE9/AFgVWh/rG0lXlOnMhVQecZdpnOTEryk\nTaQE3wrYE/Z8b+i1aCTSVmpz4gTMmsU023FIzJbRj6Ycpivv2w5FPCA7wv5E5mtF3bawsPDkY7/f\nj9/vT6BbD3jtNejenT2BgO1IJEZB6vBPbuFWnmcSv7MdjrhIIBAgEONnPtL3+z5AIZy8yedkoAJ4\nuJpjHwCOAY/G2Fbz4GM1ZgyMHInvjjuwc7cgd89Jt93nN9hCEfm0YQ/lZ4yxNA9eopOMefCrgI5A\nLlAXuBGYU1N/CbSVaB06BEVFcP31tiOROH1AZ0rIZSiLbIciGS5Sgi8DJgALgWJgBrAZGB/aAFpi\nau0/BX4BfAg0rqWtJGLmTCgogCa6sbabPcdt3MZztsOQDOeEKRgq0cTiiivggQdg+PDQVzSVaNzY\n59l8Rgm5XMxODtPstD71eZBoaKmCTLNuHezfD9dcYzsSSdDnnMN8hunKVkkpJXg3efppuOMOyMqy\nHYkkwXPcxu383XYYksFUonGLL76ANm1g/XrzL6hE4/I+61DODtozlpdYQ++TferzINFQiSaTzJoF\n/fqdTO7ifhVk8STj+QFP2A5FMpRG8G5x5ZVw//0watTJlzSCd3+fLfiED/gGF7OTz2iKRvASLY3g\nM8XGjbB7NwwfbjsSSbIDnMd8hnErz9sORTKQErwbTJkCd98N2ZFWlhA3eoIfhMo0GrlLciljON2h\nQ+bipi1bbEciKfI2V1HKWeRTRJHtYCSjaATvdE8/DdddB+efbzsSSRkff+GH/IjHbQciGUYnWZ2s\ntBTatYO5c6Hnmfdd1UnWzOmzAV9SQi79OcAH+jxIFHSS1e1eegk6dKg2uUtmOU5DnuAHZ9zzUiQR\nGsE7VTAIeXnw85+bEk01NILPrD6bc4CtnEfTjz9WSU4i0gjezZYsgS+/PG3eu2S2g7QwK9NMmWI7\nFMkQGsEnSZMmzTh69HDS3m8p8AxEcVs+jeAzqc8O+NjWvDmUlECjRmntW9xFI/g0Msk9mJStL8to\nSzumUxrhWMk02wH8fnhCyxdI4jSCT5Jk1sP/zbXM5Vqe5J5IvSatz9h4YzRtq8/ghg0wZAjs2KFR\nvNRII3gX+ibv0ZN1PMdttkMRW7p1g6uvhr/8xXYk4nIawSdJskbwixnMTG7gae6Optek9Bk774ym\nrYzgg0HYtAkGDjSj+MaN0xyDuEGyRvAFwBZgGzCphmMeD+1fD/QKe70E2ACsBVZG0ZenDeR12rKb\nZ/m+7VDEtq5dTYJ/XFe3SvwijeCzgA+AwcA+4D1gHKffPHs45ubaw4ErgD8BfUL7dgG9gU9r6UMj\neACCvEsf/shPmcFN0faaYJ/x8s5o2toIHmDbNujbF4qL4bzz0hyHOF0yRvB5mBP7JUApMB0YXeWY\nUcA/Qo9XAOcA4VdpOKEM5HhjeYl6nGAmN9gORZyiY0e45RYoLLQdibhUpATfCtgT9nxv6LVojwkC\nS4BVwF3xh5nZ6nOcR/gZ/8UfCOq8t4T71a/M3byKi21HIi4UKZtE+/20plH6VZia/DDgh0D/KN/P\nUybye97jcooYaDsUcZpmzWDyZPjZz2xHIi4UaT34fUD4TUDbYEbotR3TOvQawEehfw8Ar2BKPm9V\n7aQw7Cuo3+/H7/dHCCtzXMRufsyfuIw1tkMRp5owAZ56CmbPrnFdIsl8gUCAQCAQU5tI9fFszEnW\nQZhkvZLaT7L2AR4L/dsQc5L2KNAIWAT8OvRvOE+fZH2RsWykG7+mMJ5e4+ozcd454Wn1JGu4QAC+\n+11TqsnJSXNM4kTJOMlahkneC4FiYAYmuY8PbQDzgJ2Yk7FPAv8n9HpLzGh9Hebk61zOTO6eNpYX\n6com/of7bYciTuf3w+DBpiYvEiUnzHDx5Ai+GYd4n0sZy0ss58p4e42pz+TxzmjaMSN4gIMHzfz4\nuXPh8svTG5Y4jpYqcDAz3/3GBJK7eE7z5ubCp1tugS++sB2NuIBG8EkSywj+embxEJPpwXq+JJHF\npDSCz8Q+I34ebrkFmjSBv/41PSGJI0UzgleCT5JoE3w7dvIufRjOPFbzzUR7jarP5PNOsnVkgv/s\nM+jRwywpPHx4esISx1GJxmHO4mteYBy/5f8mIbmLZ51zDkydCrffbm4MIlIDjeCTJJoR/J+ZQFt2\nM4o5JOdHrxF8JvYZ9efhD3+AadPg7behQYPUhiWOoxJNGkVK8PfwBPfyZ/qynCOcnaxea+0zdbyT\nbB2d4INB+M53oF49ePZZ8Dnh4yzpogSfRrUl+IG8zjRuph/L2En7ZPZaY5+p5Z1k6+gED2Y2Tf/+\nMGYM/PKXqQtLHCeaBB9pqQJJ0GWs5gXGcQMzk5zcRTC39Js3D668Ei68EO64w3ZE4iBK8CnUhU28\nxgju5inewG87HMlULVvCggUwYAA0bQrf+pbtiMQhlOBT5BKKWcg13MejvIoWiJIU69TJjOSHDYPS\nUrjxRtsRiQOoBp8k4TX4PFbwKqOZyO/5J99NZa+oBp9pfZ6FWQIqPt0wC0dNAqZG2SYnpylHjtR2\n0zVxIs2Dt6CA+fybkdzJ31Kc3CUzlWH+qMS3bSTIIDbx/2jLrygEKiK2OXr0cNr+d5JeGsEnic/n\nYxIPcS9/5tvMStMaMxrBq8/qnc/HzGEU2+jI3TwVYUmMGGfuiCNoBJ8uhw7xIjCGV8hjpRYQE+v+\nQ0v8BKigDivJoyvv2w5JLFCCT9SCBdCjB7uBAbzBR2fcslbEjuM05Fae5/dMpIh8fsxj1KHcdliS\nRirRxGvvXpg4EZYvh7//Hd+gQbj167w7+lWfiejIVp5kPDkc5W6eYi2XndanKz+DHqcSTSocPQoP\nPgg9e0LHjrBpEwzUzbLF2bbRiYEsZQoTmM8wnuN7tKXEdliSYhk3D/7EiROsXr066SOSrCNHuODl\nl7ngxRf5/PLL2f3Xv3KiVStYty6p/Yikjo9/cBuvMIb7eJTV9GYmN/CY7bAkZTKuRPPyyy/zne/c\nRf36nRN/s2CQy8uP8f2vP2ZE6afMO6sZj9Zrzfas01fuKy//gmPH1pMpX+ed2a/6TLYWfMK9/Jnx\nPMh5I0bA3XdDQQHUrZuW/iUx0ZRoolEAbAG2Ya6fqM7jof3rgV4xtg0m08yZM4M5OdcHzVJ78WwV\nwd68F/wt9we30Cm4lQ7Bn/G7YHM+qaXNutCk4nj7jHez0aetftVnqrb6EAz+7W/BYP/+weC55waD\n48cHg4sWBYPHjyf1synJZX5XahepBp8FTAkl6i7AOOCSKscMBzoAHYG7gSdiaGudjwq+wRbu4ile\n4Cb2cwFCqNnYAAAEJElEQVQvMA4fQW7hn3RiK48wkYO0SEM0gTT04RYB2wE4SCCl7/4VmEXK3nwT\nVq+G3Fz49a/hvPPMHaP+8Ad45x04fjylcUQjEAjYDsFVItXg84DtcPJszHRgNLA57JhRwD9Cj1cA\n5wAtgXZRtE2bbEq5iA9pxy7as4PubKAXa+nGRg7SnLe5ikUMZTIPUUI7GyFiPsh+S307TQD9LCoF\nSNvPom1buP9+sx0+DK+/DkuXwr/+BcXFcMkl5naBnTuf2nJz01bWCQQC+P3+tPSVCSIl+FbAnrDn\ne4ErojimFXBhFG1jV1Zm1sA+dsxs4Y+PHKHDokX84sT7NOEeWnCA8/iE1uzlQj5iPxewi3bs5GI2\n0o1ZfJt19ORzzkk4LBH3yq6s59aqPtBrzRouXbOGznByaw0cBvZhPuT7gEOh1w4Dn4b+/Qw4Htqy\nGp1NyScfQf36UEeT+VIlUoKPWOMJSazQ37u3SdylpWarfFzdaxUV0LixWQe7cePTH+fk0OTIEU5U\n7GVb/WW846vLQV899tdpxx5fF0p9lb9InwCvhzZoklDwUFHxOceOJfgmItZUrn9Tu6+A5aEtXB3K\nTw6kWrGPVuzjXA7RmsN04zBNOUwzPuVsPqcBx832xT4491w4ccKM/hs0MHemys6ufevdOwX/f+/q\nAywIez6ZM0+W/i9wU9jzLcD5UbYFU8YJatOmTZu2mLbtJCgb2AHkAnWBdVR/knVe6HEf4N0Y2oqI\niEXDgA8wfy0mh14bH9oqTQntXw+nXQNdXVsREREREckU92HuUNDMdiAW/R4zlXQ98DJwtt1wrIjm\nAjkvaAMUAZuA94Ef2Q3HuixgLfBv24FYdg7wIiZPFGNK447XBnNSdhfeTvBDOHUB2v+ENi/JwpT0\ncjH3r/PyuZuWQM/Q48aYcqdXfxYA/wVMA+bYDsSyfwC3hx5n45JB4CygO0rw4cYA/7QdRJr15fTZ\nV/eHNoHZwCDbQVjSGlgC5OPtEfzZwM5oD3bKFQajMddIbLAdiMPczqkZSl5R04VzXpeLWedpheU4\nbPkjMBFTwvWydsAB4FlgDfA00LCmg9OZ4BcDG6vZRmFm2DwQdqwTVrlMpZp+FiPDjvk58DXwr7RH\nZ1fQdgAO1BhTc/0x4MVL6q7FXJ24lszPDZFkY2Yq/jX07xc4/BvupcB/MKWZXUApZv2a8yzGZNtt\nwDLM1eFeE+0Fcl5xFrAQ+IntQCz6LeZb3S5gPyapPW81IntaYn4Ola4C5lqKJS5er8EXYGZNNLcd\niCW6QO4UHyaR/dF2IA4yAG/X4AHeBDqFHhcCD9sLJXY78XaC3wbsxnwdXYv5KuY1ukDOuApTc17H\nqd+HAqsR2TcAzaLpAbyHt6dSi4iIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIhb/H+GZXb8krYgXgAA\nAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sampling = ImportanceSampling(log_p=lambda x: np.log(p(x)), q=norm(loc=1,scale=2))\n", "\n", "show_sampling(sampling, plotter=weighted_gauss_hist)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also approximate the normalization constant of $p(x) = e^{-\\frac{1}{2} x^2}$ by the mean of the sample weights:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEACAYAAACwB81wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGYhJREFUeJzt3Xu0VOV9//H3AeQuooIoNzFe2uAlihWJ6HLSJt7qz9ou\n09CuVRtNlZpqrWk0/aVpxT9Ss2JsjG1Vqvml3hqaqhBjsGpsBjUqagS8IAookZug3O8I5/n98Z1x\n5sy5zYE5c+Ds92utWXvP3nv2fuaZPZ+959l7zwZJkiRJkiRJkiRJkiRJkiTV0WDgIeAtYD4woYVp\nbgcWAvOAU+pXNElSLd0LXF7o7wUcVDH+AmBmof904MU6lUuSVEMHAe+2M81dwJfKni8AhnVaiSRJ\nHdajimmOAj4EfgS8CtwN9K+YZgSwtOz5MmBkLQooSaqNagK/FzAOuKPQ3QL8XQvTNVQ8T3tXNElS\nLfWqYpplhcfLhecP0TzwlwOjyp6PLAz7xNFHH50WL168h8WUpMxaDBxTixlVs4f/AdFcc1zh+eeB\nNyumeRS4tNA/AVgPrCqfYPHixaSUfKTEjTfe2OVl2Fce1oV1YV20/QCO3tOAr1TNHj7ANcCDQG9i\na3M5MLkwbipxhs4FwCKiyeeyWhVQklQb1Qb+POC0imFTK55fvffFkSR1lmqadFRjuVyuq4uwz7Au\nSqyLEuuic1SeWdOZUqE9SpJUpYaGBqhRVruHL0kZYeBLUkYY+JKUEQa+JGWEgS9JGWHgS1JGGPiS\nlBEGviRlhIEvSRlh4EtSRhj4kpQRBr4kZYSBL0kZYeBLUkYY+JKUEQa+JGWEgS9JGVFt4C8BXgPm\nAC+1MD4HbCiMnwN8qwZlkyTVULU3MU9EqK9tY5pZwEV7WyBJUufoSJNOe/dUrOf9cSVJHVRt4Cfg\nF8ArwBWtjD8DmAfMBMbWpHSSpJqptklnIrASGAo8BSwAni0b/yowCtgKnA/MAI6rXTElSXur2sBf\nWeh+CEwHxtM08DeV9T8O3AEcQkWb/5QpUz7pz+Vy5HK5DhVWkrq7fD5PPp/vlHlX0+7eH+hJhPoA\n4EngpkK3aBiwmmjaGQ/8BBhTMZ+UUtrL4kpStjQ0NECNjpFWs4c/jNirL07/IBH2kwvDpgKXAFcB\nu4hmnUm1KJwkqXbqeWaNe/iS1EG13MP3SltJyggDX5IywsCXpIww8CUpIwx8ScoIA1+SMsLAl6SM\nMPAlKSMMfEnKCANfkjLCwJekjDDwJSkjDHxJyggDX5IywsCXpIww8CUpIwx8ScoIA1+SMsLAl6SM\nqDbwlwCvAXOAl1qZ5nZgITAPOGWvSyZJqqleVU6XgBywtpXxFwDHAMcCpwN3AhP2tnCSpNrpSJNO\nW3dNvwi4t9A/GxgMDNvTQkmSaq/awE/AL4BXgCtaGD8CWFr2fBkwcu+KJkmqpWqbdCYCK4GhwFPA\nAuDZimkqfwGkyplMmTLlk/5cLkcul6ty8ZKUDfl8nnw+3ynzbquZpjU3ApuBW8uG3QXkgWmF5wuA\ns4FVZdOklJptAyRJbWhoaIA9y+pmqmnS6Q8cWOgfAJwDvF4xzaPApYX+CcB6moa9JKmLVdOkMwyY\nXjb9g8CTwOTCsKnATOJMnUXAFuCy2hZTkrS3avIzoUo26UhSB9W7SUeS1A0Y+JKUEQa+JGWEgS9J\nGWHgS1JGGPiSlBEGviRlhIEvSRlh4EtSRhj4kpQRBr4kZYSBL0kZYeBLUkYY+JKUEQa+JGWEgS9J\nGWHgS1JGGPiSlBEGviRlRLWB3xOYA/yshXE5YENh/BzgWzUpmSSppnpVOd21wHzgwFbGzwIuqkmJ\nJEmdopo9/JHABcA9tH7n9JrcUV2S1HmqCfzvA9cDja2MT8AZwDxgJjC2NkWTJNVSe006FwKribb5\nXCvTvAqMArYC5wMzgONamnDKlCmf9OdyOXK51mYpSdmUz+fJ5/OdMu/2mmL+CfgzYBfQFxgEPAxc\n2sZr3gNOBdZWDE8ppT0spiRlU0NDA9So2bwjMzkb+DrwfyqGDyN+BSRgPPATYEwLrzfwJamDahn4\n1Z6lU1RM7MmF7lTgEuAq4lfAVmBSLQomSaqtep5d4x6+JHVQLffwvdJWkjLCwJekjDDwJSkjDHxJ\nyggDX5IywsCXpIww8CUpIwx8ScoIA1+SMsLAl6SMMPAlKSMMfEnKCANfkjLCwJekjDDwJSkjDHxJ\nyoi6Bv6OHfVcmiSpXF0Df/36ei5NklTOJh1JyohqA78nMAf4WSvjbwcWAvOAU1qbSUM976ArSWqi\n2sC/FpgPtHQX8guAY4BjgSuBO2tTNElSLVUT+COJUL+Hlu+cfhFwb6F/NjAYGFaT0kmSaqaawP8+\ncD3Q2Mr4EcDSsufLiI2EJGkf0qud8RcCq4n2+1wb01Xu+bfU9MMtt0xhwIDoz+Vy5HJtzVKSsief\nz5PP5ztl3u0dRv0n4M+AXUBfYBDwMHBp2TR3AXlgWuH5AuBsYFXFvNKqVYnDDtvLEktShjTE2S41\nOeWlvSadbwKjgKOAScD/0jTsAR4tGzYBWE/zsJckdbH2mnQqFZtqJhe6U4GZxEHdRcAW4LLWXuxp\nmZLUdeoZwWn16sTQoXVcoiTt5+rZpFNTqcVDuZKkejDwJSkjDHxJyggDX5Iywn/LlKSMqGvgv/Za\nPZcmSSpX19MyJ05MPPdcHZcoSfu5/fa0zM2b67k0SVK5ugb+li31XJokqVxdA3/XrnouTZJUrq6B\n39jaP+pLkjpdXQN/9+56Lk2SVM49fEnKCANfkjLCwJekjLANX5Iywj18ScoIA1+SMsImHUnKiGoC\nvy8wG5gLzAdubmGaHLABmFN4fKulGbmHL0ldp1cV02wHPgdsLUz/HHBmoVtuFnBRWzMy8CWp61Tb\npLO10O0N9ATWtjBNu3/faeBLUtepNvB7EE06q4BfEk075RJwBjAPmAmMbWkmtuFLUteppkkHoBE4\nGTgIeIJos8+XjX8VGEX8EjgfmAEc12wmjVOYMiX6c7kcuVxuT8osSd1WPp8nn893yrz35C4q/wBs\nA77XxjTvAafStOknQfJG5pLUAfW+49UQYHChvx/wBeJMnHLDygo0vtDfUju/JKmLVNOkcwRwL7Fx\n6AHcDzwNTC6MnwpcAlwF7CKadSbVvKSSpL1S15uY26QjSR2z397EXJLUdeoa+KNH13NpkqRydQ38\nk06q59IkSeX88zRJyggDX5IywsCXpIww8CUpIwx8ScoIA1+SMsLAl6SMMPAlKSMMfEnKCANfkjKi\n7oHf2AgNDfivmZJUZ3UP/J07o//jj+u5ZElS3QN/x47oL3YlSfVR18BvbCzt4Rv4klRfXdakY+BL\nUn3VNfCXLIHTT4/+YvBLkuqjvcDvC8wG5gLzgZtbme52YCEwDzilrRkuXx5d9/Alqb7aC/ztwOeA\nk4GTCv1nVkxzAXAMcCxwJXBnNQvesQO++U1Pz5SkeqmmSWdrodsb6AmsrRh/EXBvoX82MBgY1t5M\nN22Cm2+2aUeS6qWawO9BNOmsAn5JNO2UGwEsLXu+DBjZ3kzXrInutm1VlECStNd6VTFNI9GkcxDw\nBJAD8hXTNFQ8b6WhZsonfc8/nwNybN0KgwdXUQpJyoB8Pk8+n++UeVcGdXv+AdgGfK9s2F3EBmBa\n4fkC4GziF0G5VL4duOUWuP56mDYNjjsOTmnzUK8kZVNDQwN0PKtb1F6TzhCiTR6gH/AFYE7FNI8C\nlxb6JwDraR72zfz859GdNAnGjauusJKkPdfeVuNE4oBsj8LjfuAWYHJh/NRC91+B84AtwGXAqy3M\nK7Xa0oNn60hSS2q5h1+TmVTpk8A/6yx49tmKkQa+JDVTzyadTnHkkV2xVEnKti4J/IEDS/2/+Q2M\nbPckTknS3uqSwO/fP7q33x6nZG7Y0BWlkKRs6ZLAnzUruiecEHv7mzbB9u3eFEWSOlOXBP6vfx3d\n/v2hR6EEgwdD795dURpJyoYuCfzDD49usWkH/PdMSepsXRL40wrX5BYD/9prS+NuuaX+5ZGkLKh7\n4H/ta9C3b/QXA/9TnyqNv+GGepdIkrKh7oF/xBHNA//AA+tdCknKni5p0unXL7rFwD/kkOiedlpX\nlEaSsqFLAr9X4U+ZDzggumPHRvcHP4hh/s2CJNVe3QM/pdKpmEWDBkX3qKOiuccLsSSp9rok8I88\nEp5/vjRsyBC49dY4XXPIkNLdsCRJtVP3wG9shIYG+OxnS8N69oyzdwAOPdTAl6TO0CWB3xYDX5I6\nxz4X+EOGwEcftT7em55L0p7pkjb8thT38B9/vPnGYc6cpn/HIEmq3j63hz94MDzxBFxwAcyb13Tc\nypXRdS9fkjqumsAfBfwSeBN4A/jrFqbJARuIG5zPAb7V2syGD297YdOnw//8T/Rv3Nh03OTCnXQn\nTWr+uu9+Nw4G//d/w/vvt70MScqiau6TeHjhMRcYCPwauBh4q2yaHPA14KI25pPWrUsMGtT8PPxy\nzzwDZ58d/VOnwkUXweuvwxe+EIH+ycwS3Hdf3Bv3vPPgkkuazmfSJLj//tJFXpK0P6r3PW0/IMIe\nYDMR9C3tp7dboMGD2w57aPr3CpMnx3/vnHMOjB8fw6ZMie6vfgV//udwzz2lsL/11tJrp02Db3yj\nvRJJUnZ0dKsxBpgFHE+Ef9HZwCPAMmA58HVgfsVrU6ryPxMaWinVyJGwdGnL4884IzYCp54KZ54Z\n0w0aBD/6Uevzk6R9Xb338IsGAg8B19I07AFeJdr6PwP8CzBjbwq1cGFpT77ciBHR/fa3o9uvH1x1\nVTT7PPZYDHvhBfjnf4brr4d7741fFZXHAiQpi6rdahwAPAY8DtxWxfTvAacCa8uGpRvLnuQKD0lS\nSb7wKLopOjXZw69mJg3AvcAa4LpWphkGrAYSMB74CdH8U67qJp2ifD720E8+udQs09F/0vzbv409\n/iuvjF8N554LDz4IJ54Y8/r3f4e//MuY9j/+I44LTJ8On/tcLLsjVq+GRYuieSmlOI7wp39aGv/l\nL8Ndd8U1BsceGzd+6ds3TjPt27f94xvadzU2wkMPwYwZMGFC6YyylSvhsMNg8WI44YSubV7ctav5\nSQybNsW9pGfNgtNPh4MO6pqyFb/XS5dGeQ49tPRvups3x4kbL74IP/whrFgBn/kMnHIKjBsXf7o4\nZgxs3x71v2pVXMC5YgUsWxYtAYcfDgMHxnd62TLYsgX69IHduyMTRo6M72S/fnG71WXL4Oij4/Pa\nuDE+xyOOiDJt3x7LmDMH3nor6nX9+vj+jhoV3+dVq2Dt2mhWLv4/2AknxLDFi+PklDVrYll9+sCw\nYfE+hw+PLHj33Xg+aBB88EHtmnSqmcmZwDPAa0SgA3wTGF3onwr8FXAVsAvYSpyx82LFfDoc+OUu\nvDDO1Cm/HWI1li2LD6FSYyOcdBK88UY8//a34aabYOfOeJ7LwdVXl1aoNWviAPFpp8Hv/m7z+W3b\nFv8HdNddbZenrb+OGDoU5s6ND79nz/bfW/GahuKGorERnnoq3tOf/El8ef/xH2ND9kd/FBu9Aw+M\nDc3u3fHFqlxOShECxX8wrYUPP4yN7pw58f4nTowyHHNMPCBW/F69SoHY3sZv06aY34oVsfG+5564\nfmP06Gj6GzculgXwe78HAwZE/fTuHV/gbdtiutaW8847URejR8PMmfElf+edqNPjjoMnn4RHHomA\nGDAggmrIkDh+tGhRfKEh6nvrVjj44HivF18cfxz40Ucx70GDIkh27Ijy9ukTy50wIYJhzpwIm09/\nOsb17BlnoA0dWvrsdu6MMHznnRg+enQE2rRp8dqNG6NMS5bA8cdHedeujRCcOzeCZezYeA8TJkQ5\nNmyIupo4Mc6EO+AAWLAgXjd2bFwAuWQJfPwx/OY3EbjFwLr44qijDRsiYNevj7B9441Y3uDBsG5d\nfIYNDXEa9YoVUcdDhsRn09AQ/b17RxPv8OHxOX75y1GP8+fHRuqdd2D58ijLtm1xxt5v/VYse+TI\nqI9NmyKAd+2KMg4bFp/jRx9FXT/7bKyjixaV7q09ZEi8t6FD4YMPos42bozvza5d8R5OPDGOGfbt\nG/f02L073segQbGMgw+OZa9eHZ/d22/HfI85Jup58OAoY3s7AbVsw6/n/sZeBf7e2LEDvvKV2LOf\nPh3+8A9L43r2jK3p6NGlih87Nlaootdfj3vt3ndfPH/ssVjBf/u34/Xlf/l8xx3w1a/GBuOrX4Uv\nfjGGpwSXXgoPPADPPRcrwIYN8Bd/AX//97EyfO97pWV+5ztxltHFF8NPf1oafu218dp16+Duu2PF\n+dnPYsN2221x3KIjxo6NwOjVK24vefnlMfyKKyK8vvSlWN6e2LIljqXcfXcETZ8+EW4//Wl8gbZs\nib2rhobStRdFN9wQ9XvuuREEq1fHe3377aib//qv+IIPHRpf1iOOgM9/PgJi1Kiok1Wrol5ac8gh\nMc9Jk+Jz2roVXnkl9hLvvLM03Uknxcbh4IPhvffiONHv/35sxN59NzYkJ5/c9BqTbdsiAPr3j8/5\nwAPh4Yej3J/9bGzwnnsuwmLp0giIdetiZ2DAAHj66QiUk06K95TPRxD36xfzgdJe8AcfRICddlos\na8WKCKtrron18pBD4Hd+Jz6DZ56JQOrVK4K6GPD9+kU5Zs2KejjssKi7l1+OX6wDB8YGYsSIuCCy\nd+8IrD59YmP0xhsxj40bo3xvvx3Pd+4s7RUff3ycbffhh/F+e/SI93/kkdEdMSLee48esRF4//1Y\nN048sbY7IK3ZsSO+p717x3LXro2NwvDhXXtXvloGfj2lrrRrV0obNkT/xIkpQUq33950mvXrU1q5\nMvo3bUrp9ddjuuLjK19J6d/+remw8sell+5dGXfuTOnHP07puuuaz/ucc5oP+5u/SelTn2o6bPr0\nlLZvT+npp1OaMSOljz+Oee/eHXWwZk1Ky5bFdHfckdLXv55SLpfSNdfE6//4j1N64YWUBg8uzTOX\nS+nMM1N6//2UtmxJ6cYbU3r22ZTmz486XL48pZdeSumBB1L6+c9Tuu22lCZMiNeeeGJKL7+cUmNj\n8/e7eXNK99yT0oUXpvTIIynNnp1SPh/luvzypu+roSEekNIXv5jSo4+W5tPSvMutW5fS2rVRL2vW\nRHfLlpTefDOl669P6ayzSssZNy6lq69OadGieO1rr0W97WvWr0/p3Xfjs/rww64ujToTpZaV/UpX\n19seWb48gmDSpNIX/+abI9whpUGDonvFFbVd7gsvpHTccSk9/njT4du3R1AWNTZG8D7/fATY3iif\nb0qxkbjpppQOPbRp+B56aOl9t/a44Yaou/bCuD2NjSlt3RoB/cQTKS1cuHfza01xYybta6hh4Gei\nSaezFA+4FN9Wdz/f/803o3ng8MOjGWPdumiT3Lw56qFXr5bvaCZpz9mGL0kZ0VUXXkmS9mMGviRl\nhIEvSRlh4EtSRhj4kpQRBr4kZYSBL0kZYeBLUkYY+JKUEQa+JGWEgS9JGWHgS1JGGPiSlBEGviRl\nhIEvSRlRTeCPAn4JvAm8Afx1K9PdDiwE5gGn1KR0kqSaqSbwPwauA44HJgB/BXy6YpoLgGOAY4Er\ngTtRq/L5fFcXYZ9hXZRYFyXWReeoJvA/AOYW+jcDbwHDK6a5CLi30D8bGAwMq0UBuyNX5hLrosS6\nKLEuOkdH2/DHEM01syuGjwCWlj1fBozc82JJkmqtI4E/EHgIuJbY069Uec9Fb2ArSfuQam+MewDw\nGPA4cFsL4+8C8sC0wvMFwNnAqrJpFgFH71EpJSm7FhPHSOuiAbgP+H4b01wAzCz0TwBe7OxCSZJq\n70ygkThwO6fwOB+YXHgU/SuxFz8PGFfnMkqSJEmqp/OIdv2FwDe6uCz1sAR4jfg19FJh2CHAU8A7\nwJPEqatF/5eomwXAOXUrZef4f8Sxm9fLhu3Jez+1MI+FwA86sbydqaW6mEKcxVb+a7moO9dFaxdw\nZnHdaK0uptAN1o2eRFPPGOLg71yaX7jV3bxHrMjlvgvcUOj/BvCdQv9Yok4OIOpoEfv3X16cRZy6\nWx5yHXnvxRMJXgLGF/pnEjsN+5uW6uJG4GstTNvd6+Jw4ORC/0DgbSIHsrhutFYXnb5u1CNYxhMF\nXEJctTsN+IM6LLerVZ4BVX5x2r3AxYX+PwB+TNTNEqKuxrP/ehZYVzGsI+/9dOAI4EBKv47uK3vN\n/qSluoCWz47r7nXR0gWcI8jmutFaXUAnrxv1CPyWLsoa0cq03UUCfgG8AlxRGDaM0mmqqyhdiTyc\nqJOi7lg/HX3vlcOX073q5Bri5IYfUmrCyFJdjKF0AWfW140xRF0Uz2zs1HWjHoGfxQuwJhIf4vnE\nfw+dVTE+0Xa9dOc6a++9d3d3AkcRP+lXArd2bXHqbiDwMHEB56aKcVlbNyovZu30daMegb+cOEhR\nNIqmW6XuaGWh+yEwnWiiWUW03UH8FFtd6K+sn5GFYd1JR977ssLwkRXDu0udrKYUbPdQar7LQl0c\nQIT9/cCMwrCsrhvFuniAUl10i3WjF3Gl2BigN93/oG1/ol0NYADwK+Ko+ncpnaH0dzQ/ONWb2Lov\npvoroPdVY2h+0Laj73020U7ZwP55YK5oDE3r4oiy/uuA/yz0d/e6aO0CziyuG63VRbdZN84njkQv\nIk4v6s6OIj6cucQpV8X3ewjRrt/S6WffJOpmAXBu3UraOX4MrAB2EsduLmPP3nvxdLNFxL0W9keV\ndXE58UV/jWinnUHTf5XtznXR0gWc55HNdaO1i1mzum5IkiRJkiRJkiRJkiRJkiRJkiRJUtf5/7yX\n/rNVJn2eAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_expectation(sampling, lambda x: x[1]) # f extracts weight\n", "plt.axhline(np.sqrt(2*np.pi), color='r'); # true normalization constant" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Importance sampling can fail miserably if $q$ has thinner tails as $p$. In this case, the weights $\\frac{p(x)}{q(x)}$ can become arbitrarily large and dominate the weighted samples!\n", "\n", "This effect is demonstrated below:" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEACAYAAACuzv3DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFo9JREFUeJzt3Xu0lHW9x/H3sDcXFZBbonJxc9Mib6ghqeWYlEItslKX\nWsuTlXI0L5l18lLHnadzzGMrXR5vpF3wki7ykigYmTmKtkJKriIFiIKIYICCoFzn/PGb7Z69mb1n\nNgz7md/e79davzXPPPPMzJfNzGd+83t+zzMgSZIkSZIkSZIkSZIkSZJUVl2AGcBsYAFwfRPb3QIs\nAuYAI1qnNElSqfbOXVYDfwVOaHT7WGBqbvnY3DaSpFbSoYRtNuUuOwFVwNpGt48DJuaWZwA9gL5l\nqU6SVFQpQd6BMLSyCniGMMSSrx+wPO/6G0D/slQnSSqqlCDfARxJCOdPA+kC26QaXc/uXlmSpFJV\nt2Dbd4EpwDFAJm/9CmBA3vX+uXUNDBkyJLtkyZJdKFGS2rUlwNDmNijWI+9DGPMG2Av4LDCr0TaT\ngXNzy6OAdwjDMA0rWbKEbDZb8e3aa69NvIa2UmcMNVqndVZ6A4YUyemiPfIDCDsyO+TavcDTwPjc\n7RMIM1bGAouBjcB5xZ5Uaq+6d+/Fhg3rGqzr1q0n69c3nkMgla5YkM8DjiqwfkKj6xeXpxypbQsh\nnm20rvEuJqllStnZ2a6k0+mkSyhJDHXGUCNYZ7lZZ+trza5ANjfeI7VbqVSKnSd1pfC9oaaE10zz\nWW2PXJIiZ5BLUuQMckmKnEEuSZEzyCUpcga5JEXOIJekyBnkkhQ5g1ySImeQS1LkDHJJipxBLkmR\nM8glKXIGuSRFziCXpMgZ5JIUOYNckiJnkEtS5AxySYqcQS5JkTPIJSlyBrkkRc4gl6TIGeSSFDmD\nXJIiZ5BLUuSKBfkA4BngZWA+cGmBbdLAu8CsXPthGeuTJBVRXeT2rcDlwGygK/B34CnglUbbPQuM\nK3t1kqSiivXI3yKEOMB7hAA/sMB2qXIWJUkqXUvGyGuAEcCMRuuzwHHAHGAqMLwslUmSSlJsaKVO\nV+Ah4DJCzzzfS4Sx9E3AGOD3wMHlKlCS1LxSgrwj8DBwHyGkG9uQt/wkcDvQC1jbeMPa2toPl9Pp\nNOl0uvRKJakdyGQyZDKZFt2n2Nh2CpgIrCHs9CykL7CaMMQyEphEGIZpLJvNZltUnNTWpFIpwlul\nwVp8b6gp4TXTfFYX65EfD3wNmEuYWghwNTAwtzwBOB24ENhGGF45a9fKlSTtitacbWKPXO2ePXK1\nVCk9co/slKTIGeSSFDmDXJIiZ5BLUuQMckmKnEEuSZEzyCUpcga5JEXOIJekyBnkkhQ5g1ySImeQ\nS1LkDHJJipxBLkmRM8glKXIGuSRFziCXpMgZ5JIUOYNckiJnkEtS5AxySYqcQS5JkTPIJSlyBrkk\nRc4gl6TIGeSSFDmDXJIiZ5BLUuQMckmKXLEgHwA8A7wMzAcubWK7W4BFwBxgRNmqkyQVVV3k9q3A\n5cBsoCvwd+Ap4JW8bcYCQ4FhwLHAHcCoslcqSSqoWI/8LUKIA7xHCPADG20zDpiYW54B9AD6lqtA\nSVLzWjJGXkMYNpnRaH0/YHne9TeA/rtXliSpVMWGVup0BR4CLiP0zBtLNbqeLfQgtbW1Hy6n02nS\n6XSJTy9J7UMmkyGTybToPo0DuJCOwBPAk8DNBW6/E8gAD+auLwROBFY12i6bzRbMd6ndSKVS7NzP\nSeF7Q00Jr5nms7rY0EoK+CWwgMIhDjAZODe3PAp4h51DXJK0hxTrkZ8APAfMpb4bcTUwMLc8IXd5\nK3AqsBE4D3ipwGPZI1e7Z49cLVVKj7yUoZVyMcjV7hnkaqlyDK1IkiqcQS5JkTPIJSlyBrkkRc4g\nl6TIGeSSFDmDXJIiZ5BLUuQMckmKnEEuSZEzyCUpcga5JEXOIJekyBnkkhQ5g1ySImeQS1LkDHJJ\nipxBLkmRM8glKXIGuSRFziCXpMgZ5JIUOYNckiJnkEtS5AxySYqcQS5JkTPIJSlyBrkkRa6UIP8V\nsAqY18TtaeBdYFau/bAslUmSSlJdwja/Bv4PuKeZbZ4FxpWlIklSi5TSI58OrCuyTaoMtUiSdkE5\nxsizwHHAHGAqMLwMjylJKlEpQyvFvAQMADYBY4DfAwcX2rC2tvbD5XQ6TTqdLsPTS1LbkclkyGQy\nLbpPqUMiNcDjwGElbLsUOBpY22h9NpvNll6Z1AalUinCl9gGa/G9oaaE10zzWV2OoZW+eU8yMrfc\nOMQlSXtIKUMrDwAnAn2A5cC1QMfcbROA04ELgW2E4ZWzyl+mJKkprTnbxKEVtXsOrailWmtoRZKU\nIINckiJnkEtS5AxySYqcQS5JkTPIJSlyBrkkRc4gl6TIGeSSFDmDXJIiZ5BLUuQMckmKnEEuSZEz\nyCUpcga5JEXOIJekyBnkkhQ5g1ySImeQS1LkDHJJipxBLkmRM8glKXIGuSRFziCXpMgZ5JIUOYNc\nkiJnkEtS5AxySYpcKUH+K2AVMK+ZbW4BFgFzgBFlqEuSVKJSgvzXwKnN3D4WGAoMAy4A7ihDXZKk\nEpUS5NOBdc3cPg6YmFueAfQA+u5mXZKkEpVjjLwfsDzv+htA/zI8riSpBNVlepxUo+vZQhvV1tZ+\nuJxOp0mn02V6eklqGzKZDJlMpkX3aRzATakBHgcOK3DbnUAGeDB3fSFwImEHab5sNlsw36V2I5VK\nsXM/J4XvDTUlvGaaz+pyDK1MBs7NLY8C3mHnEJck7SGlDK08QOhh9yGMhV8LdMzdNgGYSpi5shjY\nCJxX/jIlSU0pdWilHBxaUbvn0IpaqrWGViRJCTLIJSlyBrkkRc4gl6TIGeSSFDmDXJIiZ5BLUuQM\nckmKnEEuSZEzyCUpcga5JEXOIJekyBnkkhQ5g1ztSvfuvUilUg1a9+69ki5L2i2exlbtStKnkU36\n+RUfT2MrSe2AQS5JkTPIJSlypfxmp1QxduzYsdO6up2WUntlj1zRuPvuu6mqqqKqqjqvVXHIIUcm\nXZqUKINc0Vi3bh3V1d8DduS111mzZl2yhUkJM8glKXIGuSRFziCXpMgZ5JIUOYNckiJnkEtS5Axy\nSYpcKUF+KrAQWAT8oMDtaeBdYFau/bBcxUlJ8pS3ikWxQ/SrgFuB0cAKYCYwGXil0XbPAuPKXp2U\noA0b1tH4lLMbNnTc6XQA3br1ZP36ta1YmdRQsR75SGAx8BqwFXgQ+GKB7TzRhSpOoR717ttGCPf6\nFgJfSk6xIO8HLM+7/kZuXb4scBwwB5gKDC9bdVIJ1q59a6fATqVSeT3q/NZ69swHibSzYkMrpbzy\nXwIGAJuAMcDvgYN3sy6pBbZS+KWabHAWGppJuia1TcWCfAUhpOsMIPTK823IW34SuB3oBew0aFhb\nW/vhcjqdJp1Ol16pJLUDmUyGTCbTovsU6x5UA/8ATgbeBF4Ezqbhzs6+wGpC12MkMAmoKfBY/man\ndsuNN97I1VevZtu2G/PWLgMOoukeeaEecWm/mdnU72u21v0lKO03O4v1yLcBFwPTCDNYfkkI8fG5\n2ycApwMX5rbdBJy1yxVLklqslF8IejLX8k3IW74t1yRJCfDITkmKnEEuSZHzx5fVLlSxjT78i95A\nb56jF2vpzGaq2B56M/feC127Qrdu0L079OxJF+CDZMuWSmKQq03Zm40cwRxGMItDgSF8jsG8ygCW\n8w49WAOs4RrW0ovNdGY7VWwH+MMfYONG2LAB1q+HNWt4B1jPR1jGQBYzlPkcyjxgHkt4lcE4J1yV\nwiBX1A5gFScCacbzKaZTw2u8zMeZxQjmAZO5nCUM4XUOYgudCeE7vdGjPMjX7r9/p8feK5ViP+Yx\nkGUczD85lPmcDxxOmi58wAsczwscz/MAW7ZAp057+F8rFWaQKyrV2e18ij8zjsmM4Uk+wiqeBTIM\n507+nfkcyjY65ra+i3CwcdFHbfLw+VXszyr2ZyYjc2tuAJZzICs4nhc4gee5DXinc2f+TDhHxZOE\ngy6k1tKa3w09IEi7ZtMmeOIJFlx/PfvPXshiDmMy45jKWGbTiyyD2N0Dgnb3/n1YzSlMYyxTOYVp\nvM5BTGIWk1jCUgYXvb/vDTWllAOCDHJVpm3b4Omn4f774fHHYeRIpnXtyvmP7cfy7XfkbVieIzvL\nef8ObOdTTOdMTuIr7McyBjKJM3mAs1nBgIL3972hppQS5E4/VGV55RX4znegf3/4z/+EY46BhQth\n2jTmjhrFylTXpCssagdVPEuabwP9WMGV/JSD+SdzOZypwBlMohObky5TbYhBruRt2wYPPwyf+Qyc\ndFKYBjh9OsyYAZdeCn37Jl3hLttONX/mZC7gLvrzBvcB45nACvpxC5dwBLOTLlFtgEGu5KxcCddd\nBzU1cPPNcP75sGwZ/OQnMGxY0tWV3fvszW+B0TzNJ5jJGnozmXG8AHDffbDZXrp2jUGu1pXNht72\nWWfB8OGwYgVMmRLWnX12u5nC9xqD+DG1DOZV/hfgnntg4EC46ip4/fWky1NkDHK1jvfegzvvhCOO\nCD3v446DpUthwoSwrp3aTjWPAfzxj/Dcc/DBB3DUUTBuHEybBjt2JF2iImCQa8965ZUwzj1wYAim\nn/+8fl2PHklXV1kOOQRuuikML40bB1deWb/unXeSrk4VzCBX+W3bBo88AiefHHZedu8Oc+bAo4/C\n6NHgb1c2b5994FvfgpdegokTYeZMGDQIxo+HuXOTrk4VyCM7VT5vvQV33RWGS2pq4KKL4Ctfgc6d\nk64sTqlUGII67rj6v+2YMTBkCFx8MXzpS9CxY/HHUZtnj1y7J3/n5cc+BsuXwxNPwPPPwznnGOLl\nsv/+8KMfwWuvwSWXwO23hw/L664Ls3/Urhnk2jXr18Ntt8Hhh4dhgE9+Muy8/MUv4Mgjk66u7erY\nEc44AzKZcMbGN98Ms3/OPjt8eHqEaLtkkKtlZs2CCy6Agw6CZ5+FW24JR15edpk7L1vbYYeFmUBL\nl8KoUfCNb8CIEXD33eH8NGo3DHIVt2kT/OY3cOyxcNppIcQXLIBJk8LOTHdeJqtHj/BBunAh3HAD\nTJ4cZgldcUWYIaQ2zyBXYdksvPBC6H337w+/+10Yo331VbjmGjjggKQrVGMdOsApp4QgnzkTqqvD\nzKFRo8IOaKcwtlkGuRqqO0T+kEPCgTtDhsC8eeHoyy98Aaqqkq5QpRg0KPTOly0LJx/705/CztFz\nzgkHH23fnnSFKiOnHwrefjvM+540Kcz3PvPMcO6PT3zCYZPYVVfD2LGhrVkDDzwAV18dpjOecUb4\nvz722NCbV7T832uv3n47fN0ePTqcoCqTgW9/G954I0xtGznSEG9revcO88//9rfQK+/RA775zdBT\nv+IKePFFZ71EyiBvL7LZsOPrZz8LOyjrwvuii8I85AcegC9/Gbp0SbpStYbhw+Haa+Hll2Hq1HA0\n6bnnhh3ZF14YhtLefz/pKlUih1basvfeCwfrTJkS3qzbtsHnPw/f/W7oie+1V9IVKmmpFBx6aGg/\n/nH4sJ8yBW68McxN//Snw2tm9GgYOtRvaRXKIG9LNm6Ev/wl9LSfeSacl+Poo8Nh3Y89Ft6svhHV\nlFQq9NSHD4fvfx/WrQsnOpsyJewA79AhfJurazU1SVesHIM8Vjt2wKJFYVxz5sxwOX9+OCAknQ5v\nvFGjYO+9k65UserZM5x64ayzwtDcokWhgzBtWjhveseOYUdpXTv66DBEo1ZXSpCfCtwMVAF3AzcU\n2OYWYAywCfg6MKtM9QlCT3vBgjCeOX8+zJ4ddlj17Bl2So4cCaef7htJe04qBQcfHNr48SHYlywJ\nP8f34ovhp/rmzQvDL4cfHo46rWv9+vlNcA8rFuRVwK3AaGAFMBOYDOQfLjYWGAoMA44F7gBGlb3S\nVpLJZEin063/xJs3hxMivfpqfVu0KAT3W2+FN9DHPx6GRy6/nMzWraRPO63162yBxP6WbVRF/T1T\nqRDaQ4fCV78a1m3eDPPmkZk0ifTKlWFmzLx5sGVLeO0OGxaOS6hrQ4dCr16J/RMq6u+5m4oF+Uhg\nMfBa7vqDwBdpGOTjgIm55RlAD6AvsKpsVbaisv/nbt0axhpXrw4nOFq5MrT85WXLwu0DBsDgwaEN\nGgTHHx/eAEOGhPnA+XXW1hrk7UzF/z07d4ZjjiHzxBOka2vr169eHb5NLlkS2qOPhsvFi8O4e//+\nodd+4IENLw84IEyZ7N07nNO+zL36iv97tkCxIO8HLM+7/gah111sm/7EEuTZbDjKbevW0DZtCj3g\nrVtDT2LTptA2bmz6cuPGENZ1be3a+uX33w/zdffbL7xADzggXA4eHIL6wANDgPfvv1NYS23CfvuF\ndtJJDddns+EgpRUrQnvzzXA5e3bYwbpyZbh97drwPurZsz7Ye/eGffeFrl0btm7dGl7v0iV8wHTq\nFC7z29at4b3fBo5WLpYcpR4d0PijsvD9Tjkl/Oc1bjt2FF7f3G0tuc+OHfVBXah16BB23HTsGKbo\nTZwYljt1CjsL99mn6ct99oG+feGjHw0vtJ49w9fFuuVu3RwfLJMOHTpQXf0Ie++98MN12ewmNmxI\nsCjtulQK+vQJrdjvtm7ZEgJ9zZr6tn59mGJb15YvD5cbNtSv++CDcN/Nmxu2LVvC7ddfH97/nTuH\n93yHDiHY61qx6/nr6o6OTaXqW/715m5r6vrIkaX9KYvcPgqoJezwBLgK2EHDHZ53AhnCsAvAQuBE\ndu6RLwaGlFSVJKnOEsJ+yF1WnXuQGqATMBv4WKNtxgJTc8ujgL/uzhNKkspvDPAPQo/6qty68blW\n59bc7XOAo1q1OkmSJEktcwVhnD25CaTN+y/CN4vZwNPAgGTLadKNhGmgc4BHgH2TLadJZwAvA9up\nzG9rpxL26ywCfpBwLU35FWGf07ykCyliAPAM4f97PnBpsuUU1IUwTXo2sAC4PtlyiqoiHGD5eNKF\n5BsA/AFYSuUGebe85UsIR7NWos9Sf/bKn+ZaJfoocDDhDV5pQV5FGBKsATpSeB9QJfgUMILKD/L9\ngbpf3u5KGJKtxL9n3Xkrqgn79E5IsJZivgvcTzgQs0mtfRrbnwP/0crP2VL5k9m6Av9KqpAiniJ8\ns4HQw+ifYC3NWQj8M+kimpB/wNtW6g94qzTTgXVJF1GCtwgfhgDvEb4xHphcOU2q+2XqToQP87UJ\n1tKc/oTJJHdTZIZhawb5FwkHC81txefcVf8NLAP+jcrt6eb7BvUzh1S6Qgez9UuolramhvAtYkbC\ndRTSgfCBs4rwTXFBsuU06Sbg+9R32JpU7kMJnyJ8vWrsGsKMl8/lrUvyKJmm6ryaMBZ1Ta5dSfhj\nntd6pTVQrE4IdW4BfttaRRVQSp2VyJ/D2TO6Ag8BlxF65pVmB2EIaF9gGpAmHAtTSb4ArCaMj6eT\nLaXeoYRPv6W5tpXwdXa/BGsqxUDCTptK9XXgBcIOnEpXiWPkowj7bOpcReXu8Kyh8sfIIexrmAZ8\nJ+lCSvQj4HtJF1HA/xC+LS4FVgIbgXsSraiASt7ZOSxv+RLg3qQKKeJUwuyAPkkXUqJngKOTLqKR\nUg54qxQ1VH6Qpwhhc1PShTSjD+HEfgB7Ac8BJydXTklOpEK/2b5K5Qb5Q4Q3zGzgYSr3W8Mi4HXC\nV69ZwO3JltOkLxF6Fu8TdoY9mWw5Oyl0wFuleQB4E9hM+FsmNdRXzAmEYYvZ1L8uT232Hq3vMOAl\nQo1zCWPQle5EisxakSRJkiRJkiRJkiRJkiRJkiRJkhSR/wdOBa3S30YOmQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sampling = ImportanceSampling(log_p=lambda x: np.log(p(x)), q=norm(loc=0,scale=0.25))\n", "\n", "show_sampling(sampling, plotter=weighted_gauss_hist)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Is there a diagnostic to detect this type of problem?\n", "\n", "In general, detecting the failure of a sampling methods can be quite difficult. Nevertheless the importance weights can give us some information about the effectiveness of importance sampling. We noted above, that the variance of the standard Monte-Carlo estimate using $N$ samples is given by\n", "$$ \\mathbb{V}ar_p[\\frac{1}{N} \\sum_{i=1}^N f(x_i)] = \\frac{1}{N} \\mathbb{V}ar_p[f] $$\n", "Similarly, we could compute the variance of the weighted average used in importance sampling\n", "$$ \\mathbb{V}ar_p[\\sum_{i=1}^N \\frac{w_i}{\\sum_{i=1}^N w_i} f(x_i)] = \\frac{\\sum_{i=1}^N w_i^2}{(\\sum_{i=1}^N w_i)^2} \\mathbb{V}ar_p[f] $$\n", "Thus, by comparing coefficients we can define\n", "$$ N_{eff} = \\frac{(\\sum_{i=1}^N w_i)^2}{\\sum_{i=1}^N w_i^2} $$\n", "as an approximate effective sampling size.\n", "\n", "Note: This derivation is by no means exact, as the weights $w_i$ are actually not fixed, but dependent on the samples $x_i$. Thus, they are in particular not necessarily independent of $f(x_i)$ ..." ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def ess (x):\n", " \"\"\"\n", " Effective sample size of weighted samples x\n", " \"\"\"\n", " w = np.array(map(lambda x: x[1], x)) # extract weight\n", " return np.sum(w)**2/np.sum(w**2)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEACAYAAACuzv3DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNX9//FXgKBgSGRxAYKNhDUQIICRLTK4ILhiv24g\nIirV+nugUrFa91Arlq+0uGHFBav2JyIioq1gUZiIbIJhU6AkQGxYfggKBBBIQs7vjzMZss8NDLlz\nw/v5eNxH7tx7Z/IhGd45c+4994CIiIiIiIiIiIiIiIiIiIiIiIiISNhNBXYCayvZ3wyYC6wCvgNG\n1kxZIiLiVBqQQuVBng48G1hvBvwE1Dv5ZYmICEAdB8csBPZUsX8HEBtYj8UGeeEJ1iUiImGWQOUt\n8jqAH9gO7AcG10xJIiICzlrkoTyK7R9vAXQDJgONwvC6IiLiQDj6svsAzwTWNwFbgPbAipIHJSYm\nmk2bNoXh24mInFI2AW2qOiAcLfINwKWB9XOwIb65XCWbNmGMifjlqaeecr0G1akaVafqLF6AxFAh\n7KRFPg3oj70iJRd4CogO7JsCjAfeAlZj/zA8BPzs4HVFRCQMnAT50BD7dwNXh6EWERE5DuHoWqlV\nfD6f2yU4ojrDxws1guoMN6/U6URUDX4vE+jvERERh6KioiBEVqtFLiLicQpyERGPU5CLiHicglxE\nxOMU5CIiHqcgFxHxOAW5iIjHKchFRDxOQS4i4nEKchERj1OQi4h4nIJcRMTjFOQiIh6nIBcR8Tgn\nQT4V2AmsreIYH7AS+A7wn3BVIiLimJP7kacBB4B3gOQK9p8JLAIuB7Zip4TbXcFxuh+5iEg1het+\n5AuBPVXsHwbMxIY4VBziIiJykoSjj7wt0ARYAKwAbg3Da4o4FhvbhKioqOASG9vE7ZJEapSTyZdD\niQa6A5cADYElwFIgq+yB6enpwXWfz1er5swT9+zfvwcwJR7X5AyGIuHl9/vx+/3Veo7Td3wC8CkV\n95E/DDQA0gOP3wDmAh+WOU595HJS2D7Eku+tKPRek9qipubsnA30A+piW+QXAuvC8LoiIuKAk66V\naUB/7NUoucBT2O4UgCnABmwLfA1QBLyOglxEpMbUZGeiulbkpFDXitRmNdW1IiIiLlKQi4h4nIJc\nRMTjFOQiIh6nIBcR8TgFuYiIxynIRUQ8TkEuIuJxCnIREY9TkIt4SJ06dXjwwQeDjydOnMi4ceOq\n/Tr5+flceumlpKSkMGPGDBYuXEinTp3o3r07hw8fPu76HnjgAVJSUkhJSaF9+/Y0btwYgFWrVtGn\nTx86d+5M165d+eCDDyp8fnZ2NmlpaaSkpNC1a1fmzJkDgDGG++67j06dOpGUlMT9999f6nmPPfYY\n7du3JykpiZdeegmA2bNn07VrV1JSUujRowfz588PHj937lw6dOhA27ZtmTBhQnD7zTffHKz//PPP\nJyUlBYBvvvkmuL1Lly5Mnz69XO3XXHMNycml7yv4wQcf0KlTJzp37swtt9wCwA8//ECPHj1ISUmh\nU6dOvPDCC9X6GbvNiJwMgAFTYqm977XTTjvNtG7d2uzevdsYY8zEiRNNenp6tV9nyZIl5tJLLw0+\nvvvuu80//vGPsNVpjDEvvfSSufPOO40xxmzcuNFkZ2cbY4zZvn27ad68udm3b1+559x2223m1Vdf\nNcYYs27dOpOQkGCMMWbBggWmb9++pqioyBw9etT07t3bZGRkGGOMmTp1qrntttuCr/Hjjz8aY4w5\ncOBAcNuaNWtMYmKiMcaYwsJCk5iYaLZs2WLy8/NN165dzbp168rVMnbsWPP0008bY4z55ZdfzNGj\nR40xxuzYscM0bdrUFBYWBo+dOXOmGTZsmElOTg5u27hxo0lJSTF79+41xhiza9cuY4wx+fn5Jj8/\nP1jjr371K5Obm1vpz5HS95+okFrkIh4SHR3NXXfdxaRJkxwdv2vXLq6//npSU1NJTU1l8eLF7Nq1\ni+HDh7N8+XJSUlJ47bXXmDFjBk888QTDhw8PW63vvfceQ4cOBaBt27YkJiYC0Lx5c84++2x27dpV\n7jnNmzdn3759AOzdu5eWLVsCcPbZZ5Ofn8+RI0c4dOgQBQUFnHPOOQC8+uqrPPnkk8HXOOusswA4\n44wzgtsOHDhAs2bNANu6btOmDQkJCURHR3PzzTcze/bsUnUYY/jggw+C9Tdo0IA6dWxcHjp0iLi4\nOOrWrRt87UmTJvH444+XusfP66+/zujRo4mLiwMIfv/o6Giio6ODrxUdHU3Dhg2r86N1VaV/cURO\nBKdQizwmJsbk5eWZhIQEs2/fvpAt8qFDh5qvv/7aGGPMDz/8YDp27GiMMcbv95urrroqeNzIkSPN\nzJkzK3yNtLQ0061bt3LLl19+Wen3zcnJMc2bNzdFRUXl9i1btswkJSVV+Lx9+/aZpKQkEx8fbxo3\nbmy+/fbb4L7HHnvMnHnmmSYuLs48/vjjwe1NmzY1zzzzjOnZs6cZPHiwycrKCu6bNWuW6dChg4mL\nizPLli0zxhgzY8YMM2rUqOAx7777rhk9enSpOjIyMkzPnj0rrLtBgwbm448/Dm4fM2aM+fjjj01O\nTo7p3LlzcPuQIUPMQw89ZPr27Wt69epl5s6dG9yXm5trkpOTTYMGDczkyZMr/iEG4KBFHo4ZgkSk\nBjVq1IgRI0bw4osv0qBBgyqP/eKLL1i/fn3w8f79+zl48GCFd4esaBvAV199Ve0a33//fW644Ybi\nO/cF7dixgxEjRvDOO+9U+LwHHniAUaNG8bvf/Y6lS5dy66238t1337Fw4UIWLFjAtm3bMMZw2WWX\ncfnll9OvXz+OHDlCgwYNWL58ObNmzeKOO+4I1jxkyBCGDBnCwoULufXWW9mwYYOj+qdNm8awYcNK\nbUtNTeX7779nw4YNDBo0CJ/Px5YtW9i8eTOTJk0iJyen1PGFhYVkZ2eTkZFBbm4uF110EWvXriUu\nLo74+HjWrFnDjh076N+/PwMHDqRNmzYOf7rlKchFPGjMmDF0796d22+/vcrjjDEsW7aM+vXrH/f3\nSktL48CBA+W2T5w4kUsuuaTC50yfPp1XXnml1La8vDyuuuoqxo8fT2pqaoXPW7x4cfDkba9evTh8\n+DC7d+9m6dKlDB48ONgFMXjwYJYsWUK/fv2Ij4/n17/+NWCDu6KfSVpaGoWFhfz888/Ex8eTm5sb\n3Jebm0t8fHzwcWFhIbNmzSIzM7PCGjt06EBiYiJZWVmsWLGCFStWcP7551NYWMiPP/7IxRdfzPz5\n84mPj+fCCy+kbt26JCQk0K5dO7Kzs+nRo0fwtZo3b05aWhqrVq06oSBXH7l4yuHDh3nttdd45ZVX\ngsupqHHjxtx44428+eab5Vq9JQ0cOJAXX3wx+HjVqlXV/l4LFy5k5cqV5ZbKQnzDhg3s2bOHXr16\nBbfl5+dz3XXXMWLEiGDoVqRDhw588cUXAKxfv54jR45w1lln0aFDBzIyMjh69CgFBQVkZGSQlJQE\n2PAuviIlIyOD9u3bA7Bp06bgp4ziUG7atCk9e/YkKyuLnJwc8vPzmT59Otdcc02whi+++IKOHTvS\nokWL4LacnBwKCwsBe9VJVlYW7dq147e//S3btm1jy5YtfP3117Rr1y5Yy5AhQ4Jzb+7evZuNGzfS\nunVrtm3bxqFDhwDYs2cPixYtokuXLk5+FSdkKrATWBviuAuAQqCy31KV/UAiTnz77bfmtNOamtNP\nv8ecfvo9pn7960+pPvJGjRoF13fu3GkaNmxoxo0bV+nxu3fvNjfddJPp0qWLSUpKMvfcc48xxvaR\nX3311cHjquojr6709HTzyCOPlNr27rvvmujo6FJ97KtXrzbGGPPkk0+aTz75xBhjTHZ2tunfv7/p\n2rWr6datm5k3b17wNcaMGWM6depkkpKSzNixY4Pb9+7da6688kqTnJxs+vTpY9asWWOMMWbChAmm\nU6dOplu3bqZfv37mm2++CT7ns88+M+3atTOJiYlm/PjxpWodOXKkmTJlSrn6i1/rggsuMHPmzCn3\n796yZUupq1aMMeaBBx4wSUlJJjk52UyfPt0YY8y8efNMly5dgv/Gt99+u8qfJw76yJ3MEJQGHADe\noeLJl8HO1zkP+AV4C5hZSZA7+HYilcvMzGTAgFHk5RV/7M0EeqAZgqS2CtcMQQuBPSGOuRf4ECh/\nPZGIiJxU4TjZ2RK4FrgY272ippBIDRs/fjwzZswote3GG2/kkUcecakiqUnhCPLngT9gAzyKKj4C\npKenB9d9Ph8+ny8M315EHn30UR599FG3y5Aw8Pv9wZOkTjnpIwdIAD6l4j7yzSVepxm2n/w3wCdl\njlMfuZww9ZHLqcZJH3k4WuStS6y/hQ38siEuIiIniZMgnwb0x7a2c4GngOjAviknqS4REXHISZAP\nrcbrVT3MTEREwk4jO0VEPE5BLiLicQpyERGPU5CLiHicglxExOMU5CIiHqcgFxHxOAW5iIjHKchF\nRDxOQS4i4nEKchERj1OQi4h4nIJcRMTjFOQiIh6nIBcR8TgFuYiIxzkJ8qnATmBtJftvAVYDa4BF\nQJfwlCYiIk44CfK3gEFV7N8MXIQN8KeB18JQl4iIOOQkyBcCe6rYvwTYF1hfBsSfaFEiIuJcuPvI\n7wQ+C/NriohIFZxMvuzUAOAOoG9lB6SnpwfXfT4fPp8vjN9eRMT7/H4/fr+/Ws+JcnhcAvApkFzJ\n/i7AR9i+9OxKjjHGmGoVJ1JWZmYmAwaMIi8vs3gL0AMo+d6KQu81qS2ioqIgRFaHo2vlPGyID6fy\nEBeJGLGxTYiKigousbFN3C5J5IQ46VqZBvQHmgG5wFNAdGDfFOBJoDHwt8C2AiA1vGWKhM/+/Xso\n2YLfv9/pB1ORyOQkyIeG2D8qsIiIiAs0slNExOMU5CIiHqcgFxHxOAW5iIjHKchFRDxOQS4i4nEK\nchHqaYCQeFo477Ui4lGFaICQeJla5CIiHqcgFxHxOAW5iIjHKchFRDxOQS4i4nEKcpETpPubi9t0\n+aHICdL9zcVtTlrkU4GdwNoqjnkRyAJWAylhqEtERBxyEuRvYefirMwVQBugLXAXx2YKEhGRGuAk\nyBcCe6rYfw3wdmB9GXAmcM4J1iUiIg6F42RnS+xcnsW2AvFheF0REXEgXFetlD27Yyo8SkREwi4c\nV61sA1qVeBwf2FZOenp6cN3n8+Hz+cLw7UVEag+/34/f76/Wc5xeJ5UAfAokV7DvCmB04Gsv4PnA\n17KMMWqoy4nJzMxkwIBR5OVlFm8BelD6Q2AUVb3XoqKiyh1fnec7eT291yVc7Pur6qx20iKfBvQH\nmmH7wp8CogP7pgCfYUM8GzgI3H585YqIyPFwEuRDHRwz+kQLEYlUsbFNAoN+rEaNGpOX97OLFYmU\nppGdIiFo5KZEOt1rRUTE4xTkIiIepyAXEfE4BbmIiMcpyEVEPE5BLiLicQpyERGPU5BLrVd2KjaR\n2kYDgqTWKzugx/kthkS8QS1yERGPU4tcaqF66kKRU4qCXGqhQtSVIqcSda2IiHicglxExOMU5CIi\nHuckyAcBG4As4OEK9jcD5gKrgO+AkeEqTkREQgsV5HWBl7FhnoSdLahjmWNGAyuBboAP+As6iSqe\nVk8DiMRTQgV5KnYuzhygAHgfuLbMMTuA2MB6LPAT9rIBEY8qvuqleBGJbKFazi2xEy4X2wpcWOaY\n14H5wHagEXBj2KoTEZGQQrXInTRHHsX2j7fAdq9Mxga6iIjUgFAt8m1AqxKPW2Fb5SX1AZ4JrG8C\ntgDtgRVlXyw9PT247vP58Pl81SpWRKS28/v9+P3+aj0n1JmcesB/gEuwXSffYE94ri9xzF+BfcA4\n4BzgW6AL8HOZ1zLGqL9RTkxmZiYDBowiLy+zeAvQg/IjOU/u45LvZXtCtPL9IicicMK9yqwO1SIv\nxF6V8jn2CpY3sSF+d2D/FGA88BawGttV8xDlQ1xERE4SJ5cJzgksJU0psb4buDpsFYmISLVoZKeI\niMcpyEVEPE5BLiLicQpyERGPU5CLiHicglxExOMU5CIiHqcgFxHxOAW5iIjHKchFRDxOQS4i4nEK\nchERj1OQi4h4nIJcRMTjFOQiIh6nIBcR8TgnQT4I2ABkAQ9XcowPWAl8B/jDUZiIiDgTaoagusDL\nwKXYiZiXA59Qes7OM4HJwOXYiZmbhb9MERGpTKgWeSqQDeQABcD7wLVljhkGzMSGONip30REpIaE\nCvKWQG6Jx1sD20pqCzQBFgArgFvDVp2IiIQUqmvFOHiNaKA7cAnQEFgCLMX2qZeSnp4eXPf5fPh8\nPodlioicGvx+P36/v1rPCRXk24BWJR634lgXSrFcbHfKocDyFdCVEEEuIiLllW3kjhs3LuRzQnWt\nrMB2nSQA9YGbsCc7S5oN9MOeGG0IXAisc1ayiIicqFAt8kJgNPA5NqjfxF6xcndg/xTspYlzgTVA\nEfA6CnIRkRoTKsgB5gSWkqaUeTwxsIiISA3TyE4REY9TkIuIeJyCXETE4xTkIiIepyAXEfE4BbmI\niMcpyEVEPE5BLiLicQpyERGPU5CLiHicglxExOOc3GtFREqpR1RUlNtFiAQpyEWqrZDSc64o1MVd\n6loREfE4BbmIiMcpyEVEPM5JkA/CzgKUBTxcxXEXYDsPfx2GukRExKFQQV4XeBkb5knAUKBjJcdN\nwE75pjM/IiI1KFSQpwLZQA5QALwPXFvBcfcCHwK7wlmciIiEFuryw5ZAbonHW4ELKzjmWuBibPeK\nQSTCNeAXYskjljyigMP8wGFOZw+NKXC7OJFqChXkTkL5eeAPgWOjqKJrJT09Pbju8/nw+XwOXl7k\n+DVjF6nAhTxJZ74jkU20BurRlDxi2U8jioDTuIjTOUxj9vATkEsq6+lIJt1ZCSznEIdp4Oq/RU4N\nfr8fv99freeE6s/uBaRj+8gBHgGKsP3hxTaXeJ1mwC/Ab4BPyryWMUaNdTkxmZmZDBgwiry8zOIt\nQA+K2xx1OEof6nElD3Ml/+I8/sty8ljGo6ymK9m0YTM92FduQM+x559LPc5jMZ34nu5k0oO/kUQM\nS+nFvxnILB5iUyXPL36s97qES2AUcZVZHapFvgJoCyQA24GbsCc8S2pdYv0t4FPKh7jISZXE94zg\nHYbzD3YB/6Qed/Ea35BKEfWAZxy9ThF12Q5spzdL6c2bAPyNWLbiw88g5rII+C8X8D438w4j2H2y\n/lEiDoU62VkIjAY+B9YB04H1wN2BRcQ1dSnkBv7NEuBzLscQxUD+TQrwBH9iKb0pom5YvlcecXzC\ntfwf/kZL4BGeJZm1ZNGWt4FUlqHTQ+KWmrxUUF0rcsIyMzO50nc7Q/ffxv28wA805a+s5FMKS4R2\n+a6Ok/W4CT9xO824h9bs5Bye5gnmcoW6ViRsnHStaGSneMeRI5w1bRorD6yjD4u5ng/pzxvMhrC1\nvKvrZ5ryF6AdG3mR+3iO3/MNwKefgsJcaoha5BL5jh6Fv/8dxo1jX0ICV2X+yNcHNwR2lj7ZadVc\ni7zs4yiKGEJdPurUCZo1g4kToWdPh/9QkfLUIhfvW7wYUlPhrbfggw/Y9PzzrKnb0O2qKmWowyyA\nVavgllvgmmtg+HD473/dLk1qMQW5RKbt2+HWW+HGG2HsWFi4EHr1crsq5+rVg9/8BjZuhNatoXt3\neO45KNBwIwk/BblElqIimDwZunSBVq1gwwYYNgy8OiNPTAz88Y+wdCl8+aUN9EWL3K5KahnNECSR\nY8MG24otKrIt8I4V3Z/No9q0gTlz4MMP4aab4IorbAs9Ls7tyqQWUItc3FdQAOPHQ79+NuRqW4gX\ni4qCG26AdeugTh1IToZ//9vtqqQWUItc3PXdd7Yv/Nxz4dtv4Ve/cruiky82Fl59FebNg1GjYOBA\n+Mtf7HaR46AWubijqAheeAEGDIB774XPPjs1Qrykyy6DtWttSz052Qa7yHFQi1xq3vbtMHIk7N9v\nTwImJrpdkXtiY+G112wXyx13wP/8D/z5z3D66W5XJh6iFrnUrI8+gpQU6NvX9oWfyiFe0sCBsHq1\n/SN3wQW2pS7ikFrkUjP274cxYyAjA2bP9tY14TWlSROYPh3eeQcuvhgeewzuu8+eGBWpgt4hcvIt\nXWpb4VFRsHKlQrwqUVFw2232ZzZ9OgwaZFvpIlVQkMvJU1gI6elw7bXwv/8Lb7wBjRq5XZU3JCba\nrqe+fe0golmz3K5IIpi6VuTkyM629xiJjbWt8BYt3K7Ie+rVg6eesv3nw4fDv/4Fzz9vR4uKlKAW\nuYSXMTB1KvTuDUOHwty5CvET1bu3vQlXUZHtolq2zO2KJMI4DfJBwAYgC3i4gv23AKuBNcAioEtY\nqhNv2b0brr/ethrnz4f779eJunBp1Mj+gXz2WXtHxT/+0XZdieAsyOsCL2PDPAk7Z2fZ8dObgYuw\nAf408FoYaxQvmDsXunaF88+Hb76xA1wk/K6/HjIzbf95Whps2uR2RRIBnAR5KpAN5AAFwPvAtWWO\nWQLsC6wvA+LDVJ9Eul9+sSMz77oL3n3XTqSgwSwnV8uW8PnncPPN9gqgqVM1G9EpzkmQtwRySzze\nGthWmTuBz06kKPGIzEzo0QN++skOZrn4YrcrOnXUqWO7rubPt11Z119vfw9ySnJy1Up1/tQPAO4A\n+la0Mz09Pbju8/nw+XzVeGmJGEeP2ssJJ02yITJsmNsVnbqSk21X1uOP266tqVPtVS7iWX6/H7/f\nX63nOLlbfy8gHdtHDvAIUARMKHNcF+CjwHHZFbyO5uysDTZutPcEqVfPjkA877wa/faZmZkMGDCK\nvLzM4i1E0pydxY9dea9/+aW9h81119mTomecUfM1SNiFa87OFUBbIAGoD9wEfFLmmPOwIT6cikNc\nvO7oUdv/3aePnX5t/vwaD3EJ4ZJLbBfX3r12hqX5892uSGqIk66VQmA08Dn2CpY3gfXA3YH9U4An\ngcbA3wLbCrAnSaU2WL8ebr8dGjSw1zDrRleRq0kT+0npX/+yQ/2vvNJ2g+le57Wa04t85wDtgTbA\ns4FtUwILwCigKZASWBTitUFBgb2laloajBhhP7orxL3hyivtpB1FRdC5s51mTmotDdGXii1aBL/9\nrR2VuWIFJCS4XZFUV1ycvdf5l1/auVD79LHdY+ee63ZlEmYadiel7d4Nd95p58584gk70Ech7m2X\nXAJr1kB8vL3K5cUXNSq0llGQi1VUBG++CZ062eHg69bZk5pRTi5skogXE2O7yb76Cj7+GHr2tJ+6\npFZQ14rAggUwdqwdkTl3rr0xk9ROHTvarpbp0+2nrosvhj/9SVcgeZxa5KeyDRvsDZjuvBP+8Afb\nQlOI135RUXZ4//r1NsBTUuzvf+9etyuT46QgPxVt3Qr33GOvRunf3/6HVjfKqadRI9saX7PGnhtp\n3x5eeAGOHHG7MqkmBfmpZNs2GD3aDuWOjbUBPnYsnHaa25WJm1q2tLM3ffEFzJsHbdrAyy/D4cNu\nVyYOKchPBbm59g6Fycl2UM/69TBhAjRr5nZlEkmSk+Gf/7TTys2bZ8cMTJpk73ApEU1BXpt9+629\noVW3bvZE5vr18NxzcPbZblcmkaxnT5g924b611/be8w/8YQmgY5gCvLaJj8fZs60fd/XXWdvM7t5\nsw3wc85xuzrxkpQU+17KyICff7aXpg4fDsuXu12ZlKEgry3+8x946CF7FcJLL9mTmZs22T7wuDi3\nqxMv69ABJk+2DYJu3eCGG2yrffJkG/DiOgW5l+3aBVOmwEUX2RZ4VJQd8OH328vLoqPdrlBqk8aN\n4cEHbQPhmWdst0vr1vaKp88+s/fmEVcoyL1m9254/XW47DJo29aG9tix9oTmhAnQrp3bFUptV7cu\nXH45TJsGW7YcG1R07rn2jouzZ8OhQ25XeUqpyQuHNbHE8SgqsjetmjPHLuvWwaBBthV0xRXQsKHb\nFdYoTSwRwbZutcP/Z8600wBeeqmdrWjgQHvCVI6Lk4klFOSRprDQDtD4+ms7U3pGBpx1FgwebJd+\n/U7p674V5B7x44/2dg/z5tklJsYGelqavQvjeedpAJpD4QryQcDz2Ekl3qD8FG8ALwKDgV+AkcDK\nCo5RkJdVVGQ/mq5caZfly2HpUmjVyr7h+/Wz/d+6D0aQgtyDjIG1a22gL1oEixfbyaN797ZL9+52\nRiONa6hQOIK8LvAf4FJgG7AcGIqdIajYFdgZhK4ALgRewM7zWZYngtzv94d/Uuj8fBvYWVl2zsuN\nG20XyerVcOaZ9kqAlBR7qWCfPtC0qTt1ngThrvNUDvJa8zs3BnJyYMkSu6xaZT+FnnGGDfTkZHul\nTJs2dlBSixY2+Gu6zgjhJMhD3f0wFTsHZ07g8fvAtZQO8muAtwPry4AzgXOAndWqNkJU+5d7+DD8\n9BPs3Gn7CLdutUPhi9d/+MF+jY+3JyLbtrVv1BtusAHuILTDUqdLvFKnF3jlZxmyzqgo22d+/vl2\nwBrYcM/NtYG+Zo3tVvz73yE7G/bts8cmJtpPqy1aHFuaN7dfmzSpdth75efpRKggbwnklni8Fdvq\nDnVMPJEW5EVFtmVcUGC/Fi8FBTaMDx6EAwfs9djvvWfXi7cdPAj799vALrvk59swPvts+yaLj7eL\nz2e/tmplL9GqX9/tn4BI5IqKsl2I550HV11Vet+BA/Ya9k2bbCNp+3Z77mj7drts22b/f555pg30\nsktcnG3tFy8xMfZrTo7tzjzjDDvyuX790kt0NNSr54m+/FBB7vTzYdl/acXPu+wy+5e3qKjir1Xt\nc3pMUVHFYV1UVPoXVPaXFhNjl61b7S+u+JcdE2PfIPHxNrDLLjExnvhF1xZ16tTh8OFNxMZeDcDR\no3s5eNDlouTkiomxXS5dulR+TEEB7NljByj9/HPp9b177dfcXNsoK16+/97Oa3rggL3jY9ncyM+H\no0fLZ0W9erb1X7eu/VrRUtm+4u1RUccWqHzd4excoRKoF5COPeEJ8AhQROkTnq8Cfmy3C8AGoD/l\nW+TZgGYFNCd+AAADCElEQVTuFRGpnk3Yie+PW73AiyQA9YFVQMcyx1wBfBZY7wUsPZFvKCIi4TcY\ne+VKNrZFDnB3YCn2cmD/aqB7jVYnIiIiIiLVMxbbz97E7UIq8TT2k8Uq4EuglbvlVOo57GWgq4GP\ngEi9xeENwPfAUSLz09og7HmdLOBhl2upzFTsOae1bhcSQitgAfb3/R1wn7vlVOp07KXSq4B1wLPu\nllOlutgBlp+6XUhJrYC5wBYiN8gblVi/FzuaNRJdxrGbnv05sESiDkA77H/wSAvyutguwQQgmorP\nAUWCNCCFyA/yc4FugfUYbJdsJP48AYpvUlQPe16vn4u1VOUB4P8Cn1R1UE3f/fCvwEM1/D2ra3+J\n9Rhgt1uFhDAP+8kGbOsi3sVaqrIB2Oh2EZUoOeCtgGMD3iLNQmCP20U48P+wfwwBDmA/MbZwr5wq\nFc9fVx/7Bz0Sb6wej72Y5A1CXGFYk0F+LXaw0Joa/J7H6xngv8BtRG5Lt6Q7OHblkDhX0WC2li7V\nUtskYD9FLHO5jsrUwf7R2Yn9tLjO3XIqNAn4PccabJUKNSCouuZhP16V9Rj2ipeBJba5OYqmsjof\nxfZFPRZY/oD9Yd5ec6WVEqpOsHXmA+/VVFEVcFJnJIr8m/94UwzwIXA/tmUeiYqw3UBxwOeADzse\nJlJcBfyI7R/3uVvKMZ2xf/m2BJYC7MfZSJ8F+DzsSZtINRJYhD15E+kisY+8F/acTbFHiNwTnglE\nfh852HMNnwNj3C6kGp4AHnS7iDLGYz8tbgF2AAeBd1ytqAKRfLKzbYn1e4F33SokhEHYqwO8cu/P\nBdjbFEYSJwPeIkUCkR/kUdiwmeR2ISE0w97cD6AB8BVwiXvlhNSfCP1ku5nIDfIPsf9hVgEzidxP\nDVnAD9iPXiuBV9wtp1LXYVsWh7Anw+a4W045FQ14izTTgO3AEezP0q2uvlD6YbssVnHsfTmoyme4\nIxl7/+NV2HN2v3e3nJD6E+KqFRERERERERERERERERERERERERERERERERHxkP8PkBJHSGUDTw8A\nAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plt_ess (ax, weighted_data):\n", " weighted_gauss_hist(ax, weighted_data)\n", " y = 0.8*ax.axes().get_ylim()[1]\n", " ax.text(1, y, \"N_eff = \"+str(ess(weighted_data)))\n", " \n", "show_sampling(sampling, plotter=plt_ess)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Some theoretical remarks on importance sampling:\n", "\n", "* What is the optimal choice for $q$?\n", "\n", " Remember that the variance of a Monte-Carlo estimate for $\\mu = \\mathbb{E}_p[f]$ is given by $\\frac{1}{N} \\mathbb{V}ar_p[f]$. In importance sampling, the integrand is changed to $f \\frac{p}{q}$ and the expectation is then evaluated with respect to the sampling density $q$. It is easy to see that the optimal choice would be $q(x) = \\frac{p(x) f(x)}{\\mu}$ with variance\n", " $$ \\mathbb{V}ar_q[ f \\frac{p}{q} ] = \\mathbb{V}ar_q[ \\mu ] = 0 $$\n", " Note: While optimal this choice is infeasible since the normalization constant of this $q$ is equal to $\\mu$, i.e. the mean of $f$ which is to be computed in the first place.\n", "\n", "* In general, importance sampling can be considered a variance reduction technique. We compute $$ \\mathbb{V}ar_p[f] - \\mathbb{V}ar_q[f \\frac{p}{q}] = \\int f^2(x) (1 - \\frac{p(x)}{q(x)}) p(x) dx $$\n", " Thus, importance sampling reduces the variances when $\\frac{p(x)}{q(x)} < 1$ if $f(x) p(x)$ large, while allowing for $\\frac{p(x)}{q(x)} > 1$ if $f(x) p(x)$ is small." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Example**: Pricing an (out-of-the-money) Asian call option with importance sampling\n", "\n", "Consider a Black-Scholes world, i.e. the stock price $S_t$ follows a geometric Brownian motion, i.e. $$ S_t = S_0 e^{(r - \\frac{\\sigma^2}{2}) t + \\sigma W_t}$$ where $W_t$ denotes a Brownian motion.\n", "\n", "Now, assume that an Asian call with strike $K$ and expiring at time $T$ pays\n", "$$ a(S) = \\max ( 0, \\frac{1}{M} \\sum_{i=1}^M S_{i \\frac{T}{M}} - K ), $$\n", "i.e. the stock price is averaged over days $i \\frac{T}{M}$ for $i=1,\\ldots,M$ and then compared with the strike $K$. Using the above formula for the stock price, we can compute $S_{i \\frac{T}{M}} = S_0 e^{(r - \\frac{\\sigma^2}{2}) i \\frac{T}{M} + \\sigma \\sqrt{\\frac{T}{M}} (X_1 + \\ldots + X_i)}$ where $X_i \\sim \\mathcal{N}(0,1)$ since the Brownian motion is the cumulative sum of standard normal increments.\n", "\n", "The risk-neutral option price is then computed as\n", "$$ \\mathbb{E}_S[ e^{-rT} a(S) ] $$\n", "\n", "Below, we compute the price for an option that is vastly out of the money, i.e. $K \\gg S_0$" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEACAYAAAC+gnFaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcE/X9x/FXdpcVD5BC5T5F9AdFhaqIiLK0VgErhyjg\nheIBFlEKavH4qVBErLXFUhRR0aKtgKJSqhxFZPFAQETwACqgKKDiUWmVnxYW9vfHZ7bJhmQzSSaZ\nbOb9fDzy2GQyM/nsbHY+8z0HREREREREREREREREREREREREREREKukBbAQ2AWPirDPZeX8d0DFi\neR1gDrABWA90zlyYIiKSCYXAZqAlUANYC7SNWqcXMN95fjKwIuK9GcDlzvMi4PBMBSoiIplxCrAw\n4vVNziPSg8DAiNcbgQbYSf+DjEYnIiJpK0jwfhNgW8Tr7c6yROs0BVoBXwCPAWuAh4FD0glWRES8\nlygRlLvcTyjGdkXAj4EHnJ+7ObA0ISIiPitK8P4OoFnE62bYFX9V6zR1loWcdd9wls8hRiJo3bp1\n+ZYtW5IIWUREgC3AUV7sKFGJYDXQBmssLsbaAuZFrTMPGOw87wzsAnYCn2FVRkc7750BvBf9AVu2\nbKG8vDwwj337yvn++9jv3XHHHb7HlysPHQsdCx2Lqh9A66TP+HEkKhGUASOARVgPoulYV9BhzvvT\nsB5DvbDeRbuBIRHbXwv8BUsiW6LeC6Snn4Znn4XZs/2ORETEJEoEAAucR6RpUa9HxNl2HXBSskHl\ns927YcEC2LMHiov9jkZEJHHVkGTAN9/Aa68duLykpCTrseQqHYswHYswHYvMiO7t44dyp74rEKZP\nhyuvhBtugN/+1u9oRKS6CoVC4NE5XCWCLCsvh/btrXpIRCQXKBH44MQT4fPP4aOP/I5ERESJIOvK\ny6GwEHr0UKlARHKDEoEPQiHo2RPmz0+8rohIpikRZFlFu/iZZ8KyZfCf//gbj4iIEkGWlZdbiaBe\nPWs0XrbM74hEJOiUCHwQcjp89e4Nf/2rv7GIiCgRZFnkkIm+fWHuXNi/3794RESUCLKsomoI4Jhj\noHZtWL3a35hEJNiUCHwQihgL2K+flQpERPyiROCBf/0Ldu1yt270bBp9+8Jzz3kfk4iIW0oEHpgy\nBW691d26kVVDYKOMv/kGNm7MTGwiIokoEXigrAxeeim1bQsKoE8fVQ+JiH+UCDyycSN8+mni9aJL\nBBDuPSQi4gclAg9U1PuXlrpbNzoRlJTA++/Djh1eRyYikpgSgUfq1oWlS1PbtkYNqx56+mlvYxIR\ncUOJwCPdurlLBLFKBACDBuk+xiLiDyUCj7Rvb11It2+ver14ieAnP4HNm+HDDzMTn4hIPEoEHigv\nt94/bksFsdSoAf37w1NPeRubiEgiSgQeCYWge/fEDcbxSgQAAweqekhEsk+JwEPdu9t4gujRw5Gq\nSgSnn25dUN9/PzPxiYjEokTgobZtYe9eq+tPRWEhnH++SgUikl1KBB6oKAGEQnbnsUWLql43XokA\nrPfQzJlVlypERLykROCRipN7jx6wcGH89RIlgs6d4fvv4a23vI1PRCQeJQKPnXEGvPxy6vciLiiA\nwYPhT3/yNCwRkbiUCDxWty60awevvRb7/UQlArBEMHMm7NnjfXwiItHcJIIewEZgEzAmzjqTnffX\nAR0jlm8F3gbeAlalHGWOi67PT1Q9lCgRHHmkNTzPn59+bCIiiSRKBIXAFCwZtAMuANpGrdMLOApo\nAwwFpka8Vw6UYMmhU/rh5q7Ik/tZZ8VvMHbbCHzppTBjRvpxiYgkkigRdAI2Y1f2e4FZQJ+odXoD\nFaeslUAdoEHE+wmuf/PPSSfBtm3wyScHvuemagisG+nSpfDll97HJyISKVEiaAJsi3i93Vnmdp1y\n4EVgNXBV6mHmtuir/KIiazSOVypwkwhq14azz7a2AhGRTEqUCNz2Zo93auuKVQv1BK4BTnO5v2on\n+uTeqxe88MKB6yUzPuDSS1PvPfTmm1a1tH9/atuLSHAUJXh/B9As4nUz7Iq/qnWaOssAKipHvgCe\nw6qaXon+kLFjx/73eUlJCSUlJQnCyn1nnw0jR9qYgJo1w8vdVg0B/PSnVjX05ptwwgnJff7q1TBm\nDNx2mzU6t2+f3PYikltKS0spdXP3qxQkSgSrsUbglthJfSDWYBxpHjACaz/oDOwCdgKHYI3N3wCH\nAmcC42J9SGQiyBdHHAHHHWf1/D17Vn7PbSIoLISrroJp0+Chh5L7/PJy+Ne/7DF+vKatEKnuoi+S\nx42LeTpNSaKqoTLsJL8IWA/MBjYAw5wHwHzgA6xReRow3FneELv6X4s1Ij8P/N2zyHNIvOqe3r1h\n3jx368ZzxRV257J//zv5mLp0Cd85bf365LYXkeBwM45gAXAM1kV0orNsmvOoMMJ5/3hgjbPsA6CD\n82gfsW1einWVX5EIIk/+yVQNATRqZDetefLJ5OIpL7cSyccfw6hRcOedyW0vIsGhkcUZdMwxUKuW\n1fFHSiYRAAwbZtVDyZQmKhLOoYfCiBGweDFs3Jjc54pIMCgRZFh09VAqs4qecYZVDb3xhvttIkse\ntWpZw/WECcl/tojkPyUCD1R1co+VCJItERQUwNCh8OCDycUU+TnXXmvTXuimNyISTYnAI/FO7qec\nYncd27Il8bpVGTIEnnvO/Ujj6ERw+OFw3XXgYUcDEckTSgQZVlgI555rPX8g9RvO1K8P/frBww+7\nWz9WyWPUKFiyBNauTS0GEclPSgQeSHRyHzCgciJIpUQAVs9///12O8xE9u8/8HMOOwxuvRVuuSW1\nzxeR/KRE4JGqTu6nnw47doTvZZxqIjj+eDj6aJgzJ/G68RLO0KGwYQMsW5ZaDOn67DM7FiKSO5QI\nsqCwEPr3t1JBuvci/uUvYdKkxPuJlwgOOgh+/Wu4+WZ/7ov80EPQvTt8+62VbFK9k5uIeEeJIEsG\nDICnnkqvaghsDqOvvoIVK6per7zcehvFcuGFdiKOHvWcDfv2WYlgxAibFK9zZ/juu+zHISJhSgQe\ncHNl3bWrVYu8/356iaCw0Hr/TJqUOKZ4n1NYCHfdZW0FZWWpx5KK8nK45hpLZFOn2tQXI0dmNwYR\nqUyJwCOJTu6FhXDeeeFG43QMGQIvvQQffBB/nUQlj7PPtp5IyU5ml67ycmu0njUL3n3XejItWwZP\nPJHdOEQkTIkgiwYOtCvwdEoEYDetGTYMfvvb+OskSgShENx3n40r+Prr9OJJRYcOdtOd/v2t8Xv0\naHjvvezHISJKBFnVpYv9fOed9Pc1cqRdVX/6aez33bRFHH889O1rjcfZEhnXuefabT2PPdaS2nnn\nWduFiGSXEoEH3DYAFxTYya9Nm/Q/s359uOSS+G0FbmMaP96qZbI1IV28uC67zBLl0KH+9GYSCTIl\ngix75hmY6NGE3DfcANOnx67acZsI6teHm26yfWVDVXFNmQL/+Af8/vfZiUVEjBJBNda8OZxzjo02\njpZMN9XrrrMT8MKF3sYXT7y4Dj7Y5lP63e9g0aLsxCIiSgSe8LMqY8wY+OMfYffuysuTSQTFxTB5\nsvXtz3Sf/kTHqnlzG28xeDBs2pTZWETEKBF4JN2eQKlq2xa6dTuwVBBrrqGq9OxpPXnuvtvb+KK5\nSVBdu1rbRe/eyd+iU0SSp0SQB8aOhXvvrXzSrGpkcTz33WcJJZP3LHBbUhk61KaiuOgiS2oikjlK\nBHmgXTs480z4wx/Cy1KZyqJpU/jf/4XhwzNX3ZVMXPfdZ8ltzJjMxCIiRonAA7nQ3fGOOywRVPQg\nSnVOoxEjbC6jmTO9jS8VxcXw7LM2J9LUqX5HI5K/lAg84lcbQYU2baBPn3DXy1QTQVGR3RLzhhvg\nn//0NsZU4qpXD+bPt0FvL7zgfTwiokSQV267DR54wG5nmc4spyefbKN8R43yNj5ILa7Wra1kcNll\nsGaN9zGJBJ0SQR5p2RIGDbKZRdOd7nriRHj1Ve+vwlON65RTrKTSuzd8/LG3MYkEnRKBB9I96Xrp\n9tttnv8PPkgvpkMPtVHLV18Nu3Z5F186x6p/f5ucrlevzFRbiQSVEkGeadAgPCFdusmppMRGLo8e\n7Ulonhg1Cnr0sGm0owfRiUhqlAjy0OjRlhC8KKX85jd27wOvpp9It/QUCtlMpW3b2gR+utWlSPqU\nCPLQYYfBo4/agKx01aoFjzwCV13lTXWMF9VooZDdUOeQQ2wG1n370o9LJMiUCDyQS20EFXr2tAZW\nL5xxhvUi8mKKaK+OVVGRjXX46iv4xS9yYyyHSHXlJhH0ADYCm4B4YzwnO++vAzpGvVcIvAX8LcUY\nJQdMnGhTTzz2WPr78ipp1qwJc+fC2rXwq18pGYikKlEiKASmYMmgHXAB0DZqnV7AUUAbYCgQPQZ0\nJLAe0L9pNVazJjz5pE33kM6soF6frGvVggULYPFiuOUWJQORVCRKBJ2AzcBWYC8wC+gTtU5vYIbz\nfCVQB2jgvG6KJYpHgByrPPFOUE4+7dvbVBYXXQR796a2j0xUo9WrBy++aGMebrstOH8PEa8kSgRN\ngG0Rr7c7y9yuMwm4Ecj7+SNzrY0gU665Bo44whJCKjLVnvLDH8KSJfDXv9psrCLiXlGC991eW0X/\na4eAnwOfY+0DJVVtPDbiP7ekpISSkipXFx+FQtZOcMIJcNpp1iidjEw2rB9xhCWD7t1tCu5Uk5VI\nLiotLaW0tDQj+06UCHYAzSJeN8Ou+Ktap6mzrD9WbdQLqAnUBh4HBkd/yFhdwlUr9etbj53+/WHV\nKmjRIrntM1l6ql/fxj10727dSseNC05pTfJb9EXyuHHjPNt3oqqh1VgjcEugGBgIzItaZx7hk3tn\nYBfwGXALliBaAYOAl4iRBPJBLnYfzbSuXeHGG2HAANizx/122ai/b9AASktt+urRo9VmIJJIokRQ\nBowAFmE9f2YDG4BhzgNgPvAB1qg8DRgeZ1/6d8wz118PjRrZlNVuZStp1q8PS5fCihU2/kGDzkTi\nS1Q1BLDAeUSaFvV6RIJ9LHMekkdCIfjTn6y94NRTYeDAxNtks/T0gx9Yt9I+fayn0+OP281uRKQy\njSyWtNSpA3Pm2J3N1q1LvH62q9EOO8y6lf7f/9ncRN99l73PFqkulAg8EMQ2gkgdO8KUKXbl/fnn\nfkdzoJo14ZlnoHZtu7ezprAWqSxwieC772x+GvHWwIFw8cU2J1FVjcd+Jc0aNeDPf4bOna2h+6OP\nsh+DSK4KXCJ4+mlrPBTv/frXULeuVRPF66njZ+mpoMCmsB42zNo01q71Jw6RXBO4RLB3Lyxf7m2X\nQnVPNAUF8MQTdnzvvz/2OrlQjTZyJEyaZNVEixf7G4tILghcIgD47DPYti3xesnw++SWK2rVsv77\nEybA88/HXicXjtX551sj98UXW88nkSALZCIAWLnS7wjy15FH2pw/Q4bYyONIuVR6Ov10G3h25502\njbXGGkhQBS4RVJyIVqzwN45816kTTJ9uPYm2bAkvz4WqoUht29pFwRtvQN++8M03fkckkn2BSwRg\nV6xeJoJcO7nlit69beK3nj3hiy9sWS4eq3r1YNEiaNwYunSBDz/0OyKR7ApkIjjxROsxkswcOZKa\nq6+2LqW9e8Pu3bmZCMBGHD/4oN2buUsXeOUVvyMSyZ5AJoJataB1a3cjYSV9EybA0UeHxxjkYiIA\ni+u666zxuH9/mDw5t9o0RDIlcImg4h+7c2d4/XV/YwmKUMjaC4qL4amn/I4msbPOsqrDxx6zOYq+\n/dbviEQyK3CJAOzE1LWrd8X/XK3uyCVFRTB7Nvz859Ak+h53OejII208RHGxXTS8/77fEYlkTiAT\nAUC3bvDyyyr6Z1PNmjB3LvzsZ35H4s7BB1up4Npr7cLhuef8jkgkMwKbCFq0sH/0jRv9jkRyWShk\nU1K88AKMGmWP//zH76hEvBW4RBBZAujWDZZ5cJcEVQ3lv5NOgjVrYOtW61WkqiLJJ4FLBBA+aXuV\nCCQY6taFZ5+FK66wSesef9zviES8EchEUKEiEaidQNwKhWD4cFiyBO6+Gy65RKORpfoLXCKIPOkf\neSQUFsLmzf7FI9XTccfZtBQHH2w35lm+3O+IRFIXuEQQKRSCkhJ46aX09qM2gmA69FB46CG45x67\nDebNN6shWaqnQCaCyJP2z34Gf/+7f7FI9XfuuTZKfcMGm2xPI9aluglkIoh05plWIigr8zsSqc4a\nNLBxBtdfD2ecAXfdpe+UVB+BSwTRDcMNG0LLlro/gaQvFILBg+HNN+3iomtXeO89v6MSSSxwiSCW\ns86yaYhTpTYCidS8uVU3DhlibVB33KG2A8ltgUwE0SftdBOBSLSCAhuRvHYtvP02dOgAr73md1Qi\nsQUyEUQ79VRr6PvyS78jkXzTpIm1HUyYAAMG2BiEf//b76hEKgtcIog1eKy42Irwixenvl9VDUlV\nzj3X2gvKyuBHP7LpuDWQUXJF4BJBPOecA/Pmpbat/qHFjTp1bNzBk0/CnXda1+UNG/yOSsRdIugB\nbAQ2AWPirDPZeX8d0NFZVhNYCawF1gMT04rUQ7Gu3s85BxYuVKOeZN5pp9kEdr17w+mnw403apoK\n8VeiRFAITMGSQTvgAqBt1Dq9gKOANsBQYKqz/HugO9ABOM553tWTqDOgYUNo1w5KS/2ORIKgqMhu\ni/nuu/DFF9C2LcyapdKl+CNRIugEbAa2AnuBWUCfqHV6AzOc5yuBOkAD5/X/OT+LsaTyz/TCTV9V\n/2h9+9qNU1LZp9oIJBUNGtg9kmfPtknsunWD1av9jkqCJlEiaAJsi3i93VmWaJ2mzvNCrGpoJ7AU\nqyLKWX36WDvB/v1+RyJBc+qpNhDtkkusyuiSS2DbtsTbiXihKMH7bguq0dfDFdvtw6qGDgcWASVA\nafTGY8eO/e/zkpISSkpKXH5sauJdvR99NBx+uM0qefLJGQ1B5ACFhXDVVTBokE1k16ED/OIXMGYM\n1Krld3Tit9LSUkozVHedKBHsAJpFvG6GXfFXtU5TZ1mkfwEvACeSIBH47dxz4emnlQjEP7Vqwfjx\nMHQo3HqrXaCMGweXX25tCxJM0RfJ48aN82zfiaqGVmONwC2xev6BQHQny3nAYOd5Z2AXVhX0Q6y9\nAOBg4GfAW2lHnKZEjXEXXmiNdslUD6mNQDKhWTO7C9rzz8PMmdC+vV2kqOqyetiwofp0D06UCMqA\nEVi1znpgNrABGOY8AOYDH2CNytOA4c7yRsBLWBvBSuBvwBIPY8+Idu2gXj145RW/IxExJ5xgk9hN\nnmxVRieeCAsWqIdRrrvlFvvbHXecjRvJ5ftcuxlHsAA4BusiWjEWYJrzqDDCef94YI2z7B3gx4S7\nj/7Wg3g9kejq/cIL7QpMJFeEQjZl+qpVcNttcMMNNgZBFyz+eO01OP98qz3YvTv2Ot99B3PmwAMP\nwM6d1iOsY0eYOBG2bMluvIloZHEMgwbZH3DPHvfbqGpIsiEUgn79bCK7oUPh0kuhRw9YscLvyIJl\n/Xr48EOYMQMaN7Zzxty58P334XW+/x4OOcSmI//jH2H7drjvPusN1qWLlezuuQe2bvXt1/ivwCUC\nN8XpFi3gf/7H/dxDKqJLthUWWhfTjRtt/MugQTZlxcsv+x1ZMOzbZ9U+CxbY1X337vCHP0CjRnDZ\nZTZLwe7dcNBB4W0KC61U8MADsGOHJYEtW+Ckk+zOdhMn2t/TD4FLBG5ddJFle5FcVlwMV19t9c+D\nBlnPom7d4MUXdYGSSWVldmIH+OEPbcrxpUttYsGOHa2X1+rVULt27O2LiuAnP4Fp0+DTTy0J7Nhh\nd7dr29baF1atyt7fMJCJwE01zgUX2M1FNDW1VAfFxXDFFXZFeeWVMGKEVT/Mn6+EkAn79sXuytu4\nMYwcCa+/bueOdu0S76uoCH76U5gyBT7+2C5Ay8vtbnfNmtnfcskS2LvX+9+jQiATgRt16tgIzyee\nSLyuuo9Krigqsiqj996DX/7SBqP9+Mfwl79k9kQSNPv2hUsE8dSrl/x5oaCgcjXR4sWWXG66yeZD\nu/RSu79FvAbqVAUuESRzdXTllfDII7qikuqnsBAGDoR166zr4sMPw1FHwaRJmunUC24SgRcqqone\neMPudnfSSXD//dYW4aXAJYJknHaaXUWpR4ZUVwUFcPbZNqvunDlWZdGqlV1hfvKJ39HljqlT4fbb\nbTZYNxd+2UoEkSqqiV580ft5qAKZCNwW10IhKxU89JB3+xTxy0kn2Z3RVq2yqoX27a1x+e23/Y7M\nf/PmWeNur15Wr3/77fDOO/GTgh+JINLhh3u7v0AmgmQMGWL9g7/4Iv46qjqS6uTII61f+6ZN0Lq1\nnfy6dbMSQ1mZ39H5o6wMRo+2Pv2PPWaJ8uc/t6qZ2247MCn4nQi8FrhEkOxJ+4gjoH9/6+Ylkk/q\n1bNJ7T78EK65xvrBt2oFEyZUfeGTj8rKrKG9oAA6d4bf/c6SwowZNkI4MimsW6dEEEgjR1odYjIj\njUWqixo1YMAAm67ib3+zxHD00dZDJVM3yfn97+3zMlGaXr8eFi1KrpdURSKIFArZLMT33ls5KfTr\nZzcRqlHD07B9FchEkGx9/rHH2kjjp5/2bp8iuahDB+spt3kz/OhHcN551rbw0EPe9Tbavx+uv97u\ntdC6tdXHb9rkzb4BHnzQutA2amRjKxYuTJwUIgeIxRKZFLZssXaWoUO9i9lvgUwEqRg50uYJiXUF\nozYCyTf16sGvfmUnvfHj7Qq7eXO7cU66I14rrr7feQeeecYSzGmnWZXMAw/AV1+lF3tZGYwdC2vW\nWDIbN8764F9+efykEKtEEE8oBMcfb9XG+SJwiSDVL/DZZ9sX9qWXvI1HJJcVFtqkds88Y3Prt25t\no+47dLD+7Lt2Jb/PipNuKGTTMUyaZBOy3XEHvPqqfUbfvvaZkZO4Jbv/5s2tAfj11+Gtt6xkH5kU\nFiwIV/cmkwjyUeASQaoKC21gx/jxfkci4o+GDW38waZNVsf/8svQsqVVw7z4ojWguhFreoaiIujZ\nE5580qZZ6NPHSgeNGtn+n3/efRtdrJN68+YwapQlhbVrLSmMH2+/08UXw2efKREETqr1+RdeaF/S\n6DngNcWEBElBgc2NM3u2JYUTT7RqpJYt4eabE9+VK9HVd+3a1m17yRLb18knw29+YyftIUMS1/nv\n3Vv1/ps1s6SwfLlVT3XpAqeeCk2aVB13PgtkIkhVUZF90SdM8DsSkdxwxBHWfrZmjU1wt2+fJYlO\nnWwStViTNiZTDdOwoY2mfeUVO2kff7xV7zRubI21S5YcOPahrMx9j54mTWD4cBtDUbeuu23yUeAS\nQboNu4MHW/e0Vau8iUckXxx7rM2x//HHVu2yfLnV9/frV7m+P9X6+CZNbCK911+3bq1t2tikeo0b\nWyN2RZ1/0Ov7UxG4RJCugw6ytoJbb628XFVDIqaoCM46y+r7t22zwVgV9f2XXWbTu6d7om7RAm68\n0RLCypU22GvCBCtBLF2qRJCsQCaCdE/aV1wBH30UvoOZuo+KxFa7tv2/LFliU2NX9Db6wQ+8+4xW\nrax30Kuv2mfcc4/dHlLcC2QiSFeNGnDXXVYs3b/f72hEqofGja1qZ9Uq67mTCY0aWdfQfOrjnw2B\nSwReXb33728JYfZsb/YnEiQFgTvz5LZA/jm8qM8PhawIesstNv+I2ghEpLoKZCLwSrdu1of62Wf9\njkREJHVKBGm69171UBCR6i1wpzCve/i0aAHTp8Nxx3m7XxGRbAlcIgDv6/MvuMDb/YmIZJOqhkRE\nAs5tIugBbAQ2AWPirDPZeX8d0NFZ1gxYCrwHvAtcl3KkIiKSEW4SQSEwBUsG7YALgLZR6/QCjgLa\nAEOBqc7yvcAo4EdAZ+CaGNtmlUYBi4hU5iYRdAI2A1uxE/ssoE/UOr2BGc7zlUAdoAHwGVAxhvBb\nYAPQOK2IPaA+/yIiYW4SQRNgW8Tr7c6yROs0jVqnJVZltDK5EEVEJJPc9BpyW5kSfZ0dud1hwBxg\nJFYyqGTs2LH/fV5SUkJJSYnLjxQRCYbS0lJKS0szsm83iWAH1uhboRl2xV/VOk2dZQA1gGeAPwNz\nY31AZCLINLURiEh1FH2RPG7cOM/27aZqaDXWCNwSKAYGAvOi1pkHDHaedwZ2ATuxUsJ0YD1wX/rh\nekNtBCIiYW5KBGXACGAR1oNoOtboO8x5fxowH+s5tBnYDQxx3jsVuBh4G3jLWXYzsNCD2EVExANu\nRxYvcB6RpkW9HhFju1fRoDURkZwWuJO02ghERCoLXCIAtRGIiEQKZCIQEZGwwCUCVQ2JiFQWuEQg\nIiKVBTIRqI1ARCQskIlARETCApcI1EYgIlJZ4BKBiIhUFshEoDYCEZGwQCYCEREJC1wiUBuBiEhl\ngUsEIiJSWSATgdoIRETCApkIREQkLHCJQG0EIiKVBS4RiIhIZYFMBGojEBEJC2QiEBGRsMAlArUR\niIhUFrhEICIilQUyEaiNQEQkLJCJQEREwgKXCNRGICJSWeASgYiIVBbIRKA2AhGRsEAmAhERCQtc\nIlAbgYhIZW4TQQ9gI7AJGBNnncnO++uAjhHLHwV2Au+kGKOIiGSQm0RQCEzBkkE74AKgbdQ6vYCj\ngDbAUGBqxHuPOdvmDLURiIiEuUkEnYDNwFZgLzAL6BO1Tm9ghvN8JVAHaOi8fgX4Ot1ARUQkM9wk\ngibAtojX251lya6TE9RGICJSmZtE4PbUGV3holOuiEg1UORinR1As4jXzbAr/qrWaeosc2Xs2LH/\nfV5SUkJJSYnbTVOiNgIRqW5KS0spLS3NyL7dnBKLgH8APwU+AVZhDcYbItbpBYxwfnYG7nN+VmgJ\n/A04Nsb+y8uzWF8zfjzs2WM/RUSqq5Bd0XpyWeumaqgMO8kvAtYDs7EkMMx5AMwHPsAalacBwyO2\nnwksB47G2hGGeBF4qtRGICJSmZuqIYAFziPStKjXI+Jse0FSEYmISFYFbmQxqI1ARCRSIBOBiIiE\nBS4RqI12h5s7AAAFOUlEQVRARKSywCUCUNWQiEikQCYCEREJUyIQEQm4wCUCtRGIiFQWuEQAaiMQ\nEYkUyEQgIiJhSgQiIgEXuESgNgIRkcoClwhAbQQiIpECmQhERCRMiUBEJOAClwjURiAiUlngEgGo\njUBEJFIgE4GIiIQpEYiIBFzgEoHaCEREKgtcIgC1EYiIRApkIhARkbDAJQJVDYmIVBa4RCAiIpUF\nMhGojUBEJKxaJ4L9++Hjj/2OQkSkeqvWiWDFCmjRAkpKYNYs2L07/ro7d8Kzz8KXX2YtPBGRaqFa\nJ4LvvoOuXWH4cHj0UWjUCPr1g4cfhvffr9wwPGcOjBgBTz0Fder4F7OISK7Jhdry8vIUu/IsXAiT\nJsGiRfb6n/+E55+HxYth2TLYuxdOOQU6doT166FVK5gwQW0EIlL9hexE5snZzE2JoAewEdgEjImz\nzmTn/XVAxyS3TVlZGdSoEX5dty4MHgxPPAEffQTLl8OAAVZl9PXXcPLJSgIiItESJYJCYAp2Qm8H\nXAC0jVqnF3AU0AYYCkxNYtu07N0LRUWx3wuFrAQwaBDcfbeVHvr08fLTvVdaWup3CDlDxyJMxyJM\nxyIzEiWCTsBmYCuwF5gFRJ9OewMznOcrgTpAQ5fbpmXv3solgupOX/IwHYswHYswHYvMSJQImgDb\nIl5vd5a5Waexi23TUlYWv0QgIiLuJDqNum3FTavm/ZxzUttu+3Y47rh0PllERBLpDCyMeH0zBzb6\nPggMini9EWjgcluw6qNyPfTQQw89knpsJkuKgC1AS6AYWEvsxuL5zvPOwIokthURkWqgJ/APLPvc\n7Cwb5jwqTHHeXwf8OMG2IiIiIiIiJqMDznLUVuBt4C1glbOsLrAYeB/4O9YFt8LN2PHZCJyZtSi9\n9yiwE3gnYlkqv/cJzj42AX/IYLyZFOtYjMV61r3lPHpGvJfPx6IZsBR4D3gXuM5ZHsTvRrxjMZY8\n/m4UYlVGLYEaBKcN4UPsSx7pHuBXzvMxwN3O83bYcamBHafNVN/5oU7DRp1HnvyS+b0reqatwsao\ngLVN9chYxJkT61jcAYyOsW6+H4uGQAfn+WFYVXJbgvndiHcsMv7d8POkkvEBZzksurtt5KC8GUBf\n53kfYCZ2fLZix6sT1dMrwNdRy5L5vU8GGgG1CJekHo/YpjqJdSwgdjfsfD8Wn2EnM4BvgQ3YeKMg\nfjfiHQvI8HfDz0TgZrBaPioHXgRWA1c5yxpgVQU4Pxs4zxtjx6VCvh2jZH/v6OU7yK/jcS3W4WI6\n4aqQIB2LllhJaSX6brTEjkVFL8yMfjf8TATlPn62n07F/sA9gWuwaoJIFX2E48nX45bo9853U4FW\nWNXAp8Dv/A0n6w4DngFGAt9EvRe078ZhwBzsWHxLFr4bfiaCHVjjSIVmVM5i+epT5+cXwHNYVc9O\nrH4QrFj3ufM8+hg1dZbli2R+7+3O8qZRy/PleHxO+IT3COEqwCAcixpYEngCmOssC+p3o+JY/Jnw\nscjr70YQB5wdgtXdARwKvIa19N9DuNfUTRzYMFaMXRFsITfuIZGqlhzYWJzs770SqwcNUT0bBCu0\npPKxaBTxfBTwpPM8349FCKvDnhS1PIjfjXjHIu+/G0EbcNYK+8OtxbqHVfzOdbF2g1hd5W7Bjs9G\n4KysReq9mcAnwB6sbWgIqf3eFd3iNmP3waiOoo/F5dgJ4G2sHngu4TpxyO9j0RXYj/1PVHSP7EEw\nvxuxjkVPgvvdEBERERERERERERERERERERERERERERERERERr/0/qnnm4nnI684AAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def stock_price (S_0, r, sigma, T, X):\n", " \"\"\"\n", " Stock prices S_{i\\frac{T}{M}} given a sequence X[1], ..., X[M] of normal increments\n", " \"\"\"\n", " M = X.size\n", " i = np.arange(M) + 1 # i = 1,...,M\n", " return S_0*np.exp( (r-sigma**2/2.0)*i*T/M + sigma*np.sqrt(T/M)*np.cumsum(X) )\n", "\n", "def a(S, K):\n", " return np.max([0, np.mean(S) - K])\n", "\n", "class NormalIncrements (Sampling):\n", " def __init__ (self, M):\n", " self.M = M\n", " def sample(self):\n", " return np.random.normal(size=(self.M,))\n", " \n", "def out_of_money (X):\n", " sigma = 0.25/np.sqrt(365) # 25% yearly volatility\n", " S = stock_price(100, 0, sigma, 6*30, X)\n", " return a(S, 130)\n", "\n", "M = 100\n", "show_expectation(NormalIncrements(M), out_of_money)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, the same with importance sampling shifting up the mean of the sampling distribution:" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEACAYAAACpoOGTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUFOW5x/HvsIysikBEBHQQ3HDFKGI0ccSogEQU43bj\ndY+YiDEuCWK8YaJJRK+ACwmQgASNxoskeiDBLWpHTRTEICiyCwrIogLKOsww7/3jqUpX93R39Qy9\nzvw+58zprv3toqmn3x1ERERERERERERERERERERERERERESKUn9gMbAMGJ5kn0e87fOB3oH1q4AF\nwDxgTmB9BbDGWz/Pu4aIiBSRpsByoAxoDrwHHBW3z0Bglvf+FODtwLaVQPsE5x0J3JbJhIqISGY1\nCdneBwsQq4Aq4GlgcNw+5wNTvfezgXZAp8D2kiTnTrZeREQKQFiA6AKsDiyv8dalu48D/g7MBb4f\nd9zNWJHUZCyoiIhIAQkLEC7N8yTLDZyO1UkMAG4CvumtHw90B04A1gGj07yOiIjkSLOQ7WuBboHl\nblgOIdU+Xb11AJ96r58Bz2JFVm8AGwP7TwJmJrp4jx493IoVK0KSKCIiASuAnpk4UVgOYi5wGFZJ\nXQpcCsyI22cGcKX3vi+wBdgAtALaeutbA+cA73vLnQPHXxhYH2PFihU45/TnHCNHjsx7GgrhT/dB\n90L3IvUf0CPkuZ62sBxENTAMeBFr0TQZWAQM9bZPxFowDcQqs7cD13jbDgT+ErjOk8BL3vL9WPGS\nw1o6+ecTEZECERYgAJ73/oImxi0PS3DcR1gQSOTKJOtFRKRAhBUxSYEoLy/PdxIKgu5DlO5FlO5F\ndhR6XwTnlamJiEgaSkpKIEPPduUgREQkIQUIERFJSAFCREQSUoAQEZGEFCBERCQhBQgREUlIAUJE\nRBJSgBARkYQUIEREJCEFCBERSUgBQkREElKAEBGRhBQgREQkIQWIOP/1X/DMM/lOhYhI/qUzJGx/\n4CFsRrlJ2Gxw8R4BBgA7gKuBed76VcBXwB6gCpuTGqA98H/AId4+l2BTlcbL+XDfJd4defttOOWU\nnF5aRGSv5XK476bAOCxI9AIuB46K22cgNkH2YcANwPjANgeUA72JBgeAO4GXgcOBV7zlpDZtgnXr\nQlKaYW++mdvriYgUmrAA0Qeba3oVlgN4Ghgct8/5wFTv/WygHdApsD1RJAseMxW4IFUi+veHgw4K\nSWmGNG9ur7kOSCIihSYsQHQBVgeW13jr0t3HAX8H5gLfD+zTCdjgvd9AbECp5fPPQ1KZQWVl9qoA\nISKNXbOQ7elWACQr7zod+BT4GlaktBh4I8E1kl6noqKCL7+095FIedbnnj34YFi2TAFCRIpDJBIh\nEolk5dxhFRl9gQqsDgJgBFBDbEX1BCCCFT+BBYEziOYQfCOBrcAYb59yYD3QGXgNODLB9Z1zjqOP\nhg8/hFzUVw8cCEcdBc89B0uXQtOm2b+miEim5LKSei5W+VwGlAKXAjPi9pkBXOm974u1RtoAtALa\neutbA+cAHwSOucp7fxXwXKpE+PUCuVBTA9/8Jnz0EZx9du6uKyJSaMKKmKqBYcCLWIumycAiYKi3\nfSIwC2vJtBzYDlzjbTsQ+EvgOk8CL3nLo4BpwHVEm7kmT2RYKjOopgZatbL3r72Wu+uKiBSadB69\nz3t/QRPjloclOO4j4IQk59wEfDuNawO5DxBN1H1QRKQ4elLnuoipSRP41rds+frrYZ99oLIyd2kQ\nESkERREg8pGD+Mc/bHnyZNi9G1asyF0aREQKgQJEnGRFTEuWJD9m9WoboiPHo4KIiGRVUQSIfBQx\nAXz2WXT94sXJj9m+3V6feip76RIRybWiCBD5ykF07Ahz5sCkSakDRFWVvT77bPbTJyKSKwoQcbZs\niS1iOvlkOPLI1AFi9257/fOfYcGC7KZPRCRXiiJA5LKIadEiWLUqdt2RR1odRLI6hqoqOO44e3/8\n8dHxnEREillRBIhc5SC++MJev/nN2PUdOliQ2hA/eAgwZQqceiq0bQstW9q6jz9WhbWIFL+iCBC5\nykHMnGmvXeLHqwV69oRf/rL2er/5a/PmlsuYPh3atYPXX7f148bBwoXZSa+ISDYVRYDIVQ6iUyf7\nS6SyEn7zm2h9g++ww+x1/Xro1g0uugjOPRfKy6356803w+23ZzXZIiJZoQARsH07nHZa4m0dO9pr\nfGW1P0VpcP0J3gAjBx9sr8uWZS6NIiK5UhQBIldFTHv2JA9Gjz1mD/z582sf06kTjA9MtBo/l/Xn\nn8PmzZlNq4hIthVFgMhVDiLVQH1du8KNN8LQodGOcf4x551n23xnngmjRtn7U06xv+Ac1++/r7Gd\nRKTwKUAEhI3kevzxsHMnDBoUfsxPf2p1EG+/DaefDjfdZPUUYO+nT89s2kVEMq0oAkSuipjCAoTf\n16F79+i6PXsSH1NSYrkOgJNOsmDRuTPMnm25B801ISKFrigCRGlpbq4TFiC6dIFf/9qmPw0eEzYt\nab9+cI03jVLfvrBjhwKEiBS+dAJEf2wO6WXA8CT7POJtnw/0jtvWFJgHzAysqwDWeOvnEZ3zOiE/\nB1FTk0Zq68A5ywH4wgJESQn84AfwwQfRtKQzwVCLFlbJ7c9U98EHNqXpJ5/sXfpFRLIpLEA0BcZh\nD/BewOXAUXH7DAR6YnNX3wCMj9t+C/AhEOxb7IAxWDDpDbyQTmIzXbE7c2b0lz2k97Bv185yDP6Y\nS8mKmBL55BO4447o8iGH1C29IiK5FPZo64PNNb0KqAKeBgbH7XM+MNV7PxtoB/jdzbpiAWQSUBJ3\nXPxyqF276npEalu3xvZfSPdh/9VX0NvLJ9VlitIOHaCiwt5feqm97tyZdnJFRHIq7NHWBVgdWF7j\nrUt3n7HAT4BEhUM3Y0VSk7GgEiobTUM//jj6Pp36BIj2eVi/Pv1jfK1bw7vvwp/+ZJ3y3nijbukV\nEcmVsAak6Q45lyh3MAjYiNUxlMdtHw/c472/FxgNXJfoxBUVFf8Z1ygSKeeyy+JPVX/OwcaNVmnc\nqlX6uYG+fe11xAjo1Sv9HITvxBPt9Zxz4KWX7FVEpD4ikQiRSCQv1+5LbP3ACGpXVE8ALgssLwYO\nBH6N5SxWAuuA7cDjCa5RBryf5PrOOefuvdc5cG7xYpdRjz9u5120yJYffti5m25K79jOne3YNm2c\n69Gjftd/6y3njjmmfseKiCRC+j/sQ4X99p2LVT6XAaXApcCMuH1mAFd67/sCW4D1wF1AN6A7FkBe\nDezXOXD8hSQPEEB06Oxs9T6++mp4+GG45Zb0W0r5lc3bttW/buSkk2DtWvj00/odLyKSTWEBohoY\nBryItUT6P2ARMNT7A5gFfIRVZk8EfpjkXMGodj+wAKuDOAO4NZ3EZrqS2g88s2fDj39s79N9WN92\nG1x5pRUvPfBA/a7frBmcdRY88QS8956dZ82a2H12747tdyEikivpDGLxvPcXNDFueVjIOf7h/fmu\nTLZjKrkYv2jlyvT3Pf10ePxx2Gef+l9v0CDLwdx5py0PH27zR/TqZcuzZsGFF9atOa2ISCYUxSPH\n/6WfjRxE/DhP8dONpnL66fa6Nz29Bw6sve6xx6LvW7Sw19mz63+NVavgyy/rf3wiiYritm6FiROt\n4n/hQqv8F5HiVRQBwpfpHIRz0TkbfL3j+4GncOSR1rdhb3IQX/uavfboYeM1LVxoAaKqytb7I8fW\nd3C/6mobOyp+CPK98eST0KYNrFsXXbdpkw1MeOONNvz5McdYk97XXrPPBfDOO5YTevhhm9Z140ZN\nzSpSyIoqQGQ6BwGxvZk7dqzbGEklJfCzn1mg2Bs1NbB8uQ3u16uXBYt//tO2ffe79jp9evjD9OKL\na+c0/KC6ZIkN7+Hbmwfzu+9aB7/evS2AnXuuBcpgc91DD7XXfv0sCG/dCn36wF13waRJNvx5p07w\nu9/VLw2VlZZbUUdDkewpigDhP8wuvhimTk29b13P688/fdppMHZsdIa4dN16a+1cSF3FX3PQIBsG\nZMsWW779dvs1/nx8TVDAxo0WRB58MHZ9dTXsu689mP05KsDqM+69t37praqChx6ye3fFFdaXw3fG\nGRY0Fi+2iZJOPtnW+9d+4AEbi2riRPucN94I/fvXPWD9/e92bL9+sGGDAoVIY+Scc66iwvocgHPX\nXZe59sKTJzt39dV23ocfztx599aCBdHPC84tX+7chAnOXXxx8mP8fffbz7nPP4+u//xz59q3d27D\nBufatXNu2jTnxoyxfVu0cO6LL+qevqFDnfvtb5177TU7z8CBtn71aufWro3dd88e5z75xNIAzr36\nqr2+/rptf/BBW77oIts3qKYm9ectLbVr+599/vy6fxaRhoYc9oMoCMFfl37ZfKaUlFj5/EknZfa8\ne+PYY2OXv/Y1yz0984w1eZ0zxyqdFyywYp6lS22/P/wBzj8fHn0Upk2zHEd1tVXEH3AAXHYZXHKJ\nNdEF+wX//e/XPX27d9sIu+XlVgR2xBG2vmtXOOig2H2bNIFu3SwXBJbD2LHD6iDA1i9fDs8+C8OG\nwYoV0WOPP97WJbLPPvCvf8Hf/gYvv2zNlC+6KJrrEpGGzznn3MiR0V+Jl1+euUg7aZJz11yTufNl\n0mWX2ec98cTouh/+0Lnjjovei3HjYnMa1dXOffyxcx06RNf985/OHXSQHb9kSXT9Bx849+WXzh1w\ngHNvv123tF1xhXNTp9btmD17nPvDH5Jv37gxmra33nLuzTejy8OHO7d7d3TfOXNs/bZtsecYNsxy\nFHPm2L0QaYzIYA6i0DnnYgPEd7+buRv5+987d+21mTtfJlVVObd5c+xDcPny2IDQqZNzp54aXfY9\n9FB03bHHOnfwwdFtNTWxRTdTpth+EyemTs9XXzm3Y4cde/bZzj31VEY+ZozNm63Yr2PHaPrff99e\nmzWzz/+Xvzh34IG2Lr5IqrLSue7dbdsdd6QuohJpqFARU8PXrJnNPdG6dXRdjx7R9x9/bJWzn39u\nrYpuuSW6bdgwOPtsq8h9//3Y3tklJbGV4lddBX/8Iwwdaus3bkycnn33tQENmzSxIp2WLTPzOYPa\ntYNrr7U0AVxwgTWXXbPGKsT79oUhQ2wU3QkTanccLC2FRYtg/nx4+uloL/fqavtsV1yRm86WIpIb\nzjnnfv5z+yUMzvXunblI+7vfZbbSOxc+/dS5pUvtfb9+zvXpk3r/AQNicxfJ+Lm0c85x7t//rr29\nS5for/ohQ+zXerbs3Jm42Ovmm+36jz3m3KpVqc/hV6B36ODcffdF0z5ggHNbt2Yl2SJZt2CBfZ83\nbUq+D42tiOnnP3fupJNqF6XsrYkTnbv++sydL9f27KldzBJv/nznzjwzvfN9/HH0HtfUOHfbbc5F\nIrbt+OOde/dda6kUrA/Ite3b09+3stK5d96xz9O5sy23aGHLO3ZkL40i2fLAA/b9bd3aue99z4pc\nV6xwbvp0a6noXCMtYvKHnJCoJk3Cx2c67jh49dX0znfwwbZvhw7WqmvMGGuptH27tVzaZx9rqeTP\nEZ4P/rze6Sgttc/x4INWvFRaah32LrrIWlYFe4KLFIPt22HkSGvN2KOH9QXq0cNaE3bpUnvooL1V\nFAECkg9n8cQTVhYf9MUX6TV3dK7uHeMaujPPtCay//63LZ9yChx+uDVN3ZshRfLp9tujI+42awZT\npkDnztYkd9my/KZNJF0TJsAvfmH1kgcfbO/XrYN582wIm/XrbRKzTCqKAJEqBzF5slXSBp14og3r\nkM55FSBq69fPKrnHj7ehR1q3tkrxuvx6L2Rt21rl/e9/b8FvyBD45BP7HmlsKCkECxbYQJ7Tptk4\nZytW2AgEZ50F3/tedL8mTeCEE+y1Q4f6j46QTIYzJNmT7Nerc7VHKv3ii+ggd1I/weEzpk2DX/2q\ndie4Ynf99fbdueMO66gHNsfHlCkaWl3ya8EC6+j6xRdw6aW2rqTESkxy+f+waP4bpAoQX30Vu84f\nITWMchDpOeEE68XdEN1+u31/xo2zcb6WLYOKCts2d27dhn8XyZTt2+0HzOzZlot/5x0bNeCSS3Kb\njnQCRH9snull1J6P2veIt30+ED9gdlNgHjAzsK498DKwFHgJaJcqAWGV1PE5CAUIqYu2beGmmyz3\n8Oyz1ofi6qvh1FNtqPQ//znfKZTGZOdOq3z2n03l5dbYYsCA3DcQCQsQTYFxWJDoBVwOHBW3z0Cg\nJzZ39Q3A+Ljtt2DTlQZLd+/EAsThwCveckrZyEGIxOvUySroW7SwDnbPPw8/+IEFjF/9yioDRbLJ\n79h69935TQeEB4g+2FzTq4Aq4GlgcNw+5wP+INyzsdxAJ2+5KxZAJgElSY6ZClwQltBULWiSBYi3\n3059TuUgJJE2bazFyKZNNhT59Ok2tezdd1vjh02b8p1CKTS7dlmO8+yzbTDNREaNsh8a/gRayVRW\n2rwwezuNQCaEBYguQPDjrPHWpbvPWOAnQPwElZ0Av3HqBqIBJaFURUyJKqn3289ew6bZVICQVPbf\n316/9S2b1GnTJqsgPPBAG0lWipNz8Ne/wrZttvzKK+nXNS1caH2CZs2yZ8d551kRZMuWlvP8xjds\n5OXycqszqKmBzZtt3xEj7Ifu8cdbS8FzzrGhZUaPhs8+i16jsrJwmpSHtWJKt9Ff/GO2BBgEbMTq\nH8pDrpH0OhUVFbz+ut+qpBwo/88Q1pC4iMn32WdW0bj//rHjGInUx/77w4wZcM89Vh48Y4YNXy7F\n5d134TvfsU6f7dpZ89HWrW1GyUsugeuuiw5hD1Zh3Lo1/OUv1snS16KFNeD47nftde5caNrUcprP\nPGOvP/2pdWrbf39rANGhg+UkfvlLiESsD8Pnn1vz1FNOse9Vt251CxCRSIRIJJKp21MnfYEXAssj\nqF1RPQG4LLC8GDgQ+DWWs1gJrAO2A4/H7QPQ2VtOxDnn3F13OXfvvdFhIIJj6XzjG7WHkhg2zEb/\n/N//tf0POSRxt/VHHnHuppsy3x1eGr6//tW+W4cdZqPQSvE44ggbx+wf/3DujDOcu+ce5z77zLlf\n/tK5W26xIfC/8x0bpsaf4Mr/Ky93bvx45+bOtTHDnLNhaaqqal+npsaGqhk50rn161On6YsvnHv0\nUedOP92uc8QR9f98ZHCojbAcxFys8rkM+BS4FKuoDpoBDMPqJ/oCW4D1wF3eH8AZwB3AlYFjrgLu\n916fS5WI+KKgXbusnNjfFp+DcM6KAtavt+Vg9i3VeUXSdd559svzmGOiudNRo6w54jXXWHm0FB7n\nbH72X/zCig6DP7x/9jN7HTXKin2+/W17dlx/vY0sfPDBtSfzAnuGJBrioqTEcpjp5DLbt7dRmIcN\ns74PmzfX6+NlXFgdRDX28H8Ra4n0f8AiYKj3BzAL+AirzJ4I/DDJuYJRbRRwNtbMtZ+3nFLwQR4s\nLwzWQVRXW9fzmhobSmH0aFu/Y0fY2UXq7uij7ft3221WP3HDDdY7+6yzbPwn9couPJWVNiaX3/ks\nkRYtLFgsWWJFPzfeaD8IEgWHbOjQAXr2zM21wqTTD+J54AisKet93rqJ3p9vmLf9eODfCc7xD6zl\nkm8T8G2smes5WK4jKf8/2q232jAa8dHVz0FMmWI5Bz8HETQ+vvEtykFIZvzsZ/ZdWr/eBjtctMj6\nUnTvbrkMSWzaNAuqL79sQ5+ka+3aul2npgYWL7Zr7NyZ/lwm++9v9Qhf/3rdrteQFE1P6pISG120\na9fYHEEwB1Faaq81NbUDxA+T5GsUICRTOnWywQ4POQTefBMuvxxOPtlaqnz6qbWuu+22aOuZxqim\nxnqsP/ec/Yp/6ilrzXPccVY0t3Jl7P67dsFHH0WXt2yxZ8BJJ8HgwRZkFi60/eL565o2haOOsmsc\ndFB460aJKpoA4WvVKnacJecs21hZaeV4YOWG6YxXoiIAyZYWLeC++6zFzJtv2lDMX31lRaCDBsV+\nh3fsKJwy50zbsME+90MPWUC4/37rC3DhhbZ92zb7Vb9li7UUOvlky1UMGWI/3lq2tDqe3r1tVsOx\nY+HIIy3XtnmznfOYY2y/Sy6xOiBfy5YWsMEGY6ypgRdegKVLc34bilZRBIjgg7xVq8R1Cl99Fe3l\nunq1NVlLdR5/WTkIyaZevewX7gMPwKOPwpNPWtFT9+7RCtKBA+3HzY9+ZE0enbP6tHjV1dYEc/To\n4gkoQ4ZYzunWW+3X/l13WV8Bv10QWDDdbz8bD2vJEmt6+uyzFkjvv9+CzC23wGmnWfPQ0lILMK+/\nboFlwwbryNi1q3VsvOwyePFFO3evXhYkunSJVhofdlj+7odklnPOueHDbZo952zayYcfjjbpOvlk\n+6otX+7cM8/Y+y5dnJs0ybmzz7ZmrP7X8bPPYpuDjRnj3I9/XP/mZCL1UVNj371993Wuf3/7bl5x\nhc0S1rRp9PvaurW9/utfdtyFF0a3dezo3MqVzr3xhnMlJTaV6ujRzm3cmNePVsugQfZ/9q23nKuu\n3vvzVVZak9Bktm937n/+x7lDD7X7lKj5aUNHY5tRLig+B+H/Cvnyy+ivrvXr7dfCSy/ZrxLfJ5/E\nnktFTJIPJSX2i/rNN63IA+C3v7Xilt/9zuowbrrJfmn/+Mf2OmSI/ap+5hn73o4cGf1F3by5fbfH\njrVfzOeeazmWXbvs/8XChXVP46OPRtO2NyorrQVQ375WF7C3SkujRcmJtGplHRlXrLBrZ3qGtcam\nKAJEWBFTy5axRUx79sSO53/88fYa3/pBRUyST8ceGy1qadvW1l17rTXjHjfOfuCMHWsP+FatoKzM\netqCtZefNMmKUqZPt9ZSq1db7+5997WirJYtrbjmmGPse961qw0x4f8/Wbw4WhSzc2fs/7MJE6yS\nvV8/S8dHH0X///ztb/DHP1qZfiIzZ9oDGvI7bITfaEXqr2jiq/8gb9Uqdi5h5+w/RDAHEdwfbEz1\na6+1JmvnnRcbPBQgpND16mUP5HgDBthDOvgdPvXUaC5j8WIbJ+jYY638v3VrK7uvrrYWQB99VLt5\n6Z13Wv3Ghx/aqKJTplilsv9/rkmTaGC47z4LbP/93xawfvMb+/91vteg/aKLYPnywhlXSOquQeQg\nunSxZoTBnEMwCOyzj2XHFyyIHeFVRUxS7JL9wCkpsaadd99tweDJJ634atcua+nTs6cFh7fesn0G\nDLBf/uvXWw5hzBj7f3X33ZYbGDnSxhp6/XUrHlu3zprsbt4cneN75kzL5XzjG9aS8NBD7f9el/jh\nPaVoFGUOIr4OonNna8lw0EH2t2ZN7f84Z51lrxs3xh6rHIQ0Jk2bWh+Ck06ypqL772/1A75Bg2of\n07JldJY9sB9bYC2qrrsudt933rFhcDp2tHqQBx7I+EeQHCqKHERQ69a1A0SHDjbUQXW1lbNCdBwm\nX/fu9jpvXmxltQKENFb+cOaZdPLJlnORhqEoAkRYEVOHDjbA1Z490TbOH34Yu09pqbUfv+ceayUS\nf14REYlVFAECYouY4ntSt28fzUG0b2+VZsFx232HHhp7nIqYRESSK5o6CF+iOgi/iGnPHmv3/Pjj\niY8NBohFi+xVAUJEJLGiyEGkU8Q0Z461tEjVGSdYNrpihYqYRERSKYoAAbFFTEuXRlte+DkIsMCR\nqudk8+ZWV3H11db5aMkS5SBERJJJJ0D0x6YEXUbt6UZ9j3jb5wO9vXUtgNnAe9hkQ/cF9q8A1mDz\nVc/zrpFUfA6iqso6v/nbgq0x1qxJ/WHat7ee1SNGwB/+oAAhIpJMWIBoCozDHuC9sOlG4xuxDcQm\nCzoMuAHwp+bZBZwJnAAc5733WlDjgDFYMOlN7LzXCfkP8tata29r1iw63sxzKScvNf4UkZB41EwR\nEQkPEH2wqURXAVXYvNOD4/Y5H5jqvZ8NtAM6ect+bUEpFmyCgxSn/ds9mIMIzgYVHDL4gAPsNX5+\n6kSCAUJjw4uIJBYWILoAqwPLa7x1Yft43dVoihUxbQBew4qafDdjRVKTsaCS1NixsHu3vW/ePLp+\n9+5oU1W/mGnixNrHx/M7zQG89174/iIijVFYgEi3nU98bsA/bg9WxNQV+BZQ7q0fD3T3tq0DRodd\nINGUglu3ehcvsdZLBx0E3/teeGKDuZB09hcRaYzC+kGsBboFlrthOYRU+3T11gV9CfwNOAmIAIER\nkZgEzEyWgApvEJhXX4UzzyynvLz8P9u2bo0tfqrLZOZt29rsUr/+dfrHiIgUmkgkQsSfnjDDwuoB\nmgFLgLOAT4E5WEX1osA+A4Fh3mtf4CHvtSNQDWwBWgIvAr8AXgE6YzkHgFuBk4H/SnB955yjpAR+\n8pPowF9+hfX8+TZm/bRpcPTR6X5ks3mzFVe1aVO340REClmJPSAz0j4zLAdRjT38X8TqEyZjwWGo\nt30iMAsLDsuB7cA13rbOWOV1E+/vCSw4ANyPFS85YGXgfElVVdVet21b/YfLyMZAZSIiDUk6Q208\n7/0FxVcFD0tw3PvAiUnOeWUa140RbI7apo0Fh/giJhERyZyi6UkdzEH4UwkGK6lFRCSzijJA+E1d\nlYMQEcmeogwQfg5ib+ogREQktaIJEME6CH9UVhUxiYhkT9EEiGAO4tlnrdmriphERLKnaAJEMAfR\nqhV06xYNEMpBiIhkXtEEiPh+EG3bWh0EKECIiGRDwQcIvwjJH6zP17atiphERLKp4AOELz4H0aaN\niphERLKp4AOEn0OIDxD77hud+0EBQkQk89IZaqMgxM/8dsABNu1o27b5SY+ISENX8DkIX3wOopM3\nZ93WrcpBiIhkQ8EHiGSV1MFhuhUgREQyr+ADhO/yy/OdAhGRxqXQf3u7qirHPvvAnj21N7ZoAZWV\nsHIllJXlPG0iIgUnkxMGFUUOIlkR0pAhuU2HiEhjkk6A6A8sBpYBw5Ps84i3fT7Q21vXApgNvAd8\nCNwX2L898DKwFHgJaFfXhAMMHmyvNTX1OVpERFIJCxBNgXFYkOiFzUd9VNw+A4GewGHADcB4b/0u\n4ExsatHPLSvLAAAMzklEQVTjvPenedvuxALE4dg0pHcmS0CqntKXXmqvX34Z8ilERKTOwgJEH2yu\n6VVAFfA0MDhun/OxuafBcgztAK8RKju811Is2GxOcMxU4IJUiUjVSunii+HQQ1N+BhERqYewANEF\nWB1YXuOtC9unq/e+KVbEtAF4DStqAgsgG7z3G4gGlDqbNg3226++R4uISDJhASLdofDif+P7x+3B\nipi6At8CypNcI+l1NBifiEh+hA21sRboFljuhuUQUu3T1VsX9CXwN+DrQATLNRwIrAc6AxuTJeDe\neyuoqYGKCigvL6e8vDwkySIijUckEiESiWTl3GFtZZsBS4CzgE+BOVhF9aLAPgOBYd5rX+Ah77Uj\nUA1sAVoCLwK/wCqlHwC+AO7HKqjbkbii2u3a5WjbtnZPahERqS2T/SDCchDV2MP/Raw+YTIWHIZ6\n2ycCs7DgsBzYDlzjbeuMVUA38f6ewIIDwChgGnAdVgF+SapEaCgNEZHcK/RHr9u1y7HvvtZjWkRE\nUmtUPalVSS0ikh8FHyBARUwiIvlQ8AFCOQgRkfwo+AABykGIiORDUQQIERHJvYIPECpiEhHJj4IP\nEKAiJhGRfCj4AKEchIhIfhR8gADlIERE8qEoAoSIiORewQcIFTGJiORHwQcIUBGTiEg+FHyAUA5C\nRCQ/Cj5AgHIQIiL5UBQBQkREcq/gA4SKmERE8iOdANEfWAwsA4Yn2ecRb/t8oLe3rhvwGrAQ+AD4\nUWD/Cmxu63neX/9UCVARk4hI7oVNOdoUGAd8G1gLvAPMoPac1D2Bw4BTgPHYnNRVwK3Ae0Ab4F3g\nJSzYOGCM9yciIgUoLAfRB5trehX2wH8aGBy3z/nY3NMAs4F2QCdgPRYcALZhQaVL4Li08gUqYhIR\nyY+wANEFWB1YXkPsQz7ZPl3j9inDip5mB9bdjBVJTcaCSlIqYhIRyb2wIqZ0f7/HP8KDx7UBpgO3\nYDkJsGKoe7z39wKjgesSnfi++yrYtQsqKqC8vJzy8vI0kyQi0vBFIhEikUhWzh3227wvVqHsVyKP\nAGqA+wP7TAAiWPETWB3DGcAGoDnwV+B54KEk1ygDZgLHJtjmNm92lJXBli0hKRUREUqsyCUj5S5h\nRUxzscrnMqAUuBSrpA6aAVzpve8LbMGCQwlWfPQhtYND58D7C4H365huERHJsrAipmpgGPAi1qJp\nMlbZPNTbPhGYhbVkWg5sB67xtp0GXAEswJqyguVAXsByICdgRVErA+erRZXUIiL5UejVv27TJseh\nh8LmzflOiohI4ctlEVPeKQchIpIfBR8gQM1cRUTyoSgChIiI5F7BBwgVMYmI5EfBBwhQEZOISD4U\nfIBQDkJEJD8KPkCAchAiIvlQFAFCRERyr+ADhIqYRETyo+ADBKiISUQkH4oiQIiISO4VfIBQEZOI\nSH4UfIAAFTGJiORDwQcI5SBERPKj4AMEKAchIpIPRREgREQk99IJEP2xeaaXAcOT7POIt30+0Ntb\n1w14DVgIfAD8KLB/e+BlYCnwEtAu2cVVxCQikh9hAaIpMA4LEr2Ay4Gj4vYZCPTE5q6+ARjvra8C\nbgWOxuaqvgk40tt2JxYgDgde8ZaTUhGTiEjuhQWIPthc06uwB/7TwOC4fc4HpnrvZ2O5gU7AeuA9\nb/02bC7rLgmOmQpckCwBykGIiORHWIDoAqwOLK8h+pBPtU/XuH3KsKKn2d5yJ2CD936Dt5yUchAi\nIrnXLGR7ur/f4x/hwePaANOBW7CcRKJrJL3Ogw9WsHUrVFRAeXk55eXlaSZJRKThi0QiRCKRrJw7\n7Ld5X6ACq4MAGAHUAPcH9pkARLDiJ7AK7TOwnEFz4K/A88BDgWMWA+VYMVRnrDL7SGpza9c6vv51\nWLcunY8jItK4lViRS0bKXcKKmOZilc9lQClwKTAjbp8ZwJXe+77AFiw4lACTgQ+JDQ7+MVd5768C\nnkuVCBUxiYjkXlgRUzUwDHgRa9E0GatsHuptnwjMwloyLQe2A9d4204DrgAWAPO8dSOAF4BRwDTg\nOqwC/JJkCVAltYhIfhT6b3O3Zo2jTx9YuzbfSRERKXy5LGISEZFGquADhIqYRETyo+ADBKiSWkQk\nHwo+QLz9tuofRETyoeADxFNP5TsFIiKNU8EHiGZhDXFFRCQrCr103/mjcKiyWkQkXKNr5nrddflO\ngYhI41MUAUKtmEREcq8oAkSTokiliEjDUhSPXuUgRERyTwFCREQSUoAQEZGEFCBERCQhBQgREUlI\nAUJERBJKJ0D0x+aQXgYMT7LPI972+UDvwPrHsOlH34/bvwJYg800N4/onNeJE1kUYUxEpGEJe/Q2\nBcZhD/BewOXAUXH7DAR6YnNX3wCMD2ybQuKHvwPGYMGkNzYNaVLKQYiI5F5YgOiDzTW9CqgCngYG\nx+1zPjDVez8baAcc6C2/AWxOcu60H/sKECIiuRcWILoAqwPLa7x1dd0nkZuxIqnJWFBJSgFCRCT3\nwgbTTncM1fhHeNhx44F7vPf3AqOBJEPyVfDWW1BRAeXl5ZSXl6eZJBGRhi8SiRCJRLJy7rDf5n2x\nCmW/HmEEUAPcH9hnAhDBip/AKrTPwCqnAcqAmcCxSa6RarsDx+23w4MPhqRURERyOtz3XKzyuQwo\nBS4FZsTtMwO40nvfF9hCNDgk0znw/kJqt3KKoSImEZHcCytiqgaGAS9iLZomA4uAod72icAsrCXT\ncmA7cE3g+D9huYkOWD3Fz7GWTfcDJ2BFUSsD50tIAUJEJPcK/dHrwDF8OIwale+kiIgUvkY3o5xy\nECIiuVcUAUJERHKvKALE9u35ToGISONT6IU3DhwtW8KOHflOiohI4Wt0dRAu3e56IiKSMUURIA47\nLN8pEBFpfIoiQOzene8UiIg0PkURICor850CEZHGpygCxK5d+U6BiEjjUxQBYufOfKdARKTxKYoA\nccUV+U6BiEjjUxT9INTMVUQkPY2qH8STT+Y7BSIijVPBB4iOHfOdAhGRxqngA8QBB+Q7BSIijVM6\nAaI/No3oMmB4kn0e8bbPB3oH1j+GzS4XP2Nce+BlYCnwEtAu2cVPOCGNFIqISMaFBYimwDgsSPQC\nLgeOittnINATm5r0BmB8YNsUovNZB92JBYjDgVe8ZUkhW5OSFxvdhyjdiyjdi+wICxB9sKlEVwFV\nwNPA4Lh9zgemeu9nY7mBA73lN4DNCc4bPGYqcEFdEt0Y6T+A0X2I0r2I0r3IjrAA0QWbS9q3xltX\n133idcKKnvBeO4XsLyIiORYWINLtgRDf5rYuPRdcHfcXEZEC0Bd4IbA8gtoV1ROAywLLi4nNEZRR\nu5J6MdFiqM7eciLLiQYQ/elPf/rTX/jfcnKkGbACe8iXAu+RuJJ6lve+L/B23PYyageIB4gGmjuB\nURlJrYiI5NQAYAkWlUZ464Z6f75x3vb5wImB9X8CPgUqsXqKa7z17YG/k0YzVxERERERkVDpdM5r\naFYBC4B5wBxvXaoOhSOw+7MYOCdnqcyORB0q6/PZv+6dYxnwcBbTm02J7kUF1jpwnvc3ILCtod6L\nbsBrwELgA+BH3vrG+L1Idi8qaHzfC5pixVVlQHMS13s0RCuxL3/QA8BPvffDidbV9MLuS3PsPi2n\nCIZNSeGbWA/84EOxLp/db0U3B+u7A1YvlqiTZqFLdC9GArcl2Lch34sDAX8chTZYMfdRNM7vRbJ7\nkfXvRSE+VNLpnNdQxTcXTtahcDBWv1OF3aflRP/Ri1GiDpV1+eynYK3h2hLNfT1OcXbATNa5NNHw\nzQ35XqzHHnIA24BFWP+qxvi9SHYvIMvfi0IMEPXpeNcQOKzifi7wfW9dsg6FB2H3xdcQ71FdP3v8\n+rU0rHtyM9YIZDLRYpXGci/KsFzVbPS9KMPuhd9aNKvfi0IMEC7fCciT07B/+AHATVhRQ5DfxjmZ\nhnzfwj57Qzce6I4VM6wDRuc3OTnVBvgzcAuwNW5bY/tetAGmY/diGzn4XhRigFiLVcr4uhEb9Rqq\ndd7rZ8CzWJHRBmI7FG703sffo67euoakLp99jbe+a9z6hnJPNhJ9GE4iWpzY0O9Fcyw4PAE8561r\nrN8L/178kei9aJTfi3Q65zU0rbCyQYDWwD+xlgfJOhT6lVCl2C+IFRT+9LFhyqhdSV3Xzz4bK2st\noTgrI31lxN6LzoH3twJPee8b8r0owcrIx8atb4zfi2T3ojF+L4DEnfMasu7YP+h7WDM2/zOn6lB4\nF3Z/FgPn5iyl2eF3qNxNtENlfT6734RvOTZHSTGKvxfXYg+HBVhZ83PEDmXTUO/F6UAN9n/Cb8bZ\nn8b5vUh0LwbQOL8XIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIi6ft/Vh3M0iyiDdAAAAAASUVO\nRK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_expectation(ImportanceSampling(log_p = multivariate_normal(mean=np.zeros(10)).logpdf, \\\n", " q = multivariate_normal(mean=1.0*np.ones(10))), \\\n", " lambda X: X[1]*out_of_money(X[0]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Markov chain Monte-Carlo (MCMC)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "The previous sampling methods produced independent samples from the target distribution. Markov Chain Monte Carlo (MCMC) methods instead produce a sequence of dependent samples which nevertheless has a desired marginal distribution.\n", "\n", "A sequence of random variable $X_1, X_2, \\ldots$ is called a Markov chain if their joint density can be factorized as follows:\n", "$$ p(x_1, x_2, \\ldots) = p(x_1) \\prod_{i=1}^{\\infty} p(x_{i+1} | x_1, \\ldots, x_{i}) = p(x_1) \\prod_{i=1}^{\\infty} p(x_{i+1} | x_i) $$\n", "Thus, each variable is independent of its history given its direct predecessor. A Markov chain is called **_homogeneous_** if its transition density $p(x_{i+1}|x_i)$ is the same for all $i \\in \\mathbb{N}$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Detailed Balance\n", "\n", "MCMC constructs a Markov chain with a desired stationary distribution $p(x)$. The stationary distribution $p^*(x)$ of a homogeneous Markov chain with transition density $p(x' | x)$ has the property that\n", "$$ p^*(x') = \\int p(x' | x) p^*(x) dx, $$\n", "i.e. the distribution is unchanged by the action of the Markov chain.\n", "\n", "A sufficient condition to ensure that $p(x)$ is invariant is that the transition probability satisfies **_detailed balance_**:\n", "$$ p^*(x) p(x' | x) = p^*(x') p(x | x') $$\n", "Note that this is not tautological as it requires that $p^*$ is the same on both sides.\n", "\n", "Then,\n", "$$ \\int p(x' | x) p^*(x) dx = \\int p^*(x') p(x | x') dx = p^*(x') \\int p(x | x') dx = p^*(x') $$\n", "A Markov chain which satisfies detailed balance is **_reversible_**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ideally, we want that the chain converges to its target distribution $p^*(x)$ independent of where it started. This is ensured by the following technical conditions:\n", "\n", "* **Ergodic**: A Markov chain is called *ergodic* if every state $x'$ can be reached from any other state $x$, i.e. $\\forall x,x' \\; \\exists n \\mbox{ s.t. } p(X_n = x' | X_1 = x) > 0$\n", "\n", " This condition is also called *irreducibility* and implies the existence of a unique invariant measure $p^*(x)$. Further, from the ergodic theorem we get the important property that time averages converge to ensemble averages, i.e.\n", " $$ \\frac{1}{T} \\sum_{i=1}^T f(x_i) \\underset{T \\to \\infty}{\\longrightarrow} \\mathbb[E]_{p^*}[f] $$\n", "\n", "* **Aperiodic**: A Markov chain is called *aperiodic* if the period of all its state is 1. The period of state $x$ is defined as\n", "$$ k = \\mathtt{gcd}\\{n > 1: p(X_n = x | X_1 = x) > 0\\} $$\n", "where $\\mathtt{gcd}$ is the greatest common divisor. Thus, in an aperiodic chain each state can be revisited at irregular intervals.\n", "\n", "When the Markov chain is irreducible and aperiodic it is called *mixing* and it converges to its invariant distribution from any starting distribution. In practice it is usually easy to construct construct a mixing chain, e.g. by chosing a transition density which has full support, but the mixing time can be very long." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Metropolis-Hastings\n", "\n", "The Metropolis-Hastings algorithm draws a new sample from $p(x)$ starting from a sample $x$ as follows:\n", "\n", "* Draw $x'$ from a proposal distribution $q(x'|x)$\n", "* With probability $$a_{x,x'} = \\min\\{1, \\frac{p(x') q(x|x')}{p(x) q(x'|x)}\\}$$ accept $x'$ as the new sample. Otherwise stay at $x$ and return it as the new sample.\n", "\n", "**Exercise:** Show that this Markov chain satisfies detailed balance with respect to $p(x)$\n", "\n", "Note that the acceptance condition simplifies when the proposal distribution is symmetric, i.e. $q(x'|x) = q(x|x')$. In this case, the algorithm is also known as the _Metropolis algorithm_ with the intuitve acceptance probability $\\min\\{1, \\frac{p(x')}{p(x)}\\}$.\n", "\n", "The algorithm also works if the normalization constant of $p(x)$ is unavailable as it cancels in the acceptance condition." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class Proposer (object):\n", " \"\"\"\n", " Wraps two functions needed by a proposer q, i.e.\n", " Draw a new value y ~ q.propose(x)\n", " Compute log transition probability q.log_trans_prob(x,y) \n", " \"\"\"\n", " def __init__(self, propose, log_trans_prob):\n", " self.propose = propose\n", " self.log_trans_prob = log_trans_prob\n", " \n", " def propose(self, x):\n", " return self.propose(x)\n", " \n", " def log_trans_prob(self, x, y):\n", " return self.log_trans_prob(x, y)\n", "\n", "class MetropolisHastings (Sampling):\n", " def __init__(self, log_p, q, x):\n", " \"\"\"\n", " q is assumed to be a proposer and log_p computes log p(x)\n", " \"\"\"\n", " self.x = x # Current sample\n", " self.log_p = log_p\n", " self.q = q\n", " self.samples = 0\n", " self.accepted = 0\n", " \n", " def __str__ (self):\n", " return \"Metropolis Hastings: Accepted %d out of %d samples\" % (self.accepted, self.samples)\n", " \n", " def sample (self):\n", " self.samples += 1\n", " # Propose new candidate\n", " x_prime = self.q.propose(self.x)\n", " a = self.log_p(x_prime) + self.q.log_trans_prob(x_prime, self.x) \\\n", " - self.log_p(self.x) - self.q.log_trans_prob(self.x, x_prime)\n", " u = np.random.uniform()\n", " if np.log(u) < a:\n", " self.accepted += 1\n", " self.x = x_prime\n", " return self.x" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEACAYAAAC57G0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVNX9//HXsAtKcQUUQZorggiiFA2itBEBgR9SRAQh\nFiwhRhQ76jfGTdHEGFOMDTWiIggSiogi1QnYAWHB0FkQaYqIcWFpC/P748zCMOzOndkp587M+/l4\n3MdOuffcj+veD+d+7rnngoiIiIiIiIiIiIiIiIiIiIiIiIhIxuoBrAbWAaPCrPczoBgYUI5tRUQk\nybKA9UAuUBFYBjQrY735wAyOJfhItxURkQSo4PB9W0yS3gQcAiYAfUtZ707g38DOcmwrIiIJ4JTg\n6wHfBL3fEvgsdJ2+wAuB9/4othURkQRxSvB+h+8B/g48FFjXE1gi3VZERBIk2+H7rUCDoPcNMD3x\nYBdhyi8ApwM9MSWZSLblnHPO8W/YsCGKkEVEBNgANI6lgexAI7lAJZwvlI4Bro5yW38qeOyxx2yH\nEBHFWQ6bN/v93br5/c2b+/1vvOH379lz9KvHHnvM7z940O+fOdPv79LFrPPJJ37AD/4ELpEfF676\nXYahOOOLCKokTiWaYmAEMAtYCUwEVgHDA0t5thVxj1mz4KKLwOuF/Hy4/nqoWvX4dSpWhB49YO5c\n+O1voV8/7gBUhRS3cyrRAMwMLMFGl7HusAi2FXGHN96ABx+EqVOhfXvn9T0euOYaaN2a4Y0bcxYP\n8iB/5thlJxF3cerBS4DX67UdQkQUZ4QmToRHHgGfL2xyLzXOc86hM3AF8/gzDyYqwohZ/11GSHEm\nnxu6HoFykkiSzJsHQ4aYkssFF5SrCY/HQw12sZCOvMDtPMeIOAfpQceFhOPxeMAhhyvBS2bZtAna\ntTM9+M6dy92MObj85LKRT7iMG3mdOXSPW5hK8OJECV4k2IEDphwzdCjcc09MTZUkeIBO/IcJDKYN\nX7KDM+MQKCjBixMleJFgDz0Ea9bAlCnmgmkMghM8QB6PcRmfcCWz8Mfl0pYSvISnBC9S4vPPoW9f\nMxSydu2YmwtN8FkUs4BOjGMozwcGUca4ByV4CUsJXgRg/35o0wYeewwGDYpLk6EJHqAZK/kPnWlJ\nPtupG+selOAlrEgSvIZJSvp76ik491y49tqE7mYVzXmRX/IMdyV0PyKRUg9e0ts330CrVrBkCeTm\nxq3Z0nrwACexn+VcyN38nZn0imUP6sFLWOrBizz0EPzqV3FN7uEc4GTu42me4gGyKE7KPkXKogQv\n6evTT2HBApPkk2gGvdlJLYYxJqn7FQmlEo2kJ78fLr8cbrwRhoVOkRS7sko0JS5iMe/Ql6asYS/V\nyrMHlWgkLJVoJHPNnw/bt5vZIS1YwsUsoBMj+YeV/YuAevCSjvx+uOwyuOsuuO66hOzCqQcPcC5r\nWEhHzmEDezgl2j2oBy9hqQcvmWnmTNizJ25j3strLU2ZS1fu4DmrcUjmUg9e0k/79jByZELHvUfS\ngwdz89OHXM45bIiyFq8evISnHrxkno8/hh07YMAA25EA5uYnH15+yYu2Q5EMpB68pJc+faBnT7j9\n9oTuJtIePEArljKdPjSigGIqRroH9eAlLPXgJbOsXAlffAE33WQ7kuMsozXraMJAJtkORTJMJAm+\nB7AaWAeMKuX7vkA+sBRYAnQJ+m4TsDzw3RexBCri6KmnYMQIqFzZdiQn+Cv3ch9Powd1SzI5lWiy\ngDVAV2ArsAi4DlgVtE5VYG/g9QXAVKBx4P1G4CLghzD7UIlGYrdjBzRrBhs2QM2aCd9dNCUaAA9H\nWElzhjOaBUTyJCmVaCS8eJRo2gLrMT3xQ8AETI892N6g19WA70PjcNiHSOxeftmMmklCci8PPxX4\nG/dwL3+1HYpkEKcEXw/4Juj9lsBnofphevUz4bi5Uv3AXGAxcFv5wxQJo7gYRo82k4q52Fiu5zI+\noREbbIciGcIpwUd6jjgNaAZcBYwN+rw90BroCdwBdIw2QBFH06eb2SJbtrQdSVj7qMLr3MgveMl2\nKJIhsh2+3wo0CHrfANOLL8vCQJunAbuA7YHPd2Jq820D6xwnLy/v6Guv14vX63UISyTIc8/BHfF4\nTF7ijWY4H9Oe3/A7DnKS7XAkhfh8Pnw+X1TbONXHszEXWa8AtmFGwoReZD0HKMD09tsAkwKfVcFc\npC3EXIidDfw28DOYLrJK+a1aZWaN3LwZKlVK2m6jvcgabDbdGMMw3mJIuD3oIquEFY+LrMXACGAW\nsBKYiEnuwwMLwABgBWYo5D+AwYHP62B668uAz4EZnJjcRWLz/PNw661JTe6xeoHbdWerJIUbRrio\nBy/lU1QE9etDfj40aOC8fhzF0oPP5hCbyOVKZvFfWpS1B/XgJSzdySrpbfJkaNcu6ck9VsVU5F/c\nooutknDqwUvq6tLFDI285pqk7zqWHjxALhv5grbUZ0sZF1vVg5fw1IOX9FVQACtWwFVX2Y6kXDZx\nNl/Rgt7MsB2KpDEleElNr70GQ4bASak71HAMw/RgbkkolWgk9Rw+DGefDe++a+3mplhLNABV2MsW\n6tOclezgzNA9qEQjYalEI+lp/nyoVcv1d646KaIqU7ian/Om7VAkTSnBS+p59VUYNsx2FHExhmHc\nxGtoGmFJBJVoJLX89JMZFllQAKedVuZqOTk1KSzcneBg4vF362ct5zKUcSyibdDnKtFIeCrRSPqZ\nOhW83rDJHQgkd38Cl3jx8Do3cgNvxLFNEUMJXlLLuHEwdKjtKOJqPEMYyCSyKLYdiqQZJXhJHTt2\nwKJF0Lu37UjiaiONKKARVzDPdiiSZpTgJXVMnAh9+kCVKrYjibvxDGEI422HIWlGF1kldbRtC3/4\nA3Tv7rhqPMapO+whru2fwbes5jzqso39VEYXWcWJLrJK+li3zsz53qWL7UgS4jtqs4ifaeoCiSsl\neEkN48bBoEGQ7fQQstSlMo3Em0o04n5+PzRtCm++aco0EUi1Eg1ADv9jMw3JZRM/UlMlGglLJRpx\nrZycmng8noiWiytUYN26dXguuSTibVLRT5zKHLpxNVNshyJpQglerIjmRqRreYC3eSTi9VP5tn+V\naSSe3NDVUYkmA0VeQvFTQCP6M5V8WkWzhwjbL6/EtH8y+9jOmZzL//hOx4WEEa8STQ9gNbAOGFXK\n932BfMxDt5cAwcMcnLYVCesillBMNvmk9syRkdpPZWbSk/62A5G04JTgs4BnMYm6OXAd0CxknblA\nS6A1cBMcfdBkJNuKhDWQSUxiIO442UyOSQxkoO0gJC04Jfi2wHpgE3AImIDpsQfbG/S6GvB9FNuK\nhOEPSvCZYyY9uRhg507boUiKc0rw9YBvgt5vCXwWqh+wCpgJ3BXltiKlasOXHKECy6Kqvae+/VRm\nFsC0abZDkRTndNdIpFd5pgWWjsBY4LxogsjLyzv62uv14vV6o9lc0lQmlmdKTAIGTZoEt91mOxRx\nCZ/Ph8/ni2obpyOnHZCHqaMDPAwcAZ4Ms80GTHmmSYTbahRNBnIeReNnPY0ZyCSW0qY8e3BoP1aJ\nbb8yHopycmDDBjj99ITtR1JXPEbRLMYk6lygEjAImB6yzjlBOyk5EndFuK1IqVqzFICltLYciR37\nwEyq9s47tkORFOaU4IuBEcAsYCUwEVNrHx5YAAYAKzDDJP8BDHbYVsRRJpdnjho4ECZNsh2FpDA3\nHD0q0WSg8CUaP+towrW8Xc7yDKR6iQY8+AsLoV492LgRatZM4L4kFWkuGklJrViGB3/GlmeOqlYN\nunZVmUbKTQleXGcAk5nMANxxgmmZyjQSAzccQSrRZKBwJZoVtOA2XuYzLo1lD2W2Hx9JKNH4/bBn\njynTfP01VK+ewP1JqlGJRlJOY9ZxGrv4nEtsh+IO1apB587w/vu2I5EUpAQvrtKPabxDX/z60zym\nXz+YOtV2FJKCdBSJq/RnKlM1l+LxrroK5syBfftsRyIpRgleXKMO22nGKj7kctuhuEutWtCqFcyd\nazsSSTFK8OIaV/EuM+nJISrZDsV9+vfX5GMSNSV4cY3+TGUa/WyH4U59+8K770Jxse1IJIUowYsr\nnMJPtOdjPjg6N50cJzcX6teHjz+2HYmkECV4cYVevM9COlJIju1Q3Kt/f42mkagowYsr9GOayjNO\nShK8bgyUCCnBi3WVOEAPPmA6fWyH4m7nnw8VK8KyZbYjkRShBC/WdWE+X9GC76htOxR383hUppGo\nKMGLdf2YppubIqUEL1HQZGNiRclkYxU4zFbq0YGP2EDjeO6BtJhsLNSRI2bysYULoXE8f1+SajTZ\nmLjeJXzOTmrFObmnsQoVoE8f3fQkEVGCF6s090w5qEwjEVKJRqwwp5dH4vBovjL3QFqWaAAOHoTa\ntWHVKqhTJ4ExiJvFq0TTA1gNrANGlfL9UCAfWA58DFwY9N2mwOdLgS8i2JdkkOaspCKH9Gi+aFWq\nBD176lF+4sgpwWcBz2KSfHPgOqBZyDoFQCdMYv898FLQd37AC7QG2sYerqSTY3PPuOFEMsX066c6\nvDhySvBtgfWYnvghYALQN2SdT4H/BV5/DtQP+V5Hr5RKd6/GoGdPMy/NTz/ZjkRczCnB1wO+CXq/\nJfBZWW4Bgp8t5gfmAouB28oToKSnBkAum1hIR9uhpKZTToGOHfUoPwkr2+H7aK4iXQ7cDLQP+qw9\nsB2oBczB1PIXhm6Yl5d39LXX68Xr9UaxW0lFfYEZ9Oaw45+glKmkTDN4sO1IJAl8Ph8+ny+qbZzK\nJ+2APDg6h+vDwBHgyZD1LgSmBNZbX0ZbjwF7gKdDPtcomgw0z+PhGaYx/YSKX7yk8SiaEt9+C02b\nmp8nnZTAWMSN4jGKZjHQBMgFKgGDgOkh6zTEJPefc3xyrwKcEnhdFegOrHAOW9Lerl1cDMyhm+1I\nUlvt2tCiBcyfbzsScSmnBF8MjABmASuBicAqYHhgAfgNUAN4geOHQ9bBlGOWYS6+zgBmxzF2SVXv\nvcc8YB9VbEeS+jSaRsJwwwgXlWgyTf/+3DBtGmNTvIRivUQDsH49dOgA27aZaQwkY2guGnGfoiKY\nP58ZtuNIF40bQ61a8NlntiMRF1KCl+SaMwcuuojdtuNIJyrTSBmU4CW5pk0zk2VJ/OhRflIGJXhJ\nnuJimDED+iZqaGSGat0aDhyAlSttRyIuowQvyfPRR9CwoVkkfjwelWmkVErwkjzTpplEJPHXv78S\nvJxAwyQlOfx+OPtsU6Jp0eLoI/sSJ/Xbj+q4KC42Nz4tWwYNGiQuLHENDZMU98jPh+xsOP9825Gk\np+xs6N1bc8TLcZTgJTlKyjMeN5w0pimVaSSEErwkh+rvide9O3zxBfzwg+1IxCWU4CXxNm40t9Jf\neqntSNJblSrQpQu8957tSMQllOAl8d55B/r0gaws25GkP5VpJIgSvCTe1KkqzyRL794wdy7s22c7\nEnEBJXhJrJ07zdC9K66wHUlmOO00aNPGzPkjGU8JXhJrxgzo1g0qV7YdSeZQmUYClOAlsTS5WPL1\n7QvvvmtufpKMpgQvibN3L3z4IfTqZTuSzHLWWeZu1o8/th2JWKYEL4kzezZccgnUqGE7ksyjMo2g\nBC+JpJub7OnXT3PES0QJvgewGlgHjCrl+6FAPrAc+Bi4MIptJV0dOmQusPbpYzuSzNSihbnvID+/\n1K9zcmri8XgStuTk1Ezyf7CUxinBZwHPYhJ1c+A6oFnIOgVAJ0xi/z3wUhTbSrpauBAaNdLMhrZ4\nPGHLNIWFuzGzYSZmMe2LbU4Jvi2wHtgEHAImAKGP4/kU+F/g9edA/Si2lXSl8ox9JWUayVhOCb4e\n8E3Q+y2Bz8pyC/B+ObeVdOH3K8G7waWXwvbtUFBgOxKxJNvh+2iu0FwO3Ay0j3bbvLy8o6+9Xi9e\nrzeK3YrrLFpkJr5q3tx2JJktK8tcA3nnHbjnHtvRSIx8Ph8+ny+qbZwm524H5GHq6AAPA0eAJ0PW\nuxCYElhvfZTb6olO6WbUKPMAiscfL3MVPdHJuf24HBfvvQdPPgkLFhzfehJ+/zquEyseT3RaDDQB\ncoFKwCBgesg6DTHJ/eccS+6Rbivpxu+HyZNhwADbkQiYOYDy8+G772xHIhY4JfhiYAQwC1gJTARW\nAcMDC8BvgBrAC8BS4AuHbSWd5efDkSPQurXtSATg5JPNg0BmzLAdiVjghuenqUSTTh59FPbvh6ee\nCruaSjTO7cftuBg/Ht56y8xPU9K6SjQpL5ISjRK8xFfz5vDqq9CuXdjVlOCd24/bcfHjj9CwoXmq\nVrVqpnUl+JQXjxq8SORWroSffoK2bW1HIsGqV4fLLoP333deV9KKErzET8nF1Qr6s3KdgQPh7bdt\nRyFJphKNxE+rVvDMM9Cpk+OqKtE4tx/X42LXLjN1xLZtULWqSjRpQCUaSZ4NG2DHDmjf3nldSb7T\nTjPXRd57z3YkkkRK8BIfkyebqQmysmxHImUZOBAmTbIdhSSRErzEx7//DddcYzsKCad/f/MQlr17\nbUciSaIEL7HbvNlMaNW5s+1IJJySMo1G02QMJXiJ3eTJZlKrihVtRyJONJomoyjBS+wmToRBg2xH\nIZHo1w9mz6aK7TgkKZymCxYJb+NGM4KmSxfbkaSZ7JJhcHE3C+gF/DshrYubqAcvsXn7bXNzk8oz\ncVZMoh6n9zYvMzCZ/ylijRK8xGbCBBg82HYUEoVp9ONKoDJFtkORBFOCl/Jbswa+/RY6drQdiURh\nF6fzOdALjaZJd0rwUn4TJ5pRGbq5KeVMAq5Fo2nSneaikfLx+49NDXzppVFvrrlo7LZfEw8F5FCf\nLezhlATsQXPRJJrmopHEWbECiooc530Xd/oBWEAn+jHNdiiSQErwUj4lY98TNJRPEm8cQxnCeNth\nSAK54ehUiSbV+P3QuLGZuKpNm3I1oRKN/farsIct1Kcpa9jJGXFvX8d1YsWrRNMDWA2sA0aV8v15\nwKfAfuC+kO82Acs5/mHckuoWLzYP9dCDtVNaEVWZQW9dbE1jTgk+C3gWk+SbA9cBzULW2QXcCfyl\nlO39gBdoDeg5bumiZOy7yjMpbzxDGMo422FIgjgl+LbAekxP/BAwAegbss5OYHHg+9IoC6STw4fh\nrbdgyBDbkUgczKUr57CBsymwHYokgFOCrwd8E/R+S+CzSPmBuZh/AG6LLjRxpXnzoG5daBZ6Iiep\nqJiKvM21utiappwmG4v1Kkl7YDtQC5iDqeUvDF0pLy/v6Guv14vX641xt5IwY8fCDTfYjkLiaDxD\n+Be38Dj/h0643cvn8+Hz+aLaxun/ZjsgD1ODB3gYOAI8Wcq6jwF7gKfLaKus7zWKJlXs2QP168Pa\ntXBGbKMuNIrGTe37KaAR/ZlKPq3i1r6O68SKxyiaxUATIBeoBAwCppe1v5D3VeDoLXJVge7ACof9\niZtNmQIdOsSc3MVtPLrYmqYiOR/rCfwdM6LmX8AfgeGB70YDdYBFQA6md1+IGXFzBjAlsF42MC6w\nbSj14FNFt25w221w7bUxN6UevLvab8ZKZtOds/iaI8RjbiH14BMtkh68GwpuSvCpYOtWuOAC87Ny\n5ZibU4J3X/uLuJhHeII5dI9L+zquE0tz0Uj8jB8PV18dl+Qu7jSGYQxjjO0wJI7Ug5dS5eTUpLBw\n99H3y4E7KGUIVEzSqwec6u3X4AcKaEQum/gf1WNuX8d1YqkHL+Vmkrt5xFtLlnIKZ/ERh49+Fvsi\nbrObmsymO4OZYDsUiRMleHF0E68xluvx688l7alMk15UopFSlVwErcQBtlCftnzBJs6O5x5ItxJH\nOrRfgcNspiHdmMMqmsfUvo7rxFKJRmLWj2nk0zLOyV3c6ghZvMEN6sWnCSV4CetWXuEVbrUdhiTR\na9zEz3mT7DLnD5RUoQQvZcplI61YxjT62Q5FkmgtTdnI2VzJLNuhSIyU4KVMwxjDeIZwgJNthyJJ\nNoZh3MyrtsOQGOkiq5Qqy+NhE/X5f7zHCi5MwB7S8yJlurRfjUK+5ixa8BXbqVuu9nVcJ5Yuskq5\ndQe2UTdByV3cbg+nMJFB3MK/bIciMVCCl1LdCvyLW2yHIRa9yC/5BS+RRbHtUKSclODlRFu2cDnw\nFtfZjkQsWk5LtlCfXrxvOxQpJyV4OdFLLzEec5oume1FfskvedF2GFJOusgqxzt4EM46i+Y7drAq\njS8iqv3I2j+ZfXxDA37GoihvdtNF1kTTRVaJ3uTJ0Lw5q2zHIa6wn8q8wQ3cxsu2Q5FyUA9ejteh\nA9x7L54BA3BDD1Lt22//XNbwHzpzFl9zkJMibl/HdWKpBy/Ryc+Hr7+GPn1sRyIuspamrOACBjHR\ndigSJSV4Oea552D4cMjOth2JuMzfuIe7+Tuayz+1RJLgewCrgXXAqFK+Pw/4FNgP3BfltuIWu3fD\npEnmodoiIT6gB1UoohMLbIciUXBK8FnAs5hE3Ry4DmgWss4u4E7gL+XYVtxi9Gi46iqoXdt2JOJC\nfirwD0ZyD3+zHYpEwSnBtwXWA5uAQ8AEoG/IOjuBxYHvo91W3ODgQfjnP+G+0BMwkWPe4AY68BGN\n2GA7FImQU4KvB3wT9H5L4LNIxLKtJNNbb8H550PLlrYjERcroiqvcCt38YztUCRCTlfTYrmiEvG2\neXl5R197vV68Xm8Mu5Wo+P3w9NPw1FO2I5EU8CwjWM6F/Ibf8ROn2g4no/h8Pnw+X1TbOI2Dbwfk\nYeroAA8DR4AnS1n3MWAP8HSU22ocvE2zZ5vSzPLl4Dn251DyTNbEUfup2v44hrCU1vyFB8K2r+M6\nseIxDn4x0ATIBSoBg4DpZe0vhm3FlqefhvvvPy65i4TzJKO4h79xEvtthyIOnBJ8MTACmAWsBCYC\nq4DhgQWgDqbWfg/wa2AzUC3MtuIWS5fCf/8L12nWSIncclryJW24iddshyIO3NBtU4nGlquvhs6d\nYeTIE75SiUbth3MpnzCOoTRhHYdLvZSnEk2iaaoCKduKFfDJJ7qxScrlUy5jE7kMZoLtUCQMJfhM\n9fjjcO+9UKWK7UgkRT3BIzzMH/FwxHYoUoaMTPA5OTXxeDwJW3Jyatr+Twxv9WqYPx9uv912JJLC\n5tKVIqrQn6m2Q5EyZGQNPhn1ZVfXH6+/Hpo2hV//usxVVINX+5Hoyfv8mQdpST5HyDqufVcfA2lA\nNXg50erV8MEHMGKE7UgkDcykJz+Ro1q8S6kHn5g9uLf3MnAgXHwxjAo/uad68Go/Ul4+5CV+QXNW\nUkzFo+279hhIE+rBy/EWLTIjZ+6803YkkkZ8XM5mGnIjr9sORUKoB5+YPbiz99KtG1xzjXmohwP1\n4NV+NNrxKRMZRBPWBR7r59JjII2oBy/HzJsHmzbBzTfbjkTS0GdcyjJaMYJnbYciQdSDT8we3NV7\nOXIE2rUzk4oNGhTRJurBq/1oNWU1C+lIM1axi1ruOgbSkHrwYrz5pplMbOBA25FIGlvDeUxgMHnk\n2Q5FAlz3dOXvv/+eMWPG6F//eCkshIcfhilToIL+PZfEyiOP1ZzH87YDEcCFCf6jjz7i0UdHc/jw\n1Qlp3+/fnpB2Xevxx83F1UsusR2JZIAfOI0neIS/cK/tUAQXJniAk05qwU8//TlBrX8BvJmgtl1m\n3Tp45RUzsZhIkjzHHdzOvTBjBvTubTucjKZz9nTl98M998ADD8CZZ9qORjLIISrxKzB3S+/dazuc\njKYEn64mT4aCArj7btuRSAaaB9C+Pfz2t7ZDyWhK8Olo92646y5TnjnpJNvRSKb661/htdfM837F\nCiX4dPTAA+ZpTZddZjsSyWS1a8Mf/gC/+IW5F0OSLpIE3wNYDawDypqh6pnA9/lA66DPNwHLgaWY\nq5uSaPPnw+zZ8MQTtiMRgVtvhUqV4O9/tx1JRnIaRZMFPAt0BbYCi4DpHP/w7F5AY6AJcAnwAtAu\n8J0f8AI/xC1iKVthoXkE33PPQU6O7WhEzL0Xr71mhuleeSWcf77tiDKKUw++LbAe0xM/BEwA+oas\n0weOTiP3OVAdqB30vRumQ8gMI0fC5ZfDVVfZjkTkmEaN4I9/NA+aOXjQdjQZxSnB1wO+CXq/JfBZ\npOv4gbnAYkBPd06kyZNh4UKdCos73XIL1KsHv/ud7UgyilOJJtL5AsrqpXcAtgG1gDmYWv7CCNuU\nSG3dCr/6FUyfDtWq2Y5G5EQeD7z8MrRuDV27gtdrO6KM4JTgtwINgt43wPTQw61TP/AZmOQOsBOY\niin5nJDg8/LyjgWU7cqba93r0CG49lozLFLTEYib1akDr78OQ4bAkiW6AS9KPp8Pn88X1zazgQ1A\nLlAJWAY0C1mnF/B+4HU74LPA6yrAKYHXVYGPge6l7MMfbOrUqf6cnL5+cytmIpbP/UAC2/f7Q/+b\nEuruu/3+3r39/sOH49psMn5Haj+92y9TXp7f36mT33/oUFz/ZjON+X8YnlMNvhgYAcwCVgITMSNo\nhgcWAsm9AHMxdjSYu5SBOpje+jLMxdcZwGyngCQKb79tyjJvvKGZIiV1/PrXcPLJ8MgjtiNJe5HU\nQ2YGlmCjQ96PKGW7AqBVeYKSCCxZAnfcYca816hhOxqRyGVlwbhx5uHvF14IP/+57YjSlgreqWjL\nFujbF0aPNhetRFLN6afDe++ZYb0NG0KnTrYjSks6r081hYVmCtaRI810BCKp6vzzYfx4M0hg7Vrb\n0aQlJfhUsn8/9OtnRsvcf7/taERi17WreShNr16wbZvz+hIVlWhSRclwyFq14PnnzbhikXRwyy3w\n3Xcm2ft8cMYZtiNKG0rwqaC4GG64wYw+GzvWXKQSSScPP2zOULt2hQ8/hNNOsx1RWlCCd7sDB2Dw\nYPNz8mSoWNF2RCKJkZdnkny3bvDBB+rJx4Fq8G5WVAR9+pge+7RpULmy7YhEEsfjgT/9yUyW16ED\nbNxoO6KUpwTvVtu3Q+fO5nbuCRPMnNoi6c7jMY/5GzkSOnaE/HzbEaU0JXg3WrbMjJTp1w/GjAHN\nzyOZ5o7YLo8hAAAHZElEQVQ7zCP/unUzZ69SLsocbjNhAtx5pxkpM3Cg7WhE7Ln2Wjj7bBgwwNy5\nnZenAQZRUg/eLYqKzNOYHn0UZs1SchcB+NnPYNEi+M9/zFj57dttR5RSlODd4MsvoW1bk+S//BLa\ntLEdkYh71K4N8+aZsmXr1jBliu2IUoYSvE1FRTBqFPTsaX6++SaccorzdiKZpmJF8zSoqVPhwQfN\nBGU7dtiOyvWU4G3w+800vy1bwubNsHy5eV6l7k4VCe/SS80ghLp14YIL4J//NDcCSqmU4JNt8WIz\ng94jj5g/zrfeMqegUcjJqYnH40noIuJa1arBn/9s6vJTp5qO0pQpgeeMSDAl+GT56CMzC2SfPjB0\nqOmF9OhRrqYKC3djHuaSyEXE5Zo3N7X5p56C3//eXJCdPh2OHLEdmWsowSfSwYNmeoGOHeHGG80d\negUFZrSMxraLxM7jMaNrliyBhx4yib5pU3j2Wdizx3Z01inBJ0BTMBeCGjaEZ56BESNgzRoYPtw8\nqkxE4qtCBbjmGvjiC3NzoM9njr9bbzWlnAzt1asbGRd+LmQ5A5jM1UyhZsnHCxbAuefaDEwks3g8\nZh6bDh3M/PLjx5sOVmGheUBO797mjDpDJu2LpAffA1gNrANGlbHOM4Hv84HgZ8hFsm1KqscWrucN\nxnATm2nIVPpThSJu42Xqg7kIpOQuYk/duubBOMuXm9p8jRqmjFO7trlL9vnn4auv0rp379SDzwKe\nBboCW4FFwHRgVdA6vYDGQBPgEuAFoF2E26YAP3XYQX3G0o1iLmYxF7OYyuzjQy5nPl14gkdYRxPA\nDaNPfIDXcgzpxIf7f58+3B8j+Hw+vF5v8nfs8ZiHe194oblTfPt2c7f4ggVmvpvdu6FdO3MTVatW\n+A4cwHvddabsk+KcEnxbYD2wKfB+AtCX45N0H+D1wOvPgepAHeDsCLZ1harsoR5bjy6NKOBc1tKU\nNZzLWvZzMg9QhRoMZCKDuJ+/sJGzcUdCD+UjFQ721OHD/b9PH+6P0WKCD3XmmXDTTWYBU8r57DMz\nc+XYsSbO4cOhcWM45xzzs3FjaNTInBWceSacempK3LfilODrAd8Evd+C6aU7rVMPqBvBtjGpwGEq\ns48qFFGZfce9Lvl5CoXUYHfQspYaQA06U4ud1GMrFTkUlN7rsYlcZnEl/+RO1nIuu6kJ5AUWEUkr\ndeua+nzJQ+zz8uDuu2HDBrOsXw+ffgrjxpne/7Zt5uaqM880y+mnQ/XqpgRUvfqx16eeClWrmuc4\nVK4MVaqc+DrBo+mcWo90QHRs/5RdfLH5hR0+zBU//siSwu1ke6qSxRGy8ZOF3/z0B70OLEVksd+T\nxT4qBH5msc+TxX4qUOTJYi/Z/OipyI+eiuzyVGSd/xA7DsJufsv3nM5W6vEj1WP+TxCRNFK9Olx0\nkVlKs2ePSfbbt8OuXfDjj8eWgoJjr4uKYN8+s5T2GkySL1kqVjz+fehnWVmmdBSn8lE74IOg9w9z\n4sXSF4HBQe9XA7Uj3BZMGSfRd+1o0aJFS7ot64lRNrAByAUqAcuAZiHr9ALeD7xuB3wWxbYiImJR\nT2AN5l+LhwOfDQ8sJZ4NfJ8PtHHYVkRERERE0sV9wBE4djOoy/wec5ayDJgHNLAbTpmewgxHzQem\nAKfaDadUA4H/Aoc5/qzPLVLhJr1XgW+BFbYDcdAA+BDz//sr4C674ZTpZMxQ72XASuCPdsMJKwtY\nCrxrO5BINcBclN2IexN88NM47gResRWIg24cu0v5T4HFbc4DzsUc+G5L8FmYsmIuUBH3Xj/qiLlz\n3O0Jvg7QKvC6GqZs68bfJ0CVwM9szPXEDhZjCedeYBzm5tEyuelWrb8CD9oOwkFh0OtqwPe2AnEw\nB3MmBKZHUt9iLGVZDay1HUQZgm/wO8Sxm/TcZiGw23YQEdiB+UcSYA/m7LKuvXDCKgr8rIT5h/4H\ni7GUpT5mcMsrOIzvdkuC74u5EWq57UAi8DiwGbgRd/aMQ93MsVFOEpmybt6T2OVizjo+txxHWSpg\n/jH6FnN2udJuOKX6G/AAxzpxZUrmbJJzMKdqof4PM8Kme9BnNu86KivORzD1rv8LLA9hftHDkhfa\ncZziBBPnQWB8soIKEUmMbuS3HUCaqgb8GxiJ6cm70RFMOelUYBZmDgifxXhC9Qa+w9TfvXZDiUwL\nzL+WGwPLIcyp8RkWY4pEQ8wFI7e6CfgYc+HIzdxYg4/0Jj03yMX9NXgw1zJmAXfbDiQKjwL32w4i\nxBOYs8uNwHZgL/CG1Yii5OaLrE2CXt8JjLUViIMemBELp9sOJAIfAmXcD25NKt2kl4v7E7wHk4T+\nZjsQB6djJksEqAwsAK6wF46jzrj7TLhUBbg3wf8bczAtAybj3rOMdcDXmNO4pcDzdsMpVX9MT2Qf\n5iLcTLvhnCAVbtJ7C9gGHMD8Lm2VC510wJQ+lnHsb7J8DyROrAuALzFxLsfUud2sMw6jaERERERE\nRERERERERERERERERERERERERERErPv/buCDahLUUTcAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Metropolis Hastings: Accepted 2081 out of 2500 samples\n" ] } ], "source": [ "sampling = MetropolisHastings(log_p=lambda x: np.log(p(x)), \n", " q=Proposer(lambda x: norm(loc=x, scale=0.5).rvs(),\n", " lambda x, y: norm(loc=x, scale=0.5).logpdf(y)),\n", " x=-3.0)\n", "\n", "show_sampling(sampling, plotter=gauss_hist, N=2500)\n", "print sampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using again the more difficult 2-dimensional example, we can illustrate how the sampler moves around the target density:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Metropolis Hastings: Accepted 221 out of 1000 samples\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD7CAYAAAChScXIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XV81dUfx/HXXbJi1Aaju7uRmoSENCJIl3QpCAIiKSCN\npCIISCglgrTA6C7phlGjNthY797P74+vv41Js2Lj83w89vDe+61zrvDm7HzP9xxQSimllFJKKaWU\nUkoppZRSSimllFJKKaVUImCK6wsUKVJETpw4EdeXUUqppGYH4PnfD63i+qonTpxARJLsz9ChQxO8\nDFpHrZ/WL+HLEds/QOXnZWqch7ZSSqnYo6GtlFKJiIZ2DHl6eiZ0EeJcUq+j1i9xS+r1+684vxEJ\nyL/9M0oppV6TyWSC52S0trSVUioR0dBWSqlERENbKaUSEQ1tpZRKRDS0lVIqEdHQVkqpRERDWyml\nEhENbaWUSkQ0tJVSKhHR0FZKqUREQ1sppRIRDW2llEpENLSVUioR0dBWSqlERENbKaUSEQ1tpZRK\nRDS0lVIqEdHQVkqpRERDWymlEhENbaWUSkQ0tJVSKhHR0FZKqUREQ1sppRIRDW2llEpENLSVUioR\niWloZwK2A6eBU0CvGJdIKaXUC5lieHy6f3+OA87AEaABcPapfUREYngZpZR6v5hMJnhORse0pe2D\nEdgATzDCOn0Mz6mUUuoFbGLxXFmBYsCBWDynUkpFIyKcOXMfH58nPHwYTIoUyfjooxwJXax4E1uh\n7QysAHpjtLijGTZsWORrT09PPD09Y+mySqn3xYULD1m06B8WLz6JiJA1awrs7W3YuPESFsu3/+9O\nSLS8vLzw8vJ65X6xUUtb4C9gAzDlOdu1T1sp9VZCQyNYu/YCs2cf5uTJezRvXpCWLQtTvLgHJpOJ\n/ftv0r37eo4c6ZTQRY11L+rTjmlL2wTMBc7w/MBWSqk3du9eILNmHWLWrMPky+dGp07FadQoH/b2\nUZEVHm5m4MCtNGiQJwFLGv9i2tKuAOwE/gH+35weCGx8ah9taSulXou392O+/343S5acokmT/PTp\nU5b8+d2e2U9E6NVrA5cv+7F27WdYWye9R07iqqW9G31ARykVQxcvPuS773axdu0FOnQoxvnzPXB3\nd3rh/mPH7mb79mvs2tUuSQb2y8Tm6BGllHojly75MmbMLtasuUCvXqW5dKknKVM6vPSYX345xk8/\nHWXPnvav3Dcp0tBWSsW769cfMXz4DtauvUDXriW5cKHHawXwr7+eYPDgbWzd2pr06V3ioaTvnvfr\n9wqlVIK6fTuAHj3WU7z4T2TI4MLFCz0YUcSXlGOGvvLYOXOOMGDA3/z9d2vy5Xu2n/t9oaGtlIpz\nDx8G0b//FgoWnEmyZDacPdudkU3dSfFJHfjkEyhY8KXHjx27mzFjduPl1fa5NybfJ9o9opSKG2Yz\ngU9CmTbrKBMn7qNRo7ycPNmVDI5mGDYIZs6EiAiYMgVat37BKSwMGPA369dfZNeudmTIkDyeK/Hu\n0Za2UirWmbd74ZcxF9MyNubIkTvs2tWOH2fWJsP63yFfPrh3Dzw84NtvoXfv554jNDSCpk1XcOjQ\nbQ3sp2hoK6Vijdy/j3e1xjz4qCEuPtepP607y5c3Ia/vBShdGhYsgFWr4NYtqF8fnpri4mm+vsHU\nrLkYgM2bW5I6tWM81uLdpqGtlIo5EbxHTMU3Q04OHr2Ldf48WNvbkq96MWjVCj79FPr2hW3bYMwY\nyJQJpk6F58wXcumSL2XL/kzJkh78/vsn0Z6CVNqnrZSKodtbD+LfvB0hvv7cqt+OxsfWYvrnirGx\nSBHo1AnOnQNHR6PvWgTmzwerZ9uMu3d706TJcoYNq0znziXjtyKJhIa2UurNRUQQdP8Rh5t+QYHd\nq7hZqSmVsgh2C6dG7hKUNS+zW6xl092cHK8pLA7tzYd23lhv2Qi2ts+cctGif/jyy038+mtDatTI\nGZ+1SVQ0tJVSb0QsFky2tjgCthnLwsiRVJs1CS5H7+oomvcszdJCj6pQfs1wbs3Zxc5VXnzoGL1/\n2mIRBg3ayrJlp9m+vQ0FCrjHY20SHw1tpdRrO7bhGMVqFwfgWo/BlLt9FqaNI8IpOTZXLnLJITfh\nyVOTrFolzs18gpWLE0ybBlOG45O8KJ5bvoP7RaBwYciblyehQuvWf/DwYTCHDn2uNxxfQ3zMGq6z\n/CmVyN2+8YgtnwygzcGfAJDUqTE9fEhY1pyYb9/FISyAkzlrkSW3M8nXLwdra7C3h9BQLPbJsAoK\n5OC3Kyhtfx7++QdOnMBy7ToXJBWnClej3u652NlZJ3At3y0vmuVPQ1sp9UIhIREs/mohRWcPI01a\nF7LcOhNt+3GnEqRNEUG6+2cxublBQIDRit61C4Dr18JZUm0Sn2Q8Qy6vBZHH7Vq2n0etO/Ghw12c\nF82Djz+O13olBnG1sK9SKgkSEVYuOsKitNX55KcvyN7ts2cCGyBf34Z43DqBydER5s2DChWgc2cA\nLntD5Q42dAn6hVzfGSvLSFgYf3/cgwLNqlK4UXmcb1zWwH5D2tJWSkVz4oQPvXtvpKj3QcaErsOh\naWOYPDnaPiEjvifZtwOMN5Urw+rVEB4OuXLBzZtc8XOmSjuYUn4nDZZ3gdOnCV6/mYfNO3ATV7Ku\nWUi6yiUSoHaJh3aPKKVe6v79QAYP3saff55n6NDKdDEdxqpbt2j7iMmE6em/zwMGwNixxuvp02Hf\nPk4PW0yNTjC4M3T9qyWkS4f/6UsEbN3FHxU70fGvESRzeHbIn4pOQ1sp9VwRERZ++ukIw4Z50bx5\nIYZ1zIHz0MFY/7EiekAXKABPnsD168b7TJnA2ztqe7lynGnxLVV+r8XE/tAi93nImxeAiY5VSTNh\nBG26fhCPNUvc4mq5MaVUIrZz53W6d1+Pm5sjWzc0o8DfS7GUaIRNWPCzO58+Hf396NFRry9dIuTc\nFaourc78MULNGc1g2TIAKqUfzMTVXSlVKkMc1uT9oaGt1Hvo1i1/Bg7cyvbt15g8uQaNU9wmqFF1\nrLzPv3B0wv5qdUl56Tx5rl2AKlXA3XgIRgR2fbmES05N2dHzMLkrlwPgxyyNWVOgEX8sbKDjr2OR\njh5R6j0SEWFh0qR9FCkym4wZk3N2Sz0+mj8cU/XqOHmfxz9jHiRvXkLz5I123KppCzCXr0Ceaxew\n5M5tTK3q5kZoGLT5Wsi1dS7tb00j92dGYOdO9x33OvRk7drPNLBjmba0lXpPeHldo1evDaRN68xe\nr1ZkWLUIh/wfYSUWwuydsfqyD1Z/b+FwirTY3LlNsaeObeiWDN/vfuBg8w6UXjIX8ublXsqctGn5\niA3LU0bud65wFTzv1uLXhQ2pXj1H/FfyPaA3IpVK4u7dC6Rfv814eRldIfWc7xDQthupfC4AENy2\nC1fq1sWtR2d+a9iWprvXk/afo8bBDRoYw/kA/+QpSO7/CABLMgesQqL3e88u2pHjoSkZNuFj0uVK\nD87Oxo+T03Nn9FMv96IbkfFBlFLxLyLCLNOnH5D8qYbIlfT5JXDzdrn5YVMRoxtanpSoIPt3H5XR\n4+bJg5RpZP/w8ZHbBETWrhVZty5q/9Zto29/kx8nJ5G0aUVy5BApWlTkl18S+ut55wHPbe1qS1up\nJOjIkdt06bKOzNZPWHprCnY3o4bmhSVPxYUp0+n5QWM6fNeXens24ly9GlY/zo46QY8esGkTXLz4\nWtd74p4B54Z1wMHh+T/JkkV/X6gQpEoV29VOUnSctlLvgcePQxg8eBsrVpxh2lcFaNKvdrTtt/oN\nplu7r7njH8Tavk1wv38H0/XrEBYWswuvWQN168bsHCoaHaetVBImIixbdpovv9zMxx/n4tzc4qSo\nUzVye4RrCjY168jYKi1Z1KsJWbZufKvrRCxdTuBvyzGtXYuNtQnH8CCwWJ67bJiKGxraSiVyly/7\n0q3beu7cCWD5kvrkmDGBFHVmR9un7LJD7KtfiI8e38L2LQMbILBPXyYGFCLn+LW0WjUUevbUwI5n\nsfFtzwM+Bu4BhZ6zXbtHlIoDYWFmJkzYy6RJ+xgwoDztc4WSumFU6/po76+x3byRQmePx9o1C+ed\nytIVzSjgdwHatjXWfrTRtl9ciMs+7YrAE2AhGtpKxQsvr2t067aO7NlTMn1wccK+GknuPUsitwcn\nc8Bsa4dzwGPC8uXA7uzlV54z0NEJp6DAaJ8FDxrBoychePxgPLL+xNsH50xpoU4dow/732lYVeyL\ny/m0dwF+sXAepdQr3LsXSJs2q2nd+g/GjKjEjLw+pPCswMOrxl/Buxmz0Gfq7+zfe5ygr9oCvFZg\nA5GBHYE113J7AuAw+luuz1zCzt4TwckJ53Sp4ORJOHIE2rSJ9fqpV9MR70olAmazhVmzDlGw4Ezc\n3BzZs6U1uft/gffcdRzu+QMFwo/SfuoK/ti4l4H++yjRsBzu30599Yn/o26RPXi3/IrMD05yKk1+\nrtino3haC5Wq5YYsWYzAHjcO+vQxhvGpeBdbdxCyAmvR7hGlYt3Ro3fo2nUddnbWzJjxMSePWpO/\nZy0C8xbl5jdtqNauAafqN6Xo40skX/s3VhHmt7qOV6ZGVPLfxt2q9WmyPxMlm5Tn+++rYb9zO3Tv\nbozZbtYMtmyBy5fB1TWWa6qelqBD/oYNGxb52tPTE09Pz/i4rFKJmr9/KN98s41ly04zenRVSnxQ\nlCEDbzN524f4N63L0g8qM6NBBQAKHV9LoJM1Kd4ysAE8b6zifJ7yfLHdha/mNad+g3zGhurVjcV4\nc+WC334zFj7QwI51Xl5eeHl5vXI/bWkr9Y4REVatOkufPpuoUSMHQ4ZWY9pSR3b9fonNN6pxPU8u\nAsPCKXdwB+EuDizZ+jkVz6Ume+uhb3W9Uz2mkqLHJ/Rqt5xa9/bSznwYG2dH6NgRWrWKenLx1Cko\nXRp8fbVrJB7E5dwjS4HbQChwA2j3n+0J8di+UonS1at+Urv2YsmXb7ps335VVm0RyfShyJjPNkfO\n4/Ega3YRkEPdysoO2SmhAQ/faj6QScWmyNWrYbJ8+Wlxdx8vY8bskogIs4jFIrJ9u0jz5iKuriKf\nfSaybZtIeLhI8uQi9+8n9Nf0XkDnHlHq3RUebmbq1AOMHbubvn3L0ajZB/Qbb43NP8dYeLEuLvdu\nRe575tNCpPG3w2XSTzjkf7vFcfuNtfBNl3C+6reR7duvsXRp4+evLOPrC4sWwZw5EBICfn4wf74x\n5E/FKZ17RKl31IEDN/n887V4eLgweUpt/tiRiskLoX1naP5zdYru/jty3+C0rjjcffzW13pok5oD\n627insqXFi1WUa5cRqZNq4WLi/3LDxSBAwfgl1+gRQuoVOmty6Bej4a2Uu+Y/0/utGb5P8z8Kj9Z\nQ4JINXoAR3LW5WHXqqy1c2Lh180JzOGK+4ErLz9Zl5Iw+/BLd/GzdyNs7UYWHAtmwoS9TJtWi6ZN\nC8ZijVRs0tBWKqGEh4OtbdT74GDOtu+P76qN5LV7TMrQxxARgZVYAHji5ML+Oo2o9vuCV597Sgv4\nYRlcCX/hLrfsMhKRKRuujTxpciwXQUHhLFnSiCxZUsS0ZioOaWgrlVCKFjW6F5o3x9chFSGDvuWE\nyQOPIX0461CB1bMvMvdac86V/YDlVery/Tex+2i4f6FymMOCKejXjI5dSjNkSGVsbPS5unedhrZS\nCaVkSSz16mE1NGpInveQ6XQ+3ZSPri2m14m+WJvffnz1fwWmzUC3PMtZsPODyM8qphvI6GWdqVgx\nS6xdR8UtDW2lEsq/U5euca9I1nmTuT/1T8punYiTJejNz+XiDAFPXrpL+rKhHH1YmXQX93PXNgVp\nwx/hv34ryWtVeZvSqwQSlxNGKaWeFhwMZ84QPn1mtLmmq1nfpHCdklTdMvLtArtovhcG9r0ceQiz\ntmdU4Zn8k38gKW+dBiA0T37kxx9JPnnsW1VFvXs0tJWKLWazMZmSoyMUKIBtz+6Rmw7nqEftbJtY\nNPL3tz//8bPP/XjHlxOpnmEvZht7ure1wvzbr5zFDYDMm1dgatsWLlyAvXvf/trqnaHdI0rFhjNn\noF07zL5+mC5fxkosXPNsgIRYk23/SgCCHZ1w+M981THhkyELS7ps5PtNedlY7hdyLx9LsPdtfms8\nlB7Lv4L27WHuXGPnOXNgxQpjsV6VKGiftlJxISICJkyA8eN5nCId5mvXeeCWBQ8PV+oV2Yt/KAxN\nO5p6UwfH+qWHVfyNs+7lGDMpPQ4lcuHx4BoWG1us0nuAtzcEBICzs7FzWBjkzg1Ll0K5crFeFhX7\nNLSVim2nThmt67v3eegbzO0IB4J+mM2D/Q8IW7uB3bMXs60I/JMrdv+abe80lkerjtPwwW/Pbvzw\nQyhYEA4ffrY75KefYNUq2Pj2a0Sq+KM3IpWKLRERMHo0UqkS/jfv8fjmffaXasjj7efosr4mPufv\n4FgvHSsKR9Ah43YO9Ckfa5c+kasWRX4ZTy37A5wvU5u+To24WPIjY2PPnsZ/58+HevWePbhtWzh7\nFvbvj7XyqPinK3Iq9SZOnjRa19438Q80cyHcAdNvW9l+pSxL+kPzEWBaeIeQzPb0yTmD/A/vUWbK\nnlef184EYS//jTTI2pGzGTxJPn8Kbcce4datAJYcaUSudLbGPySpUxs7ijx/hXQ7Oxg4EIYPhw0b\n3qLy6l2g3SNKvY7wcBg7Fhk/niCTHSEBweyr3wPp+R19RkGZ8hGENonA/cjfzGrf8M3OncIGHkU8\nd9P1bLnxqjeG4GW7aBWxCqf73lyzTs29QmUoNm80tsWKvNm1QkONxQxWrDDmxlbvLO3TVuptnTxp\nLGJ77FjkR3catmVA7l+4sucqu3dnB8BsY4N1xPPD920F2zjxxNaV5OmTc+uJcOOxifzlcuCWM71R\npvJv0fUyaxb89ResWxerZVWxS0Nbqbdk6d0Hqx+iFsl9kKcMnzj+zIprH5PGzzvWr3cpWx5Wj93K\nxKlp6dnahg+L3KJ1q1WUL5+JqVNr4uoaw1VjtLWdKOiNSKXewtUff+fO7EVsTvcBvhVrAlA/21p+\n921EGj9vQp0dXnzwtwMhTZo3ut6YGSsY3PIcP/2SgVXTTYTd8aJ+vSWMHl2F+fMbxDywAezt4euv\nYcSImJ9LxTttaSv1HGE+9zlTqzWp/9nPqe4jyHT+HAU3z+Sxa1ou5cpNicO7oh9gZwWF0sORm8b7\npk3h91c//RjmZIddYBgAo7beY85oN2pXhM4NH9Kp4ypSp3Zk7tx6pE/vErsVDA2FnDmNIYClSsXu\nuVWs0Ja2Uq/p0rg5PMqUixt+Fm79fYqQzeexPWiMAHF9fPfZwC6QDtLbRgU2vFZgA9gFhnGuQHG6\nTDczY5gbPwwSCqY5SLUqc2nTpgjr1zeP/cAGbW0nYtrSVupfIVdvcKlmc+wvn+fywPGcyd6K5AN7\nUdXlEGlvncIxOIi7RdKT9sTtGF3nWOfSFPvxIAB/fDWe4Vf6kTktjOr2hP59V+PnF8LChQ3Ik+fN\nulbeWEgIFCoE1aoZc6a4xME/DuqtaUtbqRcR4dLgiQTmys85KzfubD3HsAutyTq+Jx8n20SAQwSO\nwUH4lM0YPbDTv2b/sr11tLf/D+xZ32yi055+dPsUPqtwiupVZlOuXEb27Gkf94ENkCwZHDpkhHeR\nIuDlFffXVDGmLW31Xgs+cwHvWs0IvX2P26OmcsS1MVMWCitS9aDyupkEJXfB4iA4332CJANTyBuc\nPI0zPHh2KtVgJxe61drLRfuCTB0QwoTR6zhxwodff21IiRLpY69yb+Kvv6BzZ2jcGMaMASenhCmH\niqQtbaWeZjZzqddQggsV52jK/PhuO8vAA43ZelpYkM0I7Ah7Ww7OrYGT2UjqNwpseG5ge+cuSan8\n58lZtSBfN7tI/VozcXd35MiRTgkX2AB16hjj0f38jOXRdu9OuLKoBCdKvUsCDx2XK+nyyj677LLh\nhw3yzVSRNOVFum80y/pq9UWMB8HF5+dKYslpHfn+TX7M9VwjXw+cNi/ydclaT2TPkTDp3HmtZMky\nWbZuvZLQX8ezVq0SSZdO5MsvRYKCEro07y3guV0U2j2i3ivn+o8j+/hBYGVFYOEy+Fx5QgrTY1zE\nD2f/RzE+v+xIw+NjOUjR5wAAnn8eZ1nPT3H3vsCest0I/q4/nT//k0qVsjBlSo3YGXcdFx48gO7d\n4cQJYwKqsmUTukTvHX0iUr3XnjwJY8CALdxYvpmhn3pwJjAPqw+7UqqHK6uzubC/Rh4AxM4KU5jl\nmeNPZchAwVu3XnoNy4n03PgxPS5bb/DlnD9x9DEz81PjMfOAHAX5uXw3xm8JYvbsOtSrlyf2KxkX\nli2DXr2MGQKHDTNuXqp4oX3a6r21bdtVChWaRWBgOD1XfU+ra1+zNF1DnvxRhS2VUkUGNvDcwA7J\nlIkCwcEvvoA1+O+sjHffZJgvQOlfT1Nj4/HIwL7YexQf2H7OnsDUnDjRJfEENsCnnxqt7QsXoEQJ\nY55uleQlbMeQem8FBIRKt25/ScaMk+SP1Rel/wQR9woi3XaIuJ+LkMVLvovsa7ZktxZJ6/hmfdfO\ntiIgF3+oJg9zu8naju2lzJlQuZsmuwjIw7yl5ft+f4i7+3hZvPgfsVgsCf2VvD2LRWTxYhE3N5Fv\nvhEJDU3oEiV5vKBPW1vaKkn6f+s6KCiCeUu7MXBOTk4/hMLzBLcTs7mb14bmzf9dAuyH4phKpMYS\n+Gwr+4V6uWHJbMfVj3ORfuQhRrcazrnybdmf3x73B1c4PX4R1Rzas+NMEMeOdaZ580L//3U3cTKZ\noHlzo9V9/Ljx6Pvx4wldKvWWagLngIvAgOdsT+h/sNR7JCAgVLp3X/ds63qn0br+/uKq6K3lMh5i\nLukW1eI2mV7euu5dXCzJkEc1s4iA+LmlkS+m/i5Hi0aNOpk0fJOkSTNOfv75SOJuXb+IxSIyf75I\nmjQiw4eLhIUldImSJOJo9Ig1cB6oBtwCDgGfAWf/E9oxvIxSr+bldY0OHdZQsWJmWnWqRc/R9mQr\nDGHtIcT+Hq3D5vBpk2m4HrsLKe0glz0cDHj9C8zOSUgNV5JlOwLAzfSZOF+5HlWXzgDAp04LGt+v\nTDIne+bNq0eWLCnioprvjps3oWNHuH8fFiww1qZUsSaubkSWBi4B14Bw4DegfgzPqdQbCQwMo1ev\nDbRsuYqJk2rhUbgBzQfY82F/ONzGTNl0u/jiSG86lBlhBHYGZyzF3KMFtowa9cppVH2P2rFrtUfk\ne6dgM8WWLwVgW4MvKLivMM1bFWHLllZJP7ABMmY0li3r2tVYUHjMGGPZMxWnYtrS/gSoAXz+7/uW\nQBmg51P7aEtbxZldu67Trt2flCuXibZda9F7bDIy5wdzRwiwu0+TtCuoP+gXsv12FFMyB7gVCKFR\nfx7NefNi/egR+Pg8/wLJrLDktsdyIRybkKhAuumUjTPpK/Hh/b/4Jn0L9qYqyi+/1CdnzlRxXeV3\n0/Xr0KEDBAQY47rz5UvoEiV6L2ppx3Rh39dK42HDhkW+9vT0xNPTM4aXVe+74OBwvvlmG0uXnuKH\naR/zz+28fDYAPh0BK9Jb6Jh2Hx5+q/j8wwXYWgdgCrGF288+Vm597twzn0nGZJhuhiBN7Lk4rwUr\nHHNy91YVCoxYSaefxwMQkacA5a5s4CNpQb2OzdjRuyxWVon4RmNMZckCW7bA7Nnw008weXJClyjR\n8fLywus1Ju2K6Z+yssAwjJuRAAMBC/D9U/toS1vFqv37b9KmzWqKF/egc+/a9PneAffsYNsV7ts+\npGnGP/hg8xbKtl2JqVxa+PPlU6mGm0xYmUxYWyyQIRnUT4MsvM3KM925nrYs12dH8O3YbzhsX4Ff\nSo2jb+snXOn2BUvdqjBuSbv4mZFPvXfi6olIG4wbkVWB28BB9EakiiOhoREMHerF/PnHmTq1Npf8\n8jN1ETQdBisyWWjjcQhXx410H7oSl7EnMb1iBJ/ZxoazNjbksbbCJjwYk70tssuJsJawp1UZ9n3Q\nnqp9Z5Dh7iO6Ok+j5sAKmB4fZui32+nfvzx9+5bD2lpHzaq4EVfdIxFAD2ATxkiSuUQPbKVixdGj\nd2jd+g9y507N8jVd6TvRCac0UGE+7LR5TP9Mf5Dj7nHq1J+D1W0r4/e9f91xccEjIPookV1ubmS1\ntSV/qD9WqYDHtoStzcLyfI0oXnAX94550GNcD2bnHMamip0Y1i2QEUN+JSAgjF272pEvn1t8Vl+p\nSDENbYAN//4oFbtWrUJ69+afVPmYczU1Q8d04oaTJ42+MNFsIKzOLTRJc4JSydfTdNsePFpvxBRq\nD77R+67/G9gAFe/fN14UdUN8HnJ9UwN+K1CMLNNDSbPlEulKFqFU9jP07JGKTwKO0Ljedvr2LUe/\nfh9gY6Ota5VwdMIo9c4KLF6G6fez4+jqQHO3B9jt3cUd56yca/QRKytVpGTDh6Swv8BnY+dj8+3r\nLwFmzuOBdQ4TrDeO2T+zJlfzl6Fwt9VYOSbnO48fuJmyKCO7+jP8m9X4+4cyf34D8ufX1rWKPzrL\nn0o0zGYLv/ZbSLMpn3Olcn1M1impFDKHZr0i8OEgbY4soejutWTc5/1a5zuXPTt5r1wx3mSxhxTJ\n4YTR0t48tQ6uuyHTruOsbzqOrw80Y2h3sHlyhG+HaOtaJZy46tNWKlZdvuxLpxa/s/VANwA8Du5i\nebbO1FkJf1qb+TLLPW5+kprq61xf+RjXkfbt2bl6NW0t5qgPHaywOARjcbDh3KcVKDlsL7uadOar\nykuxfuTMyomPGTHkTwICQtmxo622rtU7R1va6p0gIvw4Yz/nvp7IuPD12IUFU6fwbhbe/Iz2v67H\nqYQLhd1X8dGl8xTL/WPUgYVywslLr75AqbTI0GBu+pQgU8ftAGz7sA5/15vMT8tyMryHwCNjZMhX\nX31A377aulYJS1va6p116+Zjfqw7nLbnfqN5/mwcDG7E48dmPL+y4v5wFypVuQO2K+lwajeuhf82\nDsrniqU8ofxNAAAf+0lEQVRjP6z6Dok8j9nVFf/QUK6lSkWx20/1cWe1Q84+wFTHTCaMwJ4wax1L\ndtYmzSlYOeERwwb/SXBwuI4MUe88bWmrBCMibBq5hOSjhpAzpeD95Q802vgxB07nY9i0udTa8itp\nM97i8YAi1Og1DdPP/44CaVYey9H7WF24EHmuJw4OOP93oYLplZDGZzlq1YgCuRaQzN9Ymdc/uTuP\nQ5LxpEJ1rmTOTu8/w+g6+CP69Cmr467VO+NFLe34EN8zGqpEwPfQSdmTsZzcsUkhFwZOlB7DIyRT\nFZHZ07fIqdyF5Dv/veKf3kV8Z1R46VSpNzJkkDs2NhKcJo2Ya3qKJLMSKZ5e5ERm8XtYQzYsbBC5\n7/ifNkuBxiK1O1nk4JL9MjtLYwkx2Yg5mYNIQEBCfyVKRYMugqDeCffvc61ea6RMGfyy5OHy9uvU\nPfElt8OtKTgH8qyZyr0+JaiwfioutwNI2X13tMPDihSJfH2uWjWudOlCqh2bSVYhI1Zeu6BPASzr\nTBwO9iSg/kVqtl4NwJBvTzB+YXUGtLJQPct21nYcSUvf7dg2boDV4UPg7ByvX4NSb0v7tFX8CAoi\nbNxEwr4fz3bbYuRYtpMjd0swYyC0GwkL0kOvwG2U3b+Fx2VLkbbb7mfPUb06dlu2ABCxfTt5y39A\n3qVjoVUtCLRF1rhyL2cO7gyzJ98faziYuxQpUvrTouDf2D8uzMoRNzjUth+tfTZh/2ElnMZ7QaFC\n8fs9KJUIJPRvGSohRUSIzJ0rIW7p5C+nYvJVw5ly9GSolG0m8mEnkVaXRLJfDJWZYX/KrfKZn+0C\nqV5J5KOPon0WevGiyLGtIp/mEHF3EqnlJuEXcsvxqTUk0M1FVrfrICPHzBU/l7RSrfgxWf5HgGz+\nuKfcsXKRK0U8xXz0WEJ/K0q9Ei/oHtGWtoobIrB+PZb+A7j+xJpu5iZ8vrAruS35+KgztP0S1hSB\nbC53GBA6hwZNfsd9z38elkmdGnp9BXXrGu9TpIAzx7BbMAjWb4F/ApAB1twuVhqbT46Q3PU+NX/e\nRYuL1+g8qBPTmq5mZur1uH46jsspcmBZ8xfZPq4U/9+FUrFIQ1vFvkOHoH9/Qr1vM4BqXCzwAeMn\nNuCbGU5cvQ0t5sECLAxNs4P8s76n0oQdWAcbIztIaw9WTtB3ENy5ExXYXbtClezQowT4ucH1x4RO\ny8WtdSbcZu1j7IAJHKjRltpD19JsWVseVW9Er9X12R+ahrBBM6kz9LPEvbCuUv/SIX8q9ly+DIMH\nI7t2sb1yW5pvdmHE6OpkylucDkNMNGgKJ2uCg20AvU8Op2KfBThnF6yqp4R+lyCFDRQsDhmzw6hR\nkDOncd4ShaF4BNikgJ3nkHSB3C1cCNcFZ9jTviWdmk+k7hNnnnRZxdwzjQHY4VSAv0o254slPUmf\n3iUBvxSl3o4+XKPizoMHRsguWoR/+660zFuNu5fhUJ8QfjtqYdRvJjpOhtlOMCRgKzWG9iLzSR/s\nh1kwnSwI/f696VijAfg+hsaNowK7an7YfQoKlUZW7SeiYBoCrzoS6GRNixWHcM2Un/wzwqm7tAHV\n7v5JYDIXmji2pd3szoz7JL+2rlWSo0P+1NsLDoaxY431AM1m1o1fTa4FrpSsnItZv7Rn+eJA+v5Y\nhvnpOuMVeI1Ni1rSoWo9MhVJTrIVdpjmZIJFRyF/JuN8l6+Bhwc0aWK8z2gHLauAozXM34852JoA\nH5g3ZgrVJuyncEAuMtecy19T7Kh2909GpmpAl08W8OuFsTRpUkADW6m3lNA3YVVsi4gQmTdPJGNG\nkSZNJODoKWnT5g/JkWOq7NlzQybNF0nzgciwdWbZ+mGdaCM/Io6VEVlXXCRtapF8jiKLvje2JU8e\nfdSIg53IyibRPts3oJVk+idQ2p0Kl1EV5skNp+yR2+q795K//jqf0N+MUrGG11yDV0NbvdzGjSIF\nC4pkzy6ycqUc3PiP5M86Xj5vv0ouXwuVGp+LlGkmMmfBFnni4Cg+pQqJgASWzi6WknZRIVwhuciR\nzSJly0YP6xwpRSrmE8lnG+3z1odOSolz4bKg03y5bJ9DbuTzlDOfdJH7Vs4yvuF48fcPSehvRqlY\n9aLQ1huR6vWJQJUq4OODhITw5IE/EYHBuNgK1uGhRGCN2S4ZyUKfXfX8uezsICws6r1bCsgMHHkU\n+dHxz+oysXw7emxaTZm1Cwm1TobPlyP4facPHQ7/xI2p8ynavXHs1lOpd4AugqBizdWrfrRosQoX\nF3t+/Kk+U5a48McWoc/oMDKN6cgnaxZF7mveXRfr/Yeh3x3jg+H94eYlmLMq+kmLpYNjPtE+2lTn\nE2r8teKZ65+xTkeqVA6kXDIP+2qesV09pd4JLwptvRGpXpuIsHDhCUqX/pnGjfMxeUYLGvRxwfs+\nfD7wNN0rufDJmkXsGFGNK+uGI5VcsD54B8aGwW+/QZk8MH4crPZ69uRPBXaPW5fpOmkpRU+d5rJD\nbixP/bntkOcrQg8dJd29KxrY6r2kQ/7Ua3n8OIRu3dZz/LgPf//dmsOX0lK5DXzd7RHp/x7GZx9N\nBeCP9Z2pXLU2qaa2gp0B4BsCu/dAqlSwahkcOA9PfF94nR+6DORrz3qYHoQREGZDzmBj+tXvHGqQ\nbMQQfuxTThcnUCqOJWhnvoq5PXu8JUuWydKly1q55RMmn34hUrCeyLRDj+SOm4f4FCooZiuT7Fg/\nSMICp4vsSBV1E9HPT2TCBJE0aUTc3F46zer/f446lXzqdU5pXGm6XLr0MKG/BqXiFXojUr0p8/Ll\nrF9ymCG7bBkxrxWpMuSh5QCoWQUsrWGvbxCnijgRnCIZNxaMIFclb0y/roARoWBtB5kygZ8f3L0L\nT17z5uRTmrl2oPYPPWnVqrCOuVbvHb0Rqd7IjRuPuVrsQzxCH5Ld9gkP7dOxyqoGmfp8xIjalcni\nakvn+7OpWrwnAQM74dLiLPQ7B7fdod8AaN36ja/pnzY7ye9e4efkldlb43PGTm+Au7tTHNROqXef\nhrZ6batXn6Nz57/YkHU3WepXp/HpLuT3P0qbPJsQr00UP3+ccBdrnO48Ng4o7QBXgfrN4bo3/Dvn\n9X/5FM9AuqO3nrvtSfmqPDxzmZ72Dek6ryu1auWKo9oplThoaKtXCgoKp2/fTWzadJnFixuRcvY8\n1m03EzhkBD4fw+ZAmJXZmzvHxtG63Iw3OrfZzgbrsIjI9+sLdWWU21D2bksHwCinWgR06sG3I6vh\n5GQXq/VSKjHSIX/qpU6evEupUnN4/DiUffs7s+jvTCw8mI6GpXz4syrcM8O8HEe4fO0HPmu83Jgy\n1c32tc///8A+W6QWOUs+5EadzpGB3SD/CGrt+JnvJ9XWwFbqFXTI33tORJg27SAjR+5k/PjqlKtc\nhJpdTOTMDE0HenDplzu0drWQM9UmLl3eQ+ci0zGFhsLttW98rZGFZnCoeDv22w0jzZhxAMweuYEV\nX3+kw/iUek0xCe0mwDAgL1AKOBobBVLx5969QNq2Xc3Dh8Hs29eBvadTUaElDO0J3h/C+F3pWOJ/\nhzupFxN05Q7tc333VtcJsk/OWfv8DDnZHU52j/zc76shdPmmZmxVR6n3QkyaNyeBhsDOWCqLikcb\nNlykaNHZFCuWjk2b2zF2QSpG/wQr5sGf5eBYCMws7gT3LpP5yD2a5BjwVte54JAXi7UtJfz3P7Mt\n5fiRcPBgTKui1HslJqF9DrgQWwVR8SM0NIIvvthI585/sWRJY1rVKcgHLawJC4cZv0JLE5RKBvMy\n32FZhrW43HzEhyX7vda5w+2i90ffc8hA7uBzOAc9BODHesMJXLUW3N3B3h4++wyKF4/1OiqVlGmf\n9nvk3LkHNG++ksyZXTl+vAsbt5ko2syDb388Q0AjDz67B3PTQ36Xyyz3XUjf1CPe6Py2YWE8TJmV\n+U6t6GD+Hfc7xr/p+5LnJ1/RDHTePgFu5oJ796BZM1i4EGz0j6BSb+JVQ/62AOme8/kg4P93orYD\nfXlxn7YO+UtgIsL8+cfp3/9vRo78kNZtStB9pInkG5cz9dindLkUxC4cWJkRwmyOkytNORwehUQ/\nSdasyO3bmJ6eSvU//K2Tg709yYPuR34W6uCMbeMGWH38MVSqBB07gosLLF6sga3US7ztGpHVY+Pi\nw4YNi3zt6emJp6dnbJxWvYb/T/R07NgdvLzakMzFnfItIH8OGJ32VwKdXXhg58ABRz9MlSrhdPBU\n1MFZs0KnTsiUyZiuXYv2p8dsZYW1xRLtWsnN/hBkvN6TuTx5Zo8iTY1KYGUFoaHQqBE4O2tgK/Uc\nXl5eeHl5xcu1tgMlXrI9YWZbUXL48C3Jnn2qdOmyVgIDw2TlZhG38iLTFokcvHZPwmxsREAsefJE\nn7Rp8WKR3bvFnCtHtM8fN/9UIqytXzrZ06TUdWTLpovRCxISIvLxxyKNG4uEhSXMl6FUIkMcLDfW\nELgBBAM+wAYN7XeD2WyRSZP2Spo042T58tMSHi7Sf4JIlqoih06KLHholq0VajwTuNfWzxbz+bMS\nUqZYtM99Vy2W0w2aPjekH/X8WnxTZ5Q1dgVlUr8VEhISHr0woaEideuKNGyoga3UG4iL0H5dCV33\n94qvb5DUqbNEypSZI5cv+4rPfZHKrUU+6ihy/+pjWf/tuGeCN6Rja5lwqbvcqlU0+uc/TJS1Ry9K\nUDKHZ4653HaIRCRzkDs2KWR40d5y8eJzpk4NDRWpX1+kQQPjtVLqtb0otPUxtCRkzx5vihX7kZw5\nU7JrVzvuB6Sk5KfQIMsl1oW3I002V2qN6B/9IA8P7IIi6JtzBuk3HDc+a9OG3Xf8+eOoN3WK58Ih\nJDhy94Uj/uHu3SDu797JYqui7Pt5A0OOTiZnzlTRzxseDk2bGhH/++/GepBKqUQhof/BSvIsFouM\nG7db0qYdL2vWnBMZNEgeZioon+X7S3zzRC0ocLJMJQmdOOnFfdIFC8rVnftlxujZz2xb+uE4uXnH\nLAsWHJe0acdLt25/yaNHwc8vUFiYSKNGRreItrCVeivoIghJ0OjRRCxewrVbQQSZrchdMB12To7I\ntm1YiznarufKVCLvgRc/vGrOmBHrmzef+XxZtq7kG9IC5+QhjBp3gIcBEQz5sTklKuZ8/onCw42H\nZkJCYOVK4yEapdQbe9shf+oddvvqA/Zdd8C7fmd6dCqKv5+ZBUP30N6045nQfllgA88E9tVkObDP\nlZXGLie59U1HHt31Z5a1H3ZhweBlgopDnj1JeDi0aAHBwbBqlQa2UnFA+7QTIRFh1qxD9FkeQOXM\nwheLe3I4eQWOdZjA56eGYj1sCDd9Ayh64gmXPT967fNeLlQbz8LH+HKskPLuJY58N4PcPk0YWeJL\ncjaohJ1bKmNV9W++efbgiAho2dJYVkxb2ErFGe0eSWQCA8Po1OkvTp++x4p5NclZqSBLFz6k10R7\nNlVZSvEjvyJbtmCKiIh+YLVq8Pffz5xPbGx4mLcsPRxGE1iyIpP6g73VY3r12sCFM3dZVfUBeZfP\ngvbtYcgQ42nG/4qIgFatjPUgV6+GZMniqPZKvT90EYQk4MKFh5QrNxdraxN793YgW5Gs3HLNxcrv\nj7JtnlC8fi5wc4se2O3aGYH7VGCv+X4m4SVLATC83GI+zLaT9pMrsmJyBL8v2kmxYj/SIPV9TiWb\nS94Le2HnThg37sWB3bo1+PpqYCuVRCToHdikYuXKM+LmNk5mzjwoFotFngSK1O8u8ne2zyQiY2aR\nwoUjR3ocLFFBQm77iP+o0ZGfba1US0KcXURAwrLnkjm15kqwyV5++t0iEREimzdfkty5p0mbGrMk\noGFTkQwZRH77TcRieXGhIiJEWrQQqVZNJCgo/r4Mpd4D6MM1iVN4uFn69dskWbJMlkOHbomIyPUb\nZvm8/N+yL1+zqGF5xYuLuLvLpNnLxfpEuKys30oE5EaeArJ/+wGxVKokArKu7nhxLx0mE765LObM\nWeTKFV+pX3+p5Mo2SU52HCSSOrXIV1+J+Pu/vGARESKtWolUrSoSGBgP34RS75cXhbaOHnmH3b4d\nQLNmK3B0tOXIkU6kDvbldq9RyJx5jErjgttXHcGxKnz+OWTLBhs2UPP0Jdo2L4G9uxvi6EjG86fJ\n0LQuj6xTcDBVXZYX7MfhH8Htwm3uzHekVKk5TG7kyCqneVhdcYdduyBfvpcXzGw2ulxu3oS//gJH\nx/j5QpRS2qf9rtq58zolS/5EzSqZWd/RgdStmxCevzBblt3k0vhluHsfx/TgPgwebBzQty8MGkS+\nlk1I2bEdjvnyYAoyptyrn30Dtyxu5Br5OfO+E054HWHNx315HCJcrXSUVhu+x2rIEKPf+1WBbbEY\n06t6e8PatRrYSiVBCf1bRqJisVhkwoQ98kGqr+RK4w4i7u4iFSvKgS/nS9ZyT2T3kad2zpBBxMsr\nqoukZUuRLl3EkjKlXGv2pVSvf18mFxovgRlziqRJI6eO3ZRvi34hPjauUcf07y8SEPB6hTObRdq3\nF6lcWeTJk7iovlLqX2if9rsv8J6fTC/VTY4555CINO5GoJ47J4vWiHhUEjl+9j8H1PjPTH1p0sjd\nxp9Lk0+8JXctkSV/iZhDwiK3r7UrGLVv1aoiZ868fuHMZpEOHUQqVdLAVioeaGi/qywWkQMH5FHT\nNvLY2kGOZigpob+viJzG9Nc/RdJVFDl14aljAgJE+vV7Zn6Qzs3PSwZPkZ+WiYSHi4SFRcj0qXuj\n7+fmJrJs2ctHhfyX2SzSsaNIhQqv3ypXSsWIhva75sEDkSlTRAoWlECPTDLKsab8MmqtWJ4K00Vr\nRNJXFjn9/zUFLBaRlStF0qc3AtjBmDJ1bsXJ8tg6ufw4474Ehxi7bt58SRpmH/jspFBvGrpms0in\nTiLly796RIlSKta8KLT1RmR8slhgyxZjUdscOZBDh1hesTM5LT2pvOlH2g6u8/+noPhzK/QdB5vn\nQP6cwJUrUKcONG4Mt2/jV6g8X1bZxNjsYyhlcw6nD8vSKeNebng/pEntX7japBOrroyJuvbOneDm\nZpzndYlA9+5w6hRs2PD8h2uUUklOQv+D9W7w9hbJmlWkaFGR6dMl5M49adlylRQv/qNcveoXbdfb\n90RSlhE5fEqMpbpGjYpaRSZjfulcap0UqW+ReStFQi9cE0mdWkL6DZDtpZrIpy7tJMzW3tjfZDL+\ne/myceLZs0U++MBoPb+KxSLSrZtI2bIijx/H/vehlHopdGrWBBYUZCyUu2sXd1NkoHHjZXh4uLBg\nQQMcHW2j7frVeAgMhpkVtkHVqgBEWNvSLeev3KjYhL7trKhaDkwmMJst3MtfioPeZuqHnIg6yYAB\nUKQIzJgBu3cbn1ksUK4cdO5sjLN+ERHo1QsOHYJNm8DVNZa/DKXUq7xo7pH4kND/YL07vv1WHjRu\nJZkzT5YhQ7aJ2fz8m4ED+t2RU85FIlvXvbP9JF+MDJPzV6Pvt23bFSlSaIZcc/CI6rP+4AORs/8O\nM2nSxGhdP+3IEWMY4YMHzy+jxSLSq5dI6dIijx7FrL5KqbeG3ohMeJsX7xNfk4OsmrX9+TtERIj0\n7RsZwCdajJALZ4Oe6c04c+ae1Ku3VD7y+Cr6TcaJE6NGhWzZIuLqKvLwOWs39uxpjAb5L4tFpHdv\nkVKlRPz8nt2ulIo3GtoJbPr0A5Iu3QTxadhS5Jtvnt1h7dqo8P3kk+f2I9+/Hyi9eq2XjKm/k2s5\nS0Ttv3t31E7h4SKDB4t4eBjB/TyPHhkjUPbujfrMYhH54guRkiU1sJV6B2hoJxCz2SK9e2+QvHmn\ny+XLviIXLoikSRP1gMqVK1HhmyqVyN27z5wjMDBMRo7cIalTfy8Lq/SJ2n/s2Og3Fb29jaF51auL\n+Pi8vGBLlogUKWKEvMUi8uWXxqRTvr6xWHul1Nt6UWjrjcg4FBoaQdu2f3LnTgCrVzcjRYp/55pu\n1AjKlIFff4XTp43Pjh83bhw+JSLCwoIFxxk+fAd1Ctkzc313Y0PRosbETs7OUTuvWWNMHPXFF9C/\nP1i9YjSnCFSvbgwjvHULtm415h5Jlerlxyml4sWLbkRqaMemR4/AwQHs7fHzC6Zevd9Il86ZhQsb\n4ODw7wiR0FAoXx6OHDHeL1hgLCLwFBHhjz/OMWjQVjzSOvJH8HxSHNplbDx1CgoUiNo5NNQYKbJ6\nNSxZAh988PrlPX8eihWDPHmM0NbAVuqdoaEd1376CXr3hogIxNaOuxH2WFxd8SiQBVPKlMawuXXr\n4MGDqGPatjUeXkmZ0vhxdeXv7dcZNGgroaFmFtYIpMj4fsa+U6caw/CedukSNG0KmTPD3LlvF7rb\nthkt/NSp37rqSqnYp6EdVx49MsYzt2wJu3dzzSYN9avOoV2DrPRulQuTnx/88gssWhR1TN26xvjn\nsDAoXhz8/Ii4/xBT4BOemOxJLiGYnv7OwsLANvpYbpYuNUJ86FAj+E0JMpxTKRVHNLTjgo8PeHhE\n/8jaFb8S5cn3YSFIntzotggLg5IljaA+eNA4xmKBfPk433c0X/4Zyr1/LjGtUiBll4yLOlmbNjBx\nYvRWcFCQEdY7d8LvvxvdG0qpJEcfrokLTZpEGyd90db92QmamjcX2bzZGDFy+HDkoYcO3pS1mauJ\ngNzJkj/6MYMGPf96p06J5M9vrMuokzcplaSho0di2T//RI72iChfkYoBTaj9SWGGDKkMT54YLerd\nu2HlSmNfgJYtuX/7Md7/XCOd3w1cnW1xfnw/6pyFChnHJE8e/Voi8PPPMGiQsSp627baHaJUEhcX\n3SPjgTpAGHAZaAc8fs5+SS+0RSKH1EnZctS1a0O2wpn44YdakbP0AcbIjgIFkKAggi3WON69Gf08\nmTLBjRvG6337oGzZZ6/l72/MFXL6tNEd8qrlwJRSScKLQjsmU7NuBgoARYALwMAYnCtxadjQ+G+B\nAgwq+iVmR2emTKlpfMkicOIEjBoFyZLB5ctsfpKO0fbVWDZjOxHhZiPMP//cCOxu3YwpT7NkefY6\nhw8bNypTpIADBzSwlVKxpiGw6AXbErZjKLb9+Wdk3/P5veclXboJ4nvbV2T9epFu3cSSKZMEZ8gS\nuU/LOvNk27YrUYsb7N4tUqCASO3aIlevGp917y7y9ddR17BYRCZPjlplRin13iGO+7TXAkuBJS8I\n7Vi6TAI7ccJ4GhHg+HEOzlpD8LI/qBB+iYCsuTniUZJJF1NjEx7GIv9fCN6+G/cSeY39/fzg66+N\nFcynTIEmTaL6pa9cgdKl4epVY6RJu3bGyJTffoPs2ROmrkqpBPWi7hGbVxy3BUj3nM8HYQQ1wGCM\nfu3nBTYAw4YNi3zt6emJp6fnKy77DvLxgVq1ot57epK7dEV6+mXlm7yfYZXKjcJ53BkyojBlHp3C\nNOkELiXyGu3tZcuMx8vr14czZ4zujqdlzw5VqkCfPsbKNk2bwooVYGcXv3VUSiUYLy8vvLy8Xrlf\nTFvabYHPgapAyAv2Sfwt7eBg8PSEmzehdm0jVCtVenGoHjli9FmvXGn0Wd+4YTwx+bJHzI8cMfrK\nZ882rqGUeq/FxeiRmsBEoDLw4CX7Je7QFoHPPjMeZtmxA3LlevUx168bNw0dHaFvX+NHW81KqTcQ\nF6F9EbADfP99vw/o9pz9EndoDxtmLNm1Ywfkz/96x4SHG5M4de8OOXLEafGUUkmTPsb+NpYuNYJ3\n+/Znpk1VSqm4FBfjtJO+1ath40YNbKXUO0Nb2kop9Q7SlrZSSiUBGtpKKZWIaGgrpVQioqGtlFKJ\niIa2UkolIhraSimViGhoK6VUIqKhrZRSiYiGtlJKJSIa2koplYhoaCulVCKioa2UUomIhrZSSiUi\nGtpKKZWIaGgrpVQioqGtlFKJiIa2UkolIhraSimViGhoK6VUIqKhrZRSiYiGtlJKJSIa2koplYho\naCulVCKioa2UUomIhrZSSiUiGtpKKZWIxCS0RwIngOPAViBTrJRIKaXUC8UktMcBRYCiwGpgaKyU\nKJHx8vJK6CLEuaReR61f4pbU6/dfMQntgKdeOwMPYliWROl9+AOT1Ouo9Uvcknr9/ssmhsd/B7QC\ngoCyMS+OUkqpl3lVS3sLcPI5P3X/3T4YyAzMBybHTRGVUkr9nymWzpMZWA8UfM62S0COWLqOUkq9\nL05g3DOMJibdI7mAi/++rg8ce8F+OWNwDaWUUrFkBUZXyXFgJeCesMVRSimllFLqPZTUH8QZD5zF\nqOMqwDVhixPrmgCnATNQPIHLEptqAucwuvkGJHBZ4sI84C7Gb8RJUSZgO8afzVNAr4QtTtLi8tTr\nnsDPCVWQOFKdqJE4Y//9SUryArkx/oIkldC2xrhJnhWwxWhQ5EvIAsWBikAxkm5opyPqRp0zcJ6k\n9//wGfE190hSfxBnC2D59/UBIGMCliUunAMuJHQhYllpjNC+BoQDv2HcUE9KdgF+CV2IOOSD8Y8t\nwBOM33bTJ1xx4kdMH655E+/LgzjtgaUJXQj1ShmAG0+9vwmUSaCyqJjLivFbxYEELkeci83Q3oLx\n68p/DQLWYjyIMxj4GuNBnHaxeO348Kr6gVG/MGBJfBUqFr1O/ZISSegCqFjjjDGarTdGiztJi83Q\nrv6a+y3BeBAnsXlV/doCtYGqcV+UOPG6//+SiltEvyGeCaO1rRIXW4whx4swJq5TsSTXU697Ar8m\nVEHiSE2MO9hpErogcWw7UCKhCxFLbIDLGL9W25E0b0SCUb+keiPSBCxEp9CIE0n9QZz/tW+vNghF\nQRRFd2EUAJJyaIQWKIEywIPB0ALBIa5AYt7LDclaFZxkMmY+t+rR+Aq9VMe5cRa3b8x/X43lz3lu\nnMXsGhcH9+owOcsaTtWzejfq928jyV82jQOAa9/e205NBAAAAAAAAAAAAAAAAKznAxBZ/zVDteER\nAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sampling = MetropolisHastings(log_p=p2d.logpdf, \\\n", " q=Proposer(lambda x: multivariate_normal(mean=x, cov=np.sqrt(0.1)*np.eye(2)).rvs(),\n", " lambda x,y: 0), # Proposal is symmetric\n", " x=[1.5,0])\n", "samples = [sampling.sample() for _ in range(1000)]\n", "plt.plot(map(lambda x: x[0], samples), map(lambda x: x[1], samples), 'r-')\n", "plt.contour(X, Y, p2d.pdf(XY))\n", "print sampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This example also illustrates the importance of choosing a suitable proposal distribution $q(x'|x)$. If $q$ is tightly concentrated almost all proposals are accepted, but the chain moves very slowly. Contrary, if $q$ is very broad the acceptance rate is high since most proposals fall into regions of low probability and the chain moves again slowly. In both cases, the mixing time of the chain is very long and we would need many samples to estimate reliable averages." ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEKCAYAAADzQPVvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXfcFNW9/99PpSolKEiJKIqKBUHFRvSJEUXNtQWvGv1Z\nrrFFbqyJ9epjii2WaCxBry2J9WpijILdB7EEMRQLIKCigqIIonSesr8/vnOcs7NTd2d2Z3fP+/Xa\n18zOnpk9u2fmM9/5nu/5HjAYDAaDwWAwGAwGg8FgMBgMBoPBYDAYDAaDwWAwGAwGg8FgMFQAg4EO\noLbA45wEtAPfAtuE3OclYC0wpcDvNkRnMKbdK4HBmHZMFYU2RFRGA68DK4BlwKvArkX43teAjYH3\nQ5bfDzgjobr8FPgYWAX8HejlU3Yw8DKwGpgD/CjCsf4T+a9XW8coJabdo7X7b4B3gFbgioTqkw/V\n3o79gCeBxciN7PuOzzsB9wDfAJ8D5wYcL8o5UTDFFPuNgaeAm5EfNQC4ElhfhO+uKdI+QWwP/Ak4\nDugLrAFu9yn/EPBvoDdwKfAY0CfksZYBNwLXxFf9vDDtHr3d5wO/BJ4GMgnUJx9MO4rATwR+4vF5\nMzAEuQn8EPgVcKBH2ajnRFmxK/C1z+e1wPXAUuAD4CyyHwN7AHcDnwGLEOunFrmbrkD+PMUmyJ/X\nB3kMdD7OdQHuA5YD7yEX1qeOMm77FcpVwF+191siF0s3l7JDgXWOzyYDp0c81s8orWVv2j1au+v8\nhfRY9qYdbepxt+wXA/tr769EDDY38j0n8qaYlv37iO/tPmAsuY8spwGHADsjJ9Y4sq2a+4ANyJ1z\nBHAAImTrgceBY7Wy/wm0AF951OUKYAvkDz4QOJFoFtT3kRPf63WMx37DgFna+w+t+g91Kbu99flq\nbdss7Iti+wjHKiWm3aO1e1ox7ehPL2Azstv5bbJvYjqVcE74si1wL3IXbgX+AWxqffYScsIoxmBb\nBn0RK7ez9vmx1j4gvuwF2mevAcdb6yeRe4f/ADnZFKdSHMvgBbJ/I4iVs49L2f8HvOHY9jvEJxjl\nWKW27MG0e5R210mTZQ+mHRVulv0ga1ujtm0M8JHHMfI9J/KmPqkDezAXONla3wZ5jPkD0lGxGdkN\n9om2vjnQgHR6KGq1Mi1AV2AU8CUwHOnw8KK/z3clySrkcVanB7DSo+zGPmWjHKvUmHYvn7byo9rb\n0Y9V1nJj7CcSvzYu+jlR7GgcnfeB+4EdrPefk32n1Nc/RR5xvoc8LvVC/pgdrc/bgUcRa+FY4J9k\nuz+c+H1XGL6PNIrX61iP/d5DTmTFEMQSmOdRdkugu7ZtuLU9yrHS0sGnMO3u3+5O0tZ+impsRz++\ntuq1s7ZtOPCuR/lCzonUsw1wHtKLD/LY8xowwXp/BvIHDEBOhhfJ7uB5ArEiNrK2DSH7kWcU8me/\nA/yHtv0kch/nrkGsiZ7AQMS35rQOTnTZr1CGIWFZo5GOmAetlxdvAL9HHn+PRE6o74U8Vq213xlI\nx24nxLoqNqbdo7d7PdJ2DyIdmZ0prWEGph0VnREDrAPxr+uuqau1em2H/J4DcCfqOVFW9AceQfxS\nq6zlHdiWax0SKvgV4pP7OXLHVyfLxkho0qdI7/10pCNHZ761v+6eOgn33vz7EfF8F7iA4vn8jiU7\ntran9tkd1kuxOeJvX4PE2e8X4VgnISek/rqH4mPaXYjS7veR23YnJFCnKJh2FFR7tGtLRSMScfQN\nsAQ4x7HvSmBv7b3fOVES7gG+QO64XjQBM5A/viX5KkXieOSRcDneI/CayD5ZnkdG7D2faM1KyyDk\nRvIe0m6/cCnThJy4M6zXZcWqXAxUa7t3BqYCM4HZiLXpxi2IuM5ComPSSrW2Y0n4AXIyeIl9T0Qw\nBlrv+3iUSzNN5FoGlU4/bP9id8QHu52jTBMyYrBSaaIy272rtawH/oW4CnQORgYHAexulSlnmqjM\ndoyVMH7AKfgPpvgpEie7yHrvFRubdtLaEZYUSxDrD+Qxcg7yqO4kiZGIaaIS232NtWxE3CvLHZ8f\nirhBQJ4CeiLhkeVMJbZjrMTR6bM1Mpz/ZeAtJD683Ggheo9+JTEYeXqb6tieAfZCHvUnIp1KlUQL\nldnutciN/Avkupzt+HwA2ZbwIuwn83Kkhcpsx5IwGG83zq1IcqQuSKTIPOQGYCgPuiM36cNdPtsI\n2yVwEBUSFlZF9EBcNE2O7f8ku6PwBWBkkepkKBFxWPafAs8hKUWXAa+QHT8KwJAhQzKIpWhepX3p\nIxUbEBfcX5HQOCcrsV0Ck6zyvfUCpl1T9dLbFqRz/WlyM1MuRjroFQOtbd9h2jVVL2e7JspgvC37\nbRHLoA6xAt/B/XE/EydXXHFFKo+V9uMhJw+IL/7PwE0+7d4X22c/ClgYpl07OjKZp57KZL76Knr9\n0vzfpblumcx3bdsHO4SvC2J8OVNj6x20e+DeQRtr3dL836W5bplM1jVbEGHSJTwE7IucRJ8iuTrU\n4JwJyBDqZ5CBDR3AXeT6CA3pY28krO1tJKwS4BJs3+cEJJnVmUAbYuGHShC1dCn8+Mdw++1w5plw\n220wZgwMrZgUT6lmM6TztdZ6/QUZ4KSypU5AhP5gxGJcjZ0CwVDBhBH7MEOHr7dehvLhVYLdeLdZ\nr0hs2CDLtjZZjh8PF18MV13lvc/69XZ5Q0G8g7v/fYLj/fgi1MWQIko9BDtvmpqaUnmscjhe0rS2\nyrJdG1tYG3CmnXSSWP9p/u/SXLe0k+b/Ls11i5NixlBb7idDKampqYF42z2nXefPF5fN9dfD+edD\nTQ38z//Ar3/tfZCGBrHszSmSPzG3rbleU0Jc7Vq2lr0hvXzxhSx1y76uzn+ftjaoL3bCbYOhijBi\nb4idH/xAllHcOGCseoMhSYzYGxIjqth3dCRXF4Oh2jFib0gMY9kbDOnBiL0hMaL47MPcDAwGQ/6Y\nS8yQGO3ttuAHWe1xiP3IkXBPKaZnMRjKACP2ReDrr+2BRtVEezu88Ya97lVmypR4xH7GDHjllcKP\nYzBUIkbsi0Dv3nCOc4KyCmbcOFm2t8Onn9rrbvzjH7DPPvG5cYzf32Bwx4h9kfjoo1LXoHgMGgSb\nbSbRNW6jaXVmzZJld2sm0wVafr+vvoJ3/CbDdMGIvcHgjhH7IlFNItTeDo2N8PHHMHGivc2NTp1k\nOWSILJcutT875RTYaado311N/7PBEAUzZrFIVJMIKbF//HF7m1eSsxprEPjUqTKCVo+1X7s2+ndX\n0/9sMETBiH2RaGwsdQ2KR0cHrF6dvc3Lstd99Q0N2eXycX0ZsY+Pb7/NLxNpY6PtljOkByP2RaJL\nl1LXoHi0t8Nnn2Vv+/JL97K62Dc2Zov9gjzm5zGjcOOjd2/YeOPo+61eLTcK5aIzpAMj9kWic+dS\n16B4tLfDeefBjTfa22bMcC/rJ/b5YMQ+PrbYQjKYRqVbN3kiMGKfLkwHbZFIoWU/CHgZeA94F/iF\nR7lbgPnALGBEmAN3dMD222dv69bNvWyNlrhVuXEmTMh/IhPjxgnVrk3I/LQzrNdlbgeqKSCprmmH\n9GHEvkik0GffCpwLbI/MQ3oWsJ2jzMHAVsDWwGnAHWEO3N4uFvs112Rvc8Pps+/ogDPOgNnWxJYD\nB4b5RhsjMqHaFWAycvMeAfzW7UCFiL0hfRixTxglWinM1b4EmGmtrwLmAP0dZQ5F5jMFmIpMZN03\n6MDt7ZIL58IL4ZNP4Mkn4c037Tz3Orqg6OkVVCTOokUhf42Fl9i/8UaweC1bBpMnR/u+FBKmXSHE\nZBj5in1NjbnpphEj9gnT3CzLlFtJgxELb6pj+wBkknnFIiDQ1u7osBOfDRpkW+cq5t5ZVrFunS32\njz5qb1+1KugbbfRwT5299gre93//F1I6o1y+DMa9XTPAXohrbiIwzG3nlJ+zhoikz96sUArteEyQ\n7sBjwNmIJejEecnn2GzN6o6GzL/Z3t6U5Z5Rv91NPPT/ZehQ+/0jj8CJJ0o6hWLlFSqGNTppkkzu\nEkdoYktLCy0tLV4f+7XrdMS3vwY4CHgCGOo8wFdfNX9nrDQ1NYWeW9VY9oUR0K5lQSYfVqzIZN54\nI69dU8G4cZkMZDJnnVXqmghki3UD8CzglbnnT8Ax2vu55Lpxcr7jJz/JZB591H4/ebL8B3femVuf\na6+Vz8DeT73/5S8zmb59M5nPPgv32xoaZD/33y2v9nbv/a+/Xsp0dIT7vnyATOaaa5I69ndtG9Su\nTj4Cejvbddiw/OrRvXsm88038f62agYXAysfUu/Gufxy2HPPUteicPKNLkmQGuBuYDbwB48yTwIn\nWOt7ACsAF897NrobB2wrT+XJ0Wlvt8NS6+pkX/VU0Ngo4Xvr1wd9oxAm1M+tDnpdIPz35cu6dYke\nPky79sV+YhtlrS/POVABPntD+ki9G2fNmlLXoDB69JBlCt04ewPHA28j4XcAlwDft9YnIP7cg4EF\nwGrg5DAHVtE4in32kZeb0OriXlsr+yo/fpcu0cS+oSH3mE42bPC+Kagb8jffJDsuIuGbSZh2HQec\nCbQhrpxjcMGEXlYWqRf7FFrEkdhlF7j77lSK/auEe7IbH/XAKhpHUVMDu+7qbdmrsnV12Z2xnTuL\nMIfx2Xd0yLwBAC+9BPvv717uoYfgtNPcP1P1e+wxuTntuGPw94bl9NMliyckbtmHadfbrJcvxrKv\nLFLvxvF77C4H2tpgo43K/6YVhvXrpb2cbhwQqzvIsq+rkzQLakxCfb0MvQ8zilM9Af7nf8Jf/yqC\no3/fJpvI8vTTs/ebPFnCQ8EuP3589GybQdx5J/ztb7K+eHG8x04KY9lXFqkV+y+/hBUryl8k29vF\nOi333xGGHXYQoZ40KdeN4iX2usunrk7KDBgg72trRfxvuCH4u9vaxGXWty/cb40M0EM9N9rIfb+m\nJjjhBPsYxeDbb4vzPYViLPvKoqhi/9pr4csOGAAHHlgZln2XLuX/O8KgJy5zWvb19e5iqj8F1NaK\nG0f34d91l+RoCaKtTY6jhzSO1xxQ6v/fddfcfZcvzy4TN8rKPfXUZI6fFMayryzCiP09SARG0JxB\nuyEdPkd6FZg0KXzF2tpgyZLyF8n2dskLk7CfNnWEdePoln1bm6Q11sW+rk7KZDIymbhXjvu2Nrmh\n6Ba83rnf1gY33QSDB+fuq3z9SZ1rqs+ha1dZlkuyNmOhVxZhxP5eYGxAmTrgWuAZfIZhjxwZvmIg\nVt4LL8j65ZdH2zcttLWJtVltYh/WjdPRAd+34kS23dYWeHUMJfYbNsjMVXPmuH9fe7uI/cknQ38r\nOYCeU19lYXTrKFf1SsqNc5vVFaoifMrF6jXpEiqLMGI/Bfg6oMx/I6P1lvoV8sp86MXy5bYlp/tf\nL7hAIi7KgfZ2ceMUaxRoWvCy7D/6KNtn3d4ORx8t4Y4NDXJTdLPs1f/nFdWkLPt+/eB3v5Nteoij\nEns3q3rZMlkmZdmr/D5K7I1lbygFcfjsBwCHYWdE9LynFxJ+qJ94N9wAf/yjrD/5JDz1VP7HTRrV\nQZvC0MtEccapK7Hfcku45BJ7u/LZb7yxdO56ib0Sbi/rW4k9wEknZX92//3iqunSxb0d1DGTEvub\nbpJltbhxjGWfTuKIs/8DcBEi8jX4uHEKeUx2nnjqgjnsMFmm9eRSFuXKlaWuSXFxdoTqbhwVbw7Z\nPns/sVftHGTZuzFrFpx1ltxQ/G66ra1Sh6SewlRCuHIRe6+BaYbyJA6x3wV42FrvgyRWakWG2mfx\nwAPNTJ8u61ESK0Gu2OvJtdIq9ODvKy4GpUqq5GwvXez1qe70aBzlxnH67Fetgtdfl21hLHsn7e2w\n9dZ2OgYv2trkiUSJfSYTrytjmJVbslzE3oReVhZxiP2W2vq9wD9xEXqAceOaOeqo4AMudfH8u4n9\njBkiEGn2h69bJx20pRJ75031yiuvTPw73SZqaWiw23Wzzeztbpa9Cp9UYv+11mPkJ/bOfgIQwW5t\nlRuBSsfgRWuriL3qUzj66OxUy4Wyww6yVGkd0o4JvawswjyoPQS8DmyD5Db/L+B06xWJsILn5jt1\nc+OMHJluoQfxNas5OasFt8f/hgY7OsbPsl+/PtuNU1srg+sUXueQisZxcuONcMcd8lmQZa/EXvHq\nq95lo3L22VKHu++2R/OmHWPZVxZhLPtjIxzPN1FWGMG76y53y96573PPRahVCVm7FjbdtLo6aL3E\nXnWy6oLrtOzXrs312X/zjV0+qhtnpjVnUxjL3jlJdlyzi3XvDr/+taxvvHH53PiNZV9ZFLULJozg\nnXYaXHqp/f6BB2Q5b14ydUqaFSvge98zYl9fbz+F6WKvW/bKjeP02a9bB7vtJtt+/GP373Rz49TV\n2YKtLPswbhxFXO4W3fdfV1f5Ym8s+3RSVLGPepK/8w789KeyPmpU/PUpBitXSs6WcrnA48DLsncT\ne92yd3PjKLEPEt62tuwyL70Eo0fnin0UN05cYt/RYQtgfX353PiNZV9ZpM6yV51YYIeq3XILbLNN\nMnVKmg0bxGdfLhd4HLiJfV0dvPuurAdZ9rW1kl54113tz4JcKqoTVq9DJmN/l5sbRxekmhp45ZVk\n3DiZjP2f1NeXTwoQY9lXFkXNZx/GutWtqZ49ZemVRKscWL/eezBPpeJ2seu54Z1ir8qrKJ7aWnj7\nbVn/8ktZhrHsdXGuqZFj62LvtOzdrE9d7OMSLd2NYyx7Q6lIhRvn7bdh2jRZd7uovfKq6OijMtPC\nihXw1lsycjKFF3hQgrsm4BtktqMZwGVBB9RTEzvRU2V4Wfaq7XXfu/MzNxYuhLFj4fnn7W3Kstdz\n49fWynmmEvK5uXT0SJm44uGdYp+w4TIIeBl4D3gX+IVHuVuA+cAsYIRbAWOhVxapcOOMHm375L06\n94IukKuvLqxuSfAHawbQrl2961/CkbVhEtxNRoRgBPDboAOOsCTDqw0V+nngnHPWub/uxnGLowf4\n/PPcbcqy128gan81sM/N+tRH/qqbV6EUuYO2FTgX2B6ZN/gsYDtHmYOBrYCtgdOwU51kYdIlVBap\nsOx1wVMX5D772NvCWPZppEsXWXpZ9tOmZcecF5kwCe4iXe7KEg4Se6dl7yf2vXvL8qWXvCf9cLPA\nnT771lb7/OvWTdxDalSuztChsvzZz+ILCiiyG2cJYAWcsgqYA/R3lDkUsKZ4YSrQE5mEPAtj2VcW\nqbDs9flCldirzIUQ/tE3bcPQ1chPL5+9mjQjpWSAvZDH/InAsKAd1G+MKvZKVJQVru+/+eayfc0a\n++b5yiuOimZkrl+VXRJsy761VeLcR4yQlAmqnn37yixVnTrBD35g76duLo2NZevG0RmMPJVNdWwf\ngAyQVCwCBjp3Nh20lUVRO2hvvBEuuih725Qpds56sC/0gdqpF9ay18P40sC118qysdH9AvdyS6SE\n6Yj/dw2S7+gJYKhbwebmZkCNhWiitrYpp4zeLs5OUmebOd+rmb6UiOy7rwy0Uk9FHR3y9KS7XZRl\n39oq0Vxq0pILLsi+8dbXy3yzU6bI+169ZNmtW+GuiIcekjEjSYl9QN6j7kja8bMRC9+JU5Jzfu1H\nHzVjNW3kXFbGjZM/SeWzKqrYu42MffHF7PdKAHUhDHuB6LHWn31mT2JRagYNcrfs4wrtSwi9N2ES\ncDvQG8h5HlFiP2VK9kxTXni5cQYNkqVTKPQImaBjKGprbcte79x1Dqzq1En6Vk44AXbfXcZE3Hmn\nPHUV+uT18svyf9TUJOOz98l71AA8DvwVuUk7WYzcyBUDrW1ZDBlii30UjGVfGEnlsyq5HawezRXq\nRNFPmIaG7AvEuY9ClclkxMpL6nF5wwb45S/Dl/caualuaCm1gvpiW3+jrHVf+Wtvh/POg4cf9ivl\nLdRK1J3tdvHFuf+3lytIobtxgsS+vt4O821okLli1f6FoG7mRfbZ1wB3A7OR9ONuPAlY06yzB7AC\niczKPpAJvawoSi72zkku1AWmX2jOgSj6MHgddRGpffWZiuLkk0/g+uvDl/e6wFU9Z8+Op14RCUpw\nNw4Jy5yJiMYxQQdUeef32su/XJBV7vyvzj0XrrtO1pUA6WLi5QpSbhyn2Ovfr5KzqfNJlV2xQm5a\nhYiWM+5fbUvYZ783cDzwQ+yw2YPIbtuJwIfAAmAC8HO3AxmffWVRckeC8xFdXej6Bd/QIO4eFdLV\n2ur+OKzeq33XrYs+FWIYolp8+qTZ+oWg6luijtqgBHe3Wa/Q6DHzQeX0da+5CtzYZhuYOzdbhN1u\nGF6WvXMU7S67yFKVUcunn5YO32+/FddOoRRR7F8lnBE3PqiAsewri6Ja9m7WuG7Zf/yxfSHqg1uc\n+3nlLneK/fr1ctIddJD4Y+Miitg/8YTts3Xup55WVP72cuaLLySUMYxAOFMWOIXaTwz1iUUuvhh+\n8Qv3G4afZa9/v4rQcVr2ahL0QlwuaupMvW4mEZqhVBQ9XYLTCtNFe8IEubhefTVbAPWLNZPJjboZ\nPVr2URemupjWrZNO4Weeifd3hBX7zp3tWG0lMur3trXJTQjSn5M/DGoCeK9YeJ2ODnGFtbaGc+Po\nfPihLDMZmdt1/Xo4+ODwlr1T7NW8sL16wVFH2WX/7/9kWxzCrItfOaVLKCSyzVj26aPoPvvXXst+\n39ZmP0r37p0tiArdsr/zTvlcP5nOPVcsMTfLPokL67PPZOl3Qs+eLTcb5aZyPr6vXWuvl+OAMSeq\nfyTMRf6vf4k75qij3MU+jMBmMvZ3eUXjhLHsldh37iyzUqnjdO4s7ps4zh+n2Fe6ZW8QJk/OnmWt\n1BRd7J2dphMnyoV1zjm2P9Up9vrFOneuXDD6DeDTT7MvIt1nr4tqXIwZk/09bqgEXspNpUTGWUeo\nDMtetWsYcZw+Xdqma9folr3CKfb5RuP4+ePjEuZqE3uTLkFoaoLbIvV6JUvRxd55Aj32mFj76vH2\n3//2t+zV+2nT4BgtPkT3haqLecOGZMRe4XfRqt+pLPu6OumIbWjIFn2oDMveLVe9H3/4A/Tr5y7U\nylXjh54KwS8aZ9Uqb7Hfbjvx+XsRNNlJWPQggWrw2Rts1qwpdQ1siir2++1nh7o5qa8Xaw9yJ6zW\nL9aaGim79dYwfHj2/rrIq2WpxN6ZwfHbb2UeUsjO06LelztR3DggA6jWrnUXaq9zRCcofLOmBt5/\nH957D/r0sbfroZdbbZUb+qsTl3/9Uy0xQTn57E0HbWVRVLFvbIT5890/q6mRu2BdXfYEJpCbMdEZ\nOTFqVPbjsRKLDRvsG0gS+Il9R0duvPk//mHvp+9bSW4cPW+9H716wbJlkvZAF+rOncPd/ML47AHG\njbNTJajtSmzdRubqxGWFb7SRvV4Nbhwwbpw0UtRonO2ciVY1VIpitwtQt/T1tLVqOWJE9oX5hTUW\ncMOGZKyMY4+VvCd+FppfzLku9nvvXTmWfXMzbLlluPLf/77EsS9enJvlUnWA+xHks1fH/NnPsrfr\nrpkwYh+3FV4NYm8s+3RSVMu+U6dcS3vPPWXSicsvl6gct3AvPea+a1fbcldirzps1UW0bJksN2wQ\nId1xR8lyGBfq0d/vovVLyqbEfsiQ7CiicubGG6MNDhsyRBKUQfb/9D//A+MDh/tki72bK0gJjtNN\nowv4kCH+3xGHy8V581NGye23F3bcYmAs+8qiqGLfuXNuNE4mIx1YW20lflw3gdQt++uus/Pf635x\n/cJUNxQl9l27xms9q+/5+OPcdLt6GT/LXs2ZWk4+XD923RUOPTTaPqr9dFE54wx7MJIfQSNonTny\nFbrY/+pX/t9RqBtnhx1kUJ1bvS64IP/jFgtj2VcWRbfsnWK/YYNc9PX1/m4XPQ2yQomp6rRVF6b6\njg0bZFvcYq86+C64QNLtepXxE/sXX5QOxCRcBaWgoyM4i6ezbVX5fAbvOMXeLfQSvMW+vj54XttC\n28ZrpPebb4pxk3aMZV9ZFF3sr7su25WjYq7r6kSQvS78H/0od5tzJK4+chaSt+zdpsPTy/i5ce6w\nJoJ7/nn47/+Or26lQu8498L5f8Qh9iqm28uyd/rllYD7PXnp9StE7L3+k5TOSZxDvoJtLPt0UlSx\nV5bUggWyVDGo69bZmS2jXPh6WT83jpr8Ii7U96jfoSaw1nFa9rqgn3qqnely8eJwoYZpx8uK1fEa\nLFeo2HuFXoK3Ze92g3CrbyFuHK//pFxi7Y1lX1kUVew320yWKjxPdaQOH267cQoRezc3jhJ7dYHH\nwaOPZr93m1TGadnrlpybS6rcCWMpew2Wy0fslStNib1XNI5T7GtrbaMiSMxaW+GRR6LXTeFl2ZeL\n664SnjgNNkUV+5/8JPv96tUywfMmm8gFEFXsvbIJOi171SeQVIij28Atp/g5L+7NN5eQUUWS4wGK\nQRg3jlNc3TpowxKUz16NWnWmQwhyF+qcemq4zmIvvCz7cumUDwpN9cKkS0gnJZ28pK3NFog43DhO\ny/6CC2yxb2uz89UUA6cbx/nY3tgII0fa71eupKwJ48aZMCF7cu98LHuVSE+f3NzNjdOjhwiOmlNW\noYyKMHn3jz22sPTTa9faidacdSgHN06+Ym9IJ2Eus3uQKcve8fj8OGAW8DbwGrBT2C/XrcE43DhO\nnz3Az38u6WrVeqGEzf3idOM4o5BUBJFKDVyCgVVB7QpwCzAfad8RPuVCWfbHH589yCkfsVejku+9\nV5ZePnsvooh9IVMTZjJyA3e7WSTsxglq1ybgG+xZrC7zOpDTBRYWY9mnkzCXyL3AWJ/PPwT2QUT+\nN8CdYb9ctwbr6mDOHFiyJHi/0aNlqQukbtmvWwdHHmmHRarBPnEkJdKfRvxwunG8xF5Nsp1kDh8P\ngtr1YGArYGvgNOAOv4OFsewhWzwL6aDdYovsY4Z1BUURe5VMLR9UGLGbYCYs9kHtCjAZuXmPAH7r\nVchY9pVFmMtsCuCXlfkNxFIAmIrMVB8KXRDDCOh3FZoiy9697W3q0fi998Rd06NHbmhkHNZGW1tu\nfLbbcZ10hbCKAAAgAElEQVTWpjP/TU2N1Fl1WpdA7IPa9VDgfmt9KtATmYTclTAdtJA7tzBE99kf\ndJB9k/QKvfQiis++EMt+zhzvp7WEffZB7Qr2RPK+FOKzN6SPuH32pyCTGXsycSIccICs64/+YYTC\nydixkkgLRNxXrJBRi48/DhtvDPPmyWf33JP/dzhRlv1lng+/8O674rLws+zVIKRu3cR3n8IO2gHI\nROSKRfjcyMM+8Rx7LPzzn7K+xRbSARo2n45ixQqZ1wDkfz3ttGTcOIVY9tOmZWdlddahhD77DLAX\n4pqbCAzzKliIZW/cOOkjzkRoPwT+C5nd3pXm5mY+/FBEuKWlicWLm+yK5FGTmhoRdYCBA7NTyeod\nY8p61rMf5otKc7Drrt5lpk+XpS4o11+fPT3i+vX25507Jyf2LS0ttLjFhobDaaO5XsLNzc18/TXc\nfDMcdlgTTU1Nngfs0gV+/GNZb2gIlwfHyRtvyCuroiGtSRV6mbTPvq1N8j65EZcbJ8+2nQ4MAtYA\nBwFPAEPdCt52W/N3/2tTk3+76hjL3iaf/6LAa9aTuMR+J+AuxFfo+QjZ3NzM5Mnw0Ucyi4tzBp9C\n6N49O9pGnzCisRHOPz+eZGhr14oA6Glrnbh1PG6/vXQuvv66vFcDyUC2HXhgMoOrnBfplVdeGXbX\nxYgoKAZa23Jobm5mwgTJNaNurMUmKPWBoq5O+m7CWNaFWPZ+TzpxuXHybFs97msScDvQG8hJY3fl\nlc15181Y9kI+/0MB16wvcbhxvg/8DTgeWBBUuKFBZg9yEsbSUmmQ3XCmjnWKfWNjPBEvhx8uriM9\nE6feoJ9/bv8+529SN7ezzxbLXheDNM1oY/EkcIK1vgewAonyyKG9XTrW43CTheH227NDOE87DcJe\nD3V1MhNWWLHP17J/5hlv11KJ3Th9sZ/YRlnrEfKVBmMs+3QSxp5+CNgX6IP4cK8AlB01Abgc6IUd\nrdGKnESu9OsHM2fm3vHCWPZ+d8n6esmPrtDdOI2NcpOJY5KQf/9bll5PCf372+vO2Hl1EdTXZ7tx\nSkRQu05EInIWAKuBk70O9MknstQ7zJNk6FBYuNB+P3x4+O9W//kdvrFFQr4hhJmM9E0deaR3HRLs\noA1q13HAmUAb4so5xuUYBWMs+/QRRuyPDfj8Z9YrFFtu6W4xFSp8dXUwa5b93s2yj9N63nRTOZ7b\noBnFI4/Aww/b73fYAf71L1i6VNxBJRb7oHYFCOVR7+iQdi3UFReWTp2y+2eixIOr/9yv3RTq5pzJ\nRLNWVWSVl+DpUyMmQFC73ma9DFVGSUbQus3WE8bSCUoL+/HH9rruw21okBuMbvnHgZoYI+yF+8c/\nirX/tdWr4ZULv9zwy/CZBM4okXzEXjcG/Mink1a58bzEXr+JVCJmUFU6KYnYu/ks1fuePb33GzfO\n2xUzbVr2e118GhvhT3+CP/85el39qKmRFABh/a/19XKDUILzH/+R/fnXQdHRKcUvd38SFFPs8+mk\nVR3tQTcJI4iGYlISsV+7VqJPdNSFMdBnSFZNjXfUhTPhlV6usdHOsBk3+cwp2qWLLLfdNnv7TqET\nTaSLq68urmXv7C+JIvZDh8Lvfhf+v86nk/bSS2Xpt18lW7+V/NuikqbO6pIlQlMJrRTq4stXNJzR\nGLvvbq83NsZnee60U7YfPh+xb26Wpfqt6kYUt5upWDz7bHEt+7597RsmROsY7t4dLrnEP3RWJx83\nzpw5sqxWsTfYpKmNS5r1EuDVV2WphC/fO6Hz0V6faLqxMTtKphB69ZLOWYXKqAnh8vqAWJdg/1Zd\nuMqRpUuLa9mDDMxTydCGeY4BLZwwbpxMBiZPtsupJw2//QqJ4U87abJmDTYlF/u9PcfbRkOJjfL5\nd+4Mv7VSPDU25o64zBeVMlmh58n/4IPssocf7n8sFb1S7gmniu2zB3H3nXSSCKZ+842bMJb9okUy\nSHDpUnmvygdZ9glG5JScSr2RlTMlF3snhVqIgwaJq6VzZzvCp7FRBFW39hWZTLQT003slWXvtGgO\nOcT7OFddBaOs0QheM1qVE8W27ItFGAtc5T166ikZqa1EfPPNvfdpbZUcTpWIsext0vRflOQSVRaQ\nG4X+Od262fH2f/ubLBsavEctdu9uPwGEobU1u0PQT+z9rN2LL3a/+Xz2Wfi6pIkSjxlIjDAWuGr/\nG26AG2+U8o8/Dr//vf9+L74YTx3TiLHshTT9DyUR+z59cmcQUhQq9ioxGthzaNbWeucjWbPGzlcT\nxIMPyuhfL8veST4uqnJ9tK9my161vxq019EBQ4Zkn4tupEkI4iRN1qzBpmSXaL6z4AShd3aecgq8\n/76sKzHSxVTlxQ+bRuG442TpHLClfPZO0R/qmkvQnVNPlWW5unEq1bIPE3qp2l+lcAg7c1alij1U\n9m+LQppufCUTe6+kZIVaiM79dcHNZODMM2V9zRrYZx//unjhtOy//lrC7QpJbnXnnWINlqvYV6pl\nH8WNo4gyTWIlkiaBKzVpuumlTuz9RtCGIcjCvNOaNFEX1agN4hT76dMl/E+/6FVYYBQSzpmSKJVs\n2QedH85zOWz6iDQJQdxU8m8rV4qUuioX58xNIKlnCxX7sBaVLqpRBdYp9gr9oj/ppGjHBKm7sezT\nRb6WfaXe/AzRSNNTTsnE/vjjc+PL9Umk8yXsRaaL6uuvh59WD3J99gp10asRslGZO9c/UinNVKq4\nRemgVVS7z96MDk4nJRP7u+9O5rhhLUynBb1mTXD0hMIZeqlQOVH22ivccdy49FJ7VHE5UamWfZQO\nWkW1i73BJk1tXHGXaFgL05nbPkonrZsbp6ZGJhqH8HlX3CiyaI4F5gLzgQtdPm8CvgFmWC/PadYr\n1bLX3Ti77Sbht05S1kF7DzKj2Ds+ZW5B2nwWMCLuCqTJdWGwqTixd6YscCOTyZ18vFCx1yfD2GOP\n8MdyUkTRrANuRQR/GDLpxXYu5SYjgjAC8Bx+VsmWvbLO3nrLfUzGCy9kvy+xZX8v0qZeHAxsBWwN\nnIY9w1yspMmiLSVpuvFV3CWqzw3rxViXSyHKlIW6IKv1QjuWFQlMKu/FKGTKwYXIVJIPA4e5lAt1\nulaDZe/FkiXZT3MljsaZAvjNjHAocL+1PhXoicxLGxtpEjiDTUWJ/YoV8MADweWeey53W76Tka9b\nJ8s4Bol17y6x9kViADJHqWKRtU0nA+yFPO5PRJ4AXCnEdZVmnB20bgK9enV25s2w0Tglsn7d2t1n\nFon8MJZ9+qgose/RI9zcom7kG/K4/fZw1132pNuF8JvfyAAt50TlCRHmcpwODAKGA38EnvAqWK4T\nrwShLHuVb8nNyl+1KvtmVwYdtE7bO9aaGMs+nZQsGidtuIn92rUwf76/kNXUiFUXR3x8YyMsXw6P\nPQYnn1z48QJYjAi5YhBi5enot51JwO1Ab2C5Xuiyy5pZtUpCTpuammhqaoq/tiVCReP45U9auVKy\nrSrCiv0A53NURFpaWmiJ7vdztvtAa1sOzVoMcdR2NZZ9/uTZroFUldivW+eeaRLcxfrWW+FXvyre\niatcQYWkXYjAW0gn3WDgM+BopJNWpy/wJWL5jUIswuWOMvzmN80JVrO0qMlp/CbX2bAB9t1XZl/7\n8kv46qtgsb/wwtypNKPiFOArndO1ufMkMB7po9kDWIFE7+TQnOeAEWPZF0ae7RpIVYm93yQhbmIf\ndmStfjNwRvlEochi34Zc9M8ikTl3A3OA063PJwDjgDOtsmuAY4pSsxTR2Chi7ifebW2w//6SzE5N\nZB4k9nV1iY2WfgjYF+iD+OavAFT82ASk7+VgpHN+NZDIM6Sx7NNHVYm9H24Xnn7C+s10pcp16gSz\nZ+dfB3UzKmJkyyTrpTNBW7/NelUtYcW+vl76iwYMgMWLg8Vepdx+/nkYPTrWqSmdT2dujI/t2wxl\nQ0V10ObLrrsGW/YLFnjvr8S+trawi1YJRKGP94b4UGKvbsBuLgo91Ub37rIMumHX10v02AEHiPun\nkjDpEtKJEXtE6Fetyt2un7B9+njvr24KhbpfVNiluVDSQ0ODiL0Sebe20cVelQtj2a9YIetRxngY\nDPlStWK/5572+owZsN9+uWX0C9svpFOJfaE+2JEjYdy4ovnsDSFwWvZubayL/TffyDJoEvn6erts\npbW3sezTSdWKfZjQODX4avZs/7lh1fSDceSi79Yt/wFehvhpbJT2UOLlJsxtbfbN4NtvZRnGjaPE\n/mu/8a4GQ0yEEfuSJ1aKk77WwHAlzCNHepdV0xZuvz389Kfe5Robw6dHDkKF+hnSQW2tiLE6X9xu\nxO3tdvuvXh3uuPX18NJLsp7P3AdpxoReppMwYp+KxEpxsWgRPP20nWL5e9+L57hxiX19vbHs00Rd\nnUTXdHTITd3LstfbP8iFA/GdL2nFuHHSRxixL3lipTipr4eDD4btrPyOTivEzxUzYoTMpuVGXOGS\nxrJPF2oO4yhiH6bDtZLF3lj26SSOU84rsZLrqLy0oZ+YbW0itvvuC0cfnVt2xgzv2bTCPr4HYSz7\ndKF89krs3drGKfY77xx83EoWezCWfRqJ65QLlVipkFwbSXDPPXJhPvusvFdW2+TJpcvP/s038vRw\n/vnxHC+pPBvVggq9zGTCi30YIdfnRKg0jGWfTuIQ+7wSK6UBZ7IxPazu5ZeLWxfFW2/B22/Hd7yk\n8mxUC42NMgajo0N88U6xnzVLzhu3OQ78MJa9odjEYb8+CZxgrfsmVko7hfjK99xTJlEvlLvukhG9\nhnSgLHtd7GfNgttvF0H78ENJd6Bbs2GEXJVRbsG33oq/7gaDThj7IhWJlZLkwQfhH/8IHhS1zz7e\nn/mlwI1CQ0M8Pvv335fIox/9qPBjVTO6z75zZxH+e++Fm2+Wm3tbmx3Oqwgj9sr6nz8fRo2SZaXc\n5M2gqnQSRuwrPrFSbS0sXZprXXXqBDvsAP/+t7z3i7WPi7jE/oQT4M03zUVXKG6W/fr18llbm7x3\n+t+jiH1dHWy7bTwD8gwGP6p2BK1OTY0McDnwwOzt69fbKWtVuaSJS+zjmCbRIP/jJ594i72K4NKJ\n4sYBe4KUSsFY9unEiD3+kTdG7KubbbeFiRNl4hsvy94p7mE6aPUylSb2hnRixB5/ET/qqHDl4sKI\nfboYPVpu+F5i72bZR43GqTSxN6GX6cSIPf6W/VgtUUQ5ib0asu8zmnMsMBfJaXShR5myyXmUJI2N\nIvCNjbBkSbBlH8VnD4mIfVDbNgHfADOs12WxfjvGjZNGKjzaNxx+Iq4mowDo2TP5usRt2c+bJ53M\nDuqAW4H9kTER05AQ2jlaGT3n0e5IzqM9Cq9Z+dHYKJZ9v34y6G3JEtleiM9+t93g8stlPWaxD9O2\nAJORVCexYyz7dGIse/wt+402kuWll8KRRyZfl4YGybL417+KUOfLRx/Zx3NhFBIquxBoRSafPsxR\npqxyHiWJEvuePSXMUqUxnjsXLrooN1VGGLHv1g3U+LaYxT5M20LuqPdYMZZ9+jBiD6xc6f/5RRfB\nWWcVz40DcP/9sM02sHx5fsdRTyQedXbLZzQgRJmB+dWmvGloELGvrZV1NavZYYeJS2fu3OzyUZPi\nxSz2Ydo2A+yFuOcmAsNi+3aMZa+Tpv/CuHGAhQtzt3XrZltsV19dvLoosVdPG/nOfqX8yh4uobB2\nV6icR5WO8tkrsV+2LPtz9V8roqZCiFnsw7TRdCTFyRrgIOAJYGhsNcBY9oo0/Q9G7HHvxCxVoipl\nFRaSiK21Fd54w153wZnPaBBiAfqVCZXzKA0J7uJGuXFqa0XIlRtH4RTqYom9R5K7MG2rP8tOAm4H\negNZz5H5tmuarNlyJKnkhUbsgTVrcreV6oRV36tuQOvWRT+G8vVvvLFnNM5bSMfrYOAz4GhyR0o/\niYyMfpiAnEdpS3AXN21tMG0a7LSTuxHgFOrhw6MdP1+x90hyF6Zt+wJfIk8Bo5AnuByHYSHtmiaL\nttjsv7+kKoH8jLakkhcasQfOOw+uvz57W6nnBVWugcMPt9M1hEVZlj16eFr2bYiQP4tEb9yNRGuc\nbn1eETmP4mL4cHjoIXnqUmJ/3nkyCf20adk5k1at8p+c3o1PPoFbboGzz46lumHadhxwplV2DXBM\nLN9sACRP1quvwh13lC5VuhtG7IE+fUpdg1yU2E+fHn1fJfaffirZGdWE6A4mWS+dCY73ZZ3zKC52\n3z1X7K+7zr0jVh9xHZbFrs6xgghq29usVyLU1MCXXyZ19PSTycjI60GDJOeW1+x2On375nfuRMGI\nPfFNKRgnzk6/fHnwQXjggXiOVa106SLL2tp4+lScVJrLY5ddZKa31aurcyR3JiM3vCFD4L77ZM5r\nP1atkqlS77sv2Xql6CGjdLhduOecU/x66Ohi//nn0fYdr9nj224bT32qGSX2dXV2n0rSfTqjR5f+\nHMyXm2+WdNBxGSzlhhL7444Tqz7odfPNxfmvjGXvoF8/maqw1DMJrVhhr8+bB5ttFm6/3/1OUhsD\n/PrX1XvBxYku9sXitddyo37KCZUauhpRYh91n6Qxlr2DXr1g0qTSu3Z0n2cUl8Fll9kDserr84/T\nN9gMsIYk6TfgOPG60Mt5nlqv+XqrgahiX6yU0EbsLb6wggo//liWpRZ7nbBi7zxh6uoKm2rRIOy5\npyzXrCluSG4+nfNpwVj2pa5FLsaNY6GSnF17rSzHj4cttyxdfXTC3nicF5ex7OOlFB2pu+0moXzl\nZuU3NkYT+/b20ndU19TEY+Sl1bI3Ym+hfPTKP7vZZnDKKaWpywUXZMf9hz0R3n03+72x7OOlFGL0\n3nuwdm35iX2U7K2ffCKRK6UW+9paqUu/foUdp6MjutgXAyP2FspVkpRfNgrOxg8r2G6TaBjLPj46\nOoovSI2N5dmGUSz7pUthxx1L77baYQf46qvCxR7S2UFrxN5BqaNwQG48nToFJjPLQb+4br9dTjhj\n2cdHJiNpiVX66DiPq5ZOkSjXG3ZDQ3A2WcWaNdFHHSdB9+52RtN8UW2ZRjeO6aB1MDTW3H/5UVub\nLdJhxV4vd+aZIhQffADPPRdv/aqVTAYOOABOPz24bNTj6kudcnXFDRkCP/xhuDEia9dWt9gXixTY\nselBzTNaamprxZrbaitYsCA/yx7kKeXFF+VVan9oJZDUPLHquG5tVK6d7I8+KiNpFy0KHiOSFsu+\nZ0/pp9t4Y/9yt94K++7r/lk+15npoC0BaRB6sK2Ck0+WAVL5WPaQrvDRSiCpC/K002RynI6O3DYr\nVzcOwMCBMsFLkJCvXJk913OpuPPO4DxFV1whBpif2Kcp+ZmOEfsU83//J9Z9IZa9IT6SEvuf/1xS\nI7g9OZSz2P/lL+ETosXRKVoovXvLK6iM3xNePjH2xrKvYpYuleXMmbDddsFiv2KFPIK2tsIee8Av\nfynbjWUfL0m5cUCsQXXB6+JeX1+ePnsQd0iQS6Tc0NvJjTSLfUofOKob1Ul01lnhLvZevSRCpLVV\nhvaridGN2MdL0mKvjt/aKm23cmV5W/aVSE1N/JZ9sTBin0KUJX/rreEHp6xeLW4cPdbeuHHiJUlf\nrC4ira2SNbJ7dyP2aaPSLfuxwFxgPnChy+d9gGeAmcC7wElxVa5a0S35KCMRjzsOHn7Yfu9j2fcG\nngfmAc8BPT3KLQTeBmYAb4arRWXy+uvJTjyvi8iGDXYe+Pp6OP54mRkrAkHXLMAt1uezgBH51Lka\nqWTLvg64FTl5hiFzWW7nKDMeEYOdgSbgBkxfQEHkI/ZuJ5iPZX8RIvZDgRet925kkDYdgcxVWrXs\nuae4y5LC6cZRT2h1ddJ38+c/hz5UmGv2YGArZK7a04A7Cqp8FRFk2UdNlQDpsexHIXOQLgRakcmn\nD3OU+RxQ3TAbA8uQuS0NeRJW7IN8+T6W/aHA/db6/cDhPodJqZ1SWegW45dfSuw52G0YwSUX5prV\n238q8mTXN596VxtJWPZpEfsBwKfa+0XWNp27gO2RmexnAfFMm1zFhBH7KVPks5tukvduJ6CPQPQF\nrKTOfIH3hZ4BXgDeAk4Nqrchf3SLcc4c2HxzWVdtGKGzPcw161ZmYJT6VitJ+Ox1Zs+GN96QV9h0\nE2EJshfC3G8uQfz1TcAQxD0wHMipanNz83frTU1NNDU1hatlleEm9s8/D2PG2Ns/+USWype7007Z\nxxgzZgwffLAEWA2sZtNNPU/CDN7tvDfy5LYJ0q5zgSnOQqZdC0dZjJMnw29/K6mNwR474Xbjbmlp\noaWlxbk5rI3oPBty9jPtmkvSlv2IETBkSAufftrC7rvL9JRxEST2i4FB2vtBiBWgsxfwO2v9A+Aj\nYBvEGsxCP3kM3jjFftkyycmyfLntNw46oZ5//nlee80+WebOlQEhNbLjF0A/YAmwGeA19EVlNlkK\n/B1xEfiKvSE/lMV49dWSqvoHP5Dtfa1nLjexdwrwlVdeCeGuWWeZgda2LEy75pK0Zd/aCu+808Tl\nlzfRtStceul37VowQW6ct5BOnMFAI3A08KSjzFxgf2u9LyL0H8ZSuyrlqKPg6KNlvb7enmbwoYei\nHUcXCEf43pPAidb6icATLrt3BTay1rsBBwDvRKuBISy1tZLlVCUOUykG+veXZQSffZhr9kngBGt9\nD2AFtlvP4EMYyz5qiK5u2euJ1OIe1xFUrTYk2uZZYDbwCDAHON16AVwF7Ir4618AfgUsj7ea1cX4\n8XYIZUODnep44kS7jJf1oOch6akFVDqy+V0DjEFCL/ez3gP0B5621vshVvxMpBPvKSRM05AANTXi\nqnv7bXn/9dey7NxZlhEmLwlzzU5EDLIFwATg5wVWv2pIKs7e+T6JTtsw9sIk66UzQVv/CviP2Gpk\nyKKhQbJxQnbje51QO+5or3fvbq+fdVbWzWI59tOYzmfAIdb6h0g4raEI1NZKOx96KAwaZLvflNhv\nsYVddtkyOS98UhEEXbMgNwRDRJKKs3de26USe0MJ0cV+0CD/siDWoULP4hk2IZWhNNTWir+2tlZG\nTitUG+pPbIMHw7BhMHVqUatoINkRtLoLJ+h78sGkS0g569fbk4/o0ThebLSRva6LvUmdkG7UrGLO\nEEt1wet+4FWr4NNPMZSAJEfQOq17I/ZVxjff2OtRO2yUCwBsv78hnSjL3in2yp3jHECX1iH5lU6x\ncuMYsa9CunWz1/WImjApFPROvZkz46uTIX5WrZLJrp1if8gh8NhjucnQ0jpBRqUTZNnnmy4Bsm8U\npYjGMZQY/aL+4AOYMUPWldgPGVL8Ohni53vfk6c4t5GyblMTGrEvDUnF2SufvS72xrKvMvSL+rLL\nYORIWT/lFFnqPnpD+dK/v6TAWL0697O6OnjttWzBN2JfGpIcQWvEvsoJyokS5J555pn46mJIjvp6\nGeX897/nfjZypIRbLtLGwS5caKfMMBSPIBGOq4PWRONUIW5iH+UkOPDA+OpiSA6/aKn+/WUuYmcn\n+7RpydbJkIueitqNQjtojWVfxbg9rusn2yGH5H5uKD+CQmM7dcoVe8eoaEMRSMKy9+qgNWJfZbhZ\n9noY3nKTmKIiyEfsZ81Krj4Gd8JY9vn0p7j57E00TpURJPZR7v56bh1DuggSeyUyugDcdFPwBDaG\neEnKsjcdtAZXsdejMpx57P0wLp/0EtQRrzrs3nTMBPzxx8nVyZBLEj57N4zYVyE9euRu0625MI96\neq4VQzoJsuzVY73TlVOM6ewMNsWy7E00ThVyxBG526KK/XMmMXHqCePGyWRy2ztuv67BnyQte+PG\nqXLcOnui+uxVjnRDeglr2be3w377wUUXyXYj9sXF+OwNRaWtzU533NHhPhBHxzzqp58wYq8s+7o6\nOPdc2W7atrgEWfb55sbRUxyrbSYax/BdKtwTT4QTToDDD8+etMRJ7945m44C3gPagZE+XzUWmXZy\nPnBhIXU2+KM6aL2mnlRunPZ2Wd90U8lp7yIIzyMzkD0H9Mz5VFgIvA3MAN70KGNwIckUx+r4amks\newNtbWIJ3nefPNJDbqIsndtuy9n0DnAE8IrP19QBtyKCPww4FtguvxobglAZSn/4Q/fPlcgoyx48\nrczngaHAi8BFHl+XAZqAEcgk8oaQJD15SZJib6a0KEMWLoQNG7K3+cVb98y17+aG+JpRyBylC633\nDwOHIfOZGmJGTTTjFYLptOzB08q8X1u24C34JiN+HtTUwLvvwt13u3++ZEk8HbRJROMYsS9D3PLd\n+Fn2ESar1hkA6PMhLQJ2z+tIhkDURDNeYu9l2asbwC67fFf0C23Z1+PrMsALiBtvAnBXofWvFvbZ\nR0Yuv/66d5kzzoh2TC+fvRF7gytnnw3z5tnvx4wZw5IlS4BwE524EPpUa25u/m69qamJpqamvL6w\nmlGWvddQe3XxX3HFGBYuXMKOO8K8eavZb7/V1NfDV1/t59wlg3cb7g18DmyCuH3mAlOchUy75rLz\nzt5Wfb44E6G1tLTw9NMtfPEFaE1QVmQM+QGZzKBBmcwll2QyNTXKBgi//6JF9j5kC8DLeHfQ7gHo\nCZIvxr2TNrkfXkVcf720z8qV7p/vv38m89xzmcyjj2Yy48bJtl12yWSmTctk1qzJZDp3/q5t+1nt\nshnh3HVXAOebdi0dLS2ZzA9+kMksX57J9Ogh2+64I5M57TRZJ4Lh5YfpoC0TBg6E8eMlCiMqAW4c\nLw/jW8DWwGCgETgaeDL6txvCEOSz1+PsnT57fRtworZ8wuVQXQE15U034ACkw95QIkycveE71KQW\nDQ35Jb5yEfsjEH/8HsDTwCRre3/rPUAbMB54FpgNPILpnE2MIJ+9PoLWzWev7TcGCb3cD7jG2qa3\naz/EZTMTmAo8hYRpGkqMicYxsM02slyxQmYsiorLgJ2/Wy8nnwF6urRJ2DcCQ4IceKDkMPJ6CnOz\n4tV9qwUAAArOSURBVFXopUPs93fZXW/XD4GdY6y6oUDcOmhNNE6VowuBmoM26n6GdDJoEJx1lvfn\nzhG0kJ32OChrpiG96KGaxo1jAKBrV3v9xBO9yzkJGopvSD9+cfYOn72hDDE+e0MWNTVw3HGyHsWS\nM1Zf+aPH2Sth//BDmcDE4cYxlBleHbSlyI0TJj9KE5Jn411k1J4hIT75RJZRrPU4JlMwlJZMBg47\nTKahVML++efScW/EvjIo9aAqlR9lf2AxMA0Jv9OjMnoCtwEHIqMs+8RbRYPO4sWyNK6Z6mLdOll+\n+22uy2b9eiP25YxzUJVzW1wEWfZ6fpRW7PwoOj8FHkeEHuCrGOtncNDYKMuoYt/SEntVDEVEif2G\nDbnCvvXW9hOfofxQAl/qmarc8qMMcJTZGuiNjMZ8C/h/sdXOkEPQ4Bsv9t0X3nor/voYioMu9qYz\ntvJIQ9bLMF/XgAy5/xEyOu8N4F+Ij98QM0rs83HjaMmyDGWGLvbq6c5QGaQlxfFiYJD2fhC2u0bx\nKeK6WWu9XgGG4yL2JrFS4UQV+5aWFlqMD6fsUW6a1lZ7tK2hskh6pqogydDzo3yG5Ec51lHmH0gn\nbh3QCUmDe6PbwZrLNYVbijjrLJgyJbwbx3lTvfLKK5OpmCFR/Hz2hvKmWB20QWKv50epA+5GInFO\ntz6fgIRlPoNMc9aB5MaeHW81DYojjpClicapLtR8BcZnX3m4ddCWKjeOW36UCY7311svQ8IokTfW\nXXWhHumNZV+ZOH32pYjGMaQMPQmWofowln3lUayZqsxpU6bEfSIYygMvy/5//7f4dTHEj7Lsu3eH\nJ56A3r3jO7YR+zLFRGRUJ7plP2yYvX2vvUpTH0PhuIVeHnQQfPklLFgQ3/cYsS9DOjqgZ89S18JQ\nClpabMt++nT4zW9k3eQ/Kn90sQfo1ctY9lVPDBf2UcB7QDvec9CCpMl4G0ly92bB32qIBSX2nTrZ\ncxVofvww7RomuaGhSLhZ9klgxL46eQeZmvCVgHIZJKPpCCRPkiEFHHigva6EXxOJoHZVyQ3HAsOQ\ncTPbxV1HQ3iS6Ix1w0RrVydzI5Q1DoIUsXQp9NHyyqpQXE3s5wUcQk9uCHZyQzO/cIkxlr2hlGSA\nF5CR1KeWuC5Vz6abZgs9uFr2QYRJbmgoIsVy4xjLvkIZM2YMS5YsKfQwewOfA5sAzyNPBFOchUzO\no+LQuXN2u65evZqlS1cDcM45+4U9TGiHgWnX4qLEPql8VsV8RM9kTHB4yakR00G1+8vA+cD0ELte\nAawCbnBsN+1aBK64ArbbDo45Jnv7nXfC6afLFIVbbvld2/q16x5AM+KzB7gYSXNyraOcadciMXOm\nzCn92GMwdix88EH2545rNm+MZW/wOom6Ip15K4FuwAGAyaJWIrzy1/m4cbzaNUxyQ0MRcUuElgTG\nZ1+dHIH4bfcAnsbOfdTfeg/QD3HZzASmAk8BzxW3moYgVAetFnoZ1K56csPZwCOYztlUYHz2hiT4\nu/Vy8hlwiLX+IbBz0WpkyAsXy36QSzG9XcE9uaGhRJg4e4PBEIhL6KWhTDFibzAYPMkj9NKQMoo1\nqMqIvcFQxiixN2mPyxe3yUuSwJwiBkMZo9w43buXth6GwjA+e4PB4Iuy6Lt1K209DPljOmgNBkMg\narpC48Ypf5L225tTxGAoY9raSl0DQ6GYQVUGgyEQI/blj+mgNRgMgRixrwyMz95gMPjS3l7qGhgK\npaYG1q6FOQknrTBibzCUMcayL3/69IF+/eCaa2D06OS+x+TGMRjKmAFm2pGyp08fmDo1+e8x+eyr\njLhyY2uYdi0x69fL5OMxt61p15QQV7saN47BUOZ06lTqGhjKASP2BoPBUAWEEfuxyNyj84ELfcrt\nhkyMcGQM9TIky++RCStmAX8DeniUC9v2hvTwHtAOjPQpsxB4G5gBvFmEOhlSQJDY1wG3Ihf9MGT6\nsu08yl0LPEOR+gHinJA37sl90348ZMap7YHhwDxkHlInYds+VtL836W5bhpHAK8ElMkATcAIYFQS\nlXCS5v8uzXWLkyCxHwUsQCyBVuBh4DCXcv8NPAYsjbNyflRTYydw8jyPTDINMuXgQJcyYds+VtL8\n36W5bhrzQpYragb8NP93aa5bnASJ/QBkTkvFImubs8xhwB3We9OFX178FzDRZXuYtjeUJxngBWTy\n8VNLXBdDkQiKsw8j3H8ALrLK1lBki8HgzpgxY1iyZElQsUuBDcCDLp+Zm3YK8WrXq666Ksph9gY+\nBzZBnvLmIpPLG6qYPRA/vOJicjvqPgQ+sl4rgS+AQ12OtQAREPMq7WuB1R4nAa8BnV3aCsK1PZh2\nTdNLte3L+HfQ6lwBnO+y3bRrel6qXROlHvgAGAw0AjPx76S7FxONUw6MRaI2+viUidr2hvTwMrCL\nx2ddgY2s9W7IDf+AYlTKkH4OAt5H7i4qauN06+XEiH15MB/4GAm9mwHcbm3vDzytlXNre0N6OQLp\nZ1kLLAEmWdv1dt0SuXHPBN7FtKvBYDAYDIYo5DMwZxDyKPoeYn38wtreG+lQmofEivfU9rnY+o65\nuD+W1iFW7D9jOFZPJNR0DjAb2L3A411s/dZ3kM7SThGPdw/SV/KOViaf+uxiHWM+cLNLPZ1Ebdsk\n2hXia1vTrkKltSvE27bl2q6JUoe4AAYDDYT3+/YDdrbWuyOuhO2A64BfWdsvBK6x1odZx26wvmsB\nuWGl5wEPAE9a7ws51v1IyCKIb7tHAccbjHRyqwwnjwAnRjzePsgAGf3kibK/iqB6E3uQzUTkovci\nn7ZNol0hvrY17VqZ7Qrxte1gyrNdE2dPsiM6LrJeUXkC2B+5s/W1tvWz3kNupMgzSDSJYiASV/xD\nbCsh32P1QBrbSb7H641cHL2Qk/CfwJg8jjeY7JMn6v6bIVaP4hjgTy6/UxFH2xbarhBf25p2FSqt\nXSHeti3Xdk08EVocA3MGI3fBqcif8YW1/QvsP6e/dWyv77kJ+CX2qFEKONYWyEjhe4HpwF1IVEO+\nx1sO3AB8AnwGrEAe5/I9Xr6/z7l9scdxFYW27WAKb1eIr21NuwqV1q4Qb9uWa7smLvaZAvfvDjwO\nnI3E8DuP7Xd89dmPgS8R35/XgK+wxwK5m49EIlhGAqvJtXyiHG8IcA5ykfRHfvPxBRzP6/NC2yLq\nd/oRR7tCvG1r2jXcd/qRxnaFeNu2XNs1cbFfjHTeKAaRfTfyowE5cf6CPBaC3PH6WeubISeE2/cM\ntLYB7IUM8voIeAjYzzpmPsfCqv8iYJr1/jHkBFqS5/F2BV4HliFZQ/+GPErnezxFlN+3yNo+0LHd\n7biKfNs2rnaFeNvWtKv7ccq9XSHeti3Xdk2cfAfm1AB/Rh7ldK7D9l9dRG4nRiPyyPYB7hbBvtj+\nv0KO9Qow1Fpvto6V7/GGIxEMXazt9wNn5XG8weR2+EStz1QkSqGG4A6ffNo2qXaFeNrWtGtltivE\n17bl2q5FIZ+BOaMRX91M7IE/Y5HOkRdwD0+6xPqOucCBHsfdF7tnv5BjDUesBD0ffCHH+xV2KNf9\niJUU5XgPIf7DDYi/9eQ866NCuRYAt7jU00nUtk2qXSGetjXtKlRau0K8bVuu7WowGAwGg8FgMBgM\nBoPBYDAYDAaDwWAwGAwGg8FgMBgMBoPBYDAYDAaDwWAwGAzVw/8HL4cQZpXbGNMAAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def mixing_demo (q_var):\n", " n = len(q_var)\n", " for i in range(n):\n", " q = Proposer(lambda x: multivariate_normal(mean=x, cov=np.sqrt(q_var[i])*np.eye(2)).rvs(),\n", " lambda x,y: 0) # Proposal is symmetric\n", " sampling = MetropolisHastings(log_p=p2d.logpdf, q=q, x=[1.5,0])\n", " samples = [sampling.sample() for _ in range(1000)]\n", " plt.subplot(1,n,i+1)\n", " plt.title('Sdev[q] = ' + str(np.sqrt(q_var[i])))\n", " plt.plot(map(lambda x: x[0], samples), 'b-')\n", " \n", "mixing_demo(np.array([0.001, 0.1, 10])**2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Is there an optimal proposal or acceptance rate?\n", "\n", "* Obviously, choosing $q(x'|x) = p(x')$ would be optimal as it is always accepted and produces independent samples ...\n", "* When the proposal is a circular Gaussian an optimal acceptance rate of 0.234 has been derived.\n", "\n", "As a rule of thumb, the acceptance rate is often adjusted to around 0.2. But as an information theoretic argument by MacKay suggests, Metropolis-Hastings sampling is always rather inefficient as it takes a random walk across the space. In general, when the probable states of $p(x)$ cover a length scale of $L$ and the proposal takes steps of size $\\epsilon$, we need at least $(\\frac{L}{\\epsilon})^2$ steps to obtain an (almost) independent sample. This is based on the observation that a random walk/diffusion with step size/standard deviation $\\epsilon$ covers a distance of about $\\sqrt{T} \\epsilon$ after $T$ steps.\n", "\n", "Thus, many algorithms try to utilize proposals adjusted to the target distribution in order to improve mixing, speeding up sampling.\n", "\n", "**Exercise:** Illustrate the effect of the proposal width when sampling from a multi-modal target distribution." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Convergence diagnostics\n", "\n", "Most MCMC algorithms, e.g. Metropolis-Hastings, produce a Markov chain sampling from the desired target distribution. Thus, all samples produced are correct in principal, but the intial samples can depend strongly on the initialization especially if the chain mixes slowly. It is common practice to drop an initial part of the samples, the so called *burn-in* period.\n", "\n", "In general, it would be desirable to have diagnostic tools in order to access the quality of samples produced and detect sampling problems. In particular, one wants to know how well the chain mixes and how many (almost) inpedendent have been produced. In the literature, several diagnostics have been proposed ... here, some of them are listed:\n", "\n", "* Traceplot: Always look at your samples as problems often show up visually (see mixing example above)\n", "* Autocorrelation: Similar to importance sampling an estimate of the effective sample size can be computed. In the case of MCMC, the sample size is reduced as the samples are dependent and thus empirical averages are less reliable when taken over dependent samples. A commonly used measure of effective sample size is defined as follows:\n", "$$ N_{eff} = \\frac{N}{1 + 2 \\sum_{\\tau=1}^{\\infty} \\rho_{\\tau}}$$\n", "where $\\rho_{\\tau}$ denotes the correlation coefficient at lag $\\tau$.\n", "\n", " This is motivated from the sample mean of a stationary process $X_1, X_2, \\ldots$. Then, the sample mean $\\hat{\\mu}_N = \\frac{1}{N} \\sum_{i=1}^N X_i$ has mean $$\\mathbb{E}[\\hat{\\mu}_N] = \\mathbb{E[X]}$$ and variance $$\\mathbb{V}ar[\\hat{\\mu}_N] = \\frac{1}{N} \\sum_{\\tau=-N}^N (1 - \\frac{|\\tau|}{N}) \\gamma_{\\tau}$$ where $\\gamma_{\\tau} = \\mathbb{C}ov[X_t, X_{t+\\tau}]$.\n", " \n", " Now, when $\\sum_{\\tau} |\\gamma_{\\tau}| < \\infty$, we see that the variance converges as \n", " $$N \\mathbb{V}ar \\to_{\\infty} \\sum_{\\tau = -\\infty}^{\\infty} \\gamma_{\\tau} = \\sigma^2 \\sum_{\\tau = -\\infty}^{\\infty} \\rho_{\\tau}$$\n", " and compared to the case of independent samples the variance is increased by a factor of $1 + 2 \\sum_{\\tau=1}^{\\infty} \\rho_{\\tau}$.\n", " \n", " In practice, the autocorrelation function has to be estimated which is often done by fitting an AR(1) model to the produced samples. Obviously, plotting the autocorrelation function is also useful to access the quality of MCMC samples.\n", "* Convergence: To diagnose convergence, i.e. check if all samples can be considered as being drawn from the same distribution, the following two methods are commonly used:\n", " * Geweke proposed to compare the mean of two non-overlapping parts of the produced samples, e.g. the first 10% compared to the last 50%. Convergence is then diagnosed if a t-test cannot reject that the two means are the same.\n", " * Gelman and Rubin proposed a more sophisticated method which assumes that $m$ MCMC-chains have been simulated. Then, they define the *scale reduction factor*\n", "$$ \\hat{R} = \\sqrt{\\frac{\\hat{\\sigma}^2}{W}} $$\n", "This is based on the observation that the variance can be estimated in two ways: As the mean of the variances within each chain, $W$ or as the variance of all samples $\\hat{\\sigma}^2$. When the chain has not converged $W$ tends to underestimate the true variance as not all of the sampling space has been explored, while $\\hat{\\sigma}^2$ tends to overstate the variance if the initial conditions are chosen at arbitrary, over-dispersed positions. Thus, convergence is detected by $\\hat{R}$ being close to one." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Gibbs Sampling\n", "\n", "Gibbs sampling is a technique to sample from a multivariate distribution, e.g. $p(\\mathbf{x}) = p(x_1, \\ldots, x_D)$ where each sample is a $D$-dimensional vector $\\mathbf{x} \\in \\mathbb{R}^D$. Instead of sampling directly from the full joint distribution, Gibbs sampling considers each coordinate in turn and samples a new value from its conditional distribution, e.g.\n", "$$ p(x_d | x_1,\\ldots,x_{d-1},x_{d+1},\\ldots,X_D) = p(x_d | \\mathbf{x}_{-d}) $$\n", "\n", "This Gibbs sampling step can be considered as an adapted Metropolis-Hastings proposal which is always accepted. The multi-variate extension then draws on the observation that MCMC transitions can be combined, either in random or in sequential order, if each individual transition leaves the target density invariant. To see this, consider several transition densities $p_1(x'|x), \\ldots, p_K(x'|x)$ and mix them with probabilities $\\pi_k$, i.e.\n", "$$ p(x'|x) = \\sum_{k=1}^K \\pi_k p_k(x'|x) $$\n", "Then,\n", "$$ \\int p(x'|x) p^*(x) dx = \\int \\sum_{k=1}^K \\pi_k p_k(x'|x) p^*(x) dx = \\sum_{k=1}^K \\pi_k \\int p_k(x'|x) p^*(x) dx = \\sum_{k=1}^K \\pi_k p^*(x) = p^*(x) $$\n", "since each $p_k$ leaves the target density $p^*$ invariant.\n", "\n", "**Exercise**: Show that Gibbs sampling is a special type of Metropolis-Hastings sampling." ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class GibbsSampling (Sampling):\n", " def __init__ (self, transitions, x):\n", " self.transitions = transitions\n", " self.x = x\n", " \n", " def sample (self):\n", " x = self.x[:]\n", " K = len(self.transitions)\n", " for i in range(K):\n", " x[i] = self.transitions[i](x)\n", " self.x = x\n", " return x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Gibbs sampling requires that the conditional densities can be sampled from. It is most often used, when these can be computed in closed form, but it can also be combined with other methods such as slice sampling. Below is an illustration of Gibbs sampling for the 2-dimensional Gaussian example." ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD7CAYAAABKfn7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8jdcfwPFPpiRi77232KOliL1nKdpqq61Vbc1S5WfV\nqk3VKFVUtapqbyVGqdpiC0KtCIkkEtnP748vIk0kIffeuMn3/XrdV+597vM85zwtXyfnfM85oJRS\nSimllFJKKaWUUkoppZRSSimllFJKKStnk8zrnYA9QDrAEVgHDHv2hIoVKxonT55MZjFKKZXm7AHc\nzXVzl8c/7YG/gTf+872RWo0aNSqlq2BWqfn5UvOzGUbqfr5U82zvv28YWbMaRq9ehrFvn2H06WMY\nbdoYo0aONAzDMADjeUHX1gSBO+TxT0fADvAzwT2VUsq6RUbC/fvP/750aShXDr77Di5fhp07Ydky\nsEm8I8QUgdsWOAH4ALuBsya4p1JKWa+jR2H8eChQAMaNg9DQuOcMGQIODtChAwweDGvWQKZMSbq9\nnQmqaADzgbnACMAL8H7m+9EAHh4eeHh4AFC4cGETFPtqSE3PEp/U/Hyp+dkgdT/fK/1s589DnTqw\nYwcMHw7Hj8OYMVCihLyesLGBypVhwADo1QuPAgVYsmQJ3t7enDhxgj179gCMia+I5A5O/tf/gEfA\n1GeOPe6uUUqpVM7fH2rWhEuX5PO1a1CwIGzZAp9/DuXLw4wZULiwdKU0bQq3b0NICJw4AZkzP72V\njXSZxBujk9tVkh14UpIz0Bg4nsx7KqWU9YmMhLfegooV5XPx4hK0AZo3B09PqFZNXuPGwcCBYGsL\np05BvnyQJQvcvZukouyTWdU8wFLkHwBb4Cfgz2TeUymlrM/AgWBvL33Wv/8OjRvH/t7JSbpO3n1X\nWtwgP8uWhRs35POSJdL3nYjkBm5PoEoy76GUUtZt4ULYvh3+/hu+/lqONWwY/7lZskD69BAcDN7e\nsb8rXjxJxSU3cCulVNq2dy+MGAH790sf9f79crxkSejWDRwdJVAHBsKVK7BvX/z36dIFWrZMUpEa\nuJVS6mVdvQqdO8Py5ZIxEhEB//wj3+3cKccT0q4drF0r7y9elAHKmjUTLdYUedxKKZX23LkDRYtC\n7tywdatkiDg6xnx/9izMnCn91s8aN04GMg0DVq+WiTi7dknudxKCNmiLWymlkmb3bmlFnz4tmSBP\n+qfLloUcOSTdr3Jl+OYbyeFu1Ah8fSFnTjmveHEoVgxmzYJ06aBnT9iwAVxcwN39hapi6jzu+Gge\nt1LK+r37rgxAzpkDhw6BhwccPBi7ld2uHaxbJ/3Zq1dD9+5yfNo0yJULzp2T4+fPx1yzfj20bh2n\nuITyuDVwK6VUUoSESHdI5cqS8te7t7S+z5yBAwdg8uS4WSIgKYKlS0OZMjGvsmUluH/9tUzWGTMG\n3n4b7GIms2vgVkopU7h+HQoVStq5X34JH34IRYpI8H6ePXskv9vfX/q/27UDGxsN3EoplSznzkkr\nOalCQ6UfO6kMQ6bFf/WVLDw1fjw2TZuCBm6llHoJDx7AH3/IoOJHH8mA4s6dzz//448layQsTF6h\noTHvE3qFhkJUlAR8Fxds/P1BA7dSSiWRl5cE6A0b4PBhWW715s3Er8uQQbJK0qWTl5NTzPukvBwc\nnq7HnVBXiaYDKqVUVJRkiGzYIFkePj7g6irfPXworyfc3WM2QHiiWzeZEXn0KGTNavbqauBWSqVN\ngYGS3rd+vfQv580rAToiQroupKtCNGwI06dLwL56Ffr3l4k3EyZIat/GjTLV3QJBG3TmpFIqLfH2\nhm+/hSZNZCnVRYtktuKRIzJVvV49yQRxc5Oui3HjJA1w506ZJTliBLz2GtStK+uOHDgg+dx798r9\nLET7uJVSqZdhSB/1unXSDXL7tizk1Lq1BO8MGWKfHxEhy7Hu2SOBuGZNSf/LkgUWLIAGDWDSJMiW\nDd55RwYu16yJex8TMOdGCkop9WoyDBg0SDY3iI6G+fNlfZElS+DNN+MGW8OAXr1kJb8//pCW9o0b\nkCePdJusXCmb+WbMCK1ayfmbNpklaCdGW9xKqdQnKkrWAjl7VoJrUvqeR42CzZtlTRJXV5ls06WL\nLNW6bBlkzy67trdoIV0p8+cnPLEmmbTFrZRKO8LCZKnV69dlsaekBO2FC+HnnyXIP8kmKVhQukzc\n3GSa+8qV0rddr56cb8agnRhtcSulUo/gYFlHxNUVVqxI2uzFTZtk0szevbF3YX/W8OGSQZIpE/j5\nyV6RZqYtbqVU6ufvLwOOefNK6zgpQfvwYfjgA9nMIL6gHRoKgwfD4sWSjeLmJhv/PpvXnQI0cCul\nrJ+Pj0yMqVEDfvghad0YXl7Qpo0E5fg2MDh8GKpUkRTC/fsl4+TWLfkHIjDQ1E/wQjRwK6Ws27Vr\n8MYbkikyfXrSujF8faXlPHp03LWww8MlX7tVKxmczJMHqleXPO+ff5afefOa5VGSSmdOKqWs17lz\nskb24MGyA01ShIRIsH6yI82zTp6E996DCxdkNcA5cyQ7xdPTohNsEqOBWyllnd5/X9L0Bg6Ezz5L\n2jV+fjJ5BiRfu2NHeR8ZCSNHwsSJ8tnNDT79FLp2BWdn09c9mTSrRCllffbulbQ8kI0KoqKkFd26\ntfR1p0snCz4tXQpDhkhq4MKFMRv3du8uWSJPNvpt3lyOV60KU6fKvW0sER6fTzdSUEqlHps3SyZI\n167ShfHnnzLR5skyrAcOJHz9gQOy3siePVC/vhxLn142AC5a1OzVTypd1lUplTr8+iv06ycr+lWo\nIC3mgABZtS9XLpnSnlDgvnpVFoVq2hSCguTYyZNyLyuiLW6llHX48EP48ceknZspkwT0hEyfLv8I\nWGAyzcvQrhKllHWbNEkGDsuVgwIFpE87LEzWwU5IvnzP37lm3z5Zh+TJlmHVq79SQdycgbsAsAzI\nCRjA98Ds/5yjgVsp9WJOnpQNCkJCYMAAaT337Svrjly5IvnUz7K1lRUAn3BxgZw5ZfJMYnLnhsKF\nZenXnDlN+RTJYs7Anfvx6wTgChwF2gHnnjlHA7dSKnHR0bITzdSpcPmyzIJcvVq+69oVTp+Wwcgn\nsmSJvUvNExkzxj+zsW5dyR7x8IBt22Qgs0kTcHQ0y+MklznXKrmDBG2Ah0jATtkpRUop6xIaKtPU\ny5eXGYs9ekhr295eJsj07Qu//BI7aBcpEn/QBgn258/Lxgc5cshyreHhkkXy5ZeS/vfzz1LO1auW\neUYTM2VWSWGgMnDIhPdUSqVW9+/LmtZz5siyqdOnS5/0kSOSnvfEM5vyznGux0nHAiz4tiu277wt\n/dRRUVCnjkxNv39fpsAPGiR52Fu3ynojz7pxQ1rbjo6ymqAVMlXgdgV+B/ohLe9YRo8e/fS9u7s7\n7u7uJipWKWV1Ll+WaeS7dsnn0qWl5duunfQ1X7gQ6/SdJZoSOXI0BSoXpWZIBGObL2fh9GnSenZz\nk5MOHJAZlJcuyVZiU6bIFmTPTqIJC4OZM+W7Pn2kle/iYplnTgIPDw88PDySdK4pskocgI3AFmBm\nPN9rH7dSCv7+G4YNkz5mkEkwjRpJpki5cvhev8feBVu5tfsInz3cxdWaTcm1aCYu5Us/vcWUKX/h\numwRfVzPS2v70CGYPVs2TOjWTaapx7c869atkvpXqhTMmBF3jZJXUEJ93Mm+N5JVMiOBcwylVBoV\nGWkYa9YYRu3ahiG7NBpGz56GERr69JRTp+4YfTr8EPP9k1fp0obRr59hbN5sGMHBxo4dl43Xsw81\nolwzGMbw4YZRtaphFC9uGLNmGUZAQPzlX7liGG3bGkaxYoaxcaOFHto0kEy9eCW3q6Q28C5wCjj+\n+NgwYGsy76uUSg3eeUcGCsPCZKBwyRLZsxH466/rTJr0F4cP32TIJxXgj/9cW66c7KI+cSJRHTuR\nNywDB6LuyHeHD8OYMZIlEl/u9aNH8M030n8+cKDMuHRyMuujWpJOwFFKmcd330nXRevWMhj43ntQ\nsiQ3Lt7hrz1XCfIP5rVK2SnjEoLd1s3x32P4cP6tXI+z7/SnadhZOXb6tAT1+BiG7GYzcKCkE06d\nKhN2rJDOnFRKWY6Pj+wq89VX8vnUKTh2jKv7z1Bk0ZTEr+/SRVrIhQs/nUBzsVZrSm5cKrnbz3Ph\ngqzJfeOGbDPWoEGyHyUlaeBWSlmOt7fkWZtAr6zv837jHNTatRTefVdysjNlin1SUBCMGxfzj8Wn\nn4KDg0nKT0m6WbBSyjQiIqQ7Ij4rVsDQoSYJ2kHlJPd6TK6z1GpbRVL4duyQrJBly2KGMFesgDJl\npJXv6SnT41NB0E6MtriVUklXp44Exu+/l62/nvXRR9LqNaXs2aXLIyAg5hUUBOPHSx52UJAMQNaq\nZdpyXwG6HrdSKvkOHpRdzvv2lRzs3r1l4PHQIcmRNrV69eTl4hL7dfAgfPGFZJX06AF2dqYv+xWn\nLW6l1PNFRMisxosXY3ZDd3ePmUTzshwc5N4JGTdO9oIMCYl5BQfLDuvDh8fsHZlKaYtbKfViDhyQ\nfRmvXZP1Q+wfhwo7O8mddnPDWLECw88fWyM64XvFJ7GgvXy55ICreOngpFIqrjJlZHnUDz8ELy9o\n1kyOR0XJAOS33+IdYBt/0B469KWKjDhyLGbQUYN2gjRwK6XiypJFsjiOHpUgOvu/+6NAkUjf+K/9\n5hsAVhb/9IWKdNjr8aK1TLM0cCul4ufsLFkkv/zyUpd39pqT4PdfurbFY9Y6yJ9fFn76/vvYu9io\n59LBSaXSuogIGXw8cybm9WTnmeeIHPoVXmceUnpj3JZ4Uj18vS6uLRrD3bvxtujTOp05qZSKa9cu\nmSJ++bKs51GunGRxJLYBr6kMGyZbiE2aJGtnq1h05qRSKq5ff4W33gI/P9i0SbopEgjafzYbbZJi\nDff60KaNrBJ4+bLkaqsXooFbqbTq5EnZfaZ/fyhZEtavT/D0hltHv1QxV6s15q2MH8mHypWx2b1L\ndlS/ePGV3qz3VaZdJUqlRVFRMbnZZhZuY4+jESkfIiJiym3fHjp0kJ1rVBzaVaKUim3LFrPe/t+D\nZ2hWYTJATNBeuVJ+rl4N9etLqmHLlmatR2qlgVuptCQyUgYFn0xffxE//xzzfsSI5552u3Yzdjb4\nmLUXx8qBWrVkfZOLF2XlwJkzZZ0TLy/ImvXF66G0q0SpVO/GDcne2LRJdkA3s3u2roR0+5CCD2/C\n5s3QqpVM5unUSQJ4xYpmr0NqoF0lSqVF0dHwySdQsSKRW7dz9diV5N0vY0Zo2/bpx6tOcXdKn17m\nQ7h4gYJHd0mXiIuLrCR45YpMsNGgbRLa4lYqNQoIgMyZAdi44giff7mHK9cHmb3Y6E6dsF21Sj7M\nny/Lrsa3ma9KlLa4lUpLwsJkqvpjgX0HsKV1gFmLDCxRHiAmaJ88Cb16adA2E21xK5WaREcT2aw5\n9ju2c8M2E/mjzRuwASJKlcVhYD8YOxZu3pT+7EaNzF5uaqctbqXSACMqiquNOxG204MHTpktErQN\nW1scur0NuXNL0K5ZU4O2BWjgVsqaRUdD69ZEO7tgY29PkV1/kN4IJ3PoA7MWu3PJXnj4EJsNG2SH\nnCeDlpqXbRHaVaKUNRoxQnY4v3rVosVuzf4aFReMIU+HJnIgIkLS/QoXhpEjwdUVMmWyaJ1SK10d\nUKnU5NIlWVvEgu7YZWJm/1/5emJjHBweb85rGLJDjq8vrF1rsSn0aYX2cStl7YKCYMgQsLGxeNAG\nOLb+MJOmNosJ2iC7rJ8+LasMatC2KFP8114MtATuAm4muJ9S6lnR0fDuu4mu3mcu9zbspEWLErEP\n/vAD/PSTbCrs6poi9UrLTNFVUgd4CCwj/sCtXSVKvah792TLsEuX4NtvU6waRr582Hh7x25Rb90K\nH3wAe/ZAqVIpVbVUL6GuElO0uPcBhU1wH6XUE5s3w9y5hDs6kxKrVUemz4B9l7ewKVQodtA+dkyW\nYV23ToN2CtKOKaVeQVEPArE7f96iQTvQ1plNbh1ovmkWmfNli3uCt7esKrhggaz4p1KMRQL36NGj\nn753d3fH3d3dEsUqZZWOLdlGlX6fWay8MKf0fGXXlIozhvJejxrxn+TnB82bw9ChsvmBMjkPDw88\nPDySdK6p0gELAxvQPm6lkmbGDNi3D3Llgpw5IVcuAp0ycXjqChqe22zRqjSpOpP581tRNI8ThIbC\no0cxryefp06VWZFTp1q0bmmZJfK4C6OBW6mkmzMHpk2DL77A8PHh4o5jlDpood3V45M5Mzg7y8vJ\nKe778uVh3DhdNMqCzB24fwHqAdmQlMCRwI/PfK+BW6n/CguDUqXwGj2LnydsYtSlhRavwpGfdlLt\n3C5JN5w40eLlq4TpzEmlUsqjR5LO16MHZMny9PDDgEdcrtaEil77U6RaEQ2b4LBzG/z1l+xKc+JE\nitRDPZ/OnFQqJfj4yKa4S5dCgwbg64thGKz/7SQ3chSLN2hvad7JbNXJl30CJ94eCI6OOGx73I9e\nsyZcvw63bpmtXGV6GriVMoczZ2TLrqZNwdMTWrQgonZdFrp9QJvOlSgdcTvey5pvWWXyquzJXg2A\nm/e+otLpnXDkCNg9nrpuby/LsG7bZvJylflo4FbK1LZvl5b22LEwZgyR0fC9bXUcLp2n55llFq3K\nJbscOJcqStTq1TIb8+RJcHsmhyA6GipXltmQymroBBylTGnBAhg1Cn7/HerW5ehBb6rWKkLPFKhK\n53z9GbxmMDWq54s5GBwM//wja4wcOAAHD0rf+3vvpUAN1cvSwUmlXlZkJAQGQlSUrEv9xRewZg2s\nX09Qjnxs7v8dnT2+s3i1zjnm47RbQ5rv/h5XvzsxQfrAATh/HipVkpmPtWrB66/L7jXqlaNZJUqZ\nwyefyMCjYUj2CICzM9ERkdhGRli8OtG2dthGR8mH9u3h0CH5R6V27ZhAXaUKpEtn8bqpF6eBWylz\naNYMOnWCefOgfHlujZjEpfqdqHcjZVL8nmrYELp3l0BduLCs4a2sjgZupczhcUCMHvs1fwTmo9yM\nYZSJ8knZOs2cCf36pWwdlEmYe1lXpdKezZIHfXvIWDpszsPBvz+0eBW+rd6XxSGl8ci6jkz7dkK9\nevCZ5RanUilH0wGVelFz5mC8/z4ADRZFsuLRcgAW5vmcX9t+Y5EqtM3xOVdqt+Lgkd5kmj8LsmaF\nxYt1LZE0QrtKlEqqqCgYMIDg9VvoFdmc5Te/JdLFFfuQhxatxt50JXGeMIbq/d+SPOxatWTT3t69\nLVoPZV7ax61UcgUFEdGxM16nbzE5tAo/+v2Y+DUmcqFcKx7Mm81vXcYSXbES42sG47JqBYSEQNmy\nEB4uMx91EDJV0cCtVHLcuMGDOo04dtuGzPmyUib4X/60e4NWt34xe9EPqjfgu1YjmP3dUWbNakaX\nLuXlC8OQhaF+/x369IH8+c1eF2VZGriVekm+W/eSrm0rMoYHAXA3SzF6FVzMmpP1LFK+e9Vvsc+c\nkSVL2pE/f0aLlKleDbo6oFIvKDraYHvfadi0aEHG8CAiXDKyI1srHCOCLBa0l6WrQafuNdi+vZsG\nbRWLBm6l/uPixfu0rzOHJnMHc2PSIpbUnopDSCAVnc+S+eFdi9ThbPpCtPyoDn371sDWVvuuVWza\nVaLUY5GR0UyffpDJk/9iUTsb2v0wJEXqcTtvKXJcO4u9vbar0jLt41YqEceP3+ajj9aTPbsLX79d\nlJrd30i5yoSHg4NDypWvXgk6c1Kp5wgNjWTs2D0sWnSMiZMaE/ggH2U/LmbROpy3z8Wleh1oOrg9\njtWraNBWidIWt0qz9u+/zkcfrcfNLSc9P2lKSPcBtLtumfxs38YdybHjdyZmb0e9dbOpVauARcpV\n1kO7SpR6RlBQGF999SerV59j2oyWnL5aiAaTO9DQ3zK7wEQ6pcc+NJib2YqQ5dp5XNI7WqRcZV00\ncCv12LZtXvTqtZH69YvQ4d1mTBnrw969hUxejm+F/OQ4dSPhk/z9IXNmk5etUgcN3CrN8/N7xMCB\n2/Dw8GbG7DYc2WXL+FlFLFL2vlqfkblKLnYu28eAwG2EbNqGS4smFilbWS8N3CpNW7v2PGN6/0rb\nJgVoXDwbtUe1t3gdSuT/hjmLOtC0GDI93cnJ4nVQ1kUDt0qT7t4N5rPPtnDnnzPsvv41UTZ2OESF\nm7XMwKw5+cuuNs1918Q6HuAbSKbsGcxatkpddMq7SlMMw2DFCk8qVJhH4cKZ6DV7OG+W32XWoL27\ncXs21enPrbCcuHOYnZU70CzHADZtOA+GoUFbmZS2uFWqcvNmIH36bOLq1QdMnt6ORRvzcPoaNOx/\nl7n1c5mlTL9c+Yn2C8OzYiecOten29wb1HqjELNmNSNLFmezlKlSP3O3uJsB54FLwFAT3E+phPn4\nQPbssGvX00OGYbBp+BKqVJpHpUq5GTimFx+MzUP6ytC99FSzBe1zRd7g81yLOLDhGmtrtaLjjOvM\nmNWcZcvaa9BWZpPcFrcdcAFoBNwEDgNdgXPPnKMtbmVakZHg6gphYQDcWbuTk32/punNPRyct50J\npxpz1RfcRoNxaiu/vt38pYoxbG2wiY7/z65Hh54U3bKD7z7YSZ3W9vT7dC21axdg5sxmZM2qAVsl\nnzmnvNcAvADvx59/BdoSO3ArZVr29hAainHmDDbly5O7XSNyAwG5i7Nh8jnaNfqXqhcXUKn8P3Eu\nNVzssAmJinM88pdfsO/aNdax5wXth86ZybrjKLkcH+IcspdeH/sxf35LWrcuZZLHUyoxyQ3c+YB/\nn/l8A6iZzHsqlShv7wcs7TKfUc8cy3THiwn0g4XxXNA8G2y5H2/QBuIE7YS4PnpABY6yLMMbeEdl\nwNOzs7aylUUlt49b+0CU5QQFEb19B0dafEzhIlkYdXpOki4zHG3xu+CatDKKxt2wYFXX3vze+H9P\nP3vnLEEYdmRd8QNLfnpTg7ayuOS2uG8Cz66OUwBpdccyevTop+/d3d1xd3dPZrEqzfD3hzFjYM8e\noi9e4gYZqBZyJ9Yphxp3oOaOP557C5vwaLJeuZa08q4Exvp4tUw16v++mluFqnPyk6+5tvQP6gd4\nETVpMq1alXzhx1HqeTw8PPDw8EjSuckdnLRHBicbAreAf9DBSWUqfn7QuDFGpUr8kaUWyxYeZl3g\ngqdfXy5ek2Jeh1g9biBvjpge61KjoAs210MSL6OUM1x49NyvFxUYQIkBbYhcPJ0qZ//Er2M3is2f\nCFmyvPRjKZUU5p452RyYiWSY/ABM/M/3GrjVi7t/Hxo35kGVWrT3qkZr72186r8dx0A/i1Vh6xtf\nUu/sjzj7+fBnudZUX/MtGUuYfkEqpeKjU96Vdbl3D6NRI47ncKPDsZJsKbCHHAH3mcEHjPfuD4BH\nz664f//Ly5dhZwNRSftzee6zMZSZPfLly1LqJeiUd2U9fH0Jr+POL/75mexbkjMZlnP9USFaVT/A\nwEdTnp5WIPT0C986oms5+LiMfIgnaIfZxKyLfd4+N+O6zuNhUJgGbfXK0Ra3emUYPj74VanNUv9C\nFG9fj8YbvmVYrknk7VKcIePck19A1axwNG5Xy4lC9dnm3JKh5wcD0KbAEAYv/4S6dbVbRKUc7SpR\nr7wbRy8RWc+dPRkr0rB2Aey37WB6uYm0zLONSn+uIVPgA5OWF21jg61hcNmpBBldIce9S1y2y868\nfssZ+3UDXFx030eVsjRwq1eWYRjs6v0NDb8flqL1uJAuL4HbPKher0SK1kOpJ7SPW72Sbt4M5J0G\n31F86cwUKX9/lyHUzTKQ7z+YTeGrxzVoK6uR3Ak4Sr0wwzD46adTTBmwCg+Wki3MJ9b352rWoOjZ\nk6QLCjNL+VffGUjvOxW4dzGMxbvbULFibrOUo5S5aItbWZSPz0Pat1/JTxM24uk3nmx+MtF2Xd9x\n5PMMYeqjHWz+qyUh15vCunyJ3i8iW7Y4x6LS2xPpaBfr2PIcHzJrSSQzph+k+tZcNGhcjEOHPtag\nrayStriVxaxadYbPPtvCnKp36HhhGgB+GfPS4eBlXDIFMijPEspG3qLJ/eXY3qoKbW8mek+H+/ef\nvo/O4cD8m4MJNmrwWYYu2CMLSo1v+Bs1/9eAucOWYG9vy4EDH1GyZNyAr5S10MCtzCs8HP8rtxg/\naB0ORw9zx+dn2Bzz9dqWbZg7sjXOAVfJd/8BDvcfYHPPDkK8X7ioyGAbTp5qQZXF23AKl26W1XPO\nE3X/Dl07LmbMGHd6966Gra0lxuSVMh/NKlGm5e0NnTqBry/4+REd8ojIKANH4i6nuuGtd8lQBxwz\nh1LD8QT2LllgVyaYsTNJRRm5bbG5Ew3A/eL5yOYVu4W+c4MXA7/aToECmZg/vyUFCmRK9uMpZSma\nDqgsJzQUSpfm0ZfD+epgetJv/IP/RXsQijNOAXfZ8ukUgu6cpkjuCPbOLEW7YF/K/LEQmw+Cknb/\n96pBxwiip3qxfk1vbDwe8Y9fdf6q/Q4eZWNmPv7YZiTDDmVg+vSmdO1a/slfAqWsRkKB2xIMlbac\nGTXHMODpa17ZMU/fb2/S1jDA8OxZ3YhomMMwMtjGOtcAw7C3j3sMDKNvZSP6Tnbj7MPexvio8caQ\ngCNGsX/uGyv7Tn96zu7Zm4yiRWcZXbv+bty9+zCl/1Mo9dJIYL8DbXErkwkNCWdjhy95fceP5ItO\n4kzHIjnAPgIuJXD+wEYw+DKBTqX42bU6mY674rsymEZ7dlHu6H4pO1tuvmg6hQ1/3WPevJY0b645\n2cq6aVeJMrvTu05TvqGb6W9cPwfRDaO5apQnxCuEgpuv4pc+C95lmpB57yFKcYW/BkzjvUX36dix\nDBMmNCRDhnSmr4dSFqaBW5lNVFg4e9v3o/6W+QA8KFoRm2ve+BcsReGrcTfrfVHRmR05/nYlIp2K\nsNa1FpGvt+Tk9y58c/BtCueHflk7c+xmNIsWteG11/InuzylXhU65V2Zxt9/Q7FiULs2dO5MWFk3\n7JzSPQ0yidZ9AAAZPUlEQVTaW2oMIuh2AMuX7cAx5Cb7h3+a/CIHt2bol5v4rM+v3MzzOVe+uMYa\nj6qEVypOqStNKVarNMeO9dKgrdIUzeNWSRccDFmyYHz4ITYff8yzHRInMtag1pmfyPToLn3fqQFA\n3vHP2cy3dw+YH99W7LGFubpw6N+SVPeMZNX0CNqGjKfV7XkMyf0Oxx66sfev1pQund0ED6aUddGu\nEpV0Hh5Qvz6Bjq4ccSpGXcObja2mM/VCLcZVXIn7j6PjXuNqCw+jE7218akTt/73DstzFCZ9eCU2\n7C3IW3t2UXnFFqpc3U5g0Qr44Eh7/6Z8NrEdPXpU1Yk0KlVLqKtEW9wqcZGRsGgR9OkDwHb3j2l3\neDl/O9eh3povaRfqC8eec22dxrBlW4K3D/FqzprCdblvkxVPn9YcDMpD8zAYtqs8P1Swofz9w0wL\nKoVn7Q5s/64lefNmMPEDKmVdtI9bJSry08+5PGY2b+b8DIAWh1fybu5lVAk8wCerd7Jk2HgCi+fm\nVu2icS9OIGhHbytNeO4MrLuclzwT73B2ujPp/fKQfSzcWXSNSxENKH1wPg1delNl4Vj+WNNFg7ZS\naItbJeLYsdsE/LyPo5Xb8ENlG5gNXdy2MybXbC7YFaV+kQgeVYsm48Q7ZPQCSjvArz9DpbcSvbdt\n0/M4Al2b/kCkvT03289k0BqD3you4bV1g5hqvM7d9yazaWITMmbUFD+lntA+bhWv6GiDadMOMGXK\nATyz/UYmfx9s797B0YjgQY7cZPa9Y5Jyes5aRZXoLLQY9jHTuu9j9Lle+B89zZDc3RiyvA81aiS+\ntKtSqZHmcasXcuNGIO+/v5awsEiWzmtKsQr5eWjnyqqfT9G9i3SHPCyYnXAXO7Ke90nkbs/n0bkn\nE3POZ/zyuhQsnw2XE/tYGFmRqOEj6DekHg4OdonfRKlUSvO4VZKtXXueqlW/p379wiyZ0YIw9/YA\n/PHbNoZUKsKjHJJ+53r9XvxBO9PzV+ALy+zKrubtWdx3BABuq1fTN/AnqvnvJ/TQAUaW/ZR2Z1Yz\neHgDDdpKJUD7uBWcOkVw5hwMHH+EHTuusG5dF+6cDMO2bh38mnfl39PBPNz3J75v1k78XgEB8R7+\nY+RI+r4zhveCYO0oqFDwKG5RZ2mz9H2Wu7yG09xZTHuvuq7ip1QSaFeJwuednviu3sq37ScxYtqb\nLBjqSb9VrYioVons/+zCMSLCJOXcyFaah6H25M5tj9PdaxwOy0aerA4U6NSUdLNnmKQMpVIL7eNW\n8YqKimbKlAPs/eZXNj/4jocVX8fjTgFa+fwW59zVV6bRen00jv2/MG0lsmWDrFnh9GlwdEz8fKXS\nCO3jVnF4ez+gfv2lbN3qxXd/jwTA9eTBp0F7zuAJHH3gzTRjBuHZMtJhwtmXCtrHstama+d/8dx3\nkwlDN5Av6zimTd5HZESUrKB97x5cvKhBW6kXkJzA3Qk4A0QBVUxTHWUJv/ziSY3q39PDLZxNRQ+T\nrWKZOOe0aZCbW2u/oG+pmTjeD8Rm0Q/Pvd/uGe0ACMma9emxnZV7EWqbjqtLd/FB9zA6dF/NMa8Q\nDp38jEFfvIG9vbYZlHpZyekqKQ1EAwuAQTx/0rN2lbwiAgJCGfPuIooe3MzHzueItE3H5tCavHV3\n2Qvf6+53o+HHH3G+E4B/lRoUXL8dgJvFqpHl6hmul3Qnd9V89AhrwtGjt5g9uzmtWpU08RMplXqZ\na62S88m4VlmSry+XJ84jcO5iRtkGkOlRzG4zb3HhhW+3KHoRhRdup0aoIz6ZilD8cdA+mLMxw2pv\nZ/6kw+To24k3Nxan1mfZWLasHc7ODiZ7HKXSOlMMTu5GW9yvnpAQWLeO6OXLCftzD5tsSpFvaB+K\nVitPrtavv/RtV+z9hDJGFSrX+xiAzePnsulBV/rOrUvZYE8ObPak15B95M7typw5zSlVSpddVepl\nJCerZAeQO57jXwEbHr9PNHCPGjXq6Qd3d3fc3d0TKVa9lKgo2LULli+H9esJqViVqbeL8k+eaixa\n+iY3V/5N1aGt4lwW3bgRtjt2cmpEO9ymbcOmWzf4/vuXqkKrHP14b04POnUqqznZSr0ADw8PPDw8\nnn4eM2YMmDEdUFvcr4rKlcHWFt57j3VOFVkxZDkDqzyievg1Ig8fwzHiUdxrRo/mTPhJyk1YE+er\ntZ7jybtkGzWm7X167IcvJlA5xKDKd8OfHruXtzh776WnbD5H8v++GNcq5c3yeEqlJebO494NDAaO\nPud7DdyW8vrrPBo5luVTttFj97TEz2/bFv+8zpzLE0CtkVuI/vADbBcvSfCSYKfMbMvYkqploskQ\n6UvD4I5kypSOuXNbUrZsDtM8h1LKbIOT7YHZQHZgE3AcaJ6M+6lkul+4DNlaNKHH48/ePUYzfV9R\n2hf2ov7WsQCEuqTH8NiDc/WqAKxhCcFhfuS8+IDiiQRtgPGj79Dz7Sg2dx9Lob89GfzD67z9tpt2\niyhlQTpzMhUwDIN5847wSV/Z69EY/AXTy3zDlB9t+HWID+5t4humSLpOtQ5RxdmbfgHfcqZYdQ5t\nOE6jbA8o5pYfh00bEr+BUuqF6ZT3VCwgIJQPu6+j765JNAg4AUCnfgbet+D3qVEUKiS/VIXkzoPL\nJ31kEajZsyGJ649E2tgRUrQcNtkzc+BKOPdtXanbqTr5q5SAmjWhbFmzPZtSaVlCgdsSDGUef//9\nr1G6yDTjSq5ShgHGrXELjGA7F+PzLwONUB9/w5BJ5S/08lyzy5jW97DhZ5fFWNvnd8P3XqgxcOBW\nI0eOycb8+YeNqKjolH5spdIE4LktXp13bIUMw2DWrL/5uMV8jjycRhGfC5zoN5+KW3oSkTMfs3wH\nkC5XlpgLatSARPqgw7PnYMugtTSaVp+KR5bj1K45j+qWomKF7/D3D+X06U/o1aua7qyu1CtAA7eV\n8fd/RKdOqzg+dw0nIueQ3vcWO5uNoa1nL7YP9ybT7UvwwzPrivz9N+TMKe3peITlys2l90cS/iCU\nOXcasa/HHupeXUnXu7WZOHE/q1Z1YvHituTMmd5CT6iUSoz2cVuR48dv07HjKr4udJmuJ5dDQAC7\nSnRjbLXFrP3oBFnqJ77W1/fv9ye3bTRtfpyN99x1ePzgSdCDCD6IWIntHysJbdiUz4xm1Brfh969\nq+liUEqlEB2ctHKGYbBo0TFGDdvOnsonKfHvCYJzFebUmQh2vTWDYecGYOuxC4Ct9VvTbHfimR7R\nNrZcdipB9kzRZLlzCYBrdlk5W7I2VXavIFcuV7M+k1IqYboetxULDg7n/ffX8vP0nXgVXk8Jp2Du\nZS9O+r3bqBh5huHzKkrQHjWKXt7hvOF5KN77RLu6EvznAUbOiqZg9SCGTYkip895/t2wkyB7F5q4\nTeb2/pM0P7teg7ZSrzhtcb9qHj6EX36BDh0472vQseNvdC4YyP+2DIv//AIFYOtWjCJF6TR/I78P\n7ATA3y3eJFfObBRes5KoZSs48tMxcmxaRkaHcDKUKYRNjiz8GlSIjScfMSfzQbJ7ncTOTv8dV+pV\noV0l1uTGDShQgGhbO4INezIYYU+/epQuI85hgfKhbVtYtw66d5eBx7VreVi5KnR9G9dOHeDiRYxW\nrdj95VrGr8tFh4gtdHPYQsa9m+OW+eWXMHGihR5QKZUUGritSNiN26QrkPe530dny47tLyugSZOY\ng9OnQ+fOkPfxdSdOQOXK+DtmxzUiAAcjAvLmxbdIeX696EjGDI60qupKtlXPbKCwfTs0bmymp1JK\nvSgN3Fbi3z3HiW7ajEJhd2Md93XOS45Ht+Je0KIFbN4MdnaypOtzhJZx47yvwY0Qe8rWKU2RasWw\nyZ4dwsPh229h1SpZWTBdOlM/klLqJWngtgKH5m+gZp82sY5FdHiLRZcr0+zeGgpHXsPGzQ18feHk\nyQTv5VWvG4U3LeRhhMGECftYvPg4Q4bU5vPPa+LklJx1xZRSlqJZJa+w6MAgPCs2iRO0AcI2bKUy\n5yi0cCw2N27Ajh3SDeLvD717xzn/XoEKRH7aj8I7fmTBkpOUKjUHf/9HnD79CUOG1NagrVQqoX+T\nU4JhwP79RI4Yif1eD9z+8/XDgydxfb0iD7Pkp+bxpbFnq9+/jzF9BhG/reFs3gacjS5CZ99l2Dqn\nI/sb5djSrA+DKs4nb94M7NzZDTe3XJZ8MqWUBWiL25Lu3YOZM2WXmrp1sd/rEeeU6GYtWNJ3JwC5\n757F5txZ8PMDwyD8rBeB5Wuy7hcf2pbz4MisP+ncOBI7OxtCSpaj1b0m9B+4g8mTG7NjhwZtpVIr\n7eM2t+ho2LlT1g/57benh4Ns0sVK9XsiOGte0vvFMxD52PRa31NyfA+a1wG7jeuI6j8An8AoBkc2\npObYnvTpUx1HRzuzPIpSynJ0cDIldewIXl7g6gp//fVCl/oVdCPrdc+nn69O/40iA2SCTeiNO0SU\nLc/Y6DoMZx9Rl6+QLVdGk1ZdKZVydHAypXh5werVkDHj06C9IUvNuOctWQJffBHnsG90plifi/w2\nXZZntbHBqUAe/sz5Gl91zEXm/r01aCuVhmiL21zCw5/mRUcVL4n/9btMqDWU6R7/mbo+fjyEhMjP\nJLjlmI1HtukoaBeEw7Yt0K4dHDoERYua+gmUUilIu0osbf9+6NYNvL0JaNeZkI3bWPr+dIYu7YlN\nZOQL386ws6OP+1Q2XzSYPags7fo3k0k3+fPD0KHQp48ZHkIplZK0q8RS/P2hZ0+Zfl6yJADB67fy\n15gf+XJRd2z27En4+pYtY328cfEOH3+0jtzZJlHljaJcefe+BG2QLpjLlzVoK5UGaeA2BcOAFStk\n41xHRzh7Vtb+AC7PX0nHr9rJeaVLy89PP4UiRWSa+TOZJk+CPcDwXiupWHMJVR9d5t/ah+g5+wPs\nZ82Q+4eFySJTdpo9olRapF0lpvDPP7Lj+aJF8NFHnKrdgQoH1nDp5y2UqFQQvL1lcHLChJhrtm6V\nhaLOn5eAP3UqDB4MQO/0nalTMxdv3fXAIeyRBPoGDaB+ffjzT6hQIWWeUyllMdrHbU4XLkjwHTEC\nTp9O+nVubhAUJEH9P6KyZsOu1usSsJ+s2Ne0qQTur74yTb2VUq80DdzmEhkJ1aqBp6dMtIlPjRpQ\nqhQ4OcHChYnfs2tX+PprKFYs5tjcubB0qbTa7XWVAqXSAg3cphYZCcePw9698lq//oUuD50zjxVn\n7Wkzrz/ZjWAMOztsoqJkOny/frFPvnwZXnsN9u2L6SNXSqV6GrhN4dAhmbq+dy8cPAiFCkHdulC3\nLl4jpnPvTgCvPbwg59asKec/0bEjODjIlmQJmTlTBihLlIDChWWyjbs7dOgAAwaY68mUUq8gcwXu\nKUArIBy4DHQHAuI5z/oDd2SkBN7+/SUolyolGSHAyv/9SudxXWNazUng23MAOf43EHLmhN27oVkz\n+OQTyU65dElmXN66BTlySJfJ7t2yMJVSKs0wV+BuDPwJRAOTHh/7Mp7zrD9wg7R+ixWTrouX5D9q\nIlmGDYjZacYwZNCxUSMYMiT2yWFhcOUKFCwI6dMno+JKKWtkrgk4O5CgDXAIyJ+Me1mHlwzaUeXc\nYNAgsixdEDvzZNkyWep14MC4F6VLB2XKaNBWSsVhqt+/PwTi2T48FenSBX76SVrJAfH1CMVlVK0K\nH3+M3RlPmZo+ZQo0by7ZJT4+0sr+4QfNFFFKvZDEIsYOIHc8x78CNjx+Pxzp517xvJuMHj366Xt3\nd3fc3d1fpI6vhiJF4OpVCdy//57wudmyQZcu2CxYAEePyrEnKwC6ucGbb8KYMdC9+9O+cqVU2ubh\n4YGHh0eSzk1uVskHQA+gIRD6nHNe7T5uT0/InFkWbLJJ4D/HwoUyjd3ODlaujP1drVqygW+3bjB/\nvhx77TW4eRMCA+GDD6TFnSePfBccDPPmQd++4OxslsdSSlk3cw1ONgOmAfWAewmc9+oG7t27JdUu\nXTpZhrVChdivcuVi+pgnToyZtVi1qqxNsn8/fPRR/PfOmxc+/1wWncqSxTLPo5RKNcwVuC8BjoDf\n488HgU/iOe/VDNyPHkHFirJGSJs20ufs6QmnTkn3xorHPT/Fi0t63hNffw2ZMklQBsieXQYYn7V0\nqfSJOzpa5lmUUqlOQoE7OaNiJZJxbcobP14Cd5s28jlXLsmb9vGRyTOOjpK3vWpV7Ov+9z/56eIi\ni0utXQvffist9CFDoGHDhLtclFIqmdJmOoOnJyxYIK1rkAHHjRth+HAJyN9/L/3SrVrJDutBQXLO\nb79JQB83ThaHeuMNaN9eZlSWL5+ST6SUSkPSXuCOioIePST45skDHh7Sdx0UJK3w1q2lq8TZWQJz\nnTox15YqBTt2wOTJ0nd99mzMgKNSSllI2gvc8+fL9PUqVWTW4qVLMHasrMr3ZGOCNWtk0NLGRgL9\nmjUwbRrcvStrhnTvrhNjlFIpJm0tMnXjBhQoIBsXPHgga2h/9FHcQcQyZWQp1dOnZeGnXLlg0CDZ\nmFd3nVFKWYC5Biety7VrsuIewPvvyyYFLi5xz9u9WzZG6NQJ6tWT2ZK1alm0qkoplZDU3+K+e1f6\nrmfPls937kgL+nmeZIRcuiSpgEoplQLS7i7vt25BpUqytoi9vayl/SRoBwRIJsmYMZLT/axfftGg\nrZR6ZaXewB0ZCW+/DX36SIbIm29KsB48WLYby5pVMkj+/VcGK0FS/nLnlo0PlFLqFZV6+7gHDYI9\neyAkBA4flmP37skCUOfPy+SbuXNlTZEnbGzg9u2Uqa9SSiVR6mlxP+n6GDxYAvDs2bIF2OHDso+j\nr6/Mbty1S/KwDx+OHbSVUspKWEeLOzpaUvPKlo1ZuzooSPqsPTzkde6cbCtWpox8v2kTHDkiAbpm\nTWlhN20KZ87IlmFKKWWlrCOrxNtb1sPOnBkaNIAmTSRTpGBBCcbu7lCjhgT1pk0lfe/ttyVYlysn\nk2jmzdO0PqWU1bD+PO6CBSFjRllG9dgx2LZNBhVv35aNCB48gIgI2WEmMlIWgmrYUI41aSIzI3Wl\nPqVUKmEdLW6QlvbQodKiBhlk/PVXCeTbt0vfNcjmBOXKwaRJMnmmbt3kl62UUhZmrvW4k8o0gXvI\nEGl1jxghn9OlkwFJJycZeCxVCt59F2xtpUV+7560tufM0Y0MlFJWx/q7SgCqV4eff5b3YWEyYJku\nnfx87z1ZrW/SJJkZmS6dLCZlb69dJEqpVMd60gGrV4/Jxw4KggwZJO1vyhTZ1/GLL2Tjg7JlJXf7\n1CnZiUZX8VNKpTLW0+IuVEha2rduyc8MGeDAAZg+XbpRKlSAFi0kLTChtUiUUsrKWU/gtrGJaXUX\nKSIZI126yI7pS5bI7jS1a6d0LZVSyuysp6sEJHAfOSJdJbdvywDkyJGSWaJBWymVRlhX4K5WTVrc\ngYHy+exZ6SZ5skiUUkqlAdYZuB8+lK3FnmyMoJRSaYh1Be68eSVv+9QpGZxUSqk0yLoCN0g/9+7d\nGriVUmmWdQbuQ4c0cCul0izrC9zVqslCUhkzpnRNlFIqRVhn4AZtcSul0izrC9zZsskEHA3cSqk0\nKjmB+2vgJHAC+BMoYJIaJcV778XsdJOCPDw8UroKZpWany81Pxuk7udLzc8GSXu+5ATuyUBFoBKw\nFhiVjHu9mNGjZZAyhekfIOuVmp8NUvfzpeZnA/MH7qBn3rsC95JxL6WUUkmU3EWmxgPdgBBAt0xX\nSikLSGwHnB1A7niOfwVseObzl0ApoHs853oBxV6qdkoplXadRLqizaYgcNqcBSillBLJ6eMu8cz7\ntsDxZNZFKaWUmf0OeCLpgKuBnClbHaWUUkoppdKwlJusY35TgHPI8/0BZErZ6phcJ+AMEAVUSeG6\nmEoz4DxwCRiawnUxtcWAD/LbcGpTANiN/Hk8DXyestUxKSfgEBIjzwITU7Y64tn56Z8Bi1KqImbQ\nmJixgkmPX6lJaaAk8hcmNQRuOyTTqTDggPxFSflpuKZTB6hM6gzcuYnJsnAFLpC6/t+5PP5pD/wN\nvPG8Ey21VklqnqyzA4h+/P4QkD8F62IO54GLKV0JE6qBBG5vIAL4FRlcTy32Af4pXQkzuYP8Qwvw\nEPlNN2/KVcfkQh7/dEQaGH7PO9GSi0yNB64D75P6WqVPfAhsTulKqATlA/595vONx8eUdSmM/GZx\nKIXrYUq2yD9MPshvuGcTOtFUdiC/nv331frx98ORfO8lwAwTlmsJiT0byPOFAyssXrvkS8rzpRZG\nSldAJZsrktXWD2l5pxbRSFdQfqAu4P68E5M75f1ZjZN43gqsr1Wa2LN9ALQAGpq/KmaR1P93qcFN\nYg+OF0Ba3co6OCDpx8uRxe1SowBgE1AN8EjJijw7Wecz4KeUqogZNENGubOndEXMbDdQNaUrYQL2\nwGXkV21HUt/gJMizpcbBSRtgGdb3G3tSZAcyP37vDOzlFWgIpubJOpeAa8jM0ePA3JStjsm1R/qE\nHyGDQ1tStjom0RzJSPAChqVwXUztF+AWEIb8f4tv/SBr9QbSnXCCmL9vzVK0RqbjBhxDnu0U8EXK\nVkcppZRSSimllFJKKaWUUkoppZRSSimllFJKKaXUs/4PLfCkwrW/774AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def cond_prob (i, x):\n", " return np.random.normal(loc=0.99*x[i], scale=np.sqrt(1-0.99**2))\n", "\n", "sampling = GibbsSampling([lambda x: cond_prob(1,x), lambda x: cond_prob(0,x)], [1.5,0])\n", "\n", "samples = [sampling.sample() for _ in range(1000)]\n", "plt.plot(map(lambda x: x[0], samples), map(lambda x: x[1], samples), 'r-')\n", "plt.contour(X, Y, p2d.pdf(XY));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Gibbs sampling is vastly popular as\n", "\n", "* it has no free parameters which need to be tuned\n", "* appears efficient as it moves on every sample, i.e. never rejects\n", "\n", "Gibbs sampling is most efficient if all coordinates are almost independent. In the case of high dependencies the conditional distributions are sharply peaked and Gibbs sampling moves slowly across the space. The above example nicely illustrates this effect as the length scale of each move is restricted to the standard deviation $\\sqrt{1 - 0.99^2} = 0.141$ of the conditional distribution while the actual distribution has a much larger extend." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Example:** Gibbs sampling for Gaussian mixture model\n", "\n", "Many machine learning applications require that we can model probability densities, including ones with complicated shapes and in high dimensions. Parametric models, such as Gaussian distributions, are well studied and understood, but only provide good models if its assumptions of unimodality and rather light tails are matched by the data. Mixture models provide a principled and flexible way to extent the type of densities that can be modeled. Here, the density is assumed to be composed of different (mixture) components each of which can be described by a simple parametric form. Especially Gaussian mixture models, not least due to efficient estimation algorithms, are vastly popular in machine learning. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Gaussian mixture model describes a density $p(\\mathbf{x})$ as follows:\n", "$$ p(\\mathbf{x}) = \\sum_{k=1}^K \\pi_k \\mathcal{N}(\\mathbf{x} | \\mathbf{\\mu}_k, \\mathbf{Sigma}_k) $$\n", "Thus, it models the density as a sum/mixture of $K$ Gaussian components. Here, $(\\pi_k)_{k=1}^K$ denotes the probability of each component and $\\mathbf{\\mu}_k, \\mathbf{\\Sigma}_k$ are the component means and covariances respectively.\n", "\n", "Introducing a latent variable $z \\in \\{1,\\ldots,K\\}$ we can also think of a datum $\\mathbf{x}$ as being generated by the following process:\n", "$$ \\begin{array}{clc} z & \\sim & \\pi \\\\ \\mathbf{x} & \\sim & \\mathcal{N}(\\mathbf{\\mu}_z, \\mathbf{\\Sigma}_z) \\end{array} $$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Thus, to estimate parameters of a Gaussian mixture model from a data set $\\mathcal{X} = (\\mathbf{x}_n)_{n=1}^N$, we need to\n", "\n", "* infer the hidden component assignments $(z_n)_{n=1}^K$\n", "* infer the component means $(\\mathbf{\\mu}_k)_{k=1}^K$ and covariances $(\\mathbf{\\Sigma}_k)_{k=1}^K$\n", "\n", "As a Bayesian, we do this by computing the corresponding posterior distributions. Unfortunately, since the assignments $(z_n)$ are unobserved, the likelihood needs to marginalize over all possible component assignments leading to an intractable sum over exponentially many terms. Thus, in practice we need to approximate the posterior, and many such algorithms have been developed for the Gaussian mixture model:\n", "\n", "* **EM algorithm**: The *expectation-maximization* algorithm computes the maximum likelihood parameters of the mixture components\n", "* **Variational Bayes**: Similar to the EM algorithm, but computing an approximate posterior distribution over the parameters\n", "* **Gibbs sampling**: Sampling algorithm which explores the posterior distribution, over assignments and parameters.\n", "\n", "Here, we will implement Gibbs sampling. Thus, we need to be able to compute conditional posterior distributions. In the Gaussian mixture model this requires that we use conjugate priors leading to the following generative model:\n", "$$ \\begin{array}{lcl} \\pi & \\sim & \\mathcal{D}irichlet(\\mathbf{\\alpha}) \\\\\n", " \\mathbf{\\mu}_, \\mathbf{\\Lambda}_k & \\sim & \\mathcal{W}ishart-\\mathcal{N}ormal(\\mathbf{\\mu}_0, \\nu, \\kappa, \\mathbf{S}_0) \\quad \\forall k = 1,\\ldots,N \\\\\n", " z_n & \\sim & \\pi \\\\\n", " \\mathbf{x}_n & \\sim & \\mathcal{N}(\\mathbf{\\mu}_{z_n}, \\mathbf{\\Lambda}^{-1}_{z_n}) \\quad \\forall n = 1,\\ldots,N \n", " \\end{array} $$\n", "\n", "**Exercise:** Show that the Dirichlet distribution is the conjugate prior for the categorical variable $Z$.\n", "\n", "The conjugate prior for a multi-variate Gaussian with unknown mean and precision matrix is the *Normal-Wishart* distribution. For the math, e.g. posterior parameters, we refer to K. Murphy, *Conjugate Bayesian analysis of the Gaussian distribution*. Here, we use these results to implement the *collapsed Gibbs sampler*, i.e. with the mixture parameters $\\mathbf{\\mu}_k, \\mathbf{\\Lambda}_k$ integrated out analytically." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Thus, we approximate the marginal posterior $p(z_1,\\ldots,z_N | \\mathcal{X})$ by sampling from its conditional distributions:\n", "$$ \\begin{array}{lcl} p(z_i | \\mathcal{X}, \\mathbf{z}_{-i}) & \\propto & p(z_i | \\mathbf{z}_{-i}) p(\\mathcal{X} | z_i, \\mathbf{z}_{-i}) \\\\\n", "& = & p(z_i | \\mathbf{z}_{-i}) p(\\mathbf{x}_i | \\mathcal{X}_{-i}, z_i, \\mathbf{z}_{-i}) p(\\mathcal{X}_{-i} | z_i, \\mathbf{z}_{-i}) \\\\\n", "& \\propto & p(z_i | \\mathbf{z}_{-i}) p(x_i | \\mathcal{X}_{-i}, z_i, \\mathbf{z}_{-i}) \\end{array}$$\n", "where $_{-i}$ denotes a vector without the $i$th component and we have used conditional independence in the model, namely that $p(\\mathcal{X}_{-i} | z_i, \\mathbf{z}_{-i}) = p(\\mathcal{X}_{-i} | \\mathbf{z}_{-i})$.\n", "\n", "Thanks to conjugacy both terms can be computed analytically, being the predictive distributions from the Dirichlet and Normal-Wishart posteriors. For the second term note that $p(\\mathbf{x}_i | \\mathcal{X}_{-i}, z_i, \\mathbf{z}_{-i})$ only depends on the data points that are assigned to the same component as $z_i$. In order to compute the corresponding distributions for each component, we keep track of the (unnormalized) empirical means, i.e. $\\sum_{n=1, z_n = k}^K \\mathbf{x}_n$, and covariance terms, i.e. $\\sum_{n=1, z_n = k}^K \\mathbf{x}_n \\mathbf{x}_n^T$.\n", "\n", "A step of the collapsed Gibbs sampler then works as follows:\n", "\n", "* For $i = 1:N$\n", " 1. Remove $\\mathbf{x}_i$ from component $z_i$\n", " 2. For $k = 1:K$\n", " \n", " Compute $p(z_i=k | \\mathcal{X}, \\mathbf{z}_{-i}) \\propto p(z_i=k | \\mathbf{z}_{-i}) p(x_i | \\mathcal{X}_{-i}, z_i=k, \\mathbf{z}_{-i})$\n", " \n", " 3. Draw new assignment $z'_i$ from this distribution (after normalization) \n", " 4. and add $\\mathbf{x}_i$ to component $z'_i$" ] }, { "cell_type": "code", "execution_count": 52, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from __future__ import division\n", "import numpy as np\n", "from matplotlib import pyplot as plt\n", "from scipy.misc import logsumexp\n", "\n", "class GaussStatistics (object):\n", " \"\"\"\n", " Stores statistics of data for Gaussian model\n", " \"\"\"\n", " def __init__(self, X):\n", " N, D = X.shape\n", " # Store number of samples\n", " self.N = N\n", " # sum of samples\n", " self.x = np.reshape(np.sum(X, axis=0), (D,1))\n", " # and sum of outer squares\n", " self.xxT = np.dot(X.T, X)\n", "\n", "# Computations for conjugate priors\n", "def posterior_NormalWishart (stats, mu0, nu, kappa, S0):\n", " \"\"\"\n", " Computes posterior parameters of Normal-Wishart\n", "\n", " see Murphy, page 18\n", " \"\"\"\n", " nuN = nu + stats.N\n", " kappaN = kappa + stats.N\n", " # Note: Formula adapted to statistics\n", " muN = (kappa*mu0 + stats.x)/kappaN\n", " SN = S0 + stats.xxT + kappa*np.dot(mu0, mu0.T) - kappaN*np.dot(muN, muN.T)\n", " return muN, nuN, kappaN, SN\n", "\n", "def predictive_NormalWishart (x, muN, nuN, kappaN, SN):\n", " \"\"\"\n", " Predictive distribution of Normal-Wishart\n", "\n", " see Murphy, page 19\n", " \"\"\"\n", " D, k = x.shape\n", " return mvt_log_pdf(x, muN, nuN-D+1, SN*(kappaN+1)/(kappaN*(nuN-D+1)))\n", "\n", "def mvt_log_pdf (x, mu, nu, Sigma):\n", " \"\"\"\n", " Compute log density of multi-variate Student t distribution\n", " \"\"\"\n", " D, k = x.shape\n", " return np.math.lgamma((nu + D)/2.) - np.math.lgamma(nu/2.) - D/2.*np.log(nu*np.pi) \\\n", " - 1/2.*np.linalg.slogdet(Sigma)[1] \\\n", " - ((nu+D)/2.)*np.log(1 + 1/nu*np.dot((x - mu).T, np.linalg.solve(Sigma, x - mu)))" ] }, { "cell_type": "code", "execution_count": 53, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Finally, we can implement the collapsed Gibbs sampler\n", "def gmm_Gibbs_step (X, z, comp_stats, alpha, mu0, nu, kappa, S0):\n", " \"\"\"\n", " Run single Gibbs step, i.e. update each assignment z_i\n", " \n", " comp_stats contains GaussianStatistics for each mixture component\n", " \"\"\"\n", " N, D = X.shape\n", " assert len(z) == N, 'each data point must be assigned to exactly one mixture component'\n", " K = len(comp_stats)\n", " assert np.min(z) >= 0 and np.max(z) < K, 'assignment and components do not match'\n", "\n", " for i in range(len(z)):\n", " # Current assignment\n", " zi = z[i]\n", "\n", " # Fetch data point and remove it from statistics\n", " xi = np.reshape(X[i,:], (D,1))\n", " Ni = comp_stats[zi].N\n", " comp_stats[zi].N = Ni - 1\n", " comp_stats[zi].x = comp_stats[zi].x - xi\n", " comp_stats[zi].xxT = comp_stats[zi].xxT - np.dot(xi, xi.T)\n", " \n", " # Compute conditional probabilities for zi\n", " log_cond_probs = np.zeros(K)\n", " for k in range(K):\n", " post_i = posterior_NormalWishart (comp_stats[k], mu0, nu, kappa, S0)\n", " log_pred_xi = predictive_NormalWishart(xi, *post_i)\n", " # Predictive cond. probability from Dirichlet\n", " log_pred_diri = np.log((comp_stats[k].N + alpha)/(N + K*alpha - 1)) # - 1 ???\n", " log_cond_probs[k] = log_pred_diri + log_pred_xi\n", "\n", " # if i == 0:\n", " # print comp_stats[k].N, log_pred_diri, log_pred_xi\n", "\n", " # Normalize probs in log space\n", " log_cond_probs -= logsumexp(log_cond_probs)\n", " \n", " # Sample new assignment from this distribution\n", " zi_new = np.random.choice(K, p=np.exp(log_cond_probs))\n", " z[i] = zi_new\n", " # and add xi to its statistics\n", " Ni_new = comp_stats[zi_new].N\n", " comp_stats[zi_new].N = Ni_new + 1\n", " comp_stats[zi_new].x = comp_stats[zi_new].x + xi\n", " comp_stats[zi_new].xxT = comp_stats[zi_new].xxT + np.dot(xi, xi.T)\n", "\n", " # Finally, return new assignments and statistics\n", " return z, comp_stats" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, we test the whole thing on some demo data and with nice graphics $\\ldots$\n", "\n", "Note that there are different choices of how the initial assignment can be chosen:\n", "\n", "* All in one component, i.e. $z_n = 1$ for all $n$\n", "* At random, i.e. each data point is assigned to a random component $1, \\ldots, K$\n", "* Using some clustering algorithm, e.g. K-means" ] }, { "cell_type": "code", "execution_count": 54, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def gmm_Gibbs_demo (X, K, alpha, mu0, nu, kappa, S0, iter=100):\n", " \"\"\"\n", " Run Gibbs sampler on Gaussian mixture model with K components and\n", " the given hyperparameters\n", "\n", " Plot fancy stuff on each iteration\n", " \"\"\"\n", " N, D = X.shape\n", " assert D == 2, 'cannot plot data with more than 2 dimensions'\n", " # Draw initial assignment at random\n", " z = np.random.choice(K, size=N)\n", " z = np.array([0 for _ in range(N)])\n", " # z = np.concatenate(([0 for _ in range(100)], [1 for _ in range(100)], [2 for _ in range(30)]))\n", " # and initialize component statistics\n", " comp_stats = [GaussStatistics(X[z == k,:]) for k in range(K)]\n", "\n", " fig = plt.figure(1)\n", " # Plot 1-stdev ellipse for each mixture component\n", " for k in range(K):\n", " muN, nuN, kappaN, SN = posterior_NormalWishart (comp_stats[k], mu0, nu, kappa, S0)\n", " plot_cov_ellipse(SN/nuN, muN, nstd=1, alpha=0.1)\n", " plt.scatter(muN[0], muN[1], marker='x', s=50)\n", " plt.scatter(X[:,0], X[:,1], c=z)\n", "\n", " # Plot predictive density\n", " def pred (x, comp_stats, post_K):\n", " K = len(comp_stats)\n", " return np.exp(logsumexp([np.log((comp_stats[k].N + alpha)/(N + alpha*K)) + predictive_NormalWishart(x, *post_K[k]) \\\n", " for k in range(K)]))\n", " xmin, xmax, ymin, ymax = plt.axis()\n", " fun_contour(np.linspace(xmin, xmax, 50), np.linspace(ymin, ymax, 50), \\\n", " lambda x,y: pred(np.array([[x],[y]]), comp_stats, \\\n", " [posterior_NormalWishart(comp_stats[k], mu0, nu, kappa, S0) for k in range(K)]), \\\n", " cmap='gray', alpha=0.2)\n", " \n", " plt.pause(1)\n", " \n", " for i in range(iter):\n", " z, comp_stats = gmm_Gibbs_step (X, z, comp_stats, alpha, mu0, nu, kappa, S0)\n", "\n", " if i % 10 == 0:\n", " print i\n", " fig.clear()\n", " for k in range(K):\n", " muN, nuN, kappaN, SN = posterior_NormalWishart (comp_stats[k], mu0, nu, kappa, S0)\n", " plot_cov_ellipse(SN/nuN, muN, nstd=1, alpha=0.1)\n", " # Plot large mark for posterior mean\n", " plt.scatter(muN[0], muN[1], marker='x', s=50)\n", " fun_contour(np.linspace(xmin, xmax, 50), np.linspace(ymin, ymax, 50), \\\n", " lambda x,y: pred(np.array([[x],[y]]), comp_stats, \\\n", " [posterior_NormalWishart(comp_stats[k], mu0, nu, kappa, S0) for k in range(K)]), \\\n", " cmap='gray', alpha=0.2)\n", "\n", " plt.scatter(X[:,0], X[:,1], c=z)\n", " fig.canvas.draw()\n", "\n", " return z, comp_stats\n", "\n", "# Helper function from: https://github.com/joferkington/oost_paper_code/blob/master/error_ellipse.py\n", "from matplotlib.patches import Ellipse\n", "\n", "def plot_cov_ellipse(cov, pos, nstd=2, ax=None, **kwargs):\n", " \"\"\"\n", " Plots an `nstd` sigma error ellipse based on the specified covariance\n", " matrix (`cov`). Additional keyword arguments are passed on to the \n", " ellipse patch artist.\n", " Parameters\n", " ----------\n", " cov : The 2x2 covariance matrix to base the ellipse on\n", " pos : The location of the center of the ellipse. Expects a 2-element\n", " sequence of [x0, y0].\n", " nstd : The radius of the ellipse in numbers of standard deviations.\n", " Defaults to 2 standard deviations.\n", " ax : The axis that the ellipse will be plotted on. Defaults to the \n", " current axis.\n", " Additional keyword arguments are pass on to the ellipse patch.\n", " Returns\n", " -------\n", " A matplotlib ellipse artist\n", " \"\"\"\n", " def eigsorted(cov):\n", " vals, vecs = np.linalg.eigh(cov)\n", " order = vals.argsort()[::-1]\n", " return vals[order], vecs[:,order]\n", "\n", " if ax is None:\n", " ax = plt.gca()\n", "\n", " vals, vecs = eigsorted(cov)\n", " theta = np.degrees(np.arctan2(*vecs[:,0][::-1]))\n", "\n", " # Width and height are \"full\" widths, not radius\n", " width, height = 2 * nstd * np.sqrt(vals)\n", " ellip = Ellipse(xy=pos, width=width, height=height, angle=theta, **kwargs)\n", "\n", " ax.add_artist(ellip)\n", " return ellip\n", "\n", "# Helper function to plot nice, smooth contours of a function\n", "def fun_contour (xr, yr, f, **im_kargs):\n", " \"\"\"\n", " Evaluates function over a regular grid xr x yr and plots smooth,\n", " colored contours on current axes\n", " \"\"\"\n", " xg, yg = np.meshgrid(xr, yr)\n", " zg = np.reshape([f(x,y) for x,y in zip(xg.ravel(), yg.ravel())], xg.shape)\n", " plt.imshow(zg, interpolation='bilinear', vmin=zg.min(), vmax=zg.max(), \\\n", " extent = [xg.min(), xg.max(), yg.min(), yg.max()], \\\n", " origin='lower', **im_kargs)\n", " # plt.contour(xg, yg, zg, 42, **im_kargs)" ] }, { "cell_type": "code", "execution_count": 57, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0\n", "10\n", "20\n", "30\n", "40\n", "50\n", "60\n", "70\n", "80\n", "90\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAFSCAYAAADo5khvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd8HMX5/997TV2yLUu25G7ZcpEt94IBY4MxvdnENr3k\nS+BLCCQkIb+EECD5AmmQUBJCQigOxMFgBxtMc6Pb4I57N26yLFuSrXanK/v743Rl291ekawy79fr\npNvZ2dnZvd35zPM8s7OSLMsyAoFAIBCosJzpCggEAoGgdSIEQiAQCAS6CIEQCAQCgS5CIAQCgUCg\nixAIgUAgEOgiBEIgEAgEuthacmcVFTUJbd+5czpVVfVJqk3L0lbr3lbrDW237m213tB2695W6w2J\n1z0vL8twXZuyIGw265muQty01bq31XpD2617W603tN26t9V6Q/PWvU0JhEAgEAhaDiEQAoFAINBF\nCIRAIBAIdBECIRAIBAJdhEAIBAKBQBchEAKBQCDQRQiEQCAQCHRJ+EG5F154gcWLF2OxWCguLuaJ\nJ57A4XAko24CgUAgOIMkZEEcPnyY+fPn89///pd33nkHr9fLkiVLklU3gUAgEJxBErIgMjMzsdls\nNDQ0YLFYcDqddOvWLVl1EwgEAsEZJCGB6NSpE7fffjtTpkwhNTWVc845h0mTJiWrbgKBQCA4gyQk\nEAcPHuTVV19lxYoVZGVlcd9997F48WKuvPLKZNVPw6ZNG5qt7OZk2rTJbbLubbXe0Hbr3lbrDc1Z\nd6kZygwxbdq5bNq0MUouuVnrEC/Tpk1utrIlWZbjPur33nuPL774gsceewyAt99+m02bNvHwww/r\n5vd4vG16UiyBQCDoSCRkQfTv35+//vWvOJ1OUlJSWLVqFaWlpYb5E51ONy8vi2XLPk2ojDPFtGmT\n22Td22q9oe3Wva3WG5qz7s1vQSxb9lmUXK3XgkjkVQqRpvtOSCAGDx7MVVddxcyZM7FYLAwdOpRZ\ns2YlUqRAIBCoaF5xEBiT8HMQd9xxB3fccUcy6iIQCDo8rVkMJFqrFdFctOgb5QQCgUBJaxYEPQL1\n7RhCIabaEAgEZ4i2Jg7htOW6m0cIhEAgaGEk2kcD2x6OITJCIAQCgSBu2rdICIEQCAQtRHuxHNS0\nx2PyI4LUAoEgySS3wZSaof2N//FgI9rnCCchEAKBIEkkryVvDlGIVr5eWmxC0v5GOAkXk0AgSAJt\nRxxiIb66tKIDSBAhEAKBIAGSG1doTeIQoCOLhBAIgUAQJ8kVhtYoDgE6qkiIGIRAIIiDxBu/1iwI\nekhSPMHtth28FgIhEAhiJPaWva2JgREdTSSEi0kgEMRAxxWHAPG5w9rmMyDCghAIBCZpDeKQSIHJ\n7cV3BGtCCIRAIIhCWxcGdRnJa6Dbu0gIgRAIBBE4k+LQXC4ZdbmJNdbxiUTbQAiEQCAwoKV95mfK\nR5+cEVntUSSEQAgEAh1aynJoOQslegOemAuqPYqEEAiBQKCiJRpt8xsky2UVXk7khjx+oWhvIiEE\nQiAQhNHc4mAuc0tO1mfcoMcXTG5PIiEEQiAQcKathjP5rETkfbedEUfNgRAIgaDDE1vrnCxhaG0P\n0Bn3/GMXiehWRNuYGlwIhEDQoWm+WEBdXT179+6goKCAqqoqsrO70L17QVxlhdXCZL5kB5qbq0Fv\n3RaKEAiBoMPSPOIgy7BjxzbKy4/w6KOPYLH4Z/T561//RkFBQbPVMfp25hriyL1/8w26+VhE6xUJ\nMReTQNAhaT7LYdOmdcyYcQXjxo0NigNAbm7nGOrWHHMXmS8zelwi2bTOuZqEQAgEHY7mEAd/AyfL\nMoWF3SksLMTtdlNfXw+ALMuUlZVH3T5aINvsx0xdo+aKWFbrGI3V3AgXk0DQoUi2OCgzSZKEx+MG\n4KqrrmLRokUcO1aO2+2hb98BpuuTaMOq3j6R4HMyg9fRaV3uJmFBCAQdhuZw2Wg5daqOr776Go/H\ng8ViIyMjh9LS0WRnZ4dtp+3Fm7cA4qipYbmt07XTWhAWhEAg0GDOTaO/zdChw9i9ez9ffvkVvXv3\npXfvPlG3aSkC+4v1QbZErIi2/OCcEAiBQKAgEXEIUFjYg8LCHqbzx7Iv8xi3yvqNduTGvmVdTa0D\nIRACgSBIMsQhUv54yo+f8LK0Dbi+NdF+G/t4EAIhEHQIzI3aibWMWEb5xDsiKB43VCzWgbb8yHk7\nkhUhBEIg6ODEGwcw2+DHKgzJiEsYWwdgriFvnw1+rIhRTAJBuyfyswXxlGF2O7Mjh5prBJN+mWaF\nKXkC1lafhxAWhEDQrmlJcZBiyhNbHbTb+ol32gt9C6EtjzhqDoRACATtlmR0W82WEZs4JC9YbV40\ntG4nM26k5Lma2qL4CBeTQNAuSUbg12ygOVphSpdSZAFJxoNrkctQ7j++Ibht1WUUK8KCEAjaHc3T\nekUTh0TXO51OHA5HcIK/WBth/ZFLwbXKNZLakiDK+jbW9U8SCVsQp0+f5t577+WSSy7h0ksvZePG\njcmol0AgiItkDGfVlpNo419fX0djY6Pu+vr6OtatW43XW8+hQ3vYt2933IHgyFNqaPPHULqJ7duf\nWZGwBfHYY48xefJknnnmGTweDw0NDcmol0AgiJnWJw4ej4cNG75m5MgRnDp1guPHT1JSUqrIuWvX\ndn74w3uxWq0AzJ37L7xeb3DZ5/PR6Hbj88n4ZAkJCUmSg/u2WS3Y7XbdOifyEFxbjBkkm4QEoqam\nhrVr1/K73/3OX5jNRlZWVlIqJhAIYiFZvdfkiYMkwdatm/jBD75PWloaACtXruTEiQq6ds0LbpuR\nkR4UA4CCgu5UVp0iNS0drw8kyYLFakOSJJD8TXx4u93Y6MVXX8uunduQ8FFcPIhOnTor6mfkLorN\nlaRdb3Z0VFslIRfT4cOH6dKlCz//+c+55ppr+OUvfyksCIGgRTEf1E32NBeRXCyBdXa7PSgOAEVF\nRRw5coTdu3dSU3MaALfby/79B3C5GqmsrmHN+q2kZuQgWR3Y7A6sNr84GIWwLRYLm7duZs4NN3Lr\n/3yPw2VHKS8/FqGukUQwkWG47Q9JluM3ojZv3sycOXOYN28epaWlPPbYY2RmZnLffffp5vd4vNhs\nVt11AoGg/bFx40Z8Ph+jR49GlmVefPFFGhoayM/P59ixYwwZMoSpU8/n3fc+xOlqxO3xMnny5LCp\nwaOze9durDYr/fv3D6a99957XHHZRXTKzmyOw+owJORi6t69O926daO01O9TvOiii/jHP/5hmL+q\nqj6R3ZGXl8WyZZ8mVMaZYtq0yW2y7m213tB2626+3ua7t7FaD7G5lkILU6dO4uOPv1Ss27t3F2vX\nrsflclFefoyZM2cwatQoZFnmz08/x6layMjsTGqTobF9257oBxTG4SOHKB0+NLgsy1B2tJwvVm3C\n53GRlZHqd0+FrW/6FkybMmUSK1eG11vZb1Z2o7V96mjr9fMZ5jKTKci0aZOpqKiJaZtw8vKMwwIJ\nuZjy8vIoKChg//79AKxatYoBA/TeGiUQCM4ULSUORi6nAQOKGTSohNLS0WRkZDBq1Cjq6huoqKzh\n4suupPx4uVknGXpOph49erJ02XKqq6vxeX3M+/c8evT0v4PCYkuhtt4ZoTxBJBIexfTQQw/xk5/8\nBLfbTe/evXniiSeSUS+BQNAiNK84qNNPnqzi4OEyMrI6YbXZ2bVrFzk5ORHrFLneMhISo0eP47//\nfQev10O/fkVkZmSG+uEWB7X1TjLTU4N10uvJi1FLWhIWiMGDB7NgwYJk1EUgECSZWIKszS0OHo+X\nc867kGf/8ndGlA6ntq6OhgYXg4qHaMqModbBv0OHlGjWyIAkSfhkGw1OF2mpKTrbC1UwQky1IRB0\nWBJ9Ctq8aLjdHuoaPDgcaYwbN5H9Bw7hdLopLh6Mxm0kmfxo6qrzMFvTf4vFQqNbJjAmp2VfXNR2\nEQIhELRTYnnTW6T15iwE4/wNzkbqXT5sdjtVVZVUVJTxve/dzvnnn8emTRuURcTSLhsKhX6K1e6g\nwemKXKQpIew4CIEQCDo48Uw5YbSNz+fj5MmTuFz+htjpbMTtlbDZ/N7so2VHmDNnNjabjR49Chk1\nagRVVScT67CbEIkAjZ6QFWEmf0dHCIRA0CaJ3Kg1n/VgnL+urpa5c+fSvXsXTp06wdatm3F5UDwl\nbbUqm5yUFAden8+gTFO+JWV2nWMIX7LZU3A2iVdHtQpiQQiEQNDOiEUcErceQml79uzk5ptvZtKk\nSVx11VU40jLwqXrrnTvnsWTJ+4B/qp4vv1xN19yuqvLMCFgEoYiCxyuC0mYR0323CeLt6ogbQWBM\nvKOWjEhLS8ViseDxeKiuaaCgsJCaGieO4ER6Et3yu3Gy8gT/fPFlkCyMGjU2bEf6+zYeehoa6qpJ\nlhVfFPh8Zu8nMcJJCESrJlEb2OAGErRbEnMtxVq+2qKwsH37dvK69UCSrGzatJkRI0Yr9ytBbm5X\ncjVWg9E+9I8poQnyLBY8Hjc2m113tXgeIoQQiFZHczhGw8sUV37bJ9kNvbbcyEKgn1ZUVMzBQ0f5\naMUqnE4ng4qHIoV7seMcSqu7laYRNxKJUHrgm9Vqw+1pNBQIQQghEK2CloyWmd2XEJK2RmJB1/ga\naoDy8jJOnqwgt2su1TVOunTJJbdLV2WmJIqDKaIYFV5f6HkIYS0YIwTijNAWhk9Iqu/iLmod6F87\nsTz4lczANMDJkxXcddf3qKiswWqz88orc5sEQopQ5cTFIZHGXZbbwj145hEC0aIkdlHGeyMlp4ck\nROLM0pwNWmKupezsLKpP1WJtctlkZ2eRuDioV5i/9urr69i3dzdZ2VmcOlXD0KHDsFmVrxkQV7I5\nhEC0cpJheqvLiF8whEicGRJ55sF83viuNYnDh8uod3lITbXR0NDAyZNV9Cjsq5vX6/WwfdsW0tLT\naGhoYFhJKZJFPdperyL6cTQ9K2Lv3l3cfvttWCwSTqeT116bx/BhI5SZhF/JFEIgWoTY77zmfIgn\nMcEQItG+MLrQzFkPAMWDS3nttXlkZGTQuVM2Q4cO1ynav7BlyyZuuul60tLSqK2tZd68+U0jnSLV\nRcm33x7A63Vhs9morW1gyJBhivV5eV2xWPxlpaamkpMTet9BaASscDGZQQhEs2L+IjyT16u5MeeK\nLQK5m6E2guQT/eIye/2pBcPtdmNzpAR76BMmjuar1Rt0xQEgLy83+ArSzMxMcnM7m64jQHV1FV26\nZHPBBecDcODAAVatWkP//gMJNP+nT58O5pdlOH069DIdccXGhhCIZiN5N2W85Ycwf1toX/IerQ7i\nlmvdJGukkP7GzkYPVqvDXAkS1NUp3yrpXzZfsWPHyrj66suCy3379mXlSuXb9zp3zuNf/3qNnJwc\nKo6foG+/Im1dTL31zdy13Z69VUIgmoXk+YyTE5zUKyPyVW2+jkIk2i6xuJf0rQdZsirStMVKNDTU\ns3fvTjIyMjh+vIJ/vz6PPn37cODAAbp0ydOvgcEIpfz87qxft4EpU88D4NDBQ6Q4UhV58vLyycvL\nB6Bbfs+mVGVhlhjV0rwItK97QQhE0kmGOJi7eKOVFfmiNu8mij6cUIhE6yQ515FROS63B6slkvXg\n327Pnu3cfvvtWCwS1dXVLHjrbRobfQwYMFTxrmh1ffSuuy5durB//17eeONNHHY71adOU1JSGpZP\neS3KyHi9Xs0opkCMInY61nUuBCKpJCIOxisTGSeuh/YJVDBjUQiRaLskN8Yl4fP58PokbNGm+5Rl\nunfvHmyQO3XqRFZ2JhkZWcF6yTLs3ZvGgAENms2PHXOQleUlI8MbTOsXdBnJ9GgyEPSuz2+/PYDP\n5yYnO4uDhw4zZMgwHI4UZFnGapXweDxs3ryJtLQU3O5abLZ0XcGKcnjtGiEQSSORBt64F9Uc6McZ\nogtF9PiECF4nn3g7HdEuoETcS+BqVE9VoV+eZJGUQWIZampqFWVKElgssG1bBkOH1gXzlpU5OHo0\nhTFjamJuiL1eD1arzKxZs5qWvbz88lyGDRuBz+vFlmJj/fqvufvuu0hLS6OsrIw33/wvpaUjY9uR\ngmRc963r3hECkTDJsxqa/zWIyotPf/SS/nhz9XbCmmg7JPbsgzqDf9njlbFYtbn1tktPz+Tfr8+j\nc+fOHD16lP5FxZrc/fs3sG9fWlAkwsXBDOrrsa6unh49CoPLVquV9KbRU+DFYnHQo0dhcERVQUEB\n2dkZ4SWa2m9yaX33jBCIhIjXajArDAa9sjh0Qg5Of6xI1S1TXyz06xDZmmh9F3x7In4rM7L1EA1Z\nlvFhCZuGTy84HapfYaHfD+T1esntWmBYbkAkli/vQqdO7ijiYHRt+dOzs7LYsnUTEyZMRJLgyJGj\nNLo9bN26Ba+7gdEjR+iOqNK7npPlRopcTuu8V4RAxE084mBGGGK1Ksyh/3CcvktI2/gbN/aRhUKI\nRGLE++MndtFEcy85XY3YrJGaDr39S1gjbuMnLc1LVZWNgoLQu6NPVFRwrPwoPq+PoSXDDcuRZR9b\nt25Gln306dOPXr368fLLr5CWlk5DQwMNDQ3cfPMNdMpKZe6rr2K3p/Haa/9mwIAi9u3bS36+sXg1\n7UG1v6iHY5LWe4+IN8rFhXHP3kyDrp9PUpRrXJYU5ydSHfTzaZ+iTVwUBckhvgfb4itDjU/xilB9\n60G1J1PlBtxK55+/nerqNXz++Q527NiOZPFx2203c+NN1/Hpp8vZuHE9brc7uJ0s+/+sX7+G2bNn\n8t3v3sapUyfxeb2UDC2lf78B2GwOZs/+DulpqXTKyeaee76P01lPfn4PDh06xqxZs4JDY5sDYzFp\nveIAQiDiINa7StvwRyozsnhIijxmP3plGO8v3uCmEIm2QSzBaf34g8/ETKjRxEe9PiAOxcWHqamp\n5L77ruNHP5pNdXUlF188HYCUlBRmzJjB5MlnsWXLRrxeT3D7I0cPc+G088nMzESS4NprZ3K07HBw\nvSz7sNtt2KyB4/TfCw6Hnby8PGw2W7sfkRQPQiBiItbG0UyPXIrYmEuSZKLBj2w5aEUjcp7o1oyx\nNRFZJIRQJIMzPY2QT92Qqusjy/h8XiIR3hjLMpw44WDMmBr27dvLjBkzg+uKivoq8tbX17FlyxYm\nThzPzp3bQ1WQLPhkn6JMOWzDoqJi5s17A0n2IssyL7zwdwYM0AbLm7aOWPeOhBAIU8TjXjGyGrSN\nsnpdNDHQEw3jj5SAIMRnbcT7vIcgnHjPk7nt4hUZr9eLFHzITFvIJ598ytGyb6msLGfjxnVNjXT0\nZ2yGD/cPfU1NTaWysjK4bviw4fz97y9QU1PD3r17+fzzL7jyyiux2WxUHD8eFI/CggJWrPiYqqoq\nfD6ZefP+Q8+evYPl2GxWhgwewiuvzOXpp5+lZ89+ZGVlRz1eo2k3tNZG+xQVEaROkEjioO8qMm5k\n9dLNWCbRkTVlh1/4oUCzFExT7lsC1YgnMwFsY+LZRhArRtdfImW43R6sFv1m4/jxcnr3LuC88yYD\nUFVVxeLF7zFkSAn+3zv6/ouLh7BgwUImTBiHRbLwxZer6N+/mL89/3fq6mt45JFHACgpKWHXrt3I\nsg9JsoAkMXrUOBYtWoLb7aZ//yIyMzKB0JWWlmJn9Ohx/jQ5fE044roMRwhEVBJzK0XexlgwzImI\nWSRVj0dfFIyEQlkHpXiERkNpR0IJn248nAkLy/w+5QgNaGXlSc47b1JwuXPnzlFdTT6fD2QZS9NU\nGJIkMXr0eI4ePY4sy+Tnd8fpPM3VV1/Jyo9X8vHHn9DY6KJHjx44HA5kOazzI0l+MdJp/L1uNxmZ\nKf5jMHldtszw1taNcDFFJJluJa3LSM+NFClN6Q4KuJqMP8q8+i4lozR1vdVpkY5d/xxEzy9I9Jwk\ntr0Z92Gkxq5Xr16sWrU6uLxx40ZycjqFb63Iv23bN5SVHaTiRBmbNq1XrMvLyyc/vxuNjU6uuuoq\nBhYPxGKxMHLkCKZPn47X62PTps1YLFZluar6BRbtNplYp9EwKrSjuJdAWBBxEY84qJf1GmH9ZaM0\nvbL9fPLJx3z22cekpKRw663fJS8vj3ArQpLk4PYBU1tpPahdUqHjUlsM0dxNwpIwS3IFM1JbGK8V\nqvyvJSMji/z8Xsyd+xp2ux2wUFQ0UDfvoUMHmDBhPIMHDwKgvLycpUtXMmjQkGAeWfbHJAB8Xh99\n+/alUye/4JSWDmfr1u3agnXQtx7CJvSLcXrvjoQQCENiCcAai4P5ZXPCcepUNRUVx+nduzcpKaFp\njnfs2M7cuS9TXl7Ot9/ux+XyP2i0e/cu/vKXF7DbHU1lhXzBshwQC7V4SCofrZEbKlF3k4hFtB2a\nrosov1e/fv04fvwUYHSv+MupqqqkeGBIPLp164bL1ai5TioqTnLq1Cmys3NwOp2KdR6PG33rQVmI\nLQ7roSM/PR2OEIgYiHaNRRYDZWMfSQiM1r333ru88MJfqKysZODAYh599HF69+5NQ0MDjz32KAcO\n7NfUaceObRw4cIDi4kFEEoeAVRFuQQRiEUaiEL4cSgvUW4hEe0Qi0qt2lPivK9019O7dj6XLlnHR\nRf5nHFatWk3Xrtp3Q4wYMYqFCxdjt1spLy8nP38NxcUDWbnyEzp16hJepPoLMuD1NJKdYcZ6MDwK\nzTFFWt/eEAKhSyy9DX2XkLGbSTIUg0giATKvv/5qcAjg7t27ePXVf/KrX/2ab789oCsOANnZOeTl\n5SmGreqLg9H3wHZK0VAum7tJhLup7ZOsn69Tp04cPXqY116bh8ViwW530KdPf83eJEmipMT/juvi\n4qGUl5ezY8cH9OzZg4ymUUpG9fR5vaQ6LK3UemgbCIEwifnhrNqYg3lBMF7v8XhpaFCa2E6nE0mS\n6NmzJ3l5eVRUVATXpaSkkpOTww033Exubm4wPZogGFkQyvTAcWpFIpIVYYywItoKFkkm7HE0Eomd\nhCbx87Bnz2727N5BUdFAJIv+2BlZhvz8buTnd8OMa0nCR4ojkdhDtGuy/V+zQiA0JBIsjOQu0ks3\nLxh2u50xY8bxwQdLAEhPT+ecc85DkiRycjpx330/Ye7cl3A6nYwbN57vfe/7pKamYreHfmJtzMGH\nz+fDYrEGxUHfgjASCfWxRxcJYUW0bWxWK06PD4sU1ohH0AljN5Mft9vN1q2bmDXrWlwuFwsW/Jcx\no8crREKWobGxkZ07t5KZmUFdXT0lJcORJIsp11JkormQzK2LPW/buAmEQCiIPTAdSyxBmR5JJPTX\nPfjgw/Tv358TJyoYPXocU6ZMDZY6bdqFTJt2oaKGxg8DwaJFC5k37zVcLhfjxk3gZz/7RXBorNKa\nAEmKZGForQrteTJzMwgroi1gt9updzmxRHyVnHnLYvv2Ldx++63YbP6maPbsWSxZ8gGDBw/1lyQH\n8n0TzHfq1CneenMhw4YFXu6jvG58Xg/pKdagaykZsYfY17cPhEBEwbxryVgcoguAsTCEX+Tvvvs2\ntbW1nHPOeYwfP9HkcEVtw1tRUcELL/yVU6eqAViyZDH9+xcxe/Z1CiHQsyDMikQkS0FYEfET91D+\nJGK1mPnxlAMiwlFY1aCYvjs9PQ2v16vZrlu3/KCI5OTkkJ2TFbafEF6vF7uVpmG26vqgKldYD9FI\nyoNyXq+Xq6++mrvuuisZxZ0htHdetCGt4WmRrYXw8mIRCSn4/8knf8fvf/8Ec+e+zIMPPsDSpR8Q\n7UE5i0U//ejRI0FxCFBRUU5ojqfQf3+9Jd36h6fpnS/9c6JepzyPgtaPpPidZMU/JfoNoX8iPf+n\nT98i5s37D7Lsf9bh9dfnMWBAsaoRl6mrrVOUUVtbpynf6/Nhs/hIS3Uo9hU78VsP0Ye1th1xgCRZ\nEHPnzqWoqIi6urromds0oRvDTOOnzaMUAWUDbCwUsuzjiy8+C85OWVNTw/LlS7n44kt19+8n/EIM\ntyIkBg0aTFHRAPbu3QNAenoGY8aMC+4z4DcOv9jVbqbIloQ5V1O0WIZAyZm3HvxWgcNupcHtxap+\n56ihSBhXPCsrG1mGl156BVmWGTBgCA5HSti2ftLSM/n3v+dRUNCdAwcOUtC9p6IcnyxjwUt6Wiju\nkKyH4tTT1MRH27ymExaIY8eO8cknn3DXXXfxyiuvJKFKZwKzPVz9dZHjDpJKMMLFwaxgWHA4Qr0i\nAIfD4Q/SRTwmpasn8D0jI4PHHvsdL7/8Ii6Xi/POm8o550xGlpWNf8DF5BcHWSUeSpGIdK6EO6nt\nohdkttvtuNwNQEAgosUcIq/Pzs5i2LARqvxKCgt6IiPjdLoYOmS4pnR8bjIzQg+OxhJ3SI4AtM/r\nPGGBePzxx3nggQeora1NRn1aBebjDpLhdz13jHJZTxyMxENizpwbef75Z6ipqaFPnz7ccsvtQVeQ\nHtGGmvbvX8Svf/0EIBOYO19ZVihIHbIs5KBY1NfXMXfuyzQ0NHD22ecybtx4QysiUhBbWBHmaHnr\nIXKjbrdZafSqRjNF3DSB3zPQwUEiLVU5Mskny+BzkxVFHIzqEk0cOrL1AAkKxMqVK8nNzWXo0KF8\n9dVXyapTm0IbYzDOoxSEwPfIAhH4Pnv2HCZNmsT+/QcYPXoUWVk5iv3Jsswnn6zg2LFjTJlyAd27\nd1dZDqFGd8eObTzzzJ+prq5iyJASHnjg54pAYajXqGzgA+Lg9fp48MGfsWHDOgBWrlzOL3/5iEIk\nBK0Xo6Gn0YakNuUCJFIcDhrrG0ByKNJV2RKsqOEC4I992iw+0nXFQVVUwq6l5OVtS0iyHP+hPfXU\nUyxatAir1UpjYyO1tbVMnz6d3//+97r5PR4vNptVd50gMe677z7++te/4vF4KC4uZsGCBQwbNkyT\nT5Zlxo8fz9q1a4Np/+///T+eeOIJ0/vavXs3Q4cOxeMJvfLx7rvv5i9/+UtiByFoUzhdjdTUNQan\n6m5JvF4PGak2MtLTWnzfHYmELIj777+f+++/H4Cvv/6al156yVAcAKqq6hPZHXl5WSxb9mlCZeij\n7Opoe1HB72meAAAgAElEQVT67iVt7EE9PDWUfu654/niizWorYbwkUratPBlvf34/58+fYpXXnkl\n2GDv2rWLRx75Db/61aNAoHfjdyXV19eyZ88exdGtXbueNWu+acoTcjmBzMSJo/nyy7VhZchUVVWT\nnp7O6dOng2VUVZ3m44+/1OzPjxyWHloOR7+bkli3bNq0yc10vTQHoYtr2rRzWbbsM+VaU71xvUEU\nZtfrXePR0xucTjyyNehqmjBxNF+tXk/i5gPoXic0zc6aZg0OewVjV5DZ9PPPP4cVKz43WK9fF+O8\n5rdNBtOmTaaioibu7fPysgzXifdBxIVevEG7Xi0a4e6l6OLgD06Hhqdamoatqv/7C9czBJXb+4e9\npqdn0qtXH0W+fv36s3DhfB5++EH+/Oc/4nI1KGInyliHROfOnbj++pvJycnBZrMxevRYbrrpVtSN\ngjpgH0vgP3w7QcthLpALgUYvLTUVi6z3UqB4GkVZ9VGu8Xq9yF43WRl2A3FQbheraOiv189jnNf8\ntm2BpD0oN378eMaPH5+s4lo9eqKgF5hWfzd+kY+kIxDRRjz5y+3UqQsXX3wZCxbMx+fz0bNnL2bP\nvh6LRUKWJZzOelavXkVeXh5Dhw7j0Uf/j6effpKqqiqGDSslKyubJ5/8PW63G4AjR47whz/8idDF\nHX4s/uD19dffxGWXXUFNzWm6dy/A0jQ1gv4QWO25Mn/jiJhGy2AUNIgeTMhIT6Wmth7J6lCtSfx3\nk/F3fmSvm7QUG3a7Mkht7qG36MFq/e0E4knqqO4ldbq5Xq1WJJTLesIgSRI1NTUsXPgmkmRh9uzr\nyMjIVAmCjNPlxuvz4fOFbt8f/fgXDB8xlhMVx7ng/CkMGDAQkKisrODuu+9ky5ZvcDgcXHfdDfz4\nxz/jmWeeD7qS7r//3qA4AGzbthWv1xMMXAfqH5p6w38X5eR0Ijs7h5A7KfrdFWlEkyB5RAs4R1qv\nXee/yozSAbIy02lwOvF5I79iNBYC7iSHHdIyU7Xr4xCH2IPVHdd6ACEQcWEcezCyHpQWhNGnurqa\nWbOu5uTJkwCsWLGUF1+cS3p6Oh6PG6fLi8cnY7XZkSQbFpu/gV+69EPKy8uZOGEi0y++AoBTtS4k\nSebv/3iBLVu+AfwTnr355hsMGFBMXV0dU6ZcQLdu3aitDcUSALKzs7Faw6cqCBxXIJagFAr1sxPh\nz0qErIjogqBvbQjA7AijqKWQqNsukkikpaaSk5mCx9OIxWrTHwJrooY+nw/Z58FulcjITNEctzkB\niLZOZ98xuJai0z4u5A4uEGZulnhvKKW1EEoz/vz0p/cHxQFg8+ZvePfdRVxy2dX4ZAmrzY7NqnQx\nPfPMk7z99gK8Xi///ve/KCwspHPnLsyadR0TJkyktsGD1Z6K1+2fKryhoYFHHvklXq+X1157lb59\n+7F27Zqmulro0aOQu+/+AV6vl8cee4S9e3fjcKRw5533MGrUaJVQ+I8pIArKYzdzgwgrIhaSIxIR\n90Dguo0kBPrrACQcDjs5mWk0NjbS6HHj9clRxcIny02Whw+rRSLNbsVuT0Pv2kiOOJi55iLniT6l\nRvugAwuE9k6L172kZ0UorQc915LWzXTy5HFNHWWLHcnqwKrzjITH4+aTT1YEJzerr69jz57dABw8\neICxYyewfPkybI40rLYU3K46rBaCo50OHz7E4cOHgnuTZR8XXngJU6dO49ln/8SSJe8E1z311O94\n9dV/N8UaQuchJA5aKyJ0zmSd72eSRFrZ1lD/5BC74EQSicB6Pw6Hg8DD/263G7fHjUzoVbaB52os\nEqRYJWwp9mAcSy9IHdin3r6irYt6VDH+pB1FHECMYopCJBFRPyUdvl4rCKHy1EHoUMMfmOI4kDc3\nvwcXX3JFcARSYFRS6LsFq8EY9PLyct5/fwmNjf53U0sWC5OnXkRWTq5u/vD6S5J/ChVleceoq6sL\nq6/euVCUpEgz+q63/+Yl0R2EBP9MkLgLLloBRqN9YqmLciSS3W4nPS2VjLQUMtNTyExPJSsjlcz0\nFNLTUnA4HFgsEpGEwWikkrYesQmLcf3joX2JA3RYgUj+Da4VC6X1EMijti7CReKnP/0FZ511Njmd\nOtO/uIQnn3oWm82uEpOASPh7aRMmTNKtT0pKKj6fMmBYUjKMiy+9Ekeq/1WNBQWFjBkzNri+qGgA\n3/nOHCRJorh4EOGvauzbt39TQFrrGossFFHPXJLyJGtf7YfmiedoG2dzM5hG+xhsnWDjn8whrfr5\n2zcd2MWUCMpGX9+K0MsfLgpaocjPz+PZZ/9GfaOM3WZHUk3XHe6+kiQJCYlzzjmXxYv/q6mhy6V8\nPWlBQSHnT72A3n36MmLESMoOf8vFF02nR4+ezJ//BnV1dVx++eV07ZqHLMvccsvt1NfXs3//bsDC\nXXfd0zRs1ugmC3ct6Qer1eeoZW+2ZItDoLzW1GLEHoQ2dhWZcSWFViQzPmKm0Y49jxCHeBACYYj6\nao8mAsbupUDe8O+NjW6++OJT0tPTmTTpHALGXEOjF7vNATqjnPxl+Btbj8eLw25n7NhxDBo0mJ07\ndxgeyYQJZ3HXXffQr39/ZBkuuGA6ss+HBQ82m53rrrsBWfY1PUXtb9gtFom7776HUaNKWLt2c3Cd\n/k2kF6gOnItQotnRTMmnOS2H1hJX0SeaAJjJY9z4a8uJnD9yPfXLj5Y3vjzJjTsY16OtIwQiBowt\nA38g7s03/0NtbR1TpkxlyJCh6szBbZxOJ/fddzfr169DkiQuueQynnjiD9TUurCFiYPaBSVJEh99\n9AEvvfQitbW1OByOpucV7GRnZyumvghn6tTzGTJ4MHLQnJfAYsHtkWh0u3HY7SgbA3+ewE2hPG4p\nSs/LSCwSJd6GuCXcSq1JJMxaEWbymRUJNGUl9vubafT188WXJ0ptWstPewYQAhEFMz0hWfbx8MMP\n8sUX/vlzPvroPX796ycoLR2JMg7hb+ifeur3rF+/rmlbmfffX8IVV1zDsBHjsKAUhPDv9fUNPP/X\nv3CsvExTh+zsbKZPv5j6unp27tpORUUFACUlw7n4osv8giMHDsgHSNhsdpyNjaQ4/AIRagSkMKEg\nrP6y5pyE3EiB49OKQ/Ish1jKaOl4Q2t0OZnHjJUQ2TJI5Lhjde0ky7KIvv+OLA4gBCIpHD9eztdf\nrw4uV1RU8NFHH1BaOlLjYlq16gvef3+JYntZljldU+efX6ZpBNHqVZ/To2cvzjrr7GC84dSpU5w4\neUK3DqdPn2bWd2YzctQYKitP8tZb87FIErNmX096RjqyLPtv86BIBIak2nC6GklNcTTV05++adN6\n/vznP1JfX0e/fkU89NCj2O3hUymExEJfHJRTbugRvq3WFWV0ts2IRHzioG784mscWpM14ScWF1G0\nDpE595H6+OMTz/jnRIrFaohPHKqrqzl+/BjduxeSnW082V1bRwhEzGhH6qSkpJCSkqKYrsLhsOu6\npJYu/RCnUxlAHjlyNOMmnAVI7Nq1k0ce+SVHjx7B4XAwZ84N3P2/9wAS3bp1o2fPnhw4sF9Tq6Ki\nAaSmpXL48CF69erNnXfeTfjMrEEDAhlkCSS/hWCxWnC53aSmSEHXEsg88cRv2LFjOwA7d+6kS5dc\nfvSjnwT3F3qZkEy4OHz88Qo2bdpIfn53vvOdWUR+6128RGqEYxcHowYvkN7aepHRG+lYgtWxxSMC\neSCWOIP5E5h4D7/5xWH//r10757LjBlX8MUXX3Lw4El69+5rvEEbpoMOc00mEp07d+Gaa64lJcU/\nkdjgwUO58cZbFXnA3+vYtm2rYuu0tHR++7snSU3LAAkWLJjP0aNHAP/UGEuWvIOrsRFJApvNxg03\n3KypQVHRAHJzu3LrrTcxa9YMnnji//x7DXNPBeIZyvhGYJ0Nj8cdDIS7XE7Ky8sV+/Avh1qE8KGt\nAE6nkx/+8B4eeeSXLFgwn+eff4annvqDJr/eqK/40CvDfLnaZzmi5zVPuFuxdRDLw11mG+noQ1zN\nESgn1kB1c4lDNCTJxxVXXEH37t2ZOXMGHk9j3GW1doQFkSTuuOMuLrhgOpWVFYwYMYrU1MCLTCR2\n7tzB2rVf8Oyzz7F3b+hdDP4pLP6XrOzOISNcdUU3Njby7LN/plu3btx4481cdtkVLF++lNWr/e9e\nKC4uZvr0S3nuuT83beHhrbfeYP/+fVx33Q1MnXoBgZsh1OtvsiQk/7BVq82Kq9GNzWZDkiTS0tLo\n378/69ZVBvMXFxdrYgkBV5AkwcMPP8i6dWsUdV+37muUN6J594u5YbDh5cUmDvEQu0VxJt1NsVgR\nOlsbWhJoym2eoG9iVoPxfqLvPFr9UlWvPU1JUc9i234QApEA6huof/8iiooGKHrXixf/l+eee5qa\nGu0II7vdxsGDB2l0e0m1+vNfecXVbNq0gfLyciwWC05nA2+99QYAu3fv4rHHfsdTTz3NAz+9ny1b\nttDQ0NDUECtZt24NGzas43//9wfcdtv/BOMLwb+qifQ8Xh+BV4qCxOOP/54nn/wDXm8jvXv35/bb\n7wges1oknE6XxjICSE1NayoztG+9c5hYL7T5hUGvnLYhEkqiB5m1DX8sI5fCt4uPWIPVxts0lzgA\nHD9+giNHjtCjRw/27dtHZeUpevSIvl1bRAhEBGJtUNTDQQEWLnxTVxwA6urqWLjwTfIL+3DLzd9F\nkqB0xCiefPIZvvzyCz777GM2b/4mmH/Nmq9xuZysX7+ONWvW4Gp0Un2qiuPHT5Cfn8/x48q5nHw+\nH/Pm/Yvbb/+fpoC0P/3gwQP8+U9PUVl5kqKigfz4xz8FLPh8via3lExhYU/++Mc/MXz4IDZu3IbP\nF3oOQi0SDoeDrKxsqqpCFkdKSgrXX691h0U4ezRnQ5oscQgvLzaRgNYgFMkTiUD+ALGe4JYKWMe3\nvwg1obR0FAsXLkaWfdhsNoYPH5GkslsfQiB0UV/sxm+NM8ofwOfzKdbYbDbFu5wBf8MuhfbTt29/\n+vfvz6GD3yoEIjU1FbvNwd69e3A1hgLdLlcDl112A6dPn2LhwgXIcmifbrd/X6FGHR5//DfBYbbb\ntm8lPSONe+75IW63G4fDgXIEEmHLgWNTioQkWbjjjjv529/+QmXlSXr27M1DDz1K7959iBxMPvMN\nZiK0/NPgIcyPJoqn4Y5FJMK30/ueGIlbDcb5zW+vLWfIkBJTZbZ1RJC6mbnookuCweusrGx+9KOf\nUlQ0ILjearNTWNiT559/jueff5Zjx0LPONxy6+0MGjQYgJzsTtx8861YbTbGjz+L3NyuwXxZWVkc\nOngQSerNyJGTFfsfP36C6rkFH4cPH1bkOXL4CJJkwesLjEjSP5ZIjcTUqdN4/fX5zJ//Ni+++Ap9\n+vSJdFpMkYxef+wB5tjLjyF3c1UjJqIHl5svIB0L+lZD84iDidokqZy2hbAgmpkbb7yFfv3609BQ\nQ9euBYwbN56xY8fz97//lfr6errm5fPmm29QXe13z3z11WqefvqvdO3ahR49evL3v7/E3r17Kehe\nSNe8PAAGDx7MQw89yvz589i2bTvV1SdZtvwj4CO+8505TJ78Iz7/7HOOlh2hrKyMp576A/fd9xP8\nk/xZ6NmzJ8ePh0Yp9ejpd6CqjJ2YsdlsdOrUidC0HGcO8w13tIzRjyPxmERzCoexFRGruymwDSRf\ndBOZyqIlLYeOhrAgWoBzzpnMgw8+yJgx45BlGDhwIE8++TQ33HATy5cvDYoD+MdYL1v2YXA5NSWV\nkpLh5Hbtqihz4MCBHD9+nOrqk4r0devWctNNt3O65jRlZUfZtm0Lr702l7lzXw7m+fnPH+LsSecy\nZMhQrrj8Kv73rnuAeG/6xJ87iLTfeOJA0beRwj7JyRvfMNiWGg4b33j/aD32ePsA4cNazVkz8cQb\nhOWQDIQF0SxE9/3KMvz3vwuor6sjJaOTYl1OTk7UPcyb9zp79uzSpGdmZlBZeZJDhw4q0g/s3xf8\n3qdPP5760zPIPhkZ/yR9Pl8sDXd4HCKUt7mMBjNlm2ugo2dK5CnuMxmTiI7xyCNzD94Zb2smLXbi\nHXGUaPA7/rLaI8KCSArxXUj+N2gp3THTLpjOhRdepCrXXPl5eYVMmHAvnTt3oVevXop1hw4f1kwB\n3tKY69kbbxvpY2LvpspUp+uXE7merZtYffjqbZuj0ZR1PhFyG1odQhySjRCIZmLhwvnceedt3HXX\nd1m6NOAyUl501113IwUFhciyj5SUVGZ9Zw4P/erXWCyW4MUbuoiVN8+YMddTWFgULOu886ayaNFi\nZs6cxLp1qTz00KP07BkSiY0b1/Hoow+1QDAvFlqqNVXuJ5bAdWShaKskIhKB7eMVi9jEQLFlRHdU\ncsqpr69n06Z1lJUdZMuWjVRXV5sutz0iXEy6qF1EgYntIuUJmepff72a559/jvr6egC+/fZbZs68\nSlWWzKhRY/jTn57j63UbGDykhOHDRyqeTvb/l0GWNJf/5MmFDBnyDz766D1ycjpx+eVXYbFYKC/f\nx7cHVlEyrITevXsr3jm9bt1agjelHKhL0w3j82GzW4NpgeMxPj/K9a3XtRL6jRLp3es/Ra3vcmop\nV5O54aexlwmxlKt3oMk/+GR1bKL9Lrt3b+dHP7o3+H7sp59+hk6dRsa0j/aEEIgIRLsBjaYj2LZt\na1AcAKqqKnnjjTe49NJrgvnLy8v5+c9/wu7du8jN684P7vsZ4eIR/I+EVh4A/G+gC8z5JEnw2Wef\n8H//9wjHjx8nMzOTXr36KLaoqqrknXfe5rLLrvLvq6m+AB6vG0dGOj6fTEgklC4u/Seik90YJOP5\nCK3FYDZvCP06mG38W1IkAvuLcUsiWUHNIT7xkEw3kJnfo2vXLkFxAMjL6xohd/tHuJgSRtubHjx4\nSPDZhwAvvfQShw8fCjayzz33NBs2rKe2tpZv9+/hn/98QdGr1/wPjviQCX/zW8AikGWZ+W/8J/g0\ndW1tLQ0N9TgcoXr4fD7++Mff8Zvf/ApXo1tRd2vYlBh6QqBOS3ZwUu8p9DhK0WwbOY4QaT/G65NT\n1+QS37mP19ff/ER3JyVfHABOnqwKPtwqyzIVFSejbNG+EQIRM5HeluZfMWHCJM49d4pizcGDB5ti\nEf7t1dNvnKquxCd7VY00ChFADnMJ6Xx8svJBBkmyaILV9fX1vPvuYl588W9hN6GM1SJx7FgZc+e+\nzJIli8IEKJI4yKr/kYmtsYm14Y1lyGrTkqmgt75QmBGJlu6BN4dIBMptbqEwN+w1vrhHLHUfOHAw\nf/7zM/zrX6/x9NPP0rNnn+gbtWOEi6mZuOiiS1mxYqliqo2srMCLRWSys5VDWXv2KMD/7oRwN1OT\nq8K/iW6D47/4/Ssuv/xKtm3dQvWpahwOB/X19TgbGnSn9ygrO0rQveRupOrEUe655y727dsL+Kfj\n6NmzF/fe+wPGjTtbtT/Q3qhaSypxzLibojXe2nxmp/iG8OOJz/XV0kNf43MNRXY3hZdttg7JJf4C\nY61LWloapaWjASgo6B33ftsLwoJIAnrB2okTJ3LFFVfjcDiwWq1ce+21XH31zOB69TMM6akpeD2N\nTVZCyIUUdCsFhsMq1odeCCTLMtOnX8yTTz3D979/L4WFhZSXl3HqdDUej4eMjEzF/vr1K2L/vv2U\nlR3BZvU/VxEQB4BTp06xdesWfvazn7Fly2aFtRHN1RQv5oaVGj9kFm3EUfTRS9HK1Fog6n3oltpm\nLIkz5E8yJP46xWf1tLbjP/MICyIK4T0ybe8svOel/C5JEg888HNuuukW3G43s2fPYM2aTQQshLq6\nOsV+Ghtd2K0SXlnGIqnjEE29V+U/zX5lGUpLR1BaOoKFC99SlD9w4EA6d+7C6VOnGDxkKFu3bOaf\n/3wBi8XC1Vdcati8VVdXs3nzNwwdWoJaGPSH4kayMvQw2zOPtwGONtFipH2F6qW0KEJ1VloI2u30\nt29+4g8yByp4JmMr8YuCILl0UAvCzJWk70IJfNfzz6u/yzIUFPSgT59+BN6LEMgzZsy4YGkOh4OJ\nE88mPS0VrztgRYQsA4Wl0GRJyL6mjywjy76mjxycljt8QkCAEaUjefzx3/PcX/5G586d+eLLz/B6\nvdTXVPPGf15nzJgx9O9fhJq8vDxGjx5t4CPW6+GZu0sjTQiYnGcUQlaDUXr0h+8iBb0juaxahzWR\nWIPZkhaFrPrEuHXCMRKhLEYICyKM+HtdgW39vXlZhi+++JQVK5bicDi4+ebbGD68mCef/C1VVVWU\nlo7kgQd+Qc+ePSkrO8qwYaVN7ieZrAwHtQ2N2GwpoPFf+3uuMk31lNVNUWj/Dz/8G5555ilOnjzJ\nwAHF3PG9uwi8z6G2thYAr8eNz9uIzwsWi40XX3yVDz98n3379rJ//14kSeInP/kxRUWD8U8hbnQj\nRwrcqzFnMRj1uM0OWTVy/5h7mjtkHfjroLQMQlaDkSURZQ8tGJdI5JpuKiHsezLULXkHnpxzKMQh\nEkIgIqIN3kW74WQZNm5cyxNP/IbTp08BsGPHDl5++e988MEHAKxcuRxZlrnlltvxv6DHgiz75zay\nWq2kOXw0uD3YJO3PEwhgAxB4XkKnzlmZWTz4i181WRz+9MCUHhdcMI3333uHg/t3AjB8eClnnTWJ\n1NRUbrjhpjBLxMeoUSWsW7dZ9zjVPbdwyyL2EUv6Gxi79/TKMO7NxzY0NRZBiORuMj4JLelyCuwj\ncevFbGWb96CSd86EOESjAwuEtvE3zBlsnJRxBvVDbYF1X321OigOAHv37qaqKjSe2uv1snHjeq6/\n/iaAYIMfuPAdDgc+uZFGjxurzY4sy2zZspn6+jrGjh3f9GyDKgqhiY0QJgwolnv36sUffvdb3n1n\nETabjdtvv4PU1DSFS0sv3hB5GKL+zZasm9m4cYtsNZgTDC2xCwI6adGtpZYWitbw8Fs8tKaRUR2J\nDiwQySfgZurcuYsiPT09nby8PCorQ9N6Z2Vlo2zIYefOHaxdu4aKiuN06dKVGTOuxSP7eOpPT/Lh\nh+/j9XoZNWo0v//9n0hLSyO8AQq/gXbs2BYMUl999QyGDCkJioPH00i6w8bwYaUMKxmuinNoYx9N\nRxbxo29FaFE3qLE2pqrSFOWo082JhbKcELKue0mZFi4SyrrH40Jqy9NzNDfJPS9CGGKhgwuEGSsi\nkptJf/uZM2exe/dOvv56NQ5HCrNmXcfFF0/jBz+4l8rKSgYOLObWW7/LH//4W2pra5kw4SxcLhdP\nP/0UtbU1wXI+/ngZd931A5a881+w2JAkiQ0b1jN//n+46aZbgzd6+A1/9OgRfvnLX1BWdgSA9evX\n8ac/PUthYQ983kay0hxYrTYdYdAbXhs4Xm16LFZEJPRHARmVaRQwVq5Xi4N5sQjlCbllAgKsFAq1\nKKgFTmlxqI/DYK8t1HC3JZEQ4nBm6eACocXo5jEShYDVEO5uslot/PKXj9DQ4MThsGO32zn77HHk\n5OTjcrlIT0/nhz/8Pl9/vRqA5cuX0qlTZ4U4AGzYsJ5PPllBo7MWkLClpGOx2nG5nE1BY6mpDqH6\nfP75p0FxAL9gfLxyGTffeD1pmen4e/w+XWtB7xM4xoAFEvoo07U3svmbMXLvOZoohPJEFoLILicl\nWmtBb1n5PVCW0YHEaiE1L8mLSzQP8QtD6znH7YEOOsw1caJdwLIMaWmp2Gy2YF6LRSI9PY3Tp6vZ\ntm1LMK/T6eTkyRO65QwePISxY8cDMh5XHQX5Xbh4+jQ87ka83kBD7wt+8vO7Y7Xa8Pm8eBqdeBvr\n6de7kLTU1DBhCB8eq447qD8EvystB2WcIuzITaYpCR+yam4KDEB3OKs/TW85fOiqf3CAZLAfdbl6\ny3rfQ8vKNO06M+ejJWip0VRmiGydRt0aIQ7JJ2ELoqysjAceeIDKykokSWLWrFncfPPNyahbK0Lt\nSlIGqwO9Q72gdcDCgFBvOy0tnays7OBwUwC3263Z6+TJU7j00suZPv1iXn99Li5XI9dcM4NevXoD\nEo1uF15vyLqRgamTJ7Hhiot5/713kSQbl156BRdeeJFmlla9mIMy3diCUAuFfjA8Esa9aXMNo/n4\nQ2R3k7KsEOEWgtrNpOdiMrYktNaReUuivcclxGik1k/CAmGz2fjFL37BkCFDqKurY8aMGZx99tkU\nFWkfumqdRIoxEDE92DBr8qvFISQmsixhs9n57nfv5J//fIETJyo04jBtmv+tcuefPw2r1cqpU6eo\nrKzE7XZTUXGcHj16IUkyDrsD7IGtQhV48Be/4kc//AmyDOnpaTriEPivFAltWvjxyLpCEfHMRlxv\n3jev3Ua/d67XizdaZyYGYUYYJEl5HqKPbArUoWOKRPKPRYhDc5KwiykvL48hQ4YAkJGRQVFRUXDK\n6faNvitF2YsOtxyUjeull17GvHkLuOmmWxUl2Gw2brjhJi64YDoWi4WGhgbuvfduXn99LvPnz+PH\nP/4hmzdvIvypabWbyeeTSU1NIy0tNfg8Q/RYg75g+OutnlpcfbzRrIFobhdzH62rSbkukOZvVH28\n+eabrFjxEeXlxwi5lcJdSEYftTsKTbrWSlEfo/JA24q7KX4Xj7myk1xqsgsUqEhqDOLw4cNs376d\n0tLSZBbbAiR6oRk3msoGVDkBn8Nh55Zbbuess85GkiTsdgfXXHMtJSWlwcZ6w4a1bN0aelCtouI4\ny5cvDTb8Pp8c/KjFIlwclELhU+U3tiYiH2PY2ghCoUY/nqCfRz/+oCcYIQGQZf8rXw8dOkR+fle+\n/PJjjhw5qMgT+aMsP/RdKRhG8Qi9Zf3jljR5Ip2PliRcLPQad7314R2hSNsmqYbNUahAhSTLyfn5\n6urquOmmm7j77ruZNm2abh6Px4vNZk3G7toVXq+Xb775hoyMDIqLixXrtmzZwrhx43A6ncG03/72\ntyfR+XwAACAASURBVPzsZz9r6Wq2Gfbv389LL71EQUFBMM3n83HPPfecwVoJBG2PpAxzdbvd3Hvv\nvVx55ZWG4gBQVVVvuM4MeXlZLFv2aUJlRMa8W0Dt0jD+7v8/efJEPvvsK3w+maqqKnJysnE4UoI9\nWpBwuer46qsNqHus119/E/Pnz8PlcjFp0jlMmXIRGzZsRdl71UcZRwj19JVxCOO0iRNH8+WX61Tr\nQ/nCy1R/V+5fmR4dc7+F3rk/fvy4Jq5TVlbOl1+u1S1bTW1tDRs2rMHhcJCRkc3QocPQO34g6jlQ\nn/9wtF0zf8IFF5zL8uWfRaxjS8Ql4mHatHNZtixy3eOn+Q562rTJzdy2NB/Tpk2moqImekYD8vKy\nDNclLBCyLPPggw9SVFTErbfemmhxrRxtQDsQePY3TsrvgWBkYDTM8eMVPPLIg+zdu5vOnbtw990/\nYMqUC4DIwci77vo+s2Zdh8vlpHv3QqxWCz6frBgpFU5ozH54vfUbsVADZywA+uKApszoRAvORhOF\nUB7j0UkS3bp1w+v1v9MiIyODnTt3c+GFl2jy6dXF5/OxevXnzJp1LRaLhZ07d7F9+xaGDBmG+nmX\n8N9YW2Z4YFt/f20heN166FAH22pIWCDWrVvH4sWLGTRoEFdffTUA999/P5MnT064cmeaSCM71OtC\nyyGRCE8DePHF59my5RsAGhqO8M9/vsB5503FHwqSFduon9Tt3LlL0NJQxwZCSMG6qNcbicWrr77M\nypXLsdttzJ59PVOnTkNpRehZHer9hudTot8IGhOLKCjTlBbbVVfNwOOpZ/XqtVxxxQxycrJV+fTr\ncvLkCUpKhgRfXD9oUDF79uwL/iZqkVD+1z/m2EVCoESIw5kiYYEYO3YsO3bsSEZdWgFaC0ErEvpW\nhP524RYFmpcE1dTU4vF4sdslgx5povXXc/WEBGLlyqX84x/P09jYCPgHGQwaNISCggLd4Lq6Xmat\nB3WjH0l4w7bS3VY9gsjIrWexSEyZMoWUlCyDPPpkZWWza9eh4LLX68Xtdje5+4JHEFEsQmIf+i1j\nEwlzdAwrot0fYKtGTLURFyHXjpEVEWqoQy6mUaPGsnr1l0H/+NChw7DZrCox0e+Zmm8M9Nwd+gKx\nc+fOoDiAv/e8bdtmuncvwMiNZE4UIouccQOtF+MJpRs/y6AdWRRYF2rYtevUyDLU1tZy9Ogxli5d\nRm5uLjt37uLcc6c2bafnTgz/fZIpEol0FNoLHf34zzxCIExgrserFg11PAJmzJhJSoqDLVu+oXPn\nLtx22/8QeA9E5MZfr7Hwl79lyzesXr2KwsICLr30irAGUU8cQumyDIMGDSY1NTU4Qqpr1zxNQFa9\njdZlFS/6JzQ+YQj/rhx+Gkk81Kxd+xWdOmUxbdpUPv/8S7Kzc5k+/bKm30ftSlJ+j+wibF5LonVg\n5PYUtGWEQGjQcyEZu5qU6ZFEwl/GZZddwWWXXakKJPu30/NnR+Lzzz/l8cd/TVVVJRaLhW3btvGT\nn2iHvxoJxOTJU7jjjrtYuXIFNpuN2bOvo1u3AtXIHHU9tdaIPuZ86fEGoiMJw44d29ixYxuff76c\nvLweDB06FP/v4mPt2q9xu12Ulo4iOzs7WLrb7cHnczN27BgAZs68hsWLl1BcXGwo+urv4S5CpUUB\nZkVCex4iXwitQ1TOeAUEzYQQCF30RcIonxmRCKzT74XqiUJ0oXj//SVUVfnfMeHz+fjkk5Xcd9/9\n2Gyhn9VoKGUgfc6cG5gz54YwAdDGG8yIg5lGKrIVpudeMmMxhL5LksSRI4fYtu0b+vXrC8C2bRsB\nma1bv6G6uoo77vgfsrOzWbLkPYYPH01ubi6yDF6vh/T0dEWNHA5Hk0WmJwZE+B2VDXtkQdDmjbS+\n9dGa6yZIFDGbqyHaCz9SIxh5vL/S/A5vZNXDRpVpkb9brcqHDm02/zsj9GZo1Z9GQ/3kdGQxiHT8\n/gbaOF2/4fN/1E83h9JDy+HPiwRmYg3/HlhevnxpUBwA+vTpw6JFb5GWlsLVV19Fp06dsFgsXH75\nZWzduglJ8ge009LSOXq0jPp6/7M6W7ZsJSsrR3fftbU1LFr0Fm+//RYffLCEwNTr+q6saG6yUFqE\nsxtl/ZlCiEN7RwhERMzcAMa+fnWDqhWB8O/RBUHZ0MvMmXM9PXv2AiAtLY1rrpmpEQi9qbwDQqAv\nDFqh0Ds2I8IbekmCurpaVqxYysqVy3A6Xagbf2XDqRQLfWEgbHulQKxa9Tlut0vx5r6qqmpSU9Oa\n6hZupUhIkiW4rcUicdFFl/Ppp1/ywQcf4XS6GT58hGof/n2/++7b9OnTi379+tC5czYfffR+8Dj0\n/8cuEvqCanzOWx4hDh0B4WKKitLdpB+wDuVRu5vCYxDRfdgBt4WRCyNQjn950KDB/OUvf2ft2q/p\n168/xcWDmhr9kEvL8Kh0XEXR3UexNQp1dbUsWrSAYcNKkGWZBQv+w3e+cx0pKSnB4/Afp3pL8/GG\nUEMscezYUUaNGsWmTZs4fPgwHo+H3r370bVrV7p168bixYuZOXMmvXr14oMPPqSkZJRCNNavX4PF\n4p+3qrGxMbiP8N/Uby3Iwe3S0tLYsWMHBw8eoFevvoTHIdTxiND3sCNVLBu5pqKT3FiEaPwFfoRA\nxEE8IhGez0gY/NtKiu+BdSEfNgqx6NKlC9OnXxLcb6guEY9AcSx66bGVp89XX61m2LCS4ENnQ4cO\n5uuvV3PuuVMA/R60OWHQpvktJ/+aESNG4PP5OHjwIJdffjW7du3gww+XcNZZEykrK+Odd97hmmtm\n06VLbvD4tm3bSu/ehcFp6tetW8fRo0coLOxBQCSA4LGEzpFMt27dcDisbN36DSUlpbrxCK1IhMRA\nKxJhZyWCgKhJjkgIcRCEEAJhCm3QOnAjKhs5Y5FQpukLQ2BZb+ikdjmQFio/vuPSHpPR+lgJuLsC\n+Hw+rFarRhiMl81YD/7zsmzZh1RVnaSy8gQDBw7g5MlKJk+ejMXiL2DOnNn069cPgJKSEtas2cDZ\nZ58bPOYTJ44zZkxoHrERI0awfPnH9OjRM+w8+IVi4sRzWLXqcxobnWRmZjJnzhxycnJYsuT9YOOv\nfj4ivAy975Ea95YTCSEOAiUiBmEaszePUe9cnRby96uXjeMB4YHmUAwh3k+oHOU+Qp948ccTJk06\nlzVr1tLY2IjT6WTXrt1MmHBWME/Ir69e1g9KG30+++wTJMnHqFEjGTx4EGvXrmPKlAu5/PLLkSSJ\nqqoqevToEaxdVlYWXq8nGHuQJAtdu+azf//+YJ4tW7bQt29/TQxCkmDAgIHceOOtDBlSwp133klO\nTo7/CHSsnPBzEi6EenmSFUs4MzEJQXtECESC6PfWIrtw9ILZZoVD3ZCr3wAXyyd5ghBAItxVtGvX\ndrp378aePXvYuXMndrsDm82O1jIwIwyo/oc+VVUng880pKenU1RURGpqSlM9JEpLR7B8+YpgLb/8\nchVFRQMVAerhw0fw7beHWbp0GR9++BH19Y307t0nojDZ7Q527doNwLZt23A40qO4xSIHrY3SlOna\ndQJBcyFcTDGhdTWZyasViZBPO+By8ueTFGnquEOwBFmvQUm8gTdyTei709RoA84HD35L3759g8u7\ndu2msdFFSkoKVVXVHDt2lD59+pKRkRlTvCE87gD+htrtdrN79248Hg8nT1aSnR3o1VvIycmhpGQk\nH3ywFIvFQu/efenUqTMLFswHZAYNGsLw4SM4++zzUIqu9oSEJ40dO5Fdu3bwzjvv0b17ISNHjgoO\nElBPkaJ1NYXOabgLUXtO4/tdkxu0FnRUhEAkAf2gNRgLitKnrY1VKIUiPM2fbmShNG/P0vg4teIA\nUFV1khMnyrHb7bhcLtLS0rFaraxbt4YDB/aSm9uFb75Zz4QJZ9O//wBT8QY9gbjkksv5wx8eZ+zY\nMWRlZVFTU8MHH7zL2LGlTTEIicLCQgoLeyBJ/veXvPDCc3TqlIPD4WDjxnVYrbamaUYCjbJPc+zK\nY/QnFBcPYuDA4qAVp44xha4B9dPVsTf+scQiBIJkIFxMMaN/Uxr31pSNudaaULuYjNLVbiW1u0iZ\nJzkf86hdIIEYyZgxYygtLWXkyJHU1tZhs9nZs2cnAwYU0blzZwYPHsTGjeuiupXCRUH9oJzD4aBP\nnz5kZflffJKVlRV8FiI8fyDesH37NqxWC+effz4jRozA5XKyc+f2pvWBjwV91xKKeoTcalqLRy1q\nOmdNde6M8wgEZwJhQSSRyJZEpHx6VkJ4Omj91Noym9aaqqu5QKYZkVAX5F92OhvIysoMptrtdgoK\nCsN6/iECvfxYrAZtmpLy8mM88sgjzJgxB4fDoch74MA+7rzzTux2e7Buy5d/HBQAX9B4CHcRyqxY\n8RGnTlWTnp7B+edPb8qvdf1Fmn9JaUWE8qhdQkYuouPHj7F27VdIkoXc3K6MHz9Rm0kgSBLCgogL\n44Yzst/X/wBWuAWgLTdcFIzX6W8fni+yNWAcuI4F/R6wJPkfIHO5QlOJO51O0tLSAInU1PTglBYn\nTpyga9d8nSC0/v/wnrrT1UhtvZOaOieDhozi63Wb2H/oGF+t+4aCnn1xpGXz9jvvUn26npq6Burq\nXbhcLrKysoLiAJCbm8vo0WNUlobys3jxQrxeN92752O3W1i0aIGiTkqrQX1uJEVaMEcEd51eWmNj\nI59+upKion70798Ht7uBjRvXR/qBBIKEEAIRN7GJhNPp5OGHf8ENN1zLbbfdyMcfLzfMqxYC7Tp9\nsYgmTmZcSJHKiCVILUkWpk6dzp49+9m3bz8nT1YxbdrFSBJcddVMvF6JY8cq6Nq1O5Mnn69wGYHE\noUMHWbRoIYsWLaSs7BiSJOHxeKhrcHK6toFTtS48sg0sDmrrXXy49EP69O2PzWpHkiykpWeQmpaO\nLEtY7alIVgeyxU6jz0aP3sW89+EKyk9UUlV9mqVLlzNq1OgIbiWJ2trTZGZmAH7xa2ioVxyzUhiU\nYqH8r+dy0o/hqJePHSsjLy83uNy1a1cqKo7H+XsJBNERLqZmQu1Gevjhh/nii88BqKqq4sUXX2DS\npHNwOFIMRgkpA9kB9POgm1ebX1NLxbaxoW3U1D74goJCrr12tiINJCwWOO+88w1dScePH+Ozz1Yy\ncOAAAN7/YBEXXXw1Wdm5WG0OLLaQW0mSJObPf53evXsFn0fIy89j27ZteDxehg4dHnxYLtDjz+9W\nSJ3Tw6o123G7XYwcdz779n/Lvr07cTjspKamMWnSZALnM9yNpDgDTfVV/j5qCzDgZtIPKhu5kvTo\n0qULGzeepnv37oDfovAPGxYImgchEAkRuPEN1oaJREVFhWJdVVUVNTWnyc3NU+QHY6EIz6PNF55X\nP7/+Ntr9GMdSjMqKPC2G8Vvd9GMJGzduYODAATidLjZ8sxVHShpvLVzA1KnTGTBgQNDlFLA4fD6v\npm4NDU7OOedsunfvrcjrcjXi8sr06t2Xnk1zJ3m9PlauXMN5Z48lv2s2JyoqWLNmNePGTQhuO3Hi\nuXzyyTIyMzOoqalj4sRJhERSORBBGS/SG+4aikWEj24KfFef40CZmZlZ9OzZl61bt/L/2zvzIDmq\nO89/siqrqq/q+251qw8dLanVakkgdF8IoQOQjAAPw+ABPOHxHxNge8bemPGw4Y3xmGAc4dhYx4zD\neMGLvV7bgAQWAgQIoftCFzpBV6ullvpU31cdmbl/1JlVWUcfUqtb7xPRqqzM917+Klv9vvX7/d5h\nNstomsSjj26M/YsSCIaIEIhhE1skAJYtW8Yf/vAH/xafkydPISMjk1AB8NWJ3vl7ykfv/MPLB9sT\nXn4wRKooGbRpLA7BHkDoNZstgbaObk6eOsd99831JpnhyJH9lJeXI8tmfB2+JEmkpKTS2tJKenoG\nSUmJnDp1hqeffpb58++jrq4B0Ojt7cVqS2DApWGWZW99DTSJ9o5WiidMQDMn0t41gD01g67zF3RC\nNHVqJaWlpTQ1NZGVlYXFYsWzcJ9+cIGHQEevT1ZH9ybiYc6c+5g9ey6qqmAyBf/5Dq09gSAaQiBG\nhPBOPpS//du/5eTJ0xw/foykpGReeOHvQhZ+G4ynEFw+3jqDEaLYXoRxaClgTzyegpE49A84qZm7\ngN/97v9gsch+cQBITEygu7uLzMxM/AKBxMKFizlwYB+nT51CQ+OZZ54jP7+A999/nzNnviI1NZn8\n/HyuXrtJWdkUKqfNwGTydOyaBGmpqVy8cJ4KUwUut4JThp4+F4HtYD1iYrXamDBhAqqq4dsDIlII\nKjjEFGl/8WDPIfC7CT8OH+EkeT2IyL8fgWAkEAIxokT3Jtav38Ajj2yIow0I7cghvvBQ7DrhQjS8\nZGZ84aPQ5GxAJHznNLq6B8AsY7HamFhaRlPTDfr7+72jn6C+/gb19fVkZmb5xQEJKiomM2nSVMCz\n2qokwR//+HuysjKwWEysXbuGgQEnk6bM4J0tm/ny1EmefOqvkM0yaGCzJZKVmc3+AwexWiz093Sw\n5uF1dHQPYE+yYDKZ0TSJtrYWjhw5RGJiAv39AyxdugJZNuqogx+okScX7kXEl4sYuuchEAwFIRAj\nTnwhp9idcng7QxWKyB5I/CIRPd8Qq55EW9st2tramDhxIjZbgveahK/T6+zpxyzb/OdSUpKxWIo5\nceIEKSme+RSTJlXw1VdnKC4uISszE7zeSGAYrEccOjs7UFUFm81GcrKCpmkMuNyYzVbS09NJTEhk\nz55dPLhytfdbvUbltCqmTpuBqirIkoItwYaqanT3OUhOkDGbZQ4d2s+GDY96cxkO3n13K21tt8A7\nqXHlyofIzy8k+Jt/8PMd2jDiyJWG1qZAED9imOttIfZfbXx/2NHnMMRbbyhzHAbnVUT3Fvbu3c3O\nndu5cuUr/vSn/0trazMBEQsXB0mCBQsW093di4REdXU11dXVZGdnk52dRV1dLfiFIXzmtcViRVU8\nievm5mY6u3owm6309vbR2dmJLMsobsVf3j//QZIwm2RcCkheO8yyzRtuUrHbk/2ej81mo7b2MpMn\nVzBlymSmTp3Mrl07vG2GPr9QLyv0OQ/LhQu5h0AwcggP4rYR3ZOAwSSMjXMcg/UojGZwX7p0iYsX\nv2bOnPvJz8+Laq+eSJ0dOpFwu91cv36FyspKwDMpbe/e3Wza9BSSJNHZ7RGHQOjJV1diw4bHOXLk\nMN1dbdhTPcto3LrVxuzZ8wzEQUJVNXbt+hSLxUxXdxe3bt0iPz+fN377JqlpGVgsFgoLC2lqauaB\n+YuRTL6v4PqQkMlsweV2I8tmJEnDbLHR0+egr2/AX0pRFP8e4LGfU+j2s8brMQVyEkZ5B+EtCO48\nQiBuK7FFwl8yrtBTdKGIXN84nPTee5t5/fXf0NvbQ15ePv/tv/2Y2bPnRLm/8Sgl/bdgfQGXy6Wb\ntexJsJoAiZ6+fkyyfhmMwLHn54EH5rPj04+5VlcPaFROm0F2do6BOCh8vmsXVTOrSExIpKq6hq1b\nt3KjsZXlKx4iLd3O7t17uHGjgUULl1BeXg5ovpQ8tVdrOX36S0CjZtZciovysMgykuRJVCuYmFo5\ng/ff/8CbLO8lMzMTRVEwm82oqookmcM+f/DvLFwo4kHkHQSjhxCI287g/rjjSxpHHjUVPTmtF4m/\n/OVdent7AM/aRZs3/zmGQMQm1ItISkrC7Vb9QtHQ0MCECaW43S7cioRZ1odZgkc5+d4/tHpNUFLa\nl7fwjCDq7x/ApXr3iDZZSEi0owEmM0yeMp2SiZO5fLWOW3VNzKqZT3lFOVazyfs4PGLU0NjAoYP7\nKCjwTEDbu/dzHlr1EBMnFPrtMplksrILWLu2HFXVaGlpYvfunRw6dJj8/HwsFiuPPfYNw+cRSxTC\nRzNFLR1nOYFg+AiBuAuJf2RRZA8lnjZCJ5i53aphucheQ2gZYy/im9/8a3bs2IGiuCgrm0x19Wy6\nevoMQ0tG7yXJezbIs3A4HDhcKmaL1SsyJlRVw+F0YvMOje3u7sKWmMjMmdX4wlYSoEjQ2d2HzWom\nIcHGubNn/eIAUFCYz6nTpygtLkLyzmmQJDDJFhwOJ7293WzZ8jaVlVPIzp7P+fNf8Y1vPElCQiKh\ne0jc7rCQCD0JbidCIO4oIx1yGlyboSxZspy33voTbrcLu93OQw+tHmQL0ZayDnTysmxh7dp1/vID\nDgeYrLoygWN9/VDhUVWNnt5+JLMFs+xrw9PxPzB/IXv37iLBZsXpcjJnzhxMJt8OdlKQyEjIVisu\nVcPd3UtKSjKtrc3+4bR9vX3kFxQRjgmH08X+/XuprJziD29NnTqFQ4cOsnz5yjif2VBCTQLBnUcI\nxB0nfIJb1NJxh5yMw02hSengcn/3d39PRcUkrl+/Tk3NbGbNqolpTzwEd+qeyWbBg+UknC4VTYI9\ne/bhdruYMnkqJRPLCM4/BHsToLFr9+d0dnZiSUhh2bIHPZMMg8pKkoQsy6xc8ZB/Ndak5ASab/WG\neCCBdk0m0CQrU6fN4trH22hr6wA00tLSqa6ejaKq3vKBeQsqYDabcbvdNDc309zcjKZpJCYmj8iz\nC32OQkgEo4kQiFHFeDmMsFLDzEtEY8WKB+OuE++cB4ADB/ZRW3vR01EnJfP4409hMkkoioKqwXvv\nvUNxcREpKQkcO34El9vNpElTdG352vvww/eRZRPJqem4XG4++WQ7a9as95aTfP4HEhLNLU20tbVR\nXlaOxWb1ikOQQiAFyY7nSNEgJTWDFHsKKSl2Fi1cjKKouFxuNE3h6tVasrKySUtLx2S2sGjxCl7/\n3/+FLJuYPXs24Fm2/MSJ49TUzDZ6MnF7DYMbXiwQ3F6EQNxVDC+nEKuNkUKfiA4NA0FzcxONjdeZ\nNs0ztLWvr49duz6no+MWnd3d9A2oZGSk+Uc3FRdP4MKFr5g8eUp4LgLo7u4kO7cAs9kzYe1G/U1U\nVcHsXZPpxAkrc+a4OXHiC8yyifz8fA4c3ENewX2kpGSSm+d5JsFhJgDJm6jet3cP998/Fwk3jr5e\njh49zJw582hobGTHpx+Qnp5GT08fEyeWMWnSZL44tAeLxUJubrb/M2dnZ3PzZn0EgYj8HIey9ahA\ncKcQE+XuOiJ3FkMJN9zOEIXD4eC99zazZcuf2bz5T3R1dQHQ0HCTrKzAvgVJSUmcP3+GzMx0ysoq\nKC8vpb+/P8hGDTTjBDmA06n68w0AA44B3tn8ZxRvUn3yFDc7d1ro6emmcupU0tPSmDN3Lhcunic3\nVz+rOYDXe3C7SU5JxGw2YTJbSUhOZsDhme9w4MAeysvLyMzMpKRkAqdPn2T79m2kp6dSVTWdy5cv\n+zc+6u7uJj09Y1DPz7dTXfi54dPS0szNmzfDkuYRLAmzQyAAIRBjjvhnYEe/Fm9H1Nvbyzvv/Il3\n3vl/vPXWH7h06aL/2kcfbaOkpIjy8jLKy0vZvv19ACoqJnHjRoO/XFNTMxaLhaSkJEAiOTmZjo4O\nmpqa6e/v59Kly8x7YJHh/fsHHMy+zzNSqKmpifNffUVeXi75+XkcPXoYAHuKxpw5A6haYL6Fqmjk\n5vocZCnoX89RW/stPvhwK9u3f0DtldpAPc2MY8Chq+ejs7OT8vIy7whZifnz53Pw4CEuXrxEX98A\nCxcujvgc72Qu4cMP3+fEiSN89dUp3n33LRQlfDn0AEIYBJERIaYxyPAX2Iuf3/zmN0yZMsm/8uzR\no4eYPHmK1w7PJDHQz11ISbGzcOFijh8/iskkUVhYTEZGtmeZC5tnXaXMzEwqKqbQ19fHxo0LSUoK\nJHkDTUk4nQqlpRVcu3YNp7OfyZMmeZbKUFVcbsXfhadnyPT19dHb209yciK1V+vIyc4O6uMl/4ui\nKHz6yUcUeDfeUVWFDz78gEmTptDW1sac2Z5kffGEUm7erCUnJxuXy4XJZMbpdGKVPW319/ezcuUq\nFixYgqK42bp1C/39/WiaxuLFyygoKBzWsx+KqJw7d5aMjFTviree0NehQ/tZtGip0R2GZZ9g/CME\nYowS3zIbw1cRh8OhW3Jblj0jeDyrmGreUUq+EUuBemVlFZSVTdLNkP744w9pvNGAJJmYN28xpaWl\nulFFoQwMOJC8O6bdf/88tm37C6lpqaiayrW6azy+6Sld+aqqNXz88ReYTG5mzExnZvVMOjp6w55R\na0uLf64EQHp6GlabjcmTKklOSUZTVZxOB3Pvuw/710nUXb2CxWrlpZf+kT//+Q+kJJqxWsz09g7w\nzDPr0DT4+OOPSE9PpaDAs1zJjh0f8zd/85z3DndubkRXVyd2u93/3maz4XJ13r4bCsY1QiDuSgY3\nXyKWNzEcjyMnJ4e2tjaSkpLQNA23W0WWPf9tVq1aw/bt2/zJ1uXLH4za1qpVq+l1qJjNng17fESy\nzeVWkMyeeyUlJfPIIxs5evQIILFhwyb/vAWQuHHDRGuLzDPPzKOzC/YdUJl7PxiFl1LTUnE4Xf4z\nbrcbxa3Q1NTIBOsErBYrTpeC1WKiZtZsqqurUVUNVVV55q+/Re2l8yQmJlBWVuYVSZX+/l4yMwv8\nbSYkWOnr6/OG1YIZWXUIFZuqqmq2b99GVdV0JEniwoWL3HffAqOaI2qHYHwybIHYs2cPP/vZz1BV\nlSeeeILvfOc7I2GX4A6MRoqH5557jv/+3/8HjY2e8f4PP7zeLzh2expPPfWMbg0lH0aiZDKZ0FQ3\nmomYZQHcKljMgeeQnJzM8uUrPfMqgspfr5c4ffoCmZlNfLqjj/r6ejSTjf/5P+1869n1yBb9f/Ok\npCSmTp3O+fNnkEzQ09WDPdXOxUtfcfzkFyxZvIyMtExUVSW0I1U1jamVU7DIVt11i8WqW3fK4XCS\nmBgqDsYYJaujlI56NSUlhZUrH+Lo0cNIkomamvvIzy+IWkcgiMSwBEJRFP7t3/6N3/72t+TlpkEs\nwQAAIABJREFU5fHEE0/w4IMPUlFRMVL23ePcLpEIX8E00n1MJhOrV6/1vjPe/yGwq1roRe9SeF4B\nMJlMQaNq9NeC3wO4XA5M3vxG4B7G7Z49c4gZ07NIS53Etg+2UVRUDCbPiqyffLKXxYtX0NRkYsoU\n1V939uzZ1MyqweVysXnLn/35CHtKCseOHuOhhx7G7XYCCf7l0jUNVMWNLTnRKw6Bb/Br1qxn8+a3\ncLudaBosWLCEc+fOUFt7BZMJli9f5d8HY3AM/pt+ZmYWq1ev09knEAyFYY1iOnXqFCUlJUyYMAGL\nxcL69ev57LPPRso2AXAnQwHBnUmk4zhaiXreJAWOA+2Gv3e5VY+ghNint8WzGmtKSh+FhQW0d7Rj\nt6eiestYZBlNG+DUaROTJqu6dnxIJlOY++ITM5eiBe+qAWhYZCnEbs8EOFm28NRTT/P009/i6aef\npa+vj9raixQV5ZGXl8PmzX/yi0rsZxVO7N+BUALByDMsgWhqaqKgIOC+5uXl0dTUNGyjBKHcnj/+\nyJ3O0O5n1F6o0MhmIy8jPMyiqVrQuWCvI3RGcsBzycvPo8e7Oi1Ad08PnZ2ZzJnTHdAATW+X2Wwm\n0ZaI0+kEoL2jk4LCQjRAVfQhJsXtJsEqR7BB/+Hr6q5QVOQZxWQymcjISKelpcVfPvKzGtwwZIHg\ndjIsgYi9WYpg7DCUHsm4TnhMPXCcmGBFVVwGnaFnRFRXVxcOh9Pb50fxRoK8CastgRv1N7DIFkqK\nS6itu87XX3Xz9VctyJbLbNv2Hp9+8lGYN+A7Xv/IY9gSEhlwuKiomET1zNleuwL2eQ5VZNnirxdt\npz5NQ+cx9PX1k5SUGFZ+OEIgRERwu5G0+KZaGnLy5El++ctf8vrrrwPw61//GkmSIiaq3W4FWTYb\nXhPcO7R1dKNJ+vSX0+nkf/2vX+JyOXG73FROr2LVw2sjtBDOubPnOH78OHXX6jDJSdy82UJGegqy\nVaa0tIy+3j4Ki2awbNlMXb2uTqi9CrNmhbdpkVwU5XuW01AUNxn2BN3mR9Ho7OzklVdewWq14nQ6\nmTVrFk8//XTcn0cguBsYVpK6qqqKuro66uvryc3N5cMPP+QXv/hFxPLt7X3DuR05OXZ27NgzrDZG\ni1Wrlo6A7dGW1Y5dXl/OaInt4HOe4+XLF7B79yHd5LXACqqRX/UjmwJ7RnsmvzlxKBKSZPaX/eyz\nT8jIyPB/gTh16hSZ2RNIS08LtOVv2+RpzxR4n5GRS0trK2lp6dReu8m8B2bjdLq4fu0ax48eY2b1\nTL6+0ExBfj/efDSdnXD6tIkFC920t2vesJbmGdKKiqz243Y4URQNSXXSmmhDVVU0TUVVVVRV8x/7\n5oQE/6xbt5GbN29gt6eSlpbGvn1HvNcg3HvyPV+N5csX8vnnBwgPN0WbTxH+PS/2vuUjz8j8P7/z\njFW7wWN7S0v3kOvn5NgjXhuWQMiyzMsvv8y3v/1t/zBXMYLpzhN5nkO0UVCho4hCy+uPNc14BFPg\nmv5evpFNgWv4j202K86efvBuPaooCteu1VFdXeWvn5xko6urg9S0NPBu2hPchs9EzbtSqqpBV2cX\ni5csps+h4HK6uVpby9SpU9HQOHPqLJue/Cb1N0yASmJiQBxA4+DBfdy8cQMNKCqawNy580DyjVxy\nkJaS6B/WGtxh+46Nwk2yLFNcXGIgAoE6kUJxsTrwWOIQHRGbEsTHsOdBLFu2jGXLlo2ELYKYROvw\n76AV3s5fVaGpqQGLxUpOTk6YKAQLR/BQWE2DlOQEOnscmM0y7733NnZ7Mjdv3qSwsBBN0+jo6CQn\nNw/d8Ffvqt2aFngOnkl6GiZJIje3gOLiEo6fOsfNmzeZMtW7qQ8SM2bO4PLliyxcmMfu3TLt7SY2\nbnCiARcuXKC9rY38fM8s6LZbrVy+9DUzp1WguJ0kJ1iCPAPw5Uv0HX8ggR76oy8T/OqZpPf++++h\nqm40TWPatJksX77Q6KmP5K9QIIgLMZN6HHLkyEFu3WpFVVXmzJnnXxNoZNZw8nTOiqKwefOfsdtT\nUBQFTZPYsOFxvF04Rt5EqFAkWk3c8C7kV1RUyLVr1/nqq6/o6urhr//6WSTJ5O9Yg0NcgTbwexGa\npJGVlQ7AnJrZbN++HVWZgEn2jMNQFIWGhqtYLelY5BpKSlSamiVyc1WamhpIS0v125uenkZjww2q\nK0tJSkhANsveMJLnhpE7ft9x8E/I09ONotL4/PMdlJYW+5czOXv2S/+IquDyg0UksAUjgVjNdZzg\n6xBOnjyOyzVAeXkpkyaVc+DAbpxOR1x1Y9xBV27//n2UlU1kwoQiJk4sIS0thVOnvgzrACPH2z2h\npgSLCbfL0yGWlBRTWVlJ6cQy0tMzkU36UUT6tvSjiDQNUtMyuHyllvz8fBYvWUzd9WsMDAzQ399P\nU1MT69auIysrgRT7MebMdnPjhkRTk0RxcTHt7R3+z3arrY3iwkKSE2WsFgvBHb7He9F0nyPcYzB6\nbqHeg+fV5XLp1rqy21NobW2N+nsZXnhJIIgfIRBjjugdQktLM9nZgY1scnNzuHnz5iDa0sfYI9Vz\nOgdISAjMDLbb7XR0dBDacUYSB09nqlFYkE+SzUJLcwsul4uLFy9x/7wHAA2rxYzidgfZoqGFtHf6\n9Jfs+HQ7B/bvo6qqmqTkVE6fOU9WZh4Pr16Ny+3G7XaTl5eHySSRm5tNb68noVdTo3Cj3oTNVkbR\nhGJab7XR0nqLvNw8ZkybhM1q89sZHGKKFF4yCjPpnpqmfwVITU3z76MB0NXVTV5eXtjzHjmEoAji\nRwjEOMNiseBwBDyGjo5OMjMDm/cMJfRglJSdMWMmFy8G9ob4+uuLzJo1218u+NVIHIJFYtOmJ5lU\nXobi1li/fgNZWTlomuYZUqopYR2zTyQOHT5IQ8N1UlKScDh6+WDbVgoLCpkz9z4cDgfnz3/F0iVL\nWLp0KTNnVHH69Bk0TUNRVE8bmkZNjQtJ0phZNYs1D69j/Zp1LJ5/H1arDdlsjioKRh5E4HOHexmh\nnbOmwcKFi+nrc3DlSi0XL15m8eLlmM3mYXkPIrwkGClEDmIcoWmwZMlytm17D9/wy9LSClJTIw9j\nC04Cx7OAno/8/ALmzl3A2bOnAHjwwYdJTbX76+lfwxPXwa+gMcu7Ympv3wCqqmAymQGNpAQLvQMu\nzLIFX37DZ1dLSyN5ubkAJCQk0HrrFqqq0d/fw/nzZ/xbngIkJSfR0dHO4cNHmHvfPJ1HkJWloChu\nEixmzCZIsJrRNDcmkzVoCGvAkwgONYV6EOGeQ7goBAutJEmsXPmQoYCE1o+n4xfiIBhJhECMM8xm\nMxs2bEJVVe/cg/BePlLnHzjvEY2gKwSGuQaOS0pKKCmZGLIndXQxiCYSkiRhT0nE6XTR53Bglq2Y\nzWYsJheKqoHJV95zu66uLr9AAKjeRZiuX79OTc0saq/W+peCaW/vIDUtg/vvn49sNoGmoWoaqtuF\n1SJhS0xEUxTMJgVZlnG7FPShpWABCLyGi0L4OeMcROA4cqcerbcXSiC4/QiBGKf4doALENrpx74W\n7dtosJjoO3yjc/GKhMq2bVvp6+sF4L77F5CbPwGb1UyfwwnY/GJSd/UqJiQuXLxAfn4BLc0tFBWV\nABoZ6Rm0d7RSWFDIF0c9u9rdutXOI+s3IkngVhQkFCxmM7aURH+4SVNdJHr3vZD8z0WfoA4NNemT\n1uH5m/A8jPG5wXX44WWF5yC4HQiBGGdEDw0FhCC8XLjXEM2LMBabQJkTJ45x+fLXgER6egarV6+N\nKRKffvoJdnsSeXmeJPu+vTv51rf+DkmSMJs0Orq7MZkTkEwyddfqqJhUgdPppK2tjbz8PMyyZ1hs\nWloGX399noQEK1arjYGBAdasXoumuTBLEkmJNswmm2cmtPdhqG4H9pREvwiYTKHJaX14KdR7MMrT\nhIeHIvXiRmEko9CUQHBnEUlqgY5YSdBIHWHwcXNzM9euXWbKlMlMmTIJi8XEkSOHg8rpJ5z5Xvv6\nenQ7sCUmJtDV1YEkSaQkJTIhLwt7ohmbrJKTYaejrRmLLJGXm8VAXw/5uTl0trdycP9OykuLSEq0\nIEsaK5cuIS01iZSkRBITEjBJkqeTB1BVNLeTVHuS//NpmobZUCCMwkpG+YfQ66HPzzi0NNTEdPzi\nIVRGMDiEQIxJfJ1ShKtR+4FoSc9YHVd4WMTouK6uVjdUMyMjg7a2VkNxCI7pWyyybpJYf/8AKSl2\nXdupKYkkJ8jMmV1DUkISDTeuc7P+OnnZ2cyaOZPay+d5ePVKKspLmTu7BtnszU14e2zN25aqgeJy\nYMKF3e7xHFTVY8vp0yfZv28vbW2tuN3uKB1/+Ain4M+nf376eSRDCy0NByEOgsEjQkxjmmh5hZGt\nN5gEdllZBXv37qSiohyAW7dukZubH1RW87Yn6c6tWrWW9957B7fbBWg88MAiTCbPUNPgcFhighWr\novDgyhWoSMiy1e8VyLJZl5hPSk6kv78fm80zGU0CFMWFCY3U5AQkEzqh+nznJ0yeXIarv42DB/eS\nlJREf7/TO9Io1FsIFmojsTQa3mp8HF9oaaidvBAHwdAQAjFOiZ6LiFbOIwD689FEIfS9RmZmFpMn\nV/L11+eRJImsrGxqamZ72wyU9a2p5Dsny2aeeOKvQlaHDYxyCsZsNmNPtqCqKgMOF4oGSCbS0zO5\nfOkKFZPKcbld3LzZyNSpM3G7HUiYsJglUhJtnrkG6IewdnS0k5GZjsvloKCggBUrVgAegfvii8PM\nnRsYHqvPP4QKQvBxsNcVfhx4rz+nPx9+zbiMYYlYBQSCiAiBGPMM34uIJCahHZSRKEDoaCbP8fTp\nM5kxw7f3gtEQWM95fbI6cDePgIQLQ6hQmUwmUpIT/EuCV0+fwrkLV/ji8H7s9hQy7ckcP7IHSdIo\nLi5n6tQpgBQmDqqq4Xa7kWWZ5qZGli9d7L9LVlYW/f19hIqCPtRnJBLBzzG+8F34+eEgxEEwPIRA\njGPi9SLCy0YWHaNyweGn0HATBLyG4OPw4bAQLBig+UUiuH60JK0kSdhsNqpmzMRkkmhra6Svr5/i\n4gkAHDp0hOzsbDIzM/z1gpPQGekZHDlygDnVlRw4cIDly5cDcPLklxQWFhuEmCJ5EMYhpmi5nOgM\nxXsQ4iAYPkIgxgXxdujR68UWicjhp1CRgMCKq0Z7QugFIXDc3t7BgQN7kSSJ6dNnUFY2ydtWlE8f\nVN8kgaooSJKZhoYGqqoCe0xkZqTz+eefkJ6eTlNTE/PnL2LixDLvxj8e+1YsX8W5019w82YjW7a8\nR1JSEunpWUyfPlW3MVDAe4jkQRjlHwLvhzpiybhc2NVoFwWCuBECcc8TLUQV6MyMBAH0ievQRPZg\nPYv+/gHef/9dqqqmI0kSx44dxmyWKSkpJXan5+mQzWYzqurEpJlIS0unpeUWOTmetajOnT/Hxo0b\nAc/IpnfeeZuO9g4uXbqA0+VEkiRWLF/GokVLQnaM87zqvQb9T/DzMrQuSCwUReGDD/6CorhRVZXJ\nk6cxc2a1SEgL7jrEMNd7gMEkMuP/ZhtP2CQ8aWs8HNRzfPbsacrLS/2jkMrLyzl37rT3erSfQKhI\nkiRU742WLVvKmTNn2bdvP59/vpvU1MCeDyaTRFFREXv27iIzK4MJEwopLMxn+0fbQhLR+olxkX58\nn9PIiwj1JPbs2UVxcRGVlVOZPn0aly9/TW9v/NvxGv+Ogr0ZgWBkEB6EAE3T2LPnM/r6+gFYunQl\nKSnJwSUwWtDP2HPAMHnteR96XR9eSktL4/LlJpKTPfd2uVyYTBaCd5CL8An8CwOChKR55jsALF2y\nwltV4rPPPvKLiNPpxOl0YbVa/Ptga5oGmjtEdHyfQZ+vCN6PInZoSS/ATqeDhITAkuxpaancutVK\nUlIJsTr5kUleCwTxITyIccPQO5Zdu3ZitydTXj6R0tJiPv54m0H5gIdgFAox8jCMvIlQryHQqWqU\nl0/G5VKoq6ujoaGBCxcusXz5CsNv85GWv/DNmfDlFDQCvfi8eQv505/+zGeffcb+ffspLy/H7XZ7\n95oGTVWwyJaI9wkPK4V6EJGHtgY/o4yMTNrb2/3n29raycvL15U1IvLvUKiG4PYgPIhxRfRv2pES\n1gMDvRQWelZFNZlM/nWIIiWpw9sy8hw850O9CX258Pfr1j1Ka2srDscAhYUTMJuNth0N/1wePKOf\nzCbPSq2eaxpIUFdXx7W6WpYufZDW1ha6ujro6OjhmWf+lg+2bUXVFDTFwZrVawzFJ/g1VODCPQYM\nhCLwTO6//wH27t3FpUtX0DSNBx5YjNVqMf5wYZ9RILhzCIEYdwxeJBRF9YdefO+NlgkPbT9ayAmi\nh51854zeZ2dn++/vuUfwMNfQ+3k4ceIY/f29gEZSUiqTp9V462scPHiApqYb5OTksnfvZ0yrrGLB\ngsV+EXjiyb/C7XaTbDMhyzKehLQ+zBSeU/DdP9RDMBKKcHsXL14e8bOEfUohDoJRQgjEuGRwIrF0\n6Uo+/fRDZFnG5XIH7QxnXD7QmUXKSxhdM8pjRJ5sFxCswFyJ8M/hqXvp0kVycjKZNm0hAMePn6D+\n2lVmzJgMaFy9eoWysokAFBUVce78WabPmKkLHUmagixbDcJWPttDj+ObNR3+XCKLhhGxxUGoh+D2\nIQRi3BK/SKSmprJp01+hKG7MZt9/ifBvwuFOhZHHEHwtsucQ+VxkkQht23e/xsYbrF+/xn919uwa\n3tu6PYrdoBu2qqrYLGb0uQa9EISLBoTmUVpammlpaaGkZCLJyckhnkbgOYZ+jkgIz0Ew2ogk9bhm\ncB1QQBwil4+UvI513Sg0E+lc4BVCv8WHxv41zbP205UrV/x3PH/+PCUTivwdflZmDp2dnQC0tLRS\nVFSsq6+6HVitljARMJo57bM1NPdw+vRJnM4+ampmcPHiOW7evGH4fIyemxFCHAR3A0Igxj2xO6L4\nwhiROjrP9dbWZs6cOUVPT7dBm4H68QhFqEjoRUELas9zvrJyBpcuXeXjjz/ho48+pqmpjalTp6Gp\nniGrK1auIjsrn66uXirKJzNv3gK/eLjdLhIS9OIQKhJ6oQgPO2maiqI4uf/++8jOzmb9+nXU11+L\n8sxGKt8gVERwexEhpnuCQF4gYokIoZh4yh879gWNjTfIzc1hx46PmDVrLmVlFQahp2j5CaMwk34G\ndnBdXz1fG/PmzSewB7dn6e8Eq+zfm7t61mx/2eDVWE0oWGQrgc4/VBDQ/fjuG9yRq6qKxWLVPR9Z\nDvxpBZd1uZwcOrQfVVWZO3ceKSkphuViI8RBcPsRHsQ9xcgmRH3l6+quUFFRjt1up7JyKmfOfEnk\nb8++dkI9h9D2wz2J+vp6rly5hKoqEOJVGHXu9pQkVLcjqB39nAbFNUBKUoLBtcizp408HZPJzK1b\nbXR1dQFw/vxXWK22sKfndrt59923ychIJTc3iw8//As9Pd0RnlE0hDgI7gzCg7jn0BieJ6Gv7ymv\nr2Ayha68Gi2RHT4s1mi117/85V2sVjMWi4WDB/fzzW/+DRaL7LXBaL6Gh6QEKz39TmSLb56BJ+mt\nuJ2eDYMkdKIR7cfXtpHgLVq0jF279qGqCunpmf6lzoPLnjx5zLsFq8eWmTOrOHLkMCtWrAptMApC\nHAR3DiEQ9yQjIRL420hISKSnp4eUlBRaWlrIysqNWD6+sJNeJM6dO0daWgpZWZ5F9zIzM9mz53NW\nrnyI4DCTz/bgzybLMmkpJrp7B1A0CTSwyhJpKUbiEO4xxPIeAp/HxJw59xPcgYd6R5IkoWlqlOca\nCyEOgjuLCDHds4zEKBpPoYcffgSXS+X69QbS03N44IGFEepHSlYHrhl9U+/u7vavzwRgsVj825KG\nhpn0nbqnHUmSSE1JJMOeSEZqIinJCV4boi/ZEa/3YHQcmogHqKmZzaVLV3A4HCiKwqlTp5k/f6HR\ng4r43ASCO4kQiHuakREJSYL58xexatXDVFVV6+oPRShCO+SZM6u5cOEivvWVvv76IlVVvuWxjZLJ\nmr9spLkNejEg7H1w24FrwfYFD8X1lY3+3Mxmmccff5LeXgdtbV089tgTJCYmxXy+AsFoIUJM9zz6\ncJFhiZghp+htRa4fCHVFy0MkJCSybt0GDh3a7xWjheTnF/rLeeoFd9D6HEnw9eDOO1wc9MKgnxQX\n/Pni7bTDy5nNMvPnLxpyfYHgTiI8CIGXkfAmgtvSV4jHm9DfR+9JpKamMW/efJKTk2lubmDfvl24\nXG5d/fA8QqD98LBRuDgYC4N+tFWo1xDZezB+YPE/RyEOgtFHeBCCIALf6A2v6r7lx9sehI56Mm4j\nlvehcebMlzz55CYkSaKvr4+PP97BokVL/Z205N33IbjT1rTQdvU5A2Oh8L0P9ShC64d8gih9uhAG\nwVhECIQghOgiAcMRCjAOKenL6ifHBTr8jIw0/5DapKQkbDarV0D05TyjkwL3uXTpAs3NjZSWllNU\nNCHocxh7EcaL9IV+fi1EiCJ93mjiIMRAcHcjQkwCA+LruMJH+cTbtj4kYzQyKDxUI9Hd3esvoSiK\ndwe80AS0Pmn8xReHSEtLZv36NfT3d3P+/NkIyWnC6oeHu2J7EUIcBOMJ4UEIIhDbk9CVDurv4k9o\nRw89BYeXQKK4uIx3332PpKQkOjq6vGsq6T2I0E2INM3N1KlTALj//vv44IOPQjp+o87feD2oSJ83\nnjLhn10guPsRAiGIwtA6svhDUMZ5h8A1/RLfRUVFFBVN8OcawsUkXChCZ3kPDAxw8OBeAPLzCygt\nLSe6IEQezio6esF4Z1gC8eqrr7Jr1y4sFgslJSW88sor2O32kbJNcNcwOG/CX2tQQmG8S13we1/H\nr1/IL/AaLA4+D8Lt1rh6tY7S0omcPPklAwN9bNr0DQD27NlLU1MjOTl59PX1cvLkMQCmTZtJWlqq\noVgEf67wzyAQjC+GlYNYvHgxH3zwAVu3bqW0tJRf//rXI2WX4K5DC/oZZM248hRGBaJ1yHrcbjc7\ndnzM9u3bOHfurL/uAw8s4Pr1m2zd+iEnTnzJM88846+zZMliLl26wMBAP4cO7ePRR9exYcMjnD17\nwrt/RKj3EMm2cANFeEkwHhiWQCxatAiTydPErFmzaGxsHBGjBHc7msFPHLViJrVjjQgyns2saSrv\nvPMnMjJSKSzM4+rVi3z55QlvWxqVldNZsGAREyeW6v6PNjY2YrencurUSR5/fCNmsxlJkvjGNzZy\n7tyZkPuHC8LgkvMCwdhjxHIQmzdvZv369SPVnGDMES2fYFBaixR6Mg5nRSvf09NNYqLNv0pqUVER\n16/XhZWcOXMWe/fuIj3dEwZtb+9m8eJlnDhxlP7+fqxWz54OLpfLu7Be4N6B11BVEN6DYPwiaVr0\n70HPP/88ra2tYee///3vs3LlSgB+9atfce7cOX75y19GvZnbrSDL5mGYKxCE09/fz8svv0x5eTng\n8RqcTiff//73Dcs7HA4AbDbPng2qqvKf//mfzJ07F1mWOXjwIN/97nf91wWCe5WYAhGLLVu28NZb\nb/Hmm2/G/INqaekezq3IybGzY8eeYbUxWqxatXRM2j50u+NPaht7BpLBNSlkBFNgJNPevbvo7Gwn\nNdVOQ0MjDz+8ng0b1rFv35GI9w0OU6mqwrlzZ1EUN9OnV3n35448/2FwQ1sH9yc2Vv+vwNi1faza\nDR7bh9O35uREHlg0rBDTnj17eP311/n9738vvm0JQog/5BQ5fBT7Hr5Z10uWrKC1tZn29nbmz19G\nQoLN33b0+p5Xk8nEjBlVQXUiT4q7neIgENxNDEsgfvrTn+JyuXjhhRcAqKmp4Sc/+clI2CUYN4R2\nkMZKEC4SwcNWw9sMLMcROJednUtOTq6/PeP76+8ZmnA2EoXYs6dFwlowPhmWQHzyyScjZYfgniG4\nJ421LHioSBgLQ/gWpYH7xPIgAmWM1laKz3sQSWnBeEWsxSS4qxjMqqfhayWFHnveR/4JFYdwGyId\nx2OfQDDWEQIhGBMMrgM32ho0Upv6eRaxj43eCwTjE7EWk2AUMZ7zEL1M8PvwY/0Cf4MLMRkfDwUh\nIILxgfAgBGOSSOEl/TXfudghpvB6kdqMZ+SSQDA+EAIhuOsYyrpN0UUi2n2MBWDoQ1aFYgjGDyLE\nJBhl4g8zRV7p1Si8pN+5LnK70cQhuthEWsBPIBgvCA9CcBcwlI7VOHEcef5DtBBTvPUFgnsL4UEI\n7hL0nkS8E+dUVePUqeO4XC4yM7OoqJgSUiZyxx6PGAjvQXAvIzwIwV3EYDtZjQMH9rBgwTw2bnyU\nnJxMzp495bkSpSn9kuORxUAkpgX3OkIgBGMWTdNIT08lMzMTgGnTKnE6+wnt9EN/glrQlQs9Z3xP\nw7NDsF4guPsRAiG4y4gW3tHnCSRJwu1260q43UpYO4H34TkI/X3iPScQ3BsIgRCMSQIiIXPs2DG6\nurrYsWMHeXlFoSUZbB4iXm8i/jICwdhECITgriZWMri6ugZNk9m37xCFhWUUF5fE/U1/MF6CCC0J\n7kXEKCbBmMU3Wik/v4D8/IKwa0blg3E6HRw//gU2mxVNg9mz70O/dIdAcG8jBEJw1xNpyGuE0oQO\nl43E4cP7eeKJx7FYLLS0tLB//2Hmzp0Xr1VxlhMIxi4ixCQYE4xc6MeTk9A0laysDCwWCwA5OTnI\nsimoTOx7CwTjHSEQgruQwSWHo4uE8cglSZLo73cEtaExMDAw6HsLBOMZEWISjGH0ay4Ndl/rjIws\ntm59n7y8XGpr65g+febImygQjGGEQAjuUsLzDLFEYLAiUVY2CUUpo6+vj4ULSw3riiU1BPcyQiAE\nYxzjpHQkoQjt8M1mM3a7HRFaEgjCEQIhuIuJ14swLhf8Gv0eEa4IPRDc44gktWCcEH29YIU/AAAI\nIUlEQVTGdOQ6Ea4IcRAIhEAI7nbCe+rYmwBFKmA8ommodggE4x0hEIIxSfyho8ELgvAeBAIPQiAE\nY4A712MLcRAIAgiBEIwRBhtqGsIdIrYnVENwbyIEQjCmGSmREJ6DQBCOEAjBGMK4Fx9O5x6+y1x8\n9xQI7gWEQAjGBUMRieHMkRAI7gWEQAjGDYMRCSEOAkFshEAIxhjRO+5YHX/skFLsewgE9wpiqQ3B\nuCN0OY7BhZ+EOAgEPoQHIRiXxL8Wk67W7TBFIBizCIEQjEFuR0cuxEEgCEUIhGCMMlId+lDWZRII\n7g2EQAjGMMPt2IUwCATREAIhGOMMtZMX4iAQxGLYAvHGG29QWVlJR0fHSNgjEAwBsXS3QHA7GJZA\nNDQ0sH//fgoLC0fKHoFgiMSbSxDiIBDEy7AE4pVXXuGHP/zhSNkiENxmhDgIBINhyAKxY8cO8vPz\nqaysHEl7BAKBQHCXEHUm9fPPP09ra2vY+e9973u89tprvPHGG/5zmlgvWSAQCMYVkjaEnv3ChQs8\n99xzJCQkANDU1EReXh5vv/02WVlZEeu53QqybB66tQKBQCC4YwxJIEJZuXIlW7ZsIT09PWq5lpbu\nYd0nJ8c+7DZGi7Fq+1i1G8au7WPVbhi7to9Vu2H4tufk2CNeG5F5EFLwymgCgUAgGBeMyGqun332\n2Ug0IxAIBIK7CDGTWiAQCASGCIEQCAQCgSFCIAQCgUBgiBAIgUAgEBgiBEIgEAgEhgiBEAgEAoEh\nQiAEAoFAYIgQCIFAIBAYIgRCIBAIBIYIgRAIBAKBISOyWJ9AIBAIxh/CgxAIBAKBIUIgBAKBQGCI\nEAiBQCAQGCIEQiAQCASGCIEQCAQCgSFCIAQCgUBgyJgUiN///vesXbuWRx55hJ///Oejbc6geOON\nN6isrKSjo2O0TYmbV199lbVr1/LYY4/xD//wD3R339179+7Zs4c1a9awevVqXnvttdE2J24aGhp4\n9tlnWb9+PY888gi/+93vRtukQaEoChs3buS73/3uaJsyKLq6unjxxRdZu3Yt69at4+TJk6NtUlz8\n+te/Zv369Tz66KP84z/+I06nc+Rvoo0xDh48qD333HOa0+nUNE3Tbt26NcoWxc/Nmze1F154QVux\nYoXW3t4+2ubEzb59+zRFUTRN07Sf//zn2s9//vNRtigybrdbW7VqlXb9+nXN6XRqjz32mHbp0qXR\nNisumpubtXPnzmmapmk9PT3a6tWrx4ztmqZpb7zxhvaDH/xA+/u///vRNmVQ/OhHP9LefvttTdM0\nzeVyaV1dXaNsUWyuX7+urVy5UnM4HJqmadpLL72kbdmyZcTvM+Y8iD/+8Y985zvfwWKxAJCZmTnK\nFsXPK6+8wg9/+MPRNmPQLFq0CJPJ819l1qxZNDY2jrJFkTl16hQlJSVMmDABi8XC+vXrx8ye6Tk5\nOUybNg2A5ORkKioqaG5uHmWr4qOxsZHdu3fz5JNPjrYpg6K7u5ujR4/yxBNPACDLMna7fZStik1K\nSgqyLNPf34/b7WZgYIC8vLwRv8+YE4i6ujqOHj3KU089xbPPPsvp06dH26S42LFjB/n5+VRWVo62\nKcNi8+bNLFu2bLTNiEhTUxMFBQX+93l5eTQ1NY2iRUOjvr6e8+fPU11dPdqmxMXPfvYzfvSjH/m/\nSIwV6uvryczM5J//+Z/5xje+wb/+67/S398/2mbFJD09nRdeeIHly5ezZMkS7HY7CxcuHPH7yCPe\n4gjw/PPP09raGnb+e9/7Hoqi0NnZyVtvvcWpU6f43ve+d9d8Q4xm92uvvcYbb7zhP6fdZSucRLL9\n+9//PitXrgTgV7/6FRaLhUcfffROmxc3kiSNtgnDpre3lxdffJEf//jHJCcnj7Y5Mfn888/Jyspi\n+vTpHD58eLTNGRRut5tz587x8ssvU11dzb//+7/z2muv8dJLL422aVG5du0ab775Jjt37sRut/PS\nSy+xdetWHnvssRG9z10pEL/97W8jXvvjH//I6tWrAaiursZkMtHe3k5GRsadMi8ikey+cOEC9fX1\n/l9eU1MTmzZt4u233yYrK+tOmhiRaM8cYMuWLezevZs333zzDlk0NPLy8mhoaPC/b2xsvC2u9+3C\n5XLx4osv8thjj7Fq1arRNicuTpw4wc6dO9m9ezdOp5Oenh5+9KMf8R//8R+jbVpM8vPzycvL83tq\nDz/8ML/5zW9G2arYnDlzhtmzZ/v7vYceeogTJ06MuECMLX8QWLVqFYcOHQKgtrYWl8t1V4hDNKZM\nmcKBAwfYuXMnO3fuJC8vjy1bttw14hCLPXv28Prrr/Nf//Vf2Gy20TYnKlVVVdTV1VFfX4/T6eTD\nDz/kwQcfHG2z4kLTNH784x9TUVHBc889N9rmxM0PfvADdu/ezc6dO/nFL37B/Pnzx4Q4gCfvU1BQ\nQG1tLQAHDx5k0qRJo2xVbMrLy/nyyy8ZGBhA07TbZvdd6UFEY9OmTfzLv/wLjz76KBaLhVdffXW0\nTRo0Yy0M8tOf/hSXy8ULL7wAQE1NDT/5yU9G16gIyLLMyy+/zLe//W1UVeWJJ56goqJitM2Ki2PH\njrF161amTp3Kxo0bAU/nu3Tp0lG2bHzz8ssv80//9E+4XC5KSkp45ZVXRtukmFRWVrJhwwY2bdqE\nyWRi+vTpPPXUUyN+H7Hct0AgEAgMGXMhJoFAIBDcGYRACAQCgcAQIRACgUAgMEQIhEAgEAgMEQIh\nEAgEAkOEQAgEAoHAECEQAoFAIDBECIRAIBAIDPn/DrWuar8OYHUAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Test the whole thing on some demo data\n", "X_mix = np.concatenate((np.random.multivariate_normal(np.array([0, 0]), np.array([[1, 0], [0, 1]]), size=100), \\\n", " np.random.multivariate_normal(np.array([-3, 3]), np.array([[0.5, 0.2], [0.2, 0.5]]), size=70), \\\n", " np.random.multivariate_normal(np.array([5, 5]), np.array([[1, -0.8], [-0.8, 1]]), size=30)), \\\n", " axis = 0)\n", "\n", "N, D = X_mix.shape\n", "K = 5\n", "gmm_Gibbs_demo(X_mix, K, 1, np.zeros((2,1)), D, 1, np.eye(D));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Exercise**: Explore the collapsed Gibbs sampler on different data sets, e.g. with two components of varying size and separation. When does it work well and when does it fail to mix?\n", "\n", "**Exercise**: Implement the uncollapsed Gibbs sampler, i.e. sampling from the full posterior\n", "$$ p(z_1,\\ldots,z_N, \\{\\mathbf{\\mu}\\}_{k=1}^N, \\{\\mathbf{\\Lambda}\\}_{k=1}^N | \\mathcal{X}) $$\n", "Hint: Use conjugate priors, i.e. $\\mu_k \\sim \\mathcal{N}(\\mathbf{0}, \\mathbf{I})$ and $\\Lambda_k \\sim \\mathcal{W}ishart(\\nu, \\mathbf{S}_0)$, and exploit conditional independencies, i.e. $$p(x_i | \\mathcal{X}_{-i}, (z_n)_{n=1}^N, \\{\\mathbf{\\mu}\\}_{k=1}^N, \\{\\mathbf{\\Lambda}\\}_{k=1}^N) = p(x_i | \\mathbf{\\mu}_{z_i}, \\mathbf{\\Lambda}_{z_i}) $$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Example__: Gibbs sampling for linear regression\n", "\n", "Let $\\mathbf{X}$ denote the design matrix, i.e. values of $D$ regressors (plus bias input) for $N$ training cases collected in a $N \\times D$ matrix, and $\\mathbf{t} \\in \\mathbb{R}^N$ denote the target outputs.\n", "\n", "Linear regression then models the targets as\n", "$$ p(\\mathbf{t} | \\mathbf{X}, \\mathbf{w}, \\tau) = \\prod_{i=1}^N \\mathcal{N}(t_i ; \\mathbf{w}^T \\mathbf{x}_i, \\tau) $$\n", "with weight vector $\\mathbf{w}$ and observation noise precision $\\tau = \\frac{1}{\\sigma^2}$.\n", "\n", "With conjugate priors the conditional posterior distributions can be computed analytically:\n", "* $\\mathbf{w}$: Conjugate prior is Gaussian with mean $\\mathbf{0}$ and covariance $\\Sigma_0$\n", " \n", " Posterior is also Gaussian with covariance\n", " $$ \\Sigma_N = ( \\Sigma_0^{-1} + \\tau \\mathbf{X}^T \\mathbf{X} )^{-1} $$\n", " and mean\n", " $$ \\mu_N = \\tau \\Sigma_N \\mathbf{X}^T \\mathbf{t} $$\n", "* $\\tau$: Conjugate prior is Gamma with shape $\\alpha_0$ and rate $\\beta_0$\n", "\n", " Posterior is again a Gamma distribution with shape $\\alpha_N = \\alpha_0 + \\frac{N}{2}$ and rate $\\beta_N = \\beta_0 + \\frac{1}{2} (\\mathbf{t} - \\mathbf{X} \\mathbf{w})^T (\\mathbf{t} - \\mathbf{X} \\mathbf{w})$\n", " \n", "The program below implements the Gibbs updates according to the above formulas" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def lm_Gibbs_w (sample, X, t, Sigma_0_inv):\n", " # Compute posterior covariance\n", " tau = sample[1]\n", " Sigma_N = np.linalg.inv( Sigma_0_inv + tau*np.dot(X.T, X) )\n", " # and mean\n", " mu_N = tau*np.dot(Sigma_N, np.dot(X.T, t))\n", " # Draw new weight vector for next sample\n", " w_new = np.random.multivariate_normal(mu_N.squeeze(), Sigma_N, 1)\n", " return w_new.T\n", "\n", "def lm_Gibbs_tau (sample, X, t, alpha_0, beta_0):\n", " # Compute posterior shape\n", " N, D = X.shape\n", " alpha_N = alpha_0 + 0.5*N\n", " # and rate\n", " w = sample[0]\n", " err = t - np.dot(X, w)\n", " beta_N = beta_0 + 0.5*np.dot(err.T, err)\n", " # Draw new precision for next sample\n", " tau_new = np.random.gamma(alpha_N, 1.0/beta_N, 1)\n", " return tau_new" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEACAYAAACTXJylAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHa9JREFUeJzt3X9s1Pd9x/EnCYbiuvYZ7qhBV8BLW5KupU5wo6pOxHVz\nqmJ19W2TtZVttEzzSV21VfuB13bSCPtjmxZV7T9TJNqFpEodFVgTlU1Ii5ecW2jYmoQQhmloTe2Q\nUvPzbIomAWm+++N738vX5/vxve+P+9737vWQLHy/vvdxAu/7+P15f94fEBERERERERERERERERER\nEREREZEm9xhwEThV4rG/At4CVtd1RCIiLe6OKo/vBz5Z4v73AA8Bs76PSEREPNvE0hn3QWAL8DM0\n4xYRqatqM+5ShoE3gFd9HouIiDiwvMbntwNfwUyTWJb5NxwREamm1sB9F2bq5GT+dhJ4CbgfuLTo\niXfdZUxPT3sdn4hIq5kG3lvpCbWmSk4B7wZ6819vAPdRFLQBpqenMQwjsl979uwJfQwaf/jjaLWx\na/zhf2FOkCuqFrifAn4IvB84D+wqetyo9gYiIuKvaqmSz1R5/Nf8GoiIiDjjpqqkJaRSqbCH4InG\nH54ojx00/igIsiLEyOdrRETEoWXLlkGV2KwZt4jURSaTIZVKMTQ0xPz8fNjDiTQFbhGpi7NnzzI5\nOcmRI0fIZDJhDyfSFLhFpC7a29sB6O/vZ9++fSGPJtqU4xaRupifnyeTybBv3z5isVjYw2lYTnLc\nCtwiIg1Ei5MiIk1IgVtEJGIUuEVEIkaBW0QkYhS4RaQpNfOGHwVuEWlKzbzhR4FbRJpSM2/4UR23\niDSlqG740QYcEZGI0QYcEWkpzbwgaafALSKh8yvgNvOCpJ0Ct4jUxd13300sFiORSDA7O7voMb8C\nbjMvSNopcItIoKzZ9PT0NAsLC1y5coUHHnhg0XP8Crjj4+OMjIzw7LPPRmpBslZanBSRQKVSKSYn\nJwu329vbmZqaYuPGjYX7KlWAZDIZzp49S3t7O+Pj44ser/RYVGlxUkRCZ82mt2zZwvr165cEbYBY\nLMaBAwdKBt5KaZRWyWkXU+AWkUX8rsyw0heTk5P8/Oc/XxK0q6mURmmVnHYxpUpEZFHK4fr16xw7\ndgyAkZERDhw44Pt71JLWqJRGieomm0q0AUdEHLHnoXt6epibm6O/v9/XRT77e/j5gdBslOMWEUfs\nKYfjx48vqczwI33iZ1qjVTbahMEQkWjI5XLGyMiIkcvlSj6+bds2AzAAY2RkxNf3GB0dNbZt22Zs\n37697PsHMZ5Glf+5KloecPAWkQiwqjrKcTNbLpXTLvUeVmWI9RonKZRWXZSsxWPAReCU7b5HgDPA\nSeC7QFeJ14X9wSUiPqk2Iy/F6ax4+/btBmD09/c7vr6b8UQFDmbcThYnHwRuAN8CPpS/7yHgv4C3\ngH/K3/elEoHbweVFpBkNDQ1x5MiRqouczVgZ4oWfVSWbgMO8Hbjtfhv4XeAPi+5X4BYJUdi7ChWQ\n3alX4D4MPAWMF92vwC0SECdBWeV30VSPcsC/BW6xNGiLSICcbPXWAl55US8n9FJV8jlgCPjNck94\n+OGHC9+nUilSqZSHtxMRi5OgPD4+HqlURT1TO24qWYKSzWbJZrOBXHsTi6tKPgmcBuIVXhPmwqxI\nU2vGqop61ma7qWSpF3yqKnkK2JYP0heBPcCXgRXAtfxzXgD+tETgdnB5ERHnVSh+aOSFU/UqERHX\nMpkMTz75JLdu3SIWi/HSSy/V3Nmv2vXtqRHrvkYMpvWkwC0iSzjNJRcfgJBMJjl//rxv41DVS2lq\nMiUiSzg9fMBaAAVYtWoVR48e9XUcqnpxT4FbpMU4DZjj4+MMDQ2xatUqtmzZwuc//3lfS+da5XzI\nIChVItJi7AtzY2Nj2sjTYJykStQdUCSi3NY927v0OalnVkqj8ShVIhJRfhyU63Qjj1IajUUzbpGI\ncjsTts/UH330UXbv3l2xBK9ar26pP+W4RSLK6SaS4pRKOp1WzrqBqY5bRJYsLt64ccPVDsVSOfWw\nW8c2Iy1OijQBr8GxVEolk8mwatUq0um04+uWWshspGZN4o9wOrSINBmvzZfKNaSq9bqlGjM1crOm\nqMJBkylVlYg0OK/leNbion1GnclkePXVVwHo6+tj3759VXtUJxIJ4vH4ouuo4iQcynGLNLggOtnZ\n895r1qzh/vvv5/r16xw7dgwovWhpf01PTw9nzpxRsA6AepWINIFSM+ZaFc+mrVl8R0cHV69e5ciR\nI0xPTwPlZ/b23iVzc3Oua8elsYWdKhKRvOJ8tpX3HhwcLOSoZ2ZmKh7OkMvljJ6eHuW0A4Zy3CIC\nS/Pk1iz+4MGD9Pb28vrrr3PvvfeSy+XKXiMWi/GJT3yCRCKhFEnIFLhFWsD4+Di9vb2sXLmSHTt2\nFBYfY7EYGzZs4NKlS+RyOSYmJiqmQGZnZ7l8+XLV50mwFLhFIubuu+8mFouRSCSYnZ119JqxsTGu\nXbvGsWPHlvQ2seeurQqTctRwqjGoqkQkYmKxGAsLC4B5Ks327dur7mi0b3Pv7u7m3LlzhXTH/Pw8\nu3btwjAMHn/88YppkEoVLtpF6Q8nVSVBCjnFL9Kc4vG4ARjt7e3GzMxMyY00xfdZG2WWLVtmtLW1\nGbFYzBgcHPR1gbGep7Q3M7Q4KdJ8XnzxRZLJJFNTU2zcuHFRaV8ul1tU7hePx7lw4QK3b99mzZo1\nGIbB7du3mZ+fr5inrrYZpxSlUZpD2B9cIi0hl8sZiUSiZLnfwMBA4X6rlM/66uvrKzvjdjN7Lre1\nXmqDZtwijcvNrLaUWCxGf38/sLTcr7Ozs3D/8ePHGR4eZmhoiOHhYZ5//vmyeWg3s+dyG4X8+jml\nPsL+4BJpaG5zwqOjo0ZPT4/R3d1dyFOXm+0W379582ajq6vLiMfjxszMTMlrb9u2zRgcHDTS6bQv\ns2flvmuDgxm3ArdISNx21rMHQsAYHh52/Nqurq7C65LJZMVrVwqyVoDfvn171bGrg2BtUKpEpHG5\n7axnr7sGeOGFFwopiGppiba2tsI1HnjggSXPdZoiqeW8S3UQjJawP7hEmlIulzPWrl27aNadTqcN\nw1g6Yy5OjczMzBjJZLJsGaHTBUbNooODUiUizSmXyxltbW1L0iXFAbVSasRL8FUFSXBQ4BYJVy25\n4FpZnf3sZX3FAbV4s46dgm9j8iNwPwZcBE7Z7lsNPAucBf4TKJe0CvvnFwmd08W+atUepdgDb7kP\nCHtqRKLBSeCuth/+QeAG8C3gQ/n7/hm4kv/zb4Bu4EtlAne19xdpakNDQ45OVC/uP3L+/Pma3qf4\nJPd6Hdqr/iT+8+MEnB8AxQ16Pw08kf/+CSDtZnAircBpRYW92uPo0aM1v09Y281rqS4R/7gpB3w3\nZvqE/J/v9m84Is3F6bFjxf1HamV9QHzgAx8gnU7XtEsxk8mwbt06Vq9ezUMPPVTT7kb1JwmHk9aB\nm4DDvJ0qyWGmRyzXMPPexYw9e/YUbqRSKVKplKtBirSyWtIRblIm9tfU8joI5iDjVpPNZslms4Xb\ne/fuhSqx2U3g/jGQAuaAdcDzwN0lXqcct4gP7IG1t7eXDRs2lA3iTnPqpV4D5kEKlXqYSPCCOuX9\ne8Bn899/FnjGxTVEmk65XYu1NlkqdyJ7f38/69evr5hTdrNL0TpooVrjKYmOp4ALwC3gPLALMy0y\ngcoBRRYpV/pX7v5yJXzlTmTP5XLasdgC0AYckfqxgmpHR8ei02XKBdtSAX10dNTo7u4u2y9bm2aa\nn5PAvTzY2C0SPW5rk8fHx3nf+97HlStXCqfLxGIxrl+/Tk9PD4cOHVp0rVIVGWfPniWXMytwN23a\ntOS9rSoVP8YrUkrYH1wirnjpH108u7Zfq6enZ9FMudTsOZlMGoBx5513Gtu2bXM0s7afbFNLi1dp\nTKitq0jtvNQmj4+P09vby8qVK9mxYwc/+clPCo/Nzc3xuc99rnC7VI23VcP9q1/9isnJSUebWm7e\nvFn4/ujRozpppgUocIsU8dI/emxsjGvXrnHs2DGOHDmyKKhCodRrCauS5LXXXivc19fX5+iDY+vW\nrQDccccdXL16tWTFiY4PE6fC/o1DxDdOu/zZUyPW0WLW7S1btjg6nHf9+vXG8PDwkueWG0PxYcHd\n3d1LXqvjw6IDpUqk1fk103Tak8NKs3R3d3PixAkOHjzI8PAw6XSayclJR4fznj59mrVr1y7Zul5u\nDGNjY7z55psAdHV1ceLEiSXvo63p4lTYH1wiNc00K82q7YuOO3fuLPs8t+V6xa8rNW4nZYXWSTjV\nrl/pZw2yh7hUh+q4pdXVsmGlUpDP5XJGb2+vMTAwUKizLvU8w/An8JUat30M9mu72ZRT6WdVWiVc\nKHBLq6tlBlwtABafru7kebUGPivoDw4OGul0umKu2iovdDPLr/SzandmuFDgFnE+A64WAK2A1tfX\ntyiojo6OGj09PYXFSGtBslzgqzSeakHfqvO2vsqlRqqp9LNqd2a4UOAWqX0GXC6w7ty504jH44u2\nsxdfn/wmmEqBr9J4qs12BwYGlryXNBcUuKVV2YNvtRlwsVqbRVnBljL9RYoVB2f7WGdmZhzN+gHj\ngx/8YMX30iJjNKHALa2quNKi1KJeOeVmveXuz+VyRjqdLll7XYqTCpJKr3X6XlpkjCYUuKVVVeoZ\nUq21arkcr3V/pXJAP8bqFy0yRpMCt7Ss4uBbS2vVaio1jvJjrH6lOLTIGE1OArd2TkpTKm7gVK7/\niLWjMB6Pc+HCBUc7LK3XgNk4yuvp5mNjY1y6dIkdO3YwPz/v28npTg8qluhxcuakW/kPD5HGZR12\n+/3vf5+LFy8CMDw8zNq1a8v2uJ6fn+eee+5hbm6uprMdyyk+4PfGjRs1nxspzSOoMydFmoY1K711\n61bhvmXLllWc9cZiMc6cOVNyBu+mN0pxHxEv3Qm9UAdBAeW4JULWr19vAMa73vUuY2ZmxvXCnpuc\neaPkolWF0hhQjluaRdCzwd7eXgB++ctfsnv3btezXmv23NHRQS6XczTWRslFq4OggGbc4iM/+n9U\nqtLwq3SuuDd2lGaujTLzb3WoHFCahZfA6iToOwlaTsv0Ko1VuxmlGgVuaRpeZoNOg361oFrpA8Dp\ntnXlkaUaBW5pGE5nml5mpJWO9nIS9KsF1UofAE4Dcr13M2qGHz0ocEujcBrYvMxIvc5mq51yU+kD\nwGlArnceWTP86EGBWxqF08DmZUZay2ur9ShxmhZxEtTDpH4l0YMCtzQKp4HN/rxafs0fHR01BgYG\njJ6eHmNmZqbqeOqRFglKLf9dGvUDRcpT4JZIqyVA1hpMK/XELnUcmJf+3n4L+4NDgkXAgfvLwGng\nFDAOrFTgFj/V8mu+9dx4PF617/bmzZuNzs5OY8WKFcbJkycNw6geDIv7e4c5i1X6o7kFGbg3Aeds\nwfo7wGcVuMVPtfyav3PnTiORSBhdXV1VZ6P25ySTScMwygdDa6Ydj8cbJlgq/dHcggzcq4HXgG5g\nOXAYGFTglrA4PYHdMIxCEG5vby/kw8sFQ/t1k8mkgqUEzkngdtur5BrwVeB14AIwD0y4vJaIZ1af\njTVr1rBixQp++tOfMjIyUrJXyIsvvkgymWRqaoqNGzcC5fuF2Pt3nDp1KvR+IiLgvh/3XZiz7AeB\nBeAgcAj4tu05xp49ewo3UqkUqVTK5duJVDY/P899993H1atXuX79euH+kZERDhw44Om6mUyGffv2\nlQzamUymbN9uESey2SzZbLZwe+/evVAlNrsN3L8HPAT8Sf72HwEfBb5ge05+1i9SH/YDCQDuvfde\nnnvuOWKxWGABtvgQBC8fEiIQ7EEKP8YM1KvybzAITLm8logvrLRGLBZjaGioELQB344DK/eeaoUq\n9eTl6LIxzEqSt4CXMWfft22Pa8YtvrHPmBOJBLOzs0tmz5XSGkNDQ4EcB1YtlSJSKyczbp05KZFg\nT0nE43GuXLkCOEtPZDIZpqamePnll9myZQurV692lC5R/lrCoMAtkZPJZDh8+DA3b95k69atHDx4\nsJD6sGbMsViMiYkJR7PnTCbDgQMHWFhYWHR/IpGgv7+/7OwdYN26dczNzQGQTqd5+umng/mhRWyc\nBO4ghVcIKZFVXI9tbaKx11lX24Bi354+MDBQuNby5csNwOjo6CjcZ9V0U2LDTnd3d+Gx4eHhwH92\nEcNQrxJpQMU9QYpvWzsYAaOvr89V72578O/p6TEAo7u72zh58qQxMjKyqNeIdUhwV1fXkuZU1vOq\njaPaeERqgQK3NJriniDFt3O5nJFOp43h4WHXBx/Yt6+XOo3GPmO3z8iLZ9y1bi132/xJAV/sUOCW\nRlPcE8TP/ttWABwcHHQU+Etdwwu311K3P7FDgVsaTfEs1kvDpOLXugmAfjZscnstdfsTOyeBW1Ul\n0jSsypN4PM7mzZvp7OyMRBmfasHFTuWA0hKseuu2tjY6Ojq4fPkyx44dA8w671gsVrLEsNx1VLct\nYQpyy7tIw7C2s09MTNDW1sbs7CwAXV1dPPLII5w9e5a5uTlyuRwTExNlt7wHtS1exG8K3BKoTCZD\nKpViaGioZItVP0xPTwPQ2dnJI488UmjVurCwwO7duwv9RAD6+vrK9hRR3xERLU6KUZ+KieKSvmQy\nuag222mJoU6WkUZAgAcpiDhSj1lsZ2fnovconnHHYjGefvppnnnmmYp563KHKYg0GgVuCdT4+Dgj\nIyO+duSr9h7Fgbwe6RqRelJViTSd4vI6HXYgUeKkqmR5fYYi4k6lEj3rsenpaTZu3LiobtsenL2m\na1QmKK0k7By/NDgnPToqLW4WdxIs9RzD8L7oqC3pUk9ocVIamZO66UqzZeux4px2Ma+LjioTlEaj\nwC018XOhz0lATCQSxOPxkkF3fHyc3t5e7rnnHnp6ejh06JDr4Fzp56rHAqtIowj7Nw4JgD1tkEgk\nPLUizeVyRm9vrzEwMGBs377d2Llz55LUSbU0hV9pDK/XUWtW8QsOUiVanJSaWLNkqyeIleZwU6kR\ni8XYsGFDybMkrWvaZ+WrVq0ilUotOjD49OnThce9pDHKvY/TxUgr7WMfu0gUhf3BJQGwFvrsp8h4\nmWHaW5qWuqZ9YbF4tm99n0wmPc9yy72P09m3WrOKX1A/bgmKX9vDazlLslqQ94ubIKzt8uIXJ4Fb\nG3DEsbDrme0ba6zxBNHDWv2xJUzqxy2+8roD0e1mGpFWop2T4iuv9cylFvCsgP3qq6+Sy+UAeOON\nNxY9pxZh/1YgUg8K3FKRPRA++uij7N6923UKoVTgtwdzMA8/WFhY8PXDQaTZaAOOVGTf3fjxj3+c\nS5cusWPHDlebb0ptZLGCeV9fH+l0mpMnT3ra7KJdjtIKlOOWiqwDePv7+1m5cuWisxydzGar5bWn\npqaYnp7m+PHjhT7aXmhhUaLOSY7bixhwCDgDTAEfLXo8zIqaplevnXr2Mjc3ZXLlaqJHR0eNrq4u\nNW8SKULA5YBPAJPAY5i58ncCC0WB28PlpZIwekw7nc3aZ9m3b99mYmKC/v7+RekP+/i7u7s5d+6c\nZsgiBHvKexfwIGbQBniTxUFbAhZGLtdplz17Xryjo6Nkztoaf3d3NydOnFDQFqmB28DdC1wG9gMv\nA98A2iu+QnzVyB3r7B8q+/fvLxnsrfGfO3fOl9y2SCtxmyrpB14APgb8CPg6cB34O9tzjD179hRu\npFIpUqmUy7eTKNECoYhz2WyWbDZbuL13714IaOdkD2bg7s3ffgD4EvAp23OU424y2twiErwgc9xz\nwHng/fnbg8Bpl9eSOvF6CIKTE2tEJHheNuD8GfBt4CSwBfgHX0YkgfEaeEv1rPbjJBwRqY2XwH0S\n+AjwYeB3UFVJw/NaiWJfEJ2dndXsWyQk2jnZQvxcNLTvqGzEyhZQTl6iSW1dxbNMJsPhw4e5efMm\nW7du5eDBg8RisUhUjoSxSUnEqyAXJ6VFnD17lrm5OXK5HBMTE4W0iNPNOGFSwylpVgrcUpEV/MDs\n4BelANjIm5REvFCqRCqan59n165dGIbB448/7jgAKr8s4o5y3BIa5ZdF3FGOu4V43Vzj9/sovywS\nHAXuJlGvXY1O30f5ZZHg6MzJJlGvGa7T97GqTkTEf8pxNwk/6qqdLCjOz89z3333sX79ejo7O31d\neNSCpkjwR5dVE8KhP2Jxc7RZuWPGnDzPj6PUnL6/SDPDwdFlynE3KTc5b6dpkFLP8yPHrgVNEWcU\nuJuUmyDodEExkUiQSCRKHkXmJehqQVPEGeW4m4yVJ25ra6Ojo4P9+/f7HgRL1WhHoXeJSBQ4yXGr\nqqTJWCkLgHg8zo4dO3xf6Cs1u1YViUj9KFXSZKyg2tHRwZUrVwKp61ZKQyRcSpU0GStlYXXza+R+\n2SKylHqVtDDlnEWiSYFbRCRi1GRKfFOvJlYiUp0CtzjiZIONgrtIfShwiyNONtjUq0OhSKtT4BZH\nnJQAasu6SH1ocVJ8o0oWEe9UVSIiEjGqKhERaUIK3CIiEaPAXYLK2kSkkXkN3HcCJ4DDPoylYais\nTUQamdfA/UVgCgdH7URJ2GVtmvGLSCVeAncSGAK+SYgHWwYh7LalmvGLSCVeDlL4GrAb6PRpLA0j\n7EMBwp7xi0hjczvj/hRwCTO/3VSz7UYQ9oxfRBqb2xn3x4BPY6ZK3oE56/4WsNP+pIcffrjwfSqV\nIpVKuXy71hL2jF9E6iebzZLNZmt6jR+z5W3AXwO/VXS/dk6GxDowuL293ffzJkUkWPXcOakI3UC0\nuCnS3Pw45X0y/yUNQoubIs1NTaaakLr0iUSXugOK75Q/FwmWugOK75Q/FwmfArfURPlzkfApVSI1\nUf5cJFjKcYuIRIxy3CIiTUiBW0QkYhS4RUQiRoFbRCRiFLhFRCJGgVtEJGIUuEVEIkaBW0QkYhS4\nRUQiRoFbRCRiFLhFRCJGgVtEJGIUuEVEIkaB2weZTIZUKsXQ0BDz8/NhD0dEmpwCtw90KoyI1JMC\ntw90KoyI1JMOUvCBToUREb/oBBwRkYjRCTgiIk1IgVtEJGIUuEVEIkaBW0QkYrwE7vcAzwOngf8F\n/tyXEYmISEVeAvdt4C+AXwc+CnwBuMePQTWCbDYb9hA80fjDE+Wxg8YfBV4C9xzwSv77G8AZYL3n\nETWIqP/P1/jDE+Wxg8YfBX7luDcB9wL/7dP1RESkDD8CdwdwCPgi5sxbREQC5HXnZBvw78AR4OtF\nj/0UuMvj9UVEWs008N6gLr4M+BbwtaDeQERE/PUA8BbmAuWJ/NcnQx2RiIiIiEgreAy4CJwKeyAu\nRXlj0TswK3teAaaAfwx3OK7difkb3OGwB+LCDPAq5vj/J9yhuBLDLDY4g/l36KPhDqcmm3n7t/8T\nwALR+vf7Zcy4cwoYB1bW880fxCwNjGrg7gH68t93AK8RrY1F7fk/lwPHMVNaUfOXwLeB74U9EBd+\nBqwOexAePAH8cf775UBXiGPx4g7gF5gTsSjYBJzj7WD9HeCz5Z4cRK+SHwC5AK5bL1HfWPR/+T9X\nYM5cr4U4FjeSwBDwTYLtFx+kqI67C3Pi9Vj+9puYs9YoGsSszjgf9kAcuo65G70d8wOzHfh5uSer\nyVRlm4jexqI7MD94LmKmfKbCHU7Nvgbsxlz4jiIDmABeBEZDHkuteoHLwH7gZeAbvP0bXNT8Pma6\nISquAV8FXgcuAPOYf4/qahPRTZVYOjD/8aXDHohLXZipklTI46jFp4B/yX+fIpo57nX5PxOYH6AP\nhjiWWvVjzvo+kr/9deDvwxuOayswP4ASYQ+kBndhTrLWYM64nwb+oNyTNeMurQ34N+BJ4JmQx+LW\nAvAfmP8Yo+JjwKcx88RPAb+BuVcgSn6R//My5j+++0McS63eyH/9KH/7EHBfeMNxbTvwEub/g6jo\nB34IXMVMUX0X899DSQrcSy0D/hXz0694N2iji2NWBQCsAh7CXF2Piq9gLib1Yv6q+xywM9QR1aYd\neFf++3cCnyBav3nOYeaE35+/PYhZ5RA1n8H84I+SH2NW8KzCjEGD1DnN+RRmjuYm5l+CXfV8cx9E\neWPRhzBzk69glqTtDnc4nmwjelUlvZj/7V/BLCX9crjDceXDmDPuk5izvqhVlbwTuMLbH6BRMsbb\n5YBPYP7mLyIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiK1+n+Ua4/VRAzALgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Small demo data set\n", "N = 250\n", "x = np.hstack([np.ones((N,1)), 5+np.random.normal(size=(N,1))])\n", "t = (np.dot(x, np.array([-1,2])) + np.random.normal(size=(N,), scale=0.8))[:,np.newaxis]\n", "plt.plot(x[:,1], t, 'k.');" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEKCAYAAAD+XoUoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X20HHWd5/F35wEUci/kDjEJIXBDeIzLLoxDYAS0VXAS\nzxLAOfLguAbjZjiHOZjj7MxAAmsu6grCwjozLMwZediAgkR5WHDkIWFzFYiGxUlCNASSayJPeQLB\nXBYYSOj94/srq26nqrv6oR666/M6p86trq7u+vXvdv/qV7/6/b4/EBERERERERERERERERERERER\nEREREQHgV8DHsk6EiIjkyxbgk1knIsQAcEfWiRAZlXUCRNqsApSafG2phdeKiEgbbQE+hdWYlwJL\ngF1Ys89H3D53AHuAt4Bh4G/c9pOBlcDrwBrg44H3HQS+CTzpXnc48GFgGfAasA1Y6PYdBVwGbAJe\nBe4Gxrvn+oH3gfnAy8ArwH9xz80C/g1416VrdXNZICJSHJvxC/23sYK0BHwL+HnVfsHmnSlYAT3L\nPT7dPf4j93gQO6EcixXqPcBW4KvAPsA4YKbbdwF28jgYGAv8E3Cne64fK/S/D3wQ+HfADpdmgMXA\n7Y1/bBGRYvIK/cXAo4HtM7AaenC/YKF/KXsXtg8DX3TrK7ATiecC4JcRaVhf9d6Tsdr7KPxC/6jA\n898GbnbrA6hNX3JgTNYJEGnC9sD6W8AHsIL3/ZB9DwM+B5wZ2DYG+D+Bxy8G1qcCv4k4bj9wX9Vx\ndgMTI97rBeC4iPcSyYRu5Eq3qVQ9fgGrYY8PLD3ANRGveQFr1w/zAtZMFHyv/bDmIM+hVesvR6RL\nJBMq9KXT1Otdsx2YHnj8PayW/2lgNHZVUMba+sPe88dYs80CYF/sBOG16f8Tdg/BK9gnAHOqjn8F\n1qb/YeBC7GYv2A3h/hjpF0mUCn3pJJXAUr3dcxVW8L4O/DXwEnAWsAi7sfoC1qumFPH6N4EzsBPF\nVuB57CQB8PfAA9g9hV3YDeSZjPRTrHfPcuBa9xfgh+7va8DT9T+qSDamYje6fo11jfuK2z6A/ZhW\nu2V24DULgY3ABqx2JVIE/VhbvypS0tEmAce79XHAc1jXtsVYLaraDKwf9FjsR7AJ/QikGPpRoS8d\noN4XdBtWiINd9j6L3xYa1jZ5FnAX8B7W93kTe1/+inQr3ayV3GukVtIPnAD8wj2+BFgL3AIc6LYd\njDX7eF5i5A0zkW61BbtRHNZtVCQ34hb644AfYT0a3gRuAqZhTT9bgetqvFa1HxGRnIgzOGsscA/W\n9e1+t21H4PmbgQfd+svYzV/PIfj9lP9g+vTplaGhoYYTKyJScEPAEa28Qb2afglrvlkPfCewfXJg\n/RxgnVt/ADgfi1kyDTgSeKr6TYeGhqhUKrlaFi9enHkaGk1TT8/46qytqadnfFfmU17TpTQpTe1e\nGDkGpSn1avqnAF8AnsGPDLgIi09yPNZ0sxm4yD23HouAuB4bnn4xat5JzPDw6zSSvcPDGhckUnT1\nCv0nCL8aeKjGa77lFhERyRn1KXbK5XLWSdiL0hRfHtOlNMWjNKUrq+v9imufkoDe3j7XZNOIRvKx\nhPJdpHOVSiVosdxWoZ8j9g9trBBXoS9SHO0o9NW8IyJSIPUK/aiAa33YHKLPYxEHDwy8RgHXRERy\nqt5lwiS3rMFG5f4SOBv4EjbP6DXYdHTjsQmjZ2Bzhp6IhV9Yjk0fVz00Xc07IdS8IyK1pNG8ExVw\nbQ6wxG1fgp0IQAHXRERyrZmAa6uwOUG9eUq3488RqoBrIiI51kjAtXuwgGvDVc+FzWRU/byIiORA\nIwHX7sAPuLYda+vfhsXh8QKwxQq4BjAwMPCH9XK53NWDIUREmjE4OMjg4GBb37PeDYES1mb/GvDV\nwPZr3LZvYzdwD2TkjdyZ+DdyjyBkTlPdUNybbuSKSC1pDM46FfgZFnDNKy0WYpEzlwKHYjdszwXe\ncM8vAuZhAdcWAI+EvK8K/RAq9EWkFo3I7TIq9EWkFo3IFRGRhqjQFxEpEBX6IiIFokJfRKRA4hT6\nt2L98tcFtg1go21Xu2V24DkFXMutMZRKpVhLb29f1okVkQTEuQt8GhZ353bgOLdtMTYy9/qqfRVw\nrQVp9N6Jv796+ojkTVq9dx4HwqZzCjuwAq6JiORYK236lwBrgVvw4+kr4JqISI41W+jfBEwDjge2\nAtfV2FdtBCIiOREn4FqYHYH1m4EH3boCrgU0N9G5iIjJIuCapx8r2L0buZOxGj5YILYTgc+jgGsj\n5OvGbKP760auSN6040ZunJr+XcDHgYOAF7GeO2WsaacCbAYucvuuxwKxrccCrl2MmndERHJDAdcS\npJq+iLSTAq6JiEhDVOiLiBSICn0RkQJRoS8iUiAq9EVECqTZKJt9wDLgeeBR/DAMoCibIiK5FafQ\nvw2YVbXtMqzQPwp4zD0GG5x1nvs7C7gx5jFERCQFzUbZnAMscetLgLPduqJsiojkWLO18IlYkw/u\n70S3riibIiI51mzAtaAKtYd5hj5XhIBrIiKtyFPAtQ1Y/J1tWPC1FcAx+G37V7u/D2OxelZVvZ/C\nMIS/Ikf7KwyDSN5kGYbhAWCuW58L3B/Yfj6wDxZv/0jgqVYSKCIi7dNMlM2vYTX5pcCXsRu257p9\nFWVTRCTHFGUzQZ3dvDMWO2/H09Mznl27ftdAWkSkUe1o3lGhn6DOLvQbf+8i/E9FsqTQyiIi0hAV\n+iIiBaJCX0SkQFodnLUF2AXswUIvzMSCsd0NHIbfs+eNFo8jIiJt0GpNv4IN0joBP8ZOVDC2rtDb\n20epVIq1iIjkTasl02bgT4DXAts2YP36twOTgEFstG5Qx/beaaxHTp564zS6v3rviORNHnrvVIDl\nwNPAfLctKhibiIhkrNU2/VOArcAErElnQ9XzkcHYFHBNRKS2LAOuxbEYeBOr8ZfZOxhbkJp3cr+/\nmndE8ibr5p39gB63vj82NeI6ooOxiYhIxlo5Y0wD7nPrY4DvA1dhXTaXAocS3WVTNf3c76+avkje\nKPZOBlToR+/fqf9TkU6RdfOOSMCY2OMXSqUSvb19WSdYpJBU02+Qavrt279TvwMiWVFNX0REGlL4\nQr+RsAoKrSAinS6pQn8WNlBrI3BpQsdoi+Hh17FmiRX4Y8lqLWkaTPl4cQxmnYBQ7R7A0g5KUzxK\nU7qSKPRHAzdgBf8M4ALg2ASO02aDWScgxGDWCQgxmHUCQuXxR6o0xaM0pavVMAxhZgKbsD76AD8A\nzgKeTeBYI+zcuZOf//znsfcfNarwrVsZGtNQc5nm4BVpjyQK/SnAi4HHLwEnJXCcvXz3u9/liisG\nKJX2jbV/pfJWwimSaLtppLlseFj3U0TaIYlf0p9jTTte1M0vYIX+JYF9NgHTEzi2iEg3GwKOaOUN\nkqjpvwxMDTyeitX2g1pKtEjKvgGcjQUO/CZwZbbJEWleEo3aTwNHAv3APsB5WBA2kU61Efhb4F9I\nvwuXSEeYDTyHNeMszDgtIl9iZMVjIxYU0PMi8O9jvM8dWAhxERHJsWnA6279YKxn2Qvu8eFA3G5B\nKvSl46XVZ/FarMvmWuBe4ICI/dIc1PU54NfAHuCPa+y3BXgGWA08lZM0pZlPfdisaM8DjwIHRuy3\nheTzKc7n/gf3/FrgBLdtMzDsHn8MeAR4BTgam8/5Zwmnqwz8Hsub1cAVLR6vnluxqUrX1dgnLJ+y\nTFOZdPMI7H7jCuw39yvgKxH7pZlXcdJUJv28atgZ+CeYq91SbTTWHNQPjAXWkOygrmOAo7AMrlXA\nbsYKvjTESVPa+XQN8Hdu/VLC/3eQfD7F+dyfAX7i1k8CfhF47g7gq8D/BD4PfBv4S+A2tz2OsJp+\nnHSVSfe+1mlY4RRVwNbKp6zSVCb9e3+TgOPd+jisSbqR71RWaSrTQl6lVdNfBrzv1lcBh4TsExzU\n9R7+oK6kbMBqr3Gk1Uk8TprSzqc5wBK3vgTrxRIlyXyK87mDaV2FXZVMdI9/CnwCK3wG3eMyVtP/\naQPpqL6RG/f/keZAg8fxm7PC1MqnrNIE6Uf93YadpMGmen0Wa/4LSjuv4qQJWsirLIakzsM/cwaF\nDeqakkqKaqsAy7FeSfPr7JuGtPNpInZZjvsb9YVPOp/ifO6wfbwKhlfofwBr2nkCa5bpwy6Raxnj\nXjcaq81/AP+3EyddFeCjWPPAT7DwJFmqlU9ZyTqP+rErkVVV27PMq6g0tZRX9frpfwD7seyLdb/8\n31hvnD7gbuAw/CkRf4hdmkwAxruEbcXapB5073c58C5wZ8ixkugKt8ylqdqiQJrqOQX7HBPc+23A\nai1ZpSnNfLo85NhRx293PlVrZCKAsNdtxNr1vTTtwga67Ijx3jcDXww8vhy4ELg9Zrr+FWurfQvr\n2XY/1oyXpah8ykqWeTQO+BGwAKtdV8sir2qlqaW8qlfov4PVjt5y+z4BnIpd8izD2nsvBS7D2u1n\nYAX6VOwMuRzr2wz2I/kM8KmIY8UZ1NWoM1p8PVhBBrATmxN4Jq0VZq2mKe182o6dELYBk7FCMky7\n86lanM9dvc8hbpun+jL5xJjHvtAtzaZrOLD+EHAjVnHKKphQvXzKQlZ5NBa4B/geVnhWyyKv6qUp\ntbzaD/i/wIexWpx3mT/JPQa7Cgj2XngYOBm7jP41cFCN9x+D1bz6sauKpG9QelYAH4l4bj+gx63v\nDzwJfDrjNKWdT96JHezkHnYjN418ivO5gzfdTiadG5Rx0jURv7Y4Ez8YYZL6iXcjN618gtppyiKP\nStjV2v+osU/aeRUnTYnn1SjsizyMFQAw8oZMKfD4H4G/CDx3MxaLZyPwW/wuRje65w/GvxKAdAd1\nnYO11b2N1WIfCknT4dhnX4N1n8pDmiDdfOrDrtiqu2xmkU9hn/sit3hucM+vpXavrKDTsO939bKr\nTen6Kyxf1gArscIjSXdh9y3exb5P82hPPiWZprTzCKzV4n13TK9smk22eRUnTanl1QHYWe4T7H0X\n3rusCCv0P1v9RtOnT48zW4kWLVq0aBm5bKJFjfTe+T1Ws/sIfjsvjGznjdX+NTQ0RKVSydWyePHi\nzNOgNHVXupQmpandC22ITlyv0D8I/3L+g9gNv9XYwIC5bvtc/JsNDwDnY22b07DAa0mPYhURkZjq\n9d6ZjA1MGOWWO4DHsIJ/KfBl/C6bAOvd9vXYLBkXY5ckIiKSA/UK/XWE37j4HXB6xGu+5ZaOUi6X\ns07CXpSm+PKYLqUpHqUpXVnNQVdx7VMiIhKTm1e6pXJbM4OLiBRIvUI/KsznADbqMNiP1LMQ65e/\ngXQGMomISEz1LhMmuWUNFgvil1iUxXOxwSvXV+3vhWE4ET8Mw1H4ETY9at4REWlQGs07YWE+vQiC\nYQc+Cxt59x7Wq2cTNkxYRERyoJE2/X4szKcXe+ISbFjyLYwcmh8MNpWX8MgiIkL9Lpue6jCfNwFf\nd899A7gO67MfJrQdZ2Bg4A/r5XK5q7tIiUhn6O3tY3j4dXp6xrNrV1ZBUH2Dg4MMDg629T3jtA2N\nBX6MBf/6Tsjz/Vgc+OOwKIzgR2J8GJtebq9JANSmLyJ5Y23mFaBEHsuoNNr0S1jzzXpGFviTA+vn\n4IdLVRgGEZEcq9e8cwrwBeAZ/CnlFgEXYJP3VrAJsb2QnwrDICKSYxqRKyLiqHlHRES6igp9EZEC\naTYMQx82MXr1FHqgMAwiIrnVbBiGLwGv4k+aPR7rrqkwDCLSsdSmHx2GYQ42uQru79luXWEYRERy\nrJkwDKuAidg8ubi/E926wjCISMfp7e3zatFdr5EwDPdgYRiGq57zZmmPojAMIpJrw8Ov4zXr5Eme\nwjBsAMpY889k7GbvMSgMg4h0oGBbftHb9KPCMDwAzHXrc4H7A9sVhkFEJKfqnTFOBX6GhWHwTnsL\nsYJ8KXAodsP2XOAN9/wiYB4WhmEB8EjI+6qmLyK5UaSavsIwiEjhFanQ14hcEZECUaEvIlIgcQr9\nW7G++OsC2wawPvir3TI78JzCMIiI5FScQv82YFbVtgpwPTZY6wSsOydYGIbz3N9ZwI0xjyEikiNj\nKJVKlEolenv7sk5MW8UpkB8HXg/ZHnYzQWEYRKQL7MYbd2oDt7pHK7XwS4C1WD9+L8qmwjCIiORY\n3DAM1W4Cvu7WvwFcB3w5Yl+FYRARaUJWYRjAgq09CBxX5zmFYRCRjhPWT9+vr+anz36W/fQnB9bP\nwe/ZozAMItIxihRd0xOneecu4OPAQcCLWM29DByPnQo3Axe5fddj4RnWY3dCLqZ2BE4RkczkNbpm\nkhSGQUQKK7pZR807IiLSBVToi4gUSLNhGPqAZcDzwKP4/fRBYRhERHKr2TAMl2GF/lHAY/hdNRWG\nQUQkx5oNwzAHWOLWlwBnu3WFYRARybFma+ETsSYf3N+Jbl1hGEREcqzZMAxBFWr3xVcYBhGRJuQp\nDMMGbIDWNmx07grgGBSGQUQ6iPrpx/cAMNetzwXuD2xXGAYR6SJjuiqufjNhGL6G1eSXYpE1twDn\nun0VhkFEuozF1h8e7o5QDQrDICKFFbd5x9uWdbmlMAwiItIQFfoiIgXSapfNLcAuYA82IGsmFqLh\nbuAw/Pb+N1o8joiItEGrNf0K1nXzBPyRt1EhGkREcqGIk6d4Wv3Um4E/AV4LbNuA9fbZDkwCBrE+\n/EG6kSsimal/A1c3cqNUgOXA08B8ty0qRIOIiGSs1Tb9U4CtwASsSWdD1fORIRoUhkFEpLYswzDE\nsRh4E6vxl9k7REOQmndEJDNq3mnOfkCPW98fmzBlHdEhGkREJGOtnDGmAfe59THA94GrsC6bS4FD\nie6yqZq+iGSmyDV9hWEQkcJprtAfC+ymp2c8u3b9LvU0Q/bNOyIiHcPrm998/3wv8Fr1RIKdRTV9\nESkEv3YP9Wv4tffNqvzKc01/FtZ9cyNwaULHaKt2d4tqB6UpvjymS2mKJ49p6mZJFPqjgRuwgn8G\ncAFwbALHaas8fvGUpvjymC6lKZ4k0/TOO++wcuVKVq5cmdgxOk075sitNhPYhPXcAfgBcBbwbALH\nEhGJdO+99zJv3gLGjp3Sxne1mbSyvKHbiiRq+lOwGbY8L7ltIpm46qqrmD9/fv0dpevs2bOHSmUK\nb75ZbuO7dvYN3SRu5P451rTj/cq+AJwEXBLYZxMwPYFji4h0syHgiFbeIInmnZeBqYHHU7HaflBL\niRYRkfwYg52N+oF9gDV0wI1c6RqXYpWMXVgPsk8CA8AdgX2+CPwWeBW4Arv/9En33ADwQ7f/LuAZ\n4EhgIRY19rfAGYH3+hKw3u07BPxluz+QSCeYDTyHNeMszDgtUhxHAy9g8ziAhQI5HAsG6BX6M4Bh\n4KPYEMtrgXcZWei/jRXso4El2ElhoXv8n4HfBI75GSwkCcDHgP+HTSokIiIJOwKrjX8KK9A9A/iF\n/tewOFGeDwL/xshC/5HA82diJwnv/lcP8D7QG5GG+4CvNJN4kTSkFYbhWqzL5lrgXuCAiP3SHNT1\nOeDX2Py+f1xjvy3YJf5q4KmcpCnNfOrD5kp4HngUODBivy0kn0/1Pvcmd/wfY7X1h7Hw3kEHM/Ie\n09uMnPkNYEfV86/iD8982/0d5/7Oxr7be9xyJvBHVe9XBn7v0rYaa1JK0q3YyW9djX3+AcvHtaRz\nZVIvTWXSzSOw+40rsN/cr4g+WaeZV3HSVCb9vGrYGfgnmKvdUm009qPtx2ppSd8LOAabx3cFtQvY\nzVjBl4Y4aUo7n64B/s6tX0r4/w6Sz6c4n/szwE/c+iewwvp2RjbvxKnpB9v/T8c+m2cMVtM/GNgX\neAubSOhwl643gBur0lXGQo6n5TSscIoqYIP5dBLwixykqUy6eQTWDHi8Wx+HNUnX+k6lkVdx0lSm\nhbxKq6a/DPuhAKwCDgnZJzio6z38QV1J2YDVXuNIK0ZRnDSlnU9zsHZt3N+za+ybZD7F+dz/CZu6\nc1/gSayAHlu1zz1YbfxPsY4GAzSf7n3c8gJ2Yjgdm2fi6JB904xz9ThQqxN58H+6Crt6S3pa03pp\ngvRjgW3DKg9gE0A9i53Mg9LOqzhpghbyKosom/Pwz5xBeR3UFTYPcJbSzqe4cx4nnU9xPvdkbJzI\nTqz2/R5wWyB9YJfNl2AnjVew9vodWG3f2686mlbU42H3/scBv8NCjjyNP7lQcP+PYs0DP8FuJmcp\nLC/DKmJpyjqP+rErkVVV27PMq6g0tZRX9frpT8Uujz/kDvTPWPvWANaLYafbbxHw19ilyQRgvNt/\nK9Ym9aDb73Ksp8SdIcdKImzdMvyeHEGLAmmqJ2we4MczTFOa+XR5yLGjjt/ufKoW53MPY2l+0j1e\njjXxXFm13xL82ts4rPnHa+ev3nc51nTj2Y01NXkedn+rByMG/Sv2W3oLuwdwP9aMl6XqmmLWYW+z\nzKNxwI+ABVjtuloWeVUrTS3lVb1C/z3gq9jlxjjgl9gPugJc7xbPQ9gZ506XoCnYD+Zf3PMXYu1j\nn4o4VpxBXY06o/4udW11f3diPTNm0lph1mqa0s6n7dgJwZvzeEfEfu3Op2pxPnf1Poe4bdXOBB7D\nfsz/HbsB/dsE0zUcWH8Ia/Pvw64OshA3n9KUVR6NxZr8vkf41K5Z5FW9NLWUV/Wad8Lal7xL6rA2\npbOAu7CTxRasDXYm1uvib93z70Qc62lsEEw/1k56Hund2IlqH4uaBzgNUWlKO5/izHmcRj7F+dwP\nYAOvAE7GbqpuZ29zsB/uy1g4kPMTTtdE/P/nTLeeZaSuuPmUpizyqATcgg2u+07EPmnnVZw0pZZX\n/VhtyLsc3oK1Kd2C343vH4G/CLzmZqyNdaN7rdfFyOvdcDD+lQCkO6jrHKyt7m3s5PZQSJoOx056\na7DuU3lIE6SbT33YFVt1l80s8insc1/kFs8N7vm11O6VlWa6/grLlzXASqzwSNJd2P2Kd7Hv0zyy\nz6d6aUo7jwBOxTqYrMEvm2aTbV7FSVMqeTUOq9F4PTc+hJ1dSsA3sYIfwgv9z1a/2fTp0722YS1a\ntGjREn/ZRIvi9N4Ja1/aEUjEzdglBsRs/xoaGqJSqWipVFi8eHHmacjLorxQXigvai+0ITpxvUI/\nqn0pOMrxHPz22wew9tF9sHgkR5L8KFbpAMFJqaOWK6+8su4+jS69vWmNqxPpDPV675yCdUHzhteD\ndS28ABs1VsEGpXhtTeuBpe7vbuBit48UnE04Ue+rMOCWdh437fE+IvlWr9B/gvCrgYdCtnm+5RaJ\noVwuZ52EHClnnYDc0PfCp7xor6yqQRXXPiUFUSqVyOair4S+a9It7HfUWrmdRRgGERHJiAp9EZEC\nUaEvIlIg9Qr9qID+tSbWWIiNwN2ADccXEZGcqHdDYJJbggHXzsYmg34Vm2DjUiyq5mX4AddOxA+4\ndhR+LH2PbuQWjG7kirQujRu5UQHXoibWiAq4JiIiOdBIm34/fkD/qIk1qucfzctEKCIiQv3BWZ5x\nWPydBYyM5Qx+DJ4ooc8NDAz8Yb1cLmsAhiRkjHdJnJqenvHs2pVl5GTpFoODgwwODrb1PeP8GsYC\nP8ZG4XrxdzZgwye9iTVWYJN6X+ae9ybPfhgLw7zXdF9qZ81Ob2+fC4uQtmza9NM/ru4jSDLSaNOP\nCrgWNbGGAq51AD8OTpqLiORBvTPGqcDPsIBr3i93IVaQLwUOxW7YnovNKAMWkG0eFnBtAfBIyPuq\npp+hbHrSZNd7RzV96RbtqOkr9k4BqdBP/pj6fksSFHtHREQaokJfRKRAVOiLiBSICn0RkQKJU+jf\nio26XRfYNoCNtl3tltmB5xRwTUQkp+LcBT4Ni7tzO3Cc27YYG5l7fdW+CrjWAdR7J/lj6vstSUir\n987jQNjwzbADK+CaiEiOtdKmfwmwFhux68XTV8A1EZEcixtwrdpNwNfd+jeA64AvR+yrgGsiIk3I\nKuAaWFjlB/Hb9KOeU8C1DqA2/eSPqe+3JCHLEbmTA+vn4PfsUcA1EZEci9O8cxfwceAg4EWs5l4G\njseqUJuBi9y+67FAbOuxgGsXoxCLIiK5oYBrBaTmnaSNxeo86dHELcWgKJvSFBX63XlM/aa6n6Js\niohIQ1Toi4gUiAp9EZECaTbgWh+wDHgeeBR/RC4o4JqISG7FKfRvA2ZVbbsMK/SPAh7DH5Q1AzjP\n/Z0F3BjzGCIikoJmA67NAZa49SXA2W5dAddERHKs2Vr4RKzJB/d3oltXwDURkRxrR9NLhdqdktV5\nWEQkJ5qNsrkdmARsw+Lw7HDbXwamBvY7xG3bi6JsiojUlqcom9cArwHfxm7iHuj+ejNnzcSfOesI\n9q7ta0RuhjQitzuPqd9U92vHiNxmAq59DQudvBSLob8FONftq4BrIiI5ptg7Gert7WN4OGwmyjQU\nofab1XFV05dkKOBah8ummQWKUxBmdVwV+pIMBVwTEZGGqNAXESmQZrtsikiujPEu/VOlyVs6T6vf\nki3ALmAPFnphJhaM7W7gMPyePW9UvU5t+qhNv3uPW5Rj2nH1W05PHtr0K9h8uSfgx9iJCsYmIiIZ\na0ebfvVZJyoYm4iIZKwdNf3lwNPAfLctKhibiIhkrNUbuacAW4EJWJPOhqrn6wVjExGRFLVa6G91\nf3cC92Ht+lHB2EZQwDURkdqyDLgWZj9gNDAM7I9Nm3glcDrhwdiC1HsH9d7p3uMW5Zh2XP2W05N1\nGIZpWO0e7Irh+8BVWJfNpcChqMtmTSr0u/W4RTmmHVe/5fRkXei3IneFfnbBz1QQdt9xi3JMO27e\nfsvdTIV+GynGfDceM6vjFuWYdty8/Za7WR4GZ4mISAdRoS8iUiAq9EVECiSpQn8WNlBrI3BpQscQ\nEZEGJVHojwZuwAr+GcAFwLEJHKdLDGadgBwZzDoBOTKYdQJispDOaS69vX1Zf+iOlkQ8/ZnAJqyP\nPsAPgLOAZ+O8+Oqrr2H79lcTSFa0DMKQBwxigUpFeRE0SGfkxW6S7zU04BYzPJzpD7bjJVHoTwFe\nDDx+CTj/xv3kAAAC/UlEQVQp7osXLVpIpfLfSPN2w5gx99XfSURyQhPGtCKJQr+l0/6oUaPYf/8n\nSHMIwbvvvsLu3akdTkRaksbVxd665QojiU9xMnYtNss9Xgi8j8Xi8WwCpidwbBGRbjYEHJF1IqqN\nwRLWD+wDrEE3ckVEutps4DmsRr8w47SIiIiIiEir6g3K+htgtVvWYXdkDoz52k7TSl5sAZ5xzz2V\ndEJTUC8vDgIexpoCfwVc2MBrO00rebGFYn0vxmPh29cCq4APN/DaTtNKXmwho+/FaKw5px8YS/22\n/P+Iza/bzGvzrpW8ANiMzUvQDeLkxQA2FwNYofcadm+oiN+LAcLzAor3vbgW+K9u/WiKXV5E5QU0\n+L1oZ2f44KCs9/AHZUX5PHBXk6/Nu1bywtMd/cPi5cVWoNet92IF3e6Yr+0kreSFp0jfi2OBFW79\nOaxQ/FDM13aSZvNiQuD52N+Ldhb6YYOypkTsux/wZ8A9Tby2E7SSF2CdkJcDTwPzk0hgiuLkxXex\ny9VXsMvXBQ28tpO0khdQvO/FWuCzbn0mcBhwSMzXdpJW8gIa/F60c3BWI6MlzgSewJ9GsdtmYWgl\nLwBOwWp8E4BlWFvf421LXbri5MUi7JK2jI3fWAb8hwTTlJVW8mKY4n0vrgb+Hv++12pgT8zXdpJW\n8gLgVKySEOt70c6a/svA1MDjqdgZK8z5jGzOaOS1naCVvAD7YQPsxG7ezGxr6tIVJy8+CvzQrQ9h\nbZRHu/2K9r2Iygso3vdiGJgHnAB8ESvUhmK+tpM0mxe/cc+94v6m/r2IOyjrAKyd8oNNvLZTtJIX\n+wE9bn1/4Eng00klNAVx8uJ6YLFbn4h94ftivraTtJIXRfxeHOCeA2u2+F8NvLaTtJIXmX8vwgZl\nXeQWz1zgzpiv7WTN5sU07J/uddkrQl4cBDyItVuuw25s13ptJ2s2Lw6neN+LP3XPbwB+hBV8tV7b\nyZrNi24sL0RERERERERERERERERERERERERERERERERE8uf/A1u77qirzOmAAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Hyper parameters\n", "N, D = x.shape\n", "alpha_0 = beta_0 = 0.01;\n", "Sigma_0_inv = np.linalg.inv( np.eye(D) )\n", "\n", "# Gibbs samples\n", "w = np.reshape(np.zeros(D), (D,1))\n", "sampling = GibbsSampling([lambda s: lm_Gibbs_w(s, x, t, Sigma_0_inv), \n", " lambda s: lm_Gibbs_tau(s, x, t, alpha_0, beta_0)],\n", " [w, 1.0])\n", "\n", "samples = [sampling.sample() for _ in range(1000)]\n", "\n", "# Plot histogramm of weights and precision\n", "ax = plt.subplot(311)\n", "plt.hist(map(lambda x: float(x[0][0]), samples))\n", "plt.title('intercept')\n", "plt.subplot(312, sharex=ax)\n", "plt.hist(map(lambda x: float(x[0][1]), samples))\n", "plt.title('w_1')\n", "plt.subplot(313)\n", "plt.hist(map(lambda x: 1.0/np.sqrt(float(x[1])), samples))\n", "plt.title('sigma');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Exercise**: Define a Gibbs sampler which samples each weight individually. What are the advantages/disadvantages of this approach?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Slice Sampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Slice sampling is an example of an auxiliary variable method. In this case, the state space is enlarged to $x, u$ and sampling from a target distribution $p(x,u)$ instead. To obtain samples from $p(x)$ the $u$-component can simpy be dropped as\n", "$$ \\mathbb{E}_p[f] = \\int f(x) p(x) dx = \\int \\int f(x) p(x,u) dx du \\approx \\frac{1}{N} \\sum_{i=1}^N f(x_i) $$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Slice sampling aims to sample uniformly from the region below $p(x)$, i.e. $p(x,u) = Uniform(\\{x,u: 0 < u < p(x)\\})$. It does this by interleaving two Gibbs steps:\n", "\n", "1. Sample $u \\sim p(u|x) = Uniform(0, p(x))$\n", "2. Sample $x \\sim p(x|u) = Uniform(\\{x : u < p(x)\\})$\n", "\n", "The second step requires a method to sample uniformly from the so called *slice*, i.e. the set of all $x$ with probability at least $u$. In high-dimensional spaces this is often infeasible and slice sampling is usually formulated for a 1-dimensional $x$. In this case, the slice can be obtain by an adaptive procedure which expands and shrinks a range around the current sample. In this sense, slice sampling can be considered as a Metropolis-Hastings sampling which adapts its proposal distribution to the width of the probable region.\n", "\n", "**Exercise:** Show that the slice adjustment in the algorithm below leaves the uniform target density invariant." ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "collapsed": false }, "outputs": [], "source": [ "class SliceSampling (Sampling):\n", " def __init__ (self, log_p, x, w):\n", " self.log_p = log_p\n", " self.x = x\n", " self.w = w\n", " self.samples = 0\n", " self.evals = 0\n", " \n", " def _log_p (self, x):\n", " self.evals += 1\n", " return self.log_p(x)\n", " \n", " def sample (self):\n", " self.samples += 1\n", " # Slice sampling as in MacKay pp. 375\n", " log_px = self._log_p(self.x)\n", " log_u_prime = np.log(np.random.uniform(low=0, high=np.exp(log_px)))\n", " # Create interval\n", " r = np.random.uniform()\n", " xl = self.x - r*self.w\n", " xr = self.x + (1-r)*self.w\n", " while (self._log_p(xl) > log_u_prime): \n", " xl -= self.w\n", " while (self._log_p(xr) > log_u_prime):\n", " xr += self.w\n", " # Main sampling loop\n", " while True:\n", " x_prime = np.random.uniform(low=xl, high=xr)\n", " log_px_prime = self._log_p(x_prime)\n", " if log_px_prime > log_u_prime:\n", " self.x = x_prime\n", " return x_prime # Found new sample\n", " else:\n", " # Adjust interval\n", " if x_prime > self.x:\n", " xr = x_prime\n", " else:\n", " xl = x_prime\n", " \n", " def __str__ (self):\n", " return \"Slice sampling: %d evaluations for %d samples\" % (self.evals, self.samples)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEACAYAAAC57G0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYFFW6x/HvMGQYFBREiQoKqCBBkV0UWsmogOBKUkBX\nxOuKG4xcdZ1dWV3XrFwVI0GSK2ExIALSihgQJSkgDEqUVQQUGFFmmL5/nAaapnuqQ82c6u7f53nq\nmQ5Vp16g5+XtU6fOARERERERERERERERERERERERERGRjNUdWAusB+4oZr/zgEKgX8hrG4GVwDJg\nSQnFJyIiCcgG8oCGQDlgOdAsyn7vAm9wdIL/BqhRsiGKiEgkZRzeb4tJ8BuBAmAq0DvCfiOB14Ad\nEd7LSiI+ERFJkFOCrwNsCXm+Nfha+D69gWeCzwMh7wWA+cBSYHjiYYqISLzKOrwfcHgf4HHgzuC+\nWRxdsbcHtgM1gXmYvvxF8YcpIiLxckrw24B6Ic/rYar4UG0wXTcAJwI9MN05szHJHUzXzUxMl89R\nCb5Ro0aBDRs2xB24iEiG2wA0TqaBssFGGgLliX6R9ZCXgb7Bx5WBnODjKsBioGuEYwKp4N5777Ud\nQkwUp3tycqoHMN9MXdlycqqXSJyp8HcZCChOtxFDD4tTH3whcBMwF1gNTAPWACOCW3FqY6r15cAn\nmBE27zgFJOIVe/fuBu7FrRxv2hMpPU5dNABzgluosVH2vSbk8ddAy0SCEhGR5DlV8BLk8/lshxAT\nxek2n+0AHKXK36XiLH1eGKMe7E4S8ZasrCxiG0gWc4vosy5uMZ/P4nO4KngRkTSlBC8ikqaU4EVE\n0pQSvIhImlKCFxFJU0rwIiJpSgleRCRNKcGLiKQpJXgRkTSlBC8ikqaU4EVE0pQSvIhImlKCFxFJ\nU0rwIiJpKpYE3x2zWPZ64I5i9jsPswJUvwSOFRERlzkl+GxgDCZRnwkMJPKarNnAg8DbCRwrIiIl\nwCnBtwXygI1AATAV6B1hv5HAa8COBI4VcUW1ajXIyspybRNJdU4Jvg6wJeT51uBr4fv0Bp4JPg+E\nvO50rIhrzKLW7iyQ7e5KTiJ2OC26Hcun/HHgzuC+WRxZQirm35Dc3NzDj30+X1qtiSip5Xh2cxEL\nac4qqgP7uJuvaIIfH1upZzs8yWB+vx+/3x/XMU7fQ9sBuZh+dIBRQBGmv/2Qr0PaORH4GRgOfB/D\nsaA1WcUlyayh2ozV3MN99GAOH/JbltGKndxPDrmcxZdczLusojl/56/4uSjRCLUmq7gmljVZnRJ8\nWeAroBPwLbAEc7F0TZT9XwZeB2bEcawSvLgikQRfkf38kzsZyBT+xe28xLXspsahFg+3V44DXMmr\n5JLLl5zFcJ5nB7XijVAJXlzjxqLbhcBNwFxgNTANk6BHBLdEjhXxhEbk8THtOInvaMYaHuHWkOR+\ntALKM4mrOJPVfMlZLKclF/J+KUcsEh8vDBVQBS+uiKeCP4flvEVP/sFdPM2NRP5ViN5eZ+YxicGM\n5ClepX+sEaqCF9fEUsE7XWQVSTut+Jw59OBGnmbGUfflxW4+XejCPObQgyLK8Bq/czlKkeQpwUtG\naUQeb3ApN/Ass7g8qbZWcg49mMM7dGUvOcw9PJ5AxBvURSNpw6mL5jh+5FPO4yFu43muj6XFYts7\n5Dd8yCz60JH3WFvszdrqohH3uDGKpjQowYsrik/wAWZyOVupy0jGxNpiMe0dbRgvM4oHaMsSfuL4\nqO3psy5uUYIXz6tWrUbwDlS3RP4s3ca/6MsMOvIeB6gQY1vxDbv8P26kOrsZxGSiXbTVZ13cogQv\nnpfMzUkRWovYVis+522604bP4rwbNb7YKvEzn9GG0dzNZAZHbE+fdXGLErx4Xkkn+HIcYCnn8hC3\n8QpXJ92ek5Ys4x26ci5L2UyDY9rTZ13c4saNTiIp7R7uYyMNeYWrSuV8y2nF4/yJ/+MPaMIysU0V\nvFhVkhV8M1bzHh1pwUr+y8lJtxerchxgOS25m9HMpO9R7emzLm5RF414Xskl+ADz6cws+jCGkS60\nF58LeZ9JDOZMVrOPnMPt6bMublEXjWSsfkynJjt4hv+xcv5FdGA+nbmH+6ycXwRUwYtlJVHBV2Q/\na2nKECbwPh2Tbi9RtdnOF5xNGz5jEw1RBS9uUgUvGekmxvAZbZJM7sn7LyfzJDdzP/9rNQ7JXKrg\nxSq3K/jj2M16TqcD7ztMGxBbe8nGVpl81nEGfZnBEtqpghfXqIKXjHMrD/M6l7mQ3N3xM1XIJZd/\ncJftUCQDqYIXq9ys4E8iiy+pQSuWsYX6LrToTmxlKeArmjCUb1ikz7q4xK0KvjuwFlgP3BHh/d7A\nCmAZ8Blwcch7G4GVwfeWxHAukYSNAiYwxKXk7p5CyjGau8m1HYhkHKcKPhuzrmpnYBvwKceuq1oF\nyA8+bg7MBBoHn38DtAF2FXMOVfAZzK0KvhbfsYbanMW3Cd7UFIl73y7KUsBaytPovfegQwdX2pTM\n5kYF3xbIw1TiBcBUTMUeKj/kcVXgh/A4HM4hkrQ/8xhTwMXk7i5TxQP33ms7FMkgTgm+DrAl5PnW\n4Gvh+mCq+jnAzSGvB4D5wFJgeOJhikR3PLsZzvP8y3YgDiYCbNoEH35oOxTJEE5L9sX6/XRWcLsQ\n8zluEny9PbAdqAnMw/TlLwo/ODc39/Bjn8+Hz+eL8bQiZtz761zGZsbbDqVYBwFuuQUeeghmzrQd\njqQYv9+P3++P6xin7pN2QC4cXmxyFFAEPFjMMRswXTs7w16/F9gHPBL2uvrgM1iyffCVyecbTuVC\nFrGOpkm1dSw3x+ib9gL5+dCwISxaBE2aOB4hEo0bffBLgdOBhkB5oD8wO2yfRiEnaR38uROoDIdn\nWaoCdAVWOYctEruhjGcx7VlHiiTLypXhxhvhkfA6R8R9sVwA7QE8jhlR8yLwADAi+N5Y4HZgCOYi\n7D7gL5jRNqcBM4L7lQUmBY8Npwo+gyVTwWdRxBqaMZznWUQHSqLidr2CDwRgxw444wxYswZq13ax\nfckkmi5YPC+ZBN+TN/k7f+VclmI+yimS4AH+8AeoXh1Gj3axfckkSvDieckk+HfowgSGhCzFl0IJ\n/quvzHj4TZugYkUXzyGZQnPRSNo6m1WcxZdMo7/tUBLTpAm0bAmvvmo7EkljSvCSkv7IEzzNjRRQ\n3nYoiRs5Ep56CvQNVkqIumjEqkS6aGqwkzwacwbr+IGaoa3F3ZZDdK63d9Rn/eBBc7F10iRo187F\n80gmUBeNpKVhjGM2vcKSewrKzjYXW8eMsR2JpClV8GJVvBV8FkWspSlDGc/H/OaYd1OqggfYvRtO\nO01DJiVuquAl7VzEQn6hIh+TJl0a1avDlVfC88/bjkTSkBK8pJQbeJZnuQFvfPl0yYgR8OKLUFRk\nOxJJM0rwkjJqs53OzOcVrrIdirtat4YTToB582xHImlGCV5SxrW8xL/5HXupZjsU9w0frm4acZ0X\nvufqImsGi/UiaxkO8jWn0ZcZfE6baK3F1FYc0bneXtTP+p490KABrF0LJ53k4jklXekiq6SNbszl\ne2oVk9xTXLVq0LcvjBtnOxJJI0rwkhKu5SVe4DrbYZSs4cPhhRd0Z6u4RglePO8EfqAz85nKANuh\nlKzzzzcTj8W5ao9INErw4nmDmMybXMIejrMdSsnKyoLrr9fFVnGNLrKKVbFcZP2cVtzGQyygs1Nr\njm3FpxQvsh6ya5e5s3XjRjj+eBfPLenGrYus3TGLZa8H7ojwfm9gBbAM+Ay4OI5jRYp1DsupwS7e\nPepjlcZq1IAuXWDaNNuRSBpwSvDZwBhMoj4TGAg0C9tnPnAO0AoYBjwXx7EixbqGlxnPUAKZ1Js4\ndCiMH287CkkDTr81bYE8YCNmzdWpmIo9VH7I46rAD3EcKxJVOQ4wiMmMY5jtUEpXt27w9dewbp3t\nSCTFOSX4OsCWkOdbg6+F6wOsAeYAN8d5rEhEl/E6X3IW33Ca7VBKV7lyMGgQTJhgOxJJcWUd3o/1\nCtOs4HYhMBFoGk8Qubm5hx/7fD58Pl88h0uaGsY4XuYa22HYMXQoXHYZ/P3vUCaDuqckKr/fjz/O\nIbROo2jaAbmYfnSAUUAR8GAxx2zAdM+cHuOxGkWTwaKNojmJ/7KGZtRlKz9TJdbWIraVRHSutxfX\nZ71lS3jkEejUycUYJF24MYpmKSZRNwTKA/2B2WH7NAo5Sevgz50xHisS0dVMZAZ940juaWjYMF1s\nlaQ4JfhC4CZgLrAamIbpax8R3AD6AaswwySfgMO3G0Y7VsTRVbzCBIbYDsOuQYNg9mzYu9d2JJKi\ndKOTWBWpi6Y5K3mDS2nIxjiHR6ZZFw1Ar15mErJhw1yMQ9KBZpOUlDSYSUxicGaNfY9GY+IlCarg\nxarwCj6LIjbRgB7M4UvOjrc10q6C//VXqFMHPvvMzBcvEqQKXlJOB95nJyckkNzTVIUK0K8fTJli\nOxJJQUrw4imHumckxKBBMHmy7SgkBamLRqwK7aKpwC98yym0YCXbqJtIa6RdFw1AURE0bAhvvgnN\nm7sYj6QyddFISrmEN1lOywSTexorUwYGDoRJk2xHIilGCV48YzCTeIWrbIfhTYMHm26aoiLbkUgK\nUYIXT6jOLjqxgOn0sx2KN7VoAccdB4sX245EUogSvHjCFbzGO3RN/2X5kjF4sLppJC5K8OIJ6p6J\nwcCB8NprcOCA7UgkRSjBi3X12cRZfMkcetgOxdsaNIBmzeDtt21HIilCCV6sG8gUXuMKCihvOxTv\nO3SxVSQGGgcvVmVlZfEFZ3IDz/IBFybbGmk5Dj7Uzp3QqBFs2QI5Oe6EJSlJ4+DF884BqpDPYtrb\nDiU1nHACdOgAM2fajkRSgBK8WDUImMJAzRwZj0GDNJpGYhLLb1V3YC2wHrgjwvuDgRXASmAx0CLk\nvY3B15cBS5IJVNJQUREDQXPPxKtXL/jkE/juO9uRiMc5JfhsYAwmyZ8JDASahe3zNdABk9jvA54L\neS8A+IBWmHVaRY744AN2g2aOjFflyibJT5tmOxLxOKcE3xbIw1TiBcBUoHfYPh8BPwUffwLHTCTi\nhQu54pJq1WqQlZXlyvZMx45oEtwE6aYniYFTgq8DbAl5vjX4WjS/B94KeR4A5mMW4B6eSIDiLXv3\n7sb8sya3leNXruBEppb6nyBNdOoEmzZBXp7tSMTDnBJ8PGO6LgKu5eh++vaY7pkewB8g6XFwkia6\nMI91nMFG24GkqrJl4cortRCIFKusw/vbgHohz+thqvhwLYDnMX31u0Ne3x78uQOYienyWRR+cG5u\n7uHHPp8Pn8/nEJakuoFMYQoDgQ9th5K6Bg0yi3HffTdkqSc03fn9fvx+f1zHOH0qygJfAZ2AbzEj\nYQYCa0L2qQ+8C1wFfBzyemXMRdq9QBXgHeBvwZ+hdKNTCglfQzURlclnG3Vowld8T+2k2zsiA250\nChUIQOPG8O9/Q+vW7rUrKSGWG52cKvhC4CZgLiZZv4hJ7iOC748F/gpUB54JvlaAqdRrAzNCzjOJ\nY5O7ZKDLeJ2Pacf3nGQ7lNSWlXVkOT8leInAC9/rVMGnEDcq+Fn0ZgZ9mcBQ3K2SM6yCB1izBjp3\nhs2bITvb3bbF0zRVgXhOdXbhw89MLrcdSnpo1gxq1YL337cdiXiQEryUqn5M5x26spdqtkNJH5ph\nUqJQgpdSdWT0jLhmwACYMQN+/dV2JOIxSvBSak5hGy1Zzlv0tB1KeqlbF5o3hzlzbEciHqMEL6Wm\nP9OYRR9+paLtUNKPumkkAo2ikbgkM4pmCecxigdYQOfQFhNu71gZOIrmkF274NRTzUIg1XR9IxNo\nFI14xumsox5bWMhFtkNJTzVqgM8Hs2bZjkQ8RAleSsVApjCN/hShsdolRjNMShh10UhcEuuiCbCG\nZgxlPEs4P7zFBNqLJoO7aAB+/hnq1IG1a+Ek3SWc7tRFI57QimWUo4AlWvOlZFWuDJddBq++ajsS\n8QgleClxg5gcHPvuhS+MaU7rtUoIL/zGqYsmhcTbRZNFEZupT1feYQ1nRtxDXTQuKiw03TQffgiN\nGpXsucQqddGIdReyiJ2cECW5i+u0EIiEUIKXEjWIyUxmkO0wMsuhbhp9M8546qKRuMTTRVOOA3zL\nKbThMzbTIFqLMbcXQ3QutlUS7ZXDLLGQvJyc6uzZsyvym4GA6Z6ZPh1atXLlfOI96qIRq7ryDmtp\nWkxyzzSFuLFgOQSCi59HEboQiGS0WBJ8d2AtsJ6jF9Q+ZDCwAlgJLMaszxrrsZLGjoyekVI3aJDp\nhy8qsh2JWOTURZONWZO1M2YB7k85dk3W3wCrgZ8wCT0XaBfjsaAumpQSaxdNVfayhXo0Jo+dnFhc\nizG1F2N0Lrbl9fZiGJHTsiU88QR07OjSOcVL3OiiaQvkARsxa61OBXqH7fMRJrkDfALUjeNYSVP9\nmM57dHRI7lKiNHVBxnNK8HWALSHPtwZfi+b3wFsJHitp5GomMpGrbYeR2QYMMBdatRBIxnJK8PF8\nn7wIuJYjfe3qd8lQddhKK5bxBpfaDiWz1asHZ58Nc+fajkQsKevw/jagXsjzephKPFwL4HlMH/yh\ny/uxHktubu7hxz6fD5/P5xCWeNlgJjGdflrYwwsOddP06mU7EkmS3+/H7/fHdYzTRdaymAulnYBv\ngSUce6G0PvAucBXwcZzHgi6yphTni6wBVtGcG3maRXSIpUWH9uLh5YuibrcX47QHO3fCaafB1q2Q\nk+PSucUL3LjIWgjcBMzFjJSZhknQI4IbwF+B6sAzwDJMIi/uWElj57CCquzjAy6wHYoAnHCCGUWj\nhUAyku5klbg4VfAPcwv7qcQ9jI61xWLbi4+XK26324tj4rKpU+Hll9UXn2ZiqeCV4CUuxSX4bArZ\nQj18+FlHk1hbjNpeAtG52JbX24sjwe/fb2aYXLXK/JS0oKkKpFR1YgFbqBdHcpdSUakS9OsHr7xi\nOxIpZUrw4pqrmcgrXGU7DIlk2DAYP14zTGYYJXhxRRX2cSlvMJUBtkORSH77WygogKVLbUcipUgJ\nXlzRlxl8wAXsoJbtUCSSrCwYMgTGjbMdiZQiJXhxxRAmaGoCrxsyBKZN09QFGUQJXpLWgI20ZDn/\n0Vxy3tagAbRoAW+8YTsSKSVK8JK0oYxnKgM0NUEqGDpU3TQZROPgJS7h4+CzKGIDjejHdJbROpEW\n0Tj4xNpK6Pdm3z6oWxe++gpOOsmlWMQGjYOXEufDzx6qsQyt/ZkSqlaFPn00T3yGUAUvcQmv4Cdw\nNUs5lyf5Y6Itogo+EYkv4O0DHgdahrxW7CLe4kmaqkBcF5rgq/ETm2gQw7J8xbaIEnzptpVFEV9z\nGpczk+WHv3kl2OUj1qiLRkpUf6Yxn85ali/FBCjDOIbxe160HYqUMFXwEpfQCv4j2nEf9/AWlyTT\nIl6oalOvveTaqs8mPqc19djCfiqjCj71qIKXEtOM1dRnM3PpZjsUScBmGrCEtlzBa7ZDkRKkBC8J\nuYaXmcAQDjqu+ihe9RzXM5znbYchJSiWBN8dWAus58iC2qGaAh8BvwC3hL23EVjJ0Ss9SYorxwGu\nZiIvc43tUCQJb3ApjcmjqRZaS1tOCT4bGINJ8mdi1lRtFrbPTmAk8HCE4wOYUVmtgLbJBCre0YdZ\nrKGZ5n1PcYWUYxzDuI4XbIciJcQpwbcF8jCVeAEwFY6ZcGQHsDT4fiReuJArLhrBWJ7lBtthiAte\n4DquZiLlbQciJcIpwdcBtoQ83xp8LVYBYD7mP4Dh8YUmXnQ6cDZfMJPLbYciLviaRqyiOX1sByIl\nwukKWbLjptoD24GawDxMX/6i8J1yc3MPP/b5fPh8viRPKyXlemAcwyhQzZc2nmc4w3nXdhjiwO/3\n4/f74zrGqfukHZCL6YMHGAUUAQ9G2PdeYB/wSJS2or2vcfCp4pdf+L5SJX5DHl/TyKVGvTM2PLXa\nc6+t8vzKVipSMy8PGrn17yolzY1x8Esx38obAuWB/sDsaOcLe14ZyAk+rgJ0BVY5nE+8bPp0loOL\nyV284AAVmAjwgi62pptYLoD2wMxNlA28CDwAjAi+NxaoDXwKVMNU93sxI25qATOC+5UFJgWPDacK\nPlV06EDfRYuYmQFVrffbcze208liXc2asHkzVNS8/qlAk42Je778Erp0odz27RRmSNLzdnvuxxbo\n1g0GDjSLgojnaaoCcc/YsXDttQlOUCspYeRIeOopUMGVNlTBi7M9e6BhQ1i5kqx69cikqta77ZVA\nBV9YCGecAZMnw/nnu9i2lARV8OKOceOgSxez1Jukr+xsuPFGGDPGdiTiElXwUryiImjSxCT59u2P\nWdEped6uar3bXglU8IEA7NplhkquXas1Wz1OFbwk7+23oVo1+O1vbUcipaFGDbjiCg2ZTBOq4KV4\n3brB4MEwZAhw7JqsyfN2Vevd9kqoggdYsQIuvRS++QbKajpor1IFL8lZs8b8svfvbzsSKU3nnAOn\nngozZjjvK56mBC/RjRkD118PFSrYjkRK21/+Ag8/rCGTKU5dNBLZrl3QuDF88QWccsrhl9VF45X2\nSrCLBuDgQWjWDF58ES680MXziFvURSOJe/pp6NPnqOQuGSQ721TxDz1kOxJJgip4Odb+/aYPduFC\nU8WFUAXvlfZKuIIH+Plnc4Pb++9D06YunkvcoApeEjNunLmTsVn46oySUSpXNjc+Pfqo7UgkQarg\n5WgHD5rb1SdMgPbtj3lbFbxX2iuFCh7g++/NjW668clzVMFL/KZPh9q1IyZ3yUC1aplhspq+ICWp\ngpcjAgE47zz461+hV6+Iu6iC90p7pVTBA+TlQbt2sGEDHHeci+eUZKiCl/gsWAD5+eYuRpFDGjeG\nHj3MVMKSUmJJ8N0xi2WvB+6I8H5T4CPgF+CWOI8VrwgE4G9/g7vugjL6f1/C3HUXPPkk7N1rOxKJ\ng9NvcjYwBpOozwQGAuFDK3YCI4GHEzhWvGLhQvjuOxgwwHYk4kVNm8LFF8Mzz9iOROLglODbAnnA\nRqAAmAr0DttnB2Zx7oIEjhWv+Nvf4J57NLmURHf33WbIZH6+7UgkRk4Jvg6wJeT51uBrsUjmWClN\nfj98+61Zj1MkmrPPNqOrnnvOdiQSI6dyLZnL9DEfm5ube/ixz+fD5/MlcVqJW26uqc5UvYuTe+6B\nnj1hxAhzI5SUGr/fj9/vj+sYp2GS7YBcTD86wCigCHgwwr73AvuAR+I8VsMkbVq4EIYPNzeyxJDg\nNUzSK+2V4jDJcFdcAW3bwu23u3h+iZcbwySXAqcDDYHyQH9gdrTzJXGs2BAIwJ13wn33qXqX2I0e\nbSYh+/FH25GIA6cEXwjcBMwFVgPTgDXAiOAGUBvT1/5n4G5gM1C1mGPFK2bMgIICLegh8Wna1NwI\np5kmPU93smaqwkI46yxz80rXrjEfpi4ar7RnsYsGYPNmaNXKrBdw8skuxiGx0p2sEt1LL0HdutCl\ni+1IJBXVrw/DhpnuGvEsVfCZKD/fzBg5a5aZeyYOquC90p7lCh7ghx/MlNKLFmm+eAtUwUtkDz5o\nlmGLM7mLHOXEE2HUKLPyk3iSKvhMs3EjtGkDy5dDvXpxH64K3ivtuR1bOcy4iPiPWoUZYTEn5PWc\nnOrs2bPLndAkIlXwcqxbb4U//Smh5C7prBDzH0Z8WwEB/sybPEoTyvHr4df37t1t4c8g4ZTgM8nC\nhfDZZybJi7hkDj35hlP5A/9nOxQJoy6aTFFYCK1bm2kJ+vZNuBl10XilPW/F1oS1LOJCzmEF2zmF\nhC7aSlzURSNHPPaYWYrv8sttRyJp6CuaMpYRPMEfbYciIVTBZ4ING+D882HJEjjttKSaUgXvlfa8\nF1tF9rOSFvyZx3iTy1TBl7BYKngl+HQXCJibmbp3d6XvXQneK+15M7aLWcBLXMtZbGaffq9LlLpo\nBCZMgN27zcgZkRL2Lp14j47cZzsQAVTBp7ft26FlS5gzx1xgdYEqeK+0593YTuAHVlKTU957Dzp0\ncKVNOZa6aFLQ4sWL2b59e/INBQJccP/9BM47j5NdXIFHCd4r7Xk5NriULF5v2BBWrIBq1VxrV45Q\ngk9B1aufTGFhG8qUqZhUO9cd2MDAAxtoH/iVA4EDLkV3iFcTi7eTXqb9WQPXXQdFRfDiiy62K4co\nwaeg446rzZ49yzHT7CemKWt4nw6052nWcyVeTwSZlPQy6c8a2LPHdBE++ij07u1i2wK6yJqRKvAL\nr3AVdzOa9dS3HY5kspwcc5F/xAgzf7yUulgSfHdgLbAeuCPKPk8G318BtAp5fSOwElgGLEk4SonZ\n4/yJbziV57jedigi0L69mW3yyivhgNtdheLEKcFnA2MwSf5MYCDQLGyfnkBjzPqr1wPPhLwXAHyY\npN82+XClOEMYjw8/1/IS3uh9E8Hcf1GrlhbptsApwbcF8jCVeAEwFQjvTOsFjA8+/gQ4Hjgp5H1l\nmlLQghU8zK30Yzp70agF8ZAyZWD8ePjPf+Df/7YdTUZxSvB1MAtqH7I1+Fqs+wSA+cBSYHjiYUpx\nTmQHM+jLzTzJas6yHY7IsapXh9degxtvNDOaSqko6/B+rJfVo1XpFwDfAjWBeZi+/EXhO+Xm5h5+\n7PP58Pl8MZ5WKrKf/9CbqQxgKgNthyMSXZs2MHasGVHz8cdmTWCJmd/vx+/3x3WMU/dJOyAX0wcP\nMAooAh4M2edZwI/pvgGTxDsC34W1dS+wD3gk7HUNkwwRzzDJLIqYwkCKKMNgJhE45gvZJ5h/Qm8P\np8ukoYOZ9GeN+nv9r3/BlClmLdeqVV08Z2ZxY5jkUszF04ZAeaA/MDtsn9nAkODjdsCPmOReGcgJ\nvl4F6IpZ3Utc8gCjqMM2ruHlCMldxKNuu81U87/7nUbWlDCnrFAI3ATMBVYD04A1wIjgBvAW8DXm\nYuxY4Mbg67Ux3THLMaXkG8A7Lsae0e5iNJfyBn2Yxa8kd9erSKnKyoJnn4VKlWDQILMYjZQIL4xw\nURdNiFjZPHdgAAAIbUlEQVS6aG7hYYbzPB15j++K7cpRF036tufl2CCWRbzLA/8BvgeGFXN2LeAd\nme5kTUMjeZL/4Rk6scAhuYvY5LyI9wEC9CWfBnTgea6lTJRjtIB34pTgU0aAe/g7N/MknVjANjQC\nQVLffipzCW/SgE1MZQDl+dV2SGlFCT4FZFHEk9xMX2ZwAR+wiYa2QxJxTT5VuYQ3yeYgs+lFZfJt\nh5Q2lOA9rjL5TGUALVgZQ5+7SGo6QAWu5FW2UYf36UAdttoOKS0owXtYAzaymPbspxLdmMsejrMd\nkkiJOUhZfs+LTKM/n3A+5/Ox7ZBSnhK8R13Eu3zEbxjHMIYxTkMhJUNk8RC3cz3PMZteXM9Y2wGl\nNKepCqSUlQsE+CejuYqZXM1EFtDZdkgipe4tLuFCFjGVAXQF2LULatSwHVbKUQXvJatXMz9/F81Y\nT0uWK7lLRltHE9rxMZvArAw1d67tkFKOErwX7N8Pd90FHTowrnwlejOOH6hpOyoR6w5QgVsAXngB\nbrgBBg+G77+3HVbKUIK3KRAwc2SffTbk5cHKlbxcvjLeuMFYxEO6doUvvoBTToHmzeGppzSPTQyU\n4G354AO44AK4+254+mmYNs18eEUksipV4KGHYN48eOstOPNMs4BIUZHtyDxLCb40BQLwzjvQpQtc\ndZVZjHj5cujWzXZkIqmjRQuYM8dMWPbgg+b5xIlQUGA7Ms9Rgi8NP/9slixr1cosQDx4MKxbB0OG\nQHa27ehEUlPnzvDpp/DYYzBuHDRubCp89dEfpgRfUgIB+OQTU6XXrWu6YO6/H1atgmHDoHx52xGK\npL6sLPONeMECsyTg6tVwxhlwxRWmGyfD++m9cDUvfaYLLiw0q9TMnAmzZkHlyjB0qKnU64QvZRtZ\nPCs6OdN0wenbnpdjc7u9YlaHiuSnn2DqVPOtee1auPRS6NfPXKitVMmlmOyLZbrgWBJ8d+BxIBt4\ngaOX6zvkSaAH8DNmaudlcRybugm+qAhWroSFC8HvN8m9USO4/HLo0weaNTMVRhyU4L3Sltfb83Js\nbrcXZ4IPtW2bKbamTzfdOeedBxdfbLZzz03pb9JuJPhs4CugM7AN+BQYiFnV6ZCemFWfegLnA09g\nskosx0KKJHj/ggX46tSBZcvg88+PbLVrg89nto4dzfMkJJ/g/YAv+NjLCd6PidPrSW8hR/4+3Wiv\nJP6sfpKPsTQSvJ/E4kwiwYfau9cUYQsWmG3dOjNE+bzzTLJv2RKaNMG/ZAk+XyJxlq5YErzTVAVt\nMUvxbQw+nwr05ugk3QsYH3z8CXA8JjudGsOx3nHwIOzYAd9+a7bNm2H9+sObf8MGfPXrQ+vWZrvt\nNvOzVi3bkYfx415CKkl+FKdb/Hg/RrAeZ04O9OxpNoB9+8wotk8/NUMvH30U8vLwV6iA7/zzoWlT\naNAA6tU7stWunVIDI5wSfB1gS8jzrZgq3WmfOsApMRybvKIicydocVt+Pvz4I+zefeTnocc7d8L2\n7ebKe/XqZiz6KaeYPvPTTzdV+RlnwOTJMHq06+GLiCVVq5p7US644MhrBw+akW7dupn++82bYfFi\n2LLFbDt3Qs2acMIJR28nnmjmyqla1WxVqhz7s0oV0yVUvjyUK2e2OLtw4+WU4GP9XpRclG3bmr/Y\nwkLzM/RxpNdCHxcUQIUK5uJJtK1KFZO8jz/e/Kxb98jjGjXg5JPN/8zF9ceVLZ152bKzy5CTczVZ\nWYnNHvnLL19RseJnABw8uJt8rZ0gErvsbJMXQiv9UAcOwHffmUQfuv3wA2zdaorJffsi/8zPN/nq\nwAGzFRaaJH8o4Ycm/7JloUwZE0+0ny5oB7wd8nwUcEfYPs8CA0KerwVOivFYMN04xS/eqE2bNm3a\nwrc8klQW2AA0xCyCvhxoFrZPT+Ct4ON2cHiW/liOFRERi3pgRsPkYapwgBHB7ZAxwfdXAK0djhUR\nERERkXRxC1AEeHXplvsw31KWAwuAenbDieohzHDUFcAM8ORirr8DvgQOcvS3Pq/ojrmetJ7I1468\n4CXgO2CV7UAc1MPcUPAl8AVws91woqqIGeq9HFgNPGA3nGJlY24ofd12ILGqh7ko+w3eTfA5IY9H\nYu7O9aIuHJln6J/BzWuaAmdgfvG9luCzMd2KDYFyePf60YVAK7yf4GsDLYOPq2K6bb349wlQOfiz\nLOZ64gXF7GvTX4BJwOzidvLSZGOPArfbDsLB3pDHVYEfbAXiYB7mmxCYiqSuxViiWQussx1EFKE3\n+BVw5CY9r1kE7LYdRAz+i/lPEmAf5tulVxc/+Dn4szzmP/pdFmOJpi5mcMsLOAxR90qC7425EWql\n7UBi8A9gMzAUb1bG4a7lyCgniU20m/ckeQ0x3zo+sRxHNGUw/xl9h/l2udpuOBE9BtzGkSIuqtK5\ne8eYR+QJVu7CjLDpGvKazVkuo8X5v5j+rruC252Yv+hrSi+0ozjFCSbOA8Dk0goqTCwxelHAdgBp\nqirwGvBHTCXvRUWY7qTjgLmYuRX8FuMJdynwPab/3Wc3lNicjfnf8pvgVoD5auy1SV7C1cdcMPKq\nYcBizIUjL/NiH3ysN+l5QUO83wcP5lrGXOBPtgOJwz3ArbaDCHM/5tvlN8B2IB+YYDWiOHn5Iuvp\nIY9HAhNtBeKgO2bEwom2A4nBQqCN7SDCpNJNeg3xfoLPwiShx2wH4uBEzGSJAJWA94FO9sJx1BFv\nfxOO6Gu8m+Bfw/wyLQem491vGeuBTZivccuAp+2GE9HlmEpkP+Yi3By74RwjFW7SmwJ8C/yK+bu0\n1V3o5AJM18dyjnwmu1uNKLLmwOeYOFdi+rm9rCMOo2hERERERERERERERERERERERERERERERERE\nRKz7f1ZW+mMCqcjLAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Slice sampling: 6551 evaluations for 1000 samples\n" ] } ], "source": [ "sampling = SliceSampling(log_p=lambda x: np.log(p(x)), x=-3.0, w=1.5)\n", "\n", "show_sampling(sampling, plotter=gauss_hist, N=1000)\n", "print sampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Slice sampling can be applied to multivariate distributions by sampling in different 1-dimensional directions. Most often the directions are just chosen as the coordinate axis and, similar to Gibbs sampling, all coordinates are sampled in sequence.\n", "\n", "Here, we apply this scheme to the 2-dimensional Gaussian example:" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class MultiSampling (Sampling):\n", " \"\"\"\n", " Class that wraps a one-dimensional sampler and applies it to \n", " a sequence of given directions to produce a new sample\n", " \"\"\"\n", " def __init__(self, uni_sampler, log_p, x, directions):\n", " \"\"\"\n", " uni_sampler is called with a log_p function and current sample x.\n", " It needs to return a valid sampler which is then asked to draw a sample.\n", " \"\"\"\n", " self.uni_sampler = uni_sampler\n", " self.log_p = log_p\n", " self.x = x\n", " self.directions = directions\n", " \n", " def sample (self):\n", " # Loop through directions\n", " for d in self.directions:\n", " uni_log_p = lambda ux: self.log_p(self.x + ux*d)\n", " ux_prime = self.uni_sampler(uni_log_p, 0).sample()\n", " self.x = self.x + ux_prime*d\n", " return self.x" ] }, { "cell_type": "code", "execution_count": 40, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD7CAYAAAChScXIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8zdcfx/FXhiQksWPvEXuPEFSKmLVaWq1Ru3YHahRV\natQotWmp8StKqV2jKjaxiT0SRYxEI3vf8/vjEIIUzb25ucnn+Xjch5vv/Y7zjXg7Od8zQAghhBBC\nCCGEEEIIIYQQQgghhBBCCCGEEBbAytQXqFSpkjpz5oypLyOEEGnNXsDj+Y3Wpr7qmTNnUEql2dfX\nX39t9jLIPcr9yf2ZvxzGfgH1X5apJg9tIYQQxiOhLYQQFkRCO5k8PDzMXQSTS+v3KPdn2dL6/T3P\n5A8iAfW4fUYIIcRrsrKygpdktNS0hRDCgkhoCyGEBZHQFkIICyKhLYQQFkRCWwghLIiEthBCWBAJ\nbSGEsCAS2kIIYUEktIUQwoJIaAshhAWR0BZCCAsioS2EEBZEQlsIISyIhLYQQlgQCW0hhLAgEtpC\nCGFBJLSFEMKCSGgLIYQFkdAWQggLIqEthBAWREJbCCEsiIS2EEJYEAltIYSwIBLaQghhQSS0hRDC\ngtgm8/iCwHIgF6CARcCs5BZKCCESxMTAokUQHw+VK0OlSpA1q7lLZTZWyTw+z+PXacAJOAG0AS4+\ns49SSiXzMkKIdOnwYejVCwoVgqJF4fRpOHsWwsL05599Bh4eOswLFQKr5EZa6mGl7+WFG0pu88g9\ndGADhKHDOl8yzymESO+Cg6F/f3B3h9atYetWmD5dB3jJkk/3i4/XtXB3d8ieXQf49OlmK3ZKMGab\ndhGgCnDUiOcUQqQ3v/8O5crB9ev664kTwdoaMmWC1atR48bht3A1j8pV44+inlzOXxFcXSE2Vu+T\nP795y29iyW3TfsIJ+A34FF3jTmTs2LEJ7z08PPDw8DDSZYUQacbt2zBwIFy6BCtX6qaPZynF9vii\nrOq1jWX35gNQY8TH/BZdDNeNY7Bq2BBCQ+HMGZgyRZ9n3DgoUMAMN/PmvLy88PLyeuV+xmgAygBs\nAf4AZr7kc2nTFkIkLT4eFiyAsWOhe3fdNj1xIvj7E/flcLaUa8uc5ZeIPOXDwqz7KOIUh9PZE9Cr\nF9fylOXwwm10rqh0WMfH6weVvr7g6AhHjoCTk7nv8D9Jqk07uaFtBSwDHgKfJ7GPhLYQ4uXOndPt\n1P7++mHi4cPg5kbs3QecIxfN73hQu7g9A+s78FaWR9iePwdeXnDrFgB/udSESpVo8HkbqFhRN438\n+Sd06QIHD0KxYua9v2QwVWjXBfYBZ9Fd/gBGANuf2UdCWwiRWGSkbrqYPFl/nS0bdO2Kf5vObP92\nFd13TeVB5jxkszOQIS5G154rVdIPIQcOBOCHtpPZEVWAzZs/xMbm8eO5K1egXj1YuxbeestMN2cc\nSYV2ctu0DyADdIQQb2L3bvjkk6cPGufO5Zp7C5aO+Z2mDdvRPe4GALl++VHXngsWhLt3dTe/9u0B\nuFOwHD9ecWb//nefBnZQELRsCRMmWHxg/5uU6NQoNW0hhHb1qu7pMWQIDB9OZNN3uOEfycmH9rxr\newWrkSPJNGs6/PEHVKmi26i//loHccaMuoYOtMg7hB+PjyFfPmd93thYaN4cypeHGTPMeIPGY6p+\n2kIIkVhcHFSvDlOn6v7WzypZErZsIW7Vag65teOHC86U8z9D52hvHE8cIZNLVj2I5vJl6NQJbG11\nYD8+77V6rfnHKhPTdg15GtgAn3+u9506NeXu00yM1eVPCCE0W1t4+2348kvdZt2jB3z6KeTPj79/\nKBP/sOJuSBPW3VmMO+ha95UrULr003PkyAG5cz/9umZNLtrnZ8FZO8Z7NqRMucefXbmiA/vOHdi7\nV187jZPmESGE8cXEQN26+mUwYFi2nBMFajLerxDj8l2m8pV9r3+uEiW4miEPzSPbcqLSUTI3fRs6\ndtQ18J9+gmHD9H8Kdnamux8zkOYRIYTx7dmjH/qNGwfe3roNGnSArl6NWr6Cn+MrUNGqHzV8drAp\n7Mc3Cuz4ffv5wW0Q7VQ7vPb3ILP3frh/H8qU0d0Ez52DoUPTXGD/m7T/u4QQwnT+9z895Dw0VDeD\n3L0LjRtj8PTkl4C8hEeVpc+cnnT7t3PY2Oia8vffJ9oc/fARHXvvICDAmb8OvE+OnRv1+TdtgjVr\n9Hwj6ZA0jwgh/huDAfLl04NYihcHQN26xbXPx1Ny3Y8v7l+6tO7qN20a1Kmjg7dwYbh586WnX1a4\nNX9Uac+yuY2wHz9Wj5oE/aDTxsZEN5V6mGpwzeuQ0BYiLTp+HDp3hosX4cEDbk//keD5S8gTEUBI\nszYUGdYbq08+gQsXXn78vXt6UI29faLNoc3b4LxtQ+J9Bwx4OnqyY0cT3VDqIm3aQgjjWr0aHj0i\n0sOT8AJFOTJrHTc/HkTWsACKbl6OVdasEBCg26A3b37x+OrV9fwgz3khsEH3JDlzBho0MMGNWBap\naQsh3szly/DNN7BqFTEZHBhg14YC/Tvx+ahGODvrWrMyKJSdHdbxcRiwwppnMuDbb2HUqMTntLeH\n6OjE2woU0DP/AbRqBdeuwfnzJryx1EVq2kIIo1Bbt8KqVQB83GoJI33mM+a7Fjg72xMRodg25k+s\nbKyxjo/Dq8noxIENLwY2JA7sDh30kPVbt0ApPQHU8ePg6WnCu7IcUtMWQry2Y8fu4NeqC+3v/aU3\nxMeDtTX3AxQ7v/mTOov74WQbQ66wvzGsXYe1/23w8mL30PUMHB+Pz+B9WDd63MSRL5/utvcsHx/d\nG+V5oaF6IYSXNKekVaaaMEoIYakMBl2TfY2eGP7+oYwYsZtdu65zUV1P2B4wYT7/u+yK+8axdA47\nlOgY66GDwc8PgLADHfi9cUmsjzk+e9IXy5PUGo/Ozi/fng5JTVuI9Kp/f1i8WE/gVKZM4perKzg4\nEBUVx4wZh5k+/TC9e1djxKAqOOfN8cKpopu3xt4lKzx6BBs3vnlZGjbUzSAigXT5E0I8FRcHefPq\nBQWiovTSXBcvJrzUjRuEZ3XhaEhmwvIVxb1bQ1wa1CJgxzFcvvn01ed3dtZNGsDdErXJW6s4bN8O\ngYEJu4QVccXp2EHd7S8+Pl2NanwdEtpCiKf+/BNGjtRDz59z5sw9vhi4hcwBd5jYxokyk4e88emj\nKtXg2K0sOLtXpnLAfjiaeL3v6Jru2B/aly4GyfxXEtpCiKd699bTpA4dmrAp8IIvK4as4NH+Y3xU\nNh7XyNtYnTtn9Esre3usAgKknfoVJLSFENqTphFvb/Dzw/Ddd0QePk5saASB+UpQoEU9bAoWIsPo\n4ca/9uefw/TpST9wFAmk94gQ6dGxY7obXbdnpmzy8gIXF5g+neg16/navgl+5UYycv5HlA+9Qbxn\nYzJEhf/nS8bb2WPTrStq91/4B8eTP0AvH8a+fXr9RpEsUtMWIi1r0kSH9N69UKuW3lamDFy6hHeh\n2nSPa8G3kxvTOuw4auAgrOPj3vgSKmtWrB49errBzY14Xz/ORmWlSshlvS0oCLJmTf79pCMyIlKI\n9ObyZTh9GhYtgtq14f59DP36654iQM2/D+PjP4o2XWpi1a9fkoF9o2TZJC8R9+kXfP7h2UTbAuyz\n8U9gOFVCLqPe/0D3v5bANhqpaQuRVg0aBA4Oes6O338HIMA2My5xIck7b5UqcOoUAI0+COfPX58O\nmNndchANN8/SX2zapFdHF/+JtGkLkZ6EhsLSpXqR3LNPa8KvE9jhuXKT6WEgVk9WoXle5cpw6hQL\nCg5JFNgLK3XnkyeB/eCBbjcXRic1bSHSopkzdU+NNxS3dBm2/zyEiRMTDYR5Fa9MZfGIuIChVi2s\nDx2S3iFGIF3+hEgvwsKS1wfa0VGv+Th48Asfxdhlwi4mIuHrk237UvX3+fqLJUsS91IRySKhLUQ6\nENu0BRl2bDPJuf2ylaNI0NP5rGfXH8rAvVMff+inlw4TRiOhLUQapqKiCC9UAqeAOylyvXBrexwN\n0Rjy5sP69i09baowKunyJ0QadWuTF1YZM6ZYYAM4GqJRW7Zg7X9HAjuFGaOmvQRoATwAKrzkc6lp\nC2ECsV77yPB2/RS/rrK1xerOHciVK8WvnZ6Ysqb9M9DUCOcRQiTl0SM9UGbpUvjpJx7WbmiWwI4e\n9TVW0dES2GZkjH7a+4EiRjiPEOJlwsKgdGm9qvljLy5DYFoR2VzItO5X7N9+O4WvLJ4njVFCpHZL\nl6IqVTLLpcOsHQh3q0umC2dBAjtVkBGRQqRWR4/Cu++Cv3+KdPN6GYfhQ7AdN1YWK0hFUiS0x44d\nm/Dew8MDDw+PlLisEJYrMpKYhT9h9/zitylp505sPT3Nd/10xsvLCy8vr1fuZ6z/wIsAm5HeI0L8\nd3Fx0KULqnt3wjv3wOne3+Yri7U1fPCBnp2vZUto1sx8ZUmnTNl7ZBVwCHAFbgEyjlWI/yI8HFat\nwsrTk/3hOc1blqZNIWNGHdpOTuYti0hERkQKYS4GAwwfDnfuwMqV5i4NbNsGs2ZBx47QqZO5S5Pu\nyTB2IVKb+HiwNW9fgMgMGbGrWgmbtWt0k0iFCnD7NmTKZNZyCRnGLkTqohRRI0abuxRkXL4Em/37\noGBBWL4c2reXwE7lpMufECnNz4+Alh/g4uNt3nLcvg358+v3SsHPP+vgFqma1LSFSCnx8Tz65jtC\nSlU0a2CrUqV0T5UngQ1w6JDui+3mZrZyidcjoS1EClD9+4OtLQ/Gf0/mmFCzlCE2Szbw8sLq0qXE\ng2V8fWHKFL2Agaw4k+pJaAthStHRPOg3FKt58wBwjX+Q4kUIss9KTJNmZLh2BerX15NPrV8PfftC\niRJ6pfYsWaBnzxQvm3hz0ntECBOJ2XeAR+064RcE5RzDcQx+/TUXjcVgbYPV5ElYubnBrl36df48\n1KkDnp76VaGC1LBTIenyJ0RKCQvDv9sgMvy+jku5y1HP/7D5ypI3rx60U7Lk05B2dwcHB/OVSbyW\npEJbeo8IYUTh67cQ1bUH52NyUKF8eSoGpEzt+p9y7mQ/fyjxxvff1xNONWwIOc08wlIYjdS0hTCG\nhw+53b4Hhn37uV6yJnUenmZLjvZcz1SCoSc/TdmyVKoEp05Jk4eFk+YRIYzlt99g4kSIiYHoaLh2\nzdwleur0aR3awuLJiEghjOXMGXBzQ02dSnB4HOFWdpwrWcesRVLvvgsRERLY6YCEthBvKi6OuI2b\nCWnzAZtii2FwLU2FqwfNV56lS7Fat07PyifSPGkeEeJ1KYXh8hWsy5QG4G7ekuS9exWAgCyFcAk2\nw/zX169DsWIpf11hctI8IkRybNsG1tYJgQ2Q9+5VAuxyca9I6ZQP7H79dJu6BHa6IzVtIV4hfuky\nbLp1NXcxntq5U/e3Fmma9NMW4k3cvatXbcmUiVSzpK2TE1y9CnnymLskwoykeUQI0AsSXLkCa9ZA\n9+6QL1+qmFf6ZI7yxGfLDpMmQXCwBLaQmrZIh8LD4dw53XXv9Gn9OncOXFx0l7mNG81dQgBuVKxP\nlWA/rFZt0ZM6CYHUtEV6cuAAlC6tw7l/f/D2hjJl4Lvv4PZtoi5eZULBj8xdygTFSuTA6tQpCWyR\niNS0Rfpx7x4ULqxr1RkyJPro0KFb9OjxCyNszTe5k69NDuKKlqDktaPQtq0eeSlD0cVzJLRF+pE/\nv55L+pnAjoiIZdSov1i1yocZM5vRocOAFC/WykqdmBVZkS2ux8h57XGzTcWKKV4OYRmkeUSkH/nz\nw507CV/u23eTSpUWcP9uCH/2yUGHDuVSvEg18ozmdlk3DqkfyeniBMePS2CLfyX9tEX6ERsLjo6E\nBwYzYpQXm387x9o2sRRetwLHwL/JZIhMsaJsr/weff6pw47WIZRaNQdmzoSOHVPs+iL1k1n+hABi\nsrvwjmMverjcos3ff3LUpgp3C1fkg2PfpVgZGhcYgWu9sswIWUOGu3dg9Wq9SIEQz5DQFulaWFgM\nEwb8yqRlXRK2+ThWonz4GZNc73CjxtT+c2eibWdc6/FuSDOWfVaEuvOG6wUKJk8Ge3uTlEFYNpl7\nRKRbf/3lS4UK86l6TodoiG0WdlTsbrLAjnDI+EJgh1o7sD27Gz6eV6k743OYMwdmzJDAFm9Matoi\nzQoLi2HYsF1s2nSF2XNacvhaCcpM60u9nGcp7nPo1ScwMuXpiVWjRrrtOn/+FL++sCzSPCLSlb/+\n8qVHj014eBThox5NGTTZnlLlYtgwJWVrtr8XaUa9qQPI2bKh1KrFGzFl80hT4BJwFRhmhPMJ8Z+F\nhcUwYMA2Pv54AzNmtiBX+dZ0GmFPh4+upGhgD3dszeKfTtDmxlZytmsugS2MJrk1bRvgMtAIuAMc\nAz4ELj6zj9S0RYrw8vKjR49N1KtXiM69mzFwoj1FK0KbuCn0mppy9YkdeepS+shmChfOmmLXFGmP\nqaZmrQlcA/wef70aaE3i0BbCpMLDYxgxYjfr119kztyWHL1Wgp5DIhjSdg/9v2yWomUJzFcCz1v7\nsLaW4efCNJLbPJIfuPXM17cfbxMiRezfr0c1BgVFsWxNP376wUCv7yrje9gpxQP7hvdVct65KoEt\nTCq5Ne3XavcYO3ZswnsPDw88PDySeVmR3kVGPp0zZNbsFpz1L81fnX5gi+9nKV6WsMw5yXT/NsUc\npN1a/HdeXl54eXm9cr/kVglqAWPRDyMBRgAG4NnhZdKmLYzqyJHbfPzxBqpWzcsnnzZnyph7bNtl\n/LUSfTuU5V5RV2pP2pDkPtHFSmJ/+gQ4Oxv9+iJ9M1WXP1v0g8iGgD/gjTyIFCYSHR3H1197sXTp\naX74oTnXgspycN5h1vq1wDE0yHjXqZgd+7P//Os+/gXLktc+BquDByFXLqNdW4gnTNXlLw4YAOwA\nLgC/Ig8hhQmcPHmXatUWcePSfdZu6svBebd4Z9xbbDvnbtTABhIC+6/2PV/6eWDPQeSLC8Jq+3YJ\nbJHiZHCNSNViY+OZOHE/c+ceY/r3TfAc3Z0fDe0YeXsU4ZkcyRwWkrwLFHKGv0OT/FhZWbGl30xq\nzhtFbvXMfidOQNWqybu2EP9C5h4RFsfH5wG1ai3m6NE7bNjWhx93VOR+hAOj/x6BjSE++YFd1AGD\nbfS/7vJz0Xepu/Ar4sZP0IsTAAwYIIEtzEZWrhGpTny8ge+/P8yUKYeYOLEhhsxVaD0Iplb/H5Ue\nHHzl8cGOjmQJD3/1hXyjXlprOV+/OyeavsfJCT/T2ykI5+OHyJY1M9StC6tWQYcOb35TQhiJhLZI\nVa5f/4cuXTZgZ2fDlu29GbswC+XP/0bA4fbwmss3JhnY1lbQOzcsuJfksafm/cGQ3x4Quv4OS4/O\no2xZFwgM1IE9ZIgEtjA7adMWqYJSigULjjN69B5GjXqLPMWqc2TwcmZe65FiZbjtWp3KD9szdKg7\ngwe7Y2trDeHh0LAheHjoua+FSCGmGsYuRLLduRNCt24befQoip1Lm3Dz27W0PVqbJ3XaOAc7gotm\nJsfFQJNc33vlUebM2E1sVAz793ejTBkX/UFsLLz/PpQuDZMmmeTaQrwpqWkLs1FKsXLlOb78bCsz\nGsbxtu9BXLx3APAwZ27mTf6O0T27mrQMZ+q24/wxX6qVyoJrfnusIiJ07To8HIKC9APHDRsSreAu\nREqQ+bRFqvLwYQTjOy6i0vEtdDScITYqHsfIIP7JW5i+4+bTtORdunmYrmkk3j4jP+ZpTqS1PR16\n1iRv8TyQKRM4OiZ+FSoENjYmK4cQSZHQFilPKbB67kcsPJwzo+YQNW8hZWwfEdGwFb6HblE28hwr\nh4zmUI3KrGj5llGL8Shrdo5V7cTMwA6sudOegx4d+cgrB6NHv8WAATWxsZGeryL1kdAWKcdgAGtr\nGD4c8uaFQYPgxAli5y8k5pfVHLEuRM7hn3LNLzONlvfhWqM2zHunFV+tmkSxQ8eNXpz36nhTsgCM\n29WC8Vlbc6BQXX76qSXFi2c3+rWEMBYJbZEy9uyBVq2gUSPdFuzoCMWLExUYxNyoitxq0JaPv+5M\nvwl2NA5cxzd/tnvxHD2zw0//PvfH81Rma8JzZsHpRuIh7QXrxbLsnf24jXuXPtatqTNlAL17V5Pp\nU0Wql1RopwQl0okdO5TKkUOpIUOU0o0jSoFa+e4YlSfXd2rdugvqxzUG1arcHnWyTgcV7JxF3StS\nJNG+b/oy5LRSClRIXme1bNMPCdt9nCqp2W3XqjPTVqp/bJzU8OrDlJ9fkLm/Q0K8NpKY+loa84Rx\nLFoETZrAw4ewYwc4OSV81GzLTE72gJW7ilNqYAvWXWnC73Wrc79xNXL7+SXrslaBiq1rB7K3Tks+\nfG8IAAPKr6aErT9Fw/aQd2hvDg2bzUTvSbL8l0gTpHlEJN/ChdCnj37v7Y2qVo2gfMVZGVKIZmXt\n8B84g5jPv6SC7VW2dvgIzy2/UMD3utGLccLZjWqhRzHYZsA6LpZ/7LMSt2kLuRrXMfq1hDA1mTBK\nmMavv8LXX+vRgiVKEDlmHCNrjCA2KJiWk3qS9fpVVk/xwa5VBXIFXKPb7HFGCWzvke+/sK1a6FEA\nVmeoQkSOPGQ7dUgCW6Q5UtMW/93y5TBsmG4OqViRrYv3UbdXE7KoKOOcP5M1dGwAJ71R4dEcWdiJ\nyu+uJtQpB7lu/p1o13t1m/HTBXsGRHjhnDc7Nn/thiJFjFMOIcxAeo8I41q8GMaMgV27CClQnEGD\n/iBilxe/hC0jQ8gj41yjcBbUmHj+8a9EjtFJz+43tek4Zvs48letS5S4cBB27YJ8+YxTBiHMROYe\nEcYzb55uDvHy4sjDjHSsspAPq2XgW/8fjHeN+g6wNxirHpCDlwd2SIHifBDVkuJFK3OtwFbsTp2E\nvXshZ07jlUOIVEbatMWbmTEDpk4lbvcexqy4TevWq5k61ZM+wS/Om3psWreXn+NJ3aFdO7BLYk6P\nvUk3sUSUKMuXHhOokekzRvw6gDlha7G7dB5275bAFmmehLZ4fZMnw9y5/L1iE2997MXRo3fw3v0h\nmWctp8DOFS/snn/vaf1mVC6YMuLpB09ay/ZvgZhY/T5npoSPDTbWPKjh+tIi7O0xjiLBXcng7sYZ\n7268NWcoPHig29WzZDHGXQqRqknziHg1pWDcONSqVfw2cD792m5nxJe16BB7iVwVC1NYxb/0sLx3\nr+g372+Bxq0gZ2YIfGaJMOUMROmfwsCIhM3W8QZyHbuS6FzXp/yM95y1TD+Vix07WlGlVBZo2xac\nnWHjRrC3N/JNC5E6yYNI8e+Ugq++In7jJgaU+oJ9lyLZ2fIR+aeMSvKQqEGtcJi1KfHG2rXhrj/k\nzwkHT7z25Y8tPMDeECsmTz7AiBF1+fTTWtiGh8I770CxYvqBqK3UPUTaI71HxJtTCgYPJuL3LQwP\nc+OTjBcod+skABE2mcgUH/GKE/x3y/rtpFSX8vTru4kcOTKxYEELPcFTYKAeeVm7NsyapSemEiIN\nktAWb+bBA1Tx4liFhSXafC1TafY3H0m337qY5LL9PPfTfXpNfl3uxfLlZ5k61ZPOnSvqH2B/f/D0\nhNatYcKEF6d9FSINkdAWr+fgQb2A7ZEjANzKmIcCUfe5VdSdQjde7HoXntuZjGFxWIdHJuuyrcrv\npfXYtyjgdJ0+fbbg7l6QGTOakCuXo97B11cHds+eespXIdI4CW3xao8eQdmycPduyl+69YdcOeVH\nxIMgKhZ3Irtd/NNlv8LCICoKpk+H/v1TvGxCmIMMrhGvFP9+B2yeC+wwGyec4sOSOOK/u56zMtuz\ntKRN2TtkPbiNr/+Kp7pHA9p3q4lDjqx6Hm4np8R/yjqNQkhoC8Bg4M7oKeTftSNh07GKH2Hrd50q\nIUdffsyIEaBWwMoQ+PuZbnxFHcA3CpXBBqvYF7sCXq7QhOHxX1KlfwPavvWQzoO2EFzEjUWL3qFa\nNRl6LkRqkPKzh4vXZggISLSowNUjd1TNVpHqukuVhG3H+w9WgfXKvXwhggL2Sh39WKnfcihVwl7F\nrx6k4hwzvrDfrQq1VPu3jqsmvZS6cDpI/eXZS+XKPlHNmHFYxcbGm/vbIESqgwkWQWgPnAfigarJ\nOI8wB6UIXrgUKxeXhE1rl9zhr7bfcXRTRooFnGLy4EkcePiIjBl8iDPEJD6+ello0RxuR4PbMmj3\nEK5Fc3bfSWyeeyh5pUBt3HMc5P2R1ZhQcysONUsRf+kyxw515bPPamFrK932hEgJpQFXYA//Htrm\n/g9LPO/UKfWwXLVENeETpdqoR7ZZVXS2nCrSIaNq5GtQvrHB6tC3LdWjCoVV3D+Pa+Q5c+o/u7d+\nreXA7trlVevfmal8b4aplW49VaC1ozr2+VRzfweESPUwQU37EnDllXuJ1OPePeK7dSfMvT7Zzyce\nlXgpXz1OX7pO8+V7CClYmCWF73Hlp25UXnKCzDsOYZMtJ5QqBXPm6AOWbyVy+Kds8ZvHvnGNX3q5\n2h0UdO2Ka8wxLpesSR3fvdid8Kb690NMfadCiH8hNe3ULiJCqQkTVGzW7Morc8UXasPLV4apRf8o\nVeR4iNr2d5CKd7BX69Z2VDF5XZS6fv3pefLkUSp7dqVAReXM/NKa9d/5q6gQG2fl1ftHdfbsfXUk\nW3mlQN3r9IlS0dHm+x4IYWFIoqb9qt4ju4A8L9k+Etj8WpEuzEcpWLMGNWwYN3MWZ1N0RQZFeiXa\n5WHLTnjVd+TIP3Cjdg6sYvWse++2/0XvULw4NG8Ob78N9+4lHGf/7MRPgJ/bOwyNHkyrnOf4KGo8\ntwLv4O3WkaGRPqgSJci9YoFJb1WI9OJVoe1pjIuMHTs24b2HhwceHh7GOK14lS1bMAz9kgmuPVnl\nn50LkQMAOJC7GRVDjuEcE8SYDt0JjoejReFszHH231rNgEKTwMYG4h932du2Tb+ScCFXbUKuR7I2\n8G0AJmRrTefdi8ldrRg0/Rar68ZfxFeItMbLywsvL69X7meMEZF7gCFAUlO3Pa7pi5R2dfYv3Bs2\nnpUfT2P8uL7PAAASlUlEQVRWht1kmD0TgFtlG+BQIS+xe/ey+PRNvspljZ+VL7+ylp63PXEpmLi1\nKyy3E3H5chHT5RNyfT4sYfvaD5dy8GgsM2/0AiDSxoGM8c8sXuDgoAfGNG4Mv/xi+hsWIg0xxTD2\ntsAsICcQDJwCmr1kPwntFGYwKH744Qgbx61ho/NWnDu9j/WkCXhnqUvN4AMA7HVvRP6aVSkx4zvi\niWcO82m9y4HC7YcQWyQ//lVy89s0d0pmr0ldLxuyNH0X2xjd7W/j/26QbVAf8tiHUDzqKscqNaPz\nmWL84nqOGndPYrVqJbi7Q2go3LkD2bJB7tzm/JYIYXFk7pF0Iigoki5dNhAQEM7qWW9RxK0UANtd\nu1O/rgMZl8wj3toaG4NBH9CpE+GN6zLL8zLtxvlws5gtZ4c0oVp8Rdx+v0vMyHFkvnoZgNPzf2fY\niTbk99nNkiONAGhRYgx5czsy9+EK7MuVgp9+gqxZzXLvQqQlSYV2SjDnA9h05cCBm6pw4Rnqs8/+\nUDExcery9+sTenVEPnyU8D50yjSlKldW6sYNpRYuVIb33lOx2XRvkNi6tZR67z0Vmb/A0x4hnXup\nrycEq4rV/laXarRXcQULqU1VO6jrNjmVd9/xypAzp1ILFihlMJj7WyBEmkESvUcktNMAg8Ggpkw5\noHLnnqo2bbqklFJqy1c71YMMLgnBe6ZW/add8zJmVGr69MQnOXHipV34bldvrppX9VZr3pqg4rLn\nUKfbfKKK5JqgBvdYrffJkkWpc+dS/qaFSOMktNOowMBw1br1KlWz5o/q5s1HKipaqUldD6hAOxd1\nc83+pEcr+vvrE/z9t1JffJGw/bsBY1Vo7rzqXptuaqz7L+pvx2IJn53NUlJ9Vbibujx9qVLFi+vt\nCxaY9xsgRBqVVGhLm7YFO37cn3bt1vDee2WYNKkRD4NtGNnlJDP2NsNu9QpCmzcmd4Yk/orPnYMJ\nE1C//YZVXBwAEVmzYZszF9/XW8J6n7ys5XMKPvRhRfXuzN4VwpSGBhr8NvXpOapVgwoV4OefU+Bu\nhUhfkmrTlpl6LJBSivnzj9G8+S98/30Tpk9vwokLNnzU4gKzDrUg84iBxJw7SVCFSkmfpEIFWL0a\nq7g44h7PU321Yjt6WU+k1tV1HLleg4hSRSln6MtWVZItXr1pEHYOatWCq1fh0CFo2RIq/cs1hBBG\nJzVtCxMeHkPv3ls4f/4Ba9a0x9U1B79shu++uY33TTccAv0xFCrErnJuNPlj7Zud2y4zVjVqEFul\nCoMvF2b/TZg9uxmNbfygSxf4+GP45htZjECIFCA17TTgypWH1K69GBsbKw4d6kHx4jkYMgXGzIG1\nEyNxGDwQTpxgabcv3iiwv60wl0PLz2Eb9IDvm46h2EoXCterwNkTPWjstVgH9rJlMHGiBLYQ6YBZ\nG/NTtZCQ1+55sW7dBeXiMkXNm+etDAaDCgtXqnV/pTw+Vuph0NP9HnTo9K9TpRqcnZUCNbXdNpX3\nLaUWrVEqLk6pnTuvKVfX2ap161XK1zdIKV9fpWrXVqpJE6Xu3zfJ7QshkoYJpmYVybV3r25bjn9x\nWa4n4uIMDB26ky++2MG2bR3p27cGt+5aUacjZMsMOxZB9idjWc6dY0eJygnHum84k/DeMHsOIaGK\nr74JYU3OThTLcI/L26BR9SDee281ffpsZdo0TzZs6ECR439CzZrw7rt6zpFcuUz1HRBCvCEJbXNq\n0UL/+fXXL/3Y3z+UBg2Wce7cA06c6E316vk4dQHcO0KnlrBkAtjZPXPA2LF0+lbPVR1Zpx4HruxM\n+OjgcQOuzeBuADRrm5N3Kj/khxn7qFHjR2rWzM+FC/1o2agQ9OkDw4bBli0wZAhYy4+IEOmNuX/L\nSN1KltRNF8/Zu9dP5c07TY0fv1fFx+uRhtv3K+VSR6nfdrzkPPv3P11Zxs9PqV9/VQYnp4Rmkd/L\nDlSnLuiBOFdbdVXjs7VS7733q7px4x99vI+PUuXKKfXhh0oFB5vwhoUQrwMZXJNKXbmig/XiRaWU\nDtVp0w6qXLmmqu3brybstuYPpXLVVerAieeO9/FRqvUzS399+qkyuLur2IyOiduzJ0xQPj73VePG\nK9RP2Ruqq72Hq8cXVGrhQh32S5bIUHQhUomkQvtV82kLUytZUv/ZogUR5y7RtesGfH0fcfRoT4oU\n0Y3Vv2yGodPgRMMFFMhYH6KLwYYN0KHDC6e7Fe/CDKfxeNWtwbxy63AL2E38mTNs3XaNXjOXMWrU\nW3StUAWbnFng0SPo3RsuX4b9+6F06ZS8cyHEfyANlqnB6NFw4wZv1VqIg4Mt+/d3Swjs/22C6d/e\nxu9aYQp80xfKltXzVD8J7Fmz4MoVIspVY0SbvdT2+YoyPRvgvc2ZGs6+HA/MwN4rMcTHxHHhQn8G\nDXLDJoMtHDsGVaroh4xHj0pgC2EhpKadCuyq/j6ejGdmiWvUWdQTq1PH4MgRbq47TIPjh+kUfTvx\nAV27wuTJkDs3x31g7pQHzLx0jfyDa3PtI3Cwh127rhM6eyd+ecrQtVkVsld3hZyZ9PF2drBnDyxZ\nAm3apPj9CiH+OxkRaUZKKX4cs5Fjc9bz46MVemOmTODqim/B2sy45MbEXL/gdHCX/qxzZ/jmGwyF\ni7LrEExfCpduwI+lfqGh/zpsN67n6tWHDB68k/PnAzjsuBKXqd9gdeyYrp0PebwK+sOHEBcnCxMI\nkYrJIgipRWwszJ9P/N59BG/fg1VMDPb165LJxgA7d4K3N3eL1KBcS9g/4jzlejeABw8SDo/LYE+E\nciDGxgEHZ3scszlgFfQPkaO/YcytEvz882mGDHHn889rYV+uNGzdCiVK6INtbMx000KINyWhnVqM\nGkXMzj/57lF5HhSvxHe/fUImRzu4eROKFIH69RnawovwSJg3Rh9y8w4s+BV+Xm+gpms0X3wQTf2K\nUVjFRBMfHsGGNWf5bMFNPJuXYuLEhuTJ4wQGg661BwVBxoxmvWUhxJtLKrSlTTsl7dtH7MIfqW0/\ngBbd6/LDWA+soyLhm0n6gWL37rB2LbF1w9l62JH2n8ExHwgJgy6tYd//rHEtkhHQIbxnjy+ffbaX\nrFkdWL+5IzVq5H96rXv3IEsWCWwh0hgJbVNSCq5fBy8vyJuXiO6f0DOmBUNnt6XDB+Vg7VoYOlRP\nd3ryJBQuDIGBfF9oFVVr9MTWBiZ8BiUKJR6YePFiAMOH7+bMmXtMm9aY994r8+R/5adu3tTnE0Kk\nKRLapuTuDhcu6FDeuZPVmdwZtHsctewDoH59CAmB5cv1+yf69sV65Ei6nOgBzwVxYGAE48fvZeVK\nH4YPr8Ovv7bDwSGJv0IJbSHSJOmnbUrffovKmBH/E1cA8Fw3lVpX/oKmTaFjRzhxInFgAzRuDMHB\n4O2dsCkiIpZvv91H6dJziIszcOFCPwYPdk86sAH+/ltCW4g0SELbhKLr1md82b7ke+gHQMED22Dj\nRpg5Ez75JHFvDqV0E8nIkXDjBrRsSVycgcWLT1K69BzOnr3PkSM9mTu3BS4ujq++uNS0hUiTJLRN\nJCgokiYNl/L+2VXEjhkLxYrBhAlw+LAeiQi6eWTFCqhYUTdaV6sGDx6gvv8eAgKoUm4WK1acZfXq\ndqxZ054SJbInfcGwMH3uBQugb19Yv/7pEHkhRJohXf5M4P79MBp7rmCu9Tbq+O7DatEiXfMdNkzv\nkDmzDuzn9e/PHs9eDJvgTbbQAD7/vj1NmpZI/JBRKd30ceZM4tedO3qIe6VK+lW5MtSr90K7uBDC\nMkg/bWOLjdVBefNmolfkpevc8z5P0dgAvV+ePFC7NuTPD3PmvHieNm3g4485lLU8o8Yd5PbtEMaP\nf5v27cthHR0F588nDuezZ3U3vifh/OTl6gq28lxZiLRCQttYJk2CefPg/n09DLxw4YRXoJMLg3+4\nTIvONXn/l6/0flWrwuLF+hUQoMPe1la3a3/wASduxjBm9B4Czl7j2/Y5aOgSis25szqgfX11GD8f\n0C4u5v4uCCFMTELbWObOhc2b9euZRW7v3QvD3X0xX3xRmwHec+CPP6BGDT2DXrNmepWa7Nl1zxHg\nQfuuXN3nQ8agB5S2CyWjgw1Wz4dzmTLPLU0jhEgvJLSNJSREDzc/d043eQDBwVF4eCyj29tZGBS+\nBxYtevG4EiV0jXz3bu5kys1829rUfb8WDTq5Y1eyOOTNK+3PQogEpgjtqcA7QAxwHegGBL9kv7QV\n2gD9++smirFjiYuMZpzbl3QIPkiZcD+s4uJ0O/bChbqWfeoU6vRpDL5+XM+Qm9PkIV/zetTo1QL7\nmlX1Q0khhHiOKULbE9gNGIDJj7cNf8l+aS+0z5+H8uXhyy9hyhQADN9NwXrgAOjSRTeFdO1KcHAU\nq1b5MGvWUbLYxjPm3ew0dgnRbdanT4OPjw742bN1E4oQQjxmigmjdj3z/ijwXjLOZTkiImD44/+b\nHgd2XJWq2MbFgo0NatcuzvQazbzem1m79gINGhRl7tzmeHgUSdx178oV6NQJsmXTDyuFEOI1GGtw\nTXdgm5HOlbrNn6//nDQJgG+Ld8N32GSifpjLyl6zOBOdjff6elGoUBYuXuzPunXv8/bbRZ8GtlJ6\nAIy7u66Vb98uixEIIV7bq2rau4A8L9k+Etj8+P1X6HbtlUmdZOzYsQnvPTw88PDweJMyph6hobp2\nvW4d9OwJwOrrDvz67TVWRzrw9pbZxH78AdfmD3xx1j3Q06X26KH/3L9f9w4RQgjAy8sLLy+vV+6X\n3O4KXYFeQEMgKol90k6b9oQJepKnBw/AzQ2cnfX7efNg2TK9duPp07q73vM2bIA+fXTYjxkjXfmE\nEP/KFA8imwLTgfpA4L/slzZCOyhIzx+SJw9Ur65D+t49KFdOj4bMkEE3mXzzTeKue6Gh8OmnsHev\nnoa1Th3z3YMQwmIkFdrJadOeDTihm1BOAfOSca7U7/vv4dEjKFpUr2JubQ358kGjRnrSp4wZYdy4\nxIF98KCeA8TaWtfAJbCFEMkkg2teR0CArmWXLw9//gmOz0yN6uUF/frpboBPAjsmRte4Fy/WDx3b\ntDFLsYUQlssUNe30Y9o0KFAAtmxJHNigFzGwstLhDXDxop4g6skETxLYQggjktB+HW5usHMn5Mjx\n4mdWVrqmPWeOXpy3Xj3o3VvPTSJd+YQQRibNI8YQEqLnISlbVrdvu7qau0RCCAsnE0aZ2rVreiIp\nmdNaCGEEEtpCCGFB5EGkEEKkARLaQghhQSS0hRDCgkhoCyGEBZHQFkIICyKhLYQQFkRCWwghLIiE\nthBCWBAJbSGEsCAS2kIIYUEktIUQwoJIaAshhAWR0BZCCAsioS2EEBZEQlsIISyIhLYQQlgQCW0h\nhLAgEtpCCGFBJLSFEMKCSGgLIYQFkdAWQggLIqEthBAWREJbCCEsSHJCezxwBjgN7AYKGqVEQggh\nkpSc0J4CVAIqAxuAr41SIgvj5eVl7iKYXFq/R7k/y5bW7+95yQnt0GfeOwGBySyLRUoPPzBp/R7l\n/ixbWr+/59km8/gJQGcgAqiV/OIIIYT4N6+qae8Czr3k1fLx518BhYClwAzTFFEIIcQTVkY6TyFg\nG1D+JZ9dA4ob6TpCCJFenEE/M0wkOc0jJYGrj9+3Bk4lsV+JZFxDCCGEkfyGbio5DawDcpm3OEII\nIYQQQqRDaX0gzlTgIvoe1wNZzFsco2sPnAfigapmLosxNQUuoZv5hpm5LKawBLiP/o04LSoI7EH/\nbPoAg8xbnLTF+Zn3A4GfzFUQE/HkaU+cyY9faUlpwBX9DySthLYN+iF5ESADukJRxpwFMoF6QBXS\nbmjn4emDOifgMmnv7/AFKTX3SFofiLMLMDx+fxQoYMaymMIl4Iq5C2FkNdGh7QfEAqvRD9TTkv1A\nkLkLYUL30P/ZAoShf9vNZ77ipIzkDq55E+llIE53YJW5CyFeKT9w65mvbwNuZiqLSL4i6N8qjpq5\nHCZnzNDehf515Xkjgc3ogThfAcPRA3G6GfHaKeFV9wf6/mKAlSlVKCN6nftLS5S5CyCMxgndm+1T\ndI07TTNmaHu+5n4r0QNxLM2r7q8r0BxoaPqimMTr/v2lFXdI/EC8ILq2LSxLBnSX4/+hJ64TRlLy\nmfcDgRXmKoiJNEU/wc5p7oKY2B6gmrkLYSS2wHX0r9V2pM0HkaDvL60+iLQCliNTaJhEWh+IcxW4\niR4VegqYZ97iGF1bdPtvJPrhzx/mLY7RNEP3OLgGjDBzWUxhFeAPRKP//iytSfJV6qI7AJzm6b+9\npmYtkRBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBDCdP4PTYr2BzIP/zsAAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sampling = MultiSampling(lambda log_p, x: SliceSampling(log_p, x, w=1.0), \\\n", " log_p=p2d.logpdf, \\\n", " x=np.array([1.5,0]), \\\n", " directions=[np.array([1,0]), np.array([0,1])])\n", "samples = [sampling.sample() for _ in range(1000)]\n", "plt.plot(map(lambda x: x[0], samples), map(lambda x: x[1], samples), 'r-')\n", "plt.contour(X, Y, p2d.pdf(XY));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By choosing the sampling directions more clever, we can substantially reduce the random walk behavior and improve the efficiency of sampling from the 2-dimensional Gaussian." ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD7CAYAAAChScXIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XVYVNkfx/E3ICGKBXa3Yge74uqK3YG9ugYqBvaqa7eu\nGKuu3bpgd7smKJioSFiAnRigdM79/TH+dFkxmQEGvq/n4XmGmXvvOcf4cDj33HNACCGEEEIIIYQQ\nQgghhBBCCCGEEEIIIYQQOkBP2wVUqlRJ8fLy0nYxQgiR1pwGbP77pr62S/Xy8kJRlDT7NXny5BSv\ng7RR2iftS/l6aPoLqJNYpmo9tIUQQmiOhLYQQugQCe0ksrGxSekqaF1ab6O0T7el9fb9l9ZvRALK\nu/EZIYQQX0lPTw8SyWjpaQshhA6R0BZCCB0ioS2EEDpEQlsIIXSIhLYQQugQCW0hhNAhEtpCCKFD\nJLSFEEKHSGgLIYQOkdAWQggdIqEthBA6REJbCCF0iIS2EELoEAltIYTQIRLaQgihQyS0hRBCh0ho\nCyGEDpHQFkIIHSKhLYQQOkRCWwghdIiEthAi/bhwAX74AdzcUrom301CWwiR9gUHQ//+0Lw5ODhA\nrVopXaPvJqEthEi7FAU2bYIyZeDFC/D1hZ49QU8vpWv23TKkdAWEEEIr/PzUvWofH1iyBNq31+mw\n/j/paQsh0paoKJgyBSpUgLx54cYN6NAhTQQ2JL2nXRBwAnIBCrAKWJTUSgkhxHc5cULdu46IgN27\n1WPYaUxSe9qxwHCgHFADGAiUTWqlhBDimzx/Dl27QsOGUKUKXL+eJgMbQNO/L+wFFgMn//WeoiiK\nhosRQghApYKVK9W96/+7fRtKlUq5OmmInno456OM1uSYdhGgCnBRg9cUQogEFEXh+vUXXFx5kKAS\nFT4EdosW8OpVmgjsz9HU7JHMwE5gKBD23w+nTJny/rWNjQ02NjYaKlYIkV74+b1m40Zv9jp7MDTo\nML1DTr//TNm4Eb0uXXT6ZqOrqyuurq5fPE4TLTQEDgJHgIWJfC7DI0KI7xIdHceBA36sWHEZH+9A\nZv0Ygt3hP9BTqQAIqfwjneJsOeIzOoVrqnmfGh5Jak9bD1gL3CDxwBZCiG/24kU4y5d7sHz5ZcqW\nzckw2zy0NNiM/sGj74+J/2sRrXebUa9+0RSsafJL6pj2T8CvQF3A891Xk6RWSgiRPj18+JaBAw9R\nuvQSnjwJ5dTRLrg0DaT10CboH3sX2FWroty6xTD/4mQ0NWTcuNopW+lkltSetjvygI4QIon8/V8z\nc6YbBw740bt3FW7fHkQu/2tQvyK8fv3hwOnTYcwYHOeex8XlPm5udhgYpK8IksfYhRApJiAgiFmz\n3Ni/348hQ34gIGAw2ZVI+G0g/P33hwNLl1avIVKtGuvXe7Jq1VXOnu1F9uwZU67yKSR9/YgSQqQK\nDx68oVevfVhbryV//iz4e/zKxCdbyF44N5ibJwzs4cPB0xOqVcPZ2Yvx409x+HAX8uUzS7kGpCDp\naQshks3Tp6H88YcbW7b44uBQHX+/QWQ7dRiK5vv44EKFYMMGqFsXgNWrrzB5sisnTnSnbNmcyVvx\nVERCWwihda9fRzB79lnWrLlKr15VuHlzILmig6BTazh+/OMTevaEhQsha1YAHB3dWbXqCq6uPSlV\nyjx5K5/KSGgLIbQmPDyGxYsv8eef52nbtgw+PgPInycTLF8Ogwd/fELu3OrPbG0BiI9XMXr0CQ4f\n9sfNzY78+bMkcwtSHwltIYTGxcerWLPmKlOnnuannwrh5mZHmTIW6rWta7aAhw8TnpA3L1hZwapV\n6uBG/WBN1667efkyAjc3O8zNTVOgJamPhLYQQmMUReHAAT/Gjz9FjhwZ2b//F6pXzweRkTBmDMye\nnfCESpXgyROYOTPBjjJBQZG0a7cdc/OMHDv2K8bGElX/J38SQgiNuHbtOUOH/kNQUCSzZtWnefOS\n6kexT52C+vUTHpwnD1hYqMes9+6FIkXefxQQEESzZpto3bo0jo4N0t087C+RPw0hRJI8fPiWbt32\n0KTJRrp0Kc+1a/1o0aIUekFB0K7dx4HdoQPExal71i4uCQLb3f0htWuvZ8QIa+bObSSBnQjpaQsh\nvktERCxz555l0aJL6ul7/oMxMzMGRUHZuAm9br8mON67eFMs80WSwd9fHdblyyf4fONGb3777SjO\nzrY0blwiOZuiUyS0hRDfRFEUNm3yYcyYE9SsWZArV/pSpEg2ACJv3iO6fhOyPfNLcM6b3sPJ47SZ\nR7X6UHTVJDAygpAQCAtDlScv48adZPv267i49KBcuVwp0SydkRyLz8rSrEKkER4eTxg69B9iY1Us\nWtQEa+uCAAQ+j8O37xzqHxif8IR+/eD1ayIuXKNXQSc2u1ujf8MXli4FZ2eifxvFL75leP06kt27\nO8oMkX9Jjp1rhBBp1NOnofTosZfWrbfSp09VLl7sg7V1QQIewAz7K+TOa5gwsIsWVc8UOXKENyY5\nqVrIgxH1H6Fft456l/Tdu3nquJQqOy0wN8/I8ePdJLC/kgyPCCE+KSoqjgULzvPnn+fp27cat28P\nwszMmFt3wXFhGE23DmTCS6eEJ40ZA2/fwqJFvBo+DeclD7ny1pJMnm/UU/86deJMx5F0GODKtGk2\n9OtXPUXapqtkeEQI8RFFUdix4wajR5+gcuU8/PlnI4oVy07AA5i2HGL3H2bLzf/sdm5lpd6v0dER\n3r4lopglsZeu8sS6DZamz8DLC2XxYhz9c7No0SU2b25L3brpawODb6GtnWuEEGmMl5d6vvXbt9Gs\nX98aG5siPHsJg2bAyb2BuD2ug0Xg7Q8nGBnBtGnw/DnY2QEQU6Iss1615ceRv9Ni9xCoXJmwc5fp\nPeY89+/fxsPDngIF5JH01EoRQqR+L16EKfb2+5VcueYqS5deUuLi4pXgt4ry+zxFyf6DStnVarGi\nQMKvOnUUZfbsD99bWyt3nU4pxWuHK54tRipKnjyKsmOHcuPGC6Vs2SWKnd1eJTIyNqWbqhOARIco\npKctRDoXF6di1aorTJniSpcuFbh1ayAmGTPylzPMXgt9KvgRdKl0wpNMTaFGDfXTjqdPq59u9PTk\n3MsCzOx5niuvq5K1ZkXw9ma32yv619mAo2MDevWqkjKNTENkTFuIdOzMmQcMHHiYnDlNWbiwCeXL\n52bLIRi3EKxKxbAqaBA5tq/+/EV8faFcOXbti+S5wyT6RDljvGIxsW3aMnr0CXbvvsmOHR2wssqf\nPI1KI2RMWwjx3pMnIYwdexIXl/ssWNCYdu3K4nJRjx86gb4+7O/sRqW+Pyd6bnymTBiEh8OMGTB6\nNIpBBpzGXKDmYjvq1a6AsbMPT2NN6FTPiSxZjLlypa9M59MgmactRDoSF6di/vzzVKq0ggIFsnDj\nhgMVqlnSZpAefSbB2M4hXLxUNNHA9unZj2M1G0LhwnD1KowfT3REHIdr/E6LhW0wXzCV7P9s5/SN\ncKysVtOoUTEOHPhFAlvDpKctRDrh6nqfIUOOkDt3Zs6d6415LnMmLIbNh2B0b9hZaR2G7Xt/fOKA\nAShNm1GgT1+C2nbBYMF+MDHh9dGLhHbsSY6c5ch425uMhXIyZ85Z5s8/j7OzLQ0bFk/+RgqNSOmb\nsEKka4GBYUq3bruVggXnKzt3Xleio1XK/A2KYlFTURymKcorn8cfzwoBRalbV1Hu31cUe3vlTcHC\niv1mVyUiXlGUyEjlee/flUCj3Mp2u21KfLyivH0bpdjablV+/HG18vDhm5RucprAJ2aPyPCIEGlU\nfLyKpUsvUa7cMnLlysT16wMxNrekoq0e/7iD6zoV4wL6YF6hwMcnHzqknntdrx43w+OosdebSe3r\nkPHyJd6WqsqlPXc4v8GbDus64u39nGrVVpE7dyZOn+5JwYJZk7+xQqNS+geWEOnO5ctPlOrVVyk/\n/7xe8fUNVG7fU5TG9opSupmiHHRVlAvuVxPvXffrpygvXijK778rSp48yh6nvUpRP0W5+zZSiR81\nWgkxy60MLL9VueqrUhRFUdav91QsLOYoW7b4pGyD0yBknrYQad/bt1GMH3+KnTtv4OjYgNa2lZi+\nQg+nfTDWHn5uEUXG5g0p7+H+8ckuLpA9O9Svj1K8OH8c98LJJBfnX1zCvJUd56LKML+pF6tW5Caj\nUQw9ehzm0qUnuLrKcqrJSYZHhEgDFEVh2zZfLC2XERMTj6/vQMhaGcuWeoRFwLE9YBizE6tcGT8O\n7OHD1WtbX7gADRqgGv4bfZfs5ohBVjxXjyVHq5aMMJzI7n472bYxN69fvOLHH9egpweXL9tLYCcz\n6WkLoePu3AnCweEwz56Fsn17ezJkLkSzgeqnMg71cCHw7gWq5hlH1cROvnBB/TRjo0Zw4QKRXj50\nzlaeAlc8OD22J0+zlqZxUW8mTM5NlxawYcM1Ro06zh9/1KNPn6r/fwBE6Jh1QCDg84nPU3poSIg0\nKTo6Tpk584xibj5bmTPHXXkaGKf0naQoeWoryro9KuXcJMfEx61BcRtbRzkReVhR+vR5/15c7txK\n56PXlX2DxirxuXIpf3farBRvqFK8bilKWFi0Yme3Vylbdoni6xuY0k1PF9DimPZ6YDHg9KUDhRCa\n4ep6HweHQxQrlp2LF/tyzCMbldpC56awbO1bKjf+gaL3/BI99+GuBRQ8fpzCGZu9fy+44y8MadKT\nv4Z0wKxoKVrV9EIvRx48VsLD+8+pXn0XVlb5uHTJnsyZjZKrmSIRmghtN6CIBq4jhPiCFy/CGTXq\nOC4u91i0qCl5i5am81g9TIxgzTo4ducatqUSX5RJ1aQx+sFvKNRu+Ic3s2fn4uxFnLtyk9Wju3Fn\n0AKqnPwFh9Z6jLFXWLPmChMmuLBwYWO6dq2YTK0UnyNj2kLogPh49Up8kye70r17JdzPD2TGKiMO\nLoCJI+FGdYjv05klh7clfoE6ddA/fUa9c8w7SvPmOHXqz48zxtIntzkH2yxj8Il2/D0LqpeNpGPH\nA9y5E4Sbmx1lylgkU0vFl0hoC5HKXb36jAEDDmFkZMCJEz247J+LHzpDuyYwcissevCKm/lyJn5y\nyZLw5g14eiYI7PAlyzjlcYMe3VsCEBtgiMV9c65cbsf9gEdUrbqLNm3KsHlzW4yNJSZSk2T525gy\nZcr71zY2NtjY2CRHsULotJCQaCZMOMX27df544/6VKtZmQHT9YiPh+krYKkJNPmtLze3fmLp1EKF\nQE8PXr58/1asRTaiKlTDbJADLd+955O5Kud+mUP3RXWZO+c0y5Z5sHJlC1q3LqP9Ror3XF1dcXV1\n/eJxmpqvUwQ4AFRI5LN3N0KFEF9DURR2777JsGFHady4OBMnN2DxFlOc9sOIIfF4Whvw9KY/ZxqX\n+vRFLCzUGxU8fPjJQ95YFGWM+UxaLOtEpZKhdOu2hwwZ9HFysiVfPjMttEx8C22up70FqAOYA4+A\nSahnlAghvtH9+28YOPAw9+4F4+xsS3BcEWr3gHpW0WyrOoO6nWZ8/gKGhuoNds+d++xhK60X8o9F\nG9aV3cjpJ1eo/stphg+vwahRNTEwkGfuUjPZuUaIVCA2Np6//rqIo6M7I0ZY07ZzTUbONcD/AUy0\ndafGLHuK372V5HLu2I2j342uTDZYQS3PNbgX/An7uGY4b+0oO8ukMp/qaUtoC5HCLl58jL39AfLm\nNWPBwmbsOZ2DBU4woHsI1ufG0sx5mcbKumlanlKxfkQVL41deENM69Vm8eKmmJkZa6wMoRkS2kKk\nMv9e3Gn+/MbkKVaegTP0KFYU2lY8SJNJA8j//PE3X9e3pxXlN3i8/z6wTiNynz4GgCqjKScaD6C7\ne07+WtKcTp3Ka6w9QrMktIVIRfbuvcWgQYdp2rQEI8c0ZMaqjJy5DIOGvsBy1RCaH/rEfOvPiCmU\nl9iIEDK9CgfgTukKFL/9YXWJsE7d6PnsR57FZWTz5rYULpxNY+0RmiehLUQq8PhxCIMHH+HGjZes\nWNEC/5dFmLAIbDsolI9zYvDInl++SLZsEB4OsbHv3wotXwQz3/ufPOVtsbKUDu9Fv37VmDixDhky\nyM3G1E5CW4gUpFIprFx5mUmTXHFwqE7rjrUZNDMDih607nmP5kNsqXDL6/MXMTJSz70OCPimspe1\nc8TxkhGbNrWldu3CSWiFSE7anPInhPiM69dfYG9/AIAjR3uwyzUXjftCr5Hx5PL4ixGNRnz5Itmy\nqZ9s/IbADurSiya+VSigysm1a63IkSPj9zZBpCLS0xZCS6Kj45g1y52lSz2YNs2GYhWq4zBdj8rV\nwPInL6a3qKyVclVGxuyatAWHhQHMnFkPe3tZ91oXyfCIEMnI3f0hffseoGRJc6bMaM5cJzPOeSr8\n3t4dhwE/a63c4D+X09stG3fvvWHTprayq4wO+1Roy90IITQoJCSaQYMO07HjDqZNq0vrHp1oPMCM\nipzn/kl9rQV24A9NObv3CpUWRlKkaHYuXuwjgZ1GSU9bCA05eNCPAQMO0ahRMfoPaczvC0wIjVLh\naDqaBuvnaa3cp9tP8tfleDZu9GHt2lY0aVJCa2WJ5CM3IoXQkhcvwhk+/CgXLjxm7bo2XLlXlKYD\nYECv50y3z6u1ct2bTSDHjP50tz9E3rxmXLvWj5w5M2mtPJE6yPCIEN9JURQ2bvSmQoXl5MuXmXVb\nBjByaVFO+cCMts5aC+yATGXw3HMT35atqdNoM717V2H//s4S2OmE9LSF+A4PH75lwIBDPH4cwo5d\nXdnjnpeOI6DbxBhmtMyCSUy0Vspd2fxv6v9ly+ThB3nyJBR3dztKl5ZdZdIT6WkL8Q1UKoVlyzyo\nWnUl1tYFmLHQnu5T8vI4BuZazWBeY2OtBPaOnL+ya9NL8vevQe1aqyhfPhfnz/eWwE6H5EakEF/J\nz+819vYHiImJZ96C1qzca8GZq9DPzoexXRJueqsYgV7Mt11flSMj+kGRCd6LyJCZ0TYHGLCsJovn\nH+PIkQCcnW3lycZ0QKb8CfGd4uJUzJ17lpo112JrW4bB4+1oN8qCHGZPOfGw4keBDd8W2EpdE4CP\nAntOwSksXvCKLtNL0rrZSiIj4/Dy6i+Bnc7JmLYQn+HjE0ivXvvJksWYfYf7MvvvbLz55znOxebS\ncML8BMcqP2VA72zc11+8vAmqED30XRKG9e0C1gwtso7Ji0pydJ8brVt5sHRpMzp0KKeJJgkdJ6Et\nRCJiYuJxdHRnyZJLzJhZHyVLFToOjWFJgT+wPTr+4xOssqN3NvjbCvGN+uhX3RFl1xPRsQeObYPo\n22cd5uamXLvWX/ZsFO/J8IgQ/3H58lOqV1/FpUtP2HOoP1vdK/Nw0VZu38yH7c6Ega3kffdfyOMb\nA/s/fC2bUK7GS35e0oPyOT1oUG8tPXpU4vDhLhLYIgHpaQvxTlRUHFOnurJu3TXmzWvEy/gKzOt5\nig1Rv1Po7tWPT8hnit7TiCSX26/WcZ5VbMAWhzB+H7GJ4OAozp7tJTNDRKKkpy0EcO7cI6pUWYm/\nfxDb9zlwdKPCTxObsuV+68QDGyCJgX2vlDUFakRSbXADfqnlS8N6K7C2LiCBLT5LetoiXYuIiGXC\nhFNs2eLLgoVNef7AnOet+rMs9ihPc+XC5HG4Vsr9rflJLmWrx77RUcz7YxdeXs85fLgL1arl00p5\nIu2QnrZIt86ceUClSit4/jyMzbsdcFn8jF+nVaVg1QiyvHlBGT9fjZcZkiMP+axjMG9RjzGd/Wnd\ndBm5cply5UpfCWzxVeThGpHuhIfHMHbsSU5t92BXjUf4mNXhvqsffcNWcrdAISr7XtZKuWdLtWNI\nCWcWT8+A06qj/PNPAOvWtaZevaJaKU/oNtkEQQjA1fU+Q+22MSXLVVoGHOEZOdE30if/m2/bd/F7\nPMtdHr9lf9NrlDs//1yYhQsbkzWridbLFbpJlmYV6VpYWAwTfjtAzm3ruaQ6i+GDt3hnqoplzHWM\nIqK0Wna4QSYeD57BOv3SOA9yYcWKFrRqVVqrZYq0S0JbpHkuR2/j0mU8UyJPkS0ymGBDc17lqkLl\nwCtaL/tMFXsMpg7BfvRpLC1D8fLqL0uoiiSR4RGRZoW9iWBP23HYnHEiS4FcmDx7zCOD/OQxfEXm\nkCCtlv3CMDc3FuzD9WUky5dfZsGCxvzyS3nZYFd8NVkwSqQfKhW+k5YQmLMoVjdOENe5N7HPgrlv\nXo4SkX5aD+zVDVfh5+LFb2uv4eHxFE/PfnTpUkECW2iEJkK7CXAL8AdGa+B6QnwfRSFy514e5S5J\nvOMcXoyezsvSjci03ZkMWTJQ+tklrRb/2LggO7YHE1S/LLZtnBg40IqDB3+Rx9CFRiV1TNsAWAI0\nAJ4AHsB+4GYSryvEt3Fx4e3gkQT6PeWfWj2p3r8bmfv2JKthOLliA+GVdouf3mw/dSfWZP7wXZia\nGnL5sj2FC2fTbqEiXUpqaP8ABAD3332/FWiNhLZILg8fEte7D8EevkzTr0v9rZuw2OtNqa51sIjT\nclIDLwxzcWTDXTI996ZNi7VMnWrDgAFW6OvLUIjQjqSGdn7g0b++fwz8mMRrCvF1du4kxr4/i/St\n8W3+Fz36NOCl3Vi6PFiZLMUvrLWKanM7sPb37SgKXLjQhxIlciRL2SL9Smpof9W0kClTprx/bWNj\ng42NTRKLFelaeDhxAwfzZu8/dM/QlV4rB2DkCdUbFsYsPlTrxYcamHF49nkMjMKxbbGK8eNrM3Ro\nDeldiyRxdXXF1dX1i8cl9V9ZDWAK6puRAGMBFTD7X8fIlD+hOVevEmnbgSNvLDjQcDA9RrbDv8dM\n7P1mJkvxx4p3I/MaR8ZPPU50dBzr17eWFfmEVmhryt9loCRQBDACOqG+ESmEZqlUxDnOIaxWPYa/\nqUHsqvX8XLwkNtamWg/spwUseWOQjZN9VuM/fBCt2m+kadMSuLnZSWCLZJfU4ZE4YBBwFPVMkrXI\nTUihac+eEWLbmTvej1lRezr9u1QhZ++WFAi/q9ViY4yMOZe7KcXfXOfO+kPM+juAUG9v3NzsKFs2\np1bLFuJT5IlIkarF7d1HZLderFBVpeLwLpTav52iPv9ovVy3BvYUdDtBdFVrzrS3Z9wsD0aMsGbk\nyJpkyCDPpAntk1X+hG6JjOR170FE79rHycI2tMr6En1vb8xi3mi12CjTTFw0b0CJN74ET53DsENh\nhIREs2FDGywtpXctko88xi50RryXNy+LlsNo6yYymmelZag3ux9Zaj2wfao0JTzGhIzVKnBoxgbq\n/nGHBg2Kce5cbwlskWpIT1ukHorCyymzyTltLAARxcqyuPAESj5wpe3d1Vot+nHmYrzIXIS4+bMZ\nu+YGoaHSuxYpS4ZHRKqmPHuGXj71dlsxxqZcHOGMwwlrfC5pdwuuyIxmhMRlwst+Hv6WJZk82ZVR\no2oyYoSMXYuUJcMjItV6tXH3+8C+PedvegwM48zeR1oPbIDjRbrhf+ACs27E4eysnhkyenQtCWyR\naskmCCLFKNHR3GzbF8vDTih6ehw9HsHQqXHcdtN+YPqaVsR//GoeZlLo1XUbY8fWYtiwGhgYSFiL\n1E1CW6SI4IteBDexxfLNPaJz5WW8/S2ih+zm9o2uWinvQYEiFH58H4AVPy3CctYvzB93AEVROHu2\nlzwkI3SGdCtE8lIUvIfNQlXzJwxMjInKW5i2pU8w/q/SLNZCYJ9uZwtA4cf3OZWtMTvXPSKsTTXa\n2q6jffuynD7dUwJb6BTpaYtkE/44kFt12pHp0V1e9x1Ovk2rWJxtAIfcymm8rCdWZThdqAZddm0A\nYHqbo9QZVZW5w/diamrIxYt9KF5cVuQTukd62iJZ+CzbyZuiZQkyyobeuq2UWjGDzKEvGPVoqsbL\nmrJmPfk9btFl1wZ25+zCzj2hGFhlpF3r9djZVebkye4S2EJnSU9baFVMRDRnG/fB8tw+7oyZQ1C+\nRjTsVlwrZR2fOgDPFzmZ0scOgHF9fGncw4JhgzeRO3cm2U1GpAkS2kJr/I9dJqJtJ7JkzESwiyd7\nF71kzh+aD+yw/Nmxcz7GjnpWNAQ257PHzGkZ+i5n6ND2MHPmNKRHj0qysa5IEyS0hcYpisJxuxlU\ndZrNq3b2vO3vyJluM5jzcIbGy1roshX9zZ7sqGcFwNTf7lC7pTGDHFZSurQF1671l411RZoioS00\n6umtx9yo15lSwXcI3nYAt1NmjGhgShNUGi3n8qh2TKw3jiN1qwGwvaA95luWELzjFF1/uc6iRU1o\n395SetcizZEbkUJjXGZvI7p8ZbLny07I8Wtc+n03v6+wwkCDgf2qXF6G3vbE76E5R5qqA3vJWF+y\nrB6LfbflvH4dia/vADp0KCeBLdIk6WmLJAsJjuBog37U9drLq6l/ci+kOO1r56FiIscqBqAX/33l\nrPPZyKmzZmwsXQWAo4W6YLZlGZ5rXZljf5fly5vTvHmp72+IEDpAetoiSa7u9+BGvkpUfHqNqIOu\nvF59hPZzGnzy+O8J7GvDG9PzkhfFh2xlY//WAOwddZSwBRPp0GEtJiYZuH7dQQJbpAvS0xbfJS5O\nxY5fptNg1zxedO5FeAlrSjetToHEjq2anwxXn3xXOX88OU/UX+5s+KESAJ75G4DTKpyXeuC7z48t\nW9rx88+Fk9ASIXSLLM0qvtn964+5Vq8rP4bexGD+fKKnz6fgU89Ej1WKZEbvftg3l3HSeRTOJk2Z\nNGEoxW77AHC5/2K8q9dkzNiT2NtXZeLEOpiYSL9DpE2ynrZIMkVRODhjO6WnDiG+YiVy1K9L7nnj\nNF7O79evUmnKMjoc2IxRVARPcpbjzbKVDF4WQEhINKtXt6RKlbwaL1eI1ORToS3dFPFV3gZHsL/B\nQJp77SSmtz3Ztjhh4nlco2XsOrYct6vGTKnTlCD9vETGGRHY3o5dVdsxo/9Zxo6txdChNWSta5Gu\nSU9bfJHHgStEdPyVopljMW/0M/G79xOql5n8kQ80VsaAY+foM34MeV++5fqb/NRQXeXJpNl02RSJ\nubkpK1awj97cAAATeUlEQVQ0l/VCRLoiO9eIbxYfr2JL9zkUalOXsmUtMM9izKWjz5nrcIE8sd93\nY/G/tq9exEL7Mczt1IqbIVa8CTXmB8sYFnRdQJ05Lxg2rAbHjv0qgS3EOxLaIlGP/APZXagxzbZO\nJ1v5EpgG+DMqwzhCJ4xk+vySGMTFAXC7Y+XvLmPwom3YTPmTmlcfMDfrGNo9c0K/XQOqPGnG7bdG\n+PgMoHt3WTNEiH+T4RHxkZOLD5Lnt74UNwjB0MSQk9mbs/qHuSz3aYLFTW8AQoqYY/o8hAxRsd98\n/QvN2/A2Gqrdus74bI50U+3BKuoS08v0ZeN1A5Yvb07TpiU13SwhdIoMj4gvigiPYfPPA/lpaFvK\nxT1DldWcX/Nv5l63kezYnu99YN+2LU+W+6+/K7CdOthT/pwbr0IrMyzfahYEjSJn7ihKh/Yg2rIC\n1687SGAL8RnS0xYA3Dhzg2ctfqF+qDeKnh6uPw5mmPEUjkV2IPelkwBE57XA+Nmr7y7jUdFSRGYq\nTPcMi5lVYAe1zi1ier7OHDIqz6pVLahWTfu7rwuhK2TKn0iUoijsGbwM6+XjsFSFEF3Skr5512BR\nIBNemz/c/Av6yZIcZ298dzkhOfPxV45pBJay5mTAr7y8HkYVlT19erfk4qAfZBqfEF8pKT3tDsAU\noAxgBVz9xHHS006lXjwOxrVOd9rfPYSeYQa82o2n9b2RnA1tSYEbLhor56jVYIabTGdt7WNUWzaA\nxYa1cPuxA38taS47yQjxCdroafsAtsDKJFxDpJDTa46T1aEXHWMfE/dDDaZWWct13ygeXMz8/pin\nF1zJV8Pmu8t4VaAMv1hspFqL0ly8MZCIJcdpa9KNPiv7MaJNGQ20Qoj0JymhfUtjtRDJJjoqll2t\nx9D6+BKMTQx5NmYxTTzsObyzCvlf3wTgXo2fMRvSOUmBPa/6EvYU68+6Dp7kGVSOg29z42m3lM2O\nzciSxVhDrREi/ZGBxHTE79IdXPLWpMux+Rg2qMv+xTew31MZr39M3gf2Zeed5FXeYtHF4bvLKW/1\njCwjBrAz31RydamHo2kjSp/dx5xlthLYQiTRl3rax4E8ibw/Djig+eoIbVAUhcPjN1Bh9gjqmCpE\nb9jEwBsdWNI3C21VUQA8zVcIg6FDqd6t/VddMyaHGUZBoQneG1lmLS/b9OJI2/tEd6zOg8fB3J7o\nzIzxLTEwkP6BEJrwpdBuqIlCpkyZ8v61jY0NNjY2mris+ApvX4dxom4fWvvsIKxlO55PW8aCARdY\nc8Ho/TF7+42i5dkjGIwe8dXX/W9gWzULZ/5UU4z3LMOkxhjOWzaj9t3l1CiYXWNtESItc3V1xdXV\n9YvHaWKetgswErjyic9l9kgK8Tl4iej2nSlqGE4m57UcMW6MbbMPYR1jaMj1Dt2osnndV11PlTkz\n+mEJ18Y+mMOWm3N2067GM2636E6Fx1d5Nm8FVkM7aLQtQqQ32ngi0hZ4BNQADgFHknAtoUEqlcKu\nAYvI36ouWRvUIuujOxzY8iJBYF+tYIWSLftXBzbwUWBP7XSaCt47sPDZQGylyhTIAjmf+ElgC6FF\n8kRkGhMcHEn37nvJe9eLyTMakKFaTXIXzpTgmIBiZShx9/sn/1zJUYvQzQcwNH2DW8eR9H99jKhZ\nc8gzon9Sqy+EeEfWHkkHzp59SJUqKylRIjtLr81EdeL8R4ENJCmwL7ScQAHfIxzcuJ/ohk2xN/Mn\n601PCWwhkon0tNMARVGYN+8cf/55ntWrW9KypgVYWHz43MwMvdDQz1zh66gymxFYyJLQ2/copB8K\nQ4di8sc0MDRM8rWFEAnJ2iNp1OvXEfTuvZ9nz8K4dMme/JOGQ6v1AMQUKobRw7saCey3zduy8k52\nHkSbMmDjTEyaVoesWZN8XSHEt5HhER12+fJTqlVbRfHi2XH7uw6FCmfD4O/17z83enj3u689usRy\nwiwKEte4GdN/20uJizUw6teHv24tpHzn+hLYQqQQCW0dpCgKy5d70KzZJubPbcCfpu4YlS31dSdn\nSPjLlWe5agm+n5d/IlPGPWFWyBSe1m9JGb+G+DyKwcurP8OGyaa6QqQ0GdPWMeHhMfTte5Dr11+w\nd3o5ivzxO1y48M3X2dq6O533Ob3/PkbPkIG/3OT3mcUxVr1mQa9VHH5mxuLFTWnUqLgmmyCE+Aoy\neyQN8PN7jbX1WkyUGDwa3adw23pw4QLPjPN//sQSJT5669+B/VeZmbieiWbJusJs23iGKlZryVG/\nJt7e/SWwhUhl5Eakjti9+yb9+x9kTZdMtDw0Eb0tAVwr2JAHGYvT2m9Foue8f4IxIACA4GIlyX7X\nP8ExB6a4MmhCHU6dukPFikcoW9aCK1f6UqSIrHMtRGokwyOpXFycirFjT7Bnuw8XKl7C4p/dhLfq\nRJ+g4cy5bkfBlz6JnvfKPBcWr18AEFGsOKZ377z/bF+O9rQO2glAWAtbBsY1xt0vmoULG9OyZWnt\nN0oI8UUyPKKDnj4NpV69v/HxecGl012xsCqH75EA2j4YwRbX6okGtspI/aj6/wObzp3fB/Yt03Is\nbH+A5qYexKxei+OkY6w7EcRS93HcmpiNli2+8mamECLFSE87lTpz5gGdO+/EwcGKceNqo6+vx1E3\nhbC2v9Lu1eYvX8DYGKVwYfT8/AC4lfMHYvceorxDA26Wr0sz98JUr56PuXMbUvT1HejVCwoVguXL\noWBBLbdOCPEln+ppS2inMoqiMH/+eebMOYeTUxsaN1bfRDy04R7N7Yp93TVKlkTPXz12HaVvgokq\nCiIiCKvXBNfHGRiVyZaly5pTr17RDyfFxICjIyxeDDNmgL096GvwFzGVCsLDITRU/RUS8vHr8HAY\nNUpzZQqhwyS0dUBERCw9e+7l3r037NjRQX0zUKXiesfxlNvl+FXXUBkZo8TGggI3O42lXAk9ogNf\n4u3uz+u7zwmYvZr+A3/E0NAg8Qtcv67udZuawsqVkCdP4gGb2Huf+zw8HExMIEsWMDNTfyX22tFR\nsz8shNBREtqpXEBAEO3bb6dixdysWtUSE5MMcOdOotP1ADAyUveO/yVOPwNxigERBUqQZfsGlKqV\nCS1UkptB+uTJbkhWD3csClkkfr1/i4+HRYtg8mRQlC8H7dd8njkzGHziB4UQ4iMS2qnY0aMBdOu2\nh8mT6+DgYIWeSgVz58LYsV99jdeGFmTRi0Bv2DAyTJvE8TOPWdF/DbvuOhJdqBjGVy4mWERKCJG6\nyYJRqZCiKCxYcIG5c8+xe3cnatUqBDduQO3aEBT08Qk5c8LLlx+9HZ41N9nzW6D/9wb8sxZlRIfd\nXL/+kiMVX6KE5cLY5bgEthBphAweppDo6Di6d9/Lpk0+nD/fm1o/5lXfACxXLvHAzp79o8AOyVMC\nJWtWMjn0Isz1HKO2BWFtvZaaNQty44YDpRpUQe/IESj2dTcwhRCpn/S0U0BgYBjt2m0nb14z3Nzs\nML3tC406gb//xweXLKl+Pzg4wdtKyZJkMTQkfu1R1vsaMrHiapo2LYGvrwN58mRWHzRwYDK0RgiR\nnCS0k5m3dyAtW26hR49KTBlTA/0Zk2HWrESPfZqtJPn+G+SWlhAYiF67dpy26cGQfq5ky2bC/v2d\nsbL6whokQgidJ6GdjA4d8sPObh+LFjWlc6EQqFYVbn289Zdq+AhOZWlIg6lNEn5QpQpERXFv6SaG\nbQ7Ga8s/zJvXiHbtyv7/poUQIo2T0E4mS5deYsYMNw5ubcUP+5bBbzs+WtsaADc39C0saNDkX4Fd\nogSEhBBRqy4T43/GaZA3Y8b8xLZt7dVTA4UQ6YbciNQylUph2LB/WLLEg6tzivFDn6bqG43dusHj\nxx8ObN5cPW598yaULQsPHqjft7JClSED69pMpdDm3ESRgRs3HBgxoqYEthDpkPyv16Lo6Dh69txH\n6KNArlldxnj8SZgwAbZsgY0bPxz499/q0O7ZE/btU7+XNSuKiQne5pa0f2ZJlWBzLlzoSIkSOVKk\nLUKI1EFCW0uCgyNp1WorzVW3GH1/E3oVWqqfMBw5Et68UR9UqhQcPgwPHybcPd3SktC3UfQxsOVF\nZHn+3lafmjVlESchhAyPaEVgYBi2tZbw10snRgfuQm/lCoiOhj59PgT2+PHg6QmrV0O9eu/PjcmW\nA+eggtTPPoxeK/vj4tJDAlsI8Z70tDXs/r1gFtQYzsGI/WTqa4dei0nqFfPuvNuEwNQUjhyBvHnB\n2hq8vQGIy5yFJ6pMjMzcmfbzenGxQzn09WVGiBAiIelpa9CjizfxL/czk/TPkPnoQfSyZ4eGDT8E\ndseO6puPAQHqoZF3gR2rn4E1+tU4PX8XW+7No1On8hLYQohESU9bExSFtwuXk3HkaLI26oj5X6Oh\nd29wd/9wzLp10KYN9OsHO3a8fzsgQy48B8/EblZPjI3lr0MI8XmSEkl1/z6xvfrw5II/7v0X0reW\nKVhZqdeQBqheHTZvhmfPoHjx94+jq9DDq35XLHcuo0Q2sxRsgBBClyRleGQucBPwAnYDWTVSI12h\nUsHixSjVq7PhoTl/d52DfYgrdOmiXvgfYMwYOHsW1fbtUKfO+8B+k6sQsS6nqXLCGWMJbCHEN0jK\nwGlD4CSgAv6/rcqYRI5Le+tp376tngmiKPxRvCevbz9iXqATek+fgrExZM0KTk68rWrNls0+NBzW\nkuIxgSh6eijDhqM/cwZkzJjSrRBCpGLa2I39OOrABrgIFEjCtXRDXBzMng0//QQdO3Jr2XaMd23n\nz4vT0Lt/HzJmRGnYEM8N/9B3SxhVC/1BtWkD1IFdqhR67u7oz/9TAlsI8d00NUXhALAFSGyb8LTR\n0/b2Vu+dmD27em61SkV8mbIYxKq3/AquVINTpRsz/po5sXEKk22g65HZGDx/BsOHq9fKlrAWQnyl\n79255jiQJ5H3x6EOaoDxQAyJBzYAU6ZMef/axsYGGxubLxSbikRHw8yZsGKFetPZFi2gSRPw9MQA\ncKIim0u1JzJrQSrmysWGdeX48dg69GbMUN94dHNT98yFEOIzXF1dcXV1/eJxSe1p9wTsgfpA1CeO\n0d2e9sWL6ql7hQtDy5awbRv8/w+1ZUvYvTvhSn2PH0PXruqgHjZM3bs2NU2RqgshdJs2NvZtAvwJ\n1AFefeY43QvtiAiYOBEWLoR8+SA2FgID1Z+VLQt79kDp0gnP2b8f7OwgRw5Yvx5q1Ur+egsh0gxt\n3IhcDGRGPYTiCSxLwrVSj9OnoWJFmD8f8uSBTp3UAZwxo3qHGW/vhIEdHQ1Dh4KtLXTvDl5eEthC\nCK1JysM1JTVWi9Ri2zb18qitWsGSJVCkiHrda5UKLl2C8uUTHu/nB507q+dlnz4tYS2E0DpZe+Tf\nmjeHp09h61Z48kS9oFPLlnDhwseB7eSkftrx55+ldy2ESDbyGPu/ZX63i/n+/bBoEZw6pd6X8b/m\nzoWVK+HQIahdO3nrKIRI15JjKTnduxGpKPC5jXJfvoRMmWRmiBBCa7Qxe+Rr6V5oCyFECtPG7BEh\nhBDJTEJbCCF0iIS2EELoEAltIYTQIRLaQgihQyS0hRBCh0hoCyGEDpHQFkIIHSKhLYQQOkRCWwgh\ndIiEthBC6BAJbSGE0CES2kIIoUMktIUQQodIaAshhA6R0BZCCB0ioS2EEDpEQlsIIXSIhLYQQugQ\nCW0hhNAhEtpCCKFDJLSFEEKHSGgLIYQOkdAWQggdkpTQng54AdeAk0BBjdRICCHEJyUltOcAlYDK\nwF5gskZqpGNcXV1Tugpal9bbKO3TbWm9ff+VlNAO/dfrzMCrJNZFJ6WHfzBpvY3SPt2W1tv3XxmS\neP5MoBsQAdRIenWEEEJ8zpd62scBn0S+Wr77fDxQCNgALNBOFYUQQvyfnoauUwg4DJRP5LMAoLiG\nyhFCiPTCC/U9wwSSMjxSEvB/97o14PmJ40okoQwhhBAashP1UMk1YBeQK2WrI4QQQgghRDqU1h/E\nmQvcRN3G3UDWlK2OxnUArgPxQNUUrosmNQFuoR7mG53CddGGdUAg6t+I06KCgAvqf5u+wJCUrU7a\nYvav14OBNSlVES1pyIeZOI7vvtKSMkAp1P9B0kpoG6C+SV4EMETdoSibkhXSgtpAFdJuaOfhw426\nzMBt0t7f4UeSa+2RtP4gznFA9e71RaBACtZFG24BfildCQ37AXVo3wdiga2ob6inJW5AcEpXQoue\no/5hCxCG+rfdfClXneSR1IdrvkV6eRCnF7AlpSshvig/8Ohf3z8GfkyhuoikK4L6t4qLKVwPrdNk\naB9H/evKf40DDqB+EGc8MAb1gzh2Giw7OXypfaBuXwywObkqpUFf0760REnpCgiNyYx6NttQ1D3u\nNE2Tod3wK4/bjPpBHF3zpfb1BJoB9bVfFa342r+/tOIJCW+IF0Td2xa6xRD1lOONqBeuExpS8l+v\nBwPOKVURLWmC+g62RUpXRMtcgGopXQkNyQDcQf1rtRFp80YkqNuXVm9E6gFOyBIaWpHWH8TxBx6g\nfirUE1iWstXROFvU47+RqG/+HEnZ6mhMU9QzDgKAsSlcF23YAjwFolH//enakOSX1EI9AeAaH/7v\nNUnRGgkhhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghtOd/RfNceZqepi8AAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sampling = MultiSampling(lambda log_p, x: SliceSampling(log_p, x, w=1.0), \\\n", " log_p=p2d.logpdf, \\\n", " x=np.array([1.5,0]), \\\n", " directions=[np.array([1,1]), np.array([1,-1])])\n", "samples = [sampling.sample() for _ in range(100)]\n", "plt.plot(map(lambda x: x[0], samples), map(lambda x: x[1], samples), 'r-')\n", "plt.contour(X, Y, p2d.pdf(XY));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Hamiltonian Monte Carlo (HMC)\n", "\n", "HMC uses gradient information to automatically find suitable directions. In this sense it fixes the above shortcoming that we have to specify a suitable proposal distribution in Metropolis-Hastings or adapted directions in multi-variate slice sampling. HMC makes clever use of a physical analogy and augments the state space with an auxiliary momentum vector $\\mathbf{p}$.\n", "\n", "Write the probability density $p(x)$ as\n", "$$ p(x) = \\frac{1}{Z} e^{-E(x)} $$\n", "and assume that the *energy* $E(x)$ as well as its gradient with respect to $x$ can be evaluated. Note that any $p(x)$ can be written in the above form by choosing $E(x) = - \\log p(x)$.\n", "\n", "Now, define the Hamiltonian $H(x) = E(x) + K(\\mathbf{p})$, i.e. the total (potential $E(x)$ plus kinetic $K(\\mathbf{p}) = \\frac{1}{2} \\mathbf{p}^t \\mathbf{p}$) energy of the system. It is well known from physics that the following dynamics conserves energy\n", "$$ \\begin{array}{lcl} \\dot{x} & = & \\mathbf{p} \\\\ \\dot{\\mathbf{p}} & = & - \\frac{\\partial}{\\partial x} E(x) \\end{array} $$\n", "\n", "Thus, we can sample from the joint density $p(x,\\mathbf{p}) \\propto e^{E(x)} e^{K(\\mathbf{p})}$ using Gibbs-like steps:\n", "\n", "1. Sample $\\mathbf{p} \\sim p(\\mathbf{p}|x) = \\mathcal{N}(0, \\mathbb{1})$\n", "2. Produce new state $x'$ by simulating the Hamiltonian dynamics for some time. Due to energy conservation the proposed $x'$ will always be accepted.\n", "\n", "Also volume is preserved by the Hamiltonian dynamics. This simplifies the algorithm considerably as no Jacobian adjustment is necessary to account for the change in volume." ] }, { "cell_type": "code", "execution_count": 42, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def integrate_H(xp, dE_dx, T, dt, method='euler'):\n", " \"\"\"\n", " Integrate the state xp for some time with the Hamiltonian H(x) = E(x) + K(x) \n", " \"\"\"\n", " x,p = xp\n", " t = 0\n", " while (t < T):\n", " t += dt\n", " if method=='euler':\n", " gradE = dE_dx(x)\n", " x += p*dt\n", " p += - gradE*dt\n", " else:\n", " error('Unknown integratin method')\n", " return x,p" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Unfortunately, Euler integration is very unstable for the Hamiltonian dynamics. Further, to ensure detailed balance we need to make sure that the integration scheme is reversible. In practice, a leap-frog integration is applied where half steps to the momentum are interleaved with an update to the state. In addition, leap-frog integration leads to a deterministic, discrete dynamics that is exactly invertible and preserves volume (as each update changes one variable based on the state of the other variable only, i.e. it is a simple shift in each variable)." ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def integrate_H(xp, dE_dx, T, dt, method='leap_frog'):\n", " \"\"\"\n", " Integrate the state xp for some time with the Hamiltonian H(x) = E(x) + K(x) \n", " \"\"\"\n", " x,p = np.copy(xp)\n", " t = 0\n", " while (t < T):\n", " t += dt\n", " if method=='euler':\n", " gradE = dE_dx(x)\n", " x += p*dt\n", " p += - gradE*dt\n", " elif method=='leap_frog':\n", " p += - dE_dx(x)*dt/2\n", " x += p*dt\n", " p += - dE_dx(x)*dt/2\n", " else:\n", " error('Unknown integratin method')\n", " return x,p" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Demonstration on the 2-dimensional Gaussian example:" ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXVcVecfxx+xVrrf4BIKiJ2zna0z5nROp3N2x2w3Wzen\nswu7W2d3zw4QA1tUVBBQUJDuhnvP+/fHY05BFATR5/168VLuOfec5xy3z/mebwqhUCgUCoVCoVAo\nFAqFQqFQKBQKhUKhUCgUCoVCoVAoFAqFQvFWfCKEuCCEcBJC3BZCTM3Y5SgUCoXiTfjs8Z/ZhBDn\nhRA1M3AtCoVC8dFglAbHiHn8Zw4hRFYhREgaHFOhUCgUryEtBNxISBeKvxDCTkhXikKhUCgyEV8K\n6UKpk8HrUCgUio+CbGl4rHAhxAEhRCUhhP2TDwsVKoSHh0cankahUCg+CjyEEIWT2yG1LhSdEOJ/\nj//+qRCigRDi2gsr8PAQwAf7M3bs2Axfg7o+dW3q+j68HyFEodcJcGot8DxCiLVCPgiMhBDrhRAn\nUnlMhUKhUKSA1Ar4TSFEhbRYiEKhUCjejLTIQvmoqVOnTkYv4Z3yIV/fh3xtQqjr+xjIkg7n4LE/\nR6FQKBQpJEuWLEK8RqOVBa5QKBSZFCXgCoVCkUlRAq5QKBSZFCXgCoVCkUlRAq5QKBSZFCXgCoVC\nkUlRAq5QKBSZFCXgCoVCkUlRAq5QKBSZFCXgCoVCkUlRAq5QKBSZFCXgCoVCkUlRAq5QKBSZFCXg\nCoVCkUlRAq5QKBSZFCXgCoVCkUlRAq5QKBSZFCXgCoVCkUlRAq5QKBSZFCXgCoVCkUlRAi6EEJGR\nQkRHZ/QqFAqF4o1QAi6EEEOGCPHFF0J07y6Eh0dGr0ahUChSRLIj69MIgHQ4TSpwdxeiTBkhPv9c\nCBCiWTMhxo0Twto6o1emUCg+UrJkySLEazRaWeBCCFG4sBDOzkLodEI0bCj/LFdOiNGjpXtFoVAo\n3kOUgD+hYEEhzp8XIjRUiCtXhDh+XIgHD4QoWlSIFSuEMBgyeoUKhULxAsqF8l8MBml5b9woxJYt\nQuTMKcTAgULExgqxcKEQ1apl9AoVCsVHQHq4UKyFEHZCiFtCCGchxO+pPF7GkzWrEFOnCrF4sRA/\n/yyEvb0QDg5CDB4sRMuWQnTrJoS/f0avUqFQKFIt4IlCiMFCiFJCiKpCiP5CiBKpXdR7QZMmQly8\nKMS2bUK0aCHEjz8KceeO9I+XKiXE3LlCJCZm9CoVCsVHTGoF3E8I4fT471FCiDtCiLypPOb7g42N\nEKdPC5E/vxAVKwrh5ibEjBnyswMHhChfXgg7u4xepUKh+EhJSx94fiHEKSGt8ajnPs9cPvCk2LFD\niH79ZHph377ys927ZQ551apCzJwphJVVhi5RoVB8OKTEB55WAv6FEMJeCDFJCLHnP9s+DAEXQuaL\nt2olRPHiQixfLkSuXELExAgxfboQixYJMXSoFPScOTN6pQrFB0VQUIzYtu2WOHPmgQgOjhWPHkWK\n/v2/EX36VMropb0zUiLg2dLgPNmFEDuFEBvEy+IthBBi3LhxT/9ep04dUadOnTQ4bQZQuLAQ587J\nrJSKFWWWSoUKQowfL0SXLjLQWbq0DIB+913GrXPFCpkK2b+/XI9CkQkxGDRx4ICbWLHiqjh92ks0\nblxENGpUWJiZfS7Wrbsu3NyCM3qJaYq9vb2wt7dP13NmEUKsE0LMSWYfPkg2bwZTU5g9GwyGZ5/v\n3w/580PbtvDoUcasbfx4EAJy5oSmTeH69YxZh0LxFjx6FMHYsXZYW8+matWVrFlzjYiIuKfbNU2j\nQYN1rFlzLQNX+e4RQrzWdZHaIGYNIURHIURdIcS1xz+NUnnMzEHbtkJcuCCzVJo0ESIgQH7epIkQ\nt24JUaCALM9fsCD9i4DGjBFi+HAhLCyEKFRIiAYNhOjRQwgfn/Rdh0LxBty86S+6dt0jSpZcLAIC\nosW+fe2Eo2MP0bVrOZEr1zO35PTpZ0VYWJxo2/brDFztx0NGP8jeLQkJMGoU5M0LR4++uO32bahT\nBypUgIsX039tCxdCnjywdy+MHAnGxjBmDEREpP9aFIokOH/+Ic2bb8HcfAZTpjgQHByT5L47dtwi\nb95ZPHwYno4rzBhECizw9CCj70P6cOIEWFnBiBEQH//sc02DdevAwgL69oWQkPRd19GjYG4OM2eC\npyd07CjXsnQpJCam71oUiue4eNGbBg3WkS/fHObPP090dEKy+584cQ9TU1uuXfNNpxVmLEIJeDoT\nGCh9zt98A+7uL24LCYE+faR4rl8vhT298PSESpWgZUtpfV++LN8MSpeGU6fSbx0KBeDo+JBGjTZg\naTmLJUsuER+vf+13LlzwxsxsBidP3kuHFb4fCCXgGYCmwYIFoNNJof4vFy5A+fJSQG/fTr91xcZC\nz55QvLg8r6bB9u1gbQ3t2oG3d/qtRfFRcv26Hz/9tBkrq9ksW3aZuLiUvQGeP/8QM7MZ/Puv6zte\n4fuFUAKegTg5SbHs1Olln3NiIsybJ0X+zz8hOjr91rVypTzv9u3y96go+OsvMDGBadNedP+8C3bu\nhGzZpFtJ/3rLS5H5cXUNok2b7VhYzGT27HPExqbcdXf0qDs6ne1HJ96gBDzjiYqSVm/hwnDp0svb\nHz2S6Yb588v0w/Ti8mV5zoEDnwm2mxs0aQJFi8KhQ+/u3J6eMsVRCChRQj5Ink/DVHww+PhE0LPn\nPnQ6W6ZMcSAq6s2Mgx07bmFmNoMzZ7ze0Qrfb4QS8PeEbdtkzrit7avF6uhRKFIEfv4ZHj5MnzUF\nB0OzZtI3/ry/fv9+KFRIbrv3jvyN3t7yvF99JUW8SpWMydJRvBOCg2MYPvwoxsbTGT78KCEhSWeV\nJMWsWeewtJzF1asZVEvxHiCUgL9HeHpCjRrw3Xfg4/Py9thYGDtWujLmzUsf94KmyXOZmsLWrS+u\nZfJkmXY4dqz8Pa2JiYEOHWQ8YMIEGdz99VcICEj7cynShbi4RObOdcTU1JZevfbh7f3mqX4Gg8bw\n4UcpWXIRXl5h72CVmQehBPw9IzFRipW5Oeza9ep97tyBb7+VFurVq+mzrsuXpdXdp48U1id4eUGL\nFnLb4cNpf15Nk28lFhbyATJokPTPL1igUhwzEZqmsWHDdWxs5tCkySacnf3f6jjR0Qm0bLmNmjVX\nExSUjnGh9xShBPw95dw5KFhQ+sejol7ermmwZg2YmcHgwRAZ+e7XFB4ObdpAmTLyIfI8Bw7I9bZs\n+W6yVeztwdJSFkQ5OUHdujLF0d4+7c+lSFNOn/aiatWVVKy4DAcHz7c+jp9fJFWqrKBjx10pzk75\n0BFKwN9jwsOhSxcZNLx8+dX7BATIfaytZTXlu0bTYPlyaQWvW/fitpgYGD1aunhmz057C9nfHxo0\nkG8fPj4ybpAvnwzyvk1cIDERgoLSdo2Kp9y7F0KLFlvJl28Oa9c6YTC8WNfg7Qe7j8GUZeD5mmf+\nzZv+5M8/lzFjTqKlZ33Ee45QAp4J2LJF+qCnTUva733iRPoGOW/ckCmQXbu+/Ibg4gL16kHZsvJN\nIi3R66WLKU8eGdiNinr20Jg6VbYtSCmOjjLTpU6dV7/lKN6K8PA4Ro48hrHxdCZOPEVMzLN/k7AI\nmLcOiv8IuurwQy/o+Td8WVkK+qvYv98VU1Nb1q9XDdf+i1ACnknw8oLataXYPHjw6n3SO8gZFSUF\nvHhx6dZ4Hk2DTZuk0P76a9pbuidPyt4yo0dLS9rdHRo1gq+/frOHxrJlz1IWd+5M3+rXDwyDQWP1\n6qvkyTOTbt32vBCg9HgAAybC/6pAmyHgcOnZrXZ0Apv6L7+waZrGzJlnyZt3Fo6O6ZR5lckQSsAz\nEXo9TJki/d7btiW93/NBzitX3v261q+Xbwjz5r0sgGFh8NtvMii7enXa5nP7+sL330P16jKDR9Pk\n20qePLKnTFgKMxQ8PJ6JeIUK8liKN+L0aS8qVVpO1aoruXjxmT/k7FX4+TcwqQZ/zHrZyo6Khipt\nYOHGFz+PiUmgS5fdlCmz5KPPNEkOoQQ8E3Lxoiz86dYt6a6B6R3kdHeHypWhcWPpq/4vly/L/i81\na8LNm2l3XoMBZsyQD5AnD7WQEOjVS1ro27enzKpOSJBNxp4IeXqlaWZyvLzCaNNmO9bWs9m48Qaa\npqFpcPg01O4E+b+DBRsg8hUeqrh4aNADuv754nP90aMIvvlmOW3b7njjwp6PDaEEPJMSGQk9esj0\nvfPnk97vSZAzX753H+RMSJBl/3nzwpEjL2/X62HxYhkAHTYsbR8qFy/Ke/Hrr8/82adPyyKgJk2k\nCyolnDoFQmAQWdiefxiJTs5pt8YPiKioeP7++yTGxtMZO9aO6OgENA1OOEKNDtLHvX5v0nHs2Dho\n3BtaDXrxOeng4Iml5SwmTTqlgpUpQCgBz+Ts2CGt7IkTk7cYT55MvyDnk7a5Q4e+um+Kn5/s/2Jt\nLXPd0+p/1IgIedznffJxcTLo+SaZMSEhRDXv8NQYP9hhw7vv/5JJeJLPbWk5i3btdjx1b1y8AXW6\nQOGGUriT+08xLh4a9pS+8OdjzosXX8TMbAYHD959txfxASGUgH8APHwo86Jr1kzef5ueQc6gIFlq\nX6GCzEp5FXZ20kL+8ce0Lclfv15a+fPnP3s4uLo+G5yRkriAphE0f+Mz1/gnznjvTSKV8yPh4kVv\nqldfRfnySzl3TgbSb96F5gMg77ewbOvrn4+RUdJt0nrws32joxPo1m0PJUsuws0t+N1exAeGUAL+\ngWAwyIpFU1OZ/ZEc6RXk1DRYskSK6apVr7a04+NlYNbEBCZNkhZzWuDmJq+vaVPZg/3Jep7EBYYM\nSZELx+DmwYR8y58K+eK62zCEp0PR1HtEQEAUv/66lzx5ZrJ69VX0egPeftBjNJjVhFlrICYFnRSC\nQ6FaO+g26pl437sXQtmyS+jQYafyd78FQgn4B8aVK1CsmOwhklwWRnoGOZ2dZdVk69YQGvrqfe7f\nl2JbrFjaVVfGx8vApKWldOs8ISBAulpsbODff19/HL2e833/eWaN57jBreVn0maN7zEJCXpmzz6H\nTmfLwIGHCA+PIzQchs+Ar6rAiJkQmsJWJh4PoOgPMOy5Xm379rlgZjaDefPOK3/3WyKUgH+AREXJ\nNDobm9dP00mvIGdsrEwntLGRwcWk2L1b+s+7dUu73PEjR2RgddSoF5yucQeOy8Bnq1aybe9rCDvl\nROtcB58K+dgyu4jz+TArOR0cPCldejENGqzj9u0A4uNh/nppcfcYDT6PE430GjjHwrpQGOoLvq9w\noTg6QZ7asOjxi6Feb+DPP49jbT2bs2eTqGlQpAihBPwD5t9/ZU70H3+8Pgh38qQs2X/XQc79+2VO\n+NixSTtMIyJkH3Jzc1munxbWmb+/LPSpWhXu3SM4WIpwsaIGVv+4g0QTc+nueU2euhYdw/qG656K\neLGsbpwed/yDKQDy8Yl4mha4ZctNDAaNXcdkcLJRT7juIi/1Ugz87gtmLlDYDRp7gbgFnv/5z2z9\nXllx+e/jlyp//yjq1VtL/fprCQhQ1a+pRSgB/8Dx95euifLlXz+eLb2CnI8eyZa5NWokH3S9dEmu\nu149uJsGmQkGg8xE0elgwwb695ciXKoUFLGJY2Phv9FXS1me+v31p6mZ48JTIe9jc5CwW5l35NwT\nd4mJyXRGjTpOdHQCl51lLnfpZnDkjLS2d4ZDZQ8ocBf+9oe7j0MW0wOh5XPGtMEAf8+XFZbOj//p\nzpzxwtp6Nn/9dQK9Xg3oSAuEEvCPAE2TJeM6HSxc+HprMT2CnM8HXZ/vM/5fEhOl6JqYyHTAtAhy\nXrsGxYujtWvPqKGxFC8ub0/VqhplrQKxy/2TdLfEJD9kQB8QzOTSm8ku4vkyWySWRj7s6n0o000P\ncnDwpFSpRXz//Xpu3w7goS90/gMsasnMkthEWB0KRdykeO8Ih+f7UjlEganLMzGPjJKZKTU6gG+A\nTD2cN+88pqYf59izd4lQAv4R4eoqqyF/+EGWoSfHW2RsvBWXLsn89O7dkz+Hp6csyClRAhwcUn/e\n6GgZJ8ifn8VD3DAzg2PHZDGnjVUiLSzP42FTF44ff/0lTDxE0axu5PssEIvsgfxscgofB/fXfi+j\n8fOLpEuX3VhazmL79ltEx2hMWiLL3kfNgcBIWBIsre2698Eu6uVn/81YsHCFw4//6e57Q9nm0k8e\nHy8bW7Vtu4OyZZfg4RGS7tf4oSOUgH9kJCTAmDHSv7xnz+v3DwiAjh1l8PHAgXezpshI2RSraNHk\nLX5Nkw2nLC1lxWVwGuQM79sHFhbYdVqFubnGvHnS8J48GYxzxTMi12LC2/R87RSgKJeH9LHci0X2\nQOpY3sU0SwCrfjmAlvD+9a02GDSWLr2Eqaktw4YdISwsjs0HwLoutBwIt71gUTBYuUrf9pkk5iZc\njgFzF9j0ONnp+Dkwrwlz1sp/Kmdnf4oVW8Cvv+59oSOhIu0QSsA/Us6ehQIFpBCmxLo+elQObGjb\nVlZSvgs2b5YulZkzk3dDhIfDgAFySs+GDakPIPr6QsOG3CvbnNLF4ujcWRrojx5B1w4J5Pk8jJW5\nBqFf9U/y5zIY2N9jNxZZ/Ghc2JXSX3jQIJcjnodeE3tIR65ceUSVKiuoVm0lN274cfUWfNtZWs12\nl2BtKOS7Cz94wYVkPEj2j90meyLkLZmzVrpc7C7I7f/8cw2dzpbVq9NpYtRHilAC/hETHi7T9QoX\nTr6fyhOio2HkSCmyK1a8G1/v/fuyu2BSc0Gf58IF2XO8QYMXhy6/DQYDzJtHlEk+OlZzp3RpDdfH\n7tpLl6BG2UjKf3oHxwr9eLohCfzsbvNjrlOUzX2PLhVuYJIliEWN/8UQm3GFKhERcQwadAgzsxms\nWnWVgGCN3mOlxbx0CxwIg9LuUP0enH7NpLJd4VK8T0RBdAy0HwblW0j3SUxMAt2776FEiYXcuPGO\nHvSKpwgl4Aq2b5e+7vHjU9YrxMlJ+tJr1355tFpakJgo15LcXNAnJCTIboQmJtLvkdqeJTduoJX6\nmiUVlmOqM7Bjh/xY02DjOj15ckfSK+c/BP9hm+y5tNg4FjfYhUmWYH6re4Oq/7tN7c8v4bY7DTsx\npgBN09i58zZWVrPp0mU3vr7RLNgApjWg/0RwCITvPWWAcnf4619mlodIn/flGHD3gjLNodNIWYl5\n61YAZcosoU2b7URGqqrK9EAoAVcA0tpt0EDmSafEmtXrZaqhiYkU27QqgX8eR0fptkmJm+f+fdnK\ntmTJ5AuFUkJMDPz+O5fMf8TGPJYhzzVdCg2F/p3DMc8Zwj95/kBzSP5cd9ZdpEKOGzSyvskfDa9g\nkiWImfUOoI9KQe15KvH0DKVp000UL76QU6c8OXdNukrqdIHjd6CbjxTjhcEQr2kEEsR5LrKV7dzi\nRbePpsEYfyh0V2abHDwli3oWbpTb1q+/jk5ny4oVV1RVZToi0knAVwsh/IUQN5WAv8c8diMk27vk\nvzx4IPPMixdPm+yQ/xIRIQOcRYrIlrH/4fp1GZM9dw40gybTSPLmlf3AQ1KZ9XDwIEHmJfmhoAs1\naxhe8OhcuqhRsWAwtXOcw7nl2GTPFR8Yzqgy+7DI6s/sbjeoo7tJlU+cuLXxWurWlwQJCXqmTz+D\nicl0Jkywx9M7kU4jZcOp1QdgrD+YuMAIPwjSJ3CRy8xlAVOxZRs72MluZjD76fGiDdD2oUwhfBQH\nY+aDZR04fVm6Zrp23UPRogu4fl25TNKb9BLwWkKI8krAMwnOztK3/PPPzxpBJYemyba2T4QzqX4n\nqWHbNunmmTz5hQKjyZNlIU3RotKVP3s2RD8Kg379ZBXq5s2pC3L6+2P4sSkT8y4mj2niCy1V9HpY\nOCMG3ScRjPx8AVH/JD88wmHSKfIbefFruUvMaHcFkyzBTKp+gISw1zid34DTp734+uvFNGq0ARfX\nYBZskJWQI2bBan+wdpVifDc+DntOMZlp/MM63HFHQ0OPngUs4jo3AHiYABU9oKM3eIfIasxvO4Nf\nINy44UexYgvo3n2PcplkEOkl4EIIkV8JeCYiLg6GD5eifOhQyr4TGgp9+kjh3LYt7cvLHzyQLWFr\n1XpawRkRAWXKyAJSR0do0UK6zqdOhYhj52UTrYYN5di0t0XTYPFijuVugUXuaMaN1V4oUvX1hfbf\nB2CT3Zu9FcZJd04ShLn60dnyOIVzerFt3C0aWjhRLsctrq249PbrQ3YM7NZtD5aWs9i2zZkzVzTK\nPHaXbHaFKvekENtFPxPuzWzFn2fTk/To2cQW1rEBDY1TUdLFMj0Qzl6TaYZDp0NCgsaCBRcwMZnO\n2rVOSS9K8c5RAq5InpMn5eCFAQNeW5n4lDNnpC/6TSbhpJRXtM3185MiPnTok/xjaNdOeoKmTtIT\nPWHm202t/y+3buFT8jvq6G5Qr1bCS7VQxw8lUFQXxE/ZD+I5alnSAWFNY1uvo5hl8WdsQ0dW9r+K\nqVEgYyodIC4wiRF5SWAwaCxffhlTU1uGDDnM/Qdx9Bgt3SWLDkMXb8jrCmtCEzijnWMK09nCtheE\nGyCRRDazlTWsJV5LYHGw7HNyJIKnQc+9JyAkJIbmzbdQseIyXF0/zEZemYn3RsDHjh379MfOzi6j\n74vieUJCZP53iRJwNYV5vfHxckqQiQnMmZP2fVX+0zY3JASqVJHxzie66eICLVvKup/lkwNIbPDD\nm0+t/y9xcegHD+PvXLPJYxzHsWMvbWbioCBMsocxLc9cEs4lbVl7n75Hgy8vUCX3bRzWuNMs3xVK\nZnfl/LwUpHQCTk6+VKu2kipVVnD1qi8rtsvAYr+pMM5b+rlH+ulxMFxiGjNYx0Ye8XIFbiyxrGQ1\n69lEhCGBzt4ypfBKsBxIXOEXcPMEe/v7WFvPZuDAQ8TFvX8FSh8DdnZ2L2jleyPgikzAxo3S8p02\nLeWC7OLyrK9KSsU/pURHS5dN/vxw+jSRkdJb0rDhi63QL1yQnpdixTR2DTmNZpFHfi81vvpjxzih\na0XeL8L4a2TiS8a2h7vGD2W8KZnVhVO/zEty+LQhPpG5Px7FJEsQy3+9wOaR1zA38mdY6UPEPHr1\n+qKi4hk+/CimprYsX36Za7c1qreX093n35IpgU28DBxJvMEs5rKKNTzk1Y22wghjPovYy37uxRuo\n5CF95GduyQ6EAyZCVLSBv/8+SZ48M9W4s/cMJeCKN8PTU+Z/166dfCfB59E0mdViair96lFp3EZ0\n3z7p+B49msSYBPr3lx6c56e0aZp05ZctC9W+SeR886nSv79169v76gMD8WvUhe++cKRWxeiXuvBq\nGuxYE4HVZ8F0+WwbAWsPJnko5223KJvzDj/lvcStYz60KXSJItk8cJh29oX99u51IV++OXTsuAuP\n+5EMnS7dGxN3wY+P87k3Rd9jEUtZyBLcSDol1BsfpjODUzhwMFLD3AVmBMLcdfKYmw/IiTnVq6+i\nQYN1PHr0Zu4dxbsnvQR8sxDikRAiXgjxUAjRTQl4Jkavh+nTpSBv2JDy7/n7Q/v2soT/8OG0XZOv\nr+z3XbkyuLmxYIGstD/zn8E5BoPs0ZU3L7RvEMCDovVl/ngygcdk0TQMS5cz6bPJmOeO5sC/Lz8M\nIiJgSCsvTLMGsazcYgwPXm0Nx4XGMKLicfIY+XFg4hV2j79O3qy+DCh2hLvnPWnRYitFiy7g+PF7\n7DwK+epBu1Ew2FO6S6YEB7FW28QMZuHEdQw8VykbdxgMzx6cd3BhMtO4od1ifID0kx8IkL1QKvwi\ni3Q2bbqBqakts2adw2BQud3vI+kl4K8jo++D4m24elX6xdu1e7Oc60OHpNujfXsp6mmFpslBxo/z\n2A8d1DA1hZUrX941MlLmjxsba4yue4ZI43wyOPq2QU4XFxyK9sDqkwCG94965WGcLsRRzcqLqtku\ncm3k5iTdUPazLpMv60P6fu2A19UAmprbYS3uM6r2Wm7dTeSHXlD8Rxh/GWzuQjvvWLbqDzOJqZzC\ngYQEJ4j4CwJKQNhvUrgTPUHT0NA4zRmmYsuNxAd87wm17sPea7J392+TwD8gjo4dd1Gs2AKuXHn9\npCJFxiGUgCtSRUyMHJWWL5/MWEkpUVEwbJjM7V69Om1TDm/elOmDv/zCHccQihWTLu9XVb4/eCDj\noHnNE1ldcgaG0mVT1hfmVcTHE/jbeH7IeZyqxUNe6WEyGGDFeB/MsgczyGITEedeXVof6hlGy7z2\nFMjixs/FlrBq2AWsjLz5LudxRs4IoOF9KOluYEPcFaZiy052Ex1/BIJqg58lhA+D+DMQVA+CfwJk\npslOdjOfhZyICcXmLgz3hSkrZfBz7wlwdHxIwYLz6NVrnxoynAkQSsAVacKhQ9IvMXz4m5XVX70K\nFSvKKONrmkS9EbGxclizlRXhe+1o3lz2yEpq9OX581CtmkaF/MGcMm4O/fsnPxQ6GQwn7LD9chJm\nn4azZ+ur70WAn4Fu1V2wMvJmW7MNaFHPinmioxMYNuwIZmYzGF1/N6ZZAvnR5ARNu0bQpMQ58hp5\n89eAfSw0LGYpy/FNOArBjcHfBqL/Ae2xZZ94H/zMIc6OcCJYynI2aJuZFRSHqQuseQjfdZeDFzwe\nGJg82QEzsxns3Pn+dE9UJI9QAq5IMwIDoXlzGSl0dk759/R6mDtXphxOnJj6hlTPc+QIWFpiGDqc\niWMTsbSUnXRfhabBli1gY62nRf7LuJtVk42+3ubtICSEc/X+wia7N4M6BCR5SQ67gyj15QMafnoK\n9zUOHDrkRsGC82jXbgfOLlF0GgllKj6iaq6rlP3MiW5bbjJl0WZsst+njeVBAs91BT8ziJoH2nMP\ni0RX8C8CUXO5z32mYssBgx0/P5CZJv+clYOGR88DT69w6tb9h9q11/Dgwds9tBQZg1ACrkhTnmSc\n6HRSlJNoOXv9uhz0s3Hjc32qvLzgxx9lCklSKvs2PHmwlCvHv0seYGoKS5cmrcsxMTBlCph8mcCf\nJsuIbNShbJ+YAAAgAElEQVQy5Rk3z6NpBC/aTLMcB6hk7Yv73Vffi4QEmND+JsYiiHbZl3Dgnwus\n2C4zQXrNhyb3oIhLIr8334MuSwCTOqwh7P4QBlWYTx6jR+zoe/DFi4k7AX5maNHLOcNZJjON/XGu\nFLgL/bzhd1uwqgsnz8P+/a6Ym89g4sRTak5lJkQoAVe8E9zdoVo12df7FVPux4+Hzz6Tev3ll9Jj\n4eGBFKJt22Q5ft++b+3GeAlNkz3MdTrujt1AyZIaPXpIT0tS+PhAx/Z6rHKHs+mLnmgzZqas3e5/\nT+3uwbz8szHNHsrWJS9OEdI0jTVrrmFmNoO+XY/QKN8NCmVxp0vRgwy+pWHiAmNC7jFHW8AabTnn\nt0yh9Cc3aGF6gECn65xd40qxnPdoae6A3+WHELUI/MyIjzvERjazUFvMzJAQTF1gnpvMMGnWH7x9\nE+nf/wA2NnM4cyaNq2UV6YZQAq54ZyQmwqRJMlC5ZcsLm4KDoVAhWL5c+qX//FN6UFq3hsuXkVkt\nvXpJv/qOHWkX5HR1hYoViWz4Cy2bxFK58iufLy9w5gyULxlLrf9dx6lY61d2RXwtiYlc6rmMQlnv\n0fv7e8TEgIdHCN99t47y5Zdy9pwPo+fJxlM9eniQN7sPjUyOsdxuObbaDLxi5qD5F4LgH4kNdmJo\ntbPkNXrEoVGniI1I4I/aZzDL4sf6tv3xjzvLLOaw1bCP5g8S+cYDpu2Vx168GZydAyhdejGtW28n\nNPTdt7VVvDuEEnDFO+fSJVn23r79C9WPd+/KNisLF8rfIyJkN0ErK9mh9to1ZG/vEiXgp59kykha\nEB8Pf/6JZpGHaV3vYGHx+tnFej0sW6phljuGfp+uJujXkXKi0RsSfuwCbT/fR5FP75Pny0VMn36G\nI6cNFGkETYZDB3fI66pntv8Z2ny3DV2WQObVH4PeszTEHX3hWCcW3cY6mw8Dih4k2rkuF1f2pNSn\nLtQzOcE/J45h7Qp9veCXoVC6Gdy8K5tQ6XS2rFyp+nZ/CAgl4Ip0ITpaNsTKl4/ne7Leuyfremxt\nn+0aGyvbkltYSIv8tlO89LmYmMgNadVXxd4e8uXjRLN55LHQmDTp9VPigoOhf49YzD4JY8n//kC/\n/TUTg/7DtWu+1Cgzj4FfTkNnFEzzGh5Y14Nh52Tnv98CHzBXW8Qm/SxiQ9tye0d1vjW5RIWczlwc\nvxvamsOVZ0Iect+RdgW3UjybC8uWLGFW9AI617XDJEsgQxoeo2B9Pf0mgNfDKH76aTMVKy7j7l3V\nhOpDQSgBV6Qrhw/L7lJDhjx1QD98KA30YcNeFNCoKNl2xdQUOneG+8fdZQn/N9/IsW5pQWgotG2L\nd5E61CgXSePGKRt2f/061C4bSrmctzhd84/Xvh3ExCQwcuQxTE1tWbnyKmv3aLTIf5hiWVyoVvI6\nFZwiWJK4B1ttCj6RA9F8TSB8OLjYo83vy7rma7Ew8qPv54sI3bhWHjR6DfjpCIlZzqARK9EZBdC+\n0j7qOccy6HdPihvd5tvPL7J16iny5JnJiBFHiY9P46ZiigxFKAFXpDtBQbJNYKlST4U4KEhq888/\nS2P9ecLC4O+/wdgYhg3VCJm3Tqr6yJEv7/w2yJlgJJhYMKT2JfLn17iUgvbcmgab1yVglTuMDjm2\n4T12+SuDnKdOeVK06AJatdrGhWtRNOwJZVpA/5tQ6uADWpnvpdCn7mzZOx2DfyEIagL2C+D3ytDJ\nBtaPg37lCBnclj6lHLAw8mN992kYfAtzI2ELk5jKmhgnCu8JoZLxTSplvUzrus7cdo2jY8FdGIsg\n/q6+B0O86iD4oSGUgCsyBE2DdetkuuH06aDXExcHHTtKA/u/vbZBBjt79pTaPXtcOHGtOsqZmUeO\npM2a7t2D6tXZ/vU4dMb6ZFMNnycqCkb1ftxCNu884s5dASAsLJbevfdjaTmL7dtvM2MVmFSDvpug\nlBs09w5gvmEla+MnsObnYeiyBLCk+Wq0QdWhb1k4t0e6i+JiYNi3sLA/6B9yfm47yn9yjRpfnWfK\n4QUMDAjA5i5MsANdNY0ula6gyxJI+1xzaPXLFi7sdKP6lzep+cVVXPenYbGUIsMRSsAVGcrz3Q3v\n30fTpLvbxgZu3Hj1V5ydZfphgQKwZeRVtHw2UvkDAlK/nsREmDABV+OqfJ0vjE6dUt480d1No2m5\nBxTJ6s7KKrMobjmFnj334XAxjkqtoHYv6OQKlncTmBN7jCmGiXhG9pPukntDce3QlbJGTrQyO07Y\n/ce9ZaLC4K9GMKUtRB8FvzyERo7ANtqWoU02YJIlkKbVztJxVAIFv4dz1zRmzz5HkVzzKZbTnVYm\nJwi+ch99osa8NmcxyRLEjO+Poo9NxWALxXuDUAKuyHD0ehnF1Olg7VrQNDZulL9u3570106cgPLl\noXIlPQ6tF8h0xX/+SZuUQ0dHogp8TadCZ/m6pD7FVf4BAVE0qnaEvFk8aZTjGLa/HMO0Bvx2GKxc\noU+gO9O1ORyPG4M+oBj4NoS1A6GlMawdQ6xPAP1K2VMgmxcXpx2EHsVgQV8IGYfmZ4FT3Cwma9OY\nGeGMzgXGbPCiXg47rI0esnKkE/XqraV69VW4uwcTG21gUO0rWBl5c3SwLPbxOO1DXWMnKn92A+dt\nt1J/nxQZilACrnhvcHKSE3N++QWCgrh8WSatjBqVdOKJwQDr18v9mtcJwaXkz1CvnsxRTC0REWhd\nu7HM9C90/0tI9mGiaRqbNt3A3HwGw4YdYd/xRMpbufNVlhA6599O7aN3mZ+4gwX68YSG/gy+VnBi\nGHTOD5Nagd+LlZ47OqzHVPgzu/wSNN8GJAZWY4t+Dgu1FfziE0Z5dxiyVlZrbj2gMafZNgoId+p8\n5cj9Gy8WPx1bcR+r7L4MLLCXmPt+aAaNZV3PocsSyMRvj5EQ+Qa9axTvFUIJuOK9IjZWZqg8Hqbs\n7y+9Kz/+mHxRZmysdKXrdBoDalwl6KvCcmR9WvRV2bGDS181IP//Qhg80PBSq1hv73CaNNnE118v\n5oSdD73+Bsu60P8s5DkWw3elHLE0esii9uMxeJvCrS4w4lvo/TVcO/HiwRITYPlQ6GTDvU2LqJzr\nAo2/PMKs86NZFn8CS1c9vd2hRhc5sPimSywdOuykWLEFnDvgzLgKezExCmZG5xskxD97Ewn2jad1\nieuUynYHp9nynA8u+fGD+WXK5bzN1bVJ+KsU7zVCCbjiveTECWlW9+9PQlg0AwZA0aJw507yXwsM\nhH79wNREz/wSi0koVS51MzCf4O1NcO3mNP7fWWpUjMXbW1rdy5dfRqezZexYO3Yc0WNVF9rYQi13\nqO0VxFz9ajYm/I3d0k6U/+watXKew6lqTdg9D/T/yQoJeAhDasJfDcF7IJpfXhzCJ9Oy3m6sjB7S\n5tf9/HFEw7QGTFsBx47fw8ZmDn367Cc6+tlT5e7GC3z/+WlK57rPmZ1+Tz/XNFg/xhWdURC232xF\nHxKOZtBY2+88plkC+avKMeJCUzi4WvFeIJSAK95bQkNls+5ixeDSpadT2fbte/1Xb96E+vU1SlqF\nceSrNlLVU9tXxWDAYDvz8fSdKMqXO0ylSsuxO+1P26FQ6AfofRVMXfRMjLZnqmECDyK6yCDl4W7o\n21iytMJ0zLL407fsGYI8I58d+/IRaGsBG4eBf2XiguuxXD+FJfqtlL8XQ9MJnuiMAun6xUYunPBj\n6NAj5M07K8kZlVpsHFtbbcfSyIfuVZwJ9Hvmg/J0jqS2hSvffuKI1/YLADy6GcTPVhcokcON80uv\npe4+KdINoQRc8d6zZYsMUE6cyPkziVhZwYQJr6+a1DTYswcKFdDTxNoJV7OasOvNKif/i8GgsXHE\nelYb/YIuWxBtWkVjVgM6L4FybtDU5yEzDAs5HDcCvX8huNkARtSAPmXAWc53C3YJoH+xo5gZBbCk\njxP6VX9B+7xwcRSan457kb8zyTCZyZFX0LloDL8kuwf2GBxHVVNXamc5ydDKUwkMfH0OfPjluwy0\n3I5ZtiBW/u359J7p9TCtyy1MjQLZ2HQTxMWhabB12AUsjPwYWu440f6RyR9ckeEIJeCKTMHDh1C/\nPlSrxqNz96lWTRb9pKQdSVycTHIxyZ3A0K9WEfZj+9d3sHoFrq5B1Ky5mho1VnH6pDdrio/mG6OL\nFC8dhPmFeKbHHmS2fizBoT+hPbSCla2hlQnsmgP6RE5Pm4bHc01Xrs88SO2sDpTLeYPTy4aj9y/C\n3oQJzDUso7F3ENU8oNt8sKwDB04ZmDbtNCYmM+hY/RJ5jXw51nJpyoZnaBpXx+6hSrZL1Mh7jxvn\nnwn/1RMhlMj1gLb/O0jIGZmVEuAaQrsC5yic7R6nZl9+4/ukSD+EEnBFpsFgkD3GdTriFq2kT2+N\nokVTPjvCzw96dE3E/PMIln0xGP38RSnqq6LXG5gx4ywmJtOZM8eRpVsM6KpD97XQbelh+n+2GIvc\nvqw+YYvBzxzsmkIXmV2S+NCduAg5zX2cEOzr1Use1HEftDVHW9mdzT90w8roAfUrnWTcXXvMXPT0\nuwUlfoJfBsLFK3IyfJ06/+DlJd1Ax3eFkfeTIP4yW06iU8pugMHXn6WVV2FqFMjQnz2e9mGPidb4\n/btbWBs95ETvrU9fbfaMuUzerL70K3mSiIdq0MP7iFACrsh03LoF5cpB06asnR+GTgebNqX861eu\nQK2KUZT9/C72JfokXTEE3LoVQOXKK6hb9x/sz4ZRvxtU6AAdboONWzQz43eyxncw28p1xjSrP4u+\nnYHWpSB6x3+5tHQpsywtsTU15e7Bg2gGA1pcLCwZCJ3ywcUeaH5mOMb+waKLf9Enzyr+lzWUhg0f\noasG6/ZqLF0qg6Rz5zq+NBnez1ejQUlvamZz5OGElM8V9d9qR+cvdmL9WSA7V4Y8/dqRtb5Y5vBn\niNUWYl1kWmOIZzjdi53BJusDjky8kPKbrEgXhBJwRabkcUtYLCxwmmtHoUJytnJKswY1DbZuNmBj\nEsEvOfZxr6+tHMXzmMREOSNSp7Nl0eJLzFunYVINem6D/K4avYKcmWKYxs2o3mg+xrCtCW6lylDW\n6DrNC5xjRoHSrK1fH8e5c5lhZobTunXw4A70KwdjG8D9SsQG1WSZfizzE3ZS0C2WTi4GhpTfQiOj\nQxTM5U/VcoepUGEZt24lXWFqMMDUoQGYZwtiX8Vx8jUjJURHY99+GSWyutD4a0/uuUurO8hfzy9f\nu1A66y1uTNr79KFwZNpVbLI9pHthe0LvhaTsHIp3jlACrsjUnD4NBQoQ2uk3mv6QSLVqb+bejomB\niSMiMMkRzqj/LSJyvx03bvhRseIyGjRYx8kz4dTsAFW6QysXKOoewayETfyT8CexgRXhankYWA4G\nV8dr+zo2lK5M+6xrKP7JPfb+uV5a3wcOwKGV0EoHuzqj+ZrgGtWPyYYpDAlxxsoVBh6QAxdm/wO7\nltgxKlt/Chu50biSH25ur7+OM/aJ5MsdyuDPlhK/+0CKrz/+yk2mWi/CJFsok37zIy5OavY/E7zQ\nZQ1mVuk1GPwDAYh4FEm/0g5YGvmw7880SM1UpBqhBFyR6QkPh+7dMRQszJQ+nuTJ80LL8RTh7Q3t\nv31AniyP6Jvtb1bPOM6M1dLq7rUHrFw1fgu5yhTDZNwiuqB5GcOKH6CVjtCVE9nWsiWzra1xWrcO\nfVQU0wpMwEQEsLzDfpj4C/QpBdfrkxhQks0JY5irX0u5e+H85Ab1f5Ojzk5fjKF16+0UL76Qixe9\nif/3KLbGUzHJEc4fAyKfzQ5NguBg+KlGEN/kuIZHhzEp79RoMHB/wjqa5ThAUeNAjv4rX2Pu3Ymj\nhuU96uU4jffaZ8FX+/nXKZTdk/b5ThN4J/DNbrQiTRFKwBUfDLt2gbk5xzusxsJCY9q0lLdFcXLy\npXz5pXz7zV7KGd+nfJar/Fr+IC1cNErdC2Vm4lq2JPxBQkBJOFsFehQifvRPnBjyO9ONjbGfMIGE\n6GiiAgJYW68e67//nosjV1A4y136Wqwj9mY+fMI7MMUwkdER5zF10fjdHsxqwt/zYc9eV/LmncXg\nwYeJiXmu1DMmBp8BU+iUcwuW/4tkwzpDstekaTB3ajSmOcPYZjno8VijFOLjw/7qUyiQ7QGt6/jh\n7S1jvBO638PcKID9jRY+nUAdHRTDkEqnsDDyY9vAM2iGNOg/o3hjhBJwxQeFry80bsyD0o2pUiaG\n5s2Tr99JSNAzbpwdOp0ty1dcZdoKaXW3au+BeTY/mpodZLLDWDzD26Pd18Hsumjt8nB78gjm5MvH\njnbtCH/ss3l4/jyzra05NnwYhqWDoX0ewrY2o0Xe3VTIeZXZe6ZT+0EA9d2hyV9Q/Ec4eS6e7t33\nkD//XOzs7ie90Js3OVe6JxU/v031clFybmgyXLoEhcwi6PPJGmKmzHl90vxzxGz/lzG552CSM4KZ\n46NISIAzR6Kw+TyQ375cS6z9+af7nlt5ixI53GmR5xy+Tin0vyvSDKEEXPHBoWmwZAlxxnnoV+s6\nhQtrXL/+8m5OTr6UK7eUxo03cvJsJJVbQ60+0OwulL0fwtTA1fSssw7jLEGMrzCT6NYFCfu7LRsb\nfs+ikiW5b2f3+HQa5+fPx9bUlDvL5sle3mNro7kXIDi0KTMSx9Gt7QF0WQIY3XgblrX1DJoKBw/d\nx8ZmDr167SMiIgX53AYDhsVLWfnFQMw/j6Bnt8RkO+iGhUGbJlGU+cyNO1W6vFlwICKCu10m8X2O\nk5SyDOGUvUZICLSs9pAy2Zy53WceT5rCxIbFMaqGPWZZAljX00FZ4+mIUAKu+GBxdYXKlVlfejo6\nYz3r18uPn1jdpqa2rFh5lanLNXTVodcByONqYFj4eaYaxvMwvDWauxn3e/1Eq092YG30gH6fdsV+\nwkT0j9NdYoKD2dysGcsrVSJ44Z8yULm7MQZfUxxihmCrzecHHx+quUPzDr6YZfFnkNl6/ur0D1ZW\ns5MshU8WX19CW3RncO6V6HLHM3cuLzXYeoKmwfKlBnSfRbE2V//k+/O+6vsXL7G9wHCscvrTqVk4\nfn6wfEYYuhxhrMw3Hu3W7af7Xtl4hzKfuNLY9AIPL/i8+XUp3hiRTgLeSAjhIoRwE0KMVAKuSDcS\nEmDcOG4Yf0uRPBG0aRNNmTLL+eGHDdg9trpr94Wf7kK5+8HM1K9iT/xwEv0LwOEa0M6coPGdWVOj\nOnMt21Eu63Vq5nXn8qko3I8eZba1NYd790A/pDYMKgfOxYkIrsu8xDFMjT2ImWsCva9Avu/g1zGw\ndasXRbJfo2aWU7jP3JK6azt0iFuWDfjO/DoliyZw7FjSu964AcXzx9Al106iO/WGx8VFKSIxkchp\nCxn+yXx0n0WxYE4CN65rlLYMolWOPYROXfLURRMflcCEenbosgSyrMMpZY2/Y9JDwLMKIdyFEPmF\nENmFEE5CiBJKwBXpSeKZs9z7qgTfGJ2kkE04f0yXVnfv/1jd3mGt0O6awcSaGLoW4eyw/kw3McFx\nzhwMej16/yCWVV2JeRZf6udcj+Of46TVvb4R2iMTLkcPYKphFi1871HBDdrPlaXwe47rGTr0CHny\nzGT79tv83dMHy2y+nPpuQuqabEVHo438g925OlHAJJyfm2vcu/fqXaOioFO7BL7+6iGu1vXB0fHN\nzvXgAc51B/DtpxcoXyQCe3sY0CkUmxyPOFvp9xdcNDd33eWbz52p99UVPOy83v76FMmSHgJeTQhx\n+Lnf/3j8owRckS7cvOlPxYrLaPbdSh4268Go7FP5LGc81dZIX/cLVvehWtDWnMDxXVlZpTJrvv2W\nYHf3p8fyOnOG+YULc7BGI4bmnIOJURDTus8kxKc2SxJHMT1uL3ld4+jhJLsTdhgOx076ULz4Qlq2\n3EZAwLP5bId2x2L+aRi2X01BO+WQuou8cYPYKt8yyXoJxl8mMmbMq0fBaRosXQq63HHsyN1Nzq97\nxSDm5NB27WaD8W/k+TSErm1jWbpIj/kXkUz8fCqGDc9KYhNjE5nxo7TG5/9ijyEx5YFURcpIDwFv\nKYRY8dzvHYUQC5SAK941iYkGpkyR1ZRLl15+mte94O9/2fllC0w+D+H7MSfwCm2D5m4Ok160ui8s\nWID22DWQGBvL0eHDmWlhwe2JQ6CNGSysi2unUjTNvg8LnS+1V3vz9V3otATy1IatB/WMGnUcM7MZ\nbNlyE+0V+X9eXlClaAjNch4idMiEpJ3ZKcFggKVLefBVGdoWv4a1lYEtW16dSnnpEuS3TmRwvu0k\nVK1FkmZ7UkREEN53JCM+m4/JF7H076dRuVQUjT+3J+SXX194q3A95EHN3E7UyHkR1/WqHD8tSYmA\nZ0mlgP8ipA+853MCXkUI8dvzAj527Ninv9SpU0fUqVMnladVfMzcuRMounbdK3LnzinGTGwu/lqY\nS2ifCpFnuBAu2cNEv0/XiSpdtolBp6eJXIWE2FD0V0HZamLvIRdhlDOn+GnVKmFcqJAQQgjvCxfE\n3q5dhVmxIqJxySzi8wBnIX7NJeIKZxcbvmwgsjqbiZhW98Q8n0Ei5BMzUbHJF2LEr35i0IDdomDB\nr8SyZU2EhcUXSa41IUGI4f2ixb+bIsT2AiNEhd1jhCha9O0v3s9PiCFDhIOdQfz+6QrxpXVuMXeu\nEOXLv7hbSIgQnToiwu88ElvDGwnLeSOE6NhRiCxv8L/8lSvCo8sEMcx/mLiSvaoQWbOKHBHBYvcX\nnUTpraOFqFlTCCGEptfE4nLLhVaqtPh9a423v7aPHHt7e2Fvb//09/HjxwuReo1OlqriRRfKn+Ll\nQGZGP8gUHwhPOgfqdLYsXHSReeul1d17t6ymHBR6hSmGiXiFd0Dz0JHQohRDss7E6pNH/Ja7Hufn\nzXtqdSfExHB0xAhmmJvjPH6IHLiwuD6alzHOkb8y2TCNzgF3KHEXui4zMKzgEuZ8MoKvsofx+SeX\nWbzY+ZVWd1Js3aKh+yKG5Z8PQlu2PPXDmY8cQV+gMEsqLMfcVE+3bvDo0Yu7GAwwaRLkMU3ghE03\naNtWDtJ4ExITYe5cjuduztdmfggBQsCWL3vBmDFv7KJRpByRDi6UbEIIDyGDmDmECmIq3hF37wZR\no8Yqatdeg4NjGHW7wjedoY0rFPOIYFbiejYn/EFCQAk4VQM6WhI2viM7q1ZhxScd0WULYemUYDRN\n+roXFCvGtmZNifqjKfQoCOfKEhtYkVUJf2Ibvx3ru9F0uQnFm8PPv8Fxe38alJzKQdP6DMi9Bt2X\n8cye/WZjOV1coFThWDp/tY/oH1vJGXGpISYGRo0izLgAIxpcwdhYY8KEl6vsjx8HC3ONyVX3YbC2\nAXv7Nz/Xw4ckNvuFFWajnor42cqDoGpV8PBI3XUoXkl6CLgQQvwghHAVMhvlz1dsz+j7oMjEGAwa\nCxZceNqve9nWx3ndWyH/XegXfIPJhsm4R3RFe6CD+d9De0uuTxiOrakpDlOmYIiLw3XIUr7Oeosm\neY8yxSw/t8YNgrbmsLgempcxtyJ7MOWx1V38LnRZJUvh1+01MHasrOZcs+aatLp37+a2eR0aWd6g\naCE9+/en3KCOioJO7fV8rXuEi1ktOHw49TfJ2Rlq1MCjTHNafheKtTVs2PBigaa3N1SvDj9V8SPc\nvAj88cfbDYXes4cEqwKsrbGMB1cDYc4c0Olg3brUv1UoXiC9BPx1ZPR9UGRSPD1DqVdvLVWrrsTB\nMZjGvaFsa+h4Bwq4RTMrYStrEv8iLrA8XKoMvxYmfmwL9nRox4KiRXl09erTY7kfPcoq84K0ybaZ\n0jmdudusLjiWJTaw0gtWd2dnKPEzNB8gre5y5Zbyww8bePjwP+OBwsNhwAAO/q8dxfOG0aCBluLh\nE5oGy5aB7st4thr3gd9/f6Hd7VthMMCKFWBqyum2C6lUQU/lynDmzLNd4uOhVy8oWSwRt7o9oWJF\n+VrwpkRGwpAhchTe6tWyJ0vJkm/nolEkiVACrsiMaJrGypVX0OlsmTr1NOv3GjCtAT3WQLG70C3Q\nlSkGW25G9ZX9utc1hdam+K+azsLixdnduTPxjxszRfn7s7tLF+bky4fbxEFoP+tYXOxPTI38WTxq\nIpMN0+n02OruvFJa3Wv3PLO6V6++mryv29GRhFLlmF98EaYmevr1S7ln5MoVKGCjZ3CR/SSWLMMr\newK8Kf7+0LEjhnz5WTf4KlZW0Lr1i4koS5aAmZnGkQH7pPW8bNnbWc9XrkClSlC7tvx7//5gYwMO\nqUybVABKwBWZEB+fCBo33ki5cktxOBtA68FQ/Cfo7gx5XeOYGbeXxfoxRAfVgttlYUglGF6Hm8sW\nYqvTcW3NGgAMej0XFy3C1tSUw717EDeoNvQvBZe/JjqoKnPmjiJfVi++L3WM7o4xT33dJ5KzupMi\nIQGmTiX4q0L8XvMyOp2WYv94cDA0bKhRp/gj/I2Lw6xZb9ScKkmOHYPChYlu1o7xQ8MxNpZDMZ7M\nhDh1CiwsYP4oXyhfHpo1I9nmK0mh18P8+WBiAiNHyiHVFhYwenTq0iYVSsAVmYstW25iZjaDMWNO\nsvuYnjy1octCKOsGLX29mK7NwTFmCJqfGRxoAa1NMWyZxuFBg5hXsCC+j9ur3rezY2m5cqypVQv/\nWUPl8OH136P5mHA1qi9TDLa0979LpWMhVM19jUpGF1k7zomxY2UPldda3Unh5gb163O7RAsa1wil\nSBHYt+/1xq1eD3/9BfnyJnCxdHf47jvptE4tMTFSSHU6/Cev4PffDBgby+SR8HC4fx+KFYORw/Ro\nw0dA3rxw5MjbnevRI2jfHqytYeFCaNgQqlSB5wqlFG+GUAKuyAwEBUXTps12ihVbgN0pH3r+LfuL\n9L8Kpi56psccZ5Z+HKGhP8H9QjClEfQoRvTFk6ytX5/1339PTHAwwe7ubG3Rgjk2NtycOxWtX3kY\n8XhlR+UAACAASURBVA3cLEpk8LcsSvyLaXF7yXs3lu43oOhP0PI3jX61LmIhfPjTeizeXqkcKaZp\nsHYtmJlx6OdllChmoEEDuHnz9V/dtQt0Oo1VzfZK//KOHalbyxNu34ZataBSJe7/60yXLvLwkyfL\nZ06VKtC5MyQePQlWVjBoEMTGvt257OykP7xBA+jXT7ponnQaU7wRQgm44n3nwIG7T4cdHDubSIEG\n0GYK1HKH7x4GMNuwhENxozD454OLP8uJ8PP7EHDtCvMKFeLIsGFE+vlxaOBAppuY4DB+LAkLf4c2\nprC3EQZfUxxjfmeaYTa/+HpQxg06LAaLWrBpv57Ro09gamrL6F6XMM8RzIz8C9Hc0yAtLjAQOncm\nIV8hFvS+iakp9O37ei/F7dvSKu7d3I+4giWgW7c3a06VFAYDrFollXvIEG5diqJzZzA2ljpbpIj8\nk+BgaNkSSpdOdiB0siQkwMyZ0q3SpIl8KPTu/fYPhY8UoQRc8b4SERH3//bOOzyqcuvby3aKh+94\nNAkJvSlFASnii0oJeuiKFAVEQEDlIIo0UUHpNaG3IB1poRuahB4IvYQe0oBAIIUUEtIzmX1/fzyh\nSYAAM5lk8tzXhQwze/ZeW8iaNb/V+PrrDZQqNZmt20IZMB6c60Lv/eAYYDA88TBjzCO5lvAFRrgL\nLO6kWtz3rSPY2xt3JycOz5jBnlGjcHd0ZPO335K0ZRl0Lg1j/4sRXJq4uKZMzhzMmJQtuASm8/UJ\nKNsEOvwA23dFULmyBy1aeHLtmnKQoRfN1CoRQau/bSR+uoXK4rZvh7Jlif24K72/SsLRUcncD9PH\nExKUJF27VibX2veDcuUefzjVg7h+XYXbJUvChg2EhkLv3vD3v6vabsNA/WfhQhU9T5ny5Jr81auq\nMuXll+G551TVy+O29RdgRDtwTV7E1/cyZcpMoVs3Lw4cT6fKx9B8AHwUAjUv3WRa5u8sMw0l43pV\nCPkv/FgHfqgP19VEvFlVq+Lu5MQEFxdWt2tHzGFfGPkJdCkDexphjirOjtT+uJun0+xaGLWCof1k\nNTlwtXcmP/20ncKFx7N06an7tO60NOjZLobX/hbKqQa9nyyx91eSk1XdtaMj54evpFlTg1dfVZLJ\ngz4jzGYYOVLJ0r6jfFTkPGyY5Tofd+5UYXerVqpJx5TNgKyQEKWvNG58f5vn416rYkVudwBt3PhU\nphcURDtwTV4iLc3EgAHbcHGZwNp153Gbp7a199sKxQKhf9x5RpvHcS65D0aEA+zpqVrclwyDzEx8\nRozgvJcXaQkJ7HNzI/zoEfhjqhr5Oq8VxhVnIuI/wT1zGCOSduEUYKL7EaWndx0EW3eEUbHiDNq0\nWUlk5MO3CC9dmIHjPxP5/aVesGmTZf4HnDoFb78NdeviPecyVarAe+/BgYcsgf/zT+W7Z4+LU8nN\nd96xXOdjaioMHaqkjqlTVTb1r2RkwJAh4OwMXl5Pfq2MDPDwuOPEe/V68nMVEEQ7cE1e4fTpSKpU\n8aBVqxUcP51MvU7wXlfoHAxlgtKZmbGe6eaRJMU2hvCqMKcbfF4cTu+5fY5R//gHni1aqD8EHIae\n1WHAO3C6Hqbrlfgj/WcmmudQ/0oU9UKg9Wgo9QGs32mib19vXFwmsGpVDrttUInH8iWS6V5oGalf\nfZf9DNfH5a6yu8zBw1g4J4PixaFNGwh6wAKfoCAoXx56f29gGp/V+bhwoeU6HwMCwNVVSRwPWsi5\nbx+UKaO07Kf5/3DzpjpH27ZPfo4CgmgHrrE1mZlmxo3zzWqKOcHvXgZO70Gv5VAxGDpFXmOCMRWf\ntOEYkcXhQjfoUxt+bQrx2XTEJMTAlO4qMt/cASP8FUJufslo80h+uXkIxwAz3+yDIvXh25Hw55ZQ\nypWbSocOa4mOTr7/fI8gIQHatMig5isXuVTaFQ4devSbcsKVK9CiBVSsSMo2X8aOVYHwd9+pXpy/\nEpcVgDduDPH7zkDlyirZGBtrGXsMAxYtUpF2797ZJ04TEpR+Xr78gx29xmKIduAaW3Lp0g3q1VtI\n/foLOXUmnrZ94Y2Pobc/OAWYmZqyl9HGKCJvdofIIuA7XCUqV7nfnzgzm8F7vppfMq0tXKpKSsy7\nLDL9ypTMJVS/FE/TYGg2ECo0g6170/nmm00UKzaR9eufoF38LgwDJk2Cwi+lsvmlz5TsYIkmFcOA\ntWuV0P3110QH36B3b+XIR426fyiVyaSaHStVgpBzacrRFi+uNGZLER2tKl+KF4dVq7KP8lesACcn\nGDfOMk1HmmwR7cA1tsAwDBYuPIGjoztubvvw9jVTvAF0nQR1L8AHVxKYYV7AMtNYTNFvw/WGMK+X\nkkzO+N5/wsCj8P3b0LsW+LXHiHTGL7kPY8xu9I47Q5FAg57bwOk9+HkieK0PpmTJyXTt6sWNG5Yr\nXfP1hWIumQwuu5TMWrXVYmVLEB+vagyLFIGVKwkJNmjbFooVg3nz7s9bzpypmh2PHkUNwypaFPr3\nVxlYS+Hrq6L8xo1VsfhfuXwZ6tRR9d4REZa7ruY2oh24JreJjk6mTZuVVKniwbHjkfR3U9Ufvx4B\npwAYkXCeMYYbp1KGYUQ6QdgQGOAKAxvCjb9UfNy4fkcu2dgTI7wYcTdaMsU8jCnpXpQLTqFtELj2\nhBptwOdQKp07/0Hp0lPYts06HYAREeDqatCw4mWuv1xeDRaxlBa9b59qgmneHEJDOXQI6tdXisWy\nZffmGL28lBS+fTsqam7ZEt58kxxP1MoJGRkwfrz6SjBs2P113CaTSnAWKQJbtljuuhpAO3BNLrNl\nSzBFi06kXz9vjp0xUbUlfNgPPrsIFUIymGvayERjHDfjO0FUWTgxBz4vBr8Pvtc7ZZrAaxq0dQKP\nbnClIZnXK+Cd9ivjjal0jrpEuSD4Zi04vAPj5sKKFecoUmQCvXr9SWLiE4xJfQxMJjX2o1TRDI5X\n6qAcrqWi0PR0pZ84OMDkyRimTHbuVKNgK1WClSvvqBZ79iglY9Uq1IfI3LnqfdOnW3a065Urqtzw\n1Vdh27b7X/fxUZJL//5PNqJWky2iHbgmN0hOzqBXrz8pUWISO3deZMYyVR746yZ4LRi6R11nijGT\n9aZpmK9Xg7g2sHGyctCH/lKid3wbdH8DfnQF/+8xIhwITezOGPNIxqfswjnQRDd/eKsDuH4B+48m\n0qbNSipUmM7+/Vdy9b5Xr1at70tarFLJvz/+sNzJAwPvVIb4+WEYSi15+23VJPn770oxOXlSXfq2\nXw0Kglq1oGlTy0sbmzapSpS2beHatXtfi4lRSdmaNbOXXDSPjWgHrrE2fn7hVKw4g/bt1xB4IYVm\n/4OabaFfEDgFGHgkH2cUYwlMnayGUCVOhtn94OvX4epddXNhgTC0BXQpCz4jILICKbH/ZZFpBNPM\nC2gcFk2NEPgyq3Z89kqDufP8cHJyZ+DAHaSm2ma115kzqlGyT9trmMq8Bt26Wab1HVQUvWCBCrN/\n+AGSkjAMpVY0aqR08BEj7uQUb4/2zshQ07FcXNQ0LUuSnKzOfatL826B3jBU9K/nn1gE0Q5cYy1u\n7ad0cnJn2bLTePtC0frQywPevwiuoaksMK9mqjGVpJt9ILI4pO+HpcOhR1W4mTU0Kj4aZn6nJgZ6\nDoLITzGiSnIk9VfGGG4MTTiBQ4DBd35Qvrka+Xrw6A0aNlxMjRqzOXnS9gm0uDiV63Ota+J6h95Q\ntuy9mxSelqgoNemvdOl7tOazZ+Hrr1Wnuojqk7kHX1/1nqet3c6O8+ehQQOoVu3+Nv+TJ1XnZefO\nlvswK4CIduAaaxAWloCr6yLq1l3A+cA4+oyFEg1g9FFwDoAhceFMMCazybwUc2xDiGkAmVFgyoDG\nAj4r4MJJlaBs8x+Y2ROuDsGIcCDy5teMN0bhYfKi6oVkGl+AdmOheANYu9XMpEkHcHBwY9w4X0ym\nvFPClpkJAweqfQbHJ+xSusagQZbVhL29lYTx2Wd3BnujLhEYmH0jJfHx0LGjdWq3DUPtbitSRK36\nubsmPSkJvvxStevrmvEnQrQD11iaNWvOUbjweEaP3svZIDPVWkGLPtAjFEoGGSxNP6Qkk4xVKlGZ\n0BcMExzdAktHwL510KGoKhlcNgLCl0BUedJjG7LGNI5Jxgx6Xr9M0UDo6wMu9aDHMDhwKIq3355L\n/foLCQ62UPOKFVC6OCyZEa+SmzVqqBGDliIpCQYMUJrJ/Pk5T1Z6eqr3jBnzAE//FNy4oQrUnZ3v\n7xC9pe9MmqR3Zj4moh24xlIkJqbTtasXr746jYMHw5i3RmnRw9ZCtRD4JCyVhYYn0/EgIWUeRDpC\niuedE/zYQEXfJ3ZCRjqknoKYRhhRr+GXOppRxlimpfhSJDCTzsHQ6Ht44yPYfch0e+TrnDnHMJvz\nvhO4rYv3NjDNnK0qQ6ZNs2zTi5+fShi6uua8Hv3yZVWXWK8ehIZazpZbHD2qbKpb995yxosX1VCs\nZs2ybzPVZItoB66xBEeOXKV8+el07erF1Yh02vWDyi3ALQAcA2BK/DUmGpP5w1hHZsIPEFUKMk7c\ne5LUZLgRBZnXIf5biHQiMmkQk4zxzM5cQdOweKoEw/erVGngCA/YsTOUihVn0LLlitsjX/MLt3Tx\nhg3hxtFgVT7SqJFlNu3cwmRSW+EdHFQ2MyeNPJmZqoPS0VEVl1uazEzVaeToqL4p3NLAMzLURMai\nRWHHDstf1w4R7cA1T4PZbODmtg8nJ3dWrDjDgRNQ+r/w1SjoeBkqBBuszTjKKMZyxnwAYptl6d3Z\njGA10iDRHSIdSY3/HyvMvzHRmMLQG0E4BECfs1D9U3i/Cxw/k0aPHhspVmwi69ZZUH7IZUwmNXSv\nUiW4EGhSzTCFC2cVbluQy5fV4oSKFVVxeE44dkxtjujQwTqb5CMioFMn1U66bNkd+WT7dqWZDx9u\neSnHzhDtwDVPyrVrN2nYcDF16izg4sV4Rs4C5zowzQdeD4HO19JZbqxlCtOJMR2G669DfA8w/jIj\nxDBD8hKIKoU59kP2m35nFGOZm+ZDhWATzS+qFnvnOrDoD1i9+hxFi06ke/cNxMfbxwaXGTOUPOzr\nCxw+rBJ7nTqpBKOluDVXpVgxlTzMyZCr5GTo0UNlXnPq+B+XfftUpUq9emqcLqjZ4vXqqW8klpi3\nbqeIduCaJ2HTpkCcncczdOhuLoaZqd8Z6n8BE0OVZDIrPoapxnRWs5aMtO0Q6QxJf61hA9J2QfRb\nGNG1CE1fygQmMd/sSafwG5QKgkF7VfVK10Fwxv8mrVqtoGLFGezdawV91sZ4e6tc3pIlqETkLcfp\n42PZCyUkqLDf2VldLCeJww0bVM34wIHW6aTMzFQjB5yclG1xcerrycCBqoPTN5v5NxrtwDWPR1qa\niT59tlCy5GT27g3FawcUrgND50DnMKgUDN7p/oxmHIc5gpGyDCKdIG37vSfKOAExjSCqLAkps1lo\nLGSSMRW3BCWXfBMCTXpBxeaw46CZ6dMP4+jozuDBu2zWkJMbnD2rlrZPnZr1xKZNSk4YMMCyg6hA\nRfrVqsEHHzx40PjdREaqJGPNmnd1BFmYmBj1weXsrKZ0mc2webP68/jxukrlL4h24Jqc4u9/nTff\nnEXr1iu5Gp7CtyOV3u15Et4IgU5Xzfxp7MSNCVwhDNIPKuedcde6dVMwxHWASGcykibjbWxiFGNZ\nkr6P6hcycb0I/ZeqJOXIWXDcL5J33plHnToL8PcvGF+lQ0OVgjJsWJa/un79ziCqnKyufxxMpjvL\nhUeNenR0bRhK73FwgN9+s55DPX5cbRaqVUutI7p8WVWptGhhWVkpnyPagWsehWEYzJ17HEdHVabn\nH2JQ5WP4tA/MjVCSyZy4NJYYy5nNXG6SVVUQLpA0U2ncaXvgRleIcMB8czgnzL6MxZ2l5nV8GXGT\nYoEw9AS8/hE0+RrOBmbw00/bcXJy57ffjuaL0kBLEhmp/PUPP9y1RHjBAm5vPLb0jO3QUFWTXqlS\nzuSKc+dU9N6ihfU0arNZDXQpXhw+/VR1dt4edm6dSZL5DdEOXPMwYmKSadVqBVWrzuLs2essXKdq\nu6evhG/CoVwQHEiNZzozWcM6TNwlb2RVlBBZHK5XhcRxXDGfYCaz8DBmMyn+Kk4B0P0SdBisOilX\ne6uJhWXLTqVt29WP3Etpz8TGwhtvqID3NhcuqCWZDRqoCYCWxDBgzRqV5Pzqq0cnOdPSlLRTpIha\nzGktkpPvTF/s00c9dnaG3butd818gljZgX8qIudExCwiNbQDz1/s2nWR4sUn0b//VmJvmOj4I7z+\nIWwLgLcuQOsrEJB5jbG4sxdfDLKi5MzrkLJSPTYFQcYZYonDk5W4MZ4VaSd5M8RM/YswxEstWejv\nBiEXE2nffg1lykzhzz9zoMkWAIKDVcB5j/ydmQmjR6uEnzXqtOPj1d42FxfVBv8omWTXLiXc9+59\n/zxwSxIZqZZaODpCkyZQqJDSyQsw1nbgFUWkvIjs1g48/5CenslPP22nSJEJeHsHcypAJRO/GAgb\nYsElENyi4ZxxnlGM5Szn7j2BkQ7mGACSSWYzWxjJGLwyd9HpWjrFA2GcP/xfe3i3A/idM+PhcQRH\nR3d+/HEbyckWWEVmRzxwM9vx48q7t2+vqjYszaFDULWq6jR6lGQRGwutWyvd59y5hx/7tPj7qw3P\nt7bXb95s3evlYaztwG+hHXg+4cKFON5+ey7Nmy8jKirpdjv8Ii8YEw1FAmF3EhzgEGNxV8nKbEgj\njV34MIqxrDXWMzbmJg4B0C8Mvh2vou45q+DEiUhq157He+/N5/TpyGzPpXkIKSmq7K5ECet0L969\ncWf06IcnOQ0D5syxfoLzFv7+amvGXycdFiC0A9cAKlG5ePFJHB3dmTz5IDcTDTr/DJWaw5FgaHMF\nal2AsAwz3mxlElOI4/6oL4MM9nOQMbjhaaxibWIM5YPhw8swfYfSubsMhIuX0+jb1xsnJ3dmz84f\n80vyNFu3Ku26Tx/l1C3NpUuqhPD11x+d5Dx/XiU4W7ZUZYEaq5ETB/78I17fLiIu2Tw/SEQ2Purk\ntxg2bNjtx66uruLq6prTt2qekqSkDPnuuz/lyJFrsmNHJ/lbIRd5u73IO2+KLF8m8lm0SL3nRJYU\nM2TLsxskWqKlu3wl/5J/3T6HWcziJydkt/hIESki9TM6yciIInLJJNL/GZG1I0VmXxdZ6obEhgVI\n/Tre8v77ZeTcuZ7i5PSvh1inyRGNGomcOiXyzTcib70lsmyZSLVqljt/6dIimzaJrF0r0q6dyIcf\niowbJ/Lyy/cfW7GiyKFDIoMGKRt+/13k/fctZ0sBxsfHR3x8fHL9ujoCz6McO3aN116bRpcuXiQm\nprNso5JM5q2BzTfVkuHZcWDGzFr+YA7zSONORs2EiSMcZTwTmc9CzmZeoW8EOATA2Aj4Zbqq6Z6w\nAM4HxNK06VJef30mu3dfst1N2zOGAYsXq0TfuHHWmSUSHw89e6rqk+XLHy6VeHur437++SFivuZJ\nkVyUUGpqB553MAyDGTMO4+TkjqfnGdLToecIeK0J+J2DsdFQNBB8k9XxxzjODDxuO+800tjPQcYz\nifks5IIRikcsFA6A/4XD7z6qyadtXwgONTFy5B4cHNxwc9tHRoYeUGR1QkPVWNi6dZX8YQ0OHlRJ\nzsaNVXnjg4iKUvJLrVp6F6aFsbYDbyUiYSKSKiKRIrJFO3DbExOTTIsWntSoMZugoBiuhMM7n8HH\n30JkAnx+FWpegLCsgCmddAYxmBBC8Oc869nISMawnBWEGpfZdFO10LtegvUXoHkPKN8Utu+HnTsv\nUqHCdFq08CQ01AoT7TQPJjMT3N1VNL5okXWSihkZ6hoODjB27IOjbMNQ8wEcHVVzjm6JtwjWduA5\nxdb/HwoMhw9fpXTpKfTr5016eia7DoFLXRg7B6IzoO4l+DQMku9q9DMwGM4ohjGS+SzEhz3EEcep\nVGgUChWDYU0M/DJVySXu8+DqtSQ6dVpHyZKT8fI6b7P71aD2T1aurErvrJVUvHhRbbmvXBn273/w\ncadOqUToZ5/plngLINqBFwzuntu9dq0/hqF0aec6sOMAXMlQUXT/CMiuICSTTDJQ0dXVDPjiqpJL\npsbA0j/VxMC2feHyNYM5c45RuPB4+vffSmKiFSbXaR6f1FTo108tS7hr6bFFMQw1x7xoUbUk+UG1\n6cnJqiGndOmHO3vNIxHtwO2fuLgUWrTwpHbteVy+HE9SMrTrBzU/gdCrcD4NSgXBhEcEZ/GZMCgK\nXgmAnyNhf4BarlC1Jew9CqdPR/Luu/OpXXseJ07YfhO8Jht27oSSJdVMkeRk61zjxg3loB+V5PTy\nUssrhg9XQ7U0j43kwIE/mwsOXGMlDh++KtWrz5bSpV+SPXu6iPnZl6T2ZyL//LuI7xKRqFdEXENF\nhjmJ9HfI/hwphsjEWJHXQkQiMkV8HEUSZou0/FKk5QciexZmyPoV2+T99xdLp05VZf/+blKtWnaV\npRqb8/77qtwwPl6kRg2R48ctf43//EfEw0Nk3ToRNzeRhg1FgoLuP+7jj0X8/ET27BFp0EDkyhXL\n26LJFWz9QWZ3GIaBh8cRnJzcb2vQ2/ZnbczJmuHvm6zKBDc8YJVkshkmxqjW+TZXwC9RvbdwHfh2\nJMTEGfzxx3lKlJhEp07rCvTgqXzJrW3wI0daLwK+eyfnkCHZz0oxm8HNTdli6VVydo5oCcX+SExM\np1271VStOougoBgMAyYtUslKnyPqGN9kNQZ2azY+NyETRl9XGnfrK3AyFbbuU12ZDb+E04EQEhJL\nkyZLqVhxBjt3XszdG9RYjrAwtdDhnXesO6I1LEwlUV99VXWNZseRI+r1bt0gUQcDOUG0A7cvAgKi\nqVRpBt26eZGaaiItHb78VW2ID81adn45Q8002fKXn5EoE/wapRz751fBPw3OBUPT7vBqY1i/E1JS\n7q3pTk/XNd35HrMZpkxRJX5z51q3xG/zZihTBtq1g2vX7n/95k3o0gXKl1dLlTUPRbQDtx9WrTp7\ne+kCwPVYNe2v9fdwM+nOca6X1DRBUD+rR1PUbO+Xz0OPcAhOh6gY+Ga4Gjo1ZbGaYbR5cxBly06l\ndeuVuqbbHjl79s6Shqgo610nORkGDVIfGNOmZd8t6umpJJXx4y2/vMKOEO3A8z8ZGZn06bOF0qWn\ncOyYimrOBUO5RjBo8r3//o+kgJyD1QnwS5Sq4S4bBEOjINIEKakwbq6q5+4zFmJvqAmFrVqtoFy5\nqXh76046uyY9XU34K1IENm607rX8/VW3aM2acPTo/a9fuqSknaZN9Wb6ByDagedvrl5NoE6dBTRr\ntozYWDWFbus+FTn/7nX/8d6JyoE3u6zkkv3JKgo3m2HxelXP3aoXBF6C1FQTw4btxsHBjdGj99r1\nMmHNX9i7V9Vpd+9uXT361uwWFxdV2njjL9/sMjLUB0qxYnoDTzaIduD5lx07LuDiMoFRo/bcHsc6\nf42qNPHNoXxoGLBlL9RoA7Xbw34/VcGydq0/ZcpMoXXrlVy5ojvmCiQJCfDFFyqxaO2Z27Gxqvnn\nQbXj3t7KyQ8ZYp0BXfkU0Q48/2EYBu7u+3BxmcCOHReynoMxs6FsIwjIYVHI8XOqEadCM7WL0jDg\n/PloGjVaQuXKHrq6RKNYs0btoBwyxPoTBQ8eVFt9PvgAAgLufS08XD1frx5cvWpdO/IJoh14/iI+\nPpWWLVdQq9ac24lEw4AfJ8AbH0F4DqRC/xBo0xuK1AMPT1WqGxeXwvff/4mjozuTJh3QEwM19xIe\nrvZQ1qoFgYHWvZbJBJMmqdrxgQMh6a4MfGbmnaXGmzZZ1458gGgHnn8ICoqhUqUZ9OixkbQ0pUcb\nBgwYrySQmEcUhoReVSWFju+C2zxITgGTycysWUdxdh7P//63kehoK7VXa/I/hgEzZijH6uFh/YmC\n4eHw+edqXdzq1fdez9dXPd+v38PXvNk5oh14/uCPP87j6OjOb7/dydYbBvQdB9VbP9x5XwlXJYGv\n1IZfpkBclqS9Y8cFKlf2wNV1EX5+4Va+A43dcP48vPWWqg6JyIWZNz4+asrhf/+rrn2LmBj46CP1\nreBh88jtGNEOPG9jGAajR++lePFJHDlyr+43aLJy3nEPyDEGXISug+Dl/1NRenTWcLjg4Fg+/tiT\nMmWmsG6dP4aezax5XDIyYPBgJWWsW5c715s8WdWO//TTncoYw7jThLR6tfXtyGOIduB5l8TEdNq0\nWUmtWnO4ejXhntemLVGt7ddj732PYcCuQ2qpgtN7MHymquUGSEhI48cft+Hg4MaYMbosUGMBDhyA\ncuWga1fVRWltwsOhY0cln6xadUdWOXpUlT3261egVreJduB5k7CwBKpV+40uXbxu6923SExSOnbg\npTvPpaTCwnVKC6/QDOauVs+BmgW+cOEJihSZQNeuXly7lgs/aJqCQ2IifPWVapF/1MZ6S7FnD1Sp\noqpSbskqsbFK1qlbVzn6AoBoB573OHgwjKJFJ+Lmti9beWPfcZBKcPI8LPpD6duO76qZJZt87u28\n9PG5RPXqv1G79rz7JBiNxqKsX69qtQcOzJ3Eosl0Rz4ZMEDVrZvNar540aLKyds5oh143sLT8wxO\nTu5s3PjgUq2EROj4o1pA3L6/2qwTdOneY0JCYvn001WULDmZFSvOaJ1bkztERqrEYvXqqlU+N4iI\nUAOwXFzgt9+UY/f2Vvr8+PF2vX9TtAPPG9xKVpYqNZlTpyKf+DxxcSn06bMFBwc3Ro7cQ3JywdED\nNXkEw1CO1NERZs7MPQfq5weurvDGG8qBh4aqCpXWrVV0bodIDhz4M7nkwHPhMnmTjAyzdO++Uc6c\nuS4bN34mRYv+v8c+R1papnh4HBU3t/3SsmUFGTnyfSlc+F9WsFajySGBgSIdO4o4O4vMn69+51Ck\nEAAACgNJREFUtzYgsmGDyIABIuXKiYweLTJnjoivr8jmzSKlS1vfhlzkmWeeEXmEj9Yr1axIUlKG\nNG++XBIS0mXv3i6P7bwNA/H0PCMVK86QPXsuy65dnWX27I+089bYngoVRA4cEKlWTf3atMn613zm\nGbWq7exZkSZN1K9nnhFp2VLk3XdFjhyxvg15DB2BW4lr127KRx95yltvFZVZs5rLc8/l/LMSEG/v\nEBk0aJe88MKz4u7eUFxdS1vPWI3mafD1FencWTnUiRNFXnwxd64bFycyapTIokUiL78scvGiyLFj\nIjVr5s71rYyOwG1EcHCsvPfeAvn009dl9uwPH8t5HzwYJu+/v1j69t0qgwfXk8OHv9LOW5O3qVtX\n5ORJkaQk6y1Tzo5XXhGZNEnE31+kdWv1nIdH7lw7j6AjcAtz4kSEfPihpwwf7ipffVUjx+87eTJS\nhgzZLSdPRsrQofXliy+qyfPP689XTT5jxQqR778X6dtX5McfRZ57LveubTar33PzmlYkJxG4duAW\n5NixcGnefLnMnNlMPvnk9Ry9JzAwRoYM8ZG9ey/LTz+9Jz16vCX/+MfzVrZUo7EiYWFKUjGbRZYs\nESlVytYW5Uu0hJKL+PlFSPPmy2XOnA9z5LzPn4+WDh3WSp06C6V6dRcJCeklffrU1s5bk/8pUUJk\n506Rjz4SqVVLZNkyW1tkt+gI3AJcvHhD6tZdKNOmNZE2bR7uvP39o2XcuH3i7R0i/fq9Iz171pJ/\n//vvuWSpRpPLnDgh8vnnqlLFw0PkP/+xtUX5BmtH4ONF5LyInBKRdSLy0lOcK98SHZ0sDRsukV9/\nrftQ5+3nFyGtW6+UBg1+lwoVHCQ4uJf8/HMd7bw19k316qoyxMFB5M03RfbssbVFdsXTROANRWSn\niBgiMi7ruZ+zOc5uI3CTySwNGy6Rd98tIWPGfHDf64aBbNoUJJMnH5KQkDjp3/8d6d69prz44gs2\nsFajsTFbtoh8+aXSx0eMEPnb32xtUZ4mN5OYrUSkjYh0zOY1u3Xgw4f7yJEj4bJhQ/t7SgXDwxNl\n3jw/mTfPT5ydC0m/frXlk09elxdesI/suEbzxERHi3z1lUp0LlsmUqmSrS3Ks+RmErObiPxpoXPl\nG3btCpV+/WrLc889KykpJvH0PCMffrhc3njDQyIiEmXDhs/k6NGv5bPPqmjnrdGIiDg5iXh5ifTo\nIVKvntLF7TTAyw0eFYFvFxGXbJ4fJCIbsx7/IiI1REXg2WG3EXiJEpMlI8Mszz77jMTHp4mra2np\n2LGKfPxxRSlUSH891GgeSlCQSnAWLiyyYEHuzFPJR+QkAn9UzVrDR7zeRUSaicj9AvBdDBs27PZj\nV1dXcXV1fcRp8wf79nWVmzfTxcHhRXFw+Kf8/e+6BFCjyTHly6t5KsOHqyqVOXNU6WEBxcfHR3x8\nfB7rPU+jgTcRkYkiUl9EYh5ynN1G4BqNxkLs2yfSqZNI48Zqnsq/9MA2a2vg00WkkCiZ5YSIFKwh\nBBqNxnLUqaPmqaSkqGFUuTVPJZ+jG3k0Gk3eYuVKkV69bDNPJQ+hZ6FoNJr8SViYyBdfiGRmFth5\nKnoWikajyZ+UKCGyY4eep/IIdASu0WjyNidPinToUODmqegIXKPR5H+qVVNJzVdeUbNVDhywtUV5\nBh2BazSa/MPGjSJffy3Ss6fIoEEiz9tv74VOYmo0GvsjPFwlONPSRJYutdsEp5ZQNBqN/VG0qMjW\nrSItWqgE56pVtrbIZugIXKPR5F+OHVMJzjp1RKZNEylUyNYWWQwdgWs0GvvmrbdE/PxEnnlGpEYN\n5dALENqBazSa/E2hQiLz54uMGiXSrJmIu7uIYdjaqlxBSygajcZ+uHxZpGNHte1n8WKRYsVsbdET\noyUUjUZTsChVSmT3bhFXVzUUa/16W1tkVXQErtFo7JMDB9TCiKZNRSZMEHnxRVtb9FjoCFyj0RRc\n3n1XteHfuKHKDc+ds7VFFkc7cI1GY7+89JLI8uUiP/ygZJWFC+1qB6eWUDQaTcHg3DmRtm1VueGs\nWXm+ZlxLKBqNRnOLN94QOXpUVajUrCly+rStLXpqtAPXaDQFhxdfVDXjgweLfPCByOzZ+VpS0RKK\nRqMpmAQGKkmlUiWROXNE/v1vW1t0D1pC0Wg0mgdRoYLIoUNqQUSNGqolP5+hHbhGoym4/POfIr/9\nJjJ6tEjjxiLTp+crSUU7cI1Go2nXTuTgQZG4OFtb8lhoDVyj0WjyIFoD12g0GjtGO3CNRqPJp2gH\nrtFoNPkU7cA1Go0mn6IduEaj0eRTnsaBjxSRUyJyUkR2ikgJi1ik0Wg0mhzxNA7cXUTeFJFqIuIl\nIkMtYlE+w8fHx9YmWBV7vj97vjcRfX8Fgadx4Il3PS4kIjFPaUu+xN7/Ednz/dnzvYno+ysIPP+U\n7x8tIp1EJEVEaj+9ORqNRqPJKY+KwLeLyJlsfn2U9fovIlJSRBaJyGTrmKjRaDSa7LBUK31JEflT\nRCpn81qIiJSz0HU0Go2moHBBRF592AFPI6G8JiLBWY8/FpETDzjuoQZoNBqNJvdZI0pOOSkia0Wk\nsG3N0Wg0Go1Go9FoNLex56af8SJyXtT9rRORl2xrjsX5VETOiYhZRGrY2BZL0kREAkTJgD/Z2BZL\ns0BEokR9Q7ZHSojIblH/Ls+KyPe2Ncei/ENEDovylf4iMta25ij+312Pe4nIPFsZYgUayp1qnnFZ\nv+yJiiJSXtQPjL048OdEJddLi8gLon5YKtnSIAtTV0Sqi/06cBdRDYQiqgclUOzr7+/FrN+fF5FD\nIlLnQQfm1iwUe2762S4iRtbjwyJS3Ia2WIMAEQmytREW5m1RDjxUREwiskJUIt5e8BWRG7Y2wopE\nivrQFRFJEvUNuKjtzLE4KVm//01UsPHANUG5OcxqtIhcEZEvxP6i1Ft0E1VOqcnbFBORsLv+fDXr\nOU3+o7SobxuHbWyHJXlW1AdUlKhvvv4PO9BS2HPTz6PuTUTdX4aILM91656enNyfPaF3/NkHhURV\nw/UWFYnbC4Yoiai4iNQTEdcHHfi0rfR30zCHxy2X/BelPureuohIMxH5wPqmWIWc/t3ZC9fk3kR6\nCVFRuCb/8IKo8uWloobp2SMJIrJZRN4SER9bGvLaXY97icgSWxliBZqIyoY72toQK7NbRGra2ggL\n8byoLrfSonRGe0tiiqh7s9ck5jMisljy3zf5nOAoIv/JevxPEdkreSAwtOemn2ARuSyqE/WEiHjY\n1hyL00qUXpwqKnm0xbbmWIymoqoXQkRkoI1tsTSeIhIuIumi/u662tYci1NHlMxwUu783DWxqUWW\no4qI+Im6t9MiMsC25mg0Go1Go9FoNBqNRqPRaDQajUaj0Wg0Go1Go9FoNBqNRqPRaDQajUaj0Wg0\nGvn/KaVZux6WNgAAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Simple numeric differentiation\n", "def grad (f, x):\n", " eps = 1e-4\n", " gradx = np.zeros_like(x)\n", " for i in range(x.size):\n", " dx = np.zeros_like(x)\n", " dx[i] = 1\n", " gradx[i] = (f(x + dx*eps/2) - f(x - dx*eps/2))/eps\n", " return gradx\n", "\n", "def dE2d_dx(x):\n", " return grad(lambda x: - p2d.logpdf(x), x)\n", "\n", "x0 = np.array([1.5, 0]); p0 = np.array([0.1, -0.1])\n", "traj_euler = [(x0,p0)]; traj_leap_frog = [(x0,p0)]\n", "dt = 0.01\n", "for i in range(250):\n", " traj_euler.append(integrate_H(traj_euler[-1], dE2d_dx, dt, dt, 'euler'))\n", " traj_leap_frog.append(integrate_H(traj_leap_frog[-1], dE2d_dx, dt, dt, 'leap_frog'))\n", "traj_euler = np.array(traj_euler).squeeze(); traj_leap_frog = np.array(traj_leap_frog).squeeze()\n", "\n", "plt.contour(X, Y, p2d.pdf(XY))\n", "plt.plot(traj_euler[:,0,0], traj_euler[:,0,1], 'r-')\n", "plt.plot(traj_leap_frog[:,0,0], traj_leap_frog[:,0,1], 'b-');\n", "plt.axis([-3,3,-3,3]);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In practice, also the leap frog integration is not perfect and energy is not precisely conserved. Thus, the actual algorithm includes a Metropolis step which discards samples with lower energy to ensure that it samples from the correct distribution." ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "collapsed": false }, "outputs": [], "source": [ "class HMC (Sampling):\n", " def __init__(self, x, E, dE_dx=None, Tau=42, dtau=0.04):\n", " self.x = x\n", " self.E = E\n", " if dE_dx is None:\n", " dE_dx = lambda x: grad(E, x)\n", " self.dE_dx = dE_dx\n", " self.Tau = Tau\n", " self.dtau = dtau\n", " \n", " def _H (self, x, p):\n", " return self.E(x) + 0.5*np.dot(p.T, p)\n", " \n", " def sample (self):\n", " # Gibbs step for momentum\n", " x = self.x\n", " p = np.random.normal(size=x.shape)\n", " H = self._H(x, p)\n", " \n", " # Simulate dynamics\n", " xnew, pnew = integrate_H((x,p), self.dE_dx, self.Tau*self.dtau, self.dtau, 'leap_frog')\n", " Hnew = self._H(xnew, pnew)\n", " \n", " # Metropolis step\n", " if np.log(np.random.uniform()) < H - Hnew: # Remember: H = - logp\n", " self.x = xnew\n", " return self.x" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD7CAYAAAChScXIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XVUVdkXwPEvISqgghjYPbbY7YhjYefY7diiY43dY7eO\n3R1jB4qOgmJiYXej2ALS8e7vj+OIjPoT5T1yf9ZizXv33XfPuctxe9h3n3NACCGEEEIIIYQQQggh\nhBBCCCGEEEIIIYQQ8YCRoRuws7PTLl26ZOhmhBAioTkK2P/3oLGhW7106RKapiXYn9GjR8d6H+Qe\n5f7k/mK/H/r+ASp/KaYaPGgLIYTQHwnaQggRj0jQjiZ7e/vY7oLBJfR7lPuL3xL6/f2XwR9EAtqH\n/IwQQogoMjIygi/EaBlpCyGEIQQFgaen3i8rQVsIIfTt+HGws4OFC/V+aVO9X1EIIRKr9+9h6FBY\nuxbmzoV27fTehARtIYTQB2dn6NoVMmeGixchZ06DNCPpESGEiI43b6B9e6hbVwXto0c/D9hPnuit\nOQnaQgjxIzQNtm6FAgXg9Gk4eRKGDwfTTxIY/v7QqxeULw/BwXppVoK2EEJ8Ly8vaNIEfv0VGjWC\nCxegVKnI55w4oR5G3runPk+aVC9NS9AWQoio0jRYuVKNro8fh927YdEisLCIOCcoCP74A6pUgc6d\nwckJ0qbVWxfkQaQQQkTFgwcqZ/3PP1CnDixfDunTRz7nwgVVMeLtDYcPQ6VKeu+GjLSFEOL/CQ9X\n5XuFCqmUx8KFsGdP5IAdGgpjx0KZMpA1K3h4GCRgg4y0hRDi627cUCmOU6egRAlYvx7y5o18zvXr\nanTt4QETJsCgQWBsuPGwjLSFEOK/QkNh9OiIypBhw1R1yKcBOzwcpk+H4sXhxQtV6jd4sEEDNshI\nWwghIjt/HkqWjHh/9OjnqY5796BDB/UwsnZtWL0a0qSJke7JSFsIIQACA6F//8gB29s7csDWNFUt\nYmenUiZTp6r8dgwFbJCgLYQQcOwYmJvDrFnq/bhxKkCnShVxjqcnODhAjx6QOrX6joHz118i6REh\nROLl6wsDB8LSpRHH7t+HHDki3msarFsHjo7g46PK/VavBhubmO8vErSFEInV/v0qH/2pmzcjB+yX\nL6F7d9ixQ01PnzZNpVBieHT9KQnaQojE5c0b6N0bNm2KOGZrC3fugKVlxLHt26FbN3j9GrJkgc2b\noVy5mO/vf0hOWwiROGgabNmiHhp+GrDHjIFnzyIC9rt30LYttGihAna9eqoGOw4EbJCRthAiMXj2\nTE1B37cv8nFnZ6hRI/L7334DIyO1wNPkydCvn3ofR0jQFkIkXJoGK1aoQPypTJlU9ce/6177+akH\nkrt2qcWdfH3h0CEoWzbm+/wNkh4RQiRM9++rlfb+G7BbtYLbtyMC9rFjqu761i1V4pcjh1r4KQ4G\nbJCgLYRIaMLDYfZsyJVLzWb81MyZqnzP3FxNphkwQOWuK1dWa4h06wY7d6o67DgqJhI1mqZpMdCM\nECLRu3ZNTS8/dy7ycWtr2LZNjbxBfd6uHeTJAylSqNX7Nm1Sq/TFEUYqj/5ZjJaRthAi/gsJgfHj\n1fKp/w3YJUqo6o8qVdR5o0erCTJt26odaPz8VDokDgXs/ye6QTsL4AJcA64CfaLdIyGE+B5nz0Kx\nYjBqVOTj5uZq1O3mpta4vnpV5anPnVOBe9YsaN1aTZyxto6Vrv+I6AbtUKAfUBAoC/QC8ke3U0II\n8U0BAWpbr9KlVT76U2nTqtmLK1aAmZla2Onfh5K5cqklVfftg75941Q5X1REt+Tv+YcfAD/gBpDx\nw3+FEMIwjh6Fjh3VFmCfcnCAy5fV7MWKFeHuXWjfXgXuzZtVkM+aVaVDrKxip+/RpM+cdnagGHBG\nj9cUQogIvr5o3buDvf3nAbt+fbWUqrs7lC8P8+erdEizZtCrl6oSaddOPZCMpwEb9De5xhLYCvRF\njbgjGTNmzMfX9vb22Nvb66lZIURi8XTJRlIMdCTl+zeRjk+mAoPzv8XI1lZNU3/xAmrWhPfv4cgR\ntYLfnDkqHVKqVCz1/ttcXV1xdXX95nn6SOYkAfYC+4HZX/hcSv6EED8kODgM5/VnMB8+iGrPT332\n+Z0xc7H5czipF8xQ+erVq9Ua1/36QdOmaiJNtmxq5/R4Nrr+WslfdEfaRsBy4DpfDthCCPHdXr70\nZ+ECd57NXs7kgN1Yh76PfEKfPoSnSEmqSWPY2XEynerVg4YN4eFD+OcfNeOxYkVVUdKrV7x72Pj/\nRDdoVwDaAJeBix+ODQUORPO6QohE6PFjH6ZMOY7LuhNssD5MkQAPjDVd5JOuXkUbMoSH5x8wqsI4\n1lRLA0WLql3T166FoUPhwAG1XnaJErFzIwYU3aB9HJmgI4SIpjt33jBhght7dt9icannzDNZg7F3\nOISGRJxUqhSsXAmNGnHepjADUrbhkPUxTEZ6qIWebGzUA8rcuVV1yKdbhSUgEnCFELHm7t23dO68\ni/LlV1DSOoDnhfbT9OxqjHXhamuvf505o8r17O1x+7kN8++n4YjPPMwy2cLFiyotUr68ymtv3pxg\nAzbI2iNCiFjw6JE3Y8ceZc+e2/TsVozByc5jPnOK2ojg2TO16NO/QkJg5EiYMgWA02Y5KJ4mDLN1\nq9XGBP37w8GDqnKkePFYuiP9M9SDSCGEiLJnz94zcaIbGzdepWfPktzb9Qsp+/VUgTpZMnjyJOLk\n9evVZJlcuSIdz1uvPGYrFqjSvnLl4KefVDokZcpYuKOYJ+kRIYTBvXkTwB9/HKJQoQUkS2bKjUtd\nGG/qRsp6NcDfHzw91eJN//L3V6V6NjYfA/ZbYwueLt6A9dZ14OQEFSqopVQ3bUo0ARtkpC2EMCB/\n/xDmzXNnxoxTNG6cjytXepDp6U2o9TPodKp2+tq1iC906wYLF0Ljxmpd6w/2pixBiRNbyZQrvdod\n/fBhtTVYsWKxcFexS4K2EELvwsN1LFt2gbFjj1KhQlbc3DqSL6u5qptes0atY33yZOQvPX2qNtU1\njkgA+CRNycycbejjNg2bN54qHZIvH5w/n6hG15+S9IgQQm80TWP37lsULbqYDRuusnt3S/7++1fy\neV2BIkXUZgNp034esP38VEAvVEhdJ2lSTtrYMaDaNIZdnIPNwV0qHdKjB2zcmGgDNshIWwihJx4e\nz+nb9wBv3wYyaVJV6tTJg5Gvr0p57NsHhQurSS+f2rFDVYpYWn48FG5lzbCktdG1acviMRUx6dML\nXFzURrtFi8bwXcU9MtIWQkTL48c+tG27AweHdbRqVQgPj27UrfsTRnv3qpHzw4dqdP3fgO3hAYsW\nqTVCPvAuXp4ySXqSc+zvTOuSHZMK5dXCT+fPS8D+QEbaQogfEhAQyrRpJ5g7152ePUty544jKVIk\nhVev0Pr2JejYGe6mLUfhg39H/qK9vUp1FCsGn8zhcG83hHpOVqxZ14Sar89Axd9h4kQ1YSYBrR0S\nXTK5RgjxXTRNY/36K4wY7IzHmwkYtWtLqiXzQNMIXr2R8N/7c8KiEun8H2Pn4x7puw9bDSL7iS3w\n6FHE9QoUYHqFQSz8x4e9fzekwKI/4dgxNVnGzi6mby/OkMk1QohoO3v2KX37HqDg21s8fDZNHWzf\ngleXPXnbugfa/Qc8zlqHGjdXfPbdi1blKLZhGmTM+PFY8OjxtL6Ui1e3gjm/4WesO9ZTue9z59Qu\n6eIzktMWQnzTs2fvad9+Jw3qb2Sz3yqW3poGqVNz93Yw6yZdxahEMV6ZpiNrvlRfDNghRknIZ/pY\nrbr37BkAj3ccZsYiD6aeGM3hDkZY16sBffrAunUSsP8PSY8IIb4qKCiMWbNOMWPGKQY3y8ighW0/\nfvZ7pzs03dqF7NYB2NQsQ/Il8754DR1G3KrRjfyHFqscdvPmXM5XGfM/R2OTxhzrTDZqBuSWLaos\nUABfT49I0BZCfEbTNP7++zqDB/9D0aK2rMhyHut5Kh1yK3NF9mJPj3cLMe3QFrPL58HN7YvXuW5p\nh1dzR6ou/01dd/ZsHv61jgx3PfCuXg/bx9fVkqsLF0Yq+xMStIUQUXTpkqq39vEJZu6EClSqU/jj\nZ+vTdqRK8iuky54C0/JlYPLkL15DMzZmYs4p1Mn7nKL7ZgAQ2rINJpvWc8UiJzk6NCTlptVq5b6O\nHaU65AskaAsh/q9Xr/wZPvwIu3bdYvToynRPfQ/jli0/fn4rT3Xy3jmk3lSq9NXR9fufa9E4YBKH\nzn2oq06ShPCkyQj2D2Zf8RY0zq9hcu6sSocULvzFa4ivB215EClEIhcWpmPBgrMULLgAc/Mk3LzW\nne7Tmn8M2P7JrAEiAjZ8NWC/qtmCnq97RgRsgNBQ3IJtOd53Br8GncfECDh7VgL2D5KRthCJ2LFj\nj+jVy4m0ac2ZPduBQkFPMC5T6vsvZGVFaHA4SQIjb8AbkNSSUZb16NmrFDkXTISpU6FDB0mHRIGk\nR4QQHz196svQoYdxcXnIrFk1adI4Hy9qt8PWef13X0tLmxajV68+O37UpjhrCrVhfsYLJPM4D3//\nDQUL6qP7iYKkR4QQhIXpmDnzFHZ2i8icOSXXr/ekaGYrjExMfihgA58F7BBrG7pat+NGS0eWvVpG\nMjMTlQ6RgK0XMtIWIpFwdX1Inz77SZ/ekvnza2OTzga3ltNpeGCQ3tq4Wdge13s6ugccA1NTWLYM\n2rfX2/UTE0mPCJFIvXzpz8CBB3F1VamQevXys3hVII7dLPTWhmZljXvyXBR6cRULXZAK2JcuQYEC\nemsjsZH0iBCJTHi4jvnz3SlYcAHp0llw7VovktoUwLGCk14DNoCR9zvKeJ1TAbtjR/D2loBtILJg\nlBAJ0Pnzz+jefR/m5klwdW1PEot0NOuvY/qW4iz2vWjYxps2BTMzw7aRiEl6RIgExMcniOHDj7B1\n63UmT65Gg0Z2jF9khPvGqxy/pN+66BDjJCxOXZ12JUxJ5XkP5s+Hy5dhwwa4dw9+/RVat1b7OkqJ\n33eT9IgQCZimaWzefJUCBRYQEhLO1au9IFVRCtQzovm61tEO2D6mVpHer0ldhZnVR9HL+i6pMqcF\nd3eoXBkcHeHUKVi6FNavV8cePIj44tWrkdbSFt9P0iNCxHP37r2lZ08nvLzes2VLU0wts1K7F1gH\nvcLLLV20rx9gnJxUYd4APC5mT9OHpZlex4y2++dgNGsWtGmjTgwJge3bYcECNdLu31/tOmNtDatX\nw+LFcOuWWno1W7Zo9yux0sdIewXwAriih2sJIaIoJCSciRPdKFNmGdWq5cD5n66sOZiVho4wz3IW\nzrujF7Bv5/wFAHNdIAATak+msX9dDpa/y88Xd2Lk5qYCtqcnjBypAvGSJdC3r9oXsmlTtSBUxoww\naBA0bKiO16oVzTtP3PQRtFcCDnq4jhAiilxdH1K06CJOnnzCmTNdscxSAbvGJlgYBeHlZkSZFf1/\n+NpakiQA/HT/CABPB40jf955BBib4a5bjFWG1HD6tArWjRurLcF8fODIEXBygqAgqFpVTabZsgVG\njVIpkj/+kM0N9EAf6RE3ILseriOE+IaXL/0ZNOgQLi4PmDu3Fhly5KXFUCOSmcGuDgco1yX6o1ij\n0NCPr5fPPsqQ8afZ2+AlZXYvhLFjITRU7UBjZga9esGaNfD0qRplr14Nb95Apkwwd65KjyRPHu0+\nCf3LztfTI5oQInrCwsK1BQvctbRpp2oDBjhrjzyDtS6jNC3Dz5q2YGe45mdlrWlqXxi9/Pj9OUVr\n3HizVq7QLM27TmN1/OefNc3KStNatNA0NzdNCwrStI0bNc3ePuK7WbNq2sKF6jMRLcAXy+7kQaQQ\ncdyFC16M7rSWvKFeuG7sxOl3hSjdwojxWbYxzncKtg3P6rW9S8v20GD8XfqU9aL/1X5wFbCxgerV\nYfNmtTXYkiXQpAm8fg06HeTMCcOGQdu2UqNtYDEStMeMGfPxtb29Pfb29jHRrBDxmq9vMCNGHGHL\nlmtcSL2NjDfc8a6zlsYadAp5o/f2wn+2Z1aZvmwacorbwfMx2+yjPtixA2rWhH37VFD28ABbWxWs\nc+eG4cOhVSs1dV38MFdXV1xdXb95nr4q3rMDe4AvFYN+GOkLIaJC0zS2b7/B7787U7NmLiZWDCNd\nxybsT92Am7NW03ZoYdI8e6K39kItUhHUvCUT3JPRwvMARb1vqg9271YbFSxdCitWQJ48kDkzXLsG\nYWEwYgQ0awYmJnrri4hgyAWjNgKVARvgJTAKVVHyLwnaQkTRw4fe9OrlxIMH71iwoA7a9TdU6VUS\ngL+cLtK7djG9tXUwawuqP/2bd9ny4f3oBTnDX6sPWraE+vXVQ8WzZ9WsxnTpYNs2CA9X5X2NG4Ox\nzM0zJFnlT4g4LDQ0nDlzzjB58nEGDChH4xbl2ffbBvofaQfAkSp1+cVlr0HavmWWAZuq5Umzfxv8\n9JPKWWfLBl26qOnn06erPPWoUVCvngTrGCJBW4g46swZT7p02UOGDCmYNbs2O4+kJPWEQXT3mm3w\ntq/bFqLA86sRB3r1UmV6167Bn39CqlQqWNeqJeuHxLCvBW15ciBELPl0caeZM2tim7MQv/V5zcIb\nNbDzcjFYu6/TZadmWAtWVX1P4dO7oXRp6NZNVYPs2KHy1OnTqzrratUkWMcxErSFiAU7d96kd28n\natXKzdETvfhzSXLezr7AybMlDN52mpcPOc9kCG8Mu3apmYtr1kCxYpA9uyrnq1xZgnUcJekRIWKQ\np6cvjo77uX79FYsW1eXOq+yMnhPGQe8GFL7uFHMdyZULnj9X+et/5cihHkJmyRL5x8pKAngskKVZ\nhYhFOp3GwoVnKVZsMUWKpGPj9u6MW5gO/ylz8DqZxCAB+2yOeh9fOyUvwsUlu9W8xYAA6NNHBWN7\ne1UlcuiQqgpJmhTOn4d586BFCxW0U6SA4sUhOFjvfRTfT9IjQhjYtWsv6dJlDwD7nduzzykJzrUm\n88/TMZjowg3WbqkHe3hnYkEfhznMWdOK1El1MGOGqgYpW1alRkp8JR3j6amWWF22TOW8+/VTAV3E\nOgnaQhhI8MMnrJroxIgdvowbZ0/+9LZ4NJ5C/xcreZszGya6cB7VKE+2gyf12m5Ipmw4txnFjMVX\naDWlDWta5MFowRyYNUvlqp2doUiRz7+oaWoDgzlz1Mi7bVs4cUJNqhFxhqRHhDCAU/uv8CRvaSyP\nHeb4rGIUnTOdos3t+KmcxmzHoaR+8ZiQFMn0GrDd7dridf0pLUpPYeSBEBY69aTrCyeMcueGK1fA\nxUUtlfrfgB0cDGvXQqlS0L49VKig1r2eM0cCdhwkI20h9MjXN5hRgw/QfEV/coc8J6X2hHRtqwPg\nuPUkI3s14ecXXnpvd49zCCnMnlKm5mba1c3ClnyemNYrB3XqgJsb5M37+ZeeP4dFi9SOMoULq2VX\na9WSyTNxnARtIfRk797b9Oq+hxsvR2EeGgCA9uQJ94pV4FjeQsxrWl7vbbZreIMxc/NwY/xaiq+Z\nyvW0wVhu9oZGjeDMGVUl8l/nzqlR9N696mHj4cNQoIDe+yYMQ4K2ENH08qU/A/o6YXNkL49ervl4\nfMnQvTRb2A5P62R03LRYr20OzzkHi4G9GZHLg6eFi/GHz4fl7Cu3VjMZs2eP/IXQULV/45w58OwZ\n9O6tJs9YW+u1X8LwpE5biB+kaRqbVp7Fo+9k/jA5hY3PcwC22/UlXY63FDqyBytfb722eSN7ZX7L\nuo65wzRS/9GGHJePqb40aIDRX3+pVfg+9fq1miyzYIFaRrVPH7UYlCyjGufJ2iNC6NGTK49waTqI\n2vedoWw5LhsVJv+ZDaxedIju/Spi5fNO7232L7aR0PL2TLrTFcuDqoQwoGQ5zHdvgwwZIp98+bIa\nVW/fDg4OkDUrDBwIadPqvV/CMGRyjRB6oAsL52L1tlgX+YniSd9yZclxugWOpOT5VUzasof8+ybp\nPWCfr9iW8sVu8HvyrcybnwHLg3t4aZOVkMdPMT97MiJgh4ertUPs7aF2bVXCV78+HDwIV6+qNbBF\nvCcjbSGi6PbtN/TqtI1ed9ZTPcVrTB89QKeD5LpAXMtXI2fYS7K6X9Zbe7426ZhTaDblH+yk6uMt\nAPgYJ+fGtqOUbVgq4sR372D5cvjrL7XQU9684OUF7u7QoYPKX0vpXrwj6REhflBYmI5Zs04xZcoJ\nRoz4mXS5S9N/ijFtqj9k+ugcBmnzYs0OvHR/Tc13EWtoD2yymJHL25EqVTJ14MYN9TBx0yYoVw4s\nLVVliIkJODqqgJ0ypUH6JwxPgrYQP+DKlRd06rSblCmTMm5SfaastuLuU6g21I++zYuT69Ed/NJb\nYvnCTy/tvcqQhfCgJNi+u//xWGXrfvRe3Jlffy2o9mXcv1/lqy9fVhNiwsLg6FGoVEk9aJRa6wRB\ngrYQ3yEkJJzJk4/z11/u/DmhKlrKYoyYa0SjbuBeOgyP/EkAeFc2E9annxqkDz3y9udhjuIsX16f\njJYarFqlFnJKmhQyZoRXr+D2bTWL0dER8uc3SD9E7JCgLUQUnTv3jE6ddpE1ayqGjq7HiPkpeB8K\nBUfCRW9PLpfLYtD2D7YfR6s9ZowdV4WeNaxVKd/atSpXnTQpXL8OFhYqUHfqpFbrEwmOBG0hviEo\nKIyxY11ZscKD6dNr8Cq8MBOXGNFsADgX1hh2aBqdew+OdjvhpsaYhOk+O+7VfzxdrmTmlXcoW7tb\nk2XHarWAU44cqhLEw0Mt+NSnD9StK7ugJ3AStIX4P06efELnzrspWDAtjgPrMni2OaYpIPMgCLx+\nnF2NKumlHc3ICKP//H24UqsPtxo3Z9AwV+aVfk2d+wcw8vdXI2hvb3j5Uq245+io1ggRiYLsESnE\nFwQEhDJixBE2brzKrNm1uPeuAE0HQLNhGp5+p9iUt4Je2/s0YLtkaUKaxX+yZKErRQeP5k74eUxv\nplazFV+/VqPr3r2hc2ewsdFrP0T8JSNtkWgdO/aIzp13U6pURro41qb/tOSktw2kndl06i+fhqXf\ne4O0ey5FWa70mE1eq7t4j5tCFd09kqazwdgIePIEfv5ZpUAaNJDp5omYpEdE4qZpqkyuTRv8k6dk\n6NDDbNt2gzlz63DpaV72rnnElNwLqbF5isG68DxVDmbm/JOu9fxJOn8Sqd56kSyVBWbhIRASAq1a\nqRRIsWIG64OIPyRoi8TNyQkaNOBVuWqU9XSgQsWstO/mwMaBJ+j06i8KvXHjQa7c2J1313vT4Sam\nLMrQjwI5ocLlFYT4+GFsakJyXTBG6dNDr17QpYusCyIikaAtEq/QUHSFCrMiRyMqHF5NQLc+uCWv\nS635zbDMYMrGWo0YOH9c9NowBj4vCOFWtvI890tBJZ8j6HQaYTpIRhiUL69SII0bQ5Ik0WtbJEgS\ntEWidavPWF4t3cCyZpPp9ksWynVQ63Zs6DGcOhsXkMpb/yvyAbxJkgab0Ncf34camRDetBnJBvVT\nMxmF+D8kaItEx88vhDG/72Dwyt+4MmMDzuH1uLbmBHsvV4zRfrw0tuR5w7YU/mskRv9dQlWIr5Cg\nLRKVI0ce0LnzbhZauFAoR2pqhy9ipmdXql1ZHWN9uGqejf156tB65zgyZpeSPfF9DFmn7QDMBkyA\nZYDhHr8L8Q1+fiEMGfIPu3bdYsXwIlTv8TvL3jpy2StpjLSvMzHlZiF7+j/My6/TOzKwc/F///IJ\noRfR/b/JBLgFVAOeAmeBlsCNT86RkbaIEa6uD+nceTeVKmWlW82clGtlF2Nt+yazwbtFB3pftsXf\nKh0rVtQnWzZZE0T8OEPtXFMauAs8BEKBTUCDaF5TiO8SvGgp5/JWYUzzhaxpn5Ih7ktjNGAD+KdL\nz/5Np+mb34dDy36WgC0MJrrpkUzAk0/eewJlonlNIaLMze0Rgya/4PQjV1xxhdEx1/bJbA0I37iS\nqf3WUMD3PuMzHcBs/Z9g46sm8ghhANEN2lHKe4wZM+bja3t7e+zt7aPZrEjsAgNDGT9oD1arl3A8\n5GiMtl222E0OPq3Fg5bNGVFnLmvLP6fCo50YlS4N61ZD2bIx2h+RMLi6uuLq6vrN86Kb0y4LjEE9\njAQYippi8OnDSMlpC726vMaZ4/1m0fOtc4y2297BHU/bUkypfQG75qXZkLEmrf1PY1q3NgweDIUK\nxWh/RMJmqJz2OSAPkB0wA5oDu6N5TSG+KDg4jJW/TqBIe4dIAfufZoPIfsmfYa+PG6TdUFMz0pYN\np1SbkrTPuZuSzUqQRAunbf1smF44pzYokIAtYkh00yNhQG/AGVVJspzIlSNC6MWFC15cq96ajm9d\nANic25EDJbtRLcNOLDxvMCDHWhxTdtd7u6cy1mRYVSf2O5zj3YB+lH51SX0wdy7Gjo56b0+Ib5HJ\nNSJOCw0NZ+JEN8KnzWCc/17O1hpIk7djaTDCnJ0/abQ3PcqfuaoYpO0byQtyddASSuwchuWV89yo\n3pKK9YtiMmwoPHwI1tYGaVcIkE0QRDx09epL2rfbwWCfvdRNe5umNR7jZZWFCrPhoKk/A7Lsovx8\nw81wzJjXCuupDVibrhrNFi2j8rIZsOoi7NkjAVvEGhlpizgnPFzHzJmnmDHFjaP5TmP14jmVrPdR\n/Q8bDthB9dQ3KX9zLu0qLtRru/MH76bz1F9JpgUD0MuyOcV+q0OnB9sw9vCAiROhRQswju6jICG+\nTdYeEfHCvXtvadduJ5Ym4Wwy2sG9+zr6lt7KTxMsOWwSzPjApbS3028ueX/bvpT6ezNpgp4DsCRX\nM7anKM2G/NdJfWg3DBqkllFNlkyv7Qrx/0jQFnGapmksWnSOkSNdGD+gGE3Wj+f4y8wcnriKo5WT\n0PTWDpqvG0/+zR56bfdS8brYXL1E5hA1R+yVkQVpNX/1YZ8+MHIkpEmj1zaFiArJaYs46+lTXzp2\n3IW3dxDOa2uTpn1rnC0q82BpD3I5j2XCxJVYPfAyTOPP3mOV2RLuq7dptQ+7oLu7Q548hmlTiGiQ\nkbaINZoX4Ro6AAAXBklEQVSmsWHDFfr1c6Z379JUzp2BUm2LEZbcnKc5spH+5RMCf0pBpuP39N72\n/WS5ud1nBg5TI5bK0VKkwMjJCSrG7HrbQnyJoSbXCPF179+rfLDu83243rwJoHnzrUyadJzd6xtS\n8cBOKrfOi7kuAKfqdbg27FeSh77Xe8AOMLFg2i/rSFk6b6SAzebNGPn4SMAWcZ4EbWE4o0bBmzef\nVVs4Od2heOH5VA+5wZFMxyhb4yd+OTWbY7WbUen6c/Jle07TVkNI/i5Qr90Zn30y/rkK0fdEF9Ic\n2weArl9/CAqCZs1A1r0W8YDktIVhXLwIGzbAtWsfD/n7BfNXm3lkOLKLOyY3eHElLzfempPGyIhf\nVx3GweoRbgVsDdIdv6RW9EuyDk9PX9IGf/jH4MIFjIsVM0h7QhiKBG2hf+Hh0L27qmtOkwZu3+bJ\n5AXo1q6nnXlStHa/0erhKsr5HqGT32jar9zHtva/GKQrOox4XqoWLzQT0l84Rj6dj/rgzh3Indsg\nbQphSJIeEfq3dCm8fQu+vuhKluJ98bI4bb7AvQkL2LfsCXYXRtM55WaaP5jI28wZWNuhtsG6cnfA\nZPyuXMLs7k0sO7UBW1u4fl0Ctoi3pHpE6NfDh5AjBwDe9X9l+PWMPM5dgkkzGzNivgUPn+qYk7I/\nldfEzCYB901seNR9CPbV82DUtSscPAh2MburjRA/QqpHRMwYOhSAv/svI8/JUtgNakvPIa1p9JuO\nImnv4XbExmAB+6Z5QbbVnk1AZvWPxrLC7Ul29xZVGthh1KUL7N0rAVvEezLSFvrj4kJY2/YcD8+I\n/fMzBGfJQdhrH5IG+WCqhRus2ZO5G+KYaw1/Z55G+vWzWWVSgkx/TaRB+7IYHTsGTZvCjh1Szifi\nFZnGLgwrOJj3OfOzxDsX/YKOYKzT0byiO+bFcmDROoS/ymYySLMVi15lUL5T1Dw0AuegzLhW78KI\nZW2xsTGH06ehXj3YtAmqVjVI+0IYigRtYTC+vsEcqtyJ+h6bwMqK/W2XE7Lxb1LVt6P8pnFYBPjr\nvc2zGauxNUdPRj0fiac/DNBVp8eKHtSp8xNoGhw9Cs2bw8qVUNtwDzqFMBQJ2sIgTp/2ZGizJbg8\nGQ9Av/onOWVeDmeXzKR68dQgbS6y7Uttm2tYv7tHn0B7zFs0ZvKU6qR4/QzWr4d16yAsDGbOhPr1\nDdIHIQxNFowSehUWpmPcuKMsXnSOF69UwK5Q4hb2lVJzepDhxgLXbCvQOXQjK03rMseiLksX21P+\n6WmoVRVu3VIzG1etgjJlZIajSJAkaIvv9uDBO1q33o6VhQleAeMAqFT2LtMKr6H8oHEGazc0mQVh\nxX/C7kxVRuSCy6EumLQeBQ4Oajf0mjXBzMxg7QsRF0jQFlGmaRpr115mwICDjPy9KL9tG42xvx+u\n+drgdjo3nDZc22/qNGPRfSvyu1/ncthOTN8VhTZtYO1qSJXKcA0LEcdITltEiY9PED17OuHh8Zwt\ncypi26U9Ng/1uyHB1zzIWxaTO7cwT2uFdd+umLRpDVmyxEjbQsQWmVwjftjJk0+ws1tEypRmuCyq\nSNom9XnmneTj54emTzVY269MU3Eu0BqjXTtJ43UPk6FDJGCLRE3SI+KrwsN1TJp0nHnz3Fm6tB7Z\nX77E0r4cwZbWFPY+y9OMWXm0ZDjV63aLdlt3S9mT+6zrx/cX8v/CJK9c1J/VnTbti/076hAi0ZP0\niPiiJ098aN16O6amxqxa1YizY5xosrLlx89fZslOmmePMQ7/fIOD6PDNVYAyoR0oXjEXs2bVJF06\nC71eX4j4QtIjIsp27rxJyZJLqVUrNyvWtuVRhXY0WdmSUHMLmi4/CEC6Jw/1HrCH155BkbAuzFzU\nkPXrG0vAFuILJD0iPgoICGXAAGecne+xc2dz/G8HkT2zCdkBp74jOJw5L1s71zBI2/nTjKFugUJc\n22KPhYWU7QnxNZIeEQBcufKCFi22YWeXnrk9c5GmUszs6HLSpij9snRhwbKGlCiRMUbaFCI+kPSI\n+CJN05g79wy//LKGP/qXZmaBl5EC9iubdHppx88mPS+SZeSKbaWPx4aZ18N9+AJOnO0uAVuIKIpO\neuRXYAyQDygFXNBHh0TMefnSnw4ddvLmTSAe03JiProD71/4RXyeJj3pXr/QS1uLUvSkdc4zFD7r\nBMCoskPptukPsmWz0sv1hUgsopMeyQfogMXAAL4etCU9Egft33+Hzp13M6heano/2or3mRu8CzPn\nJ79r3/7yd3C1qo5/mSpUfbuXZGdPst28OJYzp1CjWzW9tiNEQmOIBaNuRuO7IpYEB4cxZMg//LPl\nHO7l7mG7eRe3tBwU9H1AWuBw5dpUPeqkt/Yq6s7i9TyY7g9zY9N7CKMn1CBlyqR6u74QiY1UjyQi\nN2++pl3LLXQJc2eG/y50e4PwC09GwXD1S9LjjNn0GrC9fh9Jz1OpeRJmzpIj9ShePIPeri1EYvWt\noH0IsP3C8WHAHv13RxiCpmmsWnmRQ/3+4p8kzqR88wyA0zZVKZRbB2dcAMj67JFe2tOZWzDScTtL\nll9i9Ojy9OhREhMTeeYthD58K2hX10cjY8aM+fja3t4ee3t7fVxWRIGPTxATWy2imcs8OgbeB8At\nfW1cqgzhd5OVpFy/Ui/trMw1kDJdy5Fjak8qmfcm90NfLl/uToYMKfRyfSESOldXV1xdXb95nj7q\ntF2AgcD5r3wuDyJjyaV/rvCgUUca+qk/Gq+y9WgTOIZmjj/R7Tf9BNNQI1PmLgygSZF7pKxSjr5p\n29F+ZV+qVcupl+sLkVgZok67EfAEKAvsA/ZH41pCj3SBQRyr70imGuU+Buzlv+2nnPkulv16RW8B\ne17t9Tx7EoTu7Ume/lyHyxUas+zuDAnYQhiQzIhMYN7v2k+KhrUJMjYjsFdfklimwnLSCL23c2zX\nM0xsQunWbS9DAw/RyPYd5scOg4mJ3tsSIjGSGZGJwEuHJqRoqHYeN6tUDu3aXbSpU/TaxqMC1Xjh\n5cf6fedo1mwr8xuZ0SrIHfPtmyVgCxEDJGgnAJpOh2udXqRz3s6Ntv3Az4/T2RoSePwsKcLf662d\nN8u3cnTwdOyKLsTU1JgbRxpRefkojNatg/Tp9daOEOLrpE47nnvzwhf3sk2o9fAfwtOkJU9Oa96n\nzUb5wDd6bef2iRv0GOnOu3en2bOnJaWK20L16tCtG1Spote2hBBfJyPteOyi602uZi9DnrCXaFZW\n+Hf9HZdV1wk0tdRbG2E1HBg+9BAVGuyhXr2fcHfvQqlSmWD8eDAyghH6z5cLIb5ORtrxkKZprJ+w\nmxJjupGqWhVyOpTh3bZDtN//C7sfDddbO+4jFtByQyglUr3j0qXuZMyYAl69gi1bYMkSuHBB8thC\nxDCpHoln/P1DmNJ4Jr2PTIZBg0g3oh+hNukICjWOVv76edKM2AarmZJ+zdrQKaA6l277Mm+uAzUy\nB8OePern6lWoVg0GD4bSpfV1W0KI/5DqkQTg9u03jCrQi4FHJ5Fq40psxg3jWe5yJAny/+GAvcJu\nLG/N0vByyHSCg0KZOPofch/KT/3Ur7hW/RY1etSA2rXB0xNGjYKXL2HbNgnYQsQSSY/EE9u3Xed6\n+wGMTXIBi5MuBJqlwjhpUjLqQn/oehuqzMD86lkavNtKqovHuXzdl5HZO9LA+A5Dwm5gfCs/1KsH\nO3dC4cIqfy2EiHWSHonjwsJ0jPhjP8WXjKNO5gAs9u/GZ/ZyUs2d8EPXC0xuRe3ch9nyvCXWudPy\nvlQFvNbtIsu7R/iVqUiGLi2hTh0p4RMilhliPW1hYM+evadLk1VMvLWQ/JVyYta7B+TMSaofvF4t\nu9OMDpmNy5USAPiEeLP9gg6LFr+Ra053MqTSX9WJEMIwZKQdRx079oghTRexR7ee1A6VMfL3U6kK\n4HlJB2zPHYjytYKMk3Ep4y+U8VRrZT/5uS5d7+TDolxJpk2vQY4c1ga5ByHEj/vaSFuCdhyjaRoz\nZ57i6J/r+FvbTNJc2VVp3Qcv6nck/e7vW041IHNuzD3v8r7iL7Q2a8mdZ8HMn1+bX37JoefeCyH0\nRapH4oGAgFCaN9+Kz/zl7PZeSFKftx8D9qysahLL9wRs1zFO6OYvJFmgN+ur9SPXLQeqNSjM5cvd\nJWALEU/JSDuOuHv3LU2bbGZi2AFqX9/98XiYWXJOJS9HJZ8jUb7WyjV+tG1ghLFjT7ydj9IotAl5\nm1Rm4sSqpEljbojuCyH0TNIjcZiz811+a7OFe+8nYBYc8PH4xfaTKbZ6SJSvs3PgARz+rEmy+zfw\nq9WAI95WLCn6GxPnNqBIEakGESI+kaAdB2maxqxZp3GeuAXnN7MjPqhYEY4fj/J1HheqTppT+zG3\nNOH5tIUkHzGYySnrUHrxSBo2yv/vH74QIh6RoB3HBAeH0a3TDho6z6HhmxMRH2TIAF5eUb5O6MWr\nJClaEN8X3tyo3hKba+c42nsabaa2IWlSqegUIr6SB5FxyIsXfvQpNYpVG5pFDtgQ5YCtDRgIOh3G\nhfPz95/b8cxcAPz8SXHrMp3ndJCALUQCJSPtGHbN7SYetTvR2u/Uj12gZUuYPRvSpcPF5QEH2v/J\n0Odb8O4/nOyTBst0cyESCEmPxDZNw2PgNLLMGouNFvDt87/k77+haVNu3HjFyEH7aXh0MQ0tnmDh\ntBOj4sX1218hRKySaeyx6eZNPBu0oejt89//XSMj0DSYPJnX9rUZ33c/p9a6sjf5DmyqF8FkpROk\n+tGJ7UKI+EZy2oYUGIg2YgThBQuR+UcCdv78MGUKuqzZmBhYknz5/qLwzWOcNl5OuqF9MNn2twRs\nIRIZGWkbirMzWs9eGN2/R5T3drG1BT8/CA6GYcMIGzSYV8UrMf19ObyuvOBurdtYnTgCBw5AyZKG\n7L0QIo6Skba+eXlBixbg4IDR/XtR+07y5Go51LdvoWBBtPPn2V6kOV3yDSL0yVNaL+jOhkczsfL/\nMK1dArYQiZY8iNSX8HBYuBCGDwdf36h/7/ff4fp1OHECJk7kn7y1GDbSleCgMI6GLCKVXT6MXFxg\n2DDo21eqQ4RIJORBpCGdPw/du8O5c1H/Tv360KCB2s3czo5zqw/yx/w7eHoeYPz4Kvya7D7GDS9D\noLfam7FMGcP1XwgRb8hIOzp8fWHkSJg7N2rnt2oFAQHQuTOYm0PLljxwHEnvk9Zcu/6K0aMr07at\nHaamxlC5snrIuGoVpE5t0NsQQsQ9UqetT5oGW7dCx47g7x+177x6BWnSfHx7/uQDJow5wtkbvvzx\nR3m6di0ReRajpydkyiTpECESKQna+nL/Pvz2G7i4RO38AwegZs2Pb48ff8ykScfx8HjO8OGV6Ny5\nmEw5F0J8xhBBexpQFwgB7gEdAZ8vnJcwgnZICEyfrh40RkWrVrBmDZiYoNNpODvfZerUkzx65M0f\nf1SgQ4eiJEsmwVoI8WWGCNrVgcOADpj84diXFn+O/0H76FHo0QNsbFRZ3vXr///8x48hSxZ8fILY\nuPEqc+eeIWlSU/r3L0vLloVVzloIIf4PQ6zydwgVsAHOAJmjca246dUr6NBB1V336QN58kQE7OTJ\nPz9/+XK08HDOvTCha9c9ZM8+h0OH7jN/fm0uXOga8ZBRCCF+kL5+P+8EbNTTteKOefPAykrVXzdq\npI4lSQKhoRAY+PE0zdgYtx0XOHbZh3UFFhAaqqNjx6LcuNELW1vLWOq8ECIh+lZ65BBg+4Xjw4A9\nH14PB4oDTb5yDW306NEf39jb22Nvb/99vYwtgYHg4ADHjqn3Bw9CjRqRTnGgNU8LVSB16uQUKZKO\n1q2LUKZMJtktRgjxXVxdXXF1df34fuzYsWCA6pEOQBegKhD0lXPiZ07bz0+tXb13r3r4mCWLmkDz\nr9y54epVSJo09voohEiwDDEj0gEYBFTm6wE7/vm3Brt/f6hSRb1u2jTyOWfPyvofQohYEZ2R9h3A\nDHj74f0poOcXzos/I+1bt8DRUS361K+f2nTgwIGIzytUUKkSY3mYKIQwLJlc8//4+8OECbBkCXTr\npsr6Fi2KfM79+5AjR+z0TwiR6EjQ/hJNg5071Up7JUpA1qywYgW8fx9xzqpV0L59rHVRCJE4SdD+\nr7t3Ve31gwdQsSIcOgTPnqlyvn/5+6uFnYQQIoYZYnJN/BQQAKNGQdmyYGKiqj8uXIBHjyIC9ooV\nahQuAVsIEcckrpH2nj1qdJ0yJZiaqtx19eqwdGnEOX5+YGERe30UQggS+0j7/n2oV0/lppMkUWmQ\n5s3h4cOIgL10qRpdS8AWQsRhCXuZuaAgmDoVJk9WqRAjI7UX4y+/qJ1j/iWjayFEPJFwR9pOTlCw\nIIweHTEd/cIFNaL+N2AvXCijayFEvJLwRtoPH6oSvl271PuyZWHGDPXAMU+eiPPevwdLWcxJCBG/\nJJyRdnCwmiBToIAK2DlzqhmNJ09C+fIR087nzFGjawnYQoh4KGGMtJ2d1fTzO3fA2homTlSbFny6\nmNP9+5A2rQRrIUS8Fr9L/p48UWuEbNsGZmYqcA8frgK3EELEY4ZY5S/2hITArFkwbpyaLNOihRpd\ny9ogQogELv4F7cOHoVcvtSJfxYpqs90yZWK7V0IIESPiz4PIp0/VhJhq1UCngx071DKpErCFEIlI\n3A/aoaFqNJ0vnxplz5sH165Bw4ZqsowQQiQicTs94uqqUiH37qna66FDIVWq2O6VEELEmrg50vby\ngtat1XTz4sVV/nryZAnYQohEL24F7bAwmD0b8uZVizqdPQtr10K2bLHdMyGEiBPiTnrEzU2lQkJD\nYf16qFtXctZCCPEfcWOkPWmS2vG8Z0+4ckUtoyoBWwghPhM3ZkQ+eAA2NmpzAiGEELJHpBBCxCeJ\ne+caIYRIICRoCyFEPCJBWwgh4hEJ2kIIEY9I0BZCiHhEgrYQQsQj0Qna44FLgAdwGMiilx4JIYT4\nqugE7amAHVAU2AmM1kuP4hlXV9fY7oLBJfR7lPuL3xL6/f1XdIL2+09eWwKvo9mXeCkx/A+T0O9R\n7i9+S+j391/RXTBqAtAWCADKRr87Qggh/p9vjbQPAVe+8FPvw+fDgazAKmCWYboohBDiX/paeyQr\n4AQU+sJnd4FcempHCCESi0uoZ4aRRCc9kge48+F1A+DiV87LHY02hBBC6MlWVKrEA9gGpIvd7ggh\nhBBCCJEIJfSJONOAG6h73A4ktB2IfwWuAeFA8Vjuiz45ADdRab7BsdwXQ1gBvED9RpwQZQFcUP9v\nXgX6xG53EpYUn7x2BJbFVkcMpDoRlTiTP/wkJPmAn1B/QRJK0DZBPSTPDiRBDSjyx2aHDKASUIyE\nG7RtiXhQZwncIuH9GX4mptYeSegTcQ4Bug+vzwCZY7EvhnATuB3bndCz0qig/RAIBTahHqgnJG7A\nu9juhAE9R/1jC+CH+m03Y+x1J2bE5G7siWUiTidgY2x3QnxTJuDJJ+89gTKx1BcRfdlRv1WcieV+\nGJw+g/Yh1K8r/zUM2IOaiDMcGIKaiNNRj23HhG/dH6j7CwE2xFSn9Cgq95eQyMalCYclqpqtL2rE\nnaDpM2hXj+J5G1ATceKbb91fB6A2UNXwXTGIqP75JRRPifxAPAtqtC3ilySokuN1qIXrhJ7k+eS1\nI7A2tjpiIA6oJ9hpYrsjBuYClIjtTuiJKXAP9Wu1GQnzQSSo+0uoDyKNgDXIEhoGkdAn4twBHqFm\nhV4EFsRud/SuESr/G4h6+LM/drujN7VQFQd3gaGx3BdD2Ag8A4JRf37xLSX5LRVRBQAeRPzdc4jV\nHgkhhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghDOd/3Nj1YLG0TDcAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sampling = HMC(np.array([1.5,0]), lambda x: - p2d.logpdf(x))\n", "\n", "samples = [sampling.sample() for _ in range(250)]\n", "plt.plot(map(lambda x: x[0], samples), map(lambda x: x[1], samples), 'r-')\n", "plt.contour(X, Y, p2d.pdf(XY));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## PyStan\n", "\n", "*Stan* is a probabilistic programming language which uses automatic differentiation and Hamiltonian MC under the hood. Its default sampler is called **NUTS** (No U-Turn Sampler) which has no free parameters. It is based on the idea that a good proposal should efficiently explore the distribution and thus, move as far away from the starting point as possible. NUTS integrates the Hamiltonian until the direction of motion starts to point back to the starting point, i.e. it would make a U-turn.\n", "\n", "Note: The actual algorithm is more complicated as it needs to ensure that the proposal is reversible and obeys detailed balance.\n", "\n", "Below is the linear regression model in PyStan $\\ldots$ it would be very similar when using RStan" ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import pystan\n", "\n", "# 2-dimensional Gaussian example\n", "model_code = \"\"\"\n", " data {\n", " }\n", " transformed data {\n", " matrix[2,2] Sigma;\n", " vector[2] mu;\n", " \n", " mu[1] <- 0; mu[2] <- 0;\n", " Sigma[1,1] <- 1; Sigma[1,2] <- 0.99; Sigma[2,1] <- 0.99; Sigma[2,2] <- 1;\n", " }\n", " parameters {\n", " vector[2] x;\n", " }\n", " model {\n", " x ~ multi_normal(mu, Sigma);\n", " }\n", "\"\"\"\n", "\n", "samples = pystan.stan(model_code=model_code, iter=1000, chains=4)" ] }, { "cell_type": "code", "execution_count": 48, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcjvX+x/HXjCXLSE6o7CXLKX5JkgrdKlJnIaXt1EHb\nSakoOakw0iKdihx12mQpKkQLJzoy7SgMQrJkqxDJNlnn+v3xGc0Mg5m5r7m+93Xf7+fjMQ/3fc3t\n/n5In/s7n+vz/X5BRERERERERERERERERERERERERKRQSgGzgHRgMfC423BERKQgymT9WhyYCTR3\nGIuISMJI9uE9MrJ+LQkUA37x4T1FROQo/EjgyVgJZQMwAyuliIhIiJTHSigRx3GIiCSE4j6+11Zg\nMtAESDtwsXbt2t6KFSt8HEZEJCGsAE490guiLaFUBI7LelwaaA3MyxXBihV4nhfar379+jmPQfG7\njyMR4w9z7PEQP1D7aAk42hn4ScBI7IMgGRgNTI/yPUVEJB+iTeALgcZ+BCIiIgXjRxdKXItEIq5D\niIridyvM8Yc5dgh//PmRFMAYXlY9R0RE8ikpKQmOkqM1AxcRCSklcBGRkFICFxEJKSVwEZGQUgIX\nEQkpJXARkZBSAhcRCSklcBGRI/E8GD0ann469/V334U77nATUxY/dyMUEYkva9bAP/4Bq1fD66/b\ntS1b4O674ZNPYNQop+FpBi4icrDMTBg2DBo2hJNPhtRUOPNMeP99OP10KF4cFiyAli2dhqml9CKS\nWLZuhX794LbboH79Q7+/dCncfLPNvrt1g1dfhVNOgYoV4b//hRdfhHbtijxMLaUXEcnp88+hUSN4\n/nn46afc39u3DwYOhDPOgBo1oH17eOIJe316Ovz6KyxcGEjyzi8lcBGJf/v22az7iitg8GCoXRsq\nVIAdO+z76elwzjl2o/LBB2HxYkv2ZcvC2LHwyCMwcSJUruz2z3EQlVBEJD5lZkLfvnD11XDrrXDs\nsTBiBJx0EiQlQZUqcOON9ronnoC2baFOHRg+HJo3hylT7H2uuALGj7fHe/bA9u1w/PFFHn5+SihK\n4CISX376CVJSLGGDzaJTU62eXawYPPoo9O8PV14JEybYa++4A2bMgF9+gWXLst/rj3+EuXOhVCkY\nORI6d4Zq1WDt2iL/YyiBi0jiyMyEZ5+FHj1yX7/kErshuWYN7NwJZcpARkb29+++G155BU47DWbP\nzr6ekmKtgikpULdu9vX9+yG56KvPSuAikhjWrIEuXeCjjwr+e088Edavz35er551ouRl3z6bxQdA\nXSgiEt88D157Df7v/wqXvCF38oa8k/c779hYASXv/NJKTBEJp82boWtXGDeuaMfZtg3KlbPZ95gx\n1q1Sr17RjplPmoGLSPh88IGtkizK5P366zbrLlfOxmvUCJ57DkqXLroxC0gzcBEJj5074b77bCFO\nUTpwo3LBAhvvq6+s1fCmmwK5gZlffkRSHZgBLAK+Ae7y4T1FRHKbNcv2Ixk+vOjGmDHDZt0bNthy\n+saNrW1w6VK45ZaYSt7gzwx8L9ADSAdSgDnAh8ASH95bRBLd3r0wYIDtQeJ5sHt30YzTt6/VuZOy\nGj8aNoSPP4bzz89+zapVUL68reKMAX58nKzHkjfADixxV/HhfUUk0S1ZAueeC198YbPijRuLZpxr\nroGHH4bWre15//4wZ44l7x07rEUxKcl2Jvz3v4smhkLwuw+8FvAxcDqWzEF94CJSUJmZligHDIDu\n3eGhh4IZt2RJWLHCltv/+9829gHnnANvvgk1awYSSn76wP28iZkCjAfuJjt5A5Camvr740gkQiQS\n8XFYEYkr69bZkvWMDLj/fujZM5hxJ0+2fb5POcXKNmBL6KdMgVatinz4tLQ00tLSCvR7/JqBlwDe\nB/4LDD7oe5qBi8jReZ7t/Ne9u31NnmylkyA0aADffJP9/Nln4fbbnS7cCWopfRIwEtiM3cw8mBK4\niBzZL7/YopxvvoGXXsp94zAoN90ETz1lNyljQFBL6c8HrgdaAfOyvtr68L4ikgimTrWl8FWr2n7c\nQSbvJk3sRqnnwcsvx0zyzi9tZiUibmRkQK9e8N571tv98MO2+18Q3nzTtpONsb7unIK+iSkikj+z\nZkGzZnY82bRpeZ9NWRQmTrSj0uJE7H78iEj82bvX9utu1syen312MMn7T3+yZfhxlLxBJRQRCcKe\nPbBypa1u3Lcv2LE/+8zNTdEoaT9wEXHv6afhmGPseLIgk3eXLjbrDmHyzi/VwEWk6LRqBQcWp5Qq\nBbt2BTPu55/DeecFM5ZDKqGIiP88z02Hx6WX2kHFMbRnd2GphCIiwdu40U3ynjrVlr3HQfLOLyVw\nEfHPG2/ACScEO2aJEtZT3qZNsOPGACVwEYleRgZcdBFce22w4z75pHW4JNCsOyfdxBSRwhs92vbp\nvu++4Mf+/nuoVSv4cWOIEriIFNz+/XDPPbZrX9CaN7ejz4orfelvQEQKZscOW0H57bfBj/3663Dd\ndcGPG6OUwEUk/9asCexEmkMsXQp167oZO0bpJqaIHNn27bbpVNeubpJ3hw4261fyPoRm4CJyeOvX\n20ZQc+e6GX/oULjjjuyT4iUXJXARyduyZXaQ75YtwY9dsSK8+66dSC+HpRKKiBxq9mwrWbhI3q1a\nwaJFSt75oAQuIrlNmGAz76CVLGmn0E+bBpUrBz9+CCmBi0i2bt3sqLGgpKTADTdApUrw1lvw+OPq\n7y4A/U2JiO3TXa5ccNu9gn1YrF8P6em2/WudOsGNHSc0AxdJdN9+axtCBZW8q1a1Jfgff2x7mMyc\nqeRdSErgIonK8+CZZ+yknCCUKGGn0D/1lC3D79oVRo6EMmWCGT8O6UAHkUT0449wwQWwfHnRj3XO\nObB1K7z8Mrz3HowZA+PGublRGiJBHegwHNgALPThvUSkqD36qJUxgkjeYB8UU6dCnz4wZ459KXn7\nwo8E/irQ1of3EZGismgR9OhhKxofeiiYMWvVsvp2+/Z2sPB558EHH1jHifjCjwT+KeCg219EDmvn\nTqtx//YbNGkCDRrA4MHBjd+rFyxZYguC2rWD556DRx6BYsWCiyEBqI1QJN4MHWo3CfftC37sSpWs\nzt2gAdx4IyxeDF9+CbVrBx9LAggkgaempv7+OBKJEIlEghhWJLF4Hjz4oK2kbNfOfg3SPfdYfX3N\nGqtxN2kCX3yhLpN8SktLIy0trUC/x68ulFrAe0DDPL6nLhSRorZ3L9x6q/VWf/998OPPnGlJ++23\n4bbbrFxyyy3aRTAK+elCUQlFJOx27oR69eCHH4Ifu3FjW0VZvLidizluHEyebCf2SJHz4ybmWOAL\noC6wFujiw3uKSH5s2mT7ibhI3qNGWUvgr7/CxRfDggXw9ddK3gHSQh6RsFqyBE47zc3Ya9dCtWo2\n+776arjpJujbV10mPlIJRSRejR8PHTu6GXvPHiuZDBkCjz0Gr74Kl13mJpYEp71QRMLmqqvcJO9+\n/exsyt274dprbR+TmTOVvB3SDFwkLPbutUMPXOjUCYYPtwQ+ZYqdlvP557aboDijGbhIGHzxhbvk\nXb48zJ9vS+JHjrR+71deUfKOAZqBi8SyjAw46STYts3N+OXK2Sk5S5fCO+/YXiZnneUmFjmEZuAi\nsWj/fnjxRShb1l3yvvde28tk7FhL4HPmKHnHGM3ARWKJ59ks1+WNweRk28Nk40a46CJb4dmnj12X\nmKL/IiKxYs4cWxDjuqsjErFyyZVXWq27Xz8l7xilGbiIS5s328EK/frZoQcuJSdbieSjj+Dnn2HW\nLNvTW2KWVmKKuOB5VmN+5hnXkRyqQgU7cq1UKdeRJLSgjlQTkYLYscNmu888EztJskYN+3X4cPjl\nl9iJS45ICVwkKJmZ8Pzz1pp3wK5d7uIBO2i4a1fbw2TuXOiivejCRDVwkSB8+CH8/e+wfr3rSMzk\nydCokS3LP+44u4FaoYLrqKSANAMXKUrp6XDJJdCmTWwk7+HD7SeBsmXtxJy2beHdd5W8Q0ozcJGi\nsGMH3H23teNt3uw6Gtvu9cknbbb91FPwr3/Zft5t2riOTKKgLhQRvy1ZAldcYa14mza5jsZkZsL2\n7VbjXrvWTs6pWdN1VHIE6kIRCdpbb0GLFpbEYyV5e56trDz7bDs1/tNPlbzjhBK4iB/27IHu3eH+\n++H6611HY+66y2bdY8fa6soHHoD//AeOOcZ1ZOITlVBEovXDD9bNsW0bfPON62igbl27WXn22dCz\np+3fPWECnHGG68ikAHSkmkhRmzHD6t2ZmbB1q9tYkpPtZPh+/ezGaSQCFSvaQcPHHec2NikSKqGI\nFIbnwcCBcOGFsGULnHKKu1iSk6FhQ9u7ZOBA+PJLm33/+c8waZKSdxxTCUWkoLZutSPGZs6EDRvc\nxlKhgrUr9u4NJUrAoEEweDCMHm07G0poqQtFxG8LFtgCmOrVYdUqd3Ecc4zF8fHHVjL57Tfo0AEm\nTrRDGJS8E4IfCbwt8C2wDPinD+8nEptGjbIDDvr3t5PZXZ0JmZwMAwZYqaRhQ1i40JJ5lSqW0KtX\ndxOXBC7aEkoxYClwMfAD8BVwLbAkx2tUQpFw273byhQzZlgrXufO7mI56ywYM8Y6TQBeew169LCd\nDWOlfVF8EUQXSlNgObAq6/kbQDtyJ3CR8Fq92k6mqVnTVla6TN6DB8Odd9oMfPduOx1+2jQ7gKFh\nQ3dxiTPRJvCqwNocz9cB50T5niKx4YMPLGFXrWp91C6tXAknn2yP166Fjh3ttPqvv4by5d3GJs5E\nm8DzVRtJTU39/XEkEiESiUQ5rEgRysy0GvOBf7cuO02aNrVul6Ssn6SnT7dSSffu0KtX9nUJvbS0\nNNLS0gr0e6L9r98MSMVuZAL0BjKBJ3K8RjVwCY/Nm22L1a+/dh0JDBsGt99ujzMzrcd76FB4/XXr\nP5e4FkQN/GugDlAL+BG4GruJKRI+w4ZBt26uozCffQbnn2+Pf/3VDoPYtMk+WKpWdRubxIxoE/g+\noBswFetIeQXdwJSw2b8fijveVaJGDVizxvq7586F006z6/Pn21L9yy6D8eOhZEm3cUpM0UpMSVye\nZ+dBvvCC2ziqVLHZ/86dcMEFUK2aXR81yk6uHzIErrvObYwSuPyUUJTAJfF4np1I06uX60igZUvr\nMd++3VoW16yxr1Wr7AblhAnQoIHrKMUBJXCRnDzP9gjp1Ml1JNlq1bLySY0a1mue83Ht2iqZJDBt\nJytywPvvw1/+4jqKbIsWQb16UKyY60gkxJTAJb6tWAGnnuo6imxpaVbnFvGBdiOU+DVtWuwk78qV\ns29SivhECVzij+fZDcBLLnEdibn3XlvNWaaM60gkzqiEIvFl925o3tx1FNnGj7c+bpEioAQu8WPJ\nkuwFMLFg2jRo3dp1FBLHVEKR8MvMhC5dYit5f/KJkrcUOc3AJdy+/BLOO891FLk99xy0aOE6CkkA\nSuASPp5n7YF16riOJFskYi2Cc+fCmWe6jkYShEooEi7XX28n0sRK8m7Z0g46njHDPliUvCVAmoFL\nOHge/PnPMGWK60jMscfCd9/BCSe4jkQSmGbgEvvGjbNZd6wk75degq1blbzFOSVwiV2zZ9tinKuu\nch2JGTHCOl5uvtl1JCKAErjEogULoF076NDBNqA6/ni38VSoAD/+aLsY6gxKiSFK4BI7li6Fa66B\nNm2yuzruvNPOqXRpyxY7xmziRLdxiBxECVzcW7PGFuI0bw5nnGErKnfvdt9p8vDDcPLJFte779pP\nBSIxRF0o4t7QoTBmjC0937PHDjnYts1tTCecAF99ZQdAHDhcWCTGKIGLe4MGQalSVjaJBZ06wX33\nwemnu45E5IhUQhG3MjKgXz94/nk46yy3sfztb1bOGTFCyVtCQWdiihueZ/3dPXvCzz/Drl1u41m0\nKLY2w5KEpzMxJTYtXAh33WVdJq5VqWK17ipVXEciUmDRlFA6AouA/UBjf8KRuPbrr9YW2KiRbUbV\nsKG7WNq3tw6TmTOVvCW0okngC4HLgU98ikXi3fTp1tM9fLgtRV+4MPgYeveGYcNs18CPPoLq1YOP\nQcQn0ZRQvvUtCkkM7dvD8uXQubOb8b/7znrLJ0yw8k2tWm7iEPGJauASjAULbJGOC3/9q51NWaKE\nPdcZlRInjpbAPwROzOP6A8B7+R0kNTX198eRSIRIrPT7StHLyLBVlm+95Wb8Dz6IndPpRY4gLS2N\ntALe2PejjXAGcC8w9zDfVxthonr/fduMyoVTTrHj1ipXdjO+SJTy00bo10IebdEm2TZtgrJl3SXv\nJ56AZcuUvCXuRZN4LweeBSoCW4F5wKV5vE4z8ESyaRNUquRm7KQkawts2tTN+CI+KuoZ+ESgOlAa\nq5PnlbwlUXge3HSTu+R91VW27auStyQQdaFI9ObMgSZN3I3/0kv24aHDFiTBKIFL4e3ebbsIunL6\n6fDmm9p4ShKWdiOUwnnySbfJ+7bbbA8TJW9JYJqBS8GsWmV7iLhSvjy8/DJceaW7GERihBK45I/n\n2Wx3yRJ3MTRrBmPHagm8SBaVUOToxo2D5GR3yTspCe6/Hz75RMlbJAfNwOXwgu7pbtECPv0097UT\nTrBzKVu3Di4OkZDQDFwO5XnQsWNwyfu442DkyEOTd5s2MH++krfIYWgGLrl9+im0bBnceI8+avuW\nXHutPS9b1toTH33UjltL1hxD5HB0JqaYHTugXLlgxxw1yvbmXroUateGKVOgZk27UdmsWbCxiMSY\nIDezkrDyPDulJsjkXa0a9O8P995rx5lVqQKTJ1tr4Lx5St4i+aQZeCJzcchCmzawZ48dqXbbbfDI\nI3Yq/ZAhcMstWg4vkkWn0kvefvvNShY//RTsuE2a2FmUPXpA6dLQrRvUrWulkwYNgo1FJA6ohJJo\nnnsOypQJPnmnpEDx4lYqWbgQ7rnHTuqZPVvJW6SQNANPFMuX24G+rgwYABdeaNu+/vSTHbHWsaO7\neETigGbg8W7XLmjVyl3ybtHCPjwqVoRzz7W9TNLTlbxFfKAEHs/GjbNacwEPSvXNf/4DU6fazoU3\n3GA1788+c7sZlkgcUQklXgXRzXH22bal68FOPRU++gj27bMZ+Nq1Oh1epAhoBh5vvvvu0ORdtaq/\nNwp797ZyyMHJu317mDXLYkhPh8aNoUIFWw6v5C3iO/WBx4u8TsepUMH6rO+4w79x5s2DM8889PqQ\nIXDnnbB/P/TpA089BQ8/DL16aTm8SCHkpw9cCTzs9u61zo5Jk3Jff/NNGDbMtmD1Q58+tginRYtD\nv/fpp9C8OaxfD9dcY4c+jB1rs3QRKRQtpY9nu3bBY49ByZK5k/eQIXbD8IYb/Eve69bBhg2HJu9T\nToEff7Tk/fHHNjOvWNHKJ0reIkVONzHDaN48qy/ndMsttpNf//526MGePf6MtXkzHH/8odc7dbLT\n4IsVgyeesFLNv/4Ft96q5fAiAYk2gT8J/BnYA6wAugBbow1KjmDrVnjmmeznZ51lmz+9/Taccw7U\nrw9ffBH9OD17wsUX5528X3kFbrwRtmyBzp1h2TIbs2HD6McVkXyLtoQyDTgdOAP4DugddUSSN8+z\n1Yu1atkJNcnJcMUVdvNy/ny7WTh9uj/Je8QIK420bXvo9+bNs+S9eLF9eFSubN0oSt4igfPzZ93L\ngSuA6w+6rpuY0VqxwjpJpk61540bW5liyxYrXcyfb2UMP7zwAvzjH4de/8tfbP/u446z5zNmwC+/\n2IeIiPgu6C6U94CxwJiDriuBF9bu3baKsU8fe56SYj3dmzdbL3bLlnDZZbYdqx+6doXnn89+XqeO\nLYMfMMDGUzugSGD82k72Q+DEPK4/gCVtgAexOvjByRuA1NTU3x9HIhEikUg+hk1wM2bYjckVK+x5\n2bL26yWXwIMPWh367LP9G69q1dzJe/hwuO8+W0HZpo1/44hIntLS0kgr4LYXfszAOwO3ABcBu/L4\nvmbgBbFxo91AHD069/V27WDQIOv6OPXUohv/3HPtwyMzEzIy8r6JKSJFLog+8LbAfUA78k7ekl+Z\nmfDiizYTzpm8GzWyfUXefttWOhZl8h4xwm6CHnOMbYKl5C0S06JN4EOBFKzMMg94LuqIEtH8+XD+\n+XbzcN8+u1alCrz6qnV4LFliM+9p06If64Yb8r6+apX1dotIaESbwOsANYEzs75ujzqiRLJjhx3s\n26gRzJxp18qUgdRUS9qlS0OJEv7sZVKihC28KVky9/VWrWzRT82a0Y8hIoFSW4ELngcTJ9q+2E8/\nbdeSkuyIscWL7Xrt2raviB/q14c5c+B//7ObnweMGWPlmRIl/BlHRAKlzayCtmqV1bLffz/7WqtW\n8Pjjdkr844/D99/7N97f/mYHKxQvbjP6A9ats3q7iMQkbWYVS/butcU2J5+cnbzr1rVdAy+/3I4Y\nGz48+uR9+eX2a4UKdlN09Gjb+OpAG+Jll1ksSt4ioacZeBA++8x6ur/91p7/4Q/WYw3w7LPWz33h\nhfDAA9a6VxiNG9u2rmXK2Hjly9vy9i+/hPPOs9dMmAAdOkT/5xGRIqf9wF3btAn++U+bWYPVmjt1\nsuXoI0dCJGKJfNIkK50U9u/pnXfgr3/Nfc3zoG9fW2oPdhL8iXmtxxKRWOTXSkwpKM+znurbb7fy\nBdgM+6ST7OblpZfaQcP791tb3/z5hRunXj3r2/7DH3Jfz8iwTa9+/tlKKuPGWRuiiMQV1cD9tmgR\nXHCB7di3a5e15513nh1yUKqUtQuOGAGTJ0OTJoVP3qNGWUnm4OS9e7fVu3/+Gd591xYAKXmLxCXN\nwP2SkWGbPg0caM+Tkux0mq1b4U9/siPGatSAlSutdDJ7duE2h6pUCebOhWrVDv3e6tXWelixorUj\nVqoU1R9JRGKbZuB+mDwZTj8d1qyxujPYcvTrr7cZ+bBhUL06vPwynHGGzY6LF88ur+TXvffauZN5\nJe933oGmTe0m5YYNSt4iCUA3MaOxbh3cfTcsXAg9etiy93fesWPF7rknO4muXw8332yz7qFDbbl8\n9+52kzO/Bg+2sQ62e7fdKJ00SQcJi8QR9YEXlX377FizRo2svtywoc28a9a048Uefzw7eY8fDw0a\n2MZQS5bA1VfbawuSvHv3zjt5r1hhe6isXm1lFSVvkYSiGnhBzZwJt91ms+8qVew09h49bOOpY4/N\n/dquXW0/7TFjbE/t7dvh73+3OvjRlC1rZ10mJ1tt/WDjxtkeKQ89ZCs7dZCwSMJRAs+vLVtsJvzi\ni9YmeNJJtnfJrbdmr3I8WLt2dqJOSgrMmgXXXQflyh19rDvusNN2eva0PUxydpHs2mXlmalTYcoU\n62QRkYSkBH40ngevv243EDdutJuR999vbYKlSh3597Zta73ejz1mhzE8+aQtuPnjH+0D4WDFi9sN\n0VNPtXLIpEm5b0YuXQpXXWX933Pn2mpLEUlYqoEfydKlcPHFttgmJcW6SJYvtwU6R0veAGvXwkUX\n2Y3Nr76ynu0TT8w7eV9zjXWPtGxp+6I88EDumvZrr0Hz5jb2m28qeYtIILzQ+e03z+vb1/NKlvS8\n+vU9b/Roz9u7t2DvMX6851Wq5Hn9+nneunWe176959l8/tCvMWOyf9+tt3pex46el5lpz3fu9Lwb\nb/S8unU9Lz3dtz+iiMQ24KjteyqhHGzZMlvqXraszXo7dCj4SkbPgzfesJn3ypV5920DtG5t+6Qc\n+P6oUbbE/quv7KbkokVWMmnc2GrhKSlR/dFEJL6oD/xgGzbYsvfWrQu3UvKAH36wjaumT8/7+88+\nazcrD4yxcKHtlzJjhi0KevVV6+8eNAg6d1aXiUiC0W6ELniezapvvjnv7zdpYnt016+ffW3bNttS\n9qGHoH17az9MT4e33oLTTgsmbhGJKVrIE7RVq6zccbjk3bev7R6YM3l7nnW0tGplC4KaNLEbpLNn\nK3mLyBGpBu6HzEx4/nno1i339fLl7fSbatVs1t206aG/d8gQS/zNm1vZZvBgOwZNROQoVEKJ1rJl\ntn3sTz9lXytRwnq5V6+2xT6DBtlJOQf7/HNrU2zQwBL9W2/ZMWsikvB0oENR+9//bNacU/36dnjw\nhg22F/cll+T9ezdutA6TXbtsZv7UU/nrLRcRyRJNDXwAMB9IB6YD1X2JKCw8L3fyLl0arrzS9gWv\nW9e6Sg6XvPfvt2X1O3bYrHvYMCVvESmwaBL4IOAMoBEwCejnS0RhkZRkOwsec4ztCNihg7UADhxo\nPeAHn5ST0/z5VjKZN89WXYqIFEI0JZTtOR6nAAXYHzXkMjJsQ6kpU+ym47ZtVk5JTz/8op2cGje2\nXQxFRKIQ7U3MR4EbgAygGfBrHq+Jr5uYq1fbJlUnnwwvvGCbW3meFtqIiK/8WMjzIXBiHtcfAN7L\n8fx+oB7QJY/Xev36ZVdXIpEIkUjkKMPGsJUrrfTRoYOStoj4Ji0tjbS0tN+f9+/fHwJaiVkDmAI0\nyON78TUDFxEJQFGvxKyT43E7YF4U7yUiIgUUzQx8PFY22Q+sALoCG/N4nWbgIiIFpM2sRERCSptZ\niYjEMSVwEZGQUgIXEQkpJXARkZBSAhcRCSklcBGRkFICFxEJKSVwEZGQUgIXEQkpJXARkZBSAhcR\nCSklcBGRkFICFxEJKSVwEZGQUgIXEQkpJXARkZBSAhcRCSklcBGRkFICFxEJKSVwEZGQUgIXEQkp\nJXARkZDyI4HfC2QCf/DhvUREJJ+iTeDVgdbAah9iiUlpaWmuQ4iK4ncrzPGHOXYIf/z5EW0Cfxro\n5UcgsSrs/wgUv1thjj/MsUP448+PaBJ4O2AdsMCnWEREpACKH+X7HwIn5nH9QaA30CbHtSS/ghIR\nkaMrbNJtAEwHMrKeVwN+AJoCGw967XKgdiHHERFJVCuAU4MY6HvUhSIiEii/+sA9n95HRERERET8\nFNZFPwOA+UA6Vvuv7jacAnsSWIL9Gd4GyrsNp0A6AouA/UBjx7EURFvgW2AZ8E/HsRTUcGADsNB1\nIIVUHZhYvxqKAAACLElEQVSB/bv5BrjLbTgFVgqYheWbxcDjbsMx1YEPCGe9vFyOx3cCL7sKpJBa\nk10uG5j1FRb1gbrY/5BhSeDFsJv3tYAS2P+If3QZUAG1AM4kvAn8RKBR1uMUYCnh+vsHKJP1a3Fg\nJtA8rxcFuRdKmBf9bM/xOAXY5CqQQvoQ+8kH7JO9msNYCupb4DvXQRRQUyyBrwL2Am9g6ybC4lNg\ni+sgorAe+9AE2IH99FnFXTiFcqDDryQ2IfglrxcFlcDjYdHPo8AaoBPhmsEe7EZgiusg4lxVYG2O\n5+uyrknwamE/TcxyHEdBJWMfQhuwnz4X5/Wioy3kKYiwL/o5XPwPAO9hf44HgfuBZ4AuwYWWL0eL\nHyz+PcCYoILKp/zEHibqyooNKcB44G5sJh4mmVgZqDwwFYgAaS4CaYB9inyf9bUX+9GysotgfFAD\nuzESNp2Bz7EbJGEUphp4M+x+zwG9Cd+NzFqEtwYOdu9hKtDddSA+6AP0dB3EAWG8iVknx+M7gdGu\nAimkttgd+YquA4nCDOAs10HkU3FsFV0trIYZtpuYEO4EngSMwn5SDqOKwHFZj0sDnwAXuQsnt5WE\nL4GPx/4xpwMTCN9PD8uwLX/nZX095zacArkcqyf/ht2c+q/bcPLtUqz7YTk2Aw+TscCPwG7s7z7W\nyoVH0xwrQaST/W++rdOICqYhMBeLfwFwn9twRERERERERERERERERERERERERERERERERBLc/wN9\ntZUrJZ/2ngAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "x = samples.extract('x')['x']\n", "plt.plot(x[:,0], x[:,1], 'r-');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Bayesian linear regression: " ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [], "source": [ "lm_code = \"\"\"\n", " data {\n", " int N; // Number of samples\n", " int D; // Number of features\n", " matrix[N,D] X; // Design matrix\n", " vector[N] t; // target values\n", " }\n", " parameters {\n", " vector[D] w; // weight vector\n", " real sigma; // Observation noise\n", " }\n", " model {\n", " vector[N] y;\n", " \n", " w ~ normal(0, 100); // Prior p(w)\n", " y <- X*w;\n", " t ~ normal(y, sigma);\n", " }\n", " generated quantities {\n", " real mse; // Training error\n", " real R2; // Explained variance\n", " vector[N] y;\n", " \n", " y <- X*w;\n", " mse <- 1.0/N*dot_self(y-t);\n", " R2 <- 1.0 - mse/variance(t);\n", " }\n", "\"\"\"\n", "\n", "stan_lm = pystan.StanModel(model_code=lm_code)" ] }, { "cell_type": "code", "execution_count": 50, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEACAYAAACTXJylAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHcVJREFUeJzt3X9wG+d95/G3LImyZBoEJFBHaUj9qJtYnp4jUaEaj6Wc\n0FTp2Kyv4LTDq5tO/CNT4p9OL3PXSpdcZyL5JtNrm7lrZm7amdNlTLm90mcrjZXzXfmHeGPIGeXs\ny7WyKP+ImMgma1WmRTWAGE2mjCRv/1gstFwugF1ggd0FPq8ZjgAQWDy0rC8efp/v93lARERERERE\nRERERERERERERERERNrcs8CHwAXH478DvAO8CfxRqwclIiKVfRoYZHng/gXgNLC2dL+31YMSEZHq\ndrA8cL8IfCacoYiIyF11vOZjwL8AXgPywFCQAxIRkerW1PmaFPAQsA9zBv4zQQ5KREQqqydwXwa+\nVbr9PeAjYBPwD/Yn3XfffcalS5caG52ISOe5BPxstSfUkyo5xZ0c98eBLhxBG+DSpUsYhhH5r6NH\nj4Y+Bo1T44zrGDXO4L+A+2oF4Voz7ueBg5gz6veBr2CWCD6LuWD5U+AJb/FeRESCUCtw/0aFxz8f\n9EBERMSbelIlbSWTyYQ9BE80zmDFYZxxGCNonGFY1cRrG6V8jYiIeLRq1SqoEZs7fsYtIhI3Ctwi\nIjGjwC0iEjMK3CIiMaPALSKRlcvlyGQyDA8PUywWI3e9sChwi0hkzczMcObMGSYnJ8nlcpG7XlgU\nuEUksjZs2ADA0NAQx48fj9z1wqI6bhGJrGKxSC6X4/jx4ySTychdrxm81HErcIuIRIgacERE2pAC\nt4hIzChwi4jEjAK3iEjMKHCLSN3apaElbhS4RaRu7dLQEjcK3CJSt3ZpaIkb1XGLSN28NrTkcjlm\nZmbYsGEDExMTkW1+iQI14IhIJGQyGc6cOQPA6OgoL774Ysgjii414IhIJCilEizNuEWk6eKwR0hU\nKFUiIhIzQaRKngU+BC64fO93gY+AjfUMTkRE6lMrcI8Dj7g8PgB8FpgLfEQiIlJVrcD9HaDg8vh/\nBo4EPxwREamlnqqSLHAZmA54LCISYa1qb1cbfW1rfD5/A/DvMdMklopJ9GPHjpVvZzIZMpmMz7cT\nkaiw2tvBDK7NqsVu1ftERT6fJ5/P+3qNl6qSHcDLwIOlryngJ6Xv9QN/D/w8cNXxOlWViLSR4eFh\nJicnSafT3H///SQSiaZ0QVrvMzQ0xOnTpzuufDCocsAd3AncTu8BnwR+5PI9BW6RNmLVYl+5coWz\nZ88CzemC7PSa7yAC9/PAQWAT5oz6K5iVJpZ3gSEUuEXagpc9RTp9RtxsasAREV+87CnS6TPiZvMS\nuP0uTopIG/Oyp0gymWz7BcOo04xbRMo0mw6fUiUiIjGjbV1FRNqQAreISMwocIt0GLWUx58Ct0iH\n0cns8afALdJhdIxY/KmqRKTDqOQv2lQOKCKeeWl3l+ZTOaBIjIS9aKjcd3wocItERKOBs9HAr9x3\nfChwi0REo4Gz0cA/MTHB6OhoIDv+OT9Ewv5tQrwzRMS7QqFgjI6OGoVCoa7XP/roowZgDA0N1X2N\noBw8eNAADMAYHR1dcV8qK/13qkozbpGIsHbdq3e2G+SMuVHO3x6UhgmWArdIm0gmkySTSUZGRkJP\nSTg/RKL0odIOVA4o0ka8HIRQi8oCw6VyQJEOE0RKQmWB0afALdJG7CmJI0eO1FXJoXx09ClVItKm\n6k2bqCU+XDpzUqSD1Ttz1pmS0acZt0hM1VpErDVz1iJkNGmTKZE21mgFSRAVKBK8oKpKngU+BC7Y\nHvsa8A5wHvgW0FPfEEXE4rct3G8qxHl9LUK2t08DgywP3J/lTtD/w9KXU3g9oyIx5Lct3G+LvPP6\njbbYS3PgoeXdy+Lkd4AdjsdO226/Dvyah+uISBV+Z8B+FxGd19ciZHwFUcf9BeCvA7iOSEdrdlu4\n2s7bh9fFyR3Ay8CDjsd/H9iL+4zbOHr0aPlOJpMhk8n4H6GIVKTKkPjL5/Pk8/ny/WeeeQYCqirZ\nwcrA/RQwBvwi8I8urymla0SklnoDsCpD2k8z9yp5BDgMZHEP2iLiQ737g9jz1uvXr9dhBR3CS+B+\nHvgucD/wPmZO+78A3ZiLlOeAP2vWAEU6Qb2lefa89dzcnDaH6hBqwBGJgCD2BxkeHmZycpKhoSEt\nQMaY9ioRiQm/pXn2nHhvby9zc3P84Ac/YNOmTQrYHUAzbpEmalbVh31RMp1Oc+3atWXfr7ZQqUqU\naNOMW6SF3GbB09PTFAqF8veDqPrI5XJMT08DMDg4yKZNm5iamiKRSLC4uFgzT24thAY5JmkfYXaN\nirScvaU8nU6Xb+Pj5PWxsTHj4MGDxqOPPlrx+fb3yWaz5db12dlZTy3sUToNXlYioJZ3EfHAXhmS\nTCaZmppicHCQbdu2ceLEiXJKwm1mbqUtvMyG7e9jXdd6npfZ88TEhA5KkIrC/uASaSn7pk3VNnCy\nz5jXrVu3bPbsZTYcxuZQXn4TkGDgYcatxUmRFrOX7f3whz8sN8uMjIwwPj7O3r172bp1K4lEou7F\nw6AXINWh2TpeFiebKewPLpFIss+YDx06ZADG4OBgeSbrZ3vXSjNhv1vE1hJEXlyzdm/wMONuprB/\nfpHIc0t7+AmSlQJ00AuQQaRngv4waVcocIu0VrVZpdcZp58gWSlAR/GQBFWzeKPALdJCY2NjRk9P\nT8VZpbNcMIiUQRQDdCVxGmuYvARuLU6KBGTLli3Mz88D0NPTw+zs7LJFQWtRsru7mxs3bgBa6JOV\nmrmtq4g4LC0tlW/v379/RSWHtZPfQw89BEB3dzeFQqFcVeL1sGC/hwqL+BH2bxwiLWVViKxevdpI\nJpPGoUOHXNMChUJhWWellVLxuninRb72hodUiWbcIgE5efIk6XSa27dvUywWmZqact0XO5lMsm/f\nPmD5/tte9+Sud+9uaR8K3CJ1cqYs7AEZYNOmTVy5csU1peF2cK/Xw3x16K9ocVKkTm7dhMVikaee\neopVq1axsLDA2bNny99PJpOeuxm19Wrn0uKkSANqLQJaKYt0Os2VK1cYGBjgscce46c//Snj4+Mk\nEgngTkrDz7mS9Z5BKdKosHP8Ip65NcfYFwH7+vpWLDRadcn79+9ftoUrpUVDZ92ynwYUNat0LtSA\nI1KdFbBTqdSKSg0reFpfvb29rk0z1vOs5ptKwbZWA4r9w+Pxxx83ent7K1amSPtCgVukOvus2hl0\nC4WC0dfXZwBGd3d3xRI8KyA3GmwrHcSgkr/OgsoBRaqz8tSDg4Nks9lllRrJZJJ33nlnWdOMWwme\ndZDBBx98wMLCQrkM0G+jjL3Mb8+ePRXfT6SZwv7gEqnJ6/4ZXp5npUzS6bSxf/9+1/SL1/fQvh6d\niwD2KnkW+GXgKvBg6bGNwAvAdmAW+FeA23SiNAaRzvDkk0/ywgsvcPPmTT766KPy40NDQ6q5Fs+C\nKAccBx5xPPYl4DTwceD/lO6LdLy5uTmWlpbKQXv16tUMDw8raEvgagXu7wAFx2O/AjxXuv0cMBL0\noETiyMpRW27fvs0999yjoC2Bq2dx8p8BH5Zuf1i6L9IxKi06TkxMkM1m2bx5M1B9YdHvwqV2BBS7\nNQ2+vmoi/dixY+XbmUyGTCbT4NuJhMPegr64uFhuZc/lcuX9tJPJJKdOnaJYLJLL5Th+/HjF2bbV\nGem8RiV+ny/xkc/nyefzgV93B3DBdv/7QF/p9pbSfTdhL85KB2r0QFovh+9atd32mm+v72s9z6rT\n9toZqU7KzkFADTjOwP3HwL8r3f4S8IcK3BIV1faq9hJcax2+m06njU996lNGX1+fMTs76+l9K12/\nv7/fcxBWeWDn8BK4a+W4nwe+C9wPvA88XQrUnwVmgM9UCdwiLVdtr2r7xk0f+9jHXPPFzo2jhoeH\nefLJJ1lcXGT9+vUsLS3x+uuvMz8/z+HDhz29r9v1u7u72bVrl+ef68iRI1y9epXPfe5zynFLU4X9\nwSUdqNrM1Jo1e2lft28ctWrVqhWbSKVSqbpOVS8UCkZvb6/vdnadetM50F4lIndYwdU6YswtR93f\n32/s379/2V4hzq9UKrUsTeJXPfnqenPcjeb8pfVQ4JZONzY2ZvT19RmpVKq8+ZPb7Ni52RQuM+0H\nH3zQGBkZ8b0g6eRldu68dr05bs3U4wcFbul0zoBcKXg5t2bt7u4u7zXyiU98wshmsyuCZjODYlDX\nVjVK/KDdAaXT2bsZBwcHXRcOc7kci4uL9PX18eqrr5JOp7lx4waFQoH+/n7OnDnDqVOnVtRkN/PQ\n3qCurfMp25POnJS2ZDXMrF27lu9///ssLCxw9913s2/fPk6ePEkymSw/Z3p6mkLB3NlhdHSUGzdu\nMDk5WXNzqGKxyN69e9m6dSuJRCLQsyG9NPFIe/KyyZQCt7Ql+0G+vb29LCwslL9nHexrfw7AmjVr\n6O7uZvfu3aRSKcbHx2sGTbcDg0UaocOCpWPZUw27d+8uP37vvffyta99bdlzBgcHWbduHbdu3aJY\nLJYDcaWgbe0bMjAwwPnz58vX0IEH0g7CzfBLW/FbweE8lGDdunXlxb6RkZEVz7EfegAYmzdv9tRd\naX1ls9lAf17pXKiqRNrB2NhYudqDOqosxsbGjNWrV5dfv23bthUfAlZtNz4qUBKJhCo2JHCoqkTa\nwczMDNevXwcglUr5TknMzMxw+/ZtAHp6ehgYGCi3vudyOQBOnjzJyMhIeUtWe8u7s8W8t7eXdDrt\nek6lSCsocEtkVNpz2spFp1Ipzp07tyxIVnqN/fG1a9cC0NXVxa5du7h48SKwvNQumUzy0ksv8cgj\nj5BOp7l16xZnz55dFtwtc3NzXLt2jTNnztDV1aWgLS3X6H7cIr7Z97a2l9BV2nO6t7eX3t5edu/e\nTU9Pz7JrVXqN/fFt27aVFx9ff/11APr7+11nylZQtnR3d1MoFCgWi+XnNrN+W8QLzbil5ey79Nln\ns5UC4tzcHAsLC0xNTa2Y/VZ6jX0XvuvXr7O0tFROl6RSKS5cuOA6U7ZXmmzcuJEbN24wNTXFAw88\nUJ7RWx8kmmlLOwo7xy8RVakNu9J+HM7n2ytMZmdnXV/j3IXP+urp6TFmZ2dXbCrltieI9b44Fiu1\n/4c0E6oqkShyC9DVyv2cz/caOK3Au3HjRmPjxo3G5s2by7v6uZX0uW3x6nbajf2D5IknntDuexIo\nFLglLvzMYt1m7G6B321vbevazk2lnLN/63qHDh1ascGU/YNEs28JGgrc0kxB7vXsZxe7WtuyOgOo\n27Wta1RKtfid1auWW4KCh8CtvUqkbkHu01HPpkr26pSbN28yNTXlujGUde3169czNze3oprFfq1L\nly6xfft2Ll68yLVr1zxtNKXNoCRIXvYqaaawP7ikyfzMNhuZnXs5eX1kZKTmQQPVZtFuOe+urq7y\n4QtB/BwiXqBUiTSTn1NZGskF1zp5vdoHhz3QVjuyzDqqzGpjdzuXstHWexEvUOCWsDkDYz254P7+\n/mWlfBYvHxxWVQhgDA8PV82N9/f3l3PebkHe/lz7YcGahUuQUOCWsDkDYz3Bzq0qpBLnde27/lm7\nAtr5qSm3nus8LFiVJRIkFLglDLXSE4ZRf/lfrbpp+3V37txpJJNJA8xzI92e7yfd47VBSKQRNDlw\nfxl4C7gATADrFLjFMJYHz2w2Ww52bgE9nU4v61x046du2gqi6XS6nK+2xtEshULB2LlzZ82fQ8SL\nZgbuHcC7tmD9AvCkArcYRuUZqFsVSLU0iFs6xbp2d3d3ueLDrQXefl2/+eh6ctZKl0hQmhm4NwIX\ngRTmDoMvA4cUuMUwaqcU7LNsa+ZtD8QWt2BYKBTKC53W427Ps+ejH3/88XIg3rx5c/m5lU65qScI\nK10iQWlm4AbIAT8GrgJ/4fL9sH9+iRi3FvRsNrtsMygv5X7Ox6t1RjrTK2vXrl1Wq93X1xdIztpP\nrlykmmYG7vuAt4FNmDPul4DfdAbuo0ePlr9eeeWVsP97SMgqlQb63S3Q+XitoGm/vtfNpRSEpVVe\neeWVZbGymYH714Fv2O5/HvhTZ+AWsatUGug3UDZ6cHA2my2nTJTakKhpZuDeDbwJrMfsqX8O+G0F\nbqnGWdbX19dnpFKpZYuMzsecgupe1KxaospL4G5kI5MjmJUkHwF/C/wWcNMRuBu4vLQb+2ZP3/72\nt8sHAIO5SdXVq1fLm1ZZjzk3rtqyZQvz8/OAeU7ke++9p82dpK142WSqkaPL/hj4OeBBzAB+s/rT\npR1UOpzXi2QyyYsvvsjc3NyyoD04OMjx48fLx4bZH3NaWloq33744YcDC9qN/Fwi7STs3zikCYKo\nV7ZSJslk0hgeHl6W685ms8bIyMiyumt7+sR6/8HBwYbSHM48ueqwJSpocqrES+Bu4uUlDAMDA1y+\nfJmenh7Onz/P9u3bfV/Dzx7W9tQIQDabpaurq+H9r517id+4cYPJycma+2+LNJv245bA+dnwyatq\nVSL2TaJ6enoqzsT9zr6dJYharJSoQJtMSdCa0SFYLU2xdetWAzDWrFljnD9/3vU1bq+rRYFaogoP\ngbuRxUnpQBMTE4yOjgaaTrAWJYeGhlYsSO7cuROAW7du8dWvfnXFawD27NnjupBZjbVQ6vYzaKFS\nok45bgldtZy3lVNfvXo1Bw4c4NSpUySTSYrFIk8//TSGYXDixIlAc9JBnqUp4peXHLcCt7SU/YBf\n54G9bg4cOMDZs2fL91sRSIeHh7VQKaHxErjXtGYoEnf2gNvb21vxtPRaZmZmyrPZXC5XMwgnEony\n7Uq13UGbmJjQye0SaZpxiyf29EE6nebatWuA/xmw39lssVjkqaeeYtWqVYyPjyuQSttTqkQCYw+4\nyWSSqampulIJfmq4RTqRArcExh5wgZrB12su22/Ou5KgriMSNjXgSFX1HNHl9bVeW8iDajVXy7q0\nCzzUcWtxsoPZFwr37t3Ltm3bPM9Yay0yVqvNrud5lkoza7/XERF3YX9wSQ32Lki/rey1Oii9dib6\n7WC0z6ztx46pE1LaBWp5l2rswc5vK3tYgdIaJ0qLSJvyEri1OClA49Ueu3bt4t1338UwDPbv31/u\ncLQEtXhYLBZ54IEHmJ+fV4OMtCVVlUhVQVZidHV1cfPmnbM0nPXdQbaRq6RQ2pkCt1RVbzB1C/h3\n3XUX1t93IpFgbm5uWVB1a7xxu47K+qTTqeVdqqq3EsOtoiSRSJSPIztw4MCKgGu1ka9fv56RkRE2\nbNjA4uJieR8S6zp+W+JFJFjhZffFEy8LjG4HFrgtZB46dMgAjD179lS9nrMqxHmdZuz3LRInqKpE\n6mU12HR1da2o4nAL+PbHqjXn2APz7Ozsius88cQTRjqdrutUG5F2oMAtdXOeMEPpcF8vwbRaF2Ot\nWb46IKXToc5JqZeV/7733nv58Y9/TDKZ5I033qh4Yox9y9fp6WnA/WQa6+SZWu+rDkiR5kgC3wTe\nAd4GHnJ8P+wPrrbVyB4jXlkzY7d0hpN9lpxOp8u3R0ZGqr6H28+hDkjpdDQ5VfIc8IXS7TVAjwJ3\nMILawKlV7Hlra5HSy+Ji1H4OkShoZuDuAd6t8Zywf/7YqhXQolZ5YZ8l+5kxR+3nEIkCL4G73gac\nPcB/xUyR7Ab+Bvgi8BNH4K7z8p2t1ikx7dI52C4/h0iQmtk5OQT8X+Bh4HvA14FF4Cu25xhHjx4t\n38lkMmQymTrfrrMooIl0jnw+Tz6fL99/5plnoEmBuw8zcO8s3T8AfAl4zPYczbhjIMgWc7WrizSu\nmS3v88D7wMeBGeAQ8Fad15KQWC3lVqt6oy3mjRzMICLeNVLH/TvAXwJdwCXg6UBGJC0zMzNTDtqp\nVKrhuml7Dfa6deu054hIk9zVwGvPA/swFyd/FbgeyIik6XK5HJlMhrfeMn9JSqVSnDt3ruFZ8cTE\nBKOjo5w+fZpEIgGokUakGbStaweyb+fa39/PhQsXAk9laIFVpD5ectyNzLglIqwZ9PDwMMVisebz\n7SmNZgRtuNParqAtEjwF7jZgLQpOTk6Sy+XKj1cK6PaUhgKrSPxok6k2UGljpkqHEtTa6ElEok05\n7jZQKZ88MDDA5cuXSSQSTE9Ps3379hBHKSJeKMfdISrlk61Avbi4yOHDhwH/+XARiR6lSlokjK5C\nqyQvnU5z5coVhoeHXc95FJF4UaqkReo9Ub0RVgrlypUr5WDd19fH/Px8xQ2sRCRcSpWEzJ6WWLt2\nLdDahhQrhWJvhnnttddUUSISc5pxN5F9lp3NZunq6gqlIUXNMCLx0cxNpsQDe5neiRMnfAXNRnPi\nztcrly3SPjTjbqJGZrqN5sS3bNnC/Pw8YM72T5065ev1IhIOzbhD1kijS6OnnS8tLZVvl/5HEJE2\nocXJiGq0Lf2Tn/wkAIODg4yPjwc9vDLVhYu0nlIlbapVC5JhlDmKtDOlSjpYq/YjaTSlIyL+acbd\nAZrZtalSQ5FgNfOUdy8UuCNC6QyR+FDnpABKZ4i0G824O4DSGSLxoVSJiEjMKFUiItKGFLjbgJpg\nRDpLo4F7NXAOeDmAsUidKh0WLCLtqdHA/UXgbUDJ7BCpakSkszQSuPuBYeAbNHeRU2podF8TEYmX\nRgLuSeAPgATwe8C/dHxfVSUiIj41c6+Sx4CrmPntTKUnHTt2rHw7k8mQyVR8qohIR8rn8+TzeV+v\nqXfG/QfA54FbwN2Ys+6/Ap6wPUczbhERn1rVgHMQpUpERALRygYcRWgRkRZRy7uISISo5V1EpA0p\ncIuIxIwCt4hIzChwi4jEjAK3iEjMKHCLiMSMAreISMx0RODWQQMi0k46InDroAERaScdEbh10ICI\ntJOOaHkvFovkcjmOHz+ugwZEJNJatTtgJZEJ3CIicaG9SkRE2pACt4hIzChwi4jEjAK3iEjMKHCL\niMSMAreISMwocIuIxIwCt4hIzChwi4jEjAK3iEjMNBK4B4BXgLeAN4F/HciIRESkqkYC903g3wA/\nBzwE/DbwQBCDaqV8Ph/2EDzROIMVh3HGYYygcYahkcA9D7xRun0DeAfY2vCIWiwuf5kaZ7DiMM44\njBE0zjAElePeAQwCrwd0PRERqSCIwN0NfBP4IubMW0REmqjR/bjXAv8LmAS+7vjeD4H7Gry+iEin\nuQT8bLMuvgr4c+BPmvUGIiISrAPAR5gLlOdKX4+EOiIRERERkU7wLPAhcCHsgdQQlwaiuzGrdd4A\n3gb+Y7jDqWo15m9eL4c9kCpmgWnMcf6/cIdSVRJz0f8dzL/3h8Idjqv7ufPb9jngOtH9d/RlzH/r\nF4AJYF24w6noi5hjfLN0u2U+jVkaGPXA3QfsKd3uBi4S3QaiDaU/1wCvYaapoujfAn8J/M+wB1LF\ne8DGsAfhwXPAF0q31wA9IY7Fi7uADzAnRFGzA3iXO8H6BeDJ0EZT2T/HjJt3Y06CTlOhwKMZe5V8\nByg04bpBi1MD0U9Kf3Zh/oX+KMSxVNIPDAPfoPFqpWaL+vh6MCdAz5bu38KczUbZIcxqiPfDHoiL\nRcxO7w2YH4IbgL8PdUTudmH+dv2PwG3gDPCrbk/UJlOmHUS7geguzA+ZDzHTO2+HOxxXfwIcxlyw\njjIDmAL+PzAW8lgq2QksAOPA3wL/jTu/dUXV45gpiCj6EfCfgL8DrgBFzP8HouZNzA/sjZh/37+M\nOSFqmR1EP1Vi6cb8RzwS9kA86MFMlWRCHofTY8Cflm5niHaOe0vpz17MD8NPhziWSoYwZ4j7Sve/\nDvyH8IZTUxfmB01v2AOp4D7Myc4mzBn3S8Bvhjqiyr6AGY/OAH9GhXLrTp9xrwX+CvjvwKmQx+LF\ndeB/Y/7DjpKHgV/BzB8/D3wGs8Y/ij4o/bmA+Q/450McSyWXS1/fK93/JrA3vOHU9CjwN5j/TaNo\nCPgu8A+YaadvYf4/G0XPYo73IOZvBhdb+eY7iP6MOy4NRGnMCgOA9cCrwC+GN5yaDhLdGfcG4N7S\n7XuAs8AvhTecql4FPl66fQz4o/CGUtP/IJqLfZbdmGmI9Zj/7p/D3M00ijaX/tyGue6WaNUbP4+Z\nR1rCXKh4ulVv7FNcGogexMxzvoFZxnY43OHUdJDoVpXsxPzv+AbmP+QvhzucqnZjzrjPY84Qo1pV\ncg9wjTsfiFF1hDvlgM9h/rYdRa9ijvMN4BdCHouIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIhIPP0T\nO73tZSEvXw8AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEKCAYAAAAFJbKyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXl8FOX9+N+bk4SQQDjlkHDIfYOiHIp4cQkqiEIrFX+l\n2lbtt7VWoeVQq1WqrWKrtl6ttSCXIFAvFKIgitwgGI5wRG4CISE3Seb3x2cnO7s7uzt7ZTfheb9e\n89qd2Tk+Ozv7fJ7ncz2gUCgUCoVCoVAoFAqFQqFQKBQKhUKhUCgUCoVCoVAoFAqFQqFQKBQKhUKh\nUCgUCoVCoVAoFAqFIrqYCqwwrO8HFhnWfwB61ahECoVCoYhq2gF59vctgcNAjn29PXAuAjIpFIo6\nRFykBVCEnEPABaAv0Bn4BOhtfz8I+DJyoikUirpAbKQFUISFfkAiojzWAaVAKjAE+Bb4JnKiKRQK\nhSIa+Sni59iJmKtGAfOBg4hSUSgUCoXCiSsQc9U++3oq4ts4D9giJZRCoagbKB9H3WQ/ojjW2dcL\ngGzgNKBFSiiFQqEIF9OB3cAuxLySGFlxFAqvtAHWIs/sd8DDJvsMA/KBbfblDzUlnEJxKZCB2OF1\nZbEQ+EnEpFEofNMC6GN/nwLsBbq67DMM59wahaJWExNpAVwoAC4CyYgZLRk4FlGJFArvnAS2298X\nAt8jAQmuKN+SQhFGfobY508D/4mwLAqFP2QAR5CRh5HrgLPADuBDoFvNiqVQ1G06AHuAxsiIYxnw\no4hKpFBYIwXYDNxm8lkDZPQMMBJHtJtCoQgBdwFvGNbvAf5u3KFDhw4aEhmkFrWEYzmA/8QjGfr/\nZ3H/Q0C660b1bKslzEsgz7Yp0ebjyAKuBpIQm/CNyAikmuzsbDRNC2qZPXu2OkcdPIe34z/44AM6\ndepE27Zt0TSNrVu3cuutt7rth4x6/cEGvGl/Tl/0sE9zHD6Oq+zv3WqGheLZDscSit9WyRX5JYBn\n2yPRpjh2AO8gQ/6d9m3/jJw4itrGot2L+P3nv3fbPmfOHDZu3EijRo0A6Nu3LwcPHgzFJW9HRsYP\nACXAUcQcdb99AZgA5AJlSOjurFBcWKGIFNGmODoDk4ByoAKxF/88ohIpahUvfP0Cz6x/xm17fHw8\nDRs2dNoWExOSx38DUhMsCWgKFCEVif9hX0BMUxuRMPPhwC8CuVBpRSlf5XwVkJBVWhXlleU+9/tw\n/4dUaVU+97tYeZGi8qLqc/va14imaZwuOu31mPOl533KEErKK8upqKqwtG9FVYXegwfkd6msqgyX\naJbRRxb6b22UMdREW+b4XuRPCKLUjiEO8pAybNgwdY46eI5hw4axat8qQP5ENpsjArZ79+7897//\npaKigv379zNv3jwGDRoU1PXsnLQv4ByO+71hn7HAv+3vNwINEfPVKdeTLdi1gAEtB7D5+GZubH8j\nnx38TE7QeSy7Tu8iJz+HwgOFXJdxHdnnsunSpAs2m43V2auJj41H0zRSE1NJT0qnrLKMdg3bAZCV\nm0VWbhbdm3UnKS6J3OJcUhNTKa8sZ+/ZvfRu3puCsgLyS/PZfXo3353+joTYBOJi4ug6oCu7Tu2i\nUqukT4s+aJpG5uFMcotzq+W+o+sdbD2xld4telNZVcnFqoucLDxJ/fj6bPhhA7d1uY16cfWw2Wwc\nu3CMdUfWMaLjCMory2mQ2ICE2AQW717MwNYDycrNIr80nzZpbfgh/wdu6XgLeSV5NKzXkE+zPwUg\nOT6ZQUMHcaHsApVaJYfPHyYrN8utsZzUcxI7Tu6Q+1ZeCMB1GdfRrH4zPs3+lMvTLqdrk64s3bO0\n+phhGcPYdXoXl6ddTuOkxuw+s5sTF04A0KdFH7aflOjrLk26kJWbBUDbhm25Iv0KkuOT6TqgKwfz\nDtKqQSs2Hd9UfV8HXz6YHSd3UFheyI3tb6RBYgOKyov4NPtThlw+hO9zv+ds8Vm6Nu1KywYt2Xx8\nM83rN+dE4Qk0TaNjekfyy/Lp2awnCbEJlFSUcPj8YXaf3s0N7W/g84Ofuz2cKQkpVGlVFF8s9vYM\n+000x5bfjAzph7hs18KpSRW1m/YvtefQ+UOc+905GiU1qt5eVFTE008/zaefSsNzyy23MHPmTOrV\nq+d0vF3ZBPq/yAC+ALojSkRnJfAnZHQC8BnwGLDF5Xht/s75AV5aofDO5F6TIURtfrSNOIzcjZQc\nUSgsoWkaJwtP0rx+c86WnHVSHPXr1+eZZ57hmWfczVghIgVYAvwKZ6Wh4/qHVb0fRa0lWhVHAnAr\n0itTKCxRdFFs7q1SW7nZyK+//nq3/W02G2vWrAnFpeOBpcC7wHKTz48hNa10WuOhIsKHr39YbVa4\nc/SddOjbgQPnvEdRJsQmkJKQwrkS35M7dkzv6PN8/tC0flPOFJ0J2flcuanDTazOXu20rWWDlvRo\n1qPabGWkbcO2HDl/JKhrNqzXkCqtioKygoCOT01M9Xrs5WmXk5Of47Qtxhbj5iu6stWVbDq2qXr9\nuozr+OLwF27n69OiD1VaFTtP7azeNq7LOD74+AOWfLgEgPQkt+jvoIhWU9U4xCk+wuQzbfbs2dUr\nw4YNC4l9XVH7OZh3kOH/Hk77Ru2ZMXQGN7a/sfqzzZs3V78vLS1l6dKlxMXFMXr0aDIzM6s/e+KJ\nJ8C//4UN8V+cBX7tYZ9RwIP216uRsN2rTfbTNE2jsqqS2BjzOdbKK8s5VXiKNmltqNKqOFt8lqb1\nm5ruu2DXAm5ofwNNkpuQW5xLQVkBHdM7UqVVEWOLqXZax8fGU1lVyfnS86QnpaOhVX8eHxsPUO1j\nyCvNo1G9Rmw+vpnL0y6neUpzUxkvlF0gOT6Z5VmiRyf1nATg9N10P1R5ZTkJsQmUVZQRFxNH8cVi\nii4WEWuLpWn9plRUVRAX497HLSovwmazkRyfTEFZAVtPbOXq1ldTL64elVWVnCk+Q4uUFtXXqqiq\noEqrIiE2gfLKciq1SiqqKqioqiAlIYW4mDhibM4BE2UVZZwpPsO6I+uqv0NheSHHCo7RrlE74mPi\nsdls1U7pbSe30aVJF1ISXIsHuPvdKqsqKbpYRGpiqsf9Dp8/TOOkxjRIbODxPMbtQPXv57p/kGZY\nJ6JVcbwHfITDoWhE+TgUpmw8upGHPnqIVqmtmNJrCrd3vd3r/ldeeSWbNm1y2hbAn2sIMh3vThzm\npxnA5fb3emTV35COUBEwFdhqcq4692wXlReRFJ/k1iArap5QKo5wmaq2AG8hPoo8P4+tjyT+TQu1\nUIq6zZniMzSt39TUVHDunMOMU1VVxebNmykoCMwU4cJ64F/AaKS+Wk+TfYYhuR564sgozBVHnaN+\nQv1Ii6AIA+FSHHcjvapNSDLf28CnWHMIFgFNwiSXog5zuug0zeo3o358fS6UX3D6rF+/ftXD+7i4\nODIyMnjzzTdDdem3gZeR5FVPfIGE5SoUtZ5wKY79yHD9D8AYZPRRZX99CZNyCwYaIvWquiOK5j7g\nmzDJqahDnCk6Q9PkpsTYYrhQ5qw4Dh8+HM5Lr0NCcb0RrWZhhcJvwhlV1RsZdYxEIk7mI/bgNTgm\nvjHjJaT09AS7fGqsq7BEbnEuTZKbUFFVUW2qWrp0qakjUeeOO+6oCdE0YBBSUucY8FtcarApFLWJ\ncPo48pGRw2NIjR6QkcNgL8elAUNxzPpXYT+PQuGTcyXn6NS4EyUVJZwqkqTslStXRoPi2IqE4xYj\nHanlQCezHefMmVP9XkUMKoIhMzPTKWIwlIRr+NwehyPQH/ogUSh7kBHLFiShypgvX+ciTxSh4Y6F\ndzC552QKygpYl7OOt8e97fc5gog8yUAyxM2c464cAvrjbrJVz7YibNSGqKqfAnMBPQurEfAI4vPw\nJU8/JOZ9ExLv/jgu1URVr0xhxvnS89XJW3pdIiOrVq1iz549lJaWVm+79tprw9YrM9AcibjS8FJW\nXaGoLYRLcYxCnOM6eUi4oi/FcdS+6MH1SxDF4YRRcSgUOvll+TSs15DyynI3xXH//fdTUlLCmjVr\nmDZtGosXL2bgwIFuHQ97AqC/ZCMjDhvwAzAbySYHGUFPAJ5EypJUoCo+K2o54crKiQGM1eOSkDIi\nvjiJ/PF0+++NwO7QiqaoqxSUFZCamEqDhAZuUVUbNmzgnXfeIT09ndmzZ/PNN9+wd+/eUF36XmAA\n8qy2QaIHQ15WXaGIFsI14vgv8DnyB7Ih0VXeYtyNPGQ/PgHpyU0Nh4CKuoeuOIrKi9xGHElJSQAk\nJydz7NgxGjduzMmTJ81OEwi+wnEtl1VXKGoD4VIczyElGG5E7LpPInMyW+EDoACoRIrBqagqhSUK\nygpokNCAwsRCtwTAMWPGkJeXx6OPPkr//v0BmDatxooTtEJG0jpHkWdbKQ5FrSSceRwf2Rd/0ZAS\nDcp5qLBMRVUFFysvkhyfTEpCSvXsdDqzZkl8xfjx4xk9ejSlpaVuMwKGGUtl1VXghyJU1MZw3PHA\ns8hwXL+GBqR6PMLBIcRefNbD5ypkUeFGXkke7V5qx/nHz1NYXkiL51tQOMNhrurVqxd33303d911\nFx06dPB4njCF474GZCLFOwGygOtwH3GoZ1sRNkIZjhsu5/hcxK6bCjSwL1aUBoiC+QypcaUKHSos\ncaH8QnXp6eT4ZIovFjvNb7BixQpiY2OZOHEiAwYM4PnnnycnJ8fT6ULNCmCK/f3VSJi6MlMpai3h\nMlWdxHnOZX8YDJwAmgKrkd7ZOuMOajivcOVC2QUaJIjiiLHFVCsPfV6EjIwMHnvsMR577DH279/P\nU089xWOPPcbnn38eiuF8JvLcxiFK4Tc4h+MWAzcgFRT0mm0KRa0lXIpjM7AQKa1Qbt+mAe9bOPaE\n/fUMsAxJmPKoOBQKcB5xAKQkpHCh7ILThDqHDx9m4cKFLFq0iNjYWObOnRuKPI5YxNF9BVKHahPw\nNe4dp09Q1XEVdYRwKY40oAS42WW7L8WRjPwRLyDFDW8GAsrIUlxaFJQVkJaYVr2ekpBSPZUswMCB\nAykvL2fixIksXryY9u3bh+rSVwEHgMP29feQGSxdFYeqjquoM4RLcdwb4HHNkVEGiGz/RebxUCi8\nUlBW4DbiMOZy/Pvf/6ZLly7huLRZqO1Al31UdVxFnSJczvHOSAKgnvXdC9/lRkAiqvogBeAuIn82\nhcInevKfToNE5+zxMCkNsDY5mV4dtzcy4dPycAmjUNQE4RpxvA48ioQhAuwCFgB/tHj8r5AeWQNf\nOyoUAPml+aQmOBRHSkKKWxJgmDiGKAWdNsiow4hRkI+AV4B0THKVVOCHIlSEM48jXIojGSmtoKMh\nIwgrtEaKJD6NRKcoFD7JL8snrZ7Dx2FWrypMbEYc4xnAceAuYJLLPpar46rAD0WoCFEBT1PCZao6\nA3Q0rE/AES3li78io5UqXzsqFDquzvEGCQ2cRhxFRUU89dRT1WVG9u/fz6pVq0Jx6QpkGoBPkFHy\nQsQxfr99AXn+dwHbkakC7g7FhRWKSBGuEceDwD+BLkgv7BDwIwvHjUF6ZtuQsiOmqOG8wpX80ny6\nNe1Wvd4gsUH19LEAU6dOpX///mzYsAGAli1bMmHCBFJSUkIxnNcMi97h+Yfh878jfr+RyGijDIWi\nFhMuxZGNJDzVR0Y1Vm0Gg5BY91FIWfZUpKruFONOajivcKWg3Nk5npqY6mSqys7OZtGiRbz3nlT9\nqF9fprIPUR7H35CCnnoexwqcw3FHISPwK5CIq1eRDHKFolYSLsUxG+l92XCOOnnSx3EzcEwAdR0S\ntjjF8+4KhZBfmu9kqkpLTOPYhWPV64mJiZSUlFSvZ2dnk5iYGIpLW8njUGXVFXWKcPk4iuxLITJ0\nH4X3+Qo8oSq+KSzhGo6bVi+N/FJHRf45c+YwYsQIjh49yuTJkxk+fDjPPfdcKC5tlsfRysI+rUNx\ncYUiEoRrxPG8y/qfsZbIVw/4ApkpLQGZm0Oh8Imb4khM43zZ+er1m2++mX79+vHNN98AMG/ePJo0\naRKKS1vt3Kiy6ooapTaG47pSH/demBmlwPVIUbg4YD0wxP6qUHjEVXE0rNeQ/NJ8tmzZopeTBuCy\nyy4DICcnh5ycHPr16xfspa3kcbju09q+zQ3lv1OEinCG44ZLcewyvI8BmuHbv6FTbH9NQByPakIn\nhU9cixw2rNeQvNI8HnnkESfF4cratWuDvbSVPI4VSKThe6iy6oo6QLgUx62G9xXIn8RqAmAMUqKh\nAxJ9omr6KLyiaRqF5YVOlXCrRxyZW8J56XQkbyMRKa9zCqmaoOdxgITlvgKkIGG4lcgoWqGotYRL\ncRS4rLuWDvE2iqhC6lWlIUlVw5D5DhQKU4ovFpMYm0hcjONxbpTUiLzSvOr1kpISXnnlFdavX4/N\nZmPo0KH8/Oc/p169esFc+nFkzpibgMeARsCf7J8Z8zg0oBOX4Oj54kWIiwMvgz5FLSRcimMrcDmg\n/3MbATk4kqSs1LTOB/6HTCObafxAORAVRlzNVCDO8fzSfDRNw2azMWXKFFJTU3n44YfRNI358+dz\nzz338Mtf/jIYB+JYJGwcJNw2E1EmZtR403n6NDRpAjHhip20wJIl8jrJ1XinUJjwOhKCqzMSyST3\nRRMkxh0gCfgSSSQ0oikURvbl7tM6vNTBbXvKMylafmm+pmma1rVrV7fPzbbhXwh4nuG9zWXdyEGk\nGoKv6ZBDel/mz9e0I0dCesqAZJg/v2autXGjppWV1cy1PHHmjKZVVERWBk/4+Wx7JVwjjmtw/oN8\nhITk+uIypOcWY1/+g5RnVyg8YjbiAGhUrxF5JXmkJqbSr18/vv76a6655hoAvvnmG/r372/l9KuB\nFibbf++y7u2P6XM6ZJ1Qj6a1SygTKjsb2rQBe+BcyLh4UUZO+qipogJWr4aRI933Xb0a+vSBrl1D\nc+3CQqhXT8x9/lIbw3GPI/NvvIv0xCbjIfzQhV1I1u07SCTWTxCH4rzwiKmoCxjnGzeSnpTO2ZKz\ntG3Yls2bNzN48GDatGmDzWYjJyeHzp0707NnT2w2Gzt37vR0+pu8XPoUolROIp2e0x728zkdso6v\ncNyCAvjoI7jrLsjLgy++gNtu83pIyKiqgoMHoUMHWLoUunWTxZXTp2HdOrjiisCuU1AA5eWQlAT2\nyjCWsaooS0qgshJSUnzvu8UlvqKkBM6fN98X5D6FipUroWNHuPJK/4+tjeG4k5CyI/psfl/iHqLo\niYvAr5FKoinAFqSX5joVp0IBeB5xpCelk1ci1qOPP/44HJdegXRunrO/mk3QFNLpkPPzHQ3TuXPS\niHlj925o2xa2b4djx2DUKNi3Dzp3htxc8YGANNS5uZCWJg23Wa/97FnYtEkc3RcvyvnMFMf69XK+\n/fvdP9u2Dfr29S7z//7neO/NN3LmjMjSpInIY6SyUuRt1sz82DVr5Ht27QplZTJKiIuD2FjHPqWl\nsGwZNG3qfOyRI97lDzWu383TPp9+Kr9rdjbccotsLyyEhARZQkm43GZngYeRsMO+yMRMViNKTiJK\nA6RkyfdAy1ALqKg7uJZU1zFGVmVkZJCWlkZBQQHnzp2rXjIyMsjIyAj00t8DMxET1TjgWfv2lkhg\nB8iIZDuS3HoOiTj0WEWhqgpO2Mcn5eVQ5Jg2nfx8aZStoDe++fnw/fdw6pQ0lGVlsHUrFBeLWWXB\nAjh8GPbuldHL5s2QmSn7nTnj6MEXFzvOXV7ufC1NE/ONTpm99m9lpbtcWVlyrkWLpGEuK5NXEFnM\nRgxm5wH47DNRAOBwwuvHHzoEn9uN3KdPwwf2GhTffScNvy5vdraMot5/33lkoWlyv8zYtcuxz4ED\ncPKkyL5ggbMMvsjPl2Mu+DFtzKZN0gnQNJFXl7GwUN4fPy4dCp2VK2V0mOfJ+xYg4RpxDALeQMJw\n9Skz7wd+4ed5MhDFs9HHfopLmIKyAnNTVb10zpXIv2jmzJn861//on379sQYwoyCTAD8GuiHhN4+\ngiT2gZhqR9vfH0EUSxcc1XO74mEEvWKFjCImTRJzz2m78euaa9wbMmOIq6bB0aPQsqUoAeO+27c7\nIqtycuT1qCG3/cAB9575++873sfHS49WT7Lfvt15X3vBYRo1ct5ubPCXLYMh9uyVr76Sz5Ytc3x+\n443yusuYOmxn0SJo2FBGKps2wa2GLDFPYb7GxvvYMVFWO3bAHntWWHKy+/HFxdKYV1bCJ584tp85\nY36NnTsd5zO7tq5Qt26VhnvECNn+3XdyP7OyZH3NGhg3Tn73gwflNz950ny0deCAvLZsKSPHvDy5\nd/r3MH5vo4Lfts38OwRKuBTHi8AIHLWmduAIW7RKCrAEGa0Uhk40RV0jv9R59j+dRkmNqk1VCxcu\nJDs7m4TQjtmzLOxjpXpuNbrpacEC54b466+d96uqgo327tRXX4nJxdtoRDdv6aMBY+/6zBlItVdr\nMevd66YS15FGbi4YLYDeerWlpTJC0N+7on+/3bvNjz9/HnQdX1HhcBZXVTl6+vr64cMycgLnz8wa\neSMnTjhGe2YsWABXG4rhezpfRYUoqy+/lPXYWLmvCxaI2dDV1FVcLIrCtQ+jy3LkiHz/UYY4Vb1j\ncOaMfGddcRw/7tjnA0OlP7N7HgzhrFWV47JeYbqXOfHAUsS57mY3VnkcCiP5ZfmmpirdOQ7QvXt3\n8vLyaN68udM+4Yw8sWNWGXeglQO9Jc0tXOh4n5Pj2aziynffmW/PzpbXU14KoZgdG4gJpNCkG2g0\nyRk5Z2LgXrwY7rhD3rs6onftklGDL3TTW5mfU2rZa2R6JSvLMZoAZ2XsyT9iNvA1dgTy851HPps2\nOd6fPy/+HCOu5kMr98QfwqU4cpAQRJCaUw9j3bltA95ESo28aLaDKgSnMHK+9DytGrjX0ExPSif7\nnLSIM2bMoG/fvvTo0aN6Hg6bzcaKFSt8RZ54CsedAay0IF7AAbFmjaYnvEX51Gb0UYorniyMoW4g\nI0mFS1fb070wmtV0Pvoo9PIYCZfieAAJoW2F2HU/BX5p8djBwI+BnUjSFMB0ICxhMYraT15pHg3r\nNXTb3jipcfWIY8qUKTz++OP06NGj2sfhrfihAW/huFawUj23miVL5lS/79ZtGN26DQvy8rUbT47x\nUDt76xr5+bBnTyZ79mSG5fzhUBxxwEtI7kYgrCd80V5eycmBJ56QSISyMhg8GB5+WELbVK2d6CWv\nJI9GSY3ctqcnOZzjKSkpPPzww+EUw9MTYqV6bjUTJswJtVyKSxTXjsf774cujyMcDXQF0BapGBoI\nbyGJVSbxFeFj+XIYMABatRLHYXa2RDX85jdw000SwaCITvJK80hPSnfb3jjZMeIYOnQo06dP5+uv\nv2br1q3VS5DcjvgvrkbCb3UDgTEctwIpqf4JYn5diMpJUtRywmWqOoiMHFbgmF9DA/5i4di3gZeR\n7PGwo2nwwgvw0kvw4YeiPHTuuUeUx8svw6BBMHMmPPRQZIvGKdw5V3LOVHE0SW7CmSLxKG7duhWb\nzVY9A6BOkOG4cUheRkukNpuuiYzhuCDTAxQgJdVvx1FBV6GolYRacfwHuAepGvpXZERjIanfiXUE\nNj+531RUwP/9nyQ+ff01tDaZBTouDn79axg7Fn70I3FEvfUWtDBzlyoiQm5xLk2S3aeBbZrclLMl\nZ6msqgxX5NQuRBH8w8d+GjI9wCVXVl1RNwm14uiP9L5ykFFD1HoGTp2CyZMlFX/9eim14I0OHSQh\n68knoVcvmDED7r9f6ukoIsfFyosUlheaOsfjY+NpWK8hucW5NE9pzqpVq9izZw+lhqD2WbNmBXN5\nK3kcOlH7X1Ao/CXUiuM1pJpte6TGlBGr83D4JNg8jlWr4Gc/g/vuE2e4sT6NN+Lj4amnpMDcH/4A\nzz4Ljz8ODz4YWPVKRfDkFueSnpROjM3cftg6tTU/FPzArEdmUVJSwpo1a5g2bRqLFy9m4MCBNZHH\nAfLsf4aYqv6BTDsQElJTZfRbUz64ceOcE8v8oXt3yXhevTq0MilqnlA3d/Psy2tISG5YCDSP49gx\n+O1vJeN2wQK4zt9cdjs9eogzfdcucZ7Pny9JSW3bBnY+ReCcLDxJ8/rNPX6e0TCDQ3mH2LBhA7t2\n7aJXr17Mnj2bRx55hBEjRlipIBpsHgf4UVbd33Dc3r0l69gXDRr4VxPJE3qpDqvUry/Jfb17Q7t2\nMkJv2NA976R3bykJ4g/x8XI+b8mPPXualzGJNPp9CYTRoyWR0FMyp044w3HD5eYNm9IIhFOnRGH0\n6gXt28sND1RpGOnZUypSTpwotYSsZJUqQsvxC8dpleqe/KdzRfoV7D+3nyS7TTE5OZljx44RFxfH\nSSstruRx9DRZrCoNMC+rbsqECXOql6lTh1Vvj4833791a98dlkmTzI+36qdzPb8vsy7IHBJ33imd\nLJAqurpZt3t3edV9irGxjiq9ZniqEtOtG9x8s/gfPdGpk/u2yy93VI8F+R/7c7w3rLYrsbEyn4en\n+pp6CRgjulUjNdWaibxbt2FOz1Moicb4oAXABmSO5h+AqYGeqKhITEp66eSdO+Hpp/3vNXnDZhOl\n9M9/SvG1N9+8tCbPiTQ5+Tm0SW3j8fOuTbqy58wexowZQ15eHo8++ij9+/cnIyODSaGdz9STDyMZ\nKfYJjrLqXvvA6fYAMb1RiYlxFAEE6bGPHQtDh8p606buRQpHjoQxY9wb3eHDpVxHQoJMeqSfwzj1\nuktVFgYNks7RhAmyPmqUe0M/caI0mhMmQP/+UpDQl/m2ZUv5LhMnOs7Xrp3zPs2aORTXTS6pmPHx\nsnibs8P1+6emSn6Wfo/btnUoN52B9oIwffrIXBhGbjDMR+qqQJOT5TsNHCgFIfV7a/xc5/rr5bvr\nsk+a5FA6PXo4Ss9nZJhHcTZr5vy9dWVcU0Sj4vg3ErqYA/wNCc/1mzVrpCdx8KAMgV9+WXI0gJDY\ntF3PMWaMRGf99a/S0/JWLM3TOUIhx6V2jn1n99ExvaPH47s3687uM7uZNWsWjRo1Yvz48Rw5coS9\ne/fy1FPhcwxKAAAgAElEQVRPBSkxy5H5Y64DvkLMUBBEWXVwKAm9wejVSxqZa6+V9ZEjpdEwRgHe\ncINzo9awoZinxo+Xdb0H27w5JCbK9o4dHedo1UqeYZBCfsYKtJmZmcTGmo9a9Hk7YmOl0YyPl166\nlUr1HTo4ZtHTv2vr1o7RwE03yXfq00fWGzd2VmrffZdZ/X7ECM+9faPCMSpIcE7srVdPFEp7uydW\n00Q56HOONGjgrKCN97tVK/H/AOTkZNK5s/u1hg513ENdiXTv7lBcet2tnj3lXk6aJJYM/ZhbbnEU\nOkxNlc6DPtrq1ct81DZ0qHOnI1REm+KIRZTFCKAbkmHr1ySMu3fLDb/3Xpg3T/wPbVw6pOFq5Lp1\nk6qcnTvLD/nCC96rUtbmBjtazrH5xGb6tujr8fgezXqwN3cv89+bT4HdGD537lymTp0aigTAvyOJ\nrjFIXTU9IMRTWfUUZKZAy8/0bbdBly7W9m3WTDotrj1zXyQlybEN7OOi2FiZGe/22yUQxNvvMniw\n+RSqOq1bO8qx+yI+3rkXrzeEcXHyn7bZZMQ0aZIsO3Y45GrUSBpbvac+YoSjx6+fJzbWoXzNiI11\nKK0BAxwKREdXrCCKOTHRXEHq96txY1FmY8bIKCw93X1WxNhYh6nMOAIxMmyYfO/UVPdRTv36DmVy\n7bXOJri0NFFoTZuKAgol0aY4jCWoL+IoQe0RTZOZxl58UR7iG28UDb5nj/MPXVPUqyfmsHXrZATS\noYOE8H73XWinlFTA3ty9fH/me4ZcPsTjPsnxyXRp0oWZc2aSmprK+vXr+fzzz7nvvvt44IGgXXGr\nAf1X3QiYZAL590ynpUnvu6V96rKkJPO5FjwRF+fdX2DGbbeZN4D16nlOdr38cun9x8dLg+eJhATp\nSBnxVL5nwgRRXsGEuHfuLD3/Ro2cR2RXXSXtg3HU1Ly5o1PZurXzPbjiCseIoVMnOd6Ibuq65hq5\n361M3Gw2m/yODRo4fCW9e3ue1TAlxdxfk57ubj40oiuTxETZt1cvWR81ynGvA5+rzJxoCyK1VIL6\nqafEFJSdLWao+Hhxks2YIa+eHIk1SZcuMinP9u2SMDhunBRm69lTTAQtW8K338Jrr8mfKz5eXm+9\n1bcP5k9/+hMHDx7k9ddDFtVZ6/j22LdMWzmN2dfNJinee0tzZ7c7+WPRH9l3dh+rVq1i2rRpjBkz\nhpkzZ4ZSpPsQ/5wrfpVV13uPoQjeMNKkiUQVhorOnd0VQqhISvI+Zaw3bDbz/0+HDu7bhg93vHf1\nR7jKYzz+yiudTVb+jvDCTffu4te9lBiPc4z7j5FEQiMHkKG/WtQSjsU+x1o1qxFntuti8ALwe2T+\nGDOsPNM66tlWSzgX12c7YKJtxGGlBLVLnINCYYn6iO9sJ7Af8TX0xIejGt9l1e9F6lTd4OFzf8qq\nq2dboQiAOCAbqVWVgESjXGKDLjceQxqaAiRxbDgwB6kLpjMFccLmAn9A7On6QHwOsNi+fwHScF6B\nzHFyyn6csXGcilRxLUB+i5+F+gvVIUYAuwFvXgX1TCsUNcBIYC8yrJoeYVkiTWckLFlP1bocKdsy\nG4fi6AZcAAYhU+7+GSjHWXGUIMohFgl3Pozc21jgp0g1Y51RgB5Nfy1QBPQN5ZeqQ+xHFO82+/KK\nfbsxHBfUM61QKGqQjsio4AZEKejMwaE4ZgH/NXyWBJThrDiMk0veiigaPbalARIZZJKrCkimc1hn\nQFIoFLWLaAvHNePPyMQ3O4D3AU8FD0Ygppz9iHnHyJ2ISaES8BZVfhjHlLXfBngOb3KkI87WfYht\n3VMgoy7HYqSRfwlJILsA/NFl35aIKWue/ZrfAK4zL582yPUqEhaq2T8rsb/eZD/uADLKKAbykBFI\nY/s+VibZ0uXYgflIxdc5htnl0Hvxf3D5vA2wFvktvsOzUvMmh5Vz+JIjlHh7ZmqCw7g/996e1emI\nrFlIJnyoMHs2ApGjv/0c+5H/TjjkmoP87/Tnw5jNUlNyeXqOo+GeRZybcCi4Z+2LK7FIg5eB9Mxd\n7chdkBIma/He6B9CbroZVs7hS465wO/s7x/z8F1c5dDP2R0J9zyHROUYRxxrgQ/t69ciIwjjiONd\ng1y3ICMSXa44+/7jgVWIwrjDfl2QEceT9vdDkUbYU6M/yiDHQESJueLrHMOQCcA80QKw5xKTgpiA\nXH0GvuSwcg5fcoQKX89MTWD23Ht6VrshMsYjMh8gdB1Qs2fDHzn0UfS3OOqBfYgo5lDLNRv4jcm+\nNSmXp+c47PesNow4QpFklYVoXyt4qjlk5Ry+5BiL+Biwv97mQ45OwM8R5+p+ZHRwEFFiOkuR6qsb\nEOfraPuxxkm4GxvkqkRGFGZJaDH2c+Qi93wkzr2SdcgoxBPG77cR6em4pi75Ogd4n7viJPLwAxQi\no9GWfsph5Ry+5AgVfie9hgnX7+rpWR2HdGAuIjIfwEvRRj8xezb8kWMgEi3XAMfI6R28/88ClQvM\nn4+alMvsOW5FDdyz2qA4jNyHoydpxCzJynPJVM9oyLwJm4FpARzvS47myJAX+6unfFBdjmXATOB6\npMJqE6R2VyoOU9Nu+/IgUuriAjKiSDOcK8lFrgoXufQ474HIw7gaOI+UfPFn9gWz72+m6L2hIY7+\nHchv3c3LvhlIT3BjEHJ4Ooc/cgRDqJ7dYDB77j09q7ppVCfc8vorh+v2Y2GU7yHk+XgThzkoUnJl\n4HiOw37PIqk4jHZDPcnqDGLLL0Hsy3cZ9v89Ei003+RcGtI71pOz/oQ0eq6JWr74ATHdJCM28mw/\nz+FNDtdiAnpjbcZg5CEYhpiOPkFGDWOR+/ItEoKrcxQxNTVBaiYlIAoA4AmcE84+w3mIXYGYS1Yj\nNtPWSG/jlP0akxFzmFVce2GevqMnttrl6I3IvdzDfinAEuBXSG8rEDm8ncNMDl++iEaIst+B/IGN\nNUsPY+4/8/f+hAP9eRsJ/BIxzRjx9qzi47NQ4kuOmuRVJPqwD9KpeyGCsqQglodfIR1HI2G5Z5FU\nHG/jsKPpcx5MRhrtJOSH0Z2a9yJ26x95ONcxxDmkz5XwCuJU93fehOvsx3RD7IKv+HkOb3KsQBpj\nPbT2MsRpbYZx/obPkLwLHbMEsjigA5Lk9jyiYI0zMFpJQruAKCmAjxA7qCd/jydcr9Pavs0frMgR\nj/xR3sVcsViRw9c5zOR4Be8FOGcgCqc3onSNTkYN6Qj0xdm040+CYLgwmy/E07Mait/YH/yR46h9\ne2uX7eGQ7zSORvkNHL9pTculP8f/wfEcR+s9CxkZeHaS3o78qUOZZLUWiR4ww3XehK8wjxjxdg5f\ncszF0VN9HHPnuJkcx72cE+BjxG55HunNbnf53Mr9aY6jl34V0kM2IwNrzvGrMXeO+zqHLzlsiA32\nrx6OtyKHlXO4ynECuc86j9sXI6sAY8XFA8isfyAO6Ma4E+kEQU/PvadnVXewJiA97mxC6wvKwN05\n7q8cGxGzq43QOKHN5LrM8P7XOCwhNSmXp+c4Wu5Z2MjAcwOyEhmBhCLJ6nbEDFWCmHA+MjlHe+Sm\nbkdGDYGcw5cc6cgIwjVMzpccZue8377o/M3++Q7Mo758neOX9uttRxztV5ucYwGixMqRe3FfAHL4\nOocvOYYgjvvtOIdC+iOHlXO4yjEd3zWnngb+Yn9/FaLM9VHzQft1zPxnkUwQbIf5c+/pWQUZWR1A\nzHaGQt5B4/psTA1QDj209ABicg61XPchDfZO5PlajrO/sqbkMnuORxAd9yysZGCuODwWjevQoUOk\nC4WppW4vngrBWSlW2ADx3W1DGpZvAXuR6+qorabIH92tHqt6ttUS5iVkRQ6jMarqXrz4M7Kzs9E0\nLSLL7Nmz1XXrwLWLiorIysoy/QzxFZlh1U90HzLKmIIoCb2cy3H7q8d5xyP5bEfrc6DkCt3i5dn2\nm2hTHCOAR5F4Yy9z5ykUgbFixQr69u3LLfap0rZt28ZYs9lz3NmMBClkIDbiu3BPEEyzfwZijvoC\nidbye95xhSKaiaTiWIDYjjvjsBu+jISWrcbZn6FQhIQ5c+awceNGGjWS/Mi+ffty8OBBH0cBErb8\nIBIavQdYiCRcGf0i3RCFoNuPf2Xf3hxJItuOOCFX4bucu0IRtURyPg6zOb7eqnEp/GDYsGHqurX8\n2vHx8TR0mes0xtP8qO58hCMoQucfhvdfIx0hVw7hKA1R64jkc+ANJVfkqImSCqFGs9vrFCacPw+V\nldDYLPBTwX333ccNN9zAs88+y/vvv8+8efO4ePEir732GgA2maQ5Uv8L9WwrwkYon+1o83EoAqCs\nDF58Ea64Atq0gfbt5f0774Bqh5x5+eWX2b17N4mJiUyaNInU1FRefPHFSItVzS4Lno+NG6Gqyvd+\niuhk3z7YsyfSUgRHpHpWbyHF+E4jWdUgsccLgbZI0tdEJKHNFdUrM7BlC9xzD3ToAL//PQwcKNvX\nr4eHH4Yrr4RXX4XYWO/nUQiRHnHMn68xycyIa2DBAhg3DpKTa0ao2kJ+Pnz/PVxtloEUBeTlQb16\nsGoVVFTg83cONXVhxPE27pmJjyNO8U7A57hn5SpcePNNGDkSZs2CFSvkD2OzyTJ0KHz5JRw4AL/+\ndaQljR6uv/56t2X48OG+D4wwR47A3r2RlsKcb7+FYxYLVFRVwblz4ZEjJwcOHYKTJ33vGwk+/hi+\n+sr7PpoGCxfWjDzBECnn+DokrNHIWKRWFEgp4EyU8jClvBweeQRWr4Z166CzmTsWaNAA3n8f+veH\nwYPhrrvM97uU+POf/1z9vrS0lKVLlxIXZ/lvMAIpIhmL1Cd6zuXzRshouj0STn4fUi7HyrFe2boV\nSksdv7XNYr/x7FlIT7e+v86yZTB6NCQk+N43OxsKCqCVhVqv+/fLdwlnb/v4cWjRwvd+kcCXsaSy\nsnaYISMZVeWK1ZLjlzRHjsDkydIYbNwIaZ7mQ7TTsCEsWiQjk+HDoWlT7/vXdQYMGOC0PmTIEK68\n8korh8YiZUxuRJIBNyF5HN8b9tGLHN6ORFf93b6/lWMtsWCBf/t/+qmMPlv7Wdy+tBSKi60pDoAz\nZ+DECWmwvSmpykr/5KjNaJqYpOLj3bcDLFkiyjkpyfGZvwreSEUFWO8DBUewpqotSE2fRr529BM9\nRd6UOXPmVC+ZmZkhvnR0Ul4O8+bBgAFi3/7gA99KQ6d/f1E2j6vxG+fOnatecnNzmTt3LocOHap+\nnrxgZcKlrkgRTJDaUxlAM4vHVrN3L3z3nWP94kVpyI3408AE04M9csTxvrjY834AJSXw3ntiMop2\njh6FoqLwXiM7W5SDJy5ehEKzCQECoKoKFi8OzbmsEKx+uhspRLYJyax9G0lsCsR7rZcCPon3kuO+\n/uB1irNn4T//EaXRqRNkZkL37j4Pc+PJJ8XMsXUr9PM2eW4dp1+/frqTkLi4ODIyMli+fDlDhkhR\n2yeeeMLToWYTLg102WcHMu3uekRZtEVKVFs5tppdu6RR6dFD1svLLXwxP/j+e1E8Xbp436+qCjZs\ngLZtZf2DD2DCBPcetI6uzAoKZGTUoQP06SOjluJi+Ogj39c0UloqzuRA2LJFAkL6eMieWbdOXu+8\n03MvvapKRlFWTHBmmCmmYEYU3qhp81awI479yPC8E1JW+C0gB5k8yN+5HFYAP7G//wmeJ/Cp8xQW\nwvz5MHashNZu2iTrH38cmNIASE2FmTNhek3XXo0yDh8+zKFDhzh06BD79+9n9erV1UrDB1Y6Q88i\nlUi3IVnm25CpekMeBuhPA+RqV9++XZZA8CegMTvb4QgvKPCsAHNzZaTiyrJlkpcUCPv2ycjtyBEJ\nHPGEWeP+/fei6I4dkwCTQAmXkoj0tSA0Po7eyKhjJFLRdj5S7ncNnrNlFyCO8CZIT2wW8qdbBPw/\nHOG4lxTbtsHLL4tDe9AguPtuePddafRDwU9/Cn/5C6xdC9dfH5pz1haWLl1aPdIw44477vB1Cn+K\nHOocQuY8SLJwLABLlswhNlZ8AZddNsxyFvLu3TIi9TQaqAl++MF9m+stN1M8eXmeFVJFRXAynTzp\nUA6FhZCY6HyPNE3yoBITHdu2b5f7bzQFh8t/4Ol7a1rwyiAzMzNspvxgb8UWZCrTN5CJQ8rs279B\npqP0hKeYihuDlKdWkpUFv/2tPLAPPig9peZhCA1ISIA5c+APf5A8j5rupUSSlStXBqs4jEUOjyNF\nDl2f4zRkvpZynIscWjkWgAkT5pCQIL1zbzrD1cm8c6c0dLoTvKAAvvE0jRbSMH35JVx7red9PFFS\nAsuXu0dGWQnJPXXK9z4FBY4GdfVq6NYNevd232/fPlEGVk2vK1fC5ZdLhKHOrl3i73D9Lq4N+uLF\nMGqUdb9ioLhe11Wp+cOwYc4dDy9mWL8JVnHciaNstCu3B3nuOo+mwSuvSGM+fTosXRr4Q2KVSZPg\nmWfgk09gRFTP8RVa/vWvfwV7CmORw1jgTRxFDkFqVnUD/oWYpr5DRs/ejvVKVZVn2/XBgw4fiBmn\nT4t/DDz3an019Ppxrr3f5X4YkV119WmPnksHn3ziPNLYu1dCy5s2lVedrCwZTbgqDm/mtIsXndd9\nOfyN+Otr0r+7pokytEJZmeOYffs8hy6fPy/3KDvbkfTrSkWFyBCO5N9gFcdPkWkKdUtkI+AR4A9B\nnrfOo2nw2GPit9iwQUqE1ASxsfDEE5JlfvPNYL2+X91h1apV7Nmzh1JDqNKsWbOsHBpokUNPx3pl\nwwZxzo4a5f6ZsXHMz7d2vpoM19TRG88DFqYQ2rdPzEhm5qmNG+V18GAZNYBvf4unz8+c8S2L2bH+\njNCLix2RcYcOOeT3JhfIPdDZutXzfh8ZniSj4jh1Sq7drp0EM6Smwk03WZfbKsE2G6NwLguSh5QS\nCYbpSNLULsRfEuY+eGR49lkZhq9dW3NKQ2f8ePkTLDWdY7Fuc//997No0SLmzZuHpmksWrSII8aY\n0yjihx+kEfXVQH74ofs2YyOnadIzX7zYucHxhX7dc+fM80esjDzOnZNRhpn/w5UtW8TP52ufYNA0\n51FHuLLYjdfwJ3fFOMrzhLcIqjVrHCbK8nIJPAgHwSqOGMAYMJeEYyKbQMhAbMP9kBpWsUjIb51i\n7VoJr121KjJVbGNi4E9/ghkzHEPjS4UNGzbwzjvvkJ6ezuzZs/nmm2/YG2W1PKyYRMwaluPHxSRi\nNgLRGzJPUUolJeL3MmvkTpzwfIwvtm6Fzz/3/PnBg/5FTpWWWvOTgNwjV4VVVVUzSYiezENnzjj/\ndsb33kyTRlwLJJaXw+bN/ssYDMEOXP+L1JV6CymeNRWZazlQCpAEqWQkjDEZiWapMxQXw9Sp8Pbb\ngceHh4KbbpKY+hdfFJPZpUKSPU03OTmZY8eO0bhxY05Ga3EjP/nhBzh8WBrGqwwT01ZU+G4sDxyQ\n4y+7TELAwVrvN1BOnZIijUYTjidcZc/Pdw4eKSqS/5WZicvVp3H6tGc/yyefQMeO8l7TRJEasWKq\nKisTWVyPNeLpfv7vf9YSAl1DiL/4wvvIIj8/9E79YBXHc8BOJBpKA55EHICBcg54AckFKbGf67Mg\nZYwqnntO/tTR4Jh+8UWxj959tyPJq64zZswY8vLyePTRR+nfvz8A06ZNi7BU/mPW+HjqTW/e7Ll0\nyA8/SCl+ndJSRyE+s16slcbTSmn4NWvMtweipDZskIbTU802KxQXi9nKW8CApknDnpLifv3u3WXk\ntGED9OrlrACsJucZj/F2H1xHlGZKwzhq/fDD0IX064TCVea3088LHYD/Q0xW+cBi4EfIyKYaY+a4\na8hZNHPypORpBJp8FWo6dJAw4ClT5I98KZRe153g48ePZ/To0Xz22Wds3rzZajUCX4UKmwDvIhUQ\n4oDnkSgrkNykAmQkfRHJLLeEt8YsO9vx3lvOgyfz1/r1zlE7O3c63usmJGP5E2uz7AaOP5FLrhFR\n+n0yOpit8sEH8pqX53mfnByJ8JowQfw7d94po5cjR5xLs7iaf705uT3hLQpLj5YL1fkCIVjFMR5J\n3GuOo867BgSq3wYg85Drt+Z9YBBeFEdt4oUX4Mc/dkSFRAOPPiqRXU88IWVJ6jq9evXi7rvv5q67\n7qJDhw6MGTOGMWPGVH/uJdbdSqFCPVt8OqJE9iKKpAL5XwxDRtV+Ydbw6E7dmq4LFUgjGGpcR1V6\n7zwUdZ90ZWSWUa67wvSghc8/Nzd7+eMyW7NGFJBroMEnwdhtCF0NLE8E6xyfi5RDTwUa2JdgBkVZ\nwNWIk92G/Elr+VxZwtmz8NZb0lBHE7GxUv//3Xdlfo+6zooVK4iNjWXixIkMGDCA559/nhxrra+V\nQoUncDz/qUgHyDgOCFnKZajdMsZRhT/s2BFaOazgOmIPh7Pb28hH98tYyUmxQmmpuy/GiJUgBFeC\nVTy+CFZxnCSA0tBe2IE41zcjvhOAf4bw/BHj9delqq3RnhwtNG8uYZqzZ8Pf/x5pacJLRkYGjz32\nGFu2bGHBggXs3LmTdu3aWTnUrFCha3jD60B3JDt8B/Arw2ca4q/bjEQO1gkiNQWqvwrLtbqwLzxF\nkoWDlSu9f758uXfFEgmCNVVtRqZ7XY6UWQD5g7wfxDnn2pc6Q0WFTN+6bFmkJfFM585SMXTECMnI\n/ctfIlv3KJwcPnyYhQsXsmjRImJjY5k719LjZsVtOwPYjpikOiAzWvZGalgNRkYkTe3bs5AJzQJm\n82bPzmp/wjN37/a9T7RhVFhWwnmD+e/5OwdKOFiypOanmvVGsIpDr81zs8v2YBRHnWPVKgm9jfZy\n5u3ayTD8Rz+CG2+UhLFmzSItVWgZOHAg5eXlTJw4kcWLF9Nejz31jZUih4OAp+3vs5Eih52RDpbe\nhz0DLENMX26KY8mSOdXvu3UbRrduwzwKdPq056xxf8psGx3iiujl3Dn/TFB79mSyZ09mWGSpjWXu\nNC0cgeVhZORImUjpnnsiLYk1qqrEbPXuuxJb3q1bpCUKHVlZWXTxMimEvRCi2f8iDnF234CYor5F\nChUaTbV/QaIBn0ACRrYAvZBpZGORkUd9ZM6aJ+yvRrT582vXs62oOQYNknDfQJk82eOz7TfB+jg6\nIwmA+mC3F8HXqWoILEH+kHsQZ3mt5Ycf4NtvJYSvthATA089JVFW118fHZE0ocKb0vCBsVDhHsRE\nqxc51AsdPoNEBu5A/Bm/Q6KoWiCji+3ARmAV7kpDofBKsOVWQkmw2udL4FHgNaCv/XzfIQ7CQPk3\nUo76LaSXVx/pxenUqhHHk09KBMwrr0RaksBYtgweeAA++wx69oy0NOHHy4ijJlAjDkXYCOWII1gf\nRzLSg9LRkFDFQEkDhuKYCbACZ6VRq6iqktIi3uYdjnZuv10iUkaPhq+/jmyZFIVCER0Ea6o6A3Q0\nrE/A4QQMhHb2c74NbEXCG5ODOF9EycyUVP9od4r7YtIkGXXoSqQ2U1RUxFNPPVVdZmT//v2sWrUq\nwlIpFLWLYEccDyJ5Fl0Qh+EhpERIMPL0s593E1Le4XFkatlqakvJkbfegvvuqxsz7U2fLiWvH34Y\n/lmLM2umTp1K//792WD3Mh44cICf/vSnPPDAAxGWTKGoPYSqSauPjF4uBHmeFshkOHpG1hBEcYwx\n7FMrfBz5+VI48MABaNIk0tKEhgsXoH9/8dvcXUuL3ffv358tW7bQt29fttknf+jduzc77Bllyseh\nqKtEk49jNuLXsOGcIBVo1aOTSHZuJ2AfUnKkFqYnwX//K6XL64rSAJm287334JZb4OqrISMj0hL5\nT2JiIiWGGg7Z2dkkWp+vN5gih76OVShqDcH6OIrsSyFQhcwImBHkOR9CihruQMJ7nwnyfDWOpkmJ\nkVpYrdsn/frB734HP/lJzUyIE2rmzJnDiBEjOHr0KJMnT2b48OE895ylNlwvcjgCmVt8EtDVZR+9\nyGEfJHv8BUSBWDlWoag1BDvieN5l/c8EH5++A7gyyHNElC1bpAzCjTdGWpLw8JvfSH2dF1+ERx6J\ntDT+cfPNN9OvXz++sc+vOW/ePJpYGxYaixyCo8ihMQHwBNLZAecih9dYOFahqDWEeur6+rgXfrvk\nePVV+OlPJZGuLhIbC//6l0xINWZMcBPo1BRbtmzR/RcAXHbZZQDk5OSQk5NDP9+hb2ZFDge67PM6\nsAYJFGkATPTjWIWi1hCs4jDO9RUDNCNw/0adIDcX3n8/sMlkahPt28OcORI1tm5d9CvJRx55xElx\nuLJ27VpfpwimyKFCUacIVnHcanhfAZwiuARAnVikMNxRl2tEPW+8AbfdBk2bRlqS8POLX4iz/LXX\n5H00k5mZGewpgilyeNTCsYB/RQ4VCm9Ec5HDdB+f+z3bmZ3fAP2R4f5Yl8+iNhy3tFSmY/3f/6BP\nn0hLUzNkZcHQoZLj0bp1pKXxTUlJCa+88grr16/HZrMxdOhQfv7zn1OvXj0gbEUOCywcCxbDcS+7\nrGbni1DUDaKpyOFWIBfYb19y7du2ICOGQGiNRGe9QS2r3vv229C376WjNAC6dIEHH4SHHoq0JNaY\nMmUKe/bs4eGHH+bBBx9k9+7d3GOtbHEwRQ49HRsQ114LjRsHenTo8VWGpkcP/85XGzoglzrBmqpW\nI3MLfGhfHwncDvwsiHP+FSmcGMwUtDVOeTk891x0TPpS0zz+uCjLZcukLEk0s3v3bvYYZgEaPnw4\n3azXjf/Ivhj5h+F9Lp5Nq2bHBoTN5phnOxro3BmOHfP8edu2/k1Ne9VVcNTUkKeIFoJVHNfgPA3m\nR0hIbqCMAU4jsfDDPO0UjSVHXnkFuneHa66JtCQ1T2Ii/OMfMgHUDTdIfa5opV+/fnz99ddcY/+h\nXnnlFeLj452eqWgnVIqjbVs4csT8s2DNYaNGwYcfylTJ/j4PgZboiY2tnblFtZFgFcdxZP6NdxGz\n0jmewP4AACAASURBVGTEiRgogxCfxiigHjLqeAeYYtwp2v7keXnwzDNS1PBS5dpr4eab4Q9/gHnz\nIi2NZzZv3szgwYNp06YNNpuNnJwcOnfuzNKlS71GXUULeqkXX4ojLQ0KC703pD16mCuOVq38a4Dj\nvLQiSUnympEBhw9bO1+gP0Mt+Pn8JtjJm8JFsD6OSUgI7jJkuthm9m2BMgOJOGkH3I3ExE/xekQU\nMH26TNRUl2bKC4S5c2W6WXtuXVTy8ccfc/DgQb744gsyMzM5ePAgH330EStXrmTFihWRFq+aqz1M\nX+apcWzSBO64w7GuaTBxovm+Zudq186x7dprvR9nvA64+1vq1YP69Z23hdO0psvrTxXqO+8M/rqN\nGgV/jkDOe9NN8pruKzQpjASrOM4CDyPFCPsCvyLwSCozosiSa8769ZJF/UytK4wSeho3hr/8RUqt\nlJdHWhpzMjIySEtLo6CggHPnzlUvGRkZZERR8S3X8lmujYRrQ3zVVe7HAIwfL52au+7yfj1dUVlp\n4I3X0UcUxokVx451H4W4KryBJumPycnu+15+uXdZBgyQEdKkSe7fv6uXoi7eRklWGT5c7m+w3HKL\n87qv3yAlRV4jmTsV7KUHIVEiWfb13kCo5rr7AvdQ3KjiwgW4917429+gYcNISxMd3H239F6fftr3\nvpFg5syZ9OrVi4ceeohHHnmkerHICORZ3w88ZvL5bxH/3DYkObYCmQoZpNzITvtn3/q6UGqq+Mx6\n29MHk5Ik6dIXY+3/GL3xSUiA+HjzRiYhwff5bg0gi8psVOTaqDdr5r6PPsOkPyanqirHe9cG11PD\nGqqorYQEh6zBVIsOdDRmdtzgwc7r4YpQC1ZxvIj8mXLt6zuA64I8Z63hoYdg2LDojySqSWw2SQh8\n9dXomiNZZ+HChWRnZ/PFF1+wdu3a6sUCVgoVPo+MvPsC04FM4Lz9Mw0J+OiL1L0yZfRo6T2npECv\nXg7zZ2yscy/9Sg/V3OrXh44d4Yor3D9zrZ2WmOhZeeiNkt679YTeczeOiMwa/jZt3Le5kpbmeH/b\nbb73B2dfjGtDqsthPC9I3hGISc2IXrIsEBOQN2XXvLn/5/MHfbTYr59jhBYTI4p46NDwmLRCUasq\nx2W9IgTnjHreegs2boRNmyItSfTRsiW8/DJMngxbt7rbuyNJ9+7dycvLo7n//2YrRQ6NTAZcg7N9\n9qWtRiA1bSphsHv3un/mSam4NqAgPdSL9loP3br51/u99lpHo9S2rYwili83b0SbNhWFduCA5/Pp\ntn2bzWEC81Xx3jjicEV/7qyOYBo3lpJBLVrAOQ8G9+bN4dQpx3pcnMM/ZKRJE7FCHDjgfoyRbt3c\nrRUNG0qRVJDfY8wYMJukUtOkkwFQUeFQGuPGyf3Tv3fDhp6/T6AEqzhyAH1wlID4O4Kt+NkGiaRq\nhvTS/glEVZzO5s3w2GPw5Ze+e2SXKnfdJeGYv/iFFESMloiXGTNm0LdvX3r06FE9D4fNZrPiGPen\nUGEycAtgLMSiIUmBlUjux+tWZW7Z0tzWb7TT+2vu0BurFi0c23p7qKrVtCmcOeO+PSXF0cCDNPJW\nzGlmjBjheEaMz0qjRqLcvvrK/DijKUa/BxkZEhZ//Lj7/kblGRvr/Jl+3WbNwJDq41Fe/Ri9xz9p\nkigym02WqipRHG3bws6d5ucxu+cDB4p5cf9+WW/QwLss4DzCTHaZbPuqq+DgQd/n8IdgFccDSKPe\nCgnD/RT4ZZDnvAj8GikWl4Jkoa8mSkpQnzghpql//tO7800huS3XXCNmq2ipZTVlyhQef/xxevTo\nQYzdCG4xDNefpvlWYD0OMxVIB+sE0BR5nrOAdVZOdp0H42/37tIoffih+efeGDnS++c9ekguB0g1\nhE8tTJYQE+Pu9Pak0Fy3N2rk2Kb/HFdcIUozL8/8HF27OvfWddNT9+7y2qyZmPt+MKh7Y5Z7ixaQ\nnS3vb7pJlEpWljTao0dDSYkozF3GUq4Gec0w+lViYkSZFBfL+rhx8MEH5se5niMUznudcHTaghEv\nDngJGZKHkpP2BWSCqO+BlkSB4igslGHjAw8ov4YV6teXSsFDh0ovcNSoSEsEKSkpPPzww4EcaqXI\noc7duJup9HS6M0j4+lWYKA5/kltjY83NT57wpwFp1szhwLbS4/VEfLz5dr2RNyYz6j11nQED5NWT\n4nCleXOJctL9NnFxokR8ZaEbj9HlSE2VpazMsd04uvIH/R4kJ0uC7Lp1vqMOu3eH7w0t3pAhMgIx\nmrx8jTIzMzOri3uaKb9gCEZxVABtgUSgzMe+gZKBOBM3hun8likvl7j4vn1hxoxIS1N76NhRSpHc\neissXeo7RyDcDB06lOnTpzN27FinKWMtzMexGbgCeSaPA3dhnrOUBlyLc4cqGXGuX0DmrLkZKYTo\nRiDJrc2a+edHMnOceyMhQXrkq1f7d9yYMc5mE10BtW/vaEy7dYPdhsmh/YlOMlOEZs5+XyY94zGd\nOnlWxjExnn0V3oiPd/gimjUThZSb67zPhAmwYoVDoej3R5e3TRvxe/ijOIwdjwUL4P33TR+5gAh2\nQHQQGZKvAOwDMjSkSmiwpABLkNyQQuMHNV1ypKIC7rlHfsxXX40ee31t4eqrpfz6+PEwf74jgSkS\nbN26FZvNVj0D4Pnz5zl//jw/+clPfB1qLFQYC7yJo8ghOGpW3Wbfp8RwbHNklAHyn/svwc+UWc0N\nN1jbTzej6D35YOjTx/dIxPXzzp2lirKR5GRHoxouBg+WkYMVk17//s7rxsY5VP/7IUPEJJaV5djm\naWRmRjS0P4Eqjv8A9yB5Fn9FwnpD6SaOB5YipUyWu35YkyVHysvhxz+G/HyxT/rzAysc3HCDmK3u\nvFMmgLr//sj8AXzNy/HEE157Zb6KHAL8274YOYTMQx5RYmND10gH4t8zc34H8gz06QPbt1sPCKhX\nz2Ea8zeIwDiSs9m81/eySlKS+aimXz9ribOJiWL6NYvmqikCVRz9Eb9DDvAyoS1/bkN6c3uQPJGI\nkZcnQ8iUFFEarnHfCv8YOlQi0caPl4z7SCVOrlq1ij179lBaWlq9bdasWTUvSC0iLU3+B4WFvvf1\nxs03O0Yiw4c7cie8YXQ43323NODbtwcnB4gfwVdUdpMmcs0jR6TBP306+OuCucL0RxH4W0x19Gj/\n9vdFoAmArwGfI7Ob6XNv6EuwmQ2DgR8D1+PIwh0R5Dn9ZuNGGdL36iU9ZaU0QkOnTnJvU1MlQWnJ\nkpotEX7//fezaNEi5s2bh6ZpLFq0iCPBdiEvAeLjQ+OfatzY4VNo3tw9JNaM5s3dzZtdugTf465f\nX0YQvrDZpIffvLkoG6vJid647DL5L/hDMCP0aKtY/VoErqmFkzNnNO3BBzWtWTNNW7o0rJe65MnM\n1LSePTVt8GBN++QTTauqCv81e/TooWmapvXs2VPTNE27cOGCNnjw4OrPiWx9tPDfgCC5eDEy162q\n0rT584M7x4cfatqxY6GRpybYuFHTiosd62VlmvbDD4GfL5TPdrAlRx4IiRRRwJEj8Oij4sCrrJQE\nINcqoIrQct114ix94AH4zW8kwubppyV0MFyjkCR7TGVycjLHjh0jLi6OkydP+jhKoRPK/AJ/CIU/\nbORIyQupLVx1lXMIcEJC9MyOGKHHIDooL5f5wd98U2re33uvlMiwMnxVhIbYWAk++NGPJDv4vffE\nFFBQIKUzeveWZLRu3cQ8EWgsvc6YMWPIy8vj0Ucfpb89hGbatGk+jlIoFNGOrwqkQQ8Bd+7UtF//\nWtOaNtW0oUM17e23Na2oyPdxa9euDfragXCpXVfTNG3RorXa8uWa9sQTmjZxoqb16KFp9eppWqdO\nmnbnnZr2zDNi3jp7NvBrlJaWaufPn3fahvfhfDDVcX0dG5JnOxxE8jkw4vpbR4tcrkSrXD6ebb+I\nYEV3U6xUIA2II0fg+eclgW/UKIkf37BBonzuvde9vosZvkI5w8Wldl2A3bszGTcOZs2ChQvFfJWf\nL0mE48bB2bMyB0pGhpgXp06FN96Affu8m7kWL15MQUEBAHPnzmXq1Kls3brVikjBVMcN23NdE0Ty\nOTDiWuU1WuRyJVrlCiXRZqrytwKpKZWVMk3l9u2iHD77TAqejRsnymPYMGvRHIroIiFBzFY9eohp\nC+S33r1bfufMTHjySUn2+vZbc5Pjk08+yZ133sn69ev5/PPP+e1vf8sDDzzAt9/6nCIjmOq4IXmu\nFYpoIdoUhz8VSD1y++1SjbJXLym69tpr4mhSyqLuERsrv3OvXuJkBxldenIixtofglWrVjFt2jTG\njBnDzJkzrVwqmOq4IXmuFQqFOeNxLjf9YyTB0MgBxFanFrWEY/E0Y4SVZ1PnLsBYB9XqserZVks4\nFy+zofhHtPk4rFQg7Yhkl6vFsRwGbrC/b4HMxPhn+/rvEJt7LJKweQSp3hppmSO1pAATgE729ZbI\n6ED/vCPmBFMd1+qx6tlWSzgXT892rScOyAYykImhtlOLnIgR5BAw3LA+F/ifh31fIsomxqolWH02\n04CzgDFwWD3XCkWYGQnsRYZV0yMsS23hEDLiAGgN7ATMii/ZkFDRn9WQXHUNs2fzfhwVcgF+Asy3\neKxCoVBEjMPIXA8FQBVSwtvMDPkEojhUjV+FQlFnCHeCVTiuexjp4W8DfMZ0BnDtJsDHiHnjO+Be\nk2MvInOzg0wilI+EgBp5EDGXtHQ5NpDrHibw7+zruo0QxbcDmcCrux/HhvPahwnudw5GrnBzGPfv\nlo5McbsPmTvEWMd4OiJrFjIpVah4CziF/Md0ApGjv/0c+xHTbDjkmoP4qfR2wTgZb03J1QZYC+xG\n/qP61JbRcM9qjFhkGJ+B9Ih92YHHAJ8FeGyorgtiJkr3sG8orj0H+JP9fRPEfh7ncuwh+3v92D8i\nD5TOfUgJ/IwQXBcC/85WrvtnQI+P7UxofuNgrw3B/c7ByhVuzL7bXCSwAkSZPWt/3w2RMR6R+QCh\nC7IZigRyGBtof+Sw2T/7FkfH6UOCr65tJtds4Dcm+9akXC1wzPOSgphCu1ID9yyaoqqMSVIXcSRJ\necJTgpWVY0N1XR2b2Y4huvYJQC+KnIo04BW4J5WtNRz7ov3zgcCPgKeR3oW+bzDX1QnkO1u5blcc\nSm8v8oA3s3hsOK7d1PB5oL9zsHLVBK7fbSyOCan+jcxsCCLbAkTWw4jsrqPbQFkHuM4w7o8cA4HL\ngAY4Rk7vGI4JpVxg/jzUpFwnEUUAMkvq90jOUNjvWTQpDrMkqVYe9tUTrJYGcGworwsSH/0ZMheJ\nv9XyrFz7dcRkchwxofzKw7FnDMfmIg/M48BTSG9yE+IHuYD0lgK9LgT+na183x2AXpf4KmRe+9YW\njw3XtSG43zlYucKN2XdrjphnsL/qUx61xDmUONzy+iuH6/ZjYZTvIeSZeROHOShScmUgo6KN1MA9\nqwnF0RCZO/x7ZFa/gZjb4DT7/roNbi7Ose9GbkXmOj9vX9c87GcFf451vS7IxFN9ERvnL5FhbSiv\nPQPpVbREhqV/R3oHRtoh99bIL4DbgfZAov0YfXkd33i7bqDf2cr3fRZ5HrYhfpltQKXFY8N1bYAh\nBP47BytXuPH1e+oJZJ6oqe/gS46a5FXkf9cHGZ2/EEFZUpDO7K+QjqGRsNyzmlAcLyE2s65AL8Qp\n8ziiODohMwk+jmi5LkjWbTdkXvMhHmQMNMHKjGASu0AeGpAe/zL8G7ZbufYgYLH9fTZij+5s3y+c\n39nTdSHw72zluhcQn0xfYApiKsq2eGw4rn3Q/tlx+2sgv3OwcoUbs9/zFGJDBzFl6JOmusrb2r4t\nXPgjx1H79tYu28Mh32kcjfIbOJ6HmpYrHlEa/wGW27dF6z2zTBqOP56RLBzDpxb29TjEjv4sjiSp\ndcDVJucMZYJVMIldyTh64fWBr/AvysTKtf+CmJZA7tlRZMQW7u/s6brBfGcr102zfwZiNvmXH8eG\n69rB/s7ByhVOPH23uTgivB7H3cGagPS4swmt7ycDd+e4v3JsRCwbNkLjhDaT6zLD+1/jyN2pSbls\niD/iry7bo+WeBUwfu0BvA1sRE0l9nB1NNsP6CqRnpydJvYFMTxvuBKtAE7vaIT+EHrIaSGKXr2s3\nAVYittRdiHPe27Hhvm57gvvOvq57jf3zLMTEmebj2Jq4dih+Z3/lqik8fbd0xO9hFtI5A5E1C/H5\nhYoFyP+/HPH7TA1QDj209AChqZLgKtd9SIO9E/l/LMfREa5JuYYgeVvbcYQFjyA67llQDEA8+Ffa\n119EnLWuEQrn7K8vI1FAOm/gcFYC0Lt370gXClNL3V70KJUaRz3bagnzErJnO9w+jqP2ZZN9fQnQ\nDwkjC8h+umPHDjRNC2iZPXt2wMcGu0Tq2nX9Ox89qtG8uca772p066axcqVct6ioiKysLL/PB/QO\nz1/BN8E827X9d1RyhX8J5bMdbsVxEhnadbKv34hkOa5ETD/YX3WnzgrEAa3b4K4g9Fm6ijrEE0/A\nfffJxE7TpsGyZbB371769u3LLbfISHzbtm2MHTvWn9MGk1VvFkUIklz4PWLa+P/tnXmYFOW1h9+e\nBYaBQWCGbWBwQERklUVQlrBEZXFEg6jojUZNiJoYvDfKFVQWc69GccvFJTFEFFFkTwImEBeYKOIo\nICIygMjisCrKCC6MMDN9/zhVVHV1VXd1d/UyM9/7PP10d23f19VVdb7lnN9ZRuAQnEJRo0hEIqff\nAC8jxmAXMm6ZDiwCfo4EolytbVuqLS9Fgs1+hXSxFIogvvkGFi+GUs0R+Yor4IEHICOjmG3btjFs\n2DAAevXqxe7ddj4ajjyFNHIOIL3l5QRm69NddacgRmQH8BJyzepehOOQ+6uhts9riBGqRiYrpyAT\nlwpFjSMRhmMzxhyHmYsctn9Qe3nO0KFD43HYlC67Nv/mBQskDXBrzb+lsBAKCuCrr5rQpEmTgG3T\n0iLqXIdL83oIcS2HwKj6M5AYCL03XYnohoG4n+u8hyR3qhEk8xoKhapX8ki11LFxpTY/RFOt3ESU\n/dxzMM0iHj9mDCxbdiEvv/wylZWV7Ny5k1mzZjFgwIBIDh0uzetsYDXiaZOD0WNuj8RCPI+MJ29E\ngrK+t+x/M8HxQAB88QW8+SZ07Aiffgp9+sCxY3DWWVBdLfnVu3eH7GxYvRrOPlte69fDkSOyrU63\nbvDxxzBihLwfsHjmX3ON9NpOnoQ33oDhw+WYLVuKAc7IgJIS6NNnKB9/DFu2QKdOsk/9+lBeDhdc\nAM2aQVkZ7Nsn5Z97LrRrB598AsePw+7d0KQJZGZKHQcNkn3efx/y8+VVUgItWkjvsVUrOP98qKiQ\nNMCffiq/PS0Nxo2DL7+UesJQNmyAnTvh8sshKws2b4bcXMjJgVWr5DiNG8O338L+/XKuiothwADJ\nU9+uHQwcCNu3w8GD0Lu31LW6GiorpcyMDDh6FHw+2LFD0lAvXAijR8s52bdPjnfmmXDoEHTsOJTd\nu+G994xz3bq1rMvIkHO2bZuc788+k99/7BisWQPnnSfbrlwp21ZWwtixcr7XrZPtBw2S37dyJXTu\nDCdOyPWwbZv83vXajHJWllwDbdrIsbwkHto78cavTfQo6jB79kg++YMHA2+K5cvhqae+o2/fB3jt\ntdcAGDFiBFOnTiUrKyvscX0+H4g3ny698VPEcPzGtNl9yBDVfwJnIb2JnkiA5LtI8OR6xIvwOIG5\nUe5FHETsehz++fO9v7b1h5aV7Gz43mrSkkD79vJ/pipjx8KyZeG3GzcOliyJvby2bcXImbnsMlix\nIvpjNmsGI0f6wKNnfiIMx17k5qlCXHP7IX7GCxEtoL1Ii02X8ZiCtMiqEJng1yzHU4ZDwcyZ0pL9\n058Cl+/eLcNXZWXRHVczHP/CCICagsxLPGza7J+IcOQ72vc3kfmL/YjhaK8tH4TMYxRp329EDNKP\ngQqb4v1jx04//aVLl6F06TI0uh+iqPOUlhZTWloMSG9v4cL7oQYZjj1IcMlR07KZiBCfHuHYFLnB\nuiBBducjIltvIB5Z1aZ9leFQ0LevGI/hwwOXV1dDZuYwBgwI7In4fD5Wy/hGSDTDsRt5uB9EvPqu\nJXCO43Fk7uJ+JPBrIzLncRR4C/gFEnw1A1EauBsxRI8BQ5Br34649DgUCoDrrvOux5GoOQ47yeYh\n2ue5QDFiOJwkm0sSUUlFzWDXLhlXHjIkeF1aGnTu/Ag33gg9e0JFRQVLly4lI7JB3tuRXkc6ony6\nDSO6/FnEeeN5xPEjDcl9oDeM7LwIQYJb62FMkr+LeA0qFDUON3fTRiQD1nzsNenD4Ud6DlXITTeb\n0LK/ZiORDIlpRYqzcKGMJ6en26/v378vVVXSKwEYNGgQ559v59jnyErtZeZZ0+cvEaVkO5y8CM+O\npAIKRSrjxnCMR1pN6xG9/ueReQe3feqBiPtic6S1td2yXg+HdyJo3YwZM05/Hjp0aJ1wf1MYLFwI\ns0Ko6XTocJSNG8UTprq6mg0bNnD8+HHbbYuLiykuLo5PRRWKWkok411pyCTfH5E5hzlIsNPRUDtZ\nmI5kqpoADEUiy1sjWdc6YwRE6WqOq7R9TI5tao6jLrN9u8xr7Nvn3ONo1aqQY8d8tGoFGRkZFBYW\nMn36dAYNGhT2+NocR7K8DdUchyJuJGOOoyfS6xiFaL/PRzxGVmPkvLUjGxkn/gaJoL0EmVBcjgRJ\nPUyw5Mh8ZPKxDUpyRGFhwQK46ipnowGwfv1e+vdPbRdPL8nPF7dkhSJRuJ3jOIb4tt8N/KAtL0GG\noULREkkMo5f1MjLMtQElOaKIEL8f5s2ToSo7li5dis/no7paAs0WLIB69Yz1Y8eOtd+xhtOwYfht\nFAovcWM4rsI+GRNIatJQ7MG+R3KUJEiOKGo277wj0bB9+tivX7FihT7URIMGsGgRnGGSEozAcGxH\nesp/ITB+AyTw7yVE3TkDeBQj4VMTbZ+uSIPnZqSBFSpuKWbq1/fqSAqFO9wYjl8g8Rb6hd4UuBOJ\nnlUoEsbcufCzn4n0gx0vvPDC6c/798MvfwkjI8hjVlWlpxdnJN4KHOqpkvW4pcl4KHDYubNIiigU\nicKN4RiNZI3SKQcuRRkORQL55htYutT9AzIt7VWefbaU9983ArSnWYWtLLz//unptL3au1cCh05x\nS56QmenVkeoGZ5wRqOmliBw3kqFpgFnkpwFGXmY3pCMtNF1ppRnS+rJLazgFyYGwHe9yOitqAS+9\nBMOGyURwOG655RaOHFnEG2/Mwu/3s2jRIj777LOw+x2wqgDaxxHNRoaiDiIxG3doy80Ch3qa5Gxt\nnVPcUq1jhJeJZONE1672y7Oz7ZcrgnFjOF5GtHh+jgxbvYHk23XLHchktz7JrXfbO2nH1VteXYBr\ntPeRwDMu66eo5fj98Mc/wm23udt+3bp1TJr0IunpzZg+fTolJSXs2LEj7H4+pzGwQO5BEjjlI/N3\nTyMKuRmIeOEz2vt32PcqwsUt1WiaNUt2DYLJyzM+Z2eLim0ocnLiWx83pLoRczNU9TCSlP0i5IL/\nHSLH4Ia2yFDXA8BvtWVKbkQREWvXisS2VZfKiQYNGtC+PVRWZnPgwAFyc3M5fPhw2P3atAkSKShA\neh1mBiDXM4ikyB5EFdeaJnkpRvbAz5HJdD1uSU+VHMSSJTPIyxPpcLPI4cCB4hxgpnlz8R5LFC1a\niOx7tNSrJxLuOn36wMaNsdcrFFalmVAhYPq6UaPEsUKnoEDihiKhaVORnY+WIUPkXMdyfswih17j\nNo7DToLBDU8Ak5CxYB0lN6KIiCeegDvuEB0qNxQVFdG0aTk+3yT6aC5YEyZMCLMX9NU1SqAQGYq6\nBhE4NLMdaUS9g1y75yBeh0cx0iR/gogkbtX2cYpbCmLcuBm28tz1bAaHza1S3djY0bWr5GqorrZf\n70SjRrKvnleiZcvoDUe9es5ODbHg84U2BiNGwKZNxnc3hsOLekYy7zRkiOTZ2LvXWNakSeyNAqu6\n8rJl98d2QBNuDMeVSCR3S4yoQz+BxsCOIqRltQmJErcjYrkRUJIjdYndu+Gtt+DFCAZHp02bht8P\nVVVXsmXLpWRmVgRlBNRxkBzxWuDwIezjlmzx+gHbowd8/rmzYbGjqMgYsjEnJLJilztCJzMTTp2S\nz1deKfngo6FNG0lCZT6eTjjD0bixuGbruDEcocjKkt6vlVatwEWnNgD93OXlydyd2XDY4fRbGzeW\n5F7WHmk8cWM4ZiJGYFu4DS0MQIalRiOT642BeTh32w8gQwM6bbVlQZgNh6J2M2sW/Pzn0vp1S48e\nPRg/fjz5+ddw+PBZdO/unMDJ2vC4//77QXoRZmIVOAwVt+QK6wMjLy8wRiWcsYnEGDVr5n6c364n\n5MSwYZK1Tn/INm3qbj/9t19+eXSJksznLpRxyMyU82Q9V+Z9zjpLsjBase6Tlxe+d6Yf10kFwRrY\nOWKE1NGa0OnSS0WXLZG46fwfJnKjATKJWIB4m4xH5Emux+i2Q7DcyHiktdYeJTdS5ykvl57GxImR\n7bd8+XLS09Np2PBqbryxL48++ihl0WZ2SgJuHvIXX2wf+Netm7x36OC+vJwcabHqxNqB14cUrb+j\nSROpd+fO8r15c3fH0x+smZnGvlZatYq8nlbS0uAnP4nMyF6mNSHM+/TsGXz+rR3ejh2Nz06Gw+pB\n2LSpYaj79XNfx3g4LLgxHBuQqNdrkWGrK4FotBt0u/0QcDEyDjwcQ9DQLDeyEiU3Uud59lm5MYPn\nrENTWFjI3XffzebNG1mw4BU++ugj2rdvH37HFCZSXc/+lizp1oeTedLYrZGx8/Sxq1eo1CdmDye3\nNGggQ10AvXrJe7duYvAaawPmubnG9uee63wsvb7m3poTDRtKAGmoc+/UE7bu8+MfB37v0EGc2cae\n+QAAFnFJREFUA8xGOpJg1WTrvLoxHGcAJ5C4iiLt5dRVd+LfyLAVGN32TtoxzdILDwIdEaVct55b\nilrIDz/Ak0/Cb38bfls79u7dy8MPP8z48ePZvn07M2fO9LaCKYC5lRtK9NGOn5jEgvLz3T2IIhku\nBOeWu7msq692Z0zshsRGjpQejJXzLCJHdkNVo0eHrhdID8ftcFq4Y1nPRW6uGKbWrY1locpy6xhi\nR+fOkQ0pusHNHMeN3hapUITn5Zdl+KRnz8j37d+/PydPnuTqq69m8eLFdIhk3CYFSE+X+YA1a4xl\n5lZ827byXlho9AIGDBBj6yLOMeh4AJ06SSs80gnWUAbHzZBPenr0ke+RJXUUImmpR3N8u3JGjYrs\nN55tSfnVujVccEHgMuvwU6hzfeaZ4WNXIsXNqTkHCWpqhUTM9kB6D//rbVUUCqG6Gh59VHoc0TB3\n7lw6Ow2Gpzj6cIV1TLxFC+NzljbXn55utFjr149N7LB+fYlX8IJzz4XNm53XWx/ePXtCu3aG91Z6\nOhiyYbHTsydUVkYWi9G9u9EDcGNsQk2oOzj0OWI1HLm5xn+uG7NmzQLjYMzl/+hH4okI8ZOjcdMB\nmo1MdOuhO1sI9m23IwtJwPQhMm/xe225khxRhGTFChnXdhvwZ8UDo7EduQ7vtlmXhyQY+xD4mMAe\n+V4kWHYTgY4d/bTvm5AAQcc8tvrDKh4xDyAPIb2MwYMD10VTpt1DtUsXif9wkvaw0rSpMc+SlWU/\n/BQLDRqAnsPLyQhkZwc+4Lt1i3xuzUwsQ0Pm/+Hqqw2HB5Ahq2ttnr7murdpI/vFEzeGIxtLBj4k\nujscFcAwRJahh/Z5EEpyRBECvx8efBDuuSd+D08nLOq4XZAGknWqVVfHPQ+JT3oMo+fu15b1QoyF\nzkxgqrZ8mvY9Ylp6oHB10UVwidYk04e8rHhx3nv0kOGvVMPOcNSrJ04YA8NlF8L+3OTkBHt1NWxo\nTNzHQnq68/9hneA3u+/qc17xuofcPJiPIBPWOuMQhVA3fK+910MCqsqRYa652vK5wBXaZyfJEUUd\nYvVqOH48cPI2UVjUcU9hqOOaOYQR/GpWx9Wxu1UPIU4mID1s2/gkM3Y3vD6EEctEaVpa8P7hHi6X\nmVxhrrjCfpvWrd3XywuPoHCTzxde6G6/Ro0k2DEtLfxk/sUX23tsFRWJkbzM4jJkPd75jv3M0Ps5\n0bKlfe8jEbiZ47gd+DPi6XQQ0eb5D5fHT0OUQs9CcpVvRUmOKBzw+2HqVLj33tgejt999x2PP/44\nZWVlzJ49m507d7Jjxw6KiopC7uegjmtxbGU2EpN0EBE3NA8K+BER0CokaHC2tnwysBZJ+pQGODzW\n3NGjh/O6s8/2XqTP7E1ljsIGY+zdK/EGr9xMCwvdzdnUq+d+biic91c4r7OOHWXux6zX5SXWc9e1\na/zmONwYjl2I7k5D5KL/JoLjVyNd+jMQ99phlvVKckRxmuXL4dtvY29F3XTTTfTp04d169YBkJ+f\nz7hx42wNh1lypLS01M3hdXXcoUiD6HWgJ3JfDER6F8215duBtxHpkolIGuWrgDlILFMQ+rUto2ai\nNXTNNYHbhHoYZGXJQ1PHjQx9LPTsGTpuwg4vjINTq9z8KLC6KNtJdrhpoFjL6t0bPvgg/H52DB4s\nE/WRlOeWtm3hk0+M70eP2srpeIIbwzEdeYD7CHyQ/y6Cco4B/wD6oCRHFDacPAmTJ8Mjj0Qek2Bl\n165dLFq0iAULFgDQMERSbnPDo6SkhMWLF5tXR6KOuwFjCPcIYiTORwxHPwzJkSVIellb9Gu7utrI\nrR5L7yvSh3qkpKcH90JixY1hiabMAQOCj+2md2bnuhyt4TB7x3lNr16i6aXjIKfjCW4uye+017dI\nD2I0oh4ajjwMj6kGSAtrE0pyRGHDI4+IDtCll8Z+rPr163PixInT33ft2kV9F+MRFnXceoizxnLL\nZro6LgSq42YjQ1cgvfNLEK8rkPk6PZXAcMSjMCRpae6im92S7EhjM+HiI5o2Dfb4shLNEEy7dt7E\nM/h8gZHqVmKJ/4DoGwppaTBmTPjtvMDNT3zU8v0RxI02HK2Rye807TUP8aLahL1SqFlypBIlOVJn\n2LZNpNM3bPDGC2TGjBmMHDmS/fv3c9111/HOO+8E5CN3IsO446NRx+0ALNMPhajk6vfJL5GET/UR\nFYZfxvDzajytW9s/4IYMkYeuz+fs8QWyr1Oio2Sn0b300sgj7K37p3oSJ3Cfj8NMQ9xNWm9BMqFZ\nCaUU+qD2UtQRTpwQn/Pf/z5wbD4WLrnkEnr37k1JifhazJo1i7zIRJKiUcfdjczn2bGB4En2sGRl\neZcb2+seR1pa5Pk9zNiNHrqdj3EaebziCu+HzSCycxerC64XLryJwI3h2GL6nAa0ILL5DYXCFr8f\nbrlFonR/8YvYj7dx48aA9K+ttbDqsrIyysrK6N3brh2Tugwe7G0EtRNue3nJbs2HI1KjceGF0etk\n1XXcGA5z66oSmdx2EwCoUITkgQdkmOrf//ZmiOrOO+8MmTd8jVn8qQaQmRn7w1pvLcfa47jyyuQ9\nQONlsNz2cHv3lmh4MwMGSG+5ruLGcBy3fLf6IYRKIVIAvIj0UvxIPMgsRHZkIXAmxjyHrpI7BbgZ\n8YWfiLv5FEUN4/nn4bnnYN0678Z04+V6WJNxY5DdPJiT2ep28xviqTKQkRE8b9GoUWxzGTUdN4bj\nA6AdEvUN0BQow4jBCCU9egr4L8TvvRGwEfFvv0l7n4noAU3WXmbZkTZIMFUnxJtLUUv4xz9gyhTp\naZhlpb3ixIkTPPPMM6xduxafz8fgwYO57bbbyMpyzgRY23HqcSQr8lhRs3Hj+PU6koMjV3tdivQC\n2hPaaIDEanyoff4W8VBpg5IdqbOsWwc33gh//zucY52C9ogbbriB0tJSJk6cyO23387WrVu5/vrr\nIzmE1yKHIPnIt2n7PBxJZWJBb4nHY3LcLfH6nxXJw02P40Jggun7SsQlN1IKEZG391CyI3WSbdtE\ng2revOAMdV6ydevWgCjw4cOH08U6SG2DReTwAKJku5zA1Mm6yOEUxIjsAF5C5v90kUPr8O0wpLHU\nA2kUuUyaGju64TjzTEMexAvy84PH/RV1BzeG4yBwH3Jz+IDrcCHSZqERsBS4g2DJkohlR5TkSM3j\nwAFJaPPII5GlyIyG3r178+6773KhpnRXUlJCnz59bLc1S47sMxI27NXedZFDs+E4hBgAcC9yeBuS\nVkB3Kjni9reYiWUc/5xzvG/5R5sdT1HzcWM4rkVkR/6qfX8Ld/k4dDIRozEPI0o8JtkRJTlSszh+\nXFJ13nor3HBD/MvbsGEDAwcOpKCgAJ/PR1lZGeeccw7du3fH5/Px0Ucfnd7W3PBYsmQJc+bMMR/K\nK5HDs4EfITFKFcBdSGxH3Em0NL2ibuDGcHyFeDc1RKRHIsGHRN+WAn8wLddlRx4mWHZkPvA4MkSl\nZEdqOKdOwVVXifvi3XYzBnFg1apVUe0XypXXRDQihxmIU8kFiH7VIhzmB73uTceicxUKZZBSH3Nv\n2mvcGI4BiChbDtIb6IlIMPzKxb4DgZ9iTBiCjA0/hJIdqfX4/TBxogjhPflk4h42hYWFlJeXs2/f\nPipNUqThAgDbBKd8i1XksB9iOPZjyJGsR7wEc5FGWQBe9qZHj/ZW7yrRXHstvPJKdAq2iviKHLox\nHH9AJgv/rn3fjCHYFo61OHtuKdmRWs4zz8Dbb4snVazCb5EwdepUXnjhBTp06ECa6akTLgDQInJ4\nEHENtw7L6iKH7xAscpiO9Dx0kUP9Tv0bIm74b8S9vB42RsNrUsFotG4dWncqHKNGeZ9fRBE7bm/n\nMsv3MIryirrO6tXwP/8jRiPR+jsLFy5k165d1Iswai2OIodztNcW4CQQ1UxPKhiCSInVb8WcS1uR\nOrgxHGXIkBNIS2kigV4moZiDxH18AXTXlqmo8VrO7t1w3XUwfz50CBfpEwe6du1KeXk5LaNP0u21\nyOEpIKJAEjtyclTAniI1cGM4bkVkQtogHk6vAb92efzngScR2RGdyaio8VrL119LDuZp02D48OTU\n4Z577qFXr15069btdB4On8/H8uXW1BoKhSIawhmODOD/kNiNaHib4KRPYzDmSOYCxYjhcIoaL0FR\nIzhxAi6/HC66CH7lxnUiTtxwww1MnjyZbt26nZ7jcOkxpUBNNCvCE85wVCJDSvWBHzwqU0WN10K+\n+04UVAsK4A9/CL99PGnUqBETJ05MbiVqKGPHpr58uiL5uBmq2o14Ry0HvteW+ZFYi1iJOGpckXqU\nlcG4cdC1K/z5z/GLHXDL4MGDmTJlCmPGjAlIGVvT8nEkAxcZdhWKkIZjHjKhNwZ4AvEg8UJIOKao\ncVCSI6lCVRXMmQP33gt33QWTJqXGMMcHH3yAz+c7nQFQx84d1yFIajviVfUXggUJ8xD5nVbI/fMo\n8IK2bi+ShqAKGXK1CnTeiei85RE6HUFK06mT9CwVdZdQt3kp4q++ComStW7r1g+9EFiB4VU1U9v3\nYWRuownG5Ph85GbTJ8c7Etzr8Pu9lvpURER1Nfz1r3D//eIi+uSTcJ6TP1ENoqqqSnfJbY8hcngt\ngV6EM5ChW7PIYUtkWHcP0Ad7o1CASJCcE2IbdW1HwSuvQM+eqSe6+M9/SurfVPGE0+b5PGnahepx\n/Al4E/FP32hZFy4Ph84ryER4HrAPmIaKGq+xVFXBkiUSn5GVJRn8iopSo5dh5dVXX6W0tJSKiorT\ny6ZNmxZyn/ffP61us1d790rkEGRo978xAmkVihpLKMMxS3v9CXHJjQYnW6uixmsQp07BwoXw4IMS\nzKcr3KaiwQC45ZZbOHHiBKtXr2bChAksXryY/i503A8cCBoZ9Urk8HLtWB+hUNQC3MZxKOoge/ZI\n7ozZs6FjR/GWuvji1DUYOuvWrWPLli306NGD6dOnc+eddzLShZZ7nEQON2r7XGwuyungav4ucgYP\nhuhjPWsvyRY5VNQRqqpg/XoZm331Vdi3D8aPhxUratYcRoMGDQDIzs7mwIED5Obmcvjw4bD7xUnk\nsByZ59usrWuLGJN+GI4hp1EpAyInFi2s2kyyRQ4TzUhEWNHJq0XhIeXl8PrrYihWroRWrURY7okn\nYODAxIoTekVRURHl5eVMmjTpdAKnCRMmhNkrbiKHH2PEKkHoCXSFQhEF6UjEeCGSAOpD4FzLNv5o\nWbNmTdT7xkqyyl6zZo2/qsrv/+orv3/rVr9/1Sq//+mn/f7bbvP7e/Xy+3Ny/P5Ro/z+p57y+/fu\n9b7sZGAut6Kiwv/111+73heZp9ihXYdTtGvuFgyhwzzES3AzIlqoqyp00K5XPRe5vq+V3Yhemx1x\nOBuxk8z7JhSpXq9Nm/z+f/0ruXUxg4fORqnWnuyH3LB7te92Xi1RU1xcnLQx43iU7ffDV1/B/v0y\nrKS/DhyAgwfh889h9+5iTp4cSsOGMg5cUCDCg127wk9/Cn37QoQisq5J1vl++umn6d27N40bN2bm\nzJls2rSJ++67L5IAQK9FDs0kQfYxNpJ534Qi1etVk4Z3IyXVDEcbxG1Xx86rJSXx++HkSaiogMpK\n+Z6eLpG4WVmh96uoEMmOY8dk6OjLL+HIEfjiC3k/elTEA48fh2++kc9Hj4rRyMmB/HwxCAUF0K6d\niAvm54uhePFFeOihuiUj8dZbb9G4cWPWrl3Lm2++yV133cWtt95qdrdVKBQxkGqGI6au1K9/DSU2\nkoh6TNWhQ7B8uXgF6S99fXV18MuMefvqaplIPnlShP2+/15eaWliJDIyZNvKSnFlraiQ7489ZswZ\nVFXJuh9+kId6o0bi6tqsGeTmQvPm0KKFvNq3l0C7xo2N92bNIC8vvERETk7dMhpgeEe9+uqrTJgw\ngaKiIqZOnZrkWikUinhxARKprjMFkV438yGGxpV6qZfXrw9JHuraVq94vpJ5bceVDMTFsRBJGmU3\nOa5QhKMhcCVwtva9NeLlpFAoaimjCPZqUSgUCoVCoVAoFIrEMxIJyNpJ8FwIiM/9Kgzf+htN6/Yi\n2kGbADt3mzuRtLV2PvfxKvc3iOvxxzgHPsaj7H7a902IIuz5HpfbBFii/bZSZC4L5Ny+DnyCpCRu\nYnNcr8vWvfQe0ZZtBpYBZziUnQjC/b54s5fg6yLUfzMFqet2vB0CnIOkXdhiWhZNPfpox9iJZDCN\nR71mIF6fm7TXqCTUqwBYA2xFrns9e1kqnLOUxU2w4Azg99rnPETJVPck24NzIFYB8iCy2yZe5Q5D\n/mzdB6q5zTbxKrsYGKF9HoVcjF6WOxe4WfucgfGQnokoxoI8MB+yqVu8yr4YyTGDVq5d2YnAze+L\nN3bXhdN/0wWpYyZS508xzmOsDAZ6EfiAjqQeugbY+xi5UP6JGGav6zUd+K3NtomsVyuM2KFGyBD/\nuSTgnCU5V1tMmIMFT2EEC5o5hEhfQ3QS2Iks9zbk4XdK+34kgWUfwnigNiE4gVYs5Z6B3HhztHWV\nwDHt8xjkwY72foVN3eJV9utIjxLgPURDKhm4+X2JwHpdOP03lyPpEk4hdf6U4IRV0fI2ou0VbT36\nI44QORg9pxexv65irRfY30uJrNdhDE+pb5EedBsScM5qsuGwCxa0qtTNBroiukObgTtM6/yIBPYG\nwCxkFE4CO17lng38CMm7Xgz0JZh4lT0ZeAwoQ4ZwrE4JsZTbHjGCzwMfaNtla+uc8s8nomwzNyOt\nrGTg5vfFG7vrwum/ySdQ+DHe9Y20HtblB+JYv98g19tzGMNByapXIdIreo8EnLOabDj8LrbRJbDz\nkS7d04hlBZHA7oUMzfwaaZlma/tMNx3D2qqIR7kgwyhNkfH/SUhSKyvxKvs5ZHy0HfBfGC10L8rN\nAHoDz2jv3yGGyorua263PJ5l3wucRLJPJgM3vy/eOF0XOk7/jXl9IghXj0TyR6Rhch7S430siXVp\nBCxFGkzfWNbF5ZzVZMNhzVHuJIG9WPtslsAGewnsDhgS2HswJLBbxLlctGMs0z6vR4ZRci3H9bps\nfRK8n/YdZCLZOvQQS7n7tdd6bd1S5CEORv55CMw/H6+yl5jKBplEHw38h025icLN74s3dtek039j\nrW9bgoc2vSSSeuzXlre1LI9H/b7AeCj/BeOeSXS9MpF7ah7wN21Z3M9ZTTYcG5DhnUIkWPAaYLll\nG10CG4IlsPVWuC6BvQVDAru99tqPPGjMD7R4lAvypw/XPnfSjm3N6+512R9r3z9FUvyi1eETD8s9\njAzFdNLW/RjxAkE7xs+0zz/DuPDjVfZFprJHIj27ywEjv2zicfP74onTNen03ywHxiN1bY/UPZ4i\nYJHW4zBwHBm79wHXY39dxUpr0+efYNzHiayXDxktKEVSUeik6jlLGeyCBRMhgR2PcjORVsMWpJcz\nNIG/uS8yNvoh8C4ybOFVuSAZ8tYT7PraDBlbD+eOG4+ydwKfYbhTPuNQdiJIZtBre+yvi1D/zT1I\nXbdjeON5wSvIPNVJxODfFGU9dNfST5H0117X62ZkAvkj5Lr6G4Hzc4mq1yBkZOJDjOt4JKlxzhQK\nhUKhUCgUCoVCoVAoFAqFQqFQKBQKhUKhUCgUCoVCoVAoFAqFQqFQKBQKhUKhUCSa/wdggpLQ5QVk\nTAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "N = 250\n", "x = np.hstack([np.ones((N,1)), 5+np.random.normal(size=(N,1))])\n", "t = np.dot(x, np.array([-1,2])) + np.random.normal(size=(N,), scale=0.8)\n", "plt.plot(x[:,1], t, 'k.')\n", "\n", "fit = stan_lm.sampling(data={'N': N, 'D': x.shape[1], 'X': x, 't': t}, \\\n", " iter=1000, chains=4)\n", "fit.plot(['w','sigma', 'R2']);" ] }, { "cell_type": "code", "execution_count": 51, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaQAAAGlCAYAAAC1EIKQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmUVOW56P/vHmrueQYbEJqplcEBJYoGjwgxIkS9aPRE\njSS5GX43cH5knSQeSHLzO2rMaDyRdbPiMvEENYMxMWI0hiM5YrwOEQmCMiM0NPQ8d1V1DXu/vz+q\nu+iiq6oLoburu5/PWq6Vrnp719uhaj/17v0+z6MppRRCCCHECNNHegJCCCEESEASQgiRJSQgCSGE\nyAoSkIQQQmQFCUhCCCGyggQkIYQQWUECkhBCiKxgjvQEhBgOSik6OzszGgegaVracXl5eYOOEUKc\nGQlIYlzo7Oxk8yt78Hp9acc1NzWgGyZFRcUpxwQCflZecwH5+fnneppCjGsSkMSolunKp6OjA4/X\niy8nL+04v78LXXcMOk4Ice5JQBKj2pmsfHw5+eTkDNPEhBBnTAKSGPW8Xl9GK59zRSlFR0dHRmPl\nXpMQmZOAJMQZCgS6+csbrWnvM8XGyb0mIc6EBCQhPgSPZ/BVmRDizEhAEmKIyKU9Ic6MBCQhhohc\n2hPizEhAEmIIyaU9ITInpYOEEEJkBQlIQgghsoJcshNZ6UwqMCjUMMxICDHUJCCJrDSeKjDIbjwh\nYiQgiaw13BUYRorsxhMiRgKSEFlAduMJIZsahBBCZAlZIYlhl8mGBdmsIMT4IwFJDLtMNiyMhc0K\nIyHT3YkgGyRE9pGAJEbEYBsWxsJmhZGQ6e5E2SAhspEEJCHGmEx2JwqRjWRTgxBCiKwgKyQhRolM\nEmhlM4gYzSQgiXNGyv0MrUwSaGUziBjNJCCJc2Y8lfsZKYMl0MpmEDGaSUAS59R4KfcjhDj3ZFOD\nEEKIrCArJDEouTc09kiFcZGNJCCJQcm9obFHKoyLbCQBSWRE7g2NPVJhXGQbCUhjkNQzE0KMRhKQ\nxiCpZyaEGI0kII1RUs9MCDHaSEAaxzLdaSW754QQw0EC0jiW6U4r2T0nhBgOEpDGuUx2WsnuOSHE\ncJCAJIRIKdPLukrFLulmsmNTdnaKVCQgCSFSOpPLurphSqKtOCsSkIQQaWV6WVfXHbKzU5wVKa4q\nhBAiK0hAEkIIkRXkkt0ok0lZIMkbEkKMRhKQssSZtHjY9o9avN7USUGSNySEGI0kIGWJM23xIG2s\nxWiU6TZy2Ro+PklAyiLS4kGMdZlsI5et4eOXBCQhxLCSPkwiFQlIQ0zafwtxZqS9+vglAelDOpeb\nEEA2IgjRJ9PqEH5/N9dcUpnRpT0JXKODBKQP6VxuQgC5NyREf5lWh/jLG4elXNEYMu4C0v99+z00\n0n9Tcjl1pp8/Me2Yjo4OPF6vBBohRlAmgetcFojNtIjsmRSblUB5yrgLSLV1zYR6etKOifR08df/\nuwuXy51yTEdHKx5vDgUFhWmP1drajK4b9ATSX947l+NG4jXP9TiZ29ibW6bjhuI1978fJjcv/Ym/\no6MVXTPSjstkzJmMC4V6+MKdHycvTzZ5wDgMSI01u9m4ceNIT0MIIQBwW42sWbNmpKeRFTTVt7Yc\nJzo7OzPajDAclixZwtatW0d6GnHZNJ9smgvIfAYj80kv3Xzy8vJkhdRr3K2Qsu0fv7KycqSnkCCb\n5pNNcwGZz2BkPull23yykVT7FkIIkRUkIAkhhMgKEpCEEEJkBePb3/72t0d6EuPZwoULR3oKCbJp\nPtk0F5D5DEbmk162zScbjbtddkIIIbKTXLITQgiRFSQgCSGEyArjKiB1dnbyo4cepr6hBX8wgj8Y\nIRK1RnpaQgiRVmdnJ/d8fh21tbUjPZUhNe4C0qM/+yldXVLwVAgxenR2dvLGthfH/LlrXAUkIYQQ\n2WtcByTT0DD0cf1/gRBCZI1xV8sOQNPA7TTQdU26SAohRJYYlwHJ5TQwDFkZCSFENhmXZ2VZFQkh\nRPYZlwFJCCFE9pGAJIQQIitIQBJCCJEVJCAJIUbc/fffz7Jly1i5ciV79uxJOuaNN97glltuYcWK\nFdx7771YVqzKSmtrK5/97Gf5xCc+wY033sgf/vCH4Zx63PHjx7n11ltZtmwZ69atIxKJnPPXyMnJ\nOefHzCbDFpDq6uq46667WL58OTfeeCObNm0aMGbz5s2sXLmSFStWcPvtt7Nv3774c6+++irXX389\ny5Yt49FHHx2uaQsxqvWdtM8127bP2bG2bdtGTU0NW7Zs4b777iNZRxzbtvm3f/s3fvzjH/P8888z\nceJEnn32WQCeeuopLrjgAp577jk2bdrE9773PaLR6DmbX6Z++MMfsnr1arZs2UJeXh7PPPPMOX+N\nsb4ha9i2fZumyfr166mursbv93PLLbewaNEiqqqq4mMmTZrEU089RW5uLq+++irf+ta3ePrpp7Es\ni/vuu4/HH3+c8vJyVq1axZIlSxJ+90xIxw2RbWpra/nc5z7HnDlz2LNnD9OnT+f73/8+breb9957\nj+9973v4/X4KCwv57ne/S2lpKU8//TRPP/00kUiEKVOmxMffe++9OJ1O9u3bxyWXXMK1117Ld77z\nHSB2QnvqqafweDx8//vf57XXXgPgS1/6EjfccANvvfUWGzdupLCwkIMHD3LhhRfywx/+EIBrr72W\nG264gddff53Pfe5z3HDDDefkb9+6dSs333wzAPPnz6ezs5Pm5mZKSkriY9rb23E4HEyZMgWAK6+8\nkkcffZRVq1ZRWlrK/v37AfD7/RQUFGCasVPb5z//eR544AFKS0sTXvOuu+5i9uzZvP3221iWxQMP\nPMC8efM+9N+glOKtt97ixz/+MQA333wzjzzyCHfccceHPuZ4NGwBqbS0NP6m8Pl8VFVV0djYmBBU\nLr744vj/nj9/PvX19QDs2rWLyZMnU1lZCcDy5cvZunXrhw5Ilg22Uuhj/NuGGF2OHj3Kgw8+yMUX\nX8z69ev51a9+xd13383999/PT3/6UwoLC3nxxRf58Y9/zHe+8x2WLVvGbbfdBsDDDz/MM888w513\n3glAY2Mjv/3tb9E0jS9+8Yv87//9v7n44osJBoM4nU62bNnC/v372bx5M62traxatYrLLrsMgL17\n9/LCCy9QVlbGHXfcwY4dO7jkkksAKCwsTHpJ7Pnnn+fnP//5gMenTJnCf/zHf6T9uxsbG6moqIj/\nXFFRQX19fUJAKiwsxLIs3nvvPebMmcNLL71EXV0dALfeeiv33HMPV111FX6/n4cffjj+e+mupoRC\nIf74xz+yfft2NmzYwPPPP5/w/JEjR1i3bt2A39M0jSeeeCLh8llbWxt5eXnovZVfysvLaWxsTPt3\ni4FGJDG2traWvXv3pv1G8swzz7B48WIAGhoamDBhQvy58vJydu3a9aFfXwFRS6FrCkOqNYgsMWHC\nhPiXspUrV/LEE09w9dVXc/DgQVavXg3ELsGVlZUBcODAAR5++GG6uroIBAJcffXVQOyEef3118ff\n15dccgkPPvggK1asYNmyZZSXl7Njxw5uvPFGNE2juLiYyy67jN27d5OTk8O8efMoLy8HYPbs2Zw4\ncSIekFKtilasWMGKFSs+9N9++lWL0z+Tmqbx0EMP8eCDDxIOh1m0aBGGYQDws5/9jNmzZ/PEE09w\n7NgxVq9ezXPPPTfo/Zbly5cDsGDBArq7u+nu7k74nalTp/LHP/7xQ/9N4swNe0Dy+/2sXbuWDRs2\n4PP5ko558803+f3vf8+vf/1r4MNdN33kkUfYuHFj2jG2As1WGIYEJDHy+r/PlVJomoZSihkzZvCb\n3/xmwPh7772Xn/70p8yaNYtnn32Wv//97/HnPB5P/H9//vOf55/+6Z945ZVXuOOOO3jsscfir5Hs\n9Z1OZ/wxwzAS7kP1P25/mzdv5he/+MWAxydPnsxPfvKThMeeeuopfve736FpGj/72c8oKyuLXw0B\nqK+vjwfE/i666CKeeuopAF577TVqamoA+Mc//sEXv/jF+OtVVlZy5MgR5s6dm3Sumfrggw/4yle+\nkvS5J554gtzc3PjPhYWFdHZ2Yts2uq5TX18f/+JwpjI5d41VwxqQIpEIa9euZeXKlVx33XVJx+zb\nt49vfvObPPbYY+Tn5wOxFVHf8hxSv2H7W7NmDWvWrEl4rLa2liVLlpzlXyHE0Dh58iQ7d+7koosu\n4k9/+hMLFixg6tSptLa2xh+PRCLU1NQwffp0AoEApaWlRCIRNm/enHDZq79jx44xY8YMZsyYwe7d\nu/nggw9YsGABv/nNb7j55ptpb29n+/btfP3rX+fQoUMfau4rV65k5cqVGY391Kc+xac+9an4z0uW\nLOHJJ59k+fLl7Ny5k7y8vITLdX1aW1spKioiHA7z2GOP8aUvfQmAadOm8cYbb3DppZfS3NzMkSNH\nmDRpEgCf/vSn+cEPfpA0OLz44ossXLiQ7du3k5ubO2BFNW3atIxXSJqmsXDhQl566SVuuOEGnn32\n2ZTnuMGM53PXsAUkpRQbNmygqqqKe+65J+mYkydPsmbNGn7wgx/Eb14CzJkzh5qaGmpraykrK+PF\nF1/koYceGqaZCzE8pk6dylNPPcX69euZPn06d9xxBw6Hg5/85Cfcf//9dHV1YVkWn/70p5k+fTr/\n8i//wq233kpRURHz58/H7/fHj9V/tbVp0ybeeustNE1jxowZLF68GIfDwT/+8Q8+8YlPAPDVr36V\n4uJiDh8+POx/9+LFi9m2bRtLly7F4/Hw4IMPxp/rvynhscce45VXXsG2bf75n/+ZhQsXAvCFL3yB\n9evXs3LlSpRSfPWrX6WgoADbtjl+/DgFBQVJX9flcnHzzTcTjUbjmz7Oxr/+67/yla98hYcffpgL\nLriAVatWnfUxxxtNDdOWs+3bt3PnnXcya9as+Idl3bp18ZXP7bffzoYNG3j55Zfj94tM04xvndy2\nbRvf+c53sG2bVatW8YUvfOGM59D3LeOlLS9z3nnnAWBoSKFVMeJqa2v50pe+NODGuvjwDh48yB/+\n8Ae+/vWvD3jurrvu4t577+XCCy8cgZmdub5z18svvxxf/Y1Fw7ZCWrBgQUJeUTIPPPAADzzwQNLn\nFi9eHN/kIIQQg5kxY0bSYDSadXd3j/QUhtS4bD8xVGxboeuyQUKcucrKSlkdDaMnnnhipKcgkhjX\nAcnQOSe5SJZtEwxZRKI2DlPH4zKkE60QQpyhcRmQNGLty882GCmlCEcsesIWdu+duEjUxrJsXE4D\nl8OQHCchhMjQuAxI52JlZFk2wVCUiDVwT4itiK+YvC5TNk0IIUQGxuWZ8lysWiJRO2kw6i9qKVkh\nCSFEhsZlQBJCCJF9JCAJIYTIChKQhBBCZAUJSEIIMUp0dXWN6X5uEpCSsO3B/8EVmb4pMjjWGH6D\nCSHOnTffq6ezs3OkpzFkJCD1o5QiGrXoCUexbDtloLBtG2UrBtvNbZo6UUulDHBKxZ6LWCrt6wkh\nBIDX6x3pKQypcZmHlIxt24T6Jbj2hCwcpo7D1ONbt2MByyYctQEt1twPG0tB/1hi6BoOU4/nH0Vt\nha4SmwEqpbBsFX89ywabWJCT8kNCiPFo3Ack21ZELZtI1B7wXCRqE43aOB0GmgahiMXpixhd19GU\nwlax4GIaiUEs/joKbEth6gCKJC8X62SbJHgJIcR4MO4DUiRqE7WSRIdeilggSkfTNAxNw2lo6IPU\nsItmcH/KVqArkHgkhBhP5B6SEEKIrCABSQghRFaQgCSEECIrjPt7SEIIMVq0t7fR0dEBQF5e3pjb\n+DTuV0iaNvgmA00bfIOBUopIxBo0lyhqWVhpNlH0HcsfjAx6LNu2se30xxJCjB1Op5O/72tj8yt7\nxmSC7LhfITlMA9NQhMM2VpIA0JeLBBC1bMKRgQHA6t02btmKcNTG7TJxmEbCGKVUrH9S2AINnA4D\nt8sc8A2nJxSlwx8mErXp7olQlOvG5Rx4rKit4lvQdWwMTbaJCzHWFZeUk5ObP2Y/6+M+IGm9J3KX\nS8OyVHyLt6aB22kkbON2mLGfIxELy1bxjrHRfn2RLEvhD0RwOWzc7ljAiUQsekJRrL4t3wpCYYuo\nZeN2mjgcBpZl0+kPE+iJxosNBUMW9RE/uV4nhbkuNE3Dsm1OX2DZdqyUkaGrQbedCyFEthr3AamP\npmmYpoaua9i2jWEMTG6FWBUG3WnQE4riD1kpywKFIrGAo2kkBKz+LCt2aY6eCD1hK+k4y4b27jDB\nUJSSfA9aiioOsaRa0JWNKR1qhRCjkASk0+i6hq4bacdomharzDBIkqtlq4ySWyMRO2XQ6hOK2CgN\nBjvc2FzICyHGA/kqLYQQIitIQBJCCJEVJCAJIYTICnIPSQghRom21hYUOsGgH6Umj/R0zjlZIX1I\n5zIPIOO2fJk08JNdDUKMWbYdxbYj2FZ0pKcyJGSFdIaUiuUfmaaBx61iTf2S7LbTNU5t0VacykE6\nTTgSpamjB7fTxO00kwYn09BwOQx6whZuJylzjTQN9DGaMCeEiCXGFpdW4O/uHJPJsRKQzoCtFJal\n4kHD5TRxOgyCPZGECg66rsWCQ//Aodm9Lct7j2Ur2rpDBEOxbzrBUJSeUJRcnzP+e5oGLoeB22mg\naRpKxZJlHYaNw2HE35Aa9HaalQWvEGL0koCUAdXbETZZCTpN0/B6nDhMi55wLLgkCwy6rvcGJJtA\nKEJrZ2jg6wCd/jAuh06uz4XHZSZNco1YiogVxe00Yq3SpbusEGIMkK/UGRqkHioOh4HTNNKuUmKl\nf1TSYNRfKGLjdhqDVlxQtsJMUVFCCCFGGwlIQgghsoIEJCGEEFlh2O4h1dXV8bWvfY3W1lY0TeO2\n227j7rvvThhz+PBh1q9fz549e1i3bh2f+cxn4s9de+21+Hw+DMPANE2eeeaZ4Zq6EEKIYTBsAck0\nTdavX091dTV+v59bbrmFRYsWUVVVFR9TWFjIN77xDV5++eWkx3jiiScoKCgYrikLIYQYRsN2ya60\ntJTq6moAfD4fVVVVNDY2JowpKipi7ty5OByOpMcYrIPqUMpk34DToQ+al+p06OR5k/99fXRNA5X+\n71VKcbSuk/oWf9pjRaMW3YHwoMeKSvdZIbJeW2sLzU2NtLa20NHREf9vJM+N59KIbPuura1l7969\nzJs3L+Pf0TSN1atXo+s6t99+O7fddtsQznDga5t6rFBCNEUSrKFraGasU2xPOEooSWdZhxEbM6Ek\nh/yeCCeb/QMSZgt8TnK8TgBsy0bTtQE79zr9YY6c6KC5owfT0JhUlsvc6cU4HafaZvS1QQ+GoigF\n4YiNz2Picib+k9u9Df8UYBPrpyTbyIXITn2VGlyuWCtzTWsnEPCz8poLyM/PH+npnbVhD0h+v5+1\na9eyYcMGfD5fxr/361//mrKyMlpbW1m9ejXTpk1jwYIFQzjTRLHOsuDQYlUX+uKI2ZsE23cC13UN\nj8vEaSr8PRFsFUtaNU4LKl63g6kT82nv6qG5oweXQ6Mwz4PZb5ytAEuBsntLAmkcOdlBXbM/HvCi\nluJIXSfNnUFmTipk6sQ8wmGL7mA4ocdS1LLp6A7jclrkeBzoupbQBr3/a9pWrPvs6XMWQoysvkoN\nY9WwBqRIJMLatWtZuXIl11133Rn9bllZGRC7rLd06VJ27dqVNiA98sgjbNy48azmm4ymaZiGht3b\nfC/ZSqKv+2yez0mgJ3XNKUPXKM734HWZpOv1F1tFKd491EJHdzjpmC5/hB37GolEohTkulMeKxS2\niERt8nKcpCt8Z9mAinXOFUIMn6E6d40Gw3a2UUqxYcMGqqqquOeeewYd218wGKS7uxuAQCDAa6+9\nxsyZM9MeY82aNezfvz/hv61bt57V39CfnsFlrb5V1WAyOekrBT2h9AUVFZkValVKZTRQLtsJMfyG\n+tyVzYZthfTOO++wefNmZs2axU033QTAunXrqKurA+D222+nqamJVatW0d3dja7rbNq0iRdeeIHW\n1la+/OUvA2BZFitWrOCqq64arqkLIYQYBsMWkBYsWMC+ffvSjiktLWXbtm0DHvf5fDz33HNDNTUh\nhBBZQG4QCCGEyAoSkIQQQmQFaT8xxDQ01CA7CDLdO+ByGEnzm/rzByMUptllB7Et4KGINSAn6XS2\nbaNpUk1ciGzR18K8v2DQT0dH8go2eXl5o+rzKwFpiLldBuGoTTSaPJBYto1CwzA0VL/8pv603hyk\nudNLqKnrpKEtSOS04zlMnUjUZvcHrbR0hpg9pQCXY+A/b3NHgDffq8ey4doFkzivLGdgl1mliFo2\nSoFhKZwOA10fPW9qIcaqvsTY/vonyfY3GhNmJSANMU2LtR83dY1I1I5XZlBKEemXuKrrOkpTaEph\n93al1bTYdu++XfCGrjHtvHxK8j0ca+iitSsUrxLR3n2qx9LJZj/1LX7mVpUwodiLpmmEwlF2f9DC\n0bqu+Li/vFnD+RNyWXjhBHJ9vdUhbBur37wsWxEMRXE5dAzpvSTEiJLEWHFOGIaOrmuEwhahsEWy\n9VIsb0kDbOwkVRT65OU4udBXxIHj7Zxo8hOKWAPG2ArePdRMTb2TiiIv/zjYjJ1k+XW0roua+i6u\n/8gUygq9KecfithoURu305TVkhBiSMimhmGkabFLc4OVMNX1wVcimqbhdZtJg1F/7d1h9h1rSxqM\n+igFTe3BQWZFygAphBDnggQkIYQQWUECkhBCiKwgAUkIIURWkIAkhBAiK0hAGma6puF2pv+/3TQ0\n8rwOjEF2s02dmM+86SVpxxi6hkb6quMa0NDip6auM2XnSaUUDa1+9h1tJZxmI4Vt20QsGytN99m+\nDrURyx4znS6FEGdPtn0PM8PQ8Rg6TodNoCea0ERPA7weE0dvvk+Rw6AnFKUrcFoinEPH5TLRdZ0L\nphZRWerj73saaO7oiY+xbRuXwyQYimKrWENAlMJ/Wn+mfJ8Tl9OgOxhl16FmGtsCVE8tIsfjjI8J\n9kQ51tBFW1cs16m9O8ykihwqS3PiuwGVUlhK0ReHLAW2sjE0Ejre9u9QCxCRZoBCZCxZpYZU+ldw\nGC0VGyQgjRBD18nxOIhEbfw9UZwOHc9pOT6apuFxO3A4DLr9YSKWjc/jwDSNhDH5uW6uvWwSx+u7\nePP9egxdw0ZPCD7h3pJDeT5nb1tzRVGei0hUxas+KKCuJUB7V4jJFXlMn5TPySY/TW1Bwv0qQwRD\nUQ7UtNPS3sP0yny8bjMhyPRRCqIKdGx0ervRJlkQWXby4CWESJSsUkMqfRUcgsETo6ZigwSkEaRp\nGk6HgWFo6Frqhn+moZOX4yRiqZRjDF3n/In5nGz2c/B4R8rXDIYsTEMnx20SiSa/XBYMW+w/1kan\nPxyvLJFMS0cPUctm3owS0nWftW0Gzb1SKraqkngkRGofplLDaFgZ9ZGAlAUyuVyVaffZAXXpkrCs\n2OW1wQxWFBb6wtA5esOPns+NEGIIyPdRIYQQWUECkhBCiKwgAUkIIURWkIAkhBAiK0hAGkUG3a+g\nFAW5TrxuI+0wj9vAYQ6enNvRFWawThNdgTAd/XoxJZ+WIhq1pFy4ECIt2WU3SmiahqmTkHzan2Xb\nBAIRKop85Hic7DvaRl2LPyHvx+kwcDk0bFsRjti4TB2lncpR6qNrcKyhi56wTa7HwawphZiGPmBM\nU3uQmvoutu9r5J8unUT1+YUJOVKgiEZt/MFIrPusoeH1OJLuKjT0zHYICjGenUlibJ/R1OJcAtIo\nomkapqZhY2Opvm6yinA4SjB0qpxPjsfBguoyjjd0cfhEJ53+MD6PibJtlDqV7xTqTXZ1uwxCYQtD\n1+noDtHQdqo3UlcwwvZ9jUydkEtFsQ+lIByJsr+mLd7xVin46/bjvHuwkWULz6e00INSip6eaEJC\nrWUpurrDeFwGTqcZ38ouCbFCZOZMEmP7jKYW5xKQRiFd19GBnlCErkAk5ZWwSeW5VBR5eXNPPaGw\nhZbipN8TsghHLU42dSatpABwpK6L443dlBd5aGrrSTqmpSPEr7fsZ8VVUynKc6ecfzBk0RO2KMx1\nD1h5CSFSG+stzOVsMIrZavDbMg6HgdflGHRZbll2ymDUJ2opuv2Dfzsb7J4SxOatG9lzqUAIMfIk\nIAkhhMgKEpCEEEJkBQlIQgghsoIEJCGEEFlBAtIo5naa5OU40xbJdjp0FlSXcV6JL+UYn9fBhdOK\nWTC7NGUirG0rOrt7OHyinVAo9cYGp6mx61Az7x5swk6xSyIStThS18Hf36tPuwEiGIrS1hEkEAxL\nZ1khxgHZ9j2K6bqG22niyNcJBCMEw6dykXQN3K5Yro/DNLhkdhmT24Ns39cYb8inabGt4V6XiQJK\nC70suayS/UfbOdbYHT9WKByhrbOHlt6OtB3+MOWFHvJzPQlt1n0ek1DYor07zM6DzdS1+Jk3vZSJ\nvcFQKUVzR4C6liCB3uaBHe/VM6k8h9lTiuLNCaOWjT8QIdTbKj0SjBKK2Pi8Dpxm+ioUQojRSwLS\nGGAYOjk+Jy6XTac/jNPUMU7L79E0jdJCL9ctmMThEx20dAYpKfACiZ1eTcPgwqpiKityeGNXHa2d\nPTS1+uNJsAA9oSg19V0UByOUFHgpynOhaRr+YGJ79IbWINt21DJ1Yh4XnF9IQ1uQls7EFVEoYnGo\ntoPmjh5mTS4gx+2It13vLxK16egM4XYZ5HidWZVdLsRw+TCVGlLJxhbnEpDGiL7uszmeWDvxVBwO\ng+mVBegntbTt9/J9LizL5mRTd8oxLR09WLaN1+1IOSYctdl/rB1NS/967V0hjtd3MbkiL+UYRSyp\n1utWGJLDJMahD1OpIZVsbHEuAWmMiX3LSX+/JdO7MWYGJ31NZRYYNC2D2qpZ8A1NiGw2FJUasmFl\n1Ec2NQghhMgKwxaQ6urquOuuu1i+fDk33ngjmzZtGjDm8OHDfPKTn2Tu3Ln84he/SHju1Vdf5frr\nr2fZsmU8+uijwzVtIYQQw2TYLtmZpsn69euprq7G7/dzyy23sGjRIqqqquJjCgsL+cY3vsHLL7+c\n8LuWZXFEtLNyAAAgAElEQVTffffx+OOPU15ezqpVq1iyZEnC7wohhBjdhm2FVFpaSnV1NQA+n4+q\nqioaGxsTxhQVFTF37lwcjsSb5Lt27WLy5MlUVlbicDhYvnw5W7duHa6pj0mZ3Ec6V2NGguQtCTH6\njMg9pNraWvbu3cu8efMyGt/Q0MCECRPiP5eXl9PQ0DBU0xvVDEMftMuraWjk+Zxpx7mdBpdfOIHK\nspyUY1wOg6I8N7leB2m7SKgIO987jG1ZKYcYOhxv7KS5I5hyDMRylJrag1hpthIqpYhEbSzblsAk\nxCgy7Lvs/H4/a9euZcOGDfh8qasH9PdhdoE88sgjbNy48Yx/b7QzDR1D1whHbaLR5CdtTdOYUOzD\n73XS0hlMaO5n6JCf46KiOIfpkwq5/MIKnv6v/bzxXh3BnlN5RuXFXiYW5+DzOrEVeN0ObFvh7zfG\nNKC+voHX3noXpRT//fq73LbiaiZWlA5YWR2q7aA7GGXPkXbmTy9h/owSXI7EJNjO7hDNvXlMJ5v9\nzKgsIMd7qrWGUgrbVoQiFkpBJAoOQ8fh0LNqJ5EQ6YzXcxcMc0CKRCKsXbuWlStXct1112X8e+Xl\n5dTV1cV/rq+vp7y8PO3vrFmzhjVr1iQ8Vltby5IlS85s0qOQpmm4HAamrhGO2NgpVgk+jwOv26Sl\no4f27hBup0FZkRefxxkf43aa3L38QhbOmcDv/3qQk03dTCzNobTQm3CSt1Xs8l2ez0GwJ0owGOBv\nb++iufVUl8pgKMwvn9nK/Orzufaqi/F4XLR29HC80Z8wr3cPNbPnaCtLL5vExBIfkahNXUsAq1+2\nbDhi8/6RVsqKPFSW5mAaOuGohWUl/q0RyyZi2bicBoauSWASWS/duetcJsb2SdfivL/hSJ4dtoCk\nlGLDhg1UVVVxzz33DDq2vzlz5lBTU0NtbS1lZWW8+OKLPPTQQ0M427HBMHTcuhYv05OMpmmUFHgo\nzHPicqZu5DdrShH3fvpynnppD6FI8gCnaRqWDUePHePvO/akfM139x5lz8HjrPj4EkLRVPXubF58\no4Yr51TgSZN429gapKMrzIxJ+Wk/LKGwhdtpSEKtGNXOZWJsn1QtzvsbrnbnwxaQ3nnnHTZv3sys\nWbO46aabAFi3bl185XP77bfT1NTEqlWr6O7uRtd1Nm3axAsvvIDP5+Ob3/wmn/3sZ7Ftm1WrVskO\nuwxpmoauMWg3WIdpDPrtJ1Y7z0EoEk47Ttmp7xX1sWw7YcWTStROU3aiV6oVoBBjzVhvYT5sAWnB\nggXs27cv7ZjS0lK2bduW9LnFixezePHioZiaEEKILCCVGoQQQmQFCUhCCCGyggQkkTGlFHoGu2wy\nGQOx+0iDiYQHv4GrlMrofpTkJAmR3SQgjQNul4nDTP1PrWsM6J90Osuy8QfDfPTiiVxwfmHKcaWF\nHq5dNJ/Vn1xKjs+TdIzX7aJiwkR27z1Ce3tn0jG2ZfHB/l089H828ds/vEQg2JN0nD8Y5sCxNv7y\nZg3HG7uSjlFKYVk23cEogZ6IBCYhspS0nxgHYl1jdQxDIxS2EtpAmGb6pFGlFD2hKOHeZFOHaXDB\n1GLOK83h7b0NtHfHdtx5XAZlhV4sW6GA8tJi/tenb+Sd3Qd5+W8748erKC8nogzauyJAhLaOAJMm\nFDGhogSXywVAa+NJDu7fw/ET9QC88vo/OHy0lusWX85ll8xF0zSiUYvG9iAtHUEivVvHX/3HCaZU\n5LGgugy3M/bWtmw7liPV+zeHIjYRK4LbaQxIvBVCjCwJSOOEpmkYmobHpRGxbCzLxjDSn5AjEYue\ncHRAsinEqjlce+kkaho6qWv24zD1AZfNNN1gwfzZzJx6Hi/8dQcR2+gNYKfyoqKWzZHaZto7/ZSX\n5NHedJzDhz+gJ5R4qe74ySY2Pf1ndu85xLKliwmGdbqDiWOiluLwiQ6a2oPMrSrm/Al5Sbe727Yi\n0BMlErXwuEwMXS4UCJENJCCNM5qm4TB0YPD7POHIwMoH/em6RlGum4bWYNoutXl5uRQUFnKwpjnl\nmLbOIB2tDTTV1aQcY1k229/dz0WXXIZmulKO6/SH6Qlbg+ZeRaIKr0sSZcXoMRSVGjKRrJrDUFRu\nkIAkhkUmGx0yfnNnMkxuE4kxaCgqNWTi9GoOQ1W5QQKSEEKMEmO9UoNcPBdCCJEVJCAJIYTIChKQ\nxFnJ9JZmJqk/maYHZZJQa2d4EymTnCTJWxJieEhAGqcG2+msAT73IAm1OlSUeJkxKR+XM/kWcg0o\nyHHx8atnM3tqacpjFeX7mHR+FbNmV6dMqHU5HZQWF/D6G29jR0Np5/X8q4d471ATdoqtdrZS1NR1\n8sqOWvzB1DeJbVsRtRS2dJ8VYsjJpoZxSNM0TE3DxsZSA1cmug6GpqGZJg6HQTAUq3DQf2Hichjk\neB0Yhk71+S4qS3PYc6SNhrZAfIzXZVKQ5yLX46S8yMv0yaVsfWM/f9txhJaO2Di306S0JB/TlYtp\nGjg9s/DmV9B88gDHj5+IH6ukKI+esEVLR4CWjhoOHznOsmuvYvKUKfEt56ah0d7Zw4599dhK8cjx\nNi6aWcqt182mpMAbP1anP8Tru05S29sYcPfhFm64cgpVlQXxnKS+ckR98Sxqg4bCMDIvjSSEODMS\nkMYxXdfR6etNBJoGhhZ7vI+maXjdDlwOg+5ghKhl43M7cLsS3zq5PheXX1hOTX0XH5zowOXUKcr1\noOtav9fTWLpoNhdVV7L5v9+jtrGTnNx8DEdiTpHLl8/E6QvIKyilrvYIugYtHYHEDrW2zUsvv0pF\neRlLrrkSl9vHzoNNNHcEE46180ATuw4186nrL+DiWeXU1HXyxu66hAt6kajNc68eYeqEXJZcPpl8\nn4tokpWVIpZ8a+ixmn7SfVaIc0sCksDQdXRNpT3BGoZOfo4LpVKP0zSN8yfk4XUatHWnbuJXWpTD\np2+6nF88/z6BUPJmfpqmkVt6PlY0xIGDh1O+Zn1DI7/9w5+ZWn15ykRY21Y88eL7HDjWnjZZ9khd\nF399+zgrPpq++aNlg6bHArgQw2mkEmNPl6rt+dkmy0pAEkDmSamZjBuJYwkxHoxUYuzpkrU9PxfJ\nshKQhBBilJDEWCGEEGIYSEASQgiRFSQgiXPOMDLpKgslBe5BxxUW5A06JhoO0NPdknaMUooTxz4Y\nNJcoGIrQ5U+d49R3rEg0+WYMIcSHJ/eQxDlXmOcmx+uksTWAvyc64Hmv28TncfDZFXPYfbiJP7zy\nwYBeSj63wWXVFXjcU7jiktk8++J/c/RYXcIYpRSqu4bWE3tp2v9fTJv7T5TOvBbttKzfUHcTzYde\n460jB6ieM48FV99IQXHFwGMp2H+sg+MN73HF/IlcMadiwKaKSNSiJxTrEeVyWPFcLCHE2ZOAJM45\nTdNwOgzOK8uhOxihrtmPUrE8pIJcF2bvCdw0NS6eVc60iQW8+MYRdh9uBWD+9GIqy3LiW7Tz8vP5\n9B2f4MChIzz9x5exLAs73EFP016OHTkQX/U0//U3VJ3Yw+T5N+IrmYZtW3TUvEntoZ20t8d2A+3a\nuYMTx2u46LJFzL1sKbphoJQiHLHj3W+DIYsX/naEAzVtXH/FFCqKfSilCPbEOuf2CUUsIl0WXpeJ\nx+2QHYFCnCUJSGLIaJpGrteJ5zyD9q5Qyg61+bkubl86i4tntFLXEsA09ST5Qhozp0/jq1++i0d+\n/CAH971FR0fHgGMd3r+Lxroapsy+DGzF8ZoPBoxpaWlh60ubOX7kANfcuJqIchKOJtbHU8CBY+2c\nbO5m2eWTmTm5MGkZItuG7mCUUMQmP8eZkFQshDgz8ukRQ840DFzO9N99NE0jP8eFmaZ2HoDL7SHQ\nfjJpMOrT1dlBe/3RpMGovwP79xGKqgHBqL/uQJSoZaesidcnErVlhSTEWZIVkhBCjBLZUqkhmVTV\nGyDzCg4SkIQQYpTIlkoNySSr3gBnVsFBApIQQowSUqlBCCGEGAYSkETWMMzMNgXoevLdev1Zdvrq\n5bHjaEQjqauS9wn2ZHaJRBr4CXF2JCCJYeF1mzjSBBzT0Jg2MZ9500vI8SS/kux06JQXevju93/E\nxz52PU6nc+BxTJOq2RdRXr2MqjkfobikJOmxigrzyPE4+dNTPybSXZ90jKGDHe7igUee4s/bdhAO\npw5MgVCE2sZuQuGoBCYhPiS5hySGhaHr5HichCMWPWErXplB18DlNHA5jHg/pYklPt4/0kJds5+o\npdCA/BwXhbkunA6DgtxJfPcH/8GLLzzHpv/8Ofv37wdgwsRKSqdejrN4du/qqIgCdzm5Bfs5eeww\n4XAYl9NJXq6X2uPHUCq23XvTIxu4aun/4MLLlhJVDgCchsWhgwc4cvQoABt/+Wf+9PJ2vvK5FVRN\nrog3Q4paNl2BU4HqaF0XpQVu8nNd8e6zQojMaGocfZ2rra1lyZIlbN26lcrKypGezrillCIYimIr\nhcdppiy909gW4MCxNjxOkxzvwNUQxHbwPPTD77F7/zG8lVfgcHqSv2awns4T79He3khHW3vSMV5f\nHjd88n/h8OTzzo4d2Hby/KRbl1/BrTdciVI6ESv5x8c0NCaW5uBxyXc+cfb6zl3f+sEvR92mBn93\nJ9ddPiWjXXbyFU4Mu7626DkeZ9o6cGWFXqafV5AyGAF4vT6+uOZrFM9YkjIYAWieCpxuT8pgBBDw\nd/LXF37F29u3pwxGAL974Q1ONLSnDEYQa3XeFRj8/pQQ4hT5+iaEEKNENifGptI/YXawBNlhC0h1\ndXV87Wtfo7W1FU3TuO2227j77rsHjLv//vt59dVXcbvdfPe73+WCCy4A4Nprr8Xn82EYBqZp8swz\nzwzX1IUQIitkc2JsKn0Js8HgiUETZIctIJmmyfr166mursbv93PLLbewaNEiqqqq4mO2bdtGTU0N\nW7Zs4d133+Xb3/42Tz/9dPz5J554goKC5KUphBBirBvNibGZlA4atrVfaWkp1dXVAPh8Pqqqqmhs\nbEwYs3XrVm6++WYA5s+fT2dnJ83NzfHnx9H+CyGEGHfOOiC1trae8e/U1tayd+9e5s2bl/B4Y2Mj\nFRWnon9FRQUNDQ1ALLquXr2aW265JWHVJMY2n8dEH+SLVZ7PyRUXVaUdYxg68y+7CtOReoMEQDjQ\nQrgreV5SHysa4rnn/kg0OrD5YB+lFIdr22lqC6Q9lm3bRG1bvmwJwRlestu2bduAD85rr73GN77x\njYyP4ff7Wbt2LRs2bMDn8w14PtUH81e/+hXl5eW0trayevVqpk2bxoIFC1K+ziOPPMLGjRsznpfI\nTrk+Fx63g5aOIO1dA3et5XodmLku1tx5Ddd+ZCYP/efWAS3IL5w5mbnV56MwmDPvErZs/jWvv/py\n4oGUTY7LpunY+3TUH6Jy+kW4J16OYZ4KYEopjFAj3Y0f8Pi7L/GPHdv5zD2rufS092F3MMKh2nbq\nmwPsPtRC9dRirpxbkbCjUCmFpRR9m/kUCkNX0k9JjOtz1xnlIf3sZz+jrKws/rNSirfeeovvfe97\nGf1+JBLhi1/8IldffTX33HPPgOe/9a1vsXDhQpYvXw7A9ddfz5NPPknJadn2GzduxOv18pnPfCbT\nqQOShzSaKaXoCVnUNXcTsRRup47H5RgwLhAM8adX3uN3f9lBrs/DNVfOxef10f9NrgH1xw/wy5/9\niPa2FnLdOqFAG01NiZeQz5s0lZKpl+MorEJFAtidR6g99kHClvCcHB/XLbmWL33p/8Hr83HkZCfH\nG7oIhqyEY5UXefnInHKmTszHtm0sG5J98DQNTF2T3koiwWjOQ+qTST7SGa2QbrrpJsrLyxMeW7Ro\nUUa/q5Riw4YNVFVVJQ1GAEuWLOHJJ59k+fLl7Ny5k7y8PEpKSggGg1iWRU5ODoFAgNdee40vf/nL\nZzJ1McppmobHbTJlYh6tHT2k+hrl9bi47eOXMGPqBI42BJKe+BVQPmkmX/v/fsKPvnEPRw7tTboy\nP3H8CE2NJ5ky8xJspdHePrApYHe3nz8+9zy733+fz639d05bnMU1tAZ48fWjLFs4mSkTUn8glYKI\npXAYmd0EFmIsyTgg/fa3vwXg0ksvZfr06fHHTw9Qqbzzzjts3ryZWbNmcdNNNwGwbt066urqALj9\n9ttZvHgx27ZtY+nSpXg8Hh588EEAmpub4wHIsixWrFjBVVddlenUxRhi6DqGrhFNk5QKGl6PG8tO\nf/8Gw0U4FEx7/yYcCkE0RHtXikjT61jNid5VUerCr5GoQtfkkpwQqWQckGzbZunSpQSDQbZu3UpZ\nWRlz587N+IUWLFjAvn37Bh33rW99a8BjkyZN4rnnnsv4tYQQQow+GQekK664In4vZ9KkSWzZsuWM\nApIQQoizMxorNbg9LjQ0AgH/oGPP6JLdzp07yc/PZ9q0aYRCIZYtW3ZWExVCCJG50VapIRgI8NGL\nZsU3MuTl5aUdn3FA+vrXvw5AW1sbBw4c4MSJE2cxTSGEEGdqtFVq8Hd3kp+fn1GlbziDxNhNmzbx\n85//nGAwyMKFCzFNqcsqRoaewe4zr9uBw0z/9naYGrl5g39QLHRcLlfaMTm5PiKR9BsfAJrb/YMm\nwSqlUu4iFGIsyzgguVwupk6dyo9+9CPuvPPO+O44IYZbfq4Lr9skWVzSAJfDYM70EpZcNpmyouQt\nKXI8Drxuk68+8AtW/I+7kn6DKy4p46Krb2HS5XdTddHHmFg5ecAYXdepnHw+nuIZ/Oa3v6O9pT5p\nZQlTh9qT9fzbQ3/ksd+/SWd3MOm8LMsm2BOlMxAmErWkgoMYVwZd5liWxcaNG1m+fDktLS386Ec/\nGo55CZGSpmnkeJ24nAbdgQiRaCxR1TA0PC4T04xtva4sy2FiiY8d+xs5cKyVQI+F26Hj7m2ap2ka\nbreXOz6/gUsXfYw/PrWRne+8iWEYVFUvoLT6epw5sURwPa+SIl8FOQW7qKt5n66uLkpKSvAVTiSs\n56NpGqFQiD/8cTOzZ83kio98BM10o2sQCAT42/a9BIOxFdTvt7zLn/+2l69/bgkLLpyEoesopQhH\nrFPb2RV0B6M4DR2325Dus2JcyKhSw6pVq8ZEuwep1DD2KKUI9ESw7Fgr9FTJpG1dPfxtxwlCkWjK\n8jy2ZfGHX/0fjtX7yalckPJYVrCFzpO7iOgF6ObAahEQWzktWbIEfxgOfJD6asKVF53PuruvQTdS\n5y8B5HhMHGb6MWLsGq2VGs6kWyxkeMnummuu4bHHHqOlpYVgMBj/T4iR1td91u0y01Y2KMx1U1bk\nSVsrTjcMrlz2z+ROuiztsQxPMeVT5qQMRhDL23t39560wQjg9Z1H6UhV3qGfqJW6g60QY0VGOxP6\nCv398Ic/jD+maRp79+4dmlkJIYQYdzIKSJlUWBBCCDG0sj0xti8Jtk8mybD9yd5tIYQYJbI5Mfb0\nJNg+gyXD9icBSQghRolsTow90yTYZLJ37SfEGRisqyxA9flF5PtSb0QAuKS6kk9cOy/tmAllBVx0\n8aXk+Lwpxyhl09FwCOVPv6nBaVj8/Dcv0+1PvUkoErXYe7SV+pb0lz8iUZtQOJo2d8m2bSKWndDT\nSYhsISskMeppmoZpaNi2wrLVgP5HugaGrlFa6GXZwvM5crKdd/Y3JVRDKMxzMXViPg7TYMrEAi6d\nM4mf//51ak60xccYus6Vl86ioCCPqAWrbv0kB/bv5fU33kx4PZceIhps52RjI466o5w3qQqzaBaG\n81SHZGVZ+FxR2ts72PK3evYdrmXVDR/hYx+9OGGHX32Ln5NN3fh7opxs9jOxOIcLpxXhcp766Nq2\nTThiY9mq9+copmkkVKo4vUNtVIGmbGkGKLKKBCQxZui6hqaBrRSWHavaYBhaQqkh09SZMbmIipIc\nduxrpL41wKzJheTlJJYGmlhWyL/9z+t5+72j/OezbzFtcgWzqiqxbI1obzNYS+lUzbyQKZOnsG3b\nNk6cqMWj+2mqqyccjrVbj0SiHP1gP0VtjRRNnIWWOxmfUxGx/DQ0dsdf79jJZv7j8Rd5c8cBPnf7\nUooL8zla10FLR088wNo21DZ109bVw7Tz8plSkUvUUvHE4D62gnDEwrJsnI5YUEraqLC3GaChK0m8\nFVlBApIYUzRNw9A0NE2hkbrraq7XydUXnUdNfVfSVuIApmlwxUVVuNxeauq6SZUKZLpzuHbpx3jq\n0e9york56ZjWtjZa296kag402y5se+CrWpbN6zsOcOxkK//z7ptSNiH090TZfbgFj9Mg15e6xp5l\nK3rCVrxyRepxoGGnzdESYjjIO1CMSbo2+KUoXdcwjMEvV7mcDpLEjwTRqEU4PHiCq8thJg1G/fl7\nQoN0xO11Ti+1yWU7MfIkIAkhhMgKEpCEEEJkBbmHJIQQo0S2Vmpwe1wEA4GzPo4EJCGEGCWysVLD\nqQoN559RVYZkJCCJcc1pGkStaNoxpQUeaj3ddAdTnwgK8rxMnjSJ997fk3KM1+PBssHndeEPpN4A\n4dAV3Z0d5KTpZqtrcKKpm5mTCtHTZAVHorH97+Yg7S1im8JlY0O2y8ZKDeeiQkMfCUhiXCsv8tAZ\nCNPRHR6ws81p6hTkuJg2MY8ZkwrZtqOWD052JCTUOk2dsiIPE4p9XLjhG/z5hT+x5eX/oqGhKeFY\n51VOxls6CzwlODSbMm8PTc1tCVvOfR4nRLp4/50dfHPna9x9951cfMkC0BKDSTRq897hZlo6Q1RP\nLWT5lVMpynMnjFFK0ekP09IZQtNgQrGPHK9jwM5DDTB0ZMu3yAoSkMS4pmka+T4XPreD1s4euoNR\ndA1yvA6K8tzxpNrifDc3X1PFrkPNbN/bQGtniKI8FxNLfOR4nfHjLV+xkssWfoSnf/Mr3np7Oz6f\nj+KJMzDyqtB6T/qW0gnYHkrLnIR7uuno9JPvMzhZswd/V0dsjGXx+OO/5JX/3sZdd99FSdmE+Kpo\nz5FT1SP2Hmlj/9E2PvHRaVw0sxSHaRAKR6lvDcYrNygFJ5v9eF0G5cU+nI5YgNN1enO2ZGUksoME\nJCEA09ApK/Ti80TQdQ2Pc+BHQ9M05s8oZdbkQl5++xh5PmfSk3lZWRlfXvv/UvaHP7H7g3Z0Z27S\nYwWjJsrIxbTqOfjegaTzOnL0KP/+7/dx9+ovEDZLCIYHZufaCp7d9gGv767jk9fNJBCykh4rELI4\ncrKT8yfk4nObsioSWUfekUL043M7kgaj/twukykVuYOuLKbPnJM0GPWnaQYq0jXovA4frU0ajPpr\naA3SFRj8hrdlKQlGIivJu1IIIURWkIAkhBAiK8g9JCGEGCWyKTG2r135mbYpT0cCkhBCjBLZkhh7\nervys02I7SMBSYgPoaqyoLdfUfIEV7fT4PaPz2f21CJ+9J/biKboXTFj2iSc0yfy9msvUX/iaNIx\n+cXlhLRcgkE/brc36WYKpRQ+j4P/evsYCy+ooPC0vKQ+PaEIuw81c15ZDlMn5qU8VrR3y7ihSY5S\nNsmWxNhzmQzbnwQkIT4Er9vB7POLaevsYX9NWzznB6DqvHzKiryYhs7HFlVz0exKHv3dG/z33w/F\nxxQW5FJ1/iRC0VhAuPr6O2g+eYhtW57F7q0coWk6sy+9BmfueXT2WHTWNlNRkktuTg6m41Tuk8PQ\nME2D9u4w7d1hmtqCXDC1iItnlsWrOFiWTWcgTJc/jGUr2rpDNLQFuOD8IvL7NSe0bDuh71NUgY4t\n+UpiWEhAEuJD0jWN4nwPC6qdHG/oIhiKMu28ArzuxI9VeXEu6z9/HddfNZvvPPoylZUTcbt9hPp1\neg1HIa9sOjffuYbd21+hq9PPxOmX0BXSCfbLK6pv7qKjO0hpUR4ul4+8HCf+nij+0KnLOF2BCG+9\n38CJpm4unV1GYY6bDn+InnBiflJTW5A3u+uZVJ7DzEmFKI2EKhR9bBsUsc6ysloSQ0neXUKcJafD\nYNp5+VwwtXhAMOpj6DqXzZnMymWXo5sewtHkl/CiuJl58XUUnX8ZXaHkH89gT5RjJ1sxdJv27vCA\nFuZ9ahv9vLO3gYa2wIBg1CcUsThU20FXMJI0GPVRQCY9A4U4G8MWkOrq6rjrrrtYvnw5N954I5s2\nbUo67v7772fZsmWsXLmSPXtOFap89dVXuf7661m2bBmPPvrocE1biIxompa2yGkfl8Mx6BjLUkQy\nOPv3lQAaZGYZjMlslFywE0Nt2C7ZmabJ+vXrqa6uxu/3c8stt7Bo0SKqqqriY7Zt20ZNTQ1btmzh\n3Xff5dvf/jZPP/00lmVx33338fjjj1NeXs6qVatYsmRJwu8KIYQY3YZthVRaWkp1dTUAPp+Pqqoq\nGhsbE8Zs3bqVm2++GYD58+fT2dlJU1MTu3btYvLkyVRWVuJwOFi+fDlbt24drqkLIYQYBiOyqaG2\ntpa9e/cyb968hMcbGxupqDi1pbGiooKGhgYaGxuZMGFC/PHy8nJ27do1bPMVQohsMByJsX0Jr+mc\ny2TY/oY9IPn9ftauXcuGDRvw+XwDnlfp7qwKIcQ4NtSJsacnvKZzrpJh+xvWgBSJRFi7di0rV67k\nuuuuG/B8WVkZ9fX18Z/r6+upqKggGo1SV1eX8Hh5eXna13rkkUfYuHHjuZu8EGdJKcXsKQUcqeug\nsTWYdIyhQ/W0Ygq9sPNA3YCmgX2K8jz0hKL43C78Pck73pqGRmd3D/5ACJ/XlXRM37gjdZ3MmlyA\naSbfKKGUoisYweMycTkMyUkaQunOXUOdGDtUCa+ZGraApJRiw4YNVFVVcc899yQds2TJEp588kmW\nL1/Ozp07ycvLo6SkhIKCAmpqaqitraWsrIwXX3yRhx56KO3rrVmzhjVr1iQ8Vltby5IlS87VnyRE\nRpRSvdUPYM70UqZVFvDCa0d492Azocip7dilhR4uv6CcC6YWAzN4a/cxXnztIMfrO+JjXA6DksJc\nMHl7rOIAAB4ESURBVFz0RCAcDVGQ5yYYiiYEL5/boLXDz4n6IAePNbPgggnMmV5G/71yhg7hqM2J\nxgBH67rY+0ELixdMoiTfnRBwwhGLhlY/gZ7YXCuKPRTlujEMyRoZCuP53DVsAemdd95h8+bNzJo1\ni5tuugmAdevWxVc+t99+O4sXL2bbtm0sXboUj8fDgw8+GJukafLNb36Tz372s9i2zapVq2SHnch6\nSqlY/o6lElqVe90Obr1uJvNnlrLlzRrqWwJcMK2Qj15UmbCVe+HcycyfOZFntr7HmzuP4fO6cHu8\naPqpreO2gtaOHnwekxyvg0jEQtk2HxxvTpjL9j117D7YyLIrqyjJ96Jr0NgWTEi6bfeHeW7bYeZN\nL2be9FKcDoOO7hCNbYmrufqWIC3tPUwqz8HrHtgWXYgPS1Pj6KZN37eMrVu3UllZOdLTEWPc6WV4\nko6xbA4eb6Moz5N23HOvHGDHgZZBT/5WNERHd/L6en0WXzoFxyD5UG6nwZVzJwyaDzVtYi4+jzPt\nGHH2+s5d3/rBL4f8kt11l08ZsUt2suYWYohk8k3P6G2dPpiiAl9GKxHbHvxV+18mTCUStTNKzs3g\n5YTImAQkIYQQWUECkhBCiKwgAUkIIURWkPYTQowzmexj6tuqPth9q0yPBWR0LNmxl95QV2pQKnlO\n23CRgCTEEDE0DV2P5R+lYupQWuChMxDGH0x+MjB0jcUXn8fUCXn86i/7CYQGjnOaGjOnFKNrcKS2\nhZp+uUv9FeS4OXS8nZLCMBNKcjGT5BIpZRNVGm+8V8fMSQWUFCTfdOFzm3QFItgK8n3OpMHE7rfT\nMFU/JaUUlq2wFeiawtClGWAqQ1mpIRgI8PGrZg1JBYZMSUASYohovV1WHdqpE24fXSPhxFuQ48L7\n/7d398FR1fe/wN/f7zlnn7N5dgmEpwTDQy0db6l4uSodEHyIICCOzvT2AR3GzkiwpXe4onWmMw46\n9A//kT8opdOpnXYGxo4OTnTUQsXbe73a3p+YsfKrAooGSIBAsnnYp3PO9/6x2SWb7BOQ7J4k79eM\no2EPux/Icd85Z7+f78et43I4ljF91qWn3sAFmmZV4n/89/+C/91xHkf+8U36mPkNQVQFPUiYNmwF\nzG+sxaxQJf7fZ52IJZJpYOgaKgMe9A7EAJjoG4ihrz+KxlAFqiqSgZO8QgGicTs9P6lv4EJ6gF+q\nR8rQBAI+1/DvAQaGEojFLQR9BrweI/1cpq0yZiyZNiCUDX3En3v00nhbAcriMMBcJnKnhtQuDeX8\nYYCBRDTBhBDQJCAVYNmpK4DMW1hCCLhdOkI1GgaG4hiKWcg2Xsnj0rF62Wx8a34N3vj7lwj63bBs\nlTGkz1bJZvI7bp2Pzgu9uHh5CAnLHg6jqy71RnAlHEXjTRWYFQpCSon+ocyfvhOmjdNnw7jcF8WC\nxiosmluddYeGhGmjJxyDL2aiwu/OueRdKSBhKQgoIMeEWoXs4UVTH38EISqB1AA/XUv+O9ebrJQC\nPq+RNYxGmlHrx8K51RlXU6MlLAWP243wUCJjR4aRLFvhTFcY8YQ9JoxG6h2Iw7LsgtsFDcUsmIW6\ngZEMnUIfP02fln1KYSARldB4/rQvi3iuYl+tmLqKzgde0NB1YiAREZEjMJCIiMgRGEhEROQIXGVH\nNFmN62c147iCgIsRJsx4NsaOHlU+UWPJrwUDichhNCng8+iIxMycK800KbBkXg3iCQtdPdmnz/o8\nOubPrEfAq6Hj5EUMZZksKwUwf2YVaoMeDMUt9A3Esz6X163h9Nk+3FTjQ0OtP2ft/UNxXLg8iJvn\nVMPturG3FyG4e8No49UYm2tUeTmbYgEGEpHjCJFsPHW7NAwMJTJ6jIQADE1C1yVmhyowqz6Ajz+/\niP88cyUdOEIAM2p8mDczCJ/HQNOsKiyeV4v3j3fiVGdv+rnqqrxYNK8WcxuSzZC2rRDwxtDTG0V0\neESFJgGPW8dgxEQEFt758GvMnxnE9xbdlG6CBQDTsvDVuTDO9wwBAE6dDeO2b4Uwo7a4sRkZf/7h\n12Vj7Fjj1Rhb7lHluTCQiBzK0DVUVUhEYiaGoglIIWDoMuONWkqB7y66Cc2zKvHRZ13oHYhjdqgC\noZrM7X4aZ1Ti0bVBfPjpWXz8726EagL4VnNdxlWMlAI1QQ98bh094Sii8WRP0egtjb48F8aZ82H8\nt6UzMWdGAOHBOE58eQX2iMs507LxfzrOo6HWj++01BU9xG/0DhY0vTCQiBxMCAGfx4AmBcw8A/Oq\nKtxYc9scnOnqzzk0T0qB/7q0ES1zaxEezH5rDkheEYWqvTjZ2ZdzSJ+tgP/1yTksuBTMO6TvfM8g\nBj+JY83t83IfNEwTKNh4S1Mbv/tEk0AxVwyp3SAKybaharbnGq9m2WLXOPCqiBhIRETkCAwkIiJy\nBAYSERE5AgOJaAop5vMhQy/uf3tDL/yZjlbk5z6xeP5JpMkhfYV3CaepjavsiCYBQ5fQNIFY3Mra\nLKsJAZdLQ9OsIHr7Yzh3aWjMMUIAoWovvO4KzJtRgc++6kH/0Nig8Lk1+Dwe1FV5cb5nEF98M3b6\nrKFJfOfmWvg9BhSSq+msLCvyAl4DbpeO//j3RcwJBTCzPjBm8YJt27BtBXN4XpRr1NJ2uupadmoY\nvRPDSE7YlSEbBhLRJCCEgCYEvG4By7LTk2ABwG1o0LTUqjiB2kovAl4D5y8Non+4h6jSb6Cqwg1t\n+I0+4HNh2aIZ6L4yiBNfXYFSycCqrnBDDQcDADTU+lFX6cWJry6jd3gXhwWNQcysCwC4uoJuVr0f\nkYiJi31RAMleoqoKNywbMC0F0zLxn2d6cakviubGSgS8LiilYNvJf1IsSyFqWdANBUOTXHk3SrE7\nNeTaiWGkcu/KkA0DiWgSEUJA1zVIKWFaFnRNy7rU2+3SMbchiPBgcrBetm18pBRoqA2gOuDBF9/0\nwlYq68A/Q5f4zs11uNIfhc9tpEeZj2TbgNutY3YogIGhBKQms14xXeqNIjyYwLyGABpq/TknxiYS\nNixTwe3i1dJIxe7U4NSdGArhd5poEpJSwGXoefuOhBDwD98yy8fj1uFxa3mnzyoF1AY9WcNo9HEu\nQ8saRinxhAVlq4ITYW3uYzftMJCIiMgRGEhEROQIDCQiInIEBhIRETkCA4mIilTcAoNCixUA5N0h\nPPO5OH52OuGyb6IpTJMChiFhJuycu27rmkDL7Gp8eT6My+Fo1mP8XgN1VR7E4hZ6+qIZQwNHvpYQ\ngG0BmktDJG5lfS63IXH6bBi6FKiv9mVdKaiUwlDMRN9QHHWVHrj0/Kv7potCjbGpZlinNr4WwkAi\nmsKEEHDpGnQpEDftjOXYUgi4DJmeQbRkfg26e4bQeXEAkViyodbQJWqCbtRV+SCEQIUPqPC7cfHy\nYLpRNnVcKqSkJqGUgt+jIW6q9K8buoRSSDf1/vubPnReGsTCOVUIjBjglzAt9A7E01dRnRcGURt0\no8LnKmq8xlSWrzF2dDOsExtfC2EgEU0DUkp4XBKmZSORsKBpEoaeuROCEAIz6vyorfLg9Lk+xGIW\nbqrxjeljchsaGkNBBLxRXOyLwDTtMVdMqec1NAVdarCUQjwx9qpqMGLiP/59aXhbIT+icQuR2Ngr\nq55wDOHBOOqrvXAb2rTtT8rXGDtZm2FHKmkg7dq1C8eOHUNtbS3eeOONMY/39fXhmWeewTfffAO3\n240XXngBN998MwBg1apV8Pv90DQNuq7j1VdfLWXpRFOCrsmCG7AauoaW2dUZV0DZVAU9GIomcDke\ny3mMlBKWZWcNo5G+7h6ArksYeW7NJSyF8EAcN40az05TR0kXNTz00EM4cOBAzsf37duHJUuW4PDh\nw9izZw92796d8fgf//hHvP766wwjIqIpqKSBtGzZsrz3NU+fPo3ly5cDAJqamnD27Flcvnw5/ThX\n3BARTV2OWva9aNEivPvuuwCAjo4OnDt3Dl1dXQCS96S3bNmCTZs24dChQ+Usk4iIJoCjFjVs3boV\nu3fvxoYNG9DS0oLFixend/r985//jFAohMuXL2PLli1oamrCsmXLcj7Xyy+/jL1795aqdCKicTGd\n37scFUiBQAAvvvhi+utVq1Zh9uzZAIBQKAQAqKmpwZo1a9DR0ZE3kNra2tDW1pbxa52dnVi9evUE\nVE409ei6gGnmv03u9egQ/bGcPU4A4PPo0DU7PZspG4HkMvRimKYFPc/iB9tO9lxJISblarzp/N7l\nqFt2/f39iMeTK3sOHTqE2267DX6/H5FIBAMDAwCAoaEh/P3vf0dLS0s5SyWa0oQQCHgM+L25f2YV\nAKqDXrTMrUZlwDX2cZEc3DevoRItc6rRPCuIbPlQGXBhwewqaJpErjYjAaDCa8AwNAxETERiZtbP\nlC3bhmkjPRjQ5lj0SaWkV0g7duzARx99hN7eXqxcuRJtbW0wzeRPTY8++ihOnjyJp59+GkIItLS0\npFfZXbp0Cdu2bQMAWJaFdevW4Y477ihl6UTTTrqp1i8RiZtjlm6n4sDQNcwOBVFTEcfX3f2wbIWq\ngAt1Vd70hFoAqAq4sbS5BmcvDuJSXwxSCswOBeAZ0edkq2T4CHF1eyGPocHr0TNeNxq3kDBteF0a\nDEODbduwbGRcqSkApg1IZQ/vIjH5rpZGG7lTw+gR5ZN1d4aRhJpGS9dSl71HjhxBY2NjucshmjSU\nUojFTUTjubcgApJXKNGYWXCrn4FIHBD5b9BICfg9RsG+Kbch897CSzG0yRtKqfeuJ//nr1FVU4fI\n0BDuu2PsiPJgMDhp/4yAwz5DIiJnSr7JibxhBACalPAYWsHNU92GjliW/fBGsm0UDKPpJrVTw1TY\nlSEbfreJiMgRGEhEROQIDCQiInIEBhIRETkCA4mIiiLGcRZRgQV2acUsAi56+mxxh1EZcZUdERXF\nbWgQSPYAWVlSQApA1zVobg3R2Ni+pRSPW0PA58VgxETfQAzxLKvtdE1AlwKmacMwJHKNTzctC4OR\nBHxuHT6vkXP6rGnaMM3kn0FOkZ6kqYiBRERFcxkaDD3ZKBuLXw0SXRNwjRic5/O6YOgWonEzPaXW\n0AT8PhdcRrJnqMJnwO/R0ROOYiCSgFLJUDM0CX1Ez1AiYUMMh12KUmp4t4bk10MxE5G4iUr/8PML\nAUDBsmykNmtQKhmmmpZs+J2M02dTjbFK5d6GaTJjIBHRNRFCwOc24NJtRGImdO3qGPSRDEODrktE\nYyY0KeD3GmOuTKQUqK/yIuDVcTkcgxTI2N0hRSkgkbAgZbI/KdtVlVJA70AcbkMi4HfBtrLfpLMs\nhYhlwu3SJl2fk22bGBzow313LJyUI8oLYSAR0XXRNQmvW8/7GY4QAl6PAZee/43f6zbgd1tZg2Yk\n00zuVZdPLGHDk7DSkwJysSx70gVSbV0IHq8PlZWVU/K24+T6bhAR0ZTFQCIiIkdgIBERkSMwkIiI\nyBEYSER03YpZOS1FcQ2uXo+edYDfSB6PAY8r/6gJXQpoWuFxFJalYFoc4OckXGVHRNdNSglDKFi2\nyrraTpcCQqCoFWFul45aXcNgNI5I1Mp4TJMCPq8xvLxcwZOw0TcQw+icq/Ab8Lj05OupZODkykIF\nIBa3YGo2XLosuCqPJh4DiYhuiBACuiZg28lgUsBwP9G174ggpUDA64LHZSM8EIdlK/g8+ohmVwBI\nNuHWVnkRiSQwGDWTvUc+V2Y/lBDQ9eFpsjl6koDklVLUtmDoCromp+Ry6smCgURE40IOXw0phRva\nBUEIAUPXUFPpQSxPP5EUyZ0fPG4dmhTIdb9PSgkhbJhm7lBSCognbAigqOmz5TI01A97Cg/5ZiAR\n0bgRQhT8HOhanquY22jZdonI8mwoantVh18cfaepEgsXTs1dGgAGEhHRpFFRUTHlxpaPxE/xiIjI\nERhIRETkCAwkIiJyBAYSETlWqdcYFLN+zbYV7GLH1NI14aIGInIsXRM5m26vhRACui5hWSrrrhGp\nabdGnhV7arjRNjUJ120k50Cxb2n8MJCIyLGuNt3asOzirmDyPpeefC7bVukdHEZPux1NKQVbKcTi\nVsauD7GEDWnacLs07vIwThhIROR4yeZWBctSuNHd51LPZdsKLj37tNuREqaNRI6pgLYCIjELbpeC\nXsT+eTeqv78fSqkpe1XGWCeiSUEIATFO71jJ3SAKhxGAojZgVSXao/X/ftqFcDhcmhcrAwYSEdEk\n4fP5yl3ChGIgERGRIzCQiIjIERhIRETkCAwkIpo0BApPqRUCKGYDcFsBpm3nnGarlIJp2wVHaQgB\nSG1qrnorNS77JqJJQ0oJCWQMAxxp5IRamWeSbYptAwoKmlQZvUQj+56klJASsCwL9qjVdC5dQtfZ\nHDteGEhENOmkhgHaSsGys0+oTTfVqmT/Uq5cUgBMGxDKhiaSV07ZQkzTNEipYJo2pAAbYidAyf42\nd+3ahRUrVmDdunVZH+/r68OTTz6J9evX4+GHH8YXX3yRfuz999/Hvffei7Vr12L//v2lKpmIHEwI\nAU1K6JrIO3pcClHULTylksGU74pKCAHD0OBx6wyjCVCyv9GHHnoIBw4cyPn4vn37sGTJEhw+fBh7\n9uzB7t27ASQvk59//nkcOHAA7e3taG9vx6lTp0pVNhE5nCzidtl43lIT4/x81yISGSrL65ZKyQJp\n2bJlecfunj59GsuXLwcANDU14ezZs+jp6UFHRwfmzJmDxsZGGIaB1tZWHDlypFRlExE5xt233zxl\nx5cDDlplt2jRIrz77rsAgI6ODpw7dw5dXV3o7u5GQ0ND+rhQKITu7u5ylUlEVDYVFRVTegGFYxY1\nbN26Fbt378aGDRvQ0tKCxYsXQ9Ny78BbyMsvv4y9e/eOc5VERBNrOr93OSaQAoEAXnzxxfTXq1at\nwuzZsxGNRnH+/Pn0r3d1dSEUChV8vra2NrS1tWX8WmdnJ1avXj1+RRMRjbPp/N7lmFt2/f39iMfj\nAIBDhw7htttug9/vxy233IIzZ86gs7MT8Xgcb7755rT4xhDR+BrPO125mmlTbKWQMK2Cx1Gmkl0h\n7dixAx999BF6e3uxcuVKtLW1wTRNAMCjjz6KkydP4umnn4YQAi0tLelVdrqu47nnnsPjjz8O27ax\nefNmNDc3l6psIpoChBDQ5dW+pTGPI7m7g5Qy2RSrgGxZIiWgCZF3mJ9lJ4f5AclZSm5DG+6bmrqf\n/YyXkgXSSy+9lPfxW2+9FW+//XbWx1auXImVK1dORFlENE0IIZJhgszA0WRy6XgqMFK7QVjDuzUk\nfy+gCeTsPUpOlQUSCQvWiEYmpYBo3IKuCRi6lt5FgrJzzGdIRESlcHX7ITv9dTaalJBCQSlVsAnW\ntGzEE7mn9JmWgmmZ8Hn4lpsP/3aIaFoqZqcFkef23Ej5dneg4jlmUQMREU1vDCQiInIEBhIRETkC\nA4mIiByBgUREdINcuoTLyP926ja0ElUzeXGVHRHRDRIi2WekSYn4qF4kXUvOUCpmTMZ0x0AiIhon\nUgq4XRosWyGesJJTZYtcOk4MJCKicZUanT56pDoVxs+QiIgmAMPo2jGQiIjIERhIRETkCAwkIiJy\nBAYSEVGZcIBfJgYSEVEZpIb50VVc9k1EVEJKKSgFmAyjMRhIREQlkroqYhZlx0AiIiqRhMUkyoef\nIRERkSNMyyukrq6ucpdARNNQtiuk2upKBIPBMlTjPNMqkILBIL73ve/hBz/4QblLISICAGzbtg1t\nbW15jwkGg9i2bduUDy6hptlC+HA4jHA4XO4yAACrV6/GkSNHyl1GmpPqcVItAOsphPXkl6+eYDA4\n5YOmWNPqCglw3je/sbGx3CVkcFI9TqoFYD2FsJ78nFaPE3FRAxEROQIDiYiIHIGBREREjqD96le/\n+lW5i5jOli9fXu4SMjipHifVArCeQlhPfk6rx4mm3So7IiJyJt6yIyIiR2AgERGRIzCQiIjIERhI\nRETkCAwkIiJyBAYSERE5AgOpRN566y20trZi8eLF+Ne//pXzuN/85jdobW3FunXr8Itf/ALxeLys\n9YTDYWzfvh333Xcf7r//fhw/frys9QCAZVnYsGEDfvrTn05ILcXWc/78efzwhz9Ea2srHnjgAbzy\nyitlrQcA3n//fdx7771Yu3Yt9u/fP2H19Pb2YsuWLbjnnnvw2GOP5dywuBTnc7G1lOpcLrYeoDTn\n8mTCQCqRlpYW7N27F8uWLct5TGdnJw4dOoTXXnsNb7zxBizLQnt7e9nqAYDdu3fjrrvuwltvvYXD\nhw+jubm5rPUAwCuvvDJhdVxLPbqu45lnnkF7ezsOHjyIP/3pTzh16lTZ6rEsC88//zwOHDiA9vZ2\ntLe3T1g9+/fvx4oVK/D222/j9ttvzxp+pTqfi6kFKN25XGw9QGnO5cmEgVQizc3NmD9/ft5jAoEA\ndF1HJBKBaZqIRqMIhUJlq6e/vx///Oc/sXnzZgDJN+CKioqy1QMkhyseO3YMDz/88ITUcS311NfX\nY/HixQAAv9+P5uZmXLhwoWz1dHR0YM6cOWhsbIRhGGhtbZ2wEQxHjx7Fxo0bAQAbN27EX//61zHH\nlOp8LqaWUp7LxdQDlO5cnkwYSA5SVVWFxx57DN///vdx5513oqKiAitWrChbPZ2dnaipqcGuXbuw\nceNG/PKXv0QkEilbPQDwwgsvYOfOnZDSWaduZ2cnTpw4gaVLl5athu7ubjQ0NKS/DoVC6O7unpDX\n6unpQV1dHQCgrq4OPT09Y44p1flcTC2lPJeLqQdw7rlcTtNuHtJE2rJlCy5dujTm13/+859j1apV\nBX//119/jT/84Q84evQoKioq8NRTT+Hw4cNYv359WeoxTROfffYZnnvuOSxduhS7d+/G/v378dRT\nT5Wlnr/97W+ora3FkiVL8OGHH15XDeNZT8rg4CC2b9+OZ599Fn6/v2z1CCGu+7WvpZ6f/exnY143\n22uP5/l8o7WU6lwutp7xPpenCgbSOPr9739/Q7//008/xa233orq6moAwJo1a/Dxxx9fdyDdaD0z\nZsxAKBRK/9R/zz334Le//e11P9+N1vPxxx/j6NGjOHbsGOLxOAYGBrBz5078+te/Lks9AJBIJLB9\n+3asX78ed9999w09143WEwqFcP78+fTXXV1dN3SLLF89tbW1uHjxIurr63HhwgXU1NSMOWY8z+cb\nraWU53Ix9Yz3uTxV8FqxDHLtZ9vU1IRPPvkE0WgUSil88MEHWLBgQdnqqa+vR0NDA7788ksAKHs9\nO3bswLFjx3D06FG89NJLuP3220vyP3CuepRSePbZZ9Hc3Iyf/OQnE15HoXpuueUWnDlzBp2dnYjH\n43jzzTexevXqCalh1apVeO211wAAr7/+etYwLtX5XEwtpTyXi6mnXOey4ykqiXfeeUfddddd6tvf\n/rZasWKFevzxx5VSSnV1damtW7emj9u/f7+6//771QMPPKB27typ4vF4Wes5ceKE2rRpk1q3bp16\n8sknVTgcLms9KR9++KF64oknJqSWYuv5xz/+oRYuXKjWr1+vHnzwQfXggw+qY8eOla0epZR67733\n1Nq1a9Xdd9+t9u3bNyG1KKXUlStX1I9//GO1du1atWXLFtXX15e1nlKcz8XWUqpzudh6Uib6XJ5M\nOH6CiIgcgbfsiIjIERhIRETkCAwkIiJyBAYSERE5AgOJiIgcgYFERESOwEAiIiJHYCARFcGyrHKX\nQDTlMZCIcli0aBH27t2LzZs3Y+/eveUuh2jK4+aqRHl4PB68+uqrOHnyJA4ePIhYLIYf/ehH2LNn\nDx555BF88MEHkFLikUceKXepRJMer5CI8kgNWuvp6UF9fT0SiQQA4Pjx45g3bx7WrFkDIDlq4S9/\n+Qt+97vfpY8homvDQCLKw+fzAQCWL1+O9vZ23Hfffbhy5QoqKysBAJFIBN/97ndx8OBBtLa2YuHC\nhXjvvffKWDHR5MVAIipSd3c3Zs6cic8//xxNTU0AgM8//xwLFizAhQsX4PF4UFdXh6+++qq8hRJN\nUvwMiSiH0ZM+77zzThw+fBimaSIWi+Gdd97BzJkzM461bRuappW8VqKpgIFElMOJEycyvn7iiSfS\n/71p06aMx2pqapBIJNDd3Y25c+eWpD6iqYbzkIjGwalTp3D8+HFcvHgRW7du5VUS0XVgIBERkSNw\nUQMRETkCA4mIiByBgURERI7AQCIiIkdgIBERkSMwkIiIyBEYSERE5AgMJCIicgQGEhEROcL/B5jI\nIPzfdDDiAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import seaborn as sns; sns.set() # set default plot styles\n", "import pandas as pd\n", "\n", "df_2D = pd.DataFrame(fit.extract('w')['w'], columns=['r$w_0$', 'r$w_1$'])\n", "\n", "xlim, ylim = zip(df_2D.min(0), df_2D.max(0))\n", "with sns.axes_style('ticks'):\n", " jointplot = sns.jointplot('r$w_0$', 'r$w_1$',\n", " xlim=xlim, ylim=ylim,\n", " data=df_2D, kind=\"hex\");" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 0 }