{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "6f71ca5c", "metadata": {}, "source": [ "# Tutorial: Physics Informed Neural Networks on PINA\n", "\n", "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial1/tutorial.ipynb)\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "ef4949c9", "metadata": {}, "source": [ "In this tutorial, we will demonstrate a typical use case of **PINA** on a toy problem, following the standard API procedure. \n", "\n", "

\n", " \"PINA\n", "

\n", "\n", "Specifically, the tutorial aims to introduce the following topics:\n", "\n", "* Explaining how to build **PINA** Problems,\n", "* Showing how to generate data for `PINN` training\n", "\n", "These are the two main steps needed **before** starting the modelling optimization (choose model and solver, and train). We will show each step in detail, and at the end, we will solve a simple Ordinary Differential Equation (ODE) problem using the `PINN` solver." ] }, { "attachments": {}, "cell_type": "markdown", "id": "cf9c96e3", "metadata": {}, "source": [ "## Build a PINA problem" ] }, { "attachments": {}, "cell_type": "markdown", "id": "8a819659", "metadata": {}, "source": [ "Problem definition in the **PINA** framework is done by building a python `class`, which inherits from one or more problem classes (`SpatialProblem`, `TimeDependentProblem`, `ParametricProblem`, ...) depending on the nature of the problem. Below is an example:\n", "### Simple Ordinary Differential Equation\n", "Consider the following:\n", "\n", "$$\n", "\\begin{equation}\n", "\\begin{cases}\n", "\\frac{d}{dx}u(x) &= u(x) \\quad x\\in(0,1)\\\\\n", "u(x=0) &= 1 \\\\\n", "\\end{cases}\n", "\\end{equation}\n", "$$\n", "\n", "with the analytical solution $u(x) = e^x$. In this case, our ODE depends only on the spatial variable $x\\in(0,1)$ , meaning that our `Problem` class is going to be inherited from the `SpatialProblem` class:\n", "\n", "```python\n", "from pina.problem import SpatialProblem\n", "from pina.domain import CartesianProblem\n", "\n", "class SimpleODE(SpatialProblem):\n", " \n", " output_variables = ['u']\n", " spatial_domain = CartesianProblem({'x': [0, 1]})\n", "\n", " # other stuff ...\n", "```\n", "\n", "Notice that we define `output_variables` as a list of symbols, indicating the output variables of our equation (in this case only $u$), this is done because in **PINA** the `torch.Tensor`s are labelled, allowing the user maximal flexibility for the manipulation of the tensor. The `spatial_domain` variable indicates where the sample points are going to be sampled in the domain, in this case $x\\in[0,1]$.\n", "\n", "What if our equation is also time-dependent? In this case, our `class` will inherit from both `SpatialProblem` and `TimeDependentProblem`:\n" ] }, { "cell_type": "code", "execution_count": 10, "id": "2373a925", "metadata": {}, "outputs": [], "source": [ "## routine needed to run the notebook on Google Colab\n", "try:\n", " import google.colab\n", " IN_COLAB = True\n", "except:\n", " IN_COLAB = False\n", "if IN_COLAB:\n", " !pip install \"pina-mathlab\"\n", "\n", "from pina.problem import SpatialProblem, TimeDependentProblem\n", "from pina.domain import CartesianDomain\n", "\n", "class TimeSpaceODE(SpatialProblem, TimeDependentProblem):\n", " \n", " output_variables = ['u']\n", " spatial_domain = CartesianDomain({'x': [0, 1]})\n", " temporal_domain = CartesianDomain({'t': [0, 1]})\n", "\n", " # other stuff ..." ] }, { "attachments": {}, "cell_type": "markdown", "id": "ad8566b8", "metadata": {}, "source": [ "where we have included the `temporal_domain` variable, indicating the time domain wanted for the solution.\n", "\n", "In summary, using **PINA**, we can initialize a problem with a class which inherits from different base classes: `SpatialProblem`, `TimeDependentProblem`, `ParametricProblem`, and so on depending on the type of problem we are considering. Here are some examples (more on the official documentation):\n", "* ``SpatialProblem`` $\\rightarrow$ a differential equation with spatial variable(s) ``spatial_domain``\n", "* ``TimeDependentProblem`` $\\rightarrow$ a time-dependent differential equation with temporal variable(s) ``temporal_domain``\n", "* ``ParametricProblem`` $\\rightarrow$ a parametrized differential equation with parametric variable(s) ``parameter_domain``\n", "* ``AbstractProblem`` $\\rightarrow$ any **PINA** problem inherits from here" ] }, { "attachments": {}, "cell_type": "markdown", "id": "592a4c43", "metadata": {}, "source": [ "### Write the problem class\n", "\n", "Once the `Problem` class is initialized, we need to represent the differential equation in **PINA**. In order to do this, we need to load the **PINA** operators from `pina.operator` module. Again, we'll consider Equation (1) and represent it in **PINA**:" ] }, { "cell_type": "code", "execution_count": 1, "id": "f2608e2e", "metadata": {}, "outputs": [], "source": [ "from pina.problem import SpatialProblem\n", "from pina.operator import grad\n", "from pina import Condition\n", "from pina.domain import CartesianDomain\n", "from pina.equation import Equation, FixedValue\n", "\n", "import torch\n", "import matplotlib.pyplot as plt\n", "\n", "class SimpleODE(SpatialProblem):\n", "\n", " output_variables = ['u']\n", " spatial_domain = CartesianDomain({'x': [0, 1]})\n", "\n", " domains ={\n", " 'x0': CartesianDomain({'x': 0.}),\n", " 'D': CartesianDomain({'x': [0, 1]})\n", " }\n", "\n", " # defining the ode equation\n", " def ode_equation(input_, output_):\n", "\n", " # computing the derivative\n", " u_x = grad(output_, input_, components=['u'], d=['x'])\n", "\n", " # extracting the u input variable\n", " u = output_.extract(['u'])\n", "\n", " # calculate the residual and return it\n", " return u_x - u\n", "\n", " # conditions to hold\n", " conditions = {\n", " 'bound_cond': Condition(domain='x0', equation=FixedValue(1.)),\n", " 'phys_cond': Condition(domain='D', equation=Equation(ode_equation))\n", " }\n", "\n", " # defining the true solution\n", " def truth_solution(self, pts):\n", " return torch.exp(pts.extract(['x']))\n", " \n", "problem = SimpleODE()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "7cf64d01", "metadata": {}, "source": [ "After we define the `Problem` class, we need to write different class methods, where each method is a function returning a residual. These functions are the ones minimized during PINN optimization, given the initial conditions. For example, in the domain $[0,1]$, the ODE equation (`ode_equation`) must be satisfied. We represent this by returning the difference between subtracting the variable `u` from its gradient (the residual), which we hope to minimize to 0. This is done for all conditions. Notice that we do not pass directly a `python` function, but an `Equation` object, which is initialized with the `python` function. This is done so that all the computations and internal checks are done inside **PINA**.\n", "\n", "Once we have defined the function, we need to tell the neural network where these methods are to be applied. To do so, we use the `Condition` class. In the `Condition` class, we pass the location points and the equation we want minimized on those points (other possibilities are allowed, see the documentation for reference).\n", "\n", "Finally, it's possible to define a `truth_solution` function, which can be useful if we want to plot the results and see how the real solution compares to the expected (true) solution. Notice that the `truth_solution` function is a method of the `PINN` class, but it is not mandatory for problem definition.\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "78b30f95", "metadata": {}, "source": [ "## Generate data \n", "\n", "Data for training can come in form of direct numerical simulation results, or points in the domains. In case we perform unsupervised learning, we just need the collocation points for training, i.e. points where we want to evaluate the neural network. Sampling point in **PINA** is very easy, here we show three examples using the `.discretise_domain` method of the `AbstractProblem` class." ] }, { "cell_type": "code", "execution_count": 2, "id": "09ce5c3a", "metadata": {}, "outputs": [], "source": [ "# sampling 20 points in [0, 1] through discretization in all locations\n", "problem.discretise_domain(n=20, mode='grid', domains='all')\n", "\n", "# sampling 20 points in (0, 1) through latin hypercube sampling in D, and 1 point in x0\n", "problem.discretise_domain(n=20, mode='latin', domains=['D'])\n", "problem.discretise_domain(n=1, mode='random', domains=['x0'])\n", "\n", "# sampling 20 points in (0, 1) randomly\n", "problem.discretise_domain(n=20, mode='random')" ] }, { "cell_type": "markdown", "id": "8fbb679f", "metadata": {}, "source": [ "We are going to use latin hypercube points for sampling. We need to sample in all the conditions domains. In our case we sample in `D` and `x0`." ] }, { "cell_type": "code", "execution_count": 3, "id": "329962b6", "metadata": {}, "outputs": [], "source": [ "# sampling for training\n", "problem.discretise_domain(1, 'random', domains=['x0']) # TODO check\n", "problem.discretise_domain(20, 'lh', domains=['D'])" ] }, { "cell_type": "markdown", "id": "ca2ac5c2", "metadata": {}, "source": [ "The points are saved in a python `dict`, and can be accessed by calling the attribute `input_pts` of the problem " ] }, { "cell_type": "code", "execution_count": 4, "id": "d6ed9aaf", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Input points: {'x0': LabelTensor([[0.]]), 'D': LabelTensor([[0.9731],\n", " [0.0123],\n", " [0.2990],\n", " [0.8353],\n", " [0.7245],\n", " [0.7577],\n", " [0.2480],\n", " [0.1803],\n", " [0.5525],\n", " [0.9135],\n", " [0.3683],\n", " [0.3371],\n", " [0.4241],\n", " [0.6288],\n", " [0.1155],\n", " [0.0772],\n", " [0.6523],\n", " [0.4767],\n", " [0.8872],\n", " [0.5355]])}\n", "Input points labels: ['x']\n" ] } ], "source": [ "print('Input points:', problem.discretised_domains)\n", "print('Input points labels:', problem.discretised_domains['D'].labels)" ] }, { "cell_type": "markdown", "id": "669e8534", "metadata": {}, "source": [ "To visualize the sampled points we can use `matplotlib.pyplot`:" ] }, { "cell_type": "code", "execution_count": 5, "id": "3802e22a", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAIbFJREFUeJzt3Xtw1NX9//HXJiEbVJLILSEapFAVFJQxmBDUoZVMY3FURhwRKSJNpVag/gii3CStVkO9ooIy2lrqCIVilSpmYjF4JwoGqNyrBQHBDVBkgyBJIOf3B8P6XQkhG7K72TfPx8wOwyfns+d8zq7u080mepxzTgAAAIbERXsBAAAAzY3AAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkJ0V5ANNTV1Wnnzp1q06aNPB5PtJcDAAAawTmn/fv3KyMjQ3FxDb9Hc1oGzs6dO5WZmRntZQAAgCbYvn27zj333AbHnJaB06ZNG0lHNyg5OTnKqwEAAI1RVVWlzMzMwOt4Q07LwDn2bank5GQCBwCAGNOYj5fwIWMAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkRCZxZs2apS5cuSkpKUk5OjpYvX97g+IULF6p79+5KSkpSr169VFJScsKxd955pzwej2bMmNHMqwYAALEq7IGzYMECFRYWqqioSCtXrtSll16q/Px87dq1q97xy5Yt09ChQ1VQUKBVq1Zp0KBBGjRokNauXXvc2Ndee00ff/yxMjIywn0ZAAAghoQ9cJ544gndcccdGjlypC666CLNnj1bZ5xxhl588cV6xz/11FO65pprNGHCBPXo0UMPPvigLrvsMs2cOTNo3I4dOzR27FjNnTtXrVq1CvdlAACAGBLWwKmpqVFFRYXy8vK+nzAuTnl5eSovL6/3nPLy8qDxkpSfnx80vq6uTsOHD9eECRN08cUXn3Qd1dXVqqqqCroBAAC7who4e/bs0ZEjR5SWlhZ0PC0tTT6fr95zfD7fScf/8Y9/VEJCgn772982ah3FxcVKSUkJ3DIzM0O8EgAAEEti7qeoKioq9NRTT2nOnDnyeDyNOmfSpEny+/2B2/bt28O8SgAAEE1hDZz27dsrPj5elZWVQccrKyuVnp5e7znp6ekNjv/ggw+0a9cude7cWQkJCUpISNDWrVs1fvx4denSpd779Hq9Sk5ODroBAAC7who4iYmJysrKUllZWeBYXV2dysrKlJubW+85ubm5QeMlacmSJYHxw4cP12effabVq1cHbhkZGZowYYLeeuut8F0MAACIGQnhnqCwsFAjRoxQnz59lJ2drRkzZujAgQMaOXKkJOm2227TOeeco+LiYknS3Xffrf79++vxxx/Xtddeq/nz5+vTTz/V888/L0lq166d2rVrFzRHq1atlJ6ergsvvDDclwMAAGJA2ANnyJAh2r17t6ZNmyafz6fevXurtLQ08EHibdu2KS7u+zeS+vXrp3nz5mnq1KmaPHmyzj//fC1atEg9e/YM91IBAIARHueci/YiIq2qqkopKSny+/18HgcAgBgRyut3zP0UFQAAwMkQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADAnIoEza9YsdenSRUlJScrJydHy5csbHL9w4UJ1795dSUlJ6tWrl0pKSgJfq62t1X333adevXrpzDPPVEZGhm677Tbt3Lkz3JcBAABiRNgDZ8GCBSosLFRRUZFWrlypSy+9VPn5+dq1a1e945ctW6ahQ4eqoKBAq1at0qBBgzRo0CCtXbtWknTw4EGtXLlS999/v1auXKlXX31VmzZt0vXXXx/uSwEAADHC45xz4ZwgJydHl19+uWbOnClJqqurU2ZmpsaOHauJEyceN37IkCE6cOCAFi9eHDjWt29f9e7dW7Nnz653jhUrVig7O1tbt25V586dT7qmqqoqpaSkyO/3Kzk5uYlXBgAAIimU1++wvoNTU1OjiooK5eXlfT9hXJzy8vJUXl5e7znl5eVB4yUpPz//hOMlye/3y+PxKDU1td6vV1dXq6qqKugGAADsCmvg7NmzR0eOHFFaWlrQ8bS0NPl8vnrP8fl8IY0/dOiQ7rvvPg0dOvSENVdcXKyUlJTALTMzswlXAwAAYkVM/xRVbW2tbr75Zjnn9Nxzz51w3KRJk+T3+wO37du3R3CVAAAg0hLCeeft27dXfHy8Kisrg45XVlYqPT293nPS09MbNf5Y3GzdulVLly5t8HtxXq9XXq+3iVcBAABiTVjfwUlMTFRWVpbKysoCx+rq6lRWVqbc3Nx6z8nNzQ0aL0lLliwJGn8sbj7//HO9/fbbateuXXguAAAAxKSwvoMjSYWFhRoxYoT69Omj7OxszZgxQwcOHNDIkSMlSbfddpvOOeccFRcXS5Luvvtu9e/fX48//riuvfZazZ8/X59++qmef/55SUfj5qabbtLKlSu1ePFiHTlyJPD5nLZt2yoxMTHclwQAAFq4sAfOkCFDtHv3bk2bNk0+n0+9e/dWaWlp4IPE27ZtU1zc928k9evXT/PmzdPUqVM1efJknX/++Vq0aJF69uwpSdqxY4def/11SVLv3r2D5nrnnXf0k5/8JNyXBAAAWriw/x6clojfgwMAQOxpMb8HBwAAIBoIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJgTkcCZNWuWunTpoqSkJOXk5Gj58uUNjl+4cKG6d++upKQk9erVSyUlJUFfd85p2rRp6tSpk1q3bq28vDx9/vnn4bwEAAAQQ8IeOAsWLFBhYaGKioq0cuVKXXrppcrPz9euXbvqHb9s2TINHTpUBQUFWrVqlQYNGqRBgwZp7dq1gTGPPPKInn76ac2ePVuffPKJzjzzTOXn5+vQoUPhvpyT+tr/nZb9d4++9n/3/UH/DmnL+0f/jKTmmjfW198S5472YxOpvY2VfWyu+aL5nD2VtbSEdZ/qGlrCNfxfLe25H8n9aSGPhcc558I5QU5Oji6//HLNnDlTklRXV6fMzEyNHTtWEydOPG78kCFDdODAAS1evDhwrG/fvurdu7dmz54t55wyMjI0fvx43XPPPZIkv9+vtLQ0zZkzR7fccstJ11RVVaWUlBT5/X4lJyc305VKC1Zs06RX16jOSXEeqfjGXhoS/670xt2Sq5M8cdJ1T0mX3dZsc57QypeaZ97mup9YmTcSc0f7sYnU3sbKPjbXfNF8zoa61qaObQnrDcf5za2lPfcjuT9hniuU1++wvoNTU1OjiooK5eXlfT9hXJzy8vJUXl5e7znl5eVB4yUpPz8/MH7Lli3y+XxBY1JSUpSTk3PC+6yurlZVVVXQrbl97f8uEDeSVOekp199T+7YAy0d/fON/xeZ/2pujnmb635CFa15IzF3tB+bSO1trOxjc80XzedsqGtt6thwOdU1tIRriOR6Qr3/SO5PC3sswho4e/bs0ZEjR5SWlhZ0PC0tTT6fr95zfD5fg+OP/RnKfRYXFyslJSVwy8zMbNL1NGTLngOBuDmms+dreY490Me4I9Lezc0+f5C9//3+CXYq8zbX/YQqWvNGYu5oPzaR2ttY2cfmmi+az9kfCmUtLWHdp7qGlnANkVxPqPcfyf1pYY/FafFTVJMmTZLf7w/ctm/f3uxz/Kj9mYrzBB/b5jrJeX6wxZ54qW3XZp8/SNtuR98aPNV5m+t+QhWteSMxd7Qfm0jtbazsY3PNF83n7A+FspaWsO5TXUNLuIZIrifU+4/k/rSwxyKsgdO+fXvFx8ersrIy6HhlZaXS09PrPSc9Pb3B8cf+DOU+vV6vkpOTg27NrVNKaxXf2EvxnqOVE+/x6Lc39pfnuqeOPsDS0T+vmyGlnNPs8wdJOefo9z1Pdd7mup9QRWveSMwd7ccmUnsbK/vYXPNF8zkb6lqbOjZcTnUNLeEaIrmeUO8/kvvTwh6LiHzIODs7W88884ykox8y7ty5s8aMGXPCDxkfPHhQb7zxRuBYv379dMkllwR9yPiee+7R+PHjJR390FHHjh2j/iFj6ehncb7cc1Bd2p+hTimtjx707zj6Fl3brpF9oJtr3lhff0ucO9qPTaT2Nlb2sbnmi+Zz9lTW0hLWfapraAnXEMn1hHr/kdyfMM4V0uu3C7P58+c7r9fr5syZ49avX+9GjRrlUlNTnc/nc845N3z4cDdx4sTA+I8++sglJCS4xx57zG3YsMEVFRW5Vq1auTVr1gTGTJ8+3aWmprp//vOf7rPPPnM33HCD+9GPfuS+++67Rq3J7/c7Sc7v9zfvxQIAgLAJ5fU7oVnTqh5DhgzR7t27NW3aNPl8PvXu3VulpaWBDwlv27ZNcXHff6esX79+mjdvnqZOnarJkyfr/PPP16JFi9SzZ8/AmHvvvVcHDhzQqFGjtG/fPl155ZUqLS1VUlJSuC8HAADEgLB/i6olCue3qAAAQHi0mN+DAwAAEA0EDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwJW+Ds3btXw4YNU3JyslJTU1VQUKBvv/22wXMOHTqk0aNHq127djrrrLM0ePBgVVZWBr7+73//W0OHDlVmZqZat26tHj166KmnngrXJQAAgBgVtsAZNmyY1q1bpyVLlmjx4sV6//33NWrUqAbPGTdunN544w0tXLhQ7733nnbu3Kkbb7wx8PWKigp17NhRL7/8statW6cpU6Zo0qRJmjlzZrguAwAAxCCPc841951u2LBBF110kVasWKE+ffpIkkpLSzVw4EB99dVXysjIOO4cv9+vDh06aN68ebrpppskSRs3blSPHj1UXl6uvn371jvX6NGjtWHDBi1durTR66uqqlJKSor8fr+Sk5ObcIUAACDSQnn9Dss7OOXl5UpNTQ3EjSTl5eUpLi5On3zySb3nVFRUqLa2Vnl5eYFj3bt3V+fOnVVeXn7Cufx+v9q2bdt8iwcAADEvIRx36vP51LFjx+CJEhLUtm1b+Xy+E56TmJio1NTUoONpaWknPGfZsmVasGCB3nzzzQbXU11drerq6sDfq6qqGnEVAAAgVoX0Ds7EiRPl8XgavG3cuDFcaw2ydu1a3XDDDSoqKtLPfvazBscWFxcrJSUlcMvMzIzIGgEAQHSE9A7O+PHjdfvttzc4pmvXrkpPT9euXbuCjh8+fFh79+5Venp6veelp6erpqZG+/btC3oXp7Ky8rhz1q9frwEDBmjUqFGaOnXqSdc9adIkFRYWBv5eVVVF5AAAYFhIgdOhQwd16NDhpONyc3O1b98+VVRUKCsrS5K0dOlS1dXVKScnp95zsrKy1KpVK5WVlWnw4MGSpE2bNmnbtm3Kzc0NjFu3bp2uvvpqjRgxQg899FCj1u31euX1ehs1FgAAxL6w/BSVJP385z9XZWWlZs+erdraWo0cOVJ9+vTRvHnzJEk7duzQgAED9NJLLyk7O1uS9Jvf/EYlJSWaM2eOkpOTNXbsWElHP2sjHf221NVXX638/Hw9+uijgbni4+MbFV7H8FNUAADEnlBev8PyIWNJmjt3rsaMGaMBAwYoLi5OgwcP1tNPPx34em1trTZt2qSDBw8Gjj355JOBsdXV1crPz9ezzz4b+Porr7yi3bt36+WXX9bLL78cOH7eeefpyy+/DNelAACAGBO2d3BaMt7BAQAg9kT99+AAAABEE4EDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgTtgCZ+/evRo2bJiSk5OVmpqqgoICffvttw2ec+jQIY0ePVrt2rXTWWedpcGDB6uysrLesf/73/907rnnyuPxaN++fWG4AgAAEKvCFjjDhg3TunXrtGTJEi1evFjvv/++Ro0a1eA548aN0xtvvKGFCxfqvffe086dO3XjjTfWO7agoECXXHJJOJYOAABinMc555r7Tjds2KCLLrpIK1asUJ8+fSRJpaWlGjhwoL766itlZGQcd47f71eHDh00b9483XTTTZKkjRs3qkePHiovL1ffvn0DY5977jktWLBA06ZN04ABA/TNN98oNTW10eurqqpSSkqK/H6/kpOTT+1iAQBARITy+h2Wd3DKy8uVmpoaiBtJysvLU1xcnD755JN6z6moqFBtba3y8vICx7p3767OnTurvLw8cGz9+vV64IEH9NJLLykurnHLr66uVlVVVdANAADYFZbA8fl86tixY9CxhIQEtW3bVj6f74TnJCYmHvdOTFpaWuCc6upqDR06VI8++qg6d+7c6PUUFxcrJSUlcMvMzAztggAAQEwJKXAmTpwoj8fT4G3jxo3hWqsmTZqkHj166Be/+EXI5/n9/sBt+/btYVohAABoCRJCGTx+/HjdfvvtDY7p2rWr0tPTtWvXrqDjhw8f1t69e5Wenl7veenp6aqpqdG+ffuC3sWprKwMnLN06VKtWbNGr7zyiiTp2MeH2rdvrylTpuj3v/99vfft9Xrl9Xobc4kAAMCAkAKnQ4cO6tChw0nH5ebmat++faqoqFBWVpako3FSV1ennJyces/JyspSq1atVFZWpsGDB0uSNm3apG3btik3N1eS9I9//EPfffdd4JwVK1bol7/8pT744AN169YtlEsBAACGhRQ4jdWjRw9dc801uuOOOzR79mzV1tZqzJgxuuWWWwI/QbVjxw4NGDBAL730krKzs5WSkqKCggIVFhaqbdu2Sk5O1tixY5Wbmxv4CaofRsyePXsC84XyU1QAAMC2sASOJM2dO1djxozRgAEDFBcXp8GDB+vpp58OfL22tlabNm3SwYMHA8eefPLJwNjq6mrl5+fr2WefDdcSAQCAUWH5PTgtHb8HBwCA2BP134MDAAAQTQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwJyHaC4gG55wkqaqqKsorAQAAjXXsdfvY63hDTsvA2b9/vyQpMzMzyisBAACh2r9/v1JSUhoc43GNySBj6urqtHPnTrVp00Yej6dZ77uqqkqZmZnavn27kpOTm/W+cTz2O/LY88hjzyOL/Y68xu65c0779+9XRkaG4uIa/pTNafkOTlxcnM4999ywzpGcnMw/GBHEfkceex557Hlksd+R15g9P9k7N8fwIWMAAGAOgQMAAMwhcJqZ1+tVUVGRvF5vtJdyWmC/I489jzz2PLLY78gLx56flh8yBgAAtvEODgAAMIfAAQAA5hA4AADAHAIHAACYQ+CEaNasWerSpYuSkpKUk5Oj5cuXNzh+4cKF6t69u5KSktSrVy+VlJREaKV2hLLnL7zwgq666iqdffbZOvvss5WXl3fSxwjHC/V5fsz8+fPl8Xg0aNCg8C7QmFD3e9++fRo9erQ6deokr9erCy64gH+3hCjUPZ8xY4YuvPBCtW7dWpmZmRo3bpwOHToUodXGtvfff1/XXXedMjIy5PF4tGjRopOe8+677+qyyy6T1+vVj3/8Y82ZMyf0iR0abf78+S4xMdG9+OKLbt26de6OO+5wqamprrKyst7xH330kYuPj3ePPPKIW79+vZs6dapr1aqVW7NmTYRXHrtC3fNbb73VzZo1y61atcpt2LDB3X777S4lJcV99dVXEV557Ap1z4/ZsmWLO+ecc9xVV13lbrjhhsgs1oBQ97u6utr16dPHDRw40H344Yduy5Yt7t1333WrV6+O8MpjV6h7PnfuXOf1et3cuXPdli1b3FtvveU6derkxo0bF+GVx6aSkhI3ZcoU9+qrrzpJ7rXXXmtw/ObNm90ZZ5zhCgsL3fr1690zzzzj4uPjXWlpaUjzEjghyM7OdqNHjw78/ciRIy4jI8MVFxfXO/7mm2921157bdCxnJwc9+tf/zqs67Qk1D3/ocOHD7s2bdq4v/71r+FaojlN2fPDhw+7fv36uT/96U9uxIgRBE4IQt3v5557znXt2tXV1NREaonmhLrno0ePdldffXXQscLCQnfFFVeEdZ0WNSZw7r33XnfxxRcHHRsyZIjLz88PaS6+RdVINTU1qqioUF5eXuBYXFyc8vLyVF5eXu855eXlQeMlKT8//4TjEawpe/5DBw8eVG1trdq2bRuuZZrS1D1/4IEH1LFjRxUUFERimWY0Zb9ff/115ebmavTo0UpLS1PPnj318MMP68iRI5Fadkxryp7369dPFRUVgW9jbd68WSUlJRo4cGBE1ny6aa7XztPyf7bZFHv27NGRI0eUlpYWdDwtLU0bN26s9xyfz1fveJ/PF7Z1WtKUPf+h++67TxkZGcf9w4L6NWXPP/zwQ/35z3/W6tWrI7BCW5qy35s3b9bSpUs1bNgwlZSU6IsvvtBdd92l2tpaFRUVRWLZMa0pe37rrbdqz549uvLKK+Wc0+HDh3XnnXdq8uTJkVjyaedEr51VVVX67rvv1Lp160bdD+/gwKzp06dr/vz5eu2115SUlBTt5Zi0f/9+DR8+XC+88ILat28f7eWcFurq6tSxY0c9//zzysrK0pAhQzRlyhTNnj072ksz691339XDDz+sZ599VitXrtSrr76qN998Uw8++GC0l4YG8A5OI7Vv317x8fGqrKwMOl5ZWan09PR6z0lPTw9pPII1Zc+PeeyxxzR9+nS9/fbbuuSSS8K5TFNC3fP//ve/+vLLL3XdddcFjtXV1UmSEhIStGnTJnXr1i28i45hTXmOd+rUSa1atVJ8fHzgWI8ePeTz+VRTU6PExMSwrjnWNWXP77//fg0fPly/+tWvJEm9evXSgQMHNGrUKE2ZMkVxcbxX0JxO9NqZnJzc6HdvJN7BabTExERlZWWprKwscKyurk5lZWXKzc2t95zc3Nyg8ZK0ZMmSE45HsKbsuSQ98sgjevDBB1VaWqo+ffpEYqlmhLrn3bt315o1a7R69erA7frrr9dPf/pTrV69WpmZmZFcfsxpynP8iiuu0BdffBEISUn6z3/+o06dOhE3jdCUPT948OBxEXMsMB3/O8dm12yvnaF9/vn0Nn/+fOf1et2cOXPc+vXr3ahRo1xqaqrz+XzOOeeGDx/uJk6cGBj/0UcfuYSEBPfYY4+5DRs2uKKiIn5MPESh7vn06dNdYmKie+WVV9zXX38duO3fvz9alxBzQt3zH+KnqEIT6n5v27bNtWnTxo0ZM8Zt2rTJLV682HXs2NH94Q9/iNYlxJxQ97yoqMi1adPG/e1vf3ObN292//rXv1y3bt3czTffHK1LiCn79+93q1atcqtWrXKS3BNPPOFWrVrltm7d6pxzbuLEiW748OGB8cd+THzChAluw4YNbtasWfyYeCQ888wzrnPnzi4xMdFlZ2e7jz/+OPC1/v37uxEjRgSN//vf/+4uuOACl5iY6C6++GL35ptvRnjFsS+UPT/vvPOcpONuRUVFkV94DAv1ef5/ETihC3W/ly1b5nJycpzX63Vdu3Z1Dz30kDt8+HCEVx3bQtnz2tpa97vf/c5169bNJSUluczMTHfXXXe5b775JvILj0HvvPNOvf9ePrbHI0aMcP379z/unN69e7vExETXtWtX95e//CXkeT3O8f4aAACwhc/gAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5/x+eQg27TcVSTQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "variables=problem.spatial_variables\n", "fig = plt.figure()\n", "proj = \"3d\" if len(variables) == 3 else None\n", "ax = fig.add_subplot(projection=proj)\n", "for location in problem.input_pts:\n", " coords = problem.input_pts[location].extract(variables).T.detach()\n", " ax.plot(coords.flatten(),torch.zeros(coords.flatten().shape),\".\",label=location)\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "22e502dd", "metadata": {}, "source": [ "## Perform a small training" ] }, { "attachments": {}, "cell_type": "markdown", "id": "075f43f5", "metadata": {}, "source": [ "Once we have defined the problem and generated the data we can start the modelling. Here we will choose a `FeedForward` neural network available in `pina.model`, and we will train using the `PINN` solver from `pina.solver`. We highlight that this training is fairly simple, for more advanced stuff consider the tutorials in the ***Physics Informed Neural Networks*** section of ***Tutorials***. For training we use the `Trainer` class from `pina.trainer`. Here we show a very short training and some method for plotting the results. Notice that by default all relevant metrics (e.g. MSE error during training) are going to be tracked using a `lightning` logger, by default `CSVLogger`. If you want to track the metric by yourself without a logger, use `pina.callback.MetricTracker`." ] }, { "cell_type": "code", "execution_count": 6, "id": "3bb4dc9b", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "GPU available: False, used: False\n", "TPU available: False, using: 0 TPU cores\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1499: 100%|██████████| 1/1 [00:00<00:00, 22.79it/s, v_num=0, val_loss=0.000654, bound_cond_loss=3.7e-6, phys_cond_loss=0.000236, train_loss=0.000239] " ] }, { "name": "stderr", "output_type": "stream", "text": [ "`Trainer.fit` stopped: `max_epochs=1500` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1499: 100%|██████████| 1/1 [00:00<00:00, 19.11it/s, v_num=0, val_loss=0.000654, bound_cond_loss=3.7e-6, phys_cond_loss=0.000236, train_loss=0.000239]\n" ] } ], "source": [ "from pina import Trainer\n", "from pina.solver import PINN\n", "from pina.model import FeedForward\n", "from lightning.pytorch.loggers import TensorBoardLogger\n", "\n", "\n", "# build the model\n", "model = FeedForward(\n", " layers=[10, 10],\n", " func=torch.nn.Tanh,\n", " output_dimensions=len(problem.output_variables),\n", " input_dimensions=len(problem.input_variables)\n", ")\n", "\n", "# create the PINN object\n", "pinn = PINN(problem, model)\n", "\n", "# create the trainer\n", "trainer = Trainer(solver=pinn, max_epochs=1500, logger=TensorBoardLogger('tutorial_logs'), accelerator='cpu', enable_model_summary=False) # we train on CPU and avoid model summary at beginning of training (optional)\n", "\n", "# train\n", "trainer.train()" ] }, { "cell_type": "markdown", "id": "f8b4f496", "metadata": {}, "source": [ "After the training we can inspect trainer logged metrics (by default **PINA** logs mean square error residual loss). The logged metrics can be accessed online using one of the `Lightning` loggers. The final loss can be accessed by `trainer.logged_metrics`" ] }, { "cell_type": "code", "execution_count": 7, "id": "f5fbf362", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'val_loss': tensor(0.0007),\n", " 'bound_cond_loss': tensor(3.7001e-06),\n", " 'phys_cond_loss': tensor(0.0002),\n", " 'train_loss': tensor(0.0002)}" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# inspecting final loss\n", "trainer.logged_metrics" ] }, { "cell_type": "markdown", "id": "0963d7d2", "metadata": {}, "source": [ "By using `matplotlib` we can also do some qualitative plots of the solution. " ] }, { "cell_type": "code", "execution_count": 8, "id": "ffbf0d5e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqgAAAKTCAYAAADVBfoyAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAdB5JREFUeJzt3XdcVfXjx/H3vRe4DBmiIqA4yJkzLf0609LUnJWzcuSoTCszc1SOtNLMSivTcqG5y9RSs9RCc2VaZqaZe4JbEJB17/n94Vd+X3IkCJwLvJ6Px308uud+zuF9PBBvzrQYhmEIAAAAcBFWswMAAAAA/4uCCgAAAJdCQQUAAIBLoaACAADApVBQAQAA4FIoqAAAAHApFFQAAAC4FDezA2QFp9OpU6dOydfXVxaLxew4AAAA+AfDMHT58mWFhobKar31PtI8UVBPnTqlsLAws2MAAADgXxw/flzFixe/5Zg8UVB9fX0lXV1hPz8/k9MAAADgn2JjYxUWFpbW224lTxTUa4f1/fz8KKgAAAAu7HZOx+QiKQAAALgUCioAAABcCgUVAAAALiVPnIN6uxwOh1JSUsyOAeQp7u7ustlsZscAAOQh+aKgGoah6OhoXbp0yewoQJ4UEBCg4OBg7kMMAMgS+aKgXiunQUFB8vb25pcokEUMw1BCQoLOnDkjSQoJCTE5EQAgL8jzBdXhcKSV00KFCpkdB8hzvLy8JElnzpxRUFAQh/sBAHcsz18kde2cU29vb5OTAHnXtZ8vzvEGAGSFPF9Qr+GwPpB9+PkCAGSlfFNQAQAAkDtQUAEAAOBSMlRQx44dq/vuu0++vr4KCgpSu3bttG/fvlvO06hRI1ksluteLVu2TBvTo0eP6z5v3rx55tYIOa5Ro0YaMGCA2TGy3ahRo1S9evUc+3oREREKCAi44+VERkbKYrFwmzUAQK6RoYK6fv169evXT1u3btWaNWuUkpKihx56SPHx8Ted56uvvlJUVFTaa/fu3bLZbOrQoUO6cc2bN083bsGCBZlbozzkWnEfN25cuunLli3LVef8RURE3PCPjkuXLslisSgyMvK2l9WjRw+1a9cuawPmITf6Y6Fu3bqKioqSv7+/OaEAAMigDN1mavXq1eneR0REKCgoSDt27FDDhg1vOE9gYGC69wsXLpS3t/d1BdVutys4ODgjcfIFT09PvfPOO3rmmWdUsGDBHP3aKSkpcnd3z5Jlubm5ae3atfrxxx/VuHHjLFlmTjEMQw6Hw+wYmebh4cHPFgAgV7mjc1BjYmIkXV9Cb2XGjBnq3LmzfHx80k2PjIxUUFCQypcvr759++r8+fM3XUZSUpJiY2PTvTLCMAwlJKea8jIMI0NZmzRpouDgYI0dO/aW4zZu3KgGDRrIy8tLYWFheuGFF9Lt2bZYLFq2bFm6eQICAhQRESFJOnLkiCwWixYtWqT7779fnp6emjdvns6fP68uXbqoWLFi8vb2VpUqVTK1d9vHx0c9e/bU0KFDbznu+PHj6tixowICAhQYGKi2bdvqyJEjkq4eYp89e7aWL1+edipIZGSk2rdvr/79+6ctY8CAAbJYLPrrr78kScnJyfLx8dHatWslXf3+eeGFFxQUFCRPT0/Vr19fv/zyS9r81w6Jf/vtt6pZs6bsdrs2btx4XdaDBw8qPDxc/fv3v+F2NQxDo0aNUokSJWS32xUaGqoXXngh7fOLFy+qW7duKliwoLy9vdWiRQvt37//pv82N9p7PGDAADVq1Cjt8/Xr12vSpElp/z5Hjhy54SH+JUuWqFKlSrLb7SpVqpTee++9dMstVaqU3n77bfXs2VO+vr4qUaKEPvvss5tmAwAgK2X6Rv1Op1MDBgxQvXr1VLly5duaZ9u2bdq9e7dmzJiRbnrz5s316KOPqnTp0jp48KBeffVVtWjRQlu2bLnhTb/Hjh2rN954I7PRdSXFobtHfJfp+e/EntHN5O1x+//sNptNb7/9th5//HG98MILKl68+HVjDh48qObNm+vNN9/UzJkzdfbsWfXv31/9+/fXrFmzMpRv6NCheu+993TPPffI09NTiYmJqlmzpoYMGSI/Pz+tXLlSXbt21V133aVatWplaNmjRo1SmTJl9OWXX6p9+/bXfZ6SkqJmzZqpTp06+umnn+Tm5qY333xTzZs3165duzRo0CDt3btXsbGxaesVGBioP/74Q59++mnactavX6/ChQsrMjJSFSpU0C+//KKUlBTVrVtXkjR48GAtWbJEs2fPVsmSJTV+/Hg1a9ZMBw4cSPfH1tChQzVhwgSFh4erYMGC6U5F2LVrl5o1a6ZevXrpzTffvOH6LlmyRB988IEWLlyoSpUqKTo6Wr///nva5z169ND+/fv19ddfy8/PT0OGDNHDDz+sPXv2ZGrP9aRJk/T333+rcuXKGj16tCSpSJEiaQX/mh07dqhjx44aNWqUOnXqpM2bN+u5555ToUKF1KNHj7Rx7733nsaMGaNXX31VX375pfr27av7779f5cuXz3A2AAAyItN7UPv166fdu3dr4cKFtz3PjBkzVKVKleuKTefOndWmTRtVqVJF7dq104oVK/TLL7/c9NzEYcOGKSYmJu11/PjxzK5GrvDII4+oevXqGjly5A0/Hzt2rJ544gkNGDBAZcuWVd26dfXhhx9qzpw5SkxMzNDXGjBgQNofCyEhISpWrJgGDRqk6tWrKzw8XM8//7yaN2+uxYsXZ3g9QkND9eKLL+q1115TamrqdZ8vWrRITqdT06dPV5UqVVSxYkXNmjVLx44dU2RkpAoUKCAvL6+000GCg4Pl4eGhRo0aac+ePTp79qwuXryoPXv26MUXX0z7/omMjNR9990nb29vxcfHa8qUKXr33XfVokUL3X333Zo2bZq8vLyu+8Np9OjRatq0qe666650xXXz5s1q1KiRBg0adNNyKknHjh1TcHCwmjRpohIlSqhWrVrq06ePJKUV0+nTp6tBgwaqVq2a5s2bp5MnT163p/t2+fv7y8PDQ97e3mn/Pjf6A+/999/Xgw8+qOHDh6tcuXLq0aOH+vfvr3fffTfduIcffljPPfecypQpoyFDhqhw4cL68ccfM5UNAICMyNQe1P79+2vFihXasGHDDffo3Uh8fLwWLlyYtmfnVsLDw1W4cGEdOHBADz744HWf2+122e32DOe+xsvdpj2jm2V6/jvh5Z65x0C+8847euCBBzRo0KDrPvv999+1a9cuzZs3L22aYRhyOp06fPiwKlaseNtf595770333uFw6O2339bixYt18uRJJScnKykpKdNP5hoyZIg+/fRTzZw5Ux07drxuPQ4cOCBfX9900xMTE3Xw4MGbLrNy5coKDAzU+vXr5eHhoXvuuUetWrXS5MmTJV3do3rtMPjBgweVkpKievXqpc3v7u6uWrVqae/evemW+89/C+lq6WzatKneeuutf71zQYcOHTRx4kSFh4erefPmevjhh9W6dWu5ublp7969cnNzU+3atdPGFypUSOXLl78uR1bbu3ev2rZtm25avXr1NHHiRDkcjrRSW7Vq1bTPLRaLgoODdebMmWzNBgCAlMGCahiGnn/+eS1dulSRkZEqXbr0bc/7xRdfKCkpSU8++eS/jj1x4oTOnz+vkJCQjMS7bRaLJUOH2V1Bw4YN1axZMw0bNizdYVhJiouL0zPPPJPu/MZrSpQoIenqOv/zPMkbPZbyn+cGv/vuu5o0aZImTpyoKlWqyMfHRwMGDFBycnKm1iMgIEDDhg3TG2+8oVatWl23HjVr1kxXtK8pUqTITZdpsVjUsGFDRUZGym63q1GjRqpataqSkpK0e/dubd68+YbF/t/889/iWo7Q0FAtWLBAPXv2lJ+f303nDwsL0759+7R27VqtWbNGzz33nN59912tX78+w1kkyWq13tY2zCr/PM3AYrHI6XRm29cDAOCaDB3i79evn+bOnav58+fL19dX0dHRio6O1pUrV9LGdOvWTcOGDbtu3hkzZqhdu3YqVKhQuulxcXF65ZVXtHXrVh05ckTr1q1T27ZtVaZMGTVrZs5eTlc1btw4ffPNN9qyZUu66TVq1NCePXtUpkyZ614eHh6SrharqKiotHn279+vhISEf/2amzZtUtu2bfXkk0+qWrVqCg8P199//31H6/H888/LarVq0qRJ163H/v37FRQUdN16XLtFkoeHxw2vqL///vsVGRmpyMhINWrUSFarVQ0bNtS7776rpKSktD2md911lzw8PLRp06a0eVNSUvTLL7/o7rvv/tfsXl5eWrFihTw9PdWsWTNdvnz5X8e3bt1aH374oSIjI7Vlyxb98ccfqlixolJTU/Xzzz+njT1//rz27dt30xz/3IaStHPnznTvb/bv878qVqyYbv2lq9u5XLlyNzwlAACAnJahgjplyhTFxMSoUaNGCgkJSXstWrQobcyxY8eu+yW6b98+bdy4Ub169bpumTabTbt27VKbNm1Urlw59erVSzVr1tRPP/10R4fx86IqVaroiSee0Icffphu+pAhQ7R582b1799fO3fu1P79+7V8+fJ0V7Y/8MAD+vjjj/Xbb79p+/btevbZZ2/rQpyyZctqzZo12rx5s/bu3atnnnlGp0+fvqP18PT01BtvvHHdejzxxBMqXLiw2rZtq59++kmHDx9WZGSkXnjhBZ04cULS1avLd+3apX379uncuXNpexCvnYf6559/qn79+mnT5s2bp3vvvTdtb6iPj4/69u2rV155RatXr9aePXvUp08fJSQk3PD780Z8fHy0cuVKubm5qUWLFoqLi7vhuIiICM2YMUO7d+/WoUOHNHfuXHl5ealkyZIqW7as2rZtqz59+mjjxo36/fff9eSTT6pYsWLXHX6/5oEHHtD27ds1Z84c7d+/XyNHjtTu3bvTjSlVqpR+/vlnHTlyROfOnbvhHs+XX35Z69at05gxY/T3339r9uzZ+vjjjzO1lxkAgOyQoYJqGMYNX/97yDkyMjLt1kXXlC9fXoZhqGnTptct08vLS999953OnDmj5ORkHTlyRJ999pmKFi2aqRXK60aPHn1d6ahatarWr1+vv//+Ww0aNNA999yjESNGKDQ0NG3Me++9p7CwMDVo0ECPP/64Bg0adFvnkb7++uuqUaOGmjVrpkaNGik4ODhLbpTfvXt3hYeHp5vm7e2tDRs2qESJEnr00UdVsWJF9erVS4mJiWmH0vv06aPy5cvr3nvvVZEiRdL2BFapUkUBAQGqXr26ChQoIOlqQXU4HGnnn14zbtw4PfbYY+ratatq1KihAwcO6LvvvsvQfWYLFCigb7/9VoZhqGXLljd8WEVAQICmTZumevXqqWrVqlq7dq2++eabtKMIs2bNUs2aNdWqVSvVqVNHhmFo1apVN/3DoVmzZho+fLgGDx6s++67T5cvX1a3bt3SjRk0aJBsNpvuvvtuFSlSRMeOHbtuOTVq1NDixYu1cOFCVa5cWSNGjNDo0aOvO3UEAACzWIyM3pjTBcXGxsrf318xMTHXnROYmJiow4cPq3Tp0vL09DQpIZC38XMGAPg3t+pr/3RHN+oHAABA7uRwOJWc6poXv1JQAQAA8qENX03Rtnda6q/9d3bxc3agoAIAAOQzR48fU7XdY1U/ZbOSts81O851KKgAAAD5iNNp6Oi8FxVouaxj7qVVtcNrZke6DgUVAAAgH1n3zTw1TPxBTsMi+6OTZXFzvdt6UlABAADyieNRZ1Xp15GSpH2lnlDRivX+ZQ5zUFABAADyAafT0O65gxRqOacztqIq3+UdsyPdFAUVAAAgH/j++2/ULG65JMlo9YGsngVMTnRzFFRkq8jISFksFl26dOmOlnPkyBFZLJbrnj0PAAD+3Ylzl3TXlmGyWgztD2mlove0NDvSLVFQXZTFYrnla9SoUWZHzDY9evS47nGqYWFhioqKUuXKlc0JBQBALmUYhrbOGa6ylhOKsfjrric+NDvSv3IzOwBuLCoqKu2/Fy1apBEjRmjfvn1p0649b166+o3ncDjk5pZ3N6fNZlNwcLDZMQAAyHW+/XG9WsfMlyxSYpO35V+gkNmR/hV7UF1UcHBw2svf318WiyXt/V9//SVfX199++23qlmzpux2uzZu3HjDPY8DBgxQo0aN0t47nU6NHTtWpUuXlpeXl6pVq6Yvv/zyllk++eQTlS1bVp6enipatKjat2+f9llSUpJeeOEFBQUFydPTU/Xr19cvv/xy02WNGjVK1atXTzdt4sSJKlWqVNrns2fP1vLly9P2FkdGRt7wEP/69etVq1Yt2e12hYSEaOjQoUpNTU37vFGjRnrhhRc0ePBgBQYGKjg4OE/veQYA4J+iLyUoZP1g2S2pOlqovorWfcLsSLcl7+5yuxXDkFISzPna7t6SxZIlixo6dKgmTJig8PBwFSxY8LbmGTt2rObOnaupU6eqbNmy2rBhg5588kkVKVJE999//3Xjt2/frhdeeEGff/656tatqwsXLuinn35K+3zw4MFasmSJZs+erZIlS2r8+PFq1qyZDhw4oMDAwAyv06BBg7R3717FxsZq1qxZkqTAwECdOnUq3biTJ0/q4YcfVo8ePTRnzhz99ddf6tOnjzw9PdOV0NmzZ2vgwIH6+eeftWXLFvXo0UP16tVT06ZNM5wNAIDcxDAMrZ3ztp607NMVi5eKPzklyzpIdsufBTUlQXo71Jyv/eopycMnSxY1evToDBWtpKQkvf3221q7dq3q1KkjSQoPD9fGjRv16aef3rCgHjt2TD4+PmrVqpV8fX1VsmRJ3XPPPZKk+Ph4TZkyRREREWrRooUkadq0aVqzZo1mzJihV155JcPrVKBAAXl5eSkpKemWh/Q/+eQThYWF6eOPP5bFYlGFChV06tQpDRkyRCNGjJDVevXgQNWqVTVy5NX7vZUtW1Yff/yx1q1bR0EFAOR5qzdvV7vz0ySLdLneqwoqWMLsSLctfxbUPOLee+/N0PgDBw4oISHhunKWnJycVjr/qWnTpipZsqTCw8PVvHlzNW/eXI888oi8vb118OBBpaSkqF69/7/Jr7u7u2rVqqW9e/dmfIUyYO/evapTp44s//OXYL169RQXF6cTJ06oRImrP4RVq1ZNN19ISIjOnDmTrdkAADDbmZgr8l4zRAUsiYryq6qQB/qbHSlD8mdBdfe+uifTrK+dRXx80u+JtVqtMgwj3bSUlJS0/46Li5MkrVy5UsWKFUs3zm6/8WPOfH199euvvyoyMlLff/+9RowYoVGjRt3yPNNb+beMWc3d3T3de4vFIqfTmW1fDwAAsxmGoeXzPlIf7VCK3FT48U8la+667Ch/FlSLJcsOs7uSIkWKaPfu3emm7dy5M62k3X333bLb7Tp27NgND+ffjJubm5o0aaImTZpo5MiRCggI0A8//KBmzZrJw8NDmzZtUsmSJSVdLZu//PKLBgwYcNOM0dHRMgwjbe/nP+9t6uHhIYfDcctMFStW1JIlS9ItZ9OmTfL19VXx4sVve90AAMhrvvvlTz16+kPJIl2q+YKKBN9tdqQMy58FNY964IEH9O6772rOnDmqU6eO5s6dq927d6cdvvf19dWgQYP00ksvyel0qn79+oqJidGmTZvk5+en7t27X7fMFStW6NChQ2rYsKEKFiyoVatWyel0qnz58vLx8VHfvn31yiuvKDAwUCVKlND48eOVkJCgXr163TBjo0aNdPbsWY0fP17t27fX6tWr9e2338rPzy9tTKlSpfTdd99p3759KlSokPz9/a9bznPPPaeJEyfq+eefV//+/bVv3z6NHDlSAwcOTDv/FACA/OZcXJK0arAKWS7rrHcZFWkxzOxImcJv8jykWbNmGj58uAYPHqz77rtPly9fVrdu3dKNGTNmjIYPH66xY8eqYsWKat68uVauXKnSpUvfcJkBAQH66quv9MADD6hixYqaOnWqFixYoEqVKkmSxo0bp8cee0xdu3ZVjRo1dODAAX333Xc3vatAxYoV9cknn2jy5MmqVq2atm3bpkGDBqUb06dPH5UvX1733nuvihQpok2bNl23nGLFimnVqlXatm2bqlWrpmeffVa9evXS66+/npl/OgAA8oQl8z5Vc22SQ1YFdP5McvMwO1KmWIx/nhCYC8XGxsrf318xMTHp9sRJUmJiog4fPqzSpUvL09PTpIRA3sbPGQCYb82Ofar69UMqarmks9X6qsgj48yOlM6t+to/sQcVAAAgl7sQn6yEFUNV1HJJFzxLqEirkWZHuiMUVAAAgFxu4cIItTV+kFMWFeg0VXL3MjvSHaGgAgAA5GI//H5QbY69I0k6f3cPeZSu9y9zuD4KKgAAQC4Vk5Cic8tfVXHLOV3yCFGRtm+aHSlL5JuCmgeuBQNcFj9fAGCOuYvmq6NztSTJu/0nkr2AyYmyRp4vqNduUp+QkGByEiDvuvbz9c8ndwEAss/6P4/q4cNvSZLOlussj3IPmJwo6+T5G/XbbDYFBASkPX/d29s73fPbAWSeYRhKSEjQmTNnFBAQIJvNZnYkAMgXYhNTdOKr4brfelqx7kVU5NHxZkfKUnm+oEpScHCwJKWVVABZKyAgIO3nDACQ/T7/comeTf1askj2dh9Kntc/dTE3yxcF1WKxKCQkREFBQUpJSTE7DpCnuLu7s+cUAHLQpr9OqsnfY2SzGjob3k5FKj1sdqQsly8K6jU2m41fpAAAINe6nJiiv78YqXrWE4pzK6gi7T8wO1K2yPMXSQEAAOQVM5d8oydTv5IkubV+X/IONDlR9qCgAgAA5AIb/orSA/vekLvFofMlmsuz2qNmR8o2FFQAAAAXF5uYoj+/eFNVrEeUYPNVoQ4fmh0pW1FQAQAAXNy0L79Vz9RFkiTbw+9IvkVNTpS9KKgAAAAuLHJvlBr9PVp2S4ouFmske43HzY6U7SioAAAALio2MUW/fTleNa37lWT1VsGOk6V88MAhCioAAICLmrLkez2bOvfqm2ZvSv7FzQ2UQyioAAAALujHPVF6YN8oeVmSFRNST/ZaPc2OlGMoqAAAAC4m5kqKdi4Zp/usfyvJ6i3/TlPzxaH9ayioAAAALmbKku/UN3WeJMnS7C0poITJiXIWBRUAAMCF/LDnlJr+PUqelhTFhDaQR62nzI6U4yioAAAALiImIUV/fDlWNa37lWjzyXeH9q+hoAIAALiIKUtW6VnHAkmSrfnYfHPV/j9RUAEAAFzAut0n1Xz/G7JbUhRTrJHc7+1mdiTTUFABAABMdikhWXu+elvVrQeVaCsg/46f5MtD+9dQUAEAAEw29ctVetqxUJJkazFO8i9mciJzUVABAABMtOaPE2px4A3ZLamKCXtA7jWfNDuS6SioAAAAJrkYn6y/l76latZDumLzlX+H/H1o/xoKKgAAgEk+/eIb9XEskiTZWo6X/EJMTuQaKKgAAAAm+P6P42p5aLQ8LA7FlGgqj3u6mB3JZVBQAQAActiF+GQdXDpGVaxHdMXNT/4dJnNo/39QUAEAAHLYZ4uXq5fjS0mSW6sJkm9RkxO5FgoqAABADlq186haHx4jD4tDl0o2k3u1jmZHcjkUVAAAgBxy5nKiji0bo0rWo7ri5q+ADh9zaP8GKKgAAAA5wDAMTVmwVL2MryRJ7q3flwoEmZzKNVFQAQAAcsDSXw6r44m35W5xKDa8pdyqPmZ2JJdFQQUAAMhmJy9d0fmVb6ii9ZiuuAfI79FJHNq/BQoqAABANnI6DU2ft0A9tVyS5NHuI6lAEZNTuTYKKgAAQDZatGmvup8eJ5vF0OXyj8lWqY3ZkVweBRUAACCbHDkXL2PNSJWynla8vah8271vdqRcgYIKAACQDRxOQ5/PnanHrd9Lkrw6fCp5BZgbKpegoAIAAGSDOT/sVJ+L70mSLlfrJWuZxiYnyj0oqAAAAFns79OXFbjhdQVbLirWp5R8W75pdqRchYIKAACQhVIcTi35/GO1tW6UU1b5dp4ueXibHStXoaACAABkoZmrt+qZyx9Lkq7UflGWsPtMTpT7UFABAACyyB/HL6nMz68p0BKnGP+K8mn6qtmRciUKKgAAQBZITHHo+3nv6kHrr0qxuMv/8ZmSm4fZsXIlCioAAEAWmLEiUs9cmS5JSmn4mlT0bpMT5V4UVAAAgDv0y+Fzuve3V1XAkqiLhe+V9/0vmB0pV6OgAgAA3IH4pFT9PP9N1bb+pSSrlwo+Pl2y2syOlatlqKCOHTtW9913n3x9fRUUFKR27dpp3759t5wnIiJCFosl3cvT0zPdGMMwNGLECIWEhMjLy0tNmjTR/v37M742AAAAOWzGVyvVJ3muJMl46C0psLTJiXK/DBXU9evXq1+/ftq6davWrFmjlJQUPfTQQ4qPj7/lfH5+foqKikp7HT16NN3n48eP14cffqipU6fq559/lo+Pj5o1a6bExMSMrxEAAEAO2fDXKTXeO0J2S4ouhjaSZ+2eZkfKE9wyMnj16tXp3kdERCgoKEg7duxQw4YNbzqfxWJRcHDwDT8zDEMTJ07U66+/rrZt20qS5syZo6JFi2rZsmXq3LnzdfMkJSUpKSkp7X1sbGxGVgMAAOCOXYxP1r7FI9THekQJNj8V7PKpZLGYHStPuKNzUGNiYiRJgYGBtxwXFxenkiVLKiwsTG3bttWff/6Z9tnhw4cVHR2tJk2apE3z9/dX7dq1tWXLlhsub+zYsfL39097hYWF3clqAAAAZIhhGJq28As95VgiSXJr/YHke+Odcci4TBdUp9OpAQMGqF69eqpcufJNx5UvX14zZ87U8uXLNXfuXDmdTtWtW1cnTpyQJEVHR0uSihYtmm6+okWLpn32T8OGDVNMTEza6/jx45ldDQAAgAz7ZschPXZ0jNwsTl28q408qrc3O1KekqFD/P+rX79+2r17tzZu3HjLcXXq1FGdOnXS3tetW1cVK1bUp59+qjFjxmTqa9vtdtnt9kzNCwAAcCdOXbqiuBWv6i5rlOI8CqvgY5PMjpTnZGoPav/+/bVixQr9+OOPKl68eIbmdXd31z333KMDBw5IUtq5qadPn0437vTp0zc9bxUAAMAMTqehOZ9P1+O6el2OV/upkvetT3VExmWooBqGof79+2vp0qX64YcfVLp0xm+j4HA49McffygkJESSVLp0aQUHB2vdunVpY2JjY/Xzzz+n2/MKAABgtvmRv6nnuQmSpJiqvWQr19TkRHlThg7x9+vXT/Pnz9fy5cvl6+ubdo6ov7+/vLy8JEndunVTsWLFNHbsWEnS6NGj9Z///EdlypTRpUuX9O677+ro0aPq3bu3pKtX+A8YMEBvvvmmypYtq9KlS2v48OEKDQ1Vu3btsnBVAQAAMm9/dKyKRg5WkPWSLhW4SwGt3zI7Up6VoYI6ZcoUSVKjRo3STZ81a5Z69OghSTp27Jis1v/fMXvx4kX16dNH0dHRKliwoGrWrKnNmzfr7rv///m0gwcPVnx8vJ5++mldunRJ9evX1+rVq6+7oT8AAIAZklOdWvn5uxpg/UWpcpP/4zMldy+zY+VZFsMwDLND3KnY2Fj5+/srJiZGfn5+ZscBAAB5zLTla/X4r4/Lx5KkuAbDVeDBQWZHynUy0tfu6D6oAAAAed2Ow2dUc8cQ+ViSdL7wfSrQ+CWzI+V5FFQAAICbiE9K1c75I1TDekBXrAVU6MlZktVmdqw8j4IKAABwE7O/+ELdkxdJkoyWE6QAnl6ZEyioAAAANxC565Ba/D1SbhanzpZqLe+aXcyOlG9QUAEAAP7hfFySLix9RaWtpxXjUVRFOn1kdqR8hYIKAADwPwzD0MLPp+pRY62cssirw2eSV0GzY+UrFFQAAID/8c2mneoc/a4k6ULVp+VRtpG5gfIhCioAAMB/HT8fr4A1L6mQ5bLO+ZRT4TZjzI6UL1FQAQAAJDmchlbPflsNLb8pWe4q2DVCcrObHStfoqACAABIWvztOj0Z85kkKb7BcNmCK5mcKP+ioAIAgHxv99GzqrJtkLwsyYouXEcFGz9vdqR8jYIKAADytSvJDu2cN1SVLYcVZ/VT0W4zJSsVyUz86wMAgHzt80UL1CVpydU3rT6QxS/U3ECgoAIAgPwrcud+PXxgpGwWQ6fDH1OBGu3NjgRRUAEAQD51JvaKkpe/oOKWc7poL6aiHSeaHQn/RUEFAAD5jmEYWh4xQQ8Zm5Uqm3wej5A8/cyOhf+ioAIAgHznqzXr9fj5jyRJF2u/Io+StUxOhP9FQQUAAPnK36fOq9yml+RjSVJUwXtVpNlgsyPhHyioAAAg30hKdWjn7FdUxXJIcVZfBfeYLVltZsfCP1BQAQBAvvHl4s/V8b+3lEpt9aEs/sVNToQboaACAIB8Yesff6vJvpGSpBPhnRVQ41GTE+FmKKgAACDPuxiXpOSvnlNRyyWdsZdU8c4fmB0Jt0BBBQAAeZphGFoV8ZYaGr8oWW7ye3KO5OFtdizcAgUVAADkaat//FGPnf1EknT+P6/KM6y6uYHwryioAAAgzzoSfV7h61+UpyVFxwLrKuShl8yOhNtAQQUAAHlSisOp3bMHqLzlmC5ZA1SsR4RkpfrkBmwlAACQJ33zxSy1uvK1JMnR5hPZ/IqanAi3i4IKAADynJ17/tL9e6/eUupQmW4qVL2lyYmQERRUAACQp8QkJCnpy2dUyHJZJ+1lFN55gtmRkEEUVAAAkGcYhqG1M0eqtnOnEuWhgK6zJTe72bGQQRRUAACQZ3y/7nu1PvuZJOlsvVHyKV7Z5ETIDAoqAADIEw6ejFa5n16Uh8WhQ4UbK6zJc2ZHQiZRUAEAQK6XmOLQodnPqbQlShdshVWqxwzJYjE7FjKJggoAAHK9VXM/UNPkdXLIKstjM2QtUMjsSLgDFFQAAJCrbfl5i5odGS9JOlr5eRW8u5G5gXDHKKgAACDXOn3hkgp++6x8LEk67FtT4Y+ONDsSsgAFFQAA5EpOp6GdM55XBR3RJYu/QnvOkaw2s2MhC1BQAQBArvTdkhlqFn/1UaYJD38se8HiJidCVqGgAgCAXGf3nt2qs3uEJOmv8B4Kva+NyYmQlSioAAAgV7mccEXGl70UYInXEc8KKv/4eLMjIYtRUAEAQK5hGIa2zBikKs6/FCdvBXafKwuPMs1zKKgAACDX2Pjdl2pybp4k6XSj8fILKWtyImQHCioAAMgVjh49rIpbXpbVYuiP4Ed1V6OuZkdCNqGgAgAAl5eckqrzc3uqsCVGx9xK6e6nJpsdCdmIggoAAFzextnDVSPlV12Rh7weny2b3dvsSMhGFFQAAODSftv0nRoenypJOnTvCBUJr25uIGQ7CioAAHBZZ89Gq+iafnKzOLUroKkqtexvdiTkAAoqAABwSQ6HU4dn9FSozuqUNUTlek+TLBazYyEHUFABAIBL2jD/HdVK3KRkwybHo9PlWaCg2ZGQQyioAADA5fyx/SfVPfCeJGlv5UEKq1zf5ETISRRUAADgUi6cP6eAFb1lt6TozwJ1Va39MLMjIYdRUAEAgMtwOpzaP6OnwhSt05YiKtV7Dued5kMUVAAA4DI2LXpHtRPWK8WwKaHtdPkEFDE7EkxAQQUAAC5h744Nqr1vgiRpV8WBKl29kbmBYBoKKgAAMF3MhXPyW9FbHpZU/e5TXzU6vmp2JJiIggoAAExlOJ06OL27ihmnFWUJUnifCFmsVJT8jK0PAABMtW3R26qRsFHJhpvi286QL+ed5nsUVAAAYJoDv67XPX+9L0n6tcIglane0OREcAUUVAAAYIrLF8/I55te8rA4tN3nftXuNMTsSHARFFQAAJDjDKdTh6d3V4hxVicswSrbexbnnSIN3wkAACDH7Vz8lqrGb1aS4a7LbabLv2AhsyPBhVBQAQBAjjry2w+qvPcDSdK2Cq+o4j0NTE4EV0NBBQAAOSb+4ml5f91b7haHtno3Ur2Or5gdCS6IggoAAHKG06ljM7oqyDivowpVuT6zZLVRRXA9visAAECO2LX4DVWM+1mJhrtiWk9TYMFAsyPBRVFQAQBAtjvy6xrdvfdDSdLmckNUtWZ9kxPBlVFQAQBAtoo7f0oFvnlabhanNvs8qEadXzY7ElwcBRUAAGQbw5GqEzOeUGHjgo5Yiqli7+mcd4p/xXcIAADINn/MHaoKCb8qwbArod0sFeS8U9wGCioAAMgWhzYtUdXD0yRJP1cZqbur1TY5EXKLDBXUsWPH6r777pOvr6+CgoLUrl077du375bzTJs2TQ0aNFDBggVVsGBBNWnSRNu2bUs3pkePHrJYLOlezZs3z/jaAAAAlxB7ar8Kr3lekhTp31aNHnvO5ETITTJUUNevX69+/fpp69atWrNmjVJSUvTQQw8pPj7+pvNERkaqS5cu+vHHH7VlyxaFhYXpoYce0smTJ9ONa968uaKiotJeCxYsyNwaAQAAUzmTr+hCRBf5KV57rOVUo88nslgsZsdCLmIxDMPI7Mxnz55VUFCQ1q9fr4YNG97WPA6HQwULFtTHH3+sbt26Sbq6B/XSpUtatmxZpnLExsbK399fMTEx8vPzy9QyAABA1vjz0x6qFLVUFw1fnXlijcqXq2h2JLiAjPS1OzoHNSYmRpIUGHj7JzwnJCQoJSXlunkiIyMVFBSk8uXLq2/fvjp//vxNl5GUlKTY2Nh0LwAAYL6D33+qSlFL5TQs2llrAuUUmZLpPahOp1Nt2rTRpUuXtHHjxtue77nnntN3332nP//8U56enpKkhQsXytvbW6VLl9bBgwf16quvqkCBAtqyZYtsNtt1yxg1apTeeOON66azBxUAAPNcOLhD3p83l6eS9W2Rnmr+3Psc2keajOxBzXRB7du3r7799ltt3LhRxYsXv615xo0bp/HjxysyMlJVq1a96bhDhw7prrvu0tq1a/Xggw9e93lSUpKSkpLS3sfGxiosLIyCCgCASRwJF3X2vToKdkRpm1tNVX5ltbztHmbHggvJ9kP8/fv314oVK/Tjjz/edjmdMGGCxo0bp++///6W5VSSwsPDVbhwYR04cOCGn9vtdvn5+aV7AQAAkxiGDk/vpmBHlE4aRVS4WwTlFHckQwXVMAz1799fS5cu1Q8//KDSpUvf1nzjx4/XmDFjtHr1at17773/Ov7EiRM6f/68QkJCMhIPAACY4OCyt1TmwgYlGW460GiywkuUMDsScrkMFdR+/fpp7ty5mj9/vnx9fRUdHa3o6GhduXIlbUy3bt00bNiwtPfvvPOOhg8frpkzZ6pUqVJp88TFxUmS4uLi9Morr2jr1q06cuSI1q1bp7Zt26pMmTJq1qxZFq0mAADIDuf+WKtSv0+QJH0b9pLub8zvbty5DBXUKVOmKCYmRo0aNVJISEjaa9GiRWljjh07pqioqHTzJCcnq3379unmmTDh6jezzWbTrl271KZNG5UrV069evVSzZo19dNPP8lut2fRagIAgKyWfPGk3L7qJZsMrbM/qBbdh5odCXnEHd0H1VVwH1QAAHKYI0XHPnhQJeJ+198qIe++P6p40cJmp4ILy7H7oAIAgPzp8MJBKhH3u2INL51tMZ1yiixFQQUAABkStWWhSu+PkCR9X3ak6tWubW4g5DkUVAAAcNsSTv0lv+8GSJK+9mmvdl2eMTcQ8iQKKgAAuC1G0mXFRHSUj67oV8vdqvP0h3KzUSWQ9fiuAgAA/84wdHhGD4UkH9VpI0C2jrNUxN/H7FTIoyioAADgXx37ZpzCz6xVsmHTjtofqlrFCmZHQh5GQQUAALd0cff3KvbreEnS8uAX1KJFG5MTIa+joAIAgJtKOX9EtiW9ZJNT37s/qJY9X5PFYjE7FvI4CioAALixlCs6O6Oj/IxY/WmEq1yvz+Rtdzc7FfIBCioAALieYejY588qNGGfLhgFdL7ldJUK5mb8yBkUVAAAcJ3TP3yiEseWyWFYtPbusWpYq6bZkZCPUFABAEA68Qc2KfCnEZKkxQG99FiHriYnQn5DQQUAAGmMy9FKWdBV7krVOmtdNevztmxWLopCzqKgAgCAq1KTFT29kwIc57XfKK4iT05XYAG72amQD1FQAQCAJCnqi5cVErNTsYaX/rp/iqqGFzM7EvIpCioAANDFLXMUsm+OJGlJieFq1biByYmQn1FQAQDI55JP/Cbv716WJM337KIu3Z7lZvwwFQUVAID8LOGC4uZ0kV3J2qB7VL/3BHm628xOhXyOggoAQH7ldChq5hMKTI7SEaOoLI9OU4nCBcxOBVBQAQDIr6KXva6Qc5uVYNi1peZENaha1uxIgCQKKgAA+VLM9sUK3vWJJGle0UHq1KqFyYmA/0dBBQAgn0k5+bvsK/pLkr7waKcuvQbKys344UIoqAAA5Cfx53V5did5KkmbVE339v5IBexuZqcC0qGgAgCQXzhSdXpml7SLopyPzlDpID+zUwHXoaACAJBPnFkySEXP/6w4w1Ob7/2Qi6LgsiioAADkA7FbIhS0Z5YkaU7IMHVu2czkRMDNUVABAMjjUo5uk9d/nxQ1x6Ozuj3Vn4ui4NIoqAAA5GWXo3Xl8y5yV6rW6T416DOBi6Lg8iioAADkValJOjejg/xSz2mfs7jcH/tMpYv4mp0K+FcUVAAA8iLD0LnFz6vwpV2KMby1rfZHalgl3OxUwG2hoAIAkAdd/mmKCv+9SA7DolmhI/VEi8ZmRwJuGwUVAIA8JuXABnn98LokabpnD/Xu0ZuLopCrUFABAMhLLh1T8oKucpNDK1RfD/V+k4uikOtQUAEAyCuSE3RxZgf5OC7pD2cp+bafotJFCpidCsgwCioAAHmBYejC/D4qGPuXzhp+2vGfj3V/5RJmpwIyhYIKAEAeELP2XQUeWaEUw6bZxUarW/P6ZkcCMo2CCgBALpe051v5bnpbkjTV5xk916MrF0UhV6OgAgCQixmn98j5ZU9ZZehLS1M90me4vD24KAq5GwUVAIDcKv68Yme1l5czQVudd6vkk5NVvKC32amAO0ZBBQAgN0pN1oWIzvJPPKmjziCdfOhT3XdXUbNTAVmCggoAQG5jGLq4ZIACz27TZcNL39z9nh6rX9XsVECWoaACAJDLxG+cooJ758lpWDSl8DA906GV2ZGALEVBBQAgF0ndv06e616TJE316K4+PfvK3cavc+QtfEcDAJBbnDuglIXdZZNTy4z71aTXGBX08TA7FZDlKKgAAOQGVy4qduaj8nJc1g5nWfm2n6xywX5mpwKyBQUVAABX50jVpTlPyi/hqE4YhfV7vcl6sEqY2amAbENBBQDAxV3+erACojYqwbBrbqlxeuqhWmZHArIVBRUAABeWtHWGfH+fIUn6wHeQXnziUVksPMYUeRsFFQAAF+U89JPcVr8iSfrE2kU9+zwvLw+byamA7EdBBQDAFV04rKT5T8gmh1Y466p2t7cV4u9ldiogR1BQAQBwNYmxio1oL6/UGP3uDFdyq49Us1Sg2amAHENBBQDAlTgdipnXQ36xBxRtFNSP1Sfq0VplzE4F5CgKKgAALiRuxavyP75OiYa7poWO0fPtGpodCchxFFQAAFxE0s8zVeDXqZKkD3wG6KUeXWSzcsU+8h8KKgAALsBxIFK2bwdJkqZaO6lrn4EqYHczORVgDgoqAABmO/u3UhY8ITc5tNxZX7W6j1Pxgt5mpwJMQ0EFAMBM8ecVN+sReTritN1ZTta2H6lGSa7YR/5GQQUAwCypSYqd3VEFEk7omLOIfvnPx2pdM9zsVIDpKKgAAJjBMBT3RV/5ndmuWMNbEaXf1TPNa5mdCnAJFFQAAEyQ+MM7KrBviVINqyb4v6rBT7aRlSv2AUkUVAAAclzqriXy/GmsJOk99z7q37uPPN1tJqcCXAcFFQCAHGQc/0XG0mclSRFGS7Xq+ZqC/DxNTgW4FgoqAAA55dIxJX7eUe5GstY6aii0w7uqFOpvdirA5VBQAQDICYmxipv1mLySL2iPs6SONvpQD1UuZnYqwCVRUAEAyG6OVMXP76YCMX/rtBGgrypMUM8HKpudCnBZFFQAALLZlRWD5XPsR10xPPR+kTEa3LGJLBau2AduhoIKAEA2Stk8VV6/zZAkveX5koY81Vkebvz6BW6FnxAAALKJc9/3sn0/TJL0gR5Xj14vKNDHw+RUgOujoAIAkB2idillUXdZ5dQXzkb6z5NjVCaogNmpgFyBggoAQFaLOamEiMdkdyZok6OSbK0/UJ0yhc1OBeQaGSqoY8eO1X333SdfX18FBQWpXbt22rdv37/O98UXX6hChQry9PRUlSpVtGrVqnSfG4ahESNGKCQkRF5eXmrSpIn279+fsTUBAMAVJMYqbtaj8k46o7+dxbSz7kd69L5ws1MBuUqGCur69evVr18/bd26VWvWrFFKSooeeughxcfH33SezZs3q0uXLurVq5d+++03tWvXTu3atdPu3bvTxowfP14ffvihpk6dqp9//lk+Pj5q1qyZEhMTM79mAADkNEeK4uc+qQKX/tIZI0ALy72v55rXMDsVkOtYDMMwMjvz2bNnFRQUpPXr16thw4Y3HNOpUyfFx8drxYoVadP+85//qHr16po6daoMw1BoaKhefvllDRo0SJIUExOjokWLKiIiQp07d/7XHLGxsfL391dMTIz8/PwyuzoAAGSeYejKkn7y2j1PCYZdbxSZoNHPPiG7m83sZIBLyEhfu6NzUGNiYiRJgYGBNx2zZcsWNWnSJN20Zs2aacuWLZKkw4cPKzo6Ot0Yf39/1a5dO23MPyUlJSk2NjbdCwAAMyWvnyCv3fPkMCx6y2uQhvXsTDkFMinTBdXpdGrAgAGqV6+eKle++dMwoqOjVbRo0XTTihYtqujo6LTPr0272Zh/Gjt2rPz9/dNeYWFhmV0NAADumPP3xfKIfFOS9K61p57u008B3txOCsisTBfUfv36affu3Vq4cGFW5rktw4YNU0xMTNrr+PHjOZ4BAABJ0pFNci57TpI0w9lSTXu8rpKFfEwOBeRubpmZqX///lqxYoU2bNig4sWL33JscHCwTp8+nW7a6dOnFRwcnPb5tWkhISHpxlSvXv2Gy7Tb7bLb7ZmJDgBA1jm3X0lzO8tupGiVo5aKth+vmiVvftobgNuToT2ohmGof//+Wrp0qX744QeVLl36X+epU6eO1q1bl27amjVrVKdOHUlS6dKlFRwcnG5MbGysfv7557QxAAC4nLizSpjVTvbUWP3qLKPjjT5Qq2q33mkD4PZkaA9qv379NH/+fC1fvly+vr5p54j6+/vLy8tLktStWzcVK1ZMY8eOlSS9+OKLuv/++/Xee++pZcuWWrhwobZv367PPvtMkmSxWDRgwAC9+eabKlu2rEqXLq3hw4crNDRU7dq1y8JVBQAgiyQnKGF2B3nHn9BRZ5BWVnpfrz9QyexUQJ6RoYI6ZcoUSVKjRo3STZ81a5Z69OghSTp27Jis1v/fMVu3bl3Nnz9fr7/+ul599VWVLVtWy5YtS3dh1eDBgxUfH6+nn35aly5dUv369bV69Wp5enpmcrUAAMgmToeuLOol77O/6aJRQB+HjtPb7RvIYrGYnQzIM+7oPqiugvugAgBySvLKIfL4ZaqSDDe9WuBNjXy+t/w83c2OBbi8HLsPKgAA+Yljy1R5/DJVkvSG2/Ma2KcH5RTIBhRUAABug/HXSlm+GyZJes/ZRZ2fGqBiAV4mpwLyJgoqAAD/5sQOpS7uKaucWuh4QDW6vKGqxQPMTgXkWRRUAABu5fxBJc55TO7OREU6qkkt31PjikX/fT4AmUZBBQDgZuLO6MrMtvJMvqhdztL6o+4kdf5PuNmpgDyPggoAwI0kxSkh4jF5xR/XUWeQvqr4gfo3r252KiBfyNSjTgEAyNMcKboyv6u8z+3SecNXH4W+o7Ed7+dep0AOoaACAPC/DENJy16Q19EfdMXw0Jt+ozTmqTZyt3HQEcgp/LQBAPA/Uta9Jfsf8+UwLBrhMUjD+jypAnb25wA5iYIKAMB/ObdHyH3ju5Kktyx99Eyf5xTkx2O3gZxGQQUAQJKx71tpxUuSpI+dj6pFj2EqE+Rrciogf6KgAgBwYrtSF/WQVU4tdtyv8PZv6b5SgWanAvItCioAIH87f1BJc9rL3ZmoHx3VFN90gh6uGmp2KiBfo6ACAPKvuDNKnNVO9v/eiH/bve/rqYblzE4F5HsUVABA/pQUpyuzH5Vn3DEddQZpfpkJeqV1TbNTARD3QQUA5EeOFCUu6Cqvs3/ovOGr94qO1fgujWW1ciN+wBVQUAEA+ct/b8TveeTqjfhH+YzQm0+1lae7zexkAP6LQ/wAgHwlZc0baTfiH+4+SMOe7ip/b3ezYwH4HxRUAEC+4dgyRe6bP5AkvWl5Ws/0eU6hAV4mpwLwTxRUAEC+YOz6QrbvhkqSPnB2UqunhqlsUW7ED7giCioAIO87sE7Opc9KkmY7mqlq59GqWbKgyaEA3AwFFQCQt53YoZQFT8hmpOprRx15t3lXD94dbHYqALdAQQUA5F3n9itpzqNyd1zRBkcVRTWeqA73lTQ7FYB/QUEFAORNsaeUOLON7MmX9LszXJtqvq+nG5c3OxWA20BBBQDkPVcu6sqsdvJMOKWDzhAtKvu+hrS5TxYLN+IHcgNu1A8AyFuSE5Q4p4O8Lu5TtFFQH4W+o/Fd7ucpUUAuQkEFAOQdjlQlLewmz6hfFGN4682ANzXuqZbycOOAIZCb8BMLAMgbDEPJy56X/dAaJRruet3rdY3q00EF7OyLAXIbCioAIE9I/X6kPP6Yr1TDqtfcXtbgp59S4QJ2s2MByAQKKgAg13Nu+khuWyZJkt7Q0+rZ6zmFBXqbnApAZlFQAQC5mrFzgaxrXpckTXB00cPdBqtSqL/JqQDcCQoqACD3+vt7OZf3kyTNTG2hSh1GqM5dhUwOBeBOUVABALnT0c1KXfikbIZDSx315NNmnFpUDTU7FYAsQEEFAOQ+Ub8r+fMOcnMmaZ3jHp178H11qlXK7FQAsggFFQCQu5w7oKSIdvJIjdPPzgra+Z+J6tOogtmpAGQhCioAIPeIOaHEma1kT7qg3c5S+rbKRA18uJrZqQBkMe5eDADIHeLPKXFma3kmROmgM0Sfl3lfbz/2H1ksPMIUyGsoqAAA15cYo8RZ7eQZc0gnjUL6uPgEvfN4Y9mslFMgL6KgAgBcW8oVJc7pKM9zf+ic4aexhcZqfI/m8nDjLDUgr6KgAgBclyNFSfOflOeprYo1vDTSd7Te7v2ovD349QXkZfz5CQBwTU6nkr98RvbDa5VouOtVz9c18uku8vd2NzsZgGxGQQUAuB7DUMqKl+Wxd4lSDJuGur2iwU/3VJCfp9nJAOQACioAwOWkrh0j919nymlY9LrlefXt85xKFPI2OxaAHEJBBQC4FOemD+W26T1J0mijlzr1HKDywb4mpwKQkyioAACX4dwxR9Y1wyVJExyd9GDXoapRoqDJqQDkNAoqAMAlGH8uk755UZL0aWor3d1hlBqULWJuKACmoKACAMy3f42cX/aSVU4tSG2sQu3G6eGqoWanAmASCioAwFyHf1LqgidkM1K1wvEfpT78ntrfG2Z2KgAmoqACAMxzfJtS5naQmzNJaxw1dOqBSepa9y6zUwEwGQUVAGCOqN+VPPtRuTuu6CdHZf1Zb5KeblzB7FQAXAAFFQCQ887uU9KstvJIvaxfnOW0seaHerFZFbNTAXARFFQAQM66cEhJM1rJnnxRvzvDtaLyJA1tW0MWi8XsZABchJvZAQAA+UjMCSXOaCXPxDP6yxmmheU/0Jvt61BOAaRDQQUA5IzLp6+W0/iTOugM0bRS72tc54ayWSmnANKjoAIAsl/CBSXObC3P2MM6YRTWh8Xe1fhuD8rdxplmAK5HQQUAZK/EGCXOaifPi/t02gjQuCLv6N2nHpbdzWZ2MgAuioIKAMg+yfFKnN1enmd/13nDV6MLjtXY3m3l5UE5BXBzFFQAQPZISVTi3C7yjNqmGMNbr/u+qbf7tJefp7vZyQC4OE7+AQBkPUeKkhZ0leex9Yo37BrmPUpvPNNZBX08zE4GIBegoAIAspbToaTFvWU/9L0SDXcN8Xhdrz3TTUG+nmYnA5BLUFABAFnH6VDSl8/Kvm+Zkg2bhrkP0eBne6tYgJfZyQDkIhRUAEDWcDqVvPR52fcsVoph02tuL+uFZ59TiULeZicDkMtQUAEAd84wlPzNS/L4Y54chkWv217UM8+8oNKFfcxOBiAXoqACAO6MYSh5xSvy+C1CTsOi163P66mnX1KZIF+zkwHIpSioAIDMMwylfPuqPHZMk9OwaITlOT3ZZ5AqBPuZnQxALkZBBQBkjmEodc0ouW/7RJI0Wn3UsfdgVQr1NzkYgNyOggoAyJTUH96W2+aJkqQxzp5q3fNVVS0eYGomAHkDBRUAkGGpkePl9tN4SdJYZzc1e2q4apYsaHIqAHkFBRUAkCGOnybKLfItSdJ4xxO6v9tI1SodaHIqAHkJBRUAcNucWz6Rbd1ISdL7jk76T9c3VLdMYZNTAchrKKgAgNvi/HmarN8NkyR95HhU1Z8Yo4blipicCkBelOGCumHDBrVu3VqhoaGyWCxatmzZLcf36NFDFovlulelSpXSxowaNeq6zytUqJDhlQEAZA/n9tmyfjtIkjTV0UblOr2tByoUNTkVgLwqwwU1Pj5e1apV0+TJk29r/KRJkxQVFZX2On78uAIDA9WhQ4d04ypVqpRu3MaNGzMaDQCQDZy/zZNWvChJmp76sMLav6NmlUNMTgUgL3PL6AwtWrRQixYtbnu8v7+//P3//554y5Yt08WLF/XUU0+lD+LmpuDg4IzGAQBkI+fOhdLyfrLK0GzHQyry2LtqWS3U7FgA8rgcPwd1xowZatKkiUqWLJlu+v79+xUaGqrw8HA98cQTOnbs2E2XkZSUpNjY2HQvAEDWcu5cKC3rK6sMzXc8qIBHP1Dbe4qbHQtAPpCjBfXUqVP69ttv1bt373TTa9eurYiICK1evVpTpkzR4cOH1aBBA12+fPmGyxk7dmzanll/f3+FhYXlRHwAyDf+v5w6Nd/xgHwenUQ5BZBjLIZhGJme2WLR0qVL1a5du9saP3bsWL333ns6deqUPDw8bjru0qVLKlmypN5//3316tXrus+TkpKUlJSU9j42NlZhYWGKiYmRnx/PfwaAO3F9Of1Qbe9hRwCAOxMbGyt/f//b6msZPgc1swzD0MyZM9W1a9dbllNJCggIULly5XTgwIEbfm6322W327MjJgDka5RTAK4gxw7xr1+/XgcOHLjhHtF/iouL08GDBxUSwlWiAJBTKKcAXEWGC2pcXJx27typnTt3SpIOHz6snTt3pl3UNGzYMHXr1u26+WbMmKHatWurcuXK1302aNAgrV+/XkeOHNHmzZv1yCOPyGazqUuXLhmNBwDIBMopAFeS4UP827dvV+PGjdPeDxw4UJLUvXt3RUREKCoq6ror8GNiYrRkyRJNmjTphss8ceKEunTpovPnz6tIkSKqX7++tm7dqiJFeEIJAGS3f5ZT70copwDMdUcXSbmKjJx0CwD4fzcqp+1qUE4BZL2M9LUcvw8qAMA1UE4BuCoKKgDkQ5RTAK6MggoA+QzlFICro6ACQD5COQWQG1BQASCfcO5cQDkFkCvk2JOkAADmSd0xR9ZvXpBVhhY4HpRf+w/Vqlpxs2MBwA1RUAEgj0v9eYbcvr16z+q5jqYq3PFDNa8SanIqALg5DvEDQB6WsmVKWjmNcLRQSJePKacAXB4FFQDyqJSfJsn9u6GSpOnO1ir95Id68O5gk1MBwL/jED8A5EEpkRPkHjlGkjTF+Yiqdn1X9cry+GgAuQMFFQDyEsNQ8rqx8tj4jiTpQ2dH1eoxTv8JL2RyMAC4fRRUAMgrDENJa0bLvvl9SdIHxuNq2Ost1SwZaHIwAMgYCioA5AWGoaRvX5N922RJ0rvqpqa9x6h6WIC5uQAgEyioAJDbGYaSVrwi+45pkqSxlp5q3XuUKhfzNzkYAGQOBRUAcjOnU4lfD5TnzlmSpLcsT+uRPsN1d6ifycEAIPMoqACQWzmduvJVf3ntnienYdGbtr7q/MyrKlfU1+xkAHBHKKgAkBs5HUr44hl57/1CDsOi0W7Pq9uzQ3RXkQJmJwOAO0ZBBYDcxpGi+EW95fP3MqUaVo12H6Cezw5SqcI+ZicDgCxBQQWA3CQlUXHzu6nA4e+UYtg0xvNlPdN3oIoFeJmdDACyDAUVAHKL5ATFze6oAid/UpLhrjE+Q/XCs/0V5OdpdjIAyFIUVADIDRJjdXnWY/I9vU3xhl1v+4/UoGf6qKCPh9nJACDLUVABwNUlXFDcjDbyPf+HYg1vvVNojIY83V1+nu5mJwOAbEFBBQBXFndGcdNaqUDMPl0wCuj9ou/otd6d5e3B/74B5F38Hw4AXFXMCcVNa6kCcUd02gjQ5OLv6fUej8jT3WZ2MgDIVhRUAHBFFw4pflpLFbhySieMwpoRPlHDn2wpd5vV7GQAkO0oqADgas7uU8L0lvJJOqtDzmAtqvCxXu/cRDarxexkAJAjKKgA4EqidunKzNbyTrmkv5xhWlHtEw15pIGslFMA+QgFFQBchHF8m5IiHpWX47J2OUtrQ61P9XLLWrJYKKcA8hcKKgC4AOehDUqd21Geziv6xVlOOxt8pv5N7zE7FgCYgoIKACZz7PtOzoVPysNI1k+OyjradJr6NLzb7FgAYBoKKgCYKPn3L2Vd+ozclaq1zhqKaTVNT9YuY3YsADAVBRUATJL480x5fDtQVhla6awj9w7T9FiVMLNjAYDpKKgAYIKEH96T94bRkqRFRhOFPfmJ6pYranIqAHANFFQAyEmGobhVw1Xgl48kSTPVTjV7TVS1EgVNDgYAroOCCgA5xelQ7JIX5ffn55Kkj21d1fyZsSoT5GtyMABwLRRUAMgJqcmKmd9T/oe+kdOw6H3P59T52ddVvKC32ckAwOVQUAEguyUn6NLsLgo4Galkw6b3CgxS72dfVhFfu9nJAMAlUVABIDtduaRLMx9TwNntumJ46P3A4er/dF/5e7mbnQwAXBYFFQCyS9wZxUxro4CYvYo1vPVR8Fsa2Ku7vDxsZicDAJdGQQWA7HDpmGI/ayX/hKM6a/hpRqn3NbjbY3K3Wc1OBgAuj4IKAFnMOLtPcdNbyy/ptE4YhfXF3R9rcIfmslotZkcDgFyBggoAWch54jddiWgn39RLOuAM1br7PtWAVg1ksVBOAeB2UVABIIukHNggx/xO8nEmaJeztHY1mqlnHqhhdiwAyHUoqACQBa7sWiq3r/rIUyn62VlR51rP1pP3lTc7FgDkShRUALhDcZs+k9eaIbLJqTXGffLsEqGWFYubHQsAci0KKgBklmEoZvVb8v/5XUnSV5Ymuuupz1StZCGTgwFA7kZBBYDMcDp04cuXFLhntiQpwtZBDZ/5QOFBviYHA4Dcj4IKABmVmqTznz+lQkdXymlYNNW7jx7rO0ZF/TzNTgYAeQIFFQAyIumyzs3ooMJntijZsGlywcHq+czLPLoUALIQBRUAblfcWZ3/rLUKx+5VvGHXtNDR6tuzjzzdeXQpAGQlCioA3AbjwmFd+qy1CiUe13nDVwvKfqDnH28vG0+HAoAsR0EFgH/hOLVLCTPbqWDqeZ0wCuv7mlPVr3UTng4FANmEggoAt5B08Cc553WSrzNefznDtLvxTPVsXMvsWACQp1FQAeAm4n9fJvelveWlFP3irKCLbeeofU2eDgUA2c1qdgAAcEUXf/pMnkufkodS9INxrxxPLNFDlFMAyBHsQQWA/2UYOvPNSAX9OkmS9LX1QZXrPV0VQgNNDgYA+QcFFQCucaQoev6zCj74pSRpnr2THug7USEB3iYHA4D8hYIKAJKUFKfoGZ0VfOYnOQyLZga8oI7PDucG/ABgAgoqgHzPiDujM1PbKjhuj64YHvq8+Ch1f+pZ2d24AT8AmIGCCiBfSz17QDHT2qho8kmdN3y1ovJE9X7sMVm5AT8AmIaCCiDfSjzys1LmdFAhZ4yOGkH6tcF0dW9yv9mxACDfo6ACyJdifv9G9qW95Ksk7TbCdbbN53qkZmWzYwEAREEFkA+dW/+ZCv44RDY5tVH3yKfrXDUuU9zsWACA/6KgAsg/DENRy0coZOeHkqSVtgdUsc9MhQcXNDkYAOB/UVAB5A+OFJ2c+6yKHb56j9OFXp31YN9JKuLnaXIwAMA/UVAB5H1JcTr+WUeFnd8kh2HR54VeVIdnhsvHzv8CAcAV8X9nAHmaIzZa0VPbKizhL10xPLS49Gg92fUZudmsZkcDANwEBRVAnnXl5G7Fz3pUxVJP67zhqw01P1a31m1lsXCPUwBwZRRUAHnSxd1r5L6kuwob8TpsBOto8zl6pE5ts2MBAG4DBRVAnnNq/UwV+fEVuStVv6mCrE8sUKNy4WbHAgDcpgyfhLVhwwa1bt1aoaGhslgsWrZs2S3HR0ZGymKxXPeKjo5ON27y5MkqVaqUPD09Vbt2bW3bti2j0QDkd4ahI0uGK/THl+SuVP3gVl+F+n6rapRTAMhVMlxQ4+PjVa1aNU2ePDlD8+3bt09RUVFpr6CgoLTPFi1apIEDB2rkyJH69ddfVa1aNTVr1kxnzpzJaDwA+VVqsg5P76ZSf1y9x+kyn46q8dISlSgaaHIwAEBGZfgQf4sWLdSiRYsMf6GgoCAFBATc8LP3339fffr00VNPPSVJmjp1qlauXKmZM2dq6NChGf5aAPIXZ8JFHZ/6mErH7lCqYdVXIS+pbe/XZHezmR0NAJAJOXaflerVqyskJERNmzbVpk2b0qYnJydrx44datKkyf+HslrVpEkTbdmy5YbLSkpKUmxsbLoXgPwp6exhnZ54v0rG7lCc4alvKn+gDs8Mp5wCQC6W7QU1JCREU6dO1ZIlS7RkyRKFhYWpUaNG+vXXXyVJ586dk8PhUNGiRdPNV7Ro0evOU71m7Nix8vf3T3uFhYVl92oAcEExB7cpYUpjhSQfVbQRqK33z9MjHXpwGykAyOWy/Sr+8uXLq3z58mnv69atq4MHD+qDDz7Q559/nqllDhs2TAMHDkx7HxsbS0kF8pnobV/Jf9Wz8lKS9qmk4h6bryZVK5sdCwCQBUy5zVStWrW0ceNGSVLhwoVls9l0+vTpdGNOnz6t4ODgG85vt9tlt9uzPScA13Ro1Qcque0N2WRom7W6CvVcoJrFQ82OBQDIIqY862/nzp0KCQmRJHl4eKhmzZpat25d2udOp1Pr1q1TnTp1zIgHwFU5Hfpr9vMK3zZKNhn63rOZwl9cqbsopwCQp2R4D2pcXJwOHDiQ9v7w4cPauXOnAgMDVaJECQ0bNkwnT57UnDlzJEkTJ05U6dKlValSJSUmJmr69On64Ycf9P3336ctY+DAgerevbvuvfde1apVSxMnTlR8fHzaVf0A4Ei8rINTu6jCpZ8kSd8U7qOmT4+TpwfPGwGAvCbD/2ffvn27GjdunPb+2rmg3bt3V0REhKKionTs2LG0z5OTk/Xyyy/r5MmT8vb2VtWqVbV27dp0y+jUqZPOnj2rESNGKDo6WtWrV9fq1auvu3AKQP6UcPaozk17ROWSDyrJcNcPFUaqVef+XAwFAHmUxTAMw+wQdyo2Nlb+/v6KiYmRn5+f2XEAZKEz+7bKurCLChsXdN7w0977p6r+Ay3NjgUAyKCM9DWOjQFwWYd+WqiQdS/IS0k6qDAldpyv+pWqmh0LAJDNKKgAXI9haM+Xb6rC7vdktRja7lZDoX0W6i5O+wGAfIGCCsClGKlJ+nNaH1U+vVyySD/4tlGt56apgJen2dEAADmEggrAZSTGntexqY+pcsJvchgWrS05QE26j5DNZsod8QAAJqGgAnAJ54/t1ZXZ7VXOcUJxhqd23PeemrV60uxYAAATUFABmO7or98r4OunVFxxilIhnW49R/ffW9/sWAAAk1BQAZhq96qpKv/zq3K3OLTXWlY+3b9Q9ZKlzY4FADARBRWAKQynQzsiXtG9x2ZIFmmrZwNV6DtPAf7+ZkcDAJiMggogxyXGx+qvKU/o3rgNkqT1Rburbp/35e7G/5IAABRUADns9PH9iovoqOqOQ0o23LS92ijd/+jzZscCALgQCiqAHLNv+1oVXtFLd+mSzstfUS2mq+5/HjI7FgDAxVBQAeSI7csnq+qvI+RhSdVBa2l5dV+syiXLmR0LAOCCKKgAspUjNVXbZryoOlFzJYv0q3d9les7TwV8A8yOBgBwURRUANkmNuaCDk3tojpXtkqSthTrpdo935XVZjM5GQDAlVFQAWSL4wf3KHVeJ1V3HlOi4a4/a41TnZa9zY4FAMgFKKgAstyujd8obG1fFdRlnVGgYtvNVs17GpodCwCQS1BQAWQZwzC0efEE1dozVu4Wh/a7lVPBnl+oTGgps6MBAHIRCiqALJGYlKTtn/ZV/QtLrl4M5feg7n52jjy9C5gdDQCQy1BQAdyx06ejdWp6Z9VP+U2S9Et4f9375BhZrFaTkwEAciMKKoA7snvnz/Jb1k33KFoJsutwgw9034NPmB0LAJCLUVABZNr6ryNUc8dQFbBc0WlLETk6zVelCrXMjgUAyOUoqAAyLCklRRumD1HT0zMki/S3V3UVe3qRfAoGmx0NAJAHUFABZMjZc+d1YNqTapq0WZK0q1gnVXnqY1ncPExOBgDIKyioAG7b7t075fllV9XRMSXLTQdrjVbVh/uZHQsAkMdQUAHclh9XLdI9P7+kAEu8zlsClfzYbFWszM33AQBZj4IK4JaSUxxaO2uEmp2cLJvF0CF7RRXt84UKFQ4zOxoAII+ioAK4qbMXLmnvZ0/p4cQfJIu0p2hrVeg1TVYPL7OjAQDyMAoqgBvas3ePtPhJNTQOKtWw6mCNV3V3m0GSxWJ2NABAHkdBBZCOYRhau3q57tn6ggpbYhRj8VVc2xkqf08zs6MBAPIJCiqANIkpDq2c9bZan/xAHhaHjnuEq2DPL1Us+C6zowEA8hEKKgBJ0vEzF7RnxjN6LOl7ySIdKNJUd/WOkMVewOxoAIB8hoIKQFt/+10Flj+lZjooh6w6Xv1llWn7GuebAgBMQUEF8jGn09DyZQvV4PfBKmyJVazFV8ntpqtUteZmRwMA5GMUVCCfiklI1urpw/XY+c/kZnHqpGc5Fe61WH5FSpsdDQCQz1FQgXxo3/FonYzopU6OjZJFOlKstUr1mCa5c39TAID5KKhAPrN242aVWPO0HrAcV6psOl13pEo1fYHzTQEALoOCCuQTKQ6nFs+frtYHRsrPkqBL1kBZO81RsfINzI4GAEA6FFQgHzgTk6AfP3tFT8TPlSzSSd+qCu69SDb/ULOjAQBwHQoqkMft2HdECQt7qpOxQ5J0rMwTKtF5ouTmYW4wAABugoIK5FGGYWjpd2tVY0t/1bREK0kein1wvEo0eMrsaAAA3BIFFciDLiem6MtZH6hT9AR5W5J03q2ovLstUJESNc2OBgDAv6KgAnnMXyfO6q/Zz+uplG+vnm8aWFuhvebL4lPY7GgAANwWCiqQh6zYsE1h655VO8tBSVJUtedVrO0bktVmcjIAAG4fBRXIA64kO/T5vJlqf2SUAi1xirP6ymg3VSFVW5kdDQCADKOgArncodMx2jxziHonLpTVYuhMgYoq3HOhrIGlzI4GAECmUFCBXOy7X/5UgRXP6knLLskiRZftouCOEyV3T7OjAQCQaRRUIBdKTnVq9hdf6uG/hqqY5bySZFdS8/cU/J+uZkcDAOCOUVCBXObEhXitnDlGT13+TB4Why54hsmv2wL5hVYxOxoAAFmCggrkIhv+OKz4Jf30jDZJFulM8YcU9OQMydPP7GgAAGQZCiqQC6Q6nIr4+ns1/G2gyllPyiGrYusPV9CDL0kWi9nxAADIUhRUwMWduZyo+TMnqfeF91XAmqhYt0LyfPxzFQyvZ3Y0AACyBQUVcGFb/47SkYUDNcC5SrJI5wrXVuEec6UCQWZHAwAg21BQARfkcBqKWLVB92wbqM7WA5KkizX6q3DLNyQbP7YAgLyN33SAizkdm6g5EZ+oz/kJCrDGK8FaQLZHp6pg5dZmRwMAIEdQUAEXsmHvSR1bPESvGN9IFulCQBUFdp8nFSxpdjQAAHIMBRVwASkOp6avWK//7BikJ/97SP9StacV2Potyc3D5HQAAOQsCipgspOXrujzWZ+o76UJ8rcm6IrNV7ZHpiiAQ/oAgHyKggqYaO0fxxW1ZIiGaqVkkS4WrKqC3eZySB8AkK9RUAETJKc6NWXZj7p/1ytqYj0oSYqt/owKtnqTQ/oAgHyPggrksGPnE/R5xGT1j30/7ZC+22NT5Xd3K7OjAQDgEiioQA5atfOozi0dptcsVw/pXwqspoBuc6WAEmZHAwDAZVBQgRyQmOLQx1+tU5M/h+rh/x7Sv1zjWQU8PIZD+gAA/AMFFchmB8/Gae6syRoQP/G/h/T95P7YFPlySB8AgBuioALZxDAMfbXtkOJWva6RllWSRYotVE1+XTmkDwDArVBQgWwQm5iiiYu+1SMHh6uK9YgkKb5mX/m1GM0hfQAA/gUFFchivx67qFWfv6+Xkz+VjzVJV9z85fHYVPlUfNjsaAAA5AoUVCCLOJyGZqzbpSIbXtXrto2SRboc/B/5Pj5L8gs1Ox4AALkGBRXIAtExifro88Xqc+ZNlbKdlkNWpTQYIt/Gr0hWm9nxAADIVSiowB1a82eUdn35tkY658nD6lC8V4i8O8+SZ8k6ZkcDACBXoqACmZSY4tCk5ZtU+/fX9bLtd8kixd31sAq0/0TyKmh2PAAAci0KKpAJf5++rFmzZ2hg/PsqYotRisVDaj5OBWr1lCwWs+MBAJCrWTM6w4YNG9S6dWuFhobKYrFo2bJltxz/1VdfqWnTpipSpIj8/PxUp04dfffdd+nGjBo1ShaLJd2rQoUKGY0GZDvDMDR/8wGtn9xXYxNGqoglRvH+ZeXed4Pca/einAIAkAUyXFDj4+NVrVo1TZ48+bbGb9iwQU2bNtWqVau0Y8cONW7cWK1bt9Zvv/2WblylSpUUFRWV9tq4cWNGowHZ6lJCsl6b+Y3uXt1RfazfSJKuVOsun/4/SUEVTU4HAEDekeFD/C1atFCLFi1ue/zEiRPTvX/77be1fPlyffPNN7rnnnv+P4ibm4KDg29rmUlJSUpKSkp7Hxsbe9t5gMzYeui8Vs3/SMNSpsrXekVJbr5yf+RjeVVqZ3Y0AADynAzvQb1TTqdTly9fVmBgYLrp+/fvV2hoqMLDw/XEE0/o2LFjN13G2LFj5e/vn/YKCwvL7tjIp1IcTk1a9ZuOz3pKo1M/kK/liuKL3it7/y2yUk4BAMgWOV5QJ0yYoLi4OHXs2DFtWu3atRUREaHVq1drypQpOnz4sBo0aKDLly/fcBnDhg1TTExM2uv48eM5FR/5yKGzcRr6YYRab+2sDrb1csqi5HqD5PP0d1IAfxQBAJBdcvQq/vnz5+uNN97Q8uXLFRQUlDb9f08ZqFq1qmrXrq2SJUtq8eLF6tWr13XLsdvtstvtOZIZ+Y9hGFq07YiiVo7TOMsXcrc6dMUrWF4dp8ujdAOz4wEAkOflWEFduHChevfurS+++EJNmjS55diAgACVK1dOBw4cyKF0wFUX4pP1zsI1evToaHW2/iVJulK2tbwe/Yh7mwIAkENypKAuWLBAPXv21MKFC9WyZct/HR8XF6eDBw+qa9euOZAOuCpy3xmtWTxZr6V+Jj9rgpJt3nJrNUFe1R/n9lEAAOSgDBfUuLi4dHs2Dx8+rJ07dyowMFAlSpTQsGHDdPLkSc2ZM0fS1cP63bt316RJk1S7dm1FR0dLkry8vOTv7y9JGjRokFq3bq2SJUvq1KlTGjlypGw2m7p06ZIV6wjcUmKKQx+s2K7yv47WW7aNkkVKCKoh784zpMBws+MBAJDvZLigbt++XY0bN057P3DgQElS9+7dFRERoaioqHRX4H/22WdKTU1Vv3791K9fv7Tp18ZL0okTJ9SlSxedP39eRYoUUf369bV161YVKVIks+sF3JY9p2I1bd48vRw3QcVt5+SUVc4Gg+TdaIhk40FrAACYwWIYhmF2iDsVGxsrf39/xcTEyM/Pz+w4yAWcTkMzNuxT8rpxeta6TDaLoSs+xeXVaaZUorbZ8QAAyHMy0tfYRYR8JyrmisbPX6XuUW+puu2gJCnx7o7yavOe5MkfOAAAmI2Cinxl5e+ntG3ph3rTmCkfa5KS3Xzl3naSPKs8ZnY0AADwXxRU5AuXE1M0/qstqrN3jN6wbZMs0pXQOvLqNF3yL252PAAA8D8oqMjzfjlyQfMWzNGQxEkKsV2Qw2KTGr8ur/ovSlab2fEAAMA/UFCRZyWlOvThd7tUaMs4TXRbfXWvqV9peXWeJYXeY3Y8AABwExRU5El7o2I1ed6XejF2gsq6nZQkJVfvLq+Hx0oePianAwAAt0JBRZ7icBqaFvm3kn4crw+sS+VudSjJs4jsj02RR9mmZscDAAC3gYKKPOPY+QRNmL9CPc+OS7t9VFK5NrK3myR5B5qcDgAA3C4KKnI9wzC0cNsxHVg5Ue9Y5srLmnz19lGt35O9akfJYjE7IgAAyAAKKnK1M5cTNW7Rj2p39C11sf0hSUoMayDP9lO5fRQAALkUBRW51rd/RGnDV1M00jlN/rYEpVrtsj40Wp61npasVrPjAQCATKKgIteJ/e9N92vvfUtjbVuv3j6qSDV5dZwuFSlndjwAAHCHKKjIVTYfOKcvFs3S0OSPVdR2SU7Z5Gz4irzuHyTZ3M2OBwAAsgAFFblCYopD76/8TSW2v60P3NZd3Wvqf5e8Ok6XtVgNs+MBAIAsREGFy/v12EXNWrhQA+M+UGm305KklHufllez0ZK7l8npAABAVqOgwmUlpjj08fd/yG/LeE2yrZLVaijRO1ie7T+Ve3gjs+MBAIBsQkGFS9p14pKmzV+sF+M+UBm3U5Kk5Mqd5dnyHckrwNxwAAAgW1FQ4VKSU536ZO2fsm8ar4nWb2SzGlcfVfrIR/Io38LseAAAIAdQUOEy/jwVo6nzv1T/2PdV3nZCkpR0dwfZW43nUaUAAOQjFFSYLsXh1Kc//CVtGK8PrMvlZnUqyV5I9nYfyl6xldnxAABADqOgwlT7oi9r8oIl6nvxPVW0HZMkJVV4RPbW70k+hUxOBwAAzEBBhSlSHU5Ni9ynlMgJes+6VO5Wh5I8Csqj7UTZK7UzOx4AADARBRU57sCZy/pwwXI9ff5dVbYdkSQllm0lz7YTpQJFTM0GAADMR0FFjnE4Dc3asF+x6yZogvVLeVgdSnL3l0ebD+RZ+VHJYjE7IgAAcAEUVOSIw+fiNWnB13rq7HhVsx2SJCXe1Vye7T6UfIuanA4AALgSCiqyldNpaM6m/bq05l29Y/lKdmuqktz95NFqgjyrdmSvKQAAuA4FFdnm0Nk4TVn4lXqcnaBK1qOSpCulm8rrkY8kvxCT0wEAAFdFQUWWczgNRazfq5Qfx2ms5Ru5WZ1KdA+QR6t35VW1A3tNAQDALVFQkaX+Pn1ZMxYsVJ8L76uM9ZQkKaFsW3m3fY8r9AEAwG2hoCJLpDicmvHDbnlueFtjratltRq6Yi8sz3YT5V2xtdnxAABALkJBxR3781SM5s6fo76xk1TCdlaSlHB3Z3m3Hid5FTQ5HQAAyG0oqMi0pFSHpn2/U4W3vKmxth8kq5TgFSKvRz+Wd9kmZscDAAC5FAUVmbLz+CUtWTBdz8VPVojtgiTpSvWe8m4xWrL7mpwOAADkZhRUZEhiikNTV21T6e2jNca2WbJI8T4l5dNhirxK1TM7HgAAyAMoqLht2w+f18qFU9Qv8VMVtsXKKauS7ntOPg+9Lrl7mR0PAADkERRU/KuE5FRN/fonVfl9jEbadkgW6bJ/Ofl2nCqvYjXNjgcAAPIYCipuafP+0/p58bvqkzxXvrYrSrW4KbXOAPk+MERy8zA7HgAAyIMoqLih2MQUzfpqlRr8NVovWQ9IFimm8D3y7zhFbkEVzY4HAADyMAoqrrN211EdW/aGnnMsk7vVoUSrt9RklPz/00eyWs2OBwAA8jgKKtKcuZyoeQvnq83x8WpijZIs0oXiTRXYYZLkX8zseAAAIJ+goEKGYWjZlj/l/H64XtLVG+7HuReWR5v3FFi5rWSxmB0RAADkIxTUfO7ouTgtmzdZj1+YrCKWGEnShYpPKLDN25JXgLnhAABAvkRBzadSHU4tXLtFxTa/rhctv0oW6aJ3Kfl2mKzA0vXNjgcAAPIxCmo+tPv4BW1cME5Pxs9WAUuiUuWmuFovqOBDQyU3u9nxAABAPkdBzUcSUxya9/W3qvH7SD3731tHnS14jwp3maIAbh0FAABcBAU1n/j5r+M6uGS4uiV/LXerQ1esPkp5YKSK1OXWUQAAwLVQUPO4mCspWrZwuh48MkG1Lecki3S62EMq2mmSvPxCzY4HAABwHQpqHha57VcZ3w5Vd+PnqxdBuQfL3vY9Fa3cyuxoAAAAN0VBzYNOX4rThs/f1MPnZsrHkqRU2XSmch+FthkhefiYHQ8AAOCWKKh5iMNp6PvvvlHpra+rg+WoZJFO+FZTkc6TFVqsitnxAAAAbgsFNY/Yd+SYDi0comZXvpXVYijW4qu4hiNU/P7eXAQFAAByFQpqLnclKVVrFn+sugfeU3lLrGSRDoS2Ueku78vPt4jZ8QAAADKMgpqLbdu+TdZVL6uNc5dkkaLcS8qj3USVqfSA2dEAAAAyjYKaC525eEk75o7QA+fmyW5JVZI8dLxyP5Vp96rk5mF2PAAAgDtCQc1FnE5DkasWKXz7KLVQ1NXD+X7/Uejjk1UmuIzZ8QAAALIEBTWXOHRwn04vHqgHkjZKks5bAhXX+E2VafC4ZLGYnA4AACDrUFBdXGLiFW2bP0b3Hp2ucEuSUg2r/irRRRU6v6VCPgXNjgcAAJDlKKgubPdPy+X3wzA1NE5KFulve2X5t/9QlcvWNDsaAABAtqGguqCLUYd1ZP5Luufyj5KkC/LXifteVZUWT8vCPU0BAEAeR0F1IUZqknYtGaeyeyfrHiXJYVj0S1B7VXpinKoGFDY7HgAAQI6goLqIU7+tlnPlK6qWekyS9Ketomyt39d/qtc1ORkAAEDOoqCaLPH8cR1ZMFAVzn0vSTpv+GlXxZdVv31/ubuxeQAAQP5DAzKLI0UHV0xQyG8TVUGJchgW/ejXRhUef0eNQ0LMTgcAAGAaCqoJLuxep6SvX9JdyUclSbss5RX74Fg9WP8BWbinKQAAyOcoqDko9cJRHVv4ssLPrJEknTd8tan0C3qg8wAV8OQRpQAAABIFNWekXNGpVe8o8LdPFP7fq/O/935YpTuMVZvwkmanAwAAcCkU1OxkGIrf+ZWSV72q0JRoSdIvulvnGoxWsweayGrlcD4AAMA/UVCziXF6j85+MUBB536Wj6RTRqDWhb2ghzv11X2+nmbHAwAAcFkU1Kx25aIurhotvz8iFCSnkgx3feH5qCo8NkJdyxU3Ox0AAIDLo6BmFadDSb/MlmPNGyqYekmStMa4T2fqjFCnpvXlbuMRpQAAALcjw61pw4YNat26tUJDQ2WxWLRs2bJ/nScyMlI1atSQ3W5XmTJlFBERcd2YyZMnq1SpUvL09FTt2rW1bdu2jEYzz7GfFfthA9m/fUneqZe031lM7wePV8UBX+uJ5g0ppwAAABmQ4eYUHx+vatWqafLkybc1/vDhw2rZsqUaN26snTt3asCAAerdu7e+++67tDGLFi3SwIEDNXLkSP3666+qVq2amjVrpjNnzmQ0Xs6KjVL8wp7SzIfkd+lPxRremujWU8c6fq+Bzz6j4gW9zU4IAACQ61gMwzAyPbPFoqVLl6pdu3Y3HTNkyBCtXLlSu3fvTpvWuXNnXbp0SatXr5Yk1a5dW/fdd58+/vhjSZLT6VRYWJief/55DR069F9zxMbGyt/fXzExMfLz88vs6ty+1CSlbp4sI3K83J1X5DQs+sLZSGdqDVavZrXk7cGZEwAAAP8rI30t25vUli1b1KRJk3TTmjVrpgEDBkiSkpOTtWPHDg0bNiztc6vVqiZNmmjLli03XGZSUpKSkpLS3sfGxmZ98FuI/mqYgvfMkCTtcJbVl0WeV8+Oj6psUd8czQEAAJAXZfvJkdHR0SpatGi6aUWLFlVsbKyuXLmic+fOyeFw3HBMdHT0DZc5duxY+fv7p73CwsKyLf+NRAZ21GFnUQ23PK8jbb7S2/27UU4BAACySK48Fj1s2DANHDgw7X1sbGyOltT2jWvpE32jQXXD5e/tnmNfFwAAID/I9oIaHBys06dPp5t2+vRp+fn5ycvLSzabTTab7YZjgoODb7hMu90uu92ebZn/jZvNqhealDft6wMAAORl2X6Iv06dOlq3bl26aWvWrFGdOnUkSR4eHqpZs2a6MU6nU+vWrUsbAwAAgPwjwwU1Li5OO3fu1M6dOyVdvY3Uzp07dezYMUlXD79369Ytbfyzzz6rQ4cOafDgwfrrr7/0ySefaPHixXrppZfSxgwcOFDTpk3T7NmztXfvXvXt21fx8fF66qmn7nD1AAAAkNtk+BD/9u3b1bhx47T3184F7d69uyIiIhQVFZVWViWpdOnSWrlypV566SVNmjRJxYsX1/Tp09WsWbO0MZ06ddLZs2c1YsQIRUdHq3r16lq9evV1F04BAAAg77uj+6C6ihy/DyoAAAAyJCN9jWdwAgAAwKVQUAEAAOBSKKgAAABwKRRUAAAAuBQKKgAAAFwKBRUAAAAuhYIKAAAAl0JBBQAAgEuhoAIAAMClUFABAADgUiioAAAAcCkUVAAAALgUCioAAABcCgUVAAAALoWCCgAAAJdCQQUAAIBLoaACAADApVBQAQAA4FIoqAAAAHApFFQAAAC4FDezA2QFwzAkSbGxsSYnAQAAwI1c62nXetut5ImCevnyZUlSWFiYyUkAAABwK5cvX5a/v/8tx1iM26mxLs7pdOrUqVPy9fWVxWLJka8ZGxursLAwHT9+XH5+fjnyNZF12H65H9sw92Mb5n5sw9wtp7efYRi6fPmyQkNDZbXe+izTPLEH1Wq1qnjx4qZ8bT8/P34oczG2X+7HNsz92Ia5H9swd8vJ7fdve06v4SIpAAAAuBQKKgAAAFwKBTWT7Ha7Ro4cKbvdbnYUZALbL/djG+Z+bMPcj22Yu7ny9ssTF0kBAAAg72APKgAAAFwKBRUAAAAuhYIKAAAAl0JBBQAAgEuhoAIAAMClUFBvYfLkySpVqpQ8PT1Vu3Ztbdu27Zbjv/jiC1WoUEGenp6qUqWKVq1alUNJcSMZ2X7Tpk1TgwYNVLBgQRUsWFBNmjT51+2N7JfRn8FrFi5cKIvFonbt2mVvQPyrjG7DS5cuqV+/fgoJCZHdble5cuX4f6mJMrr9Jk6cqPLly8vLy0thYWF66aWXlJiYmENp8U8bNmxQ69atFRoaKovFomXLlv3rPJGRkapRo4bsdrvKlCmjiIiIbM95QwZuaOHChYaHh4cxc+ZM488//zT69OljBAQEGKdPn77h+E2bNhk2m80YP368sWfPHuP111833N3djT/++COHk8MwMr79Hn/8cWPy5MnGb7/9Zuzdu9fo0aOH4e/vb5w4cSKHk+OajG7Daw4fPmwUK1bMaNCggdG2bducCYsbyug2TEpKMu69917j4YcfNjZu3GgcPnzYiIyMNHbu3JnDyWEYGd9+8+bNM+x2uzFv3jzj8OHDxnfffWeEhIQYL730Ug4nxzWrVq0yXnvtNeOrr74yJBlLly695fhDhw4Z3t7exsCBA409e/YYH330kWGz2YzVq1fnTOD/QUG9iVq1ahn9+vVLe+9wOIzQ0FBj7NixNxzfsWNHo2XLlumm1a5d23jmmWeyNSduLKPb759SU1MNX19fY/bs2dkVEf8iM9swNTXVqFu3rjF9+nSje/fuFFSTZXQbTpkyxQgPDzeSk5NzKiJuIaPbr1+/fsYDDzyQbtrAgQONevXqZWtO3J7bKaiDBw82KlWqlG5ap06djGbNmmVjshvjEP8NJCcna8eOHWrSpEnaNKvVqiZNmmjLli03nGfLli3pxktSs2bNbjoe2Scz2++fEhISlJKSosDAwOyKiVvI7DYcPXq0goKC1KtXr5yIiVvIzDb8+uuvVadOHfXr109FixZV5cqV9fbbb8vhcORUbPxXZrZf3bp1tWPHjrTTAA4dOqRVq1bp4YcfzpHMuHOu1GXccvwr5gLnzp2Tw+FQ0aJF000vWrSo/vrrrxvOEx0dfcPx0dHR2ZYTN5aZ7fdPQ4YMUWho6HU/qMgZmdmGGzdu1IwZM7Rz584cSIh/k5lteOjQIf3www964okntGrVKh04cEDPPfecUlJSNHLkyJyIjf/KzPZ7/PHHde7cOdWvX1+GYSg1NVXPPvusXn311ZyIjCxwsy4TGxurK1euyMvLK8eysAcV+Idx48Zp4cKFWrp0qTw9Pc2Og9tw+fJlde3aVdOmTVPhwoXNjoNMcjqdCgoK0meffaaaNWuqU6dOeu211zR16lSzo+E2REZG6u2339Ynn3yiX3/9VV999ZVWrlypMWPGmB0NuRB7UG+gcOHCstlsOn36dLrpp0+fVnBw8A3nCQ4OztB4ZJ/MbL9rJkyYoHHjxmnt2rWqWrVqdsbELWR0Gx48eFBHjhxR69at06Y5nU5Jkpubm/bt26e77rore0Mjncz8HIaEhMjd3V02my1tWsWKFRUdHa3k5GR5eHhka2b8v8xsv+HDh6tr167q3bu3JKlKlSqKj4/X008/rddee01WK/vEXN3Nuoyfn1+O7j2V2IN6Qx4eHqpZs6bWrVuXNs3pdGrdunWqU6fODeepU6dOuvGStGbNmpuOR/bJzPaTpPHjx2vMmDFavXq17r333pyIipvI6DasUKGC/vjjD+3cuTPt1aZNGzVu3Fg7d+5UWFhYTsaHMvdzWK9ePR04cCDtjwtJ+vvvvxUSEkI5zWGZ2X4JCQnXldBrf2wYhpF9YZFlXKrL5PhlWbnEwoULDbvdbkRERBh79uwxnn76aSMgIMCIjo42DMMwunbtagwdOjRt/KZNmww3NzdjwoQJxt69e42RI0dymykTZXT7jRs3zvDw8DC+/PJLIyoqKu11+fJls1Yh38voNvwnruI3X0a34bFjxwxfX1+jf//+xr59+4wVK1YYQUFBxptvvmnWKuRrGd1+I0eONHx9fY0FCxYYhw4dMr7//nvjrrvuMjp27GjWKuR7ly9fNn777Tfjt99+MyQZ77//vvHbb78ZR48eNQzDMIYOHWp07do1bfy120y98sorxt69e43JkydzmylX9NFHHxklSpQwPDw8jFq1ahlbt25N++z+++83unfvnm784sWLjXLlyhkeHh5GpUqVjJUrV+ZwYvyvjGy/kiVLGpKue40cOTLngyNNRn8G/xcF1TVkdBtu3rzZqF27tmG3243w8HDjrbfeMlJTU3M4Na7JyPZLSUkxRo0aZdx1112Gp6enERYWZjz33HPGxYsXcz44DMMwjB9//PGGv9uubbfu3bsb999//3XzVK9e3fDw8DDCw8ONWbNm5XhuwzAMi2Gw3x0AAACug3NQAQAA4FIoqAAAAHApFFQAAAC4FAoqAAAAXAoFFQAAAC6FggoAAACXQkEFAACAS6GgAgAAwKVQUAEAAOBSKKgAAABwKRRUAAAAuJT/A8OgXW3NqfguAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "pts = pinn.problem.spatial_domain.sample(256, 'grid', variables='x')\n", "predicted_output = pinn.forward(pts).extract('u').as_subclass(torch.Tensor).cpu().detach()\n", "true_output = pinn.problem.truth_solution(pts).cpu().detach()\n", "pts = pts.cpu()\n", "fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(8, 8))\n", "ax.plot(pts.extract(['x']), predicted_output, label='Neural Network solution')\n", "ax.plot(pts.extract(['x']), true_output, label='True solution')\n", "plt.legend()" ] }, { "cell_type": "markdown", "id": "bf47b98a", "metadata": {}, "source": [ "The solution is overlapped with the actual one, and they are barely indistinguishable. We can also take a look at the loss using `TensorBoard`:" ] }, { "cell_type": "code", "execution_count": 9, "id": "fcac93e4", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Reusing TensorBoard on port 6006 (pid 10248), started 1:01:24 ago. (Use '!kill 10248' to kill it.)" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", " \n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Load the TensorBoard extension\n", "%load_ext tensorboard\n", "# Show saved losses\n", "%tensorboard --logdir 'tutorial_logs'" ] }, { "cell_type": "markdown", "id": "58172899", "metadata": {}, "source": [ "As we can see the loss has not reached a minimum, suggesting that we could train for longer" ] }, { "cell_type": "markdown", "id": "33e672da", "metadata": {}, "source": [ "## What's next?\n", "\n", "Congratulations on completing the introductory tutorial of **PINA**! There are several directions you can go now:\n", "\n", "1. Train the network for longer or with different layer sizes and assert the finaly accuracy\n", "\n", "2. Train the network using other types of models (see `pina.model`)\n", "\n", "3. GPU training and speed benchmarking\n", "\n", "4. Many more..." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }