{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "6f71ca5c", "metadata": {}, "source": [ "# Tutorial: Physics Informed Neural Networks on PINA\n", "\n", "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial1/tutorial.ipynb)\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "ef4949c9", "metadata": {}, "source": [ "In this tutorial, we will demonstrate a typical use case of **PINA** on a toy problem, following the standard API procedure. \n", "\n", "

\n", " \"PINA\n", "

\n", "\n", "Specifically, the tutorial aims to introduce the following topics:\n", "\n", "* Explaining how to build **PINA** Problems,\n", "* Showing how to generate data for `PINN` training\n", "\n", "These are the two main steps needed **before** starting the modelling optimization (choose model and solver, and train). We will show each step in detail, and at the end, we will solve a simple Ordinary Differential Equation (ODE) problem using the `PINN` solver." ] }, { "attachments": {}, "cell_type": "markdown", "id": "cf9c96e3", "metadata": {}, "source": [ "## Build a PINA problem" ] }, { "attachments": {}, "cell_type": "markdown", "id": "8a819659", "metadata": {}, "source": [ "Problem definition in the **PINA** framework is done by building a python `class`, which inherits from one or more problem classes (`SpatialProblem`, `TimeDependentProblem`, `ParametricProblem`, ...) depending on the nature of the problem. Below is an example:\n", "### Simple Ordinary Differential Equation\n", "Consider the following:\n", "\n", "$$\n", "\\begin{equation}\n", "\\begin{cases}\n", "\\frac{d}{dx}u(x) &= u(x) \\quad x\\in(0,1)\\\\\n", "u(x=0) &= 1 \\\\\n", "\\end{cases}\n", "\\end{equation}\n", "$$\n", "\n", "with the analytical solution $u(x) = e^x$. In this case, our ODE depends only on the spatial variable $x\\in(0,1)$ , meaning that our `Problem` class is going to be inherited from the `SpatialProblem` class:\n", "\n", "```python\n", "from pina.problem import SpatialProblem\n", "from pina.geometry import CartesianProblem\n", "\n", "class SimpleODE(SpatialProblem):\n", " \n", " output_variables = ['u']\n", " spatial_domain = CartesianProblem({'x': [0, 1]})\n", "\n", " # other stuff ...\n", "```\n", "\n", "Notice that we define `output_variables` as a list of symbols, indicating the output variables of our equation (in this case only $u$), this is done because in **PINA** the `torch.Tensor`s are labelled, allowing the user maximal flexibility for the manipulation of the tensor. The `spatial_domain` variable indicates where the sample points are going to be sampled in the domain, in this case $x\\in[0,1]$.\n", "\n", "What if our equation is also time-dependent? In this case, our `class` will inherit from both `SpatialProblem` and `TimeDependentProblem`:\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "2373a925", "metadata": {}, "outputs": [], "source": [ "## routine needed to run the notebook on Google Colab\n", "try:\n", " import google.colab\n", " IN_COLAB = True\n", "except:\n", " IN_COLAB = False\n", "if IN_COLAB:\n", " !pip install \"pina-mathlab\"\n", "\n", "from pina.problem import SpatialProblem, TimeDependentProblem\n", "from pina.domain import CartesianDomain\n", "\n", "class TimeSpaceODE(SpatialProblem, TimeDependentProblem):\n", " \n", " output_variables = ['u']\n", " spatial_domain = CartesianDomain({'x': [0, 1]})\n", " temporal_domain = CartesianDomain({'t': [0, 1]})\n", "\n", " # other stuff ..." ] }, { "attachments": {}, "cell_type": "markdown", "id": "ad8566b8", "metadata": {}, "source": [ "where we have included the `temporal_domain` variable, indicating the time domain wanted for the solution.\n", "\n", "In summary, using **PINA**, we can initialize a problem with a class which inherits from different base classes: `SpatialProblem`, `TimeDependentProblem`, `ParametricProblem`, and so on depending on the type of problem we are considering. Here are some examples (more on the official documentation):\n", "* ``SpatialProblem`` $\\rightarrow$ a differential equation with spatial variable(s) ``spatial_domain``\n", "* ``TimeDependentProblem`` $\\rightarrow$ a time-dependent differential equation with temporal variable(s) ``temporal_domain``\n", "* ``ParametricProblem`` $\\rightarrow$ a parametrized differential equation with parametric variable(s) ``parameter_domain``\n", "* ``AbstractProblem`` $\\rightarrow$ any **PINA** problem inherits from here" ] }, { "attachments": {}, "cell_type": "markdown", "id": "592a4c43", "metadata": {}, "source": [ "### Write the problem class\n", "\n", "Once the `Problem` class is initialized, we need to represent the differential equation in **PINA**. In order to do this, we need to load the **PINA** operators from `pina.operators` module. Again, we'll consider Equation (1) and represent it in **PINA**:" ] }, { "cell_type": "code", "execution_count": 2, "id": "f2608e2e", "metadata": {}, "outputs": [], "source": [ "from pina.problem import SpatialProblem\n", "from pina.operator import grad\n", "from pina import Condition\n", "from pina.domain import CartesianDomain\n", "from pina.equation import Equation, FixedValue\n", "\n", "import torch\n", "\n", "\n", "class SimpleODE(SpatialProblem):\n", "\n", " output_variables = ['u']\n", " spatial_domain = CartesianDomain({'x': [0, 1]})\n", "\n", " domains ={\n", " 'x0': CartesianDomain({'x': 0.}),\n", " 'D': CartesianDomain({'x': [0, 1]})\n", " }\n", "\n", " # defining the ode equation\n", " def ode_equation(input_, output_):\n", "\n", " # computing the derivative\n", " u_x = grad(output_, input_, components=['u'], d=['x'])\n", "\n", " # extracting the u input variable\n", " u = output_.extract(['u'])\n", "\n", " # calculate the residual and return it\n", " return u_x - u\n", "\n", " # conditions to hold\n", " conditions = {\n", " 'bound_cond': Condition(domain='x0', equation=FixedValue(1.)),\n", " 'phys_cond': Condition(domain='D', equation=Equation(ode_equation))\n", " }\n", "\n", " # defining the true solution\n", " def truth_solution(self, pts):\n", " return torch.exp(pts.extract(['x']))\n", " \n", "problem = SimpleODE()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "7cf64d01", "metadata": {}, "source": [ "After we define the `Problem` class, we need to write different class methods, where each method is a function returning a residual. These functions are the ones minimized during PINN optimization, given the initial conditions. For example, in the domain $[0,1]$, the ODE equation (`ode_equation`) must be satisfied. We represent this by returning the difference between subtracting the variable `u` from its gradient (the residual), which we hope to minimize to 0. This is done for all conditions. Notice that we do not pass directly a `python` function, but an `Equation` object, which is initialized with the `python` function. This is done so that all the computations and internal checks are done inside **PINA**.\n", "\n", "Once we have defined the function, we need to tell the neural network where these methods are to be applied. To do so, we use the `Condition` class. In the `Condition` class, we pass the location points and the equation we want minimized on those points (other possibilities are allowed, see the documentation for reference).\n", "\n", "Finally, it's possible to define a `truth_solution` function, which can be useful if we want to plot the results and see how the real solution compares to the expected (true) solution. Notice that the `truth_solution` function is a method of the `PINN` class, but it is not mandatory for problem definition.\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "78b30f95", "metadata": {}, "source": [ "## Generate data \n", "\n", "Data for training can come in form of direct numerical simulation results, or points in the domains. In case we perform unsupervised learning, we just need the collocation points for training, i.e. points where we want to evaluate the neural network. Sampling point in **PINA** is very easy, here we show three examples using the `.discretise_domain` method of the `AbstractProblem` class." ] }, { "cell_type": "code", "execution_count": 3, "id": "09ce5c3a", "metadata": {}, "outputs": [], "source": [ "# sampling 20 points in [0, 1] through discretization in all locations\n", "problem.discretise_domain(n=20, mode='grid', domains='all')\n", "\n", "# sampling 20 points in (0, 1) through latin hypercube sampling in D, and 1 point in x0\n", "problem.discretise_domain(n=20, mode='latin', domains=['D'])\n", "problem.discretise_domain(n=1, mode='random', domains=['x0'])\n", "\n", "# sampling 20 points in (0, 1) randomly\n", "problem.discretise_domain(n=20, mode='random')" ] }, { "cell_type": "markdown", "id": "8fbb679f", "metadata": {}, "source": [ "We are going to use latin hypercube points for sampling. We need to sample in all the conditions domains. In our case we sample in `D` and `x0`." ] }, { "cell_type": "code", "execution_count": 4, "id": "329962b6", "metadata": {}, "outputs": [], "source": [ "# sampling for training\n", "problem.discretise_domain(1, 'random', domains=['x0']) # TODO check\n", "problem.discretise_domain(20, 'lh', domains=['D'])" ] }, { "cell_type": "markdown", "id": "ca2ac5c2", "metadata": {}, "source": [ "The points are saved in a python `dict`, and can be accessed by calling the attribute `input_pts` of the problem " ] }, { "cell_type": "code", "execution_count": 5, "id": "d6ed9aaf", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Input points: {'x0': LabelTensor([[0.]]), 'D': LabelTensor([[0.1358],\n", " [0.9747],\n", " [0.9358],\n", " [0.7421],\n", " [0.5146],\n", " [0.4536],\n", " [0.5559],\n", " [0.3117],\n", " [0.6820],\n", " [0.2439],\n", " [0.6064],\n", " [0.7745],\n", " [0.8496],\n", " [0.4244],\n", " [0.8744],\n", " [0.2652],\n", " [0.1833],\n", " [0.0147],\n", " [0.0537],\n", " [0.3789]])}\n", "Input points labels: ['x']\n" ] } ], "source": [ "print('Input points:', problem.discretised_domains)\n", "print('Input points labels:', problem.discretised_domains['D'].labels)" ] }, { "cell_type": "markdown", "id": "669e8534", "metadata": {}, "source": [ "To visualize the sampled points we can use `matplotlib.pyplot`:" ] }, { "cell_type": "code", "execution_count": 6, "id": "3802e22a", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAIa1JREFUeJzt3XtwVPX9//HXJiEbVJLILSEapFAVFJQxmBDUsZVMY3FURhwRKSJNpVag/gii3CRtrYZ6RQVltLXUEQrFKlVkYjF4JwoGqNyrgoDgBiiyQS5JIJ/fHwzrNyWEbLpnk33zfMzsMJx8zjmf84Fln3OyG3zOOScAAABD4pp7AgAAAJFG4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMCchOaeQHOora3Vzp071aZNG/l8vuaeDgAAaATnnPbv36+MjAzFxTV8j+a0DJydO3cqMzOzuacBAACaYPv27Tr33HMbHHNaBk6bNm0kHVug5OTkZp4NAABojMrKSmVmZoZexxtyWgbO8W9LJScnEzgAAMSYxry9hDcZAwAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMiUrgzJw5U126dFFSUpJycnK0fPnyBscvWLBA3bt3V1JSknr16qXFixefdOxdd90ln8+n6dOnR3jWAAAgVnkeOPPnz1dhYaGKioq0cuVKXXrppcrPz9euXbvqHb9s2TINGTJEBQUFWrVqlQYOHKiBAwdq7dq1J4x97bXX9PHHHysjI8PrywAAADHE88B54okndOedd2rEiBG66KKLNGvWLJ1xxhl68cUX6x3/1FNP6dprr9X48ePVo0cPPfjgg7rssss0Y8aMOuN27NihMWPGaM6cOWrVqpXXlwEAAGKIp4FTXV2t8vJy5eXlfX/CuDjl5eWprKys3n3KysrqjJek/Pz8OuNra2s1bNgwjR8/XhdffPEp51FVVaXKyso6DwAAYJengbNnzx4dPXpUaWlpdbanpaUpEAjUu08gEDjl+D/84Q9KSEjQr3/960bNo7i4WCkpKaFHZmZmmFcCAABiScx9iqq8vFxPPfWUZs+eLZ/P16h9Jk6cqGAwGHps377d41kCAIDm5GngtG/fXvHx8aqoqKizvaKiQunp6fXuk56e3uD4Dz74QLt27VLnzp2VkJCghIQEbd26VePGjVOXLl3qPabf71dycnKdBwAAsMvTwElMTFRWVpZKS0tD22pra1VaWqrc3Nx698nNza0zXpKWLFkSGj9s2DB99tlnWr16deiRkZGh8ePH66233vLuYgAAQMxI8PoEhYWFGj58uPr06aPs7GxNnz5dBw4c0IgRIyRJt99+u8455xwVFxdLku655x5dffXVevzxx3Xddddp3rx5+vTTT/X8889Lktq1a6d27drVOUerVq2Unp6uCy+80OvLAQAAMcDzwBk8eLB2796tqVOnKhAIqHfv3iopKQm9kXjbtm2Ki/v+RlK/fv00d+5cTZkyRZMmTdL555+vhQsXqmfPnl5PFQAAGOFzzrnmnkS0VVZWKiUlRcFgkPfjAAAQI8J5/Y65T1EBAACcCoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAc6ISODNnzlSXLl2UlJSknJwcLV++vMHxCxYsUPfu3ZWUlKRevXpp8eLFoa/V1NTo/vvvV69evXTmmWcqIyNDt99+u3bu3On1ZQAAgBjheeDMnz9fhYWFKioq0sqVK3XppZcqPz9fu3btqnf8smXLNGTIEBUUFGjVqlUaOHCgBg4cqLVr10qSDh48qJUrV+qBBx7QypUr9eqrr2rTpk264YYbvL4UAAAQI3zOOeflCXJycnT55ZdrxowZkqTa2lplZmZqzJgxmjBhwgnjBw8erAMHDmjRokWhbX379lXv3r01a9ases+xYsUKZWdna+vWrercufMp51RZWamUlBQFg0ElJyc38coAAEA0hfP67ekdnOrqapWXlysvL+/7E8bFKS8vT2VlZfXuU1ZWVme8JOXn5590vCQFg0H5fD6lpqbW+/WqqipVVlbWeQAAALs8DZw9e/bo6NGjSktLq7M9LS1NgUCg3n0CgUBY4w8fPqz7779fQ4YMOWnNFRcXKyUlJfTIzMxswtUAAIBYEdOfoqqpqdEtt9wi55yee+65k46bOHGigsFg6LF9+/YozhIAAERbgpcHb9++veLj41VRUVFne0VFhdLT0+vdJz09vVHjj8fN1q1btXTp0ga/F+f3++X3+5t4FQAAINZ4egcnMTFRWVlZKi0tDW2rra1VaWmpcnNz690nNze3znhJWrJkSZ3xx+Pm888/19tvv6127dp5cwEAACAmeXoHR5IKCws1fPhw9enTR9nZ2Zo+fboOHDigESNGSJJuv/12nXPOOSouLpYk3XPPPbr66qv1+OOP67rrrtO8efP06aef6vnnn5d0LG5uvvlmrVy5UosWLdLRo0dD789p27atEhMTvb4kAADQwnkeOIMHD9bu3bs1depUBQIB9e7dWyUlJaE3Em/btk1xcd/fSOrXr5/mzp2rKVOmaNKkSTr//PO1cOFC9ezZU5K0Y8cOvf7665Kk3r171znXO++8ox/96EdeXxIAAGjhPP85OC0RPwcHAIDY02J+Dg4AAEBzIHAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgTlQCZ+bMmerSpYuSkpKUk5Oj5cuXNzh+wYIF6t69u5KSktSrVy8tXry4ztedc5o6dao6deqk1q1bKy8vT59//rmXlwAAAGKI54Ezf/58FRYWqqioSCtXrtSll16q/Px87dq1q97xy5Yt05AhQ1RQUKBVq1Zp4MCBGjhwoNauXRsa88gjj+jpp5/WrFmz9Mknn+jMM89Ufn6+Dh8+7PXlnNI3wUNa9uUefRM8VPcLwR3SlveP/fq/iNRxIs2reUX7eht7PivX6/W5m3LM5liDSJ7Ty/m3pOd/S31uRnrfSAtnLrG0xl4c53/kc845L0+Qk5Ojyy+/XDNmzJAk1dbWKjMzU2PGjNGECRNOGD948GAdOHBAixYtCm3r27evevfurVmzZsk5p4yMDI0bN0733nuvJCkYDCotLU2zZ8/Wrbfeeso5VVZWKiUlRcFgUMnJyRG6Umn+im2a+Ooa1TopzicV39RLgy/vLK18SXrjHsnVSr446fqnpMtuD/8EkTpOpHk1r2hfb2PPZ+V6vT53U47ZHGsQyXN6Of+W9Pxvqc/NljDXSM0lltbYi+OcRDiv357ewamurlZ5ebny8vK+P2FcnPLy8lRWVlbvPmVlZXXGS1J+fn5o/JYtWxQIBOqMSUlJUU5OzkmPWVVVpcrKyjqPSPsmeCgUN5JU66RJr65Vxddffv+HLR379Y3/F37ZBndE5jiR5tW8on29jT2flev1+txNOWZzrEEkz+nl/FvS87+lPjdbwlwjNZdYWmMvjhMhngbOnj17dPToUaWlpdXZnpaWpkAgUO8+gUCgwfHHfw3nmMXFxUpJSQk9MjMzm3Q9Ddmy50Aobo476pz2bN3w/R/2ce6otHdzeCfY+2VkjhNpXs0r2tfb2PNZuV6vz92UYzbHGkTynF7OvyU9/1vqczPS+0ZaOHOJpTX24jgRclp8imrixIkKBoOhx/bt2yN+jh+0P1Nxvrrb4n0+tT+vx7HbdP+XL15q2zW8E7TtFpnjRJpX84r29Tb2fFau1+tzN+WYzbEGkTynl/NvSc//lvrcjPS+kRbOXGJpjb04ToR4Gjjt27dXfHy8Kioq6myvqKhQenp6vfukp6c3OP74r+Ec0+/3Kzk5uc4j0jqltFbxTb0U7ztWOfE+nx6+qafSzu127HuQvvhjA33x0vXTpZRzwjtByjmROU6keTWvaF9vY89n5Xq9PndTjtkcaxDJc3o5/5b0/G+pz82WMNdIzSWW1tiL40RIVN5knJ2drWeeeUbSsTcZd+7cWaNHjz7pm4wPHjyoN954I7StX79+uuSSS+q8yfjee+/VuHHjJB1701HHjh2b/U3G0rH34ny156C6tD9DnVJaf/+F4I5jt+nadv3f/rAjdZxI82pe0b7exp7PyvV6fe6mHLM51iCS5/Ry/i3p+d9Sn5uR3jfSwplLLK2xF8epR1iv385j8+bNc36/382ePdutX7/ejRw50qWmprpAIOCcc27YsGFuwoQJofEfffSRS0hIcI899pjbsGGDKyoqcq1atXJr1qwJjZk2bZpLTU11//jHP9xnn33mbrzxRveDH/zAHTp0qFFzCgaDTpILBoORvVgAAOCZcF6/EyKaVvUYPHiwdu/eralTpyoQCKh3794qKSkJvUl427Ztiov7/jtl/fr109y5czVlyhRNmjRJ559/vhYuXKiePXuGxtx33306cOCARo4cqX379unKK69USUmJkpKSvL4cAAAQAzz/FlVL5OW3qAAAgDdazM/BAQAAaA4EDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMzxLHD27t2roUOHKjk5WampqSooKNB3333X4D6HDx/WqFGj1K5dO5111lkaNGiQKioqQl//17/+pSFDhigzM1OtW7dWjx499NRTT3l1CQAAIEZ5FjhDhw7VunXrtGTJEi1atEjvv/++Ro4c2eA+Y8eO1RtvvKEFCxbovffe086dO3XTTTeFvl5eXq6OHTvq5Zdf1rp16zR58mRNnDhRM2bM8OoyAABADPI551ykD7phwwZddNFFWrFihfr06SNJKikp0YABA/T1118rIyPjhH2CwaA6dOiguXPn6uabb5Ykbdy4UT169FBZWZn69u1b77lGjRqlDRs2aOnSpY2eX2VlpVJSUhQMBpWcnNyEKwQAANEWzuu3J3dwysrKlJqaGoobScrLy1NcXJw++eSTevcpLy9XTU2N8vLyQtu6d++uzp07q6ys7KTnCgaDatu2beQmDwAAYl6CFwcNBALq2LFj3RMlJKht27YKBAIn3ScxMVGpqal1tqelpZ10n2XLlmn+/Pl68803G5xPVVWVqqqqQr+vrKxsxFUAAIBYFdYdnAkTJsjn8zX42Lhxo1dzrWPt2rW68cYbVVRUpJ/85CcNji0uLlZKSkrokZmZGZU5AgCA5hHWHZxx48bpjjvuaHBM165dlZ6erl27dtXZfuTIEe3du1fp6en17peenq7q6mrt27evzl2cioqKE/ZZv369+vfvr5EjR2rKlCmnnPfEiRNVWFgY+n1lZSWRAwCAYWEFTocOHdShQ4dTjsvNzdW+fftUXl6urKwsSdLSpUtVW1urnJycevfJyspSq1atVFpaqkGDBkmSNm3apG3btik3Nzc0bt26dbrmmms0fPhwPfTQQ42at9/vl9/vb9RYAAAQ+zz5FJUk/fSnP1VFRYVmzZqlmpoajRgxQn369NHcuXMlSTt27FD//v310ksvKTs7W5L0q1/9SosXL9bs2bOVnJysMWPGSDr2Xhvp2LelrrnmGuXn5+vRRx8NnSs+Pr5R4XUcn6ICACD2hPP67cmbjCVpzpw5Gj16tPr376+4uDgNGjRITz/9dOjrNTU12rRpkw4ePBja9uSTT4bGVlVVKT8/X88++2zo66+88op2796tl19+WS+//HJo+3nnnaevvvrKq0sBAAAxxrM7OC0Zd3AAAIg9zf5zcAAAAJoTgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGAOgQMAAMwhcAAAgDkEDgAAMIfAAQAA5hA4AADAHAIHAACYQ+AAAABzCBwAAGCOZ4Gzd+9eDR06VMnJyUpNTVVBQYG+++67Bvc5fPiwRo0apXbt2umss87SoEGDVFFRUe/Y//znPzr33HPl8/m0b98+D64AAADEKs8CZ+jQoVq3bp2WLFmiRYsW6f3339fIkSMb3Gfs2LF64403tGDBAr333nvauXOnbrrppnrHFhQU6JJLLvFi6gAAIMb5nHMu0gfdsGGDLrroIq1YsUJ9+vSRJJWUlGjAgAH6+uuvlZGRccI+wWBQHTp00Ny5c3XzzTdLkjZu3KgePXqorKxMffv2DY197rnnNH/+fE2dOlX9+/fXt99+q9TU1EbPr7KyUikpKQoGg0pOTv7fLhYAAERFOK/fntzBKSsrU2pqaihuJCkvL09xcXH65JNP6t2nvLxcNTU1ysvLC23r3r27OnfurLKystC29evX63e/+51eeuklxcU1bvpVVVWqrKys8wAAAHZ5EjiBQEAdO3assy0hIUFt27ZVIBA46T6JiYkn3IlJS0sL7VNVVaUhQ4bo0UcfVefOnRs9n+LiYqWkpIQemZmZ4V0QAACIKWEFzoQJE+Tz+Rp8bNy40au5auLEierRo4d+9rOfhb1fMBgMPbZv3+7RDAEAQEuQEM7gcePG6Y477mhwTNeuXZWenq5du3bV2X7kyBHt3btX6enp9e6Xnp6u6upq7du3r85dnIqKitA+S5cu1Zo1a/TKK69Iko6/fah9+/aaPHmyfvvb39Z7bL/fL7/f35hLBAAABoQVOB06dFCHDh1OOS43N1f79u1TeXm5srKyJB2Lk9raWuXk5NS7T1ZWllq1aqXS0lINGjRIkrRp0yZt27ZNubm5kqS///3vOnToUGifFStW6Oc//7k++OADdevWLZxLAQAAhoUVOI3Vo0cPXXvttbrzzjs1a9Ys1dTUaPTo0br11ltDn6DasWOH+vfvr5deeknZ2dlKSUlRQUGBCgsL1bZtWyUnJ2vMmDHKzc0NfYLqvyNmz549ofOF8ykqAABgmyeBI0lz5szR6NGj1b9/f8XFxWnQoEF6+umnQ1+vqanRpk2bdPDgwdC2J598MjS2qqpK+fn5evbZZ72aIgAAMMqTn4PT0vFzcAAAiD3N/nNwAAAAmhOBAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzCFwAACAOQQOAAAwh8ABAADmEDgAAMAcAgcAAJhD4AAAAHMIHAAAYA6BAwAAzElo7gk0B+ecJKmysrKZZwIAABrr+Ov28dfxhpyWgbN//35JUmZmZjPPBAAAhGv//v1KSUlpcIzPNSaDjKmtrdXOnTvVpk0b+Xy+iB67srJSmZmZ2r59u5KTkyN6bJyI9Y4+1jy6WO/oY82jK5z1ds5p//79ysjIUFxcw++yOS3v4MTFxencc8/19BzJyck8MaKI9Y4+1jy6WO/oY82jq7Hrfao7N8fxJmMAAGAOgQMAAMwhcCLM7/erqKhIfr+/uadyWmC9o481jy7WO/pY8+jyar1PyzcZAwAA27iDAwAAzCFwAACAOQQOAAAwh8ABAADmEDhhmjlzprp06aKkpCTl5ORo+fLlDY5fsGCBunfvrqSkJPXq1UuLFy+O0kztCGfNX3jhBV111VU6++yzdfbZZysvL++Uf0Y4Ubh/z4+bN2+efD6fBg4c6O0EjQl3vfft26dRo0apU6dO8vv9uuCCC/i3JQzhrvf06dN14YUXqnXr1srMzNTYsWN1+PDhKM029r3//vu6/vrrlZGRIZ/Pp4ULF55yn3fffVeXXXaZ/H6/fvjDH2r27Nnhn9ih0ebNm+cSExPdiy++6NatW+fuvPNOl5qa6ioqKuod/9FHH7n4+Hj3yCOPuPXr17spU6a4Vq1auTVr1kR55rEr3DW/7bbb3MyZM92qVavchg0b3B133OFSUlLc119/HeWZx65w1/y4LVu2uHPOOcddddVV7sYbb4zOZA0Id72rqqpcnz593IABA9yHH37otmzZ4t599123evXqKM88NoW73nPmzHF+v9/NmTPHbdmyxb311luuU6dObuzYsVGeeexavHixmzx5snv11VedJPfaa681OH7z5s3ujDPOcIWFhW79+vXumWeecfHx8a6kpCSs8xI4YcjOznajRo0K/f7o0aMuIyPDFRcX1zv+lltucdddd12dbTk5Oe6Xv/ylp/O0JNw1/29Hjhxxbdq0cX/5y1+8mqI5TVnzI0eOuH79+rk//vGPbvjw4QROGMJd7+eee8517drVVVdXR2uKpoS73qNGjXLXXHNNnW2FhYXuiiuu8HSeVjUmcO677z538cUX19k2ePBgl5+fH9a5+BZVI1VXV6u8vFx5eXmhbXFxccrLy1NZWVm9+5SVldUZL0n5+fknHY+6mrLm/+3gwYOqqalR27ZtvZqmKU1d89/97nfq2LGjCgoKojFNM5qy3q+//rpyc3M1atQopaWlqWfPnnr44Yd19OjRaE07ZjVlvfv166fy8vLQt7E2b96sxYsXa8CAAVGZ8+koUq+dp+V/ttkUe/bs0dGjR5WWllZne1pamjZu3FjvPoFAoN7xgUDAs3la0pQ1/2/333+/MjIyTniyoH5NWfMPP/xQf/rTn7R69eoozNCWpqz35s2btXTpUg0dOlSLFy/WF198obvvvls1NTUqKiqKxrRjVlPW+7bbbtOePXt05ZVXyjmnI0eO6K677tKkSZOiMeXT0sleOysrK3Xo0CG1bt26UcfhDg7MmjZtmubNm6fXXntNSUlJzT0dk/bv369hw4bphRdeUPv27Zt7OqeF2tpadezYUc8//7yysrI0ePBgTZ48WbNmzWruqZn07rvv6uGHH9azzz6rlStX6tVXX9Wbb76pBx98sLmnhlPgDk4jtW/fXvHx8aqoqKizvaKiQunp6fXuk56eHtZ41NWUNT/uscce07Rp0/T222/rkksu8XKapoS75l9++aW++uorXX/99aFttbW1kqSEhARt2rRJ3bp183bSMawpf8c7deqkVq1aKT4+PrStR48eCgQCqq6uVmJioqdzjmVNWe8HHnhAw4YN0y9+8QtJUq9evXTgwAGNHDlSkydPVlwc9wki7WSvncnJyY2+eyNxB6fREhMTlZWVpdLS0tC22tpalZaWKjc3t959cnNz64yXpCVLlpx0POpqyppL0iOPPKIHH3xQJSUl6tOnTzSmaka4a969e3etWbNGq1evDj1uuOEG/fjHP9bq1auVmZkZzenHnKb8Hb/iiiv0xRdfhEJSkv7973+rU6dOxM0pNGW9Dx48eELEHI9Lx3/l6ImIvXaG9/7n09u8efOc3+93s2fPduvXr3cjR450qampLhAIOOecGzZsmJswYUJo/EcffeQSEhLcY4895jZs2OCKior4mHiYwl3zadOmucTERPfKK6+4b775JvTYv39/c11CzAl3zf8bn6IKT7jrvW3bNtemTRs3evRot2nTJrdo0SLXsWNH9/vf/765LiGmhLveRUVFrk2bNu6vf/2r27x5s/vnP//punXr5m655ZbmuoSYs3//frdq1Sq3atUqJ8k98cQTbtWqVW7r1q3OOecmTJjghg0bFhp//GPi48ePdxs2bHAzZ87kY+LR8Mwzz7jOnTu7xMREl52d7T7++OPQ166++mo3fPjwOuP/9re/uQsuuMAlJia6iy++2L355ptRnnHsC2fNzzvvPCfphEdRUVH0Jx7Dwv17/n8ROOELd72XLVvmcnJynN/vd127dnUPPfSQO3LkSJRnHbvCWe+amhr3m9/8xnXr1s0lJSW5zMxMd/fdd7tvv/02+hOPUe+88069/y4fX+fhw4e7q6+++oR9evfu7RITE13Xrl3dn//857DP63OOe2wAAMAW3oMDAADMIXAAAIA5BA4AADCHwAEAAOYQOAAAwBwCBwAAmEPgAAAAcwgcAABgDoEDAADMIXAAAIA5BA4AADCHwAEAAOb8fwhhEruXTNWRAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "variables=problem.spatial_variables\n", "fig = plt.figure()\n", "proj = \"3d\" if len(variables) == 3 else None\n", "ax = fig.add_subplot(projection=proj)\n", "for location in problem.input_pts:\n", " coords = problem.input_pts[location].extract(variables).T.detach()\n", " ax.plot(coords.flatten(),torch.zeros(coords.flatten().shape),\".\",label=location)\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "22e502dd", "metadata": {}, "source": [ "## Perform a small training" ] }, { "attachments": {}, "cell_type": "markdown", "id": "075f43f5", "metadata": {}, "source": [ "Once we have defined the problem and generated the data we can start the modelling. Here we will choose a `FeedForward` neural network available in `pina.model`, and we will train using the `PINN` solver from `pina.solver`. We highlight that this training is fairly simple, for more advanced stuff consider the tutorials in the ***Physics Informed Neural Networks*** section of ***Tutorials***. For training we use the `Trainer` class from `pina.trainer`. Here we show a very short training and some method for plotting the results. Notice that by default all relevant metrics (e.g. MSE error during training) are going to be tracked using a `lightning` logger, by default `CSVLogger`. If you want to track the metric by yourself without a logger, use `pina.callbacks.MetricTracker`." ] }, { "cell_type": "code", "execution_count": 7, "id": "3bb4dc9b", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "GPU available: False, used: False\n", "TPU available: False, using: 0 TPU cores\n", "HPU available: False, using: 0 HPUs\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1499: 100%|██████████| 1/1 [00:00<00:00, 22.40it/s, v_num=102, val_loss=0.000148, bound_cond_loss=1.44e-6, phys_cond_loss=0.000132, train_loss=0.000134] " ] }, { "name": "stderr", "output_type": "stream", "text": [ "`Trainer.fit` stopped: `max_epochs=1500` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1499: 100%|██████████| 1/1 [00:00<00:00, 18.88it/s, v_num=102, val_loss=0.000148, bound_cond_loss=1.44e-6, phys_cond_loss=0.000132, train_loss=0.000134]\n" ] } ], "source": [ "from pina import Trainer\n", "from pina.solver import PINN\n", "from pina.model import FeedForward\n", "from pina.callback import MetricTracker\n", "\n", "\n", "# build the model\n", "model = FeedForward(\n", " layers=[10, 10],\n", " func=torch.nn.Tanh,\n", " output_dimensions=len(problem.output_variables),\n", " input_dimensions=len(problem.input_variables)\n", ")\n", "\n", "# create the PINN object\n", "pinn = PINN(problem, model)\n", "\n", "# create the trainer\n", "trainer = Trainer(solver=pinn, max_epochs=1500, callbacks=[MetricTracker()], accelerator='cpu', enable_model_summary=False) # we train on CPU and avoid model summary at beginning of training (optional)\n", "\n", "# train\n", "trainer.train()" ] }, { "cell_type": "markdown", "id": "f8b4f496", "metadata": {}, "source": [ "After the training we can inspect trainer logged metrics (by default **PINA** logs mean square error residual loss). The logged metrics can be accessed online using one of the `Lightinig` loggers. The final loss can be accessed by `trainer.logged_metrics`" ] }, { "cell_type": "code", "execution_count": 8, "id": "f5fbf362", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "# inspecting final loss\n", "trainer.logged_metrics\n", "print(type(problem.truth_solution))" ] }, { "cell_type": "markdown", "id": "0963d7d2", "metadata": {}, "source": [ "By using `matplotlib` we can also do some qualitative plots of the solution. " ] }, { "cell_type": "code", "execution_count": 9, "id": "ffbf0d5e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqgAAAKTCAYAAADVBfoyAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAdZlJREFUeJzt3Xd0FFXjxvHvpgdIIUBIAqFJEeldOigIiBQB6b1IFQERwQKICgjoa0OxoID0jhRBQANSREFREUSq9CIlIQGSLff3By/5vZEaSDKb5Pmcs+eY3ZnNs4whD/fO3LEZYwwiIiIiIm7Cw+oAIiIiIiL/SwVVRERERNyKCqqIiIiIuBUVVBERERFxKyqoIiIiIuJWVFBFRERExK2ooIqIiIiIW/GyOkBKcLlcnDhxgoCAAGw2m9VxRERERORfjDFcunSJiIgIPDxuP0aaIQrqiRMniIyMtDqGiIiIiNzB0aNHyZs37223yRAFNSAgALj2gQMDAy1OIyIiIiL/FhMTQ2RkZGJvu50MUVCvT+sHBgaqoIqIiIi4sbs5HVMXSYmIiIiIW1FBFRERERG3ooIqIiIiIm4lQ5yDerecTid2u93qGCIZire3N56enlbHEBGRDCRTFFRjDKdOneLixYtWRxHJkIKDgwkLC9M6xCIikiIyRUG9Xk5DQ0PJkiWLfomKpBBjDJcvX+bMmTMAhIeHW5xIREQyggxfUJ1OZ2I5zZEjh9VxRDIcf39/AM6cOUNoaKim+0VE5L5l+Iukrp9zmiVLFouTiGRc13++dI63iIikhAxfUK/TtL5I6tHPl4iIpKRMU1BFREREJH1QQRURERERt5Ksgjpu3DgqVapEQEAAoaGhNG/enL179952nzp16mCz2W54NG7cOHGbrl273vB6w4YN7+0TSZqrU6cOgwYNsjpGqhs9ejRly5ZNs+83bdo0goOD7/t9oqKisNlsWmZNRETSjWQV1A0bNtC/f39++OEH1q5di91u57HHHiMuLu6W+yxevJiTJ08mPnbt2oWnpydPPfVUku0aNmyYZLs5c+bc2yfKQK4X9/Hjxyd5funSpenqnL9p06bd9B8dFy9exGazERUVddfv1bVrV5o3b56yATOQm/1joVq1apw8eZKgoCBrQomIiCRTspaZWr16dZKvp02bRmhoKDt27KBWrVo33SckJCTJ13PnziVLliw3FFRfX1/CwsKSEydT8PPz480336R3795kz549Tb+33W7H29s7Rd7Ly8uLdevW8d1331G3bt0Uec+0YozB6XRaHeOe+fj46GdLRETSlfs6BzU6Ohq4sYTeztSpU2nbti1Zs2ZN8nxUVBShoaEUK1aMvn37cu7cuVu+R3x8PDExMUkeyWGM4XKCw5KHMSZZWevVq0dYWBjjxo277XabNm2iZs2a+Pv7ExkZycCBA5OMbNtsNpYuXZpkn+DgYKZNmwbA4cOHsdlszJs3j9q1a+Pn58esWbM4d+4c7dq1I0+ePGTJkoVSpUrd0+h21qxZ6d69O8OHD7/tdkePHqV169YEBwcTEhJCs2bNOHz4MHBtin369OksW7Ys8VSQqKgoWrVqxYABAxLfY9CgQdhsNv78808AEhISyJo1K+vWrQOu/f8zcOBAQkND8fPzo0aNGvz000+J+1+fEv/666+pUKECvr6+bNq06YasBw4coFChQgwYMOCmx9UYw+jRo8mXLx++vr5EREQwcODAxNcvXLhA586dyZ49O1myZKFRo0bs27fvln82Nxs9HjRoEHXq1El8fcOGDbz77ruJfz6HDx++6RT/okWLKFGiBL6+vhQoUIC33noryfsWKFCAsWPH0r17dwICAsiXLx+ffPLJLbOJiIikpHteqN/lcjFo0CCqV69OyZIl72qfH3/8kV27djF16tQkzzds2JAWLVpQsGBBDhw4wIsvvkijRo3YunXrTRf9HjduHK+++uq9RueK3clDI9fc8/73Y/eYBmTxufs/dk9PT8aOHUv79u0ZOHAgefPmvWGbAwcO0LBhQ15//XU+//xzzp49y4ABAxgwYABffPFFsvINHz6ct956i3LlyuHn58fVq1epUKECL7zwAoGBgaxcuZJOnTrxwAMPULly5WS99+jRoylcuDALFy6kVatWN7xut9tp0KABVatW5fvvv8fLy4vXX3+dhg0b8ttvvzF06FD27NlDTExM4ucKCQnh999/5+OPP058nw0bNpAzZ06ioqJ48MEH+emnn7Db7VSrVg2AYcOGsWjRIqZPn07+/PmZMGECDRo0YP/+/Un+sTV8+HAmTZpEoUKFyJ49e5JTEX777TcaNGhAjx49eP3112/6eRctWsR//vMf5s6dS4kSJTh16hS//vpr4utdu3Zl3759fPXVVwQGBvLCCy/w+OOPs3v37nsauX733Xf566+/KFmyJGPGjAEgV65ciQX/uh07dtC6dWtGjx5NmzZt2LJlC/369SNHjhx07do1cbu33nqL1157jRdffJGFCxfSt29fateuTbFixZKdTUREJDnueQS1f//+7Nq1i7lz5971PlOnTqVUqVI3FJu2bdvStGlTSpUqRfPmzVmxYgU//fTTLc9NHDFiBNHR0YmPo0eP3uvHSBeefPJJypYty6hRo276+rhx4+jQoQODBg2iSJEiVKtWjffee48ZM2Zw9erVZH2vQYMGJf5jITw8nDx58jB06FDKli1LoUKFeOaZZ2jYsCHz589P9ueIiIjg2Wef5aWXXsLhcNzw+rx583C5XHz22WeUKlWK4sWL88UXX3DkyBGioqLIli0b/v7+iaeDhIWF4ePjQ506ddi9ezdnz57lwoUL7N69m2effTbx/5+oqCgqVapElixZiIuL46OPPmLixIk0atSIhx56iE8//RR/f/8b/uE0ZswY6tevzwMPPJCkuG7ZsoU6deowdOjQW5ZTgCNHjhAWFka9evXIly8flStXplevXgCJxfSzzz6jZs2alClThlmzZnH8+PEbRrrvVlBQED4+PmTJkiXxz+dm/8B7++23efTRR3nllVcoWrQoXbt2ZcCAAUycODHJdo8//jj9+vWjcOHCvPDCC+TMmZPvvvvunrKJiIgkxz2NoA4YMIAVK1awcePGm47o3UxcXBxz585NHNm5nUKFCpEzZ07279/Po48+esPrvr6++Pr6Jjv3df7enuwe0+Ce978f/t73dhvIN998k0ceeYShQ4fe8Nqvv/7Kb7/9xqxZsxKfM8bgcrk4dOgQxYsXv+vvU7FixSRfO51Oxo4dy/z58zl+/DgJCQnEx8ff8525XnjhBT7++GM+//xzWrdufcPn2L9/PwEBAUmev3r1KgcOHLjle5YsWZKQkBA2bNiAj48P5cqV44knnmDy5MnAtRHV69PgBw4cwG63U7169cT9vb29qVy5Mnv27Enyvv/+s4BrpbN+/fq88cYbd1y54KmnnuKdd96hUKFCNGzYkMcff5wmTZrg5eXFnj178PLyokqVKonb58iRg2LFit2QI6Xt2bOHZs2aJXmuevXqvPPOOzidzsRSW7p06cTXbTYbYWFhnDlzJlWziYiIQDILqjGGZ555hiVLlhAVFUXBggXvet8FCxYQHx9Px44d77jtsWPHOHfuHOHh4cmJd9dsNluyptndQa1atWjQoAEjRoxIMg0LEBsbS+/evZOc33hdvnz5gGuf+d/nSd7stpT/Pjd44sSJvPvuu7zzzjuUKlWKrFmzMmjQIBISEu7pcwQHBzNixAheffVVnnjiiRs+R4UKFZIU7ety5cp1y/e02WzUqlWLqKgofH19qVOnDqVLlyY+Pp5du3axZcuWmxb7O/n3n8X1HBEREcyZM4fu3bsTGBh4y/0jIyPZu3cv69atY+3atfTr14+JEyeyYcOGZGcB8PDwuKtjmFL+fZqBzWbD5XKl2vcTERG5LllT/P3792fmzJnMnj2bgIAATp06xalTp7hy5UriNp07d2bEiBE37Dt16lSaN29Ojhw5kjwfGxvL888/zw8//MDhw4dZv349zZo1o3DhwjRoYM0op7saP348y5cvZ+vWrUmeL1++PLt376Zw4cI3PHx8fIBrxerkyZOJ++zbt4/Lly/f8Xtu3ryZZs2a0bFjR8qUKUOhQoX466+/7utzPPPMM3h4ePDuu+/e8Dn27dtHaGjoDZ/j+hJJPj4+N72ivnbt2kRFRREVFUWdOnXw8PCgVq1aTJw4kfj4+MQR0wceeAAfHx82b96cuK/dbuenn37ioYceumN2f39/VqxYgZ+fHw0aNODSpUt33L5Jkya89957REVFsXXrVn7//XeKFy+Ow+Fg27ZtidueO3eOvXv33jLHv48hwM6dO5N8fas/n/9VvHjxJJ8frh3nokWL3vSUABERkbSWrIL60UcfER0dTZ06dQgPD098zJs3L3GbI0eO3PBLdO/evWzatIkePXrc8J6enp789ttvNG3alKJFi9KjRw8qVKjA999/f1/T+BlRqVKl6NChA++9916S51944QW2bNnCgAED2LlzJ/v27WPZsmVJrmx/5JFH+OCDD/jll1/Yvn07ffr0uasLcYoUKcLatWvZsmULe/bsoXfv3pw+ffq+Poefnx+vvvrqDZ+jQ4cO5MyZk2bNmvH9999z6NAhoqKiGDhwIMeOHQOuXV3+22+/sXfvXv7555/EEcTr56H+8ccf1KhRI/G5WbNmUbFixcTR0KxZs9K3b1+ef/55Vq9eze7du+nVqxeXL1++6f+fN5M1a1ZWrlyJl5cXjRo1IjY29qbbTZs2jalTp7Jr1y4OHjzIzJkz8ff3J3/+/BQpUoRmzZrRq1cvNm3axK+//krHjh3JkyfPDdPv1z3yyCNs376dGTNmsG/fPkaNGsWuXbuSbFOgQAG2bdvG4cOH+eeff2464vncc8+xfv16XnvtNf766y+mT5/OBx98cE+jzCIiIqkhWQXVGHPTx/9OOUdFRSUuXXRdsWLFMMZQv379G97T39+fNWvWcObMGRISEjh8+DCffPIJuXPnvqcPlNGNGTPmhtJRunRpNmzYwF9//UXNmjUpV64cI0eOJCIiInGbt956i8jISGrWrEn79u0ZOnToXZ1H+vLLL1O+fHkaNGhAnTp1CAsLS5GF8rt06UKhQoWSPJclSxY2btxIvnz5aNGiBcWLF6dHjx5cvXo1cSq9V69eFCtWjIoVK5IrV67EkcBSpUoRHBxM2bJlyZYtG3CtoDqdzsTzT68bP348LVu2pFOnTpQvX579+/ezZs2aZK0zmy1bNr7++muMMTRu3PimN6sIDg7m008/pXr16pQuXZp169axfPnyxFmEL774ggoVKvDEE09QtWpVjDGsWrXqlv9waNCgAa+88grDhg2jUqVKXLp0ic6dOyfZZujQoXh6evLQQw+RK1cujhw5csP7lC9fnvnz5zN37lxKlizJyJEjGTNmzA2njoiIiFjFZpK7MKcbiomJISgoiOjo6BvOCbx69SqHDh2iYMGC+Pn5WZRQJGPTz5mIiNzJ7frav93XQv0iIiIikk45HeBMvYtt74cKqoiIiEhmtO0j+LQunPjF6iQ3UEEVERERyWzOH8J8+wac+v3aw82ooIqIiIhkJsZgVgzC5rjCFlcJZsbXsjrRDVRQRURERDKTnbOxHYziqvHmFUdPKhQIufM+aUwFVURERCSziD2Da82LAPzH0YrGdapTPPz2V9RbIX3d71NERERE7t3Xw/C4epFdrgJEhbTmq0cKW53opjSCKiIiIpIZ/LkK/liCw3gw3NGL8U+Vw9fLPW9xrYIqqSoqKgqbzcbFixfv630OHz6MzWa74d7zIiIicheuxuBaOQSAz5yNqVLtEcrlu/s7KKY1FVQ3ZbPZbvsYPXq01RFTTdeuXW+4nWpkZCQnT56kZMmS1oQSERFJz9aNxuPSSQ67crMooCNDHytmdaLb0jmoburkyZOJ/z1v3jxGjhzJ3r17E5+7fr95AGMMTqcTL6+Mezg9PT0JCwuzOoaIiEj68/dW2D4VgBGOnrzaqiL+Pu45tX+dRlDdVFhYWOIjKCgIm82W+PWff/5JQEAAX3/9NRUqVMDX15dNmzbddORx0KBB1KlTJ/Frl8vFuHHjKFiwIP7+/pQpU4aFCxfeNsuHH35IkSJF8PPzI3fu3LRq1Srxtfj4eAYOHEhoaCh+fn7UqFGDn3766ZbvNXr0aMqWLZvkuXfeeYcCBQokvj59+nSWLVuWOFocFRV10yn+DRs2ULlyZXx9fQkPD2f48OE4HI7E1+vUqcPAgQMZNmwYISEhhIWFZeiRZxERkRvYr+L66hkA5jrqUKBiI6o9kNPiUHeWcYfcbscYsF+25nt7ZwGbLUXeavjw4UyaNIlChQqRPfvdnUcybtw4Zs6cyZQpUyhSpAgbN26kY8eO5MqVi9q1a9+w/fbt2xk4cCBffvkl1apV4/z583z//feJrw8bNoxFixYxffp08ufPz4QJE2jQoAH79+8nJCT566oNHTqUPXv2EBMTwxdffAFASEgIJ06cSLLd8ePHefzxx+natSszZszgzz//pFevXvj5+SUpodOnT2fIkCFs27aNrVu30rVrV6pXr079+vWTnU1ERCTd+X4SHuf2ccYE83mW7ix6/EGrE92VzFlQ7ZdhbIQ13/vFE+CTNUXeasyYMckqWvHx8YwdO5Z169ZRtWpVAAoVKsSmTZv4+OOPb1pQjxw5QtasWXniiScICAggf/78lCtXDoC4uDg++ugjpk2bRqNGjQD49NNPWbt2LVOnTuX5559P9mfKli0b/v7+xMfH33ZK/8MPPyQyMpIPPvgAm83Ggw8+yIkTJ3jhhRcYOXIkHh7XJgdKly7NqFGjAChSpAgffPAB69evV0EVEZGM7/QfuL7/Dx7ASHtXXmxflQA/b6tT3ZXMWVAziIoVKyZr+/3793P58uUbyllCQkJi6fy3+vXrkz9/fgoVKkTDhg1p2LAhTz75JFmyZOHAgQPY7XaqV6+euL23tzeVK1dmz549yf9AybBnzx6qVq2K7X9Go6tXr05sbCzHjh0jX758wLWC+r/Cw8M5c+ZMqmYTERGxnMuJa9kAPIyDNc6KZCv7JHWKhVqd6q5lzoLqneXaSKZV3zuFZM2adCTWw8MDY0yS5+x2e+J/x8bGArBy5Ury5MmTZDtfX9+bfo+AgAB+/vlnoqKi+Oabbxg5ciSjR4++7Xmmt3OnjCnN2zvpvxRtNhsulyvVvp+IiIhb2PYxHid+Jsb4857v08x+ooTViZIlcxZUmy3FptndSa5cudi1a1eS53bu3JlY0h566CF8fX05cuTITafzb8XLy4t69epRr149Ro0aRXBwMN9++y0NGjTAx8eHzZs3kz9/fuBa2fzpp58YNGjQLTOeOnUKY0zi6Oe/1zb18fHB6XTeNlPx4sVZtGhRkvfZvHkzAQEB5M2b964/m4iISIZz/iCuda/iAYx3tGdQmzoEZUkfU/vXZc6CmkE98sgjTJw4kRkzZlC1alVmzpzJrl27EqfvAwICGDp0KIMHD8blclGjRg2io6PZvHkzgYGBdOnS5Yb3XLFiBQcPHqRWrVpkz56dVatW4XK5KFasGFmzZqVv3748//zzhISEkC9fPiZMmMDly5fp0aPHTTPWqVOHs2fPMmHCBFq1asXq1av5+uuvCQz8//sAFyhQgDVr1rB3715y5MhBUFDQDe/Tr18/3nnnHZ555hkGDBjA3r17GTVqFEOGDEk8/1RERCTTMQbXsoF4OK+yxfkQl0t2oP5Dua1OlWz6TZ6BNGjQgFdeeYVhw4ZRqVIlLl26ROfOnZNs89prr/HKK68wbtw4ihcvTsOGDVm5ciUFCxa86XsGBwezePFiHnnkEYoXL86UKVOYM2cOJUpcmyoYP348LVu2pFOnTpQvX579+/ezZs2aW64qULx4cT788EMmT55MmTJl+PHHHxk6dGiSbXr16kWxYsWoWLEiuXLlYvPmzTe8T548eVi1ahU//vgjZcqUoU+fPvTo0YOXX375Xv7oREREMoYd0/D4+3uuGB/e9OnHqKalrE50T2zm3ycEpkMxMTEEBQURHR2dZCQO4OrVqxw6dIiCBQvi5+dnUUKRjE0/ZyIibiD6GM4PquBpj2WMvRMV277E46XCrU6V6HZ97d80gioiIiKS3hmDa/kgPO2x7HAV4XTxzm5VTpNL56CKiIiIpHe/zcNj/1rijRdvePbjk+ZlrE50XzSCKiIiIpKeXTqNc9UwAN51tKBLswbkzHbz5SPTCxVUERERkXTMtXIonvHR7HIV4ECRHjQtY9HdMlNQppnizwDXgom4Lf18iYhY5I+lePz5FXbjyase/ZjcomySuyymVxl+BPX6IvWXL1+2OIlIxnX95+vfd+4SEZFUdPk8jhXPAfCRswltmjQmNDBjrKSS4UdQPT09CQ4OTrz/epYsWTLEvyxE3IExhsuXL3PmzBmCg4Px9PS0OpKISKbhWj0cryv/8JcrD78V6sUz5fPcead0IsMXVICwsDCAxJIqIikrODg48edMRETSwF/f4PHbPJzGxihbP95qUSFDDcBlioJqs9kIDw8nNDQUu91udRyRDMXb21sjpyIiaelqNI6vBuIFfO5sRNOmTYkI9rc6VYrKFAX1Ok9PT/0iFRERkXTNfDMSr9iTHHblZku+3nxeKdLqSCkuUxVUERERkXTt4AZsP08DYCR9eKNV5Qw1tX9dhr+KX0RERCRDSIjDsXQAAF866lGvUQsiQ7JYHCp1qKCKiIiIpANm/Wt4xRzhuMnBujx96Vglv9WRUo0KqoiIiIi7O7INtk0BYJSrF68+VRUPj4w3tX+dCqqIiIiIO7Nfxb6kHzYMC521ePixNhTImdXqVKlKBVVERETEjZmoN/G+sJ8zJpgVYQPoVr2g1ZFSnQqqiIiIiLs6vgOz5V0Axri6M6pNDTwz8NT+dSqoIiIiIu7IfhX7oj54GCdfOatSoWFnCmbwqf3rVFBFRERE3JCJGo/3+b84a4JYGjGYLlULWB0pzaigioiIiLibY9sxm69N7b/q6sXo1jUz9FX7/6aCKiIiIuJO7FdJWNQbD1wscVan8uOdyJcjYy7IfysqqCIiIiJuxHz7Bj7/vWp/ZZ5BGXpB/ltRQRURERFxF0d/xGz9AIAxphejWtfIVFP716mgioiIiLgD+5XEqf1FzhpUbdyJyJDMNbV/nQqqiIiIiBtwrX8dn4sHOW2CWRM5mPaV81kdyTIqqCIiIiJWO/IDth8mA/AqfRjVujo2W+ab2r9OBVVERETESgmXSVjUBxuGBY5a1GrcgTzB/lanspQKqoiIiIiFXOvH4BN9iJMmhPUFBtOmUqTVkSyngioiIiJilb+3YNs2BYAxtt6Meqpqpp7av04FVURERMQKCXGJU/vzHHV45IkOhAdl7qn961RQRURERCzgWvcqPjF/c8KEsLHQIFpVyGt1JLehgioiIiKS1g5vwuPHjwEYY+vDyFaa2v9fKqgiIiIiaSk+loTFfQGY7ahLg2YdyB3oZ3Eo96KCKiIiIpKGnGtH4RNzhGMmJ1sLD6Z52TxWR3I7KqgiIiIiaeXQRjy3fwbA6x59eKVlFU3t34QKqoiIiEhaiL9E/KJrU/uzHI/yxJMdCA3Q1P7NqKCKiIiIpAHH1y/iG3uMYyYnvxR/jidKR1gdyW15WR1AREREJMP7aw1eO2cA8LrXM4x/spLFgdybRlBFREREUtPl8yQs6Q/AZ45GtHmqPcFZfCwO5d5UUEVERERSkX35YHyunGWfKw9/l32Oug+GWh3J7WmKX0RERCS1/L4Q7z1LcRgP3vQfxDtNylmdKF3QCKqIiIhIaog5gX35EADedz5JrzYtyearscG7oYIqIiIiktKMwb6kP94J0fzmKsiVyoOoUiiH1anSDdV4ERERkZS24wu8D31LvPHmnYChfNiohNWJ0hWNoIqIiIikpPMHcXz9EgATnG0Z1O4J/Lw9LQ6VvqigioiIiKQUl5OEhb3xcl5mq/MhstUaQOm8wVanSnc0xS8iIiKSQsyW9/E58SOXjD+f5RjKlEeLWh0pXdIIqoiIiEhKOP0Hrm9fB2CsqzPD2z2Gt6eq1r3Qn5qIiIjI/XIkkLCgF54uO2ud5SlUrzdFcgdYnSrdUkEVERERuU8majw+//zBORPA/LDn6V6zkNWR0rVkFdRx48ZRqVIlAgICCA0NpXnz5uzdu/e2+0ybNg2bzZbk4efnl2QbYwwjR44kPDwcf39/6tWrx759+5L/aURERETS2tGfMJv+A8AY04tX2tbB08Nmcaj0LVkFdcOGDfTv358ffviBtWvXYrfbeeyxx4iLi7vtfoGBgZw8eTLx8ffffyd5fcKECbz33ntMmTKFbdu2kTVrVho0aMDVq1eT/4lERERE0kpCHAkLe+GBi8XOGlRp3I18ObJYnSrdS9ZV/KtXr07y9bRp0wgNDWXHjh3UqlXrlvvZbDbCwsJu+poxhnfeeYeXX36ZZs2aATBjxgxy587N0qVLadu27Q37xMfHEx8fn/h1TExMcj6GiIiISIpwrh2FT/QhTpoQ1hV4jsmVI62OlCHc1zmo0dHRAISEhNx2u9jYWPLnz09kZCTNmjXjjz/+SHzt0KFDnDp1inr16iU+FxQURJUqVdi6detN32/cuHEEBQUlPiIj9T+DiIiIpLED3+L506cAjLb1Y9RT1bDZNLWfEu65oLpcLgYNGkT16tUpWbLkLbcrVqwYn3/+OcuWLWPmzJm4XC6qVavGsWPHADh16hQAuXPnTrJf7ty5E1/7txEjRhAdHZ34OHr06L1+DBEREZHku3KRhMX9AJjuqM8TT3Ygd6DfHXaSu3XPC/X379+fXbt2sWnTpttuV7VqVapWrZr4dbVq1ShevDgff/wxr7322j19b19fX3x9fe9pXxEREZH75Vj+HD5xJznkys0fDw1hQpkIqyNlKPc0gjpgwABWrFjBd999R968eZO1r7e3N+XKlWP//v0Aieemnj59Osl2p0+fvuV5qyIiIiKW+X0hXrsX4jAejPUdxEtPVrI6UYaTrIJqjGHAgAEsWbKEb7/9loIFCyb7GzqdTn7//XfCw8MBKFiwIGFhYaxfvz5xm5iYGLZt25Zk5FVERETEchePYv9qEADvO56kW9vWBPl7W5spA0rWFH///v2ZPXs2y5YtIyAgIPEc0aCgIPz9/QHo3LkzefLkYdy4cQCMGTOGhx9+mMKFC3Px4kUmTpzI33//Tc+ePYFrV/gPGjSI119/nSJFilCwYEFeeeUVIiIiaN68eQp+VBEREZH74HKRsKg3PvZL/OIqzOWHB1HtgZxWp8qQklVQP/roIwDq1KmT5PkvvviCrl27AnDkyBE8PP5/YPbChQv06tWLU6dOkT17dipUqMCWLVt46KGHErcZNmwYcXFxPP3001y8eJEaNWqwevXqGxb0FxEREbGK2foBPkc3E2d8eT/oeT5sWMLqSBmWzRhjrA5xv2JiYggKCiI6OprAwECr44iIiEhGc+p3nB/XwdM4eMnxNB37v0LxcHWO5EhOX7uvdVBFREREMjz7VRLm98DTOPjGWYF89XqrnKYyFVQRERGR23CtG43P+b2cNUHMD3+enrUesDpShqeCKiIiInIrB77FY9u1a3BG0ZfR7Wrj6aG7RaU2FVQRERGRm7l8HvuiPgDMcNSnXrNO5M2exeJQmYMKqoiIiMi/GYNz2UC8L5/mgCucn4sN4clyeaxOlWnc861ORURERDKsX+fguXc5duPJqz6DebdFRWw2Te2nFY2gioiIiPyvC4dxrBgKwH8crejRugXZs/pYHCpzUUEVERERuc7lxLGwF16OOH50FeNypf7ULprL6lSZjqb4RURERP7LbPoPXsd/5JLx592A5/js8ZJWR8qUNIIqIiIiAnD8Z8x34wB41dmV4e0a4u/jaXGozEkFVURERCQhDvuCnngYByucVShQtzul8gZZnSrT0hS/iIiIZHqub17B++IBTpnsLAh7jql1ClsdKVPTCKqIiIhkbn99g8f2qQC8Qn9ea1sTL09VJCvpT19EREQyr9gz2Bf3BeBzR0MaNWtHvhy6W5TVVFBFREQkc3K5cCzug/fVf9jjiuT34oN1tyg3oXNQRUREJHPaNgWvg+u5arx5w28ok1tU0N2i3IRGUEVERCTzOfkbzrUjAXjD0ZGB7ZoS5O9tcSi5TgVVREREMpeEOOzzu+PpsrPWWYGgmn2oXDDE6lTyPzTFLyIiIpmKa/WLeF/YxymTnemhQ/miflGrI8m/aARVREREMo/dX+Hx8zRcxsaLpj+vt6uNt5aUcjs6IiIiIpI5RB/HsXQAAB87n6BR07YUyJnV4lByMyqoIiIikvG5nDgXPY1XQjS/ugqx58FnaFUhr9Wp5BZ0DqqIiIhkfJvfwfPIJuKML6/7DuGzFuW1pJQb0wiqiIiIZGzHtuP69g0ARjm6MqTt4wRl0ZJS7kwFVURERDKu+Es4FnTHwzhZ7nyYXDW6UfWBHFankjvQFL+IiIhkWGblULyi/+aYycmsnIOZUb+Y1ZHkLmgEVURERDKm3xZg+20uTmPjBdczjG1fAx8vVZ/0QEdJREREMp4Lh3EuHwzA+84nadKkBYVyZbM4lNwtFVQRERHJWJwOnAt74mm/xHZXUf4q2pc2lSKtTiXJoHNQRUREJGPZOAHP4z8RY/x53XcwX7QsqyWl0hmNoIqIiEjG8fcWzIaJALzk6MHzbR4je1Yfi0NJcqmgioiISMZw+TyOBT2w4WKhsxZ5a3aieuGcVqeSe6ApfhEREUn/jMG1bABesSc45MrNotwDmVG/qNWp5B6poIqIiEj699NneOxdSYLx5AXbYN5qXx1vT00Up1c6ciIiIpK+nfwN1+oXARjnaE+nFs2IDMlicSi5HyqoIiIikn7Fx+Kc3xUPVwJrneW5XLYXTcpEWJ1K7pOm+EVERCTdMquew/PCAU6YED4MGsysZiWsjiQpQCOoIiIikj7tnIPt12u3Mh3qfIY32tchi4/G3jICFVQRERFJf/7Zj3PFEADecbSkweMteCgi0OJQklJUUEVERCR9sV/FtaArno7LbHE+xJ9FnqZz1fxWp5IUpHFwERERSV/WjsTj9O+cMwGM9RvCl0+V061MMxiNoIqIiEj6sWcF/PgxAM85+vJS20d0K9MMSAVVRERE0oeLR3Eu7Q/Ax47GlK7diqoP5LA4lKQGTfGLiIiI+3M6cC3qgWf8RXa6HuC7iN7MfLSI1akklaigioiIiPuLGofH0W3EGH9GeAzi03aV8NKtTDMsHVkRERFxbwejMN+/BcAIey8GtqxP3uy6lWlGpoIqIiIi7iv2DK5FvbBhmO14hKBKrWlUKtzqVJLKNMUvIiIi7snlwizpg0fcGfa68jInpC8LnnjI6lSSBlRQRURExD1teQ/bgfVcMT4McT3L2+2r4uftaXUqSQOa4hcRERH3c/RHzPrXABjt6ELHpg0pFhZgcShJKyqoIiIi4l4un8c1vys242C582GulGhP20qRVqeSNKQpfhEREXEfLhdmSW88Lh3noCuMKYHPMq9lad3KNJPRCKqIiIi4jy3vYtv3DVeNN4Ncg5nQoTrZfDWeltmooIqIiIh7+HtrkvNOn3qiESUigiwOJVZQQRURERHrxf2Da0FXbMbJEmd1LhVvT8cq+axOJRbRmLmIiIhYy+XCLH4aj9hT7HdFMCXbABa00nmnmZlGUEVERMRam95OXO90oHMQkzpUJ9DP2+pUYiEVVBEREbHO4U2Y794AYKSjK089/hil8uq808xOBVVERESsEXsG18Lu2IyLhc5axDzYhq7VClidStyAzkEVERGRtOdyYhb3wiP2NH+58jAla18WtSyj804F0AiqiIiIWGHjJGwHo7hsfBnoGMTE9lUJyqLzTuUaFVQRERFJWwc3YKLGAfCSvTutGtWjXL7sFocSd6KCKiIiImnn0mlci3piwzDXUYdLxVrSo0ZBq1OJm9E5qCIiIpI2XE7Moh54xJ1hjyuSKVl6s/QpnXcqN9IIqoiIiKSNqPHYDn9PrPHjGccg3urwMMFZfKxOJW5IBVVERERS3/71mI0TAXjR3pNWDepSIX+IxaHEXamgioiISOqKOYlr8dPYMMxyPMqlIs14umYhq1OJG9M5qCIiIpJ6nHbMgq54XP6H3a78fJb1aRa3LouHh847lVvTCKqIiIiknrWjsB39gRjjzzPOQfyn48Nkz6rzTuX2VFBFREQkdfyxFH6YDMBQex86N36EspHBlkaS9CFZBXXcuHFUqlSJgIAAQkNDad68OXv37r3tPp9++ik1a9Yke/bsZM+enXr16vHjjz8m2aZr167YbLYkj4YNGyb/04iIiIh7+GcfZmk/AKY4muBdsimdq+a3OJSkF8kqqBs2bKB///788MMPrF27FrvdzmOPPUZcXNwt94mKiqJdu3Z89913bN26lcjISB577DGOHz+eZLuGDRty8uTJxMecOXPu7ROJiIiItRLiMPM6YbPH8YOrOAuDu/Fmy9Ja71Tums0YY+5157NnzxIaGsqGDRuoVavWXe3jdDrJnj07H3zwAZ07dwaujaBevHiRpUuX3lOOmJgYgoKCiI6OJjAw8J7eQ0RERFKAMbD4afh9PmdMMC1c45na/wmKhQVYnUwslpy+dl/noEZHRwMQEnL365hdvnwZu91+wz5RUVGEhoZSrFgx+vbty7lz5275HvHx8cTExCR5iIiIiBvYPhV+n4/DeNA/YSCDm9dUOZVku+eC6nK5GDRoENWrV6dkyZJ3vd8LL7xAREQE9erVS3yuYcOGzJgxg/Xr1/Pmm2+yYcMGGjVqhNPpvOl7jBs3jqCgoMRHZGTkvX4MERERSSnHdmC+Hg7Am462FK5Un5YV8locStKje57i79u3L19//TWbNm0ib967+59v/PjxTJgwgaioKEqXLn3L7Q4ePMgDDzzAunXrePTRR294PT4+nvj4+MSvY2JiiIyM1BS/iIiIVeLOYT6phS36GF87K/F+jpEs7l8dP29Pq5OJm0j1Kf4BAwawYsUKvvvuu7sup5MmTWL8+PF88803ty2nAIUKFSJnzpzs37//pq/7+voSGBiY5CEiIiIWcTlhcS9s0cc45MrNGM/+fNSpgsqp3LNk3UnKGMMzzzzDkiVLiIqKomDBgne134QJE3jjjTdYs2YNFStWvOP2x44d49y5c4SHhycnnoiIiFhh40Q4sJ4rxoe+9sGM7liV/DmyWp1K0rFkjaD279+fmTNnMnv2bAICAjh16hSnTp3iypUridt07tyZESNGJH795ptv8sorr/D5559ToECBxH1iY2MBiI2N5fnnn+eHH37g8OHDrF+/nmbNmlG4cGEaNGiQQh9TREREUsW+dZio8QC8ZO9OzRq1aVAizOJQkt4lq6B+9NFHREdHU6dOHcLDwxMf8+bNS9zmyJEjnDx5Msk+CQkJtGrVKsk+kyZNAsDT05PffvuNpk2bUrRoUXr06EGFChX4/vvv8fX1TaGPKSIiIinu4hHM4p7YMMxyPMqRyGYMa/ig1akkA7ivdVDdhdZBFRERSWOOePi8IZz4md9cBent9QZLnn2UsCA/q5OJm0qzdVBFREQkk1rzIpz4mYsmK/3sg5jYrorKqaQYFVQRERFJnt/mw0+fATDI3o/W9apTo0hOi0NJRqKCKiIiInfvzB7M8mcBeNfxJK7CjzGgbmGLQ0lGk6xlpkRERCQTuxqNmdsBm/0yG52lWJi1A8valMXDw2Z1MslgVFBFRETkzlwuWNIH2/kDHDc5eN48w6edKhGS1cfqZJIBaYpfRERE7uz7t2DvKuKNN30SBjOkWVVK5w22OpVkUCqoIiIicnv71mK+ewOAlx3dKFmpNm0q5bM4lGRkmuIXERGRWzt/ELOoR+Ji/HvDmzG/SQmrU0kGp4IqIiIiN5cQB3M7Yrsazc+uwrzr3YMlHSvg5+1pdTLJ4DTFLyIiIjcyBpY/C2f+4KwJpL99EG+3r0KeYH+rk0kmoIIqIiIiN9o2BX5fgMN40D/hWTo+VlWL8UuaUUEVERGRpA5vwqx5CYA3HB0ILl6HfnUesDiUZCY6B1VERET+X/RxzIKu2IyTpc5qbAhuydLWZbDZtBi/pB0VVBEREbnGEQ/zO2OLO8tuV35epQ/zOlck0M/b6mSSyWiKX0RERK75+gU4vp1ok4Xe9kGMaVWJorkDrE4lmZAKqoiIiMDPM2DHF7iMjWftA3is+sM0KRNhdSrJpFRQRUREMrvjOzArhwLwtqMVl/M9wvBGD1ocSjIznYMqIiKSmcWexczrjM0Zz1pnBRb4t2Z5h3J4e2oMS6yjgioiIpJZOR2wsBu2mGMccIXzvLMfn3WsSGiAn9XJJJPTP49EREQyq/Wj4fD3xBlfetsHM6hxBSoWCLE6lYgKqoiISKb02wLY8j4AQ+19KF2uCl2qFbA2k8h/aYpfREQkszmxE/PVAGzAh46mHAt/jAVPltJi/OI2VFBFREQyk9izmLkdsDmu8p2zDF/4dGBZpwr4eXtanUwkkab4RUREMgtHwrU7Rf33oqghzmf4oGMlIoL9rU4mkoQKqoiISGaxejgc2cIl48/T9iEMeqISVQrlsDqVyA1UUEVERDKDHdNg+1Rc2Bhk70f58lXoXDW/1alEbkrnoIqIiGR0R37ArByKDXjb3op/8jzK5OYldVGUuC0VVBERkYws+jhmXidsLjurnJWZ69eG5R3L66IocWsqqCIiIhmV/SrM64gt7gx7XJEMd/ZlaqcKhAfpoihxbzoHVUREJCMyBlYMghM/c8Fk42n7EIY1rUAl3SlK0gEVVBERkYzoh4/g1zk48KC/fSDVK1akQ5V8VqcSuSsqqCIiIhnNwSjMNy8DMNbegSt5a/BqsxK6KErSDZ2DKiIikpGcP4RZ0BWbcbLQWYvl/s1Y0bECvl66KErSDxVUERGRjCI+FuZ2wHblAjtdhRjl6sGMThXIHehndTKRZNEUv4iISEZgDCzrB2f+4KwJok/CYF5uVp4K+XVRlKQ/KqgiIiIZwfeTYPcy7HjSO2Ewj1QpR7vKuihK0icVVBERkfTuz1WYb98A4BV7NzzyVWF0kxIWhxK5dzoHVUREJD07/QdmUU9sGGY46rMx2+N81akCPl4ag5L0SwVVREQkvYr7B+a0xWaPY7OzBBNtXZnbpSI5s/lanUzkvuifVyIiIumRIwHmdYKLRzjsyk0/+7NMaF2BEhFBVicTuW8qqCIiIumNMbByCBzZwiXjTw/7ULo+Wo5GpcKtTiaSIlRQRURE0pttU+CXL3HiwTP2ZyhaogLPPlrE6lQiKUbnoIqIiKQn+9dh1ryIDRhrb8/p3LVY1LoMHh66jalkHCqoIiIi6cXZvzALumEzLuY56rDUtxnLOlcgi49+nUvGov+jRURE0oPL52FOG2zxMfzoKsYY053pnSuSN3sWq5OJpDidgyoiIuLunHZY0BXOH+SYyUnfhEGMal6eigV0G1PJmFRQRURE3N2aF+HQBuKMLz0ThtK0ehlaV4q0OpVIqlFBFRERcWc/TYUfP8GFjUH2/uQqXJ6XHi9udSqRVKVzUEVERNzVoe8xXw/DBkyyt2Z/SG2WtiuPl6fGlyRjU0EVERFxR+cPYeZ3wuZysNRZjS+9W7Ckc0WCsnhbnUwk1amgioiIuJurMTCnLbYrF9jpeoARjqf5sGN5CodmszqZSJrQHIGIiIg7cTlhUU84+ycnTQi9EoYwpFEZ6hYLtTqZSJpRQRUREXEna0fCvjVcMT70ShhCrfKl6FmzoNWpRNKUCqqIiIi72P4FbP0AgKH2PmTJX5GxLUpis+k2ppK56BxUERERd3DgO8zK5/57xf5T/B78CEs7VcDXy9PqZCJpTgVVRETEamf3YuZ3xmacLHLWYLp3K5Z0rUhIVh+rk4lYQgVVRETESnHnYHZrbPEx/OgqxsvOp/mkcwUKhwZYnUzEMjoHVURExCqOeJjXAS4c5m9XKL0TBvNS07LULJLL6mQillJBFRERsYIx8NUzcGQrMSYL3e3P07x6aTo+nN/qZCKWU0EVERGxwsZJ8Ns8HHjQ1/4s+YqW5eXGD1mdSsQt6BxUERGRtLZrMXz3OgAj7d34J1c1FrYrh6eHlpMSARVUERGRtHVsO2ZpX2zAZ45GfOPfiCVdKhLg5211MhG3oYIqIiKSVi4egTltsTmustZZnkl0YlanikSGZLE6mYhb0TmoIiIiaeFqDMxuA3Fn2e3Kz7P2AbzZqiwV8me3OpmI21FBFRERSW1OByzsDmd2c9oE0yNhKL0eLUWzsnmsTibillRQRUREUtuaF2H/Wq7gQ8+EoVQsU4pB9YpYnUrEbamgioiIpKYfP4UfPwZgUEI/PPOWZ2Kr0thsumJf5FZ0kZSIiEhq2bcO8/UwbMCb9rbsCqzNks4V8PP2tDqZiFtTQRUREUkNp37HLOiCzbiY76jNl15PsqBLRUID/KxOJuL2VFBFRERSWvRxmNUaW0IsW5wPMdLVk086V6B4eKDVyUTShWSdgzpu3DgqVapEQEAAoaGhNG/enL17995xvwULFvDggw/i5+dHqVKlWLVqVZLXjTGMHDmS8PBw/P39qVevHvv27UveJxEREXEHV2Ngdmu4dIK/XHnoYx/MqOZlqVU0l9XJRNKNZBXUDRs20L9/f3744QfWrl2L3W7nscceIy4u7pb7bNmyhXbt2tGjRw9++eUXmjdvTvPmzdm1a1fiNhMmTOC9995jypQpbNu2jaxZs9KgQQOuXr16759MREQkrTntsKArnN7FWRNEd/swOtYpTbvK+axOJpKu2Iwx5l53Pnv2LKGhoWzYsIFatWrddJs2bdoQFxfHihUrEp97+OGHKVu2LFOmTMEYQ0REBM899xxDhw4FIDo6mty5czNt2jTatm17xxwxMTEEBQURHR1NYKCmT0RExALGwPJn4efpXMaXNvGvkL9Udd5rWw4PD12xL5KcvnZfy0xFR0cDEBIScstttm7dSr169ZI816BBA7Zu3QrAoUOHOHXqVJJtgoKCqFKlSuI2/xYfH09MTEySh4iIiKU2vwM/T8eJBwMTBuCbrwKTniqjcipyD+65oLpcLgYNGkT16tUpWbLkLbc7deoUuXPnTvJc7ty5OXXqVOLr15+71Tb/Nm7cOIKCghIfkZGR9/oxRERE7t+uRbBuNACv2juxP3tNPulcUctJidyjey6o/fv3Z9euXcydOzcl89yVESNGEB0dnfg4evRommcQEREB4O+tmCV9APjM0Yjlvk/wRbfKhGT1sTiYSPp1T8tMDRgwgBUrVrBx40by5s17223DwsI4ffp0kudOnz5NWFhY4uvXnwsPD0+yTdmyZW/6nr6+vvj6+t5LdBERkZTzz36Y2w6bM4HVzkpMohMzO1ekYM6sVicTSdeSNYJqjGHAgAEsWbKEb7/9loIFC95xn6pVq7J+/fokz61du5aqVasCULBgQcLCwpJsExMTw7Zt2xK3ERERcTtx/8CsVnDlAjtdDzDI3o+JT5WjYoFbX5chIncnWSOo/fv3Z/bs2SxbtoyAgIDEc0SDgoLw9/cHoHPnzuTJk4dx48YB8Oyzz1K7dm3eeustGjduzNy5c9m+fTuffPIJADabjUGDBvH6669TpEgRChYsyCuvvEJERATNmzdPwY8qIiKSQuxXYE47uHCIIyYXPROG8mzDMjQpE2F1MpEMIVkF9aOPPgKgTp06SZ7/4osv6Nq1KwBHjhzBw+P/B2arVavG7Nmzefnll3nxxRcpUqQIS5cuTXJh1bBhw4iLi+Ppp5/m4sWL1KhRg9WrV+Pnp9vBiYiIm3G5YElvOPYjF01WuiUMo37lkvSpXcjqZCIZxn2tg+outA6qiIikmW9egS3vkYAXHeNH4FekFlO7VMTb875WbhTJ8JLT1+7pIikREZFM6afPYMt7AAxN6E1M7spMbV9O5VQkhamgioiI3I2/1mBWPY8NmGhvzbZsj7CkayUC/LytTiaS4aigioiI3MnxnzELumEzLuY56jDNswXzulQiItjf6mQiGZIKqoiIyO2cPwSzW2Ozx7HRWYpRrh580rkiJfMEWZ1MJMPSSTMiIiK3EvcPzGwJcWfZ5SpAX/sgXm9ZnlpFc1mdTCRDU0EVERG5mYTLMLsNnD/AUZOLbgnD6PtYGVpVuP0dFEXk/qmgioiI/JvTAQu7w/HtXDDZ6JLwAvWrlKZ/3cJWJxPJFFRQRURE/pcxsOo5+OtrruJNj4ShFHqwLGOalsBms1mdTiRTUEEVERH5XxsnwY5pOPFgYMIAXHkr83678nhprVORNKOr+EVERK77ZSZ89zoAo+xd+Ct7bRZ1qYi/j6fFwUQyFxVUERERgH3rMF8NxAZMdjRltX9jFnWvTI5svlYnE8l0VFBFRERO/IKZ3xmbcbLIWYPJHu2Z27US+XNktTqZSKakE2pERCRzO38IZj2VuBD/S87eTO5QgdJ5g61OJpJpqaCKiEjmFXcucSH+P1z56WsfxJgny1G3WKjVyUQyNRVUERHJnBIuw+zWcP4Ax0xOuiYMo1e90rSuFGl1MpFMTwVVREQyH6cDFvVIshD/o5VK8+yjRaxOJiKooIqISGZjDKwaCntXcRVveiY8R76iZXm9eUktxC/iJlRQRUQkc9kwAXZ8gQsbzyYMwJGnMh+010L8Iu5Ey0yJiEjm8dNUiBoLwEh7V/aF1GFB10pk9dWvQxF3op9IERHJHHYvw6x8DhvwrqMFa7M+waIeWohfxB2poIqISMZ3aCNmUU9sGGY5HmWqVxsWdK9C3uxZrE4mIjehgioiIhnbyV8xc9pjcyawylmZ1+nBjK6VKRYWYHUyEbkFnREuIiIZ1/mDMLMVtoRLbHU+xHPO/rzXviKVCoRYnUxEbkMFVUREMqZLp+HLJyHuDH+48vO0fQivPlmB+g/ltjqZiNyBCqqIiGQ8V2NgVku4cJi/XaF0TXiBvg3L6S5RIumECqqIiGQs9qswtz2c+p1/TCCd7CNoUr0cfWs/YHUyEblLKqgiIpJxuJywuBcc/p5Y40+XhOGUL1OOlxsX112iRNIRFVQREckYrt/CdM9XJOBFL/sQchSpxIRWZfDwUDkVSU+0zJSIiGQMUeNh++f/vYVpfy7nqc7sDuXx8dJYjEh6o4IqIiLp30+fwYbxALxi78beHI+wULcwFUm39JMrIiLp2x9LMCuHYgPecbRgfdYnWNi9MiFZfaxOJiL3SAVVRETSr4MbMIufxoZhpuNRPvdqw4LulXULU5F0TgVVRETSp2M7MHPaJd7CdJytJzO66RamIhmBzhwXEZH058wezKyW2OxxfO8syfOuAXzYqRIV8usWpiIZgQqqiIikLxcOw5dPYrtygV9chenrGMLEtpWpXTSX1clEJIWooIqISPpx6RTMaA6XTvKnK5KuCcN4pUUlHi8VbnUyEUlBKqgiIpI+XLkAX7aAC4f42xVKp4Th9H+8Im0q5bM6mYikMBVUERFxfwlxMKs1nPmD0yaYjvYRtK5bkadrPWB1MhFJBSqoIiLi3hzxMLcDHPuRiyYrnRJGUKdKZYY+VszqZCKSSlRQRUTEfbmcsLgXHPyOOONLt4RhFC9ThVeblsBms1mdTkRSiQqqiIi4J2Ng+bOwexkJxoun7UPIXqw6k54qg4eHyqlIRqaF+kVExP0YA2tfgV++xIkHz9gHYM9fm6kdyuPtqbEVkYxOBVVERNzPprdhy/sADLf35Hh4PeZ0qYift6fFwUQkLaigioiIe/lpKqwfA8Br9o78HNKY+d0qE+DnbXEwEUkrKqgiIuI+fl+IWfkcNuA9R3NWB7RkQY8q5Mjma3UyEUlDKqgiIuIe/voGs6Q3NgzTHfWZ4duB+T0qExHsb3UyEUljKqgiImK9Qxsx8zthczlY4qzO2149mN2jCoVyZbM6mYhYQJdCioiItY7+iJndFpvjKmudFRht68cX3R+mRESQ1clExCIaQRUREeuc/BUzsyU2exwbnaUYYp7l025VKZ8vu9XJRMRCGkEVERFrnN2L+fJJbPEx/OgqxgDXEN7vVJWHC+WwOpmIWEwFVURE0t75g5jpTbFdPsevrkL0cgxjQrtq1CkWanUyEXEDKqgiIpK2oo9dK6exp/jTFUlX+wuMaV2VhiXDrE4mIm5CBVVERNJO7BmY0Qxb9FEOusLolDCCF56sRrOyeaxOJiJuRAVVRETSxuXzMKM5nNvPMZOTDgkv0feJarStnM/qZCLiZlRQRUQk9V2NgZkt4MwfnDbBdEh4kY4NqtG9RkGrk4mIG1JBFRGR1JUQB7Nbw4lfOG+y0SHhRZ6oU53+dQtbnUxE3JQKqoiIpB5HPMztAEe2EmOy0ClhBDWq1mDoY8WsTiYibkwFVUREUofTDgu6wcHviDO+dE0YRqmKtRjV5CFsNpvV6UTEjamgiohIynM5YUkf2LuSeONNT/tQIsvU4Y0nS6mcisgdqaCKiEjKcrlg+bOwayEJxpPe9kEEPPgIk54qg6eHyqmI3JkKqoiIpBxj4Ovn4ZcvcRobz9oHYAo/xvvty+HtqV85InJ3vKwOICIiGYQxsHoE/PQZLmNjqL0PsQ805tNOFfD18rQ6nYikIyqoIiJy/4yBtSNh20cAvODoxakCzfm8U0X8vFVORSR5VFBFROT+ffcGbHkPgBftPfg7sgXTulbE30flVESSTwVVRETuz4YJsHEiAKPsXfgzT0tmdKtEFh/9ihGRe6O/PURE5N59//a10VPgNXsHdka04cvulcnmq18vInLv9DeIiIjcmy0fwPpXAXjT3pZtYe2Y1b0ygX7eFgcTkfROBVVERJJv2yfwzUsAvG1vRVRoR+b0qEKQv8qpiNw/FVQREUme7V9cW+sUeN/RnNU5OjGnR2WCs/hYHExEMgoVVBERuXu/zIQVgwCY4niCpcFdmdurKjmy+VqbS0QyFN3WQ0RE7s6v8zDLBgDwuaMh84N6MufpquQKUDkVkZSV7IK6ceNGmjRpQkREBDabjaVLl952+65du2Kz2W54lChRInGb0aNH3/D6gw8+mOwPIyIiqWTXYszSPtgwfOmox/TA3sx+uiqhgX5WJxORDCjZBTUuLo4yZcowefLku9r+3Xff5eTJk4mPo0ePEhISwlNPPZVkuxIlSiTZbtOmTcmNJiIiqWHPcsyintiMizmOunySrS+zn65KWJDKqYikjmSfg9qoUSMaNWp019sHBQURFBSU+PXSpUu5cOEC3bp1SxrEy4uwsLDkxhERkdT05yrMgm7YjJOFzlp8kKU/c5+uRp5gf6uTiUgGlubnoE6dOpV69eqRP3/+JM/v27ePiIgIChUqRIcOHThy5Mgt3yM+Pp6YmJgkDxERSWF/rsLM74zNZWeZsxrvZhnInN7ViQzJYnUyEcng0rSgnjhxgq+//pqePXsmeb5KlSpMmzaN1atX89FHH3Ho0CFq1qzJpUuXbvo+48aNSxyZDQoKIjIyMi3ii4hkHv9TTr9yVuWtLEOY3bs6+XKonIpI6rMZY8w972yzsWTJEpo3b35X248bN4633nqLEydO4ONz6/XyLl68SP78+Xn77bfp0aPHDa/Hx8cTHx+f+HVMTAyRkZFER0cTGBiY7M8hIiL/41/ldFKW55ilkVMRuU8xMTEEBQXdVV9Ls3VQjTF8/vnndOrU6bblFCA4OJiiRYuyf//+m77u6+uLr6+WNRERSXEqpyLiBtJsin/Dhg3s37//piOi/xYbG8uBAwcIDw9Pg2QiIgLcOK2fVeVURKyR7IIaGxvLzp072blzJwCHDh1i586diRc1jRgxgs6dO9+w39SpU6lSpQolS5a84bWhQ4eyYcMGDh8+zJYtW3jyySfx9PSkXbt2yY0nIiL3QuVURNxIsqf4t2/fTt26dRO/HjJkCABdunRh2rRpnDx58oYr8KOjo1m0aBHvvvvuTd/z2LFjtGvXjnPnzpErVy5q1KjBDz/8QK5cuZIbT0REkut/yuly58O8ne05Zj1dnbzZVU5FxBr3dZGUu0jOSbciIvI//lVO38o2lFm9a2idUxFJcW55kZSIiLiZG0ZOn2dW7+oqpyJiuTRfqF9ERNzATcrpTJVTEXETGkEVEcls/qecrvifkdMIlVMRcRMqqCIimcm/y2nA88x6WuVURNyLCqqISGZxs3LauzrhQSqnIuJedA6qiEhmsHsZZn6nxHL6TuAwlVMRcVsaQRURyeh2LcIs6oXNOFnqrMZHwc8z++lqhAb4WZ1MROSmVFBFRDKyX+dhlvbBZlwsctZkashzzO5VjRzZfK1OJiJySyqoIiIZ1S8zMcsGYMMw11GHWaFDmNWjKtmz+lidTETktlRQRUQyou1fwIpB2ICZjkdZFDaYmT0eJsjf2+pkIiJ3pIIqIpLRbPsEvn4egC8cDViV51lmdKtMgJ/KqYikDyqoIiIZydbJsOZFAD5xNOa7yGeY1rUSWX31172IpB/6G0tEJKPY9B9YNxqAyY6m/FCgP593roS/j6e1uUREkkkFVUQkI9gwAb57A4B3HC347YG+fNqxAn7eKqcikv6ooIqIpGfGXCumGycCMNHemv0P9mZKu/L4eOleLCKSPqmgioikV8Zcm9Lf/A4AY+3tOFHiaT5oUxZvT5VTEUm/VFBFRNIjY2DNS/DDZABetXciunRP3mlVGi+VUxFJ51RQRUTSG5cLvh4GP30KwMv2biSU68bEFqXx9LBZHE5E5P6poIqIpCcuJ3w1EHbOxGVsvOjogW/lboxvUgIPlVMRySBUUEVE0gunHbP4aWx/LMZpbAy19yGsVleGNSiGzaZyKiIZhwqqiEh6YL+KWdAF21+rSTCeDLQ/Q6n6nehft7DVyUREUpwKqoiIu0uIw8xpj+1QFFeNN33sg6jduAPdqhe0OpmISKpQQRURcWdXozGznsJ2dBtxxpde9qE0f7IdrStFWp1MRCTVqKCKiLiry+dxfdkCj5O/EGOy0MPxAp3btKZJmQirk4mIpCoVVBERd3TpNK4ZzfA4u4dzJoDuzhd5pkNL6j2U2+pkIiKpTgVVRMTdRB/DNa0JHhcOctoE09O8wvCuzaleOKfVyURE0oQKqoiIOzl/EOe0pnjGHOWYyUkv20he796ECvlDrE4mIpJmVFBFRNzFmT9xTm+KZ9xpDrrCGOA1iok9GlMyT5DVyURE0pQKqoiIOzj5G84ZzfC8cp4/XZEM9hnFe70aUiR3gNXJRETSnAqqiIjVjv6Ec2ZLPOOj+c1VkOFZRvNxr8fIlyOL1clERCyhgioiYqWDUThnt8PTcZmfXEUZEziaqb0eITzI3+pkIiKWUUEVEbHKnhW4FnTD05XA986SvJvrVaZ1r0mObL5WJxMRsZQKqoiIFXbOxizrj4dx8bWzErPyvsIXXaoR4OdtdTIREcupoIqIpLUfPoLVw7EBCxy1WFv4ZT7rUBE/b0+rk4mIuAUVVBGRtGIMRI2HDeMB+MzRiD2lhvFhq7J4eXpYHE5ExH2ooIqIpAWXC7P6BWw/fgLAJPtTxFUZxMQnSuDhYbM4nIiIe1FBFRFJbU4HZll/bL/NBWCkvQs5H3mG5x4pjM2mcioi8m8qqCIiqcl+FdeCrnj89TUO48FQex/KN+lN56oFrE4mIuK2VFBFRFJL/KVra5z+/T3xxptnHM/SuHV3mpXNY3UyERG3poIqIpIaLp/H+WULPE/+Qqzxo6/rebp16sQjD+a2OpmIiNtTQRURSWkxJ3BMb47Xub2cN9noy0s8170tlQuGWJ1MRCRdUEEVEUlJ5w7gmN4Mr5ijnDQhPOM1ktHdW1AyT5DVyURE0g0VVBGRlHLqdxzTn8TrylkOuXIz1G8ME3s9QaFc2axOJiKSrqigioikhMObccxqg5f9Entc+Xg16HUm92xAWJCf1clERNIdFVQRkfu1ZwXOBd3wciWwzfUgH4a9xsdd6xKUxdvqZCIi6ZIKqojI/dgxHdfyQXji4htnBZYUeo2POz6Mn7en1clERNItFVQRkXthDOb7t7B9+xoewDxHHX4tO4r3nyyLl6eH1elERNI1FVQRkeRyuXCtHo7Hjx8DMNnRlKs1X+KNx4rp1qUiIilABVVEJDkcCTiX9MHzj0UAvOroRP5Gz9G/ekGLg4mIZBwqqCIidys+Fsfcjngd+g678eQFZ18ead2fJ0pHWJ1MRCRDUUEVEbkbceewf9kS71O/cNn4MtgMoXPXnlQvnNPqZCIiGY4KqojInVw8gn1ac7wvHuC8ycZgrxd5vntH3R1KRCSVqKCKiNzOmT3YpzfHO+4Ux0xOhvuP5vWeLSiQM6vVyUREMiwVVBGRWznyA46ZT+GdEMNeV15ez/46b/dsRGiA7g4lIpKaVFBFRG5m72qc8zrj5Ypnu6soUyLe4MOudQnw092hRERSmwqqiMi/mO1fYFYMwRMX653lWPngOCa3qYyvl+4OJSKSFlRQRUSuMwbXd2Px2DgBGzDfUZuDD7/BpMdL4uGhBfhFRNKKCqqICIDTjuOrZ/H6dRYA7zmeJKDhSIbXKGRxMBGRzEcFVUQkPpaEuZ3xObQep7Ex2tWDam2G0qhUuNXJREQyJRVUEcncYs8SP6Mlvmd+5Yrx4QXbYDr16EOlAiFWJxMRybRUUEUk8zp3gPhpT+J76W/OmQCG+77MCz07UDg0wOpkIiKZmgqqiGROx3aQ8GVLfOMvcMSVi9eCX+eNns0JDdQapyIiVlNBFZHM5681OOZ1wcd5hd9cBfk473je7vyo1jgVEXETKqgikqmYHdMxywfjhZMoZxlWlxjPO089jLenh9XRRETkv1RQRSRzMAbnd+Pw3PgmNmCBoxZHq49nXMOHsNm0xqmIiDtRQRWRjM/pIGHZQHx++/81TnM8MZohDxewNpeIiNyUCqqIZGzxsVyd0xm/w9fWOB1jelCj/TDqP5Tb6mQiInILKqgiknHFnOTK9Fb4n9vFFePDS55D6NKtL2Uig61OJiIit6GCKiIZ0+k/uDq9Jf6XT/KPCeTVrK8wrGdHIkOyWJ1MRETuQAVVRDKeA9+RMKcjfo5YDrjCeT98HK93fYIgfy0jJSKSHiR7XZWNGzfSpEkTIiIisNlsLF269LbbR0VFYbPZbnicOnUqyXaTJ0+mQIEC+Pn5UaVKFX788cfkRhMRwfXzlzhntsTHEcs214NMK/4pE3o1UzkVEUlHkl1Q4+LiKFOmDJMnT07Wfnv37uXkyZOJj9DQ0MTX5s2bx5AhQxg1ahQ///wzZcqUoUGDBpw5cya58UQkszIGx7rX8PhqAJ7GyVJnNbbX/JwxbWvg46U1TkVE0pNkT/E3atSIRo0aJfsbhYaGEhwcfNPX3n77bXr16kW3bt0AmDJlCitXruTzzz9n+PDhyf5eIpLJOOKJX9wP390LAfjQ+SQRT75O//J5LQ4mIiL3Is2GFcqWLUt4eDj169dn8+bNic8nJCSwY8cO6tWr9/+hPDyoV68eW7duvel7xcfHExMTk+QhIpnUlQtc+aIZvrsX4jAejKYP5bu9RXOVUxGRdCvVC2p4eDhTpkxh0aJFLFq0iMjISOrUqcPPP/8MwD///IPT6SR37qRrEubOnfuG81SvGzduHEFBQYmPyMjI1P4YIuKOLvzNlSn18D++lUvGn2F+r9Cx3ys8XCiH1clEROQ+pPpV/MWKFaNYsWKJX1erVo0DBw7wn//8hy+//PKe3nPEiBEMGTIk8euYmBiVVJHM5vgOrs54Cv/4c5w0IYwPeY1XejxFzmy+VicTEZH7ZMkyU5UrV2bTpk0A5MyZE09PT06fPp1km9OnTxMWFnbT/X19ffH11S8hkczK7FmBY0F3/Fzx7HblZ0ahCbzZoT5+3p5WRxMRkRRgyaWtO3fuJDw8HAAfHx8qVKjA+vXrE193uVysX7+eqlWrWhFPRNyYY8uHmHkd8XbFE+Usw6pKnzO2cwOVUxGRDCTZI6ixsbHs378/8etDhw6xc+dOQkJCyJcvHyNGjOD48ePMmDEDgHfeeYeCBQtSokQJrl69ymeffca3337LN998k/geQ4YMoUuXLlSsWJHKlSvzzjvvEBcXl3hVv4gITgdXVgzD/5epAMx1PoKz0USGVitscTAREUlpyS6o27dvp27duolfXz8XtEuXLkybNo2TJ09y5MiRxNcTEhJ47rnnOH78OFmyZKF06dKsW7cuyXu0adOGs2fPMnLkSE6dOkXZsmVZvXr1DRdOiUgmdTWGuNmdyXrkOwDeoT0VOr1KzaKhd9hRRETSI5sxxlgd4n7FxMQQFBREdHQ0gYGBVscRkZR08QhxX7Qka/RfXDE+jPUdRJeeAykcGmB1MhERSYbk9DVLLpISEbkrx7ZzZcZTZE04zxkTzFs5X+WF7u0JyepjdTIREUlFKqgi4pZcvy/Gubg3/iaB3a78LCw6iTFtH8XXSxdDiYhkdCqoIuJejCH+uwn4bhyLB7DOWY6Dtd7llXqlsdlsVqcTEZE0oIIqIu7DEc/lRf3JsmcBANNcjQh9ahJPl9ZtS0VEMhMVVBFxD3HniJ3Rlmynf8RhPJjo2ZPHe7xEmchgq5OJiEgaU0EVEeud/Yu4aS3JFneEGOPP+GzD6d+rN3mC/a1OJiIiFlBBFRFLmYNRxM/uSFbHJY64cvFJ3vG82KU52Xz115OISGal3wAiYhn7j1/gseo5/HCyw1WEqHLv8Gqz6nh66GIoEZHMTAVVRNKey0ncypfIuuMjAL5yVuNyo/d4rloRi4OJiIg7UEEVkbR1NZqYmZ0JPBYFwEe21pTtMo6qhXNam0tERNyGCqqIpJ1zB7g0rRWBlw5yxfjwVpZBdO45mHw5slidTERE3IgKqoikCdeBDcTP6UiAI4aTJoTP8rzOoM6tCfDztjqaiIi4GRVUEUl1V7d+iteaF/DHyU7XA2yq+B4vPqGLoURE5OZUUEUk9TgdXFr2PAG/fQ7AMlcNaPouAyoWtjiYiIi4MxVUEUkdVy4QPaMDQSc3AzDZowPVur1OufwhFgcTERF3p4IqIinv7F/EfNGKoMt/E2d8+U/gUHr0fIbwIN0ZSkRE7kwFVURSlPOvtdjndSXQGcsxk5MZBcbxXIeW+Pt4Wh1NRETSCRVUEUkZxnDl+8n4fvsKfrj4yVWUX6t+wIiGlbHZdDGUiIjcPRVUEbl/jgSiFw0kaM8cABa56pCt5Xv0LJPf4mAiIpIeqaCKyP2JPcOFae3I/s92XMbGZO8uPNLtVUrkCbY6mYiIpFMqqCJyz1zHfiFuRmuyJ5whxvjzUY7h9OzelxzZfK2OJiIi6ZgKqojck6s/z8Vj+TMEmAQOuML5uuRbDGnZCG9PD6ujiYhIOqeCKiLJ43ISvfxFgn6ZAkCUqxwXGk1mQNUSFgcTEZGMQgVVRO7elQucn9GJkJPfA/CFRwvKdp1InQI5LQ4mIiIZiQqqiNwVc3o3MdNaE3LlKJeNL5ODBtOl52BCA/2sjiYiIhmMCqqI3FH8ruWYxb0Icl3hmMnJwsITGNiuOb5eWnxfRERSngqqiNyay0XMN2MJ/GEiAD+4HuLIo5N5tlY5Lb4vIiKpRgVVRG4u/hLnZ3Un5Mg3AMy1NaJg53doXTjM4mAiIpLRqaCKyA3MuYNc/KIVIbEHiDdefJStH0/1epE8wf5WRxMRkUxABVVEkkjYuw7HvK5kd13itAlmVv436NuxHf4+Ot9URETShgqqiFxjDBfXTSJg81h8cPGLqzC7a05mcL0qOt9URETSlAqqiED8Jc7O7EGuo2sAWGarS+6OH9KhaITFwUREJDNSQRXJ5Fxn93Hhi9bkunyQBOPJ1IA+NO/5MuHBWayOJiIimZQKqkgmdvn3r2BxH3KYOE6Z7CwpMo7ubVtrfVMREbGUCqpIZuRycm7lq+TY8S4AP5kHOVHvI/rWLG9xMBERERVUkcznygXOTOtM6OmNAMz3bEyxzu/SLH8ui4OJiIhco4Iqkok4TvzGpeltCY0/zlXjzdSQwbTtMZQc2XytjiYiIpJIBVUkk7i0fS7eKweS3cRz1JWLtaUm0btFU7w8PayOJiIikoQKqkhG53RwZskLhO76DIBNpjSXm3xM90oPWRxMRETk5lRQRTKy2LOc/rwduc//BMBM71ZU6f4WRcKDrc0lIiJyGyqoIhlU/OEfuDKrE7ntZ4g1fswIG0HHbv0J9PO2OpqIiMhtqaCKZDTGcCHqA7JtGE0wDg64wtlW+X36PF4PDw/dslRERNyfCqpIRhIfy6lZTxN2ZCUA31CVLG0+on2JghYHExERuXsqqCIZhOPUHi5Ob0vYlcPYjSczsvWgUc9XiciuW5aKiEj6ooIqkgHE/DQXn1XPktNc5ZTJzspi4+jUug0+XlpCSkRE0h8VVJH0zJHAqYVDCftzOgA/mJJEN55Cj8qlLA4mIiJy71RQRdIpc/EoZz5vR1jM7wDM9n2Kyt3e4uGwIIuTiYiI3B8VVJF0KG73WlwLe5DbFU20ycL8vC/ToUtvsvjoR1pERNI//TYTSU9cLs6sep2c29/GA8MfpgD7an9Iz7rVsNm0hJSIiGQMKqgi6cXl85z8ojPhZ78H4CvPehTsNJnmBcIsDiYiIpKyVFBF0oH4v7cTN7MD4fZTXDXezM75LC16vEBwFh+ro4mIiKQ4FVQRd2YM5zZMITDqZUJwcNjkZlvF/9C18eO6K5SIiGRYKqgi7io+lhOz+hBxZDkA39kqk6X1x7R5qJDFwURERFKXCqqIG0o48Tsx09sTEX8Eh/FgTkBX6vccS1iwv9XRREREUp0Kqog7MYZz339Ktm9fIicJnDQhrHtoLG1btcHbU3eFEhGRzEEFVcRdxMdybGZf8h79CoDNlMX15BQ6lS1ucTAREZG0pYIq4gbij/9OzIz25P3vlP7cgC480uMNIrJntTqaiIhImlNBFbGSMZzd+CmB371Erv9O6X9bYixtWmpKX0REMi8VVBGrxMdydGYfIo9eu0p/M2WhxRQ6lNGUvoiIZG4qqCIWuD6lH/nfKf15AV14tOdYwoKzWB1NRETEciqoImnJGM5s/Iyg715MnNKPKjmONi1a46UpfREREUAFVSTtxMdy9Ms+RB77/yl9j5Yf0670gxYHExERcS8qqCJpIP7470T/z5T+/MBrU/q5gzSlLyIi8m8qqCKpyRhOb/iM4KgXCf3vlP7GUuNp06I1nh42q9OJiIi4JRVUkVRirlzk6Je9yXdiNQCbbeXwavUxbUoWsziZiIiIe1NBFUkFcQd/4MqcruSzn8RhPFgQ1JVHe75BaKCm9EVERO5EBVUkJblcHF81ntzbJ5EVJ0dNLraVn0jrJs01pS8iInKXVFBFUogr5hTHv+hM5IVtAKz3rE7Oth/SqkgBa4OJiIikMyqoIing4q+rsC3rQ6QrmivGh8VhA2nS9QUC/X2sjiYiIpLuqKCK3A9HAkcWDCff3qkA/Gnycaj2+7SvWxubTVP6IiIi90IFVeQexZ/Zx7lpHcl3+U8AvvJtTIku79EoIqfFyURERNK3ZN9bcePGjTRp0oSIiAhsNhtLly697faLFy+mfv365MqVi8DAQKpWrcqaNWuSbDN69GhsNluSx4MP6u464r7ObJqO88OaRFz+k4smK3MfGMdjQ7/kAZVTERGR+5bsghoXF0eZMmWYPHnyXW2/ceNG6tevz6pVq9ixYwd169alSZMm/PLLL0m2K1GiBCdPnkx8bNq0KbnRRFKdib/E4c86EbpuIFm4wg6Ks6vpKtp26oeft6fV8URERDKEZE/xN2rUiEaNGt319u+8806Sr8eOHcuyZctYvnw55cqV+/8gXl6EhYXd1XvGx8cTHx+f+HVMTMxd5xG5V3GHtxM3qwsF7MdwGhtLAjtSo/ubhGXPanU0ERGRDCXZI6j3y+VycenSJUJCQpI8v2/fPiIiIihUqBAdOnTgyJEjt3yPcePGERQUlPiIjIxM7diSmRnD8a/fwmdaA0LtxzhpQlhW9lOeHPy+yqmIiEgqSPOCOmnSJGJjY2ndunXic1WqVGHatGmsXr2ajz76iEOHDlGzZk0uXbp00/cYMWIE0dHRiY+jR4+mVXzJZJwxp/j7/cbk2TYGbxxs8KjC6Q7f0uLJp7TwvoiISCpJ06v4Z8+ezauvvsqyZcsIDQ1NfP5/TxkoXbo0VapUIX/+/MyfP58ePXrc8D6+vr74+vqmSWbJvM79vAyvFc+Q3xVNvPFmae6+NOz6CkFZtLapiIhIakqzgjp37lx69uzJggULqFev3m23DQ4OpmjRouzfvz+N0on8j4TLHJozmIKH5gKw1+TjcJ33aF2njtY2FRERSQNpMsU/Z84cunXrxpw5c2jcuPEdt4+NjeXAgQOEh4enQTqR/xd3eAenJ1VJLKfL/Jvj1y+KBnXrqpyKiIikkWSPoMbGxiYZ2Tx06BA7d+4kJCSEfPnyMWLECI4fP86MGTOAa9P6Xbp04d1336VKlSqcOnUKAH9/f4KCggAYOnQoTZo0IX/+/Jw4cYJRo0bh6elJu3btUuIzityZy8XRVRMI2z6R3Dg4bYL5vuTrNGvRAW/PND9VW0REJFNLdkHdvn07devWTfx6yJAhAHTp0oVp06Zx8uTJJFfgf/LJJzgcDvr370///v0Tn7++PcCxY8do164d586dI1euXNSoUYMffviBXLly3evnErlr9gtHOTGtK/mjtwOw0bMKga0/pFWxwhYnExERyZxsxhhjdYj7FRMTQ1BQENHR0QQGBlodR9KRMz/Mx3/NYAJMLJeNL8sjBvJ452EE+OtCKBERkZSUnL6Wplfxi7gLE3+JQ18+Q6FjSwD4g0Kcrv8BbWpUtziZiIiIqKBKphOzbytX53WnkOMELmNjWbbWVOk+kRI5gqyOJiIiIqigSmbicvL30tfI89u7BOLihMnBj+XG07SpFt0XERFxJyqokinE/3OIM9M6kz/2NwC+9apB7nYf0fyBfBYnExERkX9TQZWMzRhObPycoO9eJpLLXDL+rM73HI07DiKLr7fV6UREROQmVFAlw3JeOsuR6b0o+M93AOykGLGNP+SpyhUtTiYiIiK3o4IqGdLZ7UvxXvUsBV0XSTCeLM/ehZrdXic0KKvV0UREROQOVFAlQzFXYzg061kKHV0MwD6Tl4M1/0OLR+vrVqUiIiLphAqqZBgX92zAvuhpCjlO4TI2VmRtQdkuk2iQO8TqaCIiIpIMKqiS/jniOTh/BAX++hwPDMdMTn4uN47GWj5KREQkXVJBlXQt9u9fiJndnULxBwFY41OPAh3eo2n+PBYnExERkXulgirpk8vJ38vHE/HL20Tg4B8TyIaiL/NEm574enlanU5ERETugwqqpDvxZw5wekbXxEX3v/esTLZWk2lZvKjFyURERCQlqKBK+mEMx7+dQsj3o8nHVS4Zf9ZEPkvDjkPJ5qdF90VERDIKFVRJFxzRJzk6vRcFz38PwA4eIr7JB7SqWMHiZCIiIpLSVFDFvRnDqS0zybJuBAXNJeKNF8tz9uSRbq8Sks3P6nQiIiKSClRQxW05L53h7xm9KXT2WwB2U5ATdd+mZe26WnRfREQkA1NBFbd0eusc/L4ZRiETg914sjy4A9W6jOWhkACro4mIiEgqU0EVt+KK/YfDX/aj0Ok1AOw1+fm75ls8+Wg9jZqKiIhkEiqo4jbObFuIz5rnKOS6iMN4sDyoHZW7jOOxHEFWRxMREZE0pIIqlnPFnefQzP48cHIVAPtMXvZXn0Tz+g01aioiIpIJqaCKpc5uX4rXqsE84DqP09hYHtCaCp3fpFFodqujiYiIiEVUUMUS5soFDs4cyAPHvwLggIlg78MTaNrgCTw8NGoqIiKSmamgSpr755cVeKx4lgec/+AyNpZna0nZzhN4PHcOq6OJiIiIG1BBlTRjrlzk0OzBFDq6GIDDJow/Ko+nSaPmGjUVERGRRCqokib++WUFthWDKOQ8i8vYWJm1OaU6TaRxeC6ro4mIiIibUUGVVOWKO8/h2QMpdHw5AH+b3PxWYSyPP9EST42aioiIyE2ooEqqObttPt5rhlHIdQGnsbEy65OU6DCBJnk0aioiIiK3poIqKc4Zc5q/Z/aj0Jl1AOw3efiz8jgaN2qqUVMRERG5IxVUSTnGcGrTdLJ++zKFzKVrd4MKbEuFTmN5QuuaioiIyF1SQZUUYb9wlONf9qbA+c0A7DYF+bvmBJo/Wl93gxIREZFkUUGV+2MMJ779iKDvX6MAl4k3XizP3pnqnV/loZBAq9OJiIhIOqSCKvcs/uwBTn/Zi3wxOwD4laKcffQtWtasqVFTERERuWcqqJJ8LidH1/yHnNsmkI94rhgfVuTqSZ1Or1AmKIvV6URERCSdU0GVZLl6YjdnZ/UiMm4XAD/ZSnK5wds8VbWKxclEREQko1BBlbvjSODI8rGE/fo+kTi4ZPz5OqIfj3UcRnBWP6vTiYiISAaigip3dGnfZuIW9idf/CEANnuUx9b4HVpXKGNxMhEREcmIVFDllszVaP6eP5x8B+cQgOEfE8i3BQbRsO0zBPr7WB1PREREMigVVLmpczuWYlv1HAWc/wCw2vtRcrecSOsHH7A4mYiIiGR0KqiShDP6JEdnDaDAf29T+rfJzfZSI3mieTt8vTwtTiciIiKZgQqqXONycSrqYwK+f40CJu7abUqztqRU+7G0zBtqdToRERHJRFRQhfiTezg7pw95Y3YCsMs8wOHq42hW7zE8PLTgvoiIiKQtFdTMzJHAseVvEPrrB+TFQZzxZXnOHtTu+BIls2ezOp2IiIhkUiqomVTsvs3ELexH3vjDAGy2lSOh4STaVKmg25SKiIiIpVRQMxlz5SJHFowg8uAcsv136ah1+QbTqN0AgrJo6SgRERGxngpqZmEM57cvwLZ6OPmd5wBY7fUIuVpMpO1DhS0OJyIiIvL/VFAzAcc/hzg5pz+R5zYD15aO2lZyJE2bt8PPW0tHiYiIiHtRQc3IHAmcWD2RkO3vEkk8CcaTZdlaU6bdGFpr6SgRERFxUyqoGVTcvo3ELXqGiKuHAfiJEpytM5aWtWtr6SgRERFxayqoGYyJ+4ej858n39+LyQr8YwJZk2cADdo9S6UAP6vjiYiIiNyRCmpGYQznNk/D59uR5HPFALDC6zFCnxxHhxK6CEpERETSDxXUDCDh5G7Ozu1PnuifAfjT5OPXMiNp3rQFvl66CEpERETSFxXU9CzhMieWv0au3z8hDw4uG1+WBnXi4fYv0yYsu9XpRERERO6JCmo6dWnX1yQsG0yE/SQAG6jIlcfG0a5aJd0JSkRERNI1FdR0xkQf59i8IUSeWA3ACRPC+gLP0aR1L4Kz+lqcTkREROT+qaCmF047Z9a9S8APk4g0V3AYD5b5NqXgU6/TqUik1elEREREUowKajpw5a8oYhcPIvTqIQB+MUXYV3EMTz7eEG9PD4vTiYiIiKQsFVQ3ZmJOcnz+UPIeW4E/cM4EsCK0D/XaD6Zc9qxWxxMRERFJFSqo7sjp4J9v3yPrlonkNZdxGRtfeTcgR9PX6FK6qNXpRERERFKVCqqbubr/e2IWDyL08n4AfjUP8Ee5UbRo/AR+3lrTVERERDI+FVQ3YS6d4sTCYeT5exl+wHmTjWU5evFI++donzPA6ngiIiIiaUYF1WpOB+c3TMbv+/HkuT6d71WfoMZj6FruQa1pKiIiIpmOCqqF4g9uJmbhs+S6vA+A31yF+LX0y7Rq2hx/H03ni4iISOakgmqF2LOcXPg84YeXkAu4aLKyOHsParcbSqfcQVanExEREbGUCmpactq5EDUZ380TCHfFAbDM41H8Gr1Gt4oPaTpfREREBBXUNBO/dz2xS58jx5Vri+3vchXgpxIv8VTzFmTz1WEQERERuU7NKJWZ84c4vfB5wk6sxZdri+0vzt6d2m2G0C082Op4IiIiIm5HBTW1JFzm3Jo3CdjxIWEk4DAeLPZqRPbHR9KzfDFN54uIiIjcggpqSjOGy78uwr7yRXLYTwOw1VWCv8q/TOvHG+jqfBEREZE7UEFNQa6Tu/hn4SBCz/0EwDGTkxW5+9O4TW+q5shqcToRERGR9MEjuTts3LiRJk2aEBERgc1mY+nSpXfcJyoqivLly+Pr60vhwoWZNm3aDdtMnjyZAgUK4OfnR5UqVfjxxx+TG806l89zdt4zmI9rEnruJ64ab6b5tONwm+/o028IkSqnIiIiInct2QU1Li6OMmXKMHny5Lva/tChQzRu3Ji6deuyc+dOBg0aRM+ePVmzZk3iNvPmzWPIkCGMGjWKn3/+mTJlytCgQQPOnDmT3Hhpy+UkdvMnxL1Vllx7ZuCJizXmYZZUW0r7YR9S46F8VicUERERSXdsxhhzzzvbbCxZsoTmzZvfcpsXXniBlStXsmvXrsTn2rZty8WLF1m9ejUAVapUoVKlSnzwwQcAuFwuIiMjeeaZZxg+fPgdc8TExBAUFER0dDSBgYH3+nGSxXFoM9GLBpMjdi8Ae115WV/wOVq1bE9ooF+aZBARERFJL5LT15I9gppcW7dupV69ekmea9CgAVu3bgUgISGBHTt2JNnGw8ODevXqJW7zb/Hx8cTExCR5pKUjS8fgNf1xcsTuJdpkYUqW3sR2+45+3bqrnIqIiIjcp1S/SOrUqVPkzp07yXO5c+cmJiaGK1eucOHCBZxO5023+fPPP2/6nuPGjePVV19Ntcx38qtPOSKMB0ttj+JR72V6VS+Dp4eWjRIRERFJCenyKv4RI0YwZMiQxK9jYmKIjIxMs+/fqEFjvuQrnqxTheAsPmn2fUVEREQyg1QvqGFhYZw+fTrJc6dPnyYwMBB/f388PT3x9PS86TZhYWE3fU9fX198fX1TLfOdeHl60O3xmpZ9fxEREZGMLNXPQa1atSrr169P8tzatWupWrUqAD4+PlSoUCHJNi6Xi/Xr1yduIyIiIiKZR7ILamxsLDt37mTnzp3AtWWkdu7cyZEjR4Br0++dO3dO3L5Pnz4cPHiQYcOG8eeff/Lhhx8yf/58Bg8enLjNkCFD+PTTT5k+fTp79uyhb9++xMXF0a1bt/v8eCIiIiKS3iR7in/79u3UrVs38evr54J26dKFadOmcfLkycSyClCwYEFWrlzJ4MGDeffdd8mbNy+fffYZDRo0SNymTZs2nD17lpEjR3Lq1CnKli3L6tWrb7hwSkREREQyvvtaB9VdWLEOqoiIiIjcPbdaB1VEREREJDlUUEVERETEraigioiIiIhbUUEVEREREbeigioiIiIibkUFVURERETcigqqiIiIiLgVFVQRERERcSsqqCIiIiLiVlRQRURERMStqKCKiIiIiFtRQRURERERt6KCKiIiIiJuRQVVRERERNyKCqqIiIiIuBUVVBERERFxKyqoIiIiIuJWVFBFRERExK2ooIqIiIiIW1FBFRERERG34mV1gJRgjAEgJibG4iQiIiIicjPXe9r13nY7GaKgXrp0CYDIyEiLk4iIiIjI7Vy6dImgoKDbbmMzd1Nj3ZzL5eLEiRMEBARgs9nS5HvGxMQQGRnJ0aNHCQwMTJPvKSlHxy/90zFM/3QM0z8dw/QtrY+fMYZLly4RERGBh8ftzzLNECOoHh4e5M2b15LvHRgYqB/KdEzHL/3TMUz/dAzTPx3D9C0tj9+dRk6v00VSIiIiIuJWVFBFRERExK2ooN4jX19fRo0aha+vr9VR5B7o+KV/Oobpn45h+qdjmL658/HLEBdJiYiIiEjGoRFUEREREXErKqgiIiIi4lZUUEVERETEraigioiIiIhbUUEVEREREbeignobkydPpkCBAvj5+VGlShV+/PHH226/YMECHnzwQfz8/ChVqhSrVq1Ko6RyM8k5fp9++ik1a9Yke/bsZM+enXr16t3xeEvqS+7P4HVz587FZrPRvHnz1A0od5TcY3jx4kX69+9PeHg4vr6+FC1aVH+XWii5x++dd96hWLFi+Pv7ExkZyeDBg7l69WoapZV/27hxI02aNCEiIgKbzcbSpUvvuE9UVBTly5fH19eXwoULM23atFTPeVNGbmru3LnGx8fHfP755+aPP/4wvXr1MsHBweb06dM33X7z5s3G09PTTJgwwezevdu8/PLLxtvb2/z+++9pnFyMSf7xa9++vZk8ebL55ZdfzJ49e0zXrl1NUFCQOXbsWBonl+uSewyvO3TokMmTJ4+pWbOmadasWdqElZtK7jGMj483FStWNI8//rjZtGmTOXTokImKijI7d+5M4+RiTPKP36xZs4yvr6+ZNWuWOXTokFmzZo0JDw83gwcPTuPkct2qVavMSy+9ZBYvXmwAs2TJkttuf/DgQZMlSxYzZMgQs3v3bvP+++8bT09Ps3r16rQJ/D9UUG+hcuXKpn///olfO51OExERYcaNG3fT7Vu3bm0aN26c5LkqVaqY3r17p2pOubnkHr9/czgcJiAgwEyfPj21Isod3MsxdDgcplq1auazzz4zXbp0UUG1WHKP4UcffWQKFSpkEhIS0iqi3EZyj1///v3NI488kuS5IUOGmOrVq6dqTrk7d1NQhw0bZkqUKJHkuTZt2pgGDRqkYrKb0xT/TSQkJLBjxw7q1auX+JyHhwf16tVj69atN91n69atSbYHaNCgwS23l9RzL8fv3y5fvozdbickJCS1Yspt3OsxHDNmDKGhofTo0SMtYspt3Msx/Oqrr6hatSr9+/cnd+7clCxZkrFjx+J0OtMqtvzXvRy/atWqsWPHjsTTAA4ePMiqVat4/PHH0ySz3D936jJeaf4d04F//vkHp9NJ7ty5kzyfO3du/vzzz5vuc+rUqZtuf+rUqVTLKTd3L8fv31544QUiIiJu+EGVtHEvx3DTpk1MnTqVnTt3pkFCuZN7OYYHDx7k22+/pUOHDqxatYr9+/fTr18/7HY7o0aNSovY8l/3cvzat2/PP//8Q40aNTDG4HA46NOnDy+++GJaRJYUcKsuExMTw5UrV/D390+zLBpBFfmX8ePHM3fuXJYsWYKfn5/VceQuXLp0iU6dOvHpp5+SM2dOq+PIPXK5XISGhvLJJ59QoUIF2rRpw0svvcSUKVOsjiZ3ISoqirFjx/Lhhx/y888/s3jxYlauXMlrr71mdTRJhzSCehM5c+bE09OT06dPJ3n+9OnThIWF3XSfsLCwZG0vqedejt91kyZNYvz48axbt47SpUunZky5jeQewwMHDnD48GGaNGmS+JzL5QLAy8uLvXv38sADD6RuaEniXn4Ow8PD8fb2xtPTM/G54sWLc+rUKRISEvDx8UnVzPL/7uX4vfLKK3Tq1ImePXsCUKpUKeLi4nj66ad56aWX8PDQmJi7u1WXCQwMTNPRU9AI6k35+PhQoUIF1q9fn/icy+Vi/fr1VK1a9ab7VK1aNcn2AGvXrr3l9pJ67uX4AUyYMIHXXnuN1atXU7FixbSIKreQ3GP44IMP8vvvv7Nz587ER9OmTalbty47d+4kMjIyLeML9/ZzWL16dfbv35/4jwuAv/76i/DwcJXTNHYvx+/y5cs3lNDr/9gwxqReWEkxbtVl0vyyrHRi7ty5xtfX10ybNs3s3r3bPP300yY4ONicOnXKGGNMp06dzPDhwxO337x5s/Hy8jKTJk0ye/bsMaNGjdIyUxZK7vEbP3688fHxMQsXLjQnT55MfFy6dMmqj5DpJfcY/puu4rdeco/hkSNHTEBAgBkwYIDZu3evWbFihQkNDTWvv/66VR8hU0vu8Rs1apQJCAgwc+bMMQcPHjTffPONeeCBB0zr1q2t+giZ3qVLl8wvv/xifvnlFwOYt99+2/zyyy/m77//NsYYM3z4cNOpU6fE7a8vM/X888+bPXv2mMmTJ2uZKXf0/vvvm3z58hkfHx9TuXJl88MPPyS+Vrt2bdOlS5ck28+fP98ULVrU+Pj4mBIlSpiVK1emcWL5X8k5fvnz5zfADY9Ro0alfXBJlNyfwf+lguoeknsMt2zZYqpUqWJ8fX1NoUKFzBtvvGEcDkcap5brknP87Ha7GT16tHnggQeMn5+fiYyMNP369TMXLlxI++BijDHmu+++u+nvtuvHrUuXLqZ27do37FO2bFnj4+NjChUqZL744os0z22MMTZjNO4uIiIiIu5D56CKiIiIiFtRQRURERERt6KCKiIiIiJuRQVVRERERNyKCqqIiIiIuBUVVBERERFxKyqoIiIiIuJWVFBFRERExK2ooIqIiIiIW1FBFRERERG3ooIqIiIiIm7l/wAvZojwNWzvtwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "pts = pinn.problem.spatial_domain.sample(256, 'grid', variables='x')\n", "predicted_output = pinn.forward(pts).extract('u').as_subclass(torch.Tensor).cpu().detach()\n", "true_output = pinn.problem.truth_solution(pts).cpu().detach()\n", "pts = pts.cpu()\n", "fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(8, 8))\n", "ax.plot(pts.extract(['x']), predicted_output, label='Neural Network solution')\n", "ax.plot(pts.extract(['x']), true_output, label='True solution')\n", "plt.legend()" ] }, { "cell_type": "markdown", "id": "bf47b98a", "metadata": {}, "source": [ "The solution is overlapped with the actual one, and they are barely indistinguishable. We can also plot easily the loss:" ] }, { "cell_type": "code", "execution_count": 10, "id": "bf6211e6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAGwCAYAAABFFQqPAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAQ65JREFUeJzt3Xl4VOXh/v97ZrLvIYEsZANZw5KwRIjggg3ihktttRQRtWq11GqxVqlXtdZvxbY/W6tG2/JpRVu3ahGruKOCCzuELexLEghJ2JLJQraZ8/sjZCSGQAiTnFner+vKBZk5zNyPYub2Oc95jsUwDEMAAAB+yGp2AAAAALNQhAAAgN+iCAEAAL9FEQIAAH6LIgQAAPwWRQgAAPgtihAAAPBbAWYH8HROp1OlpaWKjIyUxWIxOw4AAOgEwzBUXV2t5ORkWa0dz/tQhE6jtLRUqampZscAAABdUFJSopSUlA6fpwidRmRkpKSWf5BRUVEmpwEAAJ1ht9uVmprq+hzvCEXoNFpPh0VFRVGEAADwMqdb1sJiaQAA4LcoQh3Iz89XZmamcnJyzI4CAAC6iYW7z5+a3W5XdHS0qqqqODUGAICX6OznNzNCAADAb1GEAACA36IIAQAAv0URAgAAfosi1AGuGgMAwPdx1dhpcNUYAADeh6vGAAAAToMiBAAA/BZFCAAA+C2KkElKK49pywG72TEAAPBrFKEOdPdVY3/4YKsuf/oLzVmwQQ3Njm55DwAAcGoUoQ7MmjVLhYWFWrVqldtf2+E05DAkw5BeXVmiP3ywze3vAQAATo8iZAKb1aJnpo1S/g9HS5L+8eUeLd5SbnIqAAD8D0XIRFeMTNKM8emSpJ++sk5rio6anAgAAP9CETLZw1MzdcGg3jrW5NCMf6zQku0HzY4EAIDfoAiZLNBm1fPTR+v8gfGqa3ToR/NX6dWVxWbHAgDAL1CEPEB4cID+MTNHV2cnq9lpaM6CjfrVWxvV2Ow0OxoAAD6NIuQhggKseuqGbN0/ZbAsFumVFcWaNm+5Kuz1ZkcDAMBnUYQ6YMbd5y0Wi2ZNGqB/zsxRZEiA1hQd1ZXPfKlluw73WAYAAPwJd58/DbPuPr/3UK3u+NdqbS+vkcUi/fiCczR78iAFBdBdAQA4He4+7+Uy4sO1cNYE/SAnVYYh/XXJLl33/NfaVlZtdjQAAHwGRciDhQUF6InrRur56aMVHRqojfurdOUzX+jJj7apvonbcgAAcLYoQl7gshFJ+vDeC5Q3NEFNDkPPfLpTlz/9BWuHAAA4SxQhL5EYHaJ5N43Rc9NHq3dksHYfrNW0ect157/WaO+hWrPjAQDglVgsfRpmLZY+lapjTfrjh1v1yopiOQ0p0GbRjPEZuvviAYoNDzI7HgAApuvs5zdF6DQ8sQi12lZWrcff2+K6LUd4kE03T8jQbRP7U4gAAH6NIuQmnlyEWi3ZflC/f3+rCg/YJX1TiH40sb96UYgAAH6IInSW8vPzlZ+fL4fDoe3bt3t0EZIkwzD0cWG5nvpkh6sQhQRa9b0xKbp1Qj/17x1hckIAAHoORchNvGFG6EStheiZT3dq4/4qSZLFIn1nSB/ddn5/jevXSxaLxeSUAAB0L4qQm3hbEWplGIZW7Dmi//titz7ZUuF6fETfaN12fj9dMSJJATYuGgQA+CaKkJt4axE60a6DNfrnl3v05pp9ajh+R/uU2FDdfn5/fX9sisKCAkxOCACAe1GE3MQXilCrI7WN+vfyIr349V4drm2UJMWGBeqm3AzdlJuuuIhgkxMCAOAeFCE38aUi1Kq+yaE31uzTvKW7VXykTlLLwuobxqbqJ5MGKCEqxOSEAACcHYqQm/hiEWrlcBr6YFOZ/rpkl2thdXCAVTflpuvOC89hhggA4LUoQm7iy0WolWEY+nrXYf354+1aXXRUkhQWZNOtE/rp9gv6Kzo00OSEAACcGYqQm/hDEWplGIaWbD+oJz/a7poh6hUepNmTB+kHOalcZQYA8BoUITfxpyLUyjAMfVRYrj9+uE07K2okSYMTIvXrKzM1cWC8yekAADg9ipCb+GMRatXkcOqVFcX68yfbVVnXJEm6JDNBj149TEnRoSanAwCgY539/OZcRwfy8/OVmZmpnJwcs6OYJtBm1czzMvT5Ly7SLRMyFGC16KPCck3+01K9+PVeOZx0aACAd2NG6DT8eUbo27aVVWvOgg1aW1wpScpOjdHc747Q0CT//ucCAPA8zAjB7QYnRurNO8/TY9cMV2RwgApKKnXVs1/q+c93MTsEAPBKFCGcEavVohnj0/Xx7As1OTNBTQ5Dv/9gq6b9fblKjm/OCACAt6AIoUsSo0P09xlj9IfvjVR4kE0r9x7R5X/5Qv9bX2p2NAAAOo0ihC6zWCy6fmyq3r/nAo1Jj1V1Q7N+9uo6PfrOZjU5nGbHAwDgtChCOGtpcWF6/Y7xmjXpHEnSC1/t1bS/L1eFvd7kZAAAnBpFCG4RYLPq/ilDNO+msYoMDtDqoqO68pkvten4DtUAAHgiihDcanJmgv5390QNSohQRXWDbvjbMn22rcLsWAAAnBRFCG7XLz5cb951niYMiFNto0O3vbhar60sNjsWAADtUITQLaJCAvXCzefqu6P7yuE09OCCjfrnl3vMjgUAQBsUIXSboACrnvx+ln58YX9J0m/fLdT/fbHb5FQAAHyDIoRuZbFY9OClQ/TTSQMkSf9v0RbNW0oZAgB4BooQup3FYtF9lwzSvXkDJUm/e2+L3lhdYnIqAAAoQughFotF9+YN0o8vaDlN9uCCjVq8pdzkVAAAf0cRQo964NIhrgXUs15Zyz5DAABTUYTQo6xWi35/3UhdMKi36pucuuOl1TpU02B2LACAn6IIdSA/P1+ZmZnKyckxO4rPCbRZ9cy0UeofH67Sqnr95OW13JsMAGAKi2EYhtkhPJndbld0dLSqqqoUFRVldhyfsrOiWtfkf62ahmbdOqGfHp6aaXYkAICP6OznNzNCMM2APpH68w3ZkqR/frWHW3EAAHocRQimmpyZoJvPy5Ak3f/Geh2sZr0QAKDnUIRgugcvG6IhiZE6VNOoX7yxXpytBQD0FIoQTBcSaNPT00YpOMCqJdsPasHa/WZHAgD4CYoQPMKghEjdc3zn6ccWFXJJPQCgR1CE4DFuP7+/hiZFqbKuSY+9W2h2HACAH6AIwWME2qz6/XUjZLVIbxeUasn2g2ZHAgD4OIoQPMrIlBjNPH4V2WPvFqqZjRYBAN2IIgSPc2/eIMWGBWpnRY1eWVlsdhwAgA+jCMHjRIcGavYlgyVJf/p4u6rqmkxOBADwVRQheKRpOakalBChyromPff5TrPjAAB8FEUIHinAZtWcy4ZKkl5ctlcV1fUmJwIA+CKKEDzWRYN7a1RajOqbnHrus11mxwEA+CCKEDyWxWLRfZNb1gq9sqJYB6qOmZwIAOBrKELwaBMGxOncfr3U6HDq2U9ZKwQAcC+KEDxay6zQIEnSG6v3qcLOWiEAgPtQhODxxvWP05j0WDU6nHrh671mxwEA+BCKELzCjy/oL0n69/Ii1TQ0m5wGAOArKELwCnlDE9S/d7iq65v1GrtNAwDchCIEr2C1WnT7+S2zQv/8co+auAcZAMAN/KIIXXvttYqNjdX3vvc9s6PgLFw7qq/iI4JVWlWv9zYeMDsOAMAH+EURuueee/TSSy+ZHQNnKSTQphvHp0mSXl7O6TEAwNnziyJ00UUXKTIy0uwYcINp56bJZrVo5d4j2lpmNzsOAMDLmV6Eli5dqqlTpyo5OVkWi0ULFy5sd0x+fr4yMjIUEhKicePGaeXKlT0fFB4hISpEU4YlSGq5ggwAgLNhehGqra1VVlaW8vPzT/r866+/rtmzZ+uRRx7R2rVrlZWVpSlTpqiiosJ1THZ2toYPH97uq7S0tKeGgR5047h0SdJba/dzKT0A4KwEmB3gsssu02WXXdbh83/60590++2365ZbbpEk/fWvf9WiRYv0z3/+Uw8++KAkqaCgwG15Ghoa1NDQ4Prebuf0i6fJPSdO/XuHa/fBWr21br9mjE83OxIAwEuZPiN0Ko2NjVqzZo3y8vJcj1mtVuXl5WnZsmXd8p5z585VdHS06ys1NbVb3gddZ7FYXOXnlRUsmgYAdJ1HF6FDhw7J4XAoISGhzeMJCQkqKyvr9Ovk5eXp+9//vt577z2lpKScskTNmTNHVVVVrq+SkpIu50f3uXZUXwXZrNpywK7NpVVmxwEAeCnTT431hE8++aTTxwYHBys4OLgb08AdYsKCNDkzQYs2HtCba/ZpWHK02ZEAAF7Io2eE4uPjZbPZVF5e3ubx8vJyJSYmmpQKnuJ7Y1IkSW8XlKqxmZ2mAQBnzqOLUFBQkMaMGaPFixe7HnM6nVq8eLFyc3O79b3z8/OVmZmpnJycbn0fdN35A+PVJzJYR2ob9dm2itP/AQAAvsX0IlRTU6OCggLXlV979uxRQUGBiotbFsHOnj1b8+bN04svvqgtW7borrvuUm1tresqsu4ya9YsFRYWatWqVd36Pui6AJtV147uK0l6c80+k9MAALyR6WuEVq9erUmTJrm+nz17tiRp5syZmj9/vm644QYdPHhQDz/8sMrKypSdna0PPvig3QJq+KfvjU7R35bs1mdbK3SopkHxEazvAgB0nsUwDMPsEJ7MbrcrOjpaVVVVioqKMjsOTuLq/K+0vqRSv74yUz+a2M/sOAAAD9DZz2/TT40BZ+va7GRJ0rsb2EkcAHBmKEIdYLG097h8ZJKsFmldcaVKjtSZHQcA4EUoQh1gsbT36BMZotxz4iRJ7zArBAA4AxQh+ISpI1tOj72z/oDJSQAA3oQiBJ9w6fBEBVgt2nLArp0V1WbHAQB4CYoQfEJMWJAuGNRbErNCAIDOowh1gMXS3mdqVpKklnVC7AoBAOgMilAHWCztffKGJig4wKrdB2tVeMBudhwAgBegCMFnRIYE6uIhfSRJ727g9BgA4PQoQvApl41oOT324aYyTo8BAE6LIgSfMmlwbwXZrNp9qFY7KmrMjgMA8HAUIfiUyJBAnT8wXpL0/sYyk9MAADwdRagDXDXmvaYMT5QkfbCZIgQAODWKUAe4asx7TR6aINvxzRWLDteaHQcA4MEoQvA5seFBGt+/lyTpg03MCgEAOkYRgk+6dBinxwAAp0cRgk+aMixRFou0rrhSB6qOmR0HAOChKELwSX2iQjQ6LVaS9NHmcpPTAAA8FUUIPuuy41ePvb+JXaYBACdHEYLPmnJ8ndDKPUd0uKbB5DQAAE9EEeoA+wh5v9ReYRreN0pOQ/q4kNNjAID2KEIdYB8h39B69diHXD0GADgJihB82qXH1wl9tfOw7PVNJqcBAHgaihB82oA+kTqnd7gaHU59trXC7DgAAA9DEYLPa50VYpdpAMC3UYTg8y4dliRJ+nzbQR1rdJicBgDgSShC8HnD+0apb0yojjU5tHTHQbPjAAA8CEUIPs9isbj2FPqQ02MAgBNQhOAXWtcJfbKlXI3NTpPTAAA8BUWoA2yo6FvGpMcqPiJI9vpmLd992Ow4AAAPQRHqABsq+hab1aJLjp8e+4DNFQEAx1GE4Ddad5n+aHO5HE7D5DQAAE9AEYLfGN8/TlEhATpU06C1xUfNjgMA8AAUIfiNoACr8oYmSGJzRQBAC4oQ/MqUE3aZNgxOjwGAv6MIwa9cMLC3QgNt2l95TJtL7WbHAQCYjCIEvxIaZNNFg3tL4vQYAIAiBD/kugkrl9EDgN+jCMHvTBrSR4E2i3ZW1GhnRbXZcQAAJqIIwe9EhQRqwoB4SZweAwB/RxGCX2rdXHHRRooQAPgzilAHuNeYb7t0eKICbRZtOWDXjnJOjwGAv6IIdYB7jfm2mLAgXTCw5eqx/60vNTkNAMAsFCH4rauykyW1FCE2VwQA/0QRgt+anJmg0ECbig7Xaf2+KrPjAABMQBGC3woLCtDkzJZ7j/2vgNNjAOCPKELwa1dltZwee3dDqRxOTo8BgL+hCMGvXTCot6JDA1VR3aAVuw+bHQcA0MMoQvBrQQFWXT6iZU+htzk9BgB+hyIEv3d1dl9J0qKNB1TX2GxyGgBAT6IIwe+N69dL6XFhqmlo1vvsNA0AfoUiBL9nsVj0/TEpkqT/rC4xOQ0AoCdRhABJ141JkcUirdhzRHsP1ZodBwDQQyhCgKSk6FDXLTfeXLPP5DQAgJ5CEQKOu35sqqSWIsSeQgDgHyhCwHF5mX0UExaoMnu9lu44aHYcAEAPoAgBxwUH2HTtqJZL6V9eXmRyGgBAT6AIdSA/P1+ZmZnKyckxOwp60I3j0yVJi7dWqORInclpAADdjSLUgVmzZqmwsFCrVq0yOwp60Dm9IzRxQLwMQ3p5RbHZcQAA3YwiBHzLjNyWWaHXVxWrvslhchoAQHeiCAHf8p0hfZQcHaKjdU1atOGA2XEAAN2IIgR8S4DNqunH1wq9xKJpAPBpFCHgJG7ISVWQzar1JZVaW3zU7DgAgG5CEQJOIj4iWFdlJ0uS5i3dbXIaAEB3oQgBHbjjgv6SpA82l3H/MQDwURQhoAODEiI1aXBvGYb0jy/3mB0HANANKELAKdx+fFbojTUlOlLbaHIaAIC7UYSAU8jtH6cRfaNV3+TUS8v2mh0HAOBmFCHgFCwWi2tW6KVlRTrWyAaLAOBLKELAaVw+PFEpsaE6Utuo11Zx2w0A8CUUIeA0AmxW3XXROZKkvy3ZrYZmZoUAwFdQhIBO+N6YFCVGhajMXq831+wzOw4AwE0oQkAnBAfY9OMLW9YKPf/5LjU5nCYnAgC4A0UI6KQf5KQpPiJI+44e09sFpWbHAQC4AUUI6KTQIJtuO79lVui5z3bK4TRMTgQAOFsUIeAM3Dg+XTFhgdp9qFaLNh4wOw4A4CxRhIAzEBEcoFsn9JMk5X+6U05mhQDAq1GEgDM087wMRQYHaFt5tT4qLDc7DgDgLFCEgDMUHRqomedlSJKe/WyHDINZIQDwVl0qQi+++KIWLVrk+v6Xv/ylYmJidN5556moqMht4QBPdevEfgoNtGnTfrs+337Q7DgAgC7qUhF6/PHHFRoaKklatmyZ8vPz9Yc//EHx8fH6+c9/7taAZ6ukpEQXXXSRMjMzNXLkSL3xxhtmR4IP6BUepBvHp0mSnlnMrBAAeKsuFaGSkhINGDBAkrRw4UJdd911uuOOOzR37lx98cUXbg14tgICAvTUU0+psLBQH330ke69917V1taaHQs+4Pbz+ysowKq1xZVatuuw2XEAAF3QpSIUERGhw4dbfvB/9NFHmjx5siQpJCREx44dc186N0hKSlJ2drYkKTExUfHx8Tpy5Ii5oeAT+kSFaFpOqiTpmU93mpwGANAVXSpCkydP1m233abbbrtN27dv1+WXXy5J2rx5szIyMs7otZYuXaqpU6cqOTlZFotFCxcubHdMfn6+MjIyFBISonHjxmnlypVdia01a9bI4XAoNTW1S38e+LY7LjxHgTaLlu0+rNV7KdgA4G26VITy8/OVm5urgwcP6r///a/i4uIktRSNadOmndFr1dbWKisrS/n5+Sd9/vXXX9fs2bP1yCOPaO3atcrKytKUKVNUUVHhOiY7O1vDhw9v91Va+s1tEI4cOaKbbrpJf//730+Zp6GhQXa7vc0X0JG+MaG6bnSKJGaFAMAbWQwPWuVpsVj01ltv6ZprrnE9Nm7cOOXk5OjZZ5+VJDmdTqWmpuruu+/Wgw8+2KnXbWho0OTJk3X77bdrxowZpzz2N7/5jR599NF2j1dVVSkqKqrzg4HfKDpcq4ufXCKH09D/fjpBI1NizI4EAH7PbrcrOjr6tJ/fXZoR+uCDD/Tll1+6vs/Pz1d2drZ++MMf6ujRo115yZNqbGzUmjVrlJeX53rMarUqLy9Py5Yt69RrGIahm2++WRdffPFpS5AkzZkzR1VVVa6vkpKSLueHf0iPC9fVWcmSpGeZFQIAr9KlInT//fe7Thlt3LhR9913ny6//HLt2bNHs2fPdlu4Q4cOyeFwKCEhoc3jCQkJKisr69RrfPXVV3r99de1cOFCZWdnKzs7Wxs3buzw+ODgYEVFRbX5Ak7nJ5POkcUifVRYrq1lnE4FAG8R0JU/tGfPHmVmZkqS/vvf/+rKK6/U448/rrVr17oWTnuKiRMnyul0mh0DPm5An0hdPjxJizYe0LOf7tSzPxxtdiQAQCd0aUYoKChIdXV1kqRPPvlEl1xyiSSpV69ebl1cHB8fL5vNpvLytvdzKi8vV2Jiotve52Ty8/OVmZmpnJycbn0f+I6fXtyyt9aijQe062CNyWkAAJ3RpSI0ceJEzZ49W4899phWrlypK664QpK0fft2paSkuC1cUFCQxowZo8WLF7seczqdWrx4sXJzc932Picza9YsFRYWatWqVd36PvAdQ5OilDc0QYYhPffZLrPjAAA6oUtF6Nlnn1VAQIDefPNNPf/88+rbt68k6f3339ell156Rq9VU1OjgoICFRQUSGo57VZQUKDi4mJJ0uzZszVv3jy9+OKL2rJli+666y7V1tbqlltu6Up0oFu1zgotLNivkiN1JqcBAJyO6ZfPf/7555o0aVK7x2fOnKn58+dLailef/zjH1VWVqbs7Gw9/fTTGjduXI/k6+zld0CrGf9YoS92HNLM3HQ9evVws+MAgF/q7Od3l4uQw+HQwoULtWXLFknSsGHDdNVVV8lms3UtsYeiCOFMfbnjkG78xwqFBtr09YMXKzY8yOxIAOB3unUfoZ07d2ro0KG66aabtGDBAi1YsEA33nijhg0bpl27fGNtBIul0VUTBsRpWHKUjjU59K/lRWbHAQCcQpdmhC6//HIZhqGXX35ZvXr1kiQdPnxYN954o6xWqxYtWuT2oGZhRghd8XbBft3zWoHiwoP01YMXKyTQt2ZKAcDTdeuM0JIlS/SHP/zBVYIkKS4uTk888YSWLFnSlZcEfMoVI5LUNyZUh2sb9eaafWbHAQB0oEtFKDg4WNXV1e0er6mpUVAQ6yGAAJtVP5rYT5L0f1/slsPpMbf0AwCcoEtF6Morr9Qdd9yhFStWyDAMGYah5cuX684779RVV13l7oyAV7ohJ1XRoYHae7hOHxd27pYwAICe1aUi9PTTT+ucc85Rbm6uQkJCFBISovPOO08DBgzQU0895eaIgHcKDw7QjPHpkqS/Ld0tk3eqAACcRJfuNRYTE6O3335bO3fudF0+P3ToUA0YMMCt4cyUn5+v/Px8ORwOs6PAi808L0N//2K31hVXanXRUeVk9Dr9HwIA9JhOXzV2JneV/9Of/tTlQJ6Gq8ZwtuYs2KBXV5Yob2iC/m/mWLPjAIBf6Oznd6dnhNatW9ep4ywWS2dfEvALt53fX6+uLNHireUqOlyr9LhwsyMBAI7rdBH67LPPujMH4LPO6R2hiwb31ufbDmr+13v1yNRhZkcCABzXpcXSAM7MrRNaLqV/Y/U+Vdc3mZwGANCKIgT0gPMHxmtAnwjVNDTrP6vZYBEAPAVFqAPcawzuZLFYdMuEDEnSi1/vZYNFAPAQFKEOzJo1S4WFhVq1apXZUeAjvjsqRdGhgSo+UqfFW8rNjgMAEEUI6DGhQTZNOzdNkvTCV3vNDQMAkEQRAnrUTbnpslktWrb7sApL7WbHAQC/RxECelByTKguHZ4oSXrhqz0mpwEAUISAHtZ6Kf3b60t1qKbB5DQA4N8oQkAPG50Wo6yUaDU2O/XKimKz4wCAX6MIdYDL59FdLBaLbp3YMiv0r+VFanI4TU4EAP6LItQBLp9Hd7pseJJ6RwbrYHWDPtxcZnYcAPBbFCHABEEBVk3LSZUkvbSsyOQ0AOC/KEKASX44ruVS+pV7jmhrGZfSA4AZKEKASRKjQ3RJZoIk6V/MCgGAKShCgIlm5KZLkt5at1927koPAD2OIgSYKLd/nAb2iVBdo0ML1nBXegDoaRQhwEQWi8U1K/Sv5UUyDO5KDwA9iSIEmOzaUX0VHmTTroO1+nrXYbPjAIBfoQh1gA0V0VMiQwL13dEpkqSXlu01NwwA+BmLwVz8KdntdkVHR6uqqkpRUVFmx4GP2l5erUv+vFRWi/TlAxcrOSbU7EgA4NU6+/nNjBDgAQYlRGp8/15yGtKrK7n/GAD0FIoQ4CFuys2QJL26skSNzdx/DAB6AkUI8BCTMxOUEBWsQzUNen/TAbPjAIBfoAgBHiLQZtW0c9MksdM0APQUihDgQX54bpoCrBatLjqqbWXVZscBAJ9HEQI8SJ+oEOUNbbn/2CsrmBUCgO5GEQI8zA/HtZweW7Buv441OkxOAwC+jSIEeJiJA+KV1itM1fXNemdDqdlxAMCnUYQAD2O1WvSDc1MlSa+sYE8hAOhOFCHAA31/TKoCrBYVlFSqsNRudhwA8FkUoQ5wrzGYqXdksKYMS5QkvbKSRdMA0F0oQh2YNWuWCgsLtWrVKrOjwE+1LppeuK5UtQ3NJqcBAN9EEQI8VG7/OGXEhammoVnvrGfRNAB0B4oQ4KGsVotrp+lXuBErAHQLihDgwb43JkVBNqs27KvSpv1VZscBAJ9DEQI8WFxEsKYMb1k0/TKX0gOA21GEAA/3w+Onx/5XsF81LJoGALeiCAEebnz/XurfO1y1jQ69XbDf7DgA4FMoQoCHs1gsrlmhV1YUyzAMkxMBgO+gCAFe4LrRKQoKsGpzqV3r97FoGgDchSIEeIHY8CBdMSJJkvTKCnaaBgB3oQgBXqJ1p+l31h+Qvb7J5DQA4BsoQoCXGJseqwF9InSsyaG317FoGgDcgSIEeIkTF02/zKJpAHALihDgRb47uq+CA6zaWlatgpJKs+MAgNejCAFeJCbsm0XTr3L/MQA4axShDuTn5yszM1M5OTlmRwHaYNE0ALgPRagDs2bNUmFhoVatWmV2FKCNMemxGnh80fRCFk0DwFmhCAFexmKxuGaF2GkaAM4ORQjwQt8dleJaNL2ORdMA0GUUIcALRYcF6oqRxxdNr2DRNAB0FUUI8FLTWxdNbyhV1TEWTQNAV1CEAC81Oi1WgxIiVN/k1NsFLJoGgK6gCAFe6sSdplk0DQBdQxECvNi1o79ZNL22uNLsOADgdShCgBeLDg3UlSOTJbHTNAB0BUUI8HKtewq9y6JpADhjFCHAy41Oi9HghEjVNznZaRoAzhBFCPBy7DQNAF1HEQJ8wDWj+iok0Kpt5SyaBoAzQRECfMCJi6ZfYadpAOg0ihDgI9osmq5j0TQAdAZFCPARo1JjNCQxUg3NTr21bp/ZcQDAK1CEAB/RZtH0ShZNA0BnUIQAH3J1dsui6e3lNVpbfNTsOADg8ShCgA+JDg3U1OOLpl9m0TQAnBZFCPAxrafHFm04wKJpADgNihDgY7JPWDS9gEXTAHBKFCHAx1gsFk1np2kA6BSfL0KVlZUaO3assrOzNXz4cM2bN8/sSEC3u3pUX4UF2bSjokbLdh02Ow4AeCyfL0KRkZFaunSpCgoKtGLFCj3++OM6fJgPBvi2qJBAXTc6RZL0wtd7zQ0DAB7M54uQzWZTWFiYJKmhoUGGYXCqAH5h5nkZkqRPtpSr+HCduWEAwEOZXoSWLl2qqVOnKjk5WRaLRQsXLmx3TH5+vjIyMhQSEqJx48Zp5cqVZ/QelZWVysrKUkpKiu6//37Fx8e7KT3guQb0idAFg3rLMKSXlu01Ow4AeCTTi1Btba2ysrKUn59/0udff/11zZ49W4888ojWrl2rrKwsTZkyRRUVFa5jWtf/fPurtLRUkhQTE6P169drz549euWVV1ReXt5hnoaGBtnt9jZfgLe6ZUKGJOn11SWqbWg2NwwAeCCL4UHniSwWi9566y1dc801rsfGjRunnJwcPfvss5Ikp9Op1NRU3X333XrwwQfP+D1+8pOf6OKLL9b3vve9kz7/m9/8Ro8++mi7x6uqqhQVFXXG7weYyek0lPenJdp9qFaPXT1MM3IzzI4EAD3CbrcrOjr6tJ/fps8InUpjY6PWrFmjvLw812NWq1V5eXlatmxZp16jvLxc1dXVklrKzNKlSzV48OAOj58zZ46qqqpcXyUlJWc3CMBEVqvFtVboha/3yun0mP/vAQCP4NFF6NChQ3I4HEpISGjzeEJCgsrKyjr1GkVFRTr//POVlZWl888/X3fffbdGjBjR4fHBwcGKiopq8wV4s+vGpCgyOEC7D9Zq6Y6DZscBAI8SYHaA7nbuueeqoKDA7BiAaSKCA/T9san651d79MJXe3XR4D5mRwIAj+HRM0Lx8fGy2WztFjeXl5crMTHRpFSA95l5XrqsFmnJ9oPacoALAACglUcXoaCgII0ZM0aLFy92PeZ0OrV48WLl5uZ263vn5+crMzNTOTk53fo+QE9IjwvXZSOSJEl/W7LL5DQA4DlML0I1NTUqKChwnb7as2ePCgoKVFxcLEmaPXu25s2bpxdffFFbtmzRXXfdpdraWt1yyy3dmmvWrFkqLCzUqlWruvV9gJ5y14XnSJLe2XBAJUfYYBEAJA9YI7R69WpNmjTJ9f3s2bMlSTNnztT8+fN1ww036ODBg3r44YdVVlam7OxsffDBB+0WUAM4teF9o3X+wHh9seOQ5n2xW7+9erjZkQDAdB61j5An6uw+BIA3+HrXIf1w3goFB1j11YMXKz4i2OxIANAtfGIfIQDulds/TlmpMWpoduqFr/aYHQcATEcR6gCLpeGLLBaLa63QS18XqbKu0eREAGAuilAHWCwNX3VJZoKGJEaquqFZ877YbXYcADAVRQjwM1arRbMnD5IkvfDVXh2uaTA5EQCYhyIE+KHJmQkamRKtukaH/sq+QgD8GEUI8EMWyzezQi8tK1K5vd7kRABgDopQB1gsDV934aDeGpseq4Zmp/744Taz4wCAKdhH6DTYRwi+bF3xUV373NeSpHd+OlEjUqJNTgQA7sE+QgBOa1RarK4d1VeS9Og7m8X/FwHwNxQhwM/98tLBCgm0anXRUb274YDZcQCgR1GEAD+XFB2qO49vsvj/FhWq6liTyYkAoOdQhADozgvPUb/4cJXbG/TE+1vMjgMAPYYi1AGuGoM/CQm0ae53R0iSXl1ZomW7DpucCAB6BleNnQZXjcGf/OqtjXplRbFSYkO16GfnKzo00OxIANAlXDUG4IzNuWyIUnuFat/RY3rwvxu4igyAz6MIAXCJDAnUs9NGK9Bm0fubyvSv5UVmRwKAbkURAtBGVmqMHrh0iCTpsXcL9fWuQyYnAoDuQxEC0M6PJvbTlSOT1OQwdOe/1mhnRbXZkQCgW1CEALRjsVj0/30/S2PSY2Wvb9b0/1uhPYdqzY4FAG5HEQJwUiGBNv19xhgN7BOhcnuDfvD3ZcwMAfA5FKEOsI8QIMVFBOvVO8ZrUEJLGbo2/2t9vq3C7FgA4DbsI3Qa7CMESEdqG/Xjf63Wqr1HZbW07ER9T95ABQfYzI4GACfFPkIA3KZXeJD+fds4/SAnVU5Deu7zXZr6zJf6bFsFew0B8GrMCJ0GM0JAWx9sOqCH3tqkw7WNkqQx6bG6KTddU4YlKiSw52aInE5D1Q3Nsh9rUtWxJtU2NEuSTvyBZlHLWqfQIJtCAmwKCbIqJNCm8KAA2ayWHssKoOd19vObInQaFCGgvcq6RuV/tlMvfl2kRodTkhQWZNPEAfEa3z9OQ5OiNCghQr3Cg2SxdFw4HE7DVWSqjjXJXv/N712PHWv/WFVdk6obmnU2P72iQgIUGx6kmLAgxYYFKiY0UPERwUqMDlFSdOjxX0PUJzJYATYmzwFvQxFyE4oQ0LFye71eW1mi11YV60BVfbvnA6wWxUUEKSTQJpvVIpvFovpmh+oaHKptbFZ9k/OsMwQHWBUdGqjw4AC1dq7W6uU0pIYmh+qbnapvcuhYk+OMy5PVIvWODFZSdKgy4sKUHheujPjjv8aFKzYs8JRlD4A5KEJuQhECTs8wDG0uteuzrRXasL9KWw7Yte/osU7/+bAgm6JDAxUdGqiokEBFHf/9N18Big5r+1jrcWdyOs4wDDU6nKqpb9bRuiZV1jXqaF2TjtY1qrKuUYdqGnWgql5lVcd0oKpe5fZ6NTlO/SMyMiRAGXHhGtAnQgP6RGhQQqQG9olQaq8wTr8BJqIIuQlFCOiahmaHDtc06nBNoxodTjU7nHI4DQUH2hQe3LJOJyzIpsiQQAUFeOapJ6fT0OHaRpVV1Wvf0ToVHalT0eFa7T3U8mvpSWbBWgUHWDWgT4QG9onQwIRIDUmMVGZylBKjQphBAnoARchNKEIAOlLf5FDJkTrtPlSrnRU12l5erR3lNdp5sEaNzSc/7RcbFqjM5CgNS45WZlKUMpOj1D8+nHVIgJtRhM5Sfn6+8vPz5XA4tH37dooQgE5zOA0VH6nT9vJqV0HaeqBaOw/WyOFs/yM3OMDqmjHKTI7WyL7RGpIUyT5NwFmgCLkJM0IA3KW+yaHt5dUqLLWr8IBdhaV2bTlgV22jo92xgTaLhiZFaUTfaGWlxGhkarQG9I5g5gjoJIqQm1CEAHQnp9NQ0ZE6FZbatbm0SptK7dqwr1KVdU3tjg0NtGlYcpRGpsQoKzVaI/pGKyMuXFYWZQPtUITchCIEoKcZhqF9R49p/b5KbdhXpQ37KrVpv101xzeNPFFkSICyUmKUnRqjUWktv8ZFBJuQGvAsFCE3oQgB8AROp6Hdh2q0vqRKG/dXaf2+Sm0utZ90UXZqr1CNSo1VdmqMstNiNCw5ivVG8DsUITehCAHwVE0Op7aVVWv9vkqtK65UQUmldlbUtDsuyGbV0OQojTo+azQ6LVYpsaFcxg+fRhFyE4oQAG9SdaxJG/ZVqqC4UutKWsrRkeP3hTtR78hgjUmL1ej0lmI0vG90j94rDuhuFCE3oQgB8GaGYajkyDGtKzmqdcWVWld8VJtL7Wr+1mX8gTaLhiVHa3RarMaktxSkpOhQk1IDZ48i5CYUIQC+pr7JoQ37qrS2+KjWFh3V2uKjOlTTftYoKTpEo9NiNTo9VqPTYjQsOdpjdwEHvo0i5CYUIQC+rnXWaG1xSylaU3RUW8uq223+GBRg1ci+0a5iNDotVn2iQkxKDZwaRchNKEIA/FFdY7PWl7SdNTp6kr2NUmJDW2aN0mI0Jr2XhiRFKpBNH+EBKEJniVtsAMA3DMPQ3sN1WnO8FK0tOqrt5dX69h1DQgKtGpkS07LO6HhBYl8jmIEi5CbMCAHAyVXXN30za3S8HNnr22/6mBEXptFpsRp1/JTa4IRIbhWCbkcRchOKEAB0Tuumj2uLKl1rjXacZF+j8CCbslJjXFeojUqLUUxYkAmJ4csoQm5CEQKArqs61qSCkkqtKTqqdcVHVVBcqeqT3Cqkf+/w4/satZxSG9gngnuo4axQhNyEIgQA7uNwGtpZUfPNWqPio9p9sLbdcZHBAco6vhN2yz3UYtUrnFkjdB5FyE0oQgDQvY7WNmpdyVHXKbWCkkrVNTraHZcRF6ZRaS2n0kalxnKFGk6JIuQmFCEA6FnNDqe2l9e02Q1710lmjYIDrBqZEt1SjlJjNCotVonR7GuEFhQhN6EIAYD5quqaVLCvpRQVlLTcZLbqWPt9jZKiQ5TtOqUWqxHcQ81vUYTchCIEAJ7HMAztOVTbMmN0fOboZLthB1gtGpoU5VprNCo1VulxYbJYWIjt6yhCbkIRAgDvUNfYrI37qrSupGXmaG1xpQ5WN7Q7LjYssM3ptJGp0YoKCTQhMboTRchNKEIA4J0Mw1BpVb3WFX+z1mjTfrsaHc42x1ks0sA+ERqVGqvs4zNHA/tEysbl+16NIuQmFCEA8B0NzQ5tOVD9TTkqOaqSI8faHde66WPr6bTstBjFc6sQr0IRchOKEAD4toPVDccXYLcsxF5fUqnak1y+nxwdopEpMRqZGq2RfWM0IiVa0aGcUvNUFCE3oQgBgH9xOA3tqKh2nU5bW1ypXQdrdLJPy37x4RrRN1ojU6KVlRqjYclRCgsK6PnQaIci5CYUIQBAdX2TNu23a+P+Sq3fV6UN+ypPekrNapEG9onUyJRojUyN0ci+0RqSFKngAC7h72kUITehCAEATuZobaM27K/ShpLKll/3Varc3v4qtUBbyyX8I/pGK+v4qbUBvSMUwK7Y3Yoi5CYUIQBAZ5Xb67Xh+IxR68xRZV37jR9DA20alhzVsuYoJVojUqLVLy6cG826EUXoLOXn5ys/P18Oh0Pbt2+nCAEAzphhGNp39JjW76t0FaRN++2qaWhud2xYkE2ZSVEalhylYcnRGtY3SgP7RCoogJmjrqAIuQkzQgAAd3I6De0+VHO8GFVp/b5KbTlgV32Ts92xgTaLBiVEalhylIb3jdaw5CgNTWJBdmdQhNyEIgQA6G7NDqf2HKrV5lK7Nu2v0uZSuzaXVsle337myGKR+seHt8wanVCQYsKCTEjuuShCbkIRAgCYofW02ubS1mLUUpIqTnLbEEnqGxOqzOQo16xRZlKUUmJD/fa+ahQhN6EIAQA8ycHqhhPKUcuvRYfrTnpsRHCAhiRGakhSpIYmtRSkwQmRCg/2/VNrFCE3oQgBADydvb5JhcdnjQpL7dpaZteO8pp291WTWk6tpfcK05DEKA1OjNTgxEgNSohURlyYT13STxFyE4oQAMAbNTmc2n2wVlvL7Co8YNfWA9XacsDe4am1IJtV5/SJ0KCECA1KiNTghJaClBIb6pWX9VOE3IQiBADwJYdrGrS1rKUUbS+v1rbyGu0or1bdSe6vJrXsedRajgYlRGpQYktJSogK9uj1RxQhN6EIAQB8ndNpaH/lsePFqFrby1oK0q6DNWpsbn96TZKiQgI0KCFSAxMidU7vcJ3TJ0IDekeob4xnzCBRhNyEIgQA8FfNDqeKjtQdL0bV2lFeo23l1dpzqFYO58nrQ3CAVf17R7SUo94ROqdPy+/7x0coNKjn7rlGEXITihAAAG01NDu0+2CttpdXa2dFy8zRropa7TlUe9IF2q36xoS6itGAPhEtRal3hOIjgtx+mo0i5CYUIQAAOsfhNLTvaJ2rGLWWpJ0Ha056z7VWz0wbpalZyW7N0tnPb9/fSAAAAPQIm9Wi9LhwpceF6+IhbZ87Utt4vCAdn0E6WKtdB2tUcqRO/eLDzQksihAAAOgBvcKD1Cu8l3IyerV5vL7JoUAT9y+iCAEAANOEBPbcAuqT8Z0tJAEAAM4QRQgAAPgtihAAAPBbFCEAAOC3KEIAAMBvUYQAAIDfoggBAAC/5TdFqK6uTunp6frFL35hdhQAAOAh/KYI/e53v9P48ePNjgEAADyIXxShHTt2aOvWrbrsssvMjgIAADyI6UVo6dKlmjp1qpKTk2WxWLRw4cJ2x+Tn5ysjI0MhISEaN26cVq5ceUbv8Ytf/EJz5851U2IAAOArTC9CtbW1ysrKUn5+/kmff/311zV79mw98sgjWrt2rbKysjRlyhRVVFS4jsnOztbw4cPbfZWWlurtt9/WoEGDNGjQoJ4aEgAA8BIWwzAMs0O0slgseuutt3TNNde4Hhs3bpxycnL07LPPSpKcTqdSU1N1991368EHHzzta86ZM0f//ve/ZbPZVFNTo6amJt133316+OGHT3p8Q0ODGhoaXN/b7XalpqaqqqpKUVFRZzdAAADQI+x2u6Kjo0/7+e3Rd59vbGzUmjVrNGfOHNdjVqtVeXl5WrZsWadeY+7cua7TYvPnz9emTZs6LEGtxz/66KPtHrfb7WeYHgAAmKX1c/t08z0eXYQOHTokh8OhhISENo8nJCRo69at3fKec+bM0ezZs13f79+/X5mZmUpNTe2W9wMAAN2nurpa0dHRHT7v0UXI3W6++ebTHhMcHKzg4GDX9xERESopKVFkZKQsFovbsrSecispKfGbU27+NmbG69sYr+/ztzH72ngNw1B1dbWSk5NPeZxHF6H4+HjZbDaVl5e3eby8vFyJiYk9ksFqtSolJaXbXj8qKson/sKdCX8bM+P1bYzX9/nbmH1pvKeaCWpl+lVjpxIUFKQxY8Zo8eLFrsecTqcWL16s3NxcE5MBAABfYPqMUE1NjXbu3On6fs+ePSooKFCvXr2Ulpam2bNna+bMmRo7dqzOPfdcPfXUU6qtrdUtt9xiYmoAAOALTC9Cq1ev1qRJk1zfty5UnjlzpubPn68bbrhBBw8e1MMPP6yysjJlZ2frgw8+aLeA2tsEBwfrkUceabMeydf525gZr29jvL7P38bsb+Nt5VH7CAEAAPQkj14jBAAA0J0oQgAAwG9RhAAAgN+iCAEAAL9FETJJfn6+MjIyFBISonHjxmnlypVmRzpjc+fOVU5OjiIjI9WnTx9dc8012rZtW5tj6uvrNWvWLMXFxSkiIkLXXXdduw0yi4uLdcUVVygsLEx9+vTR/fffr+bm5p4cSpc88cQTslgsuvfee12P+eJ49+/frxtvvFFxcXEKDQ3ViBEjtHr1atfzhmHo4YcfVlJSkkJDQ5WXl6cdO3a0eY0jR45o+vTpioqKUkxMjH70ox+ppqamp4dyWg6HQ7/+9a/Vr18/hYaG6pxzztFjjz3W5l5F3jzepUuXaurUqUpOTpbFYtHChQvbPO+usW3YsEHnn3++QkJClJqaqj/84Q/dPbQOnWrMTU1NeuCBBzRixAiFh4crOTlZN910k0pLS9u8hjeN+XT/jk905513ymKx6KmnnmrzuDeN1y0M9LjXXnvNCAoKMv75z38amzdvNm6//XYjJibGKC8vNzvaGZkyZYrxwgsvGJs2bTIKCgqMyy+/3EhLSzNqampcx9x5551GamqqsXjxYmP16tXG+PHjjfPOO8/1fHNzszF8+HAjLy/PWLdunfHee+8Z8fHxxpw5c8wYUqetXLnSyMjIMEaOHGncc889rsd9bbxHjhwx0tPTjZtvvtlYsWKFsXv3buPDDz80du7c6TrmiSeeMKKjo42FCxca69evN6666iqjX79+xrFjx1zHXHrppUZWVpaxfPly44svvjAGDBhgTJs2zYwhndLvfvc7Iy4uznj33XeNPXv2GG+88YYRERFh/OUvf3Ed483jfe+994yHHnrIWLBggSHJeOutt9o8746xVVVVGQkJCcb06dONTZs2Ga+++qoRGhpq/O1vf+upYbZxqjFXVlYaeXl5xuuvv25s3brVWLZsmXHuuecaY8aMafMa3jTm0/07brVgwQIjKyvLSE5ONv785z+3ec6bxusOFCETnHvuucasWbNc3zscDiM5OdmYO3euianOXkVFhSHJWLJkiWEYLT9kAgMDjTfeeMN1zJYtWwxJxrJlywzDaPmP1mq1GmVlZa5jnn/+eSMqKspoaGjo2QF0UnV1tTFw4EDj448/Ni688EJXEfLF8T7wwAPGxIkTO3ze6XQaiYmJxh//+EfXY5WVlUZwcLDx6quvGoZhGIWFhYYkY9WqVa5j3n//fcNisRj79+/vvvBdcMUVVxi33nprm8e++93vGtOnTzcMw7fG++0PSXeN7bnnnjNiY2Pb/H1+4IEHjMGDB3fziE7vVMWg1cqVKw1JRlFRkWEY3j3mjsa7b98+o2/fvsamTZuM9PT0NkXIm8fbVZwa62GNjY1as2aN8vLyXI9ZrVbl5eVp2bJlJiY7e1VVVZKkXr16SZLWrFmjpqamNmMdMmSI0tLSXGNdtmyZRowY0WaDzClTpshut2vz5s09mL7zZs2apSuuuKLNuCTfHO///vc/jR07Vt///vfVp08fjRo1SvPmzXM9v2fPHpWVlbUZc3R0tMaNG9dmzDExMRo7dqzrmLy8PFmtVq1YsaLnBtMJ5513nhYvXqzt27dLktavX68vv/xSl112mSTfG++J3DW2ZcuW6YILLlBQUJDrmClTpmjbtm06evRoD42m66qqqmSxWBQTEyPJ98bsdDo1Y8YM3X///Ro2bFi7531tvJ1BEephhw4dksPhaLczdkJCgsrKykxKdfacTqfuvfdeTZgwQcOHD5cklZWVKSgoyPUDpdWJYy0rKzvpP4vW5zzNa6+9prVr12ru3LntnvPF8e7evVvPP/+8Bg4cqA8//FB33XWXfvazn+nFF1+U9E3mU/19LisrU58+fdo8HxAQoF69enncmB988EH94Ac/0JAhQxQYGKhRo0bp3nvv1fTp0yX53nhP5K6xedvf8RPV19frgQce0LRp01w3HfW1Mf/+979XQECAfvazn530eV8bb2eYfosN+IZZs2Zp06ZN+vLLL82O0m1KSkp0zz336OOPP1ZISIjZcXqE0+nU2LFj9fjjj0uSRo0apU2bNumvf/2rZs6caXI69/vPf/6jl19+Wa+88oqGDRumgoIC3XvvvUpOTvbJ8eIbTU1Nuv7662UYhp5//nmz43SLNWvW6C9/+YvWrl0ri8VidhyPwYxQD4uPj5fNZmt3JVF5ebkSExNNSnV2fvrTn+rdd9/VZ599ppSUFNfjiYmJamxsVGVlZZvjTxxrYmLiSf9ZtD7nSdasWaOKigqNHj1aAQEBCggI0JIlS/T0008rICBACQkJPjVeSUpKSlJmZmabx4YOHari4mJJ32Q+1d/nxMREVVRUtHm+ublZR44c8bgx33///a5ZoREjRmjGjBn6+c9/7poB9LXxnshdY/O2v+PSNyWoqKhIH3/8sWs2SPKtMX/xxReqqKhQWlqa62dYUVGR7rvvPmVkZEjyrfF2FkWohwUFBWnMmDFavHix6zGn06nFixcrNzfXxGRnzjAM/fSnP9Vbb72lTz/9VP369Wvz/JgxYxQYGNhmrNu2bVNxcbFrrLm5udq4cWOb//BafxB9+wPYbN/5zne0ceNGFRQUuL7Gjh2r6dOnu37vS+OVpAkTJrTbEmH79u1KT0+XJPXr10+JiYltxmy327VixYo2Y66srNSaNWtcx3z66adyOp0aN25cD4yi8+rq6mS1tv2xaLPZ5HQ6JfneeE/krrHl5uZq6dKlampqch3z8ccfa/DgwYqNje2h0XReawnasWOHPvnkE8XFxbV53pfGPGPGDG3YsKHNz7Dk5GTdf//9+vDDDyX51ng7zezV2v7otddeM4KDg4358+cbhYWFxh133GHExMS0uZLIG9x1111GdHS08fnnnxsHDhxwfdXV1bmOufPOO420tDTj008/NVavXm3k5uYaubm5rudbLye/5JJLjIKCAuODDz4wevfu7bGXk3/biVeNGYbvjXflypVGQECA8bvf/c7YsWOH8fLLLxthYWHGv//9b9cxTzzxhBETE2O8/fbbxoYNG4yrr776pJdcjxo1ylixYoXx5ZdfGgMHDvSIy8m/bebMmUbfvn1dl88vWLDAiI+PN375y1+6jvHm8VZXVxvr1q0z1q1bZ0gy/vSnPxnr1q1zXSHljrFVVlYaCQkJxowZM4xNmzYZr732mhEWFmbapdWnGnNjY6Nx1VVXGSkpKUZBQUGbn2MnXhHlTWM+3b/jb/v2VWOG4V3jdQeKkEmeeeYZIy0tzQgKCjLOPfdcY/ny5WZHOmOSTvr1wgsvuI45duyY8ZOf/MSIjY01wsLCjGuvvdY4cOBAm9fZu3evcdlllxmhoaFGfHy8cd999xlNTU09PJqu+XYR8sXxvvPOO8bw4cON4OBgY8iQIcbf//73Ns87nU7j17/+tZGQkGAEBwcb3/nOd4xt27a1Oebw4cPGtGnTjIiICCMqKsq45ZZbjOrq6p4cRqfY7XbjnnvuMdLS0oyQkBCjf//+xkMPPdTmQ9Gbx/vZZ5+d9L/ZmTNnGobhvrGtX7/emDhxohEcHGz07dvXeOKJJ3pqiO2casx79uzp8OfYZ5995noNbxrz6f4df9vJipA3jdcdLIZxwpapAAAAfoQ1QgAAwG9RhAAAgN+iCAEAAL9FEQIAAH6LIgQAAPwWRQgAAPgtihAAAPBbFCEAAOC3KEIAcAY+//xzWSyWdjfXBeCdKEIAAMBvUYQAAIDfoggB8CpOp1Nz585Vv379FBoaqqysLL355puSvjlttWjRIo0cOVIhISEaP368Nm3a1OY1/vvf/2rYsGEKDg5WRkaGnnzyyTbPNzQ06IEHHlBqaqqCg4M1YMAA/eMf/2hzzJo1azR27FiFhYXpvPPO07Zt27p34AC6BUUIgFeZO3euXnrpJf31r3/V5s2b9fOf/1w33nijlixZ4jrm/vvv15NPPqlVq1apd+/emjp1qpqamiS1FJjrr79eP/jBD7Rx40b95je/0a9//WvNnz/f9edvuukmvfrqq3r66ae1ZcsW/e1vf1NERESbHA899JCefPJJrV69WgEBAbr11lt7ZPwA3Iu7zwPwGg0NDerVq5c++eQT5ebmuh6/7bbbVFdXpzvuuEOTJk3Sa6+9phtuuEGSdOTIEaWkpGj+/Pm6/vrrNX36dB08eFAfffSR68//8pe/1KJFi7R582Zt375dgwcP1scff6y8vLx2GT7//HNNmjRJn3zyib7zne9Ikt577z1dccUVOnbsmEJCQrr5nwIAd2JGCIDX2Llzp+rq6jR58mRFRES4vl566SXt2rXLddyJJalXr14aPHiwtmzZIknasmWLJkyY0OZ1J0yYoB07dsjhcKigoEA2m00XXnjhKbOMHDnS9fukpCRJUkVFxVmPEUDPCjA7AAB0Vk1NjSRp0aJF6tu3b5vngoOD25ShrgoNDe3UcYGBga7fWywWSS3rlwB4F2aEAHiNzMxMBQcHq7i4WAMGDGjzlZqa6jpu+fLlrt8fPXpU27dv19ChQyVJQ4cO1VdffdXmdb/66isNGjRINptNI0aMkNPpbLPmCIDvYkYIgNeIjIzUL37xC/385z+X0+nUxIkTVVVVpa+++kpRUVFKT0+XJP32t79VXFycEhIS9NBDDyk+Pl7XXHONJOm+++5TTk6OHnvsMd1www1atmyZnn32WT333HOSpIyMDM2cOVO33nqrnn76aWVlZamoqEgVFRW6/vrrzRo6gG5CEQLgVR577DH17t1bc+fO1e7duxUTE6PRo0frV7/6levU1BNPPKF77rlHO3bsUHZ2tt555x0FBQVJkkaPHq3//Oc/evjhh/XYY48pKSlJv/3tb3XzzTe73uP555/Xr371K/3kJz/R4cOHlZaWpl/96ldmDBdAN+OqMQA+o/WKrqNHjyomJsbsOAC8AGuEAACA36IIAQAAv8WpMQAA4LeYEQIAAH6LIgQAAPwWRQgAAPgtihAAAPBbFCEAAOC3KEIAAMBvUYQAAIDfoggBAAC/9f8DBkdXbJZxIMEAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "list_ = [\n", " idx for idx, s in enumerate(trainer.callbacks)\n", " if isinstance(s, MetricTracker)\n", " ]\n", "print(list_[0])\n", "trainer_metrics = trainer.callbacks[list_[0]].metrics\n", "\n", "loss = trainer_metrics['val_loss']\n", "epochs = range(len(loss))\n", "plt.plot(epochs, loss.cpu())\n", "# plotting\n", "plt.xlabel('epoch')\n", "plt.ylabel('loss')\n", "plt.yscale('log')" ] }, { "cell_type": "markdown", "id": "58172899", "metadata": {}, "source": [ "As we can see the loss has not reached a minimum, suggesting that we could train for longer" ] }, { "cell_type": "markdown", "id": "33e672da", "metadata": {}, "source": [ "## What's next?\n", "\n", "Congratulations on completing the introductory tutorial of **PINA**! There are several directions you can go now:\n", "\n", "1. Train the network for longer or with different layer sizes and assert the finaly accuracy\n", "\n", "2. Train the network using other types of models (see `pina.model`)\n", "\n", "3. GPU training and speed benchmarking\n", "\n", "4. Many more..." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }