folivo/PINA
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update Tutorials (#544)

Browse Source 
* update tutorials
* tutorial guidelines
* doc
This commit is contained in:
Dario Coscia
2025-04-23 16:19:07 +02:00
parent 7e403acf58
commit 29b14ee9b6
45 changed files with 6279 additions and 6726 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/source/_tutorial.rst
View File

@@ -1,35 +1,43 @@
PINA Tutorials
======================
🚀 Welcome to the PINA Tutorials!
==================================
In this folder we collect useful tutorials in order to understand the principles and the potential of **PINA**.
Whether you're just getting started or looking to deepen your understanding, these resources are here to guide you.
Getting started with PINA
-------------------------
- `Introduction to PINA for Physics Informed Neural Networks training <tutorial1/tutorial.html>`_
- `Introductory Tutorial: A Beginner's Guide to PINA <tutorial17/tutorial.html>`_
- `How to build a Problem in PINA <tutorial16/tutorial.html>`_
- `Introduction to Solver classes <tutorial18/tutorial.html>`_
- `Introduction to Trainer class <tutorial11/tutorial.html>`_
- `Data structure for SciML: Tensor, LabelTensor, Data and Graph <tutorial19/tutorial.html>`_
- `Building geometries with DomainInterface class <tutorial6/tutorial.html>`_
- `Introduction to PINA Equation class <tutorial12/tutorial.html>`_
- `PINA and PyTorch Lightning, training tips and visualizations <tutorial11/tutorial.html>`_
- `Building custom geometries with PINA Location class <tutorial6/tutorial.html>`_
Physics Informed Neural Networks
--------------------------------
- `Two dimensional Poisson problem using Extra Features Learning <tutorial2/tutorial.html>`_
- `Two dimensional Wave problem with hard constraint <tutorial3/tutorial.html>`_
- `Resolution of a 2D Poisson inverse problem <tutorial7/tutorial.html>`_
- `Periodic Boundary Conditions for Helmotz Equation <tutorial9/tutorial.html>`_
- `Multiscale PDE learning with Fourier Feature Network <tutorial13/tutorial.html>`_
- `Introductory Tutorial: Physics Informed Neural Networks with PINA <tutorial1/tutorial.html>`_
- `Enhancing PINNs with Extra Features to solve the Poisson Problem <tutorial2/tutorial.html>`_
- `Applying Hard Constraints in PINNs to solve the Wave Problem <tutorial3/tutorial.html>`_
- `Applying Periodic Boundary Conditions in PINNs to solve the Helmotz Problem <tutorial9/tutorial.html>`_
- `Inverse Problem Solving with Physics-Informed Neural Network <tutorial7/tutorial.html>`_
- `Learning Multiscale PDEs Using Fourier Feature Networks <tutorial13/tutorial.html>`_
- `Learning Bifurcating PDE Solutions with Physics-Informed Deep Ensembles <tutorial14/tutorial.html>`_
Neural Operator Learning
------------------------
- `Two dimensional Darcy flow using the Fourier Neural Operator <tutorial5/tutorial.html>`_
- `Time dependent Kuramoto Sivashinsky equation using the Averaging Neural Operator <tutorial10/tutorial.html>`_
- `Introductory Tutorial: Neural Operator Learning with PINA <tutorial21/tutorial.html>`_
- `Modeling 2D Darcy Flow with the Fourier Neural Operator <tutorial5/tutorial.html>`_
- `Solving the Kuramoto-Sivashinsky Equation with Averaging Neural Operator <tutorial10/tutorial.html>`_
Supervised Learning
-------------------
- `Unstructured convolutional autoencoder via continuous convolution <tutorial4/tutorial.html>`_
- `POD-RBF and POD-NN for reduced order modeling <tutorial8/tutorial.html>`_
- `Introductory Tutorial: Supervised Learning with PINA <tutorial20/tutorial.html>`_
- `Chemical Properties Prediction with Graph Neural Networks <tutorial25/tutorial.html>`_
- `Unstructured Convolutional Autoencoders with Continuous Convolution <tutorial4/tutorial.html>`_
- `Reduced Order Modeling with POD-RBF and POD-NN Approaches for Fluid Dynamics <tutorial8/tutorial.html>`_

43
tutorials/README.md vendored
View File

@@ -1,36 +1,47 @@
# PINA Tutorials
# 🚀 Welcome to the PINA Tutorials!
In this folder we collect useful tutorials in order to understand the principles and the potential of **PINA**. Whether you're just getting started or looking to deepen your understanding, these resources are here to guide you.
The table below provides an overview of each tutorial. All tutorials are also available in HTML in the official [PINA documentation](http://mathlab.github.io/PINA/).
In this folder we collect useful tutorials in order to understand the principles and the potential of **PINA**. Please read the following table for details about the tutorials. The HTML version of all the tutorials is available also within the [documentation](http://mathlab.github.io/PINA/).
## Getting started with PINA
| Description | Tutorial |
|---------------|-----------|
Introduction to PINA for Physics Informed Neural Networks training|[[.ipynb](tutorial1/tutorial.ipynb),&#160;[.py](tutorial1/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial1/tutorial.html)]|
Introduction to PINA `Equation` class|[[.ipynb](tutorial12/tutorial.ipynb),&#160;[.py](tutorial12/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial12/tutorial.html)]|
PINA and PyTorch Lightning, training tips and visualizations|[[.ipynb](tutorial11/tutorial.ipynb),&#160;[.py](tutorial11/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial11/tutorial.html)]|
Building custom geometries with PINA `Location` class|[[.ipynb](tutorial6/tutorial.ipynb),&#160;[.py](tutorial6/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial6/tutorial.html)]|
Introductory Tutorial: A Beginner’s Guide to PINA|[[.ipynb](tutorial17/tutorial.ipynb),[.py](tutorial17/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial17/tutorial.html)]|
How to build a `Problem` in PINA|[[.ipynb](tutorial16/tutorial.ipynb),[.py](tutorial16/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial16/tutorial.html)]|
Introduction to Solver classes|[[.ipynb](tutorial18/tutorial.ipynb),[.py](tutorial18/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial18/tutorial.html)]|
Introduction to `Trainer` class|[[.ipynb](tutorial11/tutorial.ipynb),[.py](tutorial11/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial11/tutorial.html)]|
Data structure for SciML: `Tensor`, `LabelTensor`, `Data` and `Graph` |[[.ipynb](tutorial19/tutorial.ipynb),[.py](tutorial19/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial19/tutorial.html)]|
Building geometries with `DomainInterface` class|[[.ipynb](tutorial6/tutorial.ipynb),[.py](tutorial6/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial6/tutorial.html)]|
Introduction to PINA `Equation` class|[[.ipynb](tutorial12/tutorial.ipynb),[.py](tutorial12/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial12/tutorial.html)]|
## Physics Informed Neural Networks
| Description | Tutorial |
|---------------|-----------|
Two dimensional Poisson problem using Extra Features Learning &nbsp; &nbsp; |[[.ipynb](tutorial2/tutorial.ipynb),&#160;[.py](tutorial2/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial2/tutorial.html)]|
Two dimensional Wave problem with hard constraint |[[.ipynb](tutorial3/tutorial.ipynb),&#160;[.py](tutorial3/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial3/tutorial.html)]|
Resolution of a 2D Poisson inverse problem |[[.ipynb](tutorial7/tutorial.ipynb),&#160;[.py](tutorial7/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial7/tutorial.html)]|
Periodic Boundary Conditions for Helmotz Equation |[[.ipynb](tutorial9/tutorial.ipynb),&#160;[.py](tutorial9/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial9/tutorial.html)]|
Multiscale PDE learning with Fourier Feature Network |[[.ipynb](tutorial13/tutorial.ipynb),&#160;[.py](tutorial13/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial13/tutorial.html)]|
Introductory Tutorial: Physics Informed Neural Networks with PINA |[[.ipynb](tutorial1/tutorial.ipynb),[.py](tutorial1/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial1/tutorial.html)]|
Enhancing PINNs with Extra Features to solve the Poisson Problem |[[.ipynb](tutorial2/tutorial.ipynb),[.py](tutorial2/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial2/tutorial.html)]|
Applying Hard Constraints in PINNs to solve the Wave Problem |[[.ipynb](tutorial3/tutorial.ipynb),[.py](tutorial3/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial3/tutorial.html)]|
Applying Periodic Boundary Conditions in PINNs to solve the Helmotz Problem |[[.ipynb](tutorial9/tutorial.ipynb),[.py](tutorial9/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial9/tutorial.html)]|
Inverse Problem Solving with Physics-Informed Neural Network |[[.ipynb](tutorial7/tutorial.ipynb),[.py](tutorial7/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial7/tutorial.html)]|
Learning Multiscale PDEs Using Fourier Feature Networks|[[.ipynb](tutorial13/tutorial.ipynb),[.py](tutorial13/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial13/tutorial.html)]|
Learning Bifurcating PDE Solutions with Physics-Informed Deep Ensembles|[[.ipynb](tutorial14/tutorial.ipynb),[.py](tutorial14/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial14/tutorial.html)]|
## Neural Operator Learning
| Description | Tutorial |
|---------------|-----------|
Two dimensional Darcy flow using the Fourier Neural Operator &nbsp; &nbsp; &nbsp;&nbsp; &nbsp;|[[.ipynb](tutorial5/tutorial.ipynb),&#160;[.py](tutorial5/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial5/tutorial.html)]|
Time dependent Kuramoto Sivashinsky equation using the Averaging Neural Operator &nbsp; &nbsp; &nbsp;&nbsp; &nbsp;|[[.ipynb](tutorial10/tutorial.ipynb),&#160;[.py](tutorial10/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial10/tutorial.html)]|
Introductory Tutorial: Neural Operator Learning with PINA |[[.ipynb](tutorial21/tutorial.ipynb),[.py](tutorial21/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial21/tutorial.html)]|
Modeling 2D Darcy Flow with the Fourier Neural Operator |[[.ipynb](tutorial5/tutorial.ipynb),[.py](tutorial5/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial5/tutorial.html)]|
Solving the Kuramoto–Sivashinsky Equation with Averaging Neural Operator |[[.ipynb](tutorial10/tutorial.ipynb),[.py](tutorial10/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial10/tutorial.html)]|
## Supervised Learning
| Description | Tutorial |
|---------------|-----------|
Unstructured convolutional autoencoder via continuous convolution |[[.ipynb](tutorial4/tutorial.ipynb),&#160;[.py](tutorial4/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial4/tutorial.html)]|
POD-RBF and POD-NN for reduced order modeling| [[.ipynb](tutorial8/tutorial.ipynb),&#160;[.py](tutorial8/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial8/tutorial.html)]|
POD-RBF for modelling Lid Cavity| [[.ipynb](tutorial14/tutorial.ipynb),&#160;[.py](tutorial14/tutorial.py),&#160;[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial14/tutorial.html)]|
Introductory Tutorial: Supervised Learning with PINA |[[.ipynb](tutorial20/tutorial.ipynb),[.py](tutorial20/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial20/tutorial.html)]|
Chemical Properties Prediction with Graph Neural Networks |[[.ipynb](tutorial15/tutorial.ipynb),[.py](tutorial15/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial15/tutorial.html)]|
Unstructured Convolutional Autoencoders with Continuous Convolution |[[.ipynb](tutorial4/tutorial.ipynb),[.py](tutorial4/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial4/tutorial.html)]|
Reduced Order Modeling with POD-RBF and POD-NN Approaches for Fluid Dynamics| [[.ipynb](tutorial8/tutorial.ipynb),[.py](tutorial8/tutorial.py),[.html](http://mathlab.github.io/PINA/_rst/tutorials/tutorial8/tutorial.html)]|

128
tutorials/TUTORIAL_GUIDELINES.md vendored Normal file
View File

@@ -0,0 +1,128 @@
# PINA Tutorial Guidelines
Welcome to the **PINA Tutorial Guidelines** — a guiding document that defines the structure, style, and pedagogical philosophy for all tutorials in the **PINA** package. The goal of this guideline is to ensure that all learning materials are **clear, consistent, pedagogically sound, and beginner-friendly**, while remaining powerful enough to support advanced use cases.
## Purpose
The purpose of the PINA tutorials is to help users:
- Gaining a solid understanding of the PINA library and its core functionalities.
- Learning how to work with the PINA modules.
- Explore practical and advanced applications using consistent, hands-on code examples.
## Guiding Principles
1. **Clarity Over Cleverness**
Tutorials should aim to teach, not impress. Prioritize readable and understandable code and explanations.
2. **Progressive Disclosure of Complexity**
Start simple and gradually introduce complexity. Avoid overwhelming users early on.
3. **Consistency is Key**
All tutorials should follow a common structure (see below), use the same markdown and code formatting, and have a predictable flow.
4. **Real Applications, Real Problems**
Ground tutorials in real Scientific Applications or datasets, wherever possible. Bridge theory and implementation.
## Tutorial Structure
To ensure clarity, consistency, and accessibility, all PINA tutorials should follow the same standardized format.
### 1. Title
Each tutorial must begin with a clear and descriptive title in the following format: **Tutorial: TUTORIAL_TITLE**. The title should succinctly communicate the focus and objective of the tutorial.
### 2. Introducing the Topic
Immediately after the title, include a short introduction that outlines the tutorial's purpose and scope.
- Briefly explain what the tutorial covers and why it’s useful.
- Link to relevant research papers, publications, or external resources if applicable.
- List the core PINA components or modules that will be utilized.
### 3. Imports and Setup
Include a Python code cell with the necessary setup. This ensures that the tutorial runs both locally and on platforms like Google Colab.
```python
## Routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
!pip install "pina-mathlab[tutorial]"
import torch # if used
import matplotlib.pyplot as plt # if used
import warnings # if needed
warnings.filterwarnings("ignore")
# Additional PINA and problem-specific imports
...
```
### 3. Data Generation or Loading
* Describe how the data is generated or loaded.
* Include commentary on data structure, format, and content.
* If applicable, visualize key features of the dataset or simulation domain.
### 4. Main Body
The core section of the tutorial should present the problem-solving process in a clear, structured, and pedagogical way. This is where the tutorial delivers the key learning objectives.
- Guide the user step-by-step through the PINA workflow.
- Introduce relevant PINA components as they are used.
- Provide context and explain the rationale behind modeling decisions.
- Break down complex sections with inline comments and markdown explanations.
- Emphasize the relevance of each step to the broader goal of the tutorial.
### 5. Results, Visualization and Error Analysis
- Show relevant plots of results (e.g., predicted vs. ground truth).
- Quantify performance using metrics like loss or relative error.
- Discuss the outcomes: strengths, limitations, and any unexpected behavior
### 6. What's Next?
All the tutorials are concluded with the **What's Next?** section,giving suggestions for further exploration. For this use the following format:
```markdown
## What's Next?
Congratulations on completing the ..., here are a few directions you can explore:
1. **Direction 1** — Suggestion ....
2. **Direction 2** — Suggestion ....
3. **...and many more!** — Other suggestions ....
For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/).
```
## Writing Style
- Use **clear markdown headers** to segment sections.
- Include **inline math** with `$...$` and display math with `$$...$$`.
- Keep paragraphs short and focused.
- Use **bold** and *italic* for emphasis and structure.
- Include comments in code for clarity.
## Testing Tutorials
Every tutorial should:
- Be executable from top to bottom.
- Use the `tutorial` requirements in the [`pyproject.toml`](https://github.com/mathLab/PINA/blob/6ed3ca04fee3ae3673d53ea384437ce270f008da/pyproject.toml#L40) file.
## Contributing Checklist
We welcome contributions! If you’re writing a tutorial:
1. The tutorial follows this guidelines for structure and tone.
2. The tutorial is simple and modular — one tutorial per concept.
3. The tutorial PRs contains only the `.ipynb` file, and the updated `README.md` file.

BIN
tutorials/static/API_color.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 121 KiB

BIN
tutorials/static/deep_ensemble.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 128 KiB

BIN
tutorials/static/logging.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 231 KiB

BIN
tutorials/static/neural_operator.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 35 KiB

BIN
tutorials/static/pina_logo.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 51 KiB

BIN
tutorials/static/pina_wokflow.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 442 KiB

545
tutorials/tutorial1/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

340
tutorials/tutorial1/tutorial.py vendored
View File

@@ -1,340 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Physics Informed Neural Networks on PINA
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial1/tutorial.ipynb)
#
# In this tutorial, we will demonstrate a typical use case of **PINA** on a toy problem, following the standard API procedure.
#
# <p align="center">
# <img src="../../readme/API_color.png" alt="PINA API" width="400"/>
# </p>
#
# Specifically, the tutorial aims to introduce the following topics:
#
# * Explaining how to build **PINA** Problems,
# * Showing how to generate data for `PINN` training
#
# These are the two main steps needed **before** starting the modelling optimization (choose model and solver, and train). We will show each step in detail, and at the end, we will solve a simple Ordinary Differential Equation (ODE) problem using the `PINN` solver.
# ## Build a PINA problem
# Problem definition in the **PINA** framework is done by building a python `class`, which inherits from one or more problem classes (`SpatialProblem`, `TimeDependentProblem`, `ParametricProblem`, ...) depending on the nature of the problem. Below is an example:
# ### Simple Ordinary Differential Equation
# Consider the following:
#
# $$
# \begin{equation}
# \begin{cases}
# \frac{d}{dx}u(x) &= u(x) \quad x\in(0,1)\\
# u(x=0) &= 1 \\
# \end{cases}
# \end{equation}
# $$
#
# with the analytical solution $u(x) = e^x$. In this case, our ODE depends only on the spatial variable $x\in(0,1)$ , meaning that our `Problem` class is going to be inherited from the `SpatialProblem` class:
#
# ```python
# from pina.problem import SpatialProblem
# from pina.domain import CartesianProblem
#
# class SimpleODE(SpatialProblem):
#
# output_variables = ['u']
# spatial_domain = CartesianProblem({'x': [0, 1]})
#
# # other stuff ...
# ```
#
# Notice that we define `output_variables` as a list of symbols, indicating the output variables of our equation (in this case only $u$), this is done because in **PINA** the `torch.Tensor`s are labelled, allowing the user maximal flexibility for the manipulation of the tensor. The `spatial_domain` variable indicates where the sample points are going to be sampled in the domain, in this case $x\in[0,1]$.
#
# What if our equation is also time-dependent? In this case, our `class` will inherit from both `SpatialProblem` and `TimeDependentProblem`:
#
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import warnings
from pina.problem import SpatialProblem, TimeDependentProblem
from pina.domain import CartesianDomain
warnings.filterwarnings("ignore")
class TimeSpaceODE(SpatialProblem, TimeDependentProblem):
output_variables = ["u"]
spatial_domain = CartesianDomain({"x": [0, 1]})
temporal_domain = CartesianDomain({"t": [0, 1]})
# other stuff ...
# where we have included the `temporal_domain` variable, indicating the time domain wanted for the solution.
#
# In summary, using **PINA**, we can initialize a problem with a class which inherits from different base classes: `SpatialProblem`, `TimeDependentProblem`, `ParametricProblem`, and so on depending on the type of problem we are considering. Here are some examples (more on the official documentation):
# * ``SpatialProblem`` $\rightarrow$ a differential equation with spatial variable(s) ``spatial_domain``
# * ``TimeDependentProblem`` $\rightarrow$ a time-dependent differential equation with temporal variable(s) ``temporal_domain``
# * ``ParametricProblem`` $\rightarrow$ a parametrized differential equation with parametric variable(s) ``parameter_domain``
# * ``AbstractProblem`` $\rightarrow$ any **PINA** problem inherits from here
# ### Write the problem class
#
# Once the `Problem` class is initialized, we need to represent the differential equation in **PINA**. In order to do this, we need to load the **PINA** operators from `pina.operator` module. Again, we'll consider Equation (1) and represent it in **PINA**:
# In[ ]:
import torch
import matplotlib.pyplot as plt
from pina.problem import SpatialProblem
from pina.operator import grad
from pina import Condition
from pina.domain import CartesianDomain
from pina.equation import Equation, FixedValue
# defining the ode equation
def ode_equation(input_, output_):
# computing the derivative
u_x = grad(output_, input_, components=["u"], d=["x"])
# extracting the u input variable
u = output_.extract(["u"])
# calculate the residual and return it
return u_x - u
class SimpleODE(SpatialProblem):
output_variables = ["u"]
spatial_domain = CartesianDomain({"x": [0, 1]})
domains = {
"x0": CartesianDomain({"x": 0.0}),
"D": CartesianDomain({"x": [0, 1]}),
}
# conditions to hold
conditions = {
"bound_cond": Condition(domain="x0", equation=FixedValue(1.0)),
"phys_cond": Condition(domain="D", equation=Equation(ode_equation)),
}
# defining the true solution
def solution(self, pts):
return torch.exp(pts.extract(["x"]))
problem = SimpleODE()
# After we define the `Problem` class, we need to write different class methods, where each method is a function returning a residual. These functions are the ones minimized during PINN optimization, given the initial conditions. For example, in the domain $[0,1]$, the ODE equation (`ode_equation`) must be satisfied. We represent this by returning the difference between subtracting the variable `u` from its gradient (the residual), which we hope to minimize to 0. This is done for all conditions. Notice that we do not pass directly a `python` function, but an `Equation` object, which is initialized with the `python` function. This is done so that all the computations and internal checks are done inside **PINA**.
#
# Once we have defined the function, we need to tell the neural network where these methods are to be applied. To do so, we use the `Condition` class. In the `Condition` class, we pass the location points and the equation we want minimized on those points (other possibilities are allowed, see the documentation for reference).
#
# Finally, it's possible to define a `solution` function, which can be useful if we want to plot the results and see how the real solution compares to the expected (true) solution. Notice that the `solution` function is a method of the `PINN` class, but it is not mandatory for problem definition.
#
# ## Generate data
#
# Data for training can come in form of direct numerical simulation results, or points in the domains. In case we perform unsupervised learning, we just need the collocation points for training, i.e. points where we want to evaluate the neural network. Sampling point in **PINA** is very easy, here we show three examples using the `.discretise_domain` method of the `AbstractProblem` class.
# In[ ]:
# sampling 20 points in [0, 1] through discretization in all locations
problem.discretise_domain(n=20, mode="grid", domains="all")
# sampling 20 points in (0, 1) through latin hypercube sampling in D, and 1 point in x0
problem.discretise_domain(n=20, mode="latin", domains=["D"])
problem.discretise_domain(n=1, mode="random", domains=["x0"])
# sampling 20 points in (0, 1) randomly
problem.discretise_domain(n=20, mode="random")
# We are going to use latin hypercube points for sampling. We need to sample in all the conditions domains. In our case we sample in `D` and `x0`.
# In[ ]:
# sampling for training
problem.discretise_domain(1, "random", domains=["x0"])
problem.discretise_domain(20, "lh", domains=["D"])
# The points are saved in a python `dict`, and can be accessed by calling the attribute `input_pts` of the problem
# In[ ]:
print("Input points:", problem.discretised_domains)
print("Input points labels:", problem.discretised_domains["D"].labels)
# To visualize the sampled points we can use `matplotlib.pyplot`:
# In[ ]:
for location in problem.input_pts:
coords = (
problem.input_pts[location].extract(problem.spatial_variables).flatten()
)
plt.scatter(coords, torch.zeros_like(coords), s=10, label=location)
plt.legend()
# ## Perform a small training
# Once we have defined the problem and generated the data we can start the modelling. Here we will choose a `FeedForward` neural network available in `pina.model`, and we will train using the `PINN` solver from `pina.solver`. We highlight that this training is fairly simple, for more advanced stuff consider the tutorials in the ***Physics Informed Neural Networks*** section of ***Tutorials***. For training we use the `Trainer` class from `pina.trainer`. Here we show a very short training and some method for plotting the results. Notice that by default all relevant metrics (e.g. MSE error during training) are going to be tracked using a `lightning` logger, by default `CSVLogger`. If you want to track the metric by yourself without a logger, use `pina.callback.MetricTracker`.
# In[ ]:
from pina import Trainer
from pina.solver import PINN
from pina.model import FeedForward
from lightning.pytorch.loggers import TensorBoardLogger
from pina.optim import TorchOptimizer
# build the model
model = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
# create the PINN object
pinn = PINN(problem, model, TorchOptimizer(torch.optim.Adam, lr=0.005))
# create the trainer
trainer = Trainer(
solver=pinn,
max_epochs=1500,
logger=TensorBoardLogger("tutorial_logs"),
accelerator="cpu",
train_size=1.0,
test_size=0.0,
val_size=0.0,
enable_model_summary=False,
) # we train on CPU and avoid model summary at beginning of training (optional)
# train
trainer.train()
# After the training we can inspect trainer logged metrics (by default **PINA** logs mean square error residual loss). The logged metrics can be accessed online using one of the `Lightning` loggers. The final loss can be accessed by `trainer.logged_metrics`
# In[27]:
# inspecting final loss
trainer.logged_metrics
# By using `matplotlib` we can also do some qualitative plots of the solution.
# In[ ]:
pts = pinn.problem.spatial_domain.sample(256, "grid", variables="x")
predicted_output = pinn.forward(pts).extract("u").tensor.detach()
true_output = pinn.problem.solution(pts).detach()
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(8, 8))
ax.plot(pts.extract(["x"]), predicted_output, label="Neural Network solution")
ax.plot(pts.extract(["x"]), true_output, label="True solution")
plt.legend()
# The solution is overlapped with the actual one, and they are barely indistinguishable. We can also take a look at the loss using `TensorBoard`:
# In[ ]:
print("\nTo load TensorBoard run load_ext tensorboard on your terminal")
print(
"To visualize the loss you can run tensorboard --logdir 'tutorial_logs' on your terminal\n"
)
# # uncomment for running tensorboard
# %load_ext tensorboard
# %tensorboard --logdir=tutorial_logs
# As we can see the loss has not reached a minimum, suggesting that we could train for longer! Alternatively, we can also take look at the loss using callbacks. Here we use `MetricTracker` from `pina.callback`:
# In[ ]:
from pina.callback import MetricTracker
# create the model
newmodel = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
# create the PINN object
newpinn = PINN(
problem, newmodel, optimizer=TorchOptimizer(torch.optim.Adam, lr=0.005)
)
# create the trainer
newtrainer = Trainer(
solver=newpinn,
max_epochs=1500,
logger=True, # enable parameter logging
callbacks=[MetricTracker()],
accelerator="cpu",
train_size=1.0,
test_size=0.0,
val_size=0.0,
enable_model_summary=False,
) # we train on CPU and avoid model summary at beginning of training (optional)
# train
newtrainer.train()
# plot loss
trainer_metrics = newtrainer.callbacks[0].metrics
loss = trainer_metrics["train_loss"]
epochs = range(len(loss))
plt.plot(epochs, loss.cpu())
# plotting
plt.xlabel("epoch")
plt.ylabel("loss")
plt.yscale("log")
# ## What's next?
#
# Congratulations on completing the introductory tutorial of **PINA**! There are several directions you can go now:
#
# 1. Train the network for longer or with different layer sizes and assert the finaly accuracy
#
# 2. Train the network using other types of models (see `pina.model`)
#
# 3. GPU training and speed benchmarking
#
# 4. Many more...

135
tutorials/tutorial10/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

314
tutorials/tutorial10/tutorial.py vendored
View File

@@ -1,314 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Averaging Neural Operator for solving Kuramoto Sivashinsky equation
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial10/tutorial.ipynb)
#
# In this tutorial we will build a Neural Operator using the
# `AveragingNeuralOperator` model and the `SupervisedSolver`. At the end of the
# tutorial you will be able to train a Neural Operator for learning
# the operator of time dependent PDEs.
#
#
# First of all, some useful imports. Note we use `scipy` for i/o operations.
#
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
# get the data
get_ipython().system('mkdir "data"')
get_ipython().system('wget "https://github.com/mathLab/PINA/raw/refs/heads/master/tutorials/tutorial10/data/Data_KS.mat" -O "data/Data_KS.mat"')
get_ipython().system('wget "https://github.com/mathLab/PINA/raw/refs/heads/master/tutorials/tutorial10/data/Data_KS2.mat" -O "data/Data_KS2.mat"')
import torch
import matplotlib.pyplot as plt
import warnings
from scipy import io
from pina import Condition, Trainer, LabelTensor
from pina.model import AveragingNeuralOperator
from pina.solver import SupervisedSolver
from pina.problem.zoo import SupervisedProblem
warnings.filterwarnings("ignore")
# ## Data Generation
#
# We will focus on solving a specific PDE, the **Kuramoto Sivashinsky** (KS) equation.
# The KS PDE is a fourth-order nonlinear PDE with the following form:
#
# $$
# \frac{\partial u}{\partial t}(x,t) = -u(x,t)\frac{\partial u}{\partial x}(x,t)- \frac{\partial^{4}u}{\partial x^{4}}(x,t) - \frac{\partial^{2}u}{\partial x^{2}}(x,t).
# $$
#
# In the above $x\in \Omega=[0, 64]$ represents a spatial location, $t\in\mathbb{T}=[0,50]$ the time and $u(x, t)$ is the value of the function $u:\Omega \times\mathbb{T}\in\mathbb{R}$. We indicate with $\mathbb{U}$ a suitable space for $u$, i.e. we have that the solution $u\in\mathbb{U}$.
#
#
# We impose Dirichlet boundary conditions on the derivative of $u$ on the border of the domain $\partial \Omega$
# $$
# \frac{\partial u}{\partial x}(x,t)=0 \quad \forall (x,t)\in \partial \Omega\times\mathbb{T}.
# $$
#
# Initial conditions are sampled from a distribution over truncated Fourier series with random coefficients
# $\{A_k, \ell_k, \phi_k\}_k$ as
# $$
# u(x,0) = \sum_{k=1}^N A_k \sin(2 \pi \ell_k x / L + \phi_k) \ ,
# $$
#
# where $A_k \in [-0.4, -0.3]$, $\ell_k = 2$, $\phi_k = 2\pi \quad \forall k=1,\dots,N$.
#
#
# We have already generated some data for differenti initial conditions, and our objective will
# be to build a Neural Operator that, given $u(x, t)$ will output $u(x, t+\delta)$, where
# $\delta$ is a fixed time step. We will come back on the Neural Operator architecture, for now
# we first need to import the data.
#
# **Note:**
# *The numerical integration is obtained by using pseudospectral method for spatial derivative discratization and
# implicit Runge Kutta 5 for temporal dynamics.*
#
# In[2]:
# load data
data = io.loadmat("data/Data_KS.mat")
# converting to label tensor
initial_cond_train = LabelTensor(
torch.tensor(data["initial_cond_train"], dtype=torch.float),
["t", "x", "u0"],
)
initial_cond_test = LabelTensor(
torch.tensor(data["initial_cond_test"], dtype=torch.float), ["t", "x", "u0"]
)
sol_train = LabelTensor(
torch.tensor(data["sol_train"], dtype=torch.float), ["u"]
)
sol_test = LabelTensor(torch.tensor(data["sol_test"], dtype=torch.float), ["u"])
print("Data Loaded")
print(f" shape initial condition: {initial_cond_train.shape}")
print(f" shape solution: {sol_train.shape}")
# The data are saved in the form `B \times N \times D`, where `B` is the batch_size
# (basically how many initial conditions we sample), `N` the number of points in the mesh
# (which is the product of the discretization in `x` timese the one in `t`), and
# `D` the dimension of the problem (in this case we have three variables `[u, t, x]`).
#
# We are now going to plot some trajectories!
# In[3]:
# helper function
def plot_trajectory(coords, real, no_sol=None):
# find the x-t shapes
dim_x = len(torch.unique(coords.extract("x")))
dim_t = len(torch.unique(coords.extract("t")))
# if we don't have the Neural Operator solution we simply plot the real one
if no_sol is None:
fig, axs = plt.subplots(1, 1, figsize=(15, 5), sharex=True, sharey=True)
c = axs.imshow(
real.reshape(dim_t, dim_x).T.detach(),
extent=[0, 50, 0, 64],
cmap="PuOr_r",
aspect="auto",
)
axs.set_title("Real solution")
fig.colorbar(c, ax=axs)
axs.set_xlabel("t")
axs.set_ylabel("x")
# otherwise we plot the real one, the Neural Operator one, and their difference
else:
fig, axs = plt.subplots(1, 3, figsize=(15, 5), sharex=True, sharey=True)
axs[0].imshow(
real.reshape(dim_t, dim_x).T.detach(),
extent=[0, 50, 0, 64],
cmap="PuOr_r",
aspect="auto",
)
axs[0].set_title("Real solution")
axs[1].imshow(
no_sol.reshape(dim_t, dim_x).T.detach(),
extent=[0, 50, 0, 64],
cmap="PuOr_r",
aspect="auto",
)
axs[1].set_title("NO solution")
c = axs[2].imshow(
(real - no_sol).abs().reshape(dim_t, dim_x).T.detach(),
extent=[0, 50, 0, 64],
cmap="PuOr_r",
aspect="auto",
)
axs[2].set_title("Absolute difference")
fig.colorbar(c, ax=axs.ravel().tolist())
for ax in axs:
ax.set_xlabel("t")
ax.set_ylabel("x")
plt.show()
# a sample trajectory (we use the sample 5, feel free to change)
sample_number = 20
plot_trajectory(
coords=initial_cond_train[sample_number].extract(["x", "t"]),
real=sol_train[sample_number].extract("u"),
)
# As we can see, as the time progresses the solution becomes chaotic, which makes
# it really hard to learn! We will now focus on building a Neural Operator using the
# `SupervisedSolver` class to tackle the problem.
#
# ## Averaging Neural Operator
#
# We will build a neural operator $\texttt{NO}$ which takes the solution at time $t=0$ for any $x\in\Omega$,
# the time $(t)$ at which we want to compute the solution, and gives back the solution to the KS equation $u(x, t)$, mathematically:
# $$
# \texttt{NO}_\theta : \mathbb{U} \rightarrow \mathbb{U},
# $$
# such that
# $$
# \texttt{NO}_\theta[u(t=0)](x, t) \rightarrow u(x, t).
# $$
#
# There are many ways on approximating the following operator, e.g. by 2D [FNO](https://mathlab.github.io/PINA/_rst/models/fno.html) (for regular meshes),
# a [DeepOnet](https://mathlab.github.io/PINA/_rst/models/deeponet.html), [Continuous Convolutional Neural Operator](https://mathlab.github.io/PINA/_rst/layers/convolution.html),
# [MIONet](https://mathlab.github.io/PINA/_rst/models/mionet.html).
# In this tutorial we will use the *Averaging Neural Operator* presented in [*The Nonlocal Neural Operator: Universal Approximation*](https://arxiv.org/abs/2304.13221)
# which is a [Kernel Neural Operator](https://mathlab.github.io/PINA/_rst/models/base_no.html) with integral kernel:
#
# $$
# K(v) = \sigma\left(Wv(x) + b + \frac{1}{|\Omega|}\int_\Omega v(y)dy\right)
# $$
#
# where:
#
# * $v(x)\in\mathbb{R}^{\rm{emb}}$ is the update for a function $v$ with $\mathbb{R}^{\rm{emb}}$ the embedding (hidden) size
# * $\sigma$ is a non-linear activation
# * $W\in\mathbb{R}^{\rm{emb}\times\rm{emb}}$ is a tunable matrix.
# * $b\in\mathbb{R}^{\rm{emb}}$ is a tunable bias.
#
# If PINA many Kernel Neural Operators are already implemented, and the modular componets of the [Kernel Neural Operator](https://mathlab.github.io/PINA/_rst/models/base_no.html) class permits to create new ones by composing base kernel layers.
#
# **Note:*** We will use the already built class* `AveragingNeuralOperator`, *as constructive excercise try to use the* [KernelNeuralOperator](https://mathlab.github.io/PINA/_rst/models/base_no.html) *class for building a kernel neural operator from scratch. You might employ the different layers that we have in pina, e.g.* [FeedForward](https://mathlab.github.io/PINA/_rst/models/fnn.html), *and* [AveragingNeuralOperator](https://mathlab.github.io/PINA/_rst/layers/avno_layer.html) *layers*.
# In[4]:
class SIREN(torch.nn.Module):
def forward(self, x):
return torch.sin(x)
embedding_dimesion = 40 # hyperparameter embedding dimension
input_dimension = 3 # ['u', 'x', 't']
number_of_coordinates = 2 # ['x', 't']
lifting_net = torch.nn.Linear(input_dimension, embedding_dimesion)
projecting_net = torch.nn.Linear(embedding_dimesion + number_of_coordinates, 1)
model = AveragingNeuralOperator(
lifting_net=lifting_net,
projecting_net=projecting_net,
coordinates_indices=["x", "t"],
field_indices=["u0"],
n_layers=4,
func=SIREN,
)
# Super easy! Notice that we use the `SIREN` activation function, more on [Implicit Neural Representations with Periodic Activation Functions](https://arxiv.org/abs/2006.09661).
#
# ## Solving the KS problem
#
# We will now focus on solving the KS equation using the `SupervisedSolver` class
# and the `AveragingNeuralOperator` model. As done in the [FNO tutorial](https://github.com/mathLab/PINA/blob/master/tutorials/tutorial5/tutorial.ipynb) we now create the Neural Operator problem class with `SupervisedProblem`.
# In[5]:
# initialize problem
problem = SupervisedProblem(
initial_cond_train,
sol_train,
input_variables=initial_cond_train.labels,
output_variables=sol_train.labels,
)
# initialize solver
solver = SupervisedSolver(problem=problem, model=model)
# train, only CPU and avoid model summary at beginning of training (optional)
trainer = Trainer(
solver=solver,
max_epochs=40,
accelerator="cpu",
enable_model_summary=False,
batch_size=5, # we train on CPU and avoid model summary at beginning of training (optional)
train_size=1.0,
val_size=0.0,
test_size=0.0,
)
trainer.train()
# We can now see some plots for the solutions
# In[6]:
sample_number = 2
no_sol = solver(initial_cond_test)
plot_trajectory(
coords=initial_cond_test[sample_number].extract(["x", "t"]),
real=sol_test[sample_number].extract("u"),
no_sol=no_sol[5],
)
# As we can see we can obtain nice result considering the small training time and the difficulty of the problem!
# Let's take a look at the training and testing error:
# In[7]:
from pina.loss import PowerLoss
error_metric = PowerLoss(p=2) # we use the MSE loss
with torch.no_grad():
no_sol_train = solver(initial_cond_train)
err_train = error_metric(
sol_train.extract("u"), no_sol_train
).mean() # we average the error over trajectories
no_sol_test = solver(initial_cond_test)
err_test = error_metric(
sol_test.extract("u"), no_sol_test
).mean() # we average the error over trajectories
print(f"Training error: {float(err_train):.3f}")
print(f"Testing error: {float(err_test):.3f}")
# As we can see the error is pretty small, which agrees with what we can see from the previous plots.
# ## What's next?
#
# Now you know how to solve a time dependent neural operator problem in **PINA**! There are multiple directions you can go now:
#
# 1. Train the network for longer or with different layer sizes and assert the final accuracy
#
# 2. We left a more challenging dataset [Data_KS2.mat](dat/Data_KS2.mat) where $A_k \in [-0.5, 0.5]$, $\ell_k \in [1, 2, 3]$, $\phi_k \in [0, 2\pi]$ for longer training
#
# 3. Compare the performance between the different neural operators (you can even try to implement your favourite one!)

BIN
tutorials/tutorial11/logging.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 204 KiB

475
tutorials/tutorial11/tutorial.ipynb vendored
View File

@@ -4,17 +4,18 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial: PINA and PyTorch Lightning, training tips and visualizations \n",
"\n",
"# Tutorial: Introduction to `Trainer` class\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial11/tutorial.ipynb)\n",
"\n",
"In this tutorial, we will delve deeper into the functionality of the `Trainer` class, which serves as the cornerstone for training **PINA** [Solvers](https://mathlab.github.io/PINA/_rst/_code.html#solvers). \n",
"\n",
"The `Trainer` class offers a plethora of features aimed at improving model accuracy, reducing training time and memory usage, facilitating logging visualization, and more thanks to the amazing job done by the PyTorch Lightning team!\n",
"\n",
"Our leading example will revolve around solving the `SimpleODE` problem, as outlined in the [*Introduction to PINA for Physics Informed Neural Networks training*](https://github.com/mathLab/PINA/blob/master/tutorials/tutorial1/tutorial.ipynb). If you haven't already explored it, we highly recommend doing so before diving into this tutorial.\n",
"Our leading example will revolve around solving a simple regression problem where we want to approximate the following function with a Neural Net model $\\mathcal{M}_{\\theta}$:\n",
"$$y = x^3$$\n",
"by having only a set of $20$ observations $\\{x_i, y_i\\}_{i=1}^{20}$, with $x_i \\sim\\mathcal{U}[-3, 3]\\;\\;\\forall i\\in(1,\\dots,20)$.\n",
"\n",
"Let's start by importing useful modules, define the `SimpleODE` problem and the `PINN` solver."
"Let's start by importing useful modules!"
]
},
{
@@ -30,18 +31,15 @@
"except:\n",
" IN_COLAB = False\n",
"if IN_COLAB:\n",
" !pip install \"pina-mathlab\"\n",
" !pip install \"pina-mathlab[tutorial]\"\n",
"\n",
"import torch\n",
"import warnings\n",
"\n",
"from pina import Condition, Trainer\n",
"from pina.solver import PINN\n",
"from pina import Trainer\n",
"from pina.solver import SupervisedSolver\n",
"from pina.model import FeedForward\n",
"from pina.problem import SpatialProblem\n",
"from pina.operator import grad\n",
"from pina.domain import CartesianDomain\n",
"from pina.equation import Equation, FixedValue\n",
"from pina.problem.zoo import SupervisedProblem\n",
"\n",
"warnings.filterwarnings(\"ignore\")"
]
@@ -59,55 +57,22 @@
"metadata": {},
"outputs": [],
"source": [
"# defining the ode equation\n",
"def ode_equation(input_, output_):\n",
"# defining the problem\n",
"x_train = torch.empty((20, 1)).uniform_(-3, 3)\n",
"y_train = x_train.pow(3) + 3 * torch.randn_like(x_train)\n",
"\n",
" # computing the derivative\n",
" u_x = grad(output_, input_, components=[\"u\"], d=[\"x\"])\n",
"\n",
" # extracting the u input variable\n",
" u = output_.extract([\"u\"])\n",
"\n",
" # calculate the residual and return it\n",
" return u_x - u\n",
"\n",
"\n",
"class SimpleODE(SpatialProblem):\n",
"\n",
" output_variables = [\"u\"]\n",
" spatial_domain = CartesianDomain({\"x\": [0, 1]})\n",
"\n",
" domains = {\n",
" \"x0\": CartesianDomain({\"x\": 0.0}),\n",
" \"D\": CartesianDomain({\"x\": [0, 1]}),\n",
" }\n",
"\n",
" # conditions to hold\n",
" conditions = {\n",
" \"bound_cond\": Condition(domain=\"x0\", equation=FixedValue(1.0)),\n",
" \"phys_cond\": Condition(domain=\"D\", equation=Equation(ode_equation)),\n",
" }\n",
"\n",
" # defining the true solution\n",
" def solution(self, pts):\n",
" return torch.exp(pts.extract([\"x\"]))\n",
"\n",
"\n",
"# sampling for training\n",
"problem = SimpleODE()\n",
"problem.discretise_domain(1, \"random\", domains=[\"x0\"])\n",
"problem.discretise_domain(20, \"lh\", domains=[\"D\"])\n",
"problem = SupervisedProblem(x_train, y_train)\n",
"\n",
"# build the model\n",
"model = FeedForward(\n",
" layers=[10, 10],\n",
" func=torch.nn.Tanh,\n",
" output_dimensions=len(problem.output_variables),\n",
" input_dimensions=len(problem.input_variables),\n",
" output_dimensions=1,\n",
" input_dimensions=1,\n",
")\n",
"\n",
"# create the PINN object\n",
"pinn = PINN(problem, model)"
"# create the SupervisedSolver object\n",
"solver = SupervisedSolver(problem, model, use_lt=False)"
]
},
{
@@ -115,7 +80,7 @@
"metadata": {},
"source": [
"Till now we just followed the extact step of the previous tutorials. The `Trainer` object\n",
"can be initialized by simiply passing the `PINN` solver"
"can be initialized by simiply passing the `SupervisedSolver` solver"
]
},
{
@@ -134,7 +99,7 @@
}
],
"source": [
"trainer = Trainer(solver=pinn)"
"trainer = Trainer(solver=solver)"
]
},
{
@@ -143,18 +108,13 @@
"source": [
"## Trainer Accelerator\n",
"\n",
"When creating the trainer, **by defualt** the `Trainer` will choose the most performing `accelerator` for training which is available in your system, ranked as follow:\n",
"When creating the `Trainer`, **by default** the most performing `accelerator` for training which is available in your system will be chosen, ranked as follows:\n",
"1. [TPU](https://cloud.google.com/tpu/docs/intro-to-tpu)\n",
"2. [IPU](https://www.graphcore.ai/products/ipu)\n",
"3. [HPU](https://habana.ai/)\n",
"4. [GPU](https://www.intel.com/content/www/us/en/products/docs/processors/what-is-a-gpu.html#:~:text=What%20does%20GPU%20stand%20for,video%20editing%2C%20and%20gaming%20applications) or [MPS](https://developer.apple.com/metal/pytorch/)\n",
"5. CPU"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"5. CPU\n",
"\n",
"For setting manually the `accelerator` run:\n",
"\n",
"* `accelerator = {'gpu', 'cpu', 'hpu', 'mps', 'cpu', 'ipu'}` sets the accelerator to a specific one"
@@ -162,7 +122,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 15,
"metadata": {},
"outputs": [
{
@@ -176,14 +136,14 @@
}
],
"source": [
"trainer = Trainer(solver=pinn, accelerator=\"cpu\")"
"trainer = Trainer(solver=solver, accelerator=\"cpu\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"as you can see, even if in the used system `GPU` is available, it is not used since we set `accelerator='cpu'`."
"As you can see, even if a `GPU` is available on the system, it is not used since we set `accelerator='cpu'`."
]
},
{
@@ -192,16 +152,16 @@
"source": [
"## Trainer Logging\n",
"\n",
"In **PINA** you can log metrics in different ways. The simplest approach is to use the `MetricTraker` class from `pina.callbacks` as seen in the [*Introduction to PINA for Physics Informed Neural Networks training*](https://github.com/mathLab/PINA/blob/master/tutorials/tutorial1/tutorial.ipynb) tutorial.\n",
"In **PINA** you can log metrics in different ways. The simplest approach is to use the `MetricTracker` class from `pina.callbacks`, as seen in the [*Introduction to Physics Informed Neural Networks training*](https://github.com/mathLab/PINA/blob/master/tutorials/tutorial1/tutorial.ipynb) tutorial.\n",
"\n",
"However, expecially when we need to train multiple times to get an average of the loss across multiple runs, `pytorch_lightning.loggers` might be useful. Here we will use `TensorBoardLogger` (more on [logging](https://lightning.ai/docs/pytorch/stable/extensions/logging.html) here), but you can choose the one you prefer (or make your own one).\n",
"However, especially when we need to train multiple times to get an average of the loss across multiple runs, `lightning.pytorch.loggers` might be useful. Here we will use `TensorBoardLogger` (more on [logging](https://lightning.ai/docs/pytorch/stable/extensions/logging.html) here), but you can choose the one you prefer (or make your own one).\n",
"\n",
"We will now import `TensorBoardLogger`, do three runs of training and then visualize the results. Notice we set `enable_model_summary=False` to avoid model summary specifications (e.g. number of parameters), set it to true if needed.\n"
"We will now import `TensorBoardLogger`, do three runs of training, and then visualize the results. Notice we set `enable_model_summary=False` to avoid model summary specifications (e.g. number of parameters); set it to `True` if needed."
]
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 17,
"metadata": {},
"outputs": [
{
@@ -209,113 +169,108 @@
"output_type": "stream",
"text": [
"GPU available: True (mps), used: False\n",
"TPU available: False, using: 0 TPU cores\n",
"TPU available: False, using: 0 TPU cores\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"HPU available: False, using: 0 HPUs\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 999: 100%|██████████| 1/1 [00:00<00:00, 233.15it/s, v_num=0, bound_cond_loss=1.22e-5, phys_cond_loss=0.000517, train_loss=0.000529]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=1000` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 999: 100%|██████████| 1/1 [00:00<00:00, 137.95it/s, v_num=0, bound_cond_loss=1.22e-5, phys_cond_loss=0.000517, train_loss=0.000529]\n"
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "775a2d088e304b2589631b176c9e99e2",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=100` reached.\n",
"GPU available: True (mps), used: False\n",
"TPU available: False, using: 0 TPU cores\n",
"HPU available: False, using: 0 HPUs\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 999: 100%|██████████| 1/1 [00:00<00:00, 248.63it/s, v_num=1, bound_cond_loss=2.29e-5, phys_cond_loss=0.00106, train_loss=0.00108] "
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=1000` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 999: 100%|██████████| 1/1 [00:00<00:00, 149.06it/s, v_num=1, bound_cond_loss=2.29e-5, phys_cond_loss=0.00106, train_loss=0.00108]\n"
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "d858dc0a31214f5f86aae78823525b56",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=100` reached.\n",
"GPU available: True (mps), used: False\n",
"TPU available: False, using: 0 TPU cores\n",
"HPU available: False, using: 0 HPUs\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 999: 100%|██████████| 1/1 [00:00<00:00, 254.65it/s, v_num=2, bound_cond_loss=0.00029, phys_cond_loss=0.00253, train_loss=0.00282] "
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "739bf2009f7a48a1b59b7df695276672",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=1000` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 999: 100%|██████████| 1/1 [00:00<00:00, 150.72it/s, v_num=2, bound_cond_loss=0.00029, phys_cond_loss=0.00253, train_loss=0.00282]\n"
"`Trainer.fit` stopped: `max_epochs=100` reached.\n"
]
}
],
"source": [
"from lightning.pytorch.loggers import TensorBoardLogger\n",
"\n",
"# three run of training, by default it trains for 1000 epochs\n",
"# three run of training, by default it trains for 1000 epochs, we set the max to 100\n",
"# we reinitialize the model each time otherwise the same parameters will be optimized\n",
"for _ in range(3):\n",
" model = FeedForward(\n",
" layers=[10, 10],\n",
" func=torch.nn.Tanh,\n",
" output_dimensions=len(problem.output_variables),\n",
" input_dimensions=len(problem.input_variables),\n",
" output_dimensions=1,\n",
" input_dimensions=1,\n",
" )\n",
" pinn = PINN(problem, model)\n",
" solver = SupervisedSolver(problem, model, use_lt=False)\n",
" trainer = Trainer(\n",
" solver=pinn,\n",
" solver=solver,\n",
" accelerator=\"cpu\",\n",
" logger=TensorBoardLogger(save_dir=\"training_log\"),\n",
" enable_model_summary=False,\n",
" train_size=1.0,\n",
" val_size=0.0,\n",
" test_size=0.0,\n",
" max_epochs=100\n",
" )\n",
" trainer.train()"
]
@@ -324,7 +279,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We can now visualize the logs by simply running `tensorboard --logdir=training_log/` on terminal, you should obtain a webpage as the one shown below:"
"We can now visualize the logs by simply running `tensorboard --logdir=training_log/` in the terminal. You should obtain a webpage similar to the one shown below if running for 1000 epochs:"
]
},
{
@@ -332,7 +287,7 @@
"metadata": {},
"source": [
"<p align=\\\"center\\\">\n",
"<img src=\"logging.png\" alt=\\\"Logging API\\\" width=\\\"400\\\"/>\n",
"<img src=\"../static/logging.png\" alt=\\\"Logging API\\\" width=\\\"400\\\"/>\n",
"</p>"
]
},
@@ -340,39 +295,28 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"as you can see, by default, **PINA** logs the losses which are shown in the progress bar, as well as the number of epochs. You can always insert more loggings by either defining a **callback** ([more on callbacks](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html)), or inheriting the solver and modify the programs with different **hooks** ([more on hooks](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html#hooks))."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Trainer Callbacks"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Whenever we need to access certain steps of the training for logging, do static modifications (i.e. not changing the `Solver`) or updating `Problem` hyperparameters (static variables), we can use `Callabacks`. Notice that `Callbacks` allow you to add arbitrary self-contained programs to your training. At specific points during the flow of execution (hooks), the Callback interface allows you to design programs that encapsulate a full set of functionality. It de-couples functionality that does not need to be in **PINA** `Solver`s.\n",
"Lightning has a callback system to execute them when needed. Callbacks should capture NON-ESSENTIAL logic that is NOT required for your lightning module to run.\n",
"As you can see, by default, **PINA** logs the losses which are shown in the progress bar, as well as the number of epochs. You can always insert more loggings by either defining a **callback** ([more on callbacks](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html)), or inheriting the solver and modifying the programs with different **hooks** ([more on hooks](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html#hooks)).\n",
"\n",
"The following are best practices when using/designing callbacks.\n",
"## Trainer Callbacks\n",
"\n",
"Whenever we need to access certain steps of the training for logging, perform static modifications (i.e. not changing the `Solver`), or update `Problem` hyperparameters (static variables), we can use **Callbacks**. Notice that **Callbacks** allow you to add arbitrary self-contained programs to your training. At specific points during the flow of execution (hooks), the Callback interface allows you to design programs that encapsulate a full set of functionality. It de-couples functionality that does not need to be in **PINA** `Solver`s.\n",
"\n",
"Lightning has a callback system to execute them when needed. **Callbacks** should capture NON-ESSENTIAL logic that is NOT required for your lightning module to run.\n",
"\n",
"The following are best practices when using/designing callbacks:\n",
"\n",
"* Callbacks should be isolated in their functionality.\n",
"* Your callback should not rely on the behavior of other callbacks in order to work properly.\n",
"* Do not manually call methods from the callback.\n",
"* Directly calling methods (eg. on_validation_end) is strongly discouraged.\n",
"* Directly calling methods (e.g., on_validation_end) is strongly discouraged.\n",
"* Whenever possible, your callbacks should not depend on the order in which they are executed.\n",
"\n",
"We will try now to implement a naive version of `MetricTraker` to show how callbacks work. Notice that this is a very easy application of callbacks, fortunately in **PINA** we already provide more advanced callbacks in `pina.callbacks`.\n",
"\n",
"<!-- Suppose we want to log the accuracy on some validation poit -->"
"We will try now to implement a naive version of `MetricTraker` to show how callbacks work. Notice that this is a very easy application of callbacks, fortunately in **PINA** we already provide more advanced callbacks in `pina.callbacks`."
]
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
@@ -398,12 +342,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's see the results when applyed to the `SimpleODE` problem. You can define callbacks when initializing the `Trainer` by the `callbacks` argument, which expects a list of callbacks. "
"Let's see the results when applied to the problem. You can define **callbacks** when initializing the `Trainer` by using the `callbacks` argument, which expects a list of callbacks.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 19,
"metadata": {},
"outputs": [
{
@@ -416,24 +360,24 @@
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 999: 100%|██████████| 1/1 [00:00<00:00, 278.93it/s, v_num=0, bound_cond_loss=6.94e-5, phys_cond_loss=0.00116, train_loss=0.00123] "
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "f38442d749ad4702a0c99715ecf08c59",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=1000` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 999: 100%|██████████| 1/1 [00:00<00:00, 140.62it/s, v_num=0, bound_cond_loss=6.94e-5, phys_cond_loss=0.00116, train_loss=0.00123]\n"
"`Trainer.fit` stopped: `max_epochs=10` reached.\n"
]
}
],
@@ -441,12 +385,12 @@
"model = FeedForward(\n",
" layers=[10, 10],\n",
" func=torch.nn.Tanh,\n",
" output_dimensions=len(problem.output_variables),\n",
" input_dimensions=len(problem.input_variables),\n",
" output_dimensions=1,\n",
" input_dimensions=1,\n",
")\n",
"pinn = PINN(problem, model)\n",
"solver = SupervisedSolver(problem, model, use_lt=False)\n",
"trainer = Trainer(\n",
" solver=pinn,\n",
" solver=solver,\n",
" accelerator=\"cpu\",\n",
" logger=True,\n",
" callbacks=[NaiveMetricTracker()], # adding a callbacks\n",
@@ -454,6 +398,7 @@
" train_size=1.0,\n",
" val_size=0.0,\n",
" test_size=0.0,\n",
" max_epochs=10, # training only for 10 epochs\n",
")\n",
"trainer.train()"
]
@@ -467,24 +412,18 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'bound_cond_loss': tensor(0.9935),\n",
" 'phys_cond_loss': tensor(0.0303),\n",
" 'train_loss': tensor(1.0239)},\n",
" {'bound_cond_loss': tensor(0.9875),\n",
" 'phys_cond_loss': tensor(0.0293),\n",
" 'train_loss': tensor(1.0169)},\n",
" {'bound_cond_loss': tensor(0.9815),\n",
" 'phys_cond_loss': tensor(0.0284),\n",
" 'train_loss': tensor(1.0099)}]"
"[{'data_loss': tensor(126.2887), 'train_loss': tensor(126.2887)},\n",
" {'data_loss': tensor(126.2346), 'train_loss': tensor(126.2346)},\n",
" {'data_loss': tensor(126.1805), 'train_loss': tensor(126.1805)}]"
]
},
"execution_count": 8,
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
@@ -497,14 +436,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"PyTorch Lightning also has some built in `Callbacks` which can be used in **PINA**, [here an extensive list](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html#built-in-callbacks). \n",
"PyTorch Lightning also has some built-in `Callbacks` which can be used in **PINA**, [here is an extensive list](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html#built-in-callbacks). \n",
"\n",
"We can for example try the `EarlyStopping` routine, which automatically stops the training when a specific metric converged (here the `train_loss`). In order to let the training keep going forever set `max_epochs=-1`."
"We can, for example, try the `EarlyStopping` routine, which automatically stops the training when a specific metric converges (here the `train_loss`). In order to let the training keep going forever, set `max_epochs=-1`."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 22,
"metadata": {},
"outputs": [
{
@@ -515,25 +454,18 @@
"TPU available: False, using: 0 TPU cores\n",
"HPU available: False, using: 0 HPUs\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 2343: 100%|██████████| 1/1 [00:00<00:00, 64.24it/s, v_num=1, val_loss=4.79e-6, bound_cond_loss=1.15e-7, phys_cond_loss=2.33e-5, train_loss=2.34e-5] \n"
]
}
],
"source": [
"model = FeedForward(\n",
" layers=[10, 10],\n",
" func=torch.nn.Tanh,\n",
" output_dimensions=len(problem.output_variables),\n",
" input_dimensions=len(problem.input_variables),\n",
" output_dimensions=1,\n",
" input_dimensions=1,\n",
")\n",
"pinn = PINN(problem, model)\n",
"solver = SupervisedSolver(problem, model, use_lt=False)\n",
"trainer = Trainer(\n",
" solver=pinn,\n",
" solver=solver,\n",
" accelerator=\"cpu\",\n",
" max_epochs=-1,\n",
" enable_model_summary=False,\n",
@@ -559,24 +491,23 @@
"source": [
"## Trainer Tips to Boost Accuracy, Save Memory and Speed Up Training\n",
"\n",
"Untill now we have seen how to choose the right `accelerator`, how to log and visualize the results, and how to interface with the program in order to add specific parts of code at specific points by `callbacks`.\n",
"Now, we well focus on how boost your training by saving memory and speeding it up, while mantaining the same or even better degree of accuracy!\n",
"Until now we have seen how to choose the right `accelerator`, how to log and visualize the results, and how to interface with the program in order to add specific parts of code at specific points via `callbacks`.\n",
"Now, we will focus on how to boost your training by saving memory and speeding it up, while maintaining the same or even better degree of accuracy!\n",
"\n",
"\n",
"There are several built in methods developed in PyTorch Lightning which can be applied straight forward in **PINA**, here we report some:\n",
"There are several built-in methods developed in PyTorch Lightning which can be applied straightforward in **PINA**. Here we report some:\n",
"\n",
"* [Stochastic Weight Averaging](https://pytorch.org/blog/pytorch-1.6-now-includes-stochastic-weight-averaging/) to boost accuracy\n",
"* [Gradient Clippling](https://deepgram.com/ai-glossary/gradient-clipping) to reduce computational time (and improve accuracy)\n",
"* [Gradient Clipping](https://deepgram.com/ai-glossary/gradient-clipping) to reduce computational time (and improve accuracy)\n",
"* [Gradient Accumulation](https://lightning.ai/docs/pytorch/stable/common/optimization.html#id3) to save memory consumption\n",
"* [Mixed Precision Training](https://lightning.ai/docs/pytorch/stable/common/optimization.html#id3) to save memory consumption\n",
"\n",
"We will just demonstrate how to use the first two, and see the results compared to a standard training.\n",
"We use the [`Timer`](https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.callbacks.Timer.html#lightning.pytorch.callbacks.Timer) callback from `pytorch_lightning.callbacks` to take the times. Let's start by training a simple model without any optimization (train for 2000 epochs)."
"We will just demonstrate how to use the first two and see the results compared to standard training.\n",
"We use the [`Timer`](https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.callbacks.Timer.html#lightning.pytorch.callbacks.Timer) callback from `pytorch_lightning.callbacks` to track the times. Let's start by training a simple model without any optimization (train for 500 epochs)."
]
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 23,
"metadata": {},
"outputs": [
{
@@ -590,25 +521,31 @@
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1999: 100%|██████████| 1/1 [00:00<00:00, 156.69it/s, v_num=2, bound_cond_loss=1.53e-6, phys_cond_loss=0.000169, train_loss=0.000171]"
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "822b8c60e73f49a486d3d702d413d6ff",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=2000` reached.\n"
"`Trainer.fit` stopped: `max_epochs=500` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1999: 100%|██████████| 1/1 [00:00<00:00, 108.75it/s, v_num=2, bound_cond_loss=1.53e-6, phys_cond_loss=0.000169, train_loss=0.000171]\n",
"Total training time 15.36648 s\n"
"Total training time 15.49781 s\n"
]
}
],
@@ -622,16 +559,16 @@
"model = FeedForward(\n",
" layers=[10, 10],\n",
" func=torch.nn.Tanh,\n",
" output_dimensions=len(problem.output_variables),\n",
" input_dimensions=len(problem.input_variables),\n",
" output_dimensions=1,\n",
" input_dimensions=1,\n",
")\n",
"\n",
"pinn = PINN(problem, model)\n",
"solver = SupervisedSolver(problem, model, use_lt=False)\n",
"trainer = Trainer(\n",
" solver=pinn,\n",
" solver=solver,\n",
" accelerator=\"cpu\",\n",
" deterministic=True, # setting deterministic=True ensure reproducibility when a seed is imposed\n",
" max_epochs=2000,\n",
" max_epochs=500,\n",
" enable_model_summary=False,\n",
" callbacks=[Timer()],\n",
") # adding a callbacks\n",
@@ -643,12 +580,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we do the same but with StochasticWeightAveraging"
"Now we do the same but with `StochasticWeightAveraging` enabled"
]
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 24,
"metadata": {},
"outputs": [
{
@@ -662,39 +599,32 @@
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1598: 100%|██████████| 1/1 [00:00<00:00, 224.16it/s, v_num=3, bound_cond_loss=5.7e-6, phys_cond_loss=0.000257, train_loss=0.000263] "
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "dc5f3b47abff4facae7a60d0871f3bfe",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Swapping scheduler `ConstantLR` for `SWALR`\n"
"Swapping scheduler `ConstantLR` for `SWALR`\n",
"`Trainer.fit` stopped: `max_epochs=500` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1999: 100%|██████████| 1/1 [00:00<00:00, 261.43it/s, v_num=3, bound_cond_loss=2.58e-7, phys_cond_loss=9.4e-5, train_loss=9.43e-5] "
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=2000` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1999: 100%|██████████| 1/1 [00:00<00:00, 145.96it/s, v_num=3, bound_cond_loss=2.58e-7, phys_cond_loss=9.4e-5, train_loss=9.43e-5]\n",
"Total training time 17.78182 s\n"
"Total training time 15.52474 s\n"
]
}
],
@@ -707,15 +637,15 @@
"model = FeedForward(\n",
" layers=[10, 10],\n",
" func=torch.nn.Tanh,\n",
" output_dimensions=len(problem.output_variables),\n",
" input_dimensions=len(problem.input_variables),\n",
" output_dimensions=1,\n",
" input_dimensions=1,\n",
")\n",
"pinn = PINN(problem, model)\n",
"solver = SupervisedSolver(problem, model, use_lt=False)\n",
"trainer = Trainer(\n",
" solver=pinn,\n",
" solver=solver,\n",
" accelerator=\"cpu\",\n",
" deterministic=True,\n",
" max_epochs=2000,\n",
" max_epochs=500,\n",
" enable_model_summary=False,\n",
" callbacks=[Timer(), StochasticWeightAveraging(swa_lrs=0.005)],\n",
") # adding StochasticWeightAveraging callbacks\n",
@@ -727,16 +657,16 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"As you can see, the training time does not change at all! Notice that around epoch `1600`\n",
"As you can see, the training time does not change at all! Notice that around epoch 350\n",
"the scheduler is switched from the defalut one `ConstantLR` to the Stochastic Weight Average Learning Rate (`SWALR`).\n",
"This is because by default `StochasticWeightAveraging` will be activated after `int(swa_epoch_start * max_epochs)` with `swa_epoch_start=0.7` by default. Finally, the final `mean_loss` is lower when `StochasticWeightAveraging` is used.\n",
"This is because by default `StochasticWeightAveraging` will be activated after `int(swa_epoch_start * max_epochs)` with `swa_epoch_start=0.7` by default. Finally, the final `train_loss` is lower when `StochasticWeightAveraging` is used.\n",
"\n",
"We will now now do the same but clippling the gradient to be relatively small."
"We will now do the same but clippling the gradient to be relatively small."
]
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 25,
"metadata": {},
"outputs": [
{
@@ -750,39 +680,32 @@
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1598: 100%|██████████| 1/1 [00:00<00:00, 251.76it/s, v_num=4, bound_cond_loss=5.98e-8, phys_cond_loss=3.88e-5, train_loss=3.88e-5] "
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "d475613ad7f34fe6abd182eed8907004",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Swapping scheduler `ConstantLR` for `SWALR`\n"
"Swapping scheduler `ConstantLR` for `SWALR`\n",
"`Trainer.fit` stopped: `max_epochs=500` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1999: 100%|██████████| 1/1 [00:00<00:00, 239.11it/s, v_num=4, bound_cond_loss=0.000333, phys_cond_loss=0.000676, train_loss=0.00101] "
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=2000` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1999: 100%|██████████| 1/1 [00:00<00:00, 127.88it/s, v_num=4, bound_cond_loss=0.000333, phys_cond_loss=0.000676, train_loss=0.00101]\n",
"Total training time 15.12576 s\n"
"Total training time 15.94719 s\n"
]
}
],
@@ -793,14 +716,14 @@
"model = FeedForward(\n",
" layers=[10, 10],\n",
" func=torch.nn.Tanh,\n",
" output_dimensions=len(problem.output_variables),\n",
" input_dimensions=len(problem.input_variables),\n",
" output_dimensions=1,\n",
" input_dimensions=1,\n",
")\n",
"pinn = PINN(problem, model)\n",
"solver = SupervisedSolver(problem, model, use_lt=False)\n",
"trainer = Trainer(\n",
" solver=pinn,\n",
" solver=solver,\n",
" accelerator=\"cpu\",\n",
" max_epochs=2000,\n",
" max_epochs=500,\n",
" enable_model_summary=False,\n",
" gradient_clip_val=0.1, # clipping the gradient\n",
" callbacks=[Timer(), StochasticWeightAveraging(swa_lrs=0.005)],\n",
@@ -813,17 +736,21 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"As we can see we by applying gradient clipping we were able to even obtain lower error!\n",
"As we can see, by applying gradient clipping, we were able to achieve even lower error!\n",
"\n",
"## What's next?\n",
"## What's Next?\n",
"\n",
"Now you know how to use efficiently the `Trainer` class **PINA**! There are multiple directions you can go now:\n",
"Now you know how to use the `Trainer` class efficiently in **PINA**! There are several directions you can explore next:\n",
"\n",
"1. Explore training times on different devices (e.g.) `TPU` \n",
"1. **Explore Training on Different Devices**: Test training times on various devices (e.g., `TPU`) to compare performance.\n",
"\n",
"2. Try to reduce memory cost by mixed precision training and gradient accumulation (especially useful when training Neural Operators)\n",
"2. **Reduce Memory Costs**: Experiment with mixed precision training and gradient accumulation to optimize memory usage, especially when training Neural Operators.\n",
"\n",
"3. Benchmark `Trainer` speed for different precisions."
"3. **Benchmark `Trainer` Speed**: Benchmark the training speed of the `Trainer` class for different precisions to identify potential optimizations.\n",
"\n",
"4. **...and many more!**: Consider expanding to **multi-GPU** setups or other advanced configurations for large-scale training.\n",
"\n",
"For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/).\n"
]
}
],

388
tutorials/tutorial11/tutorial.py vendored
View File

@@ -1,388 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: PINA and PyTorch Lightning, training tips and visualizations
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial11/tutorial.ipynb)
#
# In this tutorial, we will delve deeper into the functionality of the `Trainer` class, which serves as the cornerstone for training **PINA** [Solvers](https://mathlab.github.io/PINA/_rst/_code.html#solvers).
#
# The `Trainer` class offers a plethora of features aimed at improving model accuracy, reducing training time and memory usage, facilitating logging visualization, and more thanks to the amazing job done by the PyTorch Lightning team!
#
# Our leading example will revolve around solving the `SimpleODE` problem, as outlined in the [*Introduction to PINA for Physics Informed Neural Networks training*](https://github.com/mathLab/PINA/blob/master/tutorials/tutorial1/tutorial.ipynb). If you haven't already explored it, we highly recommend doing so before diving into this tutorial.
#
# Let's start by importing useful modules, define the `SimpleODE` problem and the `PINN` solver.
# In[ ]:
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import torch
import warnings
from pina import Condition, Trainer
from pina.solver import PINN
from pina.model import FeedForward
from pina.problem import SpatialProblem
from pina.operator import grad
from pina.domain import CartesianDomain
from pina.equation import Equation, FixedValue
warnings.filterwarnings("ignore")
# Define problem and solver.
# In[2]:
# defining the ode equation
def ode_equation(input_, output_):
# computing the derivative
u_x = grad(output_, input_, components=["u"], d=["x"])
# extracting the u input variable
u = output_.extract(["u"])
# calculate the residual and return it
return u_x - u
class SimpleODE(SpatialProblem):
output_variables = ["u"]
spatial_domain = CartesianDomain({"x": [0, 1]})
domains = {
"x0": CartesianDomain({"x": 0.0}),
"D": CartesianDomain({"x": [0, 1]}),
}
# conditions to hold
conditions = {
"bound_cond": Condition(domain="x0", equation=FixedValue(1.0)),
"phys_cond": Condition(domain="D", equation=Equation(ode_equation)),
}
# defining the true solution
def solution(self, pts):
return torch.exp(pts.extract(["x"]))
# sampling for training
problem = SimpleODE()
problem.discretise_domain(1, "random", domains=["x0"])
problem.discretise_domain(20, "lh", domains=["D"])
# build the model
model = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
# create the PINN object
pinn = PINN(problem, model)
# Till now we just followed the extact step of the previous tutorials. The `Trainer` object
# can be initialized by simiply passing the `PINN` solver
# In[3]:
trainer = Trainer(solver=pinn)
# ## Trainer Accelerator
#
# When creating the trainer, **by defualt** the `Trainer` will choose the most performing `accelerator` for training which is available in your system, ranked as follow:
# 1. [TPU](https://cloud.google.com/tpu/docs/intro-to-tpu)
# 2. [IPU](https://www.graphcore.ai/products/ipu)
# 3. [HPU](https://habana.ai/)
# 4. [GPU](https://www.intel.com/content/www/us/en/products/docs/processors/what-is-a-gpu.html#:~:text=What%20does%20GPU%20stand%20for,video%20editing%2C%20and%20gaming%20applications) or [MPS](https://developer.apple.com/metal/pytorch/)
# 5. CPU
# For setting manually the `accelerator` run:
#
# * `accelerator = {'gpu', 'cpu', 'hpu', 'mps', 'cpu', 'ipu'}` sets the accelerator to a specific one
# In[4]:
trainer = Trainer(solver=pinn, accelerator="cpu")
# as you can see, even if in the used system `GPU` is available, it is not used since we set `accelerator='cpu'`.
# ## Trainer Logging
#
# In **PINA** you can log metrics in different ways. The simplest approach is to use the `MetricTraker` class from `pina.callbacks` as seen in the [*Introduction to PINA for Physics Informed Neural Networks training*](https://github.com/mathLab/PINA/blob/master/tutorials/tutorial1/tutorial.ipynb) tutorial.
#
# However, expecially when we need to train multiple times to get an average of the loss across multiple runs, `pytorch_lightning.loggers` might be useful. Here we will use `TensorBoardLogger` (more on [logging](https://lightning.ai/docs/pytorch/stable/extensions/logging.html) here), but you can choose the one you prefer (or make your own one).
#
# We will now import `TensorBoardLogger`, do three runs of training and then visualize the results. Notice we set `enable_model_summary=False` to avoid model summary specifications (e.g. number of parameters), set it to true if needed.
#
# In[5]:
from lightning.pytorch.loggers import TensorBoardLogger
# three run of training, by default it trains for 1000 epochs
# we reinitialize the model each time otherwise the same parameters will be optimized
for _ in range(3):
model = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
pinn = PINN(problem, model)
trainer = Trainer(
solver=pinn,
accelerator="cpu",
logger=TensorBoardLogger(save_dir="training_log"),
enable_model_summary=False,
train_size=1.0,
val_size=0.0,
test_size=0.0,
)
trainer.train()
# We can now visualize the logs by simply running `tensorboard --logdir=training_log/` on terminal, you should obtain a webpage as the one shown below:
# <p align=\"center\">
# <img src="logging.png" alt=\"Logging API\" width=\"400\"/>
# </p>
# as you can see, by default, **PINA** logs the losses which are shown in the progress bar, as well as the number of epochs. You can always insert more loggings by either defining a **callback** ([more on callbacks](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html)), or inheriting the solver and modify the programs with different **hooks** ([more on hooks](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html#hooks)).
# ## Trainer Callbacks
# Whenever we need to access certain steps of the training for logging, do static modifications (i.e. not changing the `Solver`) or updating `Problem` hyperparameters (static variables), we can use `Callabacks`. Notice that `Callbacks` allow you to add arbitrary self-contained programs to your training. At specific points during the flow of execution (hooks), the Callback interface allows you to design programs that encapsulate a full set of functionality. It de-couples functionality that does not need to be in **PINA** `Solver`s.
# Lightning has a callback system to execute them when needed. Callbacks should capture NON-ESSENTIAL logic that is NOT required for your lightning module to run.
#
# The following are best practices when using/designing callbacks.
#
# * Callbacks should be isolated in their functionality.
# * Your callback should not rely on the behavior of other callbacks in order to work properly.
# * Do not manually call methods from the callback.
# * Directly calling methods (eg. on_validation_end) is strongly discouraged.
# * Whenever possible, your callbacks should not depend on the order in which they are executed.
#
# We will try now to implement a naive version of `MetricTraker` to show how callbacks work. Notice that this is a very easy application of callbacks, fortunately in **PINA** we already provide more advanced callbacks in `pina.callbacks`.
#
# <!-- Suppose we want to log the accuracy on some validation poit -->
# In[6]:
from lightning.pytorch.callbacks import Callback
from lightning.pytorch.callbacks import EarlyStopping
import torch
# define a simple callback
class NaiveMetricTracker(Callback):
def __init__(self):
self.saved_metrics = []
def on_train_epoch_end(
self, trainer, __
): # function called at the end of each epoch
self.saved_metrics.append(
{key: value for key, value in trainer.logged_metrics.items()}
)
# Let's see the results when applyed to the `SimpleODE` problem. You can define callbacks when initializing the `Trainer` by the `callbacks` argument, which expects a list of callbacks.
# In[7]:
model = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
pinn = PINN(problem, model)
trainer = Trainer(
solver=pinn,
accelerator="cpu",
logger=True,
callbacks=[NaiveMetricTracker()], # adding a callbacks
enable_model_summary=False,
train_size=1.0,
val_size=0.0,
test_size=0.0,
)
trainer.train()
# We can easily access the data by calling `trainer.callbacks[0].saved_metrics` (notice the zero representing the first callback in the list given at initialization).
# In[8]:
trainer.callbacks[0].saved_metrics[:3] # only the first three epochs
# PyTorch Lightning also has some built in `Callbacks` which can be used in **PINA**, [here an extensive list](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html#built-in-callbacks).
#
# We can for example try the `EarlyStopping` routine, which automatically stops the training when a specific metric converged (here the `train_loss`). In order to let the training keep going forever set `max_epochs=-1`.
# In[ ]:
model = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
pinn = PINN(problem, model)
trainer = Trainer(
solver=pinn,
accelerator="cpu",
max_epochs=-1,
enable_model_summary=False,
enable_progress_bar=False,
val_size=0.2,
train_size=0.8,
test_size=0.0,
callbacks=[EarlyStopping("val_loss")],
) # adding a callbacks
trainer.train()
# As we can see the model automatically stop when the logging metric stopped improving!
# ## Trainer Tips to Boost Accuracy, Save Memory and Speed Up Training
#
# Untill now we have seen how to choose the right `accelerator`, how to log and visualize the results, and how to interface with the program in order to add specific parts of code at specific points by `callbacks`.
# Now, we well focus on how boost your training by saving memory and speeding it up, while mantaining the same or even better degree of accuracy!
#
#
# There are several built in methods developed in PyTorch Lightning which can be applied straight forward in **PINA**, here we report some:
#
# * [Stochastic Weight Averaging](https://pytorch.org/blog/pytorch-1.6-now-includes-stochastic-weight-averaging/) to boost accuracy
# * [Gradient Clippling](https://deepgram.com/ai-glossary/gradient-clipping) to reduce computational time (and improve accuracy)
# * [Gradient Accumulation](https://lightning.ai/docs/pytorch/stable/common/optimization.html#id3) to save memory consumption
# * [Mixed Precision Training](https://lightning.ai/docs/pytorch/stable/common/optimization.html#id3) to save memory consumption
#
# We will just demonstrate how to use the first two, and see the results compared to a standard training.
# We use the [`Timer`](https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.callbacks.Timer.html#lightning.pytorch.callbacks.Timer) callback from `pytorch_lightning.callbacks` to take the times. Let's start by training a simple model without any optimization (train for 2000 epochs).
# In[10]:
from lightning.pytorch.callbacks import Timer
from lightning.pytorch import seed_everything
# setting the seed for reproducibility
seed_everything(42, workers=True)
model = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
pinn = PINN(problem, model)
trainer = Trainer(
solver=pinn,
accelerator="cpu",
deterministic=True, # setting deterministic=True ensure reproducibility when a seed is imposed
max_epochs=2000,
enable_model_summary=False,
callbacks=[Timer()],
) # adding a callbacks
trainer.train()
print(f'Total training time {trainer.callbacks[0].time_elapsed("train"):.5f} s')
# Now we do the same but with StochasticWeightAveraging
# In[11]:
from lightning.pytorch.callbacks import StochasticWeightAveraging
# setting the seed for reproducibility
seed_everything(42, workers=True)
model = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
pinn = PINN(problem, model)
trainer = Trainer(
solver=pinn,
accelerator="cpu",
deterministic=True,
max_epochs=2000,
enable_model_summary=False,
callbacks=[Timer(), StochasticWeightAveraging(swa_lrs=0.005)],
) # adding StochasticWeightAveraging callbacks
trainer.train()
print(f'Total training time {trainer.callbacks[0].time_elapsed("train"):.5f} s')
# As you can see, the training time does not change at all! Notice that around epoch `1600`
# the scheduler is switched from the defalut one `ConstantLR` to the Stochastic Weight Average Learning Rate (`SWALR`).
# This is because by default `StochasticWeightAveraging` will be activated after `int(swa_epoch_start * max_epochs)` with `swa_epoch_start=0.7` by default. Finally, the final `mean_loss` is lower when `StochasticWeightAveraging` is used.
#
# We will now now do the same but clippling the gradient to be relatively small.
# In[12]:
# setting the seed for reproducibility
seed_everything(42, workers=True)
model = FeedForward(
layers=[10, 10],
func=torch.nn.Tanh,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
pinn = PINN(problem, model)
trainer = Trainer(
solver=pinn,
accelerator="cpu",
max_epochs=2000,
enable_model_summary=False,
gradient_clip_val=0.1, # clipping the gradient
callbacks=[Timer(), StochasticWeightAveraging(swa_lrs=0.005)],
)
trainer.train()
print(f'Total training time {trainer.callbacks[0].time_elapsed("train"):.5f} s')
# As we can see we by applying gradient clipping we were able to even obtain lower error!
#
# ## What's next?
#
# Now you know how to use efficiently the `Trainer` class **PINA**! There are multiple directions you can go now:
#
# 1. Explore training times on different devices (e.g.) `TPU`
#
# 2. Try to reduce memory cost by mixed precision training and gradient accumulation (especially useful when training Neural Operators)
#
# 3. Benchmark `Trainer` speed for different precisions.

147
tutorials/tutorial12/tutorial.ipynb vendored
View File

@@ -4,38 +4,25 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial: The `Equation` Class\n",
"# Tutorial: Introduction to PINA `Equation` class\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial12/tutorial.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this tutorial, we will show how to use the `Equation` Class in PINA. Specifically, we will see how use the Class and its inherited classes to enforce residuals minimization in PINNs."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Example: The Burgers 1D equation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial12/tutorial.ipynb)\n",
"\n",
"\n",
"In this tutorial, we will explore how to use the `Equation` class in **PINA**. We will focus on how to leverage this class, along with its inherited subclasses, to enforce residual minimization in **Physics-Informed Neural Networks (PINNs)**.\n",
"\n",
"By the end of this guide, you'll understand how to integrate physical laws and constraints directly into your model training, ensuring that the solution adheres to the underlying differential equations.\n",
"\n",
"\n",
"## Example: The Burgers 1D equation\n",
"We will start implementing the viscous Burgers 1D problem Class, described as follows:\n",
"\n",
"\n",
"$$\n",
"\\begin{equation}\n",
"\\begin{cases}\n",
"\\frac{\\partial u}{\\partial t} + u \\frac{\\partial u}{\\partial x} &= \\nu \\frac{\\partial^2 u}{ \\partial x^2}, \\quad x\\in(0,1), \\quad t>0\\\\\n",
"u(x,0) &= -\\sin (\\pi x)\\\\\n",
"u(x,t) &= 0 \\quad x = \\pm 1\\\\\n",
"u(x,0) &= -\\sin (\\pi x), \\quad x\\in(0,1)\\\\\n",
"u(x,t) &= 0, \\quad x = \\pm 1, \\quad t>0\\\\\n",
"\\end{cases}\n",
"\\end{equation}\n",
"$$\n",
@@ -47,7 +34,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -59,7 +46,7 @@
"except:\n",
" IN_COLAB = False\n",
"if IN_COLAB:\n",
" !pip install \"pina-mathlab\"\n",
" !pip install \"pina-mathlab[tutorial]\"\n",
"\n",
"import torch\n",
"\n",
@@ -68,7 +55,14 @@
"from pina.problem import SpatialProblem, TimeDependentProblem\n",
"from pina.equation import Equation, FixedValue\n",
"from pina.domain import CartesianDomain\n",
"from pina.operator import grad, laplacian"
"from pina.operator import grad, fast_grad, laplacian"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's begin by defining the Burgers equation and its initial condition as Python functions. These functions will take the model's `input` (spatial and temporal coordinates) and `output` (predicted solution) as arguments. The goal is to compute the residuals for the Burgers equation, which we will minimize during training."
]
},
{
@@ -79,7 +73,7 @@
"source": [
"# define the burger equation\n",
"def burger_equation(input_, output_):\n",
" du = grad(output_, input_)\n",
" du = fast_grad(output_, input_, components=[\"u\"], d=[\"x\"])\n",
" ddu = grad(du, input_, components=[\"dudx\"])\n",
" return (\n",
" du.extract([\"dudt\"])\n",
@@ -91,9 +85,32 @@
"# define initial condition\n",
"def initial_condition(input_, output_):\n",
" u_expected = -torch.sin(torch.pi * input_.extract([\"x\"]))\n",
" return output_.extract([\"u\"]) - u_expected\n",
" return output_.extract([\"u\"]) - u_expected"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Above we use the `grad` operator from `pina.operator` to compute the gradient. In PINA each differential operator takes the following inputs:\n",
"- `output_`: A tensor on which the operator is applied.\n",
"- `input_`: A tensor with respect to which the operator is computed.\n",
"- `components`: The names of the output variables for which the operator is evaluated.\n",
"- `d`: The names of the variables with respect to which the operator is computed.\n",
"\n",
"Each differential operator has its **fast** version, which performs no internal checks on input and output tensors. For these methods, the user is always required to specify both ``components`` and ``d`` as lists of strings.\n",
"\n",
"Let's define now the problem!\n",
"\n",
"> **👉 Do you want to learn more on Problems? Check the dedicated [tutorial](https://mathlab.github.io/PINA/tutorial16/tutorial.html) to learn how to build a Problem from scratch.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class Burgers1D(TimeDependentProblem, SpatialProblem):\n",
"\n",
" # assign output/ spatial and temporal variables\n",
@@ -128,36 +145,29 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"The `Equation` class takes as input a function (in this case it happens twice, with `initial_condition` and `burger_equation`) which computes a residual of an equation, such as a PDE. In a problem class such as the one above, the `Equation` class with such a given input is passed as a parameter in the specified `Condition`. \n",
"\n",
"The `FixedValue` class takes as input a value of same dimensions of the output functions; this class can be used to enforce a fixed value for a specific condition, e.g. Dirichlet boundary conditions, as it happens for instance in our example.\n",
"The `FixedValue` class takes as input a value of the same dimensions as the output functions. This class can be used to enforce a fixed value for a specific condition, such as Dirichlet boundary conditions, as demonstrated in our example.\n",
"\n",
"Once the equations are set as above in the problem conditions, the PINN solver will aim to minimize the residuals described in each equation in the training phase. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Available classes of equations include also:\n",
"- `FixedGradient` and `FixedFlux`: they work analogously to `FixedValue` class, where we can require a constant value to be enforced, respectively, on the gradient of the solution or the divergence of the solution;\n",
"- `Laplace`: it can be used to enforce the laplacian of the solution to be zero;\n",
"- `SystemEquation`: we can enforce multiple conditions on the same subdomain through this class, passing a list of residual equations defined in the problem.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Defining a new Equation class"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`Equation` classes can be also inherited to define a new class. As example, we can see how to rewrite the above problem introducing a new class `Burgers1D`; during the class call, we can pass the viscosity parameter $\\nu$:"
"Once the equations are set as above in the problem conditions, the PINN solver will aim to minimize the residuals described in each equation during the training phase. \n",
"\n",
"### Available classes of equations:\n",
"- `FixedGradient` and `FixedFlux`: These work analogously to the `FixedValue` class, where we can enforce a constant value on the gradient or the divergence of the solution, respectively.\n",
"- `Laplace`: This class can be used to enforce that the Laplacian of the solution is zero.\n",
"- `SystemEquation`: This class allows you to enforce multiple conditions on the same subdomain by passing a list of residual equations defined in the problem.\n",
"\n",
"## Defining a new Equation class\n",
"`Equation` classes can also be inherited to define a new class. For example, we can define a new class `Burgers1D` to represent the Burgers equation. During the class call, we can pass the viscosity parameter $\\nu$:\n",
"\n",
"```python\n",
"class Burgers1D(Equation):\n",
" def __init__(self, nu):\n",
" self.nu = nu\n",
"\n",
" def equation(self, input_, output_):\n",
" ...\n",
"```\n",
"In this case, the `Burgers1D` class will inherit from the `Equation` class and compute the residual of the Burgers equation. The viscosity parameter $\\nu$ is passed when instantiating the class and used in the residual calculation. Let's see it in more details:"
]
},
{
@@ -239,17 +249,18 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# What's next?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Congratulations on completing the `Equation` class tutorial of **PINA**! As we have seen, you can build new classes that inherit `Equation` to store more complex equations, as the Burgers 1D equation, only requiring to pass the characteristic coefficients of the problem. \n",
"From now on, you can:\n",
"- define additional complex equation classes (e.g. `SchrodingerEquation`, `NavierStokeEquation`..)\n",
"- define more `FixedOperator` (e.g. `FixedCurl`)"
"## What's Next?\n",
"\n",
"Congratulations on completing the `Equation` class tutorial of **PINA**! As we've seen, you can build new classes that inherit from `Equation` to store more complex equations, such as the 1D Burgers equation, by simply passing the characteristic coefficients of the problem.\n",
"\n",
"From here, you can:\n",
"\n",
"- **Define Additional Complex Equation Classes**: Create your own equation classes, such as `SchrodingerEquation`, `NavierStokesEquation`, etc.\n",
"- **Define More `FixedOperator` Classes**: Implement operators like `FixedCurl`, `FixedDivergence`, and others for more advanced simulations.\n",
"- **Integrate Custom Equations and Operators**: Combine your custom equations and operators into larger systems for more complex simulations.\n",
"- **and many more!**: Explore for example different residual minimization techniques to improve the performance and accuracy of your models.\n",
"\n",
"For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/)."
]
}
],

188
tutorials/tutorial12/tutorial.py vendored
View File

@@ -1,188 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: The `Equation` Class
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial12/tutorial.ipynb)
# In this tutorial, we will show how to use the `Equation` Class in PINA. Specifically, we will see how use the Class and its inherited classes to enforce residuals minimization in PINNs.
# # Example: The Burgers 1D equation
# We will start implementing the viscous Burgers 1D problem Class, described as follows:
#
#
# $$
# \begin{equation}
# \begin{cases}
# \frac{\partial u}{\partial t} + u \frac{\partial u}{\partial x} &= \nu \frac{\partial^2 u}{ \partial x^2}, \quad x\in(0,1), \quad t>0\\
# u(x,0) &= -\sin (\pi x)\\
# u(x,t) &= 0 \quad x = \pm 1\\
# \end{cases}
# \end{equation}
# $$
#
# where we set $ \nu = \frac{0.01}{\pi}$.
#
# In the class that models this problem we will see in action the `Equation` class and one of its inherited classes, the `FixedValue` class.
# In[1]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import torch
# useful imports
from pina import Condition
from pina.problem import SpatialProblem, TimeDependentProblem
from pina.equation import Equation, FixedValue
from pina.domain import CartesianDomain
from pina.operator import grad, laplacian
# In[2]:
# define the burger equation
def burger_equation(input_, output_):
du = grad(output_, input_)
ddu = grad(du, input_, components=["dudx"])
return (
du.extract(["dudt"])
+ output_.extract(["u"]) * du.extract(["dudx"])
- (0.01 / torch.pi) * ddu.extract(["ddudxdx"])
)
# define initial condition
def initial_condition(input_, output_):
u_expected = -torch.sin(torch.pi * input_.extract(["x"]))
return output_.extract(["u"]) - u_expected
class Burgers1D(TimeDependentProblem, SpatialProblem):
# assign output/ spatial and temporal variables
output_variables = ["u"]
spatial_domain = CartesianDomain({"x": [-1, 1]})
temporal_domain = CartesianDomain({"t": [0, 1]})
domains = {
"bound_cond1": CartesianDomain({"x": -1, "t": [0, 1]}),
"bound_cond2": CartesianDomain({"x": 1, "t": [0, 1]}),
"time_cond": CartesianDomain({"x": [-1, 1], "t": 0}),
"phys_cond": CartesianDomain({"x": [-1, 1], "t": [0, 1]}),
}
# problem condition statement
conditions = {
"bound_cond1": Condition(
domain="bound_cond1", equation=FixedValue(0.0)
),
"bound_cond2": Condition(
domain="bound_cond2", equation=FixedValue(0.0)
),
"time_cond": Condition(
domain="time_cond", equation=Equation(initial_condition)
),
"phys_cond": Condition(
domain="phys_cond", equation=Equation(burger_equation)
),
}
#
# The `Equation` class takes as input a function (in this case it happens twice, with `initial_condition` and `burger_equation`) which computes a residual of an equation, such as a PDE. In a problem class such as the one above, the `Equation` class with such a given input is passed as a parameter in the specified `Condition`.
#
# The `FixedValue` class takes as input a value of same dimensions of the output functions; this class can be used to enforce a fixed value for a specific condition, e.g. Dirichlet boundary conditions, as it happens for instance in our example.
#
# Once the equations are set as above in the problem conditions, the PINN solver will aim to minimize the residuals described in each equation in the training phase.
# Available classes of equations include also:
# - `FixedGradient` and `FixedFlux`: they work analogously to `FixedValue` class, where we can require a constant value to be enforced, respectively, on the gradient of the solution or the divergence of the solution;
# - `Laplace`: it can be used to enforce the laplacian of the solution to be zero;
# - `SystemEquation`: we can enforce multiple conditions on the same subdomain through this class, passing a list of residual equations defined in the problem.
#
# # Defining a new Equation class
# `Equation` classes can be also inherited to define a new class. As example, we can see how to rewrite the above problem introducing a new class `Burgers1D`; during the class call, we can pass the viscosity parameter $\nu$:
# In[3]:
class Burgers1DEquation(Equation):
def __init__(self, nu=0.0):
"""
Burgers1D class. This class can be
used to enforce the solution u to solve the viscous Burgers 1D Equation.
:param torch.float32 nu: the viscosity coefficient. Default value is set to 0.
"""
self.nu = nu
def equation(input_, output_):
return (
grad(output_, input_, d="t")
+ output_ * grad(output_, input_, d="x")
- self.nu * laplacian(output_, input_, d="x")
)
super().__init__(equation)
# Now we can just pass the above class as input for the last condition, setting $\nu= \frac{0.01}{\pi}$:
# In[4]:
class Burgers1D(TimeDependentProblem, SpatialProblem):
# define initial condition
def initial_condition(input_, output_):
u_expected = -torch.sin(torch.pi * input_.extract(["x"]))
return output_.extract(["u"]) - u_expected
# assign output/ spatial and temporal variables
output_variables = ["u"]
spatial_domain = CartesianDomain({"x": [-1, 1]})
temporal_domain = CartesianDomain({"t": [0, 1]})
domains = {
"bound_cond1": CartesianDomain({"x": -1, "t": [0, 1]}),
"bound_cond2": CartesianDomain({"x": 1, "t": [0, 1]}),
"time_cond": CartesianDomain({"x": [-1, 1], "t": 0}),
"phys_cond": CartesianDomain({"x": [-1, 1], "t": [0, 1]}),
}
# problem condition statement
conditions = {
"bound_cond1": Condition(
domain="bound_cond1", equation=FixedValue(0.0)
),
"bound_cond2": Condition(
domain="bound_cond2", equation=FixedValue(0.0)
),
"time_cond": Condition(
domain="time_cond", equation=Equation(initial_condition)
),
"phys_cond": Condition(
domain="phys_cond", equation=Burgers1DEquation(nu=0.01 / torch.pi)
),
}
# # What's next?
# Congratulations on completing the `Equation` class tutorial of **PINA**! As we have seen, you can build new classes that inherit `Equation` to store more complex equations, as the Burgers 1D equation, only requiring to pass the characteristic coefficients of the problem.
# From now on, you can:
# - define additional complex equation classes (e.g. `SchrodingerEquation`, `NavierStokeEquation`..)
# - define more `FixedOperator` (e.g. `FixedCurl`)

191
tutorials/tutorial13/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

283
tutorials/tutorial13/tutorial.py vendored
View File

@@ -1,283 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Multiscale PDE learning with Fourier Feature Network
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial13/tutorial.ipynb)
#
# This tutorial presents how to solve with Physics-Informed Neural Networks (PINNs)
# a PDE characterized by multiscale behaviour, as
# presented in [*On the eigenvector bias of Fourier feature networks: From regression to solving
# multi-scale PDEs with physics-informed neural networks*](
# https://doi.org/10.1016/j.cma.2021.113938).
#
# First of all, some useful imports.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import torch
import matplotlib.pyplot as plt
import warnings
from pina import Condition, Trainer
from pina.problem import SpatialProblem
from pina.operator import laplacian
from pina.solver import PINN, SelfAdaptivePINN as SAPINN
from pina.loss import LpLoss
from pina.domain import CartesianDomain
from pina.equation import Equation, FixedValue
from pina.model import FeedForward
from pina.model.block import FourierFeatureEmbedding
warnings.filterwarnings("ignore")
# ## Multiscale Problem
#
# We begin by presenting the problem which also can be found in Section 2 of [*On the eigenvector bias of Fourier feature networks: From regression to solving
# multi-scale PDEs with physics-informed neural networks*](
# https://doi.org/10.1016/j.cma.2021.113938). The one-dimensional Poisson problem we aim to solve is mathematically written as:
#
# \begin{equation}
# \begin{cases}
# \Delta u (x) + f(x) = 0 \quad x \in [0,1], \\
# u(x) = 0 \quad x \in \partial[0,1], \\
# \end{cases}
# \end{equation}
#
# We impose the solution as $u(x) = \sin(2\pi x) + 0.1 \sin(50\pi x)$ and obtain the force term $f(x) = (2\pi)^2 \sin(2\pi x) + 0.1 (50 \pi)^2 \sin(50\pi x)$.
# Though this example is simple and pedagogical, it is worth noting that
# the solution exhibits low frequency in the macro-scale and high frequency in the micro-scale, which resembles many
# practical scenarios.
#
#
# In **PINA** this problem is written, as always, as a class [see here for a tutorial on the Problem class](https://mathlab.github.io/PINA/_rst/tutorials/tutorial1/tutorial.html). Below you can find the `Poisson` problem which is mathmatically described above.
# In[2]:
class Poisson(SpatialProblem):
output_variables = ["u"]
spatial_domain = CartesianDomain({"x": [0, 1]})
def poisson_equation(input_, output_):
x = input_.extract("x")
u_xx = laplacian(output_, input_, components=["u"], d=["x"])
f = ((2 * torch.pi) ** 2) * torch.sin(2 * torch.pi * x) + 0.1 * (
(50 * torch.pi) ** 2
) * torch.sin(50 * torch.pi * x)
return u_xx + f
domains = {
"bound_cond0": CartesianDomain({"x": 0.0}),
"bound_cond1": CartesianDomain({"x": 1.0}),
"phys_cond": spatial_domain,
}
# here we write the problem conditions
conditions = {
"bound_cond0": Condition(
domain="bound_cond0", equation=FixedValue(0.0)
),
"bound_cond1": Condition(
domain="bound_cond1", equation=FixedValue(0.0)
),
"phys_cond": Condition(
domain="phys_cond", equation=Equation(poisson_equation)
),
}
def solution(self, x):
return torch.sin(2 * torch.pi * x) + 0.1 * torch.sin(50 * torch.pi * x)
problem = Poisson()
# let's discretise the domain
problem.discretise_domain(128, "grid", domains=["phys_cond"])
problem.discretise_domain(1, "grid", domains=["bound_cond0", "bound_cond1"])
# A standard PINN approach would be to fit this model using a Feed Forward (fully connected) Neural Network. For a conventional fully-connected neural network is easy to
# approximate a function $u$, given sufficient data inside the computational domain. However solving high-frequency or multi-scale problems presents great challenges to PINNs especially when the number of data cannot capture the different scales.
#
# Below we run a simulation using the `PINN` solver and the self adaptive `SAPINN` solver, using a [`FeedForward`](https://mathlab.github.io/PINA/_modules/pina/model/feed_forward.html#FeedForward) model.
# In[3]:
# training with PINN and visualize results
pinn = PINN(
problem=problem,
model=FeedForward(
input_dimensions=1, output_dimensions=1, layers=[100, 100, 100]
),
)
trainer = Trainer(
pinn,
max_epochs=1500,
accelerator="cpu",
enable_model_summary=False,
val_size=0.0,
train_size=1.0,
test_size=0.0,
)
trainer.train()
# training with PINN and visualize results
sapinn = SAPINN(
problem=problem,
model=FeedForward(
input_dimensions=1, output_dimensions=1, layers=[100, 100, 100]
),
)
trainer_sapinn = Trainer(
sapinn,
max_epochs=1500,
accelerator="cpu",
enable_model_summary=False,
val_size=0.0,
train_size=1.0,
test_size=0.0,
)
trainer_sapinn.train()
# In[4]:
# define the function to plot the solution obtained using matplotlib
def plot_solution(pinn_to_use, title):
pts = pinn_to_use.problem.spatial_domain.sample(256, "grid", variables="x")
predicted_output = pinn_to_use.forward(pts).extract("u").tensor.detach()
true_output = pinn_to_use.problem.solution(pts).detach()
plt.plot(
pts.extract(["x"]), predicted_output, label="Neural Network solution"
)
plt.plot(pts.extract(["x"]), true_output, label="True solution")
plt.title(title)
plt.legend()
# plot the solution of the two PINNs
plot_solution(pinn, "PINN solution")
plt.figure()
plot_solution(sapinn, "Self Adaptive PINN solution")
# We can clearly see that the solution has not been learned by the two different solvers. Indeed the big problem is not in the optimization strategy (i.e. the solver), but in the model used to solve the problem. A simple `FeedForward` network can hardly handle multiscales if not enough collocation points are used!
#
# We can also compute the $l_2$ relative error for the `PINN` and `SAPINN` solutions:
# In[5]:
# l2 loss from PINA losses
l2_loss = LpLoss(p=2, relative=False)
# sample new test points
pts = pts = problem.spatial_domain.sample(100, "grid")
print(
f"Relative l2 error PINN {l2_loss(pinn(pts), problem.solution(pts)).item():.2%}"
)
print(
f"Relative l2 error SAPINN {l2_loss(sapinn(pts), problem.solution(pts)).item():.2%}"
)
# Which is indeed very high!
# ## Fourier Feature Embedding in PINA
# Fourier Feature Embedding is a way to transform the input features, to help the network in learning multiscale variations in the output. It was
# first introduced in [*On the eigenvector bias of Fourier feature networks: From regression to solving
# multi-scale PDEs with physics-informed neural networks*](
# https://doi.org/10.1016/j.cma.2021.113938) showing great results for multiscale problems. The basic idea is to map the input $\mathbf{x}$ into an embedding $\tilde{\mathbf{x}}$ where:
#
# $$ \tilde{\mathbf{x}} =\left[\cos\left( \mathbf{B} \mathbf{x} \right), \sin\left( \mathbf{B} \mathbf{x} \right)\right] $$
#
# and $\mathbf{B}_{ij} \sim \mathcal{N}(0, \sigma^2)$. This simple operation allow the network to learn on multiple scales!
#
# In PINA we already have implemented the feature as a `layer` called [`FourierFeatureEmbedding`](https://mathlab.github.io/PINA/_rst/layers/fourier_embedding.html). Below we will build the *Multi-scale Fourier Feature Architecture*. In this architecture multiple Fourier feature embeddings (initialized with different $\sigma$)
# are applied to input coordinates and then passed through the same fully-connected neural network, before the outputs are finally concatenated with a linear layer.
# In[6]:
class MultiscaleFourierNet(torch.nn.Module):
def __init__(self):
super().__init__()
self.embedding1 = FourierFeatureEmbedding(
input_dimension=1, output_dimension=100, sigma=1
)
self.embedding2 = FourierFeatureEmbedding(
input_dimension=1, output_dimension=100, sigma=10
)
self.layers = FeedForward(
input_dimensions=100, output_dimensions=100, layers=[100]
)
self.final_layer = torch.nn.Linear(2 * 100, 1)
def forward(self, x):
e1 = self.layers(self.embedding1(x))
e2 = self.layers(self.embedding2(x))
return self.final_layer(torch.cat([e1, e2], dim=-1))
# We will train the `MultiscaleFourierNet` with the `PINN` solver (and feel free to try also with our PINN variants (`SAPINN`, `GPINN`, `CompetitivePINN`, ...).
# In[7]:
multiscale_pinn = PINN(problem=problem, model=MultiscaleFourierNet())
trainer = Trainer(
multiscale_pinn,
max_epochs=1500,
accelerator="cpu",
enable_model_summary=False,
val_size=0.0,
train_size=1.0,
test_size=0.0,
)
trainer.train()
# Let us now plot the solution and compute the relative $l_2$ again!
# In[8]:
# plot solution obtained
plot_solution(multiscale_pinn, "Multiscale PINN solution")
# sample new test points
pts = pts = problem.spatial_domain.sample(100, "grid")
print(
f"Relative l2 error PINN with MultiscaleFourierNet: {l2_loss(multiscale_pinn(pts), problem.solution(pts)).item():.2%}"
)
# It is pretty clear that the network has learned the correct solution, with also a very low error. Obviously a longer training and a more expressive neural network could improve the results!
#
# ## What's next?
#
# Congratulations on completing the one dimensional Poisson tutorial of **PINA** using `FourierFeatureEmbedding`! There are multiple directions you can go now:
#
# 1. Train the network for longer or with different layer sizes and assert the finaly accuracy
#
# 2. Understand the role of `sigma` in `FourierFeatureEmbedding` (see original paper for a nice reference)
#
# 3. Code the *Spatio-temporal multi-scale Fourier feature architecture* for a more complex time dependent PDE (section 3 of the original reference)
#
# 4. Many more...

657
tutorials/tutorial14/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

338
tutorials/tutorial14/tutorial.py vendored
View File

@@ -1,338 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Predicting Lid-driven cavity problem parameters with POD-RBF
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial14/tutorial.ipynb)
# In this tutorial we will show how to use the **PINA** library to predict the distributions of velocity and pressure the Lid-driven Cavity problem, a benchmark in Computational Fluid Dynamics. The problem consists of a square cavity with a lid on top moving with tangential velocity (by convention to the right), with the addition of no-slip conditions on the walls of the cavity and null static pressure on the lower left angle.
#
# Our goal is to predict the distributions of velocity and pressure of the fluid inside the cavity as the Reynolds number of the inlet fluid varies. To do so we're using a Reduced Order Model (ROM) based on Proper Orthogonal Decomposition (POD). The parametric solution manifold is approximated here with Radial Basis Function (RBF) Interpolation, a common mesh-free interpolation method that doesn't require trainers or solvers as the found radial basis functions are used to interpolate new points.
# Let's start with the necessary imports. We're particularly interested in the `PODBlock` and `RBFBlock` classes which will allow us to define the POD-RBF model.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
get_ipython().run_line_magic('matplotlib', 'inline')
import matplotlib.pyplot as plt
import torch
import pina
import warnings
from pina.model.block import PODBlock, RBFBlock
from pina import LabelTensor
warnings.filterwarnings("ignore")
# In this tutorial we're gonna use the `LidCavity` class from the [Smithers](https://github.com/mathLab/Smithers) library, which contains a set of parametric solutions of the Lid-driven cavity problem in a square domain. The dataset consists of 300 snapshots of the parameter fields, which in this case are the magnitude of velocity and the pressure, and the corresponding parameter values $u$ and $p$. Each snapshot corresponds to a different value of the tangential velocity $\mu$ of the lid, which has been sampled uniformly between 0.01 m/s and 1 m/s.
#
# Let's start by importing the dataset:
# In[2]:
import smithers
from smithers.dataset import LidCavity
dataset = LidCavity()
# Let's plot two the data points and the corresponding solution for both parameters at different snapshots, in order to better visualise the data we're using:
# In[3]:
fig, axs = plt.subplots(1, 3, figsize=(14, 3))
for ax, par, u in zip(axs, dataset.params[:3], dataset.snapshots["mag(v)"][:3]):
ax.tricontourf(dataset.triang, u, levels=16)
ax.set_title(f"$u$ field for $\mu$ = {par[0]:.4f}")
fig, axs = plt.subplots(1, 3, figsize=(14, 3))
for ax, par, u in zip(axs, dataset.params[:3], dataset.snapshots["p"][:3]):
ax.tricontourf(dataset.triang, u, levels=16)
ax.set_title(f"$p$ field for $\mu$ = {par[0]:.4f}")
# To train the model we only need the snapshots for the two parameters. In order to be able to work with the snapshots in **PINA** we first need to assure they're in a compatible format, hence why we start by casting them into `LabelTensor` objects:
# In[4]:
"""velocity magnitude data, 5041 for each snapshot"""
u = torch.tensor(dataset.snapshots["mag(v)"]).float()
u = LabelTensor(u, labels=[f"s{i}" for i in range(u.shape[1])])
"""pressure data, 5041 for each snapshot"""
p = torch.tensor(dataset.snapshots["p"]).float()
p = LabelTensor(p, labels=[f"s{i}" for i in range(p.shape[1])])
"""mu corresponding to each snapshot"""
mu = torch.tensor(dataset.params).float()
mu = LabelTensor(mu, labels=["mu"])
# The goal of our training is to be able to predict the solution for new test parameters. The first thing we need to do is validate the accuracy of the model, and in order to do so we split the 300 snapshots in training and testing dataset. In the example we set the training `ratio` to 0.9, which means that 90% of the total snapshots is used for training and the remaining 10% for testing.
# In[5]:
"""number of snapshots"""
n = u.shape[0]
"""training over total snapshots ratio and number of training snapshots"""
ratio = 0.9
n_train = int(n * ratio)
"""split u and p data"""
u_train, u_test = u[:n_train], u[n_train:] # for mag(v)
p_train, p_test = p[:n_train], p[n_train:] # for p
"""split snapshots"""
mu_train, mu_test = mu[:n_train], mu[n_train:]
# We now proceed by defining the model we intend to use. We inherit from the `torch.nn.Module` class, but in addition we require a `pod_rank` for the POD part and a function `rbf_kernel` in order to perform the RBF part:
# In[6]:
class PODRBF(torch.nn.Module):
"""
Proper orthogonal decomposition with Radial Basis Function interpolation model.
"""
def __init__(self, pod_rank, rbf_kernel):
super().__init__()
self.pod = PODBlock(pod_rank)
self.rbf = RBFBlock(kernel=rbf_kernel)
# We complete our model by adding two crucial methods. The first is `forward`, and it expands the input POD coefficients. After being expanded the POD layer needs to be fit, hence why we add a `fit` method that gives us the POD basis (current **PINA** default is by performing truncated Singular Value Decomposition). The same method then uses the basis to fit the RBF interpolation. Overall, the completed class looks like this:
# In[7]:
class PODRBF(torch.nn.Module):
"""
Proper orthogonal decomposition with Radial Basis Function interpolation model.
"""
def __init__(self, pod_rank, rbf_kernel):
super().__init__()
self.pod = PODBlock(pod_rank)
self.rbf = RBFBlock(kernel=rbf_kernel)
def forward(self, x):
"""
Defines the computation performed at every call.
:param x: The tensor to apply the forward pass.
:type x: torch.Tensor
:return: the output computed by the model.
:rtype: torch.Tensor
"""
coefficients = self.rbf(x)
return self.pod.expand(coefficients)
def fit(self, p, x):
"""
Call the :meth:`pina.model.layers.PODBlock.fit` method of the
:attr:`pina.model.layers.PODBlock` attribute to perform the POD,
and the :meth:`pina.model.layers.RBFBlock.fit` method of the
:attr:`pina.model.layers.RBFBlock` attribute to fit the interpolation.
"""
self.pod.fit(x)
self.rbf.fit(p, self.pod.reduce(x))
# Now that we've built our class, we can fit the model and ask it to predict the parameters for the remaining snapshots. We remember that we don't need to train the model, as it doesn't involve any learnable parameter. The only things we have to set are the rank of the decomposition and the radial basis function (here we use thin plate). Here we focus on predicting the magnitude of velocity:
# In[8]:
"""create the model"""
pod_rbfu = PODRBF(pod_rank=20, rbf_kernel="thin_plate_spline")
"""fit the model to velocity training data"""
pod_rbfu.fit(mu_train, u_train)
"""predict the parameter using the fitted model"""
u_train_rbf = pod_rbfu(mu_train)
u_test_rbf = pod_rbfu(mu_test)
# Finally we can calculate the relative error for our model:
# In[9]:
relative_u_error_train = torch.norm(u_train_rbf - u_train) / torch.norm(u_train)
relative_u_error_test = torch.norm(u_test_rbf - u_test) / torch.norm(u_test)
print("Error summary for POD-RBF model:")
print(f" Train: {relative_u_error_train.item():e}")
print(f" Test: {relative_u_error_test.item():e}")
# The results are promising! Now let's visualise them, comparing four random predicted snapshots to the true ones:
# In[10]:
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
idx = torch.randint(0, len(u_test), (4,))
u_idx_rbf = pod_rbfu(mu_test[idx])
fig, axs = plt.subplots(3, 4, figsize=(14, 10))
relative_u_error_rbf = np.abs(u_test[idx] - u_idx_rbf.detach())
relative_u_error_rbf = np.where(
u_test[idx] < 1e-7, 1e-7, relative_u_error_rbf / u_test[idx]
)
for i, (idx_, rbf_, rbf_err_) in enumerate(
zip(idx, u_idx_rbf, relative_u_error_rbf)
):
axs[0, i].set_title("Prediction for " f"$\mu$ = {mu_test[idx_].item():.4f}")
axs[1, i].set_title(
"True snapshot for " f"$\mu$ = {mu_test[idx_].item():.4f}"
)
axs[2, i].set_title("Error for " f"$\mu$ = {mu_test[idx_].item():.4f}")
cm = axs[0, i].tricontourf(
dataset.triang, rbf_.detach()
) # POD-RBF prediction
plt.colorbar(cm, ax=axs[0, i])
cm = axs[1, i].tricontourf(dataset.triang, u_test[idx_].flatten()) # Truth
plt.colorbar(cm, ax=axs[1, i])
cm = axs[2, i].tripcolor(
dataset.triang, rbf_err_, norm=matplotlib.colors.LogNorm()
) # Error for POD-RBF
plt.colorbar(cm, ax=axs[2, i])
plt.show()
# Overall we have reached a good level of approximation while avoiding time-consuming training procedures. Let's try doing the same to predict the pressure snapshots:
# In[11]:
"""create the model"""
pod_rbfp = PODRBF(pod_rank=20, rbf_kernel="thin_plate_spline")
"""fit the model to pressure training data"""
pod_rbfp.fit(mu_train, p_train)
"""predict the parameter using the fitted model"""
p_train_rbf = pod_rbfp(mu_train)
p_test_rbf = pod_rbfp(mu_test)
relative_p_error_train = torch.norm(p_train_rbf - p_train) / torch.norm(p_train)
relative_p_error_test = torch.norm(p_test_rbf - p_test) / torch.norm(p_test)
print("Error summary for POD-RBF model:")
print(f" Train: {relative_p_error_train.item():e}")
print(f" Test: {relative_p_error_test.item():e}")
# Unfortunately here we obtain a very high relative test error, although this is likely due to the nature of the available data. Looking at the plots we can see that the pressure field is subject to high variations between subsequent snapshots, especially here:
# In[12]:
fig, axs = plt.subplots(2, 3, figsize=(14, 6))
for ax, par, u in zip(
axs.ravel(), dataset.params[66:72], dataset.snapshots["p"][66:72]
):
cm = ax.tricontourf(dataset.triang, u, levels=16)
plt.colorbar(cm, ax=ax)
ax.set_title(f"$p$ field for $\mu$ = {par[0]:.4f}")
plt.tight_layout()
plt.show()
# Or here:
# In[13]:
fig, axs = plt.subplots(2, 3, figsize=(14, 6))
for ax, par, u in zip(
axs.ravel(), dataset.params[98:104], dataset.snapshots["p"][98:104]
):
cm = ax.tricontourf(dataset.triang, u, levels=16)
plt.colorbar(cm, ax=ax)
ax.set_title(f"$p$ field for $\mu$ = {par[0]:.4f}")
plt.tight_layout()
plt.show()
# Scrolling through the velocity snapshots we can observe a more regular behaviour, with no such variations in subsequent snapshots. Moreover, if we decide not to consider the abovementioned "problematic" snapshots, we can already observe a huge improvement:
# In[14]:
"""excluding problematic snapshots"""
data = list(range(300))
data_to_consider = data[:67] + data[71:100] + data[102:]
"""proceed as before"""
newp = torch.tensor(dataset.snapshots["p"][data_to_consider]).float()
newp = LabelTensor(newp, labels=[f"s{i}" for i in range(newp.shape[1])])
newmu = torch.tensor(dataset.params[data_to_consider]).float()
newmu = LabelTensor(newmu, labels=["mu"])
newn = newp.shape[0]
ratio = 0.9
new_train = int(newn * ratio)
new_p_train, new_p_test = newp[:new_train], newp[new_train:]
new_mu_train, new_mu_test = newmu[:new_train], newmu[new_train:]
new_pod_rbfp = PODRBF(pod_rank=20, rbf_kernel="thin_plate_spline")
new_pod_rbfp.fit(new_mu_train, new_p_train)
new_p_train_rbf = new_pod_rbfp(new_mu_train)
new_p_test_rbf = new_pod_rbfp(new_mu_test)
new_relative_p_error_train = torch.norm(
new_p_train_rbf - new_p_train
) / torch.norm(new_p_train)
new_relative_p_error_test = torch.norm(
new_p_test_rbf - new_p_test
) / torch.norm(new_p_test)
print("Error summary for POD-RBF model:")
print(f" Train: {new_relative_p_error_train.item():e}")
print(f" Test: {new_relative_p_error_test.item():e}")
# ## What's next?
#
# Congratulations on completing the **PINA** tutorial on building and using a custom POD class! Now you can try:
#
# 1. Varying the inputs of the model (for a list of the supported RB functions look at the `rbf_layer.py` file in `pina.layers`)
#
# 2. Changing the POD model, for example using Artificial Neural Networks. For a more in depth overview of POD-NN and a comparison with the POD-RBF model already shown, look at [Tutorial: Reduced order model (POD-RBF or POD-NN) for parametric problems](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial9/tutorial.ipynb)
#
# 3. Building your own classes or adapt the one shown to other datasets/problems

602
tutorials/tutorial15/tutorial.ipynb vendored Normal file
View File

File diff suppressed because one or more lines are too long

574
tutorials/tutorial16/tutorial.ipynb vendored Normal file
View File

File diff suppressed because one or more lines are too long

841
tutorials/tutorial17/tutorial.ipynb vendored Normal file
View File

File diff suppressed because one or more lines are too long

443
tutorials/tutorial18/tutorial.ipynb vendored Normal file
View File

@@ -0,0 +1,443 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "6f71ca5c",
"metadata": {},
"source": [
"# Tutorial: Introduction to Solver classes\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial18/tutorial.ipynb)\n",
"\n",
"In this tutorial, we will explore the Solver classes in PINA, that are the core components for optimizing models. Solvers are designed to manage and execute the optimization process, providing the flexibility to work with various types of neural networks and loss functions. We will show how to use this class to select and implement different solvers, such as Supervised Learning, Physics-Informed Neural Networks (PINNs), and Generative Learning solvers. By the end of this tutorial, you'll be equipped to easily choose and customize solvers for your own tasks, streamlining the model training process.\n",
"\n",
"## Introduction to Solvers\n",
"\n",
"[`Solvers`](https://mathlab.github.io/PINA/_rst/_code.html#solvers) are versatile objects in PINA designed to manage the training and optimization of machine learning models. They handle key components of the learning process, including:\n",
"\n",
"- Loss function minimization \n",
"- Model optimization (optimizer, schedulers)\n",
"- Validation and testing workflows\n",
"\n",
"PINA solvers are built on top of the [PyTorch Lightning `LightningModule`](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html), which provides a structured and scalable training framework. This allows solvers to leverage advanced features such as distributed training, early stopping, and logging — all with minimal setup.\n",
"\n",
"## Solvers Hierarchy: Single and MultiSolver\n",
"\n",
"PINA provides two main abstract interfaces for solvers, depending on whether the training involves a single model or multiple models. These interfaces define the base functionality that all specific solver implementations inherit from.\n",
"\n",
"### 1. [`SingleSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/solver_interface.html)\n",
"\n",
"This is the abstract base class for solvers that train **a single model**, such as in standard supervised learning or physics-informed training. All specific solvers (e.g., `SupervisedSolver`, `PINN`) inherit from this interface.\n",
"\n",
"**Arguments:**\n",
"- `problem` – The problem to be solved.\n",
"- `model` – The neural network model.\n",
"- `optimizer` – Defaults to `torch.optim.Adam` if not provided.\n",
"- `scheduler` – Defaults to `torch.optim.lr_scheduler.ConstantLR`.\n",
"- `weighting` – Optional loss weighting schema., see [here](https://mathlab.github.io/PINA/_rst/_code.html#losses-and-weightings). We weight already for you!\n",
"- `use_lt` – Whether to use LabelTensors as input.\n",
"\n",
"---\n",
"\n",
"### 2. [`MultiSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/multi_solver_interface.html)\n",
"\n",
"This is the abstract base class for solvers involving **multiple models**, such as in GAN architectures or ensemble training strategies. All multi-model solvers (e.g., `DeepEnsemblePINN`, `GAROM`) inherit from this interface.\n",
"\n",
"**Arguments:**\n",
"- `problem` – The problem to be solved.\n",
"- `models` – The model or models used for training.\n",
"- `optimizers` – Defaults to `torch.optim.Adam`.\n",
"- `schedulers` – Defaults to `torch.optim.lr_scheduler.ConstantLR`.\n",
"- `weightings` – Optional loss weighting schema, see [here](https://mathlab.github.io/PINA/_rst/_code.html#losses-and-weightings). We weight already for you!\n",
"- `use_lt` – Whether to use LabelTensors as input.\n",
"\n",
"---\n",
"\n",
"These base classes define the structure and behavior of solvers in PINA, allowing you to create customized training strategies while leveraging PyTorch Lightning's features under the hood. \n",
"\n",
"These classes are used to define the backbone, i.e. setting the problem, the model(s), the optimizer(s) and scheduler(s), but miss a key component the `optimization_cycle` method.\n",
"\n",
"\n",
"## Optimization Cycle\n",
"The `optimization_cycle` method is the core function responsible for computing losses for **all conditions** in a given training batch. Each condition (e.g. initial condition, boundary condition, PDE residual) contributes its own loss, which is tracked and returned in a dictionary. This method should return a dictionary mapping **condition names** to their respective **scalar loss values**.\n",
"\n",
"For supervised learning tasks, where each condition consists of an input-target pair, for example, the `optimization_cycle` may look like this:\n",
"\n",
"```python\n",
"def optimization_cycle(self, batch):\n",
" \"\"\"\n",
" The optimization cycle for Supervised solvers.\n",
" Computes loss for each condition in the batch.\n",
" \"\"\"\n",
" condition_loss = {}\n",
" for condition_name, data in batch:\n",
" condition_loss[condition_name] = self.loss_data(\n",
" input=data[\"input\"], target=data[\"target\"]\n",
" )\n",
" return condition_loss\n",
"```\n",
"In PINA, a **batch** is structured as a list of tuples, where each tuple corresponds to a specific training condition. Each tuple contains:\n",
"\n",
"- The **name of the condition**\n",
"- A **dictionary of data** associated with that condition\n",
"\n",
"for example:\n",
"\n",
"```python\n",
"batch = [\n",
" (\"condition1\", {\"input\": ..., \"target\": ...}),\n",
" (\"condition2\", {\"input\": ..., \"equation\": ...}),\n",
" (\"condition3\", {\"input\": ..., \"target\": ...}),\n",
"]\n",
"```\n",
"\n",
"Fortunately, you don't need to implement the `optimization_cycle` yourself in most cases — PINA already provides default implementations tailored to common solver types. These implementations are available through the solver interfaces and cover various training strategies.\n",
"\n",
"1. [`PINNInterface`](https://mathlab.github.io/PINA/_rst/solver/physics_informed_solver/pinn_interface.html) \n",
" Implements the optimization cycle for **physics-based solvers** (e.g., PDE residual minimization) as well as other useful methods to compute PDE residuals. \n",
" ➤ [View method](https://mathlab.github.io/PINA/_rst/solver/physics_informed_solver/pinn_interface.html#pina.solver.physics_informed_solver.pinn_interface.PINNInterface.optimization_cycle)\n",
"\n",
"2. [`SupervisedSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/supervised_solver/supervised_solver_interface.html) \n",
" Defines the optimization cycle for **supervised learning tasks**, including traditional regression and classification. \n",
" ➤ [View method](https://mathlab.github.io/PINA/_rst/solver/supervised_solver/supervised_solver_interface.html#pina.solver.supervised_solver.supervised_solver_interface.SupervisedSolverInterface.optimization_cycle)\n",
"\n",
"3. [`DeepEnsembleSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/ensemble_solver/ensemble_solver_interface.html) \n",
" Provides the optimization logic for **deep ensemble methods**, commonly used for uncertainty quantification or robustness. \n",
" ➤ [View method](https://mathlab.github.io/PINA/_rst/solver/ensemble_solver/ensemble_solver_interface.html#pina.solver.ensemble_solver.ensemble_solver_interface.DeepEnsembleSolverInterface.optimization_cycle)\n",
"\n",
"These ready-to-use implementations ensure that your solvers are properly structured and compatible with PINA’s training workflow. You can also inherit and override them to fit more specialized needs. They only require, the following arguments:\n",
"**Arguments:**\n",
"- `problem` – The problem to be solved.\n",
"- `loss` - The loss to be minimized\n",
"- `weightings` – Optional loss weighting schema.\n",
"- `use_lt` – Whether to use LabelTensors as input.\n",
"\n",
"## Structure a Solver with Multiple Inheritance:\n",
"\n",
"Thanks to PINA’s modular design, creating a custom solver is straightforward using **multiple inheritance**. You can combine different interfaces to define both the **optimization logic** and the **model structure**.\n",
"\n",
"- **`PINN` Solver**\n",
" - Inherits from: \n",
" - [`PINNInterface`](https://mathlab.github.io/PINA/_rst/solver/physics_informed_solver/pinn_interface.html) → physics-based optimization loop \n",
" - [`SingleSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/solver_interface.html) → training a single model\n",
"\n",
"- **`SupervisedSolver`**\n",
" - Inherits from: \n",
" - [`SupervisedSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/supervised_solver/supervised_solver_interface.html) → data-driven optimization loop \n",
" - [`SingleSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/solver_interface.html) → training a single model\n",
"\n",
"- **`GAROM`** (a variant of GAN)\n",
" - Inherits from: \n",
" - [`SupervisedSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/supervised_solver/supervised_solver_interface.html) → data-driven optimization loop \n",
" - [`MultiSolverInterface`](https://mathlab.github.io/PINA/_rst/solver/multi_solver_interface.html) → training multiple models (e.g., generator and discriminator)\n",
"\n",
"This structure promotes **code reuse** and **extensibility**, allowing you to quickly prototype new solver strategies by reusing core training and optimization logic.\n",
"\n",
"## Let's try to build some solvers!\n",
"\n",
"We will now start building a simple supervised solver in PINA. Let's first import useful modules! "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0981f1e9",
"metadata": {},
"outputs": [],
"source": [
"## routine needed to run the notebook on Google Colab\n",
"try:\n",
" import google.colab\n",
" IN_COLAB = True\n",
"except:\n",
" IN_COLAB = False\n",
"if IN_COLAB:\n",
" !pip install \"pina-mathlab[tutorial]\"\n",
"\n",
"import warnings\n",
"import torch\n",
"import matplotlib.pyplot as plt\n",
"\n",
"warnings.filterwarnings(\"ignore\")\n",
"\n",
"from pina import Trainer\n",
"from pina.solver import SingleSolverInterface, SupervisedSolverInterface\n",
"from pina.model import FeedForward\n",
"from pina.problem.zoo import SupervisedProblem"
]
},
{
"cell_type": "markdown",
"id": "7b91de38",
"metadata": {},
"source": [
"Since we are using only one model for this task, we will inherit from two base classes:\n",
"\n",
"- `SingleSolverInterface`: This ensures we are working with a single model.\n",
"- `SupervisedSolverInterface`: This allows us to use supervised learning strategies for training the model."
]
},
{
"cell_type": "code",
"execution_count": 50,
"id": "014bbd86",
"metadata": {},
"outputs": [],
"source": [
"class MyFirstSolver(SupervisedSolverInterface, SingleSolverInterface):\n",
" def __init__(\n",
" self,\n",
" problem,\n",
" model,\n",
" loss=None,\n",
" optimizer=None,\n",
" scheduler=None,\n",
" weighting=None,\n",
" use_lt=True,\n",
" ):\n",
" super().__init__(\n",
" model=model,\n",
" problem=problem,\n",
" loss=loss,\n",
" optimizer=optimizer,\n",
" scheduler=scheduler,\n",
" weighting=weighting,\n",
" use_lt=use_lt,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "b1b1e4c4",
"metadata": {},
"source": [
"By default, Python follows a specific method resolution order (MRO) when a class inherits from multiple parent classes. This means that the initialization (`__init__`) method is called based on the order of inheritance.\n",
"\n",
"Since we inherit from `SupervisedSolverInterface` first, Python will call the `__init__` method from `SupervisedSolverInterface` (initialize `problem`, `loss`, `weighting` and `use_lt`) before calling the `__init__` method from `SingleSolverInterface` (initialize `model`, `optimizer`, `scheduler`). This allows us to customize the initialization process for our custom solver. \n",
"\n",
"We will learn a very simple problem, try to learn $y=\\sin(x)$."
]
},
{
"cell_type": "code",
"execution_count": 51,
"id": "6f25d3a6",
"metadata": {},
"outputs": [],
"source": [
"# get the data\n",
"x = torch.linspace(0, torch.pi, 100).view(-1, 1)\n",
"y = torch.sin(x)\n",
"# build the problem\n",
"problem = SupervisedProblem(x, y)\n",
"# build the model\n",
"model = FeedForward(1, 1)"
]
},
{
"cell_type": "markdown",
"id": "9f7551bf",
"metadata": {},
"source": [
"If we now try to initialize the solver `MyFirstSolver` we will get the following error:\n",
"\n",
"```python\n",
"---------------------------------------------------------------------------\n",
"TypeError Traceback (most recent call last)\n",
"Cell In[41], line 1\n",
"----> 1 MyFirstSolver(problem, model)\n",
"\n",
"TypeError: Can't instantiate abstract class MyFirstSolver with abstract method loss_data\n",
"```\n",
"\n",
"### Data and Physics Loss\n",
"The error above is because in PINA, all solvers must specify how to compute the loss during training. There are two main types of losses that can be computed, depending on the nature of the problem:\n",
"\n",
"1. **`loss_data`**: Computes the **data loss** between the model's output and the true solution. This is typically used in **supervised learning** setups, where we have ground truth data to compare the model's predictions. It expects some `input` (tensor, graph, ...) and a `target` (tensor, graph, ...)\n",
" \n",
"2. **`loss_phys`**: Computes the **physics loss** for **physics-informed solvers** (PINNs). This loss is based on the residuals of the governing equations that model physical systems, enforcing the equations during training. It expects some `samples` (`LabelTensor`) and an `equation` (`Equation`)\n",
"\n",
"Therefore our implementation becomes:"
]
},
{
"cell_type": "code",
"execution_count": 52,
"id": "336e8060",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"GPU available: True (mps), used: False\n",
"TPU available: False, using: 0 TPU cores\n",
"HPU available: False, using: 0 HPUs\n",
"\n",
" | Name | Type | Params | Mode \n",
"----------------------------------------------------\n",
"0 | _pina_models | ModuleList | 481 | train\n",
"1 | _loss_fn | MSELoss | 0 | train\n",
"----------------------------------------------------\n",
"481 Trainable params\n",
"0 Non-trainable params\n",
"481 Total params\n",
"0.002 Total estimated model params size (MB)\n",
"9 Modules in train mode\n",
"0 Modules in eval mode\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "d6d009cd7efb4c76ba2115f828e46dc8",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=500` reached.\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "c7679ddd66e748dbaef644592c18a010",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Testing: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
" Test metric DataLoader 0\n",
"────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
" test_loss 0.006782823242247105\n",
"────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n"
]
}
],
"source": [
"class MyFirstSolver(SupervisedSolverInterface, SingleSolverInterface):\n",
" def __init__(\n",
" self,\n",
" problem,\n",
" model,\n",
" loss=None,\n",
" optimizer=None,\n",
" scheduler=None,\n",
" weighting=None,\n",
" use_lt=True,\n",
" ):\n",
" super().__init__(\n",
" model=model,\n",
" problem=problem,\n",
" loss=loss,\n",
" optimizer=optimizer,\n",
" scheduler=scheduler,\n",
" weighting=weighting,\n",
" use_lt=use_lt,\n",
" )\n",
"\n",
" def loss_data(self, input, target):\n",
" # self.loss stores the loss passed in the init\n",
" network_output = self.forward(input)\n",
" return self.loss(network_output, target)\n",
"\n",
"\n",
"# initialize (we use plain tensors!)\n",
"solver = MyFirstSolver(problem, model, use_lt=False)\n",
"\n",
"# simple training\n",
"trainer = Trainer(\n",
" solver, max_epochs=500, train_size=0.8, test_size=0.2, accelerator=\"cpu\"\n",
")\n",
"trainer.train()\n",
"_ = trainer.test()"
]
},
{
"cell_type": "markdown",
"id": "9d346aac",
"metadata": {},
"source": [
"## A Summary on Solvers\n",
"\n",
"Solvers in PINA play a critical role in training and optimizing machine learning models, especially when working with complex problems like physics-informed neural networks (PINNs) or standard supervised learning. Here’s a quick recap of the key concepts we've covered:\n",
"\n",
"1. **Solver Interfaces**:\n",
" - **`SingleSolverInterface`**: For solvers using one model (e.g., a standard supervised solver or a single physics-informed model).\n",
" - **`MultiSolverInterface`**: For solvers using multiple models (e.g., Generative Adversarial Networks (GANs)).\n",
"\n",
"2. **Loss Functions**:\n",
" - **`loss_data`**: Computes the loss for supervised solvers, typically comparing the model's predictions to the true targets.\n",
" - **`loss_phys`**: Computes the physics loss for PINNs, typically using the residuals of a physical equation to enforce consistency with the physics of the system.\n",
"\n",
"3. **Custom Solver Implementation**:\n",
" - You can create custom solvers by inheriting from base classes such as `SingleSolverInterface`. The **`optimization_cycle`** method must be implemented to define how to compute the loss for each batch.\n",
" - `SupervisedSolverInterface`, `PINNInterface` already implement the `optimization_cycle` for you!\n",
"\n",
"\n",
"By understanding and implementing solvers in PINA, you can build flexible, scalable models that can be optimized both with traditional supervised learning techniques and more specialized, physics-based methods."
]
},
{
"cell_type": "markdown",
"id": "487c1d47",
"metadata": {},
"source": [
"## What's Next?\n",
"\n",
"Congratulations on completing the tutorial on solver classes! Now that you have a solid foundation, here are a few directions you can explore:\n",
"\n",
"\n",
"1. **Physics Solvers**: Try to implement your own physics-based solver. Can you do it? This will involve creating a custom loss function that enforces the physics of a given problem insied `loss_phys`.\n",
"\n",
"2. **Multi-Model Solvers**: Take it to the next level by exploring multi-model solvers, such as GANs or ensemble-based solvers. You could implement and train models that combine the strengths of multiple neural networks.\n",
"\n",
"3. **...and many more!**: There are countless directions to further explore, try to look at our `solver` for example!\n",
"\n",
"For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "pina",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.21"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

604
tutorials/tutorial19/tutorial.ipynb vendored Normal file
View File

@@ -0,0 +1,604 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "6f71ca5c",
"metadata": {},
"source": [
"# Tutorial: Data structure for SciML: `Tensor`, `LabelTensor`, `Data` and `Graph`\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial19/tutorial.ipynb)\n",
"\n",
"In this tutorial, we’ll quickly go through the basics of Data Structures for Scientific Machine Learning, convering:\n",
"1. **PyTorch Tensors** / **PINA LabelTensors**\n",
"2. **PyTorch Geometric Data** / **PINA Graph**\n",
"\n",
"first let's import the data structures we will use!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0981f1e9",
"metadata": {},
"outputs": [],
"source": [
"## routine needed to run the notebook on Google Colab\n",
"try:\n",
" import google.colab\n",
" IN_COLAB = True\n",
"except:\n",
" IN_COLAB = False\n",
"if IN_COLAB:\n",
" !pip install \"pina-mathlab[tutorial]\"\n",
"\n",
"import warnings\n",
"import torch\n",
"from torch_geometric.data import Data\n",
"\n",
"warnings.filterwarnings(\"ignore\")\n",
"\n",
"from pina import LabelTensor, Graph"
]
},
{
"cell_type": "markdown",
"id": "8afae117",
"metadata": {},
"source": [
"## PyTorch Tensors\n",
"\n",
"A **tensor** is a multi-dimensional matrix used for storing and manipulating data in PyTorch. It's the basic building block for all computations in PyTorch, including deep learning models.\n",
"\n",
"You can create a tensor in several ways:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "6558c37a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([1, 2, 3, 4])\n",
"tensor([[0., 0., 0.],\n",
" [0., 0., 0.]])\n",
"tensor([[1., 1., 1.],\n",
" [1., 1., 1.]])\n",
"tensor([[-0.4420, 0.9948, 0.3727],\n",
" [-0.2328, 0.0719, -0.1929]])\n"
]
}
],
"source": [
"# Creating a tensor from a list\n",
"tensor_1 = torch.tensor([1, 2, 3, 4])\n",
"print(tensor_1)\n",
"\n",
"# Creating a tensor of zeros\n",
"tensor_zeros = torch.zeros(2, 3) # 2x3 tensor of zeros\n",
"print(tensor_zeros)\n",
"\n",
"# Creating a tensor of ones\n",
"tensor_ones = torch.ones(2, 3) # 2x3 tensor of ones\n",
"print(tensor_ones)\n",
"\n",
"# Creating a random tensor\n",
"tensor_random = torch.randn(2, 3) # 2x3 tensor with random values\n",
"print(tensor_random)"
]
},
{
"cell_type": "markdown",
"id": "f015f61d",
"metadata": {},
"source": [
"### Basic Tensor Operations\n",
"Tensors support a variety of operations, such as element-wise arithmetic, matrix operations, and more:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "d5369bf3",
"metadata": {},
"outputs": [],
"source": [
"# Addition\n",
"sum_tensor = tensor_1 + tensor_1\n",
"\n",
"# Matrix multiplication\n",
"result = torch.matmul(tensor_zeros, tensor_ones.T)\n",
"\n",
"# Element-wise multiplication\n",
"elementwise_prod = tensor_1 * tensor_1"
]
},
{
"cell_type": "markdown",
"id": "619364cc",
"metadata": {},
"source": [
"### Device Management\n",
"PyTorch allows you to move tensors to different devices (CPU or GPU). For instance:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "6b82839b",
"metadata": {},
"outputs": [],
"source": [
"# Move tensor to GPU\n",
"if torch.cuda.is_available():\n",
" tensor_gpu = tensor_1.cuda()"
]
},
{
"cell_type": "markdown",
"id": "75fd37ca",
"metadata": {},
"source": [
"To know more about PyTorch Tensors, see the dedicated tutorial done by the PyTorch team [here](https://pytorch.org/tutorials/beginner/introyt/tensors_deeper_tutorial.html)."
]
},
{
"cell_type": "markdown",
"id": "6073dc6d",
"metadata": {},
"source": [
"## Label Tensors\n",
"\n",
"In scientific machine learning, especially when working with **Physics-Informed Neural Networks (PINNs)**, handling tensors effectively is crucial. Often, we deal with many indices that represent physical quantities such as spatial and temporal coordinates, making it vital to ensure we use the correct indexing.\n",
"\n",
"For instance, in PINNs, if the wrong index is used to represent the coordinates of a physical domain, it could lead to incorrect calculations of derivatives, integrals, or residuals. This can significantly affect the accuracy and correctness of the model.\n",
"\n",
"### What are Label Tensors?\n",
"\n",
"**Label Tensors** are a specialized type of tensor used to keep track of indices that represent specific labels. Similar to torch tensor we can perform operation, but the slicing is simplified by using indeces:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "25e8353e",
"metadata": {},
"outputs": [],
"source": [
"# standard torch tensor\n",
"tensor = torch.randn(10, 2)\n",
"\n",
"# PINA LabelTensor\n",
"label_tensor = LabelTensor(tensor, labels=[\"x\", \"y\"])"
]
},
{
"cell_type": "markdown",
"id": "bb21b45c",
"metadata": {},
"source": [
"The label tensor is initialized by passing the tensor, and a set of labels. Specifically, the labels must match the following conditions:\n",
"\n",
"- At each dimension, the number of labels must match the size of the dimension.\n",
"- At each dimension, the labels must be unique.\n",
"\n",
"For example:\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "0e9dc23e",
"metadata": {},
"outputs": [],
"source": [
"# full labels\n",
"tensor = LabelTensor(\n",
" torch.rand((2000, 3)), {1: {\"name\": \"space\", \"dof\": [\"a\", \"b\", \"c\"]}}\n",
")\n",
"# if you index the last column you can simply pass a list\n",
"tensor = LabelTensor(torch.rand((2000, 3)), [\"a\", \"b\", \"c\"])"
]
},
{
"cell_type": "markdown",
"id": "cfe2d8dd",
"metadata": {},
"source": [
"You can access last column labels by `.labels` attribute, or using `.full_labels` to access all labels"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "235b92d4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor.labels=['a', 'b', 'c']\n",
"tensor.full_labels={0: {'dof': range(0, 2000), 'name': 0}, 1: {'dof': ['a', 'b', 'c'], 'name': 1}}\n"
]
}
],
"source": [
"print(f'{tensor.labels=}')\n",
"print(f\"{tensor.full_labels=}\")"
]
},
{
"cell_type": "markdown",
"id": "e8b230ea",
"metadata": {},
"source": [
"### Label Tensors slicing\n",
"\n",
"One of the powerful features of label tensors is the ability to easily slice and extract specific parts of the tensor based on labels, just like regular PyTorch tensors but with the ease of labels. \n",
"\n",
"Here’s how slicing works with label tensors. Suppose we have a label tensor that contains both spatial and temporal data, and we want to slice specific parts of this data to focus on certain time intervals or spatial regions."
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "45365ea8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Tensor:\n",
" tensor([[0.0000, 0.0000],\n",
" [1.0000, 0.5000],\n",
" [2.0000, 1.0000],\n",
" [3.0000, 1.5000]])\n",
"Torch methods can be used, label_tensor.shape=torch.Size([4, 2])\n",
"also label_tensor.requires_grad=False \n",
"\n",
"We can slice with labels: \n",
" label_tensor[\"x\"]=LabelTensor([[0.],\n",
" [1.],\n",
" [2.],\n",
" [3.]])\n",
"Similarly to: \n",
" label_tensor[:, 0]=LabelTensor([[0.],\n",
" [1.],\n",
" [2.],\n",
" [3.]])\n"
]
}
],
"source": [
"# Create a label tensor containing spatial and temporal coordinates\n",
"x = torch.tensor([0.0, 1.0, 2.0, 3.0]) # Spatial coordinates\n",
"t = torch.tensor([0.0, 0.5, 1.0, 1.5]) # Time coordinates\n",
"\n",
"# Combine x and t into a label tensor (2D tensor)\n",
"tensor = torch.stack([x, t], dim=-1) # Shape: [4, 2]\n",
"print(\"Tensor:\\n\", tensor)\n",
"\n",
"# Build the LabelTensor\n",
"label_tensor = LabelTensor(tensor, [\"x\", \"t\"])\n",
"\n",
"print(f\"Torch methods can be used, {label_tensor.shape=}\")\n",
"print(f\"also {label_tensor.requires_grad=} \\n\")\n",
"print(f'We can slice with labels: \\n {label_tensor[\"x\"]=}')\n",
"print(f\"Similarly to: \\n {label_tensor[:, 0]=}\")"
]
},
{
"cell_type": "markdown",
"id": "ea4adc6e",
"metadata": {},
"source": [
"You can do more complex slicing by using the extract method. For example:"
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "caec2d14",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Extract labels: label_tensor.extract({\"points\" : [0, 2]})=LabelTensor([[[0., 0.]],\n",
" [[2., 1.]]])\n",
"Similar to: label_tensor[slice(0, 4, 2), :]=LabelTensor([[[0., 0.]],\n",
" [[2., 1.]]])\n"
]
}
],
"source": [
"label_tensor = LabelTensor(\n",
" tensor,\n",
" {\n",
" 0: {\"dof\": range(4), \"name\": \"points\"},\n",
" 1: {\"dof\": [\"x\", \"t\"], \"name\": \"coords\"},\n",
" },\n",
")\n",
"\n",
"print(f'Extract labels: {label_tensor.extract({\"points\" : [0, 2]})=}')\n",
"print(f'Similar to: {label_tensor[slice(0, 4, 2), :]=}')"
]
},
{
"cell_type": "markdown",
"id": "331d6080",
"metadata": {},
"source": [
"## PyTorch Geometric Data\n",
"PyTorch Geometric (PyG) extends PyTorch to handle graph-structured data. It provides utilities to represent graphs and perform graph-based learning tasks such as node classification, graph classification, and more.\n",
"\n",
"### Graph Data Structure\n",
"PyTorch Geometric uses a custom `Data` object to store graph data. The `Data` object contains the following attributes:\n",
"\n",
"- **x**: Node features (tensor of shape `[num_nodes, num_features]`)\n",
"\n",
"- **edge_index**: Edge indices (tensor of shape `[2, num_edges]`), representing the graph's connectivity\n",
"\n",
"- **edge_attr**: Edge features (optional, tensor of shape `[num_edges, num_edge_features]`)\n",
"\n",
"- **y**: Target labels for nodes/graphs (optional)"
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "9427b274",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Data(x=[2, 3], edge_index=[2, 2])\n"
]
}
],
"source": [
"# Node features: [2 nodes, 3 features]\n",
"x = torch.tensor([[1, 2, 3], [4, 5, 6]], dtype=torch.float)\n",
"\n",
"# Edge indices: representing a graph with two edges (node 0 to node 1, node 1 to node 0)\n",
"edge_index = torch.tensor([[0, 1], [1, 0]], dtype=torch.long)\n",
"\n",
"# Create a PyG data object\n",
"data = Data(x=x, edge_index=edge_index)\n",
"\n",
"print(data)"
]
},
{
"cell_type": "markdown",
"id": "fde2dcc7",
"metadata": {},
"source": [
"Once you have your graph in a Data object, you can easily perform graph-based operations using PyTorch Geometric’s built-in functions:"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "bdebb42e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1., 2., 3.],\n",
" [4., 5., 6.]])\n",
"tensor([[0, 1],\n",
" [1, 0]])\n",
"tensor([[ 7.4528, -3.2700],\n",
" [ 7.4528, -3.2700]], grad_fn=<AddBackward0>)\n"
]
}
],
"source": [
"# Accessing node features\n",
"print(data.x) # Node features\n",
"\n",
"# Accessing edge list\n",
"print(data.edge_index) # Edge indices\n",
"\n",
"# Applying Graph Convolution (Graph Neural Networks - GCN)\n",
"from torch_geometric.nn import GCNConv\n",
"\n",
"# Define a simple GCN layer\n",
"conv = GCNConv(3, 2) # 3 input features, 2 output features\n",
"out = conv(data.x, data.edge_index)\n",
"print(out) # Output node features after applying GCN"
]
},
{
"cell_type": "markdown",
"id": "287a0d4f",
"metadata": {},
"source": [
"## PINA Graph\n",
"\n",
"If you've understood Label Tensors and Data in PINA, then you're well on your way to grasping how **PINA Graph** works. Simply put, a **Graph** in PINA is a `Data` object with extra methods for handling label tensors. We highly suggest to use `Graph` instead of `Data` in PINA, expecially when using label tensors."
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "27f5c9ac",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Graph(x=[2, 3], edge_index=[2, 2])\n",
"tensor([[1., 2., 3.],\n",
" [4., 5., 6.]])\n",
"tensor([[0, 1],\n",
" [1, 0]])\n",
"tensor([[-0.0606, 5.7191],\n",
" [-0.0606, 5.7191]], grad_fn=<AddBackward0>)\n"
]
}
],
"source": [
"# Node features: [2 nodes, 3 features]\n",
"x = torch.tensor([[1, 2, 3], [4, 5, 6]], dtype=torch.float)\n",
"\n",
"# Edge indices: representing a graph with two edges (node 0 to node 1, node 1 to node 0)\n",
"edge_index = torch.tensor([[0, 1], [1, 0]], dtype=torch.long)\n",
"\n",
"# Create a PINA graph object (similar to PyG)\n",
"data = Graph(x=x, edge_index=edge_index)\n",
"\n",
"print(data)\n",
"\n",
"# Accessing node features\n",
"print(data.x) # Node features\n",
"\n",
"# Accessing edge list\n",
"print(data.edge_index) # Edge indices\n",
"\n",
"# Applying Graph Convolution (Graph Neural Networks - GCN)\n",
"from torch_geometric.nn import GCNConv\n",
"\n",
"# Define a simple GCN layer\n",
"conv = GCNConv(3, 2) # 3 input features, 2 output features\n",
"out = conv(data.x, data.edge_index)\n",
"print(out) # Output node features after applying GCN"
]
},
{
"cell_type": "markdown",
"id": "6ee7cc14",
"metadata": {},
"source": [
"But we can also use labeltensors...."
]
},
{
"cell_type": "code",
"execution_count": 40,
"id": "3866a8ae",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Graph(x=[2, 3], edge_index=[2, 2])\n",
"Graph(x=[2, 1], edge_index=[2, 2])\n"
]
}
],
"source": [
"# Node features: [2 nodes, 3 features]\n",
"x = LabelTensor(torch.tensor([[1, 2, 3], [4, 5, 6]], dtype=torch.float),\n",
" [\"a\", \"b\", \"c\"])\n",
"\n",
"# Edge indices: representing a graph with two edges (node 0 to node 1, node 1 to node 0)\n",
"edge_index = torch.tensor([[0, 1], [1, 0]], dtype=torch.long)\n",
"\n",
"# Create a PINA graph object (similar to PyG)\n",
"data = Graph(x=x, edge_index=edge_index)\n",
"\n",
"print(data)\n",
"print(data.extract(attr=\"x\", labels=[\"a\"])) # here we extract 1 feature"
]
},
{
"cell_type": "markdown",
"id": "7a2ef072",
"metadata": {},
"source": [
"In PINA Conditions, you always need to pass a list of `Graph` or `Data`, see [here]() for details. In case you are loading a PyG dataset remember to put it in this format!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c8edb68f",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Downloading https://deepchemdata.s3-us-west-1.amazonaws.com/datasets/qm7b.mat\n",
"Processing...\n",
"Done!\n"
]
},
{
"data": {
"text/plain": [
"Data(edge_index=[2, 324], edge_attr=[324], y=[1, 14], num_nodes=18)"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from torch_geometric.datasets import QM7b\n",
"\n",
"dataset = QM7b(root=\"./tutorial_logs\").shuffle()\n",
"\n",
"# save the dataset\n",
"input_ = [data for data in dataset]\n",
"input_[0]"
]
},
{
"cell_type": "markdown",
"id": "487c1d47",
"metadata": {},
"source": [
"## What's Next?\n",
"\n",
"Congratulations on completing the tutorials on the **PINA Data Structures**! You now have a solid foundation in using the different data structures within PINA, such as **Tensors**, **Label Tensors**, and **Graphs**. Here are some exciting next steps you can take to continue your learning journey:\n",
"\n",
"1. **Deep Dive into Label Tensors**: Check the documentation of [`LabelTensor`](https://mathlab.github.io/PINA/_rst/label_tensor.html) to learn more about the available methods.\n",
"\n",
"2. **Working with Graphs in PINA**: In PINA we implement many graph structures, e.g. `KNNGraph`, `RadiusGraph`, .... see [here](https://mathlab.github.io/PINA/_rst/_code.html#graphs-structures) for further details.\n",
"\n",
"3. **...and many more!**: Consider exploring `LabelTensor` for PINNs!\n",
"\n",
"For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "pina",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.21"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

205
tutorials/tutorial2/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

360
tutorials/tutorial2/tutorial.py vendored
View File

@@ -1,360 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Two dimensional Poisson problem using Extra Features Learning
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial2/tutorial.ipynb)
#
# This tutorial presents how to solve with Physics-Informed Neural Networks (PINNs) a 2D Poisson problem with Dirichlet boundary conditions. We will train with standard PINN's training, and with extrafeatures. For more insights on extrafeature learning please read [*An extended physics informed neural network for preliminary analysis of parametric optimal control problems*](https://www.sciencedirect.com/science/article/abs/pii/S0898122123002018).
#
# First of all, some useful imports.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import torch
import matplotlib.pyplot as plt
import warnings
from pina import LabelTensor, Trainer
from pina.model import FeedForward
from pina.solver import PINN
from torch.nn import Softplus
warnings.filterwarnings("ignore")
# ## The problem definition
# The two-dimensional Poisson problem is mathematically written as:
# \begin{equation}
# \begin{cases}
# \Delta u = 2\pi^2\sin{(\pi x)} \sin{(\pi y)} \text{ in } D, \\
# u = 0 \text{ on } \Gamma_1 \cup \Gamma_2 \cup \Gamma_3 \cup \Gamma_4,
# \end{cases}
# \end{equation}
# where $D$ is a square domain $[0,1]^2$, and $\Gamma_i$, with $i=1,...,4$, are the boundaries of the square.
#
# The Poisson problem is written in **PINA** code as a class. The equations are written as *conditions* that should be satisfied in the corresponding domains. The *solution*
# is the exact solution which will be compared with the predicted one. If interested in how to write problems see [this tutorial](https://mathlab.github.io/PINA/_rst/tutorials/tutorial1/tutorial.html).
#
# We will directly import the problem from `pina.problem.zoo`, which contains a vast list of PINN problems and more.
# In[2]:
from pina.problem.zoo import Poisson2DSquareProblem as Poisson
# initialize the problem
problem = Poisson()
# print the conditions
print(
f"The problem is made of {len(problem.conditions.keys())} conditions: \n"
f"They are: {list(problem.conditions.keys())}"
)
# let's discretise the domain
problem.discretise_domain(30, "grid", domains=["D"])
problem.discretise_domain(
100,
"grid",
domains=["g1", "g2", "g3", "g4"],
)
# ## Solving the problem with standard PINNs
# After the problem, the feed-forward neural network is defined, through the class `FeedForward`. This neural network takes as input the coordinates (in this case $x$ and $y$) and provides the unkwown field of the Poisson problem. The residual of the equations are evaluated at several sampling points and the loss minimized by the neural network is the sum of the residuals.
#
# In this tutorial, the neural network is composed by two hidden layers of 10 neurons each, and it is trained for 1000 epochs with a learning rate of 0.006 and $l_2$ weight regularization set to $10^{-8}$. These parameters can be modified as desired. We set the `train_size` to 0.8 and `test_size` to 0.2, this mean that the discretised points will be divided in a 80%-20% fashion, where 80% will be used for training and the remaining 20% for testing.
# In[3]:
# make model + solver + trainer
from pina.optim import TorchOptimizer
model = FeedForward(
layers=[10, 10],
func=Softplus,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
pinn = PINN(
problem,
model,
optimizer=TorchOptimizer(torch.optim.Adam, lr=0.006, weight_decay=1e-8),
)
trainer_base = Trainer(
solver=pinn, # setting the solver, i.e. PINN
max_epochs=1000, # setting max epochs in training
accelerator="cpu", # we train on cpu, also other are available
enable_model_summary=False, # model summary statistics not printed
train_size=0.8, # set train size
val_size=0.0, # set validation size
test_size=0.2, # set testing size
shuffle=True, # shuffle the data
)
# train
trainer_base.train()
# Now we plot the results using `matplotlib`.
# The solution predicted by the neural network is plotted on the left, the exact one is represented at the center and on the right the error between the exact and the predicted solutions is showed.
# In[4]:
@torch.no_grad()
def plot_solution(solver):
# get the problem
problem = solver.problem
# get spatial points
spatial_samples = problem.spatial_domain.sample(30, "grid")
# compute pinn solution, true solution and absolute difference
data = {
"PINN solution": solver(spatial_samples),
"True solution": problem.solution(spatial_samples),
"Absolute Difference": torch.abs(
solver(spatial_samples) - problem.solution(spatial_samples)
),
}
# plot the solution
for idx, (title, field) in enumerate(data.items()):
plt.subplot(1, 3, idx + 1)
plt.title(title)
plt.tricontourf( # convert to torch tensor + flatten
spatial_samples.extract("x").tensor.flatten(),
spatial_samples.extract("y").tensor.flatten(),
field.tensor.flatten(),
)
plt.colorbar(), plt.tight_layout()
# Here the solution:
# In[5]:
plt.figure(figsize=(12, 6))
plot_solution(solver=pinn)
# As you can see the solution is not very accurate, in what follows we will use **Extra Feature** as introduced in [*An extended physics informed neural network for preliminary analysis of parametric optimal control problems*](https://www.sciencedirect.com/science/article/abs/pii/S0898122123002018) to boost the training accuracy. Of course, even extra training will benefit, this tutorial is just to show that convergence using Extra Features is usally faster.
# ## Solving the problem with extra-features PINNs
# Now, the same problem is solved in a different way.
# A new neural network is now defined, with an additional input variable, named extra-feature, which coincides with the forcing term in the Laplace equation.
# The set of input variables to the neural network is:
#
# \begin{equation}
# [x, y, k(x, y)], \text{ with } k(x, y)= 2\pi^2\sin{(\pi x)}\sin{(\pi y)},
# \end{equation}
#
# where $x$ and $y$ are the spatial coordinates and $k(x, y)$ is the added feature which is equal to the forcing term.
#
# This feature is initialized in the class `SinSin`, which is a simple `torch.nn.Module`. After declaring such feature, we can just adjust the `FeedForward` class by creating a subclass `FeedForwardWithExtraFeatures` with an adjusted forward method and the additional attribute `extra_features`.
#
# Finally, we perform the same training as before: the problem is `Poisson`, the network is composed by the same number of neurons and optimizer parameters are equal to previous test, the only change is the new extra feature.
# In[6]:
class SinSin(torch.nn.Module):
"""Feature: sin(x)*sin(y)"""
def __init__(self):
super().__init__()
def forward(self, pts):
x, y = pts.extract(["x"]), pts.extract(["y"])
f = 2 * torch.pi**2 * torch.sin(x * torch.pi) * torch.sin(y * torch.pi)
return LabelTensor(f, ["feat"])
class FeedForwardWithExtraFeatures(FeedForward):
def __init__(self, *args, extra_features, **kwargs):
super().__init__(*args, **kwargs)
self.extra_features = extra_features
def forward(self, x):
extra_feature = self.extra_features(x) # we append extra features
x = x.append(extra_feature)
return super().forward(x)
model_feat = FeedForwardWithExtraFeatures(
input_dimensions=len(problem.input_variables) + 1,
output_dimensions=len(problem.output_variables),
func=Softplus,
layers=[10, 10],
extra_features=SinSin(),
)
pinn_feat = PINN(
problem,
model_feat,
optimizer=TorchOptimizer(torch.optim.Adam, lr=0.006, weight_decay=1e-8),
)
trainer_feat = Trainer(
solver=pinn_feat, # setting the solver, i.e. PINN
max_epochs=1000, # setting max epochs in training
accelerator="cpu", # we train on cpu, also other are available
enable_model_summary=False, # model summary statistics not printed
train_size=0.8, # set train size
val_size=0.0, # set validation size
test_size=0.2, # set testing size
shuffle=True, # shuffle the data
)
trainer_feat.train()
# The predicted and exact solutions and the error between them are represented below.
# We can easily note that now our network, having almost the same condition as before, is able to reach additional order of magnitudes in accuracy.
# In[7]:
plt.figure(figsize=(12, 6))
plot_solution(solver=pinn_feat)
# ## Solving the problem with learnable extra-features PINNs
# We can still do better!
#
# Another way to exploit the extra features is the addition of learnable parameter inside them.
# In this way, the added parameters are learned during the training phase of the neural network. In this case, we use:
#
# \begin{equation}
# k(x, \mathbf{y}) = \beta \sin{(\alpha x)} \sin{(\alpha y)},
# \end{equation}
#
# where $\alpha$ and $\beta$ are the abovementioned parameters.
# Their implementation is quite trivial: by using the class `torch.nn.Parameter` we cam define all the learnable parameters we need, and they are managed by `autograd` module!
# In[8]:
class SinSinAB(torch.nn.Module):
""" """
def __init__(self):
super().__init__()
self.alpha = torch.nn.Parameter(torch.tensor([1.0]))
self.beta = torch.nn.Parameter(torch.tensor([1.0]))
def forward(self, x):
t = (
self.beta
* torch.sin(self.alpha * x.extract(["x"]) * torch.pi)
* torch.sin(self.alpha * x.extract(["y"]) * torch.pi)
)
return LabelTensor(t, ["b*sin(a*x)sin(a*y)"])
# make model + solver + trainer
model_learn = FeedForwardWithExtraFeatures(
input_dimensions=len(problem.input_variables)
+ 1, # we add one as also we consider the extra feature dimension
output_dimensions=len(problem.output_variables),
func=Softplus,
layers=[10, 10],
extra_features=SinSinAB(),
)
pinn_learn = PINN(
problem,
model_learn,
optimizer=TorchOptimizer(torch.optim.Adam, lr=0.006, weight_decay=1e-8),
)
trainer_learn = Trainer(
solver=pinn_learn, # setting the solver, i.e. PINN
max_epochs=1000, # setting max epochs in training
accelerator="cpu", # we train on cpu, also other are available
enable_model_summary=False, # model summary statistics not printed
train_size=0.8, # set train size
val_size=0.0, # set validation size
test_size=0.2, # set testing size
shuffle=True, # shuffle the data
)
# train
trainer_learn.train()
# Umh, the final loss is not appreciabily better than previous model (with static extra features), despite the usage of learnable parameters. This is mainly due to the over-parametrization of the network: there are many parameter to optimize during the training, and the model in unable to understand automatically that only the parameters of the extra feature (and not the weights/bias of the FFN) should be tuned in order to fit our problem. A longer training can be helpful, but in this case the faster way to reach machine precision for solving the Poisson problem is removing all the hidden layers in the `FeedForward`, keeping only the $\alpha$ and $\beta$ parameters of the extra feature.
# In[9]:
# make model + solver + trainer
model_learn = FeedForwardWithExtraFeatures(
layers=[],
func=Softplus,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables) + 1,
extra_features=SinSinAB(),
)
pinn_learn = PINN(
problem,
model_learn,
optimizer=TorchOptimizer(torch.optim.Adam, lr=0.006, weight_decay=1e-8),
)
trainer_learn = Trainer(
solver=pinn_learn, # setting the solver, i.e. PINN
max_epochs=1000, # setting max epochs in training
accelerator="cpu", # we train on cpu, also other are available
enable_model_summary=False, # model summary statistics not printed
train_size=0.8, # set train size
val_size=0.0, # set validation size
test_size=0.2, # set testing size
shuffle=True, # shuffle the data
)
# train
trainer_learn.train()
# In such a way, the model is able to reach a very high accuracy!
# Of course, this is a toy problem for understanding the usage of extra features: similar precision could be obtained if the extra features are very similar to the true solution. The analyzed Poisson problem shows a forcing term very close to the solution, resulting in a perfect problem to address with such an approach.
# We conclude here by showing the test error for the analysed methodologies: the standard PINN, PINN with extra features, and PINN with learnable extra features.
# In[12]:
# test error base pinn
print("PINN")
trainer_base.test()
# test error extra features pinn
print("PINN with extra features")
trainer_feat.test()
# test error learnable extra features pinn
print("PINN with learnable extra features")
_ = trainer_learn.test()
# ## What's next?
#
# Congratulations on completing the two dimensional Poisson tutorial of **PINA**! There are multiple directions you can go now:
#
# 1. Train the network for longer or with different layer sizes and assert the finaly accuracy
#
# 2. Propose new types of extrafeatures and see how they affect the learning
#
# 3. Exploit extrafeature training in more complex problems
#
# 4. Many more...

463
tutorials/tutorial20/tutorial.ipynb vendored Normal file
View File

File diff suppressed because one or more lines are too long

459
tutorials/tutorial21/tutorial.ipynb vendored Normal file
View File

File diff suppressed because one or more lines are too long

149
tutorials/tutorial3/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

336
tutorials/tutorial3/tutorial.py vendored
View File

@@ -1,336 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Two dimensional Wave problem with hard constraint
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial3/tutorial.ipynb)
#
# In this tutorial we present how to solve the wave equation using hard constraint PINNs. For doing so we will build a costum `torch` model and pass it to the `PINN` solver.
#
# First of all, some useful imports.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import torch
import matplotlib.pyplot as plt
import warnings
from pina import Condition, LabelTensor, Trainer
from pina.problem import SpatialProblem, TimeDependentProblem
from pina.operator import laplacian, grad
from pina.domain import CartesianDomain
from pina.solver import PINN
from pina.equation import Equation, FixedValue
from pina.callback import MetricTracker
warnings.filterwarnings("ignore")
# ## The problem definition
# The problem is written in the following form:
#
# \begin{equation}
# \begin{cases}
# \Delta u(x,y,t) = \frac{\partial^2}{\partial t^2} u(x,y,t) \quad \text{in } D, \\\\
# u(x, y, t=0) = \sin(\pi x)\sin(\pi y), \\\\
# u(x, y, t) = 0 \quad \text{on } \Gamma_1 \cup \Gamma_2 \cup \Gamma_3 \cup \Gamma_4,
# \end{cases}
# \end{equation}
#
# where $D$ is a squared domain $[0,1]^2$, and $\Gamma_i$, with $i=1,...,4$, are the boundaries of the square, and the velocity in the standard wave equation is fixed to one.
# Now, the wave problem is written in PINA code as a class, inheriting from `SpatialProblem` and `TimeDependentProblem` since we deal with spatial, and time dependent variables. The equations are written as `conditions` that should be satisfied in the corresponding domains. `solution` is the exact solution which will be compared with the predicted one.
# In[2]:
def wave_equation(input_, output_):
u_t = grad(output_, input_, components=["u"], d=["t"])
u_tt = grad(u_t, input_, components=["dudt"], d=["t"])
nabla_u = laplacian(output_, input_, components=["u"], d=["x", "y"])
return nabla_u - u_tt
def initial_condition(input_, output_):
u_expected = torch.sin(torch.pi * input_.extract(["x"])) * torch.sin(
torch.pi * input_.extract(["y"])
)
return output_.extract(["u"]) - u_expected
class Wave(TimeDependentProblem, SpatialProblem):
output_variables = ["u"]
spatial_domain = CartesianDomain({"x": [0, 1], "y": [0, 1]})
temporal_domain = CartesianDomain({"t": [0, 1]})
domains = {
"g1": CartesianDomain({"x": 1, "y": [0, 1], "t": [0, 1]}),
"g2": CartesianDomain({"x": 0, "y": [0, 1], "t": [0, 1]}),
"g3": CartesianDomain({"x": [0, 1], "y": 0, "t": [0, 1]}),
"g4": CartesianDomain({"x": [0, 1], "y": 1, "t": [0, 1]}),
"initial": CartesianDomain({"x": [0, 1], "y": [0, 1], "t": 0}),
"D": CartesianDomain({"x": [0, 1], "y": [0, 1], "t": [0, 1]}),
}
conditions = {
"g1": Condition(domain="g1", equation=FixedValue(0.0)),
"g2": Condition(domain="g2", equation=FixedValue(0.0)),
"g3": Condition(domain="g3", equation=FixedValue(0.0)),
"g4": Condition(domain="g4", equation=FixedValue(0.0)),
"initial": Condition(
domain="initial", equation=Equation(initial_condition)
),
"D": Condition(domain="D", equation=Equation(wave_equation)),
}
def solution(self, pts):
f = (
torch.sin(torch.pi * pts.extract(["x"]))
* torch.sin(torch.pi * pts.extract(["y"]))
* torch.cos(
torch.sqrt(torch.tensor(2.0)) * torch.pi * pts.extract(["t"])
)
)
return LabelTensor(f, self.output_variables)
# define problem
problem = Wave()
# ## Hard Constraint Model
# After the problem, a **torch** model is needed to solve the PINN. Usually, many models are already implemented in **PINA**, but the user has the possibility to build his/her own model in `torch`. The hard constraint we impose is on the boundary of the spatial domain. Specifically, our solution is written as:
#
# $$ u_{\rm{pinn}} = xy(1-x)(1-y)\cdot NN(x, y, t), $$
#
# where $NN$ is the neural net output. This neural network takes as input the coordinates (in this case $x$, $y$ and $t$) and provides the unknown field $u$. By construction, it is zero on the boundaries. The residuals of the equations are evaluated at several sampling points (which the user can manipulate using the method `discretise_domain`) and the loss minimized by the neural network is the sum of the residuals.
# In[3]:
class HardMLP(torch.nn.Module):
def __init__(self, input_dim, output_dim):
super().__init__()
self.layers = torch.nn.Sequential(
torch.nn.Linear(input_dim, 40),
torch.nn.ReLU(),
torch.nn.Linear(40, 40),
torch.nn.ReLU(),
torch.nn.Linear(40, output_dim),
)
# here in the foward we implement the hard constraints
def forward(self, x):
hard = (
x.extract(["x"])
* (1 - x.extract(["x"]))
* x.extract(["y"])
* (1 - x.extract(["y"]))
)
return hard * self.layers(x)
# ## Train and Inference
# In this tutorial, the neural network is trained for 1000 epochs with a learning rate of 0.001 (default in `PINN`). As always, we will log using `Tensorboard`.
# In[4]:
# generate the data
problem.discretise_domain(1000, "random", domains="all")
# define model
model = HardMLP(len(problem.input_variables), len(problem.output_variables))
# crete the solver
pinn = PINN(problem=problem, model=model)
# create trainer and train
trainer = Trainer(
solver=pinn,
max_epochs=1000,
accelerator="cpu",
enable_model_summary=False,
train_size=1.0,
val_size=0.0,
test_size=0.0,
callbacks=[MetricTracker(["train_loss", "initial_loss", "D_loss"])],
)
trainer.train()
# Let's now plot the losses inside `MetricTracker` to see how they vary during training.
# In[5]:
trainer_metrics = trainer.callbacks[0].metrics
for metric, loss in trainer_metrics.items():
plt.plot(range(len(loss)), loss, label=metric)
# plotting
plt.xlabel("epoch")
plt.ylabel("loss")
plt.yscale("log")
plt.legend()
# Notice that the loss on the boundaries of the spatial domain is exactly zero, as expected! After the training is completed one can now plot some results using the `matplotlib`. We plot the predicted output on the left side, the true solution at the center and the difference on the right side using the `plot_solution` function.
# In[6]:
@torch.no_grad()
def plot_solution(solver, time):
# get the problem
problem = solver.problem
# get spatial points
spatial_samples = problem.spatial_domain.sample(30, "grid")
# get temporal value
time = LabelTensor(torch.tensor([[time]]), "t")
# cross data
points = spatial_samples.append(time, mode="cross")
# compute pinn solution, true solution and absolute difference
data = {
"PINN solution": solver(points),
"True solution": problem.solution(points),
"Absolute Difference": torch.abs(
solver(points) - problem.solution(points)
),
}
# plot the solution
plt.suptitle(f"Solution for time {time.item()}")
for idx, (title, field) in enumerate(data.items()):
plt.subplot(1, 3, idx + 1)
plt.title(title)
plt.tricontourf( # convert to torch tensor + flatten
points.extract("x").tensor.flatten(),
points.extract("y").tensor.flatten(),
field.tensor.flatten(),
)
plt.colorbar(), plt.tight_layout()
# Let's take a look at the results at different times, for example `0.0`, `0.5` and `1.0`:
# In[7]:
plt.figure(figsize=(12, 6))
plot_solution(solver=pinn, time=0)
plt.figure(figsize=(12, 6))
plot_solution(solver=pinn, time=0.5)
plt.figure(figsize=(12, 6))
plot_solution(solver=pinn, time=1)
# The results are not so great, and we can clearly see that as time progresses the solution gets worse.... Can we do better?
#
# A valid option is to impose the initial condition as hard constraint as well. Specifically, our solution is written as:
#
# $$ u_{\rm{pinn}} = xy(1-x)(1-y)\cdot NN(x, y, t)\cdot t + \cos(\sqrt{2}\pi t)\sin(\pi x)\sin(\pi y), $$
#
# Let us build the network first
# In[8]:
class HardMLPtime(torch.nn.Module):
def __init__(self, input_dim, output_dim):
super().__init__()
self.layers = torch.nn.Sequential(
torch.nn.Linear(input_dim, 40),
torch.nn.ReLU(),
torch.nn.Linear(40, 40),
torch.nn.ReLU(),
torch.nn.Linear(40, output_dim),
)
# here in the foward we implement the hard constraints
def forward(self, x):
hard_space = (
x.extract(["x"])
* (1 - x.extract(["x"]))
* x.extract(["y"])
* (1 - x.extract(["y"]))
)
hard_t = (
torch.sin(torch.pi * x.extract(["x"]))
* torch.sin(torch.pi * x.extract(["y"]))
* torch.cos(
torch.sqrt(torch.tensor(2.0)) * torch.pi * x.extract(["t"])
)
)
return hard_space * self.layers(x) * x.extract(["t"]) + hard_t
# Now let's train with the same configuration as the previous test
# In[9]:
# define model
model = HardMLPtime(len(problem.input_variables), len(problem.output_variables))
# crete the solver
pinn = PINN(problem=problem, model=model)
# create trainer and train
trainer = Trainer(
solver=pinn,
max_epochs=1000,
accelerator="cpu",
enable_model_summary=False,
train_size=1.0,
val_size=0.0,
test_size=0.0,
callbacks=[MetricTracker(["train_loss", "initial_loss", "D_loss"])],
)
trainer.train()
# We can clearly see that the loss is way lower now. Let's plot the results
# In[10]:
plt.figure(figsize=(12, 6))
plot_solution(solver=pinn, time=0)
plt.figure(figsize=(12, 6))
plot_solution(solver=pinn, time=0.5)
plt.figure(figsize=(12, 6))
plot_solution(solver=pinn, time=1)
# We can see now that the results are way better! This is due to the fact that previously the network was not learning correctly the initial conditon, leading to a poor solution when time evolved. By imposing the initial condition the network is able to correctly solve the problem.
# ## What's next?
#
# Congratulations on completing the two dimensional Wave tutorial of **PINA**! There are multiple directions you can go now:
#
# 1. Train the network for longer or with different layer sizes and assert the finaly accuracy
#
# 2. Propose new types of hard constraints in time, e.g. $$ u_{\rm{pinn}} = xy(1-x)(1-y)\cdot NN(x, y, t)(1-\exp(-t)) + \cos(\sqrt{2}\pi t)sin(\pi x)\sin(\pi y), $$
#
# 3. Exploit extrafeature training for model 1 and 2
#
# 4. Many more...

363
tutorials/tutorial4/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

676
tutorials/tutorial4/tutorial.py vendored
View File

@@ -1,676 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Unstructured convolutional autoencoder via continuous convolution
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial4/tutorial.ipynb)
# In this tutorial, we will show how to use the Continuous Convolutional Filter, and how to build common Deep Learning architectures with it. The implementation of the filter follows the original work [*A Continuous Convolutional Trainable Filter for Modelling Unstructured Data*](https://arxiv.org/abs/2210.13416).
# First of all we import the modules needed for the tutorial:
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import torch
import matplotlib.pyplot as plt
import torchvision # for MNIST dataset
import warnings
from pina import Trainer
from pina.problem.zoo import SupervisedProblem
from pina.solver import SupervisedSolver
from pina.trainer import Trainer
from pina.model.block import ContinuousConvBlock
from pina.model import FeedForward # for building AE and MNIST classification
warnings.filterwarnings("ignore")
# The tutorial is structured as follow:
# * [Continuous filter background](#continuous-filter-background): understand how the convolutional filter works and how to use it.
# * [Building a MNIST Classifier](#building-a-mnist-classifier): show how to build a simple classifier using the MNIST dataset and how to combine a continuous convolutional layer with a feedforward neural network.
# * [Building a Continuous Convolutional Autoencoder](#building-a-continuous-convolutional-autoencoder): show how to use the continuous filter to work with unstructured data for autoencoding and up-sampling.
# ## Continuous filter background
# As reported by the authors in the original paper: in contrast to discrete convolution, continuous convolution is mathematically defined as:
#
# $$
# \mathcal{I}_{\rm{out}}(\mathbf{x}) = \int_{\mathcal{X}} \mathcal{I}(\mathbf{x} + \mathbf{\tau}) \cdot \mathcal{K}(\mathbf{\tau}) d\mathbf{\tau},
# $$
# where $\mathcal{K} : \mathcal{X} \rightarrow \mathbb{R}$ is the *continuous filter* function, and $\mathcal{I} : \Omega \subset \mathbb{R}^N \rightarrow \mathbb{R}$ is the input function. The continuous filter function is approximated using a FeedForward Neural Network, thus trainable during the training phase. The way in which the integral is approximated can be different, currently on **PINA** we approximate it using a simple sum, as suggested by the authors. Thus, given $\{\mathbf{x}_i\}_{i=1}^{n}$ points in $\mathbb{R}^N$ of the input function mapped on the $\mathcal{X}$ filter domain, we approximate the above equation as:
# $$
# \mathcal{I}_{\rm{out}}(\mathbf{\tilde{x}}_i) = \sum_{{\mathbf{x}_i}\in\mathcal{X}} \mathcal{I}(\mathbf{x}_i + \mathbf{\tau}) \cdot \mathcal{K}(\mathbf{x}_i),
# $$
# where $\mathbf{\tau} \in \mathcal{S}$, with $\mathcal{S}$ the set of available strides, corresponds to the current stride position of the filter, and $\mathbf{\tilde{x}}_i$ points are obtained by taking the centroid of the filter position mapped on the $\Omega$ domain.
# We will now try to pratically see how to work with the filter. From the above definition we see that what is needed is:
# 1. A domain and a function defined on that domain (the input)
# 2. A stride, corresponding to the positions where the filter needs to be $\rightarrow$ `stride` variable in `ContinuousConv`
# 3. The filter rectangular domain $\rightarrow$ `filter_dim` variable in `ContinuousConv`
# ### Input function
#
# The input function for the continuous filter is defined as a tensor of shape: $$[B \times N_{in} \times N \times D]$$ where $B$ is the batch_size, $N_{in}$ is the number of input fields, $N$ the number of points in the mesh, $D$ the dimension of the problem. In particular:
# * $D$ is the number of spatial variables + 1. The last column must contain the field value. For example for 2D problems $D=3$ and the tensor will be something like `[first coordinate, second coordinate, field value]`
# * $N_{in}$ represents the number of vectorial function presented. For example a vectorial function $f = [f_1, f_2]$ will have $N_{in}=2$
#
# Let's see an example to clear the ideas. We will be verbose to explain in details the input form. We wish to create the function:
# $$
# f(x, y) = [\sin(\pi x) \sin(\pi y), -\sin(\pi x) \sin(\pi y)] \quad (x,y)\in[0,1]\times[0,1]
# $$
#
# using a batch size equal to 1.
# In[2]:
# batch size fixed to 1
batch_size = 1
# points in the mesh fixed to 200
N = 200
# vectorial 2 dimensional function, number_input_fields=2
number_input_fields = 2
# 2 dimensional spatial variables, D = 2 + 1 = 3
D = 3
# create the function f domain as random 2d points in [0, 1]
domain = torch.rand(size=(batch_size, number_input_fields, N, D - 1))
print(f"Domain has shape: {domain.shape}")
# create the functions
pi = torch.acos(torch.tensor([-1.0])) # pi value
f1 = torch.sin(pi * domain[:, 0, :, 0]) * torch.sin(pi * domain[:, 0, :, 1])
f2 = -torch.sin(pi * domain[:, 1, :, 0]) * torch.sin(pi * domain[:, 1, :, 1])
# stacking the input domain and field values
data = torch.empty(size=(batch_size, number_input_fields, N, D))
data[..., :-1] = domain # copy the domain
data[:, 0, :, -1] = f1 # copy first field value
data[:, 1, :, -1] = f1 # copy second field value
print(f"Filter input data has shape: {data.shape}")
# ### Stride
#
# The stride is passed as a dictionary `stride` which tells the filter where to go. Here is an example for the $[0,1]\times[0,5]$ domain:
#
# ```python
# # stride definition
# stride = {"domain": [1, 5],
# "start": [0, 0],
# "jump": [0.1, 0.3],
# "direction": [1, 1],
# }
# ```
# This tells the filter:
# 1. `domain`: square domain (the only implemented) $[0,1]\times[0,5]$. The minimum value is always zero, while the maximum is specified by the user
# 2. `start`: start position of the filter, coordinate $(0, 0)$
# 3. `jump`: the jumps of the centroid of the filter to the next position $(0.1, 0.3)$
# 4. `direction`: the directions of the jump, with `1 = right`, `0 = no jump`, `-1 = left` with respect to the current position
#
# **Note**
#
# We are planning to release the possibility to directly pass a list of possible strides!
# ### Filter definition
#
# Having defined all the previous blocks, we are now able to construct the continuous filter.
#
# Suppose we would like to get an output with only one field, and let us fix the filter dimension to be $[0.1, 0.1]$.
# In[3]:
# filter dim
filter_dim = [0.1, 0.1]
# stride
stride = {
"domain": [1, 1],
"start": [0, 0],
"jump": [0.08, 0.08],
"direction": [1, 1],
}
# creating the filter
cConv = ContinuousConvBlock(
input_numb_field=number_input_fields,
output_numb_field=1,
filter_dim=filter_dim,
stride=stride,
)
# That's it! In just one line of code we have created the continuous convolutional filter. By default the `pina.model.FeedForward` neural network is intitialised, more on the [documentation](https://mathlab.github.io/PINA/_rst/fnn.html). In case the mesh doesn't change during training we can set the `optimize` flag equals to `True`, to exploit optimizations for finding the points to convolve.
# In[4]:
# creating the filter + optimization
cConv = ContinuousConvBlock(
input_numb_field=number_input_fields,
output_numb_field=1,
filter_dim=filter_dim,
stride=stride,
optimize=True,
)
# Let's try to do a forward pass:
# In[5]:
print(f"Filter input data has shape: {data.shape}")
# input to the filter
output = cConv(data)
print(f"Filter output data has shape: {output.shape}")
# If we don't want to use the default `FeedForward` neural network, we can pass a specified torch model in the `model` keyword as follow:
#
# In[6]:
class SimpleKernel(torch.nn.Module):
def __init__(self) -> None:
super().__init__()
self.model = torch.nn.Sequential(
torch.nn.Linear(2, 20),
torch.nn.ReLU(),
torch.nn.Linear(20, 20),
torch.nn.ReLU(),
torch.nn.Linear(20, 1),
)
def forward(self, x):
return self.model(x)
cConv = ContinuousConvBlock(
input_numb_field=number_input_fields,
output_numb_field=1,
filter_dim=filter_dim,
stride=stride,
optimize=True,
model=SimpleKernel,
)
# Notice that we pass the class and not an already built object!
# ## Building a MNIST Classifier
#
# Let's see how we can build a MNIST classifier using a continuous convolutional filter. We will use the MNIST dataset from PyTorch. In order to keep small training times we use only 6000 samples for training and 1000 samples for testing.
# In[7]:
from torch.utils.data import DataLoader, SubsetRandomSampler
numb_training = 6000 # get just 6000 images for training
numb_testing = 1000 # get just 1000 images for training
seed = 111 # for reproducibility
batch_size = 8 # setting batch size
# setting the seed
torch.manual_seed(seed)
# downloading the dataset
train_data = torchvision.datasets.MNIST(
"./data/",
download=True,
train=False,
transform=torchvision.transforms.Compose(
[
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize((0.1307,), (0.3081,)),
]
),
)
# Let's now build a simple classifier. The MNIST dataset is composed by vectors of shape `[batch, 1, 28, 28]`, but we can image them as one field functions where the pixels $ij$ are the coordinate $x=i, y=j$ in a $[0, 27]\times[0,27]$ domain, and the pixels values are the field values. We just need a function to transform the regular tensor in a tensor compatible for the continuous filter:
# In[8]:
def transform_input(x):
batch_size = x.shape[0]
dim_grid = tuple(x.shape[:-3:-1])
# creating the n dimensional mesh grid for a single channel image
values_mesh = [torch.arange(0, dim).float() for dim in dim_grid]
mesh = torch.meshgrid(values_mesh)
coordinates_mesh = [m.reshape(-1, 1).to(x.device) for m in mesh]
coordinates = (
torch.cat(coordinates_mesh, dim=1)
.unsqueeze(0)
.repeat((batch_size, 1, 1))
.unsqueeze(1)
)
return torch.cat((coordinates, x.flatten(2).unsqueeze(-1)), dim=-1)
# We can now build a simple classifier! We will use just one convolutional filter followed by a feedforward neural network
# In[9]:
# setting the seed
torch.manual_seed(seed)
class ContinuousClassifier(torch.nn.Module):
def __init__(self):
super().__init__()
# number of classes for classification
numb_class = 10
# convolutional block
self.convolution = ContinuousConvBlock(
input_numb_field=1,
output_numb_field=4,
stride={
"domain": [27, 27],
"start": [0, 0],
"jumps": [4, 4],
"direction": [1, 1.0],
},
filter_dim=[4, 4],
optimize=True,
)
# feedforward net
self.nn = FeedForward(
input_dimensions=196,
output_dimensions=numb_class,
layers=[120, 64],
func=torch.nn.ReLU,
)
def forward(self, x):
# transform input + convolution
x = transform_input(x)
x = self.convolution(x)
# feed forward classification
return self.nn(x[..., -1].flatten(1))
# We now aim to solve the classification problem. For this we will use the `SupervisedSolver` and the `SupervisedProblem`. The input of the supervised problems are the images, while the output the corresponding class.
# In[10]:
# setting the problem
problem = SupervisedProblem(
input_=train_data.train_data.unsqueeze(1), # adding channel dimension
output_=train_data.train_labels,
)
# setting the solver
solver = SupervisedSolver(
problem=problem,
model=ContinuousClassifier(),
loss=torch.nn.CrossEntropyLoss(),
use_lt=False,
)
# setting the trainer
trainer = Trainer(
solver=solver,
max_epochs=1,
accelerator="cpu",
enable_model_summary=False,
train_size=0.7,
val_size=0.1,
test_size=0.2,
batch_size=64,
)
trainer.train()
# Let's see the performance on the test set!
# In[11]:
correct = 0
total = 0
trainer.data_module.setup("test")
with torch.no_grad():
for data in trainer.data_module.test_dataloader():
test_data = data["data"]
images, labels = test_data["input"], test_data["target"]
# calculate outputs by running images through the network
outputs = solver(images)
# the class with the highest energy is what we choose as prediction
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print(f"Accuracy of the network on the test images: {(correct / total):.3%}")
# As we can see we have very good performance for having trained only for 1 epoch! Nevertheless, we are still using structured data... Let's see how we can build an autoencoder for unstructured data now.
# ## Building a Continuous Convolutional Autoencoder
#
# Just as toy problem, we will now build an autoencoder for the following function $f(x,y)=\sin(\pi x)\sin(\pi y)$ on the unit circle domain centered in $(0.5, 0.5)$. We will also see the ability to up-sample (once trained) the results without retraining. Let's first create the input and visualize it, we will use firstly a mesh of $100$ points.
# In[12]:
# create inputs
def circle_grid(N=100):
"""Generate points withing a unit 2D circle centered in (0.5, 0.5)
:param N: number of points
:type N: float
:return: [x, y] array of points
:rtype: torch.tensor
"""
PI = torch.acos(torch.zeros(1)).item() * 2
R = 0.5
centerX = 0.5
centerY = 0.5
r = R * torch.sqrt(torch.rand(N))
theta = torch.rand(N) * 2 * PI
x = centerX + r * torch.cos(theta)
y = centerY + r * torch.sin(theta)
return torch.stack([x, y]).T
# create the grid
grid = circle_grid(500)
# create input
input_data = torch.empty(size=(1, 1, grid.shape[0], 3))
input_data[0, 0, :, :-1] = grid
input_data[0, 0, :, -1] = torch.sin(pi * grid[:, 0]) * torch.sin(
pi * grid[:, 1]
)
# visualize data
plt.title("Training sample with 500 points")
plt.scatter(grid[:, 0], grid[:, 1], c=input_data[0, 0, :, -1])
plt.colorbar()
plt.show()
# Let's now build a simple autoencoder using the continuous convolutional filter. The data is clearly unstructured and a simple convolutional filter might not work without projecting or interpolating first. Let's first build and `Encoder` and `Decoder` class, and then a `Autoencoder` class that contains both.
# In[13]:
class Encoder(torch.nn.Module):
def __init__(self, hidden_dimension):
super().__init__()
# convolutional block
self.convolution = ContinuousConvBlock(
input_numb_field=1,
output_numb_field=2,
stride={
"domain": [1, 1],
"start": [0, 0],
"jumps": [0.05, 0.05],
"direction": [1, 1.0],
},
filter_dim=[0.15, 0.15],
optimize=True,
)
# feedforward net
self.nn = FeedForward(
input_dimensions=400,
output_dimensions=hidden_dimension,
layers=[240, 120],
)
def forward(self, x):
# convolution
x = self.convolution(x)
# feed forward pass
return self.nn(x[..., -1])
class Decoder(torch.nn.Module):
def __init__(self, hidden_dimension):
super().__init__()
# convolutional block
self.convolution = ContinuousConvBlock(
input_numb_field=2,
output_numb_field=1,
stride={
"domain": [1, 1],
"start": [0, 0],
"jumps": [0.05, 0.05],
"direction": [1, 1.0],
},
filter_dim=[0.15, 0.15],
optimize=True,
)
# feedforward net
self.nn = FeedForward(
input_dimensions=hidden_dimension,
output_dimensions=400,
layers=[120, 240],
)
def forward(self, weights, grid):
# feed forward pass
x = self.nn(weights)
# transpose convolution
return torch.sigmoid(self.convolution.transpose(x, grid))
# Very good! Notice that in the `Decoder` class in the `forward` pass we have used the `.transpose()` method of the `ContinuousConvolution` class. This method accepts the `weights` for upsampling and the `grid` on where to upsample. Let's now build the autoencoder! We set the hidden dimension in the `hidden_dimension` variable. We apply the sigmoid on the output since the field value is between $[0, 1]$.
# In[14]:
class Autoencoder(torch.nn.Module):
def __init__(self, hidden_dimension=10):
super().__init__()
self.encoder = Encoder(hidden_dimension)
self.decoder = Decoder(hidden_dimension)
def forward(self, x):
# saving grid for later upsampling
grid = x.clone().detach()
# encoder
weights = self.encoder(x)
# decoder
out = self.decoder(weights, grid)
return out
# Let's now train the autoencoder, minimizing the mean square error loss and optimizing using Adam. We use the `SupervisedSolver` as solver, and the problem is a simple problem created by inheriting from `AbstractProblem`. It takes approximately two minutes to train on CPU.
# In[15]:
# define the problem
problem = SupervisedProblem(input_data, input_data)
# define the solver
solver = SupervisedSolver(
problem=problem,
model=Autoencoder(),
loss=torch.nn.MSELoss(),
use_lt=False,
)
# train
trainer = Trainer(
solver,
max_epochs=100,
accelerator="cpu",
enable_model_summary=False, # we train on CPU and avoid model summary at beginning of training (optional)
train_size=1.0,
val_size=0.0,
test_size=0.0,
)
trainer.train()
# Let's visualize the two solutions side by side!
# In[16]:
solver.eval()
# get output and detach from computational graph for plotting
output = solver(input_data).detach()
# visualize data
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(8, 3))
pic1 = axes[0].scatter(grid[:, 0], grid[:, 1], c=input_data[0, 0, :, -1])
axes[0].set_title("Real")
fig.colorbar(pic1)
plt.subplot(1, 2, 2)
pic2 = axes[1].scatter(grid[:, 0], grid[:, 1], c=output[0, 0, :, -1])
axes[1].set_title("Autoencoder")
fig.colorbar(pic2)
plt.tight_layout()
plt.show()
# As we can see, the two solutions are really similar! We can compute the $l_2$ error quite easily as well:
# In[17]:
def l2_error(input_, target):
return torch.linalg.norm(input_ - target, ord=2) / torch.linalg.norm(
input_, ord=2
)
print(f"l2 error: {l2_error(input_data[0, 0, :, -1], output[0, 0, :, -1]):.2%}")
# More or less $4\%$ in $l_2$ error, which is really low considering the fact that we use just **one** convolutional layer and a simple feedforward to decrease the dimension. Let's see now some peculiarity of the filter.
# ### Filter for upsampling
#
# Suppose we have already the hidden representation and we want to upsample on a differen grid with more points. Let's see how to do it:
# In[18]:
# setting the seed
torch.manual_seed(seed)
grid2 = circle_grid(1500) # triple number of points
input_data2 = torch.zeros(size=(1, 1, grid2.shape[0], 3))
input_data2[0, 0, :, :-1] = grid2
input_data2[0, 0, :, -1] = torch.sin(pi * grid2[:, 0]) * torch.sin(
pi * grid2[:, 1]
)
# get the hidden representation from original input
latent = solver.model.encoder(input_data)
# upsample on the second input_data2
output = solver.model.decoder(latent, input_data2).detach()
# show the picture
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(8, 3))
pic1 = axes[0].scatter(grid2[:, 0], grid2[:, 1], c=input_data2[0, 0, :, -1])
axes[0].set_title("Real")
fig.colorbar(pic1)
plt.subplot(1, 2, 2)
pic2 = axes[1].scatter(grid2[:, 0], grid2[:, 1], c=output[0, 0, :, -1])
axes[1].set_title("Up-sampling")
fig.colorbar(pic2)
plt.tight_layout()
plt.show()
# As we can see we have a very good approximation of the original function, even thought some noise is present. Let's calculate the error now:
# In[19]:
print(
f"l2 error: {l2_error(input_data2[0, 0, :, -1], output[0, 0, :, -1]):.2%}"
)
# ### Autoencoding at different resolutions
# In the previous example we already had the hidden representation (of the original input) and we used it to upsample. Sometimes however we could have a finer mesh solution and we would simply want to encode it. This can be done without retraining! This procedure can be useful in case we have many points in the mesh and just a smaller part of them are needed for training. Let's see the results of this:
# In[20]:
# setting the seed
torch.manual_seed(seed)
grid2 = circle_grid(3500) # very fine mesh
input_data2 = torch.zeros(size=(1, 1, grid2.shape[0], 3))
input_data2[0, 0, :, :-1] = grid2
input_data2[0, 0, :, -1] = torch.sin(pi * grid2[:, 0]) * torch.sin(
pi * grid2[:, 1]
)
# get the hidden representation from finer mesh input
latent = solver.model.encoder(input_data2)
# upsample on the second input_data2
output = solver.model.decoder(latent, input_data2).detach()
# show the picture
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(8, 3))
pic1 = axes[0].scatter(grid2[:, 0], grid2[:, 1], c=input_data2[0, 0, :, -1])
axes[0].set_title("Real")
fig.colorbar(pic1)
plt.subplot(1, 2, 2)
pic2 = axes[1].scatter(grid2[:, 0], grid2[:, 1], c=output[0, 0, :, -1])
axes[1].set_title("Autoencoder not re-trained")
fig.colorbar(pic2)
plt.tight_layout()
plt.show()
# calculate l2 error
print(
f"l2 error: {l2_error(input_data2[0, 0, :, -1], output[0, 0, :, -1]):.2%}"
)
# ## What's next?
#
# We have shown the basic usage of a convolutional filter. There are additional extensions possible:
#
# 1. Train using Physics Informed strategies
#
# 2. Use the filter to build an unstructured convolutional autoencoder for reduced order modelling
#
# 3. Many more...

128
tutorials/tutorial5/tutorial.ipynb vendored
View File

@@ -5,18 +5,13 @@
"id": "e80567a6",
"metadata": {},
"source": [
"# Tutorial: Two dimensional Darcy flow using the Fourier Neural Operator\n",
"# Tutorial: Modeling 2D Darcy Flow with the Fourier Neural Operator\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial5/tutorial.ipynb)\n"
]
},
{
"cell_type": "markdown",
"id": "8762bbe5",
"metadata": {},
"source": [
"In this tutorial we are going to solve the Darcy flow problem in two dimensions, presented in [*Fourier Neural Operator for\n",
"Parametric Partial Differential Equation*](https://openreview.net/pdf?id=c8P9NQVtmnO). First of all we import the modules needed for the tutorial. Importing `scipy` is needed for input-output operations."
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial5/tutorial.ipynb)\n",
"\n",
"In this tutorial, we are going to solve the **Darcy flow problem** in two dimensions, as presented in the paper [*Fourier Neural Operator for Parametric Partial Differential Equations*](https://openreview.net/pdf?id=c8P9NQVtmnO).\n",
"\n",
"We begin by importing the necessary modules for the tutorial:\n"
]
},
{
@@ -39,7 +34,7 @@
"except:\n",
" IN_COLAB = False\n",
"if IN_COLAB:\n",
" !pip install \"pina-mathlab\"\n",
" !pip install \"pina-mathlab[tutorial]\"\n",
" !pip install scipy\n",
" # get the data\n",
" !wget https://github.com/mathLab/PINA/raw/refs/heads/master/tutorials/tutorial5/Data_Darcy.mat\n",
@@ -48,10 +43,9 @@
"import matplotlib.pyplot as plt\n",
"import warnings\n",
"\n",
"# !pip install scipy # install scipy\n",
"from scipy import io\n",
"from pina.model import FNO, FeedForward # let's import some models\n",
"from pina import Condition, Trainer\n",
"from pina.model import FNO, FeedForward\n",
"from pina import Trainer\n",
"from pina.solver import SupervisedSolver\n",
"from pina.problem.zoo import SupervisedProblem\n",
"\n",
@@ -65,13 +59,15 @@
"source": [
"## Data Generation\n",
"\n",
"We will focus on solving a specific PDE, the **Darcy Flow** equation. The Darcy PDE is a second-order elliptic PDE with the following form:\n",
"We will focus on solving a specific PDE: the **Darcy Flow** equation. This is a second-order elliptic PDE given by:\n",
"\n",
"$$\n",
"-\\nabla\\cdot(k(x, y)\\nabla u(x, y)) = f(x) \\quad (x, y) \\in D.\n",
"-\\nabla\\cdot(k(x, y)\\nabla u(x, y)) = f(x, y), \\quad (x, y) \\in D.\n",
"$$\n",
"\n",
"Specifically, $u$ is the flow pressure, $k$ is the permeability field and $f$ is the forcing function. The Darcy flow can parameterize a variety of systems including flow through porous media, elastic materials and heat conduction. Here you will define the domain as a 2D unit square Dirichlet boundary conditions. The dataset is taken from the authors original reference.\n"
"Here, $u$ represents the flow pressure, $k$ is the permeability field, and $f$ is the forcing function. The Darcy flow equation can be used to model various systems, including flow through porous media, elasticity in materials, and heat conduction.\n",
"\n",
"In this tutorial, the domain $D$ is defined as a 2D unit square with Dirichlet boundary conditions. The dataset used is taken from the authors' original implementation in the referenced paper."
]
},
{
@@ -103,12 +99,12 @@
"id": "9a9defd4",
"metadata": {},
"source": [
"Let's visualize some data"
"Before diving into modeling, it's helpful to visualize some examples from the dataset. This will give us a better understanding of the input (permeability field) and the corresponding output (pressure field) that our model will learn to predict."
]
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 4,
"id": "c8501b6f",
"metadata": {
"ExecuteTime": {
@@ -143,12 +139,12 @@
"id": "89a77ff1",
"metadata": {},
"source": [
"We now create the Neural Operators problem class. Learning Neural Operators is similar as learning in a supervised manner, therefore we will use `SupervisedProblem`."
"We now define the problem class for learning the Neural Operator. Since this task is essentially a supervised learning problem—where the goal is to learn a mapping from input functions to output solutions—we will use the `SupervisedProblem` class provided by **PINA**."
]
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 6,
"id": "8b27d283",
"metadata": {
"ExecuteTime": {
@@ -169,14 +165,14 @@
"id": "1096cc20",
"metadata": {},
"source": [
"## Solving the problem with a FeedForward Neural Network\n",
"## Solving the Problem with a Feedforward Neural Network\n",
"\n",
"We will first solve the problem using a Feedforward neural network. We will use the `SupervisedSolver` for solving the problem, since we are training using supervised learning."
"We begin by solving the Darcy flow problem using a standard Feedforward Neural Network (FNN). Since we are approaching this task with supervised learning, we will use the `SupervisedSolver` provided by **PINA** to train the model."
]
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 7,
"id": "e34f18b0",
"metadata": {
"ExecuteTime": {
@@ -195,25 +191,25 @@
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 9: 100%|██████████| 100/100 [00:00<00:00, 289.72it/s, v_num=3, data_loss_step=0.102, train_loss_step=0.102, data_loss_epoch=0.105, train_loss_epoch=0.105] "
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "0b77243fe0274dada29b6bb5a15c47e8",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=10` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 9: 100%|██████████| 100/100 [00:00<00:00, 286.77it/s, v_num=3, data_loss_step=0.102, train_loss_step=0.102, data_loss_epoch=0.105, train_loss_epoch=0.105]\n"
]
}
],
"source": [
@@ -243,12 +239,12 @@
"id": "7b2c35be",
"metadata": {},
"source": [
"The final loss is pretty high... We can calculate the error by importing `LpLoss`."
"The final loss is relatively high, indicating that the model might not be capturing the solution accurately. To better evaluate the model's performance, we can compute the error using the `LpLoss` metric."
]
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 9,
"id": "0e2a6aa4",
"metadata": {
"ExecuteTime": {
@@ -261,8 +257,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Final error training 28.57%\n",
"Final error testing 28.59%\n"
"Final error training 28.54%\n",
"Final error testing 28.58%\n"
]
}
],
@@ -293,14 +289,14 @@
"id": "6b5e5aa6",
"metadata": {},
"source": [
"## Solving the problem with a Fourier Neural Operator (FNO)\n",
"## Solving the Problem with a Fourier Neural Operator\n",
"\n",
"We will now move to solve the problem using a FNO. Since we are learning operator this approach is better suited, as we shall see."
"We will now solve the Darcy flow problem using a Fourier Neural Operator (FNO). Since we are learning a mapping between functions—i.e., an operator—this approach is more suitable and often yields better performance, as we will see."
]
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 10,
"id": "9af523a5",
"metadata": {
"ExecuteTime": {
@@ -319,25 +315,25 @@
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 9: 100%|██████████| 100/100 [00:02<00:00, 36.66it/s, v_num=4, data_loss_step=0.00164, train_loss_step=0.00164, data_loss_epoch=0.00229, train_loss_epoch=0.00229]"
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "6fbb56905e4c4799973669f533a2d73c",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=10` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 9: 100%|██████████| 100/100 [00:02<00:00, 36.56it/s, v_num=4, data_loss_step=0.00164, train_loss_step=0.00164, data_loss_epoch=0.00229, train_loss_epoch=0.00229]\n"
]
}
],
"source": [
@@ -376,12 +372,14 @@
"id": "84964cb9",
"metadata": {},
"source": [
"We can clearly see that the final loss is lower. Let's see in testing.. Notice that the number of parameters is way higher than a `FeedForward` network. We suggest to use GPU or TPU for a speed up in training, when many data samples are used."
"We can clearly observe that the final loss is significantly lower when using the FNO. Let's now evaluate its performance on the test set.\n",
"\n",
"Note that the number of trainable parameters in the FNO is considerably higher compared to a `FeedForward` network. Therefore, we recommend using a GPU or TPU to accelerate training, especially when working with large datasets."
]
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 11,
"id": "58e2db89",
"metadata": {
"ExecuteTime": {
@@ -394,8 +392,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Final error training 3.36%\n",
"Final error testing 3.54%\n"
"Final error training 3.52%\n",
"Final error testing 3.67%\n"
]
}
],
@@ -421,7 +419,7 @@
"id": "26e3a6e4",
"metadata": {},
"source": [
"As we can see the loss is way lower!"
"As we can see, the loss is significantly lower with the Fourier Neural Operator!"
]
},
{
@@ -429,9 +427,17 @@
"id": "ba1dfa4b",
"metadata": {},
"source": [
"## What's next?\n",
"## What's Next?\n",
"\n",
"We have made a very simple example on how to use the `FNO` for learning neural operator. Currently in **PINA** we implement 1D/2D/3D cases. We suggest to extend the tutorial using more complex problems and train for longer, to see the full potential of neural operators."
"Congratulations on completing the tutorial on solving the Darcy flow problem using **PINA**! There are many potential next steps you can explore:\n",
"\n",
"1. **Train the network longer or with different hyperparameters**: Experiment with different configurations of the neural network. You can try varying the number of layers, activation functions, or learning rates to improve accuracy.\n",
"\n",
"2. **Solve more complex problems**: The Darcy flow problem is just the beginning! Try solving other complex problems from the field of parametric PDEs. The original paper and **PINA** documentation offer many more examples to explore.\n",
"\n",
"3. **...and many more!**: There are countless directions to further explore. For instance, you could try to add physics informed learning!\n",
"\n",
"For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/)."
]
}
],

209
tutorials/tutorial5/tutorial.py vendored
View File

@@ -1,209 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Two dimensional Darcy flow using the Fourier Neural Operator
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial5/tutorial.ipynb)
#
# In this tutorial we are going to solve the Darcy flow problem in two dimensions, presented in [*Fourier Neural Operator for
# Parametric Partial Differential Equation*](https://openreview.net/pdf?id=c8P9NQVtmnO). First of all we import the modules needed for the tutorial. Importing `scipy` is needed for input-output operations.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
get_ipython().system('pip install scipy')
# get the data
get_ipython().system('wget https://github.com/mathLab/PINA/raw/refs/heads/master/tutorials/tutorial5/Data_Darcy.mat')
import torch
import matplotlib.pyplot as plt
import warnings
# !pip install scipy # install scipy
from scipy import io
from pina.model import FNO, FeedForward # let's import some models
from pina import Condition, Trainer
from pina.solver import SupervisedSolver
from pina.problem.zoo import SupervisedProblem
warnings.filterwarnings("ignore")
# ## Data Generation
#
# We will focus on solving a specific PDE, the **Darcy Flow** equation. The Darcy PDE is a second-order elliptic PDE with the following form:
#
# $$
# -\nabla\cdot(k(x, y)\nabla u(x, y)) = f(x) \quad (x, y) \in D.
# $$
#
# Specifically, $u$ is the flow pressure, $k$ is the permeability field and $f$ is the forcing function. The Darcy flow can parameterize a variety of systems including flow through porous media, elastic materials and heat conduction. Here you will define the domain as a 2D unit square Dirichlet boundary conditions. The dataset is taken from the authors original reference.
#
# In[2]:
# download the dataset
data = io.loadmat("Data_Darcy.mat")
# extract data (we use only 100 data for train)
k_train = torch.tensor(data["k_train"], dtype=torch.float)
u_train = torch.tensor(data["u_train"], dtype=torch.float)
k_test = torch.tensor(data["k_test"], dtype=torch.float)
u_test = torch.tensor(data["u_test"], dtype=torch.float)
x = torch.tensor(data["x"], dtype=torch.float)[0]
y = torch.tensor(data["y"], dtype=torch.float)[0]
# Let's visualize some data
# In[3]:
plt.subplot(1, 2, 1)
plt.title("permeability")
plt.imshow(k_train[0])
plt.subplot(1, 2, 2)
plt.title("field solution")
plt.imshow(u_train[0])
plt.show()
# We now create the Neural Operators problem class. Learning Neural Operators is similar as learning in a supervised manner, therefore we will use `SupervisedProblem`.
# In[4]:
# make problem
problem = SupervisedProblem(
input_=k_train.unsqueeze(-1), output_=u_train.unsqueeze(-1)
)
# ## Solving the problem with a FeedForward Neural Network
#
# We will first solve the problem using a Feedforward neural network. We will use the `SupervisedSolver` for solving the problem, since we are training using supervised learning.
# In[5]:
# make model
model = FeedForward(input_dimensions=1, output_dimensions=1)
# make solver
solver = SupervisedSolver(problem=problem, model=model, use_lt=False)
# make the trainer and train
trainer = Trainer(
solver=solver,
max_epochs=10,
accelerator="cpu",
enable_model_summary=False,
batch_size=10,
train_size=1.0,
val_size=0.0,
test_size=0.0,
)
trainer.train()
# The final loss is pretty high... We can calculate the error by importing `LpLoss`.
# In[6]:
from pina.loss import LpLoss
# make the metric
metric_err = LpLoss(relative=False)
model = solver.model
err = (
float(
metric_err(u_train.unsqueeze(-1), model(k_train.unsqueeze(-1))).mean()
)
* 100
)
print(f"Final error training {err:.2f}%")
err = (
float(metric_err(u_test.unsqueeze(-1), model(k_test.unsqueeze(-1))).mean())
* 100
)
print(f"Final error testing {err:.2f}%")
# ## Solving the problem with a Fourier Neural Operator (FNO)
#
# We will now move to solve the problem using a FNO. Since we are learning operator this approach is better suited, as we shall see.
# In[7]:
# make model
lifting_net = torch.nn.Linear(1, 24)
projecting_net = torch.nn.Linear(24, 1)
model = FNO(
lifting_net=lifting_net,
projecting_net=projecting_net,
n_modes=8,
dimensions=2,
inner_size=24,
padding=8,
)
# make solver
solver = SupervisedSolver(problem=problem, model=model, use_lt=False)
# make the trainer and train
trainer = Trainer(
solver=solver,
max_epochs=10,
accelerator="cpu",
enable_model_summary=False,
batch_size=10,
train_size=1.0,
val_size=0.0,
test_size=0.0,
)
trainer.train()
# We can clearly see that the final loss is lower. Let's see in testing.. Notice that the number of parameters is way higher than a `FeedForward` network. We suggest to use GPU or TPU for a speed up in training, when many data samples are used.
# In[8]:
model = solver.model
err = (
float(
metric_err(u_train.unsqueeze(-1), model(k_train.unsqueeze(-1))).mean()
)
* 100
)
print(f"Final error training {err:.2f}%")
err = (
float(metric_err(u_test.unsqueeze(-1), model(k_test.unsqueeze(-1))).mean())
* 100
)
print(f"Final error testing {err:.2f}%")
# As we can see the loss is way lower!
# ## What's next?
#
# We have made a very simple example on how to use the `FNO` for learning neural operator. Currently in **PINA** we implement 1D/2D/3D cases. We suggest to extend the tutorial using more complex problems and train for longer, to see the full potential of neural operators.

117
tutorials/tutorial6/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

293
tutorials/tutorial6/tutorial.py vendored
View File

@@ -1,293 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Building custom geometries with PINA `DomainInterface` class
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial6/tutorial.ipynb)
#
# In this tutorial we will show how to use geometries in PINA. Specifically, the tutorial will include how to create geometries and how to visualize them. The topics covered are:
#
# * Creating CartesianDomains and EllipsoidDomains
# * Getting the Union and Difference of Geometries
# * Sampling points in the domain (and visualize them)
#
# We import the relevant modules first.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import matplotlib.pyplot as plt
from pina.domain import (
EllipsoidDomain,
Difference,
CartesianDomain,
Union,
SimplexDomain,
DomainInterface,
)
from pina.label_tensor import LabelTensor
def plot_scatter(ax, pts, title):
ax.title.set_text(title)
ax.scatter(pts.extract("x"), pts.extract("y"), color="blue", alpha=0.5)
# ## Built-in Geometries
# We will create one cartesian and two ellipsoids. For the sake of simplicity, we show here the 2-dimensional case, but the extension to 3D (and higher) cases is trivial. The geometries allow also the generation of samples belonging to the boundary. So, we will create one ellipsoid with the border and one without.
# In[ ]:
cartesian = CartesianDomain({"x": [0, 2], "y": [0, 2]})
ellipsoid_no_border = EllipsoidDomain({"x": [1, 3], "y": [1, 3]})
ellipsoid_border = EllipsoidDomain(
{"x": [2, 4], "y": [2, 4]}, sample_surface=True
)
# The `{'x': [0, 2], 'y': [0, 2]}` are the bounds of the `CartesianDomain` being created.
#
# To visualize these shapes, we need to sample points on them. We will use the `sample` method of the `CartesianDomain` and `EllipsoidDomain` classes. This method takes a `n` argument which is the number of points to sample. It also takes different modes to sample, such as `'random'`.
# In[ ]:
cartesian_samples = cartesian.sample(n=1000, mode="random")
ellipsoid_no_border_samples = ellipsoid_no_border.sample(n=1000, mode="random")
ellipsoid_border_samples = ellipsoid_border.sample(n=1000, mode="random")
# We can see the samples of each geometry to see what we are working with.
# In[4]:
print(f"Cartesian Samples: {cartesian_samples}")
print(f"Ellipsoid No Border Samples: {ellipsoid_no_border_samples}")
print(f"Ellipsoid Border Samples: {ellipsoid_border_samples}")
# Notice how these are all `LabelTensor` objects. You can read more about these in the [documentation](https://mathlab.github.io/PINA/_rst/label_tensor.html). At a very high level, they are tensors where each element in a tensor has a label that we can access by doing `<tensor_name>.labels`. We can also access the values of the tensor by doing `<tensor_name>.extract(['x'])`.
#
# We are now ready to visualize the samples using matplotlib.
# In[ ]:
fig, axs = plt.subplots(1, 3, figsize=(16, 4))
pts_list = [
cartesian_samples,
ellipsoid_no_border_samples,
ellipsoid_border_samples,
]
title_list = ["Cartesian Domain", "Ellipsoid Domain", "Ellipsoid Border Domain"]
for ax, pts, title in zip(axs, pts_list, title_list):
plot_scatter(ax, pts, title)
# We have now created, sampled, and visualized our first geometries! We can see that the `EllipsoidDomain` with the border has a border around it. We can also see that the `EllipsoidDomain` without the border is just the ellipse. We can also see that the `CartesianDomain` is just a square.
# ### Simplex Domain
#
# Among the built-in shapes, we quickly show here the usage of `SimplexDomain`, which can be used for polygonal domains!
# In[ ]:
import torch
spatial_domain = SimplexDomain(
[
LabelTensor(torch.tensor([[0, 0]]), labels=["x", "y"]),
LabelTensor(torch.tensor([[1, 1]]), labels=["x", "y"]),
LabelTensor(torch.tensor([[0, 2]]), labels=["x", "y"]),
]
)
spatial_domain2 = SimplexDomain(
[
LabelTensor(torch.tensor([[0.0, -2.0]]), labels=["x", "y"]),
LabelTensor(torch.tensor([[-0.5, -0.5]]), labels=["x", "y"]),
LabelTensor(torch.tensor([[-2.0, 0.0]]), labels=["x", "y"]),
]
)
pts = spatial_domain2.sample(100)
fig, axs = plt.subplots(1, 2, figsize=(16, 6))
for domain, ax in zip([spatial_domain, spatial_domain2], axs):
pts = domain.sample(1000)
plot_scatter(ax, pts, "Simplex Domain")
# ## Boolean Operations
# To create complex shapes we can use the boolean operations, for example to merge two default geometries. We need to simply use the `Union` class: it takes a list of geometries and returns the union of them.
#
# Let's create three unions. Firstly, it will be a union of `cartesian` and `ellipsoid_no_border`. Next, it will be a union of `ellipse_no_border` and `ellipse_border`. Lastly, it will be a union of all three geometries.
# In[7]:
cart_ellipse_nb_union = Union([cartesian, ellipsoid_no_border])
cart_ellipse_b_union = Union([cartesian, ellipsoid_border])
three_domain_union = Union([cartesian, ellipsoid_no_border, ellipsoid_border])
# We can of course sample points over the new geometries, by using the `sample` method as before. We highlight that the available sample strategy here is only *random*.
# In[ ]:
c_e_nb_u_points = cart_ellipse_nb_union.sample(n=2000, mode="random")
c_e_b_u_points = cart_ellipse_b_union.sample(n=2000, mode="random")
three_domain_union_points = three_domain_union.sample(n=3000, mode="random")
# We can plot the samples of each of the unions to see what we are working with.
# In[ ]:
fig, axs = plt.subplots(1, 3, figsize=(16, 4))
pts_list = [c_e_nb_u_points, c_e_b_u_points, three_domain_union_points]
title_list = [
"Cartesian with Ellipsoid No Border Union",
"Cartesian with Ellipsoid Border Union",
"Three Domain Union",
]
for ax, pts, title in zip(axs, pts_list, title_list):
plot_scatter(ax, pts, title)
# Now, we will find the differences of the geometries. We will find the difference of `cartesian` and `ellipsoid_no_border`.
# In[ ]:
cart_ellipse_nb_difference = Difference([cartesian, ellipsoid_no_border])
c_e_nb_d_points = cart_ellipse_nb_difference.sample(n=2000, mode="random")
fig, ax = plt.subplots(1, 1, figsize=(8, 6))
plot_scatter(ax, c_e_nb_d_points, "Difference")
# ## Create Custom DomainInterface
# We will take a look on how to create our own geometry. The one we will try to make is a heart defined by the function $$(x^2+y^2-1)^3-x^2y^3 \le 0$$
# Let's start by importing what we will need to create our own geometry based on this equation.
# In[11]:
import torch
from pina import LabelTensor
# Next, we will create the `Heart(DomainInterface)` class and initialize it.
# In[ ]:
class Heart(DomainInterface):
"""Implementation of the Heart Domain."""
def __init__(self, sample_border=False):
super().__init__()
# Because the `DomainInterface` class we are inheriting from requires both a `sample` method and `is_inside` method, we will create them and just add in "pass" for the moment. We also observe that the methods `sample_modes` and `variables` of the `DomainInterface` class are initialized as `abstractmethod`, so we need to redefine them both in the subclass `Heart` .
# In[ ]:
class Heart(DomainInterface):
"""Implementation of the Heart Domain."""
def __init__(self, sample_border=False):
super().__init__()
def is_inside(self):
pass
def sample(self):
pass
@property
def sample_modes(self):
pass
@property
def variables(self):
pass
# Now we have the skeleton for our `Heart` class. Also the `sample` method is where most of the work is done so let's fill it out.
# In[ ]:
class Heart(DomainInterface):
"""Implementation of the Heart Domain."""
def __init__(self, sample_border=False):
super().__init__()
def is_inside(self):
pass
def sample(self, n):
sampled_points = []
while len(sampled_points) < n:
x = torch.rand(1) * 3.0 - 1.5
y = torch.rand(1) * 3.0 - 1.5
if ((x**2 + y**2 - 1) ** 3 - (x**2) * (y**3)) <= 0:
sampled_points.append([x.item(), y.item()])
return LabelTensor(torch.tensor(sampled_points), labels=["x", "y"])
@property
def sample_modes(self):
pass
@property
def variables(self):
pass
# To create the Heart geometry we simply run:
# In[15]:
heart = Heart()
# To sample from the Heart geometry we simply run:
# In[ ]:
pts_heart = heart.sample(1500)
fig, ax = plt.subplots()
plot_scatter(ax, pts_heart, "Heart Domain")
# ## What's next?
#
# We have made a very simple tutorial on how to build custom geometries and use domain operation to compose base geometries. Now you can play around with different geometries and build your own!

159
tutorials/tutorial7/tutorial.ipynb vendored
View File

@@ -5,47 +5,34 @@
"id": "dbbb73cb-a632-4056-bbca-b483b2ad5f9c",
"metadata": {},
"source": [
"# Tutorial: Resolution of an inverse problem\n",
"# Tutorial: Inverse Problem Solving with Physics-Informed Neural Network\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial7/tutorial.ipynb)\n",
"\n",
"## Introduction to the Inverse Problem\n",
"\n",
"This tutorial demonstrates how to solve an inverse Poisson problem using Physics-Informed Neural Networks (PINNs).\n",
"\n",
"The problem is defined as a Poisson equation with homogeneous boundary conditions:\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial7/tutorial.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "84508f26-1ba6-4b59-926b-3e340d632a15",
"metadata": {},
"source": [
"### Introduction to the inverse problem"
]
},
{
"cell_type": "markdown",
"id": "cae54664-4572-49df-8b2d-9e7dd1e45ec0",
"metadata": {},
"source": [
"This tutorial shows how to solve an inverse Poisson problem with Physics-Informed Neural Networks. The problem definition is that of a Poisson problem with homogeneous boundary conditions and it reads:\n",
"\\begin{equation}\n",
"\\begin{cases}\n",
"\\Delta u = e^{-2(x-\\mu_1)^2-2(y-\\mu_2)^2} \\text{ in } \\Omega\\, ,\\\\\n",
"u = 0 \\text{ on }\\partial \\Omega,\\\\\n",
"\\Delta u = e^{-2(x - \\mu_1)^2 - 2(y - \\mu_2)^2} \\quad \\text{in } \\Omega, \\\\\n",
"u = 0 \\quad \\text{on } \\partial \\Omega, \\\\\n",
"u(\\mu_1, \\mu_2) = \\text{data}\n",
"\\end{cases}\n",
"\\end{equation}\n",
"where $\\Omega$ is a square domain $[-2, 2] \\times [-2, 2]$, and $\\partial \\Omega=\\Gamma_1 \\cup \\Gamma_2 \\cup \\Gamma_3 \\cup \\Gamma_4$ is the union of the boundaries of the domain.\n",
"\n",
"This kind of problem, namely the \"inverse problem\", has two main goals:\n",
"- find the solution $u$ that satisfies the Poisson equation;\n",
"- find the unknown parameters ($\\mu_1$, $\\mu_2$) that better fit some given data (third equation in the system above).\n",
"Here, $\\Omega$ is the square domain $[-2, 2] \\times [-2, 2]$, and $\\partial \\Omega = \\Gamma_1 \\cup \\Gamma_2 \\cup \\Gamma_3 \\cup \\Gamma_4$ represents the union of its boundaries.\n",
"\n",
"In order to achieve both goals we will need to define an `InverseProblem` in PINA."
]
},
{
"cell_type": "markdown",
"id": "c1f8cb1b-c1bc-4495-96e2-ce8e9102fe56",
"metadata": {},
"source": [
"Let's start with useful imports."
"This type of setup defines an *inverse problem*, which has two primary objectives:\n",
"\n",
"- **Find the solution** $u$ that satisfies the Poisson equation,\n",
"- **Identify the unknown parameters** $(\\mu_1, \\mu_2)$ that best fit the given data (as described by the third equation in the system).\n",
"\n",
"To tackle both objectives, we will define an `InverseProblem` using **PINA**.\n",
"\n",
"Let's begin with the necessary imports:\n"
]
},
{
@@ -67,7 +54,7 @@
"883"
]
},
"execution_count": 20,
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
@@ -81,7 +68,7 @@
"except:\n",
" IN_COLAB = False\n",
"if IN_COLAB:\n",
" !pip install \"pina-mathlab\"\n",
" !pip install \"pina-mathlab[tutorial]\"\n",
" # get the data\n",
" !mkdir \"data\"\n",
" !wget \"https://github.com/mathLab/PINA/raw/refs/heads/master/tutorials/tutorial7/data/pinn_solution_0.5_0.5\" -O \"data/pinn_solution_0.5_0.5\"\n",
@@ -91,6 +78,9 @@
"import torch\n",
"import warnings\n",
"\n",
"from lightning.pytorch import seed_everything\n",
"from lightning.pytorch.callbacks import Callback\n",
"\n",
"from pina import Condition, Trainer\n",
"from pina.problem import SpatialProblem, InverseProblem\n",
"from pina.operator import laplacian\n",
@@ -99,8 +89,6 @@
"from pina.solver import PINN\n",
"from pina.domain import CartesianDomain\n",
"from pina.optim import TorchOptimizer\n",
"from lightning.pytorch import seed_everything\n",
"from lightning.pytorch.callbacks import Callback\n",
"\n",
"warnings.filterwarnings(\"ignore\")\n",
"seed_everything(883)"
@@ -111,12 +99,20 @@
"id": "5138afdf-bff6-46bf-b423-a22673190687",
"metadata": {},
"source": [
"Then, we import the pre-saved data, for ($\\mu_1$, $\\mu_2$)=($0.5$, $0.5$). These two values are the optimal parameters that we want to find through the neural network training. In particular, we import the `input` points (the spatial coordinates), and the `target` points (the corresponding $u$ values evaluated at the `input`)."
"Next, we import the pre-saved data corresponding to the true parameter values $(\\mu_1, \\mu_2) = (0.5, 0.5)$. \n",
"These values represent the *optimal parameters* that we aim to recover through neural network training.\n",
"\n",
"In particular, we load:\n",
"\n",
"- `input` points — the spatial coordinates where observations are available,\n",
"- `target` points — the corresponding $u$ values (i.e., the solution evaluated at the `input` points).\n",
"\n",
"This data will be used to guide the inverse problem and supervise the network’s prediction of the unknown parameters."
]
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 11,
"id": "2c55d972-09a9-41de-9400-ba051c28cdcb",
"metadata": {},
"outputs": [],
@@ -132,12 +128,17 @@
"id": "6541ffbe-7940-421a-9048-a796ec56f1d6",
"metadata": {},
"source": [
"Moreover, let's plot also the data points and the reference solution: this is the expected output of the neural network."
"Next, let's visualize the data:\n",
"\n",
"- We'll plot the data points, i.e., the spatial coordinates where measurements are available.\n",
"- We'll also display the reference solution corresponding to $(\\mu_1, \\mu_2) = (0.5, 0.5)$.\n",
"\n",
"This serves as the ground truth or expected output that our neural network should learn to approximate through training."
]
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": 12,
"id": "55cef553-7495-401d-9d17-1acff8ec5953",
"metadata": {},
"outputs": [
@@ -167,20 +168,17 @@
"id": "de7c4c83",
"metadata": {},
"source": [
"### Inverse problem definition in PINA"
]
},
{
"cell_type": "markdown",
"id": "c46410fa-2718-4fc9-977a-583fe2390028",
"metadata": {},
"source": [
"Then, we initialize the Poisson problem, that is inherited from the `SpatialProblem` and from the `InverseProblem` classes. We here have to define all the variables, and the domain where our unknown parameters ($\\mu_1$, $\\mu_2$) belong. Notice that the Laplace equation takes as inputs also the unknown variables, that will be treated as parameters that the neural network optimizes during the training process."
"## Inverse Problem Definition in PINA\n",
"\n",
"Next, we initialize the Poisson problem, which inherits from the `SpatialProblem` and `InverseProblem` classes. \n",
"In this step, we need to define all the variables and specify the domain in which our unknown parameters $(\\mu_1, \\mu_2)$ reside.\n",
"\n",
"Note that the Laplace equation also takes these unknown parameters as inputs. These parameters will be treated as variables that the neural network will optimize during the training process, enabling it to learn the optimal values for $(\\mu_1, \\mu_2)$."
]
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": 13,
"id": "8ec0d95d-72c2-40a4-a310-21c3d6fe17d2",
"metadata": {},
"outputs": [],
@@ -204,11 +202,6 @@
"\n",
"\n",
"class Poisson(SpatialProblem, InverseProblem):\n",
" r\"\"\"\n",
" Implementation of the inverse 2-dimensional Poisson problem in the square\n",
" domain :math:`[0, 1] \\times [0, 1]`,\n",
" with unknown parameter domain :math:`[-1, 1] \\times [-1, 1]`.\n",
" \"\"\"\n",
"\n",
" output_variables = [\"u\"]\n",
" x_min, x_max = -2, 2\n",
@@ -242,12 +235,12 @@
"id": "6b264569-57b3-458d-bb69-8e94fe89017d",
"metadata": {},
"source": [
"Then, we define the neural network model we want to use. Here we used a model which imposes hard constrains on the boundary conditions, as also done in the Wave tutorial!"
"Next, we define the neural network model that will be used for solving the inverse problem. In this case, we use a simple FeedForeard model, but you could build one that imposes *hard constraints* on the boundary conditions, similar to the approach used in the [Wave tutorial](https://mathlab.github.io/PINA/tutorial3/tutorial.html) to have better performances!"
]
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": 14,
"id": "c4170514-eb73-488e-8942-0129070e4e13",
"metadata": {},
"outputs": [],
@@ -270,7 +263,7 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": 15,
"id": "e3e0ae40-d8c6-4c08-81e8-85adc60a94e6",
"metadata": {},
"outputs": [],
@@ -288,19 +281,21 @@
"id": "b272796a-888c-4795-9d88-3e13121e8f38",
"metadata": {},
"source": [
"Here, we define a simple callback for the trainer. We use this callback to save the parameters predicted by the neural network during the training. The parameters are saved every 100 epochs as `torch` tensors in a specified directory (`tmp_dir` in our case).\n",
"The goal is to read the saved parameters after training and plot their trend across the epochs."
"Here, we define a simple callback for the trainer. This callback is used to save the parameters predicted by the neural network during training. \n",
"The parameters are saved every 100 epochs as `torch` tensors in a specified directory (in our case, `tutorial_logs`).\n",
"\n",
"The goal of this setup is to read the saved parameters after training and visualize their trend across the epochs. This allows us to monitor how the predicted parameters evolve throughout the training process.\n"
]
},
{
"cell_type": "code",
"execution_count": 26,
"execution_count": 16,
"id": "e1409953-eb1b-443b-923d-c7ec3af0dfb0",
"metadata": {},
"outputs": [],
"source": [
"# temporary directory for saving logs of training\n",
"tmp_dir = \"tmp_poisson_inverse\"\n",
"tmp_dir = \"tutorial_logs\"\n",
"\n",
"\n",
"class SaveParameters(Callback):\n",
@@ -321,12 +316,12 @@
"id": "fc6e0030-f6ae-40cf-a3b3-d21d6538e7f2",
"metadata": {},
"source": [
"Then, we define the `PINN` object and train the solver using the `Trainer`."
"Then, we define the `PINN` object and train the solver using the `Trainer`"
]
},
{
"cell_type": "code",
"execution_count": 27,
"execution_count": 17,
"id": "05a0f311-3cca-429b-be2c-1fa899b14e62",
"metadata": {},
"outputs": [
@@ -340,25 +335,25 @@
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1499: 100%|██████████| 1/1 [00:00<00:00, 68.34it/s, v_num=2, g1_loss=0.000142, g2_loss=3.78e-5, g3_loss=0.000105, g4_loss=3.2e-5, D_loss=0.000561, data_loss=2.71e-5, train_loss=0.000906] "
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "ca3282f5c0654d9d8d4335107e7254e1",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: | | 0/? [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"`Trainer.fit` stopped: `max_epochs=1500` reached.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1499: 100%|██████████| 1/1 [00:00<00:00, 54.70it/s, v_num=2, g1_loss=0.000142, g2_loss=3.78e-5, g3_loss=0.000105, g4_loss=3.2e-5, D_loss=0.000561, data_loss=2.71e-5, train_loss=0.000906]\n"
]
}
],
"source": [
@@ -433,15 +428,17 @@
"id": "f1fa4406",
"metadata": {},
"source": [
"## What's next?\n",
"## What's Next?\n",
"\n",
"We have shown the basic usage PINNs in inverse problem modelling, further extensions include:\n",
"We have covered the basic usage of PINNs for inverse problem modeling. Here are some possible directions for further exploration:\n",
"\n",
"1. Train using different Physics Informed strategies\n",
"1. **Experiment with different Physics-Informed strategies**: Explore variations in PINN training techniques to improve performance or tackle different types of problems.\n",
"\n",
"2. Try on more complex problems\n",
"2. **Apply to more complex problems**: Scale the approach to higher-dimensional or time-dependent inverse problems.\n",
"\n",
"3. Many more..."
"3. **...and many more!**: The possibilities are endless, from integrating additional physical constraints to testing on real-world datasets.\n",
"\n",
"For more resources and tutorials, check out the [PINA Documentation](https://mathlab.github.io/PINA/)."
]
}
],

255
tutorials/tutorial7/tutorial.py vendored
View File

@@ -1,255 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Resolution of an inverse problem
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial7/tutorial.ipynb)
# ### Introduction to the inverse problem
# This tutorial shows how to solve an inverse Poisson problem with Physics-Informed Neural Networks. The problem definition is that of a Poisson problem with homogeneous boundary conditions and it reads:
# \begin{equation}
# \begin{cases}
# \Delta u = e^{-2(x-\mu_1)^2-2(y-\mu_2)^2} \text{ in } \Omega\, ,\\
# u = 0 \text{ on }\partial \Omega,\\
# u(\mu_1, \mu_2) = \text{ data}
# \end{cases}
# \end{equation}
# where $\Omega$ is a square domain $[-2, 2] \times [-2, 2]$, and $\partial \Omega=\Gamma_1 \cup \Gamma_2 \cup \Gamma_3 \cup \Gamma_4$ is the union of the boundaries of the domain.
#
# This kind of problem, namely the "inverse problem", has two main goals:
# - find the solution $u$ that satisfies the Poisson equation;
# - find the unknown parameters ($\mu_1$, $\mu_2$) that better fit some given data (third equation in the system above).
#
# In order to achieve both goals we will need to define an `InverseProblem` in PINA.
# Let's start with useful imports.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
# get the data
get_ipython().system('mkdir "data"')
get_ipython().system('wget "https://github.com/mathLab/PINA/raw/refs/heads/master/tutorials/tutorial7/data/pinn_solution_0.5_0.5" -O "data/pinn_solution_0.5_0.5"')
get_ipython().system('wget "https://github.com/mathLab/PINA/raw/refs/heads/master/tutorials/tutorial7/data/pts_0.5_0.5" -O "data/pts_0.5_0.5"')
import matplotlib.pyplot as plt
import torch
import warnings
from pina import Condition, Trainer
from pina.problem import SpatialProblem, InverseProblem
from pina.operator import laplacian
from pina.model import FeedForward
from pina.equation import Equation, FixedValue
from pina.solver import PINN
from pina.domain import CartesianDomain
from pina.optim import TorchOptimizer
from lightning.pytorch import seed_everything
from lightning.pytorch.callbacks import Callback
warnings.filterwarnings("ignore")
seed_everything(883)
# Then, we import the pre-saved data, for ($\mu_1$, $\mu_2$)=($0.5$, $0.5$). These two values are the optimal parameters that we want to find through the neural network training. In particular, we import the `input` points (the spatial coordinates), and the `target` points (the corresponding $u$ values evaluated at the `input`).
# In[21]:
data_output = torch.load(
"data/pinn_solution_0.5_0.5", weights_only=False
).detach()
data_input = torch.load("data/pts_0.5_0.5", weights_only=False)
# Moreover, let's plot also the data points and the reference solution: this is the expected output of the neural network.
# In[22]:
points = data_input.extract(["x", "y"]).detach().numpy()
truth = data_output.detach().numpy()
plt.scatter(points[:, 0], points[:, 1], c=truth, s=8)
plt.axis("equal")
plt.colorbar()
plt.show()
# ### Inverse problem definition in PINA
# Then, we initialize the Poisson problem, that is inherited from the `SpatialProblem` and from the `InverseProblem` classes. We here have to define all the variables, and the domain where our unknown parameters ($\mu_1$, $\mu_2$) belong. Notice that the Laplace equation takes as inputs also the unknown variables, that will be treated as parameters that the neural network optimizes during the training process.
# In[23]:
def laplace_equation(input_, output_, params_):
"""
Implementation of the laplace equation.
:param LabelTensor input_: Input data of the problem.
:param LabelTensor output_: Output data of the problem.
:param dict params_: Parameters of the problem.
:return: The residual of the laplace equation.
:rtype: LabelTensor
"""
force_term = torch.exp(
-2 * (input_.extract(["x"]) - params_["mu1"]) ** 2
- 2 * (input_.extract(["y"]) - params_["mu2"]) ** 2
)
delta_u = laplacian(output_, input_, components=["u"], d=["x", "y"])
return delta_u - force_term
class Poisson(SpatialProblem, InverseProblem):
r"""
Implementation of the inverse 2-dimensional Poisson problem in the square
domain :math:`[0, 1] \times [0, 1]`,
with unknown parameter domain :math:`[-1, 1] \times [-1, 1]`.
"""
output_variables = ["u"]
x_min, x_max = -2, 2
y_min, y_max = -2, 2
spatial_domain = CartesianDomain({"x": [x_min, x_max], "y": [y_min, y_max]})
unknown_parameter_domain = CartesianDomain({"mu1": [-1, 1], "mu2": [-1, 1]})
domains = {
"g1": CartesianDomain({"x": [x_min, x_max], "y": y_max}),
"g2": CartesianDomain({"x": [x_min, x_max], "y": y_min}),
"g3": CartesianDomain({"x": x_max, "y": [y_min, y_max]}),
"g4": CartesianDomain({"x": x_min, "y": [y_min, y_max]}),
"D": CartesianDomain({"x": [x_min, x_max], "y": [y_min, y_max]}),
}
conditions = {
"g1": Condition(domain="g1", equation=FixedValue(0.0)),
"g2": Condition(domain="g2", equation=FixedValue(0.0)),
"g3": Condition(domain="g3", equation=FixedValue(0.0)),
"g4": Condition(domain="g4", equation=FixedValue(0.0)),
"D": Condition(domain="D", equation=Equation(laplace_equation)),
"data": Condition(input=data_input, target=data_output),
}
problem = Poisson()
# Then, we define the neural network model we want to use. Here we used a model which imposes hard constrains on the boundary conditions, as also done in the Wave tutorial!
# In[24]:
model = FeedForward(
layers=[20, 20, 20],
func=torch.nn.Softplus,
output_dimensions=len(problem.output_variables),
input_dimensions=len(problem.input_variables),
)
# After that, we discretize the spatial domain.
# In[25]:
problem.discretise_domain(20, "grid", domains=["D"])
problem.discretise_domain(
1000,
"random",
domains=["g1", "g2", "g3", "g4"],
)
# Here, we define a simple callback for the trainer. We use this callback to save the parameters predicted by the neural network during the training. The parameters are saved every 100 epochs as `torch` tensors in a specified directory (`tmp_dir` in our case).
# The goal is to read the saved parameters after training and plot their trend across the epochs.
# In[26]:
# temporary directory for saving logs of training
tmp_dir = "tmp_poisson_inverse"
class SaveParameters(Callback):
"""
Callback to save the parameters of the model every 100 epochs.
"""
def on_train_epoch_end(self, trainer, __):
if trainer.current_epoch % 100 == 99:
torch.save(
trainer.solver.problem.unknown_parameters,
"{}/parameters_epoch{}".format(tmp_dir, trainer.current_epoch),
)
# Then, we define the `PINN` object and train the solver using the `Trainer`.
# In[27]:
max_epochs = 1500
pinn = PINN(
problem, model, optimizer=TorchOptimizer(torch.optim.Adam, lr=0.005)
)
# define the trainer for the solver
trainer = Trainer(
solver=pinn,
accelerator="cpu",
max_epochs=max_epochs,
default_root_dir=tmp_dir,
enable_model_summary=False,
callbacks=[SaveParameters()],
train_size=1.0,
val_size=0.0,
test_size=0.0,
)
trainer.train()
# One can now see how the parameters vary during the training by reading the saved solution and plotting them. The plot shows that the parameters stabilize to their true value before reaching the epoch $1000$!
# In[28]:
epochs_saved = range(99, max_epochs, 100)
parameters = torch.empty((int(max_epochs / 100), 2))
for i, epoch in enumerate(epochs_saved):
params_torch = torch.load(
"{}/parameters_epoch{}".format(tmp_dir, epoch), weights_only=False
)
for e, var in enumerate(pinn.problem.unknown_variables):
parameters[i, e] = params_torch[var].data
# Plot parameters
plt.close()
plt.plot(epochs_saved, parameters[:, 0], label="mu1", marker="o")
plt.plot(epochs_saved, parameters[:, 1], label="mu2", marker="s")
plt.ylim(-1, 1)
plt.grid()
plt.legend()
plt.xlabel("Epoch")
plt.ylabel("Parameter value")
plt.show()
# ## What's next?
#
# We have shown the basic usage PINNs in inverse problem modelling, further extensions include:
#
# 1. Train using different Physics Informed strategies
#
# 2. Try on more complex problems
#
# 3. Many more...

683
tutorials/tutorial8/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

315
tutorials/tutorial8/tutorial.py vendored
View File

@@ -1,315 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: Reduced order models (POD-NN and POD-RBF) for parametric problems
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial9/tutorial.ipynb)
# The tutorial aims to show how to employ the **PINA** library in order to apply a reduced order modeling technique [1]. Such methodologies have several similarities with machine learning approaches, since the main goal consists in predicting the solution of differential equations (typically parametric PDEs) in a real-time fashion.
#
# In particular we are going to use the Proper Orthogonal Decomposition with either Radial Basis Function Interpolation (POD-RBF) or Neural Network (POD-NN) [2]. Here we basically perform a dimensional reduction using the POD approach, approximating the parametric solution manifold (at the reduced space) using a regression technique (NN) and comparing it to an RBF interpolation. In this example, we use a simple multilayer perceptron, but the plenty of different architectures can be plugged as well.
# Let's start with the necessary imports.
# It's important to note the minimum PINA version to run this tutorial is the `0.1`.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
get_ipython().run_line_magic('matplotlib', 'inline')
import matplotlib
import matplotlib.pyplot as plt
import torch
import numpy as np
import warnings
from pina import Trainer
from pina.model import FeedForward
from pina.solver import SupervisedSolver
from pina.optim import TorchOptimizer
from pina.problem.zoo import SupervisedProblem
from pina.model.block import PODBlock, RBFBlock
warnings.filterwarnings("ignore")
# We exploit the [Smithers](https://github.com/mathLab/Smithers) library to collect the parametric snapshots. In particular, we use the `NavierStokesDataset` class that contains a set of parametric solutions of the Navier-Stokes equations in a 2D L-shape domain. The parameter is the inflow velocity.
# The dataset is composed by 500 snapshots of the velocity (along $x$, $y$, and the magnitude) and pressure fields, and the corresponding parameter values.
#
# To visually check the snapshots, let's plot also the data points and the reference solution: this is the expected output of our model.
# In[83]:
from smithers.dataset import NavierStokesDataset
dataset = NavierStokesDataset()
fig, axs = plt.subplots(1, 4, figsize=(14, 3))
for ax, p, u in zip(axs, dataset.params[:4], dataset.snapshots["mag(v)"][:4]):
ax.tricontourf(dataset.triang, u, levels=16)
ax.set_title(f"$\mu$ = {p[0]:.2f}")
# The *snapshots* - aka the numerical solutions computed for several parameters - and the corresponding parameters are the only data we need to train the model, in order to predict the solution for any new test parameter. To properly validate the accuracy, we will split the 500 snapshots into the training dataset (90% of the original data) and the testing one (the reamining 10%) inside the `Trainer`.
#
# It is now time to define the problem!
# In[84]:
u = torch.tensor(dataset.snapshots["mag(v)"]).float()
p = torch.tensor(dataset.params).float()
problem = SupervisedProblem(input_=p, output_=u)
# We can then build a `POD-NN` model (using an MLP architecture as approximation) and compare it with a `POD-RBF` model (using a Radial Basis Function interpolation as approximation).
# ## POD-NN reduced order model
# Let's build the `PODNN` class
# In[85]:
class PODNN(torch.nn.Module):
"""
Proper orthogonal decomposition with neural network model.
"""
def __init__(self, pod_rank, layers, func):
""" """
super().__init__()
self.pod = PODBlock(pod_rank)
self.nn = FeedForward(
input_dimensions=1,
output_dimensions=pod_rank,
layers=layers,
func=func,
)
def forward(self, x):
"""
Defines the computation performed at every call.
:param x: The tensor to apply the forward pass.
:type x: torch.Tensor
:return: the output computed by the model.
:rtype: torch.Tensor
"""
coefficents = self.nn(x)
return self.pod.expand(coefficents)
def fit_pod(self, x):
"""
Just call the :meth:`pina.model.layers.PODBlock.fit` method of the
:attr:`pina.model.layers.PODBlock` attribute.
"""
self.pod.fit(x)
# We highlight that the POD modes are directly computed by means of the singular value decomposition (computed over the input data), and not trained using the backpropagation approach. Only the weights of the MLP are actually trained during the optimization loop.
# In[86]:
pod_nn = PODNN(pod_rank=20, layers=[10, 10, 10], func=torch.nn.Tanh)
pod_nn_stokes = SupervisedSolver(
problem=problem,
model=pod_nn,
optimizer=TorchOptimizer(torch.optim.Adam, lr=0.0001),
use_lt=False,
)
# Before starting we need to fit the POD basis on the training dataset, this can be easily done in PINA as well:
# In[87]:
trainer = Trainer(
solver=pod_nn_stokes,
max_epochs=1000,
batch_size=None,
accelerator="cpu",
train_size=0.9,
val_size=0.0,
test_size=0.1,
)
# fit the pod basis
trainer.data_module.setup("fit") # set up the dataset
x_train = trainer.data_module.train_dataset.conditions_dict["data"][
"target"
] # extract data for training
pod_nn.fit_pod(x=x_train)
# now train
trainer.train()
# Done! Now that the computational expensive part is over, we can load in future the model to infer new parameters (simply loading the checkpoint file automatically created by `Lightning`) or test its performances. We measure the relative error for the training and test datasets, printing the mean one.
# In[ ]:
# extract train and test data
trainer.data_module.setup("test") # set up the dataset
p_train = trainer.data_module.train_dataset.conditions_dict["data"]["input"]
u_train = trainer.data_module.train_dataset.conditions_dict["data"]["target"]
p_test = trainer.data_module.test_dataset.conditions_dict["data"]["input"]
u_test = trainer.data_module.test_dataset.conditions_dict["data"]["target"]
# compute statistics
u_test_nn = pod_nn_stokes(p_test)
u_train_nn = pod_nn_stokes(p_train)
relative_error_train = torch.norm(u_train_nn - u_train) / torch.norm(u_train)
relative_error_test = torch.norm(u_test_nn - u_test) / torch.norm(u_test)
print("Error summary for POD-NN model:")
print(f" Train: {relative_error_train.item():e}")
print(f" Test: {relative_error_test.item():e}")
# ## POD-RBF reduced order model
# Then, we define the model we want to use, with the POD (`PODBlock`) and the RBF (`RBFBlock`) objects.
# In[89]:
class PODRBF(torch.nn.Module):
"""
Proper orthogonal decomposition with Radial Basis Function interpolation model.
"""
def __init__(self, pod_rank, rbf_kernel):
""" """
super().__init__()
self.pod = PODBlock(pod_rank)
self.rbf = RBFBlock(kernel=rbf_kernel)
def forward(self, x):
"""
Defines the computation performed at every call.
:param x: The tensor to apply the forward pass.
:type x: torch.Tensor
:return: the output computed by the model.
:rtype: torch.Tensor
"""
coefficents = self.rbf(x)
return self.pod.expand(coefficents)
def fit(self, p, x):
"""
Call the :meth:`pina.model.layers.PODBlock.fit` method of the
:attr:`pina.model.layers.PODBlock` attribute to perform the POD,
and the :meth:`pina.model.layers.RBFBlock.fit` method of the
:attr:`pina.model.layers.RBFBlock` attribute to fit the interpolation.
"""
self.pod.fit(x)
self.rbf.fit(p, self.pod.reduce(x))
# We can then fit the model and ask it to predict the required field for unseen values of the parameters. Note that this model does not need a `Trainer` since it does not include any neural network or learnable parameters.
# In[90]:
pod_rbf = PODRBF(pod_rank=20, rbf_kernel="thin_plate_spline")
pod_rbf.fit(p_train, u_train)
# Compute errors
# In[91]:
u_test_rbf = pod_rbf(p_test)
u_train_rbf = pod_rbf(p_train)
relative_error_train = torch.norm(u_train_rbf - u_train) / torch.norm(u_train)
relative_error_test = torch.norm(u_test_rbf - u_test) / torch.norm(u_test)
print("Error summary for POD-RBF model:")
print(f" Train: {relative_error_train.item():e}")
print(f" Test: {relative_error_test.item():e}")
# ## POD-RBF vs POD-NN
# We can of course also plot the solutions predicted by the `PODRBF` and by the `PODNN` model, comparing them to the original ones. We can note here, in the `PODNN` model and for low velocities, some differences, but improvements can be accomplished thanks to longer training.
# In[92]:
idx = torch.randint(0, len(u_test), (4,))
u_idx_rbf = pod_rbf(p_test[idx])
u_idx_nn = pod_nn_stokes(p_test[idx])
fig, axs = plt.subplots(4, 5, figsize=(14, 9))
relative_error_rbf = np.abs(u_test[idx] - u_idx_rbf.detach())
relative_error_rbf = np.where(
u_test[idx] < 1e-7, 1e-7, relative_error_rbf / u_test[idx]
)
relative_error_nn = np.abs(u_test[idx] - u_idx_nn.detach())
relative_error_nn = np.where(
u_test[idx] < 1e-7, 1e-7, relative_error_nn / u_test[idx]
)
for i, (idx_, rbf_, nn_, rbf_err_, nn_err_) in enumerate(
zip(idx, u_idx_rbf, u_idx_nn, relative_error_rbf, relative_error_nn)
):
axs[0, 0].set_title(f"Real Snapshots")
axs[0, 1].set_title(f"POD-RBF")
axs[0, 2].set_title(f"POD-NN")
axs[0, 3].set_title(f"Error POD-RBF")
axs[0, 4].set_title(f"Error POD-NN")
cm = axs[i, 0].tricontourf(
dataset.triang, rbf_.detach()
) # POD-RBF prediction
plt.colorbar(cm, ax=axs[i, 0])
cm = axs[i, 1].tricontourf(
dataset.triang, nn_.detach()
) # POD-NN prediction
plt.colorbar(cm, ax=axs[i, 1])
cm = axs[i, 2].tricontourf(dataset.triang, u_test[idx_].flatten()) # Truth
plt.colorbar(cm, ax=axs[i, 2])
cm = axs[i, 3].tripcolor(
dataset.triang, rbf_err_, norm=matplotlib.colors.LogNorm()
) # Error for POD-RBF
plt.colorbar(cm, ax=axs[i, 3])
cm = axs[i, 4].tripcolor(
dataset.triang, nn_err_, norm=matplotlib.colors.LogNorm()
) # Error for POD-NN
plt.colorbar(cm, ax=axs[i, 4])
plt.show()
# #### References
# 1. Rozza G., Stabile G., Ballarin F. (2022). Advanced Reduced Order Methods and Applications in Computational Fluid Dynamics, Society for Industrial and Applied Mathematics.
# 2. Hesthaven, J. S., & Ubbiali, S. (2018). Non-intrusive reduced order modeling of nonlinear problems using neural networks. Journal of Computational Physics, 363, 55-78.

211
tutorials/tutorial9/tutorial.ipynb vendored
View File

File diff suppressed because one or more lines are too long

246
tutorials/tutorial9/tutorial.py vendored
View File

@@ -1,246 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
# # Tutorial: One dimensional Helmholtz equation using Periodic Boundary Conditions
#
# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mathLab/PINA/blob/master/tutorials/tutorial9/tutorial.ipynb)
#
# This tutorial presents how to solve with Physics-Informed Neural Networks (PINNs)
# a one dimensional Helmholtz equation with periodic boundary conditions (PBC).
# We will train with standard PINN's training by augmenting the input with
# periodic expansion as presented in [*An expert’s guide to training
# physics-informed neural networks*](
# https://arxiv.org/abs/2308.08468).
#
# First of all, some useful imports.
# In[ ]:
## routine needed to run the notebook on Google Colab
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
if IN_COLAB:
get_ipython().system('pip install "pina-mathlab"')
import torch
import matplotlib.pyplot as plt
import warnings
from pina import Condition, Trainer
from pina.problem import SpatialProblem
from pina.operator import laplacian
from pina.model import FeedForward
from pina.model.block import PeriodicBoundaryEmbedding # The PBC module
from pina.solver import PINN
from pina.domain import CartesianDomain
from pina.equation import Equation
from pina.callback import MetricTracker
warnings.filterwarnings("ignore")
# ## The problem definition
#
# The one-dimensional Helmholtz problem is mathematically written as:
# $$
# \begin{cases}
# \frac{d^2}{dx^2}u(x) - \lambda u(x) -f(x) &= 0 \quad x\in(0,2)\\
# u^{(m)}(x=0) - u^{(m)}(x=2) &= 0 \quad m\in[0, 1, \cdots]\\
# \end{cases}
# $$
# In this case we are asking the solution to be $C^{\infty}$ periodic with
# period $2$, on the infinite domain $x\in(-\infty, \infty)$. Notice that the
# classical PINN would need infinite conditions to evaluate the PBC loss function,
# one for each derivative, which is of course infeasible...
# A possible solution, diverging from the original PINN formulation,
# is to use *coordinates augmentation*. In coordinates augmentation you seek for
# a coordinates transformation $v$ such that $x\rightarrow v(x)$ such that
# the periodicity condition $ u^{(m)}(x=0) - u^{(m)}(x=2) = 0 \quad m\in[0, 1, \cdots] $ is
# satisfied.
#
# For demonstration purposes, the problem specifics are $\lambda=-10\pi^2$,
# and $f(x)=-6\pi^2\sin(3\pi x)\cos(\pi x)$ which give a solution that can be
# computed analytically $u(x) = \sin(\pi x)\cos(3\pi x)$.
# In[15]:
def helmholtz_equation(input_, output_):
x = input_.extract("x")
u_xx = laplacian(output_, input_, components=["u"], d=["x"])
f = (
-6.0
* torch.pi**2
* torch.sin(3 * torch.pi * x)
* torch.cos(torch.pi * x)
)
lambda_ = -10.0 * torch.pi**2
return u_xx - lambda_ * output_ - f
class Helmholtz(SpatialProblem):
output_variables = ["u"]
spatial_domain = CartesianDomain({"x": [0, 2]})
# here we write the problem conditions
conditions = {
"phys_cond": Condition(
domain=spatial_domain, equation=Equation(helmholtz_equation)
),
}
def solution(self, pts):
return torch.sin(torch.pi * pts) * torch.cos(3.0 * torch.pi * pts)
problem = Helmholtz()
# let's discretise the domain
problem.discretise_domain(200, "grid", domains=["phys_cond"])
# As usual, the Helmholtz problem is written in **PINA** code as a class.
# The equations are written as `conditions` that should be satisfied in the
# corresponding domains. The `solution`
# is the exact solution which will be compared with the predicted one. We used
# Latin Hypercube Sampling for choosing the collocation points.
# ## Solving the problem with a Periodic Network
# Any $\mathcal{C}^{\infty}$ periodic function
# $u : \mathbb{R} \rightarrow \mathbb{R}$ with period
# $L\in\mathbb{N}$ can be constructed by composition of an
# arbitrary smooth function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ and a
# given smooth periodic function $v : \mathbb{R} \rightarrow \mathbb{R}^n$ with
# period $L$, that is $u(x) = f(v(x))$. The formulation is generalizable for
# arbitrary dimension, see [*A method for representing periodic functions and
# enforcing exactly periodic boundary conditions with
# deep neural networks*](https://arxiv.org/pdf/2007.07442).
#
# In our case, we rewrite
# $v(x) = \left[1, \cos\left(\frac{2\pi}{L} x\right),
# \sin\left(\frac{2\pi}{L} x\right)\right]$, i.e
# the coordinates augmentation, and $f(\cdot) = NN_{\theta}(\cdot)$ i.e. a neural
# network. The resulting neural network obtained by composing $f$ with $v$ gives
# the PINN approximate solution, that is
# $u(x) \approx u_{\theta}(x)=NN_{\theta}(v(x))$.
#
# In **PINA** this translates in using the `PeriodicBoundaryEmbedding` layer for $v$, and any
# `pina.model` for $NN_{\theta}$. Let's see it in action!
#
# In[16]:
# we encapsulate all modules in a torch.nn.Sequential container
model = torch.nn.Sequential(
PeriodicBoundaryEmbedding(input_dimension=1, periods=2),
FeedForward(
input_dimensions=3, # output of PeriodicBoundaryEmbedding = 3 * input_dimension
output_dimensions=1,
layers=[10, 10],
),
)
# As simple as that! Notice that in higher dimension you can specify different periods
# for all dimensions using a dictionary, e.g. `periods={'x':2, 'y':3, ...}`
# would indicate a periodicity of $2$ in $x$, $3$ in $y$, and so on...
#
# We will now solve the problem as usually with the `PINN` and `Trainer` class, then we will look at the losses using the `MetricTracker` callback from `pina.callback`.
# In[17]:
pinn = PINN(
problem=problem,
model=model,
)
trainer = Trainer(
pinn,
max_epochs=5000,
accelerator="cpu",
enable_model_summary=False,
callbacks=[MetricTracker()],
train_size=1.0,
val_size=0.0,
test_size=0.0,
)
trainer.train()
# In[18]:
# plot loss
trainer_metrics = trainer.callbacks[0].metrics
plt.plot(
range(len(trainer_metrics["train_loss"])), trainer_metrics["train_loss"]
)
# plotting
plt.xlabel("epoch")
plt.ylabel("loss")
plt.yscale("log")
# We are going to plot the solution now!
# In[19]:
pts = pinn.problem.spatial_domain.sample(256, "grid", variables="x")
predicted_output = pinn.forward(pts).extract("u").tensor.detach()
true_output = pinn.problem.solution(pts)
plt.plot(pts.extract(["x"]), predicted_output, label="Neural Network solution")
plt.plot(pts.extract(["x"]), true_output, label="True solution")
plt.legend()
# Great, they overlap perfectly! This seems a good result, considering the simple neural network used to some this (complex) problem. We will now test the neural network on the domain $[-4, 4]$ without retraining. In principle the periodicity should be present since the $v$ function ensures the periodicity in $(-\infty, \infty)$.
# In[20]:
# plotting solution
with torch.no_grad():
# Notice here we put [-4, 4]!!!
new_domain = CartesianDomain({"x": [0, 4]})
x = new_domain.sample(1000, mode="grid")
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
# Plot 1
axes[0].plot(x, problem.solution(x), label=r"$u(x)$", color="blue")
axes[0].set_title(r"True solution $u(x)$")
axes[0].legend(loc="upper right")
# Plot 2
axes[1].plot(x, pinn(x), label=r"$u_{\theta}(x)$", color="green")
axes[1].set_title(r"PINN solution $u_{\theta}(x)$")
axes[1].legend(loc="upper right")
# Plot 3
diff = torch.abs(problem.solution(x) - pinn(x))
axes[2].plot(x, diff, label=r"$|u(x) - u_{\theta}(x)|$", color="red")
axes[2].set_title(r"Absolute difference $|u(x) - u_{\theta}(x)|$")
axes[2].legend(loc="upper right")
# Adjust layout
plt.tight_layout()
# Show the plots
plt.show()
# It is pretty clear that the network is periodic, with also the error following a periodic pattern. Obviously a longer training and a more expressive neural network could improve the results!
#
# ## What's next?
#
# Congratulations on completing the one dimensional Helmholtz tutorial of **PINA**! There are multiple directions you can go now:
#
# 1. Train the network for longer or with different layer sizes and assert the finaly accuracy
#
# 2. Apply the `PeriodicBoundaryEmbedding` layer for a time-dependent problem (see reference in the documentation)
#
# 3. Exploit extrafeature training ?
#
# 4. Many more...