From 4f7cfbf200d98f82dca46179384a233a3fc3dec9 Mon Sep 17 00:00:00 2001 From: tromain <77615882+tromain@users.noreply.github.com> Date: Wed, 6 Nov 2024 11:20:03 +0100 Subject: [PATCH 01/37] Add tuto greenIT --- tuto_greenit.ipynb | 676 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 676 insertions(+) create mode 100644 tuto_greenit.ipynb diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb new file mode 100644 index 0000000..dbe6618 --- /dev/null +++ b/tuto_greenit.ipynb @@ -0,0 +1,676 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "371254df-a3b4-4f3e-b34f-3c3db06828ba", + "metadata": {}, + "source": [ + "# Reading rasters with rasterio\n" + ] + }, + { + "cell_type": "markdown", + "id": "fb2fcaf9-5b7b-4f37-b457-6e01e5035fbf", + "metadata": {}, + "source": [ + "# Downscaling rasters thanks to dask\n", + "\n", + "In this notebook we will look at a concrete case of data aggregation, with the example of changing the resolution using dask.\n", + "\n", + "Dask is a Python library that enables calculations to be parallelized and large quantities of data to be handled in a scalable way, using available resources (CPU, memory, etc.). Unlike tools such as Pandas or NumPy, Dask allows you to work on data sets that exceed the available RAM memory by chunking the data into smaller pieces (chunks) and parallelizing calculations.\n", + "Dask works with lazy computation: instead of executing immediately, it builds a task graph. Calculations are only executed when the final result is explicitly requested (for example, by calling .compute()).\n", + "\n", + "Dask offers a wide range of [modules](https://docs.dask.org/en/stable/#how-to-use-dask) (dask dataframe...) that can be used to distribute calculations. In this notebook we will focus on the use of [dask-arrays](https://docs.dask.org/en/stable/array.html) to manipulate raster data.\n", + "\n", + "In order to change the resolution, we will look at two possible methods. One of them will prove less effective than the other, which will allow us to establish some general principles to follow for optimal use of Dask. The first one will use [map_blocks](https://docs.dask.org/en/stable/generated/dask.array.map_blocks.html) and the second one only [dask.array.mean](https://docs.dask.org/en/stable/generated/dask.array.mean.html)\n", + "\n", + "For this tutorial, we will use a Sentinel-2 acquisition stored on the local disk. We will target the storage directory under the variable ``sentinel_2_dir``. Users can modify this directory and the associated paths.\n", + "\n", + "## Python scripts\n", + "\n", + "### Import libraries\n", + "\n", + "First let's import libraries needed for this tutorial and create our dask [LocalCluster](https://docs.dask.org/en/stable/deploying-python.html#localcluster) which allow us to create workers and use [dask's dashboard](https://docs.dask.org/en/latest/dashboard.html).\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.10/dist-packages/distributed/node.py:182: UserWarning: Port 8787 is already in use.\n", + "Perhaps you already have a cluster running?\n", + "Hosting the HTTP server on port 37319 instead\n", + " warnings.warn(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Dask Dashboard: http://127.0.0.1:37319/status\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "
\n", + "
\n", + "

Client

\n", + "

Client-d925221f-9c1b-11ef-af09-00001029fe80

\n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + "
Connection method: Cluster objectCluster type: distributed.LocalCluster
\n", + " Dashboard: http://127.0.0.1:37319/status\n", + "
\n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "

Cluster Info

\n", + "
\n", + "
\n", + "
\n", + "
\n", + "

LocalCluster

\n", + "

c8311a55

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + "\n", + " \n", + "
\n", + " Dashboard: http://127.0.0.1:37319/status\n", + " \n", + " Workers: 1\n", + "
\n", + " Total threads: 1\n", + " \n", + " Total memory: 0.98 TiB\n", + "
Status: runningUsing processes: True
\n", + "\n", + "
\n", + " \n", + "

Scheduler Info

\n", + " \n", + "\n", + "
\n", + "
\n", + "
\n", + "
\n", + "

Scheduler

\n", + "

Scheduler-94223597-08d7-4d25-b3dc-e1eb43105923

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " Comm: tcp://127.0.0.1:41157\n", + " \n", + " Workers: 1\n", + "
\n", + " Dashboard: http://127.0.0.1:37319/status\n", + " \n", + " Total threads: 1\n", + "
\n", + " Started: Just now\n", + " \n", + " Total memory: 0.98 TiB\n", + "
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "

Workers

\n", + " \n", + "\n", + " \n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "

Worker: 0

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + "\n", + " \n", + "\n", + "
\n", + " Comm: tcp://127.0.0.1:44851\n", + " \n", + " Total threads: 1\n", + "
\n", + " Dashboard: http://127.0.0.1:38405/status\n", + " \n", + " Memory: 0.98 TiB\n", + "
\n", + " Nanny: tcp://127.0.0.1:46553\n", + "
\n", + " Local directory: /tmp/slurm-25873630/dask-worker-space/worker-uw4h_qgs\n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "\n", + "
\n", + "
\n", + "\n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "\n", + "
\n", + "
" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from pathlib import Path\n", + "\n", + "from typing import List, Tuple, Union, Dict\n", + "import dask.array as da\n", + "import numpy as np\n", + "import rasterio\n", + "import rioxarray as rxr\n", + "from dask import delayed\n", + "from dask.distributed import Client, LocalCluster\n", + "from rasterio.transform import Affine\n", + "\n", + "from utils import create_map_with_rasters\n", + "\n", + "cluster = LocalCluster()\n", + "client = Client(cluster)\n", + "\n", + "print(\"Dask Dashboard: \", client.dashboard_link)\n", + "client" + ] + }, + { + "cell_type": "markdown", + "id": "2a86c2ad-975b-4ed5-a8b0-8add6c8714d4", + "metadata": {}, + "source": [ + "Dask return an url where the dashboard is availaible (usually http://127.0.0.1:8787/status). This is not a tutorial on how to use this dashboard, but we recommend using it in a separate window while using this notebook.\n", + "\n", + "### Open raster thanks to rioxarray\n", + "\n", + "Here we are going to open the raster data required for this tutorial, the RGB bands from a Sentinel-2 acquisition. To do this, we're going to use rioxarray and, more specifically, the [open_rasterio](https://corteva.github.io/rioxarray/html/rioxarray.html#rioxarray-open-rasterio) method, which opens the images lazily (without loading data into memory) and returns a `dask.array` object. \n", + "From this method we will use the ``chunks`` and ``lock`` arguments, which respectively set a chunk size and limit access to the data to one thread at a time to avoid read problems. Here ``chunks`` is set to ``True`` to allow dask to automatically size chunks.\n" + ] + }, + { + "cell_type": "markdown", + "id": "0e86755c-f505-4d6b-830e-396485e12054", + "metadata": {}, + "source": [ + "## Calculation of the Average NDVI with dask\n", + "\n", + "In this example, we will use what we have learned to: \n", + "\n", + "1. Read the data from the disk and stack them.\n", + "2. Calculate the associated NDVI, which combines multi-band information into a single band.\n", + "3. Reduce the information by calculating the average NDVI within a window.\n", + "4. Write the resulting image to the disk.\n", + "\n", + "First, let's read the data we need to perform the NDVI." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "5e0c156e-01ef-4c1d-b300-e2c639fd87a9", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "def open_raster_and_get_metadata(raster_paths: List[str], chunks: Union[int, Tuple, Dict, None]):\n", + " \"\"\"\n", + " Opens multiple raster files, extracts shared geospatial metadata, \n", + " and returns the concatenated data along with resolution and CRS info.\n", + "\n", + " Parameters:\n", + " -----------\n", + " raster_paths : List[str]\n", + " Paths to the raster files.\n", + " chunks : Union[int, Tuple, Dict, bool, None]\n", + " Chunk sizes for Dask (bands, height, width).\n", + "\n", + " Returns:\n", + " --------\n", + " Tuple[dask.array.Array, float, float, float, float, Union[str, CRS]]\n", + " Concatenated raster data, x and y resolution, top-left coordinates, and CRS.\n", + " \"\"\"\n", + " results = []\n", + " for raster_path in raster_paths:\n", + " with rxr.open_rasterio(raster_path, chunks=chunks, lock=True) as tif:\n", + " reprojection = tif\n", + " transform = reprojection.rio.transform()\n", + " crs = reprojection.rio.crs\n", + " x_res = transform[0]\n", + " y_res = -transform[4]\n", + " top_left_x = transform[2]\n", + " top_left_y = transform[5]\n", + " results.append(reprojection)\n", + "\n", + " return da.concatenate(results), x_res, y_res, top_left_x, top_left_y, crs\n", + "\n", + "# Paths to RGB Sentinel-2 bands\n", + "sentinel_2_dir = \"/work/scratch/data/romaint/\" # change it for your own directory\n", + "\n", + "s2_1 = f\"{sentinel_2_dir}/SENTINEL2A_20210415-105852-555_L2A_T31TCJ_C_V3-0_FRE_STACK.tif\"\n", + "s2_2 = f\"{sentinel_2_dir}/SENTINEL2B_20210407-104900-035_L2A_T31TCJ_C_V3-0/SENTINEL2B_20210407-104900-035_L2A_T31TCJ_C_V3-0_FRE_STACK.tif\"\n", + "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", + "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", + "reading_chunks = True\n", + "#(-1,2200,2200)\n", + "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks)" + ] + }, + { + "cell_type": "markdown", + "id": "fc56a7e0-edb5-4635-b5a2-0b274d08ce38", + "metadata": {}, + "source": [ + "When the data is read, we can express the NDVI calculation as if it were a numpy array. We add ``[None, :, :]`` to keep the shape as ``(bands, rows, cols)``. Then we can apply reduction on the dask.array and use ``compute()`` on it to triger the computation." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Array Chunk
Bytes 459.90 MiB 127.98 MiB
Shape (2, 10980, 10980) (1, 6111, 10980)
Dask graph 4 chunks in 5 graph layers
Data type int16 numpy.ndarray
\n", + " 		\n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " 10980\n", + " 10980\n", + " 2\n", + "\n", + "
" + ], + "text/plain": [ + "dask.array" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "input_data_array" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "2dad7fe0-23bc-40b2-b27e-4c7cb0f8dc0e", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(2, 10980, 10980)\n" + ] + } + ], + "source": [ + "print(input_data_array.shape)\n", + "ndvi_array = (input_data_array[1] - input_data_array[0]) / (input_data_array[1] + input_data_array[0])[None, :, :]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "19fc2fce-4aca-4b7a-abd6-6baba3b5c464", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 165 ms, sys: 455 ms, total: 619 ms\n", + "Wall time: 2.06 s\n" + ] + } + ], + "source": [ + "%%time\n", + "\n", + "mean_ndvi = ndvi_array.compute()" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "c4f6ae3e-4511-448f-97d8-ff85172dea6c", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "def create_raster(data: np.ndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n", + " transform = Affine.translation(top_left_x, top_left_y) * Affine.scale(x_res, -y_res)\n", + " with rasterio.open(\n", + " output_file, \"w\",\n", + " driver=\"GTiff\",\n", + " height=data.shape[1],\n", + " width=data.shape[2],\n", + " count=data.shape[0],\n", + " dtype=data.dtype,\n", + " crs=crs,\n", + " transform=transform\n", + " ) as dst:\n", + " dst.write(data)\n", + "\n", + "crs=\"EPSG:4326\"\n", + "output_file = Path(\"ndvi_dask.tif\")\n", + "quicklook_img = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_QKL_ALL.jpg\"\n", + "create_raster(ndvi_array, output_file, x_res , y_res,\n", + " top_left_x, top_left_y, crs)" + ] + }, + { + "cell_type": "markdown", + "id": "7f5d09f0-cf39-4e3b-afcb-6db0bccc877a", + "metadata": {}, + "source": [ + "# Calculate NDVI With OTB in python" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning 1: Invalid value for NUM_THREADS: \n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Writing /work/scratch/data/romaint/img_ndvi_otb.tif...: 100% [**************************************************] (7s)\n" + ] + }, + { + "data": { + "text/plain": [ + "0" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import otbApplication as otb\n", + "\n", + "sentinel_2_dir = \"/work/scratch/data/romaint\"\n", + "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", + "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", + "out_ndvi_otb_py=\"/work/scratch/data/romaint/img_ndvi_otb.tif\"\n", + "#Compute NDVI with OTB in python\n", + "app_ndvi_otb = otb.Registry.CreateApplication(\"BandMath\")\n", + "app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n", + "app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n", + "app_ndvi_otb.SetParameterString(\"out\",out_ndvi_otb_py)\n", + "app_ndvi_otb.ExecuteAndWriteOutput()\n" + ] + }, + { + "cell_type": "markdown", + "id": "c030e442-871e-4d59-8cb6-1310b5867df1", + "metadata": {}, + "source": [ + "# Calculate NDVI with OTB in C++\n", + "This part will call BandMath with the otb CLI to compare performances with the python swig interface" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "73a731cf-a534-4b49-9a6d-425b58d87e45", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning 1: Invalid value for NUM_THREADS: \n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Writing /work/scratch/data/romaint/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (5s)\n" + ] + } + ], + "source": [ + "%%bash\n", + "otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\" " + ] + }, + { + "cell_type": "markdown", + "id": "7966cdad-a3d7-467d-bd1a-d207bc85bde0", + "metadata": {}, + "source": [ + "# Compute SuperImpose with OTB\n", + "\n", + "Superimpose does a resampling then a crop to have a new raster that has the same resolution as the reference input image. This app is using multithreading a lot, it is interesting to compare it with a solution like dask and full python / rasterio" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "71d9a855-86f3-48aa-a9eb-32a1dc853649", + "metadata": {}, + "outputs": [], + "source": [ + "import otbApplication as otb\n", + "\n", + "product_dir = \"/work/scratch/data/romaint\"\n", + "appSI = otb.Registry.CreateApplication(\"Superimpose\")\n", + "\n", + "appSI.SetParameterString(\"inr\", \"QB_Toulouse_Ortho_PAN.tif\")\n", + "appSI.SetParameterString(\"inm\", \"QB_Toulouse_Ortho_XS.tif\")\n", + "appSI.SetParameterString(\"out\", \"SuperimposedXS_to_PAN.tif\")\n", + "\n", + "appSI.ExecuteAndWriteOutput()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From fd1ac172963f740dc4e7ed8b4b47afdb22124b46 Mon Sep 17 00:00:00 2001 From: Thibaut ROMAIN Date: Fri, 8 Nov 2024 17:19:10 +0100 Subject: [PATCH 02/37] Add superimpose and numba --- tuto_greenit.ipynb | 582 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 529 insertions(+), 53 deletions(-) diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb index dbe6618..1d534f9 100644 --- a/tuto_greenit.ipynb +++ b/tuto_greenit.ipynb @@ -35,7 +35,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 53, "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21", "metadata": { "tags": [] @@ -45,9 +45,9 @@ "name": "stderr", "output_type": "stream", "text": [ - "/usr/local/lib/python3.10/dist-packages/distributed/node.py:182: UserWarning: Port 8787 is already in use.\n", + "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n", "Perhaps you already have a cluster running?\n", - "Hosting the HTTP server on port 37319 instead\n", + "Hosting the HTTP server on port 37043 instead\n", " warnings.warn(\n" ] }, @@ -55,7 +55,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Dask Dashboard: http://127.0.0.1:37319/status\n" + "Dask Dashboard: http://127.0.0.1:37043/status\n" ] }, { @@ -65,7 +65,7 @@ "
\n", "
\n", "

Client

\n", - "

Client-d925221f-9c1b-11ef-af09-00001029fe80

\n", + "

Client-9723ed70-9de5-11ef-863d-589671ac823c

\n", " \n", "\n", " \n", @@ -78,7 +78,7 @@ " \n", " \n", " \n", " \n", " \n", @@ -96,22 +96,22 @@ " \n", "
\n", "

LocalCluster

\n", - "

c8311a55

\n", + "

a20598c6

\n", "
\n", - " Dashboard: http://127.0.0.1:37319/status\n", + " Dashboard: http://127.0.0.1:37043/status\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", @@ -133,22 +133,22 @@ "
\n", "
\n", "

Scheduler

\n", - "

Scheduler-94223597-08d7-4d25-b3dc-e1eb43105923

\n", + "

Scheduler-3665a499-b764-4923-9552-72773bafd7f6

\n", "
\n", - " Dashboard: http://127.0.0.1:37319/status\n", + " Dashboard: http://127.0.0.1:37043/status\n", " \n", - " Workers: 1\n", + " Workers: 4\n", "
\n", - " Total threads: 1\n", + " Total threads: 16\n", " \n", - " Total memory: 0.98 TiB\n", + " Total memory: 30.63 GiB\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", @@ -156,7 +156,7 @@ " Started: Just now\n", " \n", " \n", " \n", "
\n", - " Comm: tcp://127.0.0.1:41157\n", + " Comm: tcp://127.0.0.1:41837\n", " \n", - " Workers: 1\n", + " Workers: 4\n", "
\n", - " Dashboard: http://127.0.0.1:37319/status\n", + " Dashboard: http://127.0.0.1:37043/status\n", " \n", - " Total threads: 1\n", + " Total threads: 16\n", "
\n", - " Total memory: 0.98 TiB\n", + " Total memory: 30.63 GiB\n", "
\n", @@ -179,29 +179,164 @@ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", + " \n", + "\n", + " \n", + "\n", + " \n", + "\n", + "
\n", - " Comm: tcp://127.0.0.1:44851\n", + " Comm: tcp://127.0.0.1:39503\n", " \n", - " Total threads: 1\n", + " Total threads: 4\n", "
\n", - " Dashboard: http://127.0.0.1:38405/status\n", + " Dashboard: http://127.0.0.1:40587/status\n", " \n", - " Memory: 0.98 TiB\n", + " Memory: 7.66 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:46553\n", + " Nanny: tcp://127.0.0.1:42947\n", "
\n", - " Local directory: /tmp/slurm-25873630/dask-worker-space/worker-uw4h_qgs\n", + " Local directory: /tmp/dask-scratch-space/worker-3of2mpm6\n", + "
\n", + " \n", + "
\n", + " \n", + " \n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "

Worker: 1

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + "\n", + " \n", + "\n", + "
\n", + " Comm: tcp://127.0.0.1:33637\n", + " \n", + " Total threads: 4\n", + "
\n", + " Dashboard: http://127.0.0.1:46855/status\n", + " \n", + " Memory: 7.66 GiB\n", + "
\n", + " Nanny: tcp://127.0.0.1:34317\n", + "
\n", + " Local directory: /tmp/dask-scratch-space/worker-5wbyfhxh\n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "

Worker: 2

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + "\n", + " \n", + "\n", + "
\n", + " Comm: tcp://127.0.0.1:40539\n", + " \n", + " Total threads: 4\n", + "
\n", + " Dashboard: http://127.0.0.1:45731/status\n", + " \n", + " Memory: 7.66 GiB\n", + "
\n", + " Nanny: tcp://127.0.0.1:37707\n", + "
\n", + " Local directory: /tmp/dask-scratch-space/worker-_e6spiif\n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "
\n", + "
\n", + "
\n", + "
\n", + " \n", + "

Worker: 3

\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", " \n", "\n", @@ -228,10 +363,10 @@ "" ], "text/plain": [ - "" + "" ] }, - "execution_count": 2, + "execution_count": 53, "metadata": {}, "output_type": "execute_result" } @@ -289,8 +424,8 @@ }, { "cell_type": "code", - "execution_count": 3, - "id": "5e0c156e-01ef-4c1d-b300-e2c639fd87a9", + "execution_count": 74, + "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62", "metadata": { "tags": [] }, @@ -328,14 +463,11 @@ " return da.concatenate(results), x_res, y_res, top_left_x, top_left_y, crs\n", "\n", "# Paths to RGB Sentinel-2 bands\n", - "sentinel_2_dir = \"/work/scratch/data/romaint/\" # change it for your own directory\n", - "\n", - "s2_1 = f\"{sentinel_2_dir}/SENTINEL2A_20210415-105852-555_L2A_T31TCJ_C_V3-0_FRE_STACK.tif\"\n", - "s2_2 = f\"{sentinel_2_dir}/SENTINEL2B_20210407-104900-035_L2A_T31TCJ_C_V3-0/SENTINEL2B_20210407-104900-035_L2A_T31TCJ_C_V3-0_FRE_STACK.tif\"\n", + "sentinel_2_dir = \"/home/tromain/Data/S2\"\n", "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", + "#reading_chunks = (-1,2200,2200)\n", "reading_chunks = True\n", - "#(-1,2200,2200)\n", "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks)" ] }, @@ -349,7 +481,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 75, "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f", "metadata": { "tags": [] @@ -446,7 +578,7 @@ "dask.array" ] }, - "execution_count": 4, + "execution_count": 75, "metadata": {}, "output_type": "execute_result" } @@ -457,7 +589,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 76, "id": "2dad7fe0-23bc-40b2-b27e-4c7cb0f8dc0e", "metadata": { "tags": [] @@ -478,7 +610,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 77, "id": "19fc2fce-4aca-4b7a-abd6-6baba3b5c464", "metadata": { "tags": [] @@ -488,8 +620,8 @@ "name": "stdout", "output_type": "stream", "text": [ - "CPU times: user 165 ms, sys: 455 ms, total: 619 ms\n", - "Wall time: 2.06 s\n" + "CPU times: user 469 ms, sys: 1.01 s, total: 1.48 s\n", + "Wall time: 2.1 s\n" ] } ], @@ -501,7 +633,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 79, "id": "c4f6ae3e-4511-448f-97d8-ff85172dea6c", "metadata": { "tags": [] @@ -523,8 +655,7 @@ " dst.write(data)\n", "\n", "crs=\"EPSG:4326\"\n", - "output_file = Path(\"ndvi_dask.tif\")\n", - "quicklook_img = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_QKL_ALL.jpg\"\n", + "output_file = Path(\"/home/tromain/Data/S2/ndvi_dask.tif\")\n", "create_raster(ndvi_array, output_file, x_res , y_res,\n", " top_left_x, top_left_y, crs)" ] @@ -539,7 +670,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 8, "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7", "metadata": { "tags": [] @@ -556,7 +687,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Writing /work/scratch/data/romaint/img_ndvi_otb.tif...: 100% [**************************************************] (7s)\n" + "Writing /work/scratch/data/romaint/img_ndvi_otb.tif...: 100% [**************************************************] (3s)\n" ] }, { @@ -565,7 +696,7 @@ "0" ] }, - "execution_count": 16, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -596,7 +727,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 9, "id": "73a731cf-a534-4b49-9a6d-425b58d87e45", "metadata": { "tags": [] @@ -613,7 +744,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Writing /work/scratch/data/romaint/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (5s)\n" + "Writing /work/scratch/data/romaint/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (2s)\n" ] } ], @@ -634,22 +765,367 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 10, "id": "71d9a855-86f3-48aa-a9eb-32a1dc853649", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2024-11-07 09:04:44 (WARNING) Superimpose: Forcing PHR mode with PHR data. You need to add \"-mode default\" to force the default mode with PHR images.\n", + "2024-11-07 09:04:44 (WARNING) Superimpose: Forcing PHR mode with PHR data. You need to add \"-mode default\" to force the default mode with PHR images.\n", + "2024-11-07 09:04:44 (WARNING) Superimpose: Forcing PHR mode with PHR data. You need to add \"-mode default\" to force the default mode with PHR images.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning 1: Invalid value for NUM_THREADS: \n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Writing /work/scratch/data/romaint/SuperimposedXS_to_PAN.tif?&writerpctags=true...: 100% [**************************************************] (2m 59s)\n" + ] + }, + { + "data": { + "text/plain": [ + "0" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "import otbApplication as otb\n", "\n", - "product_dir = \"/work/scratch/data/romaint\"\n", + "#product_dir = \"/work/scratch/data/romaint\"\n", + "pan_raster_url = \"/work/scratch/data/tanguyy/public/PHR_OTB/IMG_PHR1B_P_001/IMG_PHR1B_P_202302281104151_SEN_6967639101-1_R1C1.JP2\"\n", + "xs_raster_url = \"/work/scratch/data/tanguyy/public/PHR_OTB/IMG_PHR1B_MS_004/IMG_PHR1B_MS_202302281104151_SEN_6967639101-2_R1C1.JP2\"\n", "appSI = otb.Registry.CreateApplication(\"Superimpose\")\n", "\n", - "appSI.SetParameterString(\"inr\", \"QB_Toulouse_Ortho_PAN.tif\")\n", - "appSI.SetParameterString(\"inm\", \"QB_Toulouse_Ortho_XS.tif\")\n", - "appSI.SetParameterString(\"out\", \"SuperimposedXS_to_PAN.tif\")\n", + "appSI.SetParameterString(\"inr\", pan_raster_url)\n", + "appSI.SetParameterString(\"inm\", xs_raster_url)\n", + "appSI.SetParameterString(\"out\", \"/work/scratch/data/romaint/SuperimposedXS_to_PAN.tif\")\n", "\n", "appSI.ExecuteAndWriteOutput()" ] + }, + { + "cell_type": "markdown", + "id": "49b7321e-1cfb-4994-83fc-93312252c84f", + "metadata": {}, + "source": [ + "# Performance improvement with xarray.coarsen method ?" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "e7568371-4a69-426e-9247-c7e4abe7aef2", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " Size: 3GB\n", + "dask.array, shape=(1, 23796, 31280), dtype=float32, chunksize=(1, 1072, 31280), chunktype=numpy.ndarray>\n", + "Coordinates:\n", + " * band (band) int64 8B 1\n", + " * x (x) float64 250kB 0.5 1.5 2.5 ... 3.128e+04 3.128e+04 3.128e+04\n", + " * y (y) float64 190kB 0.5 1.5 2.5 ... 2.379e+04 2.379e+04 2.38e+04\n", + " spatial_ref int64 8B 0\n", + "Attributes: (12/33)\n", + " METADATATYPE: OTB\n", + " OTB_VERSION: 9.1.0\n", + " TimeRangeStart: 2023-02-28T11:04:15.2040000Z\n", + " ProductionDate: 2024-04-15T13:25:33.956Z\n", + " AcquisitionDate: 2023-02-28T11:04:15.1Z\n", + " SatAzimuth: 202.389\n", + " ... ...\n", + " PhysicalBias: 0\n", + " PhysicalGain: 11.71\n", + " EnhancedBandName: PAN\n", + " BandName: P\n", + " scale_factor: 1.0\n", + " add_offset: 0.0\n", + " Size: 186MB\n", + "dask.array\n", + "Coordinates:\n", + " * band (band) int64 8B 1\n", + " * x (x) float64 63kB 2.0 6.0 10.0 ... 3.127e+04 3.127e+04 3.128e+04\n", + " * y (y) float64 48kB 2.0 6.0 10.0 ... 2.379e+04 2.379e+04 2.379e+04\n", + " spatial_ref int64 8B 0\n", + "Attributes: (12/33)\n", + " METADATATYPE: OTB\n", + " OTB_VERSION: 9.1.0\n", + " TimeRangeStart: 2023-02-28T11:04:15.2040000Z\n", + " ProductionDate: 2024-04-15T13:25:33.956Z\n", + " AcquisitionDate: 2023-02-28T11:04:15.1Z\n", + " SatAzimuth: 202.389\n", + " ... ...\n", + " PhysicalBias: 0\n", + " PhysicalGain: 11.71\n", + " EnhancedBandName: PAN\n", + " BandName: P\n", + " scale_factor: 1.0\n", + " add_offset: 0.0\n", + "True\n", + "CPU times: user 1.87 s, sys: 633 ms, total: 2.5 s\n", + "Wall time: 4.95 s\n" + ] + } + ], + "source": [ + "%%time\n", + "\n", + "import xarray\n", + "from rioxarray.merge import merge_arrays\n", + "\n", + "pan_raster_url = \"/home/tromain/Data/phr_pan.tif\"\n", + "xs_raster_url = \"/home/tromain/Data/phr_xs.tif\"\n", + "origin_raster = rxr.open_rasterio(pan_raster_url,chunks=True)\n", + "#.squeeze('band', drop=True)\n", + "print(origin_raster)\n", + "#origin_raster = origin_raster.load()\n", + "pxs_raster = origin_raster.coarsen(x=4, y=4, boundary='pad').mean()\n", + "xs_raster = rxr.open_rasterio(xs_raster_url,chunks=True)\n", + "print(pxs_raster)\n", + "#print(\"X == X? {} Y==Y? {}\".format((origin_raster.x == xs_raster.x),(origin_raster.y == xs_raster.y)))\n", + "print(all(list(origin_raster.x == xs_raster.x)))\n", + "#ppxs_raster = xarray.concat([pxs_raster,xs_raster],dim=\"band\",coords=\"all\")\n", + "#ppxs_raster = merge_arrays([pxs_raster,xs_raster])\n", + "crs=\"EPSG:4326\"\n", + "output_file = Path(\"/home/tromain/Data/superimpose_coarsen.tif\")\n", + "pxs_raster.rio.to_raster(output_file)" + ] + }, + { + "cell_type": "markdown", + "id": "b2571ad8-9af5-43c2-8c5a-4c8a73a6c885", + "metadata": {}, + "source": [ + "# SuperImpose using rasterio reproject_match" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4407913c-a383-451a-9967-8a7908e9664c", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:365: UserWarning: The specified chunks separate the stored chunks along dimension \"y\" starting at index 1. This could degrade performance. Instead, consider rechunking after loading.\n", + " var_chunks = _get_chunk(var, chunks, chunkmanager)\n", + "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:365: UserWarning: The specified chunks separate the stored chunks along dimension \"x\" starting at index 1. This could degrade performance. Instead, consider rechunking after loading.\n", + " var_chunks = _get_chunk(var, chunks, chunkmanager)\n", + "ERROR 1: PROJ: internal_proj_create_from_database: /home/tromain/Apps/miniconda3/envs/env_greenit/otb9/share/proj/proj.db contains DATABASE.LAYOUT.VERSION.MINOR = 2 whereas a number >= 3 is expected. It comes from another PROJ installation.\n", + "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:365: UserWarning: The specified chunks separate the stored chunks along dimension \"y\" starting at index 1. This could degrade performance. Instead, consider rechunking after loading.\n", + " var_chunks = _get_chunk(var, chunks, chunkmanager)\n", + "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:365: UserWarning: The specified chunks separate the stored chunks along dimension \"x\" starting at index 1. This could degrade performance. Instead, consider rechunking after loading.\n", + " var_chunks = _get_chunk(var, chunks, chunkmanager)\n", + "ERROR 1: PROJ: internal_proj_create_from_database: /home/tromain/Apps/miniconda3/envs/env_greenit/otb9/share/proj/proj.db contains DATABASE.LAYOUT.VERSION.MINOR = 2 whereas a number >= 3 is expected. It comes from another PROJ installation.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(1, 23796, 31283)\n", + "float32\n", + "(4, 5949, 7822)\n" + ] + } + ], + "source": [ + "%%time\n", + "\n", + "import xarray\n", + "from pathlib import Path\n", + "\n", + "pan_raster_url = \"/home/tromain/Data/PHR_OTB/IMG_PHR1B_P_001/IMG_PHR1B_P_202302281104151_SEN_6967639101-1_R1C1.JP2\"\n", + "xs_raster_url = \"/home/tromain/Data/PHR_OTB/IMG_PHR1B_MS_004/IMG_PHR1B_MS_202302281104151_SEN_6967639101-2_R1C1.JP2\"\n", + "\n", + "xds = xarray.open_dataarray(pan_raster_url,chunks=True)\n", + "xds.rio.write_crs(\"epsg:4326\", inplace=True)\n", + "xds.rio.write_nodata(0, inplace=True)\n", + "xds_match = xarray.open_dataarray(xs_raster_url,chunks=True)\n", + "print(xds.shape)\n", + "print(xds.dtype)\n", + "print(xds_match.shape)\n", + "xds_match.rio.write_crs(\"epsg:4326\", inplace=True)\n", + "xds_match.rio.write_nodata(0, inplace=True)\n", + "xds_repr_match = xds.rio.reproject_match(xds_match)\n", + "crs=\"EPSG:4326\"\n", + "output_file = Path(\"/home/tromain/Data/superimpose_reproject_match.tif\")\n", + "xds_repr_match.rio.to_raster(output_file)" + ] + }, + { + "cell_type": "markdown", + "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386", + "metadata": {}, + "source": [ + "# Using numba for NDVI ?\n", + "Here we will try to use numba for ndvi computation" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Elapsed with Raster IO = 1.1344427589974657s\n", + "Elapsed with RIOXarray + dask = 0.027490641001350014s\n", + "Elapsed with list comprehension = 0.0035700430016731843s\n", + "Elapsed with normal compute = 0.0015671629989810754s\n" + ] + }, + { + "ename": "AttributeError", + "evalue": "'list' object has no attribute 'shape'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)", + "File \u001b[0;32m:61\u001b[0m\n", + "Cell \u001b[0;32mIn[79], line 6\u001b[0m, in \u001b[0;36mcreate_raster\u001b[0;34m(data, output_file, x_res, y_res, top_left_x, top_left_y, crs)\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcreate_raster\u001b[39m(data: np\u001b[38;5;241m.\u001b[39mndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n\u001b[1;32m 2\u001b[0m transform \u001b[38;5;241m=\u001b[39m Affine\u001b[38;5;241m.\u001b[39mtranslation(top_left_x, top_left_y) \u001b[38;5;241m*\u001b[39m Affine\u001b[38;5;241m.\u001b[39mscale(x_res, \u001b[38;5;241m-\u001b[39my_res)\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m rasterio\u001b[38;5;241m.\u001b[39mopen(\n\u001b[1;32m 4\u001b[0m output_file, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mw\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m 5\u001b[0m driver\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGTiff\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m----> 6\u001b[0m height\u001b[38;5;241m=\u001b[39m\u001b[43mdata\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mshape\u001b[49m[\u001b[38;5;241m1\u001b[39m],\n\u001b[1;32m 7\u001b[0m width\u001b[38;5;241m=\u001b[39mdata\u001b[38;5;241m.\u001b[39mshape[\u001b[38;5;241m2\u001b[39m],\n\u001b[1;32m 8\u001b[0m count\u001b[38;5;241m=\u001b[39mdata\u001b[38;5;241m.\u001b[39mshape[\u001b[38;5;241m0\u001b[39m],\n\u001b[1;32m 9\u001b[0m dtype\u001b[38;5;241m=\u001b[39mdata\u001b[38;5;241m.\u001b[39mdtype,\n\u001b[1;32m 10\u001b[0m crs\u001b[38;5;241m=\u001b[39mcrs,\n\u001b[1;32m 11\u001b[0m transform\u001b[38;5;241m=\u001b[39mtransform\n\u001b[1;32m 12\u001b[0m ) \u001b[38;5;28;01mas\u001b[39;00m dst:\n\u001b[1;32m 13\u001b[0m dst\u001b[38;5;241m.\u001b[39mwrite(data)\n", + "\u001b[0;31mAttributeError\u001b[0m: 'list' object has no attribute 'shape'" + ] + } + ], + "source": [ + "%%time\n", + "\n", + "#from numba import jit,njit\n", + "from xarray import DataArray \n", + "from typing import List, Tuple, Union, Dict\n", + "import rioxarray as rxr\n", + "import numpy as np\n", + "import time\n", + "import rasterio\n", + "\n", + "@njit\n", + "def one_pixel_ndvi(p1,p2):\n", + " return (p2-p1) / (p2+p1) \n", + "\n", + "@njit\n", + "def compute_ndvi(input_data_1: np.ndarray,input_data_2: np.ndarray):\n", + " ndvi_array = [[one_pixel_ndvi(i,j) for i in input_data_1] for j in input_data_2]\n", + " return ndvi_array\n", + "\n", + "def compute_ndvi_dask(input_data_1: DataArray,input_data_2: DataArray):\n", + " ndvi_array = [[(j-i)/(i+j) for i in input_data_1] for j in input_data_2]\n", + " return ndvi_array\n", + "\n", + "sentinel_2_dir = \"/home/tromain/Data/S2\"\n", + "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", + "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", + "\n", + "with rasterio.open(s2_b4, 'r') as ds:\n", + " input_data_b4 = ds.read() \n", + "\n", + "with rasterio.open(s2_b8, 'r') as ds:\n", + " input_data_b8 = ds.read()\n", + "\n", + "start = time.perf_counter()\n", + "ndvi_computed = compute_ndvi(input_data_b4,input_data_b8)\n", + "end = time.perf_counter()\n", + "print(\"Elapsed with Raster IO = {}s\".format((end - start)))\n", + "# Example with xarray\n", + "#input_data_b4 = xarray.open_dataarray(s2_b4,chunks=False)\n", + "#input_data_b8 = xarray.open_dataarray(s2_b8,chunks=False)\n", + "#ndvi_computed = compute_ndvi(input_data_b4.to_numpy(),input_data_b8.to_numpy())\n", + "\n", + "# Example with rioxarray \n", + "input_data_b4 = rxr.open_rasterio(s2_b4,chunks=True)\n", + "input_data_b8 = rxr.open_rasterio(s2_b8,chunks=True)\n", + "start = time.perf_counter()\n", + "ndvi_computed = compute_ndvi_dask(input_data_b4,input_data_b8)\n", + "end = time.perf_counter()\n", + "\n", + "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n", + "\n", + "start = time.perf_counter()\n", + "ndvi_array = [[(j-i)/(i+j) for i in input_data_b4] for j in input_data_b8]\n", + "end = time.perf_counter()\n", + "print(\"Elapsed with list comprehension = {}s\".format((end - start)))\n", + "start = time.perf_counter()\n", + "ndvi_array = (input_data_array[1] - input_data_array[0]) / (input_data_array[1] + input_data_array[0])[None, :, :]\n", + "end = time.perf_counter()\n", + "print(\"Elapsed with normal compute = {}s\".format((end - start)))\n", + "\n", + "crs=\"EPSG:4326\"\n", + "output_file = Path(\"/home/tromain/Data/S2/ndvi_dask_optimized.tif\")\n", + "create_raster(ndvi_computed, output_file, x_res , y_res,\n", + " top_left_x, top_left_y, crs)" + ] + }, + { + "cell_type": "markdown", + "id": "6ffcc752-1d32-42f0-86f6-de25ce216688", + "metadata": {}, + "source": [ + "# Performances multiprocessing vs multithreading\n", + "\n", + "Depending on the processing applied to an image it is interesting to use the multiprocessing approach instead of the multithreading which is not really a multithreading in python as every thread is executed one after the other. (in Python 3.13, the GIL has been reworked to improve the situation)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5bd9b9de-33df-48d7-b643-9e767bbc9d2a", + "metadata": {}, + "outputs": [], + "source": [ + "import multiprocessing\n" + ] + }, + { + "cell_type": "markdown", + "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43", + "metadata": {}, + "source": [ + "# Optimizing the size of your data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "69b8bd27-926e-4373-86d1-4dafe7fdbeed", + "metadata": {}, + "outputs": [], + "source": [ + "using COG ?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4cbef49-ee8d-4948-b5c9-1a958edcc8d4", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { @@ -668,7 +1144,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.12" + "version": "3.11.10" } }, "nbformat": 4, From 5d8af401ce1945e9fe28a6c01ec0350191343ce4 Mon Sep 17 00:00:00 2001 From: Thibaut ROMAIN Date: Tue, 12 Nov 2024 17:22:04 +0100 Subject: [PATCH 03/37] Add example without numba and adapt timers to take account of read and write --- tuto_greenit.ipynb | 284 ++++++++++++++++++++++++++++----------------- 1 file changed, 178 insertions(+), 106 deletions(-) diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb index 1d534f9..7294b8b 100644 --- a/tuto_greenit.ipynb +++ b/tuto_greenit.ipynb @@ -35,27 +35,17 @@ }, { "cell_type": "code", - "execution_count": 53, + "execution_count": 2, "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21", "metadata": { "tags": [] }, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n", - "Perhaps you already have a cluster running?\n", - "Hosting the HTTP server on port 37043 instead\n", - " warnings.warn(\n" - ] - }, { "name": "stdout", "output_type": "stream", "text": [ - "Dask Dashboard: http://127.0.0.1:37043/status\n" + "Dask Dashboard: http://127.0.0.1:8787/status\n" ] }, { @@ -65,7 +55,7 @@ "
\n", "
\n", "

Client

\n", - "

Client-9723ed70-9de5-11ef-863d-589671ac823c

\n", + "

Client-52c9be04-a10b-11ef-b7e1-589671ac823c

\n", "
\n", + " Comm: tcp://127.0.0.1:32929\n", + " \n", + " Total threads: 4\n", + "
\n", + " Dashboard: http://127.0.0.1:38331/status\n", + " \n", + " Memory: 7.66 GiB\n", + "
\n", + " Nanny: tcp://127.0.0.1:46609\n", + "
\n", + " Local directory: /tmp/dask-scratch-space/worker-u77lvci7\n", "
\n", "\n", " \n", @@ -78,7 +68,7 @@ " \n", " \n", " \n", " \n", " \n", @@ -96,11 +86,11 @@ " \n", "
\n", "

LocalCluster

\n", - "

a20598c6

\n", + "

e120fc0c

\n", "
\n", - " Dashboard: http://127.0.0.1:37043/status\n", + " Dashboard: http://127.0.0.1:8787/status\n", "
\n", " \n", " \n", "
\n", - " Dashboard: http://127.0.0.1:37043/status\n", + " Dashboard: http://127.0.0.1:8787/status\n", " \n", " Workers: 4\n", @@ -133,11 +123,11 @@ "
\n", "
\n", "

Scheduler

\n", - "

Scheduler-3665a499-b764-4923-9552-72773bafd7f6

\n", + "

Scheduler-bbff2036-0b0c-46eb-b353-6cab67208131

\n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", - " Comm: tcp://127.0.0.1:41837\n", + " Comm: tcp://127.0.0.1:38879\n", " \n", " Workers: 4\n", @@ -145,7 +135,7 @@ "
\n", - " Dashboard: http://127.0.0.1:37043/status\n", + " Dashboard: http://127.0.0.1:8787/status\n", " \n", " Total threads: 16\n", @@ -179,7 +169,7 @@ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -224,7 +214,7 @@ "
\n", - " Comm: tcp://127.0.0.1:39503\n", + " Comm: tcp://127.0.0.1:37533\n", " \n", " Total threads: 4\n", @@ -187,7 +177,7 @@ "
\n", - " Dashboard: http://127.0.0.1:40587/status\n", + " Dashboard: http://127.0.0.1:42723/status\n", " \n", " Memory: 7.66 GiB\n", @@ -195,13 +185,13 @@ "
\n", - " Nanny: tcp://127.0.0.1:42947\n", + " Nanny: tcp://127.0.0.1:34589\n", "
\n", - " Local directory: /tmp/dask-scratch-space/worker-3of2mpm6\n", + " Local directory: /tmp/dask-scratch-space/worker-ba7l_i29\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -269,7 +259,7 @@ "
\n", - " Comm: tcp://127.0.0.1:33637\n", + " Comm: tcp://127.0.0.1:37301\n", " \n", " Total threads: 4\n", @@ -232,7 +222,7 @@ "
\n", - " Dashboard: http://127.0.0.1:46855/status\n", + " Dashboard: http://127.0.0.1:43305/status\n", " \n", " Memory: 7.66 GiB\n", @@ -240,13 +230,13 @@ "
\n", - " Nanny: tcp://127.0.0.1:34317\n", + " Nanny: tcp://127.0.0.1:39845\n", "
\n", - " Local directory: /tmp/dask-scratch-space/worker-5wbyfhxh\n", + " Local directory: /tmp/dask-scratch-space/worker-zgmrf6uh\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -314,7 +304,7 @@ "
\n", - " Comm: tcp://127.0.0.1:40539\n", + " Comm: tcp://127.0.0.1:34071\n", " \n", " Total threads: 4\n", @@ -277,7 +267,7 @@ "
\n", - " Dashboard: http://127.0.0.1:45731/status\n", + " Dashboard: http://127.0.0.1:42767/status\n", " \n", " Memory: 7.66 GiB\n", @@ -285,13 +275,13 @@ "
\n", - " Nanny: tcp://127.0.0.1:37707\n", + " Nanny: tcp://127.0.0.1:46785\n", "
\n", - " Local directory: /tmp/dask-scratch-space/worker-_e6spiif\n", + " Local directory: /tmp/dask-scratch-space/worker-f7acc4xk\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -363,10 +353,10 @@ "" ], "text/plain": [ - "" + "" ] }, - "execution_count": 53, + "execution_count": 2, "metadata": {}, "output_type": "execute_result" } @@ -424,7 +414,7 @@ }, { "cell_type": "code", - "execution_count": 74, + "execution_count": 9, "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62", "metadata": { "tags": [] @@ -481,7 +471,7 @@ }, { "cell_type": "code", - "execution_count": 75, + "execution_count": 13, "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f", "metadata": { "tags": [] @@ -578,7 +568,7 @@ "dask.array" ] }, - "execution_count": 75, + "execution_count": 13, "metadata": {}, "output_type": "execute_result" } @@ -633,12 +623,24 @@ }, { "cell_type": "code", - "execution_count": 79, + "execution_count": 8, "id": "c4f6ae3e-4511-448f-97d8-ff85172dea6c", "metadata": { "tags": [] }, - "outputs": [], + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'x_res' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[8], line 17\u001b[0m\n\u001b[1;32m 15\u001b[0m crs\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEPSG:4326\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 16\u001b[0m output_file \u001b[38;5;241m=\u001b[39m Path(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/home/tromain/Data/S2/ndvi_dask.tif\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m---> 17\u001b[0m create_raster(ndvi_array, output_file, \u001b[43mx_res\u001b[49m , y_res,\n\u001b[1;32m 18\u001b[0m top_left_x, top_left_y, crs)\n", + "\u001b[0;31mNameError\u001b[0m: name 'x_res' is not defined" + ] + } + ], "source": [ "def create_raster(data: np.ndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n", " transform = Affine.translation(top_left_x, top_left_y) * Affine.scale(x_res, -y_res)\n", @@ -670,35 +672,22 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 27, "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7", "metadata": { "tags": [] }, "outputs": [ { - "name": "stderr", - "output_type": "stream", - "text": [ - "Warning 1: Invalid value for NUM_THREADS: \n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Writing /work/scratch/data/romaint/img_ndvi_otb.tif...: 100% [**************************************************] (3s)\n" + "ename": "ModuleNotFoundError", + "evalue": "No module named 'otbApplication'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[27], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01motbApplication\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01motb\u001b[39;00m\n\u001b[1;32m 3\u001b[0m sentinel_2_dir \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/work/scratch/data/romaint\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 4\u001b[0m s2_b4 \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00msentinel_2_dir\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\u001b[39m\u001b[38;5;124m\"\u001b[39m\n", + "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'otbApplication'" ] - }, - { - "data": { - "text/plain": [ - "0" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" } ], "source": [ @@ -727,7 +716,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 28, "id": "73a731cf-a534-4b49-9a6d-425b58d87e45", "metadata": { "tags": [] @@ -737,14 +726,21 @@ "name": "stderr", "output_type": "stream", "text": [ - "Warning 1: Invalid value for NUM_THREADS: \n" + "bash: ligne 1: otbcli_BandMath : commande introuvable\n" ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Writing /work/scratch/data/romaint/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (2s)\n" + "ename": "CalledProcessError", + "evalue": "Command 'b'otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\" \\n'' returned non-zero exit status 127.", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mCalledProcessError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[28], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mget_ipython\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_cell_magic\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mbash\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43motbcli_BandMath -il \u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m \u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m -exp \u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m -out \u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m \u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m)\u001b[49m\n", + "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2541\u001b[0m, in \u001b[0;36mInteractiveShell.run_cell_magic\u001b[0;34m(self, magic_name, line, cell)\u001b[0m\n\u001b[1;32m 2539\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbuiltin_trap:\n\u001b[1;32m 2540\u001b[0m args \u001b[38;5;241m=\u001b[39m (magic_arg_s, cell)\n\u001b[0;32m-> 2541\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 2543\u001b[0m \u001b[38;5;66;03m# The code below prevents the output from being displayed\u001b[39;00m\n\u001b[1;32m 2544\u001b[0m \u001b[38;5;66;03m# when using magics with decorator @output_can_be_silenced\u001b[39;00m\n\u001b[1;32m 2545\u001b[0m \u001b[38;5;66;03m# when the last Python token in the expression is a ';'.\u001b[39;00m\n\u001b[1;32m 2546\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(fn, magic\u001b[38;5;241m.\u001b[39mMAGIC_OUTPUT_CAN_BE_SILENCED, \u001b[38;5;28;01mFalse\u001b[39;00m):\n", + "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/IPython/core/magics/script.py:155\u001b[0m, in \u001b[0;36mScriptMagics._make_script_magic..named_script_magic\u001b[0;34m(line, cell)\u001b[0m\n\u001b[1;32m 153\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 154\u001b[0m line \u001b[38;5;241m=\u001b[39m script\n\u001b[0;32m--> 155\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mshebang\u001b[49m\u001b[43m(\u001b[49m\u001b[43mline\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcell\u001b[49m\u001b[43m)\u001b[49m\n", + "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/IPython/core/magics/script.py:315\u001b[0m, in \u001b[0;36mScriptMagics.shebang\u001b[0;34m(self, line, cell)\u001b[0m\n\u001b[1;32m 310\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m args\u001b[38;5;241m.\u001b[39mraise_error \u001b[38;5;129;01mand\u001b[39;00m p\u001b[38;5;241m.\u001b[39mreturncode \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m0\u001b[39m:\n\u001b[1;32m 311\u001b[0m \u001b[38;5;66;03m# If we get here and p.returncode is still None, we must have\u001b[39;00m\n\u001b[1;32m 312\u001b[0m \u001b[38;5;66;03m# killed it but not yet seen its return code. We don't wait for it,\u001b[39;00m\n\u001b[1;32m 313\u001b[0m \u001b[38;5;66;03m# in case it's stuck in uninterruptible sleep. -9 = SIGKILL\u001b[39;00m\n\u001b[1;32m 314\u001b[0m rc \u001b[38;5;241m=\u001b[39m p\u001b[38;5;241m.\u001b[39mreturncode \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m9\u001b[39m\n\u001b[0;32m--> 315\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m CalledProcessError(rc, cell)\n", + "\u001b[0;31mCalledProcessError\u001b[0m: Command 'b'otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\" \\n'' returned non-zero exit status 127." ] } ], @@ -987,7 +983,7 @@ }, { "cell_type": "code", - "execution_count": 81, + "execution_count": 35, "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50", "metadata": {}, "outputs": [ @@ -995,35 +991,28 @@ "name": "stdout", "output_type": "stream", "text": [ - "Elapsed with Raster IO = 1.1344427589974657s\n", - "Elapsed with RIOXarray + dask = 0.027490641001350014s\n", - "Elapsed with list comprehension = 0.0035700430016731843s\n", - "Elapsed with normal compute = 0.0015671629989810754s\n" - ] - }, - { - "ename": "AttributeError", - "evalue": "'list' object has no attribute 'shape'", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)", - "File \u001b[0;32m:61\u001b[0m\n", - "Cell \u001b[0;32mIn[79], line 6\u001b[0m, in \u001b[0;36mcreate_raster\u001b[0;34m(data, output_file, x_res, y_res, top_left_x, top_left_y, crs)\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcreate_raster\u001b[39m(data: np\u001b[38;5;241m.\u001b[39mndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n\u001b[1;32m 2\u001b[0m transform \u001b[38;5;241m=\u001b[39m Affine\u001b[38;5;241m.\u001b[39mtranslation(top_left_x, top_left_y) \u001b[38;5;241m*\u001b[39m Affine\u001b[38;5;241m.\u001b[39mscale(x_res, \u001b[38;5;241m-\u001b[39my_res)\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m rasterio\u001b[38;5;241m.\u001b[39mopen(\n\u001b[1;32m 4\u001b[0m output_file, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mw\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m 5\u001b[0m driver\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGTiff\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m----> 6\u001b[0m height\u001b[38;5;241m=\u001b[39m\u001b[43mdata\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mshape\u001b[49m[\u001b[38;5;241m1\u001b[39m],\n\u001b[1;32m 7\u001b[0m width\u001b[38;5;241m=\u001b[39mdata\u001b[38;5;241m.\u001b[39mshape[\u001b[38;5;241m2\u001b[39m],\n\u001b[1;32m 8\u001b[0m count\u001b[38;5;241m=\u001b[39mdata\u001b[38;5;241m.\u001b[39mshape[\u001b[38;5;241m0\u001b[39m],\n\u001b[1;32m 9\u001b[0m dtype\u001b[38;5;241m=\u001b[39mdata\u001b[38;5;241m.\u001b[39mdtype,\n\u001b[1;32m 10\u001b[0m crs\u001b[38;5;241m=\u001b[39mcrs,\n\u001b[1;32m 11\u001b[0m transform\u001b[38;5;241m=\u001b[39mtransform\n\u001b[1;32m 12\u001b[0m ) \u001b[38;5;28;01mas\u001b[39;00m dst:\n\u001b[1;32m 13\u001b[0m dst\u001b[38;5;241m.\u001b[39mwrite(data)\n", - "\u001b[0;31mAttributeError\u001b[0m: 'list' object has no attribute 'shape'" + "Elapsed with Raster IO + numba = 1.105684628993913s\n", + "Elapsed with Raster IO + without numba = 1.2050202210011776s\n", + "Elapsed with RIOXarray + dask + function call = 3.137614963001397s\n", + "Elapsed with RIOXarray + dask + list comprehension = 3.91687624000042s\n", + "Elapsed with RIOXarray + dask + standard compute = 3.168711420999898s\n", + "CPU times: user 3.08 s, sys: 8.13 s, total: 11.2 s\n", + "Wall time: 12.5 s\n" ] } ], "source": [ "%%time\n", "\n", - "#from numba import jit,njit\n", + "from numba import jit,njit\n", "from xarray import DataArray \n", "from typing import List, Tuple, Union, Dict\n", "import rioxarray as rxr\n", "import numpy as np\n", "import time\n", "import rasterio\n", + "from pathlib import Path\n", + "import xarray\n", "\n", "@njit\n", "def one_pixel_ndvi(p1,p2):\n", @@ -1031,54 +1020,98 @@ "\n", "@njit\n", "def compute_ndvi(input_data_1: np.ndarray,input_data_2: np.ndarray):\n", - " ndvi_array = [[one_pixel_ndvi(i,j) for i in input_data_1] for j in input_data_2]\n", + " #ndvi_array = [one_pixel_ndvi(i,j) for i in input_data_1 for j in input_data_2]\n", + " ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n", " return ndvi_array\n", "\n", "def compute_ndvi_dask(input_data_1: DataArray,input_data_2: DataArray):\n", - " ndvi_array = [[(j-i)/(i+j) for i in input_data_1] for j in input_data_2]\n", + " ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n", + " ndvi_array.compute()\n", + " return ndvi_array\n", + "\n", + "\n", + "def compute_ndvi_std(input_data_1: np.ndarray,input_data_2: np.ndarray):\n", + " #ndvi_array = [one_pixel_ndvi(i,j) for i in input_data_1 for j in input_data_2]\n", + " ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n", " return ndvi_array\n", "\n", "sentinel_2_dir = \"/home/tromain/Data/S2\"\n", "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", "\n", + "start = time.perf_counter()\n", "with rasterio.open(s2_b4, 'r') as ds:\n", " input_data_b4 = ds.read() \n", "\n", "with rasterio.open(s2_b8, 'r') as ds:\n", " input_data_b8 = ds.read()\n", "\n", - "start = time.perf_counter()\n", "ndvi_computed = compute_ndvi(input_data_b4,input_data_b8)\n", + "crs=\"EPSG:4326\"\n", + "output_file = Path(\"/home/tromain/Data/S2/ndvi_numba.tif\")\n", + "create_raster(ndvi_computed, output_file, x_res , y_res,\n", + " top_left_x, top_left_y, crs)\n", "end = time.perf_counter()\n", - "print(\"Elapsed with Raster IO = {}s\".format((end - start)))\n", + "print(\"Elapsed with Raster IO + numba = {}s\".format((end - start)))\n", + "\n", + "start = time.perf_counter()\n", + "with rasterio.open(s2_b4, 'r') as ds:\n", + " input_data_b4 = ds.read() \n", + "\n", + "with rasterio.open(s2_b8, 'r') as ds:\n", + " input_data_b8 = ds.read()\n", + "\n", + "ndvi_computed = compute_ndvi_std(input_data_b4,input_data_b8)\n", + "crs=\"EPSG:4326\"\n", + "output_file = Path(\"/home/tromain/Data/S2/ndvi_without_numba.tif\")\n", + "create_raster(ndvi_computed, output_file, x_res , y_res,\n", + " top_left_x, top_left_y, crs)\n", + "end = time.perf_counter()\n", + "print(\"Elapsed with Raster IO + without numba = {}s\".format((end - start)))\n", "# Example with xarray\n", "#input_data_b4 = xarray.open_dataarray(s2_b4,chunks=False)\n", "#input_data_b8 = xarray.open_dataarray(s2_b8,chunks=False)\n", - "#ndvi_computed = compute_ndvi(input_data_b4.to_numpy(),input_data_b8.to_numpy())\n", + "#start = time.perf_counter()\n", + "#input_data_b4.load()\n", + "#input_data_b8.load()\n", + "#ndvi_computed = xarray.apply_ufunc(compute_ndvi_dask,input_data_b4,input_data_b8)\n", + "#compute_ndvi(input_data_b4.to_numpy(),input_data_b8.to_numpy())\n", + "#end = time.perf_counter()\n", + "#print(\"Elapsed with Xarray + numba = {}s\".format((end - start)))\n", "\n", "# Example with rioxarray \n", + "start = time.perf_counter()\n", "input_data_b4 = rxr.open_rasterio(s2_b4,chunks=True)\n", "input_data_b8 = rxr.open_rasterio(s2_b8,chunks=True)\n", - "start = time.perf_counter()\n", + "#(-1,2200,2200)\n", "ndvi_computed = compute_ndvi_dask(input_data_b4,input_data_b8)\n", + "output_file = Path(\"/home/tromain/Data/S2/ndvi_rioxarray.tif\")\n", + "create_raster(np.asarray(ndvi_computed), output_file, x_res , y_res,\n", + " top_left_x, top_left_y, crs)\n", "end = time.perf_counter()\n", "\n", - "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n", + "print(\"Elapsed with RIOXarray + dask + function call = {}s\".format((end - start)))\n", "\n", "start = time.perf_counter()\n", - "ndvi_array = [[(j-i)/(i+j) for i in input_data_b4] for j in input_data_b8]\n", + "ndvi_array = [(j-i)/(i+j) for i in input_data_b4 for j in input_data_b8]\n", + "dsk_array = da.asarray(ndvi_array)\n", + "dsk_array.visualize(\"/home/tromain/Data/S2/dsk_array.png\")\n", + "crs=\"EPSG:4326\"\n", + "output_file = Path(\"/home/tromain/Data/S2/ndvi_list_comp.tif\")\n", + "create_raster(np.asarray(ndvi_array), output_file, x_res , y_res,\n", + " top_left_x, top_left_y, crs)\n", "end = time.perf_counter()\n", - "print(\"Elapsed with list comprehension = {}s\".format((end - start)))\n", + "\n", + "print(\"Elapsed with RIOXarray + dask + list comprehension = {}s\".format((end - start)))\n", "start = time.perf_counter()\n", - "ndvi_array = (input_data_array[1] - input_data_array[0]) / (input_data_array[1] + input_data_array[0])[None, :, :]\n", + "ndvi_array = (input_data_b8 - input_data_b4) / (input_data_b8 + input_data_b4)\n", + "ndvi_array.compute()\n", + "ndvi_array.data.visualize(\"/home/tromain/Data/S2/visu_std.png\")\n", + "output_file = Path(\"/home/tromain/Data/S2/ndvi_std.tif\")\n", + "create_raster(np.asarray(ndvi_array), output_file, x_res , y_res,\n", + " top_left_x, top_left_y, crs)\n", "end = time.perf_counter()\n", - "print(\"Elapsed with normal compute = {}s\".format((end - start)))\n", - "\n", - "crs=\"EPSG:4326\"\n", - "output_file = Path(\"/home/tromain/Data/S2/ndvi_dask_optimized.tif\")\n", - "create_raster(ndvi_computed, output_file, x_res , y_res,\n", - " top_left_x, top_left_y, crs)" + "print(\"Elapsed with RIOXarray + dask + standard compute = {}s\".format((end - start)))\n" ] }, { @@ -1088,7 +1121,8 @@ "source": [ "# Performances multiprocessing vs multithreading\n", "\n", - "Depending on the processing applied to an image it is interesting to use the multiprocessing approach instead of the multithreading which is not really a multithreading in python as every thread is executed one after the other. (in Python 3.13, the GIL has been reworked to improve the situation)" + "Depending on the processing applied to an image it is interesting to use the multiprocessing approach instead of the multithreading which is not really a multithreading in python as every thread is executed one after the other. (in Python 3.13, the GIL has been reworked to improve the situation)\n", + "\n" ] }, { @@ -1098,7 +1132,45 @@ "metadata": {}, "outputs": [], "source": [ - "import multiprocessing\n" + "import multiprocessing\n", + "from xarray import DataArray\n", + "import rioxarray as rxr\n", + "import time\n", + " \n", + "def task_ndvi(input_data_1: DataArray,input_data_2: DataArray):\n", + " ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n", + " return ndvi_array\n", + " \n", + "if __name__ == \"__main__\":\n", + " sentinel_2_dir = \"/home/tromain/Data/S2\"\n", + " s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", + " s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", + " with rasterio.open(s2_b4, 'r') as ds:\n", + " input_data_b4 = ds.read() \n", + "\n", + " with rasterio.open(s2_b8, 'r') as ds:\n", + " input_data_b8 = ds.read()\n", + "\n", + " input_data_b4.load()\n", + " input_data_b8.load()\n", + " \n", + " start_time = time.perf_counter()\n", + " \n", + " # Creates two processes\n", + " p1 = multiprocessing.Process(target=task_ndvi)\n", + " p2 = multiprocessing.Process(target=task_ndvi)\n", + " p3 = multiprocessing.Process(target=task_ndvi)\n", + " p4 = multiprocessing.Process(target=task_ndvi)\n", + " \n", + " # Starts both processes\n", + " p1.start()\n", + " p2.start()\n", + " p3.start()\n", + " p4.start()\n", + " \n", + " finish_time = time.perf_counter()\n", + " \n", + " print(f\"Program finished in {finish_time-start_time} seconds\")\n" ] }, { From c554e8015ef76f1ab7891757f672d7ee68918588 Mon Sep 17 00:00:00 2001 From: Thibaut ROMAIN Date: Wed, 18 Dec 2024 15:15:38 +0100 Subject: [PATCH 04/37] Update with latest version --- tuto_greenit.ipynb | 2508 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 2069 insertions(+), 439 deletions(-) diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb index 7294b8b..31e1c74 100644 --- a/tuto_greenit.ipynb +++ b/tuto_greenit.ipynb @@ -35,17 +35,27 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 8, "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21", "metadata": { "tags": [] }, "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.10/dist-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n", + "Perhaps you already have a cluster running?\n", + "Hosting the HTTP server on port 36839 instead\n", + " warnings.warn(\n" + ] + }, { "name": "stdout", "output_type": "stream", "text": [ - "Dask Dashboard: http://127.0.0.1:8787/status\n" + "Dask Dashboard: http://127.0.0.1:36839/status\n" ] }, { @@ -55,7 +65,7 @@ "
\n", "
\n", "

Client

\n", - "

Client-52c9be04-a10b-11ef-b7e1-589671ac823c

\n", + "

Client-70dfeac2-bd46-11ef-9045-00620ba454ee

\n", "
\n", - " Comm: tcp://127.0.0.1:32929\n", + " Comm: tcp://127.0.0.1:43133\n", " \n", " Total threads: 4\n", @@ -322,7 +312,7 @@ "
\n", - " Dashboard: http://127.0.0.1:38331/status\n", + " Dashboard: http://127.0.0.1:43303/status\n", " \n", " Memory: 7.66 GiB\n", @@ -330,13 +320,13 @@ "
\n", - " Nanny: tcp://127.0.0.1:46609\n", + " Nanny: tcp://127.0.0.1:36083\n", "
\n", - " Local directory: /tmp/dask-scratch-space/worker-u77lvci7\n", + " Local directory: /tmp/dask-scratch-space/worker-onkpffvq\n", "
\n", "\n", " \n", @@ -68,7 +78,7 @@ " \n", " \n", " \n", " \n", " \n", @@ -77,6 +87,10 @@ "
\n", - " Dashboard: http://127.0.0.1:8787/status\n", + " Dashboard: http://127.0.0.1:36839/status\n", "
\n", "\n", " \n", + " \n", + " \n", "\n", " \n", "
\n", @@ -86,11 +100,11 @@ " \n", "
\n", "

LocalCluster

\n", - "

e120fc0c

\n", + "

02f9fee4

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", @@ -123,11 +137,11 @@ "
\n", "
\n", "

Scheduler

\n", - "

Scheduler-bbff2036-0b0c-46eb-b353-6cab67208131

\n", + "

Scheduler-7238732b-a7d6-43b3-99a6-b17a85b0dcd9

\n", "
\n", - " Dashboard: http://127.0.0.1:8787/status\n", + " Dashboard: http://127.0.0.1:36839/status\n", " \n", " Workers: 4\n", @@ -98,10 +112,10 @@ "
\n", - " Total threads: 16\n", + " Total threads: 4\n", " \n", - " Total memory: 30.63 GiB\n", + " Total memory: 28.00 GiB\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", @@ -146,7 +160,7 @@ " Started: Just now\n", " \n", " \n", " \n", "
\n", - " Comm: tcp://127.0.0.1:38879\n", + " Comm: tcp://127.0.0.1:33599\n", " \n", " Workers: 4\n", @@ -135,10 +149,10 @@ "
\n", - " Dashboard: http://127.0.0.1:8787/status\n", + " Dashboard: http://127.0.0.1:36839/status\n", " \n", - " Total threads: 16\n", + " Total threads: 4\n", "
\n", - " Total memory: 30.63 GiB\n", + " Total memory: 28.00 GiB\n", "
\n", @@ -169,29 +183,29 @@ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -214,29 +228,29 @@ "
\n", - " Comm: tcp://127.0.0.1:37533\n", + " Comm: tcp://127.0.0.1:35379\n", " \n", - " Total threads: 4\n", + " Total threads: 1\n", "
\n", - " Dashboard: http://127.0.0.1:42723/status\n", + " Dashboard: http://127.0.0.1:42093/status\n", " \n", - " Memory: 7.66 GiB\n", + " Memory: 7.00 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:34589\n", + " Nanny: tcp://127.0.0.1:43625\n", "
\n", - " Local directory: /tmp/dask-scratch-space/worker-ba7l_i29\n", + " Local directory: /tmp/slurm-28574237/dask-scratch-space/worker-15rnueb2\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -259,29 +273,29 @@ "
\n", - " Comm: tcp://127.0.0.1:37301\n", + " Comm: tcp://127.0.0.1:34907\n", " \n", - " Total threads: 4\n", + " Total threads: 1\n", "
\n", - " Dashboard: http://127.0.0.1:43305/status\n", + " Dashboard: http://127.0.0.1:43573/status\n", " \n", - " Memory: 7.66 GiB\n", + " Memory: 7.00 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:39845\n", + " Nanny: tcp://127.0.0.1:34007\n", "
\n", - " Local directory: /tmp/dask-scratch-space/worker-zgmrf6uh\n", + " Local directory: /tmp/slurm-28574237/dask-scratch-space/worker-p8x8_6ao\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -304,29 +318,29 @@ "
\n", - " Comm: tcp://127.0.0.1:34071\n", + " Comm: tcp://127.0.0.1:44905\n", " \n", - " Total threads: 4\n", + " Total threads: 1\n", "
\n", - " Dashboard: http://127.0.0.1:42767/status\n", + " Dashboard: http://127.0.0.1:36541/status\n", " \n", - " Memory: 7.66 GiB\n", + " Memory: 7.00 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:46785\n", + " Nanny: tcp://127.0.0.1:34321\n", "
\n", - " Local directory: /tmp/dask-scratch-space/worker-f7acc4xk\n", + " Local directory: /tmp/slurm-28574237/dask-scratch-space/worker-a5wqbi7k\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -353,10 +367,10 @@ "" ], "text/plain": [ - "" + "" ] }, - "execution_count": 2, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -370,7 +384,7 @@ "import rasterio\n", "import rioxarray as rxr\n", "from dask import delayed\n", - "from dask.distributed import Client, LocalCluster\n", + "from dask.distributed import Client, LocalCluster, Lock\n", "from rasterio.transform import Affine\n", "\n", "from utils import create_map_with_rasters\n", @@ -414,7 +428,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 3, "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62", "metadata": { "tags": [] @@ -453,7 +467,7 @@ " return da.concatenate(results), x_res, y_res, top_left_x, top_left_y, crs\n", "\n", "# Paths to RGB Sentinel-2 bands\n", - "sentinel_2_dir = \"/home/tromain/Data/S2\"\n", + "sentinel_2_dir = \"/work/scratch/data/romaint\"\n", "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", "#reading_chunks = (-1,2200,2200)\n", @@ -471,128 +485,24 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": null, "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f", "metadata": { "tags": [] }, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - " Comm: tcp://127.0.0.1:43133\n", + " Comm: tcp://127.0.0.1:45347\n", " \n", - " Total threads: 4\n", + " Total threads: 1\n", "
\n", - " Dashboard: http://127.0.0.1:43303/status\n", + " Dashboard: http://127.0.0.1:46387/status\n", " \n", - " Memory: 7.66 GiB\n", + " Memory: 7.00 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:36083\n", + " Nanny: tcp://127.0.0.1:42059\n", "
\n", - " Local directory: /tmp/dask-scratch-space/worker-onkpffvq\n", + " Local directory: /tmp/slurm-28574237/dask-scratch-space/worker-sm2i26gx\n", "
\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Array Chunk
Bytes 459.90 MiB 127.98 MiB
Shape (2, 10980, 10980) (1, 6111, 10980)
Dask graph 4 chunks in 5 graph layers
Data type int16 numpy.ndarray
\n", - " 		\n", - " \n", - "\n", - " \n", - " \n", - " \n", - " \n", - "\n", - " \n", - " \n", - " \n", - " \n", - "\n", - " \n", - " \n", - "\n", - " \n", - " \n", - " \n", - " \n", - "\n", - " \n", - " \n", - " \n", - "\n", - " \n", - " \n", - "\n", - " \n", - " \n", - " \n", - " \n", - "\n", - " \n", - " \n", - " \n", - "\n", - " \n", - " \n", - "\n", - " \n", - " 10980\n", - " 10980\n", - " 2\n", - "\n", - "
" - ], - "text/plain": [ - "dask.array" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "input_data_array" ] }, { "cell_type": "code", - "execution_count": 76, + "execution_count": null, "id": "2dad7fe0-23bc-40b2-b27e-4c7cb0f8dc0e", "metadata": { "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "(2, 10980, 10980)\n" - ] - } - ], + "outputs": [], "source": [ "print(input_data_array.shape)\n", "ndvi_array = (input_data_array[1] - input_data_array[0]) / (input_data_array[1] + input_data_array[0])[None, :, :]" @@ -600,21 +510,12 @@ }, { "cell_type": "code", - "execution_count": 77, + "execution_count": null, "id": "19fc2fce-4aca-4b7a-abd6-6baba3b5c464", "metadata": { "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "CPU times: user 469 ms, sys: 1.01 s, total: 1.48 s\n", - "Wall time: 2.1 s\n" - ] - } - ], + "outputs": [], "source": [ "%%time\n", "\n", @@ -623,24 +524,12 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 4, "id": "c4f6ae3e-4511-448f-97d8-ff85172dea6c", "metadata": { "tags": [] }, - "outputs": [ - { - "ename": "NameError", - "evalue": "name 'x_res' is not defined", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[8], line 17\u001b[0m\n\u001b[1;32m 15\u001b[0m crs\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEPSG:4326\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 16\u001b[0m output_file \u001b[38;5;241m=\u001b[39m Path(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/home/tromain/Data/S2/ndvi_dask.tif\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m---> 17\u001b[0m create_raster(ndvi_array, output_file, \u001b[43mx_res\u001b[49m , y_res,\n\u001b[1;32m 18\u001b[0m top_left_x, top_left_y, crs)\n", - "\u001b[0;31mNameError\u001b[0m: name 'x_res' is not defined" - ] - } - ], + "outputs": [], "source": [ "def create_raster(data: np.ndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n", " transform = Affine.translation(top_left_x, top_left_y) * Affine.scale(x_res, -y_res)\n", @@ -658,8 +547,8 @@ "\n", "crs=\"EPSG:4326\"\n", "output_file = Path(\"/home/tromain/Data/S2/ndvi_dask.tif\")\n", - "create_raster(ndvi_array, output_file, x_res , y_res,\n", - " top_left_x, top_left_y, crs)" + "#create_raster(ndvi_array, output_file, x_res , y_res,\n", + "# top_left_x, top_left_y, crs)" ] }, { @@ -672,22 +561,35 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": 9, "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7", "metadata": { "tags": [] }, "outputs": [ { - "ename": "ModuleNotFoundError", - "evalue": "No module named 'otbApplication'", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[27], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01motbApplication\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01motb\u001b[39;00m\n\u001b[1;32m 3\u001b[0m sentinel_2_dir \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/work/scratch/data/romaint\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 4\u001b[0m s2_b4 \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00msentinel_2_dir\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\u001b[39m\u001b[38;5;124m\"\u001b[39m\n", - "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'otbApplication'" + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning 1: Invalid value for NUM_THREADS: \n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Writing /work/scratch/data/romaint/img_ndvi_otb.tif...: 100% [**************************************************] (1m 45s)\n" ] + }, + { + "data": { + "text/plain": [ + "0" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" } ], "source": [ @@ -697,10 +599,14 @@ "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", "out_ndvi_otb_py=\"/work/scratch/data/romaint/img_ndvi_otb.tif\"\n", + "#phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n", + "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", "#Compute NDVI with OTB in python\n", "app_ndvi_otb = otb.Registry.CreateApplication(\"BandMath\")\n", - "app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n", - "app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n", + "#app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n", + "#app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n", + "app_ndvi_otb.SetParameterStringList(\"il\",[phr_product])\n", + "app_ndvi_otb.SetParameterString(\"exp\",\"(im1b4-im1b1)/(im1b4+im1b1)\")\n", "app_ndvi_otb.SetParameterString(\"out\",out_ndvi_otb_py)\n", "app_ndvi_otb.ExecuteAndWriteOutput()\n" ] @@ -716,34 +622,12 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": null, "id": "73a731cf-a534-4b49-9a6d-425b58d87e45", "metadata": { "tags": [] }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "bash: ligne 1: otbcli_BandMath : commande introuvable\n" - ] - }, - { - "ename": "CalledProcessError", - "evalue": "Command 'b'otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\" \\n'' returned non-zero exit status 127.", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mCalledProcessError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[28], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mget_ipython\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_cell_magic\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mbash\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43motbcli_BandMath -il \u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m \u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m -exp \u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m -out \u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m \u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2541\u001b[0m, in \u001b[0;36mInteractiveShell.run_cell_magic\u001b[0;34m(self, magic_name, line, cell)\u001b[0m\n\u001b[1;32m 2539\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbuiltin_trap:\n\u001b[1;32m 2540\u001b[0m args \u001b[38;5;241m=\u001b[39m (magic_arg_s, cell)\n\u001b[0;32m-> 2541\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 2543\u001b[0m \u001b[38;5;66;03m# The code below prevents the output from being displayed\u001b[39;00m\n\u001b[1;32m 2544\u001b[0m \u001b[38;5;66;03m# when using magics with decorator @output_can_be_silenced\u001b[39;00m\n\u001b[1;32m 2545\u001b[0m \u001b[38;5;66;03m# when the last Python token in the expression is a ';'.\u001b[39;00m\n\u001b[1;32m 2546\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(fn, magic\u001b[38;5;241m.\u001b[39mMAGIC_OUTPUT_CAN_BE_SILENCED, \u001b[38;5;28;01mFalse\u001b[39;00m):\n", - "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/IPython/core/magics/script.py:155\u001b[0m, in \u001b[0;36mScriptMagics._make_script_magic..named_script_magic\u001b[0;34m(line, cell)\u001b[0m\n\u001b[1;32m 153\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 154\u001b[0m line \u001b[38;5;241m=\u001b[39m script\n\u001b[0;32m--> 155\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mshebang\u001b[49m\u001b[43m(\u001b[49m\u001b[43mline\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcell\u001b[49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/IPython/core/magics/script.py:315\u001b[0m, in \u001b[0;36mScriptMagics.shebang\u001b[0;34m(self, line, cell)\u001b[0m\n\u001b[1;32m 310\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m args\u001b[38;5;241m.\u001b[39mraise_error \u001b[38;5;129;01mand\u001b[39;00m p\u001b[38;5;241m.\u001b[39mreturncode \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m0\u001b[39m:\n\u001b[1;32m 311\u001b[0m \u001b[38;5;66;03m# If we get here and p.returncode is still None, we must have\u001b[39;00m\n\u001b[1;32m 312\u001b[0m \u001b[38;5;66;03m# killed it but not yet seen its return code. We don't wait for it,\u001b[39;00m\n\u001b[1;32m 313\u001b[0m \u001b[38;5;66;03m# in case it's stuck in uninterruptible sleep. -9 = SIGKILL\u001b[39;00m\n\u001b[1;32m 314\u001b[0m rc \u001b[38;5;241m=\u001b[39m p\u001b[38;5;241m.\u001b[39mreturncode \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m9\u001b[39m\n\u001b[0;32m--> 315\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m CalledProcessError(rc, cell)\n", - "\u001b[0;31mCalledProcessError\u001b[0m: Command 'b'otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\" \\n'' returned non-zero exit status 127." - ] - } - ], + "outputs": [], "source": [ "%%bash\n", "otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\" " @@ -761,44 +645,10 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "id": "71d9a855-86f3-48aa-a9eb-32a1dc853649", "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "2024-11-07 09:04:44 (WARNING) Superimpose: Forcing PHR mode with PHR data. You need to add \"-mode default\" to force the default mode with PHR images.\n", - "2024-11-07 09:04:44 (WARNING) Superimpose: Forcing PHR mode with PHR data. You need to add \"-mode default\" to force the default mode with PHR images.\n", - "2024-11-07 09:04:44 (WARNING) Superimpose: Forcing PHR mode with PHR data. You need to add \"-mode default\" to force the default mode with PHR images.\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Warning 1: Invalid value for NUM_THREADS: \n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Writing /work/scratch/data/romaint/SuperimposedXS_to_PAN.tif?&writerpctags=true...: 100% [**************************************************] (2m 59s)\n" - ] - }, - { - "data": { - "text/plain": [ - "0" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "import otbApplication as otb\n", "\n", @@ -824,64 +674,12 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": null, "id": "e7568371-4a69-426e-9247-c7e4abe7aef2", "metadata": { "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - " Size: 3GB\n", - "dask.array, shape=(1, 23796, 31280), dtype=float32, chunksize=(1, 1072, 31280), chunktype=numpy.ndarray>\n", - "Coordinates:\n", - " * band (band) int64 8B 1\n", - " * x (x) float64 250kB 0.5 1.5 2.5 ... 3.128e+04 3.128e+04 3.128e+04\n", - " * y (y) float64 190kB 0.5 1.5 2.5 ... 2.379e+04 2.379e+04 2.38e+04\n", - " spatial_ref int64 8B 0\n", - "Attributes: (12/33)\n", - " METADATATYPE: OTB\n", - " OTB_VERSION: 9.1.0\n", - " TimeRangeStart: 2023-02-28T11:04:15.2040000Z\n", - " ProductionDate: 2024-04-15T13:25:33.956Z\n", - " AcquisitionDate: 2023-02-28T11:04:15.1Z\n", - " SatAzimuth: 202.389\n", - " ... ...\n", - " PhysicalBias: 0\n", - " PhysicalGain: 11.71\n", - " EnhancedBandName: PAN\n", - " BandName: P\n", - " scale_factor: 1.0\n", - " add_offset: 0.0\n", - " Size: 186MB\n", - "dask.array\n", - "Coordinates:\n", - " * band (band) int64 8B 1\n", - " * x (x) float64 63kB 2.0 6.0 10.0 ... 3.127e+04 3.127e+04 3.128e+04\n", - " * y (y) float64 48kB 2.0 6.0 10.0 ... 2.379e+04 2.379e+04 2.379e+04\n", - " spatial_ref int64 8B 0\n", - "Attributes: (12/33)\n", - " METADATATYPE: OTB\n", - " OTB_VERSION: 9.1.0\n", - " TimeRangeStart: 2023-02-28T11:04:15.2040000Z\n", - " ProductionDate: 2024-04-15T13:25:33.956Z\n", - " AcquisitionDate: 2023-02-28T11:04:15.1Z\n", - " SatAzimuth: 202.389\n", - " ... ...\n", - " PhysicalBias: 0\n", - " PhysicalGain: 11.71\n", - " EnhancedBandName: PAN\n", - " BandName: P\n", - " scale_factor: 1.0\n", - " add_offset: 0.0\n", - "True\n", - "CPU times: user 1.87 s, sys: 633 ms, total: 2.5 s\n", - "Wall time: 4.95 s\n" - ] - } - ], + "outputs": [], "source": [ "%%time\n", "\n", @@ -921,41 +719,15 @@ "metadata": { "tags": [] }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:365: UserWarning: The specified chunks separate the stored chunks along dimension \"y\" starting at index 1. This could degrade performance. Instead, consider rechunking after loading.\n", - " var_chunks = _get_chunk(var, chunks, chunkmanager)\n", - "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:365: UserWarning: The specified chunks separate the stored chunks along dimension \"x\" starting at index 1. This could degrade performance. Instead, consider rechunking after loading.\n", - " var_chunks = _get_chunk(var, chunks, chunkmanager)\n", - "ERROR 1: PROJ: internal_proj_create_from_database: /home/tromain/Apps/miniconda3/envs/env_greenit/otb9/share/proj/proj.db contains DATABASE.LAYOUT.VERSION.MINOR = 2 whereas a number >= 3 is expected. It comes from another PROJ installation.\n", - "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:365: UserWarning: The specified chunks separate the stored chunks along dimension \"y\" starting at index 1. This could degrade performance. Instead, consider rechunking after loading.\n", - " var_chunks = _get_chunk(var, chunks, chunkmanager)\n", - "/home/tromain/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:365: UserWarning: The specified chunks separate the stored chunks along dimension \"x\" starting at index 1. This could degrade performance. Instead, consider rechunking after loading.\n", - " var_chunks = _get_chunk(var, chunks, chunkmanager)\n", - "ERROR 1: PROJ: internal_proj_create_from_database: /home/tromain/Apps/miniconda3/envs/env_greenit/otb9/share/proj/proj.db contains DATABASE.LAYOUT.VERSION.MINOR = 2 whereas a number >= 3 is expected. It comes from another PROJ installation.\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "(1, 23796, 31283)\n", - "float32\n", - "(4, 5949, 7822)\n" - ] - } - ], + "outputs": [], "source": [ "%%time\n", "\n", "import xarray\n", "from pathlib import Path\n", "\n", - "pan_raster_url = \"/home/tromain/Data/PHR_OTB/IMG_PHR1B_P_001/IMG_PHR1B_P_202302281104151_SEN_6967639101-1_R1C1.JP2\"\n", - "xs_raster_url = \"/home/tromain/Data/PHR_OTB/IMG_PHR1B_MS_004/IMG_PHR1B_MS_202302281104151_SEN_6967639101-2_R1C1.JP2\"\n", + "pan_raster_url = \"/work/scratch/data/tanguyy/public/PHR_OTB/IMG_PHR1B_P_001/IMG_PHR1B_P_202302281104151_SEN_6967639101-1_R1C1.JP2\"\n", + "xs_raster_url = \"/work/scratch/data/tanguyy/public/PHR_OTB/IMG_PHR1B_MS_004/IMG_PHR1B_MS_202302281104151_SEN_6967639101-2_R1C1.JP2\"\n", "\n", "xds = xarray.open_dataarray(pan_raster_url,chunks=True)\n", "xds.rio.write_crs(\"epsg:4326\", inplace=True)\n", @@ -968,7 +740,7 @@ "xds_match.rio.write_nodata(0, inplace=True)\n", "xds_repr_match = xds.rio.reproject_match(xds_match)\n", "crs=\"EPSG:4326\"\n", - "output_file = Path(\"/home/tromain/Data/superimpose_reproject_match.tif\")\n", + "output_file = Path(\"/work/scratch/data/romaint/superimpose_reproject_match.tif\")\n", "xds_repr_match.rio.to_raster(output_file)" ] }, @@ -977,27 +749,685 @@ "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386", "metadata": {}, "source": [ - "# Using numba for NDVI ?\n", + "# Using numba vs xarray vs numpy for NDVI ?\n", "Here we will try to use numba for ndvi computation" ] }, { "cell_type": "code", - "execution_count": 35, + "execution_count": 23, "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50", - "metadata": {}, + "metadata": { + "tags": [] + }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Elapsed with Raster IO + numba = 1.105684628993913s\n", - "Elapsed with Raster IO + without numba = 1.2050202210011776s\n", - "Elapsed with RIOXarray + dask + function call = 3.137614963001397s\n", - "Elapsed with RIOXarray + dask + list comprehension = 3.91687624000042s\n", - "Elapsed with RIOXarray + dask + standard compute = 3.168711420999898s\n", - "CPU times: user 3.08 s, sys: 8.13 s, total: 11.2 s\n", - "Wall time: 12.5 s\n" + "Elapsed with Raster IO + numba = 1.2825823966413736s\n", + "Elapsed with Raster IO + without numba = 1.1891403198242188s\n", + "Elapsed with Xarray + apply ufunc = 2.707308219745755s\n", + "Elapsed with RIOXarray + dask = 5.868392776697874s\n", + "(4, 41663, 39844)\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-11-14 15:29:30,639 - distributed.core - ERROR - Exception while handling op get_data\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 2)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 820, in _handle_comm\n", + " result = await result\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1734, in get_data\n", + " data = {k: self.data[k] for k in keys if k in self.data}\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1734, in \n", + " data = {k: self.data[k] for k in keys if k in self.data}\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "2024-11-14 15:29:30,645 - distributed.core - ERROR - Exception while handling op get_data\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 3)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 820, in _handle_comm\n", + " result = await result\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1734, in get_data\n", + " data = {k: self.data[k] for k in keys if k in self.data}\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1734, in \n", + " data = {k: self.data[k] for k in keys if k in self.data}\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "2024-11-14 15:29:36,735 - distributed.worker.memory - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker-memory.html#memory-not-released-back-to-the-os for more information. -- Unmanaged memory: 4.94 GiB -- Worker memory limit: 7.00 GiB\n", + "2024-11-14 15:29:36,819 - distributed.worker.memory - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker-memory.html#memory-not-released-back-to-the-os for more information. -- Unmanaged memory: 4.94 GiB -- Worker memory limit: 7.00 GiB\n", + "2024-11-14 15:29:36,918 - distributed.worker.memory - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker-memory.html#memory-not-released-back-to-the-os for more information. -- Unmanaged memory: 4.94 GiB -- Worker memory limit: 7.00 GiB\n", + "Task exception was never retrieved\n", + "future: exception=Exception('AssertionError()')>\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 2870, in get_data_from_worker\n", + " return await retry_operation(_get_data, operation=\"get_data_from_worker\")\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py\", line 400, in retry_operation\n", + " return await retry(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py\", line 385, in retry\n", + " return await coro()\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 2850, in _get_data\n", + " response = await send_recv(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 1013, in send_recv\n", + " raise Exception(response[\"exception_text\"])\n", + "Exception: AssertionError()\n" + ] + }, + { + "ename": "Exception", + "evalue": "AssertionError()", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mException\u001b[0m Traceback (most recent call last)", + "File \u001b[0;32m:90\u001b[0m\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/xarray/core/dataarray.py:1092\u001b[0m, in \u001b[0;36mDataArray.compute\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 1073\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"Manually trigger loading of this array's data from disk or a\u001b[39;00m\n\u001b[1;32m 1074\u001b[0m \u001b[38;5;124;03mremote source into memory and return a new array. The original is\u001b[39;00m\n\u001b[1;32m 1075\u001b[0m \u001b[38;5;124;03mleft unaltered.\u001b[39;00m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 1089\u001b[0m \u001b[38;5;124;03mdask.compute\u001b[39;00m\n\u001b[1;32m 1090\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m 1091\u001b[0m new \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mcopy(deep\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m)\n\u001b[0;32m-> 1092\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mnew\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mload\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/xarray/core/dataarray.py:1066\u001b[0m, in \u001b[0;36mDataArray.load\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 1048\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mload\u001b[39m(\u001b[38;5;28mself\u001b[39m: T_DataArray, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m T_DataArray:\n\u001b[1;32m 1049\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"Manually trigger loading of this array's data from disk or a\u001b[39;00m\n\u001b[1;32m 1050\u001b[0m \u001b[38;5;124;03m remote source into memory and return this array.\u001b[39;00m\n\u001b[1;32m 1051\u001b[0m \n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 1064\u001b[0m \u001b[38;5;124;03m dask.compute\u001b[39;00m\n\u001b[1;32m 1065\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[0;32m-> 1066\u001b[0m ds \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_to_temp_dataset\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mload\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1067\u001b[0m new \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_from_temp_dataset(ds)\n\u001b[1;32m 1068\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_variable \u001b[38;5;241m=\u001b[39m new\u001b[38;5;241m.\u001b[39m_variable\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/xarray/core/dataset.py:739\u001b[0m, in \u001b[0;36mDataset.load\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 736\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mdask\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01marray\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mda\u001b[39;00m\n\u001b[1;32m 738\u001b[0m \u001b[38;5;66;03m# evaluate all the dask arrays simultaneously\u001b[39;00m\n\u001b[0;32m--> 739\u001b[0m evaluated_data \u001b[38;5;241m=\u001b[39m \u001b[43mda\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompute\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mlazy_data\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mvalues\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 741\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m k, data \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mzip\u001b[39m(lazy_data, evaluated_data):\n\u001b[1;32m 742\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mvariables[k]\u001b[38;5;241m.\u001b[39mdata \u001b[38;5;241m=\u001b[39m data\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/dask/base.py:600\u001b[0m, in \u001b[0;36mcompute\u001b[0;34m(traverse, optimize_graph, scheduler, get, *args, **kwargs)\u001b[0m\n\u001b[1;32m 597\u001b[0m keys\u001b[38;5;241m.\u001b[39mappend(x\u001b[38;5;241m.\u001b[39m__dask_keys__())\n\u001b[1;32m 598\u001b[0m postcomputes\u001b[38;5;241m.\u001b[39mappend(x\u001b[38;5;241m.\u001b[39m__dask_postcompute__())\n\u001b[0;32m--> 600\u001b[0m results \u001b[38;5;241m=\u001b[39m \u001b[43mschedule\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdsk\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkeys\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 601\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m repack([f(r, \u001b[38;5;241m*\u001b[39ma) \u001b[38;5;28;01mfor\u001b[39;00m r, (f, a) \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mzip\u001b[39m(results, postcomputes)])\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/client.py:3125\u001b[0m, in \u001b[0;36mClient.get\u001b[0;34m(self, dsk, keys, workers, allow_other_workers, resources, sync, asynchronous, direct, retries, priority, fifo_timeout, actors, **kwargs)\u001b[0m\n\u001b[1;32m 3123\u001b[0m should_rejoin \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mFalse\u001b[39;00m\n\u001b[1;32m 3124\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m-> 3125\u001b[0m results \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mgather\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpacked\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43masynchronous\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43masynchronous\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdirect\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdirect\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 3126\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m 3127\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m futures\u001b[38;5;241m.\u001b[39mvalues():\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/client.py:2294\u001b[0m, in \u001b[0;36mClient.gather\u001b[0;34m(self, futures, errors, direct, asynchronous)\u001b[0m\n\u001b[1;32m 2292\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 2293\u001b[0m local_worker \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m-> 2294\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msync\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 2295\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_gather\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2296\u001b[0m \u001b[43m \u001b[49m\u001b[43mfutures\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2297\u001b[0m \u001b[43m \u001b[49m\u001b[43merrors\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43merrors\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2298\u001b[0m \u001b[43m \u001b[49m\u001b[43mdirect\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdirect\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2299\u001b[0m \u001b[43m \u001b[49m\u001b[43mlocal_worker\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlocal_worker\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2300\u001b[0m \u001b[43m \u001b[49m\u001b[43masynchronous\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43masynchronous\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2301\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils.py:339\u001b[0m, in \u001b[0;36mSyncMethodMixin.sync\u001b[0;34m(self, func, asynchronous, callback_timeout, *args, **kwargs)\u001b[0m\n\u001b[1;32m 337\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m future\n\u001b[1;32m 338\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m--> 339\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43msync\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 340\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mloop\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunc\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcallback_timeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcallback_timeout\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\n\u001b[1;32m 341\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils.py:406\u001b[0m, in \u001b[0;36msync\u001b[0;34m(loop, func, callback_timeout, *args, **kwargs)\u001b[0m\n\u001b[1;32m 404\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m error:\n\u001b[1;32m 405\u001b[0m typ, exc, tb \u001b[38;5;241m=\u001b[39m error\n\u001b[0;32m--> 406\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\u001b[38;5;241m.\u001b[39mwith_traceback(tb)\n\u001b[1;32m 407\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 408\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m result\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils.py:379\u001b[0m, in \u001b[0;36msync..f\u001b[0;34m()\u001b[0m\n\u001b[1;32m 377\u001b[0m future \u001b[38;5;241m=\u001b[39m asyncio\u001b[38;5;241m.\u001b[39mwait_for(future, callback_timeout)\n\u001b[1;32m 378\u001b[0m future \u001b[38;5;241m=\u001b[39m asyncio\u001b[38;5;241m.\u001b[39mensure_future(future)\n\u001b[0;32m--> 379\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01myield\u001b[39;00m future\n\u001b[1;32m 380\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m:\n\u001b[1;32m 381\u001b[0m error \u001b[38;5;241m=\u001b[39m sys\u001b[38;5;241m.\u001b[39mexc_info()\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/tornado/gen.py:762\u001b[0m, in \u001b[0;36mRunner.run\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 759\u001b[0m exc_info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m 761\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 762\u001b[0m value \u001b[38;5;241m=\u001b[39m \u001b[43mfuture\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresult\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 763\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m:\n\u001b[1;32m 764\u001b[0m exc_info \u001b[38;5;241m=\u001b[39m sys\u001b[38;5;241m.\u001b[39mexc_info()\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/client.py:2186\u001b[0m, in \u001b[0;36mClient._gather\u001b[0;34m(self, futures, errors, direct, local_worker)\u001b[0m\n\u001b[1;32m 2184\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 2185\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_gather_future \u001b[38;5;241m=\u001b[39m future\n\u001b[0;32m-> 2186\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m future\n\u001b[1;32m 2188\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstatus\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124merror\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m 2189\u001b[0m log \u001b[38;5;241m=\u001b[39m logger\u001b[38;5;241m.\u001b[39mwarning \u001b[38;5;28;01mif\u001b[39;00m errors \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mraise\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m logger\u001b[38;5;241m.\u001b[39mdebug\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/client.py:2237\u001b[0m, in \u001b[0;36mClient._gather_remote\u001b[0;34m(self, direct, local_worker)\u001b[0m\n\u001b[1;32m 2234\u001b[0m response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdata\u001b[39m\u001b[38;5;124m\"\u001b[39m]\u001b[38;5;241m.\u001b[39mupdate(data2)\n\u001b[1;32m 2236\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m: \u001b[38;5;66;03m# ask scheduler to gather data for us\u001b[39;00m\n\u001b[0;32m-> 2237\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m retry_operation(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mscheduler\u001b[38;5;241m.\u001b[39mgather, keys\u001b[38;5;241m=\u001b[39mkeys)\n\u001b[1;32m 2239\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m response\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:400\u001b[0m, in \u001b[0;36mretry_operation\u001b[0;34m(coro, operation, *args, **kwargs)\u001b[0m\n\u001b[1;32m 394\u001b[0m retry_delay_min \u001b[38;5;241m=\u001b[39m parse_timedelta(\n\u001b[1;32m 395\u001b[0m dask\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdistributed.comm.retry.delay.min\u001b[39m\u001b[38;5;124m\"\u001b[39m), default\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 396\u001b[0m )\n\u001b[1;32m 397\u001b[0m retry_delay_max \u001b[38;5;241m=\u001b[39m parse_timedelta(\n\u001b[1;32m 398\u001b[0m dask\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdistributed.comm.retry.delay.max\u001b[39m\u001b[38;5;124m\"\u001b[39m), default\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 399\u001b[0m )\n\u001b[0;32m--> 400\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m retry(\n\u001b[1;32m 401\u001b[0m partial(coro, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs),\n\u001b[1;32m 402\u001b[0m count\u001b[38;5;241m=\u001b[39mretry_count,\n\u001b[1;32m 403\u001b[0m delay_min\u001b[38;5;241m=\u001b[39mretry_delay_min,\n\u001b[1;32m 404\u001b[0m delay_max\u001b[38;5;241m=\u001b[39mretry_delay_max,\n\u001b[1;32m 405\u001b[0m operation\u001b[38;5;241m=\u001b[39moperation,\n\u001b[1;32m 406\u001b[0m )\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:385\u001b[0m, in \u001b[0;36mretry\u001b[0;34m(coro, count, delay_min, delay_max, jitter_fraction, retry_on_exceptions, operation)\u001b[0m\n\u001b[1;32m 383\u001b[0m delay \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m1\u001b[39m \u001b[38;5;241m+\u001b[39m random\u001b[38;5;241m.\u001b[39mrandom() \u001b[38;5;241m*\u001b[39m jitter_fraction\n\u001b[1;32m 384\u001b[0m \u001b[38;5;28;01mawait\u001b[39;00m asyncio\u001b[38;5;241m.\u001b[39msleep(delay)\n\u001b[0;32m--> 385\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m coro()\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/core.py:1221\u001b[0m, in \u001b[0;36mPooledRPCCall.__getattr__..send_recv_from_rpc\u001b[0;34m(**kwargs)\u001b[0m\n\u001b[1;32m 1219\u001b[0m prev_name, comm\u001b[38;5;241m.\u001b[39mname \u001b[38;5;241m=\u001b[39m comm\u001b[38;5;241m.\u001b[39mname, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mConnectionPool.\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;241m+\u001b[39m key\n\u001b[1;32m 1220\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m-> 1221\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m send_recv(comm\u001b[38;5;241m=\u001b[39mcomm, op\u001b[38;5;241m=\u001b[39mkey, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[1;32m 1222\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m 1223\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mpool\u001b[38;5;241m.\u001b[39mreuse(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39maddr, comm)\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/core.py:1011\u001b[0m, in \u001b[0;36msend_recv\u001b[0;34m(comm, reply, serializers, deserializers, **kwargs)\u001b[0m\n\u001b[1;32m 1009\u001b[0m _, exc, tb \u001b[38;5;241m=\u001b[39m clean_exception(\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mresponse)\n\u001b[1;32m 1010\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m exc\n\u001b[0;32m-> 1011\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\u001b[38;5;241m.\u001b[39mwith_traceback(tb)\n\u001b[1;32m 1012\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 1013\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m(response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mexception_text\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/core.py:820\u001b[0m, in \u001b[0;36m_handle_comm\u001b[0;34m()\u001b[0m\n\u001b[1;32m 818\u001b[0m result \u001b[38;5;241m=\u001b[39m handler(\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mmsg)\n\u001b[1;32m 819\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m inspect\u001b[38;5;241m.\u001b[39miscoroutine(result):\n\u001b[0;32m--> 820\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m result\n\u001b[1;32m 821\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m inspect\u001b[38;5;241m.\u001b[39misawaitable(result):\n\u001b[1;32m 822\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\n\u001b[1;32m 823\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mComm handler returned unknown awaitable. Expected coroutine, instead got \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mtype\u001b[39m(result)\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 824\u001b[0m )\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/scheduler.py:5654\u001b[0m, in \u001b[0;36mgather\u001b[0;34m()\u001b[0m\n\u001b[1;32m 5651\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 5652\u001b[0m who_has[key] \u001b[38;5;241m=\u001b[39m []\n\u001b[0;32m-> 5654\u001b[0m data, missing_keys, missing_workers \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m gather_from_workers(\n\u001b[1;32m 5655\u001b[0m who_has, rpc\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mrpc, close\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m, serializers\u001b[38;5;241m=\u001b[39mserializers\n\u001b[1;32m 5656\u001b[0m )\n\u001b[1;32m 5657\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m missing_keys:\n\u001b[1;32m 5658\u001b[0m result \u001b[38;5;241m=\u001b[39m {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstatus\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mOK\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdata\u001b[39m\u001b[38;5;124m\"\u001b[39m: data}\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:80\u001b[0m, in \u001b[0;36mgather_from_workers\u001b[0;34m()\u001b[0m\n\u001b[1;32m 78\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m worker, c \u001b[38;5;129;01min\u001b[39;00m coroutines\u001b[38;5;241m.\u001b[39mitems():\n\u001b[1;32m 79\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m---> 80\u001b[0m r \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m c\n\u001b[1;32m 81\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mOSError\u001b[39;00m:\n\u001b[1;32m 82\u001b[0m missing_workers\u001b[38;5;241m.\u001b[39madd(worker)\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/worker.py:2870\u001b[0m, in \u001b[0;36mget_data_from_worker\u001b[0;34m()\u001b[0m\n\u001b[1;32m 2867\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m 2868\u001b[0m rpc\u001b[38;5;241m.\u001b[39mreuse(worker, comm)\n\u001b[0;32m-> 2870\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m retry_operation(_get_data, operation\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mget_data_from_worker\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:400\u001b[0m, in \u001b[0;36mretry_operation\u001b[0;34m()\u001b[0m\n\u001b[1;32m 394\u001b[0m retry_delay_min \u001b[38;5;241m=\u001b[39m parse_timedelta(\n\u001b[1;32m 395\u001b[0m dask\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdistributed.comm.retry.delay.min\u001b[39m\u001b[38;5;124m\"\u001b[39m), default\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 396\u001b[0m )\n\u001b[1;32m 397\u001b[0m retry_delay_max \u001b[38;5;241m=\u001b[39m parse_timedelta(\n\u001b[1;32m 398\u001b[0m dask\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdistributed.comm.retry.delay.max\u001b[39m\u001b[38;5;124m\"\u001b[39m), default\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 399\u001b[0m )\n\u001b[0;32m--> 400\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m retry(\n\u001b[1;32m 401\u001b[0m partial(coro, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs),\n\u001b[1;32m 402\u001b[0m count\u001b[38;5;241m=\u001b[39mretry_count,\n\u001b[1;32m 403\u001b[0m delay_min\u001b[38;5;241m=\u001b[39mretry_delay_min,\n\u001b[1;32m 404\u001b[0m delay_max\u001b[38;5;241m=\u001b[39mretry_delay_max,\n\u001b[1;32m 405\u001b[0m operation\u001b[38;5;241m=\u001b[39moperation,\n\u001b[1;32m 406\u001b[0m )\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:385\u001b[0m, in \u001b[0;36mretry\u001b[0;34m()\u001b[0m\n\u001b[1;32m 383\u001b[0m delay \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m1\u001b[39m \u001b[38;5;241m+\u001b[39m random\u001b[38;5;241m.\u001b[39mrandom() \u001b[38;5;241m*\u001b[39m jitter_fraction\n\u001b[1;32m 384\u001b[0m \u001b[38;5;28;01mawait\u001b[39;00m asyncio\u001b[38;5;241m.\u001b[39msleep(delay)\n\u001b[0;32m--> 385\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m coro()\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/worker.py:2850\u001b[0m, in \u001b[0;36m_get_data\u001b[0;34m()\u001b[0m\n\u001b[1;32m 2848\u001b[0m comm\u001b[38;5;241m.\u001b[39mname \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEphemeral Worker->Worker for gather\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 2849\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m-> 2850\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m send_recv(\n\u001b[1;32m 2851\u001b[0m comm,\n\u001b[1;32m 2852\u001b[0m serializers\u001b[38;5;241m=\u001b[39mserializers,\n\u001b[1;32m 2853\u001b[0m deserializers\u001b[38;5;241m=\u001b[39mdeserializers,\n\u001b[1;32m 2854\u001b[0m op\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mget_data\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m 2855\u001b[0m keys\u001b[38;5;241m=\u001b[39mkeys,\n\u001b[1;32m 2856\u001b[0m who\u001b[38;5;241m=\u001b[39mwho,\n\u001b[1;32m 2857\u001b[0m max_connections\u001b[38;5;241m=\u001b[39mmax_connections,\n\u001b[1;32m 2858\u001b[0m )\n\u001b[1;32m 2859\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m 2860\u001b[0m status \u001b[38;5;241m=\u001b[39m response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstatus\u001b[39m\u001b[38;5;124m\"\u001b[39m]\n", + "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/core.py:1013\u001b[0m, in \u001b[0;36msend_recv\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1011\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\u001b[38;5;241m.\u001b[39mwith_traceback(tb)\n\u001b[1;32m 1012\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m-> 1013\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m(response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mexception_text\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n\u001b[1;32m 1014\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m response\n", + "\u001b[0;31mException\u001b[0m: AssertionError()" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-11-14 15:29:39,214 - distributed.worker - ERROR - \n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 2)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "2024-11-14 15:29:39,222 - tornado.application - ERROR - Exception in callback functools.partial(>, exception=AssertionError()>)\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 2)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", + " ret = callback()\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", + " future.result()\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", + " return await method(self, *args, **kwargs) # type: ignore\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", + " await self.handle_stream(comm)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", + " handler(**merge(extra, msg))\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", + " self.handle_stimulus(event)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "unhandled exception during asyncio.run() shutdown\n", + "task: exception=AssertionError()>\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 2)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", + " ret = callback()\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", + " future.result()\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", + " return await method(self, *args, **kwargs) # type: ignore\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", + " await self.handle_stream(comm)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", + " handler(**merge(extra, msg))\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", + " self.handle_stimulus(event)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "2024-11-14 15:29:39,351 - distributed.worker - ERROR - \n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 3)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "2024-11-14 15:29:39,357 - tornado.application - ERROR - Exception in callback functools.partial(>, exception=AssertionError()>)\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 3)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", + " ret = callback()\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", + " future.result()\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", + " return await method(self, *args, **kwargs) # type: ignore\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", + " await self.handle_stream(comm)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", + " handler(**merge(extra, msg))\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", + " self.handle_stimulus(event)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "unhandled exception during asyncio.run() shutdown\n", + "task: exception=AssertionError()>\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 3)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", + " ret = callback()\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", + " future.result()\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", + " return await method(self, *args, **kwargs) # type: ignore\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", + " await self.handle_stream(comm)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", + " handler(**merge(extra, msg))\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", + " self.handle_stimulus(event)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "2024-11-14 15:29:39,666 - distributed.worker - ERROR - \n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 4, 3)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "2024-11-14 15:29:40,022 - tornado.application - ERROR - Exception in callback functools.partial(>, exception=AssertionError()>)\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 4, 3)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", + " ret = callback()\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", + " future.result()\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", + " return await method(self, *args, **kwargs) # type: ignore\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", + " await self.handle_stream(comm)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", + " handler(**merge(extra, msg))\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", + " self.handle_stimulus(event)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "unhandled exception during asyncio.run() shutdown\n", + "task: exception=AssertionError()>\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", + " return self.fast[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", + " result = self.d[key]\n", + "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 4, 3)\"\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", + " ret = callback()\n", + " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", + " future.result()\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", + " return await method(self, *args, **kwargs) # type: ignore\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", + " await self.handle_stream(comm)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", + " handler(**merge(extra, msg))\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", + " self.handle_stimulus(event)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", + " return method(self, *args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", + " super().handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", + " instructions = self.state.handle_stimulus(*stims)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", + " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", + " process_recs(recommendations.copy())\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", + " a_recs, a_instructions = self._transition(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", + " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", + " recs, instructions = self._transition_generic_released(\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", + " self._purge_state(ts)\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", + " self.data.pop(key, None)\n", + " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", + " value = self[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", + " return super().__getitem__(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", + " return self.slow_to_fast(key)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", + " value = self.slow[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", + " return func(*args, **kwargs)\n", + " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", + " value = self.data[key]\n", + " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", + " assert isinstance(pickled, bytes)\n", + "AssertionError\n", + "2024-11-14 15:29:41,223 - distributed.nanny - ERROR - Worker process died unexpectedly\n" ] } ], @@ -1019,25 +1449,25 @@ " return (p2-p1) / (p2+p1) \n", "\n", "@njit\n", - "def compute_ndvi(input_data_1: np.ndarray,input_data_2: np.ndarray):\n", + "def compute_ndvi_numba(input_data_1: np.ndarray,input_data_2: np.ndarray):\n", " #ndvi_array = [one_pixel_ndvi(i,j) for i in input_data_1 for j in input_data_2]\n", " ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n", " return ndvi_array\n", "\n", "def compute_ndvi_dask(input_data_1: DataArray,input_data_2: DataArray):\n", - " ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n", + " ndvi_array = ((input_data_2 - input_data_1) / (input_data_2 + input_data_1))[None,:,:]\n", " ndvi_array.compute()\n", " return ndvi_array\n", "\n", - "\n", "def compute_ndvi_std(input_data_1: np.ndarray,input_data_2: np.ndarray):\n", " #ndvi_array = [one_pixel_ndvi(i,j) for i in input_data_1 for j in input_data_2]\n", " ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n", " return ndvi_array\n", "\n", - "sentinel_2_dir = \"/home/tromain/Data/S2\"\n", + "sentinel_2_dir = \"/work/scratch/data/romaint\"\n", "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", + "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\" \n", "\n", "start = time.perf_counter()\n", "with rasterio.open(s2_b4, 'r') as ds:\n", @@ -1045,12 +1475,11 @@ "\n", "with rasterio.open(s2_b8, 'r') as ds:\n", " input_data_b8 = ds.read()\n", - "\n", - "ndvi_computed = compute_ndvi(input_data_b4,input_data_b8)\n", + " \n", + "ndvi_computed = compute_ndvi_numba(input_data_b4,input_data_b8)\n", "crs=\"EPSG:4326\"\n", - "output_file = Path(\"/home/tromain/Data/S2/ndvi_numba.tif\")\n", - "create_raster(ndvi_computed, output_file, x_res , y_res,\n", - " top_left_x, top_left_y, crs)\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_numba.tif\")\n", + "#create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n", "end = time.perf_counter()\n", "print(\"Elapsed with Raster IO + numba = {}s\".format((end - start)))\n", "\n", @@ -1063,116 +1492,1292 @@ "\n", "ndvi_computed = compute_ndvi_std(input_data_b4,input_data_b8)\n", "crs=\"EPSG:4326\"\n", - "output_file = Path(\"/home/tromain/Data/S2/ndvi_without_numba.tif\")\n", - "create_raster(ndvi_computed, output_file, x_res , y_res,\n", - " top_left_x, top_left_y, crs)\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_without_numba.tif\")\n", + "#create_raster(ndvi_computed, output_file, x_res , y_res,top_left_x, top_left_y, crs)\n", "end = time.perf_counter()\n", "print(\"Elapsed with Raster IO + without numba = {}s\".format((end - start)))\n", "# Example with xarray\n", - "#input_data_b4 = xarray.open_dataarray(s2_b4,chunks=False)\n", - "#input_data_b8 = xarray.open_dataarray(s2_b8,chunks=False)\n", - "#start = time.perf_counter()\n", - "#input_data_b4.load()\n", - "#input_data_b8.load()\n", - "#ndvi_computed = xarray.apply_ufunc(compute_ndvi_dask,input_data_b4,input_data_b8)\n", - "#compute_ndvi(input_data_b4.to_numpy(),input_data_b8.to_numpy())\n", - "#end = time.perf_counter()\n", - "#print(\"Elapsed with Xarray + numba = {}s\".format((end - start)))\n", - "\n", - "# Example with rioxarray \n", "start = time.perf_counter()\n", - "input_data_b4 = rxr.open_rasterio(s2_b4,chunks=True)\n", - "input_data_b8 = rxr.open_rasterio(s2_b8,chunks=True)\n", - "#(-1,2200,2200)\n", - "ndvi_computed = compute_ndvi_dask(input_data_b4,input_data_b8)\n", - "output_file = Path(\"/home/tromain/Data/S2/ndvi_rioxarray.tif\")\n", - "create_raster(np.asarray(ndvi_computed), output_file, x_res , y_res,\n", - " top_left_x, top_left_y, crs)\n", + "input_data_b4 = xarray.open_dataarray(s2_b4)\n", + "input_data_b8 = xarray.open_dataarray(s2_b8)\n", + "ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n", + "ndvi_computed.rio.to_raster(\"/work/scratch/data/romaint/output_greenit/ndvi_ufunc.tif\")\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_ufunc.tif\")\n", "end = time.perf_counter()\n", - "\n", - "print(\"Elapsed with RIOXarray + dask + function call = {}s\".format((end - start)))\n", + "print(\"Elapsed with Xarray + apply ufunc = {}s\".format((end - start)))\n", "\n", "start = time.perf_counter()\n", - "ndvi_array = [(j-i)/(i+j) for i in input_data_b4 for j in input_data_b8]\n", - "dsk_array = da.asarray(ndvi_array)\n", - "dsk_array.visualize(\"/home/tromain/Data/S2/dsk_array.png\")\n", - "crs=\"EPSG:4326\"\n", - "output_file = Path(\"/home/tromain/Data/S2/ndvi_list_comp.tif\")\n", - "create_raster(np.asarray(ndvi_array), output_file, x_res , y_res,\n", - " top_left_x, top_left_y, crs)\n", + "input_data_b4 = rxr.open_rasterio(s2_b4,chunks=True)\n", + "input_data_b8 = rxr.open_rasterio(s2_b8,chunks=True)\n", + "ndvi_array = (input_data_b8 - input_data_b4) / (input_data_b8 + input_data_b4)\n", + "ndvi_array.compute()\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_std.tif\")\n", + "ndvi_array.rio.to_raster(output_file)\n", "end = time.perf_counter()\n", + "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n", "\n", - "print(\"Elapsed with RIOXarray + dask + list comprehension = {}s\".format((end - start)))\n", + "#phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n", + "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", "start = time.perf_counter()\n", - "ndvi_array = (input_data_b8 - input_data_b4) / (input_data_b8 + input_data_b4)\n", - "ndvi_array.compute()\n", - "ndvi_array.data.visualize(\"/home/tromain/Data/S2/visu_std.png\")\n", - "output_file = Path(\"/home/tromain/Data/S2/ndvi_std.tif\")\n", - "create_raster(np.asarray(ndvi_array), output_file, x_res , y_res,\n", - " top_left_x, top_left_y, crs)\n", + "reading_chunks = True\n", + "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n", + "print(input_data_array.shape)\n", + "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[0] + input_data_array[3])\n", + "ndvi_phr.compute()\n", + "print(ndvi_phr.shape)\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n", + "ndvi_phr.rio.to_raster(output_file,tiled=True)\n", "end = time.perf_counter()\n", - "print(\"Elapsed with RIOXarray + dask + standard compute = {}s\".format((end - start)))\n" + "print(\"Elapsed with RIOXarray + dask + big product = {}s\".format((end - start)))" ] }, { - "cell_type": "markdown", - "id": "6ffcc752-1d32-42f0-86f6-de25ce216688", - "metadata": {}, + "cell_type": "code", + "execution_count": null, + "id": "bfa94789-8576-4b26-a962-226db7d37f01", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(4, 41663, 39844)\n", + "(41663, 39844)\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n", + " dataset = writer(\n", + "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n", + " return self.func(*new_argspec)\n", + "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n", + " return self.func(*new_argspec)\n", + "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n", + " return self.func(*new_argspec)\n", + "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n", + " return self.func(*new_argspec)\n" + ] + } + ], "source": [ - "# Performances multiprocessing vs multithreading\n", + "%%time\n", + "\n", + "from numba import jit,njit\n", + "from xarray import DataArray \n", + "from typing import List, Tuple, Union, Dict\n", + "import rioxarray as rxr\n", + "import numpy as np\n", + "import time\n", + "import rasterio\n", + "from pathlib import Path\n", + "import xarray\n", "\n", - "Depending on the processing applied to an image it is interesting to use the multiprocessing approach instead of the multithreading which is not really a multithreading in python as every thread is executed one after the other. (in Python 3.13, the GIL has been reworked to improve the situation)\n", - "\n" + "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", + "start = time.perf_counter()\n", + "reading_chunks = True\n", + "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n", + "print(input_data_array.shape)\n", + "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[0] + input_data_array[3])\n", + "#ndvi_phr.compute()\n", + "print(ndvi_phr.shape)\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n", + "ndvi_phr.rio.to_raster(output_file,tiled=True)\n", + "end = time.perf_counter()\n", + "print(\"Elapsed with RIOXarray + dask + big product = {}s\".format((end - start)))" ] }, { "cell_type": "code", - "execution_count": null, - "id": "5bd9b9de-33df-48d7-b643-9e767bbc9d2a", + "execution_count": 4, + "id": "4228aa64-b886-43eb-b4b0-2995df37c0e8", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
<xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
+       "dask.array<open_rasterio-fc2e18048eec7fd8788f8fc93c7ebea6<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n",
+       "Coordinates:\n",
+       "  * band         (band) int64 32B 1 2 3 4\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0\n",
+       "Attributes: (12/29)\n",
+       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
+       "    BlueDisplayChannel:   0\n",
+       "    DataType:             3\n",
+       "    GeometricLevel:       SENSOR\n",
+       "    GreenDisplayChannel:  0\n",
+       "    ImageID:              6967638101-1\n",
+       "    ...                   ...\n",
+       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
+       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
+       "    NoData:               0\n",
+       "    _FillValue:           0\n",
+       "    scale_factor:         1.0\n",
+       "    add_offset:           0.0
" + ], + "text/plain": [ + " Size: 13GB\n", + "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n", + "Coordinates:\n", + " * band (band) int64 32B 1 2 3 4\n", + " * x (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n", + " * y (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n", + " spatial_ref int64 8B 0\n", + "Attributes: (12/29)\n", + " AcquisitionDate: 2023-04-15T11:00:24.3Z\n", + " BlueDisplayChannel: 0\n", + " DataType: 3\n", + " GeometricLevel: SENSOR\n", + " GreenDisplayChannel: 0\n", + " ImageID: 6967638101-1\n", + " ... ...\n", + " TimeRangeEnd: 2023-04-15T11:00:27.3815980Z\n", + " TimeRangeStart: 2023-04-15T11:00:24.3193675Z\n", + " NoData: 0\n", + " _FillValue: 0\n", + " scale_factor: 1.0\n", + " add_offset: 0.0" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "input_data_array" + ] + }, + { + "cell_type": "markdown", + "id": "f7746770-c831-4598-96ff-f13e1d372e1d", "metadata": {}, + "source": [ + "# Parallel write" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "4a41f37f-db1b-462b-b990-5bf0db8e1c84", + "metadata": { + "tags": [] + }, "outputs": [], "source": [ - "import multiprocessing\n", - "from xarray import DataArray\n", + "from xarray import DataArray \n", + "from typing import List, Tuple, Union, Dict\n", "import rioxarray as rxr\n", + "import numpy as np\n", "import time\n", - " \n", - "def task_ndvi(input_data_1: DataArray,input_data_2: DataArray):\n", - " ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n", - " return ndvi_array\n", - " \n", - "if __name__ == \"__main__\":\n", - " sentinel_2_dir = \"/home/tromain/Data/S2\"\n", - " s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", - " s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", - " with rasterio.open(s2_b4, 'r') as ds:\n", - " input_data_b4 = ds.read() \n", - "\n", - " with rasterio.open(s2_b8, 'r') as ds:\n", - " input_data_b8 = ds.read()\n", + "import rasterio\n", + "from pathlib import Path\n", + "import xarray\n", "\n", - " input_data_b4.load()\n", - " input_data_b8.load()\n", - " \n", - " start_time = time.perf_counter()\n", - " \n", - " # Creates two processes\n", - " p1 = multiprocessing.Process(target=task_ndvi)\n", - " p2 = multiprocessing.Process(target=task_ndvi)\n", - " p3 = multiprocessing.Process(target=task_ndvi)\n", - " p4 = multiprocessing.Process(target=task_ndvi)\n", - " \n", - " # Starts both processes\n", - " p1.start()\n", - " p2.start()\n", - " p3.start()\n", - " p4.start()\n", - " \n", - " finish_time = time.perf_counter()\n", - " \n", - " print(f\"Program finished in {finish_time-start_time} seconds\")\n" + "#phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", + "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n", + "reading_chunks = (-1,8192,8192)\n", + "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks)\n", + "#input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product], reading_chunks)" ] }, + { + "cell_type": "code", + "execution_count": 6, + "id": "81a22b13-3bff-45a9-b840-03d070bef283", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
<xarray.DataArray (band: 4, y: 41663, x: 39844)>\n",
+       "dask.array<open_rasterio-3fec247de0a7ca9c3f643f1ebfebdd92<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 8192, 8192), chunktype=numpy.ndarray>\n",
+       "Coordinates:\n",
+       "  * band         (band) int64 1 2 3 4\n",
+       "  * x            (x) float64 0.5 1.5 2.5 3.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 0.5 1.5 2.5 3.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 0\n",
+       "Attributes: (12/29)\n",
+       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
+       "    BlueDisplayChannel:   0\n",
+       "    DataType:             3\n",
+       "    GeometricLevel:       SENSOR\n",
+       "    GreenDisplayChannel:  0\n",
+       "    ImageID:              6967638101-1\n",
+       "    ...                   ...\n",
+       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
+       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
+       "    NoData:               0\n",
+       "    _FillValue:           0\n",
+       "    scale_factor:         1.0\n",
+       "    add_offset:           0.0
" + ], + "text/plain": [ + "\n", + "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 8192, 8192), chunktype=numpy.ndarray>\n", + "Coordinates:\n", + " * band (band) int64 1 2 3 4\n", + " * x (x) float64 0.5 1.5 2.5 3.5 ... 3.984e+04 3.984e+04 3.984e+04\n", + " * y (y) float64 0.5 1.5 2.5 3.5 ... 4.166e+04 4.166e+04 4.166e+04\n", + " spatial_ref int64 0\n", + "Attributes: (12/29)\n", + " AcquisitionDate: 2023-04-15T11:00:24.3Z\n", + " BlueDisplayChannel: 0\n", + " DataType: 3\n", + " GeometricLevel: SENSOR\n", + " GreenDisplayChannel: 0\n", + " ImageID: 6967638101-1\n", + " ... ...\n", + " TimeRangeEnd: 2023-04-15T11:00:27.3815980Z\n", + " TimeRangeStart: 2023-04-15T11:00:24.3193675Z\n", + " NoData: 0\n", + " _FillValue: 0\n", + " scale_factor: 1.0\n", + " add_offset: 0.0" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "input_data_array" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "909e5746-eee9-4f0c-a840-d3e8850dee83", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(41663, 39844)\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:314: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n", + " dataset = writer(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Elapsed with RIOXarray + dask + big product with write = 106.9249369930476s\n" + ] + } + ], + "source": [ + "start = time.perf_counter()\n", + "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n", + "ndvi_phr.compute()\n", + "print(ndvi_phr.shape)\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n", + "#create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n", + "ndvi_phr.rio.to_raster(output_file)\n", + "end = time.perf_counter()\n", + "print(\"Elapsed with RIOXarray + dask + big product with write = {}s\".format((end - start)))\n" + ] + }, + { + "cell_type": "markdown", + "id": "ffd1783b-a0e3-43f2-861d-13d043653dc6", + "metadata": {}, + "source": [ + "# Conclusions and recommandations about parallel write\n" + ] + }, + { + "cell_type": "markdown", + "id": "6ffcc752-1d32-42f0-86f6-de25ce216688", + "metadata": {}, + "source": [ + "# Performances multiprocessing example\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5bd9b9de-33df-48d7-b643-9e767bbc9d2a", + "metadata": {}, + "outputs": [], + "source": [] + }, { "cell_type": "markdown", "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43", @@ -1192,12 +2797,37 @@ ] }, { - "cell_type": "code", - "execution_count": null, - "id": "b4cbef49-ee8d-4948-b5c9-1a958edcc8d4", + "cell_type": "markdown", + "id": "97205c02-6fc9-42f3-83ed-c8fd4af704af", "metadata": {}, - "outputs": [], - "source": [] + "source": [ + "# Estimate the carbon impact of your code\n", + "Using code carbon you can have an estimate of your code footprint" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "ee231cc6-e45c-4209-9135-8c80e1a4eaf4", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "ename": "ModuleNotFoundError", + "evalue": "No module named 'codecarbon'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[6], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mcodecarbon\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m track_emissions\n", + "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'codecarbon'" + ] + } + ], + "source": [ + "from codecarbon import track_emissions\n" + ] } ], "metadata": { @@ -1216,7 +2846,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.10" + "version": "3.10.12" } }, "nbformat": 4, From 0dba817c6e3fe320a63ebdc8ccd4cffc60bb9f05 Mon Sep 17 00:00:00 2001 From: Thibaut ROMAIN Date: Tue, 7 Jan 2025 16:42:32 +0100 Subject: [PATCH 05/37] Add good practices about for loops and list search --- tuto_greenit.ipynb | 139 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 126 insertions(+), 13 deletions(-) diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb index 31e1c74..107b367 100644 --- a/tuto_greenit.ipynb +++ b/tuto_greenit.ipynb @@ -2,33 +2,146 @@ "cells": [ { "cell_type": "markdown", - "id": "371254df-a3b4-4f3e-b34f-3c3db06828ba", + "id": "3c6a05a0-c506-43e1-a352-c5e8e0d6f36e", "metadata": {}, "source": [ - "# Reading rasters with rasterio\n" + "# Good pratices for loop optimisations\n", + "\n", + "using a for loop vs numpy for basic loop coding shows that you must avoid for loops in your code, here is a code snippet that shows the big performance difference to apply a square on 250 000 elements" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "cecef79a-2ada-4df8-bde5-abb60db8a037", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Time with for loop : 83.00238009542227 ms\n", + "Time with for list comprehension : 63.063559122383595 ms\n", + "Time with numpy : 0.2981601282954216 ms\n" + ] + } + ], + "source": [ + "import time\n", + "import numpy as np\n", + "\n", + "start = time.perf_counter()\n", + "# Calculate square with for loop\n", + "array_forloop = []\n", + "for i in range(0,500):\n", + " for j in range(0,500):\n", + " array_forloop.append((i * j) ** 2)\n", + " \n", + "end = time.perf_counter()\n", + "print(\"Time with for loop : {} ms\".format((end-start)*1000))\n", + "\n", + "# Using list comprehension instead of a for loop is a nice optimisation if your use case needs a for loop anyway\n", + "start = time.perf_counter()\n", + "array_forloop = [i**2 for i in range(0,250000)]\n", + "end = time.perf_counter()\n", + "print(\"Time with for list comprehension : {} ms\".format((end-start)*1000))\n", + " \n", + "# Using numpy\n", + "start = time.perf_counter()\n", + "arr = np.arange(250000)\n", + "squared_arr = np.square(arr)\n", + "end = time.perf_counter()\n", + "print(\"Time with numpy : {} ms\".format((end-start)*1000))" ] }, { "cell_type": "markdown", - "id": "fb2fcaf9-5b7b-4f37-b457-6e01e5035fbf", - "metadata": {}, + "id": "452e6bf2-d7f3-4914-a5f5-3e9d017314d2", + "metadata": { + "tags": [] + }, "source": [ - "# Downscaling rasters thanks to dask\n", + "Conclusion\n", "\n", - "In this notebook we will look at a concrete case of data aggregation, with the example of changing the resolution using dask.\n", + "* Avoid using for loops to apply a computing on a bunch of pixels, use numpy instead\n", + "* If you need a for loop for another processing, use list comprehension as much as possible, it is very well documented \n", "\n", - "Dask is a Python library that enables calculations to be parallelized and large quantities of data to be handled in a scalable way, using available resources (CPU, memory, etc.). Unlike tools such as Pandas or NumPy, Dask allows you to work on data sets that exceed the available RAM memory by chunking the data into smaller pieces (chunks) and parallelizing calculations.\n", - "Dask works with lazy computation: instead of executing immediately, it builds a task graph. Calculations are only executed when the final result is explicitly requested (for example, by calling .compute()).\n", + "For an in-depth comparison of how much numpy is faster than for and while loops, this link provides a complete performance comparison : https://www.blog.duomly.com/loops-in-python-comparison-and-performance/" + ] + }, + { + "cell_type": "markdown", + "id": "1b0b3e1a-d31c-43c3-a662-720b0e603f14", + "metadata": {}, + "source": [ + "# Good pratices for searching informations\n", + "\n", + "If your code needs to search elements in an image for example looking at NoData pixels, it is highly recommended to do that search in a set instead of a list of pixels :" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "35d9ffa0-a498-4ab5-ae73-47cb3fb70eb1", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "List search time: 42.37660344410688 s\n", + "Set search time: 0.0062899962067604065 s\n" + ] + } + ], + "source": [ + "from timeit import timeit\n", "\n", - "Dask offers a wide range of [modules](https://docs.dask.org/en/stable/#how-to-use-dask) (dask dataframe...) that can be used to distribute calculations. In this notebook we will focus on the use of [dask-arrays](https://docs.dask.org/en/stable/array.html) to manipulate raster data.\n", + "nd_list = [i for i in range(0,100000)]\n", + "nd_set = set([i for i in range(0,100000)])\n", "\n", - "In order to change the resolution, we will look at two possible methods. One of them will prove less effective than the other, which will allow us to establish some general principles to follow for optimal use of Dask. The first one will use [map_blocks](https://docs.dask.org/en/stable/generated/dask.array.map_blocks.html) and the second one only [dask.array.mean](https://docs.dask.org/en/stable/generated/dask.array.mean.html)\n", + "#Look at an element\n", + "def search_list():\n", + " if 98950 in nd_list:\n", + " pass\n", + " \n", + "#Look at the same element in the set\n", + "def search_set():\n", + " if 98950 in nd_set:\n", + " pass\n", "\n", - "For this tutorial, we will use a Sentinel-2 acquisition stored on the local disk. We will target the storage directory under the variable ``sentinel_2_dir``. Users can modify this directory and the associated paths.\n", + "#lets say nodata value is 98950 in that case\n", + "t1 = timeit(search_list, number=100000)\n", + "t2 = timeit(search_set, number=100000)\n", + "print(\"List search time: {} s\".format(t1))\n", + "print(\"Set search time: {} s\".format(t2))" + ] + }, + { + "cell_type": "markdown", + "id": "371254df-a3b4-4f3e-b34f-3c3db06828ba", + "metadata": { + "tags": [] + }, + "source": [ + "# A Green IT approach for coding image processing chains\n" + ] + }, + { + "cell_type": "markdown", + "id": "fb2fcaf9-5b7b-4f37-b457-6e01e5035fbf", + "metadata": {}, + "source": [ + "## Downscaling rasters thanks to dask\n", "\n", - "## Python scripts\n", + "In this notebook we will look at a green IT approach of coding, via good pratices in Python. We will have a look at Dask, RioXarray, rasterio, numpy... all these libraries that are widely used in the satellite image processing chains. \n", + "In order to use dask we first need to create a local cluster.\n", "\n", - "### Import libraries\n", + "## Import libraries\n", "\n", "First let's import libraries needed for this tutorial and create our dask [LocalCluster](https://docs.dask.org/en/stable/deploying-python.html#localcluster) which allow us to create workers and use [dask's dashboard](https://docs.dask.org/en/latest/dashboard.html).\n" ] From f2972d28e3ca216fea4a3358ac5716fc6f757d8c Mon Sep 17 00:00:00 2001 From: Thibaut ROMAIN Date: Wed, 8 Jan 2025 16:28:18 +0100 Subject: [PATCH 06/37] Add an example of chunk size computation --- tuto_greenit.ipynb | 2347 ++++++++++++++++++++++++-------------------- 1 file changed, 1302 insertions(+), 1045 deletions(-) diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb index 107b367..a59a49d 100644 --- a/tuto_greenit.ipynb +++ b/tuto_greenit.ipynb @@ -5,7 +5,7 @@ "id": "3c6a05a0-c506-43e1-a352-c5e8e0d6f36e", "metadata": {}, "source": [ - "# Good pratices for loop optimisations\n", + "# Good pratices for loop optimisations : benefits of using numpy\n", "\n", "using a for loop vs numpy for basic loop coding shows that you must avoid for loops in your code, here is a code snippet that shows the big performance difference to apply a square on 250 000 elements" ] @@ -83,7 +83,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 29, "id": "35d9ffa0-a498-4ab5-ae73-47cb3fb70eb1", "metadata": { "tags": [] @@ -93,16 +93,19 @@ "name": "stdout", "output_type": "stream", "text": [ - "List search time: 42.37660344410688 s\n", - "Set search time: 0.0062899962067604065 s\n" + "List search time: 42.939479635097086 s\n", + "Set search time: 0.006291795987635851 s\n", + "Numpy search time: 4.1151342540979385 s\n" ] } ], "source": [ "from timeit import timeit\n", + "import numpy as np\n", "\n", "nd_list = [i for i in range(0,100000)]\n", "nd_set = set([i for i in range(0,100000)])\n", + "nd_array = np.arange(0,100000)\n", "\n", "#Look at an element\n", "def search_list():\n", @@ -114,11 +117,29 @@ " if 98950 in nd_set:\n", " pass\n", "\n", + "def search_numpy():\n", + " if np.where(nd_array==98950):\n", + " pass\n", + "\n", "#lets say nodata value is 98950 in that case\n", "t1 = timeit(search_list, number=100000)\n", "t2 = timeit(search_set, number=100000)\n", + "t3 = timeit(search_numpy, number=100000)\n", "print(\"List search time: {} s\".format(t1))\n", - "print(\"Set search time: {} s\".format(t2))" + "print(\"Set search time: {} s\".format(t2))\n", + "print(\"Numpy search time: {} s\".format(t3))" + ] + }, + { + "cell_type": "markdown", + "id": "0d276261-97a0-4123-8b69-b28aca357328", + "metadata": { + "tags": [] + }, + "source": [ + "Conclusion\n", + "\n", + "Use sets when you need to lookup for values in your image but beware that the set is not intended to be modified. You can alternatively use numpy.where but it is not as efficient as the set because it returns a boolean array for each element of the array, nevertheless it is 10 times more efficient than a list." ] }, { @@ -141,14 +162,14 @@ "In this notebook we will look at a green IT approach of coding, via good pratices in Python. We will have a look at Dask, RioXarray, rasterio, numpy... all these libraries that are widely used in the satellite image processing chains. \n", "In order to use dask we first need to create a local cluster.\n", "\n", - "## Import libraries\n", + "### Creating the Dask Cluster\n", "\n", "First let's import libraries needed for this tutorial and create our dask [LocalCluster](https://docs.dask.org/en/stable/deploying-python.html#localcluster) which allow us to create workers and use [dask's dashboard](https://docs.dask.org/en/latest/dashboard.html).\n" ] }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 1, "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21", "metadata": { "tags": [] @@ -160,7 +181,7 @@ "text": [ "/usr/local/lib/python3.10/dist-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n", "Perhaps you already have a cluster running?\n", - "Hosting the HTTP server on port 36839 instead\n", + "Hosting the HTTP server on port 40751 instead\n", " warnings.warn(\n" ] }, @@ -168,7 +189,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Dask Dashboard: http://127.0.0.1:36839/status\n" + "Dask Dashboard: http://127.0.0.1:40751/status\n" ] }, { @@ -178,7 +199,7 @@ "
\n", "
\n", "

Client

\n", - "

Client-70dfeac2-bd46-11ef-9045-00620ba454ee

\n", + "

Client-28da73fa-cdcc-11ef-95b3-00620ba4892c

\n", " \n", "\n", " \n", @@ -191,7 +212,7 @@ " \n", " \n", " \n", " \n", " \n", @@ -200,7 +221,7 @@ "
\n", - " Dashboard: http://127.0.0.1:36839/status\n", + " Dashboard: http://127.0.0.1:40751/status\n", "
\n", "\n", " \n", - " \n", " \n", @@ -213,11 +234,11 @@ "
\n", "
\n", "

LocalCluster

\n", - "

02f9fee4

\n", + "

ef4c3e91

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", @@ -250,11 +271,11 @@ "
\n", "
\n", "

Scheduler

\n", - "

Scheduler-7238732b-a7d6-43b3-99a6-b17a85b0dcd9

\n", + "

Scheduler-1139ebda-c011-43bf-8799-43276f0dfdb0

\n", "
\n", - " Dashboard: http://127.0.0.1:36839/status\n", + " Dashboard: http://127.0.0.1:40751/status\n", " \n", " Workers: 4\n", @@ -225,10 +246,10 @@ "
\n", - " Total threads: 4\n", + " Total threads: 8\n", " \n", - " Total memory: 28.00 GiB\n", + " Total memory: 56.00 GiB\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", @@ -273,7 +294,7 @@ " Started: Just now\n", " \n", " \n", " \n", "
\n", - " Comm: tcp://127.0.0.1:33599\n", + " Comm: tcp://127.0.0.1:42135\n", " \n", " Workers: 4\n", @@ -262,10 +283,10 @@ "
\n", - " Dashboard: http://127.0.0.1:36839/status\n", + " Dashboard: http://127.0.0.1:40751/status\n", " \n", - " Total threads: 4\n", + " Total threads: 8\n", "
\n", - " Total memory: 28.00 GiB\n", + " Total memory: 56.00 GiB\n", "
\n", @@ -296,29 +317,29 @@ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -341,29 +362,29 @@ "
\n", - " Comm: tcp://127.0.0.1:35379\n", + " Comm: tcp://127.0.0.1:39021\n", " \n", - " Total threads: 1\n", + " Total threads: 2\n", "
\n", - " Dashboard: http://127.0.0.1:42093/status\n", + " Dashboard: http://127.0.0.1:44165/status\n", " \n", - " Memory: 7.00 GiB\n", + " Memory: 14.00 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:43625\n", + " Nanny: tcp://127.0.0.1:42625\n", "
\n", - " Local directory: /tmp/slurm-28574237/dask-scratch-space/worker-15rnueb2\n", + " Local directory: /tmp/slurm-29119419/dask-scratch-space/worker-p7_isrec\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -386,29 +407,29 @@ "
\n", - " Comm: tcp://127.0.0.1:34907\n", + " Comm: tcp://127.0.0.1:35287\n", " \n", - " Total threads: 1\n", + " Total threads: 2\n", "
\n", - " Dashboard: http://127.0.0.1:43573/status\n", + " Dashboard: http://127.0.0.1:39989/status\n", " \n", - " Memory: 7.00 GiB\n", + " Memory: 14.00 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:34007\n", + " Nanny: tcp://127.0.0.1:45251\n", "
\n", - " Local directory: /tmp/slurm-28574237/dask-scratch-space/worker-p8x8_6ao\n", + " Local directory: /tmp/slurm-29119419/dask-scratch-space/worker-lm1_6txw\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -431,29 +452,29 @@ "
\n", - " Comm: tcp://127.0.0.1:44905\n", + " Comm: tcp://127.0.0.1:44657\n", " \n", - " Total threads: 1\n", + " Total threads: 2\n", "
\n", - " Dashboard: http://127.0.0.1:36541/status\n", + " Dashboard: http://127.0.0.1:33045/status\n", " \n", - " Memory: 7.00 GiB\n", + " Memory: 14.00 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:34321\n", + " Nanny: tcp://127.0.0.1:37065\n", "
\n", - " Local directory: /tmp/slurm-28574237/dask-scratch-space/worker-a5wqbi7k\n", + " Local directory: /tmp/slurm-29119419/dask-scratch-space/worker-2j7v2i0j\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", @@ -480,10 +501,10 @@ "" ], "text/plain": [ - "" + "" ] }, - "execution_count": 8, + "execution_count": 1, "metadata": {}, "output_type": "execute_result" } @@ -522,12 +543,36 @@ "From this method we will use the ``chunks`` and ``lock`` arguments, which respectively set a chunk size and limit access to the data to one thread at a time to avoid read problems. Here ``chunks`` is set to ``True`` to allow dask to automatically size chunks.\n" ] }, + { + "cell_type": "markdown", + "id": "200d60a9-7747-439c-a4fb-e82516575c6b", + "metadata": {}, + "source": [ + "## Work directories" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "sentinel_2_dir = \"/work/scratch/data/romaint\"\n", + "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", + "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", + "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n", + "phr_product_cog = \"/work/scratch/data/romaint/phr_cog.tif\"" + ] + }, { "cell_type": "markdown", "id": "0e86755c-f505-4d6b-830e-396485e12054", "metadata": {}, "source": [ - "## Calculation of the Average NDVI with dask\n", + "# Calculation of the Average NDVI on a satellite image\n", "\n", "In this example, we will use what we have learned to: \n", "\n", @@ -539,6 +584,16 @@ "First, let's read the data we need to perform the NDVI." ] }, + { + "cell_type": "markdown", + "id": "19ad8906-30ee-4511-aed4-a0a32abd3458", + "metadata": { + "tags": [] + }, + "source": [ + "### Defining usefull functions for our notebook" + ] + }, { "cell_type": "code", "execution_count": 3, @@ -579,13 +634,19 @@ "\n", " return da.concatenate(results), x_res, y_res, top_left_x, top_left_y, crs\n", "\n", - "# Paths to RGB Sentinel-2 bands\n", - "sentinel_2_dir = \"/work/scratch/data/romaint\"\n", - "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", - "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", - "#reading_chunks = (-1,2200,2200)\n", - "reading_chunks = True\n", - "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks)" + "def create_raster(data: np.ndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n", + " transform = Affine.translation(top_left_x, top_left_y) * Affine.scale(x_res, -y_res)\n", + " with rasterio.open(\n", + " output_file, \"w\",\n", + " driver=\"GTiff\",\n", + " height=data.shape[1],\n", + " width=data.shape[2],\n", + " count=data.shape[0],\n", + " dtype=data.dtype,\n", + " crs=crs,\n", + " transform=transform\n", + " ) as dst:\n", + " dst.write(data)\n" ] }, { @@ -598,70 +659,158 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 13, "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f", "metadata": { "tags": [] }, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "
\n", - " Comm: tcp://127.0.0.1:45347\n", + " Comm: tcp://127.0.0.1:41857\n", " \n", - " Total threads: 1\n", + " Total threads: 2\n", "
\n", - " Dashboard: http://127.0.0.1:46387/status\n", + " Dashboard: http://127.0.0.1:43739/status\n", " \n", - " Memory: 7.00 GiB\n", + " Memory: 14.00 GiB\n", "
\n", - " Nanny: tcp://127.0.0.1:42059\n", + " Nanny: tcp://127.0.0.1:35379\n", "
\n", - " Local directory: /tmp/slurm-28574237/dask-scratch-space/worker-sm2i26gx\n", + " Local directory: /tmp/slurm-29119419/dask-scratch-space/worker-ohokz8ot\n", "
\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Array Chunk
Bytes 459.90 MiB 9.23 MiB
Shape (2, 10980, 10980) (1, 2200, 2200)
Dask graph 50 chunks in 5 graph layers
Data type int16 numpy.ndarray
\n", + " 		\n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " 10980\n", + " 10980\n", + " 2\n", + "\n", + "
" + ], + "text/plain": [ + "dask.array" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ + "# Open the raster\n", + "# Define our own chunks for better performance\n", + "reading_chunks = (-1,2200,2200)\n", + "#reading_chunks = True\n", + "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks)\n", "input_data_array" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 12, "id": "2dad7fe0-23bc-40b2-b27e-4c7cb0f8dc0e", "metadata": { "tags": [] }, - "outputs": [], - "source": [ - "print(input_data_array.shape)\n", - "ndvi_array = (input_data_array[1] - input_data_array[0]) / (input_data_array[1] + input_data_array[0])[None, :, :]" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "19fc2fce-4aca-4b7a-abd6-6baba3b5c464", - "metadata": { - "tags": [] - }, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CPU times: user 528 ms, sys: 2.08 s, total: 2.61 s\n", + "Wall time: 6.19 s\n" + ] + } + ], "source": [ "%%time\n", "\n", - "mean_ndvi = ndvi_array.compute()" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "c4f6ae3e-4511-448f-97d8-ff85172dea6c", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "def create_raster(data: np.ndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n", - " transform = Affine.translation(top_left_x, top_left_y) * Affine.scale(x_res, -y_res)\n", - " with rasterio.open(\n", - " output_file, \"w\",\n", - " driver=\"GTiff\",\n", - " height=data.shape[1],\n", - " width=data.shape[2],\n", - " count=data.shape[0],\n", - " dtype=data.dtype,\n", - " crs=crs,\n", - " transform=transform\n", - " ) as dst:\n", - " dst.write(data)\n", - "\n", + "ndvi_array = (input_data_array[1] - input_data_array[0]) / (input_data_array[1] + input_data_array[0])[None, :, :]\n", + "# Launch the computing with dask with the compute call\n", + "mean_ndvi = ndvi_array.compute() \n", "crs=\"EPSG:4326\"\n", - "output_file = Path(\"/home/tromain/Data/S2/ndvi_dask.tif\")\n", - "#create_raster(ndvi_array, output_file, x_res , y_res,\n", - "# top_left_x, top_left_y, crs)" + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_dask.tif\")\n", + "create_raster(mean_ndvi, output_file, x_res , y_res,\n", + " top_left_x, top_left_y, crs)" ] }, { @@ -669,12 +818,12 @@ "id": "7f5d09f0-cf39-4e3b-afcb-6db0bccc877a", "metadata": {}, "source": [ - "# Calculate NDVI With OTB in python" + "## Calculate NDVI With OTB in python" ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 12, "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7", "metadata": { "tags": [] @@ -691,7 +840,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Writing /work/scratch/data/romaint/img_ndvi_otb.tif...: 100% [**************************************************] (1m 45s)\n" + "Writing /work/scratch/data/romaint/img_ndvi_otb.tif...: 100% [**************************************************] (4s)\n" ] }, { @@ -700,7 +849,7 @@ "0" ] }, - "execution_count": 9, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } @@ -708,20 +857,15 @@ "source": [ "import otbApplication as otb\n", "\n", - "sentinel_2_dir = \"/work/scratch/data/romaint\"\n", - "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n", - "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n", - "out_ndvi_otb_py=\"/work/scratch/data/romaint/img_ndvi_otb.tif\"\n", - "#phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n", - "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", + "out_ndvi_otb_py=\"/work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif\"\n", "#Compute NDVI with OTB in python\n", "app_ndvi_otb = otb.Registry.CreateApplication(\"BandMath\")\n", - "#app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n", - "#app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n", - "app_ndvi_otb.SetParameterStringList(\"il\",[phr_product])\n", - "app_ndvi_otb.SetParameterString(\"exp\",\"(im1b4-im1b1)/(im1b4+im1b1)\")\n", + "app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n", + "app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n", + "#app_ndvi_otb.SetParameterStringList(\"il\",[phr_product])\n", + "#app_ndvi_otb.SetParameterString(\"exp\",\"(im1b4-im1b1)/(im1b4+im1b1)\")\n", "app_ndvi_otb.SetParameterString(\"out\",out_ndvi_otb_py)\n", - "app_ndvi_otb.ExecuteAndWriteOutput()\n" + "app_ndvi_otb.ExecuteAndWriteOutput()" ] }, { @@ -729,7 +873,7 @@ "id": "c030e442-871e-4d59-8cb6-1310b5867df1", "metadata": {}, "source": [ - "# Calculate NDVI with OTB in C++\n", + "## Calculate NDVI with OTB in C++\n", "This part will call BandMath with the otb CLI to compare performances with the python swig interface" ] }, @@ -740,820 +884,69 @@ "metadata": { "tags": [] }, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning 1: Invalid value for NUM_THREADS: \n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Writing /work/scratch/data/romaint/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (4s)\n" + ] + } + ], "source": [ "%%bash\n", - "otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/img_ndvi_otb_cpp.tif\" " - ] - }, - { - "cell_type": "markdown", - "id": "7966cdad-a3d7-467d-bd1a-d207bc85bde0", - "metadata": {}, - "source": [ - "# Compute SuperImpose with OTB\n", - "\n", - "Superimpose does a resampling then a crop to have a new raster that has the same resolution as the reference input image. This app is using multithreading a lot, it is interesting to compare it with a solution like dask and full python / rasterio" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "71d9a855-86f3-48aa-a9eb-32a1dc853649", - "metadata": {}, - "outputs": [], - "source": [ - "import otbApplication as otb\n", - "\n", - "#product_dir = \"/work/scratch/data/romaint\"\n", - "pan_raster_url = \"/work/scratch/data/tanguyy/public/PHR_OTB/IMG_PHR1B_P_001/IMG_PHR1B_P_202302281104151_SEN_6967639101-1_R1C1.JP2\"\n", - "xs_raster_url = \"/work/scratch/data/tanguyy/public/PHR_OTB/IMG_PHR1B_MS_004/IMG_PHR1B_MS_202302281104151_SEN_6967639101-2_R1C1.JP2\"\n", - "appSI = otb.Registry.CreateApplication(\"Superimpose\")\n", "\n", - "appSI.SetParameterString(\"inr\", pan_raster_url)\n", - "appSI.SetParameterString(\"inm\", xs_raster_url)\n", - "appSI.SetParameterString(\"out\", \"/work/scratch/data/romaint/SuperimposedXS_to_PAN.tif\")\n", - "\n", - "appSI.ExecuteAndWriteOutput()" + "otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/output_greenit/img_ndvi_otb_cpp.tif\" " ] }, { "cell_type": "markdown", - "id": "49b7321e-1cfb-4994-83fc-93312252c84f", + "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386", "metadata": {}, "source": [ - "# Performance improvement with xarray.coarsen method ?" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e7568371-4a69-426e-9247-c7e4abe7aef2", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "%%time\n", - "\n", - "import xarray\n", - "from rioxarray.merge import merge_arrays\n", + "# Optimizing the computing of NDVI\n", "\n", - "pan_raster_url = \"/home/tromain/Data/phr_pan.tif\"\n", - "xs_raster_url = \"/home/tromain/Data/phr_xs.tif\"\n", - "origin_raster = rxr.open_rasterio(pan_raster_url,chunks=True)\n", - "#.squeeze('band', drop=True)\n", - "print(origin_raster)\n", - "#origin_raster = origin_raster.load()\n", - "pxs_raster = origin_raster.coarsen(x=4, y=4, boundary='pad').mean()\n", - "xs_raster = rxr.open_rasterio(xs_raster_url,chunks=True)\n", - "print(pxs_raster)\n", - "#print(\"X == X? {} Y==Y? {}\".format((origin_raster.x == xs_raster.x),(origin_raster.y == xs_raster.y)))\n", - "print(all(list(origin_raster.x == xs_raster.x)))\n", - "#ppxs_raster = xarray.concat([pxs_raster,xs_raster],dim=\"band\",coords=\"all\")\n", - "#ppxs_raster = merge_arrays([pxs_raster,xs_raster])\n", - "crs=\"EPSG:4326\"\n", - "output_file = Path(\"/home/tromain/Data/superimpose_coarsen.tif\")\n", - "pxs_raster.rio.to_raster(output_file)" - ] - }, - { - "cell_type": "markdown", - "id": "b2571ad8-9af5-43c2-8c5a-4c8a73a6c885", - "metadata": {}, - "source": [ - "# SuperImpose using rasterio reproject_match" + "Here we will try to use numba for ndvi computation" ] }, { "cell_type": "code", - "execution_count": null, - "id": "4407913c-a383-451a-9967-8a7908e9664c", + "execution_count": 16, + "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50", "metadata": { "tags": [] }, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Elapsed with Raster IO + numba = 1.2573514231480658s\n", + "Elapsed with Raster IO + without numba = 1.6734004551544785s\n", + "Elapsed with Xarray + apply ufunc = 2.9065667972899973s\n", + "Elapsed with RIOXarray + dask = 6.7410209202207625s\n", + "CPU times: user 2.53 s, sys: 8.62 s, total: 11.1 s\n", + "Wall time: 12.6 s\n" + ] + } + ], "source": [ "%%time\n", "\n", - "import xarray\n", - "from pathlib import Path\n", - "\n", - "pan_raster_url = \"/work/scratch/data/tanguyy/public/PHR_OTB/IMG_PHR1B_P_001/IMG_PHR1B_P_202302281104151_SEN_6967639101-1_R1C1.JP2\"\n", - "xs_raster_url = \"/work/scratch/data/tanguyy/public/PHR_OTB/IMG_PHR1B_MS_004/IMG_PHR1B_MS_202302281104151_SEN_6967639101-2_R1C1.JP2\"\n", - "\n", - "xds = xarray.open_dataarray(pan_raster_url,chunks=True)\n", - "xds.rio.write_crs(\"epsg:4326\", inplace=True)\n", - "xds.rio.write_nodata(0, inplace=True)\n", - "xds_match = xarray.open_dataarray(xs_raster_url,chunks=True)\n", - "print(xds.shape)\n", - "print(xds.dtype)\n", - "print(xds_match.shape)\n", - "xds_match.rio.write_crs(\"epsg:4326\", inplace=True)\n", - "xds_match.rio.write_nodata(0, inplace=True)\n", - "xds_repr_match = xds.rio.reproject_match(xds_match)\n", - "crs=\"EPSG:4326\"\n", - "output_file = Path(\"/work/scratch/data/romaint/superimpose_reproject_match.tif\")\n", - "xds_repr_match.rio.to_raster(output_file)" - ] - }, - { - "cell_type": "markdown", - "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386", - "metadata": {}, - "source": [ - "# Using numba vs xarray vs numpy for NDVI ?\n", - "Here we will try to use numba for ndvi computation" - ] - }, - { - "cell_type": "code", - "execution_count": 23, - "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Elapsed with Raster IO + numba = 1.2825823966413736s\n", - "Elapsed with Raster IO + without numba = 1.1891403198242188s\n", - "Elapsed with Xarray + apply ufunc = 2.707308219745755s\n", - "Elapsed with RIOXarray + dask = 5.868392776697874s\n", - "(4, 41663, 39844)\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2024-11-14 15:29:30,639 - distributed.core - ERROR - Exception while handling op get_data\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 2)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 820, in _handle_comm\n", - " result = await result\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1734, in get_data\n", - " data = {k: self.data[k] for k in keys if k in self.data}\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1734, in \n", - " data = {k: self.data[k] for k in keys if k in self.data}\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "2024-11-14 15:29:30,645 - distributed.core - ERROR - Exception while handling op get_data\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 3)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 820, in _handle_comm\n", - " result = await result\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1734, in get_data\n", - " data = {k: self.data[k] for k in keys if k in self.data}\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1734, in \n", - " data = {k: self.data[k] for k in keys if k in self.data}\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "2024-11-14 15:29:36,735 - distributed.worker.memory - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker-memory.html#memory-not-released-back-to-the-os for more information. -- Unmanaged memory: 4.94 GiB -- Worker memory limit: 7.00 GiB\n", - "2024-11-14 15:29:36,819 - distributed.worker.memory - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker-memory.html#memory-not-released-back-to-the-os for more information. -- Unmanaged memory: 4.94 GiB -- Worker memory limit: 7.00 GiB\n", - "2024-11-14 15:29:36,918 - distributed.worker.memory - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker-memory.html#memory-not-released-back-to-the-os for more information. -- Unmanaged memory: 4.94 GiB -- Worker memory limit: 7.00 GiB\n", - "Task exception was never retrieved\n", - "future: exception=Exception('AssertionError()')>\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 2870, in get_data_from_worker\n", - " return await retry_operation(_get_data, operation=\"get_data_from_worker\")\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py\", line 400, in retry_operation\n", - " return await retry(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py\", line 385, in retry\n", - " return await coro()\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 2850, in _get_data\n", - " response = await send_recv(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 1013, in send_recv\n", - " raise Exception(response[\"exception_text\"])\n", - "Exception: AssertionError()\n" - ] - }, - { - "ename": "Exception", - "evalue": "AssertionError()", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mException\u001b[0m Traceback (most recent call last)", - "File \u001b[0;32m:90\u001b[0m\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/xarray/core/dataarray.py:1092\u001b[0m, in \u001b[0;36mDataArray.compute\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 1073\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"Manually trigger loading of this array's data from disk or a\u001b[39;00m\n\u001b[1;32m 1074\u001b[0m \u001b[38;5;124;03mremote source into memory and return a new array. The original is\u001b[39;00m\n\u001b[1;32m 1075\u001b[0m \u001b[38;5;124;03mleft unaltered.\u001b[39;00m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 1089\u001b[0m \u001b[38;5;124;03mdask.compute\u001b[39;00m\n\u001b[1;32m 1090\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m 1091\u001b[0m new \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mcopy(deep\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m)\n\u001b[0;32m-> 1092\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mnew\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mload\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/xarray/core/dataarray.py:1066\u001b[0m, in \u001b[0;36mDataArray.load\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 1048\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mload\u001b[39m(\u001b[38;5;28mself\u001b[39m: T_DataArray, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m T_DataArray:\n\u001b[1;32m 1049\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"Manually trigger loading of this array's data from disk or a\u001b[39;00m\n\u001b[1;32m 1050\u001b[0m \u001b[38;5;124;03m remote source into memory and return this array.\u001b[39;00m\n\u001b[1;32m 1051\u001b[0m \n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 1064\u001b[0m \u001b[38;5;124;03m dask.compute\u001b[39;00m\n\u001b[1;32m 1065\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[0;32m-> 1066\u001b[0m ds \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_to_temp_dataset\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mload\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1067\u001b[0m new \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_from_temp_dataset(ds)\n\u001b[1;32m 1068\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_variable \u001b[38;5;241m=\u001b[39m new\u001b[38;5;241m.\u001b[39m_variable\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/xarray/core/dataset.py:739\u001b[0m, in \u001b[0;36mDataset.load\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 736\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mdask\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01marray\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mda\u001b[39;00m\n\u001b[1;32m 738\u001b[0m \u001b[38;5;66;03m# evaluate all the dask arrays simultaneously\u001b[39;00m\n\u001b[0;32m--> 739\u001b[0m evaluated_data \u001b[38;5;241m=\u001b[39m \u001b[43mda\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompute\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mlazy_data\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mvalues\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 741\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m k, data \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mzip\u001b[39m(lazy_data, evaluated_data):\n\u001b[1;32m 742\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mvariables[k]\u001b[38;5;241m.\u001b[39mdata \u001b[38;5;241m=\u001b[39m data\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/dask/base.py:600\u001b[0m, in \u001b[0;36mcompute\u001b[0;34m(traverse, optimize_graph, scheduler, get, *args, **kwargs)\u001b[0m\n\u001b[1;32m 597\u001b[0m keys\u001b[38;5;241m.\u001b[39mappend(x\u001b[38;5;241m.\u001b[39m__dask_keys__())\n\u001b[1;32m 598\u001b[0m postcomputes\u001b[38;5;241m.\u001b[39mappend(x\u001b[38;5;241m.\u001b[39m__dask_postcompute__())\n\u001b[0;32m--> 600\u001b[0m results \u001b[38;5;241m=\u001b[39m \u001b[43mschedule\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdsk\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkeys\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 601\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m repack([f(r, \u001b[38;5;241m*\u001b[39ma) \u001b[38;5;28;01mfor\u001b[39;00m r, (f, a) \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mzip\u001b[39m(results, postcomputes)])\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/client.py:3125\u001b[0m, in \u001b[0;36mClient.get\u001b[0;34m(self, dsk, keys, workers, allow_other_workers, resources, sync, asynchronous, direct, retries, priority, fifo_timeout, actors, **kwargs)\u001b[0m\n\u001b[1;32m 3123\u001b[0m should_rejoin \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mFalse\u001b[39;00m\n\u001b[1;32m 3124\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m-> 3125\u001b[0m results \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mgather\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpacked\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43masynchronous\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43masynchronous\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdirect\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdirect\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 3126\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m 3127\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m futures\u001b[38;5;241m.\u001b[39mvalues():\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/client.py:2294\u001b[0m, in \u001b[0;36mClient.gather\u001b[0;34m(self, futures, errors, direct, asynchronous)\u001b[0m\n\u001b[1;32m 2292\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 2293\u001b[0m local_worker \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m-> 2294\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msync\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 2295\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_gather\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2296\u001b[0m \u001b[43m \u001b[49m\u001b[43mfutures\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2297\u001b[0m \u001b[43m \u001b[49m\u001b[43merrors\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43merrors\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2298\u001b[0m \u001b[43m \u001b[49m\u001b[43mdirect\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdirect\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2299\u001b[0m \u001b[43m \u001b[49m\u001b[43mlocal_worker\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlocal_worker\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2300\u001b[0m \u001b[43m \u001b[49m\u001b[43masynchronous\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43masynchronous\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 2301\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils.py:339\u001b[0m, in \u001b[0;36mSyncMethodMixin.sync\u001b[0;34m(self, func, asynchronous, callback_timeout, *args, **kwargs)\u001b[0m\n\u001b[1;32m 337\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m future\n\u001b[1;32m 338\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m--> 339\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43msync\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 340\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mloop\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunc\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcallback_timeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcallback_timeout\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\n\u001b[1;32m 341\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils.py:406\u001b[0m, in \u001b[0;36msync\u001b[0;34m(loop, func, callback_timeout, *args, **kwargs)\u001b[0m\n\u001b[1;32m 404\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m error:\n\u001b[1;32m 405\u001b[0m typ, exc, tb \u001b[38;5;241m=\u001b[39m error\n\u001b[0;32m--> 406\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\u001b[38;5;241m.\u001b[39mwith_traceback(tb)\n\u001b[1;32m 407\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 408\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m result\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils.py:379\u001b[0m, in \u001b[0;36msync..f\u001b[0;34m()\u001b[0m\n\u001b[1;32m 377\u001b[0m future \u001b[38;5;241m=\u001b[39m asyncio\u001b[38;5;241m.\u001b[39mwait_for(future, callback_timeout)\n\u001b[1;32m 378\u001b[0m future \u001b[38;5;241m=\u001b[39m asyncio\u001b[38;5;241m.\u001b[39mensure_future(future)\n\u001b[0;32m--> 379\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01myield\u001b[39;00m future\n\u001b[1;32m 380\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m:\n\u001b[1;32m 381\u001b[0m error \u001b[38;5;241m=\u001b[39m sys\u001b[38;5;241m.\u001b[39mexc_info()\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/tornado/gen.py:762\u001b[0m, in \u001b[0;36mRunner.run\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 759\u001b[0m exc_info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m 761\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 762\u001b[0m value \u001b[38;5;241m=\u001b[39m \u001b[43mfuture\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresult\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 763\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m:\n\u001b[1;32m 764\u001b[0m exc_info \u001b[38;5;241m=\u001b[39m sys\u001b[38;5;241m.\u001b[39mexc_info()\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/client.py:2186\u001b[0m, in \u001b[0;36mClient._gather\u001b[0;34m(self, futures, errors, direct, local_worker)\u001b[0m\n\u001b[1;32m 2184\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 2185\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_gather_future \u001b[38;5;241m=\u001b[39m future\n\u001b[0;32m-> 2186\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m future\n\u001b[1;32m 2188\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstatus\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124merror\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m 2189\u001b[0m log \u001b[38;5;241m=\u001b[39m logger\u001b[38;5;241m.\u001b[39mwarning \u001b[38;5;28;01mif\u001b[39;00m errors \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mraise\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m logger\u001b[38;5;241m.\u001b[39mdebug\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/client.py:2237\u001b[0m, in \u001b[0;36mClient._gather_remote\u001b[0;34m(self, direct, local_worker)\u001b[0m\n\u001b[1;32m 2234\u001b[0m response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdata\u001b[39m\u001b[38;5;124m\"\u001b[39m]\u001b[38;5;241m.\u001b[39mupdate(data2)\n\u001b[1;32m 2236\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m: \u001b[38;5;66;03m# ask scheduler to gather data for us\u001b[39;00m\n\u001b[0;32m-> 2237\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m retry_operation(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mscheduler\u001b[38;5;241m.\u001b[39mgather, keys\u001b[38;5;241m=\u001b[39mkeys)\n\u001b[1;32m 2239\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m response\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:400\u001b[0m, in \u001b[0;36mretry_operation\u001b[0;34m(coro, operation, *args, **kwargs)\u001b[0m\n\u001b[1;32m 394\u001b[0m retry_delay_min \u001b[38;5;241m=\u001b[39m parse_timedelta(\n\u001b[1;32m 395\u001b[0m dask\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdistributed.comm.retry.delay.min\u001b[39m\u001b[38;5;124m\"\u001b[39m), default\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 396\u001b[0m )\n\u001b[1;32m 397\u001b[0m retry_delay_max \u001b[38;5;241m=\u001b[39m parse_timedelta(\n\u001b[1;32m 398\u001b[0m dask\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdistributed.comm.retry.delay.max\u001b[39m\u001b[38;5;124m\"\u001b[39m), default\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 399\u001b[0m )\n\u001b[0;32m--> 400\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m retry(\n\u001b[1;32m 401\u001b[0m partial(coro, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs),\n\u001b[1;32m 402\u001b[0m count\u001b[38;5;241m=\u001b[39mretry_count,\n\u001b[1;32m 403\u001b[0m delay_min\u001b[38;5;241m=\u001b[39mretry_delay_min,\n\u001b[1;32m 404\u001b[0m delay_max\u001b[38;5;241m=\u001b[39mretry_delay_max,\n\u001b[1;32m 405\u001b[0m operation\u001b[38;5;241m=\u001b[39moperation,\n\u001b[1;32m 406\u001b[0m )\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:385\u001b[0m, in \u001b[0;36mretry\u001b[0;34m(coro, count, delay_min, delay_max, jitter_fraction, retry_on_exceptions, operation)\u001b[0m\n\u001b[1;32m 383\u001b[0m delay \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m1\u001b[39m \u001b[38;5;241m+\u001b[39m random\u001b[38;5;241m.\u001b[39mrandom() \u001b[38;5;241m*\u001b[39m jitter_fraction\n\u001b[1;32m 384\u001b[0m \u001b[38;5;28;01mawait\u001b[39;00m asyncio\u001b[38;5;241m.\u001b[39msleep(delay)\n\u001b[0;32m--> 385\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m coro()\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/core.py:1221\u001b[0m, in \u001b[0;36mPooledRPCCall.__getattr__..send_recv_from_rpc\u001b[0;34m(**kwargs)\u001b[0m\n\u001b[1;32m 1219\u001b[0m prev_name, comm\u001b[38;5;241m.\u001b[39mname \u001b[38;5;241m=\u001b[39m comm\u001b[38;5;241m.\u001b[39mname, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mConnectionPool.\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;241m+\u001b[39m key\n\u001b[1;32m 1220\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m-> 1221\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m send_recv(comm\u001b[38;5;241m=\u001b[39mcomm, op\u001b[38;5;241m=\u001b[39mkey, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[1;32m 1222\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m 1223\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mpool\u001b[38;5;241m.\u001b[39mreuse(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39maddr, comm)\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/core.py:1011\u001b[0m, in \u001b[0;36msend_recv\u001b[0;34m(comm, reply, serializers, deserializers, **kwargs)\u001b[0m\n\u001b[1;32m 1009\u001b[0m _, exc, tb \u001b[38;5;241m=\u001b[39m clean_exception(\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mresponse)\n\u001b[1;32m 1010\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m exc\n\u001b[0;32m-> 1011\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\u001b[38;5;241m.\u001b[39mwith_traceback(tb)\n\u001b[1;32m 1012\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 1013\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m(response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mexception_text\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/core.py:820\u001b[0m, in \u001b[0;36m_handle_comm\u001b[0;34m()\u001b[0m\n\u001b[1;32m 818\u001b[0m result \u001b[38;5;241m=\u001b[39m handler(\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mmsg)\n\u001b[1;32m 819\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m inspect\u001b[38;5;241m.\u001b[39miscoroutine(result):\n\u001b[0;32m--> 820\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m result\n\u001b[1;32m 821\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m inspect\u001b[38;5;241m.\u001b[39misawaitable(result):\n\u001b[1;32m 822\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\n\u001b[1;32m 823\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mComm handler returned unknown awaitable. Expected coroutine, instead got \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mtype\u001b[39m(result)\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 824\u001b[0m )\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/scheduler.py:5654\u001b[0m, in \u001b[0;36mgather\u001b[0;34m()\u001b[0m\n\u001b[1;32m 5651\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 5652\u001b[0m who_has[key] \u001b[38;5;241m=\u001b[39m []\n\u001b[0;32m-> 5654\u001b[0m data, missing_keys, missing_workers \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m gather_from_workers(\n\u001b[1;32m 5655\u001b[0m who_has, rpc\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mrpc, close\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m, serializers\u001b[38;5;241m=\u001b[39mserializers\n\u001b[1;32m 5656\u001b[0m )\n\u001b[1;32m 5657\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m missing_keys:\n\u001b[1;32m 5658\u001b[0m result \u001b[38;5;241m=\u001b[39m {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstatus\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mOK\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdata\u001b[39m\u001b[38;5;124m\"\u001b[39m: data}\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:80\u001b[0m, in \u001b[0;36mgather_from_workers\u001b[0;34m()\u001b[0m\n\u001b[1;32m 78\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m worker, c \u001b[38;5;129;01min\u001b[39;00m coroutines\u001b[38;5;241m.\u001b[39mitems():\n\u001b[1;32m 79\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m---> 80\u001b[0m r \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m c\n\u001b[1;32m 81\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mOSError\u001b[39;00m:\n\u001b[1;32m 82\u001b[0m missing_workers\u001b[38;5;241m.\u001b[39madd(worker)\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/worker.py:2870\u001b[0m, in \u001b[0;36mget_data_from_worker\u001b[0;34m()\u001b[0m\n\u001b[1;32m 2867\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m 2868\u001b[0m rpc\u001b[38;5;241m.\u001b[39mreuse(worker, comm)\n\u001b[0;32m-> 2870\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m retry_operation(_get_data, operation\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mget_data_from_worker\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:400\u001b[0m, in \u001b[0;36mretry_operation\u001b[0;34m()\u001b[0m\n\u001b[1;32m 394\u001b[0m retry_delay_min \u001b[38;5;241m=\u001b[39m parse_timedelta(\n\u001b[1;32m 395\u001b[0m dask\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdistributed.comm.retry.delay.min\u001b[39m\u001b[38;5;124m\"\u001b[39m), default\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 396\u001b[0m )\n\u001b[1;32m 397\u001b[0m retry_delay_max \u001b[38;5;241m=\u001b[39m parse_timedelta(\n\u001b[1;32m 398\u001b[0m dask\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdistributed.comm.retry.delay.max\u001b[39m\u001b[38;5;124m\"\u001b[39m), default\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 399\u001b[0m )\n\u001b[0;32m--> 400\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m retry(\n\u001b[1;32m 401\u001b[0m partial(coro, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs),\n\u001b[1;32m 402\u001b[0m count\u001b[38;5;241m=\u001b[39mretry_count,\n\u001b[1;32m 403\u001b[0m delay_min\u001b[38;5;241m=\u001b[39mretry_delay_min,\n\u001b[1;32m 404\u001b[0m delay_max\u001b[38;5;241m=\u001b[39mretry_delay_max,\n\u001b[1;32m 405\u001b[0m operation\u001b[38;5;241m=\u001b[39moperation,\n\u001b[1;32m 406\u001b[0m )\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/utils_comm.py:385\u001b[0m, in \u001b[0;36mretry\u001b[0;34m()\u001b[0m\n\u001b[1;32m 383\u001b[0m delay \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m1\u001b[39m \u001b[38;5;241m+\u001b[39m random\u001b[38;5;241m.\u001b[39mrandom() \u001b[38;5;241m*\u001b[39m jitter_fraction\n\u001b[1;32m 384\u001b[0m \u001b[38;5;28;01mawait\u001b[39;00m asyncio\u001b[38;5;241m.\u001b[39msleep(delay)\n\u001b[0;32m--> 385\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m coro()\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/worker.py:2850\u001b[0m, in \u001b[0;36m_get_data\u001b[0;34m()\u001b[0m\n\u001b[1;32m 2848\u001b[0m comm\u001b[38;5;241m.\u001b[39mname \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEphemeral Worker->Worker for gather\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 2849\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m-> 2850\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m send_recv(\n\u001b[1;32m 2851\u001b[0m comm,\n\u001b[1;32m 2852\u001b[0m serializers\u001b[38;5;241m=\u001b[39mserializers,\n\u001b[1;32m 2853\u001b[0m deserializers\u001b[38;5;241m=\u001b[39mdeserializers,\n\u001b[1;32m 2854\u001b[0m op\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mget_data\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m 2855\u001b[0m keys\u001b[38;5;241m=\u001b[39mkeys,\n\u001b[1;32m 2856\u001b[0m who\u001b[38;5;241m=\u001b[39mwho,\n\u001b[1;32m 2857\u001b[0m max_connections\u001b[38;5;241m=\u001b[39mmax_connections,\n\u001b[1;32m 2858\u001b[0m )\n\u001b[1;32m 2859\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m 2860\u001b[0m status \u001b[38;5;241m=\u001b[39m response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstatus\u001b[39m\u001b[38;5;124m\"\u001b[39m]\n", - "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/distributed/core.py:1013\u001b[0m, in \u001b[0;36msend_recv\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1011\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\u001b[38;5;241m.\u001b[39mwith_traceback(tb)\n\u001b[1;32m 1012\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m-> 1013\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m(response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mexception_text\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n\u001b[1;32m 1014\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m response\n", - "\u001b[0;31mException\u001b[0m: AssertionError()" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2024-11-14 15:29:39,214 - distributed.worker - ERROR - \n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 2)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "2024-11-14 15:29:39,222 - tornado.application - ERROR - Exception in callback functools.partial(>, exception=AssertionError()>)\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 2)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", - " ret = callback()\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", - " future.result()\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", - " return await method(self, *args, **kwargs) # type: ignore\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", - " await self.handle_stream(comm)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", - " handler(**merge(extra, msg))\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", - " self.handle_stimulus(event)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "unhandled exception during asyncio.run() shutdown\n", - "task: exception=AssertionError()>\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 2)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", - " ret = callback()\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", - " future.result()\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", - " return await method(self, *args, **kwargs) # type: ignore\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", - " await self.handle_stream(comm)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", - " handler(**merge(extra, msg))\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", - " self.handle_stimulus(event)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "2024-11-14 15:29:39,351 - distributed.worker - ERROR - \n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 3)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "2024-11-14 15:29:39,357 - tornado.application - ERROR - Exception in callback functools.partial(>, exception=AssertionError()>)\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 3)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", - " ret = callback()\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", - " future.result()\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", - " return await method(self, *args, **kwargs) # type: ignore\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", - " await self.handle_stream(comm)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", - " handler(**merge(extra, msg))\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", - " self.handle_stimulus(event)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "unhandled exception during asyncio.run() shutdown\n", - "task: exception=AssertionError()>\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 0, 3)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", - " ret = callback()\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", - " future.result()\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", - " return await method(self, *args, **kwargs) # type: ignore\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", - " await self.handle_stream(comm)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", - " handler(**merge(extra, msg))\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", - " self.handle_stimulus(event)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "2024-11-14 15:29:39,666 - distributed.worker - ERROR - \n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 4, 3)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "2024-11-14 15:29:40,022 - tornado.application - ERROR - Exception in callback functools.partial(>, exception=AssertionError()>)\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 4, 3)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", - " ret = callback()\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", - " future.result()\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", - " return await method(self, *args, **kwargs) # type: ignore\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", - " await self.handle_stream(comm)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", - " handler(**merge(extra, msg))\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", - " self.handle_stimulus(event)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "unhandled exception during asyncio.run() shutdown\n", - "task: exception=AssertionError()>\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 184, in __getitem__\n", - " return self.fast[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/lru.py\", line 117, in __getitem__\n", - " result = self.d[key]\n", - "KeyError: \"('truediv-f9d8cf4534ba5caa42181c228e4602cd', 4, 3)\"\n", - "\n", - "During handling of the above exception, another exception occurred:\n", - "\n", - "Traceback (most recent call last):\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 741, in _run_callback\n", - " ret = callback()\n", - " File \"/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py\", line 765, in _discard_future_result\n", - " future.result()\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 178, in wrapper\n", - " return await method(self, *args, **kwargs) # type: ignore\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1261, in handle_scheduler\n", - " await self.handle_stream(comm)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/core.py\", line 904, in handle_stream\n", - " handler(**merge(extra, msg))\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1911, in _\n", - " self.handle_stimulus(event)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 191, in wrapper\n", - " return method(self, *args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker.py\", line 1936, in handle_stimulus\n", - " super().handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 3659, in handle_stimulus\n", - " instructions = self.state.handle_stimulus(*stims)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1338, in handle_stimulus\n", - " instructions += self._transitions(recs, stimulus_id=stim.stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2715, in _transitions\n", - " process_recs(recommendations.copy())\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2709, in process_recs\n", - " a_recs, a_instructions = self._transition(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2627, in _transition\n", - " recs, instructions = func(self, ts, *args, stimulus_id=stimulus_id)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 2003, in _transition_memory_released\n", - " recs, instructions = self._transition_generic_released(\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1933, in _transition_generic_released\n", - " self._purge_state(ts)\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/worker_state_machine.py\", line 1467, in _purge_state\n", - " self.data.pop(key, None)\n", - " File \"/usr/lib/python3.10/_collections_abc.py\", line 962, in pop\n", - " value = self[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 269, in __getitem__\n", - " return super().__getitem__(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 186, in __getitem__\n", - " return self.slow_to_fast(key)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/buffer.py\", line 153, in slow_to_fast\n", - " value = self.slow[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/common.py\", line 127, in wrapper\n", - " return func(*args, **kwargs)\n", - " File \"/usr/local/lib/python3.10/dist-packages/zict/cache.py\", line 70, in __getitem__\n", - " value = self.data[key]\n", - " File \"/usr/local/lib/python3.10/dist-packages/distributed/spill.py\", line 384, in __getitem__\n", - " assert isinstance(pickled, bytes)\n", - "AssertionError\n", - "2024-11-14 15:29:41,223 - distributed.nanny - ERROR - Worker process died unexpectedly\n" - ] - } - ], - "source": [ - "%%time\n", - "\n", - "from numba import jit,njit\n", - "from xarray import DataArray \n", - "from typing import List, Tuple, Union, Dict\n", - "import rioxarray as rxr\n", - "import numpy as np\n", - "import time\n", - "import rasterio\n", + "from numba import jit,njit\n", + "from xarray import DataArray \n", + "from typing import List, Tuple, Union, Dict\n", + "import rioxarray as rxr\n", + "import numpy as np\n", + "import time\n", + "import rasterio\n", "from pathlib import Path\n", "import xarray\n", "\n", @@ -1592,7 +985,7 @@ "ndvi_computed = compute_ndvi_numba(input_data_b4,input_data_b8)\n", "crs=\"EPSG:4326\"\n", "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_numba.tif\")\n", - "#create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n", + "create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n", "end = time.perf_counter()\n", "print(\"Elapsed with Raster IO + numba = {}s\".format((end - start)))\n", "\n", @@ -1606,7 +999,7 @@ "ndvi_computed = compute_ndvi_std(input_data_b4,input_data_b8)\n", "crs=\"EPSG:4326\"\n", "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_without_numba.tif\")\n", - "#create_raster(ndvi_computed, output_file, x_res , y_res,top_left_x, top_left_y, crs)\n", + "create_raster(ndvi_computed, output_file, x_res , y_res,top_left_x, top_left_y, crs)\n", "end = time.perf_counter()\n", "print(\"Elapsed with Raster IO + without numba = {}s\".format((end - start)))\n", "# Example with xarray\n", @@ -1630,84 +1023,637 @@ "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n", "\n", "#phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n", - "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", - "start = time.perf_counter()\n", - "reading_chunks = True\n", - "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n", - "print(input_data_array.shape)\n", - "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[0] + input_data_array[3])\n", - "ndvi_phr.compute()\n", - "print(ndvi_phr.shape)\n", - "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n", - "ndvi_phr.rio.to_raster(output_file,tiled=True)\n", - "end = time.perf_counter()\n", - "print(\"Elapsed with RIOXarray + dask + big product = {}s\".format((end - start)))" + "#phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", + "#start = time.perf_counter()\n", + "#reading_chunks = True\n", + "#input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n", + "#print(input_data_array.shape)\n", + "#ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[0] + input_data_array[3])\n", + "#ndvi_phr.compute()\n", + "#print(ndvi_phr.shape)\n", + "#output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n", + "#ndvi_phr.rio.to_raster(output_file,tiled=True)\n", + "#end = time.perf_counter()\n", + "#print(\"Elapsed with RIOXarray + dask + big product = {}s\".format((end - start)))" ] }, { - "cell_type": "code", - "execution_count": null, - "id": "bfa94789-8576-4b26-a962-226db7d37f01", + "cell_type": "markdown", + "id": "56e25516-3d90-4ee6-8c4b-085b94d3d096", "metadata": { "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "(4, 41663, 39844)\n", - "(41663, 39844)\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n", - " dataset = writer(\n", - "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n", - " return self.func(*new_argspec)\n", - "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n", - " return self.func(*new_argspec)\n", - "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n", - " return self.func(*new_argspec)\n", - "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n", - " return self.func(*new_argspec)\n" - ] - } - ], "source": [ - "%%time\n", + "## Differences between using Compute vs using RIO tiled write" + ] + }, + { + "cell_type": "markdown", + "id": "d3ab5833-1672-478a-8b26-0ed2bc4d6ee1", + "metadata": {}, + "source": [ + "## Optimize the chunk size for dask\n", "\n", - "from numba import jit,njit\n", - "from xarray import DataArray \n", - "from typing import List, Tuple, Union, Dict\n", - "import rioxarray as rxr\n", - "import numpy as np\n", - "import time\n", - "import rasterio\n", - "from pathlib import Path\n", - "import xarray\n", + "Chunk size is becoming very important when your data size grows. You can let chunks=True to dask which will automatically determine a chunk size\n", + "Most of the time this chunk size is coherent but users can tweak it to be more efficient.\n", "\n", - "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", - "start = time.perf_counter()\n", - "reading_chunks = True\n", - "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n", - "print(input_data_array.shape)\n", - "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[0] + input_data_array[3])\n", - "#ndvi_phr.compute()\n", - "print(ndvi_phr.shape)\n", - "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n", - "ndvi_phr.rio.to_raster(output_file,tiled=True)\n", - "end = time.perf_counter()\n", - "print(\"Elapsed with RIOXarray + dask + big product = {}s\".format((end - start)))" + "Also be careful about the compute() method, which is not recommended when your data size => 10Go, the memory consumption increases a lot" ] }, { "cell_type": "code", - "execution_count": 4, - "id": "4228aa64-b886-43eb-b4b0-2995df37c0e8", + "execution_count": 18, + "id": "57ae950e-2ea3-48a9-87ba-3b8a91de7c93", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
<xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
+       "dask.array<open_rasterio-fc2e18048eec7fd8788f8fc93c7ebea6<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n",
+       "Coordinates:\n",
+       "  * band         (band) int64 32B 1 2 3 4\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0\n",
+       "Attributes: (12/29)\n",
+       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
+       "    BlueDisplayChannel:   0\n",
+       "    DataType:             3\n",
+       "    GeometricLevel:       SENSOR\n",
+       "    GreenDisplayChannel:  0\n",
+       "    ImageID:              6967638101-1\n",
+       "    ...                   ...\n",
+       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
+       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
+       "    NoData:               0\n",
+       "    _FillValue:           0\n",
+       "    scale_factor:         1.0\n",
+       "    add_offset:           0.0
" + ], + "text/plain": [ + " Size: 13GB\n", + "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n", + "Coordinates:\n", + " * band (band) int64 32B 1 2 3 4\n", + " * x (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n", + " * y (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n", + " spatial_ref int64 8B 0\n", + "Attributes: (12/29)\n", + " AcquisitionDate: 2023-04-15T11:00:24.3Z\n", + " BlueDisplayChannel: 0\n", + " DataType: 3\n", + " GeometricLevel: SENSOR\n", + " GreenDisplayChannel: 0\n", + " ImageID: 6967638101-1\n", + " ... ...\n", + " TimeRangeEnd: 2023-04-15T11:00:27.3815980Z\n", + " TimeRangeStart: 2023-04-15T11:00:24.3193675Z\n", + " NoData: 0\n", + " _FillValue: 0\n", + " scale_factor: 1.0\n", + " add_offset: 0.0" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# First example letting dask compute the chunks\n", + "reading_chunks = True\n", + "data_array_autochunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks,lock=False)\n", + "data_array_autochunks" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "b8fb6e29-0a84-4ae0-882b-b973af22b054", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n", + " dataset = writer(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Elapsed with automatic chunk size = 57.985147991683334s\n" + ] + } + ], + "source": [ + "import time\n", + "import rioxarray as rxr\n", + "\n", + "start=time.perf_counter()\n", + "ndvi_array = (data_array_autochunks[3] - data_array_autochunks[0]) / (data_array_autochunks[3] + data_array_autochunks[0])\n", + "#mean_ndvi = ndvi_array.compute()\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_autochunks.tif\")\n", + "ndvi_array.rio.to_raster(output_file,tiled=True)\n", + "end=time.perf_counter()\n", + "print(\"Elapsed with automatic chunk size = {}s\".format((end - start)))" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "615be991-f60c-4353-8c18-fb4853c660eb", "metadata": { "tags": [] }, @@ -2086,7 +2032,7 @@ " fill: currentColor;\n", "}\n", "
<xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
-       "dask.array<open_rasterio-fc2e18048eec7fd8788f8fc93c7ebea6<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n",
+       "dask.array<open_rasterio-171af4db274709fd77ecac80bd238d05<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 2048, 2048), chunktype=numpy.ndarray>\n",
        "Coordinates:\n",
        "  * band         (band) int64 32B 1 2 3 4\n",
        "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
@@ -2105,7 +2051,7 @@
        "    NoData:               0\n",
        "    _FillValue:           0\n",
        "    scale_factor:         1.0\n",
-       "    add_offset:           0.0
" + " dtype='float64', name='y', length=41663))
  • AcquisitionDate :
    2023-04-15T11:00:24.3Z
    BlueDisplayChannel :
    0
    DataType :
    3
    GeometricLevel :
    SENSOR
    GreenDisplayChannel :
    0
    ImageID :
    6967638101-1
    Instrument :
    PHR
    InstrumentIndex :
    1A
    LinePeriod :
    0.0735
    METADATATYPE :
    OTB
    Mission :
    Pléiades
    OTB_VERSION :
    9.0.0
    ProductionDate :
    2024-04-15T13:27:06.762Z
    RedDisplayChannel :
    0
    SatAzimuth :
    35.1156
    SatElevation :
    80.4005
    SensorID :
    PHR 1A
    SunAzimuth :
    155.095
    SunElevation :
    53.1755
    SwathFirstCol :
    1
    SwathLastCol :
    39845
    TileHintX :
    2048
    TileHintY :
    2048
    TimeRangeEnd :
    2023-04-15T11:00:27.3815980Z
    TimeRangeStart :
    2023-04-15T11:00:24.3193675Z
    NoData :
    0
    _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • " ], "text/plain": [ " Size: 13GB\n", - "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n", + "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 2048, 2048), chunktype=numpy.ndarray>\n", "Coordinates:\n", " * band (band) int64 32B 1 2 3 4\n", " * x (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n", @@ -2245,26 +2239,266 @@ " add_offset: 0.0" ] }, - "execution_count": 4, + "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "input_data_array" + "reading_chunks = (-1,2048,2048)\n", + "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks,lock=False)\n", + "data_array_manualchunks" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "58bad330-9498-48ac-b3a0-cad211aaf998", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n", + " dataset = writer(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Elapsed with manual chunk size = 45.831558969803154s\n" + ] + } + ], + "source": [ + "import time\n", + "import rioxarray as rxr\n", + "\n", + "start=time.perf_counter()\n", + "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n", + "#mean_ndvi = ndvi_array.compute()\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_manualchunks.tif\")\n", + "#create_raster(mean_ndvi, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n", + "ndvi_array.rio.to_raster(output_file,tiled=True)\n", + "end=time.perf_counter()\n", + "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))" + ] + }, + { + "cell_type": "markdown", + "id": "8622d62f-a986-4893-a932-a44bc2e5c496", + "metadata": {}, + "source": [ + "Conclusion\n", + "\n", + "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. We recommand in that case defining chunks using \"(-1,sizex,sizey)\". The time gain can go about 20% !\n", + "\n", + "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of 8 for read/write efficiency in RAM." ] }, + { + "cell_type": "code", + "execution_count": null, + "id": "8cfce203-c7c2-4553-ac33-9d483f451263", + "metadata": {}, + "outputs": [], + "source": [] + }, { "cell_type": "markdown", "id": "f7746770-c831-4598-96ff-f13e1d372e1d", "metadata": {}, "source": [ - "# Parallel write" + "# Parallel write with dask vs multithreading only " + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "bfa94789-8576-4b26-a962-226db7d37f01", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(4, 41663, 39844)\n", + "(41663, 39844)\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n", + " dataset = writer(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Elapsed with RIOXarray + dask + big product = 55.51729126833379s\n", + "CPU times: user 1min 42s, sys: 28.8 s, total: 2min 11s\n", + "Wall time: 55.5 s\n" + ] + } + ], + "source": [ + "%%time\n", + "\n", + "from xarray import DataArray \n", + "from typing import List, Tuple, Union, Dict\n", + "import rioxarray as rxr\n", + "import numpy as np\n", + "import time\n", + "import rasterio\n", + "from pathlib import Path\n", + "import xarray\n", + "\n", + "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n", + "start = time.perf_counter()\n", + "reading_chunks = True\n", + "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n", + "print(input_data_array.shape)\n", + "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[0] + input_data_array[3])\n", + "#ndvi_phr.compute()\n", + "print(ndvi_phr.shape)\n", + "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n", + "ndvi_phr.rio.to_raster(output_file,tiled=True)\n", + "end = time.perf_counter()\n", + "print(\"Elapsed with RIOXarray + dask + big product = {}s\".format((end - start)))" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "4228aa64-b886-43eb-b4b0-2995df37c0e8", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
    \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
    Array Chunk
    Bytes 459.90 MiB 9.23 MiB
    Shape (2, 10980, 10980) (1, 2200, 2200)
    Dask graph 50 chunks in 5 graph layers
    Data type int16 numpy.ndarray
    \n", + "     		\n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + " 10980\n", + " 10980\n", + " 2\n", + "\n", + "
    " + ], + "text/plain": [ + "dask.array" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "input_data_array" ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 1, "id": "4a41f37f-db1b-462b-b990-5bf0db8e1c84", "metadata": { "tags": [] @@ -2289,7 +2523,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 2, "id": "81a22b13-3bff-45a9-b840-03d070bef283", "metadata": { "tags": [] @@ -2328,13 +2562,14 @@ " --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);\n", "}\n", "\n", - "html[theme=dark],\n", - "body[data-theme=dark],\n", + "html[theme=\"dark\"],\n", + "html[data-theme=\"dark\"],\n", + "body[data-theme=\"dark\"],\n", "body.vscode-dark {\n", " --xr-font-color0: rgba(255, 255, 255, 1);\n", " --xr-font-color2: rgba(255, 255, 255, 0.54);\n", " --xr-font-color3: rgba(255, 255, 255, 0.38);\n", - " --xr-border-color: #1F1F1F;\n", + " --xr-border-color: #1f1f1f;\n", " --xr-disabled-color: #515151;\n", " --xr-background-color: #111111;\n", " --xr-background-color-row-even: #111111;\n", @@ -2379,7 +2614,7 @@ ".xr-sections {\n", " padding-left: 0 !important;\n", " display: grid;\n", - " grid-template-columns: 150px auto auto 1fr 20px 20px;\n", + " grid-template-columns: 150px auto auto 1fr 0 20px 0 20px;\n", "}\n", "\n", ".xr-section-item {\n", @@ -2387,7 +2622,9 @@ "}\n", "\n", ".xr-section-item input {\n", - " display: none;\n", + " display: inline-block;\n", + " opacity: 0;\n", + " height: 0;\n", "}\n", "\n", ".xr-section-item input + label {\n", @@ -2399,6 +2636,10 @@ " color: var(--xr-font-color2);\n", "}\n", "\n", + ".xr-section-item input:focus + label {\n", + " border: 2px solid var(--xr-font-color0);\n", + "}\n", + "\n", ".xr-section-item input:enabled + label:hover {\n", " color: var(--xr-font-color0);\n", "}\n", @@ -2420,7 +2661,7 @@ "\n", ".xr-section-summary-in + label:before {\n", " display: inline-block;\n", - " content: '►';\n", + " content: \"►\";\n", " font-size: 11px;\n", " width: 15px;\n", " text-align: center;\n", @@ -2431,7 +2672,7 @@ "}\n", "\n", ".xr-section-summary-in:checked + label:before {\n", - " content: '▼';\n", + " content: \"▼\";\n", "}\n", "\n", ".xr-section-summary-in:checked + label > span {\n", @@ -2503,15 +2744,15 @@ "}\n", "\n", ".xr-dim-list:before {\n", - " content: '(';\n", + " content: \"(\";\n", "}\n", "\n", ".xr-dim-list:after {\n", - " content: ')';\n", + " content: \")\";\n", "}\n", "\n", ".xr-dim-list li:not(:last-child):after {\n", - " content: ',';\n", + " content: \",\";\n", " padding-right: 5px;\n", "}\n", "\n", @@ -2661,13 +2902,13 @@ " stroke: currentColor;\n", " fill: currentColor;\n", "}\n", - "
    <xarray.DataArray (band: 4, y: 41663, x: 39844)>\n",
-       "dask.array<open_rasterio-3fec247de0a7ca9c3f643f1ebfebdd92<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 8192, 8192), chunktype=numpy.ndarray>\n",
+       "
    <xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
+       "dask.array<open_rasterio-e809baba0b7130783671e0472b541e4c<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 8192, 8192), chunktype=numpy.ndarray>\n",
        "Coordinates:\n",
-       "  * band         (band) int64 1 2 3 4\n",
-       "  * x            (x) float64 0.5 1.5 2.5 3.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
-       "  * y            (y) float64 0.5 1.5 2.5 3.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 0\n",
+       "  * band         (band) int64 32B 1 2 3 4\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0\n",
        "Attributes: (12/29)\n",
        "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
        "    BlueDisplayChannel:   0\n",
@@ -2681,7 +2922,7 @@
        "    NoData:               0\n",
        "    _FillValue:           0\n",
        "    scale_factor:         1.0\n",
-       "    add_offset:           0.0
    "
       ],
       "text/plain": [
-       "\n",
-       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 8192, 8192), chunktype=numpy.ndarray>\n",
+       " Size: 13GB\n",
+       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 8192, 8192), chunktype=numpy.ndarray>\n",
        "Coordinates:\n",
-       "  * band         (band) int64 1 2 3 4\n",
-       "  * x            (x) float64 0.5 1.5 2.5 3.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
-       "  * y            (y) float64 0.5 1.5 2.5 3.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 0\n",
+       "  * band         (band) int64 32B 1 2 3 4\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0\n",
        "Attributes: (12/29)\n",
        "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
        "    BlueDisplayChannel:   0\n",
@@ -2815,7 +3056,7 @@
        "    add_offset:           0.0"
       ]
      },
-     "execution_count": 6,
+     "execution_count": 2,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -2826,12 +3067,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 3,
    "id": "909e5746-eee9-4f0c-a840-d3e8850dee83",
    "metadata": {
     "tags": []
    },
    "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n"
+     ]
+    },
     {
      "name": "stdout",
      "output_type": "stream",
@@ -2843,7 +3092,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:314: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
       "  dataset = writer(\n"
      ]
     },
@@ -2851,7 +3100,15 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with RIOXarray + dask + big product with write = 106.9249369930476s\n"
+      "Elapsed with RIOXarray + dask + big product with write = 182.01451965374872s\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n"
      ]
     }
    ],

From f8189fd1aad0e264288909bf5866dfcb3ad5688c Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 30 Jan 2025 10:22:28 +0100
Subject: [PATCH 07/37] add data compression example and start writing
 conclusions

---
 tuto_greenit.ipynb | 1201 ++++++++++++++------------------------------
 1 file changed, 370 insertions(+), 831 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index a59a49d..e22bde2 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -76,7 +76,7 @@
    "id": "1b0b3e1a-d31c-43c3-a662-720b0e603f14",
    "metadata": {},
    "source": [
-    "# Good pratices for searching informations\n",
+    "# Good pratices for searching informations in your data\n",
     "\n",
     "If your code needs to search elements in an image for example looking at NoData pixels, it is highly recommended to do that search in a set instead of a list of pixels :"
    ]
@@ -175,21 +175,11 @@
     "tags": []
    },
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n",
-      "Perhaps you already have a cluster running?\n",
-      "Hosting the HTTP server on port 40751 instead\n",
-      "  warnings.warn(\n"
-     ]
-    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Dask Dashboard:  http://127.0.0.1:40751/status\n"
+      "Dask Dashboard:  http://127.0.0.1:8787/status\n"
      ]
     },
     {
@@ -199,7 +189,7 @@
        "    
     
    \n",
        "    
    \n",
        "        
    Client
    \n",
-       "        
    Client-28da73fa-cdcc-11ef-95b3-00620ba4892c
    \n",
+       "        
    Client-9f81b2e0-ce90-11ef-94b3-84160c4431ea
    \n",
        "        \n",
        "\n",
        "        \n",
@@ -212,7 +202,7 @@
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
@@ -221,7 +211,7 @@
        "        
    \n",
-       "                    Dashboard:  http://127.0.0.1:40751/status\n",
+       "                    Dashboard:  http://127.0.0.1:8787/status\n",
        "                
    \n",
        "\n",
        "        \n",
-       "            \n",
        "        \n",
@@ -234,11 +224,11 @@
        "    
    \n",
        "    
    \n",
        "        
    LocalCluster
    \n",
-       "        
    ef4c3e91
    \n",
+       "        
    9a0d8c16
    \n",
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "        \n",
        "    \n",
        "
    \n",
-       "                    Dashboard: http://127.0.0.1:40751/status\n",
+       "                    Dashboard: http://127.0.0.1:8787/status\n",
        "                \n",
        "                    Workers: 4\n",
@@ -271,11 +261,11 @@
        "        
     
    \n",
        "        
    \n",
        "            
    Scheduler
    \n",
-       "            
    Scheduler-1139ebda-c011-43bf-8799-43276f0dfdb0
    \n",
+       "            
    Scheduler-ece24460-1a01-4462-b4b0-4b38a5acc434
    \n",
        "            \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                        \n",
-       "                        \n",
-       "                        \n",
+       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                    \n",
        "                        \n",
-       "                        \n",
-       "                        \n",
+       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                \n",
        "            
    \n",
-       "                        Comm: tcp://127.0.0.1:42135\n",
+       "                        Comm: tcp://127.0.0.1:42167\n",
        "                    \n",
        "                        Workers: 4\n",
@@ -283,7 +273,7 @@
        "                
    \n",
-       "                        Dashboard: http://127.0.0.1:40751/status\n",
+       "                        Dashboard: http://127.0.0.1:8787/status\n",
        "                    \n",
        "                        Total threads: 8\n",
@@ -317,7 +307,7 @@
        "                \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -362,7 +352,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:39021\n",
+       "                            Comm:  tcp://127.0.0.1:42057\n",
        "                        \n",
        "                            Total threads:  2\n",
@@ -325,7 +315,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:44165/status\n",
+       "                            Dashboard:  http://127.0.0.1:45895/status\n",
        "                        \n",
        "                            Memory:  14.00 GiB\n",
@@ -333,13 +323,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:42625\n",
+       "                            Nanny:  tcp://127.0.0.1:43477\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-29119419/dask-scratch-space/worker-p7_isrec\n",
+       "                            Local directory:  /tmp/slurm-29170710/dask-scratch-space/worker-_o85e4md\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -407,7 +397,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:35287\n",
+       "                            Comm:  tcp://127.0.0.1:35051\n",
        "                        \n",
        "                            Total threads:  2\n",
@@ -370,7 +360,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:39989/status\n",
+       "                            Dashboard:  http://127.0.0.1:33005/status\n",
        "                        \n",
        "                            Memory:  14.00 GiB\n",
@@ -378,13 +368,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:45251\n",
+       "                            Nanny:  tcp://127.0.0.1:33879\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-29119419/dask-scratch-space/worker-lm1_6txw\n",
+       "                            Local directory:  /tmp/slurm-29170710/dask-scratch-space/worker-npfxg7zg\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -452,7 +442,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:44657\n",
+       "                            Comm:  tcp://127.0.0.1:38599\n",
        "                        \n",
        "                            Total threads:  2\n",
@@ -415,7 +405,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:33045/status\n",
+       "                            Dashboard:  http://127.0.0.1:45393/status\n",
        "                        \n",
        "                            Memory:  14.00 GiB\n",
@@ -423,13 +413,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:37065\n",
+       "                            Nanny:  tcp://127.0.0.1:46333\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-29119419/dask-scratch-space/worker-2j7v2i0j\n",
+       "                            Local directory:  /tmp/slurm-29170710/dask-scratch-space/worker-80mu6_hn\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -501,7 +491,7 @@
        ""
       ],
       "text/plain": [
-       ""
+       ""
       ]
      },
      "execution_count": 1,
@@ -553,7 +543,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 2,
    "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
    "metadata": {
     "tags": []
@@ -567,23 +557,6 @@
     "phr_product_cog = \"/work/scratch/data/romaint/phr_cog.tif\""
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "0e86755c-f505-4d6b-830e-396485e12054",
-   "metadata": {},
-   "source": [
-    "# Calculation of the Average NDVI on a satellite image\n",
-    "\n",
-    "In this example, we will use what we have learned to: \n",
-    "\n",
-    "1. Read the data from the disk and stack them.\n",
-    "2. Calculate the associated NDVI, which combines multi-band information into a single band.\n",
-    "3. Reduce the information by calculating the average NDVI within a window.\n",
-    "4. Write the resulting image to the disk.\n",
-    "\n",
-    "First, let's read the data we need to perform the NDVI."
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "19ad8906-30ee-4511-aed4-a0a32abd3458",
@@ -603,7 +576,7 @@
    },
    "outputs": [],
    "source": [
-    "def open_raster_and_get_metadata(raster_paths: List[str], chunks: Union[int, Tuple, Dict, None]):\n",
+    "def open_raster_and_get_metadata(raster_paths: List[str], chunks: Union[int, Tuple, Dict, None], lock: Union[int,None]):\n",
     "    \"\"\"\n",
     "    Opens multiple raster files, extracts shared geospatial metadata, \n",
     "    and returns the concatenated data along with resolution and CRS info.\n",
@@ -622,7 +595,7 @@
     "    \"\"\"\n",
     "    results = []\n",
     "    for raster_path in raster_paths:\n",
-    "        with rxr.open_rasterio(raster_path, chunks=chunks, lock=True) as tif:\n",
+    "        with rxr.open_rasterio(raster_path, chunks=chunks, lock=lock) as tif:\n",
     "            reprojection = tif\n",
     "            transform = reprojection.rio.transform()\n",
     "            crs = reprojection.rio.crs\n",
@@ -649,6 +622,23 @@
     "        dst.write(data)\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "0e86755c-f505-4d6b-830e-396485e12054",
+   "metadata": {},
+   "source": [
+    "# Calculation of the Average NDVI on a satellite image\n",
+    "\n",
+    "In this example, we will use what we have learned to: \n",
+    "\n",
+    "1. Read the data from the disk and stack them.\n",
+    "2. Calculate the associated NDVI, which combines multi-band information into a single band.\n",
+    "3. Reduce the information by calculating the average NDVI within a window.\n",
+    "4. Write the resulting image to the disk.\n",
+    "\n",
+    "First, let's read the data we need to perform the NDVI."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "fc56a7e0-edb5-4635-b5a2-0b274d08ce38",
@@ -659,7 +649,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 4,
    "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f",
    "metadata": {
     "tags": []
@@ -684,17 +674,17 @@
        "                    \n",
        "                        \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
@@ -708,10 +698,11 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -729,10 +720,11 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -740,18 +732,20 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -767,26 +761,24 @@
        "
    \n",
-       "                            Comm:  tcp://127.0.0.1:41857\n",
+       "                            Comm:  tcp://127.0.0.1:34153\n",
        "                        \n",
        "                            Total threads:  2\n",
@@ -460,7 +450,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:43739/status\n",
+       "                            Dashboard:  http://127.0.0.1:43799/status\n",
        "                        \n",
        "                            Memory:  14.00 GiB\n",
@@ -468,13 +458,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:35379\n",
+       "                            Nanny:  tcp://127.0.0.1:44307\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-29119419/dask-scratch-space/worker-ohokz8ot\n",
+       "                            Local directory:  /tmp/slurm-29170710/dask-scratch-space/worker-lrt2z9lk\n",
        "                        
     Bytes  459.90 MiB  9.23 MiB  7.63 MiB 
     Shape  (2, 10980, 10980)  (1, 2200, 2200)  (1, 2000, 2000) 
     Dask graph  50 chunks in 5 graph layers  72 chunks in 5 graph layers 
     Data type 
    "
       ],
       "text/plain": [
-       "dask.array"
+       "dask.array"
       ]
      },
-     "execution_count": 13,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
     "# Open the raster\n",
-    "# Define our own chunks for better performance\n",
-    "reading_chunks = (-1,2200,2200)\n",
-    "#reading_chunks = True\n",
-    "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks)\n",
+    "reading_chunks = (-1,2000,2000)\n",
+    "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks, False)\n",
     "input_data_array"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 16,
    "id": "2dad7fe0-23bc-40b2-b27e-4c7cb0f8dc0e",
    "metadata": {
     "tags": []
@@ -796,8 +788,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "CPU times: user 528 ms, sys: 2.08 s, total: 2.61 s\n",
-      "Wall time: 6.19 s\n"
+      "CPU times: user 381 ms, sys: 789 ms, total: 1.17 s\n",
+      "Wall time: 1.6 s\n"
      ]
     }
    ],
@@ -823,7 +815,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 4,
    "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7",
    "metadata": {
     "tags": []
@@ -840,7 +832,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Writing /work/scratch/data/romaint/img_ndvi_otb.tif...: 100% [**************************************************] (4s)\n"
+      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif...: 100% [**************************************************] (1m 27s)\n"
      ]
     },
     {
@@ -849,7 +841,7 @@
        "0"
       ]
      },
-     "execution_count": 12,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -860,11 +852,12 @@
     "out_ndvi_otb_py=\"/work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif\"\n",
     "#Compute NDVI with OTB in python\n",
     "app_ndvi_otb = otb.Registry.CreateApplication(\"BandMath\")\n",
-    "app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n",
-    "app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n",
-    "#app_ndvi_otb.SetParameterStringList(\"il\",[phr_product])\n",
-    "#app_ndvi_otb.SetParameterString(\"exp\",\"(im1b4-im1b1)/(im1b4+im1b1)\")\n",
+    "#app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n",
+    "#app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n",
+    "app_ndvi_otb.SetParameterStringList(\"il\",[phr_product_cog])\n",
+    "app_ndvi_otb.SetParameterString(\"exp\",\"(im1b4-im1b1)/(im1b4+im1b1)\")\n",
     "app_ndvi_otb.SetParameterString(\"out\",out_ndvi_otb_py)\n",
+    "app_ndvi_otb.SetParameterInt(\"ram\",2048)\n",
     "app_ndvi_otb.ExecuteAndWriteOutput()"
    ]
   },
@@ -879,7 +872,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 7,
    "id": "73a731cf-a534-4b49-9a6d-425b58d87e45",
    "metadata": {
     "tags": []
@@ -896,7 +889,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Writing /work/scratch/data/romaint/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (4s)\n"
+      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (2s)\n"
      ]
     }
    ],
@@ -918,22 +911,30 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 41,
    "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
    "metadata": {
     "tags": []
    },
    "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n"
+     ]
+    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with Raster IO + numba = 1.2573514231480658s\n",
-      "Elapsed with Raster IO + without numba = 1.6734004551544785s\n",
-      "Elapsed with Xarray + apply ufunc = 2.9065667972899973s\n",
-      "Elapsed with RIOXarray + dask = 6.7410209202207625s\n",
-      "CPU times: user 2.53 s, sys: 8.62 s, total: 11.1 s\n",
-      "Wall time: 12.6 s\n"
+      "Elapsed with Raster IO + numba = 1.6279641180299222s\n",
+      "Elapsed with Raster IO + without numba = 1.7432440733537078s\n",
+      "Elapsed with Xarray + apply ufunc = 3.0273265461437404s\n",
+      "Elapsed with RIOXarray + dask = 4.102793791797012s\n",
+      "CPU times: user 2.25 s, sys: 5.03 s, total: 7.28 s\n",
+      "Wall time: 10.5 s\n"
      ]
     }
    ],
@@ -942,7 +943,6 @@
     "\n",
     "from numba import jit,njit\n",
     "from xarray import DataArray \n",
-    "from typing import List, Tuple, Union, Dict\n",
     "import rioxarray as rxr\n",
     "import numpy as np\n",
     "import time\n",
@@ -970,11 +970,6 @@
     "    ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n",
     "    return ndvi_array\n",
     "\n",
-    "sentinel_2_dir = \"/work/scratch/data/romaint\"\n",
-    "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
-    "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
-    "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\" \n",
-    "\n",
     "start = time.perf_counter()\n",
     "with rasterio.open(s2_b4, 'r') as ds:\n",
     "    input_data_b4 = ds.read() \n",
@@ -1013,28 +1008,24 @@
     "print(\"Elapsed with Xarray + apply ufunc = {}s\".format((end - start)))\n",
     "\n",
     "start = time.perf_counter()\n",
-    "input_data_b4 = rxr.open_rasterio(s2_b4,chunks=True)\n",
-    "input_data_b8 = rxr.open_rasterio(s2_b8,chunks=True)\n",
-    "ndvi_array = (input_data_b8 - input_data_b4) / (input_data_b8 + input_data_b4)\n",
-    "ndvi_array.compute()\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_std.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file)\n",
+    "input_data, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], (-1,2200,2200), False)\n",
+    "ndvi_array = compute_ndvi_dask(input_data[0],input_data[1])\n",
+    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_dask.tif\")\n",
+    "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n",
-    "\n",
-    "#phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
-    "#phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n",
-    "#start = time.perf_counter()\n",
-    "#reading_chunks = True\n",
-    "#input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n",
-    "#print(input_data_array.shape)\n",
-    "#ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[0] + input_data_array[3])\n",
-    "#ndvi_phr.compute()\n",
-    "#print(ndvi_phr.shape)\n",
-    "#output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n",
-    "#ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
-    "#end = time.perf_counter()\n",
-    "#print(\"Elapsed with RIOXarray + dask + big product = {}s\".format((end - start)))"
+    "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eace634b-3c4c-4db8-8c4f-92ffea2bf1e6",
+   "metadata": {},
+   "source": [
+    "**Conclusion**\n",
+    "\n",
+    "* Using RIOXarray + dask on single band products is counter productive in a case of a simple calculation as NDVI.\n",
+    "* Xarray + applyufunc also leads to low performance compared to opening the whole product with rasterio.\n",
+    "* Using numba do enhances the performance but with a low pourcentage \n"
    ]
   },
   {
@@ -1062,7 +1053,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 5,
    "id": "57ae950e-2ea3-48a9-87ba-3b8a91de7c93",
    "metadata": {
     "tags": []
@@ -1461,7 +1452,7 @@
        "    NoData:               0\n",
        "    _FillValue:           0\n",
        "    scale_factor:         1.0\n",
-       "    add_offset:           0.0"
+       "      dtype='float64', name='y', length=41663))
  • AcquisitionDate :
    2023-04-15T11:00:24.3Z
    BlueDisplayChannel :
    0
    DataType :
    3
    GeometricLevel :
    SENSOR
    GreenDisplayChannel :
    0
    ImageID :
    6967638101-1
    Instrument :
    PHR
    InstrumentIndex :
    1A
    LinePeriod :
    0.0735
    METADATATYPE :
    OTB
    Mission :
    Pléiades
    OTB_VERSION :
    9.0.0
    ProductionDate :
    2024-04-15T13:27:06.762Z
    RedDisplayChannel :
    0
    SatAzimuth :
    35.1156
    SatElevation :
    80.4005
    SensorID :
    PHR 1A
    SunAzimuth :
    155.095
    SunElevation :
    53.1755
    SwathFirstCol :
    1
    SwathLastCol :
    39845
    TileHintX :
    2048
    TileHintY :
    2048
    TimeRangeEnd :
    2023-04-15T11:00:27.3815980Z
    TimeRangeStart :
    2023-04-15T11:00:24.3193675Z
    NoData :
    0
    _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -1601,7 +1592,7 @@
        "    add_offset:           0.0"
       ]
      },
-     "execution_count": 18,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1615,7 +1606,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 6,
    "id": "b8fb6e29-0a84-4ae0-882b-b973af22b054",
    "metadata": {
     "tags": []
@@ -1626,14 +1617,22 @@
      "output_type": "stream",
      "text": [
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n"
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n"
      ]
     },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with automatic chunk size = 57.985147991683334s\n"
+      "Elapsed with automatic chunk size = 75.2928059049882s\n"
      ]
     }
    ],
@@ -1652,7 +1651,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 23,
+   "execution_count": 50,
    "id": "615be991-f60c-4353-8c18-fb4853c660eb",
    "metadata": {
     "tags": []
@@ -2051,7 +2050,7 @@
        "    NoData:               0\n",
        "    _FillValue:           0\n",
        "    scale_factor:         1.0\n",
-       "    add_offset:           0.0"
+       "      dtype='float64', name='y', length=41663))
  • AcquisitionDate :
    2023-04-15T11:00:24.3Z
    BlueDisplayChannel :
    0
    DataType :
    3
    GeometricLevel :
    SENSOR
    GreenDisplayChannel :
    0
    ImageID :
    6967638101-1
    Instrument :
    PHR
    InstrumentIndex :
    1A
    LinePeriod :
    0.0735
    METADATATYPE :
    OTB
    Mission :
    Pléiades
    OTB_VERSION :
    9.0.0
    ProductionDate :
    2024-04-15T13:27:06.762Z
    RedDisplayChannel :
    0
    SatAzimuth :
    35.1156
    SatElevation :
    80.4005
    SensorID :
    PHR 1A
    SunAzimuth :
    155.095
    SunElevation :
    53.1755
    SwathFirstCol :
    1
    SwathLastCol :
    39845
    TileHintX :
    2048
    TileHintY :
    2048
    TimeRangeEnd :
    2023-04-15T11:00:27.3815980Z
    TimeRangeStart :
    2023-04-15T11:00:24.3193675Z
    NoData :
    0
    _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -2239,20 +2238,16 @@
        "    add_offset:           0.0"
       ]
      },
-     "execution_count": 23,
+     "execution_count": 50,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
-   "source": [
-    "reading_chunks = (-1,2048,2048)\n",
-    "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks,lock=False)\n",
-    "data_array_manualchunks"
-   ]
+   "source": []
   },
   {
    "cell_type": "code",
-   "execution_count": 24,
+   "execution_count": 7,
    "id": "58bad330-9498-48ac-b3a0-cad211aaf998",
    "metadata": {},
    "outputs": [
@@ -2268,7 +2263,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with manual chunk size = 45.831558969803154s\n"
+      "Elapsed with manual chunk size = 100.88597717694938s\n"
      ]
     }
    ],
@@ -2277,11 +2272,12 @@
     "import rioxarray as rxr\n",
     "\n",
     "start=time.perf_counter()\n",
-    "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n",
-    "#mean_ndvi = ndvi_array.compute()\n",
+    "reading_chunks = (-1,2048,2048)\n",
+    "data_array_manualchunks, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product], reading_chunks, False)\n",
+    "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])[None,:,:]\n",
+    "ndvi_array.compute()\n",
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_manualchunks.tif\")\n",
-    "#create_raster(mean_ndvi, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
-    "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
+    "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))"
    ]
@@ -2316,7 +2312,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 20,
    "id": "bfa94789-8576-4b26-a962-226db7d37f01",
    "metadata": {
     "tags": []
@@ -2326,8 +2322,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "(4, 41663, 39844)\n",
-      "(41663, 39844)\n"
+      "(4, 41663, 39844)\n"
      ]
     },
     {
@@ -2342,9 +2337,9 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with RIOXarray + dask + big product = 55.51729126833379s\n",
-      "CPU times: user 1min 42s, sys: 28.8 s, total: 2min 11s\n",
-      "Wall time: 55.5 s\n"
+      "Elapsed with RasterIO + dask + product > 5Go = 71.1713912091218s\n",
+      "CPU times: user 10.2 s, sys: 40.5 s, total: 50.7 s\n",
+      "Wall time: 1min 11s\n"
      ]
     }
    ],
@@ -2362,22 +2357,21 @@
     "\n",
     "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n",
     "start = time.perf_counter()\n",
-    "reading_chunks = True\n",
-    "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n",
+    "reading_chunks = (-1,2048,2048)\n",
+    "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product], reading_chunks, False)\n",
     "print(input_data_array.shape)\n",
-    "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[0] + input_data_array[3])\n",
-    "#ndvi_phr.compute()\n",
-    "print(ndvi_phr.shape)\n",
+    "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])[None,:,:]\n",
+    "ndvi_phr.compute()\n",
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n",
-    "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
+    "create_raster(ndvi_phr, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with RIOXarray + dask + big product = {}s\".format((end - start)))"
+    "print(\"Elapsed with RasterIO + dask + product > 5Go = {}s\".format((end - start)))"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
-   "id": "4228aa64-b886-43eb-b4b0-2995df37c0e8",
+   "execution_count": 16,
+   "id": "f4346248-866b-4b31-a54c-2a25b61c431d",
    "metadata": {
     "tags": []
    },
@@ -2400,94 +2394,114 @@
        "                    \n",
        "                    
     Bytes  459.90 MiB  9.23 MiB  12.37 GiB  128.00 MiB 
     Shape  (2, 10980, 10980)  (1, 2200, 2200)  (4, 41663, 39844)  (4, 4096, 4096) 
     Dask graph  50 chunks in 5 graph layers  110 chunks in 2 graph layers 
     Data type  int16 numpy.ndarray  uint16 numpy.ndarray 
    \n",
        "        
    		\n",
-       "        \n",
+       "        \n",
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
        "  \n",
-       "  \n",
        "  \n",
        "\n",
        "  \n",
        "  \n",
        "\n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "\n",
        "  \n",
-       "  \n",
+       "  \n",
        "\n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "\n",
        "  \n",
        "  \n",
+       "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "\n",
        "  \n",
-       "  \n",
+       "  \n",
        "\n",
        "  \n",
-       "  10980\n",
-       "  10980\n",
-       "  2\n",
+       "  39844\n",
+       "  41663\n",
+       "  4\n",
        "\n",
        "        
    "
       ],
       "text/plain": [
-       "dask.array"
+       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 4096, 4096), chunktype=numpy.ndarray>"
       ]
      },
-     "execution_count": 19,
+     "execution_count": 16,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -2498,672 +2512,197 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
-   "id": "4a41f37f-db1b-462b-b990-5bf0db8e1c84",
+   "execution_count": 18,
+   "id": "909e5746-eee9-4f0c-a840-d3e8850dee83",
    "metadata": {
     "tags": []
    },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Elapsed with RIOXarray + dask + product > 5Go with tiled write = 46.12782822502777s\n"
+     ]
+    }
+   ],
+   "source": [
+    "start = time.perf_counter()\n",
+    "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n",
+    "reading_chunks = (-1,2048,2048)\n",
+    "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n",
+    "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
+    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr_rxr.tif\")\n",
+    "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
+    "end = time.perf_counter()\n",
+    "print(\"Elapsed with RIOXarray + dask + product > 5Go with tiled write = {}s\".format((end - start)))\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ffd1783b-a0e3-43f2-861d-13d043653dc6",
+   "metadata": {},
+   "source": [
+    "# Conclusions and recommandations about parallel write\n",
+    "\n",
+    "* Huge time gain without precision loss\n",
+    "* Better RAM usage\n",
+    "* You have to carefully choose your chunk size"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43",
+   "metadata": {},
+   "source": [
+    "# Optimizing the size of your data\n",
+    "\n",
+    "## Simple LZW Compression\n",
+    "\n",
+    "Can be done with gdal-translate\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "69b8bd27-926e-4373-86d1-4dafe7fdbeed",
+   "metadata": {},
    "outputs": [],
    "source": [
-    "from xarray import DataArray \n",
-    "from typing import List, Tuple, Union, Dict\n",
-    "import rioxarray as rxr\n",
-    "import numpy as np\n",
-    "import time\n",
-    "import rasterio\n",
-    "from pathlib import Path\n",
-    "import xarray\n",
+    "# Register GDAL format drivers and configuration options with a\n",
+    "# context manager.\n",
+    "with rasterio.Env():\n",
     "\n",
-    "#phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n",
-    "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
-    "reading_chunks = (-1,8192,8192)\n",
-    "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks)\n",
-    "#input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product], reading_chunks)"
+    "    # Write an array as a raster band to a new 8-bit file. For\n",
+    "    # the new file's profile, we start with the profile of the source\n",
+    "    profile = raster.profile\n",
+    "\n",
+    "    # And then change the band count to 1, set the\n",
+    "    # dtype to uint8, and specify LZW compression.\n",
+    "    profile.update(compress='lzw')\n",
+    "\n",
+    "    with rasterio.open(os.environ['TMPDIR'] + '/compressed.tif', 'w', **profile) as dst:\n",
+    "        dst.write(raster.read(1), 1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9f77319c-80a1-4060-92ff-32a5040f834f",
+   "metadata": {},
+   "source": [
+    "## CoG without overview"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
-   "id": "81a22b13-3bff-45a9-b840-03d070bef283",
+   "execution_count": null,
+   "id": "906973ba-32f1-473c-93dd-302296177950",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with rasterio.Env():\n",
+    "    profile = raster.profile\n",
+    "    profile.update(tiled=True, compress='lzw', blockxsize=512, blockysize=512)\n",
+    "    with rasterio.open(os.environ['TMPDIR'] + '/cog.tif', 'w', **profile) as dst:\n",
+    "        dst.write(raster.read(1), 1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "1071ba31-2e9b-4da5-9d94-9e6ea9a5b1db",
    "metadata": {
     "tags": []
    },
    "outputs": [
-    {
-     "data": {
-      "text/html": [
-       "
    \n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "
    <xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
-       "dask.array<open_rasterio-e809baba0b7130783671e0472b541e4c<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 8192, 8192), chunktype=numpy.ndarray>\n",
-       "Coordinates:\n",
-       "  * band         (band) int64 32B 1 2 3 4\n",
-       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
-       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 8B 0\n",
-       "Attributes: (12/29)\n",
-       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
-       "    BlueDisplayChannel:   0\n",
-       "    DataType:             3\n",
-       "    GeometricLevel:       SENSOR\n",
-       "    GreenDisplayChannel:  0\n",
-       "    ImageID:              6967638101-1\n",
-       "    ...                   ...\n",
-       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
-       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
-       "    NoData:               0\n",
-       "    _FillValue:           0\n",
-       "    scale_factor:         1.0\n",
-       "    add_offset:           0.0
    "
-      ],
-      "text/plain": [
-       " Size: 13GB\n",
-       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 8192, 8192), chunktype=numpy.ndarray>\n",
-       "Coordinates:\n",
-       "  * band         (band) int64 32B 1 2 3 4\n",
-       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
-       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 8B 0\n",
-       "Attributes: (12/29)\n",
-       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
-       "    BlueDisplayChannel:   0\n",
-       "    DataType:             3\n",
-       "    GeometricLevel:       SENSOR\n",
-       "    GreenDisplayChannel:  0\n",
-       "    ImageID:              6967638101-1\n",
-       "    ...                   ...\n",
-       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
-       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
-       "    NoData:               0\n",
-       "    _FillValue:           0\n",
-       "    scale_factor:         1.0\n",
-       "    add_offset:           0.0"
-      ]
-     },
-     "execution_count": 2,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "input_data_array"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "909e5746-eee9-4f0c-a840-d3e8850dee83",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "(41663, 39844)\n"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n"
-     ]
-    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with RIOXarray + dask + big product with write = 182.01451965374872s\n"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n"
+      "du: cannot access '/tmp/slurm-31919932/*': No such file or directory\n"
      ]
     }
    ],
    "source": [
-    "start = time.perf_counter()\n",
-    "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
-    "ndvi_phr.compute()\n",
-    "print(ndvi_phr.shape)\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n",
-    "#create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
-    "ndvi_phr.rio.to_raster(output_file)\n",
-    "end = time.perf_counter()\n",
-    "print(\"Elapsed with RIOXarray + dask + big product with write = {}s\".format((end - start)))\n"
+    "!du -sh $TMPDIR/*"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "ffd1783b-a0e3-43f2-861d-13d043653dc6",
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5f7263f1-5d4f-4025-8c74-cd83d10210aa",
    "metadata": {},
+   "outputs": [],
    "source": [
-    "# Conclusions and recommandations about parallel write\n"
+    "## Time gain after compression for an NDVI computing\n",
+    "\n",
+    "### LZW\n",
+    "\n",
+    "\n",
+    "### CoG\n"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "6ffcc752-1d32-42f0-86f6-de25ce216688",
+   "id": "6fb50a83-c68e-4a96-852c-85aa8ee90597",
    "metadata": {},
    "source": [
-    "# Performances multiprocessing example\n"
+    "Conclusion\n",
+    "\n"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "5bd9b9de-33df-48d7-b643-9e767bbc9d2a",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  },
   {
    "cell_type": "markdown",
-   "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43",
-   "metadata": {},
+   "id": "85b9e4b2-7621-48ac-8fa7-14789d40a34a",
+   "metadata": {
+    "tags": []
+   },
    "source": [
-    "# Optimizing the size of your data"
+    "# Profiling you application\n",
+    "\n",
+    "## Example with a processing chain"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "69b8bd27-926e-4373-86d1-4dafe7fdbeed",
+   "id": "ccc05e8b-cfba-4671-8995-8c6a146ebca7",
    "metadata": {},
    "outputs": [],
    "source": [
-    "using COG ?"
+    "lis_slurm_script_monitored = \"\"\"#!/bin/bash\n",
+    "#SBATCH --job-name=LIS_singularity_{cpus}_{mem}_monitored\n",
+    "#SBATCH -N 1\n",
+    "#SBATCH -n 1\n",
+    "#SBATCH -c {cpus}\n",
+    "#SBATCH --mem={mem}G            # memory per node\n",
+    "#SBATCH --time=00:30:00             # Wall Time\n",
+    "#SBATCH --account=campus       # MANDATORY : account  ( launch myaccounts to list your accounts)\n",
+    "#SBATCH --export=none               # To start the job with a clean environnement and source of ~/.bashrc\n",
+    "#SBATCH --output={share_work_dir}/LIS-{cpus}-{mem}-%j.out\n",
+    "#SBATCH --error={share_work_dir}/LIS-{cpus}-{mem}-%j.err\n",
+    "\n",
+    "module load singularity\n",
+    "module load monitoring/2.2\n",
+    "\n",
+    "cd $TMPDIR\n",
+    "\n",
+    "pin={pin}\n",
+    "pout={LIS_output_path}\n",
+    "\n",
+    "start_monitoring.sh --name lis-{cpus}-{mem} --io local\n",
+    "# monter les répertoires /data \n",
+    "singularity exec -B {share_work_dir}:/data  -B /work:/work {lis_singularity_image_path} {lis_launch_script_path} $pin $pout\n",
+    "stop_monitoring.sh --name lis-{cpus}-{mem}\n",
+    "\"\"\""
    ]
   },
   {

From 04a7bdbcb49a742a077133ceced7bac9aca36ba0 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 3 Feb 2025 11:37:39 +0100
Subject: [PATCH 08/37] Reorganise imports and conclusions

---
 tuto_greenit.ipynb | 295 +++++++++++++++++++++------------------------
 1 file changed, 140 insertions(+), 155 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index e22bde2..958a223 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -3,9 +3,13 @@
   {
    "cell_type": "markdown",
    "id": "3c6a05a0-c506-43e1-a352-c5e8e0d6f36e",
-   "metadata": {},
+   "metadata": {
+    "tags": []
+   },
    "source": [
-    "# Good pratices for loop optimisations : benefits of using numpy\n",
+    "# Good pratices\n",
+    "\n",
+    "## for loop optimisations : benefits of using numpy\n",
     "\n",
     "using a for loop vs numpy for basic loop coding shows that you must avoid for loops in your code, here is a code snippet that shows the big performance difference to apply a square on 250 000 elements"
    ]
@@ -56,27 +60,12 @@
     "print(\"Time with numpy : {} ms\".format((end-start)*1000))"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "452e6bf2-d7f3-4914-a5f5-3e9d017314d2",
-   "metadata": {
-    "tags": []
-   },
-   "source": [
-    "Conclusion\n",
-    "\n",
-    "* Avoid using for loops to apply a computing on a bunch of pixels, use numpy instead\n",
-    "* If you need a for loop for another processing, use list comprehension as much as possible, it is very well documented \n",
-    "\n",
-    "For an in-depth comparison of how much numpy is faster than for and while loops, this link provides a complete performance comparison : https://www.blog.duomly.com/loops-in-python-comparison-and-performance/"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "1b0b3e1a-d31c-43c3-a662-720b0e603f14",
    "metadata": {},
    "source": [
-    "# Good pratices for searching informations in your data\n",
+    "## Searching informations in your data\n",
     "\n",
     "If your code needs to search elements in an image for example looking at NoData pixels, it is highly recommended to do that search in a set instead of a list of pixels :"
    ]
@@ -137,9 +126,14 @@
     "tags": []
    },
    "source": [
-    "Conclusion\n",
+    "## Conclusion\n",
+    "\n",
+    "* Don't try to reinvent the wheel, use standard and well performing python libraries like numpy\n",
+    "* Avoid using for loops to apply a computing on a bunch of pixels, use numpy instead\n",
+    "* If you need a for loop for another processing, use list comprehension as much as possible, it is very well documented \n",
+    "* Use sets when you need to lookup for values in your image but beware that the set is not intended to be modified. You can alternatively use numpy.where but it is not as efficient as the set because it returns a boolean array for each element of the array, nevertheless it is 10 times more efficient than a list.\n",
     "\n",
-    "Use sets when you need to lookup for values in your image but beware that the set is not intended to be modified. You can alternatively use numpy.where but it is not as efficient as the set because it returns a boolean array for each element of the array, nevertheless it is 10 times more efficient than a list."
+    "For an in-depth comparison of how much numpy is faster than for and while loops, this link provides a complete performance comparison : https://www.blog.duomly.com/loops-in-python-comparison-and-performance/"
    ]
   },
   {
@@ -149,7 +143,63 @@
     "tags": []
    },
    "source": [
-    "# A Green IT approach for coding image processing chains\n"
+    "# A Green IT approach for coding image processing chains"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9a7fe0ae-0549-47cb-af0c-04a83d27324e",
+   "metadata": {},
+   "source": [
+    "## Necessary imports for the notebook"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "8966e0c9-d03f-4edf-b1b0-abbb4c4c2c34",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import rasterio\n",
+    "import xarray\n",
+    "import rioxarray as rxr\n",
+    "from numba import jit,njit\n",
+    "from xarray import DataArray \n",
+    "from pathlib import Path\n",
+    "from typing import List, Tuple, Union, Dict\n",
+    "from dask import delayed\n",
+    "from dask.distributed import Client, LocalCluster, Lock\n",
+    "import dask.array as da\n",
+    "from rasterio.transform import Affine\n",
+    "import time"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "200d60a9-7747-439c-a4fb-e82516575c6b",
+   "metadata": {},
+   "source": [
+    "## Work directories for the notebook"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "sentinel_2_dir = \"/work/scratch/data/romaint\"\n",
+    "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
+    "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
+    "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
+    "phr_product_cog = \"/work/scratch/data/romaint/phr_cog.tif\""
    ]
   },
   {
@@ -169,7 +219,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 2,
    "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21",
    "metadata": {
     "tags": []
@@ -189,7 +239,7 @@
        "    
     
    \n",
        "    
    \n",
        "        
    Client
    \n",
-       "        
    Client-9f81b2e0-ce90-11ef-94b3-84160c4431ea
    \n",
+       "        
    Client-e89e21b5-e20d-11ef-84df-00620ba482fc
    \n",
        "        \n",
        "\n",
        "        \n",
@@ -224,7 +274,7 @@
        "    \n",
        "    
    \n",
        "        
    LocalCluster
    \n",
-       "        
    9a0d8c16
    \n",
+       "        
    427581f7
    \n",
        "        
    \n",
        "            \n",
        "                \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
        "            \n",
@@ -261,11 +311,11 @@
        "        
     
    \n",
        "        
    \n",
        "            
    Scheduler
    \n",
-       "            
    Scheduler-ece24460-1a01-4462-b4b0-4b38a5acc434
    \n",
+       "            
    Scheduler-3ef03144-bde0-4857-9ed2-3b85e5982644
    \n",
        "            
    \n",
@@ -236,10 +286,10 @@
        "            
    \n",
-       "                    Total threads: 8\n",
+       "                    Total threads: 4\n",
        "                \n",
-       "                    Total memory: 56.00 GiB\n",
+       "                    Total memory: 28.00 GiB\n",
        "                
    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                \n",
@@ -284,7 +334,7 @@
        "                        Started: Just now\n",
        "                    \n",
        "                    \n",
        "                \n",
        "            
    \n",
-       "                        Comm: tcp://127.0.0.1:42167\n",
+       "                        Comm: tcp://127.0.0.1:40559\n",
        "                    \n",
        "                        Workers: 4\n",
@@ -276,7 +326,7 @@
        "                        Dashboard: http://127.0.0.1:8787/status\n",
        "                    \n",
-       "                        Total threads: 8\n",
+       "                        Total threads: 4\n",
        "                    
    \n",
-       "                        Total memory: 56.00 GiB\n",
+       "                        Total memory: 28.00 GiB\n",
        "                    
    \n",
@@ -307,29 +357,29 @@
        "                \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -352,29 +402,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:42057\n",
+       "                            Comm:  tcp://127.0.0.1:43405\n",
        "                        \n",
-       "                            Total threads:  2\n",
+       "                            Total threads:  1\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:45895/status\n",
+       "                            Dashboard:  http://127.0.0.1:38755/status\n",
        "                        \n",
-       "                            Memory:  14.00 GiB\n",
+       "                            Memory:  7.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:43477\n",
+       "                            Nanny:  tcp://127.0.0.1:39751\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-29170710/dask-scratch-space/worker-_o85e4md\n",
+       "                            Local directory:  /tmp/slurm-32111092/dask-scratch-space/worker-8wzm9nm6\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -397,29 +447,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:35051\n",
+       "                            Comm:  tcp://127.0.0.1:43803\n",
        "                        \n",
-       "                            Total threads:  2\n",
+       "                            Total threads:  1\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:33005/status\n",
+       "                            Dashboard:  http://127.0.0.1:33143/status\n",
        "                        \n",
-       "                            Memory:  14.00 GiB\n",
+       "                            Memory:  7.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:33879\n",
+       "                            Nanny:  tcp://127.0.0.1:34299\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-29170710/dask-scratch-space/worker-npfxg7zg\n",
+       "                            Local directory:  /tmp/slurm-32111092/dask-scratch-space/worker-ls1alhze\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -442,29 +492,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:38599\n",
+       "                            Comm:  tcp://127.0.0.1:38291\n",
        "                        \n",
-       "                            Total threads:  2\n",
+       "                            Total threads:  1\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:45393/status\n",
+       "                            Dashboard:  http://127.0.0.1:44757/status\n",
        "                        \n",
-       "                            Memory:  14.00 GiB\n",
+       "                            Memory:  7.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:46333\n",
+       "                            Nanny:  tcp://127.0.0.1:39203\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-29170710/dask-scratch-space/worker-80mu6_hn\n",
+       "                            Local directory:  /tmp/slurm-32111092/dask-scratch-space/worker-utct9_is\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -491,28 +541,15 @@
        ""
       ],
       "text/plain": [
-       ""
+       ""
       ]
      },
-     "execution_count": 1,
+     "execution_count": 2,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "from pathlib import Path\n",
-    "\n",
-    "from typing import List, Tuple, Union, Dict\n",
-    "import dask.array as da\n",
-    "import numpy as np\n",
-    "import rasterio\n",
-    "import rioxarray as rxr\n",
-    "from dask import delayed\n",
-    "from dask.distributed import Client, LocalCluster, Lock\n",
-    "from rasterio.transform import Affine\n",
-    "\n",
-    "from utils import create_map_with_rasters\n",
-    "\n",
     "cluster = LocalCluster()\n",
     "client = Client(cluster)\n",
     "\n",
@@ -533,30 +570,6 @@
     "From this method we will use the ``chunks`` and ``lock`` arguments, which respectively set a chunk size and limit access to the data to one thread at a time to avoid read problems. Here ``chunks`` is set to ``True`` to allow dask to automatically size chunks.\n"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "200d60a9-7747-439c-a4fb-e82516575c6b",
-   "metadata": {},
-   "source": [
-    "## Work directories"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "sentinel_2_dir = \"/work/scratch/data/romaint\"\n",
-    "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
-    "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
-    "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
-    "phr_product_cog = \"/work/scratch/data/romaint/phr_cog.tif\""
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "19ad8906-30ee-4511-aed4-a0a32abd3458",
@@ -619,7 +632,7 @@
     "            crs=crs,\n",
     "            transform=transform\n",
     "    ) as dst:\n",
-    "        dst.write(data)\n"
+    "        dst.write(data)"
    ]
   },
   {
@@ -649,7 +662,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 7,
    "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f",
    "metadata": {
     "tags": []
@@ -674,17 +687,17 @@
        "                    \n",
        "                        \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
@@ -698,11 +711,10 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -720,11 +732,10 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -732,20 +743,18 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -761,24 +770,24 @@
        "
    \n",
-       "                            Comm:  tcp://127.0.0.1:34153\n",
+       "                            Comm:  tcp://127.0.0.1:32857\n",
        "                        \n",
-       "                            Total threads:  2\n",
+       "                            Total threads:  1\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:43799/status\n",
+       "                            Dashboard:  http://127.0.0.1:36677/status\n",
        "                        \n",
-       "                            Memory:  14.00 GiB\n",
+       "                            Memory:  7.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:44307\n",
+       "                            Nanny:  tcp://127.0.0.1:33445\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-29170710/dask-scratch-space/worker-lrt2z9lk\n",
+       "                            Local directory:  /tmp/slurm-32111092/dask-scratch-space/worker-he9e73nn\n",
        "                        
     Bytes  459.90 MiB  7.63 MiB  9.23 MiB 
     Shape  (2, 10980, 10980)  (1, 2000, 2000)  (1, 2200, 2200) 
     Dask graph  72 chunks in 5 graph layers  50 chunks in 5 graph layers 
     Data type 
    "
       ],
       "text/plain": [
-       "dask.array"
+       "dask.array"
       ]
      },
-     "execution_count": 4,
+     "execution_count": 7,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
     "# Open the raster\n",
-    "reading_chunks = (-1,2000,2000)\n",
+    "reading_chunks = (-1,2200,2200)\n",
     "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks, False)\n",
     "input_data_array"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 8,
    "id": "2dad7fe0-23bc-40b2-b27e-4c7cb0f8dc0e",
    "metadata": {
     "tags": []
@@ -788,8 +797,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "CPU times: user 381 ms, sys: 789 ms, total: 1.17 s\n",
-      "Wall time: 1.6 s\n"
+      "CPU times: user 454 ms, sys: 1.26 s, total: 1.72 s\n",
+      "Wall time: 3.9 s\n"
      ]
     }
    ],
@@ -815,7 +824,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 9,
    "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7",
    "metadata": {
     "tags": []
@@ -832,7 +841,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif...: 100% [**************************************************] (1m 27s)\n"
+      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif...: 100% [**************************************************] (1m 34s)\n"
      ]
     },
     {
@@ -841,7 +850,7 @@
        "0"
       ]
      },
-     "execution_count": 4,
+     "execution_count": 9,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -941,15 +950,6 @@
    "source": [
     "%%time\n",
     "\n",
-    "from numba import jit,njit\n",
-    "from xarray import DataArray \n",
-    "import rioxarray as rxr\n",
-    "import numpy as np\n",
-    "import time\n",
-    "import rasterio\n",
-    "from pathlib import Path\n",
-    "import xarray\n",
-    "\n",
     "@njit\n",
     "def one_pixel_ndvi(p1,p2):\n",
     "    return (p2-p1) / (p2+p1) \n",
@@ -1021,7 +1021,7 @@
    "id": "eace634b-3c4c-4db8-8c4f-92ffea2bf1e6",
    "metadata": {},
    "source": [
-    "**Conclusion**\n",
+    "## Conclusion\n",
     "\n",
     "* Using RIOXarray + dask on single band products is counter productive in a case of a simple calculation as NDVI.\n",
     "* Xarray + applyufunc also leads to low performance compared to opening the whole product with rasterio.\n",
@@ -1637,9 +1637,6 @@
     }
    ],
    "source": [
-    "import time\n",
-    "import rioxarray as rxr\n",
-    "\n",
     "start=time.perf_counter()\n",
     "ndvi_array = (data_array_autochunks[3] - data_array_autochunks[0]) / (data_array_autochunks[3] + data_array_autochunks[0])\n",
     "#mean_ndvi = ndvi_array.compute()\n",
@@ -2268,9 +2265,6 @@
     }
    ],
    "source": [
-    "import time\n",
-    "import rioxarray as rxr\n",
-    "\n",
     "start=time.perf_counter()\n",
     "reading_chunks = (-1,2048,2048)\n",
     "data_array_manualchunks, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product], reading_chunks, False)\n",
@@ -2287,7 +2281,7 @@
    "id": "8622d62f-a986-4893-a932-a44bc2e5c496",
    "metadata": {},
    "source": [
-    "Conclusion\n",
+    "## Conclusion\n",
     "\n",
     "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. We recommand in that case defining chunks using \"(-1,sizex,sizey)\". The time gain can go about 20% !\n",
     "\n",
@@ -2346,15 +2340,6 @@
    "source": [
     "%%time\n",
     "\n",
-    "from xarray import DataArray \n",
-    "from typing import List, Tuple, Union, Dict\n",
-    "import rioxarray as rxr\n",
-    "import numpy as np\n",
-    "import time\n",
-    "import rasterio\n",
-    "from pathlib import Path\n",
-    "import xarray\n",
-    "\n",
     "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n",
     "start = time.perf_counter()\n",
     "reading_chunks = (-1,2048,2048)\n",
@@ -2543,7 +2528,7 @@
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr_rxr.tif\")\n",
     "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with RIOXarray + dask + product > 5Go with tiled write = {}s\".format((end - start)))\n"
+    "print(\"Elapsed with RIOXarray + dask + product > 5Go with tiled write = {}s\".format((end - start)))"
    ]
   },
   {
@@ -2555,7 +2540,7 @@
     "\n",
     "* Huge time gain without precision loss\n",
     "* Better RAM usage\n",
-    "* You have to carefully choose your chunk size"
+    "* You have to carefully choose your chunk size using a multiple of the cog size"
    ]
   },
   {

From a757461730de64281737c571dabda7aee2f8908f Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 3 Feb 2025 17:32:49 +0100
Subject: [PATCH 09/37] Add performance graphs computed on slurm

---
 tuto_greenit.ipynb | 121 +++++++++++++++++++++++++++++++--------------
 1 file changed, 84 insertions(+), 37 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 958a223..74a45cd 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -913,7 +913,7 @@
    "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386",
    "metadata": {},
    "source": [
-    "# Optimizing the computing of NDVI\n",
+    "# Testing optimisation methods for the computing of NDVI\n",
     "\n",
     "Here we will try to use numba for ndvi computation"
    ]
@@ -1035,7 +1035,7 @@
     "tags": []
    },
    "source": [
-    "## Differences between using Compute vs using RIO tiled write"
+    "# Optimize dask parameters and improve write speeds on disk"
    ]
   },
   {
@@ -1043,7 +1043,7 @@
    "id": "d3ab5833-1672-478a-8b26-0ed2bc4d6ee1",
    "metadata": {},
    "source": [
-    "## Optimize the chunk size for dask\n",
+    "## Find the right chunk size for dask\n",
     "\n",
     "Chunk size is becoming very important when your data size grows. You can let chunks=True to dask which will automatically determine a chunk size\n",
     "Most of the time this chunk size is coherent but users can tweak it to be more efficient.\n",
@@ -2301,7 +2301,7 @@
    "id": "f7746770-c831-4598-96ff-f13e1d372e1d",
    "metadata": {},
    "source": [
-    "# Parallel write with dask vs multithreading only "
+    "# Write with dask vs tiled write with rioxarray "
    ]
   },
   {
@@ -2340,17 +2340,15 @@
    "source": [
     "%%time\n",
     "\n",
-    "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n",
     "start = time.perf_counter()\n",
     "reading_chunks = (-1,2048,2048)\n",
-    "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product], reading_chunks, False)\n",
-    "print(input_data_array.shape)\n",
-    "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])[None,:,:]\n",
+    "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
+    "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "ndvi_phr.compute()\n",
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n",
-    "create_raster(ndvi_phr, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
+    "ndvi_phr.rio.to_raster(output_file)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with RasterIO + dask + product > 5Go = {}s\".format((end - start)))"
+    "print(\"Elapsed with RIOXarray + dask compute + product > 5Go = {}s\".format((end - start)))"
    ]
   },
   {
@@ -2521,14 +2519,60 @@
    ],
    "source": [
     "start = time.perf_counter()\n",
-    "phr_product = \"/work/scratch/data/romaint/phr_cog.tif\"\n",
     "reading_chunks = (-1,2048,2048)\n",
-    "input_data_array = rxr.open_rasterio(phr_product,chunks=reading_chunks,lock=False)\n",
+    "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr_rxr.tif\")\n",
     "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with RIOXarray + dask + product > 5Go with tiled write = {}s\".format((end - start)))"
+    "print(\"Elapsed with RIOXarray + tiled write + product > 5Go = {}s\".format((end - start)))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f55307be-456b-4a54-9415-9f46b61837fa",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Performance graphs\n",
+    "\n",
+    "**The first computing stops at 53s in these graphs that were generated using a slurm script with the exact same code as above**"
+   ]
+  },
+  {
+   "attachments": {
+    "77035e47-e99e-4a83-a3c2-cee41de28d29.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAusAAAHTCAYAAACeOFZgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90\nbGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9h\nAAAPYQGoP6dpAACBqElEQVR4nO3deXhTZfo38O/J3nQJ3WgoFChQFmURARFcQNlccENFRWbgB44o\niKAwKDKj1VfBQcUFRhwdFBWZMjOK+wKooAwiiKBsskjZWyqldG+SJuf9Iz0nrW0hpUnOku/nunJB\nk5PkySH0uXPnfu5HEEVRBBERERERqY5B6QEQEREREVHDGKwTEREREakUg3UiIiIiIpVisE5ERERE\npFIM1omIiIiIVIrBOhERERGRSjFYJyIiIiJSKQbrREREREQqxWCdiIiIiEilGKwTUdC+//573HTT\nTWjbti2sVivS0tIwYMAAzJgxo85x7du3x8iRIxt8jB9++AGCIGDp0qXyddnZ2RAEAQaDAQcOHKh3\nn/LyciQkJEAQBIwfP/6s41y8eDHat2+PxMREjB07FqdPn65ze3V1NS644AI8+uijjT7GiRMn8PDD\nD6NHjx6Ii4uDzWZDVlYWpk2bhn379tUbu3SxWCzIzMzEtGnT6jyvdNzJkycbfL7u3btj8ODBZ31t\nkpdeegnJycmorq4O+j5ERKQ9DNaJKCiffPIJBg4ciJKSEsyfPx+rVq3Ciy++iEsuuQQrVqwIyXPE\nxcXhjTfeqHf9f/7zH3g8HpjN5rM+xjfffIOpU6figQcewLJly7Bp0ybMnDmzzjELFixARUUF5syZ\n0+BjbNq0CT169MCSJUtwyy234L333sPnn3+OmTNn4scff8RFF11U7z6ff/45vvvuO3zyySe48cYb\nsXDhQlx99dUQRTHIV9807777Lm644QaYTKawPD4REakDf8sTUVDmz5+PzMxMfPHFF3UCxNtvvx3z\n588PyXPcdtttePPNN/H444/DYAjkEpYsWYKbbroJH3744Vkf45NPPsGQIUMwbdo0AEBxcTEefPBB\n+fbc3Fw8/vjj+Pjjj2G1Wuvdv6SkBDfccANsNhs2bNiANm3ayLcNHjwYkyZNwn//+9969+vTpw9S\nUlIAAMOGDUNhYSHefvttbNiwAZdccknwJyEIJ06cwPr16zFr1qyQPi4REakPM+tEFJTCwkKkpKQ0\nmMmtHVg3x4QJE3DkyBGsXr1avm7v3r1Yv349JkyYENRjVFVVITY2Vv45Li4OVVVV8s/33nsvbrvt\nNlxxxRUN3v+1115Dfn4+5s+fXydQr+2WW2456zguvvhiAMChQ4eCGndTrFy5EnFxcRg6dCgAoKKi\nAjNnzkRmZiZsNhuSkpLQt29f/Otf/wr5cxMRUWQxWCeioAwYMADff/897r//fnz//ffweDwhf46s\nrCxcdtlleP311+XrXn/9dbRv3x5DhgwJ6jEGDhyIVatW4bvvvkNBQQFeeuklDBw4EACwfPly/Pjj\nj3jmmWcavf+qVatgNBpx3XXXNeu17N+/HwCQmprarMdpyLvvvouRI0fK3ww8+OCDWLx4Me6//358\n/vnnePvtt3HrrbeisLAw5M9NRESRxTIYIgrK008/jV9++QULFy7EwoULYTab0a9fP1x33XW47777\nEBcXF5LnmTBhAu655x6cOnUKDocDb731FiZNmgRBEIK6/+jRo/HZZ5/JAXqXLl3w0Ucf4dSpU3jg\ngQewYMECJCcnN3r/w4cPIzU1tU52PhherxfV1dUoKyvDJ598gldeeQUZGRm47LLLmvQ4Z1NYWIi1\na9fWWSfwv//9D8OHD8cDDzwgX3fttdeG9HmJiEgZzKwTUVCSk5Px7bffYvPmzXj66adxww03YO/e\nvZg9ezZ69OjRaJeTprr11lthsVjwzjvv4NNPP0V+fn5QHWAkUqeZgoIC7Nu3D7t27UJWVhb+/Oc/\no1evXhg7diy2b9+OQYMGITExEX379sW3337b7HE7nU6YzWa5A82FF16Izz//HDabrdmPXdsHH3wA\ni8WCq666Sr7uoosuwmeffYaHH34Ya9euRWVlZUifk4iIlMPMOhE1Sd++fdG3b18AgMfjwUMPPYTn\nn38e8+fPlxeamkwmeL3eBu8vtRpsrLNLbGwsbrvtNrz++uto164dhg4dinbt2jV5nKmpqXIJyrp1\n65CTk4Off/4ZHo8HN954I8aOHSuXjNxwww3Yv38/kpKS0LZtW+zbtw/l5eVNyq6vWbMGDocDZrMZ\nbdq0qZe9l2r9z3Regul289///hdXX3017Ha7fN1LL72ENm3aYMWKFfjb3/4Gm82GESNG4JlnnkFW\nVlbQr4GIiNSHmXUiOmdmsxmPPfYYAGDHjh3y9WlpaTh27FiD95GuT0tLa/RxJ0yYgG3btuGjjz4K\nemFpY1wuFyZNmoS//vWv6NixI/bs2YMDBw5g5syZiImJwd133w1BEPDdd98BAEaMGAGv14uPPvqo\nSc/Tq1cv9O3bF7169WqwzEZ6vQ2dF1EUkZeXd8ZzAvg723z55Ze4+eab61wfGxuLxx9/HL/88gvy\n8/OxePFibNy4sdl190REpDwG60QUlLy8vAav3717NwAgPT1dvm7o0KHYsWMHdu3aVe/4f//734iL\ni0P//v0bfa4BAwZgwoQJuOmmm3DTTTc1a9xz586FxWKRe61Lfc/Ly8sB+L8dcLlc8vUTJ06E0+nE\nrFmzGv3A8d577zV5HFdeeSUEQWiwJ/3nn3+OkpISubtLYz766CMIgtDohlOA/0PB+PHjcccdd2DP\nnj2oqKho8liJiEg9WAZDREEZMWIE2rRpg+uuuw5du3aFz+fDtm3b8NxzzyEuLk7uaw4A06ZNw1tv\nvYXBgwfjkUceQY8ePVBUVIQVK1bgv//9LxYsWID4+PgzPt+SJUuaPeZffvkF8+fPx9dffy2XoXTp\n0gXt2rXDvffeiylTpmDFihUwmUxyq0WHw4EPPvgAI0eORO/evXHfffdhwIABsFgs2LdvH5YtW4af\nfvoJo0aNatJYOnbsiPvuuw/PPPMMTp8+jWuuuQYxMTHyGoC+fftizJgxZ3yM//73vxg2bFi9c9e/\nf3+MHDkSPXv2RGJiInbv3o23334bAwYMqFMuQ0REGiQSEQVhxYoV4pgxY8SsrCwxLi5ONJvNYtu2\nbcU//OEP4q5du+odn5+fL957771i27ZtRZPJJMbHx4uXXnqp+J///KfesY899pgIQPztt9/OOIbY\n2Fhx3LhxQY3X5/OJl112mThlypR6t23ZskW8+OKLxdjYWLFHjx7imjVrGhz/Qw89JJ5//vmi3W4X\nrVar2KlTJ3HSpEni9u3bmzx2aUyLFy8W+/btK9rtdtFisYhZWVniQw89JJaWlp7xvmVlZaLNZhPf\neOONerc9/PDDYt++fcXExETRarWKHTp0EB944AHx5MmTZx0TERGpmyCKYdoLm4iIQubf//437rzz\nTpw4cQJJSUlKD4eIiCKEwToRERERkUpxgSkRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhI\npRisExERERGpFDdFAuDz+XD8+HHEx8dDEASlh0NEREQRIIoiSktLkZ6eDoOB+UtSJwbrAI4fP46M\njAylh0FEREQKOHLkCNq0aaP0MIgaxGAdkLfuPnLkCBISEhQeDREREUVCSUkJMjIy5DiASI0YrANy\n6UtCQgKDdSIioijDElhSMxZoERERERGpFIN1IiIiIiKVYrBORERERKRSDNaJiIiIiFSKwToRERER\nkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExERERGpFIN1IiIiIiKVYrBORERERKRSDNaJ\niIiIiFSKwToRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExERERGpFIN1IiIiIiKV\nYrBORERERKRSDNaJiIiIiFSKwToRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExER\nERGpFIN1IiIiIiKVYrBORERERKRSDNaJiIiIiFSKwToRERERkUoxWCciIiIiUikG60QKOVxYgY0H\nCpUeBhEREakYg3UiBfxW6sINf1+PO17biNyT5UoPh4iIiFSKwTqRAh79YAeKKjwQRWB3XonSwyEi\nIiKVUjRYb9++PQRBqHeZMmUKAEAURWRnZyM9PR0xMTEYPHgwdu7cWecxXC4Xpk6dipSUFMTGxuL6\n66/H0aNHlXg5REH55Oc8fLYjX/6ZmXUiIiJqjKLB+ubNm5GXlydfVq9eDQC49dZbAQDz58/HggUL\nsGjRImzevBlOpxPDhg1DaWmp/BjTp0/HypUrkZOTg/Xr16OsrAwjR46E1+tV5DURnUlhmQt//WAH\nAKBlvBUAcKiQwToRERE1TNFgPTU1FU6nU758/PHH6NixIwYNGgRRFPHCCy9gzpw5GDVqFLp37443\n33wTFRUVWL58OQCguLgYS5YswXPPPYehQ4eid+/eWLZsGbZv3441a9Yo+dKIGvTohztxqtyNrs54\nzLqqKwDgYGGFwqMiIiIitVJNzbrb7cayZcswYcIECIKA3Nxc5OfnY/jw4fIxVqsVgwYNwoYNGwAA\nW7ZsgcfjqXNMeno6unfvLh/TEJfLhZKSkjoXIsBfehUun23Pwyc/58FoEPDMLb2Q1TIOADPrRERE\n1DjVBOvvv/8+Tp8+jfHjxwMA8vP9Nb1paWl1jktLS5Nvy8/Ph8ViQWJiYqPHNGTevHlwOBzyJSMj\nI4SvhLTq6c9+wYB5X+GHg6dC/tinyt1y+cu9gzqiRxsH2ifHAgBOlLhQ4a4O+XMSERGR9qkmWF+y\nZAmuvvpqpKen17leEIQ6P4uiWO+63zvbMbNnz0ZxcbF8OXLkyLkPnHRjze4TyC+pwvg3NuPHw0Uh\nfezHP9qJk2VudE6Lw9QhnQAADrsZiXYzAOAQS2GIiIioAaoI1g8dOoQ1a9bgrrvukq9zOp0AUC9D\nXlBQIGfbnU4n3G43ioqKGj2mIVarFQkJCXUuRJVu/6LkMlc1xi3ZhJ+OnA7J436xMx8fbDsOgwA8\nc0svWE1G+bZ2Ndl1lsIQERFRQ1QRrL/xxhto2bIlrr32Wvm6zMxMOJ1OuUMM4K9rX7duHQYOHAgA\n6NOnD8xmc51j8vLysGPHDvkYomBVevzBevtkO0pd1fjDku+x41hxsx7zdIUbc1b6y18mDeqIXhkt\n6tzePtkOgItMiYiIqGGKB+s+nw9vvPEGxo0bB5PJJF8vCAKmT5+OuXPnYuXKldixYwfGjx8Pu92O\nMWPGAAAcDgcmTpyIGTNm4Msvv8TWrVsxduxY9OjRA0OHDlXqJZFGSXXjr/yhD/q0S0RJVTXGLvke\nu46f+wLkJz7ahZNlLnRqGYdpQ7Lq3S5l1g+y1zoRERE1wHT2Q8JrzZo1OHz4MCZMmFDvtlmzZqGy\nshKTJ09GUVER+vfvj1WrViE+Pl4+5vnnn4fJZMLo0aNRWVmJIUOGYOnSpTAajfUej6gxPp+IKo8P\nAJASZ8XS/+uHPyzZhG1HTmPsku/xrz9djC7O+LM8Sl1f7j6B97Yeg0EA5t/SEzZz/fdkZkpNsM4y\nGCIiImqAIIazV51GlJSUwOFwoLi4mPXrUarCXY3zHv0CALDz8RGItZpQUuXB2H9+j5+PFiMlzoKc\nuy9Gp5bBBezFFR4Mf2EdTpS4cPflHfDINd0aPG7r4SLc9PIGtHLY8N3sISF7PaQd7mofthwqQu+2\nLRr8QEdE4cP5n7RA8cw6kRpIi0sBIKYmYEqwmfH2hP4Y88+N2Hm8BHe89j1y7r4YHVPjGn0cURRx\nvLgKT3/2C06UuNAhJRYPDuvc6PFS+8a84ipUebwM1qLQkvW5+Nvnv6BbqwT8fUxvdDjD+6u53NU+\nrNl9At/9WgijQUCMxQibyYgYiwE2s//vNosRNpMBcVYT+rRPrLMgmoiIIo/BOhGAippg3WoywGAI\ntP102M1YNrE/7nhtI37JL8WY1zZixd0D0D4lFj6fiMOnKrDjeDF2HCvBzuPF2HGsGEUVHgCAIADP\n3Npw+Yukhd2MBJsJJVXVOFRY0eRSG9K+46crAQC780pw3cL1mHdzT1zfK/0s92qa/QWlWLH5CN77\n8RgKy91B3+/GC9Lxwu29QzoWIiJqGgbrRACqajrB2C31A+vEWAveucsfsO89UYbbXv0O7ZNjset4\nCUpd9TczMhkEZKXF4/8uaY8+7ZLO+LyCICAzJRY/HS3GwcJyButRyFUdeO+Vu724/19bsfFAIR4d\neV6zvmmpcFfjk5/zsGLzEfxwKNDetmW8FSN7piPGYkCl24eqai+qPNLFhyqPFydKqvDrb+XI5cJn\nIiLFMVgnQiCzHtNIcJQcZ8U7d12MO17biP0FZThR4gIAWEwGdGuVgPPTE9A93YHurRPQOS2+SUFW\nu2R/sM5e69HJVe1f2Dx9aBZKq6qx6Ov9WP79YWw9fLrJZTGiKGLHsRLkbD6MD7cdlz9MGg0CrujS\nErf3y8DgLqkwGc/cCGz9vpMYu+R7edE1EREph8E6EQI91mMayKxLUuOtyLn7Yiz//jDSW8Sge+sE\ndEyNg/ksgc/ZsNd6dHPVBMQxFhPuvrwjLspMwvScbXJZzNxRPXDDBa0bvX+Vx4uNBwqxbu9vWLf3\nNxz4LfChr12yHaP7ZuCWPm2QlmALekw2s/89XVXtPcuRREQUbgzWiRBYYHqmYB3wt3W8v4F+6c3B\nXuvRTSqDsZr8AfJlWan4dNpluP9fW/F97ilMy9mGjQdO4bHr/GUxoihif0GZHJx/n3sK7upABtxi\nMuDq7k7c1i8DF2cm11mDESzpmyGpPIyIiJTDYJ0Igcy63Rz5/xLta3qtH2JmPSpJZTBSsA4AaQk2\nvHNXf7z45T4s+no//rXpMLYeLsIFGS2wbu9vyCuuqvMY6Q4bBnVJxeVZqbgkKwUJNnOzxiRn1lkG\nQ0SkOAbrRAjUrNvOklkPB6kM5nhxJds3RqFAsF73391kNGDG8C5yWcwv+aX4Jb8UgD97fnGHZFye\nlYLBXVLRMTUOgtD0DHpjpLEws05EpDwG60SonVmPfKCcFGtBvNWEUlc1jhZVBL3xEumDXAZjbnjt\ng1QWs3jtrzAIAi7vnIL+mclnLdlqDukDo6vaB1EUQ/pBgIiImobBOhGASre/a0Y4A6DGCIKAdil2\n7DhWgtyTDNajjbTAtHYZzO+lJdiQff35kRqSXAYD+AN2fttDRKSc5rWxoKhyoqQKOZsOy5lAPal0\nSx05lAlKpJ1M2b4x+ri9DZfBKKl2cM5SGCIiZTGzTkHx+kTc9eYP2H6sGL/kl0Y0yxcJFZ6azLpC\nGUQpWD/IYD3qBJNZjzSz0QCjQYDXJ3KRKRGRwtQzO5Cqrdh8BNuPFQMA3vzuILYeLjrLPbSlyt34\nDqaR0K5mkSk7wkQf6ZsqWyM160qxmaSOMMysExEpSV2zA6lSUbkb87/4BQDQukUMRBGY/d52eLz6\nybjJ3WCUyqzXtG/k9u7Rp7FuMEqTe63rsOyNiEhLGKzTWT27ag9OV3jQ1RmPlVMGItFuxi/5pXj1\nmwNKDy1k5G4wCtesHz9dqcs1AdS4hvqsq0FgYyT9fCgnItIidc0OpDrbjxZj+abDAIDHrz8fLeNt\n+OvI8wAAL365TzeZYHkHU4Uy6ylxFsRajPCJwNGiSkXGQJFX7fXB6xMBqC+zLrWSdLEMhohIUQzW\nqVE+n4hHP9wBUQRuvCAd/TskAwBu6t0al2WlwF3tw5yV2yGKosIjbT4ps65UNxhBENCOHWGijpRV\nBxrvs64Um7QxUjUz60RESlLX7ECq8t8fj2Lr4dOItRgx+5pu8vWCIOCpG3vAZjZgw6+FePfHYwqO\nMjQqFM6sA0CmXLfORabRonawbjGq69extOCVC0yJiJSlrtmBVKO4woO/feZfVDp9aGekJdjq3N42\n2Y7pQzsDAJ78ZBdOlrkiPsZQqpJr1pXrZhroCMPMerSQ1idYjAYYDOraJTRQs85gnYhISQzWqUHP\nr9mLwnI3OrWMw/hL2jd4zMRLM9GtVQJOV3jw5Me7IjvAEJMz6xbl/ksEeq0zsx4tpB7rFpUtLgUC\nwbqLC0yJiBSlvhmCFLfreAne+u4gAP+iUnMjX8+bjQY8PaoHDALw/rbjWLf3twiOMrTkmnUzM+sU\nOWrtBAPUKoNhdyIiIkWpb4YgRYmiiMc+3AGfCFzboxUu6ZRyxuN7ZbTA+IGZAIA5K7ejwl0diWGG\nnNwNRqEFpkCgZv1oUaWuethT46QyGFUG6yaWwRARqYH6ZghS1PvbjmHzwSLEmI2Yc223s98BwIzh\nndG6RQyOFlXihTX7wjzC0BNFUfE+6wCQGm9FjNkIr09k+8YoIWfWFVzY3Bgr+6wTEakCg3WSlVZ5\nMPdT/6LS+67shPQWMUHdL9Zqwv+78XwAwD+/PYAdx4rDNsZwcNfqda3UDqaA1L7RXwpzkKUwUUGq\nB1dlZp3dYIiIVEF9MwQp5sU1+/BbqQuZKbG467LMJt33yq5puLZnK/hEYPZ721GtoTKOKndgrEpm\n1oHAItNDOtlsis5M1WUwzKwTEamC+mYIUsTeE6V4Y8NBAMBj1513TrspPnbdeUiwmbD9WDHe3ngo\nxCMMnwqPv87eZBAaXUwbKe1T2BEmmrjlBabqK4MJbIrEzDoRkZIYrBNEUcTjH+2E1ydi+HlpGNyl\n5Tk9Tst4G+4fkgUAWL3rRCiHGFZqWFwqac8ymKgSqFlX369ilsEQEamD+mYIirjvfi3E//YXwmIy\n4K8jz2vWY3VI9WeGS6u00xVGDbuXStpJZTDMrEcFLZTBsM86EZGy1DdDUESJoih3cBlzUVtkJNmb\n9XhxVjMAoMylnWC9SgWdYCTtU/zn/8ipCk3V/dO5cam5DIaZdSIiVWCwHuW+O1CITQdPwWI04J5B\nHZv9eHFW/6ZCWsysK9kJRpIWb4PNbEC1T8Sx02zfqHfq7gbDmnUiIjVQ3wxBESVl1e+4KANOh63Z\njxdv8wfrZS5Psx8rUtTQY11iMAhol8RFptFCLoNRYc261cRuMEREaqC+GYIi5rtfC7EptyarPrj5\nWXUgkFmv8vg0swunmhaYApB7rR/iIlPdYxkMERGdDYP1KPbCmr0AgNsvykArR3AbIJ1NbE2wDgDl\nGqlblzLrMWbTWY6MDLl940lm1vUuEKyr71dxoM86g3UiIiWpb4agiPju10J8X5NVvzdEWXUAsJgM\ncuChlbr1CpVl1qWNkdi+Uf9cNYGwRdXBuja+ISMi0iv1zRAUES9+6c+q39YvdFl1SaBuXRvButwN\nRgULTAH2Wo8m6s6s+8fk4gJTIiJFqW+GoLD77tdCbDwQ+qy6RKpb10qwXuH2j1MtmfV2NWUwR05V\nwOsTFR4NhZOqa9a5wJSISBUYrEchKas+ul8bpLcIbVYdAOI0llmvdPuDEbUE660SbLCYDPB4RRxn\n+0ZdU3M3GNasExGpg/pmCAqrjQf8WXWzUcDkwZ3C8hxyZl0jNeuVnprMukrKYPztG1kKEw3U3Wfd\nP6Zqn8gNuoiIFKT4DHHs2DGMHTsWycnJsNvtuOCCC7Blyxb5dlEUkZ2djfT0dMTExGDw4MHYuXNn\nncdwuVyYOnUqUlJSEBsbi+uvvx5Hjx6N9EvRhBdr+qrf1i8jLFl1QHu7mEoLTNXQZ13SLpm91qOB\nqstgan14rapmsE5EpBRFg/WioiJccsklMJvN+Oyzz7Br1y4899xzaNGihXzM/PnzsWDBAixatAib\nN2+G0+nEsGHDUFpaKh8zffp0rFy5Ejk5OVi/fj3KysowcuRIeL38+ra27w8U4rsDhTAbBdwbpqw6\nUGuBqVYy6yrawVQiLTI9dJKZdT1zq3iBae0xsRSGiEg5ijaW/tvf/oaMjAy88cYb8nXt27eX/y6K\nIl544QXMmTMHo0aNAgC8+eabSEtLw/LlyzFp0iQUFxdjyZIlePvttzF06FAAwLJly5CRkYE1a9Zg\nxIgREX1Navbil/6s+ui+GWgdpqw6ECiDKdVIZl1NO5hKpEWmzKzrm5pr1gVBgNVkgKvax2CdiEhB\nis4QH374Ifr27Ytbb70VLVu2RO/evfHaa6/Jt+fm5iI/Px/Dhw+Xr7NarRg0aBA2bNgAANiyZQs8\nHk+dY9LT09G9e3f5mN9zuVwoKSmpc9G7TbmnsOFXf1Z98hXhy6oDtRaYaiyzrpaadQDIZK/1qKDm\nMhiAvdaJiNRA0WD9wIEDWLx4MbKysvDFF1/gnnvuwf3334+33noLAJCfnw8ASEtLq3O/tLQ0+bb8\n/HxYLBYkJiY2eszvzZs3Dw6HQ75kZGSE+qWpjtQB5tYwZ9WB2q0bPWF9nlBR26ZIANCupgzmcCHb\nN+qZmvusA4FFpsysExEpR9EZwufz4cILL8TcuXPRu3dvTJo0CX/605+wePHiOscJglDnZ1EU6133\ne2c6Zvbs2SguLpYvR44cad4LUblNuafwv/01WfUw9FX/Pc1uimRRtCqsjvQWMTAbBbi9PuSXVCk9\nHAoTaQdTtWfWuTESEZFyFA3WW7VqhfPOO6/Odd26dcPhw4cBAE6nEwDqZcgLCgrkbLvT6YTb7UZR\nUVGjx/ye1WpFQkJCnYueSVn1W/pkoE2iPezPJ9esa6QMpkKFZTBGg4CMJC4y1Ts5s67CmnUgkPFn\nGQwRkXIUnSEuueQS7Nmzp851e/fuRbt27QAAmZmZcDqdWL16tXy72+3GunXrMHDgQABAnz59YDab\n6xyTl5eHHTt2yMdEsx8O+rPqJoOAKVeEP6sOaG8HU2mBqZrKYIBA3Xou69Z1S/1lMNwYiYhIaYp+\n7//AAw9g4MCBmDt3LkaPHo1Nmzbh1VdfxauvvgrAX/4yffp0zJ07F1lZWcjKysLcuXNht9sxZswY\nAIDD4cDEiRMxY8YMJCcnIykpCTNnzkSPHj3k7jDR7KWv9gMAbu3bJiJZdUDDC0xVFqxLvdYPsSOM\nbsndYNRaBmPiAlMiIqUpGqz369cPK1euxOzZs/HEE08gMzMTL7zwAu688075mFmzZqGyshKTJ09G\nUVER+vfvj1WrViE+Pl4+5vnnn4fJZMLo0aNRWVmJIUOGYOnSpTAa1TkBRsq2I6fxzd7fYDSEb7fS\nhsRraFOkaq8P7prdGe0qKoMBgPYpNbuYsgxGl7w+ER6vf/GwWjPrVi4wJSJSnOIr6kaOHImRI0c2\nersgCMjOzkZ2dnajx9hsNixcuBALFy4Mwwi1a2FNX/WbereW658jQUuZ9cpaQQgz6xRJ7lq7glpU\nGqzLZTBcYEpEpBh1zhDUbDuOFePLXwpgEIApYe6r/ntyzbq7Gj6Vtx2UgnVBUF92s3avdbWfR2q6\n2h1W1Pbek7DPOhGR8tQ5Q1CzLfzKn1W/vlc6Mmt2w4wUqXWjKAIVKv/6vPaGSGdrBxpp6S1sMBkE\nuKp9OFHK9o16Iy0uNRoEmIzq/FVsM7EMhohIaeqcIahZfskvwRc7T0AQgPuujGxWHfBnCU0Gf+Cr\n9lKYSrnHurpKYADAZDTI5UsHT7IURm9cHnV3ggFq9VlnsE5EpBj1zhJ0zhbWdIC5pnsrdGoZf5aj\nQ08QhEDdusp3MZV6rNtUtrhUIu1keojtG3Un0AlGvb+G5R1Mq1kGQ0SkFPXOEnRO9heU4tPteQCU\nyapLtLIxUpVbvZl1AMioabd57HSlwiOhUAv0WFfnew9gn3UiIjVgsK4zf//6V4giMPy8NHRrpdzO\nrFrZGEmNu5fWFltzHstdDJb0Ru27lwIM1omI1EC9swQ1We7Jcnyw7RgAYOqVWYqOJV4j7RvVunup\nRMr4V3rUfR6p6bRQBmOVF5iyDIaISCnqnSWoyV7+ej98InBl15bo0cah6FjkMhiVZ9YrVZ5Zl4N1\nNzObesMyGCIiCgaDdZ04cqoCK7dKWXXlatUlUvmGVjLrdovi+4M1SMr4VzBY1x0tdYPhAlMiIuWo\nd5agJnl57a+o9om4LCsFvdsmKj2cQBmMyjPrau8GEyiDYbCuN3IZjKpr1tlnnYhIaeqdJShox09X\n4r9bjgAA7h+ibK26RCsLTNXcZx0AYsz+88jMuv5oogzGxD7rRERKY7CuA/9Y9ys8XhEXd0hCv/ZJ\nSg8HABBnNQNQf+vGSrd/fGpfYMpgXX8Cwbp6fw0HatZZBkNEpBT1zhIUlIKSKvxrc01WXeEOMLXF\naaQMRu4Go9IymBh5gam6zyM1nZStVnew7h+bVLJDRESRp95ZgoLyj28OwF3tQ992iRjQMVnp4cji\n5QWm2tjBVK2ZdelDBDPr+qOJMhhm1omIFMdgXcNOlrnwzveHAABTh2RBEASFRxSglcx6lcpr1rnA\nVL+kYN2igcx6FTPrRESKUe8sQWe18Mt9qPL40KuNA5dnpSg9nDrkPusqr1lXfzcY/3lkn3X90cam\nSOyzTkSkNPXOEnRGm3JP4a2N/qz6zBFdVJVVBwKZ9XKV11pLQbBaM+tSeU61T4Sbva51Re6zrurW\njYEyGFEUFR4NEVF0Uu8sQY2qdHvx5//+BFEERvdtg8uyUpUeUj3xGtsUSa0LTGt/iGB2XV+0UbMe\nmCJc/LBIRKQIBusaNP+LX3CosAKtHDb8ZeR5Sg+nQbVr1tWckatU+QJTs9EAs9H/rUmFR90ffKhp\n3Bpq3QgEvgkgIqLIUu8sQQ3alHsKSzccBAA8fXNPJNjMyg6oEVLNuscrqjojJ3eDUWlmHQgETOwI\noy9aqFk3Gw0wGvwfFrnIlIhIGeqdJaieCne1XP5yW98MDOqsvvIXSWzNwkhA3R1hAt1gTGc5Ujly\nRxgG67oil8Go+IMiANhqPkxwkSkRkTIYrGvIM1/skctf5ozspvRwzshgEOTsuprr1rWQWZc+SDCz\nri9a2MEUYK91IiKlqXuWIJlWyl9qk4N1lWbWRVEMLDBVac06EPggwV7r+hLYwVS97z2gdrDO9x8R\nkRIYrGuAlspfapMWmaq113rtTKGag/VAGYw6zyOdG61k1qXWkgzWiYiUoe5ZggAA8z/XTvlLbWrP\nrNfOVKu5DEb6IMEyGH0J1Kyr+9ewTdoYScULxYmI9EzdswTh+wOFmit/kcTL7Rs9Co+kYRU1mWqL\nKdDxQo3sDNZ1KdANRr0fFIFAr3Vm1omIlMFgXcUq3NWY9e7PALRV/iJR+wLTQCcYdQdLcs06g3Vd\nkXcwVXkZDGvWiYiUpe5ZIspptfxFIgXrpSotg9FCJxgAiGE3GF3STBlMzf8PbopERKQMdc8SUUzL\n5S8SeRdTlWbW1b57qUQug+EOproilcFYjOr+NSyXwXBTJCIiRah7lohiT3y8C4A2y18k8SpfYFrh\n0UZmnZsi6ZN2NkViGQwRkZIYrKuQq9qL3XklAIAHhnVWeDTnTu2Z9Sq3RmrWGazrjiiKcGumdSM3\nRSIiUpK6Z4kodeRUBXyiv+Y7LcGq9HDOWZzVX7qj9pp1m8ozm3azVAbDYF0vXLXaIKo9WGc3GCIi\nZal7lohSuScrAADtU+wQBPW2FDwbtWfWKzXSDcZes8CUmXX9cHtrB+vqfv/ZmFknIlIUg3UVyj1Z\nBgBonxyr8EiaR+0165Wa6QYj9VlX53mkppM6qwgCYDaq+wN5YFMkflgkIlICg3UVkjLrHVK0HazL\nmXW1BuvSAtOazLVasc+6/gQ2RDKo/tszlsEQESmLwboKyZl1rQfrUp91lZbBaKXPOncw1R+5E4zK\nS2AA9lknIlIag3UVOijXrOsjWC9zeRQeScM0s4Mpg3Xd0crupQAz60RESlP/TBFlKtzVyC+pAqD9\nMpj4mjKYKo8PHq/6snJSDbj6N0WSziODJb2Qy2BUvnspUGuBKWvWiYgUoehMkZ2dDUEQ6lycTqd8\nuyiKyM7ORnp6OmJiYjB48GDs3LmzzmO4XC5MnToVKSkpiI2NxfXXX4+jR49G+qWEjJRVb2E3o4Xd\novBomifWGqgFL1dh3XplTXaTZTAUaVoqg7Ga2A2GiEhJiqd1zj//fOTl5cmX7du3y7fNnz8fCxYs\nwKJFi7B582Y4nU4MGzYMpaWl8jHTp0/HypUrkZOTg/Xr16OsrAwjR46E16vNwOZgYTkAIFPjWXUA\nMBsN8lfoaqxbr9RIZl3eFMnjhc8nKjwaCgWXRjZEAgLZf36zQ0SkDMVnCpPJBKfTKV9SU1MB+LPq\nL7zwAubMmYNRo0ahe/fuePPNN1FRUYHly5cDAIqLi7FkyRI899xzGDp0KHr37o1ly5Zh+/btWLNm\njZIv65zlnqwJ1jXetlEibYykxo4w2umzHhgfSxH0weUJdINRO7l1I4N1IiJFKD5T7Nu3D+np6cjM\nzMTtt9+OAwcOAAByc3ORn5+P4cOHy8darVYMGjQIGzZsAABs2bIFHo+nzjHp6eno3r27fExDXC4X\nSkpK6lzUQgrWtb64VBKv4vaNWtnB1FarVIKlMPqgpTKYwAJTlsEQESlB0WC9f//+eOutt/DFF1/g\ntddeQ35+PgYOHIjCwkLk5+cDANLS0urcJy0tTb4tPz8fFosFiYmJjR7TkHnz5sHhcMiXjIyMEL+y\nc3fwpH7KYIBaHWFUWQajjcy6wSDIARN7reuDHKxraIGpi9/qEBEpQtGZ4uqrr8bNN9+MHj16YOjQ\nofjkk08AAG+++aZ8zO83DBFF8aybiJztmNmzZ6O4uFi+HDlypBmvIrRydRasx1r9E32pCjPr8qZI\nKs+sA4GOMMys64MU+FqM2gnWmVknIlKGqmaK2NhY9OjRA/v27ZO7wvw+Q15QUCBn251OJ9xuN4qK\niho9piFWqxUJCQl1LmpQUuVBYbkbgH7KYOSadRVn1tW+wBQIfKCQ2k2Stsl91jXwQZF91omIlKWq\nYN3lcmH37t1o1aoVMjMz4XQ6sXr1avl2t9uNdevWYeDAgQCAPn36wGw21zkmLy8PO3bskI/REqkE\nJjXeKpePaF2gZl19GyNVamQHUyBQqlPJgEkXtNQNRlozUe0TUa3C/RKIiPRO0Yhw5syZuO6669C2\nbVsUFBTgySefRElJCcaNGwdBEDB9+nTMnTsXWVlZyMrKwty5c2G32zFmzBgAgMPhwMSJEzFjxgwk\nJycjKSkJM2fOlMtqtEZvnWCA2ruYqi/IDHSDUf8HIzlYZxmMLri1FKyba3cj8iFOA6U7RER6omiU\ncvToUdxxxx04efIkUlNTcfHFF2Pjxo1o164dAGDWrFmorKzE5MmTUVRUhP79+2PVqlWIj4+XH+P5\n55+HyWTC6NGjUVlZiSFDhmDp0qUwGtWfLf09vdWrA0CcTZ0LTN3VPlTX9CzXQmY9hhsj6Yq8g6kG\nusHU/kBR5fHq5ls/IiKtUPS3bk5OzhlvFwQB2dnZyM7ObvQYm82GhQsXYuHChSEeXeQd1FnbRqB2\nZl1dZTC1y0m0ULMuZf+ZWdcHLXWDMRgEWEwGuKt9rFsnIlKA+meKKBLIrNsVHknoqLXPuhT0Gg0C\nzMYzdxdSAy4w1ZdAZl0bv4JtJvZaJyJSijZmiiggimKtYD1O4dGEjpRZL1VZGYxcr242nrUVqBrI\nZTDMbOqC3A1GA2UwQO32jXz/ERFFGoN1lSiq8KCkJqBtl6yfzHqgDEZdwbqUobZpoAQG4AJTvdFS\nNxiAGyMRESlJGzNFFMg9WQYASHfY6nRf0Dq1LjCt8mhj91JJDIN1XZHLYDRQsw7U7rXOMhgiokjT\nxkwRBXJPVgAAMlP1s7gUAOKlTZFUl1nXTo91ALCba3YwZRmCLgQy69p4/7EMhohIOQzWVULKrLfX\nUY91QL2ZdS3tXgqwDEZvAjXr2vgVLG2MxMw6EVHkaWOmiAIHpcy6jto2ArVq1t3V8NX0NVeDSo2W\nwbAbjD5orRuMVS6D4YdFIqJI08ZMEQX0uCESEGjdKIrqKuGo1FgZTKB1o3rOIZ27QJ91bbz/5DIY\nLjAlIoo4BusqIIoiDhbqb0MkwJ85NBn8rRHVVAoj16xbtLEbI8tg9EWr3WBYBkNEFHnamCl0rqDU\nhQq3F0aDgIxE/bRtBPy70Mp16yraxVQqg4nRSDeOQBkMg3U9kMpgLFoJ1k0sgyEiUoo2Zgqdk0pg\n2iTGaGbybgo1bowkZajtmsms+8dZyWBJFzS3wFTus87MOhFRpAUdqUyYMCGo415//fVzHky0koJ1\nvXWCkahxYyQp6NVKT3uWweiL9lo3+j9UuPhhkYgo4oIO1pcuXYp27dqhd+/eEEX1dPXQg4M6XVwq\niVdh+8YKN7vBkHLcmq1ZZ7BORBRpQQfr99xzD3JycnDgwAFMmDABY8eORVJSUjjHFjX02glGIpfB\nqCizXuXRVjcYObPOYEnzRFHU4A6mXGBKRKSUoGeKl19+GXl5eXjooYfw0UcfISMjA6NHj8YXX3zB\nTHsz6T5Yt9XsYqqqzLp/LJrZFKlmB1OPV4THy4BJy6p9IqQtB7RSBiN9A8DWjUREkdektI7VasUd\nd9yB1atXY9euXTj//PMxefJktGvXDmVlZeEao655fSIOndLnhkgSddas+wNerWTWbZbAf1V2hNG2\n2os0WQZDRERnc84zhSAIEAQBoijC52Om71wdP10Jd7UPFqMB6S1ilB5OWMg162oK1msy61qpWbcY\nDTDW9KvnIlNtq71IU3vBOn/XExFFWpNmCpfLhX/9618YNmwYunTpgu3bt2PRokU4fPgw4uLiwjVG\nXZM2Q2qbbJeDMb1RZetGqRuMRoJ1QRBgN3ORqR5ImXWLyQBB0Mb/eakbDDPrRESRF/QC08mTJyMn\nJwdt27bF//3f/yEnJwfJycnhHFtUOKjzto2AOstg5G4wGimDAfz19aWuapbBaJzWdi8FAFtNbX0V\n+6wTEUVc0MH6K6+8grZt2yIzMxPr1q3DunXrGjzuvffeC9ngosEBeXGpvnYurU3ewbRKPTuYVtUE\nvFpZYAoESnaY3dQ2uROMRhaXArU2ReJ7j4go4oIO1v/4xz9q5itbLQn0WNdvGVG8GjPrHm31WQeA\nmJpdTJlZ1zat7V4KsAyGiEhJTdoUiULvYKG/E0z7KMisq6pm3a2tHUyBwAcLBuvaJpfBaKTHOsAF\npkREStLObKFDHq8Ph3XethFQX8261yfKAZPdEvTnVcVJbSYrPeo4j3RutFkGwz7rRERKCTpSueKK\nKxosg3E4HOjSpQumTJmCjIyMkA5O744WVcLrExFjNiIt3qb0cMJGba0ba3+Vr5U+60Cgvp6ZdW2T\nymAsGiqDkT5YsAyGiCjygg7WL7jgggavP336ND799FMsWrQI69evb/Q4qi/3pH8jqXbJdhh02rYR\nAOKsgR1MRVFUfO1D7WDXpqFSBKkMhn3WtU2T3WBqlcGo4f8wEVE0CTpYf/755894+5QpU/DII4/g\n008/bfagokXuSX8JTIdU/ZbAAIGa9eqa8hOl68SlYDfGbNRU0MGadX0IlMFoKVgPjFUN/4eJiKJJ\nyGaLSZMmYevWraF6uKgQDT3WAX8vcykmVkMpTKUGO8EAQIzZ/6GnkqUImuaWM+vaef/VDs5dXGRK\nRBRRIQvWY2JiUFVVFaqHiwq5UrCu48WlAGAwCIizSL3WlQ/WpR1AtZYdZBmMPmixG4zZaJB3WOYi\nUyKiyArZbLFq1Sp07tw5VA8XFaRgvYPOg3Wg1sZIzKyfs8ACU+XPIZ07LZbBAIDNxF7rRERKCLpm\n/cMPP2zw+uLiYmzevBlLlixhL/YmqPJ4cby4EoD+M+sAEGtVT6/1Sg3uXgoEOtewZl3bApsiaev9\nZzMbUe72stc6EVGEBR2s33jjjQ1eHx8fj65du2Lp0qW49dZbQzUu3Tt8qgKi6N/dMznWovRwwk5N\nvdalzLqW2jYCLIPRCy12gwFqd4Th+4+IKJKCDtZ9PmZTQkkqgclMjdVUR5JzFei17lF4JIHMtOYy\n6+wGowtyGYyGataBwHgZrBMRRZa2ZgsdiZZOMBI5s66CMpgqjdasS7utVjBY0jSXBrvBAIBN2hip\nmokbIqJICjpY//777/HZZ5/Vue6tt95CZmYmWrZsibvvvhsulyvkA9QrObMeBfXqQCBYL1VBGYyU\nmdZuNxjlzyGdu0DNurZyJTZm1omIFBH0bJGdnY2ff/5Z/nn79u2YOHEihg4diocffhgfffQR5s2b\nF5ZB6lHUBes29WTWpZpvrWXWpTIY9lnXNq12g5G+CWCwTkQUWUHPFtu2bcOQIUPkn3NyctC/f3+8\n9tprePDBB/HSSy/h3//+d1gGqUfR0mNdEs8Fps3GBab6EOizrq33n5RZ56ZIRESRFXSwXlRUhLS0\nNPnndevW4aqrrpJ/7tevH44cORLa0elUuasaBaX+kqHMaKlZV2FmPcYS9PpqVbDX7GDKBabapvlu\nMNwUiYgoooKeLdLS0pCbmwsAcLvd+PHHHzFgwAD59tLSUpjN5tCPUIcOFvqz6kmxFjjs0XHO4qz+\n16mmmnWtZdZtFv9/10qPF6IoKjwaOldaLYNh60YiImUEPVtcddVVePjhh/Htt99i9uzZsNvtuOyy\ny+Tbf/75Z3Ts2PGcBzJv3jwIgoDp06fL14miiOzsbKSnpyMmJgaDBw/Gzp0769zP5XJh6tSpSElJ\nQWxsLK6//nocPXr0nMcRCXIJTLJd4ZFEjpoy61rvBiOK4MY0Gqb9BaZ87xERRVLQs8WTTz4Jo9GI\nQYMG4bXXXsNrr70GiyWwmc/rr7+O4cOHn9MgNm/ejFdffRU9e/asc/38+fOxYMECLFq0CJs3b4bT\n6cSwYcNQWloqHzN9+nSsXLkSOTk5WL9+PcrKyjBy5Eh4verN/hyUF5fGKTySyFFTzXpFTTcVrWXW\na4+3gh1hNEurrRu5wJSISBlBF+2mpqbi22+/RXFxMeLi4mA01p1o/vOf/yAurunBZ1lZGe688068\n9tprePLJJ+XrRVHECy+8gDlz5mDUqFEAgDfffBNpaWlYvnw5Jk2ahOLiYixZsgRvv/02hg4dCgBY\ntmwZMjIysGbNGowYMaLJ44mE3JMVAIDMlCjMrKsgWJcXmGoss240CLCaDHBV+1Dh9iJZ6QHROXFr\nvWadmXUioohq8mzhcDjqBeoAkJSUVCfTHqwpU6bg2muvlYNtSW5uLvLz8+tk661WKwYNGoQNGzYA\nALZs2QKPx1PnmPT0dHTv3l0+piEulwslJSV1LpGUe7IMQHRl1uU+6yoog6nUaM06UKsjDLObmqXV\nHUzlMhguMCUiiihFZ4ucnBz8+OOPDfZnz8/PB4A6HWikn6Xb8vPzYbFYkJiY2OgxDZk3bx4cDod8\nycjIaO5LaZKDhf7MevtoyqzLZTAehUcSCHS1VrMOBOrW2b5Ru7RaBsMFpkREylAsWD9y5AimTZuG\nZcuWwWazNXqcIAh1fhZFsd51v3e2Y2bPno3i4mL5EsmWk8UVHpwqdwMA2kdJ20YAiK8pg6ny+ODx\nKvs1uryDqQaDdal0h+0btUuzrRtN7LNORKQExWaLLVu2oKCgAH369IHJZILJZMK6devw0ksvwWQy\nyRn132fICwoK5NucTifcbjeKiooaPaYhVqsVCQkJdS6RcviUP6ueGm9FrFVbfb6bo/ZrLVe4bl2r\n3WCAQOlOpUf5ciI6Ny6P1LpRW+8/ZtaJiJShWLA+ZMgQbN++Hdu2bZMvffv2xZ133olt27ahQ4cO\ncDqdWL16tXwft9uNdevWYeDAgQCAPn36wGw21zkmLy8PO3bskI9Rm+JKfxlIcmzT6/u1zGw0yDWv\nSteta7XPOsDMuh4EdjDVWGadmyIRESki6NTuW2+91eD1DocDXbp0QdeuXZv0xPHx8ejevXud62Jj\nY5GcnCxfP336dMydOxdZWVnIysrC3LlzYbfbMWbMGPm5J06ciBkzZiA5ORlJSUmYOXMmevToUW/B\nqlpINdvRlFWXxFnNqPK4FO0II4qiZrvBAIFvAxisa1O114dqn39DK82VwbDPOhGRIoKOGKdNm9bg\n9WVlZfD5fLjmmmuwfPlyxMfHh2xws2bNQmVlJSZPnoyioiL0798fq1atqvMczz//PEwmE0aPHo3K\nykoMGTIES5cubbBjjRpIWeW4KAzW420mnCxTNlh3Vfsgbf6pxcy63A2GwbomuWut19BaGYyVZTBE\nRIoIOmL8fV24xOfzYcuWLbjrrrvw+OOP49lnnz3nwaxdu7bOz4IgIDs7G9nZ2Y3ex2azYeHChVi4\ncOE5P28kSYGq1Hc8msgdYRQsg6kd5GoxWI8x+88hM+vaVHtxpkVrmXVuikREpIhmzxYGgwH9+vXD\nc889h48++igUY9I1aXFlfBRm1uVe6wpm1itqAg2L0QCTUVvBEsA+61on1aubjQKMhjN3tVIblsEQ\nESkjZNFKp06dcPTo0VA9nG5JgWo0lsHIu5iqILOuxXp1oHYZDLvBaJG8IZLGSmCAwAJTFxeYEhFF\nVMiC9V9//RVt2rQJ1cPplhSoRmMZTLwKNkbS8u6lALvBaJ1We6wDtVs3MrNORBRJzY4YRVHE1q1b\nMWPGDFx33XWhGJOulTGzrmxmXcM91oFafdYZrGuSVLOuzWBdKoPhe4+IKJKCjhgTExMb3BW0rKwM\nXq8XV1111RkXgpKfFKjGR2FmXRU16zXlIzaNZtbZulHbpBISrS0uBQILTKt9Iqq9Pk2u+SAi0qKg\nI8YXXnihwesTEhLQtWtXdOvWLVRj0rVAzbpZ4ZFEnhoy61revRQAYiw13WCY3dQkt1wGo733X+0P\nuFXVPsQxWCciioigg/Vx48aFcxxRQwpUY63am6ybS6pZL1dwcWQFF5iSgrS6eylQt3SnyuONylI+\nIiIlBD1jiKKIZ555BpdccgkuuugiPPLII6iqqgrn2HRJqlmPyjKYmtdcqoKadS4wJSUEusFoL1g3\nGAS5fEf60EFEROEX9Izx9NNP4+GHH0ZsbCxatWqFBQsW4P777w/n2HSpLJrLYGpes5I7mGq+daOZ\nfda1zKXhMhgAsJm4yJSIKNKCDtaXLl2KhQsXYtWqVfjggw/w/vvv46233oIo7d1OQeEOpuros67V\nmnV7Tc06u8Fok5a7wQC12zfy/UdEFClBzxiHDh3CyJEj5Z9HjBgBURRx/PjxsAxMj1zVXnmBWTTW\ne0qlP0pm1qWFmVrtBhNj8f+XZRmMNsllMBqsWQfYa52ISAlBzxhutxsxMTHyz4IgwGKxwOVyhWVg\nelTuCgRY0RisM7PefDHMrGua5stgaj5kuJhZJyKKmCZFjH/9619ht9vln91uN5566ik4HA75ugUL\nFoRudDojBal2ixFGQ/2e9Xont250V8PnE2FQ4BxofQdTqWbd7fWx17UGaXkHU6BWZr2awToRUaQE\nHaxffvnl2LNnT53rBg4ciAMHDsg/N7RpEgWUujwAojOrDgRetyj6y1GUOA9yNxiLNv8Nai+MrfB4\nkcBgXVOkjLRmg3UTy2CIiCIt6Ihl7dq1YRxGdJAy69G4uBTwBygmg4Bqn4iyqmpFgvUKjWfWrSYD\nDALgE/3fEiTYoq+rkJYF+qxr9P1nZjcYIqJIa1J6p6SkBD5f/YyKz+dDSUlJyAalV3KP9SjNrAuC\nECiFqfmWIdK0voOpIAjsCKNhuimDYWadiChigp4xVq5cib59+za4EVJVVRX69euHjz76KKSD0xsp\nWI+N0mAdCJTCKLUxUkXNzp9a7QYDcGMkLdPypkgAWzcSESkh6Blj8eLFmDVrVp0FphK73Y6HHnoI\nixYtCung9EYKUKO1Zh2o1RFGofaNlTUZQa1m1oFACU+lR7muOnRuAn3Wtfn+kzdF4gJTIqKICTpY\n37FjBwYPHtzo7Zdffjm2b98eijHpVjRviCSRe60rlFmvdAc68miVnZl1zZLKYCyaz6yzDIaIKFKC\nnjGKiopQXd14gOXxeFBUVBSSQelVeZTXrAO1ymAUy6xre1MkgGUwWqb9mnX2WSciirSgZ4z27dvj\nhx9+aPT2H374Ae3atQvJoPSqNMq7wQBAXE33EqUy6xUa3xQJCIydC0y1Rz87mPK9R0QUKUHPGKNG\njcKcOXNw4sSJerfl5+fjL3/5C26++eaQDk5v5DIYa/S221O6Zr1K7rOu3WA9xuw/h8ysa4/2dzBl\nGQwRUaQFneJ9+OGH8cEHHyArKwtjx45Fly5dIAgCdu/ejXfeeQcZGRl4+OGHwzlWzYv2PutArZp1\nBYJ1j9cHj1cEANjN2v03CNSsc4Gp1mi9DMbKBaZERBEXdMQSHx+P//3vf5g9ezZWrFgh16cnJiZi\n7NixmDt3LuLj48M2UD2I9j7rgLKtGytrfXVvs2gzWAICwTpLEbQnsIOp1jPrfO8REUVKk6JGh8OB\nl19+GX//+99x8uRJiKKI1NRUCIIQrvHpSqmLrRuVLIORaryNBgEWo3aDdS4w1S63vIOpNt9/LIMh\nIoq8c4oaBUFAamoqTp48iU8//RRerxf9+vVDq1atQj0+XSmr8u/aGdWbIsmtGyO/g6kUrMeYjZr+\ngCn1WWewrj1aL4ORusEws05EFDnnHDW+++67mDhxIjp37gyPx4M9e/bg73//O/7v//4vlOPTFbkM\nJppr1hXMrEvBrZYXlwLsBqNlgR1MtfketNWMu6qamXUiokgJOr1TVlZW5+fHH38cmzZtwqZNm7B1\n61b85z//wZw5c0I+QD0p4w6mcmZdyZr1GA33WAeAGEtNNxhmNzUnsIOpVjPr/v877LNORBQ5Qc8Y\nffr0wQcffCD/bDKZUFBQIP984sQJWCyW0I5OR3w+EeU1mdBo7gajhpp1LfdYB2pn1tkNRmtcGq9Z\nt7IMhogo4oKOGr/44gtMnjwZS5cuxd///ne8+OKLuO222+D1elFdXQ2DwYClS5eGcajaVl4rsIrm\nzLqSrRv1sHspULt1IwMmLfH5RLi9Gu+zbuICUyKiSAs6amzfvj0+/fRTLF++HIMGDcK0adOwf/9+\n7N+/H16vF127doXNZgvnWDVNCk7NRkGzX4GHgrQhVFlVNURRjOhCT6kvudYz61xgqk1SoA5ouQyG\nfdaJiCKtyTPGmDFj5Dr1wYMHw+fz4YILLmCgfha169W13ImkuaQSoGqfKJcEREqVTmrW7TU16yxF\n0BaXRw/BOvusExFFWpPqMT777DPs2rULvXr1wpIlS7B27VqMGTMG11xzDZ544gnExMSEa5yaJ/dY\nj+J6dQCwm40QBEAU/YtMI1mSopduMDE1Gzoxs64tUicYo0GASaN9/gM1676IfzNGRBStgp4xZs2a\nhfHjx2Pz5s2YNGkS/t//+38YPHgwtm7dCqvVigsuuACfffZZOMeqaYHMulnhkSjLYBAQZ1Gmbl03\n3WDMNd1gGKxrivRNkpY35Kr94TrS34wREUWroGeN119/HZ9++ilycnKwefNmvP322wAAi8WCJ598\nEu+99x6eeuqpsA1U68rk3Uu1HSiGQmBjpAgH6+wGQwrSeicYILDAFKhb1kNEROET9Kxht9uRm5sL\nADhy5Ei9GvXzzz8f69evD+3odIQ91gOkc1DqiuwuplKwbtNJsF7h8UIURYVHQ8EKbIik3WDdbBRg\nqKl84SJTIqLICHrWmDdvHv74xz8iPT0dgwYNwv/7f/8vnOPSnUDNenSXwQCBzHq5K7KTvbSJkN2s\n7Q9MUs29KLIUQUvkzLpG2zYCgCAIXGRKRBRhQUctd955J6666iocOHAAWVlZaNGiRRiHpT/lLmbW\nJYGNkSKbWa+SF5hqN7MJBLrBAP66da33jY8WWt+9VGIzG1Hh9rLXOhFRhDRp1khOTka/fv1CFqgv\nXrwYPXv2REJCAhISEjBgwIA6i1RFUUR2djbS09MRExODwYMHY+fOnXUew+VyYerUqUhJSUFsbCyu\nv/56HD16NCTjCyWpZj0+yrvBALU2RopwzXqgG4y2/w2MBgGWmoCvktlNzZDLYDRcsw4ANhN3MSUi\niiRFZ402bdrg6aefxg8//IAffvgBV155JW644QY5IJ8/fz4WLFiARYsWYfPmzXA6nRg2bBhKS0vl\nx5g+fTpWrlyJnJwcrF+/HmVlZRg5ciS8XnVNJKWsWZcFatbZDeZccZGp9uihDAZgr3UiokhTNFi/\n7rrrcM0116Bz587o3LkznnrqKcTFxWHjxo0QRREvvPAC5syZg1GjRqF79+548803UVFRgeXLlwMA\niouLsWTJEjz33HMYOnQoevfujWXLlmH79u1Ys2aNki+tnjKWwchq72IaSXrpBgNwF1MtCgTr2s6s\nW6VgnesliIgiQjWzhtfrRU5ODsrLyzFgwADk5uYiPz8fw4cPl4+xWq0YNGgQNmzYAADYsmULPB5P\nnWPS09PRvXt3+ZiGuFwulJSU1LmEW1mVvz472jdFAmq1bmRm/ZxJi0wZrGuHy6P9bjAAYDOzDIaI\nKJIUnzW2b9+OuLg4WK1W3HPPPVi5ciXOO+885OfnAwDS0tLqHJ+Wlibflp+fD4vFgsTExEaPaci8\nefPgcDjkS0ZGRohfVX1yzToz6/I5iHzNuv/5tL6DKVC7DIYBk1bopgzGxDIYIqJIUjxY79KlC7Zt\n24aNGzfi3nvvxbhx47Br1y759t9vZx3MFtdnO2b27NkoLi6WL0eOHGneiwiCVLMey2BdzqxHumZd\n6l6hh8y6nbuYao4eNkUCApl1bopERBQZis8aFosFnTp1Qt++fTFv3jz06tULL774IpxOJwDUy5AX\nFBTI2Xan0wm3242ioqJGj2mI1WqVO9BIl3CTa9ZZBhNo3ahQZl0XNetyGQwXmGqFHjZFAmotMOWm\nSEREEaG6WUMURbhcLmRmZsLpdGL16tXybW63G+vWrcPAgQMBAH369IHZbK5zTF5eHnbs2CEfoxYs\ngwlQumZdD33J5TIYliJoRqDPurbff+wGQ0QUWYpGjo888giuvvpqZGRkoLS0FDk5OVi7di0+//xz\nCIKA6dOnY+7cucjKykJWVhbmzp0Lu92OMWPGAAAcDgcmTpyIGTNmIDk5GUlJSZg5cyZ69OiBoUOH\nKvnS6hBFMbApEjPrgZr1CAbrPp8ol8HoKbPOmnXt0Es3mMACU5bBEBFFgqKR44kTJ/CHP/wBeXl5\ncDgc6NmzJz7//HMMGzYMADBr1ixUVlZi8uTJKCoqQv/+/bFq1SrEx8fLj/H888/DZDJh9OjRqKys\nxJAhQ7B06VIYjeoJyFzVPni8IgC2bgRq1axHsAym9lf2elhgytaN2qOXTZGsXGBKRBRRikaOS5Ys\nOePtgiAgOzsb2dnZjR5js9mwcOFCLFy4MMSjC53aGeRYje+eGQpyzbrLE7HnrB3U2jRehgCwDEaL\n3DWZdYuKEgnnIlAGw8w6EVEkaDvFoxFltXYvNRjO3MkmGsTXbIpU5fHJAUy4SeUiNrNBF/8GMRap\nGwwXmGqF3rrBcIEpEVFkaHvW0AjuXlpXvM0Ei9H/1isorYrIc0oZaLtOvtmwc1MkzdFPzTrLYIiI\nIknbs4ZGSLXZXFzqZzAIaNXCBgA4fjpCwbpbP7uXAtwUSYsCO5hq+z1oM7HPOhFRJDFYjwAps84N\nkQJat4gBABw7XRGR55My0HpYXApwgakWMbNORETnQtuzhkZICynZYz1ADtaLKiPyfFJgoZ/Muv+9\nxAWm2qGXbjDcFImIKLK0PWtoRO0FpuSXLmfWI1MGo7fMOstgtCeQWdf2e1BaYMoyGCKiyGCwHgFl\nLn9AxZr1gNaJUrAemcx6pc4y6za5DIbdYLQisIOptn/tWplZJyKKKG3PGhohlcEwsx7QRi6DiUzN\nemVNUKuH3UsBZta1SC6D0XiwbjOxzzoRUSRpe9bQCKkMJp6ZdZlUBnP8dBVEUQz78+ktsy63bmTN\numYE+qxr+z0o91nne4+IKCIYrEdAKfus1yO1bqz0eFFUEf6dTPVWsx7DPuuao79uMMysExFFgrZn\nDY0oY5/1eqwmI1rGWwFEpiOM/jLr/veSu9oHry/830xQ8wX6rGv7164UrLuYWSciightzxoawR1M\nGxboCBOBYN0t7WCql2A98Dq4yFQbdFcGwwWmREQRwWA9AhisNyySHWGkchGbToJ1q8kAQfD/nb3W\n1U8URf2UwdQsMPV4RX6rQ0QUAdqeNTSCfdYb1iaCGyNJAa1d41lNiSAIckkPO8Kon8cbCGotWg/W\na/0f4iJTIqLw0/asoRHyAlPWrNcR6AgTuTIYvSwwBWp1hGGwrnquWiUjWs+s1x4/g3UiovDT9qyh\nEXLrRqtZ4ZGoS2sFatZjLPr5wMSOMNohlcAAgMWo7V+7BoMgfztQVc2OMERE4abtWUMDvD5RLsFg\nZr2uiNas66wbDADYzf73E8tg1K92vbogLTbQMJuJvdaJiCKFwXqYSYtLASDWqp9AMRSkMphT5e6w\nB5xVOusGA9TOrLMbjNrppW2jJNBrncE6EVG46WPmUDEpWLeYDLCa9BMohoIjxoz4mkW34c6uV3j8\n/w42PWXWa4J1doNRP720bZRwYyQioshhsB5mgXp1lsA0JFK91ivd/qBCT5l1LjDVDr20bZRIvda5\nMRIRUfjpY+ZQsTKXBwDr1Rsj1a2HuyNMZU2piJ5q1m1s3agZui2D4cZIRERhp4+ZQ8VK2WP9jFpH\noNe6KAYW+eoxs84yGPULZNb18f6TNkZiGQwRUfgxWA8zqWY9lsF6gyJRBuOq9kHaaFEvO5gCgL2m\nDSUXmKpfoGZdH79ypdfBBaZEROGnj5lDxVizfmaRaN9YO6DQUxkM+6xrh7Qpkl7KYKzMrBMRRYw+\nZg4VK+PupWcUiTKY8ppg1mwUYNb4hjS12Vmzrhkuj87KYJhZJyKKGP1ELiolB+vMrDdICtbzS6pQ\n7Q1Plu5QYTmAQMmNXjCzrh366wbDBaZERJGij5lDxaQyGGbWG9Yy3gqzUYDXJ6Kg1BWW59ibXwoA\nyGoZH5bHV0qgZp0Bk9rJZTA6KcMKZNZZBkNEFG4M1sNMyqyzZr1hBoOAVo7w1q3vOVEGAOjijAvL\n4ytF6gbDUgT1c+sts15TzsM+60RE4aePmUPFSlkGc1bpLWwAwle3vveEP7PeOU1fmXWpFIHdYNRP\nKoOx6CVYN/ODIhFRpOhj5lCxQBmMWeGRqFfrFnYA4cmsi6IoB+tdnPoK1rmDqXborRsMy2CIiCJH\nHzOHinGB6dmFs31jfkkVSquqYTQIyEyJDfnjK4mbImmH/rrBcIEpEVGkMFgPszLuYHpWrcNYBrOn\nZnFpZkqsbgIlCbvBaIfeusFYWQZDRBQx+pg5VIx91s9OKoM5HobM+j5pcanO6tWBQDcY9llXv0A3\nGH38yrWZWAZDRBQp+pg5VKy0ygOAmfUzqV0GI4piSB97j04XlwK1a9arQ37eKLQCmXV9fLvDBaZE\nRJHDYD2MRFGUd8+MZ2a9Ua0c/jKYCrcXpys8IX3sQCcYfbVtBAJlMD4xEAySOgVq1vXxKzdQs873\nHRFRuOlj5lCpKo8PXp8/48nMeuNsZiNS4qwAQrvI1OcT5TKYzjrrBAMAMbU22GGGU9302g2GfdaJ\niMJPHzOHSpW6/FliQQiULFDDwtER5mhRJSo9XlhMBrRLsofscdXCbDTAbBQAcJGp2sllMLrZwZRl\nMEREkcJgPYxqd4IRBEHh0ahbODrCSPXqnVLjYDLq860eY2ZHGC3QWzcYaQdTLjAlIgo/fcwcKiV1\ngolnCcxZtW7hz6yHsiOMnuvVJewIow16LYNhn3UiovBTdOaYN28e+vXrh/j4eLRs2RI33ngj9uzZ\nU+cYURSRnZ2N9PR0xMTEYPDgwdi5c2edY1wuF6ZOnYqUlBTExsbi+uuvx9GjRyP5UhoU2L2UwfrZ\nSMF6KMtgpB7reqxXl9TuCEPqpdtNkVgGQ0QUdooG6+vWrcOUKVOwceNGrF69GtXV1Rg+fDjKy8vl\nY+bPn48FCxZg0aJF2Lx5M5xOJ4YNG4bS0lL5mOnTp2PlypXIycnB+vXrUVZWhpEjR8LrVXYiKa3J\nrMcys35W6WHMrOuxx7pE3hiJQZOqBWrW9ZFZl15HlcfHtqFERGGmaBT5+eef1/n5jTfeQMuWLbFl\nyxZcfvnlEEURL7zwAubMmYNRo0YBAN58802kpaVh+fLlmDRpEoqLi7FkyRK8/fbbGDp0KABg2bJl\nyMjIwJo1azBixIh6z+tyueByueSfS0pKwvL6uHtp8EK9wNTj9eHAb/4PfXrssS6RMussg1E3t95q\n1mstlHVV++r8TEREoaWqmaO4uBgAkJSUBADIzc1Ffn4+hg8fLh9jtVoxaNAgbNiwAQCwZcsWeDye\nOsekp6eje/fu8jG/N2/ePDgcDvmSkZERltcj16yzDOas2tTsYnqyzB2Sr9YPFZbD7fXBbjHKJTZ6\nFFNTs84FpuoWqFnXR1Brq/U6XFxkSkQUVqoJ1kVRxIMPPohLL70U3bt3BwDk5+cDANLS0uocm5aW\nJt+Wn58Pi8WCxMTERo/5vdmzZ6O4uFi+HDlyJNQvB0AgWGdm/ewSYkyIrckSh6IUZm9Nf/WstHgY\nDPrtxBNTU45QyTIYVdNbNxizUYD034qLTImIwks1UeR9992Hn3/+GevXr6932+/bHoqieNZWiGc6\nxmq1wmq1nvtggxQI1s1hfy6tEwQBrRNjsPdEGY6drkSH1OZ1cJEWl3bRcScYoHY3GC4wVTO9BeuC\nIMBmNqLC7eUiUyKiMFPFzDF16lR8+OGH+Prrr9GmTRv5eqfTCQD1MuQFBQVytt3pdMLtdqOoqKjR\nY5TCbjBNI3eECUGv9UDbRv3WqwO1FpiyDEa1qr2BnYz1UgYD1O4IwzIYIqJwUjRYF0UR9913H957\n7z189dVXyMzMrHN7ZmYmnE4nVq9eLV/ndruxbt06DBw4EADQp08fmM3mOsfk5eVhx44d8jFKYZ/1\npgllR5g9URKs281cYKp2UlYd0E83GACwmaSOMHzvERGFk6JR5JQpU7B8+XJ88MEHiI+PlzPoDocD\nMTExEAQB06dPx9y5c5GVlYWsrCzMnTsXdrsdY8aMkY+dOHEiZsyYgeTkZCQlJWHmzJno0aOH3B1G\nKaXMrDeJ1BHmaDOD9SqPF4cKKwAAXXTcYx2o3WedAZNa1Q7WLTraSZe91omIIkPRKHLx4sUAgMGD\nB9e5/o033sD48eMBALNmzUJlZSUmT56MoqIi9O/fH6tWrUJ8fCAIe/7552EymTB69GhUVlZiyJAh\nWLp0KYxGZb9yLnN5AHCBabBCVQZz4LdyeH0iHDFmtIwP/9oEJbEbjPpJnWAsRoOuFjtbpWC9mmUw\nREThpGgUGcxmGoIgIDs7G9nZ2Y0eY7PZsHDhQixcuDCEo2s+doNpGilYP17cvGC99mZIZ1uIrHVy\nn3UPF5iqVWD3Uv1k1QHAZmYZDBFRJDCKDCMuMG0aqQwm73QVvD4RxnPMQkr16lk67wQDADGsWVc9\nve1eKpF6rTNYJwoNr9cLj8ej9DAoQiwWCwyG4OYFRpFhxMx607SMt8FkEFDtE1FQWoVWjnPbzGif\nlFnXeb06wG4wWqC3DZEkUmadmyIRNY8oisjPz8fp06eVHgpFkMFgQGZmJiwWy1mPZRQZRgzWm8Zo\nEOB02HC0qBLHT1eec7AeLZ1ggNplMAzW1UpvPdYl0gJTFzdFImoWKVBv2bIl7Ha77ss3CfD5fDh+\n/Djy8vLQtm3bs/6bM4oME4/XJ/cfjmcZTNBat4jB0aJKHC2qRJ92Tb9/uasaR075a96jIVhnZl39\npMyzRafBOvusE507r9crB+rJyclKD4ciKDU1FcePH0d1dTXM5jNvnqmv2UNFyl2BBX+xzKwHTapb\nP3aO7Rv3FZQBAFLirEiKPftXS1oX2MGUwbpayWUwZn2WwbBmnejcSTXqdrtd4ZFQpEnlL17v2X+H\nMlgPE6nHus1sgFlHvZXDrXUzN0aSO8E49b+4FKjdZ53dYNTKrdMyGKkGv4plMETNxtKX6NOUf3N9\nzR4qEqhXP/NXG1RXc3ut782Pnnp1INANhmUw6qX3mnWWwRARhZe+Zg8VkYJ11qs3TXqL5pXB7KnV\nYz0aSJl1V7UPXt/Z9y2gyAt0g9HXr1uWwRARRYa+Zg8VkXqsx1r1VacabnLNelFlUJtm/d6+E/6a\n9awoCdalBaYAgya1CmTW9fW7gJl1ouhWUFCASZMmoW3btrBarXA6nRgxYgS+++67iI/l4MGDuPzy\nyxEXF4dBgwbh0KFDdW6/9tpr8e677zZ433fffReDBw+Gw+FAXFwcevbsiSeeeAKnTp0CACxduhSC\nIMiXVq1aYfTo0cjNzZUfQxAEvP/++/Uee/r06Rg8eHCzXx+D9TApZdvGcyKVwZS7vSipbFoddnGF\nB/klVQCAzlGwIRIQ2JgGYCmMWul2B9Oa18OadaLodPPNN+Onn37Cm2++ib179+LDDz/E4MGD5SA3\nkmbMmIHWrVtj69atcDqdmDlzpnxbTk4OjEYjbr755nr3mzNnDm677Tb069cPn332GXbs2IHnnnsO\nP/30E95++235uISEBOTl5eH48eNYvnw5tm3bhuuvvz6oxaGhoK/ZQ0Xk3UtZs94kNrMRyTVdXI6e\nrmjSffcW+EtgWreIQbwtOs67wSBwF1OVC3SD0devW7nPOr/RIYo6p0+fxvr16/G3v/0NV1xxBdq1\na4eLLroIs2fPxrXXXgvAn+0WBAHbtm2rcz9BELB27VoAwNq1ayEIAr744gv07t0bMTExuPLKK1FQ\nUIDPPvsM3bp1Q0JCAu644w5UVDQeE+zevRvjxo1DVlYWxo8fj127dsnP95e//AWLFi2qd59NmzZh\n7ty5eO655/DMM89g4MCBaN++PYYNG4Z3330X48aNk48VBAFOpxOtWrXCFVdcgcceeww7duzA/v37\nQ3A2z45p3zApc/nbMbFmvelaJ8agsNyN46ercH66I+j77ZEXl0ZHVl1itxhR6fGiwsOOMGrEMhgi\nagpRFBXb6C7GbAyqS0lcXBzi4uLw/vvv4+KLL4bVam3W82ZnZ2PRokWw2+0YPXo0Ro8eDavViuXL\nl6OsrAw33XQTFi5ciIceeqjB+/fq1Qtr1qzB8OHDsWrVKvTs2RMAMHPmTNx3331o27Ztvfu88847\niIuLw+TJkxt8zBYtWjQ63pgYfxWA1Hoz3BhJhkmZy/8fjWUwTde6RQx+PlqMY0VNy6zvi6KdS2uL\nsRiBcpbBqJV+u8FwgSlROFR6vDjv0S8Uee5dT4yQ9+84E5PJhKVLl+JPf/oTXnnlFVx44YUYNGgQ\nbr/9djlQboonn3wSl1xyCQBg4sSJmD17Nn799Vd06NABAHDLLbfg66+/bjRYf/bZZzFp0iS0b98e\nPXv2xD/+8Q988803+OmnnzB//nyMHj0aP/zwA4YPH46XXnoJFosF+/btQ4cOHc66IdHvHT16FM88\n8wzatGmDzp07N/m1ngt9zR4qIpfBMLPeZOfaEWZPlAbrUkcYlsGok1QmordgXdrkiTXrRNHp5ptv\nxvHjx/Hhhx9ixIgRWLt2LS688EIsXbq0yY9VO8BPS0uD3W6XA3XpuoKCgkbv37p1a3z88cc4fPgw\nPv74Y6SkpGDy5Mn4xz/+gSeffBLx8fHYs2cP9u3bh3/84x8A/N9gBNvrvLi4GHFxcYiNjUVGRgbc\nbjfee+89eWOjcGMkGSZSGQwz600X2BipKuj7iKIol8F0cUZXsB5TkwVhZl2d5My63nYwNbEMhigc\nYsxG7HpihGLP3RQ2mw3Dhg3DsGHD8Oijj+Kuu+7CY489hvHjx8Ng8Ccoand2a6xspHZ2WxCEetlu\nQRDg8wX/u+app57C8OHDceGFF+Kuu+7Ck08+CbPZjFGjRuGrr77C1KlT0blzZ6xfvx4ej+es2fX4\n+Hj8+OOPMBgMSEtLQ2xsbL3bi4uL693v9OnTcDiCL+dtjL5SPSrCPuvnTmrfeLQJmfWTZW4UVXgg\nCECnltFVsx5TU46gVI0jnRnLYIioKQRBgN1iUuTS3J1UzzvvPJSXlwMAUlNTAQB5eXny7bUXm4bL\n7t278a9//QtPPPEEAMDr9cofEjwej9zBZcyYMSgrK8PLL7/c4OOcPn1a/rvBYECnTp3QoUOHeoE6\nAHTt2hWbN2+uc50oitiyZQu6dOnS7NfESDJMSqvYuvFcncsuplK9ersku7zwLVpI9YWVbi4wVSP9\nborEzDpRtCosLMStt96KCRMmoGfPnoiPj8cPP/yA+fPn44YbbgDgX4R58cUX4+mnn0b79u1x8uRJ\n/OUvfwnruERRxN13343nn38ecXH+xN0ll1yC1157DZ07d8Zbb72FO+64AwDQv39/zJo1CzNmzMCx\nY8dw0003IT09Hfv378crr7yCSy+9FNOmTQvqeWfOnIlx48aha9euGD58OCorK/Hqq6/i119/xZQp\nU5r9uhhJhkkZ+6yfMylYP1nmQpXHG1TwHa316kBgYySWwahToM+6vj5ESh8+2LqRKPrExcWhf//+\neP755/Hrr7/C4/EgIyMDf/rTn/DII4/Ix73++uuYMGEC+vbtiy5dumD+/PkYPnx42Mb16quvIi0t\nDSNHjpSvy87OxpgxY9C/f39cddVVdYLnv/3tb+jTpw/+/ve/45VXXoHP50PHjh1xyy231GndeDaj\nR4+GKIp49tlnMWfOHNhsNvTu3Rvffvst2rVr1+zXJYjnsk2kzpSUlMDhcKC4uBgJCQkhecxhC9Zh\nX0EZlt/VHwM7pYTkMaOFKIo4/7EvUOH24uuZg5GZUv8rp9+b/d7P+NemI5h6ZSfMGN78r5y05M//\n+Qn/2XIUfx7RBVOu6KT0cOh3xv7ze6zffxIv3n4BbrigtdLDCZnjpysx8OmvYDYK2PfUNUoPh+ic\nhGP+b4qqqirk5uYiMzMTNpst4s9PymnKv72+vpdVETmzzpr1JhMEIdARJshSmECP9ejLrLMbjLq5\ndVuz7n/febwivL6oz/kQEYWNvmYPFSljzXqzBDrCnD1YF0UR+06UAYjOYJ3dYNQtULOurzIYW60d\nWQvLXQqOhIhI3xish4EoiihzM7PeHE3pCJNXXIVSVzVMBiGokhm9kTPr3MFUlXTbDcZkRJua/6dj\nXvseR041bRMzIiIKjr5mD5WocHshrQSItzZtZyzya0pHGGlxaYfUWFh0FhAFQ+qLy8y6OknBut7e\nmwaDgH+O64tWDhv2F5Thppc34Oejp5UeFhGR7uhr9lAJqV7daBDqfFVMwWtKGcy+KO4EAwS6wbBm\nXZ0CO5jqqwwGALo6E7By8iXo1ioBJ8tcuO0fG7Fm1wmlh0WkOez1EX2a8m/OSDIMavdYb+4GA9FK\nKoM5FkSwvic/euvVgdplMAzW1Siwg6k+f906HTb8554BGNQ5FZUeL+5++we89d1BpYdFpAnSzpkV\nFSwjizZutxsAYDSePZHDguowYI/15pO6weQVV8LnE2EwNP6hZ2+UZ9alYP23UhcOnixHm8QYmIz6\nDAy1SK8167XFWU3457i+ePSDHfjXpiN49IOdOHKqArOv7nbG/7tE0c5oNKJFixYoKCgAANjtdib5\nooDP58Nvv/0Gu90Ok+nssSKjyTCQOsHEc3HpOUuLt8JoEODxivitzIW0hIZ7kPp8IvYV+IP1Ls7o\nDNbjbf7MzC/5pRj87FpYjAa0TbajY2osOqTGoUOK/8+OqbFoYbcoPNroo9duML9nNhow96YeaJNo\nxzNf7MFr3+bi2OlKLBh9QdTtKkzUFE6nEwDkgJ2ig8FgQNu2bYP6cMZoMgzKXB4AQCwz6+fMZDTA\nmWDDsdOVWP79YXRv7UBSrBmJdguSY62It5lgMAg4UlSBKo8PVpMBbZPsSg9bEX3bJ2JM/7b48VAR\nck+Ww1Xtw/6CMuwvKANQt344Nd6KXm1a4IIMB3pltEDP1i3gsHMRdLh4fSI8Xn9dop4z6xJBEDDl\nik5okxiDP//nZ3y6PR/5xRvx2h/7IjnOqvTwiFRJEAS0atUKLVu2hMfjUXo4FCEWiwUGQ3DzAqPJ\nMChlj/WQaJdsx7HTlXjxy331bjMaBCTazXK2slPLOBij9Ot2q8mIuTf1AOD/puHY6UocOFmOA7+V\n4cBv5Thw0v9nXnEVfit1Yc3uE1izOxDEd0iJRa+MFujVxh/AZ6XFo6jcjRMlVcgvqUJ+cVXN3104\nUey/7rdSF2KtJjgdVjgTbEhLsPn/dPj/dDr81yXYwrduo9rrQ7nLi1KXB+UuL8pcHtgtJmQk2VXz\nf0/aEAnQb816Q264oDWcCTbc/fYW/Hj4NG56eQP+OKAdrujaEh1SYpv8nhBFEQcLK7Dh15M4XeGB\n2SjAYjTAYjL6/24ywGoywGw0wGIyIN5mRlbLOCZMSFOMRmNQ9csUffibLAy4e2lozL66G9787iAK\ny1w4VeFBUbkbp8rdKHNVw+sTcbLMLR/bp12igiNVD4NBQEaSHRlJdgzqnFrntnJXNX7JL8G2I8X4\n6chp/HT0NA4VVvgD+5PlWLn1WJOeq9LjxckyF3YcK2l8PIK/ZaHZ6A+mLEYDzDV/Wkz+i9lgAM4S\nu4miiAq3F+WuapTVXKo8vkaPT4q1ICPJjrZJdmQkxqCt9PckO1o5bBGr6ZdKYADAEmXrCPp3SMa7\n9w7E+Dc24fCpCjz5yW48+clutE2y44ouqbiia0tc3CG50RKZgtIqbNhfiP/tP4n/7T+J48VVTXp+\nQQDaJ8eiW6t4dHMm4Lz0BHRrlYBWDhtrgolIUwSR/YJQUlICh8OB4uJiJCQkNPvxFn21D8+u2ovb\n+2Xg6Zt7hmCEVJur2ovTFR6cqgneK91eDOiYzCzaOThV7sZPR0/7g/cjp/HT0WKcKnfDYjQg7XdZ\ncylb7nTYkBpnRZmrWs6+Sxn32tn34srIfJ1rMRkQZzUh1mpEWVU1iirO/Lwmg4D0FjFy8J6RFAjm\n2ybZ4YgxhyyYO1FShf5zv4TJIGD/3GtC8phac7rCjXd/PIa1ewrw/YFTcHsDH7JsZgMGdkzBFV1S\nMaBjCg4VlmN9TXC+t2ZXYonZKODCtolonxwLj9cHl9cHd7X/4pH+XvNnYbkbv5U2vKtqC7sZ3ZwJ\n6OKMr1kXE3gcl9cHT63H8Xh9sJmNSLRbkBTrvyTGWpBktyAx1ozkWCsSY81IsltC8gFQFEV+kIiw\nUM//ROHA6CYMStkNJqysJiPSEoyNLjql4CXFWnBFl5a4oktLAP5godRVjfgg2452b+1o9LZKtxel\nVR64pGCqVlBUOzCSarrPxm41Is5qqnOJtZrqbTZUWuXBkVOVOHyqAkdOVfj/LPL/efRUJdxeHw7X\nXN+QeKsJrRNjgqoxv7ZnK9x9ecdGb3d59N8J5mxa2C2YeGkmJl6aiXJXNTb8WoivfinA2j0FyCuu\nwle/FOCrX+ovrBME4LxWCbi0UwoGdkpBv/aJsFuC/516ssyF3Xkl2J1Xgl3HS7A7rxT7fyvD6QoP\nvjtQiO8OFIbsNZqNAjq1jEe3VvE4r1UCzmvlz+Inxja+oLuo3O0fW81ld14pfi0oQ4fUWEy4NBM3\nXJCu+0XJRBQcRpNhIHWDYRkMaY0gCEiwhWbBaYzFKG/YFEnxNjPOSzfjvPT6WTKfT8SJ0io5mJcC\neimoLyh1odRVjV/yS4N6rh3HS3B9r9ZwOhr+4Ch3gmE3FAD+RffDzkvDsPPSIIoi9pwo9Qfuv/yG\nLYeL0CYxBgM7puDSTikY0DEZSWcIds8mJc6Ky7JScVlWoBysyuPF/oIy7Morwa+/+TP31tolWdLf\na/1Z6fHiVLkbRRVu+du8onIPCstdKKrwoKjCDY9XlD8YvIdAOZkzweYP4NMT0C45FocKy7E7rxS7\njpcgv6Thsp5f8ksx678/45kv9mDcgHa4s3+7Mwb9RKR/jCbDgH3WidTJYBDQyhGDVo4YXJSZVO/2\nSrcXR4sqcOx0Jby+M2f8X/pyH346WowVm49g2tCsBo+Jhh7r50oQBHR1JqCrMwGTB3c6634KoWAz\nG9G9teOM3wg1ldcn4vjpyppgvRS78oqxO68Uh09V1JSGVeHrPb81eN+2SfaabLwD3VrFo0NqHL7c\nfQJLNxxEXnEVnl21F4u+3o9b+2RgwqWZyEyJDdm4iUg7GE2GAfusE2lTjMWIrLR4ZAWxwVaZqxrT\ncrYhZ/NhTLmiY4M1ywzWg6fVzZOMtRZ1Dz/fKV9fWuXBL/mlcsb9UGEF2iXb0a2mRKarM17eI6G2\nTi3jMOHSTHzycx5e+/YAdh4vwdsbD2HZ94cwtFsa/nRZB/Rrn8jadqIowmgyDAI16+xfTaRXV3V3\nIinWgrxif+Z02Hlp9Y6Jlg2RqL54mxn92iehX/v63+CcjdlowI29W+OGC9Kx8cAp/PPbA/jylwKs\n3nUCq3edQP/MJLw18SK+r4iiBNM9YSBl1mOt/EVKpFdWkxG39GkDAFj+/aEGj5Ey679fBEsUDEEQ\nMKBjMpaM74c1Dw7CHRe1hdko4PvcU/j5aLHSwyOiCOEMEgZSzTrLYIj07Y6L2gIA1u79DUca6C7D\nbjAUKp1axmHeqB4Y2DEFALD3RHCLoIlI+xSdQb755htcd911SE9PhyAIeP/99+vcLooisrOzkZ6e\njpiYGAwePBg7d+6sc4zL5cLUqVORkpKC2NhYXH/99Th69GgEX0V9ZSyDIYoKmSmxuLRTCkQRyNl8\nuN7tgW4wDNYpNLo4/esp9gbZsYiItE/RGaS8vBy9evXCokWLGrx9/vz5WLBgARYtWoTNmzfD6XRi\n2LBhKC0N/JKaPn06Vq5ciZycHKxfvx5lZWUYOXIkvF5vg48ZCdzBlCh6jOnvz66v2HwUHm/dXVUD\nC0xZEkeh0blm8fPvN40iIv1SNJq8+uqrcfXVVzd4myiKeOGFFzBnzhyMGjUKAPDmm28iLS0Ny5cv\nx6RJk1BcXIwlS5bg7bffxtChQwEAy5YtQ0ZGBtasWYMRI0ZE7LVIXNVeuGsmaLZuJNK/YeelITXe\nit9KXVi96wSu6dFKvo3dYCjUOqfFAWAZDFE0Ue0Mkpubi/z8fAwfPly+zmq1YtCgQdiwYQMAYMuW\nLfB4PHWOSU9PR/fu3eVjGuJyuVBSUlLnEirlrkBGn8E6kf6ZjQbc1jcDAPDO7xaaujxSNxjV/qol\njenUMg6CABSWu3GyzKX0cIgoAlQ7g+Tn5wMA0tLqtkNLS0uTb8vPz4fFYkFiYmKjxzRk3rx5cDgc\n8iUjIyNk45Y6wdgtRhg12jeYiJrm9osyIAjA//YXIvdkuXw9y2Ao1OwWE9om2QEwu04ULVQbrEt+\nv/GDKIpn3QzibMfMnj0bxcXF8uXIkSMhGSsAlLo8AJhVJ4ombRLtGNzZv639vzYFFprKwToXmFII\nZbXkIlOiaKLaGcTp9O8E9/sMeUFBgZxtdzqdcLvdKCoqavSYhlitViQkJNS5AMBvpVXNHreUWefi\nUqLocmf/dgCA//xwBFU15S+BTZFU+6uWNKiL01+3voeLTImigmpnkMzMTDidTqxevVq+zu12Y926\ndRg4cCAAoE+fPjCbzXWOycvLw44dO+RjmuLed36UO7mcq0DbRgbrRNFkcJdUtHLYUFThwRc7/UmG\nQJ91lsFQ6EgdYfaxDIYoKigarJeVlWHbtm3Ytm0bAP+i0m3btuHw4cMQBAHTp0/H3LlzsXLlSuzY\nsQPjx4+H3W7HmDFjAAAOhwMTJ07EjBkz8OWXX2Lr1q0YO3YsevToIXeHaYpf8kpx77ItcjeXc3pN\nDNaJopLJaMDt/fxtHN/Z6C+FYTcYCgep1/qeE6UQRVHh0RBRuCkaUf7www+44oor5J8ffPBBAMC4\nceOwdOlSzJo1C5WVlZg8eTKKiorQv39/rFq1CvHx8fJ9nn/+eZhMJowePRqVlZUYMmQIli5dCqOx\n6Zksm9mAb/edxEPv/oznbu0FwzksEC2tYrBOFK1u65eBl77ah00HT2HviVJuikRhkZkSC6NBQGlV\nNfJLqtDKEaP0kIgojBSdQQYPHgxRFOtdli5dCsC/uDQ7Oxt5eXmoqqrCunXr0L179zqPYbPZsHDh\nQhQWFqKiogIfffTROXd3WXDbBTAaBKzcegx/++KXc3oMbohEFL2cDhuGdG0JAFj+/WH5WzqWwVAo\nWU1GZKbEAuDmSETRgOmeWi7PSsXTo3oAAP6x7gDe+F9ukx+jvCZYj2dmnSgq3Xmxf6Hpuz8eRXGl\nvzsUy2Ao1LqksSMMUbTgDPI7t/bNwJ9HdAEAPPHxLnzyc16T7l/KbjBEUe2yTinISIpBaVU1Nvxa\nCIBlMBR6WWlSRxgG60R6xxmkAZMHd8QfLm4HUQQeWLENGw8UBn3fwAJTc7iGR0QqZjAIuOMi/0JT\nr8+/+M9yDmtoiM5EzqwzWCfSPQbrDRAEAdnXn48R56fB7fXhT2/9gF/yS4K6L/usE9GtfTJgNgYW\nqLMMhkKts1Nq31gGn48dYYj0jDNII4wGAS/e3ht92yWitKoa41/fjOOnK896vzLWrBNFvdR4K4af\n75R/ZhkMhVq7JDssRgMqPV4cLTr73ERE2sUZ5AxsZiP+Oa4vOrWMQ35JFca9vgklVZ4z3qe0JliP\nZbBOFNXu7N9W/ju7wVComYwGdGzJunWiaMBg/Sxa2C14c8JFSEuwYl9BGf6ycscZN6Eoqwnm2Wed\nKLoN6JCM7q0TYDYKaJtkV3o4pENdahaZsm6dSN8YrAehdYsYvHznhTAaBHz403G8++OxRo+Vy2BY\ns04U1QRBwDt3XYyvZgyG02FTejikQ1lcZEoUFRisB6lPuyQ8MDQLAPDoBztw4LeGN6Io4w6mRFTD\nEWNGBrPqFCZSR5g97LVOpGsM1pvg3sGdcHGHJFS4vZj6r63yVuISn09Eudt/HbvBEBFROHWp6Qhz\n4LdyVHt9Co+GiMKFwXoTGA0CXritN1rYzdh5vATPfL6nzu3l7mr578ysExFROLVuEQO7xQi314eD\nhRVKD4eIwoTBehM5HTY8c0svAMA/1+di7Z4C+TapXt1sFNhXmYiIwspgEJDVkotMifSOEeU5GHZe\nGsYNaAcAmPmfn1BQWgWgbr26IAiN3p+IiCgUOnORKZHuMVg/R7Ov6YauznicLHNjxr9/gs8nyj3W\nWa9ORESRINWtM1gn0i8G6+fIZjZi4R29YTMb8O2+k/jn+gNyZj3WwmCdiIjCL4sdYYh0j8F6M2Sl\nxePRkecDAOZ/vgf/+/UkAPZYJyKiyJDaNx4srKjXoYyI9IHBejPdcVEGru7uRLVPxD/WHQDATjBE\nRBQZaQlWJNhM8PpEHPitXOnhEFEYMFhvJkEQ8PSonkivtUNhnM2s4IiIiChaCILARaZEOsdgPQQc\ndjNevKM3DDUNYJhZJyKiSOnsZN06kZ4xWA+Rfu2T8OcRXQEAF2Q4FB4NERFFiy5yZr1M4ZEQUTgw\nBRxC9w7uiDH928IRwzIYIiKKjKw0boxEpGfMrIcYA3UiIookKbN++FQFKtzVCo+GiEKNwToREZGG\nJcdZkRJnAQDsL2ApDJHeMFgnIiLSuKyWXGRKpFcM1omIiDSui5PtG4n0isE6ERGRxnVmRxgi3WKw\nTkREpHFdnOwIQ6RXDNaJiIg0rlNNzXpecRWKKz0Kj4aIQonBOhERkcY5Ysxo5bABAPYxu06kKwzW\niYiIdIB160T6xGCdiIhIBzpzJ1MiXWKwTkREpANSZp291on0hcE6ERGRDki91vcVMFgn0hMG60RE\nRDrQqaW/DOZkmRsny1wKj4aIQoXBOhERkQ7YLSa0TbIDYN06kZ4wWCciItIJqW59HzvCEOkGg3Ui\nIiKdkDrC7GFmnUg3GKwTERHphLTIdC87whDphm6C9ZdffhmZmZmw2Wzo06cPvv32W6WHREREFFGB\njZFKIYqiwqMholDQRbC+YsUKTJ8+HXPmzMHWrVtx2WWX4eqrr8bhw4eVHhoREVHEdEiNhdEgoKSq\nGidK2BGGSA8EUQcfvfv3748LL7wQixcvlq/r1q0bbrzxRsybN++s9y8pKYHD4UBxcTESEhLCOVQi\nIqKwGvLcWvz6WzkeGNoZXZxxjR4nioDHJ8Jd7YO72geP1/+n2+uDq9bPoghYTAb/xSjU/GmAxWSE\nueZnq8mA89MdyKjpRqMVnP9JC0xKD6C53G43tmzZgocffrjO9cOHD8eGDRsavI/L5YLLFcg4FBcX\nA/D/pyUiItKyzAQB+45W4LlPtkX0ef96XTfc1rdtRJ+zuaR5Xwd5S9IxzQfrJ0+ehNfrRVpaWp3r\n09LSkJ+f3+B95s2bh8cff7ze9RkZGWEZIxERkd7d/QJwt9KDOEelpaVwOBxKD4OoQZoP1iWCINT5\nWRTFetdJZs+ejQcffFD++fTp02jXrh0OHz7M/6wRUlJSgoyMDBw5coRfPUYIz3nk8ZxHHs955Gn5\nnIuiiNLSUqSnpys9FKJGaT5YT0lJgdForJdFLygoqJdtl1itVlit1nrXOxwOzf2i0bqEhASe8wjj\nOY88nvPI4zmPPK2ecybpSO003w3GYrGgT58+WL16dZ3rV69ejYEDByo0KiIiIiKi5tN8Zh0AHnzw\nQfzhD39A3759MWDAALz66qs4fPgw7rnnHqWHRkRERER0znQRrN92220oLCzEE088gby8PHTv3h2f\nfvop2rVrF9T9rVYrHnvssQZLYyg8eM4jj+c88njOI4/nPPJ4zonCSxd91omIiIiI9EjzNetERERE\nRHrFYJ2IiIiISKUYrBMRERERqRSDdSIiIiIilYr6YP3ll19GZmYmbDYb+vTpg2+//VbpIenGvHnz\n0K9fP8THx6Nly5a48cYbsWfPnjrHiKKI7OxspKenIyYmBoMHD8bOnTsVGrH+zJs3D4IgYPr06fJ1\nPOehd+zYMYwdOxbJycmw2+244IILsGXLFvl2nvPQqq6uxl/+8hdkZmYiJiYGHTp0wBNPPAGfzycf\nw3PefN988w2uu+46pKenQxAEvP/++3VuD+Ycu1wuTJ06FSkpKYiNjcX111+Po0ePRvBVEGlfVAfr\nK1aswPTp0zFnzhxs3boVl112Ga6++mocPnxY6aHpwrp16zBlyhRs3LgRq1evRnV1NYYPH47y8nL5\nmPnz52PBggVYtGgRNm/eDKfTiWHDhqG0tFTBkevD5s2b8eqrr6Jnz551ruc5D62ioiJccsklMJvN\n+Oyzz7Br1y4899xzaNGihXwMz3lo/e1vf8Mrr7yCRYsWYffu3Zg/fz6eeeYZLFy4UD6G57z5ysvL\n0atXLyxatKjB24M5x9OnT8fKlSuRk5OD9evXo6ysDCNHjoTX643UyyDSPjGKXXTRReI999xT57qu\nXbuKDz/8sEIj0reCggIRgLhu3TpRFEXR5/OJTqdTfPrpp+VjqqqqRIfDIb7yyitKDVMXSktLxays\nLHH16tXioEGDxGnTpomiyHMeDg899JB46aWXNno7z3noXXvtteKECRPqXDdq1Chx7NixoijynIcD\nAHHlypXyz8Gc49OnT4tms1nMycmRjzl27JhoMBjEzz//PGJjJ9K6qM2su91ubNmyBcOHD69z/fDh\nw7FhwwaFRqVvxcXFAICkpCQAQG5uLvLz8+v8G1itVgwaNIj/Bs00ZcoUXHvttRg6dGid63nOQ+/D\nDz9E3759ceutt6Jly5bo3bs3XnvtNfl2nvPQu/TSS/Hll19i7969AICffvoJ69evxzXXXAOA5zwS\ngjnHW7ZsgcfjqXNMeno6unfvzn8HoibQxQ6m5+LkyZPwer1IS0urc31aWhry8/MVGpV+iaKIBx98\nEJdeeim6d+8OAPJ5bujf4NChQxEfo17k5OTgxx9/xObNm+vdxnMeegcOHMDixYvx4IMP4pFHHsGm\nTZtw//33w2q14o9//CPPeRg89NBDKC4uRteuXWE0GuH1evHUU0/hjjvuAMD3eSQEc47z8/NhsViQ\nmJhY7xjOs0TBi9pgXSIIQp2fRVGsdx0133333Yeff/4Z69evr3cb/w1C58iRI5g2bRpWrVoFm83W\n6HE856Hj8/nQt29fzJ07FwDQu3dv7Ny5E4sXL8Yf//hH+Tie89BZsWIFli1bhuXLl+P888/Htm3b\nMH36dKSnp2PcuHHycTzn4Xcu55j/DkRNE7VlMCkpKTAajfU+3RcUFNTLFFDzTJ06FR9++CG+/vpr\ntGnTRr7e6XQCAP8NQmjLli0oKChAnz59YDKZYDKZsG7dOrz00kswmUzyeeU5D51WrVrhvPPOq3Nd\nt27d5IXqfJ+H3p///Gc8/PDDuP3229GjRw/84Q9/wAMPPIB58+YB4DmPhGDOsdPphNvtRlFRUaPH\nENHZRW2wbrFY0KdPH6xevbrO9atXr8bAgQMVGpW+iKKI++67D++99x6++uorZGZm1rk9MzMTTqez\nzr+B2+3GunXr+G9wjoYMGYLt27dj27Zt8qVv37648847sW3bNnTo0IHnPMQuueSSei1J9+7di3bt\n2gHg+zwcKioqYDDUnb6MRqPcupHnPPyCOcd9+vSB2Wyuc0xeXh527NjBfweiplBsaasK5OTkiGaz\nWVyyZIm4a9cucfr06WJsbKx48OBBpYemC/fee6/ocDjEtWvXinl5efKloqJCPubpp58WHQ6H+N57\n74nbt28X77jjDrFVq1ZiSUmJgiPXl9rdYESR5zzUNm3aJJpMJvGpp54S9+3bJ77zzjui3W4Xly1b\nJh/Dcx5a48aNE1u3bi1+/PHHYm5urvjee++JKSkp4qxZs+RjeM6br7S0VNy6dau4detWEYC4YMEC\ncevWreKhQ4dEUQzuHN9zzz1imzZtxDVr1og//vijeOWVV4q9evUSq6urlXpZRJoT1cG6KIri3//+\nd7Fdu3aixWIRL7zwQrmtIDUfgAYvb7zxhnyMz+cTH3vsMdHpdIpWq1W8/PLLxe3btys3aB36fbDO\ncx56H330kdi9e3fRarWKXbt2FV999dU6t/Och1ZJSYk4bdo0sW3btqLNZhM7dOggzpkzR3S5XPIx\nPOfN9/XXXzf4O3zcuHGiKAZ3jisrK8X77rtPTEpKEmNiYsSRI0eKhw8fVuDVEGmXIIqiqExOn4iI\niIiIziRqa9aJiIiIiNSOwToRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExERERGp\nFIN1IiIiIiKVYrBORERERKRSDNaJSDOys7NxwQUXRPx5165dC0EQcPr06Yg/NxERRTfuYEpEqiAI\nwhlvHzduHBYtWgSXy4Xk5OQIjcrP7Xbj1KlTSEtLO+s4iYiIQonBOhGpQn5+vvz3FStW4NFHH8We\nPXvk62JiYuBwOJQYGhERkWJYBkNEquB0OuWLw+GAIAj1rvt9Gcz48eNx4403Yu7cuUhLS0OLFi3w\n+OOPo7q6Gn/+85+RlJSENm3a4PXXX6/zXMeOHcNtt92GxMREJCcn44YbbsDBgwcbHdvvy2CWLl2K\nFi1a4IsvvkC3bt0QFxeHq666Cnl5eY0+RlFREe68806kpqYiJiYGWVlZeOONN5pzyoiIKAowWCci\nTfvqq69w/PhxfPPNN1iwYAGys7MxcuRIJCYm4vvvv8c999yDe+65B0eOHAEAVFRU4IorrkBcXBy+\n+eYbrF+/Xg623W530M9bUVGBZ599Fm+//Ta++eYbHD58GDNnzmz0+L/+9a/YtWsXPvvsM+zevRuL\nFy9GSkpKs18/ERHpm0npARARNUdSUhJeeuklGAwGdOnSBfPnz0dFRQUeeeQRAMDs2bPx9NNP43//\n+x9uv/125OTkwGAw4J///Kdcf/7GG2+gRYsWWLt2LYYPHx7U83o8Hrzyyivo2LEjAOC+++7DE088\n0ejxhw8fRu/evdG3b18AQPv27ZvxqomIKFowWCciTTv//PNhMAS+JExLS0P37t3ln41GI5KTk1FQ\nUAAA2LJlC/bv34/4+Pg6j1NVVYVff/016Oe12+1yoA4ArVq1kp+jIffeey9uvvlm/Pjjjxg+fDhu\nvPFGDBw4MOjnIyKi6MRgnYg0zWw21/lZEIQGr/P5fAAAn8+HPn364J133qn3WKmpqc163jOt17/6\n6qtx6NAhfPLJJ1izZg2GDBmCKVOm4Nlnnw36OYmIKPowWCeiqHLhhRdixYoVaNmyJRISEiL63Kmp\nqRg/fjzGjx+Pyy67DH/+858ZrBMR0RlxgSkRRZU777wTKSkpuOGGG/Dtt98iNzcX69atw7Rp03D0\n6NGwPe+jjz6KDz74APv378fOnTvx8ccfo1u3bmF7PiIi0gcG60QUVex2O7755hu0bdsWo0aNQrdu\n3TBhwgRUVlaGNdNusVgwe/Zs9OzZE5dffjmMRiNycnLC9nxERKQP3BSJiIiIiEilmFknIiIiIlIp\nButERERERCrFYJ2IiIiISKUYrBMRERERqRSDdSIiIiIilWKwTkRERESkUgzWiYiIiIhUisE6ERER\nEZFKMVgnIiIiIlIpButERERERCrFYJ2IiIiISKX+P9TxfO4/i7UCAAAAAElFTkSuQmCC\n"
+    },
+    "8b8360f6-274e-47ae-b601-dfb2fb33e812.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAssAAAHTCAYAAADRZVWwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90\nbGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9h\nAAAPYQGoP6dpAADF/0lEQVR4nOydeZwcVbn+n+p11kz2TEIWAoIsiWxhERCCQCDsIJdFhHBBBFmU\nG5BFrxK9ahAF8YLoRQkg+w9ZDCJgWJKAbIEQDEQhQEISkmFISGbt6bV+f/ScU6eX6jqn6lRv834/\nn3xgerqnq6u7ut56zvM+r2GapgmCIAiCIAiCIAoIVHoDCIIgCIIgCKJaoWKZIAiCIAiCIGygYpkg\nCIIgCIIgbKBimSAIgiAIgiBsoGKZIAiCIAiCIGygYpkgCIIgCIIgbKBimSAIgiAIgiBsoGKZIAiC\nIAiCIGygYpkgCIIgCIIgbKBimSCGIHfddRcMw8Abb7xR6U0pYPvtt8e5554rdb/jjjuu4PYtW7bg\n2muvxW677YampiYMGzYMBxxwAH77298imUz6sMWlYft67dq12v7m9ttvD8MwMHPmzKK//9Of/gTD\nMGAYBhYvXsxvnzdvHr/dMAwEAgGMHz8exxxzDP7xj39o2z6CIIh6IlTpDSAIgtDFv//9b8yaNQu9\nvb244oorcOCBByIWi+Gvf/0rvvvd7+Lhhx/G3/72NzQ1NZVtm4499li88sorGD9+vNa/29raiqVL\nl+LDDz/EjjvumPO7BQsWYNiwYeju7i762KeffhptbW3IZDJYt24dbrjhBsycOROvvfYa9t57b63b\nSRAEUetQsUwQRF2QTqfxta99Dd3d3Xj99dex8847898dc8wxOPTQQ3HGGWdg7ty5+P3vf1+27Roz\nZgzGjBmj/e8efPDBWLlyJRYsWICf/exn/PYPP/wQS5cuxTe/+U384Q9/KPrYffbZB6NHjwYAHHjg\ngdhvv/2w44474s9//jMVywRBEHmQDYMgCFteeuklHH744WhtbUVTUxMOPPBAPPnkkwX3++STT/Ct\nb30LkyZNQiQSwYQJE3Dqqafi008/BQAMDAzgiiuuwJ577om2tjaMHDkSX/7yl/GXv/xF27Y+9thj\nWLVqFa655pqcQplx+umnY9asWbjjjjvQ0dEBAFi7di0Mw8ANN9yAn/3sZ5g8eTIaGhowY8YMPPfc\nczmP/+yzz/hrjEajGDNmDA466CA8++yzJbermA1j5syZmDZtGpYtW4avfOUraGpqwg477IDrr78e\nmUxG6vUGAgGcc845uPvuu3Mes2DBAkyaNAlHHHGE1N8BgLa2NgBAOByWfgxBEMRQgYplgiCKsmTJ\nEnz1q19FV1cX7rjjDjzwwANobW3F8ccfj4ceeojf75NPPsG+++6Lxx57DHPnzsVTTz2Fm2++GW1t\nbdi6dSsAIB6P4/PPP8eVV16Jxx9/HA888AAOPvhgnHLKKfjTn/6kZXsXLVoEADjppJNs73PSSSch\nlUrl+HgB4NZbb8XTTz+Nm2++Gffeey8CgQBmz56NV155hd/n7LPPxuOPP44f/ehH+Pvf/44//vGP\nOOKII7BlyxZX29vR0YGzzjoL3/jGN7Bw4ULMnj0b1157Le69917pv3Heeedh48aNeOaZZwBk1fW7\n774b5557LgIB+6/3dDqNVCqFRCKBDz74AJdccgmi0ShOPfVUV6+FIAiiniEbBkEQRbnmmmswYsQI\nLF68GC0tLQCA4447DnvuuSeuvPJKnHbaaTAMAz/60Y+wefNmvP3229h1113540877TT+/21tbbjz\nzjv5z+l0Gocffji2bt2Km2++Geecc47n7V23bh0AYOrUqbb3Yb9j9xW3Z9GiRWhoaAAAHHXUUdh+\n++3xox/9iBfh//jHP/DNb34TF1xwAX/ciSee6Hp7t2zZgr/97W/Yb7/9AABHHHEEFi9ejPvvv196\nf+y444445JBDsGDBAsyePRvPPPMMNm7ciP/8z/8s2bzZ3t6e8/OwYcPwwAMPYPr06a5fD0EQRL1C\nyjJBEAX09fXhtddew6mnnsoLZQAIBoM4++yzsWHDBrz33nsAgKeeegqHHXZYTqFcjIcffhgHHXQQ\nWlpaEAqFEA6Hcccdd+Bf//qXr69FxDRNAIBhGDm3n3LKKbxQBsAV9KVLlyKdTgMA9ttvP9x11134\n6U9/ildffdVzskZ7ezsvlBlf+tKX8PHHHyv9nfPOOw8LFy7Eli1bcMcdd+Cwww7D9ttvX/Ixzz77\nLJYtW4bXX38df/3rX3HEEUfgjDPOwGOPPab6MgiCIOoeKpYJgihg69atME2zaILDhAkTAIDbDz77\n7DNMnDix5N979NFHcdppp2G77bbDvffei1deeQXLli3Deeedh4GBAS3bPHnyZADAmjVrbO/DfMOT\nJk3KuT1faWW3JRIJ9Pb2AgAeeughzJkzB3/84x/x5S9/GSNHjsQ555zD/c+qjBo1quC2aDSKWCym\n9HdOPfVUNDQ04Ne//jWeeOIJnH/++Y6P2WOPPTBjxgzsu+++OPbYY/Hwww/jC1/4Ai655BKl5yYI\nghgKULFMEEQBI0aMQCAQwKZNmwp+t3HjRgDgaQpjxozBhg0bSv69e++9F1OnTsVDDz2Ek046CQcc\ncABmzJiBeDyubZuPPPJIAMDjjz9ue5/HH38coVCoIJ+4WMHb0dGBSCTClfXRo0fj5ptvxtq1a/Hx\nxx9j/vz5ePTRR6Uyof2kqakJZ5xxBubPn4/m5maccsopyn8jEAhg9913x6ZNm9DZ2enDVhIEQdQu\nVCwTBFFAc3Mz9t9/fzz66KM5Smcmk8G9996LiRMn8sSJ2bNn44UXXuC2jGIYhoFIJJJjf+jo6NCa\nhnHyySdjt912w/XXX4/333+/4PcPPfQQ/v73v+Ob3/xmgZL86KOP5ijcPT09eOKJJ/CVr3wFwWCw\n4G9NnjwZl156KY488kgsX75c22twy7e//W0cf/zx+NGPfpRjJ5ElnU5j5cqViEajGDZsmA9bSBAE\nUbtQgx9BDGGef/75opPljjnmGMyfPx9HHnkkDjvsMFx55ZWIRCK47bbb8M477+CBBx7ghe9PfvIT\nPPXUUzjkkEPw/e9/H9OnT8e2bdvw9NNPY+7cudhll11w3HHH4dFHH8XFF1+MU089FevXr8f//M//\nYPz48Vi9erWW1xIMBvHII4/gyCOPxJe//GVcccUV+PKXv4x4PI4nnngCt99+Ow499FDceOONRR97\n5JFHYu7cuchkMvjFL36B7u5u/PjHPwYAdHV14bDDDsPXv/517LLLLmhtbcWyZcvw9NNPu1JydbPn\nnnuWVNTzefPNN3lc3KeffooFCxbg3//+N/7rv/7LVbFNEARRz1CxTBBDmKuvvrro7WvWrMGhhx6K\n559/Htdddx3OPfdcZDIZ7LHHHli4cGHOmOntttsOr7/+Oq677jpcf/312LJlC8aMGYODDz4YI0eO\nBAD853/+Jzo7O/H73/8eCxYswA477IBrrrkGGzZs4AWpDnbZZResWLECv/rVr3DPPffgf/7nfxAK\nhbDbbrvh5ptvxre+9a2iWcKXXnopBgYG8J3vfAednZ3Yfffd8eSTT+Kggw4CADQ0NGD//ffHPffc\ng7Vr1yKZTGLy5Mm4+uqrcdVVV2nb/nJx9NFH8/8fOXIkdtppJyxYsABz5syp4FYRBEFUJ4bJ2sMJ\ngiCGGGvXrsXUqVPxy1/+EldeeWWlN4cgCIKoQsizTBAEQRAEQRA2ULFMEARBEARBEDaQDYMgCIIg\nCIIgbCBlmSAIgiAIgiBsoGKZIAiCIAiCIGygYpkgCIIgCIIgbKCcZWSnkm3cuBGtra05E8YIgiAI\ngqhfTNNET08PJkyYgECA9EOiOFQsA9i4cSMmTZpU6c0gCIIgCKICrF+/HhMnTqz0ZhBVChXLAFpb\nWwFkD5Zhw4ZVeGsIgiAIgigH3d3dmDRpEq8DCKIYVCwD3HoxbNgwKpYJgiAIYohBFkyiFGTQIQiC\nIAiCIAgbqFgmCIIgCIIgCBuoWCYIgiAIgiAIG6hYJgiCIAiCIAgbqFgmCIIgCIIgCBuoWCYIgiAI\ngiAIG6hYJgiCIAiCIAgbqFgmCIIgCIIgCBuoWCYIgiAIgiAIG6hYJgiCIAiCIAgbqFgmCIIgCIIg\nCBuoWCYIgiAIgiAIG6hYJgiCIAiCIAgbqFgmCIIgCIIgCBuoWCYIgiAIgiAIG6hYJgiCIAiCIAgb\nqFgmCIIgCIIgCBuoWCYIgiAIgiAIG6hYJgiCIAiCIAgbqFgmCIIgCIIgCBuoWCYIgiAIgiAIG6hY\nJgiCIAiCIAgbqFgmCIIgCIIgCBuoWCYIgiAIgiAIG6hYJgiCIAiCIAgbKlosz58/H/vuuy9aW1sx\nduxYnHTSSXjvvfdy7mOaJubNm4cJEyagsbERM2fOxLvvvptzn3g8jssuuwyjR49Gc3MzTjjhBGzY\nsKGcL4UgCIIgCIKoQypaLC9ZsgSXXHIJXn31VSxatAipVAqzZs1CX18fv88NN9yAm266CbfeeiuW\nLVuG9vZ2HHnkkejp6eH3ufzyy/HYY4/hwQcfxEsvvYTe3l4cd9xxSKfTlXhZBEEQBEEQRJ1gmKZp\nVnojGJ999hnGjh2LJUuW4JBDDoFpmpgwYQIuv/xyXH311QCyKvK4cePwi1/8AhdeeCG6urowZswY\n3HPPPTj99NMBABs3bsSkSZPwt7/9DUcddZTj83Z3d6OtrQ1dXV0YNmyYr6+RIAiCIIjqgM7/hAxV\n5Vnu6uoCAIwcORIAsGbNGnR0dGDWrFn8PtFoFIceeihefvllAMCbb76JZDKZc58JEyZg2rRp/D75\nxONxdHd35/wjCIIgCIIgiHyqplg2TRNz587FwQcfjGnTpgEAOjo6AADjxo3Lue+4ceP47zo6OhCJ\nRDBixAjb++Qzf/58tLW18X+TJk3S/XIIgiAIgiCIOqBqiuVLL70U//znP/HAAw8U/M4wjJyfTdMs\nuC2fUve59tpr0dXVxf+tX7/e/YYTBEEQRJWz+tMevLH280pvBkHUJFVRLF922WVYuHAhXnjhBUyc\nOJHf3t7eDgAFCnFnZydXm9vb25FIJLB161bb++QTjUYxbNiwnH8EQRAEUa/MWfA6zrj9VWztS1R6\nUwii5qhosWyaJi699FI8+uijeP755zF16tSc30+dOhXt7e1YtGgRvy2RSGDJkiU48MADAQD77LMP\nwuFwzn02bdqEd955h9+HIAiCIIYyn/bEkcqY2ELFMkEoE6rkk19yySW4//778Ze//AWtra1cQW5r\na0NjYyMMw8Dll1+On//859hpp52w00474ec//zmamprw9a9/nd/3/PPPxxVXXIFRo0Zh5MiRuPLK\nKzF9+nQcccQRlXx5BEEQBFFx0hkT6Uw2+GogSZGqBKFKRYvl3/3udwCAmTNn5tx+55134txzzwUA\nXHXVVYjFYrj44ouxdetW7L///vj73/+O1tZWfv9f//rXCIVCOO200xCLxXD44YfjrrvuQjAYLNdL\nIQiCIIiqJJHK8P+nYpkg1KmqnOVKQTmLBEEQRL3S1Z/EHj/5OwDgnvP3w1d2GlPhLaoe6PxPyFAV\nDX4EQRAEQfhDIi0qy5kS9yQIohhULBMEQRBEHSMWyzGyYRCEMlQsEwRBEEQdQ55lgvAGFcsEQRAE\nUcdQsUwQ3qBimSAIgiDqGCqWCcIbVCwTBEEQRB2T41lOUIMfQahCxTJBEARB1DE5ynKKlGWCUIWK\nZYIgCIKoY3KVZSqWCUIVKpYJgiAIoo4RleU4KcsEoQwVywRBEARRx+Q2+JFnmSBUoWKZIAiCIOqY\nRNpSk8mGQRDqULFMEARBEBIsW/s5DrnhBTz/708rvSlKJFMm/39q8CMIdahYJgiCIAgJFr/XiXWf\n9+PZf3VWelOUiFODH0F4goplgiAIgpCAeX9FD3AtkBsdV1vbThDVABXLBEEQBCFBXRTLpCwThDJU\nLBMEQRCEBPHBorPW4tdoKAlBeIOKZYIgCIKQoGaVZUrDIAhPULFMEARBEBKwRjlxIl4tkEwLaRhJ\nKpYJQhUqlgmCIAhCgppVlmkoCUF4goplgiAIgpCgVovluLC9iXQG6YxZ4t4EQeRDxTJBEARBSJDg\nDX61VSznF/dkxSAINahYJgiCIAgJWApGrSnL+R5rKpYJQg0qlgmCIAhCAlZ01p6ynFscx6hYJggl\nqFgmCIIgCAm4Z7mG0zAAavIjCFWoWCYIgiAICWq1wY88ywThDSqWCYIgCEICKpYJYmhCxTJBEARB\nSJCo0aEk8YIGv9rafoKoNFQsEwRBEIQE8cEiM50xkaqhgjlfWaYGP4JQg4plgiAIgpBAVGhrSV1O\nDm5rwMj+TDYMglCDimWCIAiCcMA0zRyFtpZ8y2xbWxvCAEhZJghVqFgmCIIgCAfy49dqsVge1hgC\nAMSpWCYIJahYJgiCIAgH8m0XtTSYhG37MFKWCcIVVCwTBEEQhAP5SnIteZbZtrc1ZotlSsMgCDUq\nWiwvXboUxx9/PCZMmADDMPD444/n/N4wjKL/fvnLX/L7zJw5s+D3Z5xxRplfCUEQBFHPxPNGRsdr\nqODkNgxSlgnCFRUtlvv6+rDHHnvg1ltvLfr7TZs25fxbsGABDMPA1772tZz7XXDBBTn3+7//+79y\nbD5BEAQxRKhVZdk0Tb6tlrJMxTJBqBCq5JPPnj0bs2fPtv19e3t7zs9/+ctfcNhhh2GHHXbIub2p\nqangvgRBEAShi4JiuUY8y2JjImvwo2KZINSoGc/yp59+iieffBLnn39+we/uu+8+jB49Grvvvjuu\nvPJK9PT0lPxb8Xgc3d3dOf8IgiAIwo78hr5aKZZFBZzZMMizTBBq1EyxfPfdd6O1tRWnnHJKzu1n\nnXUWHnjgASxevBg//OEP8cgjjxTcJ5/58+ejra2N/5s0aZKfm04QBDEk6OwZwBm3v4In3t5Y6U3R\nTr7tIpGuDXVWLOqHDdowYona2HaCqBYqasNQYcGCBTjrrLPQ0NCQc/sFF1zA/3/atGnYaaedMGPG\nDCxfvhx777130b917bXXYu7cufzn7u5uKpgJgiA88tLqzXj1o88RDBg4fo8Jld4crdSqDYNtZzBg\noCkSBAAMpKhYJggVaqJYfvHFF/Hee+/hoYcecrzv3nvvjXA4jNWrV9sWy9FoFNFoVPdmEgRBDGnY\n8n4tJUXIkm/DqJWcZVYsR4IBNLJimTzLBKFETdgw7rjjDuyzzz7YY489HO/77rvvIplMYvz48WXY\nMoIgCILB4tVqpZBUIV9JrpXXyOwjkVAADaFssRyrw4sZgvCTiirLvb29+OCDD/jPa9aswYoVKzBy\n5EhMnjwZQNYi8fDDD+PGG28sePyHH36I++67D8cccwxGjx6NVatW4YorrsBee+2Fgw46qGyvgyAI\ngrAKyPxM4nqg1m0YkVAADeFssUzjrglCjYoWy2+88QYOO+ww/jPzEc+ZMwd33XUXAODBBx+EaZo4\n88wzCx4fiUTw3HPP4Te/+Q16e3sxadIkHHvssbjuuusQDAbL8hoIgiCILAleLNdGIalCfkNfzRTL\nadGGkV1MpqEkBKFGRYvlmTNnwjTNkvf51re+hW9961tFfzdp0iQsWbLEj00jCIIgFOE2jDpc5q/V\noSSishwNkWeZINxQE55lgiAIovphRXI9pi3UvA1DaPCj6DiCUIOKZYIgCEIL3LNch8pyYRpGbRSc\nzD4iepYHaqTQJ4hqgYplgiAIQgtWGkba0WJXa9TsBL9U9n2IhAJoHCyWE6kM0pn6en8Iwk+oWCYI\ngiC0wArIjAmk6qwYq1kbhtDg1xC2Tvm1oowTRDVAxTJBEAShBVF9rbdEjMJx17Xx+lhRHxZylgHy\nLROEClQsEwRBEFrIKZbrLHGBFZ0BI/tzrVwMiA1+gYCBSCh72iffMkHIQ8UyQRAEoQVxab9WiklZ\nWNHZEg3l/FztJAbfk+hgkcx8y6QsE4Q8VCwTBEEQWkjUsw1j8PW0NoRzfq52kmmrwQ8A9y1T1jJB\nyEPFMkEQBKGFXM9yfRVj7PW0NoQGf66NYlls8AMsZZmKZYKQh4plgiAIQgtivnK9ZS2zorPWbBhx\nYYIfACtruc7eH4LwEyqWCYIgCC0MCc/yoLJcc2kYwdxiOUbKMkFIQ8UyQRAEoYX6tmHUprKcKFCW\nybNMEKpQsUwQBEFoIafBr86W+Wu1wU8cdw2INgwqlglCFiqWCYIgCC0MhaEkw8pow+iNp/D0O5s8\nFbbJwXHX+dFxVCwThDxULBMEQRBayPUs11cxxpRyZsMox9CV25d+hIvuXY77X1vn+m/kp2FQgx9B\nqEPFMkEQBOGZTMbkmb5A/SrL5Wzw+7RrIPvfngHXf6PQs0wNfgShChXLBEEQhGfyi8d6HXfNleUy\nXAwwdd6L/ztekIZBDX4EoQoVywRBEIRn8gu6ulOWeYNf+dIwmFXCi6WF2zDyx11TsUwQ0lCxTBAE\nQXgmv6Cru2I5nZeGkc7ANM1SD/GMDmU5kbJLw6iv94cg/ISKZYIgCMIz+cVxvTX45SvLpgmkMv4W\ny6ygHfCwL5mPnMZdE4R7qFgmCIIgPFOgLNeZcsleH/MsZ2/z9zXqUZazj43SUBKCcA0VywRBEIRn\nCpXl+imWxaQPsVj227dseZa9F8uUhkEQ7qFimSAIgvBMPdswxKSPhkgQwYCRvb1MyrIXFZhte7gg\nZ7l+3h+C8BsqlgmC8MxHn/Xi90s+RH8iVelNISpEPadhiMVyJBjg/t9aVJatNIz6eX8Iwm9Cznch\nCIIozc3PrsbCtzdi3LAoTt5rYqU3h6gA9exZFoviaCiAaDiAWDKNRNpfdZYVyV5UevY38if41VsO\nNkH4CSnLBEF4Zmt/IvvfvmSFt4SoFPkqa13ZMISC0zAMXnj63uCXZDYM98+TzM9ZjmT/S55lgpCH\nimWCIDwT1xBxRdQ29dzgF8+zMrD/+p+GoWEoSV4aRjREnmWCUIWKZYIgPKMj4oqobfILx3oqxvJ9\nv+y/fnqW0xmTe6U9eZZtG/zoWCUIWahYJgjCMzqGJxC1DbtgipZJdS0niTzfbzka/MS/7fbCI50x\nkR4cnGLZMCg6jiBUoWKZIAjPkLJMsPd+WGN2HHRdFcvp3JHR0TIoy2KBHE+5G60tbh/PWRa2PePz\nBEKCqBeoWCYIwjNWxBWpVUMVttzPxkHX02chbuP7FSPl/HpOIDtamw1FUSGnWA7mKssArQQRhCxU\nLBME4ZmBlPeufaK24cpyQzjn53qgEp7lfOuFm8JWLObDwewglYaQUCzX0XtEEH5CxTJBEJ6Jk7I8\n5GHvfV3aMGzTMPz7vBeki7gobBNCbJxhZIvlQMDg20++ZYKQo6LF8tKlS3H88cdjwoQJMAwDjz/+\neM7vzz33XBiGkfPvgAMOyLlPPB7HZZddhtGjR6O5uRknnHACNmzYUMZXQRBDG9M0SVkmeHE3rI5t\nGOVs8MtXlt3sz/zGRAbzLddTYglB+ElFi+W+vj7sscceuPXWW23vc/TRR2PTpk3839/+9rec319+\n+eV47LHH8OCDD+Kll15Cb28vjjvuOKR9nqxEEESWRDoD1ntEJ9+hCyvMWhssZdlNU1o1Yq8sl8ez\n7Pa58rebwRMxEnS8EoQMFR13PXv2bMyePbvkfaLRKNrb24v+rqurC3fccQfuueceHHHEEQCAe++9\nF5MmTcKzzz6Lo446Svs2EwSRi3gSr6eld0INy4aRPa2wprRIyKjkZmmB2Rmi+Z5lHxv8CjzLLi5E\nbZVlNvK6jtR/gvCTqvcsL168GGPHjsXOO++MCy64AJ2dnfx3b775JpLJJGbNmsVvmzBhAqZNm4aX\nX37Z9m/G43F0d3fn/CMIwh3iSZyU5aGLZcMIC7fVx+fBmoIXHPyv/zYMLcpy2kZZDjNlmS5uCUKG\nqi6WZ8+ejfvuuw/PP/88brzxRixbtgxf/epXEY/HAQAdHR2IRCIYMWJEzuPGjRuHjo4O2787f/58\ntLW18X+TJk3y9XUQRD0jNh6Rsjx0sdIwrAXLevk8VEMahqsGPxsbRjRMI68JQoWK2jCcOP300/n/\nT5s2DTNmzMCUKVPw5JNP4pRTTrF9nGmavPO3GNdeey3mzp3Lf+7u7qaCmSBcIqqHdPIdunCrQjiI\nSCiARCpTP8VyOq/BrwKeZS/Rcfk2jMYwpWEQhApVrSznM378eEyZMgWrV68GALS3tyORSGDr1q05\n9+vs7MS4ceNs/040GsWwYcNy/hEE4Q4xAYPSMIYu4rhrPvK6Toox9jr4BL9KpGF4UJbDoeKeZbq4\nJQg5aqpY3rJlC9avX4/x48cDAPbZZx+Ew2EsWrSI32fTpk145513cOCBB1ZqMwliSCEqy/XiUSXU\nYcVctlhmDWT1cfEUT1eBDcNDdFy0QFmmYpkgVKioDaO3txcffPAB/3nNmjVYsWIFRo4ciZEjR2Le\nvHn42te+hvHjx2Pt2rX4/ve/j9GjR+Pkk08GALS1teH888/HFVdcgVGjRmHkyJG48sorMX36dJ6O\nQRCEv4hqcj1NbSPUiAtNcNEy2BTKia1nuUzjrgG3Q0lyFXGGpSzXx/tDEH5T0WL5jTfewGGHHcZ/\nZj7iOXPm4He/+x1WrlyJP/3pT9i2bRvGjx+Pww47DA899BBaW1v5Y379618jFArhtNNOQywWw+GH\nH4677roLwWCw4PkIgtCPqE4l0hmkMyaCgdqPCyPUEG0YDeH6smFYaRiDNoxB5dzXNAwNynIylc25\nLiyWybNMECpUtFieOXNmydD6Z555xvFvNDQ04JZbbsEtt9yic9MIgpAkXwFLpDJ86AExdBDV13qz\nYdTqUJK4TYMfeZYJQo2a8iwTBFF96BieQNQ+OTaMcJ3ZMPLTMILs9fn3Wdc6lIRsGAThCSqWCYLw\nRP4J103EFVH78GI5LKRh1MlnQWxeBMrT4Kdz3HXYpsGPbBgEIQcVywRBeCK/IKImv6EJ89jmpGHU\nyWchfxJeOcddM/+/l2LZzrNcL55ygvAbKpYJgvAEKcsEIAwlGUppGGVQltlERFc2jLR1ASNCyjJB\nqEHFMkEQniBlmUhnTCTTVvICG6dcLzaMhODHzv63fDnLbY1hAO6OK/E9EaFx1wShBhXLQ5BUOoP3\nP+0pmURCELIUKMt0Ah5yiEVjzgS/OlGW81MlomXMWebFsoehJIXjrklZJggVqFgegvzmudWY9eul\neHLlpkpvClEHFHTt10mBRMgjFnK5467r47NQYMMI+p+zzI6rYYPFspvkijilYRCEFqhYHoJ8tLkP\nALB28L8E4YXCSWOkVg012GcgGDAQCoo5y/XxWWCvoxI5yzqUZbs0DFoFIgg5qFgeggwksl+QtARH\n6CC/OCZleeiRv9xfdznLFWjwy1eWXaVhpO2U5UDOcxAEURoqlocgLK0glqiPExlRWfLTL+gEPPTg\no67DuZ7eelGW8y8GypmG0cZtGG6U5VxFnNFAnmWCUIKK5SFIbFBZpogvQgf5vtR6URMJeQbyhnbU\na85yQ7iwwc+vRmm2T9s8KMssDSNqO+66Pt4fgvAbKpaHIOwLktkxCMIL7KJrcHYCeZaHIHGbaLV6\nseRYynL29YlKrV+JGEyV91Is2w0laYyQskwQKlCxPARhy3n0RUnogKmHXryV1c7vFn+Ik377D/QM\nJCu9KVVJflEWrbMJcYVpGIGC3+kmntRhwyje4Ncg2EgyGYoQJQgnqFgeglCxTOhkIE8Bq0fP8sNv\nrMeK9duwYv22Sm9KVcI9y/k2jDq4cEpnTKQyucM9xGLZj9eYyZhcsfaiLMdtG/yC/P/JjkcQzlCx\nPARhRXI9FjVE+WG2nuF1XCyzYyZG1qWiWDaM+mvwE5VjVnQGAgbCQaPg97oQC+NhDWyCn4ehJKWK\nZfItE4QjVCwPQdiXY4y+JAkNsIKonm0YMVqNKYmdZ7kePgs5xbKgKLP/96dYtj5nfMXGVYNf8Ql+\nwYDBb6vHi1uC0A0Vy0MM0zQtZZlUMkID+V379Xjy5QkydfjadMBUTx4dF66fNIx4OvvaDANcTQas\n1+hHgx87pkIBA01Ra1qgavKGnbIMWMkedAFIEM5QsTzEEJUe+pIkdMAKyDYPY3mrmUzG5McN2TCK\nk8hTMOvRhhEJBmAYVrFcDmW5IRzk+zJ7u9pzJfLsMSINNMWPIKShYnmIIX4xUrFM6EDHWN5qRmyA\nIutScZiCzNTWerRh5Kuzfo68FnOrmbUFUFfq2UVMfhoGYMXHUbFMEM5QsTzEEAtk+pIkvJLJmLyY\nGN5Un8qyqCbTBWZxChv8ypOGEUukcc+rH2NTV8y352AFZ746G/FRPWffzQ3hIMJBw8owV3yukjaM\nEA0mIQhZqFgeYohfjFQsE14Ri6F6VZbpAtOZgui4MuUsP/H2Rvzw8Xfwm2dX+/YcTM3Nb5Lz14Zh\nFeiGYbi++ChZLLPBJGQtIghHqFgeYohfjMm0ybulCcINYmHMIq7qTanKsS5RYVGUgqEkZbJhfNYb\nBwBs7k349hwJm6ziSMi/YnmAN0wO2lrC6skVpmkWeMlF2GASylkmCGeoWB5i5H8xklJGeEHs2m+O\nhgZvq6/PVCxBTbFOFEbHWUqoaoKDCuyz5udnLpH32hjsgsCPNIx8W0uDC2U5mbb2ezFluZGUZYKQ\nhorlIUZ+XFy9qYBEeRG79ll3vV/jfytFjJpiHbGzYQD+FJMMVuj5+b44Nfj5qSw3hPNsLQoqsLjf\ni6ZhMM9ynR2vBOEHVCwPMUhZJnSS27Vfn0MOcjzLpMIVxUrDyLVhAP5aMcoxWTFuUyxHfSyW7Ya8\nqIgb4naVTMOgzzRBOELF8hBDXFIGSCkjvCF27fPc1jpTqigNw5n84i6bSTz4Ox9Xr2LlsGHY+H79\njY7LVZbZsaWkLA9uVzBgIBgwCn7f4MIHTRBDFSqWhxj5X4zkVyO8IHorG8qUgFBuKJvcmXyrQjbB\nwf/BJOy98fN9YZ/nAhtGWdIw8nKrXSjLxZr7AKsAp880QThDxfIQI/+LkVQFwgti1360Tj2QOZ5l\nurgsSr5nOfv//mctl8Wz7JSG4cu46zzPMj+21D3LxZr7sn+bcpYJQhYqlocYBcoyFcuEB8STOjux\npzMmUnUUSdgvFMh0cVmc/PQG8f/LYcPw8yLGbmS0nxcD+cqytWrjQlm2K5ZDpCwThCxULA8x8k/2\ndPInvCAWSWK0Vj2py2TDcCa/uAPcJTiowsaPx1MZZDL+RNRVIg0jbqMsqxTmpTKWAaAxUp+2KYLw\nAyqWhxj5J3s6+RNeEBv8ROWtni7CRNWyn2wYRSmmvpbHhpGy/t+nz5ydsuxrsWybhqHe4Odkw6Bz\nAEE4Q8XyECPfn5afjkEQKrCTekMoiEDA8DUhoFLQuGtninuW/f8slCMD2zYNI+ifcl6Ys+xCWZZs\n8KPPNEE4Q8XyEIMa/AidWA1+ufm69fS5Eo8ZGhFfHK6Ehot5ln20YYjTFX1S/e1ylsuhLLOC1k2y\nSCJdPMWDQQ1+BCFPRYvlpUuX4vjjj8eECRNgGAYef/xx/rtkMomrr74a06dPR3NzMyZMmIBzzjkH\nGzduzPkbM2fOhGEYOf/OOOOMMr+S2oEa/AidiMoyUJ9qVeHUy/p5bbrgQ0lEz3IZbBgDZVD97ewM\nfo675heheRMR1YaSZD3cdsVyI9kwCEKaihbLfX192GOPPXDrrbcW/K6/vx/Lly/HD3/4QyxfvhyP\nPvoo3n//fZxwwgkF973ggguwadMm/u///u//yrH5NQn7EmYZ9XTiJ7wQz1OWedd+ndowiv1M2Ngw\nfP4smKZZFhsGV5aDwZzby+JZHixoG0IuhpI4NPjRUBKCkCdUySefPXs2Zs+eXfR3bW1tWLRoUc5t\nt9xyC/bbbz+sW7cOkydP5rc3NTWhvb3d122tF5gyMbwpgs/7EpQbS3hioGC5uP6U5QLrEvn8Cyim\nvvo9lCSZNpEWEjD8+i5LFLGYAP6Ou7ZTlnVGxzXW4SoQQfhFTXmWu7q6YBgGhg8fnnP7fffdh9Gj\nR2P33XfHlVdeiZ6enpJ/Jx6Po7u7O+ffUIGdUIY3hQGohdwTRD68ESmUpyzXkQ8yvwgjZbmQotFx\nTA316bNQLsXfady1HzaMQs+y+sAfViyHHRv86udYJQi/qKiyrMLAwACuueYafP3rX8ewYcP47Wed\ndRamTp2K9vZ2vPPOO7j22mvx9ttvF6jSIvPnz8ePf/zjcmx21cFOKCObIvgIfZSGQXiCe1XzTup+\nZuuWG/L5lyaVziA1qPCWMw2jXJnxiZTduGv/LgbylWU3o+QTRawxIhQdRxDy1ESxnEwmccYZZyCT\nyeC2227L+d0FF1zA/3/atGnYaaedMGPGDCxfvhx777130b937bXXYu7cufzn7u5uTJo0yZ+NrzLY\nl/DwpkjOzwThhoFU8ZN6PalVBQomWZdyEJXVnDQMnz2x5VL8ndIw4mVUllUuPJLp0g1+5FkmCHmq\nvlhOJpM47bTTsGbNGjz//PM5qnIx9t57b4TDYaxevdq2WI5Go4hGo35sbtXDvhhHDNowSFUgvMBU\ntQYPjUjVDsUtlkb07IpWBb/TMAovYvx5nkoMJSnwLLsZSuI0wU/Ibs5kTARY1zdBEAVUtWeZFcqr\nV6/Gs88+i1GjRjk+5t1330UymcT48ePLsIW1B1P8RjSTsuxEPJXGsf/7Iq555J+V3pSqJV9ZdhNx\nVe2wImxYQ1ZboAvMXFgxHAwYCAWL2TB8UpbL5Vm2Ge7Bfk748PryleUGF0NJ7BRxBvubqn+XIIYi\nFVWWe3t78cEHH/Cf16xZgxUrVmDkyJGYMGECTj31VCxfvhx//etfkU6n0dHRAQAYOXIkIpEIPvzw\nQ9x333045phjMHr0aKxatQpXXHEF9tprLxx00EGVellVTYwry5Gcn4lCPvqsD+9u7MbHW/px/de+\nVOnNqUrEcdeApSzX00UYey0jmyPoHkiRDSMPK2M5Py3CZ2U534YhjL7WCVNoC9IwwmXMWXbh/5Zt\n8AOy54HGSLDo/QiCqHCx/MYbb+Cwww7jPzMf8Zw5czBv3jwsXLgQALDnnnvmPO6FF17AzJkzEYlE\n8Nxzz+E3v/kNent7MWnSJBx77LG47rrrEAzSgV+MAhsGnfht6R/cN/2JFEzThGHQMmU+8bwlajdj\neasZMct3RHMEa7f00wVmHsUylgF3cWcqlMuznLDLWQ76Z8OwneCn1OBXWlkOBgxEggEk0pm6urgl\nCD+oaLE8c+ZMmKZp+/tSvwOASZMmYcmSJbo3q27JZEz+JUwNfs6wk3HGzJ68RCWGyDKQLH5Sr5fP\nlZjlO5KOmaLYLfeX3Ybhs2fZ/vXpfd5MxizwSbuxYTiNuwayFzSJdIYuAAnCgar2LBN6Eb9oRzaT\nDcOJPmFZt58U+KLE820YdZbdKh4fzOdPqzG5FMtYFn8uW4NfhdIwdCvL4v7iF6Fh9QuPZKowzi8f\nGkxCEHJQsTyEyDnxs6EkdVLU+IFYFPX75IesdQpsGD6rieWGFRGhgIFWavAriq0No05yluN2DX6+\nFcvW67COK/WLUKc0DEC8uKXPNEGUgorlIQT7QoyEAmiK0onfCVFNJjWxOAUNfvWmLA++743hIFfh\n6JjJhV8w2TTAqfhsVShs8Cv3UJLsz6mMiUymtGVQBXbshIR0kQYXyrKTZxkQleX6OF4Jwi+oWB5C\nsJN8QyjAxxMnUhnuySRy6ScbhiNWsez+pF7NsPe9IRKkJWsbrDSMytgw/L6I4WkYBQ2MwYL76KCY\nUs/2peihd/47pdMwABpMQhCyULE8hOAqWSSYExNEX5TFybVh0D4qRr5f1c1ycTUjFmTsmKFVhlzs\nlvv9tmGw98bv/gvboSTC69X5GvObZvOfW/ZClL8vJZRlGnlNEHJQsTyEYF+yDeEgz8MFqFi2o1/Y\nL7EkeZbzSaUzSA2qXPWqLA8IxXI5C4utfQms2dzn+/PogNksCmwYPvvXBxIs0o/1X+h/nlQ6Aybk\n5hed4aAVJanTt1xcWRaKZckLUTv7iEi92aYIwi+oWB5CsGilxnAQgYDBv4BJVSgOKculEdU0pijz\niKs6OfnGitgwyvFZOGfB6zjypiXo7Bnw/bm8kt/kyYj6/FkoGLDkw/sifsbzi07DMPhtOi8IiinL\noWAAocFx1LIqdjKdrfJLNfiRD58g5KBieQhh34xFX5TFIM9yacTPTX4axkCdKMuWDSPAbRjlOF7W\nbu5DKmPik60x35/LK/bRcX7bMLJ/108bhqgYFys6oz4MJonbKMKqGeZ29hGRBp+bMAmiXnBdLC9f\nvhwrV67kP//lL3/BSSedhO9///tIJBJaNo7QSyyvGYurCj6F+dc6lIZRmgGh2z4wqHrV2wVYjme5\nTCqcaZo847sWLtKcijvfhpIM7iM/lWXm+w0Y4MkUIjw+TmODXzFlWfxZ9uJDKg2DfPgEIYXrYvnC\nCy/E+++/DwD46KOPcMYZZ6CpqQkPP/wwrrrqKm0bSOhjIK97nCtldaIC6qYSNoz1n/ejL14b/mju\nVS3irayXcdf8mIkInmWfPwvxlOWTrYXPgp2C6ffo83I0+CVsVHNGNOSfsmyfW63W4FcqDYM35NI5\ngCBK4rpYfv/997HnnnsCAB5++GEccsghuP/++3HXXXfhkUce0bV9hEbsbBikKhQnV1n2v2hZ/3k/\nDv3lC/jWPW/4/lw6KKaA1Z2ynLCOGcuG4e+FQH+NeeXtbBhiPKVp6o+njPEGP6tY1v08dtP7GH4M\nJrFTlqOKzXhqynJ9XNwShF+4LpZN00Qmkz3Ann32WRxzzDEAgEmTJmHz5s16to7QCvuS5cpymBr8\nSiGmYZSjaFm7pQ8ZE/jos9pIQRjg6SrW14iVhlEfJ192bDRFymfDEL3yfTUwOZLnLBcMJbGKPT8+\nD9yzPGjDME39z+NUcPpRLOtSlu0mD4o0kLJMEFK4LpZnzJiBn/70p7jnnnuwZMkSHHvssQCANWvW\nYNy4cdo2kNBHjEc8UYOfDKKa3F+GfcQK8lpQE4HiwyisnOXaeA1OiJ7lpjL5O3OU5Xj170en4i57\nH/3F8kCeDUO8TRdOI6MjPtiOnJRl2XSRpETOcmNksGmwRr5zCKJSuC6Wb775ZixfvhyXXnopfvCD\nH+ALX/gCAODPf/4zDjzwQG0bSOgjf+IVTSQrTbkb/Pp5U1f1q4lAcWU5yieC+bP0Xm5YEZGfs+zn\naxM/d7WgLNupr6GAgcG+T1+a/Ngx2doQ4sWsbtW/mC9fhD2vzmLZ6eJDVgWWS8MgZZkgZAipPuD9\n99/HzjvvjC996Us5aRiMX/7ylwgGizdDEJWlYDQxdUKXJLfBz/+ipW9QRUymTSRSmZKKUDVQTFkW\n1bBEOmPbGFUr8ASZvKmX8VSmQPnTRX+8tiIL7TzLhmEgGgoilkz7krUcE3owGsIBJNIZ7d9lTlPw\nKpKGIetZVpngVwOfM4KoJMpn47322gu77rorrr76arzyyisFv29oaEA4HNaycYReCtIwuFJWH/5S\n3ZS70So317n6FcV4MWVZODHXw1SwmODzbxBem5/FRZ+oLNdAGoadEgpYKw3+eJatpBLeqKbbhuGg\nzrILhPJ6lp2fK50xkR6MVCmVhkET/AhCDuViecuWLbjhhhuwZcsWnHzyyRg3bhzOP/98LFy4EAMD\n1T9taiiTr1g0UIOfLZmMmbNfyqG89MXF5ffqf0/4SoWgKEaCARhs6b0OPlcxwYYRCgZ8W+4XqbVh\nOHYT/MTbdNsw0hmTF6hiBrZ2z3IlGvzsPMsKQ0lyhqmUUpZpiitBSKFcLDc0NOD444/HH//4R2za\ntAmPPfYYxowZg2uuuQajRo3CiSeeiAULFqCzs9OP7SU8II7uBcizXIp8D185ihbxhNVfE4piYQqC\nYRi8eK6HRAwxZxkozwVmf60py0n7gjLq02dB/M7K8ZNrjkCTtmFovBiwU5ZVhpI4TR5klHMqJUHU\nMp5MkYZh4MADD8T111+PVatWYcWKFTjkkENw1113YdKkSfjtb3+razsJDbACkBr8nMkvjsuhvPTV\nmFe1mLIMiE1+1f8anBB9sUB5Jp7VWs4yKyiL+dO5sqx5mV88HqOhgG82DKf4tagPDX62aRgKKr3o\noQ4HDdv7USISQcih3OBXip122glXXHEFrrjiCmzZsgWff/65zj9PeMQasEANfk7k75NyeIhrLQVh\ngOfr5g+jCAJI1oWyLNowxP/6WVyIqwq18DngSmi4lGdZ7/4Sv8sCAYO/L7qP00oMJSnWCwCIsYwS\nyrKgiBuGfbHcSJ5lgpDCdbG8ZcsWjBo1CgCwfv16/OEPf0AsFsMJJ5yAr3zlKxg1ahT/PVEdDAge\nP8BSBMmvVkh+kVKOvNscZbmW83XrSFkutGH4f8zkDMOphc9BspRn2R8bRrliMC3PcvHkEz/TMAom\nIipcePDGxBIWDPFvUnQcQZRG2YaxcuVKbL/99hg7dix22WUXrFixAvvuuy9+/etf4/bbb8dhhx2G\nxx9/3IdNJbwykCi+pFwPRY1umMrLljD7fc7WFZ8TqA1F0TbiSkEBq3b685Xlctgwak5ZlrBh6C6W\n894Xv1bJnNMwfJzgZ6Msq3iWww7xkxQdRxByKBfLV111FaZPn44lS5Zg5syZOO6443DMMcegq6sL\nW7duxYUXXojrr7/ej20lPGINkchXY2q/qNENO3mMao4CGOy+16geFaP2UhDsGpH8WXqvBAWe5XIo\ny7XmWZZIw9B9QS7mXwP+xWDKpmH44VnOv/hQWbFJOHitGWLTYCZT+0OECMIvlG0Yy5Ytw/PPP48v\nfelL2HPPPXH77bfj4osvRiCQPSgvu+wyHHDAAdo3lPBOgWe5DCf+WoUVKaNaIujozkYixhJpX4ds\n1FqRZN+IVD8XYbE8G0ZZPMu1loZRKmfZZxtGU0GxrHsoSfbv2Y67HhzApdeGUdyz3KBQmMsMJAGs\n/cb+rjh4hyAIC2Vl+fPPP0d7ezsAoKWlBc3NzRg5ciT//YgRI9DT06NvCwltFPj8qMHPFqbyDmsI\nIzQ4s9fvAlZccq+F6Di7k3q9eJbzs3yB8jTFip+DeCqDlM8rGl4pacNgqwyaPwsDeTaMJp8sZU42\njIgPaR92+5M10so8l5MizhAvdGv9eCUIP3EVHZffXVuq25aoHvLD7ht8WiKtB1gx1CRMB/O7WO6v\nsaEkdid1lTzYaiY/y1f8r59TL/M/Z/1VfnwWy9tm+OZZzrPH+OW9lU7DKIOy7CY6zsmGEQwYZRm0\nQxC1jqs0jHPPPRfRaNbLOTAwgIsuugjNzc0AgHg8rm/rCG2Intt8ZZmK5UJ4Y1ckiKZIED0DKd8V\n+L4aG3ftdFKv9c9VfpYvUC7PcmESy7CGsG/P54VUOsPHKhcrzMqWhuHzuGtbG4YvQ0kcLkIVlGWn\nBj8ge5GTSGdq/nglCD9RLpbPOeecHCX5G9/4RtH7ENWF+EVYzmalWoXtk+ZICM2REIC4rwVsOmPm\neHz7aigyrCANg/l6a7zBLz/LFxCtS/59FvKV5WpOxBAV1dLKsj85y/lecr+K5WKvDfAnDUOLsiwZ\nHQdk913PQIrOAwRRAuVi+a677vJhMwi/KaaSUYOfPawwbhRtGD7up/z3IJas3gKJYZuz7NPUtnIz\nkKdeAmXKWc67UKrmrGXxPS6qLIf9+Szkvze88VK3DcPBzhD1wYZh2VqKN87KNfgNNiZKKMvWFL/a\nPl4Jwk+Ui+XzzjvP8T6GYeCOO+5wtUGEP4hqBVPJxC/JTMbktxOWutc0aMMAypetC9SGsmybs1wv\nynKRYpkrmAk/PcvZz0IwYCCdMataWWaFWyhgIFQBG0ZDfuNluYeSBPUqyxmhqbTBJpJRxi6RTA1a\nYySK5XIkvBBEreNKWZ4yZQr22msv34c0EPoYyDu5AMiJCaLYoFxyG/yyh4mfDX75DX014Vm2Gcvb\n4JOaWG64DSMiFsv++rFN07RiC5sj6Ozx1/7jFba6YFeU+WXD6LexYeg+Rsuds5xra3GvLDsp4iIq\nRThBDFWUi+WLLroIDz74ID766COcd955+MY3vpETHUdUJ0wFzFlSFk4AA8k0FcsC1sk4hCauJvpX\ntOTn6daCshy3G57AT+rV/xpKkZ/lC/jXSMZIpDNIDTbMjWmNorMnXtWfBdkJd7qVZVsbhub3xeli\nIKLZs5zTW2IzRl5lgp+MshwlOx5BOKIcHXfbbbdh06ZNuPrqq/HEE09g0qRJOO200/DMM8+Q0lzF\n5C9bAkAoGKDYIBuK2TD8VJbz9381q4kMJ2W51j2QxTzLbJXBL0uO+HdHt2QTh6r5s1AqYxlQywZW\nIX/cdWPEn+8xxzQMzTYMtj+DRWwt1hh5+Qa/sGSDX/bv1vbxShB+4ipnORqN4swzz8SiRYuwatUq\n7L777rj44osxZcoU9Pb2Sv+dpUuX4vjjj8eECRNgGAYef/zxnN+bpol58+ZhwoQJaGxsxMyZM/Hu\nu+/m3Ccej+Oyyy7D6NGj0dzcjBNOOAEbNmxw87LqGquzv/gIVSqWc2ENduXKWWbKcrBMA1B0YKcs\nWznL1f8aSlHsAtPvBBlmx4mEAhjWmI2Lq2ZlmTd5OqRFaE/DyBt37VfOMrNF2L4+zZni3C5XbBqi\nT8pyA50DCMIRV8WyiGEYMAwDpmkik1H7wujr68Mee+yBW2+9tejvb7jhBtx000249dZbsWzZMrS3\nt+PII4/MmRB4+eWX47HHHsODDz6Il156Cb29vTjuuOOQTtOBL2IXR9To00mm1ukXlKsmn5fexecb\n1RzJ+blaMU2TK8v5hYSVs1zbShVr4ivW4OeXv5NZfZoiQTTzi7QqVpaTTjYMvxr8sn+vKc+G4Vt0\nnJOyrCkNwy4JA7D2cTpjOk51ZGkYdu+LSCNX/6v7O4cgKomrYjkej+OBBx7AkUceiS9+8YtYuXIl\nbr31Vqxbtw4tLS3Sf2f27Nn46U9/ilNOOaXgd6Zp4uabb8YPfvADnHLKKZg2bRruvvtu9Pf34/77\n7wcAdHV14Y477sCNN96II444AnvttRfuvfderFy5Es8++6ybl1a3FOvsBywPZq2rgLqxGvxCQoOf\n/55ltvTel0hVta0pkc6AbZ5tGkaNn3z5MZPjWfZXhWMqcnMkhKbBz101T3N0mnDnm2c5v8HP53HX\n5fYsF1OWc0ZTOzxfMi2fhlEvxytB+IlysXzxxRdj/Pjx+MUvfoHjjjsOGzZswMMPP4xjjjkGgYBn\noZqzZs0adHR0YNasWfy2aDSKQw89FC+//DIA4M0330Qymcy5z4QJEzBt2jR+n2LE43F0d3fn/Kt3\n7AZIlCMKqxbJn+An3ubn841pzRbLplndyqxY/NjmLNfJuOuiOcs+fRbEz11zdPBzF69iZdnRs8yS\nUfyxYbD3pimcvbBIpk0kNWYeOxXLum0mpZRl0TfttD+dvNYilLdPEM4op2H8/ve/x+TJkzF16lQs\nWbIES5YsKXq/Rx991NOGdXR0AADGjRuXc/u4cePw8ccf8/tEIhGMGDGi4D7s8cWYP38+fvzjH3va\nvlrDTlmmL8rilDtnmWXpjmqJ5NxWrQklrJA0jMITcrROlKpiPn+/Pcts9aI5EqwRZbn0cj8ronVO\nuAOK5SznJvvINLbJ4KSc+6UsF9ufgYCBSDA7mtrpQtRpu0VoKAlBOON53LXf5D+XaZqOz+90n2uv\nvRZz587lP3d3d2PSpEneNrTK4V/CNskFVCznInpH/cpwzX2+7N9ujYbQFAmiP5Guah85X6kIBQuO\ntQaffKrlprgNw98LgT7B/sOV5Wr2LDtExzUoNKWpkD/uOhIMIGAAGTP7vrU2hLU8j2waRsYEUulM\n0cEsKnAPeBFlOXt7tlh2+vy5ScOgcwBB2FO1467b29sBZNXj8ePH89s7Ozu52tze3o5EIoGtW7fm\nqMudnZ048MADbf92NBpFNBr1acurE1vPchlUwGQ6g46uAUwa2eTbc+jENE0+2rpRUPh8VZbjQq7z\nYLFc3ZPb7FMQonUy5KDUBD+23K9LweTPKVykcWW5qtMwnGwK/vRE5FtkDMNAYziIvkQaAxotZfG0\nc/HKSGgolnkcY4n92YOU48UHazhUScOo9eOVIPxE7ze9RqZOnYr29nYsWrSI35ZIJLBkyRJeCO+z\nzz4Ih8M599m0aRPeeeedksXyUKTYUBLAf6UMAH7yxCp85YYX8PKHm317Dp0MJK3mtabB4hWAr8Vr\n0eX3Ki6SBgRlOR8rD7bGleVEYbGc02TlwzHD3vOmaKgm0jASTp5l5unVnbNcNANbr0Jqmqa0sgzo\nsWI4KsuS/QAJh2EqIuU4BxBEraOsLOukt7cXH3zwAf95zZo1WLFiBUaOHInJkyfj8ssvx89//nPs\ntNNO2GmnnfDzn/8cTU1N+PrXvw4AaGtrw/nnn48rrrgCo0aNwsiRI3HllVdi+vTpOOKIIyr1sqqS\nYuOuxZ/9VE3fWr8VAPBBZy8O3HG0b8+jC7E4aQxbOcv+epatIqmpBoqkUsqytfRe2yffYuOuo6EA\nDCPbgKlzuZ8/J5saGA6iKVr9F02OnmUfbBimaQo5y9bz6u6/YIkSgH3RGRLsHzqKZSdlWVYFZttu\nF3mX8zfr5OKWIPykosXyG2+8gcMOO4z/zHzEc+bMwV133YWrrroKsVgMF198MbZu3Yr9998ff//7\n39Ha2sof8+tf/xqhUAinnXYaYrEYDj/8cNx1110IBquzMapSDBTxXwLlafDbsDUGAOgZqN7iT4R5\nk6OhAIIBozxpGHFLWW6ugSKplLLs19S2clNMvWTL/f2al/sZLEKwKVpjOcu2Q0kGG/zSGWQyJgIB\n7/0u8ZS18lPMIqNrf4nZyaXyiiOhAAaSzk13Mjgry3L9AEpDScogBhBErVPRYnnmzJkls2QNw8C8\nefMwb9482/s0NDTglltuwS233OLDFtYPMZsua7+bO3oGktjWnwQA9FZxBJYIV/cGTyLlUHr7iyjL\nbIpgNWI35AawVLFEOoN0xuRTCWuNUj7//kTal2MmN4WlFtIw5GwYQPbz0BDwLmKIRV1DERuGLjuB\nqBSXimCLBPUVy46eZckoPqVimQ0RqvGVIILwk6r1LBN6ye8eZzT6rAIyVRkAemtMWWbFChtK4u8E\nv0FFMRxEcw14lksVSWIBozsyrJxYqzH5CTL+XWDyz4GYhlHFF5lxB2+sWCzr+o5h+z0cNHIaLC1L\nma5petnnCQWMkop4RGM8npNSzy0TTtFxaYU0DFKWCcIRKpaHCOzLNX/Z3O8vSrFY7qvik74IK1jY\nvmnKS0DwA8uzHKwJz7JdFCGQWyDVctNQsZxlwN9jhn0OxEbP/mQamUx1TnNMOETHhYIBvrKgy8Pu\nlOyj6yLG6bUxosJKilcsZdnbkBclZTksZ+0giKEMFctDhPzxsAz2Re+Xarphaz///54aKZZjwlI4\nkC1gGX75li3Pcog/XzUrywM2EyGBbIEU4gVS7Z6AKxG3GCuSs2ya1btE7mTDyP5Ob5Of0yqZ7mLZ\nqeDkxXIZlGXZfcku6qUm+IVIWSYIJ6hYHiJwxSLvS1h33FI+tW3DsAYeMHXMrxMKVxSjlg2jmpVl\n58lttZ/datcU66fPX2zwyw58YbdX5350Gkoi/k6XslxsDDlgHa8Dmo5R2Sl4Oqf4xR2U5QbJCzUV\nZZnZjKr1gowgqgEqlquA9Z/343/+ugobt8Wc7+wS2yVln4eSrP/cUpZrpsEvz7NsGAa3YvhVwIrP\nyZffq1jpKaUsi7fX8gm4WM4yYKUH+PH+iM2lgYD/nzuvMDtAqaIsqjmaLH/UNaNB84W/7GAPXiyn\nvT+v03Eln7MsZyHJ3oeUZYJwgorlKuBPr6zFHS+twQOvr/PtOSxlubzFco6yXCPFcr5nWfx/Pwqk\nRCrDT8zNOWOOq/fk5aQsN/jcOOo3YpZvoQ3DP+sSV5YHL5iqPWuZfW5LKsuac7fLbsNwsDKw3+v4\nrJfKLwcUouMUJvix/RhPZarWG08QlYaK5SpgS18i579+wDrE7VSysniWa8WGIQyGYDT5uJ9ERacx\nYg1BqeaGSFkFrFZtGIl0BqxuaLApynQt94vk++WrPWvZKRcY0D/Fz7HBT7sNo3TcXURng1+J/HJA\nGPgjacOQScMQj+Fa7jEgCD+hYrkKYEWkn8Vk3OYE42dzR1csiW7hNfXGk9qfww/yCxbAio/zQ+1l\nY7TDQQORUEDwLFdvoVkqZxmwiieniKtqRRw4Yjci3hdlOc8CVO1Zy04rDNnf6U1bsPMs+5WzLJuG\noWUoiS5l2UXOMlC7F7cE4TdULFcBPQPJnP/6gZ3PzzrB6C9qmKrMlikHkhmkfIpe00k/X+a1ZvZw\nZdkHhU/M1hWfq69K1UTAOQVBVgGrVuyyfAGgMexf7jb7LDArTrVnLVeiwY8dn/mKv+78a9mCU2eD\nH/sedhryUmpfmqZp2TAklOVQMIBwMNtJWss9BgThJ1QsVwF+K8vJdAapwTXlcsZgMb/yjmNb+G3V\n6r0UyU/DEP/fD7WX/U225M7GXVdzw03cSVnmU8H8uTh6+p1NuPCeN9AV8+cC0+7iErDSA3S/P4lU\nBsl09jhtCteGsixTUFqe5fLYMHQdo6xhz0lZ1jqUxEFZttIw7J+LfYay2yZ3im/QbGEhiHqDiuUq\nwCqW/Tnxi4Vw/pewnzFYrFjeYXQzP+H01IAVI8aVXsGGoflELMIuIFgzVy0oy3YNowyrwc+fk+8f\nXlyDZ979FC+u/syXv2+XhCHepvsCM9+7DgjKcpV+FuRylvU2ew7YvDc8Aq1CDX7l8CzLNEuK2yGT\nhgHIFeEEMZShYrkKsGwYPsWSDZ48DKPwy7NB6Ow3Tb2d0MyGMXFEI1obsoVgLSRi9BfptrdsGP4t\nvfMhKMyzXMUqPG/scspZ9klZZoqy38pyfuIC4N+46/5krncdEJTlKv0syHmWNadhOOVf61KWFW0Y\n5UnDcH4uUeGWafAD/BVNCKIeoGK5wpimyZvgun068ccFtcJgUw4GYb4/09TfCc2U5YkjGtEyqJrW\nwmASMeuW4W+DX97EQFKWHWHHil8XmHZNZIB/4675CoPgla/6NIwUS0WRKZb9tWHovoiRHUpijbvW\nmLPsNJSklLI8uN3BgMGHKTlR6z0GBOE3VCxXmFgyjfSgn7gvYf2/7ucAiqtk4glHdyYuG0gycWQT\nWgaV5VoYec2V5XBhgx9T/7Q+nzDqGrA8ywPJjC+fBx0MOCjLDZoTEPLp9rkp1m6ID+CfClcshaXa\nc5bZd0YkKGHD0DbuejAG0+ecZZnmRfH3esZd61OWZZr7GKQsE0RpqFiuMPnKmB/KK4/5KvKlHw4G\nEGKjnDV+UZqmiU8GleVJNaYsl2rw88eGUdyznP1dde4va7m4tLfSj8bRRCrDi3W/rUvl9Cz3FfHK\nV7uyzIeSlFKWNauWjtFxFbJhaEnD4Eq9XRqG84WHykAS/nfJs0wQJaFiucLkK2PdPihlMZuoJYYf\nHszuWIqryNsNb7KK5ZpQlos0+PmahsGU5exzREMBvnxard3p0svFPhTL4jHjW7FsMyUO8G+QjxUb\nJ65oVG8aRiptrXzIeZb9tWHoHhxkxa85DCXR1OBnmqZjtrPMRahKxjKDlGWCKA0VyxWmO+9k78fJ\nn6kVxVQywJ/YoPWDzX2jWyJojAR5sVzNU+kYRRv8wv4pfH15z2cYBn++aiySAImIK80Fkoh4zPhm\nw5BQlnVfyPQXSXmo5pxl8b2VSsPQZsOozpxlr69PfLwWZVnBhtHg40oQQdQDVCxXmPzi2I+Tfyn/\nJSBELmkMpLeSMJoAwPIs14ANw/KOFip8vijLeZ5lAGgaLJKq9eLCOeKqPMpy/sWmLkrmLPu0ZM3S\nT4ory9X3ORALtpI5y36lYdhmxmeQ0eD1L3uxnBQvPhw8yxINfirKsp8rQQRRD1CxXGEKbRj6T4rs\ni9VOWeYnGY2FoJiEAQAt0TCA6rdhmKZZdhsGT8OIil7V6h557TjumkXH+eCB7I6JyrLfNozC1+fX\nuGv2uRNXNKyc5er7HLCiLOSQumB5lssz7hrQPHq6TJ5lJlYEA4VTIxkyecheGvyoWCaI4lCxXGHE\nEz/gt7Jc/O32w69mFctZZZnnLFe5shxPZcBEqXLlLLO/WVRZrkJFERCSAmwb/Nhysf791Z3jWfZ3\nkE85bRh9/HMgfu6q174kW0xqt2Eki1/IiKscOr7LnPzDjKimCX5O2eXi70oPJcn+zo2yTJ5lgigO\nFcsVJv9k74dSVmpJGbAKG73FsjWQBEDNNPiJBVBTuJiy7IdnuVDJZkVSNTb4ZTJWI1KxhBXxdj+U\n5bI0+JUolsXCQucgn2L2n2peYXC6YGLotmH029jKAgGDP5eO41Q2VUK3smz3PQ3kjg63++wlUmbO\ndslAE/wIojRULFeYcniWub/UyYah8Yuy0IZRGznL/YNFUiQYQEhYxvSzeO0vMoyCDyapwv2V09jl\nNJTED2VZWI3pjae0T54ESifI6F7uZxS9aKpi77qMEgrkFng6sBt3DQjxcRqV5XKNu5ZTlq0hUnbP\n56XBj5RlgigOFcsVppzKspNnWdcXpWmafCDJpJFZG0Yzz1n2Z9lcF7EivlFAyLv14WTCi6Qa8SyL\nBbCdsuyrZ1n4DKUzpi/7qKSyLLxmnRdPxRr8xM+BHxcFXmCfAycFkw+o0fRZkBmyxAaXeEF5gl8Z\nlGXRSmd38eElOo48ywRRHCqWKwwrjpny6kuDn0Mzlu4w/239Se6/3G54VlnmnuUqVMhEig0kAfzO\nWS7iWa7ikdesAA4FjBz1XcTfnOUyxC2WKJZDwQBX7XQqcf1FnpNdQKUypmflUjfSnt6wPhtGMp1B\narCpoEmYsMnQeeGvnobh7TlllGVRLba7+GDbbdckWAxrPH11fcYIolqgYrnCsOKYFZW+DCVxUJZ1\nL8ExC8aY1ij/Em6p8rG9DLtimVkkEin9I6iLpW8wdbG/CveXTGMXe991TDXLpzuWvxrj4zFjO8iH\neWN1KstsKElhvnf299X1WbDGQTt5lvU1+InfUQ1Fkkp0NqrFZW0YupRlPurafn8ahiGs2hR/jQnJ\nxksRGkpCEKWhYrnCsOJ4u0Fvr58qmZO/VJcKmN/cB4g5y9Vuwyj0D2d/FooWzWpv8Xzd6o0Mc/LA\nA3A8oXsh/4LSj9UY52xy/cq5NQzH+hyEggG+L6ttlUE+DUOfZ5mtfgWM4kVso8bUmoRiA6PnNAxJ\npd7qByj+fMm0eoOfn+PpCaIeoGK5wrDieMLwhsGf/VDJSk/w060qMGV50mBsHAC0CmkY1ea9FCk2\nvQ/InsCMwShZnT5V0zRt0jD8S9/wipWxXMpbOVhM+jzBD6iPYwYoHHvO4KsMVXbhJO/pZUv83rdf\nXCUzjMJs5yadDX6SjXJcWfZok5E5rgDndBE3DX6kLBNEaahYrjDsRD9huP/Kst2Ssu7mjvUllOWM\nWd1fyMUsEUDuCGqdRYuY61wsOq4ax13LKGANfBCFH2kY/jfFsmMh/3PA8GNEvN2FWrUmo8gqoTrT\nMJztMRXwLAf1jruW3Z92zbOyFzEiFB1HEKWhYrnC9OR5lv2Jjivd4Kf7xJ8/kATIFuRsyFc1DyaJ\nlSiSGn1IqBALoJx8XTa5rcoKJEDOW8nURD+UZXbMjG2N5vysE1kbhl5lubDRU/y5WpVlZ8+yxmLZ\n6X3R+F0m28Co27PspCzzdBE7ZdlNGoYPtiKCqCeoWK4gpmnydIiyKMt2nmXNJ/5inmXDMGoia5mr\ne0U67fkUv6S+7bcGLARyRgZbynL17SteLEsoy+mMiZTmFIdCn3/5G/x0Lvcz2HstNvgB1Zu1HOcX\nTbIT/PTaMIqhNQ1DcShJqUEhMgzIesAdlHpXaRghKpYJohRULFeQ/kSaJyuwYrk/kUZSc3ERc1AC\ndQ4lMU2zYCAJg0/xq2Jl2S4NQ7xNawKCnZoY9Weksg7YSdpupQLIVRt1qsuZTHkuMKWLMl9sGLWm\nLMs1+CXTpuckGUdLWQWGkkSD1raw5jo3xJPOxxXg7AF3M+6ajQ6vZoscQVQSKpYrCDvJhwIGX1IG\n9BeTsUT51JjP+xL8pL5dfrHcwOLjqrdYjtl4lgF/spb7bIagVLNnWWa5WCygdKpVPfEUmHg30ae4\nxbQwzts+blHvakwqneHPmd/gV62Z29INfkLx59WqYDfqmtGgsa+Ap304Fa/i6/MgdFjKsoMNw0FZ\nTg6Ou1aJjouSskwQJaFiuYKwk3xrQwjhYICfmHUrZY7jriP6YoOYqjxuWLTgS7+mbBgllGVfprbZ\nRNVVo2dZRlEMBIyc5WldMMtFNBTAyObI4G26jxfr/XVqitVVLPeXeM5qzdxOSHqWcwZpeLRilOvC\nP5MxrQg2yXHXgLeLAXVlWee4a2t1sZrTigiiUlR9sbz99tvDMIyCf5dccgkA4Nxzzy343QEHHFDh\nrZajhxfL4cH/sil+epUyR89ySF8RWKy5j9Ey+Dqr2YYRK2HDYD5mP5Tlpnyfao0ry4A/Wcvdsez+\nGtYY5seNbs+yWGjZXRDonnrJCuFQwCgocqpXWZbz2IaCAYQG/fherV5O32X8wt/j+yIqxE7KeSBg\n8NfnqVhm464lGyYHfGjwy24HJWIQRD6FXUxVxrJly5BOW18K77zzDo488kj8x3/8B7/t6KOPxp13\n3sl/jkQiZd1Gt7C8WFYktzaE0NkT900ps03DYCd+DQ04xZr7GGLWcrVi5xsF/Mk+jjl4lqsxZ5l7\nlh2Xi4PoGUjpLZYHC+NhDSHh4tI/21KxLF/2O0CjsizYcfKfs9pzlp2U5ex9Akgl0t6VZYdIP13v\ni0qxzO6TSqS1KMtOtg+n0dSuouOE+8YSaccLYYIYalR9sTxmzJicn6+//nrsuOOOOPTQQ/lt0WgU\n7e3t5d40z/QUFMv6lTLTNBWalbwrCsUGkjBaaqBY5kpvkX3lhw2j2ECS7M/ZfZVMZ/2zKic+v5FN\nQXDyVrrBOmbC/LjRfXHplIQB6Pcs2zV6AtbxWW1ef1asyXw2o+Eg+hJpz58F9h3VYPPe6HpfxKJX\nxs4QCQXQ7/FigIkVuoaSqKRhhIIBhIMGkmlTi2hCEPVG9ZyBJUgkErj33ntx3nnn5agvixcvxtix\nY7HzzjvjggsuQGdnZ8m/E4/H0d3dnfOvEuTbMIY1Zv+rUylLpk0+9MI5DcP7l2SxgSSMFp+KG52U\ntGHw5XD9nuXCYtn6udoSMQZSpT3wDD+ahthAEl9tGA6+WEAcq6znQoAVwvl2HEBcZaiuzwErymQa\nyXiB59GG4Xjhr+mCVkzCsFtdENExmIQry5LpInaWlkRKPQ0D0GvHI4h6o6aK5ccffxzbtm3Dueee\ny2+bPXs27rvvPjz//PO48cYbsWzZMnz1q19FPB63/Tvz589HW1sb/zdp0qQybH0hrGgcludZ1nny\nFxWWcnT2l/IsN3Nl2f3r++OLH+GhZetcP94JuQY/fcW+5VnOVRTDwQA/AVebV1UmZxnwR1kWbRjD\nfFaWSzVa6Z562V/CXsD961WnLMutMADOaqgsjp5lTe+LbCweg+0DHWkYjkNJmA3DZl/KNiYW/F0e\nu0eeZYLIp+ptGCJ33HEHZs+ejQkTJvDbTj/9dP7/06ZNw4wZMzBlyhQ8+eSTOOWUU4r+nWuvvRZz\n587lP3d3d1ekYGYqGSuS/Tj5sxNaMGAgHCztv0xnTCTTGaXlO5FsxrKEZ9nl69vcG8dPn/wXQgED\nX9t7IkIut7MUlieymGdZv3fUWn4vUiRFg0j0Z6rOtxx3SFdhOOXBuiHXhjHYMBpPwTRNKQVQBhkb\nhnbPctz+c1cOZTmZziCdMZW8qmqeZVbgebVhyOUs67JhyKqzrDDV4lmWVJadhpKoRMcB1sUh2TAI\nopCaKZY//vhjPPvss3j00UdL3m/8+PGYMmUKVq9ebXufaDSKaDRq+/tyYSnL/nmWuUoWsl9ObIgE\ncu7vtlje0pfAQDIDwwDGD28o+D2zYfS6jMDa0psAAKQyJrbFkhjdov897C+Vs8wyXLWOOGbPV6RI\nioSwrT+JviqLDJOdNMZPvhqVKsuGYTX4pTMm+hNpvnLhlQEJG0aDZv96qc9dOaY5fu13L+PT7gEs\n+d5h0gWzbBoGIE6d09Pg5zjuutzF8uDFgJdimR9XThehDuq5mzQMQFDlyYZBEAXUjA3jzjvvxNix\nY3HssceWvN+WLVuwfv16jB8/vkxb5p6C6LiofmVZRiWLBANgk5a9fFGu/zyrKrcPayiqNrV4tGFs\n608U/X+dVCxnuYhX1Y+JgTrgNgyn5WKNY44Zlg0jjKZIkI8I9+OYKVU06k/DsG/w4xP8fLpoiqfS\n+OeGLnzaHccn22IKj1No8Cu7Z9nb86hOwWP3qwpl2UWDHyBkLZOyTBAF1ESxnMlkcOedd2LOnDkI\nhayTSW9vL6688kq88sorWLt2LRYvXozjjz8eo0ePxsknn1zBLZajMA1Df7E8kHReKjUMQ4tv2W7M\nNcNSlt29vm0xq8je2q+3qYtRqsGviUd46fcsF42q8+H5RLpiSZyz4HU88uYGpcdZ0XFyk810Ksvi\naoxhGNagGx9WY0o2+On2LJe6SIv6m7PcJRxLKhehKsv9umwY1rhrm/xrzZ5lWd9vVEODn3QahkN0\nnFtlmZ8DNDWtEkQ9URPF8rPPPot169bhvPPOy7k9GAxi5cqVOPHEE7Hzzjtjzpw52HnnnfHKK6+g\ntbW1Qlsrj+i/BMQ0DI0nfgePH0OHUlaquQ/w7lkWT+pb+/Qry4lUBqnB6JCmcJHiNeyDslzKs8wi\nw3xSlp/716dY+v5nuPPlNUqPkx1K0uBHGsaAlYYBwJesZZljhhVrunOWi30Omn3wyouIF6HbFC5C\nVXOWs4/x9hr6HSwy3CqVSHmaRKduw2ANfu5fn3Iahs2+VC30GQ2aLwAJop6oCc/yrFmzin7xNTY2\n4plnnqnAFulBHHed/a/+6DhLrZALuveiApZq7gO8K8tbc2wY+pVlsQguZcPQOsEvXsKzzBq7fEpB\n+PCzXgDA1j61fSmbFMAVMJ1pGGyCH0+QCQOIaVWWnRIXAFGF0/NZYL70UsNw/ErDEC88VVZsuGdZ\nJg1DUzIK2992F2rMS54xs3YEmUK+GMppGBpsGNIXoQ7KcjLt1rOs9wKQIOqJmlCW6xW+pNzoX3Sc\nTLMSoCeftNRAEgDCkrkOG4Z+Zbk/md2ucNAoeqJp9KFY5spyUc+yvyOvP+zsA6C+Lwek0zD0j7vu\nGfA/QaYSnuVYsoSyPHjcxFMZpDxEk9mRqyzLfxb4UBIJBdNKRvF53LVw+4AHO4FrZdnTuGtVz3Lp\nBj/1NAxSlgnCDiqWK0ihsuyDZ1nSB6fD61dqIAlgFcvxVIarHyqIarIfnmWnJV5WvOpUXvq5R9pe\nWdaZ6yzClGXVyWM8X9cxDcMHZbngApMlyOi0YWS3t7QNwx9lOT9vG8j1z+tMYmFsc7liwxrJnFat\nAH02jFLRjkC2qS002PTp5Ti1imU5ZTri0HTnhGmaVi+A9EVo6QY/12kYVCwTRAFULFeITMbkdgRL\nJfMhOi4h9wXc4HEJzjRNfOLgWRajvdwsKfudhhErUbhmb7f8kLqQiwzTf/JKpTNYu6WP/9zlwqta\nbmXZNE0rOo75/H0c5CPT4JcazCb3Sn+JxtJoKMBTP/xIxBAL5G0xdWVZzbOsKQ3DpsEP0KP684JT\n0vfLc5ZdfhbE/SJ/EVr4+tIZE+nBvgvXaRg0lIQgCqBiuUL0JVJgNuz8CX4DyYyn5TwRVR+c28Lm\ns9444qkMAgbQ3laYsQxkv7xZUe5GCcxVln2wYZQoWABLTRxIZpDJuG8eEmGKYrHIMF6c++BVXb81\nxid9AWpKvfpnStdn2WrA9DdBpvTnAMh97TpWGkpdNBmGYfmWfVhl2OpixSarhKrkLOsdSlLqs6cj\nA1vVyuDVhiHaU2QvQovtS/H5VZXlKHmWCcIWKpYrBDu5h4MG//JrEZRXXUqZpZKVfqu9qjHMr9w+\nrKHkl3RL1Jq6porf0XH9PMbNzoaht0DKZExrWbnMnuUPO3tzfla5+FAfd61n+5ltKRiwikdfBvlI\nFGTRUACGhmxyRik7DuBv1nKXoCbLrjCkMibY9aJazrL77c9kLKuCjOqvx4ahViy7vRhgx0jAALeR\n2BEN2V+E5hTLisoy2TAIwh4qliuEGBvHJuuFggHe4KNLKYtLLCkD3j2YbCDJxJHFLRiMVg+JGF1l\ns2HYdNoLy806mvzEk3nRYRR8zLF+NZH5lRlKjV3SNgy9yrJlwQjxY8YPZVnGhmEYhhUlqFFZLtbg\nB/ibtSymocheNOXaBspjwxCj0mRGkXsp+lRU8+z9ss/pVlkWm2adxraXuggVbSDhoNr4dx1Z+wRR\nr1CxXCF6BqwTv4juhiWZzn7Aeyau00ASRouHrGU3y8UqWIMhiqt7gYBhqVYaimVW+BhG8SapJh/z\ndT8oUJYlFcW0ZYVwjiPUrSzn5pKL/681Z1li6qX4ex3FhRUdV/w5raxl/cWy6FOWbfBTXe7X0ewp\nHnMNJQp0Hc2X5U7D0GVpEZv7nIrufNh3m9fEEoKoR6hYrhD5A0kYuuPjZItl0Y/rBqeBJAweH6eo\nLA8k0zlFybb+hKehA8VgSQNNJfYV9xEnvRct/YJfudiJrZl7ln2wYQwqy+w5ZIskFUXRKQ9WFWsg\niXUx40vcouRqjM6sZfbZbi6ShgGIWcs+N/hJK8vZ7QgHDd58WAodaRgxwf4TKPGcOmwYccUGP685\ny7JxjPnPld87wb3WihaM7HOTZ5kg7KBiuULkx8YxdE8kk/0S9roE5zSQhNHsUlnujuUWQ8m0qd3L\nGyvRZMXQmbXc5+CRbvSpqcs0TXz4WTYJY6/JIwDIF0niyoPXSWOq5CdhAD7ZMCQ8y4DerGVrOI2N\nsuzj6HOxWO5LpKUKPpUkDPF+Xi6cBsqo+Csryx7TMAYUlGXxc5n/fGy7w4rNfeLfJc8yQRRCxXKF\nsJaU7WwYtdXg94nDQBKG5VlWe32suW9EU5ifUHSPvLZsGM7Ksg41sdSoa0AskPSevLb0JdAVS8Iw\ngL0nDwcg71UdEEbpllL3AP0n354ixww/XhQ/T6WQ+RyIv/f6+tJC45pTbKEfynL+ey8THxdXLCZ1\nTPBzykFn6LBKld2G4UJZFh/HSKTUFHER8iwThD1ULFcIaxKZnQ1Db4OfbM6ym87+TMZU9ywrnvRZ\nYTy8KYIRTZHsbZqb/Jwa/ADLz6yjgHVKQPBrzDFLwpg4ohHjBmP+ZD3LfCCJxCCKBk0jjhnchiEc\nM35M8FO3YXh7faJabKss++RZHkimrcJ3sMCSScRQjVbTYsOQLJZ1FH3u0zDcPadswgyQTctg16n5\nqzZuB5IAYmMkeZYJIh8qlitEMZUMsCaTdetWln1cuvysN45EOoNgwMB4m4xlRkuDOxsGU5aHN4Ux\nvCm7j3Q3+Vm2iOLFK2D5mXUULSw/udioa8AqkHRNiWMwC8aOY1r4hYe8DUNFAdOrLHfHcqf3AbkN\nsbo87DJpGOLvvSpx7P0NGPbFkpWGofezwC44QwED44fLXzi5TYvw1OAn3X8x6L31sK+s0dOyNhOv\n0XGDzydxXBmGYdsPoFrki1h/k5RlgsiHiuUKUS5leUDSW+hlyZz5lduHNSDksPxnKctqhS5Tu4Y3\nhpULPFmchpKIv9PjWXZQloW4MJ3NjKy5b8cxLcoXHmreSr3Kck8Rnz/7/7SQWe0F07T+TkOJKXGA\nvmK5j9txijd6st8B+gfUML/y8Ca148p1ManBs1zq+AT0RMeV24ahoiyL98tXslUnD4ro9OATRL1h\nL6ERvsKK4fzoON0jr2Oy/ksPX5SyFgzAfc4yU8CGN0X4CUm3Z1nOhqGvWC41tQ2wCqSMmS1OZNRc\nGXKK5UZWIMnaMCqoLPNjxrrAbIoEEQwYSGdM9AykbC88ZImnMnyypmw2udehJMxmU9Ir77Oy3NZo\nrdjIfBZYkabuWfaehuHndxmDFZ2yqRK6xl3LHuPZYytZcCHqTVnWO56eUCedTiOZ1B+LShQSDocR\nDMqfU6lYrhDFOvsBH5TlwZNTg8OXp3WCUf+yZ8XyJIeBJIAQHefBhsGLZc02DJkGIqvBz/v7w5q1\n7Ao8cTv64ikfiuVmjGhmBVI2is8pm5V/niQ8y1F+8s1I/W0n+DEj2DAMw0BLNISuWBLdsSTGDStt\nA3JCLBT8TpBhOMXGAf55ltmKzQihF0CqwS+p6lnWkbMsmexTgznLqsqyXWFLDX61iWma6OjowLZt\n2yq9KUOK4cOHo729XercRMVyhbDzLGsvliWVZW7DcHGC4dP7JJRlHh2nqCzz5eLGCJKD6o1uG0bM\nwRYh/k7LBL9Eac8yG4ISS6bRn0hjlOdnzJ5c2cXNjmNb+MVLKmOiN54qsAXloxIZlh9xJbtkb0cx\nGwb7uSuW1BK3yAqFcNBA2KHg0GbDcIiNE3+nOw1jq2DDaGuUt+Rw5VXiognQM8GvnF7y8g8lcaMs\nF+7PRFpN8RexrHh6Lm4JeVihPHbsWDQ1NdG+9xnTNNHf34/Ozk4AwPjx4x0fQ8VyhbAdShL1KzpO\nrinGTSau7EASAGh1mbPcFWM2jDAvlrUry0n5oqUcnmUgW0jHkmltWcsffdYH08wuu49qjgw2CwUw\nkMxgW3/SuVhWUZaFE/ZA0nuxXMyGAbBjKKblmJHNWAb0NJKJjy/1ufMrZ3lbrDBlRsqG4Tpn2f2+\nkk0p0TLu2uVQErcXA8qeZRtbSzKV9RC5SsMQPn86bV9EadLpNC+UR43SIYkQMjQ2ZsW9zs5OjB07\n1tGSQQ1+FcJOJbPSMCo0lMTFiV92IAlgpWGoxqFt7StsRNIdHVf+nOVBZbnk8+nNWhYtGEy9UNmf\n1knd+UQaCQbABBIdHfaWDcO/1RjZi0vxPl49njIXTX4py9uExlnLsyzT4OeuuBvwZMMo31ASlYhE\nwDoeyqUsN4QsFTjn73ho8GvIubglK0a5YB7lpiZnsYnQC9vnMj5xKpYrhLMNw7tKltPZ79PSZSZj\n4pNtLjzLqjYM7lmOCD5bvcqyUs6ylqlt8sW5rpHXrFj+wtgWfttwXizLNHaxk7rzV4dhGPykriMR\no8dGWdaZtSw7JQ7Q6FmWmBzpm7I8WBiPaI4oNvgpDiUZvF86YyLlsglO+bvMi2dZsejkNgy3E/w0\nKcteGvxCwQBCgwHO5FsuP2S9KD8q+5yK5QqQyZjoTdjYMDSOuxYLFKfixm103Kc9A0imTYQCBsa1\nRh3v3yKkYajEoXWxNIzGsFDcVS46TkeDH1eWSzR2cUVRU5EkZiwzhjfKK4rs89Egu/yuqcM+kcrw\nE3hxG4aeC0zWRCalLGtaZVBSlrWnYWT3WZvicWVFx6kpr+JjVZG3lLlvVma4HnddNs9y8Sg+Pu7a\nhbIM0GASgrCDiuUK0JtI8Xgqu3HXiVTGU8wSoNbZz74kk2k15Yf5lccPd85YBixPtmmqWQu2Fs2D\n9UtZdi5a9E7wK5+iyKb3icWyilLPc7sll6d1KctiIdziY1OsrHoJ6Gvw65do8PNrQE1uGkb2c9AV\nk1eWZT3LYtHpuljmNgy5xstaylm2Jq3KKsvFX6MXZVn8u7o/ZwRR61CxXAGY9zISChSclFsEldHr\nyZ8VNqGARGe/cKJW8RVyv/JwOb9VQziA4OBSn2wixkAyzQuS4cJJvTeecn1yyieVzvAl1JI2jLC+\nYlkcRmGHTq9qJmPio82DxXJRG4aKV7W8yjI7FlqiIf75Yei0LlXCs8wvmmxSUcTf6R5QY+WXh3nm\nttrnQO4UEgwYCAeNnMeqojru2ssFJk/7UC2W0xlX74+uiYEsDUN2u/Px0uhNEPUMFcsVwG4gCZA9\nqbjNIs5H5cQvfrmqqAobPpcfSAJYubiA/OtjFxcBI5umMawhDFYvyWTCyiB6kEt7iPUpfFxRLFEk\n6VQUN3bFMJDMIBw0MEl4v0YoeFVVxl0D9o1IqrDx78WOGXHktVdkoxYBK8/X64WTzEUT+51p6l0i\nzxkjP7jCMJDMOF4AsOV/FQXTSsTw2bOsM2dZcmiBV+V8QFVZtlmxSabdp2EAwvFKyjKhkcWLF8Mw\nDIwYMQIDAwM5v3v99ddhGEaOf5jdn/1rbGzE7rvvjttvv73o3z/33HNxzTXX+PoaqFiuAHaxcQxW\nEHRLLIeWgjeNSBQ2LEJMfJwMKgNJGC2KWcvshN7WGEYgYCAQMHgmrC4rBjuxBgNGyaYePsEvqcOz\nLGH7EBRFrzC/8vajmnMsM+7SMNSGJ3i1FHXH7I8ZnT5/2SlxgD4bhkyDX86AGk2WHNM0uU99eFME\nrYJq73RcqSqh2ft6i1dj+9lpSqMO3y0rllVzpAF3TX6q+9OPoSSAMJWSlGXCB1pbW/HYY4/l3LZg\nwQJMnjy56P3fe+89bNq0CatWrcKFF16Ib3/723juuedy7pPJZPDkk0/ixBNP9G27ASqWK4JdbBxD\nl1Jmnfjl3mY3J//1CrFxjBbFrGU21prZBQChwNM08poXruFgyQ5ZndFxfWWOjivmVwbcpmGo5et6\nVUPZMZMfGwfoHRHvyoZRhga/QMDQnozSn0hzJXJEUxiGYVjNng4rNqrFJCAWy+62f0Dy+4y9L4l0\nxlXyRjpjIpUZVGgVx10D7nzLupRl1ZSSfJiyzBpdicpgmib6E6mK/JO1ET3xxBMYPnw4MpnsZ2XF\nihUwDAPf+973+H0uvPBCnHnmmfznOXPmYMGCBfznWCyGBx98EHPmzCn6HGPHjkV7ezumTp2K73zn\nO9h+++2xfPnynPv84x//QCAQwP77749EIoFLL70U48ePR0NDA7bffnvMnz9fer+XgoaSVAC72DiG\nLg+manJBYziIrUi6UpZlBpIwxEQMGcSlYgb7f12DSZi/0UlR9KXBTyYNQzFqrxg8Y3lsc87tbtIw\nlPN1Paqvlg3DXlnW0uAn6YsF9OT5is9ZSlnO/j6E/oS+ATVsJSESDPDX29YUxpa+BM81t0PVswxY\nK1xeG/xkbRhAtv+iRVFlFYtd2aLTMLIrUol0xlWx7N6zXFxZdpuGwaxFlLNcWWLJNHb70TMVee5V\nPznKcfUGAA455BD09PTgrbfewj777IMlS5Zg9OjRWLJkCb/P4sWL8V//9V/857PPPhu//OUvsW7d\nOkyePBmPPPIItt9+e+y9994ln8s0TTzzzDNYv3499t9//5zfLVy4EMcffzwCgQD+93//FwsXLsT/\n+3//D5MnT8b69euxfv16xT1QHFKWK0BPiRM/oO/kr5IZC1hflLKqaTpjYuM2Nc8yoG7D6BIGJzCs\nRAw9yrJswSIWSF4arZLCSbWUstysUVn+wEZZVknDUB6e4LFAYlg2jBKe5bj3CyeVY0bbuGsJGwZg\njUXXlYyyTUiYyR9Q0+WgLHuyYXj0LDtdyLjtv2C4KZbF+3pRlqVTZmysJjwf2rWyTA1+hBxtbW3Y\nc889sXjxYgBWYfz222+jp6cHHR0deP/99zFz5kz+mLFjx2L27Nm46667AGQtGOedd57tc0ycOBEt\nLS2IRCI49thjcd111+GQQw7Juc/ChQu5BWPdunXYaaedcPDBB2PKlCk4+OCDc5RtL5CyXAG6HZVl\nNsXP28mfLaXJFjaqJ/8tfXGkMiYCBjBWImOZwZVlyde3tb/QhqFiHZDBmt5X+pBoymu0kr0QsXs+\n8W8WfT6NBVKxjGVALQ1DfblYVxoGs2EUXmDqHErSL6leAoK/M5lBJmMiEHA3VIDZKpzUHPZ7XVP8\nxGKZwS5InY4rN8v9+mwYpd8bwzDQGM6OiXfzuYsPJkoYBviQDhkioQAQL49n2TYNY3Dfui2WdWWH\nE95oDAex6idHVey5ZZk5cyYWL16MuXPn4sUXX8RPf/pTPPLII3jppZewbds2jBs3Drvssgs6Ojr4\nY8477zx897vfxTe+8Q288sorePjhh/Hiiy8W/fsvvvgiWltbEY/H8frrr+PSSy/FyJEj8e1vfxsA\n8K9//QsbNmzAEUccASDb6HfkkUfii1/8Io4++mgcd9xxmDVrloe9YUHFcgXo5p7l8ijLqiqg7Amm\nszsOABjdEpXKWGa0umzwE0/qIxRG88ogk3kM5H6R9CdSHorl7GsPBYySJzZdynJXfxKbe7Pvlxgb\nB1hqYs9ACql0puR7yVRB+UYkTcqyzfQ+INfjb5qmp0lYbjzLQPb1uf4sJJ1TUQBrBUKbshyzvwh1\nWmVIuLFheMzcVrXIxITISRXEJjmVzxLzLbtRzpUvQlnjbN7rYx70qEsbxtf3m4yZXxyD6dsNd/V4\nQg+GYUhZISrNzJkzcccdd+Dtt99GIBDAbrvthkMPPRRLlizB1q1bceihhxY85phjjsGFF16I888/\nH8cffzxGjRpl+/enTp2K4cOHAwB23313vPbaa/jZz37Gi+WFCxfiyCOPRGNjdmV77733xpo1a/DU\nU0/h2WefxWmnnYYjjjgCf/7znz2/VrJhVAAnzzJTzzwry/zEr9bgJ9uM9Wl3NgJm7DB5VRlQH3nN\nFbBGocGvWe8Uv1hSbik8GDB4geClgO2LSxbnmjzLHw7mK7cPa8jJ8gayKSMGj+KT86qWW1lmyTDF\nbRjZ29IZ07t/WOGYES9CvTwvU5ZLRccBlrddl7K8tYi9abjkRajqBD/AfkSzDKZpurqQ8WLDUM0q\nZq+PZR2roJyGwRpnC5RlbzaM/XcYhZP3mogv5F1QE0QxmG/55ptvxqGHHgrDMHDooYdi8eLFWLx4\ncdFiORgM4uyzz8bixYtLWjCKEQwGEYvF+M9/+ctfcMIJJ+TcZ9iwYTj99NPxhz/8AQ899BAeeeQR\nfP755+5eoED1X7rUIU7RcZVWlmVP/J8OKsvjWhuUtqtZMQ2jK2YNTmDob/CTPxE3RYKIC+OX3cBO\n4qVGXQP6lGWehJHX3AdkLwCGNYTRFUtiW38Co1vsL36Uc5bZBZjX6DimLBexYTRFgggGDKQzJnoG\nUp4UGZWc5eDgqkDC42dBdlVDt7LMRsiPyEmZkfOv8xUGhSVbL57lRDqDwYAK3ltRCi/Nl5bFRG2l\ngCvL5UjDsFGWvRbLBKEC8y3fe++9+M1vfgMgW0D/x3/8B5LJZI5fWeR//ud/8L3vfa+kqgwAnZ2d\nGBgY4DaMe+65B6eeeir/3bJly/D444/z+//617/G+PHjseeeeyIQCODhhx9Ge3s7V6e9QMVyBZCP\njtOThiHrQVL1q3X2MGVZrVhuVUzDYJ35uTYMzdFxkkpv9j4hbO1PelOWJZu6dOUs2/mVGSOassWy\n08XHgOLyO89Z1jaUpLBYZoNuumJJ9AwkMU7x8yiiMu4ayB5biVTGtcczI6jhMmkYgBU155WtRTzL\nbZL+daYOq+T5erFhiPtXSVn2YMNQVZbdNviZpunCs2wTHZf2loZBEKocdthhWL58OS+MR4wYgd12\n2w0bN27ErrvuWvQxkUgEo0ePdvzbX/ziFwEAoVAIkyZNwoUXXoh58+YByEbX7b///hg7diy/f0tL\nC37xi19g9erVCAaD2HffffG3v/0NgYD344GK5QpQaoKfeLuucdfy09ayHyhlZdmlDUPWWmB5lkVv\nJVOW9XqWnRr8svfxrvCxx8oqy14bbnhsnE2x3NYUAbb0O158xBU/U9ZJ3T8bBru9K5b0PJhEZamf\n3a8rpha3WOz5AOcGP56GoSFGEBAb/Iooy452nPLmLLP9FAoYUoWgJxuGy0QJt8WyWPDKKsuOQ0lI\nWSbKxK9+9Sv86le/yrltxYoVOT/PnDmzZHrUSSedlPN7p/sDxS0YF1xwAS644ALJLVejqo+oefPm\n5Yw8NAwD7e3t/PemaWLevHmYMGECGhsbMXPmTLz77rsV3GI5yhUdp6ySDRaB+Ut7dnQOepZVlbwW\nxdfHlouLR8dpsmFIepbF+3gZDsF8p05FmZWz7G+xLFskuVWWvQ8lsbdhABoH+SjYMMT7ubVhsBUD\nw3AulHQry9b0PjENQy6S0Y36GvWwyqDS3Aeox2CKuJ2Cx+6vmoYh7g9VZTm/ME+m3W07QdQaOmPh\nZKj6I2r33XfHpk2b+L+VK1fy391www246aabcOutt2LZsmVob2/HkUceiZ6engpusTOlRveKt2sb\nSqLY4CetLPewYtmdsixtwyiyXMyL5VjSU94xQzZnGbD2U78nn6qcstwkFGPpjLvXmUxnsG5LdtJi\nMc8yIJ9braosW2kYuoaSlGeQj3RR5kHBBAT7j8PkSMCPNIzsvhpRpBfAn3HXHmwYipnxrEHTUxqG\nS2VZ9WKAHRsBAwgH5dI37KPjSFkmhgZXXXUVJk2aVLbnq/ojKhQKob29nf8bM2YMgKyqfPPNN+MH\nP/gBTjnlFEybNg133303+vv7cf/991d4q0vj5FkexnOWy7uk7LbBb6xig5+KZ3lAiH8qZsNIZ0zP\n+wkQbRjyynLMkw1DsqlLKKbdqpcfb+lHKmOiKRJEu80qgEzDpGmalrKsnIbhXlnOZEz+WbG7wNRl\nXVL3LLsvygC5KY4M/WkY2QujtsbC42pbf+mLUHcT/NzbMFQHLFnJPl6SKRTTMJjaq6gsDwhxjLJR\ndXYxn2791gRBlKbqj6jVq1djwoQJmDp1Ks444wx89NFHAIA1a9ago6MjJ3A6Go3i0EMPxcsvv1zy\nb8bjcXR3d+f8KxfpjMmXUWXGXXtRTZVPMLwIdP6yT6UzPLdX2YYRzZ6QZdIwmFc1YFj5zED2ZMFO\niDqylrmyLJOGEfWeUMEe6xQXFg0FwOYiuPWqipP77E7GMspyIp0B+ziq5yy731e9iRR/Xr+bYt3a\nMNx6lvkKg8Tz6U/DGFSWmwtXbBJp+4SP3Ia0cjX4ZR/jV7OySNzlYI+oa8+y2gqg+FwFyrLHCX4E\nQRSnqo+o/fffH3/605/wzDPP4A9/+AM6Ojpw4IEHYsuWLXwizLhx43IeM27cuJxpMcWYP38+2tra\n+L9ySvligegUHZdMm56GOfAGP+nsTvlRp5t7EzDNbHzWqOaI4/1FuGdZovhjS8VtjeGCCWkjm/VN\n8evn6RQSCh+zYXjKWZYbRGEYhuf4OMuvXNyCAVhL8Sx5pBhuGpFYgeRFWWYXTNFQwFbx1Re3qFiU\nebVhKDSW6lSWTdO0GmcFZbkpEuRWALvjKpUxlS+asvf14Fn2OQZTxKsNQ7VYHlAc9APYq/TsuSkN\ngyD0UtVH1OzZs/G1r30N06dPxxFHHIEnn3wSAHD33Xfz++QrZTITvK699lp0dXXxf+vXr9e/8TYw\n72U0FLD9Mm6OhPiQCC+DSdgJXCaXFBBUMokTPx9I0hpVHvPbMlgYJFIZR8WRpTOIFgyGzkQMdzYM\n/7N1Ae/xcaxYLjVoQCYyjKmnhiHfQBS16dpXwSmXPPs778VyKp3hypxf1qV8KqUs98RT3AMv9gIY\nhiFM8Sv+WRAvmsqdhqF8EeOmWHbZJOe6wc+FsswEkGTazOllIM8yQfhDTR1Rzc3NmD59OlavXs1T\nMfJV5M7OzgK1OZ9oNIphw4bl/CsXTl39ABAIGNaUOw8nf6YQN0gnF8ifYKzpfeqZts2CmuqkkhUb\ndc2QbUqTQTbrFrBUQC3Ksoyi6FlZLp2xDFjKcleJNAxr1LX8GOAGjyOOAUtZHtZov69auc/f/cWl\nOA1N1Rvr1bMsd5GmLw1j2+AKQkO4UK1nqTN2TX5iWo5SzrKH0ecqw2IAb55lzw1+ZVSWs8+XfY2m\nabou9AmCKE1NHVHxeBz/+te/MH78eEydOhXt7e1YtGgR/30ikcCSJUtw4IEHVnArS+PU3McYpiEK\nS9l/qVIs97DmPrUkDAAIBQP8uZyylruKjORlDJewDshiKb0yxSvbTx4a/JLMsyyvZLsZeW2aJj7i\n0/tKFcvOyrKlgKmf1L0oy3x6n8/KsrhSIOvFVVmNKUafpHcd0JuzvC1WOL2P4RTLGOdL/YbSqpKd\nz1YGZWVZR3Sc62JZ7TndeZat/cAuYpNpS2EmZZkg9FLVR9SVV16JJUuWYM2aNXjttddw6qmnoru7\nG3PmzIFhGLj88svx85//HI899hjeeecdnHvuuWhqasLXv/71Sm+6Ld0SS8rZ32dPnt0OubelYCcl\nZZ+fxAnGylhWL5YB+azlbTF7G4ZWZVnFFsGXwz3YMLhnWaJI8qAsf9YTR088hYABTBnVZHs/MQ3D\nrql0QFCWZWnQ4FmWucDkyrKH40WMjZNVzj0ry3yFobzKMvMjtxW5CG1zsDe5iY3L3t+7DUM1M97b\nBD93r68cnuVgwODecraCKNo/KA2DIPRS1RP8NmzYgDPPPBObN2/GmDFjcMABB+DVV1/FlClTAGRz\n9mKxGC6++GJs3boV+++/P/7+97+jtbW1wltuT49DXixDq7KsOpRE4su+k03vU4yNY7RGQ/isJ+4Y\nH1csY5kxQiLuTJZ+yfHTgDjBz4MNQ0FRZJ5lN8/3waBfefLIppInY56CkMqmIBRT2N0oyw0e4sIY\nlg3DZ2VZMT0GsPaF28+CFR0n4VnmnwMNynJ/KWW5tCXHbTwZT8NwceFk2VVUM+PVn8ttdJz7CX5q\ncYyMaCiIZDrF96f4vNTgRxB6qeoj6sEHH8TGjRuRSCTwySef4JFHHsFuu+3Gf28YBubNm4dNmzZh\nYGAAS5YswbRp0yq4xc5YzUqliyQdQxYGFJf3VDr7rYEk7orlFp61XPr18ZG8jcUa/JytA7KUv8FP\nvji3lGX1IknGr8y2g/kc7S4+VNNVAMGn6iUNQ8KGwXOWHT5PpVC9uAS8T/CzGvzkvevJtKlckOWz\nrcRFqHODn3rGMiAmOLjwLLtt8HNxzJR73LUbZTl7/9z9yZ43GDAQVGy6JghdbL/99jAMAw8++GDB\n73bffXcYhoG77rqr4P6GYSAYDGLChAk4//zzsXXr1oLHL168GOPHj9cyiEyVqi6W6xG+pByVs2Ho\nUJb9iFviA0nc2jAkGxi7YoUjeRksH9bryOt0xorok/EsN4bdF68MPrlNybOsXpB9KOFXBrIXnnz5\nva94kcQKFhUFjDWXJtIZ1xMIZVZjdIy7dqMse2kkA9xdpGUf501dtorlUikzpT3LbnOI3ewr1+Ou\nvTT4VXEaRvb+uZ89t9tNELpIJLLnjkmTJuHOO+/M+d2rr76Kjo4ONDcXRpj+5Cc/waZNm7Bu3Trc\nd999WLp0Kb7zne8U3G/hwoU44YQTpG1yOqGjqszIK8vehixkhAJQvliWn0ZmeZbdKcvNkiOvWfNe\nKQXMq7Isvt6yeZaZDUNmcpuHyDCZjGWG0/I7/zwpKGDiZ8+tGsrGw8vaMNyqDqpNZOJ9veYsyyjL\n4aAVN+nVt8yOmaLHVaNDg59rJdTDUBJ+ISPnHPTyvni9GKiYskwDSeoH0wQSfZX5p/D9OXPmTFx6\n6aWYO3cuRo8ejSOPPBIAcNZZZ2HJkiU5sbwLFizAWWedhVCo8BhubW1Fe3s7tttuOxx22GE455xz\nsHz58oL7sWIZAP785z9j+vTpaGxsxKhRo3DEEUegr69PdU9LU9We5XqkWyI6DhAa/FwqZeIJSXXp\nMpHKIJMxbTvdE6kMtgyqj26LZTaNz2mKnxUdp961LwsrQg1Dbmm5yePSO2BlJsvlLLsfRvGhML3P\nCaeLDzfKsrg/B5JpJdWW0a2gLKczpq3n2olK2jBk90tzJIhEKuM5EYNdEI0o0QtgZ8NIpN16bHXk\nLKtZytw0lpZ7KIlbZTk/fYMyluuIZD/w8wmVee7vbwQizuIK4+6778a3v/1t/OMf/4Bpmjj66KMx\nbtw4HHXUUbj77rvx3//93+jv78dDDz2EJUuW4E9/+lPJv/fJJ5/gr3/9K/bff/+c29999110dHTg\n8MMPx6ZNm3DmmWfihhtuwMknn4yenh68+OKLvtoz6KgqM9LRcY3elpXFk7dqBzlQeorfZ4NjrsNB\no+jJVgbmWXaOjhtUwIpcXIzQNJREHHUts7zjtcHPNE0lZbmZF2Rqn4W+eAobu7IrADLFslPDpBsF\nLBQMIDR40eU2a1lmKElzJMjHgrs+ZhSH+ADeGskAcYVB7jl1JWJwZblILwCz42yzW2FwkYoCiM2e\nZchZ9mLDSHtsYCyTstyQd0FAGctEJfjCF76AG264AV/84hexyy678NvPO+883HXXXTBNE3/+85+x\n4447Ys899yz6N66++mq0tLSgsbEREydOhGEYuOmmm3Lu85e//AVHHXUUGhoasGnTJqRSKZxyyinY\nfvvtMX36dFx88cVoaXE+z7mFlOUyoxwd59KGwVTASDAg3ewhLq/HEvbqnDW9r8G1d4h7lh2K5VJD\nSZgS2p9II55KK59sGCojhwGrYHG79B5PWf5dpcgwRWV5zebsktSo5ghGSIwk50q9jWfZrQIWDQWQ\nSqRd+3q5slxiKIlhGGhtCKMrlkTPQNLVioeqegloyFke/PwzH7wTurKWSzX4OUUyuo+Oc9/sqRwd\n58WGwb47q1xZzlfqSVmuI8JNWYW3Us+twIwZM4refuyxx+LCCy/E0qVLsWDBApx33nm2f+N73/se\nzj33XJimifXr1+P73/8+jj32WCxduhTBYPZY/stf/oKLL74YALDHHnvg8MMPx/Tp03HUUUdh1qxZ\nOPXUUzFixAilbVeBjqoyI6sse/UsWycX+bc4EDD4F3ApRcZrxjIgpGGUUAHjqTQvZIvZMIY1hPiF\ngBcrhsroafF+/Ql3/lhRkZaxC7iNDLP8ynJX25YNQ5+yDAgKmMv4OB4dJ32B6a6QVE1cAHSMu66M\nsryt3z6/fDi3YRTP3GbFmVtPbzyVVj5uVP3k4vui+lxex13HFRv8eMqMwucOKEyaoQa/OsIwslaI\nSvxTFMCKNewBQCgUwtlnn43rrrsOr732Gs466yzbvzF69Gh84QtfwE477YSvfvWruPnmm/Hyyy/j\nhRdeAJCd1Lx8+XIce+yxAIBgMIhFixbhqaeewm677YZbbrkFX/ziF7FmzRqlbVeBjqoyox4d5+3E\nr/oFnL+0VwyWhOHWrwwInuUSChmb3hcwrPuLGIbB7RlerBgqA0kAS03MmO6WlFnRGw3Jqf6NLpVl\nKwlDzn823MGrOuDiAix7f2/xcTI2DPH3Xm0YbtIwvBbLsh5rXVnL20p6lrMFdCpjFj0+3eYQs4us\njJn92yooTyMV7qd6jFbKs6xsa2HpInwoibuLGILwi/POOw9LlizBiSeeqKT6MjU5FosByDb2ffnL\nX8bo0aP5fQzDwEEHHYQf//jHeOuttxCJRPDYY4/pfQECZMMoM1YMVukT/zBNxbJqQ1VjOIiuWLLk\nkrllw9CgLJcoltkJva0xbNtsOLwpjC19CU8jr/sUm6yahAuQWCKtfEGi4lcGLM+yurIsl7HMcPKA\nq6arMLxEhpmmKWXDALxnk6su9QPebRjqqxrumz0Z6YzJG/zaihTLDeEgoqEA4qkMtvUnCy5SXA8l\nES6y4qmM0uAMtznLgPox6vb1WcWy4rhrbcqymbMdBFFpdt11V2zevBlNTaWtHT09Pejo6OA2jKuu\nugqjR4/GgQceCCBbLJ944on8/q+99hqee+45zJo1C2PHjsVrr72Gzz77DLvuuqtvr4WOqjLTIzFg\nAfBuw3AzQAKQa4yxMpbdK8stUWcVkOX9FlsqZugYea2qLIeCAb7U2e+iAOxTGHGcvZ+7cddubRh2\njV08DUO5SGI2DDfDKDJIprNFgN8XmJ6i48owlCR7P+/Kcs9AkqdDFWvwA0onzbj1LIv2gLji/lK9\nkAkGDF40qr43noeSuMxZVlfqbaLjyIZBVBGjRo1CY2Njyfv86Ec/wvjx4zFhwgQcd9xxaG5uxqJF\nizBq1Cj09fXhueee45FxADBs2DAsXboUxxxzDHbeeWf893//N2688UbMnj3bt9dBynIZSaUzvOBx\nTMMQlpRN01RupHPT2Q8IXr8ShVmnx+l9gLWcLKMsF2tCYrAC73MPxTJv8JNssgKyFxWJmLsIL5Vs\nXUBcepc/6aczJj7arKosO+TrulSWeQqCi4KSXSwGA4bjxYX3C0z3xXIqYyKZVlNLM4Mxd4DCqoaH\nGEEG86Q3R4K2BeHwpjA6ugewLVZ4XLltgAsEDESCASTSGWVrhMrwFkZjOMjHt6uQcN3AyD7n5fEs\ns+OqYCgJKctEmVi8eHHBbWvXri35mG3btind/5lnnsHUqVOx00478dt23XVXPP3005JbqQc6qsqI\nqHi1SHqWU8IJVQU+6tplvFPpBj/mWXZvw2ATDEs1+HXxUdf2xfIIoRnJLSqjpxleBpPwBATVpXcF\nNfGTrTEkUhlEQgFsN6L0VT3D0Ybh0rPMbRgulOVuoSHW6YLRq8/fjWe5IWLtC9XjdCCV5gqvbIOf\nDmW5VHMfo9QUP7eeZfEx6vFq5RsY47bojJZdWc6NqmPbrXLBRhDVTktLC37xi19UejNIWS4n7CTe\nGA46fqE1RYIIBgykMyZ6BlLKQxbcnPjZtgGl/aWfalCWZXKWt8UkbBiDkWh2I5plULVhiPd1Uyyz\noko+AYHFhck/1wef9QAAdhjdLB0dyPZzVyyJdMYseNwAP6m7W61woyx3xeRsS4CGYtmFZzkSDCBg\nZJvWBhJpqe1kiOqwrF3KzYVTPqVi4xjMntFV5MKJF8uKF03sMT1xtcEkybRlxVE5Rt1mLcddpkpE\ngtZQJxVcp2EURMe5K7oJopqZNWtWpTcBACnLZaVbMjYOyHZ68ixiF8vKPLlA1bPsUCwPJNP8ZDuu\n1YtnebDBL5FCxqYzfqvMSd1hkIYM/S6aIXnWsuKgEMAqkqQTEAbvl0hnpE/EH3aqWTCAbCMlkJ12\n2l3Et2w1IpVPWZaNWszeJ7v9brPJ2WARFfXSMAzXvmXxIs2ugTUfK2fZvQ2DXYSOKHkRKqMsq+ea\nu8laFr+LVApKGUtZMdyOu3abhuG2FyA/uYhdUJANgyD0Q0dVGZGNjWN4yY1lX6CqyjLzONudYD7r\nyVowoqGAYzpBKdhrM037JjmugNk0IQGVafADvE3xs5q61GOwZE/8H23ONvftMEZ+bGkkFOAXMcWs\nGJVQlrslG2IBDXGLbldjXCqY1shz+eNIh7LMkmOKJWEw2hpLNfi5VzDd2DDYfpUdR89olLCUFSPh\nMkea3T+VMW0FgGJYSr1HZZka/GoaP0c1E8VR2ed0VJURSyWTW6r1khvrZkkZsJRou/G9n3ZbFgy3\n0/uA7Bc9G4Ns51vu4jYMZ8+yl5zlfldFixfP8mBxLhkdFwmJ6Rtyn4W1m/sBAFNHyxfLgJC1XERZ\nHnCpLDe4HAMMCANJJC7MdA3yUVGWAfcKpmpsHOCu2TOfUhnLjBElMrfdKq/iY1RsGAMJS/FX+c7h\nsX5u0zBUbRjC/lDxLbtOmcm78PDyvhCVIxzOHmv9/f0V3pKhB9vn7D0oBXmWywiPjSvRsCbiJTfW\n7QCJxkjpTNxPNTT3AYM2k4YQtvUn0RtPAii0dDAFTCYNQ8cEP5UiqclBgS/9fIPFucrzRYNI9Gek\nUxA+3pK1YUwZpVYsj2iKYMPWmE2RNHhSV86DdZ+zLDuQJHuf8nuWAffxce4aS529/k7wBr8SKzal\nLpriLic5AoXZwDK4vYjx2uCn6skWi914MiP9OXKfMpO7YkNpGLVJMBjE8OHD0dnZCQBoamryJEQR\nzpimif7+fnR2dmL48OF8CEopqFguIyr+S8Baeu6OuVeW3Z5g7IvlwYEkHpr7GM2RbLFsV9xY0XHO\nNoxyTvADrJg5dzYMNWUZyBbW25CUSkEYSKaxafB9mjKqdBh8PtwDXmTIizXu2q230o0NQ26IT/Y+\n5U/DEO+v+vpcKcsuM7dFpBr8ShxXTDUttw3DbTGpchGTSmfAHBRRiROoSChgwDCy1rJ4Og1AThRx\nn1+el7NMaRg1S3t7OwDwgpkoD8OHD+f73gkqlsuI5b+ULZa9KMvu1AonlYwnYXho7mO0Okzx6+IK\nmPNycVcsiUzGlG6UEnFVvHJl2UWDn6JnGVDL192wtR+mmW2iHNVsf6FRjFIXH0xZVrf2uIsLA8pr\nw3ATTwaINgzV7ODBz4HK527QhuHJsywTHTd4zHUV8yyz4s5NGoYLG4bXZB+VYln8jKoqtIaRzZGO\np+QbcU3T9DAZM2+CH427rlkMw8D48eMxduxYJJPuV0kJecLhsJSizKBiuYyoe5bdK2WuT/wO9oJO\nPr3Pmw0DsBIx7JaUVYaSZMysClmqALCDNRgq2SK8NPgpepYBtXxd5leePFJ9OW94idxqt1Mhox6U\nZbc2DFeDfLwu96s2+MXVn48ryx7SMLpkPMvNpS6a3DeSceuAwoWT2+8yN6PIEx6KZfYYlWJZ3A+q\nFx98KMnghUdycNw1RcfVLsFgUKmAI8oHHVVlhJ/4JYskL0qZW88yK4TsYr6sBj8NxXKJi4F4Ks0L\n0VIFcG6Cg8vGLhfeUZ6G4WbctRtlWWH5/ePPs8Xy9qPVLBhA6eX3AZeKottBFIBow5BXllMZkxf2\nspim+jQ9huWNVbuoZRekSspyRKeyXCpnOXfFRsRtegMgTrlTUJY9FssqFzHMYhIMGNL55CKqg0nE\nY0L5IrRAWaY0DILwCzqqyojb6LhypmE0OijLvFjWYMPgWctFlGW2/BswnC8uhntMxHAzStdbg5+X\nFATnz4Lb5j5ASEEo1tjlsRHJlWeZ2zCcleXmSBCsvlG9wIynMnyaXvmj41Q+B9ljYSCZQVohnkzE\n8izbX4SyWLmMWfj942X4Rf7UORnY8dJQBhtGwoNqLj5OWlkWYvHCQbXivHAoCTX4EYRf0FFVRrpd\nRse5y1n21tlvV9h09jAbhkbPcpHXx4q1tsawow/Za9ay1eAnr/A1cqVX/b3p9/B8Mp7lj7dkleUp\nI9WVZbt9mcmY/GSsOkKdDyVRVHsBtQtMcZCP6jEjXvSoj4h351n2MjkScPfZS6UzfJ+W6gWIhoL8\nudgQE4ancddh9w1+jarJPmF1q5SX6YTZx6ldDPAL0JBaLB5QOJSEouMIwj/oqCojqtFxrKnJzUQy\nN9PIgNId5P2JFH8NWmwYJZRlpn6VmjLGKJXgIIMbpbfJxYnYej7W2KXiVS2Psmy3L3O9lS4jrhSa\nuhgqaRiAe+sS+7xHggGEFFVF155lF/ne0VCA2wPcfPa6hBWDNofvIavZs/hnwd0EPzc5y+oXl4C7\nlJJyK8turU2A/VASSsMgCP3QUVVGVMZdZ+/nfigJO8GoL5kPTr0qciJmzX1NkSAvdL3AlpR7ihTL\nzFJRasoYw0t8XCbjzqvqxYahOu5avG+fw/Ol0hls2BoDoB4bB9gry2JxU05lmcUmOhV2DLfWpZhL\njz9gfRbKER1nGIblW3aRtcwK39aGkONFAdvnBZ8FdmHhxYbhImfZdXSckrLsLVFCdeS1qCyrkm9p\ncTt5kCAIZ+ioKiPuPcsuGvxSrAB0p5INFFF+dE3vY3BluUhh08VHXcsUy/YJDk6Ir7P8464V0jCi\ncif+jdsGkMqYiIQCaHdhlbFTE1mhGwwYysqrW89yMp3hhZJyNrmqsuwynkx8jPIEPxepKIC3rGU2\nFVNmxWZEc/HjqlI5y+VIKfHq+40ovj4vynKDMOzHNE0k01kPOzX4EYR+6KgqIz2KS8pehizEXCrL\npeKWPmV+5VbvFgygdM7ytphzFiyjVIKDE/05XlX1dArVpfdMxnTZUCg3uW3toAVj8sgmV5nTTMmP\nJdM5xS33wLsoIthnUFZtY4ife9mVDLfKstt4MsDd8AtAsGEoPifPWnajLEtMxWSwCX+isizmAruy\nDoTVbRjWhYxbe4z8585rogS3YSimYXhRljNmNgEm4cFLThBEaeioKhPJdIarc6o2jN54NjdWBa8N\nfsVO/J2CsqyDlqj1+vLZKjFljOFFWe4Xsm5VissmBQ+xiKhkq3iWZXOdeWycCwsGkL1AY55YcX+6\nTcIARBuGWjHJkjBaos6WAYbb1Ri3S/2Ae8+yFR2n9pxelGWZqZgMK2XG2pfJtMlTQ9x5lmsjZ9lt\nwalqw/DkWRYeI2Y7kw2DIPRDR1WZ8KKSpQU1UpYBHxr8dGYsA1bOctE0DG7DkFku9qAsJ9XjuwD3\nNgzmVzYMVSVbLl/3483um/uy22Vw64uYguB2JC/gbhAFoG5byt7Xnc/fmw3D3cVAn8vGNS9Zy9sk\npmIyhgvTMRmiIuzJhlFOz7ILG4abC4Hs49SUc2/KsrX/B5JpSxWnYpkgtENHVZlgSldzJCitkjWG\ngwgNqnwqHsx0xuRfnO4zcTMFavangw1++pTlEjnLMefBCYzhNj5bGdxYIoDcBj8V1b9fWHpXUbJZ\nM6TT5DamLLtp7mMUS8TwoiyL3koVVJMwACtBxm2DnxsbRqOLRrLs/d1dqMl+Fophpcy4a5zNGQft\nwqrAVxmqdNy114KznMqyYRg5Hmn2nJSGQRD6oaOqTKiM7WUYhuHKgykWJW6XLoFCJZApyzoylgEH\nz7IrG4a6suwm6xYAmsLZbU8JFyYy9Lls6uI2jKSDsuwhNo5RLBHDOqm7X3q3mwpphzWQRF1ZVm3w\nq4xnufzKspUy47xiY6VhWPtSjFZz44nnOcQulOVy2DDYdpU7DcOzkp1MexpDThBEaeioKhPsxK+y\npJy9v3purFgsqy6Vig1c+UpZp+YGv1JpGFslpowxvETHWcqyuwxXQE1R7PdRTcxkTE8DSRjFlHpv\nNozsY9IZEymFCws3F5iuo+NcTokDvHuWlT8LGjzLKsrytiLKsltPr6uc5TKmYcQ9NvhFy6gsA7kr\ngUmyYRCEb9BRVSa6XfgvxfurTCSLCYWNqvoTCgb4iSL/JPOp5gY/VgAm0pmCk2eXC2/lQDKjHt/l\nMpEgEgrw8bQqRYub6X3Z+zuriZ09ccRTGQQDBrYb0aj090WKjQ+3bBjus3UBNXXZsmG48SyrNvi5\n8/gD7hRM0zStNAzFBj8vaRjcsyxlbyocfe41h9hNdJzrcdeD909lTF5IOuE5Oq6MaRhA7sUHpWEQ\nhH/QUVUmehRHXTO82DDc+EsBS+UQFereeIqftHQry0Chumx17Tvvr5ZoiHu7VdVlt+oe4G6crpWx\nrOqRdlaWWWzcdsMbPfkWi9laLGXZeyOSLJYNowzKsg7PssJry/YEZP9f9cLJk7KssGLDVxj6hIum\npFdlWb3Z02vOMiC/r3TlLJdLWRYvPqjBjyD8o6qPqvnz52PfffdFa2srxo4di5NOOgnvvfdezn3O\nPfdcGIaR8++AAw6o0Bbb46azP3t/NzYM9yqZ+Djx5M9U5dZoiCvCXgkGxGlk1nPFU2l+cpM5qRuG\n4Tpr2W2DHyBkLSsULW49y82CsmzXULhui/fmPsDa56JXdcCDshwIGMrDGgB3qzFus8n5Ur+Lz4Eb\nz7IYOah6nHpTluWH/bAL1e6BFNKZ7GeODyTxeCHubty12nOGgwaPQZS9SPOqzqpeDHhVlhv4Bbv1\nHlGDH0Hop6qPqiVLluCSSy7Bq6++ikWLFiGVSmHWrFno6+vLud/RRx+NTZs28X9/+9vfKrTF9rjx\nXwLCRLKYug3DTWEDCMvKRYrlsZpi4xgtfOS1VZix6X0BI1ucy+A2a5ntKzfKspusZdfK8uB+yJj2\nJ2KmLG/vobkPKD7FL+51tcJF1rKbNAzXNgyXQ3wA8XjJIJORS0bhF2nhIC/oZPGmLMtP8BMLahYf\n511Zdh8dp3pRYRiGclJJIu3NZuJ2gp/b72q2P8WLQ1KWCUI/eiRCn3j66adzfr7zzjsxduxYvPnm\nmzjkkEP47dFoFO3t7eXePCX49D6Fzn7A3ZAFrzYM6wRjfeF3ao6NY7Q0hNDZE8+xYTALRltjWNpz\n7bbJz2q4Uz8UeNayQgHY5zEGC8gWScXeWx2xcUBxG4bXxq6GcBA9AymlIoldILq1YZimKT2WXYcN\nA8juJ5n3tt+D/cdtGkYileGfPxl7UygYQGs0hJ54Ctv6ExjZHOGKcCVsGO5iC4PojaekVf/aS8PI\nPk7saaE0DILQT00dVV1dXQCAkSNH5ty+ePFijB07FjvvvDMuuOACdHZ2lvw78Xgc3d3dOf/8hl35\nq6hk2furLyt7GbAAWEusxWwYuovl1iJZy1YWrLP6xSg2bUwGbzYM9Xxd9nzNisV5MGCpZHbL7zpi\n44Di48O9Kss8a1lh+d3y+as3+KUyJrcjyWAVy+6Hroh/xwm3zX2A+5xlNmTGMOS/h4Y35x5XcY+e\nXtU0jIzwPnoZGCP7vnCbSZnGXXtVltnjRDGFNR4TBKGPmimWTdPE3LlzcfDBB2PatGn89tmzZ+O+\n++7D888/jxtvvBHLli3DV7/6VcTjcdu/NX/+fLS1tfF/kyZN8n37u12c+LP3d+FZ9uiDayzS4McG\nkmi3YRTJWrayYOUvLHjMVZ+7Bj9VWwRgxc0pNfjF3RdJpUZem6YQG+fZs1xoabE8y94UMDUbhvoF\nZnMkCLYYoXTMeLhoCgqebNmijDeWhtVXNNwqy+z9VFmxYRM02ZAgS1n26lkuHHpUDFGB9qL6yyaV\n6GvwU1OyvSr1TEyJhALSqykEQchT1TYMkUsvvRT//Oc/8dJLL+Xcfvrpp/P/nzZtGmbMmIEpU6bg\nySefxCmnnFL0b1177bWYO3cu/7m7u9v3gtl9g5+LNAyPynLRBr+eQWW5Va+yzBRW8fV1KTQhMfIV\nMFnc5iwDVtxcTKFo6XOpLAPZAntLX/EiaWt/ku/DyR4ylgHhwiOW5FYGLznLgKWAKTX4uUjDMAwD\nLdEQugdS6B5IYewwucd5WeoHssdMIiUfXdjn4aKJK8uKnmVPKzaD0xx1NcCZJpBMm4iEShd24neQ\nKz+5YvNludMw2MWH1+QidmHoVhEnCKI0NVEsX3bZZVi4cCGWLl2KiRMnlrzv+PHjMWXKFKxevdr2\nPtFoFNGoXoXUCb6kHFWNjmPKskKxnPK2tFeswa/TJxtGMWV5Gx91LX9SLzZAQQYd3lE30XFunq+5\nRPoGa+5rH9bg+sTLYAVSOmOieyCFtsYwV8C8KstxBWXZjQ0je/8wugdSSsqy1ejp7iuxMRxEVywp\nrZxr8SwrpmHwFRuVi1DhwgkQPLYeG9KyfyvtWJSy9yUSCig3QgLqSSW6hpLIN/h5258NecpymJr7\nCMIXqvrIMk0Tl156KR599FE8//zzmDp1quNjtmzZgvXr12P8+PFl2EJ53CrLrCFQZXyvl85+wPoC\nFouyT3mDn96LjNYiU/y2Koy6ZowoMkhDhljSffHa6KJY5tFxbpTlEkWSrtg4IPu5YYocU/kHPDZ2\ncc+ypI84kzHRE3fn83ezGhMT0incoPpZcDucBnCfhtHFlWX144pdhPIGOI/FJCBXUOp6X6TTMPjF\ngLfUF2Vl2aOtha3CUHMfQfhDVR9Zl1xyCe69917cf//9aG1tRUdHBzo6OhCLxQAAvb29uPLKK/HK\nK69g7dq1WLx4MY4//niMHj0aJ598coW3Phfuv1RQdQCXyrJHFbBBiMICshctPDpOsw2jqLLMbRgq\ny8WFcWcy9Hs4GfMGPwW1lBXnzZqX33XFxjHyLz6s4QneLsBkG7t6Eyk+tEP5AtPFMcMb/CLexg7L\nfhbcRggCQs5yicztYmztV1+xYVaobbzBz5tn2TDUMrfdjrpmcM+yqg3DbYMfK5alG/z0KPXs+5Ni\n4wjCH6r6yPrd736Hrq4uzJw5E+PHj+f/HnroIQBAMBjEypUrceKJJ2LnnXfGnDlzsPPOO+OVV15B\na2trhbc+F/dLyi6UZU0nGPZ3umMpfmLTn7OcPRmLxXJXTH4kL2NkszsbRsyDwmc1+Cl4lj0oyzwN\no8jzMWV5sgZlGShMxLCGJ7g8qSsqy2LDkupFn5u4Ra+rMawpVlbB9OKVZ8qyacrvT0BtKiYj/3OQ\n8GjDAMSsZed9ZV3EeFSWZW0YHsd5R4LZ5yuXssw+r+LxQhCEfqras+ykmjQ2NuKZZ54p09a4J55K\n82LD7bjr3ngKmYwp1cXuNY4oX43pHGzua2sMe/bD5sOV5YEiyrIrG0b5o+PKMe4aKB0ZpltZzk/E\n0KUsyyp8vLlP8XgBXNowPF5gsosf2dfX5+FzIG5jXyIl/dllF5JqKzbZ/d+V71n2UJRFQ0H0ICWl\nLPd7vohRO0bLP+5aj7JMNgyC8Bc6ssqAeNJuURxzzIoF05SPivK8dJnn8/PLrwwUz1m2PMvqNozu\ngSQf+yqDjkarcoy7Fp+v2Il/naaBJIz8IS/elWW1YRRWEob6fnI3It6bgqlqw4h5+NwFhDHxKlnL\nPA2jWT2SMf9z4NaGkX2svA3Dy34CXKRhpPUUy/Ljrj1G8RWJjiMIQj90ZJUB9kXWEg0pd3RHQwEe\nMi+rlHmNweKjiVOsWPYnCQMQx12L0XFMAVNYLm60LiqYCiZDzEM6hapqlb2vhzSMaHHbR89AEpt7\ns/tMnw0jV6n36oNXHXftdjx89jHMuiR3vCTTGSTT2QuscjWSebloAiwlWyVr2U0aRltTcc+yl6KM\nZy1LfBZ0Xfir5ix7HUqimobhdShJ7+DngJRlgvAHOrLKgFu/MpBtiFFtWPI87jpfWe7xp7kPsArA\nXkEFdOOtDAUDfP/KJmKYpslHVbuzYdhHudnhx5hjNoxkZHPElW2hGExR7OIpCF7TMBSVZTYe3sUx\no9oUKxbwnj3Lig1+bhVT1iCq4pd3k7NsRTIOFsseh2gAYrOnhLLseXKku5xl17aIsOJQEk3KMnMs\nkrJMEP5AR1YZcBsbx1Bt8oux8bCaGvw6/bRh5KVhxFNpXlCq2DAA9azlgWSGn2S8RLn1J+UKllQ6\nwwsEV0NJbJbedVswgEJlmdswyqQs6/Esyx4v2W0yDPdFoGrqgpfoOPFxfS5sGEoNfo1WA24yneE5\nxJ48ywoDamIeegoA8btM7iLNSsNw93wq465N0/TuWc57HBXLBOEPdGSVgZ4B9yd+QN2D6VlZzjvx\nl8OGwU76LAs2YFh+ZllG5E0bc0JU5VyN0lXN1hUKKXfjrosvvetu7gMKvapWg583ZVk1DcOdZ1mt\nwW8gYV1cuh0V3KBow/DS6Ck+TklZHkyZUVGWhzWGwXbJtv6kpSx7aPS1PMsKaRgeByzJvi9xjw1+\nKjnLYkHtddgPg2wYBOEPdGSVgW5NyrLqsrLXzFhW2FjFsn5luSUv7YNZMNoaw1LJHyL5MVdOsCI3\n6nI6mGqTFbtfKGC4OqlZS+95yjKLjfM45lrELg3DfcSVfIEEiDYM9QtMbluKqynLbldixMfK2zC8\nKaZNUTVleSCZ5sdzm4KyHAxYNrCuWMLyLHsoyqxpjlWcs+yxwS9jZleSSiFeOLoeH07KMkGUBTqy\nygBbUnbTrJR9nFrDkvfCJvfEz9IwxvqoLANZxdSNr5IxIq/Ac8IacewtLkxW3esTfKpuFMwmm8lt\nXFkerbNYtknD8KiAySrL3TH3F5iqF5defbGA+2K52WWDn6qyzI6JYMBwv2LTn/Q87hpwl4bRUKaU\nkrimNAzA2YrBLjwMQ89ExPznJwhCH3RklQHvnmU1GwY/+XsN8k+kYZomz1n2w4Yhpn30xlNWx76C\n+sVwqyy79426jQtzWyAxNTG3QLKUZZ02DOvCI5XOIDUYx+d13LWsssxUYdWJl+JjuiVTUbz6YsXH\nynuWs++h11znPkl7wVYhYUb1Qq1NaPJL6IiO482eKjYM/1NKTNP0PsFPeJyTFUNslnRt/8nbL2Gy\nYRCEL9CRVQa8xGAB6uN7YwmWiet96XJrf5LHao1p0W/DMAyDq8u9AynuWVaJjWOMUBx5zQsW18py\n9nHJtImkREMPK3Ld+JWB4h7pgWQaG7uyFzPba2zwY/uyN57KycD2mkogs/QOWMqy16EkMuOgvS71\nA4KCKetZjntUlpklJ66mLKs09zHE0ed6hpIoKMtltGGw7znAvXIeClqWLqfXx6f3afB/2/1MEIQe\n6MgqA16i48THySpl8aQ3paxBiMFifuWRzRHflviahazlbXzUtQsbRjNTQ+WUZc8DD4THyTT58aV3\nt8pykbiw9YNJGC3REB/5rQOxsYvZcAAP3sq87G4nuj0cM+yiNJUxpWwf5fYsm6aZY8lxg6qyzKf3\nuTiu2IVrV39SiDrTMe5awbNchqEkom3CiyebJ2I4FMsDGmL4Chr8qFgmCF+gI6sM8M7+MjX4WR5M\n7zFYrFge26pfVWaIyrIXBYwVAp/3qdkw3BZJEUFFklEUvRZIzUXiwljG8pRRTa6XcosRDBh8eEXH\n4GcgEgwoN10yVJVlKw1D/XPQHAmCbaaMdcmrLxZQiyiLpzLI8MjC8qRhsMbZER7tTez98zSUhOcs\ny19ges6MlyiWxSEpnoplSeVch7Kc/x1PaRgE4Q90ZJUBL/5LQLRhOJ/4k4K/1G0RyIqGjAls2BoD\n4I9fmSFmLfNR141laPDzqCwbhoGmsHzR4nXpXfRIZwbfY9bcpzNjmcEUxY6u7GfAS1MXO6lLK8se\ncpZFa49MU6zXeDJAbVKcuArh2i+vmIZhTe9zoSyz4yqW1DPumn0WZFR/Tas/Mis/TFkOBw3XF4WA\nVSyTskwQ9QMdWWVA11ASGWVZzzQy63EfDxZjfsTGMbiyHE+hi9swvHiWZZVlpvS6e18AtZOxV4+0\nWGSzAs8aSKKvuY/BFMWOrqwNw1OBFLJWK5wwTdOTDSP7OPkLTK2eZYnXxz4HbiMLAXVluavfvbIs\nDvthk+n0pGGUYdz14OMSqQzSmdL+da/NfQzZwSRaPMsUHUcQZYGOrDLgPTpOvsGPqRVeppGFgwGE\nBk/iaweX+f1UllsGX593G4alLMs0dvV7jI4THytTJPVxz7L7KXjMacEsHez90dncx2CFVUd3Vll2\na+sRHyvT1BVPZXizldvVGJULTC1pGErFsrcVBsC9suz1uNLT4Kc+7tprsQw4X6h5zVhmyA4m0aEs\n5zdxUxoGQfgDHVllwKuyzKaYqahkDSH308gAS+1gsWR+ZCwzRGWZ2zBc5SxnH5NIZ6SUXq9LvADQ\naJN9XAyvSrZhGNy3zCwd6waVf52xcQy2PzcNpm1469qXV5bZxWXAcH9hoZIg05/0rvC5sWF4UbLd\n5iy7avATegG8TrgD3KVhuPWTi4Wo04WMDosJIHqWnZ7P++cuHDQgfs2TskwQ/kBHVhnwGh3HHifj\nvxzw2NzHYF/gH38+aMPwscFP9Cx3CXmwqjRFgnwJVMaKYU1R86Dw8RxX5/emj3uWvSvZfYkUkukM\n95TrHEjCsGwY2WLZk5oo+FSdVH/LgqGeCcywlGX5Bj9PF00qyvJg3Ju3z4FqGoaHFZvBY3Fzr5iK\n4t06EJfYVzFhFLkbAgHDSvdx2FcJjwNJGOVUlg3DyHl8lJRlgvAFOrJ8ZiCZ5l/CXj3LvfGUo+9O\nRwwWYI3KZl/oftowmFraM5ASuvbVFTDDMArGNJeiX0OR1OTCs+zFI82W7mOJNDZuiyGVMREJBTCu\nVf/7w2wYLBHFW9e+9VgnL2c3T8Jwv5/c+Px1RMelMqZjkdSn4SJNOWd5sBfA3WTM7GO2CCkz5bJh\n6HxvnFY1dCR9iI93HkrC/N/evqvFY4uUZYLwBzqyfIadrA0DaHF5chSL7F6HkyMrbr0UNkDhyclf\nz3L29X3eF+dFp5sJfoBak1/MY5QbYO0nlZxlHcV5XyJtxcaNbPLUvW/H8ObcIS9eVivE4sopBcFL\nEgZDpcFPx7jrhoj8cj+7aHJrMQHcTPDL7oc2Fys27FgUFwT0DCUpve2maVoX/2VQ/bmy7LXBjxXL\nDheFAxr83/mPp2KZIPyBjiyfYSfrlmjIdUETDQX5l6DTyV/HiT//8YYBjG7RN/Ain9ZBtfSTbVlL\nQcCwblNlOJ82Jq8sezkRWzYM/4eSiI/tj6d4UokfsXFAoRXGy9J7JGg1JzoVSd0ePf7iY6Wi4zR8\nDiLBAM92dlIwvY5ZB4oPqLHDNE0rDcPF4JphDaGc1I7se+n+4kzWs5xMm3wlzduFjNwxqqvBjxXb\nTq9vQNN3tXhcUoMfQfgDHVk+w5eUPahk2cfLLSvzAQuaPMsAMKo5ipCPX8JMWWb+27bGsOsLCzHm\nyomYhjQMlQY/r+Ous89XRFn2ITYOKFyy9/KZMgyDd+47DSZhF4R6lGWVnGX374thGLz4dSrK9Kww\nZJ8rmXa2ffQnLCuYm14AwzByFGnPSqjkgBpRCS7HdEVtxbKsDUOTsiwel6QsE4Q/0JHlM15HXTNk\nT/5MtfOikgG5Jyc/M5YBKw1jW797vzKDjbze2qegLIe9N/j1JyVSF3Qoy4Ki6GdsHFDYDOY1JcBq\n8nNQlmPup/cxVBr8dPhiAfmsZT0NfuKo9dKfPdYHEAkGXBfo4mfBS8YyIG/DYO9LMGAgHHSvZMt6\nlhNp76O8s4+X82T7oSzTBD+C8Ac6snzGa2wcgynLzM9pB1eWPRY2ucWyf35lwFKWGW79ykDuaF4n\ndHqIyzHuOvtYS8leN5hUMtkvZblZn7IMWJ9JpyLC60AS8bEqyrKXcdeA1RTrWCwnvV+khYMBriI6\n+Za3DjbmtTW5TxcZnqMsey3u5GwKYsSeF9uH7MjrhCalV1VZ9vpdTZ5lgvAfOrJ8pmfA20ASBleW\n46WL5QFNJ36xMPJbWc73J7tZKmaMVLBh9Oto8FNIw4jp8KoOPl/vQIrbMPxSlvOnvZVLWdZhwxgm\nebwAgmdZU1OsU9ayDmUZELKWHZp+u3jCjPv9Ka72lEt51eElB+SbcHVkSIuPd46O8z4NEchVpr2+\nNwRBFIeOLJ/RpSzLKmWxpLdcUoZ4ghrrQyyZSL6y7MWGUfYGv7CCsqzBs8wmt63Z3Id4KoNgwMCE\n4Y2u/14pGsPBnMLB80mdDyZxSsPQacOQn3qpq1h2TsPQUwTKJmLw6X2N7o8rcbXHcw6xZM6yvhhM\nxQY/beOu5YrzBkrDIIiqh44sn9HV4Cd78tc9lATw34aRP/bXiw1DqcFPg9Jr2SJKvy+maWrxLLPi\n/F+bugEAE0c0+tYBbxhGjsrvdbnYGnntlIahw4bhosHP82qMWrHs5XMAyGctexlIwtCrLKulRWhT\n/CUn+JVrKImunGXxIpbSMAjCH+jI8hndDX7dDg1Luk8wgP82jPyiwYsCxhv8HJTlbPGqwUMctdIp\nSpFIZ5AajMHSoizzMdf+WDAYYpGkqxHJSVnu0XCBKTb4OU0M1GbDkFQwdXjXs4+XU5bZhaOXYlmv\nZ9myYZR6b3j/RZkuYiqVhqGrF0B8boIg9EJHls+wJWXvnmVVZbl2GvyCASNnQIOnk7pkg188lQEb\nhliOnOX+uPX7Jg/vDdtPrMbY3qfmPkZOCoKu5XcnZZkNJdEwwS+ZNksqmOLgC13HTDlylgH5rGUd\nKTPDhWZPr3Yc8Xh7bc3ntvezbBh6no+NzraDxeuVq4GRe5Y19QIAlIZBEH5BR5bP6FKWmcrmmIbh\nw1CSsT4ry0Cub1nHcnHPQAqpEhO0xOLWS/HKEg2cChaWgBAJBTxlVjflWVb8GkjC0KksN4TllOVu\nDQ1+zZEQH4JSajVGLGh0NZLJTvDzssIACMpyvPTz8el9mpRlrwVZSzSEY6a3AwAuuPsNvPNJV9H7\nafMs17myHCVlmSB8h44sG5zUIVkq1eDnubAZLByCAQOjmstQLEfFYtlDI1JjmBdJ20pcWPDiNeix\neJVWlr2POC72eL8GkjCYrQXQ51V1TsPwbsMIBAz+mSp1zIjvm9dGqwZJBZMryx6PUfZZcMqS7opl\nV1k85Zc36VOWAeCm0/bEflNHoieewpwFr+PDz3oL7jOgyUsu71m2vhO8wCf4OY279kFZpjQMgvAH\nOrKK8M4nXfjKDS/gyX9u8vy3WHSV9wa/yniWx7REc0bd+kWLsH+8RMcFAwbf16Wa/GKD6p73RAI2\nlMTJp6pn6T1/e/2KjWMM90FZLrU83RdP8WLSiw0DEOLjShTLawe9314vmgDrmNncGy95P2bJyW9s\nVYWloNzy/Ad4+p0O2/sxZdnLcZVrx/H2OQCyn4U75szAtO2GYUtfAmf/8TU+7p5hTSPVc+FftnHX\ng/unlLKczpjoHfxc6lSWqcGPIPyBjqwi/PHFj/BZTxyX3L8clz/4Fs8pVSGZzuCW51bjvY4eAN5i\nsABg5KBn8K1123DNI/+0LQR1pWEw1crv5j6GmLXsRQHLPj67rzu77YsWHQNJAPmcZX3ZurkF1iSf\nG/xy0jA0TW6zU/je/PhzHHfLSwCyn/cWj8VkqSl+pmninlc/xpl/eBUAsNuEYZ6eCwCmjs6q/Pe8\n+jGu/vM/bRXffk0Xahd8ZQfsN3UkeuMpXHTvm7jh6X8jnSlsmLMa/LxHMgL61MvWhjDu/s/9sMOY\nZmzsGsDZf3wt50KjUjYMv4eSrN3ch9P/7xVs7BoA4P37jm1vMGCURdggiKEIFctFuOHUPXDZV7+A\ngAE8vmIjjr55Kf7xwWbpx6/c0IXjb3kJNy56H8m0iSN3G4fp27V52qa9Jg/HmftNBgA8uGw9vnrj\nEjzy5oaCbnJdyvJBO43GsdPH49szv+Dp78giFpFevJUAMHawIfFb97yJ2xZ/ULQ40511m0hlihYq\nDF3Ksrifxrc1eFbdnMiNDNPkWc5r8Iun0rj+qX/jP37/CtZs7kP7sAbc+vW9PCu9dsryZz1xnH/3\nG/jh4+9gIJnBwV8Yjd9/Yx9PzwUAp+87CRd8ZSoMA3jojfU4+uYX8cqHW3LuoytCEMhOWLzvm/vj\n/IOnAgBuW/whzr3zdXzel3shrSM6brjG6DiRUS1R3Hv+/thueCM+2tyHOQte56tnZS+W0/56ltkF\n2uzfvIg3Pt6KlmgIv/qPPbDDmBZPz8eOK2ruIwj/oKOrCJFQAFfM+iL+/O0Dsf2oJmzqGsBZf3wN\n8xa+W9L3NpBMY/5T/8KJv30J/+7owcjmCH5zxp64/ex9PH8Bh4MBzD9lOh6+6MvYeVwLPu9L4IqH\n38YZt7+KDzp7+P10NfgNawjjt2ftjaOntXv6O7K0RLMn8oBRONFPlR8dtxumbTcMvfEUbnj6PRx+\n4xI8/tYnyAjFbEyTsiw+vlSTn46YuuzjrX3jd2wckFtgec/uZsMorCLinU+yF5a/X/IhMiZwyt7b\n4Zn/OgQH7jja03MBxZXlZ1d9iqNvXorn/92JSCiAHx23G/503n5ob/Oe+BIOBvCDY3fDgxccgIkj\nGvHJthjO/MOr+MkTq/j3hq4IQfE5f3jcbvjNGXuiMRzEi6s34/hbXsLKDdmmOdM0uXffi4LZHAki\nHMyqlrqbyCYMb8Q95++HUc0RvLuxG9+86w0MJNP6JvhF5LzyupRlKw3Der5NXTGcs+B1/PDxdxBL\npnHADiPx1He/glP3mejpucTno+Y+gvCPujm6brvtNkydOhUNDQ3YZ5998OKLL3r+m3tPHoG/ffcr\n+MYBWUX3rpfX4tj/fRH/3LCt4L6vfbQFs3/zIv5vyUfImMAJe0zAov86BCfuuR0MQ9/S2L7bj8ST\n3/kKrpm9CxrCAby25nPM/s2L+NUz72EgmeZJA34rjrphhU1bYxgBj0uJ07Zrw8JLDsZNp+2B8W0N\n+GRbDJc/tAIn3/YPvPZRVunTFd8VDQV4Q2EpT6S2uDDh8X7HxgFZ9ZKhK2c5nkojmc7g5mffx0m/\n/Qfe/7QXo1siuP3sfXDTaXuizaNliSE2xfYnUvj+YyvxzT+9gS19CezS3oonLj0Y5x081fPnLZ/9\ndxiFpy8/BGfuNwkAsOAfa/j3hq4UlnxO3HM7PHZJ9uL+k20xfO33L+P/vbEePfEUX/HwoiwbhoG2\nwfxzHZ7lfHYY04K7z9sPrdEQXl/7OS6+bzlfEfDe4McSa8o87jqdzZF+7K0NmPXrpXhx9WZEBy/Q\n7v/mAdosVFQsE4T/eDtzVwkPPfQQLr/8ctx222046KCD8H//93+YPXs2Vq1ahcmTJ6v/wbcfBF68\nERi9M5rGfBE/nfpFnDRuDOY+148PP+vDKbe9jO8cvhMunrkjYsk0fvH0v3Hvq+sAZD2+PztpOo7Y\nbZzmV2kRDgZw0aE74tjp4zFv4bt47t+duPWFD7Dw7Y3YOrgE6/UEU26YP9Wrf48RCBg4Ze+JOGb6\neNzx0hrc9sIHeHtDF06//VXM2m0cdh7XCsC70msYBprCQfQl0iVPxn2aPMvi+zpltP/K8giNXlWm\nLH/Q2YuTb/sH3vkkO4XwmOnt+OlJ07kvXxesKfb1NZ/jvtfWYc3mbDPftw7ZAVfM2tmXoo/REg1h\n/ilfwqzd2nH1I//Eh5/14eTbXsbp+2YLaK8RgsXYpX0Y/nLpwbji/63As//qxFV//ieeXfUpgOy+\n93qxM6IpjM29cd8SF6Zt14YF/7kvzr7jNTz/707uvy3/uGuPF4WD7+vWviQuvm85nhpsvtxjYhtu\nPG1PfGGsN9tFPmTDIAj/qYti+aabbsL555+Pb37zmwCAm2++Gc888wx+97vfYf78+ep/8NN3gM3v\nZ//9+68AgBkAlgL4vGUcVsbHYfUL2+H3b30Bnww04vP+BI4KAF/ZaTRO2XsimvAasErf67NjEoA/\n7mdixfguPLRsPbZuG/QqBoDR63uBnkb7B5tpIN4LxHsG/3UL/z/4L9ELBCNAtHXw3zDh/4XbQhEA\n3tS56T0dOCrwCXYINAOruj39LZEGAJeMA75xUhJ/fXsTXvrgM2T+Daz+N3BUANgnMRxYZZ8kIMMx\noX+iO5XEB0s2YktTBCZMPjSEGT8y67fhqMBW7BsbA6xa7/q5IgCODS1HKmNi/9hmYJW/H7SxA0kc\nFfgnDAAjPu4DNrOCVvBnF5vClrOakv3/nbdsxtGB9TCz15XYuSGEM/ebhBnbp2B8/In2bd+7/xN0\nBjqAfwM7A9i/JYJzD9weu7SvBt5frf35inEYgOePSeGB19fhjY+3Ysuy13BUAGgOh4BV6o3DTrQB\nuH2GiacaOvDEPzfCHPycj4hGPD/frMB72CHQiy9u/QRY9S89G5zHvgAePqQLv1v8IdKmCQSAnbZ8\nCqxa6fpvjtkWw1GBVWiMBfHWM/bH3i5bN2J4YAATO7YAoeGun2/sZ704KvAe0A1kVgHHBA0c96Xx\nOGr3FIKbnwfk21+kmNSxDUcFPkQ7GoBVhRF8rhi3OzBqRz1/iyDqAMN0mgVb5SQSCTQ1NeHhhx/G\nySefzG//7ne/ixUrVmDJkiUFj4nH44jHra7rrq4uTJ48GevXr8ewYcOA3s+AT98FPv8A2Lwa2PIB\nsPkDILal4G8RBEEQRF1xxDxg329WeivKQnd3NyZNmoRt27ahrc1bIz5Rv9S8srx582ak02mMG5dr\nexg3bhw6OoorhvPnz8ePf/zjgtsnTZrkyzYSBEEQRM1w/RUArqj0VpSVnp4eKpYJW2q+WGbkN9GZ\npmnbWHfttddi7ty5/Odt27ZhypQpWLduHR0sZYJdzXM1n/Ad2uflh/Z5+aF9Xn5qeZ+bpomenh5M\nmDCh0ptCVDE1XyyPHj0awWCwQEXu7OwsUJsZ0WgU0WjhsI22traaO9BrnWHDhtE+LzO0z8sP7fPy\nQ/u8/NTqPieRjHCi5ttnI5EI9tlnHyxatCjn9kWLFuHAAw+s0FYRBEEQBEEQ9UDNK8sAMHfuXJx9\n9tmYMWMGvvzlL+P222/HunXrcNFFF1V60wiCIAiCIIgapi6K5dNPPx1btmzBT37yE2zatAnTpk3D\n3/72N0yZMkXq8dFoFNddd11RawbhD7TPyw/t8/JD+7z80D4vP7TPiXqn5qPjCIIgCIIgCMIvat6z\nTBAEQRAEQRB+QcUyQRAEQRAEQdhAxTJBEARBEARB2EDFMkEQBEEQBEHYMOSL5dtuuw1Tp05FQ0MD\n9tlnH7z44ouV3qS6Yf78+dh3333R2tqKsWPH4qSTTsJ7772Xcx/TNDFv3jxMmDABjY2NmDlzJt59\n990KbXH9MX/+fBiGgcsvv5zfRvtcP5988gm+8Y1vYNSoUWhqasKee+6JN998k/+e9rleUqkU/vu/\n/xtTp05FY2MjdthhB/zkJz9BJpPh96F97p2lS5fi+OOPx4QJE2AYBh5//PGc38vs43g8jssuuwyj\nR49Gc3MzTjjhBGzYsKGMr4IgvDOki+WHHnoIl19+OX7wgx/grbfewle+8hXMnj0b69atq/Sm1QVL\nlizBJZdcgldffRWLFi1CKpXCrFmz0NfXx+9zww034KabbsKtt96KZcuWob29HUceeSR6enoquOX1\nwbJly3D77bfjS1/6Us7ttM/1snXrVhx00EEIh8N46qmnsGrVKtx4440YPnw4vw/tc7384he/wO9/\n/3vceuut+Ne//oUbbrgBv/zlL3HLLbfw+9A+905fXx/22GMP3HrrrUV/L7OPL7/8cjz22GN48MEH\n8dJLL6G3txfHHXcc0ul0uV4GQXjHHMLst99+5kUXXZRz2y677GJec801Fdqi+qazs9MEYC5ZssQ0\nTdPMZDJme3u7ef311/P7DAwMmG1tbebvf//7Sm1mXdDT02PutNNO5qJFi8xDDz3U/O53v2uaJu1z\nP7j66qvNgw8+2Pb3tM/1c+yxx5rnnXdezm2nnHKK+Y1vfMM0TdrnfgDAfOyxx/jPMvt427ZtZjgc\nNh988EF+n08++cQMBALm008/XbZtJwivDFllOZFI4M0338SsWbNybp81axZefvnlCm1VfdPV1QUA\nGDlyJABgzZo16OjoyHkPotEoDj30UHoPPHLJJZfg2GOPxRFHHJFzO+1z/SxcuBAzZszAf/zHf2Ds\n2LHYa6+98Ic//IH/nva5fg4++GA899xzeP/99wEAb7/9Nl566SUcc8wxAGiflwOZffzmm28imUzm\n3GfChAmYNm0avQ9ETVEXE/zcsHnzZqTTaYwbNy7n9nHjxqGjo6NCW1W/mKaJuXPn4uCDD8a0adMA\ngO/nYu/Bxx9/XPZtrBcefPBBLF++HMuWLSv4He1z/Xz00Uf43e9+h7lz5+L73/8+Xn/9dXznO99B\nNBrFOeecQ/vcB66++mp0dXVhl112QTAYRDqdxs9+9jOceeaZAOhzXg5k9nFHRwcikQhGjBhRcB86\nzxK1xJAtlhmGYeT8bJpmwW2Edy699FL885//xEsvvVTwO3oP9LF+/Xp897vfxd///nc0NDTY3o/2\nuT4ymQxmzJiBn//85wCAvfbaC++++y5+97vf4ZxzzuH3o32uj4ceegj33nsv7r//fuy+++5YsWIF\nLr/8ckyYMAFz5szh96N97j9u9jG9D0StMWRtGKNHj0YwGCy4uu3s7Cy4Uia8cdlll2HhwoV44YUX\nMHHiRH57e3s7ANB7oJE333wTnZ2d2GeffRAKhRAKhbBkyRL87//+L0KhEN+vtM/1MX78eOy22245\nt+266668UZg+5/r53ve+h2uuuQZnnHEGpk+fjrPPPhv/9V//hfnz5wOgfV4OZPZxe3s7EokEtm7d\nansfgqgFhmyxHIlEsM8++2DRokU5ty9atAgHHnhghbaqvjBNE5deeikeffRRPP/885g6dWrO76dO\nnYr29vac9yCRSGDJkiX0Hrjk8MMPx8qVK7FixQr+b8aMGTjrrLOwYsUK7LDDDrTPNXPQQQcVRCK+\n//77mDJlCgD6nPtBf38/AoHc01cwGOTRcbTP/UdmH++zzz4Ih8M599m0aRPeeecdeh+I2qJirYVV\nwIMPPmiGw2HzjjvuMFetWmVefvnlZnNzs7l27dpKb1pd8O1vf9tsa2szFy9ebG7atIn/6+/v5/e5\n/vrrzba2NvPRRx81V65caZ555pnm+PHjze7u7gpueX0hpmGYJu1z3bz++utmKBQyf/azn5mrV682\n77vvPrOpqcm89957+X1on+tlzpw55nbbbWf+9a9/NdesWWM++uij5ujRo82rrrqK34f2uXd6enrM\nt956y3zrrbdMAOZNN91kvvXWW+bHH39smqbcPr7ooovMiRMnms8++6y5fPly86tf/aq5xx57mKlU\nqlIviyCUGdLFsmma5m9/+1tzypQpZiQSMffee28ea0Z4B0DRf3feeSe/TyaTMa+77jqzvb3djEaj\n5iGHHGKuXLmychtdh+QXy7TP9fPEE0+Y06ZNM6PRqLnLLruYt99+e87vaZ/rpbu72/zud79rTp48\n2WxoaDB32GEH8wc/+IEZj8f5fWife+eFF14o+h0+Z84c0zTl9nEsFjMvvfRSc+TIkWZjY6N53HHH\nmevWravAqyEI9ximaZqV0bQJgiAIgiAIoroZsp5lgiAIgiAIgnCCimWCIAiCIAiCsIGKZYIgCIIg\nCIKwgYplgiAIgiAIgrCBimWCIAiCIAiCsIGKZYIgCIIgCIKwgYplgiAIgiAIgrCBimWCIAiCIAiC\nsIGKZYIgaoZ58+Zhzz33LPvzLl68GIZhYNu2bWV/boIgCKKy0AQ/giCqAsMwSv5+zpw5uPXWWxGP\nxzFq1KgybVWWRCKBzz//HOPGjXPcToIgCKK+oGKZIIiqoKOjg///Qw89hB/96Ed47733+G2NjY1o\na2urxKYRBEEQQxiyYRAEURW0t7fzf21tbTAMo+C2fBvGueeei5NOOgk///nPMW7cOAwfPhw//vGP\nkUql8L3vfQ8jR47ExIkTsWDBgpzn+uSTT3D66adjxIgRGDVqFE488USsXbvWdtvybRh33XUXhg8f\njmeeeQa77rorWlpacPTRR2PTpk22f2Pr1q0466yzMGbMGDQ2NmKnnXbCnXfe6WWXEQRBEGWAimWC\nIGqa559/Hhs3bsTSpUtx0003Yd68eTjuuOMwYsQIvPbaa7joootw0UUXYf369QCA/v5+HHbYYWhp\nacHSpUvx0ksv8WI3kUhIP29/fz9+9atf4Z577sHSpUuxbt06XHnllbb3/+EPf4hV/7+9+3eFPw7g\nOP70uS/DFYm7k0XKoIuUH5Nyi7oyEYuyXDZRJsMZDCbJZGFQJnU3KrIY/MjIhuUkMvkDUOr6bvdN\nX59vV+f4yvOxvT99er8+7+3V+/N597m64uDggOvrazY2NojFYhWvX5JUXb+++gEkqRJNTU2sr68T\nBAGdnZ2srq7y9PTE4uIiANlslpWVFc7OzpicnCSXyxEEAVtbW6Xvj7e3t2lsbOTo6Ih0Ol1W7uvr\nK5ubm3R0dAAwNzfH8vJy6P339/f09vYyMDAAQHt7ewWrliR9FsuypG+tq6uLIPjzkqylpYXu7u7S\nOBKJ0NzczOPjIwDn5+cUCgXq6+vfzPPy8sLNzU3ZudFotFSUAVpbW0sZ75mZmWFiYoKLiwvS6TRj\nY2MMDg6WnSdJ+hqWZUnfWm1t7ZtxTU3Nu9eKxSIAxWKR/v5+dnZ2/porHo9XlPuv89IjIyPc3d2x\nv7/P4eEhw8PDzM7Osra2VnamJOnzWZYl/Sh9fX3k83kSiQQNDQ2fmh2Px8lkMmQyGYaGhlhYWLAs\nS9J/zgN+kn6UqakpYrEYo6OjnJ6ecnt7y/HxMfPz8zw8PFQtd2lpid3dXQqFApeXl+zt7ZFMJquW\nJ0n6GJZlST9KNBrl5OSEtrY2xsfHSSaTTE9P8/z8XNWd5rq6OrLZLD09PaRSKSKRCLlcrmp5kqSP\n4U9JJEmSpBDuLEuSJEkhLMuSJElSCMuyJEmSFMKyLEmSJIWwLEuSJEkhLMuSJElSCMuyJEmSFMKy\nLEmSJIWwLEuSJEkhLMuSJElSCMuyJEmSFOI3FwFkCxzjLskAAAAASUVORK5CYII=\n"
+    },
+    "d1a3d796-81df-4a8f-90d5-9324e9d1781d.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtQAAAHTCAYAAAAHwI/VAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90\nbGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9h\nAAAPYQGoP6dpAAB/T0lEQVR4nO3dd3xT9foH8M9J2qa7pS1ddAKlQKFQpjJkyBQUBQeKCqJcVFCR\n61URrhTFolxFr6K4LsOfIipDEZUlW1Rm2ZsWyuiC7pE0yff3R5qU0JamTdKMft6vV1/ak5Ocb05C\n8+Q5z/f5SkIIASIiIiIiahCZrQdAREREROTIGFATEREREZmBATURERERkRkYUBMRERERmYEBNRER\nERGRGRhQExERERGZgQE1EREREZEZGFATEREREZmBATURERERkRkYUJNTOXz4MJ588km0atUKHh4e\n8PDwQFxcHCZPnox9+/YZ7ZucnAxJkgw/MpkMYWFhuOuuu/DHH3+YdLyYmBhIkoT+/fvXePtXX31l\nePxt27aZ+ezIVKWlpUhOTrbKOd+2bVutr+fhw4chSRIOHjxo8eOaa8KECYiJibH1MIiInJKLrQdA\nZCmfffYZpk6divj4eLzwwgtISEiAJEk4ceIEvv32W3Tv3h1nz55Fq1atjO63fv16+Pn5QavV4uLF\ni5g/fz769++Pv//+G126dKnzuD4+PtixYwfOnTtX7bEXL14MX19fFBYWWvS50q2VlpZizpw5AFDr\nlx1rWLVqFWJjY5GUlNRoxyQiIttjhpqcwh9//IFnn30Ww4cPx4EDB/D888/jzjvvxMCBAzFlyhTs\n2rUL33//PTw8PKrdt2vXrrjtttvQq1cvjB07FitXroRarcbKlStNOnafPn3QokULLF682Gj7uXPn\nsGPHDjz00EMWeY7WVlZWZushOLyVK1dizJgxth6G3RNC8P1GRE6FATU5hZSUFMjlcnz22Wdwc3Or\ncZ8HHngA4eHhdT6Wn58fAMDV1dWkY8tkMjz++ONYtmwZtFqtYfvixYsRGRmJQYMG1Xi/ffv24Z57\n7kFAQADc3d2RlJSE77//3mifpUuXQpIkbNmyBZMmTUJgYCB8fX3x+OOPo6SkBJmZmXjwwQfh7++P\nsLAwvPTSS6ioqKhzzDExMRg5ciRWr16NpKQkuLu7GzK6mZmZmDx5MiIiIuDm5obY2FjMmTMHarXa\n6DGuXLmCBx98ED4+PvDz88NDDz2Ev/76C5IkYenSpYb9+vfvX2OWuKYSBJVKhblz56Jt27ZQKBRo\n3rw5nnjiCeTk5Bjtt2XLFvTv3x+BgYHw8PBAVFQUxowZg9LSUqSnp6N58+YAgDlz5hhKbiZMmGC4\n/5kzZ/DII48gODgYCoUC7dq1w8cff1xtjCdPnsSwYcPg6emJoKAgPP300ygqKqrxnJ48eRLHjx83\nBNS1lYakp6dXO0fnz5/H2LFjER4eDoVCgZCQENx5551ITU01uu93332H22+/HV5eXvD29sbQoUNr\nLC9ZunQp4uPjDc/tq6++qnHMNdG/N9asWYPExES4u7ujZcuW+PDDD6vtW1hYiJdeegmxsbFwc3ND\nixYtMG3aNJSUlBjtJ0kSpk6dik8//RTt2rWDQqHAsmXLah3DrV5fPVPfKwCwfPly3H777fD29oa3\ntzc6d+6M//3vfyafEyKiurDkgxyeRqPB1q1b0a1bN4SFhTXo/mq12lDyMWvWLCgUCtx///0mP8bE\niRMxb948bNiwAcOHD4dGo8GyZcvw5JNPQiar/r1169atGDZsGHr27IlPP/0Ufn5+WLFiBR566CGU\nlpYaBX8A8NRTT2H06NFYsWIFDh48iNdeew1qtRqnTp3C6NGj8Y9//AObN2/GO++8g/DwcEyfPr3O\nMR84cAAnTpzArFmzEBsbCy8vL2RmZqJHjx6QyWR4/fXX0apVK/z555+YO3cu0tPTsWTJEgC6bPag\nQYNw5coVzJs3D23atMEvv/xiVjZeq9Vi1KhR2LlzJ15++WX06tULFy5cwOzZs9G/f3/s27cPHh4e\nSE9Px4gRI9C3b18sXrwY/v7+uHz5MtavXw+VSoWwsDCsX78ew4YNw5NPPomnnnoKAAxB9vHjx9Gr\nVy9ERUXhvffeQ2hoKDZs2IDnn38eubm5mD17NgAgKysL/fr1g6urKz755BOEhITgm2++wdSpU2sc\n/6pVq9CiRQv07Nmz3s/9rrvugkajwfz58xEVFYXc3Fzs3r0b+fn5hn1SUlIwa9YsPPHEE5g1axZU\nKhX+85//oG/fvtizZw/at28PQBdMP/HEExg1ahTee+89FBQUIDk5GUqlssb3Yk1SU1Mxbdo0JCcn\nIzQ0FN988w1eeOEFqFQqvPTSSwB0ZTX9+vXDpUuX8NprryExMRHHjh3D66+/jiNHjmDz5s2QJMnw\nmD/++CN27tyJ119/HaGhoQgODq7x2HW9vp6enia/VwDg9ddfx5tvvonRo0fjn//8J/z8/HD06FFc\nuHCh3q8TEVGtBJGDy8zMFADE2LFjq92mVqtFRUWF4Uer1Rpumz17tgBQ7cfX11esXr3apGNHR0eL\nESNGCCGE6Nevn7j//vuFEEL88ssvQpIkkZaWJn744QcBQGzdutVwv7Zt24qkpCRRUVFh9HgjR44U\nYWFhQqPRCCGEWLJkiQAgnnvuOaP97r33XgFALFiwwGh7586dRZcuXUwat1wuF6dOnTLaPnnyZOHt\n7S0uXLhgtP3dd98VAMSxY8eEEEIsWrRIABA//fST0X6TJk0SAMSSJUsM2/r16yf69etXbQzjx48X\n0dHRht+//fZbAUCsWrXKaL+9e/cKAOKTTz4RQgixcuVKAUCkpqbW+vxycnIEADF79uxqtw0dOlRE\nRESIgoICo+1Tp04V7u7u4vr160IIIV555RUhSVK14wwePLja6ymE7tzf+Dpt3bq1xv3S0tKMzlFu\nbq4AID744INan8/FixeFi4tLtfdBUVGRCA0NFQ8++KAQQgiNRiPCw8NFly5djN7r6enpwtXV1eh8\n1yY6OrrW5+3r6ytKSkqEEELMmzdPyGQysXfvXqP99K/Pr7/+atgGQPj5+RnO7a2Y8vqa+l45f/68\nkMvlYty4cXUel4jIHCz5IKfWtWtXuLq6Gn7ee++9avts3rwZe/fuxZ49e7Bu3ToMGjQIY8eOxZo1\na+p1rIkTJ2Lt2rW4du0a/ve//2HAgAE1dlU4e/YsTp48iXHjxgEA1Gq14eeuu+7C1atXcerUKaP7\njBw50uj3du3aAQBGjBhRbbupmbfExES0adPGaNu6deswYMAAhIeHG41r+PDhAIDt27cD0GXYfXx8\ncM899xjd/5FHHjHp2DVZt24d/P39cffddxsdu3PnzggNDTWUTnTu3Blubm74xz/+gWXLluH8+fMm\nH6O8vBy///477rvvPnh6elY79+Xl5fjrr78MzzEhIQGdOnWq8zmeP38eqampDaqfDggIQKtWrfCf\n//wHCxYswMGDB41KhwBgw4YNUKvVePzxx43G7O7ujn79+hnOzalTp3DlyhU88sgjRtnh6Oho9OrV\ny+Qx1fa8CwsLceDAAQC616tDhw7o3Lmz0ZiGDh1aY6nLwIED0axZszqPbcrra+p7ZdOmTdBoNJgy\nZYrJz52IqCEYUJPDCwoKgoeHR42B5PLly7F3716sXbu21vt36tQJ3bp1Q/fu3TFixAj88MMPaN26\ndb0/hO+//364u7vj/fffx88//4wnn3yyxv2ysrIAAC+99JJRsO/q6opnn30WAJCbm2t0n4CAAKPf\n9XXiNW0vLy83abw1lcdkZWXh559/rjauhIQEo3Fdu3YNISEh1e4fGhpq0rFrkpWVhfz8fLi5uVU7\nfmZmpuHYrVq1wubNmxEcHIwpU6agVatWaNWqFf773//WeYxr165BrVbjo48+qnaMu+66q9pzrOn5\n1LRt5cqVCA4ORp8+fer9vCVJwu+//46hQ4di/vz56NKlC5o3b47nn3/eUK+tf89079692ri/++47\nozHXNsb6vDa3ur/+GFlZWTh8+HC18fj4+EAIUe09bGo5limvr6nvFX09dUREhMnPnYioIVhDTQ5P\nLpdj4MCB2LhxI65evWr0wa2vK01PTzf58WQyGRISEvDDDz8gOzu71lrPm3l6emLs2LGYN28efH19\nMXr06Br3CwoKAgDMmDGj1n3i4+NNHm9D3ZjB1AsKCkJiYiLeeuutGu+jn9QZGBiIPXv2VLs9MzOz\n2jZ3d3cUFBRU235zwBUUFITAwECsX7++xmP7+PgY/r9v377o27cvNBoN9u3bh48++gjTpk1DSEgI\nxo4dW+P9AaBZs2aQy+V47LHHav3CFBsba3iONT2fmratWrUK9957L+RyuWGbu7s7AECpVBrte/Pz\nBnQZZP0kudOnT+P7779HcnIyVCoVPv30U8N7ZuXKlYiOjq71+QUGBtY6xpq21eZW99cfQ/9F9ubu\nNnr6MevV9H6rTV2vr6nvFX3d/KVLlxAZGWny8YmI6osBNTmFGTNm4LfffsPTTz+NlStXmtyhoyYa\njQZHjhyBQqGAr69vve77zDPPGCaz6QOqm8XHxyMuLg6HDh1CSkpKg8dpDSNHjsSvv/6KVq1a3fLy\n/IABA/D9999j7dq1RmUfy5cvr7ZvTEwMfvjhByiVSigUCgC6LOfu3buNzu/IkSOxYsUKaDQakyf2\nyeVy9OzZE23btsU333yDAwcOYOzYsYbj3NyazdPTEwMGDMDBgweRmJhYa0cY/XOcP38+Dh06ZFT+\ncPNzzMjIwN69e/Hmm29We96AbrGXoUOHGrbf6moJALRp0wazZs3CqlWrDOUVQ4cOhYuLC86dO3fL\nspL4+HiEhYXh22+/xfTp0w1B7IULF7B7926TutwAwLFjx2p83j4+Pobe7CNHjkRKSgoCAwMNX0Is\nrbbX19T3ypAhQyCXy7Fo0SLcfvvtVhkjERHAgJqcRO/evfHxxx/jueeeQ5cuXfCPf/wDCQkJkMlk\nuHr1KlatWgUANQbI+/fvN7TKy8rKwuLFi3Hy5Em8+OKLtQbFtencuTN+/PHHOvf77LPPMHz4cAwd\nOhQTJkxAixYtcP36dZw4cQIHDhzADz/8UK/jWsobb7yBTZs2oVevXnj++ecRHx+P8vJypKen49df\nf8Wnn36KiIgIPP7443j//ffx+OOP46233kJcXBx+/fVXbNiwodpjPvbYY/jss8/w6KOPYtKkSbh2\n7Rrmz59f7bUYO3YsvvnmG9x111144YUX0KNHD7i6uuLSpUvYunUrRo0ahfvuuw+ffvoptmzZghEj\nRiAqKgrl5eWGLKm+RaGPjw+io6Px008/4c4770RAQACCgoIQExOD//73v+jTpw/69u2LZ555BjEx\nMSgqKsLZs2fx888/Y8uWLQCAadOmYfHixRgxYgTmzp1r6PJx8uRJo3GvWrUK/v7+GDBggNH20NBQ\nDBo0CPPmzUOzZs0QHR2N33//HatXrzba7/Dhw5g6dSoeeOABxMXFwc3NDVu2bMHhw4fx6quvAtAF\n52+88QZmzpyJ8+fPY9iwYWjWrBmysrKwZ88eeHl5Yc6cOZDJZHjzzTfx1FNP4b777sOkSZOQn59v\n6NZhqvDwcNxzzz1ITk5GWFgYvv76a2zatAnvvPMOPD09Dedn1apVuOOOO/Diiy8iMTHR0Cln48aN\n+Oc//9mgjiemvL6mvldiYmLw2muv4c0330RZWRkefvhh+Pn54fjx48jNzTW0iiQiMputZ0USWVJq\naqp44oknRGxsrFAoFMLd3V20bt1aPP744+L333832remLh8BAQGiZ8+eYvHixYZOG7dyY5eP2tTU\n5UMIIQ4dOiQefPBBERwcLFxdXUVoaKgYOHCg+PTTTw376Lt83NxJQT/2nJwco+3jx48XXl5eZo07\nJydHPP/88yI2Nla4urqKgIAA0bVrVzFz5kxRXFxs2O/SpUtizJgxwtvbW/j4+IgxY8aI3bt3V+vy\nIYQQy5YtE+3atRPu7u6iffv24rvvvqvW5UMIISoqKsS7774rOnXqJNzd3YW3t7do27atmDx5sjhz\n5owQQog///xT3HfffSI6OlooFAoRGBgo+vXrJ9auXWv0WJs3bxZJSUlCoVAIAGL8+PGG29LS0sTE\niRNFixYthKurq2jevLno1auXmDt3rtFjHD9+XAwePFi4u7uLgIAA8eSTT4qffvrJ6PXs06eP0WPf\n6OrVq+L+++8XAQEBws/PTzz66KNi3759RucoKytLTJgwQbRt21Z4eXkJb29vkZiYKN5//32hVquN\nHu/HH38UAwYMEL6+vkKhUIjo6Ghx//33i82bNxvt9+WXX4q4uDjh5uYm2rRpIxYvXlzj+a6J/r2x\ncuVKkZCQINzc3ERMTEy1jjJCCFFcXCxmzZol4uPjhZubm/Dz8xMdO3YUL774osjMzDTsB0BMmTKl\nzmMLYfrra8p7Re+rr74S3bt3N+yXlJRU7T1KRGQOSQghbBPKE5GzSU9PR2xsLJYsWVKtl7YzyszM\nRIsWLfDjjz/i7rvvtvVwLCImJgYdOnTAunXrbD0UIiKHwZIPIqIGCg0NhUajsfUwiIjIxtg2j4iI\niIjIDCz5ICIiIiIyAzPURERERERmYEBNRERERGQGBtRERERERGZw+i4fWq0WV65cgY+PT72WviUi\nIiLHJYRAUVERwsPDIZMxf0jW5fQB9ZUrVxAZGWnrYRAREZENZGRkICIiwtbDICfn9AG1j48PAN0/\nqJqWnSYiIiLnU1hYiMjISEMcQGRNTh9Q68s8fH19GVATERE1MSz3pMbAoiIiIiIiIjMwoCYiIiIi\nMgMDaiIiIiIiMzCgJiIiIiIyAwNqIiIiIiIzMKAmIiIiIjIDA2oiIiIiIjMwoCYiIiIiMgMDaiIi\nIiIiMzCgJiIiIiIyAwNqIiIiIiIzMKAmIiIiIjIDA2oiIiIiIjMwoCYiIiIiMgMDaiIiIiIiMzCg\nJiIiIiIyAwNqIiIiIiIzMKAmIiIiIjIDA2oiIiIiIjMwoCYiIiIiMgMDaiIiIiIiMzCgJiIiIiIy\nAwNqIiIiIiIzMKAmIiIiIjIDA2oiIiIiIjMwoCYiIiIiMgMDaiIiIiIiMzCgJiIiIiIyAwNqIiIi\nIiIzMKAmIiIiIjKDi60H0GhUJYBKbutREBEROR5XT0CSbD0KIrvVdALq9+IBBf8YEBER1dtrVwA3\nL1uPgshuseSDiIiIiMgMTSdD/c9TgK+vrUdBRETkeFw9bT0CIrvWdAJqNy9eriIiIiIii2PJBxER\nERGRGRhQExERERGZgQE1EREREZEZGFATEREREZmBATURERERkRkYUBMRERERmYEBNRERERGRGWwe\nUF++fBmPPvooAgMD4enpic6dO2P//v2G24UQSE5ORnh4ODw8PNC/f38cO3bMhiMmIiIiIqpi04A6\nLy8PvXv3hqurK3777TccP34c7733Hvz9/Q37zJ8/HwsWLMDChQuxd+9ehIaGYvDgwSgqKrLdwImI\niIiIKklCCGGrg7/66qv4448/sHPnzhpvF0IgPDwc06ZNwyuvvAIAUCqVCAkJwTvvvIPJkyfXeYzC\nwkL4+fmhoKAAvlx6nIiIqEng5z81JptmqNeuXYtu3brhgQceQHBwMJKSkvDFF18Ybk9LS0NmZiaG\nDBli2KZQKNCvXz/s3r27xsdUKpUoLCw0+iEiIiIishabBtTnz5/HokWLEBcXhw0bNuDpp5/G888/\nj6+++goAkJmZCQAICQkxul9ISIjhtpvNmzcPfn5+hp/IyEjrPgkiIiIiatJsGlBrtVp06dIFKSkp\nSEpKwuTJkzFp0iQsWrTIaD9Jkox+F0JU26Y3Y8YMFBQUGH4yMjKsNn4iIiIiIpsG1GFhYWjfvr3R\ntnbt2uHixYsAgNDQUAColo3Ozs6ulrXWUygU8PX1NfohIiIiIrIWmwbUvXv3xqlTp4y2nT59GtHR\n0QCA2NhYhIaGYtOmTYbbVSoVtm/fjl69ejXqWImIiIiIauJiy4O/+OKL6NWrF1JSUvDggw9iz549\n+Pzzz/H5558D0JV6TJs2DSkpKYiLi0NcXBxSUlLg6emJRx55xJZDJyIiIiICYOOAunv37lizZg1m\nzJiBN954A7Gxsfjggw8wbtw4wz4vv/wyysrK8OyzzyIvLw89e/bExo0b4ePjY8ORExERERHp2LQP\ndWNgH0oiIqKmh5//1JhsvvQ4EREREZEjY0BNRERERGQGBtRERERERGZgQE1EREREZAYG1ERERERE\nZmBATURERERkBgbURERERERmYEBNRERERGQGBtRERERERGZgQE1EREREZAYG1EREREREZmBATURE\nRERkBgbURERERERmYEBNRERERGQGBtRERERERGZgQE1EREREZAYG1EREREREZmBATURERERkBgbU\nRERERERmYEBNRERERGQGBtRERERERGZgQE1EREREZAYG1EREREREZmBATURERERkBgbURERERERm\nYEBNRERERGQGBtRERERERGZgQE1EREREZAYG1EREREREZmBATURERERkBgbURERERERmYEBNRERE\nRGQGBtRERERERGZgQE1EREREZAYG1EREREREZmBATURERERkBgbURERERERmYEBNRERERGQGBtRE\nRERERGZgQE1EREREZAYG1EREREREZmBATURERERkBgbURERERERmYEBNRERERGQGBtRERERERGaw\naUCdnJwMSZKMfkJDQw23CyGQnJyM8PBweHh4oH///jh27JgNR0xEREREZMzmGeqEhARcvXrV8HPk\nyBHDbfPnz8eCBQuwcOFC7N27F6GhoRg8eDCKiopsOGIiIiIioiouNh+Ai4tRVlpPCIEPPvgAM2fO\nxOjRowEAy5YtQ0hICJYvX47JkyfX+HhKpRJKpdLwe2FhoXUGTkREREQEO8hQnzlzBuHh4YiNjcXY\nsWNx/vx5AEBaWhoyMzMxZMgQw74KhQL9+vXD7t27a328efPmwc/Pz/ATGRlp9edARERERE2XTQPq\nnj174quvvsKGDRvwxRdfIDMzE7169cK1a9eQmZkJAAgJCTG6T0hIiOG2msyYMQMFBQWGn4yMDKs+\nByIiIiJq2mxa8jF8+HDD/3fs2BG33347WrVqhWXLluG2224DAEiSZHQfIUS1bTdSKBRQKBTWGTAR\nERER0U1sXvJxIy8vL3Ts2BFnzpwx1FXfnI3Ozs6ulrUmIiIiIrIVuwqolUolTpw4gbCwMMTGxiI0\nNBSbNm0y3K5SqbB9+3b06tXLhqMkIiIiIqpi05KPl156CXfffTeioqKQnZ2NuXPnorCwEOPHj4ck\nSZg2bRpSUlIQFxeHuLg4pKSkwNPTE4888ogth01EREREZGDTgPrSpUt4+OGHkZubi+bNm+O2227D\nX3/9hejoaADAyy+/jLKyMjz77LPIy8tDz549sXHjRvj4+Nhy2EREREREBpIQQth6ENZUWFgIPz8/\nFBQUwNfX19bDISIiokbAz39qTHZVQ01ERERE5GgYUBMRERERmYEBNRERERGRGRhQExERERGZgQE1\nEREREZEZGFATEREREZmBATURERERkRkYUBMRERERmYEBNRERERGRGRhQExERERGZgQE1EREREZEZ\nGFATEREREZmBATURERERkRkYUBMRERERmYEBNRERERGRGRhQExERERGZgQE1EREREZEZGFATERER\nEZmBATURERERkRkYUBMRERERmYEBNRERERGRGRhQExERERGZgQE1EREREZEZGFATEREREZmBATUR\nERERkRkYUBMRERERmYEBNRERERGRGRhQExERERGZgQE1EREREZEZGFATEREREZmBATURERERkRkY\nUBMRERERmYEBNRERERGRGRhQExERERGZgQE1EREREZEZGFATEREREZmBATURERERkRkYUBMRERER\nmYEBNRERERGRGRhQExERERGZgQE1EREREZEZGFATEREREZmBATURERERkRkYUBMRERERmcFuAup5\n8+ZBkiRMmzbNsE0IgeTkZISHh8PDwwP9+/fHsWPHbDdIIiIiIqKb2EVAvXfvXnz++edITEw02j5/\n/nwsWLAACxcuxN69exEaGorBgwejqKjIRiMlIiIiIjJm84C6uLgY48aNwxdffIFmzZoZtgsh8MEH\nH2DmzJkYPXo0OnTogGXLlqG0tBTLly+34YiJiIiIiKrYPKCeMmUKRowYgUGDBhltT0tLQ2ZmJoYM\nGWLYplAo0K9fP+zevbvWx1MqlSgsLDT6ISIiIiKyFhdbHnzFihU4cOAA9u7dW+22zMxMAEBISIjR\n9pCQEFy4cKHWx5w3bx7mzJlj2YESEREREdXCZhnqjIwMvPDCC/j666/h7u5e636SJBn9LoSotu1G\nM2bMQEFBgeEnIyPDYmMmIiIiIrqZzTLU+/fvR3Z2Nrp27WrYptFosGPHDixcuBCnTp0CoMtUh4WF\nGfbJzs6ulrW+kUKhgEKhsN7AiYiIiIhuYLMM9Z133okjR44gNTXV8NOtWzeMGzcOqampaNmyJUJD\nQ7Fp0ybDfVQqFbZv345evXrZathEREREREZslqH28fFBhw4djLZ5eXkhMDDQsH3atGlISUlBXFwc\n4uLikJKSAk9PTzzyyCO2GDIRERERUTUNCqj37NmDbdu2ITs7G1qt1ui2BQsWWGRgAPDyyy+jrKwM\nzz77LPLy8tCzZ09s3LgRPj4+FjsGEREREZE5JCGEqM8dUlJSMGvWLMTHxyMkJMRogqAkSdiyZYvF\nB2mOwsJC+Pn5oaCgAL6+vrYeDhERETUCfv5TY6p3hvq///0vFi9ejAkTJlhhOEREREREjqXekxJl\nMhl69+5tjbEQERERETmcegfUL774Ij7++GNrjIWIiIiIyOHUu+TjpZdewogRI9CqVSu0b98erq6u\nRrevXr3aYoMjIiIiIrJ39Q6on3vuOWzduhUDBgxAYGDgLVctJCIiIiJydvUOqL/66iusWrUKI0aM\nsMZ4iIiIiIgcSr1rqAMCAtCqVStrjIWIiIiIyOHUO6BOTk7G7NmzUVpaao3xEBERERE5lHqXfHz4\n4Yc4d+4cQkJCEBMTU21S4oEDByw2OCIiIiIie1fvgPree++1wjCIiIiIiBxTvZcedzRcepSIiKjp\n4ec/NaZ611ADQH5+Pr788kvMmDED169fB6Ar9bh8+bJFB0dEREREZO/qXfJx+PBhDBo0CH5+fkhP\nT8ekSZMQEBCANWvW4MKFC/jqq6+sMU4iIiIiIrtU7wz19OnTMWHCBJw5cwbu7u6G7cOHD8eOHTss\nOjgiIiIiIntX74B67969mDx5crXtLVq0QGZmpkUGRURERETkKOodULu7u6OwsLDa9lOnTqF58+YW\nGRQRERERkaOod0A9atQovPHGG6ioqAAASJKEixcv4tVXX8WYMWMsPkAiIiIiIntW74D63XffRU5O\nDoKDg1FWVoZ+/fqhdevW8PHxwVtvvWWNMRIRERER2a16d/nw9fXFrl27sGXLFhw4cABarRZdunTB\noEGDrDE+IiIiIiK7xoVdiIiIyOnw858ak8kZalP7Sz/++OMNHgwRERERkaMxOUMtk8ng7e0NFxcX\n1HYXSZIMKyfaC35DJSIianr4+U+NyeQMdbt27ZCVlYVHH30UEydORGJiojXHRURERETkEEzu8nHs\n2DH88ssvKCsrwx133IFu3bph0aJFNfakJiIiIiJqKurVNq9nz5747LPPcPXqVTz//PP4/vvvERYW\nhnHjxkGpVFprjEREREREdqvefagBwMPDA48//jjmzJmDHj16YMWKFSgtLbX02IiIiIiI7F69A+rL\nly8jJSUFcXFxGDt2LLp3745jx46hWbNm1hgfEREREZFdM3lS4vfff48lS5Zg+/btGDp0KN577z2M\nGDECcrncmuMjIiIiIrJr9WqbFxUVhXHjxiEkJKTW/Z5//nmLDc4S2DaHiIio6eHnPzUmkwPqmJgY\nSJJ06weTJJw/f94iA7MU/oMiIiJqevj5T43J5JKP9PR0Kw6DiIiIiMgxNajLBxERERER6TCgJiIi\nIiIyAwNqIiIiIiIzMKAmIiIiIjIDA2oiIiIiIjOY3OWjJkIIbN26FWVlZejVqxdXSyQiIiKiJsfk\nDHV+fj7Gjx+Pjh07YtKkSSgsLETfvn0xaNAg3H333Wjbti0OHz5szbESEREREdkdkwPql156CX/+\n+SceeughHDlyBMOGDYNGo8Gff/6Jv//+G+3bt8fMmTOtOVYiIiIiIrtj8kqJLVq0wPLly9GvXz9c\nvnwZkZGR2LJlC/r37w8A2LNnD+655x5kZmZac7z1xpWSiIiImh5+/lNjMjlDnZWVhTZt2gDQBdfu\n7u6IjIw03B4VFYWcnBzLj5CIiIiIyI6ZHFBrtVrI5XLD73K5HJIkGX6/8f+JiIiIiJqKenX5+PLL\nL+Ht7Q0AUKvVWLp0KYKCggAARUVFlh8dEREREZGdM7mGOiYmxqQsdFpamtmDsiTWUBERETU9/Pyn\nxmRyhjo9Pd2KwyAiIiIickw2XSlx0aJFSExMhK+vL3x9fXH77bfjt99+M9wuhEBycjLCw8Ph4eGB\n/v3749ixYzYcMRERERGRMZMz1GVlZfj9998xcuRIAMCMGTOgVCoNt8vlcrz55ptwd3c3+eARERF4\n++230bp1awDAsmXLMGrUKBw8eBAJCQmYP38+FixYgKVLl6JNmzaYO3cuBg8ejFOnTsHHx8fk4xAR\nERERWYvJNdSfffYZ1q1bh59//hkA4OPjg4SEBHh4eAAATp48iZdffhkvvviiWQMKCAjAf/7zH0yc\nOBHh4eGYNm0aXnnlFQCAUqlESEgI3nnnHUyePNmkx2MNFRERUdPDz39qTCaXfHzzzTeYOHGi0bbl\ny5dj69at2Lp1K/7zn//g+++/b/BANBoNVqxYgZKSEtx+++1IS0tDZmYmhgwZYthHoVCgX79+2L17\nd62Po1QqUVhYaPRDRERkLUIIaLXGP5oafojIeZlc8nH69GnDwi4A4O7uDpmsKh7v0aMHpkyZUu8B\nHDlyBLfffjvKy8vh7e2NNWvWoH379oagOSQkxGj/kJAQXLhwodbHmzdvHubMmVPvcRCRY8stVuK3\no5lQa7S33M9FJmFoQiiCfU0vTyOqzZ/nruGpZXtRotLUue/g9iH44vFujTAqImpsJgfUBQUFcHGp\n2v3mVRG1Wq1RTbWp4uPjkZqaivz8fKxatQrjx4/H9u3bDbff3KpPCHHL9n0zZszA9OnTDb8XFhYa\nrehIRM7pxe9SsfNMrkn7/nH2Gj59rKuVR0TOrlSlxsurDpkUTAPApuNZKK/QwN1VXvfORORQTA6o\nIyIicPToUcTHx9d4++HDhxEREVHvAbi5uRkmJXbr1g179+7Ff//7X0PddGZmJsLCwgz7Z2dnV8ta\n30ihUEChUNR7HETkuPZfyMPOM7m67HOHUNT2lbugrAI7z+Ti+FWWgpH53t90GhnXyxDu545Vz/aC\nu0vNgbIAcFvK71BptMgtViKimWfjDpSIrM7kgPquu+7C66+/jhEjRlTr5FFWVoY5c+ZgxIgRZg9I\nCAGlUonY2FiEhoZi06ZNSEpKAgCoVCps374d77zzjtnHISLn8dGWMwCAMV0i8M79ibXul1VYjp4p\nv+NSXilUai3cXGzaOZQc2OFL+fjfLt1CZm+N7ogwP49b7t/cR4HL+WXIKWJATeSMTA6oX3vtNXz/\n/feIj4/H1KlT0aZNG0iShJMnT2LhwoVQq9V47bXX6nXw1157DcOHD0dkZCSKioqwYsUKbNu2DevX\nr4ckSZg2bRpSUlIQFxeHuLg4pKSkwNPTE4888ki9nygROadDGfnYdioHcpmEZwe0uuW+wT4KeLrJ\nUarS4OL1UrQO9m6kUZIzqdBo8fLKw9AKYFTncAyID67zPkGVAXV2Uf1LI4nI/pkcUIeEhGD37t14\n5pln8Oqrr0LfbU+SJAwePBiffPLJLUsxapKVlYXHHnsMV69ehZ+fHxITE7F+/XoMHjwYAPDyyy+j\nrKwMzz77LPLy8tCzZ09s3LiRPaibiMv5ZZj8f/swOikCE/vE2no4ZKc+2nIWgC6wiQ70uuW+kiQh\nJtALx68WIj23hAE1NcjnO87jZGYRmnm64vWR7U26T7CPrhQxhwE1kVMyOaAGgNjYWKxfvx7Xr1/H\n2bO6D7HWrVsjICCgQQf/3//+d8vbJUlCcnIykpOTG/T45Ng+234ORy8XQqO9xICaanTsSgE2n8iC\nJAFTBrQ26T6xQZUB9bUSK4+OnNH5nGL893ddidG/R7ZHoLdpc3aaM6Amcmr1Cqj1AgIC0KNHD0uP\nhcigqLwCq/ZfAgAUlKpsPBqyVwsrs9N3J4ajVXPTss0xQbr61bRcBtRUP1qtwIzVR6BSa9E3Lgj3\nJbUw+b76DDVLPoicE2fkkF1auf+SoRVVflmFjUdD9uhUZhF+O5oJAJg60LTsNADEVJaFMENN9fXd\nvgz8nXYdHq5ypNzX8ZYtXG/GDDWRc2NATXZHqxVYtjvd8HupSgOl2rQ+r9R0LNyqy07f1TEUbUJM\nn1cRG1QZUOeWWmVc5JyyCsuR8usJAMA/h7RBZED9OnU099YH1OUWHxsR2R4DarI728/kIP1aKXwU\nLpBVJoAKmKWmG5zNLsa6w1cAAFMHxNXrvjGVAfWVgjKUV/CLGplm9k/HUFSuRqcIPzzRu/5zOvQr\nczJDTeScGFCT3Vn6RzoA4MHukfD1cAUA5JcyoKYqn2w9CyF0Szm3D/et130Dvdzgo3CBEMDF68xS\nU93WH72K9ccy4SKTMG90IuQy00s99AwlH8VKQ5csInIeDKjJrpzPKcb20zmQJODx26PRzNMNAANq\nqpKeW4IfUy8DAJ4fWL/sNFDZOq8yS82JiVSXgrIKvP7TMQDA5H4t6/0FTi/IW/e3rEIj+PeMyAkx\noCa78tWfFwAAA+ODER3oBT9DhpqdPkjnk21noRXAgPjm6Bjh16DHiDHUUTOgplt7+7eTyC5SomWQ\nF55rwBc4PYWLHP6eur9nOcUs+yByNgyoyW4UK9VYWdkqb3yvGAAwfACx08etzfn5GPr/ZyuuOfkH\ndcb1Uqw+oMtOP3dnw4ObWGaoyQR/nb+Gb/dcBADMG90R7q5ysx5PPzExu9C5/50SNUUMqMlurNp/\nCcVKNVo290Kf1kEAAP/KDHUBL5HWKqdIia/+vID0a6XYdTbX1sOxqkXbz0GtFegbF4QuUc0a/Dix\n7EVNJnh/02kAwCM9o9CzZaDZjxfsq6+jZqcPImfDgJrsglYrsOzPdADAhF4xkFVO+vGvrKHOY8lH\nrdYeugKNVjfJ6XRWkY1HYz1X8svww74MADDr0jvAXtRUt5wiJfakXwdg+iqcdalqnccMNZGzYUBN\ndmHn2VyczymBt8IFo7tEGLaz5KNuaw5eMvz/qczGCag1WgGttnE7FXy2/RwqNAK3tQxAj9gAsx5L\nX/KRVahEqUptieE1WSq11tZDsIrNJ7IgBNApwg8t/D0s8pj6Th8s+SByPgyoyS7oF3K5v2sEvBUu\nhu0s+bi1M1lFOHq50PD7qUbKUE9cuhc95/3eaJNFswvL8e1eXXa6IZ09bubv6Wb4ssYFXhru2z0X\n0WbWb9h4LNPWQ7E4/SqcQzuEWuwxg30qe1E7+VwHoqaIATXZXHpuCbaeygZQNRlRT1/ykV/Gko+a\nrDmom6DXNVpXT5xxvQzFSutmXE9cLcT20znIKVLir/PXrHosvc92nIdKrUW36Ga4vZX5tawAyz7M\nJYTAlzvPA6gKPq3tu70X0f2tzTh6ucCqxykoq8DuyvkIwxIsF1Bz+XEi58WAmmzuqz8vQAigf3xz\nw6V4PX0WMa+EGeqbabUCP6XqVgt8oncMgis/rM9YOUutD+IB4NAl6wY2AFBUXoHlf+s6LTx3Zxwk\nqf6LatSEnT7MczKzCOdydOfO2gEuoHu//3fzGeQUKQ219Nay5WQW1FqBNiHeaNnc22KPq/83ms2A\nmsjpMKAmmypRqg0fjhNuyk4DVRlqLj1e3d9p13E5vww+ChcMaheC+FAfANato9ZoBX5KrQqoD1/K\nt9qx9H47komyCg1aBnnhjrggiz2uIUPNgLpB9Eu/A8C5nGKUqay7jPtfaddwpUDXHWNPep5Vj7W+\nMuNuyew0wAw1kTNjQE02tfrAJRQp1YgN8sIdcc2r3e7PhV1q9WNlpviujmFwd5UjPqQyoLZihnr3\nuVxkFSoNSy8fziiw+uTElQd0ky7HdI2wWHYaAGIqW+ex5KP+hBBYd/iq4XetAE5kFt7iHub78YYr\nIyczC632JbtUpcb20zkALFs/DVQF1AVlFSivsO4XECJqXAyoyWaEEFhaORnx8dujDa3ybqQv+ShR\naZy2m0BDlFdo8OsRXUBzX5cWAIA2lRlqa7bO05d7PNA1Au6uMhQp1UizYkCacb0Ue9KuQ5KA+5Ja\nWPSxWwbpLuWncVJivR29XIgL10rh7ipDjxhdx5VjViz7KK/Q4Lcjuqyxm4sMQgAHLlgnS73jdA7K\nK7SIDPBA+7CGLTNeGz8PV7jJdR+7uZyYSORUGFCTzew6m4tzOSXwcpPj/q4RNe7j6+4KfVKSExOr\nbD6RhSKlGi38PQwBTVsrl3yUqtSGS+EPdItAQrhu2W9rln3oV0Xs1SoQ4RZqXaanz1DnFitRVM6S\novrQl3sMbBuM7rG6CbHHrlgvQ73peNX7/e7EcAAw9Ii2tBvLPSx5RQQAJEli2QeRk2JATTZzY6s8\nH3fXGveRyST4sXVeNWsqA81RncMNmf24YB9IEpBbrLJK9mvjsSyUqjSICvBEl6hmSIzQBdSHMqyT\nmRRCYHVlj+0xXWr+wmUOH3dXBHnravTZOs90N5Z7jEwMR4fKL1ZHr1gvQ62/MnJvUjh6VvYg35tm\n+YBapdbi9xO6jkPDLFzuodecExOJnBIDarKJi9dK8ftJ3QfX4zVMRryRoY6aExMBANeKlYYaz9Fd\nqsogPNzkiA7QZV1PWyFLvdoQ1LSAJEnoHOkPADhkpQz1vgt5uHCtFF5ucqsFN/qJidYsW3E2BzPy\ncTm/DJ5ucgyIDzZcqTidWWyVsqwb3+/3JUWge2VAffhSgcXrkHefy0WRUo3mPgokRTZ8aftbYYaa\nyDkxoCab+OZvXau8O9o0R6s62lL56XtRM0MNAFh3+CrUWoGOLfzQOtjH6LY2VpqYmF1Ujl1n9EGN\nLohPjPAHoLvUX6GxfCC1ar8uOz28Yxg83Vzq2LthYoLY6aO+1h3SZacHtw+Bh5sckQEe8HF3gUqj\nxZlsy3+R+/nQFWi0AokRfmgd7I2YQE8EeSug0mhxKCPfosfaULlAzdCEkBrndFgCM9REzokBNTW6\nGy8ZP9Ijss792enDmD5TXNMkPWvVUa9NvQKtAJKi/A39m2MCPeHr7gKVWmvx45VXaPBL5Xvkxiy8\npcUyoK4XrVYYJsOOrKxlliTJUPZx7LLl66jX3PR+lyQJPSrrtvdZcGKiRiuw8VgWAGBYQpjFHvdm\nwcxQEzklBtTU6FIrLxl7ucnRPz64zv2beeoDamaoz+UU41BGPuQyCXd3Cq92u77Th6Uz1DcHNYAu\nsOlkpbKPjTdMQrst1jIrI9aEJR/1s+9CHjILy+Hj7oI72lT1BE8I13XDOGbhOupzOcU4dKmg2vu9\ne+VE3D0WrKPel34d10pU8PNwRc+WARZ73Jux5IPIOTGgpkanzzze2S4E7q7yOvfn8uNV9L14+8YF\nGT6Yb6TPUJ/OLLJYf+jTWUU4dqUQLjLJkJXU009MPGzhiYn6co/RXVpY7dI7cEMvamaoTaLv7jGk\nfSgULlX/dju00E9MtGyGWv9+vyMuCEHeVe93fUB94EIeNBZ6n6+vLPcY1C4ErnLrfTQ299YH1OVW\nOwYRNT4G1NSohBD4rbIt1V0dTbus6ufBDDWgO3c1ZYpvFB3oBTe5DCUqDS7nl1nkuPpj9o8PRoCX\nm9Ft+jpqS2aoswrLsfOMftKl5bt73Eifoc4rrWAXmTpotAK/VvaCHtnJ+N9uhxa6DPXxK4UWC3C1\n2hve7ze9D9qF+cJH4YIipRonrpofxAshsEHfLs9KE2D1gn3dATBDTeRsGFBTozIu96i+MmJN9Iu7\nNPUuH/su5OFSnu7cDWlf84e+q1yGls11QaIlFnjRagV+qgxqaqpl1nf6OJ1VhFKV2uzjAbqspFYA\nXaObGWqcrcVL4YIQX13GkGUft/b3+WvILVbC39MVfVobLwEfG+QND1c5yio0SLNQtl//fvdWuGBw\nuxCj2+QyCV2idXXUey3Qj/rI5QJcKSiHp5scfS24vH1NDCUfxUoIYd1VRomo8TCgpkZV33IPAGhm\n6PLRtEs+9IucDOsQBg+32s+dvuzjpAUmCv6Vdg1XCnQ1swPbVq93D/F1R4ivAlphmYU9hBBYdcB6\nvadros9Ss+zj1n6u/Lc7LCG0WkmEXCahXZjufWepOuo1lT3Ih3UIrfH93kPfj9oCAbV+MZcBbYNN\n/rvUUPre5xUa0eSvuhE5EwbU1GgaUu4BAH6clFjZ9UJXv1pX1wtLLkGuX0BmRMewWgMNQ9mHBVqY\nHbtSiNNZxXBzkWFEovU6LdxInwU/z4C6VhUaLdYfNe7ucTN9P2pLfLEqr9AYOgGNrqW8qWpiYp5Z\nmV4hhNHqiNamcJEbrrrlcPlxIqfBgJoaTUPKPYAb2+Y13YB626lsFJarEerrjtta3rrrhaVa55VX\naAxfgGqr2QZwwwIv5mcmV1ZORhzcPsRQO29t7EVdtz/O5iKvtAKBXm64rZYOGPo66qOXzX8fbD2Z\njaJyNcL8an+/J0b4wU0uQ26xEunXGr7S5ZnsYpzPLYGbXIYBNVyFsQb9xMTsQgbURM6CATU1moaU\newBVXT4KmnANtb7cY1RSOOR1dL3QL+5yLqfYrAVXNh3PQnFl6zp9NrAmhk4fZk5MVKm1WHtIl4W/\nv5HKPYAbSj5YQ10rfbZ4eMdQuNTSAUOfoT56ucDs2mB9r/V7OofX2uXF3VWOTpG6Y5qzDLk+O903\nLgjeCussIHSzYF99HTU7fRA5CwbU1CgaWu4BVPWhLlaqrbK0sb3LK1Fh6yndMu23yhTrtfD3gLfC\nBRUaYdYEsTWGpcZrD2oAILGFPwDgwrVS5JU0vM5926lsXC9RobmPwuoTw26kL/lIyy3hJLEaKNUa\nwwqCtZV7ALovcq5yCYXlalzKa3iHmbwSFbZVvt9HJ936i1U3fdmHGXXU+oB6qJW7e9yoqnUeM9RE\nzoIBNTWKhpZ7AICPuyukyniuKWap1x25igqNQLswX7QN9a1zf0mS0CZEt5x7Q8s+couV2H5av9T4\nrYMaP09XQ1B62IzL/fos/L2dw2vNglpDdKCuF3VRuRrXzfhC4Kx2ns5FUbkawT6KW16pcHORGa6O\nmDMxUf9+bx/mi/jK8qXa9Igxb2LixWulOH61EHKZhEE3dRKxJn3rPJZ8EDkPBtTUKBpa7gHoOgj4\nuuuy1AVNcHEX/eIWtU3Oqkm8mXXU6w5dgUYrkBjhh9bB3nXuX7XAS36DjpdXosLvJ3XLPo/p2njl\nHoCudCDcTxfgsOyjOv1iLnd1DKuz3KhqxcSGT0xcc6BqUZ+6dIluBknSXR3JLqx/+YQ+894zNqBa\nj3VrMmSoOSmRyGkwoCarM6fcQ8+/iXb6uHitFPsv5EEm6epJTRUfYt4S5IZyj86mBfHmLvDy8+Er\nhqykKVl4S4sxlH00fHKbMyqv0GDTcd0Xnbs71f1v17BiYgOvVKTnluDAxXzd+71T3e93Pw9Xw/ul\nIWUf+tURrb2Yy830vaiZoSZyHgyoyerMKffQMyw/3sQC6rWHdIFtr1ZBCKm8TGwKc1rnncspxqFL\nBZDLJJOD+M6Vk8MOXWrYhDT9UuONnZ3Wi2WnjxptO5WNEpUG4X7uSIpsVuf+homJDcxQ/5iqe7/3\nbh1kKIuoS4+YygVe6jkxMbuwHPsv5AFArQslWUuwDzPURM6GATVZnTnlHnr61nl5TWxxl58P6c6d\nKdm6G+kz1Bevl9Z7BUN9ickdcUEIqrw0XZf2YX6QyyTkFCmRWc9L72ezi3DoUgFcZBJG1SMLb0mG\niYks+TCiX8xlZKdbT0zVaxfmA0nSTbarbwmGEFVLjZtS7qHX3bDAS169jrehMvOeFOWPUD/Tv6xa\ngmG1RE5KJHIaDKjJqixR7gFUlXw0pUmJpzKLcCqrCK5yCUPrueBEoLcCQd4KCAGcySo2+X5abVVQ\nc189Wtd5uMkNE9Lqu8DLqsrJiP3jm5scwFsaV0usrlSlxpYTum4bI01cZMfTzQWtmutq7utbR33g\nYj4uXCuFh6u8Xhlj/cTEE5mFKCw3/e/DhkZczOVm+oC6oKwC5RWaRj8+EVkeA2qyKkuUewBNc3EX\nfblHvzbBhtUi66MhC7zsSb+OS3m612twPbse3Fj2YSqNVhhWY2yspcZrcuPiLmydp/P7iWyUVWgQ\nFeCJjpW10aaomphYvzrqG5ca96pHP+hgX3dEB3pCCBhKOOpy4moh/jx/DQDq/WXVEvw8XOFW2ckm\nl2UfRE6BATVZlSXKPQDAT19D3US6fAghqso9GlgG0aYBExO/35sBALi7Uzg83Or3euknJtZngZct\nJ7ORWVgOPw9XDGzXOKvU1SQqwBMyCShRaXgZvpK+u8fIxDBIUt3lHnodDAu8mJ6hLlNpDO93U3qt\n30zfzs+UOmqNVuCVVYeh0QoMSwg1fJlqTJIkseyDyMkwoCarsVS5B1C1uEteE8lQp2bk4+J13eXv\nQQ0MNOND69eLurC8Ar8e1QU1D3aPrPfxqlrnFUCrrTvLq9UKLNh0GgAwtkckFC4N/8JlLjcXGVo0\n8wAAsxbDcRb5pSpsPanrQ36rxVxqkqBfgrweGeofUy+joKwCEc080Lt1/Rf1qU8/6iV/pOHwpQL4\nuLtgzqiEeh/LUgydPhhQEzkFBtRkNZYq9wBuqKFuIgG1fgnuwe1D4OnWsOWQ4yvbiZmaoV6begXl\nFVrEBXsjKdK/3sdrE+IDd1cZipRqkyb3/Xr0Kk5cLYSPwgVP39Gq3sezNC5BXuWn1CtQabRoF+aL\n9uH1a2OYEKb7YnUpr8ykf69CCCzelQYAmNArps5e1zXRT0w8lFFwy5rkjOuleG+j7kvca3e1q1fn\nHEtjhprIuTCgJquxVLkHAPh7NJ2SD41WYN3hhnX3uFFc5YIsOUVKk1YA/H6frtzjoe6R9brEr+cq\nlxnaptVV9qHWaLGgMrB5qm9LNGvERTVqE8te1AY/7Ne9Fx5oQBtDP09XRAbosv2m1FHvPJOLM9nF\n8HKTN+jKCADEBHoiyFsBlUaLw7XU8Ash8NqaIyir0OC2lgEY28BjWQoz1ETOhQE1WYUlyz0AGCbl\nNYVJiX+fv4acIiX8PFxxR5uGZ/a9FC6ICtAtq11X2cfxK4U4fKkArnKpQTWsevqyj0MZtw6kVh+8\njPO5JWjm6YqJfWIafDxLYqcPnRNXC3H0ciFc5RLubeB7QV9HbUqnj8V/6LLTD3SLNKyIWl+SJKFH\nbGU/6lrKPtYcvIydZ3Lh5iLDvNGJDfrSaEnBzFATORUG1GQVliz3AIBmlZMSm0LJx8+Vk8GGdwiF\nm4t5/0T1ExPrWuBFn50e3D4EgWa0rutkwoqJSrUG/918BgDwTP9W8GlgEGVpsc1Z8gEAKysX2bmz\nbUiDl+PWd/qoq476bHYRtp3KgSQBT/SOadCx9PQTE/fUMDExt1iJN9YdBwBMGxRnuBphSyz5IHIu\nDKjJKixZ7gFUtc0rUqpRodGa/Xj2SqXW4tcjusy+OeUeevrWeSdvkaEur9AYek8/2M28y+CdKmuv\nj10prPV1WrEnA5fzyxDiq8Djt8eYdTxLir2hhtqUSZXOqEKjNSzs80C3hrcxTDBxCfIlf6QDAAa1\nC0F0oHlBrj6gPnAhD5qbXr83fj6O/NIKtAvzxaS+Lc06jqUE++jqt3OK6rcADhHZJ5sG1PPmzUP3\n7t3h4+OD4OBg3HvvvTh16pTRPkIIJCcnIzw8HB4eHujfvz+OHTtmoxGTKSxd7gEAvh5VWUxnXtxl\n55kcFJRVINhHgZ4tA81+PFOWIN94PAsFZRUI93NH3zjzribEBHrC190FKrW2xjKTMpUGC7eeBQBM\nHRhnkS9blhLRzAMuMgnlFVpkNdEgZ8vJbFwrUaG5jwL9zCg30meoz+eW1LpSZ36pCqsO6LLhE3vH\nNvhYeu3CfOGjcEGRUo0TV6tKTbaczMLaQ1cgk4B3xnSEq9w+8kjMUBM5F5v+Zdm+fTumTJmCv/76\nC5s2bYJarcaQIUNQUlJ1yXX+/PlYsGABFi5ciL179yI0NBSDBw9GUZHpvXWpcVm63AMA5DIJvu66\nbhfOXEet7+4xIjGsQd0ObqbPUJ/OLKp1wZLv9l4EANzfLdLsY0qSZOhHXVPZx7I/05FTpEREMw88\nZGY23NJc5DJEVtacN9XWeT/s0wW4o5NawMWMwDPYxx3BPrqVOm8Mbm/07Z4MlFdo0T7MF7e1DGjw\nsfTkMgldoo3rqIuVasxacxQA8GSfWMN70x4YAupiJRcTInICNg2o169fjwkTJiAhIQGdOnXCkiVL\ncPHiRezfvx+ALtP5wQcfYObMmRg9ejQ6dOiAZcuWobS0FMuXL7fl0OkWLF3uoafvBFHgpJ0+ylQa\nbDqeBUC3sIolxAR6wVUuoUipxpWC6lnXjOul+OPsNUhSwzo61KRTZFU/6hsVlldg0bZzAIAXB7Ux\nuz7cGmICdQF1ehPs9JFTpMTWU7qlxs0p99Dr0KL2iYkVGi2W7U4HAEzsE2uxCYI9Yo37Ub+74RSu\nFJQjMsADLw5uY5FjWEqQt+7vWYVGOHWSgKipsKtPtIIC3QdwQIDuj2JaWhoyMzMxZMgQwz4KhQL9\n+vXD7t27a3wMpVKJwsJCox9qPBUaraHlm6XKPfT0ddR5Jc754bP5RBZKVRpEBng0qA90TdxcZGgZ\npF/gpfq/hR8qJyP2bhVkyM6aq7YM9Zc701BQVoHWwd4N7h5hbTGG1nnFNh5J4/vx4GVotAKdI/3R\nOtjH7MczTEysoY76t6OZyCwsR5C3G+7uZLm/E1UTE/Ow/0Ielv2ZDgBIua9jg/u5W4vCRW7or5/D\n5ceJHJ7d/IURQmD69Ono06cPOnToAADIzNTV4YaEhBjtGxISggsXLtT4OPPmzcOcOXOsO1iq1S+H\nrxo+KC1V7qFXtfy4cwbU+nKPuxPDLdrSKz7UB6eyinAqsxgD21b9W9JoBX6o7OjQ0P6/NdF3+jid\nVYRSlRqebi64XqLC/3aeBwBMH9zGIuUs1tBUe1ELIap6T1sgOw3A0JO8piXI9Qu5PHpbtEVXyEyM\n8IObXIbcYiWmfHMAQgBjukSYPTfAWoJ9FMgvrUB2odLQkYeck1arhUrlnFdXnZWrqyvkctP/PtlN\nQD116lQcPnwYu3btqnbbzcGFEKLWgGPGjBmYPn264ffCwkJERtpXraazEkLg0+26S/pP9I61+IQz\nfYY6v9T5/igVlFVg+yndUs/3dLZMuYdefKgPcKh6hnrHmRxcLSiHv6crhrQPqeXe9Rfq544QXwWy\nCpU4dqUQ3WMCsGjbWZSoNOjQwhfDEkItdixLa6qrJR6+VIDTWcVQuMgsVm6kz1CfyS6CUq0xBM77\nL+QhNSMfbnIZHr0t2iLH0nN3laNTpB/2puchs7AcgV5umDWinUWPYUnNfRQ4nVWMnOKmOQm2qVCp\nVEhLS4NW67wdqpyVv78/QkNDTUpy2UVA/dxzz2Ht2rXYsWMHIiKqsiOhoboP3szMTISFVV0WzM7O\nrpa11lMoFFAoGt5Hlxpu2+kcnMwsgpebHI/2tOwHJXDD8uNOmKHecCwTKo0WbUK80Ta0fks91yW+\nMvN1Ksu4jOH7vbqM5L2dW1j8y09ihD82Hc/CoYx8RAV44qs/dVeU/jkkHjI7zU4DVRnqi9dKodEK\nu82kW5o+Oz2sQ2iDF1e5WUQzD/h5uKKgrAJnsooNNdX6hVxGdQ5HkBk9z2vTPSYAe9PzAACz70mw\ni1U4a9Pcm50+nJ0QAlevXoVcLkdkZCRkMruqtKVaCCFQWlqK7GzdvJIbY9Da2DSgFkLgueeew5o1\na7Bt2zbExhq3ToqNjUVoaCg2bdqEpKQkALpvetu3b8c777xjiyHTLXxWmZ1+uEeUYWVDS/KvLPnI\nc8IM9c+V5R6W6D19s/jKTh/nsotRodHCVS7DtWIlNp/QTYB8yApLMHeK8NMF1JcKkH6tBEq1Ft2i\nm6G/Ga3YGkO4vwfc5DKoNFpcyS+zWF25PSuv0GBtqu7990BXy70XJElChxa++OPsNRy7UoAOLfxw\nOb8M6ytbaj5hgVZ5NRnWIRSfbj+HYR1CcXeiZedxWFqwr64XdXYhA2pnpVarUVpaivDwcHh6Ov/f\nE2fi4eEBQJfEDQ4OrrP8w6YB9ZQpU7B8+XL89NNP8PHxMdRM+/n5wcPDA5IkYdq0aUhJSUFcXBzi\n4uKQkpICT09PPPLII7YcOt0kNSMff52/DheZhIl9rPNBWVXy4VwZ6pwiJf44mwvAct09btTC3wNe\nbnKUqDS4cK0ErYN9sObgZVRoBBIj/NAuzLIZcaBqgZfdZ3MNVxT+NTTe5ss910UukxAV6Imz2cVI\nv1bSJALqjcezUFiuRgt/D/RqZX7v8xslhPvhj7PXcPRyIR7qDnz1Zzo0WoFerQLRPtzy7ztAd3Vk\n/6zB8PNwtfv3myFDzUmJTkuj0QAA3Nzs90oJ1U7/JaiioqLOgNqm1x4WLVqEgoIC9O/fH2FhYYaf\n7777zrDPyy+/jGnTpuHZZ59Ft27dcPnyZWzcuBE+PpzAYU/02elRnVsg3N/DKsdw1pKPX49chVbo\nglBzV4uriUwmIU5f9pFZDCEEvqss97BGdhoAElv4AwCulaig1gr0jQuyyEI1jcFQR91EelHrO72M\n6dLC4uU4Ny5BXqJU49u/dT3PLbGQy60083Kz69IiPX0vamaonZ+9f7mjmtXndbN5yUddJElCcnIy\nkpOTrT8gapDzOcVYf0x3dWFyP+st66sPqJ0tQ73WiuUeevEhPkjNyMepzEKE+rnjTHYx3F0tNwHt\nZn6erogJ9ET6NV23jJeGxFvlONYQG6Rf3MX5O31cyS/DrsqrI/dbsNxDT9/p48TVQvywLwOF5WrE\nBHpiYNtgix/LEQX7MENN5CzsYlIiObYvdqZBCODOtsFWbf3kb2ib5zw11JfySrH/Qh4kCRhpxXpP\nfR31qawiZFVmw+7qGGaxCWg1SYpqhvRrpRjSPsRQAuII9L2om0Knj9UHLkEIoGdsAKICLV/eEhvk\nBU83OUpVGry/+QwAXe20I2SPGwOXHydyHpxuSmbJLirHqgO6XsaT+7Wy6rEMNdROtLDLz4d0i+Dc\nFhuIkMoJStagD6gPXyrAusO6jPjY7lFWOx4AvHBnHJ7sE4u593aw6nEsLbaJlHwIIbCysg/5A1Za\nBl4uk9C+ska/oKwCPu4uuN9CK3I6g2Af3b/5grIKlFdobDwaoirZ2dmYPHkyoqKioFAoEBoaiqFD\nh+LPP/9s9LFs27YNkiQZfgIDAzFw4ED88ccfRvslJycb7af/adu2rWGf8+fP4+GHH0Z4eDjc3d0R\nERGBUaNG4fTp02aPkxlqMsvSP9KhUmvRJcof3WOaWfVY+gx1kVJt6Fbh6AzdPSzce/pm+oD6auXy\n4y2DvKz+esUEeeHfI9tb9RjWoM9QX7xeCrVGCxcneJ/VZG96HtKvlcLLTY67OlqvN3hCuC/2XdC1\nsRvbPRJeCn7s6Pl6uBi6yuQWKxHRzPknwZJjGDNmDCoqKrBs2TK0bNkSWVlZ+P3333H9+nWbjenU\nqVPw9fVFTk4O5s6dixEjRuD06dMIDq4qIUtISMDmzZuN7ufiovubo1KpMHjwYLRt2xarV69GWFgY\nLl26hF9//dWwUrc5nPOTghpFUXkF/u8vXX/hp/u1svqkC1/3qg/iQieYmHg2uxjHrxbCRSZZfbGT\nIG8FAm/ox/tAt0hOkqlFqK873OQyqLXC8AXEGeknI45IDLPqstwJlf2nZRIwvleM1Y7jiCRJqpqY\nyLIPshP5+fnYtWsX3nnnHQwYMADR0dHo0aMHZsyYgREjRgAA0tPTIUkSUlNTje4nSRK2bdsGoCqz\nvGHDBiQlJcHDwwMDBw5EdnY2fvvtN7Rr1w6+vr54+OGHUVpa95yV4OBghIaGomPHjpg1axYKCgrw\n999/G+3j4uKC0NBQo5+goCAAwPHjx3H+/Hl88sknuO222xAdHY3evXvjrbfeQvfu3c0+bwyoqcFW\n7MlAUbkarZp7YVA7y620VxsXucwQVDvD8uOfbDsLALijTfNGWXxCn6WWyySM6drC6sdzVDKZhBbN\ndJ1qLuWV2Xg01lGiVOOXI7pyI2uVe+gNbBuMls29MOmOlszA1oB11E2LEAKlKrVNfkxpBAEA3t7e\n8Pb2xo8//gil0vz3ZXJyMhYuXIjdu3cjIyMDDz74ID744AMsX74cv/zyCzZt2oSPPvrI5McrLS3F\nkiVLAOiWBzdV8+bNIZPJsHLlSkM7Q0vitTdqEJVai//t0q14NvmOVo02ycjf0w2F5WqHX358x+kc\nrD5wGZIEPDewdaMcs22oL3afu4aBbYMNtZtUs4hmHkjLLcGlvFIAjtHurz5+PXIVpSoNYoO80C3a\nuqU/Qd4KbPlnf6sew5ExoG5ayio0aP/6Bpsc+/gbQ026GuXi4oKlS5di0qRJ+PTTT9GlSxf069cP\nY8eORWJiYr2PO3fuXPTu3RsA8OSTT2LGjBk4d+4cWrbUdQW7//77sXXrVrzyyiu3fBz9StqlpaUQ\nQqBr16648847jfY5cuQIvL29jbaNHTsWX375JVq0aIEPP/wQL7/8MubMmYNu3bphwIABGDdunGEs\n5mCGmhrkp9TLyCwsR4ivAqOSrFv/eyNnaJ1XqlLjtTVHAAATesUgKcq6AY3epDti8ehtUXjdAeua\nG5s+k5rhpBnqHyonI97fNYKlPzYWzJIPskNjxozBlStXsHbtWgwdOhTbtm1Dly5dsHTp0no/1o1B\neEhICDw9PY0C2JCQEMMS37eyc+dOHDhwAN9++y2io6OxdOnSahnq+Ph4pKamGv289dZbhtunTJmC\nzMxMfP3117j99tvxww8/ICEhAZs2bar387oZM9RUb1qtwGc7zgPQLdCgcLn16kGW5OcEqyW+t/E0\nLuWVoYW/R6P2Zw7z88Dcezs22vEcWYSh5MP5elEfv1KIPWnXIZOA0V1Y+mNrzFA3LR6uchx/Y6jN\njl0f7u7uGDx4MAYPHozXX38dTz31FGbPno0JEyZAJtPlY28sI6moqPlz+cagV5KkakGwJEnQarV1\njic2Nhb+/v5o06YNysvLcd999+Ho0aNQKBSGfdzc3NC69a2v+vr4+OCee+7BPffcg7lz52Lo0KGY\nO3cuBg8eXOcYboUZaqq3LSezcTa7GD4KFzzc07qt127WzNCL2jED6tSMfCz5Q1cqM/e+Dux4YKf0\nS447Ww21Rivw6urDAIDhHcMQ5medVU3JdAyomxZJkuDp5mKTH3OvRrVv3x4lJbp2os2bNwcAXL16\n1XD7jRMUre2xxx6DVqvFJ598Ytbj6Nvq6Z+XOfhpTvX2aeUy4+Nui7bqwiA1qSr5cLwa6gqNFq+u\nOgytAO7tHI4B8Vwtzl4ZMtTXnStDveSPNBy+VAAfdxfMZumPXdDPZ8gpct6OMuRYrl27hgceeAAT\nJ05EYmIifHx8sG/fPsyfPx+jRo0CAHh4eOC2227D22+/jZiYGOTm5mLWrFmNNkaZTIZp06Zh7ty5\nmDx5Mjw9dUkQtVqNzMxMo30lSUJISAhSU1Mxe/ZsPPbYY2jfvj3c3Nywfft2LF68uM76bVMwoKZ6\n2Zd+Hfsu5MFNLsPE3jGNfnx/By75+HzHeZzMLEIzT1eH7M/clOgD6szCcqjUWri5OP7FvIzrpXhv\no27xgpl3tUOwFRcSItMxQ032xtvbGz179sT777+Pc+fOoaKiApGRkZg0aRJee+01w36LFy/GxIkT\n0a1bN8THx2P+/PkYMmRIo41z4sSJmD17NhYuXIiXX34ZAHDs2DGEhRmvOqxQKFBeXo6IiAjExMRg\nzpw5hrZ/+t9ffPFFs8cjCVP7qDiowsJC+Pn5oaCgAL6+vrYejsN7atk+bD6RhbHdI/H2mPrP9jXX\n/3al4c11x3F3p3B89HBSox+/oc7lFGP4f3dCpdbi/Yc64b4krhZnz4QQaPvv9VCqtdjxrwFWWZa7\nMQkh8PjiPdh5Jhe3tQzAt5Nu42REO3E5vwy9394CV7mE03OH83WxIHv4/C8vL0daWhpiY2Ph7s4v\nsY6mPq+f46ddqNEcuJiHzSeyIEnApDvMbzHTEFUZascp+dBqBWasPgKVWot+bZrj3s6cCGbvJEky\nZKkznGBi4pqDl7HzTC7cXGSYNzqRQZsdCfLWzQup0AiHvPJGRDoMqMkkeSUqPLf8IADg3s4t0Kq5\ndx33sI5mXrqAusCBJiWu2JuBPWnX4ekmx1v3dWAw4yD0rfMcvdNHbrESb6w7DgCYNigOsZVLq5N9\nULjIDXNDcopZ9kHkqBhQU520WoHp36ficn4ZYgI98caoBJuNxc9Dl83Jc5AMdVZhOeb9egIA8NKQ\neK4U50AiA5xjtcQ31x1HfmkF2oX5YlJf21xZolsz9KIuZEBN5KgYUFOdPttxHltP5cDNRYaPx3WB\nTyN39riRIy3sIoTAv388iiKlGp0i/TG+V4yth0T1YFjcxYE7fWw5mYWfUq9AJgHvjOkIVzn/5Nsj\nw8TEYnb6IHJU/OtKt/T3+Wt4d+MpAMCcexKQEO5n0/Hoa6iLytVQa+puBG9L649mYuPxLLjIJLwz\npiPkjbQ8O1lG1eIujpmhLlaqMWvNUQDAk31ikRjhb9sBUa2aezNDTeToGFBTrXKLlXju24PQaAXu\nS2qBsd0jbT0kw0qJAFBYrrbhSG6toLQCr689BgB4pn8rtA1lhxlHE9nMsRd3eXfDKVwpKEdkgAde\nHNzG1sOhW9C3MGTrPCLHxYCaaqTRCkxbkYrsIiVaB3tj7r32MZnORS6Dj7uufbo911G/9etx5BQp\n0bK5F6YMuPUyqGSf9BnqrKJyKNUaG4+mfvZfyMOyP9MBAPPuS4SnG5ccsGf6DDUnJRI5LgbUVKOF\nW85i19lceLjKsWhcF7taItve66g3Hc/C9/suQZKAt0cnwt1VbushUQMEeLnBw1UOIYAr+Y5T26pS\n61bkFAK4v2sE+sQF2XpIVIdgX5Z8EDk6BtRUzR9nc/HB77oV1d66rwPiQnxsPCJj/pWdPgrK7C9D\nnVusxKurDgMAJvVtiR6xATYeETXUjb2oHal13qJt53AmuxhB3m6YeVc7Ww+HTMAMNZHjY0BNRrIK\ny/HCioMQAhjbPRKju9jfin72mqEWQuDVVUdwrUSFtqE++OcQ1q06usgAfacPx6ijPpNVhI+3ngUA\nzL47Ac283Gw8IjIFlx8ncnwMqMlArdHiuW8PIrdYFxAm32O7ftO34u+pCxLsLaD+fl8GNp/Igptc\nhvcf6gyFC0s9HJ0jZajLVBo89+1BqDRa3Nk2GCMTw2w9JDJRsI9uUmJBWQXKKxyrXp+c04QJEyBJ\nEp5++ulqtz377LOQJAkTJkxolDFIkgQXFxdERUXhmWeeQV5enlWP21AMqMlgwabT2JN2Hd4KF3wy\nrovd1v7a4/LjF6+V4o2fdavR/XNIG7QLY1cPZ+BIrfNmrz2Kk5lFCPJWYN6YjnYxiZhM4+vhArfK\nHuG5LPsgOxEZGYkVK1agrKzq7195eTm+/fZbREVFNcoYhg0bhqtXryI9PR1ffvklfv75Zzz77LON\ncuz6YkBNAICtJ7PxybZzAIC3x3RESxstLW4KQ8mHnSw/rqlcSbJEpUGPmAA8xdXonIa+dV6GnWeo\nf9iXge/3XYJMAj58uLMh40mOQZIkQ9lHNss+yE506dIFUVFRWL16tWHb6tWrERkZiaSkJMM2IQTm\nz5+Pli1bwsPDA506dcLKlSsNt2/btg2SJGHDhg1ISkqCh4cHBg4ciOzsbPz2229o164dfH198fDD\nD6O01PhvrUKhQGhoKCIiIjBkyBA89NBD2Lhxo9E+S5YsQbt27eDu7o62bdvik08+Mbp9z549SEpK\ngru7O7p164Y1a9ZAkiSkpqZa8GwB9tO6gWzmcn4ZXvw+FQDw+O3RGJkYbtsB1cHeSj4+23EO+y7k\nwVvhgvce7MQFXJxIhAP0oj6VWYR//6RbwOXFQW3QqxW7ejii5j4KXM4vYx21sxMCqLDRF3RXT6Ce\nV66eeOIJLFmyBOPGjQMALF68GBMnTsS2bdsM+8yaNQurV6/GokWLEBcXhx07duDRRx9F8+bN0a9f\nP8N+ycnJWLhwITw9PfHggw/iwQcfhEKhwPLly1FcXIz77rsPH330EV555ZUax3L+/HmsX78erq5V\n61F88cUXmD17NhYuXIikpCQcPHgQkyZNgpeXF8aPH4+SkhKMHDkSAwcOxNdff420tDS88MIL9ToH\npmJA3cSp1FpMXX4A+aUVSIzww8wR9t8VwFDyYQcZ6mNXCvD+Jl1HlNl3tzdMYiPnoC/5yClSorxC\nY3dlUCVKNZ75Zj/KK7S4o01z9jx3YJyY2ERUlAIpNkpavXYFcPOq110ee+wxzJgxA+np6ZAkCX/8\n8QdWrFhhCKhLSkqwYMECbNmyBbfffjsAoGXLlti1axc+++wzo4B67ty56N27NwDgySefxIwZM3Du\n3Dm0bKm7qnv//fdj69atRgH1unXr4O3tDY1Gg/JyXfvSBQsWGG5/88038d5772H06NEAgNjYWBw/\nfhyfffYZxo8fj2+++QYajQaLFy+Gp6cnEhIScOnSJTzzzDP1PHl1Y0DdxM1ffxIHL+bD190FHz/S\nxSEm0lV1+bBtDXV5hQYvfpeKCo3A0IQQ3N/V/jqikHn8PV3hrXBBsVKNS3llaB1sP6VQQgi8tuYI\nzueUINTXHe8/2AkyXh1xWMEs+SA7FBQUhBEjRmDZsmUQQmDEiBEICqq6Cnb8+HGUl5dj8ODBRvdT\nqVRGZSEAkJiYaPj/kJAQeHp6GoJp/bY9e/YY3WfAgAFYtGgRSktL8eWXX+L06dN47rnnAAA5OTnI\nyMjAk08+iUmTJhnuo1ar4efnBwA4ceIEOnXqBE/PqmSXPvC3NAbUTdj6o5n4clcaAOC9Bzs7THbV\nXtrmvbvhFE5nFSPIW4GU+zgJzBnpe1GfzCzCpbxSuwqol++5iJ9Sr0Auk7DwkSQEVvYyJsfEDHUT\n4eqpyxTb6tgNMHHiREydOhUA8PHHHxvdptVqAQC//PILWrRoYXSbQmH8N+nGUg1Jkox+12/TP56e\nl5cXWrfWXXn78MMPMWDAAMyZMwdvvvmmYd8vvvgCPXv2NLqfXK5LDgohTH+iZmJA3URdvFaKf608\nBAD4xx0tMbh9iI1HZDo/D30Nte0y1LvP5hq+jLwzpiODGSdWFVDbTx310csFmLNW11XmlWHx6BbD\nBYQcXVVA7TirclIDSFK9yy5sbdiwYVCpdJ+3Q4cONbqtffv2UCgUuHjxolF5h7XMnj0bw4cPxzPP\nPIPw8HC0aNEC58+fN9R436x9+/b4v//7P5SVlcHDQ1fC99dff1llbAyom6DyCg2eXb4fReVqdI1u\nhn8Njbf1kOqlWWWGurBcDY1WNPokwIKyCrz0g+7LyMM9onBnO8f5MkL1F2FnnT4Kyyvw7DcHoNJo\nMahdMCaxq4xT0HdmYYaa7I1cLseJEycM/38jHx8fvPTSS3jxxReh1WrRp08fFBYWYvfu3fD29sb4\n8eMtOpb+/fsjISEBKSkpWLhwIZKTk/H888/D19cXw4cPh1KpxL59+5CXl4fp06fjkUcewcyZM/Hk\nk09i1qxZSE9Px7vvvmvRMemxbV4T9NYvJ3D0ciGaebpi4SNJcJU71tvAz6PqMlGBDSYmzll7DFcK\nyhEd6IlZDjCJk8xjT72ohRB4+YfDuHi9FC38PfDeA51ZauQkWPJB9szX1xe+vjWvr/Dmm2/i9ddf\nx7x589CuXTsMHToUP//8M2JjY60ylunTp+OLL75ARkYGnnrqKXz55ZdYunQpOnbsiH79+mHp0qWG\nY3t7e+Pnn3/G8ePHkZSUhJkzZ+Kdd96xyrgk0ZgFJjZQWFgIPz8/FBQU1PpmaErWHrqC5789CEkC\nlkzojv7xwbYeUoN0nL0BRUo1tvyzX6P2zN5wLBOT/28/ZBLww9O90DW6WaMdm2xj/dFMPP31fnSK\n9MdPU3rbdCyLd6XhjXXH4SqX8MPTvdA50t+m4yHLuZJfhl5vb4GrXMLpucP5RckC7OHzv7y8HGlp\naYiNjYW7O/vD24P09HTExsbi4MGD6Ny58y33rc/r51ipSTLLuZxizFh1GAAwdUBrhw2mAcDPBou7\n5JWoMHONrt/v5H6tGEw3EZEBlRnq67Yt+Th4MQ/zftNddp15VzsG004m0Fs3N6RCI2w+4ZqI6o8B\ndRNRptJgyjcHUKLS4PaWgZg2qI2th2SWZpWLuxQ04gdP8s/HkFusRFywN6YNimu045Jt6Wuor5Wo\nUKpS22QM5RUaTKts0XhXx1CM7xVjk3GQ9Shc5IYORjlcfpzI4TCgbiJmrz2Kk5lFCPJW4L8Pd3b4\n1fyqlh9vnE4fG45l4qfUK5BJwH8e6OQQ/brJMvw8XOHjrpu/fdlGddSfbD2LC9dKEeKrwNtjElkO\n4KT0vaizCtnpg8haYmJiIISos9yjvhhQNwEr91/C9/suQSYBHz7c2TCb3JHpJybmlVg/Q31zqQcv\ntTc9tuz0cTa7GIu2nwMAJN+dAF931zruQY4q3N9+JsASUf0woHZyGddLMfsnXTA4fXAb9GoVVMc9\nHIN/I9ZQs9SDIm3U6UMIgX//eBQVGoEB8c0xrENoox6fGld05eJaF67ZR4tGshwn7//gtOrzujGg\ndmJCCLyy6jBKVBr0iAnAs/1b23pIFlNVQ23dkg+WehBQlaFu7ID6x9TL+PP8NShcZHhjVAeWejg5\n/Wq1GTaeAEuWo+/brF8YhRxLaanu3+LNqzrWhAu7OLFv/r6I3eeuwd1Vhvn3J0Lm4HXTN9KXfFgz\nQ81SD9LT96JuzECnoLQCc9fpuno8f2ecIdgi5xUdqFtB78L1EhuPhCzFxcUFnp6eyMnJgaurK2Qy\n5jEdgRACpaWlyM7Ohr+/f7UFbWrCgNpJZVwvxbxfdR/Grwxri5ggx1rqtC7+lRnqPCt2+WCpB+np\ng9nGzFC/vf4krpWo0DrYm6shNhHRgVUlH0IIXpFwApIkISwsDGlpabhw4YKth0P15O/vj9BQ00rt\nGFA7oZtLPcbfHmPrIVmcf2WG2lolHzeWerzLUo8mr2q1xMbJUO+/cB3f7rkIAHjr3g5wc2FWqymI\nrCwtKipXo6CswpA4IMfm5uaGuLg4ln04GFdXV5My03oMqJ2QM5d66FlzUuLNpR6dWOrR5OkD6rzS\nChQr1fBWWO9PZ4VGa3j/3d81Aj1bBlrtWGRfPNzkCPZRILtIiQvXShlQOxGZTMaVEp0c0x5OxtlL\nPfT0HzTWWFGMpR50Mx93V8OXOGtnqZf+kY6TmUXw93TFjOFtrXossj+Gsg9OTCRyKAyonUhTKPXQ\n0wc3heUV0Ggt146IpR5UG0PZx3Xr1VFfzi/D+5tPAwBmDG+LQG+F1Y5F9ikqQJcEYacPIsdi04B6\nx44duPvuuxEeHg5JkvDjjz8a3S6EQHJyMsLDw+Hh4YH+/fvj2LFjthmsA2gKpR56+i4fQgCFFir7\nYKkH3UqEv35iovUCnTlrj6FUpUG36GZ4oGuk1Y5D9ivK0IuanT6IHIlNA+qSkhJ06tQJCxcurPH2\n+fPnY8GCBVi4cCH27t2L0NBQDB48GEVFRY08Uvt3Y6nHy0Odt9RDz1UuM9SxWqqO+u3fTrLUg2oV\nGVDZOs9KnT42Hc/CxuNZcJFJmHtfB6f+Qky1u7HTBxE5DptOShw+fDiGDx9e421CCHzwwQeYOXMm\nRo8eDQBYtmwZQkJCsHz5ckyePLkxh2rXbi71mNArxtZDahT+nq4oVqqRX6oCYN4XiL3p1/HdvgwA\nwNtjElnqQdVULe5i+UCnVKVG8lrd1bcn+8aibaivxY9BjiEqkIu7EDkiu62hTktLQ2ZmJoYMGWLY\nplAo0K9fP+zevbvW+ymVShQWFhr9OLumVOpxI0t1+tB1VTgCAHi4RxS6Rjcze2zkfCKsuPz4f38/\ng8v5ZWjh74EX7uTVkaZMX/JxtbAcSrXGxqMhIlPZbUCdmZkJAAgJCTHaHhISYritJvPmzYOfn5/h\nJzLSuesQm1qpx438PfSdPszr7fm/XWk4nVWMQC83vDIs3hJDIydkrWWhj18pxP92pgEA3hiVAE83\ndjNtygK93ODlJocQQIYVJ8ASkWXZbUCtd/NKUXWtHjVjxgwUFBQYfjIyMqw9RJu5sdSje0yzJlPq\noeenz1Cb0Tov43opPqjsqvDaXe3Y95Vq1cJfl6EurFx0wxI0WoFXVx+GWiswLCEUd7YLqftO5NQk\nSUJUIDt9EDkauw2o9Us93pyNzs7Orpa1vpFCoYCvr6/Rj7Nae+jKDaUenZpMqYdeMzMDaiEEktce\nQ3mFFj1jAzC6SwtLDo+cjJfCBQFeui9cly1U9rHkjzQcvlQAH3cXzBmVYJHHJMcXVTkBlp0+iByH\n3QbUsbGxCA0NxaZNmwzbVCoVtm/fjl69etlwZPahVKXG27+dBABMHdAasU2o1ENPX/LR0GzhhmNZ\n+P1kNlzlEt66r8Mtr3wQAUBkM32nD/MzhxnXS/HexqqrIyG+XEWNdKIrM9Rc3IXIcdi0WK+4uBhn\nz541/J6WlobU1FQEBAQgKioK06ZNQ0pKCuLi4hAXF4eUlBR4enrikUceseGo7cNn28/jakE5Wvh7\n4Km+LW09HJvQT0rMa0ANdbFSjTk/67oqTL6jFVoH+1h0bOScIpp54tClArMnJgoh8NqaIyir0KBn\nbAAe6ubccz2ofqKsVK9PRNZj04B63759GDBggOH36dOnAwDGjx+PpUuX4uWXX0ZZWRmeffZZ5OXl\noWfPnti4cSN8fJp28HMlvwyf7TgHQJfZcndtmi3e9Iu7NKTk44NNp3G1oBxRAZ6YOrC1pYdGTqqq\n04d5gc6ag5ex80wu3FxkmDe6Y5Mr16Jbq1rchQE1kaOwaUDdv39/CFH7stGSJCE5ORnJycmNNygH\n8PZvJ1FeoUWPmADc1THU1sOxmWaVEwjr2zbv2JUCLNmdDkDXVaGpfiGh+tMH1OZ0X8gtVuKNdccB\nAC/cGYeWzb0tMjZyHvrFXS5eL4VWK/iFi8gB2G0NNdVs/4XrWHvoCiQJeP3u9k267ldf8lFQj5IP\nrVZg5pqj0GgFRnQMQ//4YGsNj5xQRID5i7u8ue448ksr0DbUB/+4o2mWa9Gthft7QC6ToFRrkVOs\ntPVwiMgEDKgdiFYrMOdnXWbrwa6R6NDCz8Yjsq2GLOzy7d6LSM3Ih7fCBf8e2d5aQyMnpZ+UeDmv\n7JZX12qz9WQ2fkq9ApkEvDMmEa5y/gmm6lzlMoT76yapsuyDyDHwr7kDWXXgEg5fKoC3wgUvDeUC\nJH43dPnQaOsObnKKlHinsjPKP4e0QagfuypQ/bTw12Woi5T170VdrFQbVuSc2DsWnSL9LT08ciLR\nAZWdPtg6j8ghMKB2EMVKNeZvOAUAmDqwNZr7KGw8ItvTT0oUAigqrzu4Sfn1BArL1ejQwheP3RZt\n7eGRE/JwkyPIW/dvr76dPt7dcApXCsoRGeCB6UPaWGN45ESiAtnpg8iRMKB2EJ9sPYucIiWiAz3x\nRO8YWw/HLri5yOCt0M2rravTx+6zuVhz8DIkCXjr3o5w4aV2aqCGdPo4cDEPy/5MBwCk3NeRy4tT\nnQydPhhQEzkERhUOION6Kb7clQYAmHlXOyhc2JVCz9A67xaX3/NKVHjph0MAgMdui+aldjJLfTt9\nqNRavLrqMIQAxnSJQN+45tYcHjmJaLbOI3IoDKgdQMqvJ6BSa9G7dSAGt6992fWmqK7FXbRagenf\np+JKQTlig7zwL9aek5ki69np49Pt53A6qxiBXm6YNaKdNYdGToQlH0SOhQG1nfvr/DX8djQTMgn4\n98im3SavJlWt82rOUH+64xy2nsqBwkWGjx/pAh9318YcHjmhqpKPujPUZ7KKsHCLbjXY2fckoJmX\nm1XHRs5DX/JxrUSFYqXaxqMhorowoLZjmhva5D3SMwptQ31tPCL7469f3KWGDPVf56/h3cqJnG+M\nSkD7cJ4/Ml9Es8rMYR0Z6lKVGlOWH4BKo8XAtsG4OzGsMYZHTsLH3RUBlV/A2OmDyP4xoLZj3+/L\nwImrhfB1d8H0wSxVqIl/LTXUOUVKPP/tQWgFMLpLCzzYLdIWwyMnFHlDhrq2XtRCCMz68ShOZxWj\nuY8C74xJ5NUlqjd9lpplH0T2jwG1nSosrzBkV18Y1MaQqSBjhsVdbij50GgFpn13ENlFSsQFe2Pu\nvR0YzJDFhPvrAupSlQZ5tZQa/bDvElYfuAyZBHz0cBLbXFKDRHFiIpHDYEBtpz7bfg7XSlRo2dwL\nj9/Onsm18feoXvLx4e9n8MfZa/BwlWPRo13Yoowsyt1VjuDKALmmzOGJq4X4909HAQD/HBKP21oG\nNur4yHlEB7J1HpGjYEBth66XqLD0j3QAwMtD23J54lu4efnxnWdy8OGWMwCAlNEd0DrYx2ZjI+dV\n1enDeGJiUXkFnv3mAJRqLfrHN8cz/VrZYnjkJFjyQeQ4GKnZoc93nEeJSoOEcF8MTWCbvFupmpRY\ngcyCckxbkQohgId7ROG+pAgbj46cVU2LuwghMGP1EaTlliDczx3vP9gZMhlLjajhWPJB5DgYUNuZ\na8VKfFW5otqLg9qw9rcO+gx1brESz317ANdKVGgf5ovZd7e38cjImRkWd7khoP7674tYd/gqXGQS\nPnqkC1vkkdmiA70AAJfzy1Ch0dp4NER0Kwyo7cznO86jVKVBYoQf7mwXbOvh2D19l49LeWXYm54H\nb4ULPhnXBe6uXE2SrEffOk9f8nHkUgHerGxx+erwtuga3cxmYyPnEeyjgMJFBo1W4Gp+ua2HQ0S3\nwIDajuQUKbGsMjs9bVAcs9Mm0Jd86M2/PxExQV42Gg01FZE3BNQFZRV4dvl+qDRaDGkfgif7xNp4\ndOQsZDLJUK9/4Tp7URPZMwbUduSz7edQXqFFp0h/DIhndtoUfh5VKx9O6BWDuzpy8QyyvhtrqP/1\nwyFkXC9DZIAH/vNAJ34RJouKZh01kUNgPzE7kV1Ujq//vgAAeJHZaZO5ucjw3MDWyCwox4y72tp6\nONREhPm7Q5KA8gotNh7Pgptchk8e6Wr0BY/IEqIC2emDyBEwoLYTn247j/IKLZKi/NGvTXNbD8eh\n/HMIV5GkxqVwkSPU1x1XC3R1rf++uz06RvjZeFTkjNjpg8gxsOTDDmQV3pidZmcPIkcQU9mB4e5O\n4Xi0Z5SNR0POiou7EDkGZqjtwKJt56BSa9E1uhn6xgXZejhEZIKZI9ph26lsPNE7ll+CyWqiAnRf\n3DKul0IIwfcakZ1iQG1jmQXlWL7nIgBg+mBmp4kcRYcWfujQgmUeZF0RzTwgSUCxUo3rJSoEeits\nPSQiqgFLPmzsk21noVJr0SMmAL1aBdp6OEREZEfcXXX1+gDLPojsGQNqG7qSX4YVezIAANMGs7MH\nERFVp5+YyE4fRPaLAbUNfbz1LFQaLXrGBqBXK9ZOExFRdez0QWT/GFDbyKW8Uny/T5edfnFwGxuP\nhoiI7JWh0wcDaiK7xYDaRj7eeg4VGoFerQJxW0vWThMRUc2iAqs6fRCRfWJAbQMZ10vxA7PTRERk\nAkPJx/USG4+EiGrDgNoGFmw6DbVWoE/rIHSPCbD1cIiIyI5FVwbUWYVKlFdobDwaIqoJA+pG9sO+\nDKw5eBmSBEwfwuw0ERHdmr+nK3zcdctGsOyDyD4xoG5Ex68UYtaPRwHolhjvEtXMxiMiIiJ7J0kS\nO30Q2TkG1I2koKwCz3yzH0q1Fv3jm2PqgNa2HhIRETkIQ6cPZqiJ7BID6kag1Qr88/tDuHCtFC38\nPfDBQ50hk3ERFyIiMk1UADt9ENkzBtSN4LMd57H5RBbc5DJ8+mhX+Hu62XpIRETkQKpKPtjpg8ge\nMaC2st3ncvGfDScBAHNGJaBjhJ+NR0RERI6GJR9E9o0BtRVlFpTj+W8PQiuA+7tGYGz3SFsPiYiI\nHJA+Q33pehm0WmHj0RDRzRhQW0mFRospyw8gt1iFdmG+eHNUB0gS66aJiKj+wvzc4SKToNJokVlY\nbuvhENFNGFBbybxfT2L/hTz4uLtg0bgu8HCT23pIRETkoFzkMkQ08wDA1nlE9ogBtRWsO3wFi/9I\nAwC890AnxAR52XhERETk6KIC2emDyF4xoLaws9lFeGXlYQDAM/1bYUhCqI1HREREziAqoDJDfZ2d\nPojsDQNqC9qXfh1PLduHEpUGt7cMxD8Hc2lxIiKyjOjKXtQs+SCyPy62HoAzyCosx7xfT+DH1CsA\ndJNHPnw4CS5yfl8hIiLLiKpsnceSDyL7w4DaDEq1Bot3peOjLWdQqtJAkoCHukXiX0PjEeitsPXw\niIjIiRgWd2FATWR3GFA30NZT2Xjj5+NIy9XVsiVF+WPOPQlIjPC37cCIiMgp6QPq/NIKFJRVwM/D\n1cYjIiI9h6hJ+OSTTxAbGwt3d3d07doVO3futNlY0nNL8OTSvXhiyV6k5ZYgyFuBdx/ohFVP92Iw\nTUREVuOlcEFQ5dVPln0Q2Re7z1B/9913mDZtGj755BP07t0bn332GYYPH47jx48jKirKIsdQa7Qo\nUWlQptKgrEKDUpUa5RUalKp0P/r/P5tdjP/78wJUGi1cZBKe6B2D5++Mg487swRERGR9UQEeyC1W\n4sK1UnRo4Wfr4RBRJUkIYddrmPbs2RNdunTBokWLDNvatWuHe++9F/Pmzavz/oWFhfDz88M/vtyB\nCrk7ipVqlCjVRv8tr9DWa0x944Iw++72aB3sU+/nQ0RE1FAvfpeKNQcvo3frQItdFZVJgARJ919J\ngkySIEkw+j0pyh+3tQy0yPEai/7zv6CgAL6+vrYeDjk5u85Qq1Qq7N+/H6+++qrR9iFDhmD37t01\n3kepVEKpVBp+LygoAAD8sv88ZArPWx5PLpPg4SqDh6sc7m5y3X9d5fB00/3Xy02OIQmhGNg2GJIk\nUFhYaOYzJCIiMl0LLwGtshQ7j5Vi57GMRjvuU31i0T7Isa7G6j+j7TxvSE7CrgPq3NxcaDQahISE\nGG0PCQlBZmZmjfeZN28e5syZU2375UUTLDKmTy3yKERERI5j9gfAbFsPooGKiorg58fyGLIuuw6o\n9SRJMvpdCFFtm96MGTMwffp0w+/5+fmIjo7GxYsX+Q+qkRQWFiIyMhIZGRm8zNZIeM4bH8954+M5\nb3yOfM6FECgqKkJ4eLith0JNgF0H1EFBQZDL5dWy0dnZ2dWy1noKhQIKRfUe0H5+fg73x8DR+fr6\n8pw3Mp7zxsdz3vh4zhufo55zJtKosdh12zw3Nzd07doVmzZtMtq+adMm9OrVy0ajIiIiIiKqYtcZ\nagCYPn06HnvsMXTr1g233347Pv/8c1y8eBFPP/20rYdGRERERGT/AfVDDz2Ea9eu4Y033sDVq1fR\noUMH/Prrr4iOjjbp/gqFArNnz66xDISsg+e88fGcNz6e88bHc974eM6JTGP3faiJiIiIiOyZXddQ\nExERERHZOwbURERERERmYEBNRERERGQGBtRERERERGZw6oD6k08+QWxsLNzd3dG1a1fs3LnT1kNy\nGvPmzUP37t3h4+OD4OBg3HvvvTh16pTRPkIIJCcnIzw8HB4eHujfvz+OHTtmoxE7n3nz5kGSJEyb\nNs2wjefc8i5fvoxHH30UgYGB8PT0ROfOnbF//37D7TznlqdWqzFr1izExsbCw8MDLVu2xBtvvAGt\nVmvYh+fdPDt27MDdd9+N8PBwSJKEH3/80eh2U86vUqnEc889h6CgIHh5eeGee+7BpUuXGvFZENkP\npw2ov/vuO0ybNg0zZ87EwYMH0bdvXwwfPhwXL1609dCcwvbt2zFlyhT89ddf2LRpE9RqNYYMGYKS\nkhLDPvPnz8eCBQuwcOFC7N27F6GhoRg8eDCKiopsOHLnsHfvXnz++edITEw02s5zbll5eXno3bs3\nXF1d8dtvv+H48eN477334O/vb9iH59zy3nnnHXz66adYuHAhTpw4gfnz5+M///kPPvroI8M+PO/m\nKSkpQadOnbBw4cIabzfl/E6bNg1r1qzBihUrsGvXLhQXF2PkyJHQaDSN9TSI7IdwUj169BBPP/20\n0ba2bduKV1991UYjcm7Z2dkCgNi+fbsQQgitVitCQ0PF22+/bdinvLxc+Pn5iU8//dRWw3QKRUVF\nIi4uTmzatEn069dPvPDCC0IInnNreOWVV0SfPn1qvZ3n3DpGjBghJk6caLRt9OjR4tFHHxVC8Lxb\nGgCxZs0aw++mnN/8/Hzh6uoqVqxYYdjn8uXLQiaTifXr1zfa2InshVNmqFUqFfbv348hQ4YYbR8y\nZAh2795to1E5t4KCAgBAQEAAACAtLQ2ZmZlGr4FCoUC/fv34GphpypQpGDFiBAYNGmS0nefc8tau\nXYtu3brhgQceQHBwMJKSkvDFF18Ybuc5t44+ffrg999/x+nTpwEAhw4dwq5du3DXXXcB4Hm3NlPO\n7/79+1FRUWG0T3h4ODp06MDXgJoku18psSFyc3Oh0WgQEhJitD0kJASZmZk2GpXzEkJg+vTp6NOn\nDzp06AAAhvNc02tw4cKFRh+js1ixYgUOHDiAvXv3VruN59zyzp8/j0WLFmH69Ol47bXXsGfPHjz/\n/PNQKBR4/PHHec6t5JVXXkFBQQHatm0LuVwOjUaDt956Cw8//DAAvtetzZTzm5mZCTc3NzRr1qza\nPvycpabIKQNqPUmSjH4XQlTbRuabOnUqDh8+jF27dlW7ja+B5WRkZOCFF17Axo0b4e7uXut+POeW\no9Vq0a1bN6SkpAAAkpKScOzYMSxatAiPP/64YT+ec8v67rvv8PXXX2P58uVISEhAamoqpk2bhvDw\ncIwfP96wH8+7dTXk/PI1oKbKKUs+goKCIJfLq31Lzs7OrvaNm8zz3HPPYe3atdi6dSsiIiIM20ND\nQwGAr4EF7d+/H9nZ2ejatStcXFzg4uKC7du348MPP4SLi4vhvPKcW05YWBjat29vtK1du3aGyc18\nn1vHv/71L7z66qsYO3YsOnbsiMceewwvvvgi5s2bB4Dn3dpMOb+hoaFQqVTIy8urdR+ipsQpA2o3\nNzd07doVmzZtMtq+adMm9OrVy0ajci5CCEydOhWrV6/Gli1bEBsba3R7bGwsQkNDjV4DlUqF7du3\n8zVooDvvvBNHjhxBamqq4adbt24YN24cUlNT0bJlS55zC+vdu3e1dpCnT59GdHQ0AL7PraW0tBQy\nmfHHk1wuN7TN43m3LlPOb9euXeHq6mq0z9WrV3H06FG+BtQ02Ww6pJWtWLFCuLq6iv/973/i+PHj\nYtq0acLLy0ukp6fbemhO4ZlnnhF+fn5i27Zt4urVq4af0tJSwz5vv/228PPzE6tXrxZHjhwRDz/8\nsAgLCxOFhYU2HLlzubHLhxA855a2Z88e4eLiIt566y1x5swZ8c033whPT0/x9ddfG/bhObe88ePH\nixYtWoh169aJtLQ0sXr1ahEUFCRefvllwz487+YpKioSBw8eFAcPHhQAxIIFC8TBgwfFhQsXhBCm\nnd+nn35aREREiM2bN4sDBw6IgQMHik6dOgm1Wm2rp0VkM04bUAshxMcffyyio6OFm5ub6NKli6Gl\nG5kPQI0/S5YsMeyj1WrF7NmzRWhoqFAoFOKOO+4QR44csd2gndDNATXPueX9/PPPokOHDkKhUIi2\nbduKzz//3Oh2nnPLKywsFC+88IKIiooS7u7uomXLlmLmzJlCqVQa9uF5N8/WrVtr/Bs+fvx4IYRp\n57esrExMnTpVBAQECA8PDzFy5Ehx8eJFGzwbItuThBDCNrlxIiIiIiLH55Q11EREREREjYUBNRER\nERGRGRhQExERERGZgQE1EREREZEZGFATEREREZmBATURERERkRkYUBMRERERmYEBNRERERGRGRhQ\nE5FdSE5ORufOnRv9uNu2bYMkScjPz2/0YxMRkXPgSolEZHWSJN3y9vHjx2PhwoVQKpUIDAxspFHp\nqFQqXL9+HSEhIXWOk4iIqCYMqInI6jIzMw3//9133+H111/HqVOnDNs8PDzg5+dni6ERERGZjSUf\nRGR1oaGhhh8/Pz9IklRt280lHxMmTMC9996LlJQUhISEwN/fH3PmzIFarca//vUvBAQEICIiAosX\nLzY61uXLl/HQQw+hWbNmCAwMxKhRo5Cenl7r2G4u+Vi6dCn8/f2xYcMGtGvXDt7e3hg2bBiuXr1a\n62Pk5eVh3LhxaN68OTw8PBAXF4clS5aYc8qIiMiBMKAmIru1ZcsWXLlyBTt27MCCBQuQnJyMkSNH\nolmzZvj777/x9NNP4+mnn0ZGRgYAoLS0FAMGDIC3tzd27NiBXbt2GQJilUpl8nFLS0vx7rvv4v/+\n7/+wY8cOXLx4ES+99FKt+//73//G8ePH8dtvv+HEiRNYtGgRgoKCzH7+RETkGFxsPQAiotoEBATg\nww8/hEwmQ3x8PObPn4/S0lK89tprAIAZM2bg7bffxh9//IGxY8dixYoVkMlk+PLLLw310EuWLIG/\nvz+2bduGIUOGmHTciooKfPrpp2jVqhUAYOrUqXjjjTdq3f/ixYtISkpCt27dAAAxMTFmPGsiInI0\nDKiJyG4lJCRAJqu6kBYSEoIOHToYfpfL5QgMDER2djYAYP/+/Th79ix8fHyMHqe8vBznzp0z+bie\nnp6GYBoAwsLCDMeoyTPPPIMxY8bgwIEDGDJkCO6991706tXL5OMREZFjY0BNRHbL1dXV6HdJkmrc\nptVqAQBarRZdu3bFN998U+2xmjdvbtZxbzV/e/jw4bhw4QJ++eUXbN68GXfeeSemTJmCd9991+Rj\nEhGR42JATUROo0uXLvjuu+8QHBwMX1/fRj128+bNMWHCBEyYMAF9+/bFv/71LwbURERNBCclEpHT\nGDduHIKCgjBq1Cjs3LkTaWlp2L59O1544QVcunTJasd9/fXX8dNPP+Hs2bM4duwY1q1bh3bt2lnt\neEREZF8YUBOR0/D09MSOHTsQFRWF0aNHo127dpg4cSLKysqsmrF2c3PDjBkzkJiYiDvuuANyuRwr\nVqyw2vGIiMi+cGEXIiIiIiIzMENNRERERGQGBtRERERERGZgQE1EREREZAYG1EREREREZmBATURE\nRERkBgbURERERERmYEBNRERERGQGBtRERERERGZgQE1EREREZAYG1EREREREZmBATURERERkhv8H\nJp97uSo8quYAAAAASUVORK5CYII=\n"
+    }
+   },
+   "cell_type": "markdown",
+   "id": "13fa7a12-eca7-436f-91da-f3feaac19118",
+   "metadata": {},
+   "source": [
+    "![CPU_GB_Mem_requested-on-used_per_sec.png](attachment:d1a3d796-81df-4a8f-90d5-9324e9d1781d.png)\n",
+    "\n",
+    "In this memory usage graph we can conclude that the memory usage for both methods is the same (due to rioxarray way to handle the memory)\n",
+    "\n",
+    "![CPU_Local_IOps_in_MB.png](attachment:8b8360f6-274e-47ae-b601-dfb2fb33e812.png)\n",
+    "\n",
+    "The local IO per second graph shows that the tiled write indeed produces a higher level of IOps to make the file write faster, but it is restricted by the speed of your SSD.\n",
+    "\n",
+    "![CPU_SUM_-percent-CPU_-on-s.png](attachment:77035e47-e99e-4a83-a3c2-cee41de28d29.png)\n",
+    "\n",
+    "The CPU usage shows 2 peaks for the computing with dask, only 1 peak with the tiled write scenario\n",
+    "\n",
+    "**Conclusion for Memory usage**\n",
+    "\n",
+    "* Both method consumes the same amount of memory\n",
+    "* Cpu usage is higher with dask\n",
+    "* You must use the tiled write in order to gain time"
    ]
   },
   {
@@ -2539,7 +2583,6 @@
     "# Conclusions and recommandations about parallel write\n",
     "\n",
     "* Huge time gain without precision loss\n",
-    "* Better RAM usage\n",
     "* You have to carefully choose your chunk size using a multiple of the cog size"
    ]
   },
@@ -2653,7 +2696,7 @@
    "source": [
     "# Profiling you application\n",
     "\n",
-    "## Example with a processing chain"
+    "### Example to show the performance graph of parrallel writing"
    ]
   },
   {
@@ -2663,30 +2706,34 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "lis_slurm_script_monitored = \"\"\"#!/bin/bash\n",
-    "#SBATCH --job-name=LIS_singularity_{cpus}_{mem}_monitored\n",
-    "#SBATCH -N 1\n",
-    "#SBATCH -n 1\n",
-    "#SBATCH -c {cpus}\n",
-    "#SBATCH --mem={mem}G            # memory per node\n",
-    "#SBATCH --time=00:30:00             # Wall Time\n",
-    "#SBATCH --account=campus       # MANDATORY : account  ( launch myaccounts to list your accounts)\n",
-    "#SBATCH --export=none               # To start the job with a clean environnement and source of ~/.bashrc\n",
-    "#SBATCH --output={share_work_dir}/LIS-{cpus}-{mem}-%j.out\n",
-    "#SBATCH --error={share_work_dir}/LIS-{cpus}-{mem}-%j.err\n",
-    "\n",
-    "module load singularity\n",
+    "greenit_slurm_script_monitored = \"\"\"#!/bin/bash\n",
+    "#\n",
+    "# Load monitoring module\n",
     "module load monitoring/2.2\n",
+    "#\n",
+    "# Use of SLURM $TMPDIR  of the interactive job\n",
+    "WORKDIR=$TMPDIR\n",
+    "SLURM_SUBMIT_DIR=/work/scratch/env/romaint/greenit/perfos_greenit\n",
+    "#\n",
+    "cd \"$WORKDIR\"\n",
+    "#\n",
+    "# Give model name\n",
+    "#\n",
+    "MODEL_NAME=test_perfo_greenit\n",
+    "#\n",
+    "# launch start_monitoring.sh to start monitoring of the job ( --name option mandatory)\n",
+    "#\n",
+    "start_monitoring.sh --name $MODEL_NAME --io local\n",
+    "#\n",
+    "# lauch program\n",
+    "#\n",
+    "source /work/scratch/env/romaint/greenit/env_greenit_py312/bin/activate\n",
+    "python3 /work/scratch/env/romaint/greenit/test_parallelisation.py\n",
+    "#\n",
+    "# Stop monitoring -it will generate output_monitoring directory\n",
+    "#\n",
+    "stop_monitoring.sh --name $MODEL_NAME\n",
     "\n",
-    "cd $TMPDIR\n",
-    "\n",
-    "pin={pin}\n",
-    "pout={LIS_output_path}\n",
-    "\n",
-    "start_monitoring.sh --name lis-{cpus}-{mem} --io local\n",
-    "# monter les répertoires /data \n",
-    "singularity exec -B {share_work_dir}:/data  -B /work:/work {lis_singularity_image_path} {lis_launch_script_path} $pin $pout\n",
-    "stop_monitoring.sh --name lis-{cpus}-{mem}\n",
     "\"\"\""
    ]
   },

From 96cf0d64113b11a4612a3fdc0f006f16a5d2d827 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Tue, 4 Feb 2025 16:37:45 +0100
Subject: [PATCH 10/37] Rearrange titles

---
 tuto_greenit.ipynb | 78 ++++++++++++++++++++++++----------------------
 1 file changed, 41 insertions(+), 37 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 74a45cd..7ac959e 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -640,13 +640,12 @@
    "id": "0e86755c-f505-4d6b-830e-396485e12054",
    "metadata": {},
    "source": [
-    "# Calculation of the Average NDVI on a satellite image\n",
+    "# Calculation of the Average NDVI on a Sentinel 2 image\n",
     "\n",
-    "In this example, we will use what we have learned to: \n",
+    "In this example, we will use standard python libraries to: \n",
     "\n",
     "1. Read the data from the disk and stack them.\n",
     "2. Calculate the associated NDVI, which combines multi-band information into a single band.\n",
-    "3. Reduce the information by calculating the average NDVI within a window.\n",
     "4. Write the resulting image to the disk.\n",
     "\n",
     "First, let's read the data we need to perform the NDVI."
@@ -824,35 +823,22 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 1,
    "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "Warning 1: Invalid value for NUM_THREADS: \n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif...: 100% [**************************************************] (1m 34s)\n"
+     "ename": "ModuleNotFoundError",
+     "evalue": "No module named 'otbApplication'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[1], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01motbApplication\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01motb\u001b[39;00m\n\u001b[1;32m      3\u001b[0m out_ndvi_otb_py\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m      4\u001b[0m \u001b[38;5;66;03m#Compute NDVI with OTB in python\u001b[39;00m\n",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'otbApplication'"
      ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "0"
-      ]
-     },
-     "execution_count": 9,
-     "metadata": {},
-     "output_type": "execute_result"
     }
    ],
    "source": [
@@ -861,10 +847,10 @@
     "out_ndvi_otb_py=\"/work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif\"\n",
     "#Compute NDVI with OTB in python\n",
     "app_ndvi_otb = otb.Registry.CreateApplication(\"BandMath\")\n",
-    "#app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n",
-    "#app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n",
-    "app_ndvi_otb.SetParameterStringList(\"il\",[phr_product_cog])\n",
-    "app_ndvi_otb.SetParameterString(\"exp\",\"(im1b4-im1b1)/(im1b4+im1b1)\")\n",
+    "app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n",
+    "app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n",
+    "#app_ndvi_otb.SetParameterStringList(\"il\",[phr_product_cog])\n",
+    "#app_ndvi_otb.SetParameterString(\"exp\",\"(im1b4-im1b1)/(im1b4+im1b1)\")\n",
     "app_ndvi_otb.SetParameterString(\"out\",out_ndvi_otb_py)\n",
     "app_ndvi_otb.SetParameterInt(\"ram\",2048)\n",
     "app_ndvi_otb.ExecuteAndWriteOutput()"
@@ -913,9 +899,9 @@
    "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386",
    "metadata": {},
    "source": [
-    "# Testing optimisation methods for the computing of NDVI\n",
+    "# Different optimisation methods for the computing of NDVI\n",
     "\n",
-    "Here we will try to use numba for ndvi computation"
+    "Let's see if we can improve the computing time with well known python libraries, with a Sentinel 2 product (size of about 2 Go if you include the bands you need only), every file is single band"
    ]
   },
   {
@@ -1018,14 +1004,28 @@
   },
   {
    "cell_type": "markdown",
-   "id": "eace634b-3c4c-4db8-8c4f-92ffea2bf1e6",
+   "id": "c9229d23-c553-4c30-8ae0-b21cee07879e",
    "metadata": {},
    "source": [
-    "## Conclusion\n",
+    "**Conclusion for a Sentinel 2 product with single band products (~2Go)**\n",
     "\n",
-    "* Using RIOXarray + dask on single band products is counter productive in a case of a simple calculation as NDVI.\n",
+    "* Using **RIOXarray + dask on single band products is counter productive** in a case of a simple calculation as NDVI.\n",
     "* Xarray + applyufunc also leads to low performance compared to opening the whole product with rasterio.\n",
-    "* Using numba do enhances the performance but with a low pourcentage \n"
+    "* Using numba do enhances the performance but with less than 10%\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eace634b-3c4c-4db8-8c4f-92ffea2bf1e6",
+   "metadata": {},
+   "source": [
+    "## The size of your image should lead you to use the right library\n",
+    "\n",
+    "**If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask**\n",
+    "\n",
+    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 4 Go**\n",
+    "\n",
+    "**If your product is multi-band, use RIOXarray or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances**"
    ]
   },
   {
@@ -1035,7 +1035,11 @@
     "tags": []
    },
    "source": [
-    "# Optimize dask parameters and improve write speeds on disk"
+    "# Use case : I have a bunch of product which average size is 10Go\n",
+    "\n",
+    "In that case you'll need to use parralellisation libraries such as dask or rioxarray with parallel read and write support to speedup your processing. Let's see what are the recommended parameters to optimize that\n",
+    "\n",
+    "## Optimize dask parameters and improve write speeds on disk"
    ]
   },
   {
@@ -2787,7 +2791,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.11.10"
   }
  },
  "nbformat": 4,

From d09ee4cf22f7c816919f297bbec54da26a10a8d0 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Tue, 4 Feb 2025 17:07:26 +0100
Subject: [PATCH 11/37] Rearrange titles part 2

---
 tuto_greenit.ipynb | 226 +++++++++++++++++++++------------------------
 1 file changed, 106 insertions(+), 120 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 7ac959e..90e2bc6 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -1007,7 +1007,7 @@
    "id": "c9229d23-c553-4c30-8ae0-b21cee07879e",
    "metadata": {},
    "source": [
-    "**Conclusion for a Sentinel 2 product with single band products (~2Go)**\n",
+    "**Conclusion for a Sentinel 2 product with single band files (~2Go)**\n",
     "\n",
     "* Using **RIOXarray + dask on single band products is counter productive** in a case of a simple calculation as NDVI.\n",
     "* Xarray + applyufunc also leads to low performance compared to opening the whole product with rasterio.\n",
@@ -1023,11 +1023,106 @@
     "\n",
     "**If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask**\n",
     "\n",
-    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 4 Go**\n",
+    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 2 Go**\n",
     "\n",
     "**If your product is multi-band, use RIOXarray or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances**"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43",
+   "metadata": {},
+   "source": [
+    "# Optimizing the size of your data\n",
+    "\n",
+    "## Simple LZW Compression\n",
+    "\n",
+    "Can be done with gdal-translate\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "69b8bd27-926e-4373-86d1-4dafe7fdbeed",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Register GDAL format drivers and configuration options with a\n",
+    "# context manager.\n",
+    "with rasterio.Env():\n",
+    "\n",
+    "    # Write an array as a raster band to a new 8-bit file. For\n",
+    "    # the new file's profile, we start with the profile of the source\n",
+    "    profile = raster.profile\n",
+    "\n",
+    "    # And then change the band count to 1, set the\n",
+    "    # dtype to uint8, and specify LZW compression.\n",
+    "    profile.update(compress='lzw')\n",
+    "\n",
+    "    with rasterio.open(os.environ['TMPDIR'] + '/compressed.tif', 'w', **profile) as dst:\n",
+    "        dst.write(raster.read(1), 1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9f77319c-80a1-4060-92ff-32a5040f834f",
+   "metadata": {},
+   "source": [
+    "## CoG without overview"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "906973ba-32f1-473c-93dd-302296177950",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with rasterio.Env():\n",
+    "    profile = raster.profile\n",
+    "    profile.update(tiled=True, compress='lzw', blockxsize=512, blockysize=512)\n",
+    "    with rasterio.open(os.environ['TMPDIR'] + '/cog.tif', 'w', **profile) as dst:\n",
+    "        dst.write(raster.read(1), 1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "1071ba31-2e9b-4da5-9d94-9e6ea9a5b1db",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "^C\n"
+     ]
+    }
+   ],
+   "source": [
+    "!du -sh $TMPDIR/*"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5f7263f1-5d4f-4025-8c74-cd83d10210aa",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "## Time gain after compression for an NDVI computing on a PLEIADES image\n",
+    "\n",
+    "Initial product size : 12 Go\n",
+    "\n",
+    "### LZW\n",
+    "Product size : \n",
+    "\n",
+    "### CoG\n",
+    "Product size : 7 Go"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "56e25516-3d90-4ee6-8c4b-085b94d3d096",
@@ -1037,6 +1132,8 @@
    "source": [
     "# Use case : I have a bunch of product which average size is 10Go\n",
     "\n",
+    "In this example we will use a PLEIADES product, which size is 13Go\n",
+    "\n",
     "In that case you'll need to use parralellisation libraries such as dask or rioxarray with parallel read and write support to speedup your processing. Let's see what are the recommended parameters to optimize that\n",
     "\n",
     "## Optimize dask parameters and improve write speeds on disk"
@@ -1047,7 +1144,7 @@
    "id": "d3ab5833-1672-478a-8b26-0ed2bc4d6ee1",
    "metadata": {},
    "source": [
-    "## Find the right chunk size for dask\n",
+    "### Find the right chunk size for dask\n",
     "\n",
     "Chunk size is becoming very important when your data size grows. You can let chunks=True to dask which will automatically determine a chunk size\n",
     "Most of the time this chunk size is coherent but users can tweak it to be more efficient.\n",
@@ -2271,7 +2368,7 @@
    "source": [
     "start=time.perf_counter()\n",
     "reading_chunks = (-1,2048,2048)\n",
-    "data_array_manualchunks, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product], reading_chunks, False)\n",
+    "data_array_manualchunks, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product_cog], reading_chunks, False)\n",
     "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])[None,:,:]\n",
     "ndvi_array.compute()\n",
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_manualchunks.tif\")\n",
@@ -2285,27 +2382,19 @@
    "id": "8622d62f-a986-4893-a932-a44bc2e5c496",
    "metadata": {},
    "source": [
-    "## Conclusion\n",
+    "### Conclusion\n",
     "\n",
     "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. We recommand in that case defining chunks using \"(-1,sizex,sizey)\". The time gain can go about 20% !\n",
     "\n",
     "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of 8 for read/write efficiency in RAM."
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "8cfce203-c7c2-4553-ac33-9d483f451263",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  },
   {
    "cell_type": "markdown",
    "id": "f7746770-c831-4598-96ff-f13e1d372e1d",
    "metadata": {},
    "source": [
-    "# Write with dask vs tiled write with rioxarray "
+    "## Write your image with dask vs tiled write with rioxarray "
    ]
   },
   {
@@ -2539,7 +2628,7 @@
     "tags": []
    },
    "source": [
-    "## Performance graphs\n",
+    "## Performance graphs with parallelisation methods\n",
     "\n",
     "**The first computing stops at 53s in these graphs that were generated using a slurm script with the exact same code as above**"
    ]
@@ -2584,113 +2673,12 @@
    "id": "ffd1783b-a0e3-43f2-861d-13d043653dc6",
    "metadata": {},
    "source": [
-    "# Conclusions and recommandations about parallel write\n",
+    "## Conclusions and recommandations about parallel write\n",
     "\n",
     "* Huge time gain without precision loss\n",
     "* You have to carefully choose your chunk size using a multiple of the cog size"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43",
-   "metadata": {},
-   "source": [
-    "# Optimizing the size of your data\n",
-    "\n",
-    "## Simple LZW Compression\n",
-    "\n",
-    "Can be done with gdal-translate\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "69b8bd27-926e-4373-86d1-4dafe7fdbeed",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Register GDAL format drivers and configuration options with a\n",
-    "# context manager.\n",
-    "with rasterio.Env():\n",
-    "\n",
-    "    # Write an array as a raster band to a new 8-bit file. For\n",
-    "    # the new file's profile, we start with the profile of the source\n",
-    "    profile = raster.profile\n",
-    "\n",
-    "    # And then change the band count to 1, set the\n",
-    "    # dtype to uint8, and specify LZW compression.\n",
-    "    profile.update(compress='lzw')\n",
-    "\n",
-    "    with rasterio.open(os.environ['TMPDIR'] + '/compressed.tif', 'w', **profile) as dst:\n",
-    "        dst.write(raster.read(1), 1)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "9f77319c-80a1-4060-92ff-32a5040f834f",
-   "metadata": {},
-   "source": [
-    "## CoG without overview"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "906973ba-32f1-473c-93dd-302296177950",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "with rasterio.Env():\n",
-    "    profile = raster.profile\n",
-    "    profile.update(tiled=True, compress='lzw', blockxsize=512, blockysize=512)\n",
-    "    with rasterio.open(os.environ['TMPDIR'] + '/cog.tif', 'w', **profile) as dst:\n",
-    "        dst.write(raster.read(1), 1)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "1071ba31-2e9b-4da5-9d94-9e6ea9a5b1db",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "du: cannot access '/tmp/slurm-31919932/*': No such file or directory\n"
-     ]
-    }
-   ],
-   "source": [
-    "!du -sh $TMPDIR/*"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "5f7263f1-5d4f-4025-8c74-cd83d10210aa",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "## Time gain after compression for an NDVI computing\n",
-    "\n",
-    "### LZW\n",
-    "\n",
-    "\n",
-    "### CoG\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6fb50a83-c68e-4a96-852c-85aa8ee90597",
-   "metadata": {},
-   "source": [
-    "Conclusion\n",
-    "\n"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "85b9e4b2-7621-48ac-8fa7-14789d40a34a",
@@ -2698,9 +2686,7 @@
     "tags": []
    },
    "source": [
-    "# Profiling you application\n",
-    "\n",
-    "### Example to show the performance graph of parrallel writing"
+    "# Profiling your processing chain with TREX monitoring"
    ]
   },
   {

From 69c7b323099d8b1d92a9d59a42921953f576b290 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Wed, 5 Feb 2025 10:57:56 +0100
Subject: [PATCH 12/37] Continue reorganisation with better conclusions

---
 tuto_greenit.ipynb | 730 +++++++--------------------------------------
 1 file changed, 104 insertions(+), 626 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 90e2bc6..f4c86d6 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -7,7 +7,7 @@
     "tags": []
    },
    "source": [
-    "# Good pratices\n",
+    "# Good pratices for general python coding\n",
     "\n",
     "## for loop optimisations : benefits of using numpy\n",
     "\n",
@@ -146,6 +146,14 @@
     "# A Green IT approach for coding image processing chains"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "fb2fcaf9-5b7b-4f37-b457-6e01e5035fbf",
+   "metadata": {},
+   "source": [
+    "In this notebook we will look at a green IT approach of coding, via good pratices in Python. We will have a look at Dask, RioXarray, rasterio, numpy... all these libraries that are widely used in the satellite image processing chains. "
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "9a7fe0ae-0549-47cb-af0c-04a83d27324e",
@@ -204,17 +212,12 @@
   },
   {
    "cell_type": "markdown",
-   "id": "fb2fcaf9-5b7b-4f37-b457-6e01e5035fbf",
+   "id": "94bd1ed8-f2e6-47b9-9b82-94fd0d40ec59",
    "metadata": {},
    "source": [
-    "## Downscaling rasters thanks to dask\n",
+    "## Creating the Dask Cluster\n",
     "\n",
-    "In this notebook we will look at a green IT approach of coding, via good pratices in Python. We will have a look at Dask, RioXarray, rasterio, numpy... all these libraries that are widely used in the satellite image processing chains. \n",
-    "In order to use dask we first need to create a local cluster.\n",
-    "\n",
-    "### Creating the Dask Cluster\n",
-    "\n",
-    "First let's import libraries needed for this tutorial and create our dask [LocalCluster](https://docs.dask.org/en/stable/deploying-python.html#localcluster) which allow us to create workers and use [dask's dashboard](https://docs.dask.org/en/latest/dashboard.html).\n"
+    "In order to use dask we first need to create a [LocalCluster](https://docs.dask.org/en/stable/deploying-python.html#localcluster) which allow us to create workers and use [dask's dashboard](https://docs.dask.org/en/latest/dashboard.html).\n"
    ]
   },
   {
@@ -562,12 +565,7 @@
    "id": "2a86c2ad-975b-4ed5-a8b0-8add6c8714d4",
    "metadata": {},
    "source": [
-    "Dask return an url where the dashboard is availaible (usually http://127.0.0.1:8787/status). This is not a tutorial on how to use this dashboard, but we recommend using it in a separate window while using this notebook.\n",
-    "\n",
-    "### Open raster thanks to rioxarray\n",
-    "\n",
-    "Here we are going to open the raster data required for this tutorial, the RGB bands from a Sentinel-2 acquisition. To do this, we're going to use rioxarray and, more specifically, the [open_rasterio](https://corteva.github.io/rioxarray/html/rioxarray.html#rioxarray-open-rasterio) method, which opens the images lazily (without loading data into memory) and returns a `dask.array` object. \n",
-    "From this method we will use the ``chunks`` and ``lock`` arguments, which respectively set a chunk size and limit access to the data to one thread at a time to avoid read problems. Here ``chunks`` is set to ``True`` to allow dask to automatically size chunks.\n"
+    "Dask return an url where the dashboard is availaible (usually http://127.0.0.1:8787/status). This is not a tutorial on how to use this dashboard, but we recommend using it in a separate window while using this notebook."
    ]
   },
   {
@@ -640,7 +638,7 @@
    "id": "0e86755c-f505-4d6b-830e-396485e12054",
    "metadata": {},
    "source": [
-    "# Calculation of the Average NDVI on a Sentinel 2 image\n",
+    "# Use Case : Calculation of the Average NDVI on a Sentinel 2 image\n",
     "\n",
     "In this example, we will use standard python libraries to: \n",
     "\n",
@@ -648,7 +646,7 @@
     "2. Calculate the associated NDVI, which combines multi-band information into a single band.\n",
     "4. Write the resulting image to the disk.\n",
     "\n",
-    "First, let's read the data we need to perform the NDVI."
+    "First, let's read the data we need to perform the NDVI with RIOXarray."
    ]
   },
   {
@@ -656,6 +654,11 @@
    "id": "fc56a7e0-edb5-4635-b5a2-0b274d08ce38",
    "metadata": {},
    "source": [
+    "### Open raster thanks to rioxarray\n",
+    "\n",
+    "Here we are going to open the raster data required for this tutorial, the RGB bands from a Sentinel-2 acquisition. To do this, we're going to use rioxarray and, more specifically, the [open_rasterio](https://corteva.github.io/rioxarray/html/rioxarray.html#rioxarray-open-rasterio) method, which opens the images lazily (without loading data into memory) and returns a `dask.array` object. \n",
+    "From this method we will use the ``chunks`` and ``lock`` arguments, which respectively set a chunk size and limit access to the data to one thread at a time to avoid read problems. Here ``chunks`` is set to ``True`` to allow dask to automatically size chunks.\n",
+    "\n",
     "When the data is read, we can express the NDVI calculation as if it were a numpy array. We add ``[None, :, :]`` to keep the shape as ``(bands, rows, cols)``. Then we can apply reduction on the dask.array and use ``compute()`` on it to triger the computation."
    ]
   },
@@ -894,12 +897,23 @@
     "otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/output_greenit/img_ndvi_otb_cpp.tif\" "
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "c1b01381-b598-4e69-9389-255da7cc72cf",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
+    "\n",
+    "The compute time with OTB for a simple NDVI computing is lower than standard python computing, this is mainly due to the overkill usage of dask with single band files.\n",
+    "Now let's see how to optimize our initial code, to determine when it using dask becomes important"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386",
    "metadata": {},
    "source": [
-    "# Different optimisation methods for the computing of NDVI\n",
+    "# Compute time optimisation methods for NDVI\n",
     "\n",
     "Let's see if we can improve the computing time with well known python libraries, with a Sentinel 2 product (size of about 2 Go if you include the bands you need only), every file is single band"
    ]
@@ -909,6 +923,7 @@
    "execution_count": 41,
    "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
    "metadata": {
+    "scrolled": true,
     "tags": []
    },
    "outputs": [
@@ -1007,11 +1022,19 @@
    "id": "c9229d23-c553-4c30-8ae0-b21cee07879e",
    "metadata": {},
    "source": [
-    "**Conclusion for a Sentinel 2 product with single band files (~2Go)**\n",
+    "## Conclusion for a Sentinel 2 product with single band files (~3Go for the full product)**\n",
     "\n",
-    "* Using **RIOXarray + dask on single band products is counter productive** in a case of a simple calculation as NDVI.\n",
+    "* Using **RIOXarray + dask on single band product is counter productive**.\n",
     "* Xarray + applyufunc also leads to low performance compared to opening the whole product with rasterio.\n",
-    "* Using numba do enhances the performance but with less than 10%\n"
+    "* Using numba do enhances the performance but with less than 10%, it is more suitable for numpy vectorize [optimisation]( https://numba.readthedocs.io/en/stable/user/vectorize.html#the-vectorize-decorator)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43",
+   "metadata": {},
+   "source": [
+    "# The importance of reducing the size of your data"
    ]
   },
   {
@@ -1023,21 +1046,19 @@
     "\n",
     "**If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask**\n",
     "\n",
-    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 2 Go**\n",
+    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 3 Go**\n",
+    "\n",
+    "**If your product is multi-band, use RIOXarray or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances**\n",
     "\n",
-    "**If your product is multi-band, use RIOXarray or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances**"
+    "Let's see the different types of compression and how they improve performance"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43",
+   "id": "7a95208c-4974-4c85-a09d-e7cd99fd09cd",
    "metadata": {},
    "source": [
-    "# Optimizing the size of your data\n",
-    "\n",
-    "## Simple LZW Compression\n",
-    "\n",
-    "Can be done with gdal-translate\n"
+    "## Simple LZW Compression with RasterIO"
    ]
   },
   {
@@ -1068,7 +1089,7 @@
    "id": "9f77319c-80a1-4060-92ff-32a5040f834f",
    "metadata": {},
    "source": [
-    "## CoG without overview"
+    "## Create a CoG without overview"
    ]
   },
   {
@@ -1114,7 +1135,7 @@
    "source": [
     "## Time gain after compression for an NDVI computing on a PLEIADES image\n",
     "\n",
-    "Initial product size : 12 Go\n",
+    "Initial product size : 13 Go\n",
     "\n",
     "### LZW\n",
     "Product size : \n",
@@ -1130,13 +1151,13 @@
     "tags": []
    },
    "source": [
-    "# Use case : I have a bunch of product which average size is 10Go\n",
+    "# Use case : I have to use multiple products which average size is > 10Go\n",
     "\n",
     "In this example we will use a PLEIADES product, which size is 13Go\n",
     "\n",
     "In that case you'll need to use parralellisation libraries such as dask or rioxarray with parallel read and write support to speedup your processing. Let's see what are the recommended parameters to optimize that\n",
     "\n",
-    "## Optimize dask parameters and improve write speeds on disk"
+    "## Optimize dask parameters"
    ]
   },
   {
@@ -1740,608 +1761,35 @@
    "source": [
     "start=time.perf_counter()\n",
     "ndvi_array = (data_array_autochunks[3] - data_array_autochunks[0]) / (data_array_autochunks[3] + data_array_autochunks[0])\n",
-    "#mean_ndvi = ndvi_array.compute()\n",
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_autochunks.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
+    "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with automatic chunk size = {}s\".format((end - start)))"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 50,
+   "execution_count": 3,
    "id": "615be991-f60c-4353-8c18-fb4853c660eb",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "data": {
-      "text/html": [
-       "
    \n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "
    <xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
-       "dask.array<open_rasterio-171af4db274709fd77ecac80bd238d05<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 2048, 2048), chunktype=numpy.ndarray>\n",
-       "Coordinates:\n",
-       "  * band         (band) int64 32B 1 2 3 4\n",
-       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
-       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 8B 0\n",
-       "Attributes: (12/29)\n",
-       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
-       "    BlueDisplayChannel:   0\n",
-       "    DataType:             3\n",
-       "    GeometricLevel:       SENSOR\n",
-       "    GreenDisplayChannel:  0\n",
-       "    ImageID:              6967638101-1\n",
-       "    ...                   ...\n",
-       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
-       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
-       "    NoData:               0\n",
-       "    _FillValue:           0\n",
-       "    scale_factor:         1.0\n",
-       "    add_offset:           0.0
    "
-      ],
-      "text/plain": [
-       " Size: 13GB\n",
-       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 2048, 2048), chunktype=numpy.ndarray>\n",
-       "Coordinates:\n",
-       "  * band         (band) int64 32B 1 2 3 4\n",
-       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
-       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 8B 0\n",
-       "Attributes: (12/29)\n",
-       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
-       "    BlueDisplayChannel:   0\n",
-       "    DataType:             3\n",
-       "    GeometricLevel:       SENSOR\n",
-       "    GreenDisplayChannel:  0\n",
-       "    ImageID:              6967638101-1\n",
-       "    ...                   ...\n",
-       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
-       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
-       "    NoData:               0\n",
-       "    _FillValue:           0\n",
-       "    scale_factor:         1.0\n",
-       "    add_offset:           0.0"
-      ]
-     },
-     "execution_count": 50,
-     "metadata": {},
-     "output_type": "execute_result"
+     "ename": "NameError",
+     "evalue": "name 'ndvi_array' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[3], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mndvi_array\u001b[49m\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'ndvi_array' is not defined"
+     ]
     }
    ],
-   "source": []
+   "source": [
+    "ndvi_array"
+   ]
   },
   {
    "cell_type": "code",
@@ -2368,15 +1816,38 @@
    "source": [
     "start=time.perf_counter()\n",
     "reading_chunks = (-1,2048,2048)\n",
-    "data_array_manualchunks, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([phr_product_cog], reading_chunks, False)\n",
-    "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])[None,:,:]\n",
-    "ndvi_array.compute()\n",
+    "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
+    "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n",
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_manualchunks.tif\")\n",
-    "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
+    "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "32e153c5-1ccc-47b2-a5c2-1270ab9d75b1",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "ename": "NameError",
+     "evalue": "name 'ndvi_array' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[3], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mndvi_array\u001b[49m\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'ndvi_array' is not defined"
+     ]
+    }
+   ],
+   "source": [
+    "ndvi_array"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "8622d62f-a986-4893-a932-a44bc2e5c496",
@@ -2384,7 +1855,13 @@
    "source": [
     "### Conclusion\n",
     "\n",
-    "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. We recommand in that case defining chunks using \"(-1,sizex,sizey)\". The time gain can go about 20% !\n",
+    "Prefer using rxr.to_raster() instead of dask.compute() when your product size is getting bigger as the later seems to be more RAM consuming\n",
+    "\n",
+    "### Chunk size recommandation for multi band products\n",
+    "\n",
+    "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. We recommand in that case defining chunks using \"(-1,sizex,sizey)\" which creates chunks with concatenated bands. The time gain can be up to 20% !\n",
+    "\n",
+    "### Don't hesitate to test different chunk sizes before making a conclusion\n",
     "\n",
     "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of 8 for read/write efficiency in RAM."
    ]
@@ -2394,7 +1871,7 @@
    "id": "f7746770-c831-4598-96ff-f13e1d372e1d",
    "metadata": {},
    "source": [
-    "## Write your image with dask vs tiled write with rioxarray "
+    "## Improve disk write speed with rio.to_raster "
    ]
   },
   {
@@ -2616,6 +2093,7 @@
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr_rxr.tif\")\n",
+    "# Add the Tiled parameter to rxr to speed up the disk write\n",
     "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with RIOXarray + tiled write + product > 5Go = {}s\".format((end - start)))"

From b6e0e43ca6ba6bafb72fbdcf5aaccc5822ef1851 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Wed, 5 Feb 2025 11:08:29 +0100
Subject: [PATCH 13/37] Declare data_dir to improve user experience

---
 tuto_greenit.ipynb | 160 ++++++++++++++++++++++-----------------------
 1 file changed, 80 insertions(+), 80 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index f4c86d6..37c669f 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -203,11 +203,76 @@
    },
    "outputs": [],
    "source": [
-    "sentinel_2_dir = \"/work/scratch/data/romaint\"\n",
-    "s2_b4 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
-    "s2_b8 = f\"{sentinel_2_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
+    "data_dir = \"/work/scratch/data/romaint\"\n",
+    "s2_b4 = f\"{data_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
+    "s2_b8 = f\"{data_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
     "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
-    "phr_product_cog = \"/work/scratch/data/romaint/phr_cog.tif\""
+    "phr_product_cog = f\"{data_dir}/phr_cog.tif\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "19ad8906-30ee-4511-aed4-a0a32abd3458",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Defining usefull functions for our notebook"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "def open_raster_and_get_metadata(raster_paths: List[str], chunks: Union[int, Tuple, Dict, None], lock: Union[int,None]):\n",
+    "    \"\"\"\n",
+    "    Opens multiple raster files, extracts shared geospatial metadata, \n",
+    "    and returns the concatenated data along with resolution and CRS info.\n",
+    "\n",
+    "    Parameters:\n",
+    "    -----------\n",
+    "    raster_paths : List[str]\n",
+    "        Paths to the raster files.\n",
+    "    chunks : Union[int, Tuple, Dict, bool, None]\n",
+    "        Chunk sizes for Dask (bands, height, width).\n",
+    "\n",
+    "    Returns:\n",
+    "    --------\n",
+    "    Tuple[dask.array.Array, float, float, float, float, Union[str, CRS]]\n",
+    "        Concatenated raster data, x and y resolution, top-left coordinates, and CRS.\n",
+    "    \"\"\"\n",
+    "    results = []\n",
+    "    for raster_path in raster_paths:\n",
+    "        with rxr.open_rasterio(raster_path, chunks=chunks, lock=lock) as tif:\n",
+    "            reprojection = tif\n",
+    "            transform = reprojection.rio.transform()\n",
+    "            crs = reprojection.rio.crs\n",
+    "            x_res = transform[0]\n",
+    "            y_res = -transform[4]\n",
+    "            top_left_x = transform[2]\n",
+    "            top_left_y = transform[5]\n",
+    "            results.append(reprojection)\n",
+    "\n",
+    "    return da.concatenate(results), x_res, y_res, top_left_x, top_left_y, crs\n",
+    "\n",
+    "def create_raster(data: np.ndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n",
+    "    transform = Affine.translation(top_left_x, top_left_y) * Affine.scale(x_res, -y_res)\n",
+    "    with rasterio.open(\n",
+    "            output_file, \"w\",\n",
+    "            driver=\"GTiff\",\n",
+    "            height=data.shape[1],\n",
+    "            width=data.shape[2],\n",
+    "            count=data.shape[0],\n",
+    "            dtype=data.dtype,\n",
+    "            crs=crs,\n",
+    "            transform=transform\n",
+    "    ) as dst:\n",
+    "        dst.write(data)"
    ]
   },
   {
@@ -568,71 +633,6 @@
     "Dask return an url where the dashboard is availaible (usually http://127.0.0.1:8787/status). This is not a tutorial on how to use this dashboard, but we recommend using it in a separate window while using this notebook."
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "19ad8906-30ee-4511-aed4-a0a32abd3458",
-   "metadata": {
-    "tags": []
-   },
-   "source": [
-    "### Defining usefull functions for our notebook"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "def open_raster_and_get_metadata(raster_paths: List[str], chunks: Union[int, Tuple, Dict, None], lock: Union[int,None]):\n",
-    "    \"\"\"\n",
-    "    Opens multiple raster files, extracts shared geospatial metadata, \n",
-    "    and returns the concatenated data along with resolution and CRS info.\n",
-    "\n",
-    "    Parameters:\n",
-    "    -----------\n",
-    "    raster_paths : List[str]\n",
-    "        Paths to the raster files.\n",
-    "    chunks : Union[int, Tuple, Dict, bool, None]\n",
-    "        Chunk sizes for Dask (bands, height, width).\n",
-    "\n",
-    "    Returns:\n",
-    "    --------\n",
-    "    Tuple[dask.array.Array, float, float, float, float, Union[str, CRS]]\n",
-    "        Concatenated raster data, x and y resolution, top-left coordinates, and CRS.\n",
-    "    \"\"\"\n",
-    "    results = []\n",
-    "    for raster_path in raster_paths:\n",
-    "        with rxr.open_rasterio(raster_path, chunks=chunks, lock=lock) as tif:\n",
-    "            reprojection = tif\n",
-    "            transform = reprojection.rio.transform()\n",
-    "            crs = reprojection.rio.crs\n",
-    "            x_res = transform[0]\n",
-    "            y_res = -transform[4]\n",
-    "            top_left_x = transform[2]\n",
-    "            top_left_y = transform[5]\n",
-    "            results.append(reprojection)\n",
-    "\n",
-    "    return da.concatenate(results), x_res, y_res, top_left_x, top_left_y, crs\n",
-    "\n",
-    "def create_raster(data: np.ndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n",
-    "    transform = Affine.translation(top_left_x, top_left_y) * Affine.scale(x_res, -y_res)\n",
-    "    with rasterio.open(\n",
-    "            output_file, \"w\",\n",
-    "            driver=\"GTiff\",\n",
-    "            height=data.shape[1],\n",
-    "            width=data.shape[2],\n",
-    "            count=data.shape[0],\n",
-    "            dtype=data.dtype,\n",
-    "            crs=crs,\n",
-    "            transform=transform\n",
-    "    ) as dst:\n",
-    "        dst.write(data)"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "0e86755c-f505-4d6b-830e-396485e12054",
@@ -811,7 +811,7 @@
     "# Launch the computing with dask with the compute call\n",
     "mean_ndvi = ndvi_array.compute() \n",
     "crs=\"EPSG:4326\"\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_dask.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_dask.tif\")\n",
     "create_raster(mean_ndvi, output_file, x_res , y_res,\n",
     "                  top_left_x, top_left_y, crs)"
    ]
@@ -980,7 +980,7 @@
     "   \n",
     "ndvi_computed = compute_ndvi_numba(input_data_b4,input_data_b8)\n",
     "crs=\"EPSG:4326\"\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_numba.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_numba.tif\")\n",
     "create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with Raster IO + numba = {}s\".format((end - start)))\n",
@@ -994,7 +994,7 @@
     "\n",
     "ndvi_computed = compute_ndvi_std(input_data_b4,input_data_b8)\n",
     "crs=\"EPSG:4326\"\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_without_numba.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_without_numba.tif\")\n",
     "create_raster(ndvi_computed, output_file, x_res , y_res,top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with Raster IO + without numba = {}s\".format((end - start)))\n",
@@ -1003,15 +1003,15 @@
     "input_data_b4 =  xarray.open_dataarray(s2_b4)\n",
     "input_data_b8 =  xarray.open_dataarray(s2_b8)\n",
     "ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n",
-    "ndvi_computed.rio.to_raster(\"/work/scratch/data/romaint/output_greenit/ndvi_ufunc.tif\")\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_ufunc.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_ufunc.tif\")\n",
+    "ndvi_computed.rio.to_raster(output_file)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with Xarray + apply ufunc = {}s\".format((end - start)))\n",
     "\n",
     "start = time.perf_counter()\n",
     "input_data, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], (-1,2200,2200), False)\n",
     "ndvi_array = compute_ndvi_dask(input_data[0],input_data[1])\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_dask.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_dask.tif\")\n",
     "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n"
@@ -1761,7 +1761,7 @@
    "source": [
     "start=time.perf_counter()\n",
     "ndvi_array = (data_array_autochunks[3] - data_array_autochunks[0]) / (data_array_autochunks[3] + data_array_autochunks[0])\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_autochunks.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_autochunks.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with automatic chunk size = {}s\".format((end - start)))"
@@ -1818,7 +1818,7 @@
     "reading_chunks = (-1,2048,2048)\n",
     "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
     "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_manualchunks.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_manualchunks.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))"
@@ -1915,7 +1915,7 @@
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "ndvi_phr.compute()\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_phr.tif\")\n",
     "ndvi_phr.rio.to_raster(output_file)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with RIOXarray + dask compute + product > 5Go = {}s\".format((end - start)))"
@@ -2092,7 +2092,7 @@
     "reading_chunks = (-1,2048,2048)\n",
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
-    "output_file = Path(\"/work/scratch/data/romaint/output_greenit/ndvi_phr_rxr.tif\")\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_phr_rxr.tif\")\n",
     "# Add the Tiled parameter to rxr to speed up the disk write\n",
     "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
     "end = time.perf_counter()\n",
@@ -2106,7 +2106,7 @@
     "tags": []
    },
    "source": [
-    "## Performance graphs with parallelisation methods\n",
+    "## Performance graphs dask compute vs rio.to_raster with tiled write\n",
     "\n",
     "**The first computing stops at 53s in these graphs that were generated using a slurm script with the exact same code as above**"
    ]

From 53d0dcbff69fd4e640861e485bb0d3af8f22a213 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Wed, 5 Feb 2025 11:32:34 +0100
Subject: [PATCH 14/37] Add ndvi compressed compute example

---
 tuto_greenit.ipynb | 73 ++++++++++++++++++++++++++++++++++++----------
 1 file changed, 58 insertions(+), 15 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 37c669f..89cb2c8 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -204,10 +204,10 @@
    "outputs": [],
    "source": [
     "data_dir = \"/work/scratch/data/romaint\"\n",
-    "s2_b4 = f\"{data_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
-    "s2_b8 = f\"{data_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
-    "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
-    "phr_product_cog = f\"{data_dir}/phr_cog.tif\""
+    "s2_b4 = f\"{data_dir}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
+    "s2_b8 = f\"{data_dir}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
+    "phr_product_cog = f\"{data_dir}/input_greenit/phr_cog.tif\"\n",
+    "phr_product_lzw = f\"{data_dir}/input_greenit/phr_lzw.tif\""
    ]
   },
   {
@@ -1080,7 +1080,7 @@
     "    # dtype to uint8, and specify LZW compression.\n",
     "    profile.update(compress='lzw')\n",
     "\n",
-    "    with rasterio.open(os.environ['TMPDIR'] + '/compressed.tif', 'w', **profile) as dst:\n",
+    "    with rasterio.open(os.environ['TMPDIR'] + '/lzw.tif', 'w', **profile) as dst:\n",
     "        dst.write(raster.read(1), 1)"
    ]
   },
@@ -1126,22 +1126,65 @@
     "!du -sh $TMPDIR/*"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "9c0589ff-d318-4b93-8b74-bf8caaee8037",
+   "metadata": {},
+   "source": [
+    "### Size gain after compression\n",
+    "\n",
+    "Initial PLEIADES product size : 13 Go\n",
+    "\n",
+    "#### LZW\n",
+    "Product size : 10 Go\n",
+    "\n",
+    "#### CoG\n",
+    "Product size : 7 Go"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "58483551-a29e-4435-bfe5-8737bf9ba7bb",
+   "metadata": {},
+   "source": [
+    "### Compute time with compressed files"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "5f7263f1-5d4f-4025-8c74-cd83d10210aa",
+   "id": "8e7a9fdb-3844-432a-bc43-fbcc2c5169cd",
    "metadata": {},
    "outputs": [],
    "source": [
-    "## Time gain after compression for an NDVI computing on a PLEIADES image\n",
+    "reading_chunks = (-1,2048,2048)\n",
+    "start=time.perf_counter()\n",
+    "data_array = rxr.open_rasterio(phr_product_lzw, chunks=reading_chunks, lock=False)\n",
+    "ndvi_array = (data_array[3] - data_array[0]) / (data_array[3] + data_array[0])\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_lzw.tif\")\n",
+    "ndvi_array.rio.to_raster(output_file)\n",
+    "end=time.perf_counter()\n",
+    "print(\"Elapsed with LZW compressed file = {}s\".format((end - start)))\n",
     "\n",
-    "Initial product size : 13 Go\n",
+    "start=time.perf_counter()\n",
+    "data_array = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
+    "ndvi_array = (data_array[3] - data_array[0]) / (data_array[3] + data_array[0])\n",
+    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_cog.tif\")\n",
+    "ndvi_array.rio.to_raster(output_file)\n",
+    "end=time.perf_counter()\n",
+    "print(\"Elapsed with CoG compressed file = {}s\".format((end - start)))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d695aecb-9e65-4111-a7c4-845cb5c6914e",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
     "\n",
-    "### LZW\n",
-    "Product size : \n",
+    "* Using LZW or CoG does reduce the size of your data on disk and reduce the computing time, but **beware than in memory the product size will still be the original product's size**\n",
     "\n",
-    "### CoG\n",
-    "Product size : 7 Go"
+    "* Using CoG is more efficient in disk size and computational time (see the next chapter)\n"
    ]
   },
   {
@@ -1153,7 +1196,7 @@
    "source": [
     "# Use case : I have to use multiple products which average size is > 10Go\n",
     "\n",
-    "In this example we will use a PLEIADES product, which size is 13Go\n",
+    "In this example we will use a PLEIADES product, which original size is 13Go, the CoG size is 7 Go\n",
     "\n",
     "In that case you'll need to use parralellisation libraries such as dask or rioxarray with parallel read and write support to speedup your processing. Let's see what are the recommended parameters to optimize that\n",
     "\n",
@@ -1857,11 +1900,11 @@
     "\n",
     "Prefer using rxr.to_raster() instead of dask.compute() when your product size is getting bigger as the later seems to be more RAM consuming\n",
     "\n",
-    "### Chunk size recommandation for multi band products\n",
+    "#### Chunk size recommandation for multi band products\n",
     "\n",
     "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. We recommand in that case defining chunks using \"(-1,sizex,sizey)\" which creates chunks with concatenated bands. The time gain can be up to 20% !\n",
     "\n",
-    "### Don't hesitate to test different chunk sizes before making a conclusion\n",
+    "#### Don't hesitate to test different chunk sizes before making a conclusion\n",
     "\n",
     "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of 8 for read/write efficiency in RAM."
    ]

From 834cdc1ea6a8478563859ca2c14135f06085042b Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 6 Feb 2025 14:14:08 +0100
Subject: [PATCH 15/37] Move the profiling section

---
 tuto_greenit.ipynb | 117 ++++++++++++++++++++++++++-------------------
 1 file changed, 67 insertions(+), 50 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 89cb2c8..4c2c0d2 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -1,5 +1,15 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "1d86416d-9940-4b33-a82f-a93a7a6e138f",
+   "metadata": {},
+   "source": [
+    "# Best practices for Green IT coding in python\n",
+    "\n",
+    "In this tutorial, we will show good pratices for a green IT approach to code an image processing chain with efficiency in mind"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "3c6a05a0-c506-43e1-a352-c5e8e0d6f36e",
@@ -136,6 +146,60 @@
     "For an in-depth comparison of how much numpy is faster than for and while loops, this link provides a complete performance comparison : https://www.blog.duomly.com/loops-in-python-comparison-and-performance/"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "85b9e4b2-7621-48ac-8fa7-14789d40a34a",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "# Profiling your processing chain with TREX monitoring\n",
+    "\n",
+    "In order to optimize your chain, you have to check where are the bottleneck of your program. The TREX cluster provides all the tools you need for that, with the monitoring module.\n",
+    "\n",
+    "In this tutorial, we use this script to execute as a slurm job, which creates the graphs for CPU, RAM, DISK in png format\n",
+    "\n",
+    "The in depth documentation is available [here](https://hpc.pages.cnes.fr/wiki-hpc-sphinx/module-monitoring-2.html)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ccc05e8b-cfba-4671-8995-8c6a146ebca7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "greenit_slurm_script_monitored = \"\"\"#!/bin/bash\n",
+    "#\n",
+    "# Load monitoring module\n",
+    "module load monitoring/2.2\n",
+    "#\n",
+    "# Use of SLURM $TMPDIR  of the interactive job\n",
+    "WORKDIR=$TMPDIR\n",
+    "SLURM_SUBMIT_DIR=/work/scratch/env/romaint/greenit/perfos_greenit\n",
+    "#\n",
+    "cd \"$WORKDIR\"\n",
+    "#\n",
+    "# Give model name\n",
+    "#\n",
+    "MODEL_NAME=test_perfo_greenit\n",
+    "#\n",
+    "# launch start_monitoring.sh to start monitoring of the job ( --name option mandatory)\n",
+    "#\n",
+    "start_monitoring.sh --name $MODEL_NAME --io local\n",
+    "#\n",
+    "# lauch program\n",
+    "#\n",
+    "source /work/scratch/env/romaint/greenit/env_greenit_py312/bin/activate\n",
+    "python3 /work/scratch/env/romaint/greenit/test_parallelisation.py\n",
+    "#\n",
+    "# Stop monitoring -it will generate output_monitoring directory\n",
+    "#\n",
+    "stop_monitoring.sh --name $MODEL_NAME\n",
+    "\n",
+    "\"\"\""
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "371254df-a3b4-4f3e-b34f-3c3db06828ba",
@@ -894,7 +958,8 @@
    "source": [
     "%%bash\n",
     "\n",
-    "otbcli_BandMath -il \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"/work/scratch/data/romaint/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"/work/scratch/data/romaint/output_greenit/img_ndvi_otb_cpp.tif\" "
+    "WORK_DIR=\"/work/scratch/data/romaint\"\n",
+    "otbcli_BandMath -il \"${WORK_DIR}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"${WORK_DIR}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"${WORK_DIR}/output_greenit/img_ndvi_otb_cpp.tif\" "
    ]
   },
   {
@@ -1196,7 +1261,7 @@
    "source": [
     "# Use case : I have to use multiple products which average size is > 10Go\n",
     "\n",
-    "In this example we will use a PLEIADES product, which original size is 13Go, the CoG size is 7 Go\n",
+    "In this example we will use a PLEIADES product (which original size is 13Go) in CoG format which size is 7 Go\n",
     "\n",
     "In that case you'll need to use parralellisation libraries such as dask or rioxarray with parallel read and write support to speedup your processing. Let's see what are the recommended parameters to optimize that\n",
     "\n",
@@ -2200,54 +2265,6 @@
     "* You have to carefully choose your chunk size using a multiple of the cog size"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "85b9e4b2-7621-48ac-8fa7-14789d40a34a",
-   "metadata": {
-    "tags": []
-   },
-   "source": [
-    "# Profiling your processing chain with TREX monitoring"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "ccc05e8b-cfba-4671-8995-8c6a146ebca7",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "greenit_slurm_script_monitored = \"\"\"#!/bin/bash\n",
-    "#\n",
-    "# Load monitoring module\n",
-    "module load monitoring/2.2\n",
-    "#\n",
-    "# Use of SLURM $TMPDIR  of the interactive job\n",
-    "WORKDIR=$TMPDIR\n",
-    "SLURM_SUBMIT_DIR=/work/scratch/env/romaint/greenit/perfos_greenit\n",
-    "#\n",
-    "cd \"$WORKDIR\"\n",
-    "#\n",
-    "# Give model name\n",
-    "#\n",
-    "MODEL_NAME=test_perfo_greenit\n",
-    "#\n",
-    "# launch start_monitoring.sh to start monitoring of the job ( --name option mandatory)\n",
-    "#\n",
-    "start_monitoring.sh --name $MODEL_NAME --io local\n",
-    "#\n",
-    "# lauch program\n",
-    "#\n",
-    "source /work/scratch/env/romaint/greenit/env_greenit_py312/bin/activate\n",
-    "python3 /work/scratch/env/romaint/greenit/test_parallelisation.py\n",
-    "#\n",
-    "# Stop monitoring -it will generate output_monitoring directory\n",
-    "#\n",
-    "stop_monitoring.sh --name $MODEL_NAME\n",
-    "\n",
-    "\"\"\""
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "97205c02-6fc9-42f3-83ed-c8fd4af704af",

From 9834fd9b6915aac101257a7a482952df45e62fd0 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 6 Feb 2025 14:21:33 +0100
Subject: [PATCH 16/37] More precise conclusions

---
 tuto_greenit.ipynb | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 4c2c0d2..5837b5a 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -1263,7 +1263,7 @@
     "\n",
     "In this example we will use a PLEIADES product (which original size is 13Go) in CoG format which size is 7 Go\n",
     "\n",
-    "In that case you'll need to use parralellisation libraries such as dask or rioxarray with parallel read and write support to speedup your processing. Let's see what are the recommended parameters to optimize that\n",
+    "This is the typical use case where dask or rioxarray with parallel read and write support is necessary to speedup your processing. Let's see what are the recommended parameters to optimize your processing with big data\n",
     "\n",
     "## Optimize dask parameters"
    ]
@@ -1963,7 +1963,7 @@
    "source": [
     "### Conclusion\n",
     "\n",
-    "Prefer using rxr.to_raster() instead of dask.compute() when your product size is getting bigger as the later seems to be more RAM consuming\n",
+    "Prefer using rxr.to_raster() instead of dask.compute() when your product size is getting bigger as the later seems to be more CPU consuming\n",
     "\n",
     "#### Chunk size recommandation for multi band products\n",
     "\n",
@@ -1971,7 +1971,11 @@
     "\n",
     "#### Don't hesitate to test different chunk sizes before making a conclusion\n",
     "\n",
-    "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of 8 for read/write efficiency in RAM."
+    "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of 8 for read/write efficiency in RAM.\n",
+    "\n",
+    "#### Adjust the number of workers for dask\n",
+    "\n",
+    "If you have a lot of CPU, you can also gain time by adjusting the number of workers for dask, here for 8 CPU we used 4 workers, which gives 2 threads per CPU which is the perfect scenario for modern CPUs with hyperthreading (8 Core => 16 Threads)"
    ]
   },
   {
@@ -1979,7 +1983,10 @@
    "id": "f7746770-c831-4598-96ff-f13e1d372e1d",
    "metadata": {},
    "source": [
-    "## Improve disk write speed with rio.to_raster "
+    "## Improve disk write speed with rio.to_raster \n",
+    "\n",
+    "In the previous computing, we didn't use the tiled write capability of rio.to_raster, which speeds up the image writing on disk.\n",
+    "Let's see how to use it"
    ]
   },
   {

From 2ddead86a4680df7fdf2504c9c83b487e77446fd Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 6 Feb 2025 14:54:08 +0100
Subject: [PATCH 17/37] Rearrangement of the section size of your data

---
 tuto_greenit.ipynb | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 5837b5a..b2872fc 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -354,6 +354,7 @@
    "execution_count": 2,
    "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21",
    "metadata": {
+    "scrolled": true,
     "tags": []
    },
    "outputs": [
@@ -1087,7 +1088,7 @@
    "id": "c9229d23-c553-4c30-8ae0-b21cee07879e",
    "metadata": {},
    "source": [
-    "## Conclusion for a Sentinel 2 product with single band files (~3Go for the full product)**\n",
+    "## Conclusion for a Sentinel 2 product with single band files (~3Go for the full product)\n",
     "\n",
     "* Using **RIOXarray + dask on single band product is counter productive**.\n",
     "* Xarray + applyufunc also leads to low performance compared to opening the whole product with rasterio.\n",
@@ -1099,23 +1100,9 @@
    "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43",
    "metadata": {},
    "source": [
-    "# The importance of reducing the size of your data"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "eace634b-3c4c-4db8-8c4f-92ffea2bf1e6",
-   "metadata": {},
-   "source": [
-    "## The size of your image should lead you to use the right library\n",
-    "\n",
-    "**If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask**\n",
-    "\n",
-    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 3 Go**\n",
-    "\n",
-    "**If your product is multi-band, use RIOXarray or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances**\n",
+    "# The importance of reducing the size of your data\n",
     "\n",
-    "Let's see the different types of compression and how they improve performance"
+    "Let's see how the size of your data can have an impact on your compute time"
    ]
   },
   {
@@ -1252,6 +1239,22 @@
     "* Using CoG is more efficient in disk size and computational time (see the next chapter)\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "eace634b-3c4c-4db8-8c4f-92ffea2bf1e6",
+   "metadata": {},
+   "source": [
+    "## The size of your image should lead you to use the right library\n",
+    "\n",
+    "**If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask**\n",
+    "\n",
+    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 3 Go**\n",
+    "\n",
+    "**If your product is multi-band, use RIOXarray or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances**\n",
+    "\n",
+    "Let's see a use case where parralellisation methods are worth it"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "56e25516-3d90-4ee6-8c4b-085b94d3d096",

From 716437bf7d2df6964dbb46239bbc73084e1f622d Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Fri, 7 Feb 2025 17:01:33 +0100
Subject: [PATCH 18/37] move sections to make the notebook more like a story

---
 tuto_greenit.ipynb | 40 +++++++++++++++++++++++++++++++---------
 1 file changed, 31 insertions(+), 9 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index b2872fc..138dbe2 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -200,6 +200,23 @@
     "\"\"\""
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "1898ca12-fdcc-48b9-b733-642f2627bbe9",
+   "metadata": {},
+   "source": [
+    "# IO libraries for spatial image processing"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5baa8de4-7502-45b3-a5a1-52dd0f254d96",
+   "metadata": {},
+   "source": [
+    "Depending on your project, the data format will vary : you will need to choose a library that can fully handle your product.\n",
+    "A Tutorial is available here for a detailed walkthrough on how to choose the right library"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "371254df-a3b4-4f3e-b34f-3c3db06828ba",
@@ -1183,15 +1200,20 @@
    "id": "9c0589ff-d318-4b93-8b74-bf8caaee8037",
    "metadata": {},
    "source": [
-    "### Size gain after compression\n",
+    "## Size and compute time gain after compression\n",
+    "\n",
+    "The computing is done with otb using Python\n",
     "\n",
-    "Initial PLEIADES product size : 13 Go\n",
+    "#### Original PLEIADES product size : 13 Go\n",
+    "Compute time : 1 min 45s\n",
     "\n",
-    "#### LZW\n",
-    "Product size : 10 Go\n",
+    "#### LZW : Product size : 10 Go\n",
+    "Compute time : 1 min 21s\n",
+    "Percentage gain : 22%\n",
     "\n",
-    "#### CoG\n",
-    "Product size : 7 Go"
+    "#### CoG : Product size : 7 Go\n",
+    "Compute time : 1 min 03s\n",
+    "Perf difference vs original : 70%"
    ]
   },
   {
@@ -1199,7 +1221,7 @@
    "id": "58483551-a29e-4435-bfe5-8737bf9ba7bb",
    "metadata": {},
    "source": [
-    "### Compute time with compressed files"
+    "### Compute time with RIOXarray and compressed files"
    ]
   },
   {
@@ -1236,7 +1258,7 @@
     "\n",
     "* Using LZW or CoG does reduce the size of your data on disk and reduce the computing time, but **beware than in memory the product size will still be the original product's size**\n",
     "\n",
-    "* Using CoG is more efficient in disk size and computational time (see the next chapter)\n"
+    "* Using CoG is more efficient in disk size and computational time, you can gain up to 70% with a simple NDVI indice computing. See the next chapter on how to optimize the disk write to gain more time.\n"
    ]
   },
   {
@@ -1268,7 +1290,7 @@
     "\n",
     "This is the typical use case where dask or rioxarray with parallel read and write support is necessary to speedup your processing. Let's see what are the recommended parameters to optimize your processing with big data\n",
     "\n",
-    "## Optimize dask parameters"
+    "## Using dask and optimizing its parameters"
    ]
   },
   {

From 60bf1b04df88149e7e8c1210f75901992e41408a Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 10 Feb 2025 11:12:52 +0100
Subject: [PATCH 19/37] Add carbon estimate and enhance some conclusions

---
 tuto_greenit.ipynb | 51 ++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 42 insertions(+), 9 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 138dbe2..0055d1a 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -205,7 +205,7 @@
    "id": "1898ca12-fdcc-48b9-b733-642f2627bbe9",
    "metadata": {},
    "source": [
-    "# IO libraries for spatial image processing"
+    "# Data format libraries for spatial image processing"
    ]
   },
   {
@@ -214,7 +214,7 @@
    "metadata": {},
    "source": [
     "Depending on your project, the data format will vary : you will need to choose a library that can fully handle your product.\n",
-    "A Tutorial is available here for a detailed walkthrough on how to choose the right library"
+    "A Tutorial is available [here](https://github.com/CNES/pluto-tuto/pull/3) for a detailed walkthrough on how to choose the right library"
    ]
   },
   {
@@ -864,7 +864,7 @@
    ],
    "source": [
     "# Open the raster\n",
-    "reading_chunks = (-1,2200,2200)\n",
+    "reading_chunks = True\n",
     "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks, False)\n",
     "input_data_array"
    ]
@@ -934,6 +934,7 @@
     "app_ndvi_otb = otb.Registry.CreateApplication(\"BandMath\")\n",
     "app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n",
     "app_ndvi_otb.SetParameterString(\"exp\",\"(im2b1-im1b1)/(im2b1+im1b1)\")\n",
+    "# Uncomment to compute NDVI for PLEIADES\n",
     "#app_ndvi_otb.SetParameterStringList(\"il\",[phr_product_cog])\n",
     "#app_ndvi_otb.SetParameterString(\"exp\",\"(im1b4-im1b1)/(im1b4+im1b1)\")\n",
     "app_ndvi_otb.SetParameterString(\"out\",out_ndvi_otb_py)\n",
@@ -1212,8 +1213,8 @@
     "Percentage gain : 22%\n",
     "\n",
     "#### CoG : Product size : 7 Go\n",
-    "Compute time : 1 min 03s\n",
-    "Perf difference vs original : 70%"
+    "Compute time : 1 min 08s\n",
+    "Perf difference vs original : 65%"
    ]
   },
   {
@@ -2233,7 +2234,7 @@
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "output_file = Path(f\"{data_dir}/output_greenit/ndvi_phr_rxr.tif\")\n",
-    "# Add the Tiled parameter to rxr to speed up the disk write\n",
+    "# Add the Tiled=true parameter to rxr to speed up the disk write\n",
     "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with RIOXarray + tiled write + product > 5Go = {}s\".format((end - start)))"
@@ -2282,7 +2283,7 @@
     "**Conclusion for Memory usage**\n",
     "\n",
     "* Both method consumes the same amount of memory\n",
-    "* Cpu usage is higher with dask\n",
+    "* Cpu usage is **higher with dask**\n",
     "* You must use the tiled write in order to gain time"
    ]
   },
@@ -2293,6 +2294,8 @@
    "source": [
     "## Conclusions and recommandations about parallel write\n",
     "\n",
+    "* Use **lock=False** for reading the file to improve READ time\n",
+    "* Use **tiled=True** parameter to rio.to_raster to improve WRITE performance\n",
     "* Huge time gain without precision loss\n",
     "* You have to carefully choose your chunk size using a multiple of the cog size"
    ]
@@ -2303,7 +2306,8 @@
    "metadata": {},
    "source": [
     "# Estimate the carbon impact of your code\n",
-    "Using code carbon you can have an estimate of your code footprint"
+    "Using code carbon you can have an estimate of your code footprint,\n",
+    "Here we would like to calculate the carbon footprint of an NDVI for a S2 and a PHR image for comparison"
    ]
   },
   {
@@ -2327,7 +2331,36 @@
     }
    ],
    "source": [
-    "from codecarbon import track_emissions\n"
+    "from codecarbon import track_emissions, EmissionsTracker\n",
+    "\n",
+    "tracker = EmissionsTracker()\n",
+    "\n",
+    "#@track_emissions()\n",
+    "def compute_ndvi_sentinel2():\n",
+    "    input_data_b4 =  xarray.open_dataarray(s2_b4)\n",
+    "    input_data_b8 =  xarray.open_dataarray(s2_b8)\n",
+    "    ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n",
+    "    output_file = Path(f\"{data_dir}/output_greenit/ndvi_ufunc_sentinel2.tif\")\n",
+    "    ndvi_computed.rio.to_raster(output_file)\n",
+    "\n",
+    "#@track_emissions()\n",
+    "def compute_ndvi_phr():\n",
+    "    input_data_phr = xarray.open_dataarray(phr_product)\n",
+    "    ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_phr[0],input_data_phr[3])\n",
+    "    output_file = Path(f\"{data_dir}/output_greenit/ndvi_ufunc_phr.tif\")\n",
+    "    ndvi_computed.rio.to_raster(output_file)\n",
+    "\n",
+    "tracker.start()\n",
+    "start = time.perf_counter()\n",
+    "compute_ndvi_sentinel2()\n",
+    "end = time.perf_counter()\n",
+    "print(\"Elapsed to compute sentinel 2 NDVI = {}s\".format((end - start)))\n",
+    "\n",
+    "start = time.perf_counter()\n",
+    "compute_ndvi_phr()\n",
+    "end = time.perf_counter()\n",
+    "print(\"Elapsed to compute PHR Ventoux NDVI = {}s\".format((end - start)))\n",
+    "tracker.stop()"
    ]
   }
  ],

From 4cc08c4ba96d26d13c6fc05a8b2f51b023f91331 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 10 Feb 2025 11:34:05 +0100
Subject: [PATCH 20/37] Add red color on important conclusions

---
 tuto_greenit.ipynb | 66 +++++++++++++++++++++++++++-------------------
 1 file changed, 39 insertions(+), 27 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 0055d1a..5fd7cff 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -1004,7 +1004,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 41,
+   "execution_count": 1,
    "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
    "metadata": {
     "scrolled": true,
@@ -1012,29 +1012,21 @@
    },
    "outputs": [
     {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Elapsed with Raster IO + numba = 1.6279641180299222s\n",
-      "Elapsed with Raster IO + without numba = 1.7432440733537078s\n",
-      "Elapsed with Xarray + apply ufunc = 3.0273265461437404s\n",
-      "Elapsed with RIOXarray + dask = 4.102793791797012s\n",
-      "CPU times: user 2.25 s, sys: 5.03 s, total: 7.28 s\n",
-      "Wall time: 10.5 s\n"
+     "ename": "NameError",
+     "evalue": "name 'njit' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "File \u001b[0;32m:2\u001b[0m\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'njit' is not defined"
      ]
     }
    ],
    "source": [
     "%%time\n",
     "\n",
+    "#Define functions for each scenario\n",
     "@njit\n",
     "def one_pixel_ndvi(p1,p2):\n",
     "    return (p2-p1) / (p2+p1) \n",
@@ -1054,14 +1046,14 @@
     "    #ndvi_array = [one_pixel_ndvi(i,j) for i in input_data_1 for j in input_data_2]\n",
     "    ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n",
     "    return ndvi_array\n",
-    "\n",
+    "#Compute with numba\n",
     "start = time.perf_counter()\n",
     "with rasterio.open(s2_b4, 'r') as ds:\n",
     "    input_data_b4 = ds.read() \n",
     "\n",
     "with rasterio.open(s2_b8, 'r') as ds:\n",
     "    input_data_b8 = ds.read()\n",
-    "   \n",
+    "\n",
     "ndvi_computed = compute_ndvi_numba(input_data_b4,input_data_b8)\n",
     "crs=\"EPSG:4326\"\n",
     "output_file = Path(f\"{data_dir}/output_greenit/ndvi_numba.tif\")\n",
@@ -1269,11 +1261,16 @@
    "source": [
     "## The size of your image should lead you to use the right library\n",
     "\n",
-    "**If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask**\n",
+    "
    \n",
+    "If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask\n",
+    "
    \n",
     "\n",
     "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 3 Go**\n",
     "\n",
-    "**If your product is multi-band, use RIOXarray or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances**\n",
+    "
    \n",
+    "If your product is multi-band, use RIOXarray and/or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances\n",
+    "
    \n",
+    "\n",
     "\n",
     "Let's see a use case where parralellisation methods are worth it"
    ]
@@ -1989,15 +1986,23 @@
    "source": [
     "### Conclusion\n",
     "\n",
+    "
    \n",
     "Prefer using rxr.to_raster() instead of dask.compute() when your product size is getting bigger as the later seems to be more CPU consuming\n",
+    "
    \n",
     "\n",
     "#### Chunk size recommandation for multi band products\n",
     "\n",
-    "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. We recommand in that case defining chunks using \"(-1,sizex,sizey)\" which creates chunks with concatenated bands. The time gain can be up to 20% !\n",
+    "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. \n",
+    "
    \n",
+    "We recommand in that case defining chunks using \"(-1,sizex,sizey)\" which creates chunks with concatenated bands. The time gain can be up to 20% !\n",
+    "
    \n",
     "\n",
     "#### Don't hesitate to test different chunk sizes before making a conclusion\n",
     "\n",
-    "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of 8 for read/write efficiency in RAM.\n",
+    "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of the tile size for read/write efficiency in RAM.\n",
+    "
    \n",
+    "Don't use chunks > 512Mo as it becomes counter-productive\n",
+    "
    \n",
     "\n",
     "#### Adjust the number of workers for dask\n",
     "\n",
@@ -2292,12 +2297,19 @@
    "id": "ffd1783b-a0e3-43f2-861d-13d043653dc6",
    "metadata": {},
    "source": [
-    "## Conclusions and recommandations about parallel write\n",
+    "## Conclusions and recommandations\n",
     "\n",
-    "* Use **lock=False** for reading the file to improve READ time\n",
+    "
    \n",
+    "    \n",
+    "* Use **lock=False** and **chunks=(-1,sizex,sizey)** for reading the file to improve READ time\n",
+    "  \n",
     "* Use **tiled=True** parameter to rio.to_raster to improve WRITE performance\n",
+    "\n",
+    "
    \n",
+    "\n",
     "* Huge time gain without precision loss\n",
-    "* You have to carefully choose your chunk size using a multiple of the cog size"
+    "\n",
+    "* You have to carefully choose your chunk size using a multiple of the cog size\n"
    ]
   },
   {

From 693acd61f30259c4514ea79df14154893103eb83 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 13 Feb 2025 14:16:15 +0100
Subject: [PATCH 21/37] Add output_dir to permit the user more customisation
 when executing the notebook

---
 tuto_greenit.ipynb | 43 +++++++++++++++++++++++++------------------
 1 file changed, 25 insertions(+), 18 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 5fd7cff..f58768d 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -288,7 +288,8 @@
     "s2_b4 = f\"{data_dir}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
     "s2_b8 = f\"{data_dir}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
     "phr_product_cog = f\"{data_dir}/input_greenit/phr_cog.tif\"\n",
-    "phr_product_lzw = f\"{data_dir}/input_greenit/phr_lzw.tif\""
+    "phr_product_lzw = f\"{data_dir}/input_greenit/phr_lzw.tif\"\n",
+    "output_dir = f\"{data_dir}/output_greenit\""
    ]
   },
   {
@@ -893,7 +894,7 @@
     "# Launch the computing with dask with the compute call\n",
     "mean_ndvi = ndvi_array.compute() \n",
     "crs=\"EPSG:4326\"\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_dask.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_dask.tif\")\n",
     "create_raster(mean_ndvi, output_file, x_res , y_res,\n",
     "                  top_left_x, top_left_y, crs)"
    ]
@@ -929,7 +930,7 @@
    "source": [
     "import otbApplication as otb\n",
     "\n",
-    "out_ndvi_otb_py=\"/work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif\"\n",
+    "out_ndvi_otb_py=f\"{output_dir}/img_ndvi_otb_py.tif\"\n",
     "#Compute NDVI with OTB in python\n",
     "app_ndvi_otb = otb.Registry.CreateApplication(\"BandMath\")\n",
     "app_ndvi_otb.SetParameterStringList(\"il\",[s2_b4,s2_b8])\n",
@@ -1056,7 +1057,7 @@
     "\n",
     "ndvi_computed = compute_ndvi_numba(input_data_b4,input_data_b8)\n",
     "crs=\"EPSG:4326\"\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_numba.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_numba.tif\")\n",
     "create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with Raster IO + numba = {}s\".format((end - start)))\n",
@@ -1070,7 +1071,7 @@
     "\n",
     "ndvi_computed = compute_ndvi_std(input_data_b4,input_data_b8)\n",
     "crs=\"EPSG:4326\"\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_without_numba.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_without_numba.tif\")\n",
     "create_raster(ndvi_computed, output_file, x_res , y_res,top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with Raster IO + without numba = {}s\".format((end - start)))\n",
@@ -1079,7 +1080,7 @@
     "input_data_b4 =  xarray.open_dataarray(s2_b4)\n",
     "input_data_b8 =  xarray.open_dataarray(s2_b8)\n",
     "ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_ufunc.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_ufunc.tif\")\n",
     "ndvi_computed.rio.to_raster(output_file)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with Xarray + apply ufunc = {}s\".format((end - start)))\n",
@@ -1087,7 +1088,7 @@
     "start = time.perf_counter()\n",
     "input_data, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], (-1,2200,2200), False)\n",
     "ndvi_array = compute_ndvi_dask(input_data[0],input_data[1])\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_dask.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_dask.tif\")\n",
     "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n"
@@ -1130,6 +1131,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "with rasterio.open(phr_product, 'r') as ds:\n",
+    "    raster = ds.read() \n",
+    "\n",
     "# Register GDAL format drivers and configuration options with a\n",
     "# context manager.\n",
     "with rasterio.Env():\n",
@@ -1142,7 +1146,7 @@
     "    # dtype to uint8, and specify LZW compression.\n",
     "    profile.update(compress='lzw')\n",
     "\n",
-    "    with rasterio.open(os.environ['TMPDIR'] + '/lzw.tif', 'w', **profile) as dst:\n",
+    "    with rasterio.open(phr_product_lzw, 'w', **profile) as dst:\n",
     "        dst.write(raster.read(1), 1)"
    ]
   },
@@ -1161,10 +1165,13 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "with rasterio.open(phr_product, 'r') as ds:\n",
+    "    raster = ds.read() \n",
+    "\n",
     "with rasterio.Env():\n",
     "    profile = raster.profile\n",
     "    profile.update(tiled=True, compress='lzw', blockxsize=512, blockysize=512)\n",
-    "    with rasterio.open(os.environ['TMPDIR'] + '/cog.tif', 'w', **profile) as dst:\n",
+    "    with rasterio.open(phr_product_cog, 'w', **profile) as dst:\n",
     "        dst.write(raster.read(1), 1)"
    ]
   },
@@ -1185,7 +1192,7 @@
     }
    ],
    "source": [
-    "!du -sh $TMPDIR/*"
+    "!du -sh /work/scratch/data/romaint/input_greenit/phr_product*"
    ]
   },
   {
@@ -1228,7 +1235,7 @@
     "start=time.perf_counter()\n",
     "data_array = rxr.open_rasterio(phr_product_lzw, chunks=reading_chunks, lock=False)\n",
     "ndvi_array = (data_array[3] - data_array[0]) / (data_array[3] + data_array[0])\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_lzw.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_lzw.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with LZW compressed file = {}s\".format((end - start)))\n",
@@ -1236,7 +1243,7 @@
     "start=time.perf_counter()\n",
     "data_array = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
     "ndvi_array = (data_array[3] - data_array[0]) / (data_array[3] + data_array[0])\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_cog.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_cog.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with CoG compressed file = {}s\".format((end - start)))"
@@ -1892,7 +1899,7 @@
    "source": [
     "start=time.perf_counter()\n",
     "ndvi_array = (data_array_autochunks[3] - data_array_autochunks[0]) / (data_array_autochunks[3] + data_array_autochunks[0])\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_autochunks.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_autochunks.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with automatic chunk size = {}s\".format((end - start)))"
@@ -1949,7 +1956,7 @@
     "reading_chunks = (-1,2048,2048)\n",
     "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
     "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_manualchunks.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_manualchunks.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))"
@@ -2061,7 +2068,7 @@
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "ndvi_phr.compute()\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_phr.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_phr.tif\")\n",
     "ndvi_phr.rio.to_raster(output_file)\n",
     "end = time.perf_counter()\n",
     "print(\"Elapsed with RIOXarray + dask compute + product > 5Go = {}s\".format((end - start)))"
@@ -2238,7 +2245,7 @@
     "reading_chunks = (-1,2048,2048)\n",
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
-    "output_file = Path(f\"{data_dir}/output_greenit/ndvi_phr_rxr.tif\")\n",
+    "output_file = Path(f\"{output_dir}/ndvi_phr_rxr.tif\")\n",
     "# Add the Tiled=true parameter to rxr to speed up the disk write\n",
     "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
     "end = time.perf_counter()\n",
@@ -2352,14 +2359,14 @@
     "    input_data_b4 =  xarray.open_dataarray(s2_b4)\n",
     "    input_data_b8 =  xarray.open_dataarray(s2_b8)\n",
     "    ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n",
-    "    output_file = Path(f\"{data_dir}/output_greenit/ndvi_ufunc_sentinel2.tif\")\n",
+    "    output_file = Path(f\"{output_dir}/ndvi_ufunc_sentinel2.tif\")\n",
     "    ndvi_computed.rio.to_raster(output_file)\n",
     "\n",
     "#@track_emissions()\n",
     "def compute_ndvi_phr():\n",
     "    input_data_phr = xarray.open_dataarray(phr_product)\n",
     "    ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_phr[0],input_data_phr[3])\n",
-    "    output_file = Path(f\"{data_dir}/output_greenit/ndvi_ufunc_phr.tif\")\n",
+    "    output_file = Path(f\"{output_dir}/ndvi_ufunc_phr.tif\")\n",
     "    ndvi_computed.rio.to_raster(output_file)\n",
     "\n",
     "tracker.start()\n",

From 07ee7f80bbca1a618117d39ac090b669bb1602a0 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 13 Feb 2025 14:19:52 +0100
Subject: [PATCH 22/37] Add link to a library for manpipulating Cog with
 overview

---
 tuto_greenit.ipynb | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index f58768d..be1c868 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -1175,6 +1175,24 @@
     "        dst.write(raster.read(1), 1)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "be984817-d0df-4987-812f-738df5d6c261",
+   "metadata": {},
+   "source": [
+    "## Create Cog with overview\n",
+    "\n",
+    "A library exists to manipulate CoG with overviews: https://github.com/cogeotiff/rio-cogeo/"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a72ca70d-0b0f-49e4-9e7e-f291d1e8cd96",
+   "metadata": {},
+   "source": [
+    "## Size and compute time gain after compression"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 2,
@@ -1200,8 +1218,6 @@
    "id": "9c0589ff-d318-4b93-8b74-bf8caaee8037",
    "metadata": {},
    "source": [
-    "## Size and compute time gain after compression\n",
-    "\n",
     "The computing is done with otb using Python\n",
     "\n",
     "#### Original PLEIADES product size : 13 Go\n",

From 82ac3a0620ff34fef162e2ccd3ba499febb657ed Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 13 Feb 2025 14:07:28 +0000
Subject: [PATCH 23/37]  Add time count on compression

---
 tuto_greenit.ipynb | 209 +++++++++++++++++++++++++++------------------
 1 file changed, 128 insertions(+), 81 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index be1c868..f4214b2 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -245,7 +245,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 1,
    "id": "8966e0c9-d03f-4edf-b1b0-abbb4c4c2c34",
    "metadata": {
     "tags": []
@@ -277,7 +277,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 19,
    "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
    "metadata": {
     "tags": []
@@ -287,6 +287,7 @@
     "data_dir = \"/work/scratch/data/romaint\"\n",
     "s2_b4 = f\"{data_dir}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
     "s2_b8 = f\"{data_dir}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
+    "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
     "phr_product_cog = f\"{data_dir}/input_greenit/phr_cog.tif\"\n",
     "phr_product_lzw = f\"{data_dir}/input_greenit/phr_lzw.tif\"\n",
     "output_dir = f\"{data_dir}/output_greenit\""
@@ -369,18 +370,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 4,
    "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21",
    "metadata": {
     "scrolled": true,
     "tags": []
    },
    "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n",
+      "Perhaps you already have a cluster running?\n",
+      "Hosting the HTTP server on port 37823 instead\n",
+      "  warnings.warn(\n"
+     ]
+    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Dask Dashboard:  http://127.0.0.1:8787/status\n"
+      "Dask Dashboard:  http://127.0.0.1:37823/status\n"
      ]
     },
     {
@@ -390,7 +401,7 @@
        "    
     
    \n",
        "    
    \n",
        "        
    Client
    \n",
-       "        
    Client-e89e21b5-e20d-11ef-84df-00620ba482fc
    \n",
+       "        
    Client-de5a8946-ea11-11ef-b9f9-5c6f69244cc4
    \n",
        "        \n",
        "\n",
        "        \n",
@@ -403,7 +414,7 @@
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
@@ -412,7 +423,7 @@
        "        
    \n",
-       "                    Dashboard:  http://127.0.0.1:8787/status\n",
+       "                    Dashboard:  http://127.0.0.1:37823/status\n",
        "                
    \n",
        "\n",
        "        \n",
-       "            \n",
        "        \n",
@@ -425,11 +436,11 @@
        "    
    \n",
        "    
    \n",
        "        
    LocalCluster
    \n",
-       "        
    427581f7
    \n",
+       "        
    145210b0
    \n",
        "        \n",
        "            \n",
        "                \n",
        "                
    \n",
-       "                    Dashboard: http://127.0.0.1:8787/status\n",
+       "                    Dashboard: http://127.0.0.1:37823/status\n",
        "                \n",
        "                    Workers: 4\n",
@@ -462,11 +473,11 @@
        "        
     
    \n",
        "        
    \n",
        "            
    Scheduler
    \n",
-       "            
    Scheduler-3ef03144-bde0-4857-9ed2-3b85e5982644
    \n",
+       "            
    Scheduler-5e671663-16ec-414a-bc99-d642721f0cf3
    \n",
        "            \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                    \n",
        "                    
    \n",
-       "                        Comm: tcp://127.0.0.1:40559\n",
+       "                        Comm: tcp://127.0.0.1:45279\n",
        "                    \n",
        "                        Workers: 4\n",
@@ -474,7 +485,7 @@
        "                
    \n",
-       "                        Dashboard: http://127.0.0.1:8787/status\n",
+       "                        Dashboard: http://127.0.0.1:37823/status\n",
        "                    \n",
        "                        Total threads: 4\n",
@@ -508,7 +519,7 @@
        "                \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -553,7 +564,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:43405\n",
+       "                            Comm:  tcp://127.0.0.1:40603\n",
        "                        \n",
        "                            Total threads:  1\n",
@@ -516,7 +527,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:38755/status\n",
+       "                            Dashboard:  http://127.0.0.1:43941/status\n",
        "                        \n",
        "                            Memory:  7.00 GiB\n",
@@ -524,13 +535,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:39751\n",
+       "                            Nanny:  tcp://127.0.0.1:42307\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32111092/dask-scratch-space/worker-8wzm9nm6\n",
+       "                            Local directory:  /tmp/slurm-32787395/dask-scratch-space/worker-chbni3_s\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -598,7 +609,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:43803\n",
+       "                            Comm:  tcp://127.0.0.1:41709\n",
        "                        \n",
        "                            Total threads:  1\n",
@@ -561,7 +572,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:33143/status\n",
+       "                            Dashboard:  http://127.0.0.1:45009/status\n",
        "                        \n",
        "                            Memory:  7.00 GiB\n",
@@ -569,13 +580,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:34299\n",
+       "                            Nanny:  tcp://127.0.0.1:36113\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32111092/dask-scratch-space/worker-ls1alhze\n",
+       "                            Local directory:  /tmp/slurm-32787395/dask-scratch-space/worker-ir_x2jjp\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -643,7 +654,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:38291\n",
+       "                            Comm:  tcp://127.0.0.1:36627\n",
        "                        \n",
        "                            Total threads:  1\n",
@@ -606,7 +617,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:44757/status\n",
+       "                            Dashboard:  http://127.0.0.1:38577/status\n",
        "                        \n",
        "                            Memory:  7.00 GiB\n",
@@ -614,13 +625,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:39203\n",
+       "                            Nanny:  tcp://127.0.0.1:44667\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32111092/dask-scratch-space/worker-utct9_is\n",
+       "                            Local directory:  /tmp/slurm-32787395/dask-scratch-space/worker-yyl1fiof\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -692,10 +703,10 @@
        ""
       ],
       "text/plain": [
-       ""
+       ""
       ]
      },
-     "execution_count": 2,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -747,7 +758,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 5,
    "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f",
    "metadata": {
     "tags": []
@@ -772,17 +783,17 @@
        "                    \n",
        "                        \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
@@ -796,10 +807,7 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -817,10 +825,6 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -828,18 +832,11 @@
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
        "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -855,10 +852,10 @@
        "
    \n",
-       "                            Comm:  tcp://127.0.0.1:32857\n",
+       "                            Comm:  tcp://127.0.0.1:42359\n",
        "                        \n",
        "                            Total threads:  1\n",
@@ -651,7 +662,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:36677/status\n",
+       "                            Dashboard:  http://127.0.0.1:33623/status\n",
        "                        \n",
        "                            Memory:  7.00 GiB\n",
@@ -659,13 +670,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:33445\n",
+       "                            Nanny:  tcp://127.0.0.1:40163\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32111092/dask-scratch-space/worker-he9e73nn\n",
+       "                            Local directory:  /tmp/slurm-32787395/dask-scratch-space/worker-lo45hp8m\n",
        "                        
     Bytes  459.90 MiB  9.23 MiB  127.98 MiB 
     Shape  (2, 10980, 10980)  (1, 2200, 2200)  (1, 6111, 10980) 
     Dask graph  50 chunks in 5 graph layers  4 chunks in 5 graph layers 
     Data type 
    "
       ],
       "text/plain": [
-       "dask.array"
+       "dask.array"
       ]
      },
-     "execution_count": 7,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -872,7 +869,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 6,
    "id": "2dad7fe0-23bc-40b2-b27e-4c7cb0f8dc0e",
    "metadata": {
     "tags": []
@@ -882,8 +879,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "CPU times: user 454 ms, sys: 1.26 s, total: 1.72 s\n",
-      "Wall time: 3.9 s\n"
+      "CPU times: user 348 ms, sys: 1.68 s, total: 2.03 s\n",
+      "Wall time: 3.72 s\n"
      ]
     }
    ],
@@ -909,22 +906,35 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 9,
    "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "ename": "ModuleNotFoundError",
-     "evalue": "No module named 'otbApplication'",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[1], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01motbApplication\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01motb\u001b[39;00m\n\u001b[1;32m      3\u001b[0m out_ndvi_otb_py\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m      4\u001b[0m \u001b[38;5;66;03m#Compute NDVI with OTB in python\u001b[39;00m\n",
-      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'otbApplication'"
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Warning 1: Invalid value for NUM_THREADS: \n"
      ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif...: 100% [**************************************************] (3s)\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "0"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
     }
    ],
    "source": [
@@ -954,7 +964,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 8,
    "id": "73a731cf-a534-4b49-9a6d-425b58d87e45",
    "metadata": {
     "tags": []
@@ -1005,7 +1015,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 10,
    "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
    "metadata": {
     "scrolled": true,
@@ -1013,14 +1023,15 @@
    },
    "outputs": [
     {
-     "ename": "NameError",
-     "evalue": "name 'njit' is not defined",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
-      "File \u001b[0;32m:2\u001b[0m\n",
-      "\u001b[0;31mNameError\u001b[0m: name 'njit' is not defined"
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Elapsed with Raster IO + numba = 1.651143898256123s\n",
+      "Elapsed with Raster IO + without numba = 1.3101588021963835s\n",
+      "Elapsed with Xarray + apply ufunc = 2.5775902681052685s\n",
+      "Elapsed with RIOXarray + dask = 5.034341523423791s\n",
+      "CPU times: user 2.68 s, sys: 6.16 s, total: 8.84 s\n",
+      "Wall time: 10.6 s\n"
      ]
     }
    ],
@@ -1116,6 +1127,33 @@
     "Let's see how the size of your data can have an impact on your compute time"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "6e31fa02-b790-4795-a056-aa727e33e6ad",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'driver': 'GTiff', 'dtype': 'uint16', 'nodata': 0.0, 'width': 39844, 'height': 41663, 'count': 4, 'crs': None, 'transform': Affine(1.0, 0.0, 0.0,\n",
+       "       0.0, 1.0, 0.0), 'blockxsize': 39844, 'blockysize': 1, 'tiled': False, 'interleave': 'pixel'}"
+      ]
+     },
+     "execution_count": 20,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "## Read the initial raster\n",
+    "\n",
+    "raster = rasterio.open(phr_product, 'r')\n",
+    "raster.profile"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "7a95208c-4974-4c85-a09d-e7cd99fd09cd",
@@ -1128,11 +1166,21 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "69b8bd27-926e-4373-86d1-4dafe7fdbeed",
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n"
+     ]
+    }
+   ],
    "source": [
-    "with rasterio.open(phr_product, 'r') as ds:\n",
-    "    raster = ds.read() \n",
+    "%%time\n",
     "\n",
     "# Register GDAL format drivers and configuration options with a\n",
     "# context manager.\n",
@@ -1165,8 +1213,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "with rasterio.open(phr_product, 'r') as ds:\n",
-    "    raster = ds.read() \n",
+    "%%time\n",
     "\n",
     "with rasterio.Env():\n",
     "    profile = raster.profile\n",
@@ -2415,7 +2462,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.10"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

From 2d18a8379f150aecddc7b4fe423d27a56e81fb83 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 13 Feb 2025 14:24:28 +0000
Subject: [PATCH 24/37] More explicit code for compression, add also timing
 comparison

---
 tuto_greenit.ipynb | 87 +++++++++++++++++++++++++++-------------------
 1 file changed, 52 insertions(+), 35 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index f4214b2..88d8813 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -277,7 +277,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 2,
    "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
    "metadata": {
     "tags": []
@@ -1164,7 +1164,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 21,
    "id": "69b8bd27-926e-4373-86d1-4dafe7fdbeed",
    "metadata": {
     "tags": []
@@ -1177,6 +1177,14 @@
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
       "  dataset = writer(\n"
      ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "CPU times: user 1min 13s, sys: 5.12 s, total: 1min 19s\n",
+      "Wall time: 1min 17s\n"
+     ]
     }
    ],
    "source": [
@@ -1208,10 +1216,21 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 22,
    "id": "906973ba-32f1-473c-93dd-302296177950",
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "CPU times: user 1min 11s, sys: 5.54 s, total: 1min 17s\n",
+      "Wall time: 1min 22s\n"
+     ]
+    }
+   ],
    "source": [
     "%%time\n",
     "\n",
@@ -1242,7 +1261,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 24,
    "id": "1071ba31-2e9b-4da5-9d94-9e6ea9a5b1db",
    "metadata": {
     "tags": []
@@ -1252,31 +1271,18 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "^C\n"
+      "1.5G\t/work/scratch/data/romaint/input_greenit/SENTINEL2A_20210415-105852-555_L2A_T31TCJ_C_V3-0_FRE_STACK.tif\n",
+      "2.8G\t/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1\n",
+      "2.0G\t/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1.tar.gz\n",
+      "299M\t/work/scratch/data/romaint/input_greenit/image_PHR_marmande.zip\n",
+      "1.6G\t/work/scratch/data/romaint/input_greenit/phr_cog.tif\n",
+      "35M\t/work/scratch/data/romaint/input_greenit/phr_cog_extract.tif\n",
+      "1.7G\t/work/scratch/data/romaint/input_greenit/phr_lzw.tif\n"
      ]
     }
    ],
    "source": [
-    "!du -sh /work/scratch/data/romaint/input_greenit/phr_product*"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "9c0589ff-d318-4b93-8b74-bf8caaee8037",
-   "metadata": {},
-   "source": [
-    "The computing is done with otb using Python\n",
-    "\n",
-    "#### Original PLEIADES product size : 13 Go\n",
-    "Compute time : 1 min 45s\n",
-    "\n",
-    "#### LZW : Product size : 10 Go\n",
-    "Compute time : 1 min 21s\n",
-    "Percentage gain : 22%\n",
-    "\n",
-    "#### CoG : Product size : 7 Go\n",
-    "Compute time : 1 min 08s\n",
-    "Perf difference vs original : 65%"
+    "!du -sh /work/scratch/data/romaint/input_greenit/*"
    ]
   },
   {
@@ -1291,23 +1297,34 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "8e7a9fdb-3844-432a-bc43-fbcc2c5169cd",
-   "metadata": {},
-   "outputs": [],
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n"
+     ]
+    }
+   ],
    "source": [
     "reading_chunks = (-1,2048,2048)\n",
     "start=time.perf_counter()\n",
-    "data_array = rxr.open_rasterio(phr_product_lzw, chunks=reading_chunks, lock=False)\n",
-    "ndvi_array = (data_array[3] - data_array[0]) / (data_array[3] + data_array[0])\n",
+    "data_array_lzw = rxr.open_rasterio(phr_product_lzw, chunks=reading_chunks, lock=False)\n",
+    "ndvi_array = (data_array_lzw[3] - data_array_lzw[0]) / (data_array_lzw[3] + data_array_lzw[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_lzw.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file)\n",
+    "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with LZW compressed file = {}s\".format((end - start)))\n",
     "\n",
     "start=time.perf_counter()\n",
-    "data_array = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
-    "ndvi_array = (data_array[3] - data_array[0]) / (data_array[3] + data_array[0])\n",
+    "data_array_cog = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
+    "ndvi_array = (data_array_cog[3] - data_array_cog[0]) / (data_array_cog[3] + data_array_cog[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_cog.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file)\n",
+    "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with CoG compressed file = {}s\".format((end - start)))"
    ]

From 6f6567c8df5bbc031eca850b12e2a51887bffbe1 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 13 Feb 2025 14:29:57 +0000
Subject: [PATCH 25/37] Compression cell conclusion now uses red color for
 important conclusions

---
 tuto_greenit.ipynb | 34 ++++++++++++++++++++++++++++++----
 1 file changed, 30 insertions(+), 4 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 88d8813..e624144 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -1295,7 +1295,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 4,
    "id": "8e7a9fdb-3844-432a-bc43-fbcc2c5169cd",
    "metadata": {
     "tags": []
@@ -1308,6 +1308,28 @@
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
       "  dataset = writer(\n"
      ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Elapsed with LZW compressed file = 190.28657131735235s\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Elapsed with CoG compressed file = 35.8363040285185s\n"
+     ]
     }
    ],
    "source": [
@@ -1316,7 +1338,7 @@
     "data_array_lzw = rxr.open_rasterio(phr_product_lzw, chunks=reading_chunks, lock=False)\n",
     "ndvi_array = (data_array_lzw[3] - data_array_lzw[0]) / (data_array_lzw[3] + data_array_lzw[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_lzw.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
+    "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with LZW compressed file = {}s\".format((end - start)))\n",
     "\n",
@@ -1324,7 +1346,7 @@
     "data_array_cog = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
     "ndvi_array = (data_array_cog[3] - data_array_cog[0]) / (data_array_cog[3] + data_array_cog[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_cog.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
+    "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with CoG compressed file = {}s\".format((end - start)))"
    ]
@@ -1338,7 +1360,11 @@
     "\n",
     "* Using LZW or CoG does reduce the size of your data on disk and reduce the computing time, but **beware than in memory the product size will still be the original product's size**\n",
     "\n",
-    "* Using CoG is more efficient in disk size and computational time, you can gain up to 70% with a simple NDVI indice computing. See the next chapter on how to optimize the disk write to gain more time.\n"
+    "
    \n",
+    "    \n",
+    "* Using CoG is **way more efficient in computational time, you can gain up to 200%** with a simple NDVI indice computing. See the next chapter on how to optimize the disk write to gain more time.\n",
+    "    \n",
+    "
    \n"
    ]
   },
   {

From 70db1d0f1ae6e14235b35b50874fcb3edce7ab80 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Thu, 13 Feb 2025 14:40:32 +0000
Subject: [PATCH 26/37] Add one more conclusion on write speedup

---
 tuto_greenit.ipynb | 1630 ++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 1489 insertions(+), 141 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index e624144..d4e9839 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -1397,7 +1397,7 @@
    "source": [
     "# Use case : I have to use multiple products which average size is > 10Go\n",
     "\n",
-    "In this example we will use a PLEIADES product (which original size is 13Go) in CoG format which size is 7 Go\n",
+    "In this example we will use a PLEIADES product (which original size is 13Go) in CoG format which size is 1.6 Go\n",
     "\n",
     "This is the typical use case where dask or rioxarray with parallel read and write support is necessary to speedup your processing. Let's see what are the recommended parameters to optimize your processing with big data\n",
     "\n",
@@ -1799,26 +1799,16 @@
        "  fill: currentColor;\n",
        "}\n",
        "
    <xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
-       "dask.array<open_rasterio-fc2e18048eec7fd8788f8fc93c7ebea6<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n",
+       "dask.array<open_rasterio-80e78a4b45c26ac423b4a17c1494e93e<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n",
        "Coordinates:\n",
        "  * band         (band) int64 32B 1 2 3 4\n",
        "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
        "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
        "    spatial_ref  int64 8B 0\n",
-       "Attributes: (12/29)\n",
-       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
-       "    BlueDisplayChannel:   0\n",
-       "    DataType:             3\n",
-       "    GeometricLevel:       SENSOR\n",
-       "    GreenDisplayChannel:  0\n",
-       "    ImageID:              6967638101-1\n",
-       "    ...                   ...\n",
-       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
-       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
-       "    NoData:               0\n",
-       "    _FillValue:           0\n",
-       "    scale_factor:         1.0\n",
-       "    add_offset:           0.0
    "
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
-       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n",
+       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(1, 8192, 8192), chunktype=numpy.ndarray>\n",
        "Coordinates:\n",
        "  * band         (band) int64 32B 1 2 3 4\n",
        "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
        "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
        "    spatial_ref  int64 8B 0\n",
-       "Attributes: (12/29)\n",
-       "    AcquisitionDate:      2023-04-15T11:00:24.3Z\n",
-       "    BlueDisplayChannel:   0\n",
-       "    DataType:             3\n",
-       "    GeometricLevel:       SENSOR\n",
-       "    GreenDisplayChannel:  0\n",
-       "    ImageID:              6967638101-1\n",
-       "    ...                   ...\n",
-       "    TimeRangeEnd:         2023-04-15T11:00:27.3815980Z\n",
-       "    TimeRangeStart:       2023-04-15T11:00:24.3193675Z\n",
-       "    NoData:               0\n",
-       "    _FillValue:           0\n",
-       "    scale_factor:         1.0\n",
-       "    add_offset:           0.0"
+       "Attributes:\n",
+       "    _FillValue:    0\n",
+       "    scale_factor:  1.0\n",
+       "    add_offset:    0.0"
       ]
      },
      "execution_count": 5,
@@ -1983,22 +1963,14 @@
      "output_type": "stream",
      "text": [
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n"
+      "  dataset = writer(\n"
      ]
     },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with automatic chunk size = 75.2928059049882s\n"
+      "Elapsed with automatic chunk size = 40.49029225949198s\n"
      ]
     }
    ],
@@ -2011,81 +1983,1026 @@
     "print(\"Elapsed with automatic chunk size = {}s\".format((end - start)))"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "615be991-f60c-4353-8c18-fb4853c660eb",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "ename": "NameError",
-     "evalue": "name 'ndvi_array' is not defined",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[3], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mndvi_array\u001b[49m\n",
-      "\u001b[0;31mNameError\u001b[0m: name 'ndvi_array' is not defined"
-     ]
-    }
-   ],
-   "source": [
-    "ndvi_array"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": 7,
-   "id": "58bad330-9498-48ac-b3a0-cad211aaf998",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Elapsed with manual chunk size = 100.88597717694938s\n"
-     ]
-    }
-   ],
-   "source": [
-    "start=time.perf_counter()\n",
-    "reading_chunks = (-1,2048,2048)\n",
-    "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
-    "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n",
-    "output_file = Path(f\"{output_dir}/ndvi_manualchunks.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file)\n",
-    "end=time.perf_counter()\n",
-    "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "32e153c5-1ccc-47b2-a5c2-1270ab9d75b1",
+   "id": "615be991-f60c-4353-8c18-fb4853c660eb",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "ename": "NameError",
-     "evalue": "name 'ndvi_array' is not defined",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[3], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mndvi_array\u001b[49m\n",
-      "\u001b[0;31mNameError\u001b[0m: name 'ndvi_array' is not defined"
-     ]
+     "data": {
+      "text/html": [
+       "
    \n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "
    <xarray.DataArray (y: 41663, x: 39844)> Size: 13GB\n",
+       "dask.array<truediv, shape=(41663, 39844), dtype=float64, chunksize=(8192, 8192), chunktype=numpy.ndarray>\n",
+       "Coordinates:\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0
    "
+      ],
+      "text/plain": [
+       " Size: 13GB\n",
+       "dask.array\n",
+       "Coordinates:\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "ndvi_array"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "58bad330-9498-48ac-b3a0-cad211aaf998",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Elapsed with manual chunk size = 35.32894667331129s\n"
+     ]
+    }
+   ],
+   "source": [
+    "start=time.perf_counter()\n",
+    "reading_chunks = (-1,2048,2048)\n",
+    "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
+    "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n",
+    "output_file = Path(f\"{output_dir}/ndvi_manualchunks.tif\")\n",
+    "ndvi_array.rio.to_raster(output_file)\n",
+    "end=time.perf_counter()\n",
+    "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "32e153c5-1ccc-47b2-a5c2-1270ab9d75b1",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "
    \n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "
    <xarray.DataArray (y: 41663, x: 39844)> Size: 13GB\n",
+       "dask.array<truediv, shape=(41663, 39844), dtype=float64, chunksize=(2048, 2048), chunktype=numpy.ndarray>\n",
+       "Coordinates:\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0
    "
+      ],
+      "text/plain": [
+       " Size: 13GB\n",
+       "dask.array\n",
+       "Coordinates:\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
     }
    ],
    "source": [
@@ -2135,19 +3052,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 10,
    "id": "bfa94789-8576-4b26-a962-226db7d37f01",
    "metadata": {
     "tags": []
    },
    "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "(4, 41663, 39844)\n"
-     ]
-    },
     {
      "name": "stderr",
      "output_type": "stream",
@@ -2160,9 +3070,9 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with RasterIO + dask + product > 5Go = 71.1713912091218s\n",
-      "CPU times: user 10.2 s, sys: 40.5 s, total: 50.7 s\n",
-      "Wall time: 1min 11s\n"
+      "Elapsed with RIOXarray + dask compute + product > 5Go = 55.76110280863941s\n",
+      "CPU times: user 1min 19s, sys: 40.2 s, total: 1min 59s\n",
+      "Wall time: 55.8 s\n"
      ]
     }
    ],
@@ -2182,7 +3092,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 11,
    "id": "f4346248-866b-4b31-a54c-2a25b61c431d",
    "metadata": {
     "tags": []
@@ -2191,7 +3101,387 @@
     {
      "data": {
       "text/html": [
-       "\n",
+       "
    \n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "\n",
+       "
    <xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
+       "dask.array<open_rasterio-378202cf7021cd8ef82947b521bf5afc<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 2048, 2048), chunktype=numpy.ndarray>\n",
+       "Coordinates:\n",
+       "  * band         (band) int64 32B 1 2 3 4\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0\n",
+       "Attributes:\n",
+       "    _FillValue:    0\n",
+       "    scale_factor:  1.0\n",
+       "    add_offset:    0.0
    \n",
        "    \n",
        "        
    \n",
        "            \n",
@@ -2207,17 +3497,17 @@
        "                    \n",
        "                        \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
-       "                        \n",
+       "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
@@ -2231,16 +3521,24 @@
        "\n",
        "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
-       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
@@ -2248,7 +3546,7 @@
        "  \n",
        "\n",
        "  \n",
-       "  \n",
+       "  \n",
        "\n",
        "  \n",
        "  \n",
@@ -2256,49 +3554,75 @@
        "\n",
        "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
        "  \n",
        "\n",
        "  \n",
-       "  \n",
+       "  \n",
        "\n",
        "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
-       "  \n",
+       "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
-       "  \n",
+       "  \n",
        "  \n",
        "\n",
        "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
+       "  \n",
        "  \n",
        "  \n",
        "\n",
        "  \n",
-       "  \n",
+       "  \n",
        "\n",
        "  \n",
        "  39844\n",
@@ -2307,13 +3631,35 @@
        "\n",
        "        \n",
        "    \n",
-       "
     Bytes  12.37 GiB  128.00 MiB  32.00 MiB 
     Shape  (4, 41663, 39844)  (4, 4096, 4096)  (4, 2048, 2048) 
     Dask graph  110 chunks in 2 graph layers  420 chunks in 2 graph layers 
     Data type 
    "
+       "
    • band
      (band)
      int64
      1 2 3 4
      array([1, 2, 3, 4])
    • x
      (x)
      float64
      0.5 1.5 2.5 ... 3.984e+04 3.984e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 3.98415e+04, 3.98425e+04,\n",
+       "       3.98435e+04])
    • y
      (y)
      float64
      0.5 1.5 2.5 ... 4.166e+04 4.166e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 4.16605e+04, 4.16615e+04,\n",
+       "       4.16625e+04])
    • spatial_ref
      ()
      int64
      0
      GeoTransform :
      0.0 1.0 0.0 0.0 0.0 1.0
      array(0)
    • band
      PandasIndex
      PandasIndex(Index([1, 2, 3, 4], dtype='int64', name='band'))
    • x
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
+       "           8.5,     9.5,\n",
+       "       ...\n",
+       "       39834.5, 39835.5, 39836.5, 39837.5, 39838.5, 39839.5, 39840.5, 39841.5,\n",
+       "       39842.5, 39843.5],\n",
+       "      dtype='float64', name='x', length=39844))
    • y
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
+       "           8.5,     9.5,\n",
+       "       ...\n",
+       "       41653.5, 41654.5, 41655.5, 41656.5, 41657.5, 41658.5, 41659.5, 41660.5,\n",
+       "       41661.5, 41662.5],\n",
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
-       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 4096, 4096), chunktype=numpy.ndarray>"
+       " Size: 13GB\n",
+       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 2048, 2048), chunktype=numpy.ndarray>\n",
+       "Coordinates:\n",
+       "  * band         (band) int64 32B 1 2 3 4\n",
+       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
+       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
+       "    spatial_ref  int64 8B 0\n",
+       "Attributes:\n",
+       "    _FillValue:    0\n",
+       "    scale_factor:  1.0\n",
+       "    add_offset:    0.0"
       ]
      },
-     "execution_count": 16,
+     "execution_count": 11,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -2324,7 +3670,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 12,
    "id": "909e5746-eee9-4f0c-a840-d3e8850dee83",
    "metadata": {
     "tags": []
@@ -2342,7 +3688,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with RIOXarray + dask + product > 5Go with tiled write = 46.12782822502777s\n"
+      "Elapsed with RIOXarray + tiled write + product > 5Go = 35.50480739586055s\n"
      ]
     }
    ],
@@ -2412,6 +3758,8 @@
    "source": [
     "## Conclusions and recommandations\n",
     "\n",
+    "For an NDVI on the same product, the performance difference is 55s vs 35s here : **57%!**\n",
+    "\n",
     "
    \n",
     "    \n",
     "* Use **lock=False** and **chunks=(-1,sizex,sizey)** for reading the file to improve READ time\n",

From ab7026b755e534f3645bb648762dee9316239695 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Fri, 14 Feb 2025 10:42:23 +0100
Subject: [PATCH 27/37] Cleanup and rearrange time counter in the code carbon
 example

---
 tuto_greenit.ipynb | 1491 +++++++++++++++++++++++++++++---------------
 1 file changed, 979 insertions(+), 512 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index d4e9839..e2748f7 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -2467,32 +2467,62 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 5,
    "id": "58bad330-9498-48ac-b3a0-cad211aaf998",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n"
+     "ename": "RasterioIOError",
+     "evalue": "/work/scratch/data/romaint/input_greenit/phr_cog.tif: No such file or directory",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mCPLE_OpenFailedError\u001b[0m                      Traceback (most recent call last)",
+      "File \u001b[0;32mrasterio/_base.pyx:310\u001b[0m, in \u001b[0;36mrasterio._base.DatasetBase.__init__\u001b[0;34m()\u001b[0m\n",
+      "File \u001b[0;32mrasterio/_base.pyx:221\u001b[0m, in \u001b[0;36mrasterio._base.open_dataset\u001b[0;34m()\u001b[0m\n",
+      "File \u001b[0;32mrasterio/_err.pyx:359\u001b[0m, in \u001b[0;36mrasterio._err.exc_wrap_pointer\u001b[0;34m()\u001b[0m\n",
+      "\u001b[0;31mCPLE_OpenFailedError\u001b[0m: /work/scratch/data/romaint/input_greenit/phr_cog.tif: No such file or directory",
+      "\nDuring handling of the above exception, another exception occurred:\n",
+      "\u001b[0;31mRasterioIOError\u001b[0m                           Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[5], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m reading_chunks \u001b[38;5;241m=\u001b[39m (\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m,\u001b[38;5;241m2048\u001b[39m,\u001b[38;5;241m2048\u001b[39m)\n\u001b[0;32m----> 2\u001b[0m data_array_manualchunks \u001b[38;5;241m=\u001b[39m \u001b[43mrxr\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_rasterio\u001b[49m\u001b[43m(\u001b[49m\u001b[43mphr_product_cog\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mchunks\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mreading_chunks\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlock\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m)\u001b[49m\n\u001b[1;32m      3\u001b[0m start\u001b[38;5;241m=\u001b[39mtime\u001b[38;5;241m.\u001b[39mperf_counter()\n\u001b[1;32m      4\u001b[0m ndvi_array \u001b[38;5;241m=\u001b[39m (data_array_manualchunks[\u001b[38;5;241m3\u001b[39m] \u001b[38;5;241m-\u001b[39m data_array_manualchunks[\u001b[38;5;241m0\u001b[39m]) \u001b[38;5;241m/\u001b[39m (data_array_manualchunks[\u001b[38;5;241m3\u001b[39m] \u001b[38;5;241m+\u001b[39m data_array_manualchunks[\u001b[38;5;241m0\u001b[39m])\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/_io.py:1128\u001b[0m, in \u001b[0;36mopen_rasterio\u001b[0;34m(filename, parse_coordinates, chunks, cache, lock, masked, mask_and_scale, variable, group, default_name, decode_times, decode_timedelta, band_as_variable, **open_kwargs)\u001b[0m\n\u001b[1;32m   1126\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m   1127\u001b[0m         manager \u001b[38;5;241m=\u001b[39m URIManager(file_opener, filename, mode\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m, kwargs\u001b[38;5;241m=\u001b[39mopen_kwargs)\n\u001b[0;32m-> 1128\u001b[0m     riods \u001b[38;5;241m=\u001b[39m \u001b[43mmanager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43macquire\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1129\u001b[0m     captured_warnings \u001b[38;5;241m=\u001b[39m rio_warnings\u001b[38;5;241m.\u001b[39mcopy()\n\u001b[1;32m   1131\u001b[0m \u001b[38;5;66;03m# raise the NotGeoreferencedWarning if applicable\u001b[39;00m\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/_io.py:263\u001b[0m, in \u001b[0;36mURIManager.acquire\u001b[0;34m(self, needs_lock)\u001b[0m\n\u001b[1;32m    259\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_local\u001b[38;5;241m.\u001b[39mthread_manager \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m    260\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_local\u001b[38;5;241m.\u001b[39mthread_manager \u001b[38;5;241m=\u001b[39m ThreadURIManager(\n\u001b[1;32m    261\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_opener, \u001b[38;5;241m*\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_args, mode\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_mode, kwargs\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_kwargs\n\u001b[1;32m    262\u001b[0m     )\n\u001b[0;32m--> 263\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_local\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mthread_manager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfile_handle\u001b[49m\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/_io.py:219\u001b[0m, in \u001b[0;36mThreadURIManager.file_handle\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    217\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_file_handle \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m    218\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_file_handle\n\u001b[0;32m--> 219\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_file_handle \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_opener\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_args\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmode\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_mode\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_kwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    220\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_file_handle\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rasterio/env.py:463\u001b[0m, in \u001b[0;36mensure_env_with_credentials..wrapper\u001b[0;34m(*args, **kwds)\u001b[0m\n\u001b[1;32m    460\u001b[0m     session \u001b[38;5;241m=\u001b[39m DummySession()\n\u001b[1;32m    462\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m env_ctor(session\u001b[38;5;241m=\u001b[39msession):\n\u001b[0;32m--> 463\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mf\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwds\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rasterio/__init__.py:355\u001b[0m, in \u001b[0;36mopen\u001b[0;34m(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, opener, **kwargs)\u001b[0m\n\u001b[1;32m    352\u001b[0m     path \u001b[38;5;241m=\u001b[39m _parse_path(raw_dataset_path)\n\u001b[1;32m    354\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[0;32m--> 355\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m \u001b[43mDatasetReader\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpath\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdriver\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdriver\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msharing\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msharing\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    356\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr+\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m    357\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m get_writer_for_path(path, driver\u001b[38;5;241m=\u001b[39mdriver)(\n\u001b[1;32m    358\u001b[0m         path, mode, driver\u001b[38;5;241m=\u001b[39mdriver, sharing\u001b[38;5;241m=\u001b[39msharing, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs\n\u001b[1;32m    359\u001b[0m     )\n",
+      "File \u001b[0;32mrasterio/_base.pyx:312\u001b[0m, in \u001b[0;36mrasterio._base.DatasetBase.__init__\u001b[0;34m()\u001b[0m\n",
+      "\u001b[0;31mRasterioIOError\u001b[0m: /work/scratch/data/romaint/input_greenit/phr_cog.tif: No such file or directory"
      ]
     },
     {
-     "name": "stdout",
+     "name": "stderr",
      "output_type": "stream",
      "text": [
-      "Elapsed with manual chunk size = 35.32894667331129s\n"
+      "[codecarbon INFO @ 10:39:55] Energy consumed for RAM : 0.013351 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:39:55] Energy consumed for all CPUs : 0.049405 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:39:55] 0.062756 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:40:10] Energy consumed for RAM : 0.013398 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:40:10] Energy consumed for all CPUs : 0.049582 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:40:10] 0.062981 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:40:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241089623840645 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:40:25] Energy consumed for RAM : 0.013446 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:40:25] Energy consumed for all CPUs : 0.049759 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:40:25] 0.063206 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:40:40] Energy consumed for RAM : 0.013494 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:40:40] Energy consumed for all CPUs : 0.049937 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:40:40] 0.063431 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:40:55] Energy consumed for RAM : 0.013542 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:40:55] Energy consumed for all CPUs : 0.050114 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:40:55] 0.063656 kWh of electricity used since the beginning.\n"
      ]
     }
    ],
    "source": [
-    "start=time.perf_counter()\n",
     "reading_chunks = (-1,2048,2048)\n",
     "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
+    "start=time.perf_counter()\n",
     "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_manualchunks.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
@@ -2502,507 +2532,22 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 4,
    "id": "32e153c5-1ccc-47b2-a5c2-1270ab9d75b1",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "data": {
-      "text/html": [
-       "
    \n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "\n",
-       "
    <xarray.DataArray (y: 41663, x: 39844)> Size: 13GB\n",
-       "dask.array<truediv, shape=(41663, 39844), dtype=float64, chunksize=(2048, 2048), chunktype=numpy.ndarray>\n",
-       "Coordinates:\n",
-       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
-       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 8B 0
    "
-      ],
-      "text/plain": [
-       " Size: 13GB\n",
-       "dask.array\n",
-       "Coordinates:\n",
-       "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
-       "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 8B 0"
-      ]
-     },
-     "execution_count": 9,
-     "metadata": {},
-     "output_type": "execute_result"
+     "ename": "NameError",
+     "evalue": "name 'ndvi_array' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[4], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mndvi_array\u001b[49m\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'ndvi_array' is not defined"
+     ]
     }
    ],
    "source": [
@@ -3785,21 +3330,943 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 3,
    "id": "ee231cc6-e45c-4209-9135-8c80e1a4eaf4",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "ename": "ModuleNotFoundError",
-     "evalue": "No module named 'codecarbon'",
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[codecarbon INFO @ 09:30:04] [setup] RAM Tracking...\n",
+      "[codecarbon INFO @ 09:30:04] [setup] GPU Tracking...\n",
+      "[codecarbon INFO @ 09:30:04] No GPU found.\n",
+      "[codecarbon INFO @ 09:30:04] [setup] CPU Tracking...\n",
+      "[codecarbon WARNING @ 09:30:04] No CPU tracking mode found. Falling back on CPU constant mode. \n",
+      " Linux OS detected: Please ensure RAPL files exist at \\sys\\class\\powercap\\intel-rapl to measure CPU\n",
+      "\n",
+      "[codecarbon WARNING @ 09:30:05] We saw that you have a AMD Ryzen 7 7840U w/ Radeon  780M Graphics but we don't know it. Please contact us.\n",
+      "[codecarbon INFO @ 09:30:05] CPU Model on constant consumption mode: AMD Ryzen 7 7840U w/ Radeon  780M Graphics\n",
+      "[codecarbon INFO @ 09:30:05] >>> Tracker's metadata:\n",
+      "[codecarbon INFO @ 09:30:05]   Platform system: Linux-6.8.0-52-generic-x86_64-with-glibc2.39\n",
+      "[codecarbon INFO @ 09:30:05]   Python version: 3.11.10\n",
+      "[codecarbon INFO @ 09:30:05]   CodeCarbon version: 2.7.4\n",
+      "[codecarbon INFO @ 09:30:05]   Available RAM : 30.629 GB\n",
+      "[codecarbon INFO @ 09:30:05]   CPU count: 16\n",
+      "[codecarbon INFO @ 09:30:05]   CPU model: AMD Ryzen 7 7840U w/ Radeon  780M Graphics\n",
+      "[codecarbon INFO @ 09:30:05]   GPU count: None\n",
+      "[codecarbon INFO @ 09:30:05]   GPU model: None\n",
+      "[codecarbon WARNING @ 09:30:09] Unable to access geographical location through primary API. Will resort to using the backup API - Exception : ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) - url=https://get.geojs.io/v1/ip/geo.json\n",
+      "[codecarbon WARNING @ 09:30:10] Unable to access geographical location. Using 'Canada' as the default value - Exception : ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) - url=https://get.geojs.io/v1/ip/geo.json\n",
+      "[codecarbon INFO @ 09:30:10] Saving emissions data to file /home/tromain/Devel/pluto-tuto/emissions.csv\n"
+     ]
+    },
+    {
+     "ename": "RasterioIOError",
+     "evalue": "/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif: No such file or directory",
      "output_type": "error",
      "traceback": [
       "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[6], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mcodecarbon\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m track_emissions\n",
-      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'codecarbon'"
+      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/file_manager.py:211\u001b[0m, in \u001b[0;36mCachingFileManager._acquire_with_cache_info\u001b[0;34m(self, needs_lock)\u001b[0m\n\u001b[1;32m    210\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 211\u001b[0m     file \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_cache\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_key\u001b[49m\u001b[43m]\u001b[49m\n\u001b[1;32m    212\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m:\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/lru_cache.py:56\u001b[0m, in \u001b[0;36mLRUCache.__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m     55\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_lock:\n\u001b[0;32m---> 56\u001b[0m     value \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_cache\u001b[49m\u001b[43m[\u001b[49m\u001b[43mkey\u001b[49m\u001b[43m]\u001b[49m\n\u001b[1;32m     57\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_cache\u001b[38;5;241m.\u001b[39mmove_to_end(key)\n",
+      "\u001b[0;31mKeyError\u001b[0m: [, ('/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif',), 'r', (('sharing', False),), 'e5170fea-2adc-416a-81d4-e7ba85237e5c']",
+      "\nDuring handling of the above exception, another exception occurred:\n",
+      "\u001b[0;31mCPLE_OpenFailedError\u001b[0m                      Traceback (most recent call last)",
+      "File \u001b[0;32mrasterio/_base.pyx:310\u001b[0m, in \u001b[0;36mrasterio._base.DatasetBase.__init__\u001b[0;34m()\u001b[0m\n",
+      "File \u001b[0;32mrasterio/_base.pyx:221\u001b[0m, in \u001b[0;36mrasterio._base.open_dataset\u001b[0;34m()\u001b[0m\n",
+      "File \u001b[0;32mrasterio/_err.pyx:359\u001b[0m, in \u001b[0;36mrasterio._err.exc_wrap_pointer\u001b[0;34m()\u001b[0m\n",
+      "\u001b[0;31mCPLE_OpenFailedError\u001b[0m: /work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif: No such file or directory",
+      "\nDuring handling of the above exception, another exception occurred:\n",
+      "\u001b[0;31mRasterioIOError\u001b[0m                           Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[3], line 22\u001b[0m\n\u001b[1;32m     20\u001b[0m tracker\u001b[38;5;241m.\u001b[39mstart()\n\u001b[1;32m     21\u001b[0m start \u001b[38;5;241m=\u001b[39m time\u001b[38;5;241m.\u001b[39mperf_counter()\n\u001b[0;32m---> 22\u001b[0m \u001b[43mcompute_ndvi_sentinel2\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     23\u001b[0m end \u001b[38;5;241m=\u001b[39m time\u001b[38;5;241m.\u001b[39mperf_counter()\n\u001b[1;32m     24\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mElapsed to compute sentinel 2 NDVI = \u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mformat((end \u001b[38;5;241m-\u001b[39m start)))\n",
+      "Cell \u001b[0;32mIn[3], line 7\u001b[0m, in \u001b[0;36mcompute_ndvi_sentinel2\u001b[0;34m()\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcompute_ndvi_sentinel2\u001b[39m():\n\u001b[0;32m----> 7\u001b[0m     input_data_b4 \u001b[38;5;241m=\u001b[39m  \u001b[43mxarray\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_dataarray\u001b[49m\u001b[43m(\u001b[49m\u001b[43ms2_b4\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m      8\u001b[0m     input_data_b8 \u001b[38;5;241m=\u001b[39m  xarray\u001b[38;5;241m.\u001b[39mopen_dataarray(s2_b8)\n\u001b[1;32m      9\u001b[0m     ndvi_computed \u001b[38;5;241m=\u001b[39m xarray\u001b[38;5;241m.\u001b[39mapply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:852\u001b[0m, in \u001b[0;36mopen_dataarray\u001b[0;34m(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)\u001b[0m\n\u001b[1;32m    694\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mopen_dataarray\u001b[39m(\n\u001b[1;32m    695\u001b[0m     filename_or_obj: \u001b[38;5;28mstr\u001b[39m \u001b[38;5;241m|\u001b[39m os\u001b[38;5;241m.\u001b[39mPathLike[Any] \u001b[38;5;241m|\u001b[39m BufferedIOBase \u001b[38;5;241m|\u001b[39m AbstractDataStore,\n\u001b[1;32m    696\u001b[0m     \u001b[38;5;241m*\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    712\u001b[0m     \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[1;32m    713\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m DataArray:\n\u001b[1;32m    714\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"Open an DataArray from a file or file-like object containing a single\u001b[39;00m\n\u001b[1;32m    715\u001b[0m \u001b[38;5;124;03m    data variable.\u001b[39;00m\n\u001b[1;32m    716\u001b[0m \n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    849\u001b[0m \u001b[38;5;124;03m    open_dataset\u001b[39;00m\n\u001b[1;32m    850\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m--> 852\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m \u001b[43mopen_dataset\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    853\u001b[0m \u001b[43m        \u001b[49m\u001b[43mfilename_or_obj\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    854\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecode_cf\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_cf\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    855\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmask_and_scale\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmask_and_scale\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    856\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecode_times\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_times\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    857\u001b[0m \u001b[43m        \u001b[49m\u001b[43mconcat_characters\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mconcat_characters\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    858\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecode_coords\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_coords\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    859\u001b[0m \u001b[43m        \u001b[49m\u001b[43mengine\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mengine\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    860\u001b[0m \u001b[43m        \u001b[49m\u001b[43mchunks\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mchunks\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    861\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcache\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcache\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    862\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdrop_variables\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdrop_variables\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    863\u001b[0m \u001b[43m        \u001b[49m\u001b[43minline_array\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minline_array\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    864\u001b[0m \u001b[43m        \u001b[49m\u001b[43mchunked_array_type\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mchunked_array_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    865\u001b[0m \u001b[43m        \u001b[49m\u001b[43mfrom_array_kwargs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfrom_array_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    866\u001b[0m \u001b[43m        \u001b[49m\u001b[43mbackend_kwargs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mbackend_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    867\u001b[0m \u001b[43m        \u001b[49m\u001b[43muse_cftime\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43muse_cftime\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    868\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecode_timedelta\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_timedelta\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    869\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    870\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    872\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(dataset\u001b[38;5;241m.\u001b[39mdata_vars) \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m1\u001b[39m:\n\u001b[1;32m    873\u001b[0m         \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(dataset\u001b[38;5;241m.\u001b[39mdata_vars) \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m0\u001b[39m:\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:671\u001b[0m, in \u001b[0;36mopen_dataset\u001b[0;34m(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)\u001b[0m\n\u001b[1;32m    659\u001b[0m decoders \u001b[38;5;241m=\u001b[39m _resolve_decoders_kwargs(\n\u001b[1;32m    660\u001b[0m     decode_cf,\n\u001b[1;32m    661\u001b[0m     open_backend_dataset_parameters\u001b[38;5;241m=\u001b[39mbackend\u001b[38;5;241m.\u001b[39mopen_dataset_parameters,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    667\u001b[0m     decode_coords\u001b[38;5;241m=\u001b[39mdecode_coords,\n\u001b[1;32m    668\u001b[0m )\n\u001b[1;32m    670\u001b[0m overwrite_encoded_chunks \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mpop(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124moverwrite_encoded_chunks\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mNone\u001b[39;00m)\n\u001b[0;32m--> 671\u001b[0m backend_ds \u001b[38;5;241m=\u001b[39m \u001b[43mbackend\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_dataset\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    672\u001b[0m \u001b[43m    \u001b[49m\u001b[43mfilename_or_obj\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    673\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdrop_variables\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdrop_variables\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    674\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mdecoders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    675\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    676\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    677\u001b[0m ds \u001b[38;5;241m=\u001b[39m _dataset_from_backend_dataset(\n\u001b[1;32m    678\u001b[0m     backend_ds,\n\u001b[1;32m    679\u001b[0m     filename_or_obj,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    689\u001b[0m     \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[1;32m    690\u001b[0m )\n\u001b[1;32m    691\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m ds\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/xarray_plugin.py:58\u001b[0m, in \u001b[0;36mRasterioBackend.open_dataset\u001b[0;34m(self, filename_or_obj, drop_variables, parse_coordinates, lock, masked, mask_and_scale, variable, group, default_name, decode_coords, decode_times, decode_timedelta, band_as_variable, open_kwargs)\u001b[0m\n\u001b[1;32m     56\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m open_kwargs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m     57\u001b[0m     open_kwargs \u001b[38;5;241m=\u001b[39m {}\n\u001b[0;32m---> 58\u001b[0m rds \u001b[38;5;241m=\u001b[39m \u001b[43m_io\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_rasterio\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m     59\u001b[0m \u001b[43m    \u001b[49m\u001b[43mfilename_or_obj\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     60\u001b[0m \u001b[43m    \u001b[49m\u001b[43mparse_coordinates\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mparse_coordinates\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     61\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcache\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m     62\u001b[0m \u001b[43m    \u001b[49m\u001b[43mlock\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlock\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     63\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmasked\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmasked\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     64\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmask_and_scale\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmask_and_scale\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     65\u001b[0m \u001b[43m    \u001b[49m\u001b[43mvariable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mvariable\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     66\u001b[0m \u001b[43m    \u001b[49m\u001b[43mgroup\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     67\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdefault_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdefault_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     68\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdecode_times\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_times\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     69\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdecode_timedelta\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_timedelta\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     70\u001b[0m \u001b[43m    \u001b[49m\u001b[43mband_as_variable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mband_as_variable\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     71\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mopen_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     72\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     73\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(rds, xarray\u001b[38;5;241m.\u001b[39mDataArray):\n\u001b[1;32m     74\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m rds\u001b[38;5;241m.\u001b[39mto_dataset()\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/_io.py:1128\u001b[0m, in \u001b[0;36mopen_rasterio\u001b[0;34m(filename, parse_coordinates, chunks, cache, lock, masked, mask_and_scale, variable, group, default_name, decode_times, decode_timedelta, band_as_variable, **open_kwargs)\u001b[0m\n\u001b[1;32m   1126\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m   1127\u001b[0m         manager \u001b[38;5;241m=\u001b[39m URIManager(file_opener, filename, mode\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m, kwargs\u001b[38;5;241m=\u001b[39mopen_kwargs)\n\u001b[0;32m-> 1128\u001b[0m     riods \u001b[38;5;241m=\u001b[39m \u001b[43mmanager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43macquire\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1129\u001b[0m     captured_warnings \u001b[38;5;241m=\u001b[39m rio_warnings\u001b[38;5;241m.\u001b[39mcopy()\n\u001b[1;32m   1131\u001b[0m \u001b[38;5;66;03m# raise the NotGeoreferencedWarning if applicable\u001b[39;00m\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/file_manager.py:193\u001b[0m, in \u001b[0;36mCachingFileManager.acquire\u001b[0;34m(self, needs_lock)\u001b[0m\n\u001b[1;32m    178\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21macquire\u001b[39m(\u001b[38;5;28mself\u001b[39m, needs_lock\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m):\n\u001b[1;32m    179\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"Acquire a file object from the manager.\u001b[39;00m\n\u001b[1;32m    180\u001b[0m \n\u001b[1;32m    181\u001b[0m \u001b[38;5;124;03m    A new file is only opened if it has expired from the\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    191\u001b[0m \u001b[38;5;124;03m        An open file object, as returned by ``opener(*args, **kwargs)``.\u001b[39;00m\n\u001b[1;32m    192\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m--> 193\u001b[0m     file, _ \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_acquire_with_cache_info\u001b[49m\u001b[43m(\u001b[49m\u001b[43mneeds_lock\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    194\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m file\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/file_manager.py:217\u001b[0m, in \u001b[0;36mCachingFileManager._acquire_with_cache_info\u001b[0;34m(self, needs_lock)\u001b[0m\n\u001b[1;32m    215\u001b[0m     kwargs \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mcopy()\n\u001b[1;32m    216\u001b[0m     kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmode\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_mode\n\u001b[0;32m--> 217\u001b[0m file \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_opener\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_args\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    218\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mw\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m    219\u001b[0m     \u001b[38;5;66;03m# ensure file doesn't get overridden when opened again\u001b[39;00m\n\u001b[1;32m    220\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_mode \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ma\u001b[39m\u001b[38;5;124m\"\u001b[39m\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rasterio/env.py:463\u001b[0m, in \u001b[0;36mensure_env_with_credentials..wrapper\u001b[0;34m(*args, **kwds)\u001b[0m\n\u001b[1;32m    460\u001b[0m     session \u001b[38;5;241m=\u001b[39m DummySession()\n\u001b[1;32m    462\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m env_ctor(session\u001b[38;5;241m=\u001b[39msession):\n\u001b[0;32m--> 463\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mf\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwds\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rasterio/__init__.py:355\u001b[0m, in \u001b[0;36mopen\u001b[0;34m(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, opener, **kwargs)\u001b[0m\n\u001b[1;32m    352\u001b[0m     path \u001b[38;5;241m=\u001b[39m _parse_path(raw_dataset_path)\n\u001b[1;32m    354\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[0;32m--> 355\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m \u001b[43mDatasetReader\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpath\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdriver\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdriver\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msharing\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msharing\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    356\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr+\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m    357\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m get_writer_for_path(path, driver\u001b[38;5;241m=\u001b[39mdriver)(\n\u001b[1;32m    358\u001b[0m         path, mode, driver\u001b[38;5;241m=\u001b[39mdriver, sharing\u001b[38;5;241m=\u001b[39msharing, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs\n\u001b[1;32m    359\u001b[0m     )\n",
+      "File \u001b[0;32mrasterio/_base.pyx:312\u001b[0m, in \u001b[0;36mrasterio._base.DatasetBase.__init__\u001b[0;34m()\u001b[0m\n",
+      "\u001b[0;31mRasterioIOError\u001b[0m: /work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif: No such file or directory"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[codecarbon INFO @ 09:30:25] Energy consumed for RAM : 0.000048 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:30:25] Energy consumed for all CPUs : 0.000177 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:30:25] 0.000225 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:30:40] Energy consumed for RAM : 0.000096 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:30:40] Energy consumed for all CPUs : 0.000354 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:30:40] 0.000450 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:30:55] Energy consumed for RAM : 0.000144 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:30:55] Energy consumed for all CPUs : 0.000531 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:30:55] 0.000675 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:31:10] Energy consumed for RAM : 0.000191 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:31:10] Energy consumed for all CPUs : 0.000708 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:31:10] 0.000900 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:31:25] Energy consumed for RAM : 0.000239 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:31:25] Energy consumed for all CPUs : 0.000885 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:31:25] 0.001125 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:31:40] Energy consumed for RAM : 0.000287 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:31:40] Energy consumed for all CPUs : 0.001063 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:31:40] 0.001350 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:31:55] Energy consumed for RAM : 0.000335 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:31:55] Energy consumed for all CPUs : 0.001240 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:31:55] 0.001575 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:32:10] Energy consumed for RAM : 0.000383 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:32:10] Energy consumed for all CPUs : 0.001417 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:32:10] 0.001800 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:32:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241030588549235 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:32:25] Energy consumed for RAM : 0.000431 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:32:25] Energy consumed for all CPUs : 0.001594 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:32:25] 0.002024 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:32:40] Energy consumed for RAM : 0.000479 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:32:40] Energy consumed for all CPUs : 0.001771 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:32:40] 0.002249 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:32:55] Energy consumed for RAM : 0.000526 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:32:55] Energy consumed for all CPUs : 0.001948 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:32:55] 0.002474 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:33:10] Energy consumed for RAM : 0.000574 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:33:10] Energy consumed for all CPUs : 0.002125 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:33:10] 0.002699 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:33:25] Energy consumed for RAM : 0.000622 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:33:25] Energy consumed for all CPUs : 0.002302 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:33:25] 0.002924 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:33:40] Energy consumed for RAM : 0.000670 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:33:40] Energy consumed for all CPUs : 0.002479 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:33:40] 0.003149 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:33:55] Energy consumed for RAM : 0.000718 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:33:55] Energy consumed for all CPUs : 0.002656 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:33:55] 0.003374 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:34:10] Energy consumed for RAM : 0.000766 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:34:10] Energy consumed for all CPUs : 0.002833 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:34:10] 0.003599 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:34:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241081374832695 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:34:25] Energy consumed for RAM : 0.000813 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:34:25] Energy consumed for all CPUs : 0.003010 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:34:25] 0.003824 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:34:40] Energy consumed for RAM : 0.000861 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:34:40] Energy consumed for all CPUs : 0.003187 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:34:40] 0.004049 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:34:55] Energy consumed for RAM : 0.000909 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:34:55] Energy consumed for all CPUs : 0.003365 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:34:55] 0.004274 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:35:10] Energy consumed for RAM : 0.000957 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:35:10] Energy consumed for all CPUs : 0.003542 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:35:10] 0.004499 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:35:25] Energy consumed for RAM : 0.001005 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:35:25] Energy consumed for all CPUs : 0.003719 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:35:25] 0.004724 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:35:40] Energy consumed for RAM : 0.001053 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:35:40] Energy consumed for all CPUs : 0.003896 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:35:40] 0.004949 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:35:55] Energy consumed for RAM : 0.001101 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:35:55] Energy consumed for all CPUs : 0.004073 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:35:55] 0.005173 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:36:10] Energy consumed for RAM : 0.001148 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:36:10] Energy consumed for all CPUs : 0.004250 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:36:10] 0.005398 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:36:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241062424850872 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:36:25] Energy consumed for RAM : 0.001196 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:36:25] Energy consumed for all CPUs : 0.004427 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:36:25] 0.005623 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:36:40] Energy consumed for RAM : 0.001244 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:36:40] Energy consumed for all CPUs : 0.004604 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:36:40] 0.005848 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:36:55] Energy consumed for RAM : 0.001292 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:36:55] Energy consumed for all CPUs : 0.004781 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:36:55] 0.006073 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:37:10] Energy consumed for RAM : 0.001340 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:37:10] Energy consumed for all CPUs : 0.004958 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:37:10] 0.006298 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:37:25] Energy consumed for RAM : 0.001388 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:37:25] Energy consumed for all CPUs : 0.005135 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:37:25] 0.006523 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:37:40] Energy consumed for RAM : 0.001436 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:37:40] Energy consumed for all CPUs : 0.005312 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:37:40] 0.006748 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:37:55] Energy consumed for RAM : 0.001483 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:37:55] Energy consumed for all CPUs : 0.005490 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:37:55] 0.006973 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:38:10] Energy consumed for RAM : 0.001531 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:38:10] Energy consumed for all CPUs : 0.005667 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:38:10] 0.007198 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:38:10] 0.000036 g.CO2eq/s mean an estimation of 1.124116242737765 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:38:25] Energy consumed for RAM : 0.001579 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:38:25] Energy consumed for all CPUs : 0.005844 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:38:25] 0.007423 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:38:40] Energy consumed for RAM : 0.001627 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:38:40] Energy consumed for all CPUs : 0.006021 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:38:40] 0.007648 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:38:55] Energy consumed for RAM : 0.001675 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:38:55] Energy consumed for all CPUs : 0.006198 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:38:55] 0.007873 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:39:10] Energy consumed for RAM : 0.001723 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:39:10] Energy consumed for all CPUs : 0.006375 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:39:10] 0.008098 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:39:25] Energy consumed for RAM : 0.001771 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:39:25] Energy consumed for all CPUs : 0.006552 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:39:25] 0.008323 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:39:40] Energy consumed for RAM : 0.001818 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:39:40] Energy consumed for all CPUs : 0.006729 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:39:40] 0.008547 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:39:55] Energy consumed for RAM : 0.001866 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:39:55] Energy consumed for all CPUs : 0.006906 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:39:55] 0.008772 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:40:10] Energy consumed for RAM : 0.001914 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:40:10] Energy consumed for all CPUs : 0.007083 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:40:10] 0.008997 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:40:10] 0.000036 g.CO2eq/s mean an estimation of 1.1240958745964122 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:40:25] Energy consumed for RAM : 0.001962 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:40:25] Energy consumed for all CPUs : 0.007260 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:40:25] 0.009222 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:40:40] Energy consumed for RAM : 0.002010 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:40:40] Energy consumed for all CPUs : 0.007437 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:40:40] 0.009447 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:40:55] Energy consumed for RAM : 0.002058 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:40:55] Energy consumed for all CPUs : 0.007615 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:40:55] 0.009672 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:41:10] Energy consumed for RAM : 0.002105 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:41:10] Energy consumed for all CPUs : 0.007792 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:41:10] 0.009897 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:41:25] Energy consumed for RAM : 0.002153 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:41:25] Energy consumed for all CPUs : 0.007969 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:41:25] 0.010122 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:41:40] Energy consumed for RAM : 0.002201 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:41:40] Energy consumed for all CPUs : 0.008146 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:41:40] 0.010347 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:41:55] Energy consumed for RAM : 0.002249 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:41:55] Energy consumed for all CPUs : 0.008323 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:41:55] 0.010572 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:42:10] Energy consumed for RAM : 0.002297 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:42:10] Energy consumed for all CPUs : 0.008500 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:42:10] 0.010797 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:42:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241136069648792 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:42:25] Energy consumed for RAM : 0.002345 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:42:25] Energy consumed for all CPUs : 0.008677 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:42:25] 0.011022 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:42:40] Energy consumed for RAM : 0.002393 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:42:40] Energy consumed for all CPUs : 0.008854 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:42:40] 0.011247 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:42:55] Energy consumed for RAM : 0.002440 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:42:55] Energy consumed for all CPUs : 0.009031 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:42:55] 0.011472 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:43:10] Energy consumed for RAM : 0.002488 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:43:10] Energy consumed for all CPUs : 0.009208 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:43:10] 0.011697 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:43:25] Energy consumed for RAM : 0.002536 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:43:25] Energy consumed for all CPUs : 0.009385 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:43:25] 0.011921 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:43:40] Energy consumed for RAM : 0.002584 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:43:40] Energy consumed for all CPUs : 0.009562 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:43:40] 0.012146 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:43:55] Energy consumed for RAM : 0.002632 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:43:55] Energy consumed for all CPUs : 0.009739 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:43:55] 0.012371 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:44:10] Energy consumed for RAM : 0.002680 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:44:10] Energy consumed for all CPUs : 0.009917 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:44:10] 0.012596 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:44:10] 0.000036 g.CO2eq/s mean an estimation of 1.1240987825380455 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:44:25] Energy consumed for RAM : 0.002728 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:44:25] Energy consumed for all CPUs : 0.010094 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:44:25] 0.012821 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:44:40] Energy consumed for RAM : 0.002775 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:44:40] Energy consumed for all CPUs : 0.010271 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:44:40] 0.013046 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:44:55] Energy consumed for RAM : 0.002823 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:44:55] Energy consumed for all CPUs : 0.010448 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:44:55] 0.013271 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:45:10] Energy consumed for RAM : 0.002871 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:45:10] Energy consumed for all CPUs : 0.010625 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:45:10] 0.013496 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:45:25] Energy consumed for RAM : 0.002919 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:45:25] Energy consumed for all CPUs : 0.010802 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:45:25] 0.013721 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:45:40] Energy consumed for RAM : 0.002967 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:45:40] Energy consumed for all CPUs : 0.010979 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:45:40] 0.013946 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:45:55] Energy consumed for RAM : 0.003015 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:45:55] Energy consumed for all CPUs : 0.011156 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:45:55] 0.014171 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:46:10] Energy consumed for RAM : 0.003062 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:46:10] Energy consumed for all CPUs : 0.011333 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:46:10] 0.014396 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:46:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241045609252327 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:46:25] Energy consumed for RAM : 0.003110 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:46:25] Energy consumed for all CPUs : 0.011510 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:46:25] 0.014621 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:46:40] Energy consumed for RAM : 0.003158 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:46:40] Energy consumed for all CPUs : 0.011687 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:46:40] 0.014846 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:46:55] Energy consumed for RAM : 0.003206 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:46:55] Energy consumed for all CPUs : 0.011864 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:46:55] 0.015070 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:47:10] Energy consumed for RAM : 0.003254 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:47:10] Energy consumed for all CPUs : 0.012042 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:47:10] 0.015295 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:47:25] Energy consumed for RAM : 0.003302 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:47:25] Energy consumed for all CPUs : 0.012219 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:47:25] 0.015520 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:47:40] Energy consumed for RAM : 0.003350 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:47:40] Energy consumed for all CPUs : 0.012396 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:47:40] 0.015745 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:47:55] Energy consumed for RAM : 0.003397 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:47:55] Energy consumed for all CPUs : 0.012573 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:47:55] 0.015970 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:48:10] Energy consumed for RAM : 0.003445 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:48:10] Energy consumed for all CPUs : 0.012750 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:48:10] 0.016195 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:48:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241122754246067 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:48:25] Energy consumed for RAM : 0.003493 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:48:25] Energy consumed for all CPUs : 0.012927 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:48:25] 0.016420 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:48:40] Energy consumed for RAM : 0.003541 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:48:40] Energy consumed for all CPUs : 0.013104 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:48:40] 0.016645 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:48:55] Energy consumed for RAM : 0.003589 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:48:55] Energy consumed for all CPUs : 0.013281 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:48:55] 0.016870 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:49:10] Energy consumed for RAM : 0.003637 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:49:10] Energy consumed for all CPUs : 0.013458 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:49:10] 0.017095 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:49:25] Energy consumed for RAM : 0.003685 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:49:25] Energy consumed for all CPUs : 0.013635 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:49:25] 0.017320 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:49:40] Energy consumed for RAM : 0.003732 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:49:40] Energy consumed for all CPUs : 0.013812 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:49:40] 0.017545 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:49:55] Energy consumed for RAM : 0.003780 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:49:55] Energy consumed for all CPUs : 0.013989 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:49:55] 0.017770 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:50:10] Energy consumed for RAM : 0.003828 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:50:10] Energy consumed for all CPUs : 0.014166 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:50:10] 0.017995 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:50:10] 0.000036 g.CO2eq/s mean an estimation of 1.124105407505187 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:50:25] Energy consumed for RAM : 0.003876 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:50:25] Energy consumed for all CPUs : 0.014344 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:50:25] 0.018219 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:50:40] Energy consumed for RAM : 0.003924 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:50:40] Energy consumed for all CPUs : 0.014521 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:50:40] 0.018444 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:50:55] Energy consumed for RAM : 0.003972 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:50:55] Energy consumed for all CPUs : 0.014698 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:50:55] 0.018669 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:51:10] Energy consumed for RAM : 0.004020 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:51:10] Energy consumed for all CPUs : 0.014875 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:51:10] 0.018894 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:51:25] Energy consumed for RAM : 0.004067 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:51:25] Energy consumed for all CPUs : 0.015052 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:51:25] 0.019119 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:51:40] Energy consumed for RAM : 0.004115 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:51:40] Energy consumed for all CPUs : 0.015229 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:51:40] 0.019344 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:51:55] Energy consumed for RAM : 0.004163 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:51:55] Energy consumed for all CPUs : 0.015406 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:51:55] 0.019569 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:52:10] Energy consumed for RAM : 0.004211 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:52:10] Energy consumed for all CPUs : 0.015583 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:52:10] 0.019794 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:52:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241152906286473 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:52:25] Energy consumed for RAM : 0.004259 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:52:25] Energy consumed for all CPUs : 0.015760 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:52:25] 0.020019 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:52:40] Energy consumed for RAM : 0.004307 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:52:40] Energy consumed for all CPUs : 0.015937 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:52:40] 0.020244 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:52:55] Energy consumed for RAM : 0.004354 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:52:55] Energy consumed for all CPUs : 0.016114 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:52:55] 0.020469 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:53:10] Energy consumed for RAM : 0.004402 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:53:10] Energy consumed for all CPUs : 0.016291 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:53:10] 0.020694 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:53:25] Energy consumed for RAM : 0.004450 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:53:25] Energy consumed for all CPUs : 0.016468 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:53:25] 0.020919 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:53:40] Energy consumed for RAM : 0.004498 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:53:40] Energy consumed for all CPUs : 0.016646 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:53:40] 0.021144 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:53:55] Energy consumed for RAM : 0.004546 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:53:55] Energy consumed for all CPUs : 0.016823 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:53:55] 0.021369 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:54:10] Energy consumed for RAM : 0.004594 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:54:10] Energy consumed for all CPUs : 0.017000 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:54:10] 0.021593 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:54:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241096598571043 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:54:25] Energy consumed for RAM : 0.004642 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:54:25] Energy consumed for all CPUs : 0.017177 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:54:25] 0.021818 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:54:40] Energy consumed for RAM : 0.004689 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:54:40] Energy consumed for all CPUs : 0.017354 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:54:40] 0.022043 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:54:55] Energy consumed for RAM : 0.004737 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:54:55] Energy consumed for all CPUs : 0.017531 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:54:55] 0.022268 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:55:10] Energy consumed for RAM : 0.004785 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:55:10] Energy consumed for all CPUs : 0.017708 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:55:10] 0.022493 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:55:25] Energy consumed for RAM : 0.004833 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:55:25] Energy consumed for all CPUs : 0.017885 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:55:25] 0.022718 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:55:40] Energy consumed for RAM : 0.004881 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:55:40] Energy consumed for all CPUs : 0.018062 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:55:40] 0.022943 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:55:55] Energy consumed for RAM : 0.004929 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:55:55] Energy consumed for all CPUs : 0.018239 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:55:55] 0.023168 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:56:10] Energy consumed for RAM : 0.004977 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:56:10] Energy consumed for all CPUs : 0.018416 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:56:10] 0.023393 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:56:10] 0.000036 g.CO2eq/s mean an estimation of 1.1240968553304007 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:56:25] Energy consumed for RAM : 0.005024 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:56:25] Energy consumed for all CPUs : 0.018593 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:56:25] 0.023618 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:56:40] Energy consumed for RAM : 0.005072 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:56:40] Energy consumed for all CPUs : 0.018770 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:56:40] 0.023843 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:56:55] Energy consumed for RAM : 0.005120 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:56:55] Energy consumed for all CPUs : 0.018948 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:56:55] 0.024068 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:57:10] Energy consumed for RAM : 0.005168 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:57:10] Energy consumed for all CPUs : 0.019125 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:57:10] 0.024293 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:57:25] Energy consumed for RAM : 0.005216 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:57:25] Energy consumed for all CPUs : 0.019302 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:57:25] 0.024517 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:57:40] Energy consumed for RAM : 0.005264 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:57:40] Energy consumed for all CPUs : 0.019479 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:57:40] 0.024742 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:57:55] Energy consumed for RAM : 0.005311 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:57:55] Energy consumed for all CPUs : 0.019656 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:57:55] 0.024967 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:58:10] Energy consumed for RAM : 0.005359 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:58:10] Energy consumed for all CPUs : 0.019833 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:58:10] 0.025192 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:58:10] 0.000036 g.CO2eq/s mean an estimation of 1.124113741107103 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 09:58:25] Energy consumed for RAM : 0.005407 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:58:25] Energy consumed for all CPUs : 0.020010 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:58:25] 0.025417 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:58:40] Energy consumed for RAM : 0.005455 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:58:40] Energy consumed for all CPUs : 0.020187 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:58:40] 0.025642 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:58:55] Energy consumed for RAM : 0.005503 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:58:55] Energy consumed for all CPUs : 0.020364 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:58:55] 0.025867 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:59:10] Energy consumed for RAM : 0.005551 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:59:10] Energy consumed for all CPUs : 0.020541 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:59:10] 0.026092 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:59:25] Energy consumed for RAM : 0.005599 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:59:25] Energy consumed for all CPUs : 0.020718 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:59:25] 0.026317 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:59:40] Energy consumed for RAM : 0.005646 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:59:40] Energy consumed for all CPUs : 0.020895 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:59:40] 0.026542 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 09:59:55] Energy consumed for RAM : 0.005694 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 09:59:55] Energy consumed for all CPUs : 0.021073 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 09:59:55] 0.026767 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:00:10] Energy consumed for RAM : 0.005742 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:00:10] Energy consumed for all CPUs : 0.021250 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:00:10] 0.026992 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:00:10] 0.000036 g.CO2eq/s mean an estimation of 1.124111541492001 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:00:25] Energy consumed for RAM : 0.005790 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:00:25] Energy consumed for all CPUs : 0.021427 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:00:25] 0.027217 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:00:40] Energy consumed for RAM : 0.005838 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:00:40] Energy consumed for all CPUs : 0.021604 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:00:40] 0.027442 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:00:55] Energy consumed for RAM : 0.005886 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:00:55] Energy consumed for all CPUs : 0.021781 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:00:55] 0.027667 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:01:10] Energy consumed for RAM : 0.005934 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:01:10] Energy consumed for all CPUs : 0.021958 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:01:10] 0.027891 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:01:25] Energy consumed for RAM : 0.005981 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:01:25] Energy consumed for all CPUs : 0.022135 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:01:25] 0.028116 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:01:40] Energy consumed for RAM : 0.006029 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:01:40] Energy consumed for all CPUs : 0.022312 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:01:40] 0.028341 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:01:55] Energy consumed for RAM : 0.006077 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:01:55] Energy consumed for all CPUs : 0.022489 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:01:55] 0.028566 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:02:10] Energy consumed for RAM : 0.006125 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:02:10] Energy consumed for all CPUs : 0.022666 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:02:10] 0.028791 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:02:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241034094395093 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:02:25] Energy consumed for RAM : 0.006173 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:02:25] Energy consumed for all CPUs : 0.022843 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:02:25] 0.029016 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:02:40] Energy consumed for RAM : 0.006221 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:02:40] Energy consumed for all CPUs : 0.023020 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:02:40] 0.029241 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:02:55] Energy consumed for RAM : 0.006268 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:02:55] Energy consumed for all CPUs : 0.023197 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:02:55] 0.029466 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:03:10] Energy consumed for RAM : 0.006316 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:03:10] Energy consumed for all CPUs : 0.023375 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:03:10] 0.029691 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:03:25] Energy consumed for RAM : 0.006364 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:03:25] Energy consumed for all CPUs : 0.023552 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:03:25] 0.029916 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:03:40] Energy consumed for RAM : 0.006412 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:03:40] Energy consumed for all CPUs : 0.023729 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:03:40] 0.030141 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:03:55] Energy consumed for RAM : 0.006460 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:03:55] Energy consumed for all CPUs : 0.023906 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:03:55] 0.030366 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:04:10] Energy consumed for RAM : 0.006508 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:04:10] Energy consumed for all CPUs : 0.024083 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:04:10] 0.030591 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:04:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241195224753218 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:04:25] Energy consumed for RAM : 0.006556 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:04:25] Energy consumed for all CPUs : 0.024260 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:04:25] 0.030816 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:04:40] Energy consumed for RAM : 0.006603 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:04:40] Energy consumed for all CPUs : 0.024437 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:04:40] 0.031041 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:04:55] Energy consumed for RAM : 0.006651 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:04:55] Energy consumed for all CPUs : 0.024614 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:04:55] 0.031265 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:05:10] Energy consumed for RAM : 0.006699 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:05:10] Energy consumed for all CPUs : 0.024791 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:05:10] 0.031490 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:05:25] Energy consumed for RAM : 0.006747 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:05:25] Energy consumed for all CPUs : 0.024968 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:05:25] 0.031715 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:05:40] Energy consumed for RAM : 0.006795 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:05:40] Energy consumed for all CPUs : 0.025145 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:05:40] 0.031940 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:05:55] Energy consumed for RAM : 0.006843 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:05:55] Energy consumed for all CPUs : 0.025322 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:05:55] 0.032165 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:06:10] Energy consumed for RAM : 0.006891 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:06:10] Energy consumed for all CPUs : 0.025500 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:06:10] 0.032390 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:06:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241065632787401 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:06:25] Energy consumed for RAM : 0.006938 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:06:25] Energy consumed for all CPUs : 0.025677 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:06:25] 0.032615 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:06:40] Energy consumed for RAM : 0.006986 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:06:40] Energy consumed for all CPUs : 0.025854 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:06:40] 0.032840 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:06:55] Energy consumed for RAM : 0.007034 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:06:55] Energy consumed for all CPUs : 0.026031 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:06:55] 0.033065 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:07:10] Energy consumed for RAM : 0.007082 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:07:10] Energy consumed for all CPUs : 0.026208 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:07:10] 0.033290 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:07:25] Energy consumed for RAM : 0.007130 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:07:25] Energy consumed for all CPUs : 0.026385 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:07:25] 0.033515 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:07:40] Energy consumed for RAM : 0.007178 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:07:40] Energy consumed for all CPUs : 0.026562 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:07:40] 0.033740 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:07:55] Energy consumed for RAM : 0.007226 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:07:55] Energy consumed for all CPUs : 0.026739 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:07:55] 0.033965 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:08:10] Energy consumed for RAM : 0.007273 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:08:10] Energy consumed for all CPUs : 0.026916 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:08:10] 0.034190 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:08:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241020064479441 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:08:25] Energy consumed for RAM : 0.007321 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:08:25] Energy consumed for all CPUs : 0.027093 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:08:25] 0.034414 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:08:40] Energy consumed for RAM : 0.007369 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:08:40] Energy consumed for all CPUs : 0.027270 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:08:40] 0.034639 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:08:55] Energy consumed for RAM : 0.007417 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:08:55] Energy consumed for all CPUs : 0.027447 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:08:55] 0.034864 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:09:10] Energy consumed for RAM : 0.007465 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:09:10] Energy consumed for all CPUs : 0.027624 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:09:10] 0.035089 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:09:25] Energy consumed for RAM : 0.007513 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:09:25] Energy consumed for all CPUs : 0.027802 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:09:25] 0.035314 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:09:40] Energy consumed for RAM : 0.007560 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:09:40] Energy consumed for all CPUs : 0.027979 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:09:40] 0.035539 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:09:55] Energy consumed for RAM : 0.007608 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:09:55] Energy consumed for all CPUs : 0.028156 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:09:55] 0.035764 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:10:10] Energy consumed for RAM : 0.007656 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:10:10] Energy consumed for all CPUs : 0.028333 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:10:10] 0.035989 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:10:10] 0.000036 g.CO2eq/s mean an estimation of 1.12411772689883 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:10:25] Energy consumed for RAM : 0.007704 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:10:25] Energy consumed for all CPUs : 0.028510 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:10:25] 0.036214 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:10:40] Energy consumed for RAM : 0.007752 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:10:40] Energy consumed for all CPUs : 0.028687 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:10:40] 0.036439 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:10:55] Energy consumed for RAM : 0.007800 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:10:55] Energy consumed for all CPUs : 0.028864 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:10:55] 0.036664 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:11:10] Energy consumed for RAM : 0.007848 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:11:10] Energy consumed for all CPUs : 0.029041 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:11:10] 0.036889 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:11:25] Energy consumed for RAM : 0.007895 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:11:25] Energy consumed for all CPUs : 0.029218 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:11:25] 0.037114 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:11:40] Energy consumed for RAM : 0.007943 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:11:40] Energy consumed for all CPUs : 0.029395 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:11:40] 0.037339 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:11:55] Energy consumed for RAM : 0.007991 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:11:55] Energy consumed for all CPUs : 0.029572 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:11:55] 0.037563 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:12:10] Energy consumed for RAM : 0.008039 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:12:10] Energy consumed for all CPUs : 0.029749 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:12:10] 0.037788 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:12:10] 0.000036 g.CO2eq/s mean an estimation of 1.124104770998259 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:12:25] Energy consumed for RAM : 0.008087 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:12:25] Energy consumed for all CPUs : 0.029927 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:12:25] 0.038013 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:12:40] Energy consumed for RAM : 0.008135 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:12:40] Energy consumed for all CPUs : 0.030104 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:12:40] 0.038238 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:12:55] Energy consumed for RAM : 0.008183 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:12:55] Energy consumed for all CPUs : 0.030281 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:12:55] 0.038463 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:13:10] Energy consumed for RAM : 0.008230 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:13:10] Energy consumed for all CPUs : 0.030458 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:13:10] 0.038688 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:13:25] Energy consumed for RAM : 0.008278 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:13:25] Energy consumed for all CPUs : 0.030635 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:13:25] 0.038913 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:13:40] Energy consumed for RAM : 0.008326 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:13:40] Energy consumed for all CPUs : 0.030812 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:13:40] 0.039138 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:13:55] Energy consumed for RAM : 0.008374 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:13:55] Energy consumed for all CPUs : 0.030989 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:13:55] 0.039363 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:14:10] Energy consumed for RAM : 0.008422 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:14:10] Energy consumed for all CPUs : 0.031166 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:14:10] 0.039588 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:14:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241176300291527 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:14:25] Energy consumed for RAM : 0.008470 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:14:25] Energy consumed for all CPUs : 0.031343 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:14:25] 0.039813 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:14:40] Energy consumed for RAM : 0.008517 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:14:40] Energy consumed for all CPUs : 0.031520 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:14:40] 0.040038 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:14:55] Energy consumed for RAM : 0.008565 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:14:55] Energy consumed for all CPUs : 0.031697 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:14:55] 0.040263 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:15:10] Energy consumed for RAM : 0.008613 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:15:10] Energy consumed for all CPUs : 0.031874 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:15:10] 0.040488 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:15:25] Energy consumed for RAM : 0.008661 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:15:25] Energy consumed for all CPUs : 0.032051 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:15:25] 0.040712 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:15:40] Energy consumed for RAM : 0.008709 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:15:40] Energy consumed for all CPUs : 0.032229 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:15:40] 0.040937 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:15:55] Energy consumed for RAM : 0.008757 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:15:55] Energy consumed for all CPUs : 0.032406 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:15:55] 0.041162 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:16:10] Energy consumed for RAM : 0.008805 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:16:10] Energy consumed for all CPUs : 0.032583 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:16:10] 0.041387 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:16:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241178147419713 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:16:25] Energy consumed for RAM : 0.008852 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:16:25] Energy consumed for all CPUs : 0.032760 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:16:25] 0.041612 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:16:40] Energy consumed for RAM : 0.008900 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:16:40] Energy consumed for all CPUs : 0.032937 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:16:40] 0.041837 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:16:55] Energy consumed for RAM : 0.008948 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:16:55] Energy consumed for all CPUs : 0.033114 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:16:55] 0.042062 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:17:10] Energy consumed for RAM : 0.008996 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:17:10] Energy consumed for all CPUs : 0.033291 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:17:10] 0.042287 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:17:25] Energy consumed for RAM : 0.009044 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:17:25] Energy consumed for all CPUs : 0.033468 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:17:25] 0.042512 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:17:40] Energy consumed for RAM : 0.009092 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:17:40] Energy consumed for all CPUs : 0.033645 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:17:40] 0.042737 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:17:55] Energy consumed for RAM : 0.009140 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:17:55] Energy consumed for all CPUs : 0.033822 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:17:55] 0.042962 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:18:10] Energy consumed for RAM : 0.009187 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:18:10] Energy consumed for all CPUs : 0.033999 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:18:10] 0.043187 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:18:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241141077367505 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:18:25] Energy consumed for RAM : 0.009235 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:18:25] Energy consumed for all CPUs : 0.034176 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:18:25] 0.043412 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:18:40] Energy consumed for RAM : 0.009283 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:18:40] Energy consumed for all CPUs : 0.034353 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:18:40] 0.043637 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:18:55] Energy consumed for RAM : 0.009331 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:18:55] Energy consumed for all CPUs : 0.034531 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:18:55] 0.043861 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:19:10] Energy consumed for RAM : 0.009379 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:19:10] Energy consumed for all CPUs : 0.034708 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:19:10] 0.044086 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:19:25] Energy consumed for RAM : 0.009427 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:19:25] Energy consumed for all CPUs : 0.034885 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:19:25] 0.044311 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:19:40] Energy consumed for RAM : 0.009475 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:19:40] Energy consumed for all CPUs : 0.035062 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:19:40] 0.044536 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:19:55] Energy consumed for RAM : 0.009522 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:19:55] Energy consumed for all CPUs : 0.035239 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:19:55] 0.044761 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:20:10] Energy consumed for RAM : 0.009570 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:20:10] Energy consumed for all CPUs : 0.035416 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:20:10] 0.044986 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:20:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241206406423676 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:20:25] Energy consumed for RAM : 0.009618 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:20:25] Energy consumed for all CPUs : 0.035593 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:20:25] 0.045211 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:20:40] Energy consumed for RAM : 0.009666 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:20:40] Energy consumed for all CPUs : 0.035770 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:20:40] 0.045436 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:20:55] Energy consumed for RAM : 0.009714 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:20:55] Energy consumed for all CPUs : 0.035947 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:20:55] 0.045661 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:21:10] Energy consumed for RAM : 0.009762 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:21:10] Energy consumed for all CPUs : 0.036124 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:21:10] 0.045886 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:21:25] Energy consumed for RAM : 0.009809 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:21:25] Energy consumed for all CPUs : 0.036301 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:21:25] 0.046111 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:21:40] Energy consumed for RAM : 0.009857 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:21:40] Energy consumed for all CPUs : 0.036478 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:21:40] 0.046336 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:21:55] Energy consumed for RAM : 0.009905 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:21:55] Energy consumed for all CPUs : 0.036655 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:21:55] 0.046561 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:22:10] Energy consumed for RAM : 0.009953 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:22:10] Energy consumed for all CPUs : 0.036833 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:22:10] 0.046786 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:22:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241260591903481 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:22:25] Energy consumed for RAM : 0.010001 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:22:25] Energy consumed for all CPUs : 0.037010 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:22:25] 0.047011 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:22:40] Energy consumed for RAM : 0.010049 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:22:40] Energy consumed for all CPUs : 0.037187 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:22:40] 0.047235 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:22:55] Energy consumed for RAM : 0.010097 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:22:55] Energy consumed for all CPUs : 0.037364 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:22:55] 0.047460 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:23:10] Energy consumed for RAM : 0.010144 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:23:10] Energy consumed for all CPUs : 0.037541 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:23:10] 0.047685 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:23:25] Energy consumed for RAM : 0.010192 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:23:25] Energy consumed for all CPUs : 0.037718 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:23:25] 0.047910 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:23:40] Energy consumed for RAM : 0.010240 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:23:40] Energy consumed for all CPUs : 0.037895 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:23:40] 0.048135 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:23:55] Energy consumed for RAM : 0.010288 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:23:55] Energy consumed for all CPUs : 0.038072 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:23:55] 0.048360 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:24:10] Energy consumed for RAM : 0.010336 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:24:10] Energy consumed for all CPUs : 0.038249 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:24:10] 0.048585 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:24:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241262053471515 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:24:25] Energy consumed for RAM : 0.010384 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:24:25] Energy consumed for all CPUs : 0.038426 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:24:25] 0.048810 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:24:40] Energy consumed for RAM : 0.010432 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:24:40] Energy consumed for all CPUs : 0.038603 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:24:40] 0.049035 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:24:55] Energy consumed for RAM : 0.010479 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:24:55] Energy consumed for all CPUs : 0.038780 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:24:55] 0.049260 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:25:10] Energy consumed for RAM : 0.010527 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:25:10] Energy consumed for all CPUs : 0.038957 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:25:10] 0.049485 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:25:25] Energy consumed for RAM : 0.010575 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:25:25] Energy consumed for all CPUs : 0.039135 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:25:25] 0.049710 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:25:40] Energy consumed for RAM : 0.010623 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:25:40] Energy consumed for all CPUs : 0.039312 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:25:40] 0.049935 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:25:55] Energy consumed for RAM : 0.010671 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:25:55] Energy consumed for all CPUs : 0.039489 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:25:55] 0.050160 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:26:10] Energy consumed for RAM : 0.010719 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:26:10] Energy consumed for all CPUs : 0.039666 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:26:10] 0.050384 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:26:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241243278754078 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:26:25] Energy consumed for RAM : 0.010767 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:26:25] Energy consumed for all CPUs : 0.039843 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:26:25] 0.050609 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:26:40] Energy consumed for RAM : 0.010814 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:26:40] Energy consumed for all CPUs : 0.040020 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:26:40] 0.050834 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:26:55] Energy consumed for RAM : 0.010862 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:26:55] Energy consumed for all CPUs : 0.040197 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:26:55] 0.051059 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:27:10] Energy consumed for RAM : 0.010910 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:27:10] Energy consumed for all CPUs : 0.040374 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:27:10] 0.051284 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:27:25] Energy consumed for RAM : 0.010958 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:27:25] Energy consumed for all CPUs : 0.040551 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:27:25] 0.051509 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:27:40] Energy consumed for RAM : 0.011006 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:27:40] Energy consumed for all CPUs : 0.040728 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:27:40] 0.051734 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:27:55] Energy consumed for RAM : 0.011054 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:27:55] Energy consumed for all CPUs : 0.040905 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:27:55] 0.051959 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:28:10] Energy consumed for RAM : 0.011101 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:28:10] Energy consumed for all CPUs : 0.041082 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:28:10] 0.052184 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:28:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241275665026853 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:28:25] Energy consumed for RAM : 0.011149 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:28:25] Energy consumed for all CPUs : 0.041260 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:28:25] 0.052409 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:28:40] Energy consumed for RAM : 0.011197 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:28:40] Energy consumed for all CPUs : 0.041437 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:28:40] 0.052634 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:28:55] Energy consumed for RAM : 0.011245 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:28:55] Energy consumed for all CPUs : 0.041614 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:28:55] 0.052859 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:29:10] Energy consumed for RAM : 0.011293 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:29:10] Energy consumed for all CPUs : 0.041791 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:29:10] 0.053084 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:29:25] Energy consumed for RAM : 0.011341 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:29:25] Energy consumed for all CPUs : 0.041968 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:29:25] 0.053309 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:29:40] Energy consumed for RAM : 0.011389 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:29:40] Energy consumed for all CPUs : 0.042145 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:29:40] 0.053534 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:29:55] Energy consumed for RAM : 0.011436 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:29:55] Energy consumed for all CPUs : 0.042322 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:29:55] 0.053758 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:30:10] Energy consumed for RAM : 0.011484 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:30:10] Energy consumed for all CPUs : 0.042499 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:30:10] 0.053983 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:30:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241325993989661 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:30:25] Energy consumed for RAM : 0.011532 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:30:25] Energy consumed for all CPUs : 0.042676 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:30:25] 0.054208 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:30:40] Energy consumed for RAM : 0.011580 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:30:40] Energy consumed for all CPUs : 0.042853 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:30:40] 0.054433 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:30:55] Energy consumed for RAM : 0.011628 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:30:55] Energy consumed for all CPUs : 0.043030 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:30:55] 0.054658 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:31:10] Energy consumed for RAM : 0.011676 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:31:10] Energy consumed for all CPUs : 0.043207 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:31:10] 0.054883 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:31:25] Energy consumed for RAM : 0.011724 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:31:25] Energy consumed for all CPUs : 0.043385 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:31:25] 0.055108 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:31:40] Energy consumed for RAM : 0.011771 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:31:40] Energy consumed for all CPUs : 0.043562 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:31:40] 0.055333 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:31:55] Energy consumed for RAM : 0.011819 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:31:55] Energy consumed for all CPUs : 0.043739 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:31:55] 0.055558 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:32:10] Energy consumed for RAM : 0.011867 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:32:10] Energy consumed for all CPUs : 0.043916 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:32:10] 0.055783 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:32:10] 0.000036 g.CO2eq/s mean an estimation of 1.124129726197665 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:32:25] Energy consumed for RAM : 0.011915 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:32:25] Energy consumed for all CPUs : 0.044093 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:32:25] 0.056008 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:32:40] Energy consumed for RAM : 0.011963 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:32:40] Energy consumed for all CPUs : 0.044270 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:32:40] 0.056233 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:32:55] Energy consumed for RAM : 0.012011 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:32:55] Energy consumed for all CPUs : 0.044447 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:32:55] 0.056458 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:33:10] Energy consumed for RAM : 0.012059 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:33:10] Energy consumed for all CPUs : 0.044624 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:33:10] 0.056683 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:33:25] Energy consumed for RAM : 0.012106 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:33:25] Energy consumed for all CPUs : 0.044801 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:33:25] 0.056908 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:33:40] Energy consumed for RAM : 0.012154 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:33:40] Energy consumed for all CPUs : 0.044978 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:33:40] 0.057132 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:33:55] Energy consumed for RAM : 0.012202 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:33:55] Energy consumed for all CPUs : 0.045155 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:33:55] 0.057357 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:34:10] Energy consumed for RAM : 0.012250 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:34:10] Energy consumed for all CPUs : 0.045332 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:34:10] 0.057582 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:34:10] 0.000036 g.CO2eq/s mean an estimation of 1.124126066416576 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:34:25] Energy consumed for RAM : 0.012298 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:34:25] Energy consumed for all CPUs : 0.045510 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:34:25] 0.057807 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:34:40] Energy consumed for RAM : 0.012346 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:34:40] Energy consumed for all CPUs : 0.045687 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:34:40] 0.058032 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:34:55] Energy consumed for RAM : 0.012393 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:34:55] Energy consumed for all CPUs : 0.045864 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:34:55] 0.058257 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:35:10] Energy consumed for RAM : 0.012441 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:35:10] Energy consumed for all CPUs : 0.046041 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:35:10] 0.058482 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:35:25] Energy consumed for RAM : 0.012489 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:35:25] Energy consumed for all CPUs : 0.046218 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:35:25] 0.058707 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:35:40] Energy consumed for RAM : 0.012537 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:35:40] Energy consumed for all CPUs : 0.046395 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:35:40] 0.058932 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:35:55] Energy consumed for RAM : 0.012585 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:35:55] Energy consumed for all CPUs : 0.046572 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:35:55] 0.059157 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:36:10] Energy consumed for RAM : 0.012633 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:36:10] Energy consumed for all CPUs : 0.046749 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:36:10] 0.059382 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:36:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241356727556815 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:36:25] Energy consumed for RAM : 0.012681 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:36:25] Energy consumed for all CPUs : 0.046926 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:36:25] 0.059607 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:36:40] Energy consumed for RAM : 0.012728 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:36:40] Energy consumed for all CPUs : 0.047103 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:36:40] 0.059832 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:36:55] Energy consumed for RAM : 0.012776 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:36:55] Energy consumed for all CPUs : 0.047280 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:36:55] 0.060057 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:37:10] Energy consumed for RAM : 0.012824 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:37:10] Energy consumed for all CPUs : 0.047457 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:37:10] 0.060282 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:37:25] Energy consumed for RAM : 0.012872 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:37:25] Energy consumed for all CPUs : 0.047635 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:37:25] 0.060507 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:37:40] Energy consumed for RAM : 0.012920 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:37:40] Energy consumed for all CPUs : 0.047812 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:37:40] 0.060731 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:37:55] Energy consumed for RAM : 0.012968 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:37:55] Energy consumed for all CPUs : 0.047989 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:37:55] 0.060956 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:38:10] Energy consumed for RAM : 0.013016 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:38:10] Energy consumed for all CPUs : 0.048166 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:38:10] 0.061181 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:38:10] 0.000036 g.CO2eq/s mean an estimation of 1.124127331411236 kg.CO2eq/year\n",
+      "[codecarbon INFO @ 10:38:25] Energy consumed for RAM : 0.013063 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:38:25] Energy consumed for all CPUs : 0.048343 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:38:25] 0.061406 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:38:40] Energy consumed for RAM : 0.013111 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:38:40] Energy consumed for all CPUs : 0.048520 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:38:40] 0.061631 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:38:55] Energy consumed for RAM : 0.013159 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:38:55] Energy consumed for all CPUs : 0.048697 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:38:55] 0.061856 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:39:10] Energy consumed for RAM : 0.013207 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:39:10] Energy consumed for all CPUs : 0.048874 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:39:10] 0.062081 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:39:25] Energy consumed for RAM : 0.013255 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:39:25] Energy consumed for all CPUs : 0.049051 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:39:25] 0.062306 kWh of electricity used since the beginning.\n",
+      "[codecarbon INFO @ 10:39:40] Energy consumed for RAM : 0.013303 kWh. RAM Power : 11.486040115356445 W\n",
+      "[codecarbon INFO @ 10:39:40] Energy consumed for all CPUs : 0.049228 kWh. Total CPU Power : 42.5 W\n",
+      "[codecarbon INFO @ 10:39:40] 0.062531 kWh of electricity used since the beginning.\n"
      ]
     }
    ],
@@ -3818,7 +4285,7 @@
     "\n",
     "#@track_emissions()\n",
     "def compute_ndvi_phr():\n",
-    "    input_data_phr = xarray.open_dataarray(phr_product)\n",
+    "    input_data_phr = xarray.open_dataarray(phr_product_cog)\n",
     "    ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_phr[0],input_data_phr[3])\n",
     "    output_file = Path(f\"{output_dir}/ndvi_ufunc_phr.tif\")\n",
     "    ndvi_computed.rio.to_raster(output_file)\n",
@@ -3853,7 +4320,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.11.10"
   }
  },
  "nbformat": 4,

From 59567050cbdad97b8c0a3a839cc955ccd3548c3b Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Fri, 14 Feb 2025 10:19:26 +0000
Subject: [PATCH 28/37] Separate data read in a cell to demonstrate the manual
 chunks size

---
 tuto_greenit.ipynb | 397 +++++++++++++++++++++++++--------------------
 1 file changed, 224 insertions(+), 173 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index e2748f7..1cce6d1 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -284,13 +284,13 @@
    },
    "outputs": [],
    "source": [
-    "data_dir = \"/work/scratch/data/romaint\"\n",
-    "s2_b4 = f\"{data_dir}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
-    "s2_b8 = f\"{data_dir}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
+    "input_dir = \"/work/scratch/data/romaint/input_greenit\"\n",
+    "output_dir = \"/work/scratch/data/romaint/output_greenit\"\n",
+    "s2_b4 = f\"{input_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
+    "s2_b8 = f\"{input_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
     "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
-    "phr_product_cog = f\"{data_dir}/input_greenit/phr_cog.tif\"\n",
-    "phr_product_lzw = f\"{data_dir}/input_greenit/phr_lzw.tif\"\n",
-    "output_dir = f\"{data_dir}/output_greenit\""
+    "phr_product_cog = f\"{input_dir}/phr_cog.tif\"\n",
+    "phr_product_lzw = f\"{input_dir}/phr_lzw.tif\""
    ]
   },
   {
@@ -383,7 +383,7 @@
      "text": [
       "/usr/local/lib/python3.10/dist-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n",
       "Perhaps you already have a cluster running?\n",
-      "Hosting the HTTP server on port 37823 instead\n",
+      "Hosting the HTTP server on port 45641 instead\n",
       "  warnings.warn(\n"
      ]
     },
@@ -391,7 +391,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Dask Dashboard:  http://127.0.0.1:37823/status\n"
+      "Dask Dashboard:  http://127.0.0.1:45641/status\n"
      ]
     },
     {
@@ -401,7 +401,7 @@
        "    
     
    \n",
        "    
    \n",
        "        
    Client
    \n",
-       "        
    Client-de5a8946-ea11-11ef-b9f9-5c6f69244cc4
    \n",
+       "        
    Client-9ef29c9c-eabc-11ef-80bd-84160c1e7634
    \n",
        "        \n",
        "\n",
        "        \n",
@@ -414,7 +414,7 @@
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
@@ -423,7 +423,7 @@
        "        
    \n",
-       "                    Dashboard:  http://127.0.0.1:37823/status\n",
+       "                    Dashboard:  http://127.0.0.1:45641/status\n",
        "                
    \n",
        "\n",
        "        \n",
-       "            \n",
        "        \n",
@@ -436,11 +436,11 @@
        "    
    \n",
        "    
    \n",
        "        
    LocalCluster
    \n",
-       "        
    145210b0
    \n",
+       "        
    ae49023c
    \n",
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
        "            \n",
@@ -473,11 +473,11 @@
        "        
     
    \n",
        "        
    \n",
        "            
    Scheduler
    \n",
-       "            
    Scheduler-5e671663-16ec-414a-bc99-d642721f0cf3
    \n",
+       "            
    Scheduler-b3c6abd1-f6ce-4fd6-91aa-5302a9068ade
    \n",
        "            
    \n",
-       "                    Dashboard: http://127.0.0.1:37823/status\n",
+       "                    Dashboard: http://127.0.0.1:45641/status\n",
        "                \n",
        "                    Workers: 4\n",
@@ -448,10 +448,10 @@
        "            
    \n",
-       "                    Total threads: 4\n",
+       "                    Total threads: 8\n",
        "                \n",
-       "                    Total memory: 28.00 GiB\n",
+       "                    Total memory: 56.00 GiB\n",
        "                
    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                \n",
@@ -496,7 +496,7 @@
        "                        Started: Just now\n",
        "                    \n",
        "                    \n",
        "                \n",
        "            
    \n",
-       "                        Comm: tcp://127.0.0.1:45279\n",
+       "                        Comm: tcp://127.0.0.1:33991\n",
        "                    \n",
        "                        Workers: 4\n",
@@ -485,10 +485,10 @@
        "                
    \n",
-       "                        Dashboard: http://127.0.0.1:37823/status\n",
+       "                        Dashboard: http://127.0.0.1:45641/status\n",
        "                    \n",
-       "                        Total threads: 4\n",
+       "                        Total threads: 8\n",
        "                    
    \n",
-       "                        Total memory: 28.00 GiB\n",
+       "                        Total memory: 56.00 GiB\n",
        "                    
    \n",
@@ -519,29 +519,29 @@
        "                \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -564,29 +564,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:40603\n",
+       "                            Comm:  tcp://127.0.0.1:46165\n",
        "                        \n",
-       "                            Total threads:  1\n",
+       "                            Total threads:  2\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:43941/status\n",
+       "                            Dashboard:  http://127.0.0.1:37937/status\n",
        "                        \n",
-       "                            Memory:  7.00 GiB\n",
+       "                            Memory:  14.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:42307\n",
+       "                            Nanny:  tcp://127.0.0.1:43337\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32787395/dask-scratch-space/worker-chbni3_s\n",
+       "                            Local directory:  /tmp/slurm-32831596/dask-scratch-space/worker-2or14pgx\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -609,29 +609,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:41709\n",
+       "                            Comm:  tcp://127.0.0.1:41899\n",
        "                        \n",
-       "                            Total threads:  1\n",
+       "                            Total threads:  2\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:45009/status\n",
+       "                            Dashboard:  http://127.0.0.1:33081/status\n",
        "                        \n",
-       "                            Memory:  7.00 GiB\n",
+       "                            Memory:  14.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:36113\n",
+       "                            Nanny:  tcp://127.0.0.1:44565\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32787395/dask-scratch-space/worker-ir_x2jjp\n",
+       "                            Local directory:  /tmp/slurm-32831596/dask-scratch-space/worker-p4xpbz51\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -654,29 +654,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:36627\n",
+       "                            Comm:  tcp://127.0.0.1:44637\n",
        "                        \n",
-       "                            Total threads:  1\n",
+       "                            Total threads:  2\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:38577/status\n",
+       "                            Dashboard:  http://127.0.0.1:39187/status\n",
        "                        \n",
-       "                            Memory:  7.00 GiB\n",
+       "                            Memory:  14.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:44667\n",
+       "                            Nanny:  tcp://127.0.0.1:38469\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32787395/dask-scratch-space/worker-yyl1fiof\n",
+       "                            Local directory:  /tmp/slurm-32831596/dask-scratch-space/worker-79_jb3ot\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -703,7 +703,7 @@
        ""
       ],
       "text/plain": [
-       ""
+       ""
       ]
      },
      "execution_count": 4,
@@ -879,8 +879,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "CPU times: user 348 ms, sys: 1.68 s, total: 2.03 s\n",
-      "Wall time: 3.72 s\n"
+      "CPU times: user 387 ms, sys: 1.65 s, total: 2.04 s\n",
+      "Wall time: 4 s\n"
      ]
     }
    ],
@@ -906,7 +906,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 7,
    "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7",
    "metadata": {
     "tags": []
@@ -932,7 +932,7 @@
        "0"
       ]
      },
-     "execution_count": 9,
+     "execution_count": 7,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1015,7 +1015,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 9,
    "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
    "metadata": {
     "scrolled": true,
@@ -1026,12 +1026,12 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with Raster IO + numba = 1.651143898256123s\n",
-      "Elapsed with Raster IO + without numba = 1.3101588021963835s\n",
-      "Elapsed with Xarray + apply ufunc = 2.5775902681052685s\n",
-      "Elapsed with RIOXarray + dask = 5.034341523423791s\n",
-      "CPU times: user 2.68 s, sys: 6.16 s, total: 8.84 s\n",
-      "Wall time: 10.6 s\n"
+      "Elapsed with Raster IO + numba = 2.0380662083625793s\n",
+      "Elapsed with Raster IO + without numba = 1.5673291310667992s\n",
+      "Elapsed with Xarray + apply ufunc = 2.8014243729412556s\n",
+      "Elapsed with RIOXarray + dask = 4.730989769101143s\n",
+      "CPU times: user 2.49 s, sys: 6.27 s, total: 8.77 s\n",
+      "Wall time: 11.1 s\n"
      ]
     }
    ],
@@ -1808,7 +1808,7 @@
        "Attributes:\n",
        "    _FillValue:    0\n",
        "    scale_factor:  1.0\n",
-       "    add_offset:    0.0
    \n",
-       "                            Comm:  tcp://127.0.0.1:42359\n",
+       "                            Comm:  tcp://127.0.0.1:40807\n",
        "                        \n",
-       "                            Total threads:  1\n",
+       "                            Total threads:  2\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:33623/status\n",
+       "                            Dashboard:  http://127.0.0.1:36543/status\n",
        "                        \n",
-       "                            Memory:  7.00 GiB\n",
+       "                            Memory:  14.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:40163\n",
+       "                            Nanny:  tcp://127.0.0.1:34625\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32787395/dask-scratch-space/worker-lo45hp8m\n",
+       "                            Local directory:  /tmp/slurm-32831596/dask-scratch-space/worker-wdk3d8jb\n",
        "                        
    \n",
+       "    add_offset:    0.0
    \n",
        "    \n",
        "        
    \n",
        "            \n",
@@ -1910,19 +1910,19 @@
        "\n",
        "        \n",
        "    \n",
-       "
    • band
      (band)
      int64
      1 2 3 4
      array([1, 2, 3, 4])
    • x
      (x)
      float64
      0.5 1.5 2.5 ... 3.984e+04 3.984e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 3.98415e+04, 3.98425e+04,\n",
-       "       3.98435e+04])
    • y
      (y)
      float64
      0.5 1.5 2.5 ... 4.166e+04 4.166e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 4.16605e+04, 4.16615e+04,\n",
-       "       4.16625e+04])
    • spatial_ref
      ()
      int64
      0
      GeoTransform :
      0.0 1.0 0.0 0.0 0.0 1.0
      array(0)
    • band
      PandasIndex
      PandasIndex(Index([1, 2, 3, 4], dtype='int64', name='band'))
    • x
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
+       "
    • band
      (band)
      int64
      1 2 3 4
      array([1, 2, 3, 4])
    • x
      (x)
      float64
      0.5 1.5 2.5 ... 3.984e+04 3.984e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 3.98415e+04, 3.98425e+04,\n",
+       "       3.98435e+04])
    • y
      (y)
      float64
      0.5 1.5 2.5 ... 4.166e+04 4.166e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 4.16605e+04, 4.16615e+04,\n",
+       "       4.16625e+04])
    • spatial_ref
      ()
      int64
      0
      GeoTransform :
      0.0 1.0 0.0 0.0 0.0 1.0
      array(0)
    • band
      PandasIndex
      PandasIndex(Index([1, 2, 3, 4], dtype='int64', name='band'))
    • x
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
        "           8.5,     9.5,\n",
        "       ...\n",
        "       39834.5, 39835.5, 39836.5, 39837.5, 39838.5, 39839.5, 39840.5, 39841.5,\n",
        "       39842.5, 39843.5],\n",
-       "      dtype='float64', name='x', length=39844))
    • y
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
+       "      dtype='float64', name='x', length=39844))
    • y
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
        "           8.5,     9.5,\n",
        "       ...\n",
        "       41653.5, 41654.5, 41655.5, 41656.5, 41657.5, 41658.5, 41659.5, 41660.5,\n",
        "       41661.5, 41662.5],\n",
-       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -1952,7 +1952,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": null,
    "id": "b8fb6e29-0a84-4ae0-882b-b973af22b054",
    "metadata": {
     "tags": []
@@ -1963,14 +1963,15 @@
      "output_type": "stream",
      "text": [
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Elapsed with automatic chunk size = 40.49029225949198s\n"
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n"
      ]
     }
    ],
@@ -1978,7 +1979,7 @@
     "start=time.perf_counter()\n",
     "ndvi_array = (data_array_autochunks[3] - data_array_autochunks[0]) / (data_array_autochunks[3] + data_array_autochunks[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_autochunks.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file)\n",
+    "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with automatic chunk size = {}s\".format((end - start)))"
    ]
@@ -1986,7 +1987,7 @@
   {
    "cell_type": "code",
    "execution_count": 7,
-   "id": "615be991-f60c-4353-8c18-fb4853c660eb",
+   "id": "8940e64c-1de7-4122-828b-e03a9f43a60b",
    "metadata": {
     "tags": []
    },
@@ -2364,12 +2365,17 @@
        "  stroke: currentColor;\n",
        "  fill: currentColor;\n",
        "}\n",
-       "
    <xarray.DataArray (y: 41663, x: 39844)> Size: 13GB\n",
-       "dask.array<truediv, shape=(41663, 39844), dtype=float64, chunksize=(8192, 8192), chunktype=numpy.ndarray>\n",
+       "
    <xarray.DataArray (band: 4, y: 41663, x: 39844)> Size: 13GB\n",
+       "dask.array<open_rasterio-378202cf7021cd8ef82947b521bf5afc<this-array>, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 2048, 2048), chunktype=numpy.ndarray>\n",
        "Coordinates:\n",
+       "  * band         (band) int64 32B 1 2 3 4\n",
        "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
        "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 8B 0
    • band
      (band)
      int64
      1 2 3 4
      array([1, 2, 3, 4])
    • x
      (x)
      float64
      0.5 1.5 2.5 ... 3.984e+04 3.984e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 3.98415e+04, 3.98425e+04,\n",
+       "       3.98435e+04])
    • y
      (y)
      float64
      0.5 1.5 2.5 ... 4.166e+04 4.166e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 4.16605e+04, 4.16615e+04,\n",
+       "       4.16625e+04])
    • spatial_ref
      ()
      int64
      0
      GeoTransform :
      0.0 1.0 0.0 0.0 0.0 1.0
      array(0)
    • band
      PandasIndex
      PandasIndex(Index([1, 2, 3, 4], dtype='int64', name='band'))
    • x
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
        "           8.5,     9.5,\n",
        "       ...\n",
        "       39834.5, 39835.5, 39836.5, 39837.5, 39838.5, 39839.5, 39840.5, 39841.5,\n",
        "       39842.5, 39843.5],\n",
-       "      dtype='float64', name='x', length=39844))
    • y
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
+       "      dtype='float64', name='x', length=39844))
    • y
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
        "           8.5,     9.5,\n",
        "       ...\n",
        "       41653.5, 41654.5, 41655.5, 41656.5, 41657.5, 41658.5, 41659.5, 41660.5,\n",
        "       41661.5, 41662.5],\n",
-       "      dtype='float64', name='y', length=41663))
    • "
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
-       " Size: 13GB\n",
-       "dask.array\n",
+       " Size: 13GB\n",
+       "dask.array, shape=(4, 41663, 39844), dtype=uint16, chunksize=(4, 2048, 2048), chunktype=numpy.ndarray>\n",
        "Coordinates:\n",
+       "  * band         (band) int64 32B 1 2 3 4\n",
        "  * x            (x) float64 319kB 0.5 1.5 2.5 ... 3.984e+04 3.984e+04 3.984e+04\n",
        "  * y            (y) float64 333kB 0.5 1.5 2.5 ... 4.166e+04 4.166e+04 4.166e+04\n",
-       "    spatial_ref  int64 8B 0"
+       "    spatial_ref  int64 8B 0\n",
+       "Attributes:\n",
+       "    _FillValue:    0\n",
+       "    scale_factor:  1.0\n",
+       "    add_offset:    0.0"
       ]
      },
      "execution_count": 7,
@@ -2462,98 +2559,52 @@
     }
    ],
    "source": [
-    "ndvi_array"
+    "reading_chunks = (-1,2048,2048)\n",
+    "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
+    "data_array_manualchunks"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 8,
    "id": "58bad330-9498-48ac-b3a0-cad211aaf998",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "ename": "RasterioIOError",
-     "evalue": "/work/scratch/data/romaint/input_greenit/phr_cog.tif: No such file or directory",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mCPLE_OpenFailedError\u001b[0m                      Traceback (most recent call last)",
-      "File \u001b[0;32mrasterio/_base.pyx:310\u001b[0m, in \u001b[0;36mrasterio._base.DatasetBase.__init__\u001b[0;34m()\u001b[0m\n",
-      "File \u001b[0;32mrasterio/_base.pyx:221\u001b[0m, in \u001b[0;36mrasterio._base.open_dataset\u001b[0;34m()\u001b[0m\n",
-      "File \u001b[0;32mrasterio/_err.pyx:359\u001b[0m, in \u001b[0;36mrasterio._err.exc_wrap_pointer\u001b[0;34m()\u001b[0m\n",
-      "\u001b[0;31mCPLE_OpenFailedError\u001b[0m: /work/scratch/data/romaint/input_greenit/phr_cog.tif: No such file or directory",
-      "\nDuring handling of the above exception, another exception occurred:\n",
-      "\u001b[0;31mRasterioIOError\u001b[0m                           Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[5], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m reading_chunks \u001b[38;5;241m=\u001b[39m (\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m,\u001b[38;5;241m2048\u001b[39m,\u001b[38;5;241m2048\u001b[39m)\n\u001b[0;32m----> 2\u001b[0m data_array_manualchunks \u001b[38;5;241m=\u001b[39m \u001b[43mrxr\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_rasterio\u001b[49m\u001b[43m(\u001b[49m\u001b[43mphr_product_cog\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mchunks\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mreading_chunks\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlock\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m)\u001b[49m\n\u001b[1;32m      3\u001b[0m start\u001b[38;5;241m=\u001b[39mtime\u001b[38;5;241m.\u001b[39mperf_counter()\n\u001b[1;32m      4\u001b[0m ndvi_array \u001b[38;5;241m=\u001b[39m (data_array_manualchunks[\u001b[38;5;241m3\u001b[39m] \u001b[38;5;241m-\u001b[39m data_array_manualchunks[\u001b[38;5;241m0\u001b[39m]) \u001b[38;5;241m/\u001b[39m (data_array_manualchunks[\u001b[38;5;241m3\u001b[39m] \u001b[38;5;241m+\u001b[39m data_array_manualchunks[\u001b[38;5;241m0\u001b[39m])\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/_io.py:1128\u001b[0m, in \u001b[0;36mopen_rasterio\u001b[0;34m(filename, parse_coordinates, chunks, cache, lock, masked, mask_and_scale, variable, group, default_name, decode_times, decode_timedelta, band_as_variable, **open_kwargs)\u001b[0m\n\u001b[1;32m   1126\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m   1127\u001b[0m         manager \u001b[38;5;241m=\u001b[39m URIManager(file_opener, filename, mode\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m, kwargs\u001b[38;5;241m=\u001b[39mopen_kwargs)\n\u001b[0;32m-> 1128\u001b[0m     riods \u001b[38;5;241m=\u001b[39m \u001b[43mmanager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43macquire\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1129\u001b[0m     captured_warnings \u001b[38;5;241m=\u001b[39m rio_warnings\u001b[38;5;241m.\u001b[39mcopy()\n\u001b[1;32m   1131\u001b[0m \u001b[38;5;66;03m# raise the NotGeoreferencedWarning if applicable\u001b[39;00m\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/_io.py:263\u001b[0m, in \u001b[0;36mURIManager.acquire\u001b[0;34m(self, needs_lock)\u001b[0m\n\u001b[1;32m    259\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_local\u001b[38;5;241m.\u001b[39mthread_manager \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m    260\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_local\u001b[38;5;241m.\u001b[39mthread_manager \u001b[38;5;241m=\u001b[39m ThreadURIManager(\n\u001b[1;32m    261\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_opener, \u001b[38;5;241m*\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_args, mode\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_mode, kwargs\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_kwargs\n\u001b[1;32m    262\u001b[0m     )\n\u001b[0;32m--> 263\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_local\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mthread_manager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfile_handle\u001b[49m\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/_io.py:219\u001b[0m, in \u001b[0;36mThreadURIManager.file_handle\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    217\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_file_handle \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m    218\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_file_handle\n\u001b[0;32m--> 219\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_file_handle \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_opener\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_args\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmode\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_mode\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_kwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    220\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_file_handle\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rasterio/env.py:463\u001b[0m, in \u001b[0;36mensure_env_with_credentials..wrapper\u001b[0;34m(*args, **kwds)\u001b[0m\n\u001b[1;32m    460\u001b[0m     session \u001b[38;5;241m=\u001b[39m DummySession()\n\u001b[1;32m    462\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m env_ctor(session\u001b[38;5;241m=\u001b[39msession):\n\u001b[0;32m--> 463\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mf\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwds\u001b[49m\u001b[43m)\u001b[49m\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rasterio/__init__.py:355\u001b[0m, in \u001b[0;36mopen\u001b[0;34m(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, opener, **kwargs)\u001b[0m\n\u001b[1;32m    352\u001b[0m     path \u001b[38;5;241m=\u001b[39m _parse_path(raw_dataset_path)\n\u001b[1;32m    354\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[0;32m--> 355\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m \u001b[43mDatasetReader\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpath\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdriver\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdriver\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msharing\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msharing\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    356\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr+\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m    357\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m get_writer_for_path(path, driver\u001b[38;5;241m=\u001b[39mdriver)(\n\u001b[1;32m    358\u001b[0m         path, mode, driver\u001b[38;5;241m=\u001b[39mdriver, sharing\u001b[38;5;241m=\u001b[39msharing, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs\n\u001b[1;32m    359\u001b[0m     )\n",
-      "File \u001b[0;32mrasterio/_base.pyx:312\u001b[0m, in \u001b[0;36mrasterio._base.DatasetBase.__init__\u001b[0;34m()\u001b[0m\n",
-      "\u001b[0;31mRasterioIOError\u001b[0m: /work/scratch/data/romaint/input_greenit/phr_cog.tif: No such file or directory"
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n",
+      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
+      "  return self.func(*new_argspec)\n"
      ]
     },
     {
-     "name": "stderr",
+     "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[codecarbon INFO @ 10:39:55] Energy consumed for RAM : 0.013351 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:39:55] Energy consumed for all CPUs : 0.049405 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:39:55] 0.062756 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:40:10] Energy consumed for RAM : 0.013398 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:40:10] Energy consumed for all CPUs : 0.049582 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:40:10] 0.062981 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:40:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241089623840645 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:40:25] Energy consumed for RAM : 0.013446 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:40:25] Energy consumed for all CPUs : 0.049759 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:40:25] 0.063206 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:40:40] Energy consumed for RAM : 0.013494 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:40:40] Energy consumed for all CPUs : 0.049937 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:40:40] 0.063431 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:40:55] Energy consumed for RAM : 0.013542 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:40:55] Energy consumed for all CPUs : 0.050114 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:40:55] 0.063656 kWh of electricity used since the beginning.\n"
+      "Elapsed with manual chunk size = 55.72775740176439s\n"
      ]
     }
    ],
    "source": [
-    "reading_chunks = (-1,2048,2048)\n",
-    "data_array_manualchunks = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
     "start=time.perf_counter()\n",
     "ndvi_array = (data_array_manualchunks[3] - data_array_manualchunks[0]) / (data_array_manualchunks[3] + data_array_manualchunks[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_manualchunks.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file)\n",
+    "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
     "end=time.perf_counter()\n",
     "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "32e153c5-1ccc-47b2-a5c2-1270ab9d75b1",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "ename": "NameError",
-     "evalue": "name 'ndvi_array' is not defined",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[4], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mndvi_array\u001b[49m\n",
-      "\u001b[0;31mNameError\u001b[0m: name 'ndvi_array' is not defined"
-     ]
-    }
-   ],
-   "source": [
-    "ndvi_array"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "8622d62f-a986-4893-a932-a44bc2e5c496",
@@ -4320,7 +4371,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.10"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

From 662103a527380ecb68bc7c9484e70ac680fc26c6 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Fri, 14 Feb 2025 10:41:50 +0000
Subject: [PATCH 29/37] Update results

---
 tuto_greenit.ipynb | 1051 +++-----------------------------------------
 1 file changed, 60 insertions(+), 991 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 1cce6d1..e3e629a 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -305,7 +305,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 11,
    "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62",
    "metadata": {
     "tags": []
@@ -758,7 +758,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 14,
    "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f",
    "metadata": {
     "tags": []
@@ -855,7 +855,7 @@
        "dask.array"
       ]
      },
-     "execution_count": 5,
+     "execution_count": 14,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1015,7 +1015,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 15,
    "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
    "metadata": {
     "scrolled": true,
@@ -1026,12 +1026,12 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with Raster IO + numba = 2.0380662083625793s\n",
-      "Elapsed with Raster IO + without numba = 1.5673291310667992s\n",
-      "Elapsed with Xarray + apply ufunc = 2.8014243729412556s\n",
-      "Elapsed with RIOXarray + dask = 4.730989769101143s\n",
-      "CPU times: user 2.49 s, sys: 6.27 s, total: 8.77 s\n",
-      "Wall time: 11.1 s\n"
+      "Time to compute with Raster IO + numba = 1.9315631836652756s\n",
+      "Time to compute with Raster IO + without numba = 2.2189621636644006s\n",
+      "Time to compute with Xarray + apply ufunc = 3.0127522442489862s\n",
+      "Time to compute with RIOXarray + dask = 5.733268056996167s\n",
+      "CPU times: user 2.72 s, sys: 6.51 s, total: 9.22 s\n",
+      "Wall time: 12.9 s\n"
      ]
     }
    ],
@@ -1071,7 +1071,7 @@
     "output_file = Path(f\"{output_dir}/ndvi_numba.tif\")\n",
     "create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with Raster IO + numba = {}s\".format((end - start)))\n",
+    "print(\"Time to compute with Raster IO + numba = {}s\".format((end - start)))\n",
     "\n",
     "start = time.perf_counter()\n",
     "with rasterio.open(s2_b4, 'r') as ds:\n",
@@ -1085,7 +1085,7 @@
     "output_file = Path(f\"{output_dir}/ndvi_without_numba.tif\")\n",
     "create_raster(ndvi_computed, output_file, x_res , y_res,top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with Raster IO + without numba = {}s\".format((end - start)))\n",
+    "print(\"Time to compute with Raster IO + without numba = {}s\".format((end - start)))\n",
     "# Example with xarray\n",
     "start = time.perf_counter()\n",
     "input_data_b4 =  xarray.open_dataarray(s2_b4)\n",
@@ -1094,7 +1094,7 @@
     "output_file = Path(f\"{output_dir}/ndvi_ufunc.tif\")\n",
     "ndvi_computed.rio.to_raster(output_file)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with Xarray + apply ufunc = {}s\".format((end - start)))\n",
+    "print(\"Time to compute with Xarray + apply ufunc = {}s\".format((end - start)))\n",
     "\n",
     "start = time.perf_counter()\n",
     "input_data, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], (-1,2200,2200), False)\n",
@@ -1102,7 +1102,7 @@
     "output_file = Path(f\"{output_dir}/ndvi_dask.tif\")\n",
     "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with RIOXarray + dask = {}s\".format((end - start)))\n"
+    "print(\"Time to compute with RIOXarray + dask = {}s\".format((end - start)))\n"
    ]
   },
   {
@@ -1295,7 +1295,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 16,
    "id": "8e7a9fdb-3844-432a-bc43-fbcc2c5169cd",
    "metadata": {
     "tags": []
@@ -1313,7 +1313,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with LZW compressed file = 190.28657131735235s\n"
+      "Time to compute with LZW compressed file = 148.64071926195174s\n"
      ]
     },
     {
@@ -1328,7 +1328,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with CoG compressed file = 35.8363040285185s\n"
+      "Time to compute with CoG compressed file = 50.54504857957363s\n"
      ]
     }
    ],
@@ -1340,7 +1340,7 @@
     "output_file = Path(f\"{output_dir}/ndvi_lzw.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
-    "print(\"Elapsed with LZW compressed file = {}s\".format((end - start)))\n",
+    "print(\"Time to compute with LZW compressed file = {}s\".format((end - start)))\n",
     "\n",
     "start=time.perf_counter()\n",
     "data_array_cog = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
@@ -1348,7 +1348,7 @@
     "output_file = Path(f\"{output_dir}/ndvi_cog.tif\")\n",
     "ndvi_array.rio.to_raster(output_file)\n",
     "end=time.perf_counter()\n",
-    "print(\"Elapsed with CoG compressed file = {}s\".format((end - start)))"
+    "print(\"Time to compute with CoG compressed file = {}s\".format((end - start)))"
    ]
   },
   {
@@ -1419,7 +1419,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 17,
    "id": "57ae950e-2ea3-48a9-87ba-3b8a91de7c93",
    "metadata": {
     "tags": []
@@ -1808,7 +1808,7 @@
        "Attributes:\n",
        "    _FillValue:    0\n",
        "    scale_factor:  1.0\n",
-       "    add_offset:    0.0"
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -1938,7 +1938,7 @@
        "    add_offset:    0.0"
       ]
      },
-     "execution_count": 5,
+     "execution_count": 17,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1952,7 +1952,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 18,
    "id": "b8fb6e29-0a84-4ae0-882b-b973af22b054",
    "metadata": {
     "tags": []
@@ -1963,15 +1963,14 @@
      "output_type": "stream",
      "text": [
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n"
+      "  dataset = writer(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Time to compute with automatic chunk size = 56.06073514930904s\n"
      ]
     }
    ],
@@ -1981,12 +1980,12 @@
     "output_file = Path(f\"{output_dir}/ndvi_autochunks.tif\")\n",
     "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
     "end=time.perf_counter()\n",
-    "print(\"Elapsed with automatic chunk size = {}s\".format((end - start)))"
+    "print(\"Time to compute with automatic chunk size = {}s\".format((end - start)))"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 19,
    "id": "8940e64c-1de7-4122-828b-e03a9f43a60b",
    "metadata": {
     "tags": []
@@ -2375,7 +2374,7 @@
        "Attributes:\n",
        "    _FillValue:    0\n",
        "    scale_factor:  1.0\n",
-       "    add_offset:    0.0"
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -2553,7 +2552,7 @@
        "    add_offset:    0.0"
       ]
      },
-     "execution_count": 7,
+     "execution_count": 19,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -2566,7 +2565,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 20,
    "id": "58bad330-9498-48ac-b3a0-cad211aaf998",
    "metadata": {
     "tags": []
@@ -2577,22 +2576,14 @@
      "output_type": "stream",
      "text": [
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n",
-      "/usr/local/lib/python3.10/dist-packages/dask/_task_spec.py:651: RuntimeWarning: invalid value encountered in divide\n",
-      "  return self.func(*new_argspec)\n"
+      "  dataset = writer(\n"
      ]
     },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with manual chunk size = 55.72775740176439s\n"
+      "Time to compute with manual chunk size = 43.58590572234243s\n"
      ]
     }
    ],
@@ -2602,7 +2593,7 @@
     "output_file = Path(f\"{output_dir}/ndvi_manualchunks.tif\")\n",
     "ndvi_array.rio.to_raster(output_file,tiled=True)\n",
     "end=time.perf_counter()\n",
-    "print(\"Elapsed with manual chunk size = {}s\".format((end - start)))"
+    "print(\"Time to compute with manual chunk size = {}s\".format((end - start)))"
    ]
   },
   {
@@ -2683,7 +2674,7 @@
     "output_file = Path(f\"{output_dir}/ndvi_phr.tif\")\n",
     "ndvi_phr.rio.to_raster(output_file)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with RIOXarray + dask compute + product > 5Go = {}s\".format((end - start)))"
+    "print(\"Time to compute with RIOXarray + dask compute + product > 5Go = {}s\".format((end - start)))"
    ]
   },
   {
@@ -3297,7 +3288,7 @@
     "# Add the Tiled=true parameter to rxr to speed up the disk write\n",
     "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed with RIOXarray + tiled write + product > 5Go = {}s\".format((end - start)))"
+    "print(\"Time to compute with RIOXarray + tiled write + product > 5Go = {}s\".format((end - start)))"
    ]
   },
   {
@@ -3343,7 +3334,7 @@
     "**Conclusion for Memory usage**\n",
     "\n",
     "* Both method consumes the same amount of memory\n",
-    "* Cpu usage is **higher with dask**\n",
+    "* Cpu usage is **higher with dask compute**\n",
     "* You must use the tiled write in order to gain time"
    ]
   },
@@ -3381,943 +3372,21 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 9,
    "id": "ee231cc6-e45c-4209-9135-8c80e1a4eaf4",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[codecarbon INFO @ 09:30:04] [setup] RAM Tracking...\n",
-      "[codecarbon INFO @ 09:30:04] [setup] GPU Tracking...\n",
-      "[codecarbon INFO @ 09:30:04] No GPU found.\n",
-      "[codecarbon INFO @ 09:30:04] [setup] CPU Tracking...\n",
-      "[codecarbon WARNING @ 09:30:04] No CPU tracking mode found. Falling back on CPU constant mode. \n",
-      " Linux OS detected: Please ensure RAPL files exist at \\sys\\class\\powercap\\intel-rapl to measure CPU\n",
-      "\n",
-      "[codecarbon WARNING @ 09:30:05] We saw that you have a AMD Ryzen 7 7840U w/ Radeon  780M Graphics but we don't know it. Please contact us.\n",
-      "[codecarbon INFO @ 09:30:05] CPU Model on constant consumption mode: AMD Ryzen 7 7840U w/ Radeon  780M Graphics\n",
-      "[codecarbon INFO @ 09:30:05] >>> Tracker's metadata:\n",
-      "[codecarbon INFO @ 09:30:05]   Platform system: Linux-6.8.0-52-generic-x86_64-with-glibc2.39\n",
-      "[codecarbon INFO @ 09:30:05]   Python version: 3.11.10\n",
-      "[codecarbon INFO @ 09:30:05]   CodeCarbon version: 2.7.4\n",
-      "[codecarbon INFO @ 09:30:05]   Available RAM : 30.629 GB\n",
-      "[codecarbon INFO @ 09:30:05]   CPU count: 16\n",
-      "[codecarbon INFO @ 09:30:05]   CPU model: AMD Ryzen 7 7840U w/ Radeon  780M Graphics\n",
-      "[codecarbon INFO @ 09:30:05]   GPU count: None\n",
-      "[codecarbon INFO @ 09:30:05]   GPU model: None\n",
-      "[codecarbon WARNING @ 09:30:09] Unable to access geographical location through primary API. Will resort to using the backup API - Exception : ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) - url=https://get.geojs.io/v1/ip/geo.json\n",
-      "[codecarbon WARNING @ 09:30:10] Unable to access geographical location. Using 'Canada' as the default value - Exception : ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) - url=https://get.geojs.io/v1/ip/geo.json\n",
-      "[codecarbon INFO @ 09:30:10] Saving emissions data to file /home/tromain/Devel/pluto-tuto/emissions.csv\n"
-     ]
-    },
-    {
-     "ename": "RasterioIOError",
-     "evalue": "/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif: No such file or directory",
+     "ename": "ModuleNotFoundError",
+     "evalue": "No module named 'codecarbon'",
      "output_type": "error",
      "traceback": [
       "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/file_manager.py:211\u001b[0m, in \u001b[0;36mCachingFileManager._acquire_with_cache_info\u001b[0;34m(self, needs_lock)\u001b[0m\n\u001b[1;32m    210\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 211\u001b[0m     file \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_cache\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_key\u001b[49m\u001b[43m]\u001b[49m\n\u001b[1;32m    212\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m:\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/lru_cache.py:56\u001b[0m, in \u001b[0;36mLRUCache.__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m     55\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_lock:\n\u001b[0;32m---> 56\u001b[0m     value \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_cache\u001b[49m\u001b[43m[\u001b[49m\u001b[43mkey\u001b[49m\u001b[43m]\u001b[49m\n\u001b[1;32m     57\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_cache\u001b[38;5;241m.\u001b[39mmove_to_end(key)\n",
-      "\u001b[0;31mKeyError\u001b[0m: [, ('/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif',), 'r', (('sharing', False),), 'e5170fea-2adc-416a-81d4-e7ba85237e5c']",
-      "\nDuring handling of the above exception, another exception occurred:\n",
-      "\u001b[0;31mCPLE_OpenFailedError\u001b[0m                      Traceback (most recent call last)",
-      "File \u001b[0;32mrasterio/_base.pyx:310\u001b[0m, in \u001b[0;36mrasterio._base.DatasetBase.__init__\u001b[0;34m()\u001b[0m\n",
-      "File \u001b[0;32mrasterio/_base.pyx:221\u001b[0m, in \u001b[0;36mrasterio._base.open_dataset\u001b[0;34m()\u001b[0m\n",
-      "File \u001b[0;32mrasterio/_err.pyx:359\u001b[0m, in \u001b[0;36mrasterio._err.exc_wrap_pointer\u001b[0;34m()\u001b[0m\n",
-      "\u001b[0;31mCPLE_OpenFailedError\u001b[0m: /work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif: No such file or directory",
-      "\nDuring handling of the above exception, another exception occurred:\n",
-      "\u001b[0;31mRasterioIOError\u001b[0m                           Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[3], line 22\u001b[0m\n\u001b[1;32m     20\u001b[0m tracker\u001b[38;5;241m.\u001b[39mstart()\n\u001b[1;32m     21\u001b[0m start \u001b[38;5;241m=\u001b[39m time\u001b[38;5;241m.\u001b[39mperf_counter()\n\u001b[0;32m---> 22\u001b[0m \u001b[43mcompute_ndvi_sentinel2\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     23\u001b[0m end \u001b[38;5;241m=\u001b[39m time\u001b[38;5;241m.\u001b[39mperf_counter()\n\u001b[1;32m     24\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mElapsed to compute sentinel 2 NDVI = \u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mformat((end \u001b[38;5;241m-\u001b[39m start)))\n",
-      "Cell \u001b[0;32mIn[3], line 7\u001b[0m, in \u001b[0;36mcompute_ndvi_sentinel2\u001b[0;34m()\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcompute_ndvi_sentinel2\u001b[39m():\n\u001b[0;32m----> 7\u001b[0m     input_data_b4 \u001b[38;5;241m=\u001b[39m  \u001b[43mxarray\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_dataarray\u001b[49m\u001b[43m(\u001b[49m\u001b[43ms2_b4\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m      8\u001b[0m     input_data_b8 \u001b[38;5;241m=\u001b[39m  xarray\u001b[38;5;241m.\u001b[39mopen_dataarray(s2_b8)\n\u001b[1;32m      9\u001b[0m     ndvi_computed \u001b[38;5;241m=\u001b[39m xarray\u001b[38;5;241m.\u001b[39mapply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:852\u001b[0m, in \u001b[0;36mopen_dataarray\u001b[0;34m(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)\u001b[0m\n\u001b[1;32m    694\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mopen_dataarray\u001b[39m(\n\u001b[1;32m    695\u001b[0m     filename_or_obj: \u001b[38;5;28mstr\u001b[39m \u001b[38;5;241m|\u001b[39m os\u001b[38;5;241m.\u001b[39mPathLike[Any] \u001b[38;5;241m|\u001b[39m BufferedIOBase \u001b[38;5;241m|\u001b[39m AbstractDataStore,\n\u001b[1;32m    696\u001b[0m     \u001b[38;5;241m*\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    712\u001b[0m     \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[1;32m    713\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m DataArray:\n\u001b[1;32m    714\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"Open an DataArray from a file or file-like object containing a single\u001b[39;00m\n\u001b[1;32m    715\u001b[0m \u001b[38;5;124;03m    data variable.\u001b[39;00m\n\u001b[1;32m    716\u001b[0m \n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    849\u001b[0m \u001b[38;5;124;03m    open_dataset\u001b[39;00m\n\u001b[1;32m    850\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m--> 852\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m \u001b[43mopen_dataset\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    853\u001b[0m \u001b[43m        \u001b[49m\u001b[43mfilename_or_obj\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    854\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecode_cf\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_cf\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    855\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmask_and_scale\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmask_and_scale\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    856\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecode_times\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_times\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    857\u001b[0m \u001b[43m        \u001b[49m\u001b[43mconcat_characters\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mconcat_characters\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    858\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecode_coords\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_coords\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    859\u001b[0m \u001b[43m        \u001b[49m\u001b[43mengine\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mengine\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    860\u001b[0m \u001b[43m        \u001b[49m\u001b[43mchunks\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mchunks\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    861\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcache\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcache\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    862\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdrop_variables\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdrop_variables\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    863\u001b[0m \u001b[43m        \u001b[49m\u001b[43minline_array\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minline_array\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    864\u001b[0m \u001b[43m        \u001b[49m\u001b[43mchunked_array_type\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mchunked_array_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    865\u001b[0m \u001b[43m        \u001b[49m\u001b[43mfrom_array_kwargs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfrom_array_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    866\u001b[0m \u001b[43m        \u001b[49m\u001b[43mbackend_kwargs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mbackend_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    867\u001b[0m \u001b[43m        \u001b[49m\u001b[43muse_cftime\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43muse_cftime\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    868\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecode_timedelta\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_timedelta\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    869\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    870\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    872\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(dataset\u001b[38;5;241m.\u001b[39mdata_vars) \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m1\u001b[39m:\n\u001b[1;32m    873\u001b[0m         \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(dataset\u001b[38;5;241m.\u001b[39mdata_vars) \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m0\u001b[39m:\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/api.py:671\u001b[0m, in \u001b[0;36mopen_dataset\u001b[0;34m(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)\u001b[0m\n\u001b[1;32m    659\u001b[0m decoders \u001b[38;5;241m=\u001b[39m _resolve_decoders_kwargs(\n\u001b[1;32m    660\u001b[0m     decode_cf,\n\u001b[1;32m    661\u001b[0m     open_backend_dataset_parameters\u001b[38;5;241m=\u001b[39mbackend\u001b[38;5;241m.\u001b[39mopen_dataset_parameters,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    667\u001b[0m     decode_coords\u001b[38;5;241m=\u001b[39mdecode_coords,\n\u001b[1;32m    668\u001b[0m )\n\u001b[1;32m    670\u001b[0m overwrite_encoded_chunks \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mpop(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124moverwrite_encoded_chunks\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mNone\u001b[39;00m)\n\u001b[0;32m--> 671\u001b[0m backend_ds \u001b[38;5;241m=\u001b[39m \u001b[43mbackend\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_dataset\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    672\u001b[0m \u001b[43m    \u001b[49m\u001b[43mfilename_or_obj\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    673\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdrop_variables\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdrop_variables\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    674\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mdecoders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    675\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    676\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    677\u001b[0m ds \u001b[38;5;241m=\u001b[39m _dataset_from_backend_dataset(\n\u001b[1;32m    678\u001b[0m     backend_ds,\n\u001b[1;32m    679\u001b[0m     filename_or_obj,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    689\u001b[0m     \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[1;32m    690\u001b[0m )\n\u001b[1;32m    691\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m ds\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/xarray_plugin.py:58\u001b[0m, in \u001b[0;36mRasterioBackend.open_dataset\u001b[0;34m(self, filename_or_obj, drop_variables, parse_coordinates, lock, masked, mask_and_scale, variable, group, default_name, decode_coords, decode_times, decode_timedelta, band_as_variable, open_kwargs)\u001b[0m\n\u001b[1;32m     56\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m open_kwargs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m     57\u001b[0m     open_kwargs \u001b[38;5;241m=\u001b[39m {}\n\u001b[0;32m---> 58\u001b[0m rds \u001b[38;5;241m=\u001b[39m \u001b[43m_io\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen_rasterio\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m     59\u001b[0m \u001b[43m    \u001b[49m\u001b[43mfilename_or_obj\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     60\u001b[0m \u001b[43m    \u001b[49m\u001b[43mparse_coordinates\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mparse_coordinates\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     61\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcache\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m     62\u001b[0m \u001b[43m    \u001b[49m\u001b[43mlock\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlock\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     63\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmasked\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmasked\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     64\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmask_and_scale\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmask_and_scale\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     65\u001b[0m \u001b[43m    \u001b[49m\u001b[43mvariable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mvariable\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     66\u001b[0m \u001b[43m    \u001b[49m\u001b[43mgroup\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     67\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdefault_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdefault_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     68\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdecode_times\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_times\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     69\u001b[0m \u001b[43m    \u001b[49m\u001b[43mdecode_timedelta\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecode_timedelta\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     70\u001b[0m \u001b[43m    \u001b[49m\u001b[43mband_as_variable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mband_as_variable\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     71\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mopen_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m     72\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     73\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(rds, xarray\u001b[38;5;241m.\u001b[39mDataArray):\n\u001b[1;32m     74\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m rds\u001b[38;5;241m.\u001b[39mto_dataset()\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rioxarray/_io.py:1128\u001b[0m, in \u001b[0;36mopen_rasterio\u001b[0;34m(filename, parse_coordinates, chunks, cache, lock, masked, mask_and_scale, variable, group, default_name, decode_times, decode_timedelta, band_as_variable, **open_kwargs)\u001b[0m\n\u001b[1;32m   1126\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m   1127\u001b[0m         manager \u001b[38;5;241m=\u001b[39m URIManager(file_opener, filename, mode\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m, kwargs\u001b[38;5;241m=\u001b[39mopen_kwargs)\n\u001b[0;32m-> 1128\u001b[0m     riods \u001b[38;5;241m=\u001b[39m \u001b[43mmanager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43macquire\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1129\u001b[0m     captured_warnings \u001b[38;5;241m=\u001b[39m rio_warnings\u001b[38;5;241m.\u001b[39mcopy()\n\u001b[1;32m   1131\u001b[0m \u001b[38;5;66;03m# raise the NotGeoreferencedWarning if applicable\u001b[39;00m\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/file_manager.py:193\u001b[0m, in \u001b[0;36mCachingFileManager.acquire\u001b[0;34m(self, needs_lock)\u001b[0m\n\u001b[1;32m    178\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21macquire\u001b[39m(\u001b[38;5;28mself\u001b[39m, needs_lock\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m):\n\u001b[1;32m    179\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"Acquire a file object from the manager.\u001b[39;00m\n\u001b[1;32m    180\u001b[0m \n\u001b[1;32m    181\u001b[0m \u001b[38;5;124;03m    A new file is only opened if it has expired from the\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    191\u001b[0m \u001b[38;5;124;03m        An open file object, as returned by ``opener(*args, **kwargs)``.\u001b[39;00m\n\u001b[1;32m    192\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m--> 193\u001b[0m     file, _ \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_acquire_with_cache_info\u001b[49m\u001b[43m(\u001b[49m\u001b[43mneeds_lock\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    194\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m file\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/xarray/backends/file_manager.py:217\u001b[0m, in \u001b[0;36mCachingFileManager._acquire_with_cache_info\u001b[0;34m(self, needs_lock)\u001b[0m\n\u001b[1;32m    215\u001b[0m     kwargs \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mcopy()\n\u001b[1;32m    216\u001b[0m     kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmode\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_mode\n\u001b[0;32m--> 217\u001b[0m file \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_opener\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_args\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    218\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mw\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m    219\u001b[0m     \u001b[38;5;66;03m# ensure file doesn't get overridden when opened again\u001b[39;00m\n\u001b[1;32m    220\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_mode \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ma\u001b[39m\u001b[38;5;124m\"\u001b[39m\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rasterio/env.py:463\u001b[0m, in \u001b[0;36mensure_env_with_credentials..wrapper\u001b[0;34m(*args, **kwds)\u001b[0m\n\u001b[1;32m    460\u001b[0m     session \u001b[38;5;241m=\u001b[39m DummySession()\n\u001b[1;32m    462\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m env_ctor(session\u001b[38;5;241m=\u001b[39msession):\n\u001b[0;32m--> 463\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mf\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwds\u001b[49m\u001b[43m)\u001b[49m\n",
-      "File \u001b[0;32m~/Apps/miniconda3/envs/env_greenit_2/lib/python3.11/site-packages/rasterio/__init__.py:355\u001b[0m, in \u001b[0;36mopen\u001b[0;34m(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, opener, **kwargs)\u001b[0m\n\u001b[1;32m    352\u001b[0m     path \u001b[38;5;241m=\u001b[39m _parse_path(raw_dataset_path)\n\u001b[1;32m    354\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[0;32m--> 355\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m \u001b[43mDatasetReader\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpath\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdriver\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdriver\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msharing\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msharing\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    356\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m mode \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mr+\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m    357\u001b[0m     dataset \u001b[38;5;241m=\u001b[39m get_writer_for_path(path, driver\u001b[38;5;241m=\u001b[39mdriver)(\n\u001b[1;32m    358\u001b[0m         path, mode, driver\u001b[38;5;241m=\u001b[39mdriver, sharing\u001b[38;5;241m=\u001b[39msharing, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs\n\u001b[1;32m    359\u001b[0m     )\n",
-      "File \u001b[0;32mrasterio/_base.pyx:312\u001b[0m, in \u001b[0;36mrasterio._base.DatasetBase.__init__\u001b[0;34m()\u001b[0m\n",
-      "\u001b[0;31mRasterioIOError\u001b[0m: /work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif: No such file or directory"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[codecarbon INFO @ 09:30:25] Energy consumed for RAM : 0.000048 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:30:25] Energy consumed for all CPUs : 0.000177 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:30:25] 0.000225 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:30:40] Energy consumed for RAM : 0.000096 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:30:40] Energy consumed for all CPUs : 0.000354 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:30:40] 0.000450 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:30:55] Energy consumed for RAM : 0.000144 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:30:55] Energy consumed for all CPUs : 0.000531 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:30:55] 0.000675 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:31:10] Energy consumed for RAM : 0.000191 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:31:10] Energy consumed for all CPUs : 0.000708 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:31:10] 0.000900 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:31:25] Energy consumed for RAM : 0.000239 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:31:25] Energy consumed for all CPUs : 0.000885 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:31:25] 0.001125 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:31:40] Energy consumed for RAM : 0.000287 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:31:40] Energy consumed for all CPUs : 0.001063 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:31:40] 0.001350 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:31:55] Energy consumed for RAM : 0.000335 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:31:55] Energy consumed for all CPUs : 0.001240 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:31:55] 0.001575 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:32:10] Energy consumed for RAM : 0.000383 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:32:10] Energy consumed for all CPUs : 0.001417 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:32:10] 0.001800 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:32:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241030588549235 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:32:25] Energy consumed for RAM : 0.000431 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:32:25] Energy consumed for all CPUs : 0.001594 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:32:25] 0.002024 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:32:40] Energy consumed for RAM : 0.000479 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:32:40] Energy consumed for all CPUs : 0.001771 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:32:40] 0.002249 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:32:55] Energy consumed for RAM : 0.000526 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:32:55] Energy consumed for all CPUs : 0.001948 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:32:55] 0.002474 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:33:10] Energy consumed for RAM : 0.000574 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:33:10] Energy consumed for all CPUs : 0.002125 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:33:10] 0.002699 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:33:25] Energy consumed for RAM : 0.000622 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:33:25] Energy consumed for all CPUs : 0.002302 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:33:25] 0.002924 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:33:40] Energy consumed for RAM : 0.000670 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:33:40] Energy consumed for all CPUs : 0.002479 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:33:40] 0.003149 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:33:55] Energy consumed for RAM : 0.000718 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:33:55] Energy consumed for all CPUs : 0.002656 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:33:55] 0.003374 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:34:10] Energy consumed for RAM : 0.000766 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:34:10] Energy consumed for all CPUs : 0.002833 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:34:10] 0.003599 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:34:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241081374832695 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:34:25] Energy consumed for RAM : 0.000813 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:34:25] Energy consumed for all CPUs : 0.003010 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:34:25] 0.003824 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:34:40] Energy consumed for RAM : 0.000861 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:34:40] Energy consumed for all CPUs : 0.003187 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:34:40] 0.004049 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:34:55] Energy consumed for RAM : 0.000909 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:34:55] Energy consumed for all CPUs : 0.003365 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:34:55] 0.004274 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:35:10] Energy consumed for RAM : 0.000957 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:35:10] Energy consumed for all CPUs : 0.003542 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:35:10] 0.004499 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:35:25] Energy consumed for RAM : 0.001005 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:35:25] Energy consumed for all CPUs : 0.003719 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:35:25] 0.004724 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:35:40] Energy consumed for RAM : 0.001053 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:35:40] Energy consumed for all CPUs : 0.003896 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:35:40] 0.004949 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:35:55] Energy consumed for RAM : 0.001101 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:35:55] Energy consumed for all CPUs : 0.004073 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:35:55] 0.005173 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:36:10] Energy consumed for RAM : 0.001148 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:36:10] Energy consumed for all CPUs : 0.004250 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:36:10] 0.005398 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:36:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241062424850872 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:36:25] Energy consumed for RAM : 0.001196 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:36:25] Energy consumed for all CPUs : 0.004427 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:36:25] 0.005623 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:36:40] Energy consumed for RAM : 0.001244 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:36:40] Energy consumed for all CPUs : 0.004604 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:36:40] 0.005848 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:36:55] Energy consumed for RAM : 0.001292 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:36:55] Energy consumed for all CPUs : 0.004781 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:36:55] 0.006073 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:37:10] Energy consumed for RAM : 0.001340 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:37:10] Energy consumed for all CPUs : 0.004958 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:37:10] 0.006298 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:37:25] Energy consumed for RAM : 0.001388 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:37:25] Energy consumed for all CPUs : 0.005135 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:37:25] 0.006523 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:37:40] Energy consumed for RAM : 0.001436 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:37:40] Energy consumed for all CPUs : 0.005312 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:37:40] 0.006748 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:37:55] Energy consumed for RAM : 0.001483 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:37:55] Energy consumed for all CPUs : 0.005490 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:37:55] 0.006973 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:38:10] Energy consumed for RAM : 0.001531 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:38:10] Energy consumed for all CPUs : 0.005667 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:38:10] 0.007198 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:38:10] 0.000036 g.CO2eq/s mean an estimation of 1.124116242737765 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:38:25] Energy consumed for RAM : 0.001579 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:38:25] Energy consumed for all CPUs : 0.005844 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:38:25] 0.007423 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:38:40] Energy consumed for RAM : 0.001627 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:38:40] Energy consumed for all CPUs : 0.006021 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:38:40] 0.007648 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:38:55] Energy consumed for RAM : 0.001675 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:38:55] Energy consumed for all CPUs : 0.006198 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:38:55] 0.007873 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:39:10] Energy consumed for RAM : 0.001723 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:39:10] Energy consumed for all CPUs : 0.006375 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:39:10] 0.008098 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:39:25] Energy consumed for RAM : 0.001771 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:39:25] Energy consumed for all CPUs : 0.006552 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:39:25] 0.008323 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:39:40] Energy consumed for RAM : 0.001818 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:39:40] Energy consumed for all CPUs : 0.006729 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:39:40] 0.008547 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:39:55] Energy consumed for RAM : 0.001866 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:39:55] Energy consumed for all CPUs : 0.006906 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:39:55] 0.008772 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:40:10] Energy consumed for RAM : 0.001914 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:40:10] Energy consumed for all CPUs : 0.007083 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:40:10] 0.008997 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:40:10] 0.000036 g.CO2eq/s mean an estimation of 1.1240958745964122 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:40:25] Energy consumed for RAM : 0.001962 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:40:25] Energy consumed for all CPUs : 0.007260 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:40:25] 0.009222 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:40:40] Energy consumed for RAM : 0.002010 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:40:40] Energy consumed for all CPUs : 0.007437 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:40:40] 0.009447 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:40:55] Energy consumed for RAM : 0.002058 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:40:55] Energy consumed for all CPUs : 0.007615 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:40:55] 0.009672 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:41:10] Energy consumed for RAM : 0.002105 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:41:10] Energy consumed for all CPUs : 0.007792 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:41:10] 0.009897 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:41:25] Energy consumed for RAM : 0.002153 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:41:25] Energy consumed for all CPUs : 0.007969 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:41:25] 0.010122 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:41:40] Energy consumed for RAM : 0.002201 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:41:40] Energy consumed for all CPUs : 0.008146 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:41:40] 0.010347 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:41:55] Energy consumed for RAM : 0.002249 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:41:55] Energy consumed for all CPUs : 0.008323 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:41:55] 0.010572 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:42:10] Energy consumed for RAM : 0.002297 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:42:10] Energy consumed for all CPUs : 0.008500 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:42:10] 0.010797 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:42:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241136069648792 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:42:25] Energy consumed for RAM : 0.002345 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:42:25] Energy consumed for all CPUs : 0.008677 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:42:25] 0.011022 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:42:40] Energy consumed for RAM : 0.002393 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:42:40] Energy consumed for all CPUs : 0.008854 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:42:40] 0.011247 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:42:55] Energy consumed for RAM : 0.002440 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:42:55] Energy consumed for all CPUs : 0.009031 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:42:55] 0.011472 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:43:10] Energy consumed for RAM : 0.002488 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:43:10] Energy consumed for all CPUs : 0.009208 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:43:10] 0.011697 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:43:25] Energy consumed for RAM : 0.002536 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:43:25] Energy consumed for all CPUs : 0.009385 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:43:25] 0.011921 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:43:40] Energy consumed for RAM : 0.002584 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:43:40] Energy consumed for all CPUs : 0.009562 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:43:40] 0.012146 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:43:55] Energy consumed for RAM : 0.002632 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:43:55] Energy consumed for all CPUs : 0.009739 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:43:55] 0.012371 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:44:10] Energy consumed for RAM : 0.002680 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:44:10] Energy consumed for all CPUs : 0.009917 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:44:10] 0.012596 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:44:10] 0.000036 g.CO2eq/s mean an estimation of 1.1240987825380455 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:44:25] Energy consumed for RAM : 0.002728 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:44:25] Energy consumed for all CPUs : 0.010094 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:44:25] 0.012821 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:44:40] Energy consumed for RAM : 0.002775 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:44:40] Energy consumed for all CPUs : 0.010271 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:44:40] 0.013046 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:44:55] Energy consumed for RAM : 0.002823 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:44:55] Energy consumed for all CPUs : 0.010448 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:44:55] 0.013271 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:45:10] Energy consumed for RAM : 0.002871 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:45:10] Energy consumed for all CPUs : 0.010625 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:45:10] 0.013496 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:45:25] Energy consumed for RAM : 0.002919 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:45:25] Energy consumed for all CPUs : 0.010802 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:45:25] 0.013721 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:45:40] Energy consumed for RAM : 0.002967 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:45:40] Energy consumed for all CPUs : 0.010979 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:45:40] 0.013946 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:45:55] Energy consumed for RAM : 0.003015 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:45:55] Energy consumed for all CPUs : 0.011156 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:45:55] 0.014171 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:46:10] Energy consumed for RAM : 0.003062 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:46:10] Energy consumed for all CPUs : 0.011333 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:46:10] 0.014396 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:46:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241045609252327 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:46:25] Energy consumed for RAM : 0.003110 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:46:25] Energy consumed for all CPUs : 0.011510 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:46:25] 0.014621 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:46:40] Energy consumed for RAM : 0.003158 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:46:40] Energy consumed for all CPUs : 0.011687 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:46:40] 0.014846 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:46:55] Energy consumed for RAM : 0.003206 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:46:55] Energy consumed for all CPUs : 0.011864 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:46:55] 0.015070 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:47:10] Energy consumed for RAM : 0.003254 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:47:10] Energy consumed for all CPUs : 0.012042 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:47:10] 0.015295 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:47:25] Energy consumed for RAM : 0.003302 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:47:25] Energy consumed for all CPUs : 0.012219 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:47:25] 0.015520 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:47:40] Energy consumed for RAM : 0.003350 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:47:40] Energy consumed for all CPUs : 0.012396 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:47:40] 0.015745 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:47:55] Energy consumed for RAM : 0.003397 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:47:55] Energy consumed for all CPUs : 0.012573 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:47:55] 0.015970 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:48:10] Energy consumed for RAM : 0.003445 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:48:10] Energy consumed for all CPUs : 0.012750 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:48:10] 0.016195 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:48:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241122754246067 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:48:25] Energy consumed for RAM : 0.003493 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:48:25] Energy consumed for all CPUs : 0.012927 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:48:25] 0.016420 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:48:40] Energy consumed for RAM : 0.003541 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:48:40] Energy consumed for all CPUs : 0.013104 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:48:40] 0.016645 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:48:55] Energy consumed for RAM : 0.003589 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:48:55] Energy consumed for all CPUs : 0.013281 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:48:55] 0.016870 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:49:10] Energy consumed for RAM : 0.003637 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:49:10] Energy consumed for all CPUs : 0.013458 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:49:10] 0.017095 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:49:25] Energy consumed for RAM : 0.003685 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:49:25] Energy consumed for all CPUs : 0.013635 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:49:25] 0.017320 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:49:40] Energy consumed for RAM : 0.003732 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:49:40] Energy consumed for all CPUs : 0.013812 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:49:40] 0.017545 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:49:55] Energy consumed for RAM : 0.003780 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:49:55] Energy consumed for all CPUs : 0.013989 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:49:55] 0.017770 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:50:10] Energy consumed for RAM : 0.003828 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:50:10] Energy consumed for all CPUs : 0.014166 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:50:10] 0.017995 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:50:10] 0.000036 g.CO2eq/s mean an estimation of 1.124105407505187 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:50:25] Energy consumed for RAM : 0.003876 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:50:25] Energy consumed for all CPUs : 0.014344 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:50:25] 0.018219 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:50:40] Energy consumed for RAM : 0.003924 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:50:40] Energy consumed for all CPUs : 0.014521 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:50:40] 0.018444 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:50:55] Energy consumed for RAM : 0.003972 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:50:55] Energy consumed for all CPUs : 0.014698 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:50:55] 0.018669 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:51:10] Energy consumed for RAM : 0.004020 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:51:10] Energy consumed for all CPUs : 0.014875 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:51:10] 0.018894 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:51:25] Energy consumed for RAM : 0.004067 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:51:25] Energy consumed for all CPUs : 0.015052 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:51:25] 0.019119 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:51:40] Energy consumed for RAM : 0.004115 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:51:40] Energy consumed for all CPUs : 0.015229 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:51:40] 0.019344 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:51:55] Energy consumed for RAM : 0.004163 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:51:55] Energy consumed for all CPUs : 0.015406 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:51:55] 0.019569 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:52:10] Energy consumed for RAM : 0.004211 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:52:10] Energy consumed for all CPUs : 0.015583 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:52:10] 0.019794 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:52:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241152906286473 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:52:25] Energy consumed for RAM : 0.004259 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:52:25] Energy consumed for all CPUs : 0.015760 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:52:25] 0.020019 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:52:40] Energy consumed for RAM : 0.004307 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:52:40] Energy consumed for all CPUs : 0.015937 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:52:40] 0.020244 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:52:55] Energy consumed for RAM : 0.004354 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:52:55] Energy consumed for all CPUs : 0.016114 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:52:55] 0.020469 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:53:10] Energy consumed for RAM : 0.004402 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:53:10] Energy consumed for all CPUs : 0.016291 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:53:10] 0.020694 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:53:25] Energy consumed for RAM : 0.004450 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:53:25] Energy consumed for all CPUs : 0.016468 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:53:25] 0.020919 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:53:40] Energy consumed for RAM : 0.004498 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:53:40] Energy consumed for all CPUs : 0.016646 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:53:40] 0.021144 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:53:55] Energy consumed for RAM : 0.004546 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:53:55] Energy consumed for all CPUs : 0.016823 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:53:55] 0.021369 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:54:10] Energy consumed for RAM : 0.004594 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:54:10] Energy consumed for all CPUs : 0.017000 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:54:10] 0.021593 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:54:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241096598571043 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:54:25] Energy consumed for RAM : 0.004642 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:54:25] Energy consumed for all CPUs : 0.017177 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:54:25] 0.021818 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:54:40] Energy consumed for RAM : 0.004689 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:54:40] Energy consumed for all CPUs : 0.017354 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:54:40] 0.022043 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:54:55] Energy consumed for RAM : 0.004737 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:54:55] Energy consumed for all CPUs : 0.017531 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:54:55] 0.022268 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:55:10] Energy consumed for RAM : 0.004785 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:55:10] Energy consumed for all CPUs : 0.017708 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:55:10] 0.022493 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:55:25] Energy consumed for RAM : 0.004833 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:55:25] Energy consumed for all CPUs : 0.017885 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:55:25] 0.022718 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:55:40] Energy consumed for RAM : 0.004881 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:55:40] Energy consumed for all CPUs : 0.018062 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:55:40] 0.022943 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:55:55] Energy consumed for RAM : 0.004929 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:55:55] Energy consumed for all CPUs : 0.018239 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:55:55] 0.023168 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:56:10] Energy consumed for RAM : 0.004977 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:56:10] Energy consumed for all CPUs : 0.018416 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:56:10] 0.023393 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:56:10] 0.000036 g.CO2eq/s mean an estimation of 1.1240968553304007 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:56:25] Energy consumed for RAM : 0.005024 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:56:25] Energy consumed for all CPUs : 0.018593 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:56:25] 0.023618 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:56:40] Energy consumed for RAM : 0.005072 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:56:40] Energy consumed for all CPUs : 0.018770 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:56:40] 0.023843 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:56:55] Energy consumed for RAM : 0.005120 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:56:55] Energy consumed for all CPUs : 0.018948 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:56:55] 0.024068 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:57:10] Energy consumed for RAM : 0.005168 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:57:10] Energy consumed for all CPUs : 0.019125 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:57:10] 0.024293 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:57:25] Energy consumed for RAM : 0.005216 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:57:25] Energy consumed for all CPUs : 0.019302 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:57:25] 0.024517 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:57:40] Energy consumed for RAM : 0.005264 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:57:40] Energy consumed for all CPUs : 0.019479 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:57:40] 0.024742 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:57:55] Energy consumed for RAM : 0.005311 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:57:55] Energy consumed for all CPUs : 0.019656 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:57:55] 0.024967 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:58:10] Energy consumed for RAM : 0.005359 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:58:10] Energy consumed for all CPUs : 0.019833 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:58:10] 0.025192 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:58:10] 0.000036 g.CO2eq/s mean an estimation of 1.124113741107103 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 09:58:25] Energy consumed for RAM : 0.005407 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:58:25] Energy consumed for all CPUs : 0.020010 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:58:25] 0.025417 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:58:40] Energy consumed for RAM : 0.005455 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:58:40] Energy consumed for all CPUs : 0.020187 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:58:40] 0.025642 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:58:55] Energy consumed for RAM : 0.005503 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:58:55] Energy consumed for all CPUs : 0.020364 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:58:55] 0.025867 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:59:10] Energy consumed for RAM : 0.005551 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:59:10] Energy consumed for all CPUs : 0.020541 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:59:10] 0.026092 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:59:25] Energy consumed for RAM : 0.005599 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:59:25] Energy consumed for all CPUs : 0.020718 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:59:25] 0.026317 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:59:40] Energy consumed for RAM : 0.005646 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:59:40] Energy consumed for all CPUs : 0.020895 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:59:40] 0.026542 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 09:59:55] Energy consumed for RAM : 0.005694 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 09:59:55] Energy consumed for all CPUs : 0.021073 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 09:59:55] 0.026767 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:00:10] Energy consumed for RAM : 0.005742 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:00:10] Energy consumed for all CPUs : 0.021250 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:00:10] 0.026992 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:00:10] 0.000036 g.CO2eq/s mean an estimation of 1.124111541492001 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:00:25] Energy consumed for RAM : 0.005790 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:00:25] Energy consumed for all CPUs : 0.021427 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:00:25] 0.027217 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:00:40] Energy consumed for RAM : 0.005838 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:00:40] Energy consumed for all CPUs : 0.021604 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:00:40] 0.027442 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:00:55] Energy consumed for RAM : 0.005886 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:00:55] Energy consumed for all CPUs : 0.021781 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:00:55] 0.027667 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:01:10] Energy consumed for RAM : 0.005934 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:01:10] Energy consumed for all CPUs : 0.021958 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:01:10] 0.027891 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:01:25] Energy consumed for RAM : 0.005981 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:01:25] Energy consumed for all CPUs : 0.022135 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:01:25] 0.028116 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:01:40] Energy consumed for RAM : 0.006029 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:01:40] Energy consumed for all CPUs : 0.022312 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:01:40] 0.028341 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:01:55] Energy consumed for RAM : 0.006077 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:01:55] Energy consumed for all CPUs : 0.022489 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:01:55] 0.028566 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:02:10] Energy consumed for RAM : 0.006125 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:02:10] Energy consumed for all CPUs : 0.022666 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:02:10] 0.028791 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:02:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241034094395093 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:02:25] Energy consumed for RAM : 0.006173 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:02:25] Energy consumed for all CPUs : 0.022843 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:02:25] 0.029016 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:02:40] Energy consumed for RAM : 0.006221 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:02:40] Energy consumed for all CPUs : 0.023020 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:02:40] 0.029241 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:02:55] Energy consumed for RAM : 0.006268 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:02:55] Energy consumed for all CPUs : 0.023197 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:02:55] 0.029466 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:03:10] Energy consumed for RAM : 0.006316 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:03:10] Energy consumed for all CPUs : 0.023375 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:03:10] 0.029691 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:03:25] Energy consumed for RAM : 0.006364 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:03:25] Energy consumed for all CPUs : 0.023552 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:03:25] 0.029916 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:03:40] Energy consumed for RAM : 0.006412 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:03:40] Energy consumed for all CPUs : 0.023729 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:03:40] 0.030141 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:03:55] Energy consumed for RAM : 0.006460 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:03:55] Energy consumed for all CPUs : 0.023906 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:03:55] 0.030366 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:04:10] Energy consumed for RAM : 0.006508 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:04:10] Energy consumed for all CPUs : 0.024083 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:04:10] 0.030591 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:04:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241195224753218 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:04:25] Energy consumed for RAM : 0.006556 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:04:25] Energy consumed for all CPUs : 0.024260 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:04:25] 0.030816 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:04:40] Energy consumed for RAM : 0.006603 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:04:40] Energy consumed for all CPUs : 0.024437 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:04:40] 0.031041 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:04:55] Energy consumed for RAM : 0.006651 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:04:55] Energy consumed for all CPUs : 0.024614 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:04:55] 0.031265 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:05:10] Energy consumed for RAM : 0.006699 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:05:10] Energy consumed for all CPUs : 0.024791 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:05:10] 0.031490 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:05:25] Energy consumed for RAM : 0.006747 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:05:25] Energy consumed for all CPUs : 0.024968 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:05:25] 0.031715 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:05:40] Energy consumed for RAM : 0.006795 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:05:40] Energy consumed for all CPUs : 0.025145 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:05:40] 0.031940 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:05:55] Energy consumed for RAM : 0.006843 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:05:55] Energy consumed for all CPUs : 0.025322 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:05:55] 0.032165 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:06:10] Energy consumed for RAM : 0.006891 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:06:10] Energy consumed for all CPUs : 0.025500 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:06:10] 0.032390 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:06:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241065632787401 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:06:25] Energy consumed for RAM : 0.006938 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:06:25] Energy consumed for all CPUs : 0.025677 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:06:25] 0.032615 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:06:40] Energy consumed for RAM : 0.006986 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:06:40] Energy consumed for all CPUs : 0.025854 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:06:40] 0.032840 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:06:55] Energy consumed for RAM : 0.007034 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:06:55] Energy consumed for all CPUs : 0.026031 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:06:55] 0.033065 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:07:10] Energy consumed for RAM : 0.007082 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:07:10] Energy consumed for all CPUs : 0.026208 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:07:10] 0.033290 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:07:25] Energy consumed for RAM : 0.007130 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:07:25] Energy consumed for all CPUs : 0.026385 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:07:25] 0.033515 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:07:40] Energy consumed for RAM : 0.007178 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:07:40] Energy consumed for all CPUs : 0.026562 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:07:40] 0.033740 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:07:55] Energy consumed for RAM : 0.007226 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:07:55] Energy consumed for all CPUs : 0.026739 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:07:55] 0.033965 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:08:10] Energy consumed for RAM : 0.007273 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:08:10] Energy consumed for all CPUs : 0.026916 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:08:10] 0.034190 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:08:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241020064479441 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:08:25] Energy consumed for RAM : 0.007321 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:08:25] Energy consumed for all CPUs : 0.027093 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:08:25] 0.034414 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:08:40] Energy consumed for RAM : 0.007369 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:08:40] Energy consumed for all CPUs : 0.027270 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:08:40] 0.034639 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:08:55] Energy consumed for RAM : 0.007417 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:08:55] Energy consumed for all CPUs : 0.027447 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:08:55] 0.034864 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:09:10] Energy consumed for RAM : 0.007465 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:09:10] Energy consumed for all CPUs : 0.027624 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:09:10] 0.035089 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:09:25] Energy consumed for RAM : 0.007513 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:09:25] Energy consumed for all CPUs : 0.027802 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:09:25] 0.035314 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:09:40] Energy consumed for RAM : 0.007560 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:09:40] Energy consumed for all CPUs : 0.027979 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:09:40] 0.035539 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:09:55] Energy consumed for RAM : 0.007608 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:09:55] Energy consumed for all CPUs : 0.028156 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:09:55] 0.035764 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:10:10] Energy consumed for RAM : 0.007656 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:10:10] Energy consumed for all CPUs : 0.028333 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:10:10] 0.035989 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:10:10] 0.000036 g.CO2eq/s mean an estimation of 1.12411772689883 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:10:25] Energy consumed for RAM : 0.007704 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:10:25] Energy consumed for all CPUs : 0.028510 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:10:25] 0.036214 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:10:40] Energy consumed for RAM : 0.007752 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:10:40] Energy consumed for all CPUs : 0.028687 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:10:40] 0.036439 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:10:55] Energy consumed for RAM : 0.007800 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:10:55] Energy consumed for all CPUs : 0.028864 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:10:55] 0.036664 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:11:10] Energy consumed for RAM : 0.007848 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:11:10] Energy consumed for all CPUs : 0.029041 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:11:10] 0.036889 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:11:25] Energy consumed for RAM : 0.007895 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:11:25] Energy consumed for all CPUs : 0.029218 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:11:25] 0.037114 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:11:40] Energy consumed for RAM : 0.007943 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:11:40] Energy consumed for all CPUs : 0.029395 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:11:40] 0.037339 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:11:55] Energy consumed for RAM : 0.007991 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:11:55] Energy consumed for all CPUs : 0.029572 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:11:55] 0.037563 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:12:10] Energy consumed for RAM : 0.008039 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:12:10] Energy consumed for all CPUs : 0.029749 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:12:10] 0.037788 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:12:10] 0.000036 g.CO2eq/s mean an estimation of 1.124104770998259 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:12:25] Energy consumed for RAM : 0.008087 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:12:25] Energy consumed for all CPUs : 0.029927 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:12:25] 0.038013 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:12:40] Energy consumed for RAM : 0.008135 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:12:40] Energy consumed for all CPUs : 0.030104 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:12:40] 0.038238 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:12:55] Energy consumed for RAM : 0.008183 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:12:55] Energy consumed for all CPUs : 0.030281 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:12:55] 0.038463 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:13:10] Energy consumed for RAM : 0.008230 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:13:10] Energy consumed for all CPUs : 0.030458 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:13:10] 0.038688 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:13:25] Energy consumed for RAM : 0.008278 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:13:25] Energy consumed for all CPUs : 0.030635 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:13:25] 0.038913 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:13:40] Energy consumed for RAM : 0.008326 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:13:40] Energy consumed for all CPUs : 0.030812 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:13:40] 0.039138 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:13:55] Energy consumed for RAM : 0.008374 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:13:55] Energy consumed for all CPUs : 0.030989 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:13:55] 0.039363 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:14:10] Energy consumed for RAM : 0.008422 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:14:10] Energy consumed for all CPUs : 0.031166 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:14:10] 0.039588 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:14:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241176300291527 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:14:25] Energy consumed for RAM : 0.008470 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:14:25] Energy consumed for all CPUs : 0.031343 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:14:25] 0.039813 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:14:40] Energy consumed for RAM : 0.008517 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:14:40] Energy consumed for all CPUs : 0.031520 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:14:40] 0.040038 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:14:55] Energy consumed for RAM : 0.008565 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:14:55] Energy consumed for all CPUs : 0.031697 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:14:55] 0.040263 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:15:10] Energy consumed for RAM : 0.008613 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:15:10] Energy consumed for all CPUs : 0.031874 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:15:10] 0.040488 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:15:25] Energy consumed for RAM : 0.008661 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:15:25] Energy consumed for all CPUs : 0.032051 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:15:25] 0.040712 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:15:40] Energy consumed for RAM : 0.008709 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:15:40] Energy consumed for all CPUs : 0.032229 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:15:40] 0.040937 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:15:55] Energy consumed for RAM : 0.008757 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:15:55] Energy consumed for all CPUs : 0.032406 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:15:55] 0.041162 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:16:10] Energy consumed for RAM : 0.008805 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:16:10] Energy consumed for all CPUs : 0.032583 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:16:10] 0.041387 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:16:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241178147419713 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:16:25] Energy consumed for RAM : 0.008852 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:16:25] Energy consumed for all CPUs : 0.032760 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:16:25] 0.041612 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:16:40] Energy consumed for RAM : 0.008900 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:16:40] Energy consumed for all CPUs : 0.032937 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:16:40] 0.041837 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:16:55] Energy consumed for RAM : 0.008948 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:16:55] Energy consumed for all CPUs : 0.033114 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:16:55] 0.042062 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:17:10] Energy consumed for RAM : 0.008996 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:17:10] Energy consumed for all CPUs : 0.033291 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:17:10] 0.042287 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:17:25] Energy consumed for RAM : 0.009044 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:17:25] Energy consumed for all CPUs : 0.033468 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:17:25] 0.042512 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:17:40] Energy consumed for RAM : 0.009092 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:17:40] Energy consumed for all CPUs : 0.033645 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:17:40] 0.042737 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:17:55] Energy consumed for RAM : 0.009140 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:17:55] Energy consumed for all CPUs : 0.033822 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:17:55] 0.042962 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:18:10] Energy consumed for RAM : 0.009187 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:18:10] Energy consumed for all CPUs : 0.033999 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:18:10] 0.043187 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:18:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241141077367505 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:18:25] Energy consumed for RAM : 0.009235 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:18:25] Energy consumed for all CPUs : 0.034176 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:18:25] 0.043412 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:18:40] Energy consumed for RAM : 0.009283 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:18:40] Energy consumed for all CPUs : 0.034353 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:18:40] 0.043637 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:18:55] Energy consumed for RAM : 0.009331 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:18:55] Energy consumed for all CPUs : 0.034531 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:18:55] 0.043861 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:19:10] Energy consumed for RAM : 0.009379 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:19:10] Energy consumed for all CPUs : 0.034708 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:19:10] 0.044086 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:19:25] Energy consumed for RAM : 0.009427 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:19:25] Energy consumed for all CPUs : 0.034885 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:19:25] 0.044311 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:19:40] Energy consumed for RAM : 0.009475 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:19:40] Energy consumed for all CPUs : 0.035062 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:19:40] 0.044536 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:19:55] Energy consumed for RAM : 0.009522 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:19:55] Energy consumed for all CPUs : 0.035239 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:19:55] 0.044761 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:20:10] Energy consumed for RAM : 0.009570 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:20:10] Energy consumed for all CPUs : 0.035416 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:20:10] 0.044986 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:20:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241206406423676 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:20:25] Energy consumed for RAM : 0.009618 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:20:25] Energy consumed for all CPUs : 0.035593 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:20:25] 0.045211 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:20:40] Energy consumed for RAM : 0.009666 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:20:40] Energy consumed for all CPUs : 0.035770 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:20:40] 0.045436 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:20:55] Energy consumed for RAM : 0.009714 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:20:55] Energy consumed for all CPUs : 0.035947 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:20:55] 0.045661 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:21:10] Energy consumed for RAM : 0.009762 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:21:10] Energy consumed for all CPUs : 0.036124 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:21:10] 0.045886 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:21:25] Energy consumed for RAM : 0.009809 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:21:25] Energy consumed for all CPUs : 0.036301 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:21:25] 0.046111 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:21:40] Energy consumed for RAM : 0.009857 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:21:40] Energy consumed for all CPUs : 0.036478 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:21:40] 0.046336 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:21:55] Energy consumed for RAM : 0.009905 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:21:55] Energy consumed for all CPUs : 0.036655 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:21:55] 0.046561 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:22:10] Energy consumed for RAM : 0.009953 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:22:10] Energy consumed for all CPUs : 0.036833 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:22:10] 0.046786 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:22:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241260591903481 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:22:25] Energy consumed for RAM : 0.010001 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:22:25] Energy consumed for all CPUs : 0.037010 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:22:25] 0.047011 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:22:40] Energy consumed for RAM : 0.010049 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:22:40] Energy consumed for all CPUs : 0.037187 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:22:40] 0.047235 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:22:55] Energy consumed for RAM : 0.010097 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:22:55] Energy consumed for all CPUs : 0.037364 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:22:55] 0.047460 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:23:10] Energy consumed for RAM : 0.010144 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:23:10] Energy consumed for all CPUs : 0.037541 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:23:10] 0.047685 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:23:25] Energy consumed for RAM : 0.010192 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:23:25] Energy consumed for all CPUs : 0.037718 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:23:25] 0.047910 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:23:40] Energy consumed for RAM : 0.010240 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:23:40] Energy consumed for all CPUs : 0.037895 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:23:40] 0.048135 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:23:55] Energy consumed for RAM : 0.010288 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:23:55] Energy consumed for all CPUs : 0.038072 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:23:55] 0.048360 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:24:10] Energy consumed for RAM : 0.010336 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:24:10] Energy consumed for all CPUs : 0.038249 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:24:10] 0.048585 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:24:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241262053471515 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:24:25] Energy consumed for RAM : 0.010384 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:24:25] Energy consumed for all CPUs : 0.038426 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:24:25] 0.048810 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:24:40] Energy consumed for RAM : 0.010432 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:24:40] Energy consumed for all CPUs : 0.038603 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:24:40] 0.049035 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:24:55] Energy consumed for RAM : 0.010479 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:24:55] Energy consumed for all CPUs : 0.038780 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:24:55] 0.049260 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:25:10] Energy consumed for RAM : 0.010527 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:25:10] Energy consumed for all CPUs : 0.038957 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:25:10] 0.049485 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:25:25] Energy consumed for RAM : 0.010575 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:25:25] Energy consumed for all CPUs : 0.039135 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:25:25] 0.049710 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:25:40] Energy consumed for RAM : 0.010623 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:25:40] Energy consumed for all CPUs : 0.039312 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:25:40] 0.049935 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:25:55] Energy consumed for RAM : 0.010671 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:25:55] Energy consumed for all CPUs : 0.039489 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:25:55] 0.050160 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:26:10] Energy consumed for RAM : 0.010719 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:26:10] Energy consumed for all CPUs : 0.039666 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:26:10] 0.050384 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:26:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241243278754078 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:26:25] Energy consumed for RAM : 0.010767 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:26:25] Energy consumed for all CPUs : 0.039843 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:26:25] 0.050609 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:26:40] Energy consumed for RAM : 0.010814 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:26:40] Energy consumed for all CPUs : 0.040020 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:26:40] 0.050834 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:26:55] Energy consumed for RAM : 0.010862 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:26:55] Energy consumed for all CPUs : 0.040197 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:26:55] 0.051059 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:27:10] Energy consumed for RAM : 0.010910 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:27:10] Energy consumed for all CPUs : 0.040374 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:27:10] 0.051284 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:27:25] Energy consumed for RAM : 0.010958 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:27:25] Energy consumed for all CPUs : 0.040551 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:27:25] 0.051509 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:27:40] Energy consumed for RAM : 0.011006 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:27:40] Energy consumed for all CPUs : 0.040728 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:27:40] 0.051734 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:27:55] Energy consumed for RAM : 0.011054 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:27:55] Energy consumed for all CPUs : 0.040905 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:27:55] 0.051959 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:28:10] Energy consumed for RAM : 0.011101 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:28:10] Energy consumed for all CPUs : 0.041082 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:28:10] 0.052184 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:28:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241275665026853 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:28:25] Energy consumed for RAM : 0.011149 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:28:25] Energy consumed for all CPUs : 0.041260 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:28:25] 0.052409 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:28:40] Energy consumed for RAM : 0.011197 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:28:40] Energy consumed for all CPUs : 0.041437 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:28:40] 0.052634 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:28:55] Energy consumed for RAM : 0.011245 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:28:55] Energy consumed for all CPUs : 0.041614 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:28:55] 0.052859 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:29:10] Energy consumed for RAM : 0.011293 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:29:10] Energy consumed for all CPUs : 0.041791 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:29:10] 0.053084 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:29:25] Energy consumed for RAM : 0.011341 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:29:25] Energy consumed for all CPUs : 0.041968 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:29:25] 0.053309 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:29:40] Energy consumed for RAM : 0.011389 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:29:40] Energy consumed for all CPUs : 0.042145 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:29:40] 0.053534 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:29:55] Energy consumed for RAM : 0.011436 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:29:55] Energy consumed for all CPUs : 0.042322 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:29:55] 0.053758 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:30:10] Energy consumed for RAM : 0.011484 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:30:10] Energy consumed for all CPUs : 0.042499 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:30:10] 0.053983 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:30:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241325993989661 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:30:25] Energy consumed for RAM : 0.011532 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:30:25] Energy consumed for all CPUs : 0.042676 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:30:25] 0.054208 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:30:40] Energy consumed for RAM : 0.011580 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:30:40] Energy consumed for all CPUs : 0.042853 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:30:40] 0.054433 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:30:55] Energy consumed for RAM : 0.011628 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:30:55] Energy consumed for all CPUs : 0.043030 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:30:55] 0.054658 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:31:10] Energy consumed for RAM : 0.011676 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:31:10] Energy consumed for all CPUs : 0.043207 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:31:10] 0.054883 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:31:25] Energy consumed for RAM : 0.011724 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:31:25] Energy consumed for all CPUs : 0.043385 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:31:25] 0.055108 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:31:40] Energy consumed for RAM : 0.011771 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:31:40] Energy consumed for all CPUs : 0.043562 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:31:40] 0.055333 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:31:55] Energy consumed for RAM : 0.011819 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:31:55] Energy consumed for all CPUs : 0.043739 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:31:55] 0.055558 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:32:10] Energy consumed for RAM : 0.011867 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:32:10] Energy consumed for all CPUs : 0.043916 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:32:10] 0.055783 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:32:10] 0.000036 g.CO2eq/s mean an estimation of 1.124129726197665 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:32:25] Energy consumed for RAM : 0.011915 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:32:25] Energy consumed for all CPUs : 0.044093 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:32:25] 0.056008 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:32:40] Energy consumed for RAM : 0.011963 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:32:40] Energy consumed for all CPUs : 0.044270 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:32:40] 0.056233 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:32:55] Energy consumed for RAM : 0.012011 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:32:55] Energy consumed for all CPUs : 0.044447 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:32:55] 0.056458 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:33:10] Energy consumed for RAM : 0.012059 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:33:10] Energy consumed for all CPUs : 0.044624 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:33:10] 0.056683 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:33:25] Energy consumed for RAM : 0.012106 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:33:25] Energy consumed for all CPUs : 0.044801 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:33:25] 0.056908 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:33:40] Energy consumed for RAM : 0.012154 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:33:40] Energy consumed for all CPUs : 0.044978 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:33:40] 0.057132 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:33:55] Energy consumed for RAM : 0.012202 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:33:55] Energy consumed for all CPUs : 0.045155 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:33:55] 0.057357 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:34:10] Energy consumed for RAM : 0.012250 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:34:10] Energy consumed for all CPUs : 0.045332 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:34:10] 0.057582 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:34:10] 0.000036 g.CO2eq/s mean an estimation of 1.124126066416576 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:34:25] Energy consumed for RAM : 0.012298 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:34:25] Energy consumed for all CPUs : 0.045510 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:34:25] 0.057807 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:34:40] Energy consumed for RAM : 0.012346 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:34:40] Energy consumed for all CPUs : 0.045687 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:34:40] 0.058032 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:34:55] Energy consumed for RAM : 0.012393 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:34:55] Energy consumed for all CPUs : 0.045864 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:34:55] 0.058257 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:35:10] Energy consumed for RAM : 0.012441 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:35:10] Energy consumed for all CPUs : 0.046041 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:35:10] 0.058482 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:35:25] Energy consumed for RAM : 0.012489 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:35:25] Energy consumed for all CPUs : 0.046218 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:35:25] 0.058707 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:35:40] Energy consumed for RAM : 0.012537 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:35:40] Energy consumed for all CPUs : 0.046395 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:35:40] 0.058932 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:35:55] Energy consumed for RAM : 0.012585 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:35:55] Energy consumed for all CPUs : 0.046572 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:35:55] 0.059157 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:36:10] Energy consumed for RAM : 0.012633 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:36:10] Energy consumed for all CPUs : 0.046749 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:36:10] 0.059382 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:36:10] 0.000036 g.CO2eq/s mean an estimation of 1.1241356727556815 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:36:25] Energy consumed for RAM : 0.012681 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:36:25] Energy consumed for all CPUs : 0.046926 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:36:25] 0.059607 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:36:40] Energy consumed for RAM : 0.012728 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:36:40] Energy consumed for all CPUs : 0.047103 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:36:40] 0.059832 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:36:55] Energy consumed for RAM : 0.012776 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:36:55] Energy consumed for all CPUs : 0.047280 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:36:55] 0.060057 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:37:10] Energy consumed for RAM : 0.012824 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:37:10] Energy consumed for all CPUs : 0.047457 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:37:10] 0.060282 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:37:25] Energy consumed for RAM : 0.012872 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:37:25] Energy consumed for all CPUs : 0.047635 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:37:25] 0.060507 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:37:40] Energy consumed for RAM : 0.012920 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:37:40] Energy consumed for all CPUs : 0.047812 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:37:40] 0.060731 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:37:55] Energy consumed for RAM : 0.012968 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:37:55] Energy consumed for all CPUs : 0.047989 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:37:55] 0.060956 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:38:10] Energy consumed for RAM : 0.013016 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:38:10] Energy consumed for all CPUs : 0.048166 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:38:10] 0.061181 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:38:10] 0.000036 g.CO2eq/s mean an estimation of 1.124127331411236 kg.CO2eq/year\n",
-      "[codecarbon INFO @ 10:38:25] Energy consumed for RAM : 0.013063 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:38:25] Energy consumed for all CPUs : 0.048343 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:38:25] 0.061406 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:38:40] Energy consumed for RAM : 0.013111 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:38:40] Energy consumed for all CPUs : 0.048520 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:38:40] 0.061631 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:38:55] Energy consumed for RAM : 0.013159 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:38:55] Energy consumed for all CPUs : 0.048697 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:38:55] 0.061856 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:39:10] Energy consumed for RAM : 0.013207 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:39:10] Energy consumed for all CPUs : 0.048874 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:39:10] 0.062081 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:39:25] Energy consumed for RAM : 0.013255 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:39:25] Energy consumed for all CPUs : 0.049051 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:39:25] 0.062306 kWh of electricity used since the beginning.\n",
-      "[codecarbon INFO @ 10:39:40] Energy consumed for RAM : 0.013303 kWh. RAM Power : 11.486040115356445 W\n",
-      "[codecarbon INFO @ 10:39:40] Energy consumed for all CPUs : 0.049228 kWh. Total CPU Power : 42.5 W\n",
-      "[codecarbon INFO @ 10:39:40] 0.062531 kWh of electricity used since the beginning.\n"
+      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[9], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mcodecarbon\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m track_emissions, EmissionsTracker\n\u001b[1;32m      3\u001b[0m tracker \u001b[38;5;241m=\u001b[39m EmissionsTracker()\n\u001b[1;32m      5\u001b[0m \u001b[38;5;66;03m#@track_emissions()\u001b[39;00m\n",
+      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'codecarbon'"
      ]
     }
    ],
@@ -4345,12 +3414,12 @@
     "start = time.perf_counter()\n",
     "compute_ndvi_sentinel2()\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed to compute sentinel 2 NDVI = {}s\".format((end - start)))\n",
+    "print(\"Time to compute to compute sentinel 2 NDVI = {}s\".format((end - start)))\n",
     "\n",
     "start = time.perf_counter()\n",
     "compute_ndvi_phr()\n",
     "end = time.perf_counter()\n",
-    "print(\"Elapsed to compute PHR Ventoux NDVI = {}s\".format((end - start)))\n",
+    "print(\"Time to compute to compute PHR Ventoux NDVI = {}s\".format((end - start)))\n",
     "tracker.stop()"
    ]
   }

From fc011f50432d6aa608ca99a8af155c706efec833 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 17 Feb 2025 10:07:19 +0000
Subject: [PATCH 30/37] Add necessary precisions on some conclusions

---
 tuto_greenit.ipynb | 166 ++++++++++++++++++++++-----------------------
 1 file changed, 83 insertions(+), 83 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index e3e629a..d55735b 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -205,7 +205,7 @@
    "id": "1898ca12-fdcc-48b9-b733-642f2627bbe9",
    "metadata": {},
    "source": [
-    "# Data format libraries for spatial image processing"
+    "# Choose a data format library adapted to your processing chain"
    ]
   },
   {
@@ -305,7 +305,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 4,
    "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62",
    "metadata": {
     "tags": []
@@ -370,28 +370,18 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 3,
    "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21",
    "metadata": {
     "scrolled": true,
     "tags": []
    },
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n",
-      "Perhaps you already have a cluster running?\n",
-      "Hosting the HTTP server on port 45641 instead\n",
-      "  warnings.warn(\n"
-     ]
-    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Dask Dashboard:  http://127.0.0.1:45641/status\n"
+      "Dask Dashboard:  http://127.0.0.1:8787/status\n"
      ]
     },
     {
@@ -401,7 +391,7 @@
        "    
     
    \n",
        "    
    \n",
        "        
    Client
    \n",
-       "        
    Client-9ef29c9c-eabc-11ef-80bd-84160c1e7634
    \n",
+       "        
    Client-20e36361-ed10-11ef-8e57-00620ba47efa
    \n",
        "        \n",
        "\n",
        "        \n",
@@ -414,7 +404,7 @@
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
@@ -423,7 +413,7 @@
        "        
    \n",
-       "                    Dashboard:  http://127.0.0.1:45641/status\n",
+       "                    Dashboard:  http://127.0.0.1:8787/status\n",
        "                
    \n",
        "\n",
        "        \n",
-       "            \n",
        "        \n",
@@ -436,11 +426,11 @@
        "    
    \n",
        "    
    \n",
        "        
    LocalCluster
    \n",
-       "        
    ae49023c
    \n",
+       "        
    a6db3b0f
    \n",
        "        \n",
        "            \n",
        "                \n",
        "                
    \n",
-       "                    Dashboard: http://127.0.0.1:45641/status\n",
+       "                    Dashboard: http://127.0.0.1:8787/status\n",
        "                \n",
        "                    Workers: 4\n",
@@ -473,11 +463,11 @@
        "        
     
    \n",
        "        
    \n",
        "            
    Scheduler
    \n",
-       "            
    Scheduler-b3c6abd1-f6ce-4fd6-91aa-5302a9068ade
    \n",
+       "            
    Scheduler-e08b1170-037d-4526-80fa-db62e17f1914
    \n",
        "            \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -717,29 +706,29 @@
        "                
    \n",
-       "                        Comm: tcp://127.0.0.1:33991\n",
+       "                        Comm: tcp://127.0.0.1:43565\n",
        "                    \n",
        "                        Workers: 4\n",
@@ -485,7 +475,7 @@
        "                
    \n",
-       "                        Dashboard: http://127.0.0.1:45641/status\n",
+       "                        Dashboard: http://127.0.0.1:8787/status\n",
        "                    \n",
        "                        Total threads: 8\n",
@@ -519,7 +509,7 @@
        "                \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -564,7 +554,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:46165\n",
+       "                            Comm:  tcp://127.0.0.1:32797\n",
        "                        \n",
        "                            Total threads:  2\n",
@@ -527,7 +517,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:37937/status\n",
+       "                            Dashboard:  http://127.0.0.1:39065/status\n",
        "                        \n",
        "                            Memory:  14.00 GiB\n",
@@ -535,13 +525,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:43337\n",
+       "                            Nanny:  tcp://127.0.0.1:45629\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32831596/dask-scratch-space/worker-2or14pgx\n",
+       "                            Local directory:  /tmp/slurm-32971832/dask-scratch-space/worker-xym2zrdk\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -609,7 +599,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:41899\n",
+       "                            Comm:  tcp://127.0.0.1:43229\n",
        "                        \n",
        "                            Total threads:  2\n",
@@ -572,7 +562,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:33081/status\n",
+       "                            Dashboard:  http://127.0.0.1:40875/status\n",
        "                        \n",
        "                            Memory:  14.00 GiB\n",
@@ -580,13 +570,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:44565\n",
+       "                            Nanny:  tcp://127.0.0.1:42161\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32831596/dask-scratch-space/worker-p4xpbz51\n",
+       "                            Local directory:  /tmp/slurm-32971832/dask-scratch-space/worker-6h1c1fcn\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -654,7 +644,7 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:44637\n",
+       "                            Comm:  tcp://127.0.0.1:45785\n",
        "                        \n",
        "                            Total threads:  2\n",
@@ -617,7 +607,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:39187/status\n",
+       "                            Dashboard:  http://127.0.0.1:44499/status\n",
        "                        \n",
        "                            Memory:  14.00 GiB\n",
@@ -625,13 +615,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:38469\n",
+       "                            Nanny:  tcp://127.0.0.1:41597\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32831596/dask-scratch-space/worker-79_jb3ot\n",
+       "                            Local directory:  /tmp/slurm-32971832/dask-scratch-space/worker-iit2xsch\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -703,10 +693,10 @@
        ""
       ],
       "text/plain": [
-       ""
+       ""
       ]
      },
-     "execution_count": 4,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -758,7 +748,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 5,
    "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f",
    "metadata": {
     "tags": []
@@ -855,7 +845,7 @@
        "dask.array"
       ]
      },
-     "execution_count": 14,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -879,8 +869,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "CPU times: user 387 ms, sys: 1.65 s, total: 2.04 s\n",
-      "Wall time: 4 s\n"
+      "CPU times: user 345 ms, sys: 1.49 s, total: 1.84 s\n",
+      "Wall time: 3.26 s\n"
      ]
     }
    ],
@@ -923,7 +913,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif...: 100% [**************************************************] (3s)\n"
+      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_py.tif...: 100% [**************************************************] (2s)\n"
      ]
     },
     {
@@ -981,7 +971,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (2s)\n"
+      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (3s)\n"
      ]
     }
    ],
@@ -1015,7 +1005,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 9,
    "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
    "metadata": {
     "scrolled": true,
@@ -1026,12 +1016,12 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Time to compute with Raster IO + numba = 1.9315631836652756s\n",
-      "Time to compute with Raster IO + without numba = 2.2189621636644006s\n",
-      "Time to compute with Xarray + apply ufunc = 3.0127522442489862s\n",
-      "Time to compute with RIOXarray + dask = 5.733268056996167s\n",
-      "CPU times: user 2.72 s, sys: 6.51 s, total: 9.22 s\n",
-      "Wall time: 12.9 s\n"
+      "Time to compute with Raster IO + numba = 2.2769373627379537s\n",
+      "Time to compute with Raster IO + without numba = 1.8911610832437873s\n",
+      "Time to compute with Xarray + apply ufunc = 3.2426639813929796s\n",
+      "Time to compute with RIOXarray + dask = 5.112378729507327s\n",
+      "CPU times: user 2.59 s, sys: 5.61 s, total: 8.2 s\n",
+      "Wall time: 12.5 s\n"
      ]
     }
    ],
@@ -1114,7 +1104,7 @@
     "\n",
     "* Using **RIOXarray + dask on single band product is counter productive**.\n",
     "* Xarray + applyufunc also leads to low performance compared to opening the whole product with rasterio.\n",
-    "* Using numba do enhances the performance but with less than 10%, it is more suitable for numpy vectorize [optimisation]( https://numba.readthedocs.io/en/stable/user/vectorize.html#the-vectorize-decorator)\n"
+    "* Using numba do enhances the performance but after the first compilation, if you start it once it will take more time that the computing without numba. It is more suitable for numpy vectorize [optimisation]( https://numba.readthedocs.io/en/stable/user/vectorize.html#the-vectorize-decorator)\n"
    ]
   },
   {
@@ -1419,7 +1409,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 10,
    "id": "57ae950e-2ea3-48a9-87ba-3b8a91de7c93",
    "metadata": {
     "tags": []
@@ -1808,7 +1798,7 @@
        "Attributes:\n",
        "    _FillValue:    0\n",
        "    scale_factor:  1.0\n",
-       "    add_offset:    0.0
    \n",
-       "                            Comm:  tcp://127.0.0.1:40807\n",
+       "                            Comm:  tcp://127.0.0.1:43737\n",
        "                        \n",
        "                            Total threads:  2\n",
@@ -662,7 +652,7 @@
        "                    
    \n",
-       "                            Dashboard:  http://127.0.0.1:36543/status\n",
+       "                            Dashboard:  http://127.0.0.1:38139/status\n",
        "                        \n",
        "                            Memory:  14.00 GiB\n",
@@ -670,13 +660,13 @@
        "                    
    \n",
-       "                            Nanny:  tcp://127.0.0.1:34625\n",
+       "                            Nanny:  tcp://127.0.0.1:33343\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32831596/dask-scratch-space/worker-wdk3d8jb\n",
+       "                            Local directory:  /tmp/slurm-32971832/dask-scratch-space/worker-bd6fd11h\n",
        "                        
    \n",
+       "    add_offset:    0.0
    \n",
        "    \n",
        "        
    \n",
        "            \n",
@@ -1910,19 +1900,19 @@
        "\n",
        "        \n",
        "    \n",
-       "
    • band
      (band)
      int64
      1 2 3 4
      array([1, 2, 3, 4])
    • x
      (x)
      float64
      0.5 1.5 2.5 ... 3.984e+04 3.984e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 3.98415e+04, 3.98425e+04,\n",
-       "       3.98435e+04])
    • y
      (y)
      float64
      0.5 1.5 2.5 ... 4.166e+04 4.166e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 4.16605e+04, 4.16615e+04,\n",
-       "       4.16625e+04])
    • spatial_ref
      ()
      int64
      0
      GeoTransform :
      0.0 1.0 0.0 0.0 0.0 1.0
      array(0)
    • band
      PandasIndex
      PandasIndex(Index([1, 2, 3, 4], dtype='int64', name='band'))
    • x
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
+       "
    • band
      (band)
      int64
      1 2 3 4
      array([1, 2, 3, 4])
    • x
      (x)
      float64
      0.5 1.5 2.5 ... 3.984e+04 3.984e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 3.98415e+04, 3.98425e+04,\n",
+       "       3.98435e+04])
    • y
      (y)
      float64
      0.5 1.5 2.5 ... 4.166e+04 4.166e+04
      array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 4.16605e+04, 4.16615e+04,\n",
+       "       4.16625e+04])
    • spatial_ref
      ()
      int64
      0
      GeoTransform :
      0.0 1.0 0.0 0.0 0.0 1.0
      array(0)
    • band
      PandasIndex
      PandasIndex(Index([1, 2, 3, 4], dtype='int64', name='band'))
    • x
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
        "           8.5,     9.5,\n",
        "       ...\n",
        "       39834.5, 39835.5, 39836.5, 39837.5, 39838.5, 39839.5, 39840.5, 39841.5,\n",
        "       39842.5, 39843.5],\n",
-       "      dtype='float64', name='x', length=39844))
    • y
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
+       "      dtype='float64', name='x', length=39844))
    • y
      PandasIndex
      PandasIndex(Index([    0.5,     1.5,     2.5,     3.5,     4.5,     5.5,     6.5,     7.5,\n",
        "           8.5,     9.5,\n",
        "       ...\n",
        "       41653.5, 41654.5, 41655.5, 41656.5, 41657.5, 41658.5, 41659.5, 41660.5,\n",
        "       41661.5, 41662.5],\n",
-       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -1938,7 +1928,7 @@
        "    add_offset:    0.0"
       ]
      },
-     "execution_count": 17,
+     "execution_count": 10,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1952,7 +1942,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 16,
    "id": "b8fb6e29-0a84-4ae0-882b-b973af22b054",
    "metadata": {
     "tags": []
@@ -1970,7 +1960,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Time to compute with automatic chunk size = 56.06073514930904s\n"
+      "Time to compute with automatic chunk size = 49.639332056976855s\n"
      ]
     }
    ],
@@ -1985,7 +1975,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 21,
    "id": "8940e64c-1de7-4122-828b-e03a9f43a60b",
    "metadata": {
     "tags": []
@@ -2374,7 +2364,7 @@
        "Attributes:\n",
        "    _FillValue:    0\n",
        "    scale_factor:  1.0\n",
-       "    add_offset:    0.0"
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -2552,7 +2542,7 @@
        "    add_offset:    0.0"
       ]
      },
-     "execution_count": 19,
+     "execution_count": 21,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -2565,7 +2555,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 22,
    "id": "58bad330-9498-48ac-b3a0-cad211aaf998",
    "metadata": {
     "tags": []
@@ -2583,7 +2573,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Time to compute with manual chunk size = 43.58590572234243s\n"
+      "Time to compute with manual chunk size = 43.15807781834155s\n"
      ]
     }
    ],
@@ -2603,13 +2593,16 @@
    "source": [
     "### Conclusion\n",
     "\n",
+    "With this example we demonstrated that using personalized chunk size can give you immediatly a 10% performance improvment, we highly encourage using chunks in coherence with your CoG size, using multi-band chunks.\n",
+    "\n",
     "
    \n",
     "Prefer using rxr.to_raster() instead of dask.compute() when your product size is getting bigger as the later seems to be more CPU consuming\n",
     "
    \n",
     "\n",
     "#### Chunk size recommandation for multi band products\n",
     "\n",
-    "When using multiple band products, dask will take chunks per band, which slows done the processing in case of a processing using more than one band. \n",
+    "When using multiple band products, dask will automaticaly take chunks per band, which slows done the processing\n",
+    "\n",
     "
    \n",
     "We recommand in that case defining chunks using \"(-1,sizex,sizey)\" which creates chunks with concatenated bands. The time gain can be up to 20% !\n",
     "
    \n",
@@ -2617,8 +2610,10 @@
     "#### Don't hesitate to test different chunk sizes before making a conclusion\n",
     "\n",
     "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of the tile size for read/write efficiency in RAM.\n",
+    "Tests have been done with chunks > 128Mo, bigger chunks tends to slow down the process.\n",
+    "\n",
     "
    \n",
-    "Don't use chunks > 512Mo as it becomes counter-productive\n",
+    "Don't use chunks > 256Mo as it becomes counter-productive\n",
     "
    \n",
     "\n",
     "#### Adjust the number of workers for dask\n",
@@ -2639,7 +2634,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 25,
    "id": "bfa94789-8576-4b26-a962-226db7d37f01",
    "metadata": {
     "tags": []
@@ -2657,9 +2652,9 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with RIOXarray + dask compute + product > 5Go = 55.76110280863941s\n",
-      "CPU times: user 1min 19s, sys: 40.2 s, total: 1min 59s\n",
-      "Wall time: 55.8 s\n"
+      "Time to compute with RIOXarray + dask compute + product > 5Go = 80.12519567552954s\n",
+      "CPU times: user 13.5 s, sys: 49.8 s, total: 1min 3s\n",
+      "Wall time: 1min 20s\n"
      ]
     }
    ],
@@ -3257,7 +3252,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 26,
    "id": "909e5746-eee9-4f0c-a840-d3e8850dee83",
    "metadata": {
     "tags": []
@@ -3275,7 +3270,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Elapsed with RIOXarray + tiled write + product > 5Go = 35.50480739586055s\n"
+      "Time to compute with RIOXarray + tiled write + product > 5Go = 51.08986523933709s\n"
      ]
     }
    ],
@@ -3285,6 +3280,7 @@
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_phr_rxr.tif\")\n",
+    "# Don't call dask.compute, use rio.to_raster to launch the dask computing on the background\n",
     "# Add the Tiled=true parameter to rxr to speed up the disk write\n",
     "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
     "end = time.perf_counter()\n",
@@ -3300,7 +3296,9 @@
    "source": [
     "## Performance graphs dask compute vs rio.to_raster with tiled write\n",
     "\n",
-    "**The first computing stops at 53s in these graphs that were generated using a slurm script with the exact same code as above**"
+    "This graph has been generated on TREX, with leads to better performance than in this notebook\n",
+    "\n",
+    "**The first computing stops at 55s in these graphs that were generated using a slurm script with the exact same code as above**"
    ]
   },
   {
@@ -3355,6 +3353,8 @@
     "\n",
     "\n",
     "\n",
+    "* Use rio.to_raster to compute/write in parallel, instead of dask.compute and then write\n",
+    "\n",
     "* Huge time gain without precision loss\n",
     "\n",
     "* You have to carefully choose your chunk size using a multiple of the cog size\n"

From 48311b1670c9c8ffa289d55a030a85044f9c3e44 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 17 Feb 2025 10:46:35 +0000
Subject: [PATCH 31/37] Add more infos on the Trex monitoring side

---
 tuto_greenit.ipynb | 240 +++++++++++++++++++++++++++++++++------------
 1 file changed, 180 insertions(+), 60 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index d55735b..4f03142 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -7,7 +7,70 @@
    "source": [
     "# Best practices for Green IT coding in python\n",
     "\n",
-    "In this tutorial, we will show good pratices for a green IT approach to code an image processing chain with efficiency in mind"
+    "In this tutorial, we will show good pratices for a green IT approach to code an image processing chain with efficiency in mind\n",
+    "Let's first define everything needed along this notebook"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9a7fe0ae-0549-47cb-af0c-04a83d27324e",
+   "metadata": {},
+   "source": [
+    "## Necessary imports for the notebook"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "id": "8966e0c9-d03f-4edf-b1b0-abbb4c4c2c34",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import rasterio\n",
+    "import xarray\n",
+    "import rioxarray as rxr\n",
+    "from numba import jit,njit\n",
+    "from xarray import DataArray \n",
+    "from pathlib import Path\n",
+    "from typing import List, Tuple, Union, Dict\n",
+    "from dask import delayed\n",
+    "from dask.distributed import Client, LocalCluster, Lock\n",
+    "import dask.array as da\n",
+    "from rasterio.transform import Affine\n",
+    "import time\n",
+    "import os\n",
+    "import subprocess\n",
+    "from timeit import timeit"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4c610ea3-563e-4409-9225-1566ae822f3c",
+   "metadata": {},
+   "source": [
+    "## Work directories for the notebook"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 58,
+   "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "input_dir = \"/work/scratch/data/romaint/input_greenit\"\n",
+    "output_dir = \"/work/scratch/data/romaint/output_greenit\"\n",
+    "perf_results_dir = \"/work/scratch/env/romaint/greenit/perfos_greenit\"\n",
+    "s2_b4 = f\"{input_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
+    "s2_b8 = f\"{input_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
+    "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
+    "phr_product_cog = f\"{input_dir}/phr_cog.tif\"\n",
+    "phr_product_lzw = f\"{input_dir}/phr_lzw.tif\""
    ]
   },
   {
@@ -43,9 +106,6 @@
     }
    ],
    "source": [
-    "import time\n",
-    "import numpy as np\n",
-    "\n",
     "start = time.perf_counter()\n",
     "# Calculate square with for loop\n",
     "array_forloop = []\n",
@@ -99,9 +159,6 @@
     }
    ],
    "source": [
-    "from timeit import timeit\n",
-    "import numpy as np\n",
-    "\n",
     "nd_list = [i for i in range(0,100000)]\n",
     "nd_set = set([i for i in range(0,100000)])\n",
     "nd_array = np.arange(0,100000)\n",
@@ -155,28 +212,40 @@
    "source": [
     "# Profiling your processing chain with TREX monitoring\n",
     "\n",
-    "In order to optimize your chain, you have to check where are the bottleneck of your program. The TREX cluster provides all the tools you need for that, with the monitoring module.\n",
+    "In order to optimize your chain, you have to check where are the bottlenecks of your program. The TREX cluster provides all the tools you need for that, with the monitoring module.\n",
     "\n",
-    "In this tutorial, we use this script to execute as a slurm job, which creates the graphs for CPU, RAM, DISK in png format\n",
+    "In this tutorial, we use this script to execute as a slurm job, which creates the graphs for CPU, RAM, DISK in png format. [See the graph section](#Performance-graphs-dask-compute-vs-rio.to_raster-with-tiled-write)\n",
     "\n",
     "The in depth documentation is available [here](https://hpc.pages.cnes.fr/wiki-hpc-sphinx/module-monitoring-2.html)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "846a9449-a9ba-40be-a372-aa43250eb940",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Write the monitoring script"
+   ]
+  },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 59,
    "id": "ccc05e8b-cfba-4671-8995-8c6a146ebca7",
-   "metadata": {},
+   "metadata": {
+    "tags": []
+   },
    "outputs": [],
    "source": [
-    "greenit_slurm_script_monitored = \"\"\"#!/bin/bash\n",
+    "greenit_script_monitored = \"\"\"#!/bin/bash\n",
     "#\n",
     "# Load monitoring module\n",
     "module load monitoring/2.2\n",
     "#\n",
     "# Use of SLURM $TMPDIR  of the interactive job\n",
     "WORKDIR=$TMPDIR\n",
-    "SLURM_SUBMIT_DIR=/work/scratch/env/romaint/greenit/perfos_greenit\n",
+    "SLURM_SUBMIT_DIR={perfo_result_dir}\n",
     "#\n",
     "cd \"$WORKDIR\"\n",
     "#\n",
@@ -201,96 +270,147 @@
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "1898ca12-fdcc-48b9-b733-642f2627bbe9",
-   "metadata": {},
+   "cell_type": "code",
+   "execution_count": 61,
+   "id": "524fa4c7-16be-44ab-9cc2-345532bb79aa",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
    "source": [
-    "# Choose a data format library adapted to your processing chain"
+    "with open(f\"{input_dir}/greenit_monitored.sh\", 'w') as f:\n",
+    "    f.write(greenit_script_monitored.format(perfo_result_dir=perf_results_dir))\n",
+    "os.chmod(f\"{input_dir}/greenit_monitored.sh\", 0o755)"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "5baa8de4-7502-45b3-a5a1-52dd0f254d96",
-   "metadata": {},
+   "cell_type": "code",
+   "execution_count": 55,
+   "id": "6a659871-a0fc-4572-960c-f8bf38d14677",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
    "source": [
-    "Depending on your project, the data format will vary : you will need to choose a library that can fully handle your product.\n",
-    "A Tutorial is available [here](https://github.com/CNES/pluto-tuto/pull/3) for a detailed walkthrough on how to choose the right library"
+    "greenIT_slurm_script = \"\"\"#!/bin/bash\n",
+    "#SBATCH --job-name=GreenIT_{cpus}_{mem}\n",
+    "#SBATCH -N 1\n",
+    "#SBATCH -n 1\n",
+    "#SBATCH -c {cpus}\n",
+    "#SBATCH --mem={mem}G            # memory per node\n",
+    "#SBATCH --time=00:30:00             # Wall Time\n",
+    "#SBATCH --account=cnes_level2       # MANDATORY : account  ( launch myaccounts to list your accounts)\n",
+    "#SBATCH --export=none               # To start the job with a clean environnement and source of ~/.bashrc\n",
+    "#SBATCH --output={output_greenit}/greenIT-{cpus}-{mem}-%j.out\n",
+    "#SBATCH --error={output_greenit}/greenIT-{cpus}-{mem}-%j.err\n",
+    "\n",
+    "sh {input_dir}/greenit_monitored.sh\n",
+    "\"\"\""
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "371254df-a3b4-4f3e-b34f-3c3db06828ba",
+   "id": "8175da03-8e09-41dc-a1ff-01adac85a07f",
    "metadata": {
     "tags": []
    },
    "source": [
-    "# A Green IT approach for coding image processing chains"
+    "## Write the slurm script and submit the job"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "fb2fcaf9-5b7b-4f37-b457-6e01e5035fbf",
-   "metadata": {},
+   "cell_type": "code",
+   "execution_count": 57,
+   "id": "b748533f-f3cd-47b8-b01d-c5e5f55d5afd",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Submitted batch job 32975088\n",
+      "Submitted job CompletedProcess(args=['sbatch', '/work/scratch/data/romaint/input_greenit/GreenIT.slurm'], returncode=0)\n"
+     ]
+    }
+   ],
    "source": [
-    "In this notebook we will look at a green IT approach of coding, via good pratices in Python. We will have a look at Dask, RioXarray, rasterio, numpy... all these libraries that are widely used in the satellite image processing chains. "
+    "with open(f\"{input_dir}/GreenIT.slurm\", 'w') as f:\n",
+    "    f.write(greenIT_slurm_script.format(cpus=8,mem=56,input_dir=f\"{input_dir}\",output_greenit=f\"{output_dir}\"))\n",
+    "    f.flush()\n",
+    "output = subprocess.run(['sbatch', f\"{input_dir}/GreenIT.slurm\"], text=True, shell=False)\n",
+    "print(f\"Submitted job {output}\")"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "9a7fe0ae-0549-47cb-af0c-04a83d27324e",
-   "metadata": {},
+   "id": "fbf6dcd7-2a2c-4116-ae51-6dc381b12d56",
+   "metadata": {
+    "tags": []
+   },
    "source": [
-    "## Necessary imports for the notebook"
+    "## View the generated graphs"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
-   "id": "8966e0c9-d03f-4edf-b1b0-abbb4c4c2c34",
+   "execution_count": 65,
+   "id": "a7cf71b8-f3e5-4d39-bb2b-2d44fcddf1b4",
    "metadata": {
     "tags": []
    },
-   "outputs": [],
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAusAAAHTCAYAAACeOFZgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAACBqElEQVR4nO3deXhTZfo38O/J3nQJ3WgoFChQFmURARFcQNlccENFRWbgB44oiKAwKDKj1VfBQcUFRhwdFBWZMjOK+wKooAwiiKBsskjZWyqldG+SJuf9Iz0nrW0hpUnOku/nunJBk5PkySH0uXPnfu5HEEVRBBERERERqY5B6QEQEREREVHDGKwTEREREakUg3UiIiIiIpVisE5EREREpFIM1omIiIiIVIrBOhERERGRSjFYJyIiIiJSKQbrREREREQqxWCdiIiIiEilGKwTUdC+//573HTTTWjbti2sVivS0tIwYMAAzJgxo85x7du3x8iRIxt8jB9++AGCIGDp0qXyddnZ2RAEAQaDAQcOHKh3n/LyciQkJEAQBIwfP/6s41y8eDHat2+PxMREjB07FqdPn65ze3V1NS644AI8+uijjT7GiRMn8PDDD6NHjx6Ii4uDzWZDVlYWpk2bhn379tUbu3SxWCzIzMzEtGnT6jyvdNzJkycbfL7u3btj8ODBZ31tkpdeegnJycmorq4O+j5ERKQ9DNaJKCiffPIJBg4ciJKSEsyfPx+rVq3Ciy++iEsuuQQrVqwIyXPExcXhjTfeqHf9f/7zH3g8HpjN5rM+xjfffIOpU6figQcewLJly7Bp0ybMnDmzzjELFixARUUF5syZ0+BjbNq0CT169MCSJUtwyy234L333sPnn3+OmTNn4scff8RFF11U7z6ff/45vvvuO3zyySe48cYbsXDhQlx99dUQRTHIV9807777Lm644QaYTKawPD4REakDf8sTUVDmz5+PzMxMfPHFF3UCxNtvvx3z588PyXPcdtttePPNN/H444/DYAjkEpYsWYKbbroJH3744Vkf45NPPsGQIUMwbdo0AEBxcTEefPBB+fbc3Fw8/vjj+Pjjj2G1Wuvdv6SkBDfccANsNhs2bNiANm3ayLcNHjwYkyZNwn//+9969+vTpw9SUlIAAMOGDUNhYSHefvttbNiwAZdccknwJyEIJ06cwPr16zFr1qyQPi4REakPM+tEFJTCwkKkpKQ0mMmtHVg3x4QJE3DkyBGsXr1avm7v3r1Yv349JkyYENRjVFVVITY2Vv45Li4OVVVV8s/33nsvbrvtNlxxxRUN3v+1115Dfn4+5s+fXydQr+2WW2456zguvvhiAMChQ4eCGndTrFy5EnFxcRg6dCgAoKKiAjNnzkRmZiZsNhuSkpLQt29f/Otf/wr5cxMRUWQxWCeioAwYMADff/897r//fnz//ffweDwhf46srCxcdtlleP311+XrXn/9dbRv3x5DhgwJ6jEGDhyIVatW4bvvvkNBQQFeeuklDBw4EACwfPly/Pjjj3jmmWcavf+qVatgNBpx3XXXNeu17N+/HwCQmprarMdpyLvvvouRI0fK3ww8+OCDWLx4Me6//358/vnnePvtt3HrrbeisLAw5M9NRESRxTIYIgrK008/jV9++QULFy7EwoULYTab0a9fP1x33XW47777EBcXF5LnmTBhAu655x6cOnUKDocDb731FiZNmgRBEIK6/+jRo/HZZ5/JAXqXLl3w0Ucf4dSpU3jggQewYMECJCcnN3r/w4cPIzU1tU52PhherxfV1dUoKyvDJ598gldeeQUZGRm47LLLmvQ4Z1NYWIi1a9fWWSfwv//9D8OHD8cDDzwgX3fttdeG9HmJiEgZzKwTUVCSk5Px7bffYvPmzXj66adxww03YO/evZg9ezZ69OjRaJeTprr11lthsVjwzjvv4NNPP0V+fn5QHWAkUqeZgoIC7Nu3D7t27UJWVhb+/Oc/o1evXhg7diy2b9+OQYMGITExEX379sW3337b7HE7nU6YzWa5A82FF16Izz//HDabrdmPXdsHH3wAi8WCq666Sr7uoosuwmeffYaHH34Ya9euRWVlZUifk4iIlMPMOhE1Sd++fdG3b18AgMfjwUMPPYTnn38e8+fPlxeamkwmeL3eBu8vtRpsrLNLbGwsbrvtNrz++uto164dhg4dinbt2jV5nKmpqXIJyrp165CTk4Off/4ZHo8HN954I8aOHSuXjNxwww3Yv38/kpKS0LZtW+zbtw/l5eVNyq6vWbMGDocDZrMZbdq0qZe9l2r9z3Regul289///hdXX3017Ha7fN1LL72ENm3aYMWKFfjb3/4Gm82GESNG4JlnnkFWVlbQr4GIiNSHmXUiOmdmsxmPPfYYAGDHjh3y9WlpaTh27FiD95GuT0tLa/RxJ0yYgG3btuGjjz4KemFpY1wuFyZNmoS//vWv6NixI/bs2YMDBw5g5syZiImJwd133w1BEPDdd98BAEaMGAGv14uPPvqoSc/Tq1cv9O3bF7169WqwzEZ6vQ2dF1EUkZeXd8ZzAvg723z55Ze4+eab61wfGxuLxx9/HL/88gvy8/OxePFibNy4sdl190REpDwG60QUlLy8vAav3717NwAgPT1dvm7o0KHYsWMHdu3aVe/4f//734iLi0P//v0bfa4BAwZgwoQJuOmmm3DTTTc1a9xz586FxWKRe61Lfc/Ly8sB+L8dcLlc8vUTJ06E0+nErFmzGv3A8d577zV5HFdeeSUEQWiwJ/3nn3+OkpISubtLYz766CMIgtDohlOA/0PB+PHjcccdd2DPnj2oqKho8liJiEg9WAZDREEZMWIE2rRpg+uuuw5du3aFz+fDtm3b8NxzzyEuLk7uaw4A06ZNw1tvvYXBgwfjkUceQY8ePVBUVIQVK1bgv//9LxYsWID4+PgzPt+SJUuaPeZffvkF8+fPx9dffy2XoXTp0gXt2rXDvffeiylTpmDFihUwmUxyq0WHw4EPPvgAI0eORO/evXHfffdhwIABsFgs2LdvH5YtW4affvoJo0aNatJYOnbsiPvuuw/PPPMMTp8+jWuuuQYxMTHyGoC+fftizJgxZ3yM//73vxg2bFi9c9e/f3+MHDkSPXv2RGJiInbv3o23334bAwYMqFMuQ0REGiQSEQVhxYoV4pgxY8SsrCwxLi5ONJvNYtu2bcU//OEP4q5du+odn5+fL957771i27ZtRZPJJMbHx4uXXnqp+J///KfesY899pgIQPztt9/OOIbY2Fhx3LhxQY3X5/OJl112mThlypR6t23ZskW8+OKLxdjYWLFHjx7imjVrGhz/Qw89JJ5//vmi3W4XrVar2KlTJ3HSpEni9u3bmzx2aUyLFy8W+/btK9rtdtFisYhZWVniQw89JJaWlp7xvmVlZaLNZhPfeOONerc9/PDDYt++fcXExETRarWKHTp0EB944AHx5MmTZx0TERGpmyCKYdoLm4iIQubf//437rzzTpw4cQJJSUlKD4eIiCKEwToRERERkUpxgSkRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExERERGpFDdFAuDz+XD8+HHEx8dDEASlh0NEREQRIIoiSktLkZ6eDoOB+UtSJwbrAI4fP46MjAylh0FEREQKOHLkCNq0aaP0MIgaxGAdkLfuPnLkCBISEhQeDREREUVCSUkJMjIy5DiASI0YrANy6UtCQgKDdSIioijDElhSMxZoERERERGpFIN1IiIiIiKVYrBORERERKRSDNaJiIiIiFSKwToRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExERERGpFIN1IiIiIiKVYrBORERERKRSDNaJiIiIiFSKwToRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExERERGpFIN1IiIiIiKVYrBORERERKRSDNaJiIiIiFSKwToRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExERERGpFIN1IiIiIiKVYrBORERERKRSDNaJiIiIiFSKwToRERERkUoxWCciIiIiUikG60QKOVxYgY0HCpUeBhEREakYg3UiBfxW6sINf1+PO17biNyT5UoPh4iIiFSKwTqRAh79YAeKKjwQRWB3XonSwyEiIiKVUjRYb9++PQRBqHeZMmUKAEAURWRnZyM9PR0xMTEYPHgwdu7cWecxXC4Xpk6dipSUFMTGxuL666/H0aNHlXg5REH55Oc8fLYjX/6ZmXUiIiJqjKLB+ubNm5GXlydfVq9eDQC49dZbAQDz58/HggULsGjRImzevBlOpxPDhg1DaWmp/BjTp0/HypUrkZOTg/Xr16OsrAwjR46E1+tV5DURnUlhmQt//WAHAKBlvBUAcKiQwToRERE1TNFgPTU1FU6nU758/PHH6NixIwYNGgRRFPHCCy9gzpw5GDVqFLp3744333wTFRUVWL58OQCguLgYS5YswXPPPYehQ4eid+/eWLZsGbZv3441a9Yo+dKIGvTohztxqtyNrs54zLqqKwDgYGGFwqMiIiIitVJNzbrb7cayZcswYcIECIKA3Nxc5OfnY/jw4fIxVqsVgwYNwoYNGwAAW7ZsgcfjqXNMeno6unfvLh/TEJfLhZKSkjoXIsBfehUun23Pwyc/58FoEPDMLb2Q1TIOADPrRERE1DjVBOvvv/8+Tp8+jfHjxwMA8vP9Nb1paWl1jktLS5Nvy8/Ph8ViQWJiYqPHNGTevHlwOBzyJSMjI4SvhLTq6c9+wYB5X+GHg6dC/tinyt1y+cu9gzqiRxsH2ifHAgBOlLhQ4a4O+XMSERGR9qkmWF+yZAmuvvpqpKen17leEIQ6P4uiWO+63zvbMbNnz0ZxcbF8OXLkyLkPnHRjze4TyC+pwvg3NuPHw0UhfezHP9qJk2VudE6Lw9QhnQAADrsZiXYzAOAQS2GIiIioAaoI1g8dOoQ1a9bgrrvukq9zOp0AUC9DXlBQIGfbnU4n3G43ioqKGj2mIVarFQkJCXUuRJVu/6LkMlc1xi3ZhJ+OnA7J436xMx8fbDsOgwA8c0svWE1G+bZ2Ndl1lsIQERFRQ1QRrL/xxhto2bIlrr32Wvm6zMxMOJ1OuUMM4K9rX7duHQYOHAgA6NOnD8xmc51j8vLysGPHDvkYomBVevzBevtkO0pd1fjDku+x41hxsx7zdIUbc1b6y18mDeqIXhkt6tzePtkOgItMiYiIqGGKB+s+nw9vvPEGxo0bB5PJJF8vCAKmT5+OuXPnYuXKldixYwfGjx8Pu92OMWPGAAAcDgcmTpyIGTNm4Msvv8TWrVsxduxY9OjRA0OHDlXqJZFGSXXjr/yhD/q0S0RJVTXGLvkeu46f+wLkJz7ahZNlLnRqGYdpQ7Lq3S5l1g+y1zoRERE1wHT2Q8JrzZo1OHz4MCZMmFDvtlmzZqGyshKTJ09GUVER+vfvj1WrViE+Pl4+5vnnn4fJZMLo0aNRWVmJIUOGYOnSpTAajfUej6gxPp+IKo8PAJASZ8XS/+uHPyzZhG1HTmPsku/xrz9djC7O+LM8Sl1f7j6B97Yeg0EA5t/SEzZz/fdkZkpNsM4yGCIiImqAIIazV51GlJSUwOFwoLi4mPXrUarCXY3zHv0CALDz8RGItZpQUuXB2H9+j5+PFiMlzoKcuy9Gp5bBBezFFR4Mf2EdTpS4cPflHfDINd0aPG7r4SLc9PIGtHLY8N3sISF7PaQd7mofthwqQu+2LRr8QEdE4cP5n7RA8cw6kRpIi0sBIKYmYEqwmfH2hP4Y88+N2Hm8BHe89j1y7r4YHVPjGn0cURRxvLgKT3/2C06UuNAhJRYPDuvc6PFS+8a84ipUebwM1qLQkvW5+Nvnv6BbqwT8fUxvdDjD+6u53NU+rNl9At/9WgijQUCMxQibyYgYiwE2s//vNosRNpMBcVYT+rRPrLMgmoiIIo/BOhGAippg3WoywGAItP102M1YNrE/7nhtI37JL8WY1zZixd0D0D4lFj6fiMOnKrDjeDF2HCvBzuPF2HGsGEUVHgCAIADP3Npw+Yukhd2MBJsJJVXVOFRY0eRSG9K+46crAQC780pw3cL1mHdzT1zfK/0s92qa/QWlWLH5CN778RgKy91B3+/GC9Lxwu29QzoWIiJqGgbrRACqajrB2C31A+vEWAveucsfsO89UYbbXv0O7ZNjset4CUpd9TczMhkEZKXF4/8uaY8+7ZLO+LyCICAzJRY/HS3GwcJyButRyFUdeO+Vu724/19bsfFAIR4deV6zvmmpcFfjk5/zsGLzEfxwKNDetmW8FSN7piPGYkCl24eqai+qPNLFhyqPFydKqvDrb+XI5cJnIiLFMVgnQiCzHtNIcJQcZ8U7d12MO17biP0FZThR4gIAWEwGdGuVgPPTE9A93YHurRPQOS2+SUFWu2R/sM5e69HJVe1f2Dx9aBZKq6qx6Ov9WP79YWw9fLrJZTGiKGLHsRLkbD6MD7cdlz9MGg0CrujSErf3y8DgLqkwGc/cCGz9vpMYu+R7edE1EREph8E6EQI91mMayKxLUuOtyLn7Yiz//jDSW8Sge+sEdEyNg/ksgc/ZsNd6dHPVBMQxFhPuvrwjLspMwvScbXJZzNxRPXDDBa0bvX+Vx4uNBwqxbu9vWLf3Nxz4LfChr12yHaP7ZuCWPm2QlmALekw2s/89XVXtPcuRREQUbgzWiRBYYHqmYB3wt3W8v4F+6c3BXuvRTSqDsZr8AfJlWan4dNpluP9fW/F97ilMy9mGjQdO4bHr/GUxoihif0GZHJx/n3sK7upABtxiMuDq7k7c1i8DF2cm11mDESzpmyGpPIyIiJTDYJ0Igcy63Rz5/xLta3qtH2JmPSpJZTBSsA4AaQk2vHNXf7z45T4s+no//rXpMLYeLsIFGS2wbu9vyCuuqvMY6Q4bBnVJxeVZqbgkKwUJNnOzxiRn1lkGQ0SkOAbrRAjUrNvOklkPB6kM5nhxJds3RqFAsF73391kNGDG8C5yWcwv+aX4Jb8UgD97fnGHZFyelYLBXVLRMTUOgtD0DHpjpLEws05EpDwG60SonVmPfKCcFGtBvNWEUlc1jhZVBL3xEumDXAZjbnjtg1QWs3jtrzAIAi7vnIL+mclnLdlqDukDo6vaB1EUQ/pBgIiImobBOhGASre/a0Y4A6DGCIKAdil27DhWgtyTDNajjbTAtHYZzO+lJdiQff35kRqSXAYD+AN2fttDRKSc5rWxoKhyoqQKOZsOy5lAPal0Sx05lAlKpJ1M2b4x+ri9DZfBKKl2cM5SGCIiZTGzTkHx+kTc9eYP2H6sGL/kl0Y0yxcJFZ6azLpCGUQpWD/IYD3qBJNZjzSz0QCjQYDXJ3KRKRGRwtQzO5Cqrdh8BNuPFQMA3vzuILYeLjrLPbSlyt34DqaR0K5mkSk7wkQf6ZsqWyM160qxmaSOMMysExEpSV2zA6lSUbkb87/4BQDQukUMRBGY/d52eLz6ybjJ3WCUyqzXtG/k9u7Rp7FuMEqTe63rsOyNiEhLGKzTWT27ag9OV3jQ1RmPlVMGItFuxi/5pXj1mwNKDy1k5G4wCtesHz9dqcs1AdS4hvqsq0FgYyT9fCgnItIidc0OpDrbjxZj+abDAIDHrz8fLeNt+OvI8wAAL365TzeZYHkHU4Uy6ylxFsRajPCJwNGiSkXGQJFX7fXB6xMBqC+zLrWSdLEMhohIUQzWqVE+n4hHP9wBUQRuvCAd/TskAwBu6t0al2WlwF3tw5yV2yGKosIjbT4ps65UNxhBENCOHWGijpRVBxrvs64Um7QxUjUz60RESlLX7ECq8t8fj2Lr4dOItRgx+5pu8vWCIOCpG3vAZjZgw6+FePfHYwqOMjQqFM6sA0CmXLfORabRonawbjGq69extOCVC0yJiJSlrtmBVKO4woO/feZfVDp9aGekJdjq3N422Y7pQzsDAJ78ZBdOlrkiPsZQqpJr1pXrZhroCMPMerSQ1idYjAYYDOraJTRQs85gnYhISQzWqUHPr9mLwnI3OrWMw/hL2jd4zMRLM9GtVQJOV3jw5Me7IjvAEJMz6xbl/ksEeq0zsx4tpB7rFpUtLgUCwbqLC0yJiBSlvhmCFLfreAne+u4gAP+iUnMjX8+bjQY8PaoHDALw/rbjWLf3twiOMrTkmnUzM+sUOWrtBAPUKoNhdyIiIkWpb4YgRYmiiMc+3AGfCFzboxUu6ZRyxuN7ZbTA+IGZAIA5K7ejwl0diWGGnNwNRqEFpkCgZv1oUaWuethT46QyGFUG6yaWwRARqYH6ZghS1PvbjmHzwSLEmI2Yc223s98BwIzhndG6RQyOFlXihTX7wjzC0BNFUfE+6wCQGm9FjNkIr09k+8YoIWfWFVzY3Bgr+6wTEakCg3WSlVZ5MPdT/6LS+67shPQWMUHdL9Zqwv+78XwAwD+/PYAdx4rDNsZwcNfqda3UDqaA1L7RXwpzkKUwUUGqB1dlZp3dYIiIVEF9MwQp5sU1+/BbqQuZKbG467LMJt33yq5puLZnK/hEYPZ721GtoTKOKndgrEpm1oHAItNDOtlsis5M1WUwzKwTEamC+mYIUsTeE6V4Y8NBAMBj1513TrspPnbdeUiwmbD9WDHe3ngoxCMMnwqPv87eZBAaXUwbKe1T2BEmmrjlBabqK4MJbIrEzDoRkZIYrBNEUcTjH+2E1ydi+HlpGNyl5Tk9Tst4G+4fkgUAWL3rRCiHGFZqWFwqac8ymKgSqFlX369ilsEQEamD+mYIirjvfi3E//YXwmIy4K8jz2vWY3VI9WeGS6u00xVGDbuXStpJZTDMrEcFLZTBsM86EZGy1DdDUESJoih3cBlzUVtkJNmb9XhxVjMAoMylnWC9SgWdYCTtU/zn/8ipCk3V/dO5cam5DIaZdSIiVWCwHuW+O1CITQdPwWI04J5BHZv9eHFW/6ZCWsysK9kJRpIWb4PNbEC1T8Sx02zfqHfq7gbDmnUiIjVQ3wxBESVl1e+4KANOh63Zjxdv8wfrZS5Psx8rUtTQY11iMAhol8RFptFCLoNRYc261cRuMEREaqC+GYIi5rtfC7EptyarPrj5WXUgkFmv8vg0swunmhaYApB7rR/iIlPdYxkMERGdDYP1KPbCmr0AgNsvykArR3AbIJ1NbE2wDgDlGqlblzLrMWbTWY6MDLl940lm1vUuEKyr71dxoM86g3UiIiWpb4agiPju10J8X5NVvzdEWXUAsJgMcuChlbr1CpVl1qWNkdi+Uf9cNYGwRdXBuja+ISMi0iv1zRAUES9+6c+q39YvdFl1SaBuXRvButwNRgULTAH2Wo8m6s6s+8fk4gJTIiJFqW+GoLD77tdCbDwQ+qy6RKpb10qwXuH2j1MtmfV2NWUwR05VwOsTFR4NhZOqa9a5wJSISBUYrEchKas+ul8bpLcIbVYdAOI0llmvdPuDEbUE660SbLCYDPB4RRxn+0ZdU3M3GNasExGpg/pmCAqrjQf8WXWzUcDkwZ3C8hxyZl0jNeuVnprMukrKYPztG1kKEw3U3WfdP6Zqn8gNuoiIFKT4DHHs2DGMHTsWycnJsNvtuOCCC7Blyxb5dlEUkZ2djfT0dMTExGDw4MHYuXNnncdwuVyYOnUqUlJSEBsbi+uvvx5Hjx6N9EvRhBdr+qrf1i8jLFl1QHu7mEoLTNXQZ13SLpm91qOBqstgan14rapmsE5EpBRFg/WioiJccsklMJvN+Oyzz7Br1y4899xzaNGihXzM/PnzsWDBAixatAibN2+G0+nEsGHDUFpaKh8zffp0rFy5Ejk5OVi/fj3KysowcuRIeL38+ra27w8U4rsDhTAbBdwbpqw6UGuBqVYy6yrawVQiLTI9dJKZdT1zq3iBae0xsRSGiEg5ijaW/tvf/oaMjAy88cYb8nXt27eX/y6KIl544QXMmTMHo0aNAgC8+eabSEtLw/LlyzFp0iQUFxdjyZIlePvttzF06FAAwLJly5CRkYE1a9ZgxIgREX1Navbil/6s+ui+GWgdpqw6ECiDKdVIZl1NO5hKpEWmzKzrm5pr1gVBgNVkgKvax2CdiEhBis4QH374Ifr27Ytbb70VLVu2RO/evfHaa6/Jt+fm5iI/Px/Dhw+Xr7NarRg0aBA2bNgAANiyZQs8Hk+dY9LT09G9e3f5mN9zuVwoKSmpc9G7TbmnsOFXf1Z98hXhy6oDtRaYaiyzrpaadQDIZK/1qKDmMhiAvdaJiNRA0WD9wIEDWLx4MbKysvDFF1/gnnvuwf3334+33noLAJCfnw8ASEtLq3O/tLQ0+bb8/HxYLBYkJiY2eszvzZs3Dw6HQ75kZGSE+qWpjtQB5tYwZ9WB2q0bPWF9nlBR26ZIANCupgzmcCHbN+qZmvusA4FFpsysExEpR9EZwufz4cILL8TcuXPRu3dvTJo0CX/605+wePHiOscJglDnZ1EU6133e2c6Zvbs2SguLpYvR44cad4LUblNuafwv/01WfUw9FX/Pc1uimRRtCqsjvQWMTAbBbi9PuSXVCk9HAoTaQdTtWfWuTESEZFyFA3WW7VqhfPOO6/Odd26dcPhw4cBAE6nEwDqZcgLCgrkbLvT6YTb7UZRUVGjx/ye1WpFQkJCnYueSVn1W/pkoE2iPezPJ9esa6QMpkKFZTBGg4CMJC4y1Ts5s67CmnUgkPFnGQwRkXIUnSEuueQS7Nmzp851e/fuRbt27QAAmZmZcDqdWL16tXy72+3GunXrMHDgQABAnz59YDab6xyTl5eHHTt2yMdEsx8O+rPqJoOAKVeEP6sOaG8HU2mBqZrKYIBA3Xou69Z1S/1lMNwYiYhIaYp+7//AAw9g4MCBmDt3LkaPHo1Nmzbh1VdfxauvvgrAX/4yffp0zJ07F1lZWcjKysLcuXNht9sxZswYAIDD4cDEiRMxY8YMJCcnIykpCTNnzkSPHj3k7jDR7KWv9gMAbu3bJiJZdUDDC0xVFqxLvdYPsSOMbsndYNRaBmPiAlMiIqUpGqz369cPK1euxOzZs/HEE08gMzMTL7zwAu688075mFmzZqGyshKTJ09GUVER+vfvj1WrViE+Pl4+5vnnn4fJZMLo0aNRWVmJIUOGYOnSpTAa1TkBRsq2I6fxzd7fYDSEb7fShsRraFOkaq8P7prdGe0qKoMBgPYpNbuYsgxGl7w+ER6vf/GwWjPrVi4wJSJSnOIr6kaOHImRI0c2ersgCMjOzkZ2dnajx9hsNixcuBALFy4Mwwi1a2FNX/WbereW658jQUuZ9cpaQQgz6xRJ7lq7glpUGqzLZTBcYEpEpBh1zhDUbDuOFePLXwpgEIApYe6r/ntyzbq7Gj6Vtx2UgnVBUF92s3avdbWfR2q62h1W1Pbek7DPOhGR8tQ5Q1CzLfzKn1W/vlc6Mmt2w4wUqXWjKAIVKv/6vPaGSGdrBxpp6S1sMBkEuKp9OFHK9o16Iy0uNRoEmIzq/FVsM7EMhohIaeqcIahZfskvwRc7T0AQgPuujGxWHfBnCU0Gf+Cr9lKYSrnHurpKYADAZDTI5UsHT7IURm9cHnV3ggFq9VlnsE5EpBj1zhJ0zhbWdIC5pnsrdGoZf5ajQ08QhEDdusp3MZV6rNtUtrhUIu1keojtG3Un0AlGvb+G5R1Mq1kGQ0SkFPXOEnRO9heU4tPteQCUyapLtLIxUpVbvZl1AMioabd57HSlwiOhUAv0WFfnew9gn3UiIjVgsK4zf//6V4giMPy8NHRrpdzOrFrZGEmNu5fWFltzHstdDJb0Ru27lwIM1omI1EC9swQ1We7Jcnyw7RgAYOqVWYqOJV4j7RvVunupRMr4V3rUfR6p6bRQBmOVF5iyDIaISCnqnSWoyV7+ej98InBl15bo0cah6FjkMhiVZ9YrVZ5Zl4N1NzObesMyGCIiCgaDdZ04cqoCK7dKWXXlatUlUvmGVjLrdovi+4M1SMr4VzBY1x0tdYPhAlMiIuWod5agJnl57a+o9om4LCsFvdsmKj2cQBmMyjPrau8GEyiDYbCuN3IZjKpr1tlnnYhIaeqdJShox09X4r9bjgAA7h+ibK26RCsLTNXcZx0AYsz+88jMuv5oogzGxD7rRERKY7CuA/9Y9ys8XhEXd0hCv/ZJSg8HABBnNQNQf+vGSrd/fGpfYMpgXX8Cwbp6fw0HatZZBkNEpBT1zhIUlIKSKvxrc01WXeEOMLXFaaQMRu4Go9IymBh5gam6zyM1nZStVnew7h+bVLJDRESRp95ZgoLyj28OwF3tQ992iRjQMVnp4cji5QWm2tjBVK2ZdelDBDPr+qOJMhhm1omIFMdgXcNOlrnwzveHAABTh2RBEASFRxSglcx6lcpr1rnAVL+kYN2igcx6FTPrRESKUe8sQWe18Mt9qPL40KuNA5dnpSg9nDrkPusqr1lXfzcY/3lkn3X90camSOyzTkSkNPXOEnRGm3JP4a2N/qz6zBFdVJVVBwKZ9XKV11pLQbBaM+tSeU61T4Sbva51Re6zrurWjYEyGFEUFR4NEVF0Uu8sQY2qdHvx5//+BFEERvdtg8uyUpUeUj3xGtsUSa0LTGt/iGB2XV+0UbMemCJc/LBIRKQIBusaNP+LX3CosAKtHDb8ZeR5Sg+nQbVr1tWckatU+QJTs9EAs9H/rUmFR90ffKhp3Bpq3QgEvgkgIqLIUu8sQQ3alHsKSzccBAA8fXNPJNjMyg6oEVLNuscrqjojJ3eDUWlmHQgETOwIoy9aqFk3Gw0wGvwfFrnIlIhIGeqdJaieCne1XP5yW98MDOqsvvIXSWzNwkhA3R1hAt1gTGc5UjlyRxgG67oil8Go+IMiANhqPkxwkSkRkTIYrGvIM1/skctf5ozspvRwzshgEOTsuprr1rWQWZc+SDCzri9a2MEUYK91IiKlqXuWIJlWyl9qk4N1lWbWRVEMLDBVac06EPggwV7r+hLYwVS97z2gdrDO9x8RkRIYrGuAlspfapMWmaq113rtTKGag/VAGYw6zyOdG61k1qXWkgzWiYiUoe5ZggAA8z/XTvlLbWrPrNfOVKu5DEb6IMEyGH0J1Kyr+9ewTdoYScULxYmI9EzdswTh+wOFmit/kcTL7Rs9Co+kYRU1mWqLKdDxQo3sDNZ1KdANRr0fFIFAr3Vm1omIlMFgXcUq3NWY9e7PALRV/iJR+wLTQCcYdQdLcs06g3VdkXcwVXkZDGvWiYiUpe5ZIspptfxFIgXrpSotg9FCJxgAiGE3GF3STBlMzf8PbopERKQMdc8SUUzL5S8SeRdTlWbW1b57qUQug+EOproilcFYjOr+NSyXwXBTJCIiRah7lohiT3y8C4A2y18k8SpfYFrh0UZmnZsi6ZN2NkViGQwRkZIYrKuQq9qL3XklAIAHhnVWeDTnTu2Z9Sq3RmrWGazrjiiKcGumdSM3RSIiUpK6Z4kodeRUBXyiv+Y7LcGq9HDOWZzVX7qj9pp1m8ozm3azVAbDYF0vXLXaIKo9WGc3GCIiZal7lohSuScrAADtU+wQBPW2FDwbtWfWKzXSDcZes8CUmXX9cHtrB+vqfv/ZmFknIlIUg3UVyj1ZBgBonxyr8EiaR+0165Wa6QYj9VlX53mkppM6qwgCYDaq+wN5YFMkflgkIlICg3UVkjLrHVK0HazLmXW1BuvSAtOazLVasc+6/gQ2RDKo/tszlsEQESmLwboKyZl1rQfrUp91lZbBaKXPOncw1R+5E4zKS2AA9lknIlIag3UVOijXrOsjWC9zeRQeScM0s4Mpg3Xd0crupQAz60RESlP/TBFlKtzVyC+pAqD9Mpj4mjKYKo8PHq/6snJSDbj6N0WSziODJb2Qy2BUvnspUGuBKWvWiYgUoehMkZ2dDUEQ6lycTqd8uyiKyM7ORnp6OmJiYjB48GDs3LmzzmO4XC5MnToVKSkpiI2NxfXXX4+jR49G+qWEjJRVb2E3o4XdovBomifWGqgFL1dh3XplTXaTZTAUaVoqg7Ga2A2GiEhJiqd1zj//fOTl5cmX7du3y7fNnz8fCxYswKJFi7B582Y4nU4MGzYMpaWl8jHTp0/HypUrkZOTg/Xr16OsrAwjR46E16vNwOZgYTkAIFPjWXUAMBsN8lfoaqxbr9RIZl3eFMnjhc8nKjwaCgWXRjZEAgLZf36zQ0SkDMVnCpPJBKfTKV9SU1MB+LPqL7zwAubMmYNRo0ahe/fuePPNN1FRUYHly5cDAIqLi7FkyRI899xzGDp0KHr37o1ly5Zh+/btWLNmjZIv65zlnqwJ1jXetlEibYykxo4w2umzHhgfSxH0weUJdINRO7l1I4N1IiJFKD5T7Nu3D+np6cjMzMTtt9+OAwcOAAByc3ORn5+P4cOHy8darVYMGjQIGzZsAABs2bIFHo+nzjHp6eno3r27fExDXC4XSkpK6lzUQgrWtb64VBKv4vaNWtnB1FarVIKlMPqgpTKYwAJTlsEQESlB0WC9f//+eOutt/DFF1/gtddeQ35+PgYOHIjCwkLk5+cDANLS0urcJy0tTb4tPz8fFosFiYmJjR7TkHnz5sHhcMiXjIyMEL+yc3fwpH7KYIBaHWFUWQajjcy6wSDIARN7reuDHKxraIGpi9/qEBEpQtGZ4uqrr8bNN9+MHj16YOjQofjkk08AAG+++aZ8zO83DBFF8aybiJztmNmzZ6O4uFi+HDlypBmvIrRydRasx1r9E32pCjPr8qZIKs+sA4GOMMys64MU+FqM2gnWmVknIlKGqmaK2NhY9OjRA/v27ZO7wvw+Q15QUCBn251OJ9xuN4qKiho9piFWqxUJCQl1LmpQUuVBYbkbgH7KYOSadRVn1tW+wBQIfKCQ2k2Stsl91jXwQZF91omIlKWqYN3lcmH37t1o1aoVMjMz4XQ6sXr1avl2t9uNdevWYeDAgQCAPn36wGw21zkmLy8PO3bskI/REqkEJjXeKpePaF2gZl19GyNVamQHUyBQqlPJgEkXtNQNRlozUe0TUa3C/RKIiPRO0Yhw5syZuO6669C2bVsUFBTgySefRElJCcaNGwdBEDB9+nTMnTsXWVlZyMrKwty5c2G32zFmzBgAgMPhwMSJEzFjxgwkJycjKSkJM2fOlMtqtEZvnWCA2ruYqi/IDHSDUf8HIzlYZxmMLri1FKyba3cj8iFOA6U7RER6omiUcvToUdxxxx04efIkUlNTcfHFF2Pjxo1o164dAGDWrFmorKzE5MmTUVRUhP79+2PVqlWIj4+XH+P555+HyWTC6NGjUVlZiSFDhmDp0qUwGtWfLf09vdWrA0CcTZ0LTN3VPlTX9CzXQmY9hhsj6Yq8g6kGusHU/kBR5fHq5ls/IiKtUPS3bk5OzhlvFwQB2dnZyM7ObvQYm82GhQsXYuHChSEeXeQd1FnbRqB2Zl1dZTC1y0m0ULMuZf+ZWdcHLXWDMRgEWEwGuKt9rFsnIlKA+meKKBLIrNsVHknoqLXPuhT0Gg0CzMYzdxdSAy4w1ZdAZl0bv4JtJvZaJyJSijZmiiggimKtYD1O4dGEjpRZL1VZGYxcr242nrUVqBrIZTDMbOqC3A1GA2UwQO32jXz/ERFFGoN1lSiq8KCkJqBtl6yfzHqgDEZdwbqUobZpoAQG4AJTvdFSNxiAGyMRESlJGzNFFMg9WQYASHfY6nRf0Dq1LjCt8mhj91JJDIN1XZHLYDRQsw7U7rXOMhgiokjTxkwRBXJPVgAAMlP1s7gUAOKlTZFUl1nXTo91ALCba3YwZRmCLgQy69p4/7EMhohIOQzWVULKrLfXUY91QL2ZdS3tXgqwDEZvAjXr2vgVLG2MxMw6EVHkaWOmiAIHpcy6jto2ArVq1t3V8NX0NVeDSo2WwbAbjD5orRuMVS6D4YdFIqJI08ZMEQX0uCESEGjdKIrqKuGo1FgZTKB1o3rOIZ27QJ91bbz/5DIYLjAlIoo4BusqIIoiDhbqb0MkwJ85NBn8rRHVVAoj16xbtLEbI8tg9EWr3WBYBkNEFHnamCl0rqDUhQq3F0aDgIxE/bRtBPy70Mp16yraxVQqg4nRSDeOQBkMg3U9kMpgLFoJ1k0sgyEiUoo2Zgqdk0pg2iTGaGbybgo1bowkZajtmsms+8dZyWBJFzS3wFTus87MOhFRpAUdqUyYMCGo415//fVzHky0koJ1vXWCkahxYyQp6NVKT3uWweiL9lo3+j9UuPhhkYgo4oIO1pcuXYp27dqhd+/eEEX1dPXQg4M6XVwqiVdh+8YKN7vBkHLcmq1ZZ7BORBRpQQfr99xzD3JycnDgwAFMmDABY8eORVJSUjjHFjX02glGIpfBqCizXuXRVjcYObPOYEnzRFHU4A6mXGBKRKSUoGeKl19+GXl5eXjooYfw0UcfISMjA6NHj8YXX3zBTHsz6T5Yt9XsYqqqzLp/LJrZFKlmB1OPV4THy4BJy6p9IqQtB7RSBiN9A8DWjUREkdektI7VasUdd9yB1atXY9euXTj//PMxefJktGvXDmVlZeEao655fSIOndLnhkgSddas+wNerWTWbZbAf1V2hNG22os0WQZDRERnc84zhSAIEAQBoijC52Om71wdP10Jd7UPFqMB6S1ilB5OWMg162oK1msy61qpWbcYDTDW9KvnIlNtq71IU3vBOn/XExFFWpNmCpfLhX/9618YNmwYunTpgu3bt2PRokU4fPgw4uLiwjVGXZM2Q2qbbJeDMb1RZetGqRuMRoJ1QRBgN3ORqR5ImXWLyQBB0Mb/eakbDDPrRESRF/QC08mTJyMnJwdt27bF//3f/yEnJwfJycnhHFtUOKjzto2AOstg5G4wGimDAfz19aWuapbBaJzWdi8FAFtNbX0V+6wTEUVc0MH6K6+8grZt2yIzMxPr1q3DunXrGjzuvffeC9ngosEBeXGpvnYurU3ewbRKPTuYVtUEvFpZYAoESnaY3dQ2uROMRhaXArU2ReJ7j4go4oIO1v/4xz9q5itbLQn0WNdvGVG8GjPrHm31WQeAmJpdTJlZ1zat7V4KsAyGiEhJTdoUiULvYKG/E0z7KMisq6pm3a2tHUyBwAcLBuvaJpfBaKTHOsAFpkREStLObKFDHq8Ph3XethFQX8261yfKAZPdEvTnVcVJbSYrPeo4j3RutFkGwz7rRERKCTpSueKKKxosg3E4HOjSpQumTJmCjIyMkA5O744WVcLrExFjNiIt3qb0cMJGba0ba3+Vr5U+60Cgvp6ZdW2TymAsGiqDkT5YsAyGiCjygg7WL7jgggavP336ND799FMsWrQI69evb/Q4qi/3pH8jqXbJdhh02rYRAOKsgR1MRVFUfO1D7WDXpqFSBKkMhn3WtU2T3WBqlcGo4f8wEVE0CTpYf/755894+5QpU/DII4/g008/bfagokXuSX8JTIdU/ZbAAIGa9eqa8hOl68SlYDfGbNRU0MGadX0IlMFoKVgPjFUN/4eJiKJJyGaLSZMmYevWraF6uKgQDT3WAX8vcykmVkMpTKUGO8EAQIzZ/6GnkqUImuaWM+vaef/VDs5dXGRKRBRRIQvWY2JiUFVVFaqHiwq5UrCu48WlAGAwCIizSL3WlQ/WpR1AtZYdZBmMPmixG4zZaJB3WOYiUyKiyArZbLFq1Sp07tw5VA8XFaRgvYPOg3Wg1sZIzKyfs8ACU+XPIZ07LZbBAIDNxF7rRERKCLpm/cMPP2zw+uLiYmzevBlLlixhL/YmqPJ4cby4EoD+M+sAEGtVT6/1Sg3uXgoEOtewZl3bApsiaev9ZzMbUe72stc6EVGEBR2s33jjjQ1eHx8fj65du2Lp0qW49dZbQzUu3Tt8qgKi6N/dMznWovRwwk5NvdalzLqW2jYCLIPRCy12gwFqd4Th+4+IKJKCDtZ9PmZTQkkqgclMjdVUR5JzFei17lF4JIHMtOYy6+wGowtyGYyGataBwHgZrBMRRZa2ZgsdiZZOMBI5s66CMpgqjdasS7utVjBY0jSXBrvBAIBN2hipmokbIqJICjpY//777/HZZ5/Vue6tt95CZmYmWrZsibvvvhsulyvkA9QrObMeBfXqQCBYL1VBGYyUmdZuNxjlzyGdu0DNurZyJTZm1omIFBH0bJGdnY2ff/5Z/nn79u2YOHEihg4diocffhgfffQR5s2bF5ZB6lHUBes29WTWpZpvrWXWpTIY9lnXNq12g5G+CWCwTkQUWUHPFtu2bcOQIUPkn3NyctC/f3+89tprePDBB/HSSy/h3//+d1gGqUfR0mNdEs8Fps3GBab6EOizrq33n5RZ56ZIRESRFXSwXlRUhLS0NPnndevW4aqrrpJ/7tevH44cORLa0elUuasaBaX+kqHMaKlZV2FmPcYS9PpqVbDX7GDKBabapvluMNwUiYgoooKeLdLS0pCbmwsAcLvd+PHHHzFgwAD59tLSUpjN5tCPUIcOFvqz6kmxFjjs0XHO4qz+16mmmnWtZdZtFv9/10qPF6IoKjwaOldaLYNh60YiImUEPVtcddVVePjhh/Htt99i9uzZsNvtuOyyy+Tbf/75Z3Ts2PGcBzJv3jwIgoDp06fL14miiOzsbKSnpyMmJgaDBw/Gzp0769zP5XJh6tSpSElJQWxsLK6//nocPXr0nMcRCXIJTLJd4ZFEjpoy61rvBiOK4MY0Gqb9BaZ87xERRVLQs8WTTz4Jo9GIQYMG4bXXXsNrr70GiyWwmc/rr7+O4cOHn9MgNm/ejFdffRU9e/asc/38+fOxYMECLFq0CJs3b4bT6cSwYcNQWloqHzN9+nSsXLkSOTk5WL9+PcrKyjBy5Eh4verN/hyUF5fGKTySyFFTzXpFTTcVrWXWa4+3gh1hNEurrRu5wJSISBlBF+2mpqbi22+/RXFxMeLi4mA01p1o/vOf/yAurunBZ1lZGe6880689tprePLJJ+XrRVHECy+8gDlz5mDUqFEAgDfffBNpaWlYvnw5Jk2ahOLiYixZsgRvv/02hg4dCgBYtmwZMjIysGbNGowYMaLJ44mE3JMVAIDMlCjMrKsgWJcXmGoss240CLCaDHBV+1Dh9iJZ6QHROXFrvWadmXUioohq8mzhcDjqBeoAkJSUVCfTHqwpU6bg2muvlYNtSW5uLvLz8+tk661WKwYNGoQNGzYAALZs2QKPx1PnmPT0dHTv3l0+piEulwslJSV1LpGUe7IMQHRl1uU+6yoog6nUaM06UKsjDLObmqXVHUzlMhguMCUiiihFZ4ucnBz8+OOPDfZnz8/PB4A6HWikn6Xb8vPzYbFYkJiY2OgxDZk3bx4cDod8ycjIaO5LaZKDhf7MevtoyqzLZTAehUcSCHS1VrMOBOrW2b5Ru7RaBsMFpkREylAsWD9y5AimTZuGZcuWwWazNXqcIAh1fhZFsd51v3e2Y2bPno3i4mL5EsmWk8UVHpwqdwMA2kdJ20YAiK8pg6ny+ODxKvs1uryDqQaDdal0h+0btUuzrRtN7LNORKQExWaLLVu2oKCgAH369IHJZILJZMK6devw0ksvwWQyyRn132fICwoK5NucTifcbjeKiooaPaYhVqsVCQkJdS6RcviUP6ueGm9FrFVbfb6bo/ZrLVe4bl2r3WCAQOlOpUf5ciI6Ny6P1LpRW+8/ZtaJiJShWLA+ZMgQbN++Hdu2bZMvffv2xZ133olt27ahQ4cOcDqdWL16tXwft9uNdevWYeDAgQCAPn36wGw21zkmLy8PO3bskI9Rm+JKfxlIcmzT6/u1zGw0yDWvSteta7XPOsDMuh4EdjDVWGadmyIRESki6NTuW2+91eD1DocDXbp0QdeuXZv0xPHx8ejevXud62JjY5GcnCxfP336dMydOxdZWVnIysrC3LlzYbfbMWbMGPm5J06ciBkzZiA5ORlJSUmYOXMmevToUW/BqlpINdvRlFWXxFnNqPK4FO0II4qiZrvBAIFvAxisa1O114dqn39DK82VwbDPOhGRIoKOGKdNm9bg9WVlZfD5fLjmmmuwfPlyxMfHh2xws2bNQmVlJSZPnoyioiL0798fq1atqvMczz//PEwmE0aPHo3KykoMGTIES5cubbBjjRpIWeW4KAzW420mnCxTNlh3Vfsgbf6pxcy63A2GwbomuWut19BaGYyVZTBERIoIOmL8fV24xOfzYcuWLbjrrrvw+OOP49lnnz3nwaxdu7bOz4IgIDs7G9nZ2Y3ex2azYeHChVi4cOE5P28kSYGq1Hc8msgdYRQsg6kd5GoxWI8x+88hM+vaVHtxpkVrmXVuikREpIhmzxYGgwH9+vXDc889h48++igUY9I1aXFlfBRm1uVe6wpm1itqAg2L0QCTUVvBEsA+61on1aubjQKMhjN3tVIblsEQESkjZNFKp06dcPTo0VA9nG5JgWo0lsHIu5iqILOuxXp1oHYZDLvBaJG8IZLGSmCAwAJTFxeYEhFFVMiC9V9//RVt2rQJ1cPplhSoRmMZTLwKNkbS8u6lALvBaJ1We6wDtVs3MrNORBRJzY4YRVHE1q1bMWPGDFx33XWhGJOulTGzrmxmXcM91oFafdYZrGuSVLOuzWBdKoPhe4+IKJKCjhgTExMb3BW0rKwMXq8XV1111RkXgpKfFKjGR2FmXRU16zXlIzaNZtbZulHbpBISrS0uBQILTKt9Iqq9Pk2u+SAi0qKgI8YXXnihwesTEhLQtWtXdOvWLVRj0rVAzbpZ4ZFEnhoy61revRQAYiw13WCY3dQkt1wGo733X+0PuFXVPsQxWCciioigg/Vx48aFcxxRQwpUY63am6ybS6pZL1dwcWQFF5iSgrS6eylQt3SnyuONylI+IiIlBD1jiKKIZ555BpdccgkuuugiPPLII6iqqgrn2HRJqlmPyjKYmtdcqoKadS4wJSUEusFoL1g3GAS5fEf60EFEROEX9Izx9NNP4+GHH0ZsbCxatWqFBQsW4P777w/n2HSpLJrLYGpes5I7mGq+daOZfda1zKXhMhgAsJm4yJSIKNKCDtaXLl2KhQsXYtWqVfjggw/w/vvv46233oIo7d1OQeEOpuros67VmnV7Tc06u8Fok5a7wQC12zfy/UdEFClBzxiHDh3CyJEj5Z9HjBgBURRx/PjxsAxMj1zVXnmBWTTWe0qlP0pm1qWFmVrtBhNj8f+XZRmMNsllMBqsWQfYa52ISAlBzxhutxsxMTHyz4IgwGKxwOVyhWVgelTuCgRY0RisM7PefDHMrGua5stgaj5kuJhZJyKKmCZFjH/9619ht9vln91uN5566ik4HA75ugULFoRudDojBal2ixFGQ/2e9Xont250V8PnE2FQ4BxofQdTqWbd7fWx17UGaXkHU6BWZr2awToRUaQEHaxffvnl2LNnT53rBg4ciAMHDsg/N7RpEgWUujwAojOrDgRetyj6y1GUOA9yNxiLNv8Nai+MrfB4kcBgXVOkjLRmg3UTy2CIiCIt6Ihl7dq1YRxGdJAy69G4uBTwBygmg4Bqn4iyqmpFgvUKjWfWrSYDDALgE/3fEiTYoq+rkJYF+qxr9P1nZjcYIqJIa1J6p6SkBD5f/YyKz+dDSUlJyAalV3KP9SjNrAuCECiFqfmWIdK0voOpIAjsCKNhuimDYWadiChigp4xVq5cib59+za4EVJVVRX69euHjz76KKSD0xspWI+N0mAdCJTCKLUxUkXNzp9a7QYDcGMkLdPypkgAWzcSESkh6Blj8eLFmDVrVp0FphK73Y6HHnoIixYtCung9EYKUKO1Zh2o1RFGofaNlTUZQa1m1oFACU+lR7muOnRuAn3Wtfn+kzdF4gJTIqKICTpY37FjBwYPHtzo7Zdffjm2b98eijHpVjRviCSRe60rlFmvdAc68miVnZl1zZLKYCyaz6yzDIaIKFKCnjGKiopQXd14gOXxeFBUVBSSQelVeZTXrAO1ymAUy6xre1MkgGUwWqb9mnX2WSciirSgZ4z27dvjhx9+aPT2H374Ae3atQvJoPSqNMq7wQBAXE33EqUy6xUa3xQJCIydC0y1Rz87mPK9R0QUKUHPGKNGjcKcOXNw4sSJerfl5+fjL3/5C26++eaQDk5v5DIYa/S221O6Zr1K7rOu3WA9xuw/h8ysa4/2dzBlGQwRUaQFneJ9+OGH8cEHHyArKwtjx45Fly5dIAgCdu/ejXfeeQcZGRl4+OGHwzlWzYv2PutArZp1BYJ1j9cHj1cEANjN2v03CNSsc4Gp1mi9DMbKBaZERBEXdMQSHx+P//3vf5g9ezZWrFgh16cnJiZi7NixmDt3LuLj48M2UD2I9j7rgLKtGytrfXVvs2gzWAICwTpLEbQnsIOp1jPrfO8REUVKk6JGh8OBl19+GX//+99x8uRJiKKI1NRUCIIQrvHpSqmLrRuVLIORaryNBgEWo3aDdS4w1S63vIOpNt9/LIMhIoq8c4oaBUFAamoqTp48iU8//RRerxf9+vVDq1atQj0+XSmr8u/aGdWbIsmtGyO/g6kUrMeYjZr+gCn1WWewrj1aL4ORusEws05EFDnnHDW+++67mDhxIjp37gyPx4M9e/bg73//O/7v//4vlOPTFbkMJppr1hXMrEvBrZYXlwLsBqNlgR1MtfketNWMu6qamXUiokgJOr1TVlZW5+fHH38cmzZtwqZNm7B161b85z//wZw5c0I+QD0p4w6mcmZdyZr1GA33WAeAGEtNNxhmNzUnsIOpVjPr/v877LNORBQ5Qc8Yffr0wQcffCD/bDKZUFBQIP984sQJWCyW0I5OR3w+EeU1mdBo7gajhpp1LfdYB2pn1tkNRmtcGq9Zt7IMhogo4oKOGr/44gtMnjwZS5cuxd///ne8+OKLuO222+D1elFdXQ2DwYClS5eGcajaVl4rsIrmzLqSrRv1sHspULt1IwMmLfH5RLi9Gu+zbuICUyKiSAs6amzfvj0+/fRTLF++HIMGDcK0adOwf/9+7N+/H16vF127doXNZgvnWDVNCk7NRkGzX4GHgrQhVFlVNURRjOhCT6kvudYz61xgqk1SoA5ouQyGfdaJiCKtyTPGmDFj5Dr1wYMHw+fz4YILLmCgfha169W13ImkuaQSoGqfKJcEREqVTmrW7TU16yxF0BaXRw/BOvusExFFWpPqMT777DPs2rULvXr1wpIlS7B27VqMGTMG11xzDZ544gnExMSEa5yaJ/dYj+J6dQCwm40QBEAU/YtMI1mSopduMDE1Gzoxs64tUicYo0GASaN9/gM1676IfzNGRBStgp4xZs2ahfHjx2Pz5s2YNGkS/t//+38YPHgwtm7dCqvVigsuuACfffZZOMeqaYHMulnhkSjLYBAQZ1Gmbl033WDMNd1gGKxrivRNkpY35Kr94TrS34wREUWroGeN119/HZ9++ilycnKwefNmvP322wAAi8WCJ598Eu+99x6eeuqpsA1U68rk3Uu1HSiGQmBjpAgH6+wGQwrSeicYILDAFKhb1kNEROET9Kxht9uRm5sLADhy5Ei9GvXzzz8f69evD+3odIQ91gOkc1DqiuwuplKwbtNJsF7h8UIURYVHQ8EKbIik3WDdbBRgqKl84SJTIqLICHrWmDdvHv74xz8iPT0dgwYNwv/7f/8vnOPSnUDNenSXwQCBzHq5K7KTvbSJkN2s7Q9MUs29KLIUQUvkzLpG2zYCgCAIXGRKRBRhQUctd955J6666iocOHAAWVlZaNGiRRiHpT/lLmbWJYGNkSKbWa+SF5hqN7MJBLrBAP66da33jY8WWt+9VGIzG1Hh9rLXOhFRhDRp1khOTka/fv1CFqgvXrwYPXv2REJCAhISEjBgwIA6i1RFUUR2djbS09MRExODwYMHY+fOnXUew+VyYerUqUhJSUFsbCyuv/56HD16NCTjCyWpZj0+yrvBALU2RopwzXqgG4y2/w2MBgGWmoCvktlNzZDLYDRcsw4ANhN3MSUiiiRFZ402bdrg6aefxg8//IAffvgBV155JW644QY5IJ8/fz4WLFiARYsWYfPmzXA6nRg2bBhKS0vlx5g+fTpWrlyJnJwcrF+/HmVlZRg5ciS8XnVNJKWsWZcFatbZDeZccZGp9uihDAZgr3UiokhTNFi/7rrrcM0116Bz587o3LkznnrqKcTFxWHjxo0QRREvvPAC5syZg1GjRqF79+548803UVFRgeXLlwMAiouLsWTJEjz33HMYOnQoevfujWXLlmH79u1Ys2aNki+tnjKWwchq72IaSXrpBgNwF1MtCgTr2s6sW6VgnesliIgiQjWzhtfrRU5ODsrLyzFgwADk5uYiPz8fw4cPl4+xWq0YNGgQNmzYAADYsmULPB5PnWPS09PRvXt3+ZiGuFwulJSU1LmEW1mVvz472jdFAmq1bmRm/ZxJi0wZrGuHy6P9bjAAYDOzDIaIKJIUnzW2b9+OuLg4WK1W3HPPPVi5ciXOO+885OfnAwDS0tLqHJ+Wlibflp+fD4vFgsTExEaPaci8efPgcDjkS0ZGRohfVX1yzToz6/I5iHzNuv/5tL6DKVC7DIYBk1bopgzGxDIYIqJIUjxY79KlC7Zt24aNGzfi3nvvxbhx47Br1y759t9vZx3MFtdnO2b27NkoLi6WL0eOHGneiwiCVLMey2BdzqxHumZd6l6hh8y6nbuYao4eNkUCApl1bopERBQZis8aFosFnTp1Qt++fTFv3jz06tULL774IpxOJwDUy5AXFBTI2Xan0wm3242ioqJGj2mI1WqVO9BIl3CTa9ZZBhNo3ahQZl0XNetyGQwXmGqFHjZFAmotMOWmSEREEaG6WUMURbhcLmRmZsLpdGL16tXybW63G+vWrcPAgQMBAH369IHZbK5zTF5eHnbs2CEfoxYsgwlQumZdD33J5TIYliJoRqDPurbff+wGQ0QUWYpGjo888giuvvpqZGRkoLS0FDk5OVi7di0+//xzCIKA6dOnY+7cucjKykJWVhbmzp0Lu92OMWPGAAAcDgcmTpyIGTNmIDk5GUlJSZg5cyZ69OiBoUOHKvnS6hBFMbApEjPrgZr1CAbrPp8ol8HoKbPOmnXt0Es3mMACU5bBEBFFgqKR44kTJ/CHP/wBeXl5cDgc6NmzJz7//HMMGzYMADBr1ixUVlZi8uTJKCoqQv/+/bFq1SrEx8fLj/H888/DZDJh9OjRqKysxJAhQ7B06VIYjeoJyFzVPni8IgC2bgRq1axHsAym9lf2elhgytaN2qOXTZGsXGBKRBRRikaOS5YsOePtgiAgOzsb2dnZjR5js9mwcOFCLFy4MMSjC53aGeRYje+eGQpyzbrLE7HnrB3U2jRehgCwDEaL3DWZdYuKEgnnIlAGw8w6EVEkaDvFoxFltXYvNRjO3MkmGsTXbIpU5fHJAUy4SeUiNrNBF/8GMRapGwwXmGqF3rrBcIEpEVFkaHvW0AjuXlpXvM0Ei9H/1isorYrIc0oZaLtOvtmwc1MkzdFPzTrLYIiIIknbs4ZGSLXZXFzqZzAIaNXCBgA4fjpCwbpbP7uXAtwUSYsCO5hq+z1oM7HPOhFRJDFYjwAps84NkQJat4gBABw7XRGR55My0HpYXApwgakWMbNORETnQtuzhkZICynZYz1ADtaLKiPyfFJgoZ/Muv+9xAWm2qGXbjDcFImIKLK0PWtoRO0FpuSXLmfWI1MGo7fMOstgtCeQWdf2e1BaYMoyGCKiyGCwHgFlLn9AxZr1gNaJUrAemcx6pc4y6za5DIbdYLQisIOptn/tWplZJyKKKG3PGhohlcEwsx7QRi6DiUzNemVNUKuH3UsBZta1SC6D0XiwbjOxzzoRUSRpe9bQCKkMJp6ZdZlUBnP8dBVEUQz78+ktsy63bmTNumYE+qxr+z0o91nne4+IKCIYrEdAKfus1yO1bqz0eFFUEf6dTPVWsx7DPuuao79uMMysExFFgrZnDY0oY5/1eqwmI1rGWwFEpiOM/jLr/veSu9oHry/830xQ8wX6rGv7164UrLuYWSciightzxoawR1MGxboCBOBYN0t7WCql2A98Dq4yFQbdFcGwwWmREQRwWA9AhisNyySHWGkchGbToJ1q8kAQfD/nb3W1U8URf2UwdQsMPV4RX6rQ0QUAdqeNTSCfdYb1iaCGyNJAa1d41lNiSAIckkPO8Kon8cbCGotWg/Wa/0f4iJTIqLw0/asoRHyAlPWrNcR6AgTuTIYvSwwBWp1hGGwrnquWiUjWs+s1x4/g3UiovDT9qyhEXLrRqtZ4ZGoS2sFatZjLPr5wMSOMNohlcAAgMWo7V+7BoMgfztQVc2OMERE4abtWUMDvD5RLsFgZr2uiNas66wbDADYzf73E8tg1K92vbogLTbQMJuJvdaJiCKFwXqYSYtLASDWqp9AMRSkMphT5e6wB5xVOusGA9TOrLMbjNrppW2jJNBrncE6EVG46WPmUDEpWLeYDLCa9BMohoIjxoz4mkW34c6uV3j8/w42PWXWa4J1doNRP720bZRwYyQioshhsB5mgXp1lsA0JFK91ivd/qBCT5l1LjDVDr20bZRIvda5MRIRUfjpY+ZQsTKXBwDr1Rsj1a2HuyNMZU2piJ5q1m1s3agZui2D4cZIRERhp4+ZQ8VK2WP9jFpHoNe6KAYW+eoxs84yGPULZNb18f6TNkZiGQwRUfgxWA8zqWY9lsF6gyJRBuOq9kHaaFEvO5gCgL2mDSUXmKpfoGZdH79ypdfBBaZEROGnj5lDxVizfmaRaN9YO6DQUxkM+6xrh7Qpkl7KYKzMrBMRRYw+Zg4VK+PupWcUiTKY8ppg1mwUYNb4hjS12Vmzrhkuj87KYJhZJyKKGP1ELiolB+vMrDdICtbzS6pQ7Q1Plu5QYTmAQMmNXjCzrh366wbDBaZERJGij5lDxaQyGGbWG9Yy3gqzUYDXJ6Kg1BWW59ibXwoAyGoZH5bHV0qgZp0Bk9rJZTA6KcMKZNZZBkNEFG4M1sNMyqyzZr1hBoOAVo7w1q3vOVEGAOjijAvL4ytF6gbDUgT1c+sts15TzsM+60RE4aePmUPFSlkGc1bpLWwAwle3vveEP7PeOU1fmXWpFIHdYNRPKoOx6CVYN/ODIhFRpOhj5lCxQBmMWeGRqFfrFnYA4cmsi6IoB+tdnPoK1rmDqXborRsMy2CIiCJHHzOHinGB6dmFs31jfkkVSquqYTQIyEyJDfnjK4mbImmH/rrBcIEpEVGkMFgPszLuYHpWrcNYBrOnZnFpZkqsbgIlCbvBaIfeusFYWQZDRBQx+pg5VIx91s9OKoM5HobM+j5pcanO6tWBQDcY9llXv0A3GH38yrWZWAZDRBQp+pg5VKy0ygOAmfUzqV0GI4piSB97j04XlwK1a9arQ37eKLQCmXV9fLvDBaZERJHDYD2MRFGUd8+MZ2a9Ua0c/jKYCrcXpys8IX3sQCcYfbVtBAJlMD4xEAySOgVq1vXxKzdQs873HRFRuOlj5lCpKo8PXp8/48nMeuNsZiNS4qwAQrvI1OcT5TKYzjrrBAMAMbU22GGGU9302g2GfdaJiMJPHzOHSpW6/FliQQiULFDDwtER5mhRJSo9XlhMBrRLsofscdXCbDTAbBQAcJGp2sllMLrZwZRlMEREkcJgPYxqd4IRBEHh0ahbODrCSPXqnVLjYDLq860eY2ZHGC3QWzcYaQdTLjAlIgo/fcwcKiV1golnCcxZtW7hz6yHsiOMnuvVJewIow16LYNhn3UiovBTdOaYN28e+vXrh/j4eLRs2RI33ngj9uzZU+cYURSRnZ2N9PR0xMTEYPDgwdi5c2edY1wuF6ZOnYqUlBTExsbi+uuvx9GjRyP5UhoU2L2UwfrZSMF6KMtgpB7reqxXl9TuCEPqpdtNkVgGQ0QUdooG6+vWrcOUKVOwceNGrF69GtXV1Rg+fDjKy8vlY+bPn48FCxZg0aJF2Lx5M5xOJ4YNG4bS0lL5mOnTp2PlypXIycnB+vXrUVZWhpEjR8LrVXYiKa3JrMcys35W6WHMrOuxx7pE3hiJQZOqBWrW9ZFZl15HlcfHtqFERGGmaBT5+eef1/n5jTfeQMuWLbFlyxZcfvnlEEURL7zwAubMmYNRo0YBAN58802kpaVh+fLlmDRpEoqLi7FkyRK8/fbbGDp0KABg2bJlyMjIwJo1azBixIh6z+tyueByueSfS0pKwvL6uHtp8EK9wNTj9eHAb/4PfXrssS6RMussg1E3t95q1mstlHVV++r8TEREoaWqmaO4uBgAkJSUBADIzc1Ffn4+hg8fLh9jtVoxaNAgbNiwAQCwZcsWeDyeOsekp6eje/fu8jG/N2/ePDgcDvmSkZERltcj16yzDOas2tTsYnqyzB2Sr9YPFZbD7fXBbjHKJTZ6FFNTs84FpuoWqFnXR1Brq/U6XFxkSkQUVqoJ1kVRxIMPPohLL70U3bt3BwDk5+cDANLS0uocm5aWJt+Wn58Pi8WCxMTERo/5vdmzZ6O4uFi+HDlyJNQvB0AgWGdm/ewSYkyIrckSh6IUZm9Nf/WstHgYDPrtxBNTU45QyTIYVdNbNxizUYD034qLTImIwks1UeR9992Hn3/+GevXr6932+/bHoqieNZWiGc6xmq1wmq1nvtggxQI1s1hfy6tEwQBrRNjsPdEGY6drkSH1OZ1cJEWl3bRcScYoHY3GC4wVTO9BeuCIMBmNqLC7eUiUyKiMFPFzDF16lR8+OGH+Prrr9GmTRv5eqfTCQD1MuQFBQVytt3pdMLtdqOoqKjRY5TCbjBNI3eECUGv9UDbRv3WqwO1FpiyDEa1qr2BnYz1UgYD1O4IwzIYIqJwUjRYF0UR9913H9577z189dVXyMzMrHN7ZmYmnE4nVq9eLV/ndruxbt06DBw4EADQp08fmM3mOsfk5eVhx44d8jFKYZ/1pgllR5g9URKs281cYKp2UlYd0E83GACwmaSOMHzvERGFk6JR5JQpU7B8+XJ88MEHiI+PlzPoDocDMTExEAQB06dPx9y5c5GVlYWsrCzMnTsXdrsdY8aMkY+dOHEiZsyYgeTkZCQlJWHmzJno0aOH3B1GKaXMrDeJ1BHmaDOD9SqPF4cKKwAAXXTcYx2o3WedAZNa1Q7WLTraSZe91omIIkPRKHLx4sUAgMGDB9e5/o033sD48eMBALNmzUJlZSUmT56MoqIi9O/fH6tWrUJ8fCAIe/7552EymTB69GhUVlZiyJAhWLp0KYxGZb9yLnN5AHCBabBCVQZz4LdyeH0iHDFmtIwP/9oEJbEbjPpJnWAsRoOuFjtbpWC9mmUwREThpGgUGcxmGoIgIDs7G9nZ2Y0eY7PZsHDhQixcuDCEo2s+doNpGilYP17cvGC99mZIZ1uIrHVyn3UPF5iqVWD3Uv1k1QHAZmYZDBFRJDCKDCMuMG0aqQwm73QVvD4RxnPMQkr16lk67wQDADGsWVc9ve1eKpF6rTNYJwoNr9cLj8ej9DAoQiwWCwyG4OYFRpFhxMx607SMt8FkEFDtE1FQWoVWjnPbzGiflFnXeb06wG4wWqC3DZEkUmadmyIRNY8oisjPz8fp06eVHgpFkMFgQGZmJiwWy1mPZRQZRgzWm8ZoEOB02HC0qBLHT1eec7AeLZ1ggNplMAzW1UpvPdYl0gJTFzdFImoWKVBv2bIl7Ha77ss3CfD5fDh+/Djy8vLQtm3bs/6bM4oME4/XJ/cfjmcZTNBat4jB0aJKHC2qRJ92Tb9/uasaR075a96jIVhnZl39pMyzRafBOvusE507r9crB+rJyclKD4ciKDU1FcePH0d1dTXM5jNvnqmv2UNFyl2BBX+xzKwHTapbP3aO7Rv3FZQBAFLirEiKPftXS1oX2MGUwbpayWUwZn2WwbBmnejcSTXqdrtd4ZFQpEnlL17v2X+HMlgPE6nHus1sgFlHvZXDrXUzN0aSO8E49b+4FKjdZ53dYNTKrdMyGKkGv4plMETNxtKX6NOUf3N9zR4qEqhXP/NXG1RXc3ut782Pnnp1INANhmUw6qX3mnWWwRARhZe+Zg8VkYJ11qs3TXqL5pXB7KnVYz0aSJl1V7UPXt/Z9y2gyAt0g9HXr1uWwRARRYa+Zg8VkXqsx1r1VacabnLNelFlUJtm/d6+E/6a9awoCdalBaYAgya1CmTW9fW7gJl1ouhWUFCASZMmoW3btrBarXA6nRgxYgS+++67iI/l4MGDuPzyyxEXF4dBgwbh0KFDdW6/9tpr8e677zZ433fffReDBw+Gw+FAXFwcevbsiSeeeAKnTp0CACxduhSCIMiXVq1aYfTo0cjNzZUfQxAEvP/++/Uee/r06Rg8eHCzXx+D9TApZdvGcyKVwZS7vSipbFoddnGFB/klVQCAzlGwIRIQ2JgGYCmMWul2B9Oa18OadaLodPPNN+Onn37Cm2++ib179+LDDz/E4MGD5SA3kmbMmIHWrVtj69atcDqdmDlzpnxbTk4OjEYjbr755nr3mzNnDm677Tb069cPn332GXbs2IHnnnsOP/30E95++235uISEBOTl5eH48eNYvnw5tm3bhuuvvz6oxaGhoK/ZQ0Xk3UtZs94kNrMRyTVdXI6ermjSffcW+EtgWreIQbwtOs67wSBwF1OVC3SD0devW7nPOr/RIYo6p0+fxvr16/G3v/0NV1xxBdq1a4eLLroIs2fPxrXXXgvAn+0WBAHbtm2rcz9BELB27VoAwNq1ayEIAr744gv07t0bMTExuPLKK1FQUIDPPvsM3bp1Q0JCAu644w5UVDQeE+zevRvjxo1DVlYWxo8fj127dsnP95e//AWLFi2qd59NmzZh7ty5eO655/DMM89g4MCBaN++PYYNG4Z3330X48aNk48VBAFOpxOtWrXCFVdcgcceeww7duzA/v37Q3A2z45p3zApc/nbMbFmvelaJ8agsNyN46ercH66I+j77ZEXl0ZHVl1itxhR6fGiwsOOMGrEMhgiagpRFBXb6C7GbAyqS0lcXBzi4uLw/vvv4+KLL4bVam3W82ZnZ2PRokWw2+0YPXo0Ro8eDavViuXLl6OsrAw33XQTFi5ciIceeqjB+/fq1Qtr1qzB8OHDsWrVKvTs2RMAMHPmTNx3331o27Ztvfu88847iIuLw+TJkxt8zBYtWjQ63pgYfxWA1Hoz3BhJhkmZy/8fjWUwTde6RQx+PlqMY0VNy6zvi6KdS2uLsRiBcpbBqJV+u8FwgSlROFR6vDjv0S8Uee5dT4yQ9+84E5PJhKVLl+JPf/oTXnnlFVx44YUYNGgQbr/9djlQboonn3wSl1xyCQBg4sSJmD17Nn799Vd06NABAHDLLbfg66+/bjRYf/bZZzFp0iS0b98ePXv2xD/+8Q988803+OmnnzB//nyMHj0aP/zwA4YPH46XXnoJFosF+/btQ4cOHc66IdHvHT16FM888wzatGmDzp07N/m1ngt9zR4qIpfBMLPeZOfaEWZPlAbrUkcYlsGok1QmordgXdrkiTXrRNHp5ptvxvHjx/Hhhx9ixIgRWLt2LS688EIsXbq0yY9VO8BPS0uD3W6XA3XpuoKCgkbv37p1a3z88cc4fPgwPv74Y6SkpGDy5Mn4xz/+gSeffBLx8fHYs2cP9u3bh3/84x8A/N9gBNvrvLi4GHFxcYiNjUVGRgbcbjfee+89eWOjcGMkGSZSGQwz600X2BipKuj7iKIol8F0cUZXsB5TkwVhZl2d5My63nYwNbEMhigcYsxG7HpihGLP3RQ2mw3Dhg3DsGHD8Oijj+Kuu+7CY489hvHjx8Ng8Ccoand2a6xspHZ2WxCEetluQRDg8wX/u+app57C8OHDceGFF+Kuu+7Ck08+CbPZjFGjRuGrr77C1KlT0blzZ6xfvx4ej+es2fX4+Hj8+OOPMBgMSEtLQ2xsbL3bi4uL693v9OnTcDiCL+dtjL5SPSrCPuvnTmrfeLQJmfWTZW4UVXggCECnltFVsx5TU46gVI0jnRnLYIioKQRBgN1iUuTS3J1UzzvvPJSXlwMAUlNTAQB5eXny7bUXm4bL7t278a9//QtPPPEEAMDr9cofEjwej9zBZcyYMSgrK8PLL7/c4OOcPn1a/rvBYECnTp3QoUOHeoE6AHTt2hWbN2+uc50oitiyZQu6dOnS7NfESDJMSqvYuvFcncsuplK9ersku7zwLVpI9YWVbi4wVSP9borEzDpRtCosLMStt96KCRMmoGfPnoiPj8cPP/yA+fPn44YbbgDgX4R58cUX4+mnn0b79u1x8uRJ/OUvfwnruERRxN13343nn38ecXH+xN0ll1yC1157DZ07d8Zbb72FO+64AwDQv39/zJo1CzNmzMCxY8dw0003IT09Hfv378crr7yCSy+9FNOmTQvqeWfOnIlx48aha9euGD58OCorK/Hqq6/i119/xZQpU5r9uhhJhkkZ+6yfMylYP1nmQpXHG1TwHa316kBgYySWwahToM+6vj5ESh8+2LqRKPrExcWhf//+eP755/Hrr7/C4/EgIyMDf/rTn/DII4/Ix73++uuYMGEC+vbtiy5dumD+/PkYPnx42Mb16quvIi0tDSNHjpSvy87OxpgxY9C/f39cddVVdYLnv/3tb+jTpw/+/ve/45VXXoHP50PHjh1xyy231GndeDajR4+GKIp49tlnMWfOHNhsNvTu3Rvffvst2rVr1+zXJYjnsk2kzpSUlMDhcKC4uBgJCQkhecxhC9ZhX0EZlt/VHwM7pYTkMaOFKIo4/7EvUOH24uuZg5GZUv8rp9+b/d7P+NemI5h6ZSfMGN78r5y05M//+Qn/2XIUfx7RBVOu6KT0cOh3xv7ze6zffxIv3n4BbrigtdLDCZnjpysx8OmvYDYK2PfUNUoPh+ichGP+b4qqqirk5uYiMzMTNpst4s9PymnKv72+vpdVETmzzpr1JhMEIdARJshSmECP9ejLrLMbjLq5dVuz7n/febwivL6oz/kQEYWNvmYPFSljzXqzBDrCnD1YF0UR+06UAYjOYJ3dYNQtULOurzIYW60dWQvLXQqOhIhI3xish4EoiihzM7PeHE3pCJNXXIVSVzVMBiGokhm9kTPr3MFUlXTbDcZkRJua/6djXvseR041bRMzIiIKjr5mD5WocHshrQSItzZtZyzya0pHGGlxaYfUWFh0FhAFQ+qLy8y6OknBut7emwaDgH+O64tWDhv2F5Thppc34Oejp5UeFhGR7uhr9lAJqV7daBDqfFVMwWtKGcy+KO4EAwS6wbBmXZ0CO5jqqwwGALo6E7By8iXo1ioBJ8tcuO0fG7Fm1wmlh0WkOez1EX2a8m/OSDIMavdYb+4GA9FKKoM5FkSwvic/euvVgdplMAzW1Siwg6k+f906HTb8554BGNQ5FZUeL+5++we89d1BpYdFpAnSzpkVFSwjizZutxsAYDSePZHDguowYI/15pO6weQVV8LnE2EwNP6hZ2+UZ9alYP23UhcOnixHm8QYmIz6DAy1SK8167XFWU3457i+ePSDHfjXpiN49IOdOHKqArOv7nbG/7tE0c5oNKJFixYoKCgAANjtdib5ooDP58Nvv/0Gu90Ok+nssSKjyTCQOsHEc3HpOUuLt8JoEODxivitzIW0hIZ7kPp8IvYV+IP1Ls7oDNbjbf7MzC/5pRj87FpYjAa0TbajY2osOqTGoUOK/8+OqbFoYbcoPNroo9duML9nNhow96YeaJNoxzNf7MFr3+bi2OlKLBh9QdTtKkzUFE6nEwDkgJ2ig8FgQNu2bYP6cMZoMgzKXB4AQCwz6+fMZDTAmWDDsdOVWP79YXRv7UBSrBmJdguSY62It5lgMAg4UlSBKo8PVpMBbZPsSg9bEX3bJ2JM/7b48VARck+Ww1Xtw/6CMuwvKANQt344Nd6KXm1a4IIMB3pltEDP1i3gsHMRdLh4fSI8Xn9dop4z6xJBEDDlik5okxiDP//nZ3y6PR/5xRvx2h/7IjnOqvTwiFRJEAS0atUKLVu2hMfjUXo4FCEWiwUGQ3DzAqPJMChlj/WQaJdsx7HTlXjxy331bjMaBCTazXK2slPLOBij9Ot2q8mIuTf1AOD/puHY6UocOFmOA7+V4cBv5Thw0v9nXnEVfit1Yc3uE1izOxDEd0iJRa+MFujVxh/AZ6XFo6jcjRMlVcgvqUJ+cVXN3104Uey/7rdSF2KtJjgdVjgTbEhLsPn/dPj/dDr81yXYwrduo9rrQ7nLi1KXB+UuL8pcHtgtJmQk2VXzf0/aEAnQb816Q264oDWcCTbc/fYW/Hj4NG56eQP+OKAdrujaEh1SYpv8nhBFEQcLK7Dh15M4XeGB2SjAYjTAYjL6/24ywGoywGw0wGIyIN5mRlbLOCZMSFOMRmNQ9csUffibLAy4e2lozL66G9787iAKy1w4VeFBUbkbp8rdKHNVw+sTcbLMLR/bp12igiNVD4NBQEaSHRlJdgzqnFrntnJXNX7JL8G2I8X46chp/HT0NA4VVvgD+5PlWLn1WJOeq9LjxckyF3YcK2l8PIK/ZaHZ6A+mLEYDzDV/Wkz+i9lgAM4Su4miiAq3F+WuapTVXKo8vkaPT4q1ICPJjrZJdmQkxqCt9PckO1o5bBGr6ZdKYADAEmXrCPp3SMa79w7E+Dc24fCpCjz5yW48+clutE2y44ouqbiia0tc3CG50RKZgtIqbNhfiP/tP4n/7T+J48VVTXp+QQDaJ8eiW6t4dHMm4Lz0BHRrlYBWDhtrgolIUwSR/YJQUlICh8OB4uJiJCQkNPvxFn21D8+u2ovb+2Xg6Zt7hmCEVJur2ovTFR6cqgneK91eDOiYzCzaOThV7sZPR0/7g/cjp/HT0WKcKnfDYjQg7XdZcylb7nTYkBpnRZmrWs6+Sxn32tn34srIfJ1rMRkQZzUh1mpEWVU1iirO/Lwmg4D0FjFy8J6RFAjm2ybZ4YgxhyyYO1FShf5zv4TJIGD/3GtC8phac7rCjXd/PIa1ewrw/YFTcHsDH7JsZgMGdkzBFV1SMaBjCg4VlmN9TXC+t2ZXYonZKODCtolonxwLj9cHl9cHd7X/4pH+XvNnYbkbv5U2vKtqC7sZ3ZwJ6OKMr1kXE3gcl9cHT63H8Xh9sJmNSLRbkBTrvyTGWpBktyAx1ozkWCsSY81IsltC8gFQFEV+kIiwUM//ROHA6CYMStkNJqysJiPSEoyNLjql4CXFWnBFl5a4oktLAP5godRVjfgg2452b+1o9LZKtxelVR64pGCqVlBUOzCSarrPxm41Is5qqnOJtZrqbTZUWuXBkVOVOHyqAkdOVfj/LPL/efRUJdxeHw7XXN+QeKsJrRNjgqoxv7ZnK9x9ecdGb3d59N8J5mxa2C2YeGkmJl6aiXJXNTb8WoivfinA2j0FyCuuwle/FOCrX+ovrBME4LxWCbi0UwoGdkpBv/aJsFuC/516ssyF3Xkl2J1Xgl3HS7A7rxT7fyvD6QoPvjtQiO8OFIbsNZqNAjq1jEe3VvE4r1UCzmvlz+Inxja+oLuo3O0fW81ld14pfi0oQ4fUWEy4NBM3XJCu+0XJRBQcRpNhIHWDYRkMaY0gCEiwhWbBaYzFKG/YFEnxNjPOSzfjvPT6WTKfT8SJ0io5mJcCeimoLyh1odRVjV/yS4N6rh3HS3B9r9ZwOhr+4Ch3gmE3FAD+RffDzkvDsPPSIIoi9pwo9Qfuv/yGLYeL0CYxBgM7puDSTikY0DEZSWcIds8mJc6Ky7JScVlWoBysyuPF/oIy7Morwa+/+TP31tolWdLfa/1Z6fHiVLkbRRVu+du8onIPCstdKKrwoKjCDY9XlD8YvIdAOZkzweYP4NMT0C45FocKy7E7rxS7jpcgv6Thsp5f8ksx678/45kv9mDcgHa4s3+7Mwb9RKR/jCbDgH3WidTJYBDQyhGDVo4YXJSZVO/2SrcXR4sqcOx0Jby+M2f8X/pyH346WowVm49g2tCsBo+Jhh7r50oQBHR1JqCrMwGTB3c6634KoWAzG9G9teOM3wg1ldcn4vjpyppgvRS78oqxO68Uh09V1JSGVeHrPb81eN+2SfaabLwD3VrFo0NqHL7cfQJLNxxEXnEVnl21F4u+3o9b+2RgwqWZyEyJDdm4iUg7GE2GAfusE2lTjMWIrLR4ZAWxwVaZqxrTcrYhZ/NhTLmiY4M1ywzWg6fVzZOMtRZ1Dz/fKV9fWuXBL/mlcsb9UGEF2iXb0a2mRKarM17eI6G2Ti3jMOHSTHzycx5e+/YAdh4vwdsbD2HZ94cwtFsa/nRZB/Rrn8jadqIowmgyDAI16+xfTaRXV3V3IinWgrxif+Z02Hlp9Y6Jlg2RqL54mxn92iehX/v63+CcjdlowI29W+OGC9Kx8cAp/PPbA/jylwKs3nUCq3edQP/MJLw18SK+r4iiBNM9YSBl1mOt/EVKpFdWkxG39GkDAFj+/aEGj5Ey679fBEsUDEEQMKBjMpaM74c1Dw7CHRe1hdko4PvcU/j5aLHSwyOiCOEMEgZSzTrLYIj07Y6L2gIA1u79DUca6C7DbjAUKp1axmHeqB4Y2DEFALD3RHCLoIlI+xSdQb755htcd911SE9PhyAIeP/99+vcLooisrOzkZ6ejpiYGAwePBg7d+6sc4zL5cLUqVORkpKC2NhYXH/99Th69GgEX0V9ZSyDIYoKmSmxuLRTCkQRyNl8uN7tgW4wDNYpNLo4/esp9gbZsYiItE/RGaS8vBy9evXCokWLGrx9/vz5WLBgARYtWoTNmzfD6XRi2LBhKC0N/JKaPn06Vq5ciZycHKxfvx5lZWUYOXIkvF5vg48ZCdzBlCh6jOnvz66v2HwUHm/dXVUDC0xZEkeh0blm8fPvN40iIv1SNJq8+uqrcfXVVzd4myiKeOGFFzBnzhyMGjUKAPDmm28iLS0Ny5cvx6RJk1BcXIwlS5bg7bffxtChQwEAy5YtQ0ZGBtasWYMRI0ZE7LVIXNVeuGsmaLZuJNK/YeelITXeit9KXVi96wSu6dFKvo3dYCjUOqfFAWAZDFE0Ue0Mkpubi/z8fAwfPly+zmq1YtCgQdiwYQMAYMuWLfB4PHWOSU9PR/fu3eVjGuJyuVBSUlLnEirlrkBGn8E6kf6ZjQbc1jcDAPDO7xaaujxSNxjV/qoljenUMg6CABSWu3GyzKX0cIgoAlQ7g+Tn5wMA0tLqtkNLS0uTb8vPz4fFYkFiYmKjxzRk3rx5cDgc8iUjIyNk45Y6wdgtRhg12jeYiJrm9osyIAjA//YXIvdkuXw9y2Ao1OwWE9om2QEwu04ULVQbrEt+v/GDKIpn3QzibMfMnj0bxcXF8uXIkSMhGSsAlLo8AJhVJ4ombRLtGNzZv639vzYFFprKwToXmFIIZbXkIlOiaKLaGcTp9O8E9/sMeUFBgZxtdzqdcLvdKCoqavSYhlitViQkJNS5AMBvpVXNHreUWefiUqLocmf/dgCA//xwBFU15S+BTZFU+6uWNKiL01+3voeLTImigmpnkMzMTDidTqxevVq+zu12Y926dRg4cCAAoE+fPjCbzXWOycvLw44dO+RjmuLed36UO7mcq0DbRgbrRNFkcJdUtHLYUFThwRc7/UmGQJ91lsFQ6EgdYfaxDIYoKigarJeVlWHbtm3Ytm0bAP+i0m3btuHw4cMQBAHTp0/H3LlzsXLlSuzYsQPjx4+H3W7HmDFjAAAOhwMTJ07EjBkz8OWXX2Lr1q0YO3YsevToIXeHaYpf8kpx77ItcjeXc3pNDNaJopLJaMDt/fxtHN/Z6C+FYTcYCgep1/qeE6UQRVHh0RBRuCkaUf7www+44oor5J8ffPBBAMC4ceOwdOlSzJo1C5WVlZg8eTKKiorQv39/rFq1CvHx8fJ9nn/+eZhMJowePRqVlZUYMmQIli5dCqOx6Zksm9mAb/edxEPv/oznbu0FwzksEC2tYrBOFK1u65eBl77ah00HT2HviVJuikRhkZkSC6NBQGlVNfJLqtDKEaP0kIgojBSdQQYPHgxRFOtdli5dCsC/uDQ7Oxt5eXmoqqrCunXr0L179zqPYbPZsHDhQhQWFqKiogIfffTROXd3WXDbBTAaBKzcegx/++KXc3oMbohEFL2cDhuGdG0JAFj+/WH5WzqWwVAoWU1GZKbEAuDmSETRgOmeWi7PSsXTo3oAAP6x7gDe+F9ukx+jvCZYj2dmnSgq3Xmxf6Hpuz8eRXGlvzsUy2Ao1LqksSMMUbTgDPI7t/bNwJ9HdAEAPPHxLnzyc16T7l/KbjBEUe2yTinISIpBaVU1NvxaCIBlMBR6WWlSRxgG60R6xxmkAZMHd8QfLm4HUQQeWLENGw8UBn3fwAJTc7iGR0QqZjAIuOMi/0JTr8+/+M9yDmtoiM5EzqwzWCfSPQbrDRAEAdnXn48R56fB7fXhT2/9gF/yS4K6L/usE9GtfTJgNgYWqLMMhkKts1Nq31gGn48dYYj0jDNII4wGAS/e3ht92yWitKoa41/fjOOnK896vzLWrBNFvdR4K4af75R/ZhkMhVq7JDssRgMqPV4cLTr73ERE2sUZ5AxsZiP+Oa4vOrWMQ35JFca9vgklVZ4z3qe0JliPZbBOFNXu7N9W/ju7wVComYwGdGzJunWiaMBg/Sxa2C14c8JFSEuwYl9BGf6ycscZN6Eoqwnm2WedKLoN6JCM7q0TYDYKaJtkV3o4pENdahaZsm6dSN8YrAehdYsYvHznhTAaBHz403G8++OxRo+Vy2BYs04U1QRBwDt3XYyvZgyG02FTejikQ1lcZEoUFRisB6lPuyQ8MDQLAPDoBztw4LeGN6Io4w6mRFTDEWNGBrPqFCZSR5g97LVOpGsM1pvg3sGdcHGHJFS4vZj6r63yVuISn09Eudt/HbvBEBFROHWp6Qhz4LdyVHt9Co+GiMKFwXoTGA0CXritN1rYzdh5vATPfL6nzu3l7mr578ysExFROLVuEQO7xQi314eDhRVKD4eIwoTBehM5HTY8c0svAMA/1+di7Z4C+TapXt1sFNhXmYiIwspgEJDVkotMifSOEeU5GHZeGsYNaAcAmPmfn1BQWgWgbr26IAiN3p+IiCgUOnORKZHuMVg/R7Ov6YauznicLHNjxr9/gs8nyj3WWa9ORESRINWtM1gn0i8G6+fIZjZi4R29YTMb8O2+k/jn+gNyZj3WwmCdiIjCL4sdYYh0j8F6M2SlxePRkecDAOZ/vgf/+/UkAPZYJyKiyJDaNx4srKjXoYyI9IHBejPdcVEGru7uRLVPxD/WHQDATjBERBQZaQlWJNhM8PpEHPitXOnhEFEYMFhvJkEQ8PSonkivtUNhnM2s4IiIiChaCILARaZEOsdgPQQcdjNevKM3DDUNYJhZJyKiSOnsZN06kZ4xWA+Rfu2T8OcRXQEAF2Q4FB4NERFFiy5yZr1M4ZEQUTgwBRxC9w7uiDH928IRwzIYIiKKjKw0boxEpGfMrIcYA3UiIookKbN++FQFKtzVCo+GiEKNwToREZGGJcdZkRJnAQDsL2ApDJHeMFgnIiLSuKyWXGRKpFcM1omIiDSui5PtG4n0isE6ERGRxnVmRxgi3WKwTkREpHFdnOwIQ6RXDNaJiIg0rlNNzXpecRWKKz0Kj4aIQonBOhERkcY5Ysxo5bABAPYxu06kKwzWiYiIdIB160T6xGCdiIhIBzpzJ1MiXWKwTkREpANSZp291on0hcE6ERGRDki91vcVMFgn0hMG60RERDrQqaW/DOZkmRsny1wKj4aIQoXBOhERkQ7YLSa0TbIDYN06kZ4wWCciItIJqW59HzvCEOkGg3UiIiKdkDrC7GFmnUg3GKwTERHphLTIdC87whDphm6C9ZdffhmZmZmw2Wzo06cPvv32W6WHREREFFGBjZFKIYqiwqMholDQRbC+YsUKTJ8+HXPmzMHWrVtx2WWX4eqrr8bhw4eVHhoREVHEdEiNhdEgoKSqGidK2BGGSA8EUQcfvfv3748LL7wQixcvlq/r1q0bbrzxRsybN++s9y8pKYHD4UBxcTESEhLCOVQiIqKwGvLcWvz6WzkeGNoZXZxxjR4nioDHJ8Jd7YO72geP1/+n2+uDq9bPoghYTAb/xSjU/GmAxWSEueZnq8mA89MdyKjpRqMVnP9JC0xKD6C53G43tmzZgocffrjO9cOHD8eGDRsavI/L5YLLFcg4FBcXA/D/pyUiItKyzAQB+45W4LlPtkX0ef96XTfc1rdtRJ+zuaR5Xwd5S9IxzQfrJ0+ehNfrRVpaWp3r09LSkJ+f3+B95s2bh8cff7ze9RkZGWEZIxERkd7d/QJwt9KDOEelpaVwOBxKD4OoQZoP1iWCINT5WRTFetdJZs+ejQcffFD++fTp02jXrh0OHz7M/6wRUlJSgoyMDBw5coRfPUYIz3nk8ZxHHs955Gn5nIuiiNLSUqSnpys9FKJGaT5YT0lJgdForJdFLygoqJdtl1itVlit1nrXOxwOzf2i0bqEhASe8wjjOY88nvPI4zmPPK2ecybpSO003w3GYrGgT58+WL16dZ3rV69ejYEDByo0KiIiIiKi5tN8Zh0AHnzwQfzhD39A3759MWDAALz66qs4fPgw7rnnHqWHRkRERER0znQRrN92220oLCzEE088gby8PHTv3h2ffvop2rVrF9T9rVYrHnvssQZLYyg8eM4jj+c88njOI4/nPPJ4zonCSxd91omIiIiI9EjzNetERERERHrFYJ2IiIiISKUYrBMRERERqRSDdSIiIiIilYr6YP3ll19GZmYmbDYb+vTpg2+//VbpIenGvHnz0K9fP8THx6Nly5a48cYbsWfPnjrHiKKI7OxspKenIyYmBoMHD8bOnTsVGrH+zJs3D4IgYPr06fJ1POehd+zYMYwdOxbJycmw2+244IILsGXLFvl2nvPQqq6uxl/+8hdkZmYiJiYGHTp0wBNPPAGfzycfw3PefN988w2uu+46pKenQxAEvP/++3VuD+Ycu1wuTJ06FSkpKYiNjcX111+Po0ePRvBVEGlfVAfrK1aswPTp0zFnzhxs3boVl112Ga6++mocPnxY6aHpwrp16zBlyhRs3LgRq1evRnV1NYYPH47y8nL5mPnz52PBggVYtGgRNm/eDKfTiWHDhqG0tFTBkevD5s2b8eqrr6Jnz551ruc5D62ioiJccsklMJvN+Oyzz7Br1y4899xzaNGihXwMz3lo/e1vf8Mrr7yCRYsWYffu3Zg/fz6eeeYZLFy4UD6G57z5ysvL0atXLyxatKjB24M5x9OnT8fKlSuRk5OD9evXo6ysDCNHjoTX643UyyDSPjGKXXTRReI999xT57quXbuKDz/8sEIj0reCggIRgLhu3TpRFEXR5/OJTqdTfPrpp+VjqqqqRIfDIb7yyitKDVMXSktLxaysLHH16tXioEGDxGnTpomiyHMeDg899JB46aWXNno7z3noXXvtteKECRPqXDdq1Chx7NixoijynIcDAHHlypXyz8Gc49OnT4tms1nMycmRjzl27JhoMBjEzz//PGJjJ9K6qM2su91ubNmyBcOHD69z/fDhw7FhwwaFRqVvxcXFAICkpCQAQG5uLvLz8+v8G1itVgwaNIj/Bs00ZcoUXHvttRg6dGid63nOQ+/DDz9E3759ceutt6Jly5bo3bs3XnvtNfl2nvPQu/TSS/Hll19i7969AICffvoJ69evxzXXXAOA5zwSgjnHW7ZsgcfjqXNMeno6unfvzn8HoibQxQ6m5+LkyZPwer1IS0urc31aWhry8/MVGpV+iaKIBx98EJdeeim6d+8OAPJ5bujf4NChQxEfo17k5OTgxx9/xObNm+vdxnMeegcOHMDixYvx4IMP4pFHHsGmTZtw//33w2q14o9//CPPeRg89NBDKC4uRteuXWE0GuH1evHUU0/hjjvuAMD3eSQEc47z8/NhsViQmJhY7xjOs0TBi9pgXSIIQp2fRVGsdx0133333Yeff/4Z69evr3cb/w1C58iRI5g2bRpWrVoFm83W6HE856Hj8/nQt29fzJ07FwDQu3dv7Ny5E4sXL8Yf//hH+Tie89BZsWIFli1bhuXLl+P888/Htm3bMH36dKSnp2PcuHHycTzn4Xcu55j/DkRNE7VlMCkpKTAajfU+3RcUFNTLFFDzTJ06FR9++CG+/vprtGnTRr7e6XQCAP8NQmjLli0oKChAnz59YDKZYDKZsG7dOrz00kswmUzyeeU5D51WrVrhvPPOq3Ndt27d5IXqfJ+H3p///Gc8/PDDuP3229GjRw/84Q9/wAMPPIB58+YB4DmPhGDOsdPphNvtRlFRUaPHENHZRW2wbrFY0KdPH6xevbrO9atXr8bAgQMVGpW+iKKI++67D++99x6++uorZGZm1rk9MzMTTqezzr+B2+3GunXr+G9wjoYMGYLt27dj27Zt8qVv37648847sW3bNnTo0IHnPMQuueSSei1J9+7di3bt2gHg+zwcKioqYDDUnb6MRqPcupHnPPyCOcd9+vSB2Wyuc0xeXh527NjBfweiplBsaasK5OTkiGazWVyyZIm4a9cucfr06WJsbKx48OBBpYemC/fee6/ocDjEtWvXinl5efKloqJCPubpp58WHQ6H+N5774nbt28X77jjDrFVq1ZiSUmJgiPXl9rdYESR5zzUNm3aJJpMJvGpp54S9+3bJ77zzjui3W4Xly1bJh/Dcx5a48aNE1u3bi1+/PHHYm5urvjee++JKSkp4qxZs+RjeM6br7S0VNy6dau4detWEYC4YMECcevWreKhQ4dEUQzuHN9zzz1imzZtxDVr1og//vijeOWVV4q9evUSq6urlXpZRJoT1cG6KIri3//+d7Fdu3aixWIRL7zwQrmtIDUfgAYvb7zxhnyMz+cTH3vsMdHpdIpWq1W8/PLLxe3btys3aB36fbDOcx56H330kdi9e3fRarWKXbt2FV999dU6t/Och1ZJSYk4bdo0sW3btqLNZhM7dOggzpkzR3S5XPIxPOfN9/XXXzf4O3zcuHGiKAZ3jisrK8X77rtPTEpKEmNiYsSRI0eKhw8fVuDVEGmXIIqiqExOn4iIiIiIziRqa9aJiIiIiNSOwToRERERkUoxWCciIiIiUikG60REREREKsVgnYiIiIhIpRisExERERGpFIN1IiIiIiKVYrBORERERKRSDNaJSDOys7NxwQUXRPx5165dC0EQcPr06Yg/NxERRTfuYEpEqiAIwhlvHzduHBYtWgSXy4Xk5OQIjcrP7Xbj1KlTSEtLO+s4iYiIQonBOhGpQn5+vvz3FStW4NFHH8WePXvk62JiYuBwOJQYGhERkWJYBkNEquB0OuWLw+GAIAj1rvt9Gcz48eNx4403Yu7cuUhLS0OLFi3w+OOPo7q6Gn/+85+RlJSENm3a4PXXX6/zXMeOHcNtt92GxMREJCcn44YbbsDBgwcbHdvvy2CWLl2KFi1a4IsvvkC3bt0QFxeHq666Cnl5eY0+RlFREe68806kpqYiJiYGWVlZeOONN5pzyoiIKAowWCciTfvqq69w/PhxfPPNN1iwYAGys7MxcuRIJCYm4vvvv8c999yDe+65B0eOHAEAVFRU4IorrkBcXBy++eYbrF+/Xg623W530M9bUVGBZ599Fm+//Ta++eYbHD58GDNnzmz0+L/+9a/YtWsXPvvsM+zevRuLFy9GSkpKs18/ERHpm0npARARNUdSUhJeeuklGAwGdOnSBfPnz0dFRQUeeeQRAMDs2bPx9NNP43//+x9uv/125OTkwGAw4J///Kdcf/7GG2+gRYsWWLt2LYYPHx7U83o8Hrzyyivo2LEjAOC+++7DE0880ejxhw8fRu/evdG3b18AQPv27ZvxqomIKFowWCciTTv//PNhMAS+JExLS0P37t3ln41GI5KTk1FQUAAA2LJlC/bv34/4+Pg6j1NVVYVff/016Oe12+1yoA4ArVq1kp+jIffeey9uvvlm/Pjjjxg+fDhuvPFGDBw4MOjnIyKi6MRgnYg0zWw21/lZEIQGr/P5fAAAn8+HPn364J133qn3WKmpqc163jOt17/66qtx6NAhfPLJJ1izZg2GDBmCKVOm4Nlnnw36OYmIKPowWCeiqHLhhRdixYoVaNmyJRISEiL63KmpqRg/fjzGjx+Pyy67DH/+858ZrBMR0RlxgSkRRZU777wTKSkpuOGGG/Dtt98iNzcX69atw7Rp03D06NGwPe+jjz6KDz74APv378fOnTvx8ccfo1u3bmF7PiIi0gcG60QUVex2O7755hu0bdsWo0aNQrdu3TBhwgRUVlaGNdNusVgwe/Zs9OzZE5dffjmMRiNycnLC9nxERKQP3BSJiIiIiEilmFknIiIiIlIpButERERERCrFYJ2IiIiISKUYrBMRERERqRSDdSIiIiIilWKwTkRERESkUgzWiYiIiIhUisE6EREREZFKMVgnIiIiIlIpButERERERCrFYJ2IiIiISKX+P9TxfO4/i7UCAAAAAElFTkSuQmCC",
+      "text/plain": [
+       ""
+      ]
+     },
+     "execution_count": 65,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
    "source": [
-    "import numpy as np\n",
-    "import rasterio\n",
-    "import xarray\n",
-    "import rioxarray as rxr\n",
-    "from numba import jit,njit\n",
-    "from xarray import DataArray \n",
-    "from pathlib import Path\n",
-    "from typing import List, Tuple, Union, Dict\n",
-    "from dask import delayed\n",
-    "from dask.distributed import Client, LocalCluster, Lock\n",
-    "import dask.array as da\n",
-    "from rasterio.transform import Affine\n",
-    "import time"
+    "from IPython.display import Image\n",
+    "Image(filename=f\"{perf_results_dir}/output_monitoring/CPU_SUM_-percent-CPU_-on-s.png\")"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "200d60a9-7747-439c-a4fb-e82516575c6b",
+   "id": "1898ca12-fdcc-48b9-b733-642f2627bbe9",
    "metadata": {},
    "source": [
-    "## Work directories for the notebook"
+    "# Choose a data format library adapted to your processing chain"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
+   "cell_type": "markdown",
+   "id": "5baa8de4-7502-45b3-a5a1-52dd0f254d96",
+   "metadata": {},
+   "source": [
+    "Depending on your project, the data format will vary : you will need to choose a library that can fully handle your product.\n",
+    "A Tutorial is available [here](https://github.com/CNES/pluto-tuto/pull/3) for a detailed walkthrough on how to choose the right library"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "371254df-a3b4-4f3e-b34f-3c3db06828ba",
    "metadata": {
     "tags": []
    },
-   "outputs": [],
    "source": [
-    "input_dir = \"/work/scratch/data/romaint/input_greenit\"\n",
-    "output_dir = \"/work/scratch/data/romaint/output_greenit\"\n",
-    "s2_b4 = f\"{input_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\"\n",
-    "s2_b8 = f\"{input_dir}/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\"\n",
-    "phr_product = \"/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\"\n",
-    "phr_product_cog = f\"{input_dir}/phr_cog.tif\"\n",
-    "phr_product_lzw = f\"{input_dir}/phr_lzw.tif\""
+    "# A Green IT approach for coding image processing chains"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fb2fcaf9-5b7b-4f37-b457-6e01e5035fbf",
+   "metadata": {},
+   "source": [
+    "In this notebook we will look at a green IT approach of coding, via good pratices in Python. We will have a look at Dask, RioXarray, rasterio, numpy... all these libraries that are widely used in the satellite image processing chains. "
    ]
   },
   {

From 2dfd49b922ad4abe81e48408ad8e99d85f64b3e7 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 17 Feb 2025 10:47:29 +0000
Subject: [PATCH 32/37] Move one cell on monitoring to be more consistent

---
 tuto_greenit.ipynb | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 4f03142..ba36f01 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -283,6 +283,16 @@
     "os.chmod(f\"{input_dir}/greenit_monitored.sh\", 0o755)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "8175da03-8e09-41dc-a1ff-01adac85a07f",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Write the slurm script and submit the job"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 55,
@@ -308,16 +318,6 @@
     "\"\"\""
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "8175da03-8e09-41dc-a1ff-01adac85a07f",
-   "metadata": {
-    "tags": []
-   },
-   "source": [
-    "## Write the slurm script and submit the job"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": 57,

From 8138f2005bc956c406aeaaaa277cd2b84041c026 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 17 Feb 2025 12:50:22 +0000
Subject: [PATCH 33/37] Modify some paths to be editable

---
 tuto_greenit.ipynb | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index ba36f01..75eb763 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -21,7 +21,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 37,
+   "execution_count": 4,
    "id": "8966e0c9-d03f-4edf-b1b0-abbb4c4c2c34",
    "metadata": {
     "tags": []
@@ -56,7 +56,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 58,
+   "execution_count": 3,
    "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
    "metadata": {
     "tags": []
@@ -1371,7 +1371,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 24,
+   "execution_count": 5,
    "id": "1071ba31-2e9b-4da5-9d94-9e6ea9a5b1db",
    "metadata": {
     "tags": []
@@ -1381,9 +1381,11 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
+      "512\t/work/scratch/data/romaint/input_greenit/GreenIT.slurm\n",
       "1.5G\t/work/scratch/data/romaint/input_greenit/SENTINEL2A_20210415-105852-555_L2A_T31TCJ_C_V3-0_FRE_STACK.tif\n",
       "2.8G\t/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1\n",
       "2.0G\t/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1.tar.gz\n",
+      "512\t/work/scratch/data/romaint/input_greenit/greenit_monitored.sh\n",
       "299M\t/work/scratch/data/romaint/input_greenit/image_PHR_marmande.zip\n",
       "1.6G\t/work/scratch/data/romaint/input_greenit/phr_cog.tif\n",
       "35M\t/work/scratch/data/romaint/input_greenit/phr_cog_extract.tif\n",
@@ -1392,7 +1394,7 @@
     }
    ],
    "source": [
-    "!du -sh /work/scratch/data/romaint/input_greenit/*"
+    "!du -sh {input_dir}/*"
    ]
   },
   {
@@ -3463,7 +3465,9 @@
    "source": [
     "## Conclusions and recommandations\n",
     "\n",
-    "For an NDVI on the same product, the performance difference is 55s vs 35s here : **57%!**\n",
+    "For an NDVI on the same product, the performance difference between these two parallelisation methods is 55s vs 35s here : **57%!**\n",
+    "\n",
+    "The initial compute time for an NDVI on the pleiade tile was around 150s, with all the optimisations we were able to go down to 35s, which **divided the compute time by 3!**\n",
     "\n",
     "
    \n",
     "    \n",
@@ -3475,7 +3479,7 @@
     "\n",
     "* Use rio.to_raster to compute/write in parallel, instead of dask.compute and then write\n",
     "\n",
-    "* Huge time gain without precision loss\n",
+    "* Using parallelisation methods dask / RIOXarray leads to a huge time gain without precision loss with big multiband products\n",
     "\n",
     "* You have to carefully choose your chunk size using a multiple of the cog size\n"
    ]

From f8bcced81da99c4cee9532fd8eccbd8df57a5ea6 Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Mon, 17 Feb 2025 12:57:53 +0000
Subject: [PATCH 34/37] Clarify the performance graphs conclusion

---
 tuto_greenit.ipynb | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 75eb763..2f9a2a2 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -3451,11 +3451,10 @@
     "\n",
     "The CPU usage shows 2 peaks for the computing with dask, only 1 peak with the tiled write scenario\n",
     "\n",
-    "**Conclusion for Memory usage**\n",
+    "**Conclusion for the performance graphs**\n",
     "\n",
-    "* Both method consumes the same amount of memory\n",
-    "* Cpu usage is **higher with dask compute**\n",
-    "* You must use the tiled write in order to gain time"
+    "* Both method consumes the same amount of memory ~ 2x the initial product size\n",
+    "* Cpu usage is **higher with dask.compute() method**"
    ]
   },
   {
@@ -3463,11 +3462,13 @@
    "id": "ffd1783b-a0e3-43f2-861d-13d043653dc6",
    "metadata": {},
    "source": [
-    "## Conclusions and recommandations\n",
+    "## Conclusions\n",
+    "\n",
+    "The initial compute time for an NDVI on the pleiade tile was around 150s, with all the optimisations (CoG compression, parallelisation dask + rioxarray_to raster with tiled=true) we were able to go down to 35s, which **divided the compute time by 3!**\n",
     "\n",
     "For an NDVI on the same product, the performance difference between these two parallelisation methods is 55s vs 35s here : **57%!**\n",
     "\n",
-    "The initial compute time for an NDVI on the pleiade tile was around 150s, with all the optimisations we were able to go down to 35s, which **divided the compute time by 3!**\n",
+    "### Recommandations\n",
     "\n",
     "
    \n",
     "    \n",

From 82188ae2e2925668c5e7ffa361dc1e95865ed1ea Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Fri, 4 Apr 2025 11:49:52 +0000
Subject: [PATCH 35/37] Add context on all chapter introductions and cut cells
 in more simple cells

---
 tuto_greenit.ipynb | 896 ++++++++++++++++++++++-----------------------
 1 file changed, 435 insertions(+), 461 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 2f9a2a2..da3c11e 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -7,7 +7,18 @@
    "source": [
     "# Best practices for Green IT coding in python\n",
     "\n",
-    "In this tutorial, we will show good pratices for a green IT approach to code an image processing chain with efficiency in mind\n",
+    "In this tutorial, we will show good pratices for a green IT approach to code an image processing chain with efficiency in mind.\n",
+    "This notebook is divided in 3 different parts from begginer level to advanced level :\n",
+    "\n",
+    "* Good practices in Python code for image processing\n",
+    "* Optimisation of an NDVI computing on a Sentinel-2 image and a Pleiades image\n",
+    "* Image compression for performance optimisation\n",
+    "* Comparison of performances between parallelisation methods and conclusions\n",
+    "\n",
+    "In this notebook we will cover two use cases for NDVI computing, the first uses a Sentinel 2 image with single band files (~450 Mo of Data).\n",
+    "The second one will use a Pleiades image which is a big multiband file (13 Go of Data).\n",
+    "In each of these use cases we will start from a basic NDVI calculation in standard python code, and optimize this code in different ways to see where you can have performance problems depending on your problematics\n",
+    "\n",
     "Let's first define everything needed along this notebook"
    ]
   },
@@ -21,7 +32,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 2,
    "id": "8966e0c9-d03f-4edf-b1b0-abbb4c4c2c34",
    "metadata": {
     "tags": []
@@ -51,12 +62,16 @@
    "id": "4c610ea3-563e-4409-9225-1566ae822f3c",
    "metadata": {},
    "source": [
-    "## Work directories for the notebook"
+    "## Work directories for the notebook\n",
+    "\n",
+    "Here we define the directories and the image files used along the notebook. \n",
+    "* A first use case uses a Sentinel 2 image with 2 files only (the two interesting bands to compute the NDVI b4 and b8)\n",
+    "* The second use case uses a Pleiades image with 1 multiband file, we will compress it and write it as a COG"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
    "metadata": {
     "tags": []
@@ -82,15 +97,17 @@
    "source": [
     "# Good pratices for general python coding\n",
     "\n",
-    "## for loop optimisations : benefits of using numpy\n",
+    "## Avoir For loops : benefits of using numpy\n",
+    "\n",
+    "Don't use for loops on your data, numpy is made for this use case and provides a big performance difference compared to standard python code. Here is a code snippet that shows the big performance difference to apply a square on 250 000 elements\n",
     "\n",
-    "using a for loop vs numpy for basic loop coding shows that you must avoid for loops in your code, here is a code snippet that shows the big performance difference to apply a square on 250 000 elements"
+    "For an in-depth comparison of how much numpy is faster than for and while loops, this link provides a complete performance comparison : https://www.blog.duomly.com/loops-in-python-comparison-and-performance/"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
-   "id": "cecef79a-2ada-4df8-bde5-abb60db8a037",
+   "execution_count": 1,
+   "id": "764e50d5-816a-4228-8ae5-9c60efca8c26",
    "metadata": {
     "tags": []
    },
@@ -99,35 +116,65 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Time with for loop : 83.00238009542227 ms\n",
-      "Time with for list comprehension : 63.063559122383595 ms\n",
-      "Time with numpy : 0.2981601282954216 ms\n"
+      "99.3 ms ± 231 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
      ]
     }
    ],
    "source": [
-    "start = time.perf_counter()\n",
+    "%%timeit\n",
+    "\n",
     "# Calculate square with for loop\n",
     "array_forloop = []\n",
     "for i in range(0,500):\n",
     "    for j in range(0,500):\n",
-    "        array_forloop.append((i * j) ** 2)\n",
-    "    \n",
-    "end = time.perf_counter()\n",
-    "print(\"Time with for loop : {} ms\".format((end-start)*1000))\n",
+    "        array_forloop.append((i * j) ** 2)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "cbc29586-c920-4f27-babc-7273f42e8d49",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "84.5 ms ± 439 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%timeit\n",
     "\n",
-    "# Using list comprehension instead of a for loop is a nice optimisation if your use case needs a for loop anyway\n",
-    "start = time.perf_counter()\n",
-    "array_forloop = [i**2 for i in range(0,250000)]\n",
-    "end = time.perf_counter()\n",
-    "print(\"Time with for list comprehension : {} ms\".format((end-start)*1000))\n",
+    "# Using list comprehension instead of a for loop is a nice optimisation\n",
+    "array_forloop = [i**2 for i in range(0,250000)]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "cecef79a-2ada-4df8-bde5-abb60db8a037",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "159 μs ± 412 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%timeit\n",
     "      \n",
     "# Using numpy\n",
-    "start = time.perf_counter()\n",
     "arr = np.arange(250000)\n",
-    "squared_arr = np.square(arr)\n",
-    "end = time.perf_counter()\n",
-    "print(\"Time with numpy : {} ms\".format((end-start)*1000))"
+    "squared_arr = np.square(arr)"
    ]
   },
   {
@@ -142,7 +189,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 29,
+   "execution_count": 6,
    "id": "35d9ffa0-a498-4ab5-ae73-47cb3fb70eb1",
    "metadata": {
     "tags": []
@@ -152,26 +199,19 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "List search time: 42.939479635097086 s\n",
-      "Set search time: 0.006291795987635851 s\n",
-      "Numpy search time: 4.1151342540979385 s\n"
+      "List search time: 45.0116172991693 s\n",
+      "Numpy search time: 4.298595204949379 s\n"
      ]
     }
    ],
    "source": [
     "nd_list = [i for i in range(0,100000)]\n",
-    "nd_set = set([i for i in range(0,100000)])\n",
     "nd_array = np.arange(0,100000)\n",
     "\n",
     "#Look at an element\n",
     "def search_list():\n",
     "    if 98950 in nd_list:\n",
     "        pass\n",
-    "    \n",
-    "#Look at the same element in the set\n",
-    "def search_set():\n",
-    "    if 98950 in nd_set:\n",
-    "        pass\n",
     "\n",
     "def search_numpy():\n",
     "    if np.where(nd_array==98950):\n",
@@ -179,11 +219,9 @@
     "\n",
     "#lets say nodata value is 98950 in that case\n",
     "t1 = timeit(search_list, number=100000)\n",
-    "t2 = timeit(search_set, number=100000)\n",
-    "t3 = timeit(search_numpy, number=100000)\n",
+    "t2 = timeit(search_numpy, number=100000)\n",
     "print(\"List search time: {} s\".format(t1))\n",
-    "print(\"Set search time: {} s\".format(t2))\n",
-    "print(\"Numpy search time: {} s\".format(t3))"
+    "print(\"Numpy search time: {} s\".format(t2))"
    ]
   },
   {
@@ -196,11 +234,23 @@
     "## Conclusion\n",
     "\n",
     "* Don't try to reinvent the wheel, use standard and well performing python libraries like numpy\n",
-    "* Avoid using for loops to apply a computing on a bunch of pixels, use numpy instead\n",
-    "* If you need a for loop for another processing, use list comprehension as much as possible, it is very well documented \n",
-    "* Use sets when you need to lookup for values in your image but beware that the set is not intended to be modified. You can alternatively use numpy.where but it is not as efficient as the set because it returns a boolean array for each element of the array, nevertheless it is 10 times more efficient than a list.\n",
+    "* Don't use for loops to apply a computing on a bunch of pixels, use numpy instead\n",
+    "* Use list comprehension as much as possible if you really need a for loop, it is very well documented \n",
+    "* Use numpy to lookup for data in your dataset\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1898ca12-fdcc-48b9-b733-642f2627bbe9",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "# Choose a data format library adapted to your processing chain\n",
     "\n",
-    "For an in-depth comparison of how much numpy is faster than for and while loops, this link provides a complete performance comparison : https://www.blog.duomly.com/loops-in-python-comparison-and-performance/"
+    "Depending on your project, the data format will vary : it is very important to choose a library that can fully handle your product.\n",
+    "\n",
+    "A Tutorial is available [here](https://github.com/CNES/pluto-tuto/pull/3) for a detailed walkthrough on how to choose the right library"
    ]
   },
   {
@@ -378,23 +428,6 @@
     "Image(filename=f\"{perf_results_dir}/output_monitoring/CPU_SUM_-percent-CPU_-on-s.png\")"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "1898ca12-fdcc-48b9-b733-642f2627bbe9",
-   "metadata": {},
-   "source": [
-    "# Choose a data format library adapted to your processing chain"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5baa8de4-7502-45b3-a5a1-52dd0f254d96",
-   "metadata": {},
-   "source": [
-    "Depending on your project, the data format will vary : you will need to choose a library that can fully handle your product.\n",
-    "A Tutorial is available [here](https://github.com/CNES/pluto-tuto/pull/3) for a detailed walkthrough on how to choose the right library"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "371254df-a3b4-4f3e-b34f-3c3db06828ba",
@@ -425,7 +458,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 18,
    "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62",
    "metadata": {
     "tags": []
@@ -490,18 +523,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 12,
    "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21",
    "metadata": {
     "scrolled": true,
     "tags": []
    },
    "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n",
+      "Perhaps you already have a cluster running?\n",
+      "Hosting the HTTP server on port 36045 instead\n",
+      "  warnings.warn(\n"
+     ]
+    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Dask Dashboard:  http://127.0.0.1:8787/status\n"
+      "Dask Dashboard:  http://127.0.0.1:36045/status\n"
      ]
     },
     {
@@ -511,7 +554,7 @@
        "    
     
    \n",
        "    
    \n",
        "        
    Client
    \n",
-       "        
    Client-20e36361-ed10-11ef-8e57-00620ba47efa
    \n",
+       "        
    Client-5ae0bd29-ef98-11ef-addd-e43d1a347da2
    \n",
        "        \n",
        "\n",
        "        \n",
@@ -524,7 +567,7 @@
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
@@ -533,7 +576,7 @@
        "        
    \n",
-       "                    Dashboard:  http://127.0.0.1:8787/status\n",
+       "                    Dashboard:  http://127.0.0.1:36045/status\n",
        "                
    \n",
        "\n",
        "        \n",
-       "            \n",
        "        \n",
@@ -546,11 +589,11 @@
        "    
    \n",
        "    
    \n",
        "        
    LocalCluster
    \n",
-       "        
    a6db3b0f
    \n",
+       "        
    01ab761d
    \n",
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
        "            \n",
@@ -583,11 +626,11 @@
        "        
     
    \n",
        "        
    \n",
        "            
    Scheduler
    \n",
-       "            
    Scheduler-e08b1170-037d-4526-80fa-db62e17f1914
    \n",
+       "            
    Scheduler-a3b547c6-a54a-46ae-8316-982ab39af35a
    \n",
        "            
    \n",
-       "                    Dashboard: http://127.0.0.1:8787/status\n",
+       "                    Dashboard: http://127.0.0.1:36045/status\n",
        "                \n",
        "                    Workers: 4\n",
@@ -558,10 +601,10 @@
        "            
    \n",
-       "                    Total threads: 8\n",
+       "                    Total threads: 4\n",
        "                \n",
-       "                    Total memory: 56.00 GiB\n",
+       "                    Total memory: 28.00 GiB\n",
        "                
    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                \n",
@@ -606,7 +649,7 @@
        "                        Started: Just now\n",
        "                    \n",
        "                    \n",
        "                \n",
        "            
    \n",
-       "                        Comm: tcp://127.0.0.1:43565\n",
+       "                        Comm: tcp://127.0.0.1:34383\n",
        "                    \n",
        "                        Workers: 4\n",
@@ -595,10 +638,10 @@
        "                
    \n",
-       "                        Dashboard: http://127.0.0.1:8787/status\n",
+       "                        Dashboard: http://127.0.0.1:36045/status\n",
        "                    \n",
-       "                        Total threads: 8\n",
+       "                        Total threads: 4\n",
        "                    
    \n",
-       "                        Total memory: 56.00 GiB\n",
+       "                        Total memory: 28.00 GiB\n",
        "                    
    \n",
@@ -629,29 +672,29 @@
        "                \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -674,29 +717,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:32797\n",
+       "                            Comm:  tcp://127.0.0.1:46371\n",
        "                        \n",
-       "                            Total threads:  2\n",
+       "                            Total threads:  1\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:39065/status\n",
+       "                            Dashboard:  http://127.0.0.1:45497/status\n",
        "                        \n",
-       "                            Memory:  14.00 GiB\n",
+       "                            Memory:  7.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:45629\n",
+       "                            Nanny:  tcp://127.0.0.1:44907\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32971832/dask-scratch-space/worker-xym2zrdk\n",
+       "                            Local directory:  /tmp/slurm-33186461/dask-scratch-space/worker-wrjzk4kn\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -719,29 +762,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:43229\n",
+       "                            Comm:  tcp://127.0.0.1:42037\n",
        "                        \n",
-       "                            Total threads:  2\n",
+       "                            Total threads:  1\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:40875/status\n",
+       "                            Dashboard:  http://127.0.0.1:41679/status\n",
        "                        \n",
-       "                            Memory:  14.00 GiB\n",
+       "                            Memory:  7.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:42161\n",
+       "                            Nanny:  tcp://127.0.0.1:34927\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32971832/dask-scratch-space/worker-6h1c1fcn\n",
+       "                            Local directory:  /tmp/slurm-33186461/dask-scratch-space/worker-z8wop2y_\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -764,29 +807,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:45785\n",
+       "                            Comm:  tcp://127.0.0.1:36507\n",
        "                        \n",
-       "                            Total threads:  2\n",
+       "                            Total threads:  1\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:44499/status\n",
+       "                            Dashboard:  http://127.0.0.1:39185/status\n",
        "                        \n",
-       "                            Memory:  14.00 GiB\n",
+       "                            Memory:  7.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:41597\n",
+       "                            Nanny:  tcp://127.0.0.1:37391\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32971832/dask-scratch-space/worker-iit2xsch\n",
+       "                            Local directory:  /tmp/slurm-33186461/dask-scratch-space/worker-nqzwkhoz\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -813,10 +856,10 @@
        ""
       ],
       "text/plain": [
-       ""
+       ""
       ]
      },
-     "execution_count": 3,
+     "execution_count": 12,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -842,7 +885,7 @@
    "id": "0e86755c-f505-4d6b-830e-396485e12054",
    "metadata": {},
    "source": [
-    "# Use Case : Calculation of the Average NDVI on a Sentinel 2 image\n",
+    "# Use Case 1 : Calculation of the Average NDVI on a Sentinel 2 image\n",
     "\n",
     "In this example, we will use standard python libraries to: \n",
     "\n",
@@ -850,7 +893,7 @@
     "2. Calculate the associated NDVI, which combines multi-band information into a single band.\n",
     "4. Write the resulting image to the disk.\n",
     "\n",
-    "First, let's read the data we need to perform the NDVI with RIOXarray."
+    "First, let's read the data we need to perform the NDVI with RasterIO."
    ]
   },
   {
@@ -858,128 +901,14 @@
    "id": "fc56a7e0-edb5-4635-b5a2-0b274d08ce38",
    "metadata": {},
    "source": [
-    "### Open raster thanks to rioxarray\n",
+    "### Basic python code for NDVI\n",
     "\n",
-    "Here we are going to open the raster data required for this tutorial, the RGB bands from a Sentinel-2 acquisition. To do this, we're going to use rioxarray and, more specifically, the [open_rasterio](https://corteva.github.io/rioxarray/html/rioxarray.html#rioxarray-open-rasterio) method, which opens the images lazily (without loading data into memory) and returns a `dask.array` object. \n",
-    "From this method we will use the ``chunks`` and ``lock`` arguments, which respectively set a chunk size and limit access to the data to one thread at a time to avoid read problems. Here ``chunks`` is set to ``True`` to allow dask to automatically size chunks.\n",
-    "\n",
-    "When the data is read, we can express the NDVI calculation as if it were a numpy array. We add ``[None, :, :]`` to keep the shape as ``(bands, rows, cols)``. Then we can apply reduction on the dask.array and use ``compute()`` on it to triger the computation."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "71481af7-7b8b-4f09-8a02-fd3bc2ec3f8f",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/html": [
-       "
    \n",
-       "                            Comm:  tcp://127.0.0.1:43737\n",
+       "                            Comm:  tcp://127.0.0.1:45933\n",
        "                        \n",
-       "                            Total threads:  2\n",
+       "                            Total threads:  1\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:38139/status\n",
+       "                            Dashboard:  http://127.0.0.1:46085/status\n",
        "                        \n",
-       "                            Memory:  14.00 GiB\n",
+       "                            Memory:  7.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:33343\n",
+       "                            Nanny:  tcp://127.0.0.1:44855\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-32971832/dask-scratch-space/worker-bd6fd11h\n",
+       "                            Local directory:  /tmp/slurm-33186461/dask-scratch-space/worker-x7prb8pz\n",
        "                        
    \n",
-       "    \n",
-       "        \n",
-       "        \n",
-       "    \n",
-       "
    \n",
-       "            \n",
-       "                \n",
-       "                    \n",
-       "                        \n",
-       "                        \n",
-       "                        \n",
-       "                    \n",
-       "                \n",
-       "                \n",
-       "                    \n",
-       "                    \n",
-       "                        \n",
-       "                        \n",
-       "                        \n",
-       "                    \n",
-       "                    \n",
-       "                    \n",
-       "                        \n",
-       "                        \n",
-       "                        \n",
-       "                    \n",
-       "                    \n",
-       "                        \n",
-       "                        \n",
-       "                    \n",
-       "                    \n",
-       "                        \n",
-       "                        \n",
-       "                    \n",
-       "                \n",
-       "            
      Array  Chunk 
     Bytes  459.90 MiB  127.98 MiB 
     Shape  (2, 10980, 10980)  (1, 6111, 10980) 
     Dask graph  4 chunks in 5 graph layers 
     Data type  int16 numpy.ndarray 
    \n",
-       "            		\n",
-       "        \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  \n",
-       "\n",
-       "  \n",
-       "  10980\n",
-       "  10980\n",
-       "  2\n",
-       "\n",
-       "        
    "
-      ],
-      "text/plain": [
-       "dask.array"
-      ]
-     },
-     "execution_count": 5,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# Open the raster\n",
-    "reading_chunks = True\n",
-    "input_data_array, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], reading_chunks, False)\n",
-    "input_data_array"
+    "Here we are going to open the raster data required for this tutorial, the Red and NIR bands from a Sentinel-2 acquisition. To do this, we're going to use rasterio which loads the entire product in memory, without using parallelisation."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 11,
    "id": "2dad7fe0-23bc-40b2-b27e-4c7cb0f8dc0e",
    "metadata": {
     "tags": []
@@ -989,34 +918,60 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "CPU times: user 345 ms, sys: 1.49 s, total: 1.84 s\n",
-      "Wall time: 3.26 s\n"
+      "1.73 s ± 195 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
      ]
     }
    ],
    "source": [
-    "%%time\n",
+    "%%timeit\n",
     "\n",
-    "ndvi_array = (input_data_array[1] - input_data_array[0]) / (input_data_array[1] + input_data_array[0])[None, :, :]\n",
-    "# Launch the computing with dask with the compute call\n",
-    "mean_ndvi = ndvi_array.compute() \n",
-    "crs=\"EPSG:4326\"\n",
-    "output_file = Path(f\"{output_dir}/ndvi_dask.tif\")\n",
-    "create_raster(mean_ndvi, output_file, x_res , y_res,\n",
-    "                  top_left_x, top_left_y, crs)"
+    "# Open the two rasters we will use for NDVI\n",
+    "with rasterio.open(s2_b4, 'r') as ds:\n",
+    "    red_band = ds.read() \n",
+    "\n",
+    "with rasterio.open(s2_b8, 'r') as ds:\n",
+    "    nir_band = ds.read()\n",
+    "\n",
+    "# Compute NDVI with NIR and RED bands\n",
+    "ndvi_array = (nir_band - red_band) / (nir_band + red_band)\n",
+    "\n",
+    "# Write the NDVI product\n",
+    "output_file = Path(f\"{output_dir}/ndvi_basic.tif\")\n",
+    "with rasterio.open(\n",
+    "            output_file, \"w\",\n",
+    "            driver=\"GTiff\",\n",
+    "            height=ndvi_array.shape[1],\n",
+    "            width=ndvi_array.shape[2],\n",
+    "            count=ndvi_array.shape[0],\n",
+    "            dtype=ndvi_array.dtype\n",
+    "    ) as dst:\n",
+    "        dst.write(ndvi_array)"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "7f5d09f0-cf39-4e3b-afcb-6db0bccc877a",
+   "id": "c867e7a7-ccc1-4387-9dce-712938b282e7",
    "metadata": {},
    "source": [
-    "## Calculate NDVI With OTB in python"
+    "## Comparing performance of a basic python code for NDVI with Orfeotoolbox NDVI computing\n",
+    "\n",
+    "For comparison purposes, we will use the OrfeoToolbox project, developed by CNES which uses streaming to improve performances.\n",
+    "OTB provides a python interface to use its applications (coded in C++ internally for performance purpose). We will see that in this case, the **streaming does not really make sense because of the size of the data**"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7f5d09f0-cf39-4e3b-afcb-6db0bccc877a",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Calculate NDVI With OTB in python"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 12,
    "id": "d83fde75-d936-4372-b7c6-412c9b2d5bf7",
    "metadata": {
     "tags": []
@@ -1042,7 +997,7 @@
        "0"
       ]
      },
-     "execution_count": 7,
+     "execution_count": 12,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1065,70 +1020,97 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c030e442-871e-4d59-8cb6-1310b5867df1",
+   "id": "c1b01381-b598-4e69-9389-255da7cc72cf",
    "metadata": {},
    "source": [
-    "## Calculate NDVI with OTB in C++\n",
-    "This part will call BandMath with the otb CLI to compare performances with the python swig interface"
+    "## Conclusion\n",
+    "\n",
+    "The compute time with OTB for a simple NDVI computing is approximatly the same as the standard python computing, this is mainly due to the size of the data, the streaming functionality of OTB is not relevant as the computing takes a couple of seconds.\n",
+    "\n",
+    "Now let's see how to optimize our initial code, to determine when using dask becomes important"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386",
+   "metadata": {},
+   "source": [
+    "# Trying to optimize NDVI calculation for 450Mo of data with parallelisation methods\n",
+    "\n",
+    "Let's see if we can improve the computing time with well known python libraries, with a Sentinel 2 product (size of about 2 Go if you include the bands you need only), every file is single band"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
-   "id": "73a731cf-a534-4b49-9a6d-425b58d87e45",
+   "execution_count": 21,
+   "id": "5f92ce7a-4e9a-4b78-b1b5-2295d939faac",
    "metadata": {
     "tags": []
    },
+   "outputs": [],
+   "source": [
+    "# Define useful functions\n",
+    "\n",
+    "def compute_ndvi_dask(input_data_red: DataArray,input_data_nir: DataArray):\n",
+    "    ndvi_array = ((input_data_nir - input_data_red) / (input_data_nir + input_data_red))[None,:,:]\n",
+    "    ndvi_array.compute()\n",
+    "    return ndvi_array\n",
+    "\n",
+    "def compute_ndvi_std(input_data_red: np.ndarray,input_data_nir: np.ndarray):\n",
+    "    ndvi_array = (input_data_nir - input_data_red) / (input_data_nir + input_data_red)\n",
+    "    return ndvi_array"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4230eeca-a80b-4550-b781-c676b65643b7",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Use Xarray with apply_ufunc"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
+   "metadata": {
+    "scrolled": true,
+    "tags": []
+   },
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "Warning 1: Invalid value for NUM_THREADS: \n"
-     ]
-    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Writing /work/scratch/data/romaint/output_greenit/img_ndvi_otb_cpp.tif...: 100% [**************************************************] (3s)\n"
+      "2.3 s ± 103 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
      ]
     }
    ],
    "source": [
-    "%%bash\n",
-    "\n",
-    "WORK_DIR=\"/work/scratch/data/romaint\"\n",
-    "otbcli_BandMath -il \"${WORK_DIR}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B4.tif\" \"${WORK_DIR}/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1_FRE_B8.tif\" -exp \"( im2b1 - im1b1 ) / ( im2b1 + im1b1 )\" -out \"${WORK_DIR}/output_greenit/img_ndvi_otb_cpp.tif\" "
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c1b01381-b598-4e69-9389-255da7cc72cf",
-   "metadata": {},
-   "source": [
-    "## Conclusion\n",
+    "%%timeit\n",
     "\n",
-    "The compute time with OTB for a simple NDVI computing is lower than standard python computing, this is mainly due to the overkill usage of dask with single band files.\n",
-    "Now let's see how to optimize our initial code, to determine when it using dask becomes important"
+    "input_data_b4 = xarray.open_dataarray(s2_b4)\n",
+    "input_data_b8 = xarray.open_dataarray(s2_b8)\n",
+    "ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n",
+    "output_file = Path(f\"{output_dir}/ndvi_ufunc.tif\")\n",
+    "ndvi_computed.rio.to_raster(output_file)"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "e7c62deb-08a1-4a39-8c5d-ebe0d0e28386",
+   "id": "d228aa26-96b1-482a-a289-55e2c6a0b366",
    "metadata": {},
    "source": [
-    "# Compute time optimisation methods for NDVI\n",
-    "\n",
-    "Let's see if we can improve the computing time with well known python libraries, with a Sentinel 2 product (size of about 2 Go if you include the bands you need only), every file is single band"
+    "### Use Dask"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
-   "id": "bd7a3e53-87f6-4dca-9d9e-550956f02a50",
+   "execution_count": 20,
+   "id": "184bc745-270b-453c-9e80-34918fbd81d2",
    "metadata": {
-    "scrolled": true,
     "tags": []
    },
    "outputs": [
@@ -1136,83 +1118,18 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Time to compute with Raster IO + numba = 2.2769373627379537s\n",
-      "Time to compute with Raster IO + without numba = 1.8911610832437873s\n",
-      "Time to compute with Xarray + apply ufunc = 3.2426639813929796s\n",
-      "Time to compute with RIOXarray + dask = 5.112378729507327s\n",
-      "CPU times: user 2.59 s, sys: 5.61 s, total: 8.2 s\n",
-      "Wall time: 12.5 s\n"
+      "1.99 s ± 130 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
      ]
     }
    ],
    "source": [
-    "%%time\n",
-    "\n",
-    "#Define functions for each scenario\n",
-    "@njit\n",
-    "def one_pixel_ndvi(p1,p2):\n",
-    "    return (p2-p1) / (p2+p1) \n",
-    "\n",
-    "@njit\n",
-    "def compute_ndvi_numba(input_data_1: np.ndarray,input_data_2: np.ndarray):\n",
-    "    #ndvi_array = [one_pixel_ndvi(i,j) for i in input_data_1 for j in input_data_2]\n",
-    "    ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n",
-    "    return ndvi_array\n",
-    "\n",
-    "def compute_ndvi_dask(input_data_1: DataArray,input_data_2: DataArray):\n",
-    "    ndvi_array = ((input_data_2 - input_data_1) / (input_data_2 + input_data_1))[None,:,:]\n",
-    "    ndvi_array.compute()\n",
-    "    return ndvi_array\n",
-    "\n",
-    "def compute_ndvi_std(input_data_1: np.ndarray,input_data_2: np.ndarray):\n",
-    "    #ndvi_array = [one_pixel_ndvi(i,j) for i in input_data_1 for j in input_data_2]\n",
-    "    ndvi_array = (input_data_2 - input_data_1) / (input_data_2 + input_data_1)\n",
-    "    return ndvi_array\n",
-    "#Compute with numba\n",
-    "start = time.perf_counter()\n",
-    "with rasterio.open(s2_b4, 'r') as ds:\n",
-    "    input_data_b4 = ds.read() \n",
-    "\n",
-    "with rasterio.open(s2_b8, 'r') as ds:\n",
-    "    input_data_b8 = ds.read()\n",
+    "%%timeit\n",
     "\n",
-    "ndvi_computed = compute_ndvi_numba(input_data_b4,input_data_b8)\n",
-    "crs=\"EPSG:4326\"\n",
-    "output_file = Path(f\"{output_dir}/ndvi_numba.tif\")\n",
-    "create_raster(ndvi_computed, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
-    "end = time.perf_counter()\n",
-    "print(\"Time to compute with Raster IO + numba = {}s\".format((end - start)))\n",
-    "\n",
-    "start = time.perf_counter()\n",
-    "with rasterio.open(s2_b4, 'r') as ds:\n",
-    "    input_data_b4 = ds.read() \n",
-    "\n",
-    "with rasterio.open(s2_b8, 'r') as ds:\n",
-    "    input_data_b8 = ds.read()\n",
-    "\n",
-    "ndvi_computed = compute_ndvi_std(input_data_b4,input_data_b8)\n",
-    "crs=\"EPSG:4326\"\n",
-    "output_file = Path(f\"{output_dir}/ndvi_without_numba.tif\")\n",
-    "create_raster(ndvi_computed, output_file, x_res , y_res,top_left_x, top_left_y, crs)\n",
-    "end = time.perf_counter()\n",
-    "print(\"Time to compute with Raster IO + without numba = {}s\".format((end - start)))\n",
-    "# Example with xarray\n",
-    "start = time.perf_counter()\n",
-    "input_data_b4 =  xarray.open_dataarray(s2_b4)\n",
-    "input_data_b8 =  xarray.open_dataarray(s2_b8)\n",
-    "ndvi_computed = xarray.apply_ufunc(compute_ndvi_std,input_data_b4,input_data_b8)\n",
-    "output_file = Path(f\"{output_dir}/ndvi_ufunc.tif\")\n",
-    "ndvi_computed.rio.to_raster(output_file)\n",
-    "end = time.perf_counter()\n",
-    "print(\"Time to compute with Xarray + apply ufunc = {}s\".format((end - start)))\n",
-    "\n",
-    "start = time.perf_counter()\n",
-    "input_data, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], (-1,2200,2200), False)\n",
+    "chunks_size = (-1,2200,2200)\n",
+    "input_data, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], chunks_size , False)\n",
     "ndvi_array = compute_ndvi_dask(input_data[0],input_data[1])\n",
     "output_file = Path(f\"{output_dir}/ndvi_dask.tif\")\n",
-    "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)\n",
-    "end = time.perf_counter()\n",
-    "print(\"Time to compute with RIOXarray + dask = {}s\".format((end - start)))\n"
+    "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)"
    ]
   },
   {
@@ -1220,11 +1137,37 @@
    "id": "c9229d23-c553-4c30-8ae0-b21cee07879e",
    "metadata": {},
    "source": [
-    "## Conclusion for a Sentinel 2 product with single band files (~3Go for the full product)\n",
+    "## Conclusion for a Sentinel 2 product with two single band files (450Mo of data)\n",
+    "\n",
+    "* Using **RIOXarray + dask on an amount of data that does not constrain your RAM is counter productive**.\n",
+    "* Xarray + applyufunc does not improve performance too\n",
+    "\n",
+    "**The use of parallelisation methods becomes necessary when you are RAM and/or CPU constrained which means the product size is a couple of Gigabytes** \n",
+    "\n",
+    "the RAM parameter is very important to take into account when developing your image processing chain, dasks memory management is decribed here :\n",
+    "https://docs.dask.org/en/stable/shared.html?highlight=memory\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "34beb54a-84da-4051-a712-17b0c66e1eff",
+   "metadata": {},
+   "source": [
+    "**In the next use case, we will use a bigger product, to see the importance of parallelisation and data size reduction in that case. We will now use a PLEIADES image which is a multi band product of 13Go.**"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "56e25516-3d90-4ee6-8c4b-085b94d3d096",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "# Use case 2 : I have to use bigger products which average size is > 10Go\n",
+    "\n",
+    "In this example we will use a PLEIADES product (which original size is 13Go) which is composed of 41663x39844 pixels\n",
     "\n",
-    "* Using **RIOXarray + dask on single band product is counter productive**.\n",
-    "* Xarray + applyufunc also leads to low performance compared to opening the whole product with rasterio.\n",
-    "* Using numba do enhances the performance but after the first compilation, if you start it once it will take more time that the computing without numba. It is more suitable for numpy vectorize [optimisation]( https://numba.readthedocs.io/en/stable/user/vectorize.html#the-vectorize-decorator)\n"
+    "This is the typical use case where dask or rioxarray with parallel read and write support is necessary to speedup your processing. Let's see what are the recommended parameters to optimize your processing with big data"
    ]
   },
   {
@@ -1232,7 +1175,11 @@
    "id": "807ff285-435b-4d82-aa67-bf9c32fd0f43",
    "metadata": {},
    "source": [
-    "# The importance of reducing the size of your data\n",
+    "## The importance of reducing the size of your data\n",
+    "\n",
+    "The most important thing to take into account when developing your image processing chain is the RAM and CPU.\n",
+    "\n",
+    "**The size of the data has a big impact on the RAM because the products can be loaded by chunks or even entirely in memory**\n",
     "\n",
     "Let's see how the size of your data can have an impact on your compute time"
    ]
@@ -1258,7 +1205,7 @@
     }
    ],
    "source": [
-    "## Read the initial raster\n",
+    "## Read the initial 13Go raster\n",
     "\n",
     "raster = rasterio.open(phr_product, 'r')\n",
     "raster.profile"
@@ -1269,36 +1216,41 @@
    "id": "7a95208c-4974-4c85-a09d-e7cd99fd09cd",
    "metadata": {},
    "source": [
-    "## Simple LZW Compression with RasterIO"
+    "## Simple LZW Compression with RasterIO\n",
+    "\n",
+    "The LZW compression is lossless and standard in image processing, all libraries like rasterio, rioxarray etc supports this compression. It permits to **save a lot of disk space**, but we will see that this compression type **does not improve compute time**.\n",
+    "\n",
+    "An In-depth guide to have a better understanding of how compression works on TIFF images is available here :\n",
+    "\n",
+    "https://kokoalberti.com/articles/geotiff-compression-optimization-guide/"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 27,
    "id": "69b8bd27-926e-4373-86d1-4dafe7fdbeed",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "CPU times: user 1min 13s, sys: 5.12 s, total: 1min 19s\n",
-      "Wall time: 1min 17s\n"
+     "ename": "NameError",
+     "evalue": "name 'raster' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[27], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mget_ipython\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_cell_magic\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mtimeit\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m# Register GDAL format drivers and configuration options with a\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m# context manager.\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43mwith rasterio.Env():\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    # Write an array as a raster band to a new 8-bit file. For\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    # the new file\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43ms profile, we start with the profile of the source\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    profile = raster.profile\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    # And then change the band count to 1, set the\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    # dtype to uint8, and specify LZW compression.\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    profile.update(compress=\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mlzw\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m)\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    with rasterio.open(phr_product_lzw, \u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mw\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m, **profile) as dst:\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m        dst.write(raster.read(1), 1)\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py:2543\u001b[0m, in \u001b[0;36mInteractiveShell.run_cell_magic\u001b[0;34m(self, magic_name, line, cell)\u001b[0m\n\u001b[1;32m   2541\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbuiltin_trap:\n\u001b[1;32m   2542\u001b[0m     args \u001b[38;5;241m=\u001b[39m (magic_arg_s, cell)\n\u001b[0;32m-> 2543\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   2545\u001b[0m \u001b[38;5;66;03m# The code below prevents the output from being displayed\u001b[39;00m\n\u001b[1;32m   2546\u001b[0m \u001b[38;5;66;03m# when using magics with decorator @output_can_be_silenced\u001b[39;00m\n\u001b[1;32m   2547\u001b[0m \u001b[38;5;66;03m# when the last Python token in the expression is a ';'.\u001b[39;00m\n\u001b[1;32m   2548\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(fn, magic\u001b[38;5;241m.\u001b[39mMAGIC_OUTPUT_CAN_BE_SILENCED, \u001b[38;5;28;01mFalse\u001b[39;00m):\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/magics/execution.py:1209\u001b[0m, in \u001b[0;36mExecutionMagics.timeit\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n\u001b[1;32m   1207\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m index \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;241m0\u001b[39m, \u001b[38;5;241m10\u001b[39m):\n\u001b[1;32m   1208\u001b[0m     number \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m10\u001b[39m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m index\n\u001b[0;32m-> 1209\u001b[0m     time_number \u001b[38;5;241m=\u001b[39m \u001b[43mtimer\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtimeit\u001b[49m\u001b[43m(\u001b[49m\u001b[43mnumber\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1210\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m time_number \u001b[38;5;241m>\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0.2\u001b[39m:\n\u001b[1;32m   1211\u001b[0m         \u001b[38;5;28;01mbreak\u001b[39;00m\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/magics/execution.py:173\u001b[0m, in \u001b[0;36mTimer.timeit\u001b[0;34m(self, number)\u001b[0m\n\u001b[1;32m    171\u001b[0m gc\u001b[38;5;241m.\u001b[39mdisable()\n\u001b[1;32m    172\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 173\u001b[0m     timing \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minner\u001b[49m\u001b[43m(\u001b[49m\u001b[43mit\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtimer\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    174\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m    175\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m gcold:\n",
+      "File \u001b[0;32m:7\u001b[0m, in \u001b[0;36minner\u001b[0;34m(_it, _timer)\u001b[0m\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'raster' is not defined"
      ]
     }
    ],
    "source": [
-    "%%time\n",
+    "%%timeit\n",
     "\n",
     "# Register GDAL format drivers and configuration options with a\n",
     "# context manager.\n",
@@ -1321,28 +1273,37 @@
    "id": "9f77319c-80a1-4060-92ff-32a5040f834f",
    "metadata": {},
    "source": [
-    "## Create a CoG without overview"
+    "## Create a CoG without overview\n",
+    "\n",
+    "CoG (Cloud Optimized Tiff) is an lzw compressed **tiled** product which optimizes the computing by allowing calculations by tile (allowing parallelisation), the **compute performance is greatly improved**"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 22,
+   "execution_count": 26,
    "id": "906973ba-32f1-473c-93dd-302296177950",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "CPU times: user 1min 11s, sys: 5.54 s, total: 1min 17s\n",
-      "Wall time: 1min 22s\n"
+     "ename": "NameError",
+     "evalue": "name 'raster' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[26], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mget_ipython\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_cell_magic\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mtimeit\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43mwith rasterio.Env():\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    profile = raster.profile\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    profile.update(tiled=True, compress=\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mlzw\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m, blockxsize=512, blockysize=512)\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m    with rasterio.open(phr_product_cog, \u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mw\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m, **profile) as dst:\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m        dst.write(raster.read(1), 1)\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py:2543\u001b[0m, in \u001b[0;36mInteractiveShell.run_cell_magic\u001b[0;34m(self, magic_name, line, cell)\u001b[0m\n\u001b[1;32m   2541\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbuiltin_trap:\n\u001b[1;32m   2542\u001b[0m     args \u001b[38;5;241m=\u001b[39m (magic_arg_s, cell)\n\u001b[0;32m-> 2543\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   2545\u001b[0m \u001b[38;5;66;03m# The code below prevents the output from being displayed\u001b[39;00m\n\u001b[1;32m   2546\u001b[0m \u001b[38;5;66;03m# when using magics with decorator @output_can_be_silenced\u001b[39;00m\n\u001b[1;32m   2547\u001b[0m \u001b[38;5;66;03m# when the last Python token in the expression is a ';'.\u001b[39;00m\n\u001b[1;32m   2548\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(fn, magic\u001b[38;5;241m.\u001b[39mMAGIC_OUTPUT_CAN_BE_SILENCED, \u001b[38;5;28;01mFalse\u001b[39;00m):\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/magics/execution.py:1209\u001b[0m, in \u001b[0;36mExecutionMagics.timeit\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n\u001b[1;32m   1207\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m index \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;241m0\u001b[39m, \u001b[38;5;241m10\u001b[39m):\n\u001b[1;32m   1208\u001b[0m     number \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m10\u001b[39m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m index\n\u001b[0;32m-> 1209\u001b[0m     time_number \u001b[38;5;241m=\u001b[39m \u001b[43mtimer\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtimeit\u001b[49m\u001b[43m(\u001b[49m\u001b[43mnumber\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1210\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m time_number \u001b[38;5;241m>\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0.2\u001b[39m:\n\u001b[1;32m   1211\u001b[0m         \u001b[38;5;28;01mbreak\u001b[39;00m\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/magics/execution.py:173\u001b[0m, in \u001b[0;36mTimer.timeit\u001b[0;34m(self, number)\u001b[0m\n\u001b[1;32m    171\u001b[0m gc\u001b[38;5;241m.\u001b[39mdisable()\n\u001b[1;32m    172\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 173\u001b[0m     timing \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minner\u001b[49m\u001b[43m(\u001b[49m\u001b[43mit\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtimer\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    174\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m    175\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m gcold:\n",
+      "File \u001b[0;32m:2\u001b[0m, in \u001b[0;36minner\u001b[0;34m(_it, _timer)\u001b[0m\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'raster' is not defined"
      ]
     }
    ],
    "source": [
-    "%%time\n",
+    "%%timeit\n",
     "\n",
     "with rasterio.Env():\n",
     "    profile = raster.profile\n",
@@ -1366,12 +1327,35 @@
    "id": "a72ca70d-0b0f-49e4-9e7e-f291d1e8cd96",
    "metadata": {},
    "source": [
-    "## Size and compute time gain after compression"
+    "## Size and compute time gain after compression\n",
+    "\n",
+    "In this section we compare the size on disk of the original product vs the compressed ones "
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 24,
+   "id": "32241de3-aa3f-4fb5-84f6-0ebc977ed281",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "13G\t/work/scratch/data/tanguyy/public/PHR_OTB/Marmande/IMG_PHR1A_PMS_202304151100243_SEN_6967638101-1_R1C1_wnodata.tif\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Size of the original product\n",
+    "!du -sh {phr_product}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 30,
    "id": "1071ba31-2e9b-4da5-9d94-9e6ea9a5b1db",
    "metadata": {
     "tags": []
@@ -1381,12 +1365,6 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "512\t/work/scratch/data/romaint/input_greenit/GreenIT.slurm\n",
-      "1.5G\t/work/scratch/data/romaint/input_greenit/SENTINEL2A_20210415-105852-555_L2A_T31TCJ_C_V3-0_FRE_STACK.tif\n",
-      "2.8G\t/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1\n",
-      "2.0G\t/work/scratch/data/romaint/input_greenit/SENTINEL2B_20240822-105857-973_L2A_T31TCJ_C_V3-1.tar.gz\n",
-      "512\t/work/scratch/data/romaint/input_greenit/greenit_monitored.sh\n",
-      "299M\t/work/scratch/data/romaint/input_greenit/image_PHR_marmande.zip\n",
       "1.6G\t/work/scratch/data/romaint/input_greenit/phr_cog.tif\n",
       "35M\t/work/scratch/data/romaint/input_greenit/phr_cog_extract.tif\n",
       "1.7G\t/work/scratch/data/romaint/input_greenit/phr_lzw.tif\n"
@@ -1394,7 +1372,18 @@
     }
    ],
    "source": [
-    "!du -sh {input_dir}/*"
+    "# Size of the compressed products\n",
+    "!du -sh {input_dir}/phr_*.tif"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3f2d0345-ee4b-4341-8444-7bd9f4fdbf48",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "The CoG and Lzw files are weighing ~15% of the original product size, the difference between lzw and cog is negligeable, but the CoG is **tiled** this is very important for the parallelisation efficiency"
    ]
   },
   {
@@ -1407,7 +1396,24 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": null,
+   "id": "533fc7e1-9519-46aa-8020-5b3885a8bc57",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%timeit\n",
+    "\n",
+    "# Compute NDVI for the LZW compressed file\n",
+    "reading_chunks = (-1,2048,2048)\n",
+    "data_array_lzw = rxr.open_rasterio(phr_product_lzw, chunks=reading_chunks, lock=False)\n",
+    "ndvi_array = (data_array_lzw[3] - data_array_lzw[0]) / (data_array_lzw[3] + data_array_lzw[0])\n",
+    "output_file = Path(f\"{output_dir}/ndvi_lzw.tif\")\n",
+    "ndvi_array.rio.to_raster(output_file)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
    "id": "8e7a9fdb-3844-432a-bc43-fbcc2c5169cd",
    "metadata": {
     "tags": []
@@ -1418,20 +1424,19 @@
      "output_type": "stream",
      "text": [
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
-      "  dataset = writer(\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Time to compute with LZW compressed file = 148.64071926195174s\n"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
       "  dataset = writer(\n"
      ]
@@ -1440,27 +1445,19 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Time to compute with CoG compressed file = 50.54504857957363s\n"
+      "26 s ± 400 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
      ]
     }
    ],
    "source": [
-    "reading_chunks = (-1,2048,2048)\n",
-    "start=time.perf_counter()\n",
-    "data_array_lzw = rxr.open_rasterio(phr_product_lzw, chunks=reading_chunks, lock=False)\n",
-    "ndvi_array = (data_array_lzw[3] - data_array_lzw[0]) / (data_array_lzw[3] + data_array_lzw[0])\n",
-    "output_file = Path(f\"{output_dir}/ndvi_lzw.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file)\n",
-    "end=time.perf_counter()\n",
-    "print(\"Time to compute with LZW compressed file = {}s\".format((end - start)))\n",
+    "%%timeit\n",
     "\n",
-    "start=time.perf_counter()\n",
+    "# Compute NDVI for the Cog compressed file\n",
+    "reading_chunks = (-1,2048,2048)\n",
     "data_array_cog = rxr.open_rasterio(phr_product_cog, chunks=reading_chunks, lock=False)\n",
     "ndvi_array = (data_array_cog[3] - data_array_cog[0]) / (data_array_cog[3] + data_array_cog[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_cog.tif\")\n",
-    "ndvi_array.rio.to_raster(output_file)\n",
-    "end=time.perf_counter()\n",
-    "print(\"Time to compute with CoG compressed file = {}s\".format((end - start)))"
+    "ndvi_array.rio.to_raster(output_file)"
    ]
   },
   {
@@ -1490,30 +1487,14 @@
     "If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask\n",
     "
    \n",
     "\n",
-    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is > 3 Go**\n",
+    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is at least 5x lower than your RAM size**\n",
     "\n",
     "
    \n",
     "If your product is multi-band, use RIOXarray and/or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances\n",
     "
    \n",
     "\n",
     "\n",
-    "Let's see a use case where parralellisation methods are worth it"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "56e25516-3d90-4ee6-8c4b-085b94d3d096",
-   "metadata": {
-    "tags": []
-   },
-   "source": [
-    "# Use case : I have to use multiple products which average size is > 10Go\n",
-    "\n",
-    "In this example we will use a PLEIADES product (which original size is 13Go) in CoG format which size is 1.6 Go\n",
-    "\n",
-    "This is the typical use case where dask or rioxarray with parallel read and write support is necessary to speedup your processing. Let's see what are the recommended parameters to optimize your processing with big data\n",
-    "\n",
-    "## Using dask and optimizing its parameters"
+    "Now Let's see how to optimize dask parameters"
    ]
   },
   {
@@ -1521,12 +1502,14 @@
    "id": "d3ab5833-1672-478a-8b26-0ed2bc4d6ee1",
    "metadata": {},
    "source": [
+    "## Using dask and optimizing its parameters\n",
+    "\n",
     "### Find the right chunk size for dask\n",
     "\n",
     "Chunk size is becoming very important when your data size grows. You can let chunks=True to dask which will automatically determine a chunk size\n",
     "Most of the time this chunk size is coherent but users can tweak it to be more efficient.\n",
     "\n",
-    "Also be careful about the compute() method, which is not recommended when your data size => 10Go, the memory consumption increases a lot"
+    "Also be careful about the compute() method, which is not recommended when your data size higher than RAM amount / 5"
    ]
   },
   {
@@ -1920,7 +1903,7 @@
        "Attributes:\n",
        "    _FillValue:    0\n",
        "    scale_factor:  1.0\n",
-       "    add_offset:    0.0
    "
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -2064,7 +2047,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 11,
    "id": "b8fb6e29-0a84-4ae0-882b-b973af22b054",
    "metadata": {
     "tags": []
@@ -2082,7 +2065,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Time to compute with automatic chunk size = 49.639332056976855s\n"
+      "Time to compute with automatic chunk size = 42.21393999271095s\n"
      ]
     }
    ],
@@ -2097,7 +2080,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 8,
    "id": "8940e64c-1de7-4122-828b-e03a9f43a60b",
    "metadata": {
     "tags": []
@@ -2486,7 +2469,7 @@
        "Attributes:\n",
        "    _FillValue:    0\n",
        "    scale_factor:  1.0\n",
-       "    add_offset:    0.0"
+       "      dtype='float64', name='y', length=41663))
  • _FillValue :
    0
    scale_factor :
    1.0
    add_offset :
    0.0
    • "
       ],
       "text/plain": [
        " Size: 13GB\n",
@@ -2664,7 +2647,7 @@
        "    add_offset:    0.0"
       ]
      },
-     "execution_count": 21,
+     "execution_count": 8,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -2677,7 +2660,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 22,
+   "execution_count": 9,
    "id": "58bad330-9498-48ac-b3a0-cad211aaf998",
    "metadata": {
     "tags": []
@@ -2695,7 +2678,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Time to compute with manual chunk size = 43.15807781834155s\n"
+      "Time to compute with manual chunk size = 35.88091850373894s\n"
      ]
     }
    ],
@@ -2718,7 +2701,7 @@
     "With this example we demonstrated that using personalized chunk size can give you immediatly a 10% performance improvment, we highly encourage using chunks in coherence with your CoG size, using multi-band chunks.\n",
     "\n",
     "
    \n",
-    "Prefer using rxr.to_raster() instead of dask.compute() when your product size is getting bigger as the later seems to be more CPU consuming\n",
+    "Prefer using rxr.to_raster() instead of dask.compute() when your product size is getting bigger as the later seems to be more RAM and CPU consuming\n",
     "
    \n",
     "\n",
     "#### Chunk size recommandation for multi band products\n",
@@ -2731,7 +2714,7 @@
     "\n",
     "#### Don't hesitate to test different chunk sizes before making a conclusion\n",
     "\n",
-    "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of the tile size for read/write efficiency in RAM.\n",
+    "In our case we used chunks of 32Mo, it is a best practice to use chunks size multiple of the tile size (multiple of 16 works well) for read/write efficiency in RAM.\n",
     "Tests have been done with chunks > 128Mo, bigger chunks tends to slow down the process.\n",
     "\n",
     "
    \n",
@@ -2781,17 +2764,14 @@
     }
    ],
    "source": [
-    "%%time\n",
+    "%%timeit\n",
     "\n",
-    "start = time.perf_counter()\n",
     "reading_chunks = (-1,2048,2048)\n",
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "ndvi_phr.compute()\n",
     "output_file = Path(f\"{output_dir}/ndvi_phr.tif\")\n",
-    "ndvi_phr.rio.to_raster(output_file)\n",
-    "end = time.perf_counter()\n",
-    "print(\"Time to compute with RIOXarray + dask compute + product > 5Go = {}s\".format((end - start)))"
+    "ndvi_phr.rio.to_raster(output_file)"
    ]
   },
   {
@@ -3374,7 +3354,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 26,
+   "execution_count": null,
    "id": "909e5746-eee9-4f0c-a840-d3e8850dee83",
    "metadata": {
     "tags": []
@@ -3384,29 +3364,23 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
       "  dataset = writer(\n"
      ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Time to compute with RIOXarray + tiled write + product > 5Go = 51.08986523933709s\n"
-     ]
     }
    ],
    "source": [
-    "start = time.perf_counter()\n",
+    "%%timeit\n",
+    "\n",
     "reading_chunks = (-1,2048,2048)\n",
     "input_data_array = rxr.open_rasterio(phr_product_cog,chunks=reading_chunks,lock=False)\n",
     "ndvi_phr = (input_data_array[3] - input_data_array[0]) / (input_data_array[3] + input_data_array[0])\n",
     "output_file = Path(f\"{output_dir}/ndvi_phr_rxr.tif\")\n",
     "# Don't call dask.compute, use rio.to_raster to launch the dask computing on the background\n",
     "# Add the Tiled=true parameter to rxr to speed up the disk write\n",
-    "ndvi_phr.rio.to_raster(output_file,tiled=True)\n",
-    "end = time.perf_counter()\n",
-    "print(\"Time to compute with RIOXarray + tiled write + product > 5Go = {}s\".format((end - start)))"
+    "ndvi_phr.rio.to_raster(output_file,tiled=True)"
    ]
   },
   {

From 987fbcee338191948f1fa73f617337125d13bf0d Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Fri, 4 Apr 2025 14:00:11 +0000
Subject: [PATCH 36/37] Continue refactoring

---
 tuto_greenit.ipynb | 166 +++++++++++++++++++++++++++------------------
 1 file changed, 100 insertions(+), 66 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index da3c11e..627d715 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -71,7 +71,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 3,
    "id": "5d7f7644-d20f-4b63-8fa9-76cb1a5c5e4a",
    "metadata": {
     "tags": []
@@ -250,7 +250,7 @@
     "\n",
     "Depending on your project, the data format will vary : it is very important to choose a library that can fully handle your product.\n",
     "\n",
-    "A Tutorial is available [here](https://github.com/CNES/pluto-tuto/pull/3) for a detailed walkthrough on how to choose the right library"
+    "A Tutorial is available [here](https://github.com/CNES/pluto-tuto/tree/main/data_type) for a detailed walkthrough on how to choose the right library"
    ]
   },
   {
@@ -458,7 +458,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 4,
    "id": "219cc690-5c64-4c56-8ba5-ce8333bc6c62",
    "metadata": {
     "tags": []
@@ -484,17 +484,16 @@
     "    \"\"\"\n",
     "    results = []\n",
     "    for raster_path in raster_paths:\n",
-    "        with rxr.open_rasterio(raster_path, chunks=chunks, lock=lock) as tif:\n",
-    "            reprojection = tif\n",
-    "            transform = reprojection.rio.transform()\n",
-    "            crs = reprojection.rio.crs\n",
+    "        with rxr.open_rasterio(raster_path, chunks=chunks, lock=lock) as data:\n",
+    "            transform = data.rio.transform()\n",
+    "            crs = data.rio.crs\n",
     "            x_res = transform[0]\n",
     "            y_res = -transform[4]\n",
     "            top_left_x = transform[2]\n",
     "            top_left_y = transform[5]\n",
-    "            results.append(reprojection)\n",
+    "            results.append(data)\n",
     "\n",
-    "    return da.concatenate(results), x_res, y_res, top_left_x, top_left_y, crs\n",
+    "    return da.concatenate(data), x_res, y_res, top_left_x, top_left_y, crs\n",
     "\n",
     "def create_raster(data: np.ndarray, output_file: Path, x_res, y_res, top_left_x, top_left_y, crs):\n",
     "    transform = Affine.translation(top_left_x, top_left_y) * Affine.scale(x_res, -y_res)\n",
@@ -523,28 +522,18 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 10,
    "id": "f89bb471-bf40-404a-82e1-ce4e77b14e21",
    "metadata": {
     "scrolled": true,
     "tags": []
    },
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/distributed/node.py:187: UserWarning: Port 8787 is already in use.\n",
-      "Perhaps you already have a cluster running?\n",
-      "Hosting the HTTP server on port 36045 instead\n",
-      "  warnings.warn(\n"
-     ]
-    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Dask Dashboard:  http://127.0.0.1:36045/status\n"
+      "Dask Dashboard:  http://127.0.0.1:8787/status\n"
      ]
     },
     {
@@ -554,7 +543,7 @@
        "    
     
    \n",
        "    
    \n",
        "        
    Client
    \n",
-       "        
    Client-5ae0bd29-ef98-11ef-addd-e43d1a347da2
    \n",
+       "        
    Client-54b7d9df-1153-11f0-85a7-b02628fb9b98
    \n",
        "        \n",
        "\n",
        "        \n",
@@ -567,7 +556,7 @@
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
@@ -576,7 +565,7 @@
        "        
    \n",
-       "                    Dashboard:  http://127.0.0.1:36045/status\n",
+       "                    Dashboard:  http://127.0.0.1:8787/status\n",
        "                
    \n",
        "\n",
        "        \n",
-       "            \n",
        "        \n",
@@ -589,11 +578,11 @@
        "    
    \n",
        "    
    \n",
        "        
    LocalCluster
    \n",
-       "        
    01ab761d
    \n",
+       "        
    6b54f300
    \n",
        "        \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
        "                \n",
        "                \n",
        "            \n",
        "            \n",
@@ -626,11 +615,11 @@
        "        
     
    \n",
        "        
    \n",
        "            
    Scheduler
    \n",
-       "            
    Scheduler-a3b547c6-a54a-46ae-8316-982ab39af35a
    \n",
+       "            
    Scheduler-b6d3792d-ff58-468f-9104-a536d3b9bccc
    \n",
        "            
    \n",
-       "                    Dashboard: http://127.0.0.1:36045/status\n",
+       "                    Dashboard: http://127.0.0.1:8787/status\n",
        "                \n",
        "                    Workers: 4\n",
@@ -601,10 +590,10 @@
        "            
    \n",
-       "                    Total threads: 4\n",
+       "                    Total threads: 8\n",
        "                \n",
-       "                    Total memory: 28.00 GiB\n",
+       "                    Total memory: 56.00 GiB\n",
        "                
    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                    \n",
        "                    \n",
        "                \n",
        "                \n",
@@ -649,7 +638,7 @@
        "                        Started: Just now\n",
        "                    \n",
        "                    \n",
        "                \n",
        "            
    \n",
-       "                        Comm: tcp://127.0.0.1:34383\n",
+       "                        Comm: tcp://127.0.0.1:33095\n",
        "                    \n",
        "                        Workers: 4\n",
@@ -638,10 +627,10 @@
        "                
    \n",
-       "                        Dashboard: http://127.0.0.1:36045/status\n",
+       "                        Dashboard: http://127.0.0.1:8787/status\n",
        "                    \n",
-       "                        Total threads: 4\n",
+       "                        Total threads: 8\n",
        "                    
    \n",
-       "                        Total memory: 28.00 GiB\n",
+       "                        Total memory: 56.00 GiB\n",
        "                    
    \n",
@@ -675,26 +664,26 @@
        "                            Comm:  tcp://127.0.0.1:46371\n",
        "                        
    		\n",
-       "                            Total threads:  1\n",
+       "                            Total threads:  2\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:45497/status\n",
+       "                            Dashboard:  http://127.0.0.1:36957/status\n",
        "                        \n",
-       "                            Memory:  7.00 GiB\n",
+       "                            Memory:  14.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:44907\n",
+       "                            Nanny:  tcp://127.0.0.1:32793\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-33186461/dask-scratch-space/worker-wrjzk4kn\n",
+       "                            Local directory:  /tmp/slurm-35311095/dask-scratch-space/worker-xm63qzgm\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -762,29 +751,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:42037\n",
+       "                            Comm:  tcp://127.0.0.1:39359\n",
        "                        \n",
-       "                            Total threads:  1\n",
+       "                            Total threads:  2\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:41679/status\n",
+       "                            Dashboard:  http://127.0.0.1:39813/status\n",
        "                        \n",
-       "                            Memory:  7.00 GiB\n",
+       "                            Memory:  14.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:34927\n",
+       "                            Nanny:  tcp://127.0.0.1:45545\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-33186461/dask-scratch-space/worker-z8wop2y_\n",
+       "                            Local directory:  /tmp/slurm-35311095/dask-scratch-space/worker-livr9vz6\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -807,29 +796,29 @@
        "                
    \n",
-       "                            Comm:  tcp://127.0.0.1:36507\n",
+       "                            Comm:  tcp://127.0.0.1:35347\n",
        "                        \n",
-       "                            Total threads:  1\n",
+       "                            Total threads:  2\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:39185/status\n",
+       "                            Dashboard:  http://127.0.0.1:45133/status\n",
        "                        \n",
-       "                            Memory:  7.00 GiB\n",
+       "                            Memory:  14.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:37391\n",
+       "                            Nanny:  tcp://127.0.0.1:44349\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-33186461/dask-scratch-space/worker-nqzwkhoz\n",
+       "                            Local directory:  /tmp/slurm-35311095/dask-scratch-space/worker-opqdz76r\n",
        "                        
    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                        \n",
        "                    \n",
        "                    \n",
        "                        \n",
        "                    \n",
        "\n",
@@ -856,10 +845,10 @@
        ""
       ],
       "text/plain": [
-       ""
+       ""
       ]
      },
-     "execution_count": 12,
+     "execution_count": 10,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1042,7 +1031,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 16,
    "id": "5f92ce7a-4e9a-4b78-b1b5-2295d939faac",
    "metadata": {
     "tags": []
@@ -1108,23 +1097,35 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 17,
    "id": "184bc745-270b-453c-9e80-34918fbd81d2",
    "metadata": {
     "tags": []
    },
    "outputs": [
     {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "1.99 s ± 130 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
+     "ename": "IndexError",
+     "evalue": "Too many indices for array",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mIndexError\u001b[0m                                Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[17], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mget_ipython\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_cell_magic\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mtimeit\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m# Use a chunk size to have \u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43mchunks_size = (-1,2200,2200)\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43minput_data, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], chunks_size , False)\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43mndvi_array = compute_ndvi_dask(input_data[0],input_data[1])\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43moutput_file = Path(f\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;132;43;01m{output_dir}\u001b[39;49;00m\u001b[38;5;124;43m/ndvi_dask.tif\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m)\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43mcreate_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py:2543\u001b[0m, in \u001b[0;36mInteractiveShell.run_cell_magic\u001b[0;34m(self, magic_name, line, cell)\u001b[0m\n\u001b[1;32m   2541\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbuiltin_trap:\n\u001b[1;32m   2542\u001b[0m     args \u001b[38;5;241m=\u001b[39m (magic_arg_s, cell)\n\u001b[0;32m-> 2543\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   2545\u001b[0m \u001b[38;5;66;03m# The code below prevents the output from being displayed\u001b[39;00m\n\u001b[1;32m   2546\u001b[0m \u001b[38;5;66;03m# when using magics with decorator @output_can_be_silenced\u001b[39;00m\n\u001b[1;32m   2547\u001b[0m \u001b[38;5;66;03m# when the last Python token in the expression is a ';'.\u001b[39;00m\n\u001b[1;32m   2548\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(fn, magic\u001b[38;5;241m.\u001b[39mMAGIC_OUTPUT_CAN_BE_SILENCED, \u001b[38;5;28;01mFalse\u001b[39;00m):\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/magics/execution.py:1209\u001b[0m, in \u001b[0;36mExecutionMagics.timeit\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n\u001b[1;32m   1207\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m index \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;241m0\u001b[39m, \u001b[38;5;241m10\u001b[39m):\n\u001b[1;32m   1208\u001b[0m     number \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m10\u001b[39m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m index\n\u001b[0;32m-> 1209\u001b[0m     time_number \u001b[38;5;241m=\u001b[39m \u001b[43mtimer\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtimeit\u001b[49m\u001b[43m(\u001b[49m\u001b[43mnumber\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1210\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m time_number \u001b[38;5;241m>\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0.2\u001b[39m:\n\u001b[1;32m   1211\u001b[0m         \u001b[38;5;28;01mbreak\u001b[39;00m\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/IPython/core/magics/execution.py:173\u001b[0m, in \u001b[0;36mTimer.timeit\u001b[0;34m(self, number)\u001b[0m\n\u001b[1;32m    171\u001b[0m gc\u001b[38;5;241m.\u001b[39mdisable()\n\u001b[1;32m    172\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 173\u001b[0m     timing \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minner\u001b[49m\u001b[43m(\u001b[49m\u001b[43mit\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtimer\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    174\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m    175\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m gcold:\n",
+      "File \u001b[0;32m:4\u001b[0m, in \u001b[0;36minner\u001b[0;34m(_it, _timer)\u001b[0m\n",
+      "Cell \u001b[0;32mIn[16], line 4\u001b[0m, in \u001b[0;36mcompute_ndvi_dask\u001b[0;34m(input_data_red, input_data_nir)\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21mcompute_ndvi_dask\u001b[39m(input_data_red: DataArray,input_data_nir: DataArray):\n\u001b[0;32m----> 4\u001b[0m     ndvi_array \u001b[38;5;241m=\u001b[39m \u001b[43m(\u001b[49m\u001b[43m(\u001b[49m\u001b[43minput_data_nir\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m-\u001b[39;49m\u001b[43m \u001b[49m\u001b[43minput_data_red\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m/\u001b[39;49m\u001b[43m \u001b[49m\u001b[43m(\u001b[49m\u001b[43minput_data_nir\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m+\u001b[39;49m\u001b[43m \u001b[49m\u001b[43minput_data_red\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m:\u001b[49m\u001b[43m,\u001b[49m\u001b[43m:\u001b[49m\u001b[43m]\u001b[49m\n\u001b[1;32m      5\u001b[0m     ndvi_array\u001b[38;5;241m.\u001b[39mcompute()\n\u001b[1;32m      6\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m ndvi_array\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/dask/array/core.py:1991\u001b[0m, in \u001b[0;36mArray.__getitem__\u001b[0;34m(self, index)\u001b[0m\n\u001b[1;32m   1983\u001b[0m     index \u001b[38;5;241m=\u001b[39m (index,)\n\u001b[1;32m   1985\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21;01mdask\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01marray\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mslicing\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m (\n\u001b[1;32m   1986\u001b[0m     normalize_index,\n\u001b[1;32m   1987\u001b[0m     slice_with_bool_dask_array,\n\u001b[1;32m   1988\u001b[0m     slice_with_int_dask_array,\n\u001b[1;32m   1989\u001b[0m )\n\u001b[0;32m-> 1991\u001b[0m index2 \u001b[38;5;241m=\u001b[39m \u001b[43mnormalize_index\u001b[49m\u001b[43m(\u001b[49m\u001b[43mindex\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mshape\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1992\u001b[0m dependencies \u001b[38;5;241m=\u001b[39m {\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mname}\n\u001b[1;32m   1993\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m index2:\n",
+      "File \u001b[0;32m/usr/local/lib/python3.10/dist-packages/dask/array/slicing.py:826\u001b[0m, in \u001b[0;36mnormalize_index\u001b[0;34m(idx, shape)\u001b[0m\n\u001b[1;32m    824\u001b[0m idx \u001b[38;5;241m=\u001b[39m idx \u001b[38;5;241m+\u001b[39m (\u001b[38;5;28mslice\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m),) \u001b[38;5;241m*\u001b[39m (\u001b[38;5;28mlen\u001b[39m(shape) \u001b[38;5;241m-\u001b[39m n_sliced_dims)\n\u001b[1;32m    825\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m([i \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m idx \u001b[38;5;28;01mif\u001b[39;00m i \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m]) \u001b[38;5;241m>\u001b[39m \u001b[38;5;28mlen\u001b[39m(shape):\n\u001b[0;32m--> 826\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mIndexError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mToo many indices for array\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m    828\u001b[0m none_shape \u001b[38;5;241m=\u001b[39m []\n\u001b[1;32m    829\u001b[0m i \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0\u001b[39m\n",
+      "\u001b[0;31mIndexError\u001b[0m: Too many indices for array"
      ]
     }
    ],
    "source": [
     "%%timeit\n",
     "\n",
+    "# Use a chunk size to have \n",
     "chunks_size = (-1,2200,2200)\n",
     "input_data, x_res, y_res, top_left_x, top_left_y, crs = open_raster_and_get_metadata([s2_b4,s2_b8], chunks_size , False)\n",
     "ndvi_array = compute_ndvi_dask(input_data[0],input_data[1])\n",
@@ -1132,6 +1133,16 @@
     "create_raster(ndvi_array, output_file, x_res , y_res, top_left_x, top_left_y, crs)"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c7fb681a-2e90-45cf-87dd-53530f280c1c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ndvi_array"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "c9229d23-c553-4c30-8ae0-b21cee07879e",
@@ -1416,6 +1427,10 @@
    "execution_count": 29,
    "id": "8e7a9fdb-3844-432a-bc43-fbcc2c5169cd",
    "metadata": {
+    "collapsed": true,
+    "jupyter": {
+     "outputs_hidden": true
+    },
     "tags": []
    },
    "outputs": [
@@ -3354,7 +3369,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 31,
    "id": "909e5746-eee9-4f0c-a840-d3e8850dee83",
    "metadata": {
     "tags": []
@@ -3364,11 +3379,30 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
+      "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
+      "  dataset = writer(\n",
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
       "  dataset = writer(\n",
       "/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py:366: NotGeoreferencedWarning: The given matrix is equal to Affine.identity or its flipped counterpart. GDAL may ignore this matrix and save no geotransform without raising an error. This behavior is somewhat driver-specific.\n",
       "  dataset = writer(\n"
      ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "26.7 s ± 728 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
+     ]
     }
    ],
    "source": [
@@ -3456,7 +3490,7 @@
     "\n",
     "* Using parallelisation methods dask / RIOXarray leads to a huge time gain without precision loss with big multiband products\n",
     "\n",
-    "* You have to carefully choose your chunk size using a multiple of the cog size\n"
+    "* You have to carefully choose your chunk size using a multiple of the cog size"
    ]
   },
   {

From 36cb1f53af7cdb3f95988da1d57756456be4942b Mon Sep 17 00:00:00 2001
From: Thibaut ROMAIN 
Date: Tue, 8 Apr 2025 12:16:53 +0000
Subject: [PATCH 37/37] move sections and add precision on introductions

---
 tuto_greenit.ipynb | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/tuto_greenit.ipynb b/tuto_greenit.ipynb
index 627d715..24ff933 100644
--- a/tuto_greenit.ipynb
+++ b/tuto_greenit.ipynb
@@ -7,16 +7,18 @@
    "source": [
     "# Best practices for Green IT coding in python\n",
     "\n",
-    "In this tutorial, we will show good pratices for a green IT approach to code an image processing chain with efficiency in mind.\n",
-    "This notebook is divided in 3 different parts from begginer level to advanced level :\n",
+    "In this tutorial, we’ll explore best practices for a Green IT approach to coding an efficient image processing pipeline. The notebook is structured into three parts, progressing from beginner to advanced levels :\n",
     "\n",
     "* Good practices in Python code for image processing\n",
     "* Optimisation of an NDVI computing on a Sentinel-2 image and a Pleiades image\n",
     "* Image compression for performance optimisation\n",
     "* Comparison of performances between parallelisation methods and conclusions\n",
     "\n",
-    "In this notebook we will cover two use cases for NDVI computing, the first uses a Sentinel 2 image with single band files (~450 Mo of Data).\n",
-    "The second one will use a Pleiades image which is a big multiband file (13 Go of Data).\n",
+    "In this notebook we will cover two use cases for NDVI computing\n",
+    "\n",
+    "* the first uses a Sentinel 2 image with two single band files (~450 Mo of Data).\n",
+    "* The second one will use a Pleiades image which is a big multiband file (13 Go of Data).\n",
+    "\n",
     "In each of these use cases we will start from a basic NDVI calculation in standard python code, and optimize this code in different ways to see where you can have performance problems depending on your problematics\n",
     "\n",
     "Let's first define everything needed along this notebook"
@@ -65,7 +67,7 @@
     "## Work directories for the notebook\n",
     "\n",
     "Here we define the directories and the image files used along the notebook. \n",
-    "* A first use case uses a Sentinel 2 image with 2 files only (the two interesting bands to compute the NDVI b4 and b8)\n",
+    "* The first use case uses a Sentinel 2 image with 2 files only (the two interesting bands to compute the NDVI b4 and b8)\n",
     "* The second use case uses a Pleiades image with 1 multiband file, we will compress it and write it as a COG"
    ]
   },
@@ -1502,7 +1504,7 @@
     "If your product is single band or its size is less than your RAM size divided by the number of CPUs (not threads), it is not recommended to use parallelisation libraries such as dask\n",
     "\n",
     "\n",
-    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is at least 5x lower than your RAM size**\n",
+    "Let's say you have an average computer that has 16 or 32Go RAM : with this amount of ram we recommend using dask if your **product size is at least your avaialble RAM / 5**\n",
     "\n",
     "
    \n",
     "If your product is multi-band, use RIOXarray and/or Dask, we will see in the next chapter that you can tune the parameters to improve the read/write performances\n",

    \n",
-       "                            Comm:  tcp://127.0.0.1:45933\n",
+       "                            Comm:  tcp://127.0.0.1:44725\n",
        "                        \n",
-       "                            Total threads:  1\n",
+       "                            Total threads:  2\n",
        "                        
    \n",
-       "                            Dashboard:  http://127.0.0.1:46085/status\n",
+       "                            Dashboard:  http://127.0.0.1:41691/status\n",
        "                        \n",
-       "                            Memory:  7.00 GiB\n",
+       "                            Memory:  14.00 GiB\n",
        "                        
    \n",
-       "                            Nanny:  tcp://127.0.0.1:44855\n",
+       "                            Nanny:  tcp://127.0.0.1:33897\n",
        "                        
    \n",
-       "                            Local directory:  /tmp/slurm-33186461/dask-scratch-space/worker-x7prb8pz\n",
+       "                            Local directory:  /tmp/slurm-35311095/dask-scratch-space/worker-1lo4sh4_\n",
        "