{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Genetic Feature Selection nodes in TPOT\n", "\n", "TPOT can use evolutionary algorithms to optimize feature selection simultaneously with pipeline optimization. It includes two node search spaces with different feature selection strategies: FSSNode and GeneticFeatureSelectorNode. \n", "\n", "1. FSSNode - (Feature Set Selector) This node is useful if you have a list of predefined feature sets you want to select from. Each FeatureSetSelector Node will select a single group of features to be passed to the next step in the pipeline. Note that FSSNode does not create its own subset of features and does not mix/match multiple predefined feature sets.\n", "\n", "2. GeneticFeatureSelectorNode—Whereas the FSSNode selects from a predefined list of subsets of features, this node uses evolutionary algorithms to optimize a novel subset of features from scratch. This is useful where there is no predefined grouping of features. \n", "\n", "This tutorial focuses on FSSNode. See Tutorial 5 for more information on GeneticFeatureSelectorNode.\n", "\n", "It may also be beneficial to pair these search spaces with a secondary objective function to minimize complexity. That would encourage TPOT to try to produce the simplest pipeline with the fewest number of features.\n", "\n", "tpot.objectives.number_of_nodes_objective - This can be used as an other_objective_function that counts the number of nodes.\n", "\n", "tpot.objectives.complexity_scorer - This is a scorer that tries to count the total number of learned parameters (number of coefficients, number of nodes in decision trees, etc.).\n" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Feature Set Selector\n", "\n", "The FeatureSetSelector is a subclass of sklearn.feature_selection.SelectorMixin that simply returns the manually specified columns. The parameter sel_subset specifies the name or index of the column that it selects. The transform function then simply indexes and returns the selected columns. You can also optionally name the group with the name parameter, though this is only for note keeping and does is not used by the class.\n", "\n", "\n", " sel_subset: list or int\n", " If X is a dataframe, items in sel_subset list must correspond to column names\n", " If X is a numpy array, items in sel_subset list must correspond to column indexes\n", " int: index of a single column\n", "\n", "\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "original DataFrame\n", " a b c d e f\n", "0 0 1 2 3 4 5\n", "1 0 1 2 3 4 5\n", "2 0 1 2 3 4 5\n", "3 0 1 2 3 4 5\n", "4 0 1 2 3 4 5\n", "5 0 1 2 3 4 5\n", "6 0 1 2 3 4 5\n", "7 0 1 2 3 4 5\n", "8 0 1 2 3 4 5\n", "9 0 1 2 3 4 5\n", "Transformed Data\n", "[[0 1 2]\n", " [0 1 2]\n", " [0 1 2]\n", " [0 1 2]\n", " [0 1 2]\n", " [0 1 2]\n", " [0 1 2]\n", " [0 1 2]\n", " [0 1 2]\n", " [0 1 2]]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/opt/anaconda3/envs/tpotenv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", " from .autonotebook import tqdm as notebook_tqdm\n" ] } ], "source": [ "import tpot\n", "import pandas as pd\n", "import numpy as np\n", "#make a dataframe with columns a,b,c,d,e,f\n", "\n", "#numpy array where columns are 1,2,3,4,5,6\n", "data = np.repeat([np.arange(6)],10,0)\n", "\n", "df = pd.DataFrame(data,columns=['a','b','c','d','e','f'])\n", "fss = tpot.builtin_modules.FeatureSetSelector(name='test',sel_subset=['a','b','c'])\n", "\n", "print(\"original DataFrame\")\n", "print(df)\n", "print(\"Transformed Data\")\n", "print(fss.fit_transform(df))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# FSSNode\n", "\n", "The `FSSNode` is a node search space that simply selects one feature set from a list of feature sets. This works identically to the EstimatorNode, but provides a easier interface for defining the feature sets.\n", "\n", "Note that the FSS is only well defined when used as the first step in a pipeline. This is because downstream nodes will receive different transformations of the data such that the original indexes no longer correspond to the same columns in the transformed data.\n", "\n", "The `FSSNode` takes in a single parameter `subsets` which defines the groups of features. There are four ways of defining the subsets. \n", "\n", " subsets : str or list, default=None\n", " Sets the subsets that the FeatureSetSeletor will select from if set as an option in one of the configuration dictionaries. \n", " Features are defined by column names if using a Pandas data frame, or ints corresponding to indexes if using numpy arrays.\n", " - str : If a string, it is assumed to be a path to a csv file with the subsets. \n", " The first column is assumed to be the name of the subset and the remaining columns are the features in the subset.\n", " - list or np.ndarray : If a list or np.ndarray, it is assumed to be a list of subsets (i.e a list of lists).\n", " - dict : A dictionary where keys are the names of the subsets and the values are the list of features.\n", " - int : If an int, it is assumed to be the number of subsets to generate. Each subset will contain one feature.\n", " - None : If None, each column will be treated as a subset. One column will be selected per subset.\n", "\n", "\n", "Lets say you want to have three groups of features, each with three columns each. The following examples are equivalent:\n", "\n", "### str\n", "\n", "sel_subsets=simple_fss.csv\n", "\n", "\n", " \\# simple_fss.csv\n", " group_one, 1,2,3\n", " group_two, 4,5,6\n", " group_three, 7,8,9\n", "\n", "\n", "### dict\n", "\n", "\n", "sel_subsets = { \"group_one\" : [1,2,3],\n", " \"group_two\" : [4,5,6],\n", " \"group_three\" : [7,8,9],\n", " }\n", "\n", "\n", "### list\n", "\n", "\n", "sel_subsets = [[1,2,3],\n", "[4,5,6],\n", "[7,8,9]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Examples\n", "\n", "For these examples, we create a dummy dataset where the first six columns are informative and the rest are uninformative." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
abcdefghijkl
02.315814-3.427720-1.314654-1.508737-0.3009320.0894480.3276510.3290220.8574950.7342380.2572180.652350
1-0.191001-1.3969220.149488-1.730145-0.3949320.5197120.8077620.5098230.8761590.0028060.4498280.671350
20.661264-0.9817370.7038790.730321-2.7504050.3965810.3803020.5326040.8771290.6109190.7801080.625689
31.4459360.3542370.7790401.2880142.3971330.1863240.5441910.4654190.5885350.9195750.5134600.831546
4-0.989027-1.824787-1.4482341.5464421.6437750.1679750.1882380.0241490.5448780.8345030.8778690.278330
\n", "
" ], "text/plain": [ " a b c d e f g \\\n", "0 2.315814 -3.427720 -1.314654 -1.508737 -0.300932 0.089448 0.327651 \n", "1 -0.191001 -1.396922 0.149488 -1.730145 -0.394932 0.519712 0.807762 \n", "2 0.661264 -0.981737 0.703879 0.730321 -2.750405 0.396581 0.380302 \n", "3 1.445936 0.354237 0.779040 1.288014 2.397133 0.186324 0.544191 \n", "4 -0.989027 -1.824787 -1.448234 1.546442 1.643775 0.167975 0.188238 \n", "\n", " h i j k l \n", "0 0.329022 0.857495 0.734238 0.257218 0.652350 \n", "1 0.509823 0.876159 0.002806 0.449828 0.671350 \n", "2 0.532604 0.877129 0.610919 0.780108 0.625689 \n", "3 0.465419 0.588535 0.919575 0.513460 0.831546 \n", "4 0.024149 0.544878 0.834503 0.877869 0.278330 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import tpot\n", "import sklearn.datasets\n", "from sklearn.linear_model import LogisticRegression\n", "import numpy as np\n", "import pandas as pd\n", "import tpot\n", "import sklearn.datasets\n", "from sklearn.linear_model import LogisticRegression\n", "import numpy as np\n", "from tpot.search_spaces.nodes import *\n", "from tpot.search_spaces.pipelines import *\n", "from tpot.config import get_search_space\n", "\n", "\n", "X, y = sklearn.datasets.make_classification(n_samples=1000, n_features=6, n_informative=6, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None)\n", "X = np.hstack([X, np.random.rand(X.shape[0],6)]) #add six uninformative features\n", "X = pd.DataFrame(X, columns=['a','b','c','d','e','f','g','h','i', 'j', 'k', 'l']) # a, b ,c the rest are uninformative\n", "X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, train_size=0.75, test_size=0.25)\n", "\n", "X.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lets say that either based on prior knowledge or interest, we know that the features can be grouped as follows" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "subsets = { \"group_one\" : ['a','b','c',],\n", " \"group_two\" : ['d','e','f'],\n", " \"group_three\" : ['g','h','i'],\n", " \"group_four\" : ['j','k','l'],\n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can create an FSSNode that will select from this subset. Each node in a pipeline only selects one subset." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "fss_search_space = FSSNode(subsets=subsets)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we randomly sample from this search space, we can see that we get a single selector that selects one of the predefined sets. In this case, it selects groups two, which includes ['d', 'e', 'f']. (A random seed was set in the generate function so that the same group would be selected when rerunning the notebook.)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fss_selector = fss_search_space.generate(rng=1).export_pipeline()\n", "fss_selector" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
def
1621.315442-1.0392580.194516
168-1.908995-0.953551-1.430472
2140.1811621.022858-2.289700
8952.825765-1.2055201.147791
154-2.3004811.0231730.449162
............
32-1.7930622.209649-0.045031
829-0.2214091.6887500.069356
1760.141471-1.8802941.984397
124-0.3599521.1417582.019301
350.1713120.0793320.178522
\n", "

750 rows × 3 columns

\n", "
" ], "text/plain": [ " d e f\n", "162 1.315442 -1.039258 0.194516\n", "168 -1.908995 -0.953551 -1.430472\n", "214 0.181162 1.022858 -2.289700\n", "895 2.825765 -1.205520 1.147791\n", "154 -2.300481 1.023173 0.449162\n", ".. ... ... ...\n", "32 -1.793062 2.209649 -0.045031\n", "829 -0.221409 1.688750 0.069356\n", "176 0.141471 -1.880294 1.984397\n", "124 -0.359952 1.141758 2.019301\n", "35 0.171312 0.079332 0.178522\n", "\n", "[750 rows x 3 columns]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fss_selector.set_output(transform=\"pandas\") #by default sklearn selectors return numpy arrays. this will make it return pandas dataframes\n", "fss_selector.fit(X_train)\n", "fss_selector.transform(X_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Under the hood, mutation will randomly select another feature set and crossover will swap the feature sets selected by two individuals" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ind1 = fss_search_space.generate(rng=1)\n", "ind1.export_pipeline()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
FeatureSetSelector(name='group_four', sel_subset=['j', 'k', 'l'])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "FeatureSetSelector(name='group_four', sel_subset=['j', 'k', 'l'])" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ind1.mutate()\n", "ind1.export_pipeline()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now use this when defining our pipelines. \n", "For this first example, we will construct a simple linear pipeline where the first step is a feature set selector, and the second is a classifier" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/ketrong/Desktop/tpotvalidation/tpot/tpot/tpot_estimator/estimator.py:456: UserWarning: Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.\n", " warnings.warn(\"Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.\")\n", "Generation: 100%|██████████| 5/5 [00:36<00:00, 7.26s/it]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "0.926166142557652\n" ] } ], "source": [ "\n", "classification_search_space = get_search_space([\"RandomForestClassifier\"])\n", "fss_and_classifier_search_space = SequentialPipeline([fss_search_space, classification_search_space])\n", "\n", "\n", "est = tpot.TPOTEstimator(generations=5, \n", " scorers=[\"roc_auc_ovr\", tpot.objectives.complexity_scorer],\n", " scorers_weights=[1.0, -1.0],\n", " n_jobs=32,\n", " classification=True,\n", " search_space = fss_and_classifier_search_space,\n", " verbose=1,\n", " )\n", "\n", "\n", "scorer = sklearn.metrics.get_scorer('roc_auc_ovr')\n", "est.fit(X_train, y_train)\n", "print(scorer(est, X_test, y_test))" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Pipeline(steps=[('featuresetselector',\n",
       "                 FeatureSetSelector(name='group_one',\n",
       "                                    sel_subset=['a', 'b', 'c'])),\n",
       "                ('randomforestclassifier',\n",
       "                 RandomForestClassifier(max_features=0.30141491087,\n",
       "                                        min_samples_leaf=4,\n",
       "                                        min_samples_split=17, n_estimators=128,\n",
       "                                        n_jobs=1))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "Pipeline(steps=[('featuresetselector',\n", " FeatureSetSelector(name='group_one',\n", " sel_subset=['a', 'b', 'c'])),\n", " ('randomforestclassifier',\n", " RandomForestClassifier(max_features=0.30141491087,\n", " min_samples_leaf=4,\n", " min_samples_split=17, n_estimators=128,\n", " n_jobs=1))])" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "est.fitted_pipeline_" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With this setup TPOT is able to identify one of the subsets used, but the performance is not optimal. In this case we happen to know that multiple feature sets are required. If we want to include multiple features in our pipelines, we will have to modify our search space. There are three options for this.\n", "\n", "1. UnionPipeline - This allows you to have a fixed number of feature sets selected. If you use a UnionPipeline with two FSSNodes, you will always select two feature sets that are simply concatenated together.\n", "2. DynamicUnionPipeline - This space allows multiple FSSNodes to be selected. Unlike UnionPipeline you don't have to specify the number of selected sets, TPOT will identify the number of sets that are optimal. Additionally, with DynamicUnionPipeline, the same feature set cannot be selected twice. Note that while DynamicUnionPipeline can select multiple feature sets, it never mixes two feature sets together.\n", "3. GraphSearchPipeline - When set as the leave_search_space, GraphSearchPipeline can also select multiple FSSNodes which act as an input to the rest of the pipeline." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### UnionPipeline + FSSNode example" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "union_fss_space = UnionPipeline([fss_search_space, fss_search_space])" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
FeatureUnion(transformer_list=[('featuresetselector-1',\n",
       "                                FeatureSetSelector(name='group_two',\n",
       "                                                   sel_subset=['d', 'e', 'f'])),\n",
       "                               ('featuresetselector-2',\n",
       "                                FeatureSetSelector(name='group_three',\n",
       "                                                   sel_subset=['g', 'h',\n",
       "                                                               'i']))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "FeatureUnion(transformer_list=[('featuresetselector-1',\n", " FeatureSetSelector(name='group_two',\n", " sel_subset=['d', 'e', 'f'])),\n", " ('featuresetselector-2',\n", " FeatureSetSelector(name='group_three',\n", " sel_subset=['g', 'h',\n", " 'i']))])" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# this union search space will always select exactly two fss_search_space\n", "selector1 = union_fss_space.generate(rng=1).export_pipeline()\n", "selector1" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
defghi
1621.315442-1.0392580.1945160.7511750.4113400.824754
168-1.908995-0.953551-1.4304720.0726970.8757660.953255
2140.1811621.022858-2.2897000.1352220.3958470.232638
8952.825765-1.2055201.1477910.9259050.4866450.710991
154-2.3004811.0231730.4491620.6451610.1316570.863514
.....................
32-1.7930622.209649-0.0450310.5029470.9946030.280062
829-0.2214091.6887500.0693560.3280660.1023810.492280
1760.141471-1.8802941.9843970.3655500.4658590.974601
124-0.3599521.1417582.0193010.3293800.7186470.365507
350.1713120.0793320.1785220.2157590.5462790.662928
\n", "

750 rows × 6 columns

\n", "
" ], "text/plain": [ " d e f g h i\n", "162 1.315442 -1.039258 0.194516 0.751175 0.411340 0.824754\n", "168 -1.908995 -0.953551 -1.430472 0.072697 0.875766 0.953255\n", "214 0.181162 1.022858 -2.289700 0.135222 0.395847 0.232638\n", "895 2.825765 -1.205520 1.147791 0.925905 0.486645 0.710991\n", "154 -2.300481 1.023173 0.449162 0.645161 0.131657 0.863514\n", ".. ... ... ... ... ... ...\n", "32 -1.793062 2.209649 -0.045031 0.502947 0.994603 0.280062\n", "829 -0.221409 1.688750 0.069356 0.328066 0.102381 0.492280\n", "176 0.141471 -1.880294 1.984397 0.365550 0.465859 0.974601\n", "124 -0.359952 1.141758 2.019301 0.329380 0.718647 0.365507\n", "35 0.171312 0.079332 0.178522 0.215759 0.546279 0.662928\n", "\n", "[750 rows x 6 columns]" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "selector1.set_output(transform=\"pandas\") \n", "selector1.fit(X_train)\n", "selector1.transform(X_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### DynamicUnionPipeline + FSSNode example\n", "The dynamic union pipeline may select a variable number of feature sets." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
FeatureUnion(transformer_list=[('featuresetselector',\n",
       "                                FeatureSetSelector(name='group_three',\n",
       "                                                   sel_subset=['g', 'h',\n",
       "                                                               'i']))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "FeatureUnion(transformer_list=[('featuresetselector',\n", " FeatureSetSelector(name='group_three',\n", " sel_subset=['g', 'h',\n", " 'i']))])" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dynamic_fss_space = DynamicUnionPipeline(fss_search_space)\n", "dynamic_fss_space.generate(rng=1).export_pipeline()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
FeatureUnion(transformer_list=[('featuresetselector-1',\n",
       "                                FeatureSetSelector(name='group_one',\n",
       "                                                   sel_subset=['a', 'b', 'c'])),\n",
       "                               ('featuresetselector-2',\n",
       "                                FeatureSetSelector(name='group_four',\n",
       "                                                   sel_subset=['j', 'k',\n",
       "                                                               'l']))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "FeatureUnion(transformer_list=[('featuresetselector-1',\n", " FeatureSetSelector(name='group_one',\n", " sel_subset=['a', 'b', 'c'])),\n", " ('featuresetselector-2',\n", " FeatureSetSelector(name='group_four',\n", " sel_subset=['j', 'k',\n", " 'l']))])" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dynamic_fss_space.generate(rng=3).export_pipeline()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### GraphSearchPipeline + FSSNode example\n", "\n", "FSSNodes must be set as the leaf search space as they act as the inputs to the pipeline.\n", "\n", "Here is an example pipeline from this search space that utilizes two feature sets." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAHWCAYAAAD6oMSKAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXWFJREFUeJzt3QlY1OX2wPEDDCqLsogKLrhi7vuOlt2sW2bezNSsW6bt+75aeTO1xRa7LVqZZrutN/9ZWbZYLrmgae6IKIpCIgiyCQP8n/Pe4DLKGArMb2b4fp6Hx3r5zcwBlN+Z97zveX1KSkpKBAAAAB7P1+oAAAAAUD1I7AAAALwEiR0AAICXILEDAADwEiR2AAAAXoLEDgAAwEuQ2AEAAHgJEjsAAAAvQWIHAADgJUjsAAAAvASJHQAAgJcgsQMAAPASJHYAAABegsQOAADAS5DYAQAAeAkSOwAAAC9BYgcAAOAlSOwAAAC8BIkdAACAlyCxAwAA8BIkdgAAAF6CxA4AAMBLkNgBAAB4CRI7AAAAL0FiBwAA4CVI7AAAALwEiR0AAICXILEDAADwEjarAwCA6lRUVCTp6emSmppqPg6lpMixvDwpLioSXz8/qRsQII0iI6VJkybmIzw8XPz8/KwOGwCqhU9JSUlJ9TwVAFgnIyNDNm7cKL+vXy/5OTlSYrdLcF6ehKSni7/dLr4lJVLs4yOFNptkhodLdkCA+NhsUi8oSLr26iXdu3eXsLAwq78MAKgSEjsAHu3AgQOycvlySYyPF//cXIlO2idR6ekSkpMj/kVFTh9X6OcnmUFBcjA8XJKiW0hhYKC0jomR2CFDJCoqyqVfAwBUFxI7AB7JbrfLihUrZO2KFRKclibt9iZJ87Q08SsuPuXnKvL1lf0REbKrZbRkR0RI39hYiY2NFZuN1SoAPAuJHQCPk5KSIosXLZKM/cnSIT5eYpKTTam1qrRUG9+smWyPiZHw5s1k+MiREhkZWS0xA4ArkNgB8Ch79+6VzxculMADB6X3tm3SIDe32l8jKzBQ4jp2lNymTWXUuLHSsmXLan8NAKgJJHYAPCqp+/SDD6Th3iTpt3Wr2E6j7FpZdl9fWd25k6RHR8vo8eNJ7gB4BPrYAfCY8qvO1IXvTZIBW7bUaFKn9PkHbt4i4UlJ8vnCj8zrA4C7I7ED4BEbJXRNnZZf+2/dWi3r6SpDX6f/lq0ScPCAfLVokYkDANwZiR0At6e7X3WjhK6pq+mZuuPp6/Xeuk3Sk5Nl5cqVLn1tADhVJHYA3L5PnbY00d2vNbFRojJCcnPljJ3xsmb5cjl48KAlMQBAZZDYAXBr2nxY+9RpSxMrtU9ONnGsWL7c0jgA4GRI7AC49TFheqKENh921bo6Z/T12+5NksSdO01cAOCOSOwAuC09+1WPCdMTJdxBi7Q0seXmyqZNm6wOBQAqRGIHwC0VFRXJ7+vXm7NfT+eYsJqgcbTct082xcWZ+ADA3ZDYAXBL6enpkp+TI1Hp6eJOog7/Ny6NDwDcDYkdAKdsNpv06NGj7CMvL++Un+OZZ545rddOTU2VErtdQrOzHcZfTtorw9fHyYj1cXLJbxtkX37+SZ/njf37qvT4fr+ucvj/kJwcE5fGdzKzZs2SgoICqarffvtNBgwYIF26dJFevXrJTz/9VOXnBOC9bFYHAMB9hYaGmsSiKjSxu//++0/pMVrm1MQpOC/PoW/d+qwsWZ2ZKV/06Cn+vr6ScuyYBPid/P3pG/v3y3XNW5z244/nX1Rk4tL4NNk6WWJ37bXXSp06dSr1vMXFxeLre2IsQUFB8t5770nbtm1l69atMmLECNm9e/cpxQyg9mDGDsApWbJkiQwcOFB69uwp//znP8tmpa6//nrp3bu3dO7cWZ599lkzNnnyZDly5IiZ7bvxxhtlz5490qdPn7Lnuvfee+Wtt94y/92qVSt58MEHzfP+8MMP8tknn8iz8+fLRevXy4w/E5lDBQUSZvM3SZmKrFtXQmz+5r9/yciQsRt/k39sWC/37tguBcXF8vyePXLUbpeRG9bLY7viT/nxx3t9/z4zy/fU3Lky/803y8anT58uXbt2lW7duskLL7wgr7zyium/N2jQIBk5cqS55p133jHXaDI4c+ZMM6bfDx277LLLpFOnThXOiMbExJikTnXs2FGys7NZ3wfAKWbsADhVmpQpTcieeuopk5Ro4hUQECCPPfaYvPHGG3LLLbeYz4WHh5tjt4YMGSLjxo0zCc9rr71WNuuniczJtGjRQjZs2CDbtm2TNWvWyPQLLpA+iXvkvh075Mf0dIkNDZWXkvbKBXHrJDY0TP7RuLF0rV9f0gsLZe7+/fJ2l65Sz89PXty7Rz5KSZG7W7WSD1MOyqKevczzZ9vtp/T4fzZtWhbb8owMM8P3afcesr5NG5m6do1s3rxZkpKSzPdj3bp1UrduXbP2Tr8P+n3SkyqCg4MlOTlZ/vWvf8natWslMDDQJHx/+9vfpGHDhuZr1Rk5TQr/yn/+8x+TPPv5+VXxJwvAW5HYAah0KfbLL780rT50xk4dO3ZMLrzwQvPfH3zwgcydO9fMJu3fv1+2b99uErVTMWbMGPPn999/Lwm7d8tDiYkSUFAg+UXF0iU4WM4OD5f/9Owlq48ckZWZR2Ti5s3yYocOUlBSLDtyc2Tspo3m8TrbNjQ8/ITnD7bZTvvxy49kyE/pGbIua4Pkbd0iuX5+snPnTlm+fLlMnDjRJHVKk7rjaUJ3zjnnlH3u0ksvNY/7xz/+Ie3bt69UUqflVy1pf/3116f0PQVQu5DYAag0XQemidz8+fNPSDq0/Lhq1SoJCQkxiYsmfRVtxtDnKHX8NTqbVfo6Zw0ZIuPDw6VnguN6MpuPj8SGhZmPcJu/LE0/LINDw2RoWLg81b79X34Np/v44hKRW6Oj5ZImTWRD27aSP2SwXHLJJSZBq4rSr/lkdBZQk0Cd/WzXrl2VXg+Ad2ONHYBK05m6H3/8Ufbu3Wv+PysrSxITE+Xo0aOm5NigQQMzW7d06dKyx2jZsHRNWOPGjc3aM71e14p99913Fb6Ozm6tjYuTdLvd/P/hggL5o6BAdufmStKf69BKSkpkZ26ONK1bV3o2qC+rM49I8p87XLXkWrrb1c/HR4r+PLXidB5fanBYqHycmiJ5RUVSYLNJ5tGjkpmZKcOGDTOJbmmSWtoGpX79+ubrVP369TOzkHpihV732WefmXJ1ZegaxlGjRsk999xjyrcAcDLM2AGotEaNGpk1daNHjzYJh+7i1N2fQ4cONQv7O3ToYDZBDB48uOwxEyZMMBsEzjzzTJkzZ44pJ+oGiejoaDNeEd2AcdWECTJt7lyZlZNjNjs8HdNejpUUy9SEBMn+M1HsHBQsV0Y1NeviprWLkdu2b5PC4mLx8fGRya3bSIt69WRU4yamtUnfkBAZGxl5yo8vdWZYuOzKzTUbLLLid0r4r6tk7PjxMnz4cImLizOtSPz9/U1Z9o477pDrrrtOzj77bFNqXbRokUyZMsV8DzSh1O+JXv9Xaw7VRx99JL/++qtJIvV7rTRJ1PV5AHA8nxL9LQMAbkY3Jnz18ccyYtnPpsWIuyj085MvzzpTho8Zc9J2JwBgBUqxANxSkyZNxEdLnkFB4k40Ho1L4wMAd0MpFoBb0h2k9YKC5GB4uERkZYm7ONjwv3FVtPu1Kg4fPmzWFpanO21Xr15dra8DwLuR2AFwS7rpomuvXvLb4cPSKSlJ/CpoGOxqRb6+srdFC+lVA73kdM1cVU/5AABKsQDcVvfu3aUwMFD2R0TUyPNnHc2SAwcPyh+H/pDCP3fgnsy+iAixBwZWqu8cAFiBxA6A2woLC5PWMTGyq2W0FPv4VOtzayKnLVdESsxpGdqmpPgke8n09RNaRkvr9u1NXADgjkjsALi12CFDJDsiQuKbNavR1ykqspu+fM7sbNbMxBFbrpULALgbEjsAbi0qKkr6xsbK9pgYyarEKQ2V5W+zSZ06/z0GrFRubo7kV3BiRmZgoOxoHyP9Bg828QCAuyKxA+D2YmNjJax5M4nr2FHsvr7Vehauj4/j82UeOeJQktXXi+vUUcKbNZNBgwZV22sDQE0gsQPg9vSM2QtHjpTcpk1ldedO1bbezubnZ45BK6+ouMic8qD0dfT18qKayvCRI00cAODOSOwAeITIyEgZNW6spEdHy6ounatt5i4oMFDq1v3f0WEqLy9XcgoKzOvo6+nr6usDgLvjSDEAHmXv3r3y+cKPJPDAAem9bZs0yM2t8nMWFRXJH4cOSUnJf3vl5TRoIDv69JWSNq1l9Pjx0rJly2qIHABqHokdAI+TkpIiixctkoz9ydIhPl5ikpPFt4q/ynLz8iQ984gcaN9e4jt0kOT0dMkrLJR33nlHfKq51QoA1BQSOwAeSXvPrVixQtauWCHBaWnSdm+StEhLO60TKvRECW0+vKVJY0kNCJAVa9fKypUrzUzeBx98IJdddlmNfA0AUN1I7AB4tAMHDsjKFSskcedOseXmSst9+yTqcLqE5OSIf1GR08cV+vlJpp5F2zDcHBOmJ0pEtWghT0yfLjt37iy7Ts+E3bx5M21OAHgEEjsAXiEjI0M2bdokm+LiJD8nR0rsdgnOy5MG6RlSx24X35JiKfbxlQKbTbLCwyQ7IEB8bDapFxQk3Xr3NseE6YkSH330kYwbN87huUeMGCGLFi2iJAvA7ZHYAfAqWj7V48FSU1PNx6GUFCnIz5ciu138tClxvXrSKDJSmjRpYj50Rs7Pz8/hOTSx0wSvvPnz58vVV1/t4q8GAE4NiR0AHCctLU26dOliEsNS2u9OS7ItWrSwNDYAOBn62AHAcSIiIuT11193GNNzZCdNmiS8FwbgzkjsAKACI0eOlAkTJjiMLV26VObMmWNZTADwVyjFAoATR44cMSXZ5OTksrGgoCDZuHGjtG3b1tLYAKAizNgBgBOhoaEyb948h7GcnByZOHGiFJ9GvzwAqGkkdgBwEuedd57ccMMNDmO//PKLvPjii5bFBADOUIoFgL9w9OhR6d69uyQmJpaN1a1bV3777Tfp0KGDpbEBQHnM2AHAX6hfv77pY1fesWPHzOYKPdoMANwFiR0AVMJZZ50ld955p8PYmjVr5JlnnrEsJgA4HqVYAKikvLw86dGjh8NZsv7+/rJu3TpzJBkAWI0ZOwCopICAAFmwYIH4+v7vV2dhYaFcddVVUlBQYGlsAKBI7ADgFAwYMEDuv/9+hzHtazdt2jTLYgKAUpRiAeAU6caJPn36mLNjS/n5+cmqVaukb9++lsYGoHYjsQOA07Bhwwbp16+fw67Yjh07yvr166VevXqWxgag9qIUCwCnoWfPnvLoo486jG3btu2EMQBwJWbsAOA06caJgQMHSlxcXNmYj4+POZkiNjbW0tgA1E4kdgBQBVu2bJFevXo57Ipt27at2VARFBRkaWwAah9KsQBQBZ07d5YnnnjCYSwhIUEefPBBy2ICUHsxYwcAVVRUVCRDhgwxu2LLW7p0qZxzzjmWxQWg9iGxA4BqEB8fL927dzenU5SKjo6W33//XRo0aGBpbABqD0qxAFANYmJi5Omnn3YYS0pKkrvvvtuymADUPszYAUA1KS4ulmHDhsmPP/7oML548WIZPny4ZXEBqD1I7ACgGu3Zs0e6du0q2dnZZWNRUVHmlIrw8HBLYwPg/SjFAkA1atWqlTz//PMOYwcPHpTbb7/dspgA1B7M2AFANdNfq1p6/eabbxzGP/30U7nkkkssiwuA9yOxA4AakJycLF26dJEjR46UjTVq1MiUZBs3bmxpbAC8F6VYAKgBzZo1k5deeslh7NChQ3LTTTeZGT0AqAkkdgBQQ6644gq5+OKLHcY+++wz+eCDDyyLCYB3oxQLADUoNTXVlGTT0tLKxkJDQ80Zs02bNrU0NgDehxk7AKhBTZo0kdmzZzuM6bq76667jpIsgGpHYgcANezSSy+V8ePHO4x99dVXMm/ePMtiAuCdKMUCgAukp6dL586dJSUlpWysfv365izZli1bWhobAO/BjB0AuICeOvHGG284jB09elQmTZpkjiIDgOpAYgcALjJixAiZOHGiw9gPP/xwwho8ADhdlGIBwIUyMzPNWbL79u0rGwsMDJSNGzdKu3btLI0NgOdjxg4AXCgkJOSETRO5ubly9dVXS1FRkWVxAfAOJHYA4GLDhg2Tm2++2WFsxYoV8sILL1gWEwDvQCkWACyQnZ0tPXr0kISEhLKxunXryvr166VTp06WxgbAczFjBwAWCA4Olrfeekt8fHzKxo4dOyYTJkwQu91uaWwAPBeJHQBYZPDgwXL33Xc7jK1bt06eeuopy2IC4NkoxQKAhfLy8qRXr16yffv2sjGbzSZr1641pVoAOBXM2AGAhQICAuTtt98WPz+/sjEtxWpJVkuzAHAqSOwAwGJ9+/aVBx980GFs06ZNMnXqVMtiAuCZKMUCgBsoKCgwCZ4mdKV8fX1l5cqV0r9/f0tjA+A5SOwAwE3o6ROa3BUWFpaNnXHGGbJhwwZTsgWAv0IpFgDcRPfu3WXKlCkOYzt27JBHHnnEspgAeBZm7ADAjejGiUGDBpldsaW0192yZctkyJAhlsYGwP2R2AGAm9m2bZv07NnTYVds69atzfo7bWwMAM5QigUAN9OxY0eZPn26w1hiYqLcf//9lsUEwDMwYwcAbqioqEiGDh0qy5cvdxj/9ttv5dxzz7UsLgDujcQOANxUQkKCdOvWTXJzc8vGmjdvLps3b5aQkBBLYwPgnijFAoCbatu2rcycOdNhbP/+/XLXXXdZFhMA98aMHQC4seLiYjnvvPPk+++/dxhftGiRXHTRRZbFBcA9kdgBgJtLSkqSLl26yNGjR8vGmjRpIlu2bJGGDRtaGhsA90IpFgDcXHR0tMyaNcthLDU1VW699VbLYgLgnpixAwAPoL+qtfS6ePFih/GPPvpIxowZY1lcANwLiR0AeIiDBw9K586dJSMjo2xMS7FaktXSLABQigUADxEVFSWvvPKKw9jhw4flhhtuMDN6AEBiBwAe5LLLLpPRo0c7jH3xxRfy7rvvWhYTAPdBKRYAPMyhQ4dMSVb/LKUNi7VxsTYwBlB7MWMHAB6mUaNG8tprrzmMZWZmyrXXXktJFqjlSOwAwAONGjVK/vnPfzqMLVmyRN544w3LYgJgPUqxAOChdHesNi4+cOBA2VhwcLBs2rRJWrdubWlsAKzBjB0AeKiwsDCZO3euw1h2drZMnDjRHEUGoPYhsQMAD3bBBReYtXXlLVu2TF5++WXLYgJgHUqxAODhsrKypFu3brJ3796ysYCAAPntt9+kffv2lsYGwLWYsQMAD9egQQOZP3++w1heXp5MmDBBioqKLIsLgOuR2AGAFzj77LPltttucxj79ddf5dlnn7UsJgCuRykWALxETk6O9OjRQ3bt2lU2VqdOHYmLizO7ZwF4P2bsAMBLBAUFyYIFC8TX93+/2gsKCuSqq66SwsJCS2MD4BokdgDgRQYNGiT33HOPw9iGDRtkxowZlsUEwHUoxQKAl8nPz5fevXvL1q1by8ZsNpusXr1aevXqZWlsAGoWM3YA4GXq1asnb7/9tvj5+ZWN2e12U5I9duyYpbEBqFkkdgDghXTGbvLkyQ5jW7ZskSlTplgWE4CaRykWALyUbpwYMGCAWWNXSjdWLF++XAYOHGhpbABqBokdAHix33//3czeld8VGxMTY06lCAwMtDQ2ANWPUiwAeLGuXbvK1KlTHcbi4+PloYcesiwmADWHGTsA8HK6cWLw4MFmV2x5P/zwgzmxAoD3ILEDgFpgx44d5lQKbYVSqlWrVrJp0yapX7++pbEBqD6UYgGgFjjjjDPkySefdBjbs2eP3HvvvZbFBKD6MWMHALVEcXGx/O1vf5Nly5Y5jH/99ddy/vnnWxYXgOpDYgcAtcju3bulW7dukpOTUzbWrFkzs3s2LCzM0tgAVB2lWACoRdq0aSPPPfecw1hycrLccccdlsUEoPowYwcAtYz+2tfS67fffusw/vnnn8vFF19sWVwAqo7EDgBqoX379pked5mZmWVjjRs3NseORUREWBobgNNHKRYAaqEWLVrIiy++6DD2xx9/yE033WRm9AB4JmbsAKCW0l//WnpdtGiRw/gHH3wgl112mWVxATh9JHYAUIulpKRI586dJT09vWwsPDxcNm/eLFFRUZbGBuDUUYoFgFosMjJSZs+e7TCmSd71119PSRbwQCR2AFDLjR071nyU9+WXX8qCBQssiwnA6aEUCwCQtLQ06dKli6SmppaNNWjQwJRkdaMFAM/AjB0AwLQ4ef311x3GsrKyZNKkSZRkAQ9CYgcAMEaOHCkTJkxwGFu6dKnMmTPHspgAnBpKsQCAMkeOHDElWT1mrFRQUJBs3LhR2rZta2lsAP4aM3YAgDKhoaEyb948h7GcnByZOHGiFBcXWxYXgMohsQMAODjvvPPkhhtucBj75ZdfTjipAoD7oRQLADjB0aNHpXv37pKYmFg2VrduXfntt9+kQ4cOlsYGwDlm7AAAJ6hfv77Mnz/fYezYsWNmc4XdbrcsLgAnR2IHAKjQWWedJXfeeafD2Jo1a+SZZ56xLCYAJ0cpFgDgVF5envTo0UN27txZNubv7y9r1641pVoA7oUZOwCAUwEBAeZoMV/f/90uCgsLTUm2oKDA0tgAnIjEDgBwUgMGDJD777/fYUz72j3xxBOWxQSgYpRiAQB/STdO9OnTx5wdW8rPz09WrVolffv2tTQ2AP9DYgcAqJQNGzZIv379HHbFduzYUdavXy/16tWzNDYA/0UpFgBQKT179pRHH33UYWzbtm0njAGwDjN2AIBK040TAwcOlLi4uLIxHx8f+fnnn2Xw4MGWxgaAxA4AcIq2bNkivXr1ctgV27ZtW7OhIigoyNLYgNqOUiwA4JR07txZpk2b5jCWkJAgDzzwgGUxAfgvZuwAAKesqKhIzjzzTFm5cqXD+NKlS+Wcc86xLC6gtiOxAwCclvj4eHP6hJ5OUSo6Olp+//13adCggaWxAbUVpVgAwGmJiYmRp59+2mEsKSlJ7r77bstiAmo7ZuwAAKetuLhYhg0bJj/++KPD+JdffikXXnihZXEBtRWJHQCgSvbs2SNdu3aV7OzssrGoqChzSkV4eLilsQG1DaVYAECVtGrVSl544QWHsYMHD8ptt91mWUxAbcWMHQCgyvRWoqXXr7/+2mH8k08+kdGjR1sWF1DbkNgBAKpFcnKydOnSRY4cOVI2FhERYRoaN27c2NLYgNqCUiwAoFo0a9ZMXnrpJYextLQ0uemmm8yMHoCaR2IHAKg2V1xxhYwaNcph7LPPPpP333/fspiA2oRSLACgWv3xxx/m2DGdrSsVGhpqSrJNmza1NDbA2zFjBwCoVrqebvbs2Q5juu7u2muvpSQL1DASOwBAtbv00ktl/PjxDmO6Y3bevHmWxQTUBpRiAQA1Ij093ZRkU1JSysaCg4PNWbLa+w5A9WPGDgBQI/TUiblz5zqM6ekUkyZNMkeRAah+JHYAgBqjTYs1kStPz5V99dVXLYsJ8GaUYgEANSozM9OcJbtv376ysYCAANm4caPExMRYGhvgbZixAwDUqJCQkBM2TeTl5cnVV18tRUVFlsUFeCMSOwBAjRs2bJjcfPPNDmMrV66U559/3rKYAG9EKRYA4BK6caJHjx6SkJBQNlanTh1Zv3692T0LoOqYsQMAuIS2OnnrrbfEx8enbKygoEAmTJgghYWFlsYGeAsSOwCAywwePFjuvvtuh7G4uDh56qmnLIsJ8CaUYgEALqUbJ3r16iXbt28vG7PZbLJmzRrp2bOnpbEBno4ZOwCAS2mrkwULFoifn1/ZmN1uNyXZY8eOWRob4OlI7AAALtevXz958MEHHcb0qLHHH3/cspgAb0ApFgBgCd040bdvX9m0aVPZmK+vr2mD0r9/f0tjAzwViR0AwDJ6+oQmd+V3xZ5xxhmyYcMGU7IFcGooxQIALNO9e3eZMmWKw9iOHTtk8uTJlsUEeDJm7AAAltKNE4MGDZK1a9eWjWmvu59++knOPPNMS2MDPA2JHQDActu2bTOtTsrvim3durVZf6eNjQFUDqVYAIDlOnbsKNOnT3cYS0xMlPvuu8+ymABPxIwdAMAtFBUVydChQ2X58uUO40uWLJHzzjvPsrgAT0JiBwBwGwkJCdKtWzfJzc0tG2vevLnpcRcaGmppbIAnoBQLAHAbbdu2lZkzZzqM7d+/X+666y7LYgI8CTN2AAC3UlxcLH//+99l6dKlDuNffPGFjBw50pRsyx9HBuB/mLEDALgVPX3izTfflAYNGjiMX3fddXLzzTebkmx0dPQJa/EAMGMHAHBT8+fPl0mTJjn9fI8ePcwJFQD+h8QOAOCW9PZ04YUXytdff13h57WJcV5entStW9eUZ9PT0yU1NdV8HEpJkWN5eVJcVCS+fn5SNyBAGkVGSpMmTcxHeHg45Vx4JZvVAQAAUJGDBw/K7t27T5r4bdmyRbKysuT39eslPydHSux2Cc7Lk5D0dAmw28W3pESKfXyk0GaTHeHhEhcQID42m9QLCpKuvXqZI83CwsJc+nUBNYkZOwCAW7rssstk4cKFFX4uMjJSBg8aJD27dpXAwkKJTtonUenpEpKTI/5FRU6fs9DPTzKDguRgeLgkRbeQwsBAaR0TI7FDhkhUVFQNfjWAazBjBwBwS2lpaSeMaflUz5WN7dtXIrKzpcO6OGl79Kj4FRdX6jk16YvIyjIfnZKSZH9EhOw6fFje27VL+sbGSmxsrNhs3BrhuZixAwC4JW13ctFFF0l+fr75/8aNG8vICy+UZmFhErN9uzTduVOCAwIlNCSkSq+jpdr4Zs1ke0yMhDdvJsNHjjQzgoAnIrEDALitXbt2yQMPPCDr1q2TsRdfLFG5udIxLk4Cs7LM5/38bNKkceNqea2swECJ69hRcps2lVHjxkrLli2r5XkBVyKxAwC4tb1798oHb78tIQkJcsaqVeJXbg1ddSZ2yu7rK6s7d5L06GgZPX48yR08Dg2KAQBuKyUlRT5fuFAiDxyUs3cnSnhwfZ2TKPt8cHBwtb6erbhYBm7eIuFJSfL5wo/M6wOehMQOAOCW7Ha7LF60SAIPHJT+W7eKX0mJBAUGSlRkpISFhUvjxk3M/1c3bZHSf8tWCTh4QL5atMjEAXgKEjsAgFtasWKFZOxPlt7btpmZtPKNiQPq1RNbDTYY1tfrvXWbpCcny8qVK2vsdYDqRmIHAHA7Bw4ckLUrVkiH+HhpkJtrSQwhublyxs54WbN8uWmWDHgCEjsAgNtZuXy5BKelSUxysqVxtE9ONnGsWL7c0jiAyiKxAwC4lYyMDEmMj5d2e5PMejcr6eu33ZskiTt3mrgAd0diBwBwKxs3bhT/3FxpXsHJE1ZokZYmttxc2bRpk9WhAH+JxA4A4DaKiork9/XrzdmvlT0mrKZpHC337ZNNcXEmPsCdcSAeAMBtpKenS35OjkSlpzuMd1z+i8QEBUmRlkYDAuXp9u0lwM9PUo4dkyd2J8j2nBwJsdmked168ljbthJRp4553IM7d8rO3Bz5rEfPk77uq0lJsjA1RfKKimTNgIEnfD7qcLok5OSY+Bo1alTNXzVQfZixAwC4jdTUVCmx2yU0O9thvL7NJot69pLFvXqLv6+PfJByUPTgpJu2bpWhYeHyfZ++Jnm7smlTSS8sNI8pKC6W1ZlHzJ9J+Xknfd3BYWHycfceTj8fkpNj4tL4AHdGYgcAcBuaOAXn5Tn0rTtenwYhkpSXLyszj0ign6+MiYws+1zfkBBpHxRk/nt5Roa59sJGjeSrQydfr9etfn1p/OcsX0X8i4pMXCR2cHckdgAAt3EoJUVCjivDlmcvKZGfM9KlfVCgJOTmSueTHCn2VdohuSAiQi6MaGT+u6oapGeY+AB3RmIHAHAbx/LyxL+CI7yO2u0ycsN6ueS3DdK0bj25tEnkyZ+nuFjWZGaaEmt0QIDYfHxkdxUbHdex26UgP79KzwHUNDZPAADcRnFRUYW960rX2JWnmyi+TTtc4fP8lJ4uWXa7/D1unfn/7KIiM2t3a3TL047Nt6RYijg3Fm6OGTsAgNvw9fOTYh+fSl07KDRUsovs8lm5dW/rMjNlZ06OSeKePaOD/Ni3n/n4tEcP+aqKffGKfXzFz8Z8CNwbiR0AwG3UDQiQwkomTz4+PvJqx07y3eHDcs66tTJ8fZy8c/CABPn5ya9Hjsjg0NCya6PrBYif+JikryKz9u6RIWtWm1k+/XNe8v4Trimw2aROvXpV+OqAmudTovvFAQBwA99//73sWLJEzl316wmfK7TbJTcnR4qKiyUoKEjqnmQXa034buAAOePvf5dzzjnHpa8LnArmlAEAbqNJkyYSp7N2fn6mxUhpQnf06FHJL9eL7lh+vjRu0kT8fF1TeNJ4sgMCTHyAOyOxAwC4DU2cfGw2yQwKkgaHD0t2tiZ0J+5ELZESKSqyi5/vqc3a/Sthl6zPynIYu69VaxkSFnbSx2k8GheJHdwdiR0AwG2Eh4eL+PlJQkCAtDhJ7zl//zrib/M/5ef/V9t2pxXXwYbhUi8o6L/xAW6MzRMAALewcuVKufDCC+XLJUtkT7OmUlRBmdXHx1fqB9eXhg0bms0TrqBx7G3RQrr17i1+fn4ueU3gdJHYAQAstWzZMhk2bJjExsbKkiVLZOPGjZLr7y+Hmzcvu8ZXE7r6DUwptH79+uLroqRO7YuIEHtgoHTr1s1lrwmcLkqxAACX04YMP/zwg0ydOlV+/vlnh89lZmZKfGKiNIyJkcbJydIgMEgCg4JcmsyV0p56CS2jpXX79hL2F+vwAHfAjB0AwKUJnc7KDR482MzSHZ/Uldq6fbvkNW4sWb16SXBwsCVJndrZrJlkR0RI7ODBlrw+cKpI7AAALknoFi9eLAMGDJDzzz/frKerSNOmTWXWrFmydu1aGTJsmOyIaS9ZgYE1FldRcZHk5edJcQUtXTMDA2VH+xjpN3iwREVF1VgMQHWiFAsAqNGEbtGiRabkun79eqfXNW/eXB566CGZNGmS1PvzdAddc7drxw6Jy+ooQzZsEFtxcbXGlpefLxkZGaZ5ioiPNGjQwDQ+1rlBu6+vxHXqKOHNmsmgQYOq9XWBmsSMHQCg2hUXF8snn3wiPXv2lIsvvthpUteyZUt57bXXZNeuXXLzzTeXJXXKZrPJhSNHSm7TprK6c6dKnyFbWdr0+L9JnSqRrKxMOXTokOQVFJjXy4tqKsNHjjRxAJ6CxA4AUG2Kiorkww8/NDtIx4wZY3a4VqRNmzby5ptvSnx8vFx//fVSt27dCq+LjIyUUePGSnp0tKzq0tnMpFWXitqlHCspll/OaC8JDRpIz/79zOsDnoSzYgEAVWa3201CN23aNNmxY4fT62JiYuSRRx6Ryy+//JRmwvbu3SufL/xIAg8ckN7btkmD3Nwqx5x19Kg52aJUToMGsr13HzkQGCAfff65HDx4UD766CMZNWpUlV8LcBUSOwDAaSssLJT33ntPpk+fbsqpznTs2NEkdOPGjTvtJr8pKSmyeNEiydifLB3i4yUmOVl8q3ALy83LkyNHMkyJ90D79hLfoYMkp6fLoq++kj/++MNc07t3b1m3bt1pvwbgaiwcAACcsoKCAnn77bdlxowZkpiY6PS6Ll26yKOPPiqjR4+u8qkNWhadMGmSrFixQtbWqyv7oyKl7d4kaZGWJn6nsbHCp46/pLZsKfvatZO04GBZsXat2a2r5eRSERERVYoZcDVm7AAAlXbs2DGZP3++PPnkk5KUlOT0uh49eshjjz0m//jHP8S3GtfFlTpw4ICsXLFCEnfuFFturrTct0+iDqdLSE6O+JdLzI5X6OcnmUFB5uzXPc2bS1pBgexMTJQVK1eaGcHyoqOjTRPltm3bVnv8QE0hsQMA/KW8vDyZO3euPP3005KcnOz0uj59+piEbsSIES45y1XblWzatEk2xcVJfk6OlNjtEpyXJw3SM6SO3S6+JcVS7OMrBTabZIWHSXZAgPjYbFIvKEi69uol48ePPyGhK6VtTn755ZcaSUyBmkJiBwBwKjc317QjeeaZZ5wmQKp///4yZcoU03zYFQnd8bR8mp6eLqmpqebjUEqKFOTnS5HdLn42m9SpV08aRUaas2b1Izw83JSGzz77bPnpp5+cPu/zzz8vd911l0u/FqAqSOwAACfIzs6W2bNny7PPPlu2kaAiejSYztDp8WBWJHRVpf31dEOHlnZ19m7p0qVmB24pbcOyYcMGs/kD8AQkdgCAMllZWfLKK6/Ic889J4cPH3Z63dChQ01Cp396YkJX0YyfzuAtW7bMfE3l9e3b12yqoFExPAELBwAAcuTIEXPsV6tWreThhx92mtSde+658vPPP8uPP/5oypjekNSp0h27Z511ltx5550On9Nza7UUDXgCZuwAoBbTdWmzZs2SF1980czWOXPBBReYtiUDBw6U2rBRRHf17ty5s2zM39/fJHjdu3e3NDbgr5DYAUAtlJaWZjYGvPTSS2Y9nTMXXXSRSei0HFmb/PrrrxIbG2vOvC2lSd2aNWukTp06lsYGnAylWACoRXTH6P33329KrtqLzllSp8do6caCRYsW1bqkTg0YMMB8n8rTc2+feOIJy2ICKoMZOwCoBfTc05kzZ8qcOXNMqbEiul5uzJgx5uivrl27Sm2nzZi1L9/mzZsd1uKtWrWqVia78AwkdgDgxfbv32+aCr/xxhsmUamINuC97LLLZPLkydKpUyeXx+jOtNVJv379xG63l41p6xOdzaxXr56lsQEVoRQLAF5Ie7HddNNN5jisl19+ucKkTmefrrrqKtm6dau89957JHUV6Nmzp1ljWN62bdtOGAPcBTN2AOBFdu/ebdbOvfXWWw6zTOVpPzZN6B566CFp166dy2P0NIWFhWY3cFxcnEPZWtu+aINmwJ2Q2AGAF4iPj5cZM2bIO++8Y5rtVkRbdkycOFEefPBBad26tctj9GRbtmyRXr16SUFBQdmYzobqhoqgoCBLYwPKoxQLAB5s+/btcuWVV0qHDh3MLF1FSZ2257jllltk165d5txXkrpT17lz5xN2xCYkJMgDDzxgWUxARZixAwAPnUGaNm2aLFy4UJz9GtfF/TfccIPcd9990qxZM5fH6G00aR4yZIjZFVueni97zjnnWBYXUB6JHQB4kNJeap9++qnTawIDA83GiXvvvVciIyNdGl9tKHlro+LyLWOio6Pl999/lwYNGlgaG6AoxQKAB9CF+xdffLE56spZUqdrvbQ0mJiYKM8++yxJXQ2IiYkx7WPKS0pKkrvvvtuymIDymLEDADe2evVqM0O3ePFip9fUr19fbr/9dnN4fUREhEvjq430mLFhw4bJjz/+6DD+5ZdfyoUXXmhZXIAisQMAN7RixQqT0C1ZssTpNSEhISaZu+OOOyQsLMyl8dV2e/bsMadzlD+SLSoqypxSER4ebmlsqN0oxQKAG1m2bJlZiK/90ZwldZo46MYJbUL8r3/9i6TOAnrW7vPPP3/CsW233XabZTEBihk7ALCY/hr+4YcfZOrUqabprTNaZtUNETfffLMpv8L6n9vw4cPlm2++cRj/5JNPZPTo0ZbFhdqNxA4ALKK/fr/99luT0K1cudLpdU2aNDEtS2688Uaa4bqZ5ORk6dKlixw5csQhAdd2NI0bN7Y0NtROlGIBwIKETjdDDBgwQM4//3ynSZ2u2Zo1a5Y5Juyee+4hqXND2h/wpZdechhLS0sz7WaYN4EVmLEDABfRX7eLFi0yM3Tr1693el3z5s3NsV/XXHONaTIM9/+5XnLJJfKf//zHYfzdd9+VK664wrK4UDuR2AGAC9pjfPbZZ2bDgzYYdqZly5by8MMPy4QJE6Ru3boujRFVk5qaakqyOltXKjQ01JRkmzZtamlsqF0oxQJADR5B9eGHH0q3bt1kzJgxTpO6Nm3ayJtvvmlONbj++utJ6jyQroOcPXu2w5iuu7v22mspycKlSOwAoJrZ7XZThtOD48ePH29mbZydYrBgwQLZsWOHTJo0Sfz9/V0eK6rPpZdean7e5X399dcyb948y2JC7UMpFgCqSWFhobz33nsyffp02bVrl9PrOnbsKI888oiMGzdO/Pz8XBojalZ6erpJ6FNSUsrGgoODzVmy2vsOqGnM2AFAFRUUFMjcuXPljDPOkIkTJzpN6nQN1sKFC81N/vLLLyep80LaPPqNN95wGNPTKXRGVtdaAjWNxA4ATtOxY8dkzpw5pqR63XXXSWJiYoXX9ejRw2ye0DV2Y8eOJaHzciNGjDCJXHl6ruyrr75qWUyoPSjFAsApysvLMzN0Tz/9tGlQ60yfPn3kscceMzd6Hx8fl8YIa2VmZpqzZPft21c2FhAQYJJ7fSMA1BQSOwCopNzcXHnttdfkmWeecVhDdbz+/fvLlClTTPNhErraa+nSpXLuuec6jA0aNMgcG8esLWoKpVgA+Au6RmrmzJnSunVrufvuu50mdYMHDzZHhK1atUouuOACkrpabtiwYeZc3/L0lJHnn3/espjg/ZixAwAnsrKy5JVXXpHnnntODh8+7PS6oUOHmpKr/kkyh+PfFOgay4SEhLKxOnXqmJNHdPcsUN1I7ADgONpY9t///rc5pzUjI8PpdVpme/TRR2XIkCEujQ+eZfny5XLmmWc6NCru3bu3mdmldyGqG6VYACjXg0xn3vRoL10j5yyp0zKrltS07EpSh7+iJXot4ZcXFxcnTz31lGUxwXsxYweg1tPzPXXd00svvWRKZ85cdNFFZoaub9++Lo0P3rGTulevXrJ9+/ayMZvNJmvWrJGePXtaGhu8C4kdgFp9cLuun9P+Yjk5OU6vGzVqlEnouAGjKjSJ012xeoZwKW2JsnbtWs4HRrWhFAug1jl48KApjekuV93tWlFSp5sgxowZY/qOaXNhkjpUVb9+/eTBBx90GNNTSB5//HHLYoL3YcYOQK2xf/9+01RYj3zSUyMq4uvrK5dddplMnjxZOnXq5PIY4f3Hz2mCp28Yyv+d0zWb2v8QqCoSOwBeb+/evWah+rx588yNtSLaMPaKK66Qhx9+2Jz5CtQUTep0nWZhYWHZmP6d27BhgzmdAqgKSrEAvNbu3bvNGa7t2rUzZ7pWlNTpAnY911MXtS9YsICkDjWue/fuZtd1eTt27DCzxEBVMWMHwOvEx8fLjBkz5J133nFYqF6e9g+bOHGiWfOka+0AV7Lb7WYjhW6cKL+u86effjI974DTRWIHwGvorNv06dPl/fffl+Li4gqv0a7/1157rTzwwAMSHR3t8hiBUtu2bTObcsqv99Q3GZs2bZLg4GBLY4PnohQLwONt3rzZbHjQzQ7vvvtuhUldvXr15PbbbzflWT0mjKQOVuvYsaN5I1JeYmKi3HfffZbFBM/HjB0Aj16E/sQTT8inn37q9BpdjK4Hsd97770SGRnp0viAv6JLBfSMYT12rLwlS5bIeeedZ1lc8FwkdgA8jh7HpAndF1984fSaoKAgufXWW02/usaNG7s0PuBUJCQkSLdu3SQ3N7dsrHnz5qbHXWhoqKWxwfNQigXgMVavXi0jRoyQPn36OE3q6tevb3YX7tmzx7Q4IamDu2vbtq1plH18z8W77rrLspjguZixA+D2VqxYYWbotDzlTEhIiNx5551yxx13SFhYmEvjA6pK14Vq6fX77793GNc3MCNHjrQsLngeEjsAbmvZsmUydepU+eGHH5xeEx4ebsqtWnbV5A7wVElJSdKlSxc5evRo2ViTJk1ky5Yt0rBhQ0tjg+egFAvAreh7TZ21OOuss8yicmdJXUREhCm1aslVS68kdfB0ulN71qxZDmOpqalyyy23WBYTPA8zdgDcgv4q+vbbb80MnZ6b6YzOYGg7iBtvvNFskAC87d/BRRddJIsXL3YYX7hwoYwdO9ayuOA5SOwAWEp/BX311VcmoVuzZo3T66KiokxTYT0iLDAw0KUxAq508OBB6dy5s2RkZJSNaSlWS7L6xgY4GUqxACxL6HRhuO5w1Z2uzpI6bfvw8ssvm8bCujGCpA7eTt/EaBPt8g4fPizXX3+9+XcDnAwzdgBcvvvvs88+k2nTppkGw860bNlSHn74YZkwYYLUrVvXpTECVtNb85gxY05ovr1gwQK56qqrLIsL7o/EDoDLOux//PHHJqHTkpIzbdq0MZshrrzySvH393dpjIA7OXTokCnJ6p+ldJOQHqGnM9lARSjFAqhRdrvdnN+qN6jx48c7TepiYmLMbMSOHTtk0qRJJHWo9Ro1aiSvvfaaw1hmZqZcc801lGThFIkdgBpRWFgob731ljnoXGffNGGriH7+vffek23btpkSk81mc3msgLsaNWqU/POf/3QY093jb7zxhmUxwb1RigVQrQoKCuTtt9+WGTNmSGJiotPrtBHro48+KqNHjxY/Pz+Xxgh4Et0dq/9eDhw4UDamrX42bdpkli4A5TFjB6BaHDt2TGbPnm1KqtqSxFlS16NHD7MgXDdOaF8ukjrg5PSIvLlz5zqM5eTkmCULuhkJKI/EDkCV5OXlyUsvvWQOMr/55pvNsUgV0bYmixYtkvXr18sll1wivr78+gEq64ILLpBrr732hCP39N8eUB6lWACnJTc31yzsfuaZZyQlJcXpdf3795cpU6bI+eefLz4+Pi6NEfAmWVlZ0q1bN9m7d2/ZWL169eS3336TM844w9LY4D5I7ACckuzsbFNyffbZZ+WPP/5wel1sbKxJ6IYNG0ZCB1STH3/8Uf72t785jA0YMEB++eUXNh7BoBYCoNKzBU8++aS0atVK7r//fqdJ3dChQ+WHH34wN5pzzz2XpA6oRmeffbbcdtttDmO//vqreaMFKGbsAJzUkSNH5N///rfMmjXL4ezK4+nMnO5yPfPMM10aH1Db6MYJ3YS0a9eusrE6derIunXrpGvXrpbGBuuR2AGoUHp6uknmXnzxRTNbd7JF3ZrQDRw40KXxAbXZypUrZciQIQ67Ynv27CmrV6+muXctRykWgIO0tDRzRque1frEE084TeouuugiWbNmjXz11VckdYCLDRo0SO655x6HsQ0bNsj06dMtiwnugRk7AEZqaqo899xz8uqrr5pSz8k64esMnc4OALBOfn6+9O7dW7Zu3Vo2pn0hddZOx1E7kdgBtdzBgwdl5syZMmfOHNOTriK6AeLSSy+VRx55xLRbAOAe4uLiTEuhoqKisjE9l1nX22krFNQ+lGKBWmr//v1md13r1q3lhRdeqDCp0ybCl19+uWzevFk++ugjkjrAzejM3OTJkx3GtmzZYloNoXZixg6oZbS56VNPPSXz5s0z57pWRMs5V1xxhVlrR+NTwL3pv2PtZadr7Mq/KdOWQ7oWD7ULiR1QS+zevdv0oXvrrbfEbrdXeI02OL3qqqvkoYceknbt2rk8RgCn5/fffzezd4WFhWVjem6znkoRGBhoaWxwLUqxgJeLj4+XiRMnSvv27c1B4hUlddoe4frrr5edO3fKm2++SVIHeBjtXzd16tQT/u3rmzTULszYAV5q+/btpvXB+++/79DrqjxtaqoHiz/wwAMSHR3t8hgBVB990zZ48GCzK7Y8PQlGT6xA7UBiB3gZ3egwbdo0s9nB2T9v3S2nM3R6NFizZs1cHiOAmrFjxw5zKoW2QimlPSm1VFu/fn1LY4NrUIoFvMTGjRtNSxItySxcuLDCpC4gIMA0NU1MTDQnSpDUAd5FNzvpWtrjN0wd38wY3osZO8AL+ljpCRFffPGF02uCgoLk1ltvlbvvvlsaN27s0vgAuJYuvfjb3/4my5Ytcxj/+uuv5fzzz7csLrgGiR3goXQdjSZ0ixcvdnqNll5uv/12ufPOOyUiIsKl8QGwdhe89p0sf4pM06ZNzVKNsLAwS2NDzaIUC3iYFStWmHfd2rfKWVIXEhJiGpRqCUbX25HUAbVLmzZtzBGB5R04cEDuuOMOy2KCazBjB3gILatoOwPd4eaMvhPXcqueKKHJHYDaS2/v+ibw22+/dRj//PPP5eKLL7YsLtQsEjvAjek/T03kNKH7+eefnV6nM3L33nuv3Hzzzex8A1Bm3759ZkNVZmZm2Zius9WSbKNGjSyNDTWDUizgpgndkiVLTE+qYcOGOU3q9Bf0s88+K3v27DG96EjqAJTXokULswO+vD/++MO8CWRexzsxYwe4Ef3n+NVXX5kZujVr1ji9LioqyiRy1113HccFAfjL3ytael20aJHD+AcffCCXXXaZZXGhZpDYAW5A/xnqL11N6NavX+/0uubNm8uDDz4o11xzjWkyDACVkZKSIp07d5b09HSHNblbtmwxbxThPSjFAhb3m/rkk0+kZ8+e5h21s6ROO8fPmTNHdu3aJbfccgtJHYBTEhkZKbNnz3YYy8jIMCfQML/jXZixAyxQVFQkH3/8sWlFou+YT9ayYPLkyXLllVeKv7+/S2ME4H3GjRtnjhssb968eTJx4kTLYkL1IrEDXHxI94cffmgSOj3T0ZmYmBh55JFH5PLLLxebzebSGAF4r7S0NOnSpYukpqaWjTVo0MCcJRsdHW1pbKgelGIBFygsLJS33npLOnbsaGbfnCV1+vn33ntPtm3bJldddRVJHYBqpa2RXn/9dYexrKwss26XeR7vQGIH1KCCggKZO3euOZhbSx26Rq4i+g564cKF5l2zztL5+fm5PFYAtcPIkSNlwoQJDmNLly4163jh+SjFAjXg2LFjMn/+fHnyySclKSnJ6XU9evSQRx991Gyc8PXlfRYA1zhy5Ih5Q5mcnFw2pq2TNm3aJG3btrU0NlQNdxKgGuXl5clLL71kfjHedNNNTpO6Pn36mPYmugv2kksuIakD4FKhoaFm00R5ubm5prKgm7vgubibANVAfyG+8MILZhfr7bff7vAuuLz+/fubBsTafPiiiy4SHx8fl8cKAOq8886TG264wWHsl19+OeGkCngWSrFAFWRnZ5veUHqslx7T40xsbKxMmTLFHA9GMgfAXRw9elS6d+8uiYmJZWN169aVDRs2mM1c8DwkdsBp0F1kr7zyijz33HNy+PBhp9cNHTpUHnvsMfMnCR0Ad7Rs2TLzO6q8vn37ysqVK9mZ74EoxQKnuOD4iSeekFatWsnDDz/sNKnTmTn9Zfnjjz/K2WefTVIHwG2dddZZcueddzqMrV27Vp555hnLYsLpY8YOqAQ9X1HXnehHZmam0+suuOACs8t14MCBLo0PAKq68Ut36e/cubNsTE+70QRPS7XwHCR2wF90aX/++efl5ZdfNmtRnNGNEJrQafkCADzRr7/+atYD6xnWpTSp081ederUsTQ2VB6lWKACetzO/fffb0qu2ovOWVI3atQoiYuLM61LSOoAeLIBAwaY33vlbdy40Sw/gedgxg4o5+DBgzJz5kzTgV1LExXR9XKXXnqpOcu1W7duLo8RAGqyubr22dy8eXPZmJ6Es2rVKt68eggSO0BE9u/fbxYK6xmK+outItpEeNy4cTJ58mTp3Lmzy2MEAFfQVif9+vUTu91eNqatT7Sher169SyNDX+NUixqtb1798rNN99sTorQEyMqSur03epVV10lW7dulffff5+kDoBX69mzp1kzXN62bdtOGIN7YsYOtdLu3bvN2rm33nrL4V1pedq/SRO6hx56SNq1a+fyGAHAKoWFhWZ3v64hLr8M5eeff5bBgwdbGhtOjsQOtUp8fLzMmDFD3nnnHafnIeoWfz0v8cEHH5TWrVu7PEYAcAdbtmyRXr16SUFBQdmYVjd0Q0VQUJClscE5SrGoFbZv3y5XXnmldOjQwczSVZTU6XZ+Lcvu2rVLXnvtNZI6ALWaLjs5fkdsQkKCPPDAA5bFhL/GjB28/h3ntGnTZOHCheLsr7ouBr7++uvNNv9mzZq5PEYAcFf6JnjIkCFmV2x5S5culXPOOceyuOAciR28UmnvpU8//dTpNQEBAWaG7t5775XIyEiXxgcAnrSERRsVl28BFR0dLb///rs0aNDA0thwIkqx8Cq60Pfiiy82R+M4S+p0bYiWEvbs2SPPPvssSR0AnERMTIw8/fTTDmNJSUly9913WxYTnGPGDl5h9erVZoZu8eLFTq+pX7++3H777eaw64iICJfGBwCeTI8ZGzZsmPz4448O419++aVceOGFlsWFE5HYwaOtXLlSpk6dKkuWLHF6TUhIiEnm7rjjDgkLC3NpfADgLbTK0bVrV8nOzi4b04qHrmUODw+3NDb8D6VYeKRly5aZd496YLWzpE6TOJ3F0ybE//rXv0jqAKAK9Ozs559/3mEsJSVFbrvtNstiwomYsYPH0L+qWgZ4/PHHTZNMZ7TMqhsidGOEll8BANX3e3j48OHyzTffOIx/8sknMnr0aMviwv+Q2MHt6V/Rb7/91pRctfTqTOPGjU3LkhtvvJHmmQBQQ5KTk6VLly5y5MgRhzfUWpLV38OwFqVYuHVCp5shBgwYIOeff77TpC4qKkpmzZoliYmJcs8995DUAUAN0n6ferZ2eWlpaeZNNXNF1mPGDm5H/0ouWrTIzNCtX7/e6XXNmzc3x35dc801pskwAMB1v6e19Pr55587jL/77rtyxRVXWBYXSOzgZtvp9ZeEbnjQBsPOtGzZUh566CG5+uqrpW7dui6NEQDwX3/88Yc5dkxn60qFhobK5s2bOcXHQpRi4RZH1nz44YfSrVs3ufTSS50mdW3atJE333zTdEG/4YYbSOoAwEK6nm727NkOY7ru7rrrrqMkayESO1jGbrebaXt9xzd+/Hiz8NZZ1/MFCxbIjh07ZNKkSeLv7+/yWAEAJ9I34/r7u7yvv/7avAmHNSjFwuUKCwvlvffek+nTp8uuXbucXtexY0d55JFHZNy4ceLn5+fSGAEAlZOenm7eoGtPu1LBwcHmLFntfQfXYsYOLlNQUCBz586VM844QyZOnOg0qdNt9AsXLjS/FC6//HKSOgBwY3rqhP5uL09Pp9AKi66dhmuR2KHGHTt2TObMmWNKqrr2QtuSVKRHjx7y6aefmjV2Y8eOJaEDAA+h58VqIleeNpR/9dVXLYuptqIUixqTn59v3sU99dRTpqGlM3369JHHHntMRowYIT4+Pi6NEQBQPTIzM81Zsvv27SsbCwgIMG/W9Y09XIPEDtUuNzdXXnvtNZk5c6YcPHjQ6XX9+/eXKVOmmObDJHQA4PmWLl0q5557rsPYoEGDzDGQVGFcg1Isqo2uqdBkrnXr1nL33Xc7TepiY2PNEWGrVq2SCy64gKQOALzEsGHDzDnd5empQc8//7xlMdU2zNihyrKysuSVV16R5557Tg4fPuz0uqFDh5qSq/5JMgcA3vsmX9dMJyQklI3VqVPHnCSku2dRs0jscNq0EaWeF/jCCy9IRkbGSd/BPfroo3LmmWe6ND4AgDWWL19ufueXTzF69+5tKjX0Iq1ZlGJxWj2LdG2c9ifSGThnSZ2WWXUK/rvvviOpA4BaZPDgwWZJTnlxcXFmMx1qFjN2qDQ9D1Bn53SW7ujRo06vu+iii8wMXd++fV0aHwDAfeTl5UmvXr1k+/btZWM2m03WrFkjPXv2tDQ2b0Zih7+Umppq1s9pP6KcnByn140aNcqcFKH/kAEA0CROd8XqmeCltCXK2rVrOe+7hlCKhVO6q1Wn0nWXq+52rSip000QY8aMMX2KPvvsM5I6AECZfv36yYMPPugwpqcKPf7445bF5O2YscMJ9u/fL88884y8/vrr5tSIivj6+pozXCdPnswuJwDASY+T1KU5mzZtcriH6Bps7WeK6kVihzJJSUlmYeubb75p/iFWRBtMXnHFFfLwww+bM18BAPgrWtXR5K6wsLBsTO8hGzZsMKdToPpQioU5u/X666+Xdu3ayezZsytM6nTBq54DqItgFyxYQFIHAKi07t27m24K5e3YscNUfVC9mLGrxeLj42XGjBnyzjvvOCxsLU/7DU2cONGskdC1dgAAnA673W42UujGifLrtH/66SdaYlUjErtaSGfdpk+fLu+//74UFxdXeI12Cb/22mvlgQcekOjoaJfHCADwPtu2bTOtTsqv39ZJA11/FxwcbGls3oJSbC2yZcsWGT9+vHTq1EnefffdCpO6evXqye233y67d+82x4SR1AEAqkvHjh3NxMLxy4Huu+8+y2LyNszY1ZJFq9OmTZNPPvnE6TW6eFUPbr733nslMjLSpfEBAGoPXfqjZ4brsWPlLVmyRM477zzL4vIWJHZeTA9cfuKJJ+Q///mP02uCgoLk1ltvNf3qGjdu7NL4AAC1U0JCgnTr1k1yc3PLxpo3b2563IWGhloam6ejFOulnb71WC89cNlZUle/fn2zG2nPnj2mxQlJHQDAVdq2bWsa3x/fQ/Wuu+6yLCZvwYydF9Fmj1OnTjXT2c6EhITInXfeKXfccYeEhYW5ND4AAErpOm8tvX7//fcO41988YWMHDnSsrg8HYmdF/j5559NQnf8P47yNInTcuttt91mkjsAANyhMX6XLl3k6NGjZWNNmjQxm/0aNmxoaWyeilKsh9J8/IcffjALUM866yynSV1ERIQpte7du1ceeeQRkjoAgNvQzguzZs1yGEtNTZVbbrnFspg8HTN2HkZ/XN99952ZoVuxYoXT63TN3P333y833nij2SABAIC73td0XfjixYsdxhcuXChjx461LC5PRWLnIfTH9PXXX5uEbvXq1U6vi4qKMk2Fr7vuOgkMDHRpjAAAnI6DBw9K586dJSMjo2xMS7FaktXSLCqPUqwHJHS6kFQPT77wwgudJnW6Tfzll182jYV1YwRJHQDAU+ikhDbFL+/w4cPmHHPmn04NM3ZuvFvo888/N33otMGwMy1btpSHH35YJkyYIHXr1nVpjAAAVBdNR8aMGSOffvqpw/iCBQvkqquusiwuT0Ni54YdufWECE3odAramTZt2pg+dFdeeaX4+/u7NEYAAGrCoUOHTElW/yylm/42b95sKlP4a5Ri3YTdbpf33nvPbPu+7LLLnCZ1MTEx5t3Ljh07ZNKkSSR1AACv0ahRI3nttdccxjIzM+Waa66hJFtJtWLGTmfB0tPTzRZq/TiUkiLH8vKkuKhIfP38pG5AgDSKjDQLNPUjPDxc/Pz8XBJbYWGhvP/+++ZQ5Pj4+JMenKztSsaNG+ey2AAAsIJWo959912HMU34dM2dp9zfreLViZ3urtH1ab+vXy/5OTlSYrdLcF6ehKSni7/dLr4lJVLs4yOFNptkhodLdkCA+NhsUi8oSLr26iXdu3evsdMZCgoK5J133pEZM2aYDQ/O6Azeo48+KqNHj/b6v4wAAJTev/X+d+DAgbIxbd2lZ8m2bt3are/vVvPKxE7/IqxcvlwS4+PFPzdXopP2SVR6uoTk5Ih/UZHTxxX6+UlmUJAcDA+XpOgWUhgYKK1jYiR2yBCzY6c6HDt2TObPny9PPvmk6bjtTI8ePUxCd/HFF4uvLxVzAEDtoi2+hg8f7jCm3SEmXHml7Nm1y+3u7+7CqxI7XaemTXvXrlghwWlp0m5vkjRPSxO/4uJTfq4iX1/ZHxEhu1pGS3ZEhPSNjZXY2Fix2WynFVt+fr7MnTvXnAKRnJzs9Lo+ffrIY489JiNGjBAfH5/Tei0AALyB9mTVe6dWrAYNGiSxfftK04IC6XjgoNvc392N1yR2KSkpsnjRIsnYnywd4uMlJjnZTMVWlU7lxjdrJttjYiS8eTMZPnKkREZGVvrxubm58vrrr8szzzxjGjA6079/f5kyZYqcf/75JHQAAIhIVlaWOTazT8+e0iwsTGK2b5dmO+OlSURElROx4ire392VVyR2eg7q5wsXSuCBg9J72zZpkJtb7a+RFRgocR07Sm7TpjJq3FjTP640oVyyZIl07dpVevXqVXZ9dna2zJkzR2bOnCl//PGH0+fVdwma0A0bNoyEDgCA4+7vH779tvgnJUnHuDgJzMoy43X860jDiAjxqcH7u6fy+MROf+iffvCBNNybJP22bhXbaUzLVpbd11dWd+4k6dHRMnr8eLMBYsiQIWYnTum5dhdccIHpnv3cc89JWlqa0+caOnSoKbnqnyR0AAA4v793+HWV5B896vD5BvUbSHBwcI3c31t6cHLn0YmdzpZpJh+auEcGbtlSLaXXykzdrurSWTJatpLPvvw/Wb58ednnGjRoYDY6HDlyxOnjdWZON0WceeaZNR4rAADecH/3KS6WPw4dkqIie7mrfEzfO/9qWhtX/Of9/Uir1nLZVVd6bFnW15M3SuiaOi2/9t+61SVJndLX6b9lq/jt3SMd2rVzaEGiawGcJXU6k7dy5Ur57rvvSOoAADiF+7tWtsJCQ00y9z8lcuRIhpRU8/094OAB+WrRIhOHJ/LYxE53v+pGCV1TV5Pl14oU5ORIu19/lWbh4WaXzslcdNFFsmbNGvnqq69k4MCBLosRAABvur/XqVNHgoOCTmjyX1hQUG2vbSsult5bt0l6crKZjPFEvp7ap05bmuju15rYKHEyBYWFZlYuKCvL7M7RrdcVTdeOGjVK4uLiZNGiRdK3b1+XxggAgDfe3+s3qC82m+NRmtVdrwvJzZUzdsbLmuXLT9rNwl15ZGKnzYe1T522NHEl/ctz+PDhsr9GTXfulIjsbIk9btauWbNm8sknnzjskgUAAFW7v/uIjzkWzN8kdz4SGBhkZvKqW/vkZBPHinLr6D2FxyV2eoyIniihzYddta6ulL2wUEpK/jctrK/fYtcuad+6tYSEhJSNawPiPXv2uDQ2AAA8WWXv7zY/P7NpomlUlISGhFRLy5Pj6eu33ZskiTt3mrg8iccldno2nB4joh2nXc3m/993COVF7NsngXa7OXeuVOPGjaV58+Yujw8AAE9l5f29Ii3S0sSWmyubNm0ST+JR52cUFRWZA3/1bLjTOUakqjSli2jYUDKzskS7xPj5+ZqdOq32J8vg/v1NPx09c+6uu+6qkalhAAC8kdX394poHC337ZNNcXEyePBghy4YtXbGThdBXnHFFU4/v27dOrnvvvsq/Xzp6emSn5MjaQm7ZOSG9eajx8oV8ve4dea/n0hIqFK8KceOyS3btso569bKJb9tkNu3bZO0ggL5LDVVnkrcba7RhK1RRIQ0btRIGoY3lPCwcGmblyeh9eubr3X37t0yfvx4+fDDDyt8Dd1M8cILL5j/3r59u/To0UN69uwpq1evPqXvhTNffvmldOnSxfTT27x5c5WfDwBQO+gRXXpPKv3Iy8s75efQ4zNPR+n9PSo93WH85aS9Mnx9nIxYH2fuy/vy80/6PG/s31elx/f7dZXD/0cd/m9cGt/JzJo1yxxaUFV6atU555xjJoruvffemm1QrOedXn/99WIlTVS++vhjueinZWVboP+5aZM81rattD9uC3SRzqidwokO+m245Lff5PKoKBnz5y7XtZmZEmKzyebsbNmZmyMPtm5z4uP0B1FYKP935pny8VeLZcuWLWZcEytNbJs0aeL0NZ966inzD+l0fnj67qaidw/x8fHmczfeeKO8/PLLJskDAOCvREREnPTEpJp6Dr1nbdu27YT7+/qsLHlh7x6Z17mL+Pv6msmXAD9fCTluV+zxidmaAQOr5fGq0M9PvjzrTBk+ZsxJ76etWrUyOUplT8IoLi42ecLxjh07ZiZ6NJdISEiQZ599VmqsFPvqq6+axC4nJ0duueUW86IamCYn5557rhw9elRuvvlmUyPX8qQmFS1atJBLL73UzMz9/vvvMmHCBPMY9e2338rWrVvNdbqDVP8iTJw40Rwhojte3nrrLfONuvrqq83GBP1C9+/fL2OGDHHat+7stWtkeKNGsjwjQ+5v1VoOFRbI2wcOSGFxiQwMDZWH2/w3MfvPH6knjK/MPCKBfr5lSZ3q++eGCE3sSn13OE3m7Nsn9pISaehnk8mNG0tASbHsWL9eEhMTy67Tr/Pf//63+dr0h6cJ3BdffGH+f+fOnWZaV48d0/FvvvlGJk2aJO+88445jiw3N9ecH6tJmj7P/fffb65/8cUXZd++fWZjRufOneXxxx8/4XugyZ5+5Ofnm2sDAwMr+yMGANRier/RqlN5P//8s7mXacIRExNj7vlauXr44YdNIqOzVKNHj5brrrvOJCHaDqxTp05mxk8nGDRf0HufmjFjhrRv397kBdqof8SIEfLLL7/IAw88YPKELz76SOYdPSoDQkLkgVatJTU/X0L8bDoDZSZfIuvWLYvrl4wMeSlprxwrLpaYwECZEdNeXk5KkqN2+3+refXrS2xomITZ/E1Sp/7q8XWOS7Re379PvklLk/QtmyUhJUVee+01Mz59+nRTldNcR/MW/X7oRI72tdW8RStzej/X2UuNW3MfrcjpvVt72+r9+7fffpMNGzZIQECAw2vWrVvXfG+O/znUyIydvrhOy+oPU9t46A9GkzFNODTT1h+Mv7+/+YL1L4cmerqTpDSxu+2226Rbt27mh6/Po8mHNv8rTexuvfVWiY6ONkmMnrn63nvvmW+OJnaazes36ZGHH5bP33pL/tO2XVlc5WfsNLG7pllz+WfTprIrN1de3LtHXujQUWw+PnLfjh0m6WtRr16F4/vy82R/fr483KbtCV+7lmJLZ+wy7YUS7Osnhw4dko/TD0tuSYlcGRYml6elSa9Bg+TLr78+7R8GAAC1UYvmzeWh2FjptG6dzEhNlbODg6V7QIDckpwsxSUl0icwSEZFRkq/Ro0kvbBQ7tq+XV7r1Enq+fmZe3pD/zrm3t+v3Ixbtt0ul23aaCp4muT9o3Fj6Vq/fqUerxNEP6QflkfbtJVf27eXp1f/anKTpKQkMymjhw5oEqYlWp2MKj9jp50xNDlbu3atmVzRhO+NN96Qhg0bSrt27WT9+vUmHzoZndzS56vRGbtSOtOm67imTZtm/l9n8FJTU2Xp0qUmEVM6Q6WzbOW3COupC1OnTjV94MaOHStt/pw9K6Vnruo3Sunn77jjjrLPXXzxxebPZpGRcvi4Q4CPd0FEhPlz1ZEj8tvRo6amrvKLiqVLcLBJ3ioar2zV9kD+MZm2K96svdNMv1O9emY8Jjxc1q1fX7knAQAAZQ6np8uT33wjdfLy5FhJibSvW1cGBgXJG82by295eRKXlyc3xu+Uf9tsUqhVstwcGbtpo3lsQXGxDA0PP+E5g202+U/PXrL6yBFTlZu4ebO82KGDFFTi8cuPZMhP6RmyLmuD5G3dIrk2m6m2aa6is3Sa1ClN6o6nCZ2ukyv9nE5w6eP+8Y9/mBnLv0rqqqrSiV2HDh3Mnzob93//93/SsmXLU3qhyy+/XPr162ceq6Xbjz/++KTX6xRnqdJvoJSUmMz9ZDT7NpdKiYyNjJTboh3jfPtAcoXjKzIy5Ns0bT58ctN2J8iExo2li4+PrMzJkW/+TDSv6tFDvsjPl0XM2AEAcEo6xMTIrW3aSJvjWotoZa1PYKD5CPWzyffph2VwaJgMDQuXp9q3/8vntfn4SGxYmPkIt/nL0ko+vrhE5NboaLmkSRPZ2Ka1HB00SC655BKToFWFK5ZHVXpXrJZK1XnnnWfq7aW0TqyGDRsms2fPLkv+MjMzHR6v9eK2bduaViD6HLq+rjwt6b7//vvmv7U0q0ngCcGewlbjgSGh8tWhQ5JRWGj+/3BBgfxRUOB0fFBoqGQX2U3ZtdS6zEzZmZPj8LzZRUXSKiTUlJK/LTd7+EdurplmBQAApyZhzx45+ud9OcNul8N2uyQVFEjyn2O+Pr6SLCXStG5d6dmgvqzOPCLJf+5w1ZJr6W5XPx8fU3pVu3NzJenPnb266kyXVP3V40sNDguVj1NTJK+oSIp9fCX9yBGT12iuM3/+fLPmUJXulq1fv75ZgqY0f/n+++9N1VKv++yzz2TIkCHiKpWesbvmmmvMn48++qgpk+pUot1uN+vt3n33XTOuCyW7du1qkh5dO1e+Sa/WpvU6XYens316lqpOV5b617/+ZdbTvf3222WbJ45XNyBASipZM40JCpKbWkTLhM2/mx+oLp58Oqa90/HGderIqx07yRO7d8sr+5Kkrq+vWVCp9fXyNIO/cetWCfW3Sa/69WXfn4nfwi1bJPG4MrEukty1a5f5fuj3STdGaPKqSe2TTz5pytmaDN50001mgeqcOXPM57W8rTtl9axZ/R7rItR58+Y5XO+MbsTQJFzXP4aGhkpsbKxZrwgAwMnohkfddFeeJiiPPfaYFBYWmkrazJkzzfoxXS+/Zs0acz/XTYDXXnutDB8+XCZPnixff/21ufe89NJLZiJIu2roc+u9/fzzz5crr7zSVAF1/X3pLtJ77rpLnvv4Y6mXn282MTwVE2OmzZ7YnSDZ9iLTSLZzULBcGdXUVOamtYuR27Zvk8LiYhPX5NZtzBr6UY2bmNYmuvlRq3NTExLMhIyqzONLnRkWbtbqj934m2Rv2ybBq1bKP6++2nyNem/We7rmM1qW1ZxIvx9nn322KbXqsjTdAKnfp9LNE3p9ZU+kOuOMM8w6fv2e6yaNX3/99ZQOPaj05gl3oH/BdixZIueu+lXcSUFhgXzTp498tW2b/PDDD2XlYz08OCwszOrwAABwa+56f1ffDRwgZ/z972bdnCfwqJMntCdcXECA6Svj/2cG7g586gVIUcOGZlt306ZNTaatM24kdQAAeO79vdDPT7IDAk7ak9bdeFxi52OzSWZQkERkZYm70Hg0Lq2h6+JKV9Aav/a1K2/MmDFmGhwAAE/i7vf3JtWc2GmHkONnALXSpz17a1Vip/X5ekFBcjA83K1+8Acb/jeuirY91xSt6+sHAACerrbd3xs2bFi2+dSjzoqtbroJoWuvXpIU3UKKKjiKwwoax94WLaRb794ec0AwAADuhPt79XGP794p6N69uxQGBsr+PxsRW21fRITYAwNrvOEgAADejPt7LU3sdENC65gY2dUyWoore1xEDdHXT2gZLa3bt2ejBAAAVcD9vZYmdip2yBDJjoiQ+GbNLI1jZ7NmJo7YwYMtjQMAAG/A/b2WJnZRUVHSNzZWtsfESJYLjueoSGZgoOxoHyP9Bg828QAAgKrh/l5LEzulXa3DmjeTuI4dxe7ihZb6enGdOkp4s2YyaNAgl742AADejPt7LU3s9AiTC0eOlNymTWV1504uq8fr6+jr5UU1leEjR5o4AABA9eD+XksTOxUZGSmjxo2V9OhoWdWlc41n9vr8+jr6evq6+voAAKB6cX8/fR51Vqwze/fulc8XfiSBBw5I723bpEFubo3U3HV6VjN5/aHrwccAAKDmcH+vpYmdSklJkcWLFknG/mTpEB8vMcnJ4lsNX5pOzeruGF1IqTV3nZ715EweAABPwv29liZ2ym63y4oVK2TtihUSnJYmbfcmSYu0NPErLj6tjtPanFD72OiWZ90dowspPbXmDgCAp+L+XksTu1IHDhyQlStWSOLOnWLLzZWW+/ZJ1OF0CcnJEf+iIqePK/TzMwf+6tlweoyIdpzW5oSxHrrlGQAAb8L9vZYmdqUyMjJk06ZNsikuTvJzcqTEbpfgvDxpkJ4hdex28S0plmIfXymw2SQrPEyyAwLEx2YzB/7q2XB6jIindZwGAMDbcX+vpYldqaKiIklPT5fU1FTzcSglRQry86XIbhc/m03q1KsnjSIjpUmTJuYjPDzcow78BQCgNuL+XksTOwAAgNrAo/vYAQAA4H9I7AAAALwEiR0AAICXILEDAADwEiR2AAAAXoLEDgAAwEuQ2AEAAHgJEjsAAAAvQWIHAADgJUjsAAAAvASJHQAAgJcgsQMAAPASJHYAAABegsQOAADAS5DYAQAAeAkSOwAAAC9BYgcAAOAlSOwAAAC8BIkdAACAlyCxAwAA8BIkdgAAAF6CxA4AAMBLkNgBAAB4CRI7AAAAL0FiBwAA4CVI7AAAALwEiR0AAIB4h/8HAPSWz2Mn9rYAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "graph_search_space = tpot.search_spaces.pipelines.GraphSearchPipeline(\n", " leaf_search_space = fss_search_space,\n", " inner_search_space = tpot.config.get_search_space([\"transformers\"]),\n", " root_search_space= tpot.config.get_search_space([\"KNeighborsClassifier\", \"LogisticRegression\", \"DecisionTreeClassifier\"]),\n", " max_size = 10,\n", ")\n", "\n", "graph_search_space.generate(rng=4).export_pipeline().plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Optimize with TPOT\n", "\n", "For this example, we will optimize the DynamicUnion search space" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/ketrong/Desktop/tpotvalidation/tpot/tpot/tpot_estimator/estimator.py:456: UserWarning: Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.\n", " warnings.warn(\"Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.\")\n", "Generation: 100%|██████████| 5/5 [00:41<00:00, 8.33s/it]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "0.9838836477987423\n" ] } ], "source": [ "import tpot\n", "import sklearn.datasets\n", "from sklearn.linear_model import LogisticRegression\n", "import numpy as np\n", "\n", "\n", "final_classification_search_space = SequentialPipeline([dynamic_fss_space, classification_search_space])\n", "\n", "est = tpot.TPOTEstimator(generations=5, \n", " scorers=[\"roc_auc_ovr\", tpot.objectives.complexity_scorer],\n", " scorers_weights=[1.0, -1.0],\n", " n_jobs=32,\n", " classification=True,\n", " search_space = final_classification_search_space,\n", " verbose=1,\n", " )\n", "\n", "\n", "scorer = sklearn.metrics.get_scorer('roc_auc_ovr')\n", "\n", "est.fit(X_train, y_train)\n", "print(scorer(est, X_test, y_test))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see that this pipeline performed slightly better and correctly identified group one and group two as the feature sets used in the generative equation." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Pipeline(steps=[('featureunion',\n",
       "                 FeatureUnion(transformer_list=[('featuresetselector-1',\n",
       "                                                 FeatureSetSelector(name='group_two',\n",
       "                                                                    sel_subset=['d',\n",
       "                                                                                'e',\n",
       "                                                                                'f'])),\n",
       "                                                ('featuresetselector-2',\n",
       "                                                 FeatureSetSelector(name='group_one',\n",
       "                                                                    sel_subset=['a',\n",
       "                                                                                'b',\n",
       "                                                                                'c']))])),\n",
       "                ('randomforestclassifier',\n",
       "                 RandomForestClassifier(max_features=0.0530704381152,\n",
       "                                        min_samples_leaf=2, min_samples_split=5,\n",
       "                                        n_estimators=128, n_jobs=1))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "Pipeline(steps=[('featureunion',\n", " FeatureUnion(transformer_list=[('featuresetselector-1',\n", " FeatureSetSelector(name='group_two',\n", " sel_subset=['d',\n", " 'e',\n", " 'f'])),\n", " ('featuresetselector-2',\n", " FeatureSetSelector(name='group_one',\n", " sel_subset=['a',\n", " 'b',\n", " 'c']))])),\n", " ('randomforestclassifier',\n", " RandomForestClassifier(max_features=0.0530704381152,\n", " min_samples_leaf=2, min_samples_split=5,\n", " n_estimators=128, n_jobs=1))])" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "est.fitted_pipeline_" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Combining with existing search spaces\n", "\n", "As with all search spaces, FSSNode can be combined with any other search space. \n", "\n", "You can also pair this with the existing prebuilt templates, for example:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Pipeline(steps=[('featuresetselector',\n",
       "                 FeatureSetSelector(name='group_two',\n",
       "                                    sel_subset=['d', 'e', 'f'])),\n",
       "                ('pipeline',\n",
       "                 Pipeline(steps=[('maxabsscaler', MaxAbsScaler()),\n",
       "                                 ('rfe',\n",
       "                                  RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,\n",
       "                                                                     min_samples_leaf=8,\n",
       "                                                                     min_samples_split=14,\n",
       "                                                                     n_jobs=1),\n",
       "                                      step=0.753983388654)),\n",
       "                                 ('featureunion-1',\n",
       "                                  FeatureUnion(transformer_lis...\n",
       "                                  FeatureUnion(transformer_list=[('skiptransformer',\n",
       "                                                                  SkipTransformer()),\n",
       "                                                                 ('passthrough',\n",
       "                                                                  Passthrough())])),\n",
       "                                 ('histgradientboostingclassifier',\n",
       "                                  HistGradientBoostingClassifier(early_stopping=True,\n",
       "                                                                 l2_regularization=9.1304e-09,\n",
       "                                                                 learning_rate=0.0036310282582,\n",
       "                                                                 max_features=0.238877814721,\n",
       "                                                                 max_leaf_nodes=1696,\n",
       "                                                                 min_samples_leaf=59,\n",
       "                                                                 n_iter_no_change=14,\n",
       "                                                                 tol=0.0001,\n",
       "                                                                 validation_fraction=None))]))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "Pipeline(steps=[('featuresetselector',\n", " FeatureSetSelector(name='group_two',\n", " sel_subset=['d', 'e', 'f'])),\n", " ('pipeline',\n", " Pipeline(steps=[('maxabsscaler', MaxAbsScaler()),\n", " ('rfe',\n", " RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,\n", " min_samples_leaf=8,\n", " min_samples_split=14,\n", " n_jobs=1),\n", " step=0.753983388654)),\n", " ('featureunion-1',\n", " FeatureUnion(transformer_lis...\n", " FeatureUnion(transformer_list=[('skiptransformer',\n", " SkipTransformer()),\n", " ('passthrough',\n", " Passthrough())])),\n", " ('histgradientboostingclassifier',\n", " HistGradientBoostingClassifier(early_stopping=True,\n", " l2_regularization=9.1304e-09,\n", " learning_rate=0.0036310282582,\n", " max_features=0.238877814721,\n", " max_leaf_nodes=1696,\n", " min_samples_leaf=59,\n", " n_iter_no_change=14,\n", " tol=0.0001,\n", " validation_fraction=None))]))])" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "linear_search_space = tpot.config.template_search_spaces.get_template_search_spaces(\"linear\", classification=True)\n", "fss_and_linear_search_space = SequentialPipeline([fss_search_space, linear_search_space])\n", "\n", "# est = tpot.TPOTEstimator( \n", "# population_size=32,\n", "# generations=10, \n", "# scorers=[\"roc_auc_ovr\", tpot.objectives.complexity_scorer],\n", "# scorers_weights=[1.0, -1.0],\n", "# other_objective_functions=[number_of_selected_features],\n", "# other_objective_functions_weights = [-1],\n", "# objective_function_names = [\"Number of selected features\"],\n", "\n", "# n_jobs=32,\n", "# classification=True,\n", "# search_space = fss_and_linear_search_space,\n", "# verbose=2,\n", "# )\n", "\n", "fss_and_linear_search_space.generate(rng=1).export_pipeline()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Getting Fancy\n", "\n", "If you want to get fancy, you can combine more search spaces in order to set up unique preprocessing pipelines per feature set. Here's an example:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "dynamic_transformers = DynamicUnionPipeline(get_search_space(\"all_transformers\"), max_estimators=4)\n", "dynamic_transformers_with_passthrough = tpot.search_spaces.pipelines.UnionPipeline([\n", " dynamic_transformers,\n", " tpot.config.get_search_space(\"Passthrough\")],\n", " )\n", "multi_step_engineering = DynamicLinearPipeline(dynamic_transformers_with_passthrough, max_length=4)\n", "fss_engineering_search_space = SequentialPipeline([fss_search_space, multi_step_engineering])\n", "union_fss_engineering_search_space = DynamicUnionPipeline(fss_engineering_search_space)\n", "\n", "final_fancy_search_space = SequentialPipeline([union_fss_engineering_search_space, classification_search_space])" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Pipeline(steps=[('featureunion',\n",
       "                 FeatureUnion(transformer_list=[('pipeline-1',\n",
       "                                                 Pipeline(steps=[('featuresetselector',\n",
       "                                                                  FeatureSetSelector(name='group_one',\n",
       "                                                                                     sel_subset=['a',\n",
       "                                                                                                 'b',\n",
       "                                                                                                 'c'])),\n",
       "                                                                 ('pipeline',\n",
       "                                                                  Pipeline(steps=[('featureunion',\n",
       "                                                                                   FeatureUnion(transformer_list=[('featureunion',\n",
       "                                                                                                                   FeatureUnion(transformer_list=[('zerocount',\n",
       "                                                                                                                                                   ZeroCount())])),\n",
       "                                                                                                                  ('passthrough',\n",
       "                                                                                                                   Passth...\n",
       "                                                                                                                                                   KBinsDiscretizer(encode='onehot-dense',\n",
       "                                                                                                                                                                    n_bins=11)),\n",
       "                                                                                                                                                  ('rbfsampler',\n",
       "                                                                                                                                                   RBFSampler(gamma=0.0925899621466,\n",
       "                                                                                                                                                              n_components=17)),\n",
       "                                                                                                                                                  ('maxabsscaler',\n",
       "                                                                                                                                                   MaxAbsScaler())])),\n",
       "                                                                                                                  ('passthrough',\n",
       "                                                                                                                   Passthrough())]))]))]))])),\n",
       "                ('randomforestclassifier',\n",
       "                 RandomForestClassifier(bootstrap=False,\n",
       "                                        class_weight='balanced',\n",
       "                                        max_features=0.8205760841606,\n",
       "                                        min_samples_leaf=16,\n",
       "                                        min_samples_split=11, n_estimators=128,\n",
       "                                        n_jobs=1))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "Pipeline(steps=[('featureunion',\n", " FeatureUnion(transformer_list=[('pipeline-1',\n", " Pipeline(steps=[('featuresetselector',\n", " FeatureSetSelector(name='group_one',\n", " sel_subset=['a',\n", " 'b',\n", " 'c'])),\n", " ('pipeline',\n", " Pipeline(steps=[('featureunion',\n", " FeatureUnion(transformer_list=[('featureunion',\n", " FeatureUnion(transformer_list=[('zerocount',\n", " ZeroCount())])),\n", " ('passthrough',\n", " Passth...\n", " KBinsDiscretizer(encode='onehot-dense',\n", " n_bins=11)),\n", " ('rbfsampler',\n", " RBFSampler(gamma=0.0925899621466,\n", " n_components=17)),\n", " ('maxabsscaler',\n", " MaxAbsScaler())])),\n", " ('passthrough',\n", " Passthrough())]))]))]))])),\n", " ('randomforestclassifier',\n", " RandomForestClassifier(bootstrap=False,\n", " class_weight='balanced',\n", " max_features=0.8205760841606,\n", " min_samples_leaf=16,\n", " min_samples_split=11, n_estimators=128,\n", " n_jobs=1))])" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "final_fancy_search_space.generate(rng=3).export_pipeline()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Other examples" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## dictionary" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
def
1621.315442-1.0392580.194516
168-1.908995-0.953551-1.430472
2140.1811621.022858-2.289700
8952.825765-1.2055201.147791
154-2.3004811.0231730.449162
............
32-1.7930622.209649-0.045031
829-0.2214091.6887500.069356
1760.141471-1.8802941.984397
124-0.3599521.1417582.019301
350.1713120.0793320.178522
\n", "

750 rows × 3 columns

\n", "
" ], "text/plain": [ " d e f\n", "162 1.315442 -1.039258 0.194516\n", "168 -1.908995 -0.953551 -1.430472\n", "214 0.181162 1.022858 -2.289700\n", "895 2.825765 -1.205520 1.147791\n", "154 -2.300481 1.023173 0.449162\n", ".. ... ... ...\n", "32 -1.793062 2.209649 -0.045031\n", "829 -0.221409 1.688750 0.069356\n", "176 0.141471 -1.880294 1.984397\n", "124 -0.359952 1.141758 2.019301\n", "35 0.171312 0.079332 0.178522\n", "\n", "[750 rows x 3 columns]" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import tpot\n", "import pandas as pd\n", "import numpy as np\n", "from sklearn.linear_model import LogisticRegression\n", "import sklearn\n", "\n", "subsets = { \"group_one\" : ['a','b','c'],\n", " \"group_two\" : ['d','e','f'],\n", " \"group_three\" : ['g','h','i'],\n", " }\n", "\n", "fss_search_space = tpot.search_spaces.nodes.FSSNode(subsets=subsets)\n", "\n", "selector = fss_search_space.generate(rng=1).export_pipeline()\n", "selector.set_output(transform=\"pandas\")\n", "selector.fit(X_train)\n", "selector.transform(X_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## list" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
def
1621.315442-1.0392580.194516
168-1.908995-0.953551-1.430472
2140.1811621.022858-2.289700
8952.825765-1.2055201.147791
154-2.3004811.0231730.449162
............
32-1.7930622.209649-0.045031
829-0.2214091.6887500.069356
1760.141471-1.8802941.984397
124-0.3599521.1417582.019301
350.1713120.0793320.178522
\n", "

750 rows × 3 columns

\n", "
" ], "text/plain": [ " d e f\n", "162 1.315442 -1.039258 0.194516\n", "168 -1.908995 -0.953551 -1.430472\n", "214 0.181162 1.022858 -2.289700\n", "895 2.825765 -1.205520 1.147791\n", "154 -2.300481 1.023173 0.449162\n", ".. ... ... ...\n", "32 -1.793062 2.209649 -0.045031\n", "829 -0.221409 1.688750 0.069356\n", "176 0.141471 -1.880294 1.984397\n", "124 -0.359952 1.141758 2.019301\n", "35 0.171312 0.079332 0.178522\n", "\n", "[750 rows x 3 columns]" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import tpot\n", "import pandas as pd\n", "import numpy as np\n", "from sklearn.linear_model import LogisticRegression\n", "import sklearn\n", "\n", "subsets = [['a','b','c'],['d','e','f'],['g','h','i']]\n", "\n", "fss_search_space = tpot.search_spaces.nodes.FSSNode(subsets=subsets)\n", "\n", "selector = fss_search_space.generate(rng=1).export_pipeline()\n", "selector.set_output(transform=\"pandas\")\n", "selector.fit(X_train)\n", "selector.transform(X_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## csv file\n", "\n", "note: watch for spaces in the csv file!" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
def
1621.315442-1.0392580.194516
168-1.908995-0.953551-1.430472
2140.1811621.022858-2.289700
8952.825765-1.2055201.147791
154-2.3004811.0231730.449162
............
32-1.7930622.209649-0.045031
829-0.2214091.6887500.069356
1760.141471-1.8802941.984397
124-0.3599521.1417582.019301
350.1713120.0793320.178522
\n", "

750 rows × 3 columns

\n", "
" ], "text/plain": [ " d e f\n", "162 1.315442 -1.039258 0.194516\n", "168 -1.908995 -0.953551 -1.430472\n", "214 0.181162 1.022858 -2.289700\n", "895 2.825765 -1.205520 1.147791\n", "154 -2.300481 1.023173 0.449162\n", ".. ... ... ...\n", "32 -1.793062 2.209649 -0.045031\n", "829 -0.221409 1.688750 0.069356\n", "176 0.141471 -1.880294 1.984397\n", "124 -0.359952 1.141758 2.019301\n", "35 0.171312 0.079332 0.178522\n", "\n", "[750 rows x 3 columns]" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import tpot\n", "import pandas as pd\n", "import numpy as np\n", "from sklearn.linear_model import LogisticRegression\n", "import sklearn\n", "\n", "subsets = 'simple_fss.csv'\n", "'''\n", "# simple_fss.csv\n", "one,a,b,c\n", "two,d,e,f\n", "three,g,h,i\n", "'''\n", "\n", "fss_search_space = tpot.search_spaces.nodes.FSSNode(subsets=subsets)\n", "\n", "selector = fss_search_space.generate(rng=1).export_pipeline()\n", "selector.set_output(transform=\"pandas\")\n", "selector.fit(X_train)\n", "selector.transform(X_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "All of the above is the same when using numpy data, but the column names are replaced int indexes." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[-0.31748616 2.20805859 -2.21719911 ... 0.5595234 0.80605806\n", " 0.41484993]\n", " [ 2.8673731 1.45905176 -1.11516833 ... 0.74646156 0.95635356\n", " 0.03575697]\n", " [-1.64867116 2.14478724 2.31196119 ... 0.22969172 0.72447325\n", " 0.81842014]\n", " ...\n", " [ 1.17772695 0.7188885 -0.52548496 ... 0.99266968 0.95436462\n", " 0.57430922]\n", " [ 0.14052568 0.15042817 -0.86281564 ... 0.25379746 0.1818071\n", " 0.55993116]\n", " [ 1.37273916 -0.14898886 -0.89938251 ... 0.767549 0.66184827\n", " 0.49174333]]\n" ] } ], "source": [ "import tpot\n", "import sklearn.datasets\n", "from sklearn.linear_model import LogisticRegression\n", "import numpy as np\n", "import pandas as pd\n", "\n", "n_features = 6\n", "X, y = sklearn.datasets.make_classification(n_samples=1000, n_features=n_features, n_informative=6, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None)\n", "X = np.hstack([X, np.random.rand(X.shape[0],3)]) #add three uninformative features\n", "\n", "X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, train_size=0.75, test_size=0.25)\n", "\n", "print(X)" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[-0.76235619, -1.97629642, 1.05447979],\n", " [ 2.16944118, -1.55515714, 0.67925075],\n", " [ 1.96557199, 0.13789923, 1.588271 ],\n", " ...,\n", " [ 0.78956322, 2.12535053, 0.63115798],\n", " [-0.80184984, -0.40793866, 1.3880617 ],\n", " [-1.38085267, 1.62568989, -1.42046795]])" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import tpot\n", "import pandas as pd\n", "import numpy as np\n", "from sklearn.linear_model import LogisticRegression\n", "import sklearn\n", "\n", "subsets = { \"group_one\" : [0,1,2],\n", " \"group_two\" : [3,4,5],\n", " \"group_three\" : [6,7,8],\n", " }\n", "\n", "fss_search_space = tpot.search_spaces.nodes.FSSNode(subsets=subsets)\n", "selector = fss_search_space.generate(rng=1).export_pipeline()\n", "selector.fit(X_train)\n", "selector.transform(X_train)" ] } ], "metadata": { "kernelspec": { "display_name": "tpotenv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.16" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }