2018-08-25 17:51:28 -07:00
# Migrating
2018-06-15 13:55:19 -07:00
2018-10-05 14:40:04 -07:00
## Migrating from ML-Agents toolkit v0.5 to v0.6
2018-10-18 17:48:45 -07:00
2018-11-15 14:15:46 -08:00
### Important Changes
2018-10-18 17:48:45 -07:00
* Brains are now Scriptable Objects instead of MonoBehaviors.
* You can no longer modify the type of a Brain. If you want to switch
between `PlayerBrain` and `LearningBrain` for multiple agents,
you will need to assign a new Brain to each agent separately.
__Note: __ You can pass the same Brain to multiple agents in a scene by
leveraging Unity's prefab system or look for all the agents in a scene
using the search bar of the `Hierarchy` window with the word `Agent` .
2018-11-15 14:15:46 -08:00
2018-10-18 17:48:45 -07:00
* We replaced the **Internal ** and **External ** Brain with **Learning Brain ** .
When you need to train a model, you need to drag it into the `Training Hub`
inside the `Academy` and check the `Control` checkbox.
* We removed the `Broadcast` checkbox of the Brain, to use the broadcast
functionality, you need to drag the Brain into the `Broadcast Hub` .
* When training multiple Brains at the same time, each model is now stored
into a separate model file rather than in the same file under different
graph scopes.
* We have changed the way ML-Agents models perform inference. All previous `.bytes`
files can no longer be used (you will have to retrain them). The models
produced by the training process and the shipped models have now a `.tf`
extension and use TensorflowSharp as a backend for the
[Inference Engine ](Inference-Engine.md ).
* To use a `.tf` model, drag it inside the `Model` property of the `Learning Brain`
2018-11-15 14:15:46 -08:00
#### Steps to Migrate
* To update a scene from v0.5 to v0.6, you must:
* Remove the `Brain` GameObjects in the scene. (Delete all of the
Brain GameObjects under Academy in the scene.)
* Create new `Brain` Scriptable Objects using `Assets -> Create ->
ML-Agents` for each type of the Brain you plan to use, and put
the created files under a folder called Brains within your project.
* Edit their ` Brain Parameters` to be the same as the parameters used
in the ` Brain` GameObjects.
* Agents have a ` Brain` field in the Inspector, you need to drag the
appropriate Brain ScriptableObject in it.
* The Academy has a ` Broadcast Hub` field in the inspector, which is
list of brains used in the scene. To train or control your Brain
from the ` mlagents-learn` Python script, you need to drag the relevant
` LearningBrain` ScriptableObjects used in your scene into entries
into this list.
* You will need to delete the previous TensorFlowSharp package
and install the new one to do inference. To correctly delete the previous
TensorFlowSharp package, Delete all of the files under ` ML-Agents/Plugins`
folder except the files under ` ML-Agents/Plugins/ProtoBuffer`.
2018-10-18 17:48:45 -07:00
2018-10-05 14:40:04 -07:00
2018-08-28 19:08:40 -07:00
## Migrating from ML-Agents toolkit v0.4 to v0.5
### Important
2018-09-04 13:45:33 -07:00
* The Unity project ` unity-environment` has been renamed ` UnitySDK`.
2018-08-29 18:28:05 -07:00
* The ` python` folder has been renamed to ` ml-agents`. It now contains two
2018-08-28 19:08:40 -07:00
packages, ` mlagents.env` and ` mlagents.trainers`. ` mlagents.env` can be used
to interact directly with a Unity environment, while ` mlagents.trainers`
contains the classes for training agents.
2018-09-05 14:34:18 -07:00
* The supported Unity version has changed from ` 2017.1 or later` to ` 2017.4
or later`. 2017.4 is an LTS (Long Term Support) version that helps us
maintain good quality and support. Earlier versions of Unity might still work,
but you may encounter an
2018-09-07 13:54:32 -07:00
[error](FAQ.md#instance-of-corebraininternal-couldnt-be-created) listed here.
2018-08-28 19:08:40 -07:00
### Unity API
2018-09-04 21:16:20 -07:00
* Discrete Actions now use [branches](https://arxiv.org/abs/1711.08946). You can
now specify concurrent discrete actions. You will need to update the Brain
Parameters in the Brain Inspector in all your environments that use discrete
actions. Refer to the
[discrete action documentation](Learning-Environment-Design-Agents.md#discrete-action-space)
for more information.
2018-08-28 19:08:40 -07:00
### Python API
* In order to run a training session, you can now use the command
` mlagents-learn` instead of ` python3 learn.py` after installing the ` mlagents`
2018-08-29 13:36:20 -07:00
packages. This change is documented
2018-08-30 16:45:16 -07:00
[here](Training-ML-Agents.md#training-with-mlagents-learn). For example,
if we previously ran
` ``sh
python3 learn.py 3DBall --train
` ``
2018-10-10 18:32:49 -07:00
from the ` python` subdirectory (which is changed to ` ml-agents` subdirectory
in v0.5), we now run
2018-08-30 16:45:16 -07:00
` ``sh
mlagents-learn config/trainer_config.yaml --env=3DBall --train
` ``
2018-10-10 18:32:49 -07:00
from the root directory where we installed the ML-Agents Toolkit.
2018-08-30 16:45:16 -07:00
2018-08-28 19:08:40 -07:00
* It is now required to specify the path to the yaml trainer configuration file
2018-08-30 16:45:16 -07:00
when running ` mlagents-learn`. For an example trainer configuration file, see
[trainer_config.yaml](../config/trainer_config.yaml). An example of passing
a trainer configuration to ` mlagents-learn` is shown above.
2018-08-28 19:08:40 -07:00
* The environment name is now passed through the ` --env` option.
2018-09-04 17:00:11 -07:00
* Curriculum learning has been changed. Refer to the
[curriculum learning documentation](Training-Curriculum-Learning.md)
for detailed information. In summary:
* Curriculum files for the same environment must now be placed into a folder.
2018-10-18 17:48:45 -07:00
Each curriculum file should be named after the Brain whose curriculum it
2018-09-04 17:00:11 -07:00
specifies.
* ` min_lesson_length` now specifies the minimum number of episodes in a lesson
and affects reward thresholding.
* It is no longer necessary to specify the ` Max Steps` of the Academy to use
curriculum learning.
2018-08-28 19:08:40 -07:00
2018-08-25 17:51:28 -07:00
## Migrating from ML-Agents toolkit v0.3 to v0.4
2018-06-15 13:55:19 -07:00
2018-08-25 17:51:28 -07:00
### Unity API
2018-06-22 11:09:47 -07:00
2018-08-25 17:51:28 -07:00
* ` using MLAgents;` needs to be added in all of the C# scripts that use
ML-Agents.
2018-03-13 11:45:16 -07:00
2018-08-25 17:51:28 -07:00
### Python API
2018-03-13 11:45:16 -07:00
2018-09-04 21:16:20 -07:00
* We've changed some of the Python packages dependencies in requirement.txt
2018-10-10 18:32:49 -07:00
file. Make sure to run ` pip3 install -e .` within your ` ml-agents/python`
folder
2018-09-04 21:16:20 -07:00
to update your Python packages.
2018-03-13 11:45:16 -07:00
2018-08-25 17:51:28 -07:00
## Migrating from ML-Agents toolkit v0.2 to v0.3
2018-03-13 11:45:16 -07:00
2018-08-25 17:51:28 -07:00
There are a large number of new features and improvements in the ML-Agents
toolkit v0.3 which change both the training process and Unity API in ways which
will cause incompatibilities with environments made using older versions. This
page is designed to highlight those changes for users familiar with v0.1 or v0.2
in order to ensure a smooth transition.
2018-03-13 11:45:16 -07:00
2018-08-25 17:51:28 -07:00
### Important
* The ML-Agents toolkit is no longer compatible with Python 2.
### Python Training
* The training script ` ppo.py` and ` PPO.ipynb` Python notebook have been
replaced with a single ` learn.py` script as the launching point for training
with ML-Agents. For more information on using ` learn.py`, see
2018-08-26 00:05:31 -07:00
[here](Training-ML-Agents.md#training-with-mlagents-learn).
2018-09-04 21:16:20 -07:00
* Hyperparameters for training Brains are now stored in the
2018-08-25 17:51:28 -07:00
` trainer_config.yaml` file. For more information on using this file, see
2018-08-26 00:05:31 -07:00
[here](Training-ML-Agents.md#training-config-file).
2018-08-25 17:51:28 -07:00
### Unity API
* Modifications to an Agent's rewards must now be done using either
` AddReward()` or ` SetReward()`.
* Setting an Agent to done now requires the use of the ` Done()` method.
* ` CollectStates()` has been replaced by ` CollectObservations()`, which now no
longer returns a list of floats.
* To collect observations, call ` AddVectorObs()` within ` CollectObservations()`.
Note that you can call ` AddVectorObs()` with floats, integers, lists and
arrays of floats, Vector3 and Quaternions.
* ` AgentStep()` has been replaced by ` AgentAction()`.
* ` WaitTime()` has been removed.
* The ` Frame Skip` field of the Academy is replaced by the Agent's ` Decision
2018-09-04 21:16:20 -07:00
Frequency` field, enabling the Agent to make decisions at different frequencies.
2018-08-25 17:51:28 -07:00
* The names of the inputs in the Internal Brain have been changed. You must
replace ` state` with ` vector_observation` and ` observation` with
` visual_observation`. In addition, you must remove the ` epsilon` placeholder.
### Semantics
In order to more closely align with the terminology used in the Reinforcement
Learning field, and to be more descriptive, we have changed the names of some of
the concepts used in ML-Agents. The changes are highlighted in the table below.
2018-03-13 11:45:16 -07:00
| Old - v0.2 and earlier | New - v0.3 and later |
| --- | --- |
| State | Vector Observation |
| Observation | Visual Observation |
| Action | Vector Action |
| N/A | Text Observation |
| N/A | Text Action |
