diff --git a/README.md b/README.md index 774c8dfe49..d934ae3974 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # Unity ML-Agents Toolkit -[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_1_docs/docs/) +[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_2_docs/docs/) [![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE) @@ -48,32 +48,33 @@ descriptions of all these features. ## Releases & Documentation -**Our latest, stable release is `Release 1`. Click [here](docs/Readme.md) to -get started with the latest release of ML-Agents.** - -The table below lists all our releases, including our `master` branch which is under active -development and may be unstable. A few helpful guidelines: -* The docs links in the table below include installation and usage instructions specific to each -release. Remember to always use the documentation that corresponds to the release version you're -using. -* See the [GitHub releases](https://github.com/Unity-Technologies/ml-agents/releases) for more -details of the changes between versions. -* If you have used an earlier version of the ML-Agents Toolkit, we strongly recommend our -[guide on migrating from earlier versions](docs/Migrating.md). +**Our latest, stable release is `Release 2`. Click +[here](https://github.com/Unity-Technologies/ml-agents/tree/release_2_docs/docs/Readme.md) +to get started with the latest release of ML-Agents.** + +The table below lists all our releases, including our `master` branch which is +under active development and may be unstable. A few helpful guidelines: +- The [Versioning page](docs/Versioning.md) overviews how we manage our GitHub + releases and the versioning process for each of the ML-Agents components. +- The [Releases page](https://github.com/Unity-Technologies/ml-agents/releases) + contains details of the changes between releases. +- The [Migration page](docs/Migrating.md) contains details on how to upgrade + from earlier releases of the ML-Agents Toolkit. +- The **Documentation** links in the table below include installation and usage + instructions specific to each release. Remember to always use the + documentation that corresponds to the release version you're using. | **Version** | **Release Date** | **Source** | **Documentation** | **Download** | |:-------:|:------:|:-------------:|:-------:|:------------:| | **master (unstable)** | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/master) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/master/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/master.zip) | -| **Release 1** | **April 30, 2020** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_1)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_1/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_1.zip)** | +| **Release 2** | **May 19, 2020** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_2)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_2/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_2.zip)** | +| **Release 1** | April 30, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_1) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_1.zip) | | **0.15.1** | March 30, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.15.1) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.15.1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.15.1.zip) | | **0.15.0** | March 18, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.15.0) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.15.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.15.0.zip) | | **0.14.1** | February 26, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.14.1) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.14.1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.14.1.zip) | | **0.14.0** | February 13, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.14.0) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.14.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.14.0.zip) | | **0.13.1** | January 21, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.13.1) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.13.1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.13.1.zip) | | **0.13.0** | January 8, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.13.0) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.13.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.13.0.zip) | -| **0.12.1** | December 11, 2019 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.12.1) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.12.1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.12.1.zip) | -| **0.12.0** | December 2, 2019 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.12.0) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.12.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.12.0.zip) | - ## Citation If you are a researcher interested in a discussion of Unity as an AI platform, @@ -83,14 +84,18 @@ see a pre-print of our If you use Unity or the ML-Agents Toolkit to conduct research, we ask that you cite the following paper as a reference: -Juliani, A., Berges, V., Vckay, E., Gao, Y., Henry, H., Mattar, M., Lange, D. -(2018). Unity: A General Platform for Intelligent Agents. _arXiv preprint -arXiv:1809.02627._ https://github.com/Unity-Technologies/ml-agents. +Juliani, A., Berges, V., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., +Gao, Y., Henry, H., Mattar, M., Lange, D. (2020). Unity: A General Platform for +Intelligent Agents. _arXiv preprint +[arXiv:1809.02627](https://arxiv.org/abs/1809.02627)._ +https://github.com/Unity-Technologies/ml-agents. ## Additional Resources We have published a series of blog posts that are relevant for ML-Agents: +- (May 12, 2020) + [Announcing ML-Agents Unity Package v1.0!](https://blogs.unity3d.com/2020/05/12/announcing-ml-agents-unity-package-v1-0/) - (February 28, 2020) [Training intelligent adversaries using self-play with ML-Agents](https://blogs.unity3d.com/2020/02/28/training-intelligent-adversaries-using-self-play-with-ml-agents/) - (November 11, 2019) diff --git a/com.unity.ml-agents/Runtime/Academy.cs b/com.unity.ml-agents/Runtime/Academy.cs index 846a4a1c2a..65ba3822bb 100644 --- a/com.unity.ml-agents/Runtime/Academy.cs +++ b/com.unity.ml-agents/Runtime/Academy.cs @@ -19,7 +19,7 @@ * API. For more information on each of these entities, in addition to how to * set-up a learning environment and train the behavior of characters in a * Unity scene, please browse our documentation pages on GitHub: - * https://github.com/Unity-Technologies/ml-agents/tree/release_1_docs/docs/ + * https://github.com/Unity-Technologies/ml-agents/tree/release_2_docs/docs/ */ namespace Unity.MLAgents @@ -51,7 +51,7 @@ void FixedUpdate() /// fall back to inference or heuristic decisions. (You can also set agents to always use /// inference or heuristics.) /// - [HelpURL("https://github.com/Unity-Technologies/ml-agents/tree/release_1_docs/" + + [HelpURL("https://github.com/Unity-Technologies/ml-agents/tree/release_2_docs/" + "docs/Learning-Environment-Design.md")] public class Academy : IDisposable { diff --git a/com.unity.ml-agents/Runtime/Agent.cs b/com.unity.ml-agents/Runtime/Agent.cs index b76e2cf0ab..032fa1ffd5 100644 --- a/com.unity.ml-agents/Runtime/Agent.cs +++ b/com.unity.ml-agents/Runtime/Agent.cs @@ -144,13 +144,13 @@ internal struct AgentAction /// [OnDisable()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnDisable.html] /// [OnBeforeSerialize()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnBeforeSerialize.html /// [OnAfterSerialize()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnAfterSerialize.html - /// [Agents]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md - /// [Reinforcement Learning in Unity]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design.md + /// [Agents]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md + /// [Reinforcement Learning in Unity]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design.md /// [Unity ML-Agents Toolkit]: https://github.com/Unity-Technologies/ml-agents - /// [Unity ML-Agents Toolkit manual]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Readme.md + /// [Unity ML-Agents Toolkit manual]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Readme.md /// /// - [HelpURL("https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/" + + [HelpURL("https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/" + "docs/Learning-Environment-Design-Agents.md")] [Serializable] [RequireComponent(typeof(BehaviorParameters))] @@ -598,8 +598,8 @@ public int CompletedEpisodes /// for information about mixing reward signals from curiosity and Generative Adversarial /// Imitation Learning (GAIL) with rewards supplied through this method. /// - /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md#rewards - /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals + /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md#rewards + /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals /// /// The new value of the reward. public void SetReward(float reward) @@ -628,8 +628,8 @@ public void SetReward(float reward) /// for information about mixing reward signals from curiosity and Generative Adversarial /// Imitation Learning (GAIL) with rewards supplied through this method. /// - /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md#rewards - /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals + /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md#rewards + /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals /// /// Incremental reward value. public void AddReward(float increment) @@ -781,8 +781,8 @@ public virtual void Initialize() {} /// implementing a simple heuristic function can aid in debugging agent actions and interactions /// with its environment. /// - /// [Demonstration Recorder]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md#recording-demonstrations - /// [Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Demonstration Recorder]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md#recording-demonstrations + /// [Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md#actions /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html /// /// @@ -973,7 +973,7 @@ void ResetSensors() /// For more information about observations, see [Observations and Sensors]. /// /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html - /// [Observations and Sensors]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md#observations-and-sensors + /// [Observations and Sensors]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md#observations-and-sensors /// public virtual void CollectObservations(VectorSensor sensor) { @@ -1004,7 +1004,7 @@ public ReadOnlyCollection GetObservations() /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md#actions /// /// public virtual void CollectDiscreteActionMasks(DiscreteActionMasker actionMasker) @@ -1074,7 +1074,7 @@ public virtual void CollectDiscreteActionMasks(DiscreteActionMasker actionMasker /// /// For more information about implementing agent actions see [Agents - Actions]. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md#actions /// /// /// An array containing the action vector. The length of the array is specified diff --git a/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs b/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs index e7deca57af..d75c821c86 100644 --- a/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs +++ b/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs @@ -19,7 +19,7 @@ namespace Unity.MLAgents.Demonstrations /// See [Imitation Learning - Recording Demonstrations] for more information. /// /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html - /// [Imitation Learning - Recording Demonstrations]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs//Learning-Environment-Design-Agents.md#recording-demonstrations + /// [Imitation Learning - Recording Demonstrations]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs//Learning-Environment-Design-Agents.md#recording-demonstrations /// [RequireComponent(typeof(Agent))] [AddComponentMenu("ML Agents/Demonstration Recorder", (int)MenuGroup.Default)] diff --git a/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs b/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs index 58643b2571..5fe8d5c35b 100644 --- a/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs +++ b/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs @@ -40,7 +40,7 @@ internal DiscreteActionMasker(BrainParameters brainParameters) /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md#actions /// /// The branch for which the actions will be masked. /// The indices of the masked actions. diff --git a/docs/Versioning.md b/docs/Versioning.md new file mode 100644 index 0000000000..0f1db0a062 --- /dev/null +++ b/docs/Versioning.md @@ -0,0 +1,95 @@ +# ML-Agents Versioning + +## Context +As the ML-Agents project evolves into a more mature product, we want to communicate the process +we use to version our packages and the data that flows into, through, and out of them clearly. +Our project now has four packages (1 Unity, 3 Python) along with artifacts that are produced as +well as consumed. This document covers the versioning for these packages and artifacts. + +## GitHub Releases +Up until now, all packages were in lockstep in-terms of versioning. As a result, the GitHub releases +were tagged with the version of all those packages (e.g. v0.15.0, v0.15.1) and labeled accordingly. +With the decoupling of package versions, we now need to revisit our GitHub release tagging. +The proposal is that we move towards an integer release numbering for our repo and each such +release will call out specific version upgrades of each package. For instance, with +[the April 30th release](https://github.com/Unity-Technologies/ml-agents/releases/tag/release_1), +we will have: +- GitHub Release 1 (branch name: *release_1_branch*) + - com.unity.ml-agents release 1.0.0 + - ml-agents release 0.16.0 + - ml-agents-envs release 0.16.0 + - gym-unity release 0.16.0 + +Our release cadence will not be affected by these versioning changes. We will keep having +monthly releases to fix bugs and release new features. + +## Packages +All of the software packages, and their generated artifacts will be versioned. Any automation +tools will not be versioned. + +### Unity package +Package name: com.unity.ml-agents +- Versioned following [Semantic Versioning Guidelines](https://www.semver.org) +- This package consumes an artifact of the training process: the `.nn` file. These files + are integer versioned and currently at version 2. The com.unity.ml-agents package + will need to support the version of `.nn` files which existed at its 1.0.0 release. + For example, consider that com.unity.ml-agents is at version 1.0.0 and the NN files + are at version 2. If the NN files change to version 3, the next release of + com.unity.ml-agents at version 1.1.0 guarantees it will be able to read both of these + formats. If the NN files were to change to version 4 and com.unity.ml-agents to + version 2.0.0, support for NN versions 2 and 3 could be dropped for com.unity.ml-agents + version 2.0.0. +- This package produces one artifact, the `.demo` files. These files will have integer + versioning. This means their version will increment by 1 at each change. The + com.unity.ml-agents package must be backward compatible with version changes + that occur between minor versions. +- To summarize, the artifacts produced and consumed by com.unity.ml-agents are guaranteed + to be supported for 1.x.x versions of com.unity.ml-agents. We intend to provide stability + for our users by moving to a 1.0.0 release of com.unity.ml-agents. + + +### Python Packages +Package names: ml-agents / ml-agents-envs / gym-unity +- The python packages remain in "Beta." This means that breaking changes to the public + API of the python packages can change without having to have a major version bump. + Historically, the python and C# packages were in version lockstep. This is no longer + the case. The python packages will remain in lockstep with each other for now, while the + C# package will follow its own versioning as is appropriate. However, the python package + versions may diverge in the future. +- While the python packages will remain in Beta for now, we acknowledge that the most + heavily used portion of our python interface is the `mlagents-learn` CLI and strive + to make this part of our API backward compatible. We are actively working on this and + expect to have a stable CLI in the next few weeks. + +## Communicator + +Packages which communicate: com.unity.ml-agents / ml-agents-envs + +Another entity of the ML-Agents Toolkit that requires versioning is the communication layer +between C# and Python, which will follow also semantic versioning. This guarantees a level of +backward compatibility between different versions of C# and Python packages which communicate. +Any Communicator version 1.x.x of the Unity package should be compatible with any 1.x.x +Communicator Version in Python. + +An RLCapabilities struct keeps track of which features exist. This struct is passed from C# to +Python, and another from Python to C#. With this feature level granularity, we can notify users +more specifically about feature limitations based on what's available in both C# and Python. +These notifications will be logged to the python terminal, or to the Unity Editor Console. + + +## Side Channels + +The communicator is what manages data transfer between Unity and Python for the core +training loop. Side Channels are another means of data transfer between Unity and Python. +Side Channels are not versioned, but have been designed to support backward compatibility +for what they are. As of today, we provide 4 side channels: +- FloatProperties: shared float data between Unity - Python (bidirectional) +- RawBytes: raw data that can be sent Unity - Python (bidirectional) +- EngineConfig: a set of numeric fields in a pre-defined order sent from Python to Unity +- Stats: (name, value, agg) messages sent from Unity to Python + +Aside from the specific implementations of side channels we provide (and use ourselves), +the Side Channel interface is made available for users to create their own custom side +channels. As such, we guarantee that the built in SideChannel interface between Unity and +Python is backward compatible in packages that share the same major version. + diff --git a/utils/make_readme_table.py b/utils/make_readme_table.py index 6a1d6b46ec..9a386a65ad 100644 --- a/utils/make_readme_table.py +++ b/utils/make_readme_table.py @@ -62,6 +62,7 @@ def display_name(self) -> str: ReleaseInfo.from_simple_tag("0.15.0", "March 18, 2020"), ReleaseInfo.from_simple_tag("0.15.1", "March 30, 2020"), ReleaseInfo("release_1", "1.0.0", "0.16.0", "April 30, 2020"), + ReleaseInfo("release_2", "1.0.1", "0.16.1", "May 19, 2020"), ] MAX_DAYS = 150 # do not print releases older than this many days