GAIL and Pretraining#2118

Merged

ervteng merged 146 commits intodevelopfrom

develop-irl-ervin

Jul 16, 2019

Commits on Apr 24, 2019

New version of GAIL

awjuliani
authored and
Ervin Teng
committed
Move Curiosity to separate class

awjuliani
authored and
Ervin Teng
committed
Curiosity fully working under new system

awjuliani
authored and
Ervin Teng
committed
Begin implementing GAIL

awjuliani
authored and
Ervin Teng
committed
fix discrete curiosity

vincentpierre
authored and
Ervin Teng
committed
Add expert demonstration

awjuliani
authored and
Ervin Teng
committed
Remove notebook

awjuliani
authored and
Ervin Teng
committed
Record intrinsic rewards properly

awjuliani
authored and
Ervin Teng
committed
Add gail model updating

awjuliani
authored and
Ervin Teng
committed
Code cleanup

awjuliani
authored and
Ervin Teng
committed
Nested structure for intrinsic rewards

awjuliani
authored and
Ervin Teng
committed
Rename files

awjuliani
authored and
Ervin Teng
committed
Update models so files

awjuliani
authored and
Ervin Teng
committed
fix typo

awjuliani
authored and
Ervin Teng
committed
Add reward strength parameter

awjuliani
authored and
Ervin Teng
committed
Use dictionary of reward signals

awjuliani
authored and
Ervin Teng
committed
Remove reward manager

awjuliani
authored and
Ervin Teng
committed
Extrinsic reward just another type

awjuliani
authored and
Ervin Teng
committed
Clean up imports

awjuliani
authored and
Ervin Teng
committed
All reward signals use strength to scale output

awjuliani
authored and
Ervin Teng
committed
produce scaled and unscaled reward

awjuliani
authored and
Ervin Teng
committed
Remove unused dictionary

awjuliani
authored and
Ervin Teng
committed
Current trainer config

awjuliani
authored and
Ervin Teng
committed
Add discrete control and pyramid experimentation

awjuliani
authored and
Ervin Teng
committed
Minor changes to GAIL

awjuliani
authored and
Ervin Teng
committed
Add relevant strength parameters

awjuliani
authored and
Ervin Teng
committed
Replace string

awjuliani
authored and
Ervin Teng
committed
Add support for visual observations w/ GAIL

awjuliani
authored and
Ervin Teng
committed
Finish implementing visual obs for GAIL

awjuliani
authored and
Ervin Teng
committed
Include demo files

awjuliani
authored and
Ervin Teng
committed
Fix for RNN w/ GAIL

awjuliani
authored and
Ervin Teng
committed
Keep track of reward streams separately

awjuliani
authored and
Ervin Teng
committed
Bootstrap value estimates separately

awjuliani
authored and
Ervin Teng
committed
Add value head

awjuliani
authored and
Ervin Teng
committed
Use sepaprate value streams for each reward

awjuliani
authored and
Ervin Teng
committed
Add VAIL

awjuliani
authored and
Ervin Teng
committed
Use adaptive B

awjuliani
authored and
Ervin Teng
committed
Comments improvements

vincentpierre
authored and
Ervin Teng
committed
Added comments and refactored a pievce of the code

vincentpierre
authored and
Ervin Teng
committed
Added Comments

vincentpierre
authored and
Ervin Teng
committed
Fix on Curriosity

vincentpierre
authored and
Ervin Teng
committed
Fixed typo

vincentpierre
authored and
Ervin Teng
committed
Added a forgotten comment

vincentpierre
authored and
Ervin Teng
committed
Stabilized Vail learning. Still no learning for Walker

vincentpierre
authored and
Ervin Teng
committed
Fixing typo on curiosity when using visual input

vincentpierre
authored and
Ervin Teng
committed
Added some comments

vincentpierre
authored and
Ervin Teng
committed
modified the hyperparameters

vincentpierre
authored and
Ervin Teng
committed
Fixed some of the tests, will need to refactor the reward signals in order to test them all appropriately

vincentpierre
authored and
Ervin Teng
committed
Putting the has_updated fags inside each reward signal

vincentpierre
authored and
Ervin Teng
committed
Added comments for the GAIL update method

vincentpierre
authored and
Ervin Teng
committed
initial commit

vincentpierre
authored and
Ervin Teng
committed
No more normalization after pre-training

vincentpierre
authored and
Ervin Teng
committed
Fixed large bug in Vail

vincentpierre
authored and
Ervin Teng
committed
BUG FIX VAIL : The noise dimension was wrong and the discriminator score was wrong

vincentpierre
authored and
Ervin Teng
committed
implemented discrete control pretraining

vincentpierre
authored and
Ervin Teng
committed
bug fixing

vincentpierre
authored and
Ervin Teng
committed
Bug fix, still not tested for recurrent

vincentpierre
authored and
Ervin Teng
committed
Fixing beta in GAIL so it will change properly

vincentpierre
authored and
Ervin Teng
committed
Allow for not specifying an extrinsic reward
Ervin Teng
committed
Rough implementation of annealed BC
Ervin Teng
committed
Fixes for rebase onto v0.8
Ervin Teng
committed

Commits on Apr 25, 2019

Moved BC trainer out of reward_signals and code cleanup
Ervin Teng
committed
Rename folder to "components"
Ervin Teng
committed
Fix renaming in Curiosity
Ervin Teng
committed

Commits on May 2, 2019

Remove demo_aided as a required param
Ervin Teng
committed
Make old BC compatible
Ervin Teng
committed

Commits on May 3, 2019

Fix visual obs for curiosity
Ervin Teng
committed

Commits on May 9, 2019

Tweaks all around
Ervin Teng
committed
Add reward normalization and bug fix
Ervin Teng
committed

Commits on May 30, 2019

Load multiple .demo files. Fix bug with csv nans
Ervin Teng
committed
Remove reward normalization
Ervin Teng
committed
Rename demo_aided to pretraining
Ervin Teng
committed
Fix bc configs
Ervin Teng
committed
Increase small val to prevent NaNs
Ervin Teng
committed

Commits on May 31, 2019

Fix init in components
Ervin Teng
committed
Merge remote-tracking branch 'origin/develop' into develop-irl-ervin
Ervin Teng
committed
Fix PPO tests
Ervin Teng
committed
Refactor components into common location
Ervin Teng
committed

Commits on Jun 3, 2019

Minor code cleanup
Ervin Teng
committed

Commits on Jun 5, 2019

Preliminary RNN support
Ervin Teng
committed

Commits on Jun 6, 2019

Revert regression with NaNs for LSTMs
Ervin Teng
committed
Better LSTM support for BC
Ervin Teng
committed
Code cleanup and black reformat
Ervin Teng
committed
Remove demo_helper and reformat signal
Ervin Teng
committed
Tests for GAIL and curiosity
Ervin Teng
committed
Fix Black again...
Ervin Teng
committed
Tests for BCModule and visual tests for RewardSignals
Ervin Teng
committed

Commits on Jun 7, 2019

Refactor to new structure and use class generator
Ervin Teng
committed

Commits on Jun 8, 2019

Generalize reward_signal interface and stats
Ervin Teng
committed

Commits on Jun 10, 2019

Fix incorrect environment reward reporting
Ervin Teng
committed
Rename reward signals for consistency. clean up comments
Ervin Teng
committed
Default trainer config (for cloud testing)
Ervin Teng
committed
Remove "curiosity_enc_size" from the regular params
Ervin Teng
committed
Fix PushBlock config
Ervin Teng
committed
Revert Pyramids environment
Ervin Teng
committed

Commits on Jun 11, 2019

Fix indexing issue with add_experiences
Ervin Teng
committed
Fix tests
Ervin Teng
committed
Change to BCModule
Ervin Teng
committed

Commits on Jun 12, 2019

Merge branch 'develop' into develop-irl-ervin
Ervin Teng
committed
Remove the bools for reward signals
Ervin Teng
committed

Commits on Jun 13, 2019

Make update take in a mini buffer rather than the
Ervin Teng
committed
Always reference reward signals name and not index
Ervin Teng
committed
More code cleanup
Ervin Teng
committed
Clean up reward_signal abstract class
Ervin Teng
committed
Fix issue with recording values
Ervin Teng
committed

Commits on Jun 17, 2019

Add use_actions to GAIL
Ervin Teng
committed
Add documentation for Reward Signals
Ervin Teng
committed
Add documentation for GAIL
Ervin Teng
committed
Remove unused variables in BCModel
Ervin Teng
committed
Remove Entropy Reward Signal
Ervin Teng
committed
Change tests to use safe_load
Ervin Teng
committed
Don't use mutable default
Ervin Teng
committed
Set defaults in parent __init__ (Reward Signals)
Ervin Teng
committed
Remove unneccesary lines
Ervin Teng
committed

Commits on Jul 3, 2019

Merge branch 'develop' into develop-irl-ervin
Ervin Teng
committed
Make some files same as develop
Ervin Teng
committed

Commits on Jul 4, 2019

Add demos for example envs
Ervin Teng
committed
Update docs
Ervin Teng
committed

Commits on Jul 8, 2019

Fix tests, imports, cleanup code
Ervin Teng
committed

Commits on Jul 9, 2019

Make pretrainer stats similar to reward signal
Ervin Teng
committed
Merge branch 'develop' of github.com:Unity-Technologies/ml-agents into develop-irl-ervin
Ervin Teng
committed

Commits on Jul 10, 2019

Fixes after merge develop
Ervin Teng
committed
Additional tests, bugfix for LSTM+BC+Visual
Ervin Teng
committed
GAIL code cleanup
Ervin Teng
committed
Add types to BCModel
Ervin Teng
committed

Commits on Jul 11, 2019

Fix bugs with incorrect return values
Ervin Teng
committed
Change tests to use RewardSignalResult
Ervin Teng
committed
Add docs for pretraining and plot for all three
Ervin Teng
committed
Fix bug with demo loading directories, add test
Ervin Teng
committed
Add typing to BCModule, GAIL, and demo loader
Ervin Teng
committed
Fix black
Ervin Teng
committed
Fix mypy issues
Ervin Teng
committed

Commits on Jul 12, 2019

Codacy cleanup
Ervin Teng
committed
Doc fixes
Ervin Teng
committed

Commits on Jul 13, 2019

More sophisticated tests for reward signals
Ervin Teng
committed
Fix bug in GAIL when num_sequences is 1
Ervin Teng
committed

Commits on Jul 15, 2019

Clean up use_vail and feed_dicts
Ervin Teng
committed
Change to swish from learningmodel
Ervin Teng
committed
Make variables more readable
Ervin Teng
committed
Code and comment cleanup
Ervin Teng
committed
Not all should be swish
Ervin Teng
committed
Remove prints
Ervin Teng
committed
Doc updates
Ervin Teng
committed
Make VAIL default false, improve logging
Ervin Teng
committed

Commits on Jul 16, 2019

Fix tests for sequences
Ervin Teng
committed
Change max_batches and set VAIL to default to false
Ervin Teng
committed