[change] Separate action outputs into OutputDistributions object#3514

Merged

ervteng merged 119 commits intomasterfrom

develop-distributions

Feb 28, 2020

Commits on Jan 17, 2020

Move some functionality to optimizer
Ervin Teng
committed
Move some functionality to optimizer-black
Ervin Teng
committed
Move some functionality to optimizer-black
Ervin Teng
committed

Commits on Jan 22, 2020

More incremental steps to separation
Ervin Teng
committed

Commits on Jan 23, 2020

More progress
Ervin Teng
committed
Move methods into common optimizer
Ervin Teng
committed
Some more bugfixes
Ervin Teng
committed
Working continuous updates
Ervin Teng
committed

Commits on Jan 24, 2020

Discrete PPO working
Ervin Teng
committed
Clean up policy files
Ervin Teng
committed
Combined model and policy for PPO
Ervin Teng
committed
Commit init file
Ervin Teng
committed
Remove PPO model
Ervin Teng
committed
Simplify creation of optimizer, breaks multi-GPU
Ervin Teng
committed

Commits on Jan 25, 2020

Move optimizer creation to Trainer, fix some of the reward signals
Ervin Teng
committed
Change reward signal creation
Ervin Teng
committed

Commits on Jan 27, 2020

Move policy to common location, remove epsilon
Ervin Teng
committed
Fix some typing issues with curiosity
Ervin Teng
committed
Unified policy
Ervin Teng
committed

Commits on Jan 29, 2020

SAC CC working
Ervin Teng
committed
Merge branch 'master' into develop-splitpolicyoptimizer
Ervin Teng
committed
Clean up value head creation
Ervin Teng
committed
Use resamp policy for SAC
Ervin Teng
committed
Add typing to value head creator
Ervin Teng
committed
Fix discrete SAC and clean up policy
Ervin Teng
committed
Remove epsilon from everywhere
Ervin Teng
committed
Add some typing to optimizer
Ervin Teng
committed

Commits on Jan 30, 2020

Zeroed version of LSTM working for PPO
Ervin Teng
committed

Commits on Jan 31, 2020

Revert learn.py
Ervin Teng
committed
Cleanup LSTM code
Ervin Teng
committed
SAC LSTM isn't broken
Ervin Teng
committed
Reduce memory sizes
Ervin Teng
committed
Fix Hallway summary freq
Ervin Teng
committed

Commits on Feb 3, 2020

Remove __init__ for LearningModel static class
Ervin Teng
committed
Add burn-in for memory PPO
Ervin Teng
committed
Fix comments for PPO
Ervin Teng
committed
Fix BC and tests
Ervin Teng
committed
Fix for visual obs
Ervin Teng
committed

Commits on Feb 4, 2020

Move initialization call around
Ervin Teng
committed
Fix SAC CC and some reward signal tests
Ervin Teng
committed

Commits on Feb 5, 2020

Fix policy tests
Ervin Teng
committed
Fix SAC scoping
Ervin Teng
committed
Separate out optimizer creation and policy graph creation (#3355)
Ervin T
authored
Fix policy's scoping
Ervin Teng
committed
Re-fix scoping and add method to get all variables
Ervin Teng
committed
Fix discrete scoping
Ervin Teng
committed
Add test for NN policy
Ervin Teng
committed
Remove PPO policy tests
Ervin Teng
committed
remove more PPO tests
Ervin Teng
committed

Commits on Feb 6, 2020

Fix ghost trainer and all tests
Ervin Teng
committed
Reformat using black
Ervin Teng
committed
Fix graph init in ghost trainer
Ervin Teng
committed
Remove float64 numpy
Ervin Teng
committed
Clean up some SAC LSTM
Ervin Teng
committed
Merge commit 'f9c05a61d574305497789b5997f1ae3ea1b1ad3b' into develop-splitpolicyoptimizer
Ervin Teng
committed
Fix Barracuda export for LSTM
Ervin Teng
committed
Remove burn-in
Ervin Teng
committed

Commits on Feb 7, 2020

Move check for creation into nn_policy
Ervin Teng
committed
Fix SAC RNN test
Ervin Teng
committed
Add optional burn-in for SAC as well
Ervin Teng
committed
Change memory size definition in docs
Ervin Teng
committed
add init
Ervin Teng
committed
Split optimizer and TFOptimizer
Ervin Teng
committed

Commits on Feb 8, 2020

Move optimizer to common folder
Ervin Teng
committed
Remove extra tf_optimizer
Ervin Teng
committed

Commits on Feb 11, 2020

Add docstring and make some methods private
Ervin Teng
committed
Delete extra tf_optimizer
Ervin Teng
committed
Temporarily remove multi-GPU
Ervin Teng
committed
Fix one more test for multi_gpu
Ervin Teng
committed
Remove multi_gpu from learn test
Ervin Teng
committed
Added enforcement of m_size to be divisible by 2
Ervin Teng
committed
Update docstring
Ervin Teng
committed
Add typing to optimizer, rename self.tf_optimizer
Ervin Teng
committed
Removed floating constants
Ervin Teng
committed
Used NamedTuple for create normalization tensors
Ervin Teng
committed

Commits on Feb 12, 2020

Clean up nn_policy
Ervin Teng
committed
Correct memory size docs
Ervin Teng
committed

Commits on Feb 13, 2020

Add CC distribution
Ervin Teng
committed
Tweak probs
Ervin Teng
committed
Fix SAC tanh
Ervin Teng
committed
Still broken...
Ervin Teng
committed
Fix tanh squashed distribution
Ervin Teng
committed
Build only one squashing function
Ervin Teng
committed
Make only one probability tensor
Ervin Teng
committed

Commits on Feb 14, 2020

Split up log prob and total log prob
Ervin Teng
committed
Remove action_holder placeholder
Ervin Teng
committed
Merge branch 'master' into develop-splitpolicyoptimizer
Ervin Teng
committed

Commits on Feb 19, 2020

Merge branch 'master' into develop-splitpolicyoptimizer
Ervin Teng
committed
Fix entropy calculation
Ervin Teng
committed

Commits on Feb 20, 2020

Add option to not condition sigma on obs
Ervin Teng
committed
Merge commit 'fbcdd83c087135f870e785cc72e5ff9a7e898e3a' into develop-splitpolicyoptimizer
Ervin Teng
committed
Fix tensor names
Ervin Teng
committed
Remove and rename tf_optimizer
Ervin Teng
committed
Fix PPO optimizer creation
Ervin Teng
committed
Make TF graph seed deterministic
Ervin Teng
committed
extend meta curriculum test steps
Ervin Teng
committed
Move learning rate reporting
Ervin Teng
committed

Commits on Feb 21, 2020

Add add_policy docstrings
Ervin Teng
committed
Add docstrings for model
Ervin Teng
committed
Make create critic methods private
Ervin Teng
committed
Make create losses private
Ervin Teng
committed
Fix docs for consistency
Ervin Teng
committed
Rename LearningModel to ModelUtils
Ervin Teng
committed
Merge branch 'develop-splitpolicyoptimizer' into develop-removeactionholder
Ervin Teng
committed

Commits on Feb 24, 2020

Merge branch 'develop-removeactionholder' into develop-distributions
Ervin Teng
committed
(nonworking) discrete distribution
Ervin Teng
committed
Fix discrete distribution
Ervin Teng
committed

Commits on Feb 27, 2020

Merge branch 'master' into develop-distributions
Ervin Teng
committed
Fix gradient passing for BC
Ervin Teng
committed
Reorder BC module tests
Ervin Teng
committed
Add tests for distributions
Ervin Teng
committed
Fix shape log probs in gaussian policy
Ervin Teng
committed
Fix SAC test
Ervin Teng
committed

Commits on Feb 28, 2020

Rename total_log_probs to avoid confusion with log_probs
Ervin Teng
committed
Revert "Rename total_log_probs to avoid confusion with log_probs"
Ervin Teng
committed
Revert "Revert "Rename total_log_probs to avoid confusion with log_probs""
Ervin Teng
committed
Combine tanh distribution with Gaussian
Ervin Teng
committed
Extend SAC test
Ervin Teng
committed
Fix SAC test again...
Ervin Teng
committed