Python Dataflow for Group Manager#4926

Merged

ervteng merged 212 commits intomainfrom

develop-agentprocessor-teammanager

Mar 4, 2021

Commits on Nov 14, 2020

Make buffer type-agnostic
Ervin Teng
committed
Edit types of Apped method
Ervin Teng
committed
Change comment
Ervin Teng
committed

Commits on Nov 17, 2020

Collaborative walljump
Ervin Teng
committed
Make collab env harder
Ervin Teng
committed

Commits on Nov 20, 2020

Merge branch 'develop-stack-walljump' into develop-centralizedcritic
Ervin Teng
committed
Add group ID
Ervin Teng
committed
Add collab obs to trajectory
Ervin Teng
committed

Commits on Nov 21, 2020

Merge branch 'develop-multitype-buffer' into develop-centralizedcritic
Ervin Teng
committed
Fix bug; add critic_obs to buffer
Ervin Teng
committed
Set group ids for some envs
Ervin Teng
committed
Merge branch 'develop-multitype-buffer' into develop-unified-obs
Ervin Teng
committed
Pretty broken
Ervin Teng
committed

Commits on Dec 1, 2020

Less broken PPO
Ervin Teng
committed
Update SAC, fix PPO batching
Ervin Teng
committed
Fix SAC interrupted condition and typing
Ervin Teng
committed
Fix SAC interrupted again
Ervin Teng
committed
Remove erroneous file
Ervin Teng
committed

Commits on Dec 2, 2020

Fix multiple obs
Ervin Teng
committed
Update curiosity reward provider
Ervin Teng
committed

Commits on Dec 3, 2020

Update GAIL and BC
Ervin Teng
committed
Merge branch 'develop-unified-obs' into develop-centralizedcritic
Ervin Teng
committed
Multi-input network
Ervin Teng
committed
Some minor tweaks but still broken
Ervin Teng
committed

Commits on Dec 9, 2020

Get next critic observations into value estimate
Ervin Teng
committed

Commits on Dec 10, 2020

Temporarily disable exporting
Ervin Teng
committed

Commits on Dec 11, 2020

Use Vince's ONNX export code
Ervin Teng
committed
Cleanup
Ervin Teng
committed

Commits on Dec 14, 2020

Add walljump collab YAML
Ervin Teng
committed
Lower max height
Ervin Teng
committed
Merge branch 'master' into develop-centralizedcritic
Ervin Teng
committed
Update prefab
Ervin Teng
committed
Update prefab
Ervin Teng
committed
Collaborative Hallway
Ervin Teng
committed
Set num teammates to 2
Ervin Teng
committed
Add config and group ids to HallwayCollab
Ervin Teng
committed

Commits on Dec 15, 2020

Fix bug with hallway collab
Ervin Teng
committed
Edits to HallwayCollab
Ervin Teng
committed
Update onnx file meta
Ervin Teng
committed
Make the env easier
Ervin Teng
committed
Remove prints
Ervin Teng
committed

Commits on Dec 17, 2020

Make Collab env harder
Ervin Teng
committed

Commits on Dec 18, 2020

Fix group ID
Ervin Teng
committed
Add cc to ghost trainer
Ervin Teng
committed
Add comment to ghost trainer
Ervin Teng
committed
Revert "Add comment to ghost trainer"
Ervin Teng
committed
Actually add comment to ghosttrainer
Ervin Teng
committed

Commits on Dec 21, 2020

Scale size of CC network
Ervin Teng
committed
Scale value network based on num agents
Ervin Teng
committed
Add 3rd symbol to hallway collab
Ervin Teng
committed
Make comms one-hot
Ervin Teng
committed

Commits on Dec 23, 2020

Fix S tag
Ervin Teng
committed

Commits on Jan 4, 2021

Merge branch 'master' into develop-centralizedcritic-mm
Ervin Teng
committed
Additional changes
Ervin Teng
committed
Some more fixes
Ervin Teng
committed

Commits on Jan 6, 2021

Self-attention Centralized Critic
Ervin Teng
committed

Commits on Jan 11, 2021

separate entity encoder and RSA
andrewcoh
committed
clean up args in mha
andrewcoh
committed
more cleanups
andrewcoh
committed
fixed tests
andrewcoh
committed
Merge branch 'develop-attention-refactor' into develop-centralizedcritic-mm
Ervin Teng
committed
Merge branch 'develop-attention-refactor' into develop-centralizedcritic-mm
Ervin Teng
committed
entity embeddings work with no max
Ervin Teng
committed
remove group id
Ruo-Ping Dong
committed
very rough sketch for TeamManager interface

Chris Elion
authored and
Ruo-Ping Dong
committed

Commits on Jan 12, 2021

One layer for entity embed
Ervin Teng
committed
Use 4 heads
Ervin Teng
committed
add defaults to linear encoder, initialize ent encoders
andrewcoh
committed
Merge branch 'master' into develop-centralizedcritic-mm
Ervin Teng
committed
Merge branch 'develop-lin-enc-def' into develop-centralizedcritic-mm
Ervin Teng
committed
add team manager id to proto
Ruo-Ping Dong
committed
team manager for hallway
Ruo-Ping Dong
committed
add manager to hallway
Ruo-Ping Dong
committed
send and process team manager id
Ruo-Ping Dong
committed
remove print
Ruo-Ping Dong
committed
Merge branch 'develop-centralizedcritic-mm' into develop-cc-teammanager
Ruo-Ping Dong
committed

Commits on Jan 13, 2021

small cleanup
Ruo-Ping Dong
committed
default behavior for baseTeamManager
Ruo-Ping Dong
committed
add back statsrecorder
Ruo-Ping Dong
committed
update
Ruo-Ping Dong
committed
Team manager prototype (#4850)

Ruo-Ping Dong
and
Chris Elion
authored
Remove statsrecorder
Ervin Teng
committed
Fix AgentProcessor for TeamManager
Ervin Teng
committed
Merge branch 'develop-centralizedcritic-mm' into develop-cc-teammanager
Ruo-Ping Dong
committed
team manager
Ruo-Ping Dong
committed

Commits on Jan 14, 2021

New buffer layout, TeamObsUtil, pad dead agents
Ervin Teng
committed
Use NaNs to get masks for attention
Ervin Teng
committed

Commits on Jan 15, 2021

Add team reward to buffer
Ervin Teng
committed
Try subtract marginalized value
Ervin Teng
committed

Commits on Jan 20, 2021

Add Q function with attention
Ervin Teng
committed
Some more progress - still broken
Ervin Teng
committed
use singular entity embedding (#4873)
andrewcoh
authored
I think it's running
Ervin Teng
committed

Commits on Jan 21, 2021

Actions added but untested
Ervin Teng
committed

Commits on Jan 22, 2021

Commits on Jan 23, 2021

Commits on Jan 26, 2021

Commits on Jan 27, 2021

Commits on Jan 29, 2021

Add trust region to COMA updates
Ervin Teng
committed
Remove Q-net for perf
Ervin Teng
committed
Weight decay, regularizaton loss
Ervin Teng
committed
Use same network
Ervin Teng
committed

Commits on Feb 1, 2021

add base team manager
Ruo-Ping Dong
committed

Commits on Feb 4, 2021

Remove reg loss, still stable
Ervin Teng
committed
Black format
Ervin Teng
committed

Commits on Feb 5, 2021

add team reward field to agent and proto
Ruo-Ping Dong
committed
set team reward
Ruo-Ping Dong
committed
add maxstep to teammanager and hook to academy
Ruo-Ping Dong
committed

Commits on Feb 8, 2021

check agent by agent.enabled
Ruo-Ping Dong
committed

Commits on Feb 9, 2021

remove manager from academy when dispose
Ruo-Ping Dong
committed
move manager
Ruo-Ping Dong
committed
put team reward in decision steps
Ruo-Ping Dong
committed
use 0 as default manager id
Ruo-Ping Dong
committed
fix setTeamReward

Ruo-Ping Dong
and
vincentpierre
authored
change method name to GetRegisteredAgents
Ruo-Ping Dong
committed
address comments
Ruo-Ping Dong
committed
Merge branch 'develop-base-teammanager' into develop-agentprocessor-teammanager
Ervin Teng
committed
Merge branch 'develop-base-teammanager' into develop-agentprocessor-teammanager
Ervin Teng
committed
Revert C# env changes
Ervin Teng
committed
Remove a bunch of stuff from envs
Ervin Teng
committed
Remove a bunch of extra files
Ervin Teng
committed
Remove changes from base-teammanager
Ervin Teng
committed
Remove remaining files
Ervin Teng
committed
Remove some unneeded changes
Ervin Teng
committed
Make buffer typing neater
Ervin Teng
committed
AgentProcessor fixes
Ervin Teng
committed
Back out trainer changes
Ervin Teng
committed
use delegate to avoid agent-manager cyclic reference
Ruo-Ping Dong
committed
put team reward in decision steps
Ruo-Ping Dong
committed

Commits on Feb 10, 2021

fix unregister agents
Ruo-Ping Dong
committed
add teamreward to decision step
Ruo-Ping Dong
committed
typo
Ruo-Ping Dong
committed
unregister on disabled
Ruo-Ping Dong
committed
remove OnTeamEpisodeBegin
Ruo-Ping Dong
committed

Commits on Feb 11, 2021

change name TeamManager to MultiAgentGroup
Ruo-Ping Dong
committed
more team -> group
Ruo-Ping Dong
committed
fix tests
Ruo-Ping Dong
committed
fix tests
Ruo-Ping Dong
committed
Merge remote-tracking branch 'origin/develop-base-teammanager' into develop-agentprocessor-teammanager
Ervin Teng
committed
Use attention tests from master
Ervin Teng
committed
Revert "Use attention tests from master"
Ervin Teng
committed
Use attention from master
Ervin Teng
committed
Renaming fest
Ervin Teng
committed
Use NamedTuples instead of attrs classes
Ervin Teng
committed
Bug fixes
Ervin Teng
committed

Commits on Feb 12, 2021

remove GroupMaxStep
Ruo-Ping Dong
committed
add some doc
Ruo-Ping Dong
committed
Fix mock brain
Ervin Teng
committed
np float32 fixes
Ervin Teng
committed
more renaming
Ervin Teng
committed
Test for team obs in agentprocessor
Ervin Teng
committed
Test for group and add team reward
Ervin Teng
committed
doc improve

Ruo-Ping Dong
and
Ervin T.
authored

Commits on Feb 13, 2021

Merge branch 'master' into develop-base-teammanager
Ruo-Ping Dong
committed
Merge branch 'master' into develop-base-teammanager
Ruo-Ping Dong
committed
Merge branch 'develop-base-teammanager' of https://github.com/Unity-Technologies/ml-agents into develop-base-teammanager
Ruo-Ping Dong
committed

Commits on Feb 16, 2021

store registered agents in set
Ruo-Ping Dong
committed

Commits on Feb 17, 2021

remove unused step counts
Ruo-Ping Dong
committed
Merge branch 'develop-base-teammanager' into develop-agentprocessor-teammanager
Ervin Teng
committed
Global group ids
Ervin Teng
committed

Commits on Feb 19, 2021

Fix Trajectory test
Ervin Teng
committed

Commits on Feb 23, 2021

Merge branch 'master' into develop-agentprocessor-teammanager
Ervin Teng
committed
Remove duplicated files
Ervin Teng
committed
Add team methods to AgentAction
Ervin Teng
committed

Commits on Feb 24, 2021

Buffer fixes
Ervin Teng
committed
Add test for GroupObs
Ervin Teng
committed
Change AgentAction back to 0 pad and add tests
Ervin Teng
committed
Addressed some comments
Ervin Teng
committed

Commits on Feb 25, 2021

Address some comments
Ervin Teng
committed
Add more comments
Ervin Teng
committed
Rename internal function
Ervin Teng
committed
Move padding method to AgentBufferField
Ervin Teng
committed
Merge branch 'main' into develop-agentprocessor-teammanager
Ervin Teng
committed
Fix slicing typing and string printing in AgentBufferField
Ervin Teng
committed
Fix to-flat and add tests
Ervin Teng
committed

Commits on Mar 1, 2021

Rename GroupmateStatus to AgentStatus
Ervin Teng
committed

Commits on Mar 4, 2021

Update comments
Ervin Teng
committed
Added GroupId, GlobalGroupId, GlobalAgentId types
Ervin Teng
committed
Update comment
Ervin Teng
committed
Make some agent processor properties internal
Ervin Teng
committed
Rename add_group_status
Ervin Teng
committed
Rename store_group_status, fix test
Ervin Teng
committed
Rename clear_group_obs
Ervin Teng
committed