Unity not passing observations to python side when run on server

[samby707]

I have created a linux build for my project in unity and I'm using unity ml agents. I have also written a python code for training the agent. When I'm running it on Ubuntu 16.04 server using batchmode and nographics as command line arguments written in environmnet.py, then only,it's running on the server,otherwise not. But the observations that I'm getting are blank images with constant values of 0.803, instead of the images that are captured from the camera and passed as observations. Any help to solve the problem would be appreciated. I have recorded the observations that are being captured from the python side and logged them and they are blank images. Is it because the unity window isn't getting rendered and hence, the observations are not getting passed?

[vincentpierre]

Hey, maybe this will help :
https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-on-Amazon-Web-Service.md
I do not know what could cause this problem but I think it might be because of the batchmode and nographics arguments you passed in.

[MarcoMeter]

Hi @samby707
I'd check the texture which is getting rendered by the camera in the Unity game instance. It shouldn't matter that the camera is not rendering to a window.

You could also try to make a headless build and not use the nographics argument.

@vincentpierre
Headless mode for linux builds is available since the 2017.3 release. That's likely outdated on the docs. However, I didn't try a training with image input, because I'm building a JupyterHub service, which provides docker containers with an ml-agents cuda environment.

[vincentpierre]

@MarcoMeter Hey, we would be interested in simplifying the use of ML-Agents, would it be possible to get some more info on this container you made ?

[MarcoMeter]

@vincentpierre I sent you a contact request on linkedin. This issue is probably not the right place. Also, I'd like to ask something else which doesn't suit an issue.

[samby707]

When I am trying to run the linux build of the unity simulation on server,its not running. When I am running it with batchmode and no graphics argument, then only its running, otherwise not. I'll try with the headless build. The amazon web services link you mentioned, helped a bit but on removing lib xrandr2 as mentioned there, I can't even log in to my Ubuntu 16.04. I choose the kernel then the ubuntu screen shows up but it doesn't open the login screen.

[samby707]

I could make it work.Thanks for your help.

[MarcoMeter]

Hi @samby707 ,
that's great to hear. Could you provide a few details on how you solved your issue? This would be really helpful.

[samby707]

I installed xserver and nvidiaxconfig as provided in that link. Then i created a virtual display and exported it. I didn't remove those 2 libraries that was mentioned in the link. I didn't do headless build also.It was a normal unity linux build. I just removed batchmode and nographics in environment.py which provides the launch string. I ran glxgears and it was working. Then I started training and it worked. But seems like if multiple persons are accessing the server and more than 1 person accesses the same virtual display, the observations get corrupted.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.