Add support for embedding the game window in the editor

[reduz]

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

Several users who want to switch to Godot (or switched to Godot already) miss the functionality provided by engines such as Unity for "exploring" the running game.

While Godot can show you all the running game contents in the remote scene tree and its sort of possible to override the remote camera, it still leaves a lot to be desired.

Engines like Unity or Unreal let you run the game within the editor window. They achieve this by running inside the same editor process, which lets you explore the game while you run it.

This approach has the clear advantage of allow you to explore the running game more intuitively, but it also has several disadvantages:

• Non-destructive editing of the running game is impossible or in a very limited fashion.
• The game has to be saved, the cleared before playing.
• The game has to be reloaded after playing.
• The editor stability can be easily compromised if the running game crashes.

None of these are problems with Godot however. Godot runs the game in a separate process and lets you explore its contents via IPC, but as the game window is not embedded, exploring the content becomes more unintuitive and annoying.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

The idea is to have an embedded game mode in Godot. This proposal describes specifically how this could be done in a way that is feasible and should allow most of the expected behavior.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The main goal is to have a new mode in the top bar called "Game", where the game can run embedded in the editor. It would look something like this:

(Note: Pretend that the icon in game is a joystick icon and I am a very skilled graphic designer).

The idea is that the game runs in this window, at either native resolution or letting you stretch the window to fit it (zooming options can be discussed further, but you get the idea).

What functions are supported and what are those icons?

• Pause: Ability to pause the running game
• Next Frame: Ability to move to the next frame while paused.

• ---

• 2D/3D selection mode: This lets you select which kind of nodes and view will be interacted with while exploring the scene.

• ---

• Game input: When this icon is toggled and the game is focused (click on it to focus), all input is captured and goes to the game until user clicks elsewhere. (Of course if the game captures the mouse, we will need some way to un-capture it with some special keybinding).
• Selection Mode: This mode no longer sends input to the game and it lets you select 2D/3D nodes (depending on the selection mode described before) by clicking on them. They will appear on the remote scene tree and inspector so you can see their content.
• List Selection Mode: Same as the mode above, but it presents a list of nodes you can choose to select instead of selecting the topmost one.

• ---

• Camera override: This lets you override the game camera and pan/zoom (if 2D is selected) or WASD/similar (if 3D is selected).

Implementation notes:

The main idea of this feature is that Godot will remain running the game as a sub process, with the main difference being that the game window is embedded into the editor. This can be done with relevant Vulkan extensions (and worse case if a platform does not support this, it just does not let you embed the game window, but most should support it).

FAQ:

Q: Are there not other 10 proposals about this?
A: Yes, but this explains how it would be done in a way that is feasible technically.

Q: Will the 2D and 3D editors show the game objects moving around?
A: No, this is not possible and probably never will. Exploring will have to happen from the game window itself. We can add tools to make this as comfortable as possible.

Q: Will I still be able to edit the scene while the game runs in a non-destructive way?
A: Yes, unlike other engines, you will not lose any change made to the scenes you are editing and the changes will also replicate to the running game.

Q: Will the editor stability be compromised by the game?
A: No, it should not be.

Q: Is this feature optional?
A: This feature is not only optional but probably only present for platforms and rendering drivers where it is possible to do this kind of embedding. I suppose users will be able to configure whether they want to use it by default or not in the editor settings.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This is core

Is there a reason why this should be core and not an add-on in the asset library?

N/A

[aaronfranke]

This would significantly increase the usability of the editor camera override. Currently you need to have both the editor and the game visible on screen, then enable the camera override, and you need to have the editor focused to move the camera in the game. This proposal significantly improves the situation so that you only need the editor visible.

[yosoyfreeman]

Could be an option to not save the changes made while running the game in this mode? I personally miss it to quickly test things, spawn objects or change values being sure it will revert when i stop it.

[aaronfranke]

@yosoyfreeman You would not be spawning objects into the running game directly. You can only spawn objects in the saved scenes, and in many cases Godot can sync that to the running game. This proposal does not change this behavior, it only improves the usability by allowing the game window to embed into the editor.

[bitbrain]

I really like this proposal! How would this behave if I were to set a breakpoint somewhere? I suspect since it still runs in a sub-process, the game would be set to paused if it reached a breakpoint?

[Bastienre4]

As a professional game developer currently working with Unity and wanting to switch to Godot, this is one of the feature I miss the most.

If I have an issue about the UI, I need to be able to inspect (both in terms of data AND visually, in the scene/game) the node/gizmos to understand what's wrong. Debugging without this is a pain 😅.

[BoneDaddy64]

Coming from Unity, all I really want is to have a scene view (probably should be embedded) that has a separate camera from the game view but renders the same scene state, as in same objects (transforms, models, animation state etc. Unity allows for different shading which is good) that I can fly around with, and to be able to pick (as in click in that view on the rendered object's model) and have the Inspector show that node (and be able to change it, like when in Remote in the Scene list currently). Whether the game view is embedded or not doesn't matter that much to me.

Basically I just want to be able to quickly fly around, select objects, have those objects appear in the Inspector, be able to real-time modify the objects (which Godot has the functionality for now). That's what's really lacking right now for me in Godot, makes it really cumbersome to "touch" my game as it's running

[aaronfranke]

@BoneDaddy64 Depending on the details of how the implementation of copying the texture to the editor works, it might be possible to have the editor camera override add a new viewport + camera and read the texture from there, instead of overriding the existing camera's transform, so the game view could keep the same camera. That would be a separate proposal made after this one, but it would help if this feature was designed with that use case in mind.

[allenwp]

As a professional game developer currently working with Unity and wanting to switch to Godot, this is one of the feature I miss the most.

If I have an issue about the UI, I need to be able to inspect (both in terms of data AND visually, in the scene/game) the node/gizmos to understand what's wrong. Debugging without this is a pain 😅.

This comment about gizmos brings up the question of whether gizmos could be drawn over top of this new Game view mode to visually show what is selected. Further to this, there are questions about whether these gizmos could modify the remote node properties, such as using the transform gizmos to translate, rotate, or scale remote nodes.

Personally, I suspect that this is something that should be kept in mind when implementing this new feature so that full gizmo support could be added at a later time. I think it would be great to get something like this implemented without gizmo overlays sooner, rather than delaying the feature to have full-featured gizmo overlays.

I also think that just being able to see gizmos in the scene would be immensely helpful to visually distinguish what node is selected and shown in the inspector panel. Being able to modify the node's properties using gizmos is a less valuable feature because this can be done through the inspector panel.

[sosasees]

i need to test window transformations often — like moving and resizing — especially when they are not triggered in the OS but in the game.

so the current way of running the game in its own window saves me much time.

so if people want the game window embedded, this should be an option and not the only way.

[allenwp]

I previously started a discussion regarding "Spacial Editor - Focus Selection should focus on remote nodes". I think this new proposal should address this feature: When in the new Game view mode in "Selection Mode", pressing the F key to Focus Selection should focus the camera override on the currently selected remote node.

This is especially helpful when you have nodes that are far away from your game or override camera, such as an off-screen world object. With this Focus Selection feature implemented for the Game view mode, it would allow the user to select the node in the remote scene tree, Focus Selection, and have the override camera focus on this node.

I also noticed that the original post mockup screenshot does not show a remote scene tree view at all, and it seems that the proposal suggests a "List Selection Mode" instead. How does this List Selection Mode compare to the existing remote scene tree? I'm curious to know more about this, especially in the context of how it will interact with node selection and the Focus Selection action.

[m4gr3d]

This would be a welcome improvement for the Android editor, given the limitations on tablets and phones to have multiple windows open side by side.