DeviceOrientationControls and react-scripts (drei)

[alexlee22]

Hello!

I've being playing around with the different helpers using the drei library and tried to get the DeviceOrientationControls to run. I was able to replicate the Storybook example in CodeSandbox, however when I downloaded the files and tried running it locally via npm run start, I noticed the device controls stopped working.

It appears to function as intended in the CodeSandbox browser but did not detect the motion controls when viewing the local build on my phone. I wasn't sure if I am doing something wrong as I am not well experienced with the back-end of React and ThreeJS.

Cheers!

[alexlee22]

I noticed the build files (using the command npm run build) does work with the Device Orientation Controls, however the dev server (using npm run start) does not work when viewed on my phone running the server on my computer. Here is the live build using the CodeSandbox above.

UPDATE:
Found out why it was not working, Device Orientation Controls require HTTPS: StackOverflow post

How to Run as HTTPS in create-react-app

[forerunrun]

@Joshimmor afaik device orientation requires permission to run on ios devices as a safety feature to prevent inferring key strokes, there's a very interesting paper on this here. The workaround to enable cross browser support (in three.js) was to create the controls themselves on the click of a button, to which the internals of DeviceOrientationControls would request permission to use the devices orientation. I will likely need to use this soon also so I'll have a look into drei's implementation and see if I can make a PR...

[atTommy98]

@forerunrun Did you ever get around to figuring this out? I've tried rendering DeviceOrentationControls at the click of a button as you suggested. I'm also running the app on HTTPS, but the application still doesn't register anything on my iPhone.

[xih]

@atTommy98 @forerunrun Running into the same issue on my iPhone as well.

Been trying out this prototype on iPhone and it's not working:
https://codesandbox.io/s/off-axis-projection-efx-ugpfn?file=/src/index.js

[atTommy98]

@xih So I actually managed to solve this issue. I believe the error was with DeviceOrientationControls not forcing iOS to grant permission. If I remember correctly, you have to use Device Motion events to activate the orientation on iOS.

I found/created a component that pops up first thing before entering the app that grants the iOS device permission to device orientation and then DeviceOrientationControls worked fine.

It looked something like this:
In App.js
const [permissionGranted, setPermissionGranted] = useState(false);
Conditional rendering for the accelerometer:
{permissionGranted ? ( <Main /> ) : ( <Accelerometer permissionGranted={permissionGranted} setPermissionGranted={setPermissionGranted} /> )}

I've attached an image of the accelerometer code itself. Didn't want to paste all the code into this textbox as it doesn't correctly indent or new line the code😬 Hope this helps!

[3dpk-interactive]

This is great, however may be overengineered as DeviceOrientationControls handles all of this internally with the connect() method, all you need to do is use deviceOrientationControlsRef.current.connect() on an initial button click...

function ThreeScene() {

    const [permissionGranted, setPermissionGranted] = useState(false)
    const deviceOrientationControlsRef = useRef()

    const handleClick = () => {
        deviceOrientationControlsRef.current.connect()
        setPermissionGranted(true)
    }

    return (
        <>
            <div className="threeContainer">
                <Canvas
                    camera={{ position: [0,0,-10] }}
                >
                    <DeviceOrientationControls ref={deviceOrientationControlsRef}/>

                </Canvas>
            </div>
            {!permissionGranted && 
                <div className="accelerometerModal">
                    <button onClick={handleClick}>
                        use device orientation
                    </button>
                </div>
            }
        </>
    )
}

As pointed out by @alexlee22 and yourself, running a local development environment requires the project to be served over HTTPS for this to work, '@vitejs/plugin-basic-ssl' works a charm for this in vite, you can just install it and add it to the plugins array in vite.config.js

[xih]

@atTommy98

thanks for the help! Unfortunately still getting errors. In particular:

can you share a sample repo or codpen that demonstrates how you have it working ?

[atTommy98]

I'll do my best to draft one up real quick. Bare with me. Though I'm sure you're not active right now considering you're in San Fran and I'm in the UK😂

[atTommy98]

@xih Orientation Repo
Things of note. You have to run your React project with HTTPS. I've done this by adding 'set HTTPS=true&&react-scripts start' to the npm start script. Finally, I had problems using TypeScript with the accelerometer too so I didn't bother. Make sure Accelerometer.js is a JavaScript file, not TypeScript. I'm sure it's fixable, but I'm not too well versed in TypeScript to figure it out.

When testing the demo, run the project from your local computer then access on your phone with https://yourNetworkIP:3000 and it should work correctly.

Hopefully that will solve it for you. Any other questions don't hesitate to ask :)

[xih]

@atTommy98

unfortunately, having trouble with running HTTPS through localhost (192.168.1.1:3000). set HTTPS=true&&react-scripts start didn't work for me.

Have you used VITE to have it deployed on local vs network? I haven't done this myself, but I'll try to play around with it

[xr-dev-saurabh]

You can use Ngrok to test your local development on any mobile device. It provides both http and https. Try https, it worked for me.

[Romajon-c137]

Мужик благодаря тебе я нашел крутую библиотеку (https://drei.pmnd.rs/)

[3dpk-interactive]

DeviceOrientationControls handles DeviceOrientationEvent.requestPermission() internally with its connect() method, all you need to do is use deviceOrientationControlsRef.current.connect() on an initial button click...

function ThreeScene() {

    const [permissionGranted, setPermissionGranted] = useState(false)
    const deviceOrientationControlsRef = useRef()

    const handleClick = () => {
        deviceOrientationControlsRef.current.connect()
        setPermissionGranted(true)
    }

    return (
        <>
            <div className="threeContainer">
                <Canvas
                    camera={{ position: [0,0,-10] }}
                >
                    <DeviceOrientationControls ref={deviceOrientationControlsRef}/>

                </Canvas>
            </div>
            {!permissionGranted && 
                <div className="accelerometerModal">
                    <button onClick={handleClick}>
                        use device orientation
                    </button>
                </div>
            }
        </>
    )
}

As pointed out by @alexlee22 and @atTommy98, running a local development environment requires the project to be served over HTTPS for this to work, '@vitejs/plugin-basic-ssl' works a charm for this in vite, you can just install it and add it to the plugins array in vite.config.js.