Welcome to NeRFRenderCore Discussions!

[JamesPerlman]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[lolo2k]

I would love to! I'm Lodewijk from Amsterdam, 360° video creator and since the pandemic very focussed on photogrammetry and subsequently NeRFs. My godmother writes children's books and I want to make an interactive NeRF based project to illustrate her stories because of the waterpaint-esque aesthetics that they provide. This is why I'm looking forward to NeRF in VR. Either as traditional 360° recordings from blender or if en when possible, 6DOF/interactive VR experience in Unity. I started to follow your progress due to the blender examples you posted on twitter, this was shared on the NeRF discord.
I'm currently trying to get equirectangular images to process in nerfstudio. Have some tests that I want to do, like renaming my Qoocam 8K render from .MP4 to .insv to spoof the ns-process script and see if it picks it up. And if that doesn't work take my 360 video and have a virtual camera rig in blender pump out a omni directional still sequence to process like normal. If anyone has any tips to make non-insta360 camera footage work. Please let me know. And If I send someone dutch treats to update the agi2nerf script to also accept spherical camera exports, then would do that in a heartbeat. I talked to the original creator and he's lost interest in this case. Thanks JPearl!

[JamesPerlman]

Hey @lolo2k! That sounds awesome, I'm so glad you're here following along! I do know of some people who have successfully used 360 camera footage with NeRFs. LumaLabs can do this, also @smallfly and @slowcon on Twitter both have run 360 footage through COLMAP. I don't work enough with 360 footage to know what tools they are using, but could definitely write some CUDA kernels to convert equirectangular to cubemap and then run those through COLMAP. For now we're a long ways away from supporting 360 cam footage but once I've done an official release, I'll potentially be open to feature requests.

[JamesPerlman]

Side note - have you tried Luma? They support 360 I believe.

[lolo2k]

I'm glad to say that all our prayers (comments) have not been for nought. Nerfstudio is working together with @gradeeterna to make the 360 cam pipeline productive! So now i'm on to the next conquest. Seeingwhat i'm doing when creating camera splines. Always with the features! But i'm used to see the camera view in blender before i start rendering.... Or am I really not workin the nerfstudio interface correctly??? Anyway! glad you are doing this! and looking forward to the blender intergration You Promissed!

[JamesPerlman]

Hmmm I'm not sure how nerfstudio works in that regard, but in this Blender addon you will see the NeRF rendered natively inside Blender. I hope to get that part done this week.

[lolo2k]

Thanks for being so easy to approach. Seeing your videos on YouTube is very inspiring and I feel super lucky to be in contact like this. Blender integration would open up so many cool possibilities. I can't wait to level up!

[Conch-212]

Hi I'm James from New York! I would like to thank you James for your contribution to the NeRF ecosystem. I've been a photographer now 3D artist so it's exciting to see this technology develop and become more accessible to people like myself thanks to you the developers.

I've been playing with the Nvidia Instant NGP binaries which has been fun and easy to play with, but I'm very excited to bring the full power of NeRFs into a proper DCC, and while my main DCC is c4d, I very much look forward to your Blender plugin.

[JamesPerlman]

Hi James! Glad you are here, and can't wait to see what you think of this when I get the Blender addon completed :)

[MrNeRF]

Hi all, I am Janusch living in Switzerland and also working full time as a C++ Software Engineer in the CAD/CAM Domain. There I mainly focus on 3D geometry processing algorithms and more recently on cloud stuff. In my spare time I am interested in NeRF as it produces astounding results. I somehow stumbled over this repo, since I wanted to learn more about NeRF implementation details and especially about cuda optimization (don't have here prior experience). I regularly try to contribute some linux (ubuntu) fixes to keep the project building under linux :)
I recently gifted myself the book "Programming Massively Parallel Processors" which goes into detail of cuda optimization. I hope I will be able to contribute what I have learned to this project. Unfortunately my schedule is quite tide so I will see how much I can do within the next weeks. Oh, by the way, I love that you often stream your development sessions, so I sometimes can peek into your implementation sessions. I saw your struggle and you did a great job so far!

[JamesPerlman]

Hey Janusch! I really appreciate your contributions and I am glad you are here. CAD/CAM stuff sounds exciting, I am really grateful to have your eyes on this project, as your time permits.