tooling is rough

[ipsod]

Hi,

I'm working on dev tooling. As a Python developer, the initial experience is pretty jarring, because I just keep finding things I know how to do everywhere but here.

• Install the package (was unexpectedly difficult on Ubuntu 24.04)
• import any python library in a PythonSCAD file by installing it in a virtual env
• import any python library in a PythonSCAD file by installing it in a virtual env (in the standard way)
• import PythonSCAD into another project
• Get code-intelligence in VSCode, vim, PyCharm, etc.
• Output text to my terminal (not the OpenSCAD console)
• Use standard REPL
• Use REPL of my choice
• Use a Jupyter notebook
• Use debuggers (pdb, PyCharm, VSCode)

In the past, I've used PyBind11 and cppyy to run C++ code inside of Python. Do you think it might be possible to reverse your architecture, and put the C inside of the Python? Because, I think that would solve most or all of these problems.

If that idea's dumb... is there one that isn't?

[gsohler]

1st of all, i appreciate your effort!
I have tried to use pybind11 in the past, but i was not able to map many/all of the features to pybind11.
I believe if we put C code into python code, we could be able to ship as a python module,
but the big disadvantage, would be, that we would not be able to g++ link into pythonscad executable and thus not having access to internal structures

[gsohler]

jupyter notpook would be absolutely cool. I think you would need to run pythonscad in command line mode, output stl and display it with javascript ?

[ipsod]

Well, we don't want to lose the advantages, or we could just stick with SolidPython, right?

I don't know how I'd use Jupyter notebooks specifically for this - I use them all over the place because some problems are just easier to get at through them. I'm sure it's easy enough to output images from openscad and easy enough to have jupyter display them, but right now I just want to be able to troubleshoot and tinker there, and the normal OpenSCAD preview is fine for output. Oh, but... I don't actually know what that might require, or what's going on inside of your library, so maybe it's not so simple without some deep architecture changes. But, thinking now, the interaction with the UI is limited to calls to things like the output function, right? So, maybe it'd be really simple, just a problem that solves itself if you can get this thing to be a pip-installable Python package?

[ipsod]

What about a client/server architecture, where the client is pip-installable but the server isn't? Something where, you build out a Python server that embodies the .pyi file that defines your application, and a Python client that talks to it. Would probably lose some cycles to encode/decode, but I'd rather have a slower application (within reason) than the current developer experience.

I don't now if this is a dumb idea or not yet - would probably have to write a few hundreds or thousands of lines of code to know. But, if it's possible to do without, then client/server is probably a bad idea whether or not it'd work.

[gsohler]

If you have a client/server solution, its very easy that the server/client versions/abilities get out of sync,
The additional time for a single communication with additionan encode/decode might be neglible, but i can think of model
creation scenarious which will calculate each voxel separately and the additional time will expand significantly. additional difficuly is that python in pytonscad already run in threads.
If you have python on top rather than C, its literally a different application Its like solidpython2 and if you prefer this type of approach, I feel you want enhance on solidpython.
Again, i feel the interface between server and client could be very volatile.
Today's case: i am almost ready anouncing that i can instanciate SCAD libraries(like BOSL2) into python, this interface does not only contain paramter passing and object returning, but also includes the abilities to pass python children with the scad modules - just today's example

[ipsod]

Yeah, that sounds tough. Do you know of other ways to solve any of the problems I listed?

I'm not really sure what I want. I got frustrated with SolidPython a year or two back, or maybe just frustrated with myself for spending more time building features that never quite worked right than building geometries (features that I think PythonSCAD might make easier or just outright already have), and have basically focused on robotics motion planning and control for the past year or two while waiting for the situation to get better. Was looking at other libraries, particularly PyMadCAD and CADQuery, when I left. I was hoping that this might be a way back that wouldn't require so much re-learning (and giving up the benefits that OpenSCAD gives - easier to build some shapes, lots of libraries).

I like where you're going. I don't mean to discount your work in any way - I'm a fan. I just get a very claustrophobic feeling from all the Python tools I'm used to using being cut off. But, maybe once I get more accustomed to your library's advantages, that'll outshine what's being left behind.

[ipsod]

Also, Python on top is a totally different program? Necessarily?

I mean, I can see how that's probably so... but, I did have the experience of... I used to write C libraries for Arduino, and then write Python libraries to talk to them from the PC, with a gnarly communication layer between. But, now, I just import the C into Python with cppyy, and run the same code on my PC as is running on the MCU, and everything got a lot simpler, as I can just dump c objects to bytes and rehydrate the bytes directly on either end. You're dealing with something probably many orders of magnitude more complex, though, so this might be totally irrelevant.

[ipsod]

Well, what you're doing is probably more complex, but your exposed API, the .pyi file, is really pretty narrowly defined. You surely aren't fighting the same realtime requirements robotics does - you say they can get out of sync, but I don't see that they need to communicate except when specific functions are called, which seems like an easy enough sync problem to solve. I can't help but feel that there's a simple-enough solution, perhaps if we allow for a bit of a hack.

[gsohler]

For some of your issues i have easy solutions, for others i dont.
redirecting python errors to openscad console can easily be turned off(redirection was the tough task)
For debugging i use "print" method instead of breakpoints, I am wondering if there are scenarios, where pdb is better.

I understand, that you are diappointed in some way, also I am sorry for that, but please understand that i am almost alone on this project and i do it in my spare time because i "believe" its a great thing. No way that will/can change the speed of direction/speed drastically. But of course you can influence with PR's.

When i started to use python, it was just my favorite language, but when i reconsidered my choice later, i still concluded to use python. its most similar to openscad because of the named parameter arguments.
BTW python is not on top, but its sitting next to SCAD parser. For SolidPython2 its on top.

may be my the out of sync error is on anther frequency/speed, here it fails qjuite permanent. If you still feel, that you want python on top, you can add an server/client interface in ADDITION as PR, i will very likely accept it, but issue is that the added functionaity needs to be in sync with the embedded python functions.

I know that my project might appear rough to others, but thats only because of my inabilities in certain topics. This is were i depend on/hope on/ the help/contribution of others.

[ipsod]

Oh, certainly, I understand - I'm in exactly the same boat with all of my development, just doing what I can with what I have. I don't mean to demean your project in any way, I wouldn't be putting time into it if I didn't also think it was worthwhile. It's not my intent to demean your project by listing problems, but hopefully to improve it.

I will keep it in mind as I work. If I find it painful enough, I will look further into creating a client/server architecture addition.

Thank you for sharing your work.

[ipsod]

Oh. For debugging, you can do better than print with log. Log seems like print with more steps, but, sometimes it's very useful to have the granular controls it gives. Turning up logs on this file, and down on that one, for example.

[gsohler]

Yeah, i believe it could be a benefit controlling pythonscad from "outside", not only from a python module. Especially i think, it could be cool, to control the viewing parameters like center of view, focus, direction ...
pdb would immediately work if we managed to connect the stdin path from the GUI to the python core ...

[ipsod]

Yeah... can you control those viewing parameters from PythonSCAD? I don't see any functions mentioned for that, can we get one?

[gsohler]

There some special variables like $vpr in OpenSCAD only yet which can be used to change the view during an animation.
Its not yet possible in PythonSCAD

But with an internal server it will be even possible to control from outside.

[gsohler]

I just had some brainstorming.
With PythonSCAD it would be rather easy to have an option where pythonscad would
behave exactly like a command line python interpreter(ipython). PDB would work out of the box, it would be a REPR.
(PLUS all the geomtrical functions would of course be available).
With some minor interface it would actually be quite easy to use it in jupyter. I would definitely support you on the pythonscad side.