Parallel download: How should we display the progress?

[McSinyx]

This issue is opened to discuss the UI to be implemented for parallel download, with accessibility as well as platform compatibility in mind (relevant ticket: GH-8518). Concerning cursor moving support, the safest approach is to have a meta progress bar at the bottom of the output, i.e. something like apt's but not stuck to the bottom of the viewport (since that would require curses-or-similar support and defeat the purpose):

@pradyunsg suggested having multiple progress bar, similar to what showed in python-poetry/poetry#2595 and what rich provides (it unfortunately requires Python >=3.6, but similar TUI can be implemented/found elsewhere):

I believe that there are other designs as well and would love to hear more suggestions and opinions!

[willmcgugan]

Hello. Author of Rich here. Text based progress bars is one of my favourite subjects. Mention me if I can contribute in any way.

[McSinyx]

Thanks for your attention @willmcgugan! I have yet to look into rich's codebase: it there any barrier to implement cross-platform multiple progress bars on Python 2 (since pip is going to support at least until the end of the year)? In addition, is there any magic involved to make it work on Jupiter Notebook? I notice that ncurses apps don't seem to work well there (e.g. top) but I don't know any further 😄

[willmcgugan]

It wouldn't me much of a problem to get progress bars running on both Python 2 and 3. Principle is the same.

There is a bit of magic for Jupyter support. I don't think anything that moves the cursor around work work in Jupyter.

Jupyter supports HTML widgets so you could either implement it in HTML, which gives you a lot of freedom, or convert you text based bars in to HTML (which is what Rich does).

I'm surprised to hear people run pip in Jupyter tbh. But I'm not a big Jupyter user.

[pfmoore]

I'm surprised to hear people run pip in Jupyter tbh. But I'm not a big Jupyter user.

I'm pretty sure it's a bad idea. I dabble in Jupyter, but wouldn't say I'm a heavy user. But I'd have thought that you'd need to restart the kernel after installing to pick up the changes correctly. I'm -1 on having any special handling in pip for it being run in Jupyter, it's not a scenario I think we should encourage.

[uranusjr]

Every data science workshop/talk I’ve ever attended (granted I seldom do) that involves live coding use %pip install in a Jupyter Notebook. It sounds like a bad idea, but is extremely prevalent. But I agree pip doesn’t need to do too much special things for it.

[willmcgugan]

It might be worth at least detecting Jupyter and disabling progress bars. Otherwise I suspect you will get a stream of progress bars in the output.

As much as I like multiple progress bars, does pip need them? I wouldn't care too much about the progress of individual downloads, but I would be interested in the a single bar for the progress as a whole.

[pfmoore]

Every data science workshop/talk I’ve ever attended (granted I seldom do) that involves live coding use %pip install

I note that uses %pip. I don't know if there's special support in the pip magic command to make it work better. Equally, installing stuff in-process doesn't often fail, it's just risky. But the whole thing is a long-standing (core Python) issue, and affects more than just Jupyter - the discussion has come up for Idle, and I assume that other IDEs like VS Code and PyCharm need to deal with this as well. I don't think it's something pip should concern itself with directly (other than to point out that it's not our issue 🙂).

It might be worth at least detecting Jupyter and disabling progress bars.

Hopefully, our code for detecting when we aren't running in a terminal can cope with that (or can be made to, without needing special "detect Jupyter" code). Or the Jupyter people can make the %pip magic command add --no-progress automatically.

[willmcgugan]

Hopefully, our code for detecting when we aren't running in a terminal can cope with that (or can be made to, without needing special "detect Jupyter" code)

Good point. sys.stdout.isatty() reports False in Jupyter. That should be enough.