DataFrame conversion from pandas fails for non-string column names

[hhaensel]

DataFrame conversion fails, when the columns of a pandas dataframe are not strings.

julia> using PythonCall, DataFrames

julia> @py import pandas as pd

julia> pdf = pd.DataFrame(pydict(Dict(:a => [1,2], 0 => [2, 3], :T => [4, "5"])))
Python DataFrame:
   a  0  T
0  1  2  4
1  2  3  5

julia> DataFrame(pdf)
ERROR: name of column '<py NULL>' is not a string
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:33
 [2] keys(df::PyPandasDataFrame)
   @ PythonCall C:\Users\m136270\.julia\packages\PythonCall\XgP8G\src\pywrap\PyPandasDataFrame.jl:40
 [3] columns(df::PyPandasDataFrame)
   @ PythonCall C:\Users\m136270\.julia\packages\PythonCall\XgP8G\src\pywrap\PyPandasDataFrame.jl:123
 [4] columns(x::Py)
   @ PythonCall C:\Users\m136270\.julia\packages\PythonCall\XgP8G\src\pywrap\PyTable.jl:17
 [5] DataFrame(x::Py; copycols::Nothing)
   @ DataFrames C:\Users\m136270\.julia\packages\DataFrames\MA4YO\src\other\tables.jl:58
 [6] DataFrame(x::Py)
   @ DataFrames C:\Users\m136270\.julia\packages\DataFrames\MA4YO\src\other\tables.jl:49
 [7] top-level scope
   @ REPL[6]:1

There's an easy workaround to this:

julia> pdf2 = pdf.copy();

julia> pdf2.columns = string.(pdf2.columns);

julia> DataFrame(pdf2)
2×3 DataFrame
 Row │ a      0      T
     │ Int64  Int64  Any
─────┼───────────────────
   1 │     1      2  4
   2 │     2      3  5

I wonder whether this is something that could be tackled within PythonCall?

[cjdoris]

Good question. The pandas stuff hasn't been especially thought through.

Right now you need the names to be strings because PyPandasDataFrame supports operations like indexing which require going from a Julia column name to a Python one.

If we only implemented the Tables interface we wouldn't have this restriction and could automatically convert all column names to strings, possibly also letting the user control how this happens.

I'm tempted to do this - there's no real need to support anything other than the Tables interface. If the user wants more they can use the underlying python object or materialise the table to a more useful type.

[cjdoris]

Tables stuff has been reworked. Column names are now all stringified by default.