`isfile` error on too long file names

[GiggleLiu]

The maximum filename length is 255, isfile should return false rather than raising an error when the file name is too long.

julia> isfile(randstring(256))
ERROR: IOError: stat("yDdEhZcxL943psM9mDEXZJhZwXH0cnrNCR8Ps1MrdDB8I8EDaN1WDyL40PQlcEra20zt1CyDOEv4cR7BlsIyJr0hBhoFl1kP2kQBJvTcdsTjVUx65g2kgCfER3adn3xQkIVA80mYfXvQBNBxNg9rLfsOh6TjGXApOLVRBBRBWsGc6dYwFRWZgDyRhkjNBecthZ7Pgg0EQFnLPRrlXpMdBaPoV0fDF8iNfhguOdfGpwjAD7U8ABJSW2fFUeHmUoMr"): name too long (ENAMETOOLONG)
Stacktrace:
 [1] uv_error
   @ ./libuv.jl:97 [inlined]
 [2] stat(path::String)
   @ Base.Filesystem ./stat.jl:69
 [3] isfile(path::String)
   @ Base.Filesystem ./stat.jl:311
 [4] top-level scope
   @ REPL[20]:1

The same applies for isdir

relevant code:

julia/base/stat.jl

Line 311 in a163d3e

@eval ($f)(path...) = ($f)(stat(path...))

Julia version: 1.6.0-rc1

[kimikage]

The maximum filename length is 255

It's actually system dependent. I believe that silently returning false can cause inexplicable bugs.

[mgkuhn]

I suspect this issue isn't really about the maximum filename length on a particular file system, but about the question whether/when isfile (and friends isdir, islink, etc.) should raise an exception in Julia.

The POSIX stat system call can fail for various reasons, including EACCES, EIO, ELOOP, ENAMETOOLONG, ENOENT, ENOTDIR.

The docstrings of the Julia functions isfile and stat are currently completely silent on what result which of these errors lead to.

This may be a problem because the equivalent functions in other languages do not raise an exception, and are in fact frequently used to check if a file exists.

For comparison: the Perl/Bash equivalent -f does not raise an exception if called with a gigabyte long filename, i.e. both ENOENT and ENAMETOOLONG result in false:

$ perl -E 'say -f ("a" x 1e9)'

In Julia, ENOENT results in false and ENAMETOOLONG in IOError: stat: name too long (ENAMETOOLONG). At a minimum, this should be documented. I can see arguments that more errnos than just ENOENT should cause isfile to return false.

[GiggleLiu]

I tend to agree with @mgkuhn . I recall most of my using cases are about using isfile to avoid exceptions. It is not expected to return an error just because the name is too long. Just to add one API to fix: ispath .

Also, another important factor is: is there an easy way to check if the filename is too long?

[mgkuhn]

Looking at the source of stat, this just calls jl_stat which calls uv_fs_stat, none of which say anything about exceptions that they can raise.

[mgkuhn]

POSIX offers the function pathconf to query path-specific limits on the maximum filename and path length permitted by a file system.

Libuv has uv__fs_pathmax_size() to query _PC_PATH_MAX that way, but apparently no equivalent yet for _PC_NAME_MAX. Since Julia tends to do all file I/O through libuv, I think libuv would have to be extended first before equivalent functions could be added to Julia. Or is there some package that just imports all POSIX functions (for users interested in such more low-level queries and less concerned about Win32 portability)?

I don't know a Win32 function to query maximum filename length, and limits there tend to be be measured in UTF-16 words, and not in UTF-8 bytes. So "255" can mean different things!

[GiggleLiu]

I just tried to attack Pluto server (locally) by inputing a too long filename in the path, Pluto is lucky and this error does not break down the server. Not sure every web application can be so lucky under such an attack. 😏

If you input a short filename, looks reasonable

If you input a too long filename, it might confuse Pluto a bit

Too many people are assuming isfile not raising an error.

[mgkuhn]

is there an easy way to check if the filename is too long?

Not sure if that qualifies as “easy”:

istoolong(filename) = try
    stat(filename)
    false
catch e
    if isa(e, Base.IOError)
        e.code == -36
    else
        rethrow()
    end
end

istoolong("a"^800)

[GiggleLiu]

is there an easy way to check if the filename is too long?

Not sure if that qualifies as “easy”:

istoolong(filename) = try
    stat(filename)
    false
catch e
    if isa(e, Base.IOError)
        e.code == -36
    else
        rethrow()
    end
end

istoolong("a"^800)

This is good. I just want to know if people want to have a safe_isfile, how can they implement one. Thanks

[mgkuhn]

So the question is: in base/stat.jl should the functions

ispath
isfifo
ischardev
isdir
isblockdev
isfile
issocket
issetuid
issetgid
issticky
islink

catch exceptions that stat/lstat can raise and return false instead?

If yes: all exceptions that stat/lstat can raise, or only some? Which?

Which exceptions can stat/lstat raise at present?

[nstiurca]

Bump.