system LIST on Win32 doesn't quote accurately

[p5pRT]

Migrated from rt.perl.org#119419 (status was 'open')

Searchable as RT119419$

[p5pRT]

From @haarg

This is a bug report for perl from haarg@​haarg.org,
generated with the help of perlbug 1.39 running under perl 5.18.0.

---

Program arguments on Win32 are passed as a single string, which is
then parsed by the program on startup. While it's possible for a
program to do arbitrary parsing of the command line string, almost
all programs either use the routine provided by the MS standard C
library, or a routine with compatible rules.

For system LIST to work, perl must assemble the list into a single
string. But the strings it generates aren't always compatible with
the standard parsing rules. Attached is a test script that shows
multiple strings that aren't quoted properly.

cmd.exe uses a different set of parsing rules from the C runtime. It
isn't possible to consistently generate strings that will be handled
the same by cmd.exe and the standard C routine. This can be
problematic because in some cases, perl will re-try failed system
calls using cmd.exe.

I have a module on CPAN, Win32​::ShellQuote, that will quote things
properly for Win32.

---

---

Flags​:
  category=core
  severity=low

---

Site configuration information for perl 5.18.0​:

Configured by gknop at Sat May 18 11​:37​:06 EDT 2013.

Summary of my perl5 (revision 5 version 18 subversion 0) configuration​:
  Commit id​: a9acda3
  Platform​:
  osname=darwin, osvers=12.3.0, archname=darwin-2level
  uname='darwin cuneus 12.3.0 darwin kernel version 12.3.0​: sun jan
6 22​:37​:10 pst 2013; root​:xnu-2050.22.13~1release_x86_64 x86_64 '
  config_args='-des -Dusedevel -Uversiononly
-Dprefix=/Users/gknop/perl5/perls/v5.18.0'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -I/opt/local/include',
  optimize='-O3',
  cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include -I/opt/local/include'
  ccversion='', gccversion='4.2.1 Compatible Apple LLVM 4.2
(clang-425.0.28)', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags ='
-fstack-protector -L/usr/local/lib -L/opt/local/lib'
  libpth=/usr/local/lib /opt/local/lib /usr/lib
  libs=-lgdbm -ldbm -ldb -ldl -lm -lutil -lc
  perllibs=-ldl -lm -lutil -lc
  libc=, so=dylib, useshrplib=false, libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup
-L/usr/local/lib -L/opt/local/lib -fstack-protector'

Locally applied patches​:

---

@​INC for perl 5.18.0​:
  /Users/gknop/perl5/perls/v5.18.0/lib/site_perl/5.18.0/darwin-2level
  /Users/gknop/perl5/perls/v5.18.0/lib/site_perl/5.18.0
  /Users/gknop/perl5/perls/v5.18.0/lib/5.18.0/darwin-2level
  /Users/gknop/perl5/perls/v5.18.0/lib/5.18.0
  .

---

Environment for perl 5.18.0​:
  DYLD_LIBRARY_PATH (unset)
  HOME=/Users/gknop
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/Users/gknop/bin​:/Users/gknop/perl5/perls/active/bin​:/opt/local/bin​:/opt/X11/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/bin​:/bin​:/usr/sbin​:/sbin
  PERL_BADLANG (unset)
  SHELL=/usr/local/bin/bash

[p5pRT]

From @haarg

use strict;
use warnings;
use Data​::Dumper ();

sub dd ($) {
  my $params = shift;
  local $Data​::Dumper​::Indent = 0;
  local $Data​::Dumper​::Terse = 1;
  local $Data​::Dumper​::Useqq = 1;
  local $Data​::Dumper​::Sortkeys = 1;
  local $Data​::Dumper​::Useqq = 1;

  my $out = Data​::Dumper​::Dumper($params);
  chomp $out;
  return $out;
}

BEGIN {
  if (@​ARGV && $ARGV[0] eq '-d') {
  shift @​ARGV;
  print dd \@​ARGV;
  exit;
  }
}

use Test​::More;
use Capture​::Tiny qw(capture_merged);

my @​strings = (
  'a',
  'a b',
  '"a b"',
  '"a" b',
  '"a" "b"',
  '\'a\'',
  '"a',
  '"a b',
  '\'a',
  '\'a b',
  '\'a b"',
  '\\a',
  '\\"a',
  '\\ a',
  '\\ "\' a',
  [ '\\ "\' a', ">\\"],
  '%a%',
  '%a b',
  '\%a b',
  ' & help & ',
  ' > out',
  ' | welp',
  '" | welp"',
  '\" | welp',
  "",

# from EUMM. Not all meant to be used like this, but still good test material
  q{print "foo'o", ' bar"ar'},
  q{$PATH = 'foo'; print $PATH},
  q{print 'foo'},
  q{print " \" "},
  q{print " < \" "},
  q{print " \" < "},
  q{print " < \"\" < \" < \" < "},
  q{print " < \" | \" < | \" < \" < "},

  q{print q[ &<>^|()@​ ! ]},
  q{print q[ &<>^|@​()!"&<>^|@​()! ]},
  q{print q[ "&<>^|@​() !"&<>^|@​() !" ]},
  q{print q[ "C​:\TEST A\" ]},
  q{print q[ "C​:\TEST %&^ A\" ]},

  "\n",
  "a\nb",
  "a\rb",
  "a\nb > welp",
  "a > welp\n219",
  "a\"b\nc",

  "a\fb",
  "a\x0bb",
  "a\x{85}b",
);

plan tests => 3*@​strings;

for my $test (@​strings) {
  my @​test_strings = ref $test ? @​$test : $test;
  for my $params ( [@​test_strings], [@​test_strings, '>out'], [@​test_strings, '%'] ) {
  my $name = 'roundtrip ' . dd $params;
  my $out = capture_merged { system $^X, __FILE__, '-d', @​$params; };
  is $out, dd $params, $name;
  }
}

[p5pRT]

From @jkeenan

On Thu Aug 22 00​:57​:49 2013, haarg wrote​:

This is a bug report for perl from haarg@​haarg.org,
generated with the help of perlbug 1.39 running under perl 5.18.0.

-----------------------------------------------------------------

Program arguments on Win32 are passed as a single string, which is
then parsed by the program on startup. While it's possible for a
program to do arbitrary parsing of the command line string, almost
all programs either use the routine provided by the MS standard C
library, or a routine with compatible rules.

For system LIST to work, perl must assemble the list into a single
string. But the strings it generates aren't always compatible with
the standard parsing rules. Attached is a test script that shows
multiple strings that aren't quoted properly.

cmd.exe uses a different set of parsing rules from the C runtime. It
isn't possible to consistently generate strings that will be handled
the same by cmd.exe and the standard C routine. This can be
problematic because in some cases, perl will re-try failed system
calls using cmd.exe.

Would you be able to provide some examples of failing cases?

I have a module on CPAN, Win32​::ShellQuote, that will quote things
properly for Win32.

Thank you very much.
Jim Keenan

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @haarg

On Thu Aug 22 18​:34​:41 2013, jkeenan wrote​:

Would you be able to provide some examples of failing cases?

The initial report has a test case attached that demonstrates the
issues. I've attached the output of running it on Windows.

[p5pRT]

From @haarg

1..141
ok 1 - roundtrip ["a"]
ok 2 - roundtrip ["a",">out"]
ok 3 - roundtrip ["a","%"]
ok 4 - roundtrip ["a b"]
ok 5 - roundtrip ["a b",">out"]
ok 6 - roundtrip ["a b","%"]
not ok 7 - roundtrip ["\"a b\""]
# Failed test 'roundtrip ["\"a b\""]'
# at quoting.t line 91.
# got​: '["a b"]'
# expected​: '["\"a b\""]'
not ok 8 - roundtrip ["\"a b\"",">out"]
# Failed test 'roundtrip ["\"a b\"",">out"]'
# at quoting.t line 91.
# got​: '["a b",">out"]'
# expected​: '["\"a b\"",">out"]'
not ok 9 - roundtrip ["\"a b\"","%"]
# Failed test 'roundtrip ["\"a b\"","%"]'
# at quoting.t line 91.
# got​: '["a b","%"]'
# expected​: '["\"a b\"","%"]'
not ok 10 - roundtrip ["\"a\" b"]
# Failed test 'roundtrip ["\"a\" b"]'
# at quoting.t line 91.
# got​: '["a","b"]'
# expected​: '["\"a\" b"]'
not ok 11 - roundtrip ["\"a\" b",">out"]
# Failed test 'roundtrip ["\"a\" b",">out"]'
# at quoting.t line 91.
# got​: '["a","b",">out"]'
# expected​: '["\"a\" b",">out"]'
not ok 12 - roundtrip ["\"a\" b","%"]
# Failed test 'roundtrip ["\"a\" b","%"]'
# at quoting.t line 91.
# got​: '["a","b","%"]'
# expected​: '["\"a\" b","%"]'
not ok 13 - roundtrip ["\"a\" \"b\""]
# Failed test 'roundtrip ["\"a\" \"b\""]'
# at quoting.t line 91.
# got​: '["a","b"]'
# expected​: '["\"a\" \"b\""]'
not ok 14 - roundtrip ["\"a\" \"b\"",">out"]
# Failed test 'roundtrip ["\"a\" \"b\"",">out"]'
# at quoting.t line 91.
# got​: '["a","b",">out"]'
# expected​: '["\"a\" \"b\"",">out"]'
not ok 15 - roundtrip ["\"a\" \"b\"","%"]
# Failed test 'roundtrip ["\"a\" \"b\"","%"]'
# at quoting.t line 91.
# got​: '["a","b","%"]'
# expected​: '["\"a\" \"b\"","%"]'
ok 16 - roundtrip ["'a'"]
ok 17 - roundtrip ["'a'",">out"]
ok 18 - roundtrip ["'a'","%"]
not ok 19 - roundtrip ["\"a"]
# Failed test 'roundtrip ["\"a"]'
# at quoting.t line 91.
# got​: '["a"]'
# expected​: '["\"a"]'
not ok 20 - roundtrip ["\"a",">out"]
# Failed test 'roundtrip ["\"a",">out"]'
# at quoting.t line 91.
# got​: '["a >out"]'
# expected​: '["\"a",">out"]'
not ok 21 - roundtrip ["\"a","%"]
# Failed test 'roundtrip ["\"a","%"]'
# at quoting.t line 91.
# got​: '["a %"]'
# expected​: '["\"a","%"]'
not ok 22 - roundtrip ["\"a b"]
# Failed test 'roundtrip ["\"a b"]'
# at quoting.t line 91.
# got​: '["a b"]'
# expected​: '["\"a b"]'
not ok 23 - roundtrip ["\"a b",">out"]
# Failed test 'roundtrip ["\"a b",">out"]'
# at quoting.t line 91.
# got​: '["a b >out"]'
# expected​: '["\"a b",">out"]'
not ok 24 - roundtrip ["\"a b","%"]
# Failed test 'roundtrip ["\"a b","%"]'
# at quoting.t line 91.
# got​: '["a b %"]'
# expected​: '["\"a b","%"]'
ok 25 - roundtrip ["'a"]
ok 26 - roundtrip ["'a",">out"]
ok 27 - roundtrip ["'a","%"]
ok 28 - roundtrip ["'a b"]
ok 29 - roundtrip ["'a b",">out"]
ok 30 - roundtrip ["'a b","%"]
not ok 31 - roundtrip ["'a b\""]
# Failed test 'roundtrip ["'a b\""]'
# at quoting.t line 91.
# got​: '["'a","b"]'
# expected​: '["'a b\""]'
not ok 32 - roundtrip ["'a b\"",">out"]
# Failed test 'roundtrip ["'a b\"",">out"]'
# at quoting.t line 91.
# got​: '["'a","b >out"]'
# expected​: '["'a b\"",">out"]'
not ok 33 - roundtrip ["'a b\"","%"]
# Failed test 'roundtrip ["'a b\"","%"]'
# at quoting.t line 91.
# got​: '["'a","b %"]'
# expected​: '["'a b\"","%"]'
ok 34 - roundtrip ["\\a"]
ok 35 - roundtrip ["\\a",">out"]
ok 36 - roundtrip ["\\a","%"]
not ok 37 - roundtrip ["\\\"a"]
# Failed test 'roundtrip ["\\\"a"]'
# at quoting.t line 91.
# got​: '["\"a"]'
# expected​: '["\\\"a"]'
not ok 38 - roundtrip ["\\\"a",">out"]
# Failed test 'roundtrip ["\\\"a",">out"]'
# at quoting.t line 91.
# got​: '["\"a",">out"]'
# expected​: '["\\\"a",">out"]'
not ok 39 - roundtrip ["\\\"a","%"]
# Failed test 'roundtrip ["\\\"a","%"]'
# at quoting.t line 91.
# got​: '["\"a","%"]'
# expected​: '["\\\"a","%"]'
ok 40 - roundtrip ["\\ a"]
ok 41 - roundtrip ["\\ a",">out"]
ok 42 - roundtrip ["\\ a","%"]
not ok 43 - roundtrip ["\\ \"' a"]
# Failed test 'roundtrip ["\\ \"' a"]'
# at quoting.t line 91.
# got​: '["\\","' a"]'
# expected​: '["\\ \"' a"]'
not ok 44 - roundtrip ["\\ \"' a",">out"]
# Failed test 'roundtrip ["\\ \"' a",">out"]'
# at quoting.t line 91.
# got​: '["\\","' a >out"]'
# expected​: '["\\ \"' a",">out"]'
not ok 45 - roundtrip ["\\ \"' a","%"]
# Failed test 'roundtrip ["\\ \"' a","%"]'
# at quoting.t line 91.
# got​: '["\\","' a %"]'
# expected​: '["\\ \"' a","%"]'
not ok 46 - roundtrip ["\\ \"' a",">\\"]
# Failed test 'roundtrip ["\\ \"' a",">\\"]'
# at quoting.t line 91.
# got​: '["\\","' a >\\"]'
# expected​: '["\\ \"' a",">\\"]'
not ok 47 - roundtrip ["\\ \"' a",">\\",">out"]
# Failed test 'roundtrip ["\\ \"' a",">\\",">out"]'
# at quoting.t line 91.
# got​: '["\\","' a >\\ >out"]'
# expected​: '["\\ \"' a",">\\",">out"]'
not ok 48 - roundtrip ["\\ \"' a",">\\","%"]
# Failed test 'roundtrip ["\\ \"' a",">\\","%"]'
# at quoting.t line 91.
# got​: '["\\","' a >\\ %"]'
# expected​: '["\\ \"' a",">\\","%"]'
ok 49 - roundtrip ["%a%"]
ok 50 - roundtrip ["%a%",">out"]
ok 51 - roundtrip ["%a%","%"]
ok 52 - roundtrip ["%a b"]
ok 53 - roundtrip ["%a b",">out"]
ok 54 - roundtrip ["%a b","%"]
ok 55 - roundtrip ["\\%a b"]
ok 56 - roundtrip ["\\%a b",">out"]
ok 57 - roundtrip ["\\%a b","%"]
ok 58 - roundtrip [" & help & "]
ok 59 - roundtrip [" & help & ",">out"]
ok 60 - roundtrip [" & help & ","%"]
ok 61 - roundtrip [" > out"]
ok 62 - roundtrip [" > out",">out"]
ok 63 - roundtrip [" > out","%"]
ok 64 - roundtrip [" | welp"]
ok 65 - roundtrip [" | welp",">out"]
ok 66 - roundtrip [" | welp","%"]
not ok 67 - roundtrip ["\" | welp\""]
# Failed test 'roundtrip ["\" | welp\""]'
# at quoting.t line 91.
# got​: '[" | welp"]'
# expected​: '["\" | welp\""]'
not ok 68 - roundtrip ["\" | welp\"",">out"]
# Failed test 'roundtrip ["\" | welp\"",">out"]'
# at quoting.t line 91.
# got​: '[" | welp",">out"]'
# expected​: '["\" | welp\"",">out"]'
not ok 69 - roundtrip ["\" | welp\"","%"]
# Failed test 'roundtrip ["\" | welp\"","%"]'
# at quoting.t line 91.
# got​: '[" | welp","%"]'
# expected​: '["\" | welp\"","%"]'
not ok 70 - roundtrip ["\\\" | welp"]
# Failed test 'roundtrip ["\\\" | welp"]'
# at quoting.t line 91.
# got​: '["\"","|","welp"]'
# expected​: '["\\\" | welp"]'
not ok 71 - roundtrip ["\\\" | welp",">out"]
# Failed test 'roundtrip ["\\\" | welp",">out"]'
# at quoting.t line 91.
# got​: '["\"","|","welp",">out"]'
# expected​: '["\\\" | welp",">out"]'
not ok 72 - roundtrip ["\\\" | welp","%"]
# Failed test 'roundtrip ["\\\" | welp","%"]'
# at quoting.t line 91.
# got​: '["\"","|","welp","%"]'
# expected​: '["\\\" | welp","%"]'
ok 73 - roundtrip [""]
ok 74 - roundtrip ["",">out"]
ok 75 - roundtrip ["","%"]
not ok 76 - roundtrip ["print \"foo'o\", ' bar\"ar'"]
# Failed test 'roundtrip ["print \"foo'o\", ' bar\"ar'"]'
# at quoting.t line 91.
# got​: '["print","foo'o,","'","barar'"]'
# expected​: '["print \"foo'o\", ' bar\"ar'"]'
not ok 77 - roundtrip ["print \"foo'o\", ' bar\"ar'",">out"]
# Failed test 'roundtrip ["print \"foo'o\", ' bar\"ar'",">out"]'
# at quoting.t line 91.
# got​: '["print","foo'o,","'","barar' >out"]'
# expected​: '["print \"foo'o\", ' bar\"ar'",">out"]'
not ok 78 - roundtrip ["print \"foo'o\", ' bar\"ar'","%"]
# Failed test 'roundtrip ["print \"foo'o\", ' bar\"ar'","%"]'
# at quoting.t line 91.
# got​: '["print","foo'o,","'","barar' %"]'
# expected​: '["print \"foo'o\", ' bar\"ar'","%"]'
ok 79 - roundtrip ["\$PATH = 'foo'; print \$PATH"]
ok 80 - roundtrip ["\$PATH = 'foo'; print \$PATH",">out"]
ok 81 - roundtrip ["\$PATH = 'foo'; print \$PATH","%"]
ok 82 - roundtrip ["print 'foo'"]
ok 83 - roundtrip ["print 'foo'",">out"]
ok 84 - roundtrip ["print 'foo'","%"]
not ok 85 - roundtrip ["print \" \\\" \""]
# Failed test 'roundtrip ["print \" \\\" \""]'
# at quoting.t line 91.
# got​: '["print"," \" "]'
# expected​: '["print \" \\\" \""]'
not ok 86 - roundtrip ["print \" \\\" \"",">out"]
# Failed test 'roundtrip ["print \" \\\" \"",">out"]'
# at quoting.t line 91.
# got​: '["print"," \" ",">out"]'
# expected​: '["print \" \\\" \"",">out"]'
not ok 87 - roundtrip ["print \" \\\" \"","%"]
# Failed test 'roundtrip ["print \" \\\" \"","%"]'
# at quoting.t line 91.
# got​: '["print"," \" ","%"]'
# expected​: '["print \" \\\" \"","%"]'
not ok 88 - roundtrip ["print \" < \\\" \""]
# Failed test 'roundtrip ["print \" < \\\" \""]'
# at quoting.t line 91.
# got​: '["print"," < \" "]'
# expected​: '["print \" < \\\" \""]'
not ok 89 - roundtrip ["print \" < \\\" \"",">out"]
# Failed test 'roundtrip ["print \" < \\\" \"",">out"]'
# at quoting.t line 91.
# got​: '["print"," < \" ",">out"]'
# expected​: '["print \" < \\\" \"",">out"]'
not ok 90 - roundtrip ["print \" < \\\" \"","%"]
# Failed test 'roundtrip ["print \" < \\\" \"","%"]'
# at quoting.t line 91.
# got​: '["print"," < \" ","%"]'
# expected​: '["print \" < \\\" \"","%"]'
not ok 91 - roundtrip ["print \" \\\" < \""]
# Failed test 'roundtrip ["print \" \\\" < \""]'
# at quoting.t line 91.
# got​: '["print"," \" < "]'
# expected​: '["print \" \\\" < \""]'
not ok 92 - roundtrip ["print \" \\\" < \"",">out"]
# Failed test 'roundtrip ["print \" \\\" < \"",">out"]'
# at quoting.t line 91.
# got​: '["print"," \" < ",">out"]'
# expected​: '["print \" \\\" < \"",">out"]'
not ok 93 - roundtrip ["print \" \\\" < \"","%"]
# Failed test 'roundtrip ["print \" \\\" < \"","%"]'
# at quoting.t line 91.
# got​: '["print"," \" < ","%"]'
# expected​: '["print \" \\\" < \"","%"]'
not ok 94 - roundtrip ["print \" < \\\"\\\" < \\\" < \\\" < \""]
# Failed test 'roundtrip ["print \" < \\\"\\\" < \\\" < \\\" < \""]'
# at quoting.t line 91.
# got​: '["print"," < \"\" < \" < \" < "]'
# expected​: '["print \" < \\\"\\\" < \\\" < \\\" < \""]'
not ok 95 - roundtrip ["print \" < \\\"\\\" < \\\" < \\\" < \"",">out"]
# Failed test 'roundtrip ["print \" < \\\"\\\" < \\\" < \\\" < \"",">out"]'
# at quoting.t line 91.
# got​: '["print"," < \"\" < \" < \" < ",">out"]'
# expected​: '["print \" < \\\"\\\" < \\\" < \\\" < \"",">out"]'
not ok 96 - roundtrip ["print \" < \\\"\\\" < \\\" < \\\" < \"","%"]
# Failed test 'roundtrip ["print \" < \\\"\\\" < \\\" < \\\" < \"","%"]'
# at quoting.t line 91.
# got​: '["print"," < \"\" < \" < \" < ","%"]'
# expected​: '["print \" < \\\"\\\" < \\\" < \\\" < \"","%"]'
not ok 97 - roundtrip ["print \" < \\\" | \\\" < | \\\" < \\\" < \""]
# Failed test 'roundtrip ["print \" < \\\" | \\\" < | \\\" < \\\" < \""]'
# at quoting.t line 91.
# got​: '["print"," < \" | \" < | \" < \" < "]'
# expected​: '["print \" < \\\" | \\\" < | \\\" < \\\" < \""]'
not ok 98 - roundtrip ["print \" < \\\" | \\\" < | \\\" < \\\" < \"",">out"]
# Failed test 'roundtrip ["print \" < \\\" | \\\" < | \\\" < \\\" < \"",">out"]'
# at quoting.t line 91.
# got​: '["print"," < \" | \" < | \" < \" < ",">out"]'
# expected​: '["print \" < \\\" | \\\" < | \\\" < \\\" < \"",">out"]'
not ok 99 - roundtrip ["print \" < \\\" | \\\" < | \\\" < \\\" < \"","%"]
# Failed test 'roundtrip ["print \" < \\\" | \\\" < | \\\" < \\\" < \"","%"]'
# at quoting.t line 91.
# got​: '["print"," < \" | \" < | \" < \" < ","%"]'
# expected​: '["print \" < \\\" | \\\" < | \\\" < \\\" < \"","%"]'
ok 100 - roundtrip ["print q[ &<>^|()\@​ ! ]"]
ok 101 - roundtrip ["print q[ &<>^|()\@​ ! ]",">out"]
ok 102 - roundtrip ["print q[ &<>^|()\@​ ! ]","%"]
not ok 103 - roundtrip ["print q[ &<>^|\@​()!\"&<>^|\@​()! ]"]
# Failed test 'roundtrip ["print q[ &<>^|\@​()!\"&<>^|\@​()! ]"]'
# at quoting.t line 91.
# got​: '["print","q[","&<>^|\@​()!&<>^|\@​()! ]"]'
# expected​: '["print q[ &<>^|\@​()!\"&<>^|\@​()! ]"]'
not ok 104 - roundtrip ["print q[ &<>^|\@​()!\"&<>^|\@​()! ]",">out"]
# Failed test 'roundtrip ["print q[ &<>^|\@​()!\"&<>^|\@​()! ]",">out"]'
# at quoting.t line 91.
# got​: '["print","q[","&<>^|\@​()!&<>^|\@​()! ] >out"]'
# expected​: '["print q[ &<>^|\@​()!\"&<>^|\@​()! ]",">out"]'
not ok 105 - roundtrip ["print q[ &<>^|\@​()!\"&<>^|\@​()! ]","%"]
# Failed test 'roundtrip ["print q[ &<>^|\@​()!\"&<>^|\@​()! ]","%"]'
# at quoting.t line 91.
# got​: '["print","q[","&<>^|\@​()!&<>^|\@​()! ] %"]'
# expected​: '["print q[ &<>^|\@​()!\"&<>^|\@​()! ]","%"]'
not ok 106 - roundtrip ["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]"]
# Failed test 'roundtrip ["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]"]'
# at quoting.t line 91.
# got​: '["print","q[","&<>^|\@​() !&<>^|\@​()","! ]"]'
# expected​: '["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]"]'
not ok 107 - roundtrip ["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]",">out"]
# Failed test 'roundtrip ["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]",">out"]'
# at quoting.t line 91.
# got​: '["print","q[","&<>^|\@​() !&<>^|\@​()","! ] >out"]'
# expected​: '["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]",">out"]'
not ok 108 - roundtrip ["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]","%"]
# Failed test 'roundtrip ["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]","%"]'
# at quoting.t line 91.
# got​: '["print","q[","&<>^|\@​() !&<>^|\@​()","! ] %"]'
# expected​: '["print q[ \"&<>^|\@​() !\"&<>^|\@​() !\" ]","%"]'
not ok 109 - roundtrip ["print q[ \"C​:\\TEST A\\\" ]"]
# Failed test 'roundtrip ["print q[ \"C​:\\TEST A\\\" ]"]'
# at quoting.t line 91.
# got​: '["print","q[","C​:\\TEST A\" ]"]'
# expected​: '["print q[ \"C​:\\TEST A\\\" ]"]'
not ok 110 - roundtrip ["print q[ \"C​:\\TEST A\\\" ]",">out"]
# Failed test 'roundtrip ["print q[ \"C​:\\TEST A\\\" ]",">out"]'
# at quoting.t line 91.
# got​: '["print","q[","C​:\\TEST A\" ] >out"]'
# expected​: '["print q[ \"C​:\\TEST A\\\" ]",">out"]'
not ok 111 - roundtrip ["print q[ \"C​:\\TEST A\\\" ]","%"]
# Failed test 'roundtrip ["print q[ \"C​:\\TEST A\\\" ]","%"]'
# at quoting.t line 91.
# got​: '["print","q[","C​:\\TEST A\" ] %"]'
# expected​: '["print q[ \"C​:\\TEST A\\\" ]","%"]'
not ok 112 - roundtrip ["print q[ \"C​:\\TEST %&^ A\\\" ]"]
# Failed test 'roundtrip ["print q[ \"C​:\\TEST %&^ A\\\" ]"]'
# at quoting.t line 91.
# got​: '["print","q[","C​:\\TEST %&^ A\" ]"]'
# expected​: '["print q[ \"C​:\\TEST %&^ A\\\" ]"]'
not ok 113 - roundtrip ["print q[ \"C​:\\TEST %&^ A\\\" ]",">out"]
# Failed test 'roundtrip ["print q[ \"C​:\\TEST %&^ A\\\" ]",">out"]'
# at quoting.t line 91.
# got​: '["print","q[","C​:\\TEST %&^ A\" ] >out"]'
# expected​: '["print q[ \"C​:\\TEST %&^ A\\\" ]",">out"]'
not ok 114 - roundtrip ["print q[ \"C​:\\TEST %&^ A\\\" ]","%"]
# Failed test 'roundtrip ["print q[ \"C​:\\TEST %&^ A\\\" ]","%"]'
# at quoting.t line 91.
# got​: '["print","q[","C​:\\TEST %&^ A\" ] %"]'
# expected​: '["print q[ \"C​:\\TEST %&^ A\\\" ]","%"]'
ok 115 - roundtrip ["\n"]
ok 116 - roundtrip ["\n",">out"]
ok 117 - roundtrip ["\n","%"]
ok 118 - roundtrip ["a\nb"]
ok 119 - roundtrip ["a\nb",">out"]
ok 120 - roundtrip ["a\nb","%"]
ok 121 - roundtrip ["a\rb"]
ok 122 - roundtrip ["a\rb",">out"]
ok 123 - roundtrip ["a\rb","%"]
ok 124 - roundtrip ["a\nb > welp"]
ok 125 - roundtrip ["a\nb > welp",">out"]
ok 126 - roundtrip ["a\nb > welp","%"]
ok 127 - roundtrip ["a > welp\n219"]
ok 128 - roundtrip ["a > welp\n219",">out"]
ok 129 - roundtrip ["a > welp\n219","%"]
not ok 130 - roundtrip ["a\"b\nc"]
# Failed test 'roundtrip ["a\"b\nc"]'
# at quoting.t line 91.
# got​: '["ab\nc"]'
# expected​: '["a\"b\nc"]'
not ok 131 - roundtrip ["a\"b\nc",">out"]
# Failed test 'roundtrip ["a\"b\nc",">out"]'
# at quoting.t line 91.
# got​: '["ab\nc >out"]'
# expected​: '["a\"b\nc",">out"]'
not ok 132 - roundtrip ["a\"b\nc","%"]
# Failed test 'roundtrip ["a\"b\nc","%"]'
# at quoting.t line 91.
# got​: '["ab\nc %"]'
# expected​: '["a\"b\nc","%"]'
ok 133 - roundtrip ["a\fb"]
ok 134 - roundtrip ["a\fb",">out"]
ok 135 - roundtrip ["a\fb","%"]
ok 136 - roundtrip ["a\13b"]
ok 137 - roundtrip ["a\13b",">out"]
ok 138 - roundtrip ["a\13b","%"]
ok 139 - roundtrip ["a\205b"]
ok 140 - roundtrip ["a\205b",">out"]
ok 141 - roundtrip ["a\205b","%"]
# Looks like you failed 66 tests of 141.

[p5pRT]

From @ikegami

On Thu, Aug 22, 2013 at 9​:34 PM, James E Keenan via RT <
perlbug-followup@​perl.org> wrote​:

On Thu Aug 22 00​:57​:49 2013, haarg wrote​:

This is a bug report for perl from haarg@​haarg.org,
generated with the help of perlbug 1.39 running under perl 5.18.0.

-----------------------------------------------------------------

Program arguments on Win32 are passed as a single string, which is
then parsed by the program on startup. While it's possible for a
program to do arbitrary parsing of the command line string, almost
all programs either use the routine provided by the MS standard C
library, or a routine with compatible rules.

For system LIST to work, perl must assemble the list into a single
string. But the strings it generates aren't always compatible with
the standard parsing rules. Attached is a test script that shows
multiple strings that aren't quoted properly.

cmd.exe uses a different set of parsing rules from the C runtime. It
isn't possible to consistently generate strings that will be handled
the same by cmd.exe and the standard C routine. This can be
problematic because in some cases, perl will re-try failed system
calls using cmd.exe.

Would you be able to provide some examples of failing cases?

C<dir> is a shell built-in, so system LIST shouldn't be able to execute it,
yet the retry logic makes it work.

perl -e"system 'dir', '.'; die $? if $?"
Volume in drive C has no label.
Volume Serial Number is 080E-1D03

Directory of C​:\Users\ikegami

2013-08-19 11​:27 PM <DIR> .
2013-08-19 11​:27 PM <DIR> ..
...

Curly bracket notation still avoids the shell, though.

perl -e"system { 'dir' } 'dir', '.'; die $? if $?"
65280 at -e line 1.