Merge pull request #18 from fglock/jline-command-line-editor

fglock · web-flow · commit 6d507d2cd8c3 · 2025-07-09T21:52:46.000+02:00
Jline command line editor
diff --git a/MILESTONES.md b/MILESTONES.md
@@ -9,6 +9,42 @@
 
 ## Completed Milestones
 
+- **v3.1.0**: Tracks Perl 5.42.0
+  - Update Perl version to `5.42.0`.
+  - Added features: `keyword_all`, `keyword_any`
+
+  - Accept input program in several ways:
+    1. **Piped input**: `echo 'print "Hello\n"' | ./jperl` - reads from pipe and executes immediately
+    2. **Interactive input**: `./jperl` - shows a prompt and waits for you to type code, then press Ctrl+D (on Unix/Linux/Mac) or Ctrl+Z (on Windows) to signal end of input
+    3. **File redirection**: `./jperl < script.pl` - reads from the file
+    4. **With arguments**: `./jperl -e 'print "Hello\n"'` or `./jperl script.pl`
+
+  - Added overload operators: `!`, `+`, `-`, `*`, `/`, `%`, `int`, `neg`, `log`, `sqrt`, `cos`, `sin`, `exp`, `abs`, `atan2`, `**`, `@{}`, `%{}`. `${}`, `&{}`, `*{}`.
+  - Subroutine prototypes are fully implemented. Added or fixed: `+`, `;`, `*`, `\@`, `\%`, `\$`, `\[@%]`.
+  - Added double quoted string escapes: `\U`, `\L`, `\u`, `\l`.
+  - Added star count (`C*`) in `pack`, `unpack`.
+  - Added operators: `read`, `tell`, `seek`, `system`, `exec`, `sysopen`, `chmod`.
+  - Added operator: `select(undef,undef,undef,$time)`.
+  - Added operator: `^^=`.
+  - Added operator: `delete`, `exists` for array indexes.
+  - Added `open` option: in-memory files.
+  - Syntax: identifiers starting with `::` are in `main` package.
+  - Added I/O layers support to `open`, `binmode`: `:raw`, `:bytes`, `:crlf`, `:utf8`, `:unix`, `:encoding()`.
+  - Add `open` support for pipe `-|`, `|-`, `ls|`, `|sort`.
+  - Added `# line` preprocessor directive.
+  - `Test::More` module: added `subtest`, `use_ok`, `require_ok`.
+  - `CORE::` operators have the same prototypes as in Perl.
+  - Added modules: `Fcntl`, `Test`, `Text::CSV`.
+  - Operator `$#` returns an lvalue.
+  - Improved autovivification handling: distinguish between contexts where undefined references should automatically create data structures versus where they should throw errors.
+  - Bugfix: fix a problem with Windows newlines and qw(). Also fixed `mkdir` in Windows.
+  - Bugfix: `-E` switch was setting strict mode.
+  - BugFix: fix calling context in operators that return list.
+  - BugFix: fix rules for overriding operators.
+  - Added Makefile.
+  - Debian package can be created with `make deb`.
+
+
 - **v3.0.0**: Performance Boost, New Modules, and Streamlined Configuration
   - Added `--upgrade` option to `Configure.pl` to upgrade dependencies.
   - Added `Dockerfile` configuration.
@@ -236,39 +272,7 @@ The following areas are currently under active development to enhance the functi
 
 ## Upcoming Milestones
 
-- **v3.0.1**: Next minor version
-  - Update Perl version to `5.42.0`.
-
-  - Accept input program in several ways:
-    1. **Piped input**: `echo 'print "Hello\n"' | ./jperl` - reads from pipe and executes immediately
-    2. **Interactive input**: `./jperl` - shows a prompt and waits for you to type code, then press Ctrl+D (on Unix/Linux/Mac) or Ctrl+Z (on Windows) to signal end of input
-    3. **File redirection**: `./jperl < script.pl` - reads from the file
-    4. **With arguments**: `./jperl -e 'print "Hello\n"'` or `./jperl script.pl`
-
-  - Added overload operators: `!`, `+`, `-`, `*`, `/`, `%`, `int`, `neg`, `log`, `sqrt`, `cos`, `sin`, `exp`, `abs`, `atan2`, `**`, `@{}`, `%{}`. `${}`, `&{}`, `*{}`.
-  - Subroutine prototypes are fully implemented. Added or fixed: `+`, `;`, `*`, `\@`, `\%`, `\$`, `\[@%]`.
-  - Added double quoted string escapes: `\U`, `\L`, `\u`, `\l`.
-  - Added star count (`C*`) in `pack`, `unpack`.
-  - Added operators: `read`, `tell`, `seek`, `system`, `exec`, `sysopen`, `chmod`.
-  - Added operator: `select(undef,undef,undef,$time)`.
-  - Added operator: `^^=`.
-  - Added operator: `delete`, `exists` for array indexes.
-  - Added `open` option: in-memory files.
-  - Syntax: identifiers starting with `::` are in `main` package.
-  - Added I/O layers support to `open`, `binmode`: `:raw`, `:bytes`, `:crlf`, `:utf8`, `:unix`, `:encoding()`.
-  - Add `open` support for pipe `-|`, `|-`, `ls|`, `|sort`.
-  - Added `# line` preprocessor directive.
-  - `Test::More` module: added `subtest`, `use_ok`, `require_ok`.
-  - `CORE::` operators have the same prototypes as in Perl.
-  - Added modules: `Fcntl`, `Test`, `Text::CSV`.
-  - Operator `$#` returns an lvalue.
-  - Improved autovivification handling: distinguish between contexts where undefined references should automatically create data structures versus where they should throw errors.
-  - Bugfix: fix a problem with Windows newlines and qw(). Also fixed `mkdir` in Windows.
-  - Bugfix: `-E` switch was setting strict mode.
-  - BugFix: fix calling context in operators that return list.
-  - BugFix: fix rules for overriding operators.
-  - Added Makefile.
-  - Debian package can be created with `make deb`.
+- **v3.2.0**: Next minor version
   - Planned release date: 2025-12-10.
 
 - Work in Progress
diff --git a/build.gradle b/build.gradle
@@ -68,6 +68,8 @@ repositories {
 // Project dependencies
 dependencies {
     // Core dependencies
+    implementation 'org.jline:jline-reader:3.30.4'
+    implementation 'org.jline:jline-terminal:3.30.4'
     implementation 'org.ow2.asm:asm:9.8'                           // ByteCode manipulation
     implementation 'org.ow2.asm:asm-util:9.8'                     // ASM utilities
     implementation 'com.ibm.icu:icu4j:77.1'                       // Unicode support
diff --git a/docs/FEATURE_MATRIX.md b/docs/FEATURE_MATRIX.md
@@ -471,7 +471,7 @@ The `:encoding()` layer supports all encodings provided by Java's `Charset.forNa
 - ✅  **subs** pragma
 - 🚧  **utf8** pragma: utf8 is always on. Disabling utf8 might work in a future version.
 - ✅  **feature** pragma
-  - ✅ Features implemented: `fc`, `say`, `current_sub`, `isa`, `state`, `try`, `bitwise`, `postderef`, `evalbytes`, `module_true`, `signatures`, `class`.
+  - ✅ Features implemented: `fc`, `say`, `current_sub`, `isa`, `state`, `try`, `bitwise`, `postderef`, `evalbytes`, `module_true`, `signatures`, `class`, `keyword_all`, `keyword_any`.
   - ❌ Features missing: `postderef_qq`, `unicode_eval`, `unicode_strings`, `defer`.
 - 🚧  **warnings** pragma
 - ❌  **version** pragma: version objects are not yet supported.
diff --git a/pom.xml b/pom.xml
@@ -17,6 +17,16 @@
     </properties>
 
     <dependencies>
+        <dependency>
+            <groupId>org.jline</groupId>
+            <artifactId>jline-reader</artifactId>
+            <version>3.30.4</version>
+        </dependency>
+        <dependency>
+            <groupId>org.jline</groupId>
+            <artifactId>jline-terminal</artifactId>
+            <version>3.30.4</version>
+        </dependency>
         <dependency>
             <groupId>org.ow2.asm</groupId>
             <artifactId>asm</artifactId>
diff --git a/src/main/java/org/perlonjava/ArgumentParser.java b/src/main/java/org/perlonjava/ArgumentParser.java
@@ -12,6 +12,12 @@
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
+import org.jline.reader.LineReader;
+import org.jline.reader.LineReaderBuilder;
+import org.jline.reader.EndOfFileException;
+import org.jline.reader.UserInterruptException;
+import org.jline.terminal.Terminal;
+import org.jline.terminal.TerminalBuilder;
 
 import static org.perlonjava.Configuration.getPerlVersionBundle;
 import static org.perlonjava.Configuration.perlVersion;
@@ -40,27 +46,58 @@ public static CompilerOptions parseArguments(String[] args) {
         // If no code was provided and no filename, try reading from stdin
         if (parsedArgs.code == null) {
             try {
-                // Try to read from stdin - this will work for pipes, redirections, and interactive input
                 StringBuilder stdinContent = new StringBuilder();
-                BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
-
-                // Check if we're reading from a pipe/redirection vs interactive terminal
                 boolean isInteractive = System.console() != null;
 
                 if (isInteractive) {
-                    // Interactive mode - prompt the user and read until EOF (Ctrl+D)
-                    System.err.println("Enter Perl code (press Ctrl+D when done):");
-                }
+                    // Interactive mode with JLine for better editing experience
+                    try {
+                        Terminal terminal = TerminalBuilder.builder()
+                                .system(true)
+                                .build();
+
+                        LineReader lineReader = LineReaderBuilder.builder()
+                                .terminal(terminal)
+                                .build();
+
+                        System.err.println("Enter Perl code (press Ctrl+D when done, or type 'exit' to quit):");
+                        System.err.println("Use arrow keys to navigate, Ctrl+A/E for home/end");
+
+                        String line;
+                        while (true) {
+                            try {
+                                line = lineReader.readLine("> ");
+                                if (line != null) {
+                                    if ("exit".equals(line.trim())) {
+                                        break;
+                                    }
+                                    stdinContent.append(line).append("\n");
+                                }
+                            } catch (EndOfFileException e) {
+                                // User pressed Ctrl+D
+                                break;
+                            } catch (UserInterruptException e) {
+                                // User pressed Ctrl+C
+                                System.err.println("\nInterrupted. Use 'exit' or Ctrl+D to quit.");
+                                break;
+                            }
+                        }
 
-                // Read from stdin regardless of whether it's interactive or not
-                String line;
-                while ((line = reader.readLine()) != null) {
-                    stdinContent.append(line).append("\n");
+                        terminal.close();
+                    } catch (Exception e) {
+                        // Fall back to basic readline if JLine fails
+                        System.err.println("Enhanced editing not available, falling back to basic mode.");
+                        System.err.println("Enter Perl code (press Ctrl+D when done):");
+                        fallbackReadlines(stdinContent);
+                    }
+                } else {
+                    // Non-interactive mode (pipes, redirections)
+                    fallbackReadlines(stdinContent);
                 }
 
                 if (stdinContent.length() > 0) {
                     parsedArgs.code = stdinContent.toString();
-                    parsedArgs.fileName = "-"; // Indicate that code came from stdin
+                    parsedArgs.fileName = "-";
                 }
             } catch (IOException e) {
                 // If we can't read from stdin, continue with normal error handling
@@ -72,6 +109,14 @@ public static CompilerOptions parseArguments(String[] args) {
         return parsedArgs;
     }
 
+    private static void fallbackReadlines(StringBuilder stdinContent) throws IOException {
+        BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
+        String line;
+        while ((line = reader.readLine()) != null) {
+            stdinContent.append(line).append("\n");
+        }
+    }
+    
     /**
      * Processes the command-line arguments, distinguishing between switch and non-switch arguments.
      *
diff --git a/src/main/java/org/perlonjava/codegen/EmitBinaryOperatorNode.java b/src/main/java/org/perlonjava/codegen/EmitBinaryOperatorNode.java
@@ -48,7 +48,7 @@ public static void emitBinaryOperatorNode(EmitterVisitor emitterVisitor, BinaryO
             case "push", "unshift" -> EmitOperator.handlePushOperator(emitterVisitor, node);
 
             // Higher-order functions
-            case "map", "sort", "grep" -> EmitOperator.handleMapOperator(emitterVisitor, node);
+            case "map", "sort", "grep", "all", "any" -> EmitOperator.handleMapOperator(emitterVisitor, node);
 
             // I/O operations
             case "eof", "open", "printf", "print", "say" ->
@@ -97,7 +97,7 @@ public static void emitBinaryOperatorNode(EmitterVisitor emitterVisitor, BinaryO
                             OperatorHandler.get(node.operator));
 
             default -> throw new PerlCompilerException(node.tokenIndex,
-                    "Unexpected infix operator: " + node.operator, emitterVisitor.ctx.errorUtil);
+                    "Not implemented operator: " + node.operator, emitterVisitor.ctx.errorUtil);
         }
     }
 }
diff --git a/src/main/java/org/perlonjava/operators/ListOperators.java b/src/main/java/org/perlonjava/operators/ListOperators.java
@@ -7,6 +7,8 @@
 import java.util.List;
 
 import static org.perlonjava.runtime.GlobalVariable.getGlobalVariable;
+import static org.perlonjava.runtime.RuntimeScalarCache.scalarFalse;
+import static org.perlonjava.runtime.RuntimeScalarCache.scalarTrue;
 
 public class ListOperators {
     /**
@@ -116,16 +118,14 @@ public static RuntimeList sort(RuntimeList runtimeList, RuntimeScalar perlCompar
      * @throws RuntimeException If the Perl filter subroutine throws an exception.
      */
     public static RuntimeList grep(RuntimeList runtimeList, RuntimeScalar perlFilterClosure, int ctx) {
-        RuntimeArray array = runtimeList.getArrayOfAlias();
-
         // Create a new list to hold the filtered elements
         List<RuntimeBaseEntity> filteredElements = new ArrayList<>();
 
         RuntimeScalar var_ = getGlobalVariable("main::_");
         RuntimeArray filterArgs = new RuntimeArray();
 
         // Iterate over each element in the current RuntimeArray
-        for (RuntimeScalar element : array.elements) {
+        for (RuntimeScalar element : runtimeList) {
             try {
                 // Create $_ argument for the filter subroutine
                 var_.set(element);
@@ -158,4 +158,70 @@ public static RuntimeList grep(RuntimeList runtimeList, RuntimeScalar perlFilter
             return filteredList;
         }
     }
+
+    /**
+     * Check the elements of this RuntimeArray using a Perl subroutine.
+     *
+     * @param runtimeList
+     * @param perlFilterClosure A RuntimeScalar representing the Perl filter subroutine.
+     * @return A new RuntimeScalar boolean true if all elements match the filter criteria.
+     * @throws RuntimeException If the Perl filter subroutine throws an exception.
+     */
+    public static RuntimeList all(RuntimeList runtimeList, RuntimeScalar perlFilterClosure, int ctx) {
+        RuntimeScalar var_ = getGlobalVariable("main::_");
+        RuntimeArray filterArgs = new RuntimeArray();
+
+        // Iterate over each element in the current RuntimeArray
+        for (RuntimeScalar element : runtimeList) {
+            try {
+                // Create $_ argument for the filter subroutine
+                var_.set(element);
+
+                // Apply the Perl filter subroutine with the argument
+                RuntimeList result = RuntimeCode.apply(perlFilterClosure, filterArgs, RuntimeContextType.SCALAR);
+
+                // Check the result of the filter subroutine
+                if (!result.getFirst().getBoolean()) {
+                    return scalarFalse.getList();
+                }
+            } catch (Exception e) {
+                // Wrap any exceptions thrown by the filter subroutine in a RuntimeException
+                throw new RuntimeException(e);
+            }
+        }
+        return scalarTrue.getList();
+    }
+
+    /**
+     * Check the elements of this RuntimeArray using a Perl subroutine.
+     *
+     * @param runtimeList
+     * @param perlFilterClosure A RuntimeScalar representing the Perl filter subroutine.
+     * @return A new RuntimeScalar boolean true if any elements match the filter criteria.
+     * @throws RuntimeException If the Perl filter subroutine throws an exception.
+     */
+    public static RuntimeList any(RuntimeList runtimeList, RuntimeScalar perlFilterClosure, int ctx) {
+        RuntimeScalar var_ = getGlobalVariable("main::_");
+        RuntimeArray filterArgs = new RuntimeArray();
+
+        // Iterate over each element in the current RuntimeArray
+        for (RuntimeScalar element : runtimeList) {
+            try {
+                // Create $_ argument for the filter subroutine
+                var_.set(element);
+
+                // Apply the Perl filter subroutine with the argument
+                RuntimeList result = RuntimeCode.apply(perlFilterClosure, filterArgs, RuntimeContextType.SCALAR);
+
+                // Check the result of the filter subroutine
+                if (result.getFirst().getBoolean()) {
+                    return scalarTrue.getList();
+                }
+            } catch (Exception e) {
+                // Wrap any exceptions thrown by the filter subroutine in a RuntimeException
+                throw new RuntimeException(e);
+            }
+        }
+        return scalarFalse.getList();
+    }
 }
diff --git a/src/main/java/org/perlonjava/operators/OperatorHandler.java b/src/main/java/org/perlonjava/operators/OperatorHandler.java
@@ -181,6 +181,12 @@ public class OperatorHandler {
         put("sort", "sort",
                 "org/perlonjava/operators/ListOperators",
                 "(Lorg/perlonjava/runtime/RuntimeList;Lorg/perlonjava/runtime/RuntimeScalar;Ljava/lang/String;)Lorg/perlonjava/runtime/RuntimeList;");
+        put("all", "all",
+                "org/perlonjava/operators/ListOperators",
+                "(Lorg/perlonjava/runtime/RuntimeList;Lorg/perlonjava/runtime/RuntimeScalar;I)Lorg/perlonjava/runtime/RuntimeList;");
+        put("any", "any",
+                "org/perlonjava/operators/ListOperators",
+                "(Lorg/perlonjava/runtime/RuntimeList;Lorg/perlonjava/runtime/RuntimeScalar;I)Lorg/perlonjava/runtime/RuntimeList;");
 
         operatorHandlers.put("scalar",
                 new OperatorHandler("org/perlonjava/runtime/RuntimeDataProvider",
diff --git a/src/main/java/org/perlonjava/parser/CoreOperatorResolver.java b/src/main/java/org/perlonjava/parser/CoreOperatorResolver.java
@@ -170,7 +170,7 @@ public static Node parseCoreOperator(Parser parser, LexerToken token, int startI
             case "sort":
                 // Handle 'sort' operator
                 return OperatorParser.parseSort(parser, token);
-            case "map", "grep":
+            case "map", "grep", "all", "any":
                 // Handle 'map' and 'grep' operators
                 return OperatorParser.parseMapGrep(parser, token);
             case "pack":
diff --git a/src/main/java/org/perlonjava/parser/ParsePrimary.java b/src/main/java/org/perlonjava/parser/ParsePrimary.java
@@ -118,6 +118,10 @@ private static Node parseIdentifier(Parser parser, int startIndex, LexerToken to
             // Check if the operator is enabled in the current scope
             // Some operators require specific features to be enabled
             operatorEnabled = switch (operator) {
+                case "all" ->
+                        parser.ctx.symbolTable.isFeatureCategoryEnabled("keyword_all");
+                case "any" ->
+                        parser.ctx.symbolTable.isFeatureCategoryEnabled("keyword_any");
                 case "say", "fc", "state", "evalbytes" ->
                         parser.ctx.symbolTable.isFeatureCategoryEnabled(operator);
                 case "__SUB__" ->
diff --git a/src/main/java/org/perlonjava/runtime/FeatureFlags.java b/src/main/java/org/perlonjava/runtime/FeatureFlags.java
@@ -18,7 +18,7 @@ public class FeatureFlags {
 
     static {
         // Initialize the hierarchy of feature bundles
-        featureBundles.put(":default", new String[]{"indirect", "multidimensional", "bareword_filehandles"});
+        featureBundles.put(":default", new String[]{"indirect", "multidimensional", "bareword_filehandles", "apostrophe_as_package_separator", "smartmatch"});
         featureBundles.put(":5.10", new String[]{"bareword_filehandles", "indirect", "multidimensional", "say", "state", "switch"});
         featureBundles.put(":5.12", new String[]{"bareword_filehandles", "indirect", "multidimensional", "say", "state", "switch", "unicode_strings"});
         featureBundles.put(":5.14", new String[]{"bareword_filehandles", "indirect", "multidimensional", "say", "state", "switch", "unicode_strings"});
@@ -38,6 +38,8 @@ public class FeatureFlags {
         featureBundles.put(":5.42", new String[]{"bitwise", "current_sub", "evalbytes", "fc", "isa", "module_true", "postderef_qq", "say", "signatures", "state", "try", "unicode_eval", "unicode_strings"});
         // Not bundled:
         featureBundles.put("postderef", new String[]{"postderef"});
+        featureBundles.put("keyword_all", new String[]{"keyword_all"});
+        featureBundles.put("keyword_any", new String[]{"keyword_any"});
     }
 
     /**
diff --git a/src/main/java/org/perlonjava/runtime/RuntimeList.java b/src/main/java/org/perlonjava/runtime/RuntimeList.java

Original file line number	Diff line number	Diff line change
`@@ -48,7 +48,7 @@ public static void emitBinaryOperatorNode(EmitterVisitor emitterVisitor, BinaryO`
`48`	`48`	`case "push", "unshift" -> EmitOperator.handlePushOperator(emitterVisitor, node);`
`49`	`49`
`50`	`50`	`// Higher-order functions`
`51`		`- case "map", "sort", "grep" -> EmitOperator.handleMapOperator(emitterVisitor, node);`
	`51`	`+ case "map", "sort", "grep", "all", "any" -> EmitOperator.handleMapOperator(emitterVisitor, node);`
`52`	`52`
`53`	`53`	`// I/O operations`
`54`	`54`	`case "eof", "open", "printf", "print", "say" ->`
`@@ -97,7 +97,7 @@ public static void emitBinaryOperatorNode(EmitterVisitor emitterVisitor, BinaryO`
`97`	`97`	`OperatorHandler.get(node.operator));`
`98`	`98`
`99`	`99`	`default -> throw new PerlCompilerException(node.tokenIndex,`
`100`		`- "Unexpected infix operator: " + node.operator, emitterVisitor.ctx.errorUtil);`
	`100`	`+ "Not implemented operator: " + node.operator, emitterVisitor.ctx.errorUtil);`
`101`	`101`	`}`
`102`	`102`	`}`
`103`	`103`	`}`