[chore] Drop py27 support and update lark dependency from lark-parser to lark>=1.3.1

.github/workflows/pythonpackage.yml

@@ -1,7 +1,7 @@
 # This workflow will install Python dependencies, run tests and lint with a variety of Python versions
 # For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
 
-name: Build 3.X
+name: Build and Test
 
 on:
   push:
@@ -15,19 +15,20 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.7]
+        python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
       fail-fast: false
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v5
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install wheel
+        python -m pip install wheel setuptools
+        python -m pip install -e ".[lint,test]"
     - name: Lint with flake8
       run: |
         python setup.py -q lint

CHANGELOG.md

@@ -1,6 +1,27 @@
 # Event Query Language - Changelog
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+# Version 1.0.0
+
+ _Released 2025-11-17_
+
+### Changed
+
+* Updated `lark-parser` dependency to `lark>=1.3.1` (migrated from deprecated `lark-parser` package to `lark`)
+* Updated GitHub Actions workflows to use Python 3.8+ and newer action versions (`actions/checkout@v5`, `actions/setup-python@v5`)
+* Fixed compatibility issues with Lark 1.3.1:
+  * Fixed parsing of macros with empty parameter lists (e.g., `macro TRUE()`)
+  * Fixed parsing of pipes with no arguments (e.g., `| count`)
+  * Fixed `Schema.current()` to always return a valid Schema object
+* Simplified dependencies by removing Python 2.7 and Python < 3.8 compatibility code
+* Removed Python 2.7 compatibility comments and code from source files
+* Updated documentation to reflect Python 3.8+ requirement
+
+### Removed
+
+* **BREAKING**: Dropped support for Python 2.7 and Python < 3.8. The minimum required Python version is now 3.8.
+* Removed Python 2.7 GitHub Actions workflow (`.github/workflows/pythonpackage27.yml`)
+
 # Version 0.9.19
 
  _Released 2023-10-31_

CONTRIBUTING.md

@@ -30,9 +30,9 @@ Contributing to EQL is a simple five-step process facilitated by Git:
 * There is plenty of literature and resources out there to help you. A great place to start is [GitHub guides](https://guides.github.com/).
 
 ## Ways to contribute
-  
+
 ### Bug Fixes
-Bug fixes are a natural area to contribute. We only ask that you please use the [bug report issue](https://github.com/endgameinc/eql/issues) to track the bug. Please elaborate on how to reproduce the bug and what behavior you would have expected. Compatibility is a priority for EQL, so be sure to capture information about your operating system and version of python. 
+Bug fixes are a natural area to contribute. We only ask that you please use the [bug report issue](https://github.com/endgameinc/eql/issues) to track the bug. Please elaborate on how to reproduce the bug and what behavior you would have expected. Compatibility is a priority for EQL, so be sure to capture information about your operating system and version of python.
 
 ### Language or Engine Changes
 For any changes within the language or the evaluation engine, propose your changes in a *Feature Request* issue to start a discussion. For new functionality function, be mindful of handling different edge cases, acceptable input, etc. We are happy to collaborate on such topics and encourage you to share ideas.
@@ -49,7 +49,7 @@ Anyone is encouraged to make a PR for open issues that have a clear path forward
 * Include end-to-end tests by updating the test [data](eql/etc/test_data.json) and [queries](eql/etc/test_queries.toml). These are used as the gold standard of expected behavior, and the queries should have a list of the serial_event_id of the events, in the expected order.
 
 ### CLI
-Finally, the CLI is an area we are always looking to expand. This may include new input file types, new processing features, new tables, etc. Some shell functionality, like tab completions ANSI coloring, and history often varies across different operating systems. If possible, please test new functionality across a few different operating systems if you have access, and Python 2.7 and 3.6+. If you find any unusual behavior in the shell related to compatibility, please let us know in an issue.
+Finally, the CLI is an area we are always looking to expand. This may include new input file types, new processing features, new tables, etc. Some shell functionality, like tab completions ANSI coloring, and history often varies across different operating systems. If possible, please test new functionality across a few different operating systems if you have access, and Python 3.8+. If you find any unusual behavior in the shell related to compatibility, please let us know in an issue.
 
 ## Resources
 See the [resources page](https://eql.readthedocs.io/en/latest/resources.html) on ReadTheDocs for a full list of resources

README.md

@@ -13,7 +13,7 @@ Since Endgame [joined forced with Elastic](https://www.elastic.co/blog/endgame-j
 
 # Getting Started
 
-The EQL module current supports Python 2.7 and 3.5+. Assuming a supported Python version is installed, run the command:
+The EQL module requires Python 3.8 or higher. Assuming a supported Python version is installed, run the command:
 
 ```console
 $ pip install eql
@@ -23,7 +23,7 @@ If Python is configured and already in the PATH, then ``eql`` will be readily av
 
 ```console
 $ eql --version
-eql 0.9
+eql 0.9.20
 ```
 
 From there, try a [sample json file](docs/_static/example.json) and test it with EQL.

docs/_static/eql-crash-course.slides.html

@@ -9181,15 +9181,15 @@
 }
 /* Flexible box model classes */
 /* Taken from Alex Russell http://infrequently.org/2009/08/css-3-progress/ */
-/* This file is a compatability layer.  It allows the usage of flexible box 
+/* This file is a compatability layer.  It allows the usage of flexible box
 model layouts accross multiple browsers, including older browsers.  The newest,
 universal implementation of the flexible box model is used when available (see
-`Modern browsers` comments below).  Browsers that are known to implement this 
+`Modern browsers` comments below).  Browsers that are known to implement this
 new spec completely include:
 
     Firefox 28.0+
     Chrome 29.0+
-    Internet Explorer 11+ 
+    Internet Explorer 11+
     Opera 17.0+
 
 Browsers not listed, including Safari, are supported via the styling under the
@@ -12571,7 +12571,7 @@
   background: #f7f7f7;
   border-top: 1px solid #cfcfcf;
   border-bottom: 1px solid #cfcfcf;
-  /* This injects handle bars (a short, wide = symbol) for 
+  /* This injects handle bars (a short, wide = symbol) for
         the resize handle. */
 }
 div#pager .ui-resizable-handle::after {
@@ -13070,7 +13070,7 @@
 .highlight .il { color: #666666 } /* Literal.Number.Integer.Long */
     </style>
 <style type="text/css">
-    
+
 /* Temporary definitions which will become obsolete with Notebook release 5.0 */
 .ansi-black-fg { color: #3E424D; }
 .ansi-black-bg { background-color: #3E424D; }
@@ -13253,7 +13253,7 @@ <h1 id="Event-Query-Language">Event Query Language<a class="anchor-link" href="#
 </div><div class="inner_cell">
 <div class="text_cell_render border-box-sizing rendered_html">
 <h2 id="Getting-Started">Getting Started<a class="anchor-link" href="#Getting-Started">&#182;</a></h2><p><a href="https://eql.readthedocs.io/en/latest/index.html#getting-started">https://eql.readthedocs.io/en/latest/index.html#getting-started</a></p>
-<p>Requires Python (confirmed with 2.7 and 3.5+)</p>
+<p>Requires Python 3.8+</p>
 <div class="highlight"><pre><span></span><span class="gp">$</span> pip install eql
 
 <span class="go">Collecting eql</span>

docs/cli.rst

@@ -13,7 +13,7 @@ the console. First install Python and then use ``pip`` to install EQL.
     $ pip install eql
 
 
-For the optimal shell experience, use Python 3.6+ and install the optional dependencies for EQL:
+For the optimal shell experience, use Python 3.8+ and install the optional dependencies for EQL:
 
 .. code-block:: console
 
@@ -27,12 +27,6 @@ Type ``help`` within the shell to get a list of commands and ``exit`` when finis
 .. |asciicast| image:: https://asciinema.org/a/259453.svg
    :target: https://asciinema.org/a/259453
 
-.. note::
-
-    In Python 2.7, the argument parsing is a little different. Instead of running ``eql`` directly
-    to invoke the interactive shell, run ``eql shell``.
-
-
 In addition, the ``query`` command within EQL will stream over `JSON`_, and
 output as matches are found. An input file can be provided with ``-f`` in JSON
 or as lines of JSON (``.jsonl``). Lines of JSON can also be processed as streams from stdin.

docs/index.rst

@@ -14,9 +14,9 @@ EQL also has a preprocessor that can perform parse and translation time evaluati
 
 .. note::
    This documentation is about EQL for Elastic Endgame. Several syntax changes were made in Elasticsearch to `bring Event Query Language to the Elastic Stack <https://www.elastic.co/guide/en/elasticsearch/reference/current/eql.html>`_. The existing Python EQL implementation remains unchanged, but please keep the below differences in mind when switching between the two different versions of EQL.
-   
+
    In the Elastic Stack:
-   
+
    - Most operators are now case-sensitive. For example, ``process_name == "cmd.exe"`` is no longer equivalent to ``process_name == "Cmd.exe"``.
    - Functions are now case-sensitive. To use the case-insensitive variant, use ``~``, such as ``endsWith~(process_name, ".exe")``.
    - For case-insensitive equality comparisons, use the ``:`` operator. For example, ``process_name : "cmd.exe"`` is equivalent to ``process_name : "Cmd.exe"``.
@@ -27,12 +27,12 @@ EQL also has a preprocessor that can perform parse and translation time evaluati
    - ``=`` can no longer be substituted for the ``==`` operator.
    - ``'`` strings are no longer supported. Use ``"""`` or ``"`` to represent strings.
    - ``?"`` and ``?'`` no longer indicate raw strings. Use the ``"""..."""`` syntax instead.
-   
+
    For more details, see the `limitations <https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-syntax.html#eql-syntax-limitations>`_ section of the Elasticsearch EQL documentation.
 
 Getting Started
 ^^^^^^^^^^^^^^^^
-The EQL module current supports Python 2.7 and 3.5+. Assuming a supported Python version is installed, run the command:
+The EQL module requires Python 3.8 or higher. Assuming a supported Python version is installed, run the command:
 
 .. code-block:: console
 
@@ -43,7 +43,7 @@ If Python is configured and already in the PATH, then ``eql`` will be readily av
 .. code-block:: console
 
      $ eql --version
-     eql 0.9
+     eql 1.0.0
 
 From there, try a :download:`sample json file <_static/example.json>` and test it with EQL.
 

eql/__init__.py

@@ -66,7 +66,7 @@
     Walker,
 )
 
-__version__ = '0.9.19'
+__version__ = '1.0.0'
 __all__ = (
     "__version__",
     "AnalyticOutput",

eql/ast.py

@@ -139,11 +139,7 @@ def __unicode__(self):
 
     def __str__(self):
         """Render the AST back as a valid EQL string."""
-        unicoded = self.__unicode__()
-        # Python 2.7
-        if not isinstance(unicoded, str):
-            unicoded = unicoded.encode('utf-8')
-        return unicoded
+        return self.__unicode__()
 
 
 # noinspection PyAbstractClass

eql/parser.py

@@ -96,6 +96,50 @@ def __getitem__(self, item):
     def child_trees(self):
         return [child for child in self.children if isinstance(child, KvTree)]
 
+    @property
+    def line(self):
+        """Get line number from meta or fallback to first token."""
+        if hasattr(self, 'meta') and self.meta and hasattr(self.meta, 'line'):
+            return self.meta.line
+        # Fallback: get line from first token child
+        for child in self.children:
+            if isinstance(child, Token) and hasattr(child, 'line'):
+                return child.line
+        return 1
+
+    @property
+    def end_line(self):
+        """Get end line number from meta or fallback to last token."""
+        if hasattr(self, 'meta') and self.meta and hasattr(self.meta, 'end_line'):
+            return self.meta.end_line
+        # Fallback: get end_line from last token child
+        for child in reversed(self.children):
+            if isinstance(child, Token) and hasattr(child, 'end_line'):
+                return child.end_line
+        return self.line
+
+    @property
+    def column(self):
+        """Get column number from meta or fallback to first token."""
+        if hasattr(self, 'meta') and self.meta and hasattr(self.meta, 'column'):
+            return self.meta.column
+        # Fallback: get column from first token child
+        for child in self.children:
+            if isinstance(child, Token) and hasattr(child, 'column'):
+                return child.column
+        return 1
+
+    @property
+    def end_column(self):
+        """Get end column number from meta or fallback to last token."""
+        if hasattr(self, 'meta') and self.meta and hasattr(self.meta, 'end_column'):
+            return self.meta.end_column
+        # Fallback: get end_column from last token child
+        for child in reversed(self.children):
+            if isinstance(child, Token) and hasattr(child, 'end_column'):
+                return child.end_column
+        return self.column
+
 
 class LarkToEQL(Interpreter):
     """Walker of Lark tree to convert it into a EQL AST."""
@@ -978,12 +1022,21 @@ def pipe(self, node):
         if pipe_cls is None or pipe_name not in self._allowed_pipes:
             raise self._error(node["name"], "Unknown pipe {NAME}")
 
+        # Handle arguments - Lark 1.3.1 includes None in children for empty optional rules
+        # The grammar is: "|" name [single_atom single_atom+ | expressions]
         args = []
-
         if node["expressions"]:
-            args = self.visit(node["expressions"])
+            # expressions node exists, visit it
+            args = self.visit(node["expressions"]) or []
+            # Filter out None values (Lark 1.3.1 includes None for empty optional rules)
+            args = [arg for arg in args if arg is not None]
         elif len(node.children) > 1:
-            args = self.visit(node.children[1:])
+            # Handle single_atom single_atom+ case - filter None before visiting
+            atom_nodes = [child for child in node.children[1:] if child is not None]
+            if atom_nodes:
+                args = self.visit(atom_nodes) or []
+                # Filter out None values
+                args = [arg for arg in args if arg is not None]
 
         self.validate_signature(node, pipe_cls, args)
         self._pipe_schemas = pipe_cls.output_schemas(args, self._pipe_schemas)
@@ -1286,15 +1339,37 @@ def definitions(self, node):
     def macro(self, node):
         """Callback function to walk the AST."""
         name = self.visit(node.children[0])
-        params = self.visit(node.children[1:-1])
+        # Extract name string from NodeInfo if needed
+        if hasattr(name, 'node'):
+            name = self.name(node.children[0])
+        elif not isinstance(name, str):
+            name = str(name)
+
+        # Handle parameter list - it might be empty for macros with no parameters
+        # The grammar is: "macro" name "(" [name ("," name)*] ")" expr
+        # Lark 1.3.1 includes None in children for empty optional rules
+        # So children are: [0]=name, [1]=param (or None), [2]=param (or None), ..., [-1]=expr
+        params = []
+        if len(node.children) > 2:
+            # Filter out None values before visiting (Lark 1.3.1 includes None for empty optional rules)
+            param_nodes = [child for child in node.children[1:-1] if child is not None]
+            if param_nodes:
+                param_list_result = self.visit(param_nodes)
+                if isinstance(param_list_result, list):
+                    # Extract parameter names - visit returns strings for name nodes
+                    params = [p for p in param_list_result if isinstance(p, str)]
+                elif isinstance(param_list_result, str):
+                    params = [param_list_result]
+
         body = self.visit(node.children[-1])
         definition = ast.Macro(name, params, body.node)
         self.new_preprocessor.add_definition(definition)
         return definition
 
     def constant(self, node):
         """Callback function to walk the AST."""
-        name = self.visit(node["name"])
+        name_node = node["name"]
+        name = self.name(name_node) if hasattr(name_node, 'children') else str(name_node)
         value = self.visit(node["literal"])
         definition = ast.Constant(name, value.node)
         self.new_preprocessor.add_definition(definition)

eql/schema.py

@@ -241,7 +241,11 @@ def current(cls):  # type: () -> Schema
         """Retrieve the active schema or the default."""
         current = cls.read_stack("schema")
         if current is None:
-            return cls.default()
+            default = cls.default()
+            if default is None:
+                # Ensure we always return a schema, never None
+                return EMPTY_SCHEMA
+            return default
         return current
 
     @classmethod