Introduce `:rigid` parsing mode #1991

karreiro · 2025-09-30T12:33:52Z

This PR introduces the :rigid parsing mode, as the existing :strict parser can still be too forgiving in certain cases.

For example:

<table>
  {% tablerow i in (1..10) limit: foo=>bar %}
    {{ i }}
  {% endtablerow %}
</table>

With this change, parsing for the case/when, cycle, for, if/else/elsif, include, render, and tablerow tags is now safe. To clarify where things stand, we now support these parsing modes:

- :rigid  # Raises a SyntaxError when invalid syntax is used in all tags
- :strict # Raises a SyntaxError when invalid syntax is used in some tags
- :warn   # Adds strict errors to template.errors but continues normal execution
- :lax    # The default mode, accepts almost anything.

While we may consider removing some of these parsing modes in the next major release, for now we're preserving the previous strict behavior for backward compatibility.

lib/liquid/environment.rb

lib/liquid/parse_context.rb

lib/liquid/tags/render.rb

… mode

- `respond_to?` was returning `false` in the parser switcher because `rigid_parse` was private It was working before because `parse_context` was doing the double-parsing thing, but when we removed that, this test fairly started breaking

…switcher: - Fixed NoMethod error with .peek (using look instead) - Add friendlier error message when {% cycle %}

@Operators

e.g. sometimes you want to only accept strings | lookups. {% render snippetName %} for example. snippetName is a string right now. We don't want safe_parse_expression because this would allow snippetName to be a number, a boolean, etc. But we still want to strict parse this. So what we'll do is use parse_expression(string, safe: true), this is an optional opt-in to say "I know what I'm doing". Usually that's because you're using the output of Parser#something as the input of parse_expression. It is true that Parser#expression is subset of Expression.parse, it is not true of the opposite (e.g. Expression.parse doesn't care about .5 and happily parses that as a global lookup of the variable named "5", Parser#expression throws for that.) diff --git a/lib/liquid/condition.rb b/lib/liquid/condition.rb index e5c321d..9ab350f 100644 --- a/lib/liquid/condition.rb +++ b/lib/liquid/condition.rb @@ -48,8 +48,8 @@ module Liquid @@Operators end - def self.parse_expression(parse_context, markup) - @@method_literals[markup] || parse_context.parse_expression(markup) + def self.parse_expression(parse_context, markup, safe: false) + @@method_literals[markup] || parse_context.parse_expression(markup, safe: safe) end attr_reader :attachment, :child_condition diff --git a/lib/liquid/parse_context.rb b/lib/liquid/parse_context.rb index 1c59fe4..82cf576 100644 --- a/lib/liquid/parse_context.rb +++ b/lib/liquid/parse_context.rb @@ -51,13 +51,13 @@ module Liquid end def safe_parse_expression(parser) - Expression.safe_parse(parser) + Expression.safe_parse(parser, @string_scanner, @expression_cache) end - def parse_expression(markup) + def parse_expression(markup, safe: false) # todo(guilherme): remove this once rigid mode is fully using safe_parse_expression - # raise Liquid::InternalError, "parse_expression is not supported in rigid mode" if @error_mode == :rigid - puts("🚨 parse_expression used in rigid mode") if @error_mode == :rigid + # raise Liquid::InternalError, "parse_expression is not supported in rigid mode" if !safe && @error_mode == :rigid + puts("🚨 parse_expression used in rigid mode") if !safe && @error_mode == :rigid Expression.parse(markup, @string_scanner, @expression_cache) end diff --git a/lib/liquid/tag.rb b/lib/liquid/tag.rb index 656d2e4..374ee51 100644 --- a/lib/liquid/tag.rb +++ b/lib/liquid/tag.rb @@ -72,8 +72,8 @@ module Liquid parse_context.safe_parse_expression(parser) end - def parse_expression(markup) - parse_context.parse_expression(markup) + def parse_expression(markup, safe: false) + parse_context.parse_expression(markup, safe: safe) end end end diff --git a/lib/liquid/tags/for.rb b/lib/liquid/tags/for.rb index c2be5db..3182983 100644 --- a/lib/liquid/tags/for.rb +++ b/lib/liquid/tags/for.rb @@ -93,7 +93,7 @@ module Liquid raise SyntaxError, options[:locale].t("errors.syntax.for_invalid_in") unless p.id?('in') collection_name = p.expression - @collection_name = parse_expression(collection_name) + @collection_name = parse_expression(collection_name, safe: true) @name = "#{@variable_name}-#{collection_name}" @reversed = p.id?('reversed') diff --git a/lib/liquid/tags/if.rb b/lib/liquid/tags/if.rb index 342374f..e25d625 100644 --- a/lib/liquid/tags/if.rb +++ b/lib/liquid/tags/if.rb @@ -81,8 +81,8 @@ module Liquid block.attach(new_body) end - def parse_expression(markup) - Condition.parse_expression(parse_context, markup) + def parse_expression(markup, safe: false) + Condition.parse_expression(parse_context, markup, safe: safe) end def lax_parse(markup) @@ -124,9 +124,9 @@ module Liquid end def parse_comparison(p) - a = parse_expression(p.expression) + a = parse_expression(p.expression, safe: true) if (op = p.consume?(:comparison)) - b = parse_expression(p.expression) + b = parse_expression(p.expression, safe: true) Condition.new(a, op, b) else Condition.new(a) diff --git a/lib/liquid/tags/include.rb b/lib/liquid/tags/include.rb index 6cdbfd6..b72a235 100644 --- a/lib/liquid/tags/include.rb +++ b/lib/liquid/tags/include.rb @@ -87,10 +87,11 @@ module Liquid def rigid_parse(markup) p = @parse_context.new_parser(markup) - template_name = p.expression + @template_name_expr = safe_parse_expression(p) with_or_for = p.id?("for") || p.id?("with") || nil + @variable_name_expr = nil if with_or_for - variable_name = p.expression + @variable_name_expr = parse_expression(p.consume(:id), safe: true) end alias_name = nil @@ -98,8 +99,6 @@ module Liquid alias_name = p.consume(:id) end - @template_name_expr = parse_expression(template_name) - @variable_name_expr = variable_name ? parse_expression(variable_name) : nil @alias_name = alias_name # optional comma @@ -109,7 +108,7 @@ module Liquid while p.look(:id) key = p.consume p.consume(:colon) - @attributes[key] = parse_expression(p.expression) + @attributes[key] = safe_parse_expression(p) p.consume?(:comma) # optional comma end end diff --git a/lib/liquid/tags/render.rb b/lib/liquid/tags/render.rb index 89c1106..4f716b2 100644 --- a/lib/liquid/tags/render.rb +++ b/lib/liquid/tags/render.rb @@ -88,10 +88,11 @@ module Liquid def rigid_parse(markup) p = @parse_context.new_parser(markup) - template_name = rigid_template_name(p) + @template_name_expr = parse_expression(rigid_template_name(p), safe: true) + @variable_name_expr = nil with_or_for = p.id?("for") || p.id?("with") || nil if with_or_for - variable_name = p.expression + @variable_name_expr = safe_parse_expression(p) end alias_name = nil @@ -99,8 +100,6 @@ module Liquid alias_name = p.consume(:id) end - @template_name_expr = parse_expression(template_name) - @variable_name_expr = variable_name ? parse_expression(variable_name) : nil @alias_name = alias_name @is_for_loop = (with_or_for == FOR) @@ -111,7 +110,7 @@ module Liquid while p.look(:id) key = p.consume p.consume(:colon) - @attributes[key] = parse_expression(p.expression) + @attributes[key] = safe_parse_expression(p) p.consume?(:comma) # optional comma end end diff --git a/lib/liquid/variable.rb b/lib/liquid/variable.rb index 2095706..a3623bc 100644 --- a/lib/liquid/variable.rb +++ b/lib/liquid/variable.rb @@ -65,11 +65,11 @@ module Liquid return if p.look(:end_of_string) - @name = parse_context.parse_expression(p.expression) + @name = parse_context.safe_parse_expression(p) while p.consume?(:pipe) filtername = p.consume(:id) filterargs = p.consume?(:colon) ? parse_filterargs(p) : Const::EMPTY_ARRAY - @filters << parse_filter_expressions(filtername, filterargs) + @filters << parse_filter_expressions(filtername, filterargs, safe: true) end p.consume(:end_of_string) end @@ -122,15 +122,15 @@ module Liquid private - def parse_filter_expressions(filter_name, unparsed_args) + def parse_filter_expressions(filter_name, unparsed_args, safe: false) filter_args = [] keyword_args = nil unparsed_args.each do |a| - if (matches = a.match(JustTagAttributes)) + if (matches = a.match(JustTagAttributes)) # we'll need to fix this keyword_args ||= {} - keyword_args[matches[1]] = parse_context.parse_expression(matches[2]) + keyword_args[matches[1]] = parse_context.parse_expression(matches[2], safe: false) else - filter_args << parse_context.parse_expression(a) + filter_args << parse_context.parse_expression(a, safe: safe) end end result = [filter_name, filter_args]

now the `safe: true` calls are considered safe Test the entire template instead

behavior

…efault.error_mode The `rake test` command gave us the impression that we were running all the tests on all the error modes, that was false.

History.md

charlespwd

Thanks for cleaning all this up :D LGTM other than I think this is at least a minor version change :D

bin/render

aswamy

Nothing blocking, but amazing work! Honestly, this was super thorough and clean 🧹

README.md

Rakefile

lib/liquid/condition.rb

test/integration/tags/cycle_tag_test.rb

test/integration/tags/render_tag_test.rb

test/unit/tags/case_tag_unit_test.rb

test/test_helper.rb

test/integration/variable_test.rb

Co-authored-by: Alok Swamy <[email protected]>

lib/liquid/tags/include.rb

lib/liquid/tags/cycle.rb

ggmichaelgo

Great stuff! Could you also add the new parser mode to rake benchmark task please? Let's make sure rigid parser is more performative than the lax/strict parsers :)

lib/liquid/tags/include.rb

The benchmark results show that rigid mode performs a bit better than both strict and lax modes across most metrics, including tokenization, parsing, rendering, and their combined operations. Rigid mode consistently delivers the highest number of iterations per second, with performance differences staying within 1–2% compared to the other modes ``` ================================================================================ /opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb lax Running benchmark for 20 seconds (with 10 seconds warmup). ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- tokenize: 332.000 i/100ms parse: 14.000 i/100ms render: 61.000 i/100ms parse & render: 11.000 i/100ms Calculating ------------------------------------- tokenize: 3.325k (± 1.0%) i/s (300.73 μs/i) - 66.732k in 20.070562s parse: 148.166 (± 0.7%) i/s (6.75 ms/i) - 2.968k in 20.032971s render: 654.428 (± 4.0%) i/s (1.53 ms/i) - 13.115k in 20.090452s parse & render: 116.108 (± 1.7%) i/s (8.61 ms/i) - 2.332k in 20.089221s ================================================================================ /opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb strict Running benchmark for 20 seconds (with 10 seconds warmup). ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- tokenize: 332.000 i/100ms parse: 14.000 i/100ms render: 61.000 i/100ms parse & render: 11.000 i/100ms Calculating ------------------------------------- tokenize: 3.332k (± 0.2%) i/s (300.14 μs/i) - 66.732k in 20.029095s parse: 145.674 (± 0.0%) i/s (6.86 ms/i) - 2.926k in 20.086104s render: 656.711 (± 4.6%) i/s (1.52 ms/i) - 13.115k in 20.050810s parse & render: 114.705 (± 0.0%) i/s (8.72 ms/i) - 2.299k in 20.043028s ================================================================================ /opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb rigid Running benchmark for 20 seconds (with 10 seconds warmup). ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- tokenize: 333.000 i/100ms parse: 14.000 i/100ms render: 62.000 i/100ms parse & render: 11.000 i/100ms Calculating ------------------------------------- tokenize: 3.334k (± 0.3%) i/s (299.93 μs/i) - 66.933k in 20.075484s parse: 148.349 (± 2.0%) i/s (6.74 ms/i) - 2.968k in 20.019775s render: 663.752 (± 2.6%) i/s (1.51 ms/i) - 13.268k in 20.010303s parse & render: 116.464 (± 2.6%) i/s (8.59 ms/i) - 2.332k in 20.037869s liquid$ ```

karreiro · 2025-10-24T07:33:35Z

Thank you for the review, @ggmichaelgo!

I've applied all suggestions and also added the new rigid mode to the benchmark task.

Looking at the numbers, rigid mode is a bit faster than both strict and lax modes on all benchmarks — tokenization, parsing, rendering, and the combined results. The improvement is small, but rigid mode does give us the best performance overall.

================================================================================
/opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb lax

Running benchmark for 20 seconds (with 10 seconds warmup).

ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
           tokenize:   332.000 i/100ms
              parse:    14.000 i/100ms
             render:    61.000 i/100ms
     parse & render:    11.000 i/100ms
Calculating -------------------------------------
           tokenize:      3.325k (± 1.0%) i/s  (300.73 μs/i) -    66.732k in  20.070562s
              parse:    148.166 (± 0.7%) i/s    (6.75 ms/i)  -     2.968k in  20.032971s
             render:    654.428 (± 4.0%) i/s    (1.53 ms/i)  -    13.115k in  20.090452s
     parse & render:    116.108 (± 1.7%) i/s    (8.61 ms/i)  -     2.332k in  20.089221s

================================================================================
/opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb strict

Running benchmark for 20 seconds (with 10 seconds warmup).

ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
           tokenize:   332.000 i/100ms
              parse:    14.000 i/100ms
             render:    61.000 i/100ms
     parse & render:    11.000 i/100ms
Calculating -------------------------------------
           tokenize:      3.332k (± 0.2%) i/s  (300.14 μs/i) -    66.732k in  20.029095s
              parse:    145.674 (± 0.0%) i/s    (6.86 ms/i)  -     2.926k in  20.086104s
             render:    656.711 (± 4.6%) i/s    (1.52 ms/i)  -    13.115k in  20.050810s
     parse & render:    114.705 (± 0.0%) i/s    (8.72 ms/i)  -     2.299k in  20.043028s

================================================================================
/opt/rubies/3.4.1/bin/ruby ./performance/benchmark.rb rigid

Running benchmark for 20 seconds (with 10 seconds warmup).

ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
           tokenize:   333.000 i/100ms
              parse:    14.000 i/100ms
             render:    62.000 i/100ms
     parse & render:    11.000 i/100ms
Calculating -------------------------------------
           tokenize:      3.334k (± 0.3%) i/s  (299.93 μs/i) -    66.933k in  20.075484s
              parse:    148.349 (± 2.0%) i/s    (6.74 ms/i)  -     2.968k in  20.019775s
             render:    663.752 (± 2.6%) i/s    (1.51 ms/i)  -    13.268k in  20.010303s
     parse & render:    116.464 (± 2.6%) i/s    (8.59 ms/i)  -     2.332k in  20.037869s

graygilmore

Awesome work both of you!

README.md

test/unit/condition_unit_test.rb

test/integration/tags/table_row_test.rb

lib/liquid/variable.rb

Co-authored-by: Gray Gilmore <[email protected]>

Introduce :rigid parsing mode

beaea6f

karreiro force-pushed the rigid-mode branch from 5fccd15 to beaea6f Compare September 30, 2025 12:37

charlespwd added 2 commits September 30, 2025 15:45

Add rigid_parse_with_error_context and clarifications

69a807e

Add a rigid_parse method to cycle

06279ec

charlespwd reviewed Sep 30, 2025

View reviewed changes

lib/liquid/environment.rb Outdated Show resolved Hide resolved

charlespwd reviewed Sep 30, 2025

View reviewed changes

lib/liquid/parse_context.rb Outdated Show resolved Hide resolved

Add rigid_parse to render

22232cc

charlespwd reviewed Oct 1, 2025

View reviewed changes

lib/liquid/tags/render.rb Show resolved Hide resolved

charlespwd and others added 20 commits October 1, 2025 08:58

Add rigid_parse method to include

40928e1

Add ExpressionParser and ExpressionConsumer

8f5361b

Use ExpressionParser in the ParseContext when parsing in :rigid…

7527f8a

… mode

Remove ExpressionParser in favor of ParseContext#safe_parse

d9caf26

Fix cycle tag

5625f6f

- `respond_to?` was returning `false` in the parser switcher because `rigid_parse` was private It was working before because `parse_context` was doing the double-parsing thing, but when we removed that, this test fairly started breaking

Fix an int the cycle tag, add extra unit tests, and updated parser …

42fcbcc

…switcher: - Fixed NoMethod error with .peek (using look instead) - Add friendlier error message when {% cycle %}

Fail with trailing elements in the cycle tag

0d7a7a5

Add rigid_parse to case/when

8e6faa4

Remove unnecessary skips

a16b955

Use safe_parse_expression instead of parse_expression

c36852d

Stricter 1:1 refactor of strict_parse for Variable

1308b97

rigid set_attribute in for parsing

5718f8b

No longer test ParseContext directly on RigidModeUnitTest as

1ba6fab

now the `safe: true` calls are considered safe Test the entire template instead

Remove redundant tests where rigid and strict modes have the same

fd81ac1

behavior

Add rigid parser to tablerow tag

376d849

render end of string is not optional

1c3c979

Fix alias parsing

bc32531

Fixup include parsing of with expression

26f092b

Fixup cycle rigid parsing to be backwards compatible

fd186dc

charlespwd force-pushed the rigid-mode branch from 82e2882 to ce4cdaa Compare October 20, 2025 19:06

Fix assert_template_result tests not picking up Liquid::Environment.d…

937b733

…efault.error_mode The `rake test` command gave us the impression that we were running all the tests on all the error modes, that was false.

jg-rp mentioned this pull request Oct 22, 2025

Rigid mode jg-rp/golden-liquid#24

Merged

charlespwd reviewed Oct 22, 2025

View reviewed changes

History.md Outdated Show resolved Hide resolved

charlespwd approved these changes Oct 22, 2025

View reviewed changes

Update History.md (5.8.8 -> 5.9.0)

195fd5a

EvilGenius13 reviewed Oct 22, 2025

View reviewed changes

bin/render Show resolved Hide resolved

aswamy approved these changes Oct 22, 2025

View reviewed changes

karreiro and others added 4 commits October 23, 2025 09:52

Update Rakefile

54b41db

Co-authored-by: Alok Swamy <[email protected]>

Update test/unit/tags/case_tag_unit_test.rb

36ec055

Co-authored-by: Alok Swamy <[email protected]>

Update test/integration/tags/render_tag_test.rb

8c8a843

Co-authored-by: Alok Swamy <[email protected]>

Rename with_error_mode(...) to with_error_modes(...)

ba5fb99

karreiro requested a review from EvilGenius13 October 23, 2025 08:28

EvilGenius13 approved these changes Oct 23, 2025

View reviewed changes