DSL Engine Integration #1143
DSL Engine Integration #1143
Conversation
WalkthroughAdds a new DSLMatcher for parsing/evaluating version DSLs, integrates it into HTTP response condition checks (version_dsl and cve_version_match), introduces a VMware Aria Operations vulnerability module using the DSL, updates packaging dependencies, and applies minor README/YAML formatting tweaks. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant Module as Vulnerability Module
participant CoreHTTP as core.module_protocols.core_http
participant DSL as core.dsl_matcher.DSLMatcher
participant Target as Target Server
User->>Module: run payload
Module->>CoreHTTP: build and send HTTP request
CoreHTTP->>Target: HTTP request
Target-->>CoreHTTP: HTTP response (status, headers, body)
Note over CoreHTTP: assemble searchable content (headers + body)
alt version_dsl condition
CoreHTTP->>DSL: extract_version_from_response(content, patterns)
DSL-->>CoreHTTP: extracted_version
CoreHTTP->>DSL: parse_dsl_expression(expression, extracted_version)
DSL-->>CoreHTTP: match boolean
CoreHTTP-->>Module: condition result (apply reverse if set)
else cve_version_match condition
CoreHTTP->>DSL: extract_version_from_response(content, patterns)
DSL-->>CoreHTTP: extracted_version
CoreHTTP->>DSL: match_cve_version_range(extracted_version, affected_versions)
DSL-->>CoreHTTP: vulnerable? (boolean)
CoreHTTP-->>Module: condition result (apply reverse if set)
end
Module-->>User: final match/outcome
sequenceDiagram
participant DSL as DSLMatcher
participant Expr as DSL Expression
participant Ver as Target Version
DSL->>DSL: _clean_version(Ver)
alt comma-separated / multiple
DSL->>DSL: _evaluate_multiple(Expr, Ver)
else range ("to" or "-")
DSL->>DSL: _evaluate_range(Expr, Ver)
else wildcard
DSL->>DSL: _evaluate_wildcard(Expr, Ver)
else comparator (~ ^ >= <= > < == !=)
DSL->>DSL: _compare_version(v1, v2, op)
Note over DSL: uses packaging, semver, or numeric-part fallback\n_tilde_/_caret_ handled by helpers
end
DSL-->>Expr: boolean result
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
This seems like a good addition to nettacker. I would suggest making a separate PR for this and removing this file from here.
|
Thank you @pUrGe12!
I've modified I will also remove CVE_2025_41244.yaml before merging - I thought it would be useful to include as a showcase for what this integration would look like in a template. |
|
@pUrGe12, with your familiarity with the project I invite you test these changes. I've:
Purpose of this PR: Thanks! |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
README.md(0 hunks)core/dsl_matcher.py(1 hunks)core/module_protocols/core_http.py(3 hunks)modules/vuln/apache_cve_2021_41773.yaml(1 hunks)modules/vuln/aria_operations_cve_2025_41244.yaml(1 hunks)requirements.txt(1 hunks)
💤 Files with no reviewable changes (1)
- README.md
🧰 Additional context used
🪛 Ruff (0.13.3)
core/dsl_matcher.py
1-1: Shebang is present but file is not executable
(EXE001)
49-49: Do not catch blind exception: Exception
(BLE001)
125-125: Do not catch blind exception: Exception
(BLE001)
182-182: Do not catch blind exception: Exception
(BLE001)
215-215: Consider moving this statement to an else block
(TRY300)
217-217: Do not catch blind exception: Exception
(BLE001)
247-247: Consider moving this statement to an else block
(TRY300)
249-249: Do not catch blind exception: Exception
(BLE001)
271-271: Do not catch blind exception: Exception
(BLE001)
307-307: Do not catch blind exception: Exception
(BLE001)
322-322: Do not catch blind exception: Exception
(BLE001)
361-361: Do not catch blind exception: Exception
(BLE001)
434-435: try-except-continue detected, consider logging the exception
(S112)
434-434: Do not catch blind exception: Exception
(BLE001)
479-479: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: test
|
amazing! I will work on these changes today/this weekend! |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
core/dsl_matcher.py (1)
204-266: Consider extracting shared logic between tilde and caret operators.
_tilde_compareand_caret_compareshare significant code structure (version splitting, padding, part comparison, boundary calculation). The primary difference is which version component gets incremented (minor vs. major). Extracting common logic into a helper method would improve maintainability.Example refactor:
def _semantic_range_compare(self, version1, version2, boundary_index): """ Helper for tilde/caret comparison. Args: boundary_index: 0 for major (caret), 1 for minor (tilde) """ try: v1_parts = [int(x) for x in version1.split('.')] v2_parts = [int(x) for x in version2.split('.')] max_len = max(len(v1_parts), len(v2_parts)) v1_parts.extend([0] * (max_len - len(v1_parts))) v2_parts.extend([0] * (max_len - len(v2_parts))) if not self._compare_version_parts(v1_parts, v2_parts, '>='): return False if len(v2_parts) > boundary_index: next_version = v2_parts.copy() next_version[boundary_index] += 1 for i in range(boundary_index + 1, len(next_version)): next_version[i] = 0 return self._compare_version_parts(v1_parts, next_version, '<') return True except Exception: return False def _tilde_compare(self, version1, version2): return self._semantic_range_compare(version1, version2, boundary_index=1) def _caret_compare(self, version1, version2): return self._semantic_range_compare(version1, version2, boundary_index=0)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
core/dsl_matcher.py(1 hunks)
🧰 Additional context used
🪛 Ruff (0.13.3)
core/dsl_matcher.py
1-1: Shebang is present but file is not executable
(EXE001)
49-49: Do not catch blind exception: Exception
(BLE001)
141-141: Do not catch blind exception: Exception
(BLE001)
198-198: Do not catch blind exception: Exception
(BLE001)
231-231: Consider moving this statement to an else block
(TRY300)
233-233: Do not catch blind exception: Exception
(BLE001)
263-263: Consider moving this statement to an else block
(TRY300)
265-265: Do not catch blind exception: Exception
(BLE001)
298-298: Do not catch blind exception: Exception
(BLE001)
334-334: Do not catch blind exception: Exception
(BLE001)
349-349: Do not catch blind exception: Exception
(BLE001)
388-388: Do not catch blind exception: Exception
(BLE001)
461-462: try-except-continue detected, consider logging the exception
(S112)
461-461: Do not catch blind exception: Exception
(BLE001)
506-506: Do not catch blind exception: Exception
(BLE001)
🔇 Additional comments (2)
core/dsl_matcher.py (2)
100-111: Logical connector parsing successfully implemented.The previous critical issue regarding unparsed logical connectors has been addressed. The code now correctly handles OR (
||,or) and AND (&&,and) operators with case-insensitive matching for word-based operators, splitting expressions before operator dispatch.
268-297: Range parsing correctly handles prerelease versions.The previous major issue regarding hyphenated prerelease versions being misdetected as ranges has been fixed. The code now requires spaces around the hyphen (
-) to treat it as a range separator, avoiding misinterpretation of versions like"8.18.0-beta1".
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (4)
core/dsl_matcher.py (4)
336-341: Escape regex safely when translating wildcardsOnly '.' is escaped; other regex metacharacters can misbehave. Prefer re.escape, then restore wildcard tokens.
Apply this diff:
- # Convert wildcard to regex - pattern = dsl_expression.replace('.', r'\.') - pattern = pattern.replace('*', '.*') - pattern = pattern.replace('?', '.') - pattern = f'^{pattern}$' + # Convert wildcard to regex: escape everything then expand wildcards + escaped = re.escape(dsl_expression) + pattern = '^' + escaped.replace(r'\*', '.*').replace(r'\?', '.') + '$'
433-447: Swallowed regex errors in extraction; log for diagnosticsExceptions are silently ignored. Log at debug to aid troubleshooting and satisfy static analysis (S112).
Apply this diff:
- except Exception: - continue + except Exception as e: + self.logger.debug(f"extract_version_from_response: pattern {pattern!r} failed: {e}") + continue
34-36: Broad Exception catches reduce debuggabilityMultiple blind excepts hide real errors and complicate testing. Narrow to specific exceptions (e.g., InvalidVersion, ValueError, RecursionError) and log expression/context. At minimum, convert to “except Exception as e: logger.debug(..., exc_info=True)”.
Example pattern (apply similarly across methods):
- except Exception as e: - self.logger.debug(f"DSL evaluation error: {e}") + except RecursionError as e: + self.logger.debug(f"DSL evaluation recursion error for {dsl_expression!r}: {e}", exc_info=True) + except Exception as e: + self.logger.debug(f"DSL evaluation error for {dsl_expression!r}: {e}", exc_info=True)Ruff BLE001 flagged these spots. As per coding guidelines
Also applies to: 126-128, 183-187, 222-223, 258-259, 291-293, 327-329, 342-343, 381-383, 444-445
1-3: Shebang without executable bitEither remove the shebang or make the file executable to satisfy EXE001.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
core/dsl_matcher.py(1 hunks)
🧰 Additional context used
🪛 Ruff (0.13.3)
core/dsl_matcher.py
1-1: Shebang is present but file is not executable
(EXE001)
34-34: Do not catch blind exception: Exception
(BLE001)
126-126: Do not catch blind exception: Exception
(BLE001)
183-183: Do not catch blind exception: Exception
(BLE001)
220-220: Consider moving this statement to an else block
(TRY300)
222-222: Do not catch blind exception: Exception
(BLE001)
256-256: Consider moving this statement to an else block
(TRY300)
258-258: Do not catch blind exception: Exception
(BLE001)
291-291: Do not catch blind exception: Exception
(BLE001)
327-327: Do not catch blind exception: Exception
(BLE001)
342-342: Do not catch blind exception: Exception
(BLE001)
381-381: Do not catch blind exception: Exception
(BLE001)
444-445: try-except-continue detected, consider logging the exception
(S112)
444-444: Do not catch blind exception: Exception
(BLE001)
478-478: Do not catch blind exception: Exception
(BLE001)
| # Handle logical connectors (OR and AND) before operator dispatch | ||
| # Check for OR operators first (|| or 'or') | ||
| for splitter, combiner in [('||', any), (' or ', any), ('&&', all), (' and ', all)]: | ||
| # Use case-insensitive search for word-based operators | ||
| if splitter in ['||', '&&']: | ||
| check = splitter in normalized | ||
| else: | ||
| check = re.search(rf'\s+{re.escape(splitter.strip())}\s+', normalized, re.IGNORECASE) | ||
|
|
||
| if check: | ||
| parts = [part.strip() for part in re.split(rf'\s*(?:{re.escape(splitter)})\s*', normalized, flags=re.IGNORECASE) if part.strip()] | ||
| return combiner(self._evaluate_dsl(part, target_version) for part in parts) | ||
|
|
There was a problem hiding this comment.
Connector split bug can cause deep recursion; unify detection and splitting
Detection allows multi-space around textual operators, but split pattern requires single spaces, so expressions like "a or b" won’t split and recurse until RecursionError. Split using compiled regexes and check len(parts) > 1.
Apply this diff:
- # Handle logical connectors (OR and AND) before operator dispatch
- # Check for OR operators first (|| or 'or')
- for splitter, combiner in [('||', any), (' or ', any), ('&&', all), (' and ', all)]:
- # Use case-insensitive search for word-based operators
- if splitter in ['||', '&&']:
- check = splitter in normalized
- else:
- check = re.search(rf'\s+{re.escape(splitter.strip())}\s+', normalized, re.IGNORECASE)
-
- if check:
- parts = [part.strip() for part in re.split(rf'\s*(?:{re.escape(splitter)})\s*', normalized, flags=re.IGNORECASE) if part.strip()]
- return combiner(self._evaluate_dsl(part, target_version) for part in parts)
+ # Handle logical connectors with robust splitting (AND has higher precedence)
+ connectors = [
+ (re.compile(r'\s*\|\|\s*'), any),
+ (re.compile(r'\s*\bOR\b\s*', re.IGNORECASE), any),
+ ]
+ for regex, combiner in connectors:
+ parts = [p.strip() for p in regex.split(normalized) if p.strip()]
+ if len(parts) > 1:
+ return combiner(self._evaluate_dsl(part, target_version) for part in parts)
+ # AND after OR to preserve precedence within each OR branch
+ connectors_and = [
+ (re.compile(r'\s*&&\s*'), all),
+ (re.compile(r'\s*\bAND\b\s*', re.IGNORECASE), all),
+ ]
+ for regex, combiner in connectors_and:
+ parts = [p.strip() for p in regex.split(normalized) if p.strip()]
+ if len(parts) > 1:
+ return combiner(self._evaluate_dsl(part, target_version) for part in parts)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Handle logical connectors (OR and AND) before operator dispatch | |
| # Check for OR operators first (|| or 'or') | |
| for splitter, combiner in [('||', any), (' or ', any), ('&&', all), (' and ', all)]: | |
| # Use case-insensitive search for word-based operators | |
| if splitter in ['||', '&&']: | |
| check = splitter in normalized | |
| else: | |
| check = re.search(rf'\s+{re.escape(splitter.strip())}\s+', normalized, re.IGNORECASE) | |
| if check: | |
| parts = [part.strip() for part in re.split(rf'\s*(?:{re.escape(splitter)})\s*', normalized, flags=re.IGNORECASE) if part.strip()] | |
| return combiner(self._evaluate_dsl(part, target_version) for part in parts) | |
| # Handle logical connectors with robust splitting (AND has higher precedence) | |
| connectors = [ | |
| (re.compile(r'\s*\|\|\s*'), any), | |
| (re.compile(r'\s*\bOR\b\s*', re.IGNORECASE), any), | |
| ] | |
| for regex, combiner in connectors: | |
| parts = [p.strip() for p in regex.split(normalized) if p.strip()] | |
| if len(parts) > 1: | |
| return combiner(self._evaluate_dsl(part, target_version) for part in parts) | |
| # AND after OR to preserve precedence within each OR branch | |
| connectors_and = [ | |
| (re.compile(r'\s*&&\s*'), all), | |
| (re.compile(r'\s*\bAND\b\s*', re.IGNORECASE), all), | |
| ] | |
| for regex, combiner in connectors_and: | |
| parts = [p.strip() for p in regex.split(normalized) if p.strip()] | |
| if len(parts) > 1: | |
| return combiner(self._evaluate_dsl(part, target_version) for part in parts) |
🤖 Prompt for AI Agents
In core/dsl_matcher.py around lines 85 to 97, the connector detection and
splitting are inconsistent: detection allows variable whitespace for textual
operators like " or " but the split pattern requires single spaces which can
leave strings unsplit and trigger deep recursion; replace ad-hoc checks with
precompiled regex patterns for each connector (OR: r'\s*\|\|\s*' and r'\s+or\s+'
and AND: r'\s*&&\s*' and r'\s+and\s+', all with re.IGNORECASE) use the same
regex both to test for a match and to split, and after splitting validate with
if len(parts) > 1 before recursing/combining to avoid accidental recursion.
Ensure parts are stripped and pass them to combiner(self._evaluate_dsl(part,
target_version) for part in parts).
| # Check if v1 < next minor version | ||
| if len(v2_parts) >= 2: | ||
| next_minor = v2_parts.copy() | ||
| next_minor[1] += 1 | ||
| for i in range(2, len(next_minor)): | ||
| next_minor[i] = 0 | ||
|
|
||
| return self._compare_version_parts(v1_parts, next_minor, '<') | ||
|
|
||
| return True | ||
|
|
There was a problem hiding this comment.
Tilde (~) semantics incorrect when only major is provided
When version2 has only a major segment (e.g., "~1"), the function returns True unconditionally. Use a sane bound (>=1.0.0 and <2.0.0) or return False.
Apply this diff to cap at next major:
- # Check if v1 < next minor version
- if len(v2_parts) >= 2:
+ # Check upper bound: next minor if minor present, else next major
+ if len(v2_parts) >= 2:
next_minor = v2_parts.copy()
next_minor[1] += 1
for i in range(2, len(next_minor)):
next_minor[i] = 0
return self._compare_version_parts(v1_parts, next_minor, '<')
-
- return True
+ else:
+ next_major = v2_parts.copy()
+ next_major[0] += 1
+ for i in range(1, len(next_major)):
+ next_major[i] = 0
+ return self._compare_version_parts(v1_parts, next_major, '<')📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Check if v1 < next minor version | |
| if len(v2_parts) >= 2: | |
| next_minor = v2_parts.copy() | |
| next_minor[1] += 1 | |
| for i in range(2, len(next_minor)): | |
| next_minor[i] = 0 | |
| return self._compare_version_parts(v1_parts, next_minor, '<') | |
| return True | |
| # Check upper bound: next minor if minor present, else next major | |
| if len(v2_parts) >= 2: | |
| next_minor = v2_parts.copy() | |
| next_minor[1] += 1 | |
| for i in range(2, len(next_minor)): | |
| next_minor[i] = 0 | |
| return self._compare_version_parts(v1_parts, next_minor, '<') | |
| else: | |
| next_major = v2_parts.copy() | |
| next_major[0] += 1 | |
| for i in range(1, len(next_major)): | |
| next_major[i] = 0 | |
| return self._compare_version_parts(v1_parts, next_major, '<') |
🧰 Tools
🪛 Ruff (0.13.3)
220-220: Consider moving this statement to an else block
(TRY300)
🤖 Prompt for AI Agents
In core/dsl_matcher.py around lines 211 to 221, the tilde (~) handling currently
returns True when version2 has only a major (e.g., "~1"); change it to cap at
the next major (treat "~1" as >=1.0.0 and <2.0.0). Specifically, when
len(v2_parts) == 1, build next_major by copying v2_parts, incrementing the major
(next_major[0] += 1) and zeroing any remaining segments, then use
self._compare_version_parts(v1_parts, next_major, '<') instead of returning
True; keep existing behavior for the case with at least two segments.
|
Sorry for going AWOL, this is still on my to-do list. I will create a new PR with the base branch being |
3 participants
Proposed change
Your PR description goes here.
Type of change
Checklist
make pre-commit, it didn't generate any changesmake test, all tests passed locallySummary by CodeRabbit
New Features
Documentation
Style
Chores