Merge branch 'main' of https://github.com/seedcase-project/check-datapackage into refactor/rename-file-to-extensions

Author: lwjohnst86

.pre-commit-config.yaml

@@ -25,6 +25,6 @@ repos:
   # sub-packages, which confuses pre-commit when it tries to find the latest
   # version
   - repo: https://github.com/adhtruong/mirrors-typos
-    rev: v1.38.1
+    rev: v1.39.0
     hooks:
       - id: typos

CHANGELOG.md

@@ -9,6 +9,30 @@ often, sometimes several in a day. It also means any individual release
 will not have many changes within it. Below is a list of releases along
 with what was changed within it.
 
+## 0.12.0 (2025-11-03)
+
+### Feat
+
+- :sparkles: adds `Extensions` meta-class (#165)
+
+## 0.11.1 (2025-11-03)
+
+### Refactor
+
+- :recycle: handle grouped errors without modifying input (#177)
+
+## 0.11.0 (2025-11-03)
+
+### Feat
+
+- :sparkles: add example field in resource (#174)
+
+## 0.10.0 (2025-11-03)
+
+### Feat
+
+- :sparkles: exclude required issues at a given JSON path (#138)
+
 ## 0.9.0 (2025-10-29)
 
 ### Feat

_quarto.yml

@@ -73,6 +73,7 @@ quartodoc:
     desc: Classes to configure behaviour of the checks.
     contents:
     - name: Config
+    - name: Extensions
     - name: CustomCheck
     - name: RequiredCheck
     - name: Exclusion
@@ -87,6 +88,7 @@ quartodoc:
     contents:
     - name: example_package_properties
     - name: example_resource_properties
+    - name: example_field_properties
 
 
 metadata-files:

docs/design/interface.qmd

@@ -144,63 +144,15 @@ Package standard that are not relevant to your use case.
 
 See the help documentation with `help(Exclusion)` for more details.
 
-#### {{< var wip >}} `Extensions`
+#### {{< var done >}} `Extensions`
 
 This sub-item of `Config` defines extensions, i.e., additional checks
-that supplement those specified by the Data Package standard. It
-contains subitems that store additional checks, such as `RequiredCheck`
-and `CustomCheck`. This `Extensions` class might be expanded to include
-more types of extensions.
-
-```` python
-@dataclass(frozen=True)
-class Extensions:
-    """Extensions to the standard checks.
-
-    This contains additional checks to be made alongside the standard
-    Data Package checks.
-
-    Attributes:
-        required_checks: A list of `RequiredCheck` objects defining properties to set as required.
-        custom_checks: A list of `CustomCheck` objects defining extra, custom checks to run alongside the standard
-            checks.
-
-    Examples:
-
-        ```python
-        # TODO: Add curly brackets around python on the line above when the class has been implemented.
-        import check_datapackage as cdp
-
-        extensions = cdp.Extensions(
-            required_checks=[
-                cdp.RequiredCheck(
-                    jsonpath="$.description",
-                    message="Data Packages must include a description."
-                ),
-                cdp.RequiredCheck(
-                    jsonpath="$.contributors[*].email",
-                    message="All contributors must have an email address."
-                )
-            ],
-            custom_checks=[cdp.CustomCheck(
-                type="only-mit",
-                jsonpath="$.licenses[*].name",
-                message="Data Packages may only be licensed under MIT.",
-                check=lambda license_name: license_name == "mit",
-            )]
-        )
-        # check(properties, config=cdp.Config(extensions=extensions))
-        ```
-    """
-    required_checks : list[RequiredCheck] = field(default_factory=list)
-    custom_checks : list[CustomCheck] = field(default_factory=list)
-````
+that supplement those specified by the Data Package standard.
 
-Each extension class must implement its own `apply()` method that takes
-the `datapackage.json` properties `dict` as input and outputs an `Issue`
-list that contains the issues found by that extension.
+See the help documentation with
+[`Extensions`](/docs/reference/Extensions.qmd) for more details.
 
-#### {{< var wip >}} `RequiredCheck`
+#### {{< var done >}} `RequiredCheck`
 
 A sub-item of `Extensions` that allows users to set specific properties
 as required that are not required by the Data Package standard. See the

docs/guide/check.qmd

@@ -100,7 +100,7 @@ Data Package standard, or add your own custom checks.
 ::: callout-tip
 For more details on configuring checks, see the [Configuring
 checks](config.qmd) guide or the [`Config` reference
-documentation](/docs/reference/config.qmd).
+documentation](/docs/reference/Config.qmd).
 :::
 
 ## Stop program on failed checks (`error=True`) {#stop-program-on-failed-checks-errortrue}

docs/guide/config.qmd

@@ -135,6 +135,7 @@ To register your custom checks with the `check()` function, you add them
 to the `Config` object passed to the function:
 
 ```{python}
+#| eval: false
 package_properties = {
     "name": "woolly-dormice",
     "title": "Hibernation Physiology of the Woolly Dormouse: A Scoping Review.",

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "check-datapackage"
-version = "0.9.0"
+version = "0.12.0"
 # TODO: Add a description of the package.
 description = ""
 authors = [

src/check_datapackage/__init__.py

@@ -2,20 +2,26 @@
 
 from .check import check
 from .config import Config
-from .examples import example_package_properties, example_resource_properties
+from .examples import (
+    example_field_properties,
+    example_package_properties,
+    example_resource_properties,
+)
 from .exclusion import Exclusion
-from .extensions import CustomCheck, RequiredCheck
+from .extensions import CustomCheck, Extensions, RequiredCheck
 from .issue import Issue
 from .read_json import read_json
 
 __all__ = [
     "Config",
     "Exclusion",
     "Issue",
+    "Extensions",
     "CustomCheck",
     "RequiredCheck",
     "example_package_properties",
     "example_resource_properties",
+    "example_field_properties",
     "check",
     "read_json",
 ]

src/check_datapackage/check.py

@@ -1,7 +1,7 @@
 import re
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from functools import reduce
-from typing import Any, Iterator, Optional
+from typing import Any, Callable, Iterator, Optional
 
 from jsonschema import Draft7Validator, FormatChecker, ValidationError
 
@@ -44,7 +44,7 @@ class for more details, especially about the default values.
         _set_should_fields_to_required(schema)
 
     issues = _check_object_against_json_schema(properties, schema)
-    issues += apply_extensions(properties, config.custom_checks)
+    issues += apply_extensions(properties, config.extensions)
     issues = exclude(issues, config.exclusions, properties)
 
     return sorted(set(issues))
@@ -137,46 +137,31 @@ def _validation_errors_to_issues(
     return _map(schema_errors, _create_issue)
 
 
-def _handle_grouped_error(
-    schema_errors: list[SchemaError], parent_error: SchemaError
-) -> list[SchemaError]:
-    """Handle grouped schema errors that need special treatment.
-
-    Args:
-        schema_errors: All remaining schema errors.
-        parent_error: The parent error of a group.
-
-    Returns:
-        The schema errors after processing.
-    """
-    # Handle issues at $.resources[x]
-
-    if parent_error.schema_path.endswith("resources/items/oneOf"):
-        schema_errors = _handle_S_resources_x(parent_error, schema_errors)
-
-    # Handle issues at $.resources[x].path
-    if parent_error.schema_path.endswith("resources/items/properties/path/oneOf"):
-        schema_errors = _handle_S_resources_x_path(parent_error, schema_errors)
+@dataclass(frozen=True)
+class SchemaErrorEdits:
+    """Expresses which errors to add to or remove from schema errors."""
 
-    return schema_errors
+    add: list[SchemaError] = field(default_factory=list)
+    remove: list[SchemaError] = field(default_factory=list)
 
 
 def _handle_S_resources_x(
     parent_error: SchemaError,
     schema_errors: list[SchemaError],
-) -> list[SchemaError]:
+) -> SchemaErrorEdits:
     """Do not flag missing `path` and `data` separately."""
+    edits = SchemaErrorEdits()
     errors_in_group = _filter(schema_errors, lambda error: error.parent == parent_error)
     # If the parent error is caused by other errors, remove it
     if errors_in_group:
-        schema_errors.remove(parent_error)
+        edits.remove.append(parent_error)
 
     path_or_data_required_errors = _filter(
         errors_in_group, _path_or_data_required_error
     )
     # If path and data are both missing, add a more informative error
     if len(path_or_data_required_errors) > 1:
-        schema_errors.append(
+        edits.add.append(
             SchemaError(
                 message=(
                     "This resource has no `path` or `data` field. "
@@ -189,31 +174,31 @@ def _handle_S_resources_x(
         )
 
     # Remove all required errors on path and data
-    return _filter(
-        schema_errors, lambda error: error not in path_or_data_required_errors
-    )
+    edits.remove.extend(path_or_data_required_errors)
+    return edits
 
 
 def _handle_S_resources_x_path(
     parent_error: SchemaError,
     schema_errors: list[SchemaError],
-) -> list[SchemaError]:
+) -> SchemaErrorEdits:
     """Only flag errors for the relevant type.
 
     If `path` is a string, flag errors for the string-based schema.
     If `path` is an array, flag errors for the array-based schema.
     """
+    edits = SchemaErrorEdits()
     errors_in_group = _filter(schema_errors, lambda error: error.parent == parent_error)
     type_errors = _filter(errors_in_group, _is_path_type_error)
     only_type_errors = len(errors_in_group) == len(type_errors)
 
     if type_errors:
-        schema_errors.remove(parent_error)
+        edits.remove.append(parent_error)
 
     # If the only error is that $.resources[x].path is of the wrong type,
     # add a more informative error
     if only_type_errors:
-        schema_errors.append(
+        edits.add.append(
             SchemaError(
                 message="The `path` property must be either a string or an array.",
                 type="type",
@@ -223,7 +208,52 @@ def _handle_S_resources_x_path(
         )
 
     # Remove all original type errors on $.resources[x].path
-    return _filter(schema_errors, lambda error: error not in type_errors)
+    edits.remove.extend(type_errors)
+    return edits
+
+
+_schema_path_to_handler: list[
+    tuple[str, Callable[[SchemaError, list[SchemaError]], SchemaErrorEdits]]
+] = [
+    ("resources/items/oneOf", _handle_S_resources_x),
+    ("resources/items/properties/path/oneOf", _handle_S_resources_x_path),
+]
+
+
+def _handle_grouped_error(
+    schema_errors: list[SchemaError], parent_error: SchemaError
+) -> list[SchemaError]:
+    """Handle grouped schema errors that need special treatment.
+
+    Args:
+        schema_errors: All remaining schema errors.
+        parent_error: The parent error of a group.
+
+    Returns:
+        The schema errors after processing.
+    """
+
+    def _get_edits(
+        handlers: list[
+            tuple[str, Callable[[SchemaError, list[SchemaError]], SchemaErrorEdits]]
+        ],
+    ) -> SchemaErrorEdits:
+        schema_path, handler = handlers[0]
+        edits = SchemaErrorEdits()
+        if parent_error.schema_path.endswith(schema_path):
+            edits = handler(parent_error, schema_errors)
+
+        if len(handlers) == 1:
+            return edits
+
+        next_edits = _get_edits(handlers[1:])
+        return SchemaErrorEdits(
+            add=edits.add + next_edits.add,
+            remove=edits.remove + next_edits.remove,
+        )
+
+    edits = _get_edits(_schema_path_to_handler)
+    return _filter(schema_errors, lambda error: error not in edits.remove) + edits.add
 
 
 def _validation_error_to_schema_errors(error: ValidationError) -> list[SchemaError]:

src/check_datapackage/config.py

@@ -2,7 +2,7 @@
 from typing import Literal
 
 from check_datapackage.exclusion import Exclusion
-from check_datapackage.extensions import CustomCheck, RequiredCheck
+from check_datapackage.extensions import Extensions
 
 
 @dataclass
@@ -12,8 +12,8 @@ class Config:
     Attributes:
         exclusions (list[Exclusion]): Any issues matching any of Exclusion objects will
             be excluded (i.e., removed from the output of the check function).
-        custom_checks (list[CustomCheck | RequiredCheck]): Custom checks listed here
-            will be done in addition to checks defined in the Data Package standard.
+        extensions (Extensions): Additional checks (called extensions)
+            that supplement those specified by the Data Package standard.
         strict (bool): Whether to include "SHOULD" checks in addition to "MUST" checks
             from the Data Package standard. If True, "SHOULD" checks will also be
             included. Defaults to False.
@@ -37,12 +37,17 @@ class Config:
         )
         config = cdp.Config(
             exclusions=[exclusion_required],
-            custom_checks=[license_check, required_title_check],
+            extensions=cdp.Extensions(
+                custom_checks=[license_check],
+                required_checks=[required_title_check]
+            )
         )
+
+        # check(properties, config=config)
         ```
     """
 
-    exclusions: list[Exclusion] = field(default_factory=list)
-    custom_checks: list[CustomCheck | RequiredCheck] = field(default_factory=list)
+    exclusions: list[Exclusion] = field(default_factory=list[Exclusion])
+    extensions: Extensions = Extensions()
     strict: bool = False
     version: Literal["v1", "v2"] = "v2"