Skip to content

Conversation

@joda9
Copy link
Collaborator

@joda9 joda9 commented Oct 14, 2025

This pull request refactors database engine handling, improves configuration loading, and enhances table import logic for better maintainability and flexibility. The main changes include replacing the toep_engine with a more general egon_engine, refactoring config loading to handle time parsing robustly, and making table/schema mapping lazy-loaded and context-aware. Additionally, the table import logic is now more flexible for both remote and local databases.

Database engine and import logic refactor:

  • Replaced the import and usage of toep_engine with egon_engine throughout edisgo.py to standardize database engine handling. [1] [2] [3] [4] [5]
  • Refactored import_tables_from_oep in config.py to support both remote (OEP/toep) and local (egon_data) database engines, including dynamic ORM class creation for local databases.

Configuration and mapping improvements:

  • Made DB table and schema mappings lazy-loaded via _ensure_db_mappings_loaded, improving performance and reliability for remote OEP access. [1] [2] [3]
  • Changed default schema references from "model_draft" to "dataset" and improved error handling when importing modules and attributes for mappings. [1] [2]

Configuration file parsing enhancements:

  • Centralized and robustified time parsing for demandlib configuration entries using _normalize_demandlib_times, replacing scattered manual parsing in both from_cfg and from_json. [1] [2] [3]

Miscellaneous improvements:

  • Simplified the get function for config value retrieval by iterating over possible type accessors, reducing nested try/except blocks.
  • Minor fix in test configuration to properly set pytest.engine_local when the --runlocal option is used.# Description

@joda9 joda9 merged commit 0d26519 into dev Oct 23, 2025
11 of 20 checks passed
@joda9 joda9 deleted the fix/allow_db_via_ssh branch October 23, 2025 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant