Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
f2066ea
working on a model observer
michaelbynum May 18, 2025
897c0ef
Merge remote-tracking branch 'origin/main' into observer
michaelbynum Jul 29, 2025
97aeb31
working on a model observer
michaelbynum Jul 30, 2025
a69d5e6
working on model change detector
michaelbynum Aug 6, 2025
9763e9a
observer
michaelbynum Aug 7, 2025
ff635b8
working on a model observer
michaelbynum Aug 8, 2025
db0fda4
Apply black
mrmundt Aug 12, 2025
149ab06
Merge remote-tracking branch 'origin/main' into observer
michaelbynum Aug 12, 2025
42c8cc8
adding copyright statements
michaelbynum Aug 12, 2025
ccb6de4
Merge branch 'main' into observer
michaelbynum Aug 12, 2025
d718e9a
Merge branch 'main' into observer
michaelbynum Aug 12, 2025
23ba4d9
typo
michaelbynum Aug 13, 2025
14f928b
Merge remote-tracking branch 'michaelbynum/observer' into observer
michaelbynum Aug 13, 2025
d7b9918
update observer tests
michaelbynum Aug 13, 2025
c313fe5
run black
michaelbynum Aug 13, 2025
a424cfb
Minor changes - removing unused imports
mrmundt Aug 25, 2025
33f831a
observer updates
michaelbynum Sep 16, 2025
5f3f403
observer fixes
michaelbynum Sep 16, 2025
79e1b47
observer config updates
michaelbynum Sep 16, 2025
7275176
minor observer updates
michaelbynum Sep 16, 2025
823e15a
disable gc for expensive parts of observer
michaelbynum Sep 16, 2025
73258f5
minor observer updates
michaelbynum Sep 16, 2025
5a8a5a6
minor observer updates
michaelbynum Sep 16, 2025
22c9169
Merge branch 'main' into observer
michaelbynum Sep 16, 2025
321755a
minor observer updates
michaelbynum Sep 16, 2025
f8cfc33
Merge remote-tracking branch 'michaelbynum/observer' into observer
michaelbynum Sep 16, 2025
2adefbc
docstring for the model change detector
michaelbynum Sep 16, 2025
df26c4e
run black
michaelbynum Sep 16, 2025
2dc566e
some comments for the observer
michaelbynum Sep 16, 2025
6b2ffbd
only need to descend into named expressions once
michaelbynum Sep 16, 2025
e68ee73
only need to descend into named expressions once
michaelbynum Sep 16, 2025
82e40b2
update observer tests
michaelbynum Sep 17, 2025
ae7e031
run black
michaelbynum Sep 17, 2025
5778688
fix docs
michaelbynum Sep 17, 2025
cd1c4ef
observer updates
michaelbynum Sep 18, 2025
3f30893
observer updates
michaelbynum Sep 18, 2025
4f7e93e
Merge branch 'main' into observer
michaelbynum Oct 2, 2025
ac9d952
Merge remote-tracking branch 'michaelbynum/observer' into observer
michaelbynum Oct 2, 2025
dc19b17
observer: docstring updates
michaelbynum Oct 2, 2025
bb60959
observer: typos
michaelbynum Oct 2, 2025
99ac089
Update pyomo/contrib/observer/model_observer.py
michaelbynum Oct 6, 2025
213d353
update check for unknown ctypes
michaelbynum Oct 6, 2025
065f43e
Merge remote-tracking branch 'michaelbynum/observer' into observer
michaelbynum Oct 6, 2025
ddf0a39
observer: use PauseGC()
michaelbynum Oct 6, 2025
643f546
Merge branch 'main' into observer
michaelbynum Oct 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions pyomo/contrib/observer/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# ___________________________________________________________________________
#
# Pyomo: Python Optimization Modeling Objects
# Copyright (c) 2008-2025
# National Technology and Engineering Solutions of Sandia, LLC
# Under the terms of Contract DE-NA0003525 with National Technology and
# Engineering Solutions of Sandia, LLC, the U.S. Government retains certain
# rights in this software.
# This software is distributed under the 3-clause BSD License.
# ___________________________________________________________________________
114 changes: 114 additions & 0 deletions pyomo/contrib/observer/component_collector.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# ___________________________________________________________________________
#
# Pyomo: Python Optimization Modeling Objects
# Copyright (c) 2008-2025
# National Technology and Engineering Solutions of Sandia, LLC
# Under the terms of Contract DE-NA0003525 with National Technology and
# Engineering Solutions of Sandia, LLC, the U.S. Government retains certain
# rights in this software.
# This software is distributed under the 3-clause BSD License.
# ___________________________________________________________________________

from pyomo.core.expr.visitor import StreamBasedExpressionVisitor
from pyomo.core.expr.numeric_expr import (
ExternalFunctionExpression,
NegationExpression,
PowExpression,
MaxExpression,
MinExpression,
ProductExpression,
MonomialTermExpression,
DivisionExpression,
SumExpression,
Expr_ifExpression,
UnaryFunctionExpression,
AbsExpression,
)
from pyomo.core.expr.relational_expr import (
RangedExpression,
InequalityExpression,
EqualityExpression,
)
from pyomo.core.base.var import VarData, ScalarVar
from pyomo.core.base.param import ParamData, ScalarParam
from pyomo.core.base.expression import ExpressionData, ScalarExpression
from pyomo.repn.util import ExitNodeDispatcher


def handle_var(node, collector):
collector.variables[id(node)] = node
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is basically reimplementing a version of ComponentSet. Is there a reason not to re-use that object?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have found that doing this manually is faster. That is the only reason. I'm not sure if the difference is enough to justify or not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to get some numbers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be about twice as fast.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - but we ought to see why ComponentSet is slower and try to fix it there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't disagree, but I don't think that needs to be part of the PR. We can always change the underlying implementation of this later. None of this is exposed to the user.

return None


def handle_param(node, collector):
collector.params[id(node)] = node
return None


def handle_named_expression(node, collector):
collector.named_expressions[id(node)] = node
return None


def handle_external_function(node, collector):
collector.external_functions[id(node)] = node
return None


def handle_skip(node, collector):
return None


collector_handlers = ExitNodeDispatcher()
collector_handlers[VarData] = handle_var
collector_handlers[ParamData] = handle_param
collector_handlers[ExpressionData] = handle_named_expression
collector_handlers[ScalarExpression] = handle_named_expression
collector_handlers[ExternalFunctionExpression] = handle_external_function
collector_handlers[NegationExpression] = handle_skip
collector_handlers[PowExpression] = handle_skip
collector_handlers[MaxExpression] = handle_skip
collector_handlers[MinExpression] = handle_skip
collector_handlers[ProductExpression] = handle_skip
collector_handlers[MonomialTermExpression] = handle_skip
collector_handlers[DivisionExpression] = handle_skip
collector_handlers[SumExpression] = handle_skip
collector_handlers[Expr_ifExpression] = handle_skip
collector_handlers[UnaryFunctionExpression] = handle_skip
collector_handlers[AbsExpression] = handle_skip
collector_handlers[RangedExpression] = handle_skip
collector_handlers[InequalityExpression] = handle_skip
collector_handlers[EqualityExpression] = handle_skip
collector_handlers[int] = handle_skip
collector_handlers[float] = handle_skip


class _ComponentFromExprCollector(StreamBasedExpressionVisitor):
def __init__(self, **kwds):
self.named_expressions = {}
self.variables = {}
self.params = {}
self.external_functions = {}
super().__init__(**kwds)

def exitNode(self, node, data):
return collector_handlers[node.__class__](node, self)

def beforeChild(self, node, child, child_idx):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I mentioned this in a previous comment, so repeating here so it is not lost for future development / reference)

For performance reasons, it may be good to add a BeforeChildDispatcher as well (especially because this collector is specifically looking for leaf nodes).

That said, this improvement can be deferred to a subsequent PR.

Copy link
Contributor Author

@michaelbynum michaelbynum Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to do this as a separate PR.

if id(child) in self.named_expressions:
return False, None
return True, None


_visitor = _ComponentFromExprCollector()


def collect_components_from_expr(expr):
_visitor.__init__()
_visitor.walk_expression(expr)
return (
list(_visitor.named_expressions.values()),
list(_visitor.variables.values()),
list(_visitor.params.values()),
list(_visitor.external_functions.values()),
)
Loading
Loading