Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
e17e348
Conditionally add layers
unshrawal Jun 20, 2025
cbf0b74
speedup
unshrawal Jun 21, 2025
ed33307
Show params
unshrawal Jun 21, 2025
06411b9
Refactor
unshrawal Jun 23, 2025
574049e
Fix serialization bottleneck
unshrawal Jun 23, 2025
f47a0f6
Accommodate class arguments
unshrawal Jun 23, 2025
c18fd79
optimize downcasts
unshrawal Jun 24, 2025
2fb2311
Remove label
unshrawal Jun 24, 2025
8b22705
Use writeln instead of tracing for variable tracking
unshrawal Jun 24, 2025
5b463d7
Put var file & log file in same dir
unshrawal Jun 24, 2025
76eaf2a
Add file rotator
unshrawal Jun 24, 2025
0ec8f51
Remove unused imports
unshrawal Jun 24, 2025
52dc200
Stub file + overwrite warning
unshrawal Jun 24, 2025
20be94b
Configurable log level for console & log layer
unshrawal Jun 24, 2025
0095e1c
Clear directory and fix stubs
unshrawal Jun 25, 2025
9587e3a
Use buffered writer & flush at python exit
unshrawal Jun 30, 2025
8b5cc49
Refactor + Fix top level API
unshrawal Jun 30, 2025
3d3c47b
Don't flush in maybe_rotate
unshrawal Jun 30, 2025
f4d672a
Setup user API for zenoh logger
unshrawal Jul 6, 2025
d750562
Update stub
unshrawal Jul 6, 2025
e01a742
benchmark the loggers
unshrawal Jul 10, 2025
bbc930c
fix readme
unshrawal Jul 10, 2025
1802737
Remote pycache
unshrawal Jul 10, 2025
116803c
Update README
unshrawal Jul 10, 2025
89a00dd
Update README
unshrawal Jul 10, 2025
d1361d9
Fix warnings
unshrawal Jul 10, 2025
1def61a
Fix dependency issues
unshrawal Jul 10, 2025
ca2ce34
Add timestamp to JSON variable logfile
unshrawal Jul 12, 2025
6dadf7a
Add nonblocking writer from tracing for records
unshrawal Jul 12, 2025
147ab3b
Remove unused imports
unshrawal Jul 12, 2025
7f95b96
Release GIL
unshrawal Jul 12, 2025
5f83461
Support tracking lambdas
unshrawal Jul 13, 2025
7159c38
Don't obtain gil when unnecessary
unshrawal Jul 13, 2025
6481fea
Slight optimization
unshrawal Jul 13, 2025
4266d16
Configure Grafana Loki for log observability
unshrawal Jul 14, 2025
4543703
Use faster hashmap
unshrawal Jul 14, 2025
b391375
Update README
unshrawal Jul 14, 2025
86cb018
Glob over rotating filepaths
unshrawal Jul 14, 2025
4432d82
Add profiler to this repo
unshrawal Jul 14, 2025
fcc9e6c
Remove unused type
unshrawal Jul 14, 2025
e7d3ee5
Update README.md
unshrawal Jul 14, 2025
2489847
Remove expensive ChronoLocal syscall 2x improvement
unshrawal Jul 15, 2025
1010d40
Merge branch 'master' of https://github.com/unshrawal/logger
unshrawal Jul 15, 2025
0eb1613
Update README.md
unshrawal Jul 15, 2025
989b384
Update README.md
unshrawal Jul 15, 2025
c7059bf
Fix Zenoh for ROS applications
unshrawal Jul 18, 2025
608fe68
Generalize zenoh config
unshrawal Jul 19, 2025
1742fc5
Update readme
unshrawal Jul 19, 2025
a728004
Add 'opensourceleg/logging/observe/' from commit '1742fc527d1212cf00e…
unshrawal Aug 8, 2025
52bbe2d
Add observe as a subtree
unshrawal Aug 8, 2025
2cfccd4
Fix tutorials
unshrawal Aug 8, 2025
74221e2
fix logger implementation in tutorials and controllers
unshrawal Aug 8, 2025
ebd6bc7
Change from old logger to new
unshrawal Aug 8, 2025
28ec25d
Fixed all usages of old logger in OSL
unshrawal Aug 11, 2025
5ad0f80
RECONFIGURATION WORKS!
unshrawal Aug 12, 2025
b66c80a
Update stub file
unshrawal Aug 12, 2025
ee5d7cc
Merge commit 'b66c80a6686bf37c2f5d5ab24451e00422161257' into opensour…
unshrawal Aug 12, 2025
019b24a
Make logging configurable at runtime
unshrawal Aug 12, 2025
10e7075
Complete integration of rust logger with OSL
unshrawal Aug 12, 2025
6b92b9d
Add benching scripts
unshrawal Aug 12, 2025
3f8f605
replace observable with opensourceleg_rs
unshrawal Aug 12, 2025
fddaf2c
Fix old logger usage in OSL
unshrawal Aug 13, 2025
aa07c73
Setup global logger
unshrawal Aug 13, 2025
b0ce90e
Fix global logger issues
unshrawal Aug 13, 2025
0a45620
Senthur fixes
unshrawal Aug 14, 2025
6343563
Senthur suggestions
unshrawal Aug 14, 2025
c80365b
Merge pull request #469 from unshrawal/opensourceleg-rs
senthurayyappan Aug 20, 2025
6c50ae7
Change record to use our rotator instead of the generic one
unshrawal Sep 4, 2025
dda0045
Change the rotator to lazily create the file upon first write.
unshrawal Sep 4, 2025
0326a67
fix: Use given log filename with .csv or .json extension for data log…
unshrawal Sep 4, 2025
c42735b
fix: Merge functions and variables categories in data logs
unshrawal Sep 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
577 changes: 577 additions & 0 deletions Cargo.lock

Large diffs are not rendered by default.

10 changes: 8 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,13 @@ name = "opensourceleg_rs"
crate-type = ["cdylib"]

[dependencies]
pyo3 = { version = "0.25", features = ["extension-module"] }
numpy = "0.25"
nalgebra = "0.33"
serde = { version = "1.0", features = ["derive"] }
once_cell = "1.21.3"
serde = {version="1.0.219", features=["derive"]}
serde_json = "1.0.140"
pyo3 = { version = "0.25.1", features = ["extension-module"]}
tracing = "0.1.41"
tracing-appender = "0.2.3"
tracing-subscriber = { version = "0.3.19", features = ["chrono", "env-filter"] }
fxhash = "0.2.1"
22 changes: 2 additions & 20 deletions docs/tutorials/logging/configuring_logger.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This guide covers the various ways to configure the Logger for your needs. All e

### Basic Configuration

The Logger can be initialized with several basic parameters that control its behavior. This configuration sets up both file and console logging, with different verbosity levels for each output. The `log_path` specifies where log files are stored, while `file_name` determines the base name for log files. The `buffer_size` parameter helps optimize performance by controlling how frequently logs are written to disk.
The Logger can be initialized with several basic parameters that control its behavior. This configuration sets up both file and console logging, with different verbosity levels for each output. The `log_directory` specifies where log files are stored, while `log_name` determines the base name for log files.

```python
--8<-- "tutorials/logging/configuring_logger.py:8:17"
Expand All @@ -16,7 +16,7 @@ The Logger can be initialized with several basic parameters that control its beh

Log levels help you control the verbosity of your logging output. You can set different levels for file logging and console output, allowing you to have detailed logs in your files while keeping console output focused on more important messages.

The levels range from `DEBUG` (most verbose) to `CRITICAL` (most severe), giving you fine-grained control over what gets logged.
The levels range from `TRACE` (most verbose) to `ERROR` (most severe), giving you fine-grained control over what gets logged.

```python
--8<-- "tutorials/logging/configuring_logger.py:22:32"
Expand All @@ -30,24 +30,6 @@ The Logger includes built-in file management capabilities to help maintain your
--8<-- "tutorials/logging/configuring_logger.py:38:43"
```

## Advanced Configuration

### Custom Formatting

The Logger supports customizable log message formatting, allowing you to include various pieces of information in each log entry. You can include timestamps, log levels, file names, line numbers, and function names in your log messages. This flexibility helps you create log outputs that match your debugging and monitoring needs.

```python
--8<-- "tutorials/logging/configuring_logger.py:48:57"
```

### Buffer Management

Buffer management allows you to optimize the Logger's performance by controlling when logs are written to disk. A larger buffer size reduces I/O operations but increases the amount of memory used. You can also manually flush the buffer when needed, ensuring critical logs are written immediately.

```python
--8<-- "tutorials/logging/configuring_logger.py:63:69"
```

## Common Configurations

### Debug Configuration
Expand Down
50 changes: 23 additions & 27 deletions docs/tutorials/logging/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The `opensourceleg` library provides an easy-to-use yet powerful logging system

## Overview

Our `Logger` class builds upon Python's native `logging` module and adds several powerful features:
Our `Logger` class utilizes the Rust ecosystem's `tracing-subscriber` crate and adds several powerful features:

- **Data Logging**: Easily track variables and measurements over time
- **Flexible Output**: Write to both console and files simultaneously
Expand All @@ -18,11 +18,11 @@ Our `Logger` class builds upon Python's native `logging` module and adds several
First, create a logger instance in your Python file:

```python
from opensourceleg.logging import Logger, LogLevel
from opensourceleg.rust import Logger, LogLevel

logger = Logger(
log_path="./logs",
file_name="my_application"
Logger.init(
log_directory="./logs",
log_name="my_application.log"
)
```

Expand All @@ -32,13 +32,13 @@ It's as simple as:

```python
# Log some basic information
logger.info("Application started")
Logger.info("Application started")

# Log when something might be wrong
logger.warning("Battery level below 20%")
Logger.warning("Battery level below 20%")

# Log errors when they occur
logger.error("Failed to connect to sensor")
Logger.error("Failed to connect to sensor")
```

### 3. Understanding Log Levels
Expand All @@ -47,31 +47,28 @@ We provide five log levels, from least to most severe:

```python
# For detailed debugging information
logger.debug("Motor position: 123.4 degrees")
Logger.debug("Motor position: 123.4 degrees")

# For general information
logger.info("Robot initialized successfully")
Logger.info("Robot initialized successfully")

# For potential issues
logger.warning("Battery running low")
Logger.warn("Battery running low")

# For serious problems
logger.error("Failed to read sensor data")

# For fatal errors
logger.critical("System shutdown required")
Logger.error("Failed to read sensor data")
```

### 4. Customizing Your Logger

You can configure the logger to match your needs:

```python
logger = Logger(
log_path="./logs", # Where to save your log files
file_name="robot_test", # Name your log files (e.g., robot_test_2024_03_20.log)
buffer_size=1000, # Save to file every 1000 entries
console_level=LogLevel.INFO # Show INFO and above in console
Logger.init(
log_directory="./logs", # Where to save your log files
log_name="robot_test", # Name your log files (e.g., robot_test_2024_03_20.log)
print_stdout=True
stdout_level=LogLevel.INFO # Show INFO and above in console
)
```

Expand All @@ -85,14 +82,14 @@ Our logging system uses a singleton pattern, which means there's only one logger
from opensourceleg.logging import Logger

# Create your own logger instance
logger = Logger(
log_path="./logs",
file_name="my_experiment"
Logger.init(
log_directory="./logs",
log_name="my_experiment.log"
)

# Use your logger instance throughout your application
logger.info("Application started")
logger.debug("Configuration loaded")
Logger.info("Application started")
Logger.debug("Configuration loaded")
```

### About the Global Logger (For Internal Use)
Expand All @@ -113,12 +110,11 @@ from opensourceleg.logging import LOGGER # Meant for library internal use
- `INFO`: General operational messages
- `WARNING`: Something unexpected but not critical
- `ERROR`: Something failed but application continues
- `CRITICAL`: Application cannot continue

2. **Include Relevant Details**

```python
logger.error(f"Sensor read failed: {sensor_id}, Error: {error_message}")
Logger.error(f"Sensor read failed: {sensor_id}, Error: {error_message}")
```

f-strings are preferred over string interpolation. This is because they are more readable and easier to debug.
Expand Down
18 changes: 5 additions & 13 deletions docs/tutorials/logging/logging_data.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,11 @@ Here's how to set it up:

In this example:

- `track_function()` tells the logger to monitor a specific variable
- The first argument is a function that returns the current value
- The second argument is the variable name (used in CSV headers)
- You can track multiple variables simultaneously
- `track_attributes()` tells the logger to monitor attributes of the `robot` object.
- The first argument is the object to track from
- The second argument is the attribute name(s)
- In the logs, the specific attributes will be prefixed with the name of the object.
- Variables are added to a buffer every time `update()` is called
- The buffer is flushed based on the `buffer_size` parameter in the `Logger` constructor or manually with `flush_buffer()`
- The buffer is automatically flushed when the Logger is destroyed
- `track_functions()` tells the logger to monitor a specific variable
- This function takes one argument, a dictionary where the key defines the identifier/tag, the value defines the function to be called when `Logger.record()` is called
- `trace_variables()` tells the logger to monitor variables.
- This function takes one argument, a dictionary where the key defines the identifier/tag, the value defines what will be logged
- Variables are added to a buffer every time `record()` is called


```python
Expand Down Expand Up @@ -107,7 +101,6 @@ Your logged data can be easily analyzed using common data analysis packages like

1. **High-Frequency Data**

- Increase buffer size for better performance
- Consider logging only essential variables
- Use appropriate data types

Expand All @@ -120,7 +113,6 @@ Your logged data can be easily analyzed using common data analysis packages like
3. **System Performance**

- Profile your logging impact
- Adjust buffer sizes as needed
- Balance logging frequency with requirements

If you have any questions or need further assistance, please post on the [Open Source Leg community forum](https://opensourceleg.org/community).
12 changes: 6 additions & 6 deletions examples/fsm_walking_ankle_python_controller.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
from opensourceleg.actuators.base import CONTROL_MODES
from opensourceleg.actuators.dephy import DephyActuator
from opensourceleg.control.fsm import State, StateMachine
from opensourceleg.logging.logger import Logger
from opensourceleg.robots.osl import OpenSourceLeg
from opensourceleg.rust import Logger
from opensourceleg.sensors.loadcell import NBLoadcellDAQ
from opensourceleg.utilities import SoftRealtimeLoop

Expand Down Expand Up @@ -204,9 +204,9 @@ def create_simple_walking_fsm(osl: OpenSourceLeg) -> StateMachine:
}

clock = SoftRealtimeLoop(dt=1 / FREQUENCY)
fsm_logger = Logger(
log_path="./logs",
file_name="fsm.log",
Logger.init(
log_directory="./logs",
log_name="fsm.log",
)

osl = OpenSourceLeg(
Expand All @@ -217,7 +217,7 @@ def create_simple_walking_fsm(osl: OpenSourceLeg) -> StateMachine:

osl_fsm = create_simple_walking_fsm(osl)

with fsm_logger, osl, osl_fsm:
with osl, osl_fsm:
osl.update()
osl.home()

Expand All @@ -237,7 +237,7 @@ def create_simple_walking_fsm(osl: OpenSourceLeg) -> StateMachine:

osl.ankle.set_output_position(np.deg2rad(osl_fsm.current_state.ankle_theta))

fsm_logger.info(
Logger.info(
f"T: {t:.3f}s, "
f"Current state: {osl_fsm.current_state.name}; "
f"Loadcell Fz: {osl.loadcell.fz:.3f} N; "
Expand Down
10 changes: 5 additions & 5 deletions examples/fsm_walking_python_controller.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
from opensourceleg.actuators.base import CONTROL_MODES
from opensourceleg.actuators.dephy import DephyActuator
from opensourceleg.control.fsm import State, StateMachine
from opensourceleg.logging.logger import Logger
from opensourceleg.robots.osl import OpenSourceLeg
from opensourceleg.rust import Logger
from opensourceleg.sensors.encoder import AS5048B
from opensourceleg.sensors.loadcell import NBLoadcellDAQ
from opensourceleg.utilities import SoftRealtimeLoop
Expand Down Expand Up @@ -232,9 +232,9 @@ def lswing_to_estance(osl: OpenSourceLeg) -> bool:
}

clock = SoftRealtimeLoop(dt=1 / FREQUENCY)
fsm_logger = Logger(
log_path="./logs",
file_name="fsm.log",
Logger.init(
log_directory="./logs",
log_name="fsm.log",
)

osl = OpenSourceLeg(
Expand Down Expand Up @@ -276,7 +276,7 @@ def lswing_to_estance(osl: OpenSourceLeg) -> bool:
osl.knee.set_output_position(np.deg2rad(osl_fsm.current_state.knee_theta))
osl.ankle.set_output_position(np.deg2rad(osl_fsm.current_state.ankle_theta))

fsm_logger.info(
Logger.info(
f"T: {t:.3f}s, "
f"Current state: {osl_fsm.current_state.name}; "
f"Loadcell Fz: {osl.loadcell.fz:.3f} N; "
Expand Down
Loading