Skip to content

Commit d30b11f

Browse files
rtylerfvaleye
authored andcommitted
chore: add a regression test for the DeltaTable.count() method
Since count is approximate, I made sure to update the docstrings for the function to helpfully inform users Signed-off-by: R. Tyler Croy <[email protected]>
1 parent dd058e6 commit d30b11f

File tree

2 files changed

+18
-3
lines changed

2 files changed

+18
-3
lines changed

python/deltalake/table.py

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -530,15 +530,24 @@ def _backwards_enumerate(
530530

531531
def count(self) -> int:
532532
"""
533-
Get the row count based on the history of the DeltaTable.
533+
Get the approximate row count based on file statistics added to the Delta table.
534+
535+
This requires that add actions have been added to the Delta table with
536+
per-file statistics enabled. Because this is an optional field this
537+
"count" will be less than or equal to the true row count of the table.
538+
In order to get an exact number of rows a full table scan must happen
534539
535540
Returns:
536-
The number of rows for this specific table
541+
The approximate number of rows for this specific table
537542
"""
538543
total_rows = 0
539544

540545
for value in self.get_add_actions().column("num_records").to_pylist():
541-
total_rows += value
546+
# Add action file statistics are optional and so while most modern
547+
# tables are _likely_ to have this information it is not
548+
# guaranteed.
549+
if value is not None:
550+
total_rows += value
542551

543552
return total_rows
544553

python/tests/test_table_read.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,12 @@ def test_read_simple_table_to_dict():
5757
).read_all()["id"].to_pylist() == [5, 7, 9]
5858

5959

60+
def test_table_count():
61+
table_path = "../crates/test/tests/data/COVID-19_NYT"
62+
dt = DeltaTable(table_path)
63+
assert dt.count() == 1
64+
65+
6066
class _SerializableException(BaseException):
6167
pass
6268

0 commit comments

Comments
 (0)