Implement limit push down for IcebergTableProvider

[krinart]

Which issue does this PR close?

N/A

What changes are included in this PR?

Previously _limit was ignored in IcebergTableProvider::scan:

iceberg-rust/crates/integrations/datafusion/src/table/mod.rs

Lines 149 to 163 in aad9e2e

	_state: &dyn Session,
	projection: Option<&Vec<usize>>,
	filters: &[Expr],
	_limit: Option<usize>,
	) -> DFResult<Arc<dyn ExecutionPlan>> {
	Ok(Arc::new(IcebergTableScan::new(
	self.table.clone(),
	self.snapshot_id,
	self.schema.clone(),
	projection,
	filters,
	)))
	}
	fn supports_filters_pushdown(

This PR propagates limit all the way to the ArrowReaderBuilder.

Note: limit push down is only applied to each batch which means that IcebergTableProvider::scan may potentially return more records than specified by limit.

Which is OK according to TableProvider::scan documentation:

If limit is specified, must only produce at least this many rows, (though it may return more).

Are these changes tested?

Unit tests

[ZENOTME]

Should we enable with_page_index as suggested by doc: https://docs.rs/parquet/latest/parquet/arrow/arrow_reader/struct.ArrowReaderBuilder.html#method.with_limit

[krinart]

Thanks! I extended should_load_page_index logic and ArrowReaderOptions is initialized with with_page_index(should_load_page_index).

[ZENOTME]

Thanks @krinart

[krinart]

Happy to help @ZENOTME and thanks for the feedback!
Any suggestions on how to proceed and merge this PR?

[krinart]

Hi. Anything else I can do on my side to get this merged?