Add integration tests for transactions #2056
Conversation
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
tests/ducktape/run_ducktape_test.py
Outdated
| """Run the ducktape test based on specified type""" | ||
| parser = argparse.ArgumentParser(description="Confluent Kafka Python - Ducktape Test Runner") | ||
| parser.add_argument('test_type', choices=['producer', 'consumer', 'producer_sr'], | ||
| parser.add_argument('test_type', choices=['producer', 'consumer', 'producer_sr', 'transactions'], |
There was a problem hiding this comment.
Will it be possible to run specific tests if we use "producer" string parameters . Eg.
ducktape tests/ducktape/test_producer.py::SimpleProducerTest.test_basic_produce
There was a problem hiding this comment.
Currently I extracted the transaction tests into a standalone file as I think those involve both producer + consumer
| from confluent_kafka.aio import AIOConsumer | ||
|
|
||
|
|
||
| class TransactionsTest(Test): |
There was a problem hiding this comment.
I think we need overall design on how to perf test transaction. One of options can be : Run n number of transaction end to end for async vs sync and compare performance
There was a problem hiding this comment.
Good point - I was mostly doing integration tests to verify behavior, but maybe we should benchmark test transactions as well. Let me think about this
There was a problem hiding this comment.
Making transactions an optional configuration of the perf tests seems best as a follow-up. Make the perf tests essentially a config driven execution engine for parameters of a performance scenario we can pick and choose for a given context.
tests/ducktape/producer_strategy.py
Outdated
| raise NotImplementedError() | ||
|
|
||
|
|
||
| class AsyncProducerStrategy(ProducerStrategy): |
There was a problem hiding this comment.
Added it temporarily to unblock this PR. Will remove once the PR to add is merged and will rebase
Analysis Details28 Issues
Coverage and Duplications
Project ID: confluent-kafka-python |
What
Add integration tests for transactions:
test_commit_transaction: committed messages visible toread_committed.test_abort_transaction_then_retry_commit: abort invisible; retry+commit visible.test_send_offsets_to_transaction: offsets committed correctly viasend_offsets_to_transaction.test_commit_multiple_topics_partitions: atomic commit across topics/partitions.Checklist
References
JIRA: https://confluentinc.atlassian.net/browse/NONJAVACLI-3991
Test & Review
Open questions / Follow-ups