Merge pull request #19 from hameerabbasi/dtensor-wrapping-docs

Add documentation on `DTensor` tests.

Author: hameerabbasi

docs/dtensor_wrap.md

@@ -0,0 +1,41 @@
+# Wrapping `ComplexTensor` in `DTensor`
+
+## Mechanism
+
+The wrapping is done implicitly, with no special code in either `ComplexTensor` or `DTensor`.
+
+## Tests
+
+The `DTensor` composition tests are placed in
+[`src/complex_tensor/test/test_ops.py::TestComplexTensor.test_distributed`](https://github.com/openteams-ai/pytorch-complex-tensor/blob/main/src/complex_tensor/test/test_ops.py).
+
+We use the `OpInfo`s to perform the tests. We first wrap every `torch.Tensor` with a complex
+`dtype` inside a `ComplexTensor`. We then proceed to wrap all tensors (regular and complex) into
+a `torch.distributed.tensor.DTensor`.
+
+## Outstanding Issues
+
+### Missing sharding strategies
+
+Some operations are missing sharding strategies in the `DTensor` implementation, making it
+impossible to compose certain ops, or other ops that use them. Examples are `aten.ne`, `aten.all`
+and `aten.cumprod`.
+
+### Missing scalar support
+
+Most of the sharding strategies in the `DTensor` implementation assume that input tensors all
+have `ndim >= 1`. This makes it impossible to support scalar inputs that are generated by the
+`OpInfo` samples in any way.
+
+### Can Only Be Composed Inside
+
+All `ComplexTensor`s must be placed inside a `DTensor` for tests to work. The reason for this
+is that `aten.complex` (an op to create an interleaved `Tensor` from its real and imaginary
+parts) is one of the ops that doesn't have a registered sharding strategy. `aten.allclose`
+also doesn't work due to `aten.all` not working.
+
+Since all of these are used extensively during testing, it makes it impossible to test
+a `ComplexTensor` with its real and imaginary parts being a `DTensor`. We can, however,
+test a `DTensor` that wraps a `ComplexTensor`, as these ops are not necessarily used on
+a `DTensor` during those tests; it is usually gathered into a local tensor, therefore
+making these ops possible.