这里是不是直接报错比较好

Done

如果dist_tensor是一个EagerParam，这里的行为应该是什么样的？

加了 EagerParam 的处理

this modification would lead to segment fault if user trying to print program with dist attr !
we should also adapt the str function of dist_tensor (static mode object) where assume that all node in program have dist_attr in dist_context

It's ok to print the variable after removing it from dist_context. I tested it with the following sample code:

ori_tensor = paddle.static.data(
    name="input",
    shape=[4, 1024, 512],
    dtype='float32',
)
d_tensor = dist.shard_tensor(ori_tensor, self.mesh, [Shard(0)])
default_dist_context = get_default_distributed_context()
dense_tensor = dist.unshard_dtensor(d_tensor)
print(dense_tensor) # output dense var
print(d_tensor) # output dense var

When dist_tensor is not in dist_context, str function will not add distributed attribute:

dist_context = get_default_distributed_context()
dist_tensor = dist_context.get_dist_tensor_for_program(self)
if dist_tensor is not None:
    var_str += ", {name} = {value}".format(
        name="dist_attr", value=dist_tensor
    )
return var_str

It seems original dense tensor is a little bit confusing. Actually, it does 2 things,

1. Make it Replicated on all mesh dims
2. Convert to Paddle.Tensor

Done, modified the doc.

Is the dist_tensor always in default context?

shard_tensor will add the dist_tensor to default context. If it's not in default context, default_dist_ctx._dist_tensors_for_program.pop(serial_tensor_id, None) will do nothing.

if they are pair APIs, unshard_tensor would be more suitable.

Only a distributed tensor (dtensor) can be unsharded, and only a dense tensor can be sharded with shard_tensor api. So, here named it unshard_dtensor.