Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mc mirror --watch --remove --overwrite doesn't replicate deletions with versionned and unversionned bucket #5020

Closed
the-glu opened this issue Aug 15, 2024 · 4 comments

Comments

@the-glu
Copy link

the-glu commented Aug 15, 2024

Expected behavior

I'm doing a

mc mirror --watch --remove --overwrite

between two clusters. The source cluster is using versioning, the target cluster is not.

When deleting a file in source, mc doesn't replicate the deletion on the old cluster.

If deleting all version of a file on the source, mc does replicate the deletion on the new cluster.

Actual behavior

I would except (or at least to have an option) mc mirror to delete the file on the target cluster when a file is marked for deletion on the source cluster.

Steps to reproduce the behavior

mc mirror --watch --remove --overwrite with two cluster/bucket with and without versionning

mc --version

mc version RELEASE.2024-06-12T14-34-03Z (commit-id=e7c9a733c680fe62066d24e8718f81938b4f6606

(minio servers, both source and target: minio version RELEASE.2024-06-13T22-53-53Z)

@harshavardhana
Copy link
Member

between two clusters. The source cluster is using versioning, the target cluster is not.

mc mirror only works on latest versions, it will not replicate all versions.

When deleting a file in source, mc doesn't replicate the deletion on the old cluster.

Deleting a file under versioning by default is soft delete, permanent deletes are replicated however (only for latest version)

If deleting all version of a file on the source, mc does replicate the deletion on the new cluster.

what I just said.

@harshavardhana
Copy link
Member

If you want all versions to be replicated use MinIO's server side replication.

@the-glu
Copy link
Author

the-glu commented Aug 15, 2024

mc mirror only works on latest versions, it will not replicate all versions.

Correct, but in that case it's not what is happening. I keep the previous "not deleted version".

Deleting a file under versioning by default is soft delete, permanent deletes are replicated however (only for latest version)

Yes I know that. However this have the effect of having the file as "deleted" for non-aware calls, e.g. is doesn't appear in listing unless "Show deleted objects" is checked.

I would really assume that it's more natural, if the latest version of the file is deleted, to replicate the delete via mc mirror.

If you want all versions to be replicated use MinIO's server side replication.

In that case I don't want all versions to be replicated, only the latest one :)

Again if the latest one is a delete, the 'soft-delete|delete' is not replicated to the second server, who will then have the 'version - 1'.

@harshavardhana
Copy link
Member

This is outside the scope of mc mirror - it is never designed to support all the complex requirements.

Use server-side replication.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants