Skip to content

Latest commit

 

History

History
371 lines (310 loc) · 13.5 KB

aws_s3_compat.md

File metadata and controls

371 lines (310 loc) · 13.5 KB
Raw

S3 API support

Reference:

Legend
🟢 Supported
🟡 Partially supported
🔵 Not supported yet, but will be in future
🔴 Not applicable or will never be supported

Object

Method Comments
🟢 CopyObject Done on gateway side
🟢 DeleteObject
🟢 DeleteObjects aka DeleteMultipleObjects
🟢 GetObject
🔴 GetObjectTorrent We don't plan implementing BT gateway
🟢 HeadObject
🟢 ListParts Parts loaded with MultipartUpload
🟢 ListObjects
🟢 ListObjectsV2
🟢 PutObject Content-MD5 header deprecated
🔵 SelectObjectContent Need to have some Lambda to execute SQL
🔵 WriteGetObjectResponse Waiting for Lambda to be developed
🟢 GetObjectAttributes
  • DeleteObjects limited by max amount of objects which can be deleted per request. See max_object_to_delete_per_request parameter.
  • For calculating object ETag, we use SHA256 hash instead of MD5.
  • PutObject into a container with public-write permissions as an anonymous user (for instance, with CLI option --no-sign-request) is impossible, if try to set custom ACL for the object. It happens because container ACL rules may be changed only by container owner.
  • Header X-Amz-Meta-Neofs-Copies-Number can be used with PutObject, CopyObject, CreateMultipartUpload methods to set object copies number. Otherwise, the default value from config will be used. See neofs section for more details.
    • Use metadata neofs-copies-number parameter for aws CLI. For instance:
    aws s3api put-object --endpoint $S3HOST --bucket $BUCKET --key $OBJECT_KEY --body /path/to/file.txt --metadata neofs-copies-number=3

ACL

For now there are some limitations:

  • Bucket policy supports only one Principal per Statement. Principal must be "AWS": "*" or "*" (to refer all users) or "CanonicalUser": "NiskPF9pfRMzg7V7PeB4d6ogLzu74a1L2Q" (base58 encoded address of desired user).
{
  "Statement": [
    {
      "Principal": "*"
    }
  ]
}
{
  "Statement": [
    {
      "Principal": {
        "AWS": "*"
      }
    }
  ]
}
  • Resource in bucket policy is a string value or array of strings. Each item MUST contain bucket name, CAN contain object name (wildcards are not supported):
{
  "Statement": [
    {
      "Resource": "arn:aws:s3:::bucket"
    }
  ]
}
{
  "Statement": [
    {
      "Resource": [
        "arn:aws:s3:::bucket",
        "arn:aws:s3:::bucket/some/object"
      ]
    }
  ]
}
  • Action is a string value or array of strings:
{
  "Statement": [
    {
      "Action": "s3:PutObject"
    }
  ]
}
{
  "Statement": [
    {
      "Action": ["s3:PutObject", "s3:PutObjectAcl"]
    }
  ]
}
  • AWS conditions and wildcard are not supported in resources
  • Only CanonicalUser (with hex encoded public key) and All Users Group are supported in ACL. Authenticated Users group is not supported. It is a part of All Users Group and can't be separated from it.
  • It is not possible to remove GRANTS from container owner. Using PutObjectAcl with empty grants has no effect to GRANTS for container owner, despite method completes without error.
{
    "Owner": {"DisplayName": "NiskPF9pfRMzg7V7PeB4d6ogLzu74a1L2Q","ID": "NiskPF9pfRMzg7V7PeB4d6ogLzu74a1L2Q"},
    "Grants": []
}
Method Comments
🟡 GetObjectAcl See Limitations
🟡 PutObjectAcl See Limitations

Locking

For now there are some limitations:

  • Retention period can't be shortened, only extended.
  • You can't delete locks or object with unexpired lock. This means PutObjectLegalHold with OFF status raise Unsupported error.
Method Comments
🟡 GetObjectLegalHold
🟢 GetObjectLockConfiguration GetBucketObjectLockConfig
🟡 GetObjectRetention
🟡 PutObjectLegalHold
🟢 PutObjectLockConfiguration PutBucketObjectLockConfig
🟡 PutObjectRetention

Multipart

Should be supported soon.

Method Comments
🟢 AbortMultipartUpload
🟢 CompleteMultipartUpload
🟢 CreateMultipartUpload
🟢 ListMultipartUploads
🟢 ListParts
🟢 UploadPart
🟢 UploadPartCopy

Tagging

Method Comments
🟢 DeleteObjectTagging
🟢 GetObjectTagging
🟢 PutObjectTagging

Versioning

See also GetObject and other method parameters.

Method Comments
🟢 ListObjectVersions ListBucketObjectVersions
🔵 RestoreObject

Bucket

Method Comments
🟢 CreateBucket PutBucket
🟢 DeleteBucket
🟢 GetBucketLocation
🟢 HeadBucket
🟢 ListBuckets
🔵 PutPublicAccessBlock
🔵 GetPublicAccessBlock
  • CreateBucket method allows you to select a placement policy using the LocationConstraint parameter in the AWS CLI. The policy name should be passed as a value.
    • Policies mapping can be defined via:
      • Authmate during bucket creation. These policies are available only for generated credentials.
      • Gate configuration. These policies are shared and available for all gate clients.
    • Example: aws s3api create-bucket --bucket $BUCKET --endpoint $S3HOST --create-bucket-configuration LocationConstraint=$POLICY_NAME

Acceleration

Method Comments
🔴 GetBucketAccelerateConfiguration GetBucketAccelerate
🔴 PutBucketAccelerateConfiguration

ACL

Method Comments
🟡 GetBucketAcl See ACL limitations
🟡 PutBucketAcl See ACL Limitations

Analytics

Method Comments
🔵 DeleteBucketAnalyticsConfiguration
🔵 GetBucketAnalyticsConfiguration
🔵 ListBucketAnalyticsConfigurations
🔵 PutBucketAnalyticsConfiguration

CORS

Method Comments
🟢 DeleteBucketCors
🟢 GetBucketCors
🟢 PutBucketCors

Encryption

Method Comments
🔵 DeleteBucketEncryption
🔵 GetBucketEncryption
🔵 PutBucketEncryption

Inventory

Method Comments
🔵 DeleteBucketInventoryConfiguration
🔵 GetBucketInventoryConfiguration
🔵 ListBucketInventoryConfigurations
🔵 PutBucketInventoryConfiguration

Lifecycle

Method Comments
🔵 DeleteBucketLifecycle
🔵 GetBucketLifecycle
🔵 GetBucketLifecycleConfiguration
🔵 PutBucketLifecycle
🔵 PutBucketLifecycleConfiguration

Logging

Method Comments
🔵 GetBucketLogging
🔵 PutBucketLogging

Metrics

Method Comments
🔵 DeleteBucketMetricsConfiguration
🔵 GetBucketMetricsConfiguration
🔵 ListBucketMetricsConfigurations
🔵 PutBucketMetricsConfiguration

Notifications

Method Comments
🔵 GetBucketNotification
🔵 GetBucketNotificationConfiguration
🔵 ListenBucketNotification non-standard?
🔵 PutBucketNotification
🔵 PutBucketNotificationConfiguration

Ownership controls

Method Comments
🔵 DeleteBucketOwnershipControls
🔵 GetBucketOwnershipControls
🔵 PutBucketOwnershipControls

Policy and replication

Method Comments
🔵 DeleteBucketPolicy
🔵 DeleteBucketReplication
🔵 DeletePublicAccessBlock
🟡 GetBucketPolicy See ACL limitations
🔵 GetBucketPolicyStatus
🔵 GetBucketReplication
🟢 PostPolicyBucket Upload file using POST form
🟡 PutBucketPolicy See ACL limitations
🔵 PutBucketReplication

By default bucket ACLs is disabled. See details here. In case you need to disable ACLs manually (for instance your bucket has ACLs enabled) you should use PutBucketPolicy command with the next policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BucketOwnerEnforced",
            "Action": "*",
            "Effect": "Deny",
            "Resource": "*",
            "Condition": {
                "StringNotEquals": {
                    "s3:x-amz-object-ownership": "BucketOwnerEnforced"
                }
            }
        }
    ]
}

In case you need to enable ACLs (not recommended) option you should use PutBucketPolicy command with the next policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BucketEnableACL",
            "Action": "s3:PutObject",
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Request payment

Method Comments
🔴 GetBucketRequestPayment
🔴 PutBucketRequestPayment

Tagging

Method Comments
🟢 DeleteBucketTagging
🟢 GetBucketTagging
🟢 PutBucketTagging

Tiering

Method Comments
🔵 DeleteBucketIntelligentTieringConfiguration
🔵 GetBucketIntelligentTieringConfiguration
🔵 ListBucketIntelligentTieringConfigurations
🔵 PutBucketIntelligentTieringConfiguration

Versioning

Method Comments
🟢 GetBucketVersioning
🟢 PutBucketVersioning

Website

Method Comments
🔵 DeleteBucketWebsite
🔵 GetBucketWebsite
🔵 PutBucketWebsite

Metadata

Each meta parameter value must be non-empty. If any parameter value is an empty, then "Your metadata headers are not supported." error will be returned on the object put operation.