All Documents
Current Document
Content is empty
If you don't find the content you expect, please try another search term
Content is empty
If you don't find the content you expect, please try another search term
Last updated：2021-09-14 16:18:22
KS3 SDK for Java user guide
The sample code in this topic is for reference only. In actual use, refer to the KS3 API documents and modify the parameters as required.
All dependency JAR packages of the project and the SDK JAR package are stored in the lib directory.
This SDK is applicable to Java 5.0 and later. It is created based on KS3 API operations.
Set up the development environment for Java 5.0 or later.
Download the KS3 SDK for Java.
Add the Maven dependency.
<dependency>
<groupId>com.ksyun</groupId>
<artifactId>ks3-kss-java-sdk</artifactId>
<version>1.0.2</version>
</dependency>Alternatively, reference all JAR packages in the lib directory.
Ks3ClientConfig config = new Ks3ClientConfig();
/**
* Set endpoints.
* China (Beijing) | ks3-cn-beijing.ksyuncs.com
* China (Shanghai) | ks3-cn-shanghai.ksyuncs.com
* China (Hong Kong) | ks3-cn-hk-1.ksyuncs.com
* China (Guangzhou) | ks3-cn-guangzhou.ksyuncs.com
* China (Qingdao) | ks3-cn-qingdao.ksyuncs.com
* Finance (BEIJING) | ks3-jr-beijing.ksyuncs.com
* Finance (SHANGHAI) | ks3-jr-shanghai.ksyuncs.com
* Russia | ks3-rus.ksyuncs.com
* Singapore | ks3-sgp.ksyuncs.com
* If you use a custom domain name, set the Endpoint parameter to the custom domain name and the DomainMode parameter to true.
*/
config.setEndpoint("ks3-cn-beijing.ksyuncs.com"); // China (Beijing) is used as an example.
/**
* true: a custom domain name is used for access.
* false: the public or private domain name of KS3 is used for access. The default value is false.
*/
config.setDomainMode(false);
config.setProtocol(PROTOCOL.http);
/**
* true: the domain name is in the format of endpoint/{bucket}/{key}.
* false: the domain name is in the format of {bucket}.endpoint/{key}.
* If the DomainMode parameter is set to true, the PathStyleAccess parameter can be left unspecified.
*/
config.setPathStyleAccess(false);
HttpClientConfig hconfig = new HttpClientConfig();
// You can specify properties of the HTTP client, such as the proxy, timeout period, and maximum number of retries, in the HttpClientConfig parameter.
config.setHttpClientConfig(hconfig);
Ks3 client = new Ks3Client("<Your AccessKeyID>","<Your SecretAccessKey>",config);
/* Alternatively, use the following format: client.setKs3config(config); */
This SDK uses commons-logging.
Use Log4j:
Reference Log4j-related JAR packages.
Create the log4j.properties file. Sample configurations:
log4j.logger.com.ksyun.ks3=DEBUG,stdout
log4j.logger.org.apache.http=DEBUG,stdout
log4j.logger.org.apache.http.wire=ERROR,stdout
log4j.addivity.org.apache=true
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss SSS} %-5p [%t]%C{1}.%M(%L) - %m%nUse Logback:
Exclude commons-logging when you reference the SDK. Instead, reference Logback-related packages, including but not limited to jcl-over-slf4j.
Configure Logback.
Register a Kingsoft Cloud account and activate KS3.
Initialize the KS3 client.
Ks3 client = new Ks3Client("<Your AccessKeyID>","<Your SecretAccessKey>");Use custom configurations. For more information, see section 3.1 "Configure parameters."
Ks3 client = new Ks3Client("<Your AccessKeyID>","<Your SecretAccessKey>",config);
Alternatively, use the following format: client.setKs3config(config);
An object is the basic data unit of user operations in KS3. An object can store 0–48.8 TB of data. An object contains a key and data. The key is the object name. The key is UTF-8 encoded, and the encoded key is 1 to 1,024 characters in length.
A key is an object name. The key is UTF-8 encoded, and the encoded key is 1 to 1,024 characters in length. The key can contain slashes (/). In this case, a directory structure is automatically organized in the console.
For more information, see Terms.
Before you start, read Terms.
Log in to the KS3 console and click Bucket in the left navigation pane. On the Bucket list page, click Create Bucket. In the Create Bucket panel, set the Bucket name parameter and other parameters.
/**
* Upload new File ("<filePath>") to the <Bucket name> bucket and rename it <key>.
*/
public void putObjectSimple(){
PutObjectRequest request = new PutObjectRequest("<Bucket name>",
"<key>", new File("<filePath>"));
// Upload a public object.
//request.setCannedAcl(CannedAccessControlList.PublicRead);
client.putObject(request);
}Note:
- This method requires three parameters: the name of the bucket created in section 4.1 "Create a bucket", the object path in KS3, such as image.jpg or 2015/10/19/image.jpg, and the object to be uploaded.
4.3 Upload an object by using a form
- Read the KS3 form upload protocol in POST Object and the signature authentication method of form upload in Post policy.
Log in to the KS3 console and click Bucket in the left navigation pane. On the Bucket list page, click the name of the target bucket. On the page that appears, click Bucket Settings. On the Bucket Settings tab, click CORS Configuration. On the CORS Configuration tab, configure a cross-origin resource sharing (CORS) rule. The CORS rule is required when you use JavaScript for cross-origin resource sharing. For more information, see the Fetch Living Standard.
Upload an object by using the KS3 SDK for JavaScript. For more information, see KS3 SDK for Java.
Calculate the policy and signature in the SDK for JavaScript by using the following method:
/**
If you are not familiar with the KS3 protocol, we recommend that you use this method to obtain the latest signature each time you upload an object.
*/
public PostObjectFormFields postObjectSimple(){
/**
* Provide all form fields, except KSSAccessKeyId, Signature, file, and Policy, in postData and unknownValueField. Otherwise, an HTTP 403 error is returned when you upload an object by using the generated signature.</br>
* Put form fields with known values in postData and form fields with unknown values in unknownValueField. For example, some form fields with random values may be added to an upload control, and these form fields must be put in unknownValueField.</br>
*
*/
Map<String,String> postData = new HashMap<String,String>();
// If an access control list (ACL) is configured when you upload an object by using the SDK for JavaScript, add the following code and ensure that the value is the same as that in the SDK. If no ACL is configured, delete the following code:
postData.put("acl","public-read");
// Provide the key in the SDK for JavaScript.
postData.put("key","20150115/Chinese/${filename}");
List<String> unknowValueField = new ArrayList<String>();
// The name field is automatically added when you upload an object by using the SDK for JavaScript. Therefore, the following code must be added:
unknowValueField.add("name");
// If the key provided for calculating the signature does not contain the ${filename} placeholder, the second parameter can be an empty string, because the key required for calculating the policy is obtained after ${filename} in the original key is replaced.
PostObjectFormFields fields = client.postObject("<Your bucket name>", "<Name of the object to be uploaded, without the path>", postData, unknownValueField);
fields.getKssAccessKeyId();
fields.getPolicy();
fields.getSignature();
return fields;
}Common errors:
- When you upload an object, the browser sends an OPTIONS request and a POST request. If an HTTP 403 error is returned for the OPTIONS request, verify the CORS configurations. If an HTTP 403 error is returned for the POST request, check whether the policy is exactly generated based on the form fields, or Base64-encode the policy and compare it with the form fields. For more information about the comparison rules, see Post policy.
4.4 Query the access URL of an object
Create a URL in the format of http://{bucket}.{endpoint}/{key}, for example, http://test-bucket.ks3-cn-beijing.ksyuncs.com/2015/10/19/image.jpg
. In the URL example, {bucket} is set to test-bucket, {endpoint} is set to ks3-cn-beijing.ksyun.com, and {key} is set to 2015/10/19/image.jpg.
Generate an access URL by using the following code:
// Generate an external link that is to expire 1,000s later.
client.generatePresignedUrl(<bucket>,<key>,1000);
// Generate an external link that is to expire 1,000s later with response headers overridden.
ResponseHeaderOverrides overrides = new ResponseHeaderOverrides();
//overrides.setContentType("text/html");
//.......
client.generatePresignedUrl(<bucket>,<key>,1000,overrides);Common errors:
- If the object does not exist, a NoSuchKey error is returned.
- If you attempt to access a private object by using a public URL, an AccessDenied error is returned.
- If the access URL of a private object expires, an URLExpired error is returned.
- The validity period of 1,000s is counted based on the local time of the client.
5. Bucket operations
5.1 Query all buckets
You can call the ListBucket operation to query all your buckets.
public List<Bucket> listBuckets(){
List<Bucket> buckets = client.listBuckets();
for(Bucket bucket:buckets){
// Query the bucket creation time.
bucket.getCreationDate();
// Query the bucket name.
bucket.getName();
// Query the Base64-encoded ID of the bucket owner.
bucket.getOwner();
// Query the region to which the bucket belongs.
bucket.getRegion();
// Query the bucket type. Valid values: BucketType.Normal and BucketType.Archive. The value BucketType.Normal indicates a non-Archive bucket, and the value BucketType.Archive indicates an Archive bucket.
bucket.getType();
}
return buckets;
}
/**
* Create a bucket in the simplest way.
* Create a private bucket in the China (Beijing) region by using the default configurations.
*/
public void createBucketSimple(){
client.createBucket("<Your bucket name>");
}
/**
* Set the region and access permission for the bucket when you create it.
*/
public void createBucketWithConfig(){
CreateBucketRequest request = new CreateBucketRequest("<Your bucket name>");
// Set the region to which the bucket belongs.
CreateBucketConfiguration config = new CreateBucketConfiguration(REGION.BEIJING);
request.setConfig(config);
// Set the bucket type. Valid values: BucketType.Normal and BucketType.Archive. The value BucketType.Normal indicates a non-Archive bucket, and the value BucketType.Archive indicates an Archive bucket. The default value is BucketType.Normal.
request.setBucketType(BucketType.Normal);
// Set the access permission of the bucket.
request.setCannedAcl(CannedAccessControlList.Private);
// Perform the operation.
client.createBucket(request);
}Note:
- The bucket name is globally unique.
5.3 Configure the access permission of a bucket
For more information about the access permissions of buckets, see ACL.
/**
* Configure a preset ACL to manage the access permission of a bucket. The permission in the preset ACL can be set to private, public-read, or public-read-write. The private permission indicates that only the bucket owner can access the bucket. The public-read permission indicates that all users have the read permission on the bucket. The public-read-write permission indicates that all users have the read and write permissions on the bucket.
*/
public void putBucketAclWithCannedAcl(){
PutBucketACLRequest request = new PutBucketACLRequest("<Your bucket name>");
// Set the ACL permission to private.
request.setCannedAcl(CannedAccessControlList.Private);
// Set the ACL permission to public-read.
//request.setCannedAcl(CannedAccessControlList.PublicRead);
// Set the ACL permission to public-read-write.
//request.setCannedAcl(CannedAccessControlList.PublicReadWrite);
client.putBucketACL(request);
}
/**
// Customize the access permission of a bucket.
*/
public void putBucketAclWithAcl(){
PutBucketACLRequest request = new PutBucketACLRequest("<Your bucket name>");
AccessControlList acl = new AccessControlList();
// Grant the read permission on the bucket to the user whose ID is 12345678.
Grant grant1 = new Grant();
grant1.setGrantee(new GranteeId("12345678"));
grant1.setPermission(Permission.Read);
acl.addGrant(grant1);
// Grant the full permissions on the bucket to the user whose ID is 123456789.
Grant grant2 = new Grant();
grant2.setGrantee(new GranteeId("123456789"));
grant2.setPermission(Permission.FullControl);
acl.addGrant(grant2);
// Grant the write permission on the bucket to the user whose ID is 12345678910.
Grant grant3 = new Grant();
grant3.setGrantee(new GranteeId("12345678910"));
grant3.setPermission(Permission.Write);
acl.addGrant(grant3);
request.setAccessControlList(acl);
client.putBucketACL(request);
}
public AccessControlPolicy getBucketAcl(){
AccessControlPolicy acl = null;
// You need to specify only the bucket name.
acl = client.getBucketACL("<Your bucket name>");
return acl;
}Note:
- The access control policy contains the public preset ACL. You can call the getCannedAccessControlList() method to obtain the preset ACL.
After you enable logging for the bucket, KS3 automatically stores the operations logs of the bucket and its objects to a specified bucket.
/**
* Store the operations logs of the bucket to <Name of the bucket for storing logs>. The log object name starts with logging-.
*/
public void putBucketLogging(){
PutBucketLoggingRequest request = new PutBucketLoggingRequest("<Your bucket name>");
// Enable logging.
request.setEnable(true);
request.setTargetBucket("<Name of the bucket for storing logs>");
// Set the prefix of the log object name to logging-.
request.setTargetPrefix("logging-");
client.putBucketLogging(request);
}Note:
- The bucket for storing logs must be in the same region as your bucket.
- The bucket for which logging is configured and the bucket for storing logs can be the same bucket. However, for ease of management, we recommend that you use two different buckets.
5.6 Query the logging setting of a bucket
public BucketLoggingStatus getBucketLogging(){
// You need to specify only the bucket name. KS3 does not support setting logging permissions. Therefore, the targetGrants parameter in the returned BucketLoggingStatus field is always empty.
BucketLoggingStatus logging = client.getBucketLogging("<Your bucket name>");
return logging;
}
/**
* Send a HEAD request for a bucket.
*/
public HeadBucketResult headBucket() {
HeadBucketResult result = client.headBucket("<Your bucket name>");
/**
* If the result.getStatusCode() method returns the status code 404, the bucket does not exist. If the result.getStatusCode() method returns the status code 403, you do not have the read permission on the bucket.
*/
return result;
}
public REGION getBucketLocation(){
// You need to specify only the bucket name.
REGION region = client.getBucketLocation("<Your bucket name>");
return region;
}
public void putBucketCors(){
BucketCorsConfiguration config = new BucketCorsConfiguration();
// Specify the CORS rule.
CorsRule rule1 = new CorsRule();
// Specify the allowed CORS request methods. Valid values: GET, PUT, DELETE, POST, and HEAD.
List<AllowedMethods> allowedMethods = new ArrayList<AllowedMethods>();
allowedMethods.add(AllowedMethods.GET);
// Specify the allowed source of CORS requests.
List<String> allowedOrigins = new ArrayList<String>();
allowedOrigins.add("http://example.com");
// Specify the response headers that you are allowed to access from applications.
List<String> exposedHeaders = new ArrayList<String>();
exposedHeaders.add("x-kss-test1");
// Control whether the headers specified by Access-Control-Request-Headers in an OPTIONS preflight request are allowed.
List<String> allowedHeaders = new ArrayList<String>();
allowedHeaders.add("x-kss-test");
rule1.setAllowedHeaders(allowedHeaders);
rule1.setAllowedMethods(allowedMethods);
rule1.setAllowedOrigins(allowedOrigins);
rule1.setExposedHeaders(exposedHeaders);
// Specify the period of time within which the browser caches the response for an OPTIONS preflight request for specific resources, in seconds.
rule1.setMaxAgeSeconds(200);
config.addRule(rule1);
// You can configure up to 10 rules for a bucket.
CorsRule rule2 = new CorsRule();
List<AllowedMethods> allowedMethods2 = new ArrayList<AllowedMethods>();
allowedMethods2.add(AllowedMethods.GET);
allowedMethods2.add(AllowedMethods.POST);
List<String> allowedOrigins2 = new ArrayList<String>();
allowedOrigins2.add("http://example.com");
allowedOrigins2.add("http://*.example.com");
List<String> exposedHeaders2 = new ArrayList<String>();
exposedHeaders2.add("x-kss-test1");
exposedHeaders2.add("x-kss-test2");
List<String> allowedHeaders2 = new ArrayList<String>();
allowedHeaders2.add("x-kss-test");
allowedHeaders2.add("x-kss-test2");
rule2.setAllowedHeaders(allowedHeaders2);
rule2.setAllowedMethods(allowedMethods2);
rule2.setAllowedOrigins(allowedOrigins2);
rule2.setExposedHeaders(exposedHeaders2);
rule2.setMaxAgeSeconds(500);
config.addRule(rule2);
PutBucketCorsRequest request = new PutBucketCorsRequest("<Your bucket name>",config);
client.putBucketCors(request);
}Note:
- The CORS rule is required when you use JavaScript for cross-origin resource sharing. For more information, see Fetch Living Standard.
- The AllowedMethods, AllowedHeaders, and AllowedOrigins parameters are required. The AllowedHeaders and AllowedOrigins parameters can contain at most one asterisk (*) as a wildcard.
5.10 Query the CORS rule of a bucket
public BucketCorsConfiguration getBucketCors(){
BucketCorsConfiguration config = client.getBucketCors("test.bucket");
List<CorsRule> rules = config.getRules();
for(CorsRule rule : rules){
// Control whether the headers specified by Access-Control-Request-Headers in an OPTIONS preflight request are allowed.
rule.getAllowedHeaders();
// Specify the allowed CORS request methods. Valid values: GET, PUT, DELETE, POST, and HEAD.
rule.getAllowedMethods();
// Specify the allowed source of CORS requests.
rule.getAllowedOrigins();
// Specify the response headers that you are allowed to access from applications.
rule.getExposedHeaders();
// Specify the period of time within which the browser caches the response for an OPTIONS preflight request for specific resources, in seconds.
rule.getMaxAgeSeconds();
}
return config;
}
public void deleteBucketCors(){
client.deleteBucketCors("test.bucket");
}
public void deleteBucket(){
client.deleteBucket("test-bucket");
}
You can enable a replication rule by using the following code:
/**
* Enable replication for the source bucket.
*/
// Initialize the replication rule.
ReplicationRule rule = new ReplicationRule();
// Set one or more prefixes. Only objects matching the prefixes are synchronized.
rule.getPrefixList().add(prefix1);// If no prefix is added, all objects are synchronized by default.
rule.getPrefixList().add(prefix2);// You can add up to 10 prefixes, and the prefixes must be unique.
// Specify whether to enable deletion replication. true: deletion replication is enabled. false or unspecified: deletion replication is disabled.
rule.setDeleteMarkerStatus(Enabled);
rule.setTargetBucket(bucketName);// Select a destination bucket.
// Create a replication rule for the source bucket.
client.putBucketReplicationConfiguration(bucketName,rule);
/**
* Query the replication rule of a bucket. If replication is not enabled for the bucket, an HTTP 404 error is returned.
*/
ReplicationRule replicationRule = client.getBucketReplicationConfiguration(bucketName);
/**
* Disable replication for a bucket. If replication is not enabled or has been disabled for the bucket, an HTTP 404 error is returned.
*/
ReplicationRule replicationRule = client.deleteBucketReplicationConfiguration(bucketName);
/*5.16.1 Create or update an asynchronous back-to-origin rule.*/
BucketMirror bucketMirror = new BucketMirror(BucketMirror.Version.V3, false, new BucketMirror.AsyncMirrorRule(
Arrays.asList("http://www.example1.com", "https://www.example2.com"), null), null);
client.putBucketMirror("bucketName", bucketMirror);/*5.16.2 Create or update a synchronous back-to-origin rule.*/
List<BucketMirror.SyncMirrorRule> syncMirrorRules = new ArrayList<>();
BucketMirror.SyncMirrorRule smr1 = new BucketMirror.SyncMirrorRule();
smr1.setMirrorUrl("https://www.example1.com");
syncMirrorRules.add(smr1);
BucketMirror.SyncMirrorRule smr2 = new BucketMirror.SyncMirrorRule();
smr2.setMirrorUrl("https://www.example1.com");
syncMirrorRules.add(smr2);
BucketMirror bucketMirror = new BucketMirror(BucketMirror.Version.V3, false, null, syncMirrorRules);
client.putBucketMirror("bucketName", bucketMirror);/*5.16.3 Create or update an asynchronous back-to-origin rule and set the ACL permission of the origin object to PublicRead.*/
BucketMirror bucketMirror = new BucketMirror(BucketMirror.Version.V3, false,
new BucketMirror.AsyncMirrorRule(Arrays.asList("http://www.example1.com", "https://www.example2.com"),
new BucketMirror.SavingSetting(CannedAccessControlList.PublicRead)),
null);
client.putBucketMirror("bucketName", bucketMirror);/*5.16.4 Create or update a synchronous back-to-origin rule and set the ACL permission of the origin object to PublicRead.*/
List<BucketMirror.SyncMirrorRule> syncMirrorRules = new ArrayList<>();
BucketMirror.SyncMirrorRule smr1 = new BucketMirror.SyncMirrorRule();
smr1.setMirrorUrl("https://www.example1.com");
smr1.setSavingSetting(new BucketMirror.SavingSetting(CannedAccessControlList.PublicRead));
syncMirrorRules.add(smr1);
BucketMirror bucketMirror = new BucketMirror(BucketMirror.Version.V3, false, null, syncMirrorRules);
client.putBucketMirror("bucketName", bucketMirror);/*5.16.5 Create or update a synchronous back-to-origin rule and set back-to-origin conditions.*/
List<BucketMirror.SyncMirrorRule> syncMirrorRules = new ArrayList<>();
BucketMirror.SyncMirrorRule smr1 = new BucketMirror.SyncMirrorRule();
smr1.setMirrorUrl("https://www.example1.com");
smr1.setMatchCondition(new BucketMirror.MatchCondition(Arrays.asList("404"), Arrays.asList("prefix_example")));
syncMirrorRules.add(smr1);
BucketMirror bucketMirror = new BucketMirror(BucketMirror.Version.V3, false, null, syncMirrorRules);
client.putBucketMirror("bucketName", bucketMirror);/*5.16.6 Create or update a synchronous back-to-origin rule and set back-to-origin actions.*/
List<BucketMirror.SyncMirrorRule> syncMirrorRules = new ArrayList<>();
BucketMirror.SyncMirrorRule smr1 = new BucketMirror.SyncMirrorRule();
smr1.setMirrorUrl("https://www.example1.com");
BucketMirror.MirrorRequestSetting setting = new BucketMirror.MirrorRequestSetting();
// Specify whether to respond to an HTTP 3xx response of the origin site. true: KS3 re-sends a request to the location specified in the HTTP 3xx response received from the origin site.
// false: KS3 returns an HTTP 424 response to the client when receiving an HTTP 3xx response from the origin site.
setting.setFollow3xx(true);
// Specify whether to pass through the query string that is received from the client to the origin site.
setting.setPassQueryString(true);
// Set transmission rules for back-to-origin headers.
BucketMirror.HeaderSetting hs = new BucketMirror.HeaderSetting();
// Pass through all request headers that are received from the client to the origin site.
hs.setPassAll(true);
// Pass through headers with specified keys to the origin site. Currently, the key values cannot be specified.
hs.setPassHeaders(Arrays.asList(new BucketMirror.Header("key1"), new BucketMirror.Header("key2")));
// Do not pass through headers with specified keys to the origin site. Currently, the key values cannot be specified.
// This parameter is used with the hs.setPassAll(true) or hs.setPassHeaders() parameter.
hs.setRemoveHeaders(Arrays.asList(new BucketMirror.Header("key1"), new BucketMirror.Header("key2")));
// Pass through headers with fixed values to the origin site. Regardless of the request headers received from the client, KS3 passes through headers with fixed values to the origin site.
hs.setSetHeaders(
Arrays.asList(new BucketMirror.Header("key1", "value1"), new BucketMirror.Header("key2", "value2")));
setting.setHeaderSetting(hs);
smr1.setMirrorRequestSetting(setting);
syncMirrorRules.add(smr1);
BucketMirror bucketMirror = new BucketMirror(BucketMirror.Version.V3, false, null, syncMirrorRules);
client.putBucketMirror("bucketName", bucketMirror);
BucketMirror bucketMirror = client.getBucketMirror("bucketName");
client.deleteBucketMirror("bucketName");
/**
* List objects in a bucket. Up to 1,000 objects can be returned for a request.
*/
public ObjectListing listObjectsSimple(){
ObjectListing list = client.listObjects("<Your bucket name>");
return list;
}
/**
* List objects matching the specified object key prefix in a bucket. Up to 1,000 objects can be returned for a request.
*/
public ObjectListing listObjectsWithPrefix(){
ObjectListing list = client.listObjects("<Your bucket name>","<Object key prefix>");
return list;
}
/**
* Modify the parameters of the listed objects.
*/
public ObjectListing listObjectsUseRequest(){
ObjectListing list = null;
// Create a request to list objects.
ListObjectsRequest request = new ListObjectsRequest("<Your bucket name>");
// Configure parameters.
request.setMaxKeys("<max keys>");// The maximum number of objects to be returned.
request.setPrefix("<Object key prefix>");// The key prefix of the objects to be returned.
request.setDelimiter("<delimiter>");// The object delimiter. KS3 organizes folders based on the delimiter. The default delimiter is a slash (/).
// Perform the operation.
list = client.listObjects(request);
return list;
}
/**
* List all objects through cycling.
*/
public void listAllObjects(){
ObjectListing list = null;
// Initialize a request.
ListObjectsRequest request = new ListObjectsRequest("<Your bucket name>");
do{
// If the value of isTruncated is true, more objects are to be listed, and this operation must be repeated.
if(list!=null&&list.isTruncated())
// The marker for the next request is returned in ObjectListing.
// If no delimiter is specified in a request, no marker is returned for the next request. In this case, the last key in the previous request is used as the marker for the next request.
request.setMarker(list.getObjectSummaries().get(list.getObjectSummaries().size() - 1).getKey());
list = client.listObjects(request);
}while(list.isTruncated());
}Note:
- Up to 1,000 objects can be returned for a request. To return more objects, configure the Marker parameter.
- For more information about the input and output parameters, see List Parts.
6.2 Upload an object
Upload an object by using a path.
/**
* Upload new File ("<filePath>") to the <Bucket name> bucket and rename it <Object key>.
*/
public void putObjectSimple(){
PutObjectRequest request = new PutObjectRequest("<Your bucket name>",
"<object key>", new File("<filePath>"));
// Upload a public object.
//request.setCannedAcl(CannedAccessControlList.PublicRead);
client.putObject(request);
}Upload an object by using a stream.
/**
* Upload the content of an input stream to the <Bucket name> bucket and rename it <Object key>.
*/
public void putObjectSimple(){
ObjectMetadata meta = new ObjectMetadata();
PutObjectRequest request = new PutObjectRequest("<Your bucket name>",
"<object key>", new ByteArrayInputStream("1234".getBytes()),
meta);
// You can specify the content length. Otherwise, the system caches the entire input stream, which may cause out of memory (OOM) on a Java virtual machine (JVM).
meta.setContentLength(4);
client.putObject(request);
}Set the storage class when you upload an object.
// Set the storage class to Standard.
request.setStorageClass(StorageClass.Standard);
// Set the storage class to Infrequent Access.
request.setStorageClass(StorageClass.StandardInfrequentAccess);
// Set the storage class to Archive.
request.setStorageClass(StorageClass.Archive);Set metadata when you upload an object.
ObjectMetadata meta = new ObjectMetadata();
// Set user metadata for the object to be uploaded. When you download the object, the response headers carry the user metadata.
//meta.setUserMeta("x-kss-meta-example", "example");
// Set metadata for the object to be uploaded.
// If the content type is specified, the response headers carry the specified content type when you download the object. Set the content type based on the object type so that the browser can recognize the metadata.
//meta.setContentType("text/html");
request.setObjectMeta(meta);Configure a callback.
// If a callback is configured, KS3 calls your callback URL by using the POST method before the object upload is completed. If the call succeeds and KS3 receives {"result":true}, the object is uploaded successfully. If the call fails or KS3 receives {"result":false}, the object fails to be uploaded. You can specify the callback body of KS3. KS3 organizes your parameters in a JSON string and returns the string to you. In the following example, the callback body of KS3 is as follows: {"bucket":<Bucket name>,"createTime":<Object creation time>,"etag":<Object Etag, which is the hexadecimal representation of the MD5 value>,"key":<Object key>,"mimType":<Object content type>,"objectSize":<Object size, in bytes>,"time":"20150222"}
CallBackConfiguration config = new CallBackConfiguration();
config.setCallBackUrl("http://10.4.2.38:19090/");// The callback URL for the KS3 server.
// The parameters in the body when the KS3 server accesses http://10.4.2.38:19090/.
Map<String,MagicVariables> magicVariables = new HashMap<String,MagicVariables>();
magicVariables.put("bucket", MagicVariables.bucket);
magicVariables.put("createTime", MagicVariables.createTime);
magicVariables.put("etag", MagicVariables.etag);
magicVariables.put("key", MagicVariables.key);
magicVariables.put("mimeType", MagicVariables.mimeType);
magicVariables.put("objectSize", MagicVariables.objectSize);
config.setBodyMagicVariables(magicVariables);
// You can customize the response parameters.
Map<String,String> kssVariables = new HashMap<String,String>();
kssVariables.put("time", "20150222");
request.setCallBackConfiguration(config);Configure server-side encryption.
ObjectMetadata meta = new ObjectMetadata();
meta.setSseAlgorithm("AES256");
request.setObjectMeta(meta);Configure server-side encryption by using a user-defined key.
// Generate a key, which is required for both encryption and decryption. Keep it properly.
KeyGenerator symKeyGenerator = KeyGenerator.getInstance("AES");
symKeyGenerator.init(256);
SecretKey symKey = symKeyGenerator.generateKey();
SSECustomerKey ssec = new SSECustomerKey(symKey);
request.setSseCustomerKey(ssec);Note: - For more information about how to download an object, see sections 6.7 "Download an object" and 6.8 "Query the access URL of an object." - For more information about asynchronous data processing operations, see Upload CallBack Processing. - If server-side encryption by using a user-defined key is configured for an object, you must provide the key when downloading the object. Otherwise, the download fails.
6.3 Upload an object by using a form
Read the KS3 form upload protocol in POST Object and the signature authentication method of form upload in Post policy.
Log in to the KS3 console and click Bucket in the left navigation pane. On the Bucket list page, click the name of the target bucket. On the page that appears, click Bucket Settings. On the Bucket Settings tab, click CORS Configuration. On the CORS Configuration tab, configure a CORS rule. The CORS rule is required when you use JavaScript for cross-origin resource sharing. For more information, see Fetch Living Standard.
Upload an object by using the KS3 SDK for JavaScript. For more information, see KS3 SDK for Java.
Calculate the policy and signature in the SDK for JavaScript by using the following method:
/**
If you are not familiar with the KS3 protocol, we recommend that you use this method to obtain the latest signature each time you upload an object.
*/
public PostObjectFormFields postObjectSimple(){
/**
* Provide all form fields, except KSSAccessKeyId, Signature, file, and Policy, in postData and unknownValueField. Otherwise, an HTTP 403 error is returned when you upload an object by using the generated signature.</br>
* Put form fields with known values in postData and form fields with unknown values in unknownValueField. For example, some form fields with random values may be added to an upload control, and these form fields must be put in unknownValueField.</br>
*
*/
Map<String,String> postData = new HashMap<String,String>();
// If an ACL is configured when you upload an object by using the SDK for JavaScript, add the following code and ensure that the value is the same as that in the SDK. If no ACL is configured, delete the following code:
postData.put("acl","public-read");
// Provide the key in the SDK for JavaScript.
postData.put("key","20150115/Chinese/${filename}");
List<String> unknowValueField = new ArrayList<String>();
// The name field is automatically added when you upload an object by using the SDK for JavaScript. Therefore, the following code must be added:
unknowValueField.add("name");
// If the key provided for calculating the signature does not contain the ${filename} placeholder, the second parameter can be an empty string, because the key required for calculating the policy is obtained after ${filename} in the original key is replaced.
PostObjectFormFields fields = client.postObject("<Your bucket name>", "<Name of the object to be uploaded, without the path>", postData, unknownValueField);
fields.getKssAccessKeyId();
fields.getPolicy();
fields.getSignature();
return fields;
}Common errors:
- When you upload an object, the browser sends an OPTIONS request and a POST request. If an HTTP 403 error is returned for the OPTIONS request, verify the CORS configurations. If an HTTP 403 error is returned for the POST request, check whether the policy is exactly generated based on the form fields, or Base64-encode the policy and compare it with the form fields. For more information about the comparison rules, see Post policy.
6.4 Replicate an object
public void copyObject(){
/** Replicate the sourceKey object from sourceBucket to destinationBucket and name the object in destinationBucket as destinationObject.
*/
CopyObjectRequest request = new CopyObjectRequest("destinationBucket","destinationObject","sourceBucket","sourceKey");
client.copyObject(request);
}Store the destination object by using server-side encryption.
ObjectMetadata meta = new ObjectMetadata();
meta.setSseAlgorithm("AES256");
request.setNewObjectMetadata(meta);Configure server-side encryption for the source object by using a user-defined key, and configure server-side encryption for the destination object.
// Generate a key, which is required for both encryption and decryption. Keep it properly.
KeyGenerator symKeyGenerator = KeyGenerator.getInstance("AES");
symKeyGenerator.init(256);
SecretKey destKey= symKeyGenerator.generateKey();
SecretKey sourceKey= ??// The key used for encryption.
// Specify the source key for encrypting the source data. The data will be decrypted by using the key before it is copied.
request.setSourceSSECustomerKey(new SSECustomerKey(sourceKey));
// Specify an encryption method for the destination data.
request.setDestinationSSECustomerKey(new SSECustomerKey(destKey));Note:
- You must have the read permission on the source object and the write permission on the destination bucket before you can replicate the object.
- If the destination object already exists, KS3 returns an HTTP 400 error: invalidKey.
6.5 Query the metadata of an object
public HeadObjectResult headObject() {
HeadObjectRequest request = new HeadObjectRequest("<Bucket name>",
"<Object name>");
/**
* <p>
* If the {@link NotFoundException} error is returned, the object does not exist.
* </p>
* <p>
* If the {@link AccessDeniedException} error is returned, you do not have the access permission on the object.
* </p>
*/
HeadObjectResult result = client.headObject(request);
// You can send a HEAD request to query the metadata of an object.
result.getObjectMetadata();
return result;
}
/**
* Check whether an object exists.
*/
public boolean objectExists() {
try {
HeadObjectRequest request = new HeadObjectRequest("<Bucket name>",
"<Object name>");
client.headObject(request);
return true;
} catch (NotFoundException e) {
return false;
}
}
public GetObjectResult getObject(){
GetObjectRequest request = new GetObjectRequest("<Bucket name>","<Object key>");
// Override the response headers.
ResponseHeaderOverrides overrides = new ResponseHeaderOverrides();
overrides.setContentType("text/html");
//.......
request.setOverrides(overrides);
// Receive only 0 to 10 bytes of the data. You can set this parameter to enable multipart download.
//request.setRange(0,10);
GetObjectResult result = client.getObject(request);
Ks3Object object = result.getObject();
// Query the metadata of the object.
ObjectMetadata meta = object.getObjectMetadata();
// Query the object size instead of the part size during multipart download.
Long length = meta.getInstanceLength();
// Query the input stream of the object.
object.getObjectContent();
return result;
}If server-side encryption by using a user-defined key is configured for the object:
SecretKey sourceKey= ??// The key used for encryption.
request.setSseCustomerKey(new SSECustomerKey(sourceKey));
Create a URL in the format of http://{bucket}.{endpoint}/{key}, for example, http://test-bucket.ks3-cn-beijing.ksyuncs.com/2015/10/19/image.jpg. In the URL example, {bucket} is set to test-bucket, {endpoint} is set to ks3-cn-beijing.ksyun.com, and {key} is set to 2015/10/19/image.jpg.
Generate an access URL by using the following code:
// Generate an external link that is to expire 1,000s later.
client.generatePresignedUrl(<bucket>,<key>,1000);
// Generate an external link that is to expire 1,000s later with response headers overridden.
ResponseHeaderOverrides overrides = new ResponseHeaderOverrides();
//overrides.setContentType("text/html");
//.......
client.generatePresignedUrl(<bucket>,<key>,1000,overrides);Common errors:
- If the object does not exist, a NoSuchKey error is returned.
- If you attempt to access a private object by using a public URL, an AccessDenied error is returned.
- If the access URL of a private object expires, an URLExpired error is returned.
- The validity period of 1,000s is counted based on the local time of the client.
6.9 Configure the access permission of an object
For more information about the access permissions of objects, see ACL.
/**
* Configure a preset ACL to manage the access permission of an object. The permission can be set to private or public-read. The private permission indicates that only the object owner can access the object. The public-read permission indicates that all users have the read permission on the object.
*/
public void putObjectAclWithCannedAcl(){
PutObjectACLRequest request = new PutObjectACLRequest("<Bucket name>","<Object name>");
// Set the ACL permission to private.
request.setCannedAcl(CannedAccessControlList.Private);
// Set the ACL permission to public-read.
//request.setCannedAcl(CannedAccessControlList.PublicRead);
client.putObjectACL(request);
}
/**
// Customize the access permission of an object.
*/
public void putObjectAclWithAcl(){
PutObjectACLRequest request = new PutObjectACLRequest("<Bucket name>","<Object key>");
AccessControlList acl = new AccessControlList();
// Grant the read permission on the object to the user whose ID is 12345678.
Grant grant1 = new Grant();
grant1.setGrantee(new GranteeId("12345678"));
grant1.setPermission(Permission.Read);
acl.addGrant(grant1);
// Grant the full permissions on the object to the user whose ID is 123456789.
Grant grant2 = new Grant();
grant2.setGrantee(new GranteeId("123456789"));
grant2.setPermission(Permission.FullControl);
acl.addGrant(grant2);
// Grant the write permission on the object to the user whose ID is 12345678910.
Grant grant3 = new Grant();
grant3.setGrantee(new GranteeId("12345678910"));
grant3.setPermission(Permission.Write);
acl.addGrant(grant3);
request.setAccessControlList(acl);
client.putObjectACL(request);
}
public AccessControlPolicy getObjectAcl(){
/**
* Query the access permission of the <Object key> object in the <Bucket name> bucket.
*/
AccessControlPolicy policy = client.getObjectACL("<Bucket name>","<Object key>");
return policy;
}Note:
- The access control policy contains the public preset ACL. You can call the getCannedAccessControlList() method to obtain the preset ACL.
6.11 Delete an object
/**
* Delete the <Object key> object from the <Bucket name> bucket.
*/
public void deleteObject(){
client.deleteObject("<Bucket name>","<Object key>");
}
/**
* Restore the <Object key> object in the <Bucket name> bucket.
*/
public RestoreObjectResult restoreObject(){
RestoreObjectResult result = client.restoreObject("<Bucket name>","<Object key>");
// Query the storage class of the <Object key> object. Valid values:
// StorageClass.Standard, StorageClass.StandardInfrequentAccess, and StorageClass.Archive
result.getCls();
// Query the restoration status of the <Object key> object. Valid values:
// RestoreCycle.RESTORE, RestoreCycle.RESTORING, and RestoreCycle.RESTORED
result.getType();
return result;
}
// Modify the metadata of an object.
public void modifyObjMetadata(){
/**Modify the metadata of the yourkey object in yourbucket.*/
CopyObjectRequest request = new CopyObjectRequest("yourbucket", "yourkey", "yourbucket", "yourkey");
ObjectMetadata meta = new ObjectMetadata();
meta.setHeader("x-kss-metadata-directive", "REPLACE");
meta.setContentType("application/json");
meta.setUserMeta("x-kss-meta-zlymeta123", "mymeta123");
request.setNewObjectMetadata(meta);
CopyResult result = client.copyObject(request);
}
// Change the storage class of an object.
public void modifyObjStorageClass(){
/**Change the storage class of the yourkey object in yourbucket to Infrequent Access.*/
CopyObjectRequest request = new CopyObjectRequest("yourbucket", "yourkey", "yourbucket", "yourkey");
request.setStorageClass(StorageClass.StandardInfrequentAccess);
CopyResult result = client.copyObject(request);
}
public Ks3Result putObjectFetch(){
// Set the bucket name, object name, and third-party URL in an object fetch request.
PutObjectFetchRequest request = new PutObjectFetchRequest(bucketName,key,sourceUrl);
// Set the ACL permission of the object to public-read.
request.setCannedAcl(CannedAccessControlList.PublicRead);
// The value that is generated by using the MD5 algorithm for encryption and encoded in Base64. The MD5 value is a 128-bit number that can be used to verify the integrity of the object.
request.setMd5Base64(md5);
ObjectMetadata metadata = new ObjectMetadata();
// Set the metadata for the object
request.setObjectMeta(metadata);
// Configure a callback. This operation is performed asynchronously. Therefore, you cannot learn of the result immediately. You must specify a callback URL for the upload success or failure.
request.setCallbackUrl(callbackurl);
Ks3Result result = client.putObjectFetch(request);
return result;
}
public void putObjectTagging() {
ObjectTag tag = new ObjectTag("aaa", "111");
ObjectTag tag1 = new ObjectTag("as_=/+-:23", null);
ObjectTag tag2 = new ObjectTag("tt", "aa");
ObjectTag tag3 = new ObjectTag("asdfasdf", "aa");
List<ObjectTag> list = new ArrayList<>();
list.add(tag);
//list.add(tag1);
//list.add(tag2);
//list.add(tag3);
ObjectTagging tagging = new ObjectTagging(list);
PutObjectTaggingRequest request = new PutObjectTaggingRequest(bucket, "txt0", tagging);
client.putObjectTagging(request);
}
public void putObjectWithTagging() {
PutObjectRequest request = new PutObjectRequest(bucket, "tag", new File("D:\\upload-test\\1.txt"));
ObjectMetadata metadata = new ObjectMetadata();
metadata.setHeader("x-kss-tagging", "aaa=111");
request.setObjectMeta(metadata);
PutObjectResult result = client.putObject(request);
System.out.println(result.getOriginResponse());
}
public void getObjectTagging() {
GetObjectTaggingRequest request = new GetObjectTaggingRequest(bucket, "txt0");
ObjectTagging tagging = client.getObjectTagging(request);
System.out.println(JSON.toJSONString(tagging));
}
public void deleteObjectTagging() {
DeleteObjectTaggingRequest request = new DeleteObjectTaggingRequest(bucket, "objecttagnew");
client.deleteObjectTagging(request);
}
Multipart upload processes:
Initiate multipart upload. For more information, see section 7.2 "Initiate multipart upload."
Upload parts. For more information, see section 7.3 "Upload parts."
Complete multipart upload. For more information, see section 7.5 "Complete multipart upload." If the operation fails, you can abort multipart upload. For more information, see section 7.6 "Abort multipart upload."
public ListMultipartUploadsResult listMultipartUploads() {
ListMultipartUploadsRequest request = new ListMultipartUploadsRequest("test.bucket");
/**
* The value of keyMarker is empty, and the value of uploadIdMarker is specified.
* The result is meaningless.
*
* The values of keyMarker and uploadIdMarker are specified.
* List the multipart uploads whose object key is the value of keyMarker and whose upload IDs are after the value of uploadIdMarker in alphabetical order.
*
* The value of keyMarker is specified, and the value of uploadIdMarker is empty.
* List the multipart uploads whose object keys are after the value of keyMarker in alphabetical order.
*/
request.setKeyMarker("keyMarker");
request.setUploadIdMarker("uploadIdMarker");
/**
* Interpretations of the prefix and delimiter
*
* The commonPrefix field is determined by the prefix and delimiter. It contains the string
from the prefix to the first delimiter (including the delimiter) in an object key starting with the prefix.
* For example, the multipart uploads for the following objects are ongoing:
*
* aaaa/bbb/ddd.txt
* aaaa/ccc/eee.txt
* ssss/eee/fff.txt
*
* Assume that the prefix is empty, and the delimiter is a slash (/).
* The commonPrefix field contains aaaa/ and ssss/, and no upload is returned.
*
* Assume that the prefix is aaaa/, and the delimiter is a slash (/).
* The commonPrefix field contains aaaa/bbb/ and aaaa/ccc/, and no upload is returned.
*
* Assume that the prefix is ssss/, and the delimiter is a slash (/).
* The commonPrefix field contains ssss/eee/, and no upload is returned.
*
* Assume that both the prefix and the delimiter are empty.
* The commonPrefix field is empty, and the multipart uploads for the aaaa/bbb/ddd.txt, aaaa/ccc/eee.txt, and ssss/eee/fff.txt objects are returned.
*
* Assume that the prefix is aaaa/, and the delimiter is empty.
* The commonPrefix field is empty, and the multipart uploads for the aaaa/bbb/ddd.txt and aaaa/ccc/eee.txt objects are returned.
*
* Assume that the prefix is ssss/, and the delimiter is empty.
* The commonPrefix field is empty, and the multipart upload for the ssss/eee/fff.txt object is returned.
* </p>
*/
request.setDelimiter("/");
request.setPrefix("prefix");
request.setMaxUploads(100);// Indicates that up to 100 multipart uploads can be returned for a request. Default value: 1,000. Valid values: 1 to 1,000.
ListMultipartUploadsResult result = client
.listMultipartUploads(request);
return result;}
Note:
- Up to 1,000 multipart uploads can be returned for a request.
- keyMarker, uploadIdMarker, delimiter, prefix, and maxUploads are optional parameters. You can specify the parameters as required.
- If the value of result.isTruncated() is true, the values returned by the result.getNextKeyMarker() and result.getNextUploadIdMarker methods serve as the values of the keyMarker and uploadIdMarker parameters for listing subsequent multipart uploads.
7.2 Initiate multipart upload
public InitiateMultipartUploadResult initMultipartUpload(){
InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(
bucket, key);
// Set the ACL permission to public-read.
request.setCannedAcl(CannedAccessControlList.PublicRead);
InitiateMultipartUploadResult result = client
.initiateMultipartUpload(request);
result.getUploadId();// Query the initial upload ID, which is required in subsequent operations.
return result;
}Set metadata in this operation.
ObjectMetadata meta = new ObjectMetadata();
// Set user metadata for the object to be uploaded. When you download the object, the response headers carry the user metadata.
//meta.setUserMeta("x-kss-meta-example", "example");
// Set metadata for the object to be uploaded.
// If the content type is specified, the response headers carry the specified content type when you download the object. Set the content type based on the object type so that the browser can recognize the metadata.
//meta.setContentType("text/html");
request.setObjectMeta(meta);Configure server-side encryption.
ObjectMetadata meta = new ObjectMetadata();
meta.setSseAlgorithm("AES256");
request.setObjectMeta(meta);Configure server-side encryption by using a user-defined key.
// Generate a key, which is required for both encryption and decryption. Keep it properly.
KeyGenerator symKeyGenerator = KeyGenerator.getInstance("AES");
symKeyGenerator.init(256);
SecretKey symKey = symKeyGenerator.generateKey();
SSECustomerKey ssec = new SSECustomerKey(symKey);
request.setSseCustomerKey(ssec);Note:
- You can call the result.getUploadId() method to obtain the current upload ID, which is required in subsequent operations.
- For more information about how to download an object, see sections 6.7 "Download an object" and 6.8 "Query the access URL of an object."
- If server-side encryption by using a user-defined key is configured for an object, you must provide the key when downloading the object. Otherwise, the download fails.
7.3 Upload parts
Specify the object name, part size, and offset for multipart upload.
public PartETag uploadPart(){
UploadPartRequest request = new UploadPartRequest(
{bucket}, {key} ,{uploadId},
{partNumber},{Name of the object to be uploaded}, {partSize},{offset});
// You can specify the MD5 value of the content so that the system performs MD5 verification on the server. Otherwise, the system performs MD5 verification only on the client.
//request.setContentMd5("yE0ZQBpRgPrLSDEe6fGAvg==");
PartETag result = client.uploadPart(request);
return result;
}Specify the input stream and part size for multipart upload.
public PartETag uploadPart(){
InputStream content = new ByteArrayInputStream("content".getBytes());
UploadPartRequest request = new UploadPartRequest(
{bucket}, {key} ,{uploadId},
{partNumber},content,{partSize});
// You can specify the MD5 value of the content so that the system performs MD5 verification on the server. Otherwise, the system performs MD5 verification only on the client.
//request.setContentMD5("yE0ZQBpRgPrLSDEe6fGAvg==");
PartETag result = client.uploadPart(request);
return result;
}Parameters:
- bucket: the bucket name that is specified when you initiate multipart upload.
- key: the key that is specified when you initiate multipart upload.
- uploadId: the upload ID that is obtained when you initiate multipart upload.
- partNumber: the serial number of the part that is being uploaded. If you specify multiple serial numbers, the serial numbers must be consecutive integers.
- partSize: the number of bytes of the part that is being uploaded.
Configure server-side encryption by using a user-defined key.
// If server-side encryption by using a user-defined key is configured when you initiate multipart upload, the same encryption information must be specified in this operation.
SecretKey symKey = ??// The key that is specified when you initiate multipart upload.
SSECustomerKey ssec = new SSECustomerKey(symKey);
request.setSseCustomerKey(ssec);Note:
- The serial numbers that are specified in the partNumber parameter must be consecutive integers.
- The part serial number and Etag that are returned for each multipart upload must be stored locally, which are required when you complete multipart upload.
7.4 List uploaded parts
List the uploaded parts corresponding to an upload ID.
public ListPartsResult listParts(){
ListPartsRequest request = new ListPartsRequest({bucket}, {key} ,{uploadId});
ListPartsResult tags = client.listParts(request);
return tags;
}Parameters:
- bucket: the bucket name that is specified when you initiate multipart upload.
- key: the key that is specified when you initiate multipart upload.
- uploadId: the upload ID that is obtained when you initiate multipart upload.
Note:
- This operation returns up to 1,000 records for a request. If more than 1,000 records are available, you can call the request.setPartNumberMarker() method to set the start and end positions of part serial numbers.
7.5 Complete multipart upload
You can perform this operation to complete a multipart upload and assemble all parts into an object.
public void completaMultipartUpload(){
List<PartETag> parts = new ArrayList<PartETag>();
// Add all parts that are returned when you upload parts.
//parts.add(...)
CompleteMultipartUploadRequest request = new CompleteMultipartUploadRequest({bucket},{key},{uploadId}, parts);
client.completeMultipartUpload(request);
}Configure a callback.
// If a callback is configured, KS3 calls your callback URL by using the POST method before the object upload is completed. If the call succeeds and KS3 receives {"result":true}, the object is uploaded successfully. If the call fails or KS3 receives {"result":false}, the object fails to be uploaded. You can specify the callback body of KS3. KS3 organizes your parameters in a JSON string and returns the string to you. In the following example, the callback body of KS3 is as follows: {"bucket":<Bucket name>,"createTime":<Object creation time>,"etag":<Object Etag, which is the hexadecimal representation of the MD5 value>,"key":<Object key>,"mimType":<Object content type>,"objectSize":<Object size, in bytes>,"time":"20150222"}.
CallBackConfiguration config = new CallBackConfiguration();
config.setCallBackUrl("http://10.4.2.38:19090/");// The callback URL for the KS3 server.
// The parameters in the body when the KS3 server accesses http://10.4.2.38:19090/.
Map<String,MagicVariables> magicVariables = new HashMap<String,MagicVariables>();
magicVariables.put("bucket", MagicVariables.bucket);
magicVariables.put("createTime", MagicVariables.createTime);
magicVariables.put("etag", MagicVariables.etag);
magicVariables.put("key", MagicVariables.key);
magicVariables.put("mimeType", MagicVariables.mimeType);
magicVariables.put("objectSize", MagicVariables.objectSize);
config.setBodyMagicVariables(magicVariables);
// You can customize the response parameters.
Map<String,String> kssVariables = new HashMap<String,String>();
kssVariables.put("time", "20150222");
request.setCallBackConfiguration(config);
You can perform this operation to abort a multipart upload as required.
public void abortMultipartUpload(){
client.abortMultipartUpload(bucketname, objectkey, uploadId);
}
Add image processing parameters to the original key. For more information, see Image Processing API and Picture Style Management. For example, the assembled key is in the format of {Original key}@base@tag=imageInfo or {Original key}@style@{Style name}.
Obtain the access URL of an object by using the assembled key. For more information, see section 6.8 "Query the access URL of an object."
GeneratePresignedUrlRequest req = new GeneratePresignedUrlRequest();
req.setMethod(HttpMethod.GET);
req.setBucket("
req.setKey("key");
req.setExpiration(Expiration time of the external link>);// The expiration time is a UNIX timestamp. If this parameter is not set, the external link expires after 15 minutes by default.
ResponseHeaderOverrides overrides = new ResponseHeaderOverrides();
//overrides.setContentType("application/xml");// Set Content-Type for the response.
req.setResponseHeaders(overrides);
String url = client.generatePresignedUrl(req);
For more information about the KS3 upload protocol, see PUT Object.
GeneratePresignedUrlRequest req = new GeneratePresignedUrlRequest();
req.setMethod(HttpMethod.PUT);
req.setBucket(bucket);// The bucket to which the object is uploaded.
req.setKey(key);// The object name.
req.setExpiration(Expiration time of the external link>);// If this parameter is not set, the external link expires after 15 minutes by default.
req.getRequestConfig().getExtendHeaders().put("x-kss-acl", "public-read");// Set the ACL permission to public-read. If this header is not added, the permission is set to private by default. If this header is specified when you generate an external link, it must also be added when you use the external link.
req.setContentType("application/ocet-stream");// Set Content-Type for the object. The value depends on the time. When you use an external link, set Content-Type to a specified value.
//req.setSseAlgorithm("AES256");// Configure server-side encryption.
String url = client.generatePresignedUrl(req);After obtaining the external link, you can identify the URI and host and initiate the following request to upload an object:
PUT /{uri} HTTP/1.1
Host: {host}
Date: {Current time}
x-kss-acl:public-read // This header is required because it is specified when the external link is generated.
Content-Type:application/ocet-stream // See the preceding descriptions.
Content-Length:100
[100 bytes of data]
You can encrypt objects on your client by using the SDK before uploading them to KS3.
Download UnlimitedJCEPolicyJDK7. Store the local_policy.jar and US_export_policy.jar files in the {JAVA_HOME}\jre\lib\security directory to overwrite the original files.
Download the bcprov-jdk15on-152.jar file and store it in the {JAVA_HOME}\jre\lib\ext directory.
Use the symmetric master keys.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.X509EncodedKeySpec;
import java.util.Arrays;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;
import org.junit.Assert;
public class GenerateSymmetricMasterKey {
private static final String keyDir = System.getProperty("java.io.tmpdir");
private static final String keyName = "secret.key";
public static void main(String[] args) throws Exception {
//Generate symmetric 256 bit AES key.
KeyGenerator symKeyGenerator = KeyGenerator.getInstance("AES");
symKeyGenerator.init(256);
SecretKey symKey = symKeyGenerator.generateKey();
//Save key.
saveSymmetricKey(keyDir, symKey);
//Load key.
SecretKey symKeyLoaded = loadSymmetricAESKey(keyDir, "AES");
Assert.assertTrue(Arrays.equals(symKey.getEncoded(), symKeyLoaded.getEncoded()));
}
public static void saveSymmetricKey(String path, SecretKey secretKey)
throws IOException {
X509EncodedKeySpec x509EncodedKeySpec = new X509EncodedKeySpec(
secretKey.getEncoded());
FileOutputStream keyfos = new FileOutputStream(path + "/" + keyName);
keyfos.write(x509EncodedKeySpec.getEncoded());
keyfos.close();
}
public static SecretKey loadSymmetricAESKey(String path, String algorithm)
throws IOException, NoSuchAlgorithmException, InvalidKeySpecException, InvalidKeyException{
//Read private key from file.
File keyFile = new File(path + "/" + keyName);
FileInputStream keyfis = new FileInputStream(keyFile);
byte[] encodedPrivateKey = new byte[(int)keyFile.length()];
keyfis.read(encodedPrivateKey);
keyfis.close();
//Generate secret key.
return new SecretKeySpec(encodedPrivateKey, "AES");
}
}Use the asymmetric master key.
import static org.junit.Assert.assertTrue;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.security.KeyFactory;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.NoSuchAlgorithmException;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.SecureRandom;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.PKCS8EncodedKeySpec;
import java.security.spec.X509EncodedKeySpec;
import java.util.Arrays;
public class GenerateAsymmetricMasterKey {
private static final String keyDir = System.getProperty("java.io.tmpdir");
private static final SecureRandom srand = new SecureRandom();
public static void main(String[] args) throws Exception {
// Generate RSA key pair of 1024 bits
KeyPair keypair = genKeyPair("RSA", 1024);
// Save to file system
saveKeyPair(keyDir, keypair);
// Loads from file system
KeyPair loaded = loadKeyPair(keyDir, "RSA");
// Sanity check
assertTrue(Arrays.equals(keypair.getPublic().getEncoded(), loaded
.getPublic().getEncoded()));
assertTrue(Arrays.equals(keypair.getPrivate().getEncoded(), loaded
.getPrivate().getEncoded()));
}
public static KeyPair genKeyPair(String algorithm, int bitLength)
throws NoSuchAlgorithmException {
KeyPairGenerator keyGenerator = KeyPairGenerator.getInstance(algorithm);
keyGenerator.initialize(1024, srand);
return keyGenerator.generateKeyPair();
}
public static void saveKeyPair(String dir, KeyPair keyPair)
throws IOException {
PrivateKey privateKey = keyPair.getPrivate();
PublicKey publicKey = keyPair.getPublic();
X509EncodedKeySpec x509EncodedKeySpec = new X509EncodedKeySpec(
publicKey.getEncoded());
FileOutputStream fos = new FileOutputStream(dir + "/public.key");
fos.write(x509EncodedKeySpec.getEncoded());
fos.close();
PKCS8EncodedKeySpec pkcs8EncodedKeySpec = new PKCS8EncodedKeySpec(
privateKey.getEncoded());
fos = new FileOutputStream(dir + "/private.key");
fos.write(pkcs8EncodedKeySpec.getEncoded());
fos.close();
}
public static KeyPair loadKeyPair(String path, String algorithm)
throws IOException, NoSuchAlgorithmException,
InvalidKeySpecException {
// read public key from file
File filePublicKey = new File(path + "/public.key");
FileInputStream fis = new FileInputStream(filePublicKey);
byte[] encodedPublicKey = new byte[(int) filePublicKey.length()];
fis.read(encodedPublicKey);
fis.close();
// read private key from file
File filePrivateKey = new File(path + "/private.key");
fis = new FileInputStream(filePrivateKey);
byte[] encodedPrivateKey = new byte[(int) filePrivateKey.length()];
fis.read(encodedPrivateKey);
fis.close();
// Convert them into KeyPair
KeyFactory keyFactory = KeyFactory.getInstance(algorithm);
X509EncodedKeySpec publicKeySpec = new X509EncodedKeySpec(
encodedPublicKey);
PublicKey publicKey = keyFactory.generatePublic(publicKeySpec);
PKCS8EncodedKeySpec privateKeySpec = new PKCS8EncodedKeySpec(
encodedPrivateKey);
PrivateKey privateKey = keyFactory.generatePrivate(privateKeySpec);
return new KeyPair(publicKey, privateKey);
}
}
SecretKey symKey = ??// The master key.
EncryptionMaterials keyMaterials = new EncryptionMaterials(symKey);
Ks3 client = new Ks3EncryptionClient("<accesskeyid>","<accesskeysecret>",keyMaterials);
Uploaded objects are encrypted.
You can call the GET Object operation on your client to download objects. Objects that are downloaded by using other methods are encrypted.
You can upload only one part in a multipart upload request. When uploading the last part, you must mark it by calling the request.setLastPart method. The parts must be uploaded in sequence. Multi-threaded multipart upload is not supported.
Keep your master key properly. It is required for decrypting data.
Pure Mode