S3 Buckets - how to use

We have taken a general strategy at using S3 Bucket as a file storage in our solutions. The reasons are:

Reliability
Security
Accessibility
Speed
Easy of use
Meta-data (tags)

In order to use S3 for your solution you'd need these:

A bucket created and tuned for security (IAM) by Chemwatch IT.
AWS SDK (https://aws.amazon.com/tools/) depending on which laguage you use.

This document contains examples in JavaScript and C# as the most commonly used languages here at Chemwatch.

To ffectively use S3 in MS Dev Studio, you'd need to install AWS-SDK from Nuget library.

Creating an object in Bucket

const AWS = require('aws-sdk');

AWS.config.update({region: 'us-east-1'});
var s3 = new AWS.S3({httpOptions: { timeout: 2000 }});

/* The following example retrieves an object for an S3 bucket. */

var params = {
	Bucket: "mybucket", 
	Key: "myfile.jpg"
};

s3bucket.putObject(params, function(err, data) {
        if (err) {
            console.log("Error uploading data: ", err);
        } else {
            res.writeHead(200, {'Content-Type':'text/plain'});
            res.write("Successfully uploaded data to bucket/sub-bucket/");
            res.end()
        }
    });

Retrieving an object from Bucket

const AWS = require('aws-sdk');

AWS.config.update({region: 'us-east-1'});
var s3 = new AWS.S3({httpOptions: { timeout: 2000 }});

/* The following example retrieves an object for an S3 bucket. */

var params = {
	Bucket: "mybucket", 
	Key: "myfile.jpg"
};

s3.getObject(params, function(err, data) {
	if (err) console.log(err, err.stack); // an error occurred
	else     console.log(data);           // successful response
	/*
	data = {
		AcceptRanges: "bytes", 
		ContentLength: 3191, 
		ContentType: "image/jpeg", 
		ETag: "\"6805f2cfc46c0f04559748bb039d69ae\"", 
		LastModified: <Date Representation>, 
		Metadata: {
		}, 
		TagCount: 2, 
		VersionId: "null"
	}
	*/
});

// Getting object as a stream

var s3 = new AWS.S3({apiVersion: '2006-03-01'});
var params = {Bucket: 'myBucket', Key: 'myImageFile.jpg'};
var file = require('fs').createWriteStream('/path/to/file.jpg');
s3.getObject(params).createReadStream().pipe(file);

Deleting an object from Bucket

Edit object Tags

List all objects in Bucket

const AWS = require('aws-sdk');
const listAllObjects = require('s3-list-all-objects');

AWS.config.update({region: 'us-east-1'});
var s3 = new AWS.S3({httpOptions: { timeout: 2000 }});

var files = [];

listAllObjects({ bucket: 'molecules2d', progress: function(err, data) {
	
	data.data.forEach((item) => {
		console.log(item.Key);
		files.push(item.Key);
	});
	
	if(data.isFinalBatch == true) {
		console.log(JSON.stringify(files));
	}
}});

CLI (Command Line)

Command line allows you to quiuckly upload and download big amounts of files to/from S3. Below are requirements and annotated examples.

Requirements: AWS CLI tools (https://aws.amazon.com/cli/).

CW-Public