Upload files directly to Amazon S3 from ASP.NET application - asp.net

My ASP.NET MVC application will take a lot of bandwidth and storage space. How can I setup an ASP.NET upload page so the file the user uploaded will go straight to Amazon S3 without using my web server's storage and bandwidth?

Update Feb 2016:
The AWS SDK can handle a lot more of this now. Check out how to build the form, and how to build the signature. That should prevent you from needing the bandwidth on your end, assuming you need to do no processing of the content yourself before sending it to S3.

If you need to upload large files and display a progress bar you should consider the flajaxian component.
It uses flash to upload files directly to amazon s3, saving your bandwidth.

The best and the easiest way to upload files to amazon S3 via asp.net . Have a look at following blog post by me . i think this one will help. Here i have explained from adding a S3 bucket to creating the API Key, Installing Amazon SDK and writing code to upload files. Following are are the sample code for uploading files to amazon S3 with asp.net C#.
using System
using System.Collections.Generic
using System.Linq
using System.Web
using Amazon
using Amazon.S3
using Amazon.S3.Transfer
///
/// Summary description for AmazonUploader
///
public class AmazonUploader
{
public bool sendMyFileToS3(System.IO.Stream localFilePath, string bucketName, string subDirectoryInBucket, string fileNameInS3)
{
// input explained :
// localFilePath = we will use a file stream , instead of path
// bucketName : the name of the bucket in S3 ,the bucket should be already created
// subDirectoryInBucket : if this string is not empty the file will be uploaded to
// a subdirectory with this name
// fileNameInS3 = the file name in the S3
// create an instance of IAmazonS3 class ,in my case i choose RegionEndpoint.EUWest1
// you can change that to APNortheast1 , APSoutheast1 , APSoutheast2 , CNNorth1
// SAEast1 , USEast1 , USGovCloudWest1 , USWest1 , USWest2 . this choice will not
// store your file in a different cloud storage but (i think) it differ in performance
// depending on your location
IAmazonS3 client = new AmazonS3Client("Your Access Key", "Your Secrete Key", Amazon.RegionEndpoint.USWest2);
// create a TransferUtility instance passing it the IAmazonS3 created in the first step
TransferUtility utility = new TransferUtility(client);
// making a TransferUtilityUploadRequest instance
TransferUtilityUploadRequest request = new TransferUtilityUploadRequest();
if (subDirectoryInBucket == "" || subDirectoryInBucket == null)
{
request.BucketName = bucketName; //no subdirectory just bucket name
}
else
{ // subdirectory and bucket name
request.BucketName = bucketName + #"/" + subDirectoryInBucket;
}
request.Key = fileNameInS3 ; //file name up in S3
//request.FilePath = localFilePath; //local file name
request.InputStream = localFilePath;
request.CannedACL = S3CannedACL.PublicReadWrite;
utility.Upload(request); //commensing the transfer
return true; //indicate that the file was sent
}
}
Here you can use the function sendMyFileToS3 to upload file stream to amazon S3.
For more details check my blog in the following link.
Upload File to Amazon S3 via asp.net
I hope the above mentioned link will help.

Look for a javascript library to handle the client side upload of these files. I stumbled upon a javascript and php example Dojo also seems to offer a clientside s3 file upload.

ThreeSharp is a library to facilitate interactions with Amazon S3 in a .NET environment.
You'll still need to host the logic to upload and send files to s3 in your mvc app, but you won't need to persist them on your server.

Save and GET data in aws s3 bucket in asp.net mvc :-
To save plain text data at amazon s3 bucket.
1.First you need a bucket created on aws than
2.You need your aws credentials like
a)aws key b) aws secretkey c) region
// code to save data at aws
// Note you can get access denied error. to remove this please check AWS account and give //read and write rights
Name space need to add from NuGet package
using Amazon;
using Amazon.S3;
using Amazon.S3.Model;
var credentials = new Amazon.Runtime.BasicAWSCredentials(awsKey, awsSecretKey);
try`
{
AmazonS3Client client = new AmazonS3Client(credentials, RegionEndpoint.APSouth1);
// simple object put
PutObjectRequest request = new PutObjectRequest()
{
ContentBody = "put your plain text here",
ContentType = "text/plain",
BucketName = "put your bucket name here",
Key = "1"
//put unique key to uniquly idenitify your data
// you can pass here any data with unique id like primary key
//in db
};
PutObjectResponse response = client.PutObject(request);
}
catch(exception ex)
{
//
}
Now go to your AWS account and check the bucket you can get data with "1" Name in the AWS s3 bucket.
Note:- if you get any other issue please ask me a question here will try to resolve it.
To get data from AWS s3 bucket:-
try
{
var credentials = new Amazon.Runtime.BasicAWSCredentials(awsKey, awsSecretKey);
AmazonS3Client client = new AmazonS3Client(credentials, RegionEndpoint.APSouth1);
GetObjectRequest request = new GetObjectRequest()
{
BucketName = bucketName,
Key = "1"// because we pass 1 as unique key while save
//data at the s3 bucket
};
using (GetObjectResponse response = client.GetObject(request))
{
StreamReader reader = new
StreamReader(response.ResponseStream);
vccEncryptedData = reader.ReadToEnd();
}
}
catch (AmazonS3Exception)
{
throw;
}

Related

Download an externalReference from PlannerTask using the Graph API

I'd like to download a file attached to a PlannerTask. I already have the external references but I can't figure out how to access the file.
An external reference is a JSON object like this:
{
"https%3A//contoso%2Esharepoint%2Ecom/sites/GroupName/Shared%20Documents/AnnualReport%2Epptx":
{
// ... snip ...
}
}
I've tried to use the following endpoint
GET /groups/{group-id}/drive/root:/sites/GroupName/Shared%20Documents/AnnualReport%2Epptx
but I get a 404 response. Indeed, when I use the query in Graph Explorer it gives me a warning about "Invalid whitespace in URL" (?).
A workaround that I've found is to use the search endpoint to look for files like this:
GET /groups/{group-id}/drive/root/search(q='AnnualReport.pptx')
and hope the file name is unique.
Anyway, with both methods I need extra information (ie. the group-id) that may not be readily available by the time I have the external reference object.
What is the proper way to get a download url for a driveItem that is referenced by an external reference object in a PlannerTask?
Do I really need the group-id to access such file?
The keys in external references are webUrl instances, so they can be used with the /shares/ endpoint. See this answer for details on how to do it.
When you get a driveItem object, the download url is available from AdditionalData: driveItem.AdditionalData["#microsoft.graph.downloadUrl"]. You can use WebClient.DownloadFile to download the file on the local machine.
Here is the final code:
var remoteUri = "https%3A//contoso%2Esharepoint%2Ecom/sites/GroupName/Shared%20Documents/AnnualReport%2Epptx";
var localFile = "/tmp/foo.pptx";
string sharingUrl = remoteUri;
string base64Value = System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(sharingUrl));
string encodedUrl = "u!" + base64Value.TrimEnd('=').Replace('/','_').Replace('+','-');
DriveItem driveItem = m_graphClient
.Shares[encodedUrl]
.DriveItem
.Request()
.GetAsync().Result;
using (WebClient client = new WebClient())
{
client.DownloadFile(driveItem.AdditionalData["#microsoft.graph.downloadUrl"].ToString(),
localFile);
}

Working way to transfer SQLite database to another device without external storage permissions (updates Android 11)

With new Shared Storage and new Android API 30 requirements I can't find a way to recover a database on new or another device.
Until this time, I used File API to create a copy of the database. Any time user could put this file to any cloud, download from cloud on another device and restore the database on it. Of course, I used WRITE_EXTERNAL_STORAGE.
Now, Google rejects the app and says that we should use MediaStore API which can solve that issues without any permissions. I tried it and it works only on same device. And until if created the database file not copied to/from cloud or moved to/from another folder. In that case cursor.getCount == 0:
String[] projections = new String[]{MediaStore.Downloads._ID};
String selection = MediaStore.Downloads.DISPLAY_NAME + "=?";
String[] selectionArgs = new String[]{"database.db"};
String sortOrder = MediaStore.Downloads.DISPLAY_NAME + " ASC";
Cursor cursor = getApplicationContext().getContentResolver().query(MediaStore.Downloads.getContentUri(MediaStore.VOLUME_EXTERNAL), projections, selection, selectionArgs, sortOrder);
Uri uri = null;
if (cursor != null) {
int idColumn = cursor.getColumnIndexOrThrow(MediaStore.Downloads._ID);
cursor.moveToNext();
long id = cursor.getLong(idColumn);
uri = ContentUris.withAppendedId(MediaStore.Downloads.EXTERNAL_CONTENT_URI, id);
try {
ContentResolver contentResolver = getApplicationContext().getContentResolver();
ParcelFileDescriptor parcelFileDescriptor = contentResolver.openFileDescriptor(uri, "rw");
FileInputStream fileInputStream = new FileInputStream(parcelFileDescriptor.getFileDescriptor());
File dbFile = new File(Environment.getDataDirectory(), "/data/com.example.app/databases/database.db");
FileOutputStream fileOutputStream = new FileOutputStream(dbFile);
byte[] buf = new byte[1024];
int len;
while ((len = fileInputStream.read(buf)) > 0) {
fileOutputStream.write(buf, 0, len);
}
fileOutputStream.close();
fileInputStream.close();
parcelFileDescriptor.close();
restartApp();
} catch (IOException e) {
e.printStackTrace();
}
cursor.close();
}
Something wrong with code or MediaStore API does not provide any actions with created files?
E/DBError: exception
android.database.CursorIndexOutOfBoundsException: Index 0 requested, with a size of 0
Also I tried Google Drive API to copy the database file on cloud directly from app and download it. But it's not working way too, because new device don't know about file ID which used to download files.
Any thoughts on this?
Figured out.
Without any storage permissions your app have access to onw created files with MediaStore API.
But haven't if:
file was moved/copied inside device storage
file was uploaded to cloud and then downloaded from it
app was reinstalled
I tested it on versions 28 (emulation), 29 (device), 30 (emulation).
So in my case anyway I need:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" android:maxSdkVersion="29"/>
Don't know why but for read own file on Android Q not enough only READ_EXTERNAL_STORAGE
I was lucky and Google approved app. Hope you too.

How to feed site details on controller in dashlet in alfresco

how to directly get SITE details (like ID and name ) on main() of controller js of dashelt in alfresco
i can use " Alfresco.constants.SITE" on FTL file to read site ID, but need to know is there any KEY to read data on controller
janaka
There isn't a service on the Share side which provides that information, because the information you want is only held on the repository. As such, you'll need to call one of the REST APIs on the Repo to get the information you need
Your code would want to look something like:
// Call the repository for the site profile
var json = remote.call("/api/sites/" + page.url.templateArgs.site);
if (json.status == 200)
{
// Create javascript objects from the repo response
var obj = eval('(' + json + ')');
if (obj)
{
var siteTitle = obj.title;
var siteShortName = obj.shortName;
}
}
You can see a fuller example of this in various Alfresco dashlets, eg the Dynamic Welcome dashlet

ASP.NET Web API 2 file upload

I would like to know how best to handle file upload and addtional information added to the file to be uploaded using ASP.NET Web API 2 without MVC components. I have google the net and I can tell you I am more confused than I expected.
The Additional info will be stored in db and the file on the disk.
So far the Web API app I am building does not support multipart/form-data. It only supports the default media types. I know I need to create a media formatter.
Pls help.
I had wrote Javascript split File and upload to WEB API . i think you can reference my backend codes
In front-end you need using below code to upload your File
var xhr = new self.XMLHttpRequest();
xhr.open('POST', url, false);
xhr.setRequestHeader('Content-Type', 'application/octet-stream');
xhr.send(chunk);
In backend use Request.InputStream.Read to catch your file bytes
[HttpPost]
[ValidateInput(false)]
public string fileUpload(string filename)
{
byte[] file = new byte[Request.InputStream.Length];
Request.InputStream.Read(file, 0, Convert.ToInt32(Request.InputStream.Length));
BinaryWriter binWriter = new BinaryWriter(new MemoryStream());
binWriter.Write(file);
StreamReader reader = new StreamReader(binWriter.BaseStream);
reader.BaseStream.Position = 0;
//This example is recevied text file
while ((line = reader.ReadLine()) != null)
{
};
}
You can just serialize your file data into BASE64 and send them as a string in case of multipart/from-data is not allowed for some reason.

Cannot upload large (>50MB) files to SharePoint 2010 document library

I'm trying to upload a large file to a document library, but it fails after just a few seconds. The upload single document fails silently, upload multiple just shows a failed message. I've turned up the file size limit on the web application to 500MB, and the IIS request length to the same (from this blog), and increased the IIS timeout for good measure. Are there any other size caps that I've missed?
Update I've tried a few files of various sizes, anything 50MB or over fails, so I assume something somewhere is still set to the webapp default.
Update 2 Just tried uploading using the following powershell:
$web = Get-SPWeb http://{site address}
$folder = $web.GetFolder("Site Documents")
$file = Get-Item "C:\mydoc.txt" // ~ 150MB
$folder.Files.Add("SiteDocuments/mydoc.txt", $file.OpenRead(), $false)
and get this exception:
Exception calling "Add" with "3" argument(s): "<nativehr>0x80070003</nativehr><nativestack></nativestack>There is no file with URL 'http://{site address}/SiteDocuments/mydoc.txt' in this Web."
which strikes me as odd as of course the file wouldn't exist until it's been uploaded? N.B. while the document library has the name Site Documents, it has the URL SiteDocuments. Not sure why...
Are you sure you updated the right webapp? Is the filetype blocked by the server? Is there adequate space in your content database? I would check ULS logs after that and see if there is another error since it seems you hit the 3 spots you would need too update.
for uploading a large file, you can use the PUT method instead of using the other ways to upload a document.
by using a put method you will save the file into content database directly. see the example below
Note: the disadvantage of the code below is you cannot catch the object that is responsible for uploading directly, on other word, you cannot update the additional custom properties of the uploaded document directly.
public static bool UploadFileToDocumentLibrary(string sourceFilePath, string targetDocumentLibraryPath)
{
//Flag to indicate whether file was uploaded successfuly or not
bool isUploaded = true;
try
{
// Create a PUT Web request to upload the file.
WebRequest request = WebRequest.Create(targetDocumentLibraryPath);
//Set credentials of the current security context
request.Credentials = CredentialCache.DefaultCredentials;
request.Method = “PUT”;
// Create buffer to transfer file
byte[] fileBuffer = new byte[1024];
// Write the contents of the local file to the request stream.
using (Stream stream = request.GetRequestStream())
{
//Load the content from local file to stream
using (FileStream fsWorkbook = File.Open(sourceFilePath, FileMode.Open, FileAccess.Read))
{
//Get the start point
int startBuffer = fsWorkbook.Read(fileBuffer, 0, fileBuffer.Length);
for (int i = startBuffer; i > 0; i = fsWorkbook.Read(fileBuffer, 0, fileBuffer.Length))
{
stream.Write(fileBuffer, 0, i);
}
}
}
// Perform the PUT request
WebResponse response = request.GetResponse();
//Close response
response.Close();
}
catch (Exception ex)
{
//Set the flag to indiacte failure in uploading
isUploaded = false;
}
//Return the final upload status
return isUploaded;
}
and here are an example of calling this method
UploadFileToDocumentLibrary(#”C:\test.txt”, #”http://home-vs/Shared Documents/textfile.pdf”);

Resources