Alamofire v4, Swift v3 Uploading Sqlite file to domain - sqlite

I’m trying to upload an Sqlite database from IOS Swift 3 to my server using Alamofire 4.0, but having problems converting the sqlite file into the data type required to upload.
The majority of posts / question examples seem to default to uploading images, but I am struggling to find example of uploading sqlite or other file types (for back-up purposes)
I have searched for the basic code and found this so far which looks very reasonable (thanks to following post: Alamofire 4 upload with parameters)
let parameters = ["file_name": "swift_file.jpeg"]
Alamofire.upload(multipartFormData: { (multipartFormData) in
multipartFormData.append(UIImageJPEGRepresentation(self.photoImageView.image!, 1)!, withName: "photo_path", fileName: "swift_file.jpeg", mimeType: "image/jpeg")
for (key, value) in parameters {
multipartFormData.append(value.data(using: String.Encoding.utf8)!, withName: key)
}
}, to:"http://sample.com/upload_img.php")
{ (result) in
switch result
{
case .success(let upload, _, _):
upload.uploadProgress(closure: { (progress) in
//Print progress
})
upload.responseJSON { response in
//print response.result
}
case .failure(let encodingError):
//print encodingError.description
}
}
The part I’m struggling with is to append the sqlite file to the upload (multipartFormData.append(………..?) I’ve searched but not found any good reference posts.
Yes, i’m a newbe, but trying hard, any help would be appreciated…..

It's exactly the same as the image example except that the mime type would be application/octet-stream.
Also, you'd probably go ahead and load it directly from the fileURL rather than loading it into a Data first.
As an aside, the parameters in that example don't quite make sense, as it looks redundant with the filename provided in the upload of the image itself. So you'd use whatever parameters your web service requires, if any. If you have no additional parameters, you'd simply omit the for (key, value) { ... } loop entirely.
Finally, obviously replace the file field name with whatever field name your web service is looking for.
// any additional parameters that must be included in the request (if any)
let parameters = ["somekey": "somevalue"]
// database to be uploaded; I'm assuming it's in Documents, but perhaps you have it elsewhere, so build the URL appropriately for where the file is
let filename = "test.sqlite"
let fileURL = try! FileManager.default.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: false)
.appendingPathComponent(filename)
// now initiate request
Alamofire.upload(multipartFormData: { multipartFormData in
multipartFormData.append(fileURL, withName: "file", fileName: filename, mimeType: "application/octet-stream")
for (key, value) in parameters {
multipartFormData.append(value.data(using: .utf8)!, withName: key)
}
}, to: urlString) { result in
switch result {
case .success(let upload, _, _):
upload
.authenticate(user: self.user, password: self.password) // only needed if you're doing server authentication
.uploadProgress { progress in
print(progress.fractionCompleted)
}
.responseJSON { response in
print("\(response.result.value)")
}
case .failure(let encodingError):
print(encodingError.localizedDescription)
}
}
Unrelated, but if you're ever unsure as to what mime type to use, you can use this routine, which will try to determine mime type from the file extension.
/// Determine mime type on the basis of extension of a file.
///
/// This requires MobileCoreServices framework.
///
/// - parameter url: The file `URL` of the local file for which we are going to determine the mime type.
///
/// - returns: Returns the mime type if successful. Returns application/octet-stream if unable to determine mime type.
func mimeType(for url: URL) -> String {
let pathExtension = url.pathExtension
if let uti = UTTypeCreatePreferredIdentifierForTag(kUTTagClassFilenameExtension, pathExtension as NSString, nil)?.takeRetainedValue() {
if let mimetype = UTTypeCopyPreferredTagWithClass(uti, kUTTagClassMIMEType)?.takeRetainedValue() {
return mimetype as String
}
}
return "application/octet-stream";
}

Related

Receiving <Fault -32700: 'parse error. not well formed'> when trying to upload video to WordPress using xmlrpc.client

Pretty much the title, attaching the python code below.
I'm trying to download files from a specific source and upload them to a WordPress site.
Photos work great, and videos return the following message on wp.call(UploadFile(data)):
<Fault -32700: 'parse error. not well formed'>
Tried using different methods for encoding to base64, reading Wordpress_xmlrpc documentation, nothing.
It's worth mentioning that this method did work on the "Demo" site, and now on once we moved to the "real" site it returns this error.
Any ideas?
def UploadPicture(PathOfFile, Type):
wp = Client(url ,user, password)
filename = PathOfFile #pathofvideo
#Setup
if Type == "jpg":
data = {
'name': 'picture.jpg',
'type': 'image/jpeg', # mimetype
}
if Type == "mp4":
data = {
'name': 'clip.mp4',
'type': 'video/mp4', # mimetype
}
# read the binary file and let the XMLRPC library encode it into base64
with open(filename, 'rb') as img:
data['bits'] = xmlrpc_client.Binary(img.read())
#upload and return ID
response = wp.call(media.UploadFile(data))
attachment_id = response['id']
return(attachment_id)

Sending multipart/form-data using GraphQL API in .NetCore

I'm trying to upload the user avatar of .png/jpeg/.jpg file types from angular client to .netCore server application using GraphQL API.
I managed to send the image to be uploaded in a request of content type multipart/form-data from client-side.
But getting a 400 Error from the API server saying the content-type is not supported.
Error message is as follows:
message: "Invalid 'Content-Type' header: non-supported media type.
Must be of 'application/json', 'application/graphql' or 'application/x-www-form-urlencoded'. See: http://graphql.org/learn/serving-over-http/."
I'm trying to implement the mutation like this.
FieldAsync<StringGraphType>(
"testImageUpload",
arguments: new QueryArguments(
new QueryArgument<StringGraphType> { Name = "testArg" },
new QueryArgument<UploadGraphType> { Name = "file" }
),
resolve: async context =>
{
var testArg = context.GetArgument<string>("testArg");
var file = context.GetArgument<IFormFile>("file");
try
{
return await uploaderService().UploadImage(file);
}
catch (Exception e)
{
context.Errors.Add(new ExecutionError("Something happened!"));
return context.Errors;
}
});
I'm using GraphQL.net and GraphQL.Upload.AspNetCore for supporting multipart files.
Sample mutation will be like this:
mutation testImageUpload($testArg: String, $file: Upload) {
fileUpload{
testImageUpload(testArg: $testArg, file: $file)
}
}
Can anybody suggest to me how to make the .NetCore webAPI application accept multipart/form-data.
Any help will be appreciated. Thanks in advance.
I, too, ran into this issue. The answer lies within your Startup.cs file.
First, the reason for which the "Invalid 'Content-Type' header" appears even while using GraphQL.Upload is because (at the time of this answer) the GraphQL.net middleware only checks for the three different Content-Type headers and errors out if none of those match. See GitHub
As for the solution, you haven't shared any of your Startup.cs file so I'm not sure what yours looks like. I'll share the relevant pieces of mine.
You'll need to add the service and the middleware.
The service:
services.AddGraphQLUpload()
.AddGraphQL((options, provider) =>
{
...
});
If you're using a endpoints to map to a middleware, you'll need to add the UseGraphQLUpload middleware, then map your endpoint.
Example:
app.UseGraphQLUpload<YourSchema>("/api/graphql", new
GraphQLUploadOptions {
UserContextFactory = (ctx) => new GraphQlUserContext(ctx.User)
});
app.UseEndpoints(endpoints =>
{
// map HTTP middleware for YourSchema at path api/graphql
endpoints
.MapGraphQL<YourSchema, GraphQLMiddleware<YourSchema>>("api/graphql")
.RequireAuthorization();
...
// Additional endpoints
...
}
Everything else looks fine to me.

Sending attachment file via http post request

I'm trying to send attachment file from youtrack to another system (in this example to trello) without the use of image url but its content
I cannot send it as image url in youtrack because my system is closed and accessible to only those that have vpn.
Problem is with reading inputStream from content of attachement in workflow. I symply have no idea how to do it and youtrack documentation havent touched it (as far as my research goes)
Code: (with truncated not important parts)
//...
exports.rule = entities.Issue.onChange({
//...
action: function(ctx) {
//...
var link = '/1/cards/' + issue['trelloIssueId'] + '/attachments';
issue.comments.added.forEach(function(comment) {
comment.attachments.forEach(function(attachment) {
var response = connection.postSync(link, {
name: attachment.name,
file: attachment.content,
mimeType: attachment.mimeType
});
//...
});
});
},
requirements: {}
});
from this I got error:
TypeError: invokeMember (forEach) on jetbrains.youtrack.workflow.sandbox.InputStreamWrapper#677a561f failed due to: Unknown identifier: forEach
How do I have to prepare content to ba abble to send it with postSync method?
It looks like you tried to iterate over issue.comments.added while the loop should be executed over issue.comments as there is no added key for an issue's comments Set as per the following documentation page suggest: https://www.jetbrains.com/help/youtrack/devportal/v1-Issue.html
Please let me know if that works.

NSFileProtectionComplete doesn't encrypt the core data file

I am using Xcode 7.3 for iOS 9.3 to try and encrypt a Core Data file. I am trying to use NSPersistentStoreFileProtectionKey and set it to NSFileProtectionComplete to enable the encryption. It is not working for some reason and I can always see the .sqlite file generated by the app and browse through the content in sqlitebrowser or iexplorer. Here is my code :
lazy var persistentStoreCoordinator: NSPersistentStoreCoordinator = {
// The persistent store coordinator for the application. This implementation creates and returns a coordinator, having added the store for the application to it. This property is optional since there are legitimate error conditions that could cause the creation of the store to fail.
// Create the coordinator and store
let coordinator = NSPersistentStoreCoordinator(managedObjectModel: self.managedObjectModel)
let url = self.applicationDocumentsDirectory.URLByAppendingPathComponent("SingleViewCoreData.sqlite")
var failureReason = "There was an error creating or loading the application's saved data."
let dict: [NSObject : AnyObject] = [
NSPersistentStoreFileProtectionKey : NSFileProtectionComplete
]
do {
try coordinator.addPersistentStoreWithType(NSSQLiteStoreType, configuration: nil, URL: url, options: dict)
} catch {
// Report any error we got.
var dict = [String: AnyObject]()
dict[NSLocalizedDescriptionKey] = "Failed to initialize the application's saved data"
dict[NSLocalizedFailureReasonErrorKey] = failureReason
dict[NSUnderlyingErrorKey] = error as NSError
let wrappedError = NSError(domain: "YOUR_ERROR_DOMAIN", code: 9999, userInfo: dict)
// Replace this with code to handle the error appropriately.
// abort() causes the application to generate a crash log and terminate. You should not use this function in a shipping application, although it may be useful during development.
NSLog("Unresolved error \(wrappedError), \(wrappedError.userInfo)")
abort()
}
do {
let url = self.applicationDocumentsDirectory.URLByAppendingPathComponent("SingleViewCoreData.sqlite")
try NSFileManager.defaultManager().setAttributes([NSFileProtectionKey : NSFileProtectionComplete], ofItemAtPath: url.path!)
} catch {
}
do {
let url = self.applicationDocumentsDirectory.URLByAppendingPathComponent("SingleViewCoreData.sqlite-wal")
try NSFileManager.defaultManager().setAttributes([NSFileProtectionKey : NSFileProtectionComplete], ofItemAtPath: url.path!)
// try print(NSFileManager.defaultManager().attributesOfFileSystemForPath(String(url)))
} catch {
}
do {
let url = self.applicationDocumentsDirectory.URLByAppendingPathComponent("SingleViewCoreData.sqlite-shm")
try NSFileManager.defaultManager().setAttributes([NSFileProtectionKey : NSFileProtectionComplete], ofItemAtPath: url.path!)
// try print(NSFileManager.defaultManager().attributesOfFileSystemForPath(String(url)))
} catch {
}
return coordinator
}()
I have also enabled Data Protection for my target in the "Capabilities". I have regenerated the provisioning profile from the Apple Developer portal and am using that with Enabled Data Protection.
I am also using the following code to check the file attributes of .sqlite , .sqlite-wal and .sqlite-shm files. NSFileProtectionKey is correctly set for all 3 of them.
func checkProtectionForLocalDb(atDir : String){
let fileManager = NSFileManager.defaultManager()
let enumerator: NSDirectoryEnumerator = fileManager.enumeratorAtPath(atDir)!
for path in enumerator {
let attr : NSDictionary = enumerator.fileAttributes!
print(attr)
}
}
I also tried disabling the Journal mode to prevent -wal and -shm files from being created. But I can still read the .sqlite file. Even though the attributes read NSFileProtectionComplete.
As described in the Apple Documentation at Apple Docs under "Protecting Data using On Disk Encryption", I tried to check whether the value of variable protectedDataAvailable changes as shown in the code below
public func applicationDidEnterBackground(application: UIApplication) {
// Use this method to release shared resources, save user data, invalidate timers, and store enough application state information to restore your application to its current state in case it is terminated later.
// If your application supports background execution, this method is called instead of applicationWillTerminate: when the user quits.
NSThread.sleepForTimeInterval(10)
sleep(10)
let dataAvailable : Bool = UIApplication.sharedApplication().protectedDataAvailable
print("Protected Data Available : " + String(dataAvailable))
}
If I check the value without the delay it's set to true but after adding the delay it's set to false. This is kind of encouraging, however, right after, when I download the container, to show the content, it still has .sqlite file that still shows the content when opened in sqlitebrowser.
Ok, I finally understand this.
Using Xcode 7.3.1
Enabling File Protection
Enable File Protection using the Capabilities tab on your app target
If you do not want the default NSFileProtectionComplete, change this setting in the developer portal under your app id
Make sure Xcode has the new provisioning profile this creates.
For protecting files your app creates, that's it.
To protect Core Data, you need to add the NSPersistentStoreFileProtectionKey: NSFileProtectionComplete option to your persistent store.
Example:
var options: [NSObject : AnyObject] = [NSMigratePersistentStoresAutomaticallyOption: true,
NSPersistentStoreFileProtectionKey: NSFileProtectionComplete,
NSInferMappingModelAutomaticallyOption: true]
do {
try coordinator!.addPersistentStoreWithType(NSSQLiteStoreType, configuration: nil, URL: url, options: options)
Testing File Protection
I am not able to test this using a non-jailbroken device connected to a computer. Every attempt to access the device this way requires that I "trust" the computer and I believe that trusted computers are always able to read the phone's data ("Trusted computers can sync with your iOS device, create backups, and access your device's photos, videos, contacts, and other content" - https://support.apple.com/en-us/HT202778). I think the other answers on SO referencing this technique are no longer valid with more recent versions of iOS. Indeed, I am always able to download the container using XCode and view the app's data using iPhone Explorer. So how to test...
1 - Create an archive and ensure that it is has the proper entitlements by running the following on the .app file from the command line:
codesign -d --entitlements :- <path_to_app_binary>
You should see a key/value pair that represents your Data Protection level. In this example, NSFileProtectionComplete:
<key>com.apple.developer.default-data-protection</key>
<string>NSFileProtectionComplete</string>
In addition, I used the following two techniques to satisfy myself that the data protection is indeed working. They both require code changes.
2 - Add some code to verify that the proper NSFileProtectionKey is being set on your files and/or core data store:
NSFileManager.defaultManager().attributesOfItemAtPath(dbPath.path!)
If I print this out on one of my files I get:
["NSFileCreationDate": 2016-10-14 02:06:39 +0000, "NSFileGroupOwnerAccountName": mobile, "NSFileType": NSFileTypeRegular, "NSFileSystemNumber": 16777218, "NSFileOwnerAccountName": mobile, "NSFileReferenceCount": 1, "NSFileModificationDate": 2016-10-14 02:06:39 +0000, "NSFileExtensionHidden": 0, "NSFileSize": 81920, "NSFileGroupOwnerAccountID": 501, "NSFileOwnerAccountID": 501, "NSFilePosixPermissions": 420, "NSFileProtectionKey": NSFileProtectionComplete, "NSFileSystemFileNumber": 270902]
Note the "NSFileProtectionKey": "NSFileProtectionComplete" pair.
3 - Modify the following code and hook it up to some button in your app.
#IBAction func settingButtonTouch(sender: AnyObject) {
updateTimer = NSTimer.scheduledTimerWithTimeInterval(0.5, target: self,
selector: #selector(TabbedOverviewViewController.runTest), userInfo: nil, repeats: true)
registerBackgroundTask()
}
var backgroundTask: UIBackgroundTaskIdentifier = UIBackgroundTaskInvalid
var updateTimer: NSTimer?
func registerBackgroundTask() {
backgroundTask = UIApplication.sharedApplication().beginBackgroundTaskWithExpirationHandler {
[unowned self] in
self.endBackgroundTask()
}
assert(backgroundTask != UIBackgroundTaskInvalid)
}
func endBackgroundTask() {
NSLog("Background task ended.")
UIApplication.sharedApplication().endBackgroundTask(backgroundTask)
backgroundTask = UIBackgroundTaskInvalid
}
func runTest() {
switch UIApplication.sharedApplication().applicationState {
case .Active:
NSLog("App is active.")
checkFiles()
case .Background:
NSLog("App is backgrounded.")
checkFiles()
case .Inactive:
break
}
}
func checkFiles() {
// attempt to access a protected resource, i.e. a core data store or file
}
When you tap the button this code begins executing the checkFiles method every .5 seconds. This should run indefinitely with the app in the foreground or background - until you lock your phone. At that point it should reliably fail after roughly 10 seconds - exactly as described in the description of NSFileProtectionComplete.
We need to understand how Data Protection works.
Actually, you don't even need to enable it. Starting with iOS7, the default protection level is “File Protection Complete until first user authentication.”
This means that the files are not accessible until the user unlocks the device for the first time. After that, the files remain accessible even when the device is locked and until it shuts down or reboots.
The other thing is that you're going to see the app's data on a trusted computer always - regardless of the Data Protection level setting.
However, the data can’t be accessed if somebody tries to read them from the flash drive directly. The purpose of Data Protection is to ensure that sensitive data can’t be extracted from a password-protected device’s storage.
After running this code, I could still access and read the contents written to protectedFileURL, even after locking the device.
do {
try data.write(to: protectedFileURL, options: .completeFileProtectionUnlessOpen)
} catch {
print(error)
}
But that's normal since I ran iExplorer on a trusted computer.
And for the same reason, it's fine if you see your sqlite file.
The situation is different if your device gets lost or stolen. A hacker won't be able to read the sqlite file since it's encrypted. Well, unless he guesses your passcode somehow.
Swift 5.0 & Xcode 11:
Enable "Data Protection" in "Capabilities".
Use the following code to protect a file or folder at a specific path:
// Protects a file or folder + excludes it from backup.
// - parameter path: Path component of the file.
// - parameter fileProtectionType: `FileProtectionType`.
// - returns: True, when protected successful.
static func protectFileOrFolderAtPath(_ path: String, fileProtectionType: FileProtectionType) -> Bool {
guard FileManager.default.fileExists(atPath: path) else { return false }
let fileProtectionAttrs = [FileAttributeKey.protectionKey: fileProtectionType]
do {
try FileManager.default.setAttributes(fileProtectionAttrs, ofItemAtPath: path)
return true
} catch {
assertionFailure("Failed protecting path with error: \(error).")
return false
}
}
(Optional) Use the following code to check whether the file or folder at the specific path is protected (note: This only works on physical devices):
/// Returns true, when the file at the provided path is protected.
/// - parameter path: Path of the file to check.
/// - note: Returns true, for simulators. Simulators do not have hardware file encryption. This feature is only available for real devices.
static func isFileProtectedAtPath(_ path: String) -> Bool {
guard !Environment.isSimulator else { return true } // file protection does not work on simulator!
do {
let attributes = try FileManager.default.attributesOfItem(atPath: path)
if attributes.contains(where: { $0.key == .protectionKey }) {
return true
} else {
return false
}
} catch {
assertionFailure(String(describing: error))
return false
}
}
Rather than encrypt a file at the local level I set NSFileProtectionComplete for the app as a whole.
Create the file 'entitlements.plist' in your apps root folder with the following content.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>DataProtectionClass</key>
<string>NSFileProtectionComplete</string>
</dict>
</plist>
Then if you haven't already done so already (this could be the problem with your file level encryption) enable Data Protection in your apps capabilities.

In Node.JS, when I do a POST request, what is the maximum size? [duplicate]

I created an upload script in node.js using express/formidable. It basically works, but I am wondering where and when to check the uploaded file e. g. for the maximum file size or if the file´s mimetype is actually allowed.
My program looks like this:
app.post('/', function(req, res, next) {
req.form.on('progress', function(bytesReceived, bytesExpected) {
// ... do stuff
});
req.form.complete(function(err, fields, files) {
console.log('\nuploaded %s to %s', files.image.filename, files.image.path);
// ... do stuff
});
});
It seems to me that the only viable place for checking the mimetype/file size is the complete event where I can reliably use the filesystem functions to get the size of the uploaded file in /tmp/ – but that seems like a not so good idea because:
the possibly malicious/too large file is already uploaded on my server
the user experience is poor – you watch the upload progress just to be told that it didnt work afterwards
Whats the best practice for implementing this? I found quite a few examples for file uploads in node.js but none seemed to do the security checks I would need.
With help from some guys at the node IRC and the node mailing list, here is what I do:
I am using formidable to handle the file upload. Using the progress event I can check the maximum filesize like this:
form.on('progress', function(bytesReceived, bytesExpected) {
if (bytesReceived > MAX_UPLOAD_SIZE) {
console.log('### ERROR: FILE TOO LARGE');
}
});
Reliably checking the mimetype is much more difficult. The basic Idea is to use the progress event, then if enough of the file is uploaded use a file --mime-type call and check the output of that external command. Simplified it looks like this:
// contains the path of the uploaded file,
// is grabbed in the fileBegin event below
var tmpPath;
form.on('progress', function validateMimetype(bytesReceived, bytesExpected) {
var percent = (bytesReceived / bytesExpected * 100) | 0;
// pretty basic check if enough bytes of the file are written to disk,
// might be too naive if the file is small!
if (tmpPath && percent > 25) {
var child = exec('file --mime-type ' + tmpPath, function (err, stdout, stderr) {
var mimetype = stdout.substring(stdout.lastIndexOf(':') + 2, stdout.lastIndexOf('\n'));
console.log('### file CALL OUTPUT', err, stdout, stderr);
if (err || stderr) {
console.log('### ERROR: MIMETYPE COULD NOT BE DETECTED');
} else if (!ALLOWED_MIME_TYPES[mimetype]) {
console.log('### ERROR: INVALID MIMETYPE', mimetype);
} else {
console.log('### MIMETYPE VALIDATION COMPLETE');
}
});
form.removeListener('progress', validateMimetype);
}
});
form.on('fileBegin', function grabTmpPath(_, fileInfo) {
if (fileInfo.path) {
tmpPath = fileInfo.path;
form.removeListener('fileBegin', grabTmpPath);
}
});
The new version of Connect (2.x.) has this already baked into the bodyParser using the limit middleware: https://github.com/senchalabs/connect/blob/master/lib/middleware/multipart.js#L44-61
I think it's much better this way as you just kill the request when it exceeds the maximum limit instead of just stopping the formidable parser (and letting the request "go on").
More about the limit middleware: http://www.senchalabs.org/connect/limit.html

Resources