Can't find memory leak in Swift app - recursion

I'm trying to learn Swift and so I wrote a little test application to that end. It just gives the total size of items in a directory, recursing into subdirectories to accumulate the total size of the their contents. The application works, but the memory usage just grows and grows while it runs. I had expected the memory usage to increase as the recursion got deeper and decrease when a recursive call returns. Instead, the memory usage just constantly climbs. Instruments doesn't identify any leaks. I've tried a few tips I've found in various google results, including:
re-using the default NSFileManager
not re-using the default NSFileManager but creating a new one for each recursive call
avoiding String interpolation
Nothing seems to make any difference. I had thought that Swift would clean up objects as their reference count reached zero.
This is the code in its entirety in its current state:
import Foundation
func sizeOfContents(path: String) -> UInt64
{
let subManager = NSFileManager()
var totalSize: UInt64 = 0;
var isDir: ObjCBool = false
if subManager.fileExistsAtPath(path, isDirectory: &isDir)
{
if !isDir.boolValue
{
var error: NSError? = nil
let attributes: NSDictionary? = subManager.attributesOfItemAtPath(path, error: &error)
let size: UInt64? = attributes?.fileSize()
totalSize += size!
}
else
{
var error: NSError? = nil
if let subContents = subManager.contentsOfDirectoryAtPath(path, error: &error)
{
for subItem in subContents
{
var subName = subItem as String
subName = path + "/" + subName
totalSize += sizeOfContents(subName)
}
}
}
}
return totalSize
}
let manager = NSFileManager.defaultManager()
var rootPath = "/Applications/"
if let contents = manager.contentsOfDirectoryAtPath(rootPath, error: nil)
{
for item in contents
{
let itemName = item as String
var isDir: ObjCBool = false
print("item: " + (rootPath + itemName))
if manager.fileExistsAtPath(rootPath + itemName, isDirectory: &isDir)
{
if !isDir.boolValue
{
var error: NSError? = nil
let attributes: NSDictionary? = manager.attributesOfItemAtPath(rootPath + itemName, error: &error)
let size: UInt64? = attributes?.fileSize()
println("\t\(size!)")
}
else
{
if(itemName != "Volumes")
{
let size = sizeOfContents(rootPath + itemName)
println("\t\(size)")
}
}
}
}
}

You need to add an autoreleasepool in the loop, possibly around the recursive call. Since it is a tight loop the idle loop is not getting a change to release the temp memory allocations.
example:
...
autoreleasepool {
totalSize += sizeOfContents(subName)
}
...

Related

Passing value by reference to Qore script function from C++ code

I need pass returnValue to a method as argument passed by reference and adjust original var value when function id done. So using ReferenceArgumentHelper class.
What's wrong in code bellow when returnValue is unintentionally deleted (when it is a node, i.e. string) and valgrind detects it. callMethod("onFunctionExit" calls an Qore script method and I can see there correct returnValue value. I suspect it's deleted when exiting onFunctionExit when ReferenceArgumentHelper is destroyed. rah.getArg() references reference variable, so it should not be deleted in callMethod.
DLLLOCAL ThreadDebugEnum callMethod(const char* name, const ThreadDebugEnum defaultResult, QoreProgram *pgm, int paramCount, AbstractQoreNode** params, ExceptionSink* xsink) {
int rv;
QoreListNode* l = new QoreListNode();
qore_program_to_object_map_t::iterator i = qore_program_to_object_map.find(pgm);
if (i == qore_program_to_object_map.end()) {
return defaultResult;
}
i->second->ref();
l->push(i->second);
for (int i=0; i<paramCount; i++) {
if (params[i])
params[i]->ref();
l->push(params[i]);
}
rv = qo->intEvalMethod(name, l, xsink);
l->deref(xsink);
return (ThreadDebugEnum) rv;
}
DLLLOCAL virtual ThreadDebugEnum onFunctionExit(QoreProgram *pgm, const StatementBlock *blockStatement, QoreValue& returnValue, ExceptionSink* xsink) {
AbstractQoreNode* params[2];
params[0] = getLocation(blockStatement);
ReferenceArgumentHelper rah(returnValue.takeNode(), xsink); // grab node from returnValue and pass to helper
params[1] = rah.getArg(); // caller owns ref
ThreadDebugEnum rv = callMethod("onFunctionExit", DBG_SB_RUN, pgm, 2, params, xsink);
AbstractQoreNode* rc = rah.getOutputValue(); // caller owns ref
returnValue.assign(rc); // takes reference
// returnValue.ref();
}
return rv;
}
When looking deeply I did not get why compiler is happy with code in /lib/ReferenceArgumentHelper.cpp:
struct lvih_intern {
LocalVar lv;
ExceptionSink* xsink;
ReferenceNode* ref;
DLLLOCAL lvih_intern(AbstractQoreNode* val, ExceptionSink* xs) : lv("ref_arg_helper", 0), xsink(xs) {
printd(5, "ReferenceArgumentHelper::ReferenceArgumentHelper() instantiating %p (val: %p type: '%s') \n", &lv, val, val ? val->getTypeName() : "n/a");
lv.instantiate(val); <--------------
VarRefNode* vr = new VarRefNode(strdup("ref_arg_helper"), VT_LOCAL);
vr->ref.id = &lv;
ref = new ReferenceNode(vr, 0, vr, 0);
}
class LocalVar {
....
DLLLOCAL void instantiate(QoreValue nval) const {
What is behind conversion AbstractQoreNode* to QoreValue in method call? I did not find an overloaded operator or so. I'm looking what exactly happens with references.
** EDIT **
To make a long story short, ReferenceArgumentHelper was buggy; it hadn't been used in years and was not up to date. The class has been fixed which should fix your issue I hope.
Thank you for pointing this out, and let me know if you have any further problems with this or the fix to the affected code.

CMBlockBuffer, UnsafeMutablePointer et al in Swift

I am trying to convert some Objective C code provided in one of Apple's code examples here: https://developer.apple.com/library/mac/samplecode/avsubtitleswriterOSX/Listings/avsubtitleswriter_SubtitlesTextReader_m.html
The result I have come up with thus far is as follows:
func copySampleBuffer() -> CMSampleBuffer? {
var textLength : Int = 0
var sampleSize : Int = 0
if (text != nil) {
textLength = text!.characters.count
sampleSize = text!.lengthOfBytesUsingEncoding(NSUTF16StringEncoding)
}
var sampleData = [UInt8]()
// Append text length
sampleData.append(UInt16(textLength).hiByte())
sampleData.append(UInt16(textLength).loByte())
// Append the text
for char in (text?.utf16)! {
sampleData.append(char.bigEndian.hiByte())
sampleData.append(char.bigEndian.loByte())
}
if (self.forced) {
// TODO
}
let samplePtr = UnsafeMutablePointer<[UInt8]>.alloc(1)
samplePtr.memory = sampleData
var sampleTiming = CMSampleTimingInfo()
sampleTiming.duration = self.timeRange.duration;
sampleTiming.presentationTimeStamp = self.timeRange.start;
sampleTiming.decodeTimeStamp = kCMTimeInvalid;
let formatDescription = copyFormatDescription()
let dataBufferUMP = UnsafeMutablePointer<Optional<CMBlockBuffer>>.alloc(1)
CMBlockBufferCreateWithMemoryBlock(kCFAllocatorDefault, samplePtr, sampleSize, kCFAllocatorMalloc, nil, 0, sampleSize, 0, dataBufferUMP);
let sampleBufferUMP = UnsafeMutablePointer<Optional<CMSampleBuffer>>.alloc(1)
CMSampleBufferCreate(kCFAllocatorDefault, dataBufferUMP.memory, true, nil, nil, formatDescription, 1, 1, &sampleTiming, 1, &sampleSize, sampleBufferUMP);
let sampleBuffer = sampleBufferUMP.memory
sampleBufferUMP.destroy()
sampleBufferUMP.dealloc(1)
dataBufferUMP.destroy()
dataBufferUMP.dealloc(1)
samplePtr.destroy()
//Crash if I call dealloc here
//Error is: error for object 0x10071e400: pointer being freed was not allocated
//samplePtr.dealloc(1)
return sampleBuffer;
}
I would like to avoid the "Unsafe*" types where possible, though I am not sure it is possible here. I also looked at using a struct and then somehow seeing to pack it somehow, but example I see seem to be based of sizeof, which uses the size of the definition, rather than the current size of the structure. This would have been the structure I would have used:
struct SubtitleAtom {
var length : UInt16
var text : [UInt16]
var forced : Bool?
}
Any advice on most suitable Swift 2 code for this function would be appreciated.
so, at first, you code use this pattern
class C { deinit { print("I got deinit'd!") } }
struct S { var objectRef:AnyObject? }
func foo() {
let ptr = UnsafeMutablePointer<S>.alloc(1)
let o = C()
let fancy = S(objectRef: o)
ptr.memory = fancy
ptr.destroy() //deinit runs here!
ptr.dealloc(1) //don't leak memory
}
// soon or later this code should crash :-)
(1..<1000).forEach{ i in
foo()
print(i)
}
Try it in a playground and most likely it crash :-). What's wrong with it? The trouble is your unbalanced retain / release cycles. How to write the same in the safe manner? You removed dealloc part. But try to do it in my snippet and see the result. The code crash again :-). The only safe way is to properly initialize and de-ininitialize (destroy) the underlying ptr's Memory as you can see in next snippet
class C { deinit { print("I got deinit'd!") } }
struct S { var objectRef:AnyObject? }
func foo() {
let ptr = UnsafeMutablePointer<S>.alloc(1)
let o = C()
let fancy = S(objectRef: o)
ptr.initialize(fancy)
ptr.destroy()
ptr.dealloc(1)
}
(1..<1000).forEach{ i in
foo()
print(i)
}
Now the code is executed as expected and all retain / release cycles are balanced.

Network timeout using node.js client at about 5 seconds

Note, this question was previously very different. This is now the real issue. Which is...
When making a call to executeStoredProcedure() using the node.js client I get a 408 code, RequestTimeout and I get no data back from the sproc's "body". This seems to occur at about 5 seconds, but when I time bound things from inside the sproc itself, any value over say 700 milliseconds causes me to get a network timeout (although I don't see it until about 5 seconds have passed).
Note, I can have longer running sprocs with read operations. This only seems to occur when I have a lot of createDocument() operations, so I don't think it's on the client side. I think something is happening on the server side.
It's still possible that my original thought is true and I'm not getting a false back from a createDocument() call which causes my sproc to keep running past its timeout and that's what's causing the 408.
Here is the time limited version of my create documents sproc
generateData = function(memo) {
var collection, collectionLink, nowTime, row, startTime, timeout;
if ((memo != null ? memo.remaining : void 0) == null) {
throw new Error('generateData must be called with an object containing a `remaining` field.');
}
if (memo.totalCount == null) {
memo.totalCount = 0;
}
memo.countForThisRun = 0;
timeout = memo.timeout || 600; // Works at 600. Fails at 800.
startTime = new Date();
memo.stillTime = true;
collection = getContext().getCollection();
collectionLink = collection.getSelfLink();
memo.stillQueueing = true;
while (memo.remaining > 0 && memo.stillQueueing && memo.stillTime) {
row = {
a: 1,
b: 2
};
getContext().getResponse().setBody(memo);
memo.stillQueueing = collection.createDocument(collectionLink, row);
if (memo.stillQueueing) {
memo.remaining--;
memo.countForThisRun++;
memo.totalCount++;
}
nowTime = new Date();
memo.nowTime = nowTime;
memo.startTime = startTime;
memo.stillTime = (nowTime - startTime) < timeout;
if (memo.stillTime) {
memo.continuation = null;
} else {
memo.continuation = 'Value does not matter';
}
}
getContext().getResponse().setBody(memo);
return memo;
};
The stored procedure above queues document creates in a while loop until the API returns false.
Keep in mind that createDocument() is an asynchronous method. The boolean returned represents whether it is time to wrap up execution right there and then. The return value isn't "smart" enough to estimate and account for how much time the async call will take; so it can't be used for queueing a bunch of calls in a while() loop.
As a result, the stored procedure above doesn't terminate gracefully when the boolean returns false because it has a bunch of createDocument() calls that are still running. The end result is a timeout (which eventually leads to blacklisting on repeated attempts).
In short, avoid this pattern:
while (stillQueueing) {
stillQueueing = collection.createDocument(collectionLink, row);
}
Instead, you should use the callback for control flow. Here is the refactored code:
function(memo) {
var collection = getContext().getCollection();
var collectionLink = collection.getSelfLink();
var row = {
a: 1,
b: 2
};
if ((memo != null ? memo.remaining : void 0) == null) {
throw new Error('generateData must be called with an object containing a `remaining` field.');
}
if (memo.totalCount == null) {
memo.totalCount = 0;
}
memo.countForThisRun = 0;
createMemo();
function createMemo() {
var isAccepted = collection.createDocument(collectionLink, row, function(err, createdDoc) {
if (err) throw err;
memo.remaining--;
memo.countForThisRun++;
memo.totalCount++;
if (memo.remaining > 0) {
createMemo();
} else {
getContext().getResponse().setBody(memo);
}
});
if (!isAccepted) {
getContext().getResponse().setBody(memo);
}
}
};

Accessing an SQLite Database in Swift

I'm looking for a way to access an SQLite database in my app with Swift code.
I know that I can use an SQLite Wrapper in Objective C and use the bridging header, but I'd rather be able to do this project entirely in Swift. Is there a way to do this, if so, can someone point me to a reference that shows how to submit a query, retrieve rows, etc?
While you should probably use one of the many SQLite wrappers, if you wanted to know how to call the SQLite library yourself, you would:
Configure your Swift project to handle SQLite C calls. If using Xcode 9 or later, you can simply do:
import SQLite3
Create/open database.
let fileURL = try! FileManager.default
.url(for: .applicationSupportDirectory, in: .userDomainMask, appropriateFor: nil, create: true)
.appendingPathComponent("test.sqlite")
// open database
var db: OpaquePointer?
guard sqlite3_open(fileURL.path, &db) == SQLITE_OK else {
print("error opening database")
sqlite3_close(db)
db = nil
return
}
Note, I know it seems weird to close the database upon failure to open, but the sqlite3_open documentation makes it explicit that we must do so to avoid leaking memory:
Whether or not an error occurs when it is opened, resources associated with the database connection handle should be released by passing it to sqlite3_close() when it is no longer required.
Use sqlite3_exec to perform SQL (e.g. create table).
if sqlite3_exec(db, "create table if not exists test (id integer primary key autoincrement, name text)", nil, nil, nil) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("error creating table: \(errmsg)")
}
Use sqlite3_prepare_v2 to prepare SQL with ? placeholder to which we'll bind value.
var statement: OpaquePointer?
if sqlite3_prepare_v2(db, "insert into test (name) values (?)", -1, &statement, nil) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("error preparing insert: \(errmsg)")
}
if sqlite3_bind_text(statement, 1, "foo", -1, SQLITE_TRANSIENT) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("failure binding foo: \(errmsg)")
}
if sqlite3_step(statement) != SQLITE_DONE {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("failure inserting foo: \(errmsg)")
}
Note, that uses the SQLITE_TRANSIENT constant which can be implemented as follows:
internal let SQLITE_STATIC = unsafeBitCast(0, to: sqlite3_destructor_type.self)
internal let SQLITE_TRANSIENT = unsafeBitCast(-1, to: sqlite3_destructor_type.self)
Reset SQL to insert another value. In this example, I'll insert a NULL value:
if sqlite3_reset(statement) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("error resetting prepared statement: \(errmsg)")
}
if sqlite3_bind_null(statement, 1) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("failure binding null: \(errmsg)")
}
if sqlite3_step(statement) != SQLITE_DONE {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("failure inserting null: \(errmsg)")
}
Finalize prepared statement to recover memory associated with that prepared statement:
if sqlite3_finalize(statement) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("error finalizing prepared statement: \(errmsg)")
}
statement = nil
Prepare new statement for selecting values from table and loop through retrieving the values:
if sqlite3_prepare_v2(db, "select id, name from test", -1, &statement, nil) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("error preparing select: \(errmsg)")
}
while sqlite3_step(statement) == SQLITE_ROW {
let id = sqlite3_column_int64(statement, 0)
print("id = \(id); ", terminator: "")
if let cString = sqlite3_column_text(statement, 1) {
let name = String(cString: cString)
print("name = \(name)")
} else {
print("name not found")
}
}
if sqlite3_finalize(statement) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("error finalizing prepared statement: \(errmsg)")
}
statement = nil
Close database:
if sqlite3_close(db) != SQLITE_OK {
print("error closing database")
}
db = nil
For Swift 2 and older versions of Xcode, see previous revisions of this answer.
The best you can do is import the dynamic library inside a bridging header:
Add libsqlite3.dylib to your "Link Binary With Libraries" build phase
Create a "Bridging-Header.h" and add #import <sqlite3.h> to the top
set "Bridging-Header.h" for the "Objective-C Bridging Header" setting in Build Settings under "Swift Compiler - Code Generation"
You will then be able to access all of the c methods like sqlite3_open from your swift code.
However, you may just want to use FMDB and import that through the bridging header as that is a more object oriented wrapper of sqlite. Dealing with C pointers and structs will be cumbersome in Swift.
I too was looking for some way to interact with SQLite the same way I was used to doing previously in Objective-C. Admittedly, because of C compatibility, I just used the straight C API.
As no wrapper currently exists for SQLite in Swift and the SQLiteDB code mentioned above goes a bit higher level and assumes certain usage, I decided to create a wrapper and get a bit familiar with Swift in the process. You can find it here: https://github.com/chrismsimpson/SwiftSQLite.
var db = SQLiteDatabase();
db.open("/path/to/database.sqlite");
var statement = SQLiteStatement(database: db);
if ( statement.prepare("SELECT * FROM tableName WHERE Id = ?") != .Ok )
{
/* handle error */
}
statement.bindInt(1, value: 123);
if ( statement.step() == .Row )
{
/* do something with statement */
var id:Int = statement.getIntAt(0)
var stringValue:String? = statement.getStringAt(1)
var boolValue:Bool = statement.getBoolAt(2)
var dateValue:NSDate? = statement.getDateAt(3)
}
statement.finalizeStatement(); /* not called finalize() due to destructor/language keyword */
I've created an elegant SQLite library written completely in Swift called SwiftData.
Some of its feature are:
Bind objects conveniently to the string of SQL
Support for transactions and savepoints
Inline error handling
Completely thread safe by default
It provides an easy way to execute 'changes' (e.g. INSERT, UPDATE, DELETE, etc.):
if let err = SD.executeChange("INSERT INTO Cities (Name, Population, IsWarm, FoundedIn) VALUES ('Toronto', 2615060, 0, '1793-08-27')") {
//there was an error during the insert, handle it here
} else {
//no error, the row was inserted successfully
}
and 'queries' (e.g. SELECT):
let (resultSet, err) = SD.executeQuery("SELECT * FROM Cities")
if err != nil {
//there was an error during the query, handle it here
} else {
for row in resultSet {
if let name = row["Name"].asString() {
println("The City name is: \(name)")
}
if let population = row["Population"].asInt() {
println("The population is: \(population)")
}
if let isWarm = row["IsWarm"].asBool() {
if isWarm {
println("The city is warm")
} else {
println("The city is cold")
}
}
if let foundedIn = row["FoundedIn"].asDate() {
println("The city was founded in: \(foundedIn)")
}
}
}
Along with many more features!
You can check it out here
Yet another SQLite wrapper for Swift 2 and Swift 3: http://github.com/groue/GRDB.swift
Features:
An API that will look familiar to users of ccgus/fmdb
A low-level SQLite API that leverages the Swift standard library
A pretty Swift query interface for SQL-allergic developers
Support for the SQLite WAL mode, and concurrent database access for extra performance
A Record class that wraps result sets, eats your custom SQL queries for breakfast, and provides basic CRUD operations
Swift type freedom: pick the right Swift type that fits your data. Use Int64 when needed, or stick with the convenient Int. Store and read NSDate or NSDateComponents. Declare Swift enums for discrete data types. Define your own database-convertible types.
Database Migrations
Speed: https://github.com/groue/GRDB.swift/wiki/Performance
AppDelegate.swift
func createDatabase()
{
var path:Array=NSSearchPathForDirectoriesInDomains(FileManager.SearchPathDirectory.documentDirectory, FileManager.SearchPathDomainMask.userDomainMask, true)
let directory:String=path[0]
let DBpath=(directory as NSString).appendingPathComponent("Food.sqlite")
print(DBpath)
if (FileManager.default.fileExists(atPath: DBpath))
{
print("Successfull database create")
}
else
{
let pathfrom:String=(Bundle.main.resourcePath! as NSString).appendingPathComponent("Food.sqlite")
var success:Bool
do {
try FileManager.default.copyItem(atPath: pathfrom, toPath: DBpath)
success = true
} catch _ {
success = false
}
if !success
{
print("database not create ")
}
else
{
print("Successfull database new create")
}
}
}
Database.swift
import UIKit
class database: NSObject
{
func databasePath() -> NSString
{
var path:Array=NSSearchPathForDirectoriesInDomains(FileManager.SearchPathDirectory.documentDirectory, FileManager.SearchPathDomainMask.userDomainMask, true)
let directory:String=path[0]
let DBpath=(directory as NSString).appendingPathComponent("Food.sqlite")
if (FileManager.default.fileExists(atPath: DBpath))
{
return DBpath as NSString
}
return DBpath as NSString
}
func ExecuteQuery(_ str:String) -> Bool
{
var result:Bool=false
let DBpath:String=self.databasePath() as String
var db: OpaquePointer? = nil
var stmt:OpaquePointer? = nil
let strExec=str.cString(using: String.Encoding.utf8)
if (sqlite3_open(DBpath, &db)==SQLITE_OK)
{
if (sqlite3_prepare_v2(db, strExec! , -1, &stmt, nil) == SQLITE_OK)
{
if (sqlite3_step(stmt) == SQLITE_DONE)
{
result=true
}
}
sqlite3_finalize(stmt)
}
sqlite3_close(db)
return result
}
func SelectQuery(_ str:String) -> Array<Dictionary<String,String>>
{
var result:Array<Dictionary<String,String>>=[]
let DBpath:String=self.databasePath() as String
var db: OpaquePointer? = nil
var stmt:OpaquePointer? = nil
let strExec=str.cString(using: String.Encoding.utf8)
if ( sqlite3_open(DBpath,&db) == SQLITE_OK)
{
if (sqlite3_prepare_v2(db, strExec! , -1, &stmt, nil) == SQLITE_OK)
{
while (sqlite3_step(stmt) == SQLITE_ROW)
{
var i:Int32=0
let icount:Int32=sqlite3_column_count(stmt)
var dict=Dictionary<String, String>()
while i < icount
{
let strF=sqlite3_column_name(stmt, i)
let strV = sqlite3_column_text(stmt, i)
let rFiled:String=String(cString: strF!)
let rValue:String=String(cString: strV!)
//let rValue=String(cString: UnsafePointer<Int8>(strV!))
dict[rFiled] = rValue
i += 1
}
result.insert(dict, at: result.count)
}
sqlite3_finalize(stmt)
}
sqlite3_close(db)
}
return result
}
func AllSelectQuery(_ str:String) -> Array<Model>
{
var result:Array<Model>=[]
let DBpath:String=self.databasePath() as String
var db: OpaquePointer? = nil
var stmt:OpaquePointer? = nil
let strExec=str.cString(using: String.Encoding.utf8)
if ( sqlite3_open(DBpath,&db) == SQLITE_OK)
{
if (sqlite3_prepare_v2(db, strExec! , -1, &stmt, nil) == SQLITE_OK)
{
while (sqlite3_step(stmt) == SQLITE_ROW)
{
let mod=Model()
mod.id=String(cString: sqlite3_column_text(stmt, 0))
mod.image=String(cString: sqlite3_column_text(stmt, 1))
mod.name=String(cString: sqlite3_column_text(stmt, 2))
mod.foodtype=String(cString: sqlite3_column_text(stmt, 3))
mod.vegtype=String(cString: sqlite3_column_text(stmt, 4))
mod.details=String(cString: sqlite3_column_text(stmt, 5))
result.insert(mod, at: result.count)
}
sqlite3_finalize(stmt)
}
sqlite3_close(db)
}
return result
}
}
Model.swift
import UIKit
class Model: NSObject
{
var uid:Int = 0
var id:String = ""
var image:String = ""
var name:String = ""
var foodtype:String = ""
var vegtype:String = ""
var details:String = ""
var mealtype:String = ""
var date:String = ""
}
Access database :
let DB=database()
var mod=Model()
database Query fire :
var DailyResult:Array<Model> = DB.AllSelectQuery("select * from food where foodtype == 'Sea Food' ORDER BY name ASC")
This is by far the best SQLite library that I've used in Swift: https://github.com/stephencelis/SQLite.swift
Look at the code examples. So much cleaner than the C API:
import SQLite
let db = try Connection("path/to/db.sqlite3")
let users = Table("users")
let id = Expression<Int64>("id")
let name = Expression<String?>("name")
let email = Expression<String>("email")
try db.run(users.create { t in
t.column(id, primaryKey: true)
t.column(name)
t.column(email, unique: true)
})
// CREATE TABLE "users" (
// "id" INTEGER PRIMARY KEY NOT NULL,
// "name" TEXT,
// "email" TEXT NOT NULL UNIQUE
// )
let insert = users.insert(name <- "Alice", email <- "alice#mac.com")
let rowid = try db.run(insert)
// INSERT INTO "users" ("name", "email") VALUES ('Alice', 'alice#mac.com')
for user in try db.prepare(users) {
print("id: \(user[id]), name: \(user[name]), email: \(user[email])")
// id: 1, name: Optional("Alice"), email: alice#mac.com
}
// SELECT * FROM "users"
let alice = users.filter(id == rowid)
try db.run(alice.update(email <- email.replace("mac.com", with: "me.com")))
// UPDATE "users" SET "email" = replace("email", 'mac.com', 'me.com')
// WHERE ("id" = 1)
try db.run(alice.delete())
// DELETE FROM "users" WHERE ("id" = 1)
try db.scalar(users.count) // 0
// SELECT count(*) FROM "users"
The documentation also says that "SQLite.swift also works as a lightweight, Swift-friendly wrapper over the C API," and follows with some examples of that.
You can use this library in Swift for SQLite
https://github.com/pmurphyjam/SQLiteDemo
SQLiteDemo
SQLite Demo using Swift with SQLDataAccess class written in Swift
Adding to Your Project
You only need three files to add to your project
* SQLDataAccess.swift
* DataConstants.swift
* Bridging-Header.h
Bridging-Header must be set in your Xcode's project 'Objective-C Bridging Header' under 'Swift Compiler - General'
Examples for Use
Just follow the code in ViewController.swift to see how to write simple SQL with SQLDataAccess.swift
First you need to open the SQLite Database your dealing with
let db = SQLDataAccess.shared
db.setDBName(name:"SQLite.db")
let opened = db.openConnection(copyFile:true)
If openConnection succeeded, now you can do a simple insert into Table AppInfo
//Insert into Table AppInfo
let status = db.executeStatement("insert into AppInfo (name,value,descrip,date) values(?,?,?,?)",
”SQLiteDemo","1.0.2","unencrypted",Date())
if(status)
{
//Read Table AppInfo into an Array of Dictionaries
let results = db.getRecordsForQuery("select * from AppInfo ")
NSLog("Results = \(results)")
}
See how simple that was!
The first term in db.executeStatement is your SQL as String, all the terms that follow are a variadic argument list of type Any, and are your parameters in an Array. All these terms are separated by commas in your list of SQL arguments. You can enter Strings, Integers, Date’s, and Blobs right after the sequel statement since all of these terms are considered to be parameters for the sequel. The variadic argument array just makes it convenient to enter all your sequel in just one executeStatement or getRecordsForQuery call. If you don’t have any parameters, don’t enter anything after your SQL.
The results array is an Array of Dictionary’s where the ‘key’ is your tables column name, and the ‘value’ is your data obtained from SQLite. You can easily iterate through this array with a for loop or print it out directly or assign these Dictionary elements to custom data object Classes that you use in your View Controllers for model consumption.
for dic in results as! [[String:AnyObject]] {
print(“result = \(dic)”)
}
SQLDataAccess will store, text, double, float, blob, Date, integer and long long integers.
For Blobs you can store binary, varbinary, blob.
For Text you can store char, character, clob, national varying character, native character, nchar, nvarchar, varchar, variant, varying character, text.
For Dates you can store datetime, time, timestamp, date.
For Integers you can store bigint, bit, bool, boolean, int2, int8, integer, mediumint, smallint, tinyint, int.
For Doubles you can store decimal, double precision, float, numeric, real, double. Double has the most precision.
You can even store Nulls of type Null.
In ViewController.swift a more complex example is done showing how to insert a Dictionary as a 'Blob'. In addition SQLDataAccess
understands native Swift Date() so you can insert these objects with out converting, and it will convert them to text and store them,
and when retrieved convert them back from text to Date.
Of course the real power of SQLite is it's Transaction capability. Here you can literally queue up 400 SQL statements with parameters
and insert them all at once which is really powerful since it's so fast. ViewController.swift also shows you an example of how to do this.
All you're really doing is creating an Array of Dictionaries called 'sqlAndParams', in this Array your storing Dictionaries with two keys
'SQL' for the String sequel statement or query, and 'PARAMS' which is just an Array of native objects SQLite understands for that query.
Each 'sqlParams' which is an individual Dictionary of sequel query plus parameters is then stored in the 'sqlAndParams' Array.
Once you've created this array, you just call.
let status = db.executeTransaction(sqlAndParams)
if(status)
{
//Read Table AppInfo into an Array of Dictionaries for the above Transactions
let results = db.getRecordsForQuery("select * from AppInfo ")
NSLog("Results = \(results)")
}
In addition all executeStatement and getRecordsForQuery methods can be done with simple String for SQL query and an Array for the parameters needed by the query.
let sql : String = "insert into AppInfo (name,value,descrip) values(?,?,?)"
let params : Array = ["SQLiteDemo","1.0.0","unencrypted"]
let status = db.executeStatement(sql, withParameters: params)
if(status)
{
//Read Table AppInfo into an Array of Dictionaries for the above Transactions
let results = db.getRecordsForQuery("select * from AppInfo ")
NSLog("Results = \(results)")
}
An Objective-C version also exists and is called the same SQLDataAccess, so now you can choose to write your sequel in Objective-C or Swift.
In addition SQLDataAccess will also work with SQLCipher, the present code isn't setup yet to work with it, but it's pretty easy to do, and
an example of how to do this is actually in the Objective-C version of SQLDataAccess.
SQLDataAccess is a very fast and efficient class, and can be used in place of CoreData which really just uses SQLite as it's underlying data
store without all the CoreData core data integrity fault crashes that come with CoreData.
I have written a SQLite3 wrapper library written in Swift.
This is actually a very high level wrapper with very simple API, but anyway, it has low-level C inter-op code, and I post here a (simplified) part of it to shows the C inter-op.
struct C
{
static let NULL = COpaquePointer.null()
}
func open(filename:String, flags:OpenFlag)
{
let name2 = filename.cStringUsingEncoding(NSUTF8StringEncoding)!
let r = sqlite3_open_v2(name2, &_rawptr, flags.value, UnsafePointer<Int8>.null())
checkNoErrorWith(resultCode: r)
}
func close()
{
let r = sqlite3_close(_rawptr)
checkNoErrorWith(resultCode: r)
_rawptr = C.NULL
}
func prepare(SQL:String) -> (statements:[Core.Statement], tail:String)
{
func once(zSql:UnsafePointer<Int8>, len:Int32, inout zTail:UnsafePointer<Int8>) -> Core.Statement?
{
var pStmt = C.NULL
let r = sqlite3_prepare_v2(_rawptr, zSql, len, &pStmt, &zTail)
checkNoErrorWith(resultCode: r)
if pStmt == C.NULL
{
return nil
}
return Core.Statement(database: self, pointerToRawCStatementObject: pStmt)
}
var stmts:[Core.Statement] = []
let sql2 = SQL as NSString
var zSql = UnsafePointer<Int8>(sql2.UTF8String)
var zTail = UnsafePointer<Int8>.null()
var len1 = sql2.lengthOfBytesUsingEncoding(NSUTF8StringEncoding);
var maxlen2 = Int32(len1)+1
while let one = once(zSql, maxlen2, &zTail)
{
stmts.append(one)
zSql = zTail
}
let rest1 = String.fromCString(zTail)
let rest2 = rest1 == nil ? "" : rest1!
return (stmts, rest2)
}
func step() -> Bool
{
let rc1 = sqlite3_step(_rawptr)
switch rc1
{
case SQLITE_ROW:
return true
case SQLITE_DONE:
return false
default:
database.checkNoErrorWith(resultCode: rc1)
}
}
func columnText(at index:Int32) -> String
{
let bc = sqlite3_column_bytes(_rawptr, Int32(index))
let cs = sqlite3_column_text(_rawptr, Int32(index))
let s1 = bc == 0 ? "" : String.fromCString(UnsafePointer<CChar>(cs))!
return s1
}
func finalize()
{
let r = sqlite3_finalize(_rawptr)
database.checkNoErrorWith(resultCode: r)
_rawptr = C.NULL
}
If you want a full source code of this low level wrapper, see these files.
https://github.com/Eonil/SQLite3/blob/master/Swift/Sources/Core.Statement.swift
https://github.com/Eonil/SQLite3/blob/master/Swift/Sources/Core.Database.swift
Configure your Swift project to handle SQLite C calls:
Create bridging header file to the project. See the Importing Objective-C into Swift section of the Using Swift with Cocoa and Objective-C. This bridging header should import sqlite3.h:
Add the libsqlite3.0.dylib to your project. See Apple's documentation regarding adding library/framework to one's project.
and used following code
func executeQuery(query: NSString ) -> Int
{
if sqlite3_open(databasePath! as String, &database) != SQLITE_OK
{
println("Databse is not open")
return 0
}
else
{
query.stringByReplacingOccurrencesOfString("null", withString: "")
var cStatement:COpaquePointer = nil
var executeSql = query as NSString
var lastId : Int?
var sqlStatement = executeSql.cStringUsingEncoding(NSUTF8StringEncoding)
sqlite3_prepare_v2(database, sqlStatement, -1, &cStatement, nil)
var execute = sqlite3_step(cStatement)
println("\(execute)")
if execute == SQLITE_DONE
{
lastId = Int(sqlite3_last_insert_rowid(database))
}
else
{
println("Error in Run Statement :- \(sqlite3_errmsg16(database))")
}
sqlite3_finalize(cStatement)
return lastId!
}
}
func ViewAllData(query: NSString, error: NSError) -> NSArray
{
var cStatement = COpaquePointer()
var result : AnyObject = NSNull()
var thisArray : NSMutableArray = NSMutableArray(capacity: 4)
cStatement = prepare(query)
if cStatement != nil
{
while sqlite3_step(cStatement) == SQLITE_ROW
{
result = NSNull()
var thisDict : NSMutableDictionary = NSMutableDictionary(capacity: 4)
for var i = 0 ; i < Int(sqlite3_column_count(cStatement)) ; i++
{
if sqlite3_column_type(cStatement, Int32(i)) == 0
{
continue
}
if sqlite3_column_decltype(cStatement, Int32(i)) != nil && strcasecmp(sqlite3_column_decltype(cStatement, Int32(i)), "Boolean") == 0
{
var temp = sqlite3_column_int(cStatement, Int32(i))
if temp == 0
{
result = NSNumber(bool : false)
}
else
{
result = NSNumber(bool : true)
}
}
else if sqlite3_column_type(cStatement,Int32(i)) == SQLITE_INTEGER
{
var temp = sqlite3_column_int(cStatement,Int32(i))
result = NSNumber(int : temp)
}
else if sqlite3_column_type(cStatement,Int32(i)) == SQLITE_FLOAT
{
var temp = sqlite3_column_double(cStatement,Int32(i))
result = NSNumber(double: temp)
}
else
{
if sqlite3_column_text(cStatement, Int32(i)) != nil
{
var temp = sqlite3_column_text(cStatement,Int32(i))
result = String.fromCString(UnsafePointer<CChar>(temp))!
var keyString = sqlite3_column_name(cStatement,Int32(i))
thisDict.setObject(result, forKey: String.fromCString(UnsafePointer<CChar>(keyString))!)
}
result = NSNull()
}
if result as! NSObject != NSNull()
{
var keyString = sqlite3_column_name(cStatement,Int32(i))
thisDict.setObject(result, forKey: String.fromCString(UnsafePointer<CChar>(keyString))!)
}
}
thisArray.addObject(NSMutableDictionary(dictionary: thisDict))
}
sqlite3_finalize(cStatement)
}
return thisArray
}
func prepare(sql : NSString) -> COpaquePointer
{
var cStatement:COpaquePointer = nil
sqlite3_open(databasePath! as String, &database)
var utfSql = sql.UTF8String
if sqlite3_prepare(database, utfSql, -1, &cStatement, nil) == 0
{
sqlite3_close(database)
return cStatement
}
else
{
sqlite3_close(database)
return nil
}
}
}
Sometimes, a Swift version of the "SQLite in 5 minutes or less" approach shown on sqlite.org is sufficient.
The "5 minutes or less" approach uses sqlite3_exec() which is a convenience wrapper for sqlite3_prepare(), sqlite3_step(), sqlite3_column(), and sqlite3_finalize().
Swift 2.2 can directly support the sqlite3_exec() callback function pointer as either a global, non-instance procedure func or a non-capturing literal closure {}.
Readable typealias
typealias sqlite3 = COpaquePointer
typealias CCharHandle = UnsafeMutablePointer<UnsafeMutablePointer<CChar>>
typealias CCharPointer = UnsafeMutablePointer<CChar>
typealias CVoidPointer = UnsafeMutablePointer<Void>
Callback Approach
func callback(
resultVoidPointer: CVoidPointer, // void *NotUsed
columnCount: CInt, // int argc
values: CCharHandle, // char **argv
columns: CCharHandle // char **azColName
) -> CInt {
for i in 0 ..< Int(columnCount) {
guard let value = String.fromCString(values[i])
else { continue }
guard let column = String.fromCString(columns[i])
else { continue }
print("\(column) = \(value)")
}
return 0 // status ok
}
func sqlQueryCallbackBasic(argc: Int, argv: [String]) -> Int {
var db: sqlite3 = nil
var zErrMsg:CCharPointer = nil
var rc: Int32 = 0 // result code
if argc != 3 {
print(String(format: "ERROR: Usage: %s DATABASE SQL-STATEMENT", argv[0]))
return 1
}
rc = sqlite3_open(argv[1], &db)
if rc != 0 {
print("ERROR: sqlite3_open " + String.fromCString(sqlite3_errmsg(db))! ?? "" )
sqlite3_close(db)
return 1
}
rc = sqlite3_exec(db, argv[2], callback, nil, &zErrMsg)
if rc != SQLITE_OK {
print("ERROR: sqlite3_exec " + String.fromCString(zErrMsg)! ?? "")
sqlite3_free(zErrMsg)
}
sqlite3_close(db)
return 0
}
Closure Approach
func sqlQueryClosureBasic(argc argc: Int, argv: [String]) -> Int {
var db: sqlite3 = nil
var zErrMsg:CCharPointer = nil
var rc: Int32 = 0
if argc != 3 {
print(String(format: "ERROR: Usage: %s DATABASE SQL-STATEMENT", argv[0]))
return 1
}
rc = sqlite3_open(argv[1], &db)
if rc != 0 {
print("ERROR: sqlite3_open " + String.fromCString(sqlite3_errmsg(db))! ?? "" )
sqlite3_close(db)
return 1
}
rc = sqlite3_exec(
db, // database
argv[2], // statement
{ // callback: non-capturing closure
resultVoidPointer, columnCount, values, columns in
for i in 0 ..< Int(columnCount) {
guard let value = String.fromCString(values[i])
else { continue }
guard let column = String.fromCString(columns[i])
else { continue }
print("\(column) = \(value)")
}
return 0
},
nil,
&zErrMsg
)
if rc != SQLITE_OK {
let errorMsg = String.fromCString(zErrMsg)! ?? ""
print("ERROR: sqlite3_exec \(errorMsg)")
sqlite3_free(zErrMsg)
}
sqlite3_close(db)
return 0
}
To prepare an Xcode project to call a C library such as SQLite, one needs to (1) add a Bridging-Header.h file reference C headers like #import "sqlite3.h", (2) add Bridging-Header.h to Objective-C Bridging Header in project settings, and (3) add libsqlite3.tbd to Link Binary With Library target settings.
The sqlite.org's "SQLite in 5 minutes or less" example is implemented in a Swift Xcode7 project here.
You can easlity configure SQLite with swift using single ton class as well.
Refer
https://github.com/hasyapanchasara/SQLite_SingleManagerClass
Method to create database
func methodToCreateDatabase() -> NSURL?{}
Method to insert, update and delete data
func methodToInsertUpdateDeleteData(strQuery : String) -> Bool{}
Method to select data
func methodToSelectData(strQuery : String) -> NSMutableArray{}

Make operation with aggregate function result

I confess that I do not get along very well with the Deferred object. I'm making a query to the database on several "Stores" and as a result I want to do a series of operations. This troubles me because the results are returned asynchronously and I have no way to perform the corresponding operation on the "store" you should. In short, the problem is that this piece of code always executes the same function on the same "Store"
for (var i = 0; i < schema['stores'].length; i++) {
storeName = schema['stores'][i].name;
var objeto = db.executeSql('SELECT MAX(date_upd) FROM ' + '"' + storeName + '"').done(
function(result, a){
//saveDataSynce(db, storeName, result);
console.log(result);
}
);
}
Whenever there is a loop on async operation, be very careful about function scope. In your example code, storeName inside the function will always be the last executed value. Use function scope as follow:
var getMax = function(storeName) {
db.executeSql('SELECT MAX(date_upd) FROM ' + '"' + storeName + '"').done(
function(result){
//saveDataSynce(db, storeName, result);
console.log(storeName, result);
}
);
}
for (var i = 0; i < schema['stores'].length; i++) {
getMax(schema['stores'][i].name);
}
However, preferred coding pattern for YDN-DB is NoSQL style as follow:
var getMax = function(storeName) {
var indexName = 'date_upd';
var key_range = null; // whole store
var limit = 1;
var offset = 0;
var reverse = true;
db.values(storeName, indexName, key_range, limit, offset, reverse).done(
function(results) {
var max_key = results[0]; // may be undefined. OK.
//saveDataSynce(db, storeName, max_key);
console.log(storeName, max_key);
}
);
}
Note that keys (primary or index) are always sorted by ascending order. Max key is the first key in reverse order.

Resources