I'm a very novice coder, and I'm working on my first CoreML project, where I use an Image Classification model. I used some tutorials online, and I converted the uploaded image to a CV pixel buffer.
This is my code:
let model = classifier()
private func performImageClassification(){
let currentImage = uploadedImage //image that was uploaded
let resizedImage = currentImage.resizeTo(size: CGSize(width: 255, height:255))
guard let pixelBuffer = resizedImage?.toBuffer() else { return }
let output = try? model.prediction(input: pixelBuffer)
if let output = output {
self.classificationLabel = output.classLabel
}
}
On the line let output = try? model.prediction(input: pixelBuffer), I get the following error: "Cannot convert value of type 'CVPixelBuffer' (aka 'CVBuffer') to expected argument type."
Please help!
Related
I am trying to get a simple example of face detection working with ML Kit on iOS. Here is excerpts of the Objective C code:
FIRVisionFaceDetectorOptions *faceDetectorOptions;
FIRVision *vision;
FIRVisionFaceDetector *faceDetector;
faceDetectorOptions = [[FIRVisionFaceDetectorOptions alloc] init];
faceDetectorOptions.performanceMode = FIRVisionFaceDetectorPerformanceModeAccurate;
faceDetectorOptions.landmarkMode = FIRVisionFaceDetectorLandmarkModeAll;
faceDetectorOptions.contourMode = FIRVisionFaceDetectorContourModeNone;
faceDetectorOptions.classificationMode = FIRVisionFaceDetectorClassificationModeAll;
faceDetectorOptions.minFaceSize = 0.1; // TODO: finalize this option value
vision = [FIRVision vision];
faceDetector = [vision faceDetectorWithOptions:faceDetectorOptions];
UIImage *staticImg = [UIImage imageNamed:#"sample.jpg"];
FIRVisionImage *visionImage = [[FIRVisionImage alloc] initWithImage:staticImg];
NSError* error = Nil;
NSArray<FIRVisionFace *> * faces = [faceDetector resultsInImage:visionImage error:&error];
NSLog(#"Synchronous result. error = %#, face count = %lu", error, faces.count);
The sample.jpg file is the following image downloaded and added as a resource to my Xcode project:
http://chwb.org/wp-content/uploads/2014/01/Theo_Janssen-Face1.jpg
The resultsInImage returns no error, but no faces either. It logs:
Synchronous result. error = (null), face count = 0
Am I doing something wrong?
I figured it out. The problem was I need to set the image metadata with orientation like this:
FIRVisionImageMetadata *imageMetadata = [FIRVisionImageMetadata new];
imageMetadata.orientation = [FcFaceDetector visionImageOrientationFromImageOrientation:uiImage.imageOrientation];
visionImage.metadata = imageMetadata;
+ (FIRVisionDetectorImageOrientation) visionImageOrientationFromImageOrientation:(UIImageOrientation)imageOrientation {
switch (imageOrientation) {
case UIImageOrientationUp:
return FIRVisionDetectorImageOrientationTopLeft;
case UIImageOrientationDown:
return FIRVisionDetectorImageOrientationBottomRight;
case UIImageOrientationLeft:
return FIRVisionDetectorImageOrientationLeftBottom;
case UIImageOrientationRight:
return FIRVisionDetectorImageOrientationRightTop;
case UIImageOrientationUpMirrored:
return FIRVisionDetectorImageOrientationTopRight;
case UIImageOrientationDownMirrored:
return FIRVisionDetectorImageOrientationBottomLeft;
case UIImageOrientationLeftMirrored:
return FIRVisionDetectorImageOrientationLeftTop;
case UIImageOrientationRightMirrored:
return FIRVisionDetectorImageOrientationRightBottom;
}
}
The docs seem to be unclear about it, because it seems to suggest to not set it:
https://firebase.google.com/docs/ml-kit/ios/detect-faces#2-run-the-face-detector
I recently saw code for reading entire contents of an InputStream into a String in Kotlin, such as:
// input is of type InputStream
val baos = ByteArrayOutputStream()
input.use { it.copyTo(baos) }
val inputAsString = baos.toString()
And also:
val reader = BufferedReader(InputStreamReader(input))
try {
val results = StringBuilder()
while (true) {
val line = reader.readLine()
if (line == null) break
results.append(line)
}
val inputAsString = results.toString()
} finally {
reader.close()
}
And even this that looks smoother since it auto-closes the InputStream:
val inputString = BufferedReader(InputStreamReader(input)).useLines { lines ->
val results = StringBuilder()
lines.forEach { results.append(it) }
results.toString()
}
Or slight variation on that one:
val results = StringBuilder()
BufferedReader(InputStreamReader(input)).forEachLine { results.append(it) }
val resultsAsString = results.toString()
Then this functional fold thingy:
val inputString = input.bufferedReader().useLines { lines ->
lines.fold(StringBuilder()) { buff, line -> buff.append(line) }.toString()
}
Or a bad variation which doesn't close the InputStream:
val inputString = BufferedReader(InputStreamReader(input))
.lineSequence()
.fold(StringBuilder()) { buff, line -> buff.append(line) }
.toString()
But they are all clunky and I keep finding newer and different versions of the same... and some of them never even close the InputStream. What is a non-clunky (idiomatic) way to read the InputStream?
Note: this question is intentionally written and answered by the author (Self-Answered Questions), so that the idiomatic answers to commonly asked Kotlin topics are present in SO.
Kotlin has a specific extension just for this purpose.
The simplest:
val inputAsString = input.bufferedReader().use { it.readText() } // defaults to UTF-8
And in this example, you could decide between bufferedReader() or just reader(). The call to the function Closeable.use() will automatically close the input at the end of the lambda's execution.
Further reading:
If you do this type of thing a lot, you could write this as an extension function:
fun InputStream.readTextAndClose(charset: Charset = Charsets.UTF_8): String {
return this.bufferedReader(charset).use { it.readText() }
}
Which you could then call easily as:
val inputAsString = input.readTextAndClose() // defaults to UTF-8
On a side note, all Kotlin extension functions that require knowing the charset already default to UTF-8, so if you require a different encoding you need to adjust the code above in calls to include encoding for reader(charset) or bufferedReader(charset).
Warning: You might see examples that are shorter:
val inputAsString = input.reader().readText()
But these do not close the stream. Make sure you check the API documentation for all of the IO functions you use to be sure which ones close and which do not. Usually, if they include the word use (such as useLines() or use()) they close the stream after. An exception is that File.readText() differs from Reader.readText() in that the former does not leave anything open and the latter does indeed require an explicit close.
See also: Kotlin IO related extension functions
【Method 1 | Manually Close Stream】
private fun getFileText(uri: Uri):String {
val inputStream = contentResolver.openInputStream(uri)!!
val bytes = inputStream.readBytes() //see below
val text = String(bytes, StandardCharsets.UTF_8) //specify charset
inputStream.close()
return text
}
inputStream.readBytes() requires manually close the stream: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/read-bytes.html
【Method 2 | Automatically Close Stream】
private fun getFileText(uri: Uri): String {
return contentResolver.openInputStream(uri)!!.bufferedReader().use {it.readText() }
}
You can specify the charset inside bufferedReader(), default is UTF-8:
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/buffered-reader.html
bufferedReader() is an upgrade version of reader(), it is more versatile:
How exactly does bufferedReader() work in Kotlin?
use() can automatically close the stream when the block is done:
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/use.html
An example that reads contents of an InputStream to a String
import java.io.File
import java.io.InputStream
import java.nio.charset.Charset
fun main(args: Array<String>) {
val file = File("input"+File.separator+"contents.txt")
var ins:InputStream = file.inputStream()
var content = ins.readBytes().toString(Charset.defaultCharset())
println(content)
}
For Reference - Kotlin Read File
Quick solution works well when converting InputStream to string.
val convertedInputStream = String(inputStream.readAllBytes(), StandardCharsets.UTF_8)
I have strings and determine the ranges of indexes. I will need later for instance .last .count for these ranges. How should I initialise the range for string to be able to get functionality .last .count for these ranges (that is obvious in swift2 but not in swift3) ?
For example, I am often using the .count for range of string in my code in swift2, like this
var str = "Hello, playground"
let myRange = str.rangeOfString("Hello")
let myCountOfRange = myRange.count
Now it is not possible to do this in swift3
var str = "Hello, playground"
let myRange = str.range(of: "Hello")
let myCountOfRange = myRange.count // type index does not conform to protocol strideable
In Swift3, to find the size of a range you can do:
var str = "Hello, playground"
let myRange = str.range(of: "Hello")
let myCountOfRange = str[myRange!].characters.count
I don't know if this is the best way, but it works.
Alternatively:
let myCountOfRange = str.distance(from: myRange!.lowerBound, to: myRange!.upperBound)
Both require access to the original collection (ie. string), and that apparently is a limitation of Swift 3. The new model for collections and indices is discussed here.
If you want to store the ranges in an array and call .count and .last on them, you can convert the Range<Index> to a CountableRange<Int> while you still have access to the collection:
var str = "Hello, playground"
let myRange = str.range(of: "Hello")!
let lb = str.distance(from: str.startIndex, to: myRange.lowerBound) as Int
let ub = str.distance(from: str.startIndex, to: myRange.upperBound) as Int
let newRange = lb..<ub
newRange.count // 5
newRange.last // 4
I am trying to pass contextInfo of typeUnsafeMutablePointer<Void> to UISaveVideoAtPathToSavedPhotosAlbum and use it in the callback function. For some reason I am unable to access contextInfo as a string using UnsafePointer<String>(x).memory when I am in the callback function.
I am pretty sure it is something simple I am missing but have spent way to many hours trying to figure this out.
Below is some code that I have tried.
The following code works.
var testStr:String = "hello"
takesAMutableVoidPointer(&testStr)
func takesAMutableVoidPointer(x: UnsafeMutablePointer<Void>){
var pStr:String = UnsafePointer<String>(x).memory
println("x = \(x)")
println("pStr = \(pStr)")
}
However the following code does not work.
var testStr:String = "hello"
if UIVideoAtPathIsCompatibleWithSavedPhotosAlbum(filePath){ //the filePath is compatible
println("Compatible")
//UISaveVideoAtPathToSavedPhotosAlbum(filePath, self, nil, nil)
UISaveVideoAtPathToSavedPhotosAlbum(filePath, self, "video:didFinishSavingWithError:contextInfo:", &testStr)
}
else{
println("Not Compatible")
}
func video(video: NSString, didFinishSavingWithError error:NSError, contextInfo:UnsafeMutablePointer<Void>){
var pStr:String = UnsafePointer<String>(contextInfo).memory
println("contextInfo = \(contextInfo)")
println("pStr = \(pStr)")
}
Once I get to the following line:
var pStr:String = UnsafePointer<String>(contextInfo).memory
I keep getting the following error:
Thread 1: EXC_BAD_ACCESS(code=1, address=0x0)
Any help with this would be greatly appreciated.
Thanks.
Update
Rintaro commented that testStr needs to be top level but the following code works.
import UIKit
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view, typically from a nib.
var testStr:String = "hello"
takesAMutableVoidPointer(&testStr)
}
override func didReceiveMemoryWarning() {
super.didReceiveMemoryWarning()
// Dispose of any resources that can be recreated.
}
func takesAMutableVoidPointer(x: UnsafeMutablePointer<Void>){
var answer = UnsafePointer<String>(x).memory
println("x = \(x)")
println("answer = \(answer)")
}
}
I am trying not to use global variables unless I have to. I may have to but since I am able to execute the above code, it seems as though I do not need to use a global variable.
As discussed in OP comments, testStr has already been freed.
Is there any way to force the retaining of a variable that has been created in a function? Then release it later?
It's not impossible, but I don't know this is the best way to do that.
Anyway, try this with Playground or OS X "Command Line Tool" template:
import Foundation
func foo() {
var str:NSString = "Hello World"
let ptr = UnsafePointer<Void>(Unmanaged<NSString>.passRetained(str).toOpaque())
bar(ptr)
}
func bar(v:UnsafePointer<Void>) {
let at = dispatch_time(
DISPATCH_TIME_NOW,
Int64(2.0 * Double(NSEC_PER_SEC))
)
dispatch_after(at, dispatch_get_main_queue()) {
baz(v)
}
}
func baz(v:UnsafePointer<Void>) {
println("notified")
let str = Unmanaged<NSString>.fromOpaque(COpaquePointer(v)).takeRetainedValue()
println("info: \(str)")
}
foo()
println("started")
dispatch_main()
Unmanaged<NSString>.passRetained(str) increments the retain count.
Unmanaged<NSString>.fromOpaque(...).takeRetainedValue() decrements it, and extract the object.
I think, using pure Swift String is impossible. because String is struct and is allocated in stack memory. Maybe the buffer of it is allocated in heap, but we cannot access it directly.
I have a file to put in a multidimensional array. I have to put to [0] a date (long) and one of the dimensions must be incremented depending on the value of the second token.
Here's the code :
BufferedReader bufStatsFile = new BufferedReader(new FileReader(statsFile));
String line = null;
List<Long[]> stats = new ArrayList<Long[]>();
stats.add(new Long[11]);
int i = 0; // will be in a loop later
while((line = bufStatsFile.readLine()) != null) {
StringTokenizer st = new StringTokenizer(line,";");
while(st.hasMoreTokens()) {
stats.get(i)[0] = Long.parseLong(st.nextToken());
stats.get(i)[Integer.parseInt(st.nextToken())]++; // Here is the problematic line.
}
}
bufStatsFile.close();
But the incrementation doesn't work. Maybe it is because of my array which is probably not correct, but I didn't found another proper way to do that.
Ok. I have found and it was, of course, stupid.
The problem was in my array declaration. I did it like that :
List<Long[]> stats = new ArrayList<Long[]>();
stats.add(new Long[11]);
And then, I tried to increment an Object and not a long number.
So now, I just do it like this :
List<long[]> stats = new ArrayList<>();
stats.add(new long[11]);
And it's perfectly working.
Check that the elements in your file are numbers from 0 to 10. Why are you having a List if you are only manipulating the row 0?
Which exception are your code throwing away?