Custom Machine learning code to import in the android studio - firebase

I am creating a project of fake news detection in android. I have trained the model and converted into the tensorflow lite. I tried to import it into the android studio in asset folder but it showed the message "This is not valid tensorflow lite model file".
enter image description here
can anyone help me to deploy it into android studio and to generate the input/output.
NOTE: I have tried export to firebase custom ML, I have run this code and didn't get any error
CustomModelDownloadConditions conditions = new CustomModelDownloadConditions.Builder().requireWifi().build();
FirebaseModelDownloader.getInstance().getModel("NewsCheckModel",
DownloadType.LOCAL_MODEL_UPDATE_IN_BACKGROUND, conditions)
.addOnSuccessListener(new OnSuccessListener<CustomModel>() {
#Override
public void onSuccess(CustomModel customModel) {
File modelFile = customModel.getFile();
if (modelFile != null) {
Interpreter interpreter = new Interpreter(modelFile);
}
}
});
I don't know the further steps.
.
.
.
if anyone can help in any way using asset folder or firebase way. I would be very thankful to you

Could you make sure that no compression option is enabled as follows while experimenting on the asset dir case?
android {
// ...
aaptOptions {
noCompress "tflite" // Your model's file extension: "tflite", "lite", etc.
}
}
See more details at this guide page.

There's not much information on your post. It would be useful to know if:
Are you able to deploy it on android?
Does the model initialize?
If yes, Can you perform inference?
If no, what is the exception you get?
In the past, I've had the same problem. So here are some basic steps:
Add Firebase to your project
2a. Make sure you are using the correct dependencies :
They have been updated recently so better follow this guide: here
Basically:
dependencies {
// Import the BoM for the Firebase platform
implementation platform('com.google.firebase:firebase-bom:28.0.1')
// Declare the dependency for the Firebase ML model downloader library
// When using the BoM, you don't specify versions in Firebase library dependencies
implementation 'com.google.firebase:firebase-ml-modeldownloader'
// Also declare the dependency for the TensorFlow Lite library and specify its version
implementation 'org.tensorflow:tensorflow-lite:2.3.0'
}
2b. Add to Manifest:
<uses-permission android:name="android.permission.INTERNET" />
3a. Double check the way you convert your model. Luckily you can convert and deploy to firebase easily:
3b. Convert the model to TensorFlow Lite and upload it to Cloud Storage
source = ml.TFLiteGCSModelSource.from_saved_model('./model_directory')
3c. Create the model object
tflite_format = ml.TFLiteFormat(model_source=source)
model = ml.Model(
display_name="example_model", # This is the name you use from your app to load the model.
tags=["examples"], # Optional tags for easier management.
model_format=tflite_format)
3d. Add the model to your Firebase project and publish it
new_model = ml.create_model(model)
ml.publish_model(new_model.model_id)
3a bis: If your model is already in .tflite:
source = ml.TFLiteGCSModelSource.from_tflite_model_file('example.tflite')
The guide is here
Regarding the inputs/outputs, I can't say much as it depends on your model. Also, be careful with the data types of input and output. Be aware of the type that your model requires (INT32, UINT8, FLOAT32....)
Another important thing is to have some metadata embedded in your model: here
Android Studio uses that metadata to display information when you open a .tflite file.
IMO is very likely that your issue is during your model conversion or lack of metadata.

Related

Keras Deploy for Tensorflow.js Usage

I need to be able to deploy a keras model for Tensorflow.js prediction, but the Firebase docs only seem to support a TFLite object, which tf.js cannot accept. Tf.js appears to accept JSON files for loading (loadGraphModel() / loadLayersModel() ), but not a keras SavedModel (.pb + /assets + /variables).
How can I attain this goal?
Note for the Tensorflow.js portion: There are a lot of pointers to the tfjs_converter, but the closest API function offered to what I'm looking for is the loadFrozenModel() function, which requires both a .pb and a weights_manifest.json. It seem to me like I'd have to programmatically assemble this before before sending it up to GCloud as a keras SavedModel doesn't contain both (mine contains .pb + /assets + /variables).
This seems tedious for a straightforward deployment feature, and I'd imagine my question only hits upon common usage of each tool.
What I'm looking for is a simple pathway from Keras => Firebase/GCloud => Tensorflow.js.
So I understand your confusion but you have half part ready.
So your keras model has the following files and folders if I understand correctly:
saved_model.pb
/assests
/variables
This is enough to convert the keras model to tensorflow.js model.
Use the converter script in the following manner. Make sure you have the latest version of tfjs. If you do not have the latest version, try creating a virtual environment and install latest tfjs otherwise it will disrupt your tensorflow version.
import tensorflowjs as tfjs
import tensorflow as tf
model=tf.keras.models.load_model('path/to/keras/model')
tfjs.converters.save_keras_model(model, 'path/where/you/will/like/to/have/js/model/converted')
Once you have converted the model you will receive following files for js model.
model.json
something.bin
You will have to host those files using a webserver and just make it available for loadLayersModel API something like this:
const model = await tf.loadLayersModel(
'location/of/model.json');
That is it and you have converted the model from Keras to Tensorflowjs and uploaded as well in js.
I hope my answer helps you.

SQLite no longer seems to work on xamarin android

We have a Xamarin.Forms project that needed to use the sqlite local db to store date. EF Core's sqlite library was used to set this up and by different developers from different PCs (vs 2019). Initially, it was used with the Database.EnsureCreated() function and later with ef core's migrations. All went smooth for more than a month.
Last week all of a sudden the android app wouldn't start on any one's PC due to some error with sqlite. It showed the following error:
Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
I spent a while trying all kinds of fixes and rollbacks thinking it was an issue with the code. This included the following:
Deleted obj and bin folders, cleaned and rebuilt for all below steps.
Downgraded the version of ef to 2.2.3 (the version we started with)
Rolled back to git commits up to a week back
Upgraded the versions of dependencies of ef core
Removed the past few migrations
Downgraded xamarin forms to 3.6.x
After trying the above and several other fixes, finally upgrading the versions of java and android SDK worked last Friday (on all PCs). Once this fix worked everything went smooth that day. On Tuesday once again the issue was back (no library updates or code changes). A deeper look at EF Cores logging shows that it crashes the moment it attempts to connect to a db.
The issue can be replicated on the android devices but not the emulators. I am not sure if there is some new permission in android we need to request for.
I finally created a new xamarin forms project with sqlite setup. I used the pcl library and ef core. I still found the same error.
Here is the git hub that replicates the issue https://github.com/neville-nazerane/xamarin-site
Update
Just something i noticed. eariler my database file was named "main.db". Now no matter what i change this file name to or no matter what variables i change. it always shows database name as "main" in logs. Not sure if changing the db name would fix the issue. However, never found a way to change this db name. I tried different connection strings, it just said "database" and "db" were unknown keys
Update
Steps to replicate:
using (var db = new AppDbContext())
{
db.Add(new Person {
Age = 55,
Name = "Neville"
});
db.SaveChanges();
Person[] alldata = db.People.ToArray();
}
The definitions of Person and AppDbContext are quite obvious. So, with the spirit of not making the question too long, I am not posting it here. However, if required I can post them too.
This is a bug with the Xamarin.Forms and Mono.
It was detected since a couple of months ago, it was fixed but then there was some regression (with VS 2019 v16.1).
Even with the latest release (v16.1.2) the bug still happens, so we need to wait for a fix.
Sources:
https://github.com/mono/mono/issues/14170
https://github.com/xamarin/xamarin-android/issues/3112
https://github.com/xamarin/xamarin-android/issues/2920
Due to slight differences of the particular file systems on the native side, I would suggest creating an interface to handle the actual database file handling on the native application level.
So here is how I implemented SQLite using the nuget package SQLite.Net-PCL:
In the shared project, create a new interface, for instance FileHandler.cs
public interface IFileHandler
{
SQLite.SQLiteConnection GetDbConnection();
}
You may want to extend that with more methods to save or retrieve various files, but for now we will just have the GetDbConnection Method to retrieve a working SQLite Connection.
Now in your Android implementation, we add the native implementation to that interface:
Create a new class called FileHandler.cs in your android project:
[assembly: Dependency(typeof(FileHandler))]
namespace YourProjectName.Droid
{
public class FileHandler : IFileHandler
{
public SQLite.SQLiteConnection GetDbConnection()
{
string sqliteFilename = "YourDbName.db3";
string path = Path.Combine(GetPersonalPath(), sqliteFilename);
SQLiteConnectionString connStr = new SQLiteConnectionString(path, true);
SQLiteConnectionWithLock connection = new SQLiteConnectionWithLock(connStr, SQLiteOpenFlags.Create | SQLiteOpenFlags.ReadWrite | SQLiteOpenFlags.NoMutex);
return connection;
}
private string GetPersonalPath()
{
return Environment.GetFolderPath(Environment.SpecialFolder.Personal);
}
}
}
Now back again in your shared code you can access that connection with the following code:
using (SQLiteConnection connection = DependencyService.Get<IFileHandler>().GetDbConnection())
{
// Do whatever you want to do with the database connection
}
Alright mate, I don't understand what issue you are facing. It might be an issue with your machine, I'd suggest using another computer/laptop.
I took the exact code that you shared on the Github. I was able to build it on my Mac computer in VS 2019 and installed the application in debug mode on my phone. I was able to add a date successfully, as you can see in the picture, and I placed an Exception Catchpoint and faced no exceptions.
I then proceeded to add another entry with the same details and it errored out with the message that you can see here
I would also suggest using Xamarin Profiler or any other Android logger to see the Stack Trace that you aren't able to see in your application output. It will give you details of the error, that you can share here for us to understand better.

onnx 1.2 models (Custom Vision) on HoloLens

This is a more specific continuation of my previous Post: Custom Vision on HoloLens
I'm still using the Unity Project from this blogpost: https://mtaulty.com/2018/03/29/third-experiment-with-image-classification-on-windows-ml-from-uwp-on-hololens-in-unity/
I had issues that my own exported models don't work with the the code at some point. Now it is possible to export onnx models of version 1.2, but the the old code seems to not be compatible with the new version.
in the line var evalOutput = await this.learningModel.EvaluateAsync(this.inputData); in the MainScript it throws The binding is incomplete or does not match the input/output description. (Exception from HRESULT: 0x88900002)
Does someone know what I need to change so it works with the new version on HoloLens? Thanks in advance!
You can find a similar question here: Windows ML's OS requirement
In summary, you are right about needing to update the PC and the Hololens, but the build number you need is 17763 to be on the production version of RS5.
You could also be hitting this issue: Cannot load model using WinML
where the binding isn't quite setup properly.
If you're still having issues, please post the SDK and OS version you're on, as well as the ONNX model version.

How to access public download folder in every platform in Xamarin.Forms

I need to create a backup service so I intend to save the SQLite database file on each platform. The saved file should be available after the uninstall of the app.
I intend to use the Downloads folder (which should be available on every platform).
I have created an interface and use the following code per platform:
Interface:
public interface IBackupService
{
string GetDownloadPath();
}
Android:
public string GetDownloadPath()
{
return Android.OS.Environment.DirectoryDownloads;
}
UWP:
public string GetDownloadPath()
{
return Windows.Storage.KnownFolders.???????;
}
What should I do about that? Is there a public library that I could use?
There does not seem to be a general downloads folder as per this documentation on KnownFolders. So your assumption on the Downloads folder being on every platform doesn't seem to be correct.
If we dive in a bit further we get to the DocumentsLibrary which seems the obvious choice for this kind of purpose, but it is not. Microsoft says:
The Documents library is not intended for general use. For more info,
see App capability declarations. Also, see the following blog post.
Dealing with Documents: How (not) to use the documentsLibrary capability in Windows Store apps
The paragraph after that seems to describe what we have to do then;
If your app has to create and update files that only your app uses,
consider using the app's local folder. Get the app's local folder from
the Windows.Storage.ApplicationData.Current.LocalFolder property.
So, as I can extract from your question you only want to use storage for your app, so the Windows.Storage.ApplicationData.Current.LocalFolder seems to be the right choice according to Microsoft.

MvvmCross - how do I access SQLite in a windows store background task?

I have a store app that uses the mvvmcross sqlite plugin (community edition). This app has a periodic background task that accesses the database to get data to be shown in a live tile. I can't see how I can get access to this database from the background task. I would like to use the mvvmcross sqlite plugin in the background task, but I don't see how to initialize the mvvmcross environment properly.
If you want to initialize the full MvvmCross framework including all of your app, then you'll need to run your Setup class.
In WinRT, this could be as simple as calling:
var setup = new Setup(null /*rootFrame*/);
setup.Initialize();
although it may require you to do a little work to:
Make sure your presenter does not use the null rootFrame
Provide some other means to create a UI thread dispatcher - currently MvxStoreViewDispatcher relies on .Dispatcher access - see https://github.com/MvvmCross/MvvmCross/blob/v3.1/Cirrious/Cirrious.MvvmCross.WindowsStore/Views/MvxStoreViewDispatcher.cs - to do this, you could override InitializeViewDispatcher with something like:
protected override void InitializeViewDispatcher()
{
if (_rootFrame != null)
{
base.InitializeViewDispatcher(); return;
}
var dispatcher = new NonMainThreadDispatcher();
Mvx.RegisterSingleton<IMvxMainThreadDispatcher>(dispatcher);
}
public class NonMainThreadDispatcher : MvxMainThreadDispatcher
{
public bool RequestMainThreadAction(Action action)
{
action();
}
}
If you want to initialize less functionality than the entire framework (e.g. for memory reasons) then you can also consider creating special Setup and App classes just for your background task.
Aside> This is similar to questions like these in Android - Using MvvmCross from content providers and activities and MvvmCross initialization
I was able to solve the problem in a straightforward way. Since the background task only needed the SQLite data service from the PCL core project, I did the following:
Included a reference to the Core project.
Added the nuget packages for MvvmCross and the SQLite community plugin.
Deleted all of the files and folders added when doing the mvvmcross install: Bootstrap/, Todo-Mvvmcross/, Views/, DebugTrace.cs, and Setup.cs.
There is a current limitation in the nuget installer that requires some additional edits to the project file to handle multiple store platforms (x86, ARM, and x64), see 'Cirrius.Mvvmcross.Community.Plugins.SQLite.WindowsStore needs platform-specific dlls for X86 and ARM' on Stack Overflow for details. Make sure you put the Choose statement after the default SQLite.WindowsStore reference and you need to leave the default reference in the project file. You will also need to adjust the HintPath based on the location/names of your references.
Initialized the SQLite data service by explicitly calling the factory and creating a new instance of the data service:
var factory = new MvxStoreSQLiteConnectionFactory();
IMyDataService repository = new MyDataService(factory);
I then have access to the data service with no other overhead associated with mvvmcross.

Resources