I have a mobile service sync table that is giving me absolutely HORRENDOUS performance.
The table is declared as:
IMobileServiceSyncTable<Myclass> myclassTable;
this.client = new MobileServiceClient("my url here");
var store = new MobileServiceSQLiteStore(“localdb.db”);
store.DefineTable<Myclass>();
this.client.SyncContext.InitializeAsync(store);
this.myclassTable = client.GetSyncTable<Myclass>();
Than later in a button handler I’m calling into:
this.myclassTable.ToCollectionAsync();
The problem is, the performance is horrific. It takes at best minutes and most times just sits there indefinitely.
Is there anything in the above that I’ve done that would explain why performance is so absolutely terrible?
this.myclassTable.ToCollectionAsync();
For IMobileServiceSyncTable table, the above method would execute the SELECT * FROM [Myclass] sql statement against your local sqlite db.
The problem is, the performance is horrific. It takes at best minutes and most times just sits there indefinitely.
AFAIK, when working with offline sync, we may invoke the pull operation for retrieving a subset of the server data, then insert the retrieved data into the local store table. For await this.myclassTable.PullAsync(), it would send request and retrieve the server data with the MaxPageSize in 50, and the client SDK would send another request to confirm whether there has more data and pull them automatically.
In summary, I would recommend you checking with your code to locate the specific code which causes this poor performance. Also, you could leverage adding diagnostic logging, capturing the network traces via Fiddler to troubleshoot with this issue.
Related
I am currently trying to upload some data to a Kusto cluster using the C# API. What I am basically trying to perform looks currently like this:
using (IKustoIngestClient client = KustoIngestFactory.CreateStreamingIngestClient(kustoConnection))
{
var ingestionProperties = new KustoIngestionProperties()
{
DatabaseName = _kustoSettings.DatabaseName,
TableName = tableName,
Format = DataSourceFormat.json
};
// ms is a Memory Stream
client.IngestFromStreamAsync(ms, ingestionProperties);
}
The problem with the current code or tactics that I see in multiple samples is that they use Thread.Sleep and then check for the result. This is something that in my opinion is not really useful since the application has to know if the operation succeeded or not.
So is there a way for the above code to validate and properly throw an exception in the case of any failure. Cause now despite failing there are no exceptions being thrown.
Here is some recommended reading:
Kusto ingest client library - Best practices
Tracking Ingestion Status (KustoQueuedIngestClient)
The bottom line:
Async ingestion has some major advantages over Sync ingestion.
Tracking each ingestion operation for a heavy load might have significant implications on the ingestion latency and the cluster responsiveness.
Firebase:
{exampleValue: "something";}
Javascript:
var ref = new Firebase(url + "/exampleValue");
ref.set("something");
I have a couple of questions around setting a value in Firebase to the same value, as shown above.
Does it use bandwidth for the user?
Does it count towards the bandwidth for my Firebase plan?
tl;dr: calling set() multiple times with the same value will use bandwidth each time.
You can easily test these things yourself. I just did the following: open the "Network" tab in the Chrome developer tools and find the WebSocket ("WS") that communicates with Firebase. In the "Frame" tab of that web socket you can see exactly what the Firebase client communicates with its server.
Then run the following in the developer console:
var newRef = ref.push();
This generates a new path/location, but does not send anything to the Firebase servers yet. Then:
newRef.set('This is a value');
This sets the value, so uses bandwidth.
Then I ran the same again:
newRef.set('This is a value');
This once again sent data to the Firebase server, so it uses bandwidth.
When you think about it, this also makes sense. The Firebase client knows only its local state. The value on the server might be different, for example because somebody else has changed the value. So when you tell it to set the value, it must send that command to the server to ensure that value actually gets written into the database.
I would like to know the Process Integration steps.
Through Outbound ports
If any of the event occurs at AX Dynamics, we just want to know that events in the form of XML(Process Integration).
Example: Sales Order Creation, Customer Creation, Purchase Order Creation..
Outbound ports are only useful for asynchronous communication.
See AX 2012 Export Data with Outbound ports for an example (using the file system).
The steps to initiate sending data is in the AIF_SendCustomer.
As this is no lightweight operation, you may consider logging the records which needs integration in a custom integration table, then doing the processing in batch.
This is done in the insert and/or update and maybe delete method.
Deletes requires you store the RecId field value in the external system to be used for delete requests. The following does not cover this.
For logged table make the following method:
void syncRecord()
{
XXXRecordLog log;
log.RefTableId = this.TableId;
log.RefRecId = this.RecId;
log.insert();
}
Then call this.syncRecord() in the insert and update methods.
In the query to the outbound service be sure to exists join your table and the log table. This way only changed records are exported.
Make a batch job to do the transfer using the AIF_SendCustomer as a template.
After a synchronous (AifSendMode::Sync) transfer of the records, delete the log records (or mark them transferred).
Finally call AIFoutboundProcessingService to flush the file:
new AIFoutboundProcessingService().run();
Try to keeps things simple. It might be simpler to do a comma file export of the changed records!
I'm working on an Adobe AIR application which can upload files to a web server, which is running Apache and PHP. Several files can be uploaded at the same time and the application also calls the web server for various API requests.
The problem I'm having is that if I start two file uploads, while they are in progress any other HTTP requests will time out, which is causing a problem for the application and from a user point of view.
Are Adobe AIR applications limited to 2 HTTP connections, or is something else probably the issue?
From searching about this issue I've not found much but one article did indicated that it wasn't limited to just two connections.
The file uploads are performed by calling the File classes upload method, and the API calls are done using the HTTPService class. The development web server I am using is a WAMP server, however when the application is released it will be talking to a LAMP server.
Thanks,
Grant
Here is the code I'm using to upload the file:
protected function btnAddFile_clickHandler(event:MouseEvent):void
{
// Create a new File object and display the browse file dialog
var uploadFile:File = new File();
uploadFile.browseForOpen("Select File to Upload");
uploadFile.addEventListener(Event.SELECT, uploadFile_SelectedHandler);
}
private function uploadFile_SelectedHandler(event:Event):void
{
// Get the File object which was used to select the file
var uploadFile:File = event.target as File;
uploadFile.addEventListener(ProgressEvent.PROGRESS, file_progressHandler);
uploadFile.addEventListener(IOErrorEvent.IO_ERROR, file_ioErrorHandler);
uploadFile.addEventListener(Event.COMPLETE, file_completeHandler);
// Create the request URL based on the download URL
var requestURL:URLRequest = new URLRequest(AppEnvironment.instance.serverHostname + "upload.php");
requestURL.method = URLRequestMethod.POST;
// Set the post parameters
var params:URLVariables = new URLVariables();
params.name = "filename.ext";
requestURL.data = params;
// Start uploading the file to the server
uploadFile.upload(requestURL, "file");
}
Here is the code for the API calls:
private function sendHTTPPost(apiFile:String, postParams:Object, resultCallback:Function, initialCallerResultCallback:Function):void
{
var httpService:mx.rpc.http.HTTPService = new mx.rpc.http.HTTPService();
httpService.url = AppEnvironment.instance.serverHostname + apiFile;
httpService.method = "POST";
httpService.requestTimeout = 10;
httpService.resultFormat = HTTPService.RESULT_FORMAT_TEXT;
httpService.addEventListener("result", resultCallback);
httpService.addEventListener("fault", httpFault);
var token:AsyncToken = httpService.send(postParams);
// Add the initial caller's result callback function to the token
token.initialCallerResultCallback = initialCallerResultCallback;
}
If you are on a windows system, Adobe AIR is using Microsofts WinINet library to access the web. This library by default limits the number of concurrent connections to a single server to 2:
WinInet limits the number of simultaneous connections that it makes to a single HTTP server. If you exceed this limit, the requests block until one of the current connections has completed. This is by design and is in agreement with the HTTP specification and industry standards.
... Connections to a single HTTP 1.1 server are limited to two simultaneous connections
There is an API to change the value of this limit but I don't know if it is accessible from AIR.
Since this limit also affects page loading speed for web sites, some sites are using multiple DNS names for artifacts such as images, javascripts and stylesheets to allow a browser to open more parallel connections.
So if you are controlling the server part, a workaround could be to create DNS aliases like www.example.com for uploads and api.example.com for API requests.
So as I was looking into this, I came across this info about using File.upload() in the documentation:
Starts the upload of the file to a remote server. Although Flash Player has no restriction on the size of files you can upload or download, the player officially supports uploads or downloads of up to 100 MB. You must call the FileReference.browse() or FileReferenceList.browse() method before you call this method.
Listeners receive events to indicate the progress, success, or failure of the upload. Although you can use the FileReferenceList object to let users select multiple files for upload, you must upload the files one by one; to do so, iterate through the FileReferenceList.fileList array of FileReference objects.
The FileReference.upload() and FileReference.download() functions are
nonblocking. These functions return after they are called, before the
file transmission is complete. In addition, if the FileReference
object goes out of scope, any upload or download that is not yet
completed on that object is canceled upon leaving the scope. Be sure
that your FileReference object remains in scope for as long as the
upload or download is expected to continue.
I wonder if something there could be giving you issues with uploading multiple files. I see that you are using browserForOpen() instead of browse(). It seems like the probably do the same thing... but maybe not.
I also saw this in the File class documentation
Note that because of new functionality added to the Flash Player, when publishing to Flash Player 10, you can have only one of the following operations active at one time: FileReference.browse(), FileReference.upload(), FileReference.download(), FileReference.load(), FileReference.save(). Otherwise, Flash Player throws a runtime error (code 2174). Use FileReference.cancel() to stop an operation in progress. This restriction applies only to Flash Player 10. Previous versions of Flash Player are unaffected by this restriction on simultaneous multiple operations.
When you say that you let users upload multiple files, do you mean subsequent calls to browse() and upload() or do you mean one call that includes multiple files? It seems that if you are trying to do multiple separate calls that that may be an issue.
Anyway, I don't know if this is much help. It definitely seems that what you are trying to do should be possible. I can only guess that what is going wrong is perhaps a problem with implementation. Good luck :)
Reference: http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/FileReference.html#upload()
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/FileReference.html#browse()
Just because I was thinking about a very similar question because of an error in one of my actual apps, I decided to write down the answer I found.
I instantiated 11
HttpConnections
and was wondering why my Flex 4 Application stopped working and threw an HTTP-Error although it was working pretty good formerly with just 5 simultanious HttpConnections to the same server.
I tested this myself because I did not find anything regarding this in the Flex docs or on the internet.
I found that using more than 5 HTTPConnections was the reason for the Flex application to throw the runtime error.
I decided to instantiate the connections one after another as a temporally workaround: Load the next one after the other has received the data and so on.
Thats of course just temporally since one of the next steps will be to alter the responding server code in that way that it answers a request that contains the results of requests to more then one table in one respond. Of course the client application logic needs to be altered, too.
I have an application that will reside within a business2business network that will communicate with our AS400 in our internal network environment. The firewall has been configured to allow the data request through to our AS400, but we are seeing a huge lag time in connection speed and response time. For example what takes less than a half second in our local development environments is taking upwards of 120 seconds in our B2B environment.
This is the function that we are utilizing to get our data. We are using the enterprise library application blocks, so the ASI object is the Database...
/// <summary>
/// Generic function to retrieve data table from AS400
/// </summary>
/// <param name="sql">SQL String</param>
/// <returns></returns>
private DataTable GetASIDataTable(string sql)
{
DataTable tbl = null;
HttpContext.Current.Trace.Warn("GetASIDataTable(" + sql + ") BEGIN");
using (var cmd = ASI.GetSqlStringCommand(sql))
{
using (var ds = ASI.ExecuteDataSet(cmd))
{
if (ds.Tables.Count > 0) tbl = ds.Tables[0];
}
}
HttpContext.Current.Trace.Warn("GetASIDataTable() END");
return tbl;
}
I am trying to brainstorm some ideas to consider as to why this is occurring.
Have never used ASP.NET or AS400 in anger, but I have seen this kind of behaviour before and it usually indicated some kind of network problem, typically a reverse DNS lookup that is timing out.
Assuming you have ping enabled through your firewall, check that you can ping in both directions.
Also run traceroute from each machine to try and diagnose where a delay might be.
Hope that helps.
Sorry but I can't tell you what is going on but I just have a couple comments...
First I would output the sql, see if it has a lot of joins and/or is hitting a table (file) with a large amount of records. If you really want to dig in fire up your profiler of choice (I use Ants Profiler) and try to find a profiler for the 400 - see what the server resources are as well as actual query after it goes thru the odbc driver.
I have worked with asp.net and as400 a few times and the way I have been most successful is actually using sql server with a linked server to AS400. I created a view to make it simpler to work with - hiding the oddities of as400 naming. It worked well in my scenario because the application needed to pull information from sql server anyway.
I thought I would mention it in case it helps... best of luck
Check the size of your iSeries system as well. Depending on the size of the query and if the system is undersized for the applications running on it, this may take time. While it shouldn't be thrown out as a posibility, I have seen a similar behavior in the past. But of course more likely is a network issue.
The other idea if you can solve the speed issue or is a sizing problem is to store it in an MS SQL Server then write the records from SQL Server to the iSeries from there.