The AMF response data structure called from ExtJS is different from that called from Flex - apache-flex

I'm trying to use ExtJS5.1.2 as a front-end to replace Flex3 UI.
I heard that ExtJS can use AMF. I tested calling AMF from ExtJS, and it was success.
I expected it returns the same response as AMF+Flex3. But it didn't.
The response it called from ExtJS is slower and heavy than Flex.
What I tested:
Get 10000 records from Oracle DB and show it at DataGrid.
The data size:
13.1MB(ExtJS)
2.6MB(Flex3)
I saw the responses at Firebug.
The binary response is garbling, but it looks that the data structures are different.
in ExtJS5.1.2
The field names appeared every record.
POST request
������foo.getTestRecords�/1ÿÿÿÿ
����
POST response
������/1/onResult��ÿÿÿÿ�!flex.messaging.io.ArrayCollection�source
��'�modules.entity.APR361�SEQNO� APM20HSEQ�KOCHGCD�HVPNO�FGAINO�TORYOKBN�NOTE1�MSIZE3
�0�SECKBN�MSIZE2�0�MSIZE1�0�REQFL�SKSEQNO�DJKBN�1�PCLS�1�ZUBAN�GACD� HUNITNONM
�SRCLS�502�PYVPNO�0�ZBNHCNT�DSPSEQ�YOSETU�PVPUNIT� APR363SEQ�UWSID�25803�PHKADOFL
�PBANNO�SECHGCD�PGDNM�PVLIN�0�APR36SEQ�892999�PJVSEQ�BA00010G0090�ITEMKB�GKBN�LINECD
�HKS�ZAICD�PVJAN�NYCHGCD�IPRGID�PA012X�PEND�0�FVPNO�0�PSYDATBt×oÏ���MKCD�PPRKBN2
�0�PPRKBN1�0�ISEQ�893000�PPRKBN3�0�TOSOCLO2�TOSOCLO1�PGDOHCLS�TOSOCLO3�SGVJAN�2621
�UPRGID�PA014X�IDATBtø5 Î��PPRDAT2�PPRDAT1�COMKBN�BUNCNT�PPRDAT3�ICHGCD�EMORI�NXGKBN
�GAOHCLS�PPRBSU3�PPRBSU2�ITEMOPE�PVCNT�WARI�B�TIM�20151019140515920� GENCHKDAT�VISEQ
�1343�SEC�NNOTE� APR362SEQ�UCHGCD�1053�MSGGRPID�PPRBSU1� APR361SEQ�HCNT�DDAT�SYUZUJK
�HINKB�UDATBuç˯��WRKPTN�00�MCOMSEQ�LVL�MKNM�PVPNO�0� APM21HSEQ�1018811�
SKKKSEREDA�A0010�BHCLS�PXFL�PLOT�PPRBAT1�GANM�PPRBAT2�CNCOMSEQ�RGENZU�ZAINM�NOTE10
�NOTE7�NOTE6�DFLG�0�CAMKBN�NOTE9�IWSID�6102�NOTE8�TOSOPRO�NOTE3�NOTE2�PPRBAT3
�NOTE5�NOTE4�PEYDATBtãýOØ���� �modules.entity.APR361�SEQNO� APM20HSEQ�KOCHGCD�HVPNO
�FGAINO�TORYOKBN�NOTE1�MSIZE3�SECKBN�MSIZE2�MSIZE1�REQFL�SKSEQNO�DJKBN�0�PCLS
�2�ZUBAN�GACD� HUNITNONM�SRCLS�503�PYVPNO�0�ZBNHCNT�DSPSEQ�YOSETU�PVPUNIT� APR363SEQ
�1064563�UWSID�7823�PHKADOFL�PBANNO�SECHGCD�PGDNM�PVLIN�0�APR36SEQ�1064307�PJVSEQ
�BA00013G0010�ITEMKB�GKBN�LINECD�HKS�ZAICD�PVJAN�NYCHGCD�IPRGID�PA014X�PEND�0�FVPNO
�0�PSYDAT�MKCD�PPRKBN2�0�PPRKBN1�0�ISEQ�1064567�PPRKBN3�0�TOSOCLO2�TOSOCLO1�PGDOHCLS
�TOSOCLO3�SGVJAN�1123�UPRGID�PA025X�IDATBtà¨ÈÆ���PPRDAT2�PPRDAT1�COMKBN�BUNCNT�
PPRDAT3�ICHGCD�EMORI�NXGKBN�GAOHCLS�PPRBSU3�PPRBSU2�ITEMOPE�PVCNT�WARI�B�TIM�20150803092306612
� GENCHKDAT�VISEQ�1691�SEC�NNOTE� APR362SEQ�UCHGCD�EMORI�MSGGRPID�PPRBSU1� APR361SEQ
�HCNT�DDAT�SYUZUJK�HINKB�UDATBtâÙh���WRKPTN�00�MCOMSEQ�LVL�MKNM�PVPNO�0� APM21HSEQ
�1018657�
SKKKSEREDA�BHCLS�1�PXFL�PLOT�PPRBAT1�GANM�PPRBAT2�CNCOMSEQ�RGENZU�ZAINM�NOTE10
�NOTE7�NOTE6�DFLG�0�CAMKBN�NOTE9�IWSID�7447�NOTE8�TOSOPRO�NOTE3�NOTE2�PPRBAT3
�NOTE5�NOTE4�PEYDAT�� �modules.entity.APR361�SEQNO�000100� APM20HSEQ�KOCHGCD�HVPNO�
FGAINO�TORYOKBN�NOTE1�MSIZE3�0�SECKBN�MSIZE2�0�MSIZE1�0�REQFL�0�SKSEQNO�000100
�DJKBN�0�PCLS�3�ZUBAN�GACD� HUNITNONM�SRCLS�505�PYVPNO�2�ZBNHCNT�DSPSEQ�10�YOSETU
�PVPUNIT�01� APR363SEQ�UWSID�PHKADOFL�0�PBANNO�00�SECHGCD�PGDNM�æ©ç¨®åï½ï½ï½
ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï
½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï¿¥�PVLIN�1�APR36SEQ�1064307�PJVSEQ�BA00013G0010�ITEMKB�0�GKBN
�G�LINECD�HKS�ZAICD�PVJAN�999000�NYCHGCD�IPRGID�PD230S�PEND�0�FVPNO�0�PSYDAT�
MKCD�PPRKBN2�0�PPRKBN1�0�ISEQ�1064589�PPRKBN3�0�TOSOCLO2�TOSOCLO1�PGDOHCLS�0�TOSOCLO3
�SGVJAN�1123�UPRGID�IDATBtà©bV��PPRDAT2�PPRDAT1�COMKBN�BUNCNT�PPRDAT3�ICHGCD�
EMORI�NXGKBN�1�GAOHCLS�PPRBSU3�PPRBSU2�ITEMOPE�PVCNT�1�WARI�B�TIM�20150803092306612
� GENCHKDAT�VISEQ�1700�SEC�NNOTE� APR362SEQ�UCHGCD�MSGGRPID�PPRBSU1� APR361SEQ�1064569
�HCNT�0�DDAT�SYUZUJK�HINKB�UDAT�WRKPTN�00�MCOMSEQ�LVL�0�MKNM�PVPNO�0� APM21HSEQ
�1018657�
in Flex3
The field names appeared once.
POST request
�����null�/10��,
���
Oflex.messaging.messages.RemotingMessageoperation
sourcedestinationtimestamp bodymessageIdclientIdtimeToLiveheadersgetTestRecords
foo� I92F7BB91-48E7-EB8E-D6A3-B69C86CDD6D8IDD3DCC46-C0C1-F47D-8317-4163090A7318�
DSEndpoint
my-amf DSIdIDD392F8D-20AF-9878-D0A1-9BEE2CFD8DB0
POST response
�����/10/onResult��ÿÿÿÿ
Uflex.messaging.messages.AcknowledgeMessagetimestampheaders bodycorrelationIdmessageIdtimeToLive
clientIddestinationBubSr0�
Cflex.messaging.io.ArrayCollection
#+modules.entity.APR361SEQNOAPM20HSEQKOCHGCDHVPNO
FGAINOTORYOKBNNOTE1
MSIZE3
SECKBN
MSIZE2
MSIZE1REQFLSKSEQNODJKBN PCLSZUBAN GACDHUNITNONMSRCLS
PYVPNOZBNHCNT
DSPSEQ
YOSETUPVPUNITAPR363SEQUWSIDPHKADOFL
PBANNOSECHGCDPGDNMPVLINAPR36SEQ
PJVSEQ
ITEMKB GKBN
LINECDZAICDPVJANNYCHGCD
IPRGID PENDFVPNO
PSYDAT MKCDPPRKBN2PPRKBN1 ISEQPPRKBN3TOSOCLO2TOSOCLO1PGDOHCLSTOSOCLO3
SGVJAN
UPRGID IDATPPRDAT2PPRDAT1
COMKBN
BUNCNTPPRDAT3
ICHGCD
NXGKBNGAOHCLSPPRBSU3PPRBSU2ITEMOPEPVCNT WARITIMGENCHKDATVISEQSECNNOTEAPR362SEQ
UCHGCDMSGGRPIDPPRBSU1APR361SEQ HCNT DDATSYUZUJKHINKB UDAT
WRKPTNMCOMSEQLVL MKNMPVPNOAPM21HSEQSKKKSEREDABHCLS PXFL PLOTPPRBAT1 GANMPPRBAT2CNCOMSEQ
RGENZUZAINM
NOTE10NOTE7NOTE6 DFLG
CAMKBNNOTE9IWSIDNOTE8TOSOPRONOTE3NOTE2PPRBAT3NOTE5NOTE4
PEYDAT0zz1|502z25803z
892999BA00010G0090HKS
PA012XzzBt×oÏ��zz
893000z 2621
PA014XBtø5 Î�EMORIB#20151019140515920 1343 1053Buç˯�00z
1018811A0010z 6102BtãýOØ��
z2503z1064563 7823z1064307BA00013G0010zzz
z1064567z 1123
PA025XBtà¨ÈÆ��#20150803092306612 1691BtâÙh��z1018657
|z 7447
000100zzzz<z3505"1001z-æ©ç¨®åï½ï½ï½ï½ï½ï½ï½ï½
ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï
½ï½ï½ï½ï½ï¿¥|*,zG
999000
PD230Szzzz1064589zz0Btà©bV�||4 17001064569z
zz8"zz 7452
000200z>zzzVz> MCM-002864#420DzC溶ãã¼ãã§ã¼ã³ãï
¼¢ï¼«ï¼´"*,z
002864Lzzzz1064590zz0Btà©b��4P1064592Rz
zSSï¼ï¼ï¼z8"zzT
The question:
In spite of both are calling same AMF, why are the responses different?
I need lightweight response like a Flex+AMF if possible.
Any help would be greatly appreciated.
I'm using Ext JS 5.1.2, BlazeDS3, Tomcat6, Adobe Flex3.4.
Updated: Nov 04 16:05(JST)
I've read the source code of Ext.direct.AmfRemotingProvider and discovered this.
if (me.binary) {
encoder = new Ext.data.amf.Encoder( {format: 0}); // AMF message sending always uses AMF0 Here!!
// encode packet
encoder.writeAmfPacket(amfHeaders, amfMessages);
request.binaryData = encoder.bytes;
request.binary = true; // Binary response
request.headers = {'Content-Type': 'application/x-amf'};
} else {
encoder = new Ext.data.amf.XmlEncoder();
// encode packet
encoder.writeAmfxRemotingPacket(amfMessages[0]);
request.xmlData = encoder.body;
}
AmfRemotingProvider uses AMF0.
To be honest, I don't understand much about what's the difference between AMF0 and AMF3.
Anyway I'm doubting there and researching how to use AMF3 from ExtJS.
The common Java code
public List<APR361> getTestRecords() throws DAOException, IOException {
List<APR361> list = new ArrayList<APR361>();
Connection c = null;
PreparedStatement ps = null;
ResultSet rs = null;
String w_sql = "";
try {
w_sql = "SELECT A.* FROM APR361 A WHERE ROWNUM<10001";
c = ConnectionHelper.getConnection();
ps = c.prepareStatement(w_sql);
rs = ps.executeQuery();
while (rs.next()) {
list.add(new APR361(
rs.getString("ISEQ"),
rs.getString("TIM"),
rs.getString("DFLG"),
rs.getString("VISEQ"),
rs.getTimestamp("IDAT"),
rs.getString("IWSID"),
rs.getString("IPRGID"),
rs.getString("ICHGCD"),
rs.getTimestamp("UDAT"),
rs.getString("UWSID"),
rs.getString("UPRGID"),
rs.getString("UCHGCD"),
rs.getTimestamp("DDAT"),
rs.getString("LINECD"),
rs.getString("APR36SEQ"),
rs.getString("PJVSEQ"),
rs.getString("SRCLS"),
rs.getString("PCLS"),
rs.getString("BHCLS"),
rs.getString("PVLIN"),
rs.getString("PVCNT"),
rs.getString("PLOT"),
rs.getString("REQFL"),
rs.getString("PXFL"),
rs.getString("DJKBN"),
rs.getString("APR362SEQ"),
rs.getString("APR361SEQ"),
rs.getString("APR363SEQ"),
rs.getString("SEQNO"),
rs.getString("SKSEQNO"),
rs.getString("DSPSEQ"),
rs.getString("HCNT"),
rs.getString("BUNCNT"),
rs.getString("PVJAN"),
rs.getString("PGDOHCLS"),
rs.getString("PGDNM"),
rs.getString("WRKPTN"),
rs.getString("APM20HSEQ"),
rs.getString("APM21HSEQ"),
rs.getString("LVL"),
rs.getString("HINKB"),
rs.getString("SGVJAN"),
rs.getString("PYVPNO"),
rs.getString("PVPUNIT"),
rs.getTimestamp("PSYDAT"),
rs.getTimestamp("PEYDAT"),
rs.getString("PHKADOFL"),
rs.getString("WARI"),
rs.getString("SEC"),
rs.getString("MSGGRPID"),
rs.getString("HVPNO"),
rs.getString("NXGKBN"),
rs.getString("GACD"),
rs.getString("GAOHCLS"),
rs.getString("GANM"),
rs.getString("PVPNO"),
rs.getString("FVPNO"),
rs.getString("PEND"),
rs.getString("PPRKBN1"),
rs.getString("PPRBAT1"),
rs.getTimestamp("PPRDAT1"),
rs.getString("PPRBSU1"),
rs.getString("PPRKBN2"),
rs.getString("PPRBAT2"),
rs.getTimestamp("PPRDAT2"),
rs.getString("PPRBSU2"),
rs.getString("PPRKBN3"),
rs.getString("PPRBAT3"),
rs.getTimestamp("PPRDAT3"),
rs.getString("PPRBSU3"),
rs.getString("ITEMKB"),
rs.getString("ZUBAN"),
rs.getString("MKCD"),
rs.getString("MKNM"),
rs.getString("ZAICD"),
rs.getString("ZAINM"),
rs.getString("SECKBN"),
rs.getString("MSIZE1"),
rs.getString("MSIZE2"),
rs.getString("MSIZE3"),
rs.getString("RGENZU"),
rs.getString("FGAINO"),
rs.getString("TORYOKBN"),
rs.getString("GKBN"),
rs.getString("PBANNO"),
rs.getString("HUNITNONM"),
rs.getString("NNOTE"),
rs.getString("SKKKSEREDA"),
rs.getTimestamp("GENCHKDAT"),
rs.getString("KOCHGCD"),
rs.getString("SECHGCD"),
rs.getString("NYCHGCD"),
rs.getString("SYUZUJK"),
rs.getString("ITEMOPE"),
rs.getString("TOSOPRO"),
rs.getString("TOSOCLO1"),
rs.getString("TOSOCLO2"),
rs.getString("TOSOCLO3"),
rs.getString("NOTE1"),
rs.getString("NOTE2"),
rs.getString("NOTE3"),
rs.getString("NOTE4"),
rs.getString("NOTE5"),
rs.getString("NOTE6"),
rs.getString("NOTE7"),
rs.getString("NOTE8"),
rs.getString("NOTE9"),
rs.getString("NOTE10"),
rs.getString("ZBNHCNT"),
rs.getString("CAMKBN"),
rs.getString("YOSETU"),
rs.getString("MCOMSEQ"),
rs.getString("CNCOMSEQ"),
rs.getString("COMKBN")
));
}
} catch (SQLException e) {
e.printStackTrace();
throw new DAOException(e);
} finally {
try {
if (rs != null) rs.close();
if (ps != null) ps.close();
} catch (SQLException e) {
e.printStackTrace();
}
ConnectionHelper.close(c);
}
return list;
}
Call AMF from ExtJS
Ext.Direct.addProvider({
"url":"http://127.0.0.1:8400/myTestApp/messagebroker/amf", // URL for the AMFEndpoint
"type":"amfremoting",
"endpoint": "my-amf", // the name of the AMFEndpoint channel as defined in the server's services-config.xml
"binary": true, // chooses AMF encoding
"actions":{
"foo":[{ // name of the destination as defined in remoting-config.xml on the server
"name":"getTestRecords", // method name of the method to call
"len":0 // number of parameters
}]
}
});
foo.getTestRecords((function(provider, response) {
var store = Ext.getStore("baz");
store.loadData(provider);
}));
Call AMF from Flex3
public var srv: RemoteObject;
public var arrTestRecords: ArrayCollection;
public function setSRV(): void
{
srv = new RemoteObject();
srv.showBusyCursor = true;
srv.destination = "foo";
var messageUrl:String = "http://{server.name}:{server.port}/myTestApp/messagebroker/amf";
var cs:ChannelSet = new ChannelSet();
cs.addChannel( new AMFChannel("my-amf", messageUrl) );
srv.channelSet = cs;
}
public function getTest(nextfunc: Function=null): void
{
var token: AsyncToken = srv.getTestRecords();
token.addResponder(new AsyncResponder(
// on success
function(e:ResultEvent, obj:Object=null):void
{
var w_arr: ArrayCollection;
var i: int;
w_arr = new ArrayCollection();
w_arr = e.result;
arrTestRecords = new ArrayCollection();
for (i=0; i<w_arr.length; i++) {
arrTestRecords.addItem(APR361(w_arr[i]));
}
arrTestRecords.refresh();
grid.dataProvider = arrTestRecords;
},
// NG
TokenFaultEvent
));
}
Updated: Feb 02 2016 11:35(JST)
I tried to use Node.js & node-oracledb driver with gzip compression instead of using the AMF & BlazeDS.
The response data size was 900KB!
ExtJS+Node.js(900KB) vs Flex3+AMF3(2.6MB)
However the AMF3 still about 2x faster than Node.js.

Related

Application Cache and Slow Process

I want to create an application wide feed on my ASP.net 3.5 web site using the application cache. The data that I am using to populate the cache is slow to obtain, maybe up to 10 seconds (from a remote server's data feed). My question/confusion is, what is the best way to structure the cache management.
private const string CacheKey = "MyCachedString";
private static string lockString = "";
public string GetCachedString()
{
string data = (string)Cache[CacheKey];
string newData = "";
if (data == null)
{
// A - Should this method call go here?
newData = SlowResourceMethod();
lock (lockString)
{
data = (string)Cache[CacheKey];
if (data != null)
{
return data;
}
// B - Or here, within the lock?
newData = SlowResourceMethod();
Cache[CacheKey] = data = newData;
}
}
return data;
}
The actual method would be presented by and HttpHandler (.ashx).
If I collect the data at point 'A', I keep the lock time short, but might end up calling the external resource many times (from web pages all trying to reference the feed). If I put it at point 'B', the lock time will be long, which I am assuming is a bad thing.
What is the best approach, or is there a better pattern that I could use?
Any advice would be appreciated.
I add the comments on the code.
private const string CacheKey = "MyCachedString";
private static readonly object syncLock = new object();
public string GetCachedString()
{
string data = (string)Cache[CacheKey];
string newData = "";
// start to check if you have it on cache
if (data == null)
{
// A - Should this method call go here?
// absolut not here
// newData = SlowResourceMethod();
// we are now here and wait for someone else to make it or not
lock (syncLock)
{
// now lets see if some one else make it...
data = (string)Cache[CacheKey];
// we have it, send it
if (data != null)
{
return data;
}
// not have it, now is the time to look for it.
// B - Or here, within the lock?
newData = SlowResourceMethod();
// set it on cache
Cache[CacheKey] = data = newData;
}
}
return data;
}
Better for me is to use mutex and lock depended on the name CacheKey and not lock all resource and the non relative one. With mutex one basic simple example will be:
private const string CacheKey = "MyCachedString";
public string GetCachedString()
{
string data = (string)Cache[CacheKey];
string newData = "";
// start to check if you have it on cache
if (data == null)
{
// lock it base on resource key
// (note that not all chars are valid for name)
var mut = new Mutex(true, CacheKey);
try
{
// Wait until it is safe to enter.
// but also add 30 seconds max
mut.WaitOne(30000);
// now lets see if some one else make it...
data = (string)Cache[CacheKey];
// we have it, send it
if (data != null)
{
return data;
}
// not have it, now is the time to look for it.
// B - Or here, within the lock?
newData = SlowResourceMethod();
// set it on cache
Cache[CacheKey] = data = newData;
}
finally
{
// Release the Mutex.
mut.ReleaseMutex();
}
}
return data;
}
You can also read
Image caching issue by using files in ASP.NET

Raven DB DocumentStore - throws out of memory exception

I have code like this:
public bool Set(IEnumerable<WhiteForest.Common.Entities.Projections.RequestProjection> requests)
{
var documentSession = _documentStore.OpenSession();
//{
try
{
foreach (var request in requests)
{
documentSession.Store(request);
}
//requests.AsParallel().ForAll(x => documentSession.Store(x));
documentSession.SaveChanges();
documentSession.Dispose();
return true;
}
catch (Exception e)
{
_log.LogDebug("Exception in RavenRequstRepository - Set. Exception is [{0}]", e.ToString());
return false;
}
//}
}
This code gets called many times. After i get to around 50,000 documents that have passed through it i get an OutOfMemoryException.
Any idea why ? perhaps after a while i need to declare a new DocumentStore ?
thank you
**
UPDATE:
**
I ended up using the Batch/Patch API to perform the update I needed.
You can see the discussion here: https://groups.google.com/d/topic/ravendb/3wRT9c8Y-YE/discussion
Basically since i only needed to update 1 property on my objects, and after considering ayendes comments about re-serializing all the objects back to JSON, i did something like this:
internal void Patch()
{
List<string> docIds = new List<string>() { "596548a7-61ef-4465-95bc-b651079f4888", "cbbca8d5-be45-4e0d-91cf-f4129e13e65e" };
using (var session = _documentStore.OpenSession())
{
session.Advanced.DatabaseCommands.Batch(GenerateCommands(docIds));
}
}
private List<ICommandData> GenerateCommands(List<string> docIds )
{
List<ICommandData> retList = new List<ICommandData>();
foreach (var item in docIds)
{
retList.Add(new PatchCommandData()
{
Key = item,
Patches = new[] { new Raven.Abstractions.Data.PatchRequest () {
Name = "Processed",
Type = Raven.Abstractions.Data.PatchCommandType.Set,
Value = new RavenJValue(true)
}}});
}
return retList;
}
Hope this helps ...
Thanks alot.
I just did this for my current project. I chunked the data into pieces and saved each chunk in a new session. This may work for you, too.
Note, this example shows chunking by 1024 documents at a time, but needing at least 2000 before we decide it's worth chunking. So far, my inserts got the best performance with a chunk size of 4096. I think that's because my documents are relatively small.
internal static void WriteObjectList<T>(List<T> objectList)
{
int numberOfObjectsThatWarrantChunking = 2000; // Don't bother chunking unless we have at least this many objects.
if (objectList.Count < numberOfObjectsThatWarrantChunking)
{
// Just write them all at once.
using (IDocumentSession ravenSession = GetRavenSession())
{
objectList.ForEach(x => ravenSession.Store(x));
ravenSession.SaveChanges();
}
return;
}
int numberOfDocumentsPerSession = 1024; // Chunk size
List<List<T>> objectListInChunks = new List<List<T>>();
for (int i = 0; i < objectList.Count; i += numberOfDocumentsPerSession)
{
objectListInChunks.Add(objectList.Skip(i).Take(numberOfDocumentsPerSession).ToList());
}
Parallel.ForEach(objectListInChunks, listOfObjects =>
{
using (IDocumentSession ravenSession = GetRavenSession())
{
listOfObjects.ForEach(x => ravenSession.Store(x));
ravenSession.SaveChanges();
}
});
}
private static IDocumentSession GetRavenSession()
{
return _ravenDatabase.OpenSession();
}
Are you trying to save it all in one call?
The DocumentSession need to turn all of the objects that you pass it into a single request to the server. That means that it may allocate a lot of memory for the write to the server.
Usually we recommend on batches of about 1,024 items in you are doing bulks saves.
DocumentStore is a disposable class, so I worked around this problem by disposing the instance after each chunk. I highly doubt this is the most efficient way to run operations, but it will prevent significant memory overhead from happening.
I was running a sort of "delete all" operation like so. You can see the using blocks disposing both the DocumentStore and the IDocumentSession objects after each chunk.
static DocumentStore GetDataStore()
{
DocumentStore ds = new DocumentStore
{
DefaultDatabase = "test",
Url = "http://localhost:8080"
};
ds.Initialize();
return ds;
}
static IDocumentSession GetDbInstance(DocumentStore ds)
{
return ds.OpenSession();
}
static void Main(string[] args)
{
do
{
using (var ds = GetDataStore())
using (var db = GetDbInstance(ds))
{
//The `Take` operation will cap out at 1,024 by default, per Raven documentation
var list = db.Query<MyClass>().Skip(deleteSum).Take(5000).ToList();
deleteCount = list.Count;
deleteSum += deleteCount;
foreach (var item in list)
{
db.Delete(item);
}
db.SaveChanges();
list.Clear();
}
} while (deleteCount > 0);
}

Flex prevent URL encoding of params with HTTPRequest

I'm trying to port an existing AJAX app to Flex, and having trouble with the encoding of parameters sent to the backend service.
When trying to perform the action of deleting a contact, the existing app performs a POST, sending the the following: (captured with firebug)
contactRequest.contacts[0].contactId=2c33ddc6012a100096326b40a501ec72
So, I create the following code:
var service:HTTPService;
function initalizeService():void
{
service = new HTTPService();
service.url = "http://someservice";
service.method = 'POST';
}
public function sendReq():void
{
var params:Object = new Object();
params['contactRequest.contacts[0].contactId'] = '2c33ddc6012a100097876b40a501ec72';
service.send(params);
}
In firebug, I see this sent out as follows:
Content-type: application/x-www-form-urlencoded
Content-length: 77
contactRequest%2Econtacts%5B0%5D%2EcontactId=2c33ddc6012a100097876b40a501ec72
Flex is URL encoding the params before sending them, and we're getting an error returned from the server.
How do I disable this encoding, and get the params sent as-is, without the URL encoding?
I feel like the contentType property should be the key - but neither of the defined values work.
Also, I've considered writing a SerializationFilter, but this seems like overkill - is there a simpler way?
Writing a SerializtionFilter seemed to do the trick:
public class MyFilter extends SerializationFilter
{
public function MyFilter()
{
super();
}
override public function serializeBody(operation:AbstractOperation, obj:Object):Object
{
var s:String = "";
var classinfo:Object = ObjectUtil.getClassInfo(obj);
for each (var p:* in classinfo.properties)
{
var val:* = obj[p];
if (val != null)
{
if (s.length > 0)
s += "&";
s += StringUtil.substitute("{0}={1}",p,val);
}
}
return s;
}
}
I'd love to know any alternative solutions that don't involve doing this though!

Webtest with session-id in url

We have an ASP.Net site that redirects you to a url that shows a session-id. like this:
http://localhost/(S(f3rjcw45q4cqarboeme53lbx))/main.aspx
This id is unique with every request.
Is it possible to test this site using a standard visual studio 2008/2010 webtest? How can I provide the test this data?
I have to call a couple of different pages using that same id.
Yes, it is relatively easy to do this. You will need to create a coded webtest however.
In my example we have a login post that will return the url including the session string.
Just after the we yield the login post request (request3) to the enumerator I call the following.
WebTestRequest request3 = new WebTestRequest((this.Context["WebServer1"].ToString() + "/ICS/Login/English/Login.aspx"));
//more request setup code removed for clarity
yield return request3;
string responseUrl = Context.LastResponse.ResponseUri.AbsoluteUri;
string cookieUrl = GetUrlCookie(responseUrl, this.Context["WebServer1"].ToString(),"/main.aspx");
request3 = null;
Where GetUrlCookie is something like this:
public static string GetUrlCookie(string fullUrl, string webServerUrl, string afterUrlPArt)
{
string result = fullUrl.Substring(webServerUrl.Length);
result = result.Substring(0, result.Length - afterUrlPArt.Length);
return result;
}
Once you have the session cookie string, you can substitute it really easy in any subsequent urls for request/post
e.g.
WebTestRequest request4 = new WebTestRequest((this.Context["WebServer1"].ToString() + cookieUrl + "/mySecureForm.aspx"));
I apologise for my code being so rough, but it was deprecated in my project and is pulled from the first version of the codebase - and for saying it was easy :)
For any load testing, depending on your application, you may have to come up with a stored procedure to call to provide distinct login information each time the test is run.
Note, because the response url cannot be determined ahead of time, for the login post you will have to temporarily turn off the urlValidationEventHandler. To do this I store the validationruleeventhandler in a local variable:
ValidateResponseUrl validationRule1 = new ValidateResponseUrl();
urlValidationRuleEventHandler = new EventHandler<ValidationEventArgs>(validationRule1.Validate);
So can then turn it on and off as I require:
this.ValidateResponse -= urlValidationRuleEventHandler ;
this.ValidateResponse += urlValidationRuleEventHandler ;
The alternative is to code your own such as this (reflectored from the Visual Studio code and changed to be case insensitive.
class QueryLessCaseInsensitiveValidateResponseUrl : ValidateResponseUrl
{
public override void Validate(object sender, ValidationEventArgs e)
{
Uri uri;
string uriString = string.IsNullOrEmpty(e.Request.ExpectedResponseUrl) ? e.Request.Url : e.Request.ExpectedResponseUrl;
if (!Uri.TryCreate(e.Request.Url, UriKind.Absolute, out uri))
{
e.Message = "The request URL could not be parsed";
e.IsValid = false;
}
else
{
Uri uri2;
string leftPart = uri.GetLeftPart(UriPartial.Path);
if (!Uri.TryCreate(uriString, UriKind.Absolute, out uri2))
{
e.Message = "The request URL could not be parsed";
e.IsValid = false;
}
else
{
uriString = uri2.GetLeftPart(UriPartial.Path);
////this removes the query string
//uriString.Substring(0, uriString.Length - uri2.Query.Length);
Uri uritemp = new Uri(uriString);
if (uritemp.Query.Length > 0)
{
string fred = "There is a problem";
}
//changed to ignore case
if (string.Equals(leftPart, uriString, StringComparison.OrdinalIgnoreCase))
{
e.IsValid = true;
}
else
{
e.Message = string.Format("The value of the ExpectedResponseUrl property '{0}' does not equal the actual response URL '{1}'. QueryString parameters were ignored.", new object[] { uriString, leftPart });
e.IsValid = false;
}
}
}
}
}

Pause and resume download in flex?

Is it possible in an air application to start a download, pause it and after that resume it?
I want to download very big files (1-3Gb) and I need to be sure if the connection is interrupted, then the next time the user tries to download the file it's start from the last position.
Any ideas and source code samples would be appreciated.
Yes, you would want to use the URLStream class (URLLoader doesn't support partial downloads) and the HTTP Range header. Note that there are some onerous security restrictions on the Range header, but it should be fine in an AIR application. Here's some untested code that should give you the general idea.
private var _us:URLStream;
private var _buf:ByteArray;
private var _offs:uint;
private var _paused:Boolean;
private var _intervalId:uint;
...
private function init():void {
_buf = new ByteArray();
_offs = 0;
var ur:URLRequest = new URLRequest( ... uri ... );
_us = new URLStream();
_paused = false;
_intervalId = setInterval(500, partialLoad);
}
...
private function partialLoad():void {
var len:uint = _us.bytesAvailable;
_us.readBytes(_buf, _offs, len);
_offs += len;
if (_paused) {
_us.close();
clearInterval(_intervalId);
}
}
...
private function pause():void {
_paused = true;
}
...
private function resume():void {
var ur:URLRequest = new URLRequest(... uri ...);
ur.requestHeaders = [new URLRequestHeader("Range", "bytes=" + _offs + "-")];
_us.load(ur);
_paused = false;
_intervalId = setInterval(500, partialLoad);
}
if you are targeting mobile devices, maybe you should take a look at this native extension: http://myappsnippet.com/download-manager-air-native-extension/ it supports simultaneous resumable downloads with multi-section chunks to download files as fast as possible.

Resources