C# Marshal byte array to struct - .net-core-3.0

I find many answers to my question and they all work. My question is are they all equal in speed and memory. How can I tell what is faster and uses less memory. I don't normally use the Marshal and GCHandle classes. So I am totally green.
public static object RawDeserializer(byte[] rawData, int position, Type anyType)
{
int rawsize = Marshal.SizeOf(anyType);
if (rawsize > rawData.Length)
return null;
IntPtr buffer = Marshal.AllocHGlobal(rawsize);
Marshal.Copy(rawData, position, buffer, rawsize);
object retobj = Marshal.PtrToStructure(buffer, anyType);
Marshal.FreeHGlobal(buffer);
return retobj;
}
public static T RawDeserializer<T>(byte[] rawData, int position = 0)
{
int rawsize = Marshal.SizeOf(typeof(T));
if (rawsize > rawData.Length)
{
throw new DataMisalignedException("byte array is not the correct size for the requested type");
}
IntPtr buffer = Marshal.AllocHGlobal(rawsize);
Marshal.Copy(rawData, position, buffer, rawsize);
T retobj = (T)Marshal.PtrToStructure(buffer, typeof(T));
Marshal.FreeHGlobal(buffer);
return retobj;
}
public static T RawDeserializer<T>(byte[] bytes) where T : struct
{
T stuff;
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
try
{
stuff = Marshal.PtrToStructure<T>(handle.AddrOfPinnedObject());
}
finally
{
handle.Free();
}
return stuff;
}
I am getting the desired results form all 3 implementations.

First and second are almost identical: difference is that you do not unbox (cast to T:struct) the result in the first example, I'd assume that you'll unbox it later though.
Third option does not copy memory to the unmanaged heap, it just pins it in manageed heap, so I'd assume it will allocate less memory and will be faster. I don't pretend to be a golden source of truth though, so just go and make performance testing of these options :) BenchmarkDotNet is a great framework for performance testing and may help you a lot.
Also third option could be more concise:
public static unsafe T RawDeserializer<T>(byte[] bytes) where T : struct
{
fixed (byte* p = bytes)
return Marshal.PtrToStructure<T>((IntPtr)p);
}
You need to change project settings to allow unsafe code though:
To do not be totally green, I'd strongly recommend to read a book CLR via C#, Chapter 21 'The Managed Heap and Garbage Collection'.

Related

Is `R_ClearExternalPtr()` needed to finalize an external pointer object in R?

While reading Writing R Extensions, Section 5.13 External pointers and weak references we see an example of a finalizer using R_ClearExternalPtr() albeit the comment next to it saying not really needed:
(...) The finalizer code is simply
static void chanFinalizer(SEXP ptr)
{
if(!R_ExternalPtrAddr(ptr)) return;
inRODBCClose(R_ExternalPtrAddr(ptr));
R_ClearExternalPtr(ptr); /* not really needed */
}
In the post Storing C objects in R by #Martin-Morgan he provides two finalizer examples, where he does include a call to R_ClearExternalPtr() in the first case but not in the second:
First example:
static void
_finalizer(SEXP ext)
{
if (NULL == R_ExternalPtrAddr(ext))
return;
Rprintf("finalizing\n");
char *ptr = (char *) R_ExternalPtrAddr(ext);
Free(ptr);
R_ClearExternalPtr(ext);
}
Second example:
static void
_finalizer(SEXP ext)
{
struct Foo *ptr = (struct Foo*) R_ExternalPtrAddr(ext);
Free(ptr);
}
Can someone help clarify the comment not really needed, and whether I should call R_ClearExternalPtr() at the end of the finalizer.

Recursive struct definitions in Solidity (Creating Linked Lists)

I'm trained mostly in Java so this has never really been an issue for me, but I'm trying to create a linked list in Solidity for a smart contract I am working on. In it, I am using a struct written as such:
struct Node {
address addr;
Node _next;
Node prev;
}
While I'm still working out the kinks on whether it's worth it to keep it as a single or doubly linked list, the main issue is that I have a recursive struct definition problem by referencing a Node struct inside my Node struct. I am a little unsure of what my alternatives are. Is this just a limitation of the language? I read this thread, but I couldn't really tell if the answer given was anything more than "it depends".
Thanks for any feedback; let me know if I can provide more information.
You could do it like this (double linked list):
pragma solidity ^0.4.24;
contract LinkedList {
struct Node {
string data;
uint pointPrevNode;
uint pointNextNode;
}
mapping(uint => Node) public nodes;
uint public nodeNumber = 0;
uint public count = 1;
function setData(string _data) public {
nodes[count].data = _data;
assignPointers(count);
}
function assignPointers(uint _count) internal {
nodes[nodeNumber].pointNextNode = _count;
nodes[_count].pointPrevNode = nodeNumber;
nodeNumber = _count;
count++;
}
}

Why is the HashMap transient inside HashSet implementation?

HashSet class itself implements Serializable, but the Hashmap it uses to store all of it's data is kept transient. Wouldn't all the data be lost on deserializing it? What's the point of serializing it then?
A HashSet serializes as a basic set. Upon deserialization, a new HashMap is constructed to hold the data. There's no need to serialize the map, with key-value pairs, when all you care about is the keys.
First, you are right ,transient hashmap in hashset will not be serialized, but hashset store data in its field hashmap, you think when do serialize, hashset may lost it's data, by read HashSet source code you will find there two methods called writeObject and readObject in HashSet, when do serializes, will invoke writeObject method in hashset to store data,and when do deserialize,will invoke readObject method in hashset to restore data,so serialize it's field hashmap is not necessary
private void writeObject(java.io.ObjectOutputStream s)
throws java.io.IOException {
// Write out any hidden serialization magic
s.defaultWriteObject();
// Write out HashMap capacity and load factor
s.writeInt(map.capacity());
s.writeFloat(map.loadFactor());
// Write out size
s.writeInt(map.size());
// Write out all elements in the proper order.
for (E e : map.keySet())
s.writeObject(e);
}
private void readObject(java.io.ObjectInputStream s)
throws java.io.IOException, ClassNotFoundException {
// Read in any hidden serialization magic
s.defaultReadObject();
// Read capacity and verify non-negative.
int capacity = s.readInt();
if (capacity < 0) {
throw new InvalidObjectException("Illegal capacity: " +
capacity);
}
// Read load factor and verify positive and non NaN.
float loadFactor = s.readFloat();
if (loadFactor <= 0 || Float.isNaN(loadFactor)) {
throw new InvalidObjectException("Illegal load factor: " +
loadFactor);
}
// Read size and verify non-negative.
int size = s.readInt();
if (size < 0) {
throw new InvalidObjectException("Illegal size: " +
size);
}
// Set the capacity according to the size and load factor ensuring that
// the HashMap is at least 25% full but clamping to maximum capacity.
capacity = (int) Math.min(size * Math.min(1 / loadFactor, 4.0f),
HashMap.MAXIMUM_CAPACITY);
// Constructing the backing map will lazily create an array when the first element is
// added, so check it before construction. Call HashMap.tableSizeFor to compute the
// actual allocation size. Check Map.Entry[].class since it's the nearest public type to
// what is actually created.
SharedSecrets.getJavaOISAccess()
.checkArray(s, Map.Entry[].class, HashMap.tableSizeFor(capacity));
// Create backing HashMap
map = (((HashSet<?>)this) instanceof LinkedHashSet ?
new LinkedHashMap<E,Object>(capacity, loadFactor) :
new HashMap<E,Object>(capacity, loadFactor));
// Read in all elements in the proper order.
for (int i=0; i<size; i++) {
#SuppressWarnings("unchecked")
E e = (E) s.readObject();
map.put(e, PRESENT);
}
}

passing a HANDLE variable to an unmanaged .dll in C++/CLI

I am trying to wrap an unmanaged c++ dll that talks to a video capture card in c++/CLI so i can reference the functions from a c# project that i have. I am having trouble getting the 1st wrapped call to work as I am new to c++/cli syntax. here is what i have.
here is the function declataion i am trying to wrap.
__declspec(dllimport) BOOL AZ_DeviceCreate(HANDLE& hLiveEvent, DWORD* hEncoderEvent, DWORD* pdwEncoderAddress, HANDLE& hAudioEvent, DWORD& dwAudioAddress);
here is my c++/cli .h file
namespace CaptureLibrary
{
public ref class CaptureCard
{
public:
HANDLE m_hLiveEvent;
DWORD *m_hEncoderEvent;
HANDLE m_hAudioEvent;
public:
CaptureCard();
bool CreateDevice();
void DisposeDevice();
};
}
and my .cpp
namespace CaptureLibrary
{
CaptureCard::CaptureCard()
{
m_hLiveEvent = INVALID_HANDLE_VALUE;
m_hEncoderEvent = new DWORD[MAX_VIDEO_CHANNEL];
for (BYTE i=0;i<MAX_VIDEO_CHANNEL;i++)
{
m_hEncoderEvent[i] = (DWORD)INVALID_HANDLE_VALUE;
}
m_hAudioEvent = INVALID_HANDLE_VALUE;
}
bool CaptureCard::CreateDevice()
{
DWORD dwEncoderBuff[MAX_VIDEO_CHANNEL];
DWORD dwACaptureBuffer = 0;
if(AZ_DeviceCreate(m_hLiveEvent, m_hEncoderEvent, dwEncoderBuff, m_hAudioEvent, dwACaptureBuffer)==FALSE)
{
return false;
}
return true;
}
void CaptureCard::DisposeDevice()
{
AZ_DeviceClose();
}
}
when i compile this with the required headers, i get this error:
error C2664: 'AZ_DeviceCreate' : cannot convert parameter 1 from 'HANDLE' to 'HANDLE &'
Can anyone help me as I know this is a stupid syntax thing that I am doing wrong.
Thanks in advance.
I mean this constructively: you're off on the wrong foot. Your goal with C++/CLI here is to wrap the unmanaged library in a manner that won't seem foreign in .NET, but your CaptureCard class doesn't do that.
Don't expose fields, expose properties (I assume they should be get-only for CaptureCard's members)
Don't expose raw pointer types (e.g. HANDLE), expose IntPtr
Don't expose raw C-arrays (e.g. DWORD*), expose array<T>^, ReadOnlyCollection<T>^, or IEnumerable<T>^ (but don't expose array<T>^s intended to be read-only via properties, only via methods + Array::Copy)
Don't only expose a DisposeDevice method, also make the class actually implement IDisposable so the device can be closed with a using statement rather than forcing use of try..finally
As the class controls unmanaged resources, it needs a finalizer
.h:
namespace CaptureLibrary
{
public ref class CaptureCard sealed
{
public:
CaptureCard();
~CaptureCard();
!CaptureCard();
property IntPtr LiveEvent { IntPtr get(); }
property IEnumerable<DWORD>^ EncoderEvent { IEnumerable<DWORD>^ get(); }
property IntPtr AudioEvent { IntPtr get(); }
bool CreateDevice();
void DisposeDevice();
private:
bool m_bOpened;
IntPtr m_hLiveEvent;
array<DWORD>^ m_hEncoderEvent;
IntPtr m_hAudioEvent;
};
}
.cpp:
namespace CaptureLibrary
{
CaptureCard::CaptureCard()
: m_hLiveEvent(INVALID_HANDLE_VALUE),
m_hEncoderEvent(gcnew array<DWORD>(MAX_VIDEO_CHANNEL)),
m_hAudioEvent(INVALID_HANDLE_VALUE)
{
for (int i = 0, i_max = m_hEncoderEvent->Length; i != i_max; ++i)
m_hEncoderEvent[i] = reinterpret_cast<DWORD>(INVALID_HANDLE_VALUE);
}
CaptureCard::~CaptureCard()
{
this->!CaptureCard();
}
CaptureCard::!CaptureCard()
{
DisposeDevice();
}
IntPtr CaptureCard::LiveEvent::get()
{
return m_hLiveEvent;
}
IEnumerable<DWORD>^ CaptureCard::EncoderEvent::get()
{
return m_hEncoderEvent;
}
IntPtr CaptureCard::AudioEvent::get()
{
return m_hAudioEvent;
}
bool CaptureCard::CreateDevice()
{
DisposeDevice();
DWORD dwAudioAddress = 0u;
DWORD dwEncoderAddress[MAX_VIDEO_CHANNEL];
HANDLE hLiveEvent = m_hLiveEvent.ToPointer();
HANDLE hAudioEvent = m_hAudioEvent.ToPointer();
{
pin_ptr<DWORD> hEncoderEvent = &m_hEncoderEvent[0];
m_bOpened = AZ_DeviceCreate(hLiveEvent, hEncoderEvent, dwEncoderAddress, hAudioEvent, dwAudioAddress) == TRUE;
}
m_hLiveEvent = IntPtr(hLiveEvent);
m_hAudioEvent = IntPtr(hAudioEvent);
return m_bOpened;
}
void CaptureCard::DisposeDevice()
{
if (m_bOpened)
{
AZ_DeviceClose();
m_bOpened = false;
}
}
}
Suggestions for further improvement:
Get rid of CreateDevice and DisposeDevice altogether. This code has a very C-ish mentality; .NET users would expect a constructed object to have a meaningful value without calling a separate initialization function, so assuming AZ_DeviceCreate is not expected to fail regularly then CreateDevice's logic should go straight in the class' constructor and an exception should be thrown upon failure
If calling AZ_DeviceClose multiple times is harmless then get rid of m_bOpened altogether
The problem here is that you are trying to pass m_hLiveHandle as a reference (i.e. HANDLE &), but this would require that m_hLiveHandle could be pointed to by a native pointer (i.e. it would be guaranteed not to move in memory). However, m_hLiveHandle is a member of a ref class (CaptureCard) which means instances of it are stored on the managed heap. This in turn means that the instance of CaptureCard can be moved in memory (by a garbage collection action). So, if you want to use m_hLiveHandle as a pointer parameter or reference parameter, you'd have to use pin_ptr to tell the garbage collector not to move this object during the duration of the call to the native method. Read up here for more:
http://msdn.microsoft.com/en-us/library/1dz8byfh(v=vs.80).aspx

How to deal with Number precision in Actionscript?

I have BigDecimal objects serialized with BlazeDS to Actionscript. Once they hit Actionscript as Number objects, they have values like:
140475.32 turns into 140475.31999999999998
How do I deal with this? The problem is that if I use a NumberFormatter with precision of 2, then the value is truncated to 140475.31. Any ideas?
This is my generic solution for the problem (I have blogged about this here):
var toFixed:Function = function(number:Number, factor:int) {
return Math.round(number * factor)/factor;
}
For example:
trace(toFixed(0.12345678, 10)); //0.1
Multiply 0.12345678 by 10; that gives us 1.2345678.
When we round 1.2345678, we get 1.0,
and finally, 1.0 divided by 10 equals 0.1.
Another example:
trace(toFixed(1.7302394309234435, 10000)); //1.7302
Multiply 1.7302394309234435 by 10000; that gives us 17302.394309234435.
When we round 17302.394309234435 we get 17302,
and finally, 17302 divided by 10000 equals 1.7302.
Edit
Based on the anonymous answer below, there is a nice simplification for the parameter on the method that makes the precision much more intuitive. e.g:
var setPrecision:Function = function(number:Number, precision:int) {
precision = Math.pow(10, precision);
return Math.round(number * precision)/precision;
}
var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9
trace(setPrecision(number,2)); //Result is 10.98
trace(setPrecision(number,3)); //Result is 10.988 and so on
N.B. I added this here just in case anyone sees this as the answer and doesn't scroll down...
Just a slight variation on Frasers Function, for anyone who is interested.
function setPrecision(number:Number, precision:int) {
precision = Math.pow(10, precision);
return (Math.round(number * precision)/precision);
}
So to use:
var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9
trace(setPrecision(number,2)); //Result is 10.98
trace(setPrecision(number,3)); //Result is 10.988 and so on
i've used Number.toFixed(precision) in ActionScript 3 to do this: http://livedocs.adobe.com/flex/3/langref/Number.html#toFixed%28%29
it handles rounding properly and specifies the number of digits after the decimal to display - unlike Number.toPrecision() that limits the total number of digits to display regardless of the position of the decimal.
var roundDown:Number = 1.434;
// will print 1.43
trace(roundDown.toFixed(2));
var roundUp:Number = 1.436;
// will print 1.44
trace(roundUp.toFixed(2));
I converted the Java of BigDecimal to ActionScript.
We had no choices since we compute for financial application.
http://code.google.com/p/bigdecimal/
You can use property: rounding = "nearest"
In NumberFormatter, rounding have 4 values which you can choice: rounding="none|up|down|nearest". I think with your situation, you can chose rounding = "nearest".
-- chary --
I discovered that BlazeDS supports serializing Java BigDecimal objects to ActionScript Strings as well. So if you don't need the ActionScript data to be Numbers (you are not doing any math on the Flex / ActionScript side) then the String mapping works well (no rounding weirdness). See this link for the BlazeDS mapping options: http://livedocs.adobe.com/blazeds/1/blazeds_devguide/help.html?content=serialize_data_2.html
GraniteDS 2.2 has BigDecimal, BigInteger and Long implementations in ActionScript3, serialization options between Java / Flex for these types, and even code generation tools options in order to generate AS3 big numbers variables for the corresponding Java ones.
See more here: http://www.graniteds.org/confluence/display/DOC22/2.+Big+Number+Implementations.
guys, just check the solution:
protected function button1_clickHandler(event:MouseEvent):void
{
var formatter:NumberFormatter = new NumberFormatter();
formatter.precision = 2;
formatter.rounding = NumberBaseRoundType.NEAREST;
var a:Number = 14.31999999999998;
trace(formatter.format(a)); //14.32
}
I ported the IBM ICU implementation of BigDecimal for the Actionscript client. Someone else has published their nearly identical version here as a google code project. Our version adds some convenience methods for doing comparisons.
You can extend the Blaze AMF endpoint to add serialization support for BigDecimal. Please note that the code in the other answer seems incomplete, and in our experience it fails to work in production.
AMF3 assumes that duplicate objects, traits and strings are sent by reference. The object reference tables need to be kept in sync while serializing, or the client will loose sync of these tables during deserialization and start throwing class cast errors, or corrupting the data in fields that don't match, but cast ok...
Here is the corrected code:
public void writeObject(final Object o) throws IOException {
if (o instanceof BigDecimal) {
write(kObjectType);
if(!byReference(o)){ // if not previously sent
String s = ((BigDecimal)o).toString();
TraitsInfo ti = new TraitsInfo("java.math.BigDecimal",false,true,0);
writeObjectTraits(ti); // will send traits by reference
writeUTF(s);
writeObjectEnd(); // for your AmfTrace to be correctly indented
}
} else {
super.writeObject(o);
}
}
There is another way to send a typed object, which does not require Externalizable on the client. The client will set the textValue property on the object instead:
TraitsInfo ti = new TraitsInfo("java.math.BigDecimal",false,false,1);
ti.addProperty("textValue");
writeObjectTraits(ti);
writeObjectProperty("textValue",s);
In either case, your Actionscript class will need this tag:
[RemoteClass(alias="java.math.BigDecimal")]
The Actionscript class also needs a text property to match the one you chose to send that will initialize the BigDecimal value, or in the case of the Externalizable object, a couple of methods like this:
public function writeExternal(output:IDataOutput):void {
output.writeUTF(this.toString());
}
public function readExternal(input:IDataInput):void {
var s:String = input.readUTF();
setValueFromString(s);
}
This code only concerns data going from server to client. To deserialize in the other direction from client to server, we chose to extend AbstractProxy, and use a wrapper class to temporarily store the string value of the BigDecimal before the actual object is created, due to the fact that you cannot instantiate a BigDecimal and then assign the value, as the design of Blaze/LCDS expects should be the case with all objects.
Here's the proxy object to circumvent the default handling:
public class BigNumberProxy extends AbstractProxy {
public BigNumberProxy() {
this(null);
}
public BigNumberProxy(Object defaultInstance) {
super(defaultInstance);
this.setExternalizable(true);
if (defaultInstance != null)
alias = getClassName(defaultInstance);
}
protected String getClassName(Object instance) {
return((BigNumberWrapper)instance).getClassName();
}
public Object createInstance(String className) {
BigNumberWrapper w = new BigNumberWrapper();
w.setClassName(className);
return w;
}
public Object instanceComplete(Object instance) {
String desiredClassName = ((BigNumberWrapper)instance).getClassName();
if(desiredClassName.equals("java.math.BigDecimal"))
return new BigDecimal(((BigNumberWrapper)instance).stringValue);
return null;
}
public String getAlias(Object instance) {
return((BigNumberWrapper)instance).getClassName();
}
}
This statement will have to execute somewhere in your application, to tie the proxy object to the class you want to control. We use a static method:
PropertyProxyRegistry.getRegistry().register(
java.math.BigDecimal.class, new BigNumberProxy());
Our wrapper class looks like this:
public class BigNumberWrapper implements Externalizable {
String stringValue;
String className;
public void readExternal(ObjectInput arg0) throws IOException, ClassNotFoundException {
stringValue = arg0.readUTF();
}
public void writeExternal(ObjectOutput arg0) throws IOException {
arg0.writeUTF(stringValue);
}
public String getStringValue() {
return stringValue;
}
public void setStringValue(String stringValue) {
this.stringValue = stringValue;
}
public String getClassName() {
return className;
}
public void setClassName(String className) {
this.className = className;
}
}
We were able to reuse one of the available BigDecimal.as classes on the web and extended blazeds by sublassing from AMF3Output, you'll need to specify your own endpoint class in the flex xml files, in that custom endpoint you can insert your own serializer that instantiates an AMF3Output subclass.
public class EnhancedAMF3Output extends Amf3Output {
public EnhancedAMF3Output(final SerializationContext context) {
super(context);
}
public void writeObject(final Object o) throws IOException {
if (o instanceof BigDecimal) {
write(kObjectType);
writeUInt29(7); // write U290-traits-ext (first 3 bits set)
writeStringWithoutType("java.math.BigDecimal");
writeAMFString(((BigDecimal)o).toString());
} else {
super.writeObject(o);
}
}
}
as simple as that! then you have native BigDecimal support using blazeds, wooohoo!
Make sure your BigDecimal as3 class implements IExternalizable
cheers, jb
Surprisingly the round function in MS Excel gives us different values then you have presented above.
For example in Excel
Round(143,355;2) = 143,36
So my workaround for Excel round is like:
public function setPrecision(number:Number, precision:int):Number {
precision = Math.pow(10, precision);
const excelFactor : Number = 0.00000001;
number += excelFactor;
return (Math.round(number * precision)/precision);
}
If you know the precision you need beforehand, you could store the numbers scaled so that the smallest amount you need is a whole value. For example, store the numbers as cents rather than dollars.
If that's not an option, how about something like this:
function printTwoDecimals(x)
{
printWithNoDecimals(x);
print(".");
var scaled = Math.round(x * 100);
printWithNoDecimals(scaled % 100);
}
(With however you print with no decimals stuck in there.)
This won't work for really big numbers, though, because you can still lose precision.
You may vote and watch the enhancement request in the Flash PLayer Jira bug tracking system at https://bugs.adobe.com/jira/browse/FP-3315
And meanwhile use the Number.toFixed() work-around see :
(http://livedocs.adobe.com/flex/3/langref/Number.html#toFixed%28%29)
or use the open source implementations out there : (http://code.google.com/p/bigdecimal/) or (http://www.fxcomps.com/money.html)
As for the serialization efforts, well, it will be small if you use Blazeds or LCDS as they do support Java BigDecimal serialization (to String) cf. (http://livedocs.adobe.com/livecycle/es/sdkHelp/programmer/lcds/wwhelp/wwhimpl/common/html/wwhelp.htm?context=LiveDocs_Parts&file=serialize_data_3.html)
It seems more like a transport problem, the number being correct but the scale ignored. If the number has to be stored as a BigDecimal on the server you may want to convert it server side to a less ambiguous format (Number, Double, Float) before sending it.

Resources