I need consume from kafka same topic with different types. The content of different types are slightly. But they are different. Inside payload of the message, there is one field named type which could determine which javatype it should be deserialized to.
It a little bit difficult for me because to fetch field value of 'type'. I need do a deserialize already.
Is there good way to do this with spring kafka?
Thanks in advance.
As Artem Bilan pointed out in the comments, it'd be easier to help if you added more details to your question. I'll try to help by making some assumptions.
When using Avro, deserialization of different payload types should be handled (and enforced) by the schema-registry first, such as documented here, and this doesn't seem to be compatible with having type information inside the payload, so I'll assume you're talking about JSON payloads.
If possible, you should probably change the design to have that kind of metadata in the record headers - this way you don't need to deserialize the payload in order to access it, and can use Spring Kafka built in tools such as DelegatingDeserializer.
Considering both initial assumptions are right - you're using JSON and can't change the design to use headers for type metadata, you should be able to implement a custom JsonTypeResolver to be set on the JsonDeserializer, where you can use an ObjectMapper to deserialize the payload to, for example, a Map - you can then fetch the type information from there and associate it with the proper JavaType.
EDIT: A more performant solution would be to implement a custom deserializer where you can use the ‘readTree’ method in ‘ObjectMapper’ to deserialize the payload to a ‘JsonNode’. You can then traverse this object to fetch type information, and use the ‘treeToValue’ method to continue deserializing the json node to the proper type. That way you’re only deserializing it once.
Related
I have a component that acts as a middle man between 2 web services. Both of them communicate using JSON.
The data that goes back and forth from the web services is very similar. However, it does need to be massaged a little.
I currently have this working by deserializing the JSON, build a new JObject and serialize it. It seems like there should be better way.
I'm looking at JsonConvert, JsonConverter, JsonSerializer, JsonReader, etc. trying to see if there's a better way to do this.
Any guidance on what classes to use/override to make this process more efficent?
Thanks!
You can write a custom JsonConverter using the approach described in the documentation, so you'll only have one concrete class, but this class can translate to a slightly different JSON representation of your object.
Another, more verbose, blog post about writing custom JsonConverters can be found here: http://blog.maskalik.com/asp-net/json-net-implement-custom-serialization/
I'm migrating a legacy PHP project (pre-OO) to Symfony2. On every request I have to:
compute some dynamic data (depending on the current date and/or some request parameter)
use that data (multiple times!) in the rendered response.
A naive approach would be:
At the start of every controller method, call some global helper function to compute the data.
At the end of every controller method, pass the data as a parameter to the twig template.
Sounds tedious. Maybe it would be better to:
Create a subscriber for request events that computes the data when a request comes in and provides access to it via getter methods.
Define that subscriber/service as a global twig variable in config.yml.
In twig templates, call the getter methods on that service as needed.
Is that viable?
In particular, are the twig variable/service and the subscriber always identical? Or could the service be a newly created instance?
Is this some sort of misuse? Or is there an officially recommended way for such a use case?
EDIT The data is not only needed in every twig template but in some controllers, too.
Calling a specific method in every Controller-Action would really be a bad solution. Your solution to use a subscriber isn't perfect either.
Without knowing your use-case in detail it´s hard to find a suitable way.
Maybe one approach would be to write a Twig-Extension and injecting a Service into this Extension. The Service would get the Request-Stack via Dependency-Injection and compute the relevant Data. You could then access this data "on demand" through the Twig-Extension during the rendering.
Another approach could be using sub-requests during the rendering (Symfony: How to handle common request scope data)
Maybe these tips already helped you a bit. Otherwise let me know more details, where / how you need the data during the rendering.
I am using webApi 2 for my application. i have gallery and i need a way to show my images. I've searched a lot and found some articles like these :
http://www.dotnetcurry.com/aspnet/1120/aspnet-webapi-binary-contents-images
https://jamessdixon.wordpress.com/2013/10/01/handling-images-in-webapi/
but they seems they don't have a efficient way.
so I decide to ask this question once again. how can I handle my images in web api 2 ?
what's the best approach?
Without a little more info on your use case, it's hard to know what the best way would be. Typically a formatter would be used. Then your client would make two requests, /api/Images to get all your images, then create the proper views for each image (HTML would be ). A route that accepts an extension is needed and a formatter to return the proper content type and binary data. Your controller could do all the binary streaming, but a formatter allows better flexibility.
As for BSON, I have never used BSON. Our API has so many different clients that JSON is universal for our DTO's. For images, we always use the the response body for the pure binary stream. This way there is no custom parsing that each client has to do. We can rely completely on the HTTP specs by using the proper content-type, content-length, etc. This also allows for a range request so we can stream and do partial downloads with resume. I have no doubt BSON will catch on and we'll probably use it in the future, but for now it doesn't fit our use case.
We're developing a business ASP.NET application. Is it better to pass an entire entity to a method or pass each property of this entity as parameters? What is the best practice?
Case 1. Pass Customer entity to a manager - InsertCustomer(Customer cust)
Case 2. Pass each property as a parameter - InsertCustomer(string name, string address...etc)
P.S. We're using Entity Framework as our data access layer
Pass the entire entity, not only for reasons given in the other answers, but generally methods with long parameter chains are bad. They are prone to error, and tough to work with from a development standpoint (just look at Interop with Office)
In general, if I see I am getting too many parameters (usually more than three), either I have a method trying to do too much, or I explore ways of encapsulating this data in a struct.
You should pass the entire entity as when you update the entity, e.g. add or remove members you do not have to update all your method calls in all your layers. You only need to change your datalayer and the layer where you are consuming the entity. asp.net is Object Oriented and therefore you should orientate your code around your objects
The whole concept of object orientation requires objects to be passed around. If all is happening internally I would go with this.
If this is being posted to a webservice / across a network etc you would need to serialize, and hence may find it better to pass each individual parameter, especially if the receiving framework is different.
Don't forget your Strings etc are all objects too.
I agree with another poster, passing a whole entity "encapsulates" everything so that it can be updated/modified so you have less to worry about.
Our Flex app would like to work with requests and responses as object graphs (nothing unusual there) e.g. response becomes the model of some view, and would be a structure with several layers of nesting.
** Now, ideally we would like to use the same client (and server) side objects for different message formats e.g. XML and AMF, and have a pluggable serialization/de-serialization layer (!)
AMF has serialization and matching of client to server using
[RemoteClass(alias="samples.contact.Contact")]
but it seems there is no equivalent for XML.
I am (somewhat optimistically) looking for a neat way of serializing the object graph to XML, to send through a HTTPService from the client.
For responses, the default 'object' and 'E4X' provide some de-serialization. This is handy, but of course we don't have the niceties of unpacking the XML back into specific AS classes like we do with AMF.
Any suggestions?
(did have one idea come through about wrapping/casting object as XML or XMLList - this does not seem to work, however)
Update:
Both these libraries look useful, and I will very likely use them at some point.
For now, I really need the simplicity of re-using the metadata set for the AMF3 serialization which we are using in any case ([RemoteClass],[Transient])
.. so the best option at the moment is AMFX - used Flex Data Services for AMF transfer using XML - classes in mx.messaging.channels.amfx package - only drawback at the moment is any Externalizable class is transformed into a Hex byte stream - and ArrayCollection is Externalizable! (hoping to workaround by serializing the internal Array in a subclass ..)
Hope that's useful to someone ..
Regarding the Xml serialization I can give you a starting point (as biased as it may be, though :D).
I am working on a project that allows for automatic conversion of AS3 objects to and from xml. It basically uses annotations on the model objects you use for communication in order to construct the xml structure or populating an object from xml.
It is called FlexXB and you can check it out at http://code.google.com/p/flexxb/.
I started this project cos I got into the same issues at work (namely I have a server that communicates through xml) and I hoped it be of use to someone else.
Cheers,
Alex
Yet another project: FleXMLer (http://code.google.com/p/flexmler/).
It has both the straightforward attitude of asx3m where you can just call:
new FleXMLer().serialize(obj);
Or you can customize XML element names, skip elements and tweak the way arrays and hash tables are serialized.
Would appreciate your input.
checkout asx3m project at http://code.google.com/p/asx3m
It's an AS3 port of Java XStream serialization library and works pretty well.
I made it because I had to connect to a server platform that used XStream for exchanging data objects and put a lot of work in it.
It can be extended to serialize AS3 objects to any format (JSON for example) and could leverage power of user defined metatags.
Cheers,
Tomislav
There's a library including JSON available from Adobe, too. And since ActionScript is a superset of JavaScript ... and JSON is increasingly supported cross-framework ...