Example Json serialization and deserialization with spring-boot:
#Configuration
class ConsumerKafkaConfiguration {
#Bean
fun defaultKafkaConsumerFactoryCustomizer() = DefaultKafkaConsumerFactoryCustomizer {
it.keyDeserializer = StringDeserializer()
it.valueDeserializer = ByteArrayDeserializer()
}
#Bean
fun messageConvertor(): RecordMessageConverter {
val converter = StringJsonMessageConverter()
val typeMapper = DefaultJackson2JavaTypeMapper()
typeMapper.typePrecedence = Jackson2JavaTypeMapper.TypePrecedence.TYPE_ID
typeMapper.addTrustedPackages(ResolverKafkaEvents.PACKAGE_GEN_EVENTS)
typeMapper.idClassMapping = ResolverKafkaEvents
.resolveEvents
.mapValues { Class.forName(it.value) }
converter.typeMapper = typeMapper
return converter
}
}
#Configuration
class ProducerKafkaConfiguration {
#Bean
fun defaultKafkaProducerFactoryCustomizer() = DefaultKafkaProducerFactoryCustomizer { factory ->
factory.keySerializer = StringSerializer()
factory.valueSerializer = JsonSerializer<Any>().apply {
configure(
mutableMapOf(
JsonSerializer.TYPE_MAPPINGS to ResolverKafkaEvents
.resolveEvents
.asSequence()
.joinToString(", ") { "${it.key}:${it.value}" }
),
false,
)
}
}
#Bean
fun customAsyncProducer(
kafkaTemplate: KafkaTemplate<String, Any>
) = KafkaJsonProducer(kafkaTemplate)
}
I know how to serialize into an byte array and accept, parse it every time in all #KafkaListener. Duplicating the code. Is it possible to make a more elegant solution?
Related
There is a quite simple case I would like to implement:
I have a base and DLT topics:
MessageBus:
Topic: my_topic
DltTopic: my_dlt_topic
Broker: event-serv:9092
So, those topics are already predefined, I don't need to create them automatically.
The only I need to handle broken messages automatically without retries, because they don't make any sense, so I have something like this:
#KafkaListener(topics = ["#{config.messageBus.topic}"], groupId = "group_id")
#RetryableTopic(
dltStrategy = DltStrategy.FAIL_ON_ERROR,
autoCreateTopics = "false",
attempts = "1"
)
#Throws(IOException::class)
fun consume(rawMessage: String?) {
...
}
#DltHandler
fun processMessage(rawMessage: String?) {
kafkaTemplate.send(config.messageBus.dltTopic, rawMessage)
}
That of course doesn't work properly.
I also tried to specify a kafkaTemplate
#Bean
fun kafkaTemplate(
config: Config,
producerFactory: ProducerFactory<String, String>
): KafkaTemplate<String, String> {
val template = KafkaTemplate(producerFactory)
template.defaultTopic = config.messageBus.dltTopic
return template
}
however, that does not change the situation.
In the end, I believe there is an obvious solution, so I please give me a hint about it.
See the documenation.
#SpringBootApplication
public class So69317126Application {
public static void main(String[] args) {
SpringApplication.run(So69317126Application.class, args);
}
#RetryableTopic(attempts = "1", autoCreateTopics = "false", dltStrategy = DltStrategy.FAIL_ON_ERROR)
#KafkaListener(id = "so69317126", topics = "so69317126")
void listen(String in) {
System.out.println(in);
throw new RuntimeException();
}
#DltHandler
void handler(String in) {
System.out.println("DLT: " + in);
}
#Bean
RetryTopicNamesProviderFactory namer() {
return new RetryTopicNamesProviderFactory() {
#Override
public RetryTopicNamesProvider createRetryTopicNamesProvider(Properties properties) {
if (properties.isMainEndpoint()) {
return new SuffixingRetryTopicNamesProviderFactory.SuffixingRetryTopicNamesProvider(properties) {
#Override
public String getTopicName(String topic) {
return "so69317126";
}
};
}
else if(properties.isDltTopic()) {
return new SuffixingRetryTopicNamesProviderFactory.SuffixingRetryTopicNamesProvider(properties) {
#Override
public String getTopicName(String topic) {
return "so69317126.DLT";
}
};
}
else {
throw new IllegalStateException("Shouldn't get here - attempts is only 1");
}
}
};
}
}
so69317126: partitions assigned: [so69317126-0]
so69317126-dlt: partitions assigned: [so69317126.DLT-0]
foo
DLT: foo
This is a Kafka server configuration so you must set it on the server. The relevant property is:
auto.create.topics.enable (true by default)
I create retrofitconfig for get data from API in first time I use it works and good but in 2 days data not showing from Api
this is my interface
interface CnnApi {
#GET("cnbc-news/")
fun getNewsCNN(): Call<NewsResponse>
#GET("cnbc-news/{type}")
fun getNewsCCNType(#Path("type") type: String ) : Call<NewsResponse>
}
and then this is object I use to make retrofit
object RetrofitConfig {
fun getData(): CnnApi {
val loggingInterceptor =
HttpLoggingInterceptor().setLevel(HttpLoggingInterceptor.Level.BODY)
val client = OkHttpClient.Builder()
.addInterceptor(loggingInterceptor)
.build()
val retrofit = Retrofit.Builder()
.addConverterFactory(GsonConverterFactory.create())
.baseUrl("https://berita-indo-api.vercel.app/v1/")
.client(client)
.build()
return retrofit.create(CnnApi::class.java)
}
}
and then this is my HomeViewModel
class HomeViewModel : ViewModel() {
private val _isLoading = MutableLiveData<Boolean>()
val isLoading: LiveData<Boolean> = _isLoading
private val _dataItem = MutableLiveData<List<DataItem>>()
val dataItem: LiveData<List<DataItem>> = _dataItem
private val _image = MutableLiveData<Image>()
val image:LiveData<Image> = _image
companion object{
private const val TAG = "News"
private const val type ="lifestyle"
}
init{
getDataFromAPI()
}
private fun getDataFromAPI(){
EspressoIdlingResource.increment()
_isLoading.value = true
val client = RetrofitConfig.getData().getNewsCCNType(type)
client.enqueue(object: Callback<NewsResponse>{
override fun onResponse(call: Call<NewsResponse>, response: Response<NewsResponse>) {
_isLoading.value = false
if(response.isSuccessful || response.body()?.total== 200){
_dataItem.value = response.body()?.data
}else{
Log.e(TAG,"onFailure: ${response.message()}")
}
}
override fun onFailure(call: Call<NewsResponse>, t: Throwable) {
_isLoading.value = false
Log.e(TAG, "onFailure: ${t.message.toString()}")
EspressoIdlingResource.decrement()
}
})
}
}
This is the view or how I show the data
homeViewModel =
ViewModelProvider(this).get(HomeViewModel::class.java)
_binding = FragmentHomeBinding.inflate(inflater, container, false)
val root: View = binding.root
homeViewModel.dataItem.observe(viewLifecycleOwner, { news ->
binding.articleTitle.text = news[0].title
Glide.with(this)
.load(news[0].image.small)
.into(binding.articleImage)
binding.articleFrom.text = "CNN Indonesia"
link = news[0].link
})
binding.imageCard.setOnClickListener {
val intent = Intent(Intent.ACTION_VIEW)
intent.data = Uri.parse(link)
startActivity(intent)
}
Logcat or error
logcat is what I think makes the application so it doesn't display the desired data, how to solve must-revalidate data ? because I think because of it I cant show the data from API
I have one activity with 2 recyclerViews, both recyclers are used with Firebase. One of the recyclers displays the results of a query, the other recycler has a listener that updates every time there is an update in Firebase. I added a searchView to filter the results from Firebase. The issue I'm having is when I'm trying to search the results from the Firebase query, when I start typing I see results but when I click the X to stop searching, the adapter does not reload the array and i don't see the list of items unless I reload the activity. I'm not sure what I'm missing here. Any help/suggestion is greatly appreciated. Here is my code:
Adapter
import android.view.LayoutInflater
import android.view.View
import android.view.ViewGroup
import android.widget.Filter
import android.widget.Filterable
import android.widget.TextView
import androidx.recyclerview.widget.RecyclerView
import com.eduardoz.ezmdapp.Model.Charges
import com.eduardoz.ezmdapp.R
class ChargesAdapter (private var charges: ArrayList<Charges>
, private var chargesAll: ArrayList<Charges>
, private val itemClick: (Charges) -> Unit)
: RecyclerView.Adapter<ChargesAdapter.ViewHolder>()
, Filterable {
inner class ViewHolder(itemView: View, val itemClick: (Charges) -> Unit) :
RecyclerView.ViewHolder(itemView) {
private val chargeCode = itemView.findViewById<TextView>(R.id.chargeCodeTxt)
private val chargeDescription = itemView.findViewById<TextView>(R.id.chargeDescriptionTxt)
fun bindCharges(charges: Charges) {
chargeCode?.text = charges.chargeCode
chargeDescription?.text = charges.chargeDescription
itemView.setOnClickListener { itemClick(charges) }
}
}
init {
this.charges = charges
chargesAll = java.util.ArrayList(charges)
}
override fun getItemCount(): Int {
return charges.count()
}
override fun onBindViewHolder(holder: ChargesAdapter.ViewHolder, position: Int) {
holder.bindCharges(charges[position])
}
override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): ChargesAdapter.ViewHolder {
val view = LayoutInflater.from(parent.context).inflate(R.layout.charges_list, parent, false)
return ViewHolder(view, itemClick)
}
override fun getFilter(): Filter {
return searchFilter
}
private val searchFilter: Filter = object: Filter() {
override fun performFiltering(constraint: CharSequence?): FilterResults {
val filteredList: ArrayList<Charges> = ArrayList()
if (constraint!!.isEmpty()) {
filteredList.addAll(chargesAll)
} else {
for(item in chargesAll) {
if
(item.chargeDescription.toLowerCase().contains(constraint.toString().toLowerCase())) {
filteredList.add(item)
}
}
}
val searchResults = FilterResults()
searchResults.values = filteredList
return searchResults
}
override fun publishResults(constraint: CharSequence?, results: FilterResults?) {
charges.clear()
charges.addAll(results!!.values as Collection<Charges>)
notifyDataSetChanged()
}
}
}
Activity
descriptionSearch.setOnQueryTextListener(object : SearchView.OnQueryTextListener {
override fun onQueryTextSubmit(query: String): Boolean {
return false
}
override fun onQueryTextChange(newText: String): Boolean {
if (newText.isNotEmpty()) {
searchViewBar(newText)
} else {
if (newText.isEmpty()) { //I ADDED THIS TO RELOAD THE ADAPTER
charges.clear()
chargeList()
}
}
return false
}
})
private fun searchViewBar(newText: String) {
chargesListener = chargesCollectionRef
.whereGreaterThanOrEqualTo(CHARGE_DESCRIPTION, newText)
.whereLessThanOrEqualTo(CHARGE_DESCRIPTION, newText+"z")
.addSnapshotListener(this) { snapshot, exception ->
if (exception != null) {
println("error")
}
if (snapshot != null) {
charges.clear()
parseData(snapshot)
}
}
}
fun parseData(snapshot: QuerySnapshot) {
for (document in snapshot.documents) {
val data = document.data
val chargeCode = data!![CHARGE_CODE] as String
val chargeDescription = data[CHARGE_DESCRIPTION] as String
val chargeSpecialty = data[CHARGE_SPECIALTY] as String
val newChargeList = Charges(chargeCode, chargeDescription, chargeSpecialty)
charges.add(newChargeList)
}
chargesFromAdapter.notifyDataSetChanged()
}
ive got some code that works very well with picocli:
#Command(name = "parse", sortOptions = false, description = "parse input files and write to database")
class CommandLineArgumentParser {
#Option(names = { "-h", "--help" }, usageHelp = true, description = "display this message")
private boolean helpRequested = false;
#Option(names = { "-s", "--startDate"}, description = "First day at which to parse data",
converter = GermanDateConverter.class, paramLabel = "dd.MM.yyyy")
public LocalDate start;
#Option(names = { "-e", "--endDate"}, description = "Last day (inclusive) at which to stop parsing",
converter = GermanDateConverter.class, paramLabel = "dd.MM.yyyy")
public LocalDate end;
private static class GermanDateConverter implements ITypeConverter<LocalDate> {
#Override
public LocalDate convert(String value) throws Exception {
LocalDate result = null;
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("dd.MM.yyyy");
result = LocalDate.parse(value, formatter);
if (result.getYear() < 1900) {
throw new IllegalArgumentException("year should be after 1900");
}
return result;
}
}
#SpringBootApplication
public class Application implements CommandLineRunner {
public void run(String... args) throws Exception {
CommandLineArgumentParser commandlineparser = new CommandLineArgumentParser();
CommandLine commandLine = new CommandLine(commandlineparser);
try {
commandLine.parseArgs(args);
} catch (MissingParameterException e) {
System.out.println(e);
System.out.println();
CommandLine.usage(CommandLineArgumentParser.class, System.out);
System.exit(1);
} catch (ParameterException e) {
System.out.println(e);
System.out.println();
CommandLine.usage(CommandLineArgumentParser.class, System.out);
System.exit(1);
}
if (commandLine.isUsageHelpRequested()) {
commandLine.usage(System.out);
return;
} else if (commandLine.isVersionHelpRequested()) {
commandLine.printVersionHelp(System.out);
return;
}
if (commandlineparser.start == null) {
log.warn("no start date specified, using: 01.01.2005");
commandlineparser.start = LocalDate.of(2005, 01, 01);
}
if (commandlineparser.end == null) {
LocalDate timePoint = LocalDate.now();
log.warn("no end date specified, using today: " + timePoint.toString());
commandlineparser.end = timePoint;
}
}
but i did not find a simple example that shows multiple commands used, for example this one:
https://github.com/remkop/picocli/blob/master/src/test/java/picocli/Demo.java
does not compile:
int exitCode = new CommandLine(new Demo()).execute(args);
The method execute(CommandLine, List<Object>) in the type CommandLine is not applicable for the arguments (String[])
could somebody please post a example on howto use multiple commands?
I suspect you’re using an older version of the library. The execute(String []) : int method was introduced in picocli 4.0.
Upgrading and using the execute method instead of the parseArgs method will allow you to remove a lot of boilerplate code from the application: handling requests for usage/version help and dealing with invalid input is done automatically with the execute method.
Your commands should implement Runnable or Callable, and this is where the business logic of each command lives.
Modifying your example and giving it a subcommand:
#Command(name = "parse", sortOptions = false,
mixinStandardHelpOptions = true, version = “1.0”,
description = "parse input files and write to database",
subcommands = MySubcommand.class)
class CommandLineArgumentParser implements Callable<Integer> {
#Option(names = { "-s", "--startDate"}, description = "First day at which to parse data",
converter = GermanDateConverter.class, paramLabel = "dd.MM.yyyy")
public LocalDate start;
#Option(names = { "-e", "--endDate"}, description = "Last day (inclusive) at which to stop parsing",
converter = GermanDateConverter.class, paramLabel = "dd.MM.yyyy")
public LocalDate end;
private static class GermanDateConverter implements ITypeConverter<LocalDate> {
#Override
public LocalDate convert(String value) throws Exception {
LocalDate result = null;
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("dd.MM.yyyy");
result = LocalDate.parse(value, formatter);
if (result.getYear() < 1900) {
throw new IllegalArgumentException("year should be after 1900");
}
return result;
}
}
#Override
public Integer call() {
if (start == null) {
log.warn("no start date specified, using: 01.01.2005");
start = LocalDate.of(2005, 01, 01);
}
if (end == null) {
LocalDate timePoint = LocalDate.now();
log.warn("no end date specified, using today: " + timePoint.toString());
end = timePoint;
}
// more business logic here ...
// add finally return an exit code
int exitCode = ok ? 0 : 1;
return exitCode;
}
}
#Command(name = "foo")
class MySubcommand implements Callable<Integer> {
#Override
public Integer call() {
System.out.println("hi");
return 0;
}
}
#SpringBootApplication
public class Application implements CommandLineRunner {
public void run(String... args) throws Exception {
int exitCode = new CommandLine(
CommandLineArgumentParser.class,
new picocli.spring.PicocliSpringFactory())
.execute(args);
System.exit(exitCode);
}
}
Note that when using Spring in combination with picocli subcommands, you need to call the CommandLine constructor with a picocli.spring.PicocliSpringFactory.
For a more complete Spring example, see the picocli-spring-boot-starter README.
I am attempting to write some Tests for a small project of mine but they seem to fail (I am starting with 1 test 'Create_Class')
I use the repository pattern and use Constructor Dependency Injection:
public KlasController() {
db = ObjectContextPerHttpRequest.Context;
KlasRepo = new KlasRepository(db);
LesRepo = new LesRepository(db);
OpdrachtRepo = new OpdrachtRepository(db);
}
//dependency injection constructor
public KlasController(IKlasRepository KlasRepo, ILesRepository LesRepo,
IOpdrachtRepository OpdrachtRepo) {
this.KlasRepo = KlasRepo;
this.LesRepo = LesRepo;
this.OpdrachtRepo = OpdrachtRepo;
}
here is my TestClass with testinitializer (which runs before every test) and the first test
[TestClass()]
public class KlasControllerTest
{
private KlasController Controller;
private IOpdrachtRepository OpdrachtRepo;
//Use TestInitialize to run code before running each test
[TestInitialize()]
public void MyTestInitialize()
{
OpdrachtRepo = new DummyOpdrachtRepository();
Controller = new KlasController(new DummyKlasRepository(),
new DummyLesRepository(), OpdrachtRepo);
Opdracht TestOpdracht = new Opdracht
{
OpdrachtID = 1,
VakID = 1,
StamNummer = "im1"
};
Vak TestVak = new Vak { VakID = 1, VakNaam = "FOOP" };
TestOpdracht.Vak = TestVak;
OpdrachtRepo.addOpdracht(TestOpdracht);
}
/// <summary>
///A test for Index
///</summary>
[TestMethod()]
public void CreateKlasDirectsToToonKlassen()
{
Klas Klas = new Klas { KlasNaam = "2dNet" };
RedirectToRouteResult view = Controller.Create(1) as RedirectToRouteResult;
Assert.IsNotNull(view);
Assert.AreEqual("ToonKlassen", view.RouteValues["action"]);
}
}
at the moment I get a nullreferenceException on the view (assert.isNotNull fails)
and here is one of my DummyRepository's:
class DummyOpdrachtRepository : IOpdrachtRepository
{
List<Opdracht> opdrachten;
public DummyOpdrachtRepository()
{
opdrachten = new List<Opdracht>();
}
public void addOpdracht(Opdracht opdracht)
{
opdrachten.Add(opdracht);
}
public string GetDocentID(int OpdrachtID)
{
var opdracht = opdrachten.Where(o => o.OpdrachtID == OpdrachtID).FirstOrDefault();
return opdracht.StamNummer;
}
public Opdracht Find(int id)
{
return opdrachten.Where(o => o.OpdrachtID == id).FirstOrDefault();
}
}
Normally I should have written the tests Before writting the code, I know (and I am convinced off TDD, as I have used it successfully in my latest Java-project). but it just doesn't seem to work..
here is the code for KlasController.Create action
public ActionResult Create(int id) //id = opdrachtID
{
var Opdracht = OpdrachtRepo.Find(id);
Vak vak;
if(Opdracht != null)
vak = Opdracht.Vak;
else
throw new NullReferenceException("Deze opdracht werd niet gevonden");
return View(new CreateKlasModel(id,vak));
}
I know this is a lot of code, but I really want to make this work.
Thanks for helping me out in advance :)
As vladimir77 already says in his comment, the method public ActionResult Create(int id) is of type ViewResult, so either you change you method to do areturn RedirectToRoute() or you change your test to
ViewResult view = Controller.Create(1);
Assert.IsNotNull(view);
A ViewResult can not be cast as a RedirectToRouteResult.