We have a spring cloud stream app consuming from Kinesis stream with a single shard.
The application has a consumer.concurrency of 2 and the spring.cloud.stream.instanceIndex and spring.cloud.stream.instanceCount are correctly set on each instance.
We have also added spring.cloud.stream.bindings.binding-target.group setting but this had no effect.
We have a statefulset with a minimum of 2 replicas.
The problem is that, when the messages are consumed from Kinesis, both instances are consuming the same data.
If we increase the number of shards from 1 to 2 then this problem goes away.
We are using the following maven dependencies:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kinesis</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>2021.0.2</version>
<type>pom</type>
<scope>import</scope>
</dependency>
Why are the services consuming the same data when there is a single shard?
Related
I'm using spring-kafka '2.2.7.RELEASE' to create a batch consumer and I'm trying to understand How can i configure a kafka batch consumer to retry a pre-defined no of times using SeekToCurrentBatchErrorHandler?
I see the one of the SeekToCurrentErrorHandler constructors takes 'maxFailures' as an argument but I don't see any such option for SeekToCurrentBatchErrorHandler. Please suggest.
2.2.x is no longer supported.
See the documentation for the reasons why recovery after some number of failures is not supported with batch listeners and older versions of the framework.
You can use the RetryingBatchErrorHandler (since 2.3.7) or RecoveringBatchErrorHandler (since 2.5.0) instead.
I'm working on wso2 API manager 2.6.0, if I'm connected to a distributed environment and tried making changes from publisher its not getting reflected soon in store! Any suggestions for this to make it happen as soon as I published.
Try disabling registry cache. Open registry.xml and set caches to false.
<currentDBConfig>wso2registry</currentDBConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache> <<<<<<<<<<<<<<<<<<<<<<<<<<<
<registryRoot>/</registryRoot>
<remoteInstance url="https://localhost">
<id>gov</id>
<cacheId>wso2carbon#jdbc:h2:./repository/database/WSO2SHARED_DB;DB_CLOSE_ON_EXIT=FALSE</cacheId>
<dbConfig>govregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache> <<<<<<<<<<<<<<<<<<<<<<<<<<<<
<registryRoot>/</registryRoot>
</remoteInstance>
Please note this change in not recommended for production.
After converting from Crashlytics via Fabric to Crashlytics via Firebase, I started seeing the below call stack in debug runs where StrictMode is enabled looking for resource leaks.
StrictMode is in use with this code, only on debug builds:
StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder()
.detectLeakedClosableObjects()
.penaltyLog()
.build());
I'm using this versions of Fabric's gradle tools in project-level gradle:
classpath "io.fabric.tools:gradle:1.27.0"
and these versions of Firebase and Crashlytics in module-level gradle:
implementation "com.google.firebase:firebase-core:16.0.7"
implementation "com.crashlytics.sdk.android:crashlytics:2.9.8"
During initialization, the Firebase instrumentation kicks off a background thread that is doing settings calls using okhttp. When it does, StrictMode causes this call stack to pop:
W/CrashlyticsCore: Received null settings, skipping report submission!
D/StrictMode: StrictMode policy violation: android.os.strictmode.LeakedClosableViolation: A resource was acquired at attached stack trace but never released. See java.io.Closeable for information on avoiding resource leaks.
at android.os.StrictMode$AndroidCloseGuardReporter.report(StrictMode.java:1786)
at dalvik.system.CloseGuard.warnIfOpen(CloseGuard.java:264)
at java.util.zip.Inflater.finalize(Inflater.java:398)
at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:250)
at java.lang.Daemons$FinalizerDaemon.runInternal(Daemons.java:237)
at java.lang.Daemons$Daemon.run(Daemons.java:103)
at java.lang.Thread.run(Thread.java:764)
Caused by: java.lang.Throwable: Explicit termination method 'end' not called
at dalvik.system.CloseGuard.open(CloseGuard.java:221)
at java.util.zip.Inflater.<init>(Inflater.java:114)
at com.android.okhttp.okio.GzipSource.<init>(GzipSource.java:62)
at com.android.okhttp.internal.http.HttpEngine.unzip(HttpEngine.java:473)
at com.android.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:648)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:471)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:407)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:538)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:26)
at io.fabric.sdk.android.services.network.HttpRequest.code(HttpRequest.java:1357)
at io.fabric.sdk.android.services.settings.DefaultSettingsSpiCall.handleResponse(DefaultSettingsSpiCall.java:104)
at io.fabric.sdk.android.services.settings.DefaultSettingsSpiCall.invoke(DefaultSettingsSpiCall.java:88)
at io.fabric.sdk.android.services.settings.DefaultSettingsController.loadSettingsData(DefaultSettingsController.java:90)
at io.fabric.sdk.android.services.settings.DefaultSettingsController.loadSettingsData(DefaultSettingsController.java:67)
at io.fabric.sdk.android.services.settings.Settings.loadSettingsData(Settings.java:153)
at io.fabric.sdk.android.Onboarding.retrieveSettingsData(Onboarding.java:126)
at io.fabric.sdk.android.Onboarding.doInBackground(Onboarding.java:99)
at io.fabric.sdk.android.Onboarding.doInBackground(Onboarding.java:45)
at io.fabric.sdk.android.InitializationTask.doInBackground(InitializationTask.java:63)
at io.fabric.sdk.android.InitializationTask.doInBackground(InitializationTask.java:28)
at io.fabric.sdk.android.services.concurrency.AsyncTask$2.call(AsyncTask.java:311)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:458)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:764)
D/FA: Event not sent since app measurement is disabled
I see this happening on pretty much every debug run of my app, during the initial activity startup in the app. However support claims they don't see this.
Does anyone know what conditions cause Fabric to kick off this onboarding/settings thread?
I've seen similar StrictMode call stacks in these other posts. I can't tell if this leak is in Fabric code, or in the okhttp library they are using. Here are links to similar cases where people are seeing what looks to me like the same underlying resource leak:
StrictMode penalising for Firebase Ads
Crashlytics with StrictMode enabled (detect all) gives "untagged socket detected"
https://github.com/cloudant/sync-android/issues/577
I have upgrade my worklight project from worklight 6.1 to mobilefirst 7.1. In previous project (WL6.1), pushnotification is working fine.
For mobilefirst, While checking ibm tutorials, I found this link for Adding Google Play services to your Android project.
http://www.ibm.com/support/knowledgecenter/en/SSHS8R_7.1.0/com.ibm.worklight.dev.doc/devref/t_adding_google_play_services.html
For pushnotification, is this step required? As per link, it will increase my application size. What is the significance of assming Google Service library? Can I skip it?
After configuring android pushnotification in mobilefirst 7.1 (Except adding goole service), I am facing this error on server side
com.ibm.pushworks.server.notification.gcm.GCMMediator addTokenToAndroidKey FPWSE1079W: GCM push token '##TOKEN_ID##' is not added to GCM notification key. GCM Service invocation failed (reason: Error while invoking request)
com.ibm.pushworks.server.exceptions.GCMException: GCM Service invocation failed (reason: Error while invoking request)
at com.ibm.pushworks.server.notification.gcm.GCMSender.sendToGCM(GCMSender.java:392)
at com.ibm.pushworks.server.notification.gcm.GCMSender.addTokenToAndroidKey(GCMSender.java:364)
at com.ibm.pushworks.server.notification.gcm.GCMMediator.addTokenToAndroidKey(GCMMediator.java:206)
at com.ibm.pushworks.server.core.PushServiceImpl.getNotificationKey(PushServiceImpl.java:2160)
at com.ibm.pushworks.server.core.PushServiceImpl.registerDevice(PushServiceImpl.java:613)
at com.worklight.integration.services.impl.DataAccessServiceImpl.updateDeviceToken(DataAccessServiceImpl.java:473)
......
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to android.googleapis.com:443 [android.googleapis.com/216.58.210.202, android.googleapis.com/216.58.210.234] failed: Connection refused: connect
at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:319)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at com.ibm.pushworks.server.notification.gcm.GCMSender.sendToGCM(GCMSender.java:388)
... 45 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:412)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:271)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:258)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:376)
at java.net.Socket.connect(Socket.java:546)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:244)
at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:125)
... 56 more
Can this be GCM port opening issue or relevant to google play services?
Google has deprecated the use of GCM.jar in a project and from its perspective you should use the Google Play Services library.
For IBM MobileFirst Platform, though, the Google Play Services library is required if you need to use the Cloud Sync feature and others (by Google). If you don't need this feature don't add the library, as there is still a built-in implementation in the MobileFirst SDK for push notifications using GCM.jar.
Notes:
Starting MobileFirst Foundation 8.0, the library is now required.
In v7.1, you can still use the library regardless of the built-in fallback, but you will need to use Android's ProGuard feature to decrease the classes size to prevent the dex error ("too many classes") and in general to reduce to size.
As for the network error, it seems that your network configuration is lacking (connection refused). The documentation also lists addresses/ports that Google requires having them open/accessible to properly function.
We move an application from JBoss AS 7.1.1 to WildFly 8.2.X (8.2.0-Final and 8.2.1-Final) and discovered the following problem:
First deployment works OK (slower than with JBoss AS 7.1.1, but that seems to me to be another problem).
After we redeploy the same EAR file (either from Eclipse or from the Web Interface), the JAX-RS requests are processed as long as they are not concurrent/sequential. When two parallel JAX-RS requests come, any Jax-RS requests (incl. the first two parallel) will simply timeout. No matter to which REST Resource the HTTP Requests will be dispatched.
I have debugged a bit the RestEasy 3.0.10 library and detected that the code simply waits for the dispatched REST method to return. On the other side once hanged, it never enters my REST method (of my Rest Resource).
Any ideas on how to debug further? I cannot reproduce this behavior with other EAR applications on exactly the same server.
After checking further with jconsole, I have seen that a deadlock is created: a thread waits in
org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:231)
org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:42)
org.jboss.logmanager.ExtHandler.publish(ExtHandler.java:79)
org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:296)
org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304)
org.jboss.logmanager.Logger.logRaw(Logger.java:721)
org.jboss.logmanager.Logger.log(Logger.java:506)
org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71)
- locked java.lang.StringBuilder#497a942
org.jboss.stdio.WriterOutputStream.finish(WriterOutputStream.java:143)
org.jboss.stdio.WriterOutputStream.flush(WriterOutputStream.java:164)
- locked sun.nio.cs.UTF_8$Decoder#e92e69
java.io.PrintStream.write(PrintStream.java:482)
- locked java.io.PrintStream#d4482dd
and another waits in
java.io.PrintStream.flush(PrintStream.java:335)
org.jboss.stdio.StdioContext$DelegatingPrintStream.flush(StdioContext.java:216)
sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
- locked java.io.OutputStreamWriter#7797a41d
java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
The problem seems to be that the EAR application comes with its own log4j library, without excluding the one from WildFly. The following part in the jboss-deployment-structure.xml file seems to solve the problem, by disabling the loading of the logging subsystem:
<jboss-deployment-structure>
<deployment>
<!-- exclude-subsystem prevents a subsystems deployment unit processors running on a deployment -->
<!-- which gives basically the same effect as removing the subsystem, but it only affects single deployment -->
<exclude-subsystems>
<subsystem name="logging" />
</exclude-subsystems>
</deployment>
</jboss-deployment-structure>