Below is my spring boot kafka consumer application to read data from kafka topic. In this application we are planning to implement heartbeat funtionally to post its heartbeat to url using #schduling annotaion to know its alive and running(which loads my json input data to db). purpose of this post request is to update the status on application monitoring tool.
to achive this i placed my heartbeat code to in manyplaces of my application but
I could'not able to achive this becuase #postconstuct or consumer.poll() is not allowing to run the heartbeat code piece.
we are using apache kafka 2.12, What could be the right approach to implment this behaviour in my spring boot app? Is their any other api to do such post request to url, every few miuntes through out the application.? Writing background thread will resolve this issue, please share any? why postconstuct() or poll() is blocking other recurresive code to run.
Please help me. Thanks in advance.
#SpringBootApplication
#EnableScheduling
public class KafkaApp {
#Autowired
ConsumerService kcService;
public static void main(String[] args) {
SpringApplication.run(KafkaApp.class, args);
}
#PostConstruct
public void init(){
kcService.getMessagesFromKafka();
}
}
and 2 #Service Definitions:
import org.apache.kafka.clients.consumer.Consumer;
#Service public class ConsumerService {
final Consumer<Long, String> consumer = createConsumer();
final int giveUp = 100;
int noRecordsCount = 0;
while (true) {
final ConsumerRecords<Long, String> consumerRecords = consumer.poll(1000);
if (consumerRecords.count()==0) {
noRecordsCount++;
if (noRecordsCount > giveUp) break;
else continue;
}
consumerRecords.forEach(record -> {
System.out.printf("Consumer Record:(%d, %s, %d, %d)\n",
record.key(), record.value(),
record.partition(), record.offset());
});
consumer.commitAsync();
}
}
#Scheduled(fixedDelay = 180000)
public void heartbeat() {
RestTemplate restTemplate = new RestTemplate();
String url = "endpoint url";
String requestJson = "{\"I am alive\":\"App name?\"}";
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
HttpEntity<String> entity = new HttpEntity<String>(requestJson,headers);
String answer = restTemplate.postForObject(url, entity, String.class);
System.out.println(answer);
}
Add annotation to your main class like:
#SpringBootApplication
#EnableScheduling
public class KafkaApp {
#Autowired
ConsumerService kcService;
public static void main(String[] args) {
SpringApplication.run(KafkaApp.class, args);
}
#PostConstruct
public void init(){
kcService.getMessagesFromKafka();
}
}
For more detail,spring-boot-task-scheduling-with-scheduled-annotation you can visit this link:
If you want to write a cron job for this purpose then in application.properties add this:
cron.expression=5 0 0 ? * * * //Its means it'll execute every 5 sec
You can make cron expression online here is a link:cron-expression-generator-quartz.
And in your heart beat function write this above function like:
#Scheduled(cron = "${cron.expression}")
public void heartbeat() {
//Your code here.
}
Related
I am attempting to authorize against an external identity provider. Everything seems setup fine, but I keep getting a validation error with my identity provider because the state parameter automatically tacked onto my authorization request is not long enough:
For example:
&state=uYG5DC
The requirements of my IDP say that this state param must be at least 32-characters long. How can I programmatically increase the size of this auto-generated number?
Even if I could generate this number myself, it is not possible to override with other methods I have seen suggested. The following attempt fails because my manual setting of ?state=abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz is superceded by the autogenerated param placed after it during the actual request:
#Bean
public OAuth2ProtectedResourceDetails loginGovOpenId() {
AuthorizationCodeResourceDetails details = new AuthorizationCodeResourceDetails() {
#Override
public String getUserAuthorizationUri() {
return super.getUserAuthorizationUri() + "?state=abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz";
}
};
details.setClientId(clientId);
details.setAccessTokenUri(accessTokenUri);
details.setUserAuthorizationUri(userAuthorizationUri);
details.setScope(Arrays.asList("openid", "email"));
details.setPreEstablishedRedirectUri(redirectUri);
details.setUseCurrentUri(true);
return details;
}
The 6-character setting seems to be set here, is there a way to override this?
https://github.com/spring-projects/spring-security-oauth/blob/master/spring-security-oauth2/src/main/java/org/springframework/security/oauth2/common/util/RandomValueStringGenerator.java
With the help of this post:
spring security StateKeyGenerator custom instance
I was able to come up with a working solution.
In my configuration class marked with these annotations:
#Configuration
#EnableOAuth2Client
I configured the following beans:
#Bean
public OAuth2ProtectedResourceDetails loginGovOpenId() {
AuthorizationCodeResourceDetails details = new AuthorizationCodeResourceDetails();
AuthorizationCodeResourceDetails details = new
details.setClientId(clientId);
details.setClientSecret(clientSecret);
details.setAccessTokenUri(accessTokenUri);
details.setUserAuthorizationUri(userAuthorizationUri);
details.setScope(Arrays.asList("openid", "email"));
details.setPreEstablishedRedirectUri(redirectUri);
details.setUseCurrentUri(true);
return details;
}
#Bean
public StateKeyGenerator stateKeyGenerator() {
return new CustomStateKeyGenerator();
}
#Bean
public AccessTokenProvider accessTokenProvider() {
AuthorizationCodeAccessTokenProvider accessTokenProvider = new AuthorizationCodeAccessTokenProvider();
accessTokenProvider.setStateKeyGenerator(stateKeyGenerator());
return accessTokenProvider;
}
#Bean
public OAuth2RestTemplate loginGovOpenIdTemplate(final OAuth2ClientContext clientContext) {
final OAuth2RestTemplate template = new OAuth2RestTemplate(loginGovOpenId(), clientContext);
template.setAccessTokenProvider(accessTokenProvider());
return template;
}
Where my CustomStateKeyGenerator implementation class looks as follows:
public class CustomStateKeyGenerator implements StateKeyGenerator {
// login.gov requires state to be at least 32-characters long
private static int length = 32;
private RandomValueStringGenerator generator = new RandomValueStringGenerator(length);
#Override
public String generateKey(OAuth2ProtectedResourceDetails resource) {
return generator.generate();
}
}
I am trying to write a Spring WS client using WebServiceGatewaySupport. I managed to test the client for a successful request and response. Now I wanted to write test cases for soap faults.
public class MyClient extends WebServiceGatewaySupport {
public ServiceResponse method(ServiceRequest serviceRequest) {
return (ServiceResponse) getWebServiceTemplate().marshalSendAndReceive(serviceRequest);
}
#ActiveProfiles("test")
#RunWith(SpringRunner.class)
#SpringBootTest(classes = SpringTestConfig.class)
#DirtiesContext
public class MyClientTest {
#Autowired
private MyClient myClient;
private MockWebServiceServer mockServer;
#Before
public void createServer() throws Exception {
mockServer = MockWebServiceServer.createServer(myClient);
}
}
My question is how do i stub the soap fault response in the mock server, so that my custom FaultMessageResolver will be able to unmarshall soap fault?
I tried couple of things below, but nothing worked.
// responsePayload being SoapFault wrapped in SoapEnvelope
mockServer.expect(payload(requestPayload))
.andRespond(withSoapEnvelope(responsePayload));
// tried to build error message
mockServer.expect(payload(requestPayload))
.andRespond(withError("soap fault string"));
// tried with Exception
mockServer.expect(payload(requestPayload))
.andRespond(withException(new RuntimeException));
Any help is appreciated. Thanks!
Follow Up:
Ok so, withSoapEnvelope(payload) I managed to get the controller to go to my custom MySoapFaultMessageResolver.
public class MyCustomSoapFaultMessageResolver implements FaultMessageResolver {
private Jaxb2Marshaller jaxb2Marshaller;
#Override
public void resolveFault(WebServiceMessage message) throws IOException {
if (message instanceof SoapMessage) {
SoapMessage soapMessage = (SoapMessage) message;
SoapFaultDetailElement soapFaultDetailElement = (SoapFaultDetailElement) soapMessage.getSoapBody()
.getFault()
.getFaultDetail()
.getDetailEntries()
.next();
Source source = soapFaultDetailElement.getSource();
jaxb2Marshaller = new Jaxb2Marshaller();
jaxb2Marshaller.setContextPath("com.company.project.schema");
Object object = jaxb2Marshaller.unmarshal(source);
if (object instanceof CustomerAlreadyExistsFault) {
throw new CustomerAlreadyExistsException(soapMessage);
}
}
}
}
But seriously!!! I had to unmarshall every message and check the instance of it. Being a client I should be thorough with all possible exceptions of the service here, and create custom runtime exceptions and throw it from the resolver. Still at the end, its been caught in WebServiceTemplate and re thrown as just a runtime exception.
You could try with something like this:
#Test
public void yourTestMethod() // with no throw here
{
Source requestPayload = new StringSource("<your request>");
String errorMessage = "Your error message from WS";
mockWebServiceServer
.expect(payload(requestPayload))
.andRespond(withError(errorMessage));
YourRequestClass request = new YourRequestClass();
// TODO: set request properties...
try {
yourClient.callMethod(request);
}
catch (Exception e) {
assertThat(e.getMessage()).isEqualTo(errorMessage);
}
mockWebServiceServer.verify();
}
In this part of code mockWebServiceServer represents the instance of MockWebServiceServer class.
I'm learning Retrofit and RxJava and I'v created test to connect github:
public class GitHubServiceTests {
RestAdapter restAdapter;
GitHubService service;
#Before
public void setUp(){
Gson gson = new GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES)
.create();
restAdapter = new RestAdapter.Builder()
.setEndpoint("https://api.github.com")
.setConverter(new GsonConverter(gson))
.build();
service = restAdapter.create(GitHubService.class);
}
#Test
public void GitHubUsersListObservableTest(){
service.getObservableUserList().flatMap(Observable::from)
.subscribe(user -> System.out.println(user.login));
}
when I execute test, I see nothing in my console. But when I execute another test
#Test
public void GitHubUsersListTest(){
List<User> users = service.getUsersList();
for (User user : users) {
System.out.println(user.login);
}
it works, and I see user's logins in my console
Here is my Interface for Retrofit:
public interface GitHubService {
#GET("/users")
List<User> getUsersList();
#GET("/users")
Observable<List<User>> getObservableUserList();
}
where I'm wrong?
Because of the asynchronous call your test completes before a result is downloaded. That's typical issue and you have to 'tell' test to wait for the result. In plain java it would be:
#Test
public void GitHubUsersListObservableTest(){
CountDownLatch latch = new CountDownLatch(N);
service.getObservableUserList()
.flatMap(Observable::from)
.subscribe(user -> {
System.out.println(user.login);
latch.countDown();
});
latch.await();
}
Or you can use BlockingObservable from RxJava:
// This does not block.
BlockingObservable<User> observable = service.getObservableUserList()
.flatMap(Observable::from)
.toBlocking();
// This blocks and is called for every emitted item.
observable.forEach(user -> System.out.println(user.login));
I am new to Netty. I am using “Netty 3.6.2.Final”. I have created a Netty Client (MyClient) that talks to a remote server (The server implements a custom protocol based on TCP). I create a new ClientBootstrap instance for each MyClient instance (within the constructor).
My question is if I share “NioClientSocketChannelFactory” factory object among all the instances of MyClient then when/how do I release all the resources associated with the “NioClientSocketChannelFactory”?
In other words, since my Netty Client runs inside a JBOSS container running 24x7, should I release all resources by calling “bootstrap.releaseExternalResources();” and when/where should I do so?
More Info: My Netty Client is called from two scenarios inside a JBOSS container. First, in an infinite for loop with each time passing the string that needs to be sent to the remote server (in effect similar to below code)
for( ; ; ){
//Prepare the stringToSend
//Send a string and receive a string
String returnedString=new MyClient().handle(stringToSend);
}
Another scenarios is my Netty Client is called within concurrent threads with each thread calling “new MyClient().handle(stringToSend);”.
I have given the skeleton code below. It is very similar to the TelnetClient example at Netty website.
MyClient
import org.jboss.netty.bootstrap.ClientBootstrap;
import org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory;
public class MyClient {
//Instantiate this only once per application
private final static Timer timer = new HashedWheelTimer();
//All below must come from configuration
private final String host ="127.0.0.1";
private final int port =9699;
private final InetSocketAddress address = new InetSocketAddress(host, port);
private ClientBootstrap bootstrap;
//Timeout when the server sends nothing for n seconds.
static final int READ_TIMEOUT = 5;
public MyClient(){
bootstrap = new ClientBootstrap(NioClientSocketFactorySingleton.getInstance());
}
public String handle(String messageToSend){
bootstrap.setOption("connectTimeoutMillis", 20000);
bootstrap.setOption("tcpNoDelay", true);
bootstrap.setOption("keepAlive", true);
bootstrap.setOption("remoteAddress", address);
bootstrap.setPipelineFactory(new MyClientPipelineFactory(messageToSend,bootstrap,timer));
// Start the connection attempt.
ChannelFuture future = bootstrap.connect();
// Wait until the connection attempt succeeds or fails.
channel = future.awaitUninterruptibly().getChannel();
if (!future.isSuccess()) {
return null;
}
// Wait until the connection is closed or the connection attempt fails.
channel.getCloseFuture().awaitUninterruptibly();
MyClientHandler myClientHandler=(MyClientHandler)channel.getPipeline().getLast();
String messageReceived=myClientHandler.getMessageReceived();
return messageReceived;
}
}
Singleton NioClientSocketChannelFactory
public class NioClientSocketFactorySingleton {
private static NioClientSocketChannelFactory nioClientSocketChannelFactory;
private NioClientSocketFactorySingleton() {
}
public static synchronized NioClientSocketChannelFactory getInstance() {
if ( nioClientSocketChannelFactory == null) {
nioClientSocketChannelFactory=new NioClientSocketChannelFactory(
Executors.newCachedThreadPool(),
Executors.newCachedThreadPool());
}
return nioClientSocketChannelFactory;
}
protected void finalize() throws Throwable {
try{
if(nioClientSocketChannelFactory!=null){
// Shut down thread pools to exit.
nioClientSocketChannelFactory.releaseExternalResources();
}
}catch(Exception e){
//Can't do anything much
}
}
}
MyClientPipelineFactory
public class MyClientPipelineFactory implements ChannelPipelineFactory {
private String messageToSend;
private ClientBootstrap bootstrap;
private Timer timer;
public MyClientPipelineFactory(){
}
public MyClientPipelineFactory(String messageToSend){
this.messageToSend=messageToSend;
}
public MyClientPipelineFactory(String messageToSend,ClientBootstrap bootstrap, Timer timer){
this.messageToSend=messageToSend;
this.bootstrap=bootstrap;
this.timer=timer;
}
public ChannelPipeline getPipeline() throws Exception {
// Create a default pipeline implementation.
ChannelPipeline pipeline = pipeline();
// Add the text line codec combination first,
//pipeline.addLast("framer", new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
pipeline.addLast("decoder", new StringDecoder());
pipeline.addLast("encoder", new StringEncoder());
//Add readtimeout
pipeline.addLast("timeout", new ReadTimeoutHandler(timer, MyClient.READ_TIMEOUT));
// and then business logic.
pipeline.addLast("handler", new MyClientHandler(messageToSend,bootstrap));
return pipeline;
}
}
MyClientHandler
public class MyClientHandler extends SimpleChannelUpstreamHandler {
private String messageToSend="";
private String messageReceived="";
public MyClientHandler(String messageToSend,ClientBootstrap bootstrap) {
this.messageToSend=messageToSend;
this.bootstrap=bootstrap;
}
#Override
public void channelConnected(ChannelHandlerContext ctx, ChannelStateEvent e){
e.getChannel().write(messageToSend);
}
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e){
messageReceived=e.getMessage().toString();
//This take the control back to the MyClient
e.getChannel().close();
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e) {
// Close the connection when an exception is raised.
e.getChannel().close();
}
}
You should only call releaseExternalResources() once you are sure you not need it anymore. This may be for example when the application gets stopped or undeployed.
I'm face a problem since few days and I can't get solution. below is my app structure:
I have ejbapp.jar inside MyearDeployedOnJboss7.ear at the same level of equinox-server-side-app.war (built using warproduct) and I want to load class from MyJarToLaoadForEjbapp.jar which is in iModernizeWebClient_1.0.0.jar which is in plugins folder of equinox-server-side-app.war (I want show image of app structure but I cannot send image because forum rules need 10 score to be able to do that)
My question is how to allow ejbapp.jar load classes from "MyJarToLaoadForEjbapp.jar" inside MyWebClient_1.0.0.jar's plugin folder which is in the equinox-server-side-app.war.
I think using servletbridge classloader but no idea how to use it.
in my launch.ini I've:
osgi.*=#null org.osgi.*=#null eclipse.*=#null osgi.parentClassloader=app osgi.contextClassLoaderParent=app
I resolved my proble using Servlet HttpServiceTracker from the OSGI spec. how to do it : write HttpServiceTracker liket that :
public class HttpServiceTracker extends ServiceTracker {
private static final Logger logger = Logger
.getLogger(HttpServiceTracker.class.getName());
public HttpServiceTracker(BundleContext context) {
super(context, HttpService.class.getName(), null);
}
public Object addingService(ServiceReference reference) {
HttpService httpService = (HttpService) context.getService(reference);
logger.info("default context path : "
+ org.eclipse.rap.ui.internal.servlet.HttpServiceTracker.ID_HTTP_CONTEXT);
try {
logger.info("will register servlet ");
httpService.registerServlet("/programLauncherServlet",
new ProgramLauncherServlet(), null, null);
logger.info("servlet has been registred with http context ");
// httpService.registerResources( "/", "/html", null );
} catch (Exception e) {
//e.printStackTrace();
logger.info("The alias '/programLauncherServlet' is already in use");
}
return httpService;
}
public void removedService(ServiceReference reference, Object service) {
logger.info("will unregister servlet ");
HttpService httpService = (HttpService) service;
httpService.unregister("/programLauncher");
super.removedService(reference, service);
logger.info("servlet has been unregistred");
}
in your plugin activator class at method start :
#Override
public void start(BundleContext context) throws Exception {
super.start(context);
Activator.plugin = this;
BundleContext osgiContext = BundleReference.class
.cast(AnyClassOfYourProject.class.getClassLoader()).getBundle()
.getBundleContext();
serviceTracker = new HttpServiceTracker(osgiContext);
serviceTracker.open();
LOGGER.info("servlet published !!");
LOGGER.info("Bundle started.");
}
and for unregister the servlet at the stop method :
public void stop(BundleContext context) throws Exception {
Activator.plugin = null;
serviceTracker.close();
serviceTracker = null;
LOGGER.info("servlet unregistered from context !!");
super.stop(context);
}
that's all. your servlet is accessible outside your eclipse bundle and you can call methods inside the bundle.