spring-integration-dsl: Make Feed-Flow Work - rss

I'm trying to code a RSS-feed reader with a configured set of RSS-feeds. I thought that a good approach is to solve that by coding a prototype-#Bean and call it with each RSS-feed found in the configuration.
However, I guess that I'm missing a point here as the application launches, but nothing happens. I mean the beans are created as I'd expect, but there is no logging happening in that handle()-method:
#Component
public class HomeServerRunner implements ApplicationRunner {
private static final Logger logger = LoggerFactory.getLogger(HomeServerRunner.class);
#Autowired
private Configuration configuration;
#Autowired
private FeedConfigurator feedConfigurator;
#Override
public void run(ApplicationArguments args) throws Exception {
List<IntegrationFlow> feedFlows = configuration.getRssFeeds()
.entrySet()
.stream()
.peek(entry -> System.out.println(entry.getKey()))
.map(entry -> feedConfigurator.feedFlow(entry.getKey(), entry.getValue()))
.collect(Collectors.toList());
// this one appears in the log-file and looks good
logger.info("Flows: " + feedFlows);
}
}
#Configuration
public class FeedConfigurator {
private static final Logger logger = LoggerFactory.getLogger(FeedConfigurator.class);
#Bean
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public IntegrationFlow feedFlow(String name, FeedConfiguration configuration) {
return IntegrationFlows
.from(Feed
.inboundAdapter(configuration.getSource(), getElementName(name, "adapter"))
.feedFetcher(new HttpClientFeedFetcher()),
spec -> spec.poller(Pollers.fixedRate(configuration.getInterval())))
.channel(MessageChannels.direct(getElementName(name, "in")))
.enrichHeaders(spec -> spec.header("feedSource", configuration))
.channel(getElementName(name, "handle"))
//
// it would be nice if the following would show something:
//
.handle(m -> logger.debug("Payload: " + m.getPayload()))
.get();
}
private String getElementName(String name, String postfix) {
name = "feedChannel" + StringUtils.capitalize(name);
if (!StringUtils.isEmpty(postfix)) {
name += "." + postfix;
}
return name;
}
}
What's missing here? It seems as if I need to "start" the flows somehow.

Prototype beans need to be "used" somewhere - if you don't have a reference to it anywhere, no instance will be created.
Further, you can't put an IntegrationFlow #Bean in that scope - it generates a bunch of beans internally which won't be in that scope.
See the answer to this question and its follow-up for one technique you can use to create multiple adapters with different properties.
Alternatively, the upcoming 1.2 version of the DSL has a mechanism to register flows dynamically.

Related

How to get threadlocal for concurrency consumer?

I am developing spring kafka consumer. Due to message volume, I need use concurrency to make sure throughput. Due to used concurrency, I used threadlocal object to save thread based data. Now I need remove this threadlocal object after use it.
Spring document with below links suggested to implement a EventListener which listen to event ConsumerStoppedEvent . But did not mention any sample eventlistener code to get threadlocal object and remove the value. May you please let me know how to get the threadlocal instance in this case?
Code samples will be appreciated.
https://docs.spring.io/spring-kafka/docs/current/reference/html/#thread-safety
Something like this:
#SpringBootApplication
public class So71884752Application {
public static void main(String[] args) {
SpringApplication.run(So71884752Application.class, args);
}
#Bean
public NewTopic topic2() {
return TopicBuilder.name("topic1").partitions(2).build();
}
#Component
static class MyListener implements ApplicationListener<ConsumerStoppedEvent> {
private static final ThreadLocal<Long> threadLocalState = new ThreadLocal<>();
#KafkaListener(topics = "topic1", groupId = "my-consumer", concurrency = "2")
public void listen() {
long id = Thread.currentThread().getId();
System.out.println("set thread id to ThreadLocal: " + id);
threadLocalState.set(id);
}
#Override
public void onApplicationEvent(ConsumerStoppedEvent event) {
System.out.println("Remove from ThreadLocal: " + threadLocalState.get());
threadLocalState.remove();
}
}
}
So, I have two concurrent listener containers for those two partitions in the topic. Each of them is going to call this my #KafkaListener method anyway. I store the thread id into the ThreadLocal. For simple use-case and testing the feature.
The I implement ApplicationListener<ConsumerStoppedEvent> which is emitted in the appropriate consumer thread. And that one helps me to extract ThreadLocal value and clean it up in the end of consumer life.
The test against embedded Kafka looks like this:
#SpringBootTest
#EmbeddedKafka(bootstrapServersProperty = "spring.kafka.bootstrap-servers")
#DirtiesContext
class So71884752ApplicationTests {
#Autowired
KafkaTemplate<String, String> kafkaTemplate;
#Autowired
KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry;
#Test
void contextLoads() throws InterruptedException {
this.kafkaTemplate.send("topic1", "1", "foo");
this.kafkaTemplate.send("topic1", "2", "bar");
this.kafkaTemplate.flush();
Thread.sleep(1000); // Give it a chance to consume data
this.kafkaListenerEndpointRegistry.stop();
}
}
Right. It doesn't verify anything, but it demonstrate how that event can happen.
I see something like this in log output:
set thread id to ThreadLocal: 125
set thread id to ThreadLocal: 127
...
Remove from ThreadLocal: 125
Remove from ThreadLocal: 127
So, whatever that doc says is correct.

Spring OAuth2 Making `state` param at least 32 characters long

I am attempting to authorize against an external identity provider. Everything seems setup fine, but I keep getting a validation error with my identity provider because the state parameter automatically tacked onto my authorization request is not long enough:
For example:
&state=uYG5DC
The requirements of my IDP say that this state param must be at least 32-characters long. How can I programmatically increase the size of this auto-generated number?
Even if I could generate this number myself, it is not possible to override with other methods I have seen suggested. The following attempt fails because my manual setting of ?state=abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz is superceded by the autogenerated param placed after it during the actual request:
#Bean
public OAuth2ProtectedResourceDetails loginGovOpenId() {
AuthorizationCodeResourceDetails details = new AuthorizationCodeResourceDetails() {
#Override
public String getUserAuthorizationUri() {
return super.getUserAuthorizationUri() + "?state=abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz";
}
};
details.setClientId(clientId);
details.setAccessTokenUri(accessTokenUri);
details.setUserAuthorizationUri(userAuthorizationUri);
details.setScope(Arrays.asList("openid", "email"));
details.setPreEstablishedRedirectUri(redirectUri);
details.setUseCurrentUri(true);
return details;
}
The 6-character setting seems to be set here, is there a way to override this?
https://github.com/spring-projects/spring-security-oauth/blob/master/spring-security-oauth2/src/main/java/org/springframework/security/oauth2/common/util/RandomValueStringGenerator.java
With the help of this post:
spring security StateKeyGenerator custom instance
I was able to come up with a working solution.
In my configuration class marked with these annotations:
#Configuration
#EnableOAuth2Client
I configured the following beans:
#Bean
public OAuth2ProtectedResourceDetails loginGovOpenId() {
AuthorizationCodeResourceDetails details = new AuthorizationCodeResourceDetails();
AuthorizationCodeResourceDetails details = new
details.setClientId(clientId);
details.setClientSecret(clientSecret);
details.setAccessTokenUri(accessTokenUri);
details.setUserAuthorizationUri(userAuthorizationUri);
details.setScope(Arrays.asList("openid", "email"));
details.setPreEstablishedRedirectUri(redirectUri);
details.setUseCurrentUri(true);
return details;
}
#Bean
public StateKeyGenerator stateKeyGenerator() {
return new CustomStateKeyGenerator();
}
#Bean
public AccessTokenProvider accessTokenProvider() {
AuthorizationCodeAccessTokenProvider accessTokenProvider = new AuthorizationCodeAccessTokenProvider();
accessTokenProvider.setStateKeyGenerator(stateKeyGenerator());
return accessTokenProvider;
}
#Bean
public OAuth2RestTemplate loginGovOpenIdTemplate(final OAuth2ClientContext clientContext) {
final OAuth2RestTemplate template = new OAuth2RestTemplate(loginGovOpenId(), clientContext);
template.setAccessTokenProvider(accessTokenProvider());
return template;
}
Where my CustomStateKeyGenerator implementation class looks as follows:
public class CustomStateKeyGenerator implements StateKeyGenerator {
// login.gov requires state to be at least 32-characters long
private static int length = 32;
private RandomValueStringGenerator generator = new RandomValueStringGenerator(length);
#Override
public String generateKey(OAuth2ProtectedResourceDetails resource) {
return generator.generate();
}
}

Quartz.net and Ninject: how to bind implementation to my job using NInject

I am actually working in an ASP.Net MVC 4 web application where we are using NInject for dependency injection. We are also using UnitOfWork and Repositories based on Entity framework.
We would like to use Quartz.net in our application to start some custom job periodically. I would like that NInject bind automatically the services that we need in our job.
It could be something like this:
public class DispatchingJob : IJob
{
private readonly IDispatchingManagementService _dispatchingManagementService;
public DispatchingJob(IDispatchingManagementService dispatchingManagementService )
{
_dispatchingManagementService = dispatchingManagementService ;
}
public void Execute(IJobExecutionContext context)
{
LogManager.Instance.Info(string.Format("Dispatching job started at: {0}", DateTime.Now));
_dispatchingManagementService.DispatchAtomicChecks();
LogManager.Instance.Info(string.Format("Dispatching job ended at: {0}", DateTime.Now));
}
}
So far, in our NInjectWebCommon binding is configured like this (using request scope):
kernel.Bind<IDispatchingManagementService>().To<DispatchingManagementService>();
Is it possible to inject the correct implementation into our custom job using NInject ? and how to do it ? I have read already few posts on stack overflow, however i need some advises and some example using NInject.
Use a JobFactory in your Quartz schedule, and resolve your job instance there.
So, in your NInject config set up the job (I'm guessing at the correct NInject syntax here)
// Assuming you only have one IJob
kernel.Bind<IJob>().To<DispatchingJob>();
Then, create a JobFactory: [edit: this is a modified version of #BatteryBackupUnit's answer here]
public class NInjectJobFactory : IJobFactory
{
private readonly IResolutionRoot resolutionRoot;
public NinjectJobFactory(IResolutionRoot resolutionRoot)
{
this.resolutionRoot = resolutionRoot;
}
public IJob NewJob(TriggerFiredBundle bundle, IScheduler scheduler)
{
// If you have multiple jobs, specify the name as
// bundle.JobDetail.JobType.Name, or pass the type, whatever
// NInject wants..
return (IJob)this.resolutionRoot.Get<IJob>();
}
public void ReturnJob(IJob job)
{
this.resolutionRoot.Release(job);
}
}
Then, when you create the scheduler, assign the JobFactory to it:
private IScheduler GetSchedule(IResolutionRoot root)
{
var schedule = new StdSchedulerFactory().GetScheduler();
schedule.JobFactory = new NInjectJobFactory(root);
return schedule;
}
Quartz will then use the JobFactory to create the job, and NInject will resolve the dependencies for you.
Regarding scoping of the IUnitOfWork, as per a comment of the answer i linked, you can do
// default for web requests
Bind<IUnitOfWork>().To<UnitOfWork>()
.InRequestScope();
// fall back to `InCallScope()` when there's no web request.
Bind<IUnitOfWork>().To<UnitOfWork>()
.When(x => HttpContext.Current == null)
.InCallScope();
There's only one caveat that you should be aware of:
With incorrect usage of async in a web request, you may mistakenly be resolving a IUnitOfWork in a worker thread where HttpContext.Current is null. Now without the fallback binding, this would fail with an exception which would show you that you've done something wrong. With the fallback binding however, the issue may present itself in an obscured way. That is, it may work sometimes, but sometimes not. This is because there will be two (or even more) IUnitOfWork instances for the same request.
To remedy this, we can make the binding more specific. For this, we need some parameter to tell us to use another than InRequestScope(). Have a look at:
public class NonRequestScopedParameter : Ninject.Parameters.IParameter
{
public bool Equals(IParameter other)
{
if (other == null)
{
return false;
}
return other is NonRequestScopedParameter;
}
public object GetValue(IContext context, ITarget target)
{
throw new NotSupportedException("this parameter does not provide a value");
}
public string Name
{
get { return typeof(NonRequestScopedParameter).Name; }
}
// this is very important
public bool ShouldInherit
{
get { return true; }
}
}
now adapt the job factory as follows:
public class NInjectJobFactory : IJobFactory
{
private readonly IResolutionRoot resolutionRoot;
public NinjectJobFactory(IResolutionRoot resolutionRoot)
{
this.resolutionRoot = resolutionRoot;
}
public IJob NewJob(TriggerFiredBundle bundle, IScheduler scheduler)
{
return (IJob) this.resolutionRoot.Get(
bundle.JobDetail.JobType,
new NonrequestScopedParameter()); // parameter goes here
}
public void ReturnJob(IJob job)
{
this.resolutionRoot.Release(job);
}
}
and adapt the IUnitOfWork bindings:
Bind<IUnitOfWork>().To<UnitOfWork>()
.InRequestScope();
Bind<IUnitOfWork>().To<UnitOfWork>()
.When(x => x.Parameters.OfType<NonRequestScopedParameter>().Any())
.InCallScope();
This way, if you use async wrong, there'll still be an exception, but IUnitOfWork scoping will still work for quartz tasks.
For any users that could be interested, here is the solution that finally worked for me.
I have made it working doing some adjustment to match my project. Please note that in the method NewJob, I have replaced the call to Kernel.Get by _resolutionRoot.Get.
As you can find here:
public class JobFactory : IJobFactory
{
private readonly IResolutionRoot _resolutionRoot;
public JobFactory(IResolutionRoot resolutionRoot)
{
this._resolutionRoot = resolutionRoot;
}
public IJob NewJob(TriggerFiredBundle bundle, IScheduler scheduler)
{
try
{
return (IJob)_resolutionRoot.Get(
bundle.JobDetail.JobType, new NonRequestScopedParameter()); // parameter goes here
}
catch (Exception ex)
{
LogManager.Instance.Info(string.Format("Exception raised in JobFactory"));
}
}
public void ReturnJob(IJob job)
{
}
}
And here is the call schedule my job:
public static void RegisterScheduler(IKernel kernel)
{
try
{
var scheduler = new StdSchedulerFactory().GetScheduler();
scheduler.JobFactory = new JobFactory(kernel);
....
}
}
Thank you very much for your help
Thanks so much for your response. I have implemented something like that and the binding is working :):
public IJob NewJob(TriggerFiredBundle bundle, IScheduler scheduler)
{
var resolver = DependencyResolver.Current;
var myJob = (IJob)resolver.GetService(typeof(IJob));
return myJob;
}
As I told before I am using in my project a service and unit of work (based on EF) that are both injected with NInject.
public class DispatchingManagementService : IDispatchingManagementService
{
private readonly IUnitOfWork _unitOfWork;
public DispatchingManagementService(IUnitOfWork unitOfWork)
{
_unitOfWork = unitOfWork;
}
}
Please find here how I am binding the implementations:
kernel.Bind<IUnitOfWork>().To<EfUnitOfWork>()
kernel.Bind<IDispatchingManagementService>().To<DispatchingManagementService>();
kernel.Bind<IJob>().To<DispatchingJob>();
To resume, the binding of IUnitOfWork is done for:
- Eevery time a new request is coming to my application ASP.Net MVC: Request scope
- Every time I am running the job: InCallScope
What are the best practices according to the behavior of EF ? I have find information to use CallInScope. Is it possible to tell NInject to get a scope ByRequest everytime a new request is coming to the application, and a InCallScope everytime my job is running ? How to do that ?
Thank you very much for your help

Why can't I use #EJB with #Named?

I basically am trying to inject a #Stateless bean with a local interface into a class annotated with #Named! My understanding is that injection is only possible when the injection point is managed (makes perfect sense), so for example it wouldn't be possible to inject into a POJO but you could inject into a Servlet, a JSF managed or another EJB.
I would have thought that it would have been possible to subsequently use it with #Named! However I get a NullPointerException that specifically seems to imply that this in fact doesn't seem possible!?
My classes look like this (stripped for clarity);
#Named
public class EmailUtil {
// Logger-------------------------------------------------------------------
private static final Logger LOG = Logger.getLogger(EmailUtil.class.getName());
// Constructor--------------------------------------------------------------
public EmailUtil() {
}
// EJB----------------------------------------------------------------------
#EJB AuditDAO audit;
// Methods------------------------------------------------------------------
public void sendEmail(
String emailSender,
String emailRecipient,
String emailSubject,
String emailHtmlBody,
String emailTextBody) throws FailedEmailException {
... code removed for clarity ...
// Call Amazon SES to send the message
try {
new SES().getClient().sendEmail(request);
// Create an audit log of the event
audit.create("Email sent to " + emailSender);
} catch (AmazonClientException ace) {
LOG.log(Level.SEVERE, ace.getMessage(), ace);
throw new FailedEmailException();
} catch (Exception e) {
LOG.log(Level.SEVERE, e.getMessage(), e);
}
}
}
#Stateless
public class AuditDAOImpl implements AuditDAO {
// Logger-------------------------------------------------------------------
private static final Logger LOG = Logger.getLogger(AuditDAOImpl.class.getName());
// EntityManager------------------------------------------------------------
#PersistenceContext(unitName = "iConsultPU")
private EntityManager em;
#Override
public void create(String event) {
String subject;
try {
/*
* If the current subject has authenticated and created a session we
* want to register their ID. However it is possible that a subject
* does not have an ID so we want to set it to unknown.
*/
subject = SecurityUtils
.getSubject()
.getPrincipals()
.asList()
.get(1)
.toString();
} catch (Exception e) {
subject = "UNKNOWN";
}
Audit audit = new Audit();
audit.setUserId(subject);
audit.setEventTime(Calendar.getInstance());
audit.setEvent(event);
em.persist(audit);
}
}
#Local
public interface AuditDAO {
public void create(String event);
}
I've tried using #Inject as well but that doesn't seem to work either. Have I misunderstood the specification or just poorly implemented it?
You should be injecting your dependencies. So if your EmailUtil is being manually constructed, injection won't work. It needs to be container managed. So if you use a servlet, or any managed bean, you can #Inject it. CDI injection only works for managed objects.
You can do some additional work arounds, such as manually invoking it against a constructed instance. Take a look at this question for an example like that: Parallel webservices access in a Weld CDI environment
Do you have the beans.xml in the correct location? Injection for #Named (and other CDI beans) is handled by CDI, which isn't started unless you have the beans.xml file in the correct location (WEB-INF for war and META-INF for jar).

Unit Testing I18N RESTful Web Services with Spring, RestTemplate and Java Config

Trying to get Unit Tests to work when using Spring RestTemplate and I18N. Everything in the setup works fine for all the other test cases.
Based upon what I read, this is what I put into the Java Config:
#Bean
public LocaleChangeInterceptor localeChangeInterceptor() {
return new LocaleChangeInterceptor();
}
#Bean
public DefaultAnnotationHandlerMapping handlerMapping() {
DefaultAnnotationHandlerMapping mapping = new DefaultAnnotationHandlerMapping();
Object[] interceptors = new Object[1];
interceptors[0] = new LocaleChangeInterceptor();
mapping.setInterceptors(interceptors);
return mapping;
}
#Bean
public AnnotationMethodHandlerAdapter handlerAdapter() {
return new AnnotationMethodHandlerAdapter();
}
Then in my usage with RestTemplate I have:
public MyEntity createMyEntity(MyEntity bean) {
Locale locale = LocaleContextHolder.getLocale();
String localeString = "";
if (locale != Locale.getDefault()) {
localeString = "?locale=" + locale.getLanguage();
}
HttpEntity<MyEntity> req = new HttpEntity<MyEntity>(bean);
ResponseEntity<MyEntity> response = restTemplate.exchange(restEndpoint + "/url_path" + localeString, HttpMethod.POST, req, MyEntity.class);
return response.getBody();
}
While this could be cleaned up a bit, it should work - but the LocalChangeInterceptor never gets invoked. I am debugging this now and will post again as soon as I figure it out - but in the hope this is a race condition that I lose - does anyone know why?
Was lucky and stumbled upon this thread. One of the notes clued me into the right direction. You don't need all those beans in the Java Config. But if you are using #EnableWebMvc as I am, but I didn't know it was important enough to even mention, all you need to do in your Java Config is:
#Bean
public LocaleChangeInterceptor localeChangeInterceptor() {
return new LocaleChangeInterceptor();
}
#Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(new LocaleChangeInterceptor());
super.addInterceptors(registry);
}
Add the one bean for the Interceptor and then override the method to add the interceptor. Here my configuration class (annotated with #Configuration and #EnableWebMvc) also extends WebMvcConfigurerAdapter, which should be common usage.
This, at least, worked for me. Hope it may help someone else.

Resources