I'm trying to use spring-cloud-sleuth to trace https requests initiated by spring-security-oauth.
But I'm stuck on that the spring-security-oauth filter OAuth2AuthenticationProcessingFilter is executed before the spring-cloud-sleuth filter TraceFilter.
Can this be changed so that the spring-cloud-sleuth filter is processed before the spring-security-oauth filter?
Version info:
spring-boot: 1.3.5
spring-cloud: Brixton.SR3
spring-cloud-sleuth: 1.0.3
spring-security-oauth2: 2.0.9
Update:
Based on the suggestion below I could solve the problem by defining my own FilterRegistrationBean as:
#Inject
TraceFilter traceFilter;
#Bean
public FilterRegistrationBean myTraceFilter() {
LOG.info("Register a TraceFilter with HIGHEST_PRECEDENCE");
FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean(traceFilter, new ServletRegistrationBean[0]);
filterRegistrationBean.setDispatcherTypes(ASYNC, new DispatcherType[]{ERROR, FORWARD, INCLUDE, REQUEST});
filterRegistrationBean.setOrder(Ordered.HIGHEST_PRECEDENCE);
return filterRegistrationBean;
}
You can register the TraceFilter yourself and manually provide the order. Just try to put it before the spring security filter. If that works fine you can file a PR / issue describing the whole flow so that we continue the discussion there.
Related
I am a happy user of the recently added RetryTopicConfiguration there is however a small issue that is bothering me.
The setup I use looks like:
#Bean
public RetryTopicConfiguration retryTopicConfiguration(
KafkaTemplate<String, String> template,
#Value("${kafka.topic.in}") String topicToInclude,
#Value("${spring.application.name}") String appName) {
return RetryTopicConfigurationBuilder
.newInstance()
.fixedBackOff(5000L)
.maxAttempts(3)
.retryTopicSuffix("-" + appName + ".retry")
.suffixTopicsWithIndexValues()
.dltSuffix("-" + appName + ".dlq")
.includeTopic(topicToInclude)
.dltHandlerMethod(KAFKA_EVENT_LISTENER, "handleDltEvent")
.create(template);
}
When the a listener throws an exception that triggers a retry, the DefaultErrorHandler will log a KafkaBackoffException at error level.
For a similar problem it was suggested to use a ListenerContainerFactoryConfigurer yet this does not remove all error logs, since I still see the following in my logs:
2022-04-02 17:34:33.340 ERROR 8054 --- [e.retry-0-0-C-1] o.s.kafka.listener.DefaultErrorHandler : Recovery of record (topic-spring-kafka-logging-issue.retry-0-0#0) failed
org.springframework.kafka.listener.ListenerExecutionFailedException: Listener failed; nested exception is org.springframework.kafka.listener.KafkaBackoffException: Partition 0 from topic topic-spring-kafka-logging-issue.retry-0 is not ready for consumption, backing off for approx. 4468 millis.
Can the log-level be changed, without adding a custom ErrorHandler?
Spring-Boot version: 2.6.6
Spring-Kafka version: 2.8.4
JDK version: 11
Sample project: here
Thanks for such a complete question. This is a known issue of Spring for Apache Kafka 2.8.4 due to the new combine blocking and non-blocking exceptions feature and has been fixed for 2.8.5.
The workaround is to clear the blocking exceptions mechanism such as:
#Bean(name = RetryTopicInternalBeanNames.LISTENER_CONTAINER_FACTORY_CONFIGURER_NAME)
public ListenerContainerFactoryConfigurer lcfc(KafkaConsumerBackoffManager kafkaConsumerBackoffManager,
DeadLetterPublishingRecovererFactory deadLetterPublishingRecovererFactory,
#Qualifier(RetryTopicInternalBeanNames
.INTERNAL_BACKOFF_CLOCK_BEAN_NAME) Clock clock) {
ListenerContainerFactoryConfigurer lcfc = new ListenerContainerFactoryConfigurer(kafkaConsumerBackoffManager, deadLetterPublishingRecovererFactory, clock);
lcfc.setBlockingRetriesBackOff(new FixedBackOff(0, 0));
lcfc.setErrorHandlerCustomizer(eh -> ((DefaultErrorHandler) eh).setClassifications(Collections.emptyMap(), true));
return lcfc;
}
Please let me know if that works for you.
Thanks.
EDIT:
This workaround disables only blocking retries, which since 2.8.4 can be used along non-blocking as per the link in the original answer. The exception classification for the non-blocking retries is in the DefaultDestinationTopicResolver class, and you can set FATAL exceptions as documented here.
EDIT: Alternatively, you can use the Spring Kafka 2.8.5-SNAPSHOT version by adding the Spring Snapshot repository such as:
repositories {
maven {
url 'https://repo.spring.io/snapshot'
}
}
dependencies {
implementation 'org.springframework.kafka:spring-kafka:2.8.5-SNAPSHOT'
}
You can also downgrade to Spring Kafka 2.8.3.
As Gary Russell pointed out, if your application is already in production you should not use the SNAPSHOT version, and 2.8.5 is out in a couple of weeks.
EDIT 2: Glad to hear you’re happy about the feature!
In a SpringBoot application, I have the following configuration:
axon:
axonserver:
servers: "${AXON_SERVER:localhost}"
serializer:
general: jackson
messages: jackson
events: jackson
logging.level:
org.axonframework.modelling.saga: debug
Downsizing the scenario to bare minimum, the relevant portion of Saga class:
#Slf4j
#Saga
#ProcessingGroup("AuctionEventManager")
public class AuctionEventManagerSaga {
#Autowired
private transient EventScheduler eventScheduler;
private ScheduleToken scheduleToken;
private Instant auctionTimerStart;
#StartSaga
#SagaEventHandler(associationProperty = "auctionEventId")
protected void on(final AuctionEventScheduled event) {
this.auctionTimerStart = event.getTimerStart();
// Cancel any pre-existing previous job, since the scheduling thread might be lost upon a crash/restart of JVM.
if (this.scheduleToken != null) {
this.eventScheduler.cancelSchedule(this.scheduleToken);
}
this.scheduleToken = this.eventScheduler.schedule(
this.auctionTimerStart,
AuctionEventStarted.builder()
.auctionEventId(event.getAuctionEventId())
.build()
);
}
#EndSaga
#SagaEventHandler(associationProperty = "auctionEventId")
protected void on(final AuctionEventStarted event) {
log.info(
"[AuctionEventManagerSaga] Current state: {scheduleToken={}, auctionTimerStart={}}",
this.scheduleToken,
this.auctionTimerStart
);
}
}
In the final compiled class, we will end up having 4 properties: log (from #Slf4j), eventScheduler (transient, #Autowired), scheduleToken and auctionTimerStart.
For reference information, here is a sample of the general approach I've been using for both Command and Event classes:
#Value
#Builder
#JsonDeserialize(builder = AuctionEventStarted.AuctionEventStartedBuilder.class)
public class AuctionEventStarted {
AuctionEventId auctionEventId;
#JsonPOJOBuilder(withPrefix = "")
public static final class AuctionEventStartedBuilder {}
}
When executing the code, you get the following output:
2020-05-12 15:40:01.180 DEBUG 1 --- [mandProcessor-4] o.a.m.saga.repository.jpa.JpaSagaStore : Updating saga id c8aff7f7-d47f-4616-8a96-a40044cb7e3b as {}
As soon as the general serializer is changed to xstream, the content is serialized properly, but I face another issue during deserialization, since I have private static final class Builder classes using Lombok.
So is there a way for Axon to handle these scenarios:
1- Axon to safely manage Jackson to ignore #Autowired, transient and static properties from #Saga classes? I've attempted to manually define #JsonIgnore at non-state properties and it still didn't work.
2- Axon to safely configure XStream to ignore inner classes (mostly Builder classes implemented as private static final)?
Thanks in advance,
EDIT: I'm pursuing a resolution using my preferred serializer: JSON. I attempted to modify the saga class and extend JsonSerializer<AuctionEventManagerSaga>. For that I implemented the methods:
#Override
public Class<AuctionEventManagerSaga> handledType() {
return AuctionEventManagerSaga.class;
}
#Override
public void serialize(
final AuctionEventManagerSaga value,
final JsonGenerator gen,
final SerializerProvider serializers
) throws IOException {
gen.writeStartObject();
gen.writeObjectField("scheduleToken", value.eventScheduler);
gen.writeObjectField("auctionTimerStart", value.auctionTimerStart);
gen.writeEndObject();
}
Right now, I have something being serialized, but it has nothing to do with the properties I've defined:
2020-05-12 16:20:01.322 DEBUG 1 --- [mandProcessor-0] o.a.m.saga.repository.jpa.JpaSagaStore : Storing saga id c4b5d94c-7251-40a5-accf-332768b1cacd as {"delegatee":null,"unwrappingSerializer":false}
EDIT 2 Decided to add more insight into the issue I experience when I switch general to use XStream (even though it's somewhat unrelated to the main issue described in the title).
Here is the issue it complains to me:
2020-05-12 17:08:06.495 DEBUG 1 --- [ault-executor-0] o.a.a.c.command.AxonServerCommandBus : Received command response [message_identifier: "79631ffb-9a87-4224-bed3-a957730dced7"
error_code: "AXONIQ-4002"
error_message {
message: "No converter available\n---- Debugging information ----\nmessage : No converter available\ntype : jdk.internal.misc.InnocuousThread\nconverter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter\nmessage[1] : Unable to make field private static final jdk.internal.misc.Unsafe jdk.internal.misc.InnocuousThread.UNSAFE accessible: module java.base does not \"opens jdk.internal.misc\" to unnamed module #7728643a\n-------------------------------"
location: "1#600b5b87a922"
details: "No converter available\n---- Debugging information ----\nmessage : No converter available\ntype : jdk.internal.misc.InnocuousThread\nconverter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter\nmessage[1] : Unable to make field private static final jdk.internal.misc.Unsafe jdk.internal.misc.InnocuousThread.UNSAFE accessible: module java.base does not \"opens jdk.internal.misc\" to unnamed module #7728643a\n-------------------------------"
}
request_identifier: "2f7020b1-f655-4649-bbe0-d6f458b3c2f3"
]
2020-05-12 17:08:06.505 WARN 1 --- [ault-executor-0] o.a.c.gateway.DefaultCommandGateway : Command 'ACommandClassDispatchedFromSaga' resulted in org.axonframework.commandhandling.CommandExecutionException(No converter available
---- Debugging information ----
message : No converter available
type : jdk.internal.misc.InnocuousThread
converter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
message[1] : Unable to make field private static final jdk.internal.misc.Unsafe jdk.internal.misc.InnocuousThread.UNSAFE accessible: module java.base does not "opens jdk.internal.misc" to unnamed module #7728643a
-------------------------------)
Still no luck on resolving this...
I've worked on Axon systems where the only used Serializer implementation was the JacksonSerializer too. Mind you though, this is not what the Axon team recommends. For messages (i.e. commands, events and queries) it makes perfect sense to use JSON as the serialized format. But switching the general Serializer to jackson means you have to litter your domain logic (e.g. your Saga) with Jackson specifics "to make it work".
Regardless, backtracking to my successful use case of jackson-serialized-sagas. In this case we used the correct match of JSON annotations on the fields we desired to take into account (the actual state) and to ignore the one's we didn't want deserialized (with either transient or #JsonIgnore). Why both do not seem to work in your scenario is not entirely clear at this stage.
What I do recall is that the referenced project's team very clearly decided against Lombok due to "overall weirdnes" when it comes to de-/serialization. As a trial it thus might be worth to not use any Lombok annotations/logic in the Saga class and see if you can de-/serialize it correctly in such a state.
If it does work at that moment, I think you have found your culprit for diving in further search.
I know this isn't an exact answer, but I hope it helps you regardless!
Might be worthwhile to share the repository where this problems occurs in; might make the problem clearer for others too.
I was able to resolve the issue #2 when using XStream as general serializer.
One of the Sagas had an #Autowired dependency property that was not transient.
XStream was throwing some cryptic message, but we managed to track the problem and address it.
As for JSON support, we had no luck. We ended up switched everything to XStream for now, as the company only uses Java and it would be ok to decode the events using XStream.
Not the greatest solution, as we really wanted (and hoped) JSON would be supported properly out of the box. Mind you, this is in conjunction with using Lombok which caused for the nuisance in this case.
I have a Spring Boot app with a few Controllers I want to track their dependencies (including outbound Http requests). That all works as expected. However, I have one controller for a health check (returning 204) that I do not want telemetry for. All other responses mention custom code components, but according to the documentation, this should be doable within the AI-Agent.xml config.
<BuiltInProcessors>
<Processor type="RequestTelemetryFilter">
<Add name="NotNeededResponseCodes" value="204" />
</Processor>
</BuiltInProcessors>
I notice on the classpath that there are two RequestTelemtryFilter instances (one from ai-core and one from ai-web, neither of which get hit when i debug).
Configuring the Agent (via AI-Agent.xml) is different than configuring custom telemetry (via Applicationinsights.xml). Spring boot + the agent requires the use of a custom Telemetry Processor and pulling into your configuration via #Bean. No additional XML in the AI-Agent is necessary.
public class HealthCheckTelemetryFilter implements TelemetryProcessor
{
public HealthCheckTelemetryFilter()
{
// TODO Auto-generated constructor stub
}
#Override
public boolean process(Telemetry telemetry)
{
RequestTelemetry reqTel = (RequestTelemetry) telemetry;
if(reqTel.getResponseCode().equals(HttpStatus.NO_CONTENT.toString()))
return false;
else
return true;
}
}
NOTE: dont forget appropriate type check
I m looking for a example how to use netflix-feign from a NO-spring-boot app.
I have a existing SpringMVC (4.2) webapp. Now i build some "microservices" with Spring-boot + (eureka, feign) and i want to use those as backend serivces from the webapp.
thanks in advance
In the github repo you can see a few examples to use feign-client:
https://github.com/OpenFeign/feign
Basically, you would need to create an interface annotated with feign annotations and not with the Spring, one example would be (you can see much more in the github page):
public interface YourClient {
#RequestLine("POST /")
#Headers("Content-Type: application/json")
Response getSomething(#Param("id") String id);
}
Then to instantiate your feign client it's need to use its builder Feign. It's easy, and configurable:
YourClient yourClient = Feign.builder()
.decoder(new GsonDecoder()) // you could use Gson, Jackson,...
.target(YourClient .class, "https://your.url");
Then you can use it just invoking the method you want:
yourClient.getSomething("myId");
I followed the instructions in this tutorial to set up Zend AMF as a way of passing data from my flash app to my PHP app:
http://codeigniter.com/forums/viewthread/180414/
So I have the directory structure and everything as described there. This is my gateway controller:
class Gateway extends CI_Controller
{
function __construct()
{
parent::__construct();
$this->load->library('zend');
//root_folder + application + controllers + amf + services
define('SERVICES_FOLDER', APPPATH.'controllers/amf/services/');
}
public function index()
{
$server = new Zend_Amf_Server();
$server->setProduction(false);
//$server->addFunction('testservice');
$server->addDirectory(SERVICES_FOLDER);
echo $server->handle();
}
}
And the APPPATH is /application/ so the path defined by SERVIES_FOLDER is "/application/controllers/amf/services" which is where my file 'testservice.php' sits.
When I try and connect to that service in flash:
var gateway:String = "http://mysite.com/amf/gateway";
con.connect(gateway);
con.call("Testservice.getMessage", new Responder(onResult, onFault));
It calls the onFault method and displays the error:
Plugin by name 'Testservice' was not found in the registry;
Which makes me think that the addDirectory() line in Gateway.php was the problem somehow. Interestingly, I also cannot access the testservice function through a URL, ie by going to mysite.com/amf/services/testservice.
Any thoughts on what might be going on here?
Figured it out, sort of.
Instead of using the addDirectory method which I was having no luck with, I used the setClass method and created another class within the gateway.php file that has the functions, and now I can connect to those functions from flash.
I had an issue with this when using parent::__construct() in my service controllers. Once I removed that, the problem went away.