I'am trying to find a solution to send tracing information of api calls in wso2am to apm server so that i can see them in kibana. i have followed instructions in the doc for Custom Tracer Implementation.
my implementation of the org.wso2.carbon.apimgt.tracing.OpenTracer interface is
public class ElasticTracer implements OpenTracer {
private static final String NAME = "elastic";
#Override
public Tracer getTracer(String serviceName) {
Tracer tracer = new ElasticApmTracer();
Reporter reporter = new TracingReporter(LogFactory.getLog("tracer"));
Tracer tracerR = new TracerR(tracer, reporter, new ThreadLocalScopeManager());
GlobalTracer.register(tracerR);
return tracerR;
}
#Override
public String getName() {
return NAME;
}
}
and I have added following options to the startup script
-javaagent:/root/Downloads/elastic-apm-agent-1.28.4.jar \
-Delastic.apm.server_urls=http://192.168.50.56:8200 \
-Delastic.apm.service_name=wso2am\
-Delastic.elastic.apm.log_level=DEBUG\
-Delastic.apm.application_packages=org.example\
but tracing information of api calls aren't sent to the elastic apm. tracing information of wso2am function calls are sent there.
when i use doc instructions for jaeger it works properly.
Related
Someone out there must have run into this already...
I created a WebApi solution with swagger implemented, full documentation, the whole 9 yards!
When I run my web api solution, see the swagger output (and I've tested the endpoints, all working fine)
I can see the swagger definition: https://localhost:5001/swagger/v1/swagger.json
Now, I want to consume this Api as a connected service on my web app.
So following every single tutorial online:
I go to my webapp
right click on Connected Services
Add Connected Service
Add Service Reference > OpenApi > add Url, namespace & class name
That generates a partial class in my solution (MyTestApiClient)
public parial class MyTestApiClient
{
// auto generated code
}
Next step, inject the service in Startup.cs
services.AddTransient(x =>
{
var client = new MyTestApiClient("https://localhost:5001", new HttpClient());
return client;
});
Then, inject the class into some class where it's consumed and this all works
public class TestService
{
private readonly MyTestApiClient _client; // this is class, not an interface -> my problem
public TestService(MyTestApiClient client)
{
_client = client;
}
public async Task<int> GetCountAsync()
{
return _client.GetCountAsync();
}
}
So everything up to here works. BUT, this generated OpenApi client doesn't have an interface which sucks for the purposes of DI and Unit Testing.
I got around this by creating a local interface IMyTestApiClient, added to the generated class (MyTestApiClient). I only have 1 endpoint in my WebApi so have to declare that on my interface.
public parial class MyTestApiClient : IMyTestApiClient
{
// auto generated code
}
public interface IMyTestApiClient
{
// implemented in generated MyTestApiClient class
Task<int> GetCountAsync();
}
services.AddTransient<IMyTestApiClient, MyTestApiClient>(x =>
{
IMyTestApiClient client = new MyTestApiClient("https://localhost:5001", new HttpClient());
return client;
});
public class TestService
{
private readonly IMyTestApiClient _client; // now injecting local interface instead of the generated class - great success
public TestService(IMyTestApiClient client)
{
_client = client;
}
public async Task<int> GetCountAsync()
{
return _client.GetCountAsync();
}
}
But this is a bad approach because it makes me manually create an interface and explicitly declare the methods I want to consume. Furthermore, every time my Api gets updated, I will have to tweak my local interface.
So question time:
How can I add an OpenApi Service Reference that automagically also generates an interface as well?
Thanks in advance for any help getting to a viable solution.
You may have already found the answer but I had the same issue and managed to resolve it by adding /GenerateClientInterfaces:true in the Options section for the OpenAPI reference in my .csproj:
<OpenApiReference Include="api.json" CodeGenerator="NSwagCSharp" Namespace="MyNamespace" ClassName="MyClassName">
<SourceUri>https://localhost:7040/swagger/v1/swagger.json</SourceUri>
<OutputPath>MyClient.cs</OutputPath>
<Options>/GenerateClientInterfaces:true</Options>
</OpenApiReference>
I'd like to implement a Pact consumer test in our Java EE application. This test shall invoke a consumer service method which would trigger the actual REST call.
Here's the Pact test so far:
#ExtendWith(PactConsumerTestExt.class)
#PactTestFor(providerName = "my-service")
public class MyServiceConsumerTest {
#Inject
private MyService myService;
#Pact(consumer = "myConsumer")
public RequestResponsePact mail(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap<>();
headers.put("Content-Type", ContentType.getJSON().asString());
PactDslJsonBody jsonBody = new PactDslJsonBody()
.stringValue("emailAddress", "foo#bar.com")
.stringValue("subject", "Test subject")
.stringValue("content", "Test content")
.asBody();
return builder
.given("DEFAULT_STATE")
.uponReceiving("request for sending a mail")
.path("/mail")
.method("POST")
.headers(headers)
.body(jsonBody)
.willRespondWith()
.status(Response.Status.OK.getStatusCode())
.toPact();
}
#Test
#PactTestFor(pactMethod = "mail")
public void sendMail() {
MailNotification mailNotification = MailNotification.builder()
.emailAddress("foo#bar.com")
.subject("Test subject")
.content("Test content")
.build();
myService.sendNotification(mailNotification);
}
}
The interesting part is this line:
myService.sendNotification(mailNotification);
As I'm running a consumer unit test, the injection of MyService does not work, i.e. results in myService being null. Moreover I think it would be necessary to tell the service to send its request against the Pact mock serveR?
Of course I could just fire the final REST request in the test but that would ignore the service logic.
I guess I'm missing something here?
Yes, you should hit the mock server in the #PactVerification test. Don't fire without the actual application code, it makes a few sense in case of future changes. Tests should fail if you change an HTTP property of that request
I have to do a POC on contract testing using pact, but I couldn't found anything helpful for a newbie. Can someone help me with the working code, how to install, execute I will be grateful.
I tried to explain below.
Consumer: Contract created by consumer.
Provider: Contracts tested by provider.
Pack Broker: After contracts are created under location (like targer/pacts) defined by you, you must publish the contracts to the common platform where consumer and provider will see.
Consumer side - Create contract for provider
public class CreateContractForProvider {
#Rule //Provider, HostInterface and Port defined with #Rule annotation (Used PactProviderRuleMk2)
public PactProviderRuleMk2 pactProviderRuleMk2 = new PactProviderRuleMk2(
// Provider Application Name
"ProviderName",
//Mock Server
"localhost",
8112,
this);
#Pact(consumer = "ConsumerName") // Consumer Application Name (Our application) - Consumer defined with #Pact annotation(
public RequestResponsePact createPact(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap();
headers.put("Content-Type", "application/json"); //Defined headers
//Defined responses with PactDslJsonBody()
DslPart expectedResultBodyWhenGetPayments = new PactDslJsonBody()
.integerType("id",308545)
.integerType("contractNo",854452)
.numberType("amount",3312.5)
.stringType("status","UNPAID")
.asBody();
return builder
.uponReceiving("A request for all payments")
.path("/payments")
.method("GET")
.willRespondWith()
.status(200)
.headers(headers)
.body(expectedResultBodyWhenGetPayments).toPact(); //Response bodyies and headers used in return builder
// We can define more than one endpoint with .uponReceiving or .given
//Then we have to test beacuse contracts are created test stage.
//When we say test with #PactVerification, the server we described above stands up(localhost:8112). İf we get localhost:8112/(definedpathlike/payments) its return expectedResultBodyWhenGetPayments.If the test is successful, the contracts is create.
#Test
#PactVerification()
public void pactVerification() {
int contractNo=((Integer) new ContractTestUtil(pactProviderRuleMk2.getPort()).getContractResponse("/payments","contractNo")).intValue();
assertTrue(contractNo == 854452);
}}
Test Util
public class ContractTestUtil {
int port=8111;
public ContractTestUtil(int port) {
this.port=port;
System.out.println("Custom port "+port);
}
public Object getContractResponse(String path,String object) {
try {
System.setProperty("pact.rootDir", "./target/pacts");
System.setProperty("pact.rootDir", "./target/pacts");
String url=String.format("Http://localhost:%d"+path, port);
System.out.println("using url: "+url);
HttpResponse httpResponse = Request.Get(url).execute().returnResponse();
String json = EntityUtils.toString(httpResponse.getEntity());
System.out.println("json="+json);
JSONObject jsonObject = new JSONObject(json);
return jsonObject.get(object);
}
catch (Exception e) {
System.out.println("Unable to get object="+e);
return null;
}
}}
Define Pact Broker
The PactBrokerUr lmust be defined before publishing in pom.
<plugin>
<!-- mvn pact:publish -->
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-maven_2.11</artifactId>
<version>3.5.10</version>
<configuration>
<pactDirectory>./target/pacts</pactDirectory> <!-- Defaults to ${project.build.directory}/pacts -->
<pactBrokerUrl>http://yourmachine:8113</pactBrokerUrl>
<projectVersion>1.1</projectVersion> <!-- Defaults to ${project.version} -->
<trimSnapshot>true</trimSnapshot> <!-- Defaults to false -->
</configuration>
</plugin>
Now, we can publish with pact:puplish command.
Provider Side - Call contracts created by consumer
In this stage you can test with failsafe plugin. Beacuse its integraion test.
#RunWith(PactRunner.class) // Say JUnit to run tests with custom Runner
#Provider("ProviderName")
#Consumer("ConsumerName")// Set up name of tested provider// Provider Application Name
#PactBroker(port = "8113", host = "yourmachine")
public class VerifyContractsWhichCreatedForProviderIT {
private static ConfigurableWebApplicationContext configurableWebApplicationContext;
#BeforeClass
public static void start() {
configurableWebApplicationContext = (ConfigurableWebApplicationContext)
SpringApplication.run(Application.class);
}
#TestTarget // Annotation denotes Target that will be used for tests
public final Target target = new HttpTarget(8080); //Test Target
}
Finally,you can create contrats and verify contrast created for you with clean test pact:publish verify command.
Trying to figure out if I can use Spring Reactive (Flux/Mono) with Spring MVC ?
The structure of microservices using Spring MVC + Feign Client, Eureka Server (Netflix OSS), Hystrix, MySQL database.
My first microservice addDistanceClient adds data to the database.
Here is an example controller:
#RequestMapping("/")
#RestController
public class RemoteMvcController {
#Autowired
EmployeeService service;
#GetMapping(path = "/show")
public List<EmployeeEntity> getAllEmployeesList() {
return service.getAllEmployees();
}
}
Here I can use Mono/Flux, I think there will be no problems.
My second microservice is showDistanceClient - it is not directly connected to the database.
He has a method that calls the method (as described above) on the first microservice to retrieve data from the database.
It uses the Feign Client.
Second microservice controller:
#Controller
#RequestMapping("/")
public class EmployeeMvcController {
private ServiceFeignClient serviceFeignClient;
#RequestMapping(path = "/getAllDataFromAddService")
public String getData2(Model model) {
List<EmployeeEntity> list = ServiceFeignClient.FeignHolder.create().getAllEmployeesList();
model.addAttribute("employees", list);
return "resultlist-employees";
}
}
and ServiceFeignClient itself, with which we call the method on the first microservice, looks like this:
#FeignClient(name = "add-client", url = "http://localhost:8081/", fallback = Fallback.class)
public interface ServiceFeignClient {
class FeignHolder {
public static ServiceFeignClient create() {
return HystrixFeign.builder().encoder(new GsonEncoder()).decoder(new GsonDecoder()).target(ServiceFeignClient.class, "http://localhost:8081/", new FallbackFactory<ServiceFeignClient>() {
#Override
public ServiceFeignClient create(Throwable throwable) {
return new ServiceFeignClient() {
#Override
public List<EmployeeEntity> getAllEmployeesList() {
System.out.println(throwable.getMessage());
return null;
}
};
}
});
}
}
#RequestLine("GET /show")
List<EmployeeEntity> getAllEmployeesList();
}
It is working properly now. Those, if both microservices are OK, I get data from the database.
If the first microservice (addDistanceClient) is dead, then when I call the method on second microservice (showDistanceClient) to get data from the database through the first microservice (using Feign Client on second microservice), I get a page on which the spinner is spinning and the text that the service is unavailable, try again later. All perfectly.
My goal:
To do this using Spring Reactive (not sure if this will help me, but I think I'm thinking in the right direction) to make the message that the service is currently unavailable and the spinning spinner on the second microservice will automatically disappear and the data from the database will be displayed as soon as the first microservice (addDistanceClient) will come to life again (without re-sending the request, i.e. without reloading the page).
Will I be able to do this through Spring WebFlux ?
I know that a stream is used through Spring WebFlux, which itself will notify us if data appears in it, we do not need to resubmit the request here.
I started thinking about this and cannot figure out how to do this:
1) using Spring Reactive
In this case, I need to implement Flux/Mono into the MVC model in the second showDistanceClient microservice, which returns HTML. I don't understand how. I know how to do this with REST.
2) If the first item is incorrect, maybe I need to use a WebSocket for this ?
If so, please share useful links with examples. I will be very grateful.
Indeed, this topic is very interesting to me and I want to understand it.
I will be very grateful for your help. Thanks everyone!
UPDATED POST:
I updated both controllers with REST + WebFlux. Everything works for me.
The first addDistanceClient service and its controller:
#RestController
#RequestMapping("/")
public class BucketController {
#Autowired
private BucketRepository bucketRepository;
// Get all Bucket from the database (every 1 second you will receive 1 record from the DB)
#GetMapping(value = "/stream/buckets/delay", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Bucket> streamAllBucketsDelay() {
return bucketRepository.findAll().delayElements(Duration.ofSeconds(5));
}
}
He pulls out all the records from the database with an interval of 5 seconds each record. I added an interval for an example to test.
The second service is showDistanceClient and its controller.
Here I used WebClient instead of Feign Client.
#RestController
#RequestMapping("/")
public class UserController {
#Autowired
private WebClient webClient;
#Autowired
private WebClientService webClientService;
// Using WebClient
#GetMapping(value = "/getDataByWebClient",produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Bucket> getDataByWebClient() {
return webClientService.getDataByWebClient();
}
}
and its Service layer (WebClientService):
#Service
public class WebClientService {
private static final String API_MIME_TYPE = "application/json";
private static final String API_BASE_URL = "http://localhost:8081";
private static final String USER_AGENT = "User Service";
private static final Logger logger = LoggerFactory.getLogger(WebClientService.class);
private WebClient webClient;
public WebClientService() {
this.webClient = WebClient.builder()
.baseUrl(API_BASE_URL)
.defaultHeader(HttpHeaders.CONTENT_TYPE, API_MIME_TYPE)
.defaultHeader(HttpHeaders.USER_AGENT, USER_AGENT)
.build();
}
public Flux<Bucket> getDataByWebClient() {
return webClient.get()
.uri("/stream/buckets/delay")
.exchange()
.flatMapMany(clientResponse -> clientResponse.bodyToFlux(Bucket.class));
}
}
Now everything works in a reactive environment. Fine.
But my problem remained unresolved.
My goal: everything works, everything is fine, and if I suddenly called on the second service a method that using WebClient called the first service to get the data, and at that moment my first service died, I received a message that the service is temporarily unavailable and then my first service My request for data was revived and I received all the data and instead of reporting that the service was temporarily unavailable I would get all the data (important: without reloading the page).
How do I achieve this ?
I am building a Spring Cloud project (Brixton.M4 with Spring Boot 1.3.1) with Eureka, Zuul and FeignClient where I am trying to add multi tenancy support (Tenants are identified by subdomain : tenant1.myservice.com). To do so, I would like to somehow pass the original subdomain along requests that are forwarded from a service to the other via Feign but I can't seem to be able to find the right way to do it.
What I have is a client that exposes a #RestController which calls a #FeignClient to communicate with my backend which exposes server operations to the client through its own #RestController.
The #FeignClient using same interface as my #RestController on the server :
#FeignClient(name = "product")
public interface ProductService extends IProductService {
}
What I am currently trying to do is set a header in a RequestInterceptor :
#Component
public class MultiTenancyRequestInterceptor implements RequestInterceptor {
private CurrentTenantProvider currentTenantProvider;
#Autowired
public MultiTenancyRequestInterceptor(CurrentTenantProvider currentTenantProvider) {
this.currentTenantProvider = currentTenantProvider;
}
#Override
public void apply(RequestTemplate template) {
try {
template.header("TENANT", currentTenantProvider.getTenant());
} catch (Exception e) {
// "oops"
}
}
}
My provider class is a simple component where I'm trying to inject a request / session scope bean :
#Component
public class CurrentTenantProvider {
#Autowired
private CurrentTenant currentTenant;
//...
}
The bean (I tried both session and request scope) :
#Bean
#Scope(value = WebApplicationContext.SCOPE_SESSION, proxyMode = ScopedProxyMode.TARGET_CLASS)
public CurrentTenant currentTenant() {
return new CurrentTenant();
}
On the server, I use Hibernate multitenant provider that is supposed to catch the header value and use it to define which DB to connect to :
#Autowired
private HttpServletRequest httpRequest;
#Override
public String resolveCurrentTenantIdentifier() {
return httpRequest.getHeader("TENANT");
}
It seems the Feign call to the server is done in another thread and out of the incoming request scope, so i'm not sure how to pass that value along.
It all works fine when I hardcode the tenant value in the RequestInterceptor so I know the rest is working properly.
I have also looked at many other posts about Zuul "X-Forwaded-For" header and cannot find it in the request received on the server. I have also tried adding a ZuulFilter to pass host name to next request but what I see is that original request to the Client is picked up by the ZuulFilter and I can add but not when the Feign request is sent to the backend service even if I map it in zuul (i guess that is intended ?).
I am not really sure what's the next step and would appreciate some suggestions.
Hope that it's of any use for you but we're doing sth similar in Spring-Cloud-Sleuth but we're using a ThreadLocal to pass span between different libraries and approaches (including Feign + Hystrix).
Here is an example with the highlighted line where we retrieve the Span from the thread local: https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-core/src/main/java/org/springframework/cloud/sleuth/instrument/web/client/TraceFeignClientAutoConfiguration.java#L123