First of all, excuse my English, it's very bad. I am using MassTransit with Azure Service Bus for asynchronous communication between microservices. By their own definition, and to avoid generating dependencies between them, messages sent between different microservices are defined in each of them, that is, they are part of different namespaces. The automatic management of MassTransit causes queues and topics to be managed by the object type, which prevents the microservices that consume a message from receiving the messages sent by the microservice publisher. The same thing happens with two classes with the same properties in the same namespace but with a different class name.
Is there any way to solve this?
The options that have occurred to me are:
Remove the namespace from the endpoint of the destination address, naming it only with the name of the class.
That MassTransit can manage the creation of queues and topics based on the serialization of the object, instead of managing it based on the object type (perhaps through some type of wrapping object?)
I leave an example that I hope can help you in understanding the problem.
//FIRST PROGRAM - MESSAGE CONSUMER
namespace Consumer
{
public class Example
{
public string PropOne { get; set; }
public string PropTwo { get; set; }
}
public class ExampleConsumer :
IConsumer<Example>
{
public List<Example> ConsumedTestObjectList { get; } = new List<Example>();
//THIS METHOD NEVER CALL !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
public Task Consume(ConsumeContext<ExampleConsumer> context)
{
ConsumedTestObjectList.Add(context.Message);
return Task.CompletedTask;
}
}
public class ConsumerProgram
{
public static void Main()
{
var bus = Bus.Factory.CreateUsingAzureServiceBus(sbc =>
{
var host = sbc.Host("connectionString", h => {});
});
sbc.ReceiveEndpoint(host, e =>
{
e.Consumer<ConsumerProgram.Example>(context =>
{
return Console.Out.WriteLineAsync($"Message Received: {JsonConvert.SerializeObject(context.Message)}");
});
});
bus.Start(); // This is important!
Console.WriteLine("Press any key to exit");
Console.ReadKey();
bus.Stop();
}
}
}
//SECOND PROGRAM - MESSAGE PUBLISHER
namespace Publisher
{
public class Example
{
public string PropOne { get; set; }
public string PropTwo { get; set; }
}
public class PublisherProgram
{
public static void Main()
{
var bus = Bus.Factory.CreateUsingAzureServiceBus(sbc =>
{
var host = sbc.Host("connectionString", h => {});
});
bus.Start(); // This is important!
//send new instance of Publisher.Example
var example = new Example() { PropOne = "1", PropTwo = "2" };
bus.Publish(example);
Console.WriteLine("Press any key to exit");
Console.ReadKey();
bus.Stop();
}
}
}
Thank you very much.
regards
Borja
The message type, and the resulting name, are a key concept within MassTransit. If you want to avoid sharing assemblies between projects, that is fine, but you will need to match the entire interface (or class, in your case) name, including namespace, or it will not route properly.
Yes, you can override the entity name formatter to change how topics are named but it won't change the message type requirement for deserialization of the message (which happens, by type).
So the recommendation here is to use the same namespace for the contracts, even if they're in separate projects.
Related
I'm working on a new project that uses CosmosDB and Entity Framework Core (via the Microsoft.EntityFrameworkCore.Cosmos NuGet package, version 5.0.7; the project itself is .NET Core 5). I'm new to both, and running into an issue I can't sort out.
In short, I need to save a complex object to the database. It's a big model that will have multiple collections of classes underneath it, each with their own properties and some with collections underneath them as well. I'm trying to configure EF with OwnsOne and OwnsMany to store these child objects underneath the top-level one. The code compiles, and will save to the database so long as all the owned objects are left empty. But whenever I put anything into an owned object, either with OwnsOne or OwnsMany, I get a pair of NullReferenceExceptions.
I've tried to strip my code down to the very basics. Here's how it currently looks.
Owner and owned classes:
public class Questionnaire
{
// Constructors
private Questionnaire() { }
public Questionnaire(Guid id)
{
Test = "Test property.";
TV = new TestQ();
Id = id;
}
public Guid Id { get; set; }
public string Test { get; set; }
public TestQ TV { get; set; }
// Public Methods
public void AddForm(Form f)
{
// not currently using this method
//Forms.Add(f);
}
}
public class TestQ
{
public TestQ()
{
TestValue = "test ownsone value";
}
public string TestValue { get; set; }
}
DbContext:
public class QuestionnaireDbContext : DbContext
{
public DbSet<Questionnaire> Questionnaires { get; set; }
public QuestionnaireDbContext(DbContextOptions<QuestionnaireDbContext> options) : base(options) { }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasDefaultContainer(nameof(Questionnaires));
modelBuilder.Entity<Questionnaire>().HasKey(q => q.Id);
modelBuilder.Entity<Questionnaire>().OwnsOne(q => q.TV);
}
}
And the code from the service that calls the dbContext (note that this is based on a generic service that I didn't set up originally). The actual exceptions are thrown here.
public virtual TEntity Add(TEntity entity)
{
_context.Entry(entity).State = EntityState.Added;
_context.SaveChanges();
return entity;
}
Ultimately I need this to work with OwnsMany and a collection, but I figured it might be simpler to get it working with OwnsOne first. The key thing to note here is that if I comment out the line
TV = new TestQ();
in the Questionnaire class, the model persists correctly into CosmosDB. It's only when I actually instantiate an owned entity that I get the NullReferenceExceptions.
Any advice would be much appreciated! Thank you!
Well, I'm not sure why this is the case, but the issue turned out to be with how we were adding the document. Using this generic code:
public virtual async Task<TEntity> Add(TEntity entity)
{
_context.Entry(entity).State = EntityState.Added;
await _context.SaveChanges();
return entity;
}
was the issue. It works just fine if I use the actual QuestionnaireDbContext class like so:
context.Add(questionnaire);
await context.SaveChangesAsync();
I have a series of class libraries that are used in asp.net-core middleware, and in an IHostedService.
To fetch the user context, I can inject IHttpContextAccessor to grab the HttpContext user:
public class MyLibrary
{
public MyLibrary(IHttpContextAccessor accessor)
{
// set the accessor - no problem
}
public async Task DoWorkAsync(SomeObject payload)
{
// get the user from the accessor
// do some work
}
}
To be a little more abstract, I have an IUserAccessor with an HttpUserAccessor implementation:
public class HttpUserAccessor: IUserAccessor
{
IHttpContextAccessor _httpaccessor;
public HttpUserAccessor(IHttpContextAccessor accessor)
{
_httpaccessor = accessor;
}
public string GetUser()
{
// return user from _httpaccessor
}
}
and then MyLibrary does not need an IHttpContextAccessor dependency:
public class MyLibrary
{
public MyLibrary(IUserAccessor accessor)
{
// set the accessor - no problem
}
public async Task DoWorkAsync(SomeObject payload)
{
// get the user from the accessor
// do some work
}
}
My IHostedService is popping message from a queue, where the message includes:
a user context, and
a serialized SomeObject to pass to MyLibrary.DoWorkAsync
So, something like:
public class MyHostedService : IHostedService
{
IServiceScopeProvider _serviceScopeFactory;
public MyHostedService(IServiceScopeFactory serviceScopeFactory)
{
_serviceScopeFactory = servicesScopeFactory;
}
public Task StartAsync(CancellationToken cancellationToken)
{ ... }
public Task StopAsync(CancellationToken cancellationToken)
{ ... }
public async Task ExecuteAsync(CancellationToken stoppingToken)
{
foreach (var message in queue)
{
using (var scope = _serviceScopeFactory.CreateScope())
{
// todo: tell IUserAccessor what message.User is!
var payload = // create a SomeObject from the queue message
var mylibrary = _services.GetRequiredService<MyLibrary>();
await myLibrary.DoWorkAsync(payload);
}
}
}
}
So, my question is, how does MyHostedService store message.User in such a way that a custom IUserAccessor can access it in a thread-safe manner via DI?
how does MyHostedService store message.User in such a way that a custom IUserAccessor can access it in a thread-safe manner via DI?
The thing you're looking for is AsyncLocal<T> - it's like a thread-local variable but scoped to a (possibly asynchronous) code block instead of a thread.
I tend to prefer a "provider" + "accessor" pairing for this: one type that provides the value, and a separate type that reads the value. This is logically the same thing as a React Context in the JS world, though the implementation is quite different.
One tricky thing about AsyncLocal<T> is that you need to overwrite its value on any change. In this case, that's not really a problem (no message processing will want to update the "user"), but in the general case it's important to keep in mind. I prefer storing immutable types in the AsyncLocal<T> to ensure they aren't mutated directly instead of overwriting the value. In this case, your "user" is a string, which is already immutable, so that's perfect.
First, you'll need to define the actual AsyncLocal<T> to hold the user value and define some low-level accessors. I strongly recommend using IDisposable to ensure the AsyncLocal<T> value is unset properly at the end of the scope:
public static class AsyncLocalUser
{
private static AsyncLocal<string> _local = new AsyncLocal<string>();
private static IDisposable Set(string newValue)
{
var oldValue = _local.Value;
_local.Value = newValue;
// I use Nito.Disposables; feel free to replace with another IDisposable implementation.
return Disposable.Create(() => _local.Value = oldValue);
}
private static string Get() => _local.Value;
}
Then you can define a provider:
public static class AsyncLocalUser
{
... // see above
public sealed class Provider
{
public IDisposable SetUser(string value) => Set(value);
}
}
and the accessor is similarly simple:
public static class AsyncLocalUser
{
... // see above
public sealed class Accessor : IUserAccessor
{
public string GetUser() => Get();
}
}
You'll want to set up your DI to point IUserAccessor to AsyncLocalUser.Accessor. You can also optionally add AsyncLocalUser.Provider to your DI, or you can just create it directly.
Usage would go something like this:
foreach (var message in queue)
{
using (var scope = _serviceScopeFactory.CreateScope())
{
var userProvider = new AsyncLocalUser.Provider(); // (or get it from DI)
using (userProvider.SetUser(message.User))
{
var payload = // create a SomeObject from the queue message
var mylibrary = _services.GetRequiredService<MyLibrary>();
await myLibrary.DoWorkAsync(payload);
}
}
}
I have a base class IntegrationEvent and all other business events inherited from that.
public abstract class IntegrationEvent
{
public Guid Id { get; private set; }
public DateTimeOffset OccuredOn { get; private set; }
protected IntegrationEvent()
{
this.Id = Guid.NewGuid();
this.OccuredOn = DateTimeOffset.Now;
}
}
public sealed class StudentRegisteredIntegrationEvent : IntegrationEvent
{
public Guid StudentId { get; set; }
public string FullName { get; set; }
public StudentRegisteredIntegrationEvent(Guid studentId, string fullName)
{
StudentId = studentId;
FullName = fullName;
}
}
then I've created a consumer :
public sealed class StudentRegisteredConsumer: IConsumer<StudentRegisteredIntegrationEvent>
{
private readonly ILogger<StudentRegisteredConsumer> _logger;
public StudentRegisteredConsumer(ILogger<StudentRegisteredConsumer> logger)
{
_logger = logger;
}
public Task Consume(ConsumeContext<StudentRegisteredIntegrationEvent> context)
{
_logger.LogWarning("========== Message Received +==========================");
_logger.LogInformation($"Sending notification to {context.Message.FullName}");
_logger.LogWarning("========== Message Received +==========================");
return Task.CompletedTask;
}
}
In the producer side I have a List<IntegrationEvent> list and I publish them via IPublishEndpoint but it does not routed to the correct queue, instead it just creates another exchange, Sample.Abstraction.Domain:IntegrationEvent. How can I tell MassTransit not to use the base class and instead use the real type class? I also tried with ISendEndpointProvider but they are routed again to another queue, student-registered-integration-event_skipped queue, since there is no consumer available for the base class.
here is the log on the consumer side:
[00:33:21 DBG] Configuring endpoint student-registered-integration-event, Consumer: Sample.University.Notification.Consumers.StudentRegisteredConsumer
[00:33:21 DBG] Declare exchange: name: student-registered-integration-event, type: fanout, durable
[00:33:21 DBG] Declare exchange: name: Sample.IntegrationEvents:StudentRegisteredIntegrationEvent, type: fanout, durable
[00:33:21 DBG] Bind exchange: source: Sample.IntegrationEvents:StudentRegisteredIntegrationEvent, destination: student-registered-integration-event
[00:33:21 DBG] Declare queue: name: student-registered-integration-event, durable
[00:33:21 DBG] Bind queue: source: student-registered-integration-event, destination: student-registered-integration-event
[00:33:21 DBG] Prefetch Count: 16
[00:33:21 DBG] Consumer Ok: rabbitmq://localhost/wrapperizer/student-registered-integration-event - amq.ctag-nqSrJ0A5UQZCXg3tIr9Hfg
I have no clue how to configure, I also used ConsumerDefinition<StudentRegisteredConsumer> but to no avail, here is the code:
public sealed class StudentRegisteredConsumerDefinition : ConsumerDefinition<StudentRegisteredConsumer>
{
public StudentRegisteredConsumerDefinition()
{
const string eventName = nameof(StudentRegisteredIntegrationEvent);
var sanitized = KebabCaseEndpointNameFormatter.Instance.SanitizeName(eventName);
this.EndpointName = sanitized;
}
}
on producer side to get the uri for send endpoint:
var eventName = logEvt.IntegrationEvent.GetType().Name;
var sanitized = KebabCaseEndpointNameFormatter.Instance.SanitizeName(eventName);
var uri = new Uri($"exchange:{sanitized}");
var sender = await _sendEndpointProvider.GetSendEndpoint(uri);
await sender.Send(logEvt.IntegrationEvent);
I know the above-code is kinda the default behavior of MT, but without that, I have no correct queues and exchanges. any solution will be appreciated.
First, you can do this entirely with Publish, there is no need to configure all of the things you've done to try to work around the issue. You can configure your consumers by convention, and let them get configured on their own endpoints. The one part you missed is the way the messages are published.
From your List<IntegrationEvent>, my suspicion is that you were calling Publish<T>(T message) where T was inferred to be IntegrationEvent. This is why you're only getting messages on that exchange. You need to use the Publish(object message) overload, so that the appropriate type is determined.
You can simply assign each message in the list to object, and then call Publish with that object. Or, you can force the use of the overload:
await Task.WhenAll(events.Select(x => bus.Publish((object)x, x.GetType()));
That way, MassTransit will use the object type to call the appropriate generic overload.
I got (more than) two Api POST endpoints. Each one needs a json as parameter. But when I use the same class name Payload in two endpoint argument classes, Swagger does not work. When I change one of it e.g. from Payload to Payload1 than it works.
Of course I set the right namespaces into the wrapper classes so it finds it Payload. But I would love to use the same name "Payload" each time. How can I use the same class name Payload?
I can keep the json name "Payload" at both cases and just set different names for the property ("Payload1", "Payload2"). It works. But would be nice to have same property names too.,
Endpoint A
[HttpPost()]
public async Task PostPerson([FromBody]JsonWrapperA jsonWrapperA)
namespace myProject.A
{
public class JsonWrapperA
{
[JsonProperty("name", Required = Required.AllowNull)]
public string Name { get; set; }
[JsonProperty("payload", Required = Required.AllowNull)]
public Payload Payload { get; set; }
}
public class Payload
{
[JsonProperty("value", Required = Required.AllowNull)]
public double Value { get; set; }
}
}
Endpoint B
[HttpPost()]
public async Task PostCompagn([FromBody]JsonWrapperB jsonWrapperB)
namespace myProject.B
{
public class JsonWrapperB
{
[JsonProperty("compagny", Required = Required.AllowNull)]
public string Compagny { get; set; }
[JsonProperty("payload", Required = Required.AllowNull)]
public Payload Payload { get; set; }
}
public class Payload
{
[JsonProperty("age", Required = Required.AllowNull)]
public double Age{ get; set; }
}
}
By default swagger will attempt to build its Schema Ids for objects that are return types or parameter types for your APIs endpoints, and it will display these objects in the "Models" section of the documentation. It will build these schema Ids based on the class names of the objects.
When you try to have two or more classes named the same, even though they are in different namespaces, then you will get the conflicting schemaIds error:
InvalidOperationException: Conflicting schemaIds: Identical schemaIds detected for types NamespaceOne.MyClass and NamespaceTwo.MyClass. See config settings - "CustomSchemaIds" for a workaround
This means Swagger needs to be configured to change the way it generates its SchemaIds. You can simply tell swagger to use an objects fully qualified name which will include namespaces in the schemaIds. You can do this in your Startup.cs file in the ConfigureServices method like this:
//add using statement for Swagger in Startup.cs
using Swashbuckle.AspNetCore.Swagger;
...
public void ConfigureServices(IServiceCollection services)
{
services.AddSwaggerGen(config =>
{
//some swagger configuration code.
//use fully qualified object names
config.CustomSchemaIds(x => x.FullName);
}
}
Using Swashbuckle.AspNetCore Version 5.5.1 i've had the same issue so i solved it using JustSomeDude answer, but afterwards all entities were shown with the full name so i needed a way to show only the name. This is what i did:
options.CustomSchemaIds(x => x.FullName); // Enables to support different classes with the same name using the full name with namespace
options.SchemaFilter<NamespaceSchemaFilter>(); // Makes the namespaces hidden for the schemas
Using this filter class:
public class NamespaceSchemaFilter : ISchemaFilter
{
public void Apply(OpenApiSchema schema, SchemaFilterContext context)
{
if (schema is null)
{
throw new System.ArgumentNullException(nameof(schema));
}
if (context is null)
{
throw new ArgumentNullException(nameof(context));
}
schema.Title = context.Type.Name; // To replace the full name with namespace with the class name only
}
}
All my SignalR clients connect using a JWT bearer token. I utilize the [Authorize] attribute in my SignalR Hub.
This token contains a userId which can be used to check if a user has read access on the resource through the resource's users property which contains a List<PuppyUserPermission> that look like this:
public class PuppyUserPermission
{
public string userId { get; set; }
public bool read { get; set; }
public bool write { get; set; }
}
The question is: how do I connect the dots here? Ideally, instead of something like
[Authorize]
public class PuppyHub : Hub
{
public async Task SendPuppy(Puppy pup)
{
await Clients.All.SendAsync(pup);
}
}
I would so something like the following (this is more pseudo code than anything else, as I don't use valid methods):
[Authorize]
public class PuppyHub : Hub
{
public async Task SendPuppy(Puppy pup)
{
var clients = Puppy.users.Where(u => u.read == true);
await clients.SendAsync(pup);
}
}
Basically, I'd like to ensure that the clients recieving the Puppy object via SignalR would be authorized users on the resource. Problem is, Clients is just a list of string client IDs, and I'm not sure how to go about tying them to actual users on my Puppy resource.
How do I go about achieving this?
From the beginning, I had the feeling that the answer lay in IUserIdProvider, but I didn't see how that would work for multiple users.
I finally found the answer, but it'll definitely need some cleanup.
First, create your own implementation of IUserIdProvider as follows:
public class MyUserIdProvider : IUserIdProvider
{
public string GetUserId(HubConnectionContext connection)
{
var username = connection.User.Claims.Where(x => x.Type == "THE_CLAIM_YOU_WANT_TO_USE_TO_IDENTIFY_USERS").First().Value;
return username;
}
}
Next, register it using DI:
services.AddSingleton<IUserIdProvider, MyUserIdProvider >();
Now, when you want to send events from the server, use DI in your constructor to pull down an instance of your SignalR Hub as per usual:
private IHubContext<PuppyHub> puppyHub { get; }
public UsersController(IHubContext<PuppyHub> _puppyHub)
{
puppyHub = _puppyHub;
}
Then, where when you want to tell your clients about the new Puppy:
// ... typical controller code
// assume we have a var, puppy, with a list of authorized users
// use System.Linq to get a list of userIds where the user is authorized to read the puppy
var authorizedUsers = (IReadOnlyList<string>)puppy.users.Where(x => x.permissions.read == true).Select(i => i._id).ToList();
// send the new puppy to the authorized users
await puppyHub.Clients.Users(authorizedUsers).SendAsync("SendPuppy", puppy);
And viola! You have now done resource-based authorization with SignalR.