Response status code does not indicate success when inserting documents - azure-cosmosdb

Have have following partion id on my container:
/vesselId
I am trying to add a collection of this object:
public class CoachVessel
{
[JsonProperty("id")]
public string vesselId { get; set; }
[JsonProperty("imo")]
public long Imo { get; set; }
[JsonProperty("name")]
public string Name { get; set; }
}
This is my code to bulk insert the documents:
CosmosClientOptions options = new CosmosClientOptions() { AllowBulkExecution = true };
CosmosClient cosmosclient = new CosmosClient(connStr, options);
Container container = cosmosclient.GetContainer("CoachAPI", "Vessels");
List<Task> concurrentTasks = new List<Task>();
foreach (var vessel in vessels.Take(1))
{
concurrentTasks.Add(container.CreateItemAsync(vessel, new PartitionKey(vessel.vesselId)));
}
await Task.WhenAll(concurrentTasks);
I get following error that does not provide much information?
Microsoft.Azure.Cosmos.CosmosException: 'Response status code does not
indicate success: BadRequest (400); Substatus: 1001; ActivityId: ;
Reason: ();'
Any pointers to what causes this? This is my settings:
I have same problem when deleting documents:
CosmosClientOptions options = new CosmosClientOptions() { AllowBulkExecution = true, MaxRetryAttemptsOnRateLimitedRequests=1000};
CosmosClient cosmosclient = new CosmosClient(connStr, options);
Container container = cosmosclient.GetContainer("CoachAPI", "Vessels");
var allItemsQuery = container.GetItemQueryIterator<string>("SELECT * FROM c.id");
List<Task> concurrentDeleteTasks = new List<Task>();
while (allItemsQuery.HasMoreResults)
{
foreach (var item in await allItemsQuery.ReadNextAsync())
{
concurrentDeleteTasks.Add(container.DeleteItemAsync<string>(item, new PartitionKey("id")));
}
}
await Task.WhenAll(concurrentDeleteTasks.Take(3));
Throws following error:
'Response status code does not indicate success: NotFound (404); Substatus: 0;

The partition key must match a property in the document body. Change the partition key for the container to be, /id and fix your deletion code to correctly specify the partition key. E.g.,
concurrentDeleteTasks.Add(container.DeleteItemAsync<string>(item, new PartitionKey(item.Id)));

I'm from the CosmosDB Engineering Team. From your question, you've defined the partition key on the container as /vesselId, whereas the document has mapped the vesselId to the "id" property in the CoachVessel class. Is this intentional?
If the optional PartitionKey is specified in the CreateItemAsync API, then it needs to match the partition key in the Document. If you intended "id" to be the partition key, then you need to define your container's partition key as "id", not "vesselId". In this case, if the container's partition key is indeed /vesselId, the code expects a property "vesselId" in the input document set to the value vessel.vesselId specified in the partition key. It looks like the "vesselId" property is missing in your input document.

In my case the below error was because I'd updated the .Net SDK from v2 to v3, which no longer auto-generates IDs if one isn't passed.
Microsoft.Azure.Cosmos.CosmosException: 'Response status code does not
indicate success: BadRequest (400); Substatus: 1001; ActivityId: ;
Reason: ();'
I use the repository pattern, so just added a check before calling CreateItemAsync:
if (item.Id == null)
{
item.Id = Guid.NewGuid().ToString();
}

Related

How can I use a default value/model on WebAPI EmptyBody?

I have dotnet WebAPI and I'm trying to get a specific behaviour but am constantly getting 415 responses.
I have reproduced this by starting a new webapi project using dotnet new webapi on the command line. From there, I added two things: a new controller, and a model class. In my real project the model class is obviously a bit more complex, with inheritance and methods etc...
Here they are:
[HttpGet("/data")]
public async Task<IActionResult> GetModel(BodyParams input)
{
var response = new { Message = "Hello", value = input.valueOne };
return Ok(response);
}
public class BodyParams {
public bool valueOne { get; set; } = true;
}
My goal is that the user can call https://localhost:7222/data with no headers or body needed at all, and will get the response - BodyParams will be used with the default value of true. Currently, from postman, or from the browser, I get a 415 response.
I've worked through several suggestions on stack and git but nothing seems to be working for me. Specifically, I have tried:
Adding [FromBody(EmptyBodyBehavior = EmptyBodyBehavior.Allow)] into the controller, but this makes no difference unless I provide an empty {} json object in the body. This is not what I want.
Making BodyParams nullable - again, no change.
Adding .AddControllers(opt => opt.AllowEmptyInputInBodyModelBinding = true)... again, no change.
I Implemented the solution suggested here using the attribute modification in the comment by #HappyGoLucky. Again, this did not give the desired outcome, but it did change the response to : 400 - "The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true."
I tried modifying the solution in (4) to manually set context.HttpContext.Request.Body to an empty json object... but I can't figure out the syntax for this because it need to be a byte array and at that point I feel like I am way over complicating this.
How can I get the controller to use BodyParams with default values in the case that the user provides no body and no headers at all?
You can achieve that using a Minimal API.
app.MapGet("/data",
async (HttpRequest httpRequest) =>
{
var value = true;
if (Equals(httpRequest.GetTypedHeaders().ContentType, MediaTypeHeaderValue.Parse("application/json")))
{
var bodyParams = await httpRequest.ReadFromJsonAsync<BodyParams>();
if (bodyParams is not null) value = bodyParams.ValueOne;
}
var response = new {Message = "Hello", value};
return Results.Ok(response);
});
So, as there doesn't seem to be a more straightforward answer, I have currently gone with the approach number 5) from the OP, and just tweaking the code from there very slightly.
All this does is act as an action which checks the if the user has passed in any body json. If not, then it adds in an empty anonymous type. The behaviour then is to use the default True value from the BodyParams class.
The full code for the action class is:
internal class AllowMissingContentTypeForEmptyBodyConvention : Attribute, IActionModelConvention
{
public void Apply(ActionModel action)
{
action.Filters.Add(new AllowMissingContentTypeForEmptyBodyFilter());
}
private class AllowMissingContentTypeForEmptyBodyFilter : IResourceFilter
{
public void OnResourceExecuting(ResourceExecutingContext context)
{
if (!context.HttpContext.Request.HasJsonContentType()
&& (context.HttpContext.Request.ContentLength == default
|| context.HttpContext.Request.ContentLength == 0))
{
context.HttpContext.Request.ContentType = "application/json";
var str = new { };
//convert string to jsontype
var json = JsonConvert.SerializeObject(str);
//modified stream
var requestData = Encoding.UTF8.GetBytes(json);
context.HttpContext.Request.Body = new MemoryStream(requestData);
}
}
public void OnResourceExecuted(ResourceExecutedContext context)
{
// Do nothing
}
}
}
Then you can add this to any of your controllers using [AllowMissingContentTypeForEmptyBodyConvention]

Need help understanding API and LINQ

I'm trying to set up an API for a system I'm working on, but the LINQ seems to not grab the parameters.
A bit of background: During covid I've been working with a local business owner to develop an info system for his business. So far, everything has been kept in the browser, but now we want to create a small windows form application the users can download instead of using the browser. The application will be much smaller in scope than the full site, but I don't want the SQL connection in the form.
So I guess my first question is, am I being overly cautious at not wanting the SQL connector in the client and wanting them to connect to the database, via an API, or is it safe enough to add the connection and calls directly in the application (I know how to do this, it's the API part I can't figure out). I'm thinking about it from a security point of view - would the users be able to find the connection and potentially do harm to my database by having it straight in the application or is it safe there?
If using API calls is the proper way to go, that leads me to my second question. How do I configure it properly?
This is my table (I've been following the Microsoft ToDoItems tutorials):
CREATE TABLE [dbo].[TodoItems] (
[Id] [int] identity(1,1) NOT NULL,
[Name] [nvarchar](50) NULL,
[IsComplete] [bit] NULL,
[Secret] [nvarchar](10) NULL
) ON [PRIMARY]
GO
My test form has a single button, which when pressed calls this method:
static async Task RunAsync()
{
// Update port # in the following line.
client.BaseAddress = new Uri("https://localhost:7217/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
try
{
// Create a new product
TodoItem product = new TodoItem
{
Name = "Gizmo",
IsComplete = false,
Secret = "false"
};
var url = await CreateProductAsync(product);
Console.WriteLine($"Created at {url}");
// Get the product
product = await GetProductAsync(url.PathAndQuery);
ShowProduct(product);
// Update the product
Console.WriteLine("Updating IsCompleted...");
product.IsComplete = true;
await UpdateProductAsync(product);
// Get the updated product
product = await GetProductAsync(url.PathAndQuery);
ShowProduct(product);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
My ToDoItem class looks like this:
public class TodoItem
{
public long Id { get; set; }
public string Name { get; set; }
public bool IsComplete { get; set; }
public string Secret { get; set; }
}
My first issue is creating the ToDoItem. This method should do the trick:
static async Task<Uri> CreateProductAsync(TodoItem product)
{
HttpResponseMessage response = await client.PostAsJsonAsync(
"api/todoitems", product);
response.EnsureSuccessStatusCode();
// return URI of the created resource.
return response.Headers.Location;
}
However, when I run the method my API logs this error and nothing is posted to the database:
Executed DbCommand (46ms) [Parameters=[#p0='?' (DbType = Boolean), #p1='?' (Size = 4000), #p2='?' (Size = 4000)], CommandType='Text', CommandTimeout='30']
SET NOCOUNT ON;
INSERT INTO [TodoItems] ([IsComplete], [Name], [Secret])
VALUES (#p0, #p1, #p2);
SELECT [Id]
FROM [TodoItems]
WHERE ##ROWCOUNT = 1 AND [Id] = scope_identity();
The way I read this, and I might be wrong, the method CreateProductAsync (which gets a product with the values "Gizmo", false and "false") simply doesn't transfer the values to the API.
For reference, my API ToDoContext class look like this:
public class TodoContext : DbContext
{
public TodoContext(DbContextOptions<TodoContext> options)
: base(options)
{
}
public DbSet<TodoItem> TodoItems { get; set; } = null!;
}
Do I need to add something to this class? I'm wholly unfamiliar with both API and LINQ, but I did figure out that changing the table name to ToDoItems made the connection for me on its own.

ParitionKey extracted from document doesn't match the one specified in the header - C#

I'm new to CosmosDB and trying to figure out what's going on. I am using Microsoft.Azure.Cosmos NuGet package for development.
This is the line that creates my container:
Container = await database.CreateContainerIfNotExistsAsync(Program.ContainerId, "/id", 400)
This is my class:
public class REProperty
{
public const string PartitionKey = "id";
[JsonProperty(PropertyName = "id")]
public string Id { get; set; }
public string Number { get; set; }
public User Owner { get; set; }
And finally the code where I try to create a new document:
ItemResponse<REProperty> Response = await Program.Container.CreateItemAsync<REProperty>(C, new PartitionKey(REProperty.PartitionKey));
I am using the exact same PartitionKey everywhere yet I am still getting this error every time. Am I missing anything obvious?
Error message:
(Message: {"Errors":["PartitionKey extracted from document doesn't match the one specified in the header"]
You've defined the collection to use the id property as the partition key. The value given as id will then be the partition key used. However, you are specifying this:
ItemResponse<REProperty> Response = await Program.Container.CreateItemAsync<REProperty>(C, new PartitionKey(REProperty.PartitionKey));
This will always set the value "id" as the partition key, which is not correct. The actual value is different from document to document. So either you set it like this: new PartitionKey(C.Id) or you just omit the partition key part in the item creation - I think it should be enough to just have the property set, but give it a try to check it.

CosmosDB C# Gremlin - Getting exception when sending query

Taken from: https://learn.microsoft.com/en-us/azure/cosmos-db/create-graph-dotnet
Am getting a exception on the .wait() part:
NullReferenceException: Object reference not set to an instance of an object.
at Gremlin.Net.Driver.Connection.ReceiveAsync[T]()
at Gremlin.Net.Driver.Connection.SubmitAsync[T](RequestMessage requestMessage)
at Gremlin.Net.Driver.ProxyConnection.SubmitAsync[T](RequestMessage requestMessage)
at Gremlin.Net.Driver.GremlinClient.SubmitAsync[T](RequestMessage requestMessage)
at Gremlin.Net.Driver.GremlinClientExtensions.SubmitAsync[T](IGremlinClient gremlinClient, String requestScript, Dictionary`2 bindings)
Code:
private static string database = "db";
private static string collection = "col";
private static string hostname = "grem-test.gremlin.cosmosdb.azure.com";
public void test()
{
var gremlinServer = new GremlinServer(hostname, 443, enableSsl: true,
username: "/dbs/" + database + "/colls/" + collection,
password: authKey);
var gremlinClient = new GremlinClient(gremlinServer);
var grem = "g.V()";
var t = gremlinClient.SubmitAsync<dynamic>(grem);
t.Wait();
foreach (var result in t.Result)
{
// The vertex results are formed as dictionaries with a nested dictionary for their properties
string output = JsonConvert.SerializeObject(result);
Console.WriteLine(String.Format("\tResult:\n\t{0}", output));
}
It should be:
var task = gremlinClient.SubmitAsync<dynamic>(grem);
task.Wait();
Taken from Gremlin C# Samples:
// Create async task to execute the Gremlin query.
var task = gremlinClient.SubmitAsync<dynamic>(query.Value);
task.Wait();
I started with the sample application where it uses:
private static Task<ResultSet<dynamic>> SubmitRequest(GremlinClient gremlinClient, string query)
{
try
{
return gremlinClient.SubmitAsync<dynamic>(query);
}
catch (ResponseException e)
{
// They have extra stuff here for the request information that isn't relevant
throw;
}
}
I expanded from there and never had any issues besides the occasional exception from trying to run a query while another one was still working. I can only assume that running the queries this way works better than directly calling SubmitAsync().
One other thing I would recommend is double-checking your values for the server parameters just in case.

How to unit-test transactions in Entity Framework Core?

I have a method that does some work in a transaction:
public async Task<int> AddAsync(Item item)
{
int result;
using (var transaction = await _context.Database.BeginTransactionAsync())
{
_context.Add(item);
// Save the item so it has an ItemId
result = await _context.SaveChangesAsync();
// perform some actions using that new item's ItemId
_otherRepository.Execute(item.ItemId);
transaction.Commit();
}
return result;
}
I'd like to add unit tests to check that if _context.SaveChangesAsync or _otherRepository.Execute fail then the transaction is rolled back, is that possible?
I can't see a way to do that using InMemory or SQLite?
#Ilya Chumakov's excellent answer allowed me to unit test for the transaction. Our discussion in the comments then exposed some interesting points that I thought were worth moving into an answer so they'd be more permanent and easier to see:
The primary point is that the events logged by Entity Framework change dependent on the database provider, which surprised me. If using the InMemory provider you get just one event:
Id:1; ExecutedCommand
Whereas if you use Sqlite for the in-memory database you get four events:
Id:1; ExecutedCommand
Id:5; BeginningTransaction
Id:1; ExecutedCommand
Id:6; CommittingTransaction
I hadn't expected the events logged to change depending on the DB provider.
To anyone wanting to look into this more, I captured the event details by changing Ilya's logging code as follows:
public class FakeLogger : ILogger
{
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception,
Func<TState, Exception, string> formatter)
{
var record = new LogRecord
{
EventId = eventId.Id,
RelationalEventId = (RelationalEventId) eventId.Id,
Description = formatter(state, exception)
};
Events.Add(record);
}
public List<LogRecord> Events { get; set; } = new List<LogRecord>();
public bool IsEnabled(LogLevel logLevel) => true;
public IDisposable BeginScope<TState>(TState state) => null;
}
public class LogRecord
{
public EventId EventId { get; set; }
public RelationalEventId RelationalEventId { get; set; }
public string Description { get; set; }
}
And then I adjusted my code that returns an in-memory database so that I could switch in-memory DB provider as follows:
public class InMemoryDatabase
{
public FakeLogger EfLogger { get; private set; }
public MyDbContext GetContextWithData(bool useSqlite = false)
{
EfLogger = new FakeLogger();
var factoryMock = Substitute.For<ILoggerFactory>();
factoryMock.CreateLogger(Arg.Any<string>()).Returns(EfLogger);
DbContextOptions<MyDbContext> options;
if (useSqlite)
{
// In-memory database only exists while the connection is open
var connection = new SqliteConnection("DataSource=:memory:");
connection.Open();
options = new DbContextOptionsBuilder<MyDbContext>()
.UseSqlite(connection)
.UseLoggerFactory(factoryMock)
.Options;
}
else
{
options = new DbContextOptionsBuilder<MyDbContext>()
.UseInMemoryDatabase(Guid.NewGuid().ToString())
// don't raise the error warning us that the in memory db doesn't support transactions
.ConfigureWarnings(x => x.Ignore(InMemoryEventId.TransactionIgnoredWarning))
.UseLoggerFactory(factoryMock)
.Options;
}
var ctx = new MyDbContext(options);
if (useSqlite)
{
ctx.Database.EnsureCreated();
}
// code to populate the context with test data
ctx.SaveChanges();
return ctx;
}
}
Finally, in my unit test I made sure to clear the event log just before the assert part of my test to ensure I don't get a false positive due to events that were logged during the arrange part of my test:
public async Task Commits_transaction()
{
using (var context = _inMemoryDatabase.GetContextWithData(useSqlite: true))
{
// Arrange
// code to set up date for test
// make sure none of our setup added the event we are testing for
_inMemoryDatabase.EfLogger.Events.Clear();
// Act
// Call the method that has the transaction;
// Assert
var result = _inMemoryDatabase.EfLogger.Events
.Any(x => x.EventId.Id == (int) RelationalEventId.CommittingTransaction);
You could check EF Core logs for a RelationalEventId.RollingbackTransaction event type. I provided full details here:
How to trace an Entity Framework Core event for integration testing?
How it could look:
Assert.True(eventList.Contains((int)RelationalEventId.CommittingTransaction));
I think you are asking about how to rollback when a commit fails, EF core will auto rollback on if any of the statement failed
Read more here
, if you are asking for other reason or you want to do something when rollback happens, just to add try catch blocks,
using (var transaction = await
_context.Database.BeginTransactionAsync()){
try {
_context.Add(item);
// Save the item so it has an ItemId
result = await _context.SaveChangesAsync();
// perform some actions using that new item's ItemId
_otherRepository.Execute(item.ItemId);
transaction.Commit();
}
catch (Exception)
{
// failed, Do something
} }

Resources