Unable to query by Timestamp in CosmosDB with Table API - .net-core

I'm creating a query to include Cosmos entities from the last 30 days:
var filter = TableQuery.GenerateFilterConditionForDate(
"Timestamp",
QueryComparisons.GreaterThanOrEqual,
DateTimeOffset.Now.Date.AddDays(-30));
Next I create a query using this filter:
var query = new TableQuery<ResponseEntity>().Where(filter);
Next I execute the query:
var result = await table.ExecuteQuerySegmentedAsync(query, null);
However, for some reason, the result always contains zero (0) hits.
If I execute the query without any filter...
var query = new TableQuery<ResponseEntity>();
...I do get all entities.
Looking at the generated filter string, it looks OK to me (and identical to the one in the Azure portal when using the query builder for Cosmos):
Timestamp ge datetime'2018-09-15T22:00:00.0000000Z'
Is there any limitation on querying based on Timestamp?
Edit: Tried switching to the new Microsoft.Azure.Cosmos.Table NuGet package (currently in preview, version 0.9.1), but I'm still not getting any results when filtering by Timestamp.

Please refer to my working code.
code:
using Microsoft.Azure.CosmosDB.Table;
using Microsoft.Azure.Storage;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace JayGongCosmosTable
{
class Program
{
static void Main(string[] args)
{
TableQuerySegment <ResponseEntity> resultE= QueryTableAsync("test").Result;
foreach(ResponseEntity re in resultE)
{
Console.WriteLine("Timestamp: "+re.Timestamp);
Console.WriteLine("------------------------------------------");
}
Console.WriteLine("execute done");
Console.ReadLine();
}
public static async Task<TableQuerySegment<ResponseEntity>> QueryTableAsync(string tableName)
{
CloudStorageAccount storageAccount = CreateStorageAccountFromConnectionString("***");
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference(tableName);
var filter = TableQuery.GenerateFilterConditionForDate(
"Timestamp",
QueryComparisons.GreaterThanOrEqual,
//QueryComparisons.LessThanOrEqual,
DateTimeOffset.Now.AddDays(-10).Date);
Console.WriteLine(filter);
var query = new TableQuery<ResponseEntity>().Where(filter);
var result = await table.ExecuteQuerySegmentedAsync(query, null);
return result;
}
}
class ResponseEntity : TableEntity
{
public string Name { get; set; }
public DateTimeOffset logtime { get; set; }
}
}
My data list as below without filter:
If I used Timestamp as filter, it works:
Another thing I'd like to mention is, please avoid timestamp interval query if possible. Such a query will result in a whole table scan in server side. If timestamp interval query is needed usually in your scenario, please consider choosing timestamp as your partition key or row key to optimize the query performance.
Just for summary, finally , the solution is uninstalling WindowsAzure.Storage and switch using statements to use types from Microsoft.Azure.CosmosDB.Table instead.

Related

Need help understanding API and LINQ

I'm trying to set up an API for a system I'm working on, but the LINQ seems to not grab the parameters.
A bit of background: During covid I've been working with a local business owner to develop an info system for his business. So far, everything has been kept in the browser, but now we want to create a small windows form application the users can download instead of using the browser. The application will be much smaller in scope than the full site, but I don't want the SQL connection in the form.
So I guess my first question is, am I being overly cautious at not wanting the SQL connector in the client and wanting them to connect to the database, via an API, or is it safe enough to add the connection and calls directly in the application (I know how to do this, it's the API part I can't figure out). I'm thinking about it from a security point of view - would the users be able to find the connection and potentially do harm to my database by having it straight in the application or is it safe there?
If using API calls is the proper way to go, that leads me to my second question. How do I configure it properly?
This is my table (I've been following the Microsoft ToDoItems tutorials):
CREATE TABLE [dbo].[TodoItems] (
[Id] [int] identity(1,1) NOT NULL,
[Name] [nvarchar](50) NULL,
[IsComplete] [bit] NULL,
[Secret] [nvarchar](10) NULL
) ON [PRIMARY]
GO
My test form has a single button, which when pressed calls this method:
static async Task RunAsync()
{
// Update port # in the following line.
client.BaseAddress = new Uri("https://localhost:7217/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
try
{
// Create a new product
TodoItem product = new TodoItem
{
Name = "Gizmo",
IsComplete = false,
Secret = "false"
};
var url = await CreateProductAsync(product);
Console.WriteLine($"Created at {url}");
// Get the product
product = await GetProductAsync(url.PathAndQuery);
ShowProduct(product);
// Update the product
Console.WriteLine("Updating IsCompleted...");
product.IsComplete = true;
await UpdateProductAsync(product);
// Get the updated product
product = await GetProductAsync(url.PathAndQuery);
ShowProduct(product);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
My ToDoItem class looks like this:
public class TodoItem
{
public long Id { get; set; }
public string Name { get; set; }
public bool IsComplete { get; set; }
public string Secret { get; set; }
}
My first issue is creating the ToDoItem. This method should do the trick:
static async Task<Uri> CreateProductAsync(TodoItem product)
{
HttpResponseMessage response = await client.PostAsJsonAsync(
"api/todoitems", product);
response.EnsureSuccessStatusCode();
// return URI of the created resource.
return response.Headers.Location;
}
However, when I run the method my API logs this error and nothing is posted to the database:
Executed DbCommand (46ms) [Parameters=[#p0='?' (DbType = Boolean), #p1='?' (Size = 4000), #p2='?' (Size = 4000)], CommandType='Text', CommandTimeout='30']
SET NOCOUNT ON;
INSERT INTO [TodoItems] ([IsComplete], [Name], [Secret])
VALUES (#p0, #p1, #p2);
SELECT [Id]
FROM [TodoItems]
WHERE ##ROWCOUNT = 1 AND [Id] = scope_identity();
The way I read this, and I might be wrong, the method CreateProductAsync (which gets a product with the values "Gizmo", false and "false") simply doesn't transfer the values to the API.
For reference, my API ToDoContext class look like this:
public class TodoContext : DbContext
{
public TodoContext(DbContextOptions<TodoContext> options)
: base(options)
{
}
public DbSet<TodoItem> TodoItems { get; set; } = null!;
}
Do I need to add something to this class? I'm wholly unfamiliar with both API and LINQ, but I did figure out that changing the table name to ToDoItems made the connection for me on its own.

ParitionKey extracted from document doesn't match the one specified in the header - C#

I'm new to CosmosDB and trying to figure out what's going on. I am using Microsoft.Azure.Cosmos NuGet package for development.
This is the line that creates my container:
Container = await database.CreateContainerIfNotExistsAsync(Program.ContainerId, "/id", 400)
This is my class:
public class REProperty
{
public const string PartitionKey = "id";
[JsonProperty(PropertyName = "id")]
public string Id { get; set; }
public string Number { get; set; }
public User Owner { get; set; }
And finally the code where I try to create a new document:
ItemResponse<REProperty> Response = await Program.Container.CreateItemAsync<REProperty>(C, new PartitionKey(REProperty.PartitionKey));
I am using the exact same PartitionKey everywhere yet I am still getting this error every time. Am I missing anything obvious?
Error message:
(Message: {"Errors":["PartitionKey extracted from document doesn't match the one specified in the header"]
You've defined the collection to use the id property as the partition key. The value given as id will then be the partition key used. However, you are specifying this:
ItemResponse<REProperty> Response = await Program.Container.CreateItemAsync<REProperty>(C, new PartitionKey(REProperty.PartitionKey));
This will always set the value "id" as the partition key, which is not correct. The actual value is different from document to document. So either you set it like this: new PartitionKey(C.Id) or you just omit the partition key part in the item creation - I think it should be enough to just have the property set, but give it a try to check it.

CosmosDb not using the ContractResolver provided when generating select queries

I have a project where I'm using CosmosDb (SQL API) as my database. It's a .Net Core project and I'm using the latest stable NuGet packages.
The document client is created as follows and use a custom contract resolver.
new DocumentClient(new Uri(settings.DatabaseUri), settings.DatabaseKey,
new JsonSerializerSettings
{
ContractResolver = new PrivateSetterCamelCasePropertyNamesContractResolver(),
Converters = new List<JsonConverter>
{
new EmailJsonConverter()
}
});
I have a collection called EmailAccount
public class EmailAccount : Entity
{
public string Email { get; private set; }
public string DisplayName { get; private set; }
public EmailAccount(DDD.Core.ValueObjects.Email email,
string displayName)
{
Email = email ?? throw new ArgumentNullException(nameof(email));
DisplayName = string.IsNullOrWhiteSpace(displayName) ? throw new ArgumentNullException(nameof(displayName)) : displayName;
}
}
All the properties are converted into camel-case when serialized which all works fine. But the problem is when I try to filter the documents. The SQL query that's generated looks something like this when I try to filter by the Email.
SELECT * FROM root WHERE (root["Email"] = "amila#iagto.com")
The problem is with the case of the property (Email). The property in the database is email but the query generator doesn't seem to be adhering to the ContractResolver provided and generates the above sql query which doesn't return any result.
If I put [JsonProperty("email")] above the Email property, the query is generated properly. Anyway to get the query generated properly without using attributes in the Entity class?
Any help much appreciated.
You need to set the JsonSerializerSettings at the CreateDocumentQuery level for the LINQ to SQL to pick it up.
This property was added in the SDK on 2.0.0+ versions.

C# Windows forms - Data Conduit to connect to a windows form app

Hi guys I need some help with this dataconduit. Its written by my tutor and we used it on a web development project. I am trying to use it for a windows forms application to connect to the database but i get the following error:
An attempt to attach an auto-named database for file C:\Users.... failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.
The data conduit works indeed if i use it on a asp.net website but not on a windows forms
i did try to researched but no luck.
I am just testing it with two text box and a save button
thank you
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Data.SqlClient;
using System.Data.OleDb;
using System.Data;
///This class uses the ado.net sql classes to provide a connection to an sql server database.
///it is free for use by anybody so long as you give credit to the original author i.e me
///Matthew Dean mjdean#dmu.ac.uk De Montfort University 2011
//you will need to modify the name of the namespace to suit your own program.
namespace MyClassLibrary
{
public class clsDataConduit
{
//connection object used to connect to the database
SqlConnection connectionToDB = new SqlConnection();
//data adapter used to transfer data to and from the database
SqlDataAdapter dataChannel = new SqlDataAdapter();
//ado.net class for building the sql commands
SqlCommandBuilder commandBuilder = new SqlCommandBuilder();
//stores a list of all of the sql parameters
List<SqlParameter> SQLParams = new List<SqlParameter>();
//data table used to store the results of the stored procedure
DataTable queryResults = new DataTable();
//data row used to store the data for a new record
DataRow newRecord;
//string variable used to store the connection string
private string connectionString;
public clsDataConduit()
{
//this is the constructor for the class
//you will need to modify this to suit your own database name and folder structure
//
//variable to store the patth to the database
string DbPath;
//variable to store the partial path and file name of your database
//modify this line to suit your own needs
string DatabaseName = "\\MyDatabase\\NamesData.mdf";
//set the DbPath concatenating the name of your database
DbPath = GetParentPath() + DatabaseName;
//build up the connection string for the sql server database
connectionString = "Data Source=.\\SQLEXPRESS;AttachDbFilename=" + DbPath + ";Integrated Security=True;User Instance=True";
}
private string GetParentPath()
///this function returns the path to the parent folder of the solution
{
//get the folder for the project
string DbPath = System.AppDomain.CurrentDomain.BaseDirectory;
//variable to store the position of the \\ characters
int Posn;
//loop through the path twice
for (int Counter = 0; Counter != 2; Counter++)
{
//find the right most instance of \\
Posn = DbPath.LastIndexOf("\\");
//split the path at that point
DbPath = DbPath.Substring(0, Posn);
//do it one more time
}
//return the new path
return DbPath;
}
public void AddParameter(string ParamName, string ParamValue)
///public method allowing the addition of an sql parameter to the list of parameters
///it accepts two parameters the name of the parameter and its value
{
//create a new instance of the sql parameter object
SqlParameter AParam = new SqlParameter(ParamName, ParamValue);
//add the parameter to the list
SQLParams.Add(AParam);
}
public void Execute(string SProcName)
{
///public method used to execute the named stored procedure
///accepts one parameter which is the name of the stored procedure to use
//open the stored procedure
//initialise the connection to the database
connectionToDB = new SqlConnection(connectionString);
//open the database
connectionToDB.Open();
//initialise the command builder for this connection
SqlCommand dataCommand = new SqlCommand(SProcName, connectionToDB);
//add the parameters to the command builder
//loop through each parameter
for (int Counter = 0; Counter < SQLParams.Count; Counter += 1)
{
//add it to the command builder
dataCommand.Parameters.Add(SQLParams[Counter]);
}
//set the command type as stored procedure
dataCommand.CommandType = CommandType.StoredProcedure;
//initialise the data adapter
dataChannel = new SqlDataAdapter(SProcName, connectionToDB);
//set the select command property for the data adapter
dataChannel.SelectCommand = dataCommand;
//use the copmmand builder to generate the sql insert delete etc
commandBuilder = new SqlCommandBuilder(dataChannel);
//fill the data adapter
dataChannel.Fill(queryResults);
//get the structure of a single record
newRecord = queryResults.NewRow();
//close the connection
connectionToDB.Close();
}
public void WriteToDatabase()
//void method that updates changes to the data adapter thus changing the database
{
//update any changes
dataChannel.Update(queryResults);
}
public DataRow NewRecord
///this method provides access to the new record as a single data row
{
get
{
//return the blank data row
return newRecord;
}
}
public void RemoveRecord(int Index)
//void method that removes a record at a specified index in the query results
{
//remove the record
queryResults.Rows[Index].Delete();
}
public void AddToDataTable()
//void method that adds the new record to the table data
{
//add the new record to the table
queryResults.Rows.Add(newRecord);
//re initialise the new record
newRecord = queryResults.NewRow();
}
public int Count
//property that returns the count of records in the query results
{
get
{
//return the count of the query results
return queryResults.Rows.Count;
}
}
public DataTable QueryResults
//public property that provides access to the query results
{
get
{
//return the query results
return queryResults;
}
set
{
//set the query results
queryResults = value;
}
}
}
}
this is the code for my name class.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace MyClassLibrary
{
public class clsName
{
private string firstName;
private string lastName;
public string FirstName
{
get
{
return firstName;
}
set
{
firstName = value;
}
}
public string LastName
{
get
{
return lastName;
}
set
{
lastName = value;
}
}
public void Save()
{
clsDataConduit Names = new clsDataConduit();
Names.Execute("sproc_tblNames_GetAll");
Names.NewRecord["FirstName"] = firstName;
Names.NewRecord["LastName"] = lastName;
Names.AddToDataTable();
Names.WriteToDatabase();
}
}
}
thank you guys but i finally managed to make it work. i am not sure if it is the right way to do it but what u have done is i have commented out the public clsDataConduit() {} method and before this method i have modified the connection string adding the full path of my database as follow: private string connectionString = (#"Data Source=.\SQLEXPRESS;AttachDbFilename=C:\Users...\NamesData.mdf;Integrated Security=True;Connect Timeout=30;User Instance=True");

ASP.NET MVC EF Calling MySQL Stored Procedure with Out Parameters

I am using Entity Framework ExecuteStoreQuery to call a MySql stored procedure with output parameter. The store procedure also returns a result set.
I get the following error.
OUT or INOUT argument 2 for routine mydb.mysp is not a variable or NEW pseudo-variable in BEFORE trigger
What is wrong? Is there a better way to call MySql sp with mix of out params and resultset?
Here is the code
var outParam = new MySqlParameter();
outParam.ParameterName = "totalRowsOut";
outParam.MySqlDbType = MySqlDbType.Int64;
outParam.Direction = ParameterDirection.Output;
var data = db.ExecuteStoreQuery<PropertySearchResultRow>
(
"call mysp(#input1, #totalRowsOut);",
new MySqlParameter("input1", 10),
outParam
);
var result = data.ToList();
int totalRowsCount = (int)outParam.Value;
Finally ended up using the ado.net MySqlCommand to call the stored proc and get the out value. Mysql and EF integration is bad.
You just need to specify the name of the stored procedure and pass in the parameters. Something like this
var data = db.ExecuteStoreQuery<PropertySearchResultRow>
(
"mysp",new MySqlParameter("input1", 10),outParam
);
Here is my sample code working with VS2012 and Mysql connector 6.8.3 hope this helps someone
public static IList<MyClass> GetCustOrderHist(string someParameter)
{
IList<MyClass> data = ((IObjectContextAdapter)TestDashboardEntities).ObjectContext.ExecuteStoreQuery<MyClass>("CALL CustOrderHist({0});", someParameter).ToList();
return data;
}
public class MyClass
{
public string ProductName { get; set; }
public int TOTAL { get; set; }
}

Resources