I am moving my datamodel to Azure Elastic Scale.
After some testing and some experiences I fall in love if it, it is simple and with that kind of approach the code remains clean and easy to maintain.
I just have one big question, where is the Sharding key defined? I cannot find info on the sample downloaded from Visual Studio and I can beat that this is a straight forward answer.
In the sample offered by Microsoft the default sharding key is CustomerId but i cannot find where the reference to that key takes place.
Could it be in ShardMapName from configuration file?
Thanks in advance.
There is no explicit link between the the sharding key in the SQL schema and its usage (in code).
So in the Getting Started sample, the Customers and Orders table both contain a CustomerId column, and you can see that in DataDependentRoutingSample.cs when we access these tables we make sure to provide the same customerId value to the shardMap.OpenConnectionForKey method that we then use for customerId column (both in the SELECT and INSERT statements) in the following query.
// Looks up the key in the shard map and opens a connection to the shard
using (SqlConnection conn = shardMap.OpenConnectionForKey(customerId, credentialsConnectionString))
{
// Create a simple command that will insert or update the customer information
SqlCommand cmd = conn.CreateCommand();
cmd.CommandText = #"
IF EXISTS (SELECT 1 FROM Customers WHERE CustomerId = #customerId)
UPDATE Customers
SET Name = #name, RegionId = #regionId
WHERE CustomerId = #customerId
ELSE
INSERT INTO Customers (CustomerId, Name, RegionId)
VALUES (#customerId, #name, #regionId)";
cmd.Parameters.AddWithValue("#customerId", customerId);
cmd.Parameters.AddWithValue("#name", name);
cmd.Parameters.AddWithValue("#regionId", regionId);
cmd.CommandTimeout = 60;
// Execute the command
cmd.ExecuteNonQuery();
}
In other words, when you provide a certain key value in the OpenConnectionForKey call, it's your responsibility to make sure that all SQL queries with that connection are restricted to that key value, otherwise you may end up with incorrect results (e.g. if it was a SELECT query) or rows living on the wrong shard (e.g. if it was an INSERT query).
It's possible to fix this safety issue by using the new Row-Level Security feature. We have a sample called Entity Framework Multi-Tenant Shards that demonstrates how to combine Shard Maps with Row-Level Security. The relevant code is in ElasticScaleContext.cs:
SqlConnection conn = null;
try
{
// Ask shard map to broker a validated connection for the given key
conn = shardMap.OpenConnectionForKey(shardingKey, connectionStr, ConnectionOptions.Validate);
// Set CONTEXT_INFO to shardingKey to enable Row-Level Security filtering
SqlCommand cmd = conn.CreateCommand();
cmd.CommandText = #"SET CONTEXT_INFO #shardingKey";
cmd.Parameters.AddWithValue("#shardingKey", shardingKey);
cmd.ExecuteNonQuery();
return conn;
}
catch (Exception)
{
if (conn != null)
{
conn.Dispose();
}
throw;
}
Thanks for your great question!
Related
I am new in database using Delphi. Actually I need to display all the tables and their data in a gridview using firedac. Several videos on youtube and docwiki teaches how to access database through visual components i.e. throwing FDConnetion, FDTable, DataSource, etc. on the form.
But I would like to do the same thing programmatically so that I can focus more on coding and separate form from the business logic.
Here is what I have done in C# while accessing MSAccess data:
public void LoadUsersInfo()
{
try {
OleDbConnection Connection = new OleDbConnection();
Connection.ConnectionString = #"Provider=Microsoft.ACE.OLEDB.12.0;DataSource=dbCyberNet.accdb;Persist Security Info=False;";
Connection.Open();
OleDbCommand command = new OleDbCommand();
command.Connection = Connection;
String query = "SELECT * FROM tblUsersInfo";
command.CommandText = query;
OleDbDataAdapter adapter = new OleDbDataAdapter(command);
DataTable dt = new DataTable();
adapter.Fill(dt);
dbGrid.DataSource = dt;
}
catch (Exception ex) {
MessageBox.Show("Error occured while loading users information . " + ex.Message.ToString());
}
finally { Connection.Close(); }
}
I would like to do similar in delphi for sqlite database. Here is what I have tried (edited).
procedure TForm1.Button1Click(Sender: TObject);
var
con: TFDConnection;
table: TFDTable;
ds: TDataSource;
begin
con := TFDConnection.Create(nil);
con.LoginPrompt := False;
table := TFDTable.Create(nil);
table.TableName := 'Students';
ds := TDataSource.Create(nil);
ds.DataSet := table;
try
con.Open('DriverID=SQLite;Database=‪studentsDB.sqlite;');
table.Connection := con;
table.Active := True;
con.Connected := True;
DBGrid1.DataSource := ds;
finally
con.Close;
con.Free;
table.Free;
ds.Free;
end;
end;
Whenever I run this code, error dialog appears showing: First chance exception at $759AF097. Exception class ESQLiteNativeException with message '[FireDAC][Phys][SQLite] ERROR: no such table: STUDENTS'. Process Project1.exe (3836)
whereas the same table name exists inside the table and using visual firedac components, I can see the data grid fills easily.
You can do the same as with C#: assign your TFDQuery instance as DataSource of the VCL grid!
Add a TDataSource to the form, set its DataSet property to your TFDQuery, and assign it to your grid.
See the official Delphi documentation and other SO questions.
You can execute your Select query by calling the query's Open method so your code will look like this :-
procedure TStudents.executeQuery;
var
con: TFDConnection;
query: TFDQuery;
begin
con := TFDConnection.Create(nil);
query := TFDQuery.Create(con);
con.LoginPrompt := False;
con.Open('DriverID=SQLite;Database=studentsDB.sqlite;');
query.Connection := con;
query.SQL.Text := 'SELECT * FROM Students';
query.Open;
query.First;
While Not Query.EOF Do
Begin
// Populate your object or do whatever processing you like here.
Query.Next;
End;
end;
You can then programatically create a TDataSource and a Grid and hook those up in code in a similar fashion. You'll need to move the Connection and Query objects out of your object's executeQuery method as they are local variables and will not be accessible to anything outside that method.
Note that Delphi does not garbage collect like C# does, so as it stand, the code leaks memory.
As an aside, your code appears to be an object method and it looks like you're attempting to create a set of business objects that know how to load and save themselves. Rather than roll your own, you might be better served by looking at one of the existing frameworks that exist (tiOPF, InstantObjects, mORMot or TMS Aurelius) to achieve this. These will allow you to concentrate on solving the problem at hand and not worry about the minutae of creating, loading and saving of objects.
I have a login page, users have ID's and ID is the primary key in the table. I also have an admin account, and admin can create users. But when i create a user account with an existing ID, the web page crashes. I want to handle this situation and give a warning indicating that this ID exists and cannot be created. Here is my code:
public void CreateStudent(int ID, String status, String email, String firstName, String lastName, String password, String level, String program)
{
SqlConnection con = new SqlConnection(GetConnectionString());
string query1 = "insert into StudentTable(Name,Surname,ID,email,level,program,status,password,Type) values(#firstName,#lastName,#ID,#email,#level,#program,#status,#password,'Student')";
SqlCommand command = new SqlCommand(query1,con);
command.Parameters.AddWithValue("#firstName", firstName);
command.Parameters.AddWithValue("#lastName", lastName);
command.Parameters.AddWithValue("#ID", ID);
command.Parameters.AddWithValue("#email", email);
command.Parameters.AddWithValue("#level", level);
command.Parameters.AddWithValue("#program", program);
command.Parameters.AddWithValue("#status", status);
command.Parameters.AddWithValue("#password", password);
int result;
con.Open();
result = command.ExecuteNonQuery();
con.Close();
}
Can anyone help me with this? Thanls
There are multiple ways. You could catch an Exception and display an error message. That will also help in other error scenario's, like a lost connection.
However, if it's a situation you expect to occur during normal operation, you should handle the situation without an Exception. One way to do that is to have your insert only insert a row with a new id:
insert YourTable
(id, col1, col2, ...)
select #id
, #col1
, #col2
, ...
where not exists
(
select *
from YourTable
where id = #id
)
Pass parameters to your query like:
command.Parameters.AddWithValue("#id", 42);
command.Parameters.AddWithValue("#col1", "value1");
command.Parameters.AddWithValue("#col2", "value2");
Now ExecuteNonQuery() returns the number of affected rows. You can use that to check if the insert actually added a new row to the table:
var result = command.ExecuteNonQuery();
if (result == 1)
{
lblResult.Text = "New row inserted!";
lblResult.Color = Color.Green;
}
else
{
lblResult.Text = "Failed to insert new row.";
lblResult.Color = Color.Red;
}
Rather than tying to handle the error that comes back from attempting to add a user with the same ID, it would be better to check for the user's existence first and only create the new account if you can. Though you still need to handle the exception to cater for the case where two people try to create the same record.
So you'd have the following SQL:
select * from StudentTable where ID = #newID
and if this found a record you could report the error. If it doesn't find anything then you can go ahead and create the new record.
Though if you are using the ID as the primary key it would be better to have that as an Identity column on the table and have it auto-increment when you create a new row. You'd still have to check to see if the student already exists - use the e-mail as the human readable uniqueness condition.
If you add uniqueness constraint on the e-mail column then you'll be able to trap the case of two people trying to create the same record.
I have made the e-mail attribute as a primary key and then checked the database whether same e-mail exists and that worked
this is a method i was reading about #MSDN ,
my question is if for an example i would like to use it on a stored procedure
with the fact that the query of that stored procedure is already specifying columns to select from the table like following :
SELECT Columnsome, columnother, , , , ...FROM thisSQLdbTable
though i would like to implement the approach of that specific method , it seems very advanced from a little research i have made on
"the best way" available to extract data from SQL Server into Asp.net DataTable.
public static DataTable GetCustomerData(string dataSetName,
string connectionString)
{
DataTable table = new DataTable(dataSetName);
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlDataAdapter adapter = new SqlDataAdapter(
"SELECT CustomerID, CompanyName, ContactName FROM dbo.Customers", connection);
DataTableMapping mapping = adapter.TableMappings.Add("Table", "Customers");
mapping.ColumnMappings.Add("CompanyName", "Name");
mapping.ColumnMappings.Add("ContactName", "Contact");
connection.Open();
adapter.FillSchema(table, SchemaType.Mapped);
adapter.Fill(table);
return table;
}
}
or is it not the method to use if i am querying via SP that specifies the selected column
i could actually drop that stored procedure if it is not requiered to select /specify columns
the stored procedure is doing a specific calculation and updates the table with results of calculation then i am switching it's "MODE" to select results from the table that was updated.
what i did is recyceling (; giving it a parameter (bit type)
stored procedure then asks for the value of supplied bool / bit Parameter,
if its is status true it updates (doing its original task it was made for)
if its false its doing a select oporation so i am using it as i would with 2 separated commands
but now that i have search for a better way to extract data from db into a Data table
i give up on the two way SP and i will make a selection via the exaple above if they're not intended to be used thogether as with my current SP that does preselection when servs the GetCustomersData() above.
So the question is do i need to let the function to make the selection or can i serve it with my sp ready made selection to implemet it with GetCustomersData() in the way that it will only do rest of task and only mapp the columns that was preselected
Still a bit confused on your actual requirement but here goes:
I See you are using a direct query in your C# code, 'best way' would be to make a SP out of it then say:
SqlCommand command = conn.CreateCommand();
SqlDataAdapter sqlAdapter = new SqlDataAdapter(command);
command.CommandType = System.Data.CommandType.StoredProcedure;
command.CommandText = "sp_GetCustomerData";
Then after you have added parameters if needed do:
conn.Open();
sqlAdapter.Fill(dtResult);
conn.Close();
Where dtResult is Datatable.
So you do not need to do any mapping in this case, and since you are using a SP from the Database it will work faster than your direct query and you can change the query logic any time without the need of re deploying your code.
Stored procedures are perfectly valid in this use case. however, if you want more of a properly mapped table, you have several options, some of which go beyond the use of DataTables.
You can use strongly typed DataSets or perhaps use an ORM (object relational mapper).
ref: Typed Datasets: http://msdn.microsoft.com/en-us/library/esbykkzb(v=vs.71).aspx
ref: What is an ORM : http://en.wikipedia.org/wiki/Object-relational_mapping
EXAMPLES OF ORM'S
ref: Entity Framework : http://msdn.microsoft.com/en-us/data/ef.aspx
ref: NHibernate: http://nhforge.org/
I've been building this project as the solo dev for a while, and while I'm comfortable in the front end and middle tier, I don't really think I'm doing the database the way I should be, and the reason why is because I simply don't really know of any other way. The way I'm currently getting data is by testing out queries in my MySQL workbench and copying and pasting the SQL as a string literal into a method that makes a call to the DB, pulls the data and hydrates my objects.
This hasn't really been a problem until recently, when I had to create a monster of a query and it got me thinking that maybe there's a better way to do this. I don't have a formal DAL separated out, so I know there's room for improvement there, but I was curious about what the correct way would be to store SQL strings. I assume there is a tool somewhere built into VS10 where I can manipulate and work with SQL as SQL instead of as a string.
You should be doing this in stored procedures. That will basically format and store your query. You set parameters that are passed in from your code, then read out the results.
Example:
The C# method:
private void SetNote()
{
const string sql = "sp_SelectControllerNoteByID";
using (var conn = MocSystem.GetMocDbConnection())
{
using (var comm = new SqlCommand(sql, conn))
{
comm.CommandType = CommandType.StoredProcedure;
comm.Parameters.Add(new SqlParameter("#ControllerNoteID", ControllerNoteId));
try
{
conn.Open();
using (var rdr = comm.ExecuteReader())
{
while (rdr.Read())
{
CommentText = rdr["NoteText"].ToString();
_commentor = new User(new Guid(rdr["NoteAuthor"].ToString()));
CommentDate = (DateTime)rdr["NoteDate"];
MocRequestId = (int)rdr["MocRequestID"];
}
}
}
catch (Exception ex)
{
HasError = true;
ErrorMessage += "\nThere was a problem building the note: " + ex.Message;
}
}
}
}
The stored procedure on the DBMS (sql server in this example):
ALTER proc [dbo].[sp_SelectControllerNoteByID]
#ControllerNoteID int
AS
SELECT
ControllerNoteID,
NoteText,
NoteDate,
NoteAuthor,
MocRequestID
FROM
ControllerNotes
WHERE
ControllerNoteID = #ControllerNoteID
So here we call the stored procedure which in this case is just a simple select statement, then we read it out into an object via ADO. Now, this way, you can modify your query without recompiling. Unless you add parameters, in which case you'll have to update those in your code as well.
I'm developing an application in AIR via Flex, but I'm not seeing where I'm going wrong with SQLite (I'm used to MySQL). Parameters work, but only in certain instances. Is this part of the built-in sanitation system against sql injection? Thanks for any help!
Works:
sqlite
"INSERT :Fields FROM Category", where the parameter is :Fields = "*"
as3
var statement:SQLStatement = new SQLStatement();
statement.connection = connection;
statement.text = "INSERT :Fields FROM Category";
statement.parameters[":Fields"] = "*";
statement.execute;
Doesn't Work (SQL syntax error at ":Table"):
sqlite
"INSERT :Fields FROM :Table", where the parameters are :Fields = "*" and :Table = "Category"
as3
var statement:SQLStatement = new SQLStatement();
statement.connection = connection;
statement.text = "INSERT :Fields FROM :Table";
statement.parameters[":Fields"] = "*";
statement.parameters[":Table"] = "Category";
statement.execute;
Generally one cannot use SQL parameters/placeholders for database identifiers (tables, columns, views, schemas, etc.) or database functions (e.g., CURRENT_DATE), but instead only for binding literal values.
With server-side support for parameterized (a.k.a. prepared) statements, the DB engine parses your query once, remembering out the peculiars of any parameters -- their types, max lengths, precisions, etc. -- that you will bind in subsequent executions of the already-parsed query. But the query cannot be properly parsed into its syntactic elements if critical bits, like database objects, are unknown.
So, one generally has to substitute table names oneself, in a stored procedure or in client code which dynamically concats/interpolates/whatevers the SQL statement to be properly executed. In any case, please remember to use your SQL API's function for quoting database identifiers, since the API won't do it for you.
Not sure if this is the same but I ran across something similar in Java. Basically you can't add a table as a parameter so you must generate the statement like so:
var statement:SQLStatement = new SQLStatement();
statement.connection = connection;
statement.text = stringUtil.substitute("INSERT :Fields FROM {0}", "Category");
statement.parameters[":Fields"] = "*";
statement.execute;
This is mostly likely not the securest solution, so you might want to some custom validation of the data before you add the table name.. so someone doesn't try to send it the table name ";drop tableName..."