Is it a good idea to store my SQL queries in a global resource file instead of having it in my codebehind? I know stored procedures would be a better solution but I don't have that luxury on this project.
I don't want queries all over my pages and thought a central repository would be a better idea.
Resource files are usually used for localization. But a string is just a string is just a string, and do you really want to be sending any old string in a resource file to your database?
I completely agree with others that you should be using linq or typed datasets, etc. Personally I've only had to resort to text queries a handful of times over the years, and when I do it's usually something like the following:
You set up a small framework and then all you need to do is maintain an Xml file. An single specific xml file is a lot easier to manage and deploy than a resource dll. You also have a well known place (repository) that stores Sql Queries and some metadata about them versus just some naming convention.
Never underestimate the utility of a (simple) class over a string literal. Once you've started using the class you can then add things down the road that you can't (easily) do with just a simple string.
Notepad compiler, so apologies if this isn't 100%. It's just a sketch of how everything interacts.
public static class SqlResource
{
private static Dictionary<string,SqlQuery> dictionary;
public static void Initialize(string file)
{
List<SqlQuery> list;
// deserialize the xml file
using (StreamReader streamReader = new StreamReader(file))
{
XmlSerializer deserializer = new XmlSerializer(typeof(List<SqlQuery>));
list = (List<SqlQuery>)deserializer.Deserialize(streamReader);
}
dictionary = new Dictionary<string,SqlQuery>();
foreach(var item in list )
{
dictionary.Add(item.Name,item);
}
}
public static SqlQuery GetQueryByName(string name)
{
SqlQuery query = dictionary[name];
if( query == null )
throw new ArgumentException("The query '" + name + "' is not valid.");
if( query.IsObsolete )
{
// TODO - log this.
}
return query;
}
}
public sealed class SqlQuery
{
[XmlAttributeAttribute("name")]
public bool Name { get; set; }
[XmlElement("Sql")]
public bool Sql { get; set; }
[XmlAttributeAttribute("obsolete")]
public bool IsObsolete { get; set; }
[XmlIgnore]
public TimeSpan Timeout { get; set;}
/// <summary>
/// Serialization only - XmlSerializer can't serialize normally
/// </summary>
[XmlAttribute("timeout")]
public string Timeout_String
{
get { return Timeout.ToString(); }
set { Timeout = TimeSpan.Parse(value); }
}
}
your xml file might look like
<?xml version="1.0" encoding="utf-8"?>
<ArrayOfSqlQuery xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<SqlQuery name="EmployeeByEmployeeID" timeout="00:00:30" >
<Sql>
SELECT * From Employee WHERE EmployeeID = #T0
</Sql>
</SqlQuery>
<SqlQuery name="EmployeesForManager" timeout="00:05:00" obsolete="true" >
<Sql>
SELECT * From Employee WHERE ManagerID = #T0
</Sql>
</SqlQuery>
</ArrayOfSqlQuery>
Ok, I'll try to answer again, now when I have more information.
I would make a query-class that hold all querystrings as shared properties or functions that could be named quite well to be easy to use.
I would look up strongly typed datasets with tableadapters and let the tableadapters handle all queries. When you are used with it you'll never go back.
Just add a dataset to your solution, add a connection, and a tableadapter for a table, then start build all querys (update, select, delete, search and so on) and handle it easy in code behind.
I am in the same situation with some developers preferring to write the queries in the resource file. We are using subsonic and I would prefer to use stored procedures rather then using direct queries.
One option, even though it is bad is to place those queries in a config file and read when needed but this is a very bad option and we may use it if everyone cannot be agreement of using the stored procedures.
You could use the XML config file to associate names with stored procedures too. I'm doing that for a current C# project. The "query" would define what procedure to call.
Since some database engines don't support stored queries, that's not always an option.
Sometimes for small projects, it's OK to use parameterized SQL queries (don't concatenate string). This is especially true for select statements.
Views can also be used for selects instead of stored procedures.
Rob
Related
I want to extend the documents that I receive from a SELECT clause.
Lets assume a I have a collection that stores documents in the following shape
{"foo": "yeah I am a foo", "bar": "And I am a bar"}
so that the query
SELECT * FROM f
would return the above document(s)
Now I want to add an additional property that is NOT part of the documents stored as part of the projection of the SELECT statement.
Basically I'd like to do something like using Javascript's spread operator (which is not possible in Cosmos DB)
SELECT {...*, "newprop": "oh! I am new here!"} FROM f
and which should then return document(s) like this
{"foo": "yeah I am a foo", "bar": "And I am a bar", "newprop": "oh! I am new here!"}
The one thing I DONT WANT TO DO is to repeat all the toplevel properties of my documents. So a solution in the form of
SELECT {"foo": f.foo, "bar":f.bar, "newprop": "oh! I am new here!"} FROM f
is not desired.
I also tried to get that done via a function. Which I was not able to do as I cant find out how to get the toplevel object / document handle within the SELECT clause.
I tried the following
SELECT udf.ExtendDocument(*) FROM f
SELECT udf.ExtendDocument($1) FROM f
SELECT udf.ExtendDocument(f) FROM f
SELECT udf.ExtendDocument(value) FROM f
most of which produced a syntax error
It's not possible to use SELECT *, then append columns to the projection.
One option you could explore is to add a static property and value to the class that you deserialize your data into.
For instance, you could create a class like this simple one for a person with a hardcoded property and default value. Then deserialize your query results into it with the static value added as another property with a default value.
class Person
{
[JsonProperty(PropertyName = "id")]
public string Id { get; set; }
[JsonProperty(PropertyName = "pk")]
public string Pk { get; set; }
[JsonProperty(PropertyName = "firstName")]
public string FirstName { get; set; }
[JsonProperty(PropertyName = "lastName")]
public string LastName { get; set; }
public string MyStaticColumn get; set; } = "Default Value";
}
Then the code to run the query...
public static async Task QueryPerson(Container container)
{
QueryDefinition query = new QueryDefinition("select * from c");
FeedIterator<Person> resultSet = container.GetItemQueryIterator<Person>(
query, requestOptions: new QueryRequestOptions()
{
MaxConcurrency = -1
});
List<Person> results = new List<Person>();
while (resultSet.HasMoreResults)
{
FeedResponse<Person> response = await resultSet.ReadNextAsync();
foreach(var p in response)
{
results.Add(p);
}
}
}
So I found a solution.
A) Build a user defined function that does the "Extension"
function extendProjection(x) {
var result = {
//usually one want to extend the returned doc
//with some new calculated properties and not
//with a static value
newprop: calculateNewPropFromObject(x)
}
return Object.assign(result, x)
}
B) Use the user defined function in your SELECT
SELECT VALUE udf.extendProjection(c) FROM c
//it is important to use the keyword "VALUE" eitherwise
//the resulting doc will look {$1: { //the extendedprojection }}
Having described that I would recommend against this approach
Your RUs will easily tripple. The reason seems to be the usage of the JS itself and not so much what the JS engine does.
its not possible to "reuse" different registered UDFs within your JS code.
So one has to copy code snippets
"Extended Properties" are not useable in your WHERE clause
Runtime error messages returned from Cosmos DB are horrible to decipher.
The lack of any decent development environment is basically a no go.
Like #mark-brown already answered, it does not seem to be possible.
I would just like to add that likely you shouldn't do that anyway and offer a workaround arguably better than the UDF (which is costly, hard-to-maintain, does not support multiple concurrent logic versions, etc).
If you want to add extra calculations to query output based on the same entire document, then it would make more sense to do it in business layer (after querying), not data layer (CosmsosDB queries). It would also be faster (less calculations, less data to move) and cheaper (less RU).
If you want to add static data (ex: a fix string or other constants), then the same argument applies - passing it back-and-forth to cosmosDB just makes things slower and costlier. That's not the responsibility of storage.
The workaround
If the goal is to query an entire CHILD object and add only a few selected properties from other areas of documents then its best not to try to flatten the object. Just keep your storage model objects and extras side-by-side, ex:
select c.childWithAllTheFutureChildren,
c.other.location.single.value as newProp
from c
If you really-really want to add some calculation/statics to query output then you could also still use the same pattern for entire document:
SELECT c as TheRealStoredThing,
'oh! I am new here!' as theNewProp
FROM c
Yes, it does require you to have a separate model on client side for this query, but that's a good clean practice anyway. And it's much simpler than using/maintaining UDFs.
Let's say we have a table in SQL represented in C# like this:
public class Product
{
public int ID { get; set; }
public string Name { get; set; }
public string Picture { get; set; } // filename of the picture, e.g. apple.jpg
public int CategoryID { get; set; }
}
Now we would query the database and retrieve the object, let's say with values like this:
ID = 1
Name = Yellow apple
Picture = apple.jpg
CategoryID = 25
All perfectly normal. The thing I'm meditating about at the moment is this: if I want to show a product, I need some additional info that wasn't queried from the database, like exact file path to the image, all we have is
apple.jpg
, but we need maybe something like
~/images/apple.jpg
So, I was thinking of 3 possibilities:
1.) add a new property to the class Product
public string PictureUrl
{
get
{
return "~/images/apple.jpg";
}
}
2.) specify the full url during performing of the presentation logic, let's say:
public void ShowProductDetails()
{
Product p = ProductRepo.GetProduct(id);
txtName.Text = p.Name;
imgPicture.ImageUrl = "~/images/" + p.Picture;
}
3.) use Decorator pattern
First approach seems wrong to me (even though I have been using it for quite a long time), because I'm trying to have a layered web application. I'm not sure hard-coding this is a good way to go.
Second approach is better, but worse in the sense it can't be easily reused. If I have multiple places where I'm doing the same thing and something changes, ... Maybe it would work if I specify some static constants holding the paths...
Third possibility seems quite complicated in terms of maintainability. The number of my classes would probably have to double. If I have 30 classes now, it would suddenly become 60 :/
What is the best/recommended way of doing things like this? If I add properties to my POCOs that aren't included in the db schema, I'm unable to use Dapper.Contrib or Rainbow and similar libraries, because even though "selects" work fine, I can't "insert" nor "delete". I have to hard-code the sql strings for every command which becomes really tedious after some time, when you're doing all the time the same stuff.
EDIT:
The solution from Govind KamalaPrakash Malviya is great, but can't be used every time. I need a way to solve this for any type of properties, even those more complex ones - for instance the number of photos of some album. It's a good idea to query the count of photos along with albums, but assign it to what? Create a decorated class using a Decorator pattern?
How do YOU solve this kind of architecture problems?
I think you should manipulate it in presentation layer because image path for presentation layer only. so use third one but make it easy using utility method
public class PathUtility
{
public static string ImageUrl(string imageName)
{
if(string.IsNullOrEmpty(imageName))
{
throw new Exception("Image name not valid!!");
}
else
{
return "YourImageDirectroyUrl" + imageName;
}
}
}
and use it easily
PathUtility.ImageUrl("apple.jpg");
I normally solve this by leaving the entity object as it is and creating an extra data container, which will either hold a reference to the corresponding entity or implement the corresponding properties from the entity object itself. In the latter case I use a mapping library (AutoMapper) to copy data from an entity to a the enhanced container.
The logic for filling the extra properties normally lies in a factory (or factory method). It's up to you, where you want to place this in your architecture. In a current project we are including them in our data access facade on client side, because we don't want to clutter the data access layer with too many DTO's. This of course means, that the data access layer still needs to support retrieving the extra properties. In your case an operation like int GetNumberOfPhotosForAlbum(Album album).
We found that the benefits outweigh the risk of an ever-growing contract of the data access layer, which of course might need to support many different calls like the example above instead of just EnhancedAlbum GetEnhancedAlbumWithAllKindsOfExtraProperties(long albumId). This might also become a performance problem in some scenarios, because of the overhead of an increased frequency of service calls. In the end you need to decide, what's best for your project.
I like this approach, because my entities (Album) stay untouched and I retain a clear separation of concerns between persistence, client logic and mapping.
Example:
class Album
{
string Name { get; set; }
}
class EnhancedAlbum
{
Album Album { get; set; }
int NumberOfPhotos { get; set; }
}
class EnhancedAlbumFactory
{
private MyDataService _dataService;
//include some means of constructing or (better) injecting the data service
EnhancedAlbum GetEnhancedAlbum(Album album)
{
return new EnhancedAlbum
{
Album = Album,
NumberOfPhotos = _dataService.GetNumberOfPhotosForAlbum(album);
};
}
}
I have a member class that returned IQueryable from a data context
public static IQueryable<TB_Country> GetCountriesQ()
{
IQueryable<TB_Country> country;
Bn_Master_DataDataContext db = new Bn_Master_DataDataContext();
country = db.TB_Countries
.OrderBy(o => o.CountryName);
return country;
}
As you can see I don't delete the data context after usage. Because if I delete it, the code that call this method cannot use the IQueryable (perhaps because of deferred execution?). How to force immediate execution to this method? So I can dispose the data context..
Thank you :D
The example given by Codeka is correct, and I would advice writing your code with this when the method is called by the presentation layer. However, disposing DataContext classes is a bit tricky, so I like to add something about this.
The domain objects generated by LINQ to SQL (in your case the TB_Countries class) often contain a reference to the DataContext class. This internal reference is needed for lazy loading. When you access for instance list of referenced objects (say for instance: TB_Country.States) LINQ to SQL will query the database for you. This will also happen with lazy loaded columns.
When you dispose the DataContext, you prevent it from being used again. Therefore, when you return a set of objects as you've done in your example, it is impossible to call the States property on a TB_Country instance, because it will throw a ObjectDisposedException.
This does not mean that you shouldn't dispose the DataContext, because I believe you should. How you should solve this depends a bit on the architecture you choose, but IMO you basically got two options:
Option 1. Supply a DataContext to the GetCountriesQ method.
You normally want to do this when your method is an internal method in your business layer and it is part of a bigger (business) transaction. When you supply a DataContext from the outside, it is created outside of the scope of the method and it shouldn't dispose it. You can dispose it at a higher layer. In that situation your method basically looks like this:
public static IQueryable<TB_Country> GetCountriesQ(
Bn_Master_DataDataContext db)
{
return db.TB_Countries.OrderBy(o => o.CountryName);
}
Option 2. Don't return any domain objects from the GetCountriesQ method.
This solution is useful when the method is a public in your business layer and will be called by the presentation layer. You can wrap the data in a specially crafted object (a DTO) that contains only data and no hidden references to the DataContext. This way you have full control over the communication with the database and you can dispose the DataContext as you should. I've written more about his on SO here. In that situation your method basically looks like this:
public static CountryDTO[] GetCountriesQ()
{
using (var db = new Bn_Master_DataDataContext())
{
var countries;
from country in db.TB_Countries
orderby country.CountryName
select new CountryDTO()
{
Name = country.CountryName,
States = (
from state in country.States
order by state.Name
select state.Name).ToList();
};
return countries.ToArray();
}
}
public class CountryDTO
{
public string Name { get; set; }
public List<StateDTO> States { get; set; }
}
As you will read here there are some smart things you can do that make using DTOs less painful.
I hope this helps.
You can convert the queryable to a list, like so:
public static List<TB_Country> GetCountriesQ()
{
using(var db = new Bn_Master_DataDataContext())
{
return db.TB_Countries
.OrderBy(o => o.CountryName).ToList();
}
}
Does anyone know of examples which show how to encrypt a dataset at the client side and send it over to a web service and have it decrypted there?
Another question:
What i need to do is a send hundreds of rows of data from a client to a web service and have the web service update the database with these records. I can't think of any other way to do this without using a dataset. Is there a better method?
Thanks in advance!
As far as the encryption is concerned, why try to reinvent the wheel? Just connect to the webservice over SSL - it'll most likely be much safer than a homegrown alternative.
I would probably create a custom struct/object and send an array of those to the webservice rather than a DataSet. It will mean (slightly) less network traffic; it will make the webservice's WSDL more descriptive; and it will make it easier for any non-Microsoft apps to talk to the webservice, if that becomes necessary in the future.
EDIT: An example...
At the server-side you can declare a custom type (eg, ExampleUser), and then setup your method to accept an array of that type instead of a DataSet:
[WebService(Namespace="http://example.yourdomain.com/ExampleWebService/")]
public class ExampleWebService : System.Web.Services.WebService
{
// this is your custom type
public class ExampleUser
{
public int UserID { get; set; }
public string Name { get; set; }
public DateTime DateOfBirth { get; set; }
}
// this is your method
// accepts an array of ExampleUser rather than a DataSet
[WebMethod]
public void UploadUsers(ExampleUser[] usersArray)
{
// do something...
}
}
In the client application you would add a reference to the webservice. This will enable you to use the ExampleUser type declared in the server-side code above.
You could then just convert your DataSet to an array of ExampleUser objects before sending it to the webservice:
// get the number of rows in the DataTable
int rowCount = yourDataSet.Tables[0].Rows.Count;
// create an array of ExampleUser with the correct capacity
ExampleWebService.ExampleUser[] usersArray =
new ExampleWebService.ExampleUser[rowCount];
// iterate through each row in the table
for (int i = 0; i < rowCount; i++)
{
DataRow dr = yourDataSet.Tables[0].Rows[i];
// create an ExampleUser object and populate it from the DataRow columns
ExampleWebService.ExampleUser eu = new ExampleWebService.ExampleUser();
eu.UserID = (int)dr["User_ID"];
eu.Name = (string)dr["Name"];
eu.DateOfBirth = (DateTime)dr["Date_Of_Birth"];
// add the ExampleUser object to the array
usersArray[i] = eu;
}
// the array is populated so let's call the webservice
ExampleWebService.UploadUsers(usersArray);
EDIT: Another example...
If you're using .NET 3.5 then you can get the client-side down to just a few lines of code by using LINQ and object initialisers to create your array:
// create and populate the array
ExampleWebService.ExampleUser[] usersArray =
yourDataSet.Tables[0].AsEnumerable().Select
(
s => new ExampleWebService.ExampleUser()
{
UserID = (int)s["User_ID"],
Name = (string)s["Name"],
DateOfBirth = (DateTime)s["Date_Of_Birth"]
}
).ToArray();
// the array is populated so let's call the webservice
ExampleWebService.UploadUsers(usersArray);
Well, there are a lot of approaches to this. Since you are sending this over the wire, you could: 1) Write the data to an XML stream (this is very much what the DataSet is meant to do) then 2) you could compress the XML (the compression ratio would be best at this stage) then 3) Encrypt using one of the .NET cryptographic schemes and finally 4) decrypt, unzip, and deserialize your XML into a DataSet object or whatever you want to do with it.
Note, you might need to Base64 the result of the encryption. Another alternative is to not encrypt and use the Web Service over SSL and just use that native encryption. Depending on the type of data a DataSet may not be the best choice in terms of performance. You could send a CSV or JSON style data block; this would potentially be smaller, especially if there is only one "DataTable" object in your DataSet. A lot of this is situation dependent as far as what the best method do use is.
I have some e-commerce code that I use often that uses Linq To SQL for saving orders to the database. I want to remove the tightly coupled Linq to SQL bit and pass in an IRepository instead but I am still a little confused on things.
Say I have a GetCustomer() method on my ICustomerRepository that returns a Customer object.
Do I need it to really return an ICustomer object that gets passed back from that method so if I switch from Linq To SQL to say SubSonic it's not a problem?
I believe I do, if that is the case is there a way in Linq To SQL to easily convert my Linq To SQL Customer object to my ICustomer object like SubSonics ExecuteSingle(Of ) method?
If you want your Customer class to be a plain object with no attachment to LINQ, then you will most likely need to write a mapper method to convert your LINQ-based Customer object to your plain Customer domain object. LINQ to SQL does not have such functionality built-in.
I have begun to wrap my mapping methods in an extension method for readability, and it really helps to keep the Repository code simple. For instance, an example CustomerRepository method my look like:
public Customer GetById(int id)
{
return dataContext.LINQCustomers.Where(c => c.Id == id)
.Single()
.ToDomainObject();
}
and the ToDomainObject() method is defined in an extension method like:
public static class ObjectMapper
{
public static Customer ToDomainObject(this Customer linqObject)
{
var domainObject = null
if (linqObject != null)
{
domainObject = new Customer
{
Id = linqObject.Id,
FirstName = linqObject.FirstName,
LastName = linqObject.LastName
}
}
return domainObject;
}
}
or something similar. You can do the same to convert your domain object back to a LINQ object to pass back into your repository for persistence.
You can have it return a Customer as long as Customer is a plain old .NET object, and not some db-generated entity. Your Customer domain object should have no knowledge about how (or if) it might be persisted to a database, and this is what should be returned from your repository. In your repository you might have some mapping code - this is quite common - that maps from [however you get the data back from its storage location] to your domain object. If you're using Linq-to-sql then this mapping would be from the Linq-To-Sql generated Customer table (and perhaps other tables - your Customer domain object likely won't map 1:1 to a particular table in the database) to your Customer domain object, which would live in a different namespace (and most likely, assembly).
There is no need to make it an ICustomer at all. A repository acts in a way as to make it look as though your persistent instances are in memory.
public interface ICustomerRepository
{
Customer GetByID(int id);
IEnumerable<Customer> GetByName(string name);
Customer GetByVatCode(string vatCode);
}
Some people would additionally include methods such as
void Add(Customer newCustomer);
void Delete(Customer deleted);
void Update(Customer modified);
The latter method implementations would most likely just update a unit of work.
The concept though is that these are just common ways of asking for Customer instances, the repository acts as a way of asking for them without defining how to ask for them.