How can I store an array of doubles to database using Entity Framework Code-First with no impact on the existing code and architecture design?
I've looked at Data Annotation and Fluent API, I've also considered converting the double array to a string of bytes and store that byte to the database in it own column.
I cannot access the public double[] Data { get; set; } property with Fluent API, the error message I then get is:
The type double[] must be a non-nullable value type in order to use
it as parameter 'T'.
The class where Data is stored is successfully stored in the database, and the relationships to this class. I'm only missing the Data column.
You can do a thing like this :
[NotMapped]
public double[] Data
{
get
{
string[] tab = this.InternalData.Split(',');
return new double[] { double.Parse(tab[0]), double.Parse(tab[1]) };
}
set
{
this.InternalData = string.Format("{0},{1}", value[0], value[1]);
}
}
[EditorBrowsable(EditorBrowsableState.Never)]
public string InternalData { get; set; }
Thank you all for your inputs, due to your help I was able to track down the best way to solve this. Which is:
public string InternalData { get; set; }
public double[] Data
{
get
{
return Array.ConvertAll(InternalData.Split(';'), Double.Parse);
}
set
{
_data = value;
InternalData = String.Join(";", _data.Select(p => p.ToString()).ToArray());
}
}
Thanks to these stackoverflow posts:
String to Doubles array and
Array of Doubles to a String
I know it is a bit expensive, but you could do this
class Primitive
{
public int PrimitiveId { get; set; }
public double Data { get; set; }
[Required]
public Reference ReferenceClass { get; set; }
}
// This is the class that requires an array of doubles
class Reference
{
// Other EF stuff
// EF-acceptable reference to an 'array' of doubles
public virtual List<Primitive> Data { get; set; }
}
This will now map a single entity (here 'Reference') to a 'list' of your Primitive class. This is basically to allow the SQL database to be happy, and allow you to use your list of data appropriately.
This may not suit your needs, but will be a way to make EF happy.
It would be far easier if you use List<double> rather then double[]. You already have a table that stores your Data values. You probably have foreign key from some table to the table where your double values are stored. Create another model that reflects the table where doubles are stored and add foreign key mappings in the mappings class. That way you will not need to add some complex background logic which retrieves or stores values in a class property.
In my opinion almost all other answers work on the opposite of how it should be.
Entity EF should manage the string and the array must be generated from it. So the array must be whole read and written only when the string is accessed by EF.
A solution involving logic on Data[] is wrong because, as I wrote in a comment, you would run into paradoxical conditions. In all other conditions the variable must remain a pure array.
By putting the "get" and "set" logic in Data[], as I've seen so far, this happens:
1 - Every time an index access is made to the array, the array is automatically recreated from the string. This is a useless work, think of an iteration in a loop...
2 - when you go to set a single element it is not stored because it passes through "get" and not "set".
If you try to declare Data=new []{0,0,0} then set Data[1]=2 , going to re-read Data[1] the result is 0.
My solution is to completely turn the logic around.
public string Data_string
{
get => string.Join(';', Data??Array.Empty());
set => Data= value == null ? Array.Empty<double>() : Array.ConvertAll(value.Split(';',StringSplitOptions.RemoveEmptyEntries), double.Parse);
}
[NotMapped]
public double[] Data {get;set;}
Obviously this only applies to storing and retrieving data on databases, access to Data_string is exclusive to EF.
Once the string is read from the DB it is associated to Data_string which, through set, creates the Data array.
At this point you can work on Data without affecting the string in any way, like a normal array.
When you will ask EF to save in the DB, through the get in the Data_string property, the string will be completely reconstructed based on the Data elements and then stored as a string.
Practically the string is modified only twice, at the moment of reading from the DB and at the moment of saving.
In my opinion this solution is much more efficient than operating continuously on the string.
Nathan White has the best answer (got my vote).
Here is a small improvement over Joffrey Kern's answer to allow lists of any length (untested):
[NotMapped]
public IEnumerable<double> Data
{
get
{
var tab = InternalData.Split(',');
return tab.Select(double.Parse).AsEnumerable();
}
set { InternalData = string.Join(",", value); }
}
[EditorBrowsable(EditorBrowsableState.Never)]
public string InternalData { get; set; }
Don't use double[] use List insted.
Like this.
public class MyModel{
...
public List<MyClass> Data { get; set; }
...
}
public class MyClass{
public int Id { get; set; }
public double Value { get; set; }
}
All that solution that I see there are bad, because:
If you create table, you don't want to store data like this: "99.5,89.65,78.5,15.5" that's not valid! Firstly its a string that means if you can type letter into it and at the moment when your ASP.NET server call double.Parse it will result in FormatException and that you really don't want!
It's slower, because your server must parse the string. Why parse the string instead getting almost ready data from SQL Server to use?
i know this post is Ancient, but in case someone still needs to do something like this, PLEASE DO NOT USE THE ABOVE SOLUTIONS,
as the above solutions are EXTREMELY inefficient (Performance and Disk Space wise).., the best way is to store the array as a Byte array
public byte[] ArrayData;
[NotMapped]
public double[] Array {
get {
var OutputArray = new double[ArrayData.Length / 8];
for (int i = 0;i < ArrayData.Length / 8;i++)
OutputArray[i] = BitConverter.ToDouble(ArrayData, i * 8);
return OutputArray;
}
set {
var OutputData = new byte[value.Length * 8];
for (int i = 0;i < value.Length;i++) {
var BinaryValue = BitConverter.GetBytes(value[i]);
OutputData[(i*8)] = BinaryValue[0];
OutputData[(i*8)+1] = BinaryValue[1];
OutputData[(i*8)+2] = BinaryValue[2];
OutputData[(i*8)+3] = BinaryValue[3];
OutputData[(i*8)+4] = BinaryValue[4];
OutputData[(i*8)+5] = BinaryValue[5];
OutputData[(i*8)+6] = BinaryValue[6];
OutputData[(i*8)+7] = BinaryValue[7];
}
ArrayData = OutputData;
}
}
`
And if you need more performance, you can go for Unsafe code and use pointers .. instead of BitConverter ..
This is way better than saving double values (that can get huge) as string, then spliting the string array !! and then parsing the strings to double !!!
These getter/setters work on the whole array, but if you need to get just one item from the array, you can make a function that gets a single item from the array with a complexity of O(1) :
for Get :
public double Array_GetValue(int Index) {
return BitConverter.ToDouble(ArrayData, Index * 8);
}
for Set :
public void Array_SetValue(int Index, double Value) {
var BinaryValue = BitConverter.GetBytes(Value);
ArrayData[(Index*8)] = BinaryValue[0];
ArrayData[(Index*8)+1] = BinaryValue[1];
ArrayData[(Index*8)+2] = BinaryValue[2];
ArrayData[(Index*8)+3] = BinaryValue[3];
ArrayData[(Index*8)+4] = BinaryValue[4];
ArrayData[(Index*8)+5] = BinaryValue[5];
ArrayData[(Index*8)+6] = BinaryValue[6];
ArrayData[(Index*8)+7] = BinaryValue[7];
}
If your collection can be null or empty, and you want this to be preserved, do this:
[NotMapped]
public double[] Data
{
get => InternalData != null ? Array.ConvertAll(Data.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries), double.Parse) : null;
set => InternalData = value != null ? string.Join(";", value) : null;
}
Also, specify [Column(TypeName = "varchar")] on the string property for a more efficient storage data type.
A perfect enhancement to #Jonas's answer will be to add the necessary annotations. So, a cleaner version would be
[EditorBrowsable(EditorBrowsableState.Never)]
[JsonIgnore]
public string InternalData { get; set; }
[NotMapped]
public double[] Data
{
get => Array.ConvertAll(InternalData.Split(';'), double.Parse);
set
{
InternalData = string.Join(";", value.Select(p => p.ToString(CultureInfo.InvariantCulture)).ToArray());
}
}
The [JsonIgnore] Annotation will ignore the InternalData field from JSON serialization and Swagger UI.
[EditorBrowsable(EditorBrowsableState.Never)] will hide the public method from the IDE IntelliSense
// Create the scoresheet
Scoresheet scoresheet = new Scoresheet();
scoresheet.tsakID = task.ID;
// Query Items to retrieve a list of tasks. Then create a scoresheet
// for each item retrieved and bind them together as foreign keys.
var Items = (from c in db.Items
where c.ID == task.ID
select c).ToList();
// Save scoresheet item to db
scoresheet.ScoresheetItems = new List<ScoresheetItem>();
System.Diagnostics.Debug.WriteLine(scoresheet.ScoresheetID);
for (int i = 0; i < Items.Count(); i++)
{
ScoresheetItem scoresheetItem = new ScoresheetItem();
scoresheetItem.ScoresheetID = scoresheet.ScoresheetID;
scoresheet.ScoresheetItems.Add(scoresheetItem);
}
db.Scoresheets.Add(scoresheet);
db.SaveChanges();
The error is on the line where I am doing scoresheetItem.ScoresheetID = scorehseet.ScoresheetID
The problem is that scoresheet.ScoresheetID is null until it is created. when it is created on the database, the value is assigned as an auto-incrementing value.
How do I get around this so that scoresheet.ScoresheetID is not null so that I can assign it to the scoresheetItem?
Here is the scoresheet model:
public class Scoresheet
{
public int ScoresheetID { get; set; }
public virtual ICollection<ScoresheetItem> ScoresheetItems { get; set; }
}
public class ScoresheetItem
{
public int ScoresheetItemID { get; set; }
public int ScoresheetItemScore { get; set; }
public Nullable<int> ScoresheetID { get; set; }
}
If ScoresheetID is an identity column then you should not be setting it in code at all when creating it. On your object you need to let EF know that is an identity column as well. If this is code first, which I'm assuming, you can either use data annotations
https://msdn.microsoft.com/en-us/library/jj591583(v=vs.113).aspx
or fluent api to configure and map properties
https://msdn.microsoft.com/en-us/library/jj591617(v=vs.113).aspx
When creating child ScoresheetItem(s) you should not set the ScoresheetID either, your models should be configured for the relationship.
Here is one with modeling relationships and navigation properties.
https://msdn.microsoft.com/en-us/library/jj713564(v=vs.113).aspx
Okay, this was actually a pretty simple fault.
There is actually no need at all to set the foreign key such as I was doing right here: scoresheetItem.ScoresheetID = scorehseet.ScoresheetID
When I call scoresheet.ScoresheetItems.Add(scoresheetItem); in the code, it actually handles the foreign key for me.
We have a table with 50 rows and want display initially only 10. Just below the table we'd like to add a "Show all" link (not a button) that allow, dynamically (or after reloading the page), to display of all the 50 rows.
What is the most straightforward way to achieve this?
Thanks.
Consider having a model composed of an int and a List<DataRow>:
public class MyDisplayViewModel
{
public int FullCount { get; set; }
public List<DataRow> Data { get; set; }
}
When populating your Data object (in your controller, for example) you can take the full count of your existing data and pass it in the FullCount property.
public ActionResult GetData(int count)
{
MyDisplayViewModel model = new MyDisplayViewModel();
//pseudocode for getting the total number of records
model.FullCount = yourDataProvider(yourDataType).Count();
//pseudocode for getting the the list of records
model.Data = yourDataProvider(yourDataType, count);
return View(model);
}
You can check then in your view if the data passed to it corresponds to all the "available" data. If not, show the link to get all the data from your database.
#{
foreach(var row in Model.Data)
{
//display your data
}
if (Model.Data.Count != Model.FullCount)
{
#Html.ActionLink("Load all", "GetData", "YourControllerName",
new { count = Model.FullCount }, null)
}
}
Of course, this suggestion is at the conceptual level. the view, for example, can implement an Ajax request, in order to get only the remaining DataRows.
I am learning EF5 and building a small website which simply displays some songs and singers. As a song can be sung by more than one singer and a singer will have many songs so my EF model as below.
I want to display all the list of songs with its relevant singers in a table so I wrote a query and this is so far I have.
Dim res = context.Songs _
.SelectMany(Function(song) song.Artists, Function(s, a) New With
{.SongTitle = s.SongTitle, _
.ArtistName = a.ArtistName, _
.Lyrics = s.Lyrics})
But I am having the result like below.
You will see Lucky is displayed twice in the table. I don't want that to happen. I just want to display it once but have join two singers in the Artist column. I tried to read tutorials and many forum posts but those tutorials don't get this complicated.
So how can i get change the query to return something like this?
I must write my answer with C#, hopefully you are able to translate it into VB.
I would do two changes:
First, simply use Select instead of SelectMany in this situation.
Second, introduce a named ViewModel instead of an anonymous type because it allows you to add a method or custom readonly property that will be helpful later.
The ViewModel would look like this:
public class SongViewModel
{
public string SongTitle { get; set; }
public string Lyrics { get; set; }
public IEnumerable<string> ArtistNames { get; set; }
public string ArtistNamesString
{
get { return string.Join(", ", ArtistNames); }
}
}
Then you can use this query:
var res = context.Songs.Select(s => new SongViewModel
{
SongTitle = s.SongTitle,
Lyrics = s.Lyrics,
ArtistNames = s.Artists.Select(a => a.ArtistName)
});
Now, to list the result, you can use a loop like this (example with console output):
foreach (var item in res)
{
Console.WriteLine(string.Format("{0} {1} {2}",
item.SongTitle, item.Lyrics, item.ArtistNamesString);
}
This lists each song only once and the artist names are displayed as a comma separated list.
I'm quite new to RavenDB so sorry if my question sounds stupid. I have a class which contains a DateTime property. I store instances of this class in RavenDB. I have defined index the following way:
from doc in docs.Orders
from docItemsItem in ((IEnumerable<dynamic>)doc.Items).DefaultIfEmpty()
select new { Items_Price_Amount = docItemsItem.Price.Amount, Items_Quantity = docItemsItem.Quantity, Date = doc.Date }
http://dl.dropbox.com/u/3055964/Capture.GIF <-- here's a screenshot
Here's class definition:
public class Order
{
public DateTime Date { get; set; }
public IList<OrderItem> Items { get; set; }
public string CustomerId { get; set; }
public Order()
{
Items = new List<OrderItem>();
}
}
Now, when I try to query RavenDB with the index shown above, the query yields no result at all.
var orders = session.Query<Order>("OrdersIndex").Where(o => o.Date > DateTime.Now).ToList(); // orders.Count == 0
If I omit the index from query, like this:
var orders = session.Query<Order>().Where(o => o.Date > DateTime.Now).ToList(); // orders.Count == 128
a temporary index is created and eveything works as expected.
Does anyone has any idea what's wrong with my query?
Thanks.
UPDATE
Allright, I removed Fields Date, Items,Price,Amount and Items,Quantity via management studio (shown in screenshot), and now the query works fine. Anyone any idea why? What's the purpose to define those fields explicitly?
Check that the date in the Index is stored as Index(x => x.Date, FieldIndexing.Default).
I had it set to FieldIndexing.Analysed, and wasn't getting the correct results back.
I need to read up more on the difference between the different FieldIndexing options :)