storing data in a xml format file - asp.net

I m trying to build a function that will retrieve 'some' SQL data from multiple tables, and store it in a file in the XML format.
If I do it with C#, is it as simple as:-
SQL statement,
retrieve data,
store data in a string list,
and then WriteXml (xmlFile, variable where data is stored) ??
Can any one show me an example?
I was looking at:-
WriteXml () and WriteXmlSchema() functions in C#
string xmlFile = Server.MapPath("Employees.xml");
ds.WriteXml(xmlFile, XmlWriteMode.WriteSchema);
Also, will xmlSerialization be something I need to take a look at?
Sample SQL query.
SqlConnection Connection1 = new SqlConnection(DBConnect.SqlServerConnection);
String strSQL1 = "SELECT xxx.MEMBERKEY, xxx.MEMBID, xyz.HPCODE, convert(varchar, OPFROMDT, 101) as OPFROMDT"
+ ", convert(varchar, OPTHRUDT, 101) as OPTHRUDT FROM [main].[dbo].[yyy] INNER JOIN [main].[dbo].[xxx] ON xxx.MEMBERKEY = yyy.MEMBERKEY "
+ "and opthrudt >= opfromdt INNER JOIN [main].[dbo].[xyz] ON yyy.HPCODEKEY = xyz.HPCODEKEY where MembID = #memID";
SqlCommand command1 = new SqlCommand(strSQL1, Connection1);
command1.Parameters.AddWithValue("#memID", memID);
SqlDataReader Dr1;
Connection1.Open();
Dr1 = command1.ExecuteReader();
while (Dr1.Read())
{
HPCODEs.Add((Dr1["HPCODE"].ToString()).TrimEnd());
OPFROMDTs.Add((Dr1["OPFROMDT"].ToString()).TrimEnd());
OPTHRUDTs.Add((Dr1["OPTHRUDT"].ToString()).TrimEnd());
}
Dr1.Close();

There are so many approaches to this.
For one, you can invest some time and use SQL built in XML capabilities to query and get an XML document directly which then you can serialize straight into a file. A very basic example could be found here. Then you would use the DataReader's GetSqlXml method, something like this
SqlDataReader r = cmd.ExecuteReader();
SqlXml data = r.GetSqlXml(0);
XmlReader xr = data.CreateReader();
Maybe another option is to read from the sql data reader into DTO (data transfer objects). This is probably the tried and true method. And then once you have a list of DTO's you can use .NET's serialization (DataContractSerializer) to serialize to XML.

I would highly recommending looking at a tool like this:
http://msdn.microsoft.com/en-us/library/x6c1kb0s(v=vs.80).aspx
This will generate .net classes for you if you have an xsd of the xml output you are trying to create.
Either way, dropping your data into POCO style classes and serializing to XML is a lot better than trying to use the XmlWriter directly.

Related

Application Insights and Azure Stream Analytics Query export the whole custom dimensions as string

I have setup a continuous export from Application Insights into Blog. With a data stream I'm able to get out the JSON files into SQL DB. So far so good.
Also with help from Phani Rahul Sivalenka I'm able to query the individual properties of custom dimensions as described here: Application Insights and Azure Stream Analytics Query a custom JSON property
My custom dimensions looks like this when exporting manually into CSV file:
"{""OperatingSystemVersion"":""10.0.18362.418"",""OperatingSystem"":""WINDOWS"",""RuntimePlatform"":""UWP"",""Manufacturer"":""LENOVO"",""ScreenHeight"":""696"",""IsSimulator"":""False"",""ScreenWidth"":""1366"",""Language"":""it"",""IsTablet"":""False"",""Model"":""LENOVO_BI_IDEAPAD4Q_BU_idea_FM_""}"
Additionally to the single columns I like to have the whole custom dimensions as a string in a SQL Table column (varchar(max)).
In the "Test results" of my Data Stream Output Query I see the column as formated above - but when really exporting / wrinting into SQL DB all my tests ended having only the value "Array" or "Record" as value in my SQL Table column.
What do I have to do in the Data Stream Query to get the whole custom dimensions value as a string and I'm able to write this into SQL Table as a whole string?
What do I have to do in the Data Stream Query to get the whole custom
dimensions value as a string and I'm able to write this into SQL Table
as a whole string?
You could use UDF to merge all key-values of single raw into one single json format string.
UDF:
function main(raw) {
let str = "{";
for(let key in raw) {
str = str + "\""+ key+"\":\""+raw[key]+"\",";
}
str += "}";
return str;
}
SQL:
SELECT udf.jsonstring(INPUT1) FROM INPUT1
Output:
The answer brought me on the right track.
The above script don't include the values as expected. So I modified the script to get it work as needed:
function main(dimensions) {
let str = "{";
for (let i in dimensions)
{
let dim = dimensions[i];
for (let key in dim)
{
str = str + "\"" + key+ "\":\""+dim[key]+"\",";
}
}
str += "}";
return str;
}
Selecting:
WITH pageViews as (
SELECT
V.ArrayValue.name as pageName
, *
, customDimensions = UDF.flattenCustomDimensions(A.context.custom.dimensions)
, customDimensionsString = UDF.createCustomDimesionsString(A.context.custom.dimensions)
FROM [AIInput] as A
CROSS APPLY GetElements(A.[view]) as V
)
With this I'm getting the custom dimensions string as follow in my SQL table:
{"Language":"tr","IsSimulator":"False","ScreenWidth":"1366","Manufacturer":"Hewlett-Packard","OperatingSystem":"WINDOWS","IsTablet":"False","Model":"K8K51ES#AB8","OperatingSystemVersion":"10.0.17763.805","ScreenHeight":"696","RuntimePlatform":"UWP",}

XQuery (saxon) failing with a schema (XPath works)

I switched in saxon from XPath to XQuery and on the selects where I have a schema I'm getting the error message:
A typed input document can only be used with a schema-aware query
My setup is:
InputSource xmlSource = new InputSource(xmlData);
SAXSource saxSource = new SAXSource(reader, xmlSource);
Source schemaSource = new StreamSource(schemaFile);
Configuration config = createEnterpriseConfiguration();
config.addSchemaSource(schemaSource);
Processor processor = new Processor(config);
SchemaValidator validator = new SchemaValidatorImpl(processor);
DocumentBuilder doc_builder = processor.newDocumentBuilder();
if(!preserveWhiteSpace)
doc_builder.setWhitespaceStrippingPolicy(WhitespaceStrippingPolicy.ALL);
doc_builder.setSchemaValidator(validator);
XdmNode root_node = doc_builder.build(saxSource);
XQueryCompiler compiler = processor.newXQueryCompiler();
Is there something additional I need to do on queries where there is a schema?
thanks - dave
Call XQueryCompiler.setSchemaAware(true);
This isn't the default because it's good for the optimizer to know whether the data is likely to be typed or untyped, and it's inefficient to generate schema-aware code if the data is untyped (conversely, when the data is typed, schema-aware code is typically faster -- though the savings can be eaten up by the extra cost of validating the input).

How to use SQLitePCL execute content of sql file

I'm trying to use SQLitePCL package to develop a simple UWP app that executes database commands (create-select-update-delete). I created a database sql file that contains some sqlite commands and I'm trying to execute them in my code:
Uri appUri = new Uri("ms-appx:///Assets/db.sql");
StorageFile sFile = StorageFile.GetFileFromApplicationUriAsync(appUri).AsTask().ConfigureAwait(false).GetAwaiter().GetResult();
string sSQL = FileIO.ReadTextAsync(sFile).AsTask().ConfigureAwait(false).GetAwaiter().GetResult();
ISQLiteStatement cnStatement = dbConnection.Prepare(sSQL);
cnStatement.Step();
But when I run the program, it only executes the first statement in the sql file which is CREATE command and exit without executing the rest of the commands. Here is the sample content of the sql file:
CREATE TABLE Superhero (
Type TEXT PRIMARY KEY,
Picture TEXT
);
INSERT INTO Superhero (
Type,
Picture
)
VALUES (
'batman',
'batman.ico'
);
Anyone knows if there is a way in SQLitePCL to execute a sql file?
Any help would be very much appreciated!
Thanks!
According to the description of sqlite3_prepare interface:
These routines only compile the first statement in zSql, so *pzTail is left pointing to what remains uncompiled.
So that it seems like only the first statement in the commands is actually executed. The remainder is silently ignored. Since every command is ended up with symbol ";", for a quick and simple solution, you may just split the sql commands into single statements and then execute one by one. For example:
string sSQL = FileIO.ReadTextAsync(sFile).AsTask().ConfigureAwait(false).GetAwaiter().GetResult();
var dbConnection = new SQLiteConnection("sun.db", SQLiteOpen.READWRITE);
//using (ISQLiteStatement cnStatement = dbConnection.Prepare(sSQL))
//{
// var result = cnStatement.Step();
//}
var statements = sSQL.Split(new[] { ';' });
foreach (string onestate in statements)
{
using (ISQLiteStatement cnStatement = dbConnection.Prepare(onestate))
{
var result = cnStatement.Step();
}
}
Otherwise, you may need to update the SQLitePCL Nuget package.

need help update a datatable cell value

I have a trivial question about updating a datatable MyDT. I googled and found several approach and got compile errors. Here is the code and here is what I tried with the error. Any help is greatly appreciated. BTW, I am using asp.net framework 2.0 and VB.NET
MyDT.Rows[1][4] = "4NF" ' Property access must assign to the property or use it value
row.Item("New_Column") = "4NF" ' Input string was not in a correct format.
Couldn't store <4NF> in New_Column Column. Expect type is Byte.
row["New_Column"] = "4NF" ' Expression is not a method
Dim StatusCode As String
For Each row As DataRow In MyDT.Rows
StatusCode= row.Item("ThisColumn").ToString()
If StatusCode= "NONF" Then
MyDT.Rows[1][4] = "4NF"
End If
Next row
You should either:
Store data as String in your DataTable
Do a conversion
Here how the conversion looks
MyDT.Rows[1][4] = Convert.ToByte("4NF")

Looping through XMLReader to replace special characters in data field

I have XML files I want to put into data sets to export to a database using VB.Net. There is a possibility that new XML files added to this list daily will have special characters (idk why anyone would include "&" in an address entry anyway). After creating the XMLReader, what is the easiest way to replace the escape characters? What would the pseudo code look like? Stream Reader maybe? Or does that work with XMLReader?
Here is my code right now that attempts the data set creation:
For Each file1 In Directory.GetFiles(My.Settings.Local_Meter_Path, "*BadMeter*.xml")
Dim filecreatedate As String = IO.File.GetLastWriteTime(file1)
FN = Path.GetFileName(file1).ToString()
xmlFile = XmlReader.Create(Path.Combine(My.Settings.Local_Meter_Path, FN), New XmlReaderSettings())
ds.ReadXml(xmlFile)
and the spot where I'm getting ampersand entity-name parsing error
<Cell ss:StyleID="Default"><Data ss:Type="String">1440 COUNTY ROAD 40 X-MAS LIGHT & RV #2 CAMP HILL</Data></Cell>

Resources