Using the following code:
Private Sub MakeMeSomeXmlBeforeRyanGetsAngry()
Dim db As New MyDBDataContext
Dim customer = From c In db.Customers Select c
Dim dcs As New DataContractSerializer(GetType(Customer))
Dim sb As StringBuilder = New StringBuilder
Dim writer As XmlWriter = XmlWriter.Create(sb)
dcs.WriteObject(writer, customer)
Dim xml As String = sb.ToString
Response.Write(xml)
End Sub
I am attempting to serialize my linq collection of customers. But it keeps throwing
Type 'System.Data.Linq.DataQuery`1[MyDB.Customer]' cannot be serialized. Consider marking it with the DataContractAttribute attribute, and marking all of its members you want serialized with the DataMemberAttribute attribute. See the Microsoft .NET Framework documentation for other supported types.
My issue is that I have already marked the dbml Serialization Mode to UniDirectional and when I check the dbml codebehind all of the DataContract()> and DataMember()> elements are there.
I am not sure how to proceed. I have tried adding various dataloadoptions and setting deferredloading to false, but no luck.
Ideas?
This is working for me with Serialization Mode property of .dbml set to UniDirectional:
Public Shared Function CreateXml(Of T)(ByVal item As T) As String
Dim result As String = ""
Dim memoryStream As New IO.MemoryStream()
Dim serializer As New DataContractSerializer(GetType(T))
serializer.WriteObject(memoryStream, item)
memoryStream.Position = 0
Using reader As New StreamReader(memoryStream)
result = reader.ReadToEnd()
End Using
memoryStream.Close()
Return result
End Function
Dim db As New SerializerDataContext()
Dim questions = db.Questions.ToList()
Dim xmlFileName As String = "D:\\xml_test.xml"
If My.Computer.FileSystem.FileExists(xmlFileName) Then My.Computer.FileSystem.DeleteFile(xmlFileName)
Dim xml As String = XmlHelper.CreateXml(Of List(Of Question))(questions)
My.Computer.FileSystem.WriteAllText(xmlFileName, xml, True)
I think the problem arises because LINQ queries are processed only on demand, and are not serializable. Try to serialize a serializable data type resulting from your LINQ query (array, list, a single item, etc.)
dcs.WriteObject(writer, customer.ToArray)
(I assume you only need to serialize the results of the query, and not the query itself)
Related
I've an XML string in database column like this
<trueFalseQuestion id="585" status="correct" maxPoints="10"
maxAttempts="1"
awardedPoints="10"
usedAttempts="1"
xmlns="http://www.ispringsolutions.com/ispring/quizbuilder/quizresults">
<direction>You have NO control over how you treat customers.</direction>
<answers correctAnswerIndex="1" userAnswerIndex="1">
<answer>True</answer>
<answer>False</answer>
</answers>
</trueFalseQuestion>
But I need to do XML operations on this string like select its name, attributes values,inner text etc. How can I make this possible from this string
EDIT
Im sharing the code snippet I tried, but not working
Dim myXML As String
Dim gDt As New DataTable
gDt.Columns.Add("id")
gDt.Columns.Add("questionid")
gDt.Columns.Add("serial")
gDt.Columns.Add("direction")
Dim dr As DataRow
myXML ='<my above shared XML>'
Dim xmlDoc As New XmlDocument
xmlDoc.LoadXml(myXML)
dr = gDt.NewRow
dr("serial") = 1
dr("id") = xmlDoc.Attributes("id").Value
dr("direction") = xmlDoc("direction").InnerText
gDt.Rows.Add(dr)
But thats not working at all as I wish
There are many ways to parse XML in .NET, such as using one of the serialization classes or the XmlReader class, but the two most popular options would be to parse it with either XElement or XmlDocument. For instance:
Dim input As String = "<trueFalseQuestion id=""585"" status=""correct"" maxPoints=""10"" maxAttempts=""1"" awardedPoints=""10"" usedAttempts=""1"" xmlns=""http://www.ispringsolutions.com/ispring/quizbuilder/quizresults""><direction>You have NO control over how you treat customers.</direction><answers correctAnswerIndex=""1"" userAnswerIndex=""1""><answer>True</answer><answer>False</answer></answers></trueFalseQuestion>"
Dim element As XElement = XElement.Parse(input)
Dim id As String = element.#id
Or:
Dim input As String = "<trueFalseQuestion id=""585"" status=""correct"" maxPoints=""10"" maxAttempts=""1"" awardedPoints=""10"" usedAttempts=""1"" xmlns=""http://www.ispringsolutions.com/ispring/quizbuilder/quizresults""><direction>You have NO control over how you treat customers.</direction><answers correctAnswerIndex=""1"" userAnswerIndex=""1""><answer>True</answer><answer>False</answer></answers></trueFalseQuestion>"
Dim doc As New XmlDocument()
doc.LoadXml(input)
Dim nsmgr As New XmlNamespaceManager(doc.NameTable)
nsmgr.AddNamespace("q", "http://www.ispringsolutions.com/ispring/quizbuilder/quizresults")
Dim id As String = doc.SelectSingleNode("/q:trueFalseQuestion/#id", nsmgr).InnerText
Based on your updated question, it looks like the trouble you were having is that you weren't properly specifying the namespace. If you use XElement, it's much more forgiving (i.e. loose), but when you use XPath to select nodes in an XmlDocument, you need to specify every namespace, even when it's the default namespace on the top-level element of the document.
I'm trying to scrape some schedules off of a website. the information is displayed in a GridView with paging.
The url is:
http://www.landmarkworldwide.com/when-and-where/register/search-results.aspx?prgid=0&pgID=270&crid=0&ctid=&sdt=0
My Issue is when I want to scrape pages other then #1 in the grid view.
The best post I found so far was This One, but it doesn't work and that topic is not complete. I tried to use Fiddler and Chrome to get the post data and use it, but I can't get it to work for me. Can you guys see what's missing?
Here's the code I am using. it's in VB, but you can answer in C# and I'll translate -) (sorry)
Protected Sub Page_Load(sender As Object, e As System.EventArgs) Handles Me.Load
Dim lcUrl As String = "http://www.landmarkworldwide.com/when-and-where/register/search-results.aspx?prgid=0&pgID=270&crid=0&ctid=&sdt=0"
' first, request the login form to get the viewstate value
Dim webRequest__1 As HttpWebRequest = TryCast(WebRequest.Create(lcUrl), HttpWebRequest)
Dim responseReader As New StreamReader(webRequest__1.GetResponse().GetResponseStream())
Dim responseData As String = responseReader.ReadToEnd()
responseReader.Close()
' extract the viewstate value and build out POST data
Dim viewState As String = ExtractViewState(responseData)
Dim loHttp As HttpWebRequest = DirectCast(WebRequest.Create(lcUrl), HttpWebRequest)
' *** Send any POST data
Dim lcPostData As String = [String].Format("__VIEWSTATE={0}&__EVENTTARGET={1}&__EVENTARGUMENT={2}", viewState, HttpUtility.UrlEncode("contentwrapper_0$maincontent_0$maincontentfullwidth_0$ucSearchResults$gvPrograms"), HttpUtility.UrlEncode("Page$3"))
loHttp.Method = "POST"
Dim lbPostBuffer As Byte() = System.Text.Encoding.GetEncoding(1252).GetBytes(lcPostData)
loHttp.ContentLength = lbPostBuffer.Length
Dim loPostData As Stream = loHttp.GetRequestStream()
loPostData.Write(lbPostBuffer, 0, lbPostBuffer.Length)
loPostData.Close()
Dim loWebResponse As HttpWebResponse = DirectCast(loHttp.GetResponse(), HttpWebResponse)
Dim enc As Encoding = System.Text.Encoding.GetEncoding(1252)
Dim loResponseStream As New StreamReader(loWebResponse.GetResponseStream(), enc)
Dim lcHtml As String = loResponseStream.ReadToEnd()
loWebResponse.Close()
loResponseStream.Close()
Response.Write(lcHtml)
End Sub
Private Function ExtractViewState(s As String) As String
Dim viewStateNameDelimiter As String = "__VIEWSTATE"
Dim valueDelimiter As String = "value="""
Dim viewStateNamePosition As Integer = s.IndexOf(viewStateNameDelimiter)
Dim viewStateValuePosition As Integer = s.IndexOf(valueDelimiter, viewStateNamePosition)
Dim viewStateStartPosition As Integer = viewStateValuePosition + valueDelimiter.Length
Dim viewStateEndPosition As Integer = s.IndexOf("""", viewStateStartPosition)
Return HttpUtility.UrlEncodeUnicode(s.Substring(viewStateStartPosition, viewStateEndPosition - viewStateStartPosition))
End Function
To make it work you need to send all input fields to the page, not only viewstate. Other critical data is the __EVENTVALIDATION for example that you do not handle it. So:
First you need to make scrape on the #1 page. So load it and use the Html Agility Pack to convert it to a usable struct.
Then extract from that struct the input data that you need to post. From this answer HTML Agility Pack get all input fields here is a code sniped on how you can do that.
foreach (HtmlNode input in doc.DocumentNode.SelectNodes("//input"))
{
// use this to create the post string
// input.Attributes["value"];
}
Then when you have the post data that is needed to be a valid post, you move to the next step. Here is an example How to pass POST parameters to ASP.Net web request?
You can also read: How to use HTML Agility pack
I am using the below code to get the Title and description of the youtube video embeded in my asp.net application. I am able to see the Title, but not description.
I use Atomfeed to do this...
Problem is i get the Description as "Google.GData.Client.AtomTextConstruct" for all my videos.
Private Function GetTitle(ByVal myFeed As AtomFeed) As String
Dim strTitle As String = ""
For Each entry As AtomEntry In myFeed.Entries
strTitle = entry.Title.Text
Next
Return strTitle
End Function
Private Function GetDesc(ByVal myFeed As AtomFeed) As String
Dim strDesc As String = ""
For Each entry As AtomEntry In myFeed.Entries
strDesc = entry.Summary.ToString()
Next
Return strDesc
End Function
I believe that when the XML from the atom feed is parsed, that the description is not handled. Take a look at this: http://code.google.com/p/google-gdata/wiki/UnderstandingTheUnknown
But what happens with things that are not understood? They end up as
an element of the ExtensionElements collection, that is a member of
all classes inherited from AtomBase, like AtomFeed, AtomEntry,
EventEntry etc...
So, what we can do is pull out the description from the extensionelement like this:
Dim query As New FeedQuery()
Dim service As New Service()
query.Uri = New Uri("https://gdata.youtube.com/feeds/api/standardfeeds/top_rated")
Dim myFeed As AtomFeed = service.Query(query)
For Each entry In myFeed.Entries
For Each obj As Object In entry.ExtensionElements
If TypeOf obj Is XmlExtension Then
Dim xel As XElement = XElement.Parse(TryCast(obj, XmlExtension).Node.OuterXml)
If xel.Name = "{http://search.yahoo.com/mrss/}group" Then
Dim descNode = xel.Descendants("{http://search.yahoo.com/mrss/}description").FirstOrDefault()
If descNode IsNot Nothing Then
Console.WriteLine(descNode.Value)
End If
Exit For
End If
End If
Next
Next
Also, the reason why you are getting "Google.GData.Client.AtomTextConstruct" is because Summary is an object of type Google.GData.Client.AtomTextConstruct, so doing entry.Summary.ToString() is just giving you the default ToString() behavior. You would normally do Summary.Text, but this of course is blank because as I say above, it's not handled properly by the library.
For youtube, I fetch the information for each video using the Google.GData.YouTube.
Something like this returns a lot of information from the video.
Dim yv As Google.YouTube.Video
url = New Uri("http://gdata.youtube.com/feeds/api/videos/" & video.Custom)
r = New YouTubeRequest(New YouTubeRequestSettings("??", "??"))
yv = r.Retrieve(Of Video)(url)
Then it's possible to get the description with: yv.Description
I'm trying to resize one image to 5 different sizes (and then upload them to amazonS3).
I'm using imageresizer.net
the problem seems to be that i cannot use the inputstream twice. it works the first time.
Dim SmallStream As Stream = New MemoryStream
Dim TinyStream As Stream = New MemoryStream
If FileUpload1.HasFile Then
**ImageResizer.ImageBuilder.Current.Build(FileUpload1.PostedFile.InputStream, SmallStream, New ResizeSettings("maxwidth=100&maxheight=100"))
ImageResizer.ImageBuilder.Current.Build(FileUpload1.PostedFile.InputStream, TinyStream, New ResizeSettings("maxwidth=100&maxheight=100"))**
AmazonUploadFile("SmallImages/" & FileUpload1.FileName, SmallStream)
AmazonUploadFile("TinyImages/" & FileUpload1.FileName, TinyStream)
End If
Public Shared Function GetS3Client() As AmazonS3
Dim appConfig As NameValueCollection = ConfigurationManager.AppSettings
Dim s3Client As AmazonS3 = AWSClientFactory.CreateAmazonS3Client(AWS_ACCESS_KEY, AWS_SECRET_KEY)
Return s3Client
End Function
Public Sub AmazonUploadFile(S3Key As String, FileStream As Stream)
Dim request As New PutObjectRequest()
request.WithBucketName(BUCKET_NAME)
request.WithKey(S3Key).InputStream = FileStream
request.WithCannedACL(S3CannedACL.PublicRead)
GetS3Client.PutObject(request)
End Sub
The code breaks when i try to retrieve the FileUpload1.PostedFile.InputStream the second time.
I believe ImageResizer will accept Image objects to that method. So, you could read the InputStream into an Image object first (Image.FromStream()), then you can use that image object repeatedly.
[Disclaimer: I'm the author of http://imageresizing.net/]
Pass FileUpload1.PostedFile instead of FileUpload1.PostedFile.InputStream to the ImageResizer and it will automatically handle re-seeking the stream to the beginning after each read. (Make sure you're using 3.1.5 or later).
Alternatively, use the ImageJob class and set ResetSourceStream=true.
I have been given the task of calling a web service which returns an xml data feed which I am doing like so;
For Each r As DataRow in SomeDataTable
Dim msFeed As String = string.format("http://some-feed.com?param={0}", r!SOME_VAL)
Dim x As XDocument = XDocument.Load(msFeed)
Next
This is all fine but as you can see x just gets overwritten with every iteration. What I need is create an xDocument and add each feed from my loop but I am unsure how to proceed.
Thanks
Solution
Dim xAllFeeds As XElement = New XElement("Feeds")
For Each r As DataRow in SomeDataTable
Dim msFeed As String = string.format("http://some-feed.com?param={0}", r!SOME_VAL)
Dim x As XDocument = XDocument.Load(msFeed)
xAllFeeds.Add(x.Root)
Next
Not 100% sure of the VB syntax (C# is my language of choice), but this should be the gist of what you're after.
Dim xAllFeeds As XElement = New XElement("Feeds")
For Each r As DataRow in SomeDataTable
Dim msFeed As String = string.format("http://some-feed.com?param={0}", r!SOME_VAL)
Dim xDoc As XDocument = XDocument.Load(msFeed)
xAllFeeds.Add(xDoc.Root)
Next