I need to load an external web (not local) page into my site (some link), but only a part of it.
What are the options for doing so?
That depends on whether or not the external page is local, or on a different domain. If it's local, you can use $.load() in the jQuery library. This has an optional parameter to specify which element in the remote-dom to load it:
$("#links").load("/Main_Page #jq-p-Getting-Started li");
If the page is on another domain, you'll need a proxy script. You can do this with PHP and the phpQuery (php port of jQuery) library. You'll just use file_get_contents() to get the actual remote-dom, and then pull out the elements you want based on jQuery-like selectors.
$f = fopen('http://www.quran.az/2/255', 'r');
and so on...
Once you get the whole page as Michael Todd outlined, you will likely need to either use substring methods for a static means to slice up the content or you can use regex's for a more dynamic way to grab the content. An intro article on Regex's in ASP.Net can be found here. Good luck!
To load a web page in .Net, use the HttpWebRequest class.
Example taken from MSDN, here:
private string StringGetWebPage(String uri)
{
const int bufSizeMax = 65536; // max read buffer size conserves memory
const int bufSizeMin = 8192; // min size prevents numerous small reads
StringBuilder sb;
// A WebException is thrown if HTTP request fails
try
{
// Create an HttpWebRequest using WebRequest.Create (see .NET docs)!
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
// Execute the request and obtain the response stream
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
// Content-Length header is not trustable, but makes a good hint.
// Responses longer than int size will throw an exception here!
int length = (int)response.ContentLength;
// Use Content-Length if between bufSizeMax and bufSizeMin
int bufSize = bufSizeMin;
if (length > bufSize)
bufSize = length > bufSizeMax ? bufSizeMax : length;
// Allocate buffer and StringBuilder for reading response
byte[] buf = new byte[bufSize];
sb = new StringBuilder(bufSize);
// Read response stream until end
while ((length = responseStream.Read(buf, 0, buf.Length)) != 0)
sb.Append(Encoding.UTF8.GetString(buf, 0, length));
}
catch (Exception ex)
{
sb = new StringBuilder(ex.Message);
}
return sb.ToString();
}
Note that this will return the entire page and not just a portion of it. You'll then need to sift through the page to find the information you're looking for.
Related
Great Article on this subject here: https://searchwindevelopment.techtarget.com/tip/Share-session-state-between-ASP-and-ASPNET-apps
Problem is, I cant get it to work without source code. The code snippets in the article show many errors in Visual Studios. The author, Dennis Hurst is unreachable, as it was written in 2004. Anybody out there have the actual Source code they can post ? Or maybe point me in the right direction ? My goal is to pull Classic ASP object data (Application) into ASP.Net code that shares the same folder. I have read that it might be possible using a COMM Wrapper, but that is way out of my skill level. This sounds like the best solution for my problem. Thank You in advance for your help.
// The constructor for this class takes a reference to the HttpContext and derives the URL it will need to send its requests to
public ASPSessionVar(HttpContext oInContext)
{
oContext = oInContext;
ASPSessionVarASP = "SessionVar.asp";
/* We now build a System.Uri object to derive the correct
URL to send the HTTP request to. oContext.Request.Url
will contain a System.Uri object that represents
this ASPXs URL.
*/
System.Uri oURL = oContext.Request.Url;
ASPSessionVarASP = oURL.Scheme + "://"
+ oURL.Host + ":" + oURL.Port.ToString()
+ ASPSessionVarASP;
}
//-------------------------------------------------------------------------//
// The primary function for this example is called GetSessionVar. It does the majority of the work done by this application,
// This includes creating a WebRequest, sending it off to the ASP page, and returning the response.
// First get the Session Cookie
string ASPCookieName = "";
string ASPCookieValue = "";
if (!GetSessionCookie
(out ASPCookieName, out ASPCookieValue))
{
return "";
}
// Initialize the WebRequest.
HttpWebRequest myRequest =
(HttpWebRequest)WebRequest.Create
(ASPSessionVarASP + "?SessionVar=" + ASPSessionVar);
myRequest.Headers.Add
("Cookie: " + ASPCookieName + "=" + ASPCookieValue);
// Send the request and get a response
HttpWebResponse myResponse =
(HttpWebResponse)myRequest.GetResponse();
Stream receiveStream = myResponse.GetResponseStream();
System.Text.Encoding encode =
System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream =
new StreamReader(receiveStream, encode);
string sResponse = readStream.ReadToEnd();
// Do a bit of cleanup
myResponse.Close();
readStream.Close();
return sResponse;
}
//------------------------------------------------------------------------------------------------------------------------//
// This function simply takes the Request that was passed by the client and extracts the ASP Session cookie from it.
// This function is called by the GetSessionVar function to retrieve the ASPSession cookie.
private bool GetSessionCookie
(out string ASPCookieName, out string ASPCookieValue)
{
int loop1;
HttpCookie myCookie; // Cookie variable
ASPCookieName = "";
ASPCookieValue = "";
// Capture all cookie names into a string array.
String[] CookieArray =
oContext.Request.Cookies.AllKeys;
// Grab individual cookie objects by cookie name.
for (loop1 = 0; loop1 < CookieArray.Length; loop1++)
{
myCookie =
oContext.Request.Cookies[CookieArray[loop1]];
if (myCookie.Name.StartsWith("ASPSESSION"))
{
ASPCookieName = myCookie.Name;
ASPCookieValue = myCookie.Value;
return true;
}
}
return false;
}
//--------------------------------------------------------------------------------------------------------------------------------//
//The ASPX page will instantiate an ASPSessionVar object, passing in the current Context to the construct or.
//The GetSessionVar function is then called, passing in the name of the ASP Session variable that is to be retrieved.
//Create an ASPSessionVar object,
//passing in the current context
SPI.WebUtilities.ASP.ASPSessionVar oASPSessionVar
= new SPI.WebUtilities.ASP.ASPSessionVar(Context);
string sTemp = oASPSessionVar.GetSessionVar("FirstName");
// CLASSIC ASP CODE BELOW !!
//The ASP code for this example was placed in an ASP file called SessionVar.asp.
// It performs two simple tasks. First, it ensures that the request is coming from the server that the ASP page is running on.
// This ensures that the request is valid and coming ONLY from the Web server's IP address.
// The ASP page then returns the session variable it was asked to provide
<%
dim sT
if Request.ServerVariables("REMOTE_ADDR") =
Request.ServerVariables("LOCAL_ADDR") then
sT = Request("SessionVar")
if trim(sT) <> "" then
Response.Write Session(sT)
end if
end if
%>
I am doing video (and also photo) uploading to the server by using HttpURLConnection.
I have an Uri of a video. I open an InputStream this way:
InputStream inputStream = context.getContentResolver().openInputStream(uri);
As video file is pretty big, I can't buffer data while writing it into the outputStream. So I need to use setFixedLengthStreamingMode(contentLength) method of HttpURLConnection. But it requires "contentLength".
The question is, how to get the length of the video?
Please don't suggest getting filepath. On some devices it works, but it often fails (especially on Android 6). They say Uri doesn't necessarily represent a file.
I also stumbled onto situations when after opening device gallery (with Intent) I receive an Uri of a picture, but I fail trying to get filepath from it. So I believe it's not a good way to get filepath from Uri?
Try something like this:
void uploadVideo() {
InputStream inputStream = context.getContentResolver().openInputStream(uri);
// Your connection.
HttpURLConnection connection;
// Do connection setup, setDoOutput etc.
// Be sure that the server is able to handle
// chunked transfer encoding.
connection.setChunkedStreamingMode(0);
OutputStream connectionOs = connection.getOutputStream();
// Read and write a 4 KiB chunk a time.
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
connectionOs.write(buffer, 0, bytesRead);
}
// Close streams, do connection etc.
}
UPDATE: added setChunkedStreamingMode
I'm trying to upload a large file to a document library, but it fails after just a few seconds. The upload single document fails silently, upload multiple just shows a failed message. I've turned up the file size limit on the web application to 500MB, and the IIS request length to the same (from this blog), and increased the IIS timeout for good measure. Are there any other size caps that I've missed?
Update I've tried a few files of various sizes, anything 50MB or over fails, so I assume something somewhere is still set to the webapp default.
Update 2 Just tried uploading using the following powershell:
$web = Get-SPWeb http://{site address}
$folder = $web.GetFolder("Site Documents")
$file = Get-Item "C:\mydoc.txt" // ~ 150MB
$folder.Files.Add("SiteDocuments/mydoc.txt", $file.OpenRead(), $false)
and get this exception:
Exception calling "Add" with "3" argument(s): "<nativehr>0x80070003</nativehr><nativestack></nativestack>There is no file with URL 'http://{site address}/SiteDocuments/mydoc.txt' in this Web."
which strikes me as odd as of course the file wouldn't exist until it's been uploaded? N.B. while the document library has the name Site Documents, it has the URL SiteDocuments. Not sure why...
Are you sure you updated the right webapp? Is the filetype blocked by the server? Is there adequate space in your content database? I would check ULS logs after that and see if there is another error since it seems you hit the 3 spots you would need too update.
for uploading a large file, you can use the PUT method instead of using the other ways to upload a document.
by using a put method you will save the file into content database directly. see the example below
Note: the disadvantage of the code below is you cannot catch the object that is responsible for uploading directly, on other word, you cannot update the additional custom properties of the uploaded document directly.
public static bool UploadFileToDocumentLibrary(string sourceFilePath, string targetDocumentLibraryPath)
{
//Flag to indicate whether file was uploaded successfuly or not
bool isUploaded = true;
try
{
// Create a PUT Web request to upload the file.
WebRequest request = WebRequest.Create(targetDocumentLibraryPath);
//Set credentials of the current security context
request.Credentials = CredentialCache.DefaultCredentials;
request.Method = “PUT”;
// Create buffer to transfer file
byte[] fileBuffer = new byte[1024];
// Write the contents of the local file to the request stream.
using (Stream stream = request.GetRequestStream())
{
//Load the content from local file to stream
using (FileStream fsWorkbook = File.Open(sourceFilePath, FileMode.Open, FileAccess.Read))
{
//Get the start point
int startBuffer = fsWorkbook.Read(fileBuffer, 0, fileBuffer.Length);
for (int i = startBuffer; i > 0; i = fsWorkbook.Read(fileBuffer, 0, fileBuffer.Length))
{
stream.Write(fileBuffer, 0, i);
}
}
}
// Perform the PUT request
WebResponse response = request.GetResponse();
//Close response
response.Close();
}
catch (Exception ex)
{
//Set the flag to indiacte failure in uploading
isUploaded = false;
}
//Return the final upload status
return isUploaded;
}
and here are an example of calling this method
UploadFileToDocumentLibrary(#”C:\test.txt”, #”http://home-vs/Shared Documents/textfile.pdf”);
In order to support a legacy application that's in the field, I need my ASP.NET MVC app to return an empty response that also has a Content-Type. One of IIS, ASP.NET, or ASP.NET MVC is removing my Content-Type when I send back a null response. Is there any way around this?
(While not requiring an empty response with a set Content-Type would obviously be the ideal solution, the clients are already out there, and many of them cannot be upgraded.)
EDIT: Since there was a request for code: I'm proxying the request from the new web application to the one that older clients rely on. To do this, I have a subclass of ActionResult, called LegacyResult, that you can simply return for those methods that need to be handled by the old software. This is the relevant part of its code:
public override void ExecuteResult(ControllerContext context)
{
using (var legacyResponse = GetLegacyResponse(context))
{
var clientResponse = context.HttpContext.Response;
clientResponse.Buffer = false;
clientResponse.ContentType = legacyResponse.ContentType; /* Yes, I checked that legacyResponse.ContentType is never string.IsNullOrEmpty */
if (legacyResponse.ContentLength >= 0) clientResponse.AddHeader("Content-Length", legacyResponse.ContentLength.ToString());
var legacyInput = legacyResponse.GetResponseStream();
using (var clientOutput = clientResponse.OutputStream)
{
var rgb = new byte[32768];
int cb;
while ((cb = legacyInput.Read(rgb, 0, rgb.Length)) > 0)
{
clientOutput.Write(rgb, 0, cb);
}
clientOutput.Flush();
}
}
}
If legacyInput has data, then Content-Type is set appropriately. Otherwise, it's not. I can actually kluge the old backend to send an empty v. non-empty response for exactly the same request, and observe the difference in Fiddler.
EDIT 2: Poking around with Reflector reveals that, if headers have not been written at the time that HttpResponse.Flush is called, then Flush writes out the headers itself. The problem is that it only writes out a tiny subset of the headers. One of the missing ones is Content-Type. So it seems that, if I can force headers out to the stream, I can avoid this problem.
You have to trick the response into writing the headers, by falsely telling it there's content, then suppressing it:
/// [inside the writing block]
var didWrite = false;
while ((cb = legacyInput.Read(rgb, 0, rgb.Length)) > 0)
{
didWrite = true;
clientOutput.Write(rgb, 0, cb);
}
if (!didWrite)
{
// The stream needs a non-zero content length to write the correct headers, but...
clientResponse.AddHeader("Content-Length", "1");
// ...this actually writes a "Content-Length: 0" header with the other headers.
clientResponse.SuppressContent = true;
}
I'm trying to create an MJPEG stream, I have a series of jpegs that I want to put together into a stream so that a user can just hit a URL and get an mjpeg stream.
I've been trying for the last few days to get this to work, and it may just not be possible. I've brought up ethereal and listened to the packets coming from an axis camera on the net somewhere, and tried to mimmick it. I originally tried using WCF, and returning a "stream" but then later found out that I would need to set the content type on that stream, so I then tried the WCF REST api, but that suffers from the same problem. so I am now just using a bare bones HTTPListener, and handling the event. I would greatly prefer to use WCF, but I'm not sure that it will allow me to return a stream with the right content type.
so here's what I have for the httpListener .
in the handler of the listener call back I put the following.
HttpListenerResponse response = context.Response;
response.ProtocolVersion = new System.Version(1, 0);
response.StatusCode = 200;
response.StatusDescription = "OK";
response.ContentType = "multipart/x-mixed-replace;boundary=" + BOUNDARY + "\r\n";
System.IO.Stream output = response.OutputStream;
Render(output);
the Render method looks like this
var writer = new StreamWriter(st);
writer.Write("--" + BOUNDARY + "\r\n");
while (true)
{
for (int i = 0; i < imageset.Length; i++)
{
var resource = Properties.Resources.ResourceManager.GetObject(imageset[i]) as Bitmap;
var memStream = new MemoryStream();
resource.Save(memStream,ImageFormat.Jpeg);
byte[] imgBinaryData = memStream.ToArray();
string s = Convert.ToBase64String(imgBinaryData);
writer.Write("Content-type: image/jpeg\r\n");
foreach (var s1 in imgBinaryData)
{
writer.Write((char)s1);
}
writer.Write("\n--" + BOUNDARY + "\n");
writer.Flush();
Thread.Sleep(500);
}
}
At this point I've just added a few jpeg images as properties on the dll, and am iterating over them, eventually these will be dynamic images, but for now I just want to get the thing to work.
From what I understand about the MJPEG (spec) is that the content must be set to multipart/x-mixed-replace and a boundary set. and then you just deliminate the jpegs within the stream by the boundary.
This seems like it should be simpler then I'm making it, but I'm wondering where I'm going wrong. if I load this URL up in IE or Firefox, it just hangs. if I try to make a stub html page with an img tag, whose source is the URL then I get a broken image.
Any ideas, thanks
Josh
Well, as far as I can tell, here are your issues:
The StreamWriter is not a correct choice. Use a regular stream write function is fine. Meaning, you should write data in Byte array instead of string.
You convert the Binary data of the image to String64, the browser does not known that, still thinking it is 32bit data.
Your jpeg frame format is not correct. You should also add Content-Length to the frame header so that the application that receive the stream know when to stop reading rather than having to check for the next boundary string every read. This will result in about 4-5 times faster in reading data. And there are also inconsistency in your new line character, some are "\r\n" while some others are "\n".
While loop is a infinite loop.
So, here is the solution.
Note: There might be some syntax errors but you probably get the general idea.
private byte[] CreateHeader(int length)
{
string header =
"--" + BOUDARY + "\r\n" +
"Content-Type:image/jpeg\r\n" +
"Content-Length:" + length + "\r\n" +
+ "\r\n"; // there are always 2 new line character before the actual data
// using ascii encoder is fine since there is no international character used in this string.
return ASCIIEncoding.ASCII.GetBytes(header);
}
public byte[] CreateFooter()
{
return ASCIIEncoding.ASCII.GetBytes("\r\n");
}
private void WriteFrame(Stream st, Bitmap image)
{
// prepare image data
byte[] imageData = null;
// this is to make sure memory stream is disposed after using
using (MemoryStream ms = new MemoryStream())
{
image.Save(ms, ImageFormat.Jpeg);
imageData = ms.ToArray();
}
// prepare header
byte[] header = CreateHeader(imageData.Length);
// prepare footer
byte[] footer = CreateFooter();
// Start writing data
st.Write(header, 0, header.Length);
st.Write(imageData, 0, imageData.Length);
st.Write(footer, 0, footer.Length);
}
private void Render(Stream st)
{
for (int i = 0; i < imageset.Length; i++)
{
var resource = Properties.Resources.ResourceManager.GetObject(imageset[i]) as Bitmap;
WriteFrame(st, resource);
Thread.Sleep(500);
}
}
There is also an implementation # https://net7mma.codeplex.com/SourceControl/latest the library there can transcode the Http to Rtp compatible on the fly!