I'm trying to download a municipal planning plan together with all the relevant documents.
All documents can be reached from the following link
I've tried the following command (that worked well for other sites) and some variations without success.
wget -E -k -r -l 3 "http://www.mavat.moin.gov.il/MavatPS/Forms/SV4.aspx?tid=4&et=1&mp_id=ppnCWTcsST9gG0%2fa0ayWnjFyZ%2bo14s221Ujlpi7UvR4jIRAHLKhJ8lOLSkomZ%2fvlHk8b2T0oENpI6Wh2hKzxQJCw9BPJP8gav%2ftgiKlk5S0%3d"
The same plan in their new site I can't get the files either,
https://mavat.iplan.gov.il/SV4/1/5000931297/310
I'd appreciate any help.
Well, these days, and especially with .net web sites?
We don't use hyper-links with a simple (full) path name to actual files from the web server. In fact in most cases one will not even give the web server rights to those folders. (they are not exposed to Internet Services).
So, no actual links as a full "url" to documents exist.
What happens is when you click on a button or button link? Then the code behind on the web server runs. (and that is code you don't have). And further more, that code behind can browser, read, retrieve any file from any folder on the server or other servers. But links from the web site don't exist and it not even possible to type in a url to resolve to a actual file name on the server.
So the server side code (not internet services) goes and grabs the document. In fact, the documents could be in a database. So, the code behind on the server side runs and pulls the binary data from the database (which represents a valid PDF file). Or the code behind reads the file from disk and then STREAMS the file for a download.
Now, this is often done for reasons of security. It means that no valid URL exists to get at a document.
Not only is this done for security, but from a developer point of view, it often better to retrieve a row from a database. That row can have the information you SEE rendered on that form, but the web page is not static, and the display of information is thus a developer coding a pull of rows from a database, and then you simply "assign" that data to some type of control - save datagrid, or listview or whatever. (this assignment of data is only 1 or two lines of code, and then the control + web server renders that datagrid control.
So, this is done since the developer thus only assigns the result of a database query to the control when then renders on the form. Thus, to add or remove documents? Then you only have to edit the database for the information on the web page to render.
As a result? There is no direct links to the actual documents on the server. To retrieve a document, you would have to send to the web site the exact command required.
You can hit f12 (most browsers support this). This will put your browser into developer mode. If we do this, and then select elements (select element feature). Now click on a pdf link. You get this:
<img src="../images/ft/file_PDF.gif" style="cursor:pointer"
onclick="openDoc('99000526871729',
'AABA7BE646E182B67DB1C15220E531DF36BBB591D8EEA7757435B2606C08E6F9')">
So, note above. The above code event openDoc is the SERVER side code you have to run to retrive a document. There is thus NO link. And you not going to be able to wire up, or run your OWN web page that hits that server and runs the routine "onclick".
However, the onclick DOES expose the internal database document numbers used to pull/read and retrieve a given document. But the path name, and how the code gets/grabs this file? You have no idea, and HAVE to run server side code (c#, or vb.net) code. That code as noted grabs the file and then uses code to "stream" the file when you download or click on a link.
So for simple HTML like pages? Well, for those that took a one day HTML course? Sure, such web sites will have scr=some path name to a valid url). And these simple systems thus allow you to enter a URL to grab/get a document. And those documents are fully exposed to the web site, and a simple valid URL path name to a file exists. Not so with asp.net, and as noted, this is not only done for security, but it a better over all developer experience to write code that grabs the files as opposed to rendering full path link names to files.
There are many additional benefits. For example, the database that drives this likely has a setting (or some settings) that contain the path names to the documents. If they run out of storage, or say want to move older files to a much slower storage system, which of course is much lower cost? Then can move the files, and update the path name columns in the database. The web site will continue to work, since we NEVER using a exposed URL on the web site. And as noted, actual direct URL's don't exist, and the web server (IIS) as opposed to the code behind will not even have rights to the file names.
As a result?
You not be able to simply pull the web page, and THEN extract the URL's to file names.
What you might be able to do is write code that loads the web page, and then scans all the event code stubs for the links, and have your code click on each button with web browser automation. But, even that don't allow you to enter file names into the download prompts.
So, what you ask is not easy, likely not possible, and a very difficult task. And the simple reason is that site does not use simple HTML and static links to files, and it never actually exposes a direct link to files, and even worse yet is the web server does not have or even allow a URL direct link to a site - they don't exist, and the web site will not even have rights or even allow such URL's to file names. (only the .net code behind does - not internet services).
and grabs the document and then code "streams" the file to to the web site or link you clicked on. So the simple HTML coders in the past would create say a folder (usually a virtual folder) that points to the files on some server/folder. But with .net, it easier (and far more secure).
Modern development tools don't use old fashioned ideas like a URL's to directly retrieve a file - they are designed differently.
In some cases, URL's are allowed or created, and this is done for reasons of sharing links. So if you have a cute video or document? Then the designers of the system will often permit use of parameters in the URL, so you can share a link to someone else. This page has no such provisions. So, you can share a link to the page, but no actual URL to documents or even provisions to allow URL's to a document even exists.
So this quite much means to retrieve a document, you have to go to that web page, and ONLY when you click on a document will the web site "stream" down that one particular document in question.
Related
I made an ASP.NET MVC application which allows user to create dynamic websites. I need to add feature which will allow to download from server off-line version of choosen website as static html files with menu, hyperlinks, images, documents etc. It should work similar to applications such as Teleport Pro, but I have to choose from Admin Panel which content should be export.
Client wants to burn static website on CD, save on pendrive.
Do you have any ideas how to begin? Please help.
I currently have implemented that in a current project...
User is able to change anything in the frontend and at the end he can publish and download the offline files... the site subscribe users and show all prizes, winners and more information about that campaign.
All was done in ASP.NET MVC3 under .NET4 and hosted in AppHarbor.
It's composed at several applications but for what you want, you develop the Backend and the Frontend, and to generate the static files, simple use the Frontend to grab the full HTML
As an example, I can show what 2 users did...
Callme.dk did http://callme.julekal.info and
Sony Nordic did http://sony.julekal.info
plus, you can simply point custom domains to it as well like http://sonynordicxmas.net/
To publish and generate all files:
one part of the editing:
So I give the users, offline access (through the .zip file), online access (through the frontend application) and the ability of using custom domains...
I think the only way this might be possible is if you go to every single page and then use your browser to "Save" the web page script and all.
However this causes several issues;
You never quite get everything and you need to massage the HTML produced, dowload all the images etc to get the page to look right
Each html file now has an associated folder with the same name and each time you do this you will get another html file with a folder. You can combine all the folders into a single one but that leads me to item 3.
You will need to edit each html file to clear up any pathing issues if you want to share a single source folder.
Data is no longer dynamic!
You need to, if you want to link all the pages to each other, edit every single html file and resolver the anchor tags.
This is too much work and I think it actually breaks the true requirement.
Don't do it! :)
I have a web application which consists of many aspx pages ... one of them shows a grid with rows that can be exported to a file via button click. This works fine. Now I want to have that feature which allows a user to access an external link to this page (or another) and to export to a file and download. I dont need any information on the page, just the file download. How could I do this also including security features like encryption?
Thanks :)
The easiest way to do this is simply implement an HttpHandler that contains the logic to create that file and write it to the Response stream.
There are lots of examples of how to do this on the web that I won't repeat in this question. Just do a Google search for "Download File HttpHandler" and you should be golden.
One of the search results: http://www.c-sharpcorner.com/UploadFile/jhblankenship/DownloadingFromMemStreamHttp11262005061852AM/DownloadingFromMemStreamHttp.aspx
What you're going to have to do is when the gridview shows the correct rows to provide a 'unique link' which will be your website URL with url variables at the end. When the page loads it can check these variables and then use the database to look up the correct data etc.
Encryption in transit will be done via HTTPS (SSL), and to secure otherwise you would require a login to view the gridview / file.
We have an application in classic ASP that allows user to 'attach' files to information. These can be PDFs, spreadsheets, Word documents, etc.
In the new ASP.NET version, one requested option was for a "Print All" (one user has a situation where there are 34 attached files and, in the current system, she has to open and print each one individually).
The files are kept in a separate directory - not embedded in the database. The database simply contains an ID number and the file's extension so the application would then go out and open "2182.xls" if the user wanted to see it (which would open an Excel window in that case).
Is there a way to send a file to a printer when all you have is the fully qualified filename? (Which I could presumably repeat 34 times in the above example)
Thanks in advance.
Remember that your asp.net code runs on the web server. It's not running on the client computer and has no knowledge of what printers (if any!) are installed there. That's just the way the web works; by design, all a web app can do is open the print dialog for the current page. Anything more, and hackers would use our web browsers for the same kind of spam you get at a fax machine.
That said, there might be some things you can do:
Add an activex control, flash app, silverlight app, firefox plugin, or other plugin code to support the feature.
Render all the attachments on the server into one html document with appropriate styles and javascript so it prints correctly and prompts the user on load.
If this if for a local intranet site (as opposed to public internet) where you have special knowledge of what printers are available for the current user, you could setup all the printers on your server and have it print to the correct location on behalf of the user.
Let's say we have a web site with a CF app that was written in-house.
Assume that:
Server 2003 IIS6 or 2008 IIS7 will be used
ColdFusion 8 will be used
Directory browsing is denied
SSL is required to connect
The account login process is secure (yeah I know that is a whole other
ball of wax but that concept is discussed ad nauseum on the web).
Say I have a file at https://domain.com/folder1/folder2/ with a name like picture92352.ext imagine it as a jpg or pdf or whatever. The entire path between the domain name and the file varies widely in naming structure, depth, etc. Files are not all lumped together in one folder.
The app restricts links by user such that a user would have to have access to that file to find it in the first place but as it stands now if a person knew the full URL to that file they could retrieve it without logging in to the app. It's the classic security by obscurity situation. A random person isn't likely to find a file they shouldn't get to but once someone is given access they know how to access it from another PC where their actions might not be traced back to them.
How do I restrict access to these files before someone logs in and still make them accessible to outside users after they log in? Is there a way to do it with permissions only or is the only answer to have code dynamically moving files around at the time of the request or is there some obvious step I'm not even thinking of?
Let me clarify this slightly. No matter how the file is presented on a page a user can use the browser IE, Firefox, etc to examine the URL the file comes from. If the image is a link there is always copy shortcut in the right click menu for IE and the same functionality in FF is called copy link location. If the image is displayed inline as part of the page an IE user can right click and choose properties to see the URL, in FF the same functionality is present to see properties but there is an even quicker more convenient option labeled copy image location. Once a user knows the URL to a file if the location or file name doesn't change they can use that URL without authenticating in the CF app.
If I change the NTFS/share permissions so that IUSR can't see the content then my CF app and IIS can't push it. What strategy do I use to provide the file in the CF app that doesn't leave this hole open?
You could write a CFM page that serves up the images. Then you just make sure they are authenticated inside the CFM.
<!-- something like this -->
http://localhost/GetFile.cfm?file=foobar.jpg
In GetFile.cfm, you would do something like:
<!-- the filename part is what the browser will pre-popualate the file name in the download dialog as -->
<CFHEADER name="Content-disposition" value="attachment;filename=picture92352.ext">
<CFCONTENT type="text/plain" file="\\fileserver\folder1\folder2\picture92352.ext">
Take a look at the various MIME types.
If you wanted to do something similar but keep a more natural URL, I think you would need to leverage the Java servlet underpinnings of ColdFusion to create a handler for any URL matching a certain pattern.
We have created a web application, using ASP.NET, that allows users to upload documents and attach them to business entities, like customers, contacts and so on.
The application runs on the intranet and all files are uploaded through the web application into a shared folder on the server.
I would like, right from the web page, for the user to open the actual file, edit it and then save the changes back to the original location. This is a piece of cake in a Windows environment, I'm just wondering what, if any, is the best way to handle this in a web environment?
The files are usually Word documents, Excel documents and images.
Clarification
We would display all the attachments in a list format. We would like it so that the user would click on an edit link and the file would be opened in the appropriate application, for example, Microsoft Word or Microsoft Excel. I think the file associations in Windows would already handle this. We are just trying to save our user the time to download the original file, make their changes, delete the old file, and the upload the new file.
SharePoint does this by exposing FrontPage extensions which Word and Excel know how to deal with.
If you want to look at a commercial product for ASP.NET that allows you to edit images with AJAX (no need for installed software), I work for a company that has one (Atalasoft)
WebDAV is probably what you want. (Free)
If all your client computers are Windows, map a shared folder on the server to the same drive letter on every client and use the file:// format.
Let's say you share \ServerName\ShareName to H: on every client's computer, the you can make the link as file://h:\pat_to_the_file_under_your_share\fileName.doc
If not every one of the client's computers are in Windows, then you might try to make your links as follows (not sure if ot works):
file://\ServerName\ShareName\pat_to_the_file_under_your_share\fileName.doc
I'm trying to do something with using file:// instead of http:// but it's real sporadic based on the browser. Seems to work fine in IE, okay in Firefox, and goes nowhere in Chrome.
Looks like I may just be stuck with downloading, editing, and re-uploading the document.
It sounds like you want something similar t eRoom, where the browser works in conjunction with a component that intercepts a stream from http, stores it in a temp folder, then fires up Word or Excel and allows you to edit the stream.
You may have to create a component that will intervene and create a temporary local copy of the file.
This tool should do what you need.
http://www.dlitools.com/dlitools/dlitoolsHome.nsf/0FA6B8B31F831F468525736B0001C606/4BBD7E8684EA8DB78525754E006C63A3?OpenDocument