I need to create a script that generates some URL's from a specific website and copy of its content to somewhere like as txt to parse some data from it afterward. I am able to generate URL's but I have no idea how to get the pages content to a text file.
I am using classic asp and I will be appreciated if you could give me an example on how to do it.
The page I need to copy contains only plain text, which is safe to keep in a text
you want to use the Fil System Object, here's a thorough example from 4Guys:
https://web.archive.org/web/20210506122630/http://www.4guysfromrolla.com/webtech/040699-1.shtml
One thing to note that I've seen people run into problems with, and that's permissions. In the url tutorial above, they mention saving to your c: root for a file location:
set myFile = fso.CreateTextFile("C:\test.txt", true)
Unless you set up special permissions in the root of your C:\ drive, which I'd strongly recommend NOT doing as it's a hge potential security risk, it will not work. I would recommend creating a folder off of the root of your web site, as permissions will probably already be in place (this is obviously reliant on what your requirements are as I'm just guessing).
Related
This is probably a dumb question, but I'm worried :
I have published a website on a server, then made some changes to a css file.
As the css file was already cached by my browser, it didn't display the changes.
deleting the cash allowed to display the changes.
Now my worry is that if some users have previously been to the website, and it is cached by their browser, if I make a change they wouldn't be able to see it.
How do you guys prevent this ? Do you just change the file names ?
Sorry for my noobness,
Thanks.
There are a number of solutions floating around the web, but as far as I can tell they all boil down to changing the CSS filenames whenever their content changes. That way you steer clear of user caches and server caches serving old content.
Variants:
Instead of changing the name of the file itself, create a symbolic link with a new name to the old file whenever content changes.
Instead of changing the name of the file, change the way it is referenced by the page. Replacing myfile.css?v=1 by myfile.css?v=2 circumvents people's caches.
Write code that automatically changes the name or the link name or the way the file is referenced
Use a framework that does one of the above.
And: remember that the same problem applies to any content that might be cached, like JS files.
I tried to make my title as direct as humanly possible.
I wrote some code, saved it in a file. I want the name of that file, and the disk path to it, at the moment it runs. This is for a website, and I have access to HttpContext.
I wrote some code in a Razor file. I put that file on my IIS website, and it got executed by some convoluted hierarchy steps with partials and the like. I want to write some code that spits out the path to that file.
I don't want to hard-code the filename and its current path in my code. I want to copy the magic piece of code I'm asking for here onto lots of other Razor pages, so each time I can spit out a comment in some script or style tag somewhere so that other developers can track down my mistakes amazing work.
I don't want to know how the end-user's browser got to my file. I'd use Server.MapPath or Request.ServerVariables["SCRIPT_NAME"] if I wanted that.
I don't want to know merely what directory my website app is running in. I'd use HttpContext.Current.Request.PhysicalPath for that.
Does anybody know how to do this?
How to prevent exe file upload in asp.net mvc.
If you are trying to prevent users uploading dangerous content, preventing them to upload exe files isn't enough. This is a black list approach. Much better is to ask yourself what are the valid file types you do support and block all others. This is a white list.
To allow certain file types you can check for the extension, but perhaps also validate the file header (the first couple of bytes from a file) to detect whether it is actually of the type you expect. You will have to figure out for each file type what the possible headers are.
Good luck.
So I'm writing a Django based website that allows users select a color scheme through an administration interface.
I already have middleware/context processors that links the current request (based on domain) to the account.
My question is how to dynamically serve the CSS with the account's custom color scheme.
I see two options:
Add a CSS block to the base template that overrides the styles w/variables passed in through a context processors.
Use a custom URL (e.g. "/static/dynamic/css/< website_id >/styles.css") that gets routed to a view that grabs all the necessary values and creates the css file.
I'm content with either option, but was wondering if anyone else out there has dealt with similar problems and could give some insight as to "Best Practices".
Update : I'm leaning towards option number 2, as I think this will allow for better caching down the road. So it's dynamic the first time, gets stored in memcache (or whatever), and invalidated when a user updates their settings in the admin site.
Update: Firstly, I'd like to thank everyone for their suggestions thus far. All the answers thus far have focused around generating static files. Though this would work great in production, it feels like a tremendous burden during development. If I wanted to add a new element to be styled, or tweak existing styles I'd have to go through and recreate each and every css file. Sure, this could be done with a management command, but I just don't feel it's worth it. Doing it dynamically would add 1 maybe 2 queries to each page load, which is something I'm not worried about at this stage. All I need to know is that at some point I will be able to cache it without rewriting the whole thing.
I've used option #2 with success. There are 2 decent ways of updating the generated static files that I know of:
Use a version querystring like /special_path.css?v=11452354234 where the v parameter is generated from a database field, key in memcached, or some other persistent file. Version gets updated by admin, or for development you would just make the generation not save if the parameter was something special like v=-1. You'll need a process to clean up the old generations after some time.
Don't use a version querystring, but have it look first for the generated file, if it can't find it, it generates it. You can create a cron job or WSGI app that looks for filesystem changes for development, and have a hook from your admin panel that deletes generations after an update. Here's an example of the monitoring, which you would have to convert to be specific to your generations and not to Django. http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Monitoring%5FFor%5FCode%5FChanges
Could generate the css and store it in a textfield in the same model as the user profile/settings. Could then have a view to recreate them if you change a style. Then do your option 1 above.
Nice question.
I would suggest to pre-generate css file after colors scheme is saved. This would have positive impact on caching and overall page loading time. You can store your css files in directory /media/css/custom/<id or stometing>/styles.css or /media/css/custom/<id or sth>.css and in template add <link rel="stylesheet" href="/media/css/custom/{{some_var_pointing _to_file_name}}" />
You can also do the trick with some random number or date in css file name that could be changed each time file is saved. Thanks to this browser will load the file immediately in case of changes.
UPDATE: example of using model to improve this example
To make managing of those file easy you can create simple model (one per user):
class UserCSS(models.Model):
bg_color = models.CharField(..)
...
...
Fields (like bg_color) can represent parts of your css file. You can ovveride save method to add logic that creates css file for user (by rendering some template).
In case your file format change you can make changes in your's model definition (with some default values for new fields), make little changes in template and run save method for each exisintg instance of class. This would renew your css files.
That should work nicely.
I would create an md5 key with the theme elements, store this key in the user profile and create a ccs file named after this md5 key : you gain static file access and automatic theme change detection.
We're building a CMS. The site will be built and managed by the users in aspx pages, but we would like to create a static site of HTML's.
The way we're doing it now is with code I found here that overloads the Render method in the Aspx Page and writes the HTML string to a file. This works fine for a single page, but the thing with our CMS is that we want to automatically create a few HTML pages for a site right from the start, even before the creator has edited anything in the system.
Does anyone know of any way to do this?
I seem to have found the solution for my problemby using the Server.Ecxcute method.
I found an article that demonstared the use of it:
TextWriter textWriter = new StringWriter();
Server.Execute("myOtherPage.aspx", textWriter);
Then I do a few maniulatons on the textWriter, and insert it into an html file. Et voila! It works!
Calling the Render method is still pretty simple. Just create an instance of your page, create a stub WebContext along with the WebRequest object, and call the Render method of the page. You are then free to do whatever you want with the results.
Alternatively, write a little curl or wget script to download and store whichever pages you want to make static.
You could use wget (a command line tool) to recursively query each page and save them to html files. It would update all necessary links in the resulting html to reference .html files instead of .aspx. This way, you can code all your site as if you were using server-generated pages (easier to test), and then convert it to static pages.
If you need static HTML for performance reasons only, my preference would be to use ASP.Net output caching.
I recommend you do this a very simple way and don't do it in code. It will allow your CMS code to do what the CMS code should do and will keep it as simple as possible.
Use a product such as HTTrack. It calls itself a "website copier". It crawls a site and creates html output. It is fast and free. You can just have it run at whatever frequency you think is best.
It decouples your HTML output needs from your CMS design and implementation. It reduces complexity and gives you some flexibility in how you output the HTML without introducing failure points in your CMS code.
#ckarras: I would rather not use an external tool, because I want the HTML pages to be created programmatically and not manually.
#jttraino: I don't have a time interval in which the site needs to be outputted- the uotput has to occur when a user creates a new site.
#Frank Krueger: I don't really understand how to create an instance of my page using WebContext and WebRequest.
I searched for "wget" in searchdotnet, and got to a post about a .net class called WebClient. It seems to do what I want if I use the DownloadString() method - gets a string from a specific url. The problem is that because our CMS needs to be logged in to, when the method tries to reach the page it's thrown to the login page, and therefore returns the login.aspx HTML...
Any thoughts as to how I can continue from here?