Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I often have to work with fragile legacy websites that break in unexpected ways when logic or configuration are updated.
I don't have the time or knowledge of the system needed to create a Selenium script. Besides, I don't want to check a specific use case - I want to verify every link and page on the site.
I would like to create an automated system test that will spider through a site and check for broken links and crashes. Ideally, there would be a tool that I could use to achieve this. It should have as many as possible of the following features, in descending order of priority:
Triggered via script
Does not require human interaction
Follows all links including anchor tags and links to CSS and js files
Produces a log of all found 404s, 500s etc.
Can be deployed locally to check sites on intranets
Supports cookie/form-based authentication
Free/Open source
There are many partial solutions out there, like FitNesse, Firefox's LinkChecker and the W3C link checker, but none of them do everything I need.
I would like to use this test with projects using a range of technologies and platforms, so the more portable the solution the better.
I realise this is no substitute for proper system testing, but it would be very useful if I had a convenient and automatable way of verifying that no part of the site was obviously broken.
We use and really like Linkchecker:
http://wummel.github.io/linkchecker/
It's open-source, Python, command-line, internally deployable, and outputs to a variety of formats. The developer has been very helpful when we've contacted him with issues.
We have a Ruby script that queries our database of internal websites, kicks off LinkChecker with appropriate parameters for each site, and parses the XML that LinkChecker gives us to create a custom error report for each site in our CMS.
I use Xenu's Link Sleuth for this sort of thing. Quickly check for no deadlinks etc. on a/any site. Just point it at any URI and it'll spider all links on that site.
Desription from site:
Xenu's Link Sleuth (TM) checks Web
sites for broken links. Link
verification is done on "normal"
links, images, frames, plug-ins,
backgrounds, local image maps, style
sheets, scripts and java applets. It
displays a continously updated list of
URLs which you can sort by different
criteria. A report can be produced at
any time.
It meets all you're requirements apart from being scriptable as it's a windows app that requires manually starting.
What part of your list does the W3C link checker not meet? That would be the one I would use.
Alternatively, twill (python-based) is an interesting little language for this kind of thing. It has a link checker module but I don't think it works recursively, so that's not so good for spidering. But you could modify it if you're comfortable with that. And I could be wrong, there might be a recursive option. Worth checking out, anyway.
You might want to try using wget for this. It can spider a site including the "page requisites" (i.e. files) and can be configured to log errors. I don't know if it will have enough information for you but it's Free and available on Windows (cygwin) as well as unix.
InSite is a commercial program that seems to do what you want (haven't used it).
If I was in your shoes, I'd probably write this sort of spider myself...
I'm not sure that it supports form authentication but it will handle cookies if you can get it going on the site and otherwise I think Checkbot will do everything on your list. I've used as a step in build process before to check that nothing broken on a site. There's an example output on the website.
I have always liked linklint for checking links on a site. However, I don't think it meets all your criteria, particularly the aspects that may be JavaScript dependent. I also think it will miss the images called from inside CSS.
But for spidering all anchors, it works great.
Try SortSite. It's not free, but seems to do everything you need and more.
Alternatively, PowerMapper from the same company has a similar-but-different approach. The latter will give you less information about detailed optimisation of your pages, but will still identify any broken links, etc.
Disclaimer: I have a financial interest in the company that makes these products.
Try http://www.thelinkchecker.com it is an online application that checks number of outgoing links, page rank , anchor, number of outgoing links. I think this is the solution you need.
Related
I am looking for the ability to enhance the appearance of the directory listing pages of a very basic corporate intranet I developed for use by our employees. I am using Windows Server 2012 R2, and the site is deployed in Internet Information Services.
I did not use anything like Visual Studio to create this, and I have already handwritten all of the CSS and HTML for the index page that serves as a jump-off point for the rest of the site so that users can get to the content they need; however, the directory listings pages where the users land leave quite a bit to be desired to say the least.
I just want to be able to add some quick styles to these directories, such as modifying the font family and perhaps the link styles. Nothing major, really. The site already functions perfectly for what it was designed to do, and has been for years. This is just something that's always kind of bugged me but I never devoted any time to it. I'd like to do that now. Ideally I'd be able to just add something in the web.config file like inline CSS, or perhaps link it to a .css file that will house the styles. The latter is probably preferred, actually, but any way is fine.
Any help is greatly appreciated. To get this out of the way early: yes, I have done lots and lots and lots of searching on this topic — I'm talking hours. I have not been able to find a solution that seems to meet my needs. I consider posting here as somewhat of a last resort because I understand that it's a free resource and users here are usually quick to let other users know when they didn't find a particular article that seems to offer the solution they're seeking — which is usually a result of not knowing exactly what keywords to use — and I don't want to waste anyone's time. Just know that I have tried everything I know to find the solution, and that I'm genuinely stumped and looking for help from some pros.
Thank you!
Since the directory page is not HTML, you can't directly style it with CSS. However, there are a few options for changing the way it looks.
Write a script to point to your own, custom-styled, directory page. See this forum thread for tips on how to do that and a sample script.
Create a custom page using this module that you can further customize yourself.
Use the DirectoryListing open source app, which allows for customization of the directory page.
Either one of those solutions should give you more control over how the directory page looks.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
If it were a windows program, I would use Auto it to automate testing. Is there something similar for browser-based apps?
Nothing too complex, it should just allow scripting (preferable for me to macro-recording) to simulate human interaction with the browser, which means being able to identify fields of a form by name, inject text into some, simulate mouse-click on others, etc and then, after submitting a form, should be able to read text certain named controls, check the status of others (checked, radio group index, read-only, etc).
While I do appreciate a full featured product, I don't appreciate a steep learning curve. so something as simple as the scripting of Auto It woudl be fine.
I don't know if it makes a difference which browser is used, but I could live with MSIE 6 or higher (maybe 7 or higher at a push).
Edit: Three posters so far (out of three) recommend Selenium. One says it is FireFox only and I need to test MSIE, however some quick googling shows that this should be possible (although you might have to set everything up in FF first, then switch over to MSIE).
See http://www.emadibrahim.com/2009/01/19/testing-web-app-with-selenium-explorer-and-net/ and http://wiki.openqa.org/display/SEL/Selenium+Core+FAQ Hope this helps someone.
You have several options to choose from:
Selenium
Watin/Watir
HtmlUnit
Sahi is another option. (We user sahi and selenuim)
Selenium would an the ideal tool for this.
I'd recommend Selenium, though it's for Firefox.
Try out Celerity. It renders javascript and works without a browser
Selenium is widely used, mostly because of their recorder (which is supported only in FF, but you can then replay the tests hitting IE). Its language is commonly referred as "Selenesse".
It's also a great choice if you expect to do some performance testing in a grid... but it seems this is not your case.
However, I would recommend not to use Salenium/Selenesse; instead, look for Selenium + Webdriver (Selenium 2), which is still in beta. The language construct are much better (but you don't have a recorder).
Other good option is Sahi. I found its API to be easier, in the sense that Sahi's code tries to deal itself with the problem of finding the button/link/etc you want him to click. For example, instead of relying in complex XPath expressions, you can say something like "click the link tag nearest to the div with attribute id="menu" (you can do this with one line of code). It has also a recorder.
I want to recommend something I found last week and definitely think that this is interesting.
QA Agent (http://qaagent.com)
It allows you to create web automated test in your browser. I really liked that it is free and you don't need to install anything. It looks like this is a new initiative so there are a lot of improvements which can be done.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I would like a general purpose code snippets library, that I could use to store bits of code for any language, organized in folders. When any developer on the network adds a snippet, it should get transferred to others.
How can I do this simply? Any app that could be used for sharing libraries of text should be okay too.
That's called "a wiki".
We use a Sharepoint Wiki page for exactly this purpose ... that way code instructions and examples can be included in the same place.
A wiki is definitely the way to go but why stop there? Why not go with an issue tracking system that adds wiki functionality?
I have first hand experience with Trac and Redmine and openly recommend both of them. We've recently switched from Trac to Redmine where I work and I think I like it a little more than Trac, and use the wiki functionality almost everyday.
Update:
If you are looking for something smaller scale, there was a previous discussion here on SO that looks like it has some interesting links.
You might be interested in snip2code.
They do exactly what you are talking about... searching, collecting and sharing snippets of any language. It solves the problems I had for similar issue.
Hope it may help
Code Warehouse is the most extensive source code snippets manager. Code Warehouse is multi-user ready, which means that an organization’s entire development team can share a central code library and benefit from code that is already written and tested.
alt text http://www.xcca.com/cwbig.png
Please have a look at gist.github.com. This is definitely much better than a wiki sharepoint.
Plus you can reuse snippets from other user.
Wiki were designed to share any type of content. Some tools have been specifically designed to share code snippets. Some IDEs also have snippet features.
A wiki is definitely not the best match because it will not integrate with your IDE requiring you to have a separate tool open.
I'm going back and forth between a few Mac applications (haven't decided on one yet but I have tried quite a few). Most of these options allow you to host your snippet library on Dropbox or another network storage location so you can share your library with others. I don't know of any that provide any sort of live changes monitoring though.
Text Expander
Though not designed as an actual code snippet application, I used this for that purpose for a while and was quite content with it. You can share your library with others whether they're on Mac or Windows because there is a compatible alternative for Windows (i forget the name of the Windows alternative but it's easily found on Google I'm sure)
Finally decided to move on because I just wanted something a bit more robust. But may be an option to consider for some.
Codebox
Dash
I just learned about this one and actually only just started trying it. So far it's pretty good and I haven't decided whether or not I'll stick to this or go back to Codebox. I really like the documentation library functionality it has but as far as storing snippets, with my limited experience thus far, all I can say, is I'm rather disappointed in the codehighlighting. It also doesn't have additional fields to write comments like some of the other applications do so your comments have to actually be written with the code. I can't say that bothers me though, it's actually a good thing.
Snippets
I was using this one for a while and liked it quite a lot but wanted something that I could not only store small snippets in but also full source files like a few self contained javascript libraries. I attempted to store a large javascript file and the program not only couldn't handle it, but it actually corrupted my entire snippet library which I was hosting on Dropbox. Not sure if it was a self contained problem because I immediately uninstalled the app after that. But overlooking that i
A few others I may try but have no opinion about thus far
Code Bank (open-source)
Code Collector Pro
Last but not least.. I've actually just been using Gist more and more lately. I'm not aware of any desktop application for Gist (there's a browser app for Chrome I think and a couple of third party applications on the AppStore but I haven't tried them)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 months ago.
Improve this question
I'm into my first 3 months of web development and I have been dabbling with some server side scripting in the form of ColdFusion, along with some Javascript, JQuery and CSS.
I have read about CSS optimization and would like to know what are the other pertinent factors contributing to the better performance of a website. What all factors can a developer profile and optimize?
How much part does picking (or rather I should say recommending) a particular browser play in this performance hunt?
cheers
Install YSlow and Pagespeed plugins for Firefox. Then start looking at all of the ways your site is unoptimized. This will be much like trying to take a sip of water from a fire hydrant.
Using minified ( and possibly aggregated ) Javascript and CSS along with a good, healthy far-future-expires is a really good way to start.
Don't forget to gzip.
And use Etags.
And put your CSS at the top of the document.
And put the javascript at the end.
And use separate domains for static resources.
And avoid URL redirects.
And remove duplicate javascript and CSS.
And... see what I mean about the fire hydrant!
Just to sum the stuff up from above:
The speed of a website depends on a few things:
Server
Connection
Client
And on each of this part you can do improvements.
Server: if you rely on a database, check if you queries are cached, and more importantly check if your data is cached. For example if on every page you get a menu from the database, then you can cache that result. In addition you can check your code and see if there is room for optimization.
Also the hardware itself plays a role. If you are on a shared hosting plan, maybe the server is full of other not-optimized apps that take a toll on the server.
Connection: Here the YSlow and Pagespeed come in handy, as well as Fiddler. You can do some caching of static content (CSS and JS). Set their expire date far in the future. Using GZIP to make their contents smaller, and combining the static files helps to a certain extent.
In addition maybe the server has a low bandwidth.
Client: if you do wacky javascript or have slow css selectors, this might hurt performance on the client. But this depends on the speed of the client's computer.
I'd recommend reading Best Practices for Speeding Up Your Web Site and all of the content on Yahoo!'s Exceptional Performance page.
If you like books, you may be interested in High Performance Websites (note that a lot of the content in this is in the Best Practices for Speeding Up Your Web Site article) and Even Faster Websites.
Here are a few of my favourite rules from Best Practices for Speeding Up Your Web Site:
Minimize HTTP Requests
Add an Expires or a Cache-Control Header
Gzip Components
Make JavaScript and CSS External
Minify JavaScript and CSS
Also, smush.it is good for compressing images (which have a big impact on how fast a webpage loads).
As far as browsers go, Safari 4 claims it is "the world's fastest browser", and I can say that the Mac version is certainly nice and fast (not to mention elegant!). However, the above suggestions will make much more of a difference than which browser you're using.
Steve
With ColdFusion you will want to make sure your queries are being cached. Use query analyzer (if using mssql server) to make sure a slow loading page isn't the result of a bad query. On the database end you'll also want to ensure proper indexing.
A big factor in performance is how many HTTP requests are sent for images, files, etc. YSlow will show you this info. Its only available for firefox.
I'd recommend this book.
Google is currently collecting all sorts of performance tips on their new 'Let's make the web faster'-page here: http://code.google.com/intl/de-CH/speed/articles/
FYI: Not all information on these pages are valid, particularily the PHP tips are way off.
There is a really great plugin for for Firefox called Dust-Me Selectors. It scans your css files and lets you find selectors that aren't used/have become redundant in your markup.
https://addons.mozilla.org/en-US/firefox/addon/5392
You should also be delivering your static content off a CDN. Parallel downloads of static files will speed up your page renders. A better explanation here: http://www.sitecheck.be/tips/speed-up-your-site-using-a-cdn/
You shouldn't recommend any particular browser, but design your webpage to current standards with some fixes for older models, if necessary. From my perspective everything can have a speed impact, but CSS is the least important one and in real world examples the user will not notice this. In most cases a clear separation of html and style declarations will do the job. What really has an impact? First of all you can simply throw money at the problem by getting a better hosting contract (maybe a dedicated server). Another thing to improve the speed a website takes to load is to reduce the quality of your images and the usage of CSS-Sprites. Very often on dynamic webpages the database is a bottleneck and therefore caching and a good database abstraction layer can improve things (PHP: PDO instead of simply using mysql()). GZip your output to the user. There are so much more things, but a lot of them are very language dependent..
I recommend the use of FireBug and loadimpact.com for testing.
Less files are more - CSS sprites may be something to consider. In my experience, you have to balance your CSS file between speed and maintainability - one rule more or less won't make the difference between night and day...
For IE, see http://www.fiddler2.com/fiddler/Perf/
The new neXpert plugin for Fiddler (http://www.fiddler2.com/fiddler2/addons/nexpert.asp) offers features similar to those found in YSlow and PageSpeed.
The biggest problem I have is creating fast-running, beautifully designed pages with rich content. This is one thing that is extremely hard to do with today's technology.
If you have lots of javascript you might wanna use Javascript compression. Dojo provides one such tool SHRINKSAFE to compress your javascript. Find the link below:
http://www.dojotoolkit.org/docs/shrinksafe
There is another tool open sourced by google called page speed, which can help you optimize the website performance. It was used internally before it was open sourced to everyone recently.
http://google-code-updates.blogspot.com/2009/06/introducing-page-speed.html
http://code.google.com/speed/page-speed/
Hope it helps.
A couple of very basic rules of performance testing:
Performance means nothing if the program/web page/whatever is wrong.
Do not try to improve performance without having a reliable form of measurement.
You must profile your site/program/whatever to find out what is making things slow.
Corrolary: Do not just change things at random to see if things get better.
Cache everything (web server and browser caching).
Statically publish as much as possible (i.e. to reduce the amount of database calls)
Also add multiple waiting icons to your website.
Show icons in such way that every time user should get different waiting icon, which should be effective to engage user. And mean while your website will get loaded.
There are probably thousands of applications out there like 'Google Web Accelerator' and all kinds of popup blockers. Then theres header blocking personal firewalls, full site blockers, and paranoid cookie monsters.
Fortunately Web Accelerator is now defunct (I suggest you read the above article - its actually quite funny what issues it caused) but there are so many other plugins and third party apps out there that its impossible to test them all with your app until its out in the wild.
What I'm looking for is advice on the most important things to remember when writing a web-app (whatever technology) with respect to ensuring the user's environment isnt going to break it. Kind of like a checklist.
Whats the craziest thing you've experienced?
PS. I may have linked to net-nanny above, but I'm not trying to make a porn site
The best advice I can give is to program defensively. For example, don't assume that all of your scripts may be loaded. I've seen cases where AdBlocker Plus will block 1/10 scripts that are included in a page just because it has the word "ad" in the name or path. While you can work around this by renaming the file, it's still good to check that a particular object exists before using it.
The weirdest thing I've seen wasn't so much a browser plugin but a firewall/proxy configuration at a user's workplace. They were using a squid proxy that was trying to remove ads by replacing any image HTTP request that it thought was an ad with a single pixel GIF image. Unfortunately it did this for non-GIF images too so when our iPhone application was expecting a PNG image and got a GIF, it would crash.
Internet Explorer 6. :)
No, but seriously. Firefox plugins like noscript and greasemonkey for one, though those are likely to be a very small minority.
Sometimes the user's environment means a screen reader (or even a braille interface like this). If your layout is in any way critical to the content being delivered as intended, you've got a problem right there.
Web pages break, fact of life; the closer you have been coding and designing up against standards, the less your fault it is.
Something I have checked in the past is loading some of the more popular toolbars that people tend to install (Google, Yahoo, MSN, etc) and seeing how that affects the users experience.
To a certain extent it is difficult to preempt which of the products you mentioned will be used by your users since there are so many. I would say your best bet is to test for the most frequent products that your user base may employ and roll with the punches for the rest. If you have the time to test other possible scenarios, by all means do.
Also, making it easy for your users to report possible issues also helps lessen the time it takes to get a fix in place should it be something you can work around.