Is it possible to see all publicly-accessible files on a website? - web-scraping

I would like to query a website that provides files for download to see all the available files for download.
For example: webpage called https://download.website.com/path/to/file has a file of interest to me, but I would also like to see other available files available in the system publicly.
Essentially I would like to be able to view a hierarchy of all of the publicly-facing files given some parent link. So if I know I want all files stored under https://download.website.com/path/, the query would turn up a recursive list of available files from https://download.website.com/path/*.
Is this even possible to do for most websites? Would allowing this behavior be too compromising to web frameworks in general, so it might not exist? Am I XYing out of control?
Any help here greatly appreciated.

This method isn't perfect, but you can try it. Just put this query in Google Search.
You can do a Google search for some publicly available and indexed path.
For example if you want to search all available page on a website/domain:
site:download.website.com
If you want to search all PDF files in this site:
site:download.website.com filetype:pdf
If you want to search all links with path download.website.com/wp-content/:
site:download.website.com inurl:/wp-content/
I hope it will help you a little bit.

Related

what stops people from downloading any website

I just learned that you can actually download an entire website using programs like httrack or IDM, what stops people from using these programs to download the whole Netflix library for example, and never pay for a subscription, it shouldn't be that easy so can someone tell me what's the catch?
Movies and shows are basically stored in separate servers/departments, and downloading just the HTML would not give you access to any of the other files. Think of it as viewing the page source for any other website, like even StackOverFlow, you cannot view the CSS, javascript, or any other files of it. You are only able to see the HTML.
BTW as a heads up, this is not a quality question and does not meet the guidelines of StackOverFlow, I would suggest you ask these type of questions in the communities of StackOverflow.
I'm pretty sure movies and series are stored on different servers, downloading the HTML of a website doesn't give you access to their files.

Plone - attach multiple files

I do not have much experience with plone, I searched on other forums an answer to my problem but without success. I saw that with plone you can enter text via "Add" event, news, page, file .... is to add a file that I have encountered a problem, that I can not add more files. I need to add multiple files with a single description, there is a way to do it?
Thank you !
Supposing you are looking for a possibility to batch-upload files into a folder, there are several add-ons for help out there. I live happily with collective.quickupload, but you might want to make your own choice and have a look at the other products, too, as for example:
wildcard.foldercontents (this will be the default behaviour in Plone 5)
ATMassloader
collective.upload
collective.uploadify
PloneFlashUpload
PloneJUpload (Java)
sc.base.uploader
Also: Plone/Zope has a native FTP/WebDAV-support. You can connect to a running Plone using a Webdav client, upload files and see them loaded in the CMS.
Additionally plone.formwidget.multifile allows you to add a file-field to your contenttype, which is capable to select multiple files at once for the upload in one single file-field. Haven't seen an out-of-the-box integration into a contenttype around, though, just in case you'd like to do it :-)
Might be interesting, too in this context:
Browse-able ZIP
collective.archiveviewer

Embedding HTML Help in a QT application

I write an application, with Qt and it has a pretty big help system, which is including images, tables, links, etc... This help system is stored on a disk in a specific location as simple HTML files, together with the images and using a QWebkit control it's loaded in the application on request, and the user can browse it, etc...
Now, I would like to move away from my "in-house, file based" help solution, meaning: I want a help system which still will use my HTML files, but without the need to ship 100 files to the user, only 1 or 2.
I am aware of Qt's help system ( http://doc-snapshot.qt-project.org/4.8/qthelp-framework.html ) but I am wondering if there are any other solutions providing the same or similar functionality.
What I am looking for:
take all the HTML files and create one of them, including images
reader possibly embeddable in application, but separate application is good too.
multiplatform
support for context sensitive help (ie: "Click on what's this", help shows relevant page)
For huge help docs the best way is to use Qt Help Framework. For example QAssistant is done in this way.
It mainly consists of several html like files and index files. Then you "compile" them to get a binary(and compressed) version. You can split help in sections and so on...
I am using QWebView to display HTML help files. You get a widget and can integrate it in your application.
Qt have a good easy example here: http://qt-project.org/wiki/Open_Web_Page_in_QWebView
Since it is a widget, you can greate a stand-alone or integrated look and it it very flexible.

How to add and share source code files in a WordPress based site?

I'm planning a personal/portfolio web site based on WordPress. What I would like to have is a list of example webdev projects/plugins/widgets along with the source code available for browsing in the least obtrusive way (if possible to skip downloading, going to another site, etc).
The alternatives:
The simplest:
Upload the code at github, sourceforge, launchpad, google code, or similar.
Share the link to the projects source code in the respective section in my site.
The easiest:
Use an existing WordPress plugin for exposing part of the uploads directory where I can upload the projects' source code.
Use a shortcode/widget/custom page for displaying the tree view with the projects and the source code within a WordPress page(s).
The most realistic:
Write the WordPress plugin from above. From my initial research, there is no such plugin for exposing the uploads directory files in the user and/or admin section of a WordPress site.
From my initial ideas the plugin is basically a file browser with a fancy tree view and a view panel for the source code file contents
(Nice to have) AJAX-ify the plugin to view the source code contents in a DIV with syntax highlighting.
What's your take on this?
Thanks
I think what you list as the simplest option is also probably the best. You have all the tools that people are used to -- syntax coloring, etc. -- and all you have to do is make a comment or two and then link to it. This may sound silly, but there is also a certain gravitas lent to your code because it's not "just on some WordPress site".

How to display a list of files in drupal?

I am very green at Drupal and I would like to recreate a list of files like the table that is shown at any module page in drupal.org [example]: it shows a table with the recommended releases and development releases. Each entry has a link to the file and some release notes.
What I would like is to clarify how this should be done, since I am new at drupal and I tend to be a bit confused. I am currently thinking that this is just a View, but I'm no sure.
In addition to this, I would also like to allow the download of these files to a specific role. I have managed to control this to pages, but not to attachments. Any ideas for this issue as well?
For the files, if you want access control on your files, you need to use the private files system which you can setup in the file settings. There are different modules that can help you with different tasks, like uploading etc.
How to best generate a table really depends where the files are coming from. If they are attached to nodes with CCK, it will be quite easy to use views. You can setup the fields, and how it should be displayed. If you want to create a download link, I believe you should use the generic file format.

Resources