I am writing a program that polls an SFTP directory and downloads very large files files. I don't want to partially download a file as its being uploaded to the server.
I have had success by checking the file size. The SFTP server I am testing with reports the file size as 0 until the upload is complete, but I am not sure whether this is a standard for all the SFTP servers I will read from.
Related
I am beginner to WebDAV/HTTP protocol. I want to transfer files across network using WebDAV. The source received camera image raw data files has to be placed in a webDAV server (a folder). These files are then copied by a remote client via webDAV. my questions below
Is it necessary to use webDAV/HTTP GET/POST/PUT method calls on the server machine to copy images
or just a normal Linux/QNX command shall work?
If we do not need to go via WebDAV just to copy files then how can we attach properties to these
files like file name and size? are these automatically supplied to remote client by webDAV using
some OS file system calls?
Thank you in advance for your answers
I've built a shiny app that requires user to upload data. I have a file of around 11.2MB and its upload is almost instantaneous locally. But when I deploy the app on an instance of shiny server on our company server, it takes around 5-10secs to upload the same file. And it gets worse if the file uploaded is 100MB.
Is there a setting in shiny-server (probably in shiny-server.conf) to determine file upload speed? Or is this behavior caused by other factors?
I have my Dash plotly app running in PCF, my app.py runs based on a excel file which is uploaded to pcf along with app.py, but the excel feed changes daily, so daily i am uploading the new file to pcf using "cf push", is it possible to avoid that, like making pcf to read excel from my file system instead of uploading the new excel file to pcf cell container everytime
Basically you need some persistent storage attached to your container so app can refer the available file at run time. There are the options that can be explored:
If NFS is enabled at your end then you can mount the file share and pick the files from that location directly.
Otherwise you can have another PCF service
(just to keep it separate for better management) that can pull the
files from your server using sftp and transfer to S3. Amend your app
to refer the file from S3.
I am having hard time figuring out how to get the file InputStream from file upload Post request to server, before it gets completely loaded into memory.
This is not problematic for smaller files, but I am worried what happens after trying to upload a larger file (1 or more GB). I found a possible solution with using HttpContext.Request.GetBufferlessInputStream(true), but this stream includes the whole request not just the uploading file and if I use it to upload a file for example into the Azure Blob Storage or anywhere else I end up with the corrupted file. I also lose all the information about the file (file name, size, etc.).
Is there any convenient way of uploading a large file to server without filling its memory? I would like to get the stream and then use it to upload a file anywhere in chunks.
Thank you.
I used DevExpress UploadControl for a similar task. It supports large file upload by chunks. A temporary file is saved on a server hard drive and you can get it using FileSteam without full loading in server memory. It also supports direct upload to Azure, Amazon and Dropbox.
The same is true for their MVC Upload control.
There is a torrent file in which the web seed is configured. Most files are loaded normally, but when downloading some files (text\american.ini) the connection to the server suddenly terminates and the download stops. This can be checked if, when adding a torrent, select only this file to download. At the same time, this file is loaded normally from the browser. Because of what it can be? Tested on uTorrent and libtorrent.
Here you can download the torrent file and check it personally.
Download
There are two different kinds of Web Seeds, BEP 19 and BEP 17, and one assumes that the server is configured to handle working with torrent clients, your torrent has a BEP 19 link that's supposed to point to a file or a directory that has the same name as the torrent and that directory should contain the files in the torrent.
Your torrent name looks like this:
files/licence.txt
and your Web Seed looks like this:
https://website.com/projects/crmp/
It's not working because the Web Seed URL is wrong.
The problem was that FileZilla, when downloading some files to FTP, changed them and the torrent considered these files different. Solution: change the transfer mode from ASCII to binary.