If I'm sending a single EDI file, then I can set the outbound filename in the orchestration by setting FILE.ReceivedFileName in a shape, and then using the %SourceFileName% macro when configuring the Send port. But what if I'm batching EDI files - is there a way to set the filename there as well?
Thanks!
I would suggest using a dynamic send with a file:// URI, but your method could work, too. Richard Seroter has a blog article covering both approaches here: http://blogs.msdn.com/b/richardbpi/archive/2006/07/05/656948.aspx
Related
I want to understand the difference between SAX.writer and CGI wrapper. I can't find any get started information, any suggested content OR video link it can be very appreciated thanks.
The SAX writer is a set of language statements/elements that allow the creation of well-formed XML documents. This XML is output to the location specified by the SET-OUTPUT-DESTINATION method. Output destinations include streams (which might include a Classic WebSpeed stream (WEB-STREAM).
CGI wrapper is more of an approach, with a bunch of (internal) procedures that let you create a fully-formed HTTP response (and read from an incoming HTTP request). This approach should not be used for new web services, even though it still works. In newer version of OpenEdge the PASOE server provides what are known as WebHandlers, which replace the CGI wrapper approach.
The {&OUT} 'syntax' is really just a preprocessor that does something like PUT UNFORMATTED STREAM WEB-STREAM - you can see this if you compile your programs with the PREPROCESS option, or use the equivalent command in PDSOE (a right-click option).
I am very new to grpc and I am refactoring some http handlers to grpc. In there I have found a handler which is relevant to upload a file. In the request, it is sending a file as http.FormFile using http multipart form data.
What I found is, there is a way using request chunk data stream to upload file. But what i need is avoid streaming and do it in stateless manner.
I have searched a way to solve this, but I couldn't find a way to do that. Highly appreciate if someone give me a propper solution to do this
Tl;dr
gRPC was not designed to handle large file uploads in the same way that you would using http multipart form data uploads. gRPC has a (slightly) arbitrary 4MB message limit (also see this). In my experience, the proper solution is to not use gRPC for large file uploads. That being said, there may be a few options you can try.
Changing gRPC Call Options
You can manually override the default 4MB message limit using connection options when you dial the gRPC server. For example, see this:
client, err := grpc.Dial("...",
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(4096)))
Using gRPC Streams
I have needed to use gRPC for file uploads without changing the default message limit options by implementing my own chunking package to handle it for me over a unary gRPC stream. You mentioned you wanted to avoid using a stream, however I'm providing this as a resource for those who want to avoid changing their gRPC message limit. Using the library, you'll need to wrap your gRPC client. Once a wrapper is created, you can upload anything that is compatible with the io.Reader interface.
err := chunk.UploadFrom(reader, WrapUploadFileClient(client))
We realize that if we want to produce a multipart query that contains a video file of 15GB, it is impossible to allocate in memory the size needed for such a large amount of data, most devices have only 2 or 3GB of RAM.
It is therefore absolutely necessary to switch to the uploadTask method which will push to the server the contents of a block file of the maximum size allowed by the IP packets sent to the server.
This is a POST method. However, it does not contain parameters such as the folder id or the file name. So you need a way to transmit these parameters. The best way is to code them in the URL.
I proposed an encoding format in the form of a path behind the endpoint of the API, but we can also very well encode these two parameters in a classic way in the URL, eg:
/api/upload?id=123&filename=video.mp4
From what I read on Stackoverflow, it's trivial with Symfony to retrieve id and filename. Then all the data received in the body of the POST request can be written in a raw way directly into a file, without also passing through a buffer in server-side memory.
The user data must imperatively be streamed, whether mobile side or server side, and whether upload or download. Loading user content in memory is also very dangerous in terms of security.
In symfony, how can I do that?
This goes way beyond Symfony and depends on the web server you are using.
By default with apache/nginx and php you will receive an already buffered request, so you cannot stream it to a file.
However, there are solutions, for example with Apache you can stream requests, see http://hc.apache.org/httpclient-3.x/performance.html#Request_Response_entity_streaming
Probably nginx also has options for it, but I don't know about those.
Another option might be websockets, see http://en.wikipedia.org/wiki/WebSocket
I am looking for a way on how to implement file download functionality using gRPC but I can't find in the documentation how this is done.
What is an optimal way to do this? I wanted to have a gRPC server that holds files and a gRPC client to request something from gRPC.
I have looked at the examples in Java but could not figure out a way on how to do this. I just started reading about gRPC today.
You must define a specific message as data container and you must transfer it to the client in chunks using a stream.
Example of proto Message and RPC service:
message DataChunk {
bytes data = 1;
}
rpc DownloadProductImage(DownloadProductImageRequest) returns(stream DataChunk){}
In java you can use BufferedInputStream to stream the resource to the client.
protobuf / grpc
server side code
client side code
There are a few possible approaches.
If your file size is small enough, you can define a proto file with a single String field, and dump the entire text content into that field.
If your file size is too large to do this, you might want to chunk the file and use the streaming API to send the file chunk by chunk.
Below is a good guide for getting started with gRPC.
https://grpc.io/docs/tutorials/basic/java.html
I have taken the working example from db80 and have created a PR with a full example: https://github.com/grpc/grpc-java/pull/5126
Hopefully they will merge this then everyone can have a working example of this common use-case for GRPC.
I am doing file transfers, but the filereference API doesn't support file chunking. Has anyone done this before? For example, I would like to be able to upload a 1 gig file from an AIR client to a custom PHP/Java/etc. service.
It seems that all you should have to do is use the upload() routine. The php or java service should be doing the chunking.
var myHugeFile = new air.File('myHugeLocal.file');
myHugeFile.upload(new URLRequest("http://your.website.com/uploadchunker.php"));
There is a much more elaborate example of using filereference in the adobe learning area here:
http://www.adobe.com/devnet/air/flex/articles/uploading_air_app_to_server.html
Three options jump out on this:
Use an FTP service that supports resumable transfers, assuming flash supports this as well. Maybe not an option if you are wanting to communicate with a custom service of your own.
Leverage the http file part header support. Only applicable if AIR allows access to the appropriate http headers (content-range & content-length). This is what BITS does. Probably a bit harder to implement.
Hand roll your own TCP or UDP protocol exchange. Not for the faint of heart. I'd look in the OSS space before going this route.
I think FileReference does chunk, at least that is what I have observed. Using a tool like Fiddler, you can watch it in action. If you analyze the outgoing headers of a FileReference upload, they are chunked.
If resumes are what you're after, I cannot say how you would go about that with FileReference. I have uploaded small files in generic posts, but that requires the flash/air client to load all bytes into the app. In Air that may or may not crash flash with a 1GB file (depends on your system I guess).