I have a http server with a multipart/form-data protocol interface to upload file.
There is a verification in the protocol, how can i reject the requst ahead of whole file stream uploaded.
The code like
#require_http_methods(['POST'])
#csrf_exempt
def Upload(request):
func_name = sys._getframe().f_code.co_name
request_id = str(uuid.uuid1())
logger = logging.LoggerAdapter(init_logger, {})
response_instance = Response(request_id, func_name, logger)
logger.info("====== request " + func_name + " " + request_id + " : " + str(request.POST) + " ======")
cert = request.POST.get("cert", None)
if cert is None or not check_cert():
logger.error("check cert failed")
return JsonResponse(response_instance.generator(ERROR_ERROR_DENIED, "permission denied"))
download_file = '/root/downloads/test.zip'
package = request.FILES.get("firmware_package", None)
destination = open(download_file, 'wb')
for chunk in package.chunks():
destination.write(chunk)
destination.close()
When i request server with wrong cert and big file, i have to wait a long time before the server returns "permission denied".
Related
I created an endpoint that takes a json body from a POST request, uses that data to fill in the fields of a PDF, and send the filled out file back. When I try to send a POST request using Send and Download on Postman, I initially get a 200 OK back and Postman goes into a loading state, showing how much time the download has taken so far.
After about 2 minutes of this, I get an ECONNRESET error:
Thinking this was just a problem with Postman, I updated a React project of mine to hit the endpoint. I was expecting making the request would start the browser's built in file download feature. Instead, I got a similar error: ERR_CONNECTION_RESET.
I debugged my controller and it seems to be parsing the request body correctly. It also doesn't seem to take too long to return from the endpoint's function. I'm using the File method from ControllerBase to make the response from the file stream, and I make sure not to dispose of the file stream too early.
Per Ali Vahidinasab's request, here is the Postman request exported to C#:
var client = new RestClient("https://localhost:44398/api/charactersheet/download");
client.Timeout = -1;
var request = new RestRequest(Method.POST);
request.AddHeader("Content-Type", "application/json");
request.AddHeader("Accept", "application/pdf");
var body = #"{
" + "\n" +
#" ""abilityScores"": {
" + "\n" +
#" ""strength"": 10,
" + "\n" +
#" ""dexterity"": 11,
" + "\n" +
#" ""constitution"": 12,
" + "\n" +
#" ""intelligence"": 13,
" + "\n" +
#" ""wisdom"": 14,
" + "\n" +
#" ""charisma"": 15
" + "\n" +
#" }
" + "\n" +
#"}";
request.AddParameter("application/json", body, ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
Console.WriteLine($"Status Code: {response.StatusCode}");
When I ran this client code, the response had a status code of 0, which I understand means that there is no response from the server.
I guess the crux of this question is: is this an issue with the server or the client? And what can I do to fix it?
I found the issue. It was on the server side. I had to set the position of the memory stream I was returning to 0 before returning it.
I am trying to upload JSON data + file (binary) to FastAPI 'POST' endpoint using requests.
This is the server code:
#app.post("/files/")
async def create_file(
file: bytes = File(...), fileb: UploadFile = File(...), timestamp: str = Form(...)
):
return {
"file_size": len(file),
"timestamp": timestamp,
"fileb_content_type": fileb.content_type,
}
This is the client code:
session = requests.Session()
adapter = requests.adapters.HTTPAdapter(max_retries=0)
session.mount('http://', adapter)
jpg_image = open(IMG_PATH, 'rb').read()
timestamp_str = datetime.datetime.now().isoformat()
files = {
'timestamp': (None, timestamp_str),
'file': ('image.jpg', jpg_image),
}
request = requests.Request('POST',
FILE_UPLOAD_ENDPOINT,
files=files)
prepared_request = request.prepare()
response = session.send(prepared_request)
The server fails with
"POST /files/ HTTP/1.1" 422 Unprocessable Entity
FastAPI endpoints usually respond 422 when the request body is missing a required field, or there are non-expected fields, etc.
It seems that you are missing the fileb from your request body.
If this field is optional, you must declare it as follows in the endpoint definition:
fileb: Optional[UploadFile] = File(None)
You will also need to make some checks inside your endpoint code...
If it is a required field then you need to add it to your request body.
I am sending data to grpc service and getting error message in return:
Encountered end-of-stream mid-frame
What does this mean. the connection was interrupted or something else like not enough data sent across. Was it a failure of my client to send enough data of the message over or was it some connection break in the middle of processing. I dont have enough information from this.
If you dont know whats wrong here just tell me if it means the connection closed too early during processing or the datafeed was just not as long as expected but there was no connection problem.
I am using this filter from Envoy proxy (Lyft):
I am using this bridge from Envoy:
https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/grpc_http1_bridge_filter
It asks for zero byte up front and 4 bytes with big indian of the length.
For me its a long ugly and meaningless message:
Jul 23, 2019 2:26:06 PM io.grpc.netty.shaded.io.grpc.netty.NettyServerStream$TransportState deframeFailed
WARNING: Exception processing message
io.grpc.StatusRuntimeException: INTERNAL: Encountered end-of-stream mid-frame
at io.grpc.Status.asRuntimeException(Status.java:524)
at io.grpc.internal.AbstractServerStream$TransportState.deframerClosed(AbstractServerStream.java:238)
at io.grpc.netty.shaded.io.grpc.netty.NettyServerStream$TransportState.deframerClosed(NettyServerStream.java:155)
at io.grpc.internal.MessageDeframer.close(MessageDeframer.java:229)
at io.grpc.internal.MessageDeframer.deliver(MessageDeframer.java:296)
at io.grpc.internal.MessageDeframer.request(MessageDeframer.java:161)
at io.grpc.internal.AbstractStream$TransportState.requestMessagesFromDeframer(AbstractStream.java:205)
at io.grpc.netty.shaded.io.grpc.netty.NettyServerStream$Sink$1.run(NettyServerStream.java:100)
at io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333)
at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905)
at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
14:26:06.445 [grpc-default-worker-ELG-3-2] DEBUG io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler - [id: 0xd01ed34c, L:/127.0.0.1:9009 - R:/127.0.0.1:48042] OUTBOUND RST_STREAM: streamId=45 errorCode=8```
Is there something wrong with the client?
//Define a postRequest request
HttpPost postRequest = new HttpPost("http://10.10.xx.xx:31380/com.test.EchoService/echo");
//Set the API media type in http content-type header
postRequest.addHeader("content-type", "application/grpc");
int messageLength=EchoRequest.newBuilder()
.setMessage("Hello"+ ": " + Thread.currentThread().getName())
.build().getMessageBytes().toByteArray().length;
byte[] lengthBytes = ByteBuffer.allocate(4).putInt(messageLength).array();
byte[] zeroByte = {0};
byte[] messageBytes = EchoRequest.newBuilder()
.setMessage("Hello" + ": " + Thread.currentThread().getName())
.build().getMessageBytes().toByteArray();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
baos.write(zeroByte);
baos.write(lengthBytes);
baos.write(messageBytes);
byte[] c = baos.toByteArray();
//Set the request post body
StringEntity userEntity = new StringEntity(content);
ByteArrayEntity bae = new ByteArrayEntity(baos.toByteArray());
postRequest.setEntity(userEntity);
//Send the request; It will immediately return the response in HttpResponse object if any
HttpResponse response = httpClient.execute(postRequest);
//verify the valid error code first
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode != 201)
{
throw new RuntimeException("Failed with HTTP error code : " + statusCode);
}
the error message means, the server received partial data, and it doesn't expect more data coming since the end of stream is true.
based on the error message, the length is probably larger than actual proto payload.
I am trying to learn about how a HTTP client works in Java. I am trying to build my own client that will make a request to a web server for a php file.
Currently when I make the request the server gives me the following error:
HTTP/1.1 400 Bad Request
However, I am able to access the file from within a browser no problem. I don't know what I could be doing wrong but I can't figure it out. Below is the code for my HTTP Client class:
public class MyHttpClient {
MyHttpRequest request;
String host;
public MyHttpResponse execute(MyHttpRequest request) throws IOException {
//Creating the response object
MyHttpResponse response = new MyHttpResponse();
//Get web server host and port from request.
String host = request.getHost();
int port = request.getPort();
//Check 1: HOST AND PORT NAME CORRECT!
System.out.println("host: " + host + " port: " + String.valueOf(port));
//Get resource path on web server from requests.
String path = request.getPath();
//Check 2: ENSURE PATH IS CORRECT!
System.out.println("path: " + path);
//Open connection to the web server
Socket s = new Socket(host, port);
//Get Socket input stream and wrap it in Buffered Reader so it can be read line by line.
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(s.getInputStream()));
//Get Socket output stream and wrap it in a Buffered Writer so it can be written to line by line.
PrintWriter outToServer = new PrintWriter(s.getOutputStream(),true);
//Get request method
String method = request.getMethod();
//Check 3: ENSURE REQUEST IS CORRECT GET/POST!
System.out.println("Method: " + method);
//GET REQUEST
if(method.equalsIgnoreCase("GET")){
//Send request to server
outToServer.println("GET " + path + " HTTP/1.1 " + "\r\n");
String line = inFromServer.readLine();
System.out.println("Line: " + line);
}
//Returning the response
return response;
}
}
If anyone could shed some light on this issue I'd appreciate it very much! Thanks.
New Request To Server:
outToServer.print("GET " + path+ " HTTP/1.1" + "\r\n");
outToServer.print("Host: " + host + "\r\n");
outToServer.print("\r\n");
Response:
Method: GET
line: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
line: <html><head>
line: <title>400 Bad Request</title>
line: </head><body>
line: <h1>Bad Request</h1>
line: <p>Your browser sent a request that this server could not understand.<br />
line: </p>
line: <hr>
line: <address>Apache Server at default.secureserver.net Port 80</address>
line: </body></html>
line: null
Do not use PrintWriter. You have to write ascii characters.
s.getOutputStream().write(("GET " + path + " HTTP/1.1\r\n\r\n").getBytes("ASCII"));
I think you need at least to add the Host header in the request.
Example taken from http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol
GET /index.html HTTP/1.1
Host: www.example.com
After the headers are complete you also need to transfer an extra \r\n so the server knows that the request is complete.
Do not use println but print. println adds another \n to every line causing the lines to be terminated with \r\n\n.
I've been working on a BlackBerry post request, and the request is getting sent, but the parameters don't seem to be. Here is my code:
HttpConnection httpConnection = (HttpConnection) Connector.open(url);
httpConnection.setRequestMethod(HttpConnection.POST);
httpConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
URLEncodedPostData encPostData = new URLEncodedPostData("UTF-8", false);
encPostData.append("time", "1314144000");
System.out.println("url: " + httpConnection.getURL());
byte[] postData = encPostData.toString().getBytes("UTF-8");
System.out.println("post data: " + encPostData.toString());
httpConnection.setRequestProperty("Content-length", String.valueOf(postData.length));
System.out.println("url: " + httpConnection.getURL());
System.out.println("message:" + httpConnection.getResponseMessage());
OutputStream os = httpConnection.openOutputStream();
os.write(postData);
os.flush();
os.close();
The response I get from the server(which we set up) is that we didn't send a time stamp. Is there something wrong with my
encPostData.append("time", "1314144000");
code?
Your call to getResponseMessage() before writing the post data is forcing a response before anything has been written on the connection.
System.out.println("message:" + httpConnection.getResponseMessage());
Move that to the end, after the output stream data has been written, and I think it will work better for you.
Make the http connection in read write mode.Might be that is the problem making http connection without mode link
HttpConnection connection = (HttpConnection) Connector.open("url", Connector.READ_WRITE);
See below link for making http connection.
blackberry server connection problem