I am using Vertx. 4.0.3 and trying to stream a request body directly to a file. For that purpose I am using the following (Kotlin) code:
router.post("/upload").handler { ctx ->
val startTime = System.currentTimeMillis()
val response = ctx.response()
val request = ctx.request()
val fs = vertx.fileSystem()
fs.open("data.bin", OpenOptions()) { res ->
if (res.succeeded()) {
val asyncFile = res.result()
request.pipeTo(asyncFile).onComplete { writeResult ->
if(writeResult.succeeded()) {
response.end("${System.currentTimeMillis() - startTime}")
} else {
response.setStatusCode(500).end(res.cause().stackTraceToString())
}
}
} else {
response.setStatusCode(500).end(res.cause().stackTraceToString())
}
}
}
Unfortunately I am getting an exception like:
java.lang.IllegalStateException: Request has already been read
at io.vertx.core.http.impl.Http1xServerRequest.checkEnded(Http1xServerRequest.java:628)
at io.vertx.core.http.impl.Http1xServerRequest.endHandler(Http1xServerRequest.java:334)
at io.vertx.core.http.impl.Http1xServerRequest.endHandler(Http1xServerRequest.java:60)
at io.vertx.core.streams.impl.PipeImpl.<init>(PipeImpl.java:35)
at io.vertx.core.streams.ReadStream.pipeTo(ReadStream.java:119)
at io.vertx.ext.web.impl.HttpServerRequestWrapper.pipeTo(HttpServerRequestWrapper.java:410)
at fileupload.AppKt$main$2$1.handle(App.kt:60)
at fileupload.AppKt$main$2$1.handle(App.kt)
at io.vertx.core.impl.future.FutureImpl$3.onSuccess(FutureImpl.java:124)
at io.vertx.core.impl.future.FutureBase.lambda$emitSuccess$0(FutureBase.java:54)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Since I do nothing to the request I have no idea where my request is already read. Can someone please give me some insights into this? Thanks!
This happens because by the time the callback of fs.open is invoked, the request has been fully read already.
You must pause the request before opening the file and resume it after:
router.post("/upload").handler { ctx ->
val startTime = System.currentTimeMillis()
val response = ctx.response()
val request = ctx.request()
val fs = vertx.fileSystem()
// Pause
request.pause()
fs.open("data.bin", OpenOptions()) { res ->
// Resume
request.resume()
if (res.succeeded()) {
val asyncFile = res.result()
request.pipeTo(asyncFile).onComplete { writeResult ->
if(writeResult.succeeded()) {
response.end("${System.currentTimeMillis() - startTime}")
} else {
response.setStatusCode(500).end(res.cause().stackTraceToString())
}
}
} else {
response.setStatusCode(500).end(res.cause().stackTraceToString())
}
}
}
Vert.x for Kotlin provide a equivalent set of suspend functions. In your case you may want to implement the equivalent openAwait and pipeToAwait functions in order to avoid the "callback hell". Now your code might look like this:
router.post("/upload").handler { ctx ->
val startTime = System.currentTimeMillis()
val response = ctx.response()
val request = ctx.request()
val fs = vertx.fileSystem()
val asyncFile = fs.openAwait("data.bin", OpenOptions())
val result = request.pipeToAwait(asyncFile)
// code for sending http response
}
Related
I am trying to get the image_list in Uri format.
In the process of retrieving photos from firebase storage, a coroutine was used to solve the async.
private fun test() {
val storageReference =
FirebaseStorage.getInstance().reference.child("Ballad_images")
val temp_time = ArrayList<String>()
val temp_time_3 = ArrayList<String>()
val image_list = ArrayList<String>()
storageReference.listAll().addOnSuccessListener { listResult ->
GlobalScope.launch() {
val job1 = launch {
for (fileRef in listResult.items) {
fileRef.downloadUrl.addOnSuccessListener { Uri ->
image_list.add(Uri.toString())
image_list.sort()
}
}
}
val job2 = launch {
for (fileRef in listResult.items) {
temp_time.add(fileRef.name)
}//for
for (i in 0..temp_time.size - 1) {
var temp_time_2 = temp_time.get(i).split('.')//2021 05 29 -1 jpg
temp_time_3.add(temp_time_2[1])
//05
}//for
}
runBlocking {
job1.join()
println("image_list : " + image_list.size)
job2.join()
println("temp_time : " + temp_time_3)
//delay(4000L)
println("image_list : " + image_list.size)
}
}//GlobalScope
}
}
end job1.join()
After finishing job2.join()
If you give it a delay, the data will come out.
When I see it, in join1
I think this is happening because of fileRef.downloadUrl.addOnSuccessListener
Is there a way to populate the image_list from job1?
I need your help since I'm struggling, I want to make some HTTP GET requests which allow me to retrieve each JSON object one by one in order to store them in a List which I will then put in an adapter. I intend to do it by incrementing idPlayer at each loop in the URL but given how the retrofit library was designed I wonder if this is possible.
Thx :)
fun getplayer() {
var instancePlayer = InstancePlayer()
var playerApi = instancePlayer.getPlayerAPI()
var call = playerApi?.getPlayer("/api/v1/players/" + idPlayer)
if (call != null) {
call.enqueue(object : Callback<Player?> {
override fun onResponse(call: Call<Player?>, response: Response<Player?>) {
var responseBody = response.body()!!
var p1: Player = Player()
p1.id = responseBody.id
p1.first_name = responseBody.first_name
p1.last_name = responseBody.last_name
playerAdapter.setData(playerList)
recyclerView?.setAdapter(playerAdapter)
playerList.add(p1)
}
override fun onFailure(call: Call<Player?>, t: Throwable) {
TODO("Not yet implemented")
}
})
}
}
I want to implement a "simple" SSDP discovering client. Means the client should send out a
M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 1
ST: ssdp:all
and afterwards listen to "the network"(?) to get the list of IP addresses.
To test the implementation I've written a unit test which creates a "fake" MulticastServer which simply hear to the SSDP IP&Port and, when receive something, send the same message back.
The problem is that this code works on my machine (macOS) most of the time but never on our CI Server (Linux). I (macOS) receive sometimes the same assertion failed error as on the CI. But as I said - only sometimes! Not always. And I don't know why.
This is the implementation on the client side:
interface GatewayDiscoverer {
companion object {
val instance: GatewayDiscoverer = DefaultGatewayDiscoverer()
}
suspend fun discoverGateways(timeoutMillis: Int = 1000): List<String>
}
internal class DefaultGatewayDiscoverer : GatewayDiscoverer {
override suspend fun discoverGateways(timeoutMillis: Int): List<String> {
require(timeoutMillis in 1000..5000) {
"timeoutMillis should be between 1000 (inclusive) and 5000 (inclusive)!"
}
val socket = DatagramSocket()
sendSsdpPacket(socket)
val gateways = receiveSsdpPacket(socket, timeoutMillis)
return gateways
}
private fun sendSsdpPacket(socket: DatagramSocket) {
val packetToSend =
"M-SEARCH * HTTP/1.1\r\nHOST: 239.255.255.250:1900\r\nMAN: \"ssdp:discover\"\r\nMX: 1\r\nST: ssdp:all\r\n\r\n"
val packetToSendAsBytes = packetToSend.toByteArray()
val packet = DatagramPacket(
packetToSendAsBytes,
packetToSendAsBytes.size,
InetAddress.getByName("239.255.255.250"),
1900
)
socket.send(packet)
}
private fun receiveSsdpPacket(socket: DatagramSocket, timeoutInMillis: Int): List<String> {
val gatewayList = mutableListOf<String>()
while (true) {
val receivedData = ByteArray(12)
val packetToReceive = DatagramPacket(receivedData, receivedData.size)
socket.soTimeout = timeoutInMillis
try {
socket.receive(packetToReceive)
packetToReceive.address?.hostName?.let { gatewayList.add(it) }
} catch (socketTimeout: SocketTimeoutException) {
return gatewayList
}
}
}
}
And this the test (includes the MulticastServer):
class DefaultGatewayDiscovererTest {
#Test
fun `discover gateways should return a list of gateway IPs`() = with(MulticastServer()) {
start()
val list = runBlocking { GatewayDiscoverer.instance.discoverGateways(1000) }
close()
assertThat(list.size).isEqualTo(1)
assertThat(list).contains(InetAddress.getLocalHost().hostAddress)
Unit
}
}
/**
* A "MulticastServer" which will join the
* 239.255.255.250:1900 group to listen on SSDP events.
* They will report back with the same package
* it received.
*/
class MulticastServer : Thread(), Closeable {
private val group = InetAddress.getByName("239.255.255.250")
private val socket: MulticastSocket = MulticastSocket(1900)
init {
// This force to use IPv4...
var netinterface: NetworkInterface? = null
// Otherwise it will (at least on macOS) use IPv6 which leads to issues
// while joining the group...
val networkInterfaces = NetworkInterface.getNetworkInterfaces()
while (networkInterfaces.hasMoreElements()) {
val networkInterface = networkInterfaces.nextElement()
val addressesFromNetworkInterface = networkInterface.inetAddresses
while (addressesFromNetworkInterface.hasMoreElements()) {
val inetAddress = addressesFromNetworkInterface.nextElement()
if (inetAddress.isSiteLocalAddress
&& !inetAddress.isAnyLocalAddress
&& !inetAddress.isLinkLocalAddress
&& !inetAddress.isLoopbackAddress
&& !inetAddress.isMulticastAddress
) {
netinterface = NetworkInterface.getByName(networkInterface.name)
}
}
}
socket.joinGroup(InetSocketAddress("239.255.255.250", 1900), netinterface!!)
}
override fun run() {
while (true) {
val buf = ByteArray(256)
val packet = DatagramPacket(buf, buf.size)
try {
socket.receive(packet)
} catch (socketEx: SocketException) {
break
}
// Print for debugging
val message = String(packet.data, 0, packet.length)
println(message)
socket.send(packet)
}
}
override fun close() = with(socket) {
leaveGroup(group)
close()
}
}
When the test fails it fails on that line:
assertThat(list.size).isEqualTo(1)
The list is empty.
After some debugging I found out that the MulticastServer don't receive the message. Therefore the client don't get the response and add the IP address to the list.
I would expect that the MulticastServer will always work without that "flakiness". Do I something wrong with the implementation?
I am trying to create an HTTP filter that logs some info about the request, say headers, and a limited (so the memory doesn't explode) part of the request body in case of an error.
To do that I've followed the docs (https://www.playframework.com/documentation/2.6.x/ScalaHttpFilters) and came up with something like this:
class RequestErrorLogFilter #Inject()(actorSystem: ActorSystem)(implicit ec: ExecutionContext)
extends EssentialFilter {
private val logger = org.slf4j.LoggerFactory.getLogger("application.AccumulatorFlowFilter")
private implicit val logging = Logging(actorSystem.eventStream, logger.getName)
override def apply(next: EssentialAction): EssentialAction = new EssentialAction {
override def apply(request: RequestHeader): Accumulator[ByteString, Result] = {
val accumulator: Accumulator[ByteString, Result] = next(request)
val data = ArrayBuffer.empty[ByteString]
var totalSize = 0
val maxSize = 1024
val flow: Flow[ByteString, ByteString, NotUsed] = Flow[ByteString]
.map((in: ByteString) => {
val left = maxSize - totalSize
if (left > 0) {
val takeNow =
if (in.size > left) {
in.slice(0, left)
} else {
in
}
data.append(takeNow)
totalSize += takeNow.size
}
in
})
val accumulatorWithResult = accumulator.through(flow).map { result =>
// this code doesn't get executed in case of an exception in a controller
logger.info(s"The flow has completed and the result is $result")
if (result.header.status >= 400) {
val headerText = data.map(_.utf8String).mkString("")
logger.warn(s"There was an error. Request head: $headerText")
}
result
}
accumulatorWithResult
}
}
}
This works fine for client errors (like 400 - bad request), or for any error returned from a controller, but in case of an exception inside of a controller, filter's "callback" isn't executed, so there's no opportunity to log what happened.
And there's a same problem with a much simpler "AccessLogHttpFilter" which I think is a recommended solution to create an access log with play applications:
class LoggingFilter #Inject() (val mat: Materializer, implicit val ec: ExecutionContext)
extends Filter {
def apply(nextFilter: RequestHeader => Future[Result])
(requestHeader: RequestHeader): Future[Result] = {
val startTime = System.currentTimeMillis
nextFilter(requestHeader).map { result =>
val endTime = System.currentTimeMillis
val requestTime = endTime - startTime
Logger.info(s"${requestHeader.method} ${requestHeader.uri} took ${requestTime}ms and " +
s"returned ${result.header.status}")
result.withHeaders("Request-Time" -> requestTime.toString)
}
}
}
Is there a way to make play invoke http filter code even in case of exceptions?
Is there some other workaround?
Figured it out.
For an EssentialFilter to handle an error you need to add a .recover() call to an accumulator:
class RequestErrorLogFilter #Inject()(actorSystem: ActorSystem)(implicit ec: ExecutionContext)
extends EssentialFilter {
private val logger = org.slf4j.LoggerFactory.getLogger("application.AccumulatorFlowFilter")
private implicit val logging = Logging(actorSystem.eventStream, logger.getName)
override def apply(next: EssentialAction): EssentialAction = new EssentialAction {
override def apply(request: RequestHeader): Accumulator[ByteString, Result] = {
val accumulator: Accumulator[ByteString, Result] = next(request)
val data = ArrayBuffer.empty[ByteString]
var totalSize = 0
val maxSize = 1024
val flow: Flow[ByteString, ByteString, NotUsed] = Flow[ByteString]
.map((in: ByteString) => {
val left = maxSize - totalSize
if (left > 0) {
val takeNow =
if (in.size > left) {
in.slice(0, left)
} else {
in
}
data.append(takeNow)
totalSize += takeNow.size
}
in
})
val accumulatorWithResult: Accumulator[ByteString, Result] = accumulator.through(flow).map { result =>
logger.info(s"The flow has completed and the result is $result")
if (result.header.status >= 400) {
val headerText = data.map(_.utf8String).mkString("")
logger.warn(s"There was an error. Request head: $headerText")
}
result
}
accumulatorWithResult.recover {
case error =>
val headerText = data.map(_.utf8String).mkString("")
logger.warn(s"There was an error: $error. Request head: $headerText")
throw error
}
}
}
}
And for a simple Filter you need a .failed.foreach call on the result future:
class LoggingFilter #Inject() (val mat: Materializer, implicit val ec: ExecutionContext)
extends Filter {
def apply(nextFilter: RequestHeader => Future[Result])
(requestHeader: RequestHeader): Future[Result] = {
val startTime = System.currentTimeMillis
val eventualResult = nextFilter(requestHeader)
eventualResult.failed.foreach { error: Throwable =>
val endTime = System.currentTimeMillis
val requestTime = endTime - startTime
Logger.info(s"${requestHeader.method} ${requestHeader.uri} took ${requestTime}ms and " +
s"returned 500 $error")
}
eventualResult.map { result =>
val endTime = System.currentTimeMillis
val requestTime = endTime - startTime
Logger.info(s"${requestHeader.method} ${requestHeader.uri} took ${requestTime}ms and " +
s"returned ${result.header.status}")
result.withHeaders("Request-Time" -> requestTime.toString)
}
}
}
I am new to the swift. Just couldn't get the below code working on the http request. Other than printing "start...", it doesn't print anything else. All the connection methods seem not called at all? Any suggestions?
class Network {
var data: NSMutableData = NSMutableData()
func getAcctSummary() {
let urlAcctSummary = "http://www.google.com"
var url: NSURL = NSURL(string: urlAcctSummary)
var request: NSURLRequest = NSURLRequest(URL: url)
var connection: NSURLConnection = NSURLConnection(request: request, delegate: self, startImmediately: false)
connection.start()
println("started...")
}//getAcctSummary
func connection(connection: NSURLConnection!, didFailWithError error: NSError!) {
println("Failed with error:\(error.localizedDescription)")
}
func connection(didReceiveResponse: NSURLConnection!, didReceiveResponse response: NSURLResponse!) {
println("didReceiveResponse")
}
func connection(connection: NSURLConnection!, didReceiveData conData: NSData!) {
self.data.appendData(conData)
println("here 1")
}
func connectionDidFinishLoading(connection: NSURLConnection!) {
println(self.data)
println("here")
}
}
In order to determine which optional protocol methods are supported, the NSURLConnection delegate is required to be a subclass of NSObject. Change the first line to:
class Network : NSObject {
And you should be good-to-go
Try using swift NSURLSession. Worked best for me here is some ex code but had to pull out my custom code.
func someFunc() ->()
{
var session = NSURLSession.sharedSession()
var personURL = "URL"
let url = NSURL(string:personURL)
var task:NSURLSessionDataTask = session.dataTaskWithURL(url, completionHandler:apiHandler)
task.resume()
}
func apiHandler(data:NSData!, response:NSURLResponse!, error:NSError!)
{
if error
{
println("API error: \(error), \(error.userInfo)")
}
var jsonError:NSError?
var json:JMSMediaDictionary = NSJSONSerialization.JSONObjectWithData(data, options: NSJSONReadingOptions.MutableContainers, error: &jsonError) as Dictionary
println(json)
//do stuff with data
}