I'm reading F# for fun and profit - Asynchronous programming. Under Cancelling workflows they have the following example:
let testLoop = async {
for i in [1..100] do
// do something
printf "%i before.." i
// sleep a bit
do! Async.Sleep 10
printfn "..after"
}
open System
open System.Threading
// create a cancellation source
let cancellationSource = new CancellationTokenSource()
// start the task, but this time pass in a cancellation token
Async.Start (testLoop,cancellationSource.Token)
// wait a bit
Thread.Sleep(200)
// cancel after 200ms
cancellationSource.Cancel()
About this they say:
In F#, any nested async call will check the cancellation token automatically!
In this case it was the line:
do! Async.Sleep(10)
As you can see from the output, this line is where the cancellation happened.
However, for me (VS2010, F# 2.0, F# Interactive) I get the following output. Notice how it also prints ..after after I have canceled the token. Are they simply wrong?
1 before....after
2 before....after
3 before....after
4 before....after
5 before....after
6 before....after
7 before....after
8 before....after
9 before....after
10 before....after
11 before....after
12 before....after
13 before..
val cancellationSource : CancellationTokenSource
>
..after
So perhaps the check for cancellation is done when entering Async.Sleep? No, then it would've printed:
13 before....after
14 before..
val cancellationSource : CancellationTokenSource
>
So it seems like the check is actually in the for-loop! I.e. it keeps running until the for loop after being cancelled. Is this how it works? What then if I'd rather want it to check after the sleep?
This question seems to hint at cancellation working like I described above: Can I explicitly check for cancellation / terminate async computation?
Edit:
Regarding if this is only in FSI 2.0: What happens with the following loop, if one sleep 200ms, 2500ms and 4000ms respectively? Does it print middle?
let testLoop = async {
for i in [1..5] do
printf "%i before.." i
do! Async.Sleep 2000
printfn "..middle.."
do! Async.Sleep 1000
printfn "..after"
}
I see the same results as you with F# 2.0 only in interactive Fsi. If I put the same code in a file and run fsi cancel.fsx, then the output does not have the final after and is what you expect.
Fsi v11 and v12 show the expected output for both ways of running the code.
This suggests there is some bug or difference when running Fsi v2.0 interactively that was fixed in a later version of FSharp.
Related
I want to make a VDP scheduler job in Denodo 8 wait for a certain amount of time. The wait function in the job creation process is not working as expected so I figured I'd write it into the VQL. However when i try the suggested function from the documentation (https://community.denodo.com/docs/html/browse/8.0/en/vdp/vql/stored_procedures/predefined_stored_procedures/wait) the Denodo 8 VQL shell doesn't recognize the function.
--Not working
SELECT WAIT('10000');
Returns the following error:
Function 'wait' with arity 1 not found
--Not working
WAIT('10000');
Returns the following error:
Error parsing command 'WAIT('10000')'
Any suggestions would be much appreciated.
There are two ways of invoking WAIT:
Option #1
-- Wait for one minute
CALL WAIT(60000);
Option #2:
-- Wait for ten seconds
SELECT timeinmillis
FROM WAIT()
WHERE timeinmillis = 10000;
I have a solid that needs to run after 2 solids. One will return a value, another doesn't return anything but has dependency solids and will take time to run.
I execute the pipeline in multiprocessing mode, where solids run at the same time if they don't have dependencies defined.
Below is the sample situation I am looking for. Say I have below solids.
#solid(input_defs=[InputDefinition("start", Nothing)])
def solid_a(context):
import time
time.sleep(2)
context.log.info('yey')
#solid
def solid_b(context):
return 1
#composite_solid
def my_composite_solid(wait_solid_a: Nothing, solid_b_output: int):
some_other_solid(solid_b_output)
And when executed, these solids will be running in the below timeline.
Time Passed
solid
0
pipeline starts...
1 sec
solid_b started
3 sec
solid_a dependency solids are running. solid_a did not started yet.
5 sec
solid_b finished
10 sec
solid_a started now
15 sec
solid_a finished
20 sec
my_composite_solid should start now.
So, according to this timeline, in order for my_composite_solid to start, I need both solid_a and solid_b to finish executing. However, when I make this, dagster throws an error saying:
dagster.core.errors.DagsterInvalidDefinitionError: #composite_solid 'my_composite_solid' has unmapped input 'wait_solid_a'. Remove it or pass it to the appropriate solid invocation.
If I don't put the solid_a output as a dependency to my_composite_solid, it will start immediately after the result of solid_b. What should I do?
In Jupyter Notebook (python 3.6.1) I went to run the basic python docs Hello_World in (18.5.3.1.1. Example: Hello World coroutine) and noticed that it was giving me a RuntimeError. After trying a long time to find the problem with the program(my understanding is that the docs may not be totally up to date), I finally noticed that it only does this after the second run and tested in a restarted Kernel. I've since then copied the same small python program in two successive cells(In 1 and 2) and found that it gives the error on the second not the first and gives the error to both there after. This repeats this after restarting the Kernel.
import asyncio
def hello_world(loop):
print('Hello World')
loop.stop()
loop = asyncio.get_event_loop()
# Schedule a call to hello_world()
loop.call_soon(hello_world, loop)
# Blocking call interrupted by loop.stop()
loop.run_forever()
loop.close()
The traceback:
RuntimeError Traceback (most recent call last)
<ipython-input-2-0930271bd896> in <module>()
6 loop = asyncio.get_event_loop()
7 # Blocking call which returns when the hello_world() coroutine
----> 8 loop.run_until_complete(hello_world())
9 loop.close()
/home/pontiac/anaconda3/lib/python3.6/asyncio/base_events.py in run_until_complete(self, future)
441 Return the Future's result, or raise its exception.
442 """
--> 443 self._check_closed()
444
445 new_task = not futures.isfuture(future)
/home/pontiac/anaconda3/lib/python3.6/asyncio/base_events.py in _check_closed(self)
355 def _check_closed(self):
356 if self._closed:
--> 357 raise RuntimeError('Event loop is closed')
358
359 def _asyncgen_finalizer_hook(self, agen):
RuntimeError: Event loop is closed
I don't get this error when running a file in the interpreter with all the Debug settings set. I am running this Notebook in my recently reinstalled Anaconda set up which only has the 3.6.1 python version installed.
the issue is that loop.close() makes the loop unavailable for future use. That is, you can never use a loop again after calling close. The loop stays around as an object, but almost all methods on th eloop will raise an exception once the loop is closed. However, asyncio.get_event_loop() returns the same loop if you call it more than once. You often want this, so that multiple parts of an application get the same event loop.
However if you plan on closing a loop, you are better off calling asyncio.new_event_loop rather than asyncio.get_event_loop. That will give you a fresh event loop. If you call new_event_loop rather than get_event_loop, you're responsible for making sure that the right loop gets used in all parts of the application that run in this thread. If you want to be able to run multiple times to test you could do something like:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
After that, you'll find that asyncio.get_event_loop returns the same thing as loop. So if you do that near the top of your program, you will have a new fresh event loop each run of the code.
I'm new to stackoverflow so forgive me if I do something wrong. I trying to understand how a simple server would work in Haskell. I think I'm missing something very simple or fundamental about how hGetContents works.
import Network
import System.IO
main = withSocketsDo $ do
socket <- listenOn $ PortNumber 5002
(h, _, _) <- accept socket
c <- hGetContents h
-- putStrLn c -- doesn't work
-- putStrLn $ head $ lines c -- works!
-- putStrLn $ unlines $ take 2 $ lines c -- works!
-- putStrLn $ unlines $ take 3 $ lines c -- works!
-- putStrLn $ unlines $ take 6 $ lines c -- works!
putStrLn $ unlines $ take 10 $ lines c -- doesn't work
hPutStr h $ "HTTP/1.0 200 OK\r\nContent-Length: 5\r\n\r\nHello!\r\n"
hClose h
After running the program, I navigate via web browser to http://localhost:5002. The problem seems to be that, depending on how much I've parsed the handle contents, I eventually am unable to send a response. I'd like to be able to parse the request before I send a response. I've commented in the code the cases that work and the cases that don't. Hoogle says that for hGetContents (lazy) the handle is "semi-closed" as it is being read. Am I misunderstanding the laziness or should I consider the handle closed once I begin parsing its contents?
The error I get is "hPutChar: resource vanished (Broken pipe)." Thanks for any help.
I tried to reproduce your problem. For that I executed your code and send it a request using nc:
printf "1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11" | nc localhost 5002
As expected the server (code from your question) printed out first 10 lines and exited without any error. The client (nc) printed:
HTTP/1.0 200 OK
Content-Length: 5
Hello!
and also exited without an error.
So, at first I couldn't understand what's your problem, but then I tried to send a smaller request:
printf "1\n2\n3\n4\n5\n6\n" | nc localhost 5002
The server printed first 6 lines and didn't exit. The client also didn't exit, so I interrupted it with Ctrl-C and after that the server exited with "resource vanished" error.
I took some thinking and it started making sense to me. I don't understand lazy IO too good, so if my explanation isn't clear or correct it would be helpful if someone with better understanding would improve it.
Let's follow your code. First:
(h, _, _) <- accept socket
c <- hGetContents h
You open a handle and read it's content. Note that the handle is lazy and the content that you get is also lazy. When we say that something is lazy we mean that it can be passed around without being evaluated (it's often referred as 'call by name' vs 'call by value').
Now:
putStrLn $ unlines $ take 10 $ lines c
Here it is, you pass your lazy, unevaluated content to another function take 10. take 10 will try to evaluate first 10 elements of a list and return them, if there are less than 10 elements in the list it would simply return all of them. After take 10 we have putStrLn and unlines which both perfectly compatible with laziness.
Now let's say that client sends an input that is only 6 lines long and then starts waiting for the respond. Our server lazily receives the content and tries to print first 10 lines. First, take 10 function happily consumes the first 6 lines and passes them over to putStrLn . unlines, what happens then? take 10 can't just finish it's output because there is absolutely no indication that it is the end. The handle is still open, bytes still can be floating from client to server, so it just waits for more input.
This behaviour can be observed by running:
nc localhost 5002
and manually typing there 10 lines. The input would appear on server line-by-line as you type. After you will type the 10th line the server will respond with "Hello" message.
P.S: I guess that the behaviour that you described happens because you web browser sends 6 to 9 lines of something with the request.
To test, debug and analyze this kind of low level servers you should use simple tools like nc and curl instead of your web browser :)
When you initiate a lazy read on a handle, you give up the right to do anything much else with the handle until the contents string is fully forced, or you close the handle manually (at which point attempting to force any more of the contents string will lead to bad behavior or an error).
TL;DR
This is not a situation where lazy I/O is appropriate. The situations where a lazy read on a socket is appropriate can probably be counted on zero fingers. You can use regular strict I/O if you like, or conduit, or pipes, or some Haskell web framework like Yesod or Scotty or various other competitors.
Calling hGetContents puts the handle into a "semi-closed" state. You should not perform any operations on the handle after that point. You should only use the string returned from hGetContents.
Put simply, don't use lazy I/O here. You need to manually read and write individual strings one at a time, since the timing matters.
In general, lazy I/O is kind of neat, but it doesn't work well for anything much beyond toy examples.
I'm trying to use libxively to update my feed, but it frequently seems to do nothing. I've got a basic call:
{
xi_datastream_t& ds = mXIFeed.datastreams[2];
::xi_str_copy_untiln(ds.datastream_id, sizeof (ds.datastream_id), "cc-output-power", '\0');
xi_datapoint_t& dp = ds.datapoints[0];
ds.datapoint_count = 1;
::xi_set_value_f32(&dp, mChargeController->outputPower());
}
const xi_context_t* ctx = ::xi_nob_feed_update(mXIContext, &mXIFeed);
it logs the following:
[io/posix/posix_io_layer.c:182 (posix_io_layer_init)] [posix_io_layer_init]
[io/posix/posix_io_layer.c:191 (posix_io_layer_init)] Creating socket...
[io/posix/posix_io_layer.c:202 (posix_io_layer_init)] Socket creation [ok]
Once or twice I saw my Xively developer page show a GET feed, but otherwise, nothing seems to get written. Any suggestions on what I should look at?
I tried to rebuild the library using blocking calls (would be nice if nob didn't mean no blocking calls), but I couldn't figure out how to build it.
Thanks!
EDIT:
I was able to build a synchronous version of the library, and that seems to work. Can anyone verify that the async version works? Is there more to it than simply calling xi_nob_feed_update()?
EDIT 2:
I tried running the async example, but I'm doing something wrong, as it always complains of no data received:
$ bin/asynch_feed_update <my key> <my feed ID> example 1 example 4 example 20 example 58 example 11 example 17
example: 1 7
example: 4 7
example: 20 7
example: 58 7
example: 11 7
example: 17 7
[io/posix_asynch/posix_asynch_io_layer.c:165 (posix_asynch_io_layer_init)] [posix_io_layer_init]
[io/posix_asynch/posix_asynch_io_layer.c:174 (posix_asynch_io_layer_init)] Creating socket...
[io/posix_asynch/posix_asynch_io_layer.c:185 (posix_asynch_io_layer_init)] Setting socket non blocking behaviour...
[io/posix_asynch/posix_asynch_io_layer.c:203 (posix_asynch_io_layer_init)] Socket creation [ok]
No data within five seconds.
The asynchronous version should work. The xi_nob_feed_update() is the right function to make a feed update request.
You have to call process_xively_nob_step() in a loop just after select().
In general, you should follow the asynchronous example.