Qore stream documentation (https://qoretechnologies.com/manual/qorus/latest/qore/lang/html/class_qore_1_1_output_stream.html) says that stream instance cannot be called from other thread than the object has been created (and I really see hardcoded check snippet if (tid != gettid()) then raise exception. It seems rather as huge limitation because even locking won't help.
What is supposed solution when I need use a stream object from more threads ?
I can imagine extra "stream" thread and a queue as only solution.
Iterators and streams both are limited to single-threaded use by design for performance reasons and also because no realistic use case for using streams from multiple threads could be identified with the built-in stream objects when the design and implementation of Qore streams was done.
You can implement your own streams that support multi-threaded use because the minimal internal C++ implementations of the abstract base classes, InputStream and OutputStream, do not contain any limitations on multi-threaded usage.
If you really need multi-threaded support in builtin streams, then Qore could be extended to allow for the appropriate locking to be implemented in subclasses for example.
Related
Is there a way to use atomic types in an asynchronous context instead of an asynchronous Mutex or RwLock? Can standard atomics be used as is in an asynchronous context?
Or, for example, is there an asynchronous equivalent of std::sync::atomic::AtomicUsize with load / store methods, which could replace something like tokio::sync::RwLock<usize> with read().await / write().await methods?
Yes, using Atomics in an async context is no problem.
Most of them are lock-free (=can't even block).
Even if you would block, it is still recommended to use normal, blocking synchronization primitives over async ones, unless you hold that lock during an await.
For more info I quote the respective chapter of the tokio tutorial:
On using std::sync::Mutex
Note, std::sync::Mutex and not tokio::sync::Mutex is used to guard the HashMap. A common error is to unconditionally use tokio::sync::Mutex from within async code. An async mutex is a mutex that is locked across calls to .await.
A synchronous mutex will block the current thread when waiting to acquire the lock. This, in turn, will block other tasks from processing. However, switching to tokio::sync::Mutex usually does not help as the asynchronous mutex uses a synchronous mutex internally.
As a rule of thumb, using a synchronous mutex from within asynchronous code is fine as long as contention remains low and the lock is not held across calls to .await. Additionally, consider using parking_lot::Mutex as a faster alternative to std::sync::Mutex.
Note that this is of course only true if all the threads accessing said lock/atomic are also async worker threads. If you have external threads that could block the mutex, you have to consider waiting for that mutex a blocking operation and treat it as such.
I want to understand how to work with unmanaged resources and when I need the SafeHandle class. What can be the problem statement when you can say: "Oh, here I need the SafeHandle class!"?
I would be grateful for links to articles, examples, explanations
I think MSDN is pretty clear in definition:
The SafeHandle class provides critical finalization of handle
resources, preventing handles from being reclaimed prematurely by
garbage collection and from being recycled by Windows to reference
unintended unmanaged objects. Before the .NET Framework version 2.0,
all operating system handles could only be encapsulated in the IntPtr
managed wrapper object.
The SafeHandle class contains a finalizer that ensures that the handle
is closed and is guaranteed to run, even during unexpected AppDomain
unloads when a host may not trust the consistency of the state of the
AppDomain.
For more information about the benefits of using a SafeHandle, see
Safe Handles and Critical Finalization.
This class is abstract because you cannot create a generic handle. To
implement SafeHandle, you must create a derived class. To create
SafeHandle derived classes, you must know how to create and free an
operating system handle. This process is different for different
handle types because some use CloseHandle, while others use more
specific methods such as UnmapViewOfFile or FindClose. For this
reason, you must create a derived class of SafeHandle for each
operating system handle type; such as MySafeRegistryHandle,
MySafeFileHandle, and MySpecialSafeFileHandle. Some of these derived
classes are prewritten and provided for you in the
Microsoft.Win32.SafeHandles namespace.
I have the following problem:
I'm trying to call sync closure from async function, but sync closure has to later call another async function.
I cannot make async closure, because they are unstable at the moment:
error[E0658]: async closures are unstable
so I have to do it this way somehow.
I found a couple of questions related to the problem, such as this, but when I tried to implement it, im receiving the following error:
Cannot start a runtime from within a runtime.
This happens because a function (like `block_on`)
attempted to block the current thread while the
thread is being used to drive asynchronous tasks.'
here is playground link which hopefully can show what I'm having problem with.
I'm using tokio as stated in the title.
As the error message states, Tokio doesn't allow you to have nested runtimes.
There's a section in the documentation for Tokio's Mutex which states the following (link):
[It] is ok and often preferred to use the ordinary Mutex from the standard library in asynchronous code. [...] the feature that the async mutex offers over the blocking mutex is that it is possible to keep the mutex locked across an .await point, which is rarely necessary for data.
Additionally, from Tokio's mini-Redis example:
A Tokio mutex is mostly intended to be used when locks need to be held
across .await yield points. All other cases are usually best
served by a std mutex. If the critical section does not include any
async operations but is long (CPU intensive or performing blocking
operations), then the entire operation, including waiting for the mutex,
is considered a "blocking" operation and tokio::task::spawn_blocking
should be used.
If the Mutex is the only async thing you need in your synchronous call, it's most likely better to make it a blocking Mutex. In that case, you can lock it from async code by first calling try_lock(), and, if that fails, attempting to lock it in a blocking context via spawn_blocking.
I'm working on library what is based on okhttp3 for kotlin. Internally okhttp3 uses its own Dispatcher and it have fields, such as maxRequests and maxRequestsPerHost that determines, how many requests can we send at the moment. I want to set amount of this requests equal to amount of coroutines in Dispatchers.IO in kotlin, to have logic close to coroutines.
So, here are the questions:
How can we get the number of threads in Dispatchers.IO scope from the program code? (Maybe there is some method/public constant that i didn't find)
Alternatively we can set this number manually by Dispatchers.IO documentation. Kotlin developers write, that amount of threads "defaults to the limit of 64 threads or the number of cores (whichever is larger)". Will this information be changed or this fact is immutable?
The answer is that you don't need the IO dispatcher to work with okhttp because it's an async HTTP library. The purpose of Dispatchers.IO is making blocking operations off the main UI thread. Async operations are to be performed on the Main dispatcher because they don't block the thread.
I'm writing web services in C++/CLI (not my choice) using Microsoft's Web API. A lot of functions in Web API are async, but because I'm using C++/CLI, I don't get the async/await support of C# or VB. So the fallback position is to use ContinueWith() to schedule a continuation delegate for reading the async task's result safely.
However, because C++/CLI also doesn't support inline anonymous delegates or managed lambdas, every delegate continuation must be written as a separate function somewhere. That quickly turns into spaghetti with the number of async functions in Web API.
So, to avoid the deadlock issues of Task<T>::Result, I've been trying this:
[HttpGet, Route( "get/some/dto" )]
Task< SomeDTO ^ > ^ MyActionMethod()
{
return Task::Run( gcnew Func< SomeDTO ^ >( this, &MyController::MyActionMethod2 ) );
}
SomeDTO ^ MyActionMethod2()
{
// execute code and use any task->Result calls I need without deadlocking
}
Okay, so I know this isn't great, but how bad is it? I don't yet understand enough of the guts of Web API or ASP.NET to comprehend the performance or scaling ramifications this will have.
Also, what other consequences may this have that aren't necessarily related to performance? For example, exceptions get wrapped in an extra AggregateException, which represents additional complexity and work for handling exceptions.
Your memory usage will increase with your application's parallelism. For every concurrent call to MyActionMethod you will need a separate thread with its own stack. That will cost you about 1 MB of RAM for each concurrent call. If MyActionMethod runs long enough so that 10000 instances run at once, you're looking at 10 GB of RAM. There is also CPU overhead in setting up each thread.
If concurrency is low, dropping async support won't be a problem. In that case, don't bother with Task::Run. Just change MyActionMethod to return SomeDTO^ (no Task wrapper).
Another potential concern is that lose easy use of cancellation tokens. However, for Web API it's usually fine to just let an exception propagate back to Web API, which ends up cancelling the synchronous call anyway.
Finally, if you were planning on performing any operation within your action method in parallel, you'll still need to use ContinueWith to accomplish that. Going non-async by default means you'll always perform one operation at a time. Fortunately, it's often just fine to do so.
Okay, so I know this isn't great, but how bad is it?
It's difficult to answer this without load-testing your specific scenario. But you can walk through the known semantics (taken largely from my blog).
First, when a request comes in, ASP.NET executes your handler on a thread pool thread within that request context. Your request handler calls Task.Run, which takes another thread from the thread pool and executes the actual request logic on it. The handler then returns the task returned from Task.Run; this releases the original request thread back to the thread pool.
Then, the Task.Run delegate will block on any asynchronous parts. So, this pattern has the scaling disadvantages of a regular synchronous handler, plus an extra thread context switch. Also, it uses a thread from the ASP.NET thread pool, which is not necessarily a bad thing, but in some scenarios it may throw off the ASP.NET thread pool heuristics.
Also, what other consequences may this have that aren't necessarily related to performance? For example, exceptions get wrapped in an extra AggregateException, which represents additional complexity and work for handling exceptions.
Yes, the exceptions from any .Result or Wait() calls will be wrapped in AggregateException. You may be able to avoid this by calling .GetAwaiter().GetResult() instead.
Another important consideration is that the code executing within the Task.Run is executing without a request context. So, ambient data like HttpContext.Current, current culture, thread principal, etc. are not going to be set correctly. You'll have to capture any important data before calling Task.Run and pass it down manually.