I am developing a feature that needs a variant of read/write lock that can allow concurrent multiple writers.
Standard read/write lock allows either multiple readers or single writer to run concurrently. I need a variant that can allow multiple readers or multiple writers concurrently. So, it should never allow a reader and a writer concurrently. But, its okay to allow multiple writers at the same time or multiple readers at the same time.
I hope I was clear. I couldn't find any existing algorithm so far. I can think of couple of ways to do this using some queues and etc. But, I dont want to take a risk of doing it myself unless none exists.
Do you guys know of any existing scheme?
Thanks,
The concept you are looking for is a Reentrant lock. You need to be able to try to acquire the lock and not get blocked if the lock is already taken (this is known as reentrant lock). There is a native implementation of a reentrant lock in java so I will illustrate this example in Java. (http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/ReentrantLock.html).
Because when using tryLock() you don't get blocked if the lock is not available your writer/reader can proceed. However, you only want to release the lock when you're sure that no one is reading/writing anymore, so you will need to keep the count of readers and writers. You will either need to synchronize this counter or use a native atomicInteger that allows atomic increment/decrement. For this example I used atomic integer.
Class ReadAndWrite {
private ReentrantLock readLock;
private ReentrantLock writeLock;
private AtomicInteger readers;
private AtomicInteger writers;
private File file;
public void write() {
if (!writeLock.isLocked()) {
readLock.tryLock();
writers.incrementAndGet(); // Increment the number of current writers
// ***** Write your stuff *****
writers.decrementAndGet(); // Decrement the number of current writers
if (readLock.isHeldByCurrentThread()) {
while(writers != 0); // Wait until all writers are finished to release the lock
readLock.unlock();
}
} else {
writeLock.lock();
write();
}
}
public void read() {
if (!readLock.isLocked()) {
writeLock.tryLock();
readers.incrementAndGet();
// ***** read your stuff *****
readers.decrementAndGet(); // Decrement the number of current read
if (writeLock.isHeldByCurrentThread()) {
while(readers != 0); // Wait until all writers are finished to release the lock
writeLock.unlock();
}
} else {
readLock.lock();
read();
}
}
What's happening here: First you check if your lock is locked to know if you can perform the action you're going to perform. If it's locked it means you can't read or write so you use lock to put yourself in wait state and re-call the same action when the lock is freed again.
If it's not locked, then you lock the other action (if you're going to read you lock writes and vice-versa) using tryLock. tryLock doesn't block if it's already locked, so several writers can write at the same time and several readers can read at the same time. When the number of threads doing the same thing as you reaches 0 it means that whoever held the lock in the first place can now release it. The only inconvenience with this solution is that the thread that holds the lock will have to stay alive until everyone is finished to be able to release it.
If you are using pthreads, take a look at the synchronization approach in this question.
You could use a similar approach with two variables readerCount and writerCount and a mutex.
In a reader thread you would lock the mutex and wait for writerCount == 0. If this is condition is met, you increment the readerCount by 1 and release the lock. Then you do the reading. When you are done, you lock the mutex again, decrement the readerCount, signal the condition change and release the lock.
The writer thread follows the same logic but waits for the condition readerCount == 0 and increments/decrements writerCount instead.
I did have a solution along the lines of nifs comment. I have posted my solution below. The problem is with fairness policy. Starvation can easily happen. In my approach, one kind of thread is less likely than other. So I am just getting away with giving priority to girls. Ideally we want this to be with some decent fairness policy.
/**
* RestRoomLock:
*
* This lock tries to simulate a gender based access to common rest room.
* It is okay to have multiple boys or multiple girls inside the room. But,
* we can't have boys and girls at the same time inside the room.
*
* This implementation doesn't really have proper fairness policy. For now,
* girls are being treated with priority as long as boys are being gentle,
* boyEntryBeGentle();
*
* #author bmuppana
*/
public class RestRoomLock {
int boysInside;
int girlsInside;
int girlsWaiting;
RestRoomLock() {
boysInside = girlsInside = girlsWaiting = 0;
}
public synchronized void boyEntry() {
while (girlsInside > 0) {
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
boysInside++;
}
public synchronized void boyEntryBeGentle() {
while (girlsInside + girlsWaiting > 0) {
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
boysInside++;
}
public synchronized void boyExit() {
boysInside--;
assert boysInside >= 0;
notifyAll();
}
public synchronized void girlEntry() {
girlsWaiting++;
while (boysInside > 0) {
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
girlsWaiting--;
girlsInside++;
}
public synchronized void girlExit() {
girlsInside--;
assert girlsInside >= 0;
notifyAll();
}
}
Related
I am running a blocking QLocalServer in a thread:
void QThread::stopServer()
{
m_abort = true;
m_server.close(); // QSocketNotifier: Socket notifiers cannot be enabled or disabled from another thread
}
void QThread::run()
{
m_server = new QLocalServer();
m_server->Listen("PipeName");
while (!m_abort)
{
if (m_server->waitForNewConnection())
{
// handle the connection
}
}
delete m_server;
}
How can the server be closed from another thread? Or is the only way to use non-blocking events?
Regards,
Why just not wait until run() closes or deletes the connection itself, after m_abort will be set?
void QThread::stopServer()
{
m_abort = true; // shall be thread-safe (std::atomic<bool>, etc)
wait(); // It’s optional to use it here
}
void QThread::run()
{
m_server = new QLocalServer();
m_server->Listen("PipeName");
while (!m_abort)
{
if (m_server->waitForNewConnection())
{
/* Most likely you cannot handle the connection
which was closed in another place, therefore сlose (delete)
it after leaving here */
}
}
delete m_server;
}
Please note you can use the standard QThread::requestInterruption and isInterruptionRequested() methods instead of creating own m_abort variable.
From the doc:
This function can be used to make long running tasks cleanly
interruptible. Never checking or acting on the value returned by this
function is safe, however it is advisable do so regularly in long
running functions. Take care not to call it too often, to keep the
overhead low.
So you can write:
void QThread::stopServer()
{
requestInterruption();
wait(); // It’s optional to use it here
}
void QThread::run()
{
m_server = new QLocalServer();
m_server->Listen("PipeName");
while (!isInterruptionRequested())
{
if (m_server->waitForNewConnection(100)) // 100 ms for not to call too often
{
/* Most likely you cannot handle the connection
which was closed in another place, therefore сlose (delete)
it after leaving here */
}
}
delete m_server;
}
To Set the context,
We have 4 tables in cassandra, out of those 4, one is data table remaining are search tables (Lets assumme DATA, SEARCH1, SEARCH2 and SEARCH3 are the tables).
We have an initial load requirement with upto 15k rows in one req for the DATA table and hence to the search tables to keep in sync.
We do it in batch inserts with each bacth as 4 queries (one to each table) to keep consistency.
But for every batch we need to read the data. If exists, just update only the DATA table's lastUpdatedDate column, else insert to all the 4 tables.
And below is the code snippet how we are doing:
public List<Items> loadData(List<Items> items) {
CountDownLatch latch = new CountDownLatch(items.size());
ForkJoinPool pool = new ForkJoinPool(6);
pool.submit(() -> items.parallelStream().forEach(item -> {
BatchStatement batch = prepareBatchForCreateOrUpdate(item);
batch.setConsistencyLevel(ConsistencyLevel.LOCAL_ONE);
ResultSetFuture future = getSession().executeAsync(batch);
Futures.addCallback(future, new AsyncCallBack(latch), pool);
}));
try {
latch.await();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
//TODO Consider what to do with the failed Items, Retry? or remove from the items in the return type
return items;
}
private BatchStatement prepareBatchForCreateOrUpdate(Item item) {
BatchStatement batch = new BatchStatement();
Item existingItem = getExisting(item) //synchronous read
if (null != data) {
existingItem.setLastUpdatedDateTime(new Timestamp(System.currentTimeMillis()));
batch.add(existingItem));
return batch;
}
batch.add(item);
batch.add(convertItemToSearch1(item));
batch.add(convertItemToSearch2(item));
batch.add(convertItemToSearch3(item));
return batch;
}
class AsyncCallBack implements FutureCallback<ResultSet> {
private CountDownLatch latch;
AsyncCallBack(CountDownLatch latch) {
this.latch = latch;
}
// Cooldown the latch for either success or failure so that the thread that is waiting on latch.await() will know when all the asyncs are completed.
#Override
public void onSuccess(ResultSet result) {
latch.countDown();
}
#Override
public void onFailure(Throwable t) {
LOGGER.warn("Failed async query execution, Cause:{}:{}", t.getCause(), t.getMessage());
latch.countDown();
}
}
The execution is taking about 1.5 to 2 mins for 15k items considering the network roundtrip b/w application and cassandra cluster(Both reside on same DNS but different pods on kubernetes)
we have ideas to make even the read call getExisting(item) also async, but handling of the failure cases is becoming complex.
Is there a better approach for data loads for cassandra(Considering only the Async wites through datastax enterprise java driver).
First thing - batches in Cassandra are other things than in the relational DBs. And by using them you're putting more load on the cluster.
Regarding the making everything async, I thought about following possibility:
make query to the DB, obtain a Future and add listener to it - that will be executed when query is finished (override the onSuccess);
from that method, you can schedule the execution of the next actions based on the result that is obtained from Cassandra.
One thing that you need to make sure to check, is that you don't issue too much simultaneous requests at the same time. In the version 3 of the protocol, you can have up to 32k in-flight requests per connection, but in your case you may issue up to 60k (4x15k) requests. I'm using following wrapper around Session class to limit the number of in-flight requests.
My application stores two related bits of data in application state. Each time I read these two values, I may (depending on their values) need to update both of them.
So to prevent updating them while another thread is in the middle of reading them, I'm locking application state.
But the documentation for HttpApplicationState.Lock Method really doesn't tell me exactly what it does.
For example:
How does it lock? Does it block any other thread from writing the data?
Does it also block read access? If not, then this exercise is pointless because the two values could be updated after another thread has read the first value but before it has read the second.
In addition to preventing multiple threads from writing the data at the same time, it is helpful to also prevent a thread from reading while another thread is writing; otherwise, the first thread could think it needs to refresh the data when it's not necessary. I want to limit the number of times I perform the refresh.
Looking at the code is locking only the write, not the read.
public void Lock()
{
this._lock.AcquireWrite();
}
public void UnLock()
{
this._lock.ReleaseWrite();
}
public object this[string name]
{
get
{
return this.Get(name);
}
set
{
// here is the effect on the lock
this.Set(name, value);
}
}
public void Set(string name, object value)
{
this._lock.AcquireWrite();
try
{
base.BaseSet(name, value);
}
finally
{
this._lock.ReleaseWrite();
}
}
public object Get(string name)
{
object obj2 = null;
this._lock.AcquireRead();
try
{
obj2 = base.BaseGet(name);
}
finally
{
this._lock.ReleaseRead();
}
return obj2;
}
The write and the read is thread safe, meaning have all ready the lock mechanism. So if you going on a loop that you read data, you can lock it outside to prevent other break the list.
Its also good to read this answer: Using static variables instead of Application state in ASP.NET
Its better to avoid use the Application to store data, and direct use a static member with your lock mechanism, because first of all MS suggest it, and second because the read/write to application static data is call the locking on every access of the data.
I want to use Rollback() or commit() functions after multiple process.
There is no error, but it does not commit() to update DB.
Here is my example code,
public void startTransaction(){
using(Ads_A_Connection = new AdsConnection(Ads_A_connection_string))
using(Ads_B_Connection = new AdsConnection(Ads_B_connection_string))
{
Ads_A_Connection.Open();
Ads_B_Connection.Open();
AdsTransaction aTxn = Ads_A_Connection.BeginTransaction();
AdsTransaction bTxn = Ads_B_Connection.BeginTransaction();
try{
string aResult = this.process1(Ads_A_Connection);
this.process2(Ads_B_Connection, aResult);
this.process3(Ads_A_Connection. Ads_B_Connection);
aTxn.Commit();
bTxn.Commit();
// there is no error, but it couldn't commit.
}catch(Exception e){
aTxn.Rollback();
bTxn.Rollback();
}
}
}
public string process1(conn){
// Insert data
return result;
}
public void process2(conn. aResult){
// update
}
public void process3(aConn, bConn){
// delete
// update
}
I guess, its because out of using scope. because I tried to put all the code into
startTransaction() method, then it works. but it look too dirty.
How can I use rollback() or commit() after multiple (METHOD) process?
anybody know, please advice me.
Thanks!
[EDIT]
I just add TransactionScope before connection,
using (TransactionScope scope = new TransactionScope())
{
using(Ads_A_Connection = new AdsConnection(Ads_A_connection_string))
using(Ads_B_Connection = new AdsConnection(Ads_B_connection_string))
{
.
.
but it makes an error, it say "Error 5047: The transaction command was not in valid sequence."
I need a little more hint please :)
To extend what Etch mentioned, their are several issues with manually managing transactions on your connections:
You need to pass the SQL connection around your methods
Need to manually remember to commit or rollback when you are finished
If you have more than one connection to manage under a transaction, you should really use DTC or XA to enroll the transactions into a Distributed / 2 phase transaction.
TransactionScopes are supported with the Advantage Database Server, although you will need to enable the MSDTC service and possibly also enable XA compliance.
Note that I'm assuming that the advantage .NET client has some sort of connection pooling mechanism - this makes the cost of obtaining connections very lightweight.
Ultimately, this means that your code can be refactored to something like the following, which is easier to maintain:
private void Method1()
{
using(Ads_A_Connection = new AdsConnection(Ads_A_connection_string))
{
Ads_A_Connection.Open();
string aResult = this.process1(Ads_A_Connection);
} // Can logically 'close' the connection here, although it is actually now held by the transaction manager
}
private void Method2()
{
using(Ads_B_Connection = new AdsConnection(Ads_B_connection_string))
{
Ads_B_Connection.Open();
this.process2(Ads_B_Connection, aResult);
} // Can logically 'close' the connection here, although it is actually now held by the transaction manager
}
public void MyServiceWhichNeedToBeTransactional(){
using(TransactionScope ts = new TransactionScope()) { // NB : Watch isolation here. Recommend change to READ_COMMITTED
try{
Method1();
Method2();
ts.Complete();
}
catch(Exception e){
// Do Logging etc. No need to rollback, as this is done by default if Complete() not called
}
}
}
TransactionScope is your friend!
TransactionScope
for synchronizing access to my NHibernate session at web environment I try use Mutex:
public class FactoryRepository
{
private FactoryRepository() { }
private static Mutex _sessionMutex = new Mutex();
private static ISessionFactory factory;
public static ISessionFactory SessionFactory
{
get
{
factory = factory ?? new Configuration().Configure().BuildSessionFactory();
return factory;
}
}
public static ISession Session
{
get
{
ISession currentSession;
_sessionMutex.WaitOne();
if (HttpContext.Current != null)
{
HttpContext context = HttpContext.Current;
currentSession = context.Items[SessionKey] as ISession;
if (currentSession == null || !currentSession.IsOpen)
{
currentSession = SessionFactory.OpenSession();
context.Items[SessionKey] = currentSession;
}
}
_sessionMutex.ReleaseMutex();
return currentSession;
}
}
}
At my error logging I get:
System.Threading.AbandonedMutexException: The wait completed due to an abandoned mutex.
Method: Boolean WaitOne(Int64, Boolean)
Stack Trace:
at System.Threading.WaitHandle.WaitOne(Int64 timeout, Boolean exitContext)
at System.Threading.WaitHandle.WaitOne(Int32 millisecondsTimeout, Boolean exitContext)
at System.Threading.WaitHandle.WaitOne()
Why do I get this exception with calling ReleaseMutex();
Your issue is on this line
_sessionMutex.WaitOne();
The WaitOne() can throw this exceptions because some other thread that lock it, exit with out releasing it
In your case the WaitOne throw this exception because of an abandon of the same mutex in an other thread.
I suggest to warp your mutex in a class and use a code like:
try
{
cLock = _sessionMutex.WaitOne();
// call your work
}
catch (AbandonedMutexException)
{
cLock = true;
// call your work
}
catch (Exception x)
{
//Error
}
finally
{
_sessionMutex.ReleaseMutex();
}
In the above code the ReleaseMutex may fail to run if a user stop/abandon the page and the thread is lost/delete it. And thats why you get this exception.
Be ware that mutex can lock for ever the way you do it ! :) Its better to add a millisecond limit on the wait, and / or handle the case of non lock to return read only data. You users can lock for long time if the mutex fails to pass the WaitOne()
Also be ware, the Mutex need to be close and dispose. ! Even if this is like the example in MSDN, in MSDN is only an simple Example, you need to be sure that you close and dispose your Mutex or else you see more problems when you update your page. For example if the Mutex stay on memory locked wile you update your page, then your page may lock for long time, until the Garbage collection kill it, if they do.
Unless you're using a very old version of NHibernate, I think this is probably overkill. NHibernate already has the ability to give you contextual session management in a web environment, and I think it will manage things more reliably for you.
Take a look at section 2.3 of this: NHibernate Chapter 2 - Architecture
Windows O/S has long had a bug where locking one mutex inside another could lock both if they are not unlocked properly in the correct reverse sequence.
Basically the race condition and locking could be due to NHibernate using a mutex to lock the resource as well as the mutex you are using.