What are the commands that I have to use to STOP or START an Oracle AQ Queue?
I need to STOP the queue in order to perform maintenance and analysis, and later START the queue once the analysis is complete.
You use the package DBMS_AQADM.
To STOP a queue the command looks like this:
BEGIN
DBMS_AQADM.STOP_QUEUE(queue_name => 'QUEUE_NAME');
END;
The optional paremeter "wait" for this procedure has the following behavior:
Specifies whether to wait for the completion of outstanding
transactions. TRUE means wait if there are any outstanding
transactions. In this state no new transactions are allowed to enqueue
to or dequeue from this queue. FALSE means return immediately either
with a success or an error.
And to START a queue it looks like this:
BEGIN
DBMS_AQADM.START_QUEUE(queue_name => 'QUEUE_NAME');
END;
You may have to add "SYS" as prefix for the package, depending on your permissions.
Often I find that I want to stop only dequeue. Then I have to use the extra options on START_QUEUE() like so
execute DBMS_AQADM.STOP_QUEUE (queue_name => 'schema.queue', enqueue => FALSE, dequeue => TRUE);
Related
My current design looks like this
task body Agent_Task is
begin
loop
select
accept Request_A do
end Request_A;
or
accept Request_B do
end Request_B;
or
...
else
Do_Other_Stuff;
end select;
end loop;
exception
when Exception_Id : others => Show_Exception (Exception_Id);
end Agent_Task;
But when calling an entry in an agent frequently (e.g. Request_A) it will become non-responsive.
Is there a better structure to make the agent never be blocked? like interrupts?
The most time consuming part is in Do_Other_Stuff, what I want the server to do is roughly:
loop
Do_Other_Stuff;
if interrupted, handle other requests.
end loop;
This a school assn which I can not modify the test program to retry a failed request to an agent. When the test is runing, mutilple agents will talk to each other concurrently.
Note that you have a spurious end; in your code that makes it illegal.
If what's important is that those making requests not block, you could use a protected queue for the inter-task communication:
loop
select
Request_Queue.Get (Item => Request);
Process (Request => Request);
else
Do_Other_Stuff;
end select;
end loop;
Requests will not be processed until Do_Other_Stuff completes, but those making requests will not be blocked by putting a request on the queue.
Using a queue would also allow you to use asynchronous transfer of control to give requests priority over Do_Other_Stuff:
loop
select
Request_Queue.Get (Item => Request);
Process (Request => Request);
then abort
Do_Other_Stuff;
end select;
end loop;
Do_Other_Stuff would then need to be abortable, and able to pick up from where it left off the next time it runs. But it would be better if Do_Other_Stuff could be moved into another task as Holsti suggested.
Finally, if you can't move Do_Other_Stuff to another task, and you can't abort it, you may need to break it into shorter parts:
loop
Do_Some_Stuff;
Handle_Requests;
Do_Some_More_Stuff;
Handle_Requests;
..
end loop;
Again, this is easier with a queue, since accept statements can't go in a subprogram.
Move the Do_Other_Stuff to another task, that is, divide your Agent_Task into two tasks.
How difficult that is depends on how much communication (data flow) there is between Do_Other_Stuff and the actions for Request_A and Request_B. If Do_Other_Stuff is moved to its own task, that task has to communicate with the original Agent_Task is some way, by rendez-vous or protected objects. If Do_Other_Stuff is a long computation that has some inputs and some outputs, you might add two entries to Agent_Task, one to provide the inputs to the Other_Stuff task, and another to receive the outputs from the Other_Stuff task.
I have a requirement where I iterate through 10,000,000 documents and for each document I do some operation and store some values in '/count.xml'. When I iterate to second document I update '/count.xml' with updated value
Currently this is what I am doing, here $total-records is 10,000,000
let $total-records := xdmp:estimate(cts:search( //some code))
let $batch-size := 5000
let $pagination := 0
let $bs :=
for $records in 1 to fn:ceiling($total-records div $batch-size )
let $start := fn:sum($pagination + 1)
let $end := fn:sum($batch-size + $pagination)
let $_ := xdmp:set($pagination, $end)
return
xdmp:spawn-function
(
function() {
for $each in cts:search( //some code)[$start to $end]
return //some operation and update '/count.xml' with some updated values
},
<options xmlns="xdmp:eval"><commit>auto</commit><update>true</update</options>
)
let $doc := doc("/count.xml")
return ()
So here the issue is I need to read the '/count.xml' file after all documents are iterated, But with above code using spawn task
let $doc := doc("/count.xml")
will not be latest one as above spawn task will run on different threads.
I need a solution where
let $doc := doc("/count.xml")
waits till all spawn task are completed.
I have came across
<result>{fn:true()}</result>
option as well, but I do not know whether it will work or not because variable
$bs
not being used anywhere and documentation says 'When the calling request uses the value future in any operation, it will automatically wait for the spawned task to complete and it will use the result.'
Is there any other alternative where
let $doc := doc("/count.xml")
line will be executed only after all spawn task are completed
To process 10 mln documents, you probably need to spawn something like 10.000 batches of a 1000 docs. I don't think that will work well from within MarkLogic.
I'd advice looking into the built-in aggregation features of MarkLogic. See for instance cts:sum-aggregate. You might be able to pre-calculate per-document intermediate results, that you could aggregate at run-time using those aggregation features. That would definitely be most performant, and would scale best.
Alternative would be to orchestrate your calculations from outside of MarkLogic. Otherwise you end up either flooding the task queue, or running into timeout limits, or both. Tools like Corb2 and DMSDK could be of help with this.
Note: you can indeed make spawns wait for result by using the <result> option, but either use <result>true</result> or <result>{fn:true()}</result> (note the parentheses behind fn:true, it is a function).
HTH!
The requirement as given, one cannot tell the difference between writing once the final result of a query across 10,mil docs vs writing the result after query of 1 document at a time. Since your example does no writes to the queried documents it need not be spawned nor run in a seperate thread or transaction, rather as HTH says, you can aka use of aggregate functions to do a single query over the entire set, compute the final result and store it in 1 operation. Likely this will run very quickly (or can be made to).
If the requirements are actually that each single document MUST be queried, then sequentially another shared document written to -- this can only be observed by using seperate transactions, serially. Its going to be horrendously slow, almost certainly longer then the timeout for the calling transaction. This means you must orchestrate it from outside -- if the requirement is that the same caller start the process as finish it (a highly implementation specific requirement that if true is likely to have other implications beyond those given).
Something close thats achievable but still horrendously slow is to have an outside query poll on the updated shared document and return 'success' once the job is done.
But again, with this many documents, if your forcing a write transaction for each one, its going to take longer (or atleast is not easily guaranteed to NOT take longer) then the a single transaction timeout so must be invoked from 'outside'.
This is where I would recommend revisiting the requirements to determine the core functionality/result that is desired and if it is truly required to implement exactly as stated vs a more performant implementation that achieves the desired result.
If the core functionality needed is that every single query be 'checkpointed' with a document update, then there are other implications such as transaction rollback that need to be considered.
I'm new to Ada and I'm trying to write a simple program but I have some issues I couldn't solve. So I have a task with an execute entry and a signalfound entry, the execute entry is being called first and has to do some calculations until the signalfound entry is being called by the main and then it has to stop.
But the problem is that when Signalfound entry is called it doesn't get executed because the task is stuck in the while loop. Is there an obvious solution to this problem in Ada. I tried googling the problem but without succes. Thanks in advance!
task Example body
ResultFound : Boolean := False;
--- other unimportant things
begin
loop
select
accept Execute do
while (ResultFound = False) loop
---do some calculations
end loop;
end Execute;
or
accept SignalFound do
ResultFound := True;
end SignalFound;
or
-- other unimportant accepts
end select;
end loop;
end Example;
Well, when Execute is called, your task enters the loop, which means it never executes the select statement again, so it can never accept the SignalFound call. Also, the entity calling Execute will never continue since your while loop repeats forever inside the accept statement. Usually, you want to make your critical regions for task synchronisation as small as possible, so that both tasks can carry on with their work after necessary data has been exchanged.
Mind that your code reflects a protocol by which your task operates. Currently, your code says „I will accept both Execute and SignalFound at any loop iteration“, which does not seem to match with the protocol you have in mind based on what you write.
I believe what you actually want to do is something like this:
task Example body
begin
loop
-- simply wait for someone to tell me to execute.
accept Execute;
Calculations : loop
-- check at every loop iteration whether there's a SignalFound waiting
select
accept SignalFound;
exit Calculations;
else
-- do some calculations
end select;
end loop Calculations;
-- will go to next iteration and wait for someone to call Execute again.
end loop;
end Example;
This code enforces a sequence of alternating Execute / SignalFound calls which seems to be what you have in mind.
I've spent a fair amount of time looking into the Realm database mechanics and I can't figure out if Realm is using row level read locks under the hood for data selected during write transactions.
As a basic example, imagine the following "queue" logic
assume the queue has an arbitrary number of jobs (we'll say 5 jobs)
async getNextJob() {
let nextJob = null;
this.realm.write(() => {
let jobs = this.realm.objects('Job')
.filtered('active == FALSE')
.sorted([['priority', true], ['created', false]]);
if (jobs.length) {
nextJob = jobs[0];
nextJob.active = true;
}
});
return nextJob;
}
If I call getNextJob() 2 times concurrently, if row level read blocking isn't occurring, there's a chance that nextJob will return the same job object when we query for jobs.
Furthermore, if I have outside logic that relies on up-to-date data in read logic (ie job.active == false when it actually is true at current time) I need the read to block until update transactions complete. MVCC reads getting stale data do not work in this situation.
If read locks are being set in write transactions, I could make sure I'm always reading the latest data like so
let active = null;
this.realm.write(() => {
const job = this.realm.pseudoQueryToGetJobByPrimaryKey();
active = job.active;
});
// Assuming the above write transaction blocked the read until
// any concurrent updates touching the same job committed
// the value for active can be trusted at this point in time.
if (active === false) {
// code to start job here
}
So basically, TL;DR does Realm support SELECT FOR UPDATE?
Postgresql
https://www.postgresql.org/docs/9.1/static/explicit-locking.html
MySql
https://dev.mysql.com/doc/refman/5.7/en/innodb-locking-reads.html
So basically, TL;DR does Realm support SELECT FOR UPDATE?
Well if I understand the question correctly, the answer is slightly trickier than that.
If there is no Realm Object Server involved, then realm.write(() => disallows any other writes at the same time, and updates the Realm to its latest version when the transaction is opened.
If there is Realm Object Server involved, then I think this still stands locally, but the Realm Sync manages the updates from remote, in which case the conflict resolution rules apply for remote data changes.
Realm does not allow concurrent writes. There is at most one ongoing
write transaction at any point in time.
If the async getNextJob() function is called twice concurrently, one of
the invocations will block on realm.write().
SELECT FOR UPDATE then works trivially, since there are no concurrent updates.
How do I create PL/SQL function which waits for update on some row for specified timeout and then returns.
What I want to accomplish is - I have long running process which will update it's status to ASYNC_PROCESS table by process_id. I need function which returns with true/false when this process has completed, but also I need this function to wait some time for this process complete, return on timeout or return imediately with true, when process has completed. I don't want to use sleep(1 sec), because in such case I will be having 1 sec lag. I don't want to use sleep(1 msec), because in such case I am spending cpu resources (and 1msec lag).
Is there a good way how experienced programmer would accomplish this?
That function will be called from .NET (So I need minimal lag between DB operation and .NET/UI)
THNX,
Beef
I think the most sensible thing to do in this case is to use update triggers on that ASYNC_PROCESS table.
You should also look into the DBMS_ALERT package. Here's an edited excerpt from that doc:
Create an alert:
DBMS_ALERT.REGISTER('emp_table_alert');
Create a trigger on your table to fire the alert:
CREATE TRIGGER emptrig AFTER INSERT ON emp
BEGIN
DBMS_ALERT.SIGNAL('emp_table_alert', 'message_text');
END;
From your .net code, you can the use something that calls this:
DBMS_ALERT.WAITONE('emp_table_alert', :message, :status, :timeout);
Make sure you read the docs for what :status and :timeout do.
You should look at Oracle Advanced Queuing. It offers the kind of functions your looking for.
You'll probably need a separate queue table where a trigger on ASYNC_PROCESS inserts messages. You then use the AQ functions to retrieve (or wait for) the next message in the queue table.
If you're doing this in C#.NET, why wouldn't you simply spawn a worker thread to do the update (via ODAC)? Why hand the responsibility over to Oracle when (it seems) you want a .NET process to make the update call (in background) and have the main process be notified of its completion.
See here and here for examples, although there are several approaches in .NET for this (delegates, events, async callbacks, thread pools, etc)