I have just built a proof of concept for an asp.net MVC controller to (1) generate a barcode from user input using barcode rendering framework and (2) embed it in a PDF document using wkhtmltopdf.exe
Before telling my client it's a working solution, I want to know it's not going to bring down their website. My main concern is long-term reliability -- whether for instance creating and disposing the unmanaged system process for wkhtmltopdf.exe might leak something. (Peak performance and load is not expected to be such an issue - only a few requests per minute at peak).
So, I run a couple of tests from the Windows command line:
(1) 1,000 Requests in Sequence (ie 1 at a time)
for /l %i in (1,1,1000) do curl ^
"http://localhost:8003/Home/Test?text=Iteration_%i___012345&scale=2&height=50" -o output.pdf
(2) Up to 40 requests sent within 2 seconds
for /l %i in (1,1,40) do start "Curl %i" curl ^
"http://localhost:8003/Home/Test?text=Iteration_%i___012345&scale=2&height=50" -o output%i.pdf
And I record some performance counters in perfmon before, during & after. Specifically I look at total processes, threads, handles, memory, disk use on the machine and on the IIS process specifically.
So, my questions:
1) What would you consider acceptable evidence that the the solution looks to be at low risk of bringing down the server? Would you amend what I've done, or would you do something completely different?
2) Given my concern is reliability, I think that the 'Before' vs 'After' figures are the ones I most care about. Agree or not?
3) Looking at the Before vs After figures, the only concern I see is the 'Processes Total Handle Count'. I conclude that launching wkhtmltopdf.exe nearly a thousand times has probably not leaked anything or destabilised the machine. But I might be wrong and should run the same tests for hours or days to reduce the level of doubt. Agree or not?
(The risk level: A couple of people's jobs might be on the line if it went pear-shaped. Revenue on the site is £1,000s per hour).
My perfmon results were as follows.
700 Requests in Sequence
1-5 Mins 10 Mins
Counter Before Test Min Ave Max After Test
System
System Processes 95 97 100 101 95
System Threads 1220 1245 1264 1281 1238
Memory Available MB 4888 4840 4850 4868 4837
Memory % Committed 23 24 24 24 24
Processes Total Handle Cou 33255 34147 34489 34775 34029
Processor % Processor Time 4 to 30 40 57 78 1 to 30
Physical Disk % Disk Time 1 0 7 75 0 to 30
IIS Express
% Processor Time 0 0 2 6 0
Handle Count 610 595 640 690 614
Thread Count 34 35 35 35 35
Working Set 138 139 139 139 139
IO Data KB/sec 0 453 491 691 0
20 Requests sent within 2 seconds followed by 40 Requests sent within 3 seconds
1-5 Mins 10 Mins
Counter Before Test Min Ave Max After Test
System
System Processes 95 98 137 257 96
System Threads 1238 1251 1425 1913 1240
Memory Available MB 4837 4309 4694 4818 4811
Memory % Committed 24 24 25 29 24
Processes Total Handle Cou 34029 34953 38539 52140 34800
Processor % Processor Time 1 to 30 1 48 100 1 to 10
Physical Disk % Disk Time 0 to 30 0 7 136 0 to 10
IIS Express
% Processor Time 0 0 1 29 0
Handle Count 610 664 818 936 834
Thread Count 34 37 50 68 37
Working Set 138 139 142 157 141
IO Data KB/sec 0 0 186 2559 0
Related
I have compiled Quantum ESPRESSO (Program PWSCF v.6.7MaX) for GPU acceleration (hibrid MPI/OpenMP) with the next options:
module load compiler/intel/2020.1
module load hpc_sdk/20.9
./configure F90=pgf90 CC=pgcc MPIF90=mpif90 --with-cuda=yes --enable-cuda-env-check=no --with-cuda-runtime=11.0 --with-cuda-cc=70 --enable-openmp BLAS_LIBS='-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core'
make -j8 pw
Apparently, the compilation ends succesfully. Then, I execute the program:
export OMP_NUM_THREADS=1
mpirun -n 2 /home/my_user/q-e-gpu-qe-gpu-6.7/bin/pw.x < silverslab32.in > silver4.out
Then, the program starts running and print out the next info:
Parallel version (MPI & OpenMP), running on 8 processor cores
Number of MPI processes: 2
Threads/MPI process: 4
...
GPU acceleration is ACTIVE
...
Estimated max dynamical RAM per process > 13.87 GB
Estimated total dynamical RAM > 27.75 GB
But after 2 minutes of execution the job ends with error:
0: ALLOCATE: 4345479360 bytes requested; status = 2(out of memory)
0: ALLOCATE: 4345482096 bytes requested; status = 2(out of memory)
--------------------------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[47946,1],1]
Exit code: 127
--------------------------------------------------------------------------
This node has > 180GB of available RAM. I check the Memory use with the top command:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
89681 my_user 20 0 30.1g 3.6g 2.1g R 100.0 1.9 1:39.45 pw.x
89682 my_user 20 0 29.8g 3.2g 2.0g R 100.0 1.7 1:39.30 pw.x
I noticed that the process stops when RES memory reaches 4GB. This are the caracteristics of the node:
(base) [my_user#gpu001]$ numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 28 29 30 31 32 33 34 35 36 37 38 39 40 41
node 0 size: 95313 MB
node 0 free: 41972 MB
node 1 cpus: 14 15 16 17 18 19 20 21 22 23 24 25 26 27 42 43 44 45 46 47 48 49 50 51 52 53 54 55
node 1 size: 96746 MB
node 1 free: 70751 MB
node distances:
node 0 1
0: 10 21
1: 21 10
(base) [my_user#gpu001]$ free -lm
total used free shared buff/cache available
Mem: 192059 2561 112716 260 76781 188505
Low: 192059 79342 112716
High: 0 0 0
Swap: 8191 0 8191
The version of MPI is:
mpirun (Open MPI) 3.1.5
This node is a compute node in a cluster, but no matter if I submit the job with SLURM or run it directly on the node, the error is the same.
Note that I compile it on the login node and run it on this GPU node, the difference is that on the login node it has no GPU connected.
I would really appreciate it if you could help me figure out what could be going on.
Thank you in advance!
I am trying to monitor the memory usage of kamailio. The PKG memory I have arranged, though the shared memory is something else.
According to documentation you can get the shared memory like this:
kamcmd mod.stats all shm
In my case, I get the following:
Module: core
{
msg_lump_cloner(984): 1560
dns_cache_mk_bad_entry(867): 192
dns_cache_mk_rd_entry(1216): 232
build_req_buf_from_sip_req(2155): 2752
sip_msg_shm_clone(495): 822664
tcpconn_new(957): 16416728
counters_prefork_init(207): 30720
cfg_clone_str(130): 208
cfg_shmize(217): 976
init_pt(105): 4200
cfg_parse_str(906): 80
init_pt(106): 16
init_pt(111): 40
create_avp(175): 10368
register_timer(1012): 720
cfg_register_ctx(47): 128
init_tcp(4643): 8192
init_tcp(4637): 32768
init_tcp(4629): 8
init_tcp(4622): 8
init_tcp(4615): 8
init_tcp(4609): 8
init_tcp(4597): 8
init_avps(90): 8
init_avps(89): 8
init_dst_blacklist(437): 16384
init_dst_blacklist(430): 8
timer_alloc(515): 96
init_dns_cache(366): 8
init_dns_cache(358): 16384
init_dns_cache(351): 16
init_dns_cache(345): 8
init_timer(284): 8
init_timer(283): 16384
init_timer(282): 8
init_timer(281): 8
init_timer(270): 8
init_timer(238): 8
init_timer(221): 278544
init_timer(220): 8
init_timer(207): 8
cfg_child_cb_new(830): 64
sr_cfg_init(361): 8
sr_cfg_init(354): 8
sr_cfg_init(347): 8
sr_cfg_init(335): 8
sr_cfg_init(323): 24
shm_core_lock_init(153): 8
Total: 17660616
}
... [ list of modules here, snipped because not relevant I think] ...
So I guess the one I need is Total: 17660616.
However, what does this mean? Is it the total allocated memory? Available memory?
The commands you used is for seeing the used of shared memory per module.
If you want the statistics for total used and free shared memory, then use the command:
kamctl stats shmem
or:
kamcmd stats.get_statistics shmem:
The output will be like:
shmem:fragments = 179
shmem:free_size = 127471848
shmem:max_used_size = 7269016
shmem:real_used_size = 6745880
shmem:total_size = 134217728
shmem:used_size = 5106728
The free_size value is the one you have to monitor for available shared memory.
I am submitting a job using qsub that runs parallelized R. My
intention is to have R programme running on 4 different cores rather than 8 cores. Here are some of my settings in PBS file:
#PBS -l nodes=1:ppn=4
....
time R --no-save < program1.R > program1.log
I am issuing the command ta job_id and I'm seeing that 4 cores are listed. However, the job occupies a large amount of memory(31944900k used vs 32949628k total). If I were to use 8 cores, the jobs got hang due to memory limitation.
top - 21:03:53 up 77 days, 11:54, 0 users, load average: 3.99, 3.75, 3.37
Tasks: 207 total, 5 running, 202 sleeping, 0 stopped, 0 zombie
Cpu(s): 30.4%us, 1.6%sy, 0.0%ni, 66.8%id, 0.0%wa, 0.0%hi, 1.2%si, 0.0%st
Mem: 32949628k total, 31944900k used, 1004728k free, 269812k buffers
Swap: 2097136k total, 8360k used, 2088776k free, 6030856k cached
Here is a snapshot when issuing command ta job_id
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1794 x 25 0 6247m 6.0g 1780 R 99.2 19.1 8:14.37 R
1795 x 25 0 6332m 6.1g 1780 R 99.2 19.4 8:14.37 R
1796 x 25 0 6242m 6.0g 1784 R 99.2 19.1 8:14.37 R
1797 x 25 0 6322m 6.1g 1780 R 99.2 19.4 8:14.33 R
1714 x 18 0 65932 1504 1248 S 0.0 0.0 0:00.00 bash
1761 x 18 0 63840 1244 1052 S 0.0 0.0 0:00.00 20016.hpc
1783 x 18 0 133m 7096 1128 S 0.0 0.0 0:00.00 python
1786 x 18 0 137m 46m 2688 S 0.0 0.1 0:02.06 R
How can I prevent other users from using the other 4 cores? I like to mask somehow that my job is using 8 cores with 4 cores idling.
Could anyone kindly help me out on this? Can this be solved using pbs?
Many Thanks
"How can I prevent other users from using the other 4 cores? I like to mask somehow that my job is using 8 cores with 4 cores idling."
Maybe a simple way around it is to send a 'sleep' job on the other 4? Seems hackish though! (ans warning, my PBS is rusty!)
Why not do the following -
ask PBS for ppn=4, additionally, ask for all the memory on the node, i e
#PBS -l nodes=1:ppn=4 -l mem=31944900k
This might not be possible on your setup.
I am not sure how R is parallelized, but if it is OPENMP you could definitely ask for 8 cores but set OMP_NUM_THREADS to 4
This is what the chart currently looks like:
This is all the data in the database that it is currently using. (please excuse how the headers are not properly aligning here)
Id(Key) Confidence Love Stress Date/Time
193 0 0 0 12/3/2010 11:33:47 PM
194 55 55 55 12/3/2010 11:34:04 PM
195 30 40 20 12/3/2010 11:34:11 PM
196 40 50 30 12/3/2010 11:34:20 PM
197 50 60 40 12/3/2010 11:34:28 PM
198 60 70 50 12/3/2010 11:34:45 PM
199 70 80 60 12/3/2010 11:34:53 PM
200 80 90 70 12/3/2010 11:34:59 PM
201 20 3 11 12/3/2010 11:36:42 PM
202 20 3 11 12/3/2010 11:37:08 PM
203 76 34 34 12/3/2010 11:37:41 PM
204 3 4 2 12/4/2010 12:14:15 AM
205 5 100 8 12/4/2010 12:17:57 AM
206 77 89 3 12/12/2010 8:08:49 PM
This is the SQL statement I have the chart configured too:
SELECT [ConfidenceLevel], [LoveLevel],
[DateTime], [StressLevel] FROM
[UserData]
My issue is in cases like this example, the data recorded around 12/4 looses it's fidelity and is un - "see able", it all blends together.
How can I configure the chart so that the last 20 days are always readable on the chart and that they don't blur together?
Thank You.
Here are some thoughts, although please be forgiving as it's hard to get to the root of your challenge without a little more context for the data and the purpose of the graph.
First, the data sample itself seems less than adequate. Indeed, it has gaping holes: no data from 12/5 to 12/11? With data, it's garbage in, garbage out. From that perspective, one of your challenges would be to start getting more consistent data.
The line graph falsely implies there is data for the 12/5 to 12/11 date range. Was the "Confidence" level on 12/8 really 38? Maybe it was 0? Maybe it was 140? The point is, we don't know, yet the connected line graph (by it's very nature) is faking us into thinking we do know.
Also, if you're going to use a scale of "days", then I don't understand how you can have multiple values for a single day and try to plot all of those? (Maybe that's not what you're doing... .) I would think you would want to take the average of each of the categories grouped by day and have those groupings be your daily data values. And if you don't have data for a day, then refrain from plotting data you don't have.
i think the issue is with the chart type you selected as compared to the density/sparsity of your data.
this maybe reminds me of a box-and-whisker type chart,
also consider a bubble chart.
I might be overlooking something due to deadline stress. But this behaviour amazes me.
It looks as if the cursor caches 100 rows and the continue statement flushes the cache
and begins with the first record of a new cache fetch.
I narrowed it down to the following script:
drop table test1;
create table test1 (test1_id NUMBER);
begin
for i in 1..300
loop
insert into test1 values (i);
end loop;
end;
/
declare
cursor c_test1 is
select *
from test1;
begin
for c in c_test1
loop
if mod(c.test1_id,10) = 0
then
dbms_output.put_line(c_test1%ROWCOUNT||' '||c.test1_id||' Continue');
continue;
end if;
dbms_output.put_line(c_test1%ROWCOUNT||' '||c.test1_id||' Process');
end loop;
end;
/
1 1 Process
2 2 Process
3 3 Process
4 4 Process
5 5 Process
6 6 Process
7 7 Process
8 8 Process
9 9 Process
10 10 Continue **Where are tes1_id's 11 to 100?**
11 101 Process
12 102 Process
13 103 Process
14 104 Process
15 105 Process
16 106 Process
17 107 Process
18 108 Process
19 109 Process
20 110 Continue **Where are tes1_id's 111 to 200?**
21 201 Process
22 202 Process
23 203 Process
24 204 Process
25 205 Process
26 206 Process
27 207 Process
28 208 Process
29 209 Process
30 210 Continue **Where are tes1_id's 211 to 300?**
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - Production
PL/SQL Release 11.1.0.7.0 - Production
redhat release 5
2 node RAC
It's a bug: 7306422
Pawel Barut wrote:
http://pbarut.blogspot.com/2009/04/caution-for-loop-and-continue-in-oracle.html
Workaround :
SQL> ALTER SESSION SET PLSQL_OPTIMIZE_LEVEL = 1;
Regards,
Rob