I am comparing x264 and x265 encoders, with lossless and medium presets.
I am so surprised to see below performance results.
| Format | Size (KB) | fps |
|********************|*************|******|
| YUV 420 | 9,182,363 | |
| | | |
| x265 (--lossless) | 442,890 | 1.8 |
| x265 (--medium) | 12,243 | 2.8 |
| | | |
| x264 (--lossless) | 319,139 | 7.2 |
| x264 (--medium) | 25,747 | 7.5 |
It raises two questions to me
Why lossless encoding processes less frames per seconds than lossy one? ( I supposed there should be less processing to do)
Why x265 lossless output is much bigger than x264 lossless output? Is this an implementation effect, or related to the HEVC standard?
I might try to answer your questions:
1. As you say, the algorithms are different and the lossy is apparently more efficient.
2. This is an implementation effect since both codecs use a different algorithm which differ in speed and size of the output (and internally ram usage, cpu usage etc)
My question back, if you are comparing these, you do that for a reason. The results should provide you the information you need. You can now make en educated quess, so please do so ;)
Related
There are several wrapper types in the Rust standard library:
The cells in the std::cell module: Cell and RefCell
The reference-counted wrappers, like Rc and Arc.
The types in the std::sync module: Mutex or AtomicBool for example
As I understand it, these are wrappers which provide further possibilities than a simple reference. While I understand some basics, I cannot see the whole picture.
What do they do exactly? Do cells and reference-counted families provide orthogonal or similar features?
There are two essential concepts in Rust:
Ownership,
Mutability.
The various pointer types (Box, Rc, Arc) are concerned with Ownership: they allow controlling whether there is a single or multiple owners for a single object.
On the other hand, the various cells (Cell, RefCell, Mutex, RwLock, AtomicXXX) are concerned with Mutability.
The founding rule of Rust's safety is Aliasing NAND Mutability. That is, an object can only be safely mutated if there is no outstanding reference to its interior.
This rule is generally enforced at compile time by the borrow checker:
if you have a &T, you cannot also have a &mut T to the same object in scope,
if you have a &mut T, you cannot also have any reference to the same object in scope.
However, sometimes, this is not flexible enough. Sometimes you DO need (or want) the ability to have multiple references to the same object and yet mutate it. Enter the cells.
The idea of Cell and RefCell is to permit mutability in the presence of aliasing in a controlled manner:
Cell prevents the formation of reference to its interior, avoiding dangling references,
RefCell shifts the enforcement of Aliasing XOR Mutability from compile time to runtime.
This functionality is sometimes described as providing interior mutability, that is where an object which otherwise looks immutable from the outside (&T) can actually be mutated.
When this mutability extends across multiple threads, you will instead use Mutex, RwLock or AtomicXXX; they provide the same functionality:
AtomicXXX are just Cell: no reference to the interior, just moving in/out,
RwLock is just RefCell: can obtain references to the interior through guards,
Mutex is a simplified version of RwLock which does not distinguish between a read-only guard and write guard; so conceptually similar to a RefCell with only a borrow_mut method.
If you come from a C++ background:
Box is unique_ptr,
Arc is shared_ptr,
Rc is a non thread-safe version of shared_ptr.
And the cells provide a similar functionality as mutable, except with additional guarantees to avoid aliasing issues; think of Cell as std::atomic and RefCell as a non thread-safe version of std::shared_mutex (which throws instead of blocking if the lock is taken).
Thanks to Matthieu's good answer, here is a diagram to help people to find the wrapper they need:
+-----------+
| Ownership |
+--+--------+ +================+
| +-Static----->| T |(1)
| | +================+
| |
| | +================+
| +-----------+ | Local Val| Cell<T> |(1)
+-Unique-->| Borrowing +--+-Dynamic---->|----------------|
| +-----------+ | Ref| RefCell<T> |(1)
| | +================+
| |
| | +================+
| | Threaded | AtomicT |(2)
| +-Dynamic---->|----------------|
| | Mutex<T> |(1)
| | RwLock<T> |(1)
| +================+
|
|
| +================+
| +-No--------->| Rc<T> |
| | +================+
| Locally +-----------+ |
+-Shared-->| Mutable? +--+ +================+
| +-----------+ | Val| Rc<Cell<T>> |
| +-Yes-------->|----------------|
| Ref| Rc<RefCell<T>> |
| +================+
|
|
| +================+
| +-No--------->| Arc<T> |
| | +================+
| Shared +-----------+ |
+-Between->| Mutable? +--+ +================+
Threads +-----------+ | | Arc<AtomicT> |(2)
+-Yes-------->|----------------|
| Arc<Mutex<T>> |
| Arc<RwLock<T>> |
+================+
In those cases, T can be replaced with Box<T>
Use AtomicT when T is a bool or a number
To know if you should use Mutex or RwLock, see this related question.
Background: I'm confuse with Erlang's scheduler for a long time until have a look at The Beam Book. I had do some research on async/non-blocking programming in some language(Elixir/Erlang, Scala/Java, Golang) which include Actor pattern, Future/Promise, coroutine mainly. Coroutine and Actor pattern is similarly in term of they can both think as lightweight process.
The Problem
I'm find a weakness of the async programming: It will cause a lightweight-process(or a task) re-queue to the end of the scheduler Ready Queue if it invoke any of the block operation. The block operation will not block OS-Thread but it occurred mainly because of invoke a async action such as 'aio_read'.
re-queue means the process will be put at the end of Scheduler even if the process is just scheduled.In server-side programming, it will make a client request delay with a relatively long time compare with it should process time.The Beam Book give a detail description:
A processs trying to do a receive on an empty mailbox or on a mailbox with no matching messages will yield and go into the waiting state.
When a message is delivered to an inbox the sending process will check whether the receiver is sleeping in the waiting state, and in that case it will wake the process, change its state to runable, and put it at the end of the appropriate ready queue.
The affect could be see in many benchmark test: More request, more response time for every request.
A good scheduler should make the response time nearly the real process time of the request if ignore OS-Thread Context Switch.
I haven't seem others discuss the aspect yet.
As a conclusion, there are two question:
1. I want make a confirm whether re-queue problem really exist in async-programming world.
2. Besides, Is Erlang really has the problem if it handle tens of thousands of the process, especially use many GenServer.call in a request-response-chain?
Your answers are here: https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-scheduler-details.html
Reposting it here, allowing edits.
Erlang Scheduling
Erlang as a real-time platform for multitasking uses Preemptive Scheduling. The responsibility of an Erlang scheduler is selecting a Process and executing their code. It also does Garbage Collection and Memory Management. The factor of selecting a process for execution is based on their priority level which is configurable per process and in each priority level processes are scheduled in a round robin fashion. On the other hand the factor of preempting a process from execution is based on a certain number of Reductions since the last time it was selected for execution, regardless of its priority level. The reduction is a counter per process that is normally incremented by one for each function call. It is used for preempting processes and context switching them when the counter of a process reaches the maximum number of reductions. For example in Erlang/OTP R12B this maximum number was 2000 reductions.
The scheduling of tasks in Erlang has a long history. It has been changing over the time. These changes were affected by the changes in SMP (Symmetric Multi-Processing) feature of Erlang.
Scheduling Before R11B
Before R11B Erlang did not have SMP support, so just one scheduler was run in the main OS process’s thread and accordingly just one Run Queue existed. The scheduler picked runnable Erlang processes and IO tasks from the run queue and executed them.
Erlang VM
+--------------------------------------------------------+
| |
| +-----------------+ +-----------------+ |
| | | | | |
| | Scheduler +--------------> Task # 1 | |
| | | | | |
| +-----------------+ | Task # 2 | |
| | | |
| | Task # 3 | |
| | | |
| | Task # 4 | |
| | | |
| | Task # N | |
| | | |
| +-----------------+ |
| | | |
| | Run Queue | |
| | | |
| +-----------------+ |
| |
+--------------------------------------------------------+
This way there was no need to lock data structures but the written application couldn’t take advantage of parallelism.
Scheduling In R11B and R12B
SMP support was added to Erlang VM so it could have 1 to 1024 schedulers each was run in one OS process’s thread. However, in this version schedulers could pick runnable tasks from just one common run queue.
Erlang VM
+--------------------------------------------------------+
| |
| +-----------------+ +-----------------+ |
| | | | | |
| | Scheduler # 1 +--------------> Task # 1 | |
| | | +---------> | |
| +-----------------+ | +----> Task # 2 | |
| | | | | |
| +-----------------+ | | | Task # 3 | |
| | | | | | | |
| | Scheduler # 2 +----+ | | Task # 4 | |
| | | | | | |
| +-----------------+ | | Task # N | |
| | | | |
| +-----------------+ | +-----------------+ |
| | | | | | |
| | Scheduler # N +---------+ | Run Queue | |
| | | | | |
| +-----------------+ +-----------------+ |
| |
+--------------------------------------------------------+
Because of the resulting parallelism of this method, all shared data structures are protected with locks. For example the run queue itself is a shared data structure which must be protected. Although the lock can provide performance penalty, the performance improvements which was achieved in multi-core processors systems was interesting.
Some known bottlenecks in this version was as follows:
The common run queue becomes a bottleneck when the number of schedulers increases.
Increasing the involved lock of ETS tables which also affects Mnesia.
Increasing the lock conflicts when many processes are sending messages to the same process.
A process waiting to get a lock can block its scheduler.
However, separating run queues per scheduler was picked to solve these bottleneck issues in next versions.
Scheduling After R13B
In this version each scheduler has its own run queue. It decreases the number of lock conflicts in systems with many schedulers on many cores and also improves the overall performance.
Erlang VM
+--------------------------------------------------------+
| |
| +-----------------+-----------------+ |
| | | | |
| | Scheduler # 1 | Run Queue # 1 <--+ |
| | | | | |
| +-----------------+-----------------+ | |
| | |
| +-----------------+-----------------+ | |
| | | | | |
| | Scheduler # 2 | Run Queue # 2 <----> Migration |
| | | | | Logic |
| +-----------------+-----------------+ | |
| | |
| +-----------------+-----------------+ | |
| | | | | |
| | Scheduler # N | Run Queue # N <--+ |
| | | | |
| +-----------------+-----------------+ |
| |
+--------------------------------------------------------+
This way the locking conflicts when accessing the run queue is solved but introduces some new concerns:
How fair is the process of dividing tasks among run queues?
What if one scheduler gets overloaded with tasks while others are idle?
Based on what order a scheduler can steal tasks from an overloaded scheduler?
What if we started many schedulers but there all so few tasks to do?
These concerns lead the Erlang team to introduce a concept for making scheduling fair and efficient, the Migration Logic. It tries to control and balance run queues based on the statistics that collects from the system.
However we should not depend on the scheduling to remain exactly as it is today, because it is likely to be changed in future releases in order to get better.
As a conclusion, there are two question:
1. I want make a confirm whether re-queue problem really exist in async-programming world.
2. Besides, Is Erlang really has the problem if it handle tens of thousands of the process, especially use many GenServer.call in a
request-response-chain?
Depending which Erlang version are you talking about there are different tradeoffs. Since there are multiple queues in newer Erlang versions your problem does not exist in the form you are specifying it.
I have seen Erlang (and its VM) handle millions of Erlang processes just fine, also used an Erlang based system to handle 100.000+ clients with really tight SLA on the latency. Not sure about the details of your problem, but a reasonably written Erlang/Elixir service can handle the workload you specified unless there is something you left out.
I would like to see the data on Demographic -> Language grid grouped by ISO 3316 language, without differences between the (optional) ISO639 country code national variant.
For example, instead of seeing:
| Language | Visits |
|----------|--------|
| it | 56,027 |
| it-it | 35,130 |
| en-us | 5,878 |
| en | 1,211 |
| es | 897 |
| es-es | 576 |
| ... | ... |
I would like to see something like this:
| Language | Visits |
|----------|--------|
| it | 91,157 |
| en | 7,089 |
| es | 1473 |
|----------|--------|
Is it possible?
You can do an advanced filter at the profile level that will search and replace the language input field and keep only the first 2 letters.
If you want to keep the original language variations in your reports, you'll need to define your own dimension and use the tracking code (or Google Tag Manager) to fill it from the browser's language setting:
// this will extract 'it' from 'it-IT' or 'it-CH'
var primaryLanguage = navigator.language.match(/[^-]+/)[0];
ga('set', 'dimension1', primaryLanguage);
https://developers.google.com/analytics/devguides/collection/analyticsjs/custom-dims-mets
HTTP uses the IEFT Language Tag, so the primary language can be provided in different standards. It can consist of up to 8 letters. This usually won't be the case, but if you need to account for this, you need some extra logic to group languages from different standards.
I want to create an static drawing (say any animals like giraffe) using some points, lines, drawing etc. Now i want to update the drawing by passing the parameters say height of his legs, its width or its color.
The parameters are supplied from the web page. The image will be a 2D image
I am searching on which technology should i implement this for more than 10 hours but cannot find any perfect solution.
Right now i am thinking i can use Adobe flash in which i can do some programming to create an drawing and change the drawing by passing the parameters to a Flash file, i think we can pass it when we embed an flv.
Whether i am right? Or there is any other solution. I have no knowledge of any thing except asp.net
Please help.
Any help is appreciated
I'd like to build on the previous post - you could also incorporate svg graphics into the mix. This would allow you control over color, width, and height. You can manipulate SVG files with javascript (Dynamic SVG). You'll probably get that going faster than learning action script.
If you just want to be able to stretch or recolor parts of an image, you could do that using ordinary HTML parameters. Just create a giraffe image, break it into the chunks that you want to be able to resize independently, and use CSS layout or tables to assemble them. Here's an artistic rendering:
___________________________
|image 1 V__ <<|
|head |oo | <<| <--- delicious acacia leaves
| | < <<<|
---------------------------
|image 2 | | |
|neck |o| |
| | | |
---------------------------
|image 3 / | |
|body /------/ \ |
| | \ |
---------------------------
|image 4| | | | | | | | |
|legs | | | | | | | | |
| \_/ \_/ \_/ \_/ | <--- I do not know what giraffe feet look like
---------------------------
If you want to give your giraffe a short neck without changing anything else about it, you can just alter the height attribute of the second image, like so:
___________________________
|image 1 V__ <<|
|head |oo | <<|
| | < <<<|
---------------------------
|image 2 | | |
---------------------------
|image 3 / | |
|body /------/ \ |
| | \ |
---------------------------
|image 4| | | | | | | | |
|legs | | | | | | | | |
| \_/ \_/ \_/ \_/ |
---------------------------
Obviously, changing the width of just one image would cause the boundaries to no longer match up, so you'd need to change them all to the same value.
To handle color changes, you can make use of image transparency. Each image would be white, with a transparent region representing the giraffe. Then, you'd set the background color of the div or table cell to the color you want the giraffe to appear. Again, this is clunky, but it would let you do what you want without needing anything other than static GIF / PNG images and basic HTML.
From the get go: sorry if I'm not using the proper emacs terminology -- I'm relatively wet behind the ears in the emacs world.
Most of my work in emacs is for programming R, and I'm using ESS and ECB to do so quite happily. I'd like to build a custom ECB layout which uses the entire bottom of the screen as my R console, while putting some ECB-specific buffers on the left.
Using ECB-esque layout diagrams, I'd like my layout to look like pretty much exactly like "left13", except I'd like the entirety of the "compilation" buffer to be my running R console (or any shell, for that matter):
-------------------------------------------------------
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| Directories | Edit |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
-------------------------------------------------------
| |
| R Console |
| |
-------------------------------------------------------
If I can just split my buffer in two (vertically), then call ecb-activate from the top buffer (and not allow it to touch my bottom buffer), I'm imagining it could work (hence the subject of my question).
That doesn't work, though, and I don't know how to get an entier "bottom pane" out of a layout to work in the way I like using trying to use ECB's customize layout functionality.
Does anybody know if/how I can do this?
Short answer: No.
Longer answer: Unfortunately, ECB completely takes over Emacs "window" management at a very low level. So it's all or nothing. You can't comfortably combine it with regular window splitting. What you might be able to do, is to adjust the layout ECB gives you or to program a custom layout. (Some assembly required.)