Hi I´m trying to create a robust facial capture system, which detects facial expressions, like blinks simles,etc, so the it can be translated into an animated model.
I have no idea where to start but I´m looking for some kind of c++ or java library to code this program.
The idea is that the video has to come from a webcam.
Thx alot
I've never used it myself, but I know several people who've used OpenCV for this sort of thing.
Related
Currently, warcaft's world editor is the only software which I know to use, I was wondering if there are other similar, but game making software out there? Thanks in advance.
There are programs that are similar in ways... Maybe not "world editors" but if you don't mind learning some code then there are programs like Game Maker: Studio that has 3D capabilities now and that you can make entire worlds from scratch also, there is Unity which is what i think you maybe looking for more likely as it is mainly made to make 3D games so it works well... Now both you will need to learn some coding but there are some shortcuts in Game Maker as in there are some drag and drop commands!
Here are some engines that you may like:
CryEngine - Great Graphics, free version is no longer updated.
Unreal Engine 4 - Good Graphics, Easy to use, No limitations for free
version.
Unity 5 - Not as good graphics, VERY beginner friendly, pro version is now free.
Hope this helps. With each one you could make stuff with no coding. UE4 and CryEngine have drag and drop languages, whereas Unity 5 requires C# or JS. You would only have to use these, however, if you wanted to makes games compared to just enviroments.
I am currently considering writing a small game. It is essentially a map where you can zoom out and in, and in certain places click on info boxes where, at some point, I hope to integrate minigames. Granted, game might be overstating it. Think of it as an interactive map. The theme is how mathematics can be applied in peoples every day life to raise awareness on the usefullness of mathematics.
The question is how I as fast as possible can make a reasonable prototype. If I recieve enough positive response on this I might try to code "the real thing" and use the prototype to obtain funding.
However, I am at a crossroad. I want something to work rather fast and have some C++ experience coding optimization problems, mainly in c-style. I am not convienced, though, that coding it in C++ is the fast way to obtain a prototype. Though I have some experience coding in C++, but have no experience in coding any sort of GUI.
As I see it there is a number of possibilites:
C++, possibly using some library, such as boost or ???.
Start out purely webbased, using e.g. HTML 5 and java.
Python
C#/.NET
Others, such as?
I have to admit I have little experience with anything besides C++ and the STL.
So my question to this wonderful forum is basically, is there a language that provides a significant advantage? Also, any additional insight or comments is more than welcome!
Python is a simpler language than C++, and for prototyping it will help you focus on the task at hand. You can use Pygame, a game library built on the excellent cross-platform SDL library. It provides 2D graphics, input, and audio mixing features. SDL is mainly a C library (and thus compatible with C++), and there are a number of very useful libraries that integrate with it:
SDL_image for loading images in various formats
SDL_ttf for rendering text using TrueType fonts
SDL_mixer for audio mixing
SDL_net for networking
SDL_gfx for graphics drawing primitives
So if you prototype in Python using Pygame, there is a reasonable chance you’ll be able to port what you make over to C++ with minimal hassle, if and when you choose to do so.
Possible options:
Go with what you know the best. Anything else will require a learning curve, which may be weeks to months long. If you're willing to take that road in order to make your prototype, then there are some really great tools available.
BlitzBasic is a good way to go, and is basically designed to be for games
I've done little games in Java using Slick2D - but you'll need good grounding in object-oriented coding to work effectively in Java. If you've got that from C++, then you can see a tech demo I built in Slick2D called Pedestrians. It's open source, and has demo videos here.
You might also ask your question on https://gamedev.stackexchange.com/ - a Q/A site dedicated to game programming
I would like to get started with particle systems in 2d. But I really don't know where to get started.
Well I suggest you read up on gamedevelopment. Also in which language are you going to develop it in? I suggest you start by searching for frameworks and such. For example for c# there is XNA from MS and if you use java you can use open GL read up about those things first and then search for some tutorials I guess there are a lot to find
We have a device that has an analog camera. We have a card that samples it and digitizes it. This is all done in directx. At this point in time, replacing hardware is not an option, but we need to code such that we can see this video feed real-time regardless of any hardware or underlying operating system changes occur in the future.
Along this line, we've chosen Qt to implement a GUI to view this camera feed. However, if we move to a linux or other embedded platform in the future and change other hardware (including the physical device where the camera/video sampler lives), we will need to change the camera display software as well, and that's going to be a pain because we need to integrate it into our GUI.
What i proposed was migrating to a more abstract model where data is sent over a socket to the GUI and the video is displayed live after being parsed from the socket stream.
First, is this a good idea or a bad idea?
Secondly, how would you implement such a thing? How do the video samplers usually give usable output? How can I push this output over a socket? Once I am on the receiving end parsing the output, how do I know what to do with the output (as in how to get the output to render)? The only thing I can think of would be to write each sample to a file and then to display the contents of the file every time a new sample arrives. This seems like an inefficient solution to me, if it would work at all.
How do you recommend I handle this? Are there any cross-platform libraries available for such a thing?
Thank you.
edit: i am willing to accept suggestions of something different rather than what is listed above.
Have you looked at QVision? It is a Qt based framework for managing video and video processing. You don't need the processing, but I think it will do what you want.
Anything that duplicates the video stream is going to cost you in performance, especially in an embedded space. In most situations for video, I think you're better off trying to use local hardware acceleration to blast the video directly to the screen. With some proper encapsulation, you should be able to use Qt for the GUI surrounding the video, and have a class that is platform specific that you use to control the actual video drawing to the screen (where to draw, and how big, etc.).
Edit:
You may also want to look at the Phonon library. I haven't looked at it much, but it appears to support showing video that may be acquired from a range of different sources.
I'm new to this field - but I need to perform a WAV-to-MIDI conversion in java.
Is there a way to know what exactly are the steps involved in WAV-to-MIDI conversion?
I have a very rough idea as in you need to;
sample the wav file, filter it, use FFT for spectral analysis, feature extraction and then write the extracted features on to MIDI.
But I cannot find solid sources or papers as in how to do all that?
Can some one give me clues as in how and where to start?
Are there any Open Source APIs available for this WAV-to-MIDI conversion process?
Advance thanks
It's a more involved process than you might imagine.
This research problem is often referred to as music transcription: the act of converting a low-level representation of music (e.g., waveform) into a higher-level representation such as MIDI or even sheet music.
The sophistication of your solution will depend upon the complexity of your input data. Tons of research papers address music transcription only on monophonic piano or drums... because they are easy to transcribe. (Relatively.) Violin is harder. Voice is even harder. Violin plus voice plus piano is much harder. A symphony is nearly impossible. You get the picture.
The basic elements of music transcription involve any of the following overlapping areas:
(multi)pitch estimation
instrument recognition, timbral modeling
rhythm detection
note onset/offset detection
form/structure modeling
Search for papers on "music transcription" on Google Scholar or from the ISMIR proceedings: http://www.ismir.net. If you are more interested in one of the above subtopics, I can point you further. Good luck.
EDIT: That being said, there are existing solutions that we can all find on the web. Feel free to try them. But as you do, evaluate them with a critical eye and ear. What types of audio signals would cause transcription to fail?
EDIT 2: Ah, you are only doing this for piano. Okay, this is doable. Music transcription has advanced to the point where it can transcribe monophonic piano pretty well. A Rachmaninov concerto will still pose problems.
Our recommendations depend upon your end goal. You state "need to perform... in Java." So it sounds like you just want something to work regardless of how it gets you there. In that case, I agree 100% with others: use something that exists.
That's actually an interesting question; all of the MIR libraries I know are typically C/C++/Python/Matlab. But not Java. The EchoNest has a Java API, but I don't think it does note-level transcription. http://developer.echonest.com. (Edit: It does note-level transcription. The returned data includes pitch, timbre, beat, tatum, and more. But I find polyphony is still a problem.)
Oh, Marsyas is Java-based. Cool. I thought it was just C++. http://marsyas.info/ I recommend this. It's developed by George Tzanetakis, a professor in MIR. It does signal-level analysis and should be a good option.
Now, if this is for a fun learning experience, I think you can use the sound manipulation utilities in Java to experiment with the WAV signal and see what comes out.
EDIT: This page describes MIR software better than I can: The Tools We Use
For Matlab, you may be interested in the MIR Toolbox
Here is a nice page of common datasets: MIR Datasets
This is a very big undertaking for being new to the field, unless you mean you are familiar with signal analysis and feature detection in general and want to look more specifically into automatic transcription.
There is no API for WAV to MIDI conversion. Vamp is a framework for feature extraction plugins, but to do automatic transcription you would need to use all the functionality of the existing plugins, plus implement functionality that exists in none of them yet.
Browse through the descriptions of the plugins on the vamp download page, any descriptions you do not understand are topics you should start researching if you want to do this.
If you don't need to automate this task (ie, for a website where people can upload MP3's and get MIDI files back), then you should consider using a tool like Melodyne which is already quite good at going this. As Steve noted, this is a very difficult task to accomplish, and even the best algorithms and solutions present at the moment are not 100% reliable.
So if you are just doing studio work and need to do a few conversions, it will probably save you a bit of time (and lots of headache) to use a tool already designed for this task.
This is a field which is still highly under development, yet, there are some (experimental) algorithms available.
You can install sonic annotator and use a few vamp plugins.
For example:
./sonic-annotator file.wav -d vamp:qm-vamp-plugins:qm-transcription:transcription -w midi
./sonic-annotator file.wav -d vamp:silvet:silvet:notes -w midi
./sonic-annotator file.wav -d vamp:ua-vamp-plugins:mf0ua:mf0ua -w midi
Dolphin, sorry to be brusque, but you have completely underestimated the problem. What you want to achieve - a full piano sound transcription involving all parameters that were used while playing would need an enormous amount of research with people who have worked in the field for many years. Even a group of PhDs in signal processing would have to invest a lot of work to even come close to what you mean. Music transcription has needed decades of work to even work halfway reliable. I'd suggest you pick a different problem which you can manage better than this.