How can the Google Vision API be used to detect if a head is completely inside an image or partly cut off by the image frame?
3 examples:
Image 1 shows a complete head
Image 2 shows a cut off head where the full "face" is visible
Image 3 shows a cut off head where also the face is cut off
To narrow down the question, the following cases should be detected:
there is a completely visible head in the image
there is a partly visible head in the image where parts of the head are outside the image bounds
The following is out of scope for this question:
heads that are spatially or scenically inside the image bounds but fully or partly covered by other objects
there are no parts of a head visible in the image, e.g. if there is only a neck visible it can't be assumed that there is or is not a head attached to it
the effectiveness or efficiency of the API in detecting faces that are fully or partly visible, file that under caveats
I have checked the documentation but it doesn't say anything about head crop-off detection.
I am not asking for code but whether / how the API can be used for the described purpose. Hence neither the question contains any code nor is an answer expected to contain any code. If you are looking for code examples for API calls, take a look at the plenty example calls in the API docs.
There was a meta discussion about this question.
Image credit
The API does provide boundingPoly and fdBoundingPoly to determine where the face (not head) is located inside the image.
Even though the API does not provide the bounds for the "head", the example below demonstrates that boundingPoly is pretty equivalent to the larger face area which can apparently be used as substitute.
Here are the API results for boundingPoly (red dots) and fdBoundingPoly (blue dots):
Image 1: shows the face bounds for a fully visible head. The boundingPoly which is described as the larger face area is apparently pretty equivalent to the "head" area.
Image 2: the bounding boxes are missing y coordinates for the 4 top dots (2 red, 2 blue; depicted with y=0 coordinates for the purpose of demonstration), which can be used as indication of a cropped-off head.
Image 3: the API does not result any faceAnnotations, so it can be easily detected as a substantially cropped-off head or an image without any partly visible head which won’t be distinguished in this case.
Related
Turns out I’m working with the Autodesk Forge viewer and Three.js, trying to render 2D text that can be interacted with (specifically select, rotate, and move).
To do this I am working with meshes (using MeshBasicMaterial, Mesh and TextGeometry) but it turns out that the text does not look perfectly sharp, it presents aliasing and I found that according to the API reference, the antialiasing is not applicable to 2d.
Here are some examples of the problem, as you can see, the more I move away from the plane, the worse the text looks (and even up close it doesn't look perfect):
I have tried to make a test representing the text with a Sprite (despite the fact that it would mean having to change the entire implementation already made with meshes of other functionalities) but apart from the fact that I cannot see it, I have seen example images and they do not appear either well: aliasing is visible from a distance and it looks really blurry up close. Here some examples:
Is there a way to correct this problem or is this the most I can get in 2D? I've tried searching for information on this but can't find anything helpful. And what has puzzled me the most has been realizing that antialiasing was not applicable in the case of 2d, like making it clear that nothing can be done to fix it.
I would be very grateful if you could solve my doubts, thank you very much in advance for your help.
An easier alternative, is to just use a higher pixel ratio for the renderer...
window.devicePixelRatio=2;
viewer.resize();
For example, using the custom geometry text, from Joao's demo, you can see the same aliasing issue at DPR=0.5 and DPR=1.0 ...
https://joaomartins-forge.github.io/textgeometry-sample/
But when I set the DPR=2.0, the text looks clean. The trade off is rendering performance, but your 2D drawings may be simple enough that it won't matter. You can use a 'mouse up' camera settle trick, to switch between DPR of 1 and 2, if you want a better UX experience.
There are a few ways to solve this aliasing issue for 2D (and 3D text).
The way I would recommend for your use case, is to use DIV elements (THREE.CSS3DRenderer), instead of text converted into three.js tessellated triangle geometry, as shown in this blog post:
https://forge.autodesk.com/blog/how-do-you-add-labels-forge-viewer
You can find out more information about THREE.CSS3DRenderer here:
https://threejs.org/docs/#examples/en/renderers/CSS3DRenderer
and an Example here: https://threejs.org/examples/#css3d_periodictable
Using CSS3DRenderer instead of CSS2DRenderer, means you will get the correct scaling (and rotation) of the div element as you zoom into your 2D drawing and the mathematics inside the calculation for the matrix transform has less edge-cases.
Once you are using DIV elements for your text, you will notice that the text is sharper and has no aliasing issues. That's because it is not being rasterized by the webGL pipeline, but by the SKIA library used by chrome/firefox/opera/etc for rasterizing text.
There is one final option, that uses signed-distance fields, but it's probably overkill for what you need.
Let me know if you want some example code.
I am using aplypy to create an RGB image of the Eagle nebula from three FITS files, representing the red, green and blue components of the image. The FITS files are available here, 673nm being the red, 656nm the green and 502nm the blue.
import aplpy
aplpy.make_rgb_cube(['673nmos.fits','656nmos.fits','502nmos.fits'], 'nmod_cube.fits')
aplpy.make_rgb_image('nmod_cube.fits','nmod.png')
f = aplpy.FITSFigure('../data/nmod_cube_2d.fits')
f.show_rgb('../data/output/nmod.png')
The image should look something like this, but comes out looking like the image below. The blue component is clearly misaligned with the red and green.
make_rgb_cube is supposed to realign the three images in to the same projection based on the WCS information in each FITS header, according to the docs, but it doesn't seem to be working in this case.
Do I need to employ a star matching library to get the alignment accuracy?
There is an error in the WCS coordinates in the header of the Blue FITS file. I have contacted spacetelescope.org to alert them.
Another example from the site, of M17, is created correctly using the aplpy script shown in the question. Blinking through each filter using DS9, as suggested in the comments, confirms the correct alignment.
Use the STScI software TweakReg to align the images and get a good alignment before combining. More information and worked examples can be found on the DrizzlePac website.
TweakReg:
Combining images using astrodrizzle requires that the WCS information in the headers of each input image align to within sub-pixel accuracy. The tweakreg task allows the user to align sets of images to each other and/or to and external astrometric reference frame or image.
It is optimized for use with HST instruments, but can be adapted for other observatories/instruments.
I'm currently developing a webpage containing a map based on Google Map Api V3 : http://www.fairaidsyria.org
I have two overlays containing polygons that appear successively as the user zooms in..
Unfortunately, annoying "white flashes" occur when zooming. The problem even occurs at max zoom levels, when both overlays are not displayed...
The blinking seems to be directly related to the presence of polygons in the zoomed area, because if you try to zoom somewhere else in the world (area without polygons), everything works fine : no flashes.
I suspect this phenomena to be "native" and not related to a mistake I could have done. In fact, the same flashes also happen when you zoom in the triangle in the following official example : https://developers.google.com/maps/documentation/javascript/examples/polygon-simple?hl=en
Did someone find a way to solve this?
based on this good article change your meta viewport tag to:
<meta name="viewport" content="initial-scale=1.0,minimum-scale=1.0,maximum-scale=1.0">
I set up a Google Maps v3 at our website at http://wallawalla.tk/services and I encounter a minor distortion issue. That is, if you look at the map very precisely, you'll see that there are random "white/gray stripes" appearing. Most important at the very left side of the map and especially when opening InfoWindows respectively closing them. It appears that "white/gray stripes" are somehow related to the border of the InfoWindow, i.e. it seems that if one of the borders doesn't show up (most of the time at the very left side of the respective InfoWindow), there is a random "white/gray stripe" appearing near or even through the InfoWindow. Unfortunately this isn't reproducible 100% of the time which makes it even more suspicious to me. Please try to open/close some of the InfoWindows to actually see it. I guess it is somehow related to our CSS. I read a lot about related distortions but nothing seem to fit my specific case. The "white/gray stripe" on the very left side of the map itself is reproducible almost 100% of attempts trying to reproduce. All "white/gray stripes" are 1px in width and they disappear if you either pan the map or scroll it out and back into the viewport of the Browser window. I'm testing with Google Chrome 22.0.1229.79/Webkit.
Doesn't seem to affect FF11.0/Gecko. Tried it just yet and while Chrome shows described behaviour, FF doesn't
My guess is that it is being caused by OverlappingMarkerSpiderfier. Can you build a simple test that displays your points and InfoWindows without using OMS and see if you still have the issue?
i'm trying to use overlays on a custom google StreetView Panorama.
I was able to simply put one on my pano, but the position is totally random (i made some attempts until the effect was alright), and the change of rendering mode (in chrome which renders as a sphere) changes the effect.
The problem is clear: the custom panorama has no information about its dimensions (it has only a LatLng), so the relative positions are wrong.
Am i missing something?
The only thing i need is to put some clickable labels on the walls of a room, which modify themselves along with the rotating panorama. I searched a lot, but my efforts were inconclusive.
Mapsicle seemed to do what i needed, but if i understood correcly it's a dead project.