I am trying to convert a mouse event to pixel coordinates within a video. By pixel coordinates, I mean coordinates relative to the original video resolution.
My video element has object-fit: contain, which means that the top left corner of the video is not necessarily located at position (0,0), as this picture shows:
If I click on the top-left corner of the white section in this video then I want to get (0,0), but in order to do this I need to discover the offset of the video content (white area) relative to the video element (black border).
How can I recover this offset?
I am already aware of width, height, videoWidth, and videoHeight, but these only let me account for the scaling, not the offset.
The offset can be deduced. I think this kind of code should do the trick:
if(videoHeight/height > videoWidth/width){
scale = videoHeight/height;
offsetX = (videoWidth - width*scale)/2;
offsetY = 0;
}
else{
scale = videoWidth/width;
offsetY = (videoHeight - height*scale)/2;
offsetX = 0;
}
I was also interested in getting the actual pixel positions from mouse or touch events when using object-fit, and this is the only result I found when searching. Although I suspect it is probably too late to be helpful to you, I thought I'd answer in case anybody else comes across this in future like I did.
Because I'm working on code with other people, I needed a robust solution that would work even if someone changed or removed the object-fit or object-property in the css
The approach that I took was:
Implement the cover, contain etc algorithms myself, just functions doing math, not dependent on the DOM
Use getComputedStyle to get the element's objectFit and objectPosition properties
Use .getBoundingClientRect() to get the DOM pixel size of the element
Pass the element's current objectFit, objectPosition, its DOM pixel size and it's natural pixel size to my function to figure out where the fitted rectangle sat within the element
You then have enough information to transform the event point to a pixel location
There's more code than would comfortably fit here, but getting the size of the fitted rectangle for cover or contain is something like:
if ( fitMode === 'cover' || fitMode === 'contain' ) {
const wr = parent.width / child.width
const hr = parent.height / child.height
const ratio = fitMode === 'cover' ? Math.max( wr, hr ) : Math.min( wr, hr )
const width = child.width * ratio
const height = child.height * ratio
const size = { width, height }
return size
}
// handle other object-fit modes here
Hopefully this gives others a rough idea of how to solve this problem themselves, alternately I have published the code at the link below, it supports all object-fit modes and it includes examples showing how to get the actual pixel point that was clicked:
https://github.com/nrkn/object-fit-math
Related
Is there a way to preserve image proportions when using an tag?
The docs mention hard coding the dimensions but that's problematic when requesting arbitrary images
https://aframe.io/docs/0.6.0/primitives/a-image.html
as far as i see, the a-image is just a a-plane with an image source in a material.
It means the image will be streched over the plane, you can only mess with the
<a-image> height and width, which are the a-plane's height and width in meters,
<img> height and width, which specify the image dimensions in px, but the image still will be streched over the plane.
You could try to do it automatically, within a component, setting the a-planewidth and height depending on the input <img> width and height, having a px-meters ratio:
AFRAME.registerComponent('imageSetter',{
schema:{
img:{type:'selector'}
},
init:function(){
let lowResRatio = 0.01;
this.el.setAttribute('height',this.data.img.height*lowResRatio);
this.el.setAttribute('width',this.data.img.width*lowResRatio);
}
});
Check it out in my fiddle: https://jsfiddle.net/gftruj/5d9j9nqm/2/.
Otherwise, You need to prepare the images earlier on.
UPDATE
You can modify the component so it gets the image dimensions, gets a height/width ratio, and sets the width and height accordingly:
AFRAME.registerComponent('foo',{
schema:{
img:{type:'selector',default:''},
height:{}
},
init:function(){
let data = this.data;
let ratio = 1/100;
this.el.setAttribute('width',data.img.width*ratio);
this.el.setAttribute('height',data.height);
this.el.addEventListener('materialtextureloaded', (e)=>{
var w = e.detail.texture.image.videoWidth || e.detail.texture.image.width;
var h = e.detail.texture.image.videoHeight || e.detail.texture.image.height;
let ratio = w/h;
this.el.setAttribute('width',data.height*ratio);
});
}
})
live fiddle here: https://jsfiddle.net/5d9j9nqm/4/
Now it's fixing itself to a given height.
I am required to create text items with exact width and height of text contents.
The height of the text is the most important requirement.
The position of the text should be relative to the text itself.
I also have to be able to place it on canvas in an exact spot.
Assuming a (printable) canvas (on a larger QGraphicsScene), say 5 inch width and 1 inch height, my text should be able to stretch top-bottom-left-right - and be placed on canvas, not part in part out.
I am sub-classing QGraphicsTextItem for my item type. I am resizing it, using QTransform(), to required size - in inches or mm or pixels (72*in).
Also setting the document() margin to 0, and anything inside (like QTextBlockFormat margins) also to 0.
I have implemented a setItemSize(QSizeF sz) (with sz in pixels), that resizes the QGraphicsTextItem as required.
The sz is initialized using the item bounding rect.
Assuming no wrap, single line text (multi-line could be solved separately once this issue is resolved).
When adding the item to canvas, I still see a top and bottom margin - and this varies based on font choice.
I drew a rectangle around the item to see it.
The top/bottom distances depend on font choices.
I have tried to use font metrics to determine these distances (in paint() I have been drawing lines to try to determine the position and rectangle in which the text fits).
I would be happy to at least be able to determine correct size to use for upper case, no accents or special characters fonts (it would be a start, though naturally I would need to be able to use any characters).
But at least some way to determine the size and position (relative to the (0,0) of item) of the text content even in the simplest case.....
The font metrics tightBoundingRect() seems the most accurate for size, but it seems impossible to determine its position so that I can somehow create my items correctly, and maybe resize/shift them correctly to fit on canvas.
Here are some examples of my struggle to determine at least exact size and position of text, relative to the (0,0) of the item (assuming that once I do that, I am able to expose that info to outside or include the shift in the item transform on resize).
Notice that the size of the text advertised by font metrics does not always cover the text, and for different fonts I am not able to position the tight bounding rect (magenta) around the text itself. (I did multiple guesses, the code below is just one - the lines are trying to show different font metrics sizes).
The above were experiments in paint function of the text item inheriting QGraphicsTextItem:
void paint(QPainter* painter, const QStyleOptionGraphicsItem* option, QWidget* widget)
{
// draw text
QGraphicsTextItem::paint(painter, option, widget);
QPen p;
p.setWidthF(0);
QFontMetricsF fm(this->font());
qreal ascent = fm.ascent(),
descent = fm.descent(),
hheight = fm.height();
QRectF r = QGraphicsTextItem::boundingRect();
QRectF rFont= fm.tightBoundingRect(toPlainText());
qreal xmax = r.right();
painter->save();
painter->setBrush(Qt::NoBrush);
// where is "ascent + descent"
p.setColor(Qt::green);
painter->setPen(p);
painter->drawLine(QPointF(2, ascent), QPointF(2, ascent + descent));
painter->drawLine(QPointF(2, ascent + descent), QPointF(xmax/2, ascent + descent));
// where is "height"
p.setColor(Qt::red);
painter->setPen(p);
painter->drawLine(QPointF(xmax/2, 0), QPointF(xmax/2, hheight));
painter->drawLine(QPointF(xmax/2, ascent + descent), QPointF(xmax, ascent + descent));
// where is "ascent"
p.setColor(Qt::yellow);
painter->setPen(p);
painter->drawLine(QPointF(6, 0), QPointF(6, ascent));
painter->drawLine(QPointF(6, ascent), QPointF(xmax, ascent));
// something that may look like top of the text
p.setColor(Qt::blue);
painter->setPen(p);
qreal yyy = ascent + rFont.y() + 1;
painter->drawLine(QPointF(5, yyy), QPointF(xmax, yyy));
// this should be useful... should be the natural offset
qreal yoffset = (r.height() - rFont.height()) / 2;
// qDebug() << yoffset << r << rFont;
//qreal y0 = (r.height() - fm.height())/2;
p.setColor(Qt::darkGreen);
painter->drawEllipse(10, yoffset, 1, 1);
// where is the font rect
p.setColor(Qt::magenta);
painter->setPen(p);
yoffset = (r.height() + rFont.height()) / 2;
painter->translate(0, yoffset);
painter->drawRect(rFont);
painter->restore();
}
I have also tried not using QGraphicsTextItem, but paint text inside a rectangle. The same thing happens.
(Qt 4.7 - 5.x)
This is not a good solution. This is an attempt to solve my own problem - in a first iteration - of setting text with given width and height, using font metrics.
Reasons for not being good -
Even after resize, text is smaller than desired, I don't understand why
The position is incorrect, based on font style the text can be above or below the canvas, meaning it gets clipped.
I resize it using a factor calculated from the item bounding rect size and the font metrics bounding rect (I used the tight bounding rect for more accurate size).
myText->setItemFontSize(12); // If I use font metrics I need to reset text size on every change, because resizing loses font info
QFontMetricsF fm(myText->font());
QRectF fmRect = fm.tightBoundingRect(myText.toPlainText().toUpper());
// without toUpper() the size is too small - even so it is a bit small
// I read tightBoundingRect is slow - but boundingRect and height and ascent all give values that result in even smaller size
//qreal absH = fm.ascent();
qreal absH = fmRect.height();
qreal absW = fmRect.width();
qreal absHeightRatio = myText->getItemSize().height() / absH;
qreal absWidthRatio = myText->getItemSize().width() / absW;
Then setting size:
myText->setItemSize(QSizeF(absWidthRatio * textLength, absHeightRatio * fontHeight));
// This function scales the `QTransform` on item
// but since I request a final position dependent on item size
// (including blank space around it) - it has no chance of being accurate.....
// This is where my next effort will go, figuring out how to get rid of the fluff
// around the item inside the scaling
The function for setting position: trying to center text:
// leftShift = 10, rightShift = 10 in example
myText->setPos(0,0);
QRectF r = myText->mapToScene(myText->boundingRect()).boundingRect();
QSizeF sz = r.size();
qreal w = sz.width();
qreal h = sz.height();
qreal cx = (m_docLength - w + leftShift - rightShift)/2 - r.left();
qreal cy = (m_docHeight - h)/2 - r.top();
myText->setPos(cx, cy);
The images below are for fontHeight = m_docHeight -
Desirable:
- either the entire size of text (ascent + descent) equals doc height, and text is centered vertically based on content
- or the upper case size text equals doc height and the descent is below document (text centered based on only upper case) - this would seem easier based on how QGraphicsTextItem seems to position it
Actual:
- the text is smaller no matter which parameters I use to scale, and centered based on upper case text
As shown above - I have no idea how I could center vertically based on content (so for edge-to-edge text the descent would fit in) - and in lieu of that, all I really want is edge-to-edge uppercase letters, but I can't seem able to achieve that.
Oh and these are for Arial type font - one of the best-behaved. Other fonts jump all over the place, either above or below the canvas. And for some fonts, the resulting text is actually smaller - which is inexplicable to me, because how can the tight bounding rectangle be smaller than the item bounding rectangle...
Still this is as far as I got to getting my text as close to "true" size and placed on a canvas that matches its size.
My LineItem inheriting from QGraphicsLineItem can change its pen width.
I have created a boundingRect that uses the QGraphicsLineItem::boundingRect adjusted by pads that get calculated based on pen width and arrows. It works.
void LineItem::calculateStuff() // called on any change including pen width
{
qreal padLeft, padRight, padT;
padLeft = 0.5 * m_pen.width(); // if no arrows
padT = padLeft;
padRight = padLeft;
m_boundingRect = QGraphicsLineItem::boundingRect().adjusted(-padLeft, -padT, padRight, padT);
update();
}
QRectF LineItem::boundingRect() const
{
return m_boundingRect;
}
QPainterPath LineItem::shape() const
{
QPainterPath p;
p.addRect(m_boundingRect);
return p;
}
There is only one artifact that I get:
if I increase the pen width, then decrease it, I get traces:
these of course disappear as soon as i move mouse or any action (I had a hard time getting the screen shots)
As pretty as they are (seriously I consider them a "feature :-) ) - I am trying to eliminate them. I tried to remember previous bounding rectangle, and update the item with the previous bounding rectangle - i thought that was what the option was for - but it didn't work.
QRectF oldRect = selectedItem->boundingRect();
item->setItemPenWidth(p);
selectedItem->update(oldRect);
selectedItem->update();
My viewport has
setViewportUpdateMode(BoundingRectViewportUpdate);
If I change to
setViewportUpdateMode(FullViewportUpdate);
I don't get artifacts - but I think this will impact performance which is a major constraint.
How can I fix these artifacts - that only occur in that specific situation, decreasing pen width / decreasing bounding rect of line, without impacting performance ?
Simple fix... I had to add
prepareGeometryChange();
in my calculateStuff() function.
I have not seen any changes from this before, it is the first time I change my boundingRect that it does not update seamlessly.
The Flex Canvas container is limited to 10,000x10,000 pixels. Yet, I've seen Flex apps that are scrolling way more than 10,000 pixels. Any idea how this can be done?
The content I want to scroll is in pieces already, but I need to add those pieces to something that can scroll vertically more than 10,000 pixels.
Depending on what you actually want to display you may be able to split your content into tiles. This is how Google Maps works, every time the map is moved the program determines which tiles are visible on the screen and loads them in. Any markers or overlays that are on the map are notified that the map has moved and determine where their new location is. If their location is off the screen they can be removed from the canvas. For example, the width of all the tiles at zoom level 20 on Google Maps is (256 pixels * 2^20) which equals 268,435,456 total pixels.
Essentially you just need to create a special Sprite that keeps track of the actual x,y location it is supposed to be positioned at. Any time the container moves you simply iterate through all of the child objects and determine where to put them.
function onCanvasScroll() {
//determine the top left coordinates of the canvas
//you will figure out how to do this depending on how the scrolling window
//is implemented
var canvas_scroll_x;
var canvas_scroll_y;
//create a bounding box for the canvas
var view_bounds = new Rectangle(canvas_scroll_x, canvas_scroll_y, canvas.width, canvas.height);
for (child in canvas) {
var x = child.actual_x - view_bounds.x;
var y = child.actual_y - view_bounds.y;
var childBounds = new Rectangle(x, y, child.width, child.height);
//determine if the component is visible on screen
if (view_bounds.intersects(child_bounds)) {
child.visible = true;
child.x = x;
child.y = y;
}
else {
child.visible = false;
}
}
}
So if you have a canvas that is positioned at (100, 20000), a sprite that is positioned at (300, 20100), and a window that is (640,408), you would place it at (200, 100) and it would be visible on the screen.
Instead of just setting visible to true or false a better approach would be to remove the items from the canvas entirely when they are not within the bounds of the view.
Ok, so I've been trying to get this concept to work for the day now and have had zero luck. I am not so good with math, so any help will be appreciated. I am trying to rotate a centered container from it's center. The problem with this code is when I use the rotatePicture method, it doesn't rotate from the center, instead it rotates from the box's top-left corner. Here's the code...
import mx.effects.Rotate;
private function init():void
{
calculateCoordinates();
}
private function rotateBox():void
{
var m:Matrix = myBox.transform.matrix;
var centerX:Number = myBox.width / 2;
var centerY:Number = myBox.height / 2;
var centerPoint:Point = new Point(centerX, centerY);
var transformPoint:Point= m.transformPoint(centerPoint);
m.translate(-transformPoint.x, -transformPoint.y);
m.rotate(90 * Math.PI / 180);
m.translate(transformPoint.x, transformPoint.y);
myBox.transform.matrix = m;
this.callLater(calculateCoordinates);
//calculateCoordinates();
}
private function calculateCoordinates():void
{
var x : Number = (myBox.parent.width - myBox.width) / 2;
x = x < 0 ? 0 : x;
var y : Number = (myBox.parent.height - myBox.height) / 2;
y = y < 0 ? 0 : y;
myBox.move(x, y);
}
Ok, this was a bit tricky and i'm working out a couple of details, but in case anyone had a similar issue, I found the general answer. Just took a movie break to refresh the brain...
I had to place a variable for how many turns the canvas had rotated which was easy since I was restricting the degrees to only 90. Then I place in a switch statement that tested the turns variable and recalculated the (x,y) coordinates based off of the turns. Since I knew that the Rotate class would create a cooler effect and end with the same result, I ended up using that instead.
Cheers!
I don't know what your background is, but in my experience this is a classic case of "out-thinking yourself".
You've written quite a bit of code to do something that is actually native to the Flash display API's.
Since you seem to be using Flex I'll just tell you that the simple way to achieve this is to dynamically reposition your display clip contents so that the center of your contents is at the 0,0 point of your clip.
This gets harder the more content you have, but if you just have something like an image or a button or what have you, it's really easy to just calculate the height and width, then divide by 2 and subtract.
Then the rotation property will work just fine.
In Flash it's even easier because you can just make a new clip, bind your class to the clip, and place all yours contents in the Flash authoring tool positioned properly for rotation to work as expected.
Yeah, what Back2Dos said.
Personally, I'd wrap the container in another sprite, position it so its center is at (0,0) in that sprites coordinate space, and then just rotate the sprite ... it's simple and robust ...
I'd like to use <s:Rotate> to rotate center. Hope useful to you.