Related
Hello guess my title is bad enough to explain question but I am trying to understand is there any way to control and automate things just finding tasks from task manager? I have seen in Visual Studio "Spy++". Firstly, i didn't understand what it's aim and how far we can go with it. I just got it, it can provide us logs in a cool range.
I would like to give an example,
I want to log in Facebook/Twitter and do casual things with developed software by myself(I don't want to use selenium or any kind of that thing) or I want to get informations from a game which is about characters actual health, attack power, ability power... or giving command that game from my software like, press a,b or 1.
Can someone tell me, exact subject name of what i am talking about?
Terminology: Selenium / AutoIt: "UI automation". Reading and modifying in-game values: "memory editor" or "trainer".
There is no universal way to control programs if you want your tool to be transparent. A browser may listen to OS input events (Windows messages telling it which keys were pressed or where the mouse was clicked), games may use DirectInput and yet other apps may subscribe to low-level system events or hooks.
For example browser automation:
Using plugins/extensions gives you a JavaScript API that allows you to inspect pages, forms on those pages, modify browser behavior and whatnot.
Browsers can also have their own external API. This can be done by linking to their DLLs, or passing command line arguments, or passing messages in other ways. For Firefox, this API is named "Marionette".
Then there's Selenium, that provides a common API for various browsers. It controls them using "drivers".
Selenium "knows" how to drive a browser, as it's coded against the browser's APIs. Spy++ "knows" that it's inspecting a Win32 window and looks for known controls, their classes and their names so you could write another program to send specific messages to those specific controls of those specific applications.
As for "log in to Facebook", no, you cannot do that in a reasonable amount of time for the currently popular browsers if you want to code it from the ground on up.
You'll have to, in one way or the other, interface with the browser and ask for a handle to the username/password textboxes, enter data into them and then submit the form. Then you'll practically be rebuilding Selenium, so why not use that tool in the first place?
Or you'll have to scrape the pixels on the screen, recognize those textboxes, click the mouse there and send some keys. And then Facebook redesigns their login form and you'll have to start over.
tl;dr: use the right tool for the job. If you want to automate a site's UI, then use Selenium.
I'm trying to remove the default pan behaviour, which is easily done with:
mapView.getGestures().disableDefaultAction(GestureType.PAN);
and use the TWO_FINGER_PAN gesture instead, but I can't quite find a solution for this other than coding the entire animation by myself. Is there an easier way? Maybe some source code I couldn't find?
To summarize I want the TWO_FINGER_PAN gesture to do the exact same thing the PAN gesture would do.
When you disable a gesture, you have to handle the gesture on your side. There is no source code exposed for this as part of the HERE SDK. But you may want to look for native Android gesture handling examples.
The Gestures example app that is shipped with the HERE SDK provides as least a starting point for custom gesture animations with the GestureAnimator class, but for a pan gesture you would usually not need any animation, so a finger movement relates to a change of the target coordinates of the MapView.
Please confirm the the Here SDK version that you looking for? To explain more about the reason this function is not available because, the pan gesture is designed for single finger including kinetic panning. Two finger pan behaves different. So the best user experience is still with normal panning.
Also two finger gestures are also used for tilting and rotating map. To achieve panning with two fingers can be confusing for users.
Detail information about default behaviour in 4.x in explained in the below link:
https://developer.here.com/documentation/android-sdk-navigate/4.7.6.0/dev_guide/topics/gestures.html
I create applications, that are divorced from any native framework. All rendering happens in OpenGL, with a context provided by GLFW, all in C, with no framework to rely on supplying compatibility. As such, standard screen readers like NVDA have no chance of picking up information ( excluding OCR ) and my applications are an accessibility black hole.
How can I provide an interface for screen readers to cling unto? I presume this is a per OS thing... How would that be possible on Windows, Linux, BSD or even android? In the *NIX world, I presume this would be Desktop environment dependent...
I'm finding a lot of information on this, with a framework as a starting point, but have a hard time finding resources on how to do it from scratch.
I'm fully aware this is far beyond the capability of a sole developer and know, that writing programs by ignoring native interfaces is a common accessibility hole, which you are advised to avoid.
However, I have a tough time finding resources and jump-in points to explore this topic. Can someone point me in the right direction?
TL;DR: How to provide screen-reader compatibility from scratch. Not in detail - but conceptually.
As you have already well identified, your app is an accessibility blackhole because you are using a rendering engine.
It's basicly the same for OpenGL, SDL, or <canvas> on the web, or any library rendering something without specific accessibility support.
WE can talk about several possibilities:
Become an accessibility server. Under windows, it means doing the necessary so that your app provide accessible components on demand from UIA / IAccessible2 interface.
Use a well known GUI toolkits having accessibility support and their provieded accessibility API to make your app.
Directly talk to screen readers via their respective API in order to make them say something and/or show something on a connected braille display.
Do specific screen reader scripting
However, it doesnt stops there. Supporting screen readers isn't sufficient to make your app really accessible. You must also think about many other things.
1. Accessibility server, UIA, IAccessible2
This option is of course the best, because users of assistive technologies in general (not only screen readers) will feel right at home with a perfectly accessible app if you do your job correctly.
However, it's also by far the hardest since you have to reinvent everything. You must decompose your interface into components, tell which category of component each of them are (more commonly called roles), make callback to fetch values and descriptions, etc.
IF you are making web development, compare that with if you had to use ARIA everywhere because there's no defaults, no titles, no paragraphs, no input fields, no buttons, etc.
That's an huge job ! But if you do it really well, your app will be well accessible.
You may get code and ideas on how to do it by looking at open source GUI toolkits or browsers which all do it.
Of course, the API to use are different for each OS. UIA and IAccessible2 are for windows, but MacOS and several linux desktops also have OS-specific accessibility API that are based on the same root principles.
Note about terminology: the accessibility server or provider is your app or the GUI toolkit you are using, while the accessibility client or consumer is the scren reader (or others assistive tools).
2. Use a GUI toolkit with good accessibility support
By chance, you aren't obliged to reinvent the wheel, of course !
Many people did the job of point 1 above and it resulted in libraries commonly called GUI toolkits.
Some of them are known to generally produce well accessible apps, while others are known to produce totally inaccessible apps.
QT, WXWidgets and Java SWT are three of them with quite good accessibility support.
So you can quite a lot simplify the job by simply using one of them and their associated accessibility API. You will be saved from talking more or less directly to the OS with UIA/IAccessible2 and similar API on other platforms.
Be careful though, it isn't as easy as it seems: all components provided by GUI toolkits aren't necessarily all accessible under all platforms.
Some components may be accessible out of the box, some other need configuration and/or a few specific code on your side, and some are unaccessible no matter what.
Some are accessible under windows but not under MacOS or vice-versa.
For example, GTK is the first choice for linux under GNOME for making accessible apps, but GTK under windows give quite poor results. Another example: wxWidgets's DataView control is known to be good under MacOS, but it is emulated under windows and therefore much less accessible.
In case of doubt, the best is to test yourself under all combinations of OS and screen readers you intent to support.
Sadly, for a game, using a GUI toolkit is perhaps not a viable option, even if there exist OpenGL components capable of displaying a 3D scene.
Here come the third possibility.
3. Talk directly to screen readers
Several screen readers provide an API to make them speak, adjust some settings and/or show something on braille display. If you can't, or don't want to use a GUI toolkit, this might be a solution.
Jaws come with an API called FSAPI, NVDA with NVDA controller client. Apple also alow to control several aspects of VoiceOver programatically.
There are still several disadvantages, though:
You are specificly targetting some screen readers. People using another one, or another assistive tool than a screen reader (a screen magnifier for example), are all out of luc. Or you may multiply support for a big forest of different API for different products on different platforms.
All of these screen reader specific API support different things that may not be supported by others. There is no standards at all here.
Thinking about WCAG and how it would be transposed to desktop apps, in fact you are bypassing most best practices, which all recommand first above anything else to use well known standard component, and only customize when really necessary.
So this third possibility should ideally be used if, and only if, using a good GUI toolkit isn't possible, or if the accessibility of the used GUI toolkit isn't sufficient.
I'm the autohr of UniversalSpeech, a small library trying to unify direct talking with several screen readers.
You may have a look at it if you are interested.
4. Screen reader scripting
If your app isn't accessible alone, you may distribute screen reader specific scripts to users.
These scripts can be instructed to fetch information to give to the user, add additional keyboard shortcuts and several other things.
Jaws has its own scripting language, while NVDA scripts are developed with Python. AS far as I know, there's also scripting capabilities with VoiceOver under MacOS.
I gave you this fourth point for your information, but since you are starting from a completely inaccessible app, I wouldn't advise you to go that way.
In order for scripts to be able to do useful things, you must have a working accessible base. A script can help fixing small accessibility issues, but it's nearly impossible to turn a completly inaccessible app into an accessible one just with a script.
Additionally, you must distribute these scripts separately from your app, and users have to install them. It may be a difficulty for some people, depending on your target audience.
Beyond screen reader support
Screen reader support isn't everything.
This is beyond your question, so I won't enter into details, but you shouldn't forget about the following points if you really want to make an accessible app which isn't only accessible but also comfortable to use for a screen reader user.
This isn't at all an exhaustive list of additional things to watch out.
Keyboard navigation: most blind and many visually impaired aren't comfortable with the mouse and/or a touch screen. You must provide a full and consist way of using your app only with a keyboard, or, on mobile, only by standard touch gestures supported by the screen reader. Navigation should be as simple as possible, and should as much as you can conform to user preferences and general OS conventions (i.e. functions of tab, space, enter, etc.). This in turn implies to have a good structure of components.
Gamepad, motion sensors and other inputs: unless it's absolutely mandatory because it's your core concept, don't force the use of them and always allow a keyboard fallback
Visual appearance: as much as you can, you should use the settings/preferences defined at OS level for disposition, colors, contrasts, fonts, text size, dark mode, high contrast mode, etc. rather than using your own
Audio: don't output anything if the user can't reasonably expect any, make sure the volume can be changed at any time very easily, and if possible if it isn't against your core concept, always allow it to be paused, resumed, stopped and muted. Same reflection can apply to other outputs like vibration which you should always be able to disable.
I'm working on a standalone Flash application (written using Flex 3/ActionScript 3) that features a text crawl, like what you might see at the bottom of your TV when watching a cable news channel; it's a long narrow box that text moves across from right to left.
I've implemented it by creating a Label element, populating it with text, and then moving it using a mx:Move object with a Linear.easeNone easing function. It works, but it has ample room for improvement. It looks a bit jerky, and tends to have a fair amount of "tearing" (the top and bottom halves of the text sometimes fall out of sync).
I tried throwing math at the problem to get the crawl's movement rate synced with the monitor's refresh rate, but that was a bust. I found out the hard way that the app's frame rate jumps around too much; the "optimized" crawl varied between looking silky smooth and like it had epilepsy.
Is there anything else folks would recommend I try to smooth this thing out? Is there some alternate design you'd recommend I try?
Edit: Some context: the crawl is part of a digital signage application (played from a standalone Flash projector -- no web browser) that does stuff elsewhere on the screen, including video playback and rendering text and images. It definitely gets choppier during video playback, but it's never as smooth as I'd like it to be.
There are two potential solutions to this problem, but both have caveats, the first because of your use of Flex and a standalone projector, the second because it is a mitigator, not a complete solution.
Hardware Acceleration
When publishing your file, you can attempt to have Flash utilize hardware acceleration to alleviate the vertical refresh issue you are running into that is causing tearing. Sadly, Flex Builder 3 is incapable of enabling this setting at the SWF (projector) level (Link to bug). This has yet to be resolved and has been pushed from 4.0 to 4.1 to 4.x... If and when it is resolved, it will likely be a compiler argument in the project settings of Flash Builder 4.
You may be able to determine if this solution works for you by outputting your projector as a standard SWF and embedding it on an HTML document with the wmode set to "direct" or "gpu". Sadly, if it does (it should), you can't use it right now anyway. If you have Flash Builder 4, certain projects are capable of making round trips between FB4 and Flash Professional CS5, though I am not sure what the criteria for that is (my current AIR project has all the project modification menu options grayed out). If you do manage to get your project into Flash, you can enable hardware acceleration in the Publish Settings of the project (File->Publish Settings->Flash tab->Hardware Acceleration option in CS5).
This method is almost a certain solution for your problem, though it has two issues, one already highlighted above, and (for people publishing for the web) that by utilizing direct or GPU rendering on a webpage, you are unable to layer any DOM elements on top of flash.
direct: This mode tries to use the fastest path to screen, or direct path if you will. In most cases it will ignore whatever the browser would want to do to have things like overlapping HTML menus or such work. A typical use case for this mode is video playback. On Windows this mode is using DirectDraw or Direct3D on Vista, on OSX and Linux we are using OpenGL. Fidelity should not be affected when you use this mode.
gpu: This is fully fledged compositing (+some extras) using some functionality of the graphics card. Think of it being similar to what OSX and Vista do for their desktop managers, the content of windows (in flash language that means movie clips) is still rendered using software, but the result is composited using hardware. When possible we also scale video natively in the card. More and more parts of our software rasterizer might move to the GPU over the next few Flash Player versions, this is just a start. On Windows this mode uses Direct3D, on OSX and Linux we are using OpenGL.
**Source*
Direct is the ideal option for this situation, as you can actually have performance degredation with "gpu" as well as visual differences from graphics card to graphics card.
Lower your framerate
The Flash player will continue to play video at its native refresh rate independent of the rest of your project as long as you keep the framerate at or above approximately 2FPS (though I suggest 5FPS minimum). You won't want to run that low for this example, but you are able to lower the framerate of the entire scene without impacting video performance. The closer your framerate is to the screen refresh rate, the more apt you are to actually create the tearing effect unless you are able to absolutely sync with the monitor's refresh rate, which you probably cannot do without the above... Hardware Acceleration.
This problem has existed in the Flash Player for as long as it has been able to move objects horizontally. What happens is that Flash updates a buffered snapshot of the running animation at the same time that the screen is refreshing. If the buffered snapshot changes partway through a screen refresh, you get a tear. This is why lowering the framerate actually reduces the amount of tearing, you are refreshing the buffer less frequently.
As #Tegeril mentioned, using Flex is one of the reasons. Flex is a pretty heavy framework and it does a lot of things behind the scenes. If you're familiar with the life cycle of a component(especially invalidating properties, invalidating the display list, etc.).
As a few minor things that might improve performance:
try to keep a simple display list. If you know the app will always be displayed at one size, then flex won't waste time traversing the display list/tree up to the top and back for measurements. Also, try to use a Canvas. I know, it's not very clean, but since it uses absolute values and doesn't check with the 'parents' much, it should be faster than other containers(like HBox,VBox, etc.)
try to display the video at it's full size(make sure the encoded video dimensions are right so there be any CPU cycles on resizing video
Ok, this was Flex stuff.
It might be very handy to read sencular's article on Asynchronous ActionScript Execution which explains how Flash Player handles updates and renders.
(source: senocular.com)
Frames both execute ActionScript and render the screen
(source: senocular.com)
ActionScript taking a long time to complete delays rendering
I imagine the jerkiness is related to this. Also, I'm guessing you might
get moments of smooth movement then sudden halts, every now and then, when
Flash Player catches it's breath(Garbage Collector cleans up)
Victor Drâmbă article on “Multithreading” in Actionscript might also
be useful.
Soo, to recap:
use Profiler or something and see if the Flex framework is slowing you down, or where the 'bottleneck' is
improve as much as you can on that side then check if it's how Flash Player handles all the actionscript('elastic' frames)
If the bottleneck comes from the Flex framework, worst case, you
can try to minimise the number of components that traverse the display list,
and use pure actionscript for the other things(as #PatrickS suggested, use TweenLite, etc.)
If it helps, try to preload data(fetch rss feed and all that) at the start, and when you've got most of the important bits that don't require 'refreshes'/loads frequently, display the app. You will use more memory, but will have more cpu cycles to spare for other tasks.
Also, if it's display objects that are the 'bottleneck' and there's plenty of them, check if you can reuse them using Object Pools.
HTH
TweenMax or even TweenLite ( http://www.greensock.com )handles this sort of job pretty well. What else is your app doing while the text is scrolling though? Is it possible that some other processes are interfering?
This may not be helpful, but have you considered putting the crawling text into the html DOM and using CSS transitions to crawl the text. Obviously there's the IE problem, but it should be supported in IE9 and you could use javascript as a fallback.
This may seem silly, but CSS transitions are getting hardware acceleration and separate processes for plugins meaning on a multicore machine you could get parallel threads.
One thing you might consider is to move your label incrementally using a Timer instead of an easing function. That way you can take advantage of the updateAfterEvent method to get smoother rendering. Here's a link to an article/video from Chet Haase (Adobe's Flex graphics dude) that explains usage along with an example app with code:
http://graphics-geek.blogspot.com/2010/04/video-event-performance-in-flex.html
Hope that helps.
I've chosen Flex 4 as the most appropriate technology to develop a graphically-rich web application (its not a simple content-driven site), but worried about how the recent negative press (i.e. security issues) may effect end-user's trust and ultimately whether the user-base may drop promptly in response. (I don't care if my app works on iphones or ipads for now)
I think Flash Builder 4 is an great development environment and has minimized development time for me/my team. After some basic testing of graphical animations similar to that used in my app - HTML5 didn't perform as fast, is inconsistent with browsers, and some animations are jagged (I expect browser performance and graphic libraries to improve over time). I also 'personally' dislike programming Javascript as I am very fond of strong-typing to uncover mistakes quickly.
If you develop Rich Internet Apps, how are you responding?
Are you preparing to potentially migrate to HTML5/Javascript? Java? No action?
BTW - I don't want pro/anti-flash arguments - just curious to see how the community is responding.
At the end of the day, Flash/Flex aren't going anywhere. If Flex 4 meets your current needs and you're aware of the limitations (ie can't deploy to iOS devices) then I say go for it. Yes it's true that the topic has become mildly politicized - but if you're offering something your clients need then they'd be silly to refuse to use it on the grounds that they support "HTML 5" - when HTML 5 clearly doesn't offer you the tools you need.
Plenty of awesome stuff is coming down the pipe in Flash, much of which simply can't be done any other way - google UJam for an example. I wouldn't let Steve Jobs scare you away from using the technology that works for your needs.
My company plans to continue with Flash, using FlashBuilder 4 and Java back end. We went with Flex/Flash several years ago to get out of the business of supporting all the different browsers and into the business of being productive and giving our users a rich client-side experience.
HTML5/Javascript have potential, but are nowhere near as robust, powerful, fast, or efficient. The class hierarchy, data typing, and event model alone put ActionScript 3 miles beyond any Javascript. So what if Steve Jobs gives Flash the thumbs down? Time-Warner and other big media companies have said they're going to continue with Flash, so it's only a matter of time before Steve Jobs either relegates Apple to permanent niche status or caves and allows Flash on Apple products. (My guess is for the immediate future he will prefer niche status to admitting he is wrong—look how long he maintained a mouse only needed a single button?—but that's just my opinion.) In any case, Flash will soon be available on a multitude of smart phones, including the Droid, so I am not worried.
Adobe will provide tools to convert to HTML5, but they are already following the HTML5 Path with some introductory tools. Just keep watch on adobe. They know what is going on. They just killed mobile flash so even though they argued with apple over it they finally did the right thing instead of stupidly holding on to it just because... hope that helps
I'm a Flex developer, but I think HTML5 is going to be huge. The full features of HTML5 are years away, and I don't think it's totally going to kill Flash. Flex will hold on to some part of the RIA market because it has a lot more going for than just a de facto standard client plugin -- LCDS/BlazeDS, plays nicely with ColdFusion and Java.
I like Flex for the long run. It'll lose some ground to HTML5, but there are areas where Flex will hold its advantage.
Disclaimer: I am author of Web Atoms JS
Flex/Flash is dead already, as usage of non PC devices is increasing everyday. Except old IE (IE<10) almost all features of Flash are already offered by browsers. File API, AJAX upload with progress bar,Canvas API, Indexed DB, Cross Domain message API & Web Sockets. And CSS3, WebGL with 3D can give flash like graphics.
Regarding Component Library & Binding, HTML5+JS lacks component driven development that flash offers. To bridge this gap, we created framework that gives similar functionality with all components to that of flex. Look at following image & see this blog which outlines similarities between Flex & Web Atoms JS.
http://akashkava.com/blog/439/migrating-from-flex-to-html5-with-web-atoms-js/
Here is link to documentation.
http://webatomsjs.neurospeech.com/docs