ALL ABOUT ROBOTICS!!

Mouse and keyboard control in Matlab:

After red object detection,if you want to do  mouse controls by red object then just follow steps as given below:

1)Get red objects centroid from current frame(Code shared already,see archive).

2)To set mouse cursor use java robot class or you can use inbuilt function in Matlab.

resource:
http://docs.oracle.com/javase/1.5.0/docs/api/java/awt/Robot.html

The following MATLAB code example demonstrates how one can programmatically control mouse motion using the java.awt.Robot class to move the mouse diagonally across the screen. First, import the class into MATLAB, create an object of this type, and then execute the mouseMove method in a loop to simulate motion.

import java.awt.Robot;
mouse = Robot;

mouse.mouseMove(0, 0);
screenSize = get(0, 'screensize');
for i = 1: screenSize(4)
mouse.mouseMove(i, i);
pause(0.01);
end

 

The following example demonstrates how one can programmatically click the right mouse button to bring up the context menu. Again, import the required Java classes, create an object of this type, and then use the mousePress and mouseRelease functions to simulate a click. Before executing this code, place the mouse over a portion of the screen where a context menu can appear.

import java.awt.Robot;
import java.awt.event.*;
mouse = Robot;

mouse.mousePress(InputEvent.BUTTON3_MASK);
mouse.mouseRelease(InputEvent.BUTTON3_MASK);

resoure-http://www.mathworks.in/support/solutions/en/data/1-2X10AT/index.html?solution=1-2X10AT

3)you can set keyboard events by this robot class.Using this events 

a)You can control any application

b)Play any flash game

void

keyPress(int keycode)           Presses a given key.

void

keyRelease(int keycode)           Releases a given key.

Short summary of Robot class(one of my favorite) functions:

void	delay(int ms)           Sleeps for the specified time.
int	getAutoDelay()           Returns the number of milliseconds this Robot sleeps after generating an event.
Color	getPixelColor(int x, int y)           Returns the color of a pixel at the given screen coordinates.
boolean	isAutoWaitForIdle()           Returns whether this Robot automatically invokes waitForIdle after generating an event.
void	keyPress(int keycode)           Presses a given key.
void	keyRelease(int keycode)           Releases a given key.
void	mouseMove(int x, int y)           Moves mouse pointer to given screen coordinates.
void	mousePress(int buttons)           Presses one or more mouse buttons.
void	mouseRelease(int buttons)           Releases one or more mouse buttons.
void	mouseWheel(int wheelAmt)           Rotates the scroll wheel on wheel-equipped mice.
void	setAutoDelay(int ms)           Sets the number of milliseconds this Robot sleeps after generating an event.
void	setAutoWaitForIdle(boolean isOn)           Sets whether this Robot automatically invokes waitForIdle after generating an event.
String	toString()           Returns a string representation of this Robot.
void	waitForIdle()           Waits until all events currently on the event queue have been processed.

You can always see my work in this video in which I played counterstrike with hand gestures at end of video!!!!

|