Demonstration number one

Today I made a little demonstration of the final robot API and the Lego Robot.
Everything approximately worked as it should, that's to say the connection with the camera and the robot was never lost during the simulation ( sometimes the wireless connection with the camera is lost for some reason... ).
Just a few times, robot and balls were not detected correctly. It uses to happen in some cases, for example when a ball is too near to a hoop, or under the hammer.

But, in a simple situation, the AI worked well, and almost hit the ball in the good direction.
The manual control of the robot didn't cause any unexpected bugs neither.

I didn't have the time to write them this week, but I will probably publish the three last features news on the blog on Monday, about :

• Image Processing - Part 2/2
• API Control Panel
• Platform / Reality Calibration

Then, I'm going to spend the two last weeks on documentation and report.
So, they'll probably be available on my google docs by the end of the placement.

AI - Part 2/2

While moving, the AI can now avoid obstacles by finding a new path to its final position.
It still uses the controlRobot() method which make it rotate to its destination, move to it, rotate again to reach the ball, and hit it.
However, now, the robot checks collisions in real time, each time, before moving ( rotating is not concerned, as it can't be the source of collisions ). When a collision occurs, the movement is stopped.
Then, the AI avoids the obstacle : 

  

In order to do this, the first step is to get the difference angle (purple arrow) between angle made with obstacle ( black line ) and robot's direction angle ( dotted red arrow ).

 The objective is to avoid the obstacle with an angle of 90° ( that's to say move in the direction of cyan arrow ). The blue angle to rotate is calculated using the difference angle calculated before and its signum.

At this point I simply move the robot using a constant small distance.






To finish, I call again the controlRobot() method I spoke about on the beginning.
This way, the method can be called recursively if there are many obstacles on the way.

Problem : At this point there is still problematic cases that can block the AI to an infinite loop :

If the robot want to reach the ball on the other side of the hoop,here it will collide one hoop foot, avoid it, collide the other foot, and again and again, as it is too big to cross in the middle.
This cans also happen if there is not enough space between two balls.




Moreover, even if almost of the time the robot reach the ball, sometimes the AI needs to recalculate the path more times than it seems to be actually needed...

Key and Mouse Controls - Part 2/2

 

• Move, Rotate and Hit speeds can now be controlled using sliders : when values are changed, the text field under the slider and the control panel on the top are automatically updated.
• The 3 commands are now identified by a color on buttons and sliders for better ergonimics.
• For ergonomics reasons as well, the basic control panel (on the top of screen) is isolated from croquet additions.
• Key and mouse controls don't have any effect on the robot when the AI is playing.
• Because the AI uses to change the Hit Speed value, the slider is updated at the beginning of each turn.
• Keyboard and mouse now work the same way : mouvements start when you press/click, and stop when you release key/button.

Image Processing - Part 1

As I said in an older news, I'm currently working at the same time on these two features, that's why I'm also writting this news.

So, the Image Processing is done using ImageJ Java library.
This library contains a software to process images I used to know how to do.
On a Java program, you can use ImageJ plugins, or "macros" files to directly use the tools from the software.
Even if macros are more limited than plugins, they're also more simple to use, so I've started with this.

As I'm currently working with only one set of pictures, sunlight changements may make the program not to work.
By the way, this set of picture is available on my google docs.

So, in order to simplify Image Processing, it will be divided in two parts :

- First, on the beginning only, locate hoops and pegs on a capture with no balls, no robot.
As these elements aren't supposed to move, you don't need to recalculate positions each time.
However, if they accidentally move, a button will enable to recalculate them.

-Then, each time you need it, you will calculate position of balls, and position + direction of the robot.

I'm basically using Binary images, with Erosion/Dilatation/Open/Close/Reconstruction operators, as I've already used them at school and they seem to fit there.

I - Hoops & Peg

• Detection

 The camera enables enough contrast between board and elements so that Find Edge command is useful.
Then, a Binarization gives the third picture.

The first thing to remove are sides.
For this, as hoops are supposed to be far enough from sides, I'm using a home-made mask to remove sides.

 

The last thing to remove are the "cracks" between the two parts of the board.

First, I use Fill Holes command to make hoops and peg bigger ( cracks are not circles ).

Then, an enough Erosion gives the first image.

Depending on the quality of the shot, peg or hoops may disappear as well, so erosions must be done carefully.

Then, using a reconstruction ( Dilatation + MIN operator) gives a cleaned image.

Then, I fill holes again, and add a dilatation to increase the chance of having 7 related elements.

A XOR 11111111 reverse bit values, in order to get positions with Find Maxima values.

ImageJ software enables users to save results in a text file. This is what I'm doing with my macro. 

Then, using file streams, I get my 7 points.

• Identification

At first, I was worrying about this, but the solution was easy :

If you position elements like in the classic game, and if you keep the camera on the same place, the board will always be organized more or less like on this picture.

So, with two levels of sort, you can identify all your elements :

{1, 2, 3, 4, 5, 6, P}

• A first sort using Y coordinates of points will divide the elements in 3 groups.

{3, 4} ; { 5, 6, P } ; { 1, 2 }

•  Then one sort using X coordinates in each group allows you to identify each element :

{3} ;  {4} ; {5} ; {P} ; {6} ; {2} ; {1}

Knowing the ratio pixels/cms, you can then determine the position of hoops and peg.



II - Robot and Balls


For the moment I'll only speak about the major interest of dividing the process :
If you save the complete binarized image of hoops, pegs, sides and "cracks", you can then substract it from a binarized picture with Balls and Robots : this will remove most of the undesirable shapes.

AI - Part1

A player cans now be controlled by the computer.
This is still a low level AI, as its only strategy is to reach hoops.
Altough a player can choose the robot to use, the AI automatically pick the one identified as "Thor" from the hashtable ( so be sure it does exist before using the AI ).
For the moment, the AI doesn't try to roquet, and neither to avoid sides of hoops or peg before the end.
Moreover, the robot currently moves straight to the position, without checking if it will collide other elements.

How to strike the ball

1. Angle of hit

The first step is to know which angle you'll hit the ball with. As you know position of your ball and hoop to reach, this is a trigonometric calcul as usual.






2. Find new position of robot

Then, you have to place the robot in the good position for your hit. As the ball rolls in the same direction wherever your hammer is, the objective is to place the hammer on the same position as the ball. So, as we know the relative angle and the relative distance of the hammer from the center of robot ( = center of circle), this can be done.





3. Control the robot

Basically, and for the moment, the control consists in 4 commands :
-Rotate the robot to look in the direction of the new position.
-Move the robot to this position.
-Rotate the robot to have the hammer on the ball.
-Hit the ball.
However, during the move, collisions will have to be checked, in order to recalculate a new path each time they actually occur.

4. Determine the initial Speed

 The acceleration of the ball A is constant ( roll without slipping hypothesis ), and as you know the distance D between ball and hoop, you can calculate the initial speed thanks to fundamental principles of dynamics :


speed(0) = sqrt(- A * 2D )

Here, I've increased D enough so that the ball goes a bit more farther than the hoop. 

---------

The AI actually works but the remaining bugs will have to be corrected before, I think, trying to think about a strongest AI than "hoops reaching" strategy.

Key and Mouse Controls

Robot can now be controlled using keyboard or mouse, but only on the croquet application (not the "basic" platform). This could be added, but you will have to check in the code that the active robot can move/rotate/hit.

Keyboard :
Two set of keys can be used :

• Traditional Arrows (Move/Rotate) + Space (Hit)
• Numeric pad using 2,8 (Move), 4,8 (Rotate), 5 (Hit)

Depending on the direction you're moving (forward/backward), the "Direction" parameter of Move will automatically be changed to 1/-1.
Depending on the direction you're rotating (right/left), the "Angle" parameter of Rotate will automatically be changed to a positive/negative value.

The command is linked to the "Pressed Listener" of the keyboard : it means that the robot will move again and again as long as you keep on pressing the key.

Mouse :

  You can also control the command using the mouse.

However, commands will only be used once even if you keep on clicking. This is because there is no event in MouseListener class for such thing.
 ( But this could be added easily, for exemple with a boolean set to true when mouse button pressed, and to false when it is released  ).








For the moment, if you want to change parameters, you have to use the forms like before.
Sliders will be added later to supplement the control panel.

Multi-shape elements

This feature has been added a few weeks ago for the need of the croquet game.

Indeed, hoops and croquet robot are composed of multiples simple shapes :

So, code has been modified to manage multi shapes collisions.

Here are the main parameters added to Shape class :

• relativeDistance : Distance between shape's center and element's center.
• relativeAngle : Angle made by two centers with horizontal.

Using polar coordinates is more simple to with rotating robots, that's why I didn't use relative positions ( like "relativepositionX" and "relativePositionY" ).

• collidable :  This boolean tell about if the shape must be manage during physical collisions. For exemple, robot's hammer and hoop's rectangle are not "collidable".

This boolean must not be mixed up with touchable boolean of Element class : touchable tells if the whole element can collide ( for exemple, it will be set to false when the element is held ).

Rectangles : About drawing rectangles, Swing can't draw rounded rectangles, so rectangles are now drawn line by line, using direction angle of element ( I supposed that the rectangle is straight when the angle has a value of 0° ).

Warning :
In a last news about collision, you may have read that elements must be repositioned during physical collisions. The feature mays bug if you want to reposition a multi-shape element shape by shape (however, this is not important for croquet game, because hoops are fixed and hammer is not collidable ).

Updates & Next Steps

In order to do the last week presentation, some documents have been updated. They're now also updated on Google docs :

• Class Diagram for the virtual platform : simplified, removing unused classes.
• CroquetRobotBuilding : details and photos of Thor, second version.

  Now, here are some informations about the decisions taken during the last meeting about the two remaining features :

Image processing using the camera :
Using Eyad advises, we've decided to use ImageJ  for image processing, a good free Java library.

AI :
One objective of the whole project is to have the Next Break Strategy implemented ( the robot shall be able to roquet balls and cross hoops to earn extra strikes, so that the peg will be reached in one turn ).
However, the fisrt AI to be made will only try to cross hoops.