Development of Image Processing Based Feedback Systems For Interactive Gaming Using Non-Traditional Controllers

Development of Image Processing Based Feedback Systems for Interactive Gaming Using Non-Traditional Controllers
Adam Hedji Mantas Pulinas Philip San III. Viktorious

SSIP 2009, Debrecen, Hungary
Full Body Motion Detection

Project Natal X-Box 360 Microsoft Corporation
New project that promises to use two cameras for full-body motion detection
CHECKERS!
Overview of Code
Board Plane Geometry Detection
Overview of Code
Board Plane Geometry Detection
Segmentation of Individual Game Squares
Identification of Square Status by Colour Classification
Relaying of Data with Checkers A.I. Feedback This is blue!
Geometry Detection
Find the corners 2 approaches were used: Edge detection and region growing Hough transforms
Geometry Detection Edge Detection and Region Growing

Edge detection with Sobel operator Dilate the image to fill the gaps in the border

Edge detection with Sobel operator Dilate the image to fill the gaps in the border Thin the image to get the true border

Edge detection with Sobel operator Dilate the image to fill the gaps in the border Thin the image to get the true border Dilate the image several times to remove useless edges

Edge detection with Sobel operator Dilate the image to fill the gaps in the border Thin the image to get the true border Dilate the image several times to remove useless edges Region growing Determine the corner tiles Track corners in real-time
Geometry Detection Projective Correction

Edge detection with Sobel operator Dilate the image to fill the gaps in the border Thin the image to get the true border Dilate the image several times to remove useless edges Region growing Determine the corner tiles Track corners in real-time
Geometry Detection Projective Correction
Calculate homography using corner coordinates

Geometry Detection Hough Transform

Hough transform We had time so we developed a better solution Based on Hough transformations Better real-time line detection
Colour Classification
How can the computer classify the colours?
(32,61,105)
(171,154,158)
Use image processing algorithms to make the RGB values only 0 or 255
(0,0,255)
(255,255,255)
(255,255,255) (0,0,0) (255,255,0) (0,0,255)
Select individual tile Analyse the predominant colour inside to classify the square state (white, black, blue, yellow)
Sample of pixels used as opposed to whole square

How to determine if each R, G and B values are 0 or 255? -Need to choose threshold value e.g.
A given pixel of value (15, 19, 250) A threshold of 126 Output is (0,0,255)
Colour Classification - But how to choose the threshold?

Webcams can be of relatively poor quality and provide poor contrast. For example, blue pieces used were relatively hard to distinguish from the black tiles.
But how to choose the threshold?

Can instead normalise the image and use a threshold of 127, given by (256/2)-1. Select a white tile and take an average of the colour values. Do the same for a black tile. Use these averages to normalise the image
p1=(241,209,210) p2=(232,204,214)
p3=(240,211,205)
Input Black Output( R, G, B) White Black
Colour Classification - Normalisation

Then reduce the image to absolute values of 0 and 255 Use a threshold of 126 (half of full intensity value)
Summary
Main results Successfully used 2 approaches to chess board detection
Edge detection and region growing Hough transform
Removing perspective distortion Identification of individual tiles and pieces, including classification Connection to engine interface with feedback system
Improvements
Aim towards real-time 60fps processing Use a more efficient programming language such as C++ Use of GPU using CUDA or Open CL programming language More complex algorithms Motion detection of hand Use of overlay of 3D structures onto camera image. Virtual humans...
Unused Virtual Human
The Team
Adam Hedji University of Zagreb, Croatia adam.hedi@fer.hr Philip San University College London, England p.san@ucl.ac.uk
Mantas Paulinas Vilnius Gediminas Technical University, Lithuania mantas.paulinas@el.vgut.lt
Viktor Blaskovics University of Szeged, Hungary blaskovics.viktor@stud.u-szeged.hu

Development of Image Processing Based Feedback Systems For Interactive Gaming Using Non-Traditional Controllers

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Development of Image Processing Based Feedback Systems For Interactive Gaming Using Non-Traditional Controllers

Enviado por

Direitos autorais:

Formatos disponíveis

Development of Image Processing Based Feedback Systems for Interactive Gaming Using Non-Traditional Controllers

Adam Hedji Mantas Pulinas Philip San III. Viktorious

Full Body Motion Detection

Segmentation of Individual Game Squares

Identification of Square Status by Colour Classification

Relaying of Data with Checkers A.I. Feedback This is blue!

Geometry Detection Edge Detection and Region Growing

Geometry Detection Edge Detection and Region Growing

Geometry Detection Edge Detection and Region Growing

Geometry Detection Edge Detection and Region Growing

Geometry Detection Edge Detection and Region Growing

Geometry Detection Projective Correction

Geometry Detection Projective Correction

Calculate homography using corner coordinates

Geometry Detection Hough Transform

Sample of pixels used as opposed to whole square

Colour Classification - But how to choose the threshold?

But how to choose the threshold?

Input Black Output( R, G, B) White Black

Colour Classification - Normalisation

Colour Classification - Normalisation

Colour Classification - Normalisation

Colour Classification - Normalisation

Colour Classification - Normalisation

Colour Classification - Normalisation

Unused Virtual Human

Mantas Paulinas Vilnius Gediminas Technical University, Lithuania mantas.paulinas@el.vgut.lt

Viktor Blaskovics University of Szeged, Hungary blaskovics.viktor@stud.u-szeged.hu

Você também pode gostar