중첩(Convolution)과 상관함수(Correlation Function)






 

1. 중첩(Convolution)
가. 정의
– 두개의 함수가 스펙트럼 영역을 엇갈리게 적분하여 하나의 함수로 융합되는 것
– 중첩은 선형시스템에서 입력에 대한 응답 출력을 구하는 데 사용됨
– 입력을 x(t), 출력을 y(t), 임펄스 응답을 h(t)라고 하면 다음 식으로 표현됨

나. 특징
– 교환법칙 성립(계산 상으로는 시스템과 입력신호 비구분)
– 두 개의 시간 함수의 중첩은 τ가 의사 변수이므로 적분 결과는 t의 함수가 됨
– 시간 영역에서의 중첩은 주파수 영역에서의 곱으로,
– 주파수 영역에서의 중첩은 시간 영역에서의 곱으로 나타냄
– convolution은 계산적 방법과 도식적 방법으로 구할 수 있음
– 구형파 시간 함수끼리의 convolution은 ramp 함수가 됨

다. 응용
– 시스템 해석
– Digital & Analog Filter 해석
– Impulse 응답
– 시스템 출력 해석
– DSP(Digital Signal Processing)

2. 상관함수(Correlation Function)
가. 정의
– 시간영역에서 두 신호 사이의 상호 연관성을 나타내는 함수
– 상관계수는 백터공간에서 두 신호점간의 유사성을 정의
– 상관함수는 시간영역에서 두 신호의 유사성을 결정하는데 사용

나. 특징
– 상관함수의 종류에는 자기상관함수와 상호상관함수로 표현됨
– 상관함수의 Fourier 변환과 중첩적분의 Fourier 변환은 같음
– 상관함수의 Fourier 변환은 전력에 관한 주파수 스펙트럼임
– 우함수 이며, τ =0에서 최대값을 가짐

다. 응용
– System의 평균전력
– 전력스펙트럼 밀도(Power Spectrum density), system의 analysis
– 통신에서 Quadrature, 특정 신호성분의 존재판별, PN부호 판별에 사용

3. 중첩(Convolution)과 상관함수(Correlation Function)의 상호 관계
– 시간 영역에서 중첩과 상관을 구하는 것은 많은 연산을 필요로 함
– FFT를 이용하여 이들 연산을 고속으로 수행 가능
– 중첩 정리 : x(t)의 주파수 변환치를 X(f), h(t)의 주파수 변환치를 H(f)라면
Y(f) = X(f) H(f)를 얻은 후 Y(f)의 역변환을 취하면 y(t)f를 얻음



Convolution Filter Matrix 값 모음

//
적청반전
var filterObj = new flash.filters.ColorMatrixFilter();
filterObj.matrix = new Array(
 0,0,1,0,0,
 0,1,0,0,0,
 1,0,0,0,0,
 0,0,0,1,0);
mc.filters = [filterObj];


 


 


//
흑백
var filterObj = new flash.filters.ColorMatrixFilter();
filterObj.matrix = new Array(
 1/3,1/3,1/3,0,0,
 1/3,1/3,1/3,0,0,
 1/3,1/3,1/3,0,0,
   0,  0,  0,1,0);

mc.filters = [filterObj];


 


 


//
세피아
var filterObj = new flash.filters.ColorMatrixFilter();
filterObj.matrix = new Array(
 1/2,1/2,1/2,0,0,
 1/3,1/3,1/3,0,0,
 1/4,1/4,1/4,0,0,
   0,  0,  0,1,0);
mc.filters = [filterObj];


 


 


//
샤프
var filterObj = new flash.filters.ConvolutionFilter();
filterObj.matrix = new Array(
  0,-1, 0,
 -1, 5,-1,
  0,-1, 0
);
filterObj.matrixX = 3;
filterObj.matrixY = 3;
filterObj.bias = 0;
filterObj.divisor = 1;
mc.filters = [filterObj];


 


 


//
윤곽 추출
var filterObj = new flash.filters.ConvolutionFilter();
filterObj.matrix = new Array(
 1, 1, 1,
 1,-7, 1,
 1, 1, 1
);
filterObj.matrixX = 3;
filterObj.matrixY = 3;
filterObj.bias = 0;
filterObj.divisor = 1;
mc.filters = [filterObj];


 


 


//
엠보스
var filterObj = new flash.filters.ConvolutionFilter();
filterObj.matrix = new Array(
 1, 1, 0,
 1, 0,-1,
 0,-1,-1
);
filterObj.matrixX = 3;
filterObj.matrixY = 3;
filterObj.bias = 128;
filterObj.divisor = 1;
mc.filters = [filterObj];


 

////////////////////////////////////////////////////////////////////////////////////////////////////


Flash 8 introduces some very powerful tools for manipulating bitmaps at the pixel level. Included in this list of tools is flash.filters.ConvolutionFilter. ConvolutionFilter combines pixel data in a bitmap with data from neighboring pixels to produce a given result. Having control at the pixel level allows you to produce a wide array of effects on a bitmap. These include things like blurring, beveling, embossing, sharpening, and more. All are possible using ConvolutionFilter.


Unlike Matrix and ColorMatrixFilter, ConvolutionFilter’s matrix does not have a set number of rows and columns. The number of rows and columns depend on the type and strength of the effect you are trying to achieve.


In a nutshell, ConvolutionFilter looks at each and every pixel in a source bitmap. As it does this, it uses the center value in the matrix as the value of the current pixel being manipulated. For example, in a 5 x 5 matrix, the center value is at (2, 2). It then multiplies the values from the matrix to the surrounding pixels and adds the resulting values for all pixels to get the value for the resulting center pixel. Here is the formula used on a 3 x 3 matrix convolution:



As you can see, for the pixel located at (x, y), ConvolutionFilter with a 3 x 3 matrix takes the pixel (x–1, y–1) and multiplies it by the value in the matrix located at (0,0), and then adds the pixel (x, y–1) multiplied by the value in the matrix at (0,1), and so on until all of the matrix values have been multiplied by the corresponding pixel value. (This is done for each color channel.) Finally, it takes that total, divides by the value of divisor, and adds the value of bias. Obviously the larger your matrix, the longer this process takes.


To apply a convolution matrix, you can pass a matrix along with the number of rows and columns into the ConvolutionMatrix constructor, as follows:



The following examples illustrate some convolution matrices. A useful exercise is to look at the matrix values and guess the effect they might produce on the image. Figure 10 shows the original photo.



Figure 10. Original image


In Figure 11, the pixel being affected gets its original value multiplied by 5, while the pixels immediately above, below, to the left, and to the right are multiplied by –1, with the resulting values added together and multiplied by the affected pixel to produce a new value for that pixel. The resultant effect is that there is an increase in contrast between neighboring pixels. If neighboring pixels have values that are quite similar, those pixels remain fairly similar. However, the greater the original color value difference between pixels, the greater the resulting difference will be.



Figure 11. A sharpening effect


Figure 12 shows the value of the affected pixel having its value added to that of each of its surrounding eight pixels. You can probably guess that this “mashing” of values results in a blur effect.



Figure 12. A blurring effect


By looking at the matrix in Figure 13, you can see that the result is trickier to guess than in the previous examples. The affected pixel tends to become closer in value to the pixels near the bottom right and further away in value than the pixels at the top left. The result is an embossed effect with the light source appearing to emanate from the top left.



Figure 13. An embossing effect


As you can see, having a basic understanding of matrices allows you to produce some powerful effects using the new ConvolutionFilter. In the convolution demo file that accompanies this article, you can play directly with values in a matrix and see the resulting convolution effect on the image. Play with the following demo to see the changes for yourself:


Where to Go from Here


Flash 8 was truly one of the biggest releases in Macromedia’s product history. It provides developers with a very granular level of control in several different areas. Matrices provide a powerful means of doing this type of manipulation. This article serves as a primer for people interested in gaining a better understanding of matrices. I recommend playing with these matrix values to see the ensuing results. That’s often where the real learning happens.


Here are some links you may find useful in your exploration:




Convolution Filter 자료

출처:http://www.roborealm.com/help/Convolution.php


Convolution Filter

Convolution filters are a great way to process images for certain features. Features are defined by an n by m matrix that is applied to the image in the following way: (grayscale only for purposes of example)


Interface




Instructions


1. Edit the 5×5 textbox grid to add in your convolution values OR
2. select a pre-created filter using the dropdown menu to help you get started
3. Specify the divisor and/or bias


Example









Source Sharpen More

An example small grayscale image (10×10):

















































































































34 22 77 48 237 205 29 212 107 41
50 150 77 158 233 251 112 165 47 229
93 0 77 219 43 56 42 113 140 94
32 19 44 30 36 94 151 101 28 84
10 90 48 73 63 148 159 183 99 22
192 70 27 88 20 230 53 34 38 106
239 202 196 205 50 123 192 88 41 37
230 174 14 22 127 100 189 186 214 187
227 86 195 6 53 168 46 166 36 249
215 165 237 110 125 191 191 94 123 8


An example convolution filter for line detection:















-1 -1 -1
-1 8 -1
-1 -1 -1


The row=2, column=2 pixel and its neighborhood from the image above: The row=2, column=2 pixel and its neighborhood from the image above:















34 22 77
50 150 77
93 0 77


To apply the convolution filter multiply the filter values with the image data block. Work with each pixel and its 3×3 neighborhood:















-1*34 -1*22 -1*77
-1*50 8*150 -1*77
-1*93 -1*0 -1*77


Then sum all the values:
(-34)+(-22)+(-77)+
(-50)+(1200)+(-77)+
(-93)+(0)+(-77) = 770

Divide by the divisor and add the bias.
(770/divisor)+bias=770 (in this example divisor=1, bias=0)

If the new pixel value is > 255 set it to 255
If the new pixel value is < 0 set it to 0

The new pixel value is 255. Store that in a new image:















34 22 77
50 255 77
93 0 77


Continue with all other 3×3 blocks in the image using original values. For example the next image block could be















22 77 48
150 77 158
0 77 219

Note the 3×3 “window” is shifted to the right by one and that the new pixel value is NOT used but stored as a second new image.

Most of the image is processed in this manner. Image borders create problems and are ignored.

Many other filters can easily be defined for other purposes

Blur:



























1 1 1     1 2 1
1 1 1     2 4 2
1 1 1     1 2 1

Sharpen:



























-1 -1 -1     0 -1 0
-1 9 -1     -1 5 -1
-1 -1 -1     0 -1 0

Edge Enhancement:







































0 0 0     0 -1 0     -1 0 0
-1 1 0     0 1 0     0 1 0
0 0 0     0 0 0     0 0 0

Find Edges:







































0 1 0     -1 -1 -1     1 -2 1
1 -4 1     -1 8 -1     -2 4 -2
0 1 0     -1 -1 -1     1 -2 1

Emboss:















-2 -1 0
-1 1 1
0 1 2
The convolution matrix is displayed in the convolution interface. Changing any number will alter the matrix and change the image as a result of applying that matrix.

Select a matrix from the pull down menu which will populate the matrix with those values.

Deface for flash10

I’ve updated the Deface face recognition library that I created last year to take advantage of the new Vector class and some other improvements.


google code page


It should be fast enough for some limited real-time use now. For instance, the demo below typically finds my face in < 80ms once it’s aquired it. That’s only about 12fps, but for certain applications that would be sufficient. And I’m sure that the tracking can be improved a lot. There’s no real prediction now as is, nor position averaging, etc.


Allow this to use your webcam if you have one. If you don’t, you’re out of luck.


source

———————————————————————————————

http://code.google.com/p/deface/


Based on ideas from OpenCV and the well-known Viola and Jones face detection paper, but NOT associated with either. Deface uses XML encoded classifiers to visually detect and track objects (especially faces) in BitmapData. Compatible with classifiers from OpenCV.

———————————————————————————————


Live Examples


You can see the code in action here:



Source code for the first demo is available on the download page. See DemoCode.

MariLena : OpenCV port to ActionScript 3 (AS3 / Flash)

I dreamed about it, they finally did it ^^.


A crazy guy named Ohtsuka Masakazu have ported the face detection part of OpenCV to AS3 and added the source code named Marilena to the Spark Project a few weeks ago (can’t believe I haven’t notice before today)


Mario Klingemann from Quasimondo.com already made some optimization (see his blog)


I made some test today with a stupid texture, starting from Mario’s webcam example and that’s just fun !!! Well yes, it’s still slow (15 fps on a good computer), but this is just the beginning as I believe that a lot of optimization can still be done, and so what anyway ? It’s flash, it’s everywhere, nothing to install, the fun is there without downloading anything. I really believe in it ! Now with this and FLARToolKit (ARToolKit port to AS3), augmented reality and computer vision is really getting into Flash for sure, and it’s going to give to the community some great opportunity !! I’m looking forward about it


Please, have a test at Mario’s example here


and take also a look at what they did at Squidder.com




Realtime Face Detection in Flash from squidder on Vimeo.


For more info, check out http://www.squidder.com/2009/02/26/realtime-face-detection-in-flash/

AS3 Version of OpenCV for Facial Detection, Web Cam Object Detection, Head Tracking, 3D Library Integration and more with Open Computer Vision Library for Flash

Libspark from Japan is a treasure trove of great flash advancements, they seem to realize the great things that can come from porting in existing solid libraries from C/C++ etc into flash and have been scoring lately including augmented reality in flash porting the ARToolkit to FLARToolkit.  Recently a port of openCV for as3 called Marilena was found and it is for object detection and decent facial recognition (it is a computer vision library from intel) considering the processing power needed to do this.


Face Detection: Here is the sample included with Marilena showing facial detection on an image.


marilena1


Lots of recent action has blown up on this front from Mr doob, quasimondo (optimizing the Marilena classes for better performance) and Boffwswana. Also there is a kit called deface by sshipman that is the first foray into this a year ago doing similar things but it was just a bit before it’s time and a bit slow in previous versions of flash, it performs decent now in this sample. Flash 10 performance of the AVM2 and future directions with Alchemy will lead to more interesting stuff just like this.


Mr. doob head tracking sample, be sure to check lots of other examples there


mrdoob_facedriven3d


Boffswana example of head tracking Johnny Lee Wii style with only a webcam and flash, no wiimote needed since it uses facial detection to check where you are and how close you are in the screen and then moves accordingly.


bofswana



This is stemming from the recent explosion of the FLARToolkit and augmented reality in flash as well as the gimmicks used by Nintendo with the wii and Johnny Lee’s great head tracking advancements. Porting great libraries to flash seems to be the phase we are entering now judging by the recent excitement around Adobe Alchemy and the LLVM along with the lead from the libspark.org contributors. We have also seen this heavily last year in ports of Box2D for 2d physics and other toolkits using established working code and porting that to flash now that is is mostly capable of handling the performance.


OpenCV (Open Computer Vision Library by Intel) is quite a powerful platform that allows you to do all this and now it is available in flash. There are other great libraries for nearly all platforms now. I have done some previous work with Aforge which is also a port of OpenCV mainly for motion detection. This was always around but not until the recent performance updates and the innovation that has come with Alchemy and the thinking that goes along with that (porting in libraries to flash from C/C++ etc), has allowed this to flourish in flash and thus the web.


The amazing new things we can do with flash by porting in existing libraries is only going to get more intense as alchemy and flash 10 are even more mainstream.  It is almost as if Flash will eventually just become a web renderer and simplified front end to many great toolkits that exist in more native environments like C/C++ but with the speed and distribution access of the web with Flash.  Exciting times ahead.

실시간 눈 위치검출하는 로직중..

1. 실시간으로 ‘눈’ 위치 검출하는 로직중에….


가장 빠른 로직은 무엇일까요?


————————————————–


이 질문의 개인적인 답은
Haar Feature 이라 생각이 들었습니다.


Haar Feature 관련 글을 많이 봤는데요.


Here


HaarFilters


how



이런 그림은 많이 봐서 원리는 대충 감은 오거든요.
얼굴에 생기는 명암으로 구별하는 것 같은데.
구현에 관해선 아무리 찾아봐도 설명되있지 않네요.


2. 로직을 구현하는 방법…. 


제가 아는 정도는
빠른 연산을 위해 테이블에 저장해 놓으면
해당 범위의 값을 빠르게 구할 수 있다는데요.


Integral

rect의 x,y,w,h 의 값을 바꿔 가면서
영상의 모든 부분을 for 문으로
전부 탐색하기엔 속도가 느릴 것 같거든요..


Haar

Eigenfaces



OpenCV의 정의

OpenCV(Open Computer Vision)은 오픈소스 컴퓨터 비전 C 라이브러리이다. 원래 인텔에서 개발되었다. 현재 버전 1.0이 나와있으며 윈도, 리눅스 플랫폼에서 사용할 수 있다. 실시간 이미지 프로세싱에 중점을 둔 라이브러리이다. 인텔 CPU에서 사용되는 경우 속도의 향상을 볼 수 있는 Intel Performance Primitives (IPP)를 지원한다.




OpenCV 1.1 버젼이 공개되었네요.
파일명에 1.1pre1 라고 써 있는거 보니까 공식 버젼은 아닐수도 있는데, 참으로 반가운 오랫만의 버젼입니다.


관련해서 소식을 전해주신 유유자적님의 글은 다음과 같고요
http://cafe.naver.com/opencv/7044


다운은 공식 홈피에서 받으시면 됩니다.
http://sourceforge.net/projects/opencvlibrary/ (홈)
http://sourceforge.net/project/showfiles.php?group_id=22870   (다운링크 페이지)


라이브러리 안의 메뉴얼 부분이 어떻게 잘 지원되었는지는 모르겠지만,
공식적으로는 O’Reilly Book을 보라고 나와 있습니다. (저는 구입해서 보고 있습니다만…)


Tutorial documentation is in O’Reilly Book: Learning OpenCV http://www.amazon.com/Learning-OpenCV-Computer-Vision-Library/dp/0596516134


오피셜 OpenCV Wiki (영문)


http://opencv.willowgarage.com/wiki/


[OpenCV] 설치와 개발환경 구축

라이브러리 다운받기


우선 라이브러리를 다운받아야 한다.
아래의 링크에, OpenCV 와 관련된 최신 뉴스 및 각종 자료, 그리고 라이브러리를 다운 받을 수 있다.
http://sourceforge.net/projects/opencvlibrary/


윈도우를 쓰니까, 당연히 opencv-win 을 받아야겠지!!!
2008년 10월18일 버전…. 파일은 실행파일! 게임 설치하듯이 가볍게 설치를 해주고…


OpenCV Document 읽어보기


설치가 끝나면, OpenCV에 관란 Document 를 읽어볼 수 있다. 제대로 공부해보고자 한다면, 한번 스윽~ 하고 읽어봐주자. 나는 제대로 공부할려는 마음이 있으니..









 OpenCV Document 아주 간단히 훓어보기


 OpenCV : Open Source Computer Vision Library
 영상처리와 컴퓨터비전 알고리즘이 C함수와 C++클래스로 구현이 되어있다.
 
상업적/비상업적으로 이용할수 있다(프리) – 자세한 라이센스는 안읽어봤음.
 FAQ : 설치방법 등이 설명되어 있음(문제점해결방법 등)
 레퍼런스 메뉴얼 : 앞으로 개발하면서 자주 볼 메뉴얼…


개발환경 설정
(visual studio 2005 사용)


툴 -> 옵션 -> VC++ 디렉토리 에서
1. 다음 파일의 디렉토리 표시에 포함파일(include file) 을 선택
2. 포함 디렉토리에 아래의 다섯개 경로를 추가해 주자.<그림1>


C:\Program Files\opencv\cv\include
C:\Program Files\opencv\cv\src
C:\Program Files\opencv\cvaux\include
C:\Program Files\opencv\cvaux\src
C:\Program Files\opencv\otherlibs\highgui

<그림1>


< ?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />


 


3. 다음 파일의 디렉토리 표시에 라이브러리 파일(library file) 을 선택


4. 아래의 세파일을 추가시킨다.<그림2>


C:\Program Files\opencv\bin
C:\Program Files\opencv\lib


C:\Program Files\opencv\interface\ch\opencv\bin


 


<그림2>



 


5. 프로젝트 -> 속성

아래의 파일을 라이브러리에 추가한다.<그림 3>

cxcore.lib cv.lib highgui.lib

 

<그림 3>


 

 

* <표 1>은 참고만 하세요. 옛날 버전 설명인데 메뉴이름이 달라서 잘 모르겠네요.

 <표 1>






 1. [프로젝트][설정][링크]항목을 선택해,카테고리를 「인풋」으로 한다.


 2. [오브젝트/라이브러리 모듈]란에 이하의Library파일을 추가한다.
     cv.lib, highgui.lib, cvaux.lib
     베타4버전에서는 cvxcore.lib,  cv.lib,   highgui.lib 또한 cxcore.dll, cv.dll, highgui.dll 이 필요하다.


이로써 기본적인 환경설정은 완료 되었다. 빌드도 잘 되고  ㅋㅋ


그렇다면 샘플 프로그램을 몇개 돌려보고, 테스트 프로그램 몇 개 짜보면서, 기분 좋은 출발을 해보자.



5월 5일 어린이날…. 캠연결을 테스트 해보다…이전 설명에 빠진 부분이 있어서, 추가 합니다.





    • C:\Program Files\OpenCV\bin 폴더 내에 있는 dll 파일을 프로젝트의 실행파일이 있는 곳에 복사를 해줍니다.







 cv110.dll cvaux110.dll cxcore.dll highgui110.dll





  • 툴 -> 옵션 -> VC++ 디렉토리 에서


    1. 다음 파일의 디렉토리 표시에 포함파일(include file) 을 선택, 아래 경로를 추가.






       C:\Program Files\OpenCV\cxcore\include