Echo cancellation apparatus, conferencing system using the same, and echo cancellation method   

Abstract: An echo cancellation apparatus is connectable to a speaker configured to output speaker signals and a microphone configured to receive a sound from the speaker and including a plurality of microphone elements. The echo cancellation apparatus includes: a generating unit configured to generate a plurality of sensitivity signals having different sensitivity patterns which represent directionality of the microphone, based on a plurality of microphone signals obtained from the respective microphone signals; a delay estimating unit configured to determine a shortest delay time as an estimated delay time, the shortest delay time being a shortest one of delay times between the speaker signals and the microphone signals, the delay times being obtained from the respective sensitivity signals; and an echo suppressing unit configured to suppress echoes of the plurality of microphone signals using the estimated delay time.

1. Technical Field

The present invention relates to an echo cancellation apparatus for a teleconferencing system and the like, a conferencing system using the echo cancellation apparatus, and an echo cancellation method.

2. Background Art

Some of echo cancellation apparatuses have used for teleconferencing systems, in which the echo cancellation apparatuses are connected to TV sets and suppress acoustic echoes generated in a case where voices are output from the speakers of the TV sets, according to the delay times of the speakers of the TV sets (see JP-A-2007-201496).

SUMMARY

However, in the above-mentioned technology according to the related art, since the time it takes for a voice from a speaker of a TV set to reach a microphone of a teleconferencing system is used to calculate the delay time of the speaker of the TV set, it is impossible to accurately measure the delay time on the basis of the characteristic of the microphone, and thus it is impossible to suppress echoes.

In other words, in a case of using a directional microphone having high sensitivity on the talker side to prevent echoes from entering the microphone, since the sensitivity of the microphone on the speaker side is low, a sound from the speaker (a direct wave) becomes smaller than a sound such as a sound reflected off walls (a reflected wave) or the like, and thus it is difficult to sense the direct wave. For this reason, the reflected wave may be mistaken as the direct wave. In this case, even if delay time estimation is performed, it is difficult to measure the accurate delay time, and thus an echo process may not be appropriately performed.

For this reason, in view of the above-mentioned problems, the present invention is intended to provide an echo cancellation apparatus capable of reducing echoes regardless of the characteristic of a microphone, a conferencing system using the echo cancellation apparatus, and an echo cancellation method.

In an aspect, an echo cancellation apparatus connectable to a speaker configured to output speaker signals and a microphone configured to receive a sound from the speaker and including a plurality of microphone elements, the echo cancellation apparatus includes: a generating unit configured to generate a plurality of sensitivity signals having different sensitivity patterns which represent directionality of the microphone, based on a plurality of microphone signals obtained from the respective microphone signals; a delay estimating unit configured to determine a shortest delay time as an estimated delay time, the shortest delay time being a shortest one of delay times between the speaker signals and the microphone signals, the delay times being obtained from the respective sensitivity signals; and an echo suppressing unit configured to suppress echoes of the plurality of microphone signals using the estimated delay time. With this configuration, it is possible to reduce echoes regardless of the characteristic of the microphone.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a conceptual view illustrating a teleconferencing system.

FIG. 2 is a block diagram illustrating the teleconferencing system.

FIG. 3 is a block diagram illustrating an echo cancellation apparatus.

FIG. 4 is a block diagram illustrating a delay control circuit.

FIG. 5 is a flow chart illustrating the delay control circuit.

FIG. 6 is a conceptual view illustrating peaks of a correlation function.

FIGS. 7A to 7C are conceptual views illustrating an echo process.

FIG. 8 is a conceptual view illustrating the vicinity of a microphone.

FIGS. 9A to 9C are conceptual views illustrating a microphone of Lch.

FIGS. 10A to 10C are conceptual views illustrating a microphone of Rch.

FIGS. 11A to 11C are conceptual views illustrating a microphone of (Lch+Rch).

FIGS. 12A to 12C are conceptual views illustrating a microphone of (Lch−Rch).

FIG. 13 is a block diagram illustrating another delay control circuit.

FIG. 14 is a block diagram illustrating another delay control circuit.

FIG. 15 is a block diagram illustrating another delay control circuit.

DETAILED DESCRIPTION

An embodiment of the present invention will be described with reference to the accompanying drawings. In the present embodiment, a teleconferencing system will be described as an example. However, it is possible to appropriately use any systems, such as teleconferencing systems, which use speakers and microphones to perform echo cancellation.

A configuration of a teleconferencing system will be described with reference to FIG. 1. FIG. 1 is a conceptual view illustrating a teleconferencing system.

In FIG. 1, in the teleconferencing system, a teleconferencing terminal 1 is connected to a TV set 2 for outputting images and voices, a microphone 3 for gathering the voice of a conference participant, and a camera 4 for acquiring images of the conference participant, and is connected to other teleconferencing terminals through the Internet 5.

As described below in detail, the teleconferencing terminal 1 acquires the images and voice of the conference participant from the microphone 3 and the camera 4, and transmits image signals and voice signals (microphone signals) to the other teleconferencing terminals through the Internet 5. Further, the teleconferencing terminal 1 receives image signals and voice signals (speaker signals) from the other teleconferencing terminals, and outputs the image signals and the voice signals to the TV set 2.

The TV set 2 is a general TV set, includes a display for displaying image signals transmitted from the teleconferencing system, a speaker (a stereo speaker in the present embodiment) for outputting voice signals (speaker signals) transmitted from the teleconferencing system, and so on, delays the speaker signals by a predetermined time (for example, 100 ms) so as to synchronize the speaker signals with the image signals, and outputs images and voices such that the images are sync with the voices at the TV set 2.

The microphone 3 includes a plurality of microphone elements. In the present embodiment, the microphone 3 is circular and includes two microphone elements disposed therein. The two microphone elements 3a and 3b are designed such that, with reference to the center of the microphone 3, the microphone element 3a and the microphone element 3b forms 120°, and a cable connection portion of the microphone 3 and each of the microphone elements 3a and 3b forms 120°.

In the present embodiment, the microphone elements 3a and 3b are directional microphones. In the whole microphone 3, the sensitivity is high on the side where the microphone elements 3a and 3b are disposed, and the sensitivity is low on the opposite side connected to a cable.

The teleconferencing system shown in FIG. 1 will be described in detail with reference to FIG. 2. FIG. 2 is a block diagram illustrating the teleconferencing system. In FIG. 2, arrows show data transmission directions.

The teleconferencing terminal 1 includes an image ADC 6, a voice ADC 7, an echo cancellation apparatus 26, an AV encoder 10, a communication unit 11, an AV decoder 12, a voice DAC 13, and an image DAC 14. The image ADC 6 converts analog image signals from the camera 4 into digital image signals. The voice ADC 7 converts analog microphone signals from the microphone 3 into digital microphone signals. The echo cancellation apparatus 26 reduces (suppresses) acoustic echoes (hereinafter, referred to as echoes) of the microphone signals. The AV encoder 10 encodes the digital image signals output from the image ADC 6 and the microphone signals output from the echo cancellation apparatus 26. The communication unit 11 transmits encoded image and voice data to the other teleconferencing terminals, and receives image and voice data from the other teleconferencing terminals. The AV decoder 12 decodes the image and voice data received from the other teleconferencing terminals by the communication unit 11. The voice DAC 13 converts digital speaker signals from the AV decoder 12 into analog speaker signals. The image DAC 14 converts digital image signals from the AV decoder 12 into analog image signals.

Here, the echo cancellation apparatus 26 includes an echo suppressing unit 8 for estimating an acoustic characteristic between the speaker and the microphone and subtracting echoes from a microphone signal, and a delay control circuit 9 for estimating a delay time necessary for suppressing echoes.

In the present embodiment, the echo cancellation apparatus 26 is mounted on a voice DSP. However, the echo cancellation apparatus 26 may be implemented by a general-purpose CPU.

Also, in the present embodiment, the echo cancellation apparatus 26 is used to suppress echoes. However, an echo suppressor for attenuating target microphone signals linearly or non-linearly may be used to suppress echoes, or both of the echo cancellation apparatus 26 and an echo suppressor may be used.

Moreover, in the present embodiment, since the microphone 3. is a stereo microphone, the microphone 3 has microphone signals of two channels. However, in FIG. 2, for simplification, the microphone signals of two channels are shown like a microphone signal of one channel is shown.

The TV set 2 includes a speaker 15, a display 16, an image processing circuit 17 for receiving analog image signals from the teleconferencing terminal 1 and converting the analog image signals into display signals for performing display on the display 16, and a delay circuit 18 for delaying analog speaker signals from the teleconferencing system for synchronizing a voice and display images of the display 16.

The speaker 15 of the TV set 2 also is stereo and thus has speaker signals of two channels. However, in FIG. 2, for simplification, the speaker signals of two channels are shown like a microphone signal of one channel.

Also, since each of the TV set 2, the microphone 3, and the camera 4 receives and outputs analog signals, the image ADC 6, the voice ADC 7, the voice DAC 13, and the image DAC 14 are provided. However, if each of the TV set 2, the microphone 3, and the camera 4 receives and outputs digital signals, the image ADC 6, the voice ADC 7, the voice DAC 13, and the image DAC 14 are unnecessary.

Now the above-mentioned echoes will be described. In the teleconferencing system described above, a voice output from the speaker 15 of the TV set 2 is input to the microphone 3 directly or after being reflected off walls and the like. As a result, a sound input from the speaker 15 directly to the microphone 3 and sounds input to the microphone 3 after being reflected off the walls may cause echoes, which may decrease the voice quality.

For this reason, the above-mentioned echo cancellation apparatus 26 is used to suppress echoes of microphone signals from the microphone 3.

The operation of the teleconferencing system configured as described above will be described.

If a teleconference starts by the teleconferencing terminal 1 and the other teleconferencing terminals, images acquired by the camera 4 are transmitted as image signals to the AV encoder 10 through the image ADC 6. Similarly, a voice acquired by the microphone 3 is transmitted as microphone signals to the echo cancellation apparatus 26, which suppresses echoes and transmits the microphone signals to the AV encoder 10.

Further, image and voice data encoded by the AV encoder 10 are transmitted from the communication unit 11 to the other teleconferencing terminals through the Internet 5.

Also, image and voice data received from the other teleconferencing terminals by the communication unit 11 are decoded by the AV decoder 12, and image signals and speaker signals are output to the TV set 2 through the voice DAC 13 and the image DAC 14, respectively.

The image signals output from the teleconferencing terminal 1 are processed by the image processing circuit 17 for performing display on the display 16. At this time, in general, a delay of about 100 ms occurs. Therefore, even if the speaker signals output from the teleconferencing terminal 1 are intactly output from the speaker 15 of the TV set 2, a lag between the images and voice at the TV set 2 occurs in response to the image processing time of the image processing circuit 17. For this reason, a preliminary delay circuit 18 delays voice signals by the delay time occurring by the image processing circuit 17 and outputs the voice signals such that the lag between the images and voice at the TV set 2 is eliminated.

Now, an echo suppressing method in a case of using the teleconferencing terminal in the present embodiment will be described in detail with reference to FIG. 3.

In the present embodiment, since there are two voice channels, the microphone 3 includes microphone elements 3a and 3b and the speaker 15 includes a speaker 15a and a speaker 15b. Microphone signals acquired from the microphone elements 3a and 3b are transmitted to echo suppressing units 8a and 8b through voice ADCs 7a and 7b, respectively.

Next, the microphone signals are processed by the echo suppressing units 8a and 8b, are encoded by an audio encoder 27 of the AV encoder 10, and are transmitted to the other teleconferencing terminals.

Also, speaker signals transmitted from the other teleconferencing terminals are decoded by an audio decoder 28 of the AV decoder 12, and the decoded speaker signals are transmitted to each of the echo suppressing unit 8, the delay control circuit 9, and the speaker 15.

Here, the delay control circuit 9 includes a correlation calculation circuit 19 and a delay estimating unit 29. The correlation calculation circuit 19 generates a plurality of sensitivity signals (to be described below) on the basis of the microphone signal acquired from the microphone element 3a and the microphone signal acquired from the microphone element 3b, and calculates the correlation functions between the sensitivity signals and speaker signals. The delay estimating unit 29 obtains the delay time of the speaker signals in the TV set 2 on the basis of the correlation functions obtained by the correlation calculation circuit 19.

Next, the echo suppressing unit 8 delays the speaker signals by the delay time estimated by the delay estimating unit 29, estimates pseudo echo signals on the basis of the delayed speaker signals and the microphone signals by an internal adaptive filter of the echo suppressing unit 8, subtracts the pseudo echo signals from the microphone signals so as to suppress echo components of the microphone signals, and transmits the microphone signals with echoes suppressed to the audio encoder 27.

In the present embodiment, the delay time used in the echo suppressing unit 8 is the delay time of the microphone signals relative to the speaker signals. Further, in the echo suppressing unit 8, the speaker signals input to the echo suppressing unit 8 are used, and the microphone signals input to the echo suppressing unit 8 are used.

Therefore, the above-mentioned delay time includes a delay time occurring by the voice DAC 13, the delay circuit 18, the speaker 15, the microphone 3, and the voice ADC 7, and a delay time it takes for a voice output from the speaker 15 to reach the microphone 3.

Now, the delay control circuit 9 for estimating (calculating) a delay time necessary for the above-mentioned echo process will be described in detail with reference to FIG. 4. FIG. 4 is a block diagram illustrating the delay control circuit.

In the present embodiment, since two microphone elements are used as described above, there are microphone signals of two channels. The microphone signal acquired from the microphone element 3a is referred to as a microphone signal Lch, and the microphone signal acquired from the microphone element 3b is referred to as a microphone signal Rch.

Also, since speaker signals of two channels are acquired from the other teleconferencing terminals, one speaker signal of them is referred to as a speaker signal Lch, and the other speaker signal is referred to as a speaker signal Rch.

In FIG. 4, the delay control circuit 9 includes the correlation calculation circuit 19 for outputting a plurality of correlation functions corresponding to microphone sensitivity signals on the basis of the microphone signals and the speaker signals, and the delay estimating unit 29 for estimating the delay time on the basis of the correlation functions.

The delay estimating unit 29 includes a peak detecting circuit 20 and a delay estimating circuit 21. The peak detecting circuit 20 detects a plurality of peaks corresponding to sharp increases in correlation value from the calculated correlation functions, and the delay estimating circuit 21 detects a peak corresponding to the smallest delay time from outputs of the peak detecting circuit 20, thereby determining the delay time between the microphone signals and the speaker signals.

The operation of the delay control circuit 9 configured as described above will be described with reference to FIG. 5. FIG. 5 is a flow chart illustrating the delay control circuit.

First, in step 1, with respect to an input Lch microphone signal mL,(t), an input Rch microphone signal mR(t), and the composite signal of Lch and Rch speaker signals, the delay control circuit 9 calculates correlation functions of Lch and Rch by the correlation calculation circuit 19.

Here, the above-mentioned correlation functions are calculated using Equations 1 and 2.

c L  ( τ ) = ∑ t  m L  ( t ) · s  ( τ - t ) [ Equation   1 ] c R  ( τ ) = ∑ t  m R  ( t ) · s  ( τ - t ) [ Equation   2 ]

In the above-mentioned Equations, τ is a predetermined shift time of a corresponding speaker signal and corresponds to the time axis of each correlation function, t is a current time of the corresponding speaker signal and a corresponding microphone signal, s(τ−t) is a signal obtained by shifting the corresponding speaker signal by the predetermined shift time τ, cL(τ) is the correlation function of Lch, and cR(τ) is the correlation function of Rch. Further, “mL(t)·s(τ−t)” and “mR(t)·s(τ−t)” are a product of mL(t) and s(τ−t) and a product of mR(t) and s(τ−t), respectively.

In step 2, the correlation calculation circuit 19 calculates the cross correlation function cadd(τ) of (Lch+Rch) on the basis of the correlation function cL(τ) of Lch and the correlation function cR(τ) of Rch. Similarly, in step 3, the correlation calculation circuit 19 calculates the cross correlation function cdiff(τ) of (Lch−Rch) on the basis of the correlation function cL(τ) of Lch and the correlation function cR(τ) of Rch.

The cross correlation function cadd(τ) of (Lch+Rch) and the cross correlation function cdiff(τ) of (Lch−Rch) are calculated using Equations 3 and 4.

cadd(τ)=cL(τ)+cR(τ)   [Equation 3]

cdiff(τ)=cL(τ)   [Equation 4]

Next, in step 4, the delay control circuit 9 time-averages each of the correlation function cL(τ) of Lch, the correlation function CR(τ) of Rch, the cross correlation function cadd(τ) of (Lch+Rch), and the cross correlation function cdiff(τ) of (Lch−Rch) so as to reduce instant peak changes of the four correlation functions, and outputs the time-averaged correlation functions to the peak detecting circuit 20.

In the present embodiment, in order to reduce instant peak changes, time-averaging is performed. However, the time-averaging may be omitted to simplify the echo process.

Next, in step 5, the peak detecting circuit 20 detects a plurality of peaks corresponding to sharp increases in correlation value from the time-averaged correlation functions.

Now, the peak detection will be described with reference to FIG. 6. FIG. 6 is a conceptual view illustrating peaks of a correlation function.

FIG. 6 shows a correlation function with correlation values on the vertical axis and a delay time on the horizontal axis. From this correlation function, peaks are detected.

The above-mentioned correlation functions correspond to sensitivity signals, and represent the correlation between the microphone signals and the speaker signals. The correlation function shown in FIG. 6 represents how much a microphone signal is correlated with a signal obtained by shifting a speaker signal by a predetermined time along a time axis (the degree of matching of the waveforms of both signals), and corresponds to Equation 1 or 2.

The detection of peaks of the correlation values of the correlation function is performed using the average and deviation of the entire correlation function, and a portion satisfying the condition of Equation 5 is determined as a peak.

cmax>μc+λc·νc   [Equation 5]

Here, cmax is a portion which is a subject of the peak determination, μc is the average of the entire correlation function, νc is the deviation of the entire correlation function, and λc is a parameter for adjusting a threshold value for peaks. If the parameter λc is large, only sharp peaks are detected, and if the parameter λc is small, it is easy to detect peaks. For this reason, the parameter λc is appropriately set according to the purpose. In the present embodiment, the parameter λc is set to 5 to 7.

According to the above-mentioned condition, in FIG. 6, three peaks marked with stars are detected.

After peaks of each of the correlation functions are detected, in step 6, the delay estimating circuit 21 estimates the shortest delay time of the delay times corresponding to the first peaks of the correlation functions, as the delay time of the speaker signals, on the basis of the detected peaks, subtracts a predetermined margin from the estimated delay time, and outputs the subtraction result as the delay time of the delay estimating circuit 21.

Now, the reason why the shortest delay time is estimated will be described. With respect to each of the correlation functions in the microphone 3, if peaks are detected, it is possible to obtain several delay time of the microphone signals. In this case, since the direct wave of the sound from the speaker 15 enters the microphone 3 earlier than the reflected waves of the sound from the speaker 15 reflected off walls and the like, the shortest delay time becomes the delay time of the direct wave.

Further, since the echo process is performed on a sound including a direct wave component, if it is possible to see the delay time of the direct wave, it is possible to effectively perform the echo process.

For these reasons, the delay time of the direct wave necessary for performing the echo process, that is, the shortest delay time is estimated.

Hereinafter, a method from the peak detection to the delay time estimation will be described with an example.

In the present embodiment, the peaks of the correlation functions of Lch, Rch, (Lch+Rch), and (Lch−Rch) are detected, the shortest delay times of the respective correlation functions are obtained, and the shortest delay time is selected from the delay times of Lch, Rch, (Lch+Rch), and (Lch−Rch).