Sound zoom method, medium, and apparatus   

Abstract: A sound zoom method, medium, and apparatus generating a signal in which a target sound is removed from sound signals input to a microphone array by adjusting a null width that restricts a directivity sensitivity of the microphone array, and extracting a signal corresponding to the target sound from the sound signals by using the generated signal. Thus, a sound located at a predetermined position away from the microphone array can be selectively obtained so that a target sound is efficiently obtained.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of copending application Ser. No. 12/010,087 filed Jan. 18, 2008, the contents of which are incorporated by reference

1. Field

One or more embodiments of the present invention relate to a sound zoom operation involving changing a received sound signal according to a change in the distance from a near-field location to a far-field location, and more particularly, to a method, medium, and apparatus which can implement a sound zoom engaged with a motion picture zoom operation through the use of a zoom lens control in a portable terminal apparatus, for example, such as a video camera, a digital camcorder, and a camera phone supporting the motion picture zoom function.

2. Description of the Related Art

As video cameras, digital camcorders, and camera phones capable of capturing motion pictures are becoming increasingly more common, the amount of user created content (UCC) has dramatically increased. Similarly, with the development of high speed Internet and web technologies, the number of channels conveying such UCC is also increasing. Accordingly, there is also an increased desire for digital devices capable of obtaining a motion picture with high image and sound qualities according to the various needs of a user.

With regard to conventional motion picture photographing technologies, a zoom function for photographing an object at a far-field distance is applied only to the image of the object. Even when a motion picture photographing device photographs the far-field object, in terms of sound, the background interference sound at a near-field distance to the device is merely recorded as it, resulting in the addition of a sense of being audibly present with respect to the far-field object becomes impossible. Thus, in order to be able to photograph an object along with a sense of being present with respect to the far-field object, when sound is recorded along with the zoom function when capturing an image, a technology for recording the far-field sound by excluding the near-field background interference sound would be needed. Herein, in order to avoid confusion with a motion picture zoom function for photographing an object at a far-field distance, descriptions below regarding a technology to selectively obtain sound separated a particular distance from a sound recording device will be referred to as sound zoom.

In order to selectively obtain sound located a particular distance away from a recording device, there are techniques of changing a directivity of a microphone by mechanically moving the microphone along with the motion of a zoom lens and of electronically engaging an interference sound removal rate with the motion of a zoom lens. However, the former technique merely changes a degree of the directivity to the front side of microphone so that the near-field background interference sound cannot be removed. According to the latter technique, when the signal-to-noise ratio (SNR) of a far-field sound is low, it may be highly likely that a target signal is also removed due to a misinterpreting of a far-field target sound as the interference sound. In addition, in the engagement with a zoom lens control unit, the amount of removal of interference sound performed by an interference sound removal filter can be applied only to stationary interference sounds.

SUMMARY

To overcome such above and/or other problems, one or more embodiments of the present invention provide a sound zoom method, medium, and apparatus which can differentiate a desired sound by overcoming a problem of an undesired sound, at a distance that a user does not desire, being recorded because sound cannot be selectively obtained and recorded based on distance, and/or overcome another problem of a target sound being misinterpreted as interference sounds and removed. Such a method, medium, and apparatus can overcome a limitation of interference sound canceling being applied only to stationary interference sound, unlike a motion picture zoom function capable of photographing an object according to the distance from a near-field location to a far-field location.

Additional aspects and/or advantages will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the invention.

According to an aspect of the present invention, a sound zoom method includes generating a signal in which a target sound is removed from sound signals input to a microphone array by adjusting a null width that restricts a directivity sensitivity of the microphone array, and extracting a signal corresponding to the target sound from the sound signals by using the generated signal.

According to another aspect of the present invention, embodiments may include a computer readable recording medium having recorded thereon a program to execute the above sound zoom method.

According to another aspect of the present invention, a sound zoom apparatus includes a null width adjustment unit generating a signal in which a target sound is removed from sound signals input to a microphone array by adjusting a null width that restricts a directivity sensitivity of the microphone array, and a signal extraction unit extracting a signal corresponding to the target sound from the sound signals by using the generated signal.

According to one or more embodiments of the present invention, like the motion picture zoom function capable of photographing an object according to the distance from a near distance to a far distance, sound may be selectively obtained according to the distance by interpreting sound located at a distance that a user does not desire as interference sound and removing that sound, in sound recording. In addition, a target sound may be efficiently obtained by adjusting a null width of a microphone array. Furthermore, in removing interference sound, by using a stationary interference sound removing technology varying according to the time, interference sound may be removed in an environment in which the characteristic of a signal varies in real time.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIGS. 1A and 1B respectively illustrate environments of a desired far-field target sound with near-field interference sound and a desired near-field target sound with far-field interference sound;

FIG. 1C illustrates a digital camcorder with example microphones for a sound zoom function, according to an embodiment of the present invention;

FIG. 2 illustrates a sound zoom apparatus, according to an embodiment of the present invention;

FIG. 3 illustrates a sound zoom apparatus, such as that of FIG. 2, with added input/output (I/O) signals for each element, according to an embodiment of the present invention;

FIG. 4 illustrates a null width adjustment unit and a signal extraction unit engaged with a zoom control unit, such as in the sound zoom apparatus of FIG. 2, according to an embodiment of the present invention;

FIG. 5 illustrates a signal synthesis unit in a sound zoom apparatus, such as that of FIG. 2, according to an embodiment of the present invention; and

FIGS. 6A and 6B illustrate polar patterns showing a null width adjustment function according to a null width adjustment parameter, such as in the sound zoom apparatus of FIG. 2, according to embodiments of the present invention.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. In this regard, embodiments of the present invention may be embodied in many different forms and should not be construed as being limited to embodiments set forth herein. Accordingly, embodiments are merely described below, by referring to the figures, to explain aspects of the present invention.

In general, directivity signifies a degree of direction for sound devices, such as a microphone or a speaker, indicating a better sensitivity with respect to sound in a particular direction. The directivity has a different sensitivity according to the direction in which a microphone is facing. The width of a directivity pattern showing the directivity characteristic is referred to as a directivity width. In contrast, the width of a portion where the sensitivity in the directivity pattern is very low, because the directivity is limited, is referred to as a null width. The directivity width and the null width have a variety of adjustment parameters. The directivity width and the null width, which are sensitivities to a target sound for a microphone, for example, can be adjusted by adjusting these parameters.

Accordingly, according to one or more embodiments of the present invention, in the adjustments of the directivity width and the null width, it is relatively easier to adjust the null width than the directivity width. That is, it has been found that when a target signal is controlled by adjusting the null width, a better effect is produced than by the adjustment of the directivity width. Thus, according to one or more embodiments, there is a desire to implement a sound zoom function according to the distance by engaging with the zoom function of motion picture photographing by using the null width adjustment rather than by using the directivity width adjustment.

FIGS. 1A and 1B respectively illustrate different potential environments. In FIG. 1A, it is assumed that a digital camcorder device recording sound is placed at the illustrated center, a target sound is located at a far-field distance, and an interference noise is located at a near-field distance. In contrast, in FIG. 1B, the target sound is located at a near-field distance and the interference noise is located at a far-field distance with respect to the digital camcorder. In FIGS. 1A and 1B, the illustrated digital camcorder device is equipped with two microphones. That is, as shown in FIG. 1C, to implement a sound zoom function according to an embodiment, two microphones, e.g., a front microphone and a side microphone, are installed in the digital camcorder device for capturing and recording sounds. As illustrated, the example microphones are arranged to record both a front sound and a lateral sound, with respect to a zoom lens of the digital camcorder, for example.

Here, in an embodiment, the zoom lens of the digital camcorder device of FIG. 1A is operated in a tele-view mode to photograph an object at a far-field distance. In order to cope with the photographing of the far-field object with respective sound, the microphones of the digital camera may desirably be able to record the far-field target sound while removing near-field interference noise. In contrast, in the environment of FIG. 1B, the zoom lens of the digital camcorder device is operated in a wide-view mode to photograph an object at a near-field distance. In order to cope with the photographing of the near-field object with respective sound, the microphones of the digital camera may desirably be able to record the near-field target sound while removing far-field interference noise.

FIG. 2 illustrates a sound zoom apparatus, according to an embodiment of the present invention. Herein, the term apparatus should be considered synonymous with the term system, and not limited to a single enclosure or all described elements embodied in single respective enclosures in all embodiments, but rather, depending on embodiment, is open to being embodied together or separately in differing enclosures and/or locations through differing elements, e.g., a respective apparatus/system could be a single processing element or implemented through a distributed system, noting that additional and alternative embodiments are equally available.

Referring to FIG. 2, the sound zoom apparatus, according to an embodiment, may include a signal input unit 100, a null width adjustment unit 200, a signal extraction unit 300, a signal synthesis unit 400, and a zoom control unit 500, for example.

The signal input unit 100 may receive signals of each of various sounds around an apparatus, such as the apparatus performing the sound zoom function. Here, in an embodiment, the signal input unit 100 can be formed of a microphone array to easily process a target sound signal after receiving the sound signals via a plurality of microphones. For example, the microphone array can be an array with omni-directional microphones having the same directivity characteristic in all directions or an array with heterogeneous microphones with directivity and non-directivity characteristics. In this and the following embodiments, solely for simplification of explanation it will be assumed that two microphones are arranged in an apparatus with a sound zoom function, similar to that of the embodiment of FIG. 1C. However, for example, since the directivity characteristic can also be controlled by implementing an array with a plurality of microphones, it should be understood that four or more microphones can also be arranged to adjust the null width of a microphone array, again noting that alternatives are equally available.

The null width adjustment unit 200 may generate a signal from which a target sound has been removed by adjusting a null width that restricts a directivity sensitivity with respect to a sound signal input to the signal input unit 100. That is, in an embodiment, when a zoom lens is operated to photograph a far-field object, a sound zoom control signal may accordingly restrict the directivity sensitivity to a near-field sound so that a far-field sound can be recorded. In contrast, when the zoom lens is operated to photograph a near-field object, a sound zoom control signal may accordingly restrict the directivity sensitivity to a far-field sound so that a near-field sound can be recorded. However, in an embodiment, in the recording of a near-field sound, the directivity sensitivity to the far-field sound may be restricted not through the adjustment of null width but by considering the sounds input through the microphone array as the near-field sound. This is because in such an embodiment the level of the near-field sound is generally greater than that of the far-field sound and it may be acceptable to regard the input sound as the near-field sound and not process the input sound.

The signal extraction unit 300 may extract a signal corresponding to the target sound by removing signals other than the target sound from the sound signals input to the microphone array, e.g., based on the signal generated by the null width adjustment unit 200. In detail, in such an embodiment, when a signal from which the target sound has been removed is generated by the null width adjustment unit 200, the signal extraction unit 300 estimates the generated signal as noise. Then, the signal extraction unit 300 may remove the signal estimated as noise from the sound signals input to the signal input unit 100 so as to extract a signal relating to the target sound. Since the sound signals input to the signal input unit 100 include sounds around the corresponding sound zoom apparatus in all directions, including the target sound, a signal relating to the target sound can be obtained by removing noise from these sound signals.

Accordingly, in an embodiment, the signal synthesis unit 400 may synthesize an output signal according to a zoom control signal of the zoom control unit 500, for example, based on the target sound signal extracted by the signal extraction unit 360 and a residual signal where the target sound is not included. Here, when the far-field sound is to be obtained, the signal extraction unit 300 may consider the far-field sound and the near-field sound as the target sound and the residual signal, respectively, and output both sounds, and the signal synthesis unit 400 may combine both signals according to the zoom control signal to synthesize a final output signal. For example, when the far-field sound is to be obtained as described above, the percentage of the target sound signal to be included in the synthesized output signal may be about 90% and the percentage of the residual signal to be included in the synthesized output signal may be about 10%. Such synthesis percentages can vary according to the distance between the target sound and the sound zoom apparatus and can be set based on the zoom control signal, for example, as output from the zoom control unit 500. Although the signal extraction unit 300 may extract a target sound signal desired by a user, the target sound signal may be more accurately synthesized by the signal synthesis unit 400 according to the zoom control signal, according to an embodiment of the present invention.

In such an embodiment, the zoom control unit 500 may, thus, control the obtaining of a signal relating to the target sound located a particular distance from the sound zoom apparatus to implement sound zoom and transmit a zoom control signal relating to the target sound to the null width adjustment unit 200 and the signal synthesis unit 400. The zoom control signal may therefore enable the obtaining of sound by reflecting information about the distance to where the target sound or the object to be photographed is located. The zoom control unit 500 can be set to be engaged along with control of the zoom lens for photographing and can independently transmit a control signal by reflecting the information about the distance to where the sound is located only for the obtaining of sound, for example. In the former case, when the zoom lens is operated to photograph a far-field object, the sound zoom may be controlled to record a far-field sound. In contrast, when the zoom lens is operated to photograph a near-field object, the sound zoom may be controlled to record a near-field sound.

FIG. 3 illustrates a sound zoom apparatus, such as the sound zoom apparatus of FIG. 2, in which input/output (I/O) signals are added to each element. Referring to FIG. 3, an example front microphone and an example side microphone may represent a microphone array corresponding to the signal input unit of FIG. 2, for example. Here, although a first-order differential microphone structure formed of only two microphones is discussed with reference to FIG. 3, it is also possible to use a second-order differential microphone structure, such a structure including four microphones, and processing an input signal using two example pairs each having two microphones or a higher order differential microphone structure including a larger number of microphones.

When the structure of FIG. 3 is described with respect to the I/O signals, the null width adjustment unit 200 may receive signals input through/from two microphones and output two types of signals, which respectively include a reference signal from which a target sound has been removed using a beam-forming algorithm and a primary signal including both background noise and the target sound, to the signal extraction unit 300. In general, the microphone array formed of two or more microphones, for example, functions as a filter capable of spatially reducing noise when the directions of a desired target signal and a noise signal are different from each other, by improving an amplitude of received signals by giving an appropriate weight to each of the received signals in the microphone array so as to receive a target signal mixed with background noise at a high sensitivity. This sort of spatial filter should be referred to as beam forming.

The signal extraction unit 300 may, thus, extract a far-field signal relating to a far-field sound and a near-field signal relating to a near-field sound by using a noise removal technology, such as that described above with reference to FIG. 2, for example. The signal synthesis unit 400 may further synthesize the two example signals received from the signal extraction unit and generate an output signal.

FIG. 4 illustrates a null width adjustment unit 200 and a signal extraction unit 300, such as that of FIG. 2, which may also be engaged with the zoom control unit in the sound zoom apparatus of FIG. 2.

In an embodiment, a first-order differential microphone structure, through which directivity is implemented, may be formed of two non-directivity microphones, e.g., the front and side microphones, as illustrated in FIG. 4. Adjustment parameters that can control the null width of the microphone array may include the distance between the microphones forming the microphone array and a delay of the signals input to the microphone array. As an example, in regard to the adjustment parameters, an embodiment in which adjusting of the null width of the target sound through adaptive delay adjustment will be described in greater detail below.

In order to amplify or extract the target signal from different directional noise, a phase difference between an array pattern and the signals input to the microphones are desirably obtained. In an embodiment, in the null width adjustment unit 200 of FIG. 4, a delay-and-subtract algorithm is used as the beam-forming algorithm which is described below.

The null width adjustment unit 200 of FIG. 4 may include a low pass filter (LPF) 220 and a subtractor 230, for example. An example directivity pattern of a sound signal input from the differential microphone structure to the null width adjustment unit 200 can be represented as follows. When the distance between the microphones is d, an acoustic pressure field considering the wavelength and incident angle when a front microphone signal X1(t) and a side microphone signal X2(t) may be input as expressed by be below Equation 1, for example.

E 1  ( w , θ ) =   P 0   - j  ( kd   cos   θ )  ( 1 -  - j  ( w   τ - kd   cos   θ ) )  ≈  P 0  w  ( τ - d   cos   θ / c ) =  P 0  w  ( τ + d

Download full PDF for full patent description/claims.




You can also Monitor Keywords and Search for tracking patents relating to this Sound zoom method, medium, and apparatus patent application.

Patent Applications in related categories:

20130121505 - Microphone array configuration and method for operating the same - An apparatus comprises a plurality of microphone units including at least a first microphone unit and a second microphone unit, each of the first and second microphone units comprising a microphone, an analog-to-digital converter, and a local memory. The microphone is configured to capture an analog audio signal. The analog-to-digital ...

20130121504 - Microphone array with daisy-chain summation - Microphone stages in a microphone array may be coupled together in a daisy chain. Each stage may include a microphone, an analog to digital converter, a decimation unit, a receiver, an adder, and a transmitter. The converter may convert analog audio microphone signals into digital codes that may be decimated. ...


###


Other recent patent applications listed under the agent Samsung Electronics Co., Ltd.: