T 0979/06 21-09-2010
Download and more information:
Method and apparatus for reducing flicker in a video image sequence
Claims - clarity (yes)
Inventive step (yes) after amendment
Patentable inventions - computer programs
I. The appeal is against the decision of the examining division to refuse European patent application No. 99 937 490.3.
II. The decision to refuse was based on the grounds that claims 1 and 6 then on file were not clear (Article 84 EPC 1973) and that their subject-matter did not involve an inventive step (Article 56 EPC 1973) having regard to the prior art documents
D1: US 5 243 422 A and
D2: US 5 631 706 A.
III. The applicant appealed. With the statement of grounds of appeal the appellant filed claims according to a sole main request.
IV. The board issued a communication pursuant to Article 15(1) of the Rules of Procedure of the Boards of Appeal (RPBA) annexed to a summons to oral proceedings and dated 3 December 2008.
V. With a letter dated 5 February 2009 the appellant filed claims according to a new main and first and second auxiliary request, respectively. The appellant also submitted arguments in support of its view that the application was in order for grant.
VI. Claim 1 of the main request reads as follows:
"A method of reducing luminance flicker in a video image sequence comprising a plurality of video images produced by a camera, comprising:
digitally filtering on a per pixel basis, using a finite impulse response filter of maximum length N+1, temporally successive video images of the video image sequence; said digitally filtering comprising:
adjusting a filter length, measured in terms of a number of pixel values, corresponding to a current image pixel value of the finite impulse response filter to increase the filter length if an absolute difference between the current image pixel value and a correspondingly spatially disposed previous pixel value of respective current and previous temporally spaced successive images of the video image sequence is less than a threshold value up to the maximum filter length of N+1, and otherwise setting the filter length to one, measured in terms of a number of pixel values."
Claim 6 of the main request reads as follows:
"A system comprising:
a finite impulse response filter of maximum length N+1 to temporally filter temporally successive video images of a video image sequence comprising luminance flicker to reduce said luminance flicker in said video image sequence;
a buffer to store a filter length, measured in terms of number of pixel values, for each pixel, and
means, responsive to an absolute difference between a current image pixel value and a correspondingly spatially disposed previous pixel value of respective current and previous temporally spaced successive images of the video image sequence, to, if the absolute difference is less than a threshold value, increase the filter length, measured in terms of a number of pixel values, up to the maximum filter length of N+1 and otherwise to set the filter length to one."
Claim 11 of the main request reads as follows:
"A computer program comprising computer program code means adapted to perform all the steps of claims 1 to 4 when the program is run on a computer."
Claims 2 to 5, 7 to 10 and 12 are dependent claims.
VII. In a further letter dated 3 March 2009 the appellant announced that it would not be represented at the oral proceedings.
VIII. Oral proceedings were held on 5 March 2009 in the absence of the appellant in application of Rule 71(2) EPC 1973 and Article 15(3) RPBA. The board noted that the appellant had requested in writing that a patent be granted on the basis of the main request comprising claims 1 to 12 as filed with the letter dated 5 February 2009, or, if the board was unable to provide satisfaction under the main request, that a patent be granted on the basis of the first or second auxiliary request, all filed with the letter dated 5 February 2009. At the end of the oral proceedings, the chairman announced that the proceedings would be continued in writing.
IX. In a communication dated 26 March 2009 the board informed the appellant that it had formed the opinion that the subject-matter of claims 1 to 10 of the main request was patentable. However, since claims 11 and 12 of the main request were directed to a computer program, the board intended to stay the appeal proceedings until the opinion of the Enlarged Board of Appeal in case G 3/08 was available, as long as claims 11 and 12 were included in the claim set of the main request.
X. With a letter dated 14 August 2009 the appellant filed claims according to a new first, second and third auxiliary request.
XI. The opinion of the Enlarged Board of Appeal in case G 3/08 was issued on 12 May 2010.
XII. The decision under appeal may be summarised as follows:
The expression "filter length" was not clear without specifying that the filter was a digital filter using delay lines and taps corresponding to a particular length of the sum of the delay lines used. In particular a filter length of one was not clear because in the context of video processing different time units were common. Moreover the expression "previous pixel value" was not clear because "previous" could relate to different directions corresponding to the three-dimensional character of a video sequence signal space.
According to the decision under appeal, the subject-matter of claim 1 then on file differed from the method disclosed in D1 mainly in that the filter length was increased if an absolute difference between a current image pixel value and a previous image pixel value was less than a predetermined threshold. D1 also disclosed an adaptive filtering wherein the adaptation was effected in accordance with detection, such as a movement detection or a track jump detection. Hence the adaptation implied comparisons of pixel values with thresholds. The features of claim 1 were rendered obvious by D1 in conjunction with the knowledge of a person skilled in the art. The difference between the flicker considered in the application and the flicker considered in D1 was not relevant because the input signal was not part of the subject-matter claimed.
Furthermore, an adaptive filter with adjustable filter lengths was known from D2. D2 (column 9, lines 32 to 41) also suggested the storage of filter values for spatial pixel positions. Hence the features of claims 1 and 6 were also rendered obvious by D1 in combination with D2.
XIII. The appellant's arguments may be summarised as follows:
The term "filter length" corresponded to the number of samples, i.e. the number of pixel values, processed by the digital filter at any one time. Another way of considering this was that the filter length corresponded to the number of taps or weights constituting the filter. "Adjusting" the filter length comprised changing the number of samples used to realise the filter. A filter length of "one" corresponded to a filter length of one pixel value. The "current pixel value" and the "previous pixel value" related to correspondingly spatially disposed pixel values within a current image and a previous image of the temporally successive video images respectively. The embodiments operated on a pixel-by-pixel basis, and the flicker addressed by embodiments of the invention was flicker in the video sequence. This flicker inherent to the video image sequence was to be contrasted with a video image sequence that did not comprise inherent flicker but which could exhibit flicker when processed by a video tape recorder under other than normal playback speed conditions.
D1 was directed to an interpolation filter for video signals whose purpose was to correct positional displacements without adding colour flicker, and not to reduce flicker already present in the video image sequence. D1 operated using analogue signals as opposed to pixel values and operated on a scan-line or field basis as opposed to a pixel-by-pixel basis.
D2 was directed to a converter for converting video signals of an interlace format to video signals of progressive format. The converter included an adaptive temporal filter which filtered the video signal of the progressive format such that as the degree of image movement in images represented by the video signal of the progressive format increased, a lesser degree of temporal filtering was performed.
Both D1 and D2 failed to disclose the adaptive filtering techniques recited in claims 1 and 6. Both D1 and D2 addressed completely different technical problems from each other and from those addressed by embodiments of the present invention.
1. The appeal is admissible.
2. Main request: amendments (Article 123(2) EPC)
2.1 The features of claim 1 are mainly disclosed in claims 1, 3 and 4 as originally filed. The feature that a plurality of video images is produced by a camera is disclosed on page 2, line 27 to page 3, line 10 of the description as originally filed. The feature of a finite impulse response filter of maximum length N+1 is disclosed on page 4, lines 7 and 8 and page 5, lines 12 and 13 of the description as originally filed. The feature that the filter length is measured in terms of a number of pixel values is disclosed on page 5, lines 12 to 20 in conjunction with page 4, lines 18 to 21 of the description as originally filed. The feature that an absolute difference between the current image pixel value and a correspondingly spatially disposed previous pixel value of respective current and previous temporally spaced successive images of the video image sequence is considered for the filter length adjustment is disclosed on page 4, lines 16 to 28 of the description and in figure 3 as originally filed.
2.2 The features of the system of claim 6 are disclosed in the same parts of the application. In particular, the feature of a buffer to store a filter length is disclosed on page 4, line 25 of the description as originally filed.
2.3 The features of claims 11 and 12 are disclosed on page 5, line 24 to page 6, line 3 of the description as originally filed.
2.4 The features of dependent claims 2 to 5 are disclosed in claims 5 and 6 as originally filed and on page 5, lines 1 to 5 and page 3, lines 8 to 10 of the description as originally filed, respectively. The features of dependent claims 7 to 10 are disclosed in claims 7, 3 and 6 and on page 5, lines 1 to 5 of the description as originally filed, respectively.
3. Main request: clarity (Article 84 EPC 1973)
3.1 Claim 1 specifies that a finite impulse response filter of maximum length N+1 is used to digitally filter video images on a per pixel basis. The filter length, which is measured in terms of a number of pixel values, is adjusted corresponding to a current image pixel value. Hence the filter length specified in claim 1 is consistent with the normal meaning of the length of a digital finite impulse response filter and corresponds to the number of samples processed by the digital finite impulse response filter at any one time, i.e. the number of taps or (non-zero) weighting coefficients constituting the digital finite impulse filter. A filter length of one thus has the technical meaning that no other pixel value than the current pixel value influences the digital filter output for the current pixel. Claim 1 also makes clear that the filter length is adjusted, within the limits imposed by the maximum length of the digital finite impulse response filter, according to the criterion specified in claim 1. Hence the board finds that the objection raised in the decision under appeal against the expression "filter length" does not apply to the present amended claims.
3.2 For a given current pixel, the criterion specified in claim 1 refers to the absolute pixel value difference between the current pixel and the correspondingly spatially disposed pixel in the previous image, whereby "previous" relates to the temporally spaced successive images of the video image sequence. Hence the board finds that the objection raised in the decision under appeal against the expression "previous pixel value" does not apply to the present amended claims.
3.3 Furthermore, as far as the present application documents including the amended claims according to the main request are concerned, the board does not see any other reason for an objection under Article 84 EPC 1973.
4. Main request: claims 11 and 12 relating to computer programs (see Article 52(2)(c) and (3) EPC)
4.1 The opinion of the Enlarged Board of Appeal in case G 3/08 (to be published in the OJ EPO) found inter alia that there was no divergence between decisions T 424/03 dated 23 February 2006 and T 1173/97 (OJ EPO 1999, 609) which would make the referral to the Enlarged Board admissible (see Headnote points 6 and 7). Furthermore the Enlarged Board in its analysis held that the definition of the "further technical effect" given in decision T 1173/97 stood as the established case law (see G 3/08, points 10.3 and 10.4, in particular point 10.4, in fine). In its opinion, the Enlarged Board of Appeal came to the conclusion that the referral by the President of the EPO was inadmissible under Article 112(1)(b) EPC.
4.2 Hence the board follows the approach concerning computer programs which has become established case law. In particular the board has examined whether the subject-matter of claims 11 and 12 has a technical character because the claimed program, when run on a computer, causes a "further technical effect" (see T 1173/97, point 9.4). The computer programs of claims 11 and 12 have the effect, when run on a computer, that luminance flicker in a video image sequence comprising a plurality of video images produced by a camera (and input into the computer) is reduced. The board finds that this effect is a "further technical effect". Hence the computer programs of claims 11 and 12 are not objectionable as being excluded from patentability under Article 52(2)(c) and (3) EPC.
5. Main request: novelty (Article 54(1) EPC 1973)
5.1 Document D1
5.1.1 D1 relates to a field converting method for generating new field data from arbitrary field data of television signals at a variable speed playback by a magnetic recording reproducing device such as a video tape recorder. It is concerned with the problem that, when image signals recorded in a video tape recorder are reproduced, displacements are produced in images at vertical transitions (see figure 2) of the luminance signal if image signals of a field are skipped or repeatedly reproduced to achieve slow playback, still picture playback, or fast playback (see column 1, lines 7 to 18). Specifically, in the case where a still picture is reproduced in the variable speed playback, transitions of the luminance signal may be displaced in the vertical direction in a manner which is easily observable by sight. This may cause a significant worsening in the image quality (see column 3, lines 21 to 31). D1 attempts to solve this problem by performing an intra-field interpolation (of only even fields or only odd fields, depending on whether the current field is even or odd) and an inter-field interpolation (of even fields and odd fields), wherein the latter depends on a movement detection signal (k) which indicates movement in the image (see column 10, lines 3 to 48).
5.1.2 D1 does not disclose a method of reducing luminance flicker in a video image sequence comprising a plurality of video images produced by a camera as set out in the present amended claim 1. For the purpose of carrying out the teaching of D1, flicker in the source video image sequence stored on the video tape recorder is irrelevant. Furthermore, D1 does not disclose the adjustment of a filter length dependent on the criterion specified in claim 1, because the movement detection signal (k) in D1 does not correspond to an absolute difference between two pixel values as specified in claim 1 and because according to D1 the filter length is not set to one if the absolute difference between the two pixel values is equal to or above the threshold.
5.1.3 As far as the system of claim 6 is concerned, D1 does not disclose a buffer to store a filter length, measured in terms of number of pixel values, for each pixel. Furthermore, D1 does not disclose means to adjust the filter length dependent on the criterion and in the manner as those specified in claim 6, which are essentially the same as those specified in claim 1.
5.1.4 Hence the method of claim 1 and the system of claim 6 according to the main request are not disclosed in D1.
5.2 Document D2
5.2.1 D2 relates to a converter and a method for converting video signals of interlace format to video signals of progressive format (see column 1, lines 8 to 10). An object of the converter and converting method described in D2 is to perform the conversion so that line flicker in the progressive format output, which may occur especially around stationary lines tilted slightly from the horizontal axis, is suppressed (see column 1, lines 46 to 50 in conjunction with column 6, lines 21 to 30).
5.2.2 D2 does not disclose a method of reducing luminance flicker in a video image sequence comprising a plurality of video images produced by a camera as set out in the present amended claim 1. Furthermore, D2 does not disclose the adjustment of a filter length dependent on the criterion specified in claim 1. According to D2, coefficients a and b given to pixel values are modified, but the filter length is not increased (see the section "Adaptive Temporal Filtering" in column 6, line 35 to column 7, line 18).
5.2.3 As far as the system of claim 6 is concerned, D2 does not disclose a buffer to store a filter length, measured in terms of number of pixel values, for each pixel. Furthermore, D2 does not disclose means to adjust the filter length dependent on the criterion and in the manner as those specified in claim 6, which are essentially the same as those specified in claim 1.
5.2.4 Hence the method of claim 1 and the system of claim 6 according to the main request are not disclosed in D2.
5.3 The board is of the opinion that the other available documents are not more relevant than D1 or D2. Hence the board finds that the subject-matter of claims 1 and 6 according to the main request is new (Article 54(1) EPC 1973).
6. Main request: inventive step (Article 56 EPC 1973)
6.1 As discussed in points 5.1.2 and 5.2.2 above, neither D1 nor D2 discloses a method of reducing luminance flicker in a video image sequence comprising a plurality of video images produced by a camera, as now specified in claim 1. Furthermore, neither D1 nor D2 discloses the adjustment of a filter length dependent on the criterion and in the manner specified in claim 1 of the main request. Specifically setting the filter length to one, measured in terms of a number of pixel values, reduces the amount of filtering across temporal discontinuities in the video image sequence (see page 5, lines 1 to 3).
6.2 Hence the board finds that, having regard to documents D1 and D2, the method of claim 1 according to the main request was not obvious to a person skilled in the art.
6.3 The system of claim 6 according to the main request comprises means to adjust the filter length dependent on the same criterion and which are suitable for filtering a video image sequence essentially in the same manner as those specified in claim 1. The system also comprises a buffer to store the filter length, measured in terms of number of pixel values, for each pixel so that the filter length adjustment can be carried out. Specifically setting the filter length to one, measured in terms of a number of pixel values, reduces the amount of filtering across temporal discontinuities in the video image sequence (see page 5, lines 1 to 3).
6.4 Hence the board finds that, having regard to documents D1 and D2, the system of claim 6 according to the main request likewise was not obvious to a person skilled in the art.
6.5 The arguments as to lack of inventive step given in the decision under appeal were based on claim wordings where general expressions, such as "filter length" and "previous", were not further specified. However, these expressions have been further specified on appeal (see section 3 above). Hence the arguments as to lack of inventive step given in the decision under appeal no longer apply in the context of the amended claims according to the main request. In view of the documents on file, the board comes to the conclusion that the combination of features specified in claim 1 or claim 6 according to the main request was not disclosed or rendered obvious in view of the prior art and the common general knowledge at the priority date. It therefore finds that the subject-matter of claims 1 and 6 according to the main request involves an inventive step (Article 56 EPC 1973).
7. The subject-matter of claims 2 to 5 and 7 to 10 according to the main request is new and involves an inventive step because these claims are dependent on claims 1 and 6, respectively.
8. The computer programs of claims 12 and 13 are new and involve an inventive step because they are adapted to perform all the steps of claim 1 when the program is run on a computer.
9. The board sees no other reason why the claims according to the main request presently on file do not meet the requirements of the EPC.
10. As far as the description and the drawings are concerned, there is a need to clarify the situation. For instance, the decision under appeal was based on page 6 filed with a letter dated 16 November 2000, even though the appellant had filed a new page 6 with a letter dated 3 June 2004. The decision under appeal was also based on drawings "received on 16.11.2000 with letter of 16.11.2000" even though there is no letter of 16 November 2000 on file. On the other hand, when submitting that the application was in order for grant, the appellant has not explicitly specified the description pages and drawings with which the grant of a patent is requested and has not addressed the above discrepancies. Furthermore, the question whether the description should be amended in view of the amended claims could not be discussed with the appellant in the oral proceedings.
ORDER
For these reasons it is decided that:
1. The decision under appeal is set aside.
2. The case is remitted to the first instance with the order to grant a patent with the following claims and a description to be adapted:
Claims:
Nos. 1 to 12 according to the main request filed with the letter of 5 February 2009.