Imagination Technologies LimitedDownload PDFPatent Trials and Appeals BoardDec 21, 20212020005875 (P.T.A.B. Dec. 21, 2021) Copy Citation UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 16/180,181 11/05/2018 Christopher Martin 2645-0428US01 9603 125968 7590 12/21/2021 Potomac Law Group PLLC (IMGTEC) 8229 Boone Boulevard Suite 430 Vienna, VA 22182 EXAMINER PAN, DANIEL H ART UNIT PAPER NUMBER 2182 NOTIFICATION DATE DELIVERY MODE 12/21/2021 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): Eofficeaction@appcoll.com Patents@potomaclaw.com vdeluca@potomaclaw.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte CHRISTOPHER MARTIN and AZZURRA PULIMENO Appeal 2020-005875 Application 16/180,181 Technology Center 2100 Before MICHAEL J. STRAUSS, MINN CHUNG, and ADAM J. PYONIN, Administrative Patent Judges. CHUNG, Administrative Patent Judge. DECISION ON APPEAL STATEMENT OF THE CASE1 Pursuant to 35 U.S.C. § 134(a), Appellant2 appeals from the Examiner’s decision to reject claims 1–4, 11, 15, 17, 18, and 20. See Final Act. 1, 3; Ans. 3, 7. The Examiner has found claims 5–10, 12–14, 16, 1 In this Decision, we refer to Appellant’s Appeal Brief filed April 20, 2020 (“Appeal Br.”) and Reply Brief filed August 12, 2020 (“Reply Br.”); the Examiner’s Final Office Action mailed November 19, 2019 (“Final Act.”) and Answer mailed June 12, 2020 (“Ans.”); and the Specification filed November 5, 2018 (“Spec.”). Rather than repeat the Examiner’s findings and Appellant’s contentions in their entirety, we refer to these documents. 2 “Appellant” herein refers to “applicant” as defined in 37 C.F.R. § 1.42. Appellant identifies the real party in interest as Imagination Technologies Limited. Appeal Br. 1. Appeal 2020-005875 Application 16/180,181 2 and 19 to contain allowable subject matter. Final Act. 13–15; Ans. 9. We have jurisdiction under 35 U.S.C. § 6(b). We REVERSE. CLAIMED SUBJECT MATTER The subject matter of the present application pertains to a “Hardware Unit for Performing Matrix Multiplication With Clock Gating.” Spec. 1:1–2 (Title). Appellant’s Specification purports to describe and claim embodiments that relate to hardware units and methods for “performing matrix multiplication via a multi-stage pipeline,” where “the storage elements associated with one or more stages of the pipeline are clock gated based on the data elements and/or portions thereof that [are] known to have a zero value (or can be treated as having a zero value).” Id., Abstract. Claims 1, 17, 18, and 20 are independent. Claim 1, reproduced below with the disputed limitation emphasized, is representative: 1. A hardware unit to perform a matrix multiplication, the hardware unit comprising: a multiplier stage comprising a plurality of multipliers, each multiplier configured to multiply a first data element and a second data element to produce a multiplication data element; one or more adder stages following the multiplier stage that form an adder tree to produce a sum of the multiplication data elements, each adder stage comprising one or more adders, each adder configured to add at least two data elements output by the multiplier stage or at least two data elements output by a previous adder stage in the adder tree to produce an addition data element; wherein at least one multiplier and/or at least one adder is preceded by a storage element corresponding to each bit of the data elements input to the at least one adder and/or the at least one multiplier; and Appeal 2020-005875 Application 16/180,181 3 control logic configured to clock gate all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of that data element is to be treated as having a zero value. Appeal Br. 16 (Claims App.) (emphasis added). REJECTIONS The Examiner relies on the following references: Designation3 Reference Date Landers US 2002/0002573 Al Jan. 3, 2002 Hirano US 2007/0046804 Al Mar. 1, 2007 Brothers US 2016/0358069 Al Dec. 8, 2016 Claims 1, 2, 11, 15, 17, 18, and 20 stand rejected under 35 U.S.C. § 103 as being unpatentable over the combined teachings of Brothers and Landers. Final Act. 7. Claims 3 and 4 stand rejected under 35 U.S.C. § 103 as being unpatentable over the combined teachings of Brothers, Landers, and Hirano. Final Act. 12. ISSUE4 The issue presented by Appellant’s contentions is whether the Examiner erred in finding that Brothers teaches or suggests “control logic configured to clock gate all or a portion of the storage elements corresponding to a data element in response to determining that all or a 3 All citations herein to the prior art are by reference to the first named inventor only. 4 Appellant’s arguments raise additional issues. Because the identified issue is dispositive of the appeal, we do not reach the additional issues. Appeal 2020-005875 Application 16/180,181 4 portion of that data element is to be treated as having a zero value,” as recited in claim 1 (emphasis added). ANALYSIS Background By way of background, Appellant’s Specification describes that matrix multiplication of two matrices A and B involves multiplying the elements in a row of matrix A with the elements in a column of matrix B and summing the results. Spec. ¶ 1. The Specification also describes a hardware unit to perform matrix multiplication, which comprises (1) a plurality of parallel multipliers which each multiply a first data element (D1) of a first set of data elements and a second data element (D2) of a second set of data elements to produce a multiplication data element and (2) an adder tree to produce a sum of the multiplication data elements. Id. ¶¶ 31, 32, Fig. 1. The multipliers and adders together are called an “arithmetic unit.” Id. ¶ 35. In an embodiment, an arithmetic unit is preceded by a storage element for each bit of the first and second data elements (D1, D2) to be input to the arithmetic unit. Id. ¶ 35, Fig. 2. The storage elements may be implemented as flip-flops, or any other form of register or memory cell. Id. ¶ 35. The Specification describes that the power consumed by a storage element is based on the dynamic power consumed by a storage element when the storage element is “clocked” to store the value to be input to the arithmetic unit. Id. ¶ 36. According to the Specification, in many matrix multiplications one or more data elements may be known in advance to have a zero value (or can be treated as having a zero value). Id. ¶ 37. In such circumstances, instead Appeal 2020-005875 Application 16/180,181 5 of storing such data elements in storage elements to send zero values for multiplication, zero values can be directly supplied to the arithmetic unit without “clocking” the storage elements to store the zero values. Id. By “not clocking” a storage element when a bit is known to be zero (or can be treated as being zero) the power consumption of the storage element and the overall hardware unit is reduced. Id. This can be achieved through a technique referred to as “clock gating,” where a circuit is used to disable the clock controlling the storage elements. Id. Figure 3 of Appellant’s Specification is reproduced below. Figure 3 is a block diagram of the storage elements and arithmetic unit with clock gating control logic. Id. ¶ 16. Figure 3 illustrates an example arithmetic unit (e.g., multiplier 110 or adder 112) which performs an arithmetic operation (e.g., multiplication or addition) on a first data element (D1) and a second data element (D2), where Appeal 2020-005875 Application 16/180,181 6 the first and second data elements (D1 and D2) are provided to the arithmetic unit via a set of storage elements 202. Id. ¶ 39. As shown in Figure 3, the clocking of the storage elements 202 is controlled by clock gating control logic 302. Id. Specifically, the clock gating control logic 302 comprises hardware logic configured to “clock gate” (i.e. not clock) all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of the data element has a zero value (or can be treated as having a zero value). Id. Figure 4 of the Specification is reproduced below. Figure 4 is a block diagram of an example implementation of the clock gating control logic of Figure 3. Id. ¶ 17. As shown in Figure 4, clock gating control logic 400 comprises gating logic units 402, AND gates 404, and gate signal generation logic 406. Id. ¶ 44. The Specification describes that clock gating control logic 400 Appeal 2020-005875 Application 16/180,181 7 receives an element gate signal, for each data element to be processed by arithmetic unit 110 or 112, that indicates whether that data element can be treated as having a zero value and thus should be “clock gated” (i.e. not clocked). Id. Each gating logic unit 402 is configured to receive an input clock signal (CLK) and the corresponding element gate signal and to provide an output clock signal to the clock input of each corresponding storage element such that the storage element is “not clocked” when the element gate signal indicates that the corresponding data element has a zero value. Id. ¶¶ 44, 48. The element gate signal is also provided to each AND gate 404, bypassing storage elements 202, such that if a data element has a zero value, the output of AND gate 404 provides zeros directly to arithmetic unit 110 or 112. Id. ¶ 51. Discussion In rejecting claim 1, the Examiner finds that Brothers teaches or suggests most limitations of claim 1, including “control logic configured to clock gate all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of that data element is to be treated as having a zero value.” Final Act. 7–8. In particular, the Examiner finds that Figure 16 and related text of Brothers teach this limitation. Id. at 8 (citing Brothers ¶¶ 35, 36, 135, 136, Fig. 16). According to the Examiner, the AND mask, input 0’s mask, and weight 0’s mask illustrated in Figure 16 of Brothers teach “control logic,” which is “configured to clock gate all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of that data element is to be treated as having a zero value.” Id. Appeal 2020-005875 Application 16/180,181 8 Appellant argues that Brothers does not teach the claimed “control logic” because “in Brothers there is no clock gating of any storage elements in memory.” Appeal Br. 5–6. According to Appellant, as well-known to those having ordinary skill in the art, “clock gating” is a technique that “suppresses or disables the clock signal in certain cycles,” which “effectively turns off the clock in certain desired cycles so that the electronic component will not take any action during those cycles.” Id. at 7 (emphases added). Appellant further explains that in the claimed invention, “a clock signal controlling a storage element is only gated or suppressed when the input to the storage element is zero or can be treated as being zero,” which “effectively turn[s] off the storage element in some clock cycles,” “reduc[ing] the power consumption of the storage element and of the overall hardware unit.” Reply Br. 2. According to Appellant, the masks of Brothers identify which of the inputs are zero and which of the weights are zero such that only non-zero input values are sent to the multiply-accumulate units (MAUs). Appeal Br. 5–6 (citing Brothers ¶¶ 132, 136, 139). In other words, the masks of Brothers operate such that “the positions containing all zero input data or all zero weight values are simply skipped,” which reduces the “idle cycles” of the MAUs. Id. (second emphasis added) (citing Brothers ¶¶ 136, 139). Appellant asserts that Brothers’s masks are not equivalent to the “control logic” recited in claim 1 because the masks do not “clock gate all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of that data element is to be treated as having a zero value.” Id. at 6 (emphasis added). We agree with Appellant. Appeal 2020-005875 Application 16/180,181 9 First, we agree with Appellant that “clock gating . . . storage elements,” as recited in claim 1, requires disabling the clock of the storage elements or not clocking the storage elements. The Specification describes that As is known to those of skill in the art, ‘clock gating’ is the technique wherein a circuit . . . is used to disable the clock controlling a circuit under certain conditions. By not ‘clocking’ a storage element when a bit is known to be zero (or can be treated as being zero) the number of state transitions for that storage element is reduced which reduces the power consumption of the storage element and the overall hardware unit. Spec. ¶ 37 (emphases added). These sentences indicate that Appellant was acting as a lexicographer and defining the term “clock gating” as is known to those ordinarily skilled in the art. The form of the first sentence is definitional in that it uses the word “clock gating” set off by quotation marks followed by a definitional description. See Sinorgchem Co., Shandong v. Int’l Trade Comm’n, 511 F.3d 1132, 1136 (Fed. Cir. 2007) (A term set off by quotation marks is often “a strong indication that what follows is a definition.”). The Specification also describes that the clock gating control logic 302 comprises hardware logic configured to clock gate (i.e. not clock) all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of the data element has a zero value (or can be treated as having a zero value). Spec. ¶ 39 (emphases added). The use of “i.e.” immediately following the words “clock gate” in this sentence indicates Appellant’s intent to define the term “clock gate” to mean “not clock.” See Edwards Lifesciences LLC v. Cook Inc., 582 F.3d 1322, 1334 (Fed. Cir. 2009) (“[U]se of ‘i.e.’ signals an Appeal 2020-005875 Application 16/180,181 10 intent to define the word to which it refers.”). This definition is also consistent with paragraph 37 discussed above, which defines “clock gating” a storage element as “not ‘clocking’” a storage element. Spec. ¶ 37. In addition, the definitions set forth in paragraphs 37 and 39 are consistent with the disclosures of the rest of the Specification. See, e.g., id. ¶¶ 44, 48, 51, Figs. 3, 4. In view of these express definitions provided in the Specification, which are consistent with the disclosures in the Specification as a whole and what is known to those ordinarily skilled in the art, we agree with Appellant’s interpretation of the claim term “clock gat[ing]” a storage element to mean “disabling the clock” of a storage element or “not clocking” a storage element. See In re Smith Int’l, Inc., 871 F.3d 1375, 1382–83 (Fed. Cir. 2017) (internal citations omitted) (“The correct inquiry in giving a claim term its broadest reasonable interpretation in light of the specification is . . . an interpretation that corresponds with what and how the inventor describes his invention in the specification, i.e., an interpretation that is ‘consistent with the specification.’”). We also agree with Appellant that, under this interpretation, Brothers does not teach or suggest “control logic configured to clock gate all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of that data element is to be treated as having a zero value,” as recited in claim 1. Appeal 2020-005875 Application 16/180,181 11 Figure 16 of Brothers relied upon by the Examiner is reproduced below. Figure 16 is a diagram illustrating exemplary engine 1600 for processing one or more classification layers of a neural network. Brothers ¶ 131. As shown in Figure 16, “engine 1600 includes memory 1602 storing input data and an input-0 mask (e.g., an alignment mask) and memory 1604 storing weights-0 mask (e.g., a bitmask such as the weight mask).” Id. In rejecting claim 1 over the combination of Brothers and Landers, the Examiner maps memory 1602 (storing input data) of Brothers as the recited “storage elements” (Ans. 12) and identifies the AND mask, input 0’s mask, and weight 0’s mask illustrated in Figure 16 of Brothers as the claimed “control logic,” which is “configured to clock gate all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of that data element is to be treated as having a zero value” (Final Act. 8). The Examiner, however, does not explain adequately how Brothers’s AND mask, input 0’s mask, and weight 0’s mask disable the Appeal 2020-005875 Application 16/180,181 12 clock signal to Brothers’s memory 1602 or control clocking of memory 1602 to “not clock” the memory. As described in the paragraphs cited by the Examiner, Brothers describes that the masks cause the engine depicted in Figure 16 to “skip or suppress the input or the weight with 0 value.” Final Act. 9 (emphasis added) (citing Brothers ¶¶ 35, 36). Brothers describes that as a result of applying the masks to input data stored in memory 1602, only “non-0 values are presented to the MAU 1626-1640 inputs each cycle and idle cycles are reduced.” Brothers ¶ 136 (emphases added). According to the Examiner, AND mask 1606 “performs an AND operation on the input data” stored in memory 1602 and the weight control input 1604 to “skip or suppress the input or the weight with 0 value.” Ans. 11–12 (citing Brothers ¶¶ 35, 36, 135, 136). The Examiner does not explain sufficiently, however, how the AND operation performed on the input data stored in memory 1602 and the data of the masks—i.e., a data operation involving the bits stored in memory 1602 and the bits of the masks (AND’ing the bits)—teaches or suggests “disabling the clock” to memory 1602 or “not clocking” memory 1602. The Examiner does not identify, nor do we discern, any disclosure in Brothers that shows that the masks control the clock or the clocking of memory 1602 to disable clocking of the memory or “not clock” the memory. Instead, the Examiner states that “the issue here is not how to clock the input into memory but rather how to clock (e.g. by AND gate) the input after the memory and before the arithmetic unit (e.g. multipliers and adders).” Ans. 11 (emphases added). But this is not what is recited in claim 1. Rather, claim 1 recites that “control logic” is configured to “clock Appeal 2020-005875 Application 16/180,181 13 gate,” i.e., disable the clock or not clock, “all or a portion of the storage elements” for a data element “in response to determining that all or a portion of that data element is to be treated as having a zero value.” The Examiner appears to take the position that the AND gate operation performed by the masks of Brothers is equivalent to the function performed by AND gate 404 depicted in Figure 4 of the Specification. Ans. 11–12 (citing Spec. ¶ 51, Fig. 4). The Examiner explains that AND gate 404 is a part of the clock gating control logic and “the critical component of the circuit” because “without it there would be no clock gating control.” Id. at 11. Appellant argues that “AND gates 404 of Fig. 4 do not perform clock gating.” Reply Br. 3. Appellant asserts that “[i]nstead, it is the gate logic units 402 in FIG. 4 that perform the clock gating of the storage elements.” Id. (emphasis added). As for AND gates 404 cited by the Examiner, according to Appellant, the gates “are configured to provide a zero to the arithmetic unit (i.e. multiplier 110 or adder 112) when the corresponding storage element is zero.” Id. (emphasis added) (citing Spec. ¶ 51). We agree with Appellant. As discussed in the Background section above, Figure 4 of the Specification shows that clock gating control logic 400 comprises gating logic units 402, AND gates 404, and gate signal generation logic 406. Spec. ¶ 44. The Specification describes that each gating logic unit 402 is configured to receive an input clock signal (CLK) and the corresponding element gate signal and to provide an output clock signal to the clock input of each corresponding storage element such that the storage element is “not clocked” when the element gate signal indicates that the corresponding data element has a zero value. Id. ¶¶ 44, 48. Appeal 2020-005875 Application 16/180,181 14 Regarding AND gates 404, the Specification describes that each AND gate 404 operates to provide zeros directly to the arithmetic unit if the corresponding data element has a zero value. Id. ¶ 51. As shown in Figure 4 (reproduced above in the Background section), AND gates 404 simply have no connection pathway to the clock inputs of storage elements 202 and, therefore, do not and cannot “clock gate” or “not clock” the storage elements. In sum, we are persuaded the Examiner erred in finding that Brothers teaches or suggests “control logic configured to clock gate all or a portion of the storage elements corresponding to a data element in response to determining that all or a portion of that data element is to be treated as having a zero value,” as recited in claim 1. As the Examiner does not rely on the other cited references for this limitation, we are persuaded the Examiner’s rejection of claim 1 is in error. Accordingly, we agree the Examiner erred, and, therefore, do not sustain the Examiner’s rejection of claim 1 or the rejection of independent claims 17, 18, and 20, which include the same limitation as the disputed limitation of claim 1. Nor do we sustain the rejection of dependent claims 2–4, 11, and 15, each of which stands with its respective base claim. CONCLUSION The decision of the Examiner to reject claims 1–4, 11, 15, 17, 18, and 20 is reversed. Appeal 2020-005875 Application 16/180,181 15 DECISION SUMMARY In summary: Claim(s) Rejected 35 U.S.C. § Reference(s)/Basis Affirmed Reversed 1, 2, 11, 15, 17, 18, 20 103 Brothers, Landers 1, 2, 11, 15, 17, 18, 20 3, 4 103 Brothers, Landers, Hirano 3, 4 Overall Outcome 1–4, 11, 15, 17, 18, 20 REVERSED Copy with citationCopy as parenthetical citation