Ex Parte EyalDownload PDFPatent Trial and Appeal BoardJun 16, 201611811857 (P.T.A.B. Jun. 16, 2016) Copy Citation UNITED STA TES p A TENT AND TRADEMARK OFFICE APPLICATION NO. FILING DATE FIRST NAMED INVENTOR 111811,857 06/11/2007 AvivEyal 51518 7590 06/20/2016 MA YER & WILLIAMS PC 928 Mountain A venue Second Floor Mountainside, NJ 07092 UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www .uspto.gov ATTORNEY DOCKET NO. CONFIRMATION NO. 50W8649.0l 7845 EXAMINER HENRY, MARIEGEORGES A ART UNIT PAPER NUMBER 2455 NOTIFICATION DATE DELIVERY MODE 06/20/2016 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address( es): docket@mwpatentlaw.com mwolf@mwpatentlaw.com kwilliams@mwpatentlaw.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte A VIV EY AL 1 Appeal2015-000097 Application 11/811,857 Technology Center 2400 Before MICHAEL J. STRAUSS, JOHN F. HORVATH, and AMBER L. HAGY, Administrative Patent Judges. HAGY, Administrative Patent Judge. DECISION ON i\.PPEi\L STATEMENT OF THE CASE Appellant appeals under 35 U.S.C. § 134(a) from the Examiner's Final Rejection of claims 1, 3, 7, 9, 16, 17, 20, 21, 23-25, and 30-36, which are all of the pending claims. 2 We have jurisdiction over these claims under 35 U.S.C. § 6(b). We reverse and enter new grounds of rejection within the provisions of 37 C.F.R. § 41.50(b). 1 Appellant identifies Crackle, Inc., as the real party in interest. (App. Br. 2.) 2 Claims 2, 5, 6, 8, 10-15, 18, 19, 22, and 26-29 were canceled in the Amendment submitted with the Request for Continuing Examination dated June 26, 2012. Appeal2015-000097 Application 11/811,857 Introduction According to Appellant, "[ t ]he invention relates to content recording, geographic location tagging of the recorded content, and sharing of the recorded and tagged content." (Spec. i-f 1.) Exemplary Claim Claims 1, 17, and 25, reproduced below, are exemplary of the claimed subject matter: 1. An integrated unitary system for detecting content and associating that content with geographic information, compnsmg: a. a detector within the integrated unitary system configured to receive an item of content, wherein the content detector is a smart phone, videocamera, still camera, or microphone; b. a location detector within the integrated unitary system configured to receive information corresponding to a geographic location relating to the item of content; c. a correspondence module within the integrated unitary system configured to associate the item of content with the geographic location information, the integrated unitary system allowing a direct association by attributing or placing metadata or metatags including geographic location information from the location detector with the item of content, whereby the integrated unitary system can perform the association while the system is at or moving from one geographic location to another; and d. an upload component within the integrated unitary system having a wireless transmitter configured to transmit the associated item of content and geographic location information to a remote system on the Internet. 2 Appeal2015-000097 Application 11/811,857 1 7. In a computer system having a graphical user interface including a display and a selection device, a method of providing and selecting from a menu on the display, the menu organizing and displaying content items having related geographic location information, the method comprising: a. retrieving a set of menu entries for the menu, each of the menu entries including data about a geographic location; b. causing a display of the set of menu entries on the display of a system, the system being a smart phone, videocamera, computer, or computing environment with a web-based browser, and wherein the displaying the set of menu entries on the display includes displaying the set of menu entries on a map, with each menu entry disposed on the map at a location corresponding to its geographic location; c. receiving a menu entry selection signal indicative of the selection device pointing at a selected menu entry from the set of menu entries; d. in response to the signal, causing a display of a set of content items corresponding to the geographic location; e. receiving a content item selection signal indicative of the selection device selecting a content item from the set of content items; and f. in response to the signal, causing a rendering of the selected content item on the display. 25. A method of sharing an item of content with a group, compnsmg: a. creating a share group having at least one member by inputting at least one contact into a share group list or analyzing a personal information manager to determine contacts; b. receiving an item of content in an integrated unitary device, the device being a smart phone, videocamera, still camera, or microphone, receiving geographic location information relating to the item of content in the integrated 3 Appeal2015-000097 Application 11/811,857 unitary device, and associating the item of content with the geographic location information in the integrated unitary device, whereby the integrated unitary device can perform the associating while the system is at or moving between geographic locations; c. uploading the associated item of content and geographic location information to a remote system on a network; and d. notifying each member of the share group about the uploaded item of content, wherein the notifying is accomplished by sending each member of the share group an instant message or an e-mail about the uploaded item, wherein the instant message or email contains a clickable link which when clicked initiates a display of the item of content. REFERENCES The prior art relied upon by the Examiner in rejecting the claims on appeal is: Tagami et al. ("Tagami") Pagonis et al. ("Pagonis") Ikeda et al. ("Ikeda") us 4,663,629 US 2005/0186965 Al US 2006/0291806 Al REJECTIONS May 5, 1987 Aug. 25, 2005 Dec. 28, 2006 Claims 1, 3, 25, 30, and 31 stand rejected under 35 U.S.C. § 102(b) as being anticipated by Pagonis. (Final Act. 2-5.) Claims 7, 9, and 16 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Pagonis and Tagami. 3 (Final Act. 5---6.) 3 Although the heading of the Examiner's rejection states the rejection is over "Pagonis in view of Ikeda et al." (Final Act. 5), the analysis in the body of the rejection finds the rejected claims unpatentable over Pagonis and Tagami. (Final Act. 5---6; see also Ans. 13-16.) Appellant addresses the rejection as based on Pagonis and Tagami. (App. Br. 12-15 and n.1.) We, 4 Appeal2015-000097 Application 11/811,857 Claims 17, 20, 21, 23, 24, and 32-36 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Pagonis and Ikeda. (Final Act. 7-12.) ISSUES ( 1) Whether the Examiner erred in finding Pagonis teaches "the integrated unitary system allowing a direct association by attributing or placing metadata or metatags including geographic location information from the location detector with the item of content," as recited in independent claim 1 and commensurately recited in independent claim 25. (2) Whether the Examiner erred in finding the combination of Pagonis and Tagami teaches or suggests "associating the item of content with the geographic location information within the integrated unitary device, wherein the associating is performed by attributing metatags or metadata containing the geographic location information to a file associated with the item of content," as recited in independent claim 7. (3) Whether the Examiner erred in finding the combination of Pagonis and Ikeda teaches or suggests causing a display of the set of menu entries on the display of a system, ... wherein the displaying the set of menu entries on the display includes displaying the set of menu entries on a map, with each menu entry disposed on the map at a location corresponding to its geographic location[,] as recited in independent claim 17 and commensurately recited in independent claim 32. therefore, find the error harmless and address the rejection as based on Pagonis and Tagami. 5 Appeal2015-000097 Application 11/811,857 ANALYSIS A. Claims 1, 3, 25, 30, and 31: § 102(b) over Pagonis Claim 1 requires, inter alia, "allowing a direct association by attributing or placing metadata or metatags including geographic location information from the location detector with the item of content." (App. Br. 20 (Claims App'x).) The Examiner finds Pagonis teaches this limitation in paragraph 11 (see Final Act. 3), which describes a "second wireless information device" as "a simple RF ID tag used in asset tracking whenever the tag is dose [sic, close] enough to a device that is broadcasting absolute location data it stores a second [sic, record] of that absolute location, hence building up a record of its journey." (Pagonis i-f 11 ). In so doing, the Examiner "equates the RF tags to metatags." (Final Act. 13.) The Examiner further explains "[a] tag has information about a device data." (Ans. 8.) Appellant argues the Examiner's findings are in error because: [A ]ccording to the claim, the meta-tags or metadata constitute geographic location information that is attributed or placed with the item of content to perform an association. The RF tags in Pagonis do not do that; they obtain absolute location data when they are close to devices that broadcast such, and can apparently store more than one, so as to "build up a record". But there is no disclosure about how the RF tags store the data, either as meta- tags, metadata or in any other form, nor is there any disclosure of tying the data to a content item. (App. Br. 9 (emphasis added).) We agree with Appellant. Even if the RF ID tag disclosed by Pagonis were deemed equivalent to "metadata or metatags" as recited in claim 1, Pagonis does not teach associating an RF ID tag with an item of content, as also recited in claim 1. Rather, Pagonis teaches the RF ID tag is used to broadcast the location of an asset. (Pagonis i-f 11.) The Examiner does not 6 Appeal2015-000097 Application 11/811,857 find that an "asset" is an "item of content," nor does the Examiner's rejection otherwise support a finding that Pagonis teaches "allowing a direct association by attributing or placing metadata or metatags including geographic location information from the location detector with the item of content," as recited in claim 1. Therefore, on this record, we do not sustain the Examiner's rejection of independent claim 1, or its dependent claim 3, under 35 U.S.C. § 102(b) as anticipated by Pagonis. 4 We also do not sustain the Examiner's rejection of independent claim 25, which contains a similar limitation reciting "associating the item of content with the geographic location information in the integrated unitary device." (App. Br. 24 (Claims App'x).) We agree with Appellant the Examiner has failed to make a prima facie case that Pagonis teaches associating an item of content with geographic location information. (App. Br. 10.) In particular, the portions of Pagonis cited by the Examiner with regard to this limitation (Final Act. 4 (citing Pagonis i-fi-1 5, 21)) teach, at most, associating a location with a device. (See Pagonis i15: discussing "provision of automated location information to mobile devices"); i-f 21: discussing "asset tracking and fleet management services" using a "wireless information device.") The Examiner does not find that a "device" is an "item of content," nor does the Examiner's rejection otherwise support a finding that Pagonis teaches "associating the item of content with the geographic location information." 4 Appellant's Briefs present additional arguments as to claims 1 and 3. Because the issue addressed herein is dispositive of the Examiner's rejection of those claims, we do not reach the additional arguments. 7 Appeal2015-000097 Application 11/811,857 Therefore, on this record, we do not sustain the Examiner's rejection of independent claim 25, 5 or dependent claim 3 0, 6 under 3 5 U.S. C. § 102 (b) as anticipated by Pagonis. For the same reasons, we do not sustain the rejection of claim 31, which recites "[a] non-transitory computer-readable medium containing instructions for causing a computer to implement the method of claim 25." B. Claims 7, 9, and 16: § 103(a) over Pagonis and Tagami Claim 7 recites, inter alia, "associating the item of content with the geographic location information within the integrated unitary device, wherein the associating is performed by attributing metatags or metadata containing the geographic location information to a file associated with the item of content." (App. Br. 21 (Claims App'x).) The Examiner's finding as to this limitation in claim 7 is substantially the same as the Examiner's finding regarding the similar limitation in claim 1, which we have addressed supra. (See Final Act. 6 (citing Pagonis i-fi-f 11, 21 ).) For the reasons stated above with regard to claim 1, we agree with Appellant the portions of Pagonis cited by the Examiner do not teach attributing metatags or metadata including geographic information to a file 5 Appellant's Briefs present additional arguments as to claim 25. Because the issue addressed herein is dispositive of the Examiner's rejection of claim 25, we do not reach the additional arguments. 6 Should there be further prosecution, we note claim 30 improperly depends from canceled claim 26. (App. Br. 24 (Claims App'x).) Claim 26, which was canceled in an Amendment submitted with the Request for Continuing Examination dated June 26, 2012, depended from claim 25. 8 Appeal2015-000097 Application 11/811,857 associated with an item of content, as recited in independent claim 7. (App. Br. 12 (incorporating for claim 7 arguments made for claim 1 ).) Therefore, on this record, we do not sustain the Examiner's rejection of independent claim 7, or its dependent claims 9 and 16, under 35 U.S.C. § 103(a) as unpatentable over Pagonis and Tagami. 7 C. Claims 17, 20, 21, 23, 24, and 32-36: § 103(a) over Pagonis and Ikeda Independent claim 17 recites, inter alia: causing a display of the set of menu entries on the display of a system, ... wherein the displaying the set of menu entries on the display includes displaying the set of menu entries on a map, with each menu entry disposed on the map at a location corresponding to its geographic location[.] (App. Br. 22 (Claims App'x).) The Examiner finds, although Pagonis discloses a "world map application;" Pagonis does not disclose this full limitation. (Final Act 8 (citing Pagonis i-f 60).) The Examiner further finds, however, Ikeda teaches or suggests: Ikeda discloses a. retrieving a set of menu entries for the menu, each of the menu entries representing a geographic location; b. displaying the set of menu entries on the display; c. receiving a menu entry selection signal indicative of the selection device pointing at a selected menu entry from the set of menu entries. (Ikeda, [0082], an interactive display having a remote control is 7 Appellant's Briefs present additional arguments as to claims 7, 9, and 16. Because the issue addressed herein is dispositive of the Examiner's rejection of those claims, we do not reach the additional arguments. 9 Appeal2015-000097 Application 11/811,857 receiving access requests from users, where operations can be switched using arrow keys and a subtitle switch for numeric key). (Final Act. 8 (citing Ikeda i-f 82 ). ) The Examiner then finds: [I]t would have been obvious for one having ordinary skill in the art at the time the invention was made to incorporate Ikeda table with location information entries into Pagonis method of obtaining and sharing content associated location method in order to create a obtaining and sharing content associated location method with a table with location information entries in order to be able to access easier information about a location. (Final Act. 8.) Appellant argues: Even combining the world mapping application of Pagonis [0060] with [Ikeda's] teaching cannot render the same obvious as Pagonis' s world map only relates to obtaining location information. That is, Pagonis states that mapping applications can republish information that may be available from a location server, e.g., time zone and local dialing codes. But there is no disclosure of using the same in any way as a user inteljace, much less for displaying menu entries. (App. Br. 16-17 (emphasis added).) We agree with Appellant. Therefore, on this record, we do not sustain the Examiner's rejection of independent claim 1 7, or its dependent claims 20, 21, 23, 24, 33, and 34, under 35 U.S.C. § 103(a) as unpatentable over Pagonis and Ikeda. Independent claim 32 contains one or more of the limitations discussed above regarding claims 1, 7, and 1 7, for which the Examiner's findings are insufficient to sustain the Examiner's rejections. Therefore, on this record and for the reasons discussed above regarding claims 1, 7, and 1 7, we do not sustain the Examiner's rejection of independent claim 32, or its 10 Appeal2015-000097 Application 11/811,857 dependent claims 35 and 36, under 35 U.S.C. § 103(a) as unpatentable over Pagonis and Ikeda. NEW GROUNDS OF REJECTION We exercise our discretion under 3 7 C.F .R. § 41. 50(b) to enter new grounds of rejection under 35 U.S.C. § 103(a) as follows: (1) Independent claims 1 and 7 are rejected under 35 U.S.C. § 103(a) as unpatentable over Mikan et al., U.S. Patent No. 8,548,424 Bl, issued Oct. 1, 2013 (filed Jan. 30, 2007) (hereafter "Mikan"). ( 2) Independent claim 2 5 is rejected under 3 5 U.S. C. § 1 0 3 (a) as unpatentable over Mikan in view of Jefferson Graham, "Flickr of idea on gaming project led to photo website," USA Today (Feb. 27, 2006), available for download at http ://usatoday3 0. usatoday .com/tech/products/2006-02-2 7 - flickr_x.htm (hereafter "Graham"). (3) Independent claims 17 and 32 are rejected under 35 U.S.C. § 103(a) as unpatentable over Mikan in view of Graham and further in view of James Fallows, "Spy's-eye View," The Atlantic (March 2006), available for download at http://www.theatlantic.com/magazine/archive/2006/03/spy- s-eye-view/304620/ (hereafter "Fallows"). A. Claims 1, 7, and 2 5 Mikan, which qualifies as prior art to the present application under 3 5 U.S.C. § 102(e), discloses a method and apparatus for "geotagging" digital photographs, wherein "[ w ]hen a photo is taken, the location of the camera is 11 Appeal2015-000097 Application 11/811,857 recorded as digital data along with the digital data representing the image itself." (Mikan 1 :25-27; see also 1 :23--44.)8 Mikan further discloses: It is known to geotag uploaded photos, including photos taken with camera phones or similar UE ["user equipment'']. Yahoo!, Inc., which provides the FLICKR Internet photo hosting service, has developed a geotagging client application (software) known as ZONETAG that runs on camera phones and operates in conjunction with Yahoo! 's FLICKR photo uploader utility and service. A user can access FLICKR to view and manipulate uploaded photos via the Internet. When a user indicates through the ZONET AG application that a photo is to be uploaded, ZONET AG determines the identity of the cell (i.e., base station) through which the camera phone or other UE is then communicating with the network and tags the photo with an identifier that identifies the network and cell ("Cell ID'') before uploading it to FLICKR. Once the photo has been uploaded, FLICKR attempts to translate the identifier into a geographic location. FLICKR has several ways in which it can gather information to assist it with the translation. When the user chooses to upload the photo, the user can enter a city name or ZIP code, which ZONETAG causes the camera phone to transmit along with the photo. . .. The FLICKR server maintains a database that relates cell identifiers to city names, ZIP codes or other such identifying information. (Mikan 1 :65-2:22 (emphases added).) 8 Although Mikan discloses commercially available digital cameras generally did not contain built-in GPS receivers circa the time of Appellant's invention (see Mikan 1 :36-37), we note digital cameras having built-in GPS receivers were known at the time. See, e.g., figure 5 of U.S. 2006/0066752 Al to Kelliher, published on Mar. 30, 2006 (showing a camera 10 having a digital camera 50 and a GPS interface 70). 12 Appeal2015-000097 Application 11/811,857 Claim 1 As detailed below, we find Mikan teaches or suggests all limitations of claim 1. • "An integrated unitary system for detecting content and associating that content with geographic information ... " Mikan discloses "camera phones" with "built-in digital cameras" that can take photographs and then run an application (e.g., "ZONET AG") that associates geographic information, such as a "Cell ID" and/or a "ZIP code," with the photographs. (Mikan 1 :50-53, 1 :65-2:22.) • "a detector within the integrated unitary system configured to receive an item of content, wherein the content detector is a smart phone, videocamera, still camera, or microphone" Mikan discloses "camera phones." (Mikan 1 :50-53, 1 :66.) • "a location detector within the integrated unitary system configured to receive information corresponding to a geographic location relating to the item of content" Mikan discloses the "camera phones" can run an application (e.g., "ZONET AG") that associates geographic information, such as a "ZIP code," with photographs taken by the camera phones. (Mikan 1 :65-2:22.) • "a correspondence module within the integrated unitary system configured to associate the item of content with the geographic location information, the integrated unitary system allowing a direct association by attributing or placing metadata or metatags including geographic location information from the location detector with the item of content, whereby the integrated unitary system can 13 Appeal2015-000097 Application 11/811,857 perform the association while the system is at or moving from one geographic location to another" Mikan discloses the "camera phones" can take photographs and then run an application (e.g., "ZONETAG") that directly associates geographic information, such as a "Cell ID" and/or a "ZIP code," with the photographs by tagging the photographs with the geographically identifying information, which can be done at least when the system is at a geographic location. (Mikan 1 :65-2:22.) • "an upload component within the integrated unitary system having a wireless transmitter configured to transmit the associated item of content and geographic location information to a remote system on the Internet" Mikan discloses that it is common for users of wireless mobile telecommunications network user equipment (UE) that include built-in digital cameras" to "upload photos taken with the UE." Mikan 1 :50-60); see also Mikan 2: 14--17 (describing that when a user uploads a photo, geographic information is sent along with the photo).) Claim 7 We incorporate our findings for claim 1 for similar limitations recited in claim 7. Claim 7 also recites: "wherein the associating [geographic location information] occurs in a manner selected from the group consisting of: every time an item of content is stored, periodically, upon user input, or a combination .... " (App. Br. 21 (Claims App'x) (emphasis added).) Mikan discloses associating geographic location information (such as a "Cell ID" and/or a "ZIP code"), at least upon user input, with a photograph taken by the camera phone through the ZONETAG application. (Mikan 2:5-25.) 14 Appeal2015-000097 Application 11/811,857 Claim 7 further recites: wirelessly uploading the associated item of content and geographic location information to a remote system on the Internet ... wherein the uploading occurs in a manner selected from the group consisting of: every time an item of content is stored, every time a network connection is established, periodically, upon user input, or a combination of the above. (App. Br. 21 (Claims App'x) (emphasis added).) Mikan discloses uploading content (e.g. a photograph) with its associated geographic location information (e.g., a "Cell ID and/or a "ZIP code") to the Internet at least upon user input. (Mikan 1 :45-2:20.) B. Claim 25 Claim 25 recites a "method of sharing an item of content with a group." (App. Br. 23 (Claims App'x).) We find Mikan teaches limitations "b" and "c" of claim 25 for the reasons set forth above with regard to the similar limitations recited in claims l and 7. (See Mikan 1:23-2:27.) With regard to limitations "a" and "d" recited in claim 25, we rely on Mikan, which describes a "FLICKR photo uploader utility" (Mikan 1 :65- 2:27), in combination with Graham, which describes further functionality of FLICKR as including "photo-sharing features," wherein photos posted online with FLICKR may be shared with "family and friends." (Graham 2.) Graham further describes FLICKR users may copy a web address for their photo collections and send the link to friends and family via e-mail. (Graham 2-3.) It would, therefore, have been obvious to one of ordinary skill in the art as of the date of Appellant's invention to create a contact group (such as an email group of family and friends) for sharing, via email, a clickable link 15 Appeal2015-000097 Application 11/811,857 to the FLICKR web site that would, in tum, allow the user to notify members of the share group that geotagged photographs had been uploaded from the user's smart phone, as taught by Mikan and Graham. (Mikan 1 :65- 2:27; Graham 2-3.) The motivation to use a contact group to share FLICKR links would be to quickly and efficiently distribute access to a collection of geotagged photographs to multiple persons, such as a group of friends and/or family. C. Claims 17 and 32 Claim 17 We incorporate our findings for independent claim 1, supra, for similar limitations recited in independent claim 1 7. Claim 1 7 also recites, inter alia: causing a display of the set of menu entries on the display of a system, ... wherein the displaying the set of menu entries on the display includes displaying the set of menu entries on a map, \'l1ith each menu entry disposed on the map at a location corresponding to its geographic location .... (App. Br. 22 (Claims App'x).) Appellant's Specification does not explicitly define the claim term "menu entries," but the Specification provides an example embodiment: Advantages of the invention may include one or more of the following. The content files with location data are easily uploadable to a computer, server or other storage. The same are navigable and may be viewed via an interactive user interface where the same are displayed as clickable links, such as by the use of thumbnails, icons or textual representations, on a map corresponding to their geographic location. The invention provides an easy-to-use and convenient process for camera users to share video with geo-data and for others to watch the video over the Internet. For example, family members may watch 16 Appeal2015-000097 Application 11/811,857 videos of a trip done by another family member and see where each video was shot on a map. (Spec. i-f 22 (emphases added); see also i-fi-155-57.) We conclude the term "menu entries" reasonably includes "thumbnails, icons or textual representations" of content, where the content includes photographs or videos. As discussed above, Mikan describes allowing content, such as photographs, to be geotagged with geographic location information and uploaded to the Internet (e.g., through use of ZONETAG and FLICKR). (Mikan 1 :23-2:27 .) Fallows, which is prior art to the present application under 35 U.S.C. § 102(b), describes additional functionality in FLICKR, in public use more than a year before Appellant's application, as allowing display of the geotagged content (such as photos) overlaid on a map using Google Earth, with clickable icons allowing users to view the overlaid content: Google Earth allows the equivalent of bloggers to attach annotations to specific geographical points. These overlays consist of pictures, written descriptions, Internet links, or other data tied to the GPS coordinates of a particular place. When a Google Earth user zooms in on that place, the link to the information appears, usually as a clickable icon or button. Perhaps the most open-ended potential involves Google Earth's interaction with Flickr, the photo-sharing site run by Yahoo. More than 7. 5 million photos have been stored at Flickr, many of them tagged by location-"Paris, Arc de Triomphe"---or "geotagged" with GPS coordinates. People pay to post these pictures, as a handy way to store files that can clog hard drives and to keep them all in one place; viewing access is generally free. Using the Flickr overlay, which works through a site called Geobloggers, you can see any available photos of the place you 're looking at. For instance, when displayed in addition to the 17 Appeal2015-000097 Application 11/811,857 Everest overlay, Flickr brings up pictures people have taken at the summit. When I'm looking at my neighborhood, I see Flickr photos of trees that were blown down two years ago in a hurricane. New shots pour into Flickr each day, and to similar sites like Panoramio.com, founded by two young Spaniards, which lets users easily link their photos to specific sites. (Fallows 4--6 (emphases added).) We find Mikan, in combination with Fallows, teaches or suggests all limitations of claim 17: • "In a computer system having a graphical user interface including a display and a selection device, a method of providing and selecting from a menu on the display, the menu organizing and displaying content items having related geographic location information": Mikan discloses a computer system with a graphical user interface (1 :54--60; 2:4--5). As found above regarding claims 1 and 7, Mikan also discloses displaying content items having related geographic location information using the FLICKR "photo uploader utility and service." (1 :65-2:27.) Fallows discloses FLICKR also allowed, over a year before the filing of the present application, overlaying icons or other representations of FLICKR geotagged photos onto a map. Under a broad but reasonable interpretation, these overlaid, clickable icons teach or suggest the recited "menu organizing and displaying content items having related geographic location information." • "retrieving a set of menu entries for the menu, each of the menu entries including data about a geographic location": As discussed above, Fallows discloses overlying icons or other representations of FLICKR geotagged content onto a map corresponding to the geographic location associated with that content. (Fallows 4--6.) Under 18 Appeal2015-000097 Application 11/811,857 a broad but reasonable interpretation, these overlaid, clickable icons are teach or suggest the recited "menu entries including data about a geographic location." • "causing a display of the set of menu entries on the display of a system, the system being a smart phone, videocamera, computer, or computing environment with a web-based browser, and wherein the displaying the set of menu entries on the display includes displaying the set of menu entries on a map, with each menu entry disposed on the map at a location corresponding to its geographic location": Fallows discloses it was known in the art, over a year before Appellant's application, that using the FLICKR overlay with Google Maps would "caus[ e] a display of the set of menu entries" at least on a "computer," wherein each menu entry (an icon of geotagged content, such as a photograph) would be "disposed on the map at a location corresponding to its geographic location." (Fallows 4---6.) • "receiving a menu entry selection signal indicative of the selection device pointing at a selected menu entry from the set of menu entries": Fallows discloses a service, such as FLICKR, as an overlay on a map, such as Google Earth, wherein the overlay would include clickable icons. (Fallows 4--6.) As described in Fallows, it was also known in the art, over a year before Appellant's application, to use the FLICKR overlay on Google Earth on a computer, and to use a mouse to point to and click icons displayed on the map representing geotagged content. For example, Fallows describes a user "zoom[ing] in" on a place on a map, to cause "the link to the 19 Appeal2015-000097 Application 11/811,857 information" tied to that location to appear, "usually as a clickable icon or button." (Fallows 4.) • "in response to the signal, causing a display of a set of content items corresponding to the geographic location": Fallows discloses use of an overlay, such as FLICKR, on Google Earth to "zoom[] in" on a place on a map and cause "the link to the information" tied to that location to appear, "usually as a clickable icon or button." (Fallows 4.) • "receiving a content item selection signal indicative of the selection device selecting a content item from the set of content items": Fallows discloses use of an overlay, such as FLICKR, on Google Earth to "zoom[] in" on a place on a map and cause "the link to the information" tied to that location to appear, "usually as a clickable icon or button." (Fallows 4.) The user could then use a mouse to select the clickable icon or button. • "in response to the signal, causing a rendering of the selected content item on the display": Fallows discloses clicking on icons of FLICKR geotagged content overlaid onto a Google Earth map to allow users to "caus[ e] a rendering of the selected content item on the display"-such as photographs corresponding to the geographic location shown on the map. (Fallows 5---6.) It would have been obvious to one of ordinary skill in the art as of the date of Appellant's invention to upload geotagged photographs to FLICKR as taught by Mikan, and to utilize Google Earth to display clickable links to the geotagged photographs uploaded to FLICKR on a map as taught by 20 Appeal2015-000097 Application 11/811,857 Fallows. (Mikan 1 :65-2:27; Fallows 4---6.) Both Mikan and Fallows suggest the combination as a complementary way of utilizing FLICKR's services in order to "see any available photos of the place you are looking at" on the Google Earth map. (Fallows 6.) Claim 32 Claim 32 contains limitations that are all addressed above in connection with one or more of claims 1, 7, 17, and 25. We incorporate our findings above regarding those claims, and find the combination of Mikan, Graham, and Fallows teaches or suggests all limitations of claim 32. It would have been obvious to one of ordinary skill in the art as of the date of Appellant's invention to upload geotagged photographs to FLICKR as taught by Mikan, to utilize Google Earth to display clickable links to the geotagged photographs uploaded to FLICKR on a map as taught by Fallows, and to share those links with a selected group of users as taught by Graham. (Mikan 1:65-2:27; Graham, 2-3; Fallows 4---6.) The motivation to do so would be to quickly and efficiently share a collection of geotagged photos using a map-based application with multiple persons, such as a group of friends and/or family. D. Dependent Claims In rejecting independent claims 1, 7, 17, 25, and 32 we note that we are primarily a reviewing body, rather than a place of initial examination. We, therefore, leave it to the Examiner to determine whether the claims that depend from claims 1, 7, 17, 25, and 32, respectively, should be rejected on similar grounds to those set forth herein or in combination with additional 21 Appeal2015-000097 Application 11/811,857 prior art. T'hefact that we did not enter new grounds of rejection for the dependent claims should not be construed to mean that we consider the dependent claims to be directed to patentable subject matter or to be patentable over the prior art of record. DECISION For the above reasons, the Examiner's rejections of claims 1, 3, 7, 9, 16, 17, 20, 21, 23-25, and 31-36 are reversed. We enter a new ground of rejection of independent claims 1, 7, 17, 25, and 32 under 35 U.S.C. § 103(a). This decision contains new grounds of rejection pursuant to 37 C.F.R. § 41.50(b). Section 41.50(b) provides "[a] new ground of rejection pursuant to this paragraph shall not be considered final for judicial review." Section 41.50(b) also provides: When the Board enters such a non-final decision, the appellant, \~1ithin t\~10 months from the date of the decision, must exercise one of the following two options with respect to the new ground of rejection to avoid termination of the appeal as to the rejected claims: (1) Reopen prosecution. Submit an appropriate amendment of the claims so rejected or new Evidence relating to the claims so rejected, or both, and have the matter reconsidered by the examiner, in which event the prosecution will be remanded to the examiner. The new ground of rejection is binding upon the examiner unless an amendment or new Evidence not previously of Record is made which, in the opinion of the examiner, overcomes the new ground of rejection designated in the decision. Should the examiner reject the claims, appellant may again appeal to the Board pursuant to this subpart. (2) Request rehearing. Request that the proceeding be reheard under § 41.52 by the Board upon the same Record. The request for rehearing must address any new ground of rejection 22 Appeal2015-000097 Application 11/811,857 and state with particularity the points believed to have been misapprehended or overlooked in entering the new ground of rejection and also state all other grounds upon which rehearing is sought. Further guidance on responding to a new ground of rejection can be found in the Manual of Patent Examining Procedure § 1214. 01. REVERSED 37 C.F.R. § 41.50(b) 23 Application/Control No. Applicant(s)/Patent Under Reexamination 11/811,857 Appeal No.2015-000097 Notice of References Cited Judge A. Hagy Art Unit n ..... ,.,. ..... -I ..... .i: -I od!::jO:: I UI I U.S. PATENT DOCUMENTS * Document Number Date Country Code-Number-Kind Code MM-YYYY Name Classification A US-8,548,424 81 10-2013 Mikan et al. B US- c US- D US- E US- F US- G US- H US- I US- J US- K US- L US- M US- FOREIGN PATENT DOCUMENTS * Document Number Date Country Code-Number-Kind Code MM-YYYY Country Name Classification N 0 p Q R s T NON-PATENT DOCUMENTS * Include as applicable: Author, Title Date, Publisher, Edition or Volume, Pertinent Pages) u Jefferson Graham, Flickr of idea on gaming project led to photo website, USA TODAY (Feb 27, 2006) available for download at httg://usatodav30.usatodav.com/tech/Qroducts/2006-02-27-flickr x.htm v James Fallows, Spy's-eye View, THE ATLANTIC (March 2006), available for download at httD://www.theatlantic.com/magazine/archive/2006/03/sDv-s-eve-view/304620/ w x *A copy of this reference 1s not being furnished with this Office action. (See MPEP § 707.05(a).) Dates in MM-YYYY format are publication dates. Classifications may be US or foreign. U.S. Patent and Trademark Office PT0-892 (Rev. 01-2001) Notice of References Cited Part of Paper No. fi.$'2016 !It/antic Spy's-eye View Google Earth and its rival programs offer (civilians) a new way to look at the world TEXTS!ZE MARCH 2006 ISSUE l T?C:/\(i.t't:{ 88 Like The Atlantic!? Subscribe to the Dally, our free weekday email newsletter. l_E_ma_il ____ ] 7i~bN UP As best I can figure, I have spent 3 5 ,000 to 40,000 hours of my life sitting at a computer. This knowledge does not improve my mood or self-esteem. But it brings into sharp relief the handful of moments at the keyboard I can distinctly remember, each involving a time when I realized that the computer had just done something important and new. In the late 19 70s, I marveled at the discovery that I could use my very first computer (a Processor Technology SOL-20) to save something I had drafted, then change it later on without having to retype the whole thing. In the early 1980s, I watched a message I had written go to its destination electronically, through·a 300-baud modem I clamped onto a telephone handset. In 199 3, I tried a program called Mosaic, one of the very first browsers, and was amazed to see pictures and docwnents stored on someone else's computer show up on my own screen. In 199 5, I entered a few words into a search box and within seconds got back some more-or-less relevant information, via the early search engine Alta Vista. ll!tp:Nw1uw theat!an!ic.ccm/maga,~ine!ard~ve.i20Clt'Y03/spy-s-eye-view/3f!4620f 117 fi.$'2016 That was it for truly memorable moments, W1til last year when I first tried Google Earth. The addictive nature of this program stems from three elements. One is a set of satellite and aerial photos of the globe's entire surface, which Google has acquired from a variety of public and private sources. These vary greatly in resolution, but the sharp ones, which already cover most major populated areas and are steadily expanding in range, are unnervingly clear. I now take it for granted that I can pick out every building-homes, schools, offices-I care about in the United States. But I can also see the tiny house where my family lived outside Tokyo, complete with ten-foot-square "back yard," and the red- roofed colonial bungalow we rented.in Kuala Lumpur, which now sits in the shadow of the Petronas Towers, two of the world's tallest buildings. The village where my wife and I once worked in Ghana, on the other hand, is just a green blur. The next element is Google Earth's viewing system, which lets you "fly" from point to point. This sowids trivial, but I think it can profonndly reorient people's sense of geography. We are used to thinking of places as being arrayed along roads or rail lines, or linked by airline routes. Taking a Google Earth "flight" is like driving through a city you knew only through subways, and seeing the surprising ways the pieces fit together. Finally, the program has an imperfect but tantalizing 3D feature. The contours of terrain and, in certain cities, the heights of downtown structures are built into the database. Although the program's standard view is straight down from overhead, you can tilt the perspective, until you're looking from a low altitude or even ground level. This gives a dramatic sense of how, say, a ski resort fits into the surronnding mo1Ultains, or how the fjords of Greenland would look if you approached from the sea. You can't connt on the program to have enough accurate data to make this feature work everyplace-those ll!tp:Nw1uw theat!an!ic.ccm/maga,~ine!ard~ve.i20Clt'Y03/spy-s-eye-view/31.14620f 217 fi.$'2016 Petronas Towers, when seen at a tilt, look no taller than my former house-- but where it works it's great. The basic version of the program, which does everything described in this article, is free (it requires a broadband connection). For now, it runs only on PCs, but its creator, John Hanke, says that a Mac version is about to appear. Hanke and others developed the program at a small company called Keyhole, which Google bought two years ago. (Microsoft has released its counterpart program, Virtual Earth, at its Windows Live site, http:/ /local.live.com. This program doesn't have the flying or 3D features of Google Earth, but it does have superior close-up shots of many U.S. cities, including "bird's-eye" views of about a dozen cities so breathtakingly detailed that in some cases you can see the individual panes in the windows. You can also see some structures that have been blurred out of Google Earth-for instance, the vice president's home in Washington.) Together these elements make Google Earth fascinating. What makes the program important is the trait it shares with other big steps forward in computing, like the previous ones I recall so sharply. It is not an end in itself but a beginning of new opportunities for innovation by others, based on the new tools it provides. A huge theme in recent discussions of the Internet's future is the growing importance of "user-created content." In the old-media days of the twentieth century, small groups of specialists produced books, newspapers, or broadcasts that larger public audiences could consume. Now millions of people publish their views over the Internet, by producing blogs, or rating products or services online, or posting pictures and Webcasts for others to view, or creating networks of like-minded people who can share recommendations about vacation sites or bikes or restaurants. The tools that have thus democratized publishing-or that have enabled user- ll!tp:Nw1uw theat!an!ic.ccm/maga,~ine!ard~ve.i20Clt'Y03/spy-s-eye-view/31.14620f 3/7 fi.$'2016 created content, if you prefer-are many and varied. But they rest upon the fundamental units of all modern Internet activity: URLs, links, and words. A lJRL (unifonn resource locator) directs a browser to a particular set of files on a particular computer somewhere on Earth (each Web "page" is really a collection of files). Links take users from one set of files to another. The words in a file, indexed by search engines or catalogued as "tags," serve as a guide to Web page content. Google Earth of course employs links and URLs. Its basic unit of classification, however, is not a word but a set of global positioning system (GPS) coordinates, pinpointing a location on the Earth with uncanny precision. The Google Earth view of my neighborhood in Washington shows a red truck parked down the street. The front bwnper of that truck has a different GPS address from the windshield. And in the same way that a "traditional" blog, if we can use that term, might have links pointing to a particular news item or posting, Google Earth allows the equivalent of bloggers to attach annotations to specific geographical points. These overlays consist of pictures, Vvntten descriptions, Internet links, or other data tied to the G PS coordinates of a particular place. When a Google Earth user zooms in on that place, the link to the information appears, usually as a clickable icon or button. Since last }Wle, when Google released the program and its toolkit for creating these overlays, many people have gotten to work. As with online book reviews at Amazon.com or entries on Wikipedia (the online encyclopedia in which anyone can write or edit an entry), the information supplied can range from authoritative to bogus. But nearly all of it is interesting. The easiest way to see what's on offer is through sites like Ogleearth.com and Gearthblog.com. They have news and screenshots, and links to files that create the overlays for Google Earth. You click to download overlays to your ll!tp:Nw1uw theat!an!ic.ccm/maga,~ine!ard~ve.i20Clt'Y03/spy-s-eye-view/31.14620f 4/7 fi.$'2016 computer. Then, when Google Earth is running, you choose the overlays you want displayed. John Hanke of Google showed me a few examples, which ranged from the fanciful to the highly prac-tical. On the practical side is an overlay from Caltrans, the agency responsible for California's freeways. When you display this data within Google Earth, it shows congestion conditions every mile or so along freeways in the state's major cities. Clusters of red, orange, yellow, or green dots show you where traffic is moving and where it is jammed; any dot you click on tells the average speed of cars passing that point. A WiFi overlay shows the nearest available commercial hot spots. Ona skiing overlay, you can click a blue snowflake symbol by major resorts and get detailed slope information and· a link to the mountain's Web site. An aviation site called FBOWeb.com offers a free demonstration showing the trajectories airliners are taking toward several major airports; by using the tilted view you can "watch" planes as they descend over Boston Harbor or through the Los Angeles basin for landing. You can display as many of these varied overlays as you choose, clicking on a check box to tu.rn them on or off. Less practical, but often more delightful, creations continue to proliferate. An overlay from Blogwise.com shows who is writing a blog in your city-or in Buenos Aires or Beijing-with a link to each blog. Another shows bird migration patterns. One called South 3 .1 shows the route of Ernest Shackleton's doomed Endurance mission to Antarctica, with photos and log entries matched to their locations. Another, covering the Mount Everest area, maps the course of a 2003 Everest expedition with aerial photos so sharp you can see individual tents. The Google Earth Community overlay has Wikipedia-style notes on interesting buildings and historic sites. Perhaps the most open-ended potential involves Google Earth's interaction with Flickr, the photo-sharing site run by Yahoo. More than 7 5 million ll!tp:Nw1uw theat!an!ic.ccm/maga,~ine!ard~ve.i20Clt'Y03/spy-s-eye-view/31.14620f fi.$'2016 photos have been stored at Flickr, many of them tagged by location-" Paris, Arc de Triomphe" -or "geotagged" with GPS coordinates .. People pay to post these pictures, as a handy way to store files that can clog hard drives and to keep them all in one place; viewing access is generally free. Using the Flickr overlay, which works through a site called Geobloggers, you can see any available photos of the place you're looking at. For instance, when displayed in addition to the Everest overlay, Flickr brings up pictures people have taken at the summit. When I'm looking at my neighborhood, I see Flickr photos of trees that were blown down two years ago in a hurricane. New shots pour into Flickr each day, and to similar sites like Panoramic.com, fowided by two yowig Spaniards, which lets users easily link their photos to specific sites. Google Earth is far less powerful than "geographic information systems," or G IS. These are complex and often expensive programs that match economic, demographic, environmental, and other data to satellite views of particular locations. A county government might use G IS to combine data on population density, fire risks, and traffic patterns to find the right location for a firehouse. Businesses use it to decide where to place distribution centers or new stores. Scientists use it to track environmental trends. But according to Jack Dangermond, founder and president of the world's leading GIS company, ESRI of Redlands, California, the free services available under Google Earth will increase the public's use of geographically linked information, for-pay and free alike. "There's no way to connect a professional data set to Google Earth, so in a sense it is pretty thin," he said. "But because it is spellbinding to customers, it can only build awareness of geography. And if this exposure makes citizens more spatially literate, they can accept more of their information this way and visualize more about their local situations." Many governments already use G IS to let citizens see public data, from the residences of registered sex offenders to the tax assessments of buildings, homes, and lots. Within a year or two, he said, standard Web ll!tp:Nw1uw theat!an!ic.ccm/maga,~ine!ard~ve.i20Clt'Y03/spy-s-eye-view/31.14620f fi.$'2016 browsers will allow citizens to "fly" tluough this data or see it in 30, using features like Google Earth's. "Honestly, I have no idea where this is going in the long nm," John Hanke of Google told me. "Ten years ago, this technology was the exclusive province of the U.S. intelligence commwiity. Five years ago, it cost $14,000 for a single satellite image. Now there's free, global high-resolution imagery." The most significant use of the technology so far, he said, has been to provide "maps for places where they had no previous maps." He read from a newspaper story about the headman of a Kurdish village who was using a Google Earth printout to plan the village's rebuilding. "This is really inspiring to me," Hanke said. "It's the sort of thing that we thought might happen as this information is released and democratized." Fooling around with this program and its overlays has put a few more hours on my computer total. These hours seem better spent than most. ABOUT THE AUTHOR ~~b\M~$ fAU.0%~W is a national correspondent for The Atlantic and has written for the magazine since the late 1970s. He has reported extensively from outside the United States and once worked as President Carter's chief speechwriter. His latest book is c.:hk·m Airl:Jcwne. ~Twitter II Email ll!tp:Nw1uw theat!an!ic.ccm/maga,~ine!ard~ve.i20Clt'Y03/spy-s-eye-view/3f!462(){ 711 I lllll llllllll Ill lllll lllll lllll lllll lllll 111111111111111111111111111111111 c12) United States Patent Mikan et al. (54) DYNAMIC GEOTAGGING OF PHOTOGRAPHS (75) Inventors: Jeffrey Mikan, Atlanta, GA (US); Fulvio Cenciarelli, Suwanee, GA (US); Justin McNamara, Dunwoody, GA (US) (73) Assignee: AT&T Mobility II LLC, Atlanta, GA (US) ( * ) Notice: Subject to any disclaimer, the term ofthis patent is extended or adjusted under 35 U.S.C. 154(b) by 1302 days. (21) Appl. No.: 11i668,538 (22) Filed: Jan.30,2007 (51) Int. Cl. H04M 11104 (2006.01) (52) U.S. Cl. USPC .................. 455/404.2; 455/456.1; 455/456.2; 455/456.3; 345/418; 719/322 ( 58) Field of Classification Search USPC ....................................................... 455/456.1 See application file for complete search history. 14 12 BASE STATIONS MSC 24 GMLC SGSN US008548424Bl (10) Patent No.: (45) Date of Patent: US 8,548,424 Bl Oct. 1, 2013 (56) References Cited U.S. PATENT DOCUMENTS 200110017668 Al* 8/2001 Wilcock et al. ............... 348/552 2004/0002305 Al* 112004 Byman-Kivivuori et al. ............................ 455/41.2 2004/0224700 Al* 1112004 Sawano ..................... 455/456.l 2005/0038876 Al* 212005 Chaudhuri .................... 709/219 2005/0070278 Al* 3/2005 Jiang .......................... 455/432.3 2006/0248154 Al * 1112006 Chu et al. ...................... 709/206 * cited by examiner Primary Examiner - Kamran Afshar Assistant Examiner - Marisol Fahnert (74) Attorney, Agent, or 1Dinn - Hartman & Citrin LLC (57) ABSTRACT ~ · · ·~ The network can determine the UE location using a conventional method such as those used in Location-Based Services. 10 Claims, 3 Drawing Sheets 16 GMSC SS? NETWORKS (INCL. 28 __../ PSTN) 32 22 USER GEO- GGSN MMSC 30 (CLIENT) TAGGER 26 INTERNET 20 ~ lfJ. 1o~R ~"~ \ '"' \ ~ """ """ ~ 14 ~ 16 ~ l = 12~ I l ~ I I I~ I MSC H GMSC ,.... '-' (') ~ :-' I N I 0 .... (.H 24 GMLC SGSN SS? NETWORKS (INCL. 28 00 PSTN) __./ =-('D ('D 32 ..... .... I 0 ..... (.H GEO- 22 USER TAGGER GGSN MMSC 30 (CLIENT) d 26 INTERNET rJl 00 20 u. ~ FIG. 1 00 ):... N ~ = ""'"' U.S. Patent Oct. 1, 2013 Sheet 2 of 3 I WIRELESS NETWORK [__ Rl"('l"l\/t"' n1r.IT~I 34 I '\.L-'--'L-1 V l-\,.J L"""l'-'1 I/ \L.. IMAGE FROM UE NETWORK ELEMENT SUBMITS LOCATION- BASED SERVICES (LBS) QUERY FOR LOCATION OF UE NETWORK TAGS IMAGE FILE WITH GEOGRAPHiC LOCATION INFORMATION UPLOADING PROCESS PROCEEDS ON GEOTAGGEDIMAGE FILE FIG. 2 52 54 56 US 8,548,424 Bl 26 ) 17rr~ ( ( GGSN 11 ... ,. -\\--. . I MEMORY \\ ~® uu I I l 11 FIG. 3 US 8,548,424 B 1 1 DYNA~1IC GEOTAGGING OF PHOTOGRAPHS BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates generally to wireless tele- communications networks and, more specifically, to geotag- ging digital photographs transmitted via a wireless mobile telecommunications network. 2. Description of the Related Art . The data file can represent any medium, with an image (e.g., photograph) being the most common. 15 2 developed a that runs on and operates in conjunction with Yahoo!'s FLICKR photo uploader utility and service. Once the photo has been uploaded, FLICKR attempts to translate the identifier into a geographic location. FLICKR has several ways in which it can gather information to assist it with the translation. When the user chooses to upload the photo, the user can enter a city name or ZIP code, which ZONETAG causes the camera phone to transmit along with the photo. Alternatively, the user can access FLICKR via the Internet and enter a ZIP code In theory, every part of a digital photograph can be associated with the location where the user knows the photo tied to a geographic location, but in most instances only the 20 was taken. The FLICKR server maintains a database that position of the camera is associated with the entire digital relates cell identifiers to city names, ZIP codes or other such ,. ... i!!E'..g~~---------------------------------------------··--------------------------------------··---------------, identifying information. If a FLICKR user enters the city, ZIP ! ! code, etc., at which a photo was taken, FLICKR applies that ~ ~ ! ! information to not only that photo but also to other photos that i When a photo is taken, the location of the camera is recorded ~5 the user may have taken within that same cell (i.e., that bear i as digital data along with the digital data representing the i the same identifier) as well as the photos of other FLICKR i image itself. Although there is currently no universal geotag- i users that were taken within that cell. i ging standard, some well-known digital image file format i j standards do specify formats for storing geographical coor- j i dinates along with the digital data representing the image. For ~o i example, the Joint Photographic Experts Group (JPEG) and i i Tagged Image File Format (TIFF) standards specify that geo- l i graphical coordinates can be included in an Exchangeable l i . The EXIF header is com- l ~ ~ i manly used in digital cameras to record the date and time a p5 i photo was taken. Although commercially available digital j ! cameras generally do not contain a built-in GPS receiver, the ! ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ! ! .A few i specialized digital cameras have been developed that can be , i electronically connected to an external GPS receiver to facili- l L!~!~J~~2.!~Ei~-~~~-1~!:.?..!2.~~--------------------------------------------------------------------J Photographers often desire to upload their digital photos to 45 a . A photographer can upload photos by coupling the digital camera to a (client) computer with a . It is also common for users of wireless mobile 50 telecommunications network user equipment (UE) that include built-in digital cameras (sometimes referred to as "camera phones") to upload photos taken with the UE via the \Vireless telecommunications nerw,ork. The \Vireless telecom- munications network service provider typically provides a 55 server on which the user can at least temporarily store the uploaded photos and with which the user can transfer them via the Internet to other computers, such as a server operated by a third-party photo hosting service or the user's home computer. It is also known to couple a digital camera to a UE 60 via a cable, wireless local area network (WLAN) link such as WIFI, or wireless personal area network (WPAN) link such as BLUETOOTH, and upload the images from the camera to the wireless telecommunications network via the UE. In a network employing such technologies, a received MMS message that includes a photo is routed to a Multimedia Messaging Service Center (MMSC), which performs many of the functions nec- essary to forward and otherwise handle the image. The geographic location of a UE can be determined to an accuracy level greater than that defined merely by the cell in which the UE is located. The earliest use of sub-cell-level UE location related to emergency calling (e.g., the E911 system used in the United States), in which the network determines with sub-cell accuracy the location of a caller's phone and routes the call to an emergency operator who can, if neces- sary, dispatch emergency personnel to the user's location. Location-Based Services (LBS) are now expanding far beyond emergency calling, to such diverse uses as navigation, mapping, tracking and gaming. One common method by which UE location can be determined is known as Assisted GPS (A-GPS). An A-GPS- enabled UE contains a limited processing power GPS receiver, which is smaller and more power-efficient than a full-function GPS receiver. InA-GPS, the W'ireless telecom- munications network transmits "assistance data" to the UE to aid the UE GPS receiver in finding and utilizing signals transmitted by the GPS satellites, as the GPS satellite signals that reach the receiver are sometimes weak or few in number due to shielding and reflection from buildings or terrain and, to the extent they can be detected at all, would otherwise require the processing power of a full-function GPS receiver to establish a location using them. As minimizing size and weight and maximizing battery charge conservation remain important goals for producers of cell phones, incorporating a . Yahoo!, Inc., which provides the FLICKR Internet photo hosting service, has 65 more powerful GPS receiver capable of rapidly determining location without assistance data is not considered a commer- cially viable option. Some cell phones whose designs empha- US 8,548,424 B 1 3 size portability and battery conserv'ation simply do not include even a limited processing power GPS receiver. Nev- ertheless, A-GPS is only one method by which a network can determine UE location. Other methods are based upon trian- gulation from known locations of base stations. Examples of such triangulation methods include Time Difference of Arrival (TDOA), Angle of Arrival (AOA), Forward Link Tri- lateration (FLT), Observed Time Difference (OTD). In net- works generally of the Global System for Mobile telecom- 4 although specific features, configtirations, arrangements and steps are discussed below, it should be understood that such specificity is for illustrative purposes only. A person skilled in the relevant art will recognize that other features, configura- tions, arrangements and steps are useful without departing from the spirit and scope of the invention. As illustrated in FIG. 1, in an exemplary embodiment of the munications (GSM) and Universal Mobile 10 invention a mobile telephone (also referred to as a cell phone or mobile station) or other user equipment (UE) 10 commu- nicates with a wireless telecommunications network (also referred to as a cellular telecommunications network) having Telecommunications System (UMTS) types, the network ele- ment that responds to an LBS query by determining the user equipment location is known as a Gateway Mobile Location Center (GMLC). . It is to the provision of such a method and system that the present invention is directed. SUMMARY OF THE INVENTION an architecture generally in conformance with the GPRS and UMTS standards. As known in the art, GPRS is a standard that has been adopted by many network service providers to expand the capabilities of telecommunications networks, such as those conforming to the Global System for Mobile telecommunications (GSM), to include services such as Wireless Application Protocol (WAP), Short Messaging Ser- vice (SMS) and Multimedia Messaging Service (MMS) as 20 well as Internet communication services such as e-mail and World Wide Web access. As described in further detail below, MMS allows UE 10 to transmit to the network messages that include multimedia objects (e.g., images, audio, video, rich text, etc.). Although in the exemplary embodiment of the The present invention relates to a wireless mo bile telecom- munications network method and system for geotagging an image (e.g., photo) or other multimedia message transmitted from wireless user equipment (UE) and received by the net- work. In an exemplary embodiment of the invention, in response to receiving the image, the network uses a conven- tional Location-Based Services (LBS) query to determine the 30 geographic location of the UE. 25 invention the network conforms to GPRS and uses MMS to handle image files or other multimedia messages, in other embodiments the network can use any other suitable means for handling such messages. The network architecture shown in FIG. 1 is intended only to be exemplary. In the exemplary embodiment, the wireless telecommuni- cations network includes one or more Base Stations 12, a Mobile Switching Center 14, a Gateway Mobile Switching Center 16, a Serving GPRS Support Node (SGSN) 18, a Gateway GPRS Support Node (GGSN) 20, a Multimedia .A Gateway Mobile Location Center (GMLC) or similar conventional means for determining UE location can respond to the LBS query by determining the geographic location of the UE in a conventional ma11ner, such as by using a triangulation method. In the exemplary embodiment of the invention, in which the network is generally of the GSM/UMTS type, < .When the network receives the image from the UE, the MMSC has the geotagger determine the UE geographic location and tag the image. The geotagger can act as an LBS client and query the GMLC for the UE location. Alternatively, in other embodi- ments of the invention, the MMSC or similar network ele- ment can itself be programmed or configured to determine the UE location and tag the image. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates a wireless mobile telecommunications system in which photos are geotagged in accordance with an exemplary embodiment of the present invention. FIG. 2 is a flow diagram, illustrating a method for geotag- ging an image in the wireless mobile telecommunications network of FIG. 1. 35 Messaging Service Center (MMSC) 22, and a Gateway Mobile Location Center (GMLC) 24. For oumoses of claritv. other elements that are c~mmonly includ;d i~ GSM, UMTS, GPRS, etc., networks, are not shown, and those well-known elements that are shown in FIG. 1 are shown in generalized 40 form. Accordingly, for example, Base Stations 12 can com- prise one or more Base Station Controllers, Base Transceiver Stations, Radio Network Controllers, Node-B's, Base Station Subsystems or other such elements that are known in the art to provide the air interface between user equipment (e.g., UE 45 10) and the remainder of the network in networks generally of the GSM or GPRS/UMTS types. The network further includes a novel geotagger 26 that, as described in further detail below, causes image files to be tagged with geographic location information as they are uploaded from UE 10 to the 50 network. The user can use UE 10 to transmit or up load the photographs or other images to the netw,ork in the form of 55 MMS messages. Accordingly, UE 10 includes an MMS mes- saging client, which is a software program having a user interface with which the user can interact to compose, address, transmit, receive, and view MMS messages. The use FIG. 3 illustrates a portion of the wireless mobile telecom- 60 munications network of FIG. 1 in further detail. of an MMS client in a cell phone or other user equipment to, for example, transmit a captured digital image, is well under- stood in the art and therefore not described in further detail DETAILED DESCRIPTION In the following description, like reference numerals indi- 65 cate like components to enhance the understanding of the invention through the description of the drawings. Also, herein. The wireless mobile telecommunications network inter- faces with external telecommunications networks 28 that operate in accordance with the Signaling System 7 (SS7) standard, such as the Public Switched Telephone Network (PSTN). The wireless telecommunications network also US 8,548,424 B 1 5 interfaces with the 30 via r-v1r-v1SC 22 and GGSi",r 20. As known in the art, MMSC 22 controls the uploading of multimedia files, such as those of MMS messages received from UE 10, via Internet 30. Ultimately, a person can use a client computer 32 coupled to Internet 30, either directly or via an intermediate service provider system (not shown), to receive and view uploaded image files. The flow diagram of FIG. 2 illustrates an exemplary method of the present invention. At step 34, the wireless mobile telecommunications network receives a digital image 10 from UE 10. The image can be, for example, in theformofan MMS message. The image can be a photograph that the user has taken using the built-in camera of UE 10 or any other suitable image. The MMS message is routed to MMSC 22 in the conventional manner, via SGSN 18 and GGSN 20. 15 MMSC 22 detects the arrival of an MMS message bearing image data. Referring briefly to FIG. 3, MMSC 22, which, like many common network elements, is essentially a net- work computing device and thus includes characteristic pro- cessor 36 and memory 38 elements that, together with soft- 20 ware elements executed from memory 38, define the programmed processing system that performs the functions for which the device has been progranmied. As the processor system ofMMSC 22 is programmed to perform various well- known functions relating to the handling ofMMS messages, 25 these message handling functions are embodied in software elements or instructions that processor 36 executes from memory 38 but which are not shown for purposes of clarity. The image detector element 40, shown for purposes of illus- tration as conceptually residing in memory 38, relates to 30 detecting the arrival of an MMS message bearing image data, as noted above with regard to step 34 (FIG. 2). The processor system, in response to detecting the arrival of an MMS mes- sage bearing image data, forwards the MMS message to geotagger 26, which is similarly computer-like in that it 35 includes a processor 42 and memory 44 that, together with software elements executed from memory 44, define a pro- grammed processing system. The software elements shown for purposes of illustration as conceptually residing in memory 44 include an image interface 46, a Location-Based 40 Services (LBS) query generator 48, and a tag generator 50. Image interface 46 represents the function of receiving the MMS message from MMSC 22 and returning the MMS mes- sage, once its image has beengeotagged, to MMSC 22. Loca- tion-Based Services query generator 48 acts as client soft- 45 ware with respect to GMLC 24 (FIG. 1) and represents the function of generating an LBS query to GMLC 24 in response to receipt of the MMS message. Thus, as indicated by step 52 (FIG. 2), geotagger 26 submits this LBS query to GMLC 24 when it receives the MMS message from MMSC 22. Tag generator 50 represents this function of trans- forming the reply from GMLC 24 indicating the geographic location ofUE 10 into a suitable format, and modifying the MMS message data accordingly. As noted above, the result- ing geotagged MMS message is returned to MMSC 22. The 60 message proceeds through any additional steps of the overall uploading process in the conventional manner, under control ofMMSC 22, as indicated by step 56 (FIG. 2). Note that in the exemplary embodiment the messages that are exchanged between MMSC 22 and geotagger 26 pass 65 through GGSN 20. Geotagger 26 tags the MMS message and returns it to MMSC 22. Alternatively, in another embodiment 6 (not shown), . In still other embodiments (not shown), the processing system of GGSN 20 (not shown) can be pro- grammed in the manner described above instead of the system including a separate geotagger 26. Still other variations will occur readily to persons skilled in the art to which the inven- tion pertains in view of the teachings herein. It will be apparent to those skilled in the art that various modifications and variations can be made to this invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention covers the modifications and variations of this invention provided that they come within the scope of any claims and their equiva- lents. With regard to the claims, no claim is intended to invoke the sixth paragraph of 35 U.S.C. Section 112 unless it includes the term "means for" followed by a participle. What is claimed is: 1. A method for geotagging an image in a wireless mobile telecommunications network, the method comprising: receiving a digital multimedia message at a Multimedia Messaging Service Center (MMSC), the digital multi- media message being a multimedia messaging service (MMS) message, the digital multimedia message trans- mitted by a user equipment (UE); determining that the digital multimedia message bears image data, the determination made via an image detec- tor element onboard the MMSC which detects the image data in the MMS message; and sending the digital multimedia message to a geotagger in response to detecting the image data, wherein the geotagger submits a Location-Based Services (LBS) query to a network element of the wireless mobile telecommu- nications network via a LBS query generator onboard the geotagger, receives a reply from the network element including the geographic location of the UE, and modifies data of the digital multimedia message to include a tag indicating the geographic location. 2. The method of claim 1, wherein the network element determines UE location using a triangulation method. 3. The method of claim 1, wherein the network element comprises a Gateway Mobile Location Center (GMLC). 4. The method of claim 1, wherein the geotaggercomprises a server device coupled to a Gateway General Packet Radio Service Support Node (GGSN). 5. A system for geotagging an image in a wireless mobile telecommunications network, comprising: a message-receiving network element for receiving a digital multimedia message transmitted by a wireless user equipment (UE), the digital multimedia message being a multimedia messaging service (MMS) message, and determining that the digital multimedia message bears image data, the determination made by an image detector element onboard the message-receiving net- work element which detects the image data in the MMS message; and a geotagging network element for receiving the digital multimedia message bearing image data from the message-receiving element, the digital multimedia message sent in response to detecting the image data, US 8,548,424 B 1 7 submitting a Location-Based Serv'ices (LBS) query via a LBS query generatoronboard the geotagging network element, and receiving a reply including the geographic location of the UE, and m~difying data of the digital multimedia message to mclude a tag indicating the geographic location; and a location-determining network element for receiving a LBS query from the geotagging network element, determining the geographic location of the UE on a sub-cell-level basis, and sending the geographic location to the geotagging net- work element. 10 6. The system of claim 5, wherein the location-determining 15 network element determines UE location using a triangula- tion method. 7. The system of claim 5, wherein the location-determining network element comprises a Gateway Mobile Location Cen- ter (GMLC). 8. The system of claim 5, wherein: the message-receiving network element comprises a Mul- timedia Messaging Service Center (MMSC). 9. The system of claim 8, wherein the geotagging network element comprises a server device coupled to a Gateway General Packet Radio Service Support Node (GGSN). 20 8 10. A system for geotagging an image in a wireless mobile telecommunications network, comprising: a Multimedia Messaging Service Center (MMSC) in the network for receiving a digital multimedia message transmitted by a user equipment (UE), the digital mul- timedia message being a multimedia messaging service (MMS) message; an image detector element onboard the MMSC for deter- mining that the digital multimedia message bears image data; means in the network for determining a geographic loca- tion of the UE, the geographic location determined on a sub-cell-level basis, the means receiving the digital mul- timedia message in response to the detection of the image data; and means in the network for submitting a Location-Based Services (LBS) query to the means for determining a geographic location via a LBS query generator receiving a reply from the means for determining a geo- graphic location, the reply including the geographic location of the UE, and modifying data of the digital multimedia message to include a tag indicating the geographic location. * * * * * 6l7l2016 Sports Life Tech Weather Search Tech Products Products home Talking Tech Edward C. Baig Kim Komando Ask Kim Gaming Gaming home Arcade Jinny Gudmundsen Marc Saltzman Science & Space Science & Space April Holladay Dan Vergano USATOD/.W.con-1 - Flickr of idea on a gaming projec! led to phtJto website Posted 2/27/2006 8:26 PM Updated 2/28/2006 3:21 AM Flickr of idea on a gaming project led to photo website By Jefferson Graham, USA TODAY SUNNYVALE, Calif. - Caterina Fake knew she was onto something when one of the engineers at her Vancouver, British Columbia-based online game start-up created a cool tool to share photos and save them to a Web page while playing. A screen shot of Flickr's site. This week in space ;:; :s;~;;;;;;; ~: ;::::;;: C ~;;:;;; :;t_~~% Courtesy: Flickr.com Wireless Center Hotspot finder Viii-Fi primer Columnists Columnists index Andrew Kantor Kevin Maney More Tech Hot Sites Tech briefs RSSfeeds Classifieds Marketplace Arcade Music Shopping Special Offer Newspaper Classifieds "It turned out the fun was in the photo sharing," she says. Fake scrapped the game. She and her programmer husband, Stewart Butterfield, transformed the project into Flickr. In less than two years, the photo-sharing site - now owned by Internet giant Yahoo - has turned into one of the Web's fastest- growing properties. By Martin Klimek, for USA TODAY Flickr co-founders Stewart Butterfield and Caterina Fake take a photo of themselves during a photo shoot in San Francisco. "Had we sat down and said, 'Let's start a photo appiication,' we wouid have faiied," Fake says. "We would have done all this research and done all the wrong things." Fake and Butterfield's company, Ludicorp, never did launch a game. Yahoo bought the company in March for an undisclosed sum and moved the 11-person team to Yahoo headquarters in Sunnyvale. Since then, Flickr has been on a roll, riding the wave of consumer fascination with digital cameras. Shutterbugs are less likely to make many prints these days but still want an outlet to show off their work. That's where a free, ad-supported site such as Flickr comes in. Flickr's traffic grew 448% to 3.4 million from December 2004 to December 2005, according to Internet measurement firm Nielsen/NetRatings. In the nearly 12 months since Yahoo purchased it, the site went from 250,000 registered users to more than 2 million. About 100 million photos have been posted at the site. Chad Hurley, CEO of video-sharing site YouTube, is a Flickr fan. He says the site has resonated so quickly with the public "because it brought innovation to a problem people thought was already solved - how to share photos online." •What is it? A photo-sharing site, owned by Yahoo, where users can share full-resolution images with friends, family and the open Flickr's snazzy photo-sharing features set it apart from the dozens of other photography sites. Friends can check out newly posted pictures via searching and add their own "notes" to photos they like. One distinctive tool lets bloggers simultaneously hUp:!lusa!.oday30.tJSi~loday .com11ecl'Vpr odur.ts/2006-02-27-lllckr ,y,.hlrn 1/3 6l7l2016 USATOD/.W.con-1 · Flickr of idea on a gaming projec! led to phtJto website Internet, in a snazzy, graphically post photos on their own biogs and at Flickr. rich setting. •How much? The ad-supported version lets users upload 20 megabytes of photos a month free. That's about 10 to 25 pictures, depending on the resoiuiion. For $24.95 a year, you get an ad-free version, and can upload 2 gigabytes a month. •What else does Flickr offer? Calendars, posters and DVDs of photos. •Another tool: Members can upload pictures and post them on their biogs. It also uses a tool called "tagging" - adding a few words of text to each posted photo - so that a picture can be easily searched online. Trip Hosley, co-owner of San Francisco club Shine, became a fan after his brother, Nate, became a father for the second time and tagged pictures of the newborn "Deuce" at the Flickr site. "Within an hour of the birth, there were 100 pictures online that all our family and friends could see, instantly, just by searching for 'Deuce,'" he says. Hosley even installed a photo booth at Shine where patrons can sit and snap their own pictures and have them instantly uploaded to Flickr. 'Getting' it Yahoo bought the company as much for Fake and Butterfield as it did to land Flickr, says analyst Charlene Li of market tracker Forrester Research. SUNNYVALE, Calif. - Biogs brought Flickr co-founders and husband-and-wife team Caterina Fake and stewart Butterfield together six years ago. Butterfield, a Web developer then living in British Columbia, was a fan of Fake's musings about life and technology at her caterina.net blog. In 2000, he happened to meet her at a San Francisco party, where she declined his request for a date, instead leaving with her then- beau Evan Williams, a co-creator of Google's Blogger. Six months later, Butterfield read on a blog that she1d bioken up with Williams. He arranged to return to San Francisco and try his luck again. This lime, he invited her to British Columbia to go skiing, and she said yes. On the slopes, he proposed they try creating a website together. The rest, as they say, is history. Fake says Flickr's team has adapted well in the Yahoo universe. She continues to blog at caterina.net about everything from novels she's read to her take on the Yahoo-Google rivalry - all without interference from Yahoo higher-ups. Butterfield says the couple survives the intensity of working together every day. "There was certainly a time where ii was Flickr in the office and Flickr at home," he says. Now, though, "We're getting slightly better at marking the boundary between work and play." - By Jefferson Graham "Caterina is very dynamic, smart and energetic, and her job is to help Yahoo with the social tools,'' Li says. She calls Butterfield "a brilliant programmer." Butterfield says Flickr's biggest innovation came from recognizing the social nature of photography. "It's meant to be shared, talked about, pointed to, saved, archived and available by as many means as possible," he says. Yahoo has two photo-sharing sites, Flickr and the more traditional "share-and-buy-prints"-oriented Yahoo Photos. But Yahoo executive Bradley Horowitz, the head of technology development, says there's room at Yahoo for multiple areas of photo sharing, just as it offers instant messaging and e-mail as two different communication tools. Yahoo has kept the Flickr site low-key, with no front- page promotion. Word of mouth is its best marketing tool, Horowitz says. "The right way to find Flickr is to be invited by a friend, to get plugged into the social network that way," he says. That way, you're more likely to "get it," he says. Flickr has a spare, hip look that displays photos larger and more stylishly than many others but isn't as simple to navigate as competitors such as Shutterfly and Kodak Easy Share Gallery. It uses terms such as "photostreams" to describe personal photo collections, "interestingness" to describe cool new pictures and "sets" to describe groups of photos. There is no simple tool to share pictures via e-mail. Instead, Flickr users must copy the Web address Flickr assigns to the photos into an e-mail. It's a deliberate omission. "With the kind of growth we've experienced, we're not sure our servers will be able to handle the onslaught from an e-mail application,'' Fake says. She says an e-mail function is coming soon: "We know people really want it, we just want to make sure we're prepared." hUp:!lusa!.oday30.tJSi~loday .com11ecl'Vpr odur.ts/2006-02-27-lllckr ,y,.hlrn 213 6l7l2016 USATOD/.W.con-1 - Flickr of idea on a gaming projec! led to phtJto website For now, Flickr users seem untroubled by the lack of a one-click e-mail feature. Ricky Fisher, a Seattle graphic designer, simply shares his pictures with others via the Flickr Web address. What he likes most about the service is its international flavor; he checks out interesting photographs from all over the world. At the bottom of Flickr's home page are four thumbnail images of the most recently uploaded pictures from the Flickr community. Fisher watches the page to see the new photos and reaches out to photographers whose work he admires. "You see a lot of amazing stuff this way," he says. Newspaper Home Deliverv - Subscribe Today USATODAY.com partners: USA WEEKEND •Sports Weekly• Education •Space.com Resources: Mobile news •Site map •FAQ •Contact us •E-mail news Jobs with us • Internships • Terms of service + Privacy policyNour California Privacy Right Advertise• Press Room• Media Lounge• Electronic print edition• Reprints and Permissions Add USATODAY.com RSS feeds £111'@ The Nation's Homepage Copyright 2008 USA TODAY, a division of Gannett Co. Inc. hUp:!lusa!.oday30.tJSi~loday .com11ectVpr odur.ts/2006-02-27-lllckr y,.hlrn Copy with citationCopy as parenthetical citation