阅读说明：本技术 玩具系统 (Toy system ) 是由 P.K.多辛 A.扎瓦拉 J.索德伯格 于 2019-06-25 设计创作，主要内容包括：一种玩具系统,包括：多个玩具构造元件,图像捕获装置和处理器。图像捕获装置可操作以捕获由玩具构造元件构造成的玩具构造模型的一个或多个图像；其中处理器配置为用以：执行数字游戏,该数字游戏包括计算机可执行代码,该计算机可执行代码被配置为用以使处理器提供数字娱乐体验；以及接收解锁码,所述解锁码指示一个或多个虚拟对象；响应于接收到解锁码,解锁与所接收的解锁码关联的一个或多个虚拟对象以用于数字娱乐体验,每个虚拟对象与所述玩具构造元件中的相应一个关联,或者与由玩具构造元件构成的相应的玩具构造模型关联；接收由所述图像捕获装置捕获的一个或多个图像；识别所述一个或多个图像中的一个或多个玩具构造元件和/或玩具构造模型；响应于识别到与解锁的虚拟对象中的第一个关联的第一玩具构造元件或第一玩具构造模型,提供涉及解锁的所述第一虚拟对象的数字娱乐体验。(A toy system, comprising: a plurality of toy construction elements, an image capture device and a processor. The image capture device is operable to capture one or more images of a toy construction model constructed from the toy construction elements; wherein the processor is configured to: executing a digital game, the digital game comprising computer executable code configured to cause a processor to provide a digital entertainment experience; and receiving an unlock code, the unlock code indicating one or more virtual objects; in response to receiving an unlocking code, unlocking one or more virtual objects associated with the received unlocking code for a digital entertainment experience, each virtual object being associated with a respective one of the toy construction elements or with a respective toy construction model composed of toy construction elements; receiving one or more images captured by the image capture device; identifying one or more toy construction elements and/or toy construction models in the one or more images; in response to identifying a first toy construction element or a first toy construction model associated with a first one of the unlocked virtual objects, providing a digital entertainment experience involving the unlocked first virtual object.)

1. A toy system comprising a plurality of toy construction elements, an image capture device operable to capture one or more images of toy construction models constructed from the toy construction elements, and a processor; (ii) a Wherein the processor is configured to:

-executing a digital game, the digital game comprising computer executable code configured to cause the processor to provide a digital entertainment experience;

-receiving an unlocking code, the unlocking code being indicative of one or more virtual objects;

-in response to receiving the unlocking code, unlocking one or more virtual objects associated with the received unlocking code for the digital entertainment experience, each virtual object being associated with a respective one of the toy construction elements or with a respective toy construction model constructed from the toy construction elements;

-receiving one or more images captured by the image capturing device;

-identifying one or more toy construction elements and/or toy construction models in the one or more images;

-in response to identifying a first toy construction element or a first toy construction model associated with a first virtual object of the unlocked virtual objects, providing a digital entertainment experience involving the unlocked first virtual object.

2. The toy system of claim 1; wherein the unlocking code is provided as a physical object carrying a machine-readable and/or human-readable code.

3. A toy system according to any one of the preceding claims; wherein the unlocking code is of limited use and wherein the processor is configured to determine whether the received unlocking code has been previously used beyond the limited use and the processor is configured to unlock the virtual object only if the code has not been previously used beyond the limited use.

4. A toy system according to any one of the preceding claims; wherein the digital game comprises computer executable code configured to cause the processor to control at least one virtual game item.

5. A toy system according to any one of the preceding claims; wherein the visual appearance of the unlocked first virtual object resembles the first toy construction element or the first toy construction model associated with the first virtual object.

6. A toy system according to any one of the preceding claims; wherein the processor is configured to associate a visual appearance with the unlocked virtual object in response to receiving the unlock code, in particular by receiving one or more captured images of a toy construction model whose visual appearance is to be associated with the unlocked virtual object; and the processor is configured to associate a visual appearance of the toy construction model with the unlocked virtual object.

7. A toy system according to any one of the preceding claims; wherein the processor identifies therein one or more toy construction elements and/or the one or more images received of toy construction models depicting a composite toy construction model constructed from at least a first toy construction model and a second toy construction model; wherein identifying one or more toy construction elements and/or toy construction models within the one or more images comprises identifying each of the first and second toy construction models included in the composite toy construction model; wherein the processor is configured to: in response to identifying the first toy construction model and the second toy construction model, wherein the first toy construction model is associated with an unlocked first virtual object and the second toy construction model is associated with an unlocked second virtual object, the processor provides a digital entertainment experience involving the unlocked first virtual object and the unlocked second virtual object.

8. The toy system of claim 7; wherein the processor is configured to further identify a spatial configuration of the first and second toy construction models relative to each other, and the processor is configured to modify the entertainment experience provided in response to the identified spatial configuration.

9. A toy system according to any one of the preceding claims; wherein one or more of the plurality of toy construction elements can comprise a visually identifiable code identifying the toy construction element or the toy construction model.

10. The toy system of claim 9; wherein the plurality of toy construction elements comprises one or more marker toy construction elements, the visual appearance of each of which represents an object code or a part of an object code.

11. A toy system according to claim 9 or 10; wherein the processor is configured to detect object codes within the one or more images and adjust the digital entertainment experience in response to the detected object codes.

12. A toy system according to any one of the preceding claims; wherein identifying the first toy construction element or the first toy construction model associated with the unlocked first virtual object comprises:

-identifying the first toy construction element as a toy construction element of a first type of toy construction element or the first toy construction model as a toy construction model of a first type of toy construction model, and

-detecting a first object code associated with the identified first toy construction element or the identified first toy construction model.

13. The toy system of claim 12; wherein the processor is configured to: in response to identifying the first toy construction element or the first toy construction model, the processor provides a digital entertainment experience involving a first instance of the plurality of instances of the unlocked first virtual object, the first virtual object being associated with the first type toy construction element or the first type toy construction model, and each of the plurality of instances of the virtual object being further associated with a respective object code.

14. A toy system according to claim 12 or 13; wherein the processor is configured to: in response to identifying a first toy construction element or a first toy construction model associated with a first one of the unlocked virtual objects, the processor stores the detected first object code associated with the unlocked first virtual object.

15. The toy system of claim 14; wherein the processor is configured to determine whether object code has been previously stored in association with the unlocked first virtual object, and the processor is configured to associate the detected first object code with the unlocked first virtual object only if no object code has been previously stored in association with the unlocked first virtual object.

16. A toy system according to claim 14 or 15; wherein the processor is configured to compare the detected first object code with a previously stored object code associated with the unlocked first virtual object, and the processor is configured to provide a digital entertainment experience involving the unlocked first virtual object only if the detected first object code corresponds to the previously stored object code associated with the unlocked first virtual object.

17. A processor-implemented method of operating a toy system, the toy system comprising a plurality of toy construction elements, an image capture device, and a processor; the image capture device operable to capture one or more images of one or more toy construction models constructed and placed by the toy construction elements within a field of view of the image capture device; wherein the method comprises the following steps:

-executing a digital game, the digital game comprising computer executable code configured to cause the processor to provide a digital entertainment experience;

-receiving an unlocking code, the unlocking code representing one or more virtual objects;

-in response to receiving the unlocking code, unlocking the one or more virtual objects associated with the received unlocking code for the digital entertainment experience, each virtual object being associated with a respective one of the toy construction elements or with a respective toy construction model constructed from the toy construction elements;

-receiving one or more images captured by the image capturing device;

-identifying one or more toy construction elements and/or toy construction models in the one or more images;

-providing a digital entertainment experience involving the unlocked first virtual object in response to identifying the first toy construction element or the first toy construction model associated with the unlocked first one of the virtual objects.

18. A processing device configured to perform the method of claim 17.

19. A computer program product comprising program code adapted to cause a processing device to perform the method according to claim 17 when executed by the processing device.

Technical Field

The present invention relates to the use of computer vision technology in life-like toys, and more particularly to a toy system employing such technology.

Background

Different attempts have been made to integrate physical objects into virtual games. However, it is still desirable to provide a way to associate the physical world with a virtual game that can stimulate the user's interactive participation and provide an entertaining game. Therefore, a new method for interactive gaming is needed.

Currently, most computer games or so-called life-like toy systems that augment toys involve systems in which the toy must have a physical component that is configured to communicate with a particular reader through some form of wireless communication (e.g., RFID, NFC, etc.). Examples in such systems are disclosed in e.g. US2012/0295703, EP2749327 and US 2014/256430. It is generally desirable to provide a toy system that does not require that the toy comprises elements that are capable of communicating with a reader device in order to be able to identify the toy element and create a virtual digital representation thereof and associate it with other digital data.

WO 2011/017393 describes a system for inspecting toy construction models in a special context using computer vision. In this prior art system, the assembled model is detected on a special background plate on which a specific pattern is printed.

EP 2714222 describes a toy construction system for augmented reality.

WO 2018/069269 describes a toy system comprising a scannable block comprising a visible scannable code. These codes represent the strength in the game.

In view of this prior art, it is still desirable to provide improved toy systems.

Disclosure of Invention

According to one aspect, embodiments of a toy system are disclosed herein. The toy system includes: a plurality of toy construction elements, an image capture device and a processor. The image capture device is operable to capture one or more images of a toy construction model constructed from the toy construction elements. The processor is configured to:

-executing a digital game, the digital game comprising computer executable code configured to cause the processor to provide a digital entertainment experience;

-receiving an unlocking code, the unlocking code being indicative of one or more virtual objects;

-in response to receiving the unlocking code, unlocking one or more virtual objects associated with the received unlocking code for the digital entertainment experience, each virtual object being associated with a respective one of the toy construction elements or with a respective toy construction model constructed from the toy construction elements;

-receiving one or more images captured by the image capturing device;

-identifying one or more toy construction elements and/or toy construction models in the one or more images;

-in response to identifying a first toy construction element or a first toy construction model associated with a first virtual object of the unlocked virtual objects, providing a digital entertainment experience involving the unlocked first virtual object.

Thus, the toy system enables a user to interact with a digital game by: the one or more toy construction elements and/or toy construction models are presented to the toy system such that the image capture device captures one or more images of the one or more toy construction elements and/or toy construction models, and the processor identifies at least the first toy construction element or the first toy construction model. The toy system provides an entertainment experience involving the first virtual object associated with the identified first toy construction element or first toy construction model if the first virtual object associated with the identified first toy construction element or first toy construction model has previously been unlocked by the respective unlocking code.

Thus, the toy construction element itself does not need to provide a recognizable unlocking code, allowing a user to construct a toy construction model using conventional toy construction elements without the need for special adaptation of the toy construction members for the toy system. However, the toy system provides an authentication mechanism that only provides access to virtual objects that are properly authenticated by the unlock code. The processor may be configured to determine whether the received unlock code is a true unlock code.

Furthermore, since embodiments of the process only require identification of toy construction models associated with the unlocked virtual object, the identification task is simplified, as the number of different toy construction models to be identified and distinguished from each other is limited to a set of known toy construction models.

The unlock code may be in various forms, such as a bar code, QR code, or other visually identifiable code. In other embodiments, the unlock code may be provided as an RFID tag or other electronic tag that may be read by the data processing system. Alternatively still, the unlock code may be provided as a code to be manually entered by the user as a sequence of alphanumeric symbols or in other suitable manner. In some embodiments, the unlock code may be provided as a combination of two or more of the above and/or in a different manner.

The unlock code may be provided as a physical article, such as a token or card having a machine-readable code and/or a human-readable code printed thereon or having an electronic tag incorporated therein. The physical object may be a toy building element comprising coupling members for attaching the physical object to other toy building elements in the set. Alternatively, the physical object may be a physical object different from the toy construction elements in the toy construction elements, i.e. without coupling members compatible with the toy construction system. The unlock code may also be provided as part of the packaging of the toy construction set, such as printed on the interior of the container, or otherwise inaccessible prior to opening the packaging.

The unlock code may be a unique code. The unlocking code may be a one-time code (or otherwise limited use code), i.e., the processor may be configured to determine whether the received unlocking code has been previously used, and to unlock the virtual object only if the code has not been previously used. Thus, unlocking the virtual object may include marking the unlocking code as used/expired.

The determination as to whether the unlock code has been previously used may be accomplished in a number of ways. For example, when the unlocking code is provided as an electronic tag, the tag may include a rewritable memory and the processor may be configured to delete the unlocking code from the tag or otherwise mark the unlocking code as used/expired. Alternatively, the toy system may comprise a central repository of authentic unlock codes, for example maintained by a server computer. The processor may be communicatively connected to the repository, for example via a suitable communication network (e.g. the internet). In response to receiving the unlock code, the processor may request the repository to validate the received unlock code. The repository may respond to the request with an indication as to whether the unlocking code is authentic and/or whether the unlocking code has been used. The repository may also communicate to the processor which one or more virtual objects are associated with the unlocking code. After unlocking the corresponding virtual object, the unlock code may be marked in the store as used (e.g., in response to an initial request or in response to a subsequent indication by the processor that the corresponding virtual object has been successfully unlocked).

Unlocking the virtual object may include unlocking the virtual object in an instance of the digital game, such as unlocking the virtual object on a particular data processing device and/or for a given user. To this end, the unlocked virtual object may be associated with a user ID, which may be stored locally on the processing device and/or in a central user repository. In some embodiments, the receiving of the unlock code may be performed as part of the digital game, particularly when the digital game is executed, for example as part of a digital entertainment experience. In some embodiments, the processor may receive the unlocking code prior to providing the digital entertainment experience (e.g., during an initial phase of the digital game), or even prior to executing the digital game, e.g., under control of a computer program different from the digital game.

In embodiments where the unlock code is provided as an electronic tag, such as an RFID tag, the toy system includes a suitable electronic tag reader, such as an RFID reader. In embodiments where the unlock code is provided as a visually detectable code, the toy system includes a suitable visual tag reader, such as a camera or other image capture device. In particular, the same image capturing device may be used for reading the unlocking code and for identifying the toy construction element and/or the model. It will be appreciated that these two operations may typically be performed in separate steps and based on different captured images.

Many types of digital entertainment experiences, particularly digital games, may be enhanced by the unlocking and identification processes described herein, including but not limited to: nurturing games, battle-type games (player-to-player, or player-to-computer), tournament games and role-playing action/resource games, virtual building games, massively multiplayer online games, strategy games, augmented reality games, games on mobile devices, community games that allow users to collect digital items, and the like. In some embodiments, the digital game includes computer executable code configured to cause the processor to control at least one virtual object. Examples of virtual objects include virtual characters, such as virtual player characters controlled by the toy system directly in response to user input, or non-player characters controlled by the toy system based on game rules. Other examples of virtual objects include inanimate objects, such as accessories that a virtual character may use, such as weapons, vehicles, clothing, armor, food, currency within a game or other types of in-game resources, and so forth. Thus, the digital game may be of a type in which the user controls a virtual object (e.g., a virtual character in a virtual game environment). Alternatively or additionally, the digital game may provide different forms of entertainment experiences, such as a digital bring-up game, a digital game in which a user may build a digital world or other structure from multiple virtual objects, a strategy game, an entertainment experience in which a user collects virtual objects, a social platform, and so forth.

In general, a virtual object may be a virtual character or other animate or inanimate virtual item, such as a vehicle, a house, a structure, an accessory for use by the virtual character (e.g., a weapon, clothing, jewelry, tools, etc.), and so forth. In general, a virtual object may represent any type of game asset.

The unlock code may represent a single virtual object or multiple virtual objects. For example, a toy construction set may include a plurality of toy construction elements (e.g., disposed in a box or other container) from which one or more toy construction models may be constructed. The toy construction set may further comprise one or more unlocking codes, for example an unlocking code contained in the container. In some embodiments, a single unlocking code may be provided to unlock a plurality of virtual objects, for example associated with respective toy construction elements in the set and/or with respective toy construction models that may be constructed from toy construction elements included in the toy construction set. In other embodiments, the toy construction set may comprise a plurality of unlocking codes, each unlocking code for unlocking a respective one of the toy construction elements and/or models.

The identification of the toy construction elements and/or models may be performed using known methods from the field of computer vision, for example as described in WO 2016/075081.

In some embodiments, unlocking a virtual object may not automatically assign or activate the virtual object, but may only make the virtual object available to a user of the digital game, e.g., so that the user may subsequently select, assign, or activate the virtual object. To this end, the digital game may create a representation of the virtual object and/or add the virtual object to a set of available virtual objects.

The video appearance of the unlocked first virtual object may resemble the first toy construction element or model associated with the first virtual object. The visual appearance may be predetermined, i.e. the first virtual object may resemble a predetermined toy construction model or a predetermined toy construction element.

In an alternative embodiment, in response to receiving the unlock code, unlocking the one or more virtual objects may include associating a visual appearance with the unlocked virtual object. To this end, in some embodiments, the system may allow a user to capture one or more images of a toy construction model whose visual appearance is associated with the unlocked virtual object. Thus, the user may customize the visual appearance of the unlocked virtual object. In particular, the processor may identify one of a set of available toy construction models and apply a corresponding visual appearance to the unlocked virtual object.

For the purposes of this specification, a toy construction model is a coherent structure constructed from two or more toy construction elements. A toy construction element is a single coherent element that cannot be broken down in a non-destructive manner into smaller toy construction elements in a toy construction system. The toy construction model or toy construction element may be part of a larger structure, such as a larger toy construction model, while still being individually identifiable. A toy construction model or element identified or recognizable by a processor in a captured image refers to a toy construction model or element that is individually identified or recognizable, whether captured in the image itself or as part of a larger toy construction model. For example, the processor may be configured to identify portions of the toy construction model at different stages of construction. Thus, as the user builds a wall that will be part of a larger building, for example, the processor may be operable to identify the wall as a partial toy construction model and the entire building as a complete toy construction model.

In any case, the received one or more images in which the processor identifies one or more toy construction elements and/or toy construction models may depict a composite toy construction model constructed from at least a first toy construction model and a second toy construction, or a composite toy construction model constructed from a first toy construction model and other first toy construction elements. The first and second toy construction models may be connected to each other directly or indirectly via other toy construction elements to form a coherent composite toy construction model. Likewise, the first toy construction model and the other first toy construction elements may be connected to each other directly or indirectly via other toy construction elements, thereby forming a coherent composite toy construction model. Thus, the composite toy construction model is formed as a coherent structure, which is formed by two or more interconnected individual toy construction models, and/or by one or individual toy construction models interconnected with one or more further toy construction elements. Herein, the terms "individual toy construction model" and "further toy construction element" refer to toy construction models or elements that are individually recognizable by a processor of the toy system.

For example, a composite toy construction model may comprise a vehicle constructed from a plurality of toy construction elements and a doll riding or driving the vehicle, wherein the doll is itself constructed from a plurality of toy construction elements. In another example, the composite toy construction model comprises a doll constructed from a plurality of toy construction elements, and the doll may carry a weapon which may be formed as a single toy construction element, or the weapon may itself be constructed from a plurality of toy construction elements.

Thus, identifying one or more toy construction elements and/or toy construction models within the one or more images may comprise identifying each of the first and second toy construction models comprised in the composite toy construction model, and/or identifying each of the first toy construction model and the first toy construction element comprised in the composite toy construction model.

In general, in some embodiments, the process may be configured to identify multiple toy construction models in the same image, such as individual toy construction models placed adjacent to one another, or interconnected toy construction models that form a composite toy construction model.

Thus, in response to identifying a first toy model and a second toy model, wherein the first toy construction model is associated with the unlocked first virtual object and the second toy construction model is associated with the unlocked second virtual object, the processor may be configured to provide a digital entertainment experience involving the unlocked first and second virtual objects, in particular to cause the first and second virtual objects to interact with each other. For example, the entertainment experience may involve a composite virtual object formed as a combination of first and second virtual objects, e.g. formed as a vehicle comprising a driver/rider.

Similarly, in response to identifying a first toy construction model and a first further toy construction element, wherein the first toy construction model is associated with an unlocked first virtual object and the first further toy construction element is associated with an unlocked second virtual object, the processor may be configured to provide a digital entertainment experience involving said unlocked first and second virtual objects, in particular such that the first and second virtual objects interact with each other. For example, the entertainment experience may involve a composite virtual object formed as a combination of first and second virtual objects, such as a virtual character formed as a weapon or other accessory.

Thus, the user may select a combination of one or more, e.g. unlocked, virtual objects by constructing and capturing images of the respective toy construction models and/or by capturing images of the respective toy construction elements. In some embodiments, the selection may be performed a single time, while in other embodiments, the toy system may allow the user to repeatedly select different virtual objects and/or different combinations of virtual objects to be included in the digital entertainment experience.

In some embodiments, a plurality of individually identifiable toy construction models and/or further toy construction elements forming a composite toy construction model may be interconnected in different spatial configurations relative to one another. For example, the doll may be placed in or on the vehicle in different riding positions. The processor may be configured to identify not only individual toy construction models and/or elements, but also their spatial configuration. The processor may then modify the entertainment experience provided in response to the identified spatial configuration. For example, a weapon carried by a doll may provide different play attributes (e.g., power) depending on the manner in which the doll carries the weapon, such as whether it is carried left-handed or right-handed.

The digital entertainment experience involving the selected one or more virtual objects may take a variety of forms. For example, a digital entertainment experience involving a selected virtual object or objects may be a digital game, where the user controls the virtual object or combination of objects thus selected. In other embodiments, the digital entertainment experience may allow a user to place a selected virtual object or objects in a digital environment (e.g., a game scene), or the digital entertainment experience may allow a user to modify a selected virtual object, such as to decorate or use the selected virtual object as part of a digital building environment, to trade or otherwise interact with other users of the online entertainment experience.

In general, in some embodiments, each toy construction element includes one or more coupling members configured to detachably attach the toy construction element to one or more other toy construction elements in the toy construction system. To this end, the toy construction elements may comprise mating coupling members configured to be mechanically and detachably interconnected with coupling members of other toy construction elements, e.g. in a frictional and/or interlocking engagement. In some embodiments, the coupling member is compatible with toy construction systems.

In some embodiments, the image capture device is a camera, such as a digital camera, such as a conventional digital camera. The image capturing means may be a notebookA built-in camera of the portable processing device. In general, examples of portable processing devices include tablet computers, laptop computers, smart phones, or other mobile devices. In some embodiments, the image capturing device comprises a 3D capturing device, such as a 3D sensing camera, for example a depth sensing camera that combines high resolution image information and depth information. One example of a depth-sensing camera isCameras, such as model F200 provided in the intel corporation developer suite. The image capture device may be operable to capture one or more still images. In some embodiments, the digital camera is a video camera configured to capture a video stream. Thus, receiving one or more images captured by the image capture device may include receiving one or more still images and/or receiving a video stream.

The processor is adapted to detect toy construction elements and/or toy construction models in the captured images and to identify the toy construction models and/or elements. To this end, the toy system may comprise a library of known toy construction models and/or elements, each associated with information of the respective virtual object and whether the virtual object has been unlocked, and/or how to combine the toy construction element with other toy construction models or elements to form a composite toy construction model. The information about the corresponding virtual object may, for example, include one or more of: a virtual object identifier, information about the visual appearance of the virtual object, one or more virtual characteristics of the virtual object, a progress level of the digital entertainment experience, and the like.

Although the toy construction element itself need not be provided with an identifiable code or indicia, in some embodiments this may be useful anyway. Thus, in some embodiments, one or more of the plurality of toy construction elements may comprise a visually identifiable object code identifying the toy construction element or the toy construction model. In particular, in some embodiments, the plurality of toy construction elements of the system may comprise one or more marker toy construction elements, which may be toy construction elements having a visual appearance representing the object code or a portion thereof. For example, the toy construction system may comprise two or more marker construction elements which together have a visual appearance representing an object code when interconnected with each other. Thus, the object code may identify an individual toy construction element or toy construction model comprising one or more marker construction elements together representing the object code. The object code may be represented in the form of a barcode, QR code, or other machine-readable code. Alternatively, the object code may be encoded in the visual appearance of the marker toy construction element in other ways, for example invisibly embedded in a graphic decoration, a badge, a combination of colors, etc. The object code may be different from the unlock code. In some embodiments, the object code is a unique object code that uniquely identifies a particular toy construction element or model, such as a serial number or other type of unique code. In other embodiments, the object code is non-unique object code, i.e. such that there are two toy construction elements or models carrying the same object code. The use of non-unique object codes may allow visual markers/features representing the object codes to be smaller and/or less complex, as the information content represented by the markers/features may be kept small. This allows the code to be applied to small objects and/or in a manner that does not unduly interfere with the desired aesthetic appearance of the object. For example, object codes to be assigned to a particular toy construction element or model may be selected randomly, sequentially, or otherwise from a pool of codes. The pool of codes may be larger or smaller than the number of toy construction elements or models to which the object codes are assigned, and may be large enough to make the codes considered unique. For example, the size of the pool of codes is comparable or smaller compared to the number of toy construction elements or models to which the object codes are assigned, but preferably large enough that the risk of having two toy construction elements or models with the same object codes is possible to an acceptable extent is smaller.

The processor may be configured to detect object code within the one or more images. The process may then adjust the digital entertainment experience in response to the detected object code. To this end, the object code may be used in various ways.

In some embodiments, the processor may be configured to unlock the virtual object in response to receiving the unlock code, wherein the digital game is configured to provide multiple instances of the virtual object. In particular, the multiple instances of the virtual object may share one or more common characteristics, such as a common visual appearance and/or one or more common virtual attributes indicative of a common virtual behavior of the virtual object. In particular, the user may identify the multiple instances as instances of the same virtual object. However, the multiple instances may differ from each other in one or more particular characteristics, such as a change in visual appearance (e.g., different clothing, accessories, etc.) and/or a change in one or more particular virtual attributes (e.g., virtual fitness level, strength level), and so forth. For example, the virtual objects may evolve during the course of a digital entertainment experience (e.g., in a game), such as in response to game events, user interactions, and so forth. Examples of such evolutions may include changes in one or more virtual object properties (e.g., virtual health values, virtual capabilities, etc.). Providing multiple instances of a virtual object may include providing instances each having a respective attribute value of one or more virtual object attributes. Alternatively or additionally, the digital game may allow the user to customize the virtual object, such as by selecting accessories, clothing, and the like. Accordingly, the digital game may maintain a plurality of differently customized instances of the virtual object.

The unlocked first virtual object may be associated with a particular recognizable type of toy construction element or toy construction model. Each instance of at least a first virtual object may further be associated with a specific object code and a specific recognizable type of a toy construction element or toy construction model.

Thus, in some embodiments, identifying the first toy construction element or the first toy construction model associated with the unlocked first virtual object may comprise:

-identifying the first toy construction element as a toy construction element of a first type of toy construction elements or the first toy construction model as a toy construction model of a first type of toy construction models, and

-detecting a first object code associated with the identified first toy construction element or the identified first toy construction model.

The processor may then be configured to provide, in response to identifying the first toy construction element or the first toy construction model, a digital entertainment experience involving the unlocked first instance of the first virtual object, the first virtual object being associated with a first type of toy construction element or a first type of toy construction model, and the first instance of the virtual object being further associated with a first object code.

In some embodiments, the processor is configured to store the detected first object code associated with the unlocked first virtual object in response to identifying the first toy construction element or the first toy construction model associated with the unlocked first one of the virtual objects. In particular, in some embodiments, the processor may be configured to determine whether the object code has been previously stored in association with the unlocked first virtual object, and to associate the detected first object code with the unlocked first virtual object only if the object code has not previously been stored in association with the unlocked first virtual object, for example only if the first toy building element or the first toy construction model associated with the unlocked first virtual object is first identified after the first virtual object has been unlocked.

The processor may be further configured to compare the detected first object code with a previously stored object code associated with the unlocked first virtual object and to provide a digital entertainment experience involving the unlocked first virtual object only if the detected first object code corresponds to (in particular is equal to) the previously stored object code associated with the unlocked first virtual object.

Here and hereinafter, the term "processor" is intended to include any circuitry and/or device suitable for performing the functions described herein. In particular, the term "processor" includes a general-purpose or special-purpose programmable data processing unit, e.g., a microprocessor, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Programmable Logic Array (PLA), a Field Programmable Gate Array (FPGA), an application specific electronic circuit, or the like, or a combination thereof, of a computer or another data processing system. The processor may be integrated into a portable processing device, for example, wherein the portable processing device further comprises an image capture device and a display. However, it should be understood that the toy system may also be implemented as a client-server or similar distributed system, where image capture and other user interactions are performed by a client device, while image processing and recognition tasks and/or unlock code verification tasks may be performed by a remote host system in communication with the client device. According to some embodiments, an image capture apparatus or a mobile device having an image capture apparatus may be in communication with a computer, for example by wireless communication with a computing device comprising a processor, a data store and a display.

In some embodiments, the image capture device is in communication with a display that displays the scene seen by the image capture device in real time to facilitate alignment of the desired toy construction model whose image is to be captured.

The present disclosure is directed to various aspects, including the toy systems, corresponding devices, systems, methods, and/or products described above and below, each yielding one or more of the benefits and advantages described in connection with one or more other aspects, and each having one or more embodiments corresponding to the embodiments described in connection with one or more other aspects and/or disclosed in the appended claims.

In particular, according to one aspect, disclosed herein is a processor-implemented method of operating a toy system comprising a plurality of toy construction elements, an image capture device, and a processor; the image capture device operable to capture one or more images of one or more toy construction models constructed by the toy construction elements and placed within a field of view of the image capture device; wherein, the method comprises the following steps:

-executing a digital game, the digital game comprising computer executable code configured to cause a processor to provide a digital entertainment experience;

-receiving an unlocking code, the unlocking code representing one or more virtual objects;

-in response to receiving an unlocking code, unlocking one or more virtual objects associated with said received unlocking code for use in a digital entertainment experience, each virtual object being associated with a respective one of said toy construction elements or a respective toy construction model constructed from toy construction elements;

-receiving one or more images captured by the image capturing device;

-identifying one or more toy construction elements and/or toy construction models in the one or more images;

-providing a digital entertainment experience involving the unlocked first virtual object in response to identifying the first toy construction element or the first toy construction model associated with the unlocked first one of the virtual objects.

According to yet another aspect, disclosed herein is a processing device, such as a portable processing device, configured to perform one or more of the methods disclosed herein. The processing device may comprise a suitably programmed computer, such as a laptop, tablet, smart phone, PDA or other programmable computing device, such as a device having a graphical user interface and optionally a camera or other image capture means.

In general, a digital game may be implemented as a computer program, for example, as a computer-readable medium having the computer program stored thereon. Accordingly, according to another aspect, disclosed herein is a computer program that may be encoded on a computer readable medium, such as a disk drive or other storage device. The computer program comprises program code adapted to cause a processing device to perform one or more of the methods described herein when executed by the processing device. The computer program may be embodied as a computer readable medium, such as a CD-ROM, a DVD, an optical disk, a memory card, a flash memory, a magnetic storage device, a floppy disk, a hard disk, or the like, on which the computer program is stored. In other embodiments, the computer program product may be provided as a downloadable software package, for example, downloadable over the Internet or other computer or communication network on a Web server, or as an application to be downloaded from an App Store to the mobile device. According to one aspect, a computer-readable medium has stored thereon instructions, which, when executed by one or more processing units, cause the processing units to perform embodiments of the processes described herein.

The present disclosure also relates to a toy construction kit comprising a plurality of toy construction elements, one or more unlocking codes and instructions for obtaining computer program code which, when executed by a processing device, causes the processing device to perform the steps of an embodiment of one or more of the methods described herein. For example, the instructions may be provided in the form of an internet address, a reference to an application store, and the like. The instructions may be provided in a machine-readable form (e.g., QR code, etc.). The toy construction set may even include a computer-readable medium having computer program code stored thereon. Such a toy construction set may further comprise a camera or other image capturing device connectable to the data processing system.

Other features and advantages will become apparent from the following detailed description of embodiments, which proceeds with reference to the accompanying drawings.

Drawings

Preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings, in which

Fig. 1-3 schematically illustrate examples of toy construction sets of the toy systems described herein.

2-6 schematically illustrate examples of uses of embodiments of toy systems described herein.

Fig. 7 shows a flow chart of an example of a process described herein.

Fig. 8A-C illustrate examples of toy construction models for use with the toy systems described herein.

Fig. 9 schematically illustrates another example of the use of an embodiment of a toy system described herein.

Fig. 10-11 schematically illustrate examples of toy systems described herein.

Detailed Description

Embodiments of the methods and systems disclosed herein may be used in conjunction with a variety of toy objects, particularly with construction toys that use modular toy construction elements based on size constants, constraints, and fit, and with various assembly systems such as magnets, studs, slots, sleeves, with or without interlocking connections. Examples of such systems include, but are not limited to, the toy construction system under the trade name LEGO. For example, US3005282 and USD253711S disclose one such interlocking toy construction system and toy figure, respectively.

Fig. 1 schematically illustrates an example of a toy construction set of the toy system described herein. The toy construction set is available in a box 110 or other packaging form. The box comprises a plurality of conventional toy construction elements 120 from which one or more (two in the example of fig. 1) toy construction models 131, 132 can be constructed. In the example of fig. 1, toy figurines 131 and toy vehicles 132 may be constructed from toy construction elements in the kit. The toy construction set further comprises two cards 141, 142, which are made of plastic or cardboard, for example. Each card displays an image or other representation of the toy construction model and one of the machine-readable codes 171, 172 (in this example a QR code) representing the unlock code of the virtual object associated with the respective toy construction model, e.g. the virtual character and the virtual car, respectively. Alternatively, the unlock code may be provided to the user in a different manner (e.g., by mail or sale separately). Thus, each unlock code is a unique code that is attached to the product or provided to the user in the form of a physical printed code or a digital code. Then, when using (e.g., scanning or typing) the unlock code, the possibility of selecting an unlocked virtual object in/for the digital experience using computer vision may be unlocked.

Fig. 2 schematically shows another example of a toy construction set, which is similar to the set of fig. 1 in that it is obtained as a box 110 and which comprises a plurality of conventional toy construction elements 120 from which one or more (two in the example of fig. 2) toy construction models 131, 132 can be constructed. However, in this example the toy construction set comprises only a single card 141, the card 141 having an unlocking code 171 associated with one of the toy construction models (in this example a doll 131) that can be constructed from the toy construction elements of the set. It should be appreciated that in other embodiments, the kit may include one or more unlocking codes for unlocking one or more virtual objects associated with any subset of toy construction models that may be constructed from the toy construction elements of the kit.

Fig. 3 schematically shows another example of a toy construction set, similar to the set of fig. 1. However, in this example the toy construction set comprises toy construction elements (not explicitly shown) for constructing the toy figures 131 and the car 132, and an additional toy construction element 123, in this example a sword 123, which sword 123 may be carried by the toy figure 131, i.e. may be attached to the hand 178 of the toy figure 131.

The kit may include three cards 141 and 143 with corresponding unlock codes 171 and 173, one code 171 associated with the doll 131, another code 172 associated with the car 132, and another code 173 associated with the sword 123. In an alternative embodiment, the kit may include a card 144 having a single unlock code 174, the unlock code 174 for unlocking multiple virtual objects associated with a boy, a car, and a sword, respectively. Again, it will be appreciated that instead of providing a card 144, a single unlock code may be provided in a different manner.

Fig. 4 shows an example of the use of the toy system described herein, e.g. using the toy construction kit of any of fig. 1-3 and a suitably programmed portable device 450, such as a tablet or smartphone, which executes an App that implements a digital game of the toy system. As shown in the previous examples, the toy construction set comprises toy construction elements (not explicitly shown) for constructing a doll 131 and a car 132.

Initially, the processing device reads one or more unlock codes included in the toy construction set, for example, from the respective cards 141, 142 described above. This results in the corresponding virtual object 451, 452 (in this example, virtual car 452 and virtual character 451) being unlocked. The user may then capture an image of the toy doll 131 in the driver's seat of the toy vehicle 132. The processing device 450 identifies the toy and car, thereby forming a composite model that elicits a digital game to provide an entertainment experience involving a virtual car 452 driven by a corresponding virtual character 451. After the entertainment experience is complete, the user may capture another image of the same or a different toy construction model and participate in the same or a different gaming experience involving the unlocked corresponding virtual object.

Thus, while the virtual object may only need to be unlocked once, once unlocked, the virtual object may be available for selection multiple times (e.g., a limited or unlimited number of times) as part of the entertainment experience. The selection is performed by capturing an image of the corresponding physical toy construction element or model.

Fig. 5 shows an example of another use of the toy system described herein, for example using the toy construction kit of any of fig. 1-3 and a suitably programmed portable device 450, such as a tablet or smartphone, which executes an App of a digital game implementing the present toy system. The example of fig. 5 is similar to the example of fig. 4. However, while the use in fig. 4 allows repeated selection of virtual objects 451, 452, optionally in different combinations with other objects, the use in fig. 5 allows only a single selection of an unlocked virtual object. Once a combination is selected, the combination so selected may be used in an entertainment experience.

Fig. 6 shows an example of another use of the toy system described herein, for example using the toy construction kit of any of fig. 1-3 and a suitably programmed portable device 450, such as a tablet or smartphone, which executes an App of a digital game implementing the toy system. The example of fig. 6 is similar to the example of fig. 4. In particular, the toy system of fig. 6 comprises toy construction elements from which a plurality of toy construction models 131 and 134 can be constructed. The toy system further includes four cards 141 & 144 with unlocking codes for unlocking the four virtual objects 451 & 454, each corresponding to one of the toy construction models 131 & 134.

In the example of FIG. 6, the user has unlocked four virtual objects 451-454, which may be combined in a digital game in different ways by capturing images of respective composite toy construction models 661-664 constructed from respective combinations of respective identifiable toy models. In particular, composite toy construction model 661 is constructed from doll 131 and car 132, composite toy construction model 662 is constructed from doll 131 and car 133, composite toy construction model 663 is constructed from doll 134 and car 132, and composite toy construction model 664 is constructed from doll 134 and car 133.

Fig. 7 shows a flowchart of an example of a computer-implemented process for controlling a digital game of a toy system, such as any of the toy systems described in conjunction with fig. 1-6. In particular, the process may be performed by a processing device including a digital camera and a display, such as a mobile phone, tablet computer, or other personal computing device.

In an initial step S1, the process begins execution of the digital game, for example by executing a computer program stored on a processing device. The digital game provides the functions of: the method includes obtaining an unlock code, capturing an image of a toy construction model, identifying the toy construction model in the captured image, and providing a digital entertainment experience involving one or more virtual objects.

In step S2, the process obtains the unlock code, such as by reading a QR code, reading an RFID tag, receiving a manually entered code entered by a user, or other suitable means.

In a subsequent step S3, the process unlocks the virtual object associated with the received unlock code. For example, the digital game may have stored information about a plurality of virtual objects, each virtual object having a stored unlocking code or set of unlocking codes associated therewith. The process may thus compare the acquired unlock code to the stored one or more unlock codes to identify which virtual object to unlock. The process may then mark the virtual object as unlocked. In some embodiments, the process may be implemented by a distributed system, such as a distributed system including client devices and remote host systems, such as described in conjunction with FIG. 11. In such a system, the processing device may forward the acquired unlock code to the host system, and the host system may respond with information about the virtual object to be unlocked.

In step S4, the process receives an image of a toy construction model. For example, the image may be an image captured by a digital camera of the device performing the process. The image may be forwarded directly from the camera to the recognition process. To do so, the process may instruct the user to capture an image of a toy construction model constructed by the user, where the toy construction model represents the unlocked virtual object. In some embodiments, the process may initially display or otherwise present construction instructions that guide the user in constructing the predetermined toy construction model. The process may receive a single captured image or multiple images, such as a video stream, for example a real-time video stream currently being captured by a camera.

In step S5, the process processes the received image to attempt to identify a known toy construction model in the received image. For example, the process may feed the captured images to a trained machine learning algorithm, e.g., a trained neural network, which is trained to identify each of a plurality of target toy construction models. An example of a process for identifying toy construction models is described in WO 2016/075081. However, it will be appreciated that other image processing and visual techniques may be used to identify toy construction models in the received image. It will also be appreciated that the identification process may identify the entirety of the toy construction model, or the process may identify individual toy construction elements in the model, for example, one or more marker toy construction elements comprising visual markers indicative of the toy construction model.

If the process fails to identify a known toy construction model, the process may repeat step S5 to receive a new image. Repeated failure to recognize a known toy construction model may cause the process to terminate or proceed in other suitable ways, such as requesting the user to capture another image of another toy construction model.

When the process has identified a known toy construction model in the received image, the process proceeds to step S6, and in step S6, the process determines whether the unlocked virtual object is associated with the identified toy construction model. To this end, the process may compare the identified toy construction model with a list of known toy construction models, each having a respective virtual object associated therewith. Further, each virtual object may have a lock/unlock flag associated therewith. Thus, the process determines that the unlocked virtual object is associated with the identified toy construction model only if the identified toy construction model is associated with the virtual model for which the unlocking flag is set.

When the process determines that the unlocked virtual object is associated with the identified toy construction model, the process proceeds to step S7; otherwise, the process may terminate, notifying the user that the corresponding virtual object needs to be unlocked, or in another suitable manner.

At step S7, the process provides a digital entertainment experience involving virtual objects associated with the identified toy construction model. For example, the process may begin an entertainment experience with the identified virtual object, or the process may add the virtual object to an ongoing entertainment experience.

After the entertainment experience is complete, or in response to a game event or user input, the process may return to step S4 to allow the user to obtain an image of another toy construction model. Alternatively, the process may terminate.

It will be appreciated that various modifications may be made to the above process.

For example, the process may identify portions of the toy construction model and determine whether unlocked virtual objects are associated with one or each of the identified portions and provide an entertainment experience involving a combination of these unlocked virtual objects, e.g., only if all of the identified portions have unlocked virtual objects associated therewith. Examples of such processes are described in connection with fig. 4-6. The identified portions may be individual toy construction elements or toy construction models which are interconnected to form a composite model. Alternatively or additionally, the process may limit use of the unlocked virtual object, e.g., to a single use or a predetermined number of uses, to certain combinations with other objects, etc.

Alternatively or additionally, steps S5 and S6 may be combined into a single operation. For example, if a toy construction model has an unlocked virtual object associated therewith, the process may only identify the toy construction model as a known toy construction model.

Still alternatively or additionally, in step S5, the process may further detect an object code applied to the recognized toy construction model, for example by reading a QR code or other type of visually recognizable code from the captured image.

For example, during the manufacture of the toy construction element, a data processing system executing an encoder may convert a bit string or other object code into a visually recognizable code, such as a QR code, a graphic decoration, or the like. The encoded visually identifiable code may then be printed on a toy construction element, such as on the torso of a doll as shown in fig. 8A-C.

During step S5, the decoding function may analyze the image of the toy construction model and extract the object code embedded by the encoder. The decoding function may be based on a QR code reading function, a neural network trained to convert encoded images into their object code counterparts, and the like. Error correction codes may be added to the object code so that many erroneous output bits can be corrected. In one embodiment, the process may initially identify a toy construction model, identify a portion of the identified toy construction model where the object code is expected, and feed a partial image describing the identified portion (e.g., the torso of the identified doll) to a decoding function.

In step S6, in addition to determining an unlocked virtual object corresponding to the identified toy construction model, the process may identify a particular instance of an unlocked virtual object based on the detected object code. To this end, the process may maintain records associated with multiple instances of a particular virtual object, each instance being associated with a respective object code and, optionally, with a respective attribute (e.g., health, capability, etc.).

Figures 8A-C illustrate examples of toy construction models 131. The toy construction figures 8A-C are toy figures, each being made up of a number of toy construction elements, in particular toy construction elements forming the head, torso and legs of the toy, respectively. It will be appreciated that each doll may alternatively be formed as a single toy construction element. It should also be understood that other toy construction models may represent other items, such as vehicles, buildings, animals. Each doll has applied to it a computer-readable visual code 735 encoding a serial number or another form of identifier that may uniquely or non-uniquely identify the particular doll. In the example of fig. 8A-C, the visual code is printed on the torso of the doll. However, in other examples, the code may be applied to other portions of the model, and may even be encoded by visual markers applied to corresponding portions of the model. Thus, even though the figurines 131 of fig. 8A-C have the same shape, size, and decoration, except for the code 735, they can be distinguished from one another by a computing device having a code reader. Thus, since the teens of fig. 8A-C are very similar in feel to a human observer (in some embodiments, they may even be substantially indistinguishable), the end user will not readily notice the difference between the two teens. Further, embodiments of the processes described herein may identify the teens as representing the same virtual object, in particular the same virtual character. However, a single unlocking code may unlock all instances of the virtual object.

Fig. 9 shows an example of the use of the toy system described herein, e.g. including the doll described in connection with fig. 8A-C and a suitably programmed portable device 450, e.g. a tablet or smartphone, which executes an App that implements the digital game of the toy system. As in the previous examples, the toy building system comprises toy building elements (not explicitly shown) for building the doll 131. The toy building system further comprises a card 141 having an unlock code 171.

Initially, the processing device 450 reads the unlock code 171 included in the toy construction set, for example, from the card 141. This causes the corresponding virtual object 451 (the virtual character in this example) to be unlocked. The user may then capture an image of the doll 131 bearing one of a set of object codes 735. The processing device identifies the teens that include the specific code 735 to apply to the teens. This enables the digital game executed by processing device 450 to provide an entertainment experience involving an instance of virtual character 451. After the entertainment experience is complete, the user may capture another image of the same or a different doll, particularly a doll similar to doll 131 but with a different object code 735 applied. This allows the user to participate in the same or different entertainment experiences involving different instances of the virtual character. Thus, the digital game may store or otherwise maintain game progress (e.g., health level, ability level, or other progress) for various instances of the virtual character. For example, if two users each have their own teens with corresponding object codes, they may both use the processing device 450 to participate in a digital game that uses respective instances of the same virtual character, particularly where the virtual character has respective in-game progress.

Fig. 10 schematically illustrates an example of a toy system described herein. The toy system comprises a plurality of toy construction elements 120 from which one or more toy construction models can be constructed, such as described in connection with fig. 1. The toy system also includes two cards 141, 142, which are made of plastic or cardboard. Each card displays an image or other representation of one of the toy construction models and a machine readable code 171, 172, in this example a QR code, which represents the unlock code of the virtual object associated with the respective toy construction model, e.g. the virtual character and the virtual car, respectively. Alternatively, the unlock code may be provided to the user in a different manner (e.g., by mail or sold separately). Thus, each unlock code is a unique code that is attached to the product or is provided to the user by a physically printed or digital code. The unlock code then unlocks the possibility to use the computer vision selection object in/for the digital experience after use (scanning or typing).

The toy system also includes a suitably programmed processing device 450, such as a tablet computer or smartphone or other portable computing device that executes an App that implements the digital game of the toy system. The processing device 450 includes a central processing unit 455, memory 456, a user interface 457, a code reader 458, and an image capture apparatus 459.

The user interface 457 may include, for example, a display (e.g., a touch screen), and an optional input device (e.g., a button, touchpad, pointing device, etc.).

The image capture device 459 may comprise a digital camera, a depth camera, a stereo camera, and the like.

The code reader 458 may be a bar code reader, an RFID reader, or the like. In some embodiments, the code reader may comprise a digital camera. In some embodiments, the code reader and the image capture device may be a single device. For example, the same digital camera may be used to read the unlock code and capture an image of the toy construction model.

Fig. 11 schematically illustrates another example of a toy system described herein. The toy system of fig. 11 is similar to the toy system of fig. 10, with the only difference that processing device 450 also includes a communication interface 460, such as a wireless or wired communication interface that allows processing device 450 to communicate with a remote system 1170. The communication may be wired or wireless. The communication may be via a communication network. The remote system may be a server computer or other suitable data processing system that may be configured to perform one or more of the process steps described herein. For example, the remote system may maintain a database of unlocking codes to determine whether a given unlocking code has been previously used to unlock a virtual object. Alternatively or additionally, the remote system may maintain a database of object code. Still alternatively or additionally, the remote system may implement an object recognition process, or a portion thereof, to identify the toy construction model in the captured image. Alternatively or additionally, the remote system may implement at least a portion of a digital game, such as in an embodiment where the digital game includes a multiplayer entertainment experience or a network entertainment experience.

Thus, typically, a virtual object only needs to be unlocked once by a unique unlocking code. The virtual object may be selected (or multiple/composite virtual objects selected at a time) each time the virtual object is used in a digital experience or used once.

While the present invention has been described with reference to certain particular embodiments, various modifications thereof will be apparent to those skilled in the art without departing from the spirit and scope of the invention as outlined in the claims appended hereto.