Patent
US 11,928,185 B2INTERPRETABILITY ANALYSIS OF IMAGE GENERATED BY GENERATIVE ADVERSERIAL NETWORK (GAN) MODEL
Ramya Malur Srinivasan, Kanji Uchino
FUJITSU LIMITED, Kawasaki (JP)·Mar. 12, 2024·US
Copyright © 2026 Matter42. All rights reserved.
Patent
Atlas literature
Patent
US 11,928,185 B2Patent drawings and their descriptions. Click a drawing to enlarge it.
FIG. 1. With reference to
FIG. 2. Although illustrated with discrete blocks, the steps and operations associated with one or more of the blocks of the flowchart 400 may be divided into …
FIG. 3. At block 1006, a first sequence associated with the first typicality score may be determined by the plurality of trained GAN models (for example, the …
FIG. 4. With reference to
FIG. 5. At block 314, a first typicality score between a first set of images from the received image dataset 114 and the first image may be determined, based on …
FIG. 6. At block 404, a set of feature maps of the first image may be determined, based on the application of the neural net- work model 110 on the first image. …
FIG. 7. With reference to
FIG. 8. Control may pass to end. Although the flowchart 700 is illustrated as discrete operations, such as 702, 704, and 706. However, in certain embodiments, …
FIG. 9. In an embodiment, the processor 204 may be configured to determine second interpretability coefficient associated with the trained GAN model 108 based …
FIG. 10. In an embodiment, the processor 204 may identify attributes of the image identified as the prototype of the first class based on at least one of, but …
FIG. 11B. With reference to
FIG. 12B, there is shown an example prototype image 1200B. The first image 1200A may belong to the set of images generated by the GAN model 108. Further, the …
FIG. 40 1, and
FIG. 60 6,
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method, executed by a processor, comprising: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient associ-ated with the trained GAN model, based on the deter-mined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined second interpretability coefficient; and rendering the determined interpretability score on a dis-play device.
The method according to claim 1, wherein the deter-mination of the first saliency map associated with the first image from the generated set of images further comprises: applying the neural network model on the first image; determining a set of feature maps of the first image based on the application of the neural network model on the first image; enhancing, based on an Attentive model, a set of salient features from the determined set of feature maps; and determining the first saliency map based on a combination of the enhanced set of salient features and a set of learned priors associated with saliency maps.
The method according to claim 1, wherein the second image from the received image dataset corresponds to an average image associated with the first class, and wherein the average image of the first class corresponds to an image whose feature vector corresponds to an average feature vector of each image of the first class, from the received image dataset.
The method according to claim 1, further comprising: determining a region of overlap between the determined first saliency map and the determined second saliency map; and determining the first interpretability coefficient associated with the trained GAN model, based on the determined region of overlap and the determined second saliency map.
The method according to claim 1, further comprising: determining a first set of distributions of feature vectors of the first set of images from the received image dataset; determining, by the trained GAN model, a first set of sample sequences associated with the first image from the generated set of images; and determining the first typicality score based on the deter-mined first set of sample sequences and the determined first set of distributions.
The method according to claim 1, further comprising: selecting the pair of images from the generated set of images; determining a second set of distributions of feature vec-tors of a third image from the selected pair of images; determining, by the trained GAN model, a second set of sample sequences associated with a fourth image from the selected pair of images; and determining the second typicality score based on the determined second set of sample sequences and the determined second set of distributions.
The method according to claim 1, further comprising: identifying content prototype attributes for the first class based on the determined first typicality score and the determined second typicality score; and rendering the identified content prototype attributes for the first class.
The method according to claim 1, further comprising: selecting a plurality of groups of images from the received image dataset; generating a plurality of trained GAN models, based on training of the GAN model using a respective group of images from the selected plurality of groups of images; and determining, by the plurality of trained GAN models, a first sequence associated with the first typicality score which is associated with the first image from the generated set of images associated with the first class.
One or more non-transitory computer-readable stor-age media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations comprising: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient associ-ated with the trained GAN model, based on the deter-mined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined second interpretability coefficient; and rendering the determined interpretability score on a dis-play device.
The one or more non-transitory computer-readable storage media according to claim 17, wherein the operations further comprise: determining a region of overlap between the determined first saliency map and the determined second saliency map; and determining the first interpretability coefficient associated with the trained GAN model, based on the determined region of overlap and the determined second saliency map.
The one or more non-transitory computer-readable storage media according to claim 17, wherein the operations further comprise: determining a first set of distributions of feature vectors of the first set of images from the received image dataset; determining, by the trained GAN model, a first set of sample sequences associated with the first image from the generated set of images; determining the first typicality score based on the deter-mined first set of sample sequences and the determined first set of distributions; selecting the pair of images from the generated set of images; determining a second set of distributions of feature vec-tors of a third image from the selected pair of images; determining, by the trained GAN model, a second set of sample sequences associated with a fourth image from the selected pair of images; and determining the second typicality score based on the determined second set of sample sequences and the determined second set of distributions.
An electronic device, comprising: a memory configured to store instructions; and a processor, coupled to the memory, that is configured to execute the instructions to perform a process compris-ing: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associ-ated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient asso-ciated with the trained GAN model, based on the determined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined sec-ond interpretability coefficient; and rendering the determined interpretability score on a display device. ∗ ∗ ∗ ∗ ∗
Measurements and analyses referenced in the patent, with their drawing references.
FIG. 2. Although illustrated with discrete blocks, the steps and operations associated with one or more of the blocks of the flowchart 400 may be divided into …
Patents and literature cited by this patent (applicant and examiner references).
Cited patents · 4
Related documents with shared materials, methods, properties, or citations.
SYSTEMS AND METHODS FOR A RADIO FREQUENCY TRANSMITTER WITH IMPROVED LINEARITY AND POWER OUT UTILIZING PRE-DISTORTION AND A GAN (GALLIUM NITRIDE) POWER AMPLIFIER DEVICE
ANSWER CLASSIFIER AND REPRESENTATION GENERATOR FOR QUESTION-ANSWERING SYSTEM USING GAN, AND COMPUTER PROGRAM FOR TRAINING THE REPRESENTATION GENERATOR
Patent
Atlas literature
Patent
US 11,928,185 B2Patent drawings and their descriptions. Click a drawing to enlarge it.
FIG. 1. With reference to
FIG. 2. Although illustrated with discrete blocks, the steps and operations associated with one or more of the blocks of the flowchart 400 may be divided into …
FIG. 3. At block 1006, a first sequence associated with the first typicality score may be determined by the plurality of trained GAN models (for example, the …
FIG. 4. With reference to
FIG. 5. At block 314, a first typicality score between a first set of images from the received image dataset 114 and the first image may be determined, based on …
FIG. 6. At block 404, a set of feature maps of the first image may be determined, based on the application of the neural net- work model 110 on the first image. …
FIG. 7. With reference to
FIG. 8. Control may pass to end. Although the flowchart 700 is illustrated as discrete operations, such as 702, 704, and 706. However, in certain embodiments, …
FIG. 9. In an embodiment, the processor 204 may be configured to determine second interpretability coefficient associated with the trained GAN model 108 based …
FIG. 10. In an embodiment, the processor 204 may identify attributes of the image identified as the prototype of the first class based on at least one of, but …
FIG. 11B. With reference to
FIG. 12B, there is shown an example prototype image 1200B. The first image 1200A may belong to the set of images generated by the GAN model 108. Further, the …
FIG. 40 1, and
FIG. 60 6,
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method, executed by a processor, comprising: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient associ-ated with the trained GAN model, based on the deter-mined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined second interpretability coefficient; and rendering the determined interpretability score on a dis-play device.
The method according to claim 1, wherein the deter-mination of the first saliency map associated with the first image from the generated set of images further comprises: applying the neural network model on the first image; determining a set of feature maps of the first image based on the application of the neural network model on the first image; enhancing, based on an Attentive model, a set of salient features from the determined set of feature maps; and determining the first saliency map based on a combination of the enhanced set of salient features and a set of learned priors associated with saliency maps.
The method according to claim 1, wherein the second image from the received image dataset corresponds to an average image associated with the first class, and wherein the average image of the first class corresponds to an image whose feature vector corresponds to an average feature vector of each image of the first class, from the received image dataset.
The method according to claim 1, further comprising: determining a region of overlap between the determined first saliency map and the determined second saliency map; and determining the first interpretability coefficient associated with the trained GAN model, based on the determined region of overlap and the determined second saliency map.
The method according to claim 1, further comprising: determining a first set of distributions of feature vectors of the first set of images from the received image dataset; determining, by the trained GAN model, a first set of sample sequences associated with the first image from the generated set of images; and determining the first typicality score based on the deter-mined first set of sample sequences and the determined first set of distributions.
The method according to claim 1, further comprising: selecting the pair of images from the generated set of images; determining a second set of distributions of feature vec-tors of a third image from the selected pair of images; determining, by the trained GAN model, a second set of sample sequences associated with a fourth image from the selected pair of images; and determining the second typicality score based on the determined second set of sample sequences and the determined second set of distributions.
The method according to claim 1, further comprising: identifying content prototype attributes for the first class based on the determined first typicality score and the determined second typicality score; and rendering the identified content prototype attributes for the first class.
The method according to claim 1, further comprising: selecting a plurality of groups of images from the received image dataset; generating a plurality of trained GAN models, based on training of the GAN model using a respective group of images from the selected plurality of groups of images; and determining, by the plurality of trained GAN models, a first sequence associated with the first typicality score which is associated with the first image from the generated set of images associated with the first class.
One or more non-transitory computer-readable stor-age media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations comprising: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient associ-ated with the trained GAN model, based on the deter-mined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined second interpretability coefficient; and rendering the determined interpretability score on a dis-play device.
The one or more non-transitory computer-readable storage media according to claim 17, wherein the operations further comprise: determining a region of overlap between the determined first saliency map and the determined second saliency map; and determining the first interpretability coefficient associated with the trained GAN model, based on the determined region of overlap and the determined second saliency map.
The one or more non-transitory computer-readable storage media according to claim 17, wherein the operations further comprise: determining a first set of distributions of feature vectors of the first set of images from the received image dataset; determining, by the trained GAN model, a first set of sample sequences associated with the first image from the generated set of images; determining the first typicality score based on the deter-mined first set of sample sequences and the determined first set of distributions; selecting the pair of images from the generated set of images; determining a second set of distributions of feature vec-tors of a third image from the selected pair of images; determining, by the trained GAN model, a second set of sample sequences associated with a fourth image from the selected pair of images; and determining the second typicality score based on the determined second set of sample sequences and the determined second set of distributions.
An electronic device, comprising: a memory configured to store instructions; and a processor, coupled to the memory, that is configured to execute the instructions to perform a process compris-ing: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associ-ated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient asso-ciated with the trained GAN model, based on the determined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined sec-ond interpretability coefficient; and rendering the determined interpretability score on a display device. ∗ ∗ ∗ ∗ ∗
Measurements and analyses referenced in the patent, with their drawing references.
FIG. 2. Although illustrated with discrete blocks, the steps and operations associated with one or more of the blocks of the flowchart 400 may be divided into …
Patents and literature cited by this patent (applicant and examiner references).
Cited patents · 4
Related documents with shared materials, methods, properties, or citations.
SYSTEMS AND METHODS FOR A RADIO FREQUENCY TRANSMITTER WITH IMPROVED LINEARITY AND POWER OUT UTILIZING PRE-DISTORTION AND A GAN (GALLIUM NITRIDE) POWER AMPLIFIER DEVICE
ANSWER CLASSIFIER AND REPRESENTATION GENERATOR FOR QUESTION-ANSWERING SYSTEM USING GAN, AND COMPUTER PROGRAM FOR TRAINING THE REPRESENTATION GENERATOR
Patent
Atlas literature
Patent
US 11,928,185 B2Patent drawings and their descriptions. Click a drawing to enlarge it.
FIG. 1. With reference to
FIG. 2. Although illustrated with discrete blocks, the steps and operations associated with one or more of the blocks of the flowchart 400 may be divided into …
FIG. 3. At block 1006, a first sequence associated with the first typicality score may be determined by the plurality of trained GAN models (for example, the …
FIG. 4. With reference to
FIG. 5. At block 314, a first typicality score between a first set of images from the received image dataset 114 and the first image may be determined, based on …
FIG. 6. At block 404, a set of feature maps of the first image may be determined, based on the application of the neural net- work model 110 on the first image. …
FIG. 7. With reference to
FIG. 8. Control may pass to end. Although the flowchart 700 is illustrated as discrete operations, such as 702, 704, and 706. However, in certain embodiments, …
FIG. 9. In an embodiment, the processor 204 may be configured to determine second interpretability coefficient associated with the trained GAN model 108 based …
FIG. 10. In an embodiment, the processor 204 may identify attributes of the image identified as the prototype of the first class based on at least one of, but …
FIG. 11B. With reference to
FIG. 12B, there is shown an example prototype image 1200B. The first image 1200A may belong to the set of images generated by the GAN model 108. Further, the …
FIG. 40 1, and
FIG. 60 6,
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method, executed by a processor, comprising: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient associ-ated with the trained GAN model, based on the deter-mined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined second interpretability coefficient; and rendering the determined interpretability score on a dis-play device.
The method according to claim 1, wherein the deter-mination of the first saliency map associated with the first image from the generated set of images further comprises: applying the neural network model on the first image; determining a set of feature maps of the first image based on the application of the neural network model on the first image; enhancing, based on an Attentive model, a set of salient features from the determined set of feature maps; and determining the first saliency map based on a combination of the enhanced set of salient features and a set of learned priors associated with saliency maps.
The method according to claim 1, wherein the second image from the received image dataset corresponds to an average image associated with the first class, and wherein the average image of the first class corresponds to an image whose feature vector corresponds to an average feature vector of each image of the first class, from the received image dataset.
The method according to claim 1, further comprising: determining a region of overlap between the determined first saliency map and the determined second saliency map; and determining the first interpretability coefficient associated with the trained GAN model, based on the determined region of overlap and the determined second saliency map.
The method according to claim 1, further comprising: determining a first set of distributions of feature vectors of the first set of images from the received image dataset; determining, by the trained GAN model, a first set of sample sequences associated with the first image from the generated set of images; and determining the first typicality score based on the deter-mined first set of sample sequences and the determined first set of distributions.
The method according to claim 1, further comprising: selecting the pair of images from the generated set of images; determining a second set of distributions of feature vec-tors of a third image from the selected pair of images; determining, by the trained GAN model, a second set of sample sequences associated with a fourth image from the selected pair of images; and determining the second typicality score based on the determined second set of sample sequences and the determined second set of distributions.
The method according to claim 1, further comprising: identifying content prototype attributes for the first class based on the determined first typicality score and the determined second typicality score; and rendering the identified content prototype attributes for the first class.
The method according to claim 1, further comprising: selecting a plurality of groups of images from the received image dataset; generating a plurality of trained GAN models, based on training of the GAN model using a respective group of images from the selected plurality of groups of images; and determining, by the plurality of trained GAN models, a first sequence associated with the first typicality score which is associated with the first image from the generated set of images associated with the first class.
One or more non-transitory computer-readable stor-age media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations comprising: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient associ-ated with the trained GAN model, based on the deter-mined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined second interpretability coefficient; and rendering the determined interpretability score on a dis-play device.
The one or more non-transitory computer-readable storage media according to claim 17, wherein the operations further comprise: determining a region of overlap between the determined first saliency map and the determined second saliency map; and determining the first interpretability coefficient associated with the trained GAN model, based on the determined region of overlap and the determined second saliency map.
The one or more non-transitory computer-readable storage media according to claim 17, wherein the operations further comprise: determining a first set of distributions of feature vectors of the first set of images from the received image dataset; determining, by the trained GAN model, a first set of sample sequences associated with the first image from the generated set of images; determining the first typicality score based on the deter-mined first set of sample sequences and the determined first set of distributions; selecting the pair of images from the generated set of images; determining a second set of distributions of feature vec-tors of a third image from the selected pair of images; determining, by the trained GAN model, a second set of sample sequences associated with a fourth image from the selected pair of images; and determining the second typicality score based on the determined second set of sample sequences and the determined second set of distributions.
An electronic device, comprising: a memory configured to store instructions; and a processor, coupled to the memory, that is configured to execute the instructions to perform a process compris-ing: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associ-ated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient asso-ciated with the trained GAN model, based on the determined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined sec-ond interpretability coefficient; and rendering the determined interpretability score on a display device. ∗ ∗ ∗ ∗ ∗
Measurements and analyses referenced in the patent, with their drawing references.
FIG. 2. Although illustrated with discrete blocks, the steps and operations associated with one or more of the blocks of the flowchart 400 may be divided into …
Patents and literature cited by this patent (applicant and examiner references).
Cited patents · 4
Related documents with shared materials, methods, properties, or citations.
SYSTEMS AND METHODS FOR A RADIO FREQUENCY TRANSMITTER WITH IMPROVED LINEARITY AND POWER OUT UTILIZING PRE-DISTORTION AND A GAN (GALLIUM NITRIDE) POWER AMPLIFIER DEVICE
ANSWER CLASSIFIER AND REPRESENTATION GENERATOR FOR QUESTION-ANSWERING SYSTEM USING GAN, AND COMPUTER PROGRAM FOR TRAINING THE REPRESENTATION GENERATOR
Patent
Atlas literature
Patent
US 11,928,185 B2Patent drawings and their descriptions. Click a drawing to enlarge it.
FIG. 1. With reference to
FIG. 2. Although illustrated with discrete blocks, the steps and operations associated with one or more of the blocks of the flowchart 400 may be divided into …
FIG. 3. At block 1006, a first sequence associated with the first typicality score may be determined by the plurality of trained GAN models (for example, the …
FIG. 4. With reference to
FIG. 5. At block 314, a first typicality score between a first set of images from the received image dataset 114 and the first image may be determined, based on …
FIG. 6. At block 404, a set of feature maps of the first image may be determined, based on the application of the neural net- work model 110 on the first image. …
FIG. 7. With reference to
FIG. 8. Control may pass to end. Although the flowchart 700 is illustrated as discrete operations, such as 702, 704, and 706. However, in certain embodiments, …
FIG. 9. In an embodiment, the processor 204 may be configured to determine second interpretability coefficient associated with the trained GAN model 108 based …
FIG. 10. In an embodiment, the processor 204 may identify attributes of the image identified as the prototype of the first class based on at least one of, but …
FIG. 11B. With reference to
FIG. 12B, there is shown an example prototype image 1200B. The first image 1200A may belong to the set of images generated by the GAN model 108. Further, the …
FIG. 40 1, and
FIG. 60 6,
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method, executed by a processor, comprising: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient associ-ated with the trained GAN model, based on the deter-mined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined second interpretability coefficient; and rendering the determined interpretability score on a dis-play device.
The method according to claim 1, wherein the deter-mination of the first saliency map associated with the first image from the generated set of images further comprises: applying the neural network model on the first image; determining a set of feature maps of the first image based on the application of the neural network model on the first image; enhancing, based on an Attentive model, a set of salient features from the determined set of feature maps; and determining the first saliency map based on a combination of the enhanced set of salient features and a set of learned priors associated with saliency maps.
The method according to claim 1, wherein the second image from the received image dataset corresponds to an average image associated with the first class, and wherein the average image of the first class corresponds to an image whose feature vector corresponds to an average feature vector of each image of the first class, from the received image dataset.
The method according to claim 1, further comprising: determining a region of overlap between the determined first saliency map and the determined second saliency map; and determining the first interpretability coefficient associated with the trained GAN model, based on the determined region of overlap and the determined second saliency map.
The method according to claim 1, further comprising: determining a first set of distributions of feature vectors of the first set of images from the received image dataset; determining, by the trained GAN model, a first set of sample sequences associated with the first image from the generated set of images; and determining the first typicality score based on the deter-mined first set of sample sequences and the determined first set of distributions.
The method according to claim 1, further comprising: selecting the pair of images from the generated set of images; determining a second set of distributions of feature vec-tors of a third image from the selected pair of images; determining, by the trained GAN model, a second set of sample sequences associated with a fourth image from the selected pair of images; and determining the second typicality score based on the determined second set of sample sequences and the determined second set of distributions.
The method according to claim 1, further comprising: identifying content prototype attributes for the first class based on the determined first typicality score and the determined second typicality score; and rendering the identified content prototype attributes for the first class.
The method according to claim 1, further comprising: selecting a plurality of groups of images from the received image dataset; generating a plurality of trained GAN models, based on training of the GAN model using a respective group of images from the selected plurality of groups of images; and determining, by the plurality of trained GAN models, a first sequence associated with the first typicality score which is associated with the first image from the generated set of images associated with the first class.
One or more non-transitory computer-readable stor-age media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations comprising: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient associ-ated with the trained GAN model, based on the deter-mined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined second interpretability coefficient; and rendering the determined interpretability score on a dis-play device.
The one or more non-transitory computer-readable storage media according to claim 17, wherein the operations further comprise: determining a region of overlap between the determined first saliency map and the determined second saliency map; and determining the first interpretability coefficient associated with the trained GAN model, based on the determined region of overlap and the determined second saliency map.
The one or more non-transitory computer-readable storage media according to claim 17, wherein the operations further comprise: determining a first set of distributions of feature vectors of the first set of images from the received image dataset; determining, by the trained GAN model, a first set of sample sequences associated with the first image from the generated set of images; determining the first typicality score based on the deter-mined first set of sample sequences and the determined first set of distributions; selecting the pair of images from the generated set of images; determining a second set of distributions of feature vec-tors of a third image from the selected pair of images; determining, by the trained GAN model, a second set of sample sequences associated with a fourth image from the selected pair of images; and determining the second typicality score based on the determined second set of sample sequences and the determined second set of distributions.
An electronic device, comprising: a memory configured to store instructions; and a processor, coupled to the memory, that is configured to execute the instructions to perform a process compris-ing: receiving an image dataset; training a Generative Adversarial Network (GAN) model based on the received image dataset; generating, by the GAN model, a set of images each associated with a first class from a set of classes associated with the received image dataset; determining, by a neural network model different from the GAN model, a first saliency map associated with a first image from the generated set of images; determining, by the neural network model, a second saliency map associated with a second image from the received image dataset, the second image is associated with the first class; determining a first interpretability coefficient associ-ated with the trained GAN model, based on the determined first saliency map and the determined second saliency map; determining a first typicality score between a first set of images from the received image dataset and the first image, based on the trained GAN model, each of the first set of images is associated with the first class; determining a second typicality score between a pair of images from the generated set of images, based on the trained GAN model; determining a second interpretability coefficient asso-ciated with the trained GAN model, based on the determined first typicality score and the determined second typicality score; determining an interpretability score associated with the trained GAN model based on the determined first interpretability coefficient and the determined sec-ond interpretability coefficient; and rendering the determined interpretability score on a display device. ∗ ∗ ∗ ∗ ∗
Measurements and analyses referenced in the patent, with their drawing references.
FIG. 2. Although illustrated with discrete blocks, the steps and operations associated with one or more of the blocks of the flowchart 400 may be divided into …
Patents and literature cited by this patent (applicant and examiner references).
Cited patents · 4
Related documents with shared materials, methods, properties, or citations.
SYSTEMS AND METHODS FOR A RADIO FREQUENCY TRANSMITTER WITH IMPROVED LINEARITY AND POWER OUT UTILIZING PRE-DISTORTION AND A GAN (GALLIUM NITRIDE) POWER AMPLIFIER DEVICE
ANSWER CLASSIFIER AND REPRESENTATION GENERATOR FOR QUESTION-ANSWERING SYSTEM USING GAN, AND COMPUTER PROGRAM FOR TRAINING THE REPRESENTATION GENERATOR
Cited non-patent literature · 18
Cited non-patent literature · 18
Cited non-patent literature · 18
Cited non-patent literature · 18