Patent
Atlas literature
Patent
US 10,395,392Patent drawings and their descriptions. Click a drawing to enlarge it.
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method for learning transformation of at least one annotated RGB image into at least one annotated Non-RGB image using a cycle G AN (Generative Adversarial Network), comprising steps of: (a) a learning device, if at least one first image in an RGB format is acquired, instructing a first transformer to transform the first image to at least one second image in a non-R G B format, instructing a first discriminator to determine whether the second image has a primary non-RGB format or a secondary non-RGB format, wherein the primary non-R G B format is the non-RGB format without a transformation from the RGB format and the secondary non-R G B format is the non-RGB format with the transformation from the R G B format, to thereby generate a 1 1-st result, and instructing a second transformer to transform the second image to at least one third image in the RGB format; (b) the learning device, if at least one fourth image in the non-R G B format is acquired, instructing the second transformer to transform the fourth image to at least one fifth image in the R G B format, instructing a second discriminator to determine whether the fifth image has a primary R G B format or a secondary RGB format, wherein the primary RGB format is the R G B format without a transformation from the non-R G B format and the secondary RGB format is the RGB format with the transformation from the non-RGB format, to thereby generate a 2 _ 1-st result, and instructing the first transformer to transform the fifth image to at least one sixth image in the non-R G B format; and (c) the learning device calculating one or more losses by referring to at least part of the first image, the second image, the third image, the fourth image, the fifth image, the sixth image, the 1 _ 1-st result, and the 2 _ 1-st result, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.1.svg 0.64 6.78 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 1-st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2_ 1 -st result, G(F(X)) is the sixth image, y and ft are cons tant s for adjusting each of weights of each of II- F(G(I)) I and IX- G(F(X)) I.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.2.svg 0.67 6.79 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G(G (I))) is the 1_ 1 -st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 1-st result, G(F(X)) is the sixth image, y and ft are constants for adjusting each of weights of each of II- F(G(I)) I and IX- G(F(X)) I, OD is an object detection loss, A is a constant for adjusting a weight of the object detection loss, and wherein the learning device instructs an RGB object detector, which has been learned, to detect one or more objects in the third image, and compare at leas t part of information on estimated locations, sizes, and classes of the objects detected in the third image and at least part of information on true locations, sizes, and classes of the objects in at least one G T corresponding to the first image, to thereby calculate the object detection loss.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.3.svg 0.3 6.05 Black and white an FD loss for the first discriminator included in the losses is defined by a formula above, NR is any arbitrary image in the non-RGB format, D G (NR) is a 1-2-nd result of determining the arbitrary image in the non-RGB format from the first discriminator, G(I) is the second image, and D G (G(I)) is the 1 _ 1-st result.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.4.svg 0.3 6 Black and white an SD loss for the second discriminator included in the losses is defined by a formula above, R is any arbitrary image in the R G B format, D F (R) is a 2 2-nd result of determining the arbitrary image in the RGB format from the second discriminator, F(X) is the fifth image, and D F (F(X)) is the 2 1- st result.
The method of Claim 1, wherein each of the first transformer and the second transformer includes at least part of an encoding layer and a decoding layer.
A method for testing transformation of at least one annotated RGB image into at leas t one annotated Non-R G B image using a cycle G AN (Generative Adversarial Network), comprising steps of: (a) a testing device, on condition that (1) a learning device has performed processes of instructing a first transformer to transform at least one acquired first training image in an R G B format to at least one second training image in a non-R G B format, instructing a first discriminator to determine whether the second training image has a primary non- RG B format or a secondary non-RGB format, wherein the primary non-RGB format is the non-R G B format without a transformation from the RGB format and the secondary non-R G B format is the nonRGB format with the transformation from the R G B format, to thereby generate a 1 1-st result for training, and instructing a second transformer to transform the second training image to at least one third training image in the RGB format, (2) the learning device has performed processes of instructing the second transformer to transform at least one acquired fourth training image in the non-R G B format to at least one fifth training image in the R G B format, instructing a second discriminator to determine whether the fifth training image has a primary R G B format or a secondary R G B format, wherein the primary R G B format is the R G B format without a transformation from the non-RGB forma t and the secondary RGB format is the RGB format with the transformation from the non- RG B format, to thereby generate a 2 1-st result for training, and instructing the first transformer to transform the fifth training image to at least one sixth training image in the non-R G B format, and (3) the learning device has performed processes of calculating one or more losses by referring to at least part of the first training image, the second training image, the third training image, the fourth training image, the fifth training image, the sixth training image, the 1 _ 1-st result for training, and the 2 _ 1-st result for training, to thereby learn at least part of parameters of the first transformer, the second t ransformer, the first discriminator, and the second discriminator; acquiring at least one test image in the RGB format; and (b) the testing device instructing the first transformer to transform the test image into at least one resulting image in the non-RGB format.
The method of Claim 7, wherein the resulting image is used for learning a non-R G B object detector to detect one or more objects in the test image in the non-RGB format.
A learning device for learning transformation of at least one annotated R G B image into at least one annotated Non-RGB image using a cycle GAN (Generative Adversarial Network), comprising: at least one memory that stores instructions; and at least one processor configured to execu t e the instructions to: perform processes of (I) instructing a first transformer to transform at leas t one first image in an RGB format to at least one second image in a non-RGB format, instructing a first discriminator to determine whether the second image has a primary non-R G B format or a secondary non- RGB format, wherein the primary non-R G B format is the non-R GB forma t without a transformation from the R G B format and the secondary non-RGB format is the non-R G B format with the transformation from the R G B format, to thereby generate a 1 1- st result, and instructing a second transformer to transform the second image to at least one third image in the R G B format, (II) instructing the second transformer to transform at least one fourth image in the non-R G B format to at least one fifth i mage in the R G B format, instructing a second discriminator to determine whether the fifth image has a primary R G B format or a secondary R G B format, wherein the primary R G B format is the RG B format without a transformation from the non-R G B format and the secondary RGB format is the RGB format with the transformation from the non-RGB format, to thereby generate a 2 1-st result, and instructing the first transformer to transform the fifth image to at least one sixth image in the non-RGB format, and (III) calculating one or more losses by referring to at least part of the first image, the second image, the third image, the fourth image, the fifth image, the sixth image, the 1 _ 1-st result, and the 2 _ 1-st result, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.5.svg 0.64 6.78 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 _ 1-st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 _ 1-st result, G(F(X)) is the sixth image, y and fl are constants for adjusting each of weights of each of II-F(G(I)) I and IX-G(F(X)).
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.6.svg 0.68 6.79 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 1 -st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 1 -st result, G(F(X)) is the sixth image, y and fl are constants for adjusting each of weights of each of I-F(G (I)) and IX-G(F(X)), OD is an object detection loss, X is a constant for adjusting a weight of the object detection loss, and wherein the processor instructs an RGB object detector, which has been learned, to detect one or more objects in the third image, and compare at least part of information on estimated locations, sizes, and classes of the objects detected in the third image and at least part of information on true locations, sizes, and classes of the objects in at least one GT corresponding to the first image, to thereby calculate the object detection loss.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.7.svg 0.3 6.05 Black and white an FD loss for the first discriminator included in the losses is defined by a formula above, NR is any arbitrary image in the non-RGB format, D G (NR) is a 1 2-nd result of determining the arbitrary image in the non-RGB format from the first discriminator, G(I) is the second image, and D G (G(I)) is the 1 1-st result.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.8.svg 0.3 6 Black and white an SD loss for the second discriminator included in the losses is defined by a formula above, R is any arbitrary image in the R G B format, D F (R) is a 2 2-nd result of determining the arbitrary image in the RGB format from the second discriminator, F(X) is the fifth image, and D F (F(X)) is the 2 1- st result.
The learning device of Claim 9, wherein each of the first transformer and the second transformer includes at least part of an encoding layer and a decoding layer.
A testing device for testing transformation of at least one annotated R G B image into at least one annotated Non-R GB image using a cycle G AN (Generative Adversarial Network), comprising: at least one memory that stores instructions; and at least one processor, on condition tha t a learning device (1) has performed processes of instructing a first transformer to transform at least one acquired first training image in an R G B format to at least one second training image in a non-RGB format, instructing a first discriminator to determine whether the second training image has a primary non- RGB format or a secondary non-RGB format, wherein the primary nonRGB format is the non-RGB format without a transformation from the R G B format and the secondary non-RGB format is the non-RGB format with the transformation from the RGB format, to thereby generate a 11 -st result for training, and instructing a second transformer to transform the second training image to at least one third training image in the RGB format, (2) has performed processes of instructing the second transformer to transform at least one acquired fourth training image in the non-R G B format to at leas t one fifth training image in the R GB format, instructing a second discriminator to determine whether the fifth training image has a primary R G B format or a secondary R G B format, wherein the primary RGB format is the RGB format without a transformation from the non-R G B format and the secondary R G B format is the R G B format with the transformation from the non-R G B format, to thereby generate a 2 1- st result for training, and instructing the first transformer to transform the fifth training image to at least one sixth training image in the non-R G B format, and (3) has performed processes of calculating one or more losses by referring to at least part of the first training image, the second training image, the third training image, the fourth training image, the fifth training image, the sixth training image, the 1 _ 1-st result for training, and the 2 1-st result for training, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator; configured to execute the instructions to: perform a process of instructing the first transformer to transform at least one test image in the R G B format into at least one resulting image in the non- RG B format.
The testing device of Claim 15, wherein the resulting image is used for learning non-R G B object detector to detect one or more objects in the test image in the non-R G B format.
Patent
Atlas literature
Patent
US 10,395,392Patent drawings and their descriptions. Click a drawing to enlarge it.
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method for learning transformation of at least one annotated RGB image into at least one annotated Non-RGB image using a cycle G AN (Generative Adversarial Network), comprising steps of: (a) a learning device, if at least one first image in an RGB format is acquired, instructing a first transformer to transform the first image to at least one second image in a non-R G B format, instructing a first discriminator to determine whether the second image has a primary non-RGB format or a secondary non-RGB format, wherein the primary non-R G B format is the non-RGB format without a transformation from the RGB format and the secondary non-R G B format is the non-RGB format with the transformation from the R G B format, to thereby generate a 1 1-st result, and instructing a second transformer to transform the second image to at least one third image in the RGB format; (b) the learning device, if at least one fourth image in the non-R G B format is acquired, instructing the second transformer to transform the fourth image to at least one fifth image in the R G B format, instructing a second discriminator to determine whether the fifth image has a primary R G B format or a secondary RGB format, wherein the primary RGB format is the R G B format without a transformation from the non-R G B format and the secondary RGB format is the RGB format with the transformation from the non-RGB format, to thereby generate a 2 _ 1-st result, and instructing the first transformer to transform the fifth image to at least one sixth image in the non-R G B format; and (c) the learning device calculating one or more losses by referring to at least part of the first image, the second image, the third image, the fourth image, the fifth image, the sixth image, the 1 _ 1-st result, and the 2 _ 1-st result, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.1.svg 0.64 6.78 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 1-st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2_ 1 -st result, G(F(X)) is the sixth image, y and ft are cons tant s for adjusting each of weights of each of II- F(G(I)) I and IX- G(F(X)) I.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.2.svg 0.67 6.79 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G(G (I))) is the 1_ 1 -st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 1-st result, G(F(X)) is the sixth image, y and ft are constants for adjusting each of weights of each of II- F(G(I)) I and IX- G(F(X)) I, OD is an object detection loss, A is a constant for adjusting a weight of the object detection loss, and wherein the learning device instructs an RGB object detector, which has been learned, to detect one or more objects in the third image, and compare at leas t part of information on estimated locations, sizes, and classes of the objects detected in the third image and at least part of information on true locations, sizes, and classes of the objects in at least one G T corresponding to the first image, to thereby calculate the object detection loss.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.3.svg 0.3 6.05 Black and white an FD loss for the first discriminator included in the losses is defined by a formula above, NR is any arbitrary image in the non-RGB format, D G (NR) is a 1-2-nd result of determining the arbitrary image in the non-RGB format from the first discriminator, G(I) is the second image, and D G (G(I)) is the 1 _ 1-st result.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.4.svg 0.3 6 Black and white an SD loss for the second discriminator included in the losses is defined by a formula above, R is any arbitrary image in the R G B format, D F (R) is a 2 2-nd result of determining the arbitrary image in the RGB format from the second discriminator, F(X) is the fifth image, and D F (F(X)) is the 2 1- st result.
The method of Claim 1, wherein each of the first transformer and the second transformer includes at least part of an encoding layer and a decoding layer.
A method for testing transformation of at least one annotated RGB image into at leas t one annotated Non-R G B image using a cycle G AN (Generative Adversarial Network), comprising steps of: (a) a testing device, on condition that (1) a learning device has performed processes of instructing a first transformer to transform at least one acquired first training image in an R G B format to at least one second training image in a non-R G B format, instructing a first discriminator to determine whether the second training image has a primary non- RG B format or a secondary non-RGB format, wherein the primary non-RGB format is the non-R G B format without a transformation from the RGB format and the secondary non-R G B format is the nonRGB format with the transformation from the R G B format, to thereby generate a 1 1-st result for training, and instructing a second transformer to transform the second training image to at least one third training image in the RGB format, (2) the learning device has performed processes of instructing the second transformer to transform at least one acquired fourth training image in the non-R G B format to at least one fifth training image in the R G B format, instructing a second discriminator to determine whether the fifth training image has a primary R G B format or a secondary R G B format, wherein the primary R G B format is the R G B format without a transformation from the non-RGB forma t and the secondary RGB format is the RGB format with the transformation from the non- RG B format, to thereby generate a 2 1-st result for training, and instructing the first transformer to transform the fifth training image to at least one sixth training image in the non-R G B format, and (3) the learning device has performed processes of calculating one or more losses by referring to at least part of the first training image, the second training image, the third training image, the fourth training image, the fifth training image, the sixth training image, the 1 _ 1-st result for training, and the 2 _ 1-st result for training, to thereby learn at least part of parameters of the first transformer, the second t ransformer, the first discriminator, and the second discriminator; acquiring at least one test image in the RGB format; and (b) the testing device instructing the first transformer to transform the test image into at least one resulting image in the non-RGB format.
The method of Claim 7, wherein the resulting image is used for learning a non-R G B object detector to detect one or more objects in the test image in the non-RGB format.
A learning device for learning transformation of at least one annotated R G B image into at least one annotated Non-RGB image using a cycle GAN (Generative Adversarial Network), comprising: at least one memory that stores instructions; and at least one processor configured to execu t e the instructions to: perform processes of (I) instructing a first transformer to transform at leas t one first image in an RGB format to at least one second image in a non-RGB format, instructing a first discriminator to determine whether the second image has a primary non-R G B format or a secondary non- RGB format, wherein the primary non-R G B format is the non-R GB forma t without a transformation from the R G B format and the secondary non-RGB format is the non-R G B format with the transformation from the R G B format, to thereby generate a 1 1- st result, and instructing a second transformer to transform the second image to at least one third image in the R G B format, (II) instructing the second transformer to transform at least one fourth image in the non-R G B format to at least one fifth i mage in the R G B format, instructing a second discriminator to determine whether the fifth image has a primary R G B format or a secondary R G B format, wherein the primary R G B format is the RG B format without a transformation from the non-R G B format and the secondary RGB format is the RGB format with the transformation from the non-RGB format, to thereby generate a 2 1-st result, and instructing the first transformer to transform the fifth image to at least one sixth image in the non-RGB format, and (III) calculating one or more losses by referring to at least part of the first image, the second image, the third image, the fourth image, the fifth image, the sixth image, the 1 _ 1-st result, and the 2 _ 1-st result, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.5.svg 0.64 6.78 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 _ 1-st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 _ 1-st result, G(F(X)) is the sixth image, y and fl are constants for adjusting each of weights of each of II-F(G(I)) I and IX-G(F(X)).
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.6.svg 0.68 6.79 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 1 -st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 1 -st result, G(F(X)) is the sixth image, y and fl are constants for adjusting each of weights of each of I-F(G (I)) and IX-G(F(X)), OD is an object detection loss, X is a constant for adjusting a weight of the object detection loss, and wherein the processor instructs an RGB object detector, which has been learned, to detect one or more objects in the third image, and compare at least part of information on estimated locations, sizes, and classes of the objects detected in the third image and at least part of information on true locations, sizes, and classes of the objects in at least one GT corresponding to the first image, to thereby calculate the object detection loss.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.7.svg 0.3 6.05 Black and white an FD loss for the first discriminator included in the losses is defined by a formula above, NR is any arbitrary image in the non-RGB format, D G (NR) is a 1 2-nd result of determining the arbitrary image in the non-RGB format from the first discriminator, G(I) is the second image, and D G (G(I)) is the 1 1-st result.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.8.svg 0.3 6 Black and white an SD loss for the second discriminator included in the losses is defined by a formula above, R is any arbitrary image in the R G B format, D F (R) is a 2 2-nd result of determining the arbitrary image in the RGB format from the second discriminator, F(X) is the fifth image, and D F (F(X)) is the 2 1- st result.
The learning device of Claim 9, wherein each of the first transformer and the second transformer includes at least part of an encoding layer and a decoding layer.
A testing device for testing transformation of at least one annotated R G B image into at least one annotated Non-R GB image using a cycle G AN (Generative Adversarial Network), comprising: at least one memory that stores instructions; and at least one processor, on condition tha t a learning device (1) has performed processes of instructing a first transformer to transform at least one acquired first training image in an R G B format to at least one second training image in a non-RGB format, instructing a first discriminator to determine whether the second training image has a primary non- RGB format or a secondary non-RGB format, wherein the primary nonRGB format is the non-RGB format without a transformation from the R G B format and the secondary non-RGB format is the non-RGB format with the transformation from the RGB format, to thereby generate a 11 -st result for training, and instructing a second transformer to transform the second training image to at least one third training image in the RGB format, (2) has performed processes of instructing the second transformer to transform at least one acquired fourth training image in the non-R G B format to at leas t one fifth training image in the R GB format, instructing a second discriminator to determine whether the fifth training image has a primary R G B format or a secondary R G B format, wherein the primary RGB format is the RGB format without a transformation from the non-R G B format and the secondary R G B format is the R G B format with the transformation from the non-R G B format, to thereby generate a 2 1- st result for training, and instructing the first transformer to transform the fifth training image to at least one sixth training image in the non-R G B format, and (3) has performed processes of calculating one or more losses by referring to at least part of the first training image, the second training image, the third training image, the fourth training image, the fifth training image, the sixth training image, the 1 _ 1-st result for training, and the 2 1-st result for training, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator; configured to execute the instructions to: perform a process of instructing the first transformer to transform at least one test image in the R G B format into at least one resulting image in the non- RG B format.
The testing device of Claim 15, wherein the resulting image is used for learning non-R G B object detector to detect one or more objects in the test image in the non-R G B format.
Patent
Atlas literature
Patent
US 10,395,392Patent drawings and their descriptions. Click a drawing to enlarge it.
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method for learning transformation of at least one annotated RGB image into at least one annotated Non-RGB image using a cycle G AN (Generative Adversarial Network), comprising steps of: (a) a learning device, if at least one first image in an RGB format is acquired, instructing a first transformer to transform the first image to at least one second image in a non-R G B format, instructing a first discriminator to determine whether the second image has a primary non-RGB format or a secondary non-RGB format, wherein the primary non-R G B format is the non-RGB format without a transformation from the RGB format and the secondary non-R G B format is the non-RGB format with the transformation from the R G B format, to thereby generate a 1 1-st result, and instructing a second transformer to transform the second image to at least one third image in the RGB format; (b) the learning device, if at least one fourth image in the non-R G B format is acquired, instructing the second transformer to transform the fourth image to at least one fifth image in the R G B format, instructing a second discriminator to determine whether the fifth image has a primary R G B format or a secondary RGB format, wherein the primary RGB format is the R G B format without a transformation from the non-R G B format and the secondary RGB format is the RGB format with the transformation from the non-RGB format, to thereby generate a 2 _ 1-st result, and instructing the first transformer to transform the fifth image to at least one sixth image in the non-R G B format; and (c) the learning device calculating one or more losses by referring to at least part of the first image, the second image, the third image, the fourth image, the fifth image, the sixth image, the 1 _ 1-st result, and the 2 _ 1-st result, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.1.svg 0.64 6.78 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 1-st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2_ 1 -st result, G(F(X)) is the sixth image, y and ft are cons tant s for adjusting each of weights of each of II- F(G(I)) I and IX- G(F(X)) I.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.2.svg 0.67 6.79 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G(G (I))) is the 1_ 1 -st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 1-st result, G(F(X)) is the sixth image, y and ft are constants for adjusting each of weights of each of II- F(G(I)) I and IX- G(F(X)) I, OD is an object detection loss, A is a constant for adjusting a weight of the object detection loss, and wherein the learning device instructs an RGB object detector, which has been learned, to detect one or more objects in the third image, and compare at leas t part of information on estimated locations, sizes, and classes of the objects detected in the third image and at least part of information on true locations, sizes, and classes of the objects in at least one G T corresponding to the first image, to thereby calculate the object detection loss.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.3.svg 0.3 6.05 Black and white an FD loss for the first discriminator included in the losses is defined by a formula above, NR is any arbitrary image in the non-RGB format, D G (NR) is a 1-2-nd result of determining the arbitrary image in the non-RGB format from the first discriminator, G(I) is the second image, and D G (G(I)) is the 1 _ 1-st result.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.4.svg 0.3 6 Black and white an SD loss for the second discriminator included in the losses is defined by a formula above, R is any arbitrary image in the R G B format, D F (R) is a 2 2-nd result of determining the arbitrary image in the RGB format from the second discriminator, F(X) is the fifth image, and D F (F(X)) is the 2 1- st result.
The method of Claim 1, wherein each of the first transformer and the second transformer includes at least part of an encoding layer and a decoding layer.
A method for testing transformation of at least one annotated RGB image into at leas t one annotated Non-R G B image using a cycle G AN (Generative Adversarial Network), comprising steps of: (a) a testing device, on condition that (1) a learning device has performed processes of instructing a first transformer to transform at least one acquired first training image in an R G B format to at least one second training image in a non-R G B format, instructing a first discriminator to determine whether the second training image has a primary non- RG B format or a secondary non-RGB format, wherein the primary non-RGB format is the non-R G B format without a transformation from the RGB format and the secondary non-R G B format is the nonRGB format with the transformation from the R G B format, to thereby generate a 1 1-st result for training, and instructing a second transformer to transform the second training image to at least one third training image in the RGB format, (2) the learning device has performed processes of instructing the second transformer to transform at least one acquired fourth training image in the non-R G B format to at least one fifth training image in the R G B format, instructing a second discriminator to determine whether the fifth training image has a primary R G B format or a secondary R G B format, wherein the primary R G B format is the R G B format without a transformation from the non-RGB forma t and the secondary RGB format is the RGB format with the transformation from the non- RG B format, to thereby generate a 2 1-st result for training, and instructing the first transformer to transform the fifth training image to at least one sixth training image in the non-R G B format, and (3) the learning device has performed processes of calculating one or more losses by referring to at least part of the first training image, the second training image, the third training image, the fourth training image, the fifth training image, the sixth training image, the 1 _ 1-st result for training, and the 2 _ 1-st result for training, to thereby learn at least part of parameters of the first transformer, the second t ransformer, the first discriminator, and the second discriminator; acquiring at least one test image in the RGB format; and (b) the testing device instructing the first transformer to transform the test image into at least one resulting image in the non-RGB format.
The method of Claim 7, wherein the resulting image is used for learning a non-R G B object detector to detect one or more objects in the test image in the non-RGB format.
A learning device for learning transformation of at least one annotated R G B image into at least one annotated Non-RGB image using a cycle GAN (Generative Adversarial Network), comprising: at least one memory that stores instructions; and at least one processor configured to execu t e the instructions to: perform processes of (I) instructing a first transformer to transform at leas t one first image in an RGB format to at least one second image in a non-RGB format, instructing a first discriminator to determine whether the second image has a primary non-R G B format or a secondary non- RGB format, wherein the primary non-R G B format is the non-R GB forma t without a transformation from the R G B format and the secondary non-RGB format is the non-R G B format with the transformation from the R G B format, to thereby generate a 1 1- st result, and instructing a second transformer to transform the second image to at least one third image in the R G B format, (II) instructing the second transformer to transform at least one fourth image in the non-R G B format to at least one fifth i mage in the R G B format, instructing a second discriminator to determine whether the fifth image has a primary R G B format or a secondary R G B format, wherein the primary R G B format is the RG B format without a transformation from the non-R G B format and the secondary RGB format is the RGB format with the transformation from the non-RGB format, to thereby generate a 2 1-st result, and instructing the first transformer to transform the fifth image to at least one sixth image in the non-RGB format, and (III) calculating one or more losses by referring to at least part of the first image, the second image, the third image, the fourth image, the fifth image, the sixth image, the 1 _ 1-st result, and the 2 _ 1-st result, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.5.svg 0.64 6.78 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 _ 1-st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 _ 1-st result, G(F(X)) is the sixth image, y and fl are constants for adjusting each of weights of each of II-F(G(I)) I and IX-G(F(X)).
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.6.svg 0.68 6.79 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 1 -st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 1 -st result, G(F(X)) is the sixth image, y and fl are constants for adjusting each of weights of each of I-F(G (I)) and IX-G(F(X)), OD is an object detection loss, X is a constant for adjusting a weight of the object detection loss, and wherein the processor instructs an RGB object detector, which has been learned, to detect one or more objects in the third image, and compare at least part of information on estimated locations, sizes, and classes of the objects detected in the third image and at least part of information on true locations, sizes, and classes of the objects in at least one GT corresponding to the first image, to thereby calculate the object detection loss.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.7.svg 0.3 6.05 Black and white an FD loss for the first discriminator included in the losses is defined by a formula above, NR is any arbitrary image in the non-RGB format, D G (NR) is a 1 2-nd result of determining the arbitrary image in the non-RGB format from the first discriminator, G(I) is the second image, and D G (G(I)) is the 1 1-st result.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.8.svg 0.3 6 Black and white an SD loss for the second discriminator included in the losses is defined by a formula above, R is any arbitrary image in the R G B format, D F (R) is a 2 2-nd result of determining the arbitrary image in the RGB format from the second discriminator, F(X) is the fifth image, and D F (F(X)) is the 2 1- st result.
The learning device of Claim 9, wherein each of the first transformer and the second transformer includes at least part of an encoding layer and a decoding layer.
A testing device for testing transformation of at least one annotated R G B image into at least one annotated Non-R GB image using a cycle G AN (Generative Adversarial Network), comprising: at least one memory that stores instructions; and at least one processor, on condition tha t a learning device (1) has performed processes of instructing a first transformer to transform at least one acquired first training image in an R G B format to at least one second training image in a non-RGB format, instructing a first discriminator to determine whether the second training image has a primary non- RGB format or a secondary non-RGB format, wherein the primary nonRGB format is the non-RGB format without a transformation from the R G B format and the secondary non-RGB format is the non-RGB format with the transformation from the RGB format, to thereby generate a 11 -st result for training, and instructing a second transformer to transform the second training image to at least one third training image in the RGB format, (2) has performed processes of instructing the second transformer to transform at least one acquired fourth training image in the non-R G B format to at leas t one fifth training image in the R GB format, instructing a second discriminator to determine whether the fifth training image has a primary R G B format or a secondary R G B format, wherein the primary RGB format is the RGB format without a transformation from the non-R G B format and the secondary R G B format is the R G B format with the transformation from the non-R G B format, to thereby generate a 2 1- st result for training, and instructing the first transformer to transform the fifth training image to at least one sixth training image in the non-R G B format, and (3) has performed processes of calculating one or more losses by referring to at least part of the first training image, the second training image, the third training image, the fourth training image, the fifth training image, the sixth training image, the 1 _ 1-st result for training, and the 2 1-st result for training, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator; configured to execute the instructions to: perform a process of instructing the first transformer to transform at least one test image in the R G B format into at least one resulting image in the non- RG B format.
The testing device of Claim 15, wherein the resulting image is used for learning non-R G B object detector to detect one or more objects in the test image in the non-R G B format.
Patent
Atlas literature
Patent
US 10,395,392Patent drawings and their descriptions. Click a drawing to enlarge it.
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method for learning transformation of at least one annotated RGB image into at least one annotated Non-RGB image using a cycle G AN (Generative Adversarial Network), comprising steps of: (a) a learning device, if at least one first image in an RGB format is acquired, instructing a first transformer to transform the first image to at least one second image in a non-R G B format, instructing a first discriminator to determine whether the second image has a primary non-RGB format or a secondary non-RGB format, wherein the primary non-R G B format is the non-RGB format without a transformation from the RGB format and the secondary non-R G B format is the non-RGB format with the transformation from the R G B format, to thereby generate a 1 1-st result, and instructing a second transformer to transform the second image to at least one third image in the RGB format; (b) the learning device, if at least one fourth image in the non-R G B format is acquired, instructing the second transformer to transform the fourth image to at least one fifth image in the R G B format, instructing a second discriminator to determine whether the fifth image has a primary R G B format or a secondary RGB format, wherein the primary RGB format is the R G B format without a transformation from the non-R G B format and the secondary RGB format is the RGB format with the transformation from the non-RGB format, to thereby generate a 2 _ 1-st result, and instructing the first transformer to transform the fifth image to at least one sixth image in the non-R G B format; and (c) the learning device calculating one or more losses by referring to at least part of the first image, the second image, the third image, the fourth image, the fifth image, the sixth image, the 1 _ 1-st result, and the 2 _ 1-st result, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.1.svg 0.64 6.78 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 1-st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2_ 1 -st result, G(F(X)) is the sixth image, y and ft are cons tant s for adjusting each of weights of each of II- F(G(I)) I and IX- G(F(X)) I.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.2.svg 0.67 6.79 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G(G (I))) is the 1_ 1 -st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 1-st result, G(F(X)) is the sixth image, y and ft are constants for adjusting each of weights of each of II- F(G(I)) I and IX- G(F(X)) I, OD is an object detection loss, A is a constant for adjusting a weight of the object detection loss, and wherein the learning device instructs an RGB object detector, which has been learned, to detect one or more objects in the third image, and compare at leas t part of information on estimated locations, sizes, and classes of the objects detected in the third image and at least part of information on true locations, sizes, and classes of the objects in at least one G T corresponding to the first image, to thereby calculate the object detection loss.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.3.svg 0.3 6.05 Black and white an FD loss for the first discriminator included in the losses is defined by a formula above, NR is any arbitrary image in the non-RGB format, D G (NR) is a 1-2-nd result of determining the arbitrary image in the non-RGB format from the first discriminator, G(I) is the second image, and D G (G(I)) is the 1 _ 1-st result.
The method of Claim 1, wherein, at the step of (c), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.4.svg 0.3 6 Black and white an SD loss for the second discriminator included in the losses is defined by a formula above, R is any arbitrary image in the R G B format, D F (R) is a 2 2-nd result of determining the arbitrary image in the RGB format from the second discriminator, F(X) is the fifth image, and D F (F(X)) is the 2 1- st result.
The method of Claim 1, wherein each of the first transformer and the second transformer includes at least part of an encoding layer and a decoding layer.
A method for testing transformation of at least one annotated RGB image into at leas t one annotated Non-R G B image using a cycle G AN (Generative Adversarial Network), comprising steps of: (a) a testing device, on condition that (1) a learning device has performed processes of instructing a first transformer to transform at least one acquired first training image in an R G B format to at least one second training image in a non-R G B format, instructing a first discriminator to determine whether the second training image has a primary non- RG B format or a secondary non-RGB format, wherein the primary non-RGB format is the non-R G B format without a transformation from the RGB format and the secondary non-R G B format is the nonRGB format with the transformation from the R G B format, to thereby generate a 1 1-st result for training, and instructing a second transformer to transform the second training image to at least one third training image in the RGB format, (2) the learning device has performed processes of instructing the second transformer to transform at least one acquired fourth training image in the non-R G B format to at least one fifth training image in the R G B format, instructing a second discriminator to determine whether the fifth training image has a primary R G B format or a secondary R G B format, wherein the primary R G B format is the R G B format without a transformation from the non-RGB forma t and the secondary RGB format is the RGB format with the transformation from the non- RG B format, to thereby generate a 2 1-st result for training, and instructing the first transformer to transform the fifth training image to at least one sixth training image in the non-R G B format, and (3) the learning device has performed processes of calculating one or more losses by referring to at least part of the first training image, the second training image, the third training image, the fourth training image, the fifth training image, the sixth training image, the 1 _ 1-st result for training, and the 2 _ 1-st result for training, to thereby learn at least part of parameters of the first transformer, the second t ransformer, the first discriminator, and the second discriminator; acquiring at least one test image in the RGB format; and (b) the testing device instructing the first transformer to transform the test image into at least one resulting image in the non-RGB format.
The method of Claim 7, wherein the resulting image is used for learning a non-R G B object detector to detect one or more objects in the test image in the non-RGB format.
A learning device for learning transformation of at least one annotated R G B image into at least one annotated Non-RGB image using a cycle GAN (Generative Adversarial Network), comprising: at least one memory that stores instructions; and at least one processor configured to execu t e the instructions to: perform processes of (I) instructing a first transformer to transform at leas t one first image in an RGB format to at least one second image in a non-RGB format, instructing a first discriminator to determine whether the second image has a primary non-R G B format or a secondary non- RGB format, wherein the primary non-R G B format is the non-R GB forma t without a transformation from the R G B format and the secondary non-RGB format is the non-R G B format with the transformation from the R G B format, to thereby generate a 1 1- st result, and instructing a second transformer to transform the second image to at least one third image in the R G B format, (II) instructing the second transformer to transform at least one fourth image in the non-R G B format to at least one fifth i mage in the R G B format, instructing a second discriminator to determine whether the fifth image has a primary R G B format or a secondary R G B format, wherein the primary R G B format is the RG B format without a transformation from the non-R G B format and the secondary RGB format is the RGB format with the transformation from the non-RGB format, to thereby generate a 2 1-st result, and instructing the first transformer to transform the fifth image to at least one sixth image in the non-RGB format, and (III) calculating one or more losses by referring to at least part of the first image, the second image, the third image, the fourth image, the fifth image, the sixth image, the 1 _ 1-st result, and the 2 _ 1-st result, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.5.svg 0.64 6.78 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 _ 1-st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 _ 1-st result, G(F(X)) is the sixth image, y and fl are constants for adjusting each of weights of each of II-F(G(I)) I and IX-G(F(X)).
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.6.svg 0.68 6.79 Black and white a first loss for transformation included in said one or more losses is defined by a formula above, I is the first image, G(I) is the second image, D G (G(I))) is the 1 1 -st result, F(G(I)) is the third image, X is the fourth image, F(X) is the fifth image, D F (F(X)) is the 2 1 -st result, G(F(X)) is the sixth image, y and fl are constants for adjusting each of weights of each of I-F(G (I)) and IX-G(F(X)), OD is an object detection loss, X is a constant for adjusting a weight of the object detection loss, and wherein the processor instructs an RGB object detector, which has been learned, to detect one or more objects in the third image, and compare at least part of information on estimated locations, sizes, and classes of the objects detected in the third image and at least part of information on true locations, sizes, and classes of the objects in at least one GT corresponding to the first image, to thereby calculate the object detection loss.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.7.svg 0.3 6.05 Black and white an FD loss for the first discriminator included in the losses is defined by a formula above, NR is any arbitrary image in the non-RGB format, D G (NR) is a 1 2-nd result of determining the arbitrary image in the non-RGB format from the first discriminator, G(I) is the second image, and D G (G(I)) is the 1 1-st result.
The learning device of Claim 9, wherein, at the process of (III), SVG 16263275.01-31-2019.JRKXNGXQRXEAPX0.CLM.8.svg 0.3 6 Black and white an SD loss for the second discriminator included in the losses is defined by a formula above, R is any arbitrary image in the R G B format, D F (R) is a 2 2-nd result of determining the arbitrary image in the RGB format from the second discriminator, F(X) is the fifth image, and D F (F(X)) is the 2 1- st result.
The learning device of Claim 9, wherein each of the first transformer and the second transformer includes at least part of an encoding layer and a decoding layer.
A testing device for testing transformation of at least one annotated R G B image into at least one annotated Non-R GB image using a cycle G AN (Generative Adversarial Network), comprising: at least one memory that stores instructions; and at least one processor, on condition tha t a learning device (1) has performed processes of instructing a first transformer to transform at least one acquired first training image in an R G B format to at least one second training image in a non-RGB format, instructing a first discriminator to determine whether the second training image has a primary non- RGB format or a secondary non-RGB format, wherein the primary nonRGB format is the non-RGB format without a transformation from the R G B format and the secondary non-RGB format is the non-RGB format with the transformation from the RGB format, to thereby generate a 11 -st result for training, and instructing a second transformer to transform the second training image to at least one third training image in the RGB format, (2) has performed processes of instructing the second transformer to transform at least one acquired fourth training image in the non-R G B format to at leas t one fifth training image in the R GB format, instructing a second discriminator to determine whether the fifth training image has a primary R G B format or a secondary R G B format, wherein the primary RGB format is the RGB format without a transformation from the non-R G B format and the secondary R G B format is the R G B format with the transformation from the non-R G B format, to thereby generate a 2 1- st result for training, and instructing the first transformer to transform the fifth training image to at least one sixth training image in the non-R G B format, and (3) has performed processes of calculating one or more losses by referring to at least part of the first training image, the second training image, the third training image, the fourth training image, the fifth training image, the sixth training image, the 1 _ 1-st result for training, and the 2 1-st result for training, to thereby learn at least part of parameters of the first transformer, the second transformer, the first discriminator, and the second discriminator; configured to execute the instructions to: perform a process of instructing the first transformer to transform at least one test image in the R G B format into at least one resulting image in the non- RG B format.
The testing device of Claim 15, wherein the resulting image is used for learning non-R G B object detector to detect one or more objects in the test image in the non-R G B format.
