Image Segmentation System FIG. 1 is a view provided to explain a segmentation-generative adversarial network (GAN) according to an embodiment of the present disclosure. The segmentation-GAN according to an embodiment is an image segmentation system that utilizes a GAN architecture. Specifically, a generator of the GAN may be implemented as a ‘semantic segmentation network for segmenting a region in an image’, instead of a ‘network for generating an image’, and a discriminator of the GAN may receive a result of segmenting a region by the semantic segmentation net-work and a ground truth (GT) and may discriminate the same, and the semantic segmentation network may be B₂ trained based on discrimination performance and perfor-mance of the semantic segmentation network. To achieve this, the image segmentation system according to an embodiment includes a semantic segmentation net-work 110 and a discriminator 120 as shown in FIG. 1. The semantic segmentation network 110 is a network that is trained to segment regions in an input image when an image is inputted, and to output the image. The semantic segmentation network 110 generates a semantic mask regarding the input image, and converts the input image into a semantic mask-added image (semantic labeling image). As shown in FIG. 2, the input to the semantic segmen-tation network 110 is an image for segmenting regions (Input Image), and the output therefrom is a semantic mask (Seg-Out). The semantic segmentation network 110 may be imple-mented by a deep learning-based segmentation network, and embodiments where the semantic segmentation network is implemented by segmentation networks of other structures are not excluded. The discriminator 120 is a network that is trained to receive the semantic mask (Seg-Out), which is the result of region segmentation outputted from the semantic segmen-tation network 110, and a GT semantic mask regarding this, and to discriminate (identify) which mask is the mask generated at the semantic segmentation network 110 and which mask is the GT semantic mask.

Training of Semantic Segmentation Network The semantic segmentation network 110 is trained to degrade discrimination performance of the discriminator 120 while enhancing its own region segmenting perfor-mance and segmentation image generation performance. A loss function of the semantic segmentation network 110 to achieve this is illustrated in FIG. 3, and FIG. 4 illustrates a loss generator 210 for generating a loss of the semantic segmentation network 110 based on the loss function. As shown in FIG. 3, the loss function of the semantic segmentation network 110 includes 1) an adversarial loss, 2) a segmentation performance loss, and 3) a content loss, and applies settable weights (α,β, γ). The loss function LG of the semantic segmentation net-work 110 may be expressed by the following equation: LG = × E[log(D(Gseg(I)))] + β SegPerformance(SegOut, GT) + γ ×ContentLoss(SegOut, GT) × where D () is the discriminator 120, Gseg() is the semantic segmentation network 110, E is Expectation cal-culation, SegPerformance() is a function for calculating region segmentation performance by comparing masks of two images, and ContentLoss() is a function for calculating region segmentation image generation performance by com-paring two mask-added images. In addition, 0≤α≤1,0≤β≤1,0≤γ≤1, α+β+γ=1 1) Adversarial Loss The adversarial loss is a loss for evaluating discrimination (identification) performance of the discriminator 120 regard-ing the semantic mask (Seg-Out) and the GT semantic mask. As the discrimination performance of the discriminator 120 is higher, the adversarial loss is greater, and, as the discrimi-nation performance of the discriminator 120 is lower, the adversarial loss is smaller. As shown in FIG. 4, the loss generator 210 calculates the adversarial loss, based on 1) the semantic mask (Seg-Out) outputted from the semantic segmentation network 110, 2) the GT semantic mask, and 3) a result of discrimination (Dis-Out) of the discriminator 120. 2) Segmentation Performance Loss The segmentation performance loss is a loss for evaluat-ing region segmentation performance of the semantic seg-mentation network 110. As the region segmentation perfor-mance of the semantic segmentation network 110 is higher, the segmentation performance loss is smaller, and, as the region segmentation performance is lower, the segmentation performance loss is greater. The loss generator 210 may calculate the region segmen-tation performance of the semantic segmentation network 110 by comparing 1) the semantic mask generated at the semantic segmentation network 110 and 2) the GT semantic mask, specifically, by calculating Dice, Intersection over Union (IoU) regarding the two images. 3) Content Loss The content loss is a loss for evaluating region segmen-tation image generation performance in the semantic seg-mentation network 110. As the semantic mask-added image generation performance of the semantic segmentation net-work 110 is higher, the content loss is smaller, and, as the image generation performance is lower, the content loss is greater. The loss generator 210 may calculate the region segmen-tation image generation performance of the semantic seg-mentation network 110 by comparing 1) an input image to which the semantic mask generated at the semantic segmen-tation network 110 is added, and 2) an input image to which the GT semantic mask is added, specifically, by calculating similarity between the two images. The similarity between the two images may be calculated by using the following Norm measurement equation: \X\p=(Σi=1 n?xi?p)1/p Another method of calculating similarity between the two images is as shown in FIG. 5. Specifically, a feature vector #1 may be extracted from the input image to which the semantic mask is added at a feature extraction network 220, and a feature vector #2 may be extracted from the image to which the GT semantic mask is added at a feature extraction network 230, and then, a similarity measurement device 240 may calculate similarity between the two images by calcu-lating similarity between the feature vector #1 and the feature vector #2 by using a measurement equation.

Training of Discriminator The discriminator 120 is trained to enhance its own discrimination performance. To achieve this, a loss function LD of the discriminator 120 may be expressed by the following equation: LD=E[log(D(MGT)]+E[log(1−D(Gseg(I))] where D () is the discriminator 120, Gseg () is the semantic segmentation network 110, E is expectation cal-culation, and MGT is a GT semantic mask. LD is a loss for evaluating discrimination (identification) performance of the discriminator 120 regarding the input image to which the semantic mask is added and the input image to which the GT semantic mask is added. As the discrimination performance of the discriminator 120 is higher, LD is greater, and, as the discrimination performance of the discriminator 120 is lower, LD is smaller. FIG. 6 illustrates a loss generator 310 for generating a loss of the discriminator 120, based on the loss function. As shown in the drawing, the loss generator 310 may calculate B₂ LD, based on the result of discrimination (Dis-Out) output-ted from the discriminator 120.

Performance Evaluation In order to evaluate performance of the segmentation-GAN according to an embodiment, a test was performed by using Inria aerial image data and SpaceNet data. The left view of FIG. 7 suggests a GT semantic mask, the center view suggests a result of image segmentation by a related-art semantic segmentation network, and the right view suggests a result of image segmentation by the seman-tic segmentation network 110 which is trained in the seg-mentation-GAN architecture according to an embodiment. As shown in FIG. 7, the result of image segmentation according to an embodiment is more similar to the GT semantic mask than the result of image segmentation by the related-art method. The left view of FIG. 8 illustrates overlap of the result of image segmentation by the related-art semantic segmenta-tion network on the GT semantic mask, and the right view illustrates overlap of the result of image segmentation by the semantic segmentation network trained in the segmentation-GAN architecture according to an embodiment, on the GT semantic mask. Referring to the drawing, specifically, the region illus-trated by the blue circle, the related-art method does not segment regions which are the GT semantic mask, whereas the method according to an embodiment well segments most of the regions appearing in the GT image.

Hardware Structure of Image Segmentation System FIG. 9 is a view illustrating a hardware structure of the image segmentation system shown in FIG. 1. As shown in the drawing, the image segmentation system may be imple-mented by a computing system that includes a communica-tion unit 410, an output unit 420, a processor 430, an input unit 440, and a storage unit 450. The communication unit 410 is a communication means for communicating with an external device and connecting to an external network. The output unit 420 is a display that displays results of processing by the processor 430. The input unit 440 is an input means for receiving a user’s command and forwarding the same to the processor 430. The processor 430 refers to graphic processing units (GPUs) and a central processing unit (CPU) for executing the networks constituting the image segmentation system shown in FIG. 1. The storage unit 450 provides a storage space necessary for operating the processor 430 and per-forming functions.

Variations The image segmentation system for training the semantic segmentation network by utilizing the GAN architecture has been described with reference to preferred embodiments. Embodiments of the present disclosure suggest a method for implementing a generator of a GAN as a semantic segmentation network for segmenting regions in an image, instead of a network for generating images, for allowing a discriminator of the GAN to receive a result of image segmentation of the semantic segmentation network, and a GT, and to discriminate between them, and for training the semantic segmentation network based on performance of the discriminator and performance of the semantic segmentation network. Accordingly, accuracy of image segmentation can be enhanced, and particularly, exact segmentation is possible for a precise image. In the above-described embodiments, the aviation image (FIG. 7) and the satellite image (FIG. 8) are segmented to evaluate performance. However, this is merely an example, and the method and the system according to embodiments may be widely applied to image segmentations of other images including medical images, CCTV images, front images of autonomous vehicles as well as aviation images and satellite images. The technical concept of the present disclosure may be applied to a computer-readable recording medium which records a computer program for performing the functions of the apparatus and the method according to the present embodiments. In addition, the technical idea according to various embodiments of the present disclosure may be implemented in the form of a computer readable code recorded on the computer-readable recording medium. The computer-readable recording medium may be any data stor-age device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a read only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. A computer readable code or program that is stored in the computer readable recording medium may be transmitted via a net-work connected between computers. In addition, while preferred embodiments of the present disclosure have been illustrated and described, the present disclosure is not limited to the above-described specific embodiments. Various changes can be made by a person skilled in the art without departing from the scope of the present disclosure claimed in claims, and also, changed embodiments should not be understood as being separate from the technical idea or prospect of the present disclosure. What is claimed is:

A system for training an image segmentation network, the system comprising: a processor configured to: input an image to a first network which is trained to output a region segmentation result regarding an input image, and generate a region segmentation result; and input the region segmentation result generated by the first generator and a GT semantic mask to a second network, and acquire a discrimination result, the second network being trained to discriminate input-ted region segmentation results as a result generated by the first network and a GT semantic mask, respec-tively; and train the first network and the second network by using the discrimination result; and a storage unit configured to provide a necessary storage space to the processor.

An image segmentation method comprising: receiving an input of an image; and generating a region segmentation result regarding the image inputted at the input step by using a first net-work, wherein the first network is trained to output a region segmentation result regarding an input image when an image is inputted, and discrimination performance of a second network is used for training of the first network, wherein the second network is trained to discriminate inputted region segmentation results as a result gener-ated by the first network and a GT semantic mask, respectively, when the region segmentation results are inputted. ∗ ∗ ∗ ∗ ∗

Image Segmentation System FIG. 1 is a view provided to explain a segmentation-generative adversarial network (GAN) according to an embodiment of the present disclosure. The segmentation-GAN according to an embodiment is an image segmentation system that utilizes a GAN architecture. Specifically, a generator of the GAN may be implemented as a ‘semantic segmentation network for segmenting a region in an image’, instead of a ‘network for generating an image’, and a discriminator of the GAN may receive a result of segmenting a region by the semantic segmentation net-work and a ground truth (GT) and may discriminate the same, and the semantic segmentation network may be B₂ trained based on discrimination performance and perfor-mance of the semantic segmentation network. To achieve this, the image segmentation system according to an embodiment includes a semantic segmentation net-work 110 and a discriminator 120 as shown in FIG. 1. The semantic segmentation network 110 is a network that is trained to segment regions in an input image when an image is inputted, and to output the image. The semantic segmentation network 110 generates a semantic mask regarding the input image, and converts the input image into a semantic mask-added image (semantic labeling image). As shown in FIG. 2, the input to the semantic segmen-tation network 110 is an image for segmenting regions (Input Image), and the output therefrom is a semantic mask (Seg-Out). The semantic segmentation network 110 may be imple-mented by a deep learning-based segmentation network, and embodiments where the semantic segmentation network is implemented by segmentation networks of other structures are not excluded. The discriminator 120 is a network that is trained to receive the semantic mask (Seg-Out), which is the result of region segmentation outputted from the semantic segmen-tation network 110, and a GT semantic mask regarding this, and to discriminate (identify) which mask is the mask generated at the semantic segmentation network 110 and which mask is the GT semantic mask.

Training of Semantic Segmentation Network The semantic segmentation network 110 is trained to degrade discrimination performance of the discriminator 120 while enhancing its own region segmenting perfor-mance and segmentation image generation performance. A loss function of the semantic segmentation network 110 to achieve this is illustrated in FIG. 3, and FIG. 4 illustrates a loss generator 210 for generating a loss of the semantic segmentation network 110 based on the loss function. As shown in FIG. 3, the loss function of the semantic segmentation network 110 includes 1) an adversarial loss, 2) a segmentation performance loss, and 3) a content loss, and applies settable weights (α,β, γ). The loss function LG of the semantic segmentation net-work 110 may be expressed by the following equation: LG = × E[log(D(Gseg(I)))] + β SegPerformance(SegOut, GT) + γ ×ContentLoss(SegOut, GT) × where D () is the discriminator 120, Gseg() is the semantic segmentation network 110, E is Expectation cal-culation, SegPerformance() is a function for calculating region segmentation performance by comparing masks of two images, and ContentLoss() is a function for calculating region segmentation image generation performance by com-paring two mask-added images. In addition, 0≤α≤1,0≤β≤1,0≤γ≤1, α+β+γ=1 1) Adversarial Loss The adversarial loss is a loss for evaluating discrimination (identification) performance of the discriminator 120 regard-ing the semantic mask (Seg-Out) and the GT semantic mask. As the discrimination performance of the discriminator 120 is higher, the adversarial loss is greater, and, as the discrimi-nation performance of the discriminator 120 is lower, the adversarial loss is smaller. As shown in FIG. 4, the loss generator 210 calculates the adversarial loss, based on 1) the semantic mask (Seg-Out) outputted from the semantic segmentation network 110, 2) the GT semantic mask, and 3) a result of discrimination (Dis-Out) of the discriminator 120. 2) Segmentation Performance Loss The segmentation performance loss is a loss for evaluat-ing region segmentation performance of the semantic seg-mentation network 110. As the region segmentation perfor-mance of the semantic segmentation network 110 is higher, the segmentation performance loss is smaller, and, as the region segmentation performance is lower, the segmentation performance loss is greater. The loss generator 210 may calculate the region segmen-tation performance of the semantic segmentation network 110 by comparing 1) the semantic mask generated at the semantic segmentation network 110 and 2) the GT semantic mask, specifically, by calculating Dice, Intersection over Union (IoU) regarding the two images. 3) Content Loss The content loss is a loss for evaluating region segmen-tation image generation performance in the semantic seg-mentation network 110. As the semantic mask-added image generation performance of the semantic segmentation net-work 110 is higher, the content loss is smaller, and, as the image generation performance is lower, the content loss is greater. The loss generator 210 may calculate the region segmen-tation image generation performance of the semantic seg-mentation network 110 by comparing 1) an input image to which the semantic mask generated at the semantic segmen-tation network 110 is added, and 2) an input image to which the GT semantic mask is added, specifically, by calculating similarity between the two images. The similarity between the two images may be calculated by using the following Norm measurement equation: \X\p=(Σi=1 n?xi?p)1/p Another method of calculating similarity between the two images is as shown in FIG. 5. Specifically, a feature vector #1 may be extracted from the input image to which the semantic mask is added at a feature extraction network 220, and a feature vector #2 may be extracted from the image to which the GT semantic mask is added at a feature extraction network 230, and then, a similarity measurement device 240 may calculate similarity between the two images by calcu-lating similarity between the feature vector #1 and the feature vector #2 by using a measurement equation.

Training of Discriminator The discriminator 120 is trained to enhance its own discrimination performance. To achieve this, a loss function LD of the discriminator 120 may be expressed by the following equation: LD=E[log(D(MGT)]+E[log(1−D(Gseg(I))] where D () is the discriminator 120, Gseg () is the semantic segmentation network 110, E is expectation cal-culation, and MGT is a GT semantic mask. LD is a loss for evaluating discrimination (identification) performance of the discriminator 120 regarding the input image to which the semantic mask is added and the input image to which the GT semantic mask is added. As the discrimination performance of the discriminator 120 is higher, LD is greater, and, as the discrimination performance of the discriminator 120 is lower, LD is smaller. FIG. 6 illustrates a loss generator 310 for generating a loss of the discriminator 120, based on the loss function. As shown in the drawing, the loss generator 310 may calculate B₂ LD, based on the result of discrimination (Dis-Out) output-ted from the discriminator 120.

Performance Evaluation In order to evaluate performance of the segmentation-GAN according to an embodiment, a test was performed by using Inria aerial image data and SpaceNet data. The left view of FIG. 7 suggests a GT semantic mask, the center view suggests a result of image segmentation by a related-art semantic segmentation network, and the right view suggests a result of image segmentation by the seman-tic segmentation network 110 which is trained in the seg-mentation-GAN architecture according to an embodiment. As shown in FIG. 7, the result of image segmentation according to an embodiment is more similar to the GT semantic mask than the result of image segmentation by the related-art method. The left view of FIG. 8 illustrates overlap of the result of image segmentation by the related-art semantic segmenta-tion network on the GT semantic mask, and the right view illustrates overlap of the result of image segmentation by the semantic segmentation network trained in the segmentation-GAN architecture according to an embodiment, on the GT semantic mask. Referring to the drawing, specifically, the region illus-trated by the blue circle, the related-art method does not segment regions which are the GT semantic mask, whereas the method according to an embodiment well segments most of the regions appearing in the GT image.

Hardware Structure of Image Segmentation System FIG. 9 is a view illustrating a hardware structure of the image segmentation system shown in FIG. 1. As shown in the drawing, the image segmentation system may be imple-mented by a computing system that includes a communica-tion unit 410, an output unit 420, a processor 430, an input unit 440, and a storage unit 450. The communication unit 410 is a communication means for communicating with an external device and connecting to an external network. The output unit 420 is a display that displays results of processing by the processor 430. The input unit 440 is an input means for receiving a user’s command and forwarding the same to the processor 430. The processor 430 refers to graphic processing units (GPUs) and a central processing unit (CPU) for executing the networks constituting the image segmentation system shown in FIG. 1. The storage unit 450 provides a storage space necessary for operating the processor 430 and per-forming functions.

Variations The image segmentation system for training the semantic segmentation network by utilizing the GAN architecture has been described with reference to preferred embodiments. Embodiments of the present disclosure suggest a method for implementing a generator of a GAN as a semantic segmentation network for segmenting regions in an image, instead of a network for generating images, for allowing a discriminator of the GAN to receive a result of image segmentation of the semantic segmentation network, and a GT, and to discriminate between them, and for training the semantic segmentation network based on performance of the discriminator and performance of the semantic segmentation network. Accordingly, accuracy of image segmentation can be enhanced, and particularly, exact segmentation is possible for a precise image. In the above-described embodiments, the aviation image (FIG. 7) and the satellite image (FIG. 8) are segmented to evaluate performance. However, this is merely an example, and the method and the system according to embodiments may be widely applied to image segmentations of other images including medical images, CCTV images, front images of autonomous vehicles as well as aviation images and satellite images. The technical concept of the present disclosure may be applied to a computer-readable recording medium which records a computer program for performing the functions of the apparatus and the method according to the present embodiments. In addition, the technical idea according to various embodiments of the present disclosure may be implemented in the form of a computer readable code recorded on the computer-readable recording medium. The computer-readable recording medium may be any data stor-age device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a read only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. A computer readable code or program that is stored in the computer readable recording medium may be transmitted via a net-work connected between computers. In addition, while preferred embodiments of the present disclosure have been illustrated and described, the present disclosure is not limited to the above-described specific embodiments. Various changes can be made by a person skilled in the art without departing from the scope of the present disclosure claimed in claims, and also, changed embodiments should not be understood as being separate from the technical idea or prospect of the present disclosure. What is claimed is:

A system for training an image segmentation network, the system comprising: a processor configured to: input an image to a first network which is trained to output a region segmentation result regarding an input image, and generate a region segmentation result; and input the region segmentation result generated by the first generator and a GT semantic mask to a second network, and acquire a discrimination result, the second network being trained to discriminate input-ted region segmentation results as a result generated by the first network and a GT semantic mask, respec-tively; and train the first network and the second network by using the discrimination result; and a storage unit configured to provide a necessary storage space to the processor.

An image segmentation method comprising: receiving an input of an image; and generating a region segmentation result regarding the image inputted at the input step by using a first net-work, wherein the first network is trained to output a region segmentation result regarding an input image when an image is inputted, and discrimination performance of a second network is used for training of the first network, wherein the second network is trained to discriminate inputted region segmentation results as a result gener-ated by the first network and a GT semantic mask, respectively, when the region segmentation results are inputted. ∗ ∗ ∗ ∗ ∗

Image Segmentation System FIG. 1 is a view provided to explain a segmentation-generative adversarial network (GAN) according to an embodiment of the present disclosure. The segmentation-GAN according to an embodiment is an image segmentation system that utilizes a GAN architecture. Specifically, a generator of the GAN may be implemented as a ‘semantic segmentation network for segmenting a region in an image’, instead of a ‘network for generating an image’, and a discriminator of the GAN may receive a result of segmenting a region by the semantic segmentation net-work and a ground truth (GT) and may discriminate the same, and the semantic segmentation network may be B₂ trained based on discrimination performance and perfor-mance of the semantic segmentation network. To achieve this, the image segmentation system according to an embodiment includes a semantic segmentation net-work 110 and a discriminator 120 as shown in FIG. 1. The semantic segmentation network 110 is a network that is trained to segment regions in an input image when an image is inputted, and to output the image. The semantic segmentation network 110 generates a semantic mask regarding the input image, and converts the input image into a semantic mask-added image (semantic labeling image). As shown in FIG. 2, the input to the semantic segmen-tation network 110 is an image for segmenting regions (Input Image), and the output therefrom is a semantic mask (Seg-Out). The semantic segmentation network 110 may be imple-mented by a deep learning-based segmentation network, and embodiments where the semantic segmentation network is implemented by segmentation networks of other structures are not excluded. The discriminator 120 is a network that is trained to receive the semantic mask (Seg-Out), which is the result of region segmentation outputted from the semantic segmen-tation network 110, and a GT semantic mask regarding this, and to discriminate (identify) which mask is the mask generated at the semantic segmentation network 110 and which mask is the GT semantic mask.

Training of Semantic Segmentation Network The semantic segmentation network 110 is trained to degrade discrimination performance of the discriminator 120 while enhancing its own region segmenting perfor-mance and segmentation image generation performance. A loss function of the semantic segmentation network 110 to achieve this is illustrated in FIG. 3, and FIG. 4 illustrates a loss generator 210 for generating a loss of the semantic segmentation network 110 based on the loss function. As shown in FIG. 3, the loss function of the semantic segmentation network 110 includes 1) an adversarial loss, 2) a segmentation performance loss, and 3) a content loss, and applies settable weights (α,β, γ). The loss function LG of the semantic segmentation net-work 110 may be expressed by the following equation: LG = × E[log(D(Gseg(I)))] + β SegPerformance(SegOut, GT) + γ ×ContentLoss(SegOut, GT) × where D () is the discriminator 120, Gseg() is the semantic segmentation network 110, E is Expectation cal-culation, SegPerformance() is a function for calculating region segmentation performance by comparing masks of two images, and ContentLoss() is a function for calculating region segmentation image generation performance by com-paring two mask-added images. In addition, 0≤α≤1,0≤β≤1,0≤γ≤1, α+β+γ=1 1) Adversarial Loss The adversarial loss is a loss for evaluating discrimination (identification) performance of the discriminator 120 regard-ing the semantic mask (Seg-Out) and the GT semantic mask. As the discrimination performance of the discriminator 120 is higher, the adversarial loss is greater, and, as the discrimi-nation performance of the discriminator 120 is lower, the adversarial loss is smaller. As shown in FIG. 4, the loss generator 210 calculates the adversarial loss, based on 1) the semantic mask (Seg-Out) outputted from the semantic segmentation network 110, 2) the GT semantic mask, and 3) a result of discrimination (Dis-Out) of the discriminator 120. 2) Segmentation Performance Loss The segmentation performance loss is a loss for evaluat-ing region segmentation performance of the semantic seg-mentation network 110. As the region segmentation perfor-mance of the semantic segmentation network 110 is higher, the segmentation performance loss is smaller, and, as the region segmentation performance is lower, the segmentation performance loss is greater. The loss generator 210 may calculate the region segmen-tation performance of the semantic segmentation network 110 by comparing 1) the semantic mask generated at the semantic segmentation network 110 and 2) the GT semantic mask, specifically, by calculating Dice, Intersection over Union (IoU) regarding the two images. 3) Content Loss The content loss is a loss for evaluating region segmen-tation image generation performance in the semantic seg-mentation network 110. As the semantic mask-added image generation performance of the semantic segmentation net-work 110 is higher, the content loss is smaller, and, as the image generation performance is lower, the content loss is greater. The loss generator 210 may calculate the region segmen-tation image generation performance of the semantic seg-mentation network 110 by comparing 1) an input image to which the semantic mask generated at the semantic segmen-tation network 110 is added, and 2) an input image to which the GT semantic mask is added, specifically, by calculating similarity between the two images. The similarity between the two images may be calculated by using the following Norm measurement equation: \X\p=(Σi=1 n?xi?p)1/p Another method of calculating similarity between the two images is as shown in FIG. 5. Specifically, a feature vector #1 may be extracted from the input image to which the semantic mask is added at a feature extraction network 220, and a feature vector #2 may be extracted from the image to which the GT semantic mask is added at a feature extraction network 230, and then, a similarity measurement device 240 may calculate similarity between the two images by calcu-lating similarity between the feature vector #1 and the feature vector #2 by using a measurement equation.

Training of Discriminator The discriminator 120 is trained to enhance its own discrimination performance. To achieve this, a loss function LD of the discriminator 120 may be expressed by the following equation: LD=E[log(D(MGT)]+E[log(1−D(Gseg(I))] where D () is the discriminator 120, Gseg () is the semantic segmentation network 110, E is expectation cal-culation, and MGT is a GT semantic mask. LD is a loss for evaluating discrimination (identification) performance of the discriminator 120 regarding the input image to which the semantic mask is added and the input image to which the GT semantic mask is added. As the discrimination performance of the discriminator 120 is higher, LD is greater, and, as the discrimination performance of the discriminator 120 is lower, LD is smaller. FIG. 6 illustrates a loss generator 310 for generating a loss of the discriminator 120, based on the loss function. As shown in the drawing, the loss generator 310 may calculate B₂ LD, based on the result of discrimination (Dis-Out) output-ted from the discriminator 120.

Performance Evaluation In order to evaluate performance of the segmentation-GAN according to an embodiment, a test was performed by using Inria aerial image data and SpaceNet data. The left view of FIG. 7 suggests a GT semantic mask, the center view suggests a result of image segmentation by a related-art semantic segmentation network, and the right view suggests a result of image segmentation by the seman-tic segmentation network 110 which is trained in the seg-mentation-GAN architecture according to an embodiment. As shown in FIG. 7, the result of image segmentation according to an embodiment is more similar to the GT semantic mask than the result of image segmentation by the related-art method. The left view of FIG. 8 illustrates overlap of the result of image segmentation by the related-art semantic segmenta-tion network on the GT semantic mask, and the right view illustrates overlap of the result of image segmentation by the semantic segmentation network trained in the segmentation-GAN architecture according to an embodiment, on the GT semantic mask. Referring to the drawing, specifically, the region illus-trated by the blue circle, the related-art method does not segment regions which are the GT semantic mask, whereas the method according to an embodiment well segments most of the regions appearing in the GT image.

Hardware Structure of Image Segmentation System FIG. 9 is a view illustrating a hardware structure of the image segmentation system shown in FIG. 1. As shown in the drawing, the image segmentation system may be imple-mented by a computing system that includes a communica-tion unit 410, an output unit 420, a processor 430, an input unit 440, and a storage unit 450. The communication unit 410 is a communication means for communicating with an external device and connecting to an external network. The output unit 420 is a display that displays results of processing by the processor 430. The input unit 440 is an input means for receiving a user’s command and forwarding the same to the processor 430. The processor 430 refers to graphic processing units (GPUs) and a central processing unit (CPU) for executing the networks constituting the image segmentation system shown in FIG. 1. The storage unit 450 provides a storage space necessary for operating the processor 430 and per-forming functions.

Variations The image segmentation system for training the semantic segmentation network by utilizing the GAN architecture has been described with reference to preferred embodiments. Embodiments of the present disclosure suggest a method for implementing a generator of a GAN as a semantic segmentation network for segmenting regions in an image, instead of a network for generating images, for allowing a discriminator of the GAN to receive a result of image segmentation of the semantic segmentation network, and a GT, and to discriminate between them, and for training the semantic segmentation network based on performance of the discriminator and performance of the semantic segmentation network. Accordingly, accuracy of image segmentation can be enhanced, and particularly, exact segmentation is possible for a precise image. In the above-described embodiments, the aviation image (FIG. 7) and the satellite image (FIG. 8) are segmented to evaluate performance. However, this is merely an example, and the method and the system according to embodiments may be widely applied to image segmentations of other images including medical images, CCTV images, front images of autonomous vehicles as well as aviation images and satellite images. The technical concept of the present disclosure may be applied to a computer-readable recording medium which records a computer program for performing the functions of the apparatus and the method according to the present embodiments. In addition, the technical idea according to various embodiments of the present disclosure may be implemented in the form of a computer readable code recorded on the computer-readable recording medium. The computer-readable recording medium may be any data stor-age device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a read only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. A computer readable code or program that is stored in the computer readable recording medium may be transmitted via a net-work connected between computers. In addition, while preferred embodiments of the present disclosure have been illustrated and described, the present disclosure is not limited to the above-described specific embodiments. Various changes can be made by a person skilled in the art without departing from the scope of the present disclosure claimed in claims, and also, changed embodiments should not be understood as being separate from the technical idea or prospect of the present disclosure. What is claimed is:

A system for training an image segmentation network, the system comprising: a processor configured to: input an image to a first network which is trained to output a region segmentation result regarding an input image, and generate a region segmentation result; and input the region segmentation result generated by the first generator and a GT semantic mask to a second network, and acquire a discrimination result, the second network being trained to discriminate input-ted region segmentation results as a result generated by the first network and a GT semantic mask, respec-tively; and train the first network and the second network by using the discrimination result; and a storage unit configured to provide a necessary storage space to the processor.

An image segmentation method comprising: receiving an input of an image; and generating a region segmentation result regarding the image inputted at the input step by using a first net-work, wherein the first network is trained to output a region segmentation result regarding an input image when an image is inputted, and discrimination performance of a second network is used for training of the first network, wherein the second network is trained to discriminate inputted region segmentation results as a result gener-ated by the first network and a GT semantic mask, respectively, when the region segmentation results are inputted. ∗ ∗ ∗ ∗ ∗

Image Segmentation System FIG. 1 is a view provided to explain a segmentation-generative adversarial network (GAN) according to an embodiment of the present disclosure. The segmentation-GAN according to an embodiment is an image segmentation system that utilizes a GAN architecture. Specifically, a generator of the GAN may be implemented as a ‘semantic segmentation network for segmenting a region in an image’, instead of a ‘network for generating an image’, and a discriminator of the GAN may receive a result of segmenting a region by the semantic segmentation net-work and a ground truth (GT) and may discriminate the same, and the semantic segmentation network may be B₂ trained based on discrimination performance and perfor-mance of the semantic segmentation network. To achieve this, the image segmentation system according to an embodiment includes a semantic segmentation net-work 110 and a discriminator 120 as shown in FIG. 1. The semantic segmentation network 110 is a network that is trained to segment regions in an input image when an image is inputted, and to output the image. The semantic segmentation network 110 generates a semantic mask regarding the input image, and converts the input image into a semantic mask-added image (semantic labeling image). As shown in FIG. 2, the input to the semantic segmen-tation network 110 is an image for segmenting regions (Input Image), and the output therefrom is a semantic mask (Seg-Out). The semantic segmentation network 110 may be imple-mented by a deep learning-based segmentation network, and embodiments where the semantic segmentation network is implemented by segmentation networks of other structures are not excluded. The discriminator 120 is a network that is trained to receive the semantic mask (Seg-Out), which is the result of region segmentation outputted from the semantic segmen-tation network 110, and a GT semantic mask regarding this, and to discriminate (identify) which mask is the mask generated at the semantic segmentation network 110 and which mask is the GT semantic mask.

Training of Semantic Segmentation Network The semantic segmentation network 110 is trained to degrade discrimination performance of the discriminator 120 while enhancing its own region segmenting perfor-mance and segmentation image generation performance. A loss function of the semantic segmentation network 110 to achieve this is illustrated in FIG. 3, and FIG. 4 illustrates a loss generator 210 for generating a loss of the semantic segmentation network 110 based on the loss function. As shown in FIG. 3, the loss function of the semantic segmentation network 110 includes 1) an adversarial loss, 2) a segmentation performance loss, and 3) a content loss, and applies settable weights (α,β, γ). The loss function LG of the semantic segmentation net-work 110 may be expressed by the following equation: LG = × E[log(D(Gseg(I)))] + β SegPerformance(SegOut, GT) + γ ×ContentLoss(SegOut, GT) × where D () is the discriminator 120, Gseg() is the semantic segmentation network 110, E is Expectation cal-culation, SegPerformance() is a function for calculating region segmentation performance by comparing masks of two images, and ContentLoss() is a function for calculating region segmentation image generation performance by com-paring two mask-added images. In addition, 0≤α≤1,0≤β≤1,0≤γ≤1, α+β+γ=1 1) Adversarial Loss The adversarial loss is a loss for evaluating discrimination (identification) performance of the discriminator 120 regard-ing the semantic mask (Seg-Out) and the GT semantic mask. As the discrimination performance of the discriminator 120 is higher, the adversarial loss is greater, and, as the discrimi-nation performance of the discriminator 120 is lower, the adversarial loss is smaller. As shown in FIG. 4, the loss generator 210 calculates the adversarial loss, based on 1) the semantic mask (Seg-Out) outputted from the semantic segmentation network 110, 2) the GT semantic mask, and 3) a result of discrimination (Dis-Out) of the discriminator 120. 2) Segmentation Performance Loss The segmentation performance loss is a loss for evaluat-ing region segmentation performance of the semantic seg-mentation network 110. As the region segmentation perfor-mance of the semantic segmentation network 110 is higher, the segmentation performance loss is smaller, and, as the region segmentation performance is lower, the segmentation performance loss is greater. The loss generator 210 may calculate the region segmen-tation performance of the semantic segmentation network 110 by comparing 1) the semantic mask generated at the semantic segmentation network 110 and 2) the GT semantic mask, specifically, by calculating Dice, Intersection over Union (IoU) regarding the two images. 3) Content Loss The content loss is a loss for evaluating region segmen-tation image generation performance in the semantic seg-mentation network 110. As the semantic mask-added image generation performance of the semantic segmentation net-work 110 is higher, the content loss is smaller, and, as the image generation performance is lower, the content loss is greater. The loss generator 210 may calculate the region segmen-tation image generation performance of the semantic seg-mentation network 110 by comparing 1) an input image to which the semantic mask generated at the semantic segmen-tation network 110 is added, and 2) an input image to which the GT semantic mask is added, specifically, by calculating similarity between the two images. The similarity between the two images may be calculated by using the following Norm measurement equation: \X\p=(Σi=1 n?xi?p)1/p Another method of calculating similarity between the two images is as shown in FIG. 5. Specifically, a feature vector #1 may be extracted from the input image to which the semantic mask is added at a feature extraction network 220, and a feature vector #2 may be extracted from the image to which the GT semantic mask is added at a feature extraction network 230, and then, a similarity measurement device 240 may calculate similarity between the two images by calcu-lating similarity between the feature vector #1 and the feature vector #2 by using a measurement equation.

Training of Discriminator The discriminator 120 is trained to enhance its own discrimination performance. To achieve this, a loss function LD of the discriminator 120 may be expressed by the following equation: LD=E[log(D(MGT)]+E[log(1−D(Gseg(I))] where D () is the discriminator 120, Gseg () is the semantic segmentation network 110, E is expectation cal-culation, and MGT is a GT semantic mask. LD is a loss for evaluating discrimination (identification) performance of the discriminator 120 regarding the input image to which the semantic mask is added and the input image to which the GT semantic mask is added. As the discrimination performance of the discriminator 120 is higher, LD is greater, and, as the discrimination performance of the discriminator 120 is lower, LD is smaller. FIG. 6 illustrates a loss generator 310 for generating a loss of the discriminator 120, based on the loss function. As shown in the drawing, the loss generator 310 may calculate B₂ LD, based on the result of discrimination (Dis-Out) output-ted from the discriminator 120.

Performance Evaluation In order to evaluate performance of the segmentation-GAN according to an embodiment, a test was performed by using Inria aerial image data and SpaceNet data. The left view of FIG. 7 suggests a GT semantic mask, the center view suggests a result of image segmentation by a related-art semantic segmentation network, and the right view suggests a result of image segmentation by the seman-tic segmentation network 110 which is trained in the seg-mentation-GAN architecture according to an embodiment. As shown in FIG. 7, the result of image segmentation according to an embodiment is more similar to the GT semantic mask than the result of image segmentation by the related-art method. The left view of FIG. 8 illustrates overlap of the result of image segmentation by the related-art semantic segmenta-tion network on the GT semantic mask, and the right view illustrates overlap of the result of image segmentation by the semantic segmentation network trained in the segmentation-GAN architecture according to an embodiment, on the GT semantic mask. Referring to the drawing, specifically, the region illus-trated by the blue circle, the related-art method does not segment regions which are the GT semantic mask, whereas the method according to an embodiment well segments most of the regions appearing in the GT image.

Hardware Structure of Image Segmentation System FIG. 9 is a view illustrating a hardware structure of the image segmentation system shown in FIG. 1. As shown in the drawing, the image segmentation system may be imple-mented by a computing system that includes a communica-tion unit 410, an output unit 420, a processor 430, an input unit 440, and a storage unit 450. The communication unit 410 is a communication means for communicating with an external device and connecting to an external network. The output unit 420 is a display that displays results of processing by the processor 430. The input unit 440 is an input means for receiving a user’s command and forwarding the same to the processor 430. The processor 430 refers to graphic processing units (GPUs) and a central processing unit (CPU) for executing the networks constituting the image segmentation system shown in FIG. 1. The storage unit 450 provides a storage space necessary for operating the processor 430 and per-forming functions.

Variations The image segmentation system for training the semantic segmentation network by utilizing the GAN architecture has been described with reference to preferred embodiments. Embodiments of the present disclosure suggest a method for implementing a generator of a GAN as a semantic segmentation network for segmenting regions in an image, instead of a network for generating images, for allowing a discriminator of the GAN to receive a result of image segmentation of the semantic segmentation network, and a GT, and to discriminate between them, and for training the semantic segmentation network based on performance of the discriminator and performance of the semantic segmentation network. Accordingly, accuracy of image segmentation can be enhanced, and particularly, exact segmentation is possible for a precise image. In the above-described embodiments, the aviation image (FIG. 7) and the satellite image (FIG. 8) are segmented to evaluate performance. However, this is merely an example, and the method and the system according to embodiments may be widely applied to image segmentations of other images including medical images, CCTV images, front images of autonomous vehicles as well as aviation images and satellite images. The technical concept of the present disclosure may be applied to a computer-readable recording medium which records a computer program for performing the functions of the apparatus and the method according to the present embodiments. In addition, the technical idea according to various embodiments of the present disclosure may be implemented in the form of a computer readable code recorded on the computer-readable recording medium. The computer-readable recording medium may be any data stor-age device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a read only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. A computer readable code or program that is stored in the computer readable recording medium may be transmitted via a net-work connected between computers. In addition, while preferred embodiments of the present disclosure have been illustrated and described, the present disclosure is not limited to the above-described specific embodiments. Various changes can be made by a person skilled in the art without departing from the scope of the present disclosure claimed in claims, and also, changed embodiments should not be understood as being separate from the technical idea or prospect of the present disclosure. What is claimed is:

A system for training an image segmentation network, the system comprising: a processor configured to: input an image to a first network which is trained to output a region segmentation result regarding an input image, and generate a region segmentation result; and input the region segmentation result generated by the first generator and a GT semantic mask to a second network, and acquire a discrimination result, the second network being trained to discriminate input-ted region segmentation results as a result generated by the first network and a GT semantic mask, respec-tively; and train the first network and the second network by using the discrimination result; and a storage unit configured to provide a necessary storage space to the processor.

An image segmentation method comprising: receiving an input of an image; and generating a region segmentation result regarding the image inputted at the input step by using a first net-work, wherein the first network is trained to output a region segmentation result regarding an input image when an image is inputted, and discrimination performance of a second network is used for training of the first network, wherein the second network is trained to discriminate inputted region segmentation results as a result gener-ated by the first network and a GT semantic mask, respectively, when the region segmentation results are inputted. ∗ ∗ ∗ ∗ ∗