Generating Synthetic Images of Marine Plastic Using Deep Convolutional Generative Adversarial…

生成海洋塑料的合成图像使用深卷积生成对抗性…

Generating Synthetic Images of Marine Plastic Using Deep Convolutional Generative Adversarial Networks

用深卷积生成对抗网络生成海洋塑料合成图像

Towards building a generative approach that is capable of synthesizing images of marine plastic using GANs

建立一种能够利用GANs合成海洋塑料图像的生成方法

PROBLEM STATEMENT

问题陈述

Marine plastic pollution has been at the forefront of climate issues for the past decade. Not only does plastic in the ocean capable of killing marine life by strangulation or starvation but it is also a major factor in warming the ocean by trapping CO2. In recent years, there have been numerous attempts at cleaning the plastic that has been circling our oceans such as by the non-profit group, The Ocean Cleanup. The problem with a lot of cleanup processes is that it requires human labor and is not cost-efficient. There has been a lot of research done to automate this process by using Computer Vision and Deep Learning to detect marine debris to utilize ROVs and AUVs for cleanup.

过去十年，海洋塑料污染一直处于气候问题的前沿。海洋中的塑料不仅能够通过勒死或饿死海洋生物，而且也是通过捕获二氧化碳使海洋变暖的一个主要因素。近年来，已经有许多人试图清理盘旋在我们海洋中的塑料，如非营利组织 "海洋清理"。很多清理过程的问题是，它需要人力，而且成本效益不高。已经做了很多研究，通过使用计算机视觉和深度学习来检测海洋废弃物，利用ROV和AUV进行清理，使这一过程自动化。

Note: This is theoretical and we are working on publishing our paper. Once it gets peer-reviewed and gets published in a journal I will update this article. So, watch this space for the release of our academic paper!

注：这是理论上的，我们正在努力发表我们的论文。一旦这篇文章得到同行评审并发表在期刊上，我将更新这篇文章。所以，关注我们学术论文发布的这个空间吧！

Reading Resources: Our Paper on Using CV and DL to detect Epipelagic plastic, The University Of Minnesota’s paper on a general-purpose marine debris detecting algorithm, and The Ocean Cleanup’s paper on floating plastic detection on rivers.

阅读资源。我们关于使用CV和DL检测上层塑料的论文，明尼苏达大学关于通用海洋废弃物检测算法的论文，以及海洋清洁组织关于河流上漂浮塑料检测的论文。

The major issue for this approach is regarding the availability of datasets to train the computer vision models. The JAMSTEC-JEDI dataset is a good collection of Marine Debris off the coast of Japan on the seafloor. But, apart from this dataset, there is a huge discrepancy in the availability of datasets. For this reason, I’ve utilized the help of Generative Adversarial Networks; The DCGAN in particular to curate synthetic datasets that could theoretically be a close replica to real datasets over time. (I wrote theoretically because there are a lot of variables such as quality of real-images available, progress with GANs, training, etc).

这种方法的主要问题是关于数据集的可用性来训练计算机视觉模型。JAMSTEC-JEDI数据集是日本沿海海底海洋废弃物的一个很好的收集。但是，除了这个数据集之外，在数据集的可用性方面存在着巨大的差异。为此，我利用生成对抗网络的帮助；特别是DCGAN来策划合成数据集，理论上可以在一段时间内接近真实数据集的复制品。(我写的是理论上，因为有很多变数，比如可用的真实图像的质量，GANs的进展，训练等等）。

GAN and DCGAN

GAN和DCGAN

GANs or Generative Adversarial Networks were proposed by Ian Goodfellow et al in 2014. GANs consist of a simple 2 components called Generators and Discriminators. An oversimplification of the process is as follows: The Generators role is to generate new data and the Discriminators role is to distinguish between the generated data and the actual data. In the ideal scenario, the discriminator is unable to distinguish between the generated data and real data which results in ideal synthetic data points.

GANs或生成式对抗网络是由Ian Goodfellow等人在2014年提出的。GANs由一个简单的2个组件组成，称为生成器和判别器。对该过程的过度简化如下。生成器的作用是生成新的数据，判别器的作用是区分生成的数据和实际数据。在理想情况下，鉴别器无法区分生成的数据和真实的数据，从而产生理想的合成数据点。

DCGAN is a direct extension of the GAN architecture mentioned above except it uses Deep Convolutional Layers in the discriminator and generator respectively. It was first described by Radford et. al in the paper Unsupervised Representation Learning With Deep Convolutional Generative Adversarial Networks. The discriminator is made up of strided convolutional layers while the generator is made up of convolutional transpose layers.

DCGAN是上述GAN结构的直接扩展，只是在鉴别器和生成器中分别使用了深卷积层。最早由Radford等人描述。本文研究了基于深层卷积生成对抗网络的无监督表示学习。鉴别器由跨过卷积层构成，生成器由卷积转置层构成。

PyTorch Implementation

PyTorch实施

In this method, we are going to be applying the DCGAN architecture on the DeepTrash dataset ( Citation Link: https://zenodo.org/record/5562940#.YZa9Er3MI-R, Released under the Creative Commons Attribution 4.0 International license). If you’re not familiar with the DeepTrash dataset, consider reading my paper A Robotic Approach towards Quantifying Epipelagic Bound Plastic Using Deep Visual Models. DeepTrash is a collection of plastic images in the epipelagic layer and abyssopelagic layer of the ocean curated for marine plastic detection using computer vision.

在这个方法中，我们将在DeepTrash数据集上应用DCGAN架构（引用链接：https://zenodo.org/record/5562940#.YZa9Er3MI-R，在知识共享署名4.0国际许可下发布）。如果你对DeepTrash数据集不熟悉，可以考虑阅读我的论文《使用深度视觉模型对表层塑料进行量化的机器人方法》。DeepTrash是一个海洋上层和深海层塑料图像的集合，它被策划用于使用计算机视觉进行海洋塑料检测。

Let's begin coding!

开始编码吧！

THE CODE

《守则》

Installing Requirements

安装要求

We start by installing all the basic requirements to building our GAN model such as Matplotlib and Numpy. We will also be utilizing all the tools (such as neural networks, transformations) from Pytorch. If you’re not familiar with PyTorch I recommend giving this article a read:

我们首先安装构建GAN模型的所有基本需求，如Matplotlib和numpy。我们还将利用PyTorch的所有工具（如神经网络，转换）。如果您不熟悉PyTorch，我建议您阅读一下这篇文章：

Initializing our Hyperparameters

初始化我们的超参数

This step is fairly straightforward. We’re going to set up the hyperparameters that we want to train the Neural Network with. These hyperparameters are borrowed straight from the paper and PyTorch’s tutorial on training GANs.

这一步相当简单。我们要设置超参数，用来训练神经网络。这些超参数直接从论文和PyTorch关于训练GAN的教程中借来。

Generator and Discriminator Architectures

发生器和鉴别器架构

Now, we define the architectures for the generator and discriminator.

现在，我们定义生成器和鉴别器的体系结构。

Defining the Training Function

定义培训职能

After defining the Generator and Discriminator classes, we move on to defining the training function. The training function takes in the generator, discriminator, optimization functions, and the number of epochs as the arguments. We train the generator and discriminator by recursively calling the train function until the required number of epochs. We do this by iterating through the dataloader and updating the discriminator with new images from the generator and calculating and updating the loss function.

在定义了生成器和鉴别器类之后，我们继续定义训练函数。训练函数将生成器、判别器、优化函数和历时数作为参数。我们通过递归调用训练函数来训练生成器和判别器，直到达到所需的历时数。我们通过迭代数据加载器，用来自生成器的新图像更新鉴别器，并计算和更新损失函数来实现这一目的。

Monitoring and Training the DCGAN

监测和培训DCGAN

After we’ve established the Generator, Discriminator, and Train functions the final step is to simply call the train function for the number of epochs we defined. I’ve also used Wandb which allows us to monitor our training.

在我们建立了生成器，鉴别器和Train函数之后，最后一步是简单地为我们定义的历元数调用Train函数。我还使用了Wandb，它允许我们监控我们的训练。

Results

结果

We plot the loss incurred by the generator and discrimination during training.

我们绘制了在训练过程中产生的损失和歧视。

We can also pull up an animation of how the generator was generating images to see a difference between the real and the fake images.

我们还可以调出生成器是如何生成图像的动画，以看到真实和虚假图像之间的区别。

Which looks something like this:

它看起来是这样的：

And the final resulting image after training (the same image as above).

以及训练后的最终结果图像（与上面相同的图像）。

CONCLUSION

结论

In this article, we discussed the use of Deep Convolution Generative Adversarial Networks to generate synthetic images of marine plastic that can be used by researchers to extend their current dataset of marine plastic. This can help with giving researchers the ability to extend their dataset with a mix of real and synthetic images. As you can see with the results, the GAN still needs a lot of work. The ocean is a complex environment with varying illumination, turbidity, blur, etc. But this is a theoretical starting point that other researchers can build off of. If you’re interested in improving the results and building a better network architecture, please reach out to me at gautamtata.blog@gmail.com.

在这篇文章中，我们讨论了使用深度卷积生成对抗网络来生成海洋塑料的合成图像，研究人员可以用它来扩展他们目前的海洋塑料数据集。这可以帮助让研究人员有能力用真实和合成图像的混合来扩展他们的数据集。正如你所看到的结果，GAN仍然需要大量的工作。海洋是一个复杂的环境，有不同的光照度、浊度、模糊度等。但这是一个理论上的起点，其他研究人员可以在此基础上进行研究。如果你对改进结果和建立一个更好的网络结构感兴趣，请联系我，gautamtata.blog@gmail.com。