Authors:

• L. Lakshmi, Department of Data Science and AI, Faculty of Science and Technology (IcfaiTech), ICFAI Foundation for Higher Education, Hyderabad, India
• A. Naga Kalyani, Department of CSE (AI and ML), BVRIT Hyderabad College of Engineering for Women, Hyderabad, India
• D. Krishna Madhuri, Department of Data Science and AI, Faculty of Science and Technology (IcfaiTech), ICFAI Foundation for Higher Education, Hyderabad, India
• Sirisha Potluri, Department of CSE, Faculty of Science and Technology (IcfaiTech), ICFAI Foundation for Higher Education, Hyderabad, India
• Geetika Silakari Pandey, Department of CSE (AI and ML), BVRIT Hyderabad College of Engineering for Women, Hyderabad, India
• Shahid Ali, Dept. of Electronics Engineering, Peking University, Beijing, China
• Muhammad Ijaz Khan, Dept. of Mathematics and Statistics, Riphah International University, Islamabad, Pakistan
• Fuad A. Awwad, Dept. of Quantitative Analysis, College of Business Administration, King Saud University, Riyadh, Saudi Arabia
• Emad A. A. Ismail, Dept. of Quantitative Analysis, College of Business Administration, King Saud University, Riyadh, Saudi Arabia

1. Introduction

Image transformation is a fundamental task in computer vision with applications ranging from healthcare to digital art. Traditional image transformation models often require large amounts of annotated data, making them resource-intensive. The Cycle Generative Adversarial Network (Cycle-GAN) provides a solution by enabling image transformations using unsupervised learning, thereby reducing the need for large datasets. This study investigates the performance of Cycle-GAN in transforming photos into portrait-style images, particularly in the style of renowned artist Claude Monet, using various deep learning optimizers.

2. Problem Statement

The challenge in computer vision, particularly in artistic image transformation, is the need for large, annotated datasets to train models. For styles like Monet's, the number of available artworks is limited, making it difficult to apply traditional supervised learning techniques. The study aims to address this challenge by leveraging Cycle-GAN’s capability for unsupervised learning to generate Monet-style portraits from realistic photos. Additionally, the study analyzes the impact of different deep learning optimizers on the performance of the Cycle-GAN model.

3. Objectives

The primary objectives of this study are:

To evaluate the effectiveness of Cycle-GAN in transforming photos into Monet-style portraits using a limited dataset.

To analyze the performance of different deep learning optimizers—RMSprop, ADAM, and SGD—on the quality of the generated images.

To determine the best optimizer for achieving high-quality Monet-style image transformations with minimal computational resources.

4. Methodology

4.1 Cycle-GAN Overview

Cycle-GAN is a type of Generative Adversarial Network (GAN) designed for image-to-image translation tasks without the need for paired training examples. It consists of two main components:

Generator Networks: Responsible for transforming input images into the target style (e.g., Monet-style portraits).

Discriminator Networks: Responsible for distinguishing between real images in the target style and the images generated by the generator.

The key innovation of Cycle-GAN is its cycle-consistency loss, which ensures that an image, when transformed to the target style and then back to the original style, remains unchanged.

4.2 Data Collection

The study uses a dataset comprising:

300 Monet Paintings: Limited availability of Monet’s artworks poses a challenge for traditional supervised learning but is suitable for Cycle-GAN’s unsupervised learning approach.

7,028 Natural Photos: These serve as the source images to be transformed into Monet-style portraits.

4.3 Training and Optimizers

The Cycle-GAN model is trained using the collected dataset. Three deep learning optimizers are tested to evaluate their impact on the model’s performance:

RMSprop: A gradient descent algorithm that adapts the learning rate based on a moving average of squared gradients.

ADAM (Adaptive Moment Estimation): Combines the advantages of RMSprop and momentum by using moving averages of both the gradients and the squared gradients.

SGD (Stochastic Gradient Descent): A simple and widely used optimization algorithm that updates the model parameters based on the gradient of the loss function.

The training process involves:

Preprocessing: Images are resized and normalized for consistency.

Training Cycles: The model is trained over multiple epochs, with the performance of each optimizer being monitored and evaluated.

Evaluation Metrics: The quality of the generated images is assessed using both qualitative visual inspection and quantitative metrics like inception score (IS) and Fréchet inception distance (FID).

5. Results

The study found that the Cycle-GAN model successfully transformed natural photos into Monet-style portraits with varying degrees of success depending on the optimizer used.

5.1 Optimizer Performance

SGD: This optimizer produced the best results, achieving a balance between convergence speed and image quality. The generated portraits closely resembled Monet’s style, with clear brushstrokes and vibrant color schemes.

ADAM: ADAM provided faster convergence but sometimes resulted in less consistent artistic features. Some generated images lacked the distinct Monet-style characteristics.

RMSprop: While RMSprop produced decent results, it often required more epochs to achieve the same quality as SGD, making it less efficient.

5.2 Image Quality Analysis

Visual Inspection: Images generated using SGD showed the highest fidelity to Monet’s style, with detailed and nuanced artistic elements. The ADAM and RMSprop-generated images, while visually appealing, sometimes deviated from the expected style.

Quantitative Metrics: The SGD-optimized model achieved the lowest FID score, indicating better alignment with the target Monet style. The inception score also favored SGD over ADAM and RMSprop.

6. Conclusion

The study demonstrates that Cycle-GAN is an effective tool for transforming photos into artistic styles like Monet’s, even with limited training data. Among the optimizers tested, SGD emerged as the most effective, producing high-quality, stylistically accurate portraits. This finding is significant for applications in digital art, where style transfiguration is desired without extensive annotated datasets.

The success of Cycle-GAN with SGD optimization highlights the potential for AI to create new artworks that capture the essence of historical art styles, opening new possibilities for creative expression and art analysis in the digital age.

7. Future Work

Future research could explore the application of Cycle-GAN to other artistic styles, such as Van Gogh or Picasso, using different datasets and optimizers. Additionally, integrating more advanced loss functions or multi-scale generators could further enhance the quality and diversity of generated images. Expanding the dataset to include more varied natural scenes could also help in improving the generalization capability of the model.